2024-09-21T21:30:18.467 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-21T21:30:18.484 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T21:30:18.561 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914423 branch: squid description: rbd/device/{base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-snappy supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_nbd} email: ceph-qa@ceph.com first_in_suite: false job_id: '7914423' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi no_nested_subset: false openstack: - machine: cpus: 1 disk: 40 ram: 8000 volumes: count: 4 size: 30 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: conf: global: mon client directed command retry: 5 mon warn on pool no app: false ms inject socket failures: 5000 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bluestore block size: 96636764160 bluestore compression algorithm: snappy bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - MON_DOWN - OSDMAP_FLAGS - OSD_DOWN - PG_DEGRADED - PG_AVAILABILITY - POOL_APP_NOT_ENABLED - mons down - mon down - out of quorum - noscrub - nodeep-scrub sha1: 4e45e6b490e6cbe082cdf155fea780438ec53fc7 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: extra_packages: - rbd-nbd flavor: default sha1: 4e45e6b490e6cbe082cdf155fea780438ec53fc7 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 4e45e6b490e6cbe082cdf155fea780438ec53fc7 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mon.c - osd.0 - osd.1 - osd.2 - - mon.b - mgr.x - osd.3 - osd.4 - osd.5 - - client.0 seed: 1478 sha1: 4e45e6b490e6cbe082cdf155fea780438ec53fc7 sleep_before_teardown: 0 subset: 24/128 suite: rbd suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_4e45e6b490e6cbe082cdf155fea780438ec53fc7/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 4e45e6b490e6cbe082cdf155fea780438ec53fc7 targets: smithi081.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEp2unxGDL0tIgrSRVRZWYs2Dj4oDFhbXUI8nnboauay/bZ5RU4VT4bMkH5V1wUS4RlSgVb7vKwojuQSfrJLHaQ= smithi122.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCN8WUfumi2LAuYCDR4FPmJIr5ZAPMNu1ntYwjvNH6dEtLpQqTYJHClxByBvQxSRPIgidJTHXeAhjJ5eux10pZY= smithi164.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGfGGZvXLWhp2U7f+GrXbBJKqErIMJw3IcModtivEVdfJlye+hVfnyVc7TKyI6NNV1RehGjJtbAQIfi7zRe2Ug0= tasks: - install: null - ceph: null - thrashosds: timeout: 1200 - rbd_fsx: clients: - client.0 holebdy: 512 nbd: true ops: 6000 punch_holes: true readbdy: 512 truncbdy: 512 writebdy: 512 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-21_21:16:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2902 2024-09-21T21:30:18.562 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_4e45e6b490e6cbe082cdf155fea780438ec53fc7/qa; will attempt to use it 2024-09-21T21:30:18.562 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_4e45e6b490e6cbe082cdf155fea780438ec53fc7/qa/tasks 2024-09-21T21:30:18.563 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-21T21:30:18.564 INFO:teuthology.task.internal:Checking packages... 2024-09-21T21:30:18.584 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '4e45e6b490e6cbe082cdf155fea780438ec53fc7' 2024-09-21T21:30:18.585 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-21T21:30:18.585 INFO:teuthology.packaging:ref: None 2024-09-21T21:30:18.585 INFO:teuthology.packaging:tag: None 2024-09-21T21:30:18.585 INFO:teuthology.packaging:branch: squid 2024-09-21T21:30:18.585 INFO:teuthology.packaging:sha1: 4e45e6b490e6cbe082cdf155fea780438ec53fc7 2024-09-21T21:30:18.585 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-21T21:30:18.824 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-373.g4e45e6b4 2024-09-21T21:30:18.826 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-21T21:30:18.839 INFO:teuthology.task.internal:no buildpackages task found 2024-09-21T21:30:18.839 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-21T21:30:18.907 INFO:teuthology.task.internal:Saving configuration 2024-09-21T21:30:18.920 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-21T21:30:18.957 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-21T21:30:19.010 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi081.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914423', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 21:25:02.551831', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEp2unxGDL0tIgrSRVRZWYs2Dj4oDFhbXUI8nnboauay/bZ5RU4VT4bMkH5V1wUS4RlSgVb7vKwojuQSfrJLHaQ='} 2024-09-21T21:30:19.035 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi122.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914423', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 21:25:02.548163', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCN8WUfumi2LAuYCDR4FPmJIr5ZAPMNu1ntYwjvNH6dEtLpQqTYJHClxByBvQxSRPIgidJTHXeAhjJ5eux10pZY='} 2024-09-21T21:30:19.055 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi164.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914423', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 21:25:02.550589', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGfGGZvXLWhp2U7f+GrXbBJKqErIMJw3IcModtivEVdfJlye+hVfnyVc7TKyI6NNV1RehGjJtbAQIfi7zRe2Ug0='} 2024-09-21T21:30:19.055 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-21T21:30:19.064 INFO:teuthology.task.internal:roles: ubuntu@smithi081.front.sepia.ceph.com - ['mon.a', 'mon.c', 'osd.0', 'osd.1', 'osd.2'] 2024-09-21T21:30:19.064 INFO:teuthology.task.internal:roles: ubuntu@smithi122.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.3', 'osd.4', 'osd.5'] 2024-09-21T21:30:19.064 INFO:teuthology.task.internal:roles: ubuntu@smithi164.front.sepia.ceph.com - ['client.0'] 2024-09-21T21:30:19.064 INFO:teuthology.run_tasks:Running task console_log... 2024-09-21T21:30:19.179 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7efc28ec0160>, signals=[15]) 2024-09-21T21:30:19.179 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-21T21:30:19.185 INFO:teuthology.task.internal:Opening connections... 2024-09-21T21:30:19.185 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi081.front.sepia.ceph.com 2024-09-21T21:30:19.186 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T21:30:19.264 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi122.front.sepia.ceph.com 2024-09-21T21:30:19.265 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T21:30:19.338 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi164.front.sepia.ceph.com 2024-09-21T21:30:19.339 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T21:30:19.415 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-21T21:30:19.421 DEBUG:teuthology.orchestra.run.smithi081:> uname -m 2024-09-21T21:30:19.440 INFO:teuthology.orchestra.run.smithi081.stdout:x86_64 2024-09-21T21:30:19.440 DEBUG:teuthology.orchestra.run.smithi081:> cat /etc/os-release 2024-09-21T21:30:19.498 INFO:teuthology.orchestra.run.smithi081.stdout:NAME="CentOS Stream" 2024-09-21T21:30:19.498 INFO:teuthology.orchestra.run.smithi081.stdout:VERSION="9" 2024-09-21T21:30:19.498 INFO:teuthology.orchestra.run.smithi081.stdout:ID="centos" 2024-09-21T21:30:19.498 INFO:teuthology.orchestra.run.smithi081.stdout:ID_LIKE="rhel fedora" 2024-09-21T21:30:19.498 INFO:teuthology.orchestra.run.smithi081.stdout:VERSION_ID="9" 2024-09-21T21:30:19.498 INFO:teuthology.orchestra.run.smithi081.stdout:PLATFORM_ID="platform:el9" 2024-09-21T21:30:19.499 INFO:teuthology.orchestra.run.smithi081.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T21:30:19.499 INFO:teuthology.orchestra.run.smithi081.stdout:ANSI_COLOR="0;31" 2024-09-21T21:30:19.499 INFO:teuthology.orchestra.run.smithi081.stdout:LOGO="fedora-logo-icon" 2024-09-21T21:30:19.499 INFO:teuthology.orchestra.run.smithi081.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T21:30:19.499 INFO:teuthology.orchestra.run.smithi081.stdout:HOME_URL="https://centos.org/" 2024-09-21T21:30:19.499 INFO:teuthology.orchestra.run.smithi081.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T21:30:19.499 INFO:teuthology.orchestra.run.smithi081.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T21:30:19.499 INFO:teuthology.orchestra.run.smithi081.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T21:30:19.500 INFO:teuthology.lock.ops:Updating smithi081.front.sepia.ceph.com on lock server 2024-09-21T21:30:19.522 DEBUG:teuthology.orchestra.run.smithi122:> uname -m 2024-09-21T21:30:19.541 INFO:teuthology.orchestra.run.smithi122.stdout:x86_64 2024-09-21T21:30:19.541 DEBUG:teuthology.orchestra.run.smithi122:> cat /etc/os-release 2024-09-21T21:30:19.598 INFO:teuthology.orchestra.run.smithi122.stdout:NAME="CentOS Stream" 2024-09-21T21:30:19.598 INFO:teuthology.orchestra.run.smithi122.stdout:VERSION="9" 2024-09-21T21:30:19.598 INFO:teuthology.orchestra.run.smithi122.stdout:ID="centos" 2024-09-21T21:30:19.598 INFO:teuthology.orchestra.run.smithi122.stdout:ID_LIKE="rhel fedora" 2024-09-21T21:30:19.598 INFO:teuthology.orchestra.run.smithi122.stdout:VERSION_ID="9" 2024-09-21T21:30:19.598 INFO:teuthology.orchestra.run.smithi122.stdout:PLATFORM_ID="platform:el9" 2024-09-21T21:30:19.598 INFO:teuthology.orchestra.run.smithi122.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T21:30:19.598 INFO:teuthology.orchestra.run.smithi122.stdout:ANSI_COLOR="0;31" 2024-09-21T21:30:19.598 INFO:teuthology.orchestra.run.smithi122.stdout:LOGO="fedora-logo-icon" 2024-09-21T21:30:19.598 INFO:teuthology.orchestra.run.smithi122.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T21:30:19.598 INFO:teuthology.orchestra.run.smithi122.stdout:HOME_URL="https://centos.org/" 2024-09-21T21:30:19.598 INFO:teuthology.orchestra.run.smithi122.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T21:30:19.599 INFO:teuthology.orchestra.run.smithi122.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T21:30:19.599 INFO:teuthology.orchestra.run.smithi122.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T21:30:19.599 INFO:teuthology.lock.ops:Updating smithi122.front.sepia.ceph.com on lock server 2024-09-21T21:30:19.622 DEBUG:teuthology.orchestra.run.smithi164:> uname -m 2024-09-21T21:30:19.640 INFO:teuthology.orchestra.run.smithi164.stdout:x86_64 2024-09-21T21:30:19.641 DEBUG:teuthology.orchestra.run.smithi164:> cat /etc/os-release 2024-09-21T21:30:19.699 INFO:teuthology.orchestra.run.smithi164.stdout:NAME="CentOS Stream" 2024-09-21T21:30:19.699 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION="9" 2024-09-21T21:30:19.699 INFO:teuthology.orchestra.run.smithi164.stdout:ID="centos" 2024-09-21T21:30:19.699 INFO:teuthology.orchestra.run.smithi164.stdout:ID_LIKE="rhel fedora" 2024-09-21T21:30:19.699 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION_ID="9" 2024-09-21T21:30:19.699 INFO:teuthology.orchestra.run.smithi164.stdout:PLATFORM_ID="platform:el9" 2024-09-21T21:30:19.700 INFO:teuthology.orchestra.run.smithi164.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T21:30:19.700 INFO:teuthology.orchestra.run.smithi164.stdout:ANSI_COLOR="0;31" 2024-09-21T21:30:19.700 INFO:teuthology.orchestra.run.smithi164.stdout:LOGO="fedora-logo-icon" 2024-09-21T21:30:19.700 INFO:teuthology.orchestra.run.smithi164.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T21:30:19.700 INFO:teuthology.orchestra.run.smithi164.stdout:HOME_URL="https://centos.org/" 2024-09-21T21:30:19.700 INFO:teuthology.orchestra.run.smithi164.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T21:30:19.700 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T21:30:19.700 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T21:30:19.700 INFO:teuthology.lock.ops:Updating smithi164.front.sepia.ceph.com on lock server 2024-09-21T21:30:19.722 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-21T21:30:19.731 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-21T21:30:19.757 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-21T21:30:19.757 DEBUG:teuthology.orchestra.run.smithi081:> test '!' -e /home/ubuntu/cephtest 2024-09-21T21:30:19.760 DEBUG:teuthology.orchestra.run.smithi122:> test '!' -e /home/ubuntu/cephtest 2024-09-21T21:30:19.764 DEBUG:teuthology.orchestra.run.smithi164:> test '!' -e /home/ubuntu/cephtest 2024-09-21T21:30:19.780 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-21T21:30:19.790 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-21T21:30:19.790 DEBUG:teuthology.orchestra.run.smithi081:> test -z $(ls -A /var/lib/ceph) 2024-09-21T21:30:19.817 DEBUG:teuthology.orchestra.run.smithi122:> test -z $(ls -A /var/lib/ceph) 2024-09-21T21:30:19.821 DEBUG:teuthology.orchestra.run.smithi164:> test -z $(ls -A /var/lib/ceph) 2024-09-21T21:30:19.915 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-21T21:30:19.996 INFO:teuthology.run_tasks:Running task kernel... 2024-09-21T21:30:20.023 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-21T21:30:20.024 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}, 'client.0': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-21T21:30:20.024 DEBUG:teuthology.orchestra.run.smithi081:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T21:30:20.024 DEBUG:teuthology.orchestra.run.smithi122:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T21:30:20.025 DEBUG:teuthology.orchestra.run.smithi164:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T21:30:20.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T21:30:20.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T21:30:20.043 DEBUG:teuthology.orchestra.run.smithi081:> uname -r 2024-09-21T21:30:20.043 DEBUG:teuthology.orchestra.run.smithi122:> uname -r 2024-09-21T21:30:20.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T21:30:20.044 DEBUG:teuthology.orchestra.run.smithi164:> uname -r 2024-09-21T21:30:20.102 INFO:teuthology.orchestra.run.smithi122.stdout:5.14.0-503.el9.x86_64 2024-09-21T21:30:20.102 INFO:teuthology.orchestra.run.smithi081.stdout:5.14.0-503.el9.x86_64 2024-09-21T21:30:20.102 INFO:teuthology.task.kernel:Running kernel on smithi122: 5.14.0-503.el9.x86_64 2024-09-21T21:30:20.102 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum install -y kernel 2024-09-21T21:30:20.103 INFO:teuthology.task.kernel:Running kernel on smithi081: 5.14.0-503.el9.x86_64 2024-09-21T21:30:20.103 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum install -y kernel 2024-09-21T21:30:20.104 INFO:teuthology.orchestra.run.smithi164.stdout:5.14.0-503.el9.x86_64 2024-09-21T21:30:20.104 INFO:teuthology.task.kernel:Running kernel on smithi164: 5.14.0-503.el9.x86_64 2024-09-21T21:30:20.104 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum install -y kernel 2024-09-21T21:30:22.897 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - BaseOS 8.2 MB/s | 8.3 MB 00:01 2024-09-21T21:30:23.471 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS Stream 9 - BaseOS 5.6 MB/s | 8.3 MB 00:01 2024-09-21T21:30:23.786 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - BaseOS 4.4 MB/s | 8.3 MB 00:01 2024-09-21T21:30:24.615 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - AppStream 30 MB/s | 20 MB 00:00 2024-09-21T21:30:25.106 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS Stream 9 - AppStream 37 MB/s | 20 MB 00:00 2024-09-21T21:30:25.562 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - AppStream 29 MB/s | 20 MB 00:00 2024-09-21T21:30:29.415 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.5 MB 00:00 2024-09-21T21:30:29.921 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.5 MB 00:00 2024-09-21T21:30:30.807 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS Stream 9 - Extras packages 120 kB/s | 19 kB 00:00 2024-09-21T21:30:31.388 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - Extras packages 75 kB/s | 19 kB 00:00 2024-09-21T21:30:31.911 INFO:teuthology.orchestra.run.smithi081.stdout:Extra Packages for Enterprise Linux 23 MB/s | 23 MB 00:00 2024-09-21T21:30:32.517 INFO:teuthology.orchestra.run.smithi164.stdout:Extra Packages for Enterprise Linux 22 MB/s | 23 MB 00:01 2024-09-21T21:30:32.758 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - CRB 1.5 MB/s | 6.5 MB 00:04 2024-09-21T21:30:34.241 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - Extras packages 72 kB/s | 19 kB 00:00 2024-09-21T21:30:35.196 INFO:teuthology.orchestra.run.smithi122.stdout:Extra Packages for Enterprise Linux 27 MB/s | 23 MB 00:00 2024-09-21T21:30:37.351 INFO:teuthology.orchestra.run.smithi081.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-21T21:30:37.915 INFO:teuthology.orchestra.run.smithi164.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-21T21:30:39.299 INFO:teuthology.orchestra.run.smithi081.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-21T21:30:39.299 INFO:teuthology.orchestra.run.smithi081.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-21T21:30:39.397 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-09-21T21:30:39.410 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-09-21T21:30:39.410 INFO:teuthology.orchestra.run.smithi081.stdout: Package Architecture Version Repository Size 2024-09-21T21:30:39.410 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-09-21T21:30:39.410 INFO:teuthology.orchestra.run.smithi081.stdout:Installing: 2024-09-21T21:30:39.410 INFO:teuthology.orchestra.run.smithi081.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-21T21:30:39.410 INFO:teuthology.orchestra.run.smithi081.stdout:Installing dependencies: 2024-09-21T21:30:39.411 INFO:teuthology.orchestra.run.smithi081.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-21T21:30:39.411 INFO:teuthology.orchestra.run.smithi081.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-21T21:30:39.411 INFO:teuthology.orchestra.run.smithi081.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-21T21:30:39.411 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:30:39.411 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction Summary 2024-09-21T21:30:39.411 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-09-21T21:30:39.411 INFO:teuthology.orchestra.run.smithi081.stdout:Install 4 Packages 2024-09-21T21:30:39.411 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:30:39.411 INFO:teuthology.orchestra.run.smithi081.stdout:Total download size: 79 M 2024-09-21T21:30:39.411 INFO:teuthology.orchestra.run.smithi081.stdout:Installed size: 126 M 2024-09-21T21:30:39.411 INFO:teuthology.orchestra.run.smithi081.stdout:Downloading Packages: 2024-09-21T21:30:39.796 INFO:teuthology.orchestra.run.smithi164.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-21T21:30:39.797 INFO:teuthology.orchestra.run.smithi164.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-21T21:30:39.892 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-21T21:30:39.900 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-21T21:30:39.900 INFO:teuthology.orchestra.run.smithi164.stdout: Package Architecture Version Repository Size 2024-09-21T21:30:39.900 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-21T21:30:39.900 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-09-21T21:30:39.900 INFO:teuthology.orchestra.run.smithi164.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-21T21:30:39.900 INFO:teuthology.orchestra.run.smithi164.stdout:Installing dependencies: 2024-09-21T21:30:39.901 INFO:teuthology.orchestra.run.smithi164.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-21T21:30:39.901 INFO:teuthology.orchestra.run.smithi164.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-21T21:30:39.901 INFO:teuthology.orchestra.run.smithi164.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-21T21:30:39.901 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:30:39.901 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-09-21T21:30:39.901 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-21T21:30:39.901 INFO:teuthology.orchestra.run.smithi164.stdout:Install 4 Packages 2024-09-21T21:30:39.901 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:30:39.902 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 79 M 2024-09-21T21:30:39.902 INFO:teuthology.orchestra.run.smithi164.stdout:Installed size: 126 M 2024-09-21T21:30:39.902 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-09-21T21:30:39.994 INFO:teuthology.orchestra.run.smithi081.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 64 kB/s | 36 kB 00:00 2024-09-21T21:30:40.041 INFO:teuthology.orchestra.run.smithi164.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 302 kB/s | 36 kB 00:00 2024-09-21T21:30:40.367 INFO:teuthology.orchestra.run.smithi164.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 35 MB/s | 16 MB 00:00 2024-09-21T21:30:40.560 INFO:teuthology.orchestra.run.smithi122.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-21T21:30:40.725 INFO:teuthology.orchestra.run.smithi164.stdout:(3/4): kernel-modules-core-5.14.0-508.el9.x86_6 42 MB/s | 29 MB 00:00 2024-09-21T21:30:40.836 INFO:teuthology.orchestra.run.smithi081.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 11 MB/s | 16 MB 00:01 2024-09-21T21:30:41.042 INFO:teuthology.orchestra.run.smithi164.stdout:(4/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 31 MB/s | 35 MB 00:01 2024-09-21T21:30:41.042 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-09-21T21:30:41.043 INFO:teuthology.orchestra.run.smithi164.stdout:Total 69 MB/s | 79 MB 00:01 2024-09-21T21:30:41.495 INFO:teuthology.orchestra.run.smithi081.stdout:(3/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 17 MB/s | 35 MB 00:02 2024-09-21T21:30:41.726 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-09-21T21:30:42.088 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-09-21T21:30:42.088 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-09-21T21:30:42.458 INFO:teuthology.orchestra.run.smithi122.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-21T21:30:42.458 INFO:teuthology.orchestra.run.smithi122.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-21T21:30:42.557 INFO:teuthology.orchestra.run.smithi122.stdout:Dependencies resolved. 2024-09-21T21:30:42.568 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-21T21:30:42.569 INFO:teuthology.orchestra.run.smithi122.stdout: Package Architecture Version Repository Size 2024-09-21T21:30:42.569 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-21T21:30:42.569 INFO:teuthology.orchestra.run.smithi122.stdout:Installing: 2024-09-21T21:30:42.569 INFO:teuthology.orchestra.run.smithi122.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-21T21:30:42.569 INFO:teuthology.orchestra.run.smithi122.stdout:Installing dependencies: 2024-09-21T21:30:42.569 INFO:teuthology.orchestra.run.smithi122.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-21T21:30:42.569 INFO:teuthology.orchestra.run.smithi122.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-21T21:30:42.569 INFO:teuthology.orchestra.run.smithi122.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-21T21:30:42.569 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:30:42.569 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction Summary 2024-09-21T21:30:42.569 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-21T21:30:42.569 INFO:teuthology.orchestra.run.smithi122.stdout:Install 4 Packages 2024-09-21T21:30:42.569 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:30:42.570 INFO:teuthology.orchestra.run.smithi122.stdout:Total download size: 79 M 2024-09-21T21:30:42.570 INFO:teuthology.orchestra.run.smithi122.stdout:Installed size: 126 M 2024-09-21T21:30:42.570 INFO:teuthology.orchestra.run.smithi122.stdout:Downloading Packages: 2024-09-21T21:30:42.670 INFO:teuthology.orchestra.run.smithi081.stdout:(4/4): kernel-modules-core-5.14.0-508.el9.x86_6 11 MB/s | 29 MB 00:02 2024-09-21T21:30:42.670 INFO:teuthology.orchestra.run.smithi081.stdout:-------------------------------------------------------------------------------- 2024-09-21T21:30:42.671 INFO:teuthology.orchestra.run.smithi081.stdout:Total 24 MB/s | 79 MB 00:03 2024-09-21T21:30:42.903 INFO:teuthology.orchestra.run.smithi122.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 452 kB/s | 36 kB 00:00 2024-09-21T21:30:43.049 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-09-21T21:30:43.049 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-09-21T21:30:43.353 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction check 2024-09-21T21:30:43.478 INFO:teuthology.orchestra.run.smithi122.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 24 MB/s | 16 MB 00:00 2024-09-21T21:30:43.708 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction check succeeded. 2024-09-21T21:30:43.709 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction test 2024-09-21T21:30:43.978 INFO:teuthology.orchestra.run.smithi122.stdout:(3/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 30 MB/s | 35 MB 00:01 2024-09-21T21:30:44.312 INFO:teuthology.orchestra.run.smithi122.stdout:(4/4): kernel-modules-core-5.14.0-508.el9.x86_6 20 MB/s | 29 MB 00:01 2024-09-21T21:30:44.313 INFO:teuthology.orchestra.run.smithi122.stdout:-------------------------------------------------------------------------------- 2024-09-21T21:30:44.313 INFO:teuthology.orchestra.run.smithi122.stdout:Total 45 MB/s | 79 MB 00:01 2024-09-21T21:30:44.678 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction test succeeded. 2024-09-21T21:30:44.679 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction 2024-09-21T21:30:44.990 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction check 2024-09-21T21:30:45.333 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction check succeeded. 2024-09-21T21:30:45.333 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction test 2024-09-21T21:30:45.493 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-09-21T21:30:46.180 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-21T21:30:46.275 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T21:30:46.313 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction test succeeded. 2024-09-21T21:30:46.313 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction 2024-09-21T21:30:47.092 INFO:teuthology.orchestra.run.smithi081.stdout: Preparing : 1/1 2024-09-21T21:30:47.757 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-21T21:30:47.856 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T21:30:48.750 INFO:teuthology.orchestra.run.smithi122.stdout: Preparing : 1/1 2024-09-21T21:30:49.043 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T21:30:49.170 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T21:30:49.447 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-21T21:30:49.524 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T21:30:50.613 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T21:30:50.742 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T21:30:52.281 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T21:30:52.409 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T21:30:54.017 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T21:30:54.128 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T21:30:55.634 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T21:30:55.676 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T21:30:57.261 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T21:30:57.301 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T21:30:59.122 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T21:31:00.501 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T21:31:02.128 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T21:31:37.380 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T21:31:37.419 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-21T21:31:37.750 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T21:31:37.750 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-21T21:31:37.750 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T21:31:37.750 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T21:31:38.448 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T21:31:38.488 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-21T21:31:38.604 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T21:31:38.604 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:31:38.604 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-09-21T21:31:38.604 INFO:teuthology.orchestra.run.smithi164.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-21T21:31:38.604 INFO:teuthology.orchestra.run.smithi164.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-21T21:31:38.604 INFO:teuthology.orchestra.run.smithi164.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-21T21:31:38.604 INFO:teuthology.orchestra.run.smithi164.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-21T21:31:38.604 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:31:38.604 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-21T21:31:38.918 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T21:31:38.918 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-21T21:31:38.918 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T21:31:38.919 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T21:31:38.959 DEBUG:teuthology.orchestra.run.smithi164:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T21:31:39.046 INFO:teuthology.orchestra.run.smithi164.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T21:31:39.046 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T21:31:39.046 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-21T21:31:39.046 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-21T21:31:39.046 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-21T21:31:39.047 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-09-21T21:31:39.047 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi164.front.sepia.ceph.com, path=None, version=distro) 2024-09-21T21:31:39.047 DEBUG:teuthology.orchestra.run.smithi164:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T21:31:39.106 INFO:teuthology.orchestra.run.smithi164.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T21:31:39.106 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T21:31:39.106 DEBUG:teuthology.orchestra.run.smithi164:> sudo rpm -qi grub2-tools 2024-09-21T21:31:39.138 INFO:teuthology.orchestra.run.smithi164.stdout:Name : grub2-tools 2024-09-21T21:31:39.138 INFO:teuthology.orchestra.run.smithi164.stdout:Epoch : 1 2024-09-21T21:31:39.138 INFO:teuthology.orchestra.run.smithi164.stdout:Version : 2.06 2024-09-21T21:31:39.138 INFO:teuthology.orchestra.run.smithi164.stdout:Release : 82.el9 2024-09-21T21:31:39.138 INFO:teuthology.orchestra.run.smithi164.stdout:Architecture: x86_64 2024-09-21T21:31:39.138 INFO:teuthology.orchestra.run.smithi164.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-21T21:31:39.139 INFO:teuthology.orchestra.run.smithi164.stdout:Group : Unspecified 2024-09-21T21:31:39.139 INFO:teuthology.orchestra.run.smithi164.stdout:Size : 8274670 2024-09-21T21:31:39.139 INFO:teuthology.orchestra.run.smithi164.stdout:License : GPLv3+ 2024-09-21T21:31:39.139 INFO:teuthology.orchestra.run.smithi164.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-21T21:31:39.139 INFO:teuthology.orchestra.run.smithi164.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-21T21:31:39.139 INFO:teuthology.orchestra.run.smithi164.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-21T21:31:39.139 INFO:teuthology.orchestra.run.smithi164.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-21T21:31:39.139 INFO:teuthology.orchestra.run.smithi164.stdout:Packager : builder@centos.org 2024-09-21T21:31:39.139 INFO:teuthology.orchestra.run.smithi164.stdout:Vendor : CentOS 2024-09-21T21:31:39.139 INFO:teuthology.orchestra.run.smithi164.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-21T21:31:39.139 INFO:teuthology.orchestra.run.smithi164.stdout:Summary : Support tools for GRUB. 2024-09-21T21:31:39.139 INFO:teuthology.orchestra.run.smithi164.stdout:Description : 2024-09-21T21:31:39.139 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:31:39.139 INFO:teuthology.orchestra.run.smithi164.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-21T21:31:39.140 INFO:teuthology.orchestra.run.smithi164.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-21T21:31:39.140 INFO:teuthology.orchestra.run.smithi164.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-21T21:31:39.140 INFO:teuthology.orchestra.run.smithi164.stdout:hardware devices. 2024-09-21T21:31:39.140 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:31:39.140 INFO:teuthology.orchestra.run.smithi164.stdout:This subpackage provides tools for support of all platforms. 2024-09-21T21:31:39.141 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-21T21:31:39.141 INFO:teuthology.task.kernel:Updating grub on smithi164 to boot 5.14.0-508.el9.x86_64 2024-09-21T21:31:39.141 DEBUG:teuthology.orchestra.run.smithi164:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-21T21:31:39.771 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T21:31:39.772 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:31:39.772 INFO:teuthology.orchestra.run.smithi081.stdout:Installed: 2024-09-21T21:31:39.772 INFO:teuthology.orchestra.run.smithi081.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-21T21:31:39.772 INFO:teuthology.orchestra.run.smithi081.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-21T21:31:39.772 INFO:teuthology.orchestra.run.smithi081.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-21T21:31:39.772 INFO:teuthology.orchestra.run.smithi081.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-21T21:31:39.772 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:31:39.772 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-09-21T21:31:39.811 INFO:teuthology.orchestra.run.smithi164.stderr:Generating grub configuration file ... 2024-09-21T21:31:40.130 DEBUG:teuthology.orchestra.run.smithi081:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T21:31:40.218 INFO:teuthology.orchestra.run.smithi081.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T21:31:40.219 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T21:31:40.219 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-21T21:31:40.219 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-21T21:31:40.219 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-21T21:31:40.219 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-21T21:31:40.219 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi081.front.sepia.ceph.com, path=None, version=distro) 2024-09-21T21:31:40.219 DEBUG:teuthology.orchestra.run.smithi081:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T21:31:40.279 INFO:teuthology.orchestra.run.smithi081.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T21:31:40.279 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T21:31:40.279 DEBUG:teuthology.orchestra.run.smithi081:> sudo rpm -qi grub2-tools 2024-09-21T21:31:40.314 INFO:teuthology.orchestra.run.smithi081.stdout:Name : grub2-tools 2024-09-21T21:31:40.314 INFO:teuthology.orchestra.run.smithi081.stdout:Epoch : 1 2024-09-21T21:31:40.314 INFO:teuthology.orchestra.run.smithi081.stdout:Version : 2.06 2024-09-21T21:31:40.314 INFO:teuthology.orchestra.run.smithi081.stdout:Release : 82.el9 2024-09-21T21:31:40.314 INFO:teuthology.orchestra.run.smithi081.stdout:Architecture: x86_64 2024-09-21T21:31:40.314 INFO:teuthology.orchestra.run.smithi081.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-21T21:31:40.314 INFO:teuthology.orchestra.run.smithi081.stdout:Group : Unspecified 2024-09-21T21:31:40.314 INFO:teuthology.orchestra.run.smithi081.stdout:Size : 8274670 2024-09-21T21:31:40.314 INFO:teuthology.orchestra.run.smithi081.stdout:License : GPLv3+ 2024-09-21T21:31:40.314 INFO:teuthology.orchestra.run.smithi081.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-21T21:31:40.315 INFO:teuthology.orchestra.run.smithi081.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-21T21:31:40.315 INFO:teuthology.orchestra.run.smithi081.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-21T21:31:40.315 INFO:teuthology.orchestra.run.smithi081.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-21T21:31:40.315 INFO:teuthology.orchestra.run.smithi081.stdout:Packager : builder@centos.org 2024-09-21T21:31:40.315 INFO:teuthology.orchestra.run.smithi081.stdout:Vendor : CentOS 2024-09-21T21:31:40.315 INFO:teuthology.orchestra.run.smithi081.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-21T21:31:40.315 INFO:teuthology.orchestra.run.smithi081.stdout:Summary : Support tools for GRUB. 2024-09-21T21:31:40.315 INFO:teuthology.orchestra.run.smithi081.stdout:Description : 2024-09-21T21:31:40.315 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:31:40.315 INFO:teuthology.orchestra.run.smithi081.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-21T21:31:40.315 INFO:teuthology.orchestra.run.smithi081.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-21T21:31:40.315 INFO:teuthology.orchestra.run.smithi081.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-21T21:31:40.315 INFO:teuthology.orchestra.run.smithi081.stdout:hardware devices. 2024-09-21T21:31:40.315 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:31:40.316 INFO:teuthology.orchestra.run.smithi081.stdout:This subpackage provides tools for support of all platforms. 2024-09-21T21:31:40.316 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-21T21:31:40.317 INFO:teuthology.task.kernel:Updating grub on smithi081 to boot 5.14.0-508.el9.x86_64 2024-09-21T21:31:40.317 DEBUG:teuthology.orchestra.run.smithi081:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-21T21:31:40.586 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T21:31:40.625 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-21T21:31:40.948 INFO:teuthology.orchestra.run.smithi081.stderr:Generating grub configuration file ... 2024-09-21T21:31:40.958 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T21:31:40.959 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-21T21:31:40.959 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T21:31:40.959 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T21:31:41.112 INFO:teuthology.orchestra.run.smithi164.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-21T21:31:41.137 INFO:teuthology.orchestra.run.smithi164.stderr:done 2024-09-21T21:31:41.143 DEBUG:teuthology.orchestra.run.smithi164:> mktemp 2024-09-21T21:31:41.159 INFO:teuthology.orchestra.run.smithi164.stdout:/tmp/tmp.wg2bWfMx6h 2024-09-21T21:31:41.159 DEBUG:teuthology.orchestra.run.smithi164:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.wg2bWfMx6h 2024-09-21T21:31:41.225 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 0666 /tmp/tmp.wg2bWfMx6h 2024-09-21T21:31:41.370 DEBUG:teuthology.orchestra.remote:smithi164:/tmp/tmp.wg2bWfMx6h is 6KB 2024-09-21T21:31:41.390 DEBUG:teuthology.orchestra.run.smithi164:> rm -fr /tmp/tmp.wg2bWfMx6h 2024-09-21T21:31:41.404 DEBUG:teuthology.orchestra.run.smithi164:> sudo /bin/ls /boot/loader/entries || true 2024-09-21T21:31:41.467 INFO:teuthology.orchestra.run.smithi164.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-21T21:31:41.467 INFO:teuthology.orchestra.run.smithi164.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-21T21:31:41.468 INFO:teuthology.orchestra.run.smithi164.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-21T21:31:41.468 INFO:teuthology.orchestra.run.smithi164.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-21T21:31:41.469 DEBUG:teuthology.orchestra.run.smithi164:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-21T21:31:41.628 DEBUG:teuthology.orchestra.run.smithi164:> sudo shutdown -r now 2024-09-21T21:31:41.790 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T21:31:41.791 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:31:41.791 INFO:teuthology.orchestra.run.smithi122.stdout:Installed: 2024-09-21T21:31:41.791 INFO:teuthology.orchestra.run.smithi122.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-21T21:31:41.791 INFO:teuthology.orchestra.run.smithi122.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-21T21:31:41.791 INFO:teuthology.orchestra.run.smithi122.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-21T21:31:41.791 INFO:teuthology.orchestra.run.smithi122.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-21T21:31:41.791 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:31:41.791 INFO:teuthology.orchestra.run.smithi122.stdout:Complete! 2024-09-21T21:31:42.171 DEBUG:teuthology.orchestra.run.smithi122:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T21:31:42.257 INFO:teuthology.orchestra.run.smithi122.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T21:31:42.258 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T21:31:42.258 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-21T21:31:42.258 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-21T21:31:42.258 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-21T21:31:42.258 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-21T21:31:42.258 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi122.front.sepia.ceph.com, path=None, version=distro) 2024-09-21T21:31:42.258 DEBUG:teuthology.orchestra.run.smithi122:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T21:31:42.289 INFO:teuthology.orchestra.run.smithi081.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-21T21:31:42.316 INFO:teuthology.orchestra.run.smithi122.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T21:31:42.317 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T21:31:42.317 DEBUG:teuthology.orchestra.run.smithi122:> sudo rpm -qi grub2-tools 2024-09-21T21:31:42.325 INFO:teuthology.orchestra.run.smithi081.stderr:done 2024-09-21T21:31:42.330 DEBUG:teuthology.orchestra.run.smithi081:> mktemp 2024-09-21T21:31:42.345 INFO:teuthology.orchestra.run.smithi081.stdout:/tmp/tmp.fw1QeyInND 2024-09-21T21:31:42.346 DEBUG:teuthology.orchestra.run.smithi081:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.fw1QeyInND 2024-09-21T21:31:42.348 INFO:teuthology.orchestra.run.smithi122.stdout:Name : grub2-tools 2024-09-21T21:31:42.348 INFO:teuthology.orchestra.run.smithi122.stdout:Epoch : 1 2024-09-21T21:31:42.348 INFO:teuthology.orchestra.run.smithi122.stdout:Version : 2.06 2024-09-21T21:31:42.348 INFO:teuthology.orchestra.run.smithi122.stdout:Release : 82.el9 2024-09-21T21:31:42.348 INFO:teuthology.orchestra.run.smithi122.stdout:Architecture: x86_64 2024-09-21T21:31:42.349 INFO:teuthology.orchestra.run.smithi122.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-21T21:31:42.349 INFO:teuthology.orchestra.run.smithi122.stdout:Group : Unspecified 2024-09-21T21:31:42.349 INFO:teuthology.orchestra.run.smithi122.stdout:Size : 8274670 2024-09-21T21:31:42.349 INFO:teuthology.orchestra.run.smithi122.stdout:License : GPLv3+ 2024-09-21T21:31:42.349 INFO:teuthology.orchestra.run.smithi122.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-21T21:31:42.349 INFO:teuthology.orchestra.run.smithi122.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-21T21:31:42.349 INFO:teuthology.orchestra.run.smithi122.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-21T21:31:42.349 INFO:teuthology.orchestra.run.smithi122.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-21T21:31:42.349 INFO:teuthology.orchestra.run.smithi122.stdout:Packager : builder@centos.org 2024-09-21T21:31:42.349 INFO:teuthology.orchestra.run.smithi122.stdout:Vendor : CentOS 2024-09-21T21:31:42.349 INFO:teuthology.orchestra.run.smithi122.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-21T21:31:42.349 INFO:teuthology.orchestra.run.smithi122.stdout:Summary : Support tools for GRUB. 2024-09-21T21:31:42.349 INFO:teuthology.orchestra.run.smithi122.stdout:Description : 2024-09-21T21:31:42.349 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:31:42.350 INFO:teuthology.orchestra.run.smithi122.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-21T21:31:42.350 INFO:teuthology.orchestra.run.smithi122.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-21T21:31:42.350 INFO:teuthology.orchestra.run.smithi122.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-21T21:31:42.350 INFO:teuthology.orchestra.run.smithi122.stdout:hardware devices. 2024-09-21T21:31:42.350 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:31:42.350 INFO:teuthology.orchestra.run.smithi122.stdout:This subpackage provides tools for support of all platforms. 2024-09-21T21:31:42.351 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-21T21:31:42.351 INFO:teuthology.task.kernel:Updating grub on smithi122 to boot 5.14.0-508.el9.x86_64 2024-09-21T21:31:42.351 DEBUG:teuthology.orchestra.run.smithi122:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-21T21:31:42.412 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod 0666 /tmp/tmp.fw1QeyInND 2024-09-21T21:31:42.544 DEBUG:teuthology.orchestra.remote:smithi081:/tmp/tmp.fw1QeyInND is 6KB 2024-09-21T21:31:42.593 DEBUG:teuthology.orchestra.run.smithi081:> rm -fr /tmp/tmp.fw1QeyInND 2024-09-21T21:31:42.608 DEBUG:teuthology.orchestra.run.smithi081:> sudo /bin/ls /boot/loader/entries || true 2024-09-21T21:31:42.671 INFO:teuthology.orchestra.run.smithi081.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-21T21:31:42.671 INFO:teuthology.orchestra.run.smithi081.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-21T21:31:42.671 INFO:teuthology.orchestra.run.smithi081.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-21T21:31:42.671 INFO:teuthology.orchestra.run.smithi081.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-21T21:31:42.673 DEBUG:teuthology.orchestra.run.smithi081:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-21T21:31:42.821 DEBUG:teuthology.orchestra.run.smithi081:> sudo shutdown -r now 2024-09-21T21:31:42.970 INFO:teuthology.orchestra.run.smithi122.stderr:Generating grub configuration file ... 2024-09-21T21:31:44.258 INFO:teuthology.orchestra.run.smithi122.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-21T21:31:44.296 INFO:teuthology.orchestra.run.smithi122.stderr:done 2024-09-21T21:31:44.299 DEBUG:teuthology.orchestra.run.smithi122:> mktemp 2024-09-21T21:31:44.314 INFO:teuthology.orchestra.run.smithi122.stdout:/tmp/tmp.xViz92tr7h 2024-09-21T21:31:44.314 DEBUG:teuthology.orchestra.run.smithi122:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.xViz92tr7h 2024-09-21T21:31:44.382 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 0666 /tmp/tmp.xViz92tr7h 2024-09-21T21:31:44.518 DEBUG:teuthology.orchestra.remote:smithi122:/tmp/tmp.xViz92tr7h is 6KB 2024-09-21T21:31:44.566 DEBUG:teuthology.orchestra.run.smithi122:> rm -fr /tmp/tmp.xViz92tr7h 2024-09-21T21:31:44.581 DEBUG:teuthology.orchestra.run.smithi122:> sudo /bin/ls /boot/loader/entries || true 2024-09-21T21:31:44.643 INFO:teuthology.orchestra.run.smithi122.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-21T21:31:44.643 INFO:teuthology.orchestra.run.smithi122.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-21T21:31:44.643 INFO:teuthology.orchestra.run.smithi122.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-21T21:31:44.644 INFO:teuthology.orchestra.run.smithi122.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-21T21:31:44.645 DEBUG:teuthology.orchestra.run.smithi122:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-21T21:31:44.819 DEBUG:teuthology.orchestra.run.smithi122:> sudo shutdown -r now 2024-09-21T21:32:11.654 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-21T21:32:11.655 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-09-21T21:32:11.655 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T21:32:12.824 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-21T21:32:12.825 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi081.front.sepia.ceph.com' 2024-09-21T21:32:12.825 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T21:32:14.822 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-21T21:32:14.823 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-09-21T21:32:14.823 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T21:32:46.195 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.164 2024-09-21T21:32:49.519 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.122 2024-09-21T21:32:55.199 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-09-21T21:32:55.199 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T21:32:58.522 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-09-21T21:32:58.523 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T21:33:12.839 DEBUG:teuthology.orchestra.remote:timed out 2024-09-21T21:33:13.583 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.164 2024-09-21T21:33:16.911 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.122 2024-09-21T21:33:21.844 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi081.front.sepia.ceph.com' 2024-09-21T21:33:21.845 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T21:33:22.270 DEBUG:teuthology.orchestra.run.smithi081:> true 2024-09-21T21:33:22.619 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi081.front.sepia.ceph.com' 2024-09-21T21:33:22.619 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-508.el9.x86_64"... 2024-09-21T21:33:22.619 DEBUG:teuthology.orchestra.run.smithi081:> uname -r 2024-09-21T21:33:22.634 INFO:teuthology.orchestra.run.smithi081.stdout:5.14.0-508.el9.x86_64 2024-09-21T21:33:22.634 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-21T21:33:22.634 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-21T21:33:22.634 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-21T21:33:23.635 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-21T21:33:23.636 DEBUG:teuthology.orchestra.run.smithi081:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-21T21:33:23.719 INFO:teuthology.orchestra.run.smithi081.stdout:ttyS1 2024-09-21T21:33:23.750 DEBUG:teuthology.parallel:result is None 2024-09-21T21:33:25.584 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-09-21T21:33:25.584 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T21:33:25.960 DEBUG:teuthology.orchestra.run.smithi164:> true 2024-09-21T21:33:26.342 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-09-21T21:33:26.342 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-508.el9.x86_64"... 2024-09-21T21:33:26.342 DEBUG:teuthology.orchestra.run.smithi164:> uname -r 2024-09-21T21:33:26.357 INFO:teuthology.orchestra.run.smithi164.stdout:5.14.0-508.el9.x86_64 2024-09-21T21:33:26.357 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-21T21:33:26.357 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-21T21:33:26.357 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-21T21:33:27.358 INFO:teuthology.task.kernel:Enabling kdb on client.0... 2024-09-21T21:33:27.358 DEBUG:teuthology.orchestra.run.smithi164:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-21T21:33:27.443 INFO:teuthology.orchestra.run.smithi164.stdout:ttyS1 2024-09-21T21:33:27.463 DEBUG:teuthology.parallel:result is None 2024-09-21T21:33:28.912 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-09-21T21:33:28.912 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T21:33:29.241 DEBUG:teuthology.orchestra.run.smithi122:> true 2024-09-21T21:33:29.610 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-09-21T21:33:29.610 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-508.el9.x86_64"... 2024-09-21T21:33:29.610 DEBUG:teuthology.orchestra.run.smithi122:> uname -r 2024-09-21T21:33:29.625 INFO:teuthology.orchestra.run.smithi122.stdout:5.14.0-508.el9.x86_64 2024-09-21T21:33:29.625 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-21T21:33:29.625 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-21T21:33:29.625 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-21T21:33:30.626 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-21T21:33:30.626 DEBUG:teuthology.orchestra.run.smithi122:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-21T21:33:30.722 INFO:teuthology.orchestra.run.smithi122.stdout:ttyS1 2024-09-21T21:33:30.740 DEBUG:teuthology.parallel:result is None 2024-09-21T21:33:30.740 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-21T21:33:30.757 INFO:teuthology.task.internal:Creating test directory... 2024-09-21T21:33:30.757 DEBUG:teuthology.orchestra.run.smithi081:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T21:33:30.760 DEBUG:teuthology.orchestra.run.smithi122:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T21:33:30.762 DEBUG:teuthology.orchestra.run.smithi164:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T21:33:30.779 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-21T21:33:30.818 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-21T21:33:30.868 INFO:teuthology.task.internal:Creating archive directory... 2024-09-21T21:33:30.868 DEBUG:teuthology.orchestra.run.smithi081:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T21:33:30.871 DEBUG:teuthology.orchestra.run.smithi122:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T21:33:30.873 DEBUG:teuthology.orchestra.run.smithi164:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T21:33:30.984 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-21T21:33:30.990 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-21T21:33:30.990 DEBUG:teuthology.orchestra.run.smithi081:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T21:33:30.993 DEBUG:teuthology.orchestra.run.smithi122:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T21:33:30.995 DEBUG:teuthology.orchestra.run.smithi164:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T21:33:31.021 INFO:teuthology.orchestra.run.smithi081.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T21:33:31.026 INFO:teuthology.orchestra.run.smithi122.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T21:33:31.032 INFO:teuthology.orchestra.run.smithi081.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T21:33:31.035 INFO:teuthology.orchestra.run.smithi122.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T21:33:31.217 INFO:teuthology.orchestra.run.smithi164.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T21:33:31.238 INFO:teuthology.orchestra.run.smithi164.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T21:33:31.240 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-21T21:33:31.247 INFO:teuthology.task.internal:Configuring sudo... 2024-09-21T21:33:31.247 DEBUG:teuthology.orchestra.run.smithi081:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T21:33:31.250 DEBUG:teuthology.orchestra.run.smithi122:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T21:33:31.252 DEBUG:teuthology.orchestra.run.smithi164:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T21:33:31.329 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-21T21:33:31.337 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-21T21:33:31.337 DEBUG:teuthology.orchestra.run.smithi081:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T21:33:31.340 DEBUG:teuthology.orchestra.run.smithi122:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T21:33:31.343 DEBUG:teuthology.orchestra.run.smithi164:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T21:33:31.381 DEBUG:teuthology.orchestra.run.smithi081:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T21:33:31.442 DEBUG:teuthology.orchestra.run.smithi081:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T21:33:31.486 DEBUG:teuthology.orchestra.run.smithi081:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T21:33:31.571 DEBUG:teuthology.orchestra.run.smithi081:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T21:33:31.610 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-09-21T21:33:31.610 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T21:33:31.676 DEBUG:teuthology.orchestra.run.smithi122:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T21:33:31.719 DEBUG:teuthology.orchestra.run.smithi122:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T21:33:31.807 DEBUG:teuthology.orchestra.run.smithi122:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T21:33:31.855 DEBUG:teuthology.orchestra.run.smithi122:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T21:33:31.898 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-21T21:33:31.899 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T21:33:31.965 DEBUG:teuthology.orchestra.run.smithi164:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T21:33:32.012 DEBUG:teuthology.orchestra.run.smithi164:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T21:33:32.063 DEBUG:teuthology.orchestra.run.smithi164:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T21:33:32.104 DEBUG:teuthology.orchestra.run.smithi164:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T21:33:32.142 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-21T21:33:32.142 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T21:33:32.206 DEBUG:teuthology.orchestra.run.smithi081:> sudo service rsyslog restart 2024-09-21T21:33:32.209 DEBUG:teuthology.orchestra.run.smithi122:> sudo service rsyslog restart 2024-09-21T21:33:32.213 DEBUG:teuthology.orchestra.run.smithi164:> sudo service rsyslog restart 2024-09-21T21:33:32.270 INFO:teuthology.orchestra.run.smithi081.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T21:33:32.293 INFO:teuthology.orchestra.run.smithi122.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T21:33:32.372 INFO:teuthology.orchestra.run.smithi164.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T21:33:32.938 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-21T21:33:32.947 INFO:teuthology.task.internal:Starting timer... 2024-09-21T21:33:32.947 INFO:teuthology.run_tasks:Running task pcp... 2024-09-21T21:33:33.009 INFO:teuthology.run_tasks:Running task selinux... 2024-09-21T21:33:33.054 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-09-21T21:33:33.054 DEBUG:teuthology.orchestra.run.smithi081:> sudo service auditd rotate 2024-09-21T21:33:33.111 INFO:teuthology.orchestra.run.smithi081.stdout:Rotating logs: 2024-09-21T21:33:33.113 DEBUG:teuthology.orchestra.run.smithi122:> sudo service auditd rotate 2024-09-21T21:33:33.234 INFO:teuthology.orchestra.run.smithi122.stdout:Rotating logs: 2024-09-21T21:33:33.240 DEBUG:teuthology.orchestra.run.smithi164:> sudo service auditd rotate 2024-09-21T21:33:33.297 INFO:teuthology.orchestra.run.smithi164.stdout:Rotating logs: 2024-09-21T21:33:33.299 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-21T21:33:33.299 DEBUG:teuthology.orchestra.run.smithi081:> /usr/sbin/getenforce 2024-09-21T21:33:33.321 INFO:teuthology.orchestra.run.smithi081.stdout:Permissive 2024-09-21T21:33:33.322 DEBUG:teuthology.orchestra.run.smithi122:> /usr/sbin/getenforce 2024-09-21T21:33:33.342 INFO:teuthology.orchestra.run.smithi122.stdout:Permissive 2024-09-21T21:33:33.343 DEBUG:teuthology.orchestra.run.smithi164:> /usr/sbin/getenforce 2024-09-21T21:33:33.363 INFO:teuthology.orchestra.run.smithi164.stdout:Permissive 2024-09-21T21:33:33.364 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi081.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi122.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi164.front.sepia.ceph.com': 'permissive'} 2024-09-21T21:33:33.364 DEBUG:teuthology.orchestra.run.smithi081:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-21T21:33:33.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T21:33:33.390 DEBUG:teuthology.orchestra.run.smithi122:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-21T21:33:33.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T21:33:33.416 DEBUG:teuthology.orchestra.run.smithi164:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-21T21:33:33.442 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T21:33:33.443 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-21T21:33:33.443 DEBUG:teuthology.orchestra.run.smithi081:> sudo /usr/sbin/setenforce permissive 2024-09-21T21:33:33.478 DEBUG:teuthology.orchestra.run.smithi122:> sudo /usr/sbin/setenforce permissive 2024-09-21T21:33:33.510 DEBUG:teuthology.orchestra.run.smithi164:> sudo /usr/sbin/setenforce permissive 2024-09-21T21:33:33.547 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-21T21:33:33.556 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-21T21:33:33.558 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-21T21:33:33.673 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-21T21:33:33.687 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-21T21:33:33.688 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi081.front.sepia.ceph.com,smithi122.front.sepia.ceph.com,smithi164.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-21T21:37:16.465 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi081.front.sepia.ceph.com'), Remote(name='ubuntu@smithi122.front.sepia.ceph.com'), Remote(name='ubuntu@smithi164.front.sepia.ceph.com')] 2024-09-21T21:37:16.466 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi081.front.sepia.ceph.com' 2024-09-21T21:37:16.467 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T21:37:16.545 DEBUG:teuthology.orchestra.run.smithi081:> true 2024-09-21T21:37:16.629 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi081.front.sepia.ceph.com' 2024-09-21T21:37:16.630 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-09-21T21:37:16.630 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T21:37:16.707 DEBUG:teuthology.orchestra.run.smithi122:> true 2024-09-21T21:37:16.793 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-09-21T21:37:16.793 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-09-21T21:37:16.794 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T21:37:16.876 DEBUG:teuthology.orchestra.run.smithi164:> true 2024-09-21T21:37:16.959 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-09-21T21:37:16.960 INFO:teuthology.run_tasks:Running task clock... 2024-09-21T21:37:16.970 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-21T21:37:16.971 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T21:37:16.971 DEBUG:teuthology.orchestra.run.smithi081:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T21:37:16.973 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T21:37:16.974 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T21:37:16.976 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T21:37:16.976 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T21:37:17.003 INFO:teuthology.orchestra.run.smithi122.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T21:37:17.004 INFO:teuthology.orchestra.run.smithi081.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T21:37:17.017 INFO:teuthology.orchestra.run.smithi122.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T21:37:17.019 INFO:teuthology.orchestra.run.smithi081.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T21:37:17.033 INFO:teuthology.orchestra.run.smithi164.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T21:37:17.041 INFO:teuthology.orchestra.run.smithi122.stderr:sudo: ntpd: command not found 2024-09-21T21:37:17.045 INFO:teuthology.orchestra.run.smithi081.stderr:sudo: ntpd: command not found 2024-09-21T21:37:17.049 INFO:teuthology.orchestra.run.smithi164.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T21:37:17.052 INFO:teuthology.orchestra.run.smithi122.stdout:506 Cannot talk to daemon 2024-09-21T21:37:17.057 INFO:teuthology.orchestra.run.smithi081.stdout:506 Cannot talk to daemon 2024-09-21T21:37:17.066 INFO:teuthology.orchestra.run.smithi122.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T21:37:17.071 INFO:teuthology.orchestra.run.smithi081.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T21:37:17.075 INFO:teuthology.orchestra.run.smithi164.stderr:sudo: ntpd: command not found 2024-09-21T21:37:17.080 INFO:teuthology.orchestra.run.smithi122.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T21:37:17.086 INFO:teuthology.orchestra.run.smithi081.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T21:37:17.087 INFO:teuthology.orchestra.run.smithi164.stdout:506 Cannot talk to daemon 2024-09-21T21:37:17.102 INFO:teuthology.orchestra.run.smithi164.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T21:37:17.117 INFO:teuthology.orchestra.run.smithi164.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T21:37:17.136 INFO:teuthology.orchestra.run.smithi122.stderr:bash: line 1: ntpq: command not found 2024-09-21T21:37:17.140 INFO:teuthology.orchestra.run.smithi122.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T21:37:17.140 INFO:teuthology.orchestra.run.smithi122.stdout:=============================================================================== 2024-09-21T21:37:17.140 INFO:teuthology.orchestra.run.smithi122.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T21:37:17.140 INFO:teuthology.orchestra.run.smithi122.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T21:37:17.140 INFO:teuthology.orchestra.run.smithi122.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T21:37:17.140 INFO:teuthology.orchestra.run.smithi122.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T21:37:17.141 INFO:teuthology.orchestra.run.smithi081.stderr:bash: line 1: ntpq: command not found 2024-09-21T21:37:17.144 INFO:teuthology.orchestra.run.smithi081.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T21:37:17.144 INFO:teuthology.orchestra.run.smithi081.stdout:=============================================================================== 2024-09-21T21:37:17.144 INFO:teuthology.orchestra.run.smithi081.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T21:37:17.144 INFO:teuthology.orchestra.run.smithi081.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T21:37:17.144 INFO:teuthology.orchestra.run.smithi081.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T21:37:17.144 INFO:teuthology.orchestra.run.smithi081.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T21:37:17.172 INFO:teuthology.orchestra.run.smithi164.stderr:bash: line 1: ntpq: command not found 2024-09-21T21:37:17.176 INFO:teuthology.orchestra.run.smithi164.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T21:37:17.176 INFO:teuthology.orchestra.run.smithi164.stdout:=============================================================================== 2024-09-21T21:37:17.176 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T21:37:17.176 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T21:37:17.176 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T21:37:17.176 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T21:37:17.177 INFO:teuthology.run_tasks:Running task install... 2024-09-21T21:37:17.187 DEBUG:teuthology.task.install:project ceph 2024-09-21T21:37:17.187 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'extra_packages': ['rbd-nbd'], 'flavor': 'default', 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7'}} 2024-09-21T21:37:17.188 DEBUG:teuthology.task.install:config {'extra_packages': ['rbd-nbd'], 'flavor': 'default', 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7'} 2024-09-21T21:37:17.188 INFO:teuthology.task.install:Using flavor: default 2024-09-21T21:37:17.193 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-21T21:37:17.193 INFO:teuthology.task.install:extra packages: ['rbd-nbd'] 2024-09-21T21:37:17.193 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': ['rbd-nbd'], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7', 'tag': None, 'wait_for_package': False} 2024-09-21T21:37:17.194 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=4e45e6b490e6cbe082cdf155fea780438ec53fc7 2024-09-21T21:37:17.195 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': ['rbd-nbd'], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7', 'tag': None, 'wait_for_package': False} 2024-09-21T21:37:17.195 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=4e45e6b490e6cbe082cdf155fea780438ec53fc7 2024-09-21T21:37:17.196 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': ['rbd-nbd'], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7', 'tag': None, 'wait_for_package': False} 2024-09-21T21:37:17.196 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=4e45e6b490e6cbe082cdf155fea780438ec53fc7 2024-09-21T21:37:17.345 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/centos/9/flavors/default/ 2024-09-21T21:37:17.345 INFO:teuthology.task.install.rpm:Package version is 19.1.1-373.g4e45e6b4 2024-09-21T21:37:17.349 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/centos/9/flavors/default/ 2024-09-21T21:37:17.349 INFO:teuthology.task.install.rpm:Package version is 19.1.1-373.g4e45e6b4 2024-09-21T21:37:17.357 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/centos/9/flavors/default/ 2024-09-21T21:37:17.357 INFO:teuthology.task.install.rpm:Package version is 19.1.1-373.g4e45e6b4 2024-09-21T21:37:17.454 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T21:37:17.454 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-09-21T21:37:17.454 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T21:37:17.460 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T21:37:17.461 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-21T21:37:17.461 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T21:37:17.467 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/4e45e6b490e6cbe082cdf155fea780438ec53fc7/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T21:37:17.468 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-21T21:37:17.468 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T21:37:17.480 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, rbd-nbd on remote rpm x86_64 2024-09-21T21:37:17.481 DEBUG:teuthology.orchestra.run.smithi081:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/4e45e6b490e6cbe082cdf155fea780438ec53fc7/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T21:37:17.487 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, rbd-nbd on remote rpm x86_64 2024-09-21T21:37:17.487 DEBUG:teuthology.orchestra.run.smithi164:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/4e45e6b490e6cbe082cdf155fea780438ec53fc7/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T21:37:17.493 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, rbd-nbd on remote rpm x86_64 2024-09-21T21:37:17.493 DEBUG:teuthology.orchestra.run.smithi122:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/4e45e6b490e6cbe082cdf155fea780438ec53fc7/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T21:37:17.546 DEBUG:teuthology.orchestra.run.smithi081:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T21:37:17.554 DEBUG:teuthology.orchestra.run.smithi164:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T21:37:17.558 DEBUG:teuthology.orchestra.run.smithi122:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T21:37:17.622 DEBUG:teuthology.orchestra.run.smithi081:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T21:37:17.630 DEBUG:teuthology.orchestra.run.smithi164:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T21:37:17.632 DEBUG:teuthology.orchestra.run.smithi122:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T21:37:17.686 INFO:teuthology.orchestra.run.smithi081.stdout:check_obsoletes = 1 2024-09-21T21:37:17.687 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum clean all 2024-09-21T21:37:17.695 INFO:teuthology.orchestra.run.smithi164.stdout:check_obsoletes = 1 2024-09-21T21:37:17.696 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum clean all 2024-09-21T21:37:17.697 INFO:teuthology.orchestra.run.smithi122.stdout:check_obsoletes = 1 2024-09-21T21:37:17.698 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum clean all 2024-09-21T21:37:17.915 INFO:teuthology.orchestra.run.smithi081.stdout:45 files removed 2024-09-21T21:37:17.924 INFO:teuthology.orchestra.run.smithi122.stdout:45 files removed 2024-09-21T21:37:17.926 INFO:teuthology.orchestra.run.smithi164.stdout:45 files removed 2024-09-21T21:37:17.942 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum -y install ceph-radosgw 2024-09-21T21:37:17.951 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum -y install ceph-radosgw 2024-09-21T21:37:17.953 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install ceph-radosgw 2024-09-21T21:37:18.495 INFO:teuthology.orchestra.run.smithi081.stdout:ceph packages for x86_64 311 kB/s | 84 kB 00:00 2024-09-21T21:37:18.501 INFO:teuthology.orchestra.run.smithi164.stdout:ceph packages for x86_64 317 kB/s | 84 kB 00:00 2024-09-21T21:37:18.518 INFO:teuthology.orchestra.run.smithi122.stdout:ceph packages for x86_64 296 kB/s | 84 kB 00:00 2024-09-21T21:37:18.779 INFO:teuthology.orchestra.run.smithi081.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-21T21:37:18.785 INFO:teuthology.orchestra.run.smithi164.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-21T21:37:18.811 INFO:teuthology.orchestra.run.smithi122.stdout:ceph noarch packages 47 kB/s | 12 kB 00:00 2024-09-21T21:37:19.043 INFO:teuthology.orchestra.run.smithi164.stdout:ceph source packages 8.5 kB/s | 1.9 kB 00:00 2024-09-21T21:37:19.055 INFO:teuthology.orchestra.run.smithi081.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-21T21:37:19.086 INFO:teuthology.orchestra.run.smithi122.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-21T21:37:19.752 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.3 MB 00:00 2024-09-21T21:37:20.255 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS Stream 9 - BaseOS 7.1 MB/s | 8.3 MB 00:01 2024-09-21T21:37:20.995 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - BaseOS 4.4 MB/s | 8.3 MB 00:01 2024-09-21T21:37:22.157 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS Stream 9 - AppStream 25 MB/s | 20 MB 00:00 2024-09-21T21:37:22.720 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - AppStream 11 MB/s | 20 MB 00:01 2024-09-21T21:37:22.880 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - AppStream 25 MB/s | 20 MB 00:00 2024-09-21T21:37:27.883 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS Stream 9 - CRB 3.4 MB/s | 6.5 MB 00:01 2024-09-21T21:37:29.189 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - CRB 2.6 MB/s | 6.5 MB 00:02 2024-09-21T21:37:29.384 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS Stream 9 - Extras packages 70 kB/s | 19 kB 00:00 2024-09-21T21:37:30.796 INFO:teuthology.orchestra.run.smithi081.stdout:Extra Packages for Enterprise Linux 18 MB/s | 23 MB 00:01 2024-09-21T21:37:30.937 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - CRB 1.5 MB/s | 6.5 MB 00:04 2024-09-21T21:37:30.972 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - Extras packages 44 kB/s | 19 kB 00:00 2024-09-21T21:37:32.319 INFO:teuthology.orchestra.run.smithi122.stdout:Extra Packages for Enterprise Linux 19 MB/s | 23 MB 00:01 2024-09-21T21:37:32.379 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - Extras packages 87 kB/s | 19 kB 00:00 2024-09-21T21:37:33.392 INFO:teuthology.orchestra.run.smithi164.stdout:Extra Packages for Enterprise Linux 26 MB/s | 23 MB 00:00 2024-09-21T21:37:36.093 INFO:teuthology.orchestra.run.smithi081.stdout:lab-extras 25 kB/s | 1.7 kB 00:00 2024-09-21T21:37:37.783 INFO:teuthology.orchestra.run.smithi122.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-21T21:37:38.338 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-09-21T21:37:38.355 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-09-21T21:37:38.356 INFO:teuthology.orchestra.run.smithi081.stdout: Package Arch Version Repository Size 2024-09-21T21:37:38.356 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-09-21T21:37:38.356 INFO:teuthology.orchestra.run.smithi081.stdout:Installing: 2024-09-21T21:37:38.356 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-radosgw x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 11 M 2024-09-21T21:37:38.356 INFO:teuthology.orchestra.run.smithi081.stdout:Upgrading: 2024-09-21T21:37:38.356 INFO:teuthology.orchestra.run.smithi081.stdout: librados2 x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 3.4 M 2024-09-21T21:37:38.356 INFO:teuthology.orchestra.run.smithi081.stdout: librbd1 x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 3.2 M 2024-09-21T21:37:38.357 INFO:teuthology.orchestra.run.smithi081.stdout:Installing dependencies: 2024-09-21T21:37:38.357 INFO:teuthology.orchestra.run.smithi081.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T21:37:38.357 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-base x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 5.4 M 2024-09-21T21:37:38.357 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-common x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 21 M 2024-09-21T21:37:38.357 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-selinux x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 25 k 2024-09-21T21:37:38.358 INFO:teuthology.orchestra.run.smithi081.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T21:37:38.358 INFO:teuthology.orchestra.run.smithi081.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T21:37:38.358 INFO:teuthology.orchestra.run.smithi081.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T21:37:38.358 INFO:teuthology.orchestra.run.smithi081.stdout: libcephfs2 x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 751 k 2024-09-21T21:37:38.358 INFO:teuthology.orchestra.run.smithi081.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T21:37:38.358 INFO:teuthology.orchestra.run.smithi081.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T21:37:38.359 INFO:teuthology.orchestra.run.smithi081.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T21:37:38.359 INFO:teuthology.orchestra.run.smithi081.stdout: libradosstriper1 x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 507 k 2024-09-21T21:37:38.359 INFO:teuthology.orchestra.run.smithi081.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T21:37:38.359 INFO:teuthology.orchestra.run.smithi081.stdout: librgw2 x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 5.3 M 2024-09-21T21:37:38.359 INFO:teuthology.orchestra.run.smithi081.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T21:37:38.359 INFO:teuthology.orchestra.run.smithi081.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T21:37:38.359 INFO:teuthology.orchestra.run.smithi081.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T21:37:38.359 INFO:teuthology.orchestra.run.smithi081.stdout: python3-ceph-argparse x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 45 k 2024-09-21T21:37:38.359 INFO:teuthology.orchestra.run.smithi081.stdout: python3-ceph-common x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 136 k 2024-09-21T21:37:38.360 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cephfs x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 162 k 2024-09-21T21:37:38.360 INFO:teuthology.orchestra.run.smithi081.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T21:37:38.360 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rados x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 321 k 2024-09-21T21:37:38.360 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rbd x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 300 k 2024-09-21T21:37:38.360 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rgw x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 99 k 2024-09-21T21:37:38.360 INFO:teuthology.orchestra.run.smithi081.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-21T21:37:38.361 INFO:teuthology.orchestra.run.smithi081.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-21T21:37:38.361 INFO:teuthology.orchestra.run.smithi081.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T21:37:38.361 INFO:teuthology.orchestra.run.smithi081.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T21:37:38.361 INFO:teuthology.orchestra.run.smithi081.stdout:Installing weak dependencies: 2024-09-21T21:37:38.361 INFO:teuthology.orchestra.run.smithi081.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-21T21:37:38.361 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:37:38.361 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction Summary 2024-09-21T21:37:38.361 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-09-21T21:37:38.361 INFO:teuthology.orchestra.run.smithi081.stdout:Install 30 Packages 2024-09-21T21:37:38.361 INFO:teuthology.orchestra.run.smithi081.stdout:Upgrade 2 Packages 2024-09-21T21:37:38.361 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:37:38.362 INFO:teuthology.orchestra.run.smithi081.stdout:Total download size: 60 M 2024-09-21T21:37:38.362 INFO:teuthology.orchestra.run.smithi081.stdout:Downloading Packages: 2024-09-21T21:37:38.889 INFO:teuthology.orchestra.run.smithi164.stdout:lab-extras 25 kB/s | 1.7 kB 00:00 2024-09-21T21:37:38.962 INFO:teuthology.orchestra.run.smithi081.stdout:(1/32): ceph-base-19.1.1-373.g4e45e6b4.el9.x86_ 10 MB/s | 5.4 MB 00:00 2024-09-21T21:37:39.020 INFO:teuthology.orchestra.run.smithi081.stdout:(2/32): ceph-selinux-19.1.1-373.g4e45e6b4.el9.x 433 kB/s | 25 kB 00:00 2024-09-21T21:37:39.121 INFO:teuthology.orchestra.run.smithi081.stdout:(3/32): libcephfs2-19.1.1-373.g4e45e6b4.el9.x86 7.3 MB/s | 751 kB 00:00 2024-09-21T21:37:39.213 INFO:teuthology.orchestra.run.smithi081.stdout:(4/32): libradosstriper1-19.1.1-373.g4e45e6b4.e 5.4 MB/s | 507 kB 00:00 2024-09-21T21:37:39.446 INFO:teuthology.orchestra.run.smithi081.stdout:(5/32): ceph-common-19.1.1-373.g4e45e6b4.el9.x8 21 MB/s | 21 MB 00:01 2024-09-21T21:37:39.580 INFO:teuthology.orchestra.run.smithi081.stdout:(6/32): ceph-radosgw-19.1.1-373.g4e45e6b4.el9.x 9.3 MB/s | 11 MB 00:01 2024-09-21T21:37:39.605 INFO:teuthology.orchestra.run.smithi081.stdout:(7/32): python3-ceph-argparse-19.1.1-373.g4e45e 285 kB/s | 45 kB 00:00 2024-09-21T21:37:39.630 INFO:teuthology.orchestra.run.smithi081.stdout:(8/32): python3-ceph-common-19.1.1-373.g4e45e6b 2.7 MB/s | 136 kB 00:00 2024-09-21T21:37:39.656 INFO:teuthology.orchestra.run.smithi081.stdout:(9/32): python3-cephfs-19.1.1-373.g4e45e6b4.el9 3.1 MB/s | 162 kB 00:00 2024-09-21T21:37:39.681 INFO:teuthology.orchestra.run.smithi081.stdout:(10/32): python3-rados-19.1.1-373.g4e45e6b4.el9 6.3 MB/s | 321 kB 00:00 2024-09-21T21:37:39.715 INFO:teuthology.orchestra.run.smithi081.stdout:(11/32): python3-rbd-19.1.1-373.g4e45e6b4.el9.x 5.0 MB/s | 300 kB 00:00 2024-09-21T21:37:39.740 INFO:teuthology.orchestra.run.smithi081.stdout:(12/32): python3-rgw-19.1.1-373.g4e45e6b4.el9.x 1.7 MB/s | 99 kB 00:00 2024-09-21T21:37:39.815 INFO:teuthology.orchestra.run.smithi081.stdout:(13/32): librgw2-19.1.1-373.g4e45e6b4.el9.x86_6 8.9 MB/s | 5.3 MB 00:00 2024-09-21T21:37:39.907 INFO:teuthology.orchestra.run.smithi081.stdout:(14/32): boost-program-options-1.75.0-8.el9.x86 557 kB/s | 107 kB 00:00 2024-09-21T21:37:39.949 INFO:teuthology.orchestra.run.smithi081.stdout:(15/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 339 kB/s | 45 kB 00:00 2024-09-21T21:37:39.983 INFO:teuthology.orchestra.run.smithi081.stdout:(16/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 661 kB/s | 160 kB 00:00 2024-09-21T21:37:40.033 INFO:teuthology.orchestra.run.smithi081.stdout:(17/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 292 kB 00:00 2024-09-21T21:37:40.058 INFO:teuthology.orchestra.run.smithi081.stdout:(18/32): librdkafka-1.6.1-102.el9.x86_64.rpm 4.3 MB/s | 662 kB 00:00 2024-09-21T21:37:40.059 INFO:teuthology.orchestra.run.smithi122.stdout:Dependencies resolved. 2024-09-21T21:37:40.075 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-21T21:37:40.075 INFO:teuthology.orchestra.run.smithi122.stdout: Package Arch Version Repository Size 2024-09-21T21:37:40.075 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-21T21:37:40.075 INFO:teuthology.orchestra.run.smithi122.stdout:Installing: 2024-09-21T21:37:40.075 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-radosgw x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 11 M 2024-09-21T21:37:40.076 INFO:teuthology.orchestra.run.smithi122.stdout:Upgrading: 2024-09-21T21:37:40.076 INFO:teuthology.orchestra.run.smithi122.stdout: librados2 x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 3.4 M 2024-09-21T21:37:40.076 INFO:teuthology.orchestra.run.smithi122.stdout: librbd1 x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 3.2 M 2024-09-21T21:37:40.076 INFO:teuthology.orchestra.run.smithi122.stdout:Installing dependencies: 2024-09-21T21:37:40.076 INFO:teuthology.orchestra.run.smithi122.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T21:37:40.076 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-base x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 5.4 M 2024-09-21T21:37:40.076 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-common x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 21 M 2024-09-21T21:37:40.076 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-selinux x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 25 k 2024-09-21T21:37:40.076 INFO:teuthology.orchestra.run.smithi122.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T21:37:40.076 INFO:teuthology.orchestra.run.smithi122.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T21:37:40.076 INFO:teuthology.orchestra.run.smithi122.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T21:37:40.076 INFO:teuthology.orchestra.run.smithi122.stdout: libcephfs2 x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 751 k 2024-09-21T21:37:40.076 INFO:teuthology.orchestra.run.smithi122.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T21:37:40.077 INFO:teuthology.orchestra.run.smithi122.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T21:37:40.077 INFO:teuthology.orchestra.run.smithi122.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T21:37:40.077 INFO:teuthology.orchestra.run.smithi122.stdout: libradosstriper1 x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 507 k 2024-09-21T21:37:40.077 INFO:teuthology.orchestra.run.smithi122.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T21:37:40.077 INFO:teuthology.orchestra.run.smithi122.stdout: librgw2 x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 5.3 M 2024-09-21T21:37:40.077 INFO:teuthology.orchestra.run.smithi122.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T21:37:40.077 INFO:teuthology.orchestra.run.smithi122.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T21:37:40.077 INFO:teuthology.orchestra.run.smithi122.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T21:37:40.077 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ceph-argparse x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 45 k 2024-09-21T21:37:40.077 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ceph-common x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 136 k 2024-09-21T21:37:40.077 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cephfs x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 162 k 2024-09-21T21:37:40.077 INFO:teuthology.orchestra.run.smithi122.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T21:37:40.077 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rados x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 321 k 2024-09-21T21:37:40.078 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rbd x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 300 k 2024-09-21T21:37:40.078 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rgw x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 99 k 2024-09-21T21:37:40.078 INFO:teuthology.orchestra.run.smithi122.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-21T21:37:40.078 INFO:teuthology.orchestra.run.smithi122.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-21T21:37:40.078 INFO:teuthology.orchestra.run.smithi122.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T21:37:40.078 INFO:teuthology.orchestra.run.smithi122.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T21:37:40.078 INFO:teuthology.orchestra.run.smithi122.stdout:Installing weak dependencies: 2024-09-21T21:37:40.078 INFO:teuthology.orchestra.run.smithi122.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-21T21:37:40.078 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:37:40.078 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction Summary 2024-09-21T21:37:40.078 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-21T21:37:40.078 INFO:teuthology.orchestra.run.smithi122.stdout:Install 30 Packages 2024-09-21T21:37:40.078 INFO:teuthology.orchestra.run.smithi122.stdout:Upgrade 2 Packages 2024-09-21T21:37:40.079 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:37:40.079 INFO:teuthology.orchestra.run.smithi122.stdout:Total download size: 60 M 2024-09-21T21:37:40.079 INFO:teuthology.orchestra.run.smithi122.stdout:Downloading Packages: 2024-09-21T21:37:40.083 INFO:teuthology.orchestra.run.smithi081.stdout:(19/32): python3-prettytable-0.7.2-27.el9.noarc 421 kB/s | 42 kB 00:00 2024-09-21T21:37:40.109 INFO:teuthology.orchestra.run.smithi081.stdout:(20/32): qatlib-24.02.0-1.el9.x86_64.rpm 2.9 MB/s | 221 kB 00:00 2024-09-21T21:37:40.134 INFO:teuthology.orchestra.run.smithi081.stdout:(21/32): qatlib-service-24.02.0-1.el9.x86_64.rp 476 kB/s | 36 kB 00:00 2024-09-21T21:37:40.159 INFO:teuthology.orchestra.run.smithi081.stdout:(22/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 626 kB/s | 47 kB 00:00 2024-09-21T21:37:40.210 INFO:teuthology.orchestra.run.smithi081.stdout:(23/32): gperftools-libs-2.9.1-3.el9.x86_64.rpm 3.0 MB/s | 308 kB 00:00 2024-09-21T21:37:40.243 INFO:teuthology.orchestra.run.smithi081.stdout:(24/32): liboath-2.6.7-2.el9.x86_64.rpm 1.6 MB/s | 49 kB 00:00 2024-09-21T21:37:40.269 INFO:teuthology.orchestra.run.smithi081.stdout:(25/32): libarrow-doc-9.0.0-11.el9.noarch.rpm 274 kB/s | 26 kB 00:00 2024-09-21T21:37:40.294 INFO:teuthology.orchestra.run.smithi081.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-21T21:37:40.336 INFO:teuthology.orchestra.run.smithi081.stdout:(27/32): re2-20211101-20.el9.x86_64.rpm 4.5 MB/s | 191 kB 00:00 2024-09-21T21:37:40.370 INFO:teuthology.orchestra.run.smithi081.stdout:(28/32): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.1 MB/s | 839 kB 00:00 2024-09-21T21:37:40.436 INFO:teuthology.orchestra.run.smithi081.stdout:(29/32): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-21T21:37:40.504 INFO:teuthology.orchestra.run.smithi081.stdout:(30/32): libarrow-9.0.0-11.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-09-21T21:37:40.620 INFO:teuthology.orchestra.run.smithi081.stdout:(31/32): librados2-19.1.1-373.g4e45e6b4.el9.x86 14 MB/s | 3.4 MB 00:00 2024-09-21T21:37:40.679 INFO:teuthology.orchestra.run.smithi081.stdout:(32/32): librbd1-19.1.1-373.g4e45e6b4.el9.x86_6 13 MB/s | 3.2 MB 00:00 2024-09-21T21:37:40.683 INFO:teuthology.orchestra.run.smithi081.stdout:-------------------------------------------------------------------------------- 2024-09-21T21:37:40.684 INFO:teuthology.orchestra.run.smithi081.stdout:Total 26 MB/s | 60 MB 00:02 2024-09-21T21:37:40.782 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction check 2024-09-21T21:37:40.818 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction check succeeded. 2024-09-21T21:37:40.818 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction test 2024-09-21T21:37:41.090 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-21T21:37:41.100 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-21T21:37:41.101 INFO:teuthology.orchestra.run.smithi164.stdout: Package Arch Version Repository Size 2024-09-21T21:37:41.101 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-21T21:37:41.101 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-09-21T21:37:41.101 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-radosgw x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 11 M 2024-09-21T21:37:41.101 INFO:teuthology.orchestra.run.smithi164.stdout:Upgrading: 2024-09-21T21:37:41.101 INFO:teuthology.orchestra.run.smithi164.stdout: librados2 x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 3.4 M 2024-09-21T21:37:41.101 INFO:teuthology.orchestra.run.smithi164.stdout: librbd1 x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 3.2 M 2024-09-21T21:37:41.101 INFO:teuthology.orchestra.run.smithi164.stdout:Installing dependencies: 2024-09-21T21:37:41.101 INFO:teuthology.orchestra.run.smithi164.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T21:37:41.101 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-base x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 5.4 M 2024-09-21T21:37:41.102 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-common x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 21 M 2024-09-21T21:37:41.102 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-selinux x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 25 k 2024-09-21T21:37:41.102 INFO:teuthology.orchestra.run.smithi164.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T21:37:41.102 INFO:teuthology.orchestra.run.smithi164.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T21:37:41.102 INFO:teuthology.orchestra.run.smithi164.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T21:37:41.102 INFO:teuthology.orchestra.run.smithi164.stdout: libcephfs2 x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 751 k 2024-09-21T21:37:41.102 INFO:teuthology.orchestra.run.smithi164.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T21:37:41.102 INFO:teuthology.orchestra.run.smithi164.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T21:37:41.102 INFO:teuthology.orchestra.run.smithi164.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T21:37:41.102 INFO:teuthology.orchestra.run.smithi164.stdout: libradosstriper1 x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 507 k 2024-09-21T21:37:41.102 INFO:teuthology.orchestra.run.smithi164.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T21:37:41.102 INFO:teuthology.orchestra.run.smithi164.stdout: librgw2 x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 5.3 M 2024-09-21T21:37:41.103 INFO:teuthology.orchestra.run.smithi164.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T21:37:41.103 INFO:teuthology.orchestra.run.smithi164.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T21:37:41.103 INFO:teuthology.orchestra.run.smithi164.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T21:37:41.103 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ceph-argparse x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 45 k 2024-09-21T21:37:41.103 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ceph-common x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 136 k 2024-09-21T21:37:41.103 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cephfs x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 162 k 2024-09-21T21:37:41.103 INFO:teuthology.orchestra.run.smithi164.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T21:37:41.103 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rados x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 321 k 2024-09-21T21:37:41.103 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rbd x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 300 k 2024-09-21T21:37:41.103 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rgw x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 99 k 2024-09-21T21:37:41.103 INFO:teuthology.orchestra.run.smithi164.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-21T21:37:41.103 INFO:teuthology.orchestra.run.smithi164.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-21T21:37:41.104 INFO:teuthology.orchestra.run.smithi164.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T21:37:41.104 INFO:teuthology.orchestra.run.smithi164.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T21:37:41.104 INFO:teuthology.orchestra.run.smithi164.stdout:Installing weak dependencies: 2024-09-21T21:37:41.104 INFO:teuthology.orchestra.run.smithi164.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-21T21:37:41.104 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:37:41.104 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-09-21T21:37:41.104 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-21T21:37:41.104 INFO:teuthology.orchestra.run.smithi164.stdout:Install 30 Packages 2024-09-21T21:37:41.104 INFO:teuthology.orchestra.run.smithi164.stdout:Upgrade 2 Packages 2024-09-21T21:37:41.104 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:37:41.105 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 60 M 2024-09-21T21:37:41.105 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-09-21T21:37:41.483 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction test succeeded. 2024-09-21T21:37:41.484 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction 2024-09-21T21:37:41.501 INFO:teuthology.orchestra.run.smithi122.stdout:(1/32): ceph-base-19.1.1-373.g4e45e6b4.el9.x86_ 4.9 MB/s | 5.4 MB 00:01 2024-09-21T21:37:41.551 INFO:teuthology.orchestra.run.smithi122.stdout:(2/32): ceph-selinux-19.1.1-373.g4e45e6b4.el9.x 505 kB/s | 25 kB 00:00 2024-09-21T21:37:41.677 INFO:teuthology.orchestra.run.smithi122.stdout:(3/32): libcephfs2-19.1.1-373.g4e45e6b4.el9.x86 5.9 MB/s | 751 kB 00:00 2024-09-21T21:37:41.794 INFO:teuthology.orchestra.run.smithi122.stdout:(4/32): libradosstriper1-19.1.1-373.g4e45e6b4.e 4.3 MB/s | 507 kB 00:00 2024-09-21T21:37:42.107 INFO:teuthology.orchestra.run.smithi164.stdout:(1/32): ceph-base-19.1.1-373.g4e45e6b4.el9.x86_ 8.4 MB/s | 5.4 MB 00:00 2024-09-21T21:37:42.144 INFO:teuthology.orchestra.run.smithi122.stdout:(5/32): ceph-radosgw-19.1.1-373.g4e45e6b4.el9.x 6.1 MB/s | 11 MB 00:01 2024-09-21T21:37:42.149 INFO:teuthology.orchestra.run.smithi164.stdout:(2/32): ceph-selinux-19.1.1-373.g4e45e6b4.el9.x 607 kB/s | 25 kB 00:00 2024-09-21T21:37:42.224 INFO:teuthology.orchestra.run.smithi164.stdout:(3/32): libcephfs2-19.1.1-373.g4e45e6b4.el9.x86 9.8 MB/s | 751 kB 00:00 2024-09-21T21:37:42.283 INFO:teuthology.orchestra.run.smithi164.stdout:(4/32): libradosstriper1-19.1.1-373.g4e45e6b4.e 8.5 MB/s | 507 kB 00:00 2024-09-21T21:37:42.369 INFO:teuthology.orchestra.run.smithi122.stdout:(6/32): ceph-common-19.1.1-373.g4e45e6b4.el9.x8 11 MB/s | 21 MB 00:01 2024-09-21T21:37:42.394 INFO:teuthology.orchestra.run.smithi122.stdout:(7/32): python3-ceph-argparse-19.1.1-373.g4e45e 180 kB/s | 45 kB 00:00 2024-09-21T21:37:42.436 INFO:teuthology.orchestra.run.smithi122.stdout:(8/32): python3-ceph-common-19.1.1-373.g4e45e6b 2.0 MB/s | 136 kB 00:00 2024-09-21T21:37:42.470 INFO:teuthology.orchestra.run.smithi122.stdout:(9/32): python3-cephfs-19.1.1-373.g4e45e6b4.el9 2.1 MB/s | 162 kB 00:00 2024-09-21T21:37:42.520 INFO:teuthology.orchestra.run.smithi122.stdout:(10/32): python3-rados-19.1.1-373.g4e45e6b4.el9 3.8 MB/s | 321 kB 00:00 2024-09-21T21:37:42.554 INFO:teuthology.orchestra.run.smithi122.stdout:(11/32): python3-rbd-19.1.1-373.g4e45e6b4.el9.x 3.5 MB/s | 300 kB 00:00 2024-09-21T21:37:42.595 INFO:teuthology.orchestra.run.smithi122.stdout:(12/32): python3-rgw-19.1.1-373.g4e45e6b4.el9.x 1.3 MB/s | 99 kB 00:00 2024-09-21T21:37:42.633 INFO:teuthology.orchestra.run.smithi164.stdout:(5/32): librgw2-19.1.1-373.g4e45e6b4.el9.x86_64 15 MB/s | 5.3 MB 00:00 2024-09-21T21:37:42.671 INFO:teuthology.orchestra.run.smithi122.stdout:(13/32): librgw2-19.1.1-373.g4e45e6b4.el9.x86_6 6.1 MB/s | 5.3 MB 00:00 2024-09-21T21:37:42.713 INFO:teuthology.orchestra.run.smithi122.stdout:(14/32): boost-program-options-1.75.0-8.el9.x86 673 kB/s | 107 kB 00:00 2024-09-21T21:37:42.763 INFO:teuthology.orchestra.run.smithi122.stdout:(15/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 958 kB/s | 160 kB 00:00 2024-09-21T21:37:42.767 INFO:teuthology.orchestra.run.smithi164.stdout:(6/32): ceph-radosgw-19.1.1-373.g4e45e6b4.el9.x 8.1 MB/s | 11 MB 00:01 2024-09-21T21:37:42.780 INFO:teuthology.orchestra.run.smithi122.stdout:(16/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 416 kB/s | 45 kB 00:00 2024-09-21T21:37:42.792 INFO:teuthology.orchestra.run.smithi164.stdout:(7/32): python3-ceph-argparse-19.1.1-373.g4e45e 285 kB/s | 45 kB 00:00 2024-09-21T21:37:42.809 INFO:teuthology.orchestra.run.smithi164.stdout:(8/32): python3-ceph-common-19.1.1-373.g4e45e6b 3.2 MB/s | 136 kB 00:00 2024-09-21T21:37:42.830 INFO:teuthology.orchestra.run.smithi122.stdout:(17/32): librdkafka-1.6.1-102.el9.x86_64.rpm 5.5 MB/s | 662 kB 00:00 2024-09-21T21:37:42.843 INFO:teuthology.orchestra.run.smithi164.stdout:(9/32): python3-cephfs-19.1.1-373.g4e45e6b4.el9 3.1 MB/s | 162 kB 00:00 2024-09-21T21:37:42.847 INFO:teuthology.orchestra.run.smithi122.stdout:(18/32): python3-prettytable-0.7.2-27.el9.noarc 632 kB/s | 42 kB 00:00 2024-09-21T21:37:42.868 INFO:teuthology.orchestra.run.smithi164.stdout:(10/32): python3-rados-19.1.1-373.g4e45e6b4.el9 5.3 MB/s | 321 kB 00:00 2024-09-21T21:37:42.872 INFO:teuthology.orchestra.run.smithi122.stdout:(19/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-09-21T21:37:42.889 INFO:teuthology.orchestra.run.smithi122.stdout:(20/32): qatlib-service-24.02.0-1.el9.x86_64.rp 861 kB/s | 36 kB 00:00 2024-09-21T21:37:42.906 INFO:teuthology.orchestra.run.smithi122.stdout:(21/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-09-21T21:37:43.001 INFO:teuthology.orchestra.run.smithi081.stdout: Preparing : 1/1 2024-09-21T21:37:43.002 INFO:teuthology.orchestra.run.smithi164.stdout:(11/32): python3-rbd-19.1.1-373.g4e45e6b4.el9.x 4.4 MB/s | 300 kB 00:00 2024-09-21T21:37:43.002 INFO:teuthology.orchestra.run.smithi164.stdout:(12/32): python3-rgw-19.1.1-373.g4e45e6b4.el9.x 1.5 MB/s | 99 kB 00:00 2024-09-21T21:37:43.002 INFO:teuthology.orchestra.run.smithi122.stdout:(22/32): qatlib-24.02.0-1.el9.x86_64.rpm 2.1 MB/s | 221 kB 00:00 2024-09-21T21:37:43.002 INFO:teuthology.orchestra.run.smithi122.stdout:(23/32): gperftools-libs-2.9.1-3.el9.x86_64.rpm 3.0 MB/s | 308 kB 00:00 2024-09-21T21:37:43.015 INFO:teuthology.orchestra.run.smithi122.stdout:(24/32): libarrow-doc-9.0.0-11.el9.noarch.rpm 375 kB/s | 26 kB 00:00 2024-09-21T21:37:43.021 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-09-21T21:37:43.041 INFO:teuthology.orchestra.run.smithi122.stdout:(25/32): liboath-2.6.7-2.el9.x86_64.rpm 970 kB/s | 49 kB 00:00 2024-09-21T21:37:43.059 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-09-21T21:37:43.066 INFO:teuthology.orchestra.run.smithi122.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-21T21:37:43.093 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-09-21T21:37:43.108 INFO:teuthology.orchestra.run.smithi122.stdout:(27/32): re2-20211101-20.el9.x86_64.rpm 4.5 MB/s | 191 kB 00:00 2024-09-21T21:37:43.115 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-09-21T21:37:43.138 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-09-21T21:37:43.150 INFO:teuthology.orchestra.run.smithi122.stdout:(28/32): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.5 MB/s | 839 kB 00:00 2024-09-21T21:37:43.244 INFO:teuthology.orchestra.run.smithi164.stdout:(13/32): ceph-common-19.1.1-373.g4e45e6b4.el9.x 12 MB/s | 21 MB 00:01 2024-09-21T21:37:43.269 INFO:teuthology.orchestra.run.smithi164.stdout:(14/32): boost-program-options-1.75.0-8.el9.x86 298 kB/s | 107 kB 00:00 2024-09-21T21:37:43.284 INFO:teuthology.orchestra.run.smithi122.stdout:(29/32): thrift-0.15.0-2.el9.x86_64.rpm 9.0 MB/s | 1.6 MB 00:00 2024-09-21T21:37:43.294 INFO:teuthology.orchestra.run.smithi164.stdout:(15/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 446 kB/s | 160 kB 00:00 2024-09-21T21:37:43.325 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/34 2024-09-21T21:37:43.376 INFO:teuthology.orchestra.run.smithi122.stdout:(30/32): libarrow-9.0.0-11.el9.x86_64.rpm 9.4 MB/s | 4.4 MB 00:00 2024-09-21T21:37:43.387 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/34 2024-09-21T21:37:43.387 INFO:teuthology.orchestra.run.smithi164.stdout:(16/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.1 MB/s | 292 kB 00:00 2024-09-21T21:37:43.412 INFO:teuthology.orchestra.run.smithi164.stdout:(17/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 270 kB/s | 45 kB 00:00 2024-09-21T21:37:43.445 INFO:teuthology.orchestra.run.smithi164.stdout:(18/32): python3-prettytable-0.7.2-27.el9.noarc 722 kB/s | 42 kB 00:00 2024-09-21T21:37:43.470 INFO:teuthology.orchestra.run.smithi164.stdout:(19/32): librdkafka-1.6.1-102.el9.x86_64.rpm 3.2 MB/s | 662 kB 00:00 2024-09-21T21:37:43.492 INFO:teuthology.orchestra.run.smithi122.stdout:(31/32): librados2-19.1.1-373.g4e45e6b4.el9.x86 10 MB/s | 3.4 MB 00:00 2024-09-21T21:37:43.496 INFO:teuthology.orchestra.run.smithi164.stdout:(20/32): qatlib-service-24.02.0-1.el9.x86_64.rp 712 kB/s | 36 kB 00:00 2024-09-21T21:37:43.521 INFO:teuthology.orchestra.run.smithi164.stdout:(21/32): qatlib-24.02.0-1.el9.x86_64.rpm 2.0 MB/s | 221 kB 00:00 2024-09-21T21:37:43.547 INFO:teuthology.orchestra.run.smithi164.stdout:(22/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 627 kB/s | 47 kB 00:00 2024-09-21T21:37:43.568 INFO:teuthology.orchestra.run.smithi122.stdout:(32/32): librbd1-19.1.1-373.g4e45e6b4.el9.x86_6 11 MB/s | 3.2 MB 00:00 2024-09-21T21:37:43.573 INFO:teuthology.orchestra.run.smithi122.stdout:-------------------------------------------------------------------------------- 2024-09-21T21:37:43.573 INFO:teuthology.orchestra.run.smithi122.stdout:Total 17 MB/s | 60 MB 00:03 2024-09-21T21:37:43.597 INFO:teuthology.orchestra.run.smithi164.stdout:(23/32): libarrow-doc-9.0.0-11.el9.noarch.rpm 600 kB/s | 26 kB 00:00 2024-09-21T21:37:43.622 INFO:teuthology.orchestra.run.smithi164.stdout:(24/32): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-21T21:37:43.647 INFO:teuthology.orchestra.run.smithi164.stdout:(25/32): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-21T21:37:43.679 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction check 2024-09-21T21:37:43.714 INFO:teuthology.orchestra.run.smithi164.stdout:(26/32): parquet-libs-9.0.0-11.el9.x86_64.rpm 12 MB/s | 839 kB 00:00 2024-09-21T21:37:43.715 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction check succeeded. 2024-09-21T21:37:43.715 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction test 2024-09-21T21:37:43.749 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-09-21T21:37:43.756 INFO:teuthology.orchestra.run.smithi164.stdout:(27/32): re2-20211101-20.el9.x86_64.rpm 4.7 MB/s | 191 kB 00:00 2024-09-21T21:37:43.790 INFO:teuthology.orchestra.run.smithi164.stdout:(28/32): gperftools-libs-2.9.1-3.el9.x86_64.rpm 1.0 MB/s | 308 kB 00:00 2024-09-21T21:37:43.827 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : librados2-2:19.1.1-373.g4e45e6b4.el9.x86_64 7/34 2024-09-21T21:37:43.857 INFO:teuthology.orchestra.run.smithi164.stdout:(29/32): libarrow-9.0.0-11.el9.x86_64.rpm 13 MB/s | 4.4 MB 00:00 2024-09-21T21:37:43.916 INFO:teuthology.orchestra.run.smithi164.stdout:(30/32): thrift-0.15.0-2.el9.x86_64.rpm 10 MB/s | 1.6 MB 00:00 2024-09-21T21:37:43.920 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: librados2-2:19.1.1-373.g4e45e6b4.el9.x86_64 7/34 2024-09-21T21:37:44.010 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-rados-2:19.1.1-373.g4e45e6b4.el9.x86_64 8/34 2024-09-21T21:37:44.052 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libcephfs2-2:19.1.1-373.g4e45e6b4.el9.x86_64 9/34 2024-09-21T21:37:44.074 INFO:teuthology.orchestra.run.smithi164.stdout:(31/32): librados2-19.1.1-373.g4e45e6b4.el9.x86 12 MB/s | 3.4 MB 00:00 2024-09-21T21:37:44.117 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: libcephfs2-2:19.1.1-373.g4e45e6b4.el9.x86_64 9/34 2024-09-21T21:37:44.132 INFO:teuthology.orchestra.run.smithi164.stdout:(32/32): librbd1-19.1.1-373.g4e45e6b4.el9.x86_6 11 MB/s | 3.2 MB 00:00 2024-09-21T21:37:44.137 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-09-21T21:37:44.137 INFO:teuthology.orchestra.run.smithi164.stdout:Total 20 MB/s | 60 MB 00:03 2024-09-21T21:37:44.154 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-09-21T21:37:44.199 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-ceph-argparse-2:19.1.1-373.g4e45e6b4.el9.x 11/34 2024-09-21T21:37:44.235 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-09-21T21:37:44.271 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-cephfs-2:19.1.1-373.g4e45e6b4.el9.x86_64 12/34 2024-09-21T21:37:44.272 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-09-21T21:37:44.272 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-09-21T21:37:44.298 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libradosstriper1-2:19.1.1-373.g4e45e6b4.el9.x86_64 13/34 2024-09-21T21:37:44.362 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: libradosstriper1-2:19.1.1-373.g4e45e6b4.el9.x86_64 13/34 2024-09-21T21:37:44.397 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/34 2024-09-21T21:37:44.409 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction test succeeded. 2024-09-21T21:37:44.410 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction 2024-09-21T21:37:44.515 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/34 2024-09-21T21:37:44.548 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/34 2024-09-21T21:37:44.597 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-09-21T21:37:44.629 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/34 2024-09-21T21:37:44.658 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/34 2024-09-21T21:37:44.691 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-09-21T21:37:44.933 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-09-21T21:37:44.934 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-09-21T21:37:45.012 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-09-21T21:37:45.063 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : librbd1-2:19.1.1-373.g4e45e6b4.el9.x86_64 22/34 2024-09-21T21:37:45.152 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: librbd1-2:19.1.1-373.g4e45e6b4.el9.x86_64 22/34 2024-09-21T21:37:45.184 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-rbd-2:19.1.1-373.g4e45e6b4.el9.x86_64 23/34 2024-09-21T21:37:45.220 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-09-21T21:37:45.705 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 25/34 2024-09-21T21:37:45.740 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : librgw2-2:19.1.1-373.g4e45e6b4.el9.x86_64 26/34 2024-09-21T21:37:45.820 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: librgw2-2:19.1.1-373.g4e45e6b4.el9.x86_64 26/34 2024-09-21T21:37:45.897 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-rgw-2:19.1.1-373.g4e45e6b4.el9.x86_64 27/34 2024-09-21T21:37:45.926 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-ceph-common-2:19.1.1-373.g4e45e6b4.el9.x86 28/34 2024-09-21T21:37:45.980 INFO:teuthology.orchestra.run.smithi122.stdout: Preparing : 1/1 2024-09-21T21:37:46.056 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-09-21T21:37:46.084 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-09-21T21:37:46.131 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-09-21T21:37:46.154 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-09-21T21:37:46.175 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-09-21T21:37:46.373 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/34 2024-09-21T21:37:46.426 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/34 2024-09-21T21:37:46.445 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-09-21T21:37:46.525 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-09-21T21:37:46.554 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-09-21T21:37:46.588 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-09-21T21:37:46.610 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-09-21T21:37:46.632 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-09-21T21:37:46.804 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/34 2024-09-21T21:37:46.812 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-09-21T21:37:46.858 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/34 2024-09-21T21:37:46.866 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : librados2-2:19.1.1-373.g4e45e6b4.el9.x86_64 7/34 2024-09-21T21:37:46.959 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: librados2-2:19.1.1-373.g4e45e6b4.el9.x86_64 7/34 2024-09-21T21:37:47.049 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-rados-2:19.1.1-373.g4e45e6b4.el9.x86_64 8/34 2024-09-21T21:37:47.076 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libcephfs2-2:19.1.1-373.g4e45e6b4.el9.x86_64 9/34 2024-09-21T21:37:47.139 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: libcephfs2-2:19.1.1-373.g4e45e6b4.el9.x86_64 9/34 2024-09-21T21:37:47.168 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-09-21T21:37:47.212 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-ceph-argparse-2:19.1.1-373.g4e45e6b4.el9.x 11/34 2024-09-21T21:37:47.239 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-09-21T21:37:47.281 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cephfs-2:19.1.1-373.g4e45e6b4.el9.x86_64 12/34 2024-09-21T21:37:47.303 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libradosstriper1-2:19.1.1-373.g4e45e6b4.el9.x86_64 13/34 2024-09-21T21:37:47.306 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : librados2-2:19.1.1-373.g4e45e6b4.el9.x86_64 7/34 2024-09-21T21:37:47.367 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: libradosstriper1-2:19.1.1-373.g4e45e6b4.el9.x86_64 13/34 2024-09-21T21:37:47.390 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librados2-2:19.1.1-373.g4e45e6b4.el9.x86_64 7/34 2024-09-21T21:37:47.401 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/34 2024-09-21T21:37:47.479 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-rados-2:19.1.1-373.g4e45e6b4.el9.x86_64 8/34 2024-09-21T21:37:47.512 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/34 2024-09-21T21:37:47.516 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libcephfs2-2:19.1.1-373.g4e45e6b4.el9.x86_64 9/34 2024-09-21T21:37:47.552 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/34 2024-09-21T21:37:47.579 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: libcephfs2-2:19.1.1-373.g4e45e6b4.el9.x86_64 9/34 2024-09-21T21:37:47.607 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-09-21T21:37:47.616 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-09-21T21:37:47.642 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/34 2024-09-21T21:37:47.661 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-ceph-argparse-2:19.1.1-373.g4e45e6b4.el9.x 11/34 2024-09-21T21:37:47.671 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/34 2024-09-21T21:37:47.705 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-09-21T21:37:47.737 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cephfs-2:19.1.1-373.g4e45e6b4.el9.x86_64 12/34 2024-09-21T21:37:47.760 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libradosstriper1-2:19.1.1-373.g4e45e6b4.el9.x86_64 13/34 2024-09-21T21:37:47.825 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: libradosstriper1-2:19.1.1-373.g4e45e6b4.el9.x86_64 13/34 2024-09-21T21:37:47.860 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/34 2024-09-21T21:37:47.970 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/34 2024-09-21T21:37:48.001 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/34 2024-09-21T21:37:48.015 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-09-21T21:37:48.047 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-09-21T21:37:48.067 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : librbd1-2:19.1.1-373.g4e45e6b4.el9.x86_64 22/34 2024-09-21T21:37:48.075 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/34 2024-09-21T21:37:48.104 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/34 2024-09-21T21:37:48.137 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-09-21T21:37:48.140 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: librbd1-2:19.1.1-373.g4e45e6b4.el9.x86_64 22/34 2024-09-21T21:37:48.172 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-rbd-2:19.1.1-373.g4e45e6b4.el9.x86_64 23/34 2024-09-21T21:37:48.208 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-09-21T21:37:48.208 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-common-2:19.1.1-373.g4e45e6b4.el9.x86_64 29/34 2024-09-21T21:37:48.256 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-common-2:19.1.1-373.g4e45e6b4.el9.x86_64 29/34 2024-09-21T21:37:48.460 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-09-21T21:37:48.499 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : librbd1-2:19.1.1-373.g4e45e6b4.el9.x86_64 22/34 2024-09-21T21:37:48.590 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librbd1-2:19.1.1-373.g4e45e6b4.el9.x86_64 22/34 2024-09-21T21:37:48.621 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-rbd-2:19.1.1-373.g4e45e6b4.el9.x86_64 23/34 2024-09-21T21:37:48.657 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-09-21T21:37:48.687 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 25/34 2024-09-21T21:37:48.720 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : librgw2-2:19.1.1-373.g4e45e6b4.el9.x86_64 26/34 2024-09-21T21:37:48.791 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: librgw2-2:19.1.1-373.g4e45e6b4.el9.x86_64 26/34 2024-09-21T21:37:48.818 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-common-2:19.1.1-373.g4e45e6b4.el9.x86_64 29/34 2024-09-21T21:37:48.860 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-base-2:19.1.1-373.g4e45e6b4.el9.x86_64 30/34 2024-09-21T21:37:48.868 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-rgw-2:19.1.1-373.g4e45e6b4.el9.x86_64 27/34 2024-09-21T21:37:48.897 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-ceph-common-2:19.1.1-373.g4e45e6b4.el9.x86 28/34 2024-09-21T21:37:48.951 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-base-2:19.1.1-373.g4e45e6b4.el9.x86_64 30/34 2024-09-21T21:37:48.951 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T21:37:48.951 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T21:37:48.951 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:37:48.985 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-selinux-2:19.1.1-373.g4e45e6b4.el9.x86_64 31/34 2024-09-21T21:37:49.144 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 25/34 2024-09-21T21:37:49.177 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : librgw2-2:19.1.1-373.g4e45e6b4.el9.x86_64 26/34 2024-09-21T21:37:49.249 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librgw2-2:19.1.1-373.g4e45e6b4.el9.x86_64 26/34 2024-09-21T21:37:49.326 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-rgw-2:19.1.1-373.g4e45e6b4.el9.x86_64 27/34 2024-09-21T21:37:49.370 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-ceph-common-2:19.1.1-373.g4e45e6b4.el9.x86 28/34 2024-09-21T21:37:51.205 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-common-2:19.1.1-373.g4e45e6b4.el9.x86_64 29/34 2024-09-21T21:37:51.260 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-common-2:19.1.1-373.g4e45e6b4.el9.x86_64 29/34 2024-09-21T21:37:51.683 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-common-2:19.1.1-373.g4e45e6b4.el9.x86_64 29/34 2024-09-21T21:37:51.726 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-common-2:19.1.1-373.g4e45e6b4.el9.x86_64 29/34 2024-09-21T21:37:51.829 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-common-2:19.1.1-373.g4e45e6b4.el9.x86_64 29/34 2024-09-21T21:37:51.872 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-base-2:19.1.1-373.g4e45e6b4.el9.x86_64 30/34 2024-09-21T21:37:51.966 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-base-2:19.1.1-373.g4e45e6b4.el9.x86_64 30/34 2024-09-21T21:37:51.966 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T21:37:51.966 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T21:37:51.966 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:37:51.998 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-selinux-2:19.1.1-373.g4e45e6b4.el9.x86_64 31/34 2024-09-21T21:37:52.298 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-common-2:19.1.1-373.g4e45e6b4.el9.x86_64 29/34 2024-09-21T21:37:52.338 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-base-2:19.1.1-373.g4e45e6b4.el9.x86_64 30/34 2024-09-21T21:37:52.415 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-base-2:19.1.1-373.g4e45e6b4.el9.x86_64 30/34 2024-09-21T21:37:52.415 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T21:37:52.415 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T21:37:52.415 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:37:52.447 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-selinux-2:19.1.1-373.g4e45e6b4.el9.x86_64 31/34 2024-09-21T21:37:59.086 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-selinux-2:19.1.1-373.g4e45e6b4.el9.x86_64 31/34 2024-09-21T21:37:59.086 INFO:teuthology.orchestra.run.smithi081.stdout:skipping the directory /sys 2024-09-21T21:37:59.086 INFO:teuthology.orchestra.run.smithi081.stdout:skipping the directory /proc 2024-09-21T21:37:59.086 INFO:teuthology.orchestra.run.smithi081.stdout:skipping the directory /mnt 2024-09-21T21:37:59.086 INFO:teuthology.orchestra.run.smithi081.stdout:skipping the directory /var/tmp 2024-09-21T21:37:59.086 INFO:teuthology.orchestra.run.smithi081.stdout:skipping the directory /home 2024-09-21T21:37:59.086 INFO:teuthology.orchestra.run.smithi081.stdout:skipping the directory /root 2024-09-21T21:37:59.086 INFO:teuthology.orchestra.run.smithi081.stdout:skipping the directory /tmp 2024-09-21T21:37:59.086 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:38:00.079 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-radosgw-2:19.1.1-373.g4e45e6b4.el9.x86_64 32/34 2024-09-21T21:38:00.111 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-373.g4e45e6b4.el9.x86_64 32/34 2024-09-21T21:38:00.111 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T21:38:00.111 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T21:38:00.111 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T21:38:00.111 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T21:38:00.111 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:38:00.113 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-21T21:38:00.199 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-21T21:38:00.199 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-21T21:38:01.344 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-21T21:38:01.344 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-base-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/34 2024-09-21T21:38:01.344 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-common-2:19.1.1-373.g4e45e6b4.el9.x86_64 2/34 2024-09-21T21:38:01.345 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-radosgw-2:19.1.1-373.g4e45e6b4.el9.x86_64 3/34 2024-09-21T21:38:01.345 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-selinux-2:19.1.1-373.g4e45e6b4.el9.x86_64 4/34 2024-09-21T21:38:01.345 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libcephfs2-2:19.1.1-373.g4e45e6b4.el9.x86_64 5/34 2024-09-21T21:38:01.345 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libradosstriper1-2:19.1.1-373.g4e45e6b4.el9.x86_64 6/34 2024-09-21T21:38:01.345 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librgw2-2:19.1.1-373.g4e45e6b4.el9.x86_64 7/34 2024-09-21T21:38:01.345 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-ceph-argparse-2:19.1.1-373.g4e45e6b4.el9.x 8/34 2024-09-21T21:38:01.345 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-ceph-common-2:19.1.1-373.g4e45e6b4.el9.x86 9/34 2024-09-21T21:38:01.346 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-cephfs-2:19.1.1-373.g4e45e6b4.el9.x86_64 10/34 2024-09-21T21:38:01.346 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-rados-2:19.1.1-373.g4e45e6b4.el9.x86_64 11/34 2024-09-21T21:38:01.346 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-rbd-2:19.1.1-373.g4e45e6b4.el9.x86_64 12/34 2024-09-21T21:38:01.346 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-rgw-2:19.1.1-373.g4e45e6b4.el9.x86_64 13/34 2024-09-21T21:38:01.346 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-09-21T21:38:01.346 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-09-21T21:38:01.346 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-09-21T21:38:01.346 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-09-21T21:38:01.346 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-09-21T21:38:01.346 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-09-21T21:38:01.346 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-09-21T21:38:01.346 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-09-21T21:38:01.347 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-09-21T21:38:01.347 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 23/34 2024-09-21T21:38:01.347 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 24/34 2024-09-21T21:38:01.347 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 25/34 2024-09-21T21:38:01.347 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-09-21T21:38:01.347 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-09-21T21:38:01.347 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 28/34 2024-09-21T21:38:01.347 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : re2-1:20211101-20.el9.x86_64 29/34 2024-09-21T21:38:01.347 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-09-21T21:38:01.347 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librados2-2:19.1.1-373.g4e45e6b4.el9.x86_64 31/34 2024-09-21T21:38:01.347 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-09-21T21:38:01.347 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librbd1-2:19.1.1-373.g4e45e6b4.el9.x86_64 33/34 2024-09-21T21:38:01.870 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-selinux-2:19.1.1-373.g4e45e6b4.el9.x86_64 31/34 2024-09-21T21:38:01.871 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /sys 2024-09-21T21:38:01.871 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /proc 2024-09-21T21:38:01.871 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /mnt 2024-09-21T21:38:01.871 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /var/tmp 2024-09-21T21:38:01.871 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /home 2024-09-21T21:38:01.871 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /root 2024-09-21T21:38:01.871 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /tmp 2024-09-21T21:38:01.871 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:38:02.247 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-09-21T21:38:02.247 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:38:02.247 INFO:teuthology.orchestra.run.smithi081.stdout:Upgraded: 2024-09-21T21:38:02.247 INFO:teuthology.orchestra.run.smithi081.stdout: librados2-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:38:02.247 INFO:teuthology.orchestra.run.smithi081.stdout: librbd1-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:38:02.247 INFO:teuthology.orchestra.run.smithi081.stdout:Installed: 2024-09-21T21:38:02.247 INFO:teuthology.orchestra.run.smithi081.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T21:38:02.247 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-base-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:38:02.247 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-common-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:38:02.247 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-radosgw-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:38:02.247 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-selinux-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:38:02.247 INFO:teuthology.orchestra.run.smithi081.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T21:38:02.248 INFO:teuthology.orchestra.run.smithi081.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T21:38:02.248 INFO:teuthology.orchestra.run.smithi081.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T21:38:02.248 INFO:teuthology.orchestra.run.smithi081.stdout: libcephfs2-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:38:02.248 INFO:teuthology.orchestra.run.smithi081.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T21:38:02.248 INFO:teuthology.orchestra.run.smithi081.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T21:38:02.248 INFO:teuthology.orchestra.run.smithi081.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T21:38:02.248 INFO:teuthology.orchestra.run.smithi081.stdout: libradosstriper1-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:38:02.248 INFO:teuthology.orchestra.run.smithi081.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T21:38:02.248 INFO:teuthology.orchestra.run.smithi081.stdout: librgw2-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:38:02.248 INFO:teuthology.orchestra.run.smithi081.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T21:38:02.248 INFO:teuthology.orchestra.run.smithi081.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T21:38:02.248 INFO:teuthology.orchestra.run.smithi081.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T21:38:02.248 INFO:teuthology.orchestra.run.smithi081.stdout: python3-ceph-argparse-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:38:02.249 INFO:teuthology.orchestra.run.smithi081.stdout: python3-ceph-common-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:38:02.249 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cephfs-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:38:02.249 INFO:teuthology.orchestra.run.smithi081.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T21:38:02.249 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rados-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:38:02.249 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rbd-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:38:02.249 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rgw-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:38:02.249 INFO:teuthology.orchestra.run.smithi081.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-21T21:38:02.249 INFO:teuthology.orchestra.run.smithi081.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-21T21:38:02.249 INFO:teuthology.orchestra.run.smithi081.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-21T21:38:02.249 INFO:teuthology.orchestra.run.smithi081.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T21:38:02.249 INFO:teuthology.orchestra.run.smithi081.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T21:38:02.249 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:38:02.249 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-09-21T21:38:02.617 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum -y install ceph-test 2024-09-21T21:38:02.817 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-selinux-2:19.1.1-373.g4e45e6b4.el9.x86_64 31/34 2024-09-21T21:38:02.817 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /sys 2024-09-21T21:38:02.817 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /proc 2024-09-21T21:38:02.817 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /mnt 2024-09-21T21:38:02.817 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /var/tmp 2024-09-21T21:38:02.817 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /home 2024-09-21T21:38:02.817 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /root 2024-09-21T21:38:02.817 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /tmp 2024-09-21T21:38:02.817 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:38:02.875 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-radosgw-2:19.1.1-373.g4e45e6b4.el9.x86_64 32/34 2024-09-21T21:38:02.903 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-373.g4e45e6b4.el9.x86_64 32/34 2024-09-21T21:38:02.903 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T21:38:02.903 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T21:38:02.903 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T21:38:02.903 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T21:38:02.904 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:38:02.905 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-21T21:38:02.986 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-21T21:38:02.986 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-21T21:38:03.202 INFO:teuthology.orchestra.run.smithi081.stdout:Last metadata expiration check: 0:00:27 ago on Sat 21 Sep 2024 09:37:36 PM UTC. 2024-09-21T21:38:03.336 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-09-21T21:38:03.336 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-09-21T21:38:03.337 INFO:teuthology.orchestra.run.smithi081.stdout: Package Arch Version Repository Size 2024-09-21T21:38:03.337 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-09-21T21:38:03.337 INFO:teuthology.orchestra.run.smithi081.stdout:Installing: 2024-09-21T21:38:03.337 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-test x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 49 M 2024-09-21T21:38:03.337 INFO:teuthology.orchestra.run.smithi081.stdout:Installing dependencies: 2024-09-21T21:38:03.337 INFO:teuthology.orchestra.run.smithi081.stdout: libcephsqlite x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 163 k 2024-09-21T21:38:03.337 INFO:teuthology.orchestra.run.smithi081.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T21:38:03.337 INFO:teuthology.orchestra.run.smithi081.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T21:38:03.337 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:38:03.337 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction Summary 2024-09-21T21:38:03.337 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-09-21T21:38:03.337 INFO:teuthology.orchestra.run.smithi081.stdout:Install 4 Packages 2024-09-21T21:38:03.337 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:38:03.338 INFO:teuthology.orchestra.run.smithi081.stdout:Total download size: 49 M 2024-09-21T21:38:03.338 INFO:teuthology.orchestra.run.smithi081.stdout:Installed size: 209 M 2024-09-21T21:38:03.338 INFO:teuthology.orchestra.run.smithi081.stdout:Downloading Packages: 2024-09-21T21:38:03.604 INFO:teuthology.orchestra.run.smithi081.stdout:(1/4): libcephsqlite-19.1.1-373.g4e45e6b4.el9.x 750 kB/s | 163 kB 00:00 2024-09-21T21:38:03.663 INFO:teuthology.orchestra.run.smithi081.stdout:(2/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.1 MB/s | 304 kB 00:00 2024-09-21T21:38:03.813 INFO:teuthology.orchestra.run.smithi081.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 305 kB/s | 64 kB 00:00 2024-09-21T21:38:03.826 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-radosgw-2:19.1.1-373.g4e45e6b4.el9.x86_64 32/34 2024-09-21T21:38:03.851 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-373.g4e45e6b4.el9.x86_64 32/34 2024-09-21T21:38:03.851 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T21:38:03.851 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T21:38:03.851 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T21:38:03.852 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T21:38:03.852 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:38:03.853 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-21T21:38:03.937 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-21T21:38:03.937 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-21T21:38:04.236 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-21T21:38:04.237 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-base-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/34 2024-09-21T21:38:04.237 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-common-2:19.1.1-373.g4e45e6b4.el9.x86_64 2/34 2024-09-21T21:38:04.237 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-radosgw-2:19.1.1-373.g4e45e6b4.el9.x86_64 3/34 2024-09-21T21:38:04.237 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-selinux-2:19.1.1-373.g4e45e6b4.el9.x86_64 4/34 2024-09-21T21:38:04.237 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libcephfs2-2:19.1.1-373.g4e45e6b4.el9.x86_64 5/34 2024-09-21T21:38:04.237 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libradosstriper1-2:19.1.1-373.g4e45e6b4.el9.x86_64 6/34 2024-09-21T21:38:04.237 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librgw2-2:19.1.1-373.g4e45e6b4.el9.x86_64 7/34 2024-09-21T21:38:04.237 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-ceph-argparse-2:19.1.1-373.g4e45e6b4.el9.x 8/34 2024-09-21T21:38:04.237 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-ceph-common-2:19.1.1-373.g4e45e6b4.el9.x86 9/34 2024-09-21T21:38:04.237 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cephfs-2:19.1.1-373.g4e45e6b4.el9.x86_64 10/34 2024-09-21T21:38:04.237 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-rados-2:19.1.1-373.g4e45e6b4.el9.x86_64 11/34 2024-09-21T21:38:04.238 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-rbd-2:19.1.1-373.g4e45e6b4.el9.x86_64 12/34 2024-09-21T21:38:04.238 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-rgw-2:19.1.1-373.g4e45e6b4.el9.x86_64 13/34 2024-09-21T21:38:04.238 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-09-21T21:38:04.238 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-09-21T21:38:04.238 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-09-21T21:38:04.238 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-09-21T21:38:04.238 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-09-21T21:38:04.238 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-09-21T21:38:04.238 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-09-21T21:38:04.238 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-09-21T21:38:04.238 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-09-21T21:38:04.238 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 23/34 2024-09-21T21:38:04.239 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 24/34 2024-09-21T21:38:04.239 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 25/34 2024-09-21T21:38:04.239 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-09-21T21:38:04.239 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-09-21T21:38:04.239 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 28/34 2024-09-21T21:38:04.239 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : re2-1:20211101-20.el9.x86_64 29/34 2024-09-21T21:38:04.239 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-09-21T21:38:04.239 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librados2-2:19.1.1-373.g4e45e6b4.el9.x86_64 31/34 2024-09-21T21:38:04.239 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-09-21T21:38:04.239 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librbd1-2:19.1.1-373.g4e45e6b4.el9.x86_64 33/34 2024-09-21T21:38:04.922 INFO:teuthology.orchestra.run.smithi081.stdout:(4/4): ceph-test-19.1.1-373.g4e45e6b4.el9.x86_6 32 MB/s | 49 MB 00:01 2024-09-21T21:38:04.924 INFO:teuthology.orchestra.run.smithi081.stdout:-------------------------------------------------------------------------------- 2024-09-21T21:38:04.924 INFO:teuthology.orchestra.run.smithi081.stdout:Total 31 MB/s | 49 MB 00:01 2024-09-21T21:38:04.950 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction check 2024-09-21T21:38:04.970 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction check succeeded. 2024-09-21T21:38:04.970 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction test 2024-09-21T21:38:05.161 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-21T21:38:05.162 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-base-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/34 2024-09-21T21:38:05.162 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-common-2:19.1.1-373.g4e45e6b4.el9.x86_64 2/34 2024-09-21T21:38:05.162 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-radosgw-2:19.1.1-373.g4e45e6b4.el9.x86_64 3/34 2024-09-21T21:38:05.162 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-selinux-2:19.1.1-373.g4e45e6b4.el9.x86_64 4/34 2024-09-21T21:38:05.162 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libcephfs2-2:19.1.1-373.g4e45e6b4.el9.x86_64 5/34 2024-09-21T21:38:05.162 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libradosstriper1-2:19.1.1-373.g4e45e6b4.el9.x86_64 6/34 2024-09-21T21:38:05.162 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librgw2-2:19.1.1-373.g4e45e6b4.el9.x86_64 7/34 2024-09-21T21:38:05.162 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-ceph-argparse-2:19.1.1-373.g4e45e6b4.el9.x 8/34 2024-09-21T21:38:05.162 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-ceph-common-2:19.1.1-373.g4e45e6b4.el9.x86 9/34 2024-09-21T21:38:05.162 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cephfs-2:19.1.1-373.g4e45e6b4.el9.x86_64 10/34 2024-09-21T21:38:05.162 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-rados-2:19.1.1-373.g4e45e6b4.el9.x86_64 11/34 2024-09-21T21:38:05.162 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-rbd-2:19.1.1-373.g4e45e6b4.el9.x86_64 12/34 2024-09-21T21:38:05.162 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-rgw-2:19.1.1-373.g4e45e6b4.el9.x86_64 13/34 2024-09-21T21:38:05.162 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-09-21T21:38:05.163 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-09-21T21:38:05.163 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-09-21T21:38:05.163 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-09-21T21:38:05.163 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-09-21T21:38:05.163 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-09-21T21:38:05.163 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-09-21T21:38:05.163 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-09-21T21:38:05.163 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-09-21T21:38:05.163 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 23/34 2024-09-21T21:38:05.163 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 24/34 2024-09-21T21:38:05.163 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 25/34 2024-09-21T21:38:05.163 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-09-21T21:38:05.164 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-09-21T21:38:05.164 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 28/34 2024-09-21T21:38:05.164 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : re2-1:20211101-20.el9.x86_64 29/34 2024-09-21T21:38:05.164 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-09-21T21:38:05.164 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librados2-2:19.1.1-373.g4e45e6b4.el9.x86_64 31/34 2024-09-21T21:38:05.165 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-09-21T21:38:05.165 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librbd1-2:19.1.1-373.g4e45e6b4.el9.x86_64 33/34 2024-09-21T21:38:05.277 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-09-21T21:38:05.277 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:38:05.277 INFO:teuthology.orchestra.run.smithi122.stdout:Upgraded: 2024-09-21T21:38:05.277 INFO:teuthology.orchestra.run.smithi122.stdout: librados2-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:38:05.277 INFO:teuthology.orchestra.run.smithi122.stdout: librbd1-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:38:05.277 INFO:teuthology.orchestra.run.smithi122.stdout:Installed: 2024-09-21T21:38:05.277 INFO:teuthology.orchestra.run.smithi122.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T21:38:05.277 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-base-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:38:05.277 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-common-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:38:05.278 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-radosgw-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:38:05.278 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-selinux-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:38:05.278 INFO:teuthology.orchestra.run.smithi122.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T21:38:05.278 INFO:teuthology.orchestra.run.smithi122.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T21:38:05.278 INFO:teuthology.orchestra.run.smithi122.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T21:38:05.278 INFO:teuthology.orchestra.run.smithi122.stdout: libcephfs2-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:38:05.278 INFO:teuthology.orchestra.run.smithi122.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T21:38:05.278 INFO:teuthology.orchestra.run.smithi122.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T21:38:05.278 INFO:teuthology.orchestra.run.smithi122.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T21:38:05.278 INFO:teuthology.orchestra.run.smithi122.stdout: libradosstriper1-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:38:05.278 INFO:teuthology.orchestra.run.smithi122.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T21:38:05.278 INFO:teuthology.orchestra.run.smithi122.stdout: librgw2-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:38:05.278 INFO:teuthology.orchestra.run.smithi122.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T21:38:05.278 INFO:teuthology.orchestra.run.smithi122.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T21:38:05.279 INFO:teuthology.orchestra.run.smithi122.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T21:38:05.279 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ceph-argparse-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:38:05.279 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ceph-common-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:38:05.279 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cephfs-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:38:05.279 INFO:teuthology.orchestra.run.smithi122.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T21:38:05.279 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rados-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:38:05.279 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rbd-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:38:05.279 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rgw-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:38:05.279 INFO:teuthology.orchestra.run.smithi122.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-21T21:38:05.279 INFO:teuthology.orchestra.run.smithi122.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-21T21:38:05.279 INFO:teuthology.orchestra.run.smithi122.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-21T21:38:05.279 INFO:teuthology.orchestra.run.smithi122.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T21:38:05.279 INFO:teuthology.orchestra.run.smithi122.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T21:38:05.279 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:38:05.280 INFO:teuthology.orchestra.run.smithi122.stdout:Complete! 2024-09-21T21:38:05.338 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction test succeeded. 2024-09-21T21:38:05.338 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction 2024-09-21T21:38:05.664 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum -y install ceph-test 2024-09-21T21:38:05.864 INFO:teuthology.orchestra.run.smithi081.stdout: Preparing : 1/1 2024-09-21T21:38:05.954 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-21T21:38:05.995 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-21T21:38:06.023 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libcephsqlite-2:19.1.1-373.g4e45e6b4.el9.x86_64 3/4 2024-09-21T21:38:06.244 INFO:teuthology.orchestra.run.smithi122.stdout:Last metadata expiration check: 0:00:29 ago on Sat 21 Sep 2024 09:37:37 PM UTC. 2024-09-21T21:38:06.289 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-09-21T21:38:06.289 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:38:06.289 INFO:teuthology.orchestra.run.smithi164.stdout:Upgraded: 2024-09-21T21:38:06.289 INFO:teuthology.orchestra.run.smithi164.stdout: librados2-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:38:06.289 INFO:teuthology.orchestra.run.smithi164.stdout: librbd1-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:38:06.289 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-09-21T21:38:06.289 INFO:teuthology.orchestra.run.smithi164.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T21:38:06.289 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-base-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:38:06.290 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-common-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:38:06.290 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-radosgw-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:38:06.290 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-selinux-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:38:06.290 INFO:teuthology.orchestra.run.smithi164.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T21:38:06.290 INFO:teuthology.orchestra.run.smithi164.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T21:38:06.290 INFO:teuthology.orchestra.run.smithi164.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T21:38:06.290 INFO:teuthology.orchestra.run.smithi164.stdout: libcephfs2-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:38:06.290 INFO:teuthology.orchestra.run.smithi164.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T21:38:06.290 INFO:teuthology.orchestra.run.smithi164.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T21:38:06.290 INFO:teuthology.orchestra.run.smithi164.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T21:38:06.290 INFO:teuthology.orchestra.run.smithi164.stdout: libradosstriper1-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:38:06.290 INFO:teuthology.orchestra.run.smithi164.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T21:38:06.290 INFO:teuthology.orchestra.run.smithi164.stdout: librgw2-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:38:06.291 INFO:teuthology.orchestra.run.smithi164.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T21:38:06.291 INFO:teuthology.orchestra.run.smithi164.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T21:38:06.291 INFO:teuthology.orchestra.run.smithi164.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T21:38:06.291 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ceph-argparse-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:38:06.291 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ceph-common-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:38:06.291 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cephfs-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:38:06.291 INFO:teuthology.orchestra.run.smithi164.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T21:38:06.291 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rados-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:38:06.291 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rbd-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:38:06.291 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rgw-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:38:06.291 INFO:teuthology.orchestra.run.smithi164.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-21T21:38:06.291 INFO:teuthology.orchestra.run.smithi164.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-21T21:38:06.291 INFO:teuthology.orchestra.run.smithi164.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-21T21:38:06.292 INFO:teuthology.orchestra.run.smithi164.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T21:38:06.292 INFO:teuthology.orchestra.run.smithi164.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T21:38:06.292 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:38:06.292 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-21T21:38:06.383 INFO:teuthology.orchestra.run.smithi122.stdout:Dependencies resolved. 2024-09-21T21:38:06.383 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-21T21:38:06.384 INFO:teuthology.orchestra.run.smithi122.stdout: Package Arch Version Repository Size 2024-09-21T21:38:06.384 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-21T21:38:06.384 INFO:teuthology.orchestra.run.smithi122.stdout:Installing: 2024-09-21T21:38:06.384 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-test x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 49 M 2024-09-21T21:38:06.384 INFO:teuthology.orchestra.run.smithi122.stdout:Installing dependencies: 2024-09-21T21:38:06.384 INFO:teuthology.orchestra.run.smithi122.stdout: libcephsqlite x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 163 k 2024-09-21T21:38:06.384 INFO:teuthology.orchestra.run.smithi122.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T21:38:06.384 INFO:teuthology.orchestra.run.smithi122.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T21:38:06.384 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:38:06.384 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction Summary 2024-09-21T21:38:06.384 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-21T21:38:06.384 INFO:teuthology.orchestra.run.smithi122.stdout:Install 4 Packages 2024-09-21T21:38:06.385 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:38:06.385 INFO:teuthology.orchestra.run.smithi122.stdout:Total download size: 49 M 2024-09-21T21:38:06.385 INFO:teuthology.orchestra.run.smithi122.stdout:Installed size: 209 M 2024-09-21T21:38:06.385 INFO:teuthology.orchestra.run.smithi122.stdout:Downloading Packages: 2024-09-21T21:38:06.608 INFO:teuthology.orchestra.run.smithi122.stdout:(1/4): libcephsqlite-19.1.1-373.g4e45e6b4.el9.x 822 kB/s | 163 kB 00:00 2024-09-21T21:38:06.633 INFO:teuthology.orchestra.run.smithi122.stdout:(2/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.3 MB/s | 304 kB 00:00 2024-09-21T21:38:06.671 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install ceph-test 2024-09-21T21:38:06.750 INFO:teuthology.orchestra.run.smithi122.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 448 kB/s | 64 kB 00:00 2024-09-21T21:38:07.239 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:00:29 ago on Sat 21 Sep 2024 09:37:38 PM UTC. 2024-09-21T21:38:07.377 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-21T21:38:07.377 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-21T21:38:07.377 INFO:teuthology.orchestra.run.smithi164.stdout: Package Arch Version Repository Size 2024-09-21T21:38:07.377 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-21T21:38:07.377 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-09-21T21:38:07.377 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-test x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 49 M 2024-09-21T21:38:07.377 INFO:teuthology.orchestra.run.smithi164.stdout:Installing dependencies: 2024-09-21T21:38:07.378 INFO:teuthology.orchestra.run.smithi164.stdout: libcephsqlite x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 163 k 2024-09-21T21:38:07.378 INFO:teuthology.orchestra.run.smithi164.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T21:38:07.378 INFO:teuthology.orchestra.run.smithi164.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T21:38:07.378 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:38:07.378 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-09-21T21:38:07.378 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-21T21:38:07.378 INFO:teuthology.orchestra.run.smithi164.stdout:Install 4 Packages 2024-09-21T21:38:07.378 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:38:07.379 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 49 M 2024-09-21T21:38:07.379 INFO:teuthology.orchestra.run.smithi164.stdout:Installed size: 209 M 2024-09-21T21:38:07.379 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-09-21T21:38:07.709 INFO:teuthology.orchestra.run.smithi164.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.4 MB/s | 304 kB 00:00 2024-09-21T21:38:07.734 INFO:teuthology.orchestra.run.smithi164.stdout:(2/4): libcephsqlite-19.1.1-373.g4e45e6b4.el9.x 694 kB/s | 163 kB 00:00 2024-09-21T21:38:07.759 INFO:teuthology.orchestra.run.smithi164.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-21T21:38:07.920 INFO:teuthology.orchestra.run.smithi122.stdout:(4/4): ceph-test-19.1.1-373.g4e45e6b4.el9.x86_6 32 MB/s | 49 MB 00:01 2024-09-21T21:38:07.922 INFO:teuthology.orchestra.run.smithi122.stdout:-------------------------------------------------------------------------------- 2024-09-21T21:38:07.922 INFO:teuthology.orchestra.run.smithi122.stdout:Total 32 MB/s | 49 MB 00:01 2024-09-21T21:38:07.950 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction check 2024-09-21T21:38:07.970 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction check succeeded. 2024-09-21T21:38:07.970 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction test 2024-09-21T21:38:08.339 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction test succeeded. 2024-09-21T21:38:08.339 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction 2024-09-21T21:38:08.866 INFO:teuthology.orchestra.run.smithi122.stdout: Preparing : 1/1 2024-09-21T21:38:08.944 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-21T21:38:08.982 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-21T21:38:09.011 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libcephsqlite-2:19.1.1-373.g4e45e6b4.el9.x86_64 3/4 2024-09-21T21:38:09.026 INFO:teuthology.orchestra.run.smithi164.stdout:(4/4): ceph-test-19.1.1-373.g4e45e6b4.el9.x86_6 32 MB/s | 49 MB 00:01 2024-09-21T21:38:09.028 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-09-21T21:38:09.029 INFO:teuthology.orchestra.run.smithi164.stdout:Total 30 MB/s | 49 MB 00:01 2024-09-21T21:38:09.055 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-09-21T21:38:09.076 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-09-21T21:38:09.076 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-09-21T21:38:09.450 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-09-21T21:38:09.450 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-09-21T21:38:09.988 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-09-21T21:38:10.079 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-21T21:38:10.125 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-21T21:38:10.153 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libcephsqlite-2:19.1.1-373.g4e45e6b4.el9.x86_64 3/4 2024-09-21T21:38:10.618 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: libcephsqlite-2:19.1.1-373.g4e45e6b4.el9.x86_64 3/4 2024-09-21T21:38:10.705 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-test-2:19.1.1-373.g4e45e6b4.el9.x86_64 4/4 2024-09-21T21:38:13.139 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-test-2:19.1.1-373.g4e45e6b4.el9.x86_64 4/4 2024-09-21T21:38:13.139 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-test-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/4 2024-09-21T21:38:13.139 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libcephsqlite-2:19.1.1-373.g4e45e6b4.el9.x86_64 2/4 2024-09-21T21:38:13.139 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-21T21:38:13.470 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-21T21:38:13.470 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:38:13.470 INFO:teuthology.orchestra.run.smithi081.stdout:Installed: 2024-09-21T21:38:13.470 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-test-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:38:13.470 INFO:teuthology.orchestra.run.smithi081.stdout: libcephsqlite-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:38:13.470 INFO:teuthology.orchestra.run.smithi081.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T21:38:13.470 INFO:teuthology.orchestra.run.smithi081.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T21:38:13.470 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:38:13.470 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-09-21T21:38:13.611 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: libcephsqlite-2:19.1.1-373.g4e45e6b4.el9.x86_64 3/4 2024-09-21T21:38:13.692 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-test-2:19.1.1-373.g4e45e6b4.el9.x86_64 4/4 2024-09-21T21:38:13.773 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum -y install ceph 2024-09-21T21:38:14.378 INFO:teuthology.orchestra.run.smithi081.stdout:Last metadata expiration check: 0:00:38 ago on Sat 21 Sep 2024 09:37:36 PM UTC. 2024-09-21T21:38:14.519 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-09-21T21:38:14.521 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-09-21T21:38:14.521 INFO:teuthology.orchestra.run.smithi081.stdout: Package Arch Version Repository Size 2024-09-21T21:38:14.522 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-09-21T21:38:14.522 INFO:teuthology.orchestra.run.smithi081.stdout:Installing: 2024-09-21T21:38:14.522 INFO:teuthology.orchestra.run.smithi081.stdout: ceph x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 6.6 k 2024-09-21T21:38:14.522 INFO:teuthology.orchestra.run.smithi081.stdout:Installing dependencies: 2024-09-21T21:38:14.522 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mds x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 2.4 M 2024-09-21T21:38:14.522 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 1.5 M 2024-09-21T21:38:14.522 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-modules-core noarch 2:19.1.1-373.g4e45e6b4.el9 ceph-noarch 249 k 2024-09-21T21:38:14.522 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mon x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 4.7 M 2024-09-21T21:38:14.522 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-osd x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 17 M 2024-09-21T21:38:14.522 INFO:teuthology.orchestra.run.smithi081.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-21T21:38:14.522 INFO:teuthology.orchestra.run.smithi081.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-21T21:38:14.522 INFO:teuthology.orchestra.run.smithi081.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-21T21:38:14.522 INFO:teuthology.orchestra.run.smithi081.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T21:38:14.523 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T21:38:14.523 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T21:38:14.523 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T21:38:14.523 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T21:38:14.523 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T21:38:14.523 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T21:38:14.523 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-21T21:38:14.523 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T21:38:14.523 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T21:38:14.523 INFO:teuthology.orchestra.run.smithi081.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T21:38:14.523 INFO:teuthology.orchestra.run.smithi081.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T21:38:14.523 INFO:teuthology.orchestra.run.smithi081.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T21:38:14.523 INFO:teuthology.orchestra.run.smithi081.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T21:38:14.523 INFO:teuthology.orchestra.run.smithi081.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T21:38:14.524 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T21:38:14.524 INFO:teuthology.orchestra.run.smithi081.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T21:38:14.524 INFO:teuthology.orchestra.run.smithi081.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T21:38:14.524 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T21:38:14.524 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T21:38:14.524 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T21:38:14.524 INFO:teuthology.orchestra.run.smithi081.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T21:38:14.524 INFO:teuthology.orchestra.run.smithi081.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T21:38:14.524 INFO:teuthology.orchestra.run.smithi081.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T21:38:14.524 INFO:teuthology.orchestra.run.smithi081.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T21:38:14.524 INFO:teuthology.orchestra.run.smithi081.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T21:38:14.524 INFO:teuthology.orchestra.run.smithi081.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T21:38:14.524 INFO:teuthology.orchestra.run.smithi081.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T21:38:14.524 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:38:14.525 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction Summary 2024-09-21T21:38:14.525 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-09-21T21:38:14.525 INFO:teuthology.orchestra.run.smithi081.stdout:Install 37 Packages 2024-09-21T21:38:14.525 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:38:14.525 INFO:teuthology.orchestra.run.smithi081.stdout:Total download size: 30 M 2024-09-21T21:38:14.525 INFO:teuthology.orchestra.run.smithi081.stdout:Installed size: 108 M 2024-09-21T21:38:14.525 INFO:teuthology.orchestra.run.smithi081.stdout:Downloading Packages: 2024-09-21T21:38:14.744 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: libcephsqlite-2:19.1.1-373.g4e45e6b4.el9.x86_64 3/4 2024-09-21T21:38:14.826 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-test-2:19.1.1-373.g4e45e6b4.el9.x86_64 4/4 2024-09-21T21:38:14.943 INFO:teuthology.orchestra.run.smithi081.stdout:(1/37): ceph-19.1.1-373.g4e45e6b4.el9.x86_64.rp 54 kB/s | 6.6 kB 00:00 2024-09-21T21:38:15.119 INFO:teuthology.orchestra.run.smithi081.stdout:(2/37): ceph-mgr-19.1.1-373.g4e45e6b4.el9.x86_6 5.1 MB/s | 1.5 MB 00:00 2024-09-21T21:38:15.194 INFO:teuthology.orchestra.run.smithi081.stdout:(3/37): ceph-mds-19.1.1-373.g4e45e6b4.el9.x86_6 6.4 MB/s | 2.4 MB 00:00 2024-09-21T21:38:15.319 INFO:teuthology.orchestra.run.smithi081.stdout:(4/37): ceph-mon-19.1.1-373.g4e45e6b4.el9.x86_6 13 MB/s | 4.7 MB 00:00 2024-09-21T21:38:15.344 INFO:teuthology.orchestra.run.smithi081.stdout:(5/37): ceph-mgr-modules-core-19.1.1-373.g4e45e 1.6 MB/s | 249 kB 00:00 2024-09-21T21:38:15.470 INFO:teuthology.orchestra.run.smithi081.stdout:(6/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.6 MB/s | 253 kB 00:00 2024-09-21T21:38:15.520 INFO:teuthology.orchestra.run.smithi081.stdout:(7/37): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-21T21:38:15.554 INFO:teuthology.orchestra.run.smithi081.stdout:(8/37): python3-cryptography-36.0.1-4.el9.x86_6 5.9 MB/s | 1.2 MB 00:00 2024-09-21T21:38:15.579 INFO:teuthology.orchestra.run.smithi081.stdout:(9/37): python3-pycparser-2.20-6.el9.noarch.rpm 2.3 MB/s | 135 kB 00:00 2024-09-21T21:38:15.605 INFO:teuthology.orchestra.run.smithi081.stdout:(10/37): python3-requests-2.25.1-8.el9.noarch.r 2.4 MB/s | 125 kB 00:00 2024-09-21T21:38:15.630 INFO:teuthology.orchestra.run.smithi081.stdout:(11/37): python3-urllib3-1.26.5-6.el9.noarch.rp 4.2 MB/s | 215 kB 00:00 2024-09-21T21:38:15.798 INFO:teuthology.orchestra.run.smithi081.stdout:(12/37): ceph-osd-19.1.1-373.g4e45e6b4.el9.x86_ 25 MB/s | 17 MB 00:00 2024-09-21T21:38:15.847 INFO:teuthology.orchestra.run.smithi081.stdout:(13/37): lua-5.4.4-4.el9.x86_64.rpm 778 kB/s | 188 kB 00:00 2024-09-21T21:38:15.897 INFO:teuthology.orchestra.run.smithi081.stdout:(14/37): python3-mako-1.1.4-6.el9.noarch.rpm 644 kB/s | 172 kB 00:00 2024-09-21T21:38:15.923 INFO:teuthology.orchestra.run.smithi081.stdout:(15/37): python3-markupsafe-1.1.1-12.el9.x86_64 277 kB/s | 35 kB 00:00 2024-09-21T21:38:15.956 INFO:teuthology.orchestra.run.smithi081.stdout:(16/37): python3-packaging-20.9-5.el9.noarch.rp 709 kB/s | 77 kB 00:00 2024-09-21T21:38:15.981 INFO:teuthology.orchestra.run.smithi081.stdout:(17/37): python3-pytz-2021.1-5.el9.noarch.rpm 608 kB/s | 51 kB 00:00 2024-09-21T21:38:16.024 INFO:teuthology.orchestra.run.smithi081.stdout:(18/37): luarocks-3.9.2-5.el9.noarch.rpm 3.5 MB/s | 151 kB 00:00 2024-09-21T21:38:16.049 INFO:teuthology.orchestra.run.smithi081.stdout:(19/37): python3-toml-0.10.2-6.el9.noarch.rpm 331 kB/s | 42 kB 00:00 2024-09-21T21:38:16.069 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-test-2:19.1.1-373.g4e45e6b4.el9.x86_64 4/4 2024-09-21T21:38:16.069 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-test-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/4 2024-09-21T21:38:16.069 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libcephsqlite-2:19.1.1-373.g4e45e6b4.el9.x86_64 2/4 2024-09-21T21:38:16.069 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-21T21:38:16.074 INFO:teuthology.orchestra.run.smithi081.stdout:(20/37): lua-devel-5.4.4-4.el9.x86_64.rpm 190 kB/s | 22 kB 00:00 2024-09-21T21:38:16.108 INFO:teuthology.orchestra.run.smithi081.stdout:(21/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 518 kB/s | 43 kB 00:00 2024-09-21T21:38:16.141 INFO:teuthology.orchestra.run.smithi081.stdout:(22/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 447 kB/s | 11 kB 00:00 2024-09-21T21:38:16.166 INFO:teuthology.orchestra.run.smithi081.stdout:(23/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.4 MB/s | 172 kB 00:00 2024-09-21T21:38:16.192 INFO:teuthology.orchestra.run.smithi081.stdout:(24/37): python3-jaraco-classes-3.2.1-5.el9.noa 364 kB/s | 18 kB 00:00 2024-09-21T21:38:16.217 INFO:teuthology.orchestra.run.smithi081.stdout:(25/37): python3-jaraco-collections-3.0.0-8.el9 460 kB/s | 23 kB 00:00 2024-09-21T21:38:16.251 INFO:teuthology.orchestra.run.smithi081.stdout:(26/37): python3-jaraco-functools-3.5.0-2.el9.n 331 kB/s | 19 kB 00:00 2024-09-21T21:38:16.268 INFO:teuthology.orchestra.run.smithi081.stdout:(27/37): python3-jaraco-text-3.2.0-6.el9.noarch 392 kB/s | 20 kB 00:00 2024-09-21T21:38:16.293 INFO:teuthology.orchestra.run.smithi081.stdout:(28/37): python3-logutils-0.3.5-21.el9.noarch.r 1.1 MB/s | 46 kB 00:00 2024-09-21T21:38:16.318 INFO:teuthology.orchestra.run.smithi081.stdout:(29/37): python3-more-itertools-8.12.0-2.el9.no 1.5 MB/s | 79 kB 00:00 2024-09-21T21:38:16.335 INFO:teuthology.orchestra.run.smithi081.stdout:(30/37): python3-portend-3.1.0-2.el9.noarch.rpm 992 kB/s | 16 kB 00:00 2024-09-21T21:38:16.361 INFO:teuthology.orchestra.run.smithi081.stdout:(31/37): python3-cherrypy-18.6.1-2.el9.noarch.r 1.3 MB/s | 358 kB 00:00 2024-09-21T21:38:16.384 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-21T21:38:16.384 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:38:16.384 INFO:teuthology.orchestra.run.smithi122.stdout:Installed: 2024-09-21T21:38:16.384 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-test-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:38:16.384 INFO:teuthology.orchestra.run.smithi122.stdout: libcephsqlite-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:38:16.384 INFO:teuthology.orchestra.run.smithi122.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T21:38:16.384 INFO:teuthology.orchestra.run.smithi122.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T21:38:16.384 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:38:16.384 INFO:teuthology.orchestra.run.smithi122.stdout:Complete! 2024-09-21T21:38:16.386 INFO:teuthology.orchestra.run.smithi081.stdout:(32/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.7 MB/s | 90 kB 00:00 2024-09-21T21:38:16.411 INFO:teuthology.orchestra.run.smithi081.stdout:(33/37): python3-pecan-1.4.2-3.el9.noarch.rpm 2.3 MB/s | 272 kB 00:00 2024-09-21T21:38:16.428 INFO:teuthology.orchestra.run.smithi081.stdout:(34/37): python3-tempora-5.0.0-2.el9.noarch.rpm 531 kB/s | 36 kB 00:00 2024-09-21T21:38:16.471 INFO:teuthology.orchestra.run.smithi081.stdout:(35/37): python3-webob-1.8.8-2.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-09-21T21:38:16.487 INFO:teuthology.orchestra.run.smithi081.stdout:(36/37): python3-zc-lockfile-2.0-10.el9.noarch. 341 kB/s | 20 kB 00:00 2024-09-21T21:38:16.512 INFO:teuthology.orchestra.run.smithi081.stdout:(37/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.1 MB/s | 427 kB 00:00 2024-09-21T21:38:16.517 INFO:teuthology.orchestra.run.smithi081.stdout:-------------------------------------------------------------------------------- 2024-09-21T21:38:16.517 INFO:teuthology.orchestra.run.smithi081.stdout:Total 15 MB/s | 30 MB 00:01 2024-09-21T21:38:16.643 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum -y install ceph 2024-09-21T21:38:16.668 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction check 2024-09-21T21:38:16.709 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction check succeeded. 2024-09-21T21:38:16.709 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction test 2024-09-21T21:38:17.126 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-test-2:19.1.1-373.g4e45e6b4.el9.x86_64 4/4 2024-09-21T21:38:17.126 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-test-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/4 2024-09-21T21:38:17.126 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libcephsqlite-2:19.1.1-373.g4e45e6b4.el9.x86_64 2/4 2024-09-21T21:38:17.126 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-21T21:38:17.189 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction test succeeded. 2024-09-21T21:38:17.189 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction 2024-09-21T21:38:17.252 INFO:teuthology.orchestra.run.smithi122.stdout:Last metadata expiration check: 0:00:40 ago on Sat 21 Sep 2024 09:37:37 PM UTC. 2024-09-21T21:38:17.401 INFO:teuthology.orchestra.run.smithi122.stdout:Dependencies resolved. 2024-09-21T21:38:17.404 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-21T21:38:17.404 INFO:teuthology.orchestra.run.smithi122.stdout: Package Arch Version Repository Size 2024-09-21T21:38:17.404 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-21T21:38:17.404 INFO:teuthology.orchestra.run.smithi122.stdout:Installing: 2024-09-21T21:38:17.404 INFO:teuthology.orchestra.run.smithi122.stdout: ceph x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 6.6 k 2024-09-21T21:38:17.404 INFO:teuthology.orchestra.run.smithi122.stdout:Installing dependencies: 2024-09-21T21:38:17.404 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mds x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 2.4 M 2024-09-21T21:38:17.404 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 1.5 M 2024-09-21T21:38:17.405 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-modules-core noarch 2:19.1.1-373.g4e45e6b4.el9 ceph-noarch 249 k 2024-09-21T21:38:17.405 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mon x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 4.7 M 2024-09-21T21:38:17.405 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-osd x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 17 M 2024-09-21T21:38:17.405 INFO:teuthology.orchestra.run.smithi122.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-21T21:38:17.405 INFO:teuthology.orchestra.run.smithi122.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-21T21:38:17.405 INFO:teuthology.orchestra.run.smithi122.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-21T21:38:17.405 INFO:teuthology.orchestra.run.smithi122.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T21:38:17.405 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T21:38:17.405 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T21:38:17.405 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T21:38:17.405 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T21:38:17.405 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T21:38:17.405 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T21:38:17.406 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-21T21:38:17.406 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T21:38:17.406 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T21:38:17.406 INFO:teuthology.orchestra.run.smithi122.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T21:38:17.406 INFO:teuthology.orchestra.run.smithi122.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T21:38:17.406 INFO:teuthology.orchestra.run.smithi122.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T21:38:17.406 INFO:teuthology.orchestra.run.smithi122.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T21:38:17.406 INFO:teuthology.orchestra.run.smithi122.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T21:38:17.406 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T21:38:17.406 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T21:38:17.406 INFO:teuthology.orchestra.run.smithi122.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T21:38:17.406 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T21:38:17.406 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T21:38:17.406 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T21:38:17.407 INFO:teuthology.orchestra.run.smithi122.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T21:38:17.407 INFO:teuthology.orchestra.run.smithi122.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T21:38:17.407 INFO:teuthology.orchestra.run.smithi122.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T21:38:17.407 INFO:teuthology.orchestra.run.smithi122.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T21:38:17.407 INFO:teuthology.orchestra.run.smithi122.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T21:38:17.407 INFO:teuthology.orchestra.run.smithi122.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T21:38:17.407 INFO:teuthology.orchestra.run.smithi122.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T21:38:17.407 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:38:17.407 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction Summary 2024-09-21T21:38:17.407 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-21T21:38:17.407 INFO:teuthology.orchestra.run.smithi122.stdout:Install 37 Packages 2024-09-21T21:38:17.407 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:38:17.408 INFO:teuthology.orchestra.run.smithi122.stdout:Total download size: 30 M 2024-09-21T21:38:17.408 INFO:teuthology.orchestra.run.smithi122.stdout:Installed size: 108 M 2024-09-21T21:38:17.408 INFO:teuthology.orchestra.run.smithi122.stdout:Downloading Packages: 2024-09-21T21:38:17.443 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-21T21:38:17.443 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:38:17.443 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-09-21T21:38:17.444 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-test-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:38:17.444 INFO:teuthology.orchestra.run.smithi164.stdout: libcephsqlite-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:38:17.444 INFO:teuthology.orchestra.run.smithi164.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T21:38:17.444 INFO:teuthology.orchestra.run.smithi164.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T21:38:17.444 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:38:17.444 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-21T21:38:17.691 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install ceph 2024-09-21T21:38:17.901 INFO:teuthology.orchestra.run.smithi122.stdout:(1/37): ceph-19.1.1-373.g4e45e6b4.el9.x86_64.rp 51 kB/s | 6.6 kB 00:00 2024-09-21T21:38:18.127 INFO:teuthology.orchestra.run.smithi122.stdout:(2/37): ceph-mds-19.1.1-373.g4e45e6b4.el9.x86_6 6.8 MB/s | 2.4 MB 00:00 2024-09-21T21:38:18.160 INFO:teuthology.orchestra.run.smithi122.stdout:(3/37): ceph-mgr-19.1.1-373.g4e45e6b4.el9.x86_6 3.9 MB/s | 1.5 MB 00:00 2024-09-21T21:38:18.261 INFO:teuthology.orchestra.run.smithi122.stdout:(4/37): ceph-mgr-modules-core-19.1.1-373.g4e45e 2.4 MB/s | 249 kB 00:00 2024-09-21T21:38:18.275 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:00:40 ago on Sat 21 Sep 2024 09:37:38 PM UTC. 2024-09-21T21:38:18.353 INFO:teuthology.orchestra.run.smithi122.stdout:(5/37): ceph-mon-19.1.1-373.g4e45e6b4.el9.x86_6 10 MB/s | 4.7 MB 00:00 2024-09-21T21:38:18.412 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-21T21:38:18.415 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-21T21:38:18.415 INFO:teuthology.orchestra.run.smithi164.stdout: Package Arch Version Repository Size 2024-09-21T21:38:18.415 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-21T21:38:18.415 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-09-21T21:38:18.415 INFO:teuthology.orchestra.run.smithi164.stdout: ceph x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 6.6 k 2024-09-21T21:38:18.415 INFO:teuthology.orchestra.run.smithi164.stdout:Installing dependencies: 2024-09-21T21:38:18.415 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mds x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 2.4 M 2024-09-21T21:38:18.415 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 1.5 M 2024-09-21T21:38:18.415 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-modules-core noarch 2:19.1.1-373.g4e45e6b4.el9 ceph-noarch 249 k 2024-09-21T21:38:18.415 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mon x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 4.7 M 2024-09-21T21:38:18.415 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-osd x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 17 M 2024-09-21T21:38:18.415 INFO:teuthology.orchestra.run.smithi164.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-21T21:38:18.415 INFO:teuthology.orchestra.run.smithi164.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-21T21:38:18.416 INFO:teuthology.orchestra.run.smithi164.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-21T21:38:18.416 INFO:teuthology.orchestra.run.smithi164.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T21:38:18.416 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T21:38:18.416 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T21:38:18.416 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T21:38:18.416 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T21:38:18.416 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T21:38:18.416 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T21:38:18.416 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-21T21:38:18.416 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T21:38:18.416 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T21:38:18.416 INFO:teuthology.orchestra.run.smithi164.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T21:38:18.416 INFO:teuthology.orchestra.run.smithi164.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T21:38:18.417 INFO:teuthology.orchestra.run.smithi164.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T21:38:18.417 INFO:teuthology.orchestra.run.smithi164.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T21:38:18.417 INFO:teuthology.orchestra.run.smithi164.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T21:38:18.417 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T21:38:18.417 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T21:38:18.417 INFO:teuthology.orchestra.run.smithi164.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T21:38:18.417 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T21:38:18.417 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T21:38:18.417 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T21:38:18.417 INFO:teuthology.orchestra.run.smithi164.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T21:38:18.417 INFO:teuthology.orchestra.run.smithi164.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T21:38:18.417 INFO:teuthology.orchestra.run.smithi164.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T21:38:18.417 INFO:teuthology.orchestra.run.smithi164.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T21:38:18.417 INFO:teuthology.orchestra.run.smithi164.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T21:38:18.418 INFO:teuthology.orchestra.run.smithi164.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T21:38:18.418 INFO:teuthology.orchestra.run.smithi164.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T21:38:18.418 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:38:18.418 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-09-21T21:38:18.418 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-21T21:38:18.418 INFO:teuthology.orchestra.run.smithi164.stdout:Install 37 Packages 2024-09-21T21:38:18.418 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:38:18.418 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 30 M 2024-09-21T21:38:18.419 INFO:teuthology.orchestra.run.smithi164.stdout:Installed size: 108 M 2024-09-21T21:38:18.419 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-09-21T21:38:18.503 INFO:teuthology.orchestra.run.smithi122.stdout:(6/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.0 MB/s | 253 kB 00:00 2024-09-21T21:38:18.555 INFO:teuthology.orchestra.run.smithi081.stdout: Preparing : 1/1 2024-09-21T21:38:18.615 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-21T21:38:18.648 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-21T21:38:18.678 INFO:teuthology.orchestra.run.smithi122.stdout:(7/37): ceph-osd-19.1.1-373.g4e45e6b4.el9.x86_6 31 MB/s | 17 MB 00:00 2024-09-21T21:38:18.684 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-21T21:38:18.703 INFO:teuthology.orchestra.run.smithi122.stdout:(8/37): python3-ply-3.11-14.el9.noarch.rpm 532 kB/s | 106 kB 00:00 2024-09-21T21:38:18.755 INFO:teuthology.orchestra.run.smithi122.stdout:(9/37): python3-requests-2.25.1-8.el9.noarch.rp 2.4 MB/s | 125 kB 00:00 2024-09-21T21:38:18.855 INFO:teuthology.orchestra.run.smithi164.stdout:(1/37): ceph-19.1.1-373.g4e45e6b4.el9.x86_64.rp 56 kB/s | 6.6 kB 00:00 2024-09-21T21:38:18.856 INFO:teuthology.orchestra.run.smithi122.stdout:(10/37): python3-urllib3-1.26.5-6.el9.noarch.rp 3.6 MB/s | 215 kB 00:00 2024-09-21T21:38:18.856 INFO:teuthology.orchestra.run.smithi122.stdout:(11/37): python3-pycparser-2.20-6.el9.noarch.rp 851 kB/s | 135 kB 00:00 2024-09-21T21:38:18.856 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-21T21:38:18.856 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-21T21:38:18.856 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-21T21:38:18.933 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-21T21:38:18.979 INFO:teuthology.orchestra.run.smithi122.stdout:(12/37): lua-5.4.4-4.el9.x86_64.rpm 1.1 MB/s | 188 kB 00:00 2024-09-21T21:38:18.986 INFO:teuthology.orchestra.run.smithi164.stdout:(2/37): ceph-mgr-19.1.1-373.g4e45e6b4.el9.x86_6 4.9 MB/s | 1.5 MB 00:00 2024-09-21T21:38:19.004 INFO:teuthology.orchestra.run.smithi122.stdout:(13/37): python3-mako-1.1.4-6.el9.noarch.rpm 1.0 MB/s | 172 kB 00:00 2024-09-21T21:38:19.028 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-21T21:38:19.030 INFO:teuthology.orchestra.run.smithi122.stdout:(14/37): python3-markupsafe-1.1.1-12.el9.x86_64 690 kB/s | 35 kB 00:00 2024-09-21T21:38:19.037 INFO:teuthology.orchestra.run.smithi164.stdout:(3/37): ceph-mds-19.1.1-373.g4e45e6b4.el9.x86_6 6.6 MB/s | 2.4 MB 00:00 2024-09-21T21:38:19.047 INFO:teuthology.orchestra.run.smithi122.stdout:(15/37): python3-packaging-20.9-5.el9.noarch.rp 1.8 MB/s | 77 kB 00:00 2024-09-21T21:38:19.078 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-21T21:38:19.082 INFO:teuthology.orchestra.run.smithi122.stdout:(16/37): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-21T21:38:19.106 INFO:teuthology.orchestra.run.smithi122.stdout:(17/37): python3-toml-0.10.2-6.el9.noarch.rpm 710 kB/s | 42 kB 00:00 2024-09-21T21:38:19.112 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-21T21:38:19.137 INFO:teuthology.orchestra.run.smithi164.stdout:(4/37): ceph-mon-19.1.1-373.g4e45e6b4.el9.x86_6 14 MB/s | 4.7 MB 00:00 2024-09-21T21:38:19.152 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-21T21:38:19.156 INFO:teuthology.orchestra.run.smithi122.stdout:(18/37): luarocks-3.9.2-5.el9.noarch.rpm 3.0 MB/s | 151 kB 00:00 2024-09-21T21:38:19.162 INFO:teuthology.orchestra.run.smithi164.stdout:(5/37): ceph-mgr-modules-core-19.1.1-373.g4e45e 1.9 MB/s | 249 kB 00:00 2024-09-21T21:38:19.189 INFO:teuthology.orchestra.run.smithi122.stdout:(19/37): python3-cryptography-36.0.1-4.el9.x86_ 1.5 MB/s | 1.2 MB 00:00 2024-09-21T21:38:19.195 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-21T21:38:19.215 INFO:teuthology.orchestra.run.smithi122.stdout:(20/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 742 kB/s | 43 kB 00:00 2024-09-21T21:38:19.228 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-21T21:38:19.249 INFO:teuthology.orchestra.run.smithi122.stdout:(21/37): lua-devel-5.4.4-4.el9.x86_64.rpm 133 kB/s | 22 kB 00:00 2024-09-21T21:38:19.269 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-21T21:38:19.271 INFO:teuthology.orchestra.run.smithi164.stdout:(6/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.9 MB/s | 253 kB 00:00 2024-09-21T21:38:19.291 INFO:teuthology.orchestra.run.smithi122.stdout:(22/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.7 MB/s | 172 kB 00:00 2024-09-21T21:38:19.312 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-21T21:38:19.313 INFO:teuthology.orchestra.run.smithi164.stdout:(7/37): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-09-21T21:38:19.316 INFO:teuthology.orchestra.run.smithi122.stdout:(23/37): python3-cherrypy-18.6.1-2.el9.noarch.r 3.7 MB/s | 358 kB 00:00 2024-09-21T21:38:19.341 INFO:teuthology.orchestra.run.smithi122.stdout:(24/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 118 kB/s | 11 kB 00:00 2024-09-21T21:38:19.346 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-21T21:38:19.355 INFO:teuthology.orchestra.run.smithi164.stdout:(8/37): python3-cryptography-36.0.1-4.el9.x86_6 6.4 MB/s | 1.2 MB 00:00 2024-09-21T21:38:19.375 INFO:teuthology.orchestra.run.smithi122.stdout:(25/37): python3-jaraco-classes-3.2.1-5.el9.noa 214 kB/s | 18 kB 00:00 2024-09-21T21:38:19.380 INFO:teuthology.orchestra.run.smithi164.stdout:(9/37): python3-pycparser-2.20-6.el9.noarch.rpm 2.0 MB/s | 135 kB 00:00 2024-09-21T21:38:19.395 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-21T21:38:19.405 INFO:teuthology.orchestra.run.smithi164.stdout:(10/37): python3-requests-2.25.1-8.el9.noarch.r 2.4 MB/s | 125 kB 00:00 2024-09-21T21:38:19.408 INFO:teuthology.orchestra.run.smithi122.stdout:(26/37): python3-jaraco-collections-3.0.0-8.el9 251 kB/s | 23 kB 00:00 2024-09-21T21:38:19.427 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-21T21:38:19.431 INFO:teuthology.orchestra.run.smithi164.stdout:(11/37): python3-urllib3-1.26.5-6.el9.noarch.rp 4.2 MB/s | 215 kB 00:00 2024-09-21T21:38:19.433 INFO:teuthology.orchestra.run.smithi122.stdout:(27/37): python3-jaraco-functools-3.5.0-2.el9.n 210 kB/s | 19 kB 00:00 2024-09-21T21:38:19.456 INFO:teuthology.orchestra.run.smithi164.stdout:(12/37): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-09-21T21:38:19.459 INFO:teuthology.orchestra.run.smithi122.stdout:(28/37): python3-jaraco-text-3.2.0-6.el9.noarch 235 kB/s | 20 kB 00:00 2024-09-21T21:38:19.481 INFO:teuthology.orchestra.run.smithi164.stdout:(13/37): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-09-21T21:38:19.484 INFO:teuthology.orchestra.run.smithi122.stdout:(29/37): python3-logutils-0.3.5-21.el9.noarch.r 921 kB/s | 46 kB 00:00 2024-09-21T21:38:19.507 INFO:teuthology.orchestra.run.smithi164.stdout:(14/37): python3-markupsafe-1.1.1-12.el9.x86_64 690 kB/s | 35 kB 00:00 2024-09-21T21:38:19.509 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-21T21:38:19.509 INFO:teuthology.orchestra.run.smithi122.stdout:(30/37): python3-more-itertools-8.12.0-2.el9.no 1.0 MB/s | 79 kB 00:00 2024-09-21T21:38:19.524 INFO:teuthology.orchestra.run.smithi164.stdout:(15/37): python3-packaging-20.9-5.el9.noarch.rp 1.8 MB/s | 77 kB 00:00 2024-09-21T21:38:19.535 INFO:teuthology.orchestra.run.smithi122.stdout:(31/37): python3-portend-3.1.0-2.el9.noarch.rpm 327 kB/s | 16 kB 00:00 2024-09-21T21:38:19.549 INFO:teuthology.orchestra.run.smithi164.stdout:(16/37): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 51 kB 00:00 2024-09-21T21:38:19.560 INFO:teuthology.orchestra.run.smithi122.stdout:(32/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.7 MB/s | 90 kB 00:00 2024-09-21T21:38:19.574 INFO:teuthology.orchestra.run.smithi164.stdout:(17/37): python3-toml-0.10.2-6.el9.noarch.rpm 828 kB/s | 42 kB 00:00 2024-09-21T21:38:19.585 INFO:teuthology.orchestra.run.smithi122.stdout:(33/37): python3-tempora-5.0.0-2.el9.noarch.rpm 715 kB/s | 36 kB 00:00 2024-09-21T21:38:19.610 INFO:teuthology.orchestra.run.smithi122.stdout:(34/37): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-09-21T21:38:19.636 INFO:teuthology.orchestra.run.smithi122.stdout:(35/37): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-21T21:38:19.661 INFO:teuthology.orchestra.run.smithi122.stdout:(36/37): python3-zc-lockfile-2.0-10.el9.noarch. 398 kB/s | 20 kB 00:00 2024-09-21T21:38:19.663 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-21T21:38:19.690 INFO:teuthology.orchestra.run.smithi122.stdout:(37/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.0 MB/s | 427 kB 00:00 2024-09-21T21:38:19.694 INFO:teuthology.orchestra.run.smithi122.stdout:-------------------------------------------------------------------------------- 2024-09-21T21:38:19.694 INFO:teuthology.orchestra.run.smithi122.stdout:Total 13 MB/s | 30 MB 00:02 2024-09-21T21:38:19.744 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-21T21:38:19.750 INFO:teuthology.orchestra.run.smithi164.stdout:(18/37): ceph-osd-19.1.1-373.g4e45e6b4.el9.x86_ 22 MB/s | 17 MB 00:00 2024-09-21T21:38:19.783 INFO:teuthology.orchestra.run.smithi164.stdout:(19/37): luarocks-3.9.2-5.el9.noarch.rpm 725 kB/s | 151 kB 00:00 2024-09-21T21:38:19.798 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-21T21:38:19.808 INFO:teuthology.orchestra.run.smithi164.stdout:(20/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 770 kB/s | 43 kB 00:00 2024-09-21T21:38:19.834 INFO:teuthology.orchestra.run.smithi164.stdout:(21/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-09-21T21:38:19.837 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-21T21:38:19.859 INFO:teuthology.orchestra.run.smithi164.stdout:(22/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 430 kB/s | 11 kB 00:00 2024-09-21T21:38:19.863 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction check 2024-09-21T21:38:19.884 INFO:teuthology.orchestra.run.smithi164.stdout:(23/37): python3-jaraco-classes-3.2.1-5.el9.noa 704 kB/s | 18 kB 00:00 2024-09-21T21:38:19.902 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-21T21:38:19.909 INFO:teuthology.orchestra.run.smithi164.stdout:(24/37): python3-jaraco-collections-3.0.0-8.el9 930 kB/s | 23 kB 00:00 2024-09-21T21:38:19.911 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction check succeeded. 2024-09-21T21:38:19.911 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction test 2024-09-21T21:38:19.938 INFO:teuthology.orchestra.run.smithi164.stdout:(25/37): python3-cherrypy-18.6.1-2.el9.noarch.r 2.7 MB/s | 358 kB 00:00 2024-09-21T21:38:19.958 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-21T21:38:19.964 INFO:teuthology.orchestra.run.smithi164.stdout:(26/37): lua-devel-5.4.4-4.el9.x86_64.rpm 54 kB/s | 22 kB 00:00 2024-09-21T21:38:19.981 INFO:teuthology.orchestra.run.smithi164.stdout:(27/37): python3-jaraco-functools-3.5.0-2.el9.n 274 kB/s | 19 kB 00:00 2024-09-21T21:38:19.998 INFO:teuthology.orchestra.run.smithi164.stdout:(28/37): python3-jaraco-text-3.2.0-6.el9.noarch 333 kB/s | 20 kB 00:00 2024-09-21T21:38:20.023 INFO:teuthology.orchestra.run.smithi164.stdout:(29/37): python3-logutils-0.3.5-21.el9.noarch.r 789 kB/s | 46 kB 00:00 2024-09-21T21:38:20.040 INFO:teuthology.orchestra.run.smithi164.stdout:(30/37): python3-portend-3.1.0-2.el9.noarch.rpm 1.1 MB/s | 16 kB 00:00 2024-09-21T21:38:20.065 INFO:teuthology.orchestra.run.smithi164.stdout:(31/37): python3-more-itertools-8.12.0-2.el9.no 938 kB/s | 79 kB 00:00 2024-09-21T21:38:20.099 INFO:teuthology.orchestra.run.smithi164.stdout:(32/37): python3-pecan-1.4.2-3.el9.noarch.rpm 2.8 MB/s | 272 kB 00:00 2024-09-21T21:38:20.115 INFO:teuthology.orchestra.run.smithi164.stdout:(33/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.2 MB/s | 90 kB 00:00 2024-09-21T21:38:20.141 INFO:teuthology.orchestra.run.smithi164.stdout:(34/37): python3-tempora-5.0.0-2.el9.noarch.rpm 474 kB/s | 36 kB 00:00 2024-09-21T21:38:20.155 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-21T21:38:20.166 INFO:teuthology.orchestra.run.smithi164.stdout:(35/37): python3-zc-lockfile-2.0-10.el9.noarch. 797 kB/s | 20 kB 00:00 2024-09-21T21:38:20.199 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-21T21:38:20.200 INFO:teuthology.orchestra.run.smithi164.stdout:(36/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 5.0 MB/s | 427 kB 00:00 2024-09-21T21:38:20.225 INFO:teuthology.orchestra.run.smithi164.stdout:(37/37): python3-webob-1.8.8-2.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-09-21T21:38:20.229 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-09-21T21:38:20.229 INFO:teuthology.orchestra.run.smithi164.stdout:Total 17 MB/s | 30 MB 00:01 2024-09-21T21:38:20.269 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-21T21:38:20.358 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-21T21:38:20.373 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-09-21T21:38:20.398 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction test succeeded. 2024-09-21T21:38:20.398 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction 2024-09-21T21:38:20.415 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-09-21T21:38:20.415 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-09-21T21:38:20.432 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-21T21:38:20.558 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-21T21:38:20.721 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-373.g4e45e6b4.el9.n 32/37 2024-09-21T21:38:20.760 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mgr-2:19.1.1-373.g4e45e6b4.el9.x86_64 33/37 2024-09-21T21:38:20.786 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-mgr-2:19.1.1-373.g4e45e6b4.el9.x86_64 33/37 2024-09-21T21:38:20.786 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T21:38:20.786 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T21:38:20.786 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T21:38:20.786 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T21:38:20.786 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:38:20.879 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-09-21T21:38:20.880 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-09-21T21:38:21.747 INFO:teuthology.orchestra.run.smithi122.stdout: Preparing : 1/1 2024-09-21T21:38:21.809 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-21T21:38:21.842 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-21T21:38:21.870 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-21T21:38:21.944 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-21T21:38:22.002 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-21T21:38:22.035 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-21T21:38:22.126 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-21T21:38:22.205 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-21T21:38:22.219 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-osd-2:19.1.1-373.g4e45e6b4.el9.x86_64 34/37 2024-09-21T21:38:22.246 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-osd-2:19.1.1-373.g4e45e6b4.el9.x86_64 34/37 2024-09-21T21:38:22.246 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T21:38:22.246 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T21:38:22.246 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T21:38:22.246 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T21:38:22.246 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:38:22.247 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-21T21:38:22.277 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-09-21T21:38:22.281 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-21T21:38:22.311 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-21T21:38:22.336 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-21T21:38:22.345 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-21T21:38:22.369 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-21T21:38:22.379 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-21T21:38:22.397 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-21T21:38:22.419 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-21T21:38:22.463 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-21T21:38:22.468 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-21T21:38:22.506 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-21T21:38:22.513 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-21T21:38:22.545 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-21T21:38:22.553 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-21T21:38:22.586 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-21T21:38:22.638 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-21T21:38:22.659 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-21T21:38:22.686 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mon-2:19.1.1-373.g4e45e6b4.el9.x86_64 35/37 2024-09-21T21:38:22.710 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-mon-2:19.1.1-373.g4e45e6b4.el9.x86_64 35/37 2024-09-21T21:38:22.710 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T21:38:22.710 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T21:38:22.710 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T21:38:22.710 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T21:38:22.710 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:38:22.724 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-21T21:38:22.765 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-21T21:38:22.800 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-21T21:38:22.805 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-21T21:38:22.830 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-21T21:38:22.864 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-21T21:38:22.885 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-21T21:38:22.898 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-21T21:38:22.937 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mds-2:19.1.1-373.g4e45e6b4.el9.x86_64 36/37 2024-09-21T21:38:22.938 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-21T21:38:22.939 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-21T21:38:22.961 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-mds-2:19.1.1-373.g4e45e6b4.el9.x86_64 36/37 2024-09-21T21:38:22.961 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T21:38:22.961 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T21:38:22.961 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T21:38:22.961 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T21:38:22.961 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:38:22.977 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-21T21:38:22.982 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-21T21:38:23.016 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-21T21:38:23.026 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-2:19.1.1-373.g4e45e6b4.el9.x86_64 37/37 2024-09-21T21:38:23.026 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-21T21:38:23.065 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-21T21:38:23.082 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-21T21:38:23.097 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-21T21:38:23.170 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-21T21:38:23.268 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-21T21:38:23.309 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-21T21:38:23.316 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-21T21:38:23.394 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-21T21:38:23.396 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-21T21:38:23.452 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-21T21:38:23.483 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-21T21:38:23.499 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-21T21:38:23.524 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-21T21:38:23.556 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-21T21:38:23.612 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-21T21:38:23.640 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-21T21:38:23.796 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-373.g4e45e6b4.el9.n 32/37 2024-09-21T21:38:23.798 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-21T21:38:23.826 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-2:19.1.1-373.g4e45e6b4.el9.x86_64 33/37 2024-09-21T21:38:23.853 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mgr-2:19.1.1-373.g4e45e6b4.el9.x86_64 33/37 2024-09-21T21:38:23.853 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T21:38:23.853 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T21:38:23.853 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T21:38:23.854 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T21:38:23.854 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:38:23.854 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-21T21:38:23.931 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-21T21:38:24.020 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-21T21:38:24.061 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-21T21:38:24.177 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-21T21:38:24.332 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-373.g4e45e6b4.el9.n 32/37 2024-09-21T21:38:24.355 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-2:19.1.1-373.g4e45e6b4.el9.x86_64 33/37 2024-09-21T21:38:24.382 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-2:19.1.1-373.g4e45e6b4.el9.x86_64 33/37 2024-09-21T21:38:24.382 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T21:38:24.383 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T21:38:24.383 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T21:38:24.383 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T21:38:24.383 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:38:24.850 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-2:19.1.1-373.g4e45e6b4.el9.x86_64 37/37 2024-09-21T21:38:24.851 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/37 2024-09-21T21:38:24.851 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mds-2:19.1.1-373.g4e45e6b4.el9.x86_64 2/37 2024-09-21T21:38:24.851 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mgr-2:19.1.1-373.g4e45e6b4.el9.x86_64 3/37 2024-09-21T21:38:24.851 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mon-2:19.1.1-373.g4e45e6b4.el9.x86_64 4/37 2024-09-21T21:38:24.851 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-osd-2:19.1.1-373.g4e45e6b4.el9.x86_64 5/37 2024-09-21T21:38:24.851 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-373.g4e45e6b4.el9.n 6/37 2024-09-21T21:38:24.851 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-21T21:38:24.851 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-21T21:38:24.851 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-21T21:38:24.851 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-21T21:38:24.851 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-21T21:38:24.852 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-21T21:38:24.852 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-21T21:38:24.852 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-21T21:38:24.852 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-21T21:38:24.852 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-21T21:38:24.852 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-21T21:38:24.852 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-21T21:38:24.852 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-21T21:38:24.852 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-21T21:38:24.852 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-21T21:38:24.852 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-21T21:38:24.852 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-21T21:38:24.852 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-21T21:38:24.853 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-21T21:38:24.853 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-21T21:38:24.853 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-21T21:38:24.854 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-21T21:38:24.854 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-21T21:38:24.854 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-21T21:38:24.854 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-21T21:38:24.854 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-21T21:38:24.854 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-21T21:38:24.854 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-21T21:38:24.854 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-21T21:38:24.854 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-21T21:38:25.294 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-osd-2:19.1.1-373.g4e45e6b4.el9.x86_64 34/37 2024-09-21T21:38:25.322 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-osd-2:19.1.1-373.g4e45e6b4.el9.x86_64 34/37 2024-09-21T21:38:25.322 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T21:38:25.322 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T21:38:25.322 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T21:38:25.322 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T21:38:25.322 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:38:25.753 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mon-2:19.1.1-373.g4e45e6b4.el9.x86_64 35/37 2024-09-21T21:38:25.776 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mon-2:19.1.1-373.g4e45e6b4.el9.x86_64 35/37 2024-09-21T21:38:25.777 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T21:38:25.777 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T21:38:25.777 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T21:38:25.777 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T21:38:25.777 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:38:25.822 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-osd-2:19.1.1-373.g4e45e6b4.el9.x86_64 34/37 2024-09-21T21:38:25.850 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-osd-2:19.1.1-373.g4e45e6b4.el9.x86_64 34/37 2024-09-21T21:38:25.850 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T21:38:25.850 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T21:38:25.850 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T21:38:25.850 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T21:38:25.850 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:38:25.982 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-21T21:38:25.982 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:38:25.982 INFO:teuthology.orchestra.run.smithi081.stdout:Installed: 2024-09-21T21:38:25.982 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:38:25.982 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mds-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:38:25.982 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:38:25.982 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-modules-core-2:19.1.1-373.g4e45e6b4.el9.noarch 2024-09-21T21:38:25.982 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mon-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:38:25.982 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-osd-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:38:25.982 INFO:teuthology.orchestra.run.smithi081.stdout: lua-5.4.4-4.el9.x86_64 2024-09-21T21:38:25.982 INFO:teuthology.orchestra.run.smithi081.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-21T21:38:25.982 INFO:teuthology.orchestra.run.smithi081.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-21T21:38:25.983 INFO:teuthology.orchestra.run.smithi081.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T21:38:25.983 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T21:38:25.983 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T21:38:25.983 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T21:38:25.983 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T21:38:25.983 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T21:38:25.983 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T21:38:25.983 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T21:38:25.983 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T21:38:25.983 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T21:38:25.983 INFO:teuthology.orchestra.run.smithi081.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T21:38:25.983 INFO:teuthology.orchestra.run.smithi081.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T21:38:25.983 INFO:teuthology.orchestra.run.smithi081.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T21:38:25.983 INFO:teuthology.orchestra.run.smithi081.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T21:38:25.984 INFO:teuthology.orchestra.run.smithi081.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T21:38:25.984 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T21:38:25.984 INFO:teuthology.orchestra.run.smithi081.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T21:38:25.984 INFO:teuthology.orchestra.run.smithi081.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T21:38:25.984 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T21:38:25.984 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T21:38:25.984 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T21:38:25.984 INFO:teuthology.orchestra.run.smithi081.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T21:38:25.984 INFO:teuthology.orchestra.run.smithi081.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T21:38:25.984 INFO:teuthology.orchestra.run.smithi081.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T21:38:25.984 INFO:teuthology.orchestra.run.smithi081.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T21:38:25.984 INFO:teuthology.orchestra.run.smithi081.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T21:38:25.984 INFO:teuthology.orchestra.run.smithi081.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T21:38:25.984 INFO:teuthology.orchestra.run.smithi081.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T21:38:25.984 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:38:25.985 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-09-21T21:38:26.020 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mds-2:19.1.1-373.g4e45e6b4.el9.x86_64 36/37 2024-09-21T21:38:26.046 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mds-2:19.1.1-373.g4e45e6b4.el9.x86_64 36/37 2024-09-21T21:38:26.046 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T21:38:26.046 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T21:38:26.046 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T21:38:26.046 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T21:38:26.046 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:38:26.108 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-2:19.1.1-373.g4e45e6b4.el9.x86_64 37/37 2024-09-21T21:38:26.282 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mon-2:19.1.1-373.g4e45e6b4.el9.x86_64 35/37 2024-09-21T21:38:26.306 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mon-2:19.1.1-373.g4e45e6b4.el9.x86_64 35/37 2024-09-21T21:38:26.306 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T21:38:26.306 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T21:38:26.306 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T21:38:26.306 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T21:38:26.306 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:38:26.351 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum -y install ceph-base 2024-09-21T21:38:26.532 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mds-2:19.1.1-373.g4e45e6b4.el9.x86_64 36/37 2024-09-21T21:38:26.556 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mds-2:19.1.1-373.g4e45e6b4.el9.x86_64 36/37 2024-09-21T21:38:26.556 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T21:38:26.556 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T21:38:26.556 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T21:38:26.556 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T21:38:26.556 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:38:26.622 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-2:19.1.1-373.g4e45e6b4.el9.x86_64 37/37 2024-09-21T21:38:26.939 INFO:teuthology.orchestra.run.smithi081.stdout:Last metadata expiration check: 0:00:50 ago on Sat 21 Sep 2024 09:37:36 PM UTC. 2024-09-21T21:38:27.029 INFO:teuthology.orchestra.run.smithi081.stdout:Package ceph-base-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-21T21:38:27.071 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-09-21T21:38:27.071 INFO:teuthology.orchestra.run.smithi081.stdout:Nothing to do. 2024-09-21T21:38:27.072 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-09-21T21:38:27.142 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum -y install cephadm 2024-09-21T21:38:27.719 INFO:teuthology.orchestra.run.smithi081.stdout:Last metadata expiration check: 0:00:51 ago on Sat 21 Sep 2024 09:37:36 PM UTC. 2024-09-21T21:38:27.851 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-09-21T21:38:27.852 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-09-21T21:38:27.852 INFO:teuthology.orchestra.run.smithi081.stdout: Package Arch Version Repository Size 2024-09-21T21:38:27.852 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-09-21T21:38:27.852 INFO:teuthology.orchestra.run.smithi081.stdout:Installing: 2024-09-21T21:38:27.852 INFO:teuthology.orchestra.run.smithi081.stdout: cephadm noarch 2:19.1.1-373.g4e45e6b4.el9 ceph-noarch 765 k 2024-09-21T21:38:27.853 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:38:27.853 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction Summary 2024-09-21T21:38:27.853 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-09-21T21:38:27.853 INFO:teuthology.orchestra.run.smithi081.stdout:Install 1 Package 2024-09-21T21:38:27.853 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:38:27.853 INFO:teuthology.orchestra.run.smithi081.stdout:Total download size: 765 k 2024-09-21T21:38:27.853 INFO:teuthology.orchestra.run.smithi081.stdout:Installed size: 772 k 2024-09-21T21:38:27.853 INFO:teuthology.orchestra.run.smithi081.stdout:Downloading Packages: 2024-09-21T21:38:27.904 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-2:19.1.1-373.g4e45e6b4.el9.x86_64 37/37 2024-09-21T21:38:27.905 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/37 2024-09-21T21:38:27.905 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mds-2:19.1.1-373.g4e45e6b4.el9.x86_64 2/37 2024-09-21T21:38:27.905 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-2:19.1.1-373.g4e45e6b4.el9.x86_64 3/37 2024-09-21T21:38:27.905 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mon-2:19.1.1-373.g4e45e6b4.el9.x86_64 4/37 2024-09-21T21:38:27.905 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-osd-2:19.1.1-373.g4e45e6b4.el9.x86_64 5/37 2024-09-21T21:38:27.905 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-373.g4e45e6b4.el9.n 6/37 2024-09-21T21:38:27.905 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-21T21:38:27.905 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-21T21:38:27.905 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-21T21:38:27.905 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-21T21:38:27.905 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-21T21:38:27.905 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-21T21:38:27.905 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-21T21:38:27.906 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-21T21:38:27.906 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-21T21:38:27.906 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-21T21:38:27.906 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-21T21:38:27.906 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-21T21:38:27.906 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-21T21:38:27.906 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-21T21:38:27.906 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-21T21:38:27.906 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-21T21:38:27.906 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-21T21:38:27.906 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-21T21:38:27.906 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-21T21:38:27.906 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-21T21:38:27.907 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-21T21:38:27.907 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-21T21:38:27.908 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-21T21:38:27.908 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-21T21:38:27.908 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-21T21:38:27.908 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-21T21:38:27.908 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-21T21:38:27.908 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-21T21:38:27.908 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-21T21:38:27.908 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-21T21:38:28.112 INFO:teuthology.orchestra.run.smithi081.stdout:cephadm-19.1.1-373.g4e45e6b4.el9.noarch.rpm 2.9 MB/s | 765 kB 00:00 2024-09-21T21:38:28.112 INFO:teuthology.orchestra.run.smithi081.stdout:-------------------------------------------------------------------------------- 2024-09-21T21:38:28.112 INFO:teuthology.orchestra.run.smithi081.stdout:Total 2.9 MB/s | 765 kB 00:00 2024-09-21T21:38:28.113 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction check 2024-09-21T21:38:28.118 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction check succeeded. 2024-09-21T21:38:28.118 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction test 2024-09-21T21:38:28.123 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction test succeeded. 2024-09-21T21:38:28.124 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction 2024-09-21T21:38:28.193 INFO:teuthology.orchestra.run.smithi081.stdout: Preparing : 1/1 2024-09-21T21:38:28.415 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-2:19.1.1-373.g4e45e6b4.el9.x86_64 37/37 2024-09-21T21:38:28.415 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/37 2024-09-21T21:38:28.415 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mds-2:19.1.1-373.g4e45e6b4.el9.x86_64 2/37 2024-09-21T21:38:28.415 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-2:19.1.1-373.g4e45e6b4.el9.x86_64 3/37 2024-09-21T21:38:28.416 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mon-2:19.1.1-373.g4e45e6b4.el9.x86_64 4/37 2024-09-21T21:38:28.416 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-osd-2:19.1.1-373.g4e45e6b4.el9.x86_64 5/37 2024-09-21T21:38:28.416 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-373.g4e45e6b4.el9.n 6/37 2024-09-21T21:38:28.416 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-21T21:38:28.416 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-21T21:38:28.416 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-21T21:38:28.416 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-21T21:38:28.416 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-21T21:38:28.416 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-21T21:38:28.416 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-21T21:38:28.416 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-21T21:38:28.416 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-21T21:38:28.416 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-21T21:38:28.417 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-21T21:38:28.417 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-21T21:38:28.417 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-21T21:38:28.417 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-21T21:38:28.417 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-21T21:38:28.417 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-21T21:38:28.417 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-21T21:38:28.417 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-21T21:38:28.417 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-21T21:38:28.417 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-21T21:38:28.417 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-21T21:38:28.417 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-21T21:38:28.417 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-21T21:38:28.417 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-21T21:38:28.417 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-21T21:38:28.418 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-21T21:38:28.418 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-21T21:38:28.418 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-21T21:38:28.418 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-21T21:38:28.418 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-21T21:38:28.597 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: cephadm-2:19.1.1-373.g4e45e6b4.el9.noarch 1/1 2024-09-21T21:38:28.681 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : cephadm-2:19.1.1-373.g4e45e6b4.el9.noarch 1/1 2024-09-21T21:38:28.823 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-21T21:38:28.823 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:38:28.823 INFO:teuthology.orchestra.run.smithi122.stdout:Installed: 2024-09-21T21:38:28.824 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:38:28.824 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mds-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:38:28.824 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:38:28.824 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-modules-core-2:19.1.1-373.g4e45e6b4.el9.noarch 2024-09-21T21:38:28.824 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mon-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:38:28.824 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-osd-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:38:28.824 INFO:teuthology.orchestra.run.smithi122.stdout: lua-5.4.4-4.el9.x86_64 2024-09-21T21:38:28.824 INFO:teuthology.orchestra.run.smithi122.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-21T21:38:28.824 INFO:teuthology.orchestra.run.smithi122.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-21T21:38:28.824 INFO:teuthology.orchestra.run.smithi122.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T21:38:28.824 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T21:38:28.824 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T21:38:28.824 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T21:38:28.824 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T21:38:28.825 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T21:38:28.825 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T21:38:28.825 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T21:38:28.825 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T21:38:28.825 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T21:38:28.825 INFO:teuthology.orchestra.run.smithi122.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T21:38:28.825 INFO:teuthology.orchestra.run.smithi122.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T21:38:28.825 INFO:teuthology.orchestra.run.smithi122.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T21:38:28.825 INFO:teuthology.orchestra.run.smithi122.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T21:38:28.825 INFO:teuthology.orchestra.run.smithi122.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T21:38:28.825 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T21:38:28.825 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T21:38:28.825 INFO:teuthology.orchestra.run.smithi122.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T21:38:28.825 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T21:38:28.825 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T21:38:28.826 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T21:38:28.826 INFO:teuthology.orchestra.run.smithi122.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T21:38:28.826 INFO:teuthology.orchestra.run.smithi122.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T21:38:28.826 INFO:teuthology.orchestra.run.smithi122.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T21:38:28.826 INFO:teuthology.orchestra.run.smithi122.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T21:38:28.826 INFO:teuthology.orchestra.run.smithi122.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T21:38:28.826 INFO:teuthology.orchestra.run.smithi122.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T21:38:28.826 INFO:teuthology.orchestra.run.smithi122.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T21:38:28.826 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:38:28.826 INFO:teuthology.orchestra.run.smithi122.stdout:Complete! 2024-09-21T21:38:29.041 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: cephadm-2:19.1.1-373.g4e45e6b4.el9.noarch 1/1 2024-09-21T21:38:29.223 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum -y install ceph-base 2024-09-21T21:38:29.269 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : cephadm-2:19.1.1-373.g4e45e6b4.el9.noarch 1/1 2024-09-21T21:38:29.269 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:38:29.269 INFO:teuthology.orchestra.run.smithi081.stdout:Installed: 2024-09-21T21:38:29.269 INFO:teuthology.orchestra.run.smithi081.stdout: cephadm-2:19.1.1-373.g4e45e6b4.el9.noarch 2024-09-21T21:38:29.269 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:38:29.269 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-09-21T21:38:29.343 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-21T21:38:29.343 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:38:29.343 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-09-21T21:38:29.343 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:38:29.343 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mds-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:38:29.343 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:38:29.343 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-modules-core-2:19.1.1-373.g4e45e6b4.el9.noarch 2024-09-21T21:38:29.343 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mon-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:38:29.343 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-osd-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:38:29.344 INFO:teuthology.orchestra.run.smithi164.stdout: lua-5.4.4-4.el9.x86_64 2024-09-21T21:38:29.344 INFO:teuthology.orchestra.run.smithi164.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-21T21:38:29.344 INFO:teuthology.orchestra.run.smithi164.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-21T21:38:29.344 INFO:teuthology.orchestra.run.smithi164.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T21:38:29.344 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T21:38:29.344 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T21:38:29.344 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T21:38:29.344 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T21:38:29.344 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T21:38:29.344 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T21:38:29.344 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T21:38:29.344 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T21:38:29.344 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T21:38:29.344 INFO:teuthology.orchestra.run.smithi164.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T21:38:29.345 INFO:teuthology.orchestra.run.smithi164.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T21:38:29.345 INFO:teuthology.orchestra.run.smithi164.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T21:38:29.345 INFO:teuthology.orchestra.run.smithi164.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T21:38:29.345 INFO:teuthology.orchestra.run.smithi164.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T21:38:29.345 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T21:38:29.345 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T21:38:29.345 INFO:teuthology.orchestra.run.smithi164.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T21:38:29.345 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T21:38:29.345 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T21:38:29.345 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T21:38:29.345 INFO:teuthology.orchestra.run.smithi164.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T21:38:29.345 INFO:teuthology.orchestra.run.smithi164.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T21:38:29.345 INFO:teuthology.orchestra.run.smithi164.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T21:38:29.345 INFO:teuthology.orchestra.run.smithi164.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T21:38:29.346 INFO:teuthology.orchestra.run.smithi164.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T21:38:29.346 INFO:teuthology.orchestra.run.smithi164.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T21:38:29.346 INFO:teuthology.orchestra.run.smithi164.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T21:38:29.346 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:38:29.346 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-21T21:38:29.485 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum -y install ceph-immutable-object-cache 2024-09-21T21:38:29.711 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install ceph-base 2024-09-21T21:38:29.800 INFO:teuthology.orchestra.run.smithi122.stdout:Last metadata expiration check: 0:00:52 ago on Sat 21 Sep 2024 09:37:37 PM UTC. 2024-09-21T21:38:29.889 INFO:teuthology.orchestra.run.smithi122.stdout:Package ceph-base-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-21T21:38:29.931 INFO:teuthology.orchestra.run.smithi122.stdout:Dependencies resolved. 2024-09-21T21:38:29.932 INFO:teuthology.orchestra.run.smithi122.stdout:Nothing to do. 2024-09-21T21:38:29.932 INFO:teuthology.orchestra.run.smithi122.stdout:Complete! 2024-09-21T21:38:29.997 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum -y install cephadm 2024-09-21T21:38:30.073 INFO:teuthology.orchestra.run.smithi081.stdout:Last metadata expiration check: 0:00:54 ago on Sat 21 Sep 2024 09:37:36 PM UTC. 2024-09-21T21:38:30.209 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-09-21T21:38:30.209 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-09-21T21:38:30.210 INFO:teuthology.orchestra.run.smithi081.stdout: Package Arch Version Repo Size 2024-09-21T21:38:30.210 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-09-21T21:38:30.210 INFO:teuthology.orchestra.run.smithi081.stdout:Installing: 2024-09-21T21:38:30.210 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 145 k 2024-09-21T21:38:30.210 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:38:30.210 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction Summary 2024-09-21T21:38:30.210 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-09-21T21:38:30.210 INFO:teuthology.orchestra.run.smithi081.stdout:Install 1 Package 2024-09-21T21:38:30.210 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:38:30.210 INFO:teuthology.orchestra.run.smithi081.stdout:Total download size: 145 k 2024-09-21T21:38:30.210 INFO:teuthology.orchestra.run.smithi081.stdout:Installed size: 435 k 2024-09-21T21:38:30.210 INFO:teuthology.orchestra.run.smithi081.stdout:Downloading Packages: 2024-09-21T21:38:30.299 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:00:52 ago on Sat 21 Sep 2024 09:37:38 PM UTC. 2024-09-21T21:38:30.388 INFO:teuthology.orchestra.run.smithi164.stdout:Package ceph-base-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-21T21:38:30.419 INFO:teuthology.orchestra.run.smithi081.stdout:ceph-immutable-object-cache-19.1.1-373.g4e45e6b 696 kB/s | 145 kB 00:00 2024-09-21T21:38:30.419 INFO:teuthology.orchestra.run.smithi081.stdout:-------------------------------------------------------------------------------- 2024-09-21T21:38:30.420 INFO:teuthology.orchestra.run.smithi081.stdout:Total 691 kB/s | 145 kB 00:00 2024-09-21T21:38:30.420 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction check 2024-09-21T21:38:30.427 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction check succeeded. 2024-09-21T21:38:30.427 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction test 2024-09-21T21:38:30.430 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-21T21:38:30.431 INFO:teuthology.orchestra.run.smithi164.stdout:Nothing to do. 2024-09-21T21:38:30.431 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-21T21:38:30.484 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction test succeeded. 2024-09-21T21:38:30.484 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction 2024-09-21T21:38:30.495 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install cephadm 2024-09-21T21:38:30.570 INFO:teuthology.orchestra.run.smithi122.stdout:Last metadata expiration check: 0:00:53 ago on Sat 21 Sep 2024 09:37:37 PM UTC. 2024-09-21T21:38:30.629 INFO:teuthology.orchestra.run.smithi081.stdout: Preparing : 1/1 2024-09-21T21:38:30.695 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-373.g4e45e6b4.e 1/1 2024-09-21T21:38:30.700 INFO:teuthology.orchestra.run.smithi122.stdout:Dependencies resolved. 2024-09-21T21:38:30.701 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-21T21:38:30.701 INFO:teuthology.orchestra.run.smithi122.stdout: Package Arch Version Repository Size 2024-09-21T21:38:30.701 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-21T21:38:30.701 INFO:teuthology.orchestra.run.smithi122.stdout:Installing: 2024-09-21T21:38:30.701 INFO:teuthology.orchestra.run.smithi122.stdout: cephadm noarch 2:19.1.1-373.g4e45e6b4.el9 ceph-noarch 765 k 2024-09-21T21:38:30.701 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:38:30.701 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction Summary 2024-09-21T21:38:30.701 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-21T21:38:30.702 INFO:teuthology.orchestra.run.smithi122.stdout:Install 1 Package 2024-09-21T21:38:30.702 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:38:30.702 INFO:teuthology.orchestra.run.smithi122.stdout:Total download size: 765 k 2024-09-21T21:38:30.702 INFO:teuthology.orchestra.run.smithi122.stdout:Installed size: 772 k 2024-09-21T21:38:30.702 INFO:teuthology.orchestra.run.smithi122.stdout:Downloading Packages: 2024-09-21T21:38:30.720 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-373.g4e45e6b4.e 1/1 2024-09-21T21:38:30.720 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T21:38:30.720 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T21:38:30.720 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:38:30.997 INFO:teuthology.orchestra.run.smithi122.stdout:cephadm-19.1.1-373.g4e45e6b4.el9.noarch.rpm 2.5 MB/s | 765 kB 00:00 2024-09-21T21:38:30.997 INFO:teuthology.orchestra.run.smithi122.stdout:-------------------------------------------------------------------------------- 2024-09-21T21:38:30.997 INFO:teuthology.orchestra.run.smithi122.stdout:Total 2.5 MB/s | 765 kB 00:00 2024-09-21T21:38:30.998 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction check 2024-09-21T21:38:31.003 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction check succeeded. 2024-09-21T21:38:31.003 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction test 2024-09-21T21:38:31.008 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction test succeeded. 2024-09-21T21:38:31.009 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction 2024-09-21T21:38:31.075 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:00:53 ago on Sat 21 Sep 2024 09:37:38 PM UTC. 2024-09-21T21:38:31.081 INFO:teuthology.orchestra.run.smithi122.stdout: Preparing : 1/1 2024-09-21T21:38:31.204 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-21T21:38:31.205 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-21T21:38:31.205 INFO:teuthology.orchestra.run.smithi164.stdout: Package Arch Version Repository Size 2024-09-21T21:38:31.205 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-21T21:38:31.205 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-09-21T21:38:31.205 INFO:teuthology.orchestra.run.smithi164.stdout: cephadm noarch 2:19.1.1-373.g4e45e6b4.el9 ceph-noarch 765 k 2024-09-21T21:38:31.205 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:38:31.205 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-09-21T21:38:31.205 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-21T21:38:31.205 INFO:teuthology.orchestra.run.smithi164.stdout:Install 1 Package 2024-09-21T21:38:31.206 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:38:31.206 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 765 k 2024-09-21T21:38:31.206 INFO:teuthology.orchestra.run.smithi164.stdout:Installed size: 772 k 2024-09-21T21:38:31.206 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-09-21T21:38:31.467 INFO:teuthology.orchestra.run.smithi164.stdout:cephadm-19.1.1-373.g4e45e6b4.el9.noarch.rpm 2.9 MB/s | 765 kB 00:00 2024-09-21T21:38:31.467 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-09-21T21:38:31.467 INFO:teuthology.orchestra.run.smithi164.stdout:Total 2.9 MB/s | 765 kB 00:00 2024-09-21T21:38:31.467 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-09-21T21:38:31.472 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-09-21T21:38:31.473 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-09-21T21:38:31.478 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-09-21T21:38:31.479 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-09-21T21:38:31.550 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-09-21T21:38:31.574 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: cephadm-2:19.1.1-373.g4e45e6b4.el9.noarch 1/1 2024-09-21T21:38:31.610 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-373.g4e45e6b4.e 1/1 2024-09-21T21:38:31.610 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:38:31.610 INFO:teuthology.orchestra.run.smithi081.stdout:Installed: 2024-09-21T21:38:31.610 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-immutable-object-cache-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:38:31.610 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:38:31.610 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-09-21T21:38:31.658 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : cephadm-2:19.1.1-373.g4e45e6b4.el9.noarch 1/1 2024-09-21T21:38:31.839 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum -y install ceph-mgr 2024-09-21T21:38:31.986 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: cephadm-2:19.1.1-373.g4e45e6b4.el9.noarch 1/1 2024-09-21T21:38:32.020 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: cephadm-2:19.1.1-373.g4e45e6b4.el9.noarch 1/1 2024-09-21T21:38:32.079 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : cephadm-2:19.1.1-373.g4e45e6b4.el9.noarch 1/1 2024-09-21T21:38:32.232 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : cephadm-2:19.1.1-373.g4e45e6b4.el9.noarch 1/1 2024-09-21T21:38:32.232 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:38:32.232 INFO:teuthology.orchestra.run.smithi122.stdout:Installed: 2024-09-21T21:38:32.232 INFO:teuthology.orchestra.run.smithi122.stdout: cephadm-2:19.1.1-373.g4e45e6b4.el9.noarch 2024-09-21T21:38:32.232 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:38:32.232 INFO:teuthology.orchestra.run.smithi122.stdout:Complete! 2024-09-21T21:38:32.426 INFO:teuthology.orchestra.run.smithi081.stdout:Last metadata expiration check: 0:00:56 ago on Sat 21 Sep 2024 09:37:36 PM UTC. 2024-09-21T21:38:32.443 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum -y install ceph-immutable-object-cache 2024-09-21T21:38:32.471 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: cephadm-2:19.1.1-373.g4e45e6b4.el9.noarch 1/1 2024-09-21T21:38:32.518 INFO:teuthology.orchestra.run.smithi081.stdout:Package ceph-mgr-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-21T21:38:32.562 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-09-21T21:38:32.562 INFO:teuthology.orchestra.run.smithi081.stdout:Nothing to do. 2024-09-21T21:38:32.562 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-09-21T21:38:32.637 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum -y install ceph-mgr-dashboard 2024-09-21T21:38:32.723 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : cephadm-2:19.1.1-373.g4e45e6b4.el9.noarch 1/1 2024-09-21T21:38:32.723 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:38:32.723 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-09-21T21:38:32.723 INFO:teuthology.orchestra.run.smithi164.stdout: cephadm-2:19.1.1-373.g4e45e6b4.el9.noarch 2024-09-21T21:38:32.723 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:38:32.723 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-21T21:38:32.894 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install ceph-immutable-object-cache 2024-09-21T21:38:33.041 INFO:teuthology.orchestra.run.smithi122.stdout:Last metadata expiration check: 0:00:56 ago on Sat 21 Sep 2024 09:37:37 PM UTC. 2024-09-21T21:38:33.178 INFO:teuthology.orchestra.run.smithi122.stdout:Dependencies resolved. 2024-09-21T21:38:33.179 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-21T21:38:33.179 INFO:teuthology.orchestra.run.smithi122.stdout: Package Arch Version Repo Size 2024-09-21T21:38:33.179 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-21T21:38:33.179 INFO:teuthology.orchestra.run.smithi122.stdout:Installing: 2024-09-21T21:38:33.179 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 145 k 2024-09-21T21:38:33.179 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:38:33.179 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction Summary 2024-09-21T21:38:33.179 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-21T21:38:33.179 INFO:teuthology.orchestra.run.smithi122.stdout:Install 1 Package 2024-09-21T21:38:33.179 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:38:33.179 INFO:teuthology.orchestra.run.smithi122.stdout:Total download size: 145 k 2024-09-21T21:38:33.180 INFO:teuthology.orchestra.run.smithi122.stdout:Installed size: 435 k 2024-09-21T21:38:33.180 INFO:teuthology.orchestra.run.smithi122.stdout:Downloading Packages: 2024-09-21T21:38:33.214 INFO:teuthology.orchestra.run.smithi081.stdout:Last metadata expiration check: 0:00:57 ago on Sat 21 Sep 2024 09:37:36 PM UTC. 2024-09-21T21:38:33.345 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-09-21T21:38:33.347 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-09-21T21:38:33.347 INFO:teuthology.orchestra.run.smithi081.stdout: Package Arch Version Repository Size 2024-09-21T21:38:33.347 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-09-21T21:38:33.347 INFO:teuthology.orchestra.run.smithi081.stdout:Installing: 2024-09-21T21:38:33.347 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-dashboard noarch 2:19.1.1-373.g4e45e6b4.el9 ceph-noarch 3.6 M 2024-09-21T21:38:33.347 INFO:teuthology.orchestra.run.smithi081.stdout:Installing dependencies: 2024-09-21T21:38:33.347 INFO:teuthology.orchestra.run.smithi081.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-21T21:38:33.347 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-grafana-dashboards noarch 2:19.1.1-373.g4e45e6b4.el9 ceph-noarch 32 k 2024-09-21T21:38:33.347 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-prometheus-alerts noarch 2:19.1.1-373.g4e45e6b4.el9 ceph-noarch 16 k 2024-09-21T21:38:33.347 INFO:teuthology.orchestra.run.smithi081.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-21T21:38:33.347 INFO:teuthology.orchestra.run.smithi081.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-21T21:38:33.347 INFO:teuthology.orchestra.run.smithi081.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-21T21:38:33.347 INFO:teuthology.orchestra.run.smithi081.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-21T21:38:33.347 INFO:teuthology.orchestra.run.smithi081.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-21T21:38:33.348 INFO:teuthology.orchestra.run.smithi081.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-21T21:38:33.348 INFO:teuthology.orchestra.run.smithi081.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T21:38:33.348 INFO:teuthology.orchestra.run.smithi081.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T21:38:33.348 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:38:33.348 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction Summary 2024-09-21T21:38:33.348 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-09-21T21:38:33.348 INFO:teuthology.orchestra.run.smithi081.stdout:Install 12 Packages 2024-09-21T21:38:33.348 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:38:33.349 INFO:teuthology.orchestra.run.smithi081.stdout:Total download size: 8.7 M 2024-09-21T21:38:33.349 INFO:teuthology.orchestra.run.smithi081.stdout:Installed size: 105 M 2024-09-21T21:38:33.349 INFO:teuthology.orchestra.run.smithi081.stdout:Downloading Packages: 2024-09-21T21:38:33.386 INFO:teuthology.orchestra.run.smithi122.stdout:ceph-immutable-object-cache-19.1.1-373.g4e45e6b 704 kB/s | 145 kB 00:00 2024-09-21T21:38:33.387 INFO:teuthology.orchestra.run.smithi122.stdout:-------------------------------------------------------------------------------- 2024-09-21T21:38:33.387 INFO:teuthology.orchestra.run.smithi122.stdout:Total 698 kB/s | 145 kB 00:00 2024-09-21T21:38:33.387 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction check 2024-09-21T21:38:33.394 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction check succeeded. 2024-09-21T21:38:33.395 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction test 2024-09-21T21:38:33.451 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction test succeeded. 2024-09-21T21:38:33.451 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction 2024-09-21T21:38:33.494 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:00:55 ago on Sat 21 Sep 2024 09:37:38 PM UTC. 2024-09-21T21:38:33.593 INFO:teuthology.orchestra.run.smithi122.stdout: Preparing : 1/1 2024-09-21T21:38:33.632 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-21T21:38:33.633 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-21T21:38:33.633 INFO:teuthology.orchestra.run.smithi164.stdout: Package Arch Version Repo Size 2024-09-21T21:38:33.633 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-21T21:38:33.633 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-09-21T21:38:33.633 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 145 k 2024-09-21T21:38:33.633 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:38:33.633 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-09-21T21:38:33.633 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-21T21:38:33.633 INFO:teuthology.orchestra.run.smithi164.stdout:Install 1 Package 2024-09-21T21:38:33.633 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:38:33.634 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 145 k 2024-09-21T21:38:33.634 INFO:teuthology.orchestra.run.smithi164.stdout:Installed size: 435 k 2024-09-21T21:38:33.634 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-09-21T21:38:33.662 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-373.g4e45e6b4.e 1/1 2024-09-21T21:38:33.687 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-373.g4e45e6b4.e 1/1 2024-09-21T21:38:33.687 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T21:38:33.687 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T21:38:33.687 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:38:33.848 INFO:teuthology.orchestra.run.smithi164.stdout:ceph-immutable-object-cache-19.1.1-373.g4e45e6b 679 kB/s | 145 kB 00:00 2024-09-21T21:38:33.848 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-09-21T21:38:33.849 INFO:teuthology.orchestra.run.smithi164.stdout:Total 674 kB/s | 145 kB 00:00 2024-09-21T21:38:33.849 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-09-21T21:38:33.856 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-09-21T21:38:33.856 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-09-21T21:38:33.914 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-09-21T21:38:33.914 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-09-21T21:38:34.060 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-09-21T21:38:34.115 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-373.g4e45e6b4.e 1/1 2024-09-21T21:38:34.140 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-373.g4e45e6b4.e 1/1 2024-09-21T21:38:34.140 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T21:38:34.140 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T21:38:34.140 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:38:34.189 INFO:teuthology.orchestra.run.smithi081.stdout:(1/12): ceph-grafana-dashboards-19.1.1-373.g4e4 196 kB/s | 32 kB 00:00 2024-09-21T21:38:34.214 INFO:teuthology.orchestra.run.smithi081.stdout:(2/12): ceph-prometheus-alerts-19.1.1-373.g4e45 88 kB/s | 16 kB 00:00 2024-09-21T21:38:34.339 INFO:teuthology.orchestra.run.smithi081.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 2.1 MB/s | 269 kB 00:00 2024-09-21T21:38:34.390 INFO:teuthology.orchestra.run.smithi081.stdout:(4/12): protobuf-3.14.0-13.el9.x86_64.rpm 5.0 MB/s | 1.0 MB 00:00 2024-09-21T21:38:34.457 INFO:teuthology.orchestra.run.smithi081.stdout:(5/12): ceph-mgr-dashboard-19.1.1-373.g4e45e6b4 8.3 MB/s | 3.6 MB 00:00 2024-09-21T21:38:34.516 INFO:teuthology.orchestra.run.smithi081.stdout:(6/12): protobuf-compiler-3.14.0-13.el9.x86_64. 4.8 MB/s | 863 kB 00:00 2024-09-21T21:38:34.541 INFO:teuthology.orchestra.run.smithi081.stdout:(7/12): grpc-data-1.46.7-10.el9.noarch.rpm 238 kB/s | 19 kB 00:00 2024-09-21T21:38:34.542 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-373.g4e45e6b4.e 1/1 2024-09-21T21:38:34.543 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:38:34.543 INFO:teuthology.orchestra.run.smithi122.stdout:Installed: 2024-09-21T21:38:34.543 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-immutable-object-cache-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:38:34.543 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:38:34.543 INFO:teuthology.orchestra.run.smithi122.stdout:Complete! 2024-09-21T21:38:34.583 INFO:teuthology.orchestra.run.smithi081.stdout:(8/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 2.8 MB/s | 548 kB 00:00 2024-09-21T21:38:34.608 INFO:teuthology.orchestra.run.smithi081.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 1.3 MB/s | 31 kB 00:00 2024-09-21T21:38:34.633 INFO:teuthology.orchestra.run.smithi081.stdout:(10/12): python3-grpcio-tools-1.46.7-10.el9.x86 1.6 MB/s | 144 kB 00:00 2024-09-21T21:38:34.659 INFO:teuthology.orchestra.run.smithi081.stdout:(11/12): python3-routes-2.5.1-5.el9.noarch.rpm 3.7 MB/s | 188 kB 00:00 2024-09-21T21:38:34.759 INFO:teuthology.orchestra.run.smithi081.stdout:(12/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 8.4 MB/s | 2.0 MB 00:00 2024-09-21T21:38:34.764 INFO:teuthology.orchestra.run.smithi081.stdout:-------------------------------------------------------------------------------- 2024-09-21T21:38:34.764 INFO:teuthology.orchestra.run.smithi081.stdout:Total 6.1 MB/s | 8.7 MB 00:01 2024-09-21T21:38:34.771 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum -y install ceph-mgr 2024-09-21T21:38:34.813 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction check 2024-09-21T21:38:34.830 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction check succeeded. 2024-09-21T21:38:34.830 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction test 2024-09-21T21:38:35.050 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction test succeeded. 2024-09-21T21:38:35.051 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction 2024-09-21T21:38:35.098 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-373.g4e45e6b4.e 1/1 2024-09-21T21:38:35.099 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:38:35.099 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-09-21T21:38:35.099 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-immutable-object-cache-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:38:35.099 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:38:35.099 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-21T21:38:35.309 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install ceph-mgr 2024-09-21T21:38:35.362 INFO:teuthology.orchestra.run.smithi122.stdout:Last metadata expiration check: 0:00:58 ago on Sat 21 Sep 2024 09:37:37 PM UTC. 2024-09-21T21:38:35.452 INFO:teuthology.orchestra.run.smithi122.stdout:Package ceph-mgr-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-21T21:38:35.494 INFO:teuthology.orchestra.run.smithi122.stdout:Dependencies resolved. 2024-09-21T21:38:35.495 INFO:teuthology.orchestra.run.smithi122.stdout:Nothing to do. 2024-09-21T21:38:35.496 INFO:teuthology.orchestra.run.smithi122.stdout:Complete! 2024-09-21T21:38:35.563 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum -y install ceph-mgr-dashboard 2024-09-21T21:38:35.592 INFO:teuthology.orchestra.run.smithi081.stdout: Preparing : 1/1 2024-09-21T21:38:35.676 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-21T21:38:35.719 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-21T21:38:35.766 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-21T21:38:35.797 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-21T21:38:35.897 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:00:57 ago on Sat 21 Sep 2024 09:37:38 PM UTC. 2024-09-21T21:38:35.902 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-21T21:38:35.988 INFO:teuthology.orchestra.run.smithi164.stdout:Package ceph-mgr-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-21T21:38:36.030 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-21T21:38:36.031 INFO:teuthology.orchestra.run.smithi164.stdout:Nothing to do. 2024-09-21T21:38:36.031 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-21T21:38:36.039 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-21T21:38:36.093 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install ceph-mgr-dashboard 2024-09-21T21:38:36.132 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-21T21:38:36.142 INFO:teuthology.orchestra.run.smithi122.stdout:Last metadata expiration check: 0:00:59 ago on Sat 21 Sep 2024 09:37:37 PM UTC. 2024-09-21T21:38:36.176 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-21T21:38:36.205 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-21T21:38:36.237 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-373.g4e45e6b4.el9. 10/12 2024-09-21T21:38:36.274 INFO:teuthology.orchestra.run.smithi122.stdout:Dependencies resolved. 2024-09-21T21:38:36.275 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-21T21:38:36.275 INFO:teuthology.orchestra.run.smithi122.stdout: Package Arch Version Repository Size 2024-09-21T21:38:36.275 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-21T21:38:36.276 INFO:teuthology.orchestra.run.smithi122.stdout:Installing: 2024-09-21T21:38:36.276 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-dashboard noarch 2:19.1.1-373.g4e45e6b4.el9 ceph-noarch 3.6 M 2024-09-21T21:38:36.276 INFO:teuthology.orchestra.run.smithi122.stdout:Installing dependencies: 2024-09-21T21:38:36.276 INFO:teuthology.orchestra.run.smithi122.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-21T21:38:36.276 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-grafana-dashboards noarch 2:19.1.1-373.g4e45e6b4.el9 ceph-noarch 32 k 2024-09-21T21:38:36.276 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-prometheus-alerts noarch 2:19.1.1-373.g4e45e6b4.el9 ceph-noarch 16 k 2024-09-21T21:38:36.276 INFO:teuthology.orchestra.run.smithi122.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-21T21:38:36.276 INFO:teuthology.orchestra.run.smithi122.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-21T21:38:36.276 INFO:teuthology.orchestra.run.smithi122.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-21T21:38:36.276 INFO:teuthology.orchestra.run.smithi122.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-21T21:38:36.276 INFO:teuthology.orchestra.run.smithi122.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-21T21:38:36.276 INFO:teuthology.orchestra.run.smithi122.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-21T21:38:36.276 INFO:teuthology.orchestra.run.smithi122.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T21:38:36.277 INFO:teuthology.orchestra.run.smithi122.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T21:38:36.277 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:38:36.277 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction Summary 2024-09-21T21:38:36.277 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-21T21:38:36.277 INFO:teuthology.orchestra.run.smithi122.stdout:Install 12 Packages 2024-09-21T21:38:36.277 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:38:36.278 INFO:teuthology.orchestra.run.smithi122.stdout:Total download size: 8.7 M 2024-09-21T21:38:36.278 INFO:teuthology.orchestra.run.smithi122.stdout:Installed size: 105 M 2024-09-21T21:38:36.278 INFO:teuthology.orchestra.run.smithi122.stdout:Downloading Packages: 2024-09-21T21:38:36.677 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:00:58 ago on Sat 21 Sep 2024 09:37:38 PM UTC. 2024-09-21T21:38:36.740 INFO:teuthology.orchestra.run.smithi122.stdout:(1/12): ceph-grafana-dashboards-19.1.1-373.g4e4 228 kB/s | 32 kB 00:00 2024-09-21T21:38:36.766 INFO:teuthology.orchestra.run.smithi122.stdout:(2/12): ceph-prometheus-alerts-19.1.1-373.g4e45 100 kB/s | 16 kB 00:00 2024-09-21T21:38:36.811 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-21T21:38:36.811 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-21T21:38:36.811 INFO:teuthology.orchestra.run.smithi164.stdout: Package Arch Version Repository Size 2024-09-21T21:38:36.811 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-21T21:38:36.812 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-09-21T21:38:36.812 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-dashboard noarch 2:19.1.1-373.g4e45e6b4.el9 ceph-noarch 3.6 M 2024-09-21T21:38:36.812 INFO:teuthology.orchestra.run.smithi164.stdout:Installing dependencies: 2024-09-21T21:38:36.812 INFO:teuthology.orchestra.run.smithi164.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-21T21:38:36.812 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-grafana-dashboards noarch 2:19.1.1-373.g4e45e6b4.el9 ceph-noarch 32 k 2024-09-21T21:38:36.812 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-prometheus-alerts noarch 2:19.1.1-373.g4e45e6b4.el9 ceph-noarch 16 k 2024-09-21T21:38:36.812 INFO:teuthology.orchestra.run.smithi164.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-21T21:38:36.812 INFO:teuthology.orchestra.run.smithi164.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-21T21:38:36.812 INFO:teuthology.orchestra.run.smithi164.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-21T21:38:36.812 INFO:teuthology.orchestra.run.smithi164.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-21T21:38:36.812 INFO:teuthology.orchestra.run.smithi164.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-21T21:38:36.812 INFO:teuthology.orchestra.run.smithi164.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-21T21:38:36.813 INFO:teuthology.orchestra.run.smithi164.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T21:38:36.813 INFO:teuthology.orchestra.run.smithi164.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T21:38:36.813 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:38:36.813 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-09-21T21:38:36.813 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-21T21:38:36.813 INFO:teuthology.orchestra.run.smithi164.stdout:Install 12 Packages 2024-09-21T21:38:36.813 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:38:36.813 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 8.7 M 2024-09-21T21:38:36.813 INFO:teuthology.orchestra.run.smithi164.stdout:Installed size: 105 M 2024-09-21T21:38:36.813 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-09-21T21:38:36.991 INFO:teuthology.orchestra.run.smithi122.stdout:(3/12): ceph-mgr-dashboard-19.1.1-373.g4e45e6b4 9.1 MB/s | 3.6 MB 00:00 2024-09-21T21:38:37.168 INFO:teuthology.orchestra.run.smithi164.stdout:(1/12): ceph-prometheus-alerts-19.1.1-373.g4e45 113 kB/s | 16 kB 00:00 2024-09-21T21:38:37.193 INFO:teuthology.orchestra.run.smithi164.stdout:(2/12): ceph-grafana-dashboards-19.1.1-373.g4e4 186 kB/s | 32 kB 00:00 2024-09-21T21:38:37.266 INFO:teuthology.orchestra.run.smithi122.stdout:(4/12): protobuf-compiler-3.14.0-13.el9.x86_64. 3.1 MB/s | 863 kB 00:00 2024-09-21T21:38:37.295 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-373.g4e45e6b4.el9 11/12 2024-09-21T21:38:37.308 INFO:teuthology.orchestra.run.smithi122.stdout:(5/12): python3-protobuf-3.14.0-13.el9.noarch.r 497 kB/s | 269 kB 00:00 2024-09-21T21:38:37.329 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-373.g4e45e6b4.el9.noar 12/12 2024-09-21T21:38:37.358 INFO:teuthology.orchestra.run.smithi122.stdout:(6/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 5.8 MB/s | 548 kB 00:00 2024-09-21T21:38:37.375 INFO:teuthology.orchestra.run.smithi122.stdout:(7/12): grpc-data-1.46.7-10.el9.noarch.rpm 290 kB/s | 19 kB 00:00 2024-09-21T21:38:37.386 INFO:teuthology.orchestra.run.smithi164.stdout:(3/12): ceph-mgr-dashboard-19.1.1-373.g4e45e6b4 9.8 MB/s | 3.6 MB 00:00 2024-09-21T21:38:37.409 INFO:teuthology.orchestra.run.smithi122.stdout:(8/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 4.2 MB/s | 144 kB 00:00 2024-09-21T21:38:37.411 INFO:teuthology.orchestra.run.smithi164.stdout:(4/12): python3-protobuf-3.14.0-13.el9.noarch.r 1.2 MB/s | 269 kB 00:00 2024-09-21T21:38:37.426 INFO:teuthology.orchestra.run.smithi122.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 1.8 MB/s | 31 kB 00:00 2024-09-21T21:38:37.468 INFO:teuthology.orchestra.run.smithi122.stdout:(10/12): python3-routes-2.5.1-5.el9.noarch.rpm 4.4 MB/s | 188 kB 00:00 2024-09-21T21:38:37.486 INFO:teuthology.orchestra.run.smithi164.stdout:(5/12): protobuf-3.14.0-13.el9.x86_64.rpm 3.2 MB/s | 1.0 MB 00:00 2024-09-21T21:38:37.528 INFO:teuthology.orchestra.run.smithi164.stdout:(6/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 4.6 MB/s | 548 kB 00:00 2024-09-21T21:38:37.553 INFO:teuthology.orchestra.run.smithi164.stdout:(7/12): grpc-data-1.46.7-10.el9.noarch.rpm 291 kB/s | 19 kB 00:00 2024-09-21T21:38:37.560 INFO:teuthology.orchestra.run.smithi122.stdout:(11/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 10 MB/s | 2.0 MB 00:00 2024-09-21T21:38:37.587 INFO:teuthology.orchestra.run.smithi164.stdout:(8/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 4.2 MB/s | 144 kB 00:00 2024-09-21T21:38:37.620 INFO:teuthology.orchestra.run.smithi164.stdout:(9/12): protobuf-compiler-3.14.0-13.el9.x86_64. 3.6 MB/s | 863 kB 00:00 2024-09-21T21:38:37.646 INFO:teuthology.orchestra.run.smithi164.stdout:(10/12): python3-repoze-lru-0.7-16.el9.noarch.r 524 kB/s | 31 kB 00:00 2024-09-21T21:38:37.677 INFO:teuthology.orchestra.run.smithi122.stdout:(12/12): protobuf-3.14.0-13.el9.x86_64.rpm 1.1 MB/s | 1.0 MB 00:00 2024-09-21T21:38:37.681 INFO:teuthology.orchestra.run.smithi122.stdout:-------------------------------------------------------------------------------- 2024-09-21T21:38:37.681 INFO:teuthology.orchestra.run.smithi122.stdout:Total 6.2 MB/s | 8.7 MB 00:01 2024-09-21T21:38:37.713 INFO:teuthology.orchestra.run.smithi164.stdout:(11/12): python3-routes-2.5.1-5.el9.noarch.rpm 2.0 MB/s | 188 kB 00:00 2024-09-21T21:38:37.731 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction check 2024-09-21T21:38:37.748 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction check succeeded. 2024-09-21T21:38:37.748 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction test 2024-09-21T21:38:37.763 INFO:teuthology.orchestra.run.smithi164.stdout:(12/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 8.7 MB/s | 2.0 MB 00:00 2024-09-21T21:38:37.767 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-09-21T21:38:37.767 INFO:teuthology.orchestra.run.smithi164.stdout:Total 9.1 MB/s | 8.7 MB 00:00 2024-09-21T21:38:37.817 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-09-21T21:38:37.832 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-09-21T21:38:37.832 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-09-21T21:38:37.961 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction test succeeded. 2024-09-21T21:38:37.961 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction 2024-09-21T21:38:38.040 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-09-21T21:38:38.041 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-09-21T21:38:38.500 INFO:teuthology.orchestra.run.smithi122.stdout: Preparing : 1/1 2024-09-21T21:38:38.574 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-09-21T21:38:38.577 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-21T21:38:38.613 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-21T21:38:38.659 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-21T21:38:38.664 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-21T21:38:38.698 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-21T21:38:38.707 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-21T21:38:38.754 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-21T21:38:38.778 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-373.g4e45e6b4.el9.noar 12/12 2024-09-21T21:38:38.778 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-373.g4e45e6b4.el9 1/12 2024-09-21T21:38:38.778 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-373.g4e45e6b4.el9.noar 2/12 2024-09-21T21:38:38.778 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-373.g4e45e6b4.el9. 3/12 2024-09-21T21:38:38.778 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-21T21:38:38.779 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-21T21:38:38.779 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-21T21:38:38.779 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-21T21:38:38.779 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-21T21:38:38.779 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-21T21:38:38.779 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-21T21:38:38.779 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-21T21:38:38.793 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-21T21:38:38.810 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-21T21:38:38.898 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-21T21:38:38.942 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-21T21:38:39.027 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-21T21:38:39.033 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-21T21:38:39.078 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-21T21:38:39.098 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-21T21:38:39.099 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:38:39.099 INFO:teuthology.orchestra.run.smithi081.stdout:Installed: 2024-09-21T21:38:39.099 INFO:teuthology.orchestra.run.smithi081.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-21T21:38:39.099 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-grafana-dashboards-2:19.1.1-373.g4e45e6b4.el9.noarch 2024-09-21T21:38:39.099 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-dashboard-2:19.1.1-373.g4e45e6b4.el9.noarch 2024-09-21T21:38:39.099 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-prometheus-alerts-2:19.1.1-373.g4e45e6b4.el9.noarch 2024-09-21T21:38:39.099 INFO:teuthology.orchestra.run.smithi081.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-21T21:38:39.099 INFO:teuthology.orchestra.run.smithi081.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-21T21:38:39.099 INFO:teuthology.orchestra.run.smithi081.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-21T21:38:39.099 INFO:teuthology.orchestra.run.smithi081.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-21T21:38:39.099 INFO:teuthology.orchestra.run.smithi081.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-21T21:38:39.099 INFO:teuthology.orchestra.run.smithi081.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-21T21:38:39.099 INFO:teuthology.orchestra.run.smithi081.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T21:38:39.100 INFO:teuthology.orchestra.run.smithi081.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T21:38:39.100 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:38:39.100 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-09-21T21:38:39.106 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-21T21:38:39.112 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-21T21:38:39.140 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-373.g4e45e6b4.el9. 10/12 2024-09-21T21:38:39.164 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-21T21:38:39.193 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-21T21:38:39.226 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-373.g4e45e6b4.el9. 10/12 2024-09-21T21:38:39.351 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-21T21:38:39.937 INFO:teuthology.orchestra.run.smithi081.stdout:Last metadata expiration check: 0:01:03 ago on Sat 21 Sep 2024 09:37:36 PM UTC. 2024-09-21T21:38:40.070 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-09-21T21:38:40.071 INFO:teuthology.orchestra.run.smithi081.stdout:====================================================================================== 2024-09-21T21:38:40.071 INFO:teuthology.orchestra.run.smithi081.stdout: Package Arch Version Repository Size 2024-09-21T21:38:40.071 INFO:teuthology.orchestra.run.smithi081.stdout:====================================================================================== 2024-09-21T21:38:40.072 INFO:teuthology.orchestra.run.smithi081.stdout:Installing: 2024-09-21T21:38:40.072 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-373.g4e45e6b4.el9 ceph-noarch 7.4 M 2024-09-21T21:38:40.072 INFO:teuthology.orchestra.run.smithi081.stdout:Installing dependencies: 2024-09-21T21:38:40.072 INFO:teuthology.orchestra.run.smithi081.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T21:38:40.072 INFO:teuthology.orchestra.run.smithi081.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T21:38:40.072 INFO:teuthology.orchestra.run.smithi081.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T21:38:40.072 INFO:teuthology.orchestra.run.smithi081.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T21:38:40.072 INFO:teuthology.orchestra.run.smithi081.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T21:38:40.072 INFO:teuthology.orchestra.run.smithi081.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T21:38:40.072 INFO:teuthology.orchestra.run.smithi081.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T21:38:40.072 INFO:teuthology.orchestra.run.smithi081.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T21:38:40.072 INFO:teuthology.orchestra.run.smithi081.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T21:38:40.073 INFO:teuthology.orchestra.run.smithi081.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T21:38:40.073 INFO:teuthology.orchestra.run.smithi081.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T21:38:40.073 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:38:40.073 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction Summary 2024-09-21T21:38:40.073 INFO:teuthology.orchestra.run.smithi081.stdout:====================================================================================== 2024-09-21T21:38:40.073 INFO:teuthology.orchestra.run.smithi081.stdout:Install 12 Packages 2024-09-21T21:38:40.073 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:38:40.074 INFO:teuthology.orchestra.run.smithi081.stdout:Total download size: 38 M 2024-09-21T21:38:40.074 INFO:teuthology.orchestra.run.smithi081.stdout:Installed size: 212 M 2024-09-21T21:38:40.074 INFO:teuthology.orchestra.run.smithi081.stdout:Downloading Packages: 2024-09-21T21:38:40.192 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-373.g4e45e6b4.el9 11/12 2024-09-21T21:38:40.231 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-373.g4e45e6b4.el9.noar 12/12 2024-09-21T21:38:40.285 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-373.g4e45e6b4.el9 11/12 2024-09-21T21:38:40.317 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-373.g4e45e6b4.el9.noar 12/12 2024-09-21T21:38:40.559 INFO:teuthology.orchestra.run.smithi081.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.5 MB/s | 191 kB 00:00 2024-09-21T21:38:40.659 INFO:teuthology.orchestra.run.smithi081.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 3.5 MB/s | 801 kB 00:00 2024-09-21T21:38:40.684 INFO:teuthology.orchestra.run.smithi081.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 261 kB/s | 33 kB 00:00 2024-09-21T21:38:40.735 INFO:teuthology.orchestra.run.smithi081.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 363 kB/s | 18 kB 00:00 2024-09-21T21:38:40.818 INFO:teuthology.orchestra.run.smithi081.stdout:(5/12): openblas-0.3.26-2.el9.x86_64.rpm 463 kB/s | 38 kB 00:00 2024-09-21T21:38:40.910 INFO:teuthology.orchestra.run.smithi081.stdout:(6/12): ceph-mgr-diskprediction-local-19.1.1-37 16 MB/s | 7.4 MB 00:00 2024-09-21T21:38:41.119 INFO:teuthology.orchestra.run.smithi081.stdout:(7/12): python3-devel-3.9.19-8.el9.x86_64.rpm 1.1 MB/s | 245 kB 00:00 2024-09-21T21:38:41.590 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-373.g4e45e6b4.el9.noar 12/12 2024-09-21T21:38:41.590 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-373.g4e45e6b4.el9 1/12 2024-09-21T21:38:41.590 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-373.g4e45e6b4.el9.noar 2/12 2024-09-21T21:38:41.590 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-373.g4e45e6b4.el9. 3/12 2024-09-21T21:38:41.590 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-21T21:38:41.591 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-21T21:38:41.591 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-21T21:38:41.591 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-21T21:38:41.591 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-21T21:38:41.591 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-21T21:38:41.591 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-21T21:38:41.591 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-21T21:38:41.605 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-373.g4e45e6b4.el9.noar 12/12 2024-09-21T21:38:41.605 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-373.g4e45e6b4.el9 1/12 2024-09-21T21:38:41.606 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-373.g4e45e6b4.el9.noar 2/12 2024-09-21T21:38:41.606 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-373.g4e45e6b4.el9. 3/12 2024-09-21T21:38:41.606 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-21T21:38:41.606 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-21T21:38:41.606 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-21T21:38:41.606 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-21T21:38:41.606 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-21T21:38:41.606 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-21T21:38:41.606 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-21T21:38:41.606 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-21T21:38:41.761 INFO:teuthology.orchestra.run.smithi081.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 2.7 MB/s | 3.0 MB 00:01 2024-09-21T21:38:41.902 INFO:teuthology.orchestra.run.smithi081.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 1.7 MB/s | 250 kB 00:00 2024-09-21T21:38:41.913 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-21T21:38:41.913 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:38:41.914 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-09-21T21:38:41.914 INFO:teuthology.orchestra.run.smithi164.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-21T21:38:41.914 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-grafana-dashboards-2:19.1.1-373.g4e45e6b4.el9.noarch 2024-09-21T21:38:41.914 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-dashboard-2:19.1.1-373.g4e45e6b4.el9.noarch 2024-09-21T21:38:41.914 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-prometheus-alerts-2:19.1.1-373.g4e45e6b4.el9.noarch 2024-09-21T21:38:41.914 INFO:teuthology.orchestra.run.smithi164.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-21T21:38:41.914 INFO:teuthology.orchestra.run.smithi164.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-21T21:38:41.915 INFO:teuthology.orchestra.run.smithi164.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-21T21:38:41.915 INFO:teuthology.orchestra.run.smithi164.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-21T21:38:41.915 INFO:teuthology.orchestra.run.smithi164.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-21T21:38:41.915 INFO:teuthology.orchestra.run.smithi164.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-21T21:38:41.915 INFO:teuthology.orchestra.run.smithi164.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T21:38:41.915 INFO:teuthology.orchestra.run.smithi164.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T21:38:41.915 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:38:41.915 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-21T21:38:41.916 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-21T21:38:41.916 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:38:41.916 INFO:teuthology.orchestra.run.smithi122.stdout:Installed: 2024-09-21T21:38:41.917 INFO:teuthology.orchestra.run.smithi122.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-21T21:38:41.917 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-grafana-dashboards-2:19.1.1-373.g4e45e6b4.el9.noarch 2024-09-21T21:38:41.917 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-dashboard-2:19.1.1-373.g4e45e6b4.el9.noarch 2024-09-21T21:38:41.917 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-prometheus-alerts-2:19.1.1-373.g4e45e6b4.el9.noarch 2024-09-21T21:38:41.917 INFO:teuthology.orchestra.run.smithi122.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-21T21:38:41.917 INFO:teuthology.orchestra.run.smithi122.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-21T21:38:41.917 INFO:teuthology.orchestra.run.smithi122.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-21T21:38:41.917 INFO:teuthology.orchestra.run.smithi122.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-21T21:38:41.917 INFO:teuthology.orchestra.run.smithi122.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-21T21:38:41.917 INFO:teuthology.orchestra.run.smithi122.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-21T21:38:41.917 INFO:teuthology.orchestra.run.smithi122.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T21:38:41.917 INFO:teuthology.orchestra.run.smithi122.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T21:38:41.917 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:38:41.917 INFO:teuthology.orchestra.run.smithi122.stdout:Complete! 2024-09-21T21:38:42.152 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-21T21:38:42.155 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-21T21:38:42.636 INFO:teuthology.orchestra.run.smithi081.stdout:(10/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 2.7 MB/s | 4.9 MB 00:01 2024-09-21T21:38:42.735 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:01:04 ago on Sat 21 Sep 2024 09:37:38 PM UTC. 2024-09-21T21:38:42.736 INFO:teuthology.orchestra.run.smithi122.stdout:Last metadata expiration check: 0:01:05 ago on Sat 21 Sep 2024 09:37:37 PM UTC. 2024-09-21T21:38:42.869 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-21T21:38:42.869 INFO:teuthology.orchestra.run.smithi122.stdout:Dependencies resolved. 2024-09-21T21:38:42.870 INFO:teuthology.orchestra.run.smithi164.stdout:====================================================================================== 2024-09-21T21:38:42.870 INFO:teuthology.orchestra.run.smithi164.stdout: Package Arch Version Repository Size 2024-09-21T21:38:42.870 INFO:teuthology.orchestra.run.smithi164.stdout:====================================================================================== 2024-09-21T21:38:42.870 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-09-21T21:38:42.870 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-373.g4e45e6b4.el9 ceph-noarch 7.4 M 2024-09-21T21:38:42.870 INFO:teuthology.orchestra.run.smithi164.stdout:Installing dependencies: 2024-09-21T21:38:42.870 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T21:38:42.871 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T21:38:42.871 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T21:38:42.871 INFO:teuthology.orchestra.run.smithi164.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T21:38:42.871 INFO:teuthology.orchestra.run.smithi164.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T21:38:42.871 INFO:teuthology.orchestra.run.smithi164.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T21:38:42.871 INFO:teuthology.orchestra.run.smithi164.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T21:38:42.871 INFO:teuthology.orchestra.run.smithi164.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T21:38:42.871 INFO:teuthology.orchestra.run.smithi164.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T21:38:42.871 INFO:teuthology.orchestra.run.smithi164.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T21:38:42.871 INFO:teuthology.orchestra.run.smithi164.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T21:38:42.871 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:38:42.871 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-09-21T21:38:42.872 INFO:teuthology.orchestra.run.smithi164.stdout:====================================================================================== 2024-09-21T21:38:42.872 INFO:teuthology.orchestra.run.smithi164.stdout:Install 12 Packages 2024-09-21T21:38:42.872 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:38:42.873 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 38 M 2024-09-21T21:38:42.873 INFO:teuthology.orchestra.run.smithi164.stdout:Installed size: 212 M 2024-09-21T21:38:42.873 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-09-21T21:38:42.873 INFO:teuthology.orchestra.run.smithi122.stdout:====================================================================================== 2024-09-21T21:38:42.873 INFO:teuthology.orchestra.run.smithi122.stdout: Package Arch Version Repository Size 2024-09-21T21:38:42.873 INFO:teuthology.orchestra.run.smithi122.stdout:====================================================================================== 2024-09-21T21:38:42.873 INFO:teuthology.orchestra.run.smithi122.stdout:Installing: 2024-09-21T21:38:42.873 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-373.g4e45e6b4.el9 ceph-noarch 7.4 M 2024-09-21T21:38:42.873 INFO:teuthology.orchestra.run.smithi122.stdout:Installing dependencies: 2024-09-21T21:38:42.873 INFO:teuthology.orchestra.run.smithi122.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T21:38:42.873 INFO:teuthology.orchestra.run.smithi122.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T21:38:42.874 INFO:teuthology.orchestra.run.smithi122.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T21:38:42.874 INFO:teuthology.orchestra.run.smithi122.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T21:38:42.874 INFO:teuthology.orchestra.run.smithi122.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T21:38:42.874 INFO:teuthology.orchestra.run.smithi122.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T21:38:42.874 INFO:teuthology.orchestra.run.smithi122.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T21:38:42.874 INFO:teuthology.orchestra.run.smithi122.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T21:38:42.874 INFO:teuthology.orchestra.run.smithi122.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T21:38:42.874 INFO:teuthology.orchestra.run.smithi122.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T21:38:42.874 INFO:teuthology.orchestra.run.smithi122.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T21:38:42.874 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:38:42.874 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction Summary 2024-09-21T21:38:42.874 INFO:teuthology.orchestra.run.smithi122.stdout:====================================================================================== 2024-09-21T21:38:42.874 INFO:teuthology.orchestra.run.smithi122.stdout:Install 12 Packages 2024-09-21T21:38:42.875 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:38:42.875 INFO:teuthology.orchestra.run.smithi122.stdout:Total download size: 38 M 2024-09-21T21:38:42.875 INFO:teuthology.orchestra.run.smithi122.stdout:Installed size: 212 M 2024-09-21T21:38:42.875 INFO:teuthology.orchestra.run.smithi122.stdout:Downloading Packages: 2024-09-21T21:38:42.903 INFO:teuthology.orchestra.run.smithi081.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 2.9 MB/s | 5.1 MB 00:01 2024-09-21T21:38:43.172 INFO:teuthology.orchestra.run.smithi164.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 915 kB/s | 191 kB 00:00 2024-09-21T21:38:43.238 INFO:teuthology.orchestra.run.smithi164.stdout:(2/12): flexiblas-3.0.4-8.el9.x86_64.rpm 492 kB/s | 33 kB 00:00 2024-09-21T21:38:43.271 INFO:teuthology.orchestra.run.smithi164.stdout:(3/12): libgfortran-11.5.0-2.el9.x86_64.rpm 2.5 MB/s | 801 kB 00:00 2024-09-21T21:38:43.347 INFO:teuthology.orchestra.run.smithi164.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 242 kB/s | 18 kB 00:00 2024-09-21T21:38:43.393 INFO:teuthology.orchestra.run.smithi122.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 516 kB/s | 191 kB 00:00 2024-09-21T21:38:43.422 INFO:teuthology.orchestra.run.smithi164.stdout:(5/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 16 MB/s | 3.0 MB 00:00 2024-09-21T21:38:43.447 INFO:teuthology.orchestra.run.smithi164.stdout:(6/12): openblas-0.3.26-2.el9.x86_64.rpm 384 kB/s | 38 kB 00:00 2024-09-21T21:38:43.540 INFO:teuthology.orchestra.run.smithi164.stdout:(7/12): ceph-mgr-diskprediction-local-19.1.1-37 13 MB/s | 7.4 MB 00:00 2024-09-21T21:38:43.560 INFO:teuthology.orchestra.run.smithi122.stdout:(2/12): ceph-mgr-diskprediction-local-19.1.1-37 14 MB/s | 7.4 MB 00:00 2024-09-21T21:38:43.585 INFO:teuthology.orchestra.run.smithi122.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 170 kB/s | 33 kB 00:00 2024-09-21T21:38:43.623 INFO:teuthology.orchestra.run.smithi164.stdout:(8/12): openblas-openmp-0.3.26-2.el9.x86_64.rpm 25 MB/s | 4.9 MB 00:00 2024-09-21T21:38:43.650 INFO:teuthology.orchestra.run.smithi164.stdout:(9/12): python3-devel-3.9.19-8.el9.x86_64.rpm 1.2 MB/s | 245 kB 00:00 2024-09-21T21:38:43.652 INFO:teuthology.orchestra.run.smithi122.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 274 kB/s | 18 kB 00:00 2024-09-21T21:38:43.674 INFO:teuthology.orchestra.run.smithi164.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 4.9 MB/s | 250 kB 00:00 2024-09-21T21:38:43.727 INFO:teuthology.orchestra.run.smithi122.stdout:(5/12): openblas-0.3.26-2.el9.x86_64.rpm 513 kB/s | 38 kB 00:00 2024-09-21T21:38:43.786 INFO:teuthology.orchestra.run.smithi122.stdout:(6/12): libgfortran-11.5.0-2.el9.x86_64.rpm 1.0 MB/s | 801 kB 00:00 2024-09-21T21:38:43.849 INFO:teuthology.orchestra.run.smithi164.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 17 MB/s | 5.1 MB 00:00 2024-09-21T21:38:43.962 INFO:teuthology.orchestra.run.smithi122.stdout:(7/12): python3-devel-3.9.19-8.el9.x86_64.rpm 1.4 MB/s | 245 kB 00:00 2024-09-21T21:38:44.699 INFO:teuthology.orchestra.run.smithi164.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 15 MB/s | 16 MB 00:01 2024-09-21T21:38:44.700 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-09-21T21:38:44.700 INFO:teuthology.orchestra.run.smithi164.stdout:Total 21 MB/s | 38 MB 00:01 2024-09-21T21:38:45.055 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-09-21T21:38:45.081 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-09-21T21:38:45.082 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-09-21T21:38:45.479 INFO:teuthology.orchestra.run.smithi122.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 1.6 MB/s | 3.0 MB 00:01 2024-09-21T21:38:45.516 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-09-21T21:38:45.517 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-09-21T21:38:45.687 INFO:teuthology.orchestra.run.smithi122.stdout:(9/12): openblas-openmp-0.3.26-2.el9.x86_64.rpm 2.5 MB/s | 4.9 MB 00:01 2024-09-21T21:38:45.729 INFO:teuthology.orchestra.run.smithi122.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 1.0 MB/s | 250 kB 00:00 2024-09-21T21:38:45.879 INFO:teuthology.orchestra.run.smithi122.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 2.7 MB/s | 5.1 MB 00:01 2024-09-21T21:38:46.292 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-09-21T21:38:46.360 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-21T21:38:46.405 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-21T21:38:46.514 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-21T21:38:46.565 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-21T21:38:46.750 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-21T21:38:46.781 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-21T21:38:46.845 INFO:teuthology.orchestra.run.smithi081.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 3.3 MB/s | 16 MB 00:04 2024-09-21T21:38:46.846 INFO:teuthology.orchestra.run.smithi081.stdout:-------------------------------------------------------------------------------- 2024-09-21T21:38:46.846 INFO:teuthology.orchestra.run.smithi081.stdout:Total 5.6 MB/s | 38 MB 00:06 2024-09-21T21:38:46.857 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-21T21:38:47.206 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction check 2024-09-21T21:38:47.234 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction check succeeded. 2024-09-21T21:38:47.234 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction test 2024-09-21T21:38:47.525 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-21T21:38:47.645 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-21T21:38:47.669 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction test succeeded. 2024-09-21T21:38:47.669 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction 2024-09-21T21:38:48.427 INFO:teuthology.orchestra.run.smithi081.stdout: Preparing : 1/1 2024-09-21T21:38:48.492 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-21T21:38:48.534 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-21T21:38:48.633 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-21T21:38:48.669 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-21T21:38:48.853 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-21T21:38:48.884 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-21T21:38:48.895 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-21T21:38:48.969 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-21T21:38:49.512 INFO:teuthology.orchestra.run.smithi122.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 4.2 MB/s | 16 MB 00:03 2024-09-21T21:38:49.513 INFO:teuthology.orchestra.run.smithi122.stdout:-------------------------------------------------------------------------------- 2024-09-21T21:38:49.514 INFO:teuthology.orchestra.run.smithi122.stdout:Total 5.8 MB/s | 38 MB 00:06 2024-09-21T21:38:49.626 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-21T21:38:49.740 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-21T21:38:49.869 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction check 2024-09-21T21:38:49.896 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction check succeeded. 2024-09-21T21:38:49.896 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction test 2024-09-21T21:38:49.963 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-21T21:38:49.989 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-373.g4e45e6 12/12 2024-09-21T21:38:50.327 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction test succeeded. 2024-09-21T21:38:50.327 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction 2024-09-21T21:38:51.002 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-21T21:38:51.118 INFO:teuthology.orchestra.run.smithi122.stdout: Preparing : 1/1 2024-09-21T21:38:51.191 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-21T21:38:51.235 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-21T21:38:51.342 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-21T21:38:51.386 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-21T21:38:51.571 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-21T21:38:51.602 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-21T21:38:51.679 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-21T21:38:52.037 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-21T21:38:52.067 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-373.g4e45e6 12/12 2024-09-21T21:38:52.324 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-21T21:38:52.448 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-21T21:38:52.553 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-373.g4e45e6 12/12 2024-09-21T21:38:52.553 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-373.g4e45e6 1/12 2024-09-21T21:38:52.553 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-21T21:38:52.553 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-21T21:38:52.553 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-21T21:38:52.553 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-21T21:38:52.553 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-21T21:38:52.553 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-21T21:38:52.553 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-21T21:38:52.553 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-21T21:38:52.554 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-21T21:38:52.554 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-21T21:38:52.964 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-21T21:38:52.964 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:38:52.964 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-09-21T21:38:52.964 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-diskprediction-local-2:19.1.1-373.g4e45e6b4.el9.noarch 2024-09-21T21:38:52.964 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T21:38:52.964 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T21:38:52.964 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T21:38:52.965 INFO:teuthology.orchestra.run.smithi164.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T21:38:52.965 INFO:teuthology.orchestra.run.smithi164.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T21:38:52.965 INFO:teuthology.orchestra.run.smithi164.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T21:38:52.965 INFO:teuthology.orchestra.run.smithi164.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T21:38:52.965 INFO:teuthology.orchestra.run.smithi164.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T21:38:52.965 INFO:teuthology.orchestra.run.smithi164.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T21:38:52.965 INFO:teuthology.orchestra.run.smithi164.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T21:38:52.965 INFO:teuthology.orchestra.run.smithi164.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T21:38:52.965 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:38:52.965 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-21T21:38:53.401 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install ceph-mgr-rook 2024-09-21T21:38:53.678 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-21T21:38:53.984 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:01:15 ago on Sat 21 Sep 2024 09:37:38 PM UTC. 2024-09-21T21:38:54.114 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-21T21:38:54.116 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-21T21:38:54.116 INFO:teuthology.orchestra.run.smithi164.stdout: Package Arch Version Repository Size 2024-09-21T21:38:54.116 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-21T21:38:54.116 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-09-21T21:38:54.116 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-rook noarch 2:19.1.1-373.g4e45e6b4.el9 ceph-noarch 49 k 2024-09-21T21:38:54.116 INFO:teuthology.orchestra.run.smithi164.stdout:Installing dependencies: 2024-09-21T21:38:54.116 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T21:38:54.116 INFO:teuthology.orchestra.run.smithi164.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T21:38:54.116 INFO:teuthology.orchestra.run.smithi164.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T21:38:54.116 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T21:38:54.116 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T21:38:54.116 INFO:teuthology.orchestra.run.smithi164.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T21:38:54.117 INFO:teuthology.orchestra.run.smithi164.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T21:38:54.117 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T21:38:54.117 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T21:38:54.117 INFO:teuthology.orchestra.run.smithi164.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T21:38:54.117 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T21:38:54.117 INFO:teuthology.orchestra.run.smithi164.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T21:38:54.117 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:38:54.117 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-09-21T21:38:54.117 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-21T21:38:54.117 INFO:teuthology.orchestra.run.smithi164.stdout:Install 13 Packages 2024-09-21T21:38:54.117 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:38:54.118 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 2.2 M 2024-09-21T21:38:54.118 INFO:teuthology.orchestra.run.smithi164.stdout:Installed size: 26 M 2024-09-21T21:38:54.118 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-09-21T21:38:54.341 INFO:teuthology.orchestra.run.smithi164.stdout:(1/13): ceph-mgr-rook-19.1.1-373.g4e45e6b4.el9. 294 kB/s | 49 kB 00:00 2024-09-21T21:38:54.466 INFO:teuthology.orchestra.run.smithi164.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 65 kB/s | 19 kB 00:00 2024-09-21T21:38:54.491 INFO:teuthology.orchestra.run.smithi164.stdout:(3/13): python3-jsonpatch-1.21-16.el9.noarch.rp 81 kB/s | 26 kB 00:00 2024-09-21T21:38:54.675 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-373.g4e45e6 12/12 2024-09-21T21:38:54.676 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-373.g4e45e6 1/12 2024-09-21T21:38:54.676 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-21T21:38:54.676 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-21T21:38:54.676 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-21T21:38:54.676 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-21T21:38:54.676 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-21T21:38:54.677 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-21T21:38:54.677 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-21T21:38:54.677 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-21T21:38:54.677 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-21T21:38:54.677 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-21T21:38:54.724 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-21T21:38:54.760 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-373.g4e45e6 12/12 2024-09-21T21:38:54.850 INFO:teuthology.orchestra.run.smithi164.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 414 kB/s | 159 kB 00:00 2024-09-21T21:38:54.933 INFO:teuthology.orchestra.run.smithi164.stdout:(5/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 375 kB/s | 222 kB 00:00 2024-09-21T21:38:54.984 INFO:teuthology.orchestra.run.smithi164.stdout:(6/13): python3-cachetools-4.2.4-1.el9.noarch.r 643 kB/s | 32 kB 00:00 2024-09-21T21:38:55.009 INFO:teuthology.orchestra.run.smithi164.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 337 kB/s | 54 kB 00:00 2024-09-21T21:38:55.034 INFO:teuthology.orchestra.run.smithi164.stdout:(8/13): python3-pyasn1-modules-0.4.8-6.el9.noar 514 kB/s | 279 kB 00:00 2024-09-21T21:38:55.059 INFO:teuthology.orchestra.run.smithi164.stdout:(9/13): python3-certifi-2023.05.07-4.el9.noarch 187 kB/s | 14 kB 00:00 2024-09-21T21:38:55.115 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-21T21:38:55.115 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:38:55.115 INFO:teuthology.orchestra.run.smithi081.stdout:Installed: 2024-09-21T21:38:55.115 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-diskprediction-local-2:19.1.1-373.g4e45e6b4.el9.noarch 2024-09-21T21:38:55.115 INFO:teuthology.orchestra.run.smithi081.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T21:38:55.115 INFO:teuthology.orchestra.run.smithi081.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T21:38:55.115 INFO:teuthology.orchestra.run.smithi081.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T21:38:55.116 INFO:teuthology.orchestra.run.smithi081.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T21:38:55.116 INFO:teuthology.orchestra.run.smithi081.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T21:38:55.116 INFO:teuthology.orchestra.run.smithi081.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T21:38:55.116 INFO:teuthology.orchestra.run.smithi081.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T21:38:55.116 INFO:teuthology.orchestra.run.smithi081.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T21:38:55.116 INFO:teuthology.orchestra.run.smithi081.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T21:38:55.116 INFO:teuthology.orchestra.run.smithi081.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T21:38:55.116 INFO:teuthology.orchestra.run.smithi081.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T21:38:55.116 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:38:55.116 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-09-21T21:38:55.118 INFO:teuthology.orchestra.run.smithi164.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 1.1 MB/s | 59 kB 00:00 2024-09-21T21:38:55.143 INFO:teuthology.orchestra.run.smithi164.stdout:(11/13): python3-google-auth-2.34.0-1.el9.noarc 1.6 MB/s | 220 kB 00:00 2024-09-21T21:38:55.168 INFO:teuthology.orchestra.run.smithi164.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.8 MB/s | 90 kB 00:00 2024-09-21T21:38:55.227 INFO:teuthology.orchestra.run.smithi164.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 6.4 MB/s | 1.0 MB 00:00 2024-09-21T21:38:55.230 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-09-21T21:38:55.230 INFO:teuthology.orchestra.run.smithi164.stdout:Total 2.0 MB/s | 2.2 MB 00:01 2024-09-21T21:38:55.293 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-09-21T21:38:55.308 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-09-21T21:38:55.308 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-09-21T21:38:55.400 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-09-21T21:38:55.400 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-09-21T21:38:55.493 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum -y install ceph-mgr-rook 2024-09-21T21:38:55.904 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-09-21T21:38:56.065 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-21T21:38:56.082 INFO:teuthology.orchestra.run.smithi081.stdout:Last metadata expiration check: 0:01:20 ago on Sat 21 Sep 2024 09:37:36 PM UTC. 2024-09-21T21:38:56.136 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-21T21:38:56.192 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-21T21:38:56.213 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-09-21T21:38:56.215 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-09-21T21:38:56.215 INFO:teuthology.orchestra.run.smithi081.stdout: Package Arch Version Repository Size 2024-09-21T21:38:56.215 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-09-21T21:38:56.215 INFO:teuthology.orchestra.run.smithi081.stdout:Installing: 2024-09-21T21:38:56.215 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-rook noarch 2:19.1.1-373.g4e45e6b4.el9 ceph-noarch 49 k 2024-09-21T21:38:56.215 INFO:teuthology.orchestra.run.smithi081.stdout:Installing dependencies: 2024-09-21T21:38:56.215 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T21:38:56.215 INFO:teuthology.orchestra.run.smithi081.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T21:38:56.215 INFO:teuthology.orchestra.run.smithi081.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T21:38:56.215 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T21:38:56.215 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T21:38:56.216 INFO:teuthology.orchestra.run.smithi081.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T21:38:56.216 INFO:teuthology.orchestra.run.smithi081.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T21:38:56.216 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T21:38:56.216 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T21:38:56.216 INFO:teuthology.orchestra.run.smithi081.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T21:38:56.216 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T21:38:56.216 INFO:teuthology.orchestra.run.smithi081.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T21:38:56.216 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:38:56.216 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction Summary 2024-09-21T21:38:56.216 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-09-21T21:38:56.216 INFO:teuthology.orchestra.run.smithi081.stdout:Install 13 Packages 2024-09-21T21:38:56.216 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:38:56.217 INFO:teuthology.orchestra.run.smithi081.stdout:Total download size: 2.2 M 2024-09-21T21:38:56.217 INFO:teuthology.orchestra.run.smithi081.stdout:Installed size: 26 M 2024-09-21T21:38:56.217 INFO:teuthology.orchestra.run.smithi081.stdout:Downloading Packages: 2024-09-21T21:38:56.229 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-21T21:38:56.271 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-21T21:38:56.389 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-21T21:38:56.445 INFO:teuthology.orchestra.run.smithi081.stdout:(1/13): ceph-mgr-rook-19.1.1-373.g4e45e6b4.el9. 296 kB/s | 49 kB 00:00 2024-09-21T21:38:56.499 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-21T21:38:56.547 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-21T21:38:56.570 INFO:teuthology.orchestra.run.smithi081.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 66 kB/s | 19 kB 00:00 2024-09-21T21:38:56.595 INFO:teuthology.orchestra.run.smithi081.stdout:(3/13): python3-jsonpatch-1.21-16.el9.noarch.rp 81 kB/s | 26 kB 00:00 2024-09-21T21:38:56.953 INFO:teuthology.orchestra.run.smithi081.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 414 kB/s | 159 kB 00:00 2024-09-21T21:38:56.996 INFO:teuthology.orchestra.run.smithi081.stdout:(5/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 403 kB/s | 222 kB 00:00 2024-09-21T21:38:57.046 INFO:teuthology.orchestra.run.smithi081.stdout:(6/13): python3-cachetools-4.2.4-1.el9.noarch.r 645 kB/s | 32 kB 00:00 2024-09-21T21:38:57.071 INFO:teuthology.orchestra.run.smithi081.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 457 kB/s | 54 kB 00:00 2024-09-21T21:38:57.096 INFO:teuthology.orchestra.run.smithi081.stdout:(8/13): python3-pyasn1-modules-0.4.8-6.el9.noar 556 kB/s | 279 kB 00:00 2024-09-21T21:38:57.122 INFO:teuthology.orchestra.run.smithi081.stdout:(9/13): python3-certifi-2023.05.07-4.el9.noarch 187 kB/s | 14 kB 00:00 2024-09-21T21:38:57.174 INFO:teuthology.orchestra.run.smithi081.stdout:(10/13): python3-google-auth-2.34.0-1.el9.noarc 2.1 MB/s | 220 kB 00:00 2024-09-21T21:38:57.197 INFO:teuthology.orchestra.run.smithi081.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 866 kB/s | 59 kB 00:00 2024-09-21T21:38:57.231 INFO:teuthology.orchestra.run.smithi081.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.5 MB/s | 90 kB 00:00 2024-09-21T21:38:57.262 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-21T21:38:57.273 INFO:teuthology.orchestra.run.smithi081.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 7.1 MB/s | 1.0 MB 00:00 2024-09-21T21:38:57.276 INFO:teuthology.orchestra.run.smithi081.stdout:-------------------------------------------------------------------------------- 2024-09-21T21:38:57.276 INFO:teuthology.orchestra.run.smithi081.stdout:Total 2.1 MB/s | 2.2 MB 00:01 2024-09-21T21:38:57.320 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-21T21:38:57.343 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction check 2024-09-21T21:38:57.354 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-21T21:38:57.358 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction check succeeded. 2024-09-21T21:38:57.358 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction test 2024-09-21T21:38:57.360 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-373.g4e45e6 12/12 2024-09-21T21:38:57.360 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-373.g4e45e6 1/12 2024-09-21T21:38:57.360 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-21T21:38:57.360 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-21T21:38:57.360 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-21T21:38:57.360 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-21T21:38:57.360 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-21T21:38:57.360 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-21T21:38:57.360 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-21T21:38:57.360 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-21T21:38:57.360 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-21T21:38:57.360 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-21T21:38:57.402 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-21T21:38:57.424 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-rook-2:19.1.1-373.g4e45e6b4.el9.noarch 13/13 2024-09-21T21:38:57.451 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction test succeeded. 2024-09-21T21:38:57.452 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction 2024-09-21T21:38:57.790 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-21T21:38:57.790 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:38:57.790 INFO:teuthology.orchestra.run.smithi122.stdout:Installed: 2024-09-21T21:38:57.790 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-diskprediction-local-2:19.1.1-373.g4e45e6b4.el9.noarch 2024-09-21T21:38:57.790 INFO:teuthology.orchestra.run.smithi122.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T21:38:57.790 INFO:teuthology.orchestra.run.smithi122.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T21:38:57.790 INFO:teuthology.orchestra.run.smithi122.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T21:38:57.790 INFO:teuthology.orchestra.run.smithi122.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T21:38:57.791 INFO:teuthology.orchestra.run.smithi122.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T21:38:57.791 INFO:teuthology.orchestra.run.smithi122.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T21:38:57.791 INFO:teuthology.orchestra.run.smithi122.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T21:38:57.791 INFO:teuthology.orchestra.run.smithi122.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T21:38:57.791 INFO:teuthology.orchestra.run.smithi122.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T21:38:57.791 INFO:teuthology.orchestra.run.smithi122.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T21:38:57.791 INFO:teuthology.orchestra.run.smithi122.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T21:38:57.791 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:38:57.791 INFO:teuthology.orchestra.run.smithi122.stdout:Complete! 2024-09-21T21:38:57.958 INFO:teuthology.orchestra.run.smithi081.stdout: Preparing : 1/1 2024-09-21T21:38:58.129 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-21T21:38:58.182 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-21T21:38:58.208 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum -y install ceph-mgr-rook 2024-09-21T21:38:58.214 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-373.g4e45e6b4.el9.noarch 13/13 2024-09-21T21:38:58.214 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-rook-2:19.1.1-373.g4e45e6b4.el9.noarch 1/13 2024-09-21T21:38:58.214 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-21T21:38:58.214 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-21T21:38:58.214 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-21T21:38:58.214 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-21T21:38:58.214 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-21T21:38:58.214 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-21T21:38:58.214 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-21T21:38:58.214 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-21T21:38:58.215 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-21T21:38:58.215 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-21T21:38:58.215 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-21T21:38:58.238 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-21T21:38:58.275 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-21T21:38:58.317 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-21T21:38:58.428 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-21T21:38:58.528 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-21T21:38:58.543 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-21T21:38:58.543 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:38:58.543 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-09-21T21:38:58.543 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-rook-2:19.1.1-373.g4e45e6b4.el9.noarch 2024-09-21T21:38:58.543 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T21:38:58.543 INFO:teuthology.orchestra.run.smithi164.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T21:38:58.543 INFO:teuthology.orchestra.run.smithi164.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T21:38:58.543 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T21:38:58.544 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T21:38:58.544 INFO:teuthology.orchestra.run.smithi164.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T21:38:58.544 INFO:teuthology.orchestra.run.smithi164.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T21:38:58.544 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T21:38:58.544 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T21:38:58.544 INFO:teuthology.orchestra.run.smithi164.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T21:38:58.544 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T21:38:58.544 INFO:teuthology.orchestra.run.smithi164.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T21:38:58.544 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:38:58.544 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-21T21:38:58.585 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-21T21:38:58.804 INFO:teuthology.orchestra.run.smithi122.stdout:Last metadata expiration check: 0:01:21 ago on Sat 21 Sep 2024 09:37:37 PM UTC. 2024-09-21T21:38:58.809 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install ceph-mgr-cephadm 2024-09-21T21:38:58.935 INFO:teuthology.orchestra.run.smithi122.stdout:Dependencies resolved. 2024-09-21T21:38:58.937 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-21T21:38:58.937 INFO:teuthology.orchestra.run.smithi122.stdout: Package Arch Version Repository Size 2024-09-21T21:38:58.937 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-21T21:38:58.937 INFO:teuthology.orchestra.run.smithi122.stdout:Installing: 2024-09-21T21:38:58.937 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-rook noarch 2:19.1.1-373.g4e45e6b4.el9 ceph-noarch 49 k 2024-09-21T21:38:58.937 INFO:teuthology.orchestra.run.smithi122.stdout:Installing dependencies: 2024-09-21T21:38:58.937 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T21:38:58.937 INFO:teuthology.orchestra.run.smithi122.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T21:38:58.937 INFO:teuthology.orchestra.run.smithi122.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T21:38:58.937 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T21:38:58.937 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T21:38:58.937 INFO:teuthology.orchestra.run.smithi122.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T21:38:58.937 INFO:teuthology.orchestra.run.smithi122.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T21:38:58.938 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T21:38:58.938 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T21:38:58.938 INFO:teuthology.orchestra.run.smithi122.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T21:38:58.938 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T21:38:58.938 INFO:teuthology.orchestra.run.smithi122.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T21:38:58.938 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:38:58.938 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction Summary 2024-09-21T21:38:58.938 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-21T21:38:58.938 INFO:teuthology.orchestra.run.smithi122.stdout:Install 13 Packages 2024-09-21T21:38:58.938 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:38:58.939 INFO:teuthology.orchestra.run.smithi122.stdout:Total download size: 2.2 M 2024-09-21T21:38:58.939 INFO:teuthology.orchestra.run.smithi122.stdout:Installed size: 26 M 2024-09-21T21:38:58.939 INFO:teuthology.orchestra.run.smithi122.stdout:Downloading Packages: 2024-09-21T21:38:59.301 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-21T21:38:59.357 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-21T21:38:59.391 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-21T21:38:59.395 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:01:21 ago on Sat 21 Sep 2024 09:37:38 PM UTC. 2024-09-21T21:38:59.439 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-21T21:38:59.469 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mgr-rook-2:19.1.1-373.g4e45e6b4.el9.noarch 13/13 2024-09-21T21:38:59.504 INFO:teuthology.orchestra.run.smithi122.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 254 kB/s | 19 kB 00:00 2024-09-21T21:38:59.528 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-21T21:38:59.529 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-21T21:38:59.529 INFO:teuthology.orchestra.run.smithi164.stdout: Package Arch Version Repository Size 2024-09-21T21:38:59.529 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-21T21:38:59.529 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-09-21T21:38:59.529 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-cephadm noarch 2:19.1.1-373.g4e45e6b4.el9 ceph-noarch 146 k 2024-09-21T21:38:59.529 INFO:teuthology.orchestra.run.smithi164.stdout:Installing dependencies: 2024-09-21T21:38:59.529 INFO:teuthology.orchestra.run.smithi164.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T21:38:59.529 INFO:teuthology.orchestra.run.smithi164.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T21:38:59.530 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T21:38:59.530 INFO:teuthology.orchestra.run.smithi164.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T21:38:59.530 INFO:teuthology.orchestra.run.smithi164.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T21:38:59.530 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:38:59.530 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-09-21T21:38:59.530 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-21T21:38:59.530 INFO:teuthology.orchestra.run.smithi164.stdout:Install 6 Packages 2024-09-21T21:38:59.530 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:38:59.531 INFO:teuthology.orchestra.run.smithi122.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 254 kB/s | 26 kB 00:00 2024-09-21T21:38:59.531 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 7.0 M 2024-09-21T21:38:59.531 INFO:teuthology.orchestra.run.smithi164.stdout:Installed size: 33 M 2024-09-21T21:38:59.531 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-09-21T21:38:59.596 INFO:teuthology.orchestra.run.smithi122.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.4 MB/s | 222 kB 00:00 2024-09-21T21:38:59.621 INFO:teuthology.orchestra.run.smithi122.stdout:(4/13): ceph-mgr-rook-19.1.1-373.g4e45e6b4.el9. 255 kB/s | 49 kB 00:00 2024-09-21T21:38:59.647 INFO:teuthology.orchestra.run.smithi122.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.3 MB/s | 159 kB 00:00 2024-09-21T21:38:59.672 INFO:teuthology.orchestra.run.smithi122.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 3.6 MB/s | 279 kB 00:00 2024-09-21T21:38:59.714 INFO:teuthology.orchestra.run.smithi122.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 481 kB/s | 32 kB 00:00 2024-09-21T21:38:59.758 INFO:teuthology.orchestra.run.smithi122.stdout:(8/13): python3-certifi-2023.05.07-4.el9.noarch 234 kB/s | 14 kB 00:00 2024-09-21T21:38:59.764 INFO:teuthology.orchestra.run.smithi122.stdout:(9/13): python3-requests-oauthlib-1.3.0-12.el9. 376 kB/s | 54 kB 00:00 2024-09-21T21:38:59.790 INFO:teuthology.orchestra.run.smithi122.stdout:(10/13): python3-google-auth-2.34.0-1.el9.noarc 2.9 MB/s | 220 kB 00:00 2024-09-21T21:38:59.823 INFO:teuthology.orchestra.run.smithi122.stdout:(11/13): python3-websocket-client-1.2.3-2.el9.n 3.4 MB/s | 90 kB 00:00 2024-09-21T21:38:59.857 INFO:teuthology.orchestra.run.smithi122.stdout:(12/13): python3-kubernetes-26.1.0-2.el9.noarch 8.7 MB/s | 1.0 MB 00:00 2024-09-21T21:38:59.882 INFO:teuthology.orchestra.run.smithi122.stdout:(13/13): python3-rsa-4.9-2.el9.noarch.rpm 503 kB/s | 59 kB 00:00 2024-09-21T21:38:59.884 INFO:teuthology.orchestra.run.smithi122.stdout:-------------------------------------------------------------------------------- 2024-09-21T21:38:59.885 INFO:teuthology.orchestra.run.smithi122.stdout:Total 2.3 MB/s | 2.2 MB 00:00 2024-09-21T21:38:59.949 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction check 2024-09-21T21:38:59.964 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction check succeeded. 2024-09-21T21:38:59.965 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction test 2024-09-21T21:39:00.022 INFO:teuthology.orchestra.run.smithi164.stdout:(1/6): ceph-mgr-cephadm-19.1.1-373.g4e45e6b4.el 767 kB/s | 146 kB 00:00 2024-09-21T21:39:00.047 INFO:teuthology.orchestra.run.smithi164.stdout:(2/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.1 MB/s | 248 kB 00:00 2024-09-21T21:39:00.054 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction test succeeded. 2024-09-21T21:39:00.056 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction 2024-09-21T21:39:00.097 INFO:teuthology.orchestra.run.smithi164.stdout:(3/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 7.1 MB/s | 548 kB 00:00 2024-09-21T21:39:00.122 INFO:teuthology.orchestra.run.smithi164.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 769 kB/s | 58 kB 00:00 2024-09-21T21:39:00.147 INFO:teuthology.orchestra.run.smithi164.stdout:(5/6): python3-typing-extensions-4.12.2-2.el9.n 1.5 MB/s | 74 kB 00:00 2024-09-21T21:39:00.275 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-373.g4e45e6b4.el9.noarch 13/13 2024-09-21T21:39:00.275 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mgr-rook-2:19.1.1-373.g4e45e6b4.el9.noarch 1/13 2024-09-21T21:39:00.275 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-21T21:39:00.275 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-21T21:39:00.275 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-21T21:39:00.275 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-21T21:39:00.275 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-21T21:39:00.275 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-21T21:39:00.276 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-21T21:39:00.276 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-21T21:39:00.276 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-21T21:39:00.276 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-21T21:39:00.276 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-21T21:39:00.289 INFO:teuthology.orchestra.run.smithi164.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 13 MB/s | 6.0 MB 00:00 2024-09-21T21:39:00.291 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-09-21T21:39:00.292 INFO:teuthology.orchestra.run.smithi164.stdout:Total 9.2 MB/s | 7.0 MB 00:00 2024-09-21T21:39:00.362 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-09-21T21:39:00.374 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-09-21T21:39:00.374 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-09-21T21:39:00.474 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-09-21T21:39:00.475 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-09-21T21:39:00.562 INFO:teuthology.orchestra.run.smithi122.stdout: Preparing : 1/1 2024-09-21T21:39:00.625 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-21T21:39:00.625 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:39:00.625 INFO:teuthology.orchestra.run.smithi081.stdout:Installed: 2024-09-21T21:39:00.625 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-rook-2:19.1.1-373.g4e45e6b4.el9.noarch 2024-09-21T21:39:00.625 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T21:39:00.625 INFO:teuthology.orchestra.run.smithi081.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T21:39:00.625 INFO:teuthology.orchestra.run.smithi081.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T21:39:00.626 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T21:39:00.626 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T21:39:00.626 INFO:teuthology.orchestra.run.smithi081.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T21:39:00.626 INFO:teuthology.orchestra.run.smithi081.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T21:39:00.626 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T21:39:00.626 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T21:39:00.626 INFO:teuthology.orchestra.run.smithi081.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T21:39:00.626 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T21:39:00.626 INFO:teuthology.orchestra.run.smithi081.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T21:39:00.626 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:39:00.626 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-09-21T21:39:00.736 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-21T21:39:00.773 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-09-21T21:39:00.791 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-21T21:39:00.855 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-21T21:39:00.891 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum -y install ceph-mgr-cephadm 2024-09-21T21:39:00.892 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-21T21:39:00.929 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-21T21:39:00.934 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-21T21:39:00.983 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-21T21:39:01.037 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-21T21:39:01.138 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-21T21:39:01.196 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-21T21:39:01.476 INFO:teuthology.orchestra.run.smithi081.stdout:Last metadata expiration check: 0:01:25 ago on Sat 21 Sep 2024 09:37:36 PM UTC. 2024-09-21T21:39:01.509 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-21T21:39:01.594 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-21T21:39:01.609 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-09-21T21:39:01.610 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-09-21T21:39:01.610 INFO:teuthology.orchestra.run.smithi081.stdout: Package Arch Version Repository Size 2024-09-21T21:39:01.610 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-09-21T21:39:01.611 INFO:teuthology.orchestra.run.smithi081.stdout:Installing: 2024-09-21T21:39:01.611 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-cephadm noarch 2:19.1.1-373.g4e45e6b4.el9 ceph-noarch 146 k 2024-09-21T21:39:01.611 INFO:teuthology.orchestra.run.smithi081.stdout:Installing dependencies: 2024-09-21T21:39:01.611 INFO:teuthology.orchestra.run.smithi081.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T21:39:01.611 INFO:teuthology.orchestra.run.smithi081.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T21:39:01.611 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T21:39:01.611 INFO:teuthology.orchestra.run.smithi081.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T21:39:01.611 INFO:teuthology.orchestra.run.smithi081.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T21:39:01.611 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:39:01.611 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction Summary 2024-09-21T21:39:01.611 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-09-21T21:39:01.611 INFO:teuthology.orchestra.run.smithi081.stdout:Install 6 Packages 2024-09-21T21:39:01.611 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:39:01.612 INFO:teuthology.orchestra.run.smithi081.stdout:Total download size: 7.0 M 2024-09-21T21:39:01.612 INFO:teuthology.orchestra.run.smithi081.stdout:Installed size: 33 M 2024-09-21T21:39:01.612 INFO:teuthology.orchestra.run.smithi081.stdout:Downloading Packages: 2024-09-21T21:39:01.658 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-21T21:39:01.687 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-373.g4e45e6b4.el9.noarch 6/6 2024-09-21T21:39:01.903 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-21T21:39:01.958 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-21T21:39:01.992 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-21T21:39:02.040 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-21T21:39:02.062 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-rook-2:19.1.1-373.g4e45e6b4.el9.noarch 13/13 2024-09-21T21:39:02.200 INFO:teuthology.orchestra.run.smithi081.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.6 MB/s | 248 kB 00:00 2024-09-21T21:39:02.259 INFO:teuthology.orchestra.run.smithi081.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 9.2 MB/s | 548 kB 00:00 2024-09-21T21:39:02.284 INFO:teuthology.orchestra.run.smithi081.stdout:(3/6): ceph-mgr-cephadm-19.1.1-373.g4e45e6b4.el 633 kB/s | 146 kB 00:00 2024-09-21T21:39:02.310 INFO:teuthology.orchestra.run.smithi081.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 1.1 MB/s | 58 kB 00:00 2024-09-21T21:39:02.351 INFO:teuthology.orchestra.run.smithi081.stdout:(5/6): python3-typing-extensions-4.12.2-2.el9.n 1.1 MB/s | 74 kB 00:00 2024-09-21T21:39:02.435 INFO:teuthology.orchestra.run.smithi081.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 6.0 MB 00:00 2024-09-21T21:39:02.438 INFO:teuthology.orchestra.run.smithi081.stdout:-------------------------------------------------------------------------------- 2024-09-21T21:39:02.438 INFO:teuthology.orchestra.run.smithi081.stdout:Total 8.5 MB/s | 7.0 MB 00:00 2024-09-21T21:39:02.450 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-373.g4e45e6b4.el9.noarch 6/6 2024-09-21T21:39:02.450 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-373.g4e45e6b4.el9.noarch 1/6 2024-09-21T21:39:02.450 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-21T21:39:02.450 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-21T21:39:02.450 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-21T21:39:02.450 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-21T21:39:02.509 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction check 2024-09-21T21:39:02.520 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction check succeeded. 2024-09-21T21:39:02.521 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction test 2024-09-21T21:39:02.621 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction test succeeded. 2024-09-21T21:39:02.623 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction 2024-09-21T21:39:02.725 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-21T21:39:02.725 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:39:02.725 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-09-21T21:39:02.725 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-cephadm-2:19.1.1-373.g4e45e6b4.el9.noarch 2024-09-21T21:39:02.726 INFO:teuthology.orchestra.run.smithi164.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T21:39:02.726 INFO:teuthology.orchestra.run.smithi164.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T21:39:02.726 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T21:39:02.726 INFO:teuthology.orchestra.run.smithi164.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T21:39:02.726 INFO:teuthology.orchestra.run.smithi164.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T21:39:02.726 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:39:02.726 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-21T21:39:02.844 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-373.g4e45e6b4.el9.noarch 13/13 2024-09-21T21:39:02.844 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-rook-2:19.1.1-373.g4e45e6b4.el9.noarch 1/13 2024-09-21T21:39:02.844 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-21T21:39:02.844 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-21T21:39:02.844 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-21T21:39:02.845 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-21T21:39:02.845 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-21T21:39:02.845 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-21T21:39:02.845 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-21T21:39:02.845 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-21T21:39:02.845 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-21T21:39:02.845 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-21T21:39:02.845 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-21T21:39:02.926 INFO:teuthology.orchestra.run.smithi081.stdout: Preparing : 1/1 2024-09-21T21:39:02.962 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install ceph-fuse 2024-09-21T21:39:03.117 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-21T21:39:03.170 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-21T21:39:03.171 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-21T21:39:03.171 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:39:03.171 INFO:teuthology.orchestra.run.smithi122.stdout:Installed: 2024-09-21T21:39:03.171 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-rook-2:19.1.1-373.g4e45e6b4.el9.noarch 2024-09-21T21:39:03.171 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T21:39:03.171 INFO:teuthology.orchestra.run.smithi122.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T21:39:03.172 INFO:teuthology.orchestra.run.smithi122.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T21:39:03.172 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T21:39:03.172 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T21:39:03.172 INFO:teuthology.orchestra.run.smithi122.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T21:39:03.172 INFO:teuthology.orchestra.run.smithi122.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T21:39:03.172 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T21:39:03.172 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T21:39:03.172 INFO:teuthology.orchestra.run.smithi122.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T21:39:03.172 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T21:39:03.172 INFO:teuthology.orchestra.run.smithi122.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T21:39:03.172 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:39:03.172 INFO:teuthology.orchestra.run.smithi122.stdout:Complete! 2024-09-21T21:39:03.423 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum -y install ceph-mgr-cephadm 2024-09-21T21:39:03.541 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:01:25 ago on Sat 21 Sep 2024 09:37:38 PM UTC. 2024-09-21T21:39:03.677 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-21T21:39:03.677 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-21T21:39:03.677 INFO:teuthology.orchestra.run.smithi164.stdout: Package Arch Version Repository Size 2024-09-21T21:39:03.677 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-21T21:39:03.677 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-09-21T21:39:03.678 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-fuse x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 897 k 2024-09-21T21:39:03.678 INFO:teuthology.orchestra.run.smithi164.stdout:Installing dependencies: 2024-09-21T21:39:03.678 INFO:teuthology.orchestra.run.smithi164.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T21:39:03.678 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:39:03.678 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-09-21T21:39:03.679 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-21T21:39:03.679 INFO:teuthology.orchestra.run.smithi164.stdout:Install 2 Packages 2024-09-21T21:39:03.679 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:39:03.679 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 976 k 2024-09-21T21:39:03.679 INFO:teuthology.orchestra.run.smithi164.stdout:Installed size: 2.9 M 2024-09-21T21:39:03.679 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-09-21T21:39:03.718 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-21T21:39:03.799 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-21T21:39:03.871 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-21T21:39:03.899 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-373.g4e45e6b4.el9.noarch 6/6 2024-09-21T21:39:03.925 INFO:teuthology.orchestra.run.smithi164.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 561 kB/s | 80 kB 00:00 2024-09-21T21:39:04.007 INFO:teuthology.orchestra.run.smithi122.stdout:Last metadata expiration check: 0:01:27 ago on Sat 21 Sep 2024 09:37:37 PM UTC. 2024-09-21T21:39:04.058 INFO:teuthology.orchestra.run.smithi164.stdout:(2/2): ceph-fuse-19.1.1-373.g4e45e6b4.el9.x86_6 3.2 MB/s | 897 kB 00:00 2024-09-21T21:39:04.059 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-09-21T21:39:04.059 INFO:teuthology.orchestra.run.smithi164.stdout:Total 2.5 MB/s | 976 kB 00:00 2024-09-21T21:39:04.071 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-09-21T21:39:04.093 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-09-21T21:39:04.093 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-09-21T21:39:04.139 INFO:teuthology.orchestra.run.smithi122.stdout:Dependencies resolved. 2024-09-21T21:39:04.140 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-21T21:39:04.140 INFO:teuthology.orchestra.run.smithi122.stdout: Package Arch Version Repository Size 2024-09-21T21:39:04.140 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-21T21:39:04.140 INFO:teuthology.orchestra.run.smithi122.stdout:Installing: 2024-09-21T21:39:04.140 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-cephadm noarch 2:19.1.1-373.g4e45e6b4.el9 ceph-noarch 146 k 2024-09-21T21:39:04.140 INFO:teuthology.orchestra.run.smithi122.stdout:Installing dependencies: 2024-09-21T21:39:04.141 INFO:teuthology.orchestra.run.smithi122.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T21:39:04.141 INFO:teuthology.orchestra.run.smithi122.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T21:39:04.141 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T21:39:04.141 INFO:teuthology.orchestra.run.smithi122.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T21:39:04.141 INFO:teuthology.orchestra.run.smithi122.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T21:39:04.141 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:39:04.141 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction Summary 2024-09-21T21:39:04.141 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-21T21:39:04.141 INFO:teuthology.orchestra.run.smithi122.stdout:Install 6 Packages 2024-09-21T21:39:04.141 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:39:04.142 INFO:teuthology.orchestra.run.smithi122.stdout:Total download size: 7.0 M 2024-09-21T21:39:04.142 INFO:teuthology.orchestra.run.smithi122.stdout:Installed size: 33 M 2024-09-21T21:39:04.142 INFO:teuthology.orchestra.run.smithi122.stdout:Downloading Packages: 2024-09-21T21:39:04.173 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-09-21T21:39:04.174 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-09-21T21:39:04.359 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-09-21T21:39:04.409 INFO:teuthology.orchestra.run.smithi122.stdout:(1/6): ceph-mgr-cephadm-19.1.1-373.g4e45e6b4.el 674 kB/s | 146 kB 00:00 2024-09-21T21:39:04.495 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-21T21:39:04.501 INFO:teuthology.orchestra.run.smithi122.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 5.8 MB/s | 548 kB 00:00 2024-09-21T21:39:04.518 INFO:teuthology.orchestra.run.smithi122.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 3.4 MB/s | 58 kB 00:00 2024-09-21T21:39:04.552 INFO:teuthology.orchestra.run.smithi122.stdout:(4/6): python3-typing-extensions-4.12.2-2.el9.n 2.2 MB/s | 74 kB 00:00 2024-09-21T21:39:04.558 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-fuse-2:19.1.1-373.g4e45e6b4.el9.x86_64 2/2 2024-09-21T21:39:04.735 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-373.g4e45e6b4.el9.noarch 6/6 2024-09-21T21:39:04.735 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-373.g4e45e6b4.el9.noarch 1/6 2024-09-21T21:39:04.735 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-21T21:39:04.735 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-21T21:39:04.735 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-21T21:39:04.735 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-21T21:39:04.735 INFO:teuthology.orchestra.run.smithi122.stdout:(5/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 457 kB/s | 248 kB 00:00 2024-09-21T21:39:05.023 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-21T21:39:05.023 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:39:05.023 INFO:teuthology.orchestra.run.smithi081.stdout:Installed: 2024-09-21T21:39:05.023 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-cephadm-2:19.1.1-373.g4e45e6b4.el9.noarch 2024-09-21T21:39:05.023 INFO:teuthology.orchestra.run.smithi081.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T21:39:05.023 INFO:teuthology.orchestra.run.smithi081.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T21:39:05.023 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T21:39:05.023 INFO:teuthology.orchestra.run.smithi081.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T21:39:05.024 INFO:teuthology.orchestra.run.smithi081.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T21:39:05.024 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:39:05.024 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-09-21T21:39:05.261 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-fuse-2:19.1.1-373.g4e45e6b4.el9.x86_64 2/2 2024-09-21T21:39:05.261 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-fuse-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/2 2024-09-21T21:39:05.275 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum -y install ceph-fuse 2024-09-21T21:39:05.527 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-21T21:39:05.527 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:39:05.527 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-09-21T21:39:05.527 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-fuse-2:19.1.1-373.g4e45e6b4.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-21T21:39:05.527 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:39:05.527 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-21T21:39:05.853 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install ceph-volume 2024-09-21T21:39:05.858 INFO:teuthology.orchestra.run.smithi081.stdout:Last metadata expiration check: 0:01:29 ago on Sat 21 Sep 2024 09:37:36 PM UTC. 2024-09-21T21:39:05.992 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-09-21T21:39:05.993 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-09-21T21:39:05.993 INFO:teuthology.orchestra.run.smithi081.stdout: Package Arch Version Repository Size 2024-09-21T21:39:05.993 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-09-21T21:39:05.993 INFO:teuthology.orchestra.run.smithi081.stdout:Installing: 2024-09-21T21:39:05.993 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-fuse x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 897 k 2024-09-21T21:39:05.993 INFO:teuthology.orchestra.run.smithi081.stdout:Installing dependencies: 2024-09-21T21:39:05.993 INFO:teuthology.orchestra.run.smithi081.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T21:39:05.993 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:39:05.993 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction Summary 2024-09-21T21:39:05.994 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-09-21T21:39:05.994 INFO:teuthology.orchestra.run.smithi081.stdout:Install 2 Packages 2024-09-21T21:39:05.994 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:39:05.994 INFO:teuthology.orchestra.run.smithi081.stdout:Total download size: 976 k 2024-09-21T21:39:05.994 INFO:teuthology.orchestra.run.smithi081.stdout:Installed size: 2.9 M 2024-09-21T21:39:05.995 INFO:teuthology.orchestra.run.smithi081.stdout:Downloading Packages: 2024-09-21T21:39:06.069 INFO:teuthology.orchestra.run.smithi122.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 3.2 MB/s | 6.0 MB 00:01 2024-09-21T21:39:06.071 INFO:teuthology.orchestra.run.smithi122.stdout:-------------------------------------------------------------------------------- 2024-09-21T21:39:06.072 INFO:teuthology.orchestra.run.smithi122.stdout:Total 3.6 MB/s | 7.0 MB 00:01 2024-09-21T21:39:06.144 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction check 2024-09-21T21:39:06.155 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction check succeeded. 2024-09-21T21:39:06.155 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction test 2024-09-21T21:39:06.255 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction test succeeded. 2024-09-21T21:39:06.255 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction 2024-09-21T21:39:06.444 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:01:28 ago on Sat 21 Sep 2024 09:37:38 PM UTC. 2024-09-21T21:39:06.562 INFO:teuthology.orchestra.run.smithi081.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 170 kB/s | 80 kB 00:00 2024-09-21T21:39:06.562 INFO:teuthology.orchestra.run.smithi122.stdout: Preparing : 1/1 2024-09-21T21:39:06.580 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-21T21:39:06.581 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-21T21:39:06.581 INFO:teuthology.orchestra.run.smithi164.stdout: Package Arch Version Repository Size 2024-09-21T21:39:06.581 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-21T21:39:06.581 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-09-21T21:39:06.582 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-volume noarch 2:19.1.1-373.g4e45e6b4.el9 ceph-noarch 285 k 2024-09-21T21:39:06.582 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:39:06.582 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-09-21T21:39:06.582 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-21T21:39:06.582 INFO:teuthology.orchestra.run.smithi164.stdout:Install 1 Package 2024-09-21T21:39:06.582 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:39:06.582 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 285 k 2024-09-21T21:39:06.582 INFO:teuthology.orchestra.run.smithi164.stdout:Installed size: 1.3 M 2024-09-21T21:39:06.582 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-09-21T21:39:06.729 INFO:teuthology.orchestra.run.smithi081.stdout:(2/2): ceph-fuse-19.1.1-373.g4e45e6b4.el9.x86_6 1.4 MB/s | 897 kB 00:00 2024-09-21T21:39:06.729 INFO:teuthology.orchestra.run.smithi081.stdout:-------------------------------------------------------------------------------- 2024-09-21T21:39:06.730 INFO:teuthology.orchestra.run.smithi081.stdout:Total 1.3 MB/s | 976 kB 00:00 2024-09-21T21:39:06.734 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-21T21:39:06.742 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction check 2024-09-21T21:39:06.765 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction check succeeded. 2024-09-21T21:39:06.765 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction test 2024-09-21T21:39:06.780 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-21T21:39:06.847 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction test succeeded. 2024-09-21T21:39:06.848 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction 2024-09-21T21:39:06.922 INFO:teuthology.orchestra.run.smithi164.stdout:ceph-volume-19.1.1-373.g4e45e6b4.el9.noarch.rpm 838 kB/s | 285 kB 00:00 2024-09-21T21:39:06.923 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-09-21T21:39:06.923 INFO:teuthology.orchestra.run.smithi164.stdout:Total 834 kB/s | 285 kB 00:00 2024-09-21T21:39:06.923 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-09-21T21:39:06.932 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-09-21T21:39:06.932 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-09-21T21:39:06.965 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-09-21T21:39:06.965 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-09-21T21:39:07.068 INFO:teuthology.orchestra.run.smithi081.stdout: Preparing : 1/1 2024-09-21T21:39:07.185 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-09-21T21:39:07.206 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-21T21:39:07.258 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-volume-2:19.1.1-373.g4e45e6b4.el9.noarch 1/1 2024-09-21T21:39:07.274 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-volume-2:19.1.1-373.g4e45e6b4.el9.noarch 1/1 2024-09-21T21:39:07.274 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T21:39:07.274 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-21T21:39:07.274 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:39:07.287 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-fuse-2:19.1.1-373.g4e45e6b4.el9.x86_64 2/2 2024-09-21T21:39:07.326 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-21T21:39:07.423 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-21T21:39:07.487 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-21T21:39:07.516 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-373.g4e45e6b4.el9.noarch 6/6 2024-09-21T21:39:07.977 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-fuse-2:19.1.1-373.g4e45e6b4.el9.x86_64 2/2 2024-09-21T21:39:07.977 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-fuse-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/2 2024-09-21T21:39:08.265 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-21T21:39:08.265 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:39:08.265 INFO:teuthology.orchestra.run.smithi081.stdout:Installed: 2024-09-21T21:39:08.265 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-fuse-2:19.1.1-373.g4e45e6b4.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-21T21:39:08.265 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:39:08.266 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-09-21T21:39:08.318 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-373.g4e45e6b4.el9.noarch 6/6 2024-09-21T21:39:08.318 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-373.g4e45e6b4.el9.noarch 1/6 2024-09-21T21:39:08.318 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-21T21:39:08.318 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-21T21:39:08.318 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-21T21:39:08.318 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-21T21:39:08.333 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-volume-2:19.1.1-373.g4e45e6b4.el9.noarch 1/1 2024-09-21T21:39:08.333 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:39:08.333 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-09-21T21:39:08.333 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-volume-2:19.1.1-373.g4e45e6b4.el9.noarch 2024-09-21T21:39:08.333 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:39:08.334 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-21T21:39:08.561 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum -y install ceph-volume 2024-09-21T21:39:08.602 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-21T21:39:08.602 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:39:08.602 INFO:teuthology.orchestra.run.smithi122.stdout:Installed: 2024-09-21T21:39:08.602 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-cephadm-2:19.1.1-373.g4e45e6b4.el9.noarch 2024-09-21T21:39:08.603 INFO:teuthology.orchestra.run.smithi122.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T21:39:08.603 INFO:teuthology.orchestra.run.smithi122.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T21:39:08.603 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T21:39:08.603 INFO:teuthology.orchestra.run.smithi122.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T21:39:08.603 INFO:teuthology.orchestra.run.smithi122.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T21:39:08.603 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:39:08.603 INFO:teuthology.orchestra.run.smithi122.stdout:Complete! 2024-09-21T21:39:08.644 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install librados-devel 2024-09-21T21:39:08.850 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum -y install ceph-fuse 2024-09-21T21:39:09.144 INFO:teuthology.orchestra.run.smithi081.stdout:Last metadata expiration check: 0:01:33 ago on Sat 21 Sep 2024 09:37:36 PM UTC. 2024-09-21T21:39:09.254 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:01:31 ago on Sat 21 Sep 2024 09:37:38 PM UTC. 2024-09-21T21:39:09.279 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-09-21T21:39:09.280 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-09-21T21:39:09.280 INFO:teuthology.orchestra.run.smithi081.stdout: Package Arch Version Repository Size 2024-09-21T21:39:09.280 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-09-21T21:39:09.280 INFO:teuthology.orchestra.run.smithi081.stdout:Installing: 2024-09-21T21:39:09.280 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-volume noarch 2:19.1.1-373.g4e45e6b4.el9 ceph-noarch 285 k 2024-09-21T21:39:09.280 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:39:09.280 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction Summary 2024-09-21T21:39:09.281 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-09-21T21:39:09.281 INFO:teuthology.orchestra.run.smithi081.stdout:Install 1 Package 2024-09-21T21:39:09.281 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:39:09.281 INFO:teuthology.orchestra.run.smithi081.stdout:Total download size: 285 k 2024-09-21T21:39:09.281 INFO:teuthology.orchestra.run.smithi081.stdout:Installed size: 1.3 M 2024-09-21T21:39:09.281 INFO:teuthology.orchestra.run.smithi081.stdout:Downloading Packages: 2024-09-21T21:39:09.388 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-21T21:39:09.389 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-21T21:39:09.389 INFO:teuthology.orchestra.run.smithi164.stdout: Package Arch Version Repo Size 2024-09-21T21:39:09.389 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-21T21:39:09.389 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-09-21T21:39:09.390 INFO:teuthology.orchestra.run.smithi164.stdout: librados-devel x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 127 k 2024-09-21T21:39:09.390 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:39:09.390 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-09-21T21:39:09.390 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-21T21:39:09.390 INFO:teuthology.orchestra.run.smithi164.stdout:Install 1 Package 2024-09-21T21:39:09.390 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:39:09.390 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 127 k 2024-09-21T21:39:09.390 INFO:teuthology.orchestra.run.smithi164.stdout:Installed size: 456 k 2024-09-21T21:39:09.390 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-09-21T21:39:09.430 INFO:teuthology.orchestra.run.smithi122.stdout:Last metadata expiration check: 0:01:32 ago on Sat 21 Sep 2024 09:37:37 PM UTC. 2024-09-21T21:39:09.510 INFO:teuthology.orchestra.run.smithi081.stdout:ceph-volume-19.1.1-373.g4e45e6b4.el9.noarch.rpm 1.2 MB/s | 285 kB 00:00 2024-09-21T21:39:09.510 INFO:teuthology.orchestra.run.smithi081.stdout:-------------------------------------------------------------------------------- 2024-09-21T21:39:09.511 INFO:teuthology.orchestra.run.smithi081.stdout:Total 1.2 MB/s | 285 kB 00:00 2024-09-21T21:39:09.511 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction check 2024-09-21T21:39:09.519 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction check succeeded. 2024-09-21T21:39:09.519 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction test 2024-09-21T21:39:09.551 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction test succeeded. 2024-09-21T21:39:09.551 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction 2024-09-21T21:39:09.562 INFO:teuthology.orchestra.run.smithi122.stdout:Dependencies resolved. 2024-09-21T21:39:09.562 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-21T21:39:09.563 INFO:teuthology.orchestra.run.smithi122.stdout: Package Arch Version Repository Size 2024-09-21T21:39:09.563 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-21T21:39:09.563 INFO:teuthology.orchestra.run.smithi122.stdout:Installing: 2024-09-21T21:39:09.563 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-fuse x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 897 k 2024-09-21T21:39:09.563 INFO:teuthology.orchestra.run.smithi122.stdout:Installing dependencies: 2024-09-21T21:39:09.563 INFO:teuthology.orchestra.run.smithi122.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T21:39:09.563 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:39:09.563 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction Summary 2024-09-21T21:39:09.563 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-21T21:39:09.563 INFO:teuthology.orchestra.run.smithi122.stdout:Install 2 Packages 2024-09-21T21:39:09.564 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:39:09.564 INFO:teuthology.orchestra.run.smithi122.stdout:Total download size: 976 k 2024-09-21T21:39:09.564 INFO:teuthology.orchestra.run.smithi122.stdout:Installed size: 2.9 M 2024-09-21T21:39:09.564 INFO:teuthology.orchestra.run.smithi122.stdout:Downloading Packages: 2024-09-21T21:39:09.602 INFO:teuthology.orchestra.run.smithi164.stdout:librados-devel-19.1.1-373.g4e45e6b4.el9.x86_64. 596 kB/s | 127 kB 00:00 2024-09-21T21:39:09.603 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-09-21T21:39:09.603 INFO:teuthology.orchestra.run.smithi164.stdout:Total 592 kB/s | 127 kB 00:00 2024-09-21T21:39:09.604 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-09-21T21:39:09.609 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-09-21T21:39:09.609 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-09-21T21:39:09.672 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-09-21T21:39:09.672 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-09-21T21:39:09.779 INFO:teuthology.orchestra.run.smithi081.stdout: Preparing : 1/1 2024-09-21T21:39:09.813 INFO:teuthology.orchestra.run.smithi122.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 603 kB/s | 80 kB 00:00 2024-09-21T21:39:09.822 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-09-21T21:39:09.845 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-volume-2:19.1.1-373.g4e45e6b4.el9.noarch 1/1 2024-09-21T21:39:09.861 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-volume-2:19.1.1-373.g4e45e6b4.el9.noarch 1/1 2024-09-21T21:39:09.861 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T21:39:09.862 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-21T21:39:09.862 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:39:09.955 INFO:teuthology.orchestra.run.smithi122.stdout:(2/2): ceph-fuse-19.1.1-373.g4e45e6b4.el9.x86_6 3.2 MB/s | 897 kB 00:00 2024-09-21T21:39:09.955 INFO:teuthology.orchestra.run.smithi122.stdout:-------------------------------------------------------------------------------- 2024-09-21T21:39:09.956 INFO:teuthology.orchestra.run.smithi122.stdout:Total 2.4 MB/s | 976 kB 00:00 2024-09-21T21:39:09.967 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction check 2024-09-21T21:39:09.984 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : librados-devel-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-21T21:39:09.989 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction check succeeded. 2024-09-21T21:39:09.989 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction test 2024-09-21T21:39:10.069 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction test succeeded. 2024-09-21T21:39:10.070 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction 2024-09-21T21:39:10.257 INFO:teuthology.orchestra.run.smithi122.stdout: Preparing : 1/1 2024-09-21T21:39:10.382 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-21T21:39:10.400 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librados-devel-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-21T21:39:10.446 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-fuse-2:19.1.1-373.g4e45e6b4.el9.x86_64 2/2 2024-09-21T21:39:10.658 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librados-devel-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-21T21:39:10.658 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:39:10.658 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-09-21T21:39:10.658 INFO:teuthology.orchestra.run.smithi164.stdout: librados-devel-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:39:10.658 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:39:10.658 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-21T21:39:10.914 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install libcephfs2 2024-09-21T21:39:10.915 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-volume-2:19.1.1-373.g4e45e6b4.el9.noarch 1/1 2024-09-21T21:39:10.915 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:39:10.915 INFO:teuthology.orchestra.run.smithi081.stdout:Installed: 2024-09-21T21:39:10.915 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-volume-2:19.1.1-373.g4e45e6b4.el9.noarch 2024-09-21T21:39:10.915 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:39:10.915 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-09-21T21:39:11.093 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-fuse-2:19.1.1-373.g4e45e6b4.el9.x86_64 2/2 2024-09-21T21:39:11.094 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-fuse-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/2 2024-09-21T21:39:11.233 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum -y install librados-devel 2024-09-21T21:39:11.394 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-21T21:39:11.394 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:39:11.394 INFO:teuthology.orchestra.run.smithi122.stdout:Installed: 2024-09-21T21:39:11.394 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-fuse-2:19.1.1-373.g4e45e6b4.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-21T21:39:11.394 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:39:11.394 INFO:teuthology.orchestra.run.smithi122.stdout:Complete! 2024-09-21T21:39:11.502 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:01:33 ago on Sat 21 Sep 2024 09:37:38 PM UTC. 2024-09-21T21:39:11.592 INFO:teuthology.orchestra.run.smithi164.stdout:Package libcephfs2-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-21T21:39:11.638 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-21T21:39:11.639 INFO:teuthology.orchestra.run.smithi164.stdout:Nothing to do. 2024-09-21T21:39:11.639 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-21T21:39:11.669 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum -y install ceph-volume 2024-09-21T21:39:11.712 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install libcephfs-devel 2024-09-21T21:39:11.840 INFO:teuthology.orchestra.run.smithi081.stdout:Last metadata expiration check: 0:01:35 ago on Sat 21 Sep 2024 09:37:36 PM UTC. 2024-09-21T21:39:11.977 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-09-21T21:39:11.978 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-09-21T21:39:11.978 INFO:teuthology.orchestra.run.smithi081.stdout: Package Arch Version Repo Size 2024-09-21T21:39:11.978 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-09-21T21:39:11.978 INFO:teuthology.orchestra.run.smithi081.stdout:Installing: 2024-09-21T21:39:11.978 INFO:teuthology.orchestra.run.smithi081.stdout: librados-devel x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 127 k 2024-09-21T21:39:11.978 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:39:11.978 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction Summary 2024-09-21T21:39:11.978 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-09-21T21:39:11.978 INFO:teuthology.orchestra.run.smithi081.stdout:Install 1 Package 2024-09-21T21:39:11.978 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:39:11.979 INFO:teuthology.orchestra.run.smithi081.stdout:Total download size: 127 k 2024-09-21T21:39:11.979 INFO:teuthology.orchestra.run.smithi081.stdout:Installed size: 456 k 2024-09-21T21:39:11.979 INFO:teuthology.orchestra.run.smithi081.stdout:Downloading Packages: 2024-09-21T21:39:12.190 INFO:teuthology.orchestra.run.smithi081.stdout:librados-devel-19.1.1-373.g4e45e6b4.el9.x86_64. 600 kB/s | 127 kB 00:00 2024-09-21T21:39:12.191 INFO:teuthology.orchestra.run.smithi081.stdout:-------------------------------------------------------------------------------- 2024-09-21T21:39:12.191 INFO:teuthology.orchestra.run.smithi081.stdout:Total 596 kB/s | 127 kB 00:00 2024-09-21T21:39:12.191 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction check 2024-09-21T21:39:12.197 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction check succeeded. 2024-09-21T21:39:12.197 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction test 2024-09-21T21:39:12.261 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction test succeeded. 2024-09-21T21:39:12.261 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction 2024-09-21T21:39:12.263 INFO:teuthology.orchestra.run.smithi122.stdout:Last metadata expiration check: 0:01:35 ago on Sat 21 Sep 2024 09:37:37 PM UTC. 2024-09-21T21:39:12.297 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:01:34 ago on Sat 21 Sep 2024 09:37:38 PM UTC. 2024-09-21T21:39:12.399 INFO:teuthology.orchestra.run.smithi122.stdout:Dependencies resolved. 2024-09-21T21:39:12.400 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-21T21:39:12.400 INFO:teuthology.orchestra.run.smithi122.stdout: Package Arch Version Repository Size 2024-09-21T21:39:12.400 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-21T21:39:12.400 INFO:teuthology.orchestra.run.smithi122.stdout:Installing: 2024-09-21T21:39:12.400 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-volume noarch 2:19.1.1-373.g4e45e6b4.el9 ceph-noarch 285 k 2024-09-21T21:39:12.400 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:39:12.400 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction Summary 2024-09-21T21:39:12.400 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-21T21:39:12.401 INFO:teuthology.orchestra.run.smithi122.stdout:Install 1 Package 2024-09-21T21:39:12.401 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:39:12.401 INFO:teuthology.orchestra.run.smithi122.stdout:Total download size: 285 k 2024-09-21T21:39:12.401 INFO:teuthology.orchestra.run.smithi122.stdout:Installed size: 1.3 M 2024-09-21T21:39:12.401 INFO:teuthology.orchestra.run.smithi122.stdout:Downloading Packages: 2024-09-21T21:39:12.409 INFO:teuthology.orchestra.run.smithi081.stdout: Preparing : 1/1 2024-09-21T21:39:12.428 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-21T21:39:12.429 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-21T21:39:12.429 INFO:teuthology.orchestra.run.smithi164.stdout: Package Arch Version Repo Size 2024-09-21T21:39:12.429 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-21T21:39:12.429 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-09-21T21:39:12.429 INFO:teuthology.orchestra.run.smithi164.stdout: libcephfs-devel x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 32 k 2024-09-21T21:39:12.429 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:39:12.430 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-09-21T21:39:12.430 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-21T21:39:12.430 INFO:teuthology.orchestra.run.smithi164.stdout:Install 1 Package 2024-09-21T21:39:12.430 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:39:12.430 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 32 k 2024-09-21T21:39:12.430 INFO:teuthology.orchestra.run.smithi164.stdout:Installed size: 142 k 2024-09-21T21:39:12.430 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-09-21T21:39:12.530 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : librados-devel-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-21T21:39:12.590 INFO:teuthology.orchestra.run.smithi164.stdout:libcephfs-devel-19.1.1-373.g4e45e6b4.el9.x86_64 199 kB/s | 32 kB 00:00 2024-09-21T21:39:12.590 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-09-21T21:39:12.590 INFO:teuthology.orchestra.run.smithi164.stdout:Total 197 kB/s | 32 kB 00:00 2024-09-21T21:39:12.591 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-09-21T21:39:12.594 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-09-21T21:39:12.594 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-09-21T21:39:12.611 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-09-21T21:39:12.612 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-09-21T21:39:12.625 INFO:teuthology.orchestra.run.smithi122.stdout:ceph-volume-19.1.1-373.g4e45e6b4.el9.noarch.rpm 1.2 MB/s | 285 kB 00:00 2024-09-21T21:39:12.626 INFO:teuthology.orchestra.run.smithi122.stdout:-------------------------------------------------------------------------------- 2024-09-21T21:39:12.626 INFO:teuthology.orchestra.run.smithi122.stdout:Total 1.2 MB/s | 285 kB 00:00 2024-09-21T21:39:12.627 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction check 2024-09-21T21:39:12.634 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction check succeeded. 2024-09-21T21:39:12.635 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction test 2024-09-21T21:39:12.666 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction test succeeded. 2024-09-21T21:39:12.666 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction 2024-09-21T21:39:12.704 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-09-21T21:39:12.820 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libcephfs-devel-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-21T21:39:12.878 INFO:teuthology.orchestra.run.smithi122.stdout: Preparing : 1/1 2024-09-21T21:39:12.936 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-volume-2:19.1.1-373.g4e45e6b4.el9.noarch 1/1 2024-09-21T21:39:12.952 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-volume-2:19.1.1-373.g4e45e6b4.el9.noarch 1/1 2024-09-21T21:39:12.953 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T21:39:12.953 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-21T21:39:12.953 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:39:12.954 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: librados-devel-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-21T21:39:13.177 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librados-devel-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-21T21:39:13.177 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:39:13.178 INFO:teuthology.orchestra.run.smithi081.stdout:Installed: 2024-09-21T21:39:13.178 INFO:teuthology.orchestra.run.smithi081.stdout: librados-devel-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:39:13.178 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:39:13.178 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-09-21T21:39:13.202 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: libcephfs-devel-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-21T21:39:13.431 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libcephfs-devel-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-21T21:39:13.431 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:39:13.431 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-09-21T21:39:13.431 INFO:teuthology.orchestra.run.smithi164.stdout: libcephfs-devel-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:39:13.431 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:39:13.431 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-21T21:39:13.433 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum -y install libcephfs2 2024-09-21T21:39:13.611 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install librados2 2024-09-21T21:39:14.011 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-volume-2:19.1.1-373.g4e45e6b4.el9.noarch 1/1 2024-09-21T21:39:14.011 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:39:14.012 INFO:teuthology.orchestra.run.smithi122.stdout:Installed: 2024-09-21T21:39:14.012 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-volume-2:19.1.1-373.g4e45e6b4.el9.noarch 2024-09-21T21:39:14.012 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:39:14.012 INFO:teuthology.orchestra.run.smithi122.stdout:Complete! 2024-09-21T21:39:14.019 INFO:teuthology.orchestra.run.smithi081.stdout:Last metadata expiration check: 0:01:38 ago on Sat 21 Sep 2024 09:37:36 PM UTC. 2024-09-21T21:39:14.107 INFO:teuthology.orchestra.run.smithi081.stdout:Package libcephfs2-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-21T21:39:14.150 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-09-21T21:39:14.151 INFO:teuthology.orchestra.run.smithi081.stdout:Nothing to do. 2024-09-21T21:39:14.151 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-09-21T21:39:14.192 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:01:36 ago on Sat 21 Sep 2024 09:37:38 PM UTC. 2024-09-21T21:39:14.219 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum -y install libcephfs-devel 2024-09-21T21:39:14.281 INFO:teuthology.orchestra.run.smithi164.stdout:Package librados2-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-21T21:39:14.282 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum -y install librados-devel 2024-09-21T21:39:14.324 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-21T21:39:14.325 INFO:teuthology.orchestra.run.smithi164.stdout:Nothing to do. 2024-09-21T21:39:14.326 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-21T21:39:14.393 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install librbd1 2024-09-21T21:39:14.800 INFO:teuthology.orchestra.run.smithi081.stdout:Last metadata expiration check: 0:01:38 ago on Sat 21 Sep 2024 09:37:36 PM UTC. 2024-09-21T21:39:14.881 INFO:teuthology.orchestra.run.smithi122.stdout:Last metadata expiration check: 0:01:37 ago on Sat 21 Sep 2024 09:37:37 PM UTC. 2024-09-21T21:39:14.937 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-09-21T21:39:14.937 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-09-21T21:39:14.938 INFO:teuthology.orchestra.run.smithi081.stdout: Package Arch Version Repo Size 2024-09-21T21:39:14.938 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-09-21T21:39:14.938 INFO:teuthology.orchestra.run.smithi081.stdout:Installing: 2024-09-21T21:39:14.938 INFO:teuthology.orchestra.run.smithi081.stdout: libcephfs-devel x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 32 k 2024-09-21T21:39:14.938 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:39:14.938 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction Summary 2024-09-21T21:39:14.938 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-09-21T21:39:14.938 INFO:teuthology.orchestra.run.smithi081.stdout:Install 1 Package 2024-09-21T21:39:14.938 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:39:14.938 INFO:teuthology.orchestra.run.smithi081.stdout:Total download size: 32 k 2024-09-21T21:39:14.939 INFO:teuthology.orchestra.run.smithi081.stdout:Installed size: 142 k 2024-09-21T21:39:14.939 INFO:teuthology.orchestra.run.smithi081.stdout:Downloading Packages: 2024-09-21T21:39:14.965 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:01:36 ago on Sat 21 Sep 2024 09:37:38 PM UTC. 2024-09-21T21:39:15.017 INFO:teuthology.orchestra.run.smithi122.stdout:Dependencies resolved. 2024-09-21T21:39:15.018 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-21T21:39:15.018 INFO:teuthology.orchestra.run.smithi122.stdout: Package Arch Version Repo Size 2024-09-21T21:39:15.018 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-21T21:39:15.018 INFO:teuthology.orchestra.run.smithi122.stdout:Installing: 2024-09-21T21:39:15.018 INFO:teuthology.orchestra.run.smithi122.stdout: librados-devel x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 127 k 2024-09-21T21:39:15.018 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:39:15.018 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction Summary 2024-09-21T21:39:15.018 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-21T21:39:15.019 INFO:teuthology.orchestra.run.smithi122.stdout:Install 1 Package 2024-09-21T21:39:15.019 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:39:15.019 INFO:teuthology.orchestra.run.smithi122.stdout:Total download size: 127 k 2024-09-21T21:39:15.019 INFO:teuthology.orchestra.run.smithi122.stdout:Installed size: 456 k 2024-09-21T21:39:15.019 INFO:teuthology.orchestra.run.smithi122.stdout:Downloading Packages: 2024-09-21T21:39:15.054 INFO:teuthology.orchestra.run.smithi164.stdout:Package librbd1-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-21T21:39:15.086 INFO:teuthology.orchestra.run.smithi081.stdout:libcephfs-devel-19.1.1-373.g4e45e6b4.el9.x86_64 216 kB/s | 32 kB 00:00 2024-09-21T21:39:15.148 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-21T21:39:15.148 INFO:teuthology.orchestra.run.smithi164.stdout:Nothing to do. 2024-09-21T21:39:15.148 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-21T21:39:15.149 INFO:teuthology.orchestra.run.smithi081.stdout:-------------------------------------------------------------------------------- 2024-09-21T21:39:15.149 INFO:teuthology.orchestra.run.smithi081.stdout:Total 214 kB/s | 32 kB 00:00 2024-09-21T21:39:15.149 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction check 2024-09-21T21:39:15.149 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction check succeeded. 2024-09-21T21:39:15.149 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction test 2024-09-21T21:39:15.149 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction test succeeded. 2024-09-21T21:39:15.149 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction 2024-09-21T21:39:15.166 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install python3-rados 2024-09-21T21:39:15.198 INFO:teuthology.orchestra.run.smithi081.stdout: Preparing : 1/1 2024-09-21T21:39:15.206 INFO:teuthology.orchestra.run.smithi122.stdout:librados-devel-19.1.1-373.g4e45e6b4.el9.x86_64. 673 kB/s | 127 kB 00:00 2024-09-21T21:39:15.207 INFO:teuthology.orchestra.run.smithi122.stdout:-------------------------------------------------------------------------------- 2024-09-21T21:39:15.207 INFO:teuthology.orchestra.run.smithi122.stdout:Total 668 kB/s | 127 kB 00:00 2024-09-21T21:39:15.207 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction check 2024-09-21T21:39:15.213 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction check succeeded. 2024-09-21T21:39:15.213 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction test 2024-09-21T21:39:15.277 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction test succeeded. 2024-09-21T21:39:15.278 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction 2024-09-21T21:39:15.317 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libcephfs-devel-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-21T21:39:15.430 INFO:teuthology.orchestra.run.smithi122.stdout: Preparing : 1/1 2024-09-21T21:39:15.521 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : librados-devel-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-21T21:39:15.680 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: libcephfs-devel-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-21T21:39:15.744 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:01:37 ago on Sat 21 Sep 2024 09:37:38 PM UTC. 2024-09-21T21:39:15.832 INFO:teuthology.orchestra.run.smithi164.stdout:Package python3-rados-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-21T21:39:15.874 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-21T21:39:15.875 INFO:teuthology.orchestra.run.smithi164.stdout:Nothing to do. 2024-09-21T21:39:15.875 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-21T21:39:15.920 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libcephfs-devel-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-21T21:39:15.920 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:39:15.920 INFO:teuthology.orchestra.run.smithi081.stdout:Installed: 2024-09-21T21:39:15.921 INFO:teuthology.orchestra.run.smithi081.stdout: libcephfs-devel-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:39:15.921 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:39:15.921 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-09-21T21:39:15.944 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install python3-rgw 2024-09-21T21:39:15.963 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: librados-devel-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-21T21:39:16.091 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum -y install librados2 2024-09-21T21:39:16.210 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librados-devel-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-21T21:39:16.211 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:39:16.211 INFO:teuthology.orchestra.run.smithi122.stdout:Installed: 2024-09-21T21:39:16.211 INFO:teuthology.orchestra.run.smithi122.stdout: librados-devel-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:39:16.211 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:39:16.211 INFO:teuthology.orchestra.run.smithi122.stdout:Complete! 2024-09-21T21:39:16.425 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum -y install libcephfs2 2024-09-21T21:39:16.517 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:01:38 ago on Sat 21 Sep 2024 09:37:38 PM UTC. 2024-09-21T21:39:16.605 INFO:teuthology.orchestra.run.smithi164.stdout:Package python3-rgw-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-21T21:39:16.648 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-21T21:39:16.649 INFO:teuthology.orchestra.run.smithi164.stdout:Nothing to do. 2024-09-21T21:39:16.649 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-21T21:39:16.668 INFO:teuthology.orchestra.run.smithi081.stdout:Last metadata expiration check: 0:01:40 ago on Sat 21 Sep 2024 09:37:36 PM UTC. 2024-09-21T21:39:16.716 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install python3-cephfs 2024-09-21T21:39:16.757 INFO:teuthology.orchestra.run.smithi081.stdout:Package librados2-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-21T21:39:16.800 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-09-21T21:39:16.801 INFO:teuthology.orchestra.run.smithi081.stdout:Nothing to do. 2024-09-21T21:39:16.801 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-09-21T21:39:16.872 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum -y install librbd1 2024-09-21T21:39:17.014 INFO:teuthology.orchestra.run.smithi122.stdout:Last metadata expiration check: 0:01:40 ago on Sat 21 Sep 2024 09:37:37 PM UTC. 2024-09-21T21:39:17.103 INFO:teuthology.orchestra.run.smithi122.stdout:Package libcephfs2-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-21T21:39:17.147 INFO:teuthology.orchestra.run.smithi122.stdout:Dependencies resolved. 2024-09-21T21:39:17.148 INFO:teuthology.orchestra.run.smithi122.stdout:Nothing to do. 2024-09-21T21:39:17.148 INFO:teuthology.orchestra.run.smithi122.stdout:Complete! 2024-09-21T21:39:17.216 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum -y install libcephfs-devel 2024-09-21T21:39:17.298 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:01:39 ago on Sat 21 Sep 2024 09:37:38 PM UTC. 2024-09-21T21:39:17.388 INFO:teuthology.orchestra.run.smithi164.stdout:Package python3-cephfs-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-21T21:39:17.432 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-21T21:39:17.433 INFO:teuthology.orchestra.run.smithi164.stdout:Nothing to do. 2024-09-21T21:39:17.433 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-21T21:39:17.450 INFO:teuthology.orchestra.run.smithi081.stdout:Last metadata expiration check: 0:01:41 ago on Sat 21 Sep 2024 09:37:36 PM UTC. 2024-09-21T21:39:17.501 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install python3-rbd 2024-09-21T21:39:17.539 INFO:teuthology.orchestra.run.smithi081.stdout:Package librbd1-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-21T21:39:17.583 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-09-21T21:39:17.584 INFO:teuthology.orchestra.run.smithi081.stdout:Nothing to do. 2024-09-21T21:39:17.584 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-09-21T21:39:17.653 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum -y install python3-rados 2024-09-21T21:39:17.804 INFO:teuthology.orchestra.run.smithi122.stdout:Last metadata expiration check: 0:01:40 ago on Sat 21 Sep 2024 09:37:37 PM UTC. 2024-09-21T21:39:17.935 INFO:teuthology.orchestra.run.smithi122.stdout:Dependencies resolved. 2024-09-21T21:39:17.935 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-21T21:39:17.936 INFO:teuthology.orchestra.run.smithi122.stdout: Package Arch Version Repo Size 2024-09-21T21:39:17.936 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-21T21:39:17.936 INFO:teuthology.orchestra.run.smithi122.stdout:Installing: 2024-09-21T21:39:17.936 INFO:teuthology.orchestra.run.smithi122.stdout: libcephfs-devel x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 32 k 2024-09-21T21:39:17.936 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:39:17.936 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction Summary 2024-09-21T21:39:17.936 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-21T21:39:17.936 INFO:teuthology.orchestra.run.smithi122.stdout:Install 1 Package 2024-09-21T21:39:17.936 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:39:17.936 INFO:teuthology.orchestra.run.smithi122.stdout:Total download size: 32 k 2024-09-21T21:39:17.936 INFO:teuthology.orchestra.run.smithi122.stdout:Installed size: 142 k 2024-09-21T21:39:17.936 INFO:teuthology.orchestra.run.smithi122.stdout:Downloading Packages: 2024-09-21T21:39:18.076 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:01:40 ago on Sat 21 Sep 2024 09:37:38 PM UTC. 2024-09-21T21:39:18.077 INFO:teuthology.orchestra.run.smithi122.stdout:libcephfs-devel-19.1.1-373.g4e45e6b4.el9.x86_64 226 kB/s | 32 kB 00:00 2024-09-21T21:39:18.077 INFO:teuthology.orchestra.run.smithi122.stdout:-------------------------------------------------------------------------------- 2024-09-21T21:39:18.078 INFO:teuthology.orchestra.run.smithi122.stdout:Total 224 kB/s | 32 kB 00:00 2024-09-21T21:39:18.078 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction check 2024-09-21T21:39:18.081 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction check succeeded. 2024-09-21T21:39:18.081 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction test 2024-09-21T21:39:18.099 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction test succeeded. 2024-09-21T21:39:18.099 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction 2024-09-21T21:39:18.164 INFO:teuthology.orchestra.run.smithi164.stdout:Package python3-rbd-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-21T21:39:18.186 INFO:teuthology.orchestra.run.smithi122.stdout: Preparing : 1/1 2024-09-21T21:39:18.207 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-21T21:39:18.208 INFO:teuthology.orchestra.run.smithi164.stdout:Nothing to do. 2024-09-21T21:39:18.208 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-21T21:39:18.234 INFO:teuthology.orchestra.run.smithi081.stdout:Last metadata expiration check: 0:01:42 ago on Sat 21 Sep 2024 09:37:36 PM UTC. 2024-09-21T21:39:18.275 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install rbd-fuse 2024-09-21T21:39:18.315 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libcephfs-devel-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-21T21:39:18.323 INFO:teuthology.orchestra.run.smithi081.stdout:Package python3-rados-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-21T21:39:18.366 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-09-21T21:39:18.368 INFO:teuthology.orchestra.run.smithi081.stdout:Nothing to do. 2024-09-21T21:39:18.368 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-09-21T21:39:18.437 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum -y install python3-rgw 2024-09-21T21:39:18.679 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: libcephfs-devel-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-21T21:39:18.859 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:01:40 ago on Sat 21 Sep 2024 09:37:38 PM UTC. 2024-09-21T21:39:18.968 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libcephfs-devel-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-21T21:39:18.968 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:39:18.969 INFO:teuthology.orchestra.run.smithi122.stdout:Installed: 2024-09-21T21:39:18.969 INFO:teuthology.orchestra.run.smithi122.stdout: libcephfs-devel-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:39:18.969 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:39:18.969 INFO:teuthology.orchestra.run.smithi122.stdout:Complete! 2024-09-21T21:39:19.003 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-21T21:39:19.004 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-21T21:39:19.004 INFO:teuthology.orchestra.run.smithi164.stdout: Package Architecture Version Repository Size 2024-09-21T21:39:19.004 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-21T21:39:19.004 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-09-21T21:39:19.005 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-fuse x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 85 k 2024-09-21T21:39:19.005 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:39:19.005 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-09-21T21:39:19.005 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-21T21:39:19.005 INFO:teuthology.orchestra.run.smithi164.stdout:Install 1 Package 2024-09-21T21:39:19.005 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:39:19.005 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 85 k 2024-09-21T21:39:19.005 INFO:teuthology.orchestra.run.smithi164.stdout:Installed size: 231 k 2024-09-21T21:39:19.005 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-09-21T21:39:19.005 INFO:teuthology.orchestra.run.smithi081.stdout:Last metadata expiration check: 0:01:43 ago on Sat 21 Sep 2024 09:37:36 PM UTC. 2024-09-21T21:39:19.092 INFO:teuthology.orchestra.run.smithi081.stdout:Package python3-rgw-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-21T21:39:19.135 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-09-21T21:39:19.136 INFO:teuthology.orchestra.run.smithi081.stdout:Nothing to do. 2024-09-21T21:39:19.136 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-09-21T21:39:19.153 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum -y install librados2 2024-09-21T21:39:19.200 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum -y install python3-cephfs 2024-09-21T21:39:19.201 INFO:teuthology.orchestra.run.smithi164.stdout:rbd-fuse-19.1.1-373.g4e45e6b4.el9.x86_64.rpm 436 kB/s | 85 kB 00:00 2024-09-21T21:39:19.201 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-09-21T21:39:19.202 INFO:teuthology.orchestra.run.smithi164.stdout:Total 434 kB/s | 85 kB 00:00 2024-09-21T21:39:19.202 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-09-21T21:39:19.209 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-09-21T21:39:19.209 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-09-21T21:39:19.272 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-09-21T21:39:19.273 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-09-21T21:39:19.415 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-09-21T21:39:19.523 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : rbd-fuse-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-21T21:39:19.726 INFO:teuthology.orchestra.run.smithi122.stdout:Last metadata expiration check: 0:01:42 ago on Sat 21 Sep 2024 09:37:37 PM UTC. 2024-09-21T21:39:19.786 INFO:teuthology.orchestra.run.smithi081.stdout:Last metadata expiration check: 0:01:43 ago on Sat 21 Sep 2024 09:37:36 PM UTC. 2024-09-21T21:39:19.813 INFO:teuthology.orchestra.run.smithi122.stdout:Package librados2-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-21T21:39:19.856 INFO:teuthology.orchestra.run.smithi122.stdout:Dependencies resolved. 2024-09-21T21:39:19.857 INFO:teuthology.orchestra.run.smithi122.stdout:Nothing to do. 2024-09-21T21:39:19.857 INFO:teuthology.orchestra.run.smithi122.stdout:Complete! 2024-09-21T21:39:19.876 INFO:teuthology.orchestra.run.smithi081.stdout:Package python3-cephfs-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-21T21:39:19.906 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: rbd-fuse-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-21T21:39:19.920 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-09-21T21:39:19.921 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum -y install librbd1 2024-09-21T21:39:19.922 INFO:teuthology.orchestra.run.smithi081.stdout:Nothing to do. 2024-09-21T21:39:19.922 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-09-21T21:39:19.995 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum -y install python3-rbd 2024-09-21T21:39:20.150 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : rbd-fuse-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-21T21:39:20.150 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:39:20.150 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-09-21T21:39:20.150 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-fuse-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:39:20.150 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:39:20.151 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-21T21:39:20.429 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install rbd-mirror 2024-09-21T21:39:20.497 INFO:teuthology.orchestra.run.smithi122.stdout:Last metadata expiration check: 0:01:43 ago on Sat 21 Sep 2024 09:37:37 PM UTC. 2024-09-21T21:39:20.574 INFO:teuthology.orchestra.run.smithi081.stdout:Last metadata expiration check: 0:01:44 ago on Sat 21 Sep 2024 09:37:36 PM UTC. 2024-09-21T21:39:20.587 INFO:teuthology.orchestra.run.smithi122.stdout:Package librbd1-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-21T21:39:20.630 INFO:teuthology.orchestra.run.smithi122.stdout:Dependencies resolved. 2024-09-21T21:39:20.631 INFO:teuthology.orchestra.run.smithi122.stdout:Nothing to do. 2024-09-21T21:39:20.631 INFO:teuthology.orchestra.run.smithi122.stdout:Complete! 2024-09-21T21:39:20.662 INFO:teuthology.orchestra.run.smithi081.stdout:Package python3-rbd-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-21T21:39:20.697 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum -y install python3-rados 2024-09-21T21:39:20.704 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-09-21T21:39:20.705 INFO:teuthology.orchestra.run.smithi081.stdout:Nothing to do. 2024-09-21T21:39:20.706 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-09-21T21:39:20.776 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum -y install rbd-fuse 2024-09-21T21:39:21.033 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:01:43 ago on Sat 21 Sep 2024 09:37:38 PM UTC. 2024-09-21T21:39:21.177 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-21T21:39:21.178 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-21T21:39:21.178 INFO:teuthology.orchestra.run.smithi164.stdout: Package Arch Version Repo Size 2024-09-21T21:39:21.178 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-21T21:39:21.178 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-09-21T21:39:21.178 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-mirror x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 3.1 M 2024-09-21T21:39:21.178 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:39:21.178 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-09-21T21:39:21.178 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-21T21:39:21.179 INFO:teuthology.orchestra.run.smithi164.stdout:Install 1 Package 2024-09-21T21:39:21.179 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:39:21.179 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 3.1 M 2024-09-21T21:39:21.179 INFO:teuthology.orchestra.run.smithi164.stdout:Installed size: 13 M 2024-09-21T21:39:21.179 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-09-21T21:39:21.271 INFO:teuthology.orchestra.run.smithi122.stdout:Last metadata expiration check: 0:01:44 ago on Sat 21 Sep 2024 09:37:37 PM UTC. 2024-09-21T21:39:21.352 INFO:teuthology.orchestra.run.smithi081.stdout:Last metadata expiration check: 0:01:45 ago on Sat 21 Sep 2024 09:37:36 PM UTC. 2024-09-21T21:39:21.358 INFO:teuthology.orchestra.run.smithi122.stdout:Package python3-rados-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-21T21:39:21.401 INFO:teuthology.orchestra.run.smithi122.stdout:Dependencies resolved. 2024-09-21T21:39:21.402 INFO:teuthology.orchestra.run.smithi122.stdout:Nothing to do. 2024-09-21T21:39:21.402 INFO:teuthology.orchestra.run.smithi122.stdout:Complete! 2024-09-21T21:39:21.471 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum -y install python3-rgw 2024-09-21T21:39:21.483 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-09-21T21:39:21.484 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-09-21T21:39:21.484 INFO:teuthology.orchestra.run.smithi081.stdout: Package Architecture Version Repository Size 2024-09-21T21:39:21.485 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-09-21T21:39:21.485 INFO:teuthology.orchestra.run.smithi081.stdout:Installing: 2024-09-21T21:39:21.485 INFO:teuthology.orchestra.run.smithi081.stdout: rbd-fuse x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 85 k 2024-09-21T21:39:21.485 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:39:21.485 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction Summary 2024-09-21T21:39:21.485 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-09-21T21:39:21.485 INFO:teuthology.orchestra.run.smithi081.stdout:Install 1 Package 2024-09-21T21:39:21.485 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:39:21.486 INFO:teuthology.orchestra.run.smithi081.stdout:Total download size: 85 k 2024-09-21T21:39:21.486 INFO:teuthology.orchestra.run.smithi081.stdout:Installed size: 231 k 2024-09-21T21:39:21.486 INFO:teuthology.orchestra.run.smithi081.stdout:Downloading Packages: 2024-09-21T21:39:21.681 INFO:teuthology.orchestra.run.smithi081.stdout:rbd-fuse-19.1.1-373.g4e45e6b4.el9.x86_64.rpm 436 kB/s | 85 kB 00:00 2024-09-21T21:39:21.681 INFO:teuthology.orchestra.run.smithi081.stdout:-------------------------------------------------------------------------------- 2024-09-21T21:39:21.681 INFO:teuthology.orchestra.run.smithi081.stdout:Total 433 kB/s | 85 kB 00:00 2024-09-21T21:39:21.681 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction check 2024-09-21T21:39:21.688 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction check succeeded. 2024-09-21T21:39:21.688 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction test 2024-09-21T21:39:21.751 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction test succeeded. 2024-09-21T21:39:21.751 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction 2024-09-21T21:39:21.814 INFO:teuthology.orchestra.run.smithi164.stdout:rbd-mirror-19.1.1-373.g4e45e6b4.el9.x86_64.rpm 4.9 MB/s | 3.1 MB 00:00 2024-09-21T21:39:21.814 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-09-21T21:39:21.815 INFO:teuthology.orchestra.run.smithi164.stdout:Total 4.9 MB/s | 3.1 MB 00:00 2024-09-21T21:39:21.815 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-09-21T21:39:21.824 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-09-21T21:39:21.824 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-09-21T21:39:21.901 INFO:teuthology.orchestra.run.smithi081.stdout: Preparing : 1/1 2024-09-21T21:39:21.905 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-09-21T21:39:21.906 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-09-21T21:39:22.019 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : rbd-fuse-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-21T21:39:22.043 INFO:teuthology.orchestra.run.smithi122.stdout:Last metadata expiration check: 0:01:45 ago on Sat 21 Sep 2024 09:37:37 PM UTC. 2024-09-21T21:39:22.130 INFO:teuthology.orchestra.run.smithi122.stdout:Package python3-rgw-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-21T21:39:22.173 INFO:teuthology.orchestra.run.smithi122.stdout:Dependencies resolved. 2024-09-21T21:39:22.174 INFO:teuthology.orchestra.run.smithi122.stdout:Nothing to do. 2024-09-21T21:39:22.174 INFO:teuthology.orchestra.run.smithi122.stdout:Complete! 2024-09-21T21:39:22.250 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum -y install python3-cephfs 2024-09-21T21:39:22.342 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-09-21T21:39:22.366 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: rbd-fuse-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-21T21:39:22.523 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : rbd-mirror-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-21T21:39:22.550 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: rbd-mirror-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-21T21:39:22.550 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T21:39:22.550 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T21:39:22.550 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T21:39:22.550 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T21:39:22.553 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:39:22.623 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : rbd-fuse-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-21T21:39:22.623 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:39:22.623 INFO:teuthology.orchestra.run.smithi081.stdout:Installed: 2024-09-21T21:39:22.623 INFO:teuthology.orchestra.run.smithi081.stdout: rbd-fuse-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:39:22.623 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:39:22.623 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-09-21T21:39:22.832 INFO:teuthology.orchestra.run.smithi122.stdout:Last metadata expiration check: 0:01:45 ago on Sat 21 Sep 2024 09:37:37 PM UTC. 2024-09-21T21:39:22.900 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum -y install rbd-mirror 2024-09-21T21:39:22.921 INFO:teuthology.orchestra.run.smithi122.stdout:Package python3-cephfs-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-21T21:39:22.965 INFO:teuthology.orchestra.run.smithi122.stdout:Dependencies resolved. 2024-09-21T21:39:22.966 INFO:teuthology.orchestra.run.smithi122.stdout:Nothing to do. 2024-09-21T21:39:22.966 INFO:teuthology.orchestra.run.smithi122.stdout:Complete! 2024-09-21T21:39:23.030 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum -y install python3-rbd 2024-09-21T21:39:23.519 INFO:teuthology.orchestra.run.smithi081.stdout:Last metadata expiration check: 0:01:47 ago on Sat 21 Sep 2024 09:37:36 PM UTC. 2024-09-21T21:39:23.599 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : rbd-mirror-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-21T21:39:23.599 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:39:23.599 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-09-21T21:39:23.599 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-mirror-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:39:23.599 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:39:23.599 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-21T21:39:23.608 INFO:teuthology.orchestra.run.smithi122.stdout:Last metadata expiration check: 0:01:46 ago on Sat 21 Sep 2024 09:37:37 PM UTC. 2024-09-21T21:39:23.658 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-09-21T21:39:23.659 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-09-21T21:39:23.659 INFO:teuthology.orchestra.run.smithi081.stdout: Package Arch Version Repo Size 2024-09-21T21:39:23.659 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-09-21T21:39:23.659 INFO:teuthology.orchestra.run.smithi081.stdout:Installing: 2024-09-21T21:39:23.660 INFO:teuthology.orchestra.run.smithi081.stdout: rbd-mirror x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 3.1 M 2024-09-21T21:39:23.660 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:39:23.660 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction Summary 2024-09-21T21:39:23.660 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-09-21T21:39:23.660 INFO:teuthology.orchestra.run.smithi081.stdout:Install 1 Package 2024-09-21T21:39:23.660 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:39:23.660 INFO:teuthology.orchestra.run.smithi081.stdout:Total download size: 3.1 M 2024-09-21T21:39:23.660 INFO:teuthology.orchestra.run.smithi081.stdout:Installed size: 13 M 2024-09-21T21:39:23.660 INFO:teuthology.orchestra.run.smithi081.stdout:Downloading Packages: 2024-09-21T21:39:23.696 INFO:teuthology.orchestra.run.smithi122.stdout:Package python3-rbd-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-21T21:39:23.738 INFO:teuthology.orchestra.run.smithi122.stdout:Dependencies resolved. 2024-09-21T21:39:23.739 INFO:teuthology.orchestra.run.smithi122.stdout:Nothing to do. 2024-09-21T21:39:23.740 INFO:teuthology.orchestra.run.smithi122.stdout:Complete! 2024-09-21T21:39:23.808 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum -y install rbd-fuse 2024-09-21T21:39:23.867 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install rbd-nbd 2024-09-21T21:39:24.227 INFO:teuthology.orchestra.run.smithi081.stdout:rbd-mirror-19.1.1-373.g4e45e6b4.el9.x86_64.rpm 5.5 MB/s | 3.1 MB 00:00 2024-09-21T21:39:24.227 INFO:teuthology.orchestra.run.smithi081.stdout:-------------------------------------------------------------------------------- 2024-09-21T21:39:24.228 INFO:teuthology.orchestra.run.smithi081.stdout:Total 5.5 MB/s | 3.1 MB 00:00 2024-09-21T21:39:24.228 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction check 2024-09-21T21:39:24.237 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction check succeeded. 2024-09-21T21:39:24.237 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction test 2024-09-21T21:39:24.318 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction test succeeded. 2024-09-21T21:39:24.318 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction 2024-09-21T21:39:24.384 INFO:teuthology.orchestra.run.smithi122.stdout:Last metadata expiration check: 0:01:47 ago on Sat 21 Sep 2024 09:37:37 PM UTC. 2024-09-21T21:39:24.472 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:01:46 ago on Sat 21 Sep 2024 09:37:38 PM UTC. 2024-09-21T21:39:24.516 INFO:teuthology.orchestra.run.smithi122.stdout:Dependencies resolved. 2024-09-21T21:39:24.516 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-21T21:39:24.516 INFO:teuthology.orchestra.run.smithi122.stdout: Package Architecture Version Repository Size 2024-09-21T21:39:24.516 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-21T21:39:24.516 INFO:teuthology.orchestra.run.smithi122.stdout:Installing: 2024-09-21T21:39:24.516 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-fuse x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 85 k 2024-09-21T21:39:24.517 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:39:24.517 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction Summary 2024-09-21T21:39:24.517 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-21T21:39:24.517 INFO:teuthology.orchestra.run.smithi122.stdout:Install 1 Package 2024-09-21T21:39:24.517 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:39:24.517 INFO:teuthology.orchestra.run.smithi122.stdout:Total download size: 85 k 2024-09-21T21:39:24.517 INFO:teuthology.orchestra.run.smithi122.stdout:Installed size: 231 k 2024-09-21T21:39:24.518 INFO:teuthology.orchestra.run.smithi122.stdout:Downloading Packages: 2024-09-21T21:39:24.605 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-21T21:39:24.605 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-21T21:39:24.606 INFO:teuthology.orchestra.run.smithi164.stdout: Package Architecture Version Repository Size 2024-09-21T21:39:24.606 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-21T21:39:24.606 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-09-21T21:39:24.606 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-nbd x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 170 k 2024-09-21T21:39:24.606 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:39:24.606 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-09-21T21:39:24.606 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-21T21:39:24.606 INFO:teuthology.orchestra.run.smithi164.stdout:Install 1 Package 2024-09-21T21:39:24.606 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:39:24.606 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 170 k 2024-09-21T21:39:24.606 INFO:teuthology.orchestra.run.smithi164.stdout:Installed size: 494 k 2024-09-21T21:39:24.607 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-09-21T21:39:24.722 INFO:teuthology.orchestra.run.smithi122.stdout:rbd-fuse-19.1.1-373.g4e45e6b4.el9.x86_64.rpm 418 kB/s | 85 kB 00:00 2024-09-21T21:39:24.722 INFO:teuthology.orchestra.run.smithi122.stdout:-------------------------------------------------------------------------------- 2024-09-21T21:39:24.723 INFO:teuthology.orchestra.run.smithi122.stdout:Total 415 kB/s | 85 kB 00:00 2024-09-21T21:39:24.723 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction check 2024-09-21T21:39:24.729 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction check succeeded. 2024-09-21T21:39:24.730 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction test 2024-09-21T21:39:24.751 INFO:teuthology.orchestra.run.smithi081.stdout: Preparing : 1/1 2024-09-21T21:39:24.792 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction test succeeded. 2024-09-21T21:39:24.792 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction 2024-09-21T21:39:24.817 INFO:teuthology.orchestra.run.smithi164.stdout:rbd-nbd-19.1.1-373.g4e45e6b4.el9.x86_64.rpm 807 kB/s | 170 kB 00:00 2024-09-21T21:39:24.818 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-09-21T21:39:24.818 INFO:teuthology.orchestra.run.smithi164.stdout:Total 801 kB/s | 170 kB 00:00 2024-09-21T21:39:24.818 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-09-21T21:39:24.819 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : rbd-mirror-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-21T21:39:24.826 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-09-21T21:39:24.826 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-09-21T21:39:24.846 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: rbd-mirror-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-21T21:39:24.846 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T21:39:24.846 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T21:39:24.846 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T21:39:24.846 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T21:39:24.847 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:39:24.889 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-09-21T21:39:24.890 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-09-21T21:39:24.931 INFO:teuthology.orchestra.run.smithi122.stdout: Preparing : 1/1 2024-09-21T21:39:25.019 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : rbd-fuse-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-21T21:39:25.041 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-09-21T21:39:25.149 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : rbd-nbd-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-21T21:39:25.386 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: rbd-fuse-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-21T21:39:25.572 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: rbd-nbd-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-21T21:39:25.661 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : rbd-fuse-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-21T21:39:25.661 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:39:25.662 INFO:teuthology.orchestra.run.smithi122.stdout:Installed: 2024-09-21T21:39:25.662 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-fuse-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:39:25.662 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:39:25.662 INFO:teuthology.orchestra.run.smithi122.stdout:Complete! 2024-09-21T21:39:25.839 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : rbd-nbd-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-21T21:39:25.839 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:39:25.839 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-09-21T21:39:25.840 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-nbd-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:39:25.840 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:39:25.840 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-21T21:39:25.905 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum -y install rbd-mirror 2024-09-21T21:39:25.987 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : rbd-mirror-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-21T21:39:25.988 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:39:25.988 INFO:teuthology.orchestra.run.smithi081.stdout:Installed: 2024-09-21T21:39:25.988 INFO:teuthology.orchestra.run.smithi081.stdout: rbd-mirror-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:39:25.988 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:39:25.988 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-09-21T21:39:26.085 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install rbd-nbd 2024-09-21T21:39:26.233 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum -y install rbd-nbd 2024-09-21T21:39:26.515 INFO:teuthology.orchestra.run.smithi122.stdout:Last metadata expiration check: 0:01:49 ago on Sat 21 Sep 2024 09:37:37 PM UTC. 2024-09-21T21:39:26.656 INFO:teuthology.orchestra.run.smithi122.stdout:Dependencies resolved. 2024-09-21T21:39:26.656 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-21T21:39:26.656 INFO:teuthology.orchestra.run.smithi122.stdout: Package Arch Version Repo Size 2024-09-21T21:39:26.656 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-21T21:39:26.656 INFO:teuthology.orchestra.run.smithi122.stdout:Installing: 2024-09-21T21:39:26.656 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-mirror x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 3.1 M 2024-09-21T21:39:26.656 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:39:26.656 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction Summary 2024-09-21T21:39:26.656 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-21T21:39:26.656 INFO:teuthology.orchestra.run.smithi122.stdout:Install 1 Package 2024-09-21T21:39:26.656 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:39:26.657 INFO:teuthology.orchestra.run.smithi122.stdout:Total download size: 3.1 M 2024-09-21T21:39:26.657 INFO:teuthology.orchestra.run.smithi122.stdout:Installed size: 13 M 2024-09-21T21:39:26.657 INFO:teuthology.orchestra.run.smithi122.stdout:Downloading Packages: 2024-09-21T21:39:26.685 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:01:48 ago on Sat 21 Sep 2024 09:37:38 PM UTC. 2024-09-21T21:39:26.777 INFO:teuthology.orchestra.run.smithi164.stdout:Package rbd-nbd-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-21T21:39:26.819 INFO:teuthology.orchestra.run.smithi081.stdout:Last metadata expiration check: 0:01:50 ago on Sat 21 Sep 2024 09:37:36 PM UTC. 2024-09-21T21:39:26.823 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-21T21:39:26.824 INFO:teuthology.orchestra.run.smithi164.stdout:Nothing to do. 2024-09-21T21:39:26.824 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-21T21:39:26.893 DEBUG:teuthology.parallel:result is None 2024-09-21T21:39:26.952 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-09-21T21:39:26.954 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-09-21T21:39:26.954 INFO:teuthology.orchestra.run.smithi081.stdout: Package Architecture Version Repository Size 2024-09-21T21:39:26.954 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-09-21T21:39:26.954 INFO:teuthology.orchestra.run.smithi081.stdout:Installing: 2024-09-21T21:39:26.954 INFO:teuthology.orchestra.run.smithi081.stdout: rbd-nbd x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 170 k 2024-09-21T21:39:26.955 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:39:26.955 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction Summary 2024-09-21T21:39:26.955 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-09-21T21:39:26.955 INFO:teuthology.orchestra.run.smithi081.stdout:Install 1 Package 2024-09-21T21:39:26.955 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:39:26.955 INFO:teuthology.orchestra.run.smithi081.stdout:Total download size: 170 k 2024-09-21T21:39:26.956 INFO:teuthology.orchestra.run.smithi081.stdout:Installed size: 494 k 2024-09-21T21:39:26.956 INFO:teuthology.orchestra.run.smithi081.stdout:Downloading Packages: 2024-09-21T21:39:27.173 INFO:teuthology.orchestra.run.smithi081.stdout:rbd-nbd-19.1.1-373.g4e45e6b4.el9.x86_64.rpm 780 kB/s | 170 kB 00:00 2024-09-21T21:39:27.174 INFO:teuthology.orchestra.run.smithi081.stdout:-------------------------------------------------------------------------------- 2024-09-21T21:39:27.174 INFO:teuthology.orchestra.run.smithi081.stdout:Total 775 kB/s | 170 kB 00:00 2024-09-21T21:39:27.174 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction check 2024-09-21T21:39:27.181 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction check succeeded. 2024-09-21T21:39:27.181 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction test 2024-09-21T21:39:27.217 INFO:teuthology.orchestra.run.smithi122.stdout:rbd-mirror-19.1.1-373.g4e45e6b4.el9.x86_64.rpm 5.5 MB/s | 3.1 MB 00:00 2024-09-21T21:39:27.218 INFO:teuthology.orchestra.run.smithi122.stdout:-------------------------------------------------------------------------------- 2024-09-21T21:39:27.219 INFO:teuthology.orchestra.run.smithi122.stdout:Total 5.5 MB/s | 3.1 MB 00:00 2024-09-21T21:39:27.219 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction check 2024-09-21T21:39:27.228 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction check succeeded. 2024-09-21T21:39:27.228 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction test 2024-09-21T21:39:27.244 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction test succeeded. 2024-09-21T21:39:27.244 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction 2024-09-21T21:39:27.309 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction test succeeded. 2024-09-21T21:39:27.309 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction 2024-09-21T21:39:27.401 INFO:teuthology.orchestra.run.smithi081.stdout: Preparing : 1/1 2024-09-21T21:39:27.511 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : rbd-nbd-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-21T21:39:27.748 INFO:teuthology.orchestra.run.smithi122.stdout: Preparing : 1/1 2024-09-21T21:39:27.819 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : rbd-mirror-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-21T21:39:27.845 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: rbd-mirror-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-21T21:39:27.845 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T21:39:27.845 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T21:39:27.845 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T21:39:27.846 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T21:39:27.846 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:39:27.945 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: rbd-nbd-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-21T21:39:28.190 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : rbd-nbd-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-21T21:39:28.191 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:39:28.191 INFO:teuthology.orchestra.run.smithi081.stdout:Installed: 2024-09-21T21:39:28.191 INFO:teuthology.orchestra.run.smithi081.stdout: rbd-nbd-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:39:28.191 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:39:28.191 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-09-21T21:39:28.432 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum -y install rbd-nbd 2024-09-21T21:39:28.978 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : rbd-mirror-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-21T21:39:28.978 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:39:28.978 INFO:teuthology.orchestra.run.smithi122.stdout:Installed: 2024-09-21T21:39:28.978 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-mirror-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:39:28.978 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:39:28.978 INFO:teuthology.orchestra.run.smithi122.stdout:Complete! 2024-09-21T21:39:29.031 INFO:teuthology.orchestra.run.smithi081.stdout:Last metadata expiration check: 0:01:53 ago on Sat 21 Sep 2024 09:37:36 PM UTC. 2024-09-21T21:39:29.121 INFO:teuthology.orchestra.run.smithi081.stdout:Package rbd-nbd-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-21T21:39:29.165 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-09-21T21:39:29.166 INFO:teuthology.orchestra.run.smithi081.stdout:Nothing to do. 2024-09-21T21:39:29.166 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-09-21T21:39:29.224 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum -y install rbd-nbd 2024-09-21T21:39:29.241 DEBUG:teuthology.parallel:result is None 2024-09-21T21:39:29.823 INFO:teuthology.orchestra.run.smithi122.stdout:Last metadata expiration check: 0:01:52 ago on Sat 21 Sep 2024 09:37:37 PM UTC. 2024-09-21T21:39:29.958 INFO:teuthology.orchestra.run.smithi122.stdout:Dependencies resolved. 2024-09-21T21:39:29.959 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-21T21:39:29.959 INFO:teuthology.orchestra.run.smithi122.stdout: Package Architecture Version Repository Size 2024-09-21T21:39:29.959 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-21T21:39:29.959 INFO:teuthology.orchestra.run.smithi122.stdout:Installing: 2024-09-21T21:39:29.960 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-nbd x86_64 2:19.1.1-373.g4e45e6b4.el9 ceph 170 k 2024-09-21T21:39:29.960 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:39:29.960 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction Summary 2024-09-21T21:39:29.960 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-09-21T21:39:29.960 INFO:teuthology.orchestra.run.smithi122.stdout:Install 1 Package 2024-09-21T21:39:29.960 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:39:29.960 INFO:teuthology.orchestra.run.smithi122.stdout:Total download size: 170 k 2024-09-21T21:39:29.960 INFO:teuthology.orchestra.run.smithi122.stdout:Installed size: 494 k 2024-09-21T21:39:29.960 INFO:teuthology.orchestra.run.smithi122.stdout:Downloading Packages: 2024-09-21T21:39:30.172 INFO:teuthology.orchestra.run.smithi122.stdout:rbd-nbd-19.1.1-373.g4e45e6b4.el9.x86_64.rpm 806 kB/s | 170 kB 00:00 2024-09-21T21:39:30.172 INFO:teuthology.orchestra.run.smithi122.stdout:-------------------------------------------------------------------------------- 2024-09-21T21:39:30.172 INFO:teuthology.orchestra.run.smithi122.stdout:Total 801 kB/s | 170 kB 00:00 2024-09-21T21:39:30.173 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction check 2024-09-21T21:39:30.180 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction check succeeded. 2024-09-21T21:39:30.180 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction test 2024-09-21T21:39:30.244 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction test succeeded. 2024-09-21T21:39:30.244 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction 2024-09-21T21:39:30.407 INFO:teuthology.orchestra.run.smithi122.stdout: Preparing : 1/1 2024-09-21T21:39:30.519 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : rbd-nbd-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-21T21:39:30.916 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: rbd-nbd-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-21T21:39:31.184 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : rbd-nbd-2:19.1.1-373.g4e45e6b4.el9.x86_64 1/1 2024-09-21T21:39:31.184 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:39:31.184 INFO:teuthology.orchestra.run.smithi122.stdout:Installed: 2024-09-21T21:39:31.184 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-nbd-2:19.1.1-373.g4e45e6b4.el9.x86_64 2024-09-21T21:39:31.184 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:39:31.184 INFO:teuthology.orchestra.run.smithi122.stdout:Complete! 2024-09-21T21:39:31.389 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum -y install rbd-nbd 2024-09-21T21:39:31.984 INFO:teuthology.orchestra.run.smithi122.stdout:Last metadata expiration check: 0:01:54 ago on Sat 21 Sep 2024 09:37:37 PM UTC. 2024-09-21T21:39:32.074 INFO:teuthology.orchestra.run.smithi122.stdout:Package rbd-nbd-2:19.1.1-373.g4e45e6b4.el9.x86_64 is already installed. 2024-09-21T21:39:32.118 INFO:teuthology.orchestra.run.smithi122.stdout:Dependencies resolved. 2024-09-21T21:39:32.119 INFO:teuthology.orchestra.run.smithi122.stdout:Nothing to do. 2024-09-21T21:39:32.119 INFO:teuthology.orchestra.run.smithi122.stdout:Complete! 2024-09-21T21:39:32.212 DEBUG:teuthology.parallel:result is None 2024-09-21T21:39:32.212 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=4e45e6b490e6cbe082cdf155fea780438ec53fc7 2024-09-21T21:39:32.356 DEBUG:teuthology.orchestra.run.smithi081:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T21:39:32.382 INFO:teuthology.orchestra.run.smithi081.stdout:19.1.1-373.g4e45e6b4.el9 2024-09-21T21:39:32.382 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-373.g4e45e6b4.el9 2024-09-21T21:39:32.383 INFO:teuthology.task.install:The correct ceph version 19.1.1-373.g4e45e6b4 is installed. 2024-09-21T21:39:32.384 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=4e45e6b490e6cbe082cdf155fea780438ec53fc7 2024-09-21T21:39:32.537 DEBUG:teuthology.orchestra.run.smithi122:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T21:39:32.558 INFO:teuthology.orchestra.run.smithi122.stdout:19.1.1-373.g4e45e6b4.el9 2024-09-21T21:39:32.558 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-373.g4e45e6b4.el9 2024-09-21T21:39:32.558 INFO:teuthology.task.install:The correct ceph version 19.1.1-373.g4e45e6b4 is installed. 2024-09-21T21:39:32.560 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=4e45e6b490e6cbe082cdf155fea780438ec53fc7 2024-09-21T21:39:32.696 DEBUG:teuthology.orchestra.run.smithi164:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T21:39:32.721 INFO:teuthology.orchestra.run.smithi164.stdout:19.1.1-373.g4e45e6b4.el9 2024-09-21T21:39:32.721 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-373.g4e45e6b4.el9 2024-09-21T21:39:32.722 INFO:teuthology.task.install:The correct ceph version 19.1.1-373.g4e45e6b4 is installed. 2024-09-21T21:39:32.723 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-21T21:39:32.724 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-09-21T21:39:32.724 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T21:39:32.754 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-21T21:39:32.754 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T21:39:32.779 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-21T21:39:32.779 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T21:39:32.808 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-21T21:39:32.808 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-09-21T21:39:32.808 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T21:39:32.836 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T21:39:32.904 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-21T21:39:32.904 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T21:39:32.929 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T21:39:32.993 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-21T21:39:32.993 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T21:39:33.023 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T21:39:33.092 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-21T21:39:33.092 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-09-21T21:39:33.092 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T21:39:33.119 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T21:39:33.186 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-21T21:39:33.186 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T21:39:33.210 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T21:39:33.275 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-21T21:39:33.276 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T21:39:33.305 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T21:39:33.372 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-21T21:39:33.372 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-09-21T21:39:33.372 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T21:39:33.398 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T21:39:33.464 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-21T21:39:33.464 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T21:39:33.489 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T21:39:33.553 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-21T21:39:33.553 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T21:39:33.580 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T21:39:33.647 INFO:teuthology.run_tasks:Running task ceph... 2024-09-21T21:39:33.729 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-21T21:39:33.729 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod 777 /var/log/ceph 2024-09-21T21:39:33.732 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 777 /var/log/ceph 2024-09-21T21:39:33.734 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 777 /var/log/ceph 2024-09-21T21:39:33.760 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-21T21:39:33.760 DEBUG:teuthology.orchestra.run.smithi081:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-21T21:39:33.798 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-21T21:39:33.800 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-21T21:39:33.826 INFO:tasks.ceph:Creating extra log directories... 2024-09-21T21:39:33.826 DEBUG:teuthology.orchestra.run.smithi081:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-21T21:39:33.863 DEBUG:teuthology.orchestra.run.smithi122:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-21T21:39:33.865 DEBUG:teuthology.orchestra.run.smithi164:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-21T21:39:33.901 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-21T21:39:33.901 INFO:tasks.ceph:config {'conf': {'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-21T21:39:33.902 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914423', 'branch': 'squid', 'description': 'rbd/device/{base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-snappy supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_nbd}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7914423', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'machine': {'cpus': 1, 'disk': 40, 'ram': 8000}, 'volumes': {'count': 4, 'size': 30}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'extra_packages': ['rbd-nbd'], 'flavor': 'default', 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 100, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mon.c', 'osd.0', 'osd.1', 'osd.2'], ['mon.b', 'mgr.x', 'osd.3', 'osd.4', 'osd.5'], ['client.0']], 'seed': 1478, 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7', 'sleep_before_teardown': 0, 'subset': '24/128', 'suite': 'rbd', 'suite_branch': 'squid', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_4e45e6b490e6cbe082cdf155fea780438ec53fc7/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7', 'targets': {'smithi081.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEp2unxGDL0tIgrSRVRZWYs2Dj4oDFhbXUI8nnboauay/bZ5RU4VT4bMkH5V1wUS4RlSgVb7vKwojuQSfrJLHaQ=', 'smithi122.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCN8WUfumi2LAuYCDR4FPmJIr5ZAPMNu1ntYwjvNH6dEtLpQqTYJHClxByBvQxSRPIgidJTHXeAhjJ5eux10pZY=', 'smithi164.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGfGGZvXLWhp2U7f+GrXbBJKqErIMJw3IcModtivEVdfJlye+hVfnyVc7TKyI6NNV1RehGjJtbAQIfi7zRe2Ug0='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'thrashosds': {'timeout': 1200}}, {'rbd_fsx': {'clients': ['client.0'], 'holebdy': 512, 'nbd': True, 'ops': 6000, 'punch_holes': True, 'readbdy': 512, 'truncbdy': 512, 'writebdy': 512}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-21_21:16:02', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2902'} 2024-09-21T21:39:33.902 DEBUG:teuthology.orchestra.run.smithi081:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-21T21:39:33.930 DEBUG:teuthology.orchestra.run.smithi122:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-21T21:39:33.932 DEBUG:teuthology.orchestra.run.smithi164:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-21T21:39:33.949 DEBUG:teuthology.orchestra.run.smithi081:> sudo install -d -m0777 -- /var/run/ceph 2024-09-21T21:39:33.985 DEBUG:teuthology.orchestra.run.smithi122:> sudo install -d -m0777 -- /var/run/ceph 2024-09-21T21:39:33.988 DEBUG:teuthology.orchestra.run.smithi164:> sudo install -d -m0777 -- /var/run/ceph 2024-09-21T21:39:34.015 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-09-21T21:39:34.015 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T21:39:34.063 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T21:39:34.063 DEBUG:teuthology.orchestra.run.smithi081:> stat /dev/vg_nvme/lv_1 2024-09-21T21:39:34.118 INFO:teuthology.orchestra.run.smithi081.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T21:39:34.119 INFO:teuthology.orchestra.run.smithi081.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T21:39:34.119 INFO:teuthology.orchestra.run.smithi081.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-09-21T21:39:34.119 INFO:teuthology.orchestra.run.smithi081.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T21:39:34.119 INFO:teuthology.orchestra.run.smithi081.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T21:39:34.119 INFO:teuthology.orchestra.run.smithi081.stdout:Access: 2024-09-21 21:39:25.086869626 +0000 2024-09-21T21:39:34.119 INFO:teuthology.orchestra.run.smithi081.stdout:Modify: 2024-09-21 21:36:54.572527603 +0000 2024-09-21T21:39:34.119 INFO:teuthology.orchestra.run.smithi081.stdout:Change: 2024-09-21 21:36:54.572527603 +0000 2024-09-21T21:39:34.119 INFO:teuthology.orchestra.run.smithi081.stdout: Birth: 2024-09-21 21:36:54.572527603 +0000 2024-09-21T21:39:34.119 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T21:39:34.181 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records in 2024-09-21T21:39:34.182 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records out 2024-09-21T21:39:34.182 INFO:teuthology.orchestra.run.smithi081.stderr:512 bytes copied, 0.000186194 s, 2.7 MB/s 2024-09-21T21:39:34.183 DEBUG:teuthology.orchestra.run.smithi081:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T21:39:34.239 DEBUG:teuthology.orchestra.run.smithi081:> stat /dev/vg_nvme/lv_2 2024-09-21T21:39:34.295 INFO:teuthology.orchestra.run.smithi081.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T21:39:34.295 INFO:teuthology.orchestra.run.smithi081.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T21:39:34.295 INFO:teuthology.orchestra.run.smithi081.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-21T21:39:34.295 INFO:teuthology.orchestra.run.smithi081.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T21:39:34.295 INFO:teuthology.orchestra.run.smithi081.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T21:39:34.295 INFO:teuthology.orchestra.run.smithi081.stdout:Access: 2024-09-21 21:39:25.087869608 +0000 2024-09-21T21:39:34.296 INFO:teuthology.orchestra.run.smithi081.stdout:Modify: 2024-09-21 21:36:54.896521600 +0000 2024-09-21T21:39:34.296 INFO:teuthology.orchestra.run.smithi081.stdout:Change: 2024-09-21 21:36:54.896521600 +0000 2024-09-21T21:39:34.296 INFO:teuthology.orchestra.run.smithi081.stdout: Birth: 2024-09-21 21:36:54.896521600 +0000 2024-09-21T21:39:34.296 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T21:39:34.357 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records in 2024-09-21T21:39:34.358 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records out 2024-09-21T21:39:34.358 INFO:teuthology.orchestra.run.smithi081.stderr:512 bytes copied, 0.000232122 s, 2.2 MB/s 2024-09-21T21:39:34.359 DEBUG:teuthology.orchestra.run.smithi081:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T21:39:34.415 DEBUG:teuthology.orchestra.run.smithi081:> stat /dev/vg_nvme/lv_3 2024-09-21T21:39:34.471 INFO:teuthology.orchestra.run.smithi081.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T21:39:34.471 INFO:teuthology.orchestra.run.smithi081.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T21:39:34.472 INFO:teuthology.orchestra.run.smithi081.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-09-21T21:39:34.472 INFO:teuthology.orchestra.run.smithi081.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T21:39:34.472 INFO:teuthology.orchestra.run.smithi081.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T21:39:34.472 INFO:teuthology.orchestra.run.smithi081.stdout:Access: 2024-09-21 21:39:25.087869608 +0000 2024-09-21T21:39:34.472 INFO:teuthology.orchestra.run.smithi081.stdout:Modify: 2024-09-21 21:36:55.211515762 +0000 2024-09-21T21:39:34.472 INFO:teuthology.orchestra.run.smithi081.stdout:Change: 2024-09-21 21:36:55.211515762 +0000 2024-09-21T21:39:34.473 INFO:teuthology.orchestra.run.smithi081.stdout: Birth: 2024-09-21 21:36:55.211515762 +0000 2024-09-21T21:39:34.473 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T21:39:34.535 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records in 2024-09-21T21:39:34.535 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records out 2024-09-21T21:39:34.535 INFO:teuthology.orchestra.run.smithi081.stderr:512 bytes copied, 0.000210055 s, 2.4 MB/s 2024-09-21T21:39:34.536 DEBUG:teuthology.orchestra.run.smithi081:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T21:39:34.593 DEBUG:teuthology.orchestra.run.smithi081:> stat /dev/vg_nvme/lv_4 2024-09-21T21:39:34.648 INFO:teuthology.orchestra.run.smithi081.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T21:39:34.649 INFO:teuthology.orchestra.run.smithi081.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T21:39:34.649 INFO:teuthology.orchestra.run.smithi081.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-09-21T21:39:34.649 INFO:teuthology.orchestra.run.smithi081.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T21:39:34.649 INFO:teuthology.orchestra.run.smithi081.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T21:39:34.649 INFO:teuthology.orchestra.run.smithi081.stdout:Access: 2024-09-21 21:39:25.087869608 +0000 2024-09-21T21:39:34.649 INFO:teuthology.orchestra.run.smithi081.stdout:Modify: 2024-09-21 21:36:55.541509646 +0000 2024-09-21T21:39:34.649 INFO:teuthology.orchestra.run.smithi081.stdout:Change: 2024-09-21 21:36:55.541509646 +0000 2024-09-21T21:39:34.649 INFO:teuthology.orchestra.run.smithi081.stdout: Birth: 2024-09-21 21:36:55.541509646 +0000 2024-09-21T21:39:34.650 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T21:39:34.711 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records in 2024-09-21T21:39:34.711 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records out 2024-09-21T21:39:34.711 INFO:teuthology.orchestra.run.smithi081.stderr:512 bytes copied, 0.000181951 s, 2.8 MB/s 2024-09-21T21:39:34.712 DEBUG:teuthology.orchestra.run.smithi081:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T21:39:34.768 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-21T21:39:34.769 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-21T21:39:34.769 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T21:39:34.786 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T21:39:34.787 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_1 2024-09-21T21:39:34.846 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T21:39:34.846 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T21:39:34.846 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-21T21:39:34.846 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T21:39:34.847 INFO:teuthology.orchestra.run.smithi122.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T21:39:34.847 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-09-21 21:39:28.086638874 +0000 2024-09-21T21:39:34.847 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-09-21 21:36:56.980104630 +0000 2024-09-21T21:39:34.847 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-09-21 21:36:56.980104630 +0000 2024-09-21T21:39:34.847 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: 2024-09-21 21:36:56.980104630 +0000 2024-09-21T21:39:34.847 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T21:39:34.912 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-09-21T21:39:34.912 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-09-21T21:39:34.912 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.000217709 s, 2.4 MB/s 2024-09-21T21:39:34.913 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T21:39:34.971 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_2 2024-09-21T21:39:35.028 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T21:39:35.028 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T21:39:35.028 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-21T21:39:35.028 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T21:39:35.029 INFO:teuthology.orchestra.run.smithi122.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T21:39:35.029 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-09-21 21:39:28.086638874 +0000 2024-09-21T21:39:35.029 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-09-21 21:36:57.329096551 +0000 2024-09-21T21:39:35.029 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-09-21 21:36:57.329096551 +0000 2024-09-21T21:39:35.029 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: 2024-09-21 21:36:57.329096551 +0000 2024-09-21T21:39:35.030 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T21:39:35.093 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-09-21T21:39:35.093 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-09-21T21:39:35.093 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.00020236 s, 2.5 MB/s 2024-09-21T21:39:35.094 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T21:39:35.151 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_3 2024-09-21T21:39:35.207 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T21:39:35.207 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T21:39:35.207 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-09-21T21:39:35.207 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T21:39:35.208 INFO:teuthology.orchestra.run.smithi122.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T21:39:35.208 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-09-21 21:39:28.087638851 +0000 2024-09-21T21:39:35.208 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-09-21 21:36:57.655089005 +0000 2024-09-21T21:39:35.208 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-09-21 21:36:57.655089005 +0000 2024-09-21T21:39:35.208 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: 2024-09-21 21:36:57.655089005 +0000 2024-09-21T21:39:35.208 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T21:39:35.272 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-09-21T21:39:35.272 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-09-21T21:39:35.272 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.000203614 s, 2.5 MB/s 2024-09-21T21:39:35.274 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T21:39:35.330 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_4 2024-09-21T21:39:35.385 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T21:39:35.385 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T21:39:35.385 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-09-21T21:39:35.385 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T21:39:35.385 INFO:teuthology.orchestra.run.smithi122.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T21:39:35.385 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-09-21 21:39:28.087638851 +0000 2024-09-21T21:39:35.385 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-09-21 21:36:57.979081505 +0000 2024-09-21T21:39:35.385 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-09-21 21:36:57.979081505 +0000 2024-09-21T21:39:35.386 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: 2024-09-21 21:36:57.979081505 +0000 2024-09-21T21:39:35.386 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T21:39:35.448 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-09-21T21:39:35.449 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-09-21T21:39:35.449 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.000171889 s, 3.0 MB/s 2024-09-21T21:39:35.449 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T21:39:35.505 INFO:tasks.ceph:osd dev map: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-09-21T21:39:35.505 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi081.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}, Remote(name='ubuntu@smithi122.front.sepia.ceph.com'): {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'}} 2024-09-21T21:39:35.505 INFO:tasks.ceph:Generating config... 2024-09-21T21:39:35.506 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-09-21T21:39:35.507 INFO:tasks.ceph:[global] mon warn on pool no app = False 2024-09-21T21:39:35.507 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-09-21T21:39:35.507 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-21T21:39:35.507 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-21T21:39:35.507 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-21T21:39:35.507 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-21T21:39:35.507 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-21T21:39:35.507 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-21T21:39:35.507 INFO:tasks.ceph:[osd] bluestore compression algorithm = snappy 2024-09-21T21:39:35.507 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2024-09-21T21:39:35.507 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-21T21:39:35.507 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-21T21:39:35.507 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-21T21:39:35.508 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-21T21:39:35.508 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-21T21:39:35.508 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-21T21:39:35.508 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-21T21:39:35.508 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-21T21:39:35.508 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-21T21:39:35.508 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-09-21T21:39:35.508 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-21T21:39:35.508 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-09-21T21:39:35.508 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-21T21:39:35.508 INFO:tasks.ceph:Setting up mon.a... 2024-09-21T21:39:35.508 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-21T21:39:35.560 INFO:teuthology.orchestra.run.smithi081.stdout:creating /etc/ceph/ceph.keyring 2024-09-21T21:39:35.562 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-21T21:39:35.604 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-21T21:39:35.668 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.81'), ('mon.c', '[v2:172.21.15.81:3301,v1:172.21.15.81:6790]'), ('mon.b', '172.21.15.122')] 2024-09-21T21:39:35.668 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.81,[v2:172.21.15.81:3301,v1:172.21.15.81:6790],172.21.15.122', 'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2024-09-21T21:39:35.669 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-09-21T21:39:35.669 DEBUG:teuthology.orchestra.run.smithi081:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-21T21:39:35.723 DEBUG:teuthology.orchestra.run.smithi081:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.81 --addv c '[v2:172.21.15.81:3301,v1:172.21.15.81:6790]' --add b 172.21.15.122 --print /home/ubuntu/cephtest/ceph.monmap 2024-09-21T21:39:35.808 INFO:teuthology.orchestra.run.smithi081.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-21T21:39:35.808 INFO:teuthology.orchestra.run.smithi081.stdout:monmaptool: generated fsid a0fdcfc2-ddd2-4e50-81bb-876d940f3578 2024-09-21T21:39:35.809 INFO:teuthology.orchestra.run.smithi081.stdout:setting min_mon_release = quincy 2024-09-21T21:39:35.809 INFO:teuthology.orchestra.run.smithi081.stdout:epoch 0 2024-09-21T21:39:35.809 INFO:teuthology.orchestra.run.smithi081.stdout:fsid a0fdcfc2-ddd2-4e50-81bb-876d940f3578 2024-09-21T21:39:35.809 INFO:teuthology.orchestra.run.smithi081.stdout:last_changed 2024-09-21T21:39:35.807663+0000 2024-09-21T21:39:35.809 INFO:teuthology.orchestra.run.smithi081.stdout:created 2024-09-21T21:39:35.807663+0000 2024-09-21T21:39:35.809 INFO:teuthology.orchestra.run.smithi081.stdout:min_mon_release 17 (quincy) 2024-09-21T21:39:35.810 INFO:teuthology.orchestra.run.smithi081.stdout:election_strategy: 1 2024-09-21T21:39:35.810 INFO:teuthology.orchestra.run.smithi081.stdout:0: [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] mon.a 2024-09-21T21:39:35.810 INFO:teuthology.orchestra.run.smithi081.stdout:1: [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] mon.b 2024-09-21T21:39:35.810 INFO:teuthology.orchestra.run.smithi081.stdout:2: [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] mon.c 2024-09-21T21:39:35.810 INFO:teuthology.orchestra.run.smithi081.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-21T21:39:35.812 DEBUG:teuthology.orchestra.run.smithi081:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-21T21:39:35.827 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID a0fdcfc2-ddd2-4e50-81bb-876d940f3578... 2024-09-21T21:39:35.828 DEBUG:teuthology.orchestra.run.smithi081:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-21T21:39:35.870 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-21T21:39:35.872 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-21T21:39:35.910 INFO:teuthology.orchestra.run.smithi081.stdout:[global] 2024-09-21T21:39:35.910 INFO:teuthology.orchestra.run.smithi081.stdout: chdir = "" 2024-09-21T21:39:35.910 INFO:teuthology.orchestra.run.smithi081.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-21T21:39:35.910 INFO:teuthology.orchestra.run.smithi081.stdout: auth supported = cephx 2024-09-21T21:39:35.910 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:39:35.910 INFO:teuthology.orchestra.run.smithi081.stdout: filestore xattr use omap = true 2024-09-21T21:39:35.910 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:39:35.910 INFO:teuthology.orchestra.run.smithi081.stdout: mon clock drift allowed = 1.000 2024-09-21T21:39:35.910 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:39:35.911 INFO:teuthology.orchestra.run.smithi081.stdout: osd crush chooseleaf type = 0 2024-09-21T21:39:35.911 INFO:teuthology.orchestra.run.smithi081.stdout: auth debug = true 2024-09-21T21:39:35.911 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:39:35.911 INFO:teuthology.orchestra.run.smithi081.stdout: ms die on old message = true 2024-09-21T21:39:35.911 INFO:teuthology.orchestra.run.smithi081.stdout: ms die on bug = true 2024-09-21T21:39:35.911 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:39:35.911 INFO:teuthology.orchestra.run.smithi081.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-21T21:39:35.911 INFO:teuthology.orchestra.run.smithi081.stdout: mon pg warn max object skew = 0 2024-09-21T21:39:35.911 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:39:35.911 INFO:teuthology.orchestra.run.smithi081.stdout: # disable pg_autoscaler by default for new pools 2024-09-21T21:39:35.911 INFO:teuthology.orchestra.run.smithi081.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-21T21:39:35.911 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:39:35.911 INFO:teuthology.orchestra.run.smithi081.stdout: osd pool default size = 2 2024-09-21T21:39:35.911 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:39:35.911 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd allow primary affinity = true 2024-09-21T21:39:35.911 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd allow pg remap = true 2024-09-21T21:39:35.912 INFO:teuthology.orchestra.run.smithi081.stdout: mon warn on legacy crush tunables = false 2024-09-21T21:39:35.912 INFO:teuthology.orchestra.run.smithi081.stdout: mon warn on crush straw calc version zero = false 2024-09-21T21:39:35.912 INFO:teuthology.orchestra.run.smithi081.stdout: mon warn on no sortbitwise = false 2024-09-21T21:39:35.912 INFO:teuthology.orchestra.run.smithi081.stdout: mon warn on osd down out interval zero = false 2024-09-21T21:39:35.912 INFO:teuthology.orchestra.run.smithi081.stdout: mon warn on too few osds = false 2024-09-21T21:39:35.912 INFO:teuthology.orchestra.run.smithi081.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-21T21:39:35.912 INFO:teuthology.orchestra.run.smithi081.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-21T21:39:35.912 INFO:teuthology.orchestra.run.smithi081.stdout: mon_allow_pool_size_one = true 2024-09-21T21:39:35.912 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:39:35.912 INFO:teuthology.orchestra.run.smithi081.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-21T21:39:35.912 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:39:35.912 INFO:teuthology.orchestra.run.smithi081.stdout: osd default data pool replay window = 5 2024-09-21T21:39:35.912 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:39:35.912 INFO:teuthology.orchestra.run.smithi081.stdout: mon allow pool delete = true 2024-09-21T21:39:35.913 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:39:35.913 INFO:teuthology.orchestra.run.smithi081.stdout: mon cluster log file level = debug 2024-09-21T21:39:35.913 INFO:teuthology.orchestra.run.smithi081.stdout: debug asserts on shutdown = true 2024-09-21T21:39:35.913 INFO:teuthology.orchestra.run.smithi081.stdout: mon health detail to clog = false 2024-09-21T21:39:35.913 INFO:teuthology.orchestra.run.smithi081.stdout: mon host = "172.21.15.81,[v2:172.21.15.81:3301,v1:172.21.15.81:6790],172.21.15.122" 2024-09-21T21:39:35.913 INFO:teuthology.orchestra.run.smithi081.stdout: mon client directed command retry = 5 2024-09-21T21:39:35.913 INFO:teuthology.orchestra.run.smithi081.stdout: mon warn on pool no app = False 2024-09-21T21:39:35.913 INFO:teuthology.orchestra.run.smithi081.stdout: ms inject socket failures = 5000 2024-09-21T21:39:35.913 INFO:teuthology.orchestra.run.smithi081.stdout: fsid = a0fdcfc2-ddd2-4e50-81bb-876d940f3578 2024-09-21T21:39:35.913 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:39:35.913 INFO:teuthology.orchestra.run.smithi081.stdout:[osd] 2024-09-21T21:39:35.913 INFO:teuthology.orchestra.run.smithi081.stdout: osd journal size = 100 2024-09-21T21:39:35.913 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:39:35.913 INFO:teuthology.orchestra.run.smithi081.stdout: osd scrub load threshold = 5.0 2024-09-21T21:39:35.913 INFO:teuthology.orchestra.run.smithi081.stdout: osd scrub max interval = 600 2024-09-21T21:39:35.914 INFO:teuthology.orchestra.run.smithi081.stdout: osd mclock profile = high_recovery_ops 2024-09-21T21:39:35.914 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:39:35.914 INFO:teuthology.orchestra.run.smithi081.stdout: osd recover clone overlap = true 2024-09-21T21:39:35.914 INFO:teuthology.orchestra.run.smithi081.stdout: osd recovery max chunk = 1048576 2024-09-21T21:39:35.914 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:39:35.914 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug shutdown = true 2024-09-21T21:39:35.914 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug op order = true 2024-09-21T21:39:35.914 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug verify stray on activate = true 2024-09-21T21:39:35.914 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug trim objects = true 2024-09-21T21:39:35.914 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:39:35.914 INFO:teuthology.orchestra.run.smithi081.stdout: osd open classes on start = true 2024-09-21T21:39:35.914 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug pg log writeout = true 2024-09-21T21:39:35.914 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:39:35.914 INFO:teuthology.orchestra.run.smithi081.stdout: osd deep scrub update digest min age = 30 2024-09-21T21:39:35.914 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:39:35.914 INFO:teuthology.orchestra.run.smithi081.stdout: osd map max advance = 10 2024-09-21T21:39:35.915 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:39:35.915 INFO:teuthology.orchestra.run.smithi081.stdout: journal zero on create = true 2024-09-21T21:39:35.915 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:39:35.915 INFO:teuthology.orchestra.run.smithi081.stdout: filestore ondisk finisher threads = 3 2024-09-21T21:39:35.915 INFO:teuthology.orchestra.run.smithi081.stdout: filestore apply finisher threads = 3 2024-09-21T21:39:35.915 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:39:35.915 INFO:teuthology.orchestra.run.smithi081.stdout: bdev debug aio = true 2024-09-21T21:39:35.915 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug misdirected ops = true 2024-09-21T21:39:35.915 INFO:teuthology.orchestra.run.smithi081.stdout: bluestore block size = 96636764160 2024-09-21T21:39:35.915 INFO:teuthology.orchestra.run.smithi081.stdout: bluestore compression algorithm = snappy 2024-09-21T21:39:35.915 INFO:teuthology.orchestra.run.smithi081.stdout: bluestore compression mode = aggressive 2024-09-21T21:39:35.915 INFO:teuthology.orchestra.run.smithi081.stdout: bluestore fsck on mount = True 2024-09-21T21:39:35.915 INFO:teuthology.orchestra.run.smithi081.stdout: debug bluefs = 1/20 2024-09-21T21:39:35.915 INFO:teuthology.orchestra.run.smithi081.stdout: debug bluestore = 1/20 2024-09-21T21:39:35.915 INFO:teuthology.orchestra.run.smithi081.stdout: debug ms = 1 2024-09-21T21:39:35.915 INFO:teuthology.orchestra.run.smithi081.stdout: debug osd = 20 2024-09-21T21:39:35.916 INFO:teuthology.orchestra.run.smithi081.stdout: debug rocksdb = 4/10 2024-09-21T21:39:35.916 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd backfillfull_ratio = 0.85 2024-09-21T21:39:35.916 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd full ratio = 0.9 2024-09-21T21:39:35.916 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd nearfull ratio = 0.8 2024-09-21T21:39:35.916 INFO:teuthology.orchestra.run.smithi081.stdout: osd blocked scrub grace period = 3600 2024-09-21T21:39:35.916 INFO:teuthology.orchestra.run.smithi081.stdout: osd failsafe full ratio = 0.95 2024-09-21T21:39:35.916 INFO:teuthology.orchestra.run.smithi081.stdout: osd max markdown count = 1000 2024-09-21T21:39:35.916 INFO:teuthology.orchestra.run.smithi081.stdout: osd objectstore = bluestore 2024-09-21T21:39:35.916 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:39:35.916 INFO:teuthology.orchestra.run.smithi081.stdout:[mgr] 2024-09-21T21:39:35.916 INFO:teuthology.orchestra.run.smithi081.stdout: debug ms = 1 2024-09-21T21:39:35.917 INFO:teuthology.orchestra.run.smithi081.stdout: debug mgr = 20 2024-09-21T21:39:35.917 INFO:teuthology.orchestra.run.smithi081.stdout: debug mon = 20 2024-09-21T21:39:35.917 INFO:teuthology.orchestra.run.smithi081.stdout: debug auth = 20 2024-09-21T21:39:35.917 INFO:teuthology.orchestra.run.smithi081.stdout: mon reweight min pgs per osd = 4 2024-09-21T21:39:35.917 INFO:teuthology.orchestra.run.smithi081.stdout: mon reweight min bytes per osd = 10 2024-09-21T21:39:35.917 INFO:teuthology.orchestra.run.smithi081.stdout: mgr/telemetry/nag = false 2024-09-21T21:39:35.918 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:39:35.918 INFO:teuthology.orchestra.run.smithi081.stdout:[mon] 2024-09-21T21:39:35.918 INFO:teuthology.orchestra.run.smithi081.stdout: debug ms = 1 2024-09-21T21:39:35.918 INFO:teuthology.orchestra.run.smithi081.stdout: debug mon = 20 2024-09-21T21:39:35.918 INFO:teuthology.orchestra.run.smithi081.stdout: debug paxos = 20 2024-09-21T21:39:35.918 INFO:teuthology.orchestra.run.smithi081.stdout: debug auth = 20 2024-09-21T21:39:35.919 INFO:teuthology.orchestra.run.smithi081.stdout: mon data avail warn = 5 2024-09-21T21:39:35.919 INFO:teuthology.orchestra.run.smithi081.stdout: mon mgr mkfs grace = 240 2024-09-21T21:39:35.919 INFO:teuthology.orchestra.run.smithi081.stdout: mon reweight min pgs per osd = 4 2024-09-21T21:39:35.919 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd reporter subtree level = osd 2024-09-21T21:39:35.919 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd prime pg temp = true 2024-09-21T21:39:35.919 INFO:teuthology.orchestra.run.smithi081.stdout: mon reweight min bytes per osd = 10 2024-09-21T21:39:35.920 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:39:35.920 INFO:teuthology.orchestra.run.smithi081.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-21T21:39:35.920 INFO:teuthology.orchestra.run.smithi081.stdout: auth mon ticket ttl = 660 # 11m 2024-09-21T21:39:35.920 INFO:teuthology.orchestra.run.smithi081.stdout: auth service ticket ttl = 240 # 4m 2024-09-21T21:39:35.920 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:39:35.920 INFO:teuthology.orchestra.run.smithi081.stdout: # don't complain about insecure global_id in the test suite 2024-09-21T21:39:35.921 INFO:teuthology.orchestra.run.smithi081.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-21T21:39:35.921 INFO:teuthology.orchestra.run.smithi081.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-21T21:39:35.921 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:39:35.921 INFO:teuthology.orchestra.run.smithi081.stdout: # 1m isn't quite enough 2024-09-21T21:39:35.921 INFO:teuthology.orchestra.run.smithi081.stdout: mon_down_mkfs_grace = 2m 2024-09-21T21:39:35.921 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:39:35.921 INFO:teuthology.orchestra.run.smithi081.stdout: mon_warn_on_filestore_osds = false 2024-09-21T21:39:35.921 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:39:35.921 INFO:teuthology.orchestra.run.smithi081.stdout:[client] 2024-09-21T21:39:35.921 INFO:teuthology.orchestra.run.smithi081.stdout: rgw cache enabled = true 2024-09-21T21:39:35.921 INFO:teuthology.orchestra.run.smithi081.stdout: rgw enable ops log = true 2024-09-21T21:39:35.921 INFO:teuthology.orchestra.run.smithi081.stdout: rgw enable usage log = true 2024-09-21T21:39:35.922 INFO:teuthology.orchestra.run.smithi081.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-21T21:39:35.922 INFO:teuthology.orchestra.run.smithi081.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-21T21:39:35.922 INFO:teuthology.orchestra.run.smithi081.stdout:[mon.a] 2024-09-21T21:39:35.922 INFO:teuthology.orchestra.run.smithi081.stdout:[mon.c] 2024-09-21T21:39:35.922 INFO:teuthology.orchestra.run.smithi081.stdout:[mon.b] 2024-09-21T21:39:35.922 INFO:teuthology.orchestra.run.smithi122.stdout:[global] 2024-09-21T21:39:35.922 INFO:teuthology.orchestra.run.smithi122.stdout: chdir = "" 2024-09-21T21:39:35.923 INFO:teuthology.orchestra.run.smithi122.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-21T21:39:35.923 INFO:teuthology.orchestra.run.smithi122.stdout: auth supported = cephx 2024-09-21T21:39:35.923 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:39:35.923 INFO:teuthology.orchestra.run.smithi122.stdout: filestore xattr use omap = true 2024-09-21T21:39:35.923 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:39:35.923 INFO:teuthology.orchestra.run.smithi122.stdout: mon clock drift allowed = 1.000 2024-09-21T21:39:35.923 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:39:35.923 INFO:teuthology.orchestra.run.smithi122.stdout: osd crush chooseleaf type = 0 2024-09-21T21:39:35.923 INFO:teuthology.orchestra.run.smithi122.stdout: auth debug = true 2024-09-21T21:39:35.923 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:39:35.923 INFO:teuthology.orchestra.run.smithi122.stdout: ms die on old message = true 2024-09-21T21:39:35.923 INFO:teuthology.orchestra.run.smithi122.stdout: ms die on bug = true 2024-09-21T21:39:35.924 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:39:35.924 INFO:teuthology.orchestra.run.smithi122.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-21T21:39:35.924 INFO:teuthology.orchestra.run.smithi122.stdout: mon pg warn max object skew = 0 2024-09-21T21:39:35.924 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:39:35.924 INFO:teuthology.orchestra.run.smithi122.stdout: # disable pg_autoscaler by default for new pools 2024-09-21T21:39:35.924 INFO:teuthology.orchestra.run.smithi122.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-21T21:39:35.924 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:39:35.925 INFO:teuthology.orchestra.run.smithi122.stdout: osd pool default size = 2 2024-09-21T21:39:35.925 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:39:35.925 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd allow primary affinity = true 2024-09-21T21:39:35.925 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd allow pg remap = true 2024-09-21T21:39:35.925 INFO:teuthology.orchestra.run.smithi122.stdout: mon warn on legacy crush tunables = false 2024-09-21T21:39:35.925 INFO:teuthology.orchestra.run.smithi122.stdout: mon warn on crush straw calc version zero = false 2024-09-21T21:39:35.925 INFO:teuthology.orchestra.run.smithi122.stdout: mon warn on no sortbitwise = false 2024-09-21T21:39:35.926 INFO:teuthology.orchestra.run.smithi122.stdout: mon warn on osd down out interval zero = false 2024-09-21T21:39:35.926 INFO:teuthology.orchestra.run.smithi122.stdout: mon warn on too few osds = false 2024-09-21T21:39:35.926 INFO:teuthology.orchestra.run.smithi122.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-21T21:39:35.926 INFO:teuthology.orchestra.run.smithi122.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-21T21:39:35.926 INFO:teuthology.orchestra.run.smithi122.stdout: mon_allow_pool_size_one = true 2024-09-21T21:39:35.926 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:39:35.927 INFO:teuthology.orchestra.run.smithi122.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-21T21:39:35.927 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:39:35.927 INFO:teuthology.orchestra.run.smithi122.stdout: osd default data pool replay window = 5 2024-09-21T21:39:35.927 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:39:35.927 INFO:teuthology.orchestra.run.smithi122.stdout: mon allow pool delete = true 2024-09-21T21:39:35.927 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:39:35.927 INFO:teuthology.orchestra.run.smithi122.stdout: mon cluster log file level = debug 2024-09-21T21:39:35.927 INFO:teuthology.orchestra.run.smithi122.stdout: debug asserts on shutdown = true 2024-09-21T21:39:35.928 INFO:teuthology.orchestra.run.smithi122.stdout: mon health detail to clog = false 2024-09-21T21:39:35.928 INFO:teuthology.orchestra.run.smithi122.stdout: mon host = "172.21.15.81,[v2:172.21.15.81:3301,v1:172.21.15.81:6790],172.21.15.122" 2024-09-21T21:39:35.928 INFO:teuthology.orchestra.run.smithi122.stdout: mon client directed command retry = 5 2024-09-21T21:39:35.928 INFO:teuthology.orchestra.run.smithi122.stdout: mon warn on pool no app = False 2024-09-21T21:39:35.928 INFO:teuthology.orchestra.run.smithi122.stdout: ms inject socket failures = 5000 2024-09-21T21:39:35.928 INFO:teuthology.orchestra.run.smithi122.stdout: fsid = a0fdcfc2-ddd2-4e50-81bb-876d940f3578 2024-09-21T21:39:35.928 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:39:35.928 INFO:teuthology.orchestra.run.smithi122.stdout:[osd] 2024-09-21T21:39:35.928 INFO:teuthology.orchestra.run.smithi122.stdout: osd journal size = 100 2024-09-21T21:39:35.928 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:39:35.928 INFO:teuthology.orchestra.run.smithi122.stdout: osd scrub load threshold = 5.0 2024-09-21T21:39:35.928 INFO:teuthology.orchestra.run.smithi122.stdout: osd scrub max interval = 600 2024-09-21T21:39:35.928 INFO:teuthology.orchestra.run.smithi122.stdout: osd mclock profile = high_recovery_ops 2024-09-21T21:39:35.928 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:39:35.928 INFO:teuthology.orchestra.run.smithi122.stdout: osd recover clone overlap = true 2024-09-21T21:39:35.928 INFO:teuthology.orchestra.run.smithi122.stdout: osd recovery max chunk = 1048576 2024-09-21T21:39:35.929 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:39:35.929 INFO:teuthology.orchestra.run.smithi122.stdout: osd debug shutdown = true 2024-09-21T21:39:35.929 INFO:teuthology.orchestra.run.smithi122.stdout: osd debug op order = true 2024-09-21T21:39:35.929 INFO:teuthology.orchestra.run.smithi122.stdout: osd debug verify stray on activate = true 2024-09-21T21:39:35.929 INFO:teuthology.orchestra.run.smithi122.stdout: osd debug trim objects = true 2024-09-21T21:39:35.929 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:39:35.929 INFO:teuthology.orchestra.run.smithi122.stdout: osd open classes on start = true 2024-09-21T21:39:35.929 INFO:teuthology.orchestra.run.smithi122.stdout: osd debug pg log writeout = true 2024-09-21T21:39:35.929 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:39:35.929 INFO:teuthology.orchestra.run.smithi122.stdout: osd deep scrub update digest min age = 30 2024-09-21T21:39:35.929 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:39:35.929 INFO:teuthology.orchestra.run.smithi122.stdout: osd map max advance = 10 2024-09-21T21:39:35.929 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:39:35.929 INFO:teuthology.orchestra.run.smithi122.stdout: journal zero on create = true 2024-09-21T21:39:35.929 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:39:35.929 INFO:teuthology.orchestra.run.smithi122.stdout: filestore ondisk finisher threads = 3 2024-09-21T21:39:35.930 INFO:teuthology.orchestra.run.smithi122.stdout: filestore apply finisher threads = 3 2024-09-21T21:39:35.930 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:39:35.930 INFO:teuthology.orchestra.run.smithi122.stdout: bdev debug aio = true 2024-09-21T21:39:35.930 INFO:teuthology.orchestra.run.smithi122.stdout: osd debug misdirected ops = true 2024-09-21T21:39:35.931 INFO:teuthology.orchestra.run.smithi122.stdout: bluestore block size = 96636764160 2024-09-21T21:39:35.931 INFO:teuthology.orchestra.run.smithi122.stdout: bluestore compression algorithm = snappy 2024-09-21T21:39:35.931 INFO:teuthology.orchestra.run.smithi122.stdout: bluestore compression mode = aggressive 2024-09-21T21:39:35.931 INFO:teuthology.orchestra.run.smithi122.stdout: bluestore fsck on mount = True 2024-09-21T21:39:35.931 INFO:teuthology.orchestra.run.smithi122.stdout: debug bluefs = 1/20 2024-09-21T21:39:35.931 INFO:teuthology.orchestra.run.smithi122.stdout: debug bluestore = 1/20 2024-09-21T21:39:35.931 INFO:teuthology.orchestra.run.smithi122.stdout: debug ms = 1 2024-09-21T21:39:35.931 INFO:teuthology.orchestra.run.smithi122.stdout: debug osd = 20 2024-09-21T21:39:35.931 INFO:teuthology.orchestra.run.smithi122.stdout: debug rocksdb = 4/10 2024-09-21T21:39:35.931 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd backfillfull_ratio = 0.85 2024-09-21T21:39:35.931 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd full ratio = 0.9 2024-09-21T21:39:35.931 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd nearfull ratio = 0.8 2024-09-21T21:39:35.932 INFO:teuthology.orchestra.run.smithi122.stdout: osd blocked scrub grace period = 3600 2024-09-21T21:39:35.932 INFO:teuthology.orchestra.run.smithi122.stdout: osd failsafe full ratio = 0.95 2024-09-21T21:39:35.932 INFO:teuthology.orchestra.run.smithi122.stdout: osd max markdown count = 1000 2024-09-21T21:39:35.932 INFO:teuthology.orchestra.run.smithi122.stdout: osd objectstore = bluestore 2024-09-21T21:39:35.932 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:39:35.932 INFO:teuthology.orchestra.run.smithi122.stdout:[mgr] 2024-09-21T21:39:35.932 INFO:teuthology.orchestra.run.smithi122.stdout: debug ms = 1 2024-09-21T21:39:35.932 INFO:teuthology.orchestra.run.smithi122.stdout: debug mgr = 20 2024-09-21T21:39:35.932 INFO:teuthology.orchestra.run.smithi122.stdout: debug mon = 20 2024-09-21T21:39:35.932 INFO:teuthology.orchestra.run.smithi122.stdout: debug auth = 20 2024-09-21T21:39:35.932 INFO:teuthology.orchestra.run.smithi122.stdout: mon reweight min pgs per osd = 4 2024-09-21T21:39:35.932 INFO:teuthology.orchestra.run.smithi122.stdout: mon reweight min bytes per osd = 10 2024-09-21T21:39:35.932 INFO:teuthology.orchestra.run.smithi122.stdout: mgr/telemetry/nag = false 2024-09-21T21:39:35.932 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:39:35.932 INFO:teuthology.orchestra.run.smithi122.stdout:[mon] 2024-09-21T21:39:35.932 INFO:teuthology.orchestra.run.smithi122.stdout: debug ms = 1 2024-09-21T21:39:35.933 INFO:teuthology.orchestra.run.smithi122.stdout: debug mon = 20 2024-09-21T21:39:35.933 INFO:teuthology.orchestra.run.smithi122.stdout: debug paxos = 20 2024-09-21T21:39:35.933 INFO:teuthology.orchestra.run.smithi122.stdout: debug auth = 20 2024-09-21T21:39:35.933 INFO:teuthology.orchestra.run.smithi122.stdout: mon data avail warn = 5 2024-09-21T21:39:35.933 INFO:teuthology.orchestra.run.smithi122.stdout: mon mgr mkfs grace = 240 2024-09-21T21:39:35.933 INFO:teuthology.orchestra.run.smithi122.stdout: mon reweight min pgs per osd = 4 2024-09-21T21:39:35.933 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd reporter subtree level = osd 2024-09-21T21:39:35.933 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd prime pg temp = true 2024-09-21T21:39:35.933 INFO:teuthology.orchestra.run.smithi122.stdout: mon reweight min bytes per osd = 10 2024-09-21T21:39:35.933 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:39:35.933 INFO:teuthology.orchestra.run.smithi122.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-21T21:39:35.934 INFO:teuthology.orchestra.run.smithi122.stdout: auth mon ticket ttl = 660 # 11m 2024-09-21T21:39:35.934 INFO:teuthology.orchestra.run.smithi122.stdout: auth service ticket ttl = 240 # 4m 2024-09-21T21:39:35.934 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:39:35.934 INFO:teuthology.orchestra.run.smithi122.stdout: # don't complain about insecure global_id in the test suite 2024-09-21T21:39:35.934 INFO:teuthology.orchestra.run.smithi122.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-21T21:39:35.934 INFO:teuthology.orchestra.run.smithi122.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-21T21:39:35.934 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:39:35.934 INFO:teuthology.orchestra.run.smithi122.stdout: # 1m isn't quite enough 2024-09-21T21:39:35.934 INFO:teuthology.orchestra.run.smithi122.stdout: mon_down_mkfs_grace = 2m 2024-09-21T21:39:35.934 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:39:35.934 INFO:teuthology.orchestra.run.smithi122.stdout: mon_warn_on_filestore_osds = false 2024-09-21T21:39:35.934 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-21T21:39:35.935 INFO:teuthology.orchestra.run.smithi122.stdout:[client] 2024-09-21T21:39:35.935 INFO:teuthology.orchestra.run.smithi122.stdout: rgw cache enabled = true 2024-09-21T21:39:35.935 INFO:teuthology.orchestra.run.smithi122.stdout: rgw enable ops log = true 2024-09-21T21:39:35.935 INFO:teuthology.orchestra.run.smithi122.stdout: rgw enable usage log = true 2024-09-21T21:39:35.935 INFO:teuthology.orchestra.run.smithi122.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-21T21:39:35.935 INFO:teuthology.orchestra.run.smithi122.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-21T21:39:35.935 INFO:teuthology.orchestra.run.smithi122.stdout:[mon.a] 2024-09-21T21:39:35.935 INFO:teuthology.orchestra.run.smithi122.stdout:[mon.c] 2024-09-21T21:39:35.935 INFO:teuthology.orchestra.run.smithi122.stdout:[mon.b] 2024-09-21T21:39:35.937 INFO:teuthology.orchestra.run.smithi164.stdout:[global] 2024-09-21T21:39:35.937 INFO:teuthology.orchestra.run.smithi164.stdout: chdir = "" 2024-09-21T21:39:35.938 INFO:teuthology.orchestra.run.smithi164.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-21T21:39:35.938 INFO:teuthology.orchestra.run.smithi164.stdout: auth supported = cephx 2024-09-21T21:39:35.938 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:39:35.938 INFO:teuthology.orchestra.run.smithi164.stdout: filestore xattr use omap = true 2024-09-21T21:39:35.938 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:39:35.938 INFO:teuthology.orchestra.run.smithi164.stdout: mon clock drift allowed = 1.000 2024-09-21T21:39:35.938 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:39:35.938 INFO:teuthology.orchestra.run.smithi164.stdout: osd crush chooseleaf type = 0 2024-09-21T21:39:35.938 INFO:teuthology.orchestra.run.smithi164.stdout: auth debug = true 2024-09-21T21:39:35.938 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:39:35.938 INFO:teuthology.orchestra.run.smithi164.stdout: ms die on old message = true 2024-09-21T21:39:35.939 INFO:teuthology.orchestra.run.smithi164.stdout: ms die on bug = true 2024-09-21T21:39:35.939 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:39:35.939 INFO:teuthology.orchestra.run.smithi164.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-21T21:39:35.939 INFO:teuthology.orchestra.run.smithi164.stdout: mon pg warn max object skew = 0 2024-09-21T21:39:35.939 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:39:35.939 INFO:teuthology.orchestra.run.smithi164.stdout: # disable pg_autoscaler by default for new pools 2024-09-21T21:39:35.939 INFO:teuthology.orchestra.run.smithi164.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-21T21:39:35.939 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:39:35.939 INFO:teuthology.orchestra.run.smithi164.stdout: osd pool default size = 2 2024-09-21T21:39:35.939 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:39:35.939 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd allow primary affinity = true 2024-09-21T21:39:35.939 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd allow pg remap = true 2024-09-21T21:39:35.939 INFO:teuthology.orchestra.run.smithi164.stdout: mon warn on legacy crush tunables = false 2024-09-21T21:39:35.939 INFO:teuthology.orchestra.run.smithi164.stdout: mon warn on crush straw calc version zero = false 2024-09-21T21:39:35.940 INFO:teuthology.orchestra.run.smithi164.stdout: mon warn on no sortbitwise = false 2024-09-21T21:39:35.940 INFO:teuthology.orchestra.run.smithi164.stdout: mon warn on osd down out interval zero = false 2024-09-21T21:39:35.940 INFO:teuthology.orchestra.run.smithi164.stdout: mon warn on too few osds = false 2024-09-21T21:39:35.940 INFO:teuthology.orchestra.run.smithi164.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-21T21:39:35.940 INFO:teuthology.orchestra.run.smithi164.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-21T21:39:35.940 INFO:teuthology.orchestra.run.smithi164.stdout: mon_allow_pool_size_one = true 2024-09-21T21:39:35.940 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:39:35.940 INFO:teuthology.orchestra.run.smithi164.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-21T21:39:35.940 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:39:35.940 INFO:teuthology.orchestra.run.smithi164.stdout: osd default data pool replay window = 5 2024-09-21T21:39:35.940 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:39:35.940 INFO:teuthology.orchestra.run.smithi164.stdout: mon allow pool delete = true 2024-09-21T21:39:35.941 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:39:35.941 INFO:teuthology.orchestra.run.smithi164.stdout: mon cluster log file level = debug 2024-09-21T21:39:35.941 INFO:teuthology.orchestra.run.smithi164.stdout: debug asserts on shutdown = true 2024-09-21T21:39:35.941 INFO:teuthology.orchestra.run.smithi164.stdout: mon health detail to clog = false 2024-09-21T21:39:35.941 INFO:teuthology.orchestra.run.smithi164.stdout: mon host = "172.21.15.81,[v2:172.21.15.81:3301,v1:172.21.15.81:6790],172.21.15.122" 2024-09-21T21:39:35.941 INFO:teuthology.orchestra.run.smithi164.stdout: mon client directed command retry = 5 2024-09-21T21:39:35.941 INFO:teuthology.orchestra.run.smithi164.stdout: mon warn on pool no app = False 2024-09-21T21:39:35.941 INFO:teuthology.orchestra.run.smithi164.stdout: ms inject socket failures = 5000 2024-09-21T21:39:35.941 INFO:teuthology.orchestra.run.smithi164.stdout: fsid = a0fdcfc2-ddd2-4e50-81bb-876d940f3578 2024-09-21T21:39:35.941 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:39:35.941 INFO:teuthology.orchestra.run.smithi164.stdout:[osd] 2024-09-21T21:39:35.941 INFO:teuthology.orchestra.run.smithi164.stdout: osd journal size = 100 2024-09-21T21:39:35.941 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:39:35.941 INFO:teuthology.orchestra.run.smithi164.stdout: osd scrub load threshold = 5.0 2024-09-21T21:39:35.942 INFO:teuthology.orchestra.run.smithi164.stdout: osd scrub max interval = 600 2024-09-21T21:39:35.942 INFO:teuthology.orchestra.run.smithi164.stdout: osd mclock profile = high_recovery_ops 2024-09-21T21:39:35.942 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:39:35.942 INFO:teuthology.orchestra.run.smithi164.stdout: osd recover clone overlap = true 2024-09-21T21:39:35.942 INFO:teuthology.orchestra.run.smithi164.stdout: osd recovery max chunk = 1048576 2024-09-21T21:39:35.942 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:39:35.942 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug shutdown = true 2024-09-21T21:39:35.942 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug op order = true 2024-09-21T21:39:35.942 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug verify stray on activate = true 2024-09-21T21:39:35.942 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug trim objects = true 2024-09-21T21:39:35.942 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:39:35.942 INFO:teuthology.orchestra.run.smithi164.stdout: osd open classes on start = true 2024-09-21T21:39:35.943 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug pg log writeout = true 2024-09-21T21:39:35.943 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:39:35.943 INFO:teuthology.orchestra.run.smithi164.stdout: osd deep scrub update digest min age = 30 2024-09-21T21:39:35.943 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:39:35.943 INFO:teuthology.orchestra.run.smithi164.stdout: osd map max advance = 10 2024-09-21T21:39:35.943 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:39:35.943 INFO:teuthology.orchestra.run.smithi164.stdout: journal zero on create = true 2024-09-21T21:39:35.943 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:39:35.943 INFO:teuthology.orchestra.run.smithi164.stdout: filestore ondisk finisher threads = 3 2024-09-21T21:39:35.943 INFO:teuthology.orchestra.run.smithi164.stdout: filestore apply finisher threads = 3 2024-09-21T21:39:35.943 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:39:35.943 INFO:teuthology.orchestra.run.smithi164.stdout: bdev debug aio = true 2024-09-21T21:39:35.943 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug misdirected ops = true 2024-09-21T21:39:35.944 INFO:teuthology.orchestra.run.smithi164.stdout: bluestore block size = 96636764160 2024-09-21T21:39:35.944 INFO:teuthology.orchestra.run.smithi164.stdout: bluestore compression algorithm = snappy 2024-09-21T21:39:35.944 INFO:teuthology.orchestra.run.smithi164.stdout: bluestore compression mode = aggressive 2024-09-21T21:39:35.944 INFO:teuthology.orchestra.run.smithi164.stdout: bluestore fsck on mount = True 2024-09-21T21:39:35.944 INFO:teuthology.orchestra.run.smithi164.stdout: debug bluefs = 1/20 2024-09-21T21:39:35.944 INFO:teuthology.orchestra.run.smithi164.stdout: debug bluestore = 1/20 2024-09-21T21:39:35.944 INFO:teuthology.orchestra.run.smithi164.stdout: debug ms = 1 2024-09-21T21:39:35.944 INFO:teuthology.orchestra.run.smithi164.stdout: debug osd = 20 2024-09-21T21:39:35.944 INFO:teuthology.orchestra.run.smithi164.stdout: debug rocksdb = 4/10 2024-09-21T21:39:35.944 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd backfillfull_ratio = 0.85 2024-09-21T21:39:35.944 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd full ratio = 0.9 2024-09-21T21:39:35.944 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd nearfull ratio = 0.8 2024-09-21T21:39:35.944 INFO:teuthology.orchestra.run.smithi164.stdout: osd blocked scrub grace period = 3600 2024-09-21T21:39:35.944 INFO:teuthology.orchestra.run.smithi164.stdout: osd failsafe full ratio = 0.95 2024-09-21T21:39:35.945 INFO:teuthology.orchestra.run.smithi164.stdout: osd max markdown count = 1000 2024-09-21T21:39:35.945 INFO:teuthology.orchestra.run.smithi164.stdout: osd objectstore = bluestore 2024-09-21T21:39:35.945 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:39:35.945 INFO:teuthology.orchestra.run.smithi164.stdout:[mgr] 2024-09-21T21:39:35.945 INFO:teuthology.orchestra.run.smithi164.stdout: debug ms = 1 2024-09-21T21:39:35.945 INFO:teuthology.orchestra.run.smithi164.stdout: debug mgr = 20 2024-09-21T21:39:35.945 INFO:teuthology.orchestra.run.smithi164.stdout: debug mon = 20 2024-09-21T21:39:35.945 INFO:teuthology.orchestra.run.smithi164.stdout: debug auth = 20 2024-09-21T21:39:35.945 INFO:teuthology.orchestra.run.smithi164.stdout: mon reweight min pgs per osd = 4 2024-09-21T21:39:35.945 INFO:teuthology.orchestra.run.smithi164.stdout: mon reweight min bytes per osd = 10 2024-09-21T21:39:35.945 INFO:teuthology.orchestra.run.smithi164.stdout: mgr/telemetry/nag = false 2024-09-21T21:39:35.946 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:39:35.946 INFO:teuthology.orchestra.run.smithi164.stdout:[mon] 2024-09-21T21:39:35.946 INFO:teuthology.orchestra.run.smithi164.stdout: debug ms = 1 2024-09-21T21:39:35.946 INFO:teuthology.orchestra.run.smithi164.stdout: debug mon = 20 2024-09-21T21:39:35.946 INFO:teuthology.orchestra.run.smithi164.stdout: debug paxos = 20 2024-09-21T21:39:35.946 INFO:teuthology.orchestra.run.smithi164.stdout: debug auth = 20 2024-09-21T21:39:35.946 INFO:teuthology.orchestra.run.smithi164.stdout: mon data avail warn = 5 2024-09-21T21:39:35.946 INFO:teuthology.orchestra.run.smithi164.stdout: mon mgr mkfs grace = 240 2024-09-21T21:39:35.946 INFO:teuthology.orchestra.run.smithi164.stdout: mon reweight min pgs per osd = 4 2024-09-21T21:39:35.946 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd reporter subtree level = osd 2024-09-21T21:39:35.946 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd prime pg temp = true 2024-09-21T21:39:35.946 INFO:teuthology.orchestra.run.smithi164.stdout: mon reweight min bytes per osd = 10 2024-09-21T21:39:35.946 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:39:35.947 INFO:teuthology.orchestra.run.smithi164.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-21T21:39:35.947 INFO:teuthology.orchestra.run.smithi164.stdout: auth mon ticket ttl = 660 # 11m 2024-09-21T21:39:35.947 INFO:teuthology.orchestra.run.smithi164.stdout: auth service ticket ttl = 240 # 4m 2024-09-21T21:39:35.947 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:39:35.947 INFO:teuthology.orchestra.run.smithi164.stdout: # don't complain about insecure global_id in the test suite 2024-09-21T21:39:35.947 INFO:teuthology.orchestra.run.smithi164.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-21T21:39:35.947 INFO:teuthology.orchestra.run.smithi164.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-21T21:39:35.947 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:39:35.947 INFO:teuthology.orchestra.run.smithi164.stdout: # 1m isn't quite enough 2024-09-21T21:39:35.947 INFO:teuthology.orchestra.run.smithi164.stdout: mon_down_mkfs_grace = 2m 2024-09-21T21:39:35.947 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:39:35.947 INFO:teuthology.orchestra.run.smithi164.stdout: mon_warn_on_filestore_osds = false 2024-09-21T21:39:35.948 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-21T21:39:35.948 INFO:teuthology.orchestra.run.smithi164.stdout:[client] 2024-09-21T21:39:35.948 INFO:teuthology.orchestra.run.smithi164.stdout: rgw cache enabled = true 2024-09-21T21:39:35.948 INFO:teuthology.orchestra.run.smithi164.stdout: rgw enable ops log = true 2024-09-21T21:39:35.948 INFO:teuthology.orchestra.run.smithi164.stdout: rgw enable usage log = true 2024-09-21T21:39:35.948 INFO:teuthology.orchestra.run.smithi164.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-21T21:39:35.948 INFO:teuthology.orchestra.run.smithi164.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-21T21:39:35.948 INFO:teuthology.orchestra.run.smithi164.stdout:[mon.a] 2024-09-21T21:39:35.948 INFO:teuthology.orchestra.run.smithi164.stdout:[mon.c] 2024-09-21T21:39:35.948 INFO:teuthology.orchestra.run.smithi164.stdout:[mon.b] 2024-09-21T21:39:35.948 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-21T21:39:35.949 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-21T21:39:36.032 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-21T21:39:36.032 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-09-21T21:39:36.032 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-21T21:39:36.048 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-09-21T21:39:36.048 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-21T21:39:36.103 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi081.front.sepia.ceph.com 2024-09-21T21:39:36.103 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-09-21T21:39:36.103 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-21T21:39:36.103 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-21T21:39:36.178 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-09-21T21:39:36.178 DEBUG:teuthology.orchestra.run.smithi081:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-21T21:39:36.233 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi122.front.sepia.ceph.com 2024-09-21T21:39:36.233 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-21T21:39:36.233 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-21T21:39:36.233 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-21T21:39:36.268 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-21T21:39:36.268 DEBUG:teuthology.orchestra.run.smithi122:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-21T21:39:36.324 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi164.front.sepia.ceph.com 2024-09-21T21:39:36.324 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-21T21:39:36.324 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-21T21:39:36.324 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-21T21:39:36.366 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-21T21:39:36.366 DEBUG:teuthology.orchestra.run.smithi164:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-21T21:39:36.425 INFO:tasks.ceph:Setting up mon nodes... 2024-09-21T21:39:36.425 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-21T21:39:36.426 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-21T21:39:36.502 INFO:teuthology.orchestra.run.smithi122.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-21T21:39:36.505 INFO:tasks.ceph:Setting up mds nodes... 2024-09-21T21:39:36.505 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-21T21:39:36.505 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-21T21:39:36.561 INFO:teuthology.orchestra.run.smithi164.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-21T21:39:36.576 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-21T21:39:36.576 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi081.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}, Remote(name='ubuntu@smithi122.front.sepia.ceph.com'): {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'}} 2024-09-21T21:39:36.576 DEBUG:teuthology.orchestra.run.smithi081:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-21T21:39:36.602 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-21T21:39:36.602 INFO:tasks.ceph:role: osd.0 2024-09-21T21:39:36.602 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi081.front.sepia.ceph.com 2024-09-21T21:39:36.602 DEBUG:teuthology.orchestra.run.smithi081:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-21T21:39:36.675 INFO:teuthology.orchestra.run.smithi081.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T21:39:36.675 INFO:teuthology.orchestra.run.smithi081.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T21:39:36.675 INFO:teuthology.orchestra.run.smithi081.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T21:39:36.675 INFO:teuthology.orchestra.run.smithi081.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-21T21:39:36.675 INFO:teuthology.orchestra.run.smithi081.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T21:39:36.675 INFO:teuthology.orchestra.run.smithi081.stdout: = sunit=0 swidth=0 blks 2024-09-21T21:39:36.675 INFO:teuthology.orchestra.run.smithi081.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T21:39:36.675 INFO:teuthology.orchestra.run.smithi081.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-21T21:39:36.675 INFO:teuthology.orchestra.run.smithi081.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T21:39:36.675 INFO:teuthology.orchestra.run.smithi081.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T21:39:37.190 INFO:teuthology.orchestra.run.smithi081.stdout:Discarding blocks...Done. 2024-09-21T21:39:37.192 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi081.front.sepia.ceph.com -o noatime 2024-09-21T21:39:37.192 DEBUG:teuthology.orchestra.run.smithi081:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-21T21:39:37.320 DEBUG:teuthology.orchestra.run.smithi081:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-21T21:39:37.353 DEBUG:teuthology.orchestra.run.smithi081:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-21T21:39:37.419 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-21T21:39:37.419 INFO:tasks.ceph:role: osd.1 2024-09-21T21:39:37.419 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi081.front.sepia.ceph.com 2024-09-21T21:39:37.420 DEBUG:teuthology.orchestra.run.smithi081:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-21T21:39:37.492 INFO:teuthology.orchestra.run.smithi081.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T21:39:37.492 INFO:teuthology.orchestra.run.smithi081.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T21:39:37.492 INFO:teuthology.orchestra.run.smithi081.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T21:39:37.493 INFO:teuthology.orchestra.run.smithi081.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-21T21:39:37.493 INFO:teuthology.orchestra.run.smithi081.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T21:39:37.493 INFO:teuthology.orchestra.run.smithi081.stdout: = sunit=0 swidth=0 blks 2024-09-21T21:39:37.493 INFO:teuthology.orchestra.run.smithi081.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T21:39:37.493 INFO:teuthology.orchestra.run.smithi081.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-21T21:39:37.494 INFO:teuthology.orchestra.run.smithi081.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T21:39:37.494 INFO:teuthology.orchestra.run.smithi081.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T21:39:37.952 INFO:teuthology.orchestra.run.smithi081.stdout:Discarding blocks...Done. 2024-09-21T21:39:37.954 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi081.front.sepia.ceph.com -o noatime 2024-09-21T21:39:37.955 DEBUG:teuthology.orchestra.run.smithi081:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-21T21:39:38.042 DEBUG:teuthology.orchestra.run.smithi081:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-21T21:39:38.070 DEBUG:teuthology.orchestra.run.smithi081:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-21T21:39:38.135 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-21T21:39:38.135 INFO:tasks.ceph:role: osd.2 2024-09-21T21:39:38.135 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi081.front.sepia.ceph.com 2024-09-21T21:39:38.135 DEBUG:teuthology.orchestra.run.smithi081:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-21T21:39:38.207 INFO:teuthology.orchestra.run.smithi081.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T21:39:38.208 INFO:teuthology.orchestra.run.smithi081.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T21:39:38.208 INFO:teuthology.orchestra.run.smithi081.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T21:39:38.208 INFO:teuthology.orchestra.run.smithi081.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-21T21:39:38.208 INFO:teuthology.orchestra.run.smithi081.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T21:39:38.208 INFO:teuthology.orchestra.run.smithi081.stdout: = sunit=0 swidth=0 blks 2024-09-21T21:39:38.208 INFO:teuthology.orchestra.run.smithi081.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T21:39:38.208 INFO:teuthology.orchestra.run.smithi081.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-21T21:39:38.208 INFO:teuthology.orchestra.run.smithi081.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T21:39:38.208 INFO:teuthology.orchestra.run.smithi081.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T21:39:38.692 INFO:teuthology.orchestra.run.smithi081.stdout:Discarding blocks...Done. 2024-09-21T21:39:38.693 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi081.front.sepia.ceph.com -o noatime 2024-09-21T21:39:38.694 DEBUG:teuthology.orchestra.run.smithi081:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-21T21:39:38.802 DEBUG:teuthology.orchestra.run.smithi081:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-21T21:39:38.830 DEBUG:teuthology.orchestra.run.smithi081:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T21:39:38.925 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-21T21:39:38.924+0000 7f763b233640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-21T21:39:38.931 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-21T21:39:38.929+0000 7f763b233640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-21T21:39:38.931 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-21T21:39:38.929+0000 7f763b233640 -1 bdev(0x556fc9993800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-21T21:39:38.931 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-21T21:39:38.929+0000 7f763b233640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-21T21:39:42.097 DEBUG:teuthology.orchestra.run.smithi081:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-21T21:39:42.126 DEBUG:teuthology.orchestra.run.smithi081:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T21:39:42.216 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-21T21:39:42.214+0000 7feb13222640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-21T21:39:42.216 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-21T21:39:42.214+0000 7feb13222640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-21T21:39:42.216 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-21T21:39:42.214+0000 7feb13222640 -1 bdev(0x55a7a4cc3800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-21T21:39:42.216 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-21T21:39:42.214+0000 7feb13222640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-21T21:39:45.379 DEBUG:teuthology.orchestra.run.smithi081:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-21T21:39:45.408 DEBUG:teuthology.orchestra.run.smithi081:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T21:39:45.498 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-21T21:39:45.497+0000 7fd84f72e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-21T21:39:45.499 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-21T21:39:45.497+0000 7fd84f72e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-21T21:39:45.499 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-21T21:39:45.497+0000 7fd84f72e640 -1 bdev(0x55f03f7bb800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-21T21:39:45.499 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-21T21:39:45.497+0000 7fd84f72e640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-21T21:39:48.905 DEBUG:teuthology.orchestra.run.smithi081:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-21T21:39:48.932 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-21T21:39:48.962 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-09-21T21:39:48.962 INFO:tasks.ceph:role: osd.3 2024-09-21T21:39:48.962 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi122.front.sepia.ceph.com 2024-09-21T21:39:48.962 DEBUG:teuthology.orchestra.run.smithi122:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-21T21:39:49.037 INFO:teuthology.orchestra.run.smithi122.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T21:39:49.038 INFO:teuthology.orchestra.run.smithi122.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T21:39:49.038 INFO:teuthology.orchestra.run.smithi122.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T21:39:49.038 INFO:teuthology.orchestra.run.smithi122.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-21T21:39:49.038 INFO:teuthology.orchestra.run.smithi122.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T21:39:49.038 INFO:teuthology.orchestra.run.smithi122.stdout: = sunit=0 swidth=0 blks 2024-09-21T21:39:49.038 INFO:teuthology.orchestra.run.smithi122.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T21:39:49.038 INFO:teuthology.orchestra.run.smithi122.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-21T21:39:49.038 INFO:teuthology.orchestra.run.smithi122.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T21:39:49.038 INFO:teuthology.orchestra.run.smithi122.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T21:39:49.505 INFO:teuthology.orchestra.run.smithi122.stdout:Discarding blocks...Done. 2024-09-21T21:39:49.507 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi122.front.sepia.ceph.com -o noatime 2024-09-21T21:39:49.508 DEBUG:teuthology.orchestra.run.smithi122:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-3 2024-09-21T21:39:49.632 DEBUG:teuthology.orchestra.run.smithi122:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-21T21:39:49.666 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-21T21:39:49.735 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-09-21T21:39:49.735 INFO:tasks.ceph:role: osd.4 2024-09-21T21:39:49.735 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi122.front.sepia.ceph.com 2024-09-21T21:39:49.735 DEBUG:teuthology.orchestra.run.smithi122:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-21T21:39:49.809 INFO:teuthology.orchestra.run.smithi122.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T21:39:49.809 INFO:teuthology.orchestra.run.smithi122.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T21:39:49.809 INFO:teuthology.orchestra.run.smithi122.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T21:39:49.809 INFO:teuthology.orchestra.run.smithi122.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-21T21:39:49.810 INFO:teuthology.orchestra.run.smithi122.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T21:39:49.810 INFO:teuthology.orchestra.run.smithi122.stdout: = sunit=0 swidth=0 blks 2024-09-21T21:39:49.810 INFO:teuthology.orchestra.run.smithi122.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T21:39:49.810 INFO:teuthology.orchestra.run.smithi122.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-21T21:39:49.810 INFO:teuthology.orchestra.run.smithi122.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T21:39:49.810 INFO:teuthology.orchestra.run.smithi122.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T21:39:50.277 INFO:teuthology.orchestra.run.smithi122.stdout:Discarding blocks...Done. 2024-09-21T21:39:50.279 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi122.front.sepia.ceph.com -o noatime 2024-09-21T21:39:50.279 DEBUG:teuthology.orchestra.run.smithi122:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-4 2024-09-21T21:39:50.361 DEBUG:teuthology.orchestra.run.smithi122:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-21T21:39:50.430 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-21T21:39:50.496 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-09-21T21:39:50.496 INFO:tasks.ceph:role: osd.5 2024-09-21T21:39:50.496 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi122.front.sepia.ceph.com 2024-09-21T21:39:50.496 DEBUG:teuthology.orchestra.run.smithi122:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-21T21:39:50.568 INFO:teuthology.orchestra.run.smithi122.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T21:39:50.569 INFO:teuthology.orchestra.run.smithi122.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T21:39:50.569 INFO:teuthology.orchestra.run.smithi122.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T21:39:50.569 INFO:teuthology.orchestra.run.smithi122.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-21T21:39:50.569 INFO:teuthology.orchestra.run.smithi122.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T21:39:50.569 INFO:teuthology.orchestra.run.smithi122.stdout: = sunit=0 swidth=0 blks 2024-09-21T21:39:50.569 INFO:teuthology.orchestra.run.smithi122.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T21:39:50.569 INFO:teuthology.orchestra.run.smithi122.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-21T21:39:50.569 INFO:teuthology.orchestra.run.smithi122.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T21:39:50.569 INFO:teuthology.orchestra.run.smithi122.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T21:39:51.057 INFO:teuthology.orchestra.run.smithi122.stdout:Discarding blocks...Done. 2024-09-21T21:39:51.059 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi122.front.sepia.ceph.com -o noatime 2024-09-21T21:39:51.059 DEBUG:teuthology.orchestra.run.smithi122:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-5 2024-09-21T21:39:51.176 DEBUG:teuthology.orchestra.run.smithi122:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-21T21:39:51.205 DEBUG:teuthology.orchestra.run.smithi122:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T21:39:51.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T21:39:51.306+0000 7f50e142c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-21T21:39:51.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T21:39:51.307+0000 7f50e142c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-21T21:39:51.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T21:39:51.307+0000 7f50e142c640 -1 bdev(0x55bb8344f800 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-21T21:39:51.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T21:39:51.308+0000 7f50e142c640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-21T21:39:54.217 DEBUG:teuthology.orchestra.run.smithi122:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-21T21:39:54.244 DEBUG:teuthology.orchestra.run.smithi122:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T21:39:54.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T21:39:54.330+0000 7f98e282f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-21T21:39:54.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T21:39:54.330+0000 7f98e282f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-21T21:39:54.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T21:39:54.331+0000 7f98e282f640 -1 bdev(0x55d44f463800 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-21T21:39:54.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T21:39:54.331+0000 7f98e282f640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-21T21:39:57.489 DEBUG:teuthology.orchestra.run.smithi122:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-21T21:39:57.519 DEBUG:teuthology.orchestra.run.smithi122:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T21:39:57.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T21:39:57.609+0000 7f9307c26640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-21T21:39:57.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T21:39:57.610+0000 7f9307c26640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-21T21:39:57.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T21:39:57.610+0000 7f9307c26640 -1 bdev(0x55cdbb807800 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-21T21:39:57.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-21T21:39:57.610+0000 7f9307c26640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-21T21:40:00.536 DEBUG:teuthology.orchestra.run.smithi122:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-21T21:40:00.563 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-21T21:40:00.563 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-09-21T21:40:00.564 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-21T21:40:00.595 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-09-21T21:40:00.595 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-21T21:40:00.663 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-09-21T21:40:00.663 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-21T21:40:00.731 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-21T21:40:00.731 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-21T21:40:00.758 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-21T21:40:00.758 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-21T21:40:00.824 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-21T21:40:00.824 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-21T21:40:00.890 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-21T21:40:00.890 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-21T21:40:00.956 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-21T21:40:00.956 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-21T21:40:00.976 INFO:tasks.ceph:Adding keys to all mons... 2024-09-21T21:40:00.976 DEBUG:teuthology.orchestra.run.smithi081:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-21T21:40:00.978 DEBUG:teuthology.orchestra.run.smithi122:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-21T21:40:01.003 INFO:teuthology.orchestra.run.smithi081.stdout:[osd.0] 2024-09-21T21:40:01.003 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQAaPe9mqxckNxAAqE4wSus76WRMX/xt7eHhlw== 2024-09-21T21:40:01.003 INFO:teuthology.orchestra.run.smithi081.stdout:[osd.1] 2024-09-21T21:40:01.003 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQAePe9mxXfWDBAAlkEY1U+7KU2u49jETsKokQ== 2024-09-21T21:40:01.003 INFO:teuthology.orchestra.run.smithi081.stdout:[osd.2] 2024-09-21T21:40:01.003 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQAhPe9m88auHRAANhqdSWsZ/juatvSJlXznVg== 2024-09-21T21:40:01.003 INFO:teuthology.orchestra.run.smithi081.stdout:[mgr.x] 2024-09-21T21:40:01.003 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQAYPe9m1sHsHRAAmhZbp34V/6PuFwaiqkhsSw== 2024-09-21T21:40:01.003 INFO:teuthology.orchestra.run.smithi081.stdout:[osd.3] 2024-09-21T21:40:01.003 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQAnPe9mGfxOEhAAyXdMWCDN3vEc53Z/YlvA4Q== 2024-09-21T21:40:01.004 INFO:teuthology.orchestra.run.smithi081.stdout:[osd.4] 2024-09-21T21:40:01.004 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQAqPe9m3uXDExAARSU/IsRAmuOpo5ZhLRGOEA== 2024-09-21T21:40:01.004 INFO:teuthology.orchestra.run.smithi081.stdout:[osd.5] 2024-09-21T21:40:01.004 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQAtPe9mibhzJBAAYCTNJc4lCtPVDMyiOeJ60w== 2024-09-21T21:40:01.004 INFO:teuthology.orchestra.run.smithi081.stdout:[client.0] 2024-09-21T21:40:01.004 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQAYPe9m9kVyIRAAa97yktXTxd4gzc7K1a3Tlg== 2024-09-21T21:40:01.019 INFO:teuthology.orchestra.run.smithi122.stdout:[osd.0] 2024-09-21T21:40:01.020 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQAaPe9mqxckNxAAqE4wSus76WRMX/xt7eHhlw== 2024-09-21T21:40:01.020 INFO:teuthology.orchestra.run.smithi122.stdout:[osd.1] 2024-09-21T21:40:01.020 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQAePe9mxXfWDBAAlkEY1U+7KU2u49jETsKokQ== 2024-09-21T21:40:01.020 INFO:teuthology.orchestra.run.smithi122.stdout:[osd.2] 2024-09-21T21:40:01.020 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQAhPe9m88auHRAANhqdSWsZ/juatvSJlXznVg== 2024-09-21T21:40:01.020 INFO:teuthology.orchestra.run.smithi122.stdout:[mgr.x] 2024-09-21T21:40:01.020 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQAYPe9m1sHsHRAAmhZbp34V/6PuFwaiqkhsSw== 2024-09-21T21:40:01.020 INFO:teuthology.orchestra.run.smithi122.stdout:[osd.3] 2024-09-21T21:40:01.020 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQAnPe9mGfxOEhAAyXdMWCDN3vEc53Z/YlvA4Q== 2024-09-21T21:40:01.020 INFO:teuthology.orchestra.run.smithi122.stdout:[osd.4] 2024-09-21T21:40:01.020 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQAqPe9m3uXDExAARSU/IsRAmuOpo5ZhLRGOEA== 2024-09-21T21:40:01.020 INFO:teuthology.orchestra.run.smithi122.stdout:[osd.5] 2024-09-21T21:40:01.020 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQAtPe9mibhzJBAAYCTNJc4lCtPVDMyiOeJ60w== 2024-09-21T21:40:01.021 INFO:teuthology.orchestra.run.smithi122.stdout:[client.0] 2024-09-21T21:40:01.021 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQAYPe9m9kVyIRAAa97yktXTxd4gzc7K1a3Tlg== 2024-09-21T21:40:01.021 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T21:40:01.047 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T21:40:01.114 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T21:40:01.116 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T21:40:01.167 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T21:40:01.259 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T21:40:01.311 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-21T21:40:01.313 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-21T21:40:01.364 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T21:40:01.366 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T21:40:01.417 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T21:40:01.420 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T21:40:01.471 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T21:40:01.474 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T21:40:01.524 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T21:40:01.527 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T21:40:01.580 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-21T21:40:01.580 DEBUG:teuthology.orchestra.run.smithi081:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-21T21:40:01.604 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-21T21:40:01.700 DEBUG:teuthology.orchestra.run.smithi081:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-21T21:40:01.725 DEBUG:teuthology.orchestra.run.smithi081:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-21T21:40:01.789 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-21T21:40:01.885 DEBUG:teuthology.orchestra.run.smithi081:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-21T21:40:01.910 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-21T21:40:01.935 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-21T21:40:02.031 DEBUG:teuthology.orchestra.run.smithi122:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-21T21:40:02.056 DEBUG:teuthology.orchestra.run.smithi081:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-21T21:40:02.058 DEBUG:teuthology.orchestra.run.smithi122:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-21T21:40:02.110 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-21T21:40:02.111 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-21T21:40:02.111 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-21T21:40:02.114 INFO:tasks.ceph.mon.a:Started 2024-09-21T21:40:02.114 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-21T21:40:02.114 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-21T21:40:02.118 INFO:tasks.ceph.mon.c:Started 2024-09-21T21:40:02.119 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-21T21:40:02.119 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-21T21:40:02.152 INFO:tasks.ceph.mon.b:Started 2024-09-21T21:40:02.153 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-21T21:40:02.153 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-21T21:40:02.153 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-21T21:40:02.155 INFO:tasks.ceph.mgr.x:Started 2024-09-21T21:40:02.155 INFO:tasks.ceph:Setting crush tunables to default 2024-09-21T21:40:02.155 DEBUG:teuthology.orchestra.run.smithi081:> sudo ceph --cluster ceph osd crush tunables default 2024-09-21T21:40:07.517 INFO:teuthology.orchestra.run.smithi081.stderr:adjusted tunables profile to default 2024-09-21T21:40:07.530 INFO:tasks.ceph:check_enable_crimson: False 2024-09-21T21:40:07.530 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-21T21:40:07.530 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-09-21T21:40:07.530 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-21T21:40:07.555 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-09-21T21:40:07.556 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-21T21:40:07.621 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-09-21T21:40:07.621 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-21T21:40:07.686 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-21T21:40:07.687 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-21T21:40:07.717 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-21T21:40:07.718 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-21T21:40:07.786 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-21T21:40:07.787 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-21T21:40:07.855 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph --cluster ceph osd new a9869830-8e74-4b02-963d-ab630a0ee286 0 2024-09-21T21:40:08.076 INFO:teuthology.orchestra.run.smithi122.stdout:0 2024-09-21T21:40:08.087 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph --cluster ceph osd new b189819d-69a3-4e95-96e7-96dc4809ac80 1 2024-09-21T21:40:08.268 INFO:teuthology.orchestra.run.smithi122.stdout:1 2024-09-21T21:40:08.280 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph --cluster ceph osd new 8437b99a-1297-4903-ab66-95c5cba2b2bf 2 2024-09-21T21:40:08.333 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-21T21:40:08.331+0000 7f30a5b23040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-21T21:40:08.375 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-21T21:40:08.373+0000 7f30a5b23040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-21T21:40:08.462 INFO:teuthology.orchestra.run.smithi122.stdout:2 2024-09-21T21:40:08.473 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph --cluster ceph osd new e8848768-c1aa-445b-9554-dc6aa4eefc8d 3 2024-09-21T21:40:08.519 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-21T21:40:08.517+0000 7f30a5b23040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-21T21:40:08.571 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-21T21:40:08.569+0000 7f30a5b23040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-21T21:40:08.651 INFO:teuthology.orchestra.run.smithi122.stdout:3 2024-09-21T21:40:08.662 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph --cluster ceph osd new b1d8e85d-a207-4ba6-9322-61d07812083e 4 2024-09-21T21:40:08.738 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-21T21:40:08.736+0000 7f30a5b23040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-21T21:40:08.842 INFO:teuthology.orchestra.run.smithi122.stdout:4 2024-09-21T21:40:08.853 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph --cluster ceph osd new 3ab0450b-d8fd-4ef2-a298-08d5aa2ea32f 5 2024-09-21T21:40:08.853 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-21T21:40:08.851+0000 7f30a5b23040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-21T21:40:09.033 INFO:teuthology.orchestra.run.smithi122.stdout:5 2024-09-21T21:40:09.045 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-21T21:40:09.045 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-21T21:40:09.047 INFO:tasks.ceph.osd.0:Started 2024-09-21T21:40:09.048 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-21T21:40:09.048 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-21T21:40:09.050 INFO:tasks.ceph.osd.1:Started 2024-09-21T21:40:09.050 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-21T21:40:09.051 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-21T21:40:09.054 INFO:tasks.ceph.osd.2:Started 2024-09-21T21:40:09.054 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-21T21:40:09.054 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-21T21:40:09.056 INFO:tasks.ceph.osd.3:Started 2024-09-21T21:40:09.057 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-21T21:40:09.057 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-21T21:40:09.059 INFO:tasks.ceph.osd.4:Started 2024-09-21T21:40:09.059 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-21T21:40:09.059 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-21T21:40:09.061 INFO:tasks.ceph.osd.5:Started 2024-09-21T21:40:09.062 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T21:40:09.214 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-21T21:40:09.212+0000 7f30a5b23040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-21T21:40:09.239 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:40:09.239 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":8,"fsid":"a0fdcfc2-ddd2-4e50-81bb-876d940f3578","created":"2024-09-21T21:40:07.413917+0000","modified":"2024-09-21T21:40:09.029953+0000","last_up_change":"0.000000","last_in_change":"2024-09-21T21:40:09.029953+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"a9869830-8e74-4b02-963d-ab630a0ee286","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"b189819d-69a3-4e95-96e7-96dc4809ac80","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"8437b99a-1297-4903-ab66-95c5cba2b2bf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"e8848768-c1aa-445b-9554-dc6aa4eefc8d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"b1d8e85d-a207-4ba6-9322-61d07812083e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"3ab0450b-d8fd-4ef2-a298-08d5aa2ea32f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T21:40:09.250 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-21T21:40:09.250 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-21T21:40:09.266 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-21T21:40:09.264+0000 7f30a5b23040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-21T21:40:09.415 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-21T21:40:09.414+0000 7f30a5b23040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-21T21:40:09.854 DEBUG:teuthology.orchestra.run.smithi081:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-21T21:40:10.002 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:40:10.000+0000 7fd4a740c640 -1 Falling back to public interface 2024-09-21T21:40:10.017 INFO:teuthology.misc.health.smithi081.stdout: 2024-09-21T21:40:10.017 INFO:teuthology.misc.health.smithi081.stdout:{"epoch":8,"fsid":"a0fdcfc2-ddd2-4e50-81bb-876d940f3578","created":"2024-09-21T21:40:07.413917+0000","modified":"2024-09-21T21:40:09.029953+0000","last_up_change":"0.000000","last_in_change":"2024-09-21T21:40:09.029953+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"a9869830-8e74-4b02-963d-ab630a0ee286","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"b189819d-69a3-4e95-96e7-96dc4809ac80","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"8437b99a-1297-4903-ab66-95c5cba2b2bf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"e8848768-c1aa-445b-9554-dc6aa4eefc8d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"b1d8e85d-a207-4ba6-9322-61d07812083e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"3ab0450b-d8fd-4ef2-a298-08d5aa2ea32f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T21:40:10.027 DEBUG:teuthology.misc:0 of 6 OSDs are up 2024-09-21T21:40:10.397 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-21T21:40:10.395+0000 7f30a5b23040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-21T21:40:10.451 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-21T21:40:10.450+0000 7f30a5b23040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-21T21:40:10.487 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:40:10.485+0000 7faee220f640 -1 Falling back to public interface 2024-09-21T21:40:10.519 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:40:10.517+0000 7fc970fa8640 -1 Falling back to public interface 2024-09-21T21:40:10.526 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:40:10.524+0000 7f0438422640 -1 Falling back to public interface 2024-09-21T21:40:10.741 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:40:10.739+0000 7ff40cb4b640 -1 Falling back to public interface 2024-09-21T21:40:10.771 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:40:10.770+0000 7f7fb954b640 -1 Falling back to public interface 2024-09-21T21:40:10.817 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-21T21:40:10.815+0000 7f30a5b23040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-21T21:40:10.902 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-21T21:40:10.900+0000 7f30a5b23040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-21T21:40:11.008 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-21T21:40:11.006+0000 7f30a5b23040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-21T21:40:11.054 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-21T21:40:11.053+0000 7f30a5b23040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-21T21:40:11.167 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-21T21:40:11.166+0000 7f30a5b23040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-21T21:40:11.219 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-21T21:40:11.217+0000 7f30a5b23040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-21T21:40:11.263 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-21T21:40:11.262+0000 7f30a5b23040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-21T21:40:11.376 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-21T21:40:11.374+0000 7f30a5b23040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-21T21:40:11.427 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-21T21:40:11.425+0000 7f30a5b23040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-21T21:40:11.478 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-21T21:40:11.476+0000 7f30a5b23040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-21T21:40:11.545 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-21T21:40:11.543+0000 7f30a5b23040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-21T21:40:11.589 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-21T21:40:11.588+0000 7f30a5b23040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-21T21:40:11.634 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-21T21:40:11.633+0000 7f30a5b23040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-21T21:40:12.283 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-21T21:40:12.281+0000 7f30a5b23040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-21T21:40:13.296 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-21T21:40:13.295+0000 7f3045938640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-21T21:40:13.297 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-21T21:40:13.295+0000 7f3045938640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-21T21:40:13.776 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:40:13.773+0000 7fd4a740c640 -1 osd.2 0 log_to_monitors true 2024-09-21T21:40:14.367 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:40:14.366+0000 7fc970fa8640 -1 osd.4 0 log_to_monitors true 2024-09-21T21:40:14.386 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:40:14.384+0000 7f0438422640 -1 osd.3 0 log_to_monitors true 2024-09-21T21:40:14.495 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:40:14.493+0000 7ff40cb4b640 -1 osd.1 0 log_to_monitors true 2024-09-21T21:40:14.589 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:40:14.587+0000 7f7fb954b640 -1 osd.5 0 log_to_monitors true 2024-09-21T21:40:14.781 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:40:14.778+0000 7faee220f640 -1 osd.0 0 log_to_monitors true 2024-09-21T21:40:15.965 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:40:15.963+0000 7fd4a2dd9640 -1 osd.2 0 waiting for initial osdmap 2024-09-21T21:40:15.974 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:40:15.972+0000 7fd49abf9640 -1 osd.2 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T21:40:16.633 DEBUG:teuthology.orchestra.run.smithi081:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-21T21:40:17.035 INFO:teuthology.misc.health.smithi081.stdout: 2024-09-21T21:40:17.035 INFO:teuthology.misc.health.smithi081.stdout:{"epoch":11,"fsid":"a0fdcfc2-ddd2-4e50-81bb-876d940f3578","created":"2024-09-21T21:40:07.413917+0000","modified":"2024-09-21T21:40:16.295100+0000","last_up_change":"2024-09-21T21:40:16.295100+0000","last_in_change":"2024-09-21T21:40:09.029953+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"a9869830-8e74-4b02-963d-ab630a0ee286","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"b189819d-69a3-4e95-96e7-96dc4809ac80","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"8437b99a-1297-4903-ab66-95c5cba2b2bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3023160932},{"type":"v1","addr":"172.21.15.81:6801","nonce":3023160932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3023160932},{"type":"v1","addr":"172.21.15.81:6803","nonce":3023160932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3023160932},{"type":"v1","addr":"172.21.15.81:6807","nonce":3023160932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3023160932},{"type":"v1","addr":"172.21.15.81:6805","nonce":3023160932}]},"public_addr":"172.21.15.81:6801/3023160932","cluster_addr":"172.21.15.81:6803/3023160932","heartbeat_back_addr":"172.21.15.81:6807/3023160932","heartbeat_front_addr":"172.21.15.81:6805/3023160932","state":["exists","up"]},{"osd":3,"uuid":"e8848768-c1aa-445b-9554-dc6aa4eefc8d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"b1d8e85d-a207-4ba6-9322-61d07812083e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"3ab0450b-d8fd-4ef2-a298-08d5aa2ea32f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T21:40:17.048 DEBUG:teuthology.misc:1 of 6 OSDs are up 2024-09-21T21:40:17.175 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:40:17.173+0000 7faeddb1e640 -1 osd.0 0 waiting for initial osdmap 2024-09-21T21:40:17.176 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:40:17.175+0000 7ff408ada640 -1 osd.1 0 waiting for initial osdmap 2024-09-21T21:40:17.189 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:40:17.187+0000 7faed593e640 -1 osd.0 11 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T21:40:17.189 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:40:17.187+0000 7ff4008fa640 -1 osd.1 11 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T21:40:17.289 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:40:17.288+0000 7f7fb54da640 -1 osd.5 0 waiting for initial osdmap 2024-09-21T21:40:17.299 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:40:17.298+0000 7f7fad2fa640 -1 osd.5 11 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T21:40:17.330 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:40:17.329+0000 7f0433d14640 -1 osd.3 0 waiting for initial osdmap 2024-09-21T21:40:17.336 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:40:17.334+0000 7fc96cf37640 -1 osd.4 0 waiting for initial osdmap 2024-09-21T21:40:17.344 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:40:17.343+0000 7f042bb34640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T21:40:17.344 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:40:17.343+0000 7fc964d57640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T21:40:23.655 DEBUG:teuthology.orchestra.run.smithi081:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-21T21:40:23.934 INFO:teuthology.misc.health.smithi081.stdout: 2024-09-21T21:40:23.934 INFO:teuthology.misc.health.smithi081.stdout:{"epoch":15,"fsid":"a0fdcfc2-ddd2-4e50-81bb-876d940f3578","created":"2024-09-21T21:40:07.413917+0000","modified":"2024-09-21T21:40:20.346779+0000","last_up_change":"2024-09-21T21:40:18.340430+0000","last_in_change":"2024-09-21T21:40:09.029953+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T21:40:18.321328+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9869830-8e74-4b02-963d-ab630a0ee286","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1657552338},{"type":"v1","addr":"172.21.15.81:6809","nonce":1657552338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1657552338},{"type":"v1","addr":"172.21.15.81:6811","nonce":1657552338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1657552338},{"type":"v1","addr":"172.21.15.81:6815","nonce":1657552338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1657552338},{"type":"v1","addr":"172.21.15.81:6813","nonce":1657552338}]},"public_addr":"172.21.15.81:6809/1657552338","cluster_addr":"172.21.15.81:6811/1657552338","heartbeat_back_addr":"172.21.15.81:6815/1657552338","heartbeat_front_addr":"172.21.15.81:6813/1657552338","state":["exists","up"]},{"osd":1,"uuid":"b189819d-69a3-4e95-96e7-96dc4809ac80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2864159502},{"type":"v1","addr":"172.21.15.81:6817","nonce":2864159502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2864159502},{"type":"v1","addr":"172.21.15.81:6819","nonce":2864159502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2864159502},{"type":"v1","addr":"172.21.15.81:6823","nonce":2864159502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2864159502},{"type":"v1","addr":"172.21.15.81:6821","nonce":2864159502}]},"public_addr":"172.21.15.81:6817/2864159502","cluster_addr":"172.21.15.81:6819/2864159502","heartbeat_back_addr":"172.21.15.81:6823/2864159502","heartbeat_front_addr":"172.21.15.81:6821/2864159502","state":["exists","up"]},{"osd":2,"uuid":"8437b99a-1297-4903-ab66-95c5cba2b2bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3023160932},{"type":"v1","addr":"172.21.15.81:6801","nonce":3023160932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3023160932},{"type":"v1","addr":"172.21.15.81:6803","nonce":3023160932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3023160932},{"type":"v1","addr":"172.21.15.81:6807","nonce":3023160932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3023160932},{"type":"v1","addr":"172.21.15.81:6805","nonce":3023160932}]},"public_addr":"172.21.15.81:6801/3023160932","cluster_addr":"172.21.15.81:6803/3023160932","heartbeat_back_addr":"172.21.15.81:6807/3023160932","heartbeat_front_addr":"172.21.15.81:6805/3023160932","state":["exists","up"]},{"osd":3,"uuid":"e8848768-c1aa-445b-9554-dc6aa4eefc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":2681016844},{"type":"v1","addr":"172.21.15.122:6809","nonce":2681016844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2681016844},{"type":"v1","addr":"172.21.15.122:6811","nonce":2681016844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2681016844},{"type":"v1","addr":"172.21.15.122:6815","nonce":2681016844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2681016844},{"type":"v1","addr":"172.21.15.122:6813","nonce":2681016844}]},"public_addr":"172.21.15.122:6809/2681016844","cluster_addr":"172.21.15.122:6811/2681016844","heartbeat_back_addr":"172.21.15.122:6815/2681016844","heartbeat_front_addr":"172.21.15.122:6813/2681016844","state":["exists","up"]},{"osd":4,"uuid":"b1d8e85d-a207-4ba6-9322-61d07812083e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":1095023540},{"type":"v1","addr":"172.21.15.122:6801","nonce":1095023540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":1095023540},{"type":"v1","addr":"172.21.15.122:6803","nonce":1095023540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":1095023540},{"type":"v1","addr":"172.21.15.122:6807","nonce":1095023540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":1095023540},{"type":"v1","addr":"172.21.15.122:6805","nonce":1095023540}]},"public_addr":"172.21.15.122:6801/1095023540","cluster_addr":"172.21.15.122:6803/1095023540","heartbeat_back_addr":"172.21.15.122:6807/1095023540","heartbeat_front_addr":"172.21.15.122:6805/1095023540","state":["exists","up"]},{"osd":5,"uuid":"3ab0450b-d8fd-4ef2-a298-08d5aa2ea32f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1974398333},{"type":"v1","addr":"172.21.15.122:6817","nonce":1974398333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1974398333},{"type":"v1","addr":"172.21.15.122:6819","nonce":1974398333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1974398333},{"type":"v1","addr":"172.21.15.122:6823","nonce":1974398333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1974398333},{"type":"v1","addr":"172.21.15.122:6821","nonce":1974398333}]},"public_addr":"172.21.15.122:6817/1974398333","cluster_addr":"172.21.15.122:6819/1974398333","heartbeat_back_addr":"172.21.15.122:6823/1974398333","heartbeat_front_addr":"172.21.15.122:6821/1974398333","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T21:40:15.766648+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T21:40:15.457496+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T21:40:14.767039+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T21:40:15.355570+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T21:40:15.335729+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T21:40:15.614733+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T21:40:23.945 DEBUG:teuthology.misc:6 of 6 OSDs are up 2024-09-21T21:40:23.945 INFO:tasks.ceph:Creating RBD pool 2024-09-21T21:40:23.946 DEBUG:teuthology.orchestra.run.smithi081:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-21T21:40:24.362 INFO:teuthology.orchestra.run.smithi081.stderr:pool 'rbd' created 2024-09-21T21:40:24.376 DEBUG:teuthology.orchestra.run.smithi081:> rbd --cluster ceph pool init rbd 2024-09-21T21:40:27.388 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-21T21:40:27.389 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-21T21:40:27.389 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-21T21:40:27.389 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-21T21:40:27.389 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-21T21:40:27.719 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:40:27.730 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":5,"flags":0,"active_gid":4120,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":3903555308},{"type":"v1","addr":"172.21.15.122:6825","nonce":3903555308}]},"active_addr":"172.21.15.122:6825/3903555308","active_change":"2024-09-21T21:40:12.284495+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.122:0","nonce":707681086}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.122:0","nonce":3913072034}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.122:0","nonce":3403571498}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.122:0","nonce":3143528999}]}]} 2024-09-21T21:40:27.732 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-21T21:40:27.732 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-21T21:40:27.732 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T21:40:28.007 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:40:28.007 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":19,"fsid":"a0fdcfc2-ddd2-4e50-81bb-876d940f3578","created":"2024-09-21T21:40:07.413917+0000","modified":"2024-09-21T21:40:27.373092+0000","last_up_change":"2024-09-21T21:40:18.340430+0000","last_in_change":"2024-09-21T21:40:09.029953+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T21:40:18.321328+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-21T21:40:24.235658+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":19,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9869830-8e74-4b02-963d-ab630a0ee286","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1657552338},{"type":"v1","addr":"172.21.15.81:6809","nonce":1657552338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1657552338},{"type":"v1","addr":"172.21.15.81:6811","nonce":1657552338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1657552338},{"type":"v1","addr":"172.21.15.81:6815","nonce":1657552338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1657552338},{"type":"v1","addr":"172.21.15.81:6813","nonce":1657552338}]},"public_addr":"172.21.15.81:6809/1657552338","cluster_addr":"172.21.15.81:6811/1657552338","heartbeat_back_addr":"172.21.15.81:6815/1657552338","heartbeat_front_addr":"172.21.15.81:6813/1657552338","state":["exists","up"]},{"osd":1,"uuid":"b189819d-69a3-4e95-96e7-96dc4809ac80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2864159502},{"type":"v1","addr":"172.21.15.81:6817","nonce":2864159502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2864159502},{"type":"v1","addr":"172.21.15.81:6819","nonce":2864159502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2864159502},{"type":"v1","addr":"172.21.15.81:6823","nonce":2864159502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2864159502},{"type":"v1","addr":"172.21.15.81:6821","nonce":2864159502}]},"public_addr":"172.21.15.81:6817/2864159502","cluster_addr":"172.21.15.81:6819/2864159502","heartbeat_back_addr":"172.21.15.81:6823/2864159502","heartbeat_front_addr":"172.21.15.81:6821/2864159502","state":["exists","up"]},{"osd":2,"uuid":"8437b99a-1297-4903-ab66-95c5cba2b2bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3023160932},{"type":"v1","addr":"172.21.15.81:6801","nonce":3023160932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3023160932},{"type":"v1","addr":"172.21.15.81:6803","nonce":3023160932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3023160932},{"type":"v1","addr":"172.21.15.81:6807","nonce":3023160932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3023160932},{"type":"v1","addr":"172.21.15.81:6805","nonce":3023160932}]},"public_addr":"172.21.15.81:6801/3023160932","cluster_addr":"172.21.15.81:6803/3023160932","heartbeat_back_addr":"172.21.15.81:6807/3023160932","heartbeat_front_addr":"172.21.15.81:6805/3023160932","state":["exists","up"]},{"osd":3,"uuid":"e8848768-c1aa-445b-9554-dc6aa4eefc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":2681016844},{"type":"v1","addr":"172.21.15.122:6809","nonce":2681016844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2681016844},{"type":"v1","addr":"172.21.15.122:6811","nonce":2681016844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2681016844},{"type":"v1","addr":"172.21.15.122:6815","nonce":2681016844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2681016844},{"type":"v1","addr":"172.21.15.122:6813","nonce":2681016844}]},"public_addr":"172.21.15.122:6809/2681016844","cluster_addr":"172.21.15.122:6811/2681016844","heartbeat_back_addr":"172.21.15.122:6815/2681016844","heartbeat_front_addr":"172.21.15.122:6813/2681016844","state":["exists","up"]},{"osd":4,"uuid":"b1d8e85d-a207-4ba6-9322-61d07812083e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":1095023540},{"type":"v1","addr":"172.21.15.122:6801","nonce":1095023540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":1095023540},{"type":"v1","addr":"172.21.15.122:6803","nonce":1095023540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":1095023540},{"type":"v1","addr":"172.21.15.122:6807","nonce":1095023540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":1095023540},{"type":"v1","addr":"172.21.15.122:6805","nonce":1095023540}]},"public_addr":"172.21.15.122:6801/1095023540","cluster_addr":"172.21.15.122:6803/1095023540","heartbeat_back_addr":"172.21.15.122:6807/1095023540","heartbeat_front_addr":"172.21.15.122:6805/1095023540","state":["exists","up"]},{"osd":5,"uuid":"3ab0450b-d8fd-4ef2-a298-08d5aa2ea32f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1974398333},{"type":"v1","addr":"172.21.15.122:6817","nonce":1974398333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1974398333},{"type":"v1","addr":"172.21.15.122:6819","nonce":1974398333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1974398333},{"type":"v1","addr":"172.21.15.122:6823","nonce":1974398333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1974398333},{"type":"v1","addr":"172.21.15.122:6821","nonce":1974398333}]},"public_addr":"172.21.15.122:6817/1974398333","cluster_addr":"172.21.15.122:6819/1974398333","heartbeat_back_addr":"172.21.15.122:6823/1974398333","heartbeat_front_addr":"172.21.15.122:6821/1974398333","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T21:40:15.766648+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T21:40:15.457496+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T21:40:14.767039+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T21:40:15.355570+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T21:40:15.335729+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T21:40:15.614733+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T21:40:28.020 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-21T21:40:28.020 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T21:40:28.295 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:40:28.295 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":19,"fsid":"a0fdcfc2-ddd2-4e50-81bb-876d940f3578","created":"2024-09-21T21:40:07.413917+0000","modified":"2024-09-21T21:40:27.373092+0000","last_up_change":"2024-09-21T21:40:18.340430+0000","last_in_change":"2024-09-21T21:40:09.029953+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T21:40:18.321328+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-21T21:40:24.235658+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":19,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9869830-8e74-4b02-963d-ab630a0ee286","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1657552338},{"type":"v1","addr":"172.21.15.81:6809","nonce":1657552338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1657552338},{"type":"v1","addr":"172.21.15.81:6811","nonce":1657552338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1657552338},{"type":"v1","addr":"172.21.15.81:6815","nonce":1657552338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1657552338},{"type":"v1","addr":"172.21.15.81:6813","nonce":1657552338}]},"public_addr":"172.21.15.81:6809/1657552338","cluster_addr":"172.21.15.81:6811/1657552338","heartbeat_back_addr":"172.21.15.81:6815/1657552338","heartbeat_front_addr":"172.21.15.81:6813/1657552338","state":["exists","up"]},{"osd":1,"uuid":"b189819d-69a3-4e95-96e7-96dc4809ac80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2864159502},{"type":"v1","addr":"172.21.15.81:6817","nonce":2864159502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2864159502},{"type":"v1","addr":"172.21.15.81:6819","nonce":2864159502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2864159502},{"type":"v1","addr":"172.21.15.81:6823","nonce":2864159502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2864159502},{"type":"v1","addr":"172.21.15.81:6821","nonce":2864159502}]},"public_addr":"172.21.15.81:6817/2864159502","cluster_addr":"172.21.15.81:6819/2864159502","heartbeat_back_addr":"172.21.15.81:6823/2864159502","heartbeat_front_addr":"172.21.15.81:6821/2864159502","state":["exists","up"]},{"osd":2,"uuid":"8437b99a-1297-4903-ab66-95c5cba2b2bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3023160932},{"type":"v1","addr":"172.21.15.81:6801","nonce":3023160932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3023160932},{"type":"v1","addr":"172.21.15.81:6803","nonce":3023160932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3023160932},{"type":"v1","addr":"172.21.15.81:6807","nonce":3023160932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3023160932},{"type":"v1","addr":"172.21.15.81:6805","nonce":3023160932}]},"public_addr":"172.21.15.81:6801/3023160932","cluster_addr":"172.21.15.81:6803/3023160932","heartbeat_back_addr":"172.21.15.81:6807/3023160932","heartbeat_front_addr":"172.21.15.81:6805/3023160932","state":["exists","up"]},{"osd":3,"uuid":"e8848768-c1aa-445b-9554-dc6aa4eefc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":2681016844},{"type":"v1","addr":"172.21.15.122:6809","nonce":2681016844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2681016844},{"type":"v1","addr":"172.21.15.122:6811","nonce":2681016844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2681016844},{"type":"v1","addr":"172.21.15.122:6815","nonce":2681016844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2681016844},{"type":"v1","addr":"172.21.15.122:6813","nonce":2681016844}]},"public_addr":"172.21.15.122:6809/2681016844","cluster_addr":"172.21.15.122:6811/2681016844","heartbeat_back_addr":"172.21.15.122:6815/2681016844","heartbeat_front_addr":"172.21.15.122:6813/2681016844","state":["exists","up"]},{"osd":4,"uuid":"b1d8e85d-a207-4ba6-9322-61d07812083e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":1095023540},{"type":"v1","addr":"172.21.15.122:6801","nonce":1095023540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":1095023540},{"type":"v1","addr":"172.21.15.122:6803","nonce":1095023540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":1095023540},{"type":"v1","addr":"172.21.15.122:6807","nonce":1095023540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":1095023540},{"type":"v1","addr":"172.21.15.122:6805","nonce":1095023540}]},"public_addr":"172.21.15.122:6801/1095023540","cluster_addr":"172.21.15.122:6803/1095023540","heartbeat_back_addr":"172.21.15.122:6807/1095023540","heartbeat_front_addr":"172.21.15.122:6805/1095023540","state":["exists","up"]},{"osd":5,"uuid":"3ab0450b-d8fd-4ef2-a298-08d5aa2ea32f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1974398333},{"type":"v1","addr":"172.21.15.122:6817","nonce":1974398333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1974398333},{"type":"v1","addr":"172.21.15.122:6819","nonce":1974398333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1974398333},{"type":"v1","addr":"172.21.15.122:6823","nonce":1974398333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1974398333},{"type":"v1","addr":"172.21.15.122:6821","nonce":1974398333}]},"public_addr":"172.21.15.122:6817/1974398333","cluster_addr":"172.21.15.122:6819/1974398333","heartbeat_back_addr":"172.21.15.122:6823/1974398333","heartbeat_front_addr":"172.21.15.122:6821/1974398333","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T21:40:15.766648+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T21:40:15.457496+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T21:40:14.767039+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T21:40:15.355570+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T21:40:15.335729+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T21:40:15.614733+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T21:40:28.308 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-21T21:40:28.308 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-21T21:40:28.309 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-21T21:40:28.309 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-21T21:40:28.309 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-21T21:40:28.310 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-21T21:40:28.473 INFO:teuthology.orchestra.run.smithi081.stdout:51539607557 2024-09-21T21:40:28.473 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-21T21:40:28.496 INFO:teuthology.orchestra.run.smithi081.stdout:55834574852 2024-09-21T21:40:28.496 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-21T21:40:28.503 INFO:teuthology.orchestra.run.smithi081.stdout:55834574852 2024-09-21T21:40:28.503 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-21T21:40:28.509 INFO:teuthology.orchestra.run.smithi081.stdout:51539607558 2024-09-21T21:40:28.509 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-21T21:40:28.521 INFO:teuthology.orchestra.run.smithi081.stdout:47244640261 2024-09-21T21:40:28.522 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-21T21:40:28.522 INFO:teuthology.orchestra.run.smithi081.stdout:51539607557 2024-09-21T21:40:28.523 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-21T21:40:28.813 INFO:teuthology.orchestra.run.smithi081.stdout:51539607555 2024-09-21T21:40:28.826 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607557 got 51539607555 for osd.5 2024-09-21T21:40:28.877 INFO:teuthology.orchestra.run.smithi081.stdout:51539607556 2024-09-21T21:40:28.890 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607558 got 51539607556 for osd.0 2024-09-21T21:40:28.975 INFO:teuthology.orchestra.run.smithi081.stdout:55834574851 2024-09-21T21:40:28.977 INFO:teuthology.orchestra.run.smithi081.stdout:55834574851 2024-09-21T21:40:28.990 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.3 2024-09-21T21:40:28.992 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.4 2024-09-21T21:40:29.020 INFO:teuthology.orchestra.run.smithi081.stdout:51539607555 2024-09-21T21:40:29.032 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607557 got 51539607555 for osd.1 2024-09-21T21:40:29.042 INFO:teuthology.orchestra.run.smithi081.stdout:47244640260 2024-09-21T21:40:29.054 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640261 got 47244640260 for osd.2 2024-09-21T21:40:29.827 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-21T21:40:29.891 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-21T21:40:29.991 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-21T21:40:29.992 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-21T21:40:30.033 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-21T21:40:30.054 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-21T21:40:30.146 INFO:teuthology.orchestra.run.smithi081.stdout:51539607555 2024-09-21T21:40:30.161 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607557 got 51539607555 for osd.5 2024-09-21T21:40:30.333 INFO:teuthology.orchestra.run.smithi081.stdout:51539607556 2024-09-21T21:40:30.351 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607558 got 51539607556 for osd.0 2024-09-21T21:40:30.410 INFO:teuthology.orchestra.run.smithi081.stdout:55834574853 2024-09-21T21:40:30.414 INFO:teuthology.orchestra.run.smithi081.stdout:47244640261 2024-09-21T21:40:30.427 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574853 for osd.4 2024-09-21T21:40:30.427 DEBUG:teuthology.parallel:result is None 2024-09-21T21:40:30.430 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640261 got 47244640261 for osd.2 2024-09-21T21:40:30.430 DEBUG:teuthology.parallel:result is None 2024-09-21T21:40:30.441 INFO:teuthology.orchestra.run.smithi081.stdout:55834574853 2024-09-21T21:40:30.455 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574853 for osd.3 2024-09-21T21:40:30.455 DEBUG:teuthology.parallel:result is None 2024-09-21T21:40:30.510 INFO:teuthology.orchestra.run.smithi081.stdout:51539607557 2024-09-21T21:40:30.524 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607557 got 51539607557 for osd.1 2024-09-21T21:40:30.524 DEBUG:teuthology.parallel:result is None 2024-09-21T21:40:31.162 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-21T21:40:31.352 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-21T21:40:31.447 INFO:teuthology.orchestra.run.smithi081.stdout:51539607557 2024-09-21T21:40:31.460 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607557 got 51539607557 for osd.5 2024-09-21T21:40:31.460 DEBUG:teuthology.parallel:result is None 2024-09-21T21:40:31.630 INFO:teuthology.orchestra.run.smithi081.stdout:51539607558 2024-09-21T21:40:31.642 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607558 got 51539607558 for osd.0 2024-09-21T21:40:31.642 DEBUG:teuthology.parallel:result is None 2024-09-21T21:40:31.642 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T21:40:31.642 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T21:40:31.914 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:40:31.915 INFO:teuthology.orchestra.run.smithi081.stderr:dumped all 2024-09-21T21:40:31.926 INFO:teuthology.orchestra.run.smithi081.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-09-21T21:40:30.293392+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":163368,"kb_used_data":1180,"kb_used_omap":9,"kb_used_meta":162038,"kb_avail":566067672,"statfs":{"total":579820584960,"available":579653296128,"internally_reserved":0,"allocated":1208320,"data_stored":1529314,"data_compressed":64790,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":9568,"internal_metadata":165927584},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.930475"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":17,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T21:40:27.380563+0000","last_change":"2024-09-21T21:40:27.380563+0000","last_active":"2024-09-21T21:40:27.380563+0000","last_peered":"2024-09-21T21:40:27.380563+0000","last_clean":"2024-09-21T21:40:27.380563+0000","last_became_active":"2024-09-21T21:40:25.501200+0000","last_became_peered":"2024-09-21T21:40:25.501200+0000","last_unstale":"2024-09-21T21:40:27.380563+0000","last_undegraded":"2024-09-21T21:40:27.380563+0000","last_fullsized":"2024-09-21T21:40:27.380563+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:03:24.900063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072643899999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T21:40:27.379903+0000","last_change":"2024-09-21T21:40:27.379903+0000","last_active":"2024-09-21T21:40:27.379903+0000","last_peered":"2024-09-21T21:40:27.379903+0000","last_clean":"2024-09-21T21:40:27.379903+0000","last_became_active":"2024-09-21T21:40:25.371095+0000","last_became_peered":"2024-09-21T21:40:25.371095+0000","last_unstale":"2024-09-21T21:40:27.379903+0000","last_undegraded":"2024-09-21T21:40:27.379903+0000","last_fullsized":"2024-09-21T21:40:27.379903+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:08:09.434308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062280799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":17,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T21:40:27.380218+0000","last_change":"2024-09-21T21:40:27.380218+0000","last_active":"2024-09-21T21:40:27.380218+0000","last_peered":"2024-09-21T21:40:27.380218+0000","last_clean":"2024-09-21T21:40:27.380218+0000","last_became_active":"2024-09-21T21:40:25.500143+0000","last_became_peered":"2024-09-21T21:40:25.500143+0000","last_unstale":"2024-09-21T21:40:27.380218+0000","last_undegraded":"2024-09-21T21:40:27.380218+0000","last_fullsized":"2024-09-21T21:40:27.380218+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:11:42.747388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042715899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T21:40:27.380005+0000","last_change":"2024-09-21T21:40:27.380005+0000","last_active":"2024-09-21T21:40:27.380005+0000","last_peered":"2024-09-21T21:40:27.380005+0000","last_clean":"2024-09-21T21:40:27.380005+0000","last_became_active":"2024-09-21T21:40:25.500109+0000","last_became_peered":"2024-09-21T21:40:25.500109+0000","last_unstale":"2024-09-21T21:40:27.380005+0000","last_undegraded":"2024-09-21T21:40:27.380005+0000","last_fullsized":"2024-09-21T21:40:27.380005+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:21:10.631127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070669899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"19'2","reported_seq":21,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T21:40:27.383288+0000","last_change":"2024-09-21T21:40:27.382736+0000","last_active":"2024-09-21T21:40:27.383288+0000","last_peered":"2024-09-21T21:40:27.383288+0000","last_clean":"2024-09-21T21:40:27.383288+0000","last_became_active":"2024-09-21T21:40:25.371972+0000","last_became_peered":"2024-09-21T21:40:25.371972+0000","last_unstale":"2024-09-21T21:40:27.383288+0000","last_undegraded":"2024-09-21T21:40:27.383288+0000","last_fullsized":"2024-09-21T21:40:27.383288+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:45:30.417508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000906268,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":17,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T21:40:27.380557+0000","last_change":"2024-09-21T21:40:27.380792+0000","last_active":"2024-09-21T21:40:27.380557+0000","last_peered":"2024-09-21T21:40:27.380557+0000","last_clean":"2024-09-21T21:40:27.380557+0000","last_became_active":"2024-09-21T21:40:25.371578+0000","last_became_peered":"2024-09-21T21:40:25.371578+0000","last_unstale":"2024-09-21T21:40:27.380557+0000","last_undegraded":"2024-09-21T21:40:27.380557+0000","last_fullsized":"2024-09-21T21:40:27.380557+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:16:44.813324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000459858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":17,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T21:40:27.380219+0000","last_change":"2024-09-21T21:40:27.380219+0000","last_active":"2024-09-21T21:40:27.380219+0000","last_peered":"2024-09-21T21:40:27.380219+0000","last_clean":"2024-09-21T21:40:27.380219+0000","last_became_active":"2024-09-21T21:40:25.371451+0000","last_became_peered":"2024-09-21T21:40:25.371451+0000","last_unstale":"2024-09-21T21:40:27.380219+0000","last_undegraded":"2024-09-21T21:40:27.380219+0000","last_fullsized":"2024-09-21T21:40:27.380219+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:05:41.445313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000438863,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"17'1","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T21:40:27.382253+0000","last_change":"2024-09-21T21:40:27.382253+0000","last_active":"2024-09-21T21:40:27.382253+0000","last_peered":"2024-09-21T21:40:27.382253+0000","last_clean":"2024-09-21T21:40:27.382253+0000","last_became_active":"2024-09-21T21:40:25.372202+0000","last_became_peered":"2024-09-21T21:40:25.372202+0000","last_unstale":"2024-09-21T21:40:27.382253+0000","last_undegraded":"2024-09-21T21:40:27.382253+0000","last_fullsized":"2024-09-21T21:40:27.382253+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:19:18.518621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036803999999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"14'117","reported_seq":169,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T21:40:27.379629+0000","last_change":"2024-09-21T21:40:19.352502+0000","last_active":"2024-09-21T21:40:27.379629+0000","last_peered":"2024-09-21T21:40:27.379629+0000","last_clean":"2024-09-21T21:40:27.379629+0000","last_became_active":"2024-09-21T21:40:19.352227+0000","last_became_peered":"2024-09-21T21:40:19.352227+0000","last_unstale":"2024-09-21T21:40:27.379629+0000","last_undegraded":"2024-09-21T21:40:27.379629+0000","last_fullsized":"2024-09-21T21:40:27.379629+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:18.340430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:18.340430+0000","last_clean_scrub_stamp":"2024-09-21T21:40:18.340430+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:19:13.902196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64790,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":180224,"data_stored":58918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":245760,"data_stored":649286,"data_compressed":32395,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":12,"seq":51539607557,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":184320,"data_stored":58937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607557,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":184320,"data_stored":58937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607558,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":233472,"data_stored":644318,"data_compressed":32395,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640261,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":180224,"data_stored":58918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32395,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32395,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T21:40:31.927 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T21:40:32.199 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:40:32.199 INFO:teuthology.orchestra.run.smithi081.stderr:dumped all 2024-09-21T21:40:32.214 INFO:teuthology.orchestra.run.smithi081.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-09-21T21:40:30.293392+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":163368,"kb_used_data":1180,"kb_used_omap":9,"kb_used_meta":162038,"kb_avail":566067672,"statfs":{"total":579820584960,"available":579653296128,"internally_reserved":0,"allocated":1208320,"data_stored":1529314,"data_compressed":64790,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":9568,"internal_metadata":165927584},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.930475"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":17,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T21:40:27.380563+0000","last_change":"2024-09-21T21:40:27.380563+0000","last_active":"2024-09-21T21:40:27.380563+0000","last_peered":"2024-09-21T21:40:27.380563+0000","last_clean":"2024-09-21T21:40:27.380563+0000","last_became_active":"2024-09-21T21:40:25.501200+0000","last_became_peered":"2024-09-21T21:40:25.501200+0000","last_unstale":"2024-09-21T21:40:27.380563+0000","last_undegraded":"2024-09-21T21:40:27.380563+0000","last_fullsized":"2024-09-21T21:40:27.380563+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:03:24.900063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072643899999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T21:40:27.379903+0000","last_change":"2024-09-21T21:40:27.379903+0000","last_active":"2024-09-21T21:40:27.379903+0000","last_peered":"2024-09-21T21:40:27.379903+0000","last_clean":"2024-09-21T21:40:27.379903+0000","last_became_active":"2024-09-21T21:40:25.371095+0000","last_became_peered":"2024-09-21T21:40:25.371095+0000","last_unstale":"2024-09-21T21:40:27.379903+0000","last_undegraded":"2024-09-21T21:40:27.379903+0000","last_fullsized":"2024-09-21T21:40:27.379903+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:08:09.434308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062280799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":17,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T21:40:27.380218+0000","last_change":"2024-09-21T21:40:27.380218+0000","last_active":"2024-09-21T21:40:27.380218+0000","last_peered":"2024-09-21T21:40:27.380218+0000","last_clean":"2024-09-21T21:40:27.380218+0000","last_became_active":"2024-09-21T21:40:25.500143+0000","last_became_peered":"2024-09-21T21:40:25.500143+0000","last_unstale":"2024-09-21T21:40:27.380218+0000","last_undegraded":"2024-09-21T21:40:27.380218+0000","last_fullsized":"2024-09-21T21:40:27.380218+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:11:42.747388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042715899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T21:40:27.380005+0000","last_change":"2024-09-21T21:40:27.380005+0000","last_active":"2024-09-21T21:40:27.380005+0000","last_peered":"2024-09-21T21:40:27.380005+0000","last_clean":"2024-09-21T21:40:27.380005+0000","last_became_active":"2024-09-21T21:40:25.500109+0000","last_became_peered":"2024-09-21T21:40:25.500109+0000","last_unstale":"2024-09-21T21:40:27.380005+0000","last_undegraded":"2024-09-21T21:40:27.380005+0000","last_fullsized":"2024-09-21T21:40:27.380005+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:21:10.631127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070669899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"19'2","reported_seq":21,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T21:40:27.383288+0000","last_change":"2024-09-21T21:40:27.382736+0000","last_active":"2024-09-21T21:40:27.383288+0000","last_peered":"2024-09-21T21:40:27.383288+0000","last_clean":"2024-09-21T21:40:27.383288+0000","last_became_active":"2024-09-21T21:40:25.371972+0000","last_became_peered":"2024-09-21T21:40:25.371972+0000","last_unstale":"2024-09-21T21:40:27.383288+0000","last_undegraded":"2024-09-21T21:40:27.383288+0000","last_fullsized":"2024-09-21T21:40:27.383288+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:45:30.417508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000906268,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":17,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T21:40:27.380557+0000","last_change":"2024-09-21T21:40:27.380792+0000","last_active":"2024-09-21T21:40:27.380557+0000","last_peered":"2024-09-21T21:40:27.380557+0000","last_clean":"2024-09-21T21:40:27.380557+0000","last_became_active":"2024-09-21T21:40:25.371578+0000","last_became_peered":"2024-09-21T21:40:25.371578+0000","last_unstale":"2024-09-21T21:40:27.380557+0000","last_undegraded":"2024-09-21T21:40:27.380557+0000","last_fullsized":"2024-09-21T21:40:27.380557+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:16:44.813324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000459858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":17,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T21:40:27.380219+0000","last_change":"2024-09-21T21:40:27.380219+0000","last_active":"2024-09-21T21:40:27.380219+0000","last_peered":"2024-09-21T21:40:27.380219+0000","last_clean":"2024-09-21T21:40:27.380219+0000","last_became_active":"2024-09-21T21:40:25.371451+0000","last_became_peered":"2024-09-21T21:40:25.371451+0000","last_unstale":"2024-09-21T21:40:27.380219+0000","last_undegraded":"2024-09-21T21:40:27.380219+0000","last_fullsized":"2024-09-21T21:40:27.380219+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:05:41.445313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000438863,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"17'1","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T21:40:27.382253+0000","last_change":"2024-09-21T21:40:27.382253+0000","last_active":"2024-09-21T21:40:27.382253+0000","last_peered":"2024-09-21T21:40:27.382253+0000","last_clean":"2024-09-21T21:40:27.382253+0000","last_became_active":"2024-09-21T21:40:25.372202+0000","last_became_peered":"2024-09-21T21:40:25.372202+0000","last_unstale":"2024-09-21T21:40:27.382253+0000","last_undegraded":"2024-09-21T21:40:27.382253+0000","last_fullsized":"2024-09-21T21:40:27.382253+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:19:18.518621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036803999999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"14'117","reported_seq":169,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T21:40:27.379629+0000","last_change":"2024-09-21T21:40:19.352502+0000","last_active":"2024-09-21T21:40:27.379629+0000","last_peered":"2024-09-21T21:40:27.379629+0000","last_clean":"2024-09-21T21:40:27.379629+0000","last_became_active":"2024-09-21T21:40:19.352227+0000","last_became_peered":"2024-09-21T21:40:19.352227+0000","last_unstale":"2024-09-21T21:40:27.379629+0000","last_undegraded":"2024-09-21T21:40:27.379629+0000","last_fullsized":"2024-09-21T21:40:27.379629+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:18.340430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:18.340430+0000","last_clean_scrub_stamp":"2024-09-21T21:40:18.340430+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:19:13.902196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64790,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":180224,"data_stored":58918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":245760,"data_stored":649286,"data_compressed":32395,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":12,"seq":51539607557,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":184320,"data_stored":58937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607557,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":184320,"data_stored":58937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607558,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":233472,"data_stored":644318,"data_compressed":32395,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640261,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":180224,"data_stored":58918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32395,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32395,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T21:40:32.215 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T21:40:32.215 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-21T21:40:32.215 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-21T21:40:32.215 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-21T21:40:32.525 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:40:32.525 INFO:teuthology.orchestra.run.smithi081.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-21T21:40:32.537 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-21T21:40:32.537 INFO:teuthology.run_tasks:Running task thrashosds... 2024-09-21T21:40:32.570 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-21T21:40:32.570 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'extra_packages': ['rbd-nbd'], 'flavor': 'default', 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7'}} 2024-09-21T21:40:32.570 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2024-09-21T21:40:32.570 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-09-21T21:40:32.863 INFO:tasks.thrashosds:Beginning thrashosds... 2024-09-21T21:40:32.863 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T21:40:32.864 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T21:40:33.137 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:40:33.137 INFO:teuthology.orchestra.run.smithi081.stderr:dumped all 2024-09-21T21:40:33.149 INFO:teuthology.orchestra.run.smithi081.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-21T21:40:32.293785+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":163368,"kb_used_data":1180,"kb_used_omap":9,"kb_used_meta":162038,"kb_avail":566067672,"statfs":{"total":579820584960,"available":579653296128,"internally_reserved":0,"allocated":1208320,"data_stored":1529314,"data_compressed":64790,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":9568,"internal_metadata":165927584},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001150"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":17,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T21:40:27.380563+0000","last_change":"2024-09-21T21:40:27.380563+0000","last_active":"2024-09-21T21:40:27.380563+0000","last_peered":"2024-09-21T21:40:27.380563+0000","last_clean":"2024-09-21T21:40:27.380563+0000","last_became_active":"2024-09-21T21:40:25.501200+0000","last_became_peered":"2024-09-21T21:40:25.501200+0000","last_unstale":"2024-09-21T21:40:27.380563+0000","last_undegraded":"2024-09-21T21:40:27.380563+0000","last_fullsized":"2024-09-21T21:40:27.380563+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:03:24.900063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072643899999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T21:40:27.379903+0000","last_change":"2024-09-21T21:40:27.379903+0000","last_active":"2024-09-21T21:40:27.379903+0000","last_peered":"2024-09-21T21:40:27.379903+0000","last_clean":"2024-09-21T21:40:27.379903+0000","last_became_active":"2024-09-21T21:40:25.371095+0000","last_became_peered":"2024-09-21T21:40:25.371095+0000","last_unstale":"2024-09-21T21:40:27.379903+0000","last_undegraded":"2024-09-21T21:40:27.379903+0000","last_fullsized":"2024-09-21T21:40:27.379903+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:08:09.434308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062280799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":17,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T21:40:27.380218+0000","last_change":"2024-09-21T21:40:27.380218+0000","last_active":"2024-09-21T21:40:27.380218+0000","last_peered":"2024-09-21T21:40:27.380218+0000","last_clean":"2024-09-21T21:40:27.380218+0000","last_became_active":"2024-09-21T21:40:25.500143+0000","last_became_peered":"2024-09-21T21:40:25.500143+0000","last_unstale":"2024-09-21T21:40:27.380218+0000","last_undegraded":"2024-09-21T21:40:27.380218+0000","last_fullsized":"2024-09-21T21:40:27.380218+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:11:42.747388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042715899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T21:40:27.380005+0000","last_change":"2024-09-21T21:40:27.380005+0000","last_active":"2024-09-21T21:40:27.380005+0000","last_peered":"2024-09-21T21:40:27.380005+0000","last_clean":"2024-09-21T21:40:27.380005+0000","last_became_active":"2024-09-21T21:40:25.500109+0000","last_became_peered":"2024-09-21T21:40:25.500109+0000","last_unstale":"2024-09-21T21:40:27.380005+0000","last_undegraded":"2024-09-21T21:40:27.380005+0000","last_fullsized":"2024-09-21T21:40:27.380005+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:21:10.631127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070669899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"19'2","reported_seq":21,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T21:40:27.383288+0000","last_change":"2024-09-21T21:40:27.382736+0000","last_active":"2024-09-21T21:40:27.383288+0000","last_peered":"2024-09-21T21:40:27.383288+0000","last_clean":"2024-09-21T21:40:27.383288+0000","last_became_active":"2024-09-21T21:40:25.371972+0000","last_became_peered":"2024-09-21T21:40:25.371972+0000","last_unstale":"2024-09-21T21:40:27.383288+0000","last_undegraded":"2024-09-21T21:40:27.383288+0000","last_fullsized":"2024-09-21T21:40:27.383288+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:45:30.417508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000906268,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":17,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T21:40:27.380557+0000","last_change":"2024-09-21T21:40:27.380792+0000","last_active":"2024-09-21T21:40:27.380557+0000","last_peered":"2024-09-21T21:40:27.380557+0000","last_clean":"2024-09-21T21:40:27.380557+0000","last_became_active":"2024-09-21T21:40:25.371578+0000","last_became_peered":"2024-09-21T21:40:25.371578+0000","last_unstale":"2024-09-21T21:40:27.380557+0000","last_undegraded":"2024-09-21T21:40:27.380557+0000","last_fullsized":"2024-09-21T21:40:27.380557+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:16:44.813324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000459858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":17,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T21:40:27.380219+0000","last_change":"2024-09-21T21:40:27.380219+0000","last_active":"2024-09-21T21:40:27.380219+0000","last_peered":"2024-09-21T21:40:27.380219+0000","last_clean":"2024-09-21T21:40:27.380219+0000","last_became_active":"2024-09-21T21:40:25.371451+0000","last_became_peered":"2024-09-21T21:40:25.371451+0000","last_unstale":"2024-09-21T21:40:27.380219+0000","last_undegraded":"2024-09-21T21:40:27.380219+0000","last_fullsized":"2024-09-21T21:40:27.380219+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:05:41.445313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000438863,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"17'1","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T21:40:27.382253+0000","last_change":"2024-09-21T21:40:27.382253+0000","last_active":"2024-09-21T21:40:27.382253+0000","last_peered":"2024-09-21T21:40:27.382253+0000","last_clean":"2024-09-21T21:40:27.382253+0000","last_became_active":"2024-09-21T21:40:25.372202+0000","last_became_peered":"2024-09-21T21:40:25.372202+0000","last_unstale":"2024-09-21T21:40:27.382253+0000","last_undegraded":"2024-09-21T21:40:27.382253+0000","last_fullsized":"2024-09-21T21:40:27.382253+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:19:18.518621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036803999999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"14'117","reported_seq":169,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T21:40:27.379629+0000","last_change":"2024-09-21T21:40:19.352502+0000","last_active":"2024-09-21T21:40:27.379629+0000","last_peered":"2024-09-21T21:40:27.379629+0000","last_clean":"2024-09-21T21:40:27.379629+0000","last_became_active":"2024-09-21T21:40:19.352227+0000","last_became_peered":"2024-09-21T21:40:19.352227+0000","last_unstale":"2024-09-21T21:40:27.379629+0000","last_undegraded":"2024-09-21T21:40:27.379629+0000","last_fullsized":"2024-09-21T21:40:27.379629+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:18.340430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:18.340430+0000","last_clean_scrub_stamp":"2024-09-21T21:40:18.340430+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:19:13.902196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64790,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":180224,"data_stored":58918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":245760,"data_stored":649286,"data_compressed":32395,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":12,"seq":51539607557,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":184320,"data_stored":58937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607557,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":184320,"data_stored":58937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607558,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":233472,"data_stored":644318,"data_compressed":32395,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640262,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":180224,"data_stored":58918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32395,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32395,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T21:40:33.150 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T21:40:33.424 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:40:33.424 INFO:teuthology.orchestra.run.smithi081.stderr:dumped all 2024-09-21T21:40:33.436 INFO:teuthology.orchestra.run.smithi081.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-21T21:40:32.293785+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":163368,"kb_used_data":1180,"kb_used_omap":9,"kb_used_meta":162038,"kb_avail":566067672,"statfs":{"total":579820584960,"available":579653296128,"internally_reserved":0,"allocated":1208320,"data_stored":1529314,"data_compressed":64790,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":9568,"internal_metadata":165927584},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001150"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":17,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T21:40:27.380563+0000","last_change":"2024-09-21T21:40:27.380563+0000","last_active":"2024-09-21T21:40:27.380563+0000","last_peered":"2024-09-21T21:40:27.380563+0000","last_clean":"2024-09-21T21:40:27.380563+0000","last_became_active":"2024-09-21T21:40:25.501200+0000","last_became_peered":"2024-09-21T21:40:25.501200+0000","last_unstale":"2024-09-21T21:40:27.380563+0000","last_undegraded":"2024-09-21T21:40:27.380563+0000","last_fullsized":"2024-09-21T21:40:27.380563+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:03:24.900063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072643899999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T21:40:27.379903+0000","last_change":"2024-09-21T21:40:27.379903+0000","last_active":"2024-09-21T21:40:27.379903+0000","last_peered":"2024-09-21T21:40:27.379903+0000","last_clean":"2024-09-21T21:40:27.379903+0000","last_became_active":"2024-09-21T21:40:25.371095+0000","last_became_peered":"2024-09-21T21:40:25.371095+0000","last_unstale":"2024-09-21T21:40:27.379903+0000","last_undegraded":"2024-09-21T21:40:27.379903+0000","last_fullsized":"2024-09-21T21:40:27.379903+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:08:09.434308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062280799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":17,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T21:40:27.380218+0000","last_change":"2024-09-21T21:40:27.380218+0000","last_active":"2024-09-21T21:40:27.380218+0000","last_peered":"2024-09-21T21:40:27.380218+0000","last_clean":"2024-09-21T21:40:27.380218+0000","last_became_active":"2024-09-21T21:40:25.500143+0000","last_became_peered":"2024-09-21T21:40:25.500143+0000","last_unstale":"2024-09-21T21:40:27.380218+0000","last_undegraded":"2024-09-21T21:40:27.380218+0000","last_fullsized":"2024-09-21T21:40:27.380218+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:11:42.747388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042715899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T21:40:27.380005+0000","last_change":"2024-09-21T21:40:27.380005+0000","last_active":"2024-09-21T21:40:27.380005+0000","last_peered":"2024-09-21T21:40:27.380005+0000","last_clean":"2024-09-21T21:40:27.380005+0000","last_became_active":"2024-09-21T21:40:25.500109+0000","last_became_peered":"2024-09-21T21:40:25.500109+0000","last_unstale":"2024-09-21T21:40:27.380005+0000","last_undegraded":"2024-09-21T21:40:27.380005+0000","last_fullsized":"2024-09-21T21:40:27.380005+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:21:10.631127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070669899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"19'2","reported_seq":21,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T21:40:27.383288+0000","last_change":"2024-09-21T21:40:27.382736+0000","last_active":"2024-09-21T21:40:27.383288+0000","last_peered":"2024-09-21T21:40:27.383288+0000","last_clean":"2024-09-21T21:40:27.383288+0000","last_became_active":"2024-09-21T21:40:25.371972+0000","last_became_peered":"2024-09-21T21:40:25.371972+0000","last_unstale":"2024-09-21T21:40:27.383288+0000","last_undegraded":"2024-09-21T21:40:27.383288+0000","last_fullsized":"2024-09-21T21:40:27.383288+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:45:30.417508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000906268,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":17,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T21:40:27.380557+0000","last_change":"2024-09-21T21:40:27.380792+0000","last_active":"2024-09-21T21:40:27.380557+0000","last_peered":"2024-09-21T21:40:27.380557+0000","last_clean":"2024-09-21T21:40:27.380557+0000","last_became_active":"2024-09-21T21:40:25.371578+0000","last_became_peered":"2024-09-21T21:40:25.371578+0000","last_unstale":"2024-09-21T21:40:27.380557+0000","last_undegraded":"2024-09-21T21:40:27.380557+0000","last_fullsized":"2024-09-21T21:40:27.380557+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:16:44.813324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000459858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":17,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T21:40:27.380219+0000","last_change":"2024-09-21T21:40:27.380219+0000","last_active":"2024-09-21T21:40:27.380219+0000","last_peered":"2024-09-21T21:40:27.380219+0000","last_clean":"2024-09-21T21:40:27.380219+0000","last_became_active":"2024-09-21T21:40:25.371451+0000","last_became_peered":"2024-09-21T21:40:25.371451+0000","last_unstale":"2024-09-21T21:40:27.380219+0000","last_undegraded":"2024-09-21T21:40:27.380219+0000","last_fullsized":"2024-09-21T21:40:27.380219+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:05:41.445313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000438863,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"17'1","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T21:40:27.382253+0000","last_change":"2024-09-21T21:40:27.382253+0000","last_active":"2024-09-21T21:40:27.382253+0000","last_peered":"2024-09-21T21:40:27.382253+0000","last_clean":"2024-09-21T21:40:27.382253+0000","last_became_active":"2024-09-21T21:40:25.372202+0000","last_became_peered":"2024-09-21T21:40:25.372202+0000","last_unstale":"2024-09-21T21:40:27.382253+0000","last_undegraded":"2024-09-21T21:40:27.382253+0000","last_fullsized":"2024-09-21T21:40:27.382253+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:19:18.518621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036803999999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"14'117","reported_seq":169,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-21T21:40:27.379629+0000","last_change":"2024-09-21T21:40:19.352502+0000","last_active":"2024-09-21T21:40:27.379629+0000","last_peered":"2024-09-21T21:40:27.379629+0000","last_clean":"2024-09-21T21:40:27.379629+0000","last_became_active":"2024-09-21T21:40:19.352227+0000","last_became_peered":"2024-09-21T21:40:19.352227+0000","last_unstale":"2024-09-21T21:40:27.379629+0000","last_undegraded":"2024-09-21T21:40:27.379629+0000","last_fullsized":"2024-09-21T21:40:27.379629+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:18.340430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:18.340430+0000","last_clean_scrub_stamp":"2024-09-21T21:40:18.340430+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:19:13.902196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64790,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":180224,"data_stored":58918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":245760,"data_stored":649286,"data_compressed":32395,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":12,"seq":51539607557,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":184320,"data_stored":58937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607557,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":184320,"data_stored":58937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607558,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":233472,"data_stored":644318,"data_compressed":32395,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640262,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":180224,"data_stored":58918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32395,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32395,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T21:40:33.437 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T21:40:33.437 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-09-21T21:40:33.713 INFO:teuthology.orchestra.run.smithi081.stdout:epoch 20 2024-09-21T21:40:33.713 INFO:teuthology.orchestra.run.smithi081.stdout:fsid a0fdcfc2-ddd2-4e50-81bb-876d940f3578 2024-09-21T21:40:33.713 INFO:teuthology.orchestra.run.smithi081.stdout:created 2024-09-21T21:40:07.413917+0000 2024-09-21T21:40:33.714 INFO:teuthology.orchestra.run.smithi081.stdout:modified 2024-09-21T21:40:32.195382+0000 2024-09-21T21:40:33.714 INFO:teuthology.orchestra.run.smithi081.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-21T21:40:33.714 INFO:teuthology.orchestra.run.smithi081.stdout:crush_version 5 2024-09-21T21:40:33.714 INFO:teuthology.orchestra.run.smithi081.stdout:full_ratio 0.95 2024-09-21T21:40:33.714 INFO:teuthology.orchestra.run.smithi081.stdout:backfillfull_ratio 0.9 2024-09-21T21:40:33.714 INFO:teuthology.orchestra.run.smithi081.stdout:nearfull_ratio 0.85 2024-09-21T21:40:33.714 INFO:teuthology.orchestra.run.smithi081.stdout:require_min_compat_client luminous 2024-09-21T21:40:33.714 INFO:teuthology.orchestra.run.smithi081.stdout:min_compat_client jewel 2024-09-21T21:40:33.714 INFO:teuthology.orchestra.run.smithi081.stdout:require_osd_release squid 2024-09-21T21:40:33.714 INFO:teuthology.orchestra.run.smithi081.stdout:stretch_mode_enabled false 2024-09-21T21:40:33.714 INFO:teuthology.orchestra.run.smithi081.stdout:pool 1 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 15 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 6.06 2024-09-21T21:40:33.714 INFO:teuthology.orchestra.run.smithi081.stdout:pool 2 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 19 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd read_balance_score 2.25 2024-09-21T21:40:33.715 INFO:teuthology.orchestra.run.smithi081.stdout:max_osd 6 2024-09-21T21:40:33.715 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0 up in weight 1 up_from 12 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.81:6808/1657552338,v1:172.21.15.81:6809/1657552338] [v2:172.21.15.81:6810/1657552338,v1:172.21.15.81:6811/1657552338] exists,up a9869830-8e74-4b02-963d-ab630a0ee286 2024-09-21T21:40:33.715 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1 up in weight 1 up_from 12 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.81:6816/2864159502,v1:172.21.15.81:6817/2864159502] [v2:172.21.15.81:6818/2864159502,v1:172.21.15.81:6819/2864159502] exists,up b189819d-69a3-4e95-96e7-96dc4809ac80 2024-09-21T21:40:33.715 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2 up in weight 1 up_from 11 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.81:6800/3023160932,v1:172.21.15.81:6801/3023160932] [v2:172.21.15.81:6802/3023160932,v1:172.21.15.81:6803/3023160932] exists,up 8437b99a-1297-4903-ab66-95c5cba2b2bf 2024-09-21T21:40:33.715 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3 up in weight 1 up_from 13 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.122:6808/2681016844,v1:172.21.15.122:6809/2681016844] [v2:172.21.15.122:6810/2681016844,v1:172.21.15.122:6811/2681016844] exists,up e8848768-c1aa-445b-9554-dc6aa4eefc8d 2024-09-21T21:40:33.715 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.122:6800/1095023540,v1:172.21.15.122:6801/1095023540] [v2:172.21.15.122:6802/1095023540,v1:172.21.15.122:6803/1095023540] exists,up b1d8e85d-a207-4ba6-9322-61d07812083e 2024-09-21T21:40:33.715 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5 up in weight 1 up_from 12 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.122:6816/1974398333,v1:172.21.15.122:6817/1974398333] [v2:172.21.15.122:6818/1974398333,v1:172.21.15.122:6819/1974398333] exists,up 3ab0450b-d8fd-4ef2-a298-08d5aa2ea32f 2024-09-21T21:40:33.726 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 12 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.81:6808/1657552338,v1:172.21.15.81:6809/1657552338] [v2:172.21.15.81:6810/1657552338,v1:172.21.15.81:6811/1657552338] exists,up a9869830-8e74-4b02-963d-ab630a0ee286', 'osd.1 up in weight 1 up_from 12 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.81:6816/2864159502,v1:172.21.15.81:6817/2864159502] [v2:172.21.15.81:6818/2864159502,v1:172.21.15.81:6819/2864159502] exists,up b189819d-69a3-4e95-96e7-96dc4809ac80', 'osd.2 up in weight 1 up_from 11 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.81:6800/3023160932,v1:172.21.15.81:6801/3023160932] [v2:172.21.15.81:6802/3023160932,v1:172.21.15.81:6803/3023160932] exists,up 8437b99a-1297-4903-ab66-95c5cba2b2bf', 'osd.3 up in weight 1 up_from 13 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.122:6808/2681016844,v1:172.21.15.122:6809/2681016844] [v2:172.21.15.122:6810/2681016844,v1:172.21.15.122:6811/2681016844] exists,up e8848768-c1aa-445b-9554-dc6aa4eefc8d', 'osd.4 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.122:6800/1095023540,v1:172.21.15.122:6801/1095023540] [v2:172.21.15.122:6802/1095023540,v1:172.21.15.122:6803/1095023540] exists,up b1d8e85d-a207-4ba6-9322-61d07812083e', 'osd.5 up in weight 1 up_from 12 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.122:6816/1974398333,v1:172.21.15.122:6817/1974398333] [v2:172.21.15.122:6818/1974398333,v1:172.21.15.122:6819/1974398333] exists,up 3ab0450b-d8fd-4ef2-a298-08d5aa2ea32f'] 2024-09-21T21:40:33.726 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-09-21T21:40:33.824 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:40:33.824 INFO:teuthology.orchestra.run.smithi081.stdout: "name": "mon.a", 2024-09-21T21:40:33.824 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster": "ceph", 2024-09-21T21:40:33.824 INFO:teuthology.orchestra.run.smithi081.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-09-21T21:40:33.824 INFO:teuthology.orchestra.run.smithi081.stdout: "admin_socket_mode": "", 2024-09-21T21:40:33.824 INFO:teuthology.orchestra.run.smithi081.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-09-21T21:40:33.824 INFO:teuthology.orchestra.run.smithi081.stdout: "auth_client_required": "cephx, none", 2024-09-21T21:40:33.824 INFO:teuthology.orchestra.run.smithi081.stdout: "auth_cluster_required": "cephx", 2024-09-21T21:40:33.824 INFO:teuthology.orchestra.run.smithi081.stdout: "auth_debug": "true", 2024-09-21T21:40:33.824 INFO:teuthology.orchestra.run.smithi081.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-09-21T21:40:33.825 INFO:teuthology.orchestra.run.smithi081.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-09-21T21:40:33.825 INFO:teuthology.orchestra.run.smithi081.stdout: "auth_service_required": "cephx", 2024-09-21T21:40:33.825 INFO:teuthology.orchestra.run.smithi081.stdout: "auth_service_ticket_ttl": "240.000000", 2024-09-21T21:40:33.825 INFO:teuthology.orchestra.run.smithi081.stdout: "auth_supported": "cephx", 2024-09-21T21:40:33.825 INFO:teuthology.orchestra.run.smithi081.stdout: "bdev_aio": "true", 2024-09-21T21:40:33.825 INFO:teuthology.orchestra.run.smithi081.stdout: "bdev_aio_max_queue_depth": "1024", 2024-09-21T21:40:33.825 INFO:teuthology.orchestra.run.smithi081.stdout: "bdev_aio_poll_ms": "250", 2024-09-21T21:40:33.825 INFO:teuthology.orchestra.run.smithi081.stdout: "bdev_aio_reap_max": "16", 2024-09-21T21:40:33.825 INFO:teuthology.orchestra.run.smithi081.stdout: "bdev_async_discard": "false", 2024-09-21T21:40:33.825 INFO:teuthology.orchestra.run.smithi081.stdout: "bdev_block_size": "4096", 2024-09-21T21:40:33.825 INFO:teuthology.orchestra.run.smithi081.stdout: "bdev_debug_aio": "false", 2024-09-21T21:40:33.825 INFO:teuthology.orchestra.run.smithi081.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-09-21T21:40:33.825 INFO:teuthology.orchestra.run.smithi081.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-09-21T21:40:33.825 INFO:teuthology.orchestra.run.smithi081.stdout: "bdev_debug_inflight_ios": "false", 2024-09-21T21:40:33.825 INFO:teuthology.orchestra.run.smithi081.stdout: "bdev_enable_discard": "false", 2024-09-21T21:40:33.825 INFO:teuthology.orchestra.run.smithi081.stdout: "bdev_flock_retry": "3", 2024-09-21T21:40:33.826 INFO:teuthology.orchestra.run.smithi081.stdout: "bdev_flock_retry_interval": "0.100000", 2024-09-21T21:40:33.826 INFO:teuthology.orchestra.run.smithi081.stdout: "bdev_inject_crash": "0", 2024-09-21T21:40:33.826 INFO:teuthology.orchestra.run.smithi081.stdout: "bdev_inject_crash_flush_delay": "2", 2024-09-21T21:40:33.826 INFO:teuthology.orchestra.run.smithi081.stdout: "bdev_ioring": "false", 2024-09-21T21:40:33.826 INFO:teuthology.orchestra.run.smithi081.stdout: "bdev_ioring_hipri": "false", 2024-09-21T21:40:33.826 INFO:teuthology.orchestra.run.smithi081.stdout: "bdev_ioring_sqthread_poll": "false", 2024-09-21T21:40:33.826 INFO:teuthology.orchestra.run.smithi081.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-09-21T21:40:33.826 INFO:teuthology.orchestra.run.smithi081.stdout: "bdev_read_buffer_alignment": "4096", 2024-09-21T21:40:33.826 INFO:teuthology.orchestra.run.smithi081.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-09-21T21:40:33.826 INFO:teuthology.orchestra.run.smithi081.stdout: "bdev_stalled_read_warn_lifetime": "86400", 2024-09-21T21:40:33.826 INFO:teuthology.orchestra.run.smithi081.stdout: "bdev_stalled_read_warn_threshold": "1", 2024-09-21T21:40:33.826 INFO:teuthology.orchestra.run.smithi081.stdout: "bdev_type": "", 2024-09-21T21:40:33.826 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_alloc_size": "1048576", 2024-09-21T21:40:33.826 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_allocator": "hybrid", 2024-09-21T21:40:33.827 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_buffered_io": "true", 2024-09-21T21:40:33.827 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_check_for_zeros": "false", 2024-09-21T21:40:33.827 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_check_volume_selector_often": "false", 2024-09-21T21:40:33.827 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-09-21T21:40:33.827 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_compact_log_sync": "false", 2024-09-21T21:40:33.827 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-09-21T21:40:33.827 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-09-21T21:40:33.827 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_log_compact_min_size": "16777216", 2024-09-21T21:40:33.827 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_log_replay_check_allocations": "true", 2024-09-21T21:40:33.827 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_max_log_runway": "4194304", 2024-09-21T21:40:33.827 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_max_prefetch": "1048576", 2024-09-21T21:40:33.827 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_min_flush_size": "524288", 2024-09-21T21:40:33.827 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_min_log_runway": "1048576", 2024-09-21T21:40:33.827 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_replay_recovery": "false", 2024-09-21T21:40:33.827 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-09-21T21:40:33.827 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_shared_alloc_size": "65536", 2024-09-21T21:40:33.828 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_sync_write": "false", 2024-09-21T21:40:33.828 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-09-21T21:40:33.828 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-09-21T21:40:33.828 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-09-21T21:40:33.828 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_allocation_from_file": "true", 2024-09-21T21:40:33.828 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_allocator": "hybrid", 2024-09-21T21:40:33.828 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-09-21T21:40:33.828 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-09-21T21:40:33.828 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-09-21T21:40:33.828 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-09-21T21:40:33.828 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_bdev_label_multi": "true", 2024-09-21T21:40:33.828 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_bdev_label_multi_upgrade": "false", 2024-09-21T21:40:33.828 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_bdev_label_require_all": "true", 2024-09-21T21:40:33.829 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-09-21T21:40:33.829 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-09-21T21:40:33.829 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_blobid_prealloc": "10240", 2024-09-21T21:40:33.829 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_block_create": "true", 2024-09-21T21:40:33.829 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_block_db_create": "false", 2024-09-21T21:40:33.829 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_block_db_path": "", 2024-09-21T21:40:33.829 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_block_db_size": "0", 2024-09-21T21:40:33.829 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_block_path": "", 2024-09-21T21:40:33.829 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_block_preallocate_file": "false", 2024-09-21T21:40:33.829 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_block_size": "107374182400", 2024-09-21T21:40:33.829 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_block_wal_create": "false", 2024-09-21T21:40:33.829 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_block_wal_path": "", 2024-09-21T21:40:33.829 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_block_wal_size": "100663296", 2024-09-21T21:40:33.829 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_bluefs": "true", 2024-09-21T21:40:33.830 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-09-21T21:40:33.830 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_bluefs_env_mirror": "false", 2024-09-21T21:40:33.830 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-09-21T21:40:33.830 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-09-21T21:40:33.830 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-09-21T21:40:33.830 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-09-21T21:40:33.830 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-09-21T21:40:33.830 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-09-21T21:40:33.830 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_autotune": "true", 2024-09-21T21:40:33.830 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-09-21T21:40:33.830 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-09-21T21:40:33.830 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-09-21T21:40:33.830 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-09-21T21:40:33.830 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_size": "0", 2024-09-21T21:40:33.830 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-09-21T21:40:33.830 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-09-21T21:40:33.831 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-09-21T21:40:33.831 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-09-21T21:40:33.831 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_type": "2q", 2024-09-21T21:40:33.831 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-09-21T21:40:33.831 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_clone_cow": "true", 2024-09-21T21:40:33.831 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_compression_algorithm": "snappy", 2024-09-21T21:40:33.831 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_compression_max_blob_size": "0", 2024-09-21T21:40:33.831 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-09-21T21:40:33.831 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-09-21T21:40:33.831 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_compression_min_blob_size": "0", 2024-09-21T21:40:33.831 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-09-21T21:40:33.831 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-09-21T21:40:33.831 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_compression_mode": "none", 2024-09-21T21:40:33.831 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-09-21T21:40:33.831 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_csum_type": "crc32c", 2024-09-21T21:40:33.832 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_debug_enforce_settings": "default", 2024-09-21T21:40:33.832 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_debug_freelist": "false", 2024-09-21T21:40:33.832 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_debug_fsck_abort": "false", 2024-09-21T21:40:33.832 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-09-21T21:40:33.832 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-09-21T21:40:33.832 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_debug_inject_read_err": "false", 2024-09-21T21:40:33.832 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_debug_legacy_omap": "false", 2024-09-21T21:40:33.832 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-09-21T21:40:33.832 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-09-21T21:40:33.832 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-09-21T21:40:33.832 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-09-21T21:40:33.832 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-09-21T21:40:33.832 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-09-21T21:40:33.833 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-09-21T21:40:33.833 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_debug_small_allocations": "0", 2024-09-21T21:40:33.833 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-09-21T21:40:33.833 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_default_buffered_read": "true", 2024-09-21T21:40:33.833 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_default_buffered_write": "false", 2024-09-21T21:40:33.833 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_deferred_batch_ops": "0", 2024-09-21T21:40:33.833 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-09-21T21:40:33.833 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-09-21T21:40:33.833 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_elastic_shared_blobs": "true", 2024-09-21T21:40:33.833 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-09-21T21:40:33.833 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-09-21T21:40:33.833 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-09-21T21:40:33.833 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-09-21T21:40:33.833 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-09-21T21:40:33.833 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_fail_eio": "false", 2024-09-21T21:40:33.833 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-09-21T21:40:33.834 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-09-21T21:40:33.834 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-09-21T21:40:33.834 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-09-21T21:40:33.834 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_fsck_on_mkfs": "true", 2024-09-21T21:40:33.834 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-09-21T21:40:33.834 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_fsck_on_mount": "false", 2024-09-21T21:40:33.834 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-09-21T21:40:33.834 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_fsck_on_umount": "false", 2024-09-21T21:40:33.834 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-09-21T21:40:33.834 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-09-21T21:40:33.834 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-09-21T21:40:33.834 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-09-21T21:40:33.834 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-09-21T21:40:33.834 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-09-21T21:40:33.834 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-09-21T21:40:33.835 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-09-21T21:40:33.835 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_ignore_data_csum": "false", 2024-09-21T21:40:33.835 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-09-21T21:40:33.835 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_kvbackend": "rocksdb", 2024-09-21T21:40:33.835 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-09-21T21:40:33.835 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-09-21T21:40:33.835 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_log_op_age": "5.000000", 2024-09-21T21:40:33.835 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_max_alloc_size": "0", 2024-09-21T21:40:33.835 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_max_blob_size": "0", 2024-09-21T21:40:33.835 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-09-21T21:40:33.835 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-09-21T21:40:33.835 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_max_defer_interval": "3.000000", 2024-09-21T21:40:33.835 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_max_deferred_txc": "32", 2024-09-21T21:40:33.836 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_min_alloc_size": "0", 2024-09-21T21:40:33.836 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-09-21T21:40:33.836 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-09-21T21:40:33.836 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_nid_prealloc": "1024", 2024-09-21T21:40:33.836 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_prefer_deferred_size": "0", 2024-09-21T21:40:33.836 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-09-21T21:40:33.836 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-09-21T21:40:33.836 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_qfsck_on_mount": "true", 2024-09-21T21:40:33.836 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_retry_disk_reads": "3", 2024-09-21T21:40:33.836 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_rocksdb_cf": "true", 2024-09-21T21:40:33.836 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-09-21T21:40:33.837 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-09-21T21:40:33.837 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_rocksdb_options_annex": "", 2024-09-21T21:40:33.837 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_slow_ops_warn_lifetime": "86400", 2024-09-21T21:40:33.837 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_slow_ops_warn_threshold": "1", 2024-09-21T21:40:33.837 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_spdk_coremask": "0x1", 2024-09-21T21:40:33.837 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_spdk_io_sleep": "5", 2024-09-21T21:40:33.837 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_spdk_max_io_completion": "0", 2024-09-21T21:40:33.837 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_spdk_mem": "512", 2024-09-21T21:40:33.837 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_sync_submit_transaction": "false", 2024-09-21T21:40:33.837 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_throttle_bytes": "67108864", 2024-09-21T21:40:33.837 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_throttle_cost_per_io": "0", 2024-09-21T21:40:33.838 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-09-21T21:40:33.838 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-09-21T21:40:33.838 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-09-21T21:40:33.838 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-09-21T21:40:33.838 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_tracing": "false", 2024-09-21T21:40:33.838 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-09-21T21:40:33.838 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-09-21T21:40:33.838 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_volume_selection_reserved": "0", 2024-09-21T21:40:33.838 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-09-21T21:40:33.838 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-09-21T21:40:33.838 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-09-21T21:40:33.838 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-09-21T21:40:33.838 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-09-21T21:40:33.839 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-09-21T21:40:33.839 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_zero_block_detection": "false", 2024-09-21T21:40:33.839 INFO:teuthology.orchestra.run.smithi081.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-09-21T21:40:33.839 INFO:teuthology.orchestra.run.smithi081.stdout: "cephfs_mirror_action_update_interval": "2", 2024-09-21T21:40:33.839 INFO:teuthology.orchestra.run.smithi081.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-09-21T21:40:33.839 INFO:teuthology.orchestra.run.smithi081.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-09-21T21:40:33.839 INFO:teuthology.orchestra.run.smithi081.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-09-21T21:40:33.839 INFO:teuthology.orchestra.run.smithi081.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-09-21T21:40:33.839 INFO:teuthology.orchestra.run.smithi081.stdout: "cephfs_mirror_mount_timeout": "10", 2024-09-21T21:40:33.839 INFO:teuthology.orchestra.run.smithi081.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-09-21T21:40:33.839 INFO:teuthology.orchestra.run.smithi081.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-09-21T21:40:33.839 INFO:teuthology.orchestra.run.smithi081.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-09-21T21:40:33.840 INFO:teuthology.orchestra.run.smithi081.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-09-21T21:40:33.840 INFO:teuthology.orchestra.run.smithi081.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-09-21T21:40:33.840 INFO:teuthology.orchestra.run.smithi081.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-09-21T21:40:33.840 INFO:teuthology.orchestra.run.smithi081.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-09-21T21:40:33.840 INFO:teuthology.orchestra.run.smithi081.stdout: "cephx_cluster_require_signatures": "false", 2024-09-21T21:40:33.840 INFO:teuthology.orchestra.run.smithi081.stdout: "cephx_cluster_require_version": "2", 2024-09-21T21:40:33.840 INFO:teuthology.orchestra.run.smithi081.stdout: "cephx_require_signatures": "false", 2024-09-21T21:40:33.840 INFO:teuthology.orchestra.run.smithi081.stdout: "cephx_require_version": "2", 2024-09-21T21:40:33.840 INFO:teuthology.orchestra.run.smithi081.stdout: "cephx_service_require_signatures": "false", 2024-09-21T21:40:33.840 INFO:teuthology.orchestra.run.smithi081.stdout: "cephx_service_require_version": "2", 2024-09-21T21:40:33.840 INFO:teuthology.orchestra.run.smithi081.stdout: "cephx_sign_messages": "true", 2024-09-21T21:40:33.840 INFO:teuthology.orchestra.run.smithi081.stdout: "chdir": "", 2024-09-21T21:40:33.840 INFO:teuthology.orchestra.run.smithi081.stdout: "client_acl_type": "", 2024-09-21T21:40:33.840 INFO:teuthology.orchestra.run.smithi081.stdout: "client_asio_thread_count": "2", 2024-09-21T21:40:33.841 INFO:teuthology.orchestra.run.smithi081.stdout: "client_cache_mid": "0.750000", 2024-09-21T21:40:33.841 INFO:teuthology.orchestra.run.smithi081.stdout: "client_cache_size": "16384", 2024-09-21T21:40:33.841 INFO:teuthology.orchestra.run.smithi081.stdout: "client_caps_release_delay": "5", 2024-09-21T21:40:33.841 INFO:teuthology.orchestra.run.smithi081.stdout: "client_check_pool_perm": "true", 2024-09-21T21:40:33.841 INFO:teuthology.orchestra.run.smithi081.stdout: "client_collect_and_send_global_metrics": "false", 2024-09-21T21:40:33.841 INFO:teuthology.orchestra.run.smithi081.stdout: "client_debug_force_sync_read": "false", 2024-09-21T21:40:33.841 INFO:teuthology.orchestra.run.smithi081.stdout: "client_debug_getattr_caps": "false", 2024-09-21T21:40:33.841 INFO:teuthology.orchestra.run.smithi081.stdout: "client_debug_inject_features": "", 2024-09-21T21:40:33.841 INFO:teuthology.orchestra.run.smithi081.stdout: "client_debug_inject_tick_delay": "0", 2024-09-21T21:40:33.841 INFO:teuthology.orchestra.run.smithi081.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-09-21T21:40:33.841 INFO:teuthology.orchestra.run.smithi081.stdout: "client_die_on_failed_remount": "false", 2024-09-21T21:40:33.841 INFO:teuthology.orchestra.run.smithi081.stdout: "client_dirsize_rbytes": "true", 2024-09-21T21:40:33.841 INFO:teuthology.orchestra.run.smithi081.stdout: "client_force_lazyio": "false", 2024-09-21T21:40:33.842 INFO:teuthology.orchestra.run.smithi081.stdout: "client_fs": "", 2024-09-21T21:40:33.842 INFO:teuthology.orchestra.run.smithi081.stdout: "client_inject_fixed_oldest_tid": "false", 2024-09-21T21:40:33.842 INFO:teuthology.orchestra.run.smithi081.stdout: "client_inject_release_failure": "false", 2024-09-21T21:40:33.842 INFO:teuthology.orchestra.run.smithi081.stdout: "client_max_inline_size": "4096", 2024-09-21T21:40:33.842 INFO:teuthology.orchestra.run.smithi081.stdout: "client_max_retries_on_remount_failure": "5", 2024-09-21T21:40:33.842 INFO:teuthology.orchestra.run.smithi081.stdout: "client_mds_namespace": "", 2024-09-21T21:40:33.842 INFO:teuthology.orchestra.run.smithi081.stdout: "client_metadata": "", 2024-09-21T21:40:33.842 INFO:teuthology.orchestra.run.smithi081.stdout: "client_mount_gid": "-1", 2024-09-21T21:40:33.842 INFO:teuthology.orchestra.run.smithi081.stdout: "client_mount_timeout": "300", 2024-09-21T21:40:33.842 INFO:teuthology.orchestra.run.smithi081.stdout: "client_mount_uid": "-1", 2024-09-21T21:40:33.842 INFO:teuthology.orchestra.run.smithi081.stdout: "client_mountpoint": "/", 2024-09-21T21:40:33.842 INFO:teuthology.orchestra.run.smithi081.stdout: "client_notify_timeout": "10", 2024-09-21T21:40:33.842 INFO:teuthology.orchestra.run.smithi081.stdout: "client_oc": "true", 2024-09-21T21:40:33.842 INFO:teuthology.orchestra.run.smithi081.stdout: "client_oc_max_dirty": "104857600", 2024-09-21T21:40:33.842 INFO:teuthology.orchestra.run.smithi081.stdout: "client_oc_max_dirty_age": "5.000000", 2024-09-21T21:40:33.843 INFO:teuthology.orchestra.run.smithi081.stdout: "client_oc_max_objects": "1000", 2024-09-21T21:40:33.843 INFO:teuthology.orchestra.run.smithi081.stdout: "client_oc_size": "209715200", 2024-09-21T21:40:33.843 INFO:teuthology.orchestra.run.smithi081.stdout: "client_oc_target_dirty": "8388608", 2024-09-21T21:40:33.843 INFO:teuthology.orchestra.run.smithi081.stdout: "client_permissions": "true", 2024-09-21T21:40:33.843 INFO:teuthology.orchestra.run.smithi081.stdout: "client_quota": "true", 2024-09-21T21:40:33.843 INFO:teuthology.orchestra.run.smithi081.stdout: "client_quota_df": "true", 2024-09-21T21:40:33.843 INFO:teuthology.orchestra.run.smithi081.stdout: "client_readahead_max_bytes": "0", 2024-09-21T21:40:33.843 INFO:teuthology.orchestra.run.smithi081.stdout: "client_readahead_max_periods": "4", 2024-09-21T21:40:33.843 INFO:teuthology.orchestra.run.smithi081.stdout: "client_readahead_min": "131072", 2024-09-21T21:40:33.843 INFO:teuthology.orchestra.run.smithi081.stdout: "client_reconnect_stale": "false", 2024-09-21T21:40:33.843 INFO:teuthology.orchestra.run.smithi081.stdout: "client_shutdown_timeout": "30", 2024-09-21T21:40:33.843 INFO:teuthology.orchestra.run.smithi081.stdout: "client_snapdir": ".snap", 2024-09-21T21:40:33.843 INFO:teuthology.orchestra.run.smithi081.stdout: "client_tick_interval": "1", 2024-09-21T21:40:33.843 INFO:teuthology.orchestra.run.smithi081.stdout: "client_trace": "", 2024-09-21T21:40:33.843 INFO:teuthology.orchestra.run.smithi081.stdout: "client_try_dentry_invalidate": "false", 2024-09-21T21:40:33.844 INFO:teuthology.orchestra.run.smithi081.stdout: "client_use_faked_inos": "false", 2024-09-21T21:40:33.844 INFO:teuthology.orchestra.run.smithi081.stdout: "client_use_random_mds": "false", 2024-09-21T21:40:33.844 INFO:teuthology.orchestra.run.smithi081.stdout: "clog_to_graylog": "false", 2024-09-21T21:40:33.844 INFO:teuthology.orchestra.run.smithi081.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-09-21T21:40:33.844 INFO:teuthology.orchestra.run.smithi081.stdout: "clog_to_graylog_port": "12201", 2024-09-21T21:40:33.844 INFO:teuthology.orchestra.run.smithi081.stdout: "clog_to_monitors": "default=true", 2024-09-21T21:40:33.844 INFO:teuthology.orchestra.run.smithi081.stdout: "clog_to_syslog": "false", 2024-09-21T21:40:33.844 INFO:teuthology.orchestra.run.smithi081.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-09-21T21:40:33.844 INFO:teuthology.orchestra.run.smithi081.stdout: "clog_to_syslog_level": "info", 2024-09-21T21:40:33.844 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster_addr": "-", 2024-09-21T21:40:33.844 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster_network": "", 2024-09-21T21:40:33.844 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster_network_interface": "", 2024-09-21T21:40:33.844 INFO:teuthology.orchestra.run.smithi081.stdout: "compressor_zlib_isal": "false", 2024-09-21T21:40:33.844 INFO:teuthology.orchestra.run.smithi081.stdout: "compressor_zlib_level": "5", 2024-09-21T21:40:33.844 INFO:teuthology.orchestra.run.smithi081.stdout: "compressor_zlib_winsize": "-15", 2024-09-21T21:40:33.844 INFO:teuthology.orchestra.run.smithi081.stdout: "compressor_zstd_level": "1", 2024-09-21T21:40:33.844 INFO:teuthology.orchestra.run.smithi081.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-09-21T21:40:33.845 INFO:teuthology.orchestra.run.smithi081.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-09-21T21:40:33.845 INFO:teuthology.orchestra.run.smithi081.stdout: "crimson_alien_op_num_threads": "6", 2024-09-21T21:40:33.845 INFO:teuthology.orchestra.run.smithi081.stdout: "crimson_alien_thread_cpu_cores": "", 2024-09-21T21:40:33.845 INFO:teuthology.orchestra.run.smithi081.stdout: "crimson_osd_obc_lru_size": "512", 2024-09-21T21:40:33.845 INFO:teuthology.orchestra.run.smithi081.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-09-21T21:40:33.845 INFO:teuthology.orchestra.run.smithi081.stdout: "crimson_osd_stat_interval": "0", 2024-09-21T21:40:33.845 INFO:teuthology.orchestra.run.smithi081.stdout: "crimson_seastar_cpu_cores": "", 2024-09-21T21:40:33.845 INFO:teuthology.orchestra.run.smithi081.stdout: "crimson_seastar_num_threads": "0", 2024-09-21T21:40:33.845 INFO:teuthology.orchestra.run.smithi081.stdout: "crush_location": "", 2024-09-21T21:40:33.845 INFO:teuthology.orchestra.run.smithi081.stdout: "crush_location_hook": "", 2024-09-21T21:40:33.845 INFO:teuthology.orchestra.run.smithi081.stdout: "crush_location_hook_timeout": "10", 2024-09-21T21:40:33.845 INFO:teuthology.orchestra.run.smithi081.stdout: "daemonize": "false", 2024-09-21T21:40:33.845 INFO:teuthology.orchestra.run.smithi081.stdout: "daos_pool": "tank", 2024-09-21T21:40:33.846 INFO:teuthology.orchestra.run.smithi081.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-09-21T21:40:33.846 INFO:teuthology.orchestra.run.smithi081.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-09-21T21:40:33.846 INFO:teuthology.orchestra.run.smithi081.stdout: "dbstore_db_name_prefix": "dbstore", 2024-09-21T21:40:33.846 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_alienstore": "0/5", 2024-09-21T21:40:33.846 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_allow_any_pool_priority": "false", 2024-09-21T21:40:33.846 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_asok": "1/5", 2024-09-21T21:40:33.846 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_asok_assert_abort": "false", 2024-09-21T21:40:33.846 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_asserts_on_shutdown": "true", 2024-09-21T21:40:33.846 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_auth": "20/20", 2024-09-21T21:40:33.846 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_bdev": "1/3", 2024-09-21T21:40:33.846 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_bluefs": "1/5", 2024-09-21T21:40:33.846 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_bluestore": "1/5", 2024-09-21T21:40:33.846 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_buffer": "0/1", 2024-09-21T21:40:33.846 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_ceph_exporter": "1/5", 2024-09-21T21:40:33.847 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_cephfs_mirror": "0/5", 2024-09-21T21:40:33.847 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_cephsqlite": "0/5", 2024-09-21T21:40:33.847 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_client": "0/5", 2024-09-21T21:40:33.847 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_compressor": "1/5", 2024-09-21T21:40:33.847 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_context": "0/1", 2024-09-21T21:40:33.847 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_crimson_interrupt": "0/5", 2024-09-21T21:40:33.847 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_crush": "1/1", 2024-09-21T21:40:33.847 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_crypto": "1/5", 2024-09-21T21:40:33.847 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_cyanstore": "0/5", 2024-09-21T21:40:33.847 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_deliberately_leak_memory": "false", 2024-09-21T21:40:33.847 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_disable_randomized_ping": "false", 2024-09-21T21:40:33.847 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_dpdk": "1/5", 2024-09-21T21:40:33.847 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_eventtrace": "1/5", 2024-09-21T21:40:33.847 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_filer": "0/1", 2024-09-21T21:40:33.847 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_filestore": "1/3", 2024-09-21T21:40:33.848 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_finisher": "1/1", 2024-09-21T21:40:33.848 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_fuse": "1/5", 2024-09-21T21:40:33.848 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_heartbeat_testing_span": "0", 2024-09-21T21:40:33.848 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_heartbeatmap": "1/5", 2024-09-21T21:40:33.848 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_immutable_obj_cache": "0/5", 2024-09-21T21:40:33.848 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_javaclient": "1/5", 2024-09-21T21:40:33.848 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_journal": "1/3", 2024-09-21T21:40:33.848 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_journaler": "0/5", 2024-09-21T21:40:33.848 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_kstore": "1/5", 2024-09-21T21:40:33.848 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_lockdep": "0/1", 2024-09-21T21:40:33.848 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_mclock": "1/5", 2024-09-21T21:40:33.848 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_mds": "1/5", 2024-09-21T21:40:33.848 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_mds_balancer": "1/5", 2024-09-21T21:40:33.849 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_mds_locker": "1/5", 2024-09-21T21:40:33.849 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_mds_log": "1/5", 2024-09-21T21:40:33.849 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_mds_log_expire": "1/5", 2024-09-21T21:40:33.849 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_mds_migrator": "1/5", 2024-09-21T21:40:33.849 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_mds_quiesce": "3/5", 2024-09-21T21:40:33.850 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_memstore": "1/5", 2024-09-21T21:40:33.850 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_mgr": "2/5", 2024-09-21T21:40:33.850 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_mgrc": "1/5", 2024-09-21T21:40:33.850 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_mon": "20/20", 2024-09-21T21:40:33.850 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_monc": "0/10", 2024-09-21T21:40:33.850 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_ms": "1/1", 2024-09-21T21:40:33.850 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_none": "0/5", 2024-09-21T21:40:33.850 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_objclass": "0/5", 2024-09-21T21:40:33.850 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_objectcacher": "0/5", 2024-09-21T21:40:33.850 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_objecter": "0/1", 2024-09-21T21:40:33.850 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_optracker": "0/5", 2024-09-21T21:40:33.850 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_osd": "1/5", 2024-09-21T21:40:33.850 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_paxos": "20/20", 2024-09-21T21:40:33.850 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_perfcounter": "1/5", 2024-09-21T21:40:33.850 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_prioritycache": "1/5", 2024-09-21T21:40:33.850 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_rados": "0/5", 2024-09-21T21:40:33.851 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_rbd": "0/5", 2024-09-21T21:40:33.851 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_rbd_mirror": "0/5", 2024-09-21T21:40:33.851 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_rbd_pwl": "0/5", 2024-09-21T21:40:33.851 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_rbd_replay": "0/5", 2024-09-21T21:40:33.851 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_refs": "0/0", 2024-09-21T21:40:33.851 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_reserver": "1/1", 2024-09-21T21:40:33.851 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_rgw": "1/5", 2024-09-21T21:40:33.851 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_rgw_access": "1/5", 2024-09-21T21:40:33.851 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_rgw_datacache": "1/5", 2024-09-21T21:40:33.851 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_rgw_dbstore": "1/5", 2024-09-21T21:40:33.851 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_rgw_flight": "1/5", 2024-09-21T21:40:33.851 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_rgw_lifecycle": "1/5", 2024-09-21T21:40:33.851 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_rgw_sync": "1/5", 2024-09-21T21:40:33.851 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_rocksdb": "4/5", 2024-09-21T21:40:33.851 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_seastore": "0/5", 2024-09-21T21:40:33.851 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_seastore_backref": "0/5", 2024-09-21T21:40:33.852 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_seastore_cache": "0/5", 2024-09-21T21:40:33.852 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_seastore_cleaner": "0/5", 2024-09-21T21:40:33.852 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_seastore_device": "0/5", 2024-09-21T21:40:33.852 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_seastore_epm": "0/5", 2024-09-21T21:40:33.852 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-09-21T21:40:33.852 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_seastore_journal": "0/5", 2024-09-21T21:40:33.852 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_seastore_lba": "0/5", 2024-09-21T21:40:33.852 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_seastore_odata": "0/5", 2024-09-21T21:40:33.852 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_seastore_omap": "0/5", 2024-09-21T21:40:33.852 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_seastore_onode": "0/5", 2024-09-21T21:40:33.852 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_seastore_t": "0/5", 2024-09-21T21:40:33.852 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_seastore_tm": "0/5", 2024-09-21T21:40:33.852 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_striper": "0/1", 2024-09-21T21:40:33.852 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_test": "0/5", 2024-09-21T21:40:33.852 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_throttle": "1/1", 2024-09-21T21:40:33.853 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_timer": "0/1", 2024-09-21T21:40:33.853 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_tp": "0/5", 2024-09-21T21:40:33.853 INFO:teuthology.orchestra.run.smithi081.stdout: "debug_trace": "1/5", 2024-09-21T21:40:33.853 INFO:teuthology.orchestra.run.smithi081.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-09-21T21:40:33.853 INFO:teuthology.orchestra.run.smithi081.stdout: "device_failure_prediction_mode": "none", 2024-09-21T21:40:33.853 INFO:teuthology.orchestra.run.smithi081.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-09-21T21:40:33.853 INFO:teuthology.orchestra.run.smithi081.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2024-09-21T21:40:33.853 INFO:teuthology.orchestra.run.smithi081.stdout: "err_to_graylog": "false", 2024-09-21T21:40:33.853 INFO:teuthology.orchestra.run.smithi081.stdout: "err_to_journald": "false", 2024-09-21T21:40:33.853 INFO:teuthology.orchestra.run.smithi081.stdout: "err_to_stderr": "true", 2024-09-21T21:40:33.853 INFO:teuthology.orchestra.run.smithi081.stdout: "err_to_syslog": "false", 2024-09-21T21:40:33.853 INFO:teuthology.orchestra.run.smithi081.stdout: "event_tracing": "false", 2024-09-21T21:40:33.853 INFO:teuthology.orchestra.run.smithi081.stdout: "exporter_addr": "0.0.0.0", 2024-09-21T21:40:33.854 INFO:teuthology.orchestra.run.smithi081.stdout: "exporter_http_port": "9926", 2024-09-21T21:40:33.854 INFO:teuthology.orchestra.run.smithi081.stdout: "exporter_prio_limit": "5", 2024-09-21T21:40:33.854 INFO:teuthology.orchestra.run.smithi081.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-09-21T21:40:33.854 INFO:teuthology.orchestra.run.smithi081.stdout: "exporter_sort_metrics": "true", 2024-09-21T21:40:33.854 INFO:teuthology.orchestra.run.smithi081.stdout: "exporter_stats_period": "5", 2024-09-21T21:40:33.854 INFO:teuthology.orchestra.run.smithi081.stdout: "fake_statfs_for_testing": "0", 2024-09-21T21:40:33.854 INFO:teuthology.orchestra.run.smithi081.stdout: "fatal_signal_handlers": "true", 2024-09-21T21:40:33.854 INFO:teuthology.orchestra.run.smithi081.stdout: "filer_max_purge_ops": "10", 2024-09-21T21:40:33.854 INFO:teuthology.orchestra.run.smithi081.stdout: "filer_max_truncate_ops": "128", 2024-09-21T21:40:33.854 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_apply_finisher_threads": "1", 2024-09-21T21:40:33.854 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_blackhole": "false", 2024-09-21T21:40:33.854 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_btrfs_clone_range": "true", 2024-09-21T21:40:33.854 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_btrfs_snap": "true", 2024-09-21T21:40:33.855 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_caller_concurrency": "10", 2024-09-21T21:40:33.855 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_collect_device_partition_information": "true", 2024-09-21T21:40:33.855 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_commit_timeout": "600.000000", 2024-09-21T21:40:33.855 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_debug_inject_read_err": "false", 2024-09-21T21:40:33.855 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_debug_omap_check": "false", 2024-09-21T21:40:33.855 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_debug_random_read_err": "0.000000", 2024-09-21T21:40:33.855 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_debug_verify_split": "false", 2024-09-21T21:40:33.855 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_dump_file": "", 2024-09-21T21:40:33.855 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-09-21T21:40:33.855 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-09-21T21:40:33.855 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_fadvise": "true", 2024-09-21T21:40:33.855 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_fail_eio": "true", 2024-09-21T21:40:33.855 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_fd_cache_shards": "16", 2024-09-21T21:40:33.856 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_fd_cache_size": "128", 2024-09-21T21:40:33.856 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_fiemap": "false", 2024-09-21T21:40:33.856 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_fiemap_threshold": "4096", 2024-09-21T21:40:33.856 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-09-21T21:40:33.856 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_index_retry_probability": "0.000000", 2024-09-21T21:40:33.856 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_inject_stall": "0", 2024-09-21T21:40:33.856 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_journal_parallel": "false", 2024-09-21T21:40:33.856 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_journal_trailing": "false", 2024-09-21T21:40:33.856 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_journal_writeahead": "false", 2024-09-21T21:40:33.856 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_kill_at": "0", 2024-09-21T21:40:33.856 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-09-21T21:40:33.856 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_max_inline_xattr_size": "0", 2024-09-21T21:40:33.856 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-09-21T21:40:33.856 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-09-21T21:40:33.857 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-09-21T21:40:33.857 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_max_inline_xattrs": "0", 2024-09-21T21:40:33.857 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-09-21T21:40:33.857 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_max_inline_xattrs_other": "2", 2024-09-21T21:40:33.857 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-09-21T21:40:33.857 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_max_sync_interval": "5.000000", 2024-09-21T21:40:33.857 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_max_xattr_value_size": "0", 2024-09-21T21:40:33.857 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-09-21T21:40:33.857 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-09-21T21:40:33.857 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-09-21T21:40:33.857 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_merge_threshold": "-10", 2024-09-21T21:40:33.857 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_min_sync_interval": "0.010000", 2024-09-21T21:40:33.857 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_odsync_write": "false", 2024-09-21T21:40:33.858 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_omap_backend": "rocksdb", 2024-09-21T21:40:33.858 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_omap_backend_path": "", 2024-09-21T21:40:33.858 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_omap_header_cache_size": "1024", 2024-09-21T21:40:33.858 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_ondisk_finisher_threads": "1", 2024-09-21T21:40:33.858 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-09-21T21:40:33.858 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_op_thread_timeout": "60", 2024-09-21T21:40:33.858 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_op_threads": "2", 2024-09-21T21:40:33.858 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_punch_hole": "false", 2024-09-21T21:40:33.858 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-09-21T21:40:33.858 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-09-21T21:40:33.858 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-09-21T21:40:33.858 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-09-21T21:40:33.858 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-09-21T21:40:33.858 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_queue_max_bytes": "104857600", 2024-09-21T21:40:33.859 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-09-21T21:40:33.859 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-09-21T21:40:33.859 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-09-21T21:40:33.859 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_queue_max_ops": "50", 2024-09-21T21:40:33.859 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-09-21T21:40:33.859 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_seek_data_hole": "false", 2024-09-21T21:40:33.859 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_sloppy_crc": "false", 2024-09-21T21:40:33.859 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-09-21T21:40:33.859 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_splice": "false", 2024-09-21T21:40:33.859 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_split_multiple": "2", 2024-09-21T21:40:33.859 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_split_rand_factor": "20", 2024-09-21T21:40:33.859 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_update_to": "1000", 2024-09-21T21:40:33.859 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-09-21T21:40:33.859 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-09-21T21:40:33.859 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-09-21T21:40:33.860 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-09-21T21:40:33.860 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-09-21T21:40:33.860 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-09-21T21:40:33.860 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_wbthrottle_enable": "true", 2024-09-21T21:40:33.860 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-09-21T21:40:33.860 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-09-21T21:40:33.860 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-09-21T21:40:33.860 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-09-21T21:40:33.860 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-09-21T21:40:33.860 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-09-21T21:40:33.860 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_xfs_extsize": "false", 2024-09-21T21:40:33.860 INFO:teuthology.orchestra.run.smithi081.stdout: "filestore_zfs_snap": "false", 2024-09-21T21:40:33.860 INFO:teuthology.orchestra.run.smithi081.stdout: "fio_dir": "/tmp/fio", 2024-09-21T21:40:33.860 INFO:teuthology.orchestra.run.smithi081.stdout: "fsid": "a0fdcfc2-ddd2-4e50-81bb-876d940f3578", 2024-09-21T21:40:33.860 INFO:teuthology.orchestra.run.smithi081.stdout: "fuse_allow_other": "true", 2024-09-21T21:40:33.860 INFO:teuthology.orchestra.run.smithi081.stdout: "fuse_atomic_o_trunc": "true", 2024-09-21T21:40:33.861 INFO:teuthology.orchestra.run.smithi081.stdout: "fuse_big_writes": "true", 2024-09-21T21:40:33.861 INFO:teuthology.orchestra.run.smithi081.stdout: "fuse_debug": "false", 2024-09-21T21:40:33.861 INFO:teuthology.orchestra.run.smithi081.stdout: "fuse_default_permissions": "false", 2024-09-21T21:40:33.861 INFO:teuthology.orchestra.run.smithi081.stdout: "fuse_disable_pagecache": "false", 2024-09-21T21:40:33.861 INFO:teuthology.orchestra.run.smithi081.stdout: "fuse_max_write": "0", 2024-09-21T21:40:33.861 INFO:teuthology.orchestra.run.smithi081.stdout: "fuse_multithreaded": "true", 2024-09-21T21:40:33.861 INFO:teuthology.orchestra.run.smithi081.stdout: "fuse_require_active_mds": "true", 2024-09-21T21:40:33.861 INFO:teuthology.orchestra.run.smithi081.stdout: "fuse_set_user_groups": "true", 2024-09-21T21:40:33.861 INFO:teuthology.orchestra.run.smithi081.stdout: "fuse_splice_move": "true", 2024-09-21T21:40:33.861 INFO:teuthology.orchestra.run.smithi081.stdout: "fuse_splice_read": "true", 2024-09-21T21:40:33.861 INFO:teuthology.orchestra.run.smithi081.stdout: "fuse_splice_write": "true", 2024-09-21T21:40:33.861 INFO:teuthology.orchestra.run.smithi081.stdout: "fuse_syncfs_on_mksnap": "true", 2024-09-21T21:40:33.861 INFO:teuthology.orchestra.run.smithi081.stdout: "fuse_use_invalidate_cb": "true", 2024-09-21T21:40:33.861 INFO:teuthology.orchestra.run.smithi081.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-09-21T21:40:33.861 INFO:teuthology.orchestra.run.smithi081.stdout: "gss_target_name": "ceph", 2024-09-21T21:40:33.861 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_file": "", 2024-09-21T21:40:33.862 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_inject_failure": "0", 2024-09-21T21:40:33.862 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_interval": "5", 2024-09-21T21:40:33.862 INFO:teuthology.orchestra.run.smithi081.stdout: "host": "smithi081", 2024-09-21T21:40:33.862 INFO:teuthology.orchestra.run.smithi081.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-09-21T21:40:33.862 INFO:teuthology.orchestra.run.smithi081.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-09-21T21:40:33.862 INFO:teuthology.orchestra.run.smithi081.stdout: "immutable_object_cache_max_size": "1073741824", 2024-09-21T21:40:33.862 INFO:teuthology.orchestra.run.smithi081.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-09-21T21:40:33.862 INFO:teuthology.orchestra.run.smithi081.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-09-21T21:40:33.862 INFO:teuthology.orchestra.run.smithi081.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-09-21T21:40:33.862 INFO:teuthology.orchestra.run.smithi081.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-09-21T21:40:33.862 INFO:teuthology.orchestra.run.smithi081.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-09-21T21:40:33.862 INFO:teuthology.orchestra.run.smithi081.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-09-21T21:40:33.862 INFO:teuthology.orchestra.run.smithi081.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-09-21T21:40:33.862 INFO:teuthology.orchestra.run.smithi081.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-09-21T21:40:33.862 INFO:teuthology.orchestra.run.smithi081.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-09-21T21:40:33.862 INFO:teuthology.orchestra.run.smithi081.stdout: "immutable_object_cache_watermark": "0.900000", 2024-09-21T21:40:33.863 INFO:teuthology.orchestra.run.smithi081.stdout: "inject_early_sigterm": "false", 2024-09-21T21:40:33.863 INFO:teuthology.orchestra.run.smithi081.stdout: "jaeger_agent_port": "6799", 2024-09-21T21:40:33.863 INFO:teuthology.orchestra.run.smithi081.stdout: "jaeger_tracing_enable": "false", 2024-09-21T21:40:33.863 INFO:teuthology.orchestra.run.smithi081.stdout: "journal_aio": "true", 2024-09-21T21:40:33.863 INFO:teuthology.orchestra.run.smithi081.stdout: "journal_align_min_size": "65536", 2024-09-21T21:40:33.863 INFO:teuthology.orchestra.run.smithi081.stdout: "journal_block_align": "true", 2024-09-21T21:40:33.863 INFO:teuthology.orchestra.run.smithi081.stdout: "journal_block_size": "4096", 2024-09-21T21:40:33.863 INFO:teuthology.orchestra.run.smithi081.stdout: "journal_dio": "true", 2024-09-21T21:40:33.863 INFO:teuthology.orchestra.run.smithi081.stdout: "journal_discard": "false", 2024-09-21T21:40:33.863 INFO:teuthology.orchestra.run.smithi081.stdout: "journal_force_aio": "false", 2024-09-21T21:40:33.863 INFO:teuthology.orchestra.run.smithi081.stdout: "journal_ignore_corruption": "false", 2024-09-21T21:40:33.863 INFO:teuthology.orchestra.run.smithi081.stdout: "journal_max_write_bytes": "10485760", 2024-09-21T21:40:33.863 INFO:teuthology.orchestra.run.smithi081.stdout: "journal_max_write_entries": "100", 2024-09-21T21:40:33.863 INFO:teuthology.orchestra.run.smithi081.stdout: "journal_replay_from": "0", 2024-09-21T21:40:33.864 INFO:teuthology.orchestra.run.smithi081.stdout: "journal_throttle_high_multiple": "0.000000", 2024-09-21T21:40:33.864 INFO:teuthology.orchestra.run.smithi081.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-09-21T21:40:33.864 INFO:teuthology.orchestra.run.smithi081.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-09-21T21:40:33.864 INFO:teuthology.orchestra.run.smithi081.stdout: "journal_throttle_max_multiple": "0.000000", 2024-09-21T21:40:33.864 INFO:teuthology.orchestra.run.smithi081.stdout: "journal_write_header_frequency": "0", 2024-09-21T21:40:33.864 INFO:teuthology.orchestra.run.smithi081.stdout: "journal_zero_on_create": "false", 2024-09-21T21:40:33.864 INFO:teuthology.orchestra.run.smithi081.stdout: "journaler_prefetch_periods": "10", 2024-09-21T21:40:33.864 INFO:teuthology.orchestra.run.smithi081.stdout: "journaler_prezero_periods": "5", 2024-09-21T21:40:33.864 INFO:teuthology.orchestra.run.smithi081.stdout: "journaler_write_head_interval": "15", 2024-09-21T21:40:33.864 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "", 2024-09-21T21:40:33.864 INFO:teuthology.orchestra.run.smithi081.stdout: "keyfile": "", 2024-09-21T21:40:33.864 INFO:teuthology.orchestra.run.smithi081.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-09-21T21:40:33.864 INFO:teuthology.orchestra.run.smithi081.stdout: "kstore_backend": "rocksdb", 2024-09-21T21:40:33.864 INFO:teuthology.orchestra.run.smithi081.stdout: "kstore_default_stripe_size": "65536", 2024-09-21T21:40:33.864 INFO:teuthology.orchestra.run.smithi081.stdout: "kstore_fsck_on_mount": "false", 2024-09-21T21:40:33.864 INFO:teuthology.orchestra.run.smithi081.stdout: "kstore_fsck_on_mount_deep": "true", 2024-09-21T21:40:33.865 INFO:teuthology.orchestra.run.smithi081.stdout: "kstore_max_bytes": "67108864", 2024-09-21T21:40:33.865 INFO:teuthology.orchestra.run.smithi081.stdout: "kstore_max_ops": "512", 2024-09-21T21:40:33.865 INFO:teuthology.orchestra.run.smithi081.stdout: "kstore_nid_prealloc": "1024", 2024-09-21T21:40:33.865 INFO:teuthology.orchestra.run.smithi081.stdout: "kstore_onode_map_size": "1024", 2024-09-21T21:40:33.865 INFO:teuthology.orchestra.run.smithi081.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-09-21T21:40:33.865 INFO:teuthology.orchestra.run.smithi081.stdout: "kstore_sync_submit_transaction": "false", 2024-09-21T21:40:33.865 INFO:teuthology.orchestra.run.smithi081.stdout: "kstore_sync_transaction": "false", 2024-09-21T21:40:33.865 INFO:teuthology.orchestra.run.smithi081.stdout: "librados_thread_count": "2", 2024-09-21T21:40:33.865 INFO:teuthology.orchestra.run.smithi081.stdout: "lockdep": "false", 2024-09-21T21:40:33.865 INFO:teuthology.orchestra.run.smithi081.stdout: "lockdep_force_backtrace": "false", 2024-09-21T21:40:33.865 INFO:teuthology.orchestra.run.smithi081.stdout: "log_coarse_timestamps": "true", 2024-09-21T21:40:33.865 INFO:teuthology.orchestra.run.smithi081.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-09-21T21:40:33.865 INFO:teuthology.orchestra.run.smithi081.stdout: "log_flush_on_exit": "false", 2024-09-21T21:40:33.865 INFO:teuthology.orchestra.run.smithi081.stdout: "log_graylog_host": "127.0.0.1", 2024-09-21T21:40:33.866 INFO:teuthology.orchestra.run.smithi081.stdout: "log_graylog_port": "12201", 2024-09-21T21:40:33.866 INFO:teuthology.orchestra.run.smithi081.stdout: "log_max_new": "1000", 2024-09-21T21:40:33.866 INFO:teuthology.orchestra.run.smithi081.stdout: "log_max_recent": "10000", 2024-09-21T21:40:33.866 INFO:teuthology.orchestra.run.smithi081.stdout: "log_stderr_prefix": "", 2024-09-21T21:40:33.866 INFO:teuthology.orchestra.run.smithi081.stdout: "log_stop_at_utilization": "0.970000", 2024-09-21T21:40:33.866 INFO:teuthology.orchestra.run.smithi081.stdout: "log_to_file": "true", 2024-09-21T21:40:33.866 INFO:teuthology.orchestra.run.smithi081.stdout: "log_to_graylog": "false", 2024-09-21T21:40:33.866 INFO:teuthology.orchestra.run.smithi081.stdout: "log_to_journald": "false", 2024-09-21T21:40:33.866 INFO:teuthology.orchestra.run.smithi081.stdout: "log_to_stderr": "false", 2024-09-21T21:40:33.866 INFO:teuthology.orchestra.run.smithi081.stdout: "log_to_syslog": "false", 2024-09-21T21:40:33.866 INFO:teuthology.orchestra.run.smithi081.stdout: "max_rotating_auth_attempts": "10", 2024-09-21T21:40:33.866 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-09-21T21:40:33.866 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_action_on_write_error": "1", 2024-09-21T21:40:33.866 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_alternate_name_max": "8192", 2024-09-21T21:40:33.867 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_asio_thread_count": "2", 2024-09-21T21:40:33.867 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_bal_export_pin": "true", 2024-09-21T21:40:33.867 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_bal_fragment_dirs": "true", 2024-09-21T21:40:33.867 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-09-21T21:40:33.867 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_bal_fragment_interval": "5", 2024-09-21T21:40:33.867 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_bal_fragment_size_max": "100000", 2024-09-21T21:40:33.867 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_bal_idle_threshold": "0.000000", 2024-09-21T21:40:33.867 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_bal_interval": "10", 2024-09-21T21:40:33.867 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_bal_max": "-1", 2024-09-21T21:40:33.867 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_bal_max_until": "-1", 2024-09-21T21:40:33.867 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_bal_merge_size": "50", 2024-09-21T21:40:33.867 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_bal_midchunk": "0.300000", 2024-09-21T21:40:33.867 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_bal_min_rebalance": "0.100000", 2024-09-21T21:40:33.867 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_bal_min_start": "0.200000", 2024-09-21T21:40:33.868 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_bal_minchunk": "0.001000", 2024-09-21T21:40:33.868 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_bal_mode": "0", 2024-09-21T21:40:33.868 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_bal_need_max": "1.200000", 2024-09-21T21:40:33.868 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_bal_need_min": "0.800000", 2024-09-21T21:40:33.868 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_bal_overload_epochs": "2", 2024-09-21T21:40:33.868 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-09-21T21:40:33.868 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_bal_sample_interval": "3.000000", 2024-09-21T21:40:33.868 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_bal_split_bits": "3", 2024-09-21T21:40:33.868 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_bal_split_rd": "25000.000000", 2024-09-21T21:40:33.868 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_bal_split_size": "10000", 2024-09-21T21:40:33.868 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_bal_split_wr": "10000.000000", 2024-09-21T21:40:33.868 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_bal_target_decay": "10.000000", 2024-09-21T21:40:33.868 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-09-21T21:40:33.868 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_beacon_grace": "15.000000", 2024-09-21T21:40:33.869 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_beacon_interval": "4.000000", 2024-09-21T21:40:33.869 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_beacon_mon_down_grace": "60", 2024-09-21T21:40:33.869 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_cache_memory_limit": "4294967296", 2024-09-21T21:40:33.869 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_cache_mid": "0.700000", 2024-09-21T21:40:33.869 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_cache_quiesce_decay_rate": "1.000000", 2024-09-21T21:40:33.869 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_cache_quiesce_delay": "0", 2024-09-21T21:40:33.869 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_cache_quiesce_sleep": "200", 2024-09-21T21:40:33.869 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_cache_quiesce_splitauth": "true", 2024-09-21T21:40:33.869 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_cache_quiesce_threshold": "524288", 2024-09-21T21:40:33.869 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_cache_release_free_interval": "10", 2024-09-21T21:40:33.869 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_cache_reservation": "0.050000", 2024-09-21T21:40:33.869 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-09-21T21:40:33.869 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_cache_trim_interval": "1", 2024-09-21T21:40:33.869 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_cache_trim_threshold": "262144", 2024-09-21T21:40:33.869 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-09-21T21:40:33.869 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-09-21T21:40:33.870 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_client_delegate_inos_pct": "50", 2024-09-21T21:40:33.870 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_client_prealloc_inos": "1000", 2024-09-21T21:40:33.870 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-09-21T21:40:33.870 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_connect_bootstrapping": "false", 2024-09-21T21:40:33.870 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_damage_table_max_entries": "10000", 2024-09-21T21:40:33.870 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-09-21T21:40:33.870 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_debug_auth_pins": "false", 2024-09-21T21:40:33.870 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_debug_frag": "false", 2024-09-21T21:40:33.870 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_debug_scatterstat": "false", 2024-09-21T21:40:33.870 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_debug_subtrees": "false", 2024-09-21T21:40:33.870 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_decay_halflife": "5.000000", 2024-09-21T21:40:33.870 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_default_dir_hash": "2", 2024-09-21T21:40:33.870 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_defer_session_stale": "true", 2024-09-21T21:40:33.870 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_deny_all_reconnect": "false", 2024-09-21T21:40:33.871 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_dir_keys_per_op": "16384", 2024-09-21T21:40:33.871 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_dir_max_commit_size": "10", 2024-09-21T21:40:33.871 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_dir_max_entries": "0", 2024-09-21T21:40:33.871 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_dir_prefetch": "true", 2024-09-21T21:40:33.871 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_dirstat_min_interval": "1.000000", 2024-09-21T21:40:33.871 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_dump_cache_after_rejoin": "false", 2024-09-21T21:40:33.871 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_dump_cache_on_map": "false", 2024-09-21T21:40:33.871 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_dump_cache_threshold_file": "0", 2024-09-21T21:40:33.871 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-09-21T21:40:33.871 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_early_reply": "true", 2024-09-21T21:40:33.871 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_enable_op_tracker": "true", 2024-09-21T21:40:33.871 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_enforce_unique_name": "true", 2024-09-21T21:40:33.872 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_export_ephemeral_distributed": "true", 2024-09-21T21:40:33.872 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-09-21T21:40:33.872 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_export_ephemeral_random": "true", 2024-09-21T21:40:33.872 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-09-21T21:40:33.872 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-09-21T21:40:33.872 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_forward_all_requests_to_auth": "false", 2024-09-21T21:40:33.872 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-09-21T21:40:33.872 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-09-21T21:40:33.872 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-09-21T21:40:33.872 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-09-21T21:40:33.872 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_health_cache_threshold": "1.500000", 2024-09-21T21:40:33.872 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_health_summarize_threshold": "10", 2024-09-21T21:40:33.872 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_heartbeat_grace": "15.000000", 2024-09-21T21:40:33.872 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_heartbeat_reset_grace": "1000", 2024-09-21T21:40:33.872 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_inject_health_dummy": "false", 2024-09-21T21:40:33.873 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-09-21T21:40:33.873 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_inject_migrator_session_race": "false", 2024-09-21T21:40:33.873 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-09-21T21:40:33.873 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-09-21T21:40:33.873 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-09-21T21:40:33.873 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_join_fs": "", 2024-09-21T21:40:33.873 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_journal_format": "1", 2024-09-21T21:40:33.873 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_kill_after_journal_logs_flushed": "false", 2024-09-21T21:40:33.873 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_kill_create_at": "0", 2024-09-21T21:40:33.873 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_kill_export_at": "0", 2024-09-21T21:40:33.873 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_kill_import_at": "0", 2024-09-21T21:40:33.873 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_kill_journal_at": "0", 2024-09-21T21:40:33.873 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_kill_journal_expire_at": "0", 2024-09-21T21:40:33.873 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_kill_journal_replay_at": "0", 2024-09-21T21:40:33.873 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_kill_link_at": "0", 2024-09-21T21:40:33.874 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_kill_mdstable_at": "0", 2024-09-21T21:40:33.874 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_kill_openc_at": "0", 2024-09-21T21:40:33.874 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_kill_rename_at": "0", 2024-09-21T21:40:33.874 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_kill_shutdown_at": "0", 2024-09-21T21:40:33.874 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_log_event_large_threshold": "524288", 2024-09-21T21:40:33.874 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_log_events_per_segment": "1024", 2024-09-21T21:40:33.874 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_log_major_segment_event_ratio": "12", 2024-09-21T21:40:33.874 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_log_max_events": "-1", 2024-09-21T21:40:33.874 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_log_max_segments": "128", 2024-09-21T21:40:33.874 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_log_pause": "false", 2024-09-21T21:40:33.874 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_log_segment_size": "0", 2024-09-21T21:40:33.874 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_log_skip_corrupt_events": "false", 2024-09-21T21:40:33.874 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_log_skip_unbounded_events": "false", 2024-09-21T21:40:33.874 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_log_trim_decay_rate": "1.000000", 2024-09-21T21:40:33.875 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_log_trim_threshold": "128", 2024-09-21T21:40:33.875 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_log_trim_upkeep_interval": "1000", 2024-09-21T21:40:33.875 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_log_warn_factor": "2.000000", 2024-09-21T21:40:33.875 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_max_caps_per_client": "1048576", 2024-09-21T21:40:33.875 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_max_completed_flushes": "100000", 2024-09-21T21:40:33.875 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_max_completed_requests": "100000", 2024-09-21T21:40:33.875 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_max_export_size": "20971520", 2024-09-21T21:40:33.875 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_max_file_recover": "32", 2024-09-21T21:40:33.875 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_max_purge_files": "64", 2024-09-21T21:40:33.875 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_max_purge_ops": "8192", 2024-09-21T21:40:33.875 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-09-21T21:40:33.875 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-09-21T21:40:33.875 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_max_snaps_per_dir": "100", 2024-09-21T21:40:33.875 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_metrics_update_interval": "2", 2024-09-21T21:40:33.875 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_min_caps_per_client": "100", 2024-09-21T21:40:33.876 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_min_caps_working_set": "10000", 2024-09-21T21:40:33.876 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-09-21T21:40:33.876 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_numa_node": "-1", 2024-09-21T21:40:33.876 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-09-21T21:40:33.876 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_op_complaint_time": "30.000000", 2024-09-21T21:40:33.876 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_op_history_duration": "600", 2024-09-21T21:40:33.876 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_op_history_size": "20", 2024-09-21T21:40:33.876 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_op_history_slow_op_size": "20", 2024-09-21T21:40:33.876 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_op_history_slow_op_threshold": "10", 2024-09-21T21:40:33.876 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_op_log_threshold": "5", 2024-09-21T21:40:33.876 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_ping_grace": "15", 2024-09-21T21:40:33.876 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_ping_interval": "5", 2024-09-21T21:40:33.876 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-09-21T21:40:33.876 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-09-21T21:40:33.876 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_recall_max_caps": "30000", 2024-09-21T21:40:33.877 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-09-21T21:40:33.877 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_recall_max_decay_threshold": "131072", 2024-09-21T21:40:33.877 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-09-21T21:40:33.877 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_recall_warning_threshold": "262144", 2024-09-21T21:40:33.877 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_reconnect_timeout": "45.000000", 2024-09-21T21:40:33.877 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_replay_interval": "1.000000", 2024-09-21T21:40:33.877 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-09-21T21:40:33.877 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-09-21T21:40:33.877 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_root_ino_gid": "0", 2024-09-21T21:40:33.877 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_root_ino_uid": "0", 2024-09-21T21:40:33.877 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-09-21T21:40:33.877 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_session_blocklist_on_evict": "true", 2024-09-21T21:40:33.877 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_session_blocklist_on_timeout": "true", 2024-09-21T21:40:33.877 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-09-21T21:40:33.878 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-09-21T21:40:33.878 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-09-21T21:40:33.878 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-09-21T21:40:33.878 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-09-21T21:40:33.878 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_session_metadata_threshold": "16777216", 2024-09-21T21:40:33.878 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-09-21T21:40:33.878 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_shutdown_check": "0", 2024-09-21T21:40:33.878 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_skip_ino": "0", 2024-09-21T21:40:33.878 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_sleep_rank_change": "0.000000", 2024-09-21T21:40:33.878 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_snap_max_uid": "4294967294", 2024-09-21T21:40:33.878 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_snap_min_uid": "0", 2024-09-21T21:40:33.878 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_snap_rstat": "false", 2024-09-21T21:40:33.878 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_standby_replay_damaged": "false", 2024-09-21T21:40:33.879 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_symlink_recovery": "true", 2024-09-21T21:40:33.879 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_task_status_update_interval": "2.000000", 2024-09-21T21:40:33.879 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_thrash_exports": "0", 2024-09-21T21:40:33.879 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_thrash_fragments": "0", 2024-09-21T21:40:33.879 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_tick_interval": "5.000000", 2024-09-21T21:40:33.879 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_valgrind_exit": "false", 2024-09-21T21:40:33.879 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_verify_backtrace": "1", 2024-09-21T21:40:33.879 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_verify_scatter": "false", 2024-09-21T21:40:33.879 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_wipe_ino_prealloc": "false", 2024-09-21T21:40:33.879 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_wipe_sessions": "false", 2024-09-21T21:40:33.879 INFO:teuthology.orchestra.run.smithi081.stdout: "mempool_debug": "false", 2024-09-21T21:40:33.879 INFO:teuthology.orchestra.run.smithi081.stdout: "memstore_debug_omit_block_device_write": "false", 2024-09-21T21:40:33.879 INFO:teuthology.orchestra.run.smithi081.stdout: "memstore_device_bytes": "1073741824", 2024-09-21T21:40:33.879 INFO:teuthology.orchestra.run.smithi081.stdout: "memstore_page_set": "false", 2024-09-21T21:40:33.880 INFO:teuthology.orchestra.run.smithi081.stdout: "memstore_page_size": "65536", 2024-09-21T21:40:33.880 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr_client_bytes": "134217728", 2024-09-21T21:40:33.880 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr_client_messages": "512", 2024-09-21T21:40:33.880 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-09-21T21:40:33.880 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr_connect_retry_interval": "1.000000", 2024-09-21T21:40:33.880 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-09-21T21:40:33.880 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-09-21T21:40:33.880 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr_disabled_modules": "", 2024-09-21T21:40:33.880 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr_enable_op_tracker": "true", 2024-09-21T21:40:33.880 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-09-21T21:40:33.880 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr_max_pg_num_change": "128", 2024-09-21T21:40:33.880 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr_mds_bytes": "134217728", 2024-09-21T21:40:33.880 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr_mds_messages": "128", 2024-09-21T21:40:33.881 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-09-21T21:40:33.881 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr_mon_bytes": "134217728", 2024-09-21T21:40:33.881 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr_mon_messages": "128", 2024-09-21T21:40:33.881 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr_num_op_tracker_shard": "32", 2024-09-21T21:40:33.881 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr_op_complaint_time": "30.000000", 2024-09-21T21:40:33.881 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr_op_history_duration": "600", 2024-09-21T21:40:33.881 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr_op_history_size": "20", 2024-09-21T21:40:33.881 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr_op_history_slow_op_size": "20", 2024-09-21T21:40:33.881 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr_op_history_slow_op_threshold": "10.000000", 2024-09-21T21:40:33.881 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr_op_log_threshold": "5", 2024-09-21T21:40:33.881 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr_osd_bytes": "536870912", 2024-09-21T21:40:33.881 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr_osd_messages": "8192", 2024-09-21T21:40:33.881 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr_pool": "true", 2024-09-21T21:40:33.881 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr_service_beacon_grace": "60.000000", 2024-09-21T21:40:33.881 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr_standby_modules": "true", 2024-09-21T21:40:33.882 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr_stats_period": "5", 2024-09-21T21:40:33.882 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr_stats_threshold": "5", 2024-09-21T21:40:33.882 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr_tick_period": "2", 2024-09-21T21:40:33.882 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-09-21T21:40:33.882 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_accept_timeout_factor": "2.000000", 2024-09-21T21:40:33.882 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_allow_pool_delete": "true", 2024-09-21T21:40:33.882 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_allow_pool_size_one": "true", 2024-09-21T21:40:33.882 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_auth_validate_all_caps": "true", 2024-09-21T21:40:33.882 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-09-21T21:40:33.882 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-09-21T21:40:33.882 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_client_bytes": "104857600", 2024-09-21T21:40:33.882 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_client_directed_command_retry": "5", 2024-09-21T21:40:33.882 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_client_hunt_interval": "3.000000", 2024-09-21T21:40:33.883 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-09-21T21:40:33.883 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-09-21T21:40:33.883 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-09-21T21:40:33.883 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_client_hunt_parallel": "3", 2024-09-21T21:40:33.883 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_client_log_interval": "1.000000", 2024-09-21T21:40:33.883 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-09-21T21:40:33.883 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_client_ping_interval": "10.000000", 2024-09-21T21:40:33.883 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_client_ping_timeout": "30.000000", 2024-09-21T21:40:33.883 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_client_target_rank": "-1", 2024-09-21T21:40:33.883 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_clock_drift_allowed": "1.000000", 2024-09-21T21:40:33.883 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-09-21T21:40:33.883 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-09-21T21:40:33.883 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_cluster_log_file_level": "debug", 2024-09-21T21:40:33.884 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_cluster_log_to_file": "true", 2024-09-21T21:40:33.884 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_cluster_log_to_graylog": "false", 2024-09-21T21:40:33.884 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-09-21T21:40:33.884 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-09-21T21:40:33.884 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_cluster_log_to_journald": "false", 2024-09-21T21:40:33.884 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_cluster_log_to_stderr": "false", 2024-09-21T21:40:33.884 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-09-21T21:40:33.884 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-09-21T21:40:33.884 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_cluster_log_to_syslog_level": "info", 2024-09-21T21:40:33.884 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_compact_on_bootstrap": "false", 2024-09-21T21:40:33.884 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_compact_on_start": "false", 2024-09-21T21:40:33.884 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_compact_on_trim": "true", 2024-09-21T21:40:33.884 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_con_tracker_persist_interval": "10", 2024-09-21T21:40:33.885 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_con_tracker_score_halflife": "43200", 2024-09-21T21:40:33.885 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_config_key_max_entry_size": "65536", 2024-09-21T21:40:33.885 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_cpu_threads": "4", 2024-09-21T21:40:33.885 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_crush_min_required_version": "hammer", 2024-09-21T21:40:33.885 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_daemon_bytes": "419430400", 2024-09-21T21:40:33.885 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-09-21T21:40:33.885 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_data_avail_crit": "5", 2024-09-21T21:40:33.885 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_data_avail_warn": "5", 2024-09-21T21:40:33.885 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_data_size_warn": "16106127360", 2024-09-21T21:40:33.885 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_debug_block_osdmap_trim": "false", 2024-09-21T21:40:33.885 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-09-21T21:40:33.885 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_debug_dump_json": "false", 2024-09-21T21:40:33.885 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-09-21T21:40:33.885 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_debug_dump_transactions": "false", 2024-09-21T21:40:33.886 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_debug_extra_checks": "false", 2024-09-21T21:40:33.886 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-09-21T21:40:33.886 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-09-21T21:40:33.886 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_debug_no_require_reef": "false", 2024-09-21T21:40:33.886 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_debug_no_require_squid": "false", 2024-09-21T21:40:33.886 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-09-21T21:40:33.886 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_delta_reset_interval": "10.000000", 2024-09-21T21:40:33.886 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_dns_srv_name": "ceph-mon", 2024-09-21T21:40:33.886 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_down_mkfs_grace": "120", 2024-09-21T21:40:33.886 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_down_uptime_grace": "60", 2024-09-21T21:40:33.886 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_election_default_strategy": "1", 2024-09-21T21:40:33.886 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_election_timeout": "5.000000", 2024-09-21T21:40:33.886 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-09-21T21:40:33.887 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_elector_ping_divisor": "2", 2024-09-21T21:40:33.887 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_elector_ping_timeout": "2.000000", 2024-09-21T21:40:33.887 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_enable_op_tracker": "true", 2024-09-21T21:40:33.887 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_fake_pool_delete": "false", 2024-09-21T21:40:33.887 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_force_quorum_join": "false", 2024-09-21T21:40:33.887 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_fsmap_prune_threshold": "300", 2024-09-21T21:40:33.887 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_globalid_prealloc": "10000", 2024-09-21T21:40:33.887 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_health_detail_to_clog": "false", 2024-09-21T21:40:33.887 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_health_log_update_period": "5", 2024-09-21T21:40:33.887 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_health_max_detail": "50", 2024-09-21T21:40:33.887 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_health_to_clog": "true", 2024-09-21T21:40:33.887 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_health_to_clog_interval": "600", 2024-09-21T21:40:33.887 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-09-21T21:40:33.888 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_host": "172.21.15.81,[v2:172.21.15.81:3301,v1:172.21.15.81:6790],172.21.15.122", 2024-09-21T21:40:33.888 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_host_override": "", 2024-09-21T21:40:33.888 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_initial_members": "", 2024-09-21T21:40:33.888 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-09-21T21:40:33.888 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-09-21T21:40:33.888 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-09-21T21:40:33.888 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-09-21T21:40:33.888 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_keyvaluedb": "rocksdb", 2024-09-21T21:40:33.888 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_lease": "5.000000", 2024-09-21T21:40:33.888 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-09-21T21:40:33.888 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-09-21T21:40:33.888 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_log_full_interval": "50", 2024-09-21T21:40:33.889 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_log_max": "10000", 2024-09-21T21:40:33.889 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_log_max_summary": "50", 2024-09-21T21:40:33.889 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_max_log_entries_per_event": "4096", 2024-09-21T21:40:33.889 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_max_log_epochs": "500", 2024-09-21T21:40:33.889 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_max_mdsmap_epochs": "500", 2024-09-21T21:40:33.889 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_max_mgrmap_epochs": "500", 2024-09-21T21:40:33.889 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_max_osd": "10000", 2024-09-21T21:40:33.889 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_max_pg_per_osd": "10000", 2024-09-21T21:40:33.889 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_max_pool_pg_num": "65536", 2024-09-21T21:40:33.889 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-09-21T21:40:33.889 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-09-21T21:40:33.889 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_mds_force_trim_to": "0", 2024-09-21T21:40:33.889 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_mds_skip_sanity": "false", 2024-09-21T21:40:33.890 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_memory_autotune": "true", 2024-09-21T21:40:33.890 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_memory_target": "2147483648", 2024-09-21T21:40:33.890 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_mgr_beacon_grace": "30", 2024-09-21T21:40:33.890 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-09-21T21:40:33.890 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_mgr_digest_period": "5", 2024-09-21T21:40:33.890 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_mgr_inactive_grace": "60", 2024-09-21T21:40:33.890 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_mgr_mkfs_grace": "240", 2024-09-21T21:40:33.890 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-09-21T21:40:33.890 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_min_osdmap_epochs": "500", 2024-09-21T21:40:33.890 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_op_complaint_time": "30", 2024-09-21T21:40:33.890 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_op_history_duration": "600", 2024-09-21T21:40:33.890 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_op_history_size": "20", 2024-09-21T21:40:33.890 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_op_history_slow_op_size": "20", 2024-09-21T21:40:33.890 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_op_history_slow_op_threshold": "10", 2024-09-21T21:40:33.891 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_op_log_threshold": "5", 2024-09-21T21:40:33.891 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-09-21T21:40:33.891 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-09-21T21:40:33.891 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-09-21T21:40:33.891 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_osd_auto_mark_in": "false", 2024-09-21T21:40:33.891 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_osd_auto_mark_new_in": "true", 2024-09-21T21:40:33.891 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-09-21T21:40:33.891 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-09-21T21:40:33.891 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_osd_cache_size": "500", 2024-09-21T21:40:33.891 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_osd_cache_size_min": "134217728", 2024-09-21T21:40:33.891 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_osd_crush_smoke_test": "true", 2024-09-21T21:40:33.891 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_osd_destroyed_out_interval": "600", 2024-09-21T21:40:33.891 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_osd_down_out_interval": "600", 2024-09-21T21:40:33.891 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-09-21T21:40:33.892 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-09-21T21:40:33.892 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_osd_force_trim_to": "0", 2024-09-21T21:40:33.892 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_osd_full_ratio": "0.950000", 2024-09-21T21:40:33.892 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-09-21T21:40:33.892 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_osd_laggy_halflife": "3600", 2024-09-21T21:40:33.892 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_osd_laggy_max_interval": "300", 2024-09-21T21:40:33.892 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_osd_laggy_weight": "0.300000", 2024-09-21T21:40:33.892 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-09-21T21:40:33.892 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_osd_max_creating_pgs": "1024", 2024-09-21T21:40:33.892 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_osd_max_initial_pgs": "1024", 2024-09-21T21:40:33.892 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_osd_min_down_reporters": "2", 2024-09-21T21:40:33.892 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-09-21T21:40:33.892 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-09-21T21:40:33.892 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-09-21T21:40:33.893 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_osd_prime_pg_temp": "true", 2024-09-21T21:40:33.893 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-09-21T21:40:33.893 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-09-21T21:40:33.893 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_osd_report_timeout": "900", 2024-09-21T21:40:33.893 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-09-21T21:40:33.893 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-09-21T21:40:33.893 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_osd_warn_num_repaired": "10", 2024-09-21T21:40:33.893 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_osd_warn_op_age": "32.000000", 2024-09-21T21:40:33.893 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-09-21T21:40:33.893 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_osdmap_full_prune_interval": "10", 2024-09-21T21:40:33.893 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_osdmap_full_prune_min": "10000", 2024-09-21T21:40:33.893 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_osdmap_full_prune_txsize": "100", 2024-09-21T21:40:33.893 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-09-21T21:40:33.893 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_pg_stuck_threshold": "60", 2024-09-21T21:40:33.894 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-09-21T21:40:33.894 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_pg_warn_min_objects": "10000", 2024-09-21T21:40:33.894 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_pg_warn_min_per_osd": "0", 2024-09-21T21:40:33.894 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-09-21T21:40:33.894 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_pool_quota_crit_threshold": "0", 2024-09-21T21:40:33.894 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_pool_quota_warn_threshold": "0", 2024-09-21T21:40:33.894 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_probe_timeout": "2.000000", 2024-09-21T21:40:33.894 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_reweight_max_change": "0.050000", 2024-09-21T21:40:33.894 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_reweight_max_osds": "4", 2024-09-21T21:40:33.894 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-09-21T21:40:33.894 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-09-21T21:40:33.894 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-09-21T21:40:33.894 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-09-21T21:40:33.894 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-09-21T21:40:33.895 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_scrub_interval": "86400", 2024-09-21T21:40:33.895 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_scrub_max_keys": "100", 2024-09-21T21:40:33.895 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_scrub_timeout": "300", 2024-09-21T21:40:33.895 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_session_timeout": "300", 2024-09-21T21:40:33.895 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_smart_report_timeout": "5", 2024-09-21T21:40:33.895 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_stat_smooth_intervals": "6", 2024-09-21T21:40:33.895 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-09-21T21:40:33.895 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_stretch_pool_min_size": "2", 2024-09-21T21:40:33.895 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_stretch_pool_size": "4", 2024-09-21T21:40:33.895 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-09-21T21:40:33.895 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_subscribe_interval": "86400.000000", 2024-09-21T21:40:33.895 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_sync_debug": "false", 2024-09-21T21:40:33.896 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_sync_max_payload_keys": "2000", 2024-09-21T21:40:33.896 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_sync_max_payload_size": "1048576", 2024-09-21T21:40:33.896 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_sync_provider_kill_at": "0", 2024-09-21T21:40:33.896 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_sync_requester_kill_at": "0", 2024-09-21T21:40:33.896 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_sync_timeout": "60.000000", 2024-09-21T21:40:33.896 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_target_pg_per_osd": "100", 2024-09-21T21:40:33.896 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_tick_interval": "5", 2024-09-21T21:40:33.896 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_timecheck_interval": "300.000000", 2024-09-21T21:40:33.896 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-09-21T21:40:33.896 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_use_min_delay_socket": "false", 2024-09-21T21:40:33.896 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_warn_older_version_delay": "604800", 2024-09-21T21:40:33.896 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-09-21T21:40:33.897 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-09-21T21:40:33.897 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-09-21T21:40:33.897 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_warn_on_filestore_osds": "false", 2024-09-21T21:40:33.897 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-09-21T21:40:33.897 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-09-21T21:40:33.897 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-09-21T21:40:33.897 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_warn_on_misplaced": "false", 2024-09-21T21:40:33.897 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-09-21T21:40:33.897 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_warn_on_older_version": "true", 2024-09-21T21:40:33.897 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-09-21T21:40:33.897 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_warn_on_pool_no_app": "false", 2024-09-21T21:40:33.897 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-09-21T21:40:33.898 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-09-21T21:40:33.898 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-09-21T21:40:33.898 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-09-21T21:40:33.898 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_warn_on_too_few_osds": "false", 2024-09-21T21:40:33.898 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-09-21T21:40:33.898 INFO:teuthology.orchestra.run.smithi081.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-09-21T21:40:33.898 INFO:teuthology.orchestra.run.smithi081.stdout: "monmap": "", 2024-09-21T21:40:33.898 INFO:teuthology.orchestra.run.smithi081.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-21T21:40:33.898 INFO:teuthology.orchestra.run.smithi081.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-09-21T21:40:33.898 INFO:teuthology.orchestra.run.smithi081.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-09-21T21:40:33.898 INFO:teuthology.orchestra.run.smithi081.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-21T21:40:33.898 INFO:teuthology.orchestra.run.smithi081.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-09-21T21:40:33.898 INFO:teuthology.orchestra.run.smithi081.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-09-21T21:40:33.899 INFO:teuthology.orchestra.run.smithi081.stdout: "motr_tracing_enabled": "false", 2024-09-21T21:40:33.899 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_async_op_threads": "3", 2024-09-21T21:40:33.899 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_async_rdma_buffer_size": "131072", 2024-09-21T21:40:33.899 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_async_rdma_cm": "false", 2024-09-21T21:40:33.899 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_async_rdma_device_name": "", 2024-09-21T21:40:33.899 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_async_rdma_dscp": "96", 2024-09-21T21:40:33.899 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-09-21T21:40:33.899 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_async_rdma_gid_idx": "0", 2024-09-21T21:40:33.899 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_async_rdma_local_gid": "", 2024-09-21T21:40:33.899 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_async_rdma_polling_us": "1000", 2024-09-21T21:40:33.899 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_async_rdma_port_num": "1", 2024-09-21T21:40:33.899 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-09-21T21:40:33.899 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-09-21T21:40:33.900 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_async_rdma_roce_ver": "1", 2024-09-21T21:40:33.900 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_async_rdma_send_buffers": "1024", 2024-09-21T21:40:33.900 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_async_rdma_sl": "3", 2024-09-21T21:40:33.900 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_async_rdma_support_srq": "true", 2024-09-21T21:40:33.900 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_async_rdma_type": "ib", 2024-09-21T21:40:33.900 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_async_reap_threshold": "5", 2024-09-21T21:40:33.900 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_bind_before_connect": "false", 2024-09-21T21:40:33.900 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_bind_ipv4": "true", 2024-09-21T21:40:33.900 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_bind_ipv6": "false", 2024-09-21T21:40:33.900 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_bind_msgr1": "true", 2024-09-21T21:40:33.900 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_bind_msgr2": "true", 2024-09-21T21:40:33.900 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_bind_port_max": "7568", 2024-09-21T21:40:33.900 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_bind_port_min": "6800", 2024-09-21T21:40:33.900 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_bind_prefer_ipv4": "false", 2024-09-21T21:40:33.901 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_bind_retry_count": "3", 2024-09-21T21:40:33.901 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_bind_retry_delay": "5", 2024-09-21T21:40:33.901 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_blackhole_client": "false", 2024-09-21T21:40:33.901 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_blackhole_mds": "false", 2024-09-21T21:40:33.901 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_blackhole_mgr": "false", 2024-09-21T21:40:33.901 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_blackhole_mon": "false", 2024-09-21T21:40:33.901 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_blackhole_osd": "false", 2024-09-21T21:40:33.901 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_client_mode": "crc secure", 2024-09-21T21:40:33.901 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_cluster_mode": "crc secure", 2024-09-21T21:40:33.901 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_cluster_type": "", 2024-09-21T21:40:33.901 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_compress_secure": "false", 2024-09-21T21:40:33.901 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_connection_idle_timeout": "900", 2024-09-21T21:40:33.901 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_connection_ready_timeout": "10", 2024-09-21T21:40:33.901 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_crc_data": "true", 2024-09-21T21:40:33.901 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_crc_header": "true", 2024-09-21T21:40:33.902 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_die_on_bad_msg": "false", 2024-09-21T21:40:33.902 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_die_on_bug": "true", 2024-09-21T21:40:33.902 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_die_on_old_message": "true", 2024-09-21T21:40:33.902 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_die_on_skipped_message": "false", 2024-09-21T21:40:33.902 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_die_on_unhandled_msg": "false", 2024-09-21T21:40:33.902 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-09-21T21:40:33.902 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_dpdk_coremask": "0xF", 2024-09-21T21:40:33.902 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-09-21T21:40:33.902 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_dpdk_devs_allowlist": "", 2024-09-21T21:40:33.902 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_dpdk_enable_tso": "true", 2024-09-21T21:40:33.902 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-09-21T21:40:33.902 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-09-21T21:40:33.902 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_dpdk_hugepages": "", 2024-09-21T21:40:33.903 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_dpdk_hw_flow_control": "true", 2024-09-21T21:40:33.903 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-09-21T21:40:33.903 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_dpdk_lro": "true", 2024-09-21T21:40:33.903 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_dpdk_memory_channel": "4", 2024-09-21T21:40:33.903 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-09-21T21:40:33.903 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_dpdk_pmd": "", 2024-09-21T21:40:33.903 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_dpdk_port_id": "0", 2024-09-21T21:40:33.903 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-09-21T21:40:33.903 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_dump_corrupt_message_level": "1", 2024-09-21T21:40:33.903 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_dump_on_send": "false", 2024-09-21T21:40:33.903 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_initial_backoff": "0.200000", 2024-09-21T21:40:33.904 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_inject_delay_max": "1.000000", 2024-09-21T21:40:33.904 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_inject_delay_probability": "0.000000", 2024-09-21T21:40:33.904 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_inject_delay_type": "", 2024-09-21T21:40:33.904 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_inject_internal_delays": "0.000000", 2024-09-21T21:40:33.904 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_inject_network_congestion": "0", 2024-09-21T21:40:33.904 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_inject_socket_failures": "5000", 2024-09-21T21:40:33.904 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_learn_addr_from_peer": "true", 2024-09-21T21:40:33.904 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_max_accept_failures": "4", 2024-09-21T21:40:33.904 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_max_backoff": "15.000000", 2024-09-21T21:40:33.904 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_mon_client_mode": "secure crc", 2024-09-21T21:40:33.904 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_mon_cluster_mode": "secure crc", 2024-09-21T21:40:33.905 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_mon_service_mode": "secure crc", 2024-09-21T21:40:33.905 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_osd_compress_min_size": "1024", 2024-09-21T21:40:33.905 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_osd_compress_mode": "none", 2024-09-21T21:40:33.905 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_osd_compression_algorithm": "snappy", 2024-09-21T21:40:33.905 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-09-21T21:40:33.905 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_pq_min_cost": "65536", 2024-09-21T21:40:33.905 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_public_type": "", 2024-09-21T21:40:33.905 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_service_mode": "crc secure", 2024-09-21T21:40:33.905 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_tcp_listen_backlog": "512", 2024-09-21T21:40:33.905 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_tcp_nodelay": "true", 2024-09-21T21:40:33.905 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-09-21T21:40:33.905 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_tcp_rcvbuf": "0", 2024-09-21T21:40:33.905 INFO:teuthology.orchestra.run.smithi081.stdout: "ms_type": "async+posix", 2024-09-21T21:40:33.905 INFO:teuthology.orchestra.run.smithi081.stdout: "no_config_file": "false", 2024-09-21T21:40:33.905 INFO:teuthology.orchestra.run.smithi081.stdout: "objecter_completion_locks_per_session": "32", 2024-09-21T21:40:33.906 INFO:teuthology.orchestra.run.smithi081.stdout: "objecter_debug_inject_relock_delay": "false", 2024-09-21T21:40:33.906 INFO:teuthology.orchestra.run.smithi081.stdout: "objecter_inflight_op_bytes": "104857600", 2024-09-21T21:40:33.906 INFO:teuthology.orchestra.run.smithi081.stdout: "objecter_inflight_ops": "1024", 2024-09-21T21:40:33.906 INFO:teuthology.orchestra.run.smithi081.stdout: "objecter_inject_no_watch_ping": "false", 2024-09-21T21:40:33.906 INFO:teuthology.orchestra.run.smithi081.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-09-21T21:40:33.906 INFO:teuthology.orchestra.run.smithi081.stdout: "objecter_tick_interval": "5.000000", 2024-09-21T21:40:33.906 INFO:teuthology.orchestra.run.smithi081.stdout: "objecter_timeout": "10.000000", 2024-09-21T21:40:33.906 INFO:teuthology.orchestra.run.smithi081.stdout: "objectstore_blackhole": "false", 2024-09-21T21:40:33.906 INFO:teuthology.orchestra.run.smithi081.stdout: "objectstore_debug_throw_on_failed_txc": "false", 2024-09-21T21:40:33.906 INFO:teuthology.orchestra.run.smithi081.stdout: "openssl_engine_opts": "", 2024-09-21T21:40:33.906 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_agent_delay_time": "5.000000", 2024-09-21T21:40:33.906 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_agent_hist_halflife": "1000", 2024-09-21T21:40:33.906 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_agent_max_low_ops": "2", 2024-09-21T21:40:33.906 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_agent_max_ops": "4", 2024-09-21T21:40:33.907 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-09-21T21:40:33.907 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_agent_quantize_effort": "0.100000", 2024-09-21T21:40:33.907 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_agent_slop": "0.020000", 2024-09-21T21:40:33.907 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-09-21T21:40:33.907 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_allow_recovery_below_min_size": "true", 2024-09-21T21:40:33.907 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_asio_thread_count": "2", 2024-09-21T21:40:33.907 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_async_recovery_min_cost": "100", 2024-09-21T21:40:33.907 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_auto_mark_unfound_lost": "false", 2024-09-21T21:40:33.907 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_backfill_retry_interval": "30.000000", 2024-09-21T21:40:33.907 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_backfill_scan_max": "512", 2024-09-21T21:40:33.907 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_backfill_scan_min": "64", 2024-09-21T21:40:33.907 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_backoff_on_degraded": "false", 2024-09-21T21:40:33.907 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_backoff_on_peering": "false", 2024-09-21T21:40:33.907 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_backoff_on_unfound": "true", 2024-09-21T21:40:33.908 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_beacon_report_interval": "300", 2024-09-21T21:40:33.908 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_bench_duration": "30", 2024-09-21T21:40:33.908 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-09-21T21:40:33.908 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_bench_max_block_size": "67108864", 2024-09-21T21:40:33.908 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_bench_small_size_max_iops": "100", 2024-09-21T21:40:33.908 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_blkin_trace_all": "false", 2024-09-21T21:40:33.908 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_blocked_scrub_grace_period": "120", 2024-09-21T21:40:33.908 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-09-21T21:40:33.908 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-09-21T21:40:33.908 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-09-21T21:40:33.908 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_check_for_log_corruption": "false", 2024-09-21T21:40:33.908 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-09-21T21:40:33.908 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-21T21:40:33.908 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_class_dir": "/usr/lib64/rados-classes", 2024-09-21T21:40:33.909 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-21T21:40:33.909 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_class_update_on_start": "true", 2024-09-21T21:40:33.909 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_client_message_cap": "256", 2024-09-21T21:40:33.909 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_client_message_size_cap": "524288000", 2024-09-21T21:40:33.909 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_client_op_priority": "63", 2024-09-21T21:40:33.909 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_client_watch_timeout": "30", 2024-09-21T21:40:33.909 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_command_max_records": "256", 2024-09-21T21:40:33.909 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_compact_on_start": "false", 2024-09-21T21:40:33.909 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-09-21T21:40:33.909 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_crush_chooseleaf_type": "0", 2024-09-21T21:40:33.909 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_crush_initial_weight": "-1.000000", 2024-09-21T21:40:33.909 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_crush_update_on_start": "true", 2024-09-21T21:40:33.909 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_crush_update_weight_set": "true", 2024-09-21T21:40:33.909 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-09-21T21:40:33.910 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-09-21T21:40:33.910 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-09-21T21:40:33.910 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_debug_drop_ping_duration": "0", 2024-09-21T21:40:33.910 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-09-21T21:40:33.910 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_debug_feed_pullee": "-1", 2024-09-21T21:40:33.910 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-09-21T21:40:33.910 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-09-21T21:40:33.910 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-09-21T21:40:33.910 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_debug_misdirected_ops": "false", 2024-09-21T21:40:33.910 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_debug_no_acting_change": "false", 2024-09-21T21:40:33.910 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_debug_no_purge_strays": "false", 2024-09-21T21:40:33.910 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_debug_op_order": "false", 2024-09-21T21:40:33.910 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_debug_pg_log_writeout": "false", 2024-09-21T21:40:33.910 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_debug_pretend_recovery_active": "false", 2024-09-21T21:40:33.910 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-09-21T21:40:33.911 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-09-21T21:40:33.911 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_debug_shutdown": "false", 2024-09-21T21:40:33.911 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-09-21T21:40:33.911 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-09-21T21:40:33.911 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_debug_trim_objects": "false", 2024-09-21T21:40:33.911 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_debug_verify_cached_snaps": "false", 2024-09-21T21:40:33.911 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_debug_verify_missing_on_start": "false", 2024-09-21T21:40:33.911 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_debug_verify_snaps": "false", 2024-09-21T21:40:33.911 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-09-21T21:40:33.911 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-09-21T21:40:33.911 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_deep_scrub_keys": "1024", 2024-09-21T21:40:33.911 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-09-21T21:40:33.911 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-09-21T21:40:33.911 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-09-21T21:40:33.912 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_deep_scrub_stride": "524288", 2024-09-21T21:40:33.912 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-09-21T21:40:33.912 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_default_data_pool_replay_window": "5", 2024-09-21T21:40:33.912 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_default_notify_timeout": "30", 2024-09-21T21:40:33.912 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_delete_sleep": "0.000000", 2024-09-21T21:40:33.912 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-09-21T21:40:33.912 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-09-21T21:40:33.912 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-09-21T21:40:33.912 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_discard_disconnected_ops": "true", 2024-09-21T21:40:33.912 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_enable_op_tracker": "true", 2024-09-21T21:40:33.912 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-09-21T21:40:33.912 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_extblkdev_plugins": "vdo", 2024-09-21T21:40:33.912 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-09-21T21:40:33.912 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-09-21T21:40:33.913 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_fast_info": "true", 2024-09-21T21:40:33.913 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_fast_shutdown": "true", 2024-09-21T21:40:33.913 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-09-21T21:40:33.913 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_fast_shutdown_timeout": "15", 2024-09-21T21:40:33.913 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-09-21T21:40:33.913 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-09-21T21:40:33.913 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-09-21T21:40:33.913 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_function_tracing": "false", 2024-09-21T21:40:33.913 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_heartbeat_grace": "20", 2024-09-21T21:40:33.913 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_heartbeat_interval": "6", 2024-09-21T21:40:33.913 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-09-21T21:40:33.913 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_heartbeat_min_peers": "10", 2024-09-21T21:40:33.913 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_heartbeat_min_size": "2000", 2024-09-21T21:40:33.913 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_heartbeat_stale": "600", 2024-09-21T21:40:33.914 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-09-21T21:40:33.914 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_hit_set_max_size": "100000", 2024-09-21T21:40:33.914 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_hit_set_min_size": "1000", 2024-09-21T21:40:33.914 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-09-21T21:40:33.914 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-09-21T21:40:33.914 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-09-21T21:40:33.914 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-09-21T21:40:33.914 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-09-21T21:40:33.914 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_journal_flush_on_shutdown": "true", 2024-09-21T21:40:33.914 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_journal_size": "5120", 2024-09-21T21:40:33.914 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_kill_backfill_at": "0", 2024-09-21T21:40:33.914 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_loop_before_reset_tphandle": "64", 2024-09-21T21:40:33.914 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_map_cache_size": "50", 2024-09-21T21:40:33.914 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_map_dedup": "true", 2024-09-21T21:40:33.914 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_map_message_max": "40", 2024-09-21T21:40:33.915 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_map_message_max_bytes": "10485760", 2024-09-21T21:40:33.915 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_map_share_max_epochs": "40", 2024-09-21T21:40:33.915 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_max_attr_name_len": "100", 2024-09-21T21:40:33.915 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_max_attr_size": "0", 2024-09-21T21:40:33.915 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_max_backfills": "1", 2024-09-21T21:40:33.915 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_max_markdown_count": "5", 2024-09-21T21:40:33.915 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_max_markdown_period": "600", 2024-09-21T21:40:33.915 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_max_object_name_len": "2048", 2024-09-21T21:40:33.915 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_max_object_namespace_len": "256", 2024-09-21T21:40:33.915 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_max_object_size": "134217728", 2024-09-21T21:40:33.915 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-09-21T21:40:33.915 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_max_omap_entries_per_request": "1024", 2024-09-21T21:40:33.915 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_max_pg_blocked_by": "16", 2024-09-21T21:40:33.915 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_max_pg_log_entries": "10000", 2024-09-21T21:40:33.916 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-09-21T21:40:33.916 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_max_pgls": "1024", 2024-09-21T21:40:33.916 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_max_push_cost": "8388608", 2024-09-21T21:40:33.916 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_max_push_objects": "10", 2024-09-21T21:40:33.916 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_max_scrubs": "3", 2024-09-21T21:40:33.916 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-09-21T21:40:33.916 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_max_trimming_pgs": "2", 2024-09-21T21:40:33.916 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_max_write_op_reply_len": "64", 2024-09-21T21:40:33.916 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_max_write_size": "90", 2024-09-21T21:40:33.916 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-09-21T21:40:33.916 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-09-21T21:40:33.916 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-09-21T21:40:33.916 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-09-21T21:40:33.916 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-09-21T21:40:33.917 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-09-21T21:40:33.917 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-09-21T21:40:33.917 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_mclock_override_recovery_settings": "false", 2024-09-21T21:40:33.917 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_mclock_profile": "balanced", 2024-09-21T21:40:33.917 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-09-21T21:40:33.917 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-09-21T21:40:33.917 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-09-21T21:40:33.917 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-09-21T21:40:33.917 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-09-21T21:40:33.917 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-09-21T21:40:33.917 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-09-21T21:40:33.917 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-09-21T21:40:33.917 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-09-21T21:40:33.917 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-09-21T21:40:33.918 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_mclock_skip_benchmark": "false", 2024-09-21T21:40:33.918 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_memory_base": "805306368", 2024-09-21T21:40:33.918 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_memory_cache_min": "134217728", 2024-09-21T21:40:33.918 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-09-21T21:40:33.918 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-09-21T21:40:33.918 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_memory_target": "4294967296", 2024-09-21T21:40:33.918 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_memory_target_autotune": "false", 2024-09-21T21:40:33.918 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-09-21T21:40:33.918 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_min_pg_log_entries": "250", 2024-09-21T21:40:33.918 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_min_recovery_priority": "0", 2024-09-21T21:40:33.918 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_mon_heartbeat_interval": "30", 2024-09-21T21:40:33.918 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-09-21T21:40:33.918 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_mon_report_interval": "5", 2024-09-21T21:40:33.919 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_mon_report_max_in_flight": "2", 2024-09-21T21:40:33.919 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-09-21T21:40:33.919 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_num_cache_shards": "32", 2024-09-21T21:40:33.919 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_num_op_tracker_shard": "32", 2024-09-21T21:40:33.919 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_numa_auto_affinity": "true", 2024-09-21T21:40:33.919 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_numa_node": "-1", 2024-09-21T21:40:33.919 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_numa_prefer_iface": "true", 2024-09-21T21:40:33.919 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-09-21T21:40:33.919 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_objecter_finishers": "1", 2024-09-21T21:40:33.919 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_objectstore": "bluestore", 2024-09-21T21:40:33.919 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_objectstore_fuse": "false", 2024-09-21T21:40:33.919 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_objectstore_tracing": "false", 2024-09-21T21:40:33.919 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_op_complaint_time": "30.000000", 2024-09-21T21:40:33.919 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_op_history_duration": "600", 2024-09-21T21:40:33.919 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_op_history_size": "20", 2024-09-21T21:40:33.920 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_op_history_slow_op_size": "20", 2024-09-21T21:40:33.920 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-09-21T21:40:33.920 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_op_log_threshold": "5", 2024-09-21T21:40:33.920 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_op_num_shards": "0", 2024-09-21T21:40:33.920 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_op_num_shards_hdd": "5", 2024-09-21T21:40:33.920 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_op_num_shards_ssd": "8", 2024-09-21T21:40:33.920 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_op_num_threads_per_shard": "0", 2024-09-21T21:40:33.920 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-09-21T21:40:33.920 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-09-21T21:40:33.920 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-09-21T21:40:33.920 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_op_pq_min_cost": "65536", 2024-09-21T21:40:33.920 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_op_queue": "mclock_scheduler", 2024-09-21T21:40:33.920 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_op_queue_cut_off": "high", 2024-09-21T21:40:33.921 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_op_thread_suicide_timeout": "150", 2024-09-21T21:40:33.921 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_op_thread_timeout": "15", 2024-09-21T21:40:33.921 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_open_classes_on_start": "true", 2024-09-21T21:40:33.921 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_os_flags": "0", 2024-09-21T21:40:33.921 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_peering_op_priority": "255", 2024-09-21T21:40:33.921 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_pg_delete_cost": "1048576", 2024-09-21T21:40:33.921 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_pg_delete_priority": "5", 2024-09-21T21:40:33.921 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-09-21T21:40:33.921 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-09-21T21:40:33.921 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_pg_log_dups_tracked": "3000", 2024-09-21T21:40:33.921 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_pg_log_trim_max": "10000", 2024-09-21T21:40:33.921 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_pg_log_trim_min": "100", 2024-09-21T21:40:33.921 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-09-21T21:40:33.921 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_pg_object_context_cache_count": "64", 2024-09-21T21:40:33.922 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_pg_stat_report_interval_max_epochs": "500", 2024-09-21T21:40:33.922 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_pg_stat_report_interval_max_seconds": "5", 2024-09-21T21:40:33.922 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-09-21T21:40:33.922 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-09-21T21:40:33.922 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-09-21T21:40:33.922 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-09-21T21:40:33.922 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-09-21T21:40:33.922 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-09-21T21:40:33.922 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_pool_default_crimson": "false", 2024-09-21T21:40:33.922 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_pool_default_crush_rule": "-1", 2024-09-21T21:40:33.922 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_pool_default_ec_fast_read": "false", 2024-09-21T21:40:33.922 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-09-21T21:40:33.922 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_pool_default_flag_bulk": "false", 2024-09-21T21:40:33.922 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-09-21T21:40:33.923 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_pool_default_flag_nodelete": "false", 2024-09-21T21:40:33.923 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-09-21T21:40:33.923 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-09-21T21:40:33.923 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_pool_default_flags": "0", 2024-09-21T21:40:33.923 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-09-21T21:40:33.923 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_pool_default_min_size": "0", 2024-09-21T21:40:33.923 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-09-21T21:40:33.923 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_pool_default_pg_num": "32", 2024-09-21T21:40:33.923 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_pool_default_pgp_num": "0", 2024-09-21T21:40:33.923 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-09-21T21:40:33.923 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_pool_default_read_ratio": "70", 2024-09-21T21:40:33.923 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_pool_default_size": "2", 2024-09-21T21:40:33.923 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_pool_default_type": "replicated", 2024-09-21T21:40:33.923 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-09-21T21:40:33.924 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_pool_use_gmt_hitset": "true", 2024-09-21T21:40:33.924 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_push_per_object_cost": "1000", 2024-09-21T21:40:33.924 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_read_ec_check_for_errors": "false", 2024-09-21T21:40:33.924 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_recover_clone_overlap": "true", 2024-09-21T21:40:33.924 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_recover_clone_overlap_limit": "10", 2024-09-21T21:40:33.924 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_recovery_cost": "20971520", 2024-09-21T21:40:33.924 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_recovery_delay_start": "0.000000", 2024-09-21T21:40:33.924 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_recovery_max_active": "0", 2024-09-21T21:40:33.924 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_recovery_max_active_hdd": "3", 2024-09-21T21:40:33.924 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_recovery_max_active_ssd": "10", 2024-09-21T21:40:33.924 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_recovery_max_chunk": "8388608", 2024-09-21T21:40:33.924 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-09-21T21:40:33.924 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_recovery_max_single_start": "1", 2024-09-21T21:40:33.924 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_recovery_op_priority": "3", 2024-09-21T21:40:33.925 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_recovery_op_warn_multiple": "16", 2024-09-21T21:40:33.925 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_recovery_priority": "5", 2024-09-21T21:40:33.925 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_recovery_retry_interval": "30.000000", 2024-09-21T21:40:33.925 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_recovery_sleep": "0.000000", 2024-09-21T21:40:33.925 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-09-21T21:40:33.925 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-09-21T21:40:33.925 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-09-21T21:40:33.925 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_repair_during_recovery": "false", 2024-09-21T21:40:33.925 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_requested_scrub_priority": "120", 2024-09-21T21:40:33.925 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-09-21T21:40:33.925 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_rollback_to_cluster_snap": "", 2024-09-21T21:40:33.925 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_scrub_auto_repair": "false", 2024-09-21T21:40:33.925 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-09-21T21:40:33.926 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-09-21T21:40:33.926 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_scrub_begin_hour": "0", 2024-09-21T21:40:33.926 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_scrub_begin_week_day": "0", 2024-09-21T21:40:33.926 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_scrub_chunk_max": "25", 2024-09-21T21:40:33.926 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_scrub_chunk_min": "5", 2024-09-21T21:40:33.926 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_scrub_cost": "52428800", 2024-09-21T21:40:33.926 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_scrub_disable_reservation_queuing": "false", 2024-09-21T21:40:33.926 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_scrub_during_recovery": "false", 2024-09-21T21:40:33.926 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_scrub_end_hour": "0", 2024-09-21T21:40:33.926 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_scrub_end_week_day": "0", 2024-09-21T21:40:33.926 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_scrub_event_cost": "4096", 2024-09-21T21:40:33.926 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-09-21T21:40:33.927 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-09-21T21:40:33.927 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_scrub_invalid_stats": "true", 2024-09-21T21:40:33.927 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_scrub_load_threshold": "0.500000", 2024-09-21T21:40:33.927 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_scrub_max_interval": "604800.000000", 2024-09-21T21:40:33.927 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_scrub_max_preemptions": "5", 2024-09-21T21:40:33.927 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_scrub_min_interval": "86400.000000", 2024-09-21T21:40:33.927 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_scrub_priority": "5", 2024-09-21T21:40:33.927 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_scrub_sleep": "0.000000", 2024-09-21T21:40:33.927 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-09-21T21:40:33.927 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-09-21T21:40:33.927 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_shutdown_pgref_assert": "false", 2024-09-21T21:40:33.928 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_skip_data_digest": "false", 2024-09-21T21:40:33.928 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_smart_report_timeout": "5", 2024-09-21T21:40:33.928 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_snap_trim_cost": "1048576", 2024-09-21T21:40:33.928 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_snap_trim_priority": "5", 2024-09-21T21:40:33.928 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_snap_trim_sleep": "0.000000", 2024-09-21T21:40:33.928 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-09-21T21:40:33.928 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-09-21T21:40:33.928 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-09-21T21:40:33.928 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-09-21T21:40:33.928 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_stats_update_period_scrubbing": "15", 2024-09-21T21:40:33.928 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-09-21T21:40:33.928 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_target_transaction_size": "30", 2024-09-21T21:40:33.928 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-09-21T21:40:33.929 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-09-21T21:40:33.929 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-09-21T21:40:33.929 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-09-21T21:40:33.929 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-09-21T21:40:33.929 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-09-21T21:40:33.929 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-09-21T21:40:33.929 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_tier_default_cache_mode": "writeback", 2024-09-21T21:40:33.929 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-09-21T21:40:33.929 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-09-21T21:40:33.929 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_tracing": "false", 2024-09-21T21:40:33.929 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_use_stale_snap": "false", 2024-09-21T21:40:33.929 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-09-21T21:40:33.929 INFO:teuthology.orchestra.run.smithi081.stdout: "osdc_blkin_trace_all": "false", 2024-09-21T21:40:33.930 INFO:teuthology.orchestra.run.smithi081.stdout: "paxos_kill_at": "0", 2024-09-21T21:40:33.930 INFO:teuthology.orchestra.run.smithi081.stdout: "paxos_max_join_drift": "10", 2024-09-21T21:40:33.930 INFO:teuthology.orchestra.run.smithi081.stdout: "paxos_min": "500", 2024-09-21T21:40:33.930 INFO:teuthology.orchestra.run.smithi081.stdout: "paxos_min_wait": "0.050000", 2024-09-21T21:40:33.930 INFO:teuthology.orchestra.run.smithi081.stdout: "paxos_propose_interval": "1.000000", 2024-09-21T21:40:33.930 INFO:teuthology.orchestra.run.smithi081.stdout: "paxos_service_trim_max": "500", 2024-09-21T21:40:33.930 INFO:teuthology.orchestra.run.smithi081.stdout: "paxos_service_trim_max_multiplier": "20", 2024-09-21T21:40:33.930 INFO:teuthology.orchestra.run.smithi081.stdout: "paxos_service_trim_min": "250", 2024-09-21T21:40:33.930 INFO:teuthology.orchestra.run.smithi081.stdout: "paxos_stash_full_interval": "25", 2024-09-21T21:40:33.930 INFO:teuthology.orchestra.run.smithi081.stdout: "paxos_trim_max": "500", 2024-09-21T21:40:33.930 INFO:teuthology.orchestra.run.smithi081.stdout: "paxos_trim_min": "250", 2024-09-21T21:40:33.930 INFO:teuthology.orchestra.run.smithi081.stdout: "perf": "true", 2024-09-21T21:40:33.930 INFO:teuthology.orchestra.run.smithi081.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-09-21T21:40:33.930 INFO:teuthology.orchestra.run.smithi081.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-09-21T21:40:33.931 INFO:teuthology.orchestra.run.smithi081.stdout: "plugin_dir": "/usr/lib64/ceph", 2024-09-21T21:40:33.931 INFO:teuthology.orchestra.run.smithi081.stdout: "public_addr": "-", 2024-09-21T21:40:33.931 INFO:teuthology.orchestra.run.smithi081.stdout: "public_addrv": "", 2024-09-21T21:40:33.931 INFO:teuthology.orchestra.run.smithi081.stdout: "public_bind_addr": "-", 2024-09-21T21:40:33.931 INFO:teuthology.orchestra.run.smithi081.stdout: "public_network": "", 2024-09-21T21:40:33.931 INFO:teuthology.orchestra.run.smithi081.stdout: "public_network_interface": "", 2024-09-21T21:40:33.931 INFO:teuthology.orchestra.run.smithi081.stdout: "qat_compressor_busy_polling": "false", 2024-09-21T21:40:33.931 INFO:teuthology.orchestra.run.smithi081.stdout: "qat_compressor_enabled": "false", 2024-09-21T21:40:33.931 INFO:teuthology.orchestra.run.smithi081.stdout: "qat_compressor_session_max_number": "256", 2024-09-21T21:40:33.931 INFO:teuthology.orchestra.run.smithi081.stdout: "rados_mon_op_timeout": "0", 2024-09-21T21:40:33.931 INFO:teuthology.orchestra.run.smithi081.stdout: "rados_osd_op_timeout": "0", 2024-09-21T21:40:33.931 INFO:teuthology.orchestra.run.smithi081.stdout: "rados_tracing": "false", 2024-09-21T21:40:33.931 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_atime_update_interval": "60", 2024-09-21T21:40:33.932 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-09-21T21:40:33.932 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_balance_parent_reads": "false", 2024-09-21T21:40:33.932 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_balance_snap_reads": "false", 2024-09-21T21:40:33.932 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_blkin_trace_all": "false", 2024-09-21T21:40:33.932 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_blocklist_expire_seconds": "0", 2024-09-21T21:40:33.932 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_blocklist_on_break_lock": "true", 2024-09-21T21:40:33.932 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_cache": "true", 2024-09-21T21:40:33.932 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_cache_block_writes_upfront": "false", 2024-09-21T21:40:33.932 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_cache_max_dirty": "25165824", 2024-09-21T21:40:33.932 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-09-21T21:40:33.932 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_cache_max_dirty_object": "0", 2024-09-21T21:40:33.932 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_cache_policy": "writearound", 2024-09-21T21:40:33.932 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_cache_size": "33554432", 2024-09-21T21:40:33.932 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_cache_target_dirty": "16777216", 2024-09-21T21:40:33.933 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-09-21T21:40:33.933 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_clone_copy_on_read": "false", 2024-09-21T21:40:33.933 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_compression_hint": "none", 2024-09-21T21:40:33.933 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_concurrent_management_ops": "10", 2024-09-21T21:40:33.933 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-09-21T21:40:33.933 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_default_clone_format": "auto", 2024-09-21T21:40:33.933 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_default_data_pool": "", 2024-09-21T21:40:33.933 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_default_features": "61", 2024-09-21T21:40:33.933 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_default_format": "2", 2024-09-21T21:40:33.933 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_default_map_options": "", 2024-09-21T21:40:33.933 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_default_order": "22", 2024-09-21T21:40:33.933 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_default_pool": "rbd", 2024-09-21T21:40:33.933 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-09-21T21:40:33.933 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_default_stripe_count": "0", 2024-09-21T21:40:33.933 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_default_stripe_unit": "0", 2024-09-21T21:40:33.933 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_disable_zero_copy_writes": "true", 2024-09-21T21:40:33.934 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_discard_granularity_bytes": "65536", 2024-09-21T21:40:33.934 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-09-21T21:40:33.934 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_enable_alloc_hint": "true", 2024-09-21T21:40:33.934 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-09-21T21:40:33.934 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_io_scheduler": "simple", 2024-09-21T21:40:33.934 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-09-21T21:40:33.934 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_journal_commit_age": "5.000000", 2024-09-21T21:40:33.934 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-09-21T21:40:33.934 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-09-21T21:40:33.934 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-09-21T21:40:33.934 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-09-21T21:40:33.934 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_journal_object_flush_interval": "0", 2024-09-21T21:40:33.934 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-09-21T21:40:33.934 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-09-21T21:40:33.934 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_journal_order": "24", 2024-09-21T21:40:33.935 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_journal_pool": "", 2024-09-21T21:40:33.935 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_journal_splay_width": "4", 2024-09-21T21:40:33.935 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_localize_parent_reads": "false", 2024-09-21T21:40:33.935 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_localize_snap_reads": "false", 2024-09-21T21:40:33.935 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-09-21T21:40:33.935 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-09-21T21:40:33.935 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-09-21T21:40:33.935 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-09-21T21:40:33.935 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-09-21T21:40:33.935 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-09-21T21:40:33.935 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_mirror_image_policy_type": "simple", 2024-09-21T21:40:33.935 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-09-21T21:40:33.935 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-09-21T21:40:33.935 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-09-21T21:40:33.936 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-09-21T21:40:33.936 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-09-21T21:40:33.936 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-09-21T21:40:33.936 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-09-21T21:40:33.936 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_mirror_memory_autotune": "true", 2024-09-21T21:40:33.936 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_mirror_memory_base": "805306368", 2024-09-21T21:40:33.936 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-09-21T21:40:33.936 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-09-21T21:40:33.936 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-09-21T21:40:33.936 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-09-21T21:40:33.936 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_mirror_memory_target": "4294967296", 2024-09-21T21:40:33.936 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-09-21T21:40:33.936 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-09-21T21:40:33.937 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-09-21T21:40:33.937 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_mirroring_delete_delay": "0", 2024-09-21T21:40:33.937 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-09-21T21:40:33.937 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_mirroring_replay_delay": "0", 2024-09-21T21:40:33.937 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-09-21T21:40:33.937 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-09-21T21:40:33.937 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_move_to_trash_on_remove": "false", 2024-09-21T21:40:33.937 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-09-21T21:40:33.937 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_mtime_update_interval": "60", 2024-09-21T21:40:33.937 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_non_blocking_aio": "true", 2024-09-21T21:40:33.937 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_op_thread_timeout": "60", 2024-09-21T21:40:33.937 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_op_threads": "1", 2024-09-21T21:40:33.937 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_parent_cache_enabled": "false", 2024-09-21T21:40:33.937 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_persistent_cache_mode": "disabled", 2024-09-21T21:40:33.938 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_persistent_cache_path": "/tmp", 2024-09-21T21:40:33.938 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_persistent_cache_size": "1073741824", 2024-09-21T21:40:33.938 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_plugins": "", 2024-09-21T21:40:33.938 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_qos_bps_burst": "0", 2024-09-21T21:40:33.938 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-09-21T21:40:33.938 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_qos_bps_limit": "0", 2024-09-21T21:40:33.938 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_qos_exclude_ops": "0", 2024-09-21T21:40:33.938 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_qos_iops_burst": "0", 2024-09-21T21:40:33.938 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-09-21T21:40:33.938 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_qos_iops_limit": "0", 2024-09-21T21:40:33.938 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_qos_read_bps_burst": "0", 2024-09-21T21:40:33.938 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-09-21T21:40:33.938 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_qos_read_bps_limit": "0", 2024-09-21T21:40:33.938 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_qos_read_iops_burst": "0", 2024-09-21T21:40:33.939 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-09-21T21:40:33.939 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_qos_read_iops_limit": "0", 2024-09-21T21:40:33.939 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_qos_schedule_tick_min": "50", 2024-09-21T21:40:33.939 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_qos_write_bps_burst": "0", 2024-09-21T21:40:33.939 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-09-21T21:40:33.939 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_qos_write_bps_limit": "0", 2024-09-21T21:40:33.939 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_qos_write_iops_burst": "0", 2024-09-21T21:40:33.939 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-09-21T21:40:33.939 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_qos_write_iops_limit": "0", 2024-09-21T21:40:33.939 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_quiesce_notification_attempts": "10", 2024-09-21T21:40:33.939 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_read_from_replica_policy": "default", 2024-09-21T21:40:33.940 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-09-21T21:40:33.940 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_readahead_max_bytes": "524288", 2024-09-21T21:40:33.940 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_readahead_trigger_requests": "10", 2024-09-21T21:40:33.940 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_request_timed_out_seconds": "30", 2024-09-21T21:40:33.940 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_skip_partial_discard": "true", 2024-09-21T21:40:33.940 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-09-21T21:40:33.940 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_tracing": "false", 2024-09-21T21:40:33.940 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_validate_names": "true", 2024-09-21T21:40:33.940 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd_validate_pool": "true", 2024-09-21T21:40:33.940 INFO:teuthology.orchestra.run.smithi081.stdout: "restapi_base_url": "", 2024-09-21T21:40:33.940 INFO:teuthology.orchestra.run.smithi081.stdout: "restapi_log_level": "", 2024-09-21T21:40:33.940 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_account_default_quota_max_objects": "-1", 2024-09-21T21:40:33.940 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_account_default_quota_max_size": "-1", 2024-09-21T21:40:33.940 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_acl_grants_max_num": "100", 2024-09-21T21:40:33.941 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_admin_entry": "admin", 2024-09-21T21:40:33.941 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-09-21T21:40:33.941 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_backend_store": "rados", 2024-09-21T21:40:33.941 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_barbican_url": "", 2024-09-21T21:40:33.941 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_beast_enable_async": "true", 2024-09-21T21:40:33.941 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_bucket_counters_cache": "false", 2024-09-21T21:40:33.941 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_bucket_counters_cache_size": "10000", 2024-09-21T21:40:33.941 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-09-21T21:40:33.941 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-09-21T21:40:33.941 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_bucket_index_max_aio": "128", 2024-09-21T21:40:33.941 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-09-21T21:40:33.941 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-09-21T21:40:33.941 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_bucket_quota_ttl": "600", 2024-09-21T21:40:33.941 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-09-21T21:40:33.942 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_cache_enabled": "true", 2024-09-21T21:40:33.942 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_cache_expiry_interval": "900", 2024-09-21T21:40:33.942 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_cache_lru_size": "25000", 2024-09-21T21:40:33.942 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_config_store": "rados", 2024-09-21T21:40:33.942 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_content_length_compat": "false", 2024-09-21T21:40:33.942 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_copy_obj_progress": "true", 2024-09-21T21:40:33.942 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-09-21T21:40:33.942 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_cors_rules_max_num": "100", 2024-09-21T21:40:33.942 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_cross_domain_policy": "", 2024-09-21T21:40:33.942 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_crypt_default_encryption_key": "", 2024-09-21T21:40:33.942 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_crypt_kmip_addr": "", 2024-09-21T21:40:33.942 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_crypt_kmip_ca_path": "", 2024-09-21T21:40:33.943 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_crypt_kmip_client_cert": "", 2024-09-21T21:40:33.943 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_crypt_kmip_client_key": "", 2024-09-21T21:40:33.943 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-09-21T21:40:33.943 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_crypt_kmip_password": "", 2024-09-21T21:40:33.943 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-09-21T21:40:33.943 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_crypt_kmip_username": "", 2024-09-21T21:40:33.943 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_crypt_require_ssl": "true", 2024-09-21T21:40:33.943 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-09-21T21:40:33.943 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-09-21T21:40:33.943 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-09-21T21:40:33.943 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-09-21T21:40:33.944 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-09-21T21:40:33.944 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-09-21T21:40:33.944 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-09-21T21:40:33.944 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-09-21T21:40:33.944 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-09-21T21:40:33.944 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-09-21T21:40:33.944 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-09-21T21:40:33.944 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-09-21T21:40:33.944 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-09-21T21:40:33.944 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-09-21T21:40:33.945 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_crypt_suppress_logs": "true", 2024-09-21T21:40:33.945 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_crypt_vault_addr": "", 2024-09-21T21:40:33.945 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_crypt_vault_auth": "token", 2024-09-21T21:40:33.945 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_crypt_vault_namespace": "", 2024-09-21T21:40:33.945 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_crypt_vault_prefix": "", 2024-09-21T21:40:33.945 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-09-21T21:40:33.945 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-09-21T21:40:33.945 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-09-21T21:40:33.945 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-09-21T21:40:33.945 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_crypt_vault_token_file": "", 2024-09-21T21:40:33.945 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-09-21T21:40:33.945 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_curl_buffersize": "524288", 2024-09-21T21:40:33.946 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_curl_low_speed_limit": "1024", 2024-09-21T21:40:33.946 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_curl_low_speed_time": "300", 2024-09-21T21:40:33.946 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_curl_tcp_keepalive": "0", 2024-09-21T21:40:33.946 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-09-21T21:40:33.946 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-09-21T21:40:33.946 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-09-21T21:40:33.946 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-09-21T21:40:33.946 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-09-21T21:40:33.946 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_d3n_l1_fadvise": "4", 2024-09-21T21:40:33.946 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-09-21T21:40:33.946 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-09-21T21:40:33.947 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-09-21T21:40:33.947 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_d4n_host": "127.0.0.1", 2024-09-21T21:40:33.947 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_d4n_port": "6379", 2024-09-21T21:40:33.947 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-09-21T21:40:33.947 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_data_log_changes_size": "1000", 2024-09-21T21:40:33.947 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_data_log_num_shards": "128", 2024-09-21T21:40:33.947 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-09-21T21:40:33.947 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_data_log_window": "30", 2024-09-21T21:40:33.947 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_data_notify_interval_msec": "0", 2024-09-21T21:40:33.947 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_data_sync_poll_interval": "20", 2024-09-21T21:40:33.947 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_data_sync_spawn_window": "20", 2024-09-21T21:40:33.948 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_debug_inject_latency_bi_unlink": "0", 2024-09-21T21:40:33.948 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-09-21T21:40:33.948 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-09-21T21:40:33.948 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_default_data_log_backing": "fifo", 2024-09-21T21:40:33.948 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-09-21T21:40:33.948 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_default_region_info_oid": "default.region", 2024-09-21T21:40:33.948 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-09-21T21:40:33.948 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-09-21T21:40:33.948 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_defer_to_bucket_acls": "", 2024-09-21T21:40:33.948 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-09-21T21:40:33.948 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-09-21T21:40:33.948 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-09-21T21:40:33.948 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-09-21T21:40:33.949 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-09-21T21:40:33.949 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-09-21T21:40:33.949 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-09-21T21:40:33.949 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-09-21T21:40:33.949 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_dmclock_data_res": "500.000000", 2024-09-21T21:40:33.949 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-09-21T21:40:33.949 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-09-21T21:40:33.949 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-09-21T21:40:33.949 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-09-21T21:40:33.949 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_dns_name": "", 2024-09-21T21:40:33.949 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_dns_s3website_name": "", 2024-09-21T21:40:33.949 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_dynamic_resharding": "true", 2024-09-21T21:40:33.949 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-09-21T21:40:33.950 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_enable_gc_threads": "true", 2024-09-21T21:40:33.950 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_enable_lc_threads": "true", 2024-09-21T21:40:33.950 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_enable_ops_log": "false", 2024-09-21T21:40:33.950 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_enable_quota_threads": "true", 2024-09-21T21:40:33.950 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_enable_static_website": "false", 2024-09-21T21:40:33.950 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_enable_usage_log": "false", 2024-09-21T21:40:33.950 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_enforce_swift_acls": "true", 2024-09-21T21:40:33.950 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_exit_timeout_secs": "120", 2024-09-21T21:40:33.950 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_expose_bucket": "false", 2024-09-21T21:40:33.950 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_extended_http_attrs": "", 2024-09-21T21:40:33.950 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_filter": "none", 2024-09-21T21:40:33.950 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-09-21T21:40:33.950 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_frontends": "beast port=7480", 2024-09-21T21:40:33.951 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_gc_max_concurrent_io": "10", 2024-09-21T21:40:33.951 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_gc_max_deferred": "50", 2024-09-21T21:40:33.951 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-09-21T21:40:33.951 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_gc_max_objs": "32", 2024-09-21T21:40:33.951 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_gc_max_queue_size": "134213632", 2024-09-21T21:40:33.951 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_gc_max_trim_chunk": "16", 2024-09-21T21:40:33.951 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_gc_obj_min_wait": "7200", 2024-09-21T21:40:33.951 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_gc_processor_max_time": "3600", 2024-09-21T21:40:33.951 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_gc_processor_period": "3600", 2024-09-21T21:40:33.951 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-09-21T21:40:33.951 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_get_obj_window_size": "16777216", 2024-09-21T21:40:33.951 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_healthcheck_disabling_path": "", 2024-09-21T21:40:33.951 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_ignore_get_invalid_range": "false", 2024-09-21T21:40:33.952 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_init_timeout": "300", 2024-09-21T21:40:33.952 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-09-21T21:40:33.952 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-09-21T21:40:33.952 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_kafka_connection_idle": "300", 2024-09-21T21:40:33.952 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_kafka_message_timeout": "5000", 2024-09-21T21:40:33.952 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_kafka_sleep_timeout": "10", 2024-09-21T21:40:33.952 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-09-21T21:40:33.952 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_keystone_accepted_reader_roles": "", 2024-09-21T21:40:33.952 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-09-21T21:40:33.952 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_keystone_admin_domain": "", 2024-09-21T21:40:33.952 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_keystone_admin_password": "", 2024-09-21T21:40:33.953 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_keystone_admin_password_path": "", 2024-09-21T21:40:33.953 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_keystone_admin_project": "", 2024-09-21T21:40:33.953 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_keystone_admin_tenant": "", 2024-09-21T21:40:33.953 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_keystone_admin_token": "", 2024-09-21T21:40:33.953 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_keystone_admin_token_path": "", 2024-09-21T21:40:33.953 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_keystone_admin_user": "", 2024-09-21T21:40:33.953 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_keystone_api_version": "2", 2024-09-21T21:40:33.953 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_keystone_barbican_domain": "", 2024-09-21T21:40:33.953 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_keystone_barbican_password": "", 2024-09-21T21:40:33.953 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_keystone_barbican_project": "", 2024-09-21T21:40:33.953 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_keystone_barbican_tenant": "", 2024-09-21T21:40:33.953 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_keystone_barbican_user": "", 2024-09-21T21:40:33.953 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-09-21T21:40:33.953 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_keystone_implicit_tenants": "false", 2024-09-21T21:40:33.953 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-09-21T21:40:33.954 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_keystone_service_token_enabled": "false", 2024-09-21T21:40:33.954 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_keystone_token_cache_size": "10000", 2024-09-21T21:40:33.954 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_keystone_url": "", 2024-09-21T21:40:33.954 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_keystone_verify_ssl": "true", 2024-09-21T21:40:33.954 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_lc_debug_interval": "-1", 2024-09-21T21:40:33.954 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_lc_lock_max_time": "90", 2024-09-21T21:40:33.954 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_lc_max_objs": "32", 2024-09-21T21:40:33.954 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_lc_max_rules": "1000", 2024-09-21T21:40:33.954 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_lc_max_worker": "3", 2024-09-21T21:40:33.954 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_lc_max_wp_worker": "3", 2024-09-21T21:40:33.954 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_lc_thread_delay": "0", 2024-09-21T21:40:33.954 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-09-21T21:40:33.954 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_ldap_dnattr": "uid", 2024-09-21T21:40:33.955 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-09-21T21:40:33.955 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_ldap_searchfilter": "", 2024-09-21T21:40:33.955 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-09-21T21:40:33.955 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_ldap_uri": "", 2024-09-21T21:40:33.955 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-09-21T21:40:33.955 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-09-21T21:40:33.955 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-09-21T21:40:33.955 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_log_http_headers": "", 2024-09-21T21:40:33.955 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_log_nonexistent_bucket": "false", 2024-09-21T21:40:33.955 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-09-21T21:40:33.955 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_log_object_name_utc": "false", 2024-09-21T21:40:33.955 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_lua_max_memory_per_state": "128000", 2024-09-21T21:40:33.955 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_luarocks_location": "/tmp/rgw_luarocks/mon.a", 2024-09-21T21:40:33.955 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_max_attr_name_len": "0", 2024-09-21T21:40:33.956 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_max_attr_size": "0", 2024-09-21T21:40:33.956 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_max_attrs_num_in_req": "0", 2024-09-21T21:40:33.956 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_max_chunk_size": "4194304", 2024-09-21T21:40:33.956 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_max_concurrent_requests": "1024", 2024-09-21T21:40:33.956 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-09-21T21:40:33.956 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_max_dynamic_shards": "1999", 2024-09-21T21:40:33.956 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_max_listing_results": "1000", 2024-09-21T21:40:33.956 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_max_notify_retries": "10", 2024-09-21T21:40:33.956 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_max_objs_per_shard": "100000", 2024-09-21T21:40:33.956 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_max_put_param_size": "1048576", 2024-09-21T21:40:33.956 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_max_put_size": "5368709120", 2024-09-21T21:40:33.956 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_max_slo_entries": "1000", 2024-09-21T21:40:33.956 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_md_log_max_shards": "64", 2024-09-21T21:40:33.957 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_md_notify_interval_msec": "200", 2024-09-21T21:40:33.957 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_meta_sync_poll_interval": "20", 2024-09-21T21:40:33.957 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_meta_sync_spawn_window": "20", 2024-09-21T21:40:33.957 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-09-21T21:40:33.957 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_mp_lock_max_time": "600", 2024-09-21T21:40:33.957 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-09-21T21:40:33.957 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_multipart_min_part_size": "5242880", 2024-09-21T21:40:33.957 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-09-21T21:40:33.957 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-09-21T21:40:33.957 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_nfs_fhcache_size": "2017", 2024-09-21T21:40:33.957 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-09-21T21:40:33.957 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-09-21T21:40:33.957 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_nfs_lru_lanes": "5", 2024-09-21T21:40:33.957 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_nfs_max_gc": "300", 2024-09-21T21:40:33.958 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-09-21T21:40:33.958 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_nfs_run_gc_threads": "false", 2024-09-21T21:40:33.958 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_nfs_run_lc_threads": "false", 2024-09-21T21:40:33.958 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_nfs_run_quota_threads": "false", 2024-09-21T21:40:33.958 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_nfs_run_sync_thread": "false", 2024-09-21T21:40:33.958 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-09-21T21:40:33.958 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-09-21T21:40:33.958 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_num_async_rados_threads": "32", 2024-09-21T21:40:33.958 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_num_control_oids": "8", 2024-09-21T21:40:33.958 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_numa_node": "-1", 2024-09-21T21:40:33.958 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_obj_stripe_size": "4194304", 2024-09-21T21:40:33.958 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-09-21T21:40:33.958 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_objexp_chunk_size": "100", 2024-09-21T21:40:33.958 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_objexp_gc_interval": "600", 2024-09-21T21:40:33.959 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_objexp_hints_num_shards": "127", 2024-09-21T21:40:33.959 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-09-21T21:40:33.959 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-09-21T21:40:33.959 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_op_thread_timeout": "600", 2024-09-21T21:40:33.959 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_op_tracing": "false", 2024-09-21T21:40:33.959 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_opa_token": "", 2024-09-21T21:40:33.959 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_opa_url": "", 2024-09-21T21:40:33.959 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_opa_verify_ssl": "true", 2024-09-21T21:40:33.959 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-09-21T21:40:33.959 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-09-21T21:40:33.959 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_ops_log_rados": "false", 2024-09-21T21:40:33.959 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_ops_log_socket_path": "", 2024-09-21T21:40:33.959 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-09-21T21:40:33.959 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-09-21T21:40:33.959 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-09-21T21:40:33.960 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_period_push_interval": "2.000000", 2024-09-21T21:40:33.960 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_period_push_interval_max": "30.000000", 2024-09-21T21:40:33.960 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_period_root_pool": ".rgw.root", 2024-09-21T21:40:33.960 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-09-21T21:40:33.960 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_posix_base_path": "/tmp/rgw_posix_driver", 2024-09-21T21:40:33.960 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_posix_cache_lanes": "3", 2024-09-21T21:40:33.960 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_posix_cache_lmdb_count": "3", 2024-09-21T21:40:33.960 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_posix_cache_max_buckets": "100", 2024-09-21T21:40:33.960 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_posix_cache_partitions": "3", 2024-09-21T21:40:33.960 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_posix_database_root": "/var/lib/ceph/radosgw", 2024-09-21T21:40:33.960 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_print_continue": "true", 2024-09-21T21:40:33.960 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_print_prohibited_content_length": "false", 2024-09-21T21:40:33.960 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-09-21T21:40:33.960 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-09-21T21:40:33.961 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-09-21T21:40:33.961 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-09-21T21:40:33.961 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_rados_tracing": "false", 2024-09-21T21:40:33.961 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_realm": "", 2024-09-21T21:40:33.961 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_realm_id": "", 2024-09-21T21:40:33.961 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-09-21T21:40:33.961 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_region": "", 2024-09-21T21:40:33.961 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_region_root_pool": ".rgw.root", 2024-09-21T21:40:33.961 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_relaxed_region_enforcement": "false", 2024-09-21T21:40:33.961 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-09-21T21:40:33.961 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-09-21T21:40:33.961 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_request_uri": "", 2024-09-21T21:40:33.961 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_reshard_batch_size": "64", 2024-09-21T21:40:33.962 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-09-21T21:40:33.962 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_reshard_max_aio": "128", 2024-09-21T21:40:33.962 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_reshard_num_logs": "16", 2024-09-21T21:40:33.962 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_reshard_thread_interval": "600", 2024-09-21T21:40:33.962 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_resolve_cname": "false", 2024-09-21T21:40:33.962 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_rest_getusage_op_compat": "false", 2024-09-21T21:40:33.962 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_run_sync_thread": "true", 2024-09-21T21:40:33.962 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_s3_auth_disable_signature_url": "false", 2024-09-21T21:40:33.962 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-09-21T21:40:33.962 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_s3_auth_use_keystone": "false", 2024-09-21T21:40:33.962 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_s3_auth_use_ldap": "false", 2024-09-21T21:40:33.962 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_s3_auth_use_rados": "true", 2024-09-21T21:40:33.962 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_s3_auth_use_sts": "false", 2024-09-21T21:40:33.962 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-09-21T21:40:33.963 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_s3_success_create_obj_status": "0", 2024-09-21T21:40:33.963 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-09-21T21:40:33.963 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_scheduler_type": "throttler", 2024-09-21T21:40:33.963 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_script_uri": "", 2024-09-21T21:40:33.963 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_service_provider_name": "", 2024-09-21T21:40:33.963 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-09-21T21:40:33.963 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_sts_client_id": "", 2024-09-21T21:40:33.963 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_sts_client_secret": "", 2024-09-21T21:40:33.963 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_sts_entry": "sts", 2024-09-21T21:40:33.963 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_sts_key": "sts", 2024-09-21T21:40:33.963 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_sts_max_session_duration": "43200", 2024-09-21T21:40:33.963 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_sts_min_session_duration": "900", 2024-09-21T21:40:33.963 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_sts_token_introspection_url": "", 2024-09-21T21:40:33.963 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_swift_account_in_url": "false", 2024-09-21T21:40:33.963 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_swift_auth_entry": "auth", 2024-09-21T21:40:33.964 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_swift_auth_url": "", 2024-09-21T21:40:33.964 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_swift_custom_header": "", 2024-09-21T21:40:33.964 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_swift_enforce_content_length": "false", 2024-09-21T21:40:33.964 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_swift_need_stats": "true", 2024-09-21T21:40:33.964 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_swift_tenant_name": "", 2024-09-21T21:40:33.964 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_swift_token_expiration": "86400", 2024-09-21T21:40:33.964 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_swift_url": "", 2024-09-21T21:40:33.964 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_swift_url_prefix": "swift", 2024-09-21T21:40:33.964 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_swift_versioning_enabled": "false", 2024-09-21T21:40:33.964 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-09-21T21:40:33.964 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-09-21T21:40:33.964 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_sync_lease_period": "120", 2024-09-21T21:40:33.964 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-09-21T21:40:33.964 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_sync_log_trim_interval": "1200", 2024-09-21T21:40:33.965 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-09-21T21:40:33.965 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-09-21T21:40:33.965 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-09-21T21:40:33.965 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_sync_obj_etag_verify": "false", 2024-09-21T21:40:33.965 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_sync_trace_history_size": "4096", 2024-09-21T21:40:33.965 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-09-21T21:40:33.965 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-09-21T21:40:33.965 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_thread_pool_size": "512", 2024-09-21T21:40:33.965 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_topic_persistency_max_retries": "0", 2024-09-21T21:40:33.965 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_topic_persistency_sleep_duration": "0", 2024-09-21T21:40:33.965 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_topic_persistency_time_to_live": "0", 2024-09-21T21:40:33.965 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_topic_require_publish_policy": "false", 2024-09-21T21:40:33.965 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_torrent_comment": "", 2024-09-21T21:40:33.965 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_torrent_createby": "", 2024-09-21T21:40:33.966 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_torrent_encoding": "", 2024-09-21T21:40:33.966 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_torrent_flag": "false", 2024-09-21T21:40:33.966 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_torrent_max_size": "5368709120", 2024-09-21T21:40:33.966 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_torrent_origin": "", 2024-09-21T21:40:33.966 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_torrent_sha_unit": "524288", 2024-09-21T21:40:33.966 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_torrent_tracker": "", 2024-09-21T21:40:33.966 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_trust_forwarded_https": "false", 2024-09-21T21:40:33.966 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-09-21T21:40:33.966 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_usage_log_tick_interval": "30", 2024-09-21T21:40:33.966 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_usage_max_shards": "32", 2024-09-21T21:40:33.966 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_usage_max_user_shards": "1", 2024-09-21T21:40:33.966 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_use_opa_authz": "false", 2024-09-21T21:40:33.966 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_user_counters_cache": "false", 2024-09-21T21:40:33.967 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_user_counters_cache_size": "10000", 2024-09-21T21:40:33.967 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-09-21T21:40:33.967 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_user_default_quota_max_size": "-1", 2024-09-21T21:40:33.967 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_user_max_buckets": "1000", 2024-09-21T21:40:33.967 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_user_policies_max_num": "100", 2024-09-21T21:40:33.967 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-09-21T21:40:33.967 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-09-21T21:40:33.967 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_user_quota_sync_interval": "86400", 2024-09-21T21:40:33.967 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-09-21T21:40:33.967 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_user_unique_email": "true", 2024-09-21T21:40:33.967 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_verify_ssl": "true", 2024-09-21T21:40:33.967 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_website_routing_rules_max_num": "50", 2024-09-21T21:40:33.967 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_zone": "", 2024-09-21T21:40:33.967 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_zone_id": "", 2024-09-21T21:40:33.968 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-09-21T21:40:33.968 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_zonegroup": "", 2024-09-21T21:40:33.968 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_zonegroup_id": "", 2024-09-21T21:40:33.968 INFO:teuthology.orchestra.run.smithi081.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-09-21T21:40:33.968 INFO:teuthology.orchestra.run.smithi081.stdout: "rgwlc_auto_session_clear": "true", 2024-09-21T21:40:33.968 INFO:teuthology.orchestra.run.smithi081.stdout: "rgwlc_skip_bucket_step": "false", 2024-09-21T21:40:33.968 INFO:teuthology.orchestra.run.smithi081.stdout: "rocksdb_block_size": "4096", 2024-09-21T21:40:33.968 INFO:teuthology.orchestra.run.smithi081.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-09-21T21:40:33.968 INFO:teuthology.orchestra.run.smithi081.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-09-21T21:40:33.968 INFO:teuthology.orchestra.run.smithi081.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-09-21T21:40:33.968 INFO:teuthology.orchestra.run.smithi081.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-09-21T21:40:33.968 INFO:teuthology.orchestra.run.smithi081.stdout: "rocksdb_cache_shard_bits": "4", 2024-09-21T21:40:33.968 INFO:teuthology.orchestra.run.smithi081.stdout: "rocksdb_cache_size": "536870912", 2024-09-21T21:40:33.968 INFO:teuthology.orchestra.run.smithi081.stdout: "rocksdb_cache_type": "binned_lru", 2024-09-21T21:40:33.968 INFO:teuthology.orchestra.run.smithi081.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-09-21T21:40:33.969 INFO:teuthology.orchestra.run.smithi081.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-09-21T21:40:33.969 INFO:teuthology.orchestra.run.smithi081.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-09-21T21:40:33.969 INFO:teuthology.orchestra.run.smithi081.stdout: "rocksdb_collect_compaction_stats": "false", 2024-09-21T21:40:33.969 INFO:teuthology.orchestra.run.smithi081.stdout: "rocksdb_collect_extended_stats": "false", 2024-09-21T21:40:33.969 INFO:teuthology.orchestra.run.smithi081.stdout: "rocksdb_collect_memory_stats": "false", 2024-09-21T21:40:33.969 INFO:teuthology.orchestra.run.smithi081.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-09-21T21:40:33.969 INFO:teuthology.orchestra.run.smithi081.stdout: "rocksdb_index_type": "binary_search", 2024-09-21T21:40:33.969 INFO:teuthology.orchestra.run.smithi081.stdout: "rocksdb_log_to_ceph_log": "true", 2024-09-21T21:40:33.969 INFO:teuthology.orchestra.run.smithi081.stdout: "rocksdb_metadata_block_size": "4096", 2024-09-21T21:40:33.969 INFO:teuthology.orchestra.run.smithi081.stdout: "rocksdb_partition_filters": "false", 2024-09-21T21:40:33.969 INFO:teuthology.orchestra.run.smithi081.stdout: "rocksdb_perf": "false", 2024-09-21T21:40:33.969 INFO:teuthology.orchestra.run.smithi081.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-09-21T21:40:33.969 INFO:teuthology.orchestra.run.smithi081.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-09-21T21:40:33.969 INFO:teuthology.orchestra.run.smithi081.stdout: "rotating_keys_renewal_timeout": "10", 2024-09-21T21:40:33.969 INFO:teuthology.orchestra.run.smithi081.stdout: "run_dir": "/var/run/ceph", 2024-09-21T21:40:33.970 INFO:teuthology.orchestra.run.smithi081.stdout: "seastore_block_create": "true", 2024-09-21T21:40:33.970 INFO:teuthology.orchestra.run.smithi081.stdout: "seastore_cache_lru_size": "67108864", 2024-09-21T21:40:33.970 INFO:teuthology.orchestra.run.smithi081.stdout: "seastore_cbjournal_size": "5368709120", 2024-09-21T21:40:33.970 INFO:teuthology.orchestra.run.smithi081.stdout: "seastore_data_delta_based_overwrite": "0", 2024-09-21T21:40:33.970 INFO:teuthology.orchestra.run.smithi081.stdout: "seastore_default_max_object_size": "16777216", 2024-09-21T21:40:33.970 INFO:teuthology.orchestra.run.smithi081.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-09-21T21:40:33.970 INFO:teuthology.orchestra.run.smithi081.stdout: "seastore_device_size": "53687091200", 2024-09-21T21:40:33.970 INFO:teuthology.orchestra.run.smithi081.stdout: "seastore_disable_end_to_end_data_protection": "true", 2024-09-21T21:40:33.970 INFO:teuthology.orchestra.run.smithi081.stdout: "seastore_full_integrity_check": "false", 2024-09-21T21:40:33.970 INFO:teuthology.orchestra.run.smithi081.stdout: "seastore_journal_batch_capacity": "16", 2024-09-21T21:40:33.970 INFO:teuthology.orchestra.run.smithi081.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-09-21T21:40:33.970 INFO:teuthology.orchestra.run.smithi081.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-09-21T21:40:33.970 INFO:teuthology.orchestra.run.smithi081.stdout: "seastore_journal_iodepth_limit": "5", 2024-09-21T21:40:33.970 INFO:teuthology.orchestra.run.smithi081.stdout: "seastore_main_device_type": "SSD", 2024-09-21T21:40:33.971 INFO:teuthology.orchestra.run.smithi081.stdout: "seastore_max_concurrent_transactions": "8", 2024-09-21T21:40:33.971 INFO:teuthology.orchestra.run.smithi081.stdout: "seastore_max_data_allocation_size": "32768", 2024-09-21T21:40:33.971 INFO:teuthology.orchestra.run.smithi081.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-09-21T21:40:33.971 INFO:teuthology.orchestra.run.smithi081.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-09-21T21:40:33.971 INFO:teuthology.orchestra.run.smithi081.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-09-21T21:40:33.971 INFO:teuthology.orchestra.run.smithi081.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-09-21T21:40:33.971 INFO:teuthology.orchestra.run.smithi081.stdout: "seastore_segment_size": "67108864", 2024-09-21T21:40:33.971 INFO:teuthology.orchestra.run.smithi081.stdout: "set_keepcaps": "false", 2024-09-21T21:40:33.971 INFO:teuthology.orchestra.run.smithi081.stdout: "setgroup": "", 2024-09-21T21:40:33.971 INFO:teuthology.orchestra.run.smithi081.stdout: "setuser": "", 2024-09-21T21:40:33.971 INFO:teuthology.orchestra.run.smithi081.stdout: "setuser_match_path": "", 2024-09-21T21:40:33.971 INFO:teuthology.orchestra.run.smithi081.stdout: "target_max_misplaced_ratio": "0.050000", 2024-09-21T21:40:33.971 INFO:teuthology.orchestra.run.smithi081.stdout: "thp": "false", 2024-09-21T21:40:33.971 INFO:teuthology.orchestra.run.smithi081.stdout: "threadpool_default_timeout": "60", 2024-09-21T21:40:33.972 INFO:teuthology.orchestra.run.smithi081.stdout: "threadpool_empty_queue_max_wait": "2", 2024-09-21T21:40:33.972 INFO:teuthology.orchestra.run.smithi081.stdout: "throttler_perf_counter": "true" 2024-09-21T21:40:33.972 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:40:33.974 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-09-21T21:40:34.084 INFO:teuthology.orchestra.run.smithi081.stdout:mon.a: {} 2024-09-21T21:40:34.084 INFO:teuthology.orchestra.run.smithi081.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-21T21:40:34.093 INFO:teuthology.orchestra.run.smithi081.stdout:mon.b: {} 2024-09-21T21:40:34.093 INFO:teuthology.orchestra.run.smithi081.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-21T21:40:34.102 INFO:teuthology.orchestra.run.smithi081.stdout:mon.c: {} 2024-09-21T21:40:34.102 INFO:teuthology.orchestra.run.smithi081.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-21T21:40:34.112 DEBUG:teuthology.orchestra.run.smithi081:> type ceph-objectstore-tool 2024-09-21T21:40:34.126 INFO:teuthology.orchestra.run.smithi081.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-21T21:40:34.127 DEBUG:teuthology.orchestra.run.smithi122:> type ceph-objectstore-tool 2024-09-21T21:40:34.144 INFO:teuthology.orchestra.run.smithi122.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-21T21:40:34.145 INFO:teuthology.run_tasks:Running task rbd_fsx... 2024-09-21T21:40:35.920 INFO:tasks.rbd_fsx:starting rbd_fsx... 2024-09-21T21:40:35.920 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-09-21T21:40:35.921 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-21T21:40:35.921 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T21:40:35.921 INFO:tasks.thrashosds.thrasher:inject_pause on osd.3 2024-09-21T21:40:35.921 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-21T21:40:35.921 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-21T21:40:35.921 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok config set filestore_inject_stall 3 2024-09-21T21:40:35.922 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-09-21T21:40:35.922 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-09-21T21:40:35.922 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T21:40:35.923 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-09-21T21:40:35.923 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T21:40:35.923 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-09-21T21:40:35.923 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T21:40:35.924 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 0 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 512 -w 512 -t 512 -h 512 -l 250000000 -S 0 -N 6000 -M pool_client.0 image_client.0 2024-09-21T21:40:35.926 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:40:35.921+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:40:36.024 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:40:36.023+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:40:36.048 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:40:36.049 INFO:teuthology.orchestra.run.smithi122.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-21T21:40:36.049 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:40:36.049 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:40:36.049 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:40:36.049 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 0 2024-09-21T21:40:36.050 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:40:36.059 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T21:40:36.061 INFO:teuthology.orchestra.run.smithi164.stdout:mapped writes DISABLED 2024-09-21T21:40:36.061 INFO:teuthology.orchestra.run.smithi164.stdout:mapped reads DISABLED 2024-09-21T21:40:36.061 INFO:teuthology.orchestra.run.smithi164.stdout:Seed set to 2814 2024-09-21T21:40:36.061 INFO:teuthology.orchestra.run.smithi164.stdout:rbd-nbd mode enabled 2024-09-21T21:40:36.065 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0: {} 2024-09-21T21:40:36.065 INFO:teuthology.orchestra.run.smithi081.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T21:40:36.085 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1: {} 2024-09-21T21:40:36.086 INFO:teuthology.orchestra.run.smithi081.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T21:40:36.117 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2: {} 2024-09-21T21:40:36.117 INFO:teuthology.orchestra.run.smithi081.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T21:40:36.125 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:40:36.124+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:40:36.134 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3: {} 2024-09-21T21:40:36.135 INFO:teuthology.orchestra.run.smithi081.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T21:40:36.152 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4: {} 2024-09-21T21:40:36.152 INFO:teuthology.orchestra.run.smithi081.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T21:40:36.169 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5: {} 2024-09-21T21:40:36.169 INFO:teuthology.orchestra.run.smithi081.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T21:40:36.176 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:36.176 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:36.185 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T21:40:36.226 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:40:36.225+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:40:36.326 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:40:36.325+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:40:36.329 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:36.330 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:36.338 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T21:40:36.408 INFO:teuthology.orchestra.run.smithi081.stderr:noscrub is set 2024-09-21T21:40:36.467 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:40:36.465+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:40:36.488 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:36.488 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:36.497 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T21:40:36.528 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:40:36.527+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:40:36.629 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:40:36.628+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:40:36.640 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:36.640 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:36.649 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T21:40:36.731 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:40:36.729+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:40:36.752 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:36.752 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:36.760 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T21:40:36.844 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:40:36.843+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:40:36.904 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:36.904 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:36.913 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T21:40:36.932 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:40:36.931+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:40:37.033 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:40:37.032+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:40:37.055 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:37.055 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:37.064 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T21:40:37.134 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:40:37.133+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:40:37.207 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:37.207 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:37.216 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T21:40:37.259 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:40:37.258+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:40:37.321 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:37.321 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:37.329 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T21:40:37.370 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:40:37.369+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:40:37.437 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:40:37.436+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:40:37.480 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:37.480 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:37.490 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T21:40:37.538 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:40:37.537+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:40:37.632 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:37.632 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:37.639 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:40:37.638+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:40:37.640 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T21:40:37.739 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:40:37.738+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:40:37.742 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:37.742 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:37.751 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T21:40:37.841 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:40:37.839+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:40:37.892 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:37.892 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:37.901 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T21:40:37.942 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:40:37.941+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:40:38.004 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:38.005 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:38.013 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T21:40:38.042 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:40:38.041+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:40:38.115 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:38.116 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:38.124 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T21:40:38.168 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:40:38.167+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:40:38.181 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T21:40:38.226 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:38.226 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:38.234 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T21:40:38.276 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:40:38.274+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:40:38.301 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0: {} 2024-09-21T21:40:38.301 INFO:teuthology.orchestra.run.smithi081.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T21:40:38.318 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1: {} 2024-09-21T21:40:38.318 INFO:teuthology.orchestra.run.smithi081.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T21:40:38.334 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2: {} 2024-09-21T21:40:38.334 INFO:teuthology.orchestra.run.smithi081.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T21:40:38.345 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:40:38.344+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:40:38.350 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3: {} 2024-09-21T21:40:38.350 INFO:teuthology.orchestra.run.smithi081.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T21:40:38.367 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4: {} 2024-09-21T21:40:38.367 INFO:teuthology.orchestra.run.smithi081.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T21:40:38.378 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:38.378 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:38.383 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5: {} 2024-09-21T21:40:38.384 INFO:teuthology.orchestra.run.smithi081.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T21:40:38.387 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T21:40:38.425 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T21:40:38.479 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:40:38.477+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:40:38.493 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:40:38.493 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:40:38.493 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 0 2024-09-21T21:40:38.493 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:40:38.503 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T21:40:38.547 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:40:38.545+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:40:38.647 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:40:38.645+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:40:38.648 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:40:38.649 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:40:38.649 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:40:38.649 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:40:38.649 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:40:38.657 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T21:40:38.748 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:40:38.746+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:40:38.803 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:40:38.803 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 20, 2024-09-21T21:40:38.803 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 600, 2024-09-21T21:40:38.803 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:40:38.804 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.804 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_repop(client.4183.0:54 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'32, mlcod=14'29)", 2024-09-21T21:40:38.804 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:19.405585+0000", 2024-09-21T21:40:38.804 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 19.395611511999999, 2024-09-21T21:40:38.804 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0014183850000000001, 2024-09-21T21:40:38.804 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:38.804 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:38.804 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:38.804 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:38.804 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.804 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:38.804 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.405585+0000", 2024-09-21T21:40:38.804 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:38.804 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.804 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.805 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:38.805 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.405585+0000", 2024-09-21T21:40:38.805 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:38.805 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.805 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.805 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:38.805 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.405585+0000", 2024-09-21T21:40:38.805 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0800000000000004e-07 2024-09-21T21:40:38.805 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.805 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.805 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:38.805 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.405595+0000", 2024-09-21T21:40:38.805 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2560000000000008e-06 2024-09-21T21:40:38.805 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.805 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.806 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:38.806 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.405606+0000", 2024-09-21T21:40:38.806 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1392999999999999e-05 2024-09-21T21:40:38.806 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.806 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.806 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:38.806 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.405612+0000", 2024-09-21T21:40:38.806 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2269999999999998e-06 2024-09-21T21:40:38.806 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.806 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.806 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:38.806 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.405671+0000", 2024-09-21T21:40:38.806 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8684000000000003e-05 2024-09-21T21:40:38.806 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.807 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.807 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:38.807 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.405690+0000", 2024-09-21T21:40:38.807 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9225e-05 2024-09-21T21:40:38.807 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.807 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.807 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:38.807 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.406978+0000", 2024-09-21T21:40:38.807 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00128773 2024-09-21T21:40:38.807 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.807 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.807 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:38.807 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.407003+0000", 2024-09-21T21:40:38.807 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.5162000000000001e-05 2024-09-21T21:40:38.807 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:38.808 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:38.808 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:38.808 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.808 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.808 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_repop(client.4183.0:55 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'33, mlcod=14'29)", 2024-09-21T21:40:38.808 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:19.406140+0000", 2024-09-21T21:40:38.808 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 19.395056294, 2024-09-21T21:40:38.808 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0011063900000000001, 2024-09-21T21:40:38.808 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:38.808 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:38.808 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:38.808 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:38.808 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.808 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:38.808 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.406140+0000", 2024-09-21T21:40:38.809 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:38.809 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.809 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.809 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:38.809 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.406140+0000", 2024-09-21T21:40:38.809 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:38.809 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.809 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.809 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:38.809 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.406140+0000", 2024-09-21T21:40:38.809 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9999999999999997e-07 2024-09-21T21:40:38.809 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.809 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.809 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:38.810 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.406206+0000", 2024-09-21T21:40:38.810 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5555999999999994e-05 2024-09-21T21:40:38.810 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.810 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.810 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:38.810 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.406217+0000", 2024-09-21T21:40:38.810 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0954e-05 2024-09-21T21:40:38.810 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.810 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.810 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:38.810 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.406222+0000", 2024-09-21T21:40:38.810 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1660000000000002e-06 2024-09-21T21:40:38.810 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.810 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.811 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:38.811 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.406254+0000", 2024-09-21T21:40:38.811 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.1712999999999999e-05 2024-09-21T21:40:38.811 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.811 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.811 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:38.811 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.406265+0000", 2024-09-21T21:40:38.811 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1144999999999999e-05 2024-09-21T21:40:38.811 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.811 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.811 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:38.811 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.407225+0000", 2024-09-21T21:40:38.811 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00096029899999999996 2024-09-21T21:40:38.811 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.812 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.812 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:38.812 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.407246+0000", 2024-09-21T21:40:38.812 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0857e-05 2024-09-21T21:40:38.812 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:38.812 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:38.812 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:38.812 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.812 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.812 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_repop(client.4183.0:62 1.0 e14/13 1:30306672:devicehealth::main.db.0000000000000000:head v 14'40, mlcod=14'38)", 2024-09-21T21:40:38.812 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:19.414660+0000", 2024-09-21T21:40:38.812 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 19.386536534000001, 2024-09-21T21:40:38.812 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0012956370000000001, 2024-09-21T21:40:38.812 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:38.812 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:38.812 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:38.813 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:38.813 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.813 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:38.813 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.414660+0000", 2024-09-21T21:40:38.813 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:38.813 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.813 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.813 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:38.813 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.414660+0000", 2024-09-21T21:40:38.813 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:38.813 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.813 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.813 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:38.813 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.414660+0000", 2024-09-21T21:40:38.813 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.8999999999999995e-07 2024-09-21T21:40:38.813 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.814 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.814 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:38.814 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.414666+0000", 2024-09-21T21:40:38.814 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4620000000000004e-06 2024-09-21T21:40:38.814 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.814 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.814 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:38.814 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.414677+0000", 2024-09-21T21:40:38.814 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1541e-05 2024-09-21T21:40:38.814 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.814 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.814 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:38.814 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.414682+0000", 2024-09-21T21:40:38.814 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1410000000000004e-06 2024-09-21T21:40:38.814 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.815 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.815 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:38.815 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.414724+0000", 2024-09-21T21:40:38.815 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1357999999999999e-05 2024-09-21T21:40:38.815 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.815 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.815 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:38.815 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.414742+0000", 2024-09-21T21:40:38.815 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.8647e-05 2024-09-21T21:40:38.815 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.815 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.815 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:38.815 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.415916+0000", 2024-09-21T21:40:38.815 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.001173597 2024-09-21T21:40:38.815 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.816 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.816 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:38.816 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.415955+0000", 2024-09-21T21:40:38.816 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.9100999999999997e-05 2024-09-21T21:40:38.816 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:38.816 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:38.816 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:38.816 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.816 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.816 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_repop(client.4183.0:67 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'41, mlcod=14'39)", 2024-09-21T21:40:38.816 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:19.417976+0000", 2024-09-21T21:40:38.816 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 19.383219778000001, 2024-09-21T21:40:38.816 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0014537339999999999, 2024-09-21T21:40:38.816 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:38.816 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:38.817 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:38.817 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:38.817 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.817 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:38.817 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.417976+0000", 2024-09-21T21:40:38.817 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:38.817 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.817 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.817 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:38.817 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.417976+0000", 2024-09-21T21:40:38.817 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:38.817 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.817 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.817 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:38.818 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.417977+0000", 2024-09-21T21:40:38.818 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.4499999999999996e-07 2024-09-21T21:40:38.818 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.818 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.818 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:38.818 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.417983+0000", 2024-09-21T21:40:38.818 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2140000000000001e-06 2024-09-21T21:40:38.818 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.818 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.818 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:38.818 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.417994+0000", 2024-09-21T21:40:38.818 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1073999999999999e-05 2024-09-21T21:40:38.818 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.818 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.819 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:38.819 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.417999+0000", 2024-09-21T21:40:38.819 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0520000000000001e-06 2024-09-21T21:40:38.819 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.819 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.819 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:38.819 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.418040+0000", 2024-09-21T21:40:38.819 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.0803000000000002e-05 2024-09-21T21:40:38.819 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.819 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.819 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:38.819 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.418057+0000", 2024-09-21T21:40:38.819 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.6779999999999999e-05 2024-09-21T21:40:38.819 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.819 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.820 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:38.820 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.419405+0000", 2024-09-21T21:40:38.820 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.001347921 2024-09-21T21:40:38.820 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.820 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.820 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:38.820 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.419430+0000", 2024-09-21T21:40:38.820 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.5145000000000001e-05 2024-09-21T21:40:38.820 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:38.820 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:38.820 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:38.820 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.820 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.820 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_repop(client.4183.0:68 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'42, mlcod=14'39)", 2024-09-21T21:40:38.821 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:19.418478+0000", 2024-09-21T21:40:38.821 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 19.382718336, 2024-09-21T21:40:38.821 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.001196852, 2024-09-21T21:40:38.821 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:38.821 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:38.821 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:38.821 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:38.821 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.821 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:38.821 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.418478+0000", 2024-09-21T21:40:38.821 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:38.821 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.821 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.821 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:38.821 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.418478+0000", 2024-09-21T21:40:38.822 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:38.822 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.822 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.822 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:38.822 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.418478+0000", 2024-09-21T21:40:38.822 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.6899999999999996e-07 2024-09-21T21:40:38.822 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.822 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.822 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:38.822 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.418485+0000", 2024-09-21T21:40:38.822 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.195e-06 2024-09-21T21:40:38.822 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.822 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.822 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:38.823 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.418496+0000", 2024-09-21T21:40:38.823 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1559000000000001e-05 2024-09-21T21:40:38.823 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.823 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.823 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:38.823 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.418501+0000", 2024-09-21T21:40:38.823 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2109999999999997e-06 2024-09-21T21:40:38.823 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.823 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.823 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:38.823 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.418545+0000", 2024-09-21T21:40:38.823 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3409999999999999e-05 2024-09-21T21:40:38.823 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.823 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.823 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:38.823 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.418562+0000", 2024-09-21T21:40:38.824 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.6917000000000001e-05 2024-09-21T21:40:38.824 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.824 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.824 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:38.824 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.419655+0000", 2024-09-21T21:40:38.824 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0010935910000000001 2024-09-21T21:40:38.824 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.824 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.824 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:38.824 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.419675+0000", 2024-09-21T21:40:38.824 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9199999999999999e-05 2024-09-21T21:40:38.824 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:38.824 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:38.824 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:38.824 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.825 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.825 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_repop(client.4183.0:83 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'53, mlcod=14'50)", 2024-09-21T21:40:38.825 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:19.430818+0000", 2024-09-21T21:40:38.825 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 19.370378527, 2024-09-21T21:40:38.825 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00114628, 2024-09-21T21:40:38.825 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:38.825 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:38.825 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:38.825 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:38.825 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.825 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:38.825 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.430818+0000", 2024-09-21T21:40:38.825 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:38.825 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.825 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.826 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:38.826 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.430818+0000", 2024-09-21T21:40:38.826 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:38.826 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.826 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.826 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:38.826 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.430818+0000", 2024-09-21T21:40:38.826 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.61e-07 2024-09-21T21:40:38.826 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.826 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.826 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:38.826 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.430825+0000", 2024-09-21T21:40:38.826 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.613e-06 2024-09-21T21:40:38.826 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.827 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.827 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:38.827 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.430836+0000", 2024-09-21T21:40:38.827 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0869e-05 2024-09-21T21:40:38.827 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.827 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.827 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:38.827 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.430841+0000", 2024-09-21T21:40:38.827 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9710000000000003e-06 2024-09-21T21:40:38.827 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.827 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.827 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:38.827 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.430882+0000", 2024-09-21T21:40:38.827 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1539000000000003e-05 2024-09-21T21:40:38.827 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.827 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.827 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:38.828 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.430895+0000", 2024-09-21T21:40:38.828 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.2880999999999999e-05 2024-09-21T21:40:38.828 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.828 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.828 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:38.828 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.431929+0000", 2024-09-21T21:40:38.828 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.001033909 2024-09-21T21:40:38.828 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.828 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.828 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:38.828 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.431964+0000", 2024-09-21T21:40:38.828 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4737000000000003e-05 2024-09-21T21:40:38.828 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:38.828 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:38.829 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:38.829 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.829 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.829 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_repop(client.4183.0:84 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'54, mlcod=14'50)", 2024-09-21T21:40:38.829 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:19.431181+0000", 2024-09-21T21:40:38.829 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 19.370014594000001, 2024-09-21T21:40:38.829 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0013644270000000001, 2024-09-21T21:40:38.829 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:38.829 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:38.829 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:38.829 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:38.829 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.829 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:38.829 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.431181+0000", 2024-09-21T21:40:38.829 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:38.829 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.829 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.830 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:38.830 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.431181+0000", 2024-09-21T21:40:38.830 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:38.830 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.830 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.830 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:38.830 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.431182+0000", 2024-09-21T21:40:38.830 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.13e-07 2024-09-21T21:40:38.830 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.830 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.830 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:38.830 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.431251+0000", 2024-09-21T21:40:38.830 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.8665999999999996e-05 2024-09-21T21:40:38.830 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.831 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.831 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:38.831 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.431257+0000", 2024-09-21T21:40:38.831 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.844e-06 2024-09-21T21:40:38.831 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.831 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.831 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:38.831 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.431262+0000", 2024-09-21T21:40:38.831 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.16e-06 2024-09-21T21:40:38.831 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.831 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.831 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:38.831 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.431302+0000", 2024-09-21T21:40:38.831 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.0268999999999997e-05 2024-09-21T21:40:38.831 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.831 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.832 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:38.832 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.431318+0000", 2024-09-21T21:40:38.832 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.6439999999999998e-05 2024-09-21T21:40:38.832 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.832 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.832 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:38.832 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.432505+0000", 2024-09-21T21:40:38.832 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0011869809999999999 2024-09-21T21:40:38.832 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.832 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.832 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:38.832 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.432546+0000", 2024-09-21T21:40:38.832 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.0553999999999998e-05 2024-09-21T21:40:38.832 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:38.832 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:38.833 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:38.833 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.833 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.833 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_repop(client.4183.0:85 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'55, mlcod=14'50)", 2024-09-21T21:40:38.833 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:19.431665+0000", 2024-09-21T21:40:38.833 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 19.369531462000001, 2024-09-21T21:40:38.833 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.001491633, 2024-09-21T21:40:38.833 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:38.833 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:38.833 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:38.833 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:38.833 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.833 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:38.833 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.431665+0000", 2024-09-21T21:40:38.833 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:38.833 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.834 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.834 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:38.834 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.431665+0000", 2024-09-21T21:40:38.834 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:38.834 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.834 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.834 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:38.834 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.431665+0000", 2024-09-21T21:40:38.834 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5000000000000002e-07 2024-09-21T21:40:38.834 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.834 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.834 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:38.834 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.431671+0000", 2024-09-21T21:40:38.834 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3850000000000001e-06 2024-09-21T21:40:38.834 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.834 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.835 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:38.835 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.431682+0000", 2024-09-21T21:40:38.835 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1024e-05 2024-09-21T21:40:38.835 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.835 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.835 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:38.835 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.431687+0000", 2024-09-21T21:40:38.835 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0540000000000002e-06 2024-09-21T21:40:38.835 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.835 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.835 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:38.835 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.431729+0000", 2024-09-21T21:40:38.835 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2085000000000001e-05 2024-09-21T21:40:38.835 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.835 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.835 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:38.836 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.431742+0000", 2024-09-21T21:40:38.836 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.3071000000000001e-05 2024-09-21T21:40:38.836 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.836 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.836 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:38.836 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.433116+0000", 2024-09-21T21:40:38.836 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.001374416 2024-09-21T21:40:38.836 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.836 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.836 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:38.836 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.433156+0000", 2024-09-21T21:40:38.836 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.9848000000000002e-05 2024-09-21T21:40:38.836 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:38.836 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:38.837 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:38.837 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.837 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.837 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_repop(client.4183.0:87 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'57, mlcod=14'51)", 2024-09-21T21:40:38.837 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:19.432500+0000", 2024-09-21T21:40:38.837 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 19.368696342, 2024-09-21T21:40:38.837 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0013807369999999999, 2024-09-21T21:40:38.837 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:38.837 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:38.837 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:38.837 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:38.837 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.837 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:38.837 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.432500+0000", 2024-09-21T21:40:38.838 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:38.838 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.838 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.838 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:38.838 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.432500+0000", 2024-09-21T21:40:38.838 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:38.838 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.838 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.838 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:38.838 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.432500+0000", 2024-09-21T21:40:38.838 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5899999999999996e-07 2024-09-21T21:40:38.838 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.838 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.838 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:38.838 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.432572+0000", 2024-09-21T21:40:38.839 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.1587000000000004e-05 2024-09-21T21:40:38.839 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.839 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.839 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:38.839 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.432580+0000", 2024-09-21T21:40:38.839 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.9410000000000007e-06 2024-09-21T21:40:38.839 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.839 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.839 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:38.839 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.432585+0000", 2024-09-21T21:40:38.839 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9420000000000003e-06 2024-09-21T21:40:38.839 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.839 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.839 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:38.839 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.432617+0000", 2024-09-21T21:40:38.840 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.1934999999999998e-05 2024-09-21T21:40:38.840 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.840 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.840 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:38.840 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.432629+0000", 2024-09-21T21:40:38.840 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.207e-05 2024-09-21T21:40:38.840 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.840 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.840 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:38.840 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.433848+0000", 2024-09-21T21:40:38.840 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0012191000000000001 2024-09-21T21:40:38.840 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.840 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.840 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:38.840 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.433880+0000", 2024-09-21T21:40:38.841 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.2602999999999999e-05 2024-09-21T21:40:38.841 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:38.841 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:38.841 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:38.841 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.841 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.841 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_repop(client.4183.0:99 1.0 e14/13 1:30306672:devicehealth::main.db.0000000000000000:head v 14'65, mlcod=14'63)", 2024-09-21T21:40:38.841 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:19.640075+0000", 2024-09-21T21:40:38.841 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 19.161121518000002, 2024-09-21T21:40:38.841 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0010927370000000001, 2024-09-21T21:40:38.841 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:38.841 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:38.841 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:38.841 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:38.841 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.842 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:38.842 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.640075+0000", 2024-09-21T21:40:38.842 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:38.842 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.842 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.842 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:38.842 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.640075+0000", 2024-09-21T21:40:38.842 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:38.842 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.842 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.842 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:38.842 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.640086+0000", 2024-09-21T21:40:38.842 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1875999999999999e-05 2024-09-21T21:40:38.842 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.843 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.843 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:38.843 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.640115+0000", 2024-09-21T21:40:38.843 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.8612999999999999e-05 2024-09-21T21:40:38.843 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.843 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.843 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:38.843 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.640145+0000", 2024-09-21T21:40:38.843 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0332999999999999e-05 2024-09-21T21:40:38.843 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.843 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.843 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:38.843 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.640183+0000", 2024-09-21T21:40:38.843 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7868999999999999e-05 2024-09-21T21:40:38.843 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.844 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.844 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:38.844 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.640380+0000", 2024-09-21T21:40:38.844 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00019657600000000001 2024-09-21T21:40:38.844 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.844 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.844 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:38.844 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.640404+0000", 2024-09-21T21:40:38.844 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3985000000000002e-05 2024-09-21T21:40:38.844 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.844 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.844 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:38.844 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.641129+0000", 2024-09-21T21:40:38.844 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00072518799999999998 2024-09-21T21:40:38.844 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.845 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.845 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:38.845 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.641167+0000", 2024-09-21T21:40:38.845 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8297000000000003e-05 2024-09-21T21:40:38.845 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:38.845 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:38.845 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:38.845 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.845 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.845 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_repop(client.4183.0:119 1.0 e14/13 1:30306672:devicehealth::main.db.0000000000000000:head v 14'81, mlcod=14'78)", 2024-09-21T21:40:38.845 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:19.652276+0000", 2024-09-21T21:40:38.845 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 19.148920315000002, 2024-09-21T21:40:38.845 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0028475499999999999, 2024-09-21T21:40:38.845 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:38.846 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:38.846 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:38.846 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:38.846 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.846 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:38.846 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.652276+0000", 2024-09-21T21:40:38.846 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:38.846 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.846 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.846 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:38.846 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.652276+0000", 2024-09-21T21:40:38.846 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:38.846 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.846 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.847 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:38.847 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.652276+0000", 2024-09-21T21:40:38.847 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6000000000000003e-07 2024-09-21T21:40:38.847 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.847 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.847 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:38.847 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.652326+0000", 2024-09-21T21:40:38.847 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9370999999999999e-05 2024-09-21T21:40:38.847 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.847 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.847 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:38.847 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.652336+0000", 2024-09-21T21:40:38.847 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.8910000000000007e-06 2024-09-21T21:40:38.847 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.847 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.848 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:38.848 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.652340+0000", 2024-09-21T21:40:38.848 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5260000000000004e-06 2024-09-21T21:40:38.848 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.848 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.848 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:38.848 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.652381+0000", 2024-09-21T21:40:38.848 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.0388e-05 2024-09-21T21:40:38.848 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.848 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.848 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:38.848 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.652393+0000", 2024-09-21T21:40:38.848 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.2500999999999999e-05 2024-09-21T21:40:38.848 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.848 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.849 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:38.849 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.655088+0000", 2024-09-21T21:40:38.849 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0026953860000000001 2024-09-21T21:40:38.849 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.849 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.849 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:38.849 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.655123+0000", 2024-09-21T21:40:38.849 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4826999999999997e-05 2024-09-21T21:40:38.849 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:38.849 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:38.849 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:38.849 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.849 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.849 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_repop(client.4183.0:120 1.0 e14/13 1:30306672:devicehealth::main.db.0000000000000000:head v 14'82, mlcod=14'78)", 2024-09-21T21:40:38.849 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:19.652809+0000", 2024-09-21T21:40:38.849 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 19.148386773999999, 2024-09-21T21:40:38.850 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.002553044, 2024-09-21T21:40:38.850 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:38.850 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:38.850 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:38.850 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:38.850 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.850 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:38.850 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.652809+0000", 2024-09-21T21:40:38.850 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:38.850 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.850 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.850 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:38.850 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.652809+0000", 2024-09-21T21:40:38.850 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:38.850 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.851 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.851 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:38.851 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.652810+0000", 2024-09-21T21:40:38.851 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8000000000000001e-07 2024-09-21T21:40:38.851 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.851 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.851 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:38.851 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.652874+0000", 2024-09-21T21:40:38.851 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4561999999999996e-05 2024-09-21T21:40:38.851 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.851 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.851 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:38.851 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.652881+0000", 2024-09-21T21:40:38.851 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.7839999999999998e-06 2024-09-21T21:40:38.852 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.852 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.852 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:38.852 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.652885+0000", 2024-09-21T21:40:38.852 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7440000000000001e-06 2024-09-21T21:40:38.852 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.852 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.852 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:38.852 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.652919+0000", 2024-09-21T21:40:38.852 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4499999999999998e-05 2024-09-21T21:40:38.852 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.852 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.852 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:38.852 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.652940+0000", 2024-09-21T21:40:38.853 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.067e-05 2024-09-21T21:40:38.853 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.853 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.853 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:38.853 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.655335+0000", 2024-09-21T21:40:38.853 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0023946290000000001 2024-09-21T21:40:38.853 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.853 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.853 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:38.853 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.655362+0000", 2024-09-21T21:40:38.853 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.7775000000000001e-05 2024-09-21T21:40:38.853 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:38.853 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:38.853 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:38.853 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.853 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.854 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_repop(client.4183.0:121 1.0 e14/13 1:30306672:devicehealth::main.db.0000000000000000:head v 14'83, mlcod=14'78)", 2024-09-21T21:40:38.854 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:19.653330+0000", 2024-09-21T21:40:38.854 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 19.147866141000002, 2024-09-21T21:40:38.854 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0020575599999999999, 2024-09-21T21:40:38.854 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:38.854 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:38.854 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:38.854 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:38.854 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.854 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:38.854 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.653330+0000", 2024-09-21T21:40:38.854 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:38.854 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.854 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.854 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:38.855 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.653330+0000", 2024-09-21T21:40:38.855 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:38.855 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.855 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.855 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:38.855 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.653331+0000", 2024-09-21T21:40:38.855 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1500000000000004e-07 2024-09-21T21:40:38.855 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.855 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.855 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:38.855 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.653379+0000", 2024-09-21T21:40:38.855 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8491e-05 2024-09-21T21:40:38.855 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.855 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.855 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:38.855 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.653389+0000", 2024-09-21T21:40:38.856 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.8779999999999993e-06 2024-09-21T21:40:38.856 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.856 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.856 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:38.856 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.653395+0000", 2024-09-21T21:40:38.856 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5050000000000004e-06 2024-09-21T21:40:38.856 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.856 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.856 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:38.856 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.653430+0000", 2024-09-21T21:40:38.856 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4171000000000002e-05 2024-09-21T21:40:38.856 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.856 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.856 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:38.856 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.653441+0000", 2024-09-21T21:40:38.857 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1636e-05 2024-09-21T21:40:38.857 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.857 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.857 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:38.857 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.655364+0000", 2024-09-21T21:40:38.857 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0019223549999999999 2024-09-21T21:40:38.857 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.857 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.857 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:38.857 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.655387+0000", 2024-09-21T21:40:38.857 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3909000000000001e-05 2024-09-21T21:40:38.857 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:38.857 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:38.857 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:38.858 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.858 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.858 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_repop(client.4183.0:123 1.0 e14/13 1:30306672:devicehealth::main.db.0000000000000000:head v 14'85, mlcod=14'83)", 2024-09-21T21:40:38.858 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:19.657211+0000", 2024-09-21T21:40:38.858 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 19.143985287, 2024-09-21T21:40:38.858 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0018177239999999999, 2024-09-21T21:40:38.858 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:38.858 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:38.858 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:38.858 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:38.858 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.858 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:38.858 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.657211+0000", 2024-09-21T21:40:38.859 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:38.859 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.859 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.859 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:38.859 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.657211+0000", 2024-09-21T21:40:38.859 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:38.859 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.859 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.859 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:38.859 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.657211+0000", 2024-09-21T21:40:38.859 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8699999999999995e-07 2024-09-21T21:40:38.859 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.859 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.859 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:38.859 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.657215+0000", 2024-09-21T21:40:38.860 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8689999999999997e-06 2024-09-21T21:40:38.860 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.860 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.860 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:38.860 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.657223+0000", 2024-09-21T21:40:38.860 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.8439999999999998e-06 2024-09-21T21:40:38.860 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.860 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.860 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:38.860 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.657227+0000", 2024-09-21T21:40:38.860 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.9330000000000003e-06 2024-09-21T21:40:38.860 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.860 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.860 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:38.860 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.657258+0000", 2024-09-21T21:40:38.861 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.1124999999999998e-05 2024-09-21T21:40:38.861 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.861 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.861 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:38.861 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.657270+0000", 2024-09-21T21:40:38.861 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1678999999999999e-05 2024-09-21T21:40:38.861 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.861 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.861 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:38.861 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.658989+0000", 2024-09-21T21:40:38.861 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0017192970000000001 2024-09-21T21:40:38.861 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.861 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.861 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:38.862 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.659028+0000", 2024-09-21T21:40:38.862 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.9490000000000003e-05 2024-09-21T21:40:38.862 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:38.862 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:38.862 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:38.862 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.862 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.862 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_repop(client.4183.0:137 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'95, mlcod=14'90)", 2024-09-21T21:40:38.862 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:19.920653+0000", 2024-09-21T21:40:38.862 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 18.880542809000001, 2024-09-21T21:40:38.862 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0031174010000000001, 2024-09-21T21:40:38.862 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:38.862 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:38.862 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:38.863 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:38.863 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.863 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:38.863 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.920653+0000", 2024-09-21T21:40:38.863 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:38.863 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.863 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.863 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:38.863 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.920653+0000", 2024-09-21T21:40:38.863 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:38.863 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.863 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.863 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:38.863 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.920654+0000", 2024-09-21T21:40:38.863 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5999999999999999e-07 2024-09-21T21:40:38.864 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.864 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.864 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:38.864 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.920713+0000", 2024-09-21T21:40:38.864 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9008999999999997e-05 2024-09-21T21:40:38.864 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.864 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.864 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:38.864 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.920718+0000", 2024-09-21T21:40:38.864 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6779999999999997e-06 2024-09-21T21:40:38.864 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.864 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.864 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:38.864 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.920722+0000", 2024-09-21T21:40:38.864 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4410000000000002e-06 2024-09-21T21:40:38.865 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.865 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.865 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:38.865 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.920756+0000", 2024-09-21T21:40:38.865 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3986999999999998e-05 2024-09-21T21:40:38.865 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.865 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.865 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:38.865 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.920768+0000", 2024-09-21T21:40:38.865 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1960999999999999e-05 2024-09-21T21:40:38.865 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.865 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.865 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:38.865 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.923744+0000", 2024-09-21T21:40:38.865 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0029758839999999998 2024-09-21T21:40:38.866 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.866 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.866 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:38.866 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.923771+0000", 2024-09-21T21:40:38.866 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.6981000000000001e-05 2024-09-21T21:40:38.866 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:38.866 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:38.866 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:38.866 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.866 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.866 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_repop(client.4183.0:138 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'96, mlcod=14'90)", 2024-09-21T21:40:38.866 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:19.921304+0000", 2024-09-21T21:40:38.866 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 18.879891625999999, 2024-09-21T21:40:38.866 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0024851589999999998, 2024-09-21T21:40:38.867 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:38.867 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:38.867 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:38.867 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:38.867 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.867 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:38.867 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.921304+0000", 2024-09-21T21:40:38.867 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:38.867 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.867 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.867 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:38.867 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.921304+0000", 2024-09-21T21:40:38.867 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:38.867 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.867 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.868 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:38.868 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.921305+0000", 2024-09-21T21:40:38.868 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.7400000000000003e-07 2024-09-21T21:40:38.868 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.868 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.868 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:38.868 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.921310+0000", 2024-09-21T21:40:38.868 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7520000000000004e-06 2024-09-21T21:40:38.868 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.868 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.868 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:38.868 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.921320+0000", 2024-09-21T21:40:38.868 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.8190000000000003e-06 2024-09-21T21:40:38.868 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.868 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.869 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:38.869 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.921324+0000", 2024-09-21T21:40:38.869 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5979999999999999e-06 2024-09-21T21:40:38.869 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.869 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.869 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:38.869 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.922948+0000", 2024-09-21T21:40:38.869 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0016239309999999999 2024-09-21T21:40:38.869 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.869 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.869 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:38.869 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.922961+0000", 2024-09-21T21:40:38.869 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.3298e-05 2024-09-21T21:40:38.869 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.870 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.870 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:38.870 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.923772+0000", 2024-09-21T21:40:38.870 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00081065599999999999 2024-09-21T21:40:38.870 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.870 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.870 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:38.870 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.923790+0000", 2024-09-21T21:40:38.870 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.7431e-05 2024-09-21T21:40:38.870 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:38.870 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:38.870 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:38.870 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.870 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.870 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_repop(client.4183.0:139 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'97, mlcod=14'90)", 2024-09-21T21:40:38.871 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:19.921649+0000", 2024-09-21T21:40:38.871 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 18.879546716, 2024-09-21T21:40:38.871 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0021538960000000002, 2024-09-21T21:40:38.871 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:38.871 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:38.871 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:38.871 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:38.871 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.871 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:38.871 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.921649+0000", 2024-09-21T21:40:38.871 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:38.871 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.871 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.871 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:38.872 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.921649+0000", 2024-09-21T21:40:38.872 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:38.872 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.872 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.872 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:38.872 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.921650+0000", 2024-09-21T21:40:38.872 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3799999999999997e-07 2024-09-21T21:40:38.872 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.872 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.872 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:38.872 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.921654+0000", 2024-09-21T21:40:38.872 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3680000000000004e-06 2024-09-21T21:40:38.872 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.872 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.873 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:38.873 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.921664+0000", 2024-09-21T21:40:38.873 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.8709999999999992e-06 2024-09-21T21:40:38.873 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.873 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.873 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:38.873 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.921669+0000", 2024-09-21T21:40:38.873 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.656e-06 2024-09-21T21:40:38.873 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.873 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.873 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:38.873 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.923150+0000", 2024-09-21T21:40:38.873 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0014808779999999999 2024-09-21T21:40:38.873 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.874 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.874 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:38.874 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.923161+0000", 2024-09-21T21:40:38.874 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1338999999999999e-05 2024-09-21T21:40:38.874 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.874 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.874 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:38.874 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.923791+0000", 2024-09-21T21:40:38.874 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00062957900000000001 2024-09-21T21:40:38.874 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.874 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.874 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:38.874 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.923803+0000", 2024-09-21T21:40:38.874 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.2567e-05 2024-09-21T21:40:38.874 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:38.875 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:38.875 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:38.875 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.875 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.875 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_repop(client.4183.0:140 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'98, mlcod=14'90)", 2024-09-21T21:40:38.875 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:19.922104+0000", 2024-09-21T21:40:38.875 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 18.879091751000001, 2024-09-21T21:40:38.875 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0020448599999999999, 2024-09-21T21:40:38.875 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:38.875 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:38.875 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:38.875 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:38.875 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.875 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:38.875 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.922104+0000", 2024-09-21T21:40:38.875 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:38.876 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.876 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.876 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:38.876 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.922104+0000", 2024-09-21T21:40:38.876 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:38.876 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.876 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.876 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:38.876 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.922105+0000", 2024-09-21T21:40:38.876 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.96e-07 2024-09-21T21:40:38.876 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.876 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.876 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:38.876 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.922192+0000", 2024-09-21T21:40:38.876 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.6897000000000005e-05 2024-09-21T21:40:38.877 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.877 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.877 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:38.877 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.922198+0000", 2024-09-21T21:40:38.877 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.8469999999999998e-06 2024-09-21T21:40:38.877 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.877 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.877 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:38.877 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.922202+0000", 2024-09-21T21:40:38.877 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.394e-06 2024-09-21T21:40:38.877 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.877 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.877 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:38.877 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.923347+0000", 2024-09-21T21:40:38.878 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0011452280000000001 2024-09-21T21:40:38.878 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.878 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.878 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:38.878 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.923359+0000", 2024-09-21T21:40:38.878 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1956e-05 2024-09-21T21:40:38.878 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.878 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.878 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:38.878 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.924133+0000", 2024-09-21T21:40:38.878 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00077395199999999995 2024-09-21T21:40:38.878 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.878 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.878 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:38.878 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.924149+0000", 2024-09-21T21:40:38.879 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.6189999999999999e-05 2024-09-21T21:40:38.879 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:38.879 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:38.879 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:38.879 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.879 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.879 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_repop(client.4183.0:145 1.0 e14/13 1:30306672:devicehealth::main.db.0000000000000000:head v 14'103, mlcod=14'98)", 2024-09-21T21:40:38.879 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:19.929302+0000", 2024-09-21T21:40:38.879 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 18.871893975999999, 2024-09-21T21:40:38.879 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0031456890000000001, 2024-09-21T21:40:38.879 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:38.879 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:38.879 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:38.879 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:38.880 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.880 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:38.880 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.929302+0000", 2024-09-21T21:40:38.880 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:38.880 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.880 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.880 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:38.880 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.929302+0000", 2024-09-21T21:40:38.880 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:38.880 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.880 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.880 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:38.880 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.929303+0000", 2024-09-21T21:40:38.880 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.06e-07 2024-09-21T21:40:38.880 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.881 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.881 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:38.881 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.929361+0000", 2024-09-21T21:40:38.881 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8436000000000001e-05 2024-09-21T21:40:38.881 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.881 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.881 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:38.881 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.929367+0000", 2024-09-21T21:40:38.881 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4620000000000001e-06 2024-09-21T21:40:38.881 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.881 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.881 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:38.881 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.929371+0000", 2024-09-21T21:40:38.881 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.9339999999999999e-06 2024-09-21T21:40:38.881 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.882 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.882 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:38.882 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.929408+0000", 2024-09-21T21:40:38.882 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6285000000000002e-05 2024-09-21T21:40:38.882 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.882 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.882 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:38.882 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.929418+0000", 2024-09-21T21:40:38.882 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.028e-05 2024-09-21T21:40:38.882 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.882 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.882 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:38.882 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.932419+0000", 2024-09-21T21:40:38.882 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0030006550000000001 2024-09-21T21:40:38.882 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.883 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.883 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:38.883 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:19.932448+0000", 2024-09-21T21:40:38.883 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.9130999999999999e-05 2024-09-21T21:40:38.883 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:38.883 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:38.883 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:38.883 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.883 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.883 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_repop(client.4183.0:161 1.0 e14/13 1:30306672:devicehealth::main.db.0000000000000000:head v 14'115, mlcod=14'112)", 2024-09-21T21:40:38.883 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:20.303190+0000", 2024-09-21T21:40:38.883 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 18.498006181000001, 2024-09-21T21:40:38.883 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0010801949999999999, 2024-09-21T21:40:38.883 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:38.884 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:38.884 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:38.884 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:38.884 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.884 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:38.884 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:20.303190+0000", 2024-09-21T21:40:38.884 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:38.884 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.884 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.884 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:38.884 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:20.303190+0000", 2024-09-21T21:40:38.884 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:38.884 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.884 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.884 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:38.885 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:20.303190+0000", 2024-09-21T21:40:38.885 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.9499999999999998e-07 2024-09-21T21:40:38.885 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.885 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.885 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:38.885 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:20.303261+0000", 2024-09-21T21:40:38.885 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.1093000000000005e-05 2024-09-21T21:40:38.885 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.885 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.885 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:38.885 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:20.303269+0000", 2024-09-21T21:40:38.885 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5700000000000004e-06 2024-09-21T21:40:38.885 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.885 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.885 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:38.886 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:20.303275+0000", 2024-09-21T21:40:38.886 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1700000000000002e-06 2024-09-21T21:40:38.886 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.886 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.886 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:38.886 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:20.303313+0000", 2024-09-21T21:40:38.886 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.79e-05 2024-09-21T21:40:38.886 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.886 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.886 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:38.886 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:20.303331+0000", 2024-09-21T21:40:38.886 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.8274000000000002e-05 2024-09-21T21:40:38.886 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.886 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.887 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:38.887 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:20.304240+0000", 2024-09-21T21:40:38.887 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00090916100000000002 2024-09-21T21:40:38.887 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:38.887 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:38.887 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:38.887 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:20.304270+0000", 2024-09-21T21:40:38.887 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.9632e-05 2024-09-21T21:40:38.887 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:38.887 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:38.887 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:38.887 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:38.887 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:38.887 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:40:38.889 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T21:40:38.889 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:40:38.887+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:40:38.989 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:40:38.988+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:40:39.033 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:40:39.033 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:40:39.033 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 0 2024-09-21T21:40:39.033 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:40:39.041 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T21:40:39.125 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:40:39.124+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:40:39.184 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:40:39.184 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:40:39.184 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:40:39.184 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:40:39.184 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:40:39.190 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:40:39.189+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:40:39.192 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T21:40:39.291 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:40:39.290+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:40:39.335 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:40:39.336 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 20, 2024-09-21T21:40:39.336 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 600, 2024-09-21T21:40:39.336 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:40:39.336 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:39.336 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_repop(client.4214.0:5 2.2 e18/16 2:5cea7035:::rbd_info:head v 18'1, mlcod=0'0)", 2024-09-21T21:40:39.336 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:26.376109+0000", 2024-09-21T21:40:39.336 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 12.958668767000001, 2024-09-21T21:40:39.336 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000779629, 2024-09-21T21:40:39.336 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:39.336 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:39.336 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:39.336 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:39.336 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:39.336 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:39.337 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:26.376109+0000", 2024-09-21T21:40:39.337 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:39.337 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:39.337 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:39.337 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:39.337 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:26.376109+0000", 2024-09-21T21:40:39.337 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:39.337 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:39.337 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:39.337 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:39.337 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:26.376109+0000", 2024-09-21T21:40:39.337 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.6400000000000001e-07 2024-09-21T21:40:39.338 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:39.338 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:39.338 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:39.338 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:26.376118+0000", 2024-09-21T21:40:39.338 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.5739999999999996e-06 2024-09-21T21:40:39.338 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:39.338 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:39.338 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:39.338 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:26.376139+0000", 2024-09-21T21:40:39.338 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1015000000000001e-05 2024-09-21T21:40:39.338 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:39.338 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:39.338 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:39.338 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:26.376148+0000", 2024-09-21T21:40:39.338 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.3649999999999993e-06 2024-09-21T21:40:39.339 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:39.339 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:39.339 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:39.339 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:26.376194+0000", 2024-09-21T21:40:39.339 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5861e-05 2024-09-21T21:40:39.339 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:39.339 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:39.339 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:39.339 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:26.376219+0000", 2024-09-21T21:40:39.339 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.5131e-05 2024-09-21T21:40:39.339 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:39.339 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:39.339 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:39.339 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:26.376846+0000", 2024-09-21T21:40:39.340 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00062638899999999998 2024-09-21T21:40:39.340 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:39.340 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:39.340 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:39.340 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:26.376889+0000", 2024-09-21T21:40:39.340 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3130000000000002e-05 2024-09-21T21:40:39.340 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:39.340 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:39.340 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:39.340 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:39.340 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:39.340 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_repop(client.4214.0:7 2.2 e19/16 2:5cea7035:::rbd_info:head v 19'2, mlcod=0'0)", 2024-09-21T21:40:39.340 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:27.382370+0000", 2024-09-21T21:40:39.340 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 11.952408225999999, 2024-09-21T21:40:39.340 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00077314500000000004, 2024-09-21T21:40:39.341 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:39.341 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:39.341 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:39.341 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:39.341 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:39.341 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:39.341 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:27.382370+0000", 2024-09-21T21:40:39.341 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:39.341 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:39.341 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:39.341 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:39.341 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:27.382370+0000", 2024-09-21T21:40:39.341 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:39.341 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:39.342 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:39.342 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:39.342 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:27.382370+0000", 2024-09-21T21:40:39.342 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.2099999999999998e-07 2024-09-21T21:40:39.342 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:39.342 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:39.342 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:39.342 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:27.382376+0000", 2024-09-21T21:40:39.342 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5500000000000002e-06 2024-09-21T21:40:39.342 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:39.342 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:39.342 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:39.342 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:27.382388+0000", 2024-09-21T21:40:39.342 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.2832e-05 2024-09-21T21:40:39.342 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:39.343 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:39.343 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:39.343 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:27.382399+0000", 2024-09-21T21:40:39.343 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0825000000000001e-05 2024-09-21T21:40:39.343 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:39.343 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:39.343 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:39.343 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:27.382456+0000", 2024-09-21T21:40:39.343 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6842000000000002e-05 2024-09-21T21:40:39.343 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:39.343 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:39.343 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:39.343 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:27.382482+0000", 2024-09-21T21:40:39.343 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.5945999999999999e-05 2024-09-21T21:40:39.344 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:39.344 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:39.344 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:39.344 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:27.383098+0000", 2024-09-21T21:40:39.344 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000615944 2024-09-21T21:40:39.344 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:39.344 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:39.344 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:39.344 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:27.383143+0000", 2024-09-21T21:40:39.344 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4885000000000001e-05 2024-09-21T21:40:39.344 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:39.344 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:39.344 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:39.344 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:39.345 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:39.345 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_repop(client.4385.0:7 3.1a e23/21 3:5cea7035:::rbd_info:head v 23'1, mlcod=0'0)", 2024-09-21T21:40:39.345 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:38.416166+0000", 2024-09-21T21:40:39.345 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.91861201800000003, 2024-09-21T21:40:39.345 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00070378499999999996, 2024-09-21T21:40:39.345 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:39.345 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:39.345 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:39.345 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:39.345 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:39.345 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:39.345 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:38.416166+0000", 2024-09-21T21:40:39.346 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:39.346 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:39.346 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:39.346 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:39.346 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:38.416166+0000", 2024-09-21T21:40:39.346 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:39.346 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:39.346 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:39.346 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:39.346 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:38.416166+0000", 2024-09-21T21:40:39.346 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.5699999999999999e-07 2024-09-21T21:40:39.346 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:39.346 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:39.346 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:39.346 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:38.416171+0000", 2024-09-21T21:40:39.347 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9590000000000003e-06 2024-09-21T21:40:39.347 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:39.347 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:39.347 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:39.347 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:38.416180+0000", 2024-09-21T21:40:39.347 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.3039999999999994e-06 2024-09-21T21:40:39.347 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:39.347 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:39.347 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:39.347 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:38.416188+0000", 2024-09-21T21:40:39.347 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7570000000000004e-06 2024-09-21T21:40:39.347 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:39.347 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:39.347 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:39.348 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:38.416223+0000", 2024-09-21T21:40:39.348 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5172999999999997e-05 2024-09-21T21:40:39.348 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:39.348 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:39.348 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:39.348 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:38.416244+0000", 2024-09-21T21:40:39.348 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0744e-05 2024-09-21T21:40:39.348 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:39.348 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:39.348 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:39.348 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:38.416838+0000", 2024-09-21T21:40:39.348 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00059400499999999999 2024-09-21T21:40:39.348 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:39.348 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:39.348 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:39.348 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:38.416870+0000", 2024-09-21T21:40:39.349 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.1585999999999999e-05 2024-09-21T21:40:39.349 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:39.349 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:39.349 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:39.349 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:39.349 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:39.349 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:40:39.350 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T21:40:39.392 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:40:39.391+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:40:39.412 INFO:teuthology.orchestra.run.smithi081.stderr:nodeep-scrub is set 2024-09-21T21:40:39.492 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:40:39.491+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:40:39.500 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:40:39.500 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:40:39.500 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 0 2024-09-21T21:40:39.500 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:40:39.509 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T21:40:39.593 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:40:39.592+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:40:39.654 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:40:39.655 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:40:39.655 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:40:39.655 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:40:39.655 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:40:39.664 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T21:40:39.707 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:40:39.706+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:40:39.766 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:40:39.766 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 20, 2024-09-21T21:40:39.767 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 600, 2024-09-21T21:40:39.767 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:40:39.767 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:39.767 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_repop(client.4214.0:1 2.3 e17/16 2:c4c92e5a:::rbd_trash:head v 17'1, mlcod=0'0)", 2024-09-21T21:40:39.767 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:25.375128+0000", 2024-09-21T21:40:39.767 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 14.39031119, 2024-09-21T21:40:39.767 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.005806243, 2024-09-21T21:40:39.767 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:39.768 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:39.768 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:39.768 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:39.768 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:39.768 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:39.768 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:25.375128+0000", 2024-09-21T21:40:39.768 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:39.768 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:39.768 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:39.768 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:39.768 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:25.375128+0000", 2024-09-21T21:40:39.768 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:39.768 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:39.769 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:39.769 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:39.769 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:25.375128+0000", 2024-09-21T21:40:39.769 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3099999999999999e-07 2024-09-21T21:40:39.769 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:39.769 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:39.769 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:39.769 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:25.375135+0000", 2024-09-21T21:40:39.769 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.8079999999999999e-06 2024-09-21T21:40:39.769 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:39.769 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:39.769 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:39.769 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:25.375144+0000", 2024-09-21T21:40:39.770 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.0159999999999997e-06 2024-09-21T21:40:39.770 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:39.770 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:39.770 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:39.770 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:25.375152+0000", 2024-09-21T21:40:39.770 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.4179999999999997e-06 2024-09-21T21:40:39.770 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:39.770 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:39.770 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:39.770 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:25.375202+0000", 2024-09-21T21:40:39.770 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0212e-05 2024-09-21T21:40:39.770 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:39.770 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:39.770 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:39.771 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:25.375229+0000", 2024-09-21T21:40:39.771 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.6417000000000001e-05 2024-09-21T21:40:39.771 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:39.771 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:39.771 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:39.771 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:25.380896+0000", 2024-09-21T21:40:39.771 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0056673060000000004 2024-09-21T21:40:39.771 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:39.771 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:39.771 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:39.771 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:25.380934+0000", 2024-09-21T21:40:39.771 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7834999999999999e-05 2024-09-21T21:40:39.772 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:39.772 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:39.772 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:39.772 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:39.772 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:39.772 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_repop(client.4385.0:12 3.0 e24/21 3:00e0eee1:::rbd_header.1121c53dac8e:head v 24'1, mlcod=0'0)", 2024-09-21T21:40:39.772 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:39.432404+0000", 2024-09-21T21:40:39.772 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.33303435199999998, 2024-09-21T21:40:39.772 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00059507899999999999, 2024-09-21T21:40:39.772 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:39.772 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:39.772 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:39.772 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:39.772 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:39.773 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:39.773 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:39.432404+0000", 2024-09-21T21:40:39.773 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:39.773 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:39.773 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:39.773 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:39.773 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:39.432404+0000", 2024-09-21T21:40:39.773 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:39.773 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:39.773 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:39.773 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:39.774 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:39.432405+0000", 2024-09-21T21:40:39.774 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.16e-07 2024-09-21T21:40:39.774 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:39.774 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:39.774 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:39.774 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:39.432410+0000", 2024-09-21T21:40:39.774 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7409999999999998e-06 2024-09-21T21:40:39.774 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:39.774 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:39.774 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:39.775 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:39.432422+0000", 2024-09-21T21:40:39.775 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1942e-05 2024-09-21T21:40:39.775 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:39.775 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:39.775 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:39.775 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:39.432430+0000", 2024-09-21T21:40:39.775 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.1999999999999997e-06 2024-09-21T21:40:39.775 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:39.775 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:39.775 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:39.775 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:39.432483+0000", 2024-09-21T21:40:39.775 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3075999999999998e-05 2024-09-21T21:40:39.776 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:39.776 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:39.776 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:39.776 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:39.432498+0000", 2024-09-21T21:40:39.776 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.502e-05 2024-09-21T21:40:39.776 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:39.776 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:39.776 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:39.776 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:39.432969+0000", 2024-09-21T21:40:39.776 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00047160300000000002 2024-09-21T21:40:39.776 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:39.776 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:39.776 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:39.776 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:39.433000+0000", 2024-09-21T21:40:39.776 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0281e-05 2024-09-21T21:40:39.777 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:39.777 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:39.777 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:39.777 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:39.777 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:39.777 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_repop(client.4385.0:25 3.0 e24/21 3:00e0eee1:::rbd_header.1121c53dac8e:head v 24'2, mlcod=0'0)", 2024-09-21T21:40:39.777 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:39.443918+0000", 2024-09-21T21:40:39.777 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.32152087600000001, 2024-09-21T21:40:39.777 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.001308497, 2024-09-21T21:40:39.777 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:39.777 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:39.777 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:39.778 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:39.778 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:39.778 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:39.778 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:39.443918+0000", 2024-09-21T21:40:39.778 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:39.778 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:39.778 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:39.778 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:39.778 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:39.443918+0000", 2024-09-21T21:40:39.778 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:39.778 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:39.778 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:39.778 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:39.778 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:39.443919+0000", 2024-09-21T21:40:39.779 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.032e-06 2024-09-21T21:40:39.779 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:39.779 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:39.779 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:39.779 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:39.443928+0000", 2024-09-21T21:40:39.779 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.0289999999999994e-06 2024-09-21T21:40:39.779 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:39.779 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:39.779 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:39.779 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:39.443949+0000", 2024-09-21T21:40:39.779 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0871999999999999e-05 2024-09-21T21:40:39.779 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:39.779 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:39.779 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:39.779 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:39.443957+0000", 2024-09-21T21:40:39.779 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.9400000000000002e-06 2024-09-21T21:40:39.780 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:39.780 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:39.780 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:39.780 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:39.444005+0000", 2024-09-21T21:40:39.780 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7729000000000003e-05 2024-09-21T21:40:39.780 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:39.780 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:39.780 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:39.780 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:39.444025+0000", 2024-09-21T21:40:39.780 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0268e-05 2024-09-21T21:40:39.780 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:39.780 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:39.780 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:39.781 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:39.445198+0000", 2024-09-21T21:40:39.781 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0011727770000000001 2024-09-21T21:40:39.781 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:39.781 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:39.781 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:39.781 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:39.445226+0000", 2024-09-21T21:40:39.781 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.885e-05 2024-09-21T21:40:39.781 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:39.781 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:39.781 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:39.781 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:39.781 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:39.781 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_repop(client.4400.0:12 3.0 e24/21 3:00e0eee1:::rbd_header.1121c53dac8e:head v 24'3, mlcod=24'1)", 2024-09-21T21:40:39.781 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:39.594339+0000", 2024-09-21T21:40:39.781 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.17109961800000001, 2024-09-21T21:40:39.782 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00063064299999999996, 2024-09-21T21:40:39.782 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:39.782 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:39.782 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:39.782 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:39.782 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:39.782 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:39.782 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:39.594339+0000", 2024-09-21T21:40:39.782 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:39.782 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:39.782 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:39.782 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:39.782 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:39.594339+0000", 2024-09-21T21:40:39.782 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:39.783 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:39.783 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:39.783 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:39.783 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:39.594340+0000", 2024-09-21T21:40:39.783 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.0400000000000005e-07 2024-09-21T21:40:39.783 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:39.783 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:39.783 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:39.783 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:39.594352+0000", 2024-09-21T21:40:39.783 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.2084e-05 2024-09-21T21:40:39.783 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:39.783 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:39.783 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:39.784 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:39.594366+0000", 2024-09-21T21:40:39.784 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.3917000000000001e-05 2024-09-21T21:40:39.784 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:39.784 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:39.784 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:39.784 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:39.594375+0000", 2024-09-21T21:40:39.784 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.3449999999999995e-06 2024-09-21T21:40:39.784 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:39.784 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:39.784 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:39.784 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:39.594437+0000", 2024-09-21T21:40:39.784 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1186999999999995e-05 2024-09-21T21:40:39.785 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:39.785 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:39.785 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:39.785 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:39.594460+0000", 2024-09-21T21:40:39.785 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3764000000000002e-05 2024-09-21T21:40:39.785 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:39.785 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:39.785 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:39.785 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:39.594918+0000", 2024-09-21T21:40:39.785 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00045801199999999999 2024-09-21T21:40:39.785 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:39.785 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:39.785 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:39.785 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:39.594970+0000", 2024-09-21T21:40:39.786 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1430000000000001e-05 2024-09-21T21:40:39.786 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:39.786 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:39.786 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:39.786 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:39.786 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:39.786 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:40:39.787 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T21:40:39.828 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:40:39.827+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:40:39.896 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:40:39.895+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:40:39.900 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:40:39.900 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:40:39.900 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-21T21:40:39.900 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:40:39.910 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T21:40:39.997 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:40:39.995+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:40:40.053 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:40:40.053 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:40:40.054 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:40:40.054 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:40:40.054 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:40:40.062 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T21:40:40.097 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:40:40.096+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:40:40.167 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:40:40.167 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-21T21:40:40.167 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-21T21:40:40.168 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:40:40.168 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.168 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4183.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T21:40:40.168 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:19.643261+0000", 2024-09-21T21:40:40.168 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 20.521267519999999, 2024-09-21T21:40:40.168 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0075300310000000004, 2024-09-21T21:40:40.168 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:40.168 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:40.168 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:40.168 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:40.168 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4183", 2024-09-21T21:40:40.169 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.122:0/3913072034", 2024-09-21T21:40:40.169 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 112 2024-09-21T21:40:40.169 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.169 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:40.169 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.169 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:40.169 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643261+0000", 2024-09-21T21:40:40.169 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:40.169 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.169 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.169 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:40.169 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643261+0000", 2024-09-21T21:40:40.169 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:40.170 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.170 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.170 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:40.170 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643262+0000", 2024-09-21T21:40:40.170 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.99e-07 2024-09-21T21:40:40.170 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.170 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.170 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:40.170 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643326+0000", 2024-09-21T21:40:40.170 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4244000000000004e-05 2024-09-21T21:40:40.171 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.171 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.171 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:40.171 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643338+0000", 2024-09-21T21:40:40.171 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1807999999999999e-05 2024-09-21T21:40:40.171 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.172 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.172 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:40.172 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643344+0000", 2024-09-21T21:40:40.172 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4479999999999999e-06 2024-09-21T21:40:40.172 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.172 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.172 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:40.173 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.647517+0000", 2024-09-21T21:40:40.173 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.004172711 2024-09-21T21:40:40.173 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.173 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.173 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:40.173 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.647570+0000", 2024-09-21T21:40:40.173 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3094999999999999e-05 2024-09-21T21:40:40.173 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.174 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.174 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:40:40.174 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.647660+0000", 2024-09-21T21:40:40.174 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.0428999999999998e-05 2024-09-21T21:40:40.174 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.174 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.174 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:40.174 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.649208+0000", 2024-09-21T21:40:40.175 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0015479269999999999 2024-09-21T21:40:40.175 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.175 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.175 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:40.175 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.650733+0000", 2024-09-21T21:40:40.175 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0015247710000000001 2024-09-21T21:40:40.175 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.175 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.175 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:40.175 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.650764+0000", 2024-09-21T21:40:40.176 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0840000000000003e-05 2024-09-21T21:40:40.176 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.176 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.176 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:40.176 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.650791+0000", 2024-09-21T21:40:40.176 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7158999999999999e-05 2024-09-21T21:40:40.176 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:40.176 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:40.176 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:40.176 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.176 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.176 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4183.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T21:40:40.176 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:19.643357+0000", 2024-09-21T21:40:40.176 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 20.521171103, 2024-09-21T21:40:40.177 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.007525398, 2024-09-21T21:40:40.177 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:40.177 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:40.177 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:40.177 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:40.177 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4183", 2024-09-21T21:40:40.177 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.122:0/3913072034", 2024-09-21T21:40:40.177 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 113 2024-09-21T21:40:40.177 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.177 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:40.177 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.177 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:40.177 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643357+0000", 2024-09-21T21:40:40.177 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:40.177 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.178 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.178 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:40.178 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643357+0000", 2024-09-21T21:40:40.178 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:40.178 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.178 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.178 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:40.178 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643358+0000", 2024-09-21T21:40:40.178 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7100000000000001e-07 2024-09-21T21:40:40.178 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.178 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.178 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:40.178 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643362+0000", 2024-09-21T21:40:40.178 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2919999999999997e-06 2024-09-21T21:40:40.179 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.179 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.179 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:40.179 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643370+0000", 2024-09-21T21:40:40.179 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7190000000000001e-06 2024-09-21T21:40:40.179 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.179 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.179 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:40.179 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643376+0000", 2024-09-21T21:40:40.179 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3219999999999998e-06 2024-09-21T21:40:40.179 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.179 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.179 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:40.179 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.648180+0000", 2024-09-21T21:40:40.179 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048035619999999999 2024-09-21T21:40:40.180 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.180 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.180 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:40.180 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.648226+0000", 2024-09-21T21:40:40.180 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6303e-05 2024-09-21T21:40:40.180 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.180 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.180 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:40:40.180 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.648303+0000", 2024-09-21T21:40:40.180 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6831999999999999e-05 2024-09-21T21:40:40.180 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.180 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.180 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:40.181 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.649249+0000", 2024-09-21T21:40:40.181 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00094576299999999996 2024-09-21T21:40:40.181 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.181 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.181 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:40.181 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.650799+0000", 2024-09-21T21:40:40.181 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001549976 2024-09-21T21:40:40.181 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.181 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.181 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:40.181 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.650845+0000", 2024-09-21T21:40:40.181 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6133e-05 2024-09-21T21:40:40.181 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.181 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.182 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:40.182 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.650883+0000", 2024-09-21T21:40:40.182 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7824999999999998e-05 2024-09-21T21:40:40.182 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:40.182 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:40.182 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:40.182 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.182 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.182 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4183.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T21:40:40.182 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:19.643392+0000", 2024-09-21T21:40:40.182 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 20.521136031000001, 2024-09-21T21:40:40.182 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0075764780000000002, 2024-09-21T21:40:40.183 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:40.183 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:40.183 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:40.183 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:40.183 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4183", 2024-09-21T21:40:40.183 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.122:0/3913072034", 2024-09-21T21:40:40.183 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 114 2024-09-21T21:40:40.183 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.183 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:40.183 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.183 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:40.183 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643392+0000", 2024-09-21T21:40:40.183 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:40.184 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.184 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.184 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:40.184 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643392+0000", 2024-09-21T21:40:40.184 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:40.184 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.184 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.184 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:40.184 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643393+0000", 2024-09-21T21:40:40.184 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2900000000000003e-07 2024-09-21T21:40:40.184 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.184 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.184 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:40.184 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643398+0000", 2024-09-21T21:40:40.185 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3159999999999998e-06 2024-09-21T21:40:40.185 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.185 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.185 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:40.185 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643403+0000", 2024-09-21T21:40:40.185 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5419999999999996e-06 2024-09-21T21:40:40.185 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.185 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.185 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:40.185 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643407+0000", 2024-09-21T21:40:40.185 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4649999999999999e-06 2024-09-21T21:40:40.185 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.185 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.186 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:40.186 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.648632+0000", 2024-09-21T21:40:40.186 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0052258000000000001 2024-09-21T21:40:40.186 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.186 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.186 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:40.186 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.648734+0000", 2024-09-21T21:40:40.186 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010178099999999999 2024-09-21T21:40:40.186 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.186 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.186 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:40:40.186 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.648880+0000", 2024-09-21T21:40:40.186 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014552799999999999 2024-09-21T21:40:40.186 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.187 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.187 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:40.187 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.650398+0000", 2024-09-21T21:40:40.187 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0015188389999999999 2024-09-21T21:40:40.187 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.187 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.187 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:40.187 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.650903+0000", 2024-09-21T21:40:40.187 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00050413400000000005 2024-09-21T21:40:40.187 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.187 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.187 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:40.187 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.650940+0000", 2024-09-21T21:40:40.187 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7354999999999997e-05 2024-09-21T21:40:40.188 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.188 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.188 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:40.188 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.650969+0000", 2024-09-21T21:40:40.188 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8989e-05 2024-09-21T21:40:40.188 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:40.188 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:40.188 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:40.188 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.188 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.188 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4183.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T21:40:40.188 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:19.643416+0000", 2024-09-21T21:40:40.189 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 20.521112091999999, 2024-09-21T21:40:40.189 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0076828269999999997, 2024-09-21T21:40:40.189 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:40.189 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:40.189 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:40.189 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:40.189 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4183", 2024-09-21T21:40:40.189 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.122:0/3913072034", 2024-09-21T21:40:40.189 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 115 2024-09-21T21:40:40.189 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.189 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:40.189 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.189 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:40.190 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643416+0000", 2024-09-21T21:40:40.190 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:40.190 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.190 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.190 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:40.190 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643416+0000", 2024-09-21T21:40:40.190 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:40.190 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.190 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.190 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:40.190 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643417+0000", 2024-09-21T21:40:40.190 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.96e-07 2024-09-21T21:40:40.190 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.191 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.191 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:40.191 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643469+0000", 2024-09-21T21:40:40.191 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2293999999999998e-05 2024-09-21T21:40:40.191 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.191 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.191 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:40.193 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643474+0000", 2024-09-21T21:40:40.194 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3179999999999997e-06 2024-09-21T21:40:40.194 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.194 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.194 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:40.194 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643488+0000", 2024-09-21T21:40:40.194 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3241000000000001e-05 2024-09-21T21:40:40.194 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.194 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.194 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:40.194 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.649349+0000", 2024-09-21T21:40:40.195 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0058617690000000002 2024-09-21T21:40:40.195 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.195 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.195 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:40.195 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.649440+0000", 2024-09-21T21:40:40.195 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.1092000000000001e-05 2024-09-21T21:40:40.195 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.195 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.195 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:40:40.195 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.649583+0000", 2024-09-21T21:40:40.195 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014218499999999999 2024-09-21T21:40:40.195 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.195 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.195 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:40.195 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.650978+0000", 2024-09-21T21:40:40.196 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0013956210000000001 2024-09-21T21:40:40.196 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.196 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.196 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:40.196 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.651032+0000", 2024-09-21T21:40:40.196 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3513000000000001e-05 2024-09-21T21:40:40.196 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.196 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.196 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:40.197 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.651068+0000", 2024-09-21T21:40:40.197 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6675999999999999e-05 2024-09-21T21:40:40.197 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.197 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.197 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:40.197 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.651099+0000", 2024-09-21T21:40:40.197 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0821999999999997e-05 2024-09-21T21:40:40.197 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:40.197 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:40.197 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:40.197 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.197 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.197 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4183.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T21:40:40.197 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:19.643496+0000", 2024-09-21T21:40:40.197 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 20.521032547000001, 2024-09-21T21:40:40.198 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0077031249999999999, 2024-09-21T21:40:40.198 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:40.198 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:40.198 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:40.198 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:40.198 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4183", 2024-09-21T21:40:40.198 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.122:0/3913072034", 2024-09-21T21:40:40.198 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 116 2024-09-21T21:40:40.198 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.198 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:40.198 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.198 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:40.198 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643496+0000", 2024-09-21T21:40:40.199 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:40.199 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.199 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.199 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:40.199 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643496+0000", 2024-09-21T21:40:40.199 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:40.199 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.199 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.199 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:40.199 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643496+0000", 2024-09-21T21:40:40.199 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3599999999999999e-07 2024-09-21T21:40:40.199 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.199 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.199 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:40.199 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643500+0000", 2024-09-21T21:40:40.199 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1790000000000001e-06 2024-09-21T21:40:40.200 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.200 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.200 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:40.200 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643504+0000", 2024-09-21T21:40:40.200 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8299999999999998e-06 2024-09-21T21:40:40.200 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.200 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.200 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:40.200 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643507+0000", 2024-09-21T21:40:40.200 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.056e-06 2024-09-21T21:40:40.200 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.200 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.200 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:40.200 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.650003+0000", 2024-09-21T21:40:40.201 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0064959129999999999 2024-09-21T21:40:40.201 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.201 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.201 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:40.201 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.650051+0000", 2024-09-21T21:40:40.201 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8180999999999998e-05 2024-09-21T21:40:40.201 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.201 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.201 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:40:40.201 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.650127+0000", 2024-09-21T21:40:40.201 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5795000000000002e-05 2024-09-21T21:40:40.201 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.201 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.202 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:40.202 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.651137+0000", 2024-09-21T21:40:40.202 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0010094699999999999 2024-09-21T21:40:40.202 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.202 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.202 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:40.202 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.651144+0000", 2024-09-21T21:40:40.202 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.985e-06 2024-09-21T21:40:40.202 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.202 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.202 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:40.202 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.651169+0000", 2024-09-21T21:40:40.202 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.563e-05 2024-09-21T21:40:40.202 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.203 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.203 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:40.203 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.651199+0000", 2024-09-21T21:40:40.203 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9750000000000001e-05 2024-09-21T21:40:40.203 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:40.203 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:40.203 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:40.203 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.203 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.203 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4183.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T21:40:40.203 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:19.643512+0000", 2024-09-21T21:40:40.203 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 20.521016932999999, 2024-09-21T21:40:40.203 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0077806539999999997, 2024-09-21T21:40:40.204 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:40.204 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:40.204 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:40.204 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:40.204 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4183", 2024-09-21T21:40:40.204 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.122:0/3913072034", 2024-09-21T21:40:40.204 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 117 2024-09-21T21:40:40.204 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.204 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:40.204 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.204 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:40.204 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643512+0000", 2024-09-21T21:40:40.204 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:40.204 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.204 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.205 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:40.205 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643512+0000", 2024-09-21T21:40:40.205 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:40.205 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.205 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.205 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:40.205 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643512+0000", 2024-09-21T21:40:40.205 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.16e-07 2024-09-21T21:40:40.205 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.205 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.205 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:40.205 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643514+0000", 2024-09-21T21:40:40.205 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3709999999999998e-06 2024-09-21T21:40:40.205 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.206 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.206 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:40.206 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643517+0000", 2024-09-21T21:40:40.206 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2310000000000001e-06 2024-09-21T21:40:40.206 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.206 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.206 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:40.206 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643522+0000", 2024-09-21T21:40:40.206 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8319999999999996e-06 2024-09-21T21:40:40.206 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.206 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.206 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:40.206 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.650426+0000", 2024-09-21T21:40:40.206 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0069033380000000002 2024-09-21T21:40:40.207 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.207 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.207 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:40.207 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.650482+0000", 2024-09-21T21:40:40.207 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6468999999999998e-05 2024-09-21T21:40:40.207 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.207 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.207 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:40:40.207 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.650565+0000", 2024-09-21T21:40:40.207 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3026000000000001e-05 2024-09-21T21:40:40.207 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.207 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.207 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:40.207 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.651209+0000", 2024-09-21T21:40:40.207 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00064387199999999998 2024-09-21T21:40:40.208 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.208 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.208 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:40.208 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.651238+0000", 2024-09-21T21:40:40.208 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8799e-05 2024-09-21T21:40:40.208 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.208 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.208 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:40.208 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.651270+0000", 2024-09-21T21:40:40.208 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1934000000000003e-05 2024-09-21T21:40:40.208 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.208 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.208 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:40.208 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.651292+0000", 2024-09-21T21:40:40.208 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2566e-05 2024-09-21T21:40:40.209 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:40.209 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:40.209 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:40.209 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.209 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.209 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4183.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T21:40:40.209 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:19.916463+0000", 2024-09-21T21:40:40.209 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 20.248065449999999, 2024-09-21T21:40:40.209 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0094252929999999995, 2024-09-21T21:40:40.209 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:40.209 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:40.209 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:40.209 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:40.209 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4183", 2024-09-21T21:40:40.210 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.122:0/3913072034", 2024-09-21T21:40:40.210 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 134 2024-09-21T21:40:40.210 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.210 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:40.210 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.210 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:40.210 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916463+0000", 2024-09-21T21:40:40.210 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:40.210 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.210 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.210 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:40.210 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916463+0000", 2024-09-21T21:40:40.210 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:40.210 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.210 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.211 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:40.211 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916464+0000", 2024-09-21T21:40:40.211 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2900000000000004e-07 2024-09-21T21:40:40.211 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.211 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.211 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:40.211 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916525+0000", 2024-09-21T21:40:40.211 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1576999999999997e-05 2024-09-21T21:40:40.211 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.211 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.211 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:40.211 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916533+0000", 2024-09-21T21:40:40.211 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7510000000000008e-06 2024-09-21T21:40:40.211 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.211 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.212 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:40.212 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916537+0000", 2024-09-21T21:40:40.212 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.155e-06 2024-09-21T21:40:40.212 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.212 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.212 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:40.212 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.919134+0000", 2024-09-21T21:40:40.212 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025973469999999999 2024-09-21T21:40:40.212 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.212 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.212 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:40.212 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.919189+0000", 2024-09-21T21:40:40.212 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4339999999999998e-05 2024-09-21T21:40:40.212 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.212 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.213 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:40:40.213 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.919261+0000", 2024-09-21T21:40:40.213 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1977000000000006e-05 2024-09-21T21:40:40.213 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.213 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.213 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:40.213 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.920953+0000", 2024-09-21T21:40:40.213 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0016926109999999999 2024-09-21T21:40:40.213 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.213 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.213 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:40.213 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.925825+0000", 2024-09-21T21:40:40.213 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048719159999999996 2024-09-21T21:40:40.213 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.213 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.214 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:40.214 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.925854+0000", 2024-09-21T21:40:40.214 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8524999999999999e-05 2024-09-21T21:40:40.214 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.214 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.214 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:40.214 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.925888+0000", 2024-09-21T21:40:40.214 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4564999999999999e-05 2024-09-21T21:40:40.214 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:40.214 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:40.214 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:40.214 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.214 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.214 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4183.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T21:40:40.215 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:19.916545+0000", 2024-09-21T21:40:40.215 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 20.247983532999999, 2024-09-21T21:40:40.215 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0094280869999999999, 2024-09-21T21:40:40.215 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:40.215 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:40.215 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:40.215 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:40.215 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4183", 2024-09-21T21:40:40.215 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.122:0/3913072034", 2024-09-21T21:40:40.215 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 135 2024-09-21T21:40:40.215 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.215 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:40.215 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.215 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:40.216 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916545+0000", 2024-09-21T21:40:40.216 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:40.216 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.216 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.216 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:40.216 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916545+0000", 2024-09-21T21:40:40.216 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:40.216 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.216 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.216 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:40.216 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916545+0000", 2024-09-21T21:40:40.216 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1100000000000002e-07 2024-09-21T21:40:40.216 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.216 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.216 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:40.216 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916549+0000", 2024-09-21T21:40:40.217 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0799999999999999e-06 2024-09-21T21:40:40.217 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.217 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.217 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:40.217 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916556+0000", 2024-09-21T21:40:40.217 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0339999999999999e-06 2024-09-21T21:40:40.217 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.217 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.217 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:40.217 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916562+0000", 2024-09-21T21:40:40.217 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5439999999999998e-06 2024-09-21T21:40:40.217 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.217 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.217 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:40.218 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.919649+0000", 2024-09-21T21:40:40.218 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0030868660000000002 2024-09-21T21:40:40.218 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.218 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.218 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:40.218 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.919693+0000", 2024-09-21T21:40:40.218 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4694000000000003e-05 2024-09-21T21:40:40.218 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.218 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.218 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:40:40.218 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.919765+0000", 2024-09-21T21:40:40.218 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1427999999999994e-05 2024-09-21T21:40:40.218 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.218 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.218 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:40.219 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.921005+0000", 2024-09-21T21:40:40.219 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001240314 2024-09-21T21:40:40.219 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.219 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.219 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:40.219 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.925897+0000", 2024-09-21T21:40:40.219 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048916660000000002 2024-09-21T21:40:40.219 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.219 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.219 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:40.219 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.925942+0000", 2024-09-21T21:40:40.219 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4721000000000001e-05 2024-09-21T21:40:40.219 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.219 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.219 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:40.220 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.925973+0000", 2024-09-21T21:40:40.220 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1429e-05 2024-09-21T21:40:40.220 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:40.220 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:40.220 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:40.220 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.220 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.220 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4183.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T21:40:40.220 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:19.916568+0000", 2024-09-21T21:40:40.220 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 20.247960598999999, 2024-09-21T21:40:40.220 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0094755120000000002, 2024-09-21T21:40:40.220 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:40.220 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:40.220 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:40.220 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:40.221 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4183", 2024-09-21T21:40:40.221 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.122:0/3913072034", 2024-09-21T21:40:40.221 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 136 2024-09-21T21:40:40.221 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.221 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:40.221 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.221 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:40.221 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916568+0000", 2024-09-21T21:40:40.221 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:40.221 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.221 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.221 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:40.221 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916568+0000", 2024-09-21T21:40:40.221 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:40.221 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.222 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.222 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:40.222 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916568+0000", 2024-09-21T21:40:40.222 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9200000000000002e-07 2024-09-21T21:40:40.222 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.222 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.222 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:40.222 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916572+0000", 2024-09-21T21:40:40.222 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.428e-06 2024-09-21T21:40:40.222 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.222 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.222 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:40.222 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916576+0000", 2024-09-21T21:40:40.222 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.814e-06 2024-09-21T21:40:40.223 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.223 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.223 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:40.223 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916580+0000", 2024-09-21T21:40:40.223 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6669999999999998e-06 2024-09-21T21:40:40.223 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.223 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.223 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:40.223 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.920045+0000", 2024-09-21T21:40:40.223 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0034645209999999999 2024-09-21T21:40:40.223 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.223 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.223 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:40.223 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.920091+0000", 2024-09-21T21:40:40.223 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6146999999999997e-05 2024-09-21T21:40:40.224 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.224 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.224 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:40:40.224 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.920159+0000", 2024-09-21T21:40:40.224 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8411000000000006e-05 2024-09-21T21:40:40.224 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.224 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.224 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:40.224 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.921037+0000", 2024-09-21T21:40:40.224 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00087794700000000002 2024-09-21T21:40:40.224 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.224 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.224 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:40.224 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.925981+0000", 2024-09-21T21:40:40.224 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0049443259999999998 2024-09-21T21:40:40.225 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.225 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.225 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:40.225 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.926014+0000", 2024-09-21T21:40:40.225 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2298999999999997e-05 2024-09-21T21:40:40.225 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.225 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.225 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:40.225 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.926043+0000", 2024-09-21T21:40:40.225 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9660000000000001e-05 2024-09-21T21:40:40.225 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:40.225 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:40.225 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:40.225 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.225 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.226 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4183.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T21:40:40.226 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:19.916587+0000", 2024-09-21T21:40:40.226 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 20.247941119, 2024-09-21T21:40:40.226 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.009527265, 2024-09-21T21:40:40.226 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:40.226 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:40.226 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:40.226 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:40.226 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4183", 2024-09-21T21:40:40.226 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.122:0/3913072034", 2024-09-21T21:40:40.226 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 137 2024-09-21T21:40:40.226 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.226 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:40.227 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.227 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:40.227 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916587+0000", 2024-09-21T21:40:40.227 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:40.227 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.227 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.227 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:40.227 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916587+0000", 2024-09-21T21:40:40.227 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:40.227 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.227 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.227 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:40.227 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916588+0000", 2024-09-21T21:40:40.227 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5699999999999999e-07 2024-09-21T21:40:40.228 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.228 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.228 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:40.228 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916644+0000", 2024-09-21T21:40:40.228 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6344000000000001e-05 2024-09-21T21:40:40.228 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.228 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.228 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:40.228 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916649+0000", 2024-09-21T21:40:40.228 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0409999999999997e-06 2024-09-21T21:40:40.228 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.228 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.228 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:40.228 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916654+0000", 2024-09-21T21:40:40.228 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9919999999999998e-06 2024-09-21T21:40:40.229 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.229 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.229 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:40.229 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.920443+0000", 2024-09-21T21:40:40.229 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0037892469999999999 2024-09-21T21:40:40.229 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.229 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.229 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:40.229 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.920493+0000", 2024-09-21T21:40:40.229 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9898000000000002e-05 2024-09-21T21:40:40.229 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.229 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.229 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:40:40.229 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.920561+0000", 2024-09-21T21:40:40.229 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7481e-05 2024-09-21T21:40:40.230 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.230 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.230 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:40.230 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.925739+0000", 2024-09-21T21:40:40.230 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.005178729 2024-09-21T21:40:40.230 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.230 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.230 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:40.230 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.926052+0000", 2024-09-21T21:40:40.230 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00031252800000000001 2024-09-21T21:40:40.230 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.230 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.230 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:40.231 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.926087+0000", 2024-09-21T21:40:40.231 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4873999999999998e-05 2024-09-21T21:40:40.231 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.231 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.231 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:40.231 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.926115+0000", 2024-09-21T21:40:40.231 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7874000000000001e-05 2024-09-21T21:40:40.231 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:40.231 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:40.231 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:40.231 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.231 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.231 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4183.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T21:40:40.231 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:19.916666+0000", 2024-09-21T21:40:40.232 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 20.247862248000001, 2024-09-21T21:40:40.232 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0095150730000000006, 2024-09-21T21:40:40.232 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:40.232 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:40.232 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:40.232 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:40.232 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4183", 2024-09-21T21:40:40.232 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.122:0/3913072034", 2024-09-21T21:40:40.232 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 138 2024-09-21T21:40:40.232 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.232 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:40.232 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.232 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:40.232 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916666+0000", 2024-09-21T21:40:40.233 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:40.233 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.233 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.233 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:40.233 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916666+0000", 2024-09-21T21:40:40.233 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:40.233 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.233 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.233 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:40.233 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916667+0000", 2024-09-21T21:40:40.233 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3200000000000005e-07 2024-09-21T21:40:40.233 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.233 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.233 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:40.233 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916670+0000", 2024-09-21T21:40:40.233 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.427e-06 2024-09-21T21:40:40.234 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.234 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.234 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:40.234 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916684+0000", 2024-09-21T21:40:40.234 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3839000000000001e-05 2024-09-21T21:40:40.234 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.234 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.234 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:40.234 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916706+0000", 2024-09-21T21:40:40.234 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1543e-05 2024-09-21T21:40:40.234 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.234 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.234 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:40.235 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.921064+0000", 2024-09-21T21:40:40.235 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0043587119999999998 2024-09-21T21:40:40.235 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.235 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.235 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:40.235 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.921119+0000", 2024-09-21T21:40:40.235 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4619e-05 2024-09-21T21:40:40.235 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.235 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.235 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:40:40.235 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.921192+0000", 2024-09-21T21:40:40.235 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3224000000000002e-05 2024-09-21T21:40:40.235 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.235 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.235 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:40.236 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.925791+0000", 2024-09-21T21:40:40.236 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0045988670000000004 2024-09-21T21:40:40.236 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.236 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.236 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:40.236 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.926123+0000", 2024-09-21T21:40:40.236 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00033195000000000002 2024-09-21T21:40:40.236 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.236 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.236 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:40.236 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.926154+0000", 2024-09-21T21:40:40.236 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0720999999999999e-05 2024-09-21T21:40:40.236 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.236 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.236 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:40.237 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.926181+0000", 2024-09-21T21:40:40.237 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7639000000000001e-05 2024-09-21T21:40:40.237 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:40.237 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:40.237 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:40.237 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.237 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.237 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4183.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T21:40:40.237 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:19.916714+0000", 2024-09-21T21:40:40.237 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 20.247814358999999, 2024-09-21T21:40:40.237 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0095298369999999993, 2024-09-21T21:40:40.237 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:40.237 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:40.237 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:40.237 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:40.238 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4183", 2024-09-21T21:40:40.238 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.122:0/3913072034", 2024-09-21T21:40:40.238 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 139 2024-09-21T21:40:40.238 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.238 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:40.238 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.238 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:40.238 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916714+0000", 2024-09-21T21:40:40.238 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:40.238 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.238 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.238 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:40.238 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916714+0000", 2024-09-21T21:40:40.238 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:40.239 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.239 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.239 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:40.239 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916714+0000", 2024-09-21T21:40:40.239 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9799999999999999e-07 2024-09-21T21:40:40.239 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.239 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.239 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:40.239 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916718+0000", 2024-09-21T21:40:40.239 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1810000000000001e-06 2024-09-21T21:40:40.239 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.239 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.239 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:40.239 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916724+0000", 2024-09-21T21:40:40.240 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7329999999999997e-06 2024-09-21T21:40:40.240 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.240 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.240 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:40.240 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916728+0000", 2024-09-21T21:40:40.240 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2909999999999999e-06 2024-09-21T21:40:40.240 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.240 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.240 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:40.240 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.921416+0000", 2024-09-21T21:40:40.240 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0046882850000000004 2024-09-21T21:40:40.240 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.240 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.240 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:40.240 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.921461+0000", 2024-09-21T21:40:40.240 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5259000000000002e-05 2024-09-21T21:40:40.241 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.241 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.241 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:40:40.241 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.921540+0000", 2024-09-21T21:40:40.241 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8765999999999998e-05 2024-09-21T21:40:40.241 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.241 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.241 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:40.241 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.925818+0000", 2024-09-21T21:40:40.241 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0042784800000000003 2024-09-21T21:40:40.241 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.241 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.241 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:40.241 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.926190+0000", 2024-09-21T21:40:40.241 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00037112400000000002 2024-09-21T21:40:40.242 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.242 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.242 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:40.242 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.926225+0000", 2024-09-21T21:40:40.242 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5420999999999998e-05 2024-09-21T21:40:40.242 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.242 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.242 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:40.242 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.926244+0000", 2024-09-21T21:40:40.242 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.8998999999999999e-05 2024-09-21T21:40:40.242 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:40.242 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:40.242 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:40.243 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.243 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.243 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4183.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T21:40:40.243 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:19.916735+0000", 2024-09-21T21:40:40.243 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 20.247793961999999, 2024-09-21T21:40:40.243 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.010302162, 2024-09-21T21:40:40.243 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:40.243 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:40.243 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:40.243 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:40.243 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4183", 2024-09-21T21:40:40.243 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.122:0/3913072034", 2024-09-21T21:40:40.243 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 140 2024-09-21T21:40:40.243 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.244 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:40.244 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.244 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:40.244 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916735+0000", 2024-09-21T21:40:40.244 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:40.244 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.244 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.244 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:40.244 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916735+0000", 2024-09-21T21:40:40.244 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:40.244 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.244 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.244 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:40.244 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916735+0000", 2024-09-21T21:40:40.244 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5400000000000002e-07 2024-09-21T21:40:40.245 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.245 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.245 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:40.245 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916788+0000", 2024-09-21T21:40:40.245 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3074e-05 2024-09-21T21:40:40.245 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.245 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.245 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:40.245 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916795+0000", 2024-09-21T21:40:40.245 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.204e-06 2024-09-21T21:40:40.245 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.245 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.245 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:40.245 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916801+0000", 2024-09-21T21:40:40.246 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4759999999999997e-06 2024-09-21T21:40:40.246 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.246 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.246 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:40.246 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.921792+0000", 2024-09-21T21:40:40.246 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0049909749999999999 2024-09-21T21:40:40.246 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.246 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.246 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:40.246 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.921879+0000", 2024-09-21T21:40:40.246 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7361999999999995e-05 2024-09-21T21:40:40.246 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.246 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.246 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:40:40.246 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.922000+0000", 2024-09-21T21:40:40.247 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000121451 2024-09-21T21:40:40.247 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.247 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.247 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:40.247 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.926291+0000", 2024-09-21T21:40:40.247 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0042906180000000004 2024-09-21T21:40:40.247 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.247 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.247 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:40.247 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.926984+0000", 2024-09-21T21:40:40.247 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00069308599999999998 2024-09-21T21:40:40.247 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.247 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.247 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:40.248 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.927014+0000", 2024-09-21T21:40:40.248 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0222999999999998e-05 2024-09-21T21:40:40.248 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.248 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.248 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:40.248 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.927037+0000", 2024-09-21T21:40:40.248 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2439e-05 2024-09-21T21:40:40.248 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:40.248 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:40.248 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:40.248 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.248 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.248 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4183.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T21:40:40.248 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:19.916814+0000", 2024-09-21T21:40:40.249 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 20.247714772999998, 2024-09-21T21:40:40.249 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.010507598999999999, 2024-09-21T21:40:40.249 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:40.249 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:40.249 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:40.249 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:40.249 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4183", 2024-09-21T21:40:40.249 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.122:0/3913072034", 2024-09-21T21:40:40.249 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 141 2024-09-21T21:40:40.249 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.249 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:40.249 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.249 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:40.249 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916814+0000", 2024-09-21T21:40:40.250 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:40.250 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.250 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.250 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:40.250 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916814+0000", 2024-09-21T21:40:40.250 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:40.250 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.250 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.250 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:40.250 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916814+0000", 2024-09-21T21:40:40.250 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9599999999999999e-07 2024-09-21T21:40:40.250 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.250 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.250 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:40.250 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916819+0000", 2024-09-21T21:40:40.251 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7090000000000001e-06 2024-09-21T21:40:40.251 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.251 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.251 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:40.251 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916830+0000", 2024-09-21T21:40:40.251 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0977e-05 2024-09-21T21:40:40.251 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.251 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.251 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:40.251 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916835+0000", 2024-09-21T21:40:40.251 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2320000000000001e-06 2024-09-21T21:40:40.251 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.251 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.251 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:40.251 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.925249+0000", 2024-09-21T21:40:40.251 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0084135170000000006 2024-09-21T21:40:40.252 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.252 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.252 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:40.252 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.925323+0000", 2024-09-21T21:40:40.252 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3919999999999997e-05 2024-09-21T21:40:40.252 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.252 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.252 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:40:40.252 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.925442+0000", 2024-09-21T21:40:40.252 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00011944500000000001 2024-09-21T21:40:40.252 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.252 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.252 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:40.252 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.926322+0000", 2024-09-21T21:40:40.252 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00087985500000000003 2024-09-21T21:40:40.252 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.253 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.253 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:40.253 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.927245+0000", 2024-09-21T21:40:40.253 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00092342499999999996 2024-09-21T21:40:40.253 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.253 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.253 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:40.253 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.927283+0000", 2024-09-21T21:40:40.253 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8157000000000001e-05 2024-09-21T21:40:40.253 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.253 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.253 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:40.253 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.927321+0000", 2024-09-21T21:40:40.253 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7866e-05 2024-09-21T21:40:40.254 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:40.254 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:40.254 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:40.254 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.254 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.254 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4183.0:145 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T21:40:40.254 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:19.927637+0000", 2024-09-21T21:40:40.254 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 20.236891515, 2024-09-21T21:40:40.254 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.006407622, 2024-09-21T21:40:40.254 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:40.254 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:40.254 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:40.254 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:40.255 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4183", 2024-09-21T21:40:40.255 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.122:0/3913072034", 2024-09-21T21:40:40.255 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 145 2024-09-21T21:40:40.255 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.255 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:40.255 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.255 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:40.255 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.927637+0000", 2024-09-21T21:40:40.255 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:40.255 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.255 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.255 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:40.255 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.927637+0000", 2024-09-21T21:40:40.256 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:40.256 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.256 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.256 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:40.256 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.927637+0000", 2024-09-21T21:40:40.256 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5199999999999998e-07 2024-09-21T21:40:40.256 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.256 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.256 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:40.256 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.927684+0000", 2024-09-21T21:40:40.256 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6529000000000001e-05 2024-09-21T21:40:40.256 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.256 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.256 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:40.257 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.927689+0000", 2024-09-21T21:40:40.257 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6959999999999996e-06 2024-09-21T21:40:40.257 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.257 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.257 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:40.257 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.927692+0000", 2024-09-21T21:40:40.257 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1659999999999998e-06 2024-09-21T21:40:40.257 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.257 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.257 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:40.257 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.929102+0000", 2024-09-21T21:40:40.257 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0014106679999999999 2024-09-21T21:40:40.257 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.257 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.258 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:40.258 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.929147+0000", 2024-09-21T21:40:40.258 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4459999999999998e-05 2024-09-21T21:40:40.258 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.258 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.258 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:40:40.258 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.929219+0000", 2024-09-21T21:40:40.258 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2547999999999997e-05 2024-09-21T21:40:40.258 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.258 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.258 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:40.258 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.932636+0000", 2024-09-21T21:40:40.258 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0034168559999999998 2024-09-21T21:40:40.259 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.259 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.259 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:40.259 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.933989+0000", 2024-09-21T21:40:40.259 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001352699 2024-09-21T21:40:40.259 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.259 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.259 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:40.259 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.934022+0000", 2024-09-21T21:40:40.259 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2829e-05 2024-09-21T21:40:40.259 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.259 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.259 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:40.259 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.934045+0000", 2024-09-21T21:40:40.260 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2819e-05 2024-09-21T21:40:40.260 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:40.260 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:40.260 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:40.260 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.260 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.260 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4183.0:155 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T21:40:40.260 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:20.296102+0000", 2024-09-21T21:40:40.260 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 19.868426867, 2024-09-21T21:40:40.260 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0061463489999999997, 2024-09-21T21:40:40.260 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:40.260 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:40.260 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:40.260 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:40.261 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4183", 2024-09-21T21:40:40.261 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.122:0/3913072034", 2024-09-21T21:40:40.261 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 155 2024-09-21T21:40:40.261 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.261 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:40.261 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.261 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:40.261 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:20.296102+0000", 2024-09-21T21:40:40.261 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:40.261 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.261 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.261 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:40.261 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:20.296102+0000", 2024-09-21T21:40:40.261 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:40.262 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.262 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.262 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:40.262 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:20.296102+0000", 2024-09-21T21:40:40.262 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4299999999999999e-07 2024-09-21T21:40:40.262 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.262 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.262 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:40.262 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:20.296130+0000", 2024-09-21T21:40:40.262 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7894000000000001e-05 2024-09-21T21:40:40.262 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.262 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.262 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:40.263 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:20.296139+0000", 2024-09-21T21:40:40.263 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9549999999999998e-06 2024-09-21T21:40:40.263 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.263 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.263 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:40.263 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:20.296142+0000", 2024-09-21T21:40:40.263 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6339999999999999e-06 2024-09-21T21:40:40.263 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.263 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.263 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:40.263 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:20.296913+0000", 2024-09-21T21:40:40.263 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00077061999999999996 2024-09-21T21:40:40.263 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.263 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.264 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:40.264 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:20.296993+0000", 2024-09-21T21:40:40.264 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0096e-05 2024-09-21T21:40:40.264 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.264 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.264 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:40:40.264 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:20.297079+0000", 2024-09-21T21:40:40.264 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5483999999999997e-05 2024-09-21T21:40:40.264 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.264 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.264 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:40.264 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:20.298640+0000", 2024-09-21T21:40:40.264 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001560996 2024-09-21T21:40:40.264 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.264 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.265 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:40.265 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:20.302160+0000", 2024-09-21T21:40:40.265 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0035209310000000001 2024-09-21T21:40:40.265 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.265 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.265 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:40.265 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:20.302210+0000", 2024-09-21T21:40:40.265 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9187000000000002e-05 2024-09-21T21:40:40.265 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.265 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.265 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:40.265 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:20.302248+0000", 2024-09-21T21:40:40.265 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8309000000000002e-05 2024-09-21T21:40:40.265 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:40.266 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:40.266 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:40.266 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.266 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.266 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4183.0:156 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T21:40:40.266 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:20.296151+0000", 2024-09-21T21:40:40.266 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 19.868377944999999, 2024-09-21T21:40:40.266 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0061646349999999999, 2024-09-21T21:40:40.266 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:40.266 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:40.266 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:40.266 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:40.266 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4183", 2024-09-21T21:40:40.266 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.122:0/3913072034", 2024-09-21T21:40:40.267 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 156 2024-09-21T21:40:40.267 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.267 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:40.267 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.267 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:40.267 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:20.296151+0000", 2024-09-21T21:40:40.267 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:40.267 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.267 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.267 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:40.267 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:20.296151+0000", 2024-09-21T21:40:40.267 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:40.267 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.267 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.268 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:40.268 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:20.296151+0000", 2024-09-21T21:40:40.268 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0600000000000001e-07 2024-09-21T21:40:40.268 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.268 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.268 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:40.268 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:20.296153+0000", 2024-09-21T21:40:40.268 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2670000000000001e-06 2024-09-21T21:40:40.268 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.268 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.268 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:40.268 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:20.296157+0000", 2024-09-21T21:40:40.268 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7459999999999998e-06 2024-09-21T21:40:40.268 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.269 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.269 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:40.269 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:20.296160+0000", 2024-09-21T21:40:40.269 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.667e-06 2024-09-21T21:40:40.269 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.269 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.269 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:40.269 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:20.297385+0000", 2024-09-21T21:40:40.269 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012259549999999999 2024-09-21T21:40:40.269 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.269 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.269 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:40.269 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:20.297438+0000", 2024-09-21T21:40:40.269 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.215e-05 2024-09-21T21:40:40.269 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.270 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.270 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:40:40.270 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:20.297506+0000", 2024-09-21T21:40:40.270 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8467999999999996e-05 2024-09-21T21:40:40.270 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.270 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.270 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:40.270 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:20.298675+0000", 2024-09-21T21:40:40.270 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00116866 2024-09-21T21:40:40.270 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.270 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.270 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:40.270 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:20.302257+0000", 2024-09-21T21:40:40.270 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0035822089999999998 2024-09-21T21:40:40.271 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.271 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.271 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:40.271 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:20.302296+0000", 2024-09-21T21:40:40.271 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8890000000000002e-05 2024-09-21T21:40:40.271 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.271 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.271 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:40.271 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:20.302315+0000", 2024-09-21T21:40:40.271 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9317000000000002e-05 2024-09-21T21:40:40.271 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:40.271 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:40.271 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:40.271 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.272 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.272 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4183.0:157 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T21:40:40.272 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:20.296163+0000", 2024-09-21T21:40:40.272 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 19.868365038, 2024-09-21T21:40:40.272 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0061922590000000003, 2024-09-21T21:40:40.272 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:40.272 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:40.272 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:40.272 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:40.272 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4183", 2024-09-21T21:40:40.272 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.122:0/3913072034", 2024-09-21T21:40:40.272 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 157 2024-09-21T21:40:40.272 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.272 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:40.272 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.273 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:40.273 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:20.296163+0000", 2024-09-21T21:40:40.273 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:40.273 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.273 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.273 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:40.273 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:20.296163+0000", 2024-09-21T21:40:40.273 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:40.273 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.273 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.273 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:40.273 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:20.296164+0000", 2024-09-21T21:40:40.273 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.22e-07 2024-09-21T21:40:40.273 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.273 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.274 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:40.274 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:20.296169+0000", 2024-09-21T21:40:40.274 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1970000000000004e-06 2024-09-21T21:40:40.274 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.274 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.274 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:40.274 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:20.296172+0000", 2024-09-21T21:40:40.274 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.298e-06 2024-09-21T21:40:40.274 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.274 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.274 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:40.274 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:20.296175+0000", 2024-09-21T21:40:40.274 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6240000000000002e-06 2024-09-21T21:40:40.274 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.275 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.275 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:40.275 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:20.297870+0000", 2024-09-21T21:40:40.275 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001695149 2024-09-21T21:40:40.275 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.275 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.275 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:40.275 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:20.297916+0000", 2024-09-21T21:40:40.275 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6078999999999997e-05 2024-09-21T21:40:40.275 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.275 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.275 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:40:40.275 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:20.297979+0000", 2024-09-21T21:40:40.275 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3099e-05 2024-09-21T21:40:40.276 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.276 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.276 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:40.276 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:20.299010+0000", 2024-09-21T21:40:40.276 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001030984 2024-09-21T21:40:40.276 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.276 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.276 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:40.276 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:20.302319+0000", 2024-09-21T21:40:40.276 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0033092880000000001 2024-09-21T21:40:40.276 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.276 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.276 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:40.276 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:20.302340+0000", 2024-09-21T21:40:40.276 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0285999999999999e-05 2024-09-21T21:40:40.276 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.277 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.277 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:40.277 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:20.302356+0000", 2024-09-21T21:40:40.277 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6033e-05 2024-09-21T21:40:40.277 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:40.277 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:40.277 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:40.277 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.277 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.277 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4183.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T21:40:40.277 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:20.296180+0000", 2024-09-21T21:40:40.277 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 19.868348060999999, 2024-09-21T21:40:40.277 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0062147499999999998, 2024-09-21T21:40:40.277 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:40.277 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:40.278 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:40.278 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:40.278 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4183", 2024-09-21T21:40:40.278 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.122:0/3913072034", 2024-09-21T21:40:40.278 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 158 2024-09-21T21:40:40.278 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.278 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:40.278 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.278 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:40.278 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:20.296180+0000", 2024-09-21T21:40:40.278 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:40.278 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.278 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.278 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:40.278 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:20.296180+0000", 2024-09-21T21:40:40.279 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:40.279 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.279 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.279 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:40.279 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:20.296181+0000", 2024-09-21T21:40:40.279 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3999999999999998e-07 2024-09-21T21:40:40.279 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.279 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.279 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:40.279 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:20.296213+0000", 2024-09-21T21:40:40.279 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2474000000000001e-05 2024-09-21T21:40:40.279 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.279 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.279 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:40.279 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:20.296218+0000", 2024-09-21T21:40:40.280 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8600000000000001e-06 2024-09-21T21:40:40.280 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.280 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.280 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:40.280 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:20.296223+0000", 2024-09-21T21:40:40.280 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8350000000000003e-06 2024-09-21T21:40:40.280 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.280 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.280 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:40.280 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:20.298191+0000", 2024-09-21T21:40:40.280 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019678289999999999 2024-09-21T21:40:40.280 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.280 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.280 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:40.281 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:20.298233+0000", 2024-09-21T21:40:40.281 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2250999999999999e-05 2024-09-21T21:40:40.281 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.281 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.281 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:40:40.281 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:20.298300+0000", 2024-09-21T21:40:40.281 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6989999999999994e-05 2024-09-21T21:40:40.281 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.281 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.281 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:40.281 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:20.299212+0000", 2024-09-21T21:40:40.281 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00091255100000000005 2024-09-21T21:40:40.281 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.281 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.281 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:40.282 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:20.302360+0000", 2024-09-21T21:40:40.282 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0031472359999999999 2024-09-21T21:40:40.282 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.282 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.282 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:40.282 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:20.302380+0000", 2024-09-21T21:40:40.282 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0205999999999999e-05 2024-09-21T21:40:40.282 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.282 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.282 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:40.282 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:20.302395+0000", 2024-09-21T21:40:40.282 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5277999999999998e-05 2024-09-21T21:40:40.282 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:40.282 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:40.283 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:40.283 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.283 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.283 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4183.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T21:40:40.283 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:20.296228+0000", 2024-09-21T21:40:40.283 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 19.868300006999998, 2024-09-21T21:40:40.283 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0062058859999999999, 2024-09-21T21:40:40.283 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:40.283 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:40.283 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:40.283 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:40.283 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4183", 2024-09-21T21:40:40.283 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.122:0/3913072034", 2024-09-21T21:40:40.283 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 159 2024-09-21T21:40:40.284 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.284 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:40.284 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.284 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:40.284 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:20.296228+0000", 2024-09-21T21:40:40.284 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:40.284 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.284 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.284 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:40.284 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:20.296228+0000", 2024-09-21T21:40:40.284 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:40.284 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.284 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.284 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:40.284 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:20.296229+0000", 2024-09-21T21:40:40.285 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8700000000000002e-07 2024-09-21T21:40:40.285 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.285 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.285 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:40.285 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:20.296231+0000", 2024-09-21T21:40:40.285 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.142e-06 2024-09-21T21:40:40.285 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.285 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.285 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:40.285 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:20.296235+0000", 2024-09-21T21:40:40.285 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7309999999999999e-06 2024-09-21T21:40:40.285 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.285 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.285 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:40.285 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:20.296237+0000", 2024-09-21T21:40:40.285 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6759999999999999e-06 2024-09-21T21:40:40.286 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.286 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.286 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:40.286 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:20.298695+0000", 2024-09-21T21:40:40.286 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0024579369999999999 2024-09-21T21:40:40.286 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.286 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.286 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:40.286 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:20.298737+0000", 2024-09-21T21:40:40.286 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1343e-05 2024-09-21T21:40:40.286 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.286 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.286 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:40:40.286 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:20.298803+0000", 2024-09-21T21:40:40.286 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6458000000000006e-05 2024-09-21T21:40:40.287 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.287 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.287 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:40.287 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:20.299937+0000", 2024-09-21T21:40:40.287 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001133474 2024-09-21T21:40:40.287 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.287 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.287 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:40.287 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:20.302399+0000", 2024-09-21T21:40:40.287 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0024622630000000001 2024-09-21T21:40:40.287 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.287 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.287 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:40.287 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:20.302418+0000", 2024-09-21T21:40:40.287 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9613999999999999e-05 2024-09-21T21:40:40.288 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:40.288 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:40.288 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:40.288 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:20.302434+0000", 2024-09-21T21:40:40.288 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5960999999999998e-05 2024-09-21T21:40:40.288 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:40.288 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:40.288 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:40.288 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:40.288 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:40.288 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:40:40.290 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T21:40:40.290 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:40:40.288+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:40:40.390 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:40:40.389+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:40:40.395 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T21:40:40.395 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:40:40.395 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:40:40.395 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-21T21:40:40.396 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:40:40.404 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T21:40:40.491 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:40:40.489+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:40:40.507 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:40:40.507 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:40:40.507 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:40:40.507 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:40:40.507 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:40:40.515 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T21:40:40.516 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0: {} 2024-09-21T21:40:40.516 INFO:teuthology.orchestra.run.smithi081.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T21:40:40.532 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1: {} 2024-09-21T21:40:40.532 INFO:teuthology.orchestra.run.smithi081.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T21:40:40.547 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2: {} 2024-09-21T21:40:40.548 INFO:teuthology.orchestra.run.smithi081.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T21:40:40.564 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3: {} 2024-09-21T21:40:40.564 INFO:teuthology.orchestra.run.smithi081.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T21:40:40.580 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4: {} 2024-09-21T21:40:40.580 INFO:teuthology.orchestra.run.smithi081.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T21:40:40.591 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:40:40.590+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:40:40.597 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5: {} 2024-09-21T21:40:40.597 INFO:teuthology.orchestra.run.smithi081.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T21:40:40.659 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:40.659 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:40.667 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T21:40:40.692 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:40:40.691+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:40:40.770 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:40.770 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:40.779 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T21:40:40.793 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:40:40.792+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:40:40.882 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:40.883 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:40.891 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T21:40:40.894 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:40:40.891+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:40:40.994 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:40:40.992+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:40:40.995 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:40.995 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:41.003 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T21:40:41.059 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-21T21:40:41.059 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T21:40:41.060 INFO:tasks.thrashosds.thrasher:inject_pause on osd.2 2024-09-21T21:40:41.060 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-21T21:40:41.060 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-21T21:40:41.060 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok config set filestore_inject_stall 3 2024-09-21T21:40:41.103 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:40:41.102+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:40:41.108 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:41.109 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:41.118 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T21:40:41.164 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:40:41.164 INFO:teuthology.orchestra.run.smithi081.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-21T21:40:41.164 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:40:41.214 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:40:41.213+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:40:41.263 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:41.263 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:41.273 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T21:40:41.274 INFO:teuthology.orchestra.run.smithi164.stdout:skipping zero size read 2024-09-21T21:40:41.274 INFO:teuthology.orchestra.run.smithi164.stdout:skipping zero size read 2024-09-21T21:40:41.274 INFO:teuthology.orchestra.run.smithi164.stdout:skipping zero size read 2024-09-21T21:40:41.275 INFO:teuthology.orchestra.run.smithi164.stdout:skipping zero size read 2024-09-21T21:40:41.296 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:40:41.294+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:40:41.301 INFO:teuthology.orchestra.run.smithi164.stdout:8 write 0x9790800 thru 0x979319d (0x299e bytes) 2024-09-21T21:40:41.301 INFO:teuthology.orchestra.run.smithi164.stdout:10 write 0x8461e00 thru 0x846806d (0x626e bytes) 2024-09-21T21:40:41.322 INFO:teuthology.orchestra.run.smithi164.stdout:13 write 0xab16400 thru 0xab198c3 (0x34c4 bytes) 2024-09-21T21:40:41.329 INFO:teuthology.orchestra.run.smithi164.stdout:14 read 0x3584400 thru 0x35900d6 (0xbcd7 bytes) 2024-09-21T21:40:41.329 INFO:teuthology.orchestra.run.smithi164.stdout:15 write 0x4fa7800 thru 0x4fb4c37 (0xd438 bytes) 2024-09-21T21:40:41.330 INFO:teuthology.orchestra.run.smithi164.stdout:16 read 0x8ec7e00 thru 0x8ed5bf0 (0xddf1 bytes) 2024-09-21T21:40:41.330 INFO:teuthology.orchestra.run.smithi164.stdout:truncating to largest ever: 0x98dfe00 2024-09-21T21:40:41.331 INFO:teuthology.orchestra.run.smithi164.stdout:17 trunc from 0xab19a00 to 0x98dfe00 2024-09-21T21:40:41.352 INFO:teuthology.orchestra.run.smithi164.stdout:18 write 0x9e6000 thru 0x9ec17b (0x617c bytes) 2024-09-21T21:40:41.354 INFO:teuthology.orchestra.run.smithi164.stdout:22 write 0x3bedc00 thru 0x3bf48d5 (0x6cd6 bytes) 2024-09-21T21:40:41.354 INFO:teuthology.orchestra.run.smithi164.stdout:23 read 0x1d5d400 thru 0x1d60b50 (0x3751 bytes) 2024-09-21T21:40:41.354 INFO:teuthology.orchestra.run.smithi164.stdout:26 write 0x9057000 thru 0x905c400 (0x5401 bytes) 2024-09-21T21:40:41.372 INFO:teuthology.orchestra.run.smithi164.stdout:27 write 0xae78200 thru 0xae7fcc9 (0x7aca bytes) 2024-09-21T21:40:41.372 INFO:teuthology.orchestra.run.smithi164.stdout:28 write 0x2d09200 thru 0x2d18cde (0xfadf bytes) 2024-09-21T21:40:41.373 INFO:teuthology.orchestra.run.smithi164.stdout:29 punch from 0x6db400 to 0x6e0600, (0x5200 bytes) 2024-09-21T21:40:41.373 INFO:teuthology.orchestra.run.smithi164.stdout:31 read 0x1042600 thru 0x104b25c (0x8c5d bytes) 2024-09-21T21:40:41.373 INFO:teuthology.orchestra.run.smithi164.stdout:32 read 0xe88c00 thru 0xe95918 (0xcd19 bytes) 2024-09-21T21:40:41.373 INFO:teuthology.orchestra.run.smithi164.stdout:34 read 0x29bb000 thru 0x29bc5be (0x15bf bytes) 2024-09-21T21:40:41.373 INFO:teuthology.orchestra.run.smithi164.stdout:37 read 0x9498200 thru 0x94a3f7c (0xbd7d bytes) 2024-09-21T21:40:41.374 INFO:teuthology.orchestra.run.smithi164.stdout:39 read 0x7e83600 thru 0x7e88804 (0x5205 bytes) 2024-09-21T21:40:41.374 INFO:teuthology.orchestra.run.smithi164.stdout:40 punch from 0x7603800 to 0x760d000, (0x9800 bytes) 2024-09-21T21:40:41.384 INFO:teuthology.orchestra.run.smithi164.stdout:41 trunc from 0xae7fe00 to 0x20d7c00 2024-09-21T21:40:41.396 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:40:41.394+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:40:41.415 INFO:teuthology.orchestra.run.smithi164.stdout:42 read 0x136a800 thru 0x136c6f3 (0x1ef4 bytes) 2024-09-21T21:40:41.429 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T21:40:41.430 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:41.430 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:41.430 INFO:teuthology.orchestra.run.smithi164.stdout:43 write 0x5e3d600 thru 0x5e495e3 (0xbfe4 bytes) 2024-09-21T21:40:41.443 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T21:40:41.444 INFO:teuthology.orchestra.run.smithi164.stdout:44 write 0x60ef400 thru 0x60fb1ee (0xbdef bytes) 2024-09-21T21:40:41.445 INFO:teuthology.orchestra.run.smithi164.stdout:45 punch from 0x30eac00 to 0x30f1c00, (0x7000 bytes) 2024-09-21T21:40:41.445 INFO:teuthology.orchestra.run.smithi164.stdout:47 read 0xa08a00 thru 0xa0f731 (0x6d32 bytes) 2024-09-21T21:40:41.446 INFO:teuthology.orchestra.run.smithi164.stdout:52 read 0x5801c00 thru 0x58066ce (0x4acf bytes) 2024-09-21T21:40:41.467 INFO:teuthology.orchestra.run.smithi164.stdout:55 write 0x78e9200 thru 0x78f6f5e (0xdd5f bytes) 2024-09-21T21:40:41.467 INFO:teuthology.orchestra.run.smithi164.stdout:56 read 0x3d9de00 thru 0x3da67c8 (0x89c9 bytes) 2024-09-21T21:40:41.467 INFO:teuthology.orchestra.run.smithi164.stdout:57 read 0x123de00 thru 0x1244912 (0x6b13 bytes) 2024-09-21T21:40:41.467 INFO:teuthology.orchestra.run.smithi164.stdout:58 write 0x3065800 thru 0x30745d7 (0xedd8 bytes) 2024-09-21T21:40:41.468 INFO:teuthology.orchestra.run.smithi164.stdout:61 punch from 0x1701800 to 0x1703800, (0x2000 bytes) 2024-09-21T21:40:41.476 INFO:teuthology.orchestra.run.smithi164.stdout:65 trunc from 0x78f7000 to 0xdf1600 2024-09-21T21:40:41.502 INFO:teuthology.orchestra.run.smithi164.stdout:66 punch from 0xc9b000 to 0xca5400, (0xa400 bytes) 2024-09-21T21:40:41.502 INFO:teuthology.orchestra.run.smithi164.stdout:67 read 0x74b800 thru 0x75a1de (0xe9df bytes) 2024-09-21T21:40:41.526 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:40:41.525+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:40:41.533 INFO:teuthology.orchestra.run.smithi164.stdout:70 write 0xc866600 thru 0xc86af8f (0x4990 bytes) 2024-09-21T21:40:41.543 INFO:teuthology.orchestra.run.smithi164.stdout:71 trunc from 0xc86b000 to 0x3dfcc00 2024-09-21T21:40:41.570 INFO:teuthology.orchestra.run.smithi164.stdout:truncating to largest ever: 0xa4c9a00 2024-09-21T21:40:41.577 INFO:teuthology.orchestra.run.smithi164.stdout:74 trunc from 0x3dfcc00 to 0xa4c9a00 2024-09-21T21:40:41.594 INFO:teuthology.orchestra.run.smithi164.stdout:75 punch from 0x2a41a00 to 0x2a47600, (0x5c00 bytes) 2024-09-21T21:40:41.594 INFO:teuthology.orchestra.run.smithi164.stdout:76 write 0xfa9c00 thru 0xfb778a (0xdb8b bytes) 2024-09-21T21:40:41.594 INFO:teuthology.orchestra.run.smithi164.stdout:77 write 0x3230e00 thru 0x3240860 (0xfa61 bytes) 2024-09-21T21:40:41.596 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:41.596 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:41.598 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:40:41.596+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:40:41.606 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T21:40:41.616 INFO:teuthology.orchestra.run.smithi164.stdout:78 write 0xad45a00 thru 0xad50362 (0xa963 bytes) 2024-09-21T21:40:41.616 INFO:teuthology.orchestra.run.smithi164.stdout:79 read 0x7e19c00 thru 0x7e21d81 (0x8182 bytes) 2024-09-21T21:40:41.616 INFO:teuthology.orchestra.run.smithi164.stdout:81 punch from 0x7b97800 to 0x7b9bc00, (0x4400 bytes) 2024-09-21T21:40:41.616 INFO:teuthology.orchestra.run.smithi164.stdout:82 read 0x9822e00 thru 0x98302a2 (0xd4a3 bytes) 2024-09-21T21:40:41.616 INFO:teuthology.orchestra.run.smithi164.stdout:83 read 0x45d4800 thru 0x45e3336 (0xeb37 bytes) 2024-09-21T21:40:41.620 INFO:teuthology.orchestra.run.smithi164.stdout:84 trunc from 0xad50400 to 0x7e01600 2024-09-21T21:40:41.635 INFO:teuthology.orchestra.run.smithi164.stdout:86 read 0x3a7f800 thru 0x3a819a9 (0x21aa bytes) 2024-09-21T21:40:41.641 INFO:teuthology.orchestra.run.smithi164.stdout:87 trunc from 0x7e01600 to 0x3c29200 2024-09-21T21:40:41.662 INFO:teuthology.orchestra.run.smithi164.stdout:89 trunc from 0x3c29200 to 0x2b68000 2024-09-21T21:40:41.682 INFO:teuthology.orchestra.run.smithi164.stdout:90 trunc from 0x2b68000 to 0x49e4c00 2024-09-21T21:40:41.699 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:40:41.697+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:40:41.713 INFO:teuthology.orchestra.run.smithi164.stdout:92 write 0x7791e00 thru 0x7793a4f (0x1c50 bytes) 2024-09-21T21:40:41.714 INFO:teuthology.orchestra.run.smithi164.stdout:93 read 0x13d6800 thru 0x13e08fc (0xa0fd bytes) 2024-09-21T21:40:41.715 INFO:teuthology.orchestra.run.smithi164.stdout:96 punch from 0x475a00 to 0x47e800, (0x8e00 bytes) 2024-09-21T21:40:41.715 INFO:teuthology.orchestra.run.smithi164.stdout:97 read 0x26e200 thru 0x270909 (0x270a bytes) 2024-09-21T21:40:41.715 INFO:teuthology.orchestra.run.smithi164.stdout:101 write 0x2982800 thru 0x2983e80 (0x1681 bytes) 2024-09-21T21:40:41.716 INFO:teuthology.orchestra.run.smithi164.stdout:103 write 0x1183e00 thru 0x118f243 (0xb444 bytes) 2024-09-21T21:40:41.716 INFO:teuthology.orchestra.run.smithi164.stdout:105 read 0x4f28200 thru 0x4f2837d (0x17e bytes) 2024-09-21T21:40:41.716 INFO:teuthology.orchestra.run.smithi164.stdout:106 read 0x58e6400 thru 0x58ec8f1 (0x64f2 bytes) 2024-09-21T21:40:41.716 INFO:teuthology.orchestra.run.smithi164.stdout:truncating to largest ever: 0xaa7a200 2024-09-21T21:40:41.719 INFO:teuthology.orchestra.run.smithi164.stdout:108 trunc from 0x7793c00 to 0xaa7a200 2024-09-21T21:40:41.746 INFO:teuthology.orchestra.run.smithi164.stdout:109 trunc from 0xaa7a200 to 0x342e00 2024-09-21T21:40:41.760 INFO:teuthology.orchestra.run.smithi164.stdout:111 read 0x2c7a00 thru 0x2c9792 (0x1d93 bytes) 2024-09-21T21:40:41.763 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:41.763 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:41.772 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T21:40:41.794 INFO:teuthology.orchestra.run.smithi164.stdout:113 write 0xea0f000 thru 0xea1e2dd (0xf2de bytes) 2024-09-21T21:40:41.801 INFO:teuthology.orchestra.run.smithi164.stdout:114 read 0xce0a00 thru 0xce3201 (0x2802 bytes) 2024-09-21T21:40:41.801 INFO:teuthology.orchestra.run.smithi164.stdout:116 read 0xb596800 thru 0xb5a60e2 (0xf8e3 bytes) 2024-09-21T21:40:41.801 INFO:teuthology.orchestra.run.smithi164.stdout:117 write 0xe8ab400 thru 0xe8b7c6a (0xc86b bytes) 2024-09-21T21:40:41.801 INFO:teuthology.orchestra.run.smithi164.stdout:118 punch from 0xc596000 to 0xc599600, (0x3600 bytes) 2024-09-21T21:40:41.802 INFO:teuthology.orchestra.run.smithi164.stdout:120 write 0x32bb600 thru 0x32c6031 (0xaa32 bytes) 2024-09-21T21:40:41.802 INFO:teuthology.orchestra.run.smithi164.stdout:121 read 0x32d0400 thru 0x32db078 (0xac79 bytes) 2024-09-21T21:40:41.802 INFO:teuthology.orchestra.run.smithi164.stdout:122 write 0xbe38a00 thru 0xbe3b8d7 (0x2ed8 bytes) 2024-09-21T21:40:41.802 INFO:teuthology.orchestra.run.smithi164.stdout:123 write 0x5dadc00 thru 0x5db9866 (0xbc67 bytes) 2024-09-21T21:40:41.802 INFO:teuthology.orchestra.run.smithi164.stdout:124 read 0xf4c00 thru 0xfaf72 (0x6373 bytes) 2024-09-21T21:40:41.802 INFO:teuthology.orchestra.run.smithi164.stdout:126 write 0x1955400 thru 0x196074f (0xb350 bytes) 2024-09-21T21:40:41.802 INFO:teuthology.orchestra.run.smithi164.stdout:130 write 0x5462200 thru 0x546c9b1 (0xa7b2 bytes) 2024-09-21T21:40:41.803 INFO:teuthology.orchestra.run.smithi164.stdout:132 read 0xee4000 thru 0xee6fc8 (0x2fc9 bytes) 2024-09-21T21:40:41.803 INFO:teuthology.orchestra.run.smithi164.stdout:133 punch from 0x633d600 to 0x6349800, (0xc200 bytes) 2024-09-21T21:40:41.803 INFO:teuthology.orchestra.run.smithi164.stdout:134 write 0x20f4c00 thru 0x20fea2f (0x9e30 bytes) 2024-09-21T21:40:41.803 INFO:teuthology.orchestra.run.smithi164.stdout:135 punch from 0xdb76400 to 0xdb7d200, (0x6e00 bytes) 2024-09-21T21:40:41.803 INFO:teuthology.orchestra.run.smithi164.stdout:136 clone 1 order 21 su 1048576 sc 14 2024-09-21T21:40:41.813 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:40:41.812+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:40:41.900 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:40:41.899+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:40:41.915 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:41.915 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:41.924 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T21:40:42.001 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:40:41.999+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:40:42.065 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:42.065 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:42.074 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T21:40:42.117 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:40:42.116+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:40:42.176 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:42.177 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:42.185 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T21:40:42.203 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:40:42.201+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:40:42.303 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:40:42.302+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:40:42.328 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:42.328 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:42.337 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T21:40:42.404 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:40:42.403+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:40:42.437 INFO:teuthology.orchestra.run.smithi081.stderr:noscrub is unset 2024-09-21T21:40:42.444 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:42.444 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:42.453 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T21:40:42.505 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:40:42.503+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:40:42.557 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:42.558 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:42.566 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T21:40:42.606 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:40:42.604+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:40:42.608 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T21:40:42.707 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:40:42.706+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:40:42.710 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:42.710 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:42.719 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T21:40:42.729 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0: {} 2024-09-21T21:40:42.730 INFO:teuthology.orchestra.run.smithi081.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T21:40:42.747 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1: {} 2024-09-21T21:40:42.747 INFO:teuthology.orchestra.run.smithi081.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T21:40:42.763 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2: {} 2024-09-21T21:40:42.764 INFO:teuthology.orchestra.run.smithi081.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T21:40:42.780 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3: {} 2024-09-21T21:40:42.780 INFO:teuthology.orchestra.run.smithi081.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T21:40:42.782 INFO:teuthology.orchestra.run.smithi164.stdout:137 clone 2 order 23 su 2097152 sc 8 2024-09-21T21:40:42.799 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4: {} 2024-09-21T21:40:42.799 INFO:teuthology.orchestra.run.smithi081.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T21:40:42.808 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:40:42.807+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:40:42.816 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5: {} 2024-09-21T21:40:42.816 INFO:teuthology.orchestra.run.smithi081.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T21:40:42.825 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:40:42.825 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:40:42.825 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-21T21:40:42.825 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:40:42.834 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T21:40:42.909 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:40:42.907+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:40:42.935 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:40:42.936 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:40:42.936 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:40:42.936 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:40:42.936 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:40:42.944 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T21:40:43.010 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:40:43.009+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:40:43.088 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:40:43.088 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-21T21:40:43.088 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-21T21:40:43.088 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:40:43.088 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.088 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4385.0:7 3.1a 3:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-21T21:40:43.088 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:38.414394+0000", 2024-09-21T21:40:43.088 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.672017383, 2024-09-21T21:40:43.088 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0026021880000000001, 2024-09-21T21:40:43.088 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:43.088 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:43.089 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:43.089 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:43.089 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4385", 2024-09-21T21:40:43.089 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:40:43.089 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7 2024-09-21T21:40:43.089 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.089 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:43.089 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.089 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:43.089 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:38.414394+0000", 2024-09-21T21:40:43.089 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:43.089 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.089 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.090 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:43.090 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:38.414394+0000", 2024-09-21T21:40:43.090 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:43.090 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.090 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.090 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:43.090 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:38.414396+0000", 2024-09-21T21:40:43.090 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.821e-06 2024-09-21T21:40:43.090 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.090 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.090 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:43.090 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:38.414407+0000", 2024-09-21T21:40:43.090 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1533e-05 2024-09-21T21:40:43.091 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.091 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.091 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:43.091 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:38.414424+0000", 2024-09-21T21:40:43.091 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6810999999999999e-05 2024-09-21T21:40:43.091 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.091 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.091 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:43.091 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:38.414432+0000", 2024-09-21T21:40:43.091 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0539999999999995e-06 2024-09-21T21:40:43.091 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.091 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.091 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:43.091 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:38.414499+0000", 2024-09-21T21:40:43.092 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6797000000000004e-05 2024-09-21T21:40:43.092 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.092 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.092 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op must wait for map", 2024-09-21T21:40:43.092 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:38.414508+0000", 2024-09-21T21:40:43.092 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8969999999999997e-06 2024-09-21T21:40:43.092 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.092 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.092 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:43.092 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:38.415839+0000", 2024-09-21T21:40:43.092 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0013311270000000001 2024-09-21T21:40:43.092 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.092 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.092 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:43.092 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:38.415935+0000", 2024-09-21T21:40:43.092 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5617000000000003e-05 2024-09-21T21:40:43.093 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.093 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.093 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2024-09-21T21:40:43.093 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:38.416029+0000", 2024-09-21T21:40:43.093 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.4691000000000005e-05 2024-09-21T21:40:43.093 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.093 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.093 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:43.093 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:38.416487+0000", 2024-09-21T21:40:43.093 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00045795999999999999 2024-09-21T21:40:43.093 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.093 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.093 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:43.093 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:38.416944+0000", 2024-09-21T21:40:43.094 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00045617699999999999 2024-09-21T21:40:43.094 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.094 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.094 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:43.094 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:38.416973+0000", 2024-09-21T21:40:43.094 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.942e-05 2024-09-21T21:40:43.094 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.094 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.094 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:43.094 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:38.416996+0000", 2024-09-21T21:40:43.094 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3283000000000001e-05 2024-09-21T21:40:43.094 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:43.094 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:43.094 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:43.095 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.095 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.095 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop_reply(client.4385.0:7 3.1a e23/21 ondisk, result = 0)", 2024-09-21T21:40:43.095 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:38.416889+0000", 2024-09-21T21:40:43.095 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.669522508, 2024-09-21T21:40:43.095 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00011225, 2024-09-21T21:40:43.095 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:43.095 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:43.095 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:40:43.095 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:43.095 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.095 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:43.096 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:38.416889+0000", 2024-09-21T21:40:43.096 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:43.096 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.096 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.096 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:43.096 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:38.416889+0000", 2024-09-21T21:40:43.096 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:43.096 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.096 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.096 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:43.096 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:38.416890+0000", 2024-09-21T21:40:43.096 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7599999999999997e-07 2024-09-21T21:40:43.097 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.097 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.097 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:43.097 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:38.416894+0000", 2024-09-21T21:40:43.097 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9439999999999998e-06 2024-09-21T21:40:43.097 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.097 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.097 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:43.097 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:38.416902+0000", 2024-09-21T21:40:43.097 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9410000000000007e-06 2024-09-21T21:40:43.097 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.097 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.097 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:43.097 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:38.416907+0000", 2024-09-21T21:40:43.097 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6130000000000003e-06 2024-09-21T21:40:43.098 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.098 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.098 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:43.098 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:38.416935+0000", 2024-09-21T21:40:43.098 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8231000000000001e-05 2024-09-21T21:40:43.098 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.098 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.098 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:43.098 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:38.416941+0000", 2024-09-21T21:40:43.098 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2360000000000004e-06 2024-09-21T21:40:43.098 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.098 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.098 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:43.099 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:38.417001+0000", 2024-09-21T21:40:43.099 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0609000000000002e-05 2024-09-21T21:40:43.099 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:43.099 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:43.099 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:43.099 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.099 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.099 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4385.0:9 3.1a 3:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-21T21:40:43.099 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:39.417430+0000", 2024-09-21T21:40:43.099 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 3.6689814419999998, 2024-09-21T21:40:43.099 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0033823809999999998, 2024-09-21T21:40:43.099 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:43.099 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:43.099 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:43.100 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:43.100 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4385", 2024-09-21T21:40:43.100 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:40:43.100 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 9 2024-09-21T21:40:43.100 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.100 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:43.100 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.100 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:43.100 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.417430+0000", 2024-09-21T21:40:43.100 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:43.100 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.100 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.101 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:43.101 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.417430+0000", 2024-09-21T21:40:43.101 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:43.101 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.101 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.101 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:43.101 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.417432+0000", 2024-09-21T21:40:43.101 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.689e-06 2024-09-21T21:40:43.101 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.101 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.101 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:43.101 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.417440+0000", 2024-09-21T21:40:43.101 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3310000000000004e-06 2024-09-21T21:40:43.101 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.102 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.102 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:43.102 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.417456+0000", 2024-09-21T21:40:43.102 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6141999999999999e-05 2024-09-21T21:40:43.102 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.102 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.102 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:43.102 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.417476+0000", 2024-09-21T21:40:43.102 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0228999999999999e-05 2024-09-21T21:40:43.102 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.102 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.102 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:43.103 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.417540+0000", 2024-09-21T21:40:43.103 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3750000000000005e-05 2024-09-21T21:40:43.103 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.103 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.103 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op must wait for map", 2024-09-21T21:40:43.103 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.417545+0000", 2024-09-21T21:40:43.103 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0390000000000004e-06 2024-09-21T21:40:43.103 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.103 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.103 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:43.103 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.419488+0000", 2024-09-21T21:40:43.103 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001942966 2024-09-21T21:40:43.103 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.103 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.103 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:43.104 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.419591+0000", 2024-09-21T21:40:43.104 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000102877 2024-09-21T21:40:43.104 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.104 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.104 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2024-09-21T21:40:43.104 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.419719+0000", 2024-09-21T21:40:43.104 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012808000000000001 2024-09-21T21:40:43.104 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.104 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.104 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:43.104 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.420710+0000", 2024-09-21T21:40:43.105 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00099065999999999998 2024-09-21T21:40:43.105 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.105 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.105 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:43.105 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.420735+0000", 2024-09-21T21:40:43.105 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4709e-05 2024-09-21T21:40:43.105 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.105 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.105 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:43.105 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.420776+0000", 2024-09-21T21:40:43.105 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1270000000000003e-05 2024-09-21T21:40:43.105 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.105 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.105 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:43.105 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.420812+0000", 2024-09-21T21:40:43.106 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6638999999999999e-05 2024-09-21T21:40:43.106 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:43.106 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:43.106 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:43.106 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.106 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.106 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop_reply(client.4385.0:9 3.1a e24/21 ondisk, result = 0)", 2024-09-21T21:40:43.106 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:39.420612+0000", 2024-09-21T21:40:43.106 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 3.6657996079999999, 2024-09-21T21:40:43.106 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00011496499999999999, 2024-09-21T21:40:43.106 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:43.106 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:43.107 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:40:43.107 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:43.107 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.107 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:43.107 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.420612+0000", 2024-09-21T21:40:43.107 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:43.107 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.107 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.107 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:43.107 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.420612+0000", 2024-09-21T21:40:43.107 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:43.107 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.107 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.107 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:43.108 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.420613+0000", 2024-09-21T21:40:43.108 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8299999999999996e-07 2024-09-21T21:40:43.108 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.108 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.108 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:43.108 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.420620+0000", 2024-09-21T21:40:43.108 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2790000000000001e-06 2024-09-21T21:40:43.108 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.108 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.108 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:43.108 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.420634+0000", 2024-09-21T21:40:43.108 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.411e-05 2024-09-21T21:40:43.108 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.108 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.109 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:43.109 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.420643+0000", 2024-09-21T21:40:43.109 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.1959999999999997e-06 2024-09-21T21:40:43.109 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.109 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.109 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:43.109 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.420690+0000", 2024-09-21T21:40:43.109 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6660999999999999e-05 2024-09-21T21:40:43.109 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.109 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.109 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:43.109 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.420703+0000", 2024-09-21T21:40:43.109 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2627e-05 2024-09-21T21:40:43.110 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.110 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.110 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:43.110 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.420727+0000", 2024-09-21T21:40:43.110 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4309000000000001e-05 2024-09-21T21:40:43.110 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:43.110 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:43.110 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:43.110 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.110 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.110 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(client.4385.0:10 3.1c e24/21 3:3831950c:::rbd_directory:head v 24'1, mlcod=0'0)", 2024-09-21T21:40:43.110 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:39.424347+0000", 2024-09-21T21:40:43.110 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 3.662064639, 2024-09-21T21:40:43.110 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00077297300000000004, 2024-09-21T21:40:43.110 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:43.111 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:43.111 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:43.111 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:43.111 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.111 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:43.111 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.424347+0000", 2024-09-21T21:40:43.111 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:43.111 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.111 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.111 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:43.111 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.424347+0000", 2024-09-21T21:40:43.111 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:43.111 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.111 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.112 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:43.112 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.424347+0000", 2024-09-21T21:40:43.112 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.39e-07 2024-09-21T21:40:43.112 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.112 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.112 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:43.112 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.424353+0000", 2024-09-21T21:40:43.112 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2090000000000002e-06 2024-09-21T21:40:43.112 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.112 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.112 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:43.112 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.424372+0000", 2024-09-21T21:40:43.112 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.8221999999999999e-05 2024-09-21T21:40:43.112 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.112 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.112 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:43.113 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.424381+0000", 2024-09-21T21:40:43.113 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.7210000000000006e-06 2024-09-21T21:40:43.113 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.113 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.113 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:43.113 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.424459+0000", 2024-09-21T21:40:43.113 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8044e-05 2024-09-21T21:40:43.113 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.113 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.113 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:43.113 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.424489+0000", 2024-09-21T21:40:43.113 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9223000000000001e-05 2024-09-21T21:40:43.113 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.113 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.113 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:43.114 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.425066+0000", 2024-09-21T21:40:43.114 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00057740299999999999 2024-09-21T21:40:43.114 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.114 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.114 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:43.114 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.425120+0000", 2024-09-21T21:40:43.114 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3811999999999999e-05 2024-09-21T21:40:43.114 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:43.114 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:43.114 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:43.114 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.114 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.114 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(client.4385.0:13 3.a e24/21 3:54765c8d:::rbd_object_map.1121c53dac8e:head v 24'1, mlcod=0'0)", 2024-09-21T21:40:43.114 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:39.433840+0000", 2024-09-21T21:40:43.114 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 3.6525719269999999, 2024-09-21T21:40:43.115 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00074857299999999999, 2024-09-21T21:40:43.115 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:43.115 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:43.115 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:43.115 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:43.115 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.115 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:43.115 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.433840+0000", 2024-09-21T21:40:43.115 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:43.115 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.115 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.115 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:43.115 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.433840+0000", 2024-09-21T21:40:43.115 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:43.115 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.115 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.115 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:43.116 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.433840+0000", 2024-09-21T21:40:43.116 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1299999999999999e-07 2024-09-21T21:40:43.116 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.116 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.116 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:43.116 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.433844+0000", 2024-09-21T21:40:43.116 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7210000000000002e-06 2024-09-21T21:40:43.116 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.116 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.116 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:43.116 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.433856+0000", 2024-09-21T21:40:43.116 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1433999999999999e-05 2024-09-21T21:40:43.116 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.116 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.116 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:43.117 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.433874+0000", 2024-09-21T21:40:43.117 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7615999999999999e-05 2024-09-21T21:40:43.117 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.117 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.117 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:43.117 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.433935+0000", 2024-09-21T21:40:43.117 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1341999999999996e-05 2024-09-21T21:40:43.117 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.117 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.117 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:43.117 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.433957+0000", 2024-09-21T21:40:43.117 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2456e-05 2024-09-21T21:40:43.117 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.117 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.117 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:43.117 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.434534+0000", 2024-09-21T21:40:43.118 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00057707600000000004 2024-09-21T21:40:43.118 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.118 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.118 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:43.118 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.434588+0000", 2024-09-21T21:40:43.118 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3715000000000003e-05 2024-09-21T21:40:43.118 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:43.118 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:43.118 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:43.118 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.118 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.118 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4385.0:23 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e24)", 2024-09-21T21:40:43.118 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:39.442135+0000", 2024-09-21T21:40:43.118 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 3.644276289, 2024-09-21T21:40:43.119 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000341789, 2024-09-21T21:40:43.119 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:43.119 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:43.119 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:40:43.119 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:43.119 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4385", 2024-09-21T21:40:43.119 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:40:43.119 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 23 2024-09-21T21:40:43.119 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.119 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:43.119 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.119 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:43.119 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.442135+0000", 2024-09-21T21:40:43.119 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:43.120 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.120 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.120 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:43.120 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.442135+0000", 2024-09-21T21:40:43.120 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:43.120 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.120 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.120 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:43.120 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.442137+0000", 2024-09-21T21:40:43.120 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.716e-06 2024-09-21T21:40:43.120 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.120 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.120 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:43.120 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.442160+0000", 2024-09-21T21:40:43.121 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2850999999999999e-05 2024-09-21T21:40:43.121 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.121 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.121 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:43.121 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.442173+0000", 2024-09-21T21:40:43.121 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3208e-05 2024-09-21T21:40:43.121 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.121 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.121 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:43.121 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.442196+0000", 2024-09-21T21:40:43.121 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3337e-05 2024-09-21T21:40:43.121 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.121 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.121 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:43.121 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.442260+0000", 2024-09-21T21:40:43.122 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3917999999999993e-05 2024-09-21T21:40:43.122 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.122 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.122 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:43.122 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.442354+0000", 2024-09-21T21:40:43.122 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3467999999999993e-05 2024-09-21T21:40:43.122 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.122 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.122 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:43.122 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.442477+0000", 2024-09-21T21:40:43.122 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012329100000000001 2024-09-21T21:40:43.122 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:43.122 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:43.122 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:43.122 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.123 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.123 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4400.0:10 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e24)", 2024-09-21T21:40:43.123 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:39.592695+0000", 2024-09-21T21:40:43.123 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 3.4937162669999999, 2024-09-21T21:40:43.123 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00026781800000000002, 2024-09-21T21:40:43.123 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:43.123 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:43.123 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:40:43.123 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:43.123 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4400", 2024-09-21T21:40:43.123 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/2130510535", 2024-09-21T21:40:43.123 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 10 2024-09-21T21:40:43.123 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.123 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:43.123 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.124 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:43.124 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.592695+0000", 2024-09-21T21:40:43.124 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:43.124 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.124 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.124 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:43.124 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.592695+0000", 2024-09-21T21:40:43.124 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:43.124 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.124 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.124 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:43.124 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.592696+0000", 2024-09-21T21:40:43.124 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.003e-06 2024-09-21T21:40:43.124 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.124 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.125 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:43.125 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.592701+0000", 2024-09-21T21:40:43.125 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0509999999999996e-06 2024-09-21T21:40:43.125 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.125 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.125 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:43.125 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.592711+0000", 2024-09-21T21:40:43.125 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.7049999999999994e-06 2024-09-21T21:40:43.125 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.125 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.125 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:43.125 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.592718+0000", 2024-09-21T21:40:43.125 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6660000000000002e-06 2024-09-21T21:40:43.125 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.125 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.126 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:43.126 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.592790+0000", 2024-09-21T21:40:43.126 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2526000000000003e-05 2024-09-21T21:40:43.126 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.126 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.126 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:43.126 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.592849+0000", 2024-09-21T21:40:43.126 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8678999999999998e-05 2024-09-21T21:40:43.126 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.126 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.126 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:43.126 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.592963+0000", 2024-09-21T21:40:43.126 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000114188 2024-09-21T21:40:43.126 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:43.126 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:43.127 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:43.127 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.127 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.127 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4400.0:13 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e24)", 2024-09-21T21:40:43.127 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:39.621758+0000", 2024-09-21T21:40:43.127 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 3.464653191, 2024-09-21T21:40:43.127 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000485828, 2024-09-21T21:40:43.127 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:43.127 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:43.127 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:40:43.127 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:43.127 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4400", 2024-09-21T21:40:43.127 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/2130510535", 2024-09-21T21:40:43.127 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 13 2024-09-21T21:40:43.128 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.128 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:43.128 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.128 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:43.128 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.621758+0000", 2024-09-21T21:40:43.128 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:43.128 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.128 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.128 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:43.128 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.621758+0000", 2024-09-21T21:40:43.128 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:43.128 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.128 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.128 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:43.128 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.621760+0000", 2024-09-21T21:40:43.129 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1100000000000001e-06 2024-09-21T21:40:43.129 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.129 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.129 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:43.129 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.621773+0000", 2024-09-21T21:40:43.129 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.307e-05 2024-09-21T21:40:43.129 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.129 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.129 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:43.129 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.621804+0000", 2024-09-21T21:40:43.129 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0188999999999998e-05 2024-09-21T21:40:43.129 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.129 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.129 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:43.130 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.621818+0000", 2024-09-21T21:40:43.130 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4293e-05 2024-09-21T21:40:43.130 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.130 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.130 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:43.130 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.621968+0000", 2024-09-21T21:40:43.130 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00015012700000000001 2024-09-21T21:40:43.130 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.130 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.130 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:43.130 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.622104+0000", 2024-09-21T21:40:43.130 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013595100000000001 2024-09-21T21:40:43.130 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.130 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.131 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:43.131 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.622244+0000", 2024-09-21T21:40:43.131 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014008799999999999 2024-09-21T21:40:43.131 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:43.131 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:43.131 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:43.131 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:43.131 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:43.131 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:40:43.132 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T21:40:43.133 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:40:43.130+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:40:43.233 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:40:43.231+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:40:43.275 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:40:43.276 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:40:43.276 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-21T21:40:43.276 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:40:43.284 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T21:40:43.334 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:40:43.333+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:40:43.427 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:40:43.427 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:40:43.427 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:40:43.427 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:40:43.427 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:40:43.436 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T21:40:43.436 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:40:43.434+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:40:43.532 INFO:teuthology.orchestra.run.smithi164.stdout:truncating image image_client.0-clone1 from 0xea1e400 (overlap 0xea1e400) to 0x2ed2c00 2024-09-21T21:40:43.536 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:40:43.535+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:40:43.551 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:40:43.551 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-21T21:40:43.551 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-21T21:40:43.551 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:40:43.552 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.552 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4214.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T21:40:43.552 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:25.374665+0000", 2024-09-21T21:40:43.552 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 18.172492942000002, 2024-09-21T21:40:43.552 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0065415279999999996, 2024-09-21T21:40:43.553 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:43.553 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:43.553 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:43.553 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:43.553 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4214", 2024-09-21T21:40:43.553 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.81:0/3228106259", 2024-09-21T21:40:43.553 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1 2024-09-21T21:40:43.554 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.554 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:43.554 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.554 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:43.554 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:25.374665+0000", 2024-09-21T21:40:43.554 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:43.555 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.555 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.555 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:43.555 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:25.374665+0000", 2024-09-21T21:40:43.555 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:43.555 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.556 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.556 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:43.556 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:25.374668+0000", 2024-09-21T21:40:43.556 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4460000000000001e-06 2024-09-21T21:40:43.556 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.556 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.557 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:43.557 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:25.374674+0000", 2024-09-21T21:40:43.557 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1189999999999996e-06 2024-09-21T21:40:43.557 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.557 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.557 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:43.557 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:25.374683+0000", 2024-09-21T21:40:43.558 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.4700000000000008e-06 2024-09-21T21:40:43.558 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.558 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.558 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:43.558 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:25.374691+0000", 2024-09-21T21:40:43.558 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1140000000000006e-06 2024-09-21T21:40:43.559 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.559 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.559 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:43.559 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:25.374743+0000", 2024-09-21T21:40:43.559 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1669000000000003e-05 2024-09-21T21:40:43.559 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.560 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.560 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:43.560 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:25.374873+0000", 2024-09-21T21:40:43.560 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012997400000000001 2024-09-21T21:40:43.560 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.560 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.560 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 2", 2024-09-21T21:40:43.561 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:25.374973+0000", 2024-09-21T21:40:43.561 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.9769e-05 2024-09-21T21:40:43.561 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.561 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.561 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:43.561 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:25.375534+0000", 2024-09-21T21:40:43.561 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00056094100000000002 2024-09-21T21:40:43.562 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.562 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.562 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:43.562 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:25.381121+0000", 2024-09-21T21:40:43.562 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0055874490000000004 2024-09-21T21:40:43.562 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.563 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.563 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:43.563 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:25.381173+0000", 2024-09-21T21:40:43.563 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2447000000000001e-05 2024-09-21T21:40:43.563 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.563 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.563 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:43.564 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:25.381207+0000", 2024-09-21T21:40:43.564 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3130000000000003e-05 2024-09-21T21:40:43.564 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:43.564 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:43.564 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:43.564 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.564 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.564 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4214.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e17)", 2024-09-21T21:40:43.565 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:25.383971+0000", 2024-09-21T21:40:43.565 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 18.16318716, 2024-09-21T21:40:43.565 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000320188, 2024-09-21T21:40:43.565 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:43.565 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:43.565 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2024-09-21T21:40:43.565 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:43.566 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4214", 2024-09-21T21:40:43.566 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.81:0/3228106259", 2024-09-21T21:40:43.566 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2 2024-09-21T21:40:43.566 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.566 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:43.566 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.566 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:43.566 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:25.383971+0000", 2024-09-21T21:40:43.566 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:43.566 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.567 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.567 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:43.567 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:25.383971+0000", 2024-09-21T21:40:43.567 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:43.567 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.567 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.567 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:43.567 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:25.383973+0000", 2024-09-21T21:40:43.567 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1160000000000002e-06 2024-09-21T21:40:43.567 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.567 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.567 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:43.567 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:25.383984+0000", 2024-09-21T21:40:43.567 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0917000000000001e-05 2024-09-21T21:40:43.568 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.568 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.568 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:43.568 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:25.384014+0000", 2024-09-21T21:40:43.568 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9958999999999999e-05 2024-09-21T21:40:43.568 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.568 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.568 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:43.568 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:25.384036+0000", 2024-09-21T21:40:43.568 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1926e-05 2024-09-21T21:40:43.568 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.568 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.568 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:43.568 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:25.384193+0000", 2024-09-21T21:40:43.569 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00015713399999999999 2024-09-21T21:40:43.569 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.569 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.569 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:43.569 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:25.384291+0000", 2024-09-21T21:40:43.569 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.8135999999999997e-05 2024-09-21T21:40:43.569 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:43.569 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:43.569 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:43.569 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.569 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.569 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4214.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-21T21:40:43.569 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:26.375481+0000", 2024-09-21T21:40:43.570 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 17.171676687000001, 2024-09-21T21:40:43.570 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0015995670000000001, 2024-09-21T21:40:43.570 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:43.570 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:43.570 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:43.570 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:43.570 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4214", 2024-09-21T21:40:43.570 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.81:0/3228106259", 2024-09-21T21:40:43.570 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 5 2024-09-21T21:40:43.570 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.570 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:43.570 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.570 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:43.571 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:26.375481+0000", 2024-09-21T21:40:43.571 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:43.571 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.571 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.571 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:43.571 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:26.375481+0000", 2024-09-21T21:40:43.571 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:43.571 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.571 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.571 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:43.571 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:26.375483+0000", 2024-09-21T21:40:43.571 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.432e-06 2024-09-21T21:40:43.571 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.572 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.572 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:43.572 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:26.375493+0000", 2024-09-21T21:40:43.572 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0285999999999999e-05 2024-09-21T21:40:43.572 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.572 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.572 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:43.572 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:26.375506+0000", 2024-09-21T21:40:43.572 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2938e-05 2024-09-21T21:40:43.572 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.572 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.572 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:43.573 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:26.375519+0000", 2024-09-21T21:40:43.573 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2995e-05 2024-09-21T21:40:43.573 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.573 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.573 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:43.573 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:26.375599+0000", 2024-09-21T21:40:43.573 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9546e-05 2024-09-21T21:40:43.573 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.573 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.573 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:43.573 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:26.375756+0000", 2024-09-21T21:40:43.574 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00015701699999999999 2024-09-21T21:40:43.574 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.574 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.574 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2024-09-21T21:40:43.574 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:26.375914+0000", 2024-09-21T21:40:43.574 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000158615 2024-09-21T21:40:43.574 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.574 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.574 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:43.574 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:26.376768+0000", 2024-09-21T21:40:43.574 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00085413600000000002 2024-09-21T21:40:43.574 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.574 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.575 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:43.575 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:26.377012+0000", 2024-09-21T21:40:43.575 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00024373 2024-09-21T21:40:43.575 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.575 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.575 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:43.575 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:26.377048+0000", 2024-09-21T21:40:43.575 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6236999999999999e-05 2024-09-21T21:40:43.575 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.575 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.575 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:43.575 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:26.377081+0000", 2024-09-21T21:40:43.576 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2635000000000002e-05 2024-09-21T21:40:43.576 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:43.576 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:43.576 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:43.576 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.576 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.576 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4214.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-21T21:40:43.576 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:27.380242+0000", 2024-09-21T21:40:43.576 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 16.166916056000002, 2024-09-21T21:40:43.576 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0030744890000000001, 2024-09-21T21:40:43.576 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:43.576 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:43.576 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:43.577 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:43.577 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4214", 2024-09-21T21:40:43.577 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.81:0/3228106259", 2024-09-21T21:40:43.577 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7 2024-09-21T21:40:43.577 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.577 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:43.577 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.577 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:43.577 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:27.380242+0000", 2024-09-21T21:40:43.577 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:43.577 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.577 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.577 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:43.577 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:27.380242+0000", 2024-09-21T21:40:43.577 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:43.578 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.578 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.578 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:43.578 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:27.380244+0000", 2024-09-21T21:40:43.578 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1900000000000002e-06 2024-09-21T21:40:43.578 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.578 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.578 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:43.578 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:27.380254+0000", 2024-09-21T21:40:43.578 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5990000000000008e-06 2024-09-21T21:40:43.578 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.578 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.578 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:43.579 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:27.380284+0000", 2024-09-21T21:40:43.579 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0406999999999999e-05 2024-09-21T21:40:43.579 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.579 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.579 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:43.579 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:27.380297+0000", 2024-09-21T21:40:43.579 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3258000000000001e-05 2024-09-21T21:40:43.579 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.579 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.579 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:43.579 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:27.380373+0000", 2024-09-21T21:40:43.579 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5823999999999998e-05 2024-09-21T21:40:43.579 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.579 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.580 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op must wait for map", 2024-09-21T21:40:43.580 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:27.380379+0000", 2024-09-21T21:40:43.580 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3770000000000004e-06 2024-09-21T21:40:43.580 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.580 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.580 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:43.580 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:27.382026+0000", 2024-09-21T21:40:43.580 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001647765 2024-09-21T21:40:43.580 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.580 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.580 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:43.580 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:27.382094+0000", 2024-09-21T21:40:43.580 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7816000000000003e-05 2024-09-21T21:40:43.580 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.581 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.581 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2024-09-21T21:40:43.581 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:27.382202+0000", 2024-09-21T21:40:43.581 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010763 2024-09-21T21:40:43.581 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.581 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.581 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:43.581 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:27.383044+0000", 2024-09-21T21:40:43.581 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00084213500000000004 2024-09-21T21:40:43.581 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.581 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.581 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:43.581 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:27.383249+0000", 2024-09-21T21:40:43.581 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00020495100000000001 2024-09-21T21:40:43.582 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.582 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.582 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:43.582 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:27.383287+0000", 2024-09-21T21:40:43.582 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8211999999999999e-05 2024-09-21T21:40:43.582 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.582 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.582 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:43.582 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:27.383316+0000", 2024-09-21T21:40:43.582 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9325000000000001e-05 2024-09-21T21:40:43.583 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:43.583 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:43.583 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:43.583 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.583 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.583 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4385.0:10 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-21T21:40:43.583 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:39.421955+0000", 2024-09-21T21:40:43.583 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.1252027829999998, 2024-09-21T21:40:43.583 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003786223, 2024-09-21T21:40:43.583 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:43.583 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:43.583 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:43.584 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:43.584 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4385", 2024-09-21T21:40:43.584 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:40:43.584 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 10 2024-09-21T21:40:43.584 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.584 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:43.584 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.584 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:43.584 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.421955+0000", 2024-09-21T21:40:43.584 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:43.584 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.584 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.584 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:43.585 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.421955+0000", 2024-09-21T21:40:43.585 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:43.585 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.585 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.585 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:43.585 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.421957+0000", 2024-09-21T21:40:43.585 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6360000000000001e-06 2024-09-21T21:40:43.585 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.585 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.585 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:43.585 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.421962+0000", 2024-09-21T21:40:43.585 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.31e-06 2024-09-21T21:40:43.586 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.586 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.586 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:43.586 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.421971+0000", 2024-09-21T21:40:43.586 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7490000000000004e-06 2024-09-21T21:40:43.586 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.586 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.586 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:43.586 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.421979+0000", 2024-09-21T21:40:43.586 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8979999999999996e-06 2024-09-21T21:40:43.587 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.587 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.587 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:43.587 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.422053+0000", 2024-09-21T21:40:43.587 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3993999999999997e-05 2024-09-21T21:40:43.587 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.587 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.587 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op must wait for map", 2024-09-21T21:40:43.587 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.422058+0000", 2024-09-21T21:40:43.587 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4249999999999997e-06 2024-09-21T21:40:43.587 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.587 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.587 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:43.588 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.423850+0000", 2024-09-21T21:40:43.588 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001791611 2024-09-21T21:40:43.588 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.588 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.588 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:43.588 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.424051+0000", 2024-09-21T21:40:43.588 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00020083199999999999 2024-09-21T21:40:43.588 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.588 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.588 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 4", 2024-09-21T21:40:43.588 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.424193+0000", 2024-09-21T21:40:43.588 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014260999999999999 2024-09-21T21:40:43.588 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.588 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.589 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:43.589 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.425326+0000", 2024-09-21T21:40:43.589 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0011330909999999999 2024-09-21T21:40:43.589 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.589 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.589 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:43.589 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.425430+0000", 2024-09-21T21:40:43.589 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000104088 2024-09-21T21:40:43.589 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.589 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.589 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:43.590 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.425484+0000", 2024-09-21T21:40:43.590 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3318e-05 2024-09-21T21:40:43.590 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.590 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.590 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:43.590 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:39.425741+0000", 2024-09-21T21:40:43.590 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00025766100000000001 2024-09-21T21:40:43.590 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:43.590 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:43.590 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:43.590 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.590 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.590 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4385.0:463 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_get_id in=13b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:43.591 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.468512+0000", 2024-09-21T21:40:43.591 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.078646211999999993, 2024-09-21T21:40:43.591 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019393800000000001, 2024-09-21T21:40:43.591 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:43.591 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:43.591 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:40:43.591 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:43.591 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4385", 2024-09-21T21:40:43.591 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:40:43.591 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 463 2024-09-21T21:40:43.591 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.591 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:43.591 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.591 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:43.592 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.468512+0000", 2024-09-21T21:40:43.592 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:43.592 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.592 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.592 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:43.592 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.468512+0000", 2024-09-21T21:40:43.592 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:43.592 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.592 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.592 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:43.592 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.468514+0000", 2024-09-21T21:40:43.592 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.781e-06 2024-09-21T21:40:43.592 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.592 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.593 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:43.593 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.468522+0000", 2024-09-21T21:40:43.593 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7849999999999997e-06 2024-09-21T21:40:43.593 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.593 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.593 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:43.593 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.468537+0000", 2024-09-21T21:40:43.593 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4958e-05 2024-09-21T21:40:43.593 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.593 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.593 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:43.594 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.468546+0000", 2024-09-21T21:40:43.594 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6079999999999996e-06 2024-09-21T21:40:43.594 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.594 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.594 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:43.594 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.468627+0000", 2024-09-21T21:40:43.594 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0858000000000004e-05 2024-09-21T21:40:43.594 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.594 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.594 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op must wait for map", 2024-09-21T21:40:43.594 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.468633+0000", 2024-09-21T21:40:43.594 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1789999999999996e-06 2024-09-21T21:40:43.594 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.594 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.594 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:43.595 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.470271+0000", 2024-09-21T21:40:43.595 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0016381950000000001 2024-09-21T21:40:43.595 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.595 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.595 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:43.595 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.470343+0000", 2024-09-21T21:40:43.595 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2068000000000002e-05 2024-09-21T21:40:43.595 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.595 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.595 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:43.595 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.470451+0000", 2024-09-21T21:40:43.595 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000107948 2024-09-21T21:40:43.595 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:43.596 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:43.596 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:43.596 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.596 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.596 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4385.0:465 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_get_name in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:43.596 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.475489+0000", 2024-09-21T21:40:43.596 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.071668894999999996, 2024-09-21T21:40:43.596 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00022832899999999999, 2024-09-21T21:40:43.596 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:43.596 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:43.596 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:40:43.596 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:43.596 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4385", 2024-09-21T21:40:43.596 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:40:43.597 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 465 2024-09-21T21:40:43.597 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.597 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:43.597 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.597 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:43.597 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.475489+0000", 2024-09-21T21:40:43.597 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:43.597 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.597 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.597 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:43.597 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.475489+0000", 2024-09-21T21:40:43.597 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:43.597 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.598 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.598 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:43.598 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.475490+0000", 2024-09-21T21:40:43.598 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6199999999999996e-07 2024-09-21T21:40:43.598 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.598 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.598 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:43.598 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.475500+0000", 2024-09-21T21:40:43.598 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0522e-05 2024-09-21T21:40:43.598 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.598 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.598 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:43.598 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.475511+0000", 2024-09-21T21:40:43.599 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.066e-05 2024-09-21T21:40:43.599 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.599 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.599 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:43.599 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.475518+0000", 2024-09-21T21:40:43.599 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0999999999999998e-06 2024-09-21T21:40:43.599 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.599 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.599 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:43.600 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.475591+0000", 2024-09-21T21:40:43.600 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2484999999999995e-05 2024-09-21T21:40:43.600 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.600 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.600 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:43.600 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.475637+0000", 2024-09-21T21:40:43.600 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6771000000000003e-05 2024-09-21T21:40:43.600 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.600 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.600 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:43.600 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.475717+0000", 2024-09-21T21:40:43.600 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9929e-05 2024-09-21T21:40:43.600 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:43.601 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:43.601 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:43.601 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.601 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.601 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4385.0:493 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:43.601 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.495778+0000", 2024-09-21T21:40:43.601 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.051379923000000001, 2024-09-21T21:40:43.601 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001181421, 2024-09-21T21:40:43.601 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:43.601 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:43.601 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:43.601 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:43.601 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4385", 2024-09-21T21:40:43.602 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:40:43.602 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 493 2024-09-21T21:40:43.602 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.602 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:43.602 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.602 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:43.602 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.495778+0000", 2024-09-21T21:40:43.602 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:43.602 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.602 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.602 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:43.602 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.495778+0000", 2024-09-21T21:40:43.602 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:43.602 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.603 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.603 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:43.603 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.495779+0000", 2024-09-21T21:40:43.603 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8899999999999998e-07 2024-09-21T21:40:43.603 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.603 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.603 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:43.603 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.495784+0000", 2024-09-21T21:40:43.603 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8400000000000002e-06 2024-09-21T21:40:43.603 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.603 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.603 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:43.603 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.495794+0000", 2024-09-21T21:40:43.603 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.066e-05 2024-09-21T21:40:43.604 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.604 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.604 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:43.604 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.495804+0000", 2024-09-21T21:40:43.604 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.1800000000000002e-06 2024-09-21T21:40:43.604 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.604 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.604 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:43.604 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.495848+0000", 2024-09-21T21:40:43.604 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4668e-05 2024-09-21T21:40:43.604 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.604 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.605 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:43.605 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.495911+0000", 2024-09-21T21:40:43.605 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2279000000000005e-05 2024-09-21T21:40:43.605 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.605 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.605 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 4", 2024-09-21T21:40:43.605 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.496051+0000", 2024-09-21T21:40:43.605 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014036499999999999 2024-09-21T21:40:43.605 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.605 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.605 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:43.605 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.496739+0000", 2024-09-21T21:40:43.606 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00068810900000000001 2024-09-21T21:40:43.606 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.606 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.606 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:43.606 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.496860+0000", 2024-09-21T21:40:43.606 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000120729 2024-09-21T21:40:43.606 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.606 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.606 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:43.606 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.496921+0000", 2024-09-21T21:40:43.606 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1017000000000002e-05 2024-09-21T21:40:43.606 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.606 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.606 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:43.607 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.496959+0000", 2024-09-21T21:40:43.607 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8685e-05 2024-09-21T21:40:43.607 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:43.607 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:43.607 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:43.607 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.607 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.607 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4385.0:495 3.f 3:f6ac0ab2:::rbd_header.1121158926db:head [create,call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:43.607 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.498860+0000", 2024-09-21T21:40:43.607 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.048297904000000003, 2024-09-21T21:40:43.607 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001250679, 2024-09-21T21:40:43.607 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:43.607 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:43.607 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:43.607 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:43.608 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4385", 2024-09-21T21:40:43.608 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:40:43.608 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 495 2024-09-21T21:40:43.608 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.608 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:43.608 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.608 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:43.608 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.498860+0000", 2024-09-21T21:40:43.608 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:43.608 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.608 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.608 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:43.608 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.498860+0000", 2024-09-21T21:40:43.608 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:43.608 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.608 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.609 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:43.609 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.498866+0000", 2024-09-21T21:40:43.609 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6840000000000001e-06 2024-09-21T21:40:43.609 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.609 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.609 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:43.609 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.498877+0000", 2024-09-21T21:40:43.609 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1586000000000001e-05 2024-09-21T21:40:43.609 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.609 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.609 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:43.609 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.498888+0000", 2024-09-21T21:40:43.609 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0755e-05 2024-09-21T21:40:43.609 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.610 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.610 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:43.610 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.498894+0000", 2024-09-21T21:40:43.610 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2960000000000004e-06 2024-09-21T21:40:43.610 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.610 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.610 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:43.610 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.498944+0000", 2024-09-21T21:40:43.610 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9734000000000002e-05 2024-09-21T21:40:43.610 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.610 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.610 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:43.610 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.499031+0000", 2024-09-21T21:40:43.611 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7328000000000001e-05 2024-09-21T21:40:43.611 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.611 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.611 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:40:43.611 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.499152+0000", 2024-09-21T21:40:43.611 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000120512 2024-09-21T21:40:43.611 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.611 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.611 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:43.611 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.499659+0000", 2024-09-21T21:40:43.611 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00050750999999999997 2024-09-21T21:40:43.611 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.611 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.611 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:43.611 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.500056+0000", 2024-09-21T21:40:43.612 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00039700199999999998 2024-09-21T21:40:43.612 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.612 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.612 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:43.612 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.500086+0000", 2024-09-21T21:40:43.612 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9366999999999999e-05 2024-09-21T21:40:43.612 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.612 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.612 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:43.612 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.500111+0000", 2024-09-21T21:40:43.612 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4905e-05 2024-09-21T21:40:43.612 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:43.612 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:43.612 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:43.613 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.613 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.613 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4385.0:496 3.f 3:f6ac0ab2:::rbd_header.1121158926db:head [call rbd.set_stripe_unit_count in=24b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:43.613 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.500292+0000", 2024-09-21T21:40:43.613 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.046866270000000002, 2024-09-21T21:40:43.613 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002161638, 2024-09-21T21:40:43.613 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:43.613 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:43.613 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:43.613 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:43.613 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4385", 2024-09-21T21:40:43.613 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:40:43.613 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 496 2024-09-21T21:40:43.613 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.614 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:43.614 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.614 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:43.614 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.500292+0000", 2024-09-21T21:40:43.614 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:43.614 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.614 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.614 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:43.614 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.500292+0000", 2024-09-21T21:40:43.614 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:43.614 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.614 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.614 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:43.615 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.500292+0000", 2024-09-21T21:40:43.615 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8999999999999996e-07 2024-09-21T21:40:43.615 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.615 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.615 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:43.615 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.500296+0000", 2024-09-21T21:40:43.615 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.399e-06 2024-09-21T21:40:43.615 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.615 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.615 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:43.615 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.500303+0000", 2024-09-21T21:40:43.615 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5859999999999999e-06 2024-09-21T21:40:43.615 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.616 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.616 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:43.616 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.500309+0000", 2024-09-21T21:40:43.616 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4219999999999999e-06 2024-09-21T21:40:43.616 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.616 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.616 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:43.616 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.500349+0000", 2024-09-21T21:40:43.616 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0244000000000003e-05 2024-09-21T21:40:43.616 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.616 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.616 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:43.616 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.500397+0000", 2024-09-21T21:40:43.616 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7843000000000003e-05 2024-09-21T21:40:43.617 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.617 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.617 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:40:43.617 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.500540+0000", 2024-09-21T21:40:43.617 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000143573 2024-09-21T21:40:43.617 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.617 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.617 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:43.617 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.502299+0000", 2024-09-21T21:40:43.617 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00175874 2024-09-21T21:40:43.617 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.617 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.617 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:43.617 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.502390+0000", 2024-09-21T21:40:43.618 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.0564999999999998e-05 2024-09-21T21:40:43.618 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.618 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.618 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:43.618 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.502423+0000", 2024-09-21T21:40:43.618 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2894e-05 2024-09-21T21:40:43.618 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.618 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.618 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:43.618 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.502453+0000", 2024-09-21T21:40:43.618 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0682000000000002e-05 2024-09-21T21:40:43.618 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:43.618 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:43.619 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:43.619 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.619 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.619 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4385.0:500 3.f 3:f6ac0ab2:::rbd_header.1121158926db:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:43.619 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.508616+0000", 2024-09-21T21:40:43.619 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.038542304999999999, 2024-09-21T21:40:43.619 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00032649900000000001, 2024-09-21T21:40:43.619 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:43.619 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:43.619 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:40:43.619 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:43.619 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4385", 2024-09-21T21:40:43.619 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:40:43.620 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 500 2024-09-21T21:40:43.620 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.620 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:43.620 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.620 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:43.620 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.508616+0000", 2024-09-21T21:40:43.620 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:43.620 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.620 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.620 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:43.620 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.508616+0000", 2024-09-21T21:40:43.620 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:43.620 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.621 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.621 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:43.621 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.508617+0000", 2024-09-21T21:40:43.621 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5120000000000001e-06 2024-09-21T21:40:43.621 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.621 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.621 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:43.621 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.508625+0000", 2024-09-21T21:40:43.621 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1030000000000002e-06 2024-09-21T21:40:43.621 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.621 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.621 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:43.622 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.508642+0000", 2024-09-21T21:40:43.622 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6996000000000001e-05 2024-09-21T21:40:43.622 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.622 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.622 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:43.622 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.508668+0000", 2024-09-21T21:40:43.622 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6021e-05 2024-09-21T21:40:43.622 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.622 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.622 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:43.622 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.508717+0000", 2024-09-21T21:40:43.622 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8780999999999999e-05 2024-09-21T21:40:43.622 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.623 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.623 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:43.623 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.508780+0000", 2024-09-21T21:40:43.623 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3147999999999999e-05 2024-09-21T21:40:43.623 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.623 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.623 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:43.623 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.508942+0000", 2024-09-21T21:40:43.623 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00016193800000000001 2024-09-21T21:40:43.623 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:43.623 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:43.623 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:43.623 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.624 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.624 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4385.0:501 3.f 3:f6ac0ab2:::rbd_header.1121158926db:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:43.624 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.509242+0000", 2024-09-21T21:40:43.624 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.037915855999999998, 2024-09-21T21:40:43.624 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00027740499999999999, 2024-09-21T21:40:43.624 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:43.624 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:43.624 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:40:43.624 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:43.624 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4385", 2024-09-21T21:40:43.624 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:40:43.624 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 501 2024-09-21T21:40:43.624 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.624 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:43.625 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.625 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:43.625 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.509242+0000", 2024-09-21T21:40:43.625 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:43.625 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.625 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.625 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:43.625 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.509242+0000", 2024-09-21T21:40:43.625 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:43.625 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.625 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.625 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:43.625 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.509243+0000", 2024-09-21T21:40:43.625 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9999999999999996e-07 2024-09-21T21:40:43.626 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.626 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.626 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:43.626 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.509249+0000", 2024-09-21T21:40:43.626 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5679999999999999e-06 2024-09-21T21:40:43.626 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.626 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.626 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:43.626 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.509261+0000", 2024-09-21T21:40:43.626 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2744e-05 2024-09-21T21:40:43.626 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.626 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.627 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:43.627 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.509269+0000", 2024-09-21T21:40:43.627 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1589999999999993e-06 2024-09-21T21:40:43.627 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.627 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.627 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:43.627 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.509364+0000", 2024-09-21T21:40:43.627 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.4648999999999995e-05 2024-09-21T21:40:43.628 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.628 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.628 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:43.628 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.509413+0000", 2024-09-21T21:40:43.628 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8887000000000001e-05 2024-09-21T21:40:43.628 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.629 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.629 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:43.629 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.509520+0000", 2024-09-21T21:40:43.629 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000106598 2024-09-21T21:40:43.630 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:43.630 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:43.630 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:43.630 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.630 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.630 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4385.0:502 3.f 3:f6ac0ab2:::rbd_header.1121158926db:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:43.630 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.509880+0000", 2024-09-21T21:40:43.631 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.03727805, 2024-09-21T21:40:43.631 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00020072499999999999, 2024-09-21T21:40:43.631 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:43.631 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:43.631 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:40:43.631 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:43.632 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4385", 2024-09-21T21:40:43.632 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:40:43.632 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 502 2024-09-21T21:40:43.632 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.632 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:43.632 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.633 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:43.633 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.509880+0000", 2024-09-21T21:40:43.633 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:43.633 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.633 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.633 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:43.633 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.509880+0000", 2024-09-21T21:40:43.634 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:43.634 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.634 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.634 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:43.634 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.509881+0000", 2024-09-21T21:40:43.634 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.2200000000000002e-07 2024-09-21T21:40:43.635 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.635 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.635 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:43.635 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.509887+0000", 2024-09-21T21:40:43.635 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.998e-06 2024-09-21T21:40:43.635 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.636 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.636 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:43.636 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.509899+0000", 2024-09-21T21:40:43.636 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2303000000000001e-05 2024-09-21T21:40:43.636 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.637 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.637 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:43.637 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.509906+0000", 2024-09-21T21:40:43.637 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0140000000000001e-06 2024-09-21T21:40:43.637 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.637 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.638 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:43.638 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.509957+0000", 2024-09-21T21:40:43.638 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0809000000000001e-05 2024-09-21T21:40:43.638 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.638 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.638 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:43.639 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.510004+0000", 2024-09-21T21:40:43.639 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7459000000000001e-05 2024-09-21T21:40:43.639 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.639 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.639 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:43.639 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.510081+0000", 2024-09-21T21:40:43.640 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6219999999999999e-05 2024-09-21T21:40:43.640 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:43.640 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:43.640 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:43.640 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.640 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.640 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4385.0:503 3.f 3:f6ac0ab2:::rbd_header.1121158926db:head [call,call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:43.641 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.510376+0000", 2024-09-21T21:40:43.641 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.036782318000000001, 2024-09-21T21:40:43.641 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00025251599999999999, 2024-09-21T21:40:43.641 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:43.641 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:43.641 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:40:43.642 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:43.642 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4385", 2024-09-21T21:40:43.642 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:40:43.642 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 503 2024-09-21T21:40:43.642 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.642 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:43.643 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.643 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:43.643 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.510376+0000", 2024-09-21T21:40:43.643 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:43.643 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.643 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.644 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:43.644 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.510376+0000", 2024-09-21T21:40:43.644 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:43.644 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.644 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.644 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:43.645 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.510377+0000", 2024-09-21T21:40:43.645 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7000000000000003e-07 2024-09-21T21:40:43.645 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.645 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.645 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:43.645 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.510383+0000", 2024-09-21T21:40:43.646 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2369999999999997e-06 2024-09-21T21:40:43.646 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.646 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.646 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:43.646 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.510396+0000", 2024-09-21T21:40:43.646 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3704e-05 2024-09-21T21:40:43.647 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.647 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.647 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:43.647 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.510406+0000", 2024-09-21T21:40:43.647 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5340000000000005e-06 2024-09-21T21:40:43.647 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.648 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.648 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:43.648 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.510482+0000", 2024-09-21T21:40:43.648 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6440999999999995e-05 2024-09-21T21:40:43.648 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.648 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.649 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:43.649 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.510531+0000", 2024-09-21T21:40:43.649 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8547999999999997e-05 2024-09-21T21:40:43.649 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.649 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.649 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:43.649 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.510628+0000", 2024-09-21T21:40:43.650 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.7182000000000006e-05 2024-09-21T21:40:43.650 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:43.650 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:43.650 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:43.650 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.650 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.651 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4385.0:505 3.f 3:f6ac0ab2:::rbd_header.1121158926db:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:43.651 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.511610+0000", 2024-09-21T21:40:43.651 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.035547927, 2024-09-21T21:40:43.651 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00041278799999999998, 2024-09-21T21:40:43.651 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:43.651 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:43.652 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:40:43.652 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:43.652 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4385", 2024-09-21T21:40:43.652 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:40:43.652 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 505 2024-09-21T21:40:43.653 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.653 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:43.653 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.653 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:43.653 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.511610+0000", 2024-09-21T21:40:43.653 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:43.654 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.654 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.654 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:43.654 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.511610+0000", 2024-09-21T21:40:43.654 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:43.654 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.655 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.655 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:43.655 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.511611+0000", 2024-09-21T21:40:43.655 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5199999999999995e-07 2024-09-21T21:40:43.655 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.655 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.656 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:43.656 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.511616+0000", 2024-09-21T21:40:43.656 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4049999999999999e-06 2024-09-21T21:40:43.656 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.656 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.656 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:43.656 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.511630+0000", 2024-09-21T21:40:43.657 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3417e-05 2024-09-21T21:40:43.657 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.657 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.657 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:43.657 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.511639+0000", 2024-09-21T21:40:43.657 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.2750000000000001e-06 2024-09-21T21:40:43.657 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.658 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.658 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:43.658 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.511716+0000", 2024-09-21T21:40:43.658 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6779000000000005e-05 2024-09-21T21:40:43.658 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.658 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.659 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:43.659 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.511772+0000", 2024-09-21T21:40:43.659 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6240999999999999e-05 2024-09-21T21:40:43.659 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.659 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.659 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:43.659 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.512023+0000", 2024-09-21T21:40:43.660 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000250819 2024-09-21T21:40:43.660 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:43.660 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:43.660 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:43.660 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.660 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.661 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4385.0:506 3.f 3:f6ac0ab2:::rbd_header.1121158926db:head [call,call rbd.parent_overlap_get in=21b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:43.661 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.512324+0000", 2024-09-21T21:40:43.661 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.034833852999999998, 2024-09-21T21:40:43.661 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00033635400000000003, 2024-09-21T21:40:43.661 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:43.661 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:43.661 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:40:43.662 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:43.662 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4385", 2024-09-21T21:40:43.662 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:40:43.662 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 506 2024-09-21T21:40:43.662 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.662 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:43.663 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.663 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:43.663 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.512324+0000", 2024-09-21T21:40:43.663 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:43.663 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.663 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.664 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:43.664 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.512324+0000", 2024-09-21T21:40:43.664 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:43.664 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.664 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.664 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:43.665 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.512325+0000", 2024-09-21T21:40:43.665 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3399999999999998e-07 2024-09-21T21:40:43.665 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.665 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.665 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:43.665 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.512330+0000", 2024-09-21T21:40:43.666 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7060000000000003e-06 2024-09-21T21:40:43.666 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.666 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.666 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:43.666 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.512343+0000", 2024-09-21T21:40:43.666 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3169999999999999e-05 2024-09-21T21:40:43.666 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.667 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.667 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:43.667 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.512351+0000", 2024-09-21T21:40:43.667 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7419999999999997e-06 2024-09-21T21:40:43.667 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.667 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.667 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:43.668 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.512430+0000", 2024-09-21T21:40:43.668 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9149999999999999e-05 2024-09-21T21:40:43.668 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.668 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.668 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:43.669 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.512481+0000", 2024-09-21T21:40:43.669 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0898e-05 2024-09-21T21:40:43.669 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.669 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.669 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:43.669 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.512660+0000", 2024-09-21T21:40:43.669 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00017985399999999999 2024-09-21T21:40:43.670 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:43.670 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:43.670 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:43.670 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.670 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.670 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4385.0:509 3.f 3:f6ac0ab2:::rbd_header.1121158926db:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:43.671 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.514126+0000", 2024-09-21T21:40:43.671 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.033031875000000002, 2024-09-21T21:40:43.671 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00024102800000000001, 2024-09-21T21:40:43.671 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:43.671 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:43.671 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:40:43.671 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:43.671 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4385", 2024-09-21T21:40:43.671 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:40:43.672 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 509 2024-09-21T21:40:43.672 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.672 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:43.672 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.672 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:43.672 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.514126+0000", 2024-09-21T21:40:43.673 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:43.673 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.673 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.673 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:43.673 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.514126+0000", 2024-09-21T21:40:43.673 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:43.674 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.674 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.674 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:43.674 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.514127+0000", 2024-09-21T21:40:43.674 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3399999999999997e-07 2024-09-21T21:40:43.674 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.675 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.675 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:43.675 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.514133+0000", 2024-09-21T21:40:43.675 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.48e-06 2024-09-21T21:40:43.675 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.675 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.675 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:43.676 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.514162+0000", 2024-09-21T21:40:43.676 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9709e-05 2024-09-21T21:40:43.676 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.676 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.676 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:43.676 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.514169+0000", 2024-09-21T21:40:43.676 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1779999999999997e-06 2024-09-21T21:40:43.676 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.676 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.677 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:43.677 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.514244+0000", 2024-09-21T21:40:43.677 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4667999999999997e-05 2024-09-21T21:40:43.677 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.677 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.677 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:43.677 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.514290+0000", 2024-09-21T21:40:43.677 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5989000000000003e-05 2024-09-21T21:40:43.677 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.677 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.677 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:43.677 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.514367+0000", 2024-09-21T21:40:43.677 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7070000000000006e-05 2024-09-21T21:40:43.678 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:43.678 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:43.678 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:43.678 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.678 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.678 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4385.0:510 3.f 3:f6ac0ab2:::rbd_header.1121158926db:head [watch watch cookie 140325440779296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:43.678 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.514650+0000", 2024-09-21T21:40:43.678 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.032507719999999997, 2024-09-21T21:40:43.678 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0010558010000000001, 2024-09-21T21:40:43.678 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:43.678 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:43.678 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:43.679 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:43.679 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4385", 2024-09-21T21:40:43.679 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:40:43.679 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 510 2024-09-21T21:40:43.679 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.679 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:43.679 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.679 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:43.679 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.514650+0000", 2024-09-21T21:40:43.679 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:43.679 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.679 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.679 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:43.679 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.514650+0000", 2024-09-21T21:40:43.680 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:43.680 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.680 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.680 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:43.680 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.514651+0000", 2024-09-21T21:40:43.680 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.1900000000000001e-07 2024-09-21T21:40:43.680 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.680 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.680 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:43.680 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.514655+0000", 2024-09-21T21:40:43.680 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0770000000000001e-06 2024-09-21T21:40:43.680 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.681 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.681 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:43.681 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.514669+0000", 2024-09-21T21:40:43.681 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3923e-05 2024-09-21T21:40:43.681 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.681 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.681 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:43.681 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.514678+0000", 2024-09-21T21:40:43.681 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5890000000000003e-06 2024-09-21T21:40:43.681 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.681 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.681 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:43.681 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.514719+0000", 2024-09-21T21:40:43.681 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0757000000000003e-05 2024-09-21T21:40:43.682 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.682 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.682 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:43.682 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.514761+0000", 2024-09-21T21:40:43.682 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2330999999999998e-05 2024-09-21T21:40:43.682 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.682 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.682 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:40:43.682 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.514843+0000", 2024-09-21T21:40:43.682 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2460000000000007e-05 2024-09-21T21:40:43.682 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.682 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.682 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:43.683 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.515355+0000", 2024-09-21T21:40:43.683 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00051183399999999997 2024-09-21T21:40:43.683 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.683 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.683 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:43.683 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.515631+0000", 2024-09-21T21:40:43.683 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00027539799999999999 2024-09-21T21:40:43.683 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.683 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.683 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:43.683 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.515664+0000", 2024-09-21T21:40:43.683 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2947000000000002e-05 2024-09-21T21:40:43.683 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.684 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.684 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:43.684 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.515706+0000", 2024-09-21T21:40:43.684 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2565999999999999e-05 2024-09-21T21:40:43.684 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:43.684 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:43.684 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:43.684 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.684 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.684 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4385.0:511 3.f 3:f6ac0ab2:::rbd_header.1121158926db:head [call rbd.parent_attach in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:43.684 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.516060+0000", 2024-09-21T21:40:43.684 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.031098020000000001, 2024-09-21T21:40:43.684 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00091921899999999998, 2024-09-21T21:40:43.684 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:43.685 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:43.685 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:43.685 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:43.685 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4385", 2024-09-21T21:40:43.685 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:40:43.685 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 511 2024-09-21T21:40:43.685 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.685 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:43.685 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.685 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:43.685 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.516060+0000", 2024-09-21T21:40:43.685 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:43.685 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.685 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.686 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:43.686 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.516060+0000", 2024-09-21T21:40:43.686 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:43.686 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.686 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.686 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:43.686 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.516061+0000", 2024-09-21T21:40:43.686 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8999999999999996e-07 2024-09-21T21:40:43.686 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.686 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.686 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:43.686 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.516064+0000", 2024-09-21T21:40:43.687 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4460000000000001e-06 2024-09-21T21:40:43.687 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.687 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.687 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:43.687 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.516071+0000", 2024-09-21T21:40:43.687 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.362e-06 2024-09-21T21:40:43.687 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.687 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.687 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:43.687 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.516076+0000", 2024-09-21T21:40:43.687 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.33e-06 2024-09-21T21:40:43.687 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.687 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.687 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:43.687 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.516112+0000", 2024-09-21T21:40:43.688 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6115000000000002e-05 2024-09-21T21:40:43.688 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.688 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.688 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:43.688 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.516161+0000", 2024-09-21T21:40:43.688 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9004e-05 2024-09-21T21:40:43.688 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.688 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.688 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:40:43.688 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.516310+0000", 2024-09-21T21:40:43.688 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000148923 2024-09-21T21:40:43.688 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.688 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.689 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:43.689 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.516701+0000", 2024-09-21T21:40:43.689 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00039093500000000002 2024-09-21T21:40:43.689 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.689 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.689 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:43.689 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.516923+0000", 2024-09-21T21:40:43.689 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00022209500000000001 2024-09-21T21:40:43.689 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.689 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.689 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:43.689 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.516952+0000", 2024-09-21T21:40:43.689 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8804000000000001e-05 2024-09-21T21:40:43.689 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.690 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.690 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:43.690 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.516979+0000", 2024-09-21T21:40:43.690 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7515e-05 2024-09-21T21:40:43.690 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:43.690 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:43.690 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:43.690 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.690 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.690 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4385.0:520 3.f 3:f6ac0ab2:::rbd_header.1121158926db:head [watch unwatch cookie 140325440779296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:43.690 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.525024+0000", 2024-09-21T21:40:43.690 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.022134489, 2024-09-21T21:40:43.690 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0009881270000000001, 2024-09-21T21:40:43.690 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:43.691 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:43.691 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:43.691 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:43.691 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4385", 2024-09-21T21:40:43.691 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:40:43.691 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 520 2024-09-21T21:40:43.691 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.691 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:43.691 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.691 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:43.691 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.525024+0000", 2024-09-21T21:40:43.691 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:43.691 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.691 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.692 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:43.692 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.525024+0000", 2024-09-21T21:40:43.692 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:43.692 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.692 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.692 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:43.692 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.525024+0000", 2024-09-21T21:40:43.692 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4499999999999996e-07 2024-09-21T21:40:43.692 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.692 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.692 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:43.693 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.525029+0000", 2024-09-21T21:40:43.693 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9760000000000003e-06 2024-09-21T21:40:43.693 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.693 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.693 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:43.693 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.525040+0000", 2024-09-21T21:40:43.693 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1221e-05 2024-09-21T21:40:43.693 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.693 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.693 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:43.693 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.525046+0000", 2024-09-21T21:40:43.693 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0669999999999999e-06 2024-09-21T21:40:43.693 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.694 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.694 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:43.694 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.525086+0000", 2024-09-21T21:40:43.694 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0027999999999997e-05 2024-09-21T21:40:43.694 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.694 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.694 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:43.694 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.525133+0000", 2024-09-21T21:40:43.694 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7054e-05 2024-09-21T21:40:43.694 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.694 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.695 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:40:43.695 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.525213+0000", 2024-09-21T21:40:43.695 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0370000000000005e-05 2024-09-21T21:40:43.695 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.695 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.695 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:43.695 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.525642+0000", 2024-09-21T21:40:43.695 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00042865500000000001 2024-09-21T21:40:43.695 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.695 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.695 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:43.695 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.525940+0000", 2024-09-21T21:40:43.696 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000298184 2024-09-21T21:40:43.696 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.696 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.696 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:43.696 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.525972+0000", 2024-09-21T21:40:43.696 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1931999999999999e-05 2024-09-21T21:40:43.696 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:43.696 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:43.696 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:43.696 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.526012+0000", 2024-09-21T21:40:43.696 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9894999999999997e-05 2024-09-21T21:40:43.696 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:43.696 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:43.697 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:43.697 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:43.697 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:43.697 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:40:43.698 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:40:43.696+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:40:43.699 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T21:40:43.799 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:40:43.797+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:40:43.803 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:40:43.804 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:40:43.804 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 0 2024-09-21T21:40:43.804 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:40:43.814 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T21:40:43.899 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:40:43.898+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:40:43.932 INFO:teuthology.orchestra.run.smithi164.stdout:checking clone #0, image image_client.0 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent1 2024-09-21T21:40:43.959 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:40:43.960 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:40:43.960 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:40:43.960 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:40:43.960 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:40:43.968 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T21:40:44.009 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:40:44.008+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:40:44.101 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:40:44.099+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:40:44.124 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:40:44.125 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 20, 2024-09-21T21:40:44.125 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 600, 2024-09-21T21:40:44.125 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:40:44.125 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.125 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "pg_update_log_missing(3.15 epoch 26/21 rep_tid 217 entries 26'3 (0'0) error 3:abf84a89:::rbd_data.11214be7c411.0000000000000025:head by client.4417.0:131 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 24'1)", 2024-09-21T21:40:44.125 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.536284+0000", 2024-09-21T21:40:44.126 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.58528428499999996, 2024-09-21T21:40:44.126 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0057258170000000002, 2024-09-21T21:40:44.126 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:44.126 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:44.126 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:40:44.126 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:44.127 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.127 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:44.127 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.536284+0000", 2024-09-21T21:40:44.127 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:44.127 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.127 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.128 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:44.128 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.536284+0000", 2024-09-21T21:40:44.128 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:44.128 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.128 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.128 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:44.129 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.536284+0000", 2024-09-21T21:40:44.129 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.6399999999999998e-07 2024-09-21T21:40:44.129 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.129 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.129 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:44.129 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.536290+0000", 2024-09-21T21:40:44.129 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2430000000000002e-06 2024-09-21T21:40:44.130 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.130 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.130 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:44.130 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.536309+0000", 2024-09-21T21:40:44.130 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.8572000000000001e-05 2024-09-21T21:40:44.130 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.131 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.131 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:44.131 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.536324+0000", 2024-09-21T21:40:44.131 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.4876e-05 2024-09-21T21:40:44.131 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.131 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.131 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:44.132 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.536414+0000", 2024-09-21T21:40:44.132 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.0847999999999995e-05 2024-09-21T21:40:44.132 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.132 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.132 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:44.132 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.542009+0000", 2024-09-21T21:40:44.133 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0055950139999999997 2024-09-21T21:40:44.133 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:44.133 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:44.133 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:44.133 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.133 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.133 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:157 3.6 3:678f83b3:::rbd_data.11214be7c411.000000000000002a:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:44.134 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.538947+0000", 2024-09-21T21:40:44.134 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.58262055300000004, 2024-09-21T21:40:44.134 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.006108998, 2024-09-21T21:40:44.134 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:44.134 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:44.134 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:40:44.135 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:44.135 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:44.135 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:44.135 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 157 2024-09-21T21:40:44.135 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.135 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:44.136 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.136 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:44.136 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.538947+0000", 2024-09-21T21:40:44.136 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:44.136 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.136 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.137 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:44.137 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.538947+0000", 2024-09-21T21:40:44.137 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:44.137 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.137 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.137 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:44.138 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.538948+0000", 2024-09-21T21:40:44.138 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.7100000000000011e-07 2024-09-21T21:40:44.138 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.138 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.138 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:44.138 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.538953+0000", 2024-09-21T21:40:44.139 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7099999999999998e-06 2024-09-21T21:40:44.139 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.139 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.139 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:44.139 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.538964+0000", 2024-09-21T21:40:44.139 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0900000000000001e-05 2024-09-21T21:40:44.139 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.140 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.140 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:44.140 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.538972+0000", 2024-09-21T21:40:44.140 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7279999999999995e-06 2024-09-21T21:40:44.140 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.140 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.141 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:44.141 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.539034+0000", 2024-09-21T21:40:44.141 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2025000000000003e-05 2024-09-21T21:40:44.141 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.141 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.141 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:44.142 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.539148+0000", 2024-09-21T21:40:44.142 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000113879 2024-09-21T21:40:44.142 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.142 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.142 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:44.142 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.545056+0000", 2024-09-21T21:40:44.143 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0059087849999999997 2024-09-21T21:40:44.143 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:44.143 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:44.143 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:44.143 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.143 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.143 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:181 3.1 3:865f7811:::rbd_data.11214be7c411.000000000000002f:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:44.143 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.545151+0000", 2024-09-21T21:40:44.143 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.57641699599999996, 2024-09-21T21:40:44.144 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0053218199999999997, 2024-09-21T21:40:44.144 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:44.144 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:44.144 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:40:44.144 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:44.144 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:44.144 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:44.144 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 181 2024-09-21T21:40:44.145 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.145 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:44.145 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.145 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:44.145 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.545151+0000", 2024-09-21T21:40:44.145 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:44.146 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.146 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.146 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:44.146 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.545151+0000", 2024-09-21T21:40:44.147 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:44.147 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.147 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.147 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:44.147 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.545152+0000", 2024-09-21T21:40:44.147 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6200000000000006e-07 2024-09-21T21:40:44.147 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.147 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.147 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:44.147 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.545157+0000", 2024-09-21T21:40:44.147 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4410000000000001e-06 2024-09-21T21:40:44.147 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.147 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.147 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:44.148 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.545171+0000", 2024-09-21T21:40:44.148 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.3209e-05 2024-09-21T21:40:44.148 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.148 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.148 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:44.148 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.545178+0000", 2024-09-21T21:40:44.148 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7759999999999997e-06 2024-09-21T21:40:44.148 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.148 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.148 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:44.148 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.545230+0000", 2024-09-21T21:40:44.148 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1641000000000002e-05 2024-09-21T21:40:44.148 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.148 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.149 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:44.149 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.545357+0000", 2024-09-21T21:40:44.149 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00012734199999999999 2024-09-21T21:40:44.149 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.149 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.149 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:44.149 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.550473+0000", 2024-09-21T21:40:44.149 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0051154490000000002 2024-09-21T21:40:44.149 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:44.149 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:44.149 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:44.149 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.149 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.149 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:199 3.12 3:4fa97af7:::rbd_data.11214be7c411.0000000000000033:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:44.149 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.547897+0000", 2024-09-21T21:40:44.150 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.57367108200000005, 2024-09-21T21:40:44.150 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0052696440000000004, 2024-09-21T21:40:44.150 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:44.150 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:44.150 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:40:44.150 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:44.150 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:44.150 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:44.150 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 199 2024-09-21T21:40:44.151 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.151 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:44.151 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.151 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:44.151 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.547897+0000", 2024-09-21T21:40:44.151 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:44.151 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.151 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.151 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:44.151 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.547897+0000", 2024-09-21T21:40:44.151 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:44.151 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.151 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.152 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:44.152 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.547898+0000", 2024-09-21T21:40:44.152 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.7700000000000003e-07 2024-09-21T21:40:44.152 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.152 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.152 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:44.152 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.547902+0000", 2024-09-21T21:40:44.152 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6099999999999999e-06 2024-09-21T21:40:44.152 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.152 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.152 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:44.152 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.547915+0000", 2024-09-21T21:40:44.152 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.289e-05 2024-09-21T21:40:44.152 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.153 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.153 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:44.153 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.547922+0000", 2024-09-21T21:40:44.153 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.2099999999999996e-06 2024-09-21T21:40:44.153 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.153 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.153 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:44.153 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.547993+0000", 2024-09-21T21:40:44.153 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0290999999999995e-05 2024-09-21T21:40:44.153 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.153 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.153 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:44.153 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.548121+0000", 2024-09-21T21:40:44.153 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000128756 2024-09-21T21:40:44.153 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.154 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.154 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:44.154 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.553166+0000", 2024-09-21T21:40:44.154 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0050450099999999999 2024-09-21T21:40:44.154 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:44.154 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:44.154 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:44.154 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.154 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.154 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:205 3.17 3:ef24eaf4:::rbd_data.11214be7c411.0000000000000034:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:44.154 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.548038+0000", 2024-09-21T21:40:44.154 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.57352998700000002, 2024-09-21T21:40:44.154 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0051384509999999996, 2024-09-21T21:40:44.155 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:44.155 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:44.155 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:40:44.155 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:44.155 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:44.155 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:44.155 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 205 2024-09-21T21:40:44.155 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.155 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:44.155 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.155 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:44.155 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.548038+0000", 2024-09-21T21:40:44.155 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:44.155 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.155 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.156 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:44.156 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.548038+0000", 2024-09-21T21:40:44.156 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:44.156 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.156 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.156 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:44.156 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.548039+0000", 2024-09-21T21:40:44.156 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0500000000000002e-07 2024-09-21T21:40:44.156 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.156 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.156 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:44.156 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.548042+0000", 2024-09-21T21:40:44.156 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.478e-06 2024-09-21T21:40:44.156 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.157 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.157 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:44.157 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.548058+0000", 2024-09-21T21:40:44.157 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.5637999999999998e-05 2024-09-21T21:40:44.157 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.157 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.157 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:44.157 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.548066+0000", 2024-09-21T21:40:44.157 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.6809999999999998e-06 2024-09-21T21:40:44.157 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.157 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.157 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:44.157 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.548124+0000", 2024-09-21T21:40:44.157 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8384999999999998e-05 2024-09-21T21:40:44.158 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.158 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.158 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:44.158 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.548249+0000", 2024-09-21T21:40:44.158 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00012460999999999999 2024-09-21T21:40:44.158 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.158 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.158 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:44.158 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.553176+0000", 2024-09-21T21:40:44.158 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0049278539999999997 2024-09-21T21:40:44.158 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:44.158 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:44.158 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:44.158 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.159 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.159 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "pg_update_log_missing(3.18 epoch 26/21 rep_tid 19 entries 26'5 (0'0) error 3:18a859ac:::rbd_data.11214be7c411.0000000000000036:head by client.4417.0:214 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 24'3)", 2024-09-21T21:40:44.159 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.550506+0000", 2024-09-21T21:40:44.159 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.57106199199999996, 2024-09-21T21:40:44.159 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00530073, 2024-09-21T21:40:44.159 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:44.159 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:44.159 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:40:44.159 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:44.159 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.159 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:44.159 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.550506+0000", 2024-09-21T21:40:44.159 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:44.159 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.160 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.160 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:44.160 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.550506+0000", 2024-09-21T21:40:44.160 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:44.160 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.160 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.160 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:44.160 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.550506+0000", 2024-09-21T21:40:44.160 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.16e-07 2024-09-21T21:40:44.160 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.161 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.161 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:44.161 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.550512+0000", 2024-09-21T21:40:44.161 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6180000000000003e-06 2024-09-21T21:40:44.161 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.161 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.161 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:44.161 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.550529+0000", 2024-09-21T21:40:44.161 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.7325000000000001e-05 2024-09-21T21:40:44.161 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.161 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.161 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:44.161 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.550553+0000", 2024-09-21T21:40:44.161 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.4012999999999999e-05 2024-09-21T21:40:44.162 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.162 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.162 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:44.162 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.550636+0000", 2024-09-21T21:40:44.162 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.3150000000000002e-05 2024-09-21T21:40:44.162 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.162 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.162 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:44.162 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.555807+0000", 2024-09-21T21:40:44.162 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0051704079999999996 2024-09-21T21:40:44.162 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:44.162 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:44.162 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:44.163 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.163 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.163 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:217 3.1 3:865f7811:::rbd_data.11214be7c411.000000000000002f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:44.163 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.550723+0000", 2024-09-21T21:40:44.163 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.57084499399999999, 2024-09-21T21:40:44.163 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0051892220000000003, 2024-09-21T21:40:44.163 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:44.163 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:44.163 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:44.163 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:44.163 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:44.163 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:44.163 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 217 2024-09-21T21:40:44.163 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.164 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:44.164 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.164 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:44.164 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.550723+0000", 2024-09-21T21:40:44.164 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:44.164 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.164 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.164 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:44.164 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.550723+0000", 2024-09-21T21:40:44.164 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:44.164 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.164 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.164 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:44.165 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.550724+0000", 2024-09-21T21:40:44.165 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.4399999999999999e-07 2024-09-21T21:40:44.165 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.165 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.165 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:44.165 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.550728+0000", 2024-09-21T21:40:44.165 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.9179999999999996e-06 2024-09-21T21:40:44.165 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.165 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.165 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:44.165 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.550737+0000", 2024-09-21T21:40:44.165 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.9789999999999999e-06 2024-09-21T21:40:44.165 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.165 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.166 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:44.166 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.550742+0000", 2024-09-21T21:40:44.166 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9830000000000004e-06 2024-09-21T21:40:44.166 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.166 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.166 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:44.166 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.550788+0000", 2024-09-21T21:40:44.166 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6807000000000001e-05 2024-09-21T21:40:44.166 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.166 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.166 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:44.166 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.550863+0000", 2024-09-21T21:40:44.166 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.4787000000000001e-05 2024-09-21T21:40:44.166 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.166 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.167 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 4", 2024-09-21T21:40:44.167 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.551062+0000", 2024-09-21T21:40:44.167 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00019837000000000001 2024-09-21T21:40:44.167 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.167 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.167 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:44.167 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.551991+0000", 2024-09-21T21:40:44.167 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00092915800000000002 2024-09-21T21:40:44.167 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.167 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.167 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:40:44.167 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.555819+0000", 2024-09-21T21:40:44.168 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0038285160000000001 2024-09-21T21:40:44.168 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.168 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.168 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:44.168 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.555862+0000", 2024-09-21T21:40:44.168 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3158999999999998e-05 2024-09-21T21:40:44.168 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.168 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.168 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:44.168 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.555912+0000", 2024-09-21T21:40:44.168 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9801e-05 2024-09-21T21:40:44.168 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:44.168 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:44.168 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:44.169 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.169 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.169 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:218 3.12 3:48deee06:::rbd_data.11214be7c411.0000000000000031:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:44.169 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.550755+0000", 2024-09-21T21:40:44.169 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.57081306200000004, 2024-09-21T21:40:44.169 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0051449260000000002, 2024-09-21T21:40:44.169 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:44.169 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:44.169 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:44.169 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:44.169 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:44.169 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:44.169 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 218 2024-09-21T21:40:44.169 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.170 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:44.170 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.170 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:44.170 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.550755+0000", 2024-09-21T21:40:44.170 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:44.170 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.170 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.170 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:44.170 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.550755+0000", 2024-09-21T21:40:44.170 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:44.170 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.171 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.171 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:44.171 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.550755+0000", 2024-09-21T21:40:44.171 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9900000000000001e-07 2024-09-21T21:40:44.171 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.171 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.171 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:44.171 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.550758+0000", 2024-09-21T21:40:44.171 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.8030000000000001e-06 2024-09-21T21:40:44.171 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.171 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.171 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:44.172 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.550765+0000", 2024-09-21T21:40:44.172 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.567e-06 2024-09-21T21:40:44.172 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.172 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.172 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:44.172 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.550770+0000", 2024-09-21T21:40:44.172 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9180000000000002e-06 2024-09-21T21:40:44.172 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.172 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.172 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:44.172 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.550842+0000", 2024-09-21T21:40:44.172 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.2397000000000005e-05 2024-09-21T21:40:44.172 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.172 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.173 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:44.173 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.550894+0000", 2024-09-21T21:40:44.173 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1953000000000002e-05 2024-09-21T21:40:44.173 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.173 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.173 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 3", 2024-09-21T21:40:44.173 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.551019+0000", 2024-09-21T21:40:44.173 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00012530500000000001 2024-09-21T21:40:44.173 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.173 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.173 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:44.173 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.551761+0000", 2024-09-21T21:40:44.173 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00074191700000000003 2024-09-21T21:40:44.173 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.174 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.174 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:40:44.174 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.555812+0000", 2024-09-21T21:40:44.174 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0040509300000000003 2024-09-21T21:40:44.174 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.174 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.174 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:44.174 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.555861+0000", 2024-09-21T21:40:44.174 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9011000000000002e-05 2024-09-21T21:40:44.174 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.174 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.174 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:44.174 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.555900+0000", 2024-09-21T21:40:44.175 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8625999999999999e-05 2024-09-21T21:40:44.175 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:44.175 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:44.175 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:44.175 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.175 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.175 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:227 3.6 3:678f83b3:::rbd_data.11214be7c411.000000000000002a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:44.175 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.553729+0000", 2024-09-21T21:40:44.175 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.56783899199999999, 2024-09-21T21:40:44.175 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0090411439999999992, 2024-09-21T21:40:44.175 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:44.175 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:44.175 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:44.176 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:44.176 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:44.176 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:44.176 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 227 2024-09-21T21:40:44.176 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.176 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:44.176 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.176 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:44.176 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.553729+0000", 2024-09-21T21:40:44.176 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:44.176 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.176 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.176 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:44.176 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.553729+0000", 2024-09-21T21:40:44.176 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:44.177 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.177 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.177 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:44.177 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.553729+0000", 2024-09-21T21:40:44.177 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.06e-07 2024-09-21T21:40:44.177 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.177 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.177 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:44.177 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.553771+0000", 2024-09-21T21:40:44.177 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2042999999999997e-05 2024-09-21T21:40:44.177 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.177 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.177 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:44.177 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.553779+0000", 2024-09-21T21:40:44.178 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.3440000000000003e-06 2024-09-21T21:40:44.178 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.178 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.178 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:44.178 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.553784+0000", 2024-09-21T21:40:44.178 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3419999999999998e-06 2024-09-21T21:40:44.178 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.178 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.178 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:44.178 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.553831+0000", 2024-09-21T21:40:44.178 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6826000000000002e-05 2024-09-21T21:40:44.178 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.178 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.179 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:44.179 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.553914+0000", 2024-09-21T21:40:44.179 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.2816000000000001e-05 2024-09-21T21:40:44.179 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.179 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.179 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 1", 2024-09-21T21:40:44.179 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.554149+0000", 2024-09-21T21:40:44.179 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00023488900000000001 2024-09-21T21:40:44.179 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.179 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.179 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:44.179 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.558852+0000", 2024-09-21T21:40:44.179 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0047028399999999998 2024-09-21T21:40:44.180 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.180 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.180 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:40:44.180 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.562673+0000", 2024-09-21T21:40:44.180 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0038215929999999999 2024-09-21T21:40:44.180 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.180 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.180 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:44.180 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.562717+0000", 2024-09-21T21:40:44.180 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4255999999999998e-05 2024-09-21T21:40:44.180 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.180 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.180 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:44.180 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.562770+0000", 2024-09-21T21:40:44.181 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2689000000000003e-05 2024-09-21T21:40:44.181 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:44.181 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:44.181 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:44.181 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.181 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.181 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:228 3.6 3:678f83b3:::rbd_data.11214be7c411.000000000000002a:head [delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:44.181 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.553796+0000", 2024-09-21T21:40:44.181 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.56777160699999996, 2024-09-21T21:40:44.181 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0098649099999999993, 2024-09-21T21:40:44.182 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:44.182 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:44.182 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:44.182 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:44.182 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:44.182 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:44.182 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 228 2024-09-21T21:40:44.182 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.182 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:44.182 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.182 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:44.182 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.553796+0000", 2024-09-21T21:40:44.182 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:44.182 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.183 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.183 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:44.183 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.553796+0000", 2024-09-21T21:40:44.183 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:44.183 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.183 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.183 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:44.183 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.553797+0000", 2024-09-21T21:40:44.183 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6899999999999997e-07 2024-09-21T21:40:44.183 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.183 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.183 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:44.183 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.553799+0000", 2024-09-21T21:40:44.183 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3e-06 2024-09-21T21:40:44.184 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.184 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.184 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:44.184 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.553805+0000", 2024-09-21T21:40:44.184 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1310000000000003e-06 2024-09-21T21:40:44.184 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.184 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.184 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:44.184 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.553816+0000", 2024-09-21T21:40:44.184 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0506000000000001e-05 2024-09-21T21:40:44.184 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.184 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.184 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:44.184 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.555870+0000", 2024-09-21T21:40:44.185 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0020538510000000002 2024-09-21T21:40:44.185 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.185 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.185 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:44.185 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.555942+0000", 2024-09-21T21:40:44.185 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.2436999999999998e-05 2024-09-21T21:40:44.185 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.185 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.185 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 1", 2024-09-21T21:40:44.185 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556121+0000", 2024-09-21T21:40:44.185 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000178526 2024-09-21T21:40:44.185 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.185 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.185 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:44.186 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.559481+0000", 2024-09-21T21:40:44.186 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0033603019999999999 2024-09-21T21:40:44.186 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.186 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.186 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:40:44.186 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.563144+0000", 2024-09-21T21:40:44.186 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0036625799999999999 2024-09-21T21:40:44.186 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.186 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.186 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:44.186 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.563185+0000", 2024-09-21T21:40:44.186 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1084000000000002e-05 2024-09-21T21:40:44.186 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.186 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.187 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:44.187 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.563661+0000", 2024-09-21T21:40:44.187 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00047662399999999998 2024-09-21T21:40:44.187 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:44.187 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:44.187 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:44.187 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.187 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.187 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "pg_update_log_missing(3.b epoch 26/21 rep_tid 94 entries 26'3 (0'0) error 3:d29f5c36:::rbd_data.11214be7c411.000000000000003b:head by client.4417.0:236 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 26'1)", 2024-09-21T21:40:44.187 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.556653+0000", 2024-09-21T21:40:44.187 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.56491539499999999, 2024-09-21T21:40:44.187 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.006055749, 2024-09-21T21:40:44.187 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:44.187 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:44.188 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:40:44.188 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:44.188 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.188 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:44.188 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556653+0000", 2024-09-21T21:40:44.188 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:44.188 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.188 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.188 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:44.188 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556653+0000", 2024-09-21T21:40:44.188 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:44.188 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.188 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.188 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:44.188 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556653+0000", 2024-09-21T21:40:44.189 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0900000000000001e-07 2024-09-21T21:40:44.189 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.189 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.189 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:44.189 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556661+0000", 2024-09-21T21:40:44.189 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.746e-06 2024-09-21T21:40:44.189 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.189 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.189 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:44.189 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556679+0000", 2024-09-21T21:40:44.189 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.8379000000000001e-05 2024-09-21T21:40:44.189 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.189 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.190 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:44.190 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556694+0000", 2024-09-21T21:40:44.190 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.4440000000000001e-05 2024-09-21T21:40:44.190 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.190 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.190 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:44.190 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556784+0000", 2024-09-21T21:40:44.190 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.9955000000000002e-05 2024-09-21T21:40:44.190 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.190 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.190 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:44.190 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.562708+0000", 2024-09-21T21:40:44.190 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00592452 2024-09-21T21:40:44.190 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:44.190 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:44.191 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:44.191 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.191 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.191 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_repop(client.4417.0:246 3.18 e26/21 3:18a859ac:::rbd_data.11214be7c411.0000000000000036:head v 26'6, mlcod=24'4)", 2024-09-21T21:40:44.191 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.557271+0000", 2024-09-21T21:40:44.191 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.56429655099999998, 2024-09-21T21:40:44.191 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0054572550000000003, 2024-09-21T21:40:44.191 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:44.191 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:44.191 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:44.191 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:44.191 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.191 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:44.191 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.557271+0000", 2024-09-21T21:40:44.192 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:44.192 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.192 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.192 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:44.192 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.557271+0000", 2024-09-21T21:40:44.192 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:44.192 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.192 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.192 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:44.192 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.557272+0000", 2024-09-21T21:40:44.192 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.06e-07 2024-09-21T21:40:44.192 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.192 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.192 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:44.193 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.557279+0000", 2024-09-21T21:40:44.193 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.4649999999999997e-06 2024-09-21T21:40:44.193 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.193 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.193 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:44.193 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.557289+0000", 2024-09-21T21:40:44.193 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.4190000000000008e-06 2024-09-21T21:40:44.193 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.193 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.193 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:44.193 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.557295+0000", 2024-09-21T21:40:44.193 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5950000000000004e-06 2024-09-21T21:40:44.193 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.193 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.194 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:44.194 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.557345+0000", 2024-09-21T21:40:44.194 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9208e-05 2024-09-21T21:40:44.194 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.194 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.194 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:44.194 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.557363+0000", 2024-09-21T21:40:44.194 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.8717000000000001e-05 2024-09-21T21:40:44.194 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.194 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.194 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:44.194 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.562693+0000", 2024-09-21T21:40:44.194 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0053298690000000001 2024-09-21T21:40:44.194 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.195 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.195 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:44.195 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.562729+0000", 2024-09-21T21:40:44.195 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5376000000000001e-05 2024-09-21T21:40:44.195 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:44.195 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:44.195 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:44.195 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.195 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.195 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:288 3.12 3:4db597f3:::rbd_data.11214be7c411.0000000000000048:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:44.195 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.565848+0000", 2024-09-21T21:40:44.195 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.55571956600000005, 2024-09-21T21:40:44.195 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0059913800000000001, 2024-09-21T21:40:44.195 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:44.196 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:44.196 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:40:44.196 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:44.196 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:44.196 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:44.196 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 288 2024-09-21T21:40:44.196 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.196 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:44.196 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.196 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:44.196 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.565848+0000", 2024-09-21T21:40:44.196 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:44.196 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.196 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.197 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:44.197 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.565848+0000", 2024-09-21T21:40:44.197 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:44.197 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.197 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.197 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:44.197 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.565849+0000", 2024-09-21T21:40:44.197 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0130000000000001e-06 2024-09-21T21:40:44.197 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.197 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.197 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:44.197 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.565854+0000", 2024-09-21T21:40:44.197 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.853e-06 2024-09-21T21:40:44.197 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.198 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.198 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:44.198 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.565867+0000", 2024-09-21T21:40:44.198 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.3208e-05 2024-09-21T21:40:44.198 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.198 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.198 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:44.198 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.565875+0000", 2024-09-21T21:40:44.198 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7840000000000004e-06 2024-09-21T21:40:44.198 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.198 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.198 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:44.198 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.565926+0000", 2024-09-21T21:40:44.198 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0346000000000002e-05 2024-09-21T21:40:44.199 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.199 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.199 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:44.199 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.566046+0000", 2024-09-21T21:40:44.199 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00012065000000000001 2024-09-21T21:40:44.199 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.199 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.199 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:44.199 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.571840+0000", 2024-09-21T21:40:44.199 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0057935260000000002 2024-09-21T21:40:44.199 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:44.199 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:44.199 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:44.199 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.200 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.200 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:309 3.1b 3:dd2a1785:::rbd_data.11214be7c411.000000000000004b:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:44.200 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.569250+0000", 2024-09-21T21:40:44.200 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.55231750199999996, 2024-09-21T21:40:44.200 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0051863389999999999, 2024-09-21T21:40:44.200 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:44.200 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:44.200 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:40:44.200 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:44.200 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:44.200 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:44.200 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 309 2024-09-21T21:40:44.200 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.200 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:44.201 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.201 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:44.201 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.569250+0000", 2024-09-21T21:40:44.201 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:44.201 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.201 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.201 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:44.201 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.569250+0000", 2024-09-21T21:40:44.201 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:44.201 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.201 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.201 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:44.201 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.569251+0000", 2024-09-21T21:40:44.201 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0399999999999995e-07 2024-09-21T21:40:44.202 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.202 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.202 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:44.202 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.569255+0000", 2024-09-21T21:40:44.202 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.121e-06 2024-09-21T21:40:44.202 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.202 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.202 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:44.202 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.569265+0000", 2024-09-21T21:40:44.202 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0068000000000001e-05 2024-09-21T21:40:44.202 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.202 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.202 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:44.202 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.569271+0000", 2024-09-21T21:40:44.203 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5999999999999997e-06 2024-09-21T21:40:44.203 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.203 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.203 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:44.203 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.569322+0000", 2024-09-21T21:40:44.203 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0776000000000003e-05 2024-09-21T21:40:44.203 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.203 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.203 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:44.203 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.569454+0000", 2024-09-21T21:40:44.203 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00013230699999999999 2024-09-21T21:40:44.203 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.203 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.203 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:44.203 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.574437+0000", 2024-09-21T21:40:44.204 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0049827630000000003 2024-09-21T21:40:44.204 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:44.204 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:44.204 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:44.204 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.204 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.204 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:344 3.1b 3:d9520152:::rbd_data.11214be7c411.000000000000004d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:44.204 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.576793+0000", 2024-09-21T21:40:44.204 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.54477527000000003, 2024-09-21T21:40:44.204 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0058611469999999997, 2024-09-21T21:40:44.204 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:44.204 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:44.204 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:44.204 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:44.205 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:44.205 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:44.205 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 344 2024-09-21T21:40:44.205 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.205 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:44.205 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.205 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:44.205 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.576793+0000", 2024-09-21T21:40:44.205 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:44.205 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.205 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.205 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:44.205 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.576793+0000", 2024-09-21T21:40:44.205 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:44.206 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.206 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.206 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:44.206 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.576793+0000", 2024-09-21T21:40:44.206 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.4499999999999996e-07 2024-09-21T21:40:44.206 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.206 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.206 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:44.206 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.576799+0000", 2024-09-21T21:40:44.206 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2299999999999999e-06 2024-09-21T21:40:44.206 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.206 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.206 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:44.206 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.576811+0000", 2024-09-21T21:40:44.207 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.2213e-05 2024-09-21T21:40:44.207 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.207 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.207 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:44.207 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.576819+0000", 2024-09-21T21:40:44.207 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7230000000000004e-06 2024-09-21T21:40:44.207 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.207 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.207 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:44.207 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.576865+0000", 2024-09-21T21:40:44.207 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5954000000000001e-05 2024-09-21T21:40:44.207 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.207 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.207 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:44.208 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.576946+0000", 2024-09-21T21:40:44.208 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.1689999999999999e-05 2024-09-21T21:40:44.208 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.208 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.208 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 4", 2024-09-21T21:40:44.208 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.577144+0000", 2024-09-21T21:40:44.208 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00019760599999999999 2024-09-21T21:40:44.208 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.208 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.208 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:40:44.208 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.579276+0000", 2024-09-21T21:40:44.208 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0021324009999999999 2024-09-21T21:40:44.208 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.208 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.208 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:44.209 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.582551+0000", 2024-09-21T21:40:44.209 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.003275086 2024-09-21T21:40:44.209 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.209 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.210 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:44.210 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.582610+0000", 2024-09-21T21:40:44.210 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8186999999999997e-05 2024-09-21T21:40:44.210 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.210 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.211 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:44.211 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.582654+0000", 2024-09-21T21:40:44.211 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4212000000000003e-05 2024-09-21T21:40:44.211 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:44.211 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:44.211 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:44.211 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.211 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.211 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:350 3.1b 3:db169869:::rbd_data.11214be7c411.0000000000000053:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:44.211 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.577617+0000", 2024-09-21T21:40:44.211 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.54395046499999999, 2024-09-21T21:40:44.211 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0053647620000000003, 2024-09-21T21:40:44.211 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:44.211 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:44.211 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:44.212 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:44.212 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:44.212 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:44.212 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 350 2024-09-21T21:40:44.212 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.212 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:44.212 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.212 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:44.212 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.577617+0000", 2024-09-21T21:40:44.212 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:44.212 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.212 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.212 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:44.212 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.577617+0000", 2024-09-21T21:40:44.213 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:44.213 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.213 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.213 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:44.213 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.577618+0000", 2024-09-21T21:40:44.213 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.8100000000000001e-07 2024-09-21T21:40:44.213 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.213 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.213 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:44.213 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.577623+0000", 2024-09-21T21:40:44.213 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8180000000000003e-06 2024-09-21T21:40:44.213 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.213 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.213 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:44.214 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.577641+0000", 2024-09-21T21:40:44.214 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.7986e-05 2024-09-21T21:40:44.214 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.214 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.214 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:44.214 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.577659+0000", 2024-09-21T21:40:44.214 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.8209000000000001e-05 2024-09-21T21:40:44.214 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.214 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.214 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:44.214 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.577838+0000", 2024-09-21T21:40:44.214 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00017856999999999999 2024-09-21T21:40:44.214 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.214 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.215 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:44.215 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.577886+0000", 2024-09-21T21:40:44.215 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8486999999999998e-05 2024-09-21T21:40:44.215 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.215 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.215 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 4", 2024-09-21T21:40:44.215 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.578110+0000", 2024-09-21T21:40:44.215 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000223277 2024-09-21T21:40:44.215 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.215 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.215 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:40:44.215 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.582200+0000", 2024-09-21T21:40:44.215 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0040905530000000002 2024-09-21T21:40:44.215 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.216 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.216 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:44.216 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.582886+0000", 2024-09-21T21:40:44.216 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00068611699999999998 2024-09-21T21:40:44.216 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.216 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.216 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:44.216 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.582943+0000", 2024-09-21T21:40:44.216 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6909e-05 2024-09-21T21:40:44.216 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.216 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.216 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:44.216 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.582982+0000", 2024-09-21T21:40:44.216 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8955000000000002e-05 2024-09-21T21:40:44.216 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:44.217 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:44.217 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:44.217 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.217 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.217 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:436 3.17 3:ea5ebfbf:::rbd_data.11214be7c411.000000000000006c:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:44.217 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.590210+0000", 2024-09-21T21:40:44.217 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.53135764100000005, 2024-09-21T21:40:44.217 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0083415569999999994, 2024-09-21T21:40:44.217 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:44.217 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:44.217 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:40:44.217 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:44.217 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:44.218 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:44.218 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 436 2024-09-21T21:40:44.218 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.218 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:44.218 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.218 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:44.218 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.590210+0000", 2024-09-21T21:40:44.218 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:44.218 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.218 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.218 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:44.218 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.590210+0000", 2024-09-21T21:40:44.218 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:44.218 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.218 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.219 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:44.219 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.590211+0000", 2024-09-21T21:40:44.219 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.8899999999999998e-07 2024-09-21T21:40:44.219 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.219 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.219 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:44.219 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.590222+0000", 2024-09-21T21:40:44.219 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0473e-05 2024-09-21T21:40:44.219 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.219 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.219 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:44.219 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.590235+0000", 2024-09-21T21:40:44.219 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.3003000000000001e-05 2024-09-21T21:40:44.219 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.220 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.220 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:44.220 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.590242+0000", 2024-09-21T21:40:44.220 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7749999999999993e-06 2024-09-21T21:40:44.220 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.220 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.220 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:44.220 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.590293+0000", 2024-09-21T21:40:44.220 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0133000000000003e-05 2024-09-21T21:40:44.220 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.220 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.220 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:44.220 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.590397+0000", 2024-09-21T21:40:44.220 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000104443 2024-09-21T21:40:44.221 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.221 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.221 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:44.221 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.598552+0000", 2024-09-21T21:40:44.221 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0081548409999999995 2024-09-21T21:40:44.221 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:44.221 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:44.221 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:44.221 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.221 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.221 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "pg_update_log_missing(3.3 epoch 26/21 rep_tid 23 entries 26'5 (0'0) error 3:c632f711:::rbd_data.11214be7c411.000000000000006f:head by client.4417.0:457 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 26'3)", 2024-09-21T21:40:44.222 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.593526+0000", 2024-09-21T21:40:44.222 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.52804215499999996, 2024-09-21T21:40:44.222 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.007353788, 2024-09-21T21:40:44.222 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:44.222 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:44.222 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:40:44.222 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:44.222 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.222 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:44.222 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593526+0000", 2024-09-21T21:40:44.222 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:44.222 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.222 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.222 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:44.223 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593526+0000", 2024-09-21T21:40:44.223 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:44.223 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.223 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.223 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:44.223 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593527+0000", 2024-09-21T21:40:44.223 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0389999999999999e-06 2024-09-21T21:40:44.223 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.223 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.223 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:44.223 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593535+0000", 2024-09-21T21:40:44.223 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.8909999999999995e-06 2024-09-21T21:40:44.223 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.223 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.224 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:44.224 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593556+0000", 2024-09-21T21:40:44.224 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0848000000000001e-05 2024-09-21T21:40:44.224 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.224 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.224 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:44.224 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593572+0000", 2024-09-21T21:40:44.224 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.6070999999999999e-05 2024-09-21T21:40:44.224 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.224 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.224 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:44.224 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593699+0000", 2024-09-21T21:40:44.224 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000127358 2024-09-21T21:40:44.224 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.225 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.225 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:44.225 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.600880+0000", 2024-09-21T21:40:44.225 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0071805810000000001 2024-09-21T21:40:44.225 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:44.225 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:44.225 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:44.225 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.225 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.225 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:466 3.1f 3:f854b118:::rbd_data.11214be7c411.0000000000000070:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:44.225 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.595543+0000", 2024-09-21T21:40:44.225 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.52602444199999998, 2024-09-21T21:40:44.225 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0053899910000000002, 2024-09-21T21:40:44.225 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:44.225 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:44.226 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:40:44.226 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:44.226 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:44.226 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:44.226 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 466 2024-09-21T21:40:44.226 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.226 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:44.226 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.226 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:44.226 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.595543+0000", 2024-09-21T21:40:44.226 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:44.226 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.226 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.226 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:44.227 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.595543+0000", 2024-09-21T21:40:44.227 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:44.227 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.227 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.227 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:44.227 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.595544+0000", 2024-09-21T21:40:44.227 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4700000000000001e-07 2024-09-21T21:40:44.227 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.227 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.227 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:44.227 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.595548+0000", 2024-09-21T21:40:44.227 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8759999999999998e-06 2024-09-21T21:40:44.227 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.227 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.228 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:44.228 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.595557+0000", 2024-09-21T21:40:44.228 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.833e-06 2024-09-21T21:40:44.228 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.228 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.228 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:44.228 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.595562+0000", 2024-09-21T21:40:44.228 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7520000000000004e-06 2024-09-21T21:40:44.228 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.228 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.228 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:44.228 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.595612+0000", 2024-09-21T21:40:44.228 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9960000000000003e-05 2024-09-21T21:40:44.228 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.229 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.229 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:44.229 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.595695+0000", 2024-09-21T21:40:44.229 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.3347e-05 2024-09-21T21:40:44.229 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.229 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.229 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:44.229 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.600933+0000", 2024-09-21T21:40:44.229 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.005238576 2024-09-21T21:40:44.229 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:44.229 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:44.229 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:44.229 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.229 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.230 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:475 3.17 3:ea5ebfbf:::rbd_data.11214be7c411.000000000000006c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:44.230 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.598715+0000", 2024-09-21T21:40:44.230 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.52285287700000005, 2024-09-21T21:40:44.230 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0051441300000000002, 2024-09-21T21:40:44.230 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:44.230 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:44.230 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:44.230 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:44.230 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:44.230 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:44.230 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 475 2024-09-21T21:40:44.230 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.230 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:44.230 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.231 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:44.231 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.598715+0000", 2024-09-21T21:40:44.231 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:44.231 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.231 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.231 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:44.231 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.598715+0000", 2024-09-21T21:40:44.231 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:44.231 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.231 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.231 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:44.231 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.598716+0000", 2024-09-21T21:40:44.231 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.4099999999999997e-07 2024-09-21T21:40:44.232 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.232 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.232 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:44.232 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.598721+0000", 2024-09-21T21:40:44.232 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2259999999999996e-06 2024-09-21T21:40:44.232 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.232 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.232 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:44.232 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.598735+0000", 2024-09-21T21:40:44.232 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.3838e-05 2024-09-21T21:40:44.232 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.232 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.232 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:44.232 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.598743+0000", 2024-09-21T21:40:44.233 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.4850000000000003e-06 2024-09-21T21:40:44.233 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.233 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.233 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:44.233 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.598798+0000", 2024-09-21T21:40:44.233 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5546e-05 2024-09-21T21:40:44.233 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.233 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.233 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:44.233 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.598844+0000", 2024-09-21T21:40:44.233 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5940999999999999e-05 2024-09-21T21:40:44.234 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.234 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.234 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:40:44.234 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.598957+0000", 2024-09-21T21:40:44.234 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000112871 2024-09-21T21:40:44.234 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.234 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.234 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:44.234 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.602463+0000", 2024-09-21T21:40:44.234 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0035060059999999999 2024-09-21T21:40:44.234 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.234 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.234 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:40:44.235 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.603772+0000", 2024-09-21T21:40:44.235 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.001309005 2024-09-21T21:40:44.235 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.235 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.235 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:44.235 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.603822+0000", 2024-09-21T21:40:44.235 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0365999999999999e-05 2024-09-21T21:40:44.235 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.235 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.235 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:44.235 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.603859+0000", 2024-09-21T21:40:44.235 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6905e-05 2024-09-21T21:40:44.235 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:44.235 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:44.235 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:44.236 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:44.236 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:44.236 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:40:44.237 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T21:40:44.237 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:40:44.235+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:40:44.338 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:40:44.336+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:40:44.382 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:40:44.382 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:40:44.382 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 0 2024-09-21T21:40:44.382 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:40:44.391 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T21:40:44.439 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:40:44.437+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:40:44.452 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T21:40:44.496 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:40:44.496 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:40:44.496 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:40:44.496 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:40:44.496 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:40:44.505 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T21:40:44.521 INFO:teuthology.orchestra.run.smithi164.stdout:138 clone 3 order 21 su 8192 sc 12 2024-09-21T21:40:44.546 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:40:44.545+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:40:44.641 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:40:44.639+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:40:44.651 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:40:44.651 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 20, 2024-09-21T21:40:44.651 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 600, 2024-09-21T21:40:44.651 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:40:44.651 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.651 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:124 3.2 3:42a46321:::rbd_data.11214be7c411.0000000000000020:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:44.651 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.534750+0000", 2024-09-21T21:40:44.651 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 1.11379508, 2024-09-21T21:40:44.651 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0073162330000000001, 2024-09-21T21:40:44.652 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:44.652 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:44.652 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:40:44.652 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:44.652 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:44.652 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:44.652 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 124 2024-09-21T21:40:44.652 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.652 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:44.652 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.652 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:44.652 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.534750+0000", 2024-09-21T21:40:44.652 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:44.652 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.653 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.653 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:44.653 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.534750+0000", 2024-09-21T21:40:44.653 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:44.653 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.653 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.653 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:44.653 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.534751+0000", 2024-09-21T21:40:44.653 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4300000000000003e-07 2024-09-21T21:40:44.653 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.653 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.653 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:44.653 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.534753+0000", 2024-09-21T21:40:44.653 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.294e-06 2024-09-21T21:40:44.654 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.654 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.654 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:44.654 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.534759+0000", 2024-09-21T21:40:44.654 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3409999999999999e-06 2024-09-21T21:40:44.654 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.654 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.654 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:44.654 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.534764+0000", 2024-09-21T21:40:44.655 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9490000000000004e-06 2024-09-21T21:40:44.655 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.655 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.655 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:44.655 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.534817+0000", 2024-09-21T21:40:44.655 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2954000000000002e-05 2024-09-21T21:40:44.655 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.655 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.655 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:44.655 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.534924+0000", 2024-09-21T21:40:44.656 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000106884 2024-09-21T21:40:44.656 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.656 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.656 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:44.656 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.542067+0000", 2024-09-21T21:40:44.656 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0071422680000000002 2024-09-21T21:40:44.656 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:44.656 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:44.656 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:44.656 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.656 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.656 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "pg_update_log_missing(3.8 epoch 26/21 rep_tid 13 entries 26'4 (0'0) error 3:108e7c9d:::rbd_data.11214be7c411.000000000000001f:head by client.4417.0:122 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 25'2)", 2024-09-21T21:40:44.656 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.535157+0000", 2024-09-21T21:40:44.657 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 1.113388346, 2024-09-21T21:40:44.657 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0068721269999999996, 2024-09-21T21:40:44.657 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:44.657 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:44.657 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:40:44.657 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:44.657 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.657 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:44.657 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.535157+0000", 2024-09-21T21:40:44.657 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:44.657 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.657 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.657 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:44.657 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.535157+0000", 2024-09-21T21:40:44.658 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:44.658 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.658 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.658 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:44.658 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.535158+0000", 2024-09-21T21:40:44.658 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7999999999999995e-07 2024-09-21T21:40:44.658 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.658 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.658 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:44.658 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.535165+0000", 2024-09-21T21:40:44.658 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.2869999999999999e-06 2024-09-21T21:40:44.658 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.658 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.658 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:44.659 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.535179+0000", 2024-09-21T21:40:44.659 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.4092e-05 2024-09-21T21:40:44.659 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.659 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.659 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:44.659 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.535192+0000", 2024-09-21T21:40:44.659 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.3107e-05 2024-09-21T21:40:44.659 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.659 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.660 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:44.660 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.535257+0000", 2024-09-21T21:40:44.660 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4543999999999997e-05 2024-09-21T21:40:44.660 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.660 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.660 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:44.660 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.542029+0000", 2024-09-21T21:40:44.660 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0067725169999999996 2024-09-21T21:40:44.660 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:44.660 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:44.660 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:44.660 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.660 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.660 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:147 3.19 3:9f7f40ef:::rbd_data.11214be7c411.000000000000001d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:44.661 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.536972+0000", 2024-09-21T21:40:44.661 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 1.111573022, 2024-09-21T21:40:44.661 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0078089780000000003, 2024-09-21T21:40:44.661 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:44.661 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:44.661 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:44.661 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:44.661 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:44.661 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:44.661 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 147 2024-09-21T21:40:44.661 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.661 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:44.662 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.662 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:44.662 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.536972+0000", 2024-09-21T21:40:44.662 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:44.662 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.662 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.662 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:44.662 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.536972+0000", 2024-09-21T21:40:44.662 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:44.662 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.662 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.662 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:44.662 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.536973+0000", 2024-09-21T21:40:44.662 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.9999999999999998e-07 2024-09-21T21:40:44.663 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.663 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.663 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:44.663 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.536977+0000", 2024-09-21T21:40:44.663 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8850000000000001e-06 2024-09-21T21:40:44.663 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.663 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.663 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:44.663 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.536982+0000", 2024-09-21T21:40:44.663 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6590000000000004e-06 2024-09-21T21:40:44.663 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.663 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.663 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:44.663 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.536986+0000", 2024-09-21T21:40:44.663 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4999999999999999e-06 2024-09-21T21:40:44.664 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.664 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.664 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:44.664 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.539409+0000", 2024-09-21T21:40:44.664 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0024234259999999998 2024-09-21T21:40:44.664 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.664 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.664 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:44.664 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.539480+0000", 2024-09-21T21:40:44.664 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.1181999999999997e-05 2024-09-21T21:40:44.664 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.664 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.665 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 4", 2024-09-21T21:40:44.665 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.539597+0000", 2024-09-21T21:40:44.665 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000116239 2024-09-21T21:40:44.665 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.665 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.665 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:44.665 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.541542+0000", 2024-09-21T21:40:44.665 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.001945801 2024-09-21T21:40:44.665 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.665 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.666 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:40:44.666 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.544698+0000", 2024-09-21T21:40:44.666 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.003156042 2024-09-21T21:40:44.666 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.666 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.666 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:44.666 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.544747+0000", 2024-09-21T21:40:44.666 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8869000000000003e-05 2024-09-21T21:40:44.666 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.666 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.666 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:44.666 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.544781+0000", 2024-09-21T21:40:44.666 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3974999999999999e-05 2024-09-21T21:40:44.667 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:44.667 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:44.667 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:44.667 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.667 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.667 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "pg_update_log_missing(3.8 epoch 26/21 rep_tid 15 entries 26'5 (0'0) error 3:15b4ee09:::rbd_data.11214be7c411.0000000000000028:head by client.4417.0:154 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 25'2)", 2024-09-21T21:40:44.667 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.538536+0000", 2024-09-21T21:40:44.667 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 1.1100092269999999, 2024-09-21T21:40:44.667 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0061585449999999996, 2024-09-21T21:40:44.667 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:44.667 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:44.667 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:40:44.667 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:44.667 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.668 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:44.668 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.538536+0000", 2024-09-21T21:40:44.668 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:44.668 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.668 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.668 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:44.668 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.538536+0000", 2024-09-21T21:40:44.668 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:44.668 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.668 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.668 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:44.668 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.538537+0000", 2024-09-21T21:40:44.668 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9200000000000001e-07 2024-09-21T21:40:44.668 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.669 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.669 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:44.669 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.538547+0000", 2024-09-21T21:40:44.669 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0319999999999999e-05 2024-09-21T21:40:44.669 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.669 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.669 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:44.669 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.538565+0000", 2024-09-21T21:40:44.669 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.8080999999999998e-05 2024-09-21T21:40:44.669 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.669 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.669 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:44.669 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.538582+0000", 2024-09-21T21:40:44.670 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.7220000000000001e-05 2024-09-21T21:40:44.670 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.670 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.670 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:44.670 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.538644+0000", 2024-09-21T21:40:44.670 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1383999999999994e-05 2024-09-21T21:40:44.670 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.670 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.670 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:44.670 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.544695+0000", 2024-09-21T21:40:44.670 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0060509479999999996 2024-09-21T21:40:44.670 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:44.670 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:44.670 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:44.671 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.671 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.671 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:195 3.19 3:9c6b57eb:::rbd_object_map.11214be7c411.0000000000000004:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:44.671 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.546530+0000", 2024-09-21T21:40:44.671 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 1.1020149930000001, 2024-09-21T21:40:44.671 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0067606400000000001, 2024-09-21T21:40:44.671 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:44.671 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:44.671 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:44.671 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:44.671 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:44.671 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:44.671 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 195 2024-09-21T21:40:44.671 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.671 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:44.672 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.672 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:44.672 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.546530+0000", 2024-09-21T21:40:44.672 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:44.672 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.672 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.672 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:44.672 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.546530+0000", 2024-09-21T21:40:44.672 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:44.672 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.672 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.672 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:44.672 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.546531+0000", 2024-09-21T21:40:44.672 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1399999999999997e-07 2024-09-21T21:40:44.672 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.673 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.673 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:44.673 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.546534+0000", 2024-09-21T21:40:44.673 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.7989999999999998e-06 2024-09-21T21:40:44.673 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.673 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.673 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:44.673 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.546540+0000", 2024-09-21T21:40:44.673 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.314e-06 2024-09-21T21:40:44.673 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.673 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.673 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:44.673 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.546544+0000", 2024-09-21T21:40:44.673 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.546e-06 2024-09-21T21:40:44.673 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.674 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.674 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:44.674 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.547355+0000", 2024-09-21T21:40:44.674 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00081132899999999996 2024-09-21T21:40:44.674 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.674 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.674 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:44.674 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.547400+0000", 2024-09-21T21:40:44.674 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5627000000000002e-05 2024-09-21T21:40:44.674 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.674 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.674 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 4", 2024-09-21T21:40:44.674 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.547755+0000", 2024-09-21T21:40:44.674 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00035492899999999998 2024-09-21T21:40:44.675 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.675 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.675 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:44.675 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.549411+0000", 2024-09-21T21:40:44.675 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0016557239999999999 2024-09-21T21:40:44.675 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.675 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.675 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:40:44.675 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.553193+0000", 2024-09-21T21:40:44.675 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0037821370000000001 2024-09-21T21:40:44.676 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.676 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.676 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:44.676 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.553256+0000", 2024-09-21T21:40:44.676 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2534999999999997e-05 2024-09-21T21:40:44.676 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.676 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.676 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:44.676 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.553291+0000", 2024-09-21T21:40:44.676 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5185999999999998e-05 2024-09-21T21:40:44.676 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:44.676 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:44.676 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:44.676 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.676 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.677 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:236 3.b 3:d29f5c36:::rbd_data.11214be7c411.000000000000003b:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:44.677 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.556219+0000", 2024-09-21T21:40:44.677 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 1.092326688, 2024-09-21T21:40:44.677 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0068065230000000001, 2024-09-21T21:40:44.677 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:44.677 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:44.677 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:40:44.677 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:44.677 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:44.677 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:44.677 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 236 2024-09-21T21:40:44.677 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.677 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:44.677 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.678 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:44.678 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556219+0000", 2024-09-21T21:40:44.678 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:44.678 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.678 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.678 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:44.678 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556219+0000", 2024-09-21T21:40:44.678 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:44.678 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.678 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.678 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:44.678 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556219+0000", 2024-09-21T21:40:44.678 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.9899999999999999e-07 2024-09-21T21:40:44.678 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.679 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.679 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:44.679 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556223+0000", 2024-09-21T21:40:44.679 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7910000000000002e-06 2024-09-21T21:40:44.679 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.679 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.679 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:44.679 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556235+0000", 2024-09-21T21:40:44.679 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1694e-05 2024-09-21T21:40:44.679 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.679 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.679 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:44.679 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556240+0000", 2024-09-21T21:40:44.679 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5199999999999997e-06 2024-09-21T21:40:44.680 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.680 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.680 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:44.680 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556300+0000", 2024-09-21T21:40:44.680 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9963000000000002e-05 2024-09-21T21:40:44.680 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.680 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.680 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:44.680 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556409+0000", 2024-09-21T21:40:44.680 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000108441 2024-09-21T21:40:44.680 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.680 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.680 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:44.680 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.563025+0000", 2024-09-21T21:40:44.680 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0066163150000000002 2024-09-21T21:40:44.681 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:44.681 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:44.681 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:44.681 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.681 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.681 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:252 3.0 3:01bf271b:::rbd_data.11214be7c411.0000000000000038:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:44.681 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.559092+0000", 2024-09-21T21:40:44.681 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 1.0894529260000001, 2024-09-21T21:40:44.681 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0066292950000000003, 2024-09-21T21:40:44.681 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:44.681 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:44.681 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:44.681 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:44.681 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:44.681 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:44.682 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 252 2024-09-21T21:40:44.682 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.682 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:44.682 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.682 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:44.682 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.559092+0000", 2024-09-21T21:40:44.682 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:44.682 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.682 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.682 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:44.682 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.559092+0000", 2024-09-21T21:40:44.682 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:44.682 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.682 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.683 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:44.683 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.559093+0000", 2024-09-21T21:40:44.683 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.1900000000000001e-07 2024-09-21T21:40:44.683 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.683 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.683 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:44.683 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.559098+0000", 2024-09-21T21:40:44.683 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6399999999999996e-06 2024-09-21T21:40:44.683 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.683 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.683 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:44.683 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.559108+0000", 2024-09-21T21:40:44.683 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6399999999999992e-06 2024-09-21T21:40:44.683 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.683 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.684 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:44.684 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.559121+0000", 2024-09-21T21:40:44.684 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.3687e-05 2024-09-21T21:40:44.684 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.684 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.684 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:44.684 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.559195+0000", 2024-09-21T21:40:44.684 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.3380999999999994e-05 2024-09-21T21:40:44.684 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.684 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.684 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:44.684 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.559273+0000", 2024-09-21T21:40:44.684 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.8236000000000001e-05 2024-09-21T21:40:44.684 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.685 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.685 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 2", 2024-09-21T21:40:44.685 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.559569+0000", 2024-09-21T21:40:44.685 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00029590600000000002 2024-09-21T21:40:44.685 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.685 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.685 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:40:44.685 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.563070+0000", 2024-09-21T21:40:44.685 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0035011880000000001 2024-09-21T21:40:44.685 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.685 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.686 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:44.686 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.565647+0000", 2024-09-21T21:40:44.686 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.002576877 2024-09-21T21:40:44.686 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.686 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.686 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:44.686 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.565692+0000", 2024-09-21T21:40:44.686 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.481e-05 2024-09-21T21:40:44.686 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.686 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.686 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:44.686 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.565722+0000", 2024-09-21T21:40:44.686 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0111e-05 2024-09-21T21:40:44.686 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:44.687 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:44.687 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:44.687 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.687 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.687 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:400 3.19 3:9c6b57eb:::rbd_object_map.11214be7c411.0000000000000004:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:44.687 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.586210+0000", 2024-09-21T21:40:44.687 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 1.062335399, 2024-09-21T21:40:44.687 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0068241939999999996, 2024-09-21T21:40:44.687 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:44.687 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:44.687 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:44.687 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:44.687 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:44.687 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:44.688 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 400 2024-09-21T21:40:44.688 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.688 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:44.688 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.688 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:44.688 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.586210+0000", 2024-09-21T21:40:44.688 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:44.688 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.688 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.688 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:44.688 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.586210+0000", 2024-09-21T21:40:44.688 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:44.688 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.688 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.688 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:44.689 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.586211+0000", 2024-09-21T21:40:44.689 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.92e-07 2024-09-21T21:40:44.689 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.689 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.689 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:44.689 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.586215+0000", 2024-09-21T21:40:44.689 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2910000000000001e-06 2024-09-21T21:40:44.689 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.689 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.689 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:44.689 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.586223+0000", 2024-09-21T21:40:44.689 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.3319999999999992e-06 2024-09-21T21:40:44.689 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.689 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.689 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:44.690 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.586230+0000", 2024-09-21T21:40:44.690 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5649999999999998e-06 2024-09-21T21:40:44.690 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.690 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.690 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:44.690 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.587427+0000", 2024-09-21T21:40:44.690 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00119736 2024-09-21T21:40:44.690 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.690 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.690 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:44.690 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.587499+0000", 2024-09-21T21:40:44.690 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.1439000000000005e-05 2024-09-21T21:40:44.690 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.690 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.690 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 4", 2024-09-21T21:40:44.691 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.587824+0000", 2024-09-21T21:40:44.691 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00032502200000000001 2024-09-21T21:40:44.691 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.691 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.691 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:40:44.691 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.589401+0000", 2024-09-21T21:40:44.691 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0015776270000000001 2024-09-21T21:40:44.691 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.691 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.691 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:44.691 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.592976+0000", 2024-09-21T21:40:44.691 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.003574565 2024-09-21T21:40:44.691 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.691 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.692 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:44.692 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593004+0000", 2024-09-21T21:40:44.692 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.7937999999999999e-05 2024-09-21T21:40:44.692 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.692 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.692 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:44.692 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593034+0000", 2024-09-21T21:40:44.692 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0263000000000001e-05 2024-09-21T21:40:44.692 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:44.692 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:44.692 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:44.692 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.692 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.692 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:410 3.19 3:981b6e71:::rbd_data.11214be7c411.0000000000000065:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:44.692 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.587078+0000", 2024-09-21T21:40:44.693 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 1.061467715, 2024-09-21T21:40:44.693 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0060338199999999996, 2024-09-21T21:40:44.693 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:44.693 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:44.693 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:40:44.693 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:44.693 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:44.693 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:44.693 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 410 2024-09-21T21:40:44.693 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.693 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:44.693 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.693 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:44.694 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.587078+0000", 2024-09-21T21:40:44.694 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:44.694 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.694 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.694 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:44.694 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.587078+0000", 2024-09-21T21:40:44.694 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:44.694 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.694 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.694 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:44.694 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.587078+0000", 2024-09-21T21:40:44.694 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6000000000000004e-07 2024-09-21T21:40:44.694 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.695 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.695 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:44.695 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.587081+0000", 2024-09-21T21:40:44.695 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.9950000000000001e-06 2024-09-21T21:40:44.695 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.695 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.695 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:44.695 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.587091+0000", 2024-09-21T21:40:44.695 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6299999999999993e-06 2024-09-21T21:40:44.695 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.695 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.695 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:44.695 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.587097+0000", 2024-09-21T21:40:44.695 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8259999999999998e-06 2024-09-21T21:40:44.695 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.696 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.696 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:44.696 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.589045+0000", 2024-09-21T21:40:44.696 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0019484179999999999 2024-09-21T21:40:44.696 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.696 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.696 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:44.696 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.589153+0000", 2024-09-21T21:40:44.696 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000107627 2024-09-21T21:40:44.697 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.697 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.697 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:44.697 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593111+0000", 2024-09-21T21:40:44.697 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.003958764 2024-09-21T21:40:44.697 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:44.697 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:44.697 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:44.697 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.697 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.697 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:459 3.4 3:22094545:::rbd_data.11214be7c411.000000000000005c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:44.697 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.593149+0000", 2024-09-21T21:40:44.697 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 1.0553964680000001, 2024-09-21T21:40:44.697 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.010963733999999999, 2024-09-21T21:40:44.698 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:44.698 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:44.698 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:44.698 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:44.698 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:44.698 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:44.698 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 459 2024-09-21T21:40:44.698 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.698 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:44.698 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.698 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:44.698 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593149+0000", 2024-09-21T21:40:44.698 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:44.698 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.699 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.699 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:44.699 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593149+0000", 2024-09-21T21:40:44.699 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:44.699 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.699 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.699 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:44.699 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593150+0000", 2024-09-21T21:40:44.699 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4300000000000003e-07 2024-09-21T21:40:44.699 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.699 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.699 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:44.699 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593204+0000", 2024-09-21T21:40:44.699 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4868999999999999e-05 2024-09-21T21:40:44.699 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.700 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.700 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:44.700 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593213+0000", 2024-09-21T21:40:44.700 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.7479999999999999e-06 2024-09-21T21:40:44.700 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.700 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.700 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:44.700 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593217+0000", 2024-09-21T21:40:44.700 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.0409999999999999e-06 2024-09-21T21:40:44.700 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.700 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.700 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:44.700 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593249+0000", 2024-09-21T21:40:44.700 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.1897999999999999e-05 2024-09-21T21:40:44.701 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.701 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.701 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:44.701 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593312+0000", 2024-09-21T21:40:44.701 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2813999999999999e-05 2024-09-21T21:40:44.701 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.701 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.701 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 2", 2024-09-21T21:40:44.701 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593501+0000", 2024-09-21T21:40:44.701 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000188887 2024-09-21T21:40:44.701 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.701 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.701 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:40:44.701 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.594562+0000", 2024-09-21T21:40:44.701 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.001060975 2024-09-21T21:40:44.702 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.702 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.702 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:44.702 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.604018+0000", 2024-09-21T21:40:44.702 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0094560800000000004 2024-09-21T21:40:44.702 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.702 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.702 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:44.702 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.604074+0000", 2024-09-21T21:40:44.702 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6063999999999997e-05 2024-09-21T21:40:44.702 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.702 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.702 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:44.702 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.604113+0000", 2024-09-21T21:40:44.703 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8714999999999998e-05 2024-09-21T21:40:44.703 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:44.703 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:44.703 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:44.703 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.703 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.703 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:460 3.4 3:22094545:::rbd_data.11214be7c411.000000000000005c:head [delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:44.703 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.593227+0000", 2024-09-21T21:40:44.703 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 1.05531872, 2024-09-21T21:40:44.703 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.01140237, 2024-09-21T21:40:44.703 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:44.703 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:44.703 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:44.703 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:44.703 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:44.704 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:44.704 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 460 2024-09-21T21:40:44.704 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.704 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:44.704 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.704 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:44.704 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593227+0000", 2024-09-21T21:40:44.704 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:44.704 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.704 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.704 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:44.704 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593227+0000", 2024-09-21T21:40:44.704 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:44.704 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.704 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.705 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:44.705 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593227+0000", 2024-09-21T21:40:44.705 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4000000000000002e-07 2024-09-21T21:40:44.705 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.705 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.705 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:44.705 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593229+0000", 2024-09-21T21:40:44.705 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1169999999999998e-06 2024-09-21T21:40:44.705 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.705 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.705 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:44.705 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593235+0000", 2024-09-21T21:40:44.705 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0510000000000002e-06 2024-09-21T21:40:44.705 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.705 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.706 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:44.706 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593248+0000", 2024-09-21T21:40:44.706 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.2403e-05 2024-09-21T21:40:44.706 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.706 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.706 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:44.706 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593895+0000", 2024-09-21T21:40:44.706 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000647358 2024-09-21T21:40:44.706 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.706 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.706 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:44.706 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593969+0000", 2024-09-21T21:40:44.706 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.4107e-05 2024-09-21T21:40:44.706 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.707 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.707 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 2", 2024-09-21T21:40:44.707 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.594139+0000", 2024-09-21T21:40:44.707 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000169457 2024-09-21T21:40:44.707 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.707 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.707 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:40:44.707 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.596005+0000", 2024-09-21T21:40:44.707 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00186608 2024-09-21T21:40:44.707 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.707 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.707 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:44.707 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.604537+0000", 2024-09-21T21:40:44.707 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.008531861 2024-09-21T21:40:44.707 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.708 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.708 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:44.708 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.604587+0000", 2024-09-21T21:40:44.708 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0544000000000003e-05 2024-09-21T21:40:44.708 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.708 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.708 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:44.708 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.604629+0000", 2024-09-21T21:40:44.708 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1851999999999998e-05 2024-09-21T21:40:44.708 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:44.708 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:44.708 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:44.708 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.708 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.708 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:463 3.19 3:981b6e71:::rbd_data.11214be7c411.0000000000000065:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:44.709 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.593716+0000", 2024-09-21T21:40:44.709 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 1.054829639, 2024-09-21T21:40:44.709 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.006390326, 2024-09-21T21:40:44.709 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:44.709 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:44.709 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:44.709 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:44.709 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:44.709 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:44.709 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 463 2024-09-21T21:40:44.709 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.709 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:44.709 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.709 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:44.709 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593716+0000", 2024-09-21T21:40:44.710 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:44.710 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.710 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.710 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:44.710 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593716+0000", 2024-09-21T21:40:44.710 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:44.710 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.710 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.710 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:44.710 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593717+0000", 2024-09-21T21:40:44.710 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.09e-07 2024-09-21T21:40:44.710 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.710 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.710 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:44.711 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593729+0000", 2024-09-21T21:40:44.711 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.2211e-05 2024-09-21T21:40:44.711 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.711 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.711 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:44.711 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593739+0000", 2024-09-21T21:40:44.711 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.8470000000000008e-06 2024-09-21T21:40:44.711 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.711 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.711 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:44.711 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593745+0000", 2024-09-21T21:40:44.711 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5869999999999998e-06 2024-09-21T21:40:44.711 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.711 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.712 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:44.712 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.595440+0000", 2024-09-21T21:40:44.712 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.001694954 2024-09-21T21:40:44.712 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.712 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.712 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:44.712 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.595489+0000", 2024-09-21T21:40:44.712 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8782999999999997e-05 2024-09-21T21:40:44.712 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.712 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.712 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 4", 2024-09-21T21:40:44.712 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.595607+0000", 2024-09-21T21:40:44.712 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000117737 2024-09-21T21:40:44.712 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.712 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.713 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:40:44.713 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.596867+0000", 2024-09-21T21:40:44.713 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0012600210000000001 2024-09-21T21:40:44.713 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.713 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.713 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:44.713 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.600050+0000", 2024-09-21T21:40:44.713 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0031829979999999998 2024-09-21T21:40:44.713 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.713 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.713 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:44.713 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.600082+0000", 2024-09-21T21:40:44.713 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.2279e-05 2024-09-21T21:40:44.713 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.714 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.714 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:44.714 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.600106+0000", 2024-09-21T21:40:44.714 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.41e-05 2024-09-21T21:40:44.714 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:44.714 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:44.714 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:44.714 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.714 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.714 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:505 3.19 3:9c6b57eb:::rbd_object_map.11214be7c411.0000000000000004:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:44.714 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.605631+0000", 2024-09-21T21:40:44.714 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 1.042914246, 2024-09-21T21:40:44.714 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0071505190000000001, 2024-09-21T21:40:44.714 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:44.714 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:44.715 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:44.715 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:44.715 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:44.715 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:44.715 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 505 2024-09-21T21:40:44.715 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.715 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:44.715 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.715 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:44.715 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605631+0000", 2024-09-21T21:40:44.716 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:44.716 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.716 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.716 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:44.716 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605631+0000", 2024-09-21T21:40:44.716 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:44.716 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.716 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.716 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:44.716 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605632+0000", 2024-09-21T21:40:44.716 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0900000000000001e-07 2024-09-21T21:40:44.716 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.716 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.716 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:44.717 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605636+0000", 2024-09-21T21:40:44.717 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8160000000000004e-06 2024-09-21T21:40:44.717 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.717 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.717 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:44.717 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605645+0000", 2024-09-21T21:40:44.717 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.0319999999999992e-06 2024-09-21T21:40:44.717 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.717 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.717 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:44.717 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605651+0000", 2024-09-21T21:40:44.717 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9529999999999996e-06 2024-09-21T21:40:44.717 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.717 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.717 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:44.718 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.606491+0000", 2024-09-21T21:40:44.718 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000840301 2024-09-21T21:40:44.718 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.718 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.718 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:44.718 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.606571+0000", 2024-09-21T21:40:44.718 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0507999999999996e-05 2024-09-21T21:40:44.718 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.718 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.718 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 4", 2024-09-21T21:40:44.718 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.606909+0000", 2024-09-21T21:40:44.718 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00033747300000000003 2024-09-21T21:40:44.718 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.718 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.719 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:44.719 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.608367+0000", 2024-09-21T21:40:44.719 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.001458188 2024-09-21T21:40:44.719 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.719 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.719 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:40:44.719 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.612683+0000", 2024-09-21T21:40:44.719 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0043163209999999997 2024-09-21T21:40:44.719 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.719 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.719 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:44.719 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.612732+0000", 2024-09-21T21:40:44.719 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8980999999999997e-05 2024-09-21T21:40:44.719 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.720 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.720 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:44.720 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.612782+0000", 2024-09-21T21:40:44.720 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9237000000000003e-05 2024-09-21T21:40:44.720 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:44.720 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:44.720 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:44.720 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.720 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.720 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_repop(client.4417.0:504 3.8 e26/21 3:10f57124:::rbd_object_map.11214be7c411:head v 26'13, mlcod=26'10)", 2024-09-21T21:40:44.720 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.607109+0000", 2024-09-21T21:40:44.720 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 1.0414360330000001, 2024-09-21T21:40:44.720 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0085878610000000005, 2024-09-21T21:40:44.721 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:44.721 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:44.721 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:44.721 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:44.721 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.721 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:44.721 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607109+0000", 2024-09-21T21:40:44.721 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:44.721 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.721 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.721 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:44.721 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607109+0000", 2024-09-21T21:40:44.721 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:44.721 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.722 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.722 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:44.722 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607110+0000", 2024-09-21T21:40:44.722 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.82e-07 2024-09-21T21:40:44.722 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.722 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.722 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:44.722 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607115+0000", 2024-09-21T21:40:44.722 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1730000000000003e-06 2024-09-21T21:40:44.722 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.722 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.722 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:44.722 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607123+0000", 2024-09-21T21:40:44.722 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7610000000000007e-06 2024-09-21T21:40:44.722 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.723 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.723 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:44.723 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607128+0000", 2024-09-21T21:40:44.723 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6420000000000004e-06 2024-09-21T21:40:44.723 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.723 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.723 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:44.723 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607302+0000", 2024-09-21T21:40:44.723 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00017391600000000001 2024-09-21T21:40:44.723 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.723 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.723 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:44.723 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607320+0000", 2024-09-21T21:40:44.723 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.7404000000000001e-05 2024-09-21T21:40:44.723 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.724 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.724 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:44.724 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.615646+0000", 2024-09-21T21:40:44.724 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0083265130000000007 2024-09-21T21:40:44.724 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.724 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.724 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:44.724 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.615697+0000", 2024-09-21T21:40:44.724 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0869999999999999e-05 2024-09-21T21:40:44.724 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:44.724 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:44.724 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:44.724 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.724 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.724 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:513 3.19 3:9c6b57eb:::rbd_object_map.11214be7c411.0000000000000004:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:44.724 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.607160+0000", 2024-09-21T21:40:44.725 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 1.0413854490000001, 2024-09-21T21:40:44.725 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0086296150000000002, 2024-09-21T21:40:44.725 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:44.725 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:44.725 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:44.725 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:44.725 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:44.725 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:44.725 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 513 2024-09-21T21:40:44.725 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.725 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:44.725 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.725 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:44.725 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607160+0000", 2024-09-21T21:40:44.725 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:44.725 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.726 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.726 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:44.726 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607160+0000", 2024-09-21T21:40:44.726 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:44.726 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.726 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.726 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:44.726 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607161+0000", 2024-09-21T21:40:44.726 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.9399999999999993e-07 2024-09-21T21:40:44.726 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.726 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.726 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:44.726 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607165+0000", 2024-09-21T21:40:44.726 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3200000000000001e-06 2024-09-21T21:40:44.726 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.727 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.727 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:44.727 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607365+0000", 2024-09-21T21:40:44.727 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00019965799999999999 2024-09-21T21:40:44.727 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.727 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.727 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:44.727 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607374+0000", 2024-09-21T21:40:44.727 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.312e-06 2024-09-21T21:40:44.727 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.727 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.727 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:44.727 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607416+0000", 2024-09-21T21:40:44.727 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1962000000000002e-05 2024-09-21T21:40:44.727 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.728 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.728 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:44.728 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607485+0000", 2024-09-21T21:40:44.728 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9274000000000001e-05 2024-09-21T21:40:44.728 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.728 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.728 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 4", 2024-09-21T21:40:44.728 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607818+0000", 2024-09-21T21:40:44.728 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00033211299999999998 2024-09-21T21:40:44.728 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.728 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.728 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:44.728 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.608616+0000", 2024-09-21T21:40:44.728 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00079832700000000004 2024-09-21T21:40:44.728 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.729 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.729 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:40:44.729 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.615697+0000", 2024-09-21T21:40:44.729 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0070815849999999996 2024-09-21T21:40:44.729 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.729 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.729 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:44.729 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.615750+0000", 2024-09-21T21:40:44.729 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2778000000000002e-05 2024-09-21T21:40:44.729 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.729 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.729 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:44.729 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.615790+0000", 2024-09-21T21:40:44.729 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.9292000000000003e-05 2024-09-21T21:40:44.729 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:44.729 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:44.730 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:44.730 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.730 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.730 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_repop(client.4417.0:507 3.8 e26/21 3:10f57124:::rbd_object_map.11214be7c411:head v 26'14, mlcod=26'10)", 2024-09-21T21:40:44.730 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.607866+0000", 2024-09-21T21:40:44.730 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 1.0406797189999999, 2024-09-21T21:40:44.730 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0078638389999999992, 2024-09-21T21:40:44.730 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:44.730 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:44.730 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:44.730 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:44.730 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.730 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:44.730 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607866+0000", 2024-09-21T21:40:44.731 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:44.731 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.731 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.731 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:44.731 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607866+0000", 2024-09-21T21:40:44.731 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:44.731 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.731 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.731 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:44.731 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607866+0000", 2024-09-21T21:40:44.731 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6000000000000004e-07 2024-09-21T21:40:44.731 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.731 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.731 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:44.731 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607872+0000", 2024-09-21T21:40:44.731 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4820000000000002e-06 2024-09-21T21:40:44.732 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.732 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.732 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:44.732 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607882+0000", 2024-09-21T21:40:44.732 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0481e-05 2024-09-21T21:40:44.732 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.732 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.732 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:44.732 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607889+0000", 2024-09-21T21:40:44.732 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.3089999999999998e-06 2024-09-21T21:40:44.732 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.732 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.732 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:44.732 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607958+0000", 2024-09-21T21:40:44.733 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.8424999999999996e-05 2024-09-21T21:40:44.733 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.733 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.733 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:44.733 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607976+0000", 2024-09-21T21:40:44.733 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.7765e-05 2024-09-21T21:40:44.733 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.733 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.733 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:44.733 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.615700+0000", 2024-09-21T21:40:44.733 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0077241799999999998 2024-09-21T21:40:44.733 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.733 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.733 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:44.734 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.615729+0000", 2024-09-21T21:40:44.734 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.9637000000000001e-05 2024-09-21T21:40:44.734 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:44.734 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:44.734 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:44.734 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.734 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.734 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_repop(client.4417.0:508 3.8 e26/21 3:10f57124:::rbd_object_map.11214be7c411:head v 26'15, mlcod=26'10)", 2024-09-21T21:40:44.734 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.608535+0000", 2024-09-21T21:40:44.734 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 1.040009897, 2024-09-21T21:40:44.734 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0072268009999999997, 2024-09-21T21:40:44.734 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:44.734 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:44.734 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:44.734 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:44.735 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.735 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:44.735 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.608535+0000", 2024-09-21T21:40:44.735 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:44.735 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.735 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.735 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:44.735 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.608535+0000", 2024-09-21T21:40:44.735 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:44.735 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.735 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.735 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:44.735 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.608536+0000", 2024-09-21T21:40:44.735 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2600000000000002e-07 2024-09-21T21:40:44.736 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.736 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.736 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:44.736 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.608540+0000", 2024-09-21T21:40:44.736 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4530000000000004e-06 2024-09-21T21:40:44.736 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.736 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.736 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:44.736 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.608549+0000", 2024-09-21T21:40:44.736 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.3270000000000001e-06 2024-09-21T21:40:44.736 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.736 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.736 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:44.737 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.608555+0000", 2024-09-21T21:40:44.737 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1009999999999997e-06 2024-09-21T21:40:44.737 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.737 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.737 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:44.737 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.608621+0000", 2024-09-21T21:40:44.737 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6351000000000002e-05 2024-09-21T21:40:44.737 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.737 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.737 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:44.737 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.608640+0000", 2024-09-21T21:40:44.737 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.8419000000000001e-05 2024-09-21T21:40:44.737 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.738 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.738 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:44.738 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.615731+0000", 2024-09-21T21:40:44.738 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.007091226 2024-09-21T21:40:44.738 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.738 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.738 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:44.738 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.615762+0000", 2024-09-21T21:40:44.738 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.1398e-05 2024-09-21T21:40:44.738 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:44.738 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:44.738 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:44.738 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.738 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.738 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_repop(client.4417.0:517 3.1a e26/21 3:5aa0c892:::rbd_data.11214be7c411.000000000000007a:head v 26'10, mlcod=26'8)", 2024-09-21T21:40:44.739 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.608704+0000", 2024-09-21T21:40:44.739 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 1.0398417719999999, 2024-09-21T21:40:44.739 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0071660500000000002, 2024-09-21T21:40:44.739 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:44.739 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:44.739 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:44.739 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:44.739 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.739 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:44.739 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.608704+0000", 2024-09-21T21:40:44.739 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:44.739 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.739 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.739 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:44.740 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.608704+0000", 2024-09-21T21:40:44.740 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:44.740 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.740 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.740 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:44.740 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.608704+0000", 2024-09-21T21:40:44.740 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2099999999999997e-07 2024-09-21T21:40:44.740 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.740 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.740 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:44.740 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.608709+0000", 2024-09-21T21:40:44.740 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8489999999999997e-06 2024-09-21T21:40:44.740 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.740 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.741 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:44.741 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.608719+0000", 2024-09-21T21:40:44.741 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0402e-05 2024-09-21T21:40:44.741 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.741 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.741 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:44.741 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.608726+0000", 2024-09-21T21:40:44.741 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.72e-06 2024-09-21T21:40:44.741 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.741 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.741 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:44.741 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.608766+0000", 2024-09-21T21:40:44.741 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.9745e-05 2024-09-21T21:40:44.741 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.741 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.742 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:44.742 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.608785+0000", 2024-09-21T21:40:44.742 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.8919e-05 2024-09-21T21:40:44.742 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.742 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.742 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:44.742 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.615831+0000", 2024-09-21T21:40:44.742 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0070464969999999997 2024-09-21T21:40:44.742 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.742 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.742 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:44.742 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.615870+0000", 2024-09-21T21:40:44.742 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8396999999999998e-05 2024-09-21T21:40:44.742 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:44.743 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:44.743 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:44.743 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.743 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.743 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_repop(client.4417.0:509 3.8 e26/21 3:10f57124:::rbd_object_map.11214be7c411:head v 26'16, mlcod=26'10)", 2024-09-21T21:40:44.743 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.609103+0000", 2024-09-21T21:40:44.743 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 1.0394423699999999, 2024-09-21T21:40:44.743 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0066864489999999997, 2024-09-21T21:40:44.743 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:44.743 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:44.743 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:44.743 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:44.743 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.743 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:44.744 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.609103+0000", 2024-09-21T21:40:44.744 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:44.744 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.744 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.744 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:44.744 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.609103+0000", 2024-09-21T21:40:44.744 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:44.744 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.744 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.744 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:44.744 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.609103+0000", 2024-09-21T21:40:44.744 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5999999999999999e-07 2024-09-21T21:40:44.744 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.744 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.745 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:44.745 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.609108+0000", 2024-09-21T21:40:44.745 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0100000000000003e-06 2024-09-21T21:40:44.745 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.745 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.745 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:44.745 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.609115+0000", 2024-09-21T21:40:44.745 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2319999999999998e-06 2024-09-21T21:40:44.745 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.745 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.745 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:44.745 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.609120+0000", 2024-09-21T21:40:44.745 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4199999999999998e-06 2024-09-21T21:40:44.745 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.745 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.746 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:44.746 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.609156+0000", 2024-09-21T21:40:44.746 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6269999999999997e-05 2024-09-21T21:40:44.746 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.746 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.746 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:44.746 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.609176+0000", 2024-09-21T21:40:44.746 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9666000000000001e-05 2024-09-21T21:40:44.746 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.746 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.746 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:44.746 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.615764+0000", 2024-09-21T21:40:44.746 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0065882989999999997 2024-09-21T21:40:44.746 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.747 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.747 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:44.747 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.615789+0000", 2024-09-21T21:40:44.747 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.5092e-05 2024-09-21T21:40:44.747 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:44.747 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:44.747 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:44.747 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.747 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.747 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_repop(client.4417.0:510 3.8 e26/21 3:10f57124:::rbd_object_map.11214be7c411:head v 26'17, mlcod=26'10)", 2024-09-21T21:40:44.747 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.609673+0000", 2024-09-21T21:40:44.747 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 1.038872258, 2024-09-21T21:40:44.747 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0063224570000000001, 2024-09-21T21:40:44.748 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:44.748 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:44.748 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:44.748 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:44.748 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.748 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:44.748 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.609673+0000", 2024-09-21T21:40:44.748 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:44.748 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.748 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.748 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:44.748 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.609673+0000", 2024-09-21T21:40:44.748 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:44.748 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.749 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.749 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:44.749 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.609674+0000", 2024-09-21T21:40:44.749 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4799999999999999e-07 2024-09-21T21:40:44.749 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.749 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.749 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:44.749 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.609677+0000", 2024-09-21T21:40:44.749 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8419999999999998e-06 2024-09-21T21:40:44.749 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.749 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.749 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:44.749 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.609684+0000", 2024-09-21T21:40:44.749 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6100000000000002e-06 2024-09-21T21:40:44.750 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.750 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.750 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:44.750 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.609689+0000", 2024-09-21T21:40:44.750 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8450000000000002e-06 2024-09-21T21:40:44.750 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.750 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.750 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:44.750 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.609733+0000", 2024-09-21T21:40:44.751 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3881999999999998e-05 2024-09-21T21:40:44.751 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.751 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.751 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:44.751 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.609752+0000", 2024-09-21T21:40:44.751 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9743000000000001e-05 2024-09-21T21:40:44.751 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.751 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.751 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:44.751 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.615791+0000", 2024-09-21T21:40:44.751 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0060380570000000003 2024-09-21T21:40:44.751 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:44.751 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:44.751 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:44.752 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.615996+0000", 2024-09-21T21:40:44.752 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00020503000000000001 2024-09-21T21:40:44.752 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:44.752 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:44.752 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:44.752 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:44.752 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:44.752 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:40:44.753 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T21:40:44.754 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:40:44.751+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:40:44.827 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T21:40:44.859 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:40:44.859 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:40:44.859 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 0 2024-09-21T21:40:44.859 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:40:44.859 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:40:44.857+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:40:44.868 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T21:40:44.949 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0: {} 2024-09-21T21:40:44.949 INFO:teuthology.orchestra.run.smithi081.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T21:40:44.954 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:40:44.953+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:40:44.979 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1: {} 2024-09-21T21:40:44.979 INFO:teuthology.orchestra.run.smithi081.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T21:40:44.996 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2: {} 2024-09-21T21:40:44.996 INFO:teuthology.orchestra.run.smithi081.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T21:40:45.012 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3: {} 2024-09-21T21:40:45.013 INFO:teuthology.orchestra.run.smithi081.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T21:40:45.025 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:45.025 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:45.029 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4: {} 2024-09-21T21:40:45.029 INFO:teuthology.orchestra.run.smithi081.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T21:40:45.034 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T21:40:45.046 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5: {} 2024-09-21T21:40:45.046 INFO:teuthology.orchestra.run.smithi081.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T21:40:45.056 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:40:45.054+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:40:45.149 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:45.149 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:45.157 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T21:40:45.158 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:40:45.155+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:40:45.258 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:40:45.257+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:40:45.264 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:45.265 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:45.273 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T21:40:45.359 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:40:45.357+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:40:45.376 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:45.376 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:45.384 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T21:40:45.453 INFO:teuthology.orchestra.run.smithi081.stderr:nodeep-scrub is unset 2024-09-21T21:40:45.468 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:40:45.466+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:40:45.532 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:45.532 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:45.541 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T21:40:45.544 INFO:teuthology.orchestra.run.smithi164.stdout:leaving image image_client.0-clone2 intact 2024-09-21T21:40:45.582 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:40:45.581+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:40:45.662 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:40:45.661+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:40:45.684 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:45.684 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:45.692 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T21:40:45.763 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:40:45.761+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:40:45.798 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:45.798 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:45.806 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T21:40:45.890 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:40:45.888+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:40:45.953 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:45.954 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:45.962 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T21:40:45.964 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:40:45.963+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:40:45.967 INFO:teuthology.orchestra.run.smithi164.stdout:checking clone #1, image image_client.0-clone1 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent2 2024-09-21T21:40:46.065 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:40:46.063+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:40:46.109 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:46.109 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:46.118 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T21:40:46.153 INFO:teuthology.orchestra.run.smithi164.stdout:140 write 0x9e7aa00 thru 0x9e7c625 (0x1c26 bytes) 2024-09-21T21:40:46.154 INFO:teuthology.orchestra.run.smithi164.stdout:145 read 0x846f600 thru 0x8475d32 (0x6733 bytes) 2024-09-21T21:40:46.159 INFO:teuthology.orchestra.run.smithi164.stdout:146 read 0x1e5fc00 thru 0x1e6152c (0x192d bytes) 2024-09-21T21:40:46.166 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:40:46.164+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:40:46.166 INFO:teuthology.orchestra.run.smithi164.stdout:147 read 0xe6f3000 thru 0xe701e1f (0xee20 bytes) 2024-09-21T21:40:46.173 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-21T21:40:46.174 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T21:40:46.174 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-21T21:40:46.174 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-21T21:40:46.180 INFO:teuthology.orchestra.run.smithi164.stdout:150 read 0x1a15c00 thru 0x1a19b38 (0x3f39 bytes) 2024-09-21T21:40:46.191 INFO:teuthology.orchestra.run.smithi164.stdout:151 trunc from 0xea1e400 to 0x6579200 2024-09-21T21:40:46.222 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:46.222 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:46.231 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T21:40:46.275 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:40:46.273+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:40:46.285 INFO:teuthology.orchestra.run.smithi164.stdout:152 write 0x5af3c00 thru 0x5af5d1c (0x211d bytes) 2024-09-21T21:40:46.285 INFO:teuthology.orchestra.run.smithi164.stdout:156 punch from 0x2473c00 to 0x247b000, (0x7400 bytes) 2024-09-21T21:40:46.295 INFO:teuthology.orchestra.run.smithi164.stdout:157 punch from 0x200c200 to 0x200c400, (0x200 bytes) 2024-09-21T21:40:46.298 INFO:teuthology.orchestra.run.smithi164.stdout:158 read 0x1bfa000 thru 0x1c04272 (0xa273 bytes) 2024-09-21T21:40:46.349 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:46.349 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:46.354 INFO:teuthology.orchestra.run.smithi164.stdout:159 write 0xaa4aa00 thru 0xaa4ec9e (0x429f bytes) 2024-09-21T21:40:46.356 INFO:teuthology.orchestra.run.smithi164.stdout:161 read 0x52dba00 thru 0x52dbf93 (0x594 bytes) 2024-09-21T21:40:46.358 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T21:40:46.366 INFO:teuthology.orchestra.run.smithi164.stdout:166 trunc from 0xaa4ee00 to 0x24d1200 2024-09-21T21:40:46.402 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:40:46.400+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:40:46.461 INFO:teuthology.orchestra.run.smithi164.stdout:167 punch from 0x1337400 to 0x1340200, (0x8e00 bytes) 2024-09-21T21:40:46.468 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:40:46.466+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:40:46.487 INFO:teuthology.orchestra.run.smithi164.stdout:171 write 0xafeca00 thru 0xaff8912 (0xbf13 bytes) 2024-09-21T21:40:46.488 INFO:teuthology.orchestra.run.smithi164.stdout:172 read 0x9ae00 thru 0xa95b7 (0xe7b8 bytes) 2024-09-21T21:40:46.494 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:46.495 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:46.501 INFO:teuthology.orchestra.run.smithi164.stdout:174 trunc from 0xaff8a00 to 0xb7c400 2024-09-21T21:40:46.504 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T21:40:46.555 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:40:46.555 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:40:46.555 INFO:teuthology.orchestra.run.smithi081.stdout: "epoch": 27, 2024-09-21T21:40:46.555 INFO:teuthology.orchestra.run.smithi081.stdout: "fsid": "a0fdcfc2-ddd2-4e50-81bb-876d940f3578", 2024-09-21T21:40:46.555 INFO:teuthology.orchestra.run.smithi081.stdout: "created": "2024-09-21T21:40:07.413917+0000", 2024-09-21T21:40:46.555 INFO:teuthology.orchestra.run.smithi081.stdout: "modified": "2024-09-21T21:40:45.449182+0000", 2024-09-21T21:40:46.555 INFO:teuthology.orchestra.run.smithi081.stdout: "last_up_change": "2024-09-21T21:40:18.340430+0000", 2024-09-21T21:40:46.555 INFO:teuthology.orchestra.run.smithi081.stdout: "last_in_change": "2024-09-21T21:40:09.029953+0000", 2024-09-21T21:40:46.555 INFO:teuthology.orchestra.run.smithi081.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-21T21:40:46.556 INFO:teuthology.orchestra.run.smithi081.stdout: "flags_num": 5799936, 2024-09-21T21:40:46.556 INFO:teuthology.orchestra.run.smithi081.stdout: "flags_set": [ 2024-09-21T21:40:46.556 INFO:teuthology.orchestra.run.smithi081.stdout: "pglog_hardlimit", 2024-09-21T21:40:46.556 INFO:teuthology.orchestra.run.smithi081.stdout: "purged_snapdirs", 2024-09-21T21:40:46.556 INFO:teuthology.orchestra.run.smithi081.stdout: "recovery_deletes", 2024-09-21T21:40:46.556 INFO:teuthology.orchestra.run.smithi081.stdout: "sortbitwise" 2024-09-21T21:40:46.556 INFO:teuthology.orchestra.run.smithi081.stdout: ], 2024-09-21T21:40:46.556 INFO:teuthology.orchestra.run.smithi081.stdout: "crush_version": 5, 2024-09-21T21:40:46.556 INFO:teuthology.orchestra.run.smithi081.stdout: "full_ratio": 0.94999998807907104, 2024-09-21T21:40:46.556 INFO:teuthology.orchestra.run.smithi081.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-21T21:40:46.556 INFO:teuthology.orchestra.run.smithi081.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-21T21:40:46.556 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster_snapshot": "", 2024-09-21T21:40:46.556 INFO:teuthology.orchestra.run.smithi081.stdout: "pool_max": 3, 2024-09-21T21:40:46.556 INFO:teuthology.orchestra.run.smithi081.stdout: "max_osd": 6, 2024-09-21T21:40:46.557 INFO:teuthology.orchestra.run.smithi081.stdout: "require_min_compat_client": "luminous", 2024-09-21T21:40:46.557 INFO:teuthology.orchestra.run.smithi081.stdout: "min_compat_client": "jewel", 2024-09-21T21:40:46.557 INFO:teuthology.orchestra.run.smithi081.stdout: "require_osd_release": "squid", 2024-09-21T21:40:46.557 INFO:teuthology.orchestra.run.smithi081.stdout: "allow_crimson": false, 2024-09-21T21:40:46.557 INFO:teuthology.orchestra.run.smithi081.stdout: "pools": [ 2024-09-21T21:40:46.557 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:46.557 INFO:teuthology.orchestra.run.smithi081.stdout: "pool": 1, 2024-09-21T21:40:46.557 INFO:teuthology.orchestra.run.smithi081.stdout: "pool_name": ".mgr", 2024-09-21T21:40:46.557 INFO:teuthology.orchestra.run.smithi081.stdout: "create_time": "2024-09-21T21:40:18.321328+0000", 2024-09-21T21:40:46.557 INFO:teuthology.orchestra.run.smithi081.stdout: "flags": 1, 2024-09-21T21:40:46.557 INFO:teuthology.orchestra.run.smithi081.stdout: "flags_names": "hashpspool", 2024-09-21T21:40:46.557 INFO:teuthology.orchestra.run.smithi081.stdout: "type": 1, 2024-09-21T21:40:46.557 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 2, 2024-09-21T21:40:46.557 INFO:teuthology.orchestra.run.smithi081.stdout: "min_size": 1, 2024-09-21T21:40:46.558 INFO:teuthology.orchestra.run.smithi081.stdout: "crush_rule": 0, 2024-09-21T21:40:46.558 INFO:teuthology.orchestra.run.smithi081.stdout: "peering_crush_bucket_count": 0, 2024-09-21T21:40:46.558 INFO:teuthology.orchestra.run.smithi081.stdout: "peering_crush_bucket_target": 0, 2024-09-21T21:40:46.558 INFO:teuthology.orchestra.run.smithi081.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T21:40:46.558 INFO:teuthology.orchestra.run.smithi081.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T21:40:46.558 INFO:teuthology.orchestra.run.smithi081.stdout: "is_stretch_pool": false, 2024-09-21T21:40:46.558 INFO:teuthology.orchestra.run.smithi081.stdout: "object_hash": 2, 2024-09-21T21:40:46.558 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_autoscale_mode": "off", 2024-09-21T21:40:46.558 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_num": 1, 2024-09-21T21:40:46.558 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_placement_num": 1, 2024-09-21T21:40:46.558 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_placement_num_target": 1, 2024-09-21T21:40:46.558 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_num_target": 1, 2024-09-21T21:40:46.558 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_num_pending": 1, 2024-09-21T21:40:46.559 INFO:teuthology.orchestra.run.smithi081.stdout: "last_pg_merge_meta": { 2024-09-21T21:40:46.559 INFO:teuthology.orchestra.run.smithi081.stdout: "source_pgid": "0.0", 2024-09-21T21:40:46.559 INFO:teuthology.orchestra.run.smithi081.stdout: "ready_epoch": 0, 2024-09-21T21:40:46.559 INFO:teuthology.orchestra.run.smithi081.stdout: "last_epoch_started": 0, 2024-09-21T21:40:46.559 INFO:teuthology.orchestra.run.smithi081.stdout: "last_epoch_clean": 0, 2024-09-21T21:40:46.559 INFO:teuthology.orchestra.run.smithi081.stdout: "source_version": "0'0", 2024-09-21T21:40:46.559 INFO:teuthology.orchestra.run.smithi081.stdout: "target_version": "0'0" 2024-09-21T21:40:46.559 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:46.559 INFO:teuthology.orchestra.run.smithi081.stdout: "last_change": "15", 2024-09-21T21:40:46.559 INFO:teuthology.orchestra.run.smithi081.stdout: "last_force_op_resend": "0", 2024-09-21T21:40:46.559 INFO:teuthology.orchestra.run.smithi081.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T21:40:46.559 INFO:teuthology.orchestra.run.smithi081.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T21:40:46.559 INFO:teuthology.orchestra.run.smithi081.stdout: "auid": 0, 2024-09-21T21:40:46.559 INFO:teuthology.orchestra.run.smithi081.stdout: "snap_mode": "selfmanaged", 2024-09-21T21:40:46.560 INFO:teuthology.orchestra.run.smithi081.stdout: "snap_seq": 0, 2024-09-21T21:40:46.560 INFO:teuthology.orchestra.run.smithi081.stdout: "snap_epoch": 0, 2024-09-21T21:40:46.560 INFO:teuthology.orchestra.run.smithi081.stdout: "pool_snaps": [], 2024-09-21T21:40:46.560 INFO:teuthology.orchestra.run.smithi081.stdout: "removed_snaps": "[]", 2024-09-21T21:40:46.560 INFO:teuthology.orchestra.run.smithi081.stdout: "quota_max_bytes": 0, 2024-09-21T21:40:46.560 INFO:teuthology.orchestra.run.smithi081.stdout: "quota_max_objects": 0, 2024-09-21T21:40:46.560 INFO:teuthology.orchestra.run.smithi081.stdout: "tiers": [], 2024-09-21T21:40:46.560 INFO:teuthology.orchestra.run.smithi081.stdout: "tier_of": -1, 2024-09-21T21:40:46.560 INFO:teuthology.orchestra.run.smithi081.stdout: "read_tier": -1, 2024-09-21T21:40:46.560 INFO:teuthology.orchestra.run.smithi081.stdout: "write_tier": -1, 2024-09-21T21:40:46.560 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_mode": "none", 2024-09-21T21:40:46.560 INFO:teuthology.orchestra.run.smithi081.stdout: "target_max_bytes": 0, 2024-09-21T21:40:46.560 INFO:teuthology.orchestra.run.smithi081.stdout: "target_max_objects": 0, 2024-09-21T21:40:46.560 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T21:40:46.561 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T21:40:46.561 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T21:40:46.561 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_min_flush_age": 0, 2024-09-21T21:40:46.561 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_min_evict_age": 0, 2024-09-21T21:40:46.561 INFO:teuthology.orchestra.run.smithi081.stdout: "erasure_code_profile": "", 2024-09-21T21:40:46.561 INFO:teuthology.orchestra.run.smithi081.stdout: "hit_set_params": { 2024-09-21T21:40:46.561 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "none" 2024-09-21T21:40:46.561 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:46.561 INFO:teuthology.orchestra.run.smithi081.stdout: "hit_set_period": 0, 2024-09-21T21:40:46.561 INFO:teuthology.orchestra.run.smithi081.stdout: "hit_set_count": 0, 2024-09-21T21:40:46.561 INFO:teuthology.orchestra.run.smithi081.stdout: "use_gmt_hitset": true, 2024-09-21T21:40:46.561 INFO:teuthology.orchestra.run.smithi081.stdout: "min_read_recency_for_promote": 0, 2024-09-21T21:40:46.561 INFO:teuthology.orchestra.run.smithi081.stdout: "min_write_recency_for_promote": 0, 2024-09-21T21:40:46.561 INFO:teuthology.orchestra.run.smithi081.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T21:40:46.561 INFO:teuthology.orchestra.run.smithi081.stdout: "hit_set_search_last_n": 0, 2024-09-21T21:40:46.562 INFO:teuthology.orchestra.run.smithi081.stdout: "grade_table": [], 2024-09-21T21:40:46.562 INFO:teuthology.orchestra.run.smithi081.stdout: "stripe_width": 0, 2024-09-21T21:40:46.562 INFO:teuthology.orchestra.run.smithi081.stdout: "expected_num_objects": 0, 2024-09-21T21:40:46.562 INFO:teuthology.orchestra.run.smithi081.stdout: "fast_read": false, 2024-09-21T21:40:46.562 INFO:teuthology.orchestra.run.smithi081.stdout: "options": { 2024-09-21T21:40:46.562 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_num_max": 32, 2024-09-21T21:40:46.562 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_num_min": 1 2024-09-21T21:40:46.562 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:46.562 INFO:teuthology.orchestra.run.smithi081.stdout: "application_metadata": { 2024-09-21T21:40:46.562 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": {} 2024-09-21T21:40:46.562 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:46.562 INFO:teuthology.orchestra.run.smithi081.stdout: "read_balance": { 2024-09-21T21:40:46.562 INFO:teuthology.orchestra.run.smithi081.stdout: "score_type": "Fair distribution", 2024-09-21T21:40:46.562 INFO:teuthology.orchestra.run.smithi081.stdout: "score_acting": 6.059999942779541, 2024-09-21T21:40:46.563 INFO:teuthology.orchestra.run.smithi081.stdout: "score_stable": 6.059999942779541, 2024-09-21T21:40:46.563 INFO:teuthology.orchestra.run.smithi081.stdout: "optimal_score": 0.33000001311302185, 2024-09-21T21:40:46.563 INFO:teuthology.orchestra.run.smithi081.stdout: "raw_score_acting": 2, 2024-09-21T21:40:46.563 INFO:teuthology.orchestra.run.smithi081.stdout: "raw_score_stable": 2, 2024-09-21T21:40:46.563 INFO:teuthology.orchestra.run.smithi081.stdout: "primary_affinity_weighted": 1, 2024-09-21T21:40:46.563 INFO:teuthology.orchestra.run.smithi081.stdout: "average_primary_affinity": 1, 2024-09-21T21:40:46.563 INFO:teuthology.orchestra.run.smithi081.stdout: "average_primary_affinity_weighted": 1 2024-09-21T21:40:46.563 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:46.563 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:46.563 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:46.563 INFO:teuthology.orchestra.run.smithi081.stdout: "pool": 2, 2024-09-21T21:40:46.563 INFO:teuthology.orchestra.run.smithi081.stdout: "pool_name": "rbd", 2024-09-21T21:40:46.563 INFO:teuthology.orchestra.run.smithi081.stdout: "create_time": "2024-09-21T21:40:24.235658+0000", 2024-09-21T21:40:46.563 INFO:teuthology.orchestra.run.smithi081.stdout: "flags": 8193, 2024-09-21T21:40:46.564 INFO:teuthology.orchestra.run.smithi081.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T21:40:46.564 INFO:teuthology.orchestra.run.smithi081.stdout: "type": 1, 2024-09-21T21:40:46.564 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 2, 2024-09-21T21:40:46.564 INFO:teuthology.orchestra.run.smithi081.stdout: "min_size": 1, 2024-09-21T21:40:46.564 INFO:teuthology.orchestra.run.smithi081.stdout: "crush_rule": 0, 2024-09-21T21:40:46.564 INFO:teuthology.orchestra.run.smithi081.stdout: "peering_crush_bucket_count": 0, 2024-09-21T21:40:46.564 INFO:teuthology.orchestra.run.smithi081.stdout: "peering_crush_bucket_target": 0, 2024-09-21T21:40:46.564 INFO:teuthology.orchestra.run.smithi081.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T21:40:46.564 INFO:teuthology.orchestra.run.smithi081.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T21:40:46.564 INFO:teuthology.orchestra.run.smithi081.stdout: "is_stretch_pool": false, 2024-09-21T21:40:46.564 INFO:teuthology.orchestra.run.smithi081.stdout: "object_hash": 2, 2024-09-21T21:40:46.564 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_autoscale_mode": "off", 2024-09-21T21:40:46.564 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_num": 8, 2024-09-21T21:40:46.564 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_placement_num": 8, 2024-09-21T21:40:46.565 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_placement_num_target": 8, 2024-09-21T21:40:46.565 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_num_target": 8, 2024-09-21T21:40:46.565 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_num_pending": 8, 2024-09-21T21:40:46.565 INFO:teuthology.orchestra.run.smithi081.stdout: "last_pg_merge_meta": { 2024-09-21T21:40:46.565 INFO:teuthology.orchestra.run.smithi081.stdout: "source_pgid": "0.0", 2024-09-21T21:40:46.565 INFO:teuthology.orchestra.run.smithi081.stdout: "ready_epoch": 0, 2024-09-21T21:40:46.565 INFO:teuthology.orchestra.run.smithi081.stdout: "last_epoch_started": 0, 2024-09-21T21:40:46.565 INFO:teuthology.orchestra.run.smithi081.stdout: "last_epoch_clean": 0, 2024-09-21T21:40:46.565 INFO:teuthology.orchestra.run.smithi081.stdout: "source_version": "0'0", 2024-09-21T21:40:46.565 INFO:teuthology.orchestra.run.smithi081.stdout: "target_version": "0'0" 2024-09-21T21:40:46.565 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:46.565 INFO:teuthology.orchestra.run.smithi081.stdout: "last_change": "19", 2024-09-21T21:40:46.565 INFO:teuthology.orchestra.run.smithi081.stdout: "last_force_op_resend": "0", 2024-09-21T21:40:46.565 INFO:teuthology.orchestra.run.smithi081.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T21:40:46.566 INFO:teuthology.orchestra.run.smithi081.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T21:40:46.566 INFO:teuthology.orchestra.run.smithi081.stdout: "auid": 0, 2024-09-21T21:40:46.566 INFO:teuthology.orchestra.run.smithi081.stdout: "snap_mode": "selfmanaged", 2024-09-21T21:40:46.566 INFO:teuthology.orchestra.run.smithi081.stdout: "snap_seq": 2, 2024-09-21T21:40:46.566 INFO:teuthology.orchestra.run.smithi081.stdout: "snap_epoch": 19, 2024-09-21T21:40:46.566 INFO:teuthology.orchestra.run.smithi081.stdout: "pool_snaps": [], 2024-09-21T21:40:46.566 INFO:teuthology.orchestra.run.smithi081.stdout: "removed_snaps": "[]", 2024-09-21T21:40:46.566 INFO:teuthology.orchestra.run.smithi081.stdout: "quota_max_bytes": 0, 2024-09-21T21:40:46.566 INFO:teuthology.orchestra.run.smithi081.stdout: "quota_max_objects": 0, 2024-09-21T21:40:46.566 INFO:teuthology.orchestra.run.smithi081.stdout: "tiers": [], 2024-09-21T21:40:46.566 INFO:teuthology.orchestra.run.smithi081.stdout: "tier_of": -1, 2024-09-21T21:40:46.566 INFO:teuthology.orchestra.run.smithi081.stdout: "read_tier": -1, 2024-09-21T21:40:46.566 INFO:teuthology.orchestra.run.smithi081.stdout: "write_tier": -1, 2024-09-21T21:40:46.566 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_mode": "none", 2024-09-21T21:40:46.567 INFO:teuthology.orchestra.run.smithi081.stdout: "target_max_bytes": 0, 2024-09-21T21:40:46.567 INFO:teuthology.orchestra.run.smithi081.stdout: "target_max_objects": 0, 2024-09-21T21:40:46.567 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T21:40:46.567 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T21:40:46.567 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T21:40:46.567 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_min_flush_age": 0, 2024-09-21T21:40:46.567 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_min_evict_age": 0, 2024-09-21T21:40:46.567 INFO:teuthology.orchestra.run.smithi081.stdout: "erasure_code_profile": "", 2024-09-21T21:40:46.567 INFO:teuthology.orchestra.run.smithi081.stdout: "hit_set_params": { 2024-09-21T21:40:46.567 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "none" 2024-09-21T21:40:46.567 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:46.567 INFO:teuthology.orchestra.run.smithi081.stdout: "hit_set_period": 0, 2024-09-21T21:40:46.567 INFO:teuthology.orchestra.run.smithi081.stdout: "hit_set_count": 0, 2024-09-21T21:40:46.567 INFO:teuthology.orchestra.run.smithi081.stdout: "use_gmt_hitset": true, 2024-09-21T21:40:46.568 INFO:teuthology.orchestra.run.smithi081.stdout: "min_read_recency_for_promote": 0, 2024-09-21T21:40:46.568 INFO:teuthology.orchestra.run.smithi081.stdout: "min_write_recency_for_promote": 0, 2024-09-21T21:40:46.568 INFO:teuthology.orchestra.run.smithi081.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T21:40:46.568 INFO:teuthology.orchestra.run.smithi081.stdout: "hit_set_search_last_n": 0, 2024-09-21T21:40:46.568 INFO:teuthology.orchestra.run.smithi081.stdout: "grade_table": [], 2024-09-21T21:40:46.568 INFO:teuthology.orchestra.run.smithi081.stdout: "stripe_width": 0, 2024-09-21T21:40:46.568 INFO:teuthology.orchestra.run.smithi081.stdout: "expected_num_objects": 0, 2024-09-21T21:40:46.568 INFO:teuthology.orchestra.run.smithi081.stdout: "fast_read": false, 2024-09-21T21:40:46.568 INFO:teuthology.orchestra.run.smithi081.stdout: "options": {}, 2024-09-21T21:40:46.568 INFO:teuthology.orchestra.run.smithi081.stdout: "application_metadata": { 2024-09-21T21:40:46.568 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd": {} 2024-09-21T21:40:46.568 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:46.568 INFO:teuthology.orchestra.run.smithi081.stdout: "read_balance": { 2024-09-21T21:40:46.568 INFO:teuthology.orchestra.run.smithi081.stdout: "score_type": "Fair distribution", 2024-09-21T21:40:46.568 INFO:teuthology.orchestra.run.smithi081.stdout: "score_acting": 2.25, 2024-09-21T21:40:46.569 INFO:teuthology.orchestra.run.smithi081.stdout: "score_stable": 2.25, 2024-09-21T21:40:46.569 INFO:teuthology.orchestra.run.smithi081.stdout: "optimal_score": 1, 2024-09-21T21:40:46.569 INFO:teuthology.orchestra.run.smithi081.stdout: "raw_score_acting": 2.25, 2024-09-21T21:40:46.569 INFO:teuthology.orchestra.run.smithi081.stdout: "raw_score_stable": 2.25, 2024-09-21T21:40:46.569 INFO:teuthology.orchestra.run.smithi081.stdout: "primary_affinity_weighted": 1, 2024-09-21T21:40:46.569 INFO:teuthology.orchestra.run.smithi081.stdout: "average_primary_affinity": 1, 2024-09-21T21:40:46.569 INFO:teuthology.orchestra.run.smithi081.stdout: "average_primary_affinity_weighted": 1 2024-09-21T21:40:46.569 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:46.569 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:46.569 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:46.569 INFO:teuthology.orchestra.run.smithi081.stdout: "pool": 3, 2024-09-21T21:40:46.569 INFO:teuthology.orchestra.run.smithi081.stdout: "pool_name": "pool_client.0", 2024-09-21T21:40:46.569 INFO:teuthology.orchestra.run.smithi081.stdout: "create_time": "2024-09-21T21:40:36.093691+0000", 2024-09-21T21:40:46.569 INFO:teuthology.orchestra.run.smithi081.stdout: "flags": 8193, 2024-09-21T21:40:46.569 INFO:teuthology.orchestra.run.smithi081.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T21:40:46.570 INFO:teuthology.orchestra.run.smithi081.stdout: "type": 1, 2024-09-21T21:40:46.570 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 2, 2024-09-21T21:40:46.570 INFO:teuthology.orchestra.run.smithi081.stdout: "min_size": 1, 2024-09-21T21:40:46.570 INFO:teuthology.orchestra.run.smithi081.stdout: "crush_rule": 0, 2024-09-21T21:40:46.570 INFO:teuthology.orchestra.run.smithi081.stdout: "peering_crush_bucket_count": 0, 2024-09-21T21:40:46.570 INFO:teuthology.orchestra.run.smithi081.stdout: "peering_crush_bucket_target": 0, 2024-09-21T21:40:46.570 INFO:teuthology.orchestra.run.smithi081.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T21:40:46.570 INFO:teuthology.orchestra.run.smithi081.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T21:40:46.570 INFO:teuthology.orchestra.run.smithi081.stdout: "is_stretch_pool": false, 2024-09-21T21:40:46.570 INFO:teuthology.orchestra.run.smithi081.stdout: "object_hash": 2, 2024-09-21T21:40:46.570 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_autoscale_mode": "off", 2024-09-21T21:40:46.570 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_num": 32, 2024-09-21T21:40:46.570 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_placement_num": 32, 2024-09-21T21:40:46.570 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_placement_num_target": 32, 2024-09-21T21:40:46.571 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_num_target": 32, 2024-09-21T21:40:46.571 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_num_pending": 32, 2024-09-21T21:40:46.571 INFO:teuthology.orchestra.run.smithi081.stdout: "last_pg_merge_meta": { 2024-09-21T21:40:46.571 INFO:teuthology.orchestra.run.smithi081.stdout: "source_pgid": "0.0", 2024-09-21T21:40:46.571 INFO:teuthology.orchestra.run.smithi081.stdout: "ready_epoch": 0, 2024-09-21T21:40:46.571 INFO:teuthology.orchestra.run.smithi081.stdout: "last_epoch_started": 0, 2024-09-21T21:40:46.571 INFO:teuthology.orchestra.run.smithi081.stdout: "last_epoch_clean": 0, 2024-09-21T21:40:46.571 INFO:teuthology.orchestra.run.smithi081.stdout: "source_version": "0'0", 2024-09-21T21:40:46.571 INFO:teuthology.orchestra.run.smithi081.stdout: "target_version": "0'0" 2024-09-21T21:40:46.571 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:46.571 INFO:teuthology.orchestra.run.smithi081.stdout: "last_change": "27", 2024-09-21T21:40:46.571 INFO:teuthology.orchestra.run.smithi081.stdout: "last_force_op_resend": "0", 2024-09-21T21:40:46.571 INFO:teuthology.orchestra.run.smithi081.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T21:40:46.571 INFO:teuthology.orchestra.run.smithi081.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T21:40:46.571 INFO:teuthology.orchestra.run.smithi081.stdout: "auid": 0, 2024-09-21T21:40:46.572 INFO:teuthology.orchestra.run.smithi081.stdout: "snap_mode": "selfmanaged", 2024-09-21T21:40:46.572 INFO:teuthology.orchestra.run.smithi081.stdout: "snap_seq": 5, 2024-09-21T21:40:46.572 INFO:teuthology.orchestra.run.smithi081.stdout: "snap_epoch": 27, 2024-09-21T21:40:46.572 INFO:teuthology.orchestra.run.smithi081.stdout: "pool_snaps": [], 2024-09-21T21:40:46.572 INFO:teuthology.orchestra.run.smithi081.stdout: "removed_snaps": "[]", 2024-09-21T21:40:46.572 INFO:teuthology.orchestra.run.smithi081.stdout: "quota_max_bytes": 0, 2024-09-21T21:40:46.572 INFO:teuthology.orchestra.run.smithi081.stdout: "quota_max_objects": 0, 2024-09-21T21:40:46.572 INFO:teuthology.orchestra.run.smithi081.stdout: "tiers": [], 2024-09-21T21:40:46.572 INFO:teuthology.orchestra.run.smithi081.stdout: "tier_of": -1, 2024-09-21T21:40:46.572 INFO:teuthology.orchestra.run.smithi081.stdout: "read_tier": -1, 2024-09-21T21:40:46.572 INFO:teuthology.orchestra.run.smithi081.stdout: "write_tier": -1, 2024-09-21T21:40:46.572 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_mode": "none", 2024-09-21T21:40:46.572 INFO:teuthology.orchestra.run.smithi081.stdout: "target_max_bytes": 0, 2024-09-21T21:40:46.572 INFO:teuthology.orchestra.run.smithi081.stdout: "target_max_objects": 0, 2024-09-21T21:40:46.572 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T21:40:46.573 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T21:40:46.573 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T21:40:46.573 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_min_flush_age": 0, 2024-09-21T21:40:46.573 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_min_evict_age": 0, 2024-09-21T21:40:46.573 INFO:teuthology.orchestra.run.smithi081.stdout: "erasure_code_profile": "", 2024-09-21T21:40:46.573 INFO:teuthology.orchestra.run.smithi081.stdout: "hit_set_params": { 2024-09-21T21:40:46.573 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "none" 2024-09-21T21:40:46.573 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:46.573 INFO:teuthology.orchestra.run.smithi081.stdout: "hit_set_period": 0, 2024-09-21T21:40:46.573 INFO:teuthology.orchestra.run.smithi081.stdout: "hit_set_count": 0, 2024-09-21T21:40:46.573 INFO:teuthology.orchestra.run.smithi081.stdout: "use_gmt_hitset": true, 2024-09-21T21:40:46.573 INFO:teuthology.orchestra.run.smithi081.stdout: "min_read_recency_for_promote": 0, 2024-09-21T21:40:46.573 INFO:teuthology.orchestra.run.smithi081.stdout: "min_write_recency_for_promote": 0, 2024-09-21T21:40:46.574 INFO:teuthology.orchestra.run.smithi081.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T21:40:46.574 INFO:teuthology.orchestra.run.smithi081.stdout: "hit_set_search_last_n": 0, 2024-09-21T21:40:46.574 INFO:teuthology.orchestra.run.smithi081.stdout: "grade_table": [], 2024-09-21T21:40:46.574 INFO:teuthology.orchestra.run.smithi081.stdout: "stripe_width": 0, 2024-09-21T21:40:46.574 INFO:teuthology.orchestra.run.smithi081.stdout: "expected_num_objects": 0, 2024-09-21T21:40:46.574 INFO:teuthology.orchestra.run.smithi081.stdout: "fast_read": false, 2024-09-21T21:40:46.574 INFO:teuthology.orchestra.run.smithi081.stdout: "options": {}, 2024-09-21T21:40:46.574 INFO:teuthology.orchestra.run.smithi081.stdout: "application_metadata": { 2024-09-21T21:40:46.574 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd": {} 2024-09-21T21:40:46.574 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:46.574 INFO:teuthology.orchestra.run.smithi081.stdout: "read_balance": { 2024-09-21T21:40:46.574 INFO:teuthology.orchestra.run.smithi081.stdout: "score_type": "Fair distribution", 2024-09-21T21:40:46.574 INFO:teuthology.orchestra.run.smithi081.stdout: "score_acting": 1.5, 2024-09-21T21:40:46.574 INFO:teuthology.orchestra.run.smithi081.stdout: "score_stable": 1.5, 2024-09-21T21:40:46.575 INFO:teuthology.orchestra.run.smithi081.stdout: "optimal_score": 1, 2024-09-21T21:40:46.575 INFO:teuthology.orchestra.run.smithi081.stdout: "raw_score_acting": 1.5, 2024-09-21T21:40:46.575 INFO:teuthology.orchestra.run.smithi081.stdout: "raw_score_stable": 1.5, 2024-09-21T21:40:46.575 INFO:teuthology.orchestra.run.smithi081.stdout: "primary_affinity_weighted": 1, 2024-09-21T21:40:46.575 INFO:teuthology.orchestra.run.smithi081.stdout: "average_primary_affinity": 1, 2024-09-21T21:40:46.575 INFO:teuthology.orchestra.run.smithi081.stdout: "average_primary_affinity_weighted": 1 2024-09-21T21:40:46.575 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:46.575 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:46.575 INFO:teuthology.orchestra.run.smithi081.stdout: ], 2024-09-21T21:40:46.575 INFO:teuthology.orchestra.run.smithi081.stdout: "osds": [ 2024-09-21T21:40:46.575 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:46.575 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": 0, 2024-09-21T21:40:46.575 INFO:teuthology.orchestra.run.smithi081.stdout: "uuid": "a9869830-8e74-4b02-963d-ab630a0ee286", 2024-09-21T21:40:46.575 INFO:teuthology.orchestra.run.smithi081.stdout: "up": 1, 2024-09-21T21:40:46.575 INFO:teuthology.orchestra.run.smithi081.stdout: "in": 1, 2024-09-21T21:40:46.576 INFO:teuthology.orchestra.run.smithi081.stdout: "weight": 1, 2024-09-21T21:40:46.576 INFO:teuthology.orchestra.run.smithi081.stdout: "primary_affinity": 1, 2024-09-21T21:40:46.576 INFO:teuthology.orchestra.run.smithi081.stdout: "last_clean_begin": 0, 2024-09-21T21:40:46.576 INFO:teuthology.orchestra.run.smithi081.stdout: "last_clean_end": 0, 2024-09-21T21:40:46.576 INFO:teuthology.orchestra.run.smithi081.stdout: "up_from": 12, 2024-09-21T21:40:46.576 INFO:teuthology.orchestra.run.smithi081.stdout: "up_thru": 21, 2024-09-21T21:40:46.576 INFO:teuthology.orchestra.run.smithi081.stdout: "down_at": 0, 2024-09-21T21:40:46.576 INFO:teuthology.orchestra.run.smithi081.stdout: "lost_at": 0, 2024-09-21T21:40:46.576 INFO:teuthology.orchestra.run.smithi081.stdout: "public_addrs": { 2024-09-21T21:40:46.576 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:40:46.576 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:46.576 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:40:46.576 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6808", 2024-09-21T21:40:46.576 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1657552338 2024-09-21T21:40:46.577 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:46.577 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:46.577 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:40:46.577 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6809", 2024-09-21T21:40:46.577 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1657552338 2024-09-21T21:40:46.577 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:46.577 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:46.577 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:46.577 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster_addrs": { 2024-09-21T21:40:46.577 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:40:46.577 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:46.577 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:40:46.577 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6810", 2024-09-21T21:40:46.577 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1657552338 2024-09-21T21:40:46.577 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:46.577 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:46.578 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:40:46.578 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6811", 2024-09-21T21:40:46.578 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1657552338 2024-09-21T21:40:46.578 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:46.578 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:46.578 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:46.578 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_back_addrs": { 2024-09-21T21:40:46.578 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:40:46.578 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:46.578 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:40:46.578 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6814", 2024-09-21T21:40:46.578 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1657552338 2024-09-21T21:40:46.578 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:46.579 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:46.579 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:40:46.579 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6815", 2024-09-21T21:40:46.579 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1657552338 2024-09-21T21:40:46.579 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:46.579 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:46.579 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:46.579 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_front_addrs": { 2024-09-21T21:40:46.579 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:40:46.579 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:46.579 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:40:46.579 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6812", 2024-09-21T21:40:46.579 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1657552338 2024-09-21T21:40:46.579 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:46.580 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:46.580 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:40:46.580 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6813", 2024-09-21T21:40:46.580 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1657552338 2024-09-21T21:40:46.580 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:46.580 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:46.580 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:46.580 INFO:teuthology.orchestra.run.smithi081.stdout: "public_addr": "172.21.15.81:6809/1657552338", 2024-09-21T21:40:46.580 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster_addr": "172.21.15.81:6811/1657552338", 2024-09-21T21:40:46.580 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_back_addr": "172.21.15.81:6815/1657552338", 2024-09-21T21:40:46.580 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_front_addr": "172.21.15.81:6813/1657552338", 2024-09-21T21:40:46.580 INFO:teuthology.orchestra.run.smithi081.stdout: "state": [ 2024-09-21T21:40:46.580 INFO:teuthology.orchestra.run.smithi081.stdout: "exists", 2024-09-21T21:40:46.580 INFO:teuthology.orchestra.run.smithi081.stdout: "up" 2024-09-21T21:40:46.581 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:46.581 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:46.581 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:46.581 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": 1, 2024-09-21T21:40:46.581 INFO:teuthology.orchestra.run.smithi081.stdout: "uuid": "b189819d-69a3-4e95-96e7-96dc4809ac80", 2024-09-21T21:40:46.581 INFO:teuthology.orchestra.run.smithi081.stdout: "up": 1, 2024-09-21T21:40:46.581 INFO:teuthology.orchestra.run.smithi081.stdout: "in": 1, 2024-09-21T21:40:46.581 INFO:teuthology.orchestra.run.smithi081.stdout: "weight": 1, 2024-09-21T21:40:46.581 INFO:teuthology.orchestra.run.smithi081.stdout: "primary_affinity": 1, 2024-09-21T21:40:46.581 INFO:teuthology.orchestra.run.smithi081.stdout: "last_clean_begin": 0, 2024-09-21T21:40:46.581 INFO:teuthology.orchestra.run.smithi081.stdout: "last_clean_end": 0, 2024-09-21T21:40:46.581 INFO:teuthology.orchestra.run.smithi081.stdout: "up_from": 12, 2024-09-21T21:40:46.581 INFO:teuthology.orchestra.run.smithi081.stdout: "up_thru": 21, 2024-09-21T21:40:46.581 INFO:teuthology.orchestra.run.smithi081.stdout: "down_at": 0, 2024-09-21T21:40:46.581 INFO:teuthology.orchestra.run.smithi081.stdout: "lost_at": 0, 2024-09-21T21:40:46.582 INFO:teuthology.orchestra.run.smithi081.stdout: "public_addrs": { 2024-09-21T21:40:46.582 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:40:46.582 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:46.582 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:40:46.582 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6816", 2024-09-21T21:40:46.582 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2864159502 2024-09-21T21:40:46.582 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:46.582 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:46.582 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:40:46.582 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6817", 2024-09-21T21:40:46.582 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2864159502 2024-09-21T21:40:46.582 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:46.582 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:46.582 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:46.583 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster_addrs": { 2024-09-21T21:40:46.583 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:40:46.583 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:46.583 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:40:46.583 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6818", 2024-09-21T21:40:46.583 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2864159502 2024-09-21T21:40:46.583 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:46.583 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:46.583 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:40:46.583 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6819", 2024-09-21T21:40:46.583 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2864159502 2024-09-21T21:40:46.583 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:46.583 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:46.583 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:46.584 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_back_addrs": { 2024-09-21T21:40:46.584 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:40:46.584 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:46.584 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:40:46.584 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6822", 2024-09-21T21:40:46.584 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2864159502 2024-09-21T21:40:46.584 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:46.584 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:46.584 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:40:46.584 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6823", 2024-09-21T21:40:46.584 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2864159502 2024-09-21T21:40:46.584 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:46.584 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:46.584 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:46.584 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_front_addrs": { 2024-09-21T21:40:46.585 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:40:46.585 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:46.585 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:40:46.585 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6820", 2024-09-21T21:40:46.585 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2864159502 2024-09-21T21:40:46.585 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:46.585 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:46.585 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:40:46.585 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6821", 2024-09-21T21:40:46.585 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2864159502 2024-09-21T21:40:46.585 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:46.585 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:46.585 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:46.585 INFO:teuthology.orchestra.run.smithi081.stdout: "public_addr": "172.21.15.81:6817/2864159502", 2024-09-21T21:40:46.586 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster_addr": "172.21.15.81:6819/2864159502", 2024-09-21T21:40:46.586 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_back_addr": "172.21.15.81:6823/2864159502", 2024-09-21T21:40:46.586 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_front_addr": "172.21.15.81:6821/2864159502", 2024-09-21T21:40:46.586 INFO:teuthology.orchestra.run.smithi081.stdout: "state": [ 2024-09-21T21:40:46.586 INFO:teuthology.orchestra.run.smithi081.stdout: "exists", 2024-09-21T21:40:46.586 INFO:teuthology.orchestra.run.smithi081.stdout: "up" 2024-09-21T21:40:46.586 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:46.586 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:46.586 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:46.586 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": 2, 2024-09-21T21:40:46.586 INFO:teuthology.orchestra.run.smithi081.stdout: "uuid": "8437b99a-1297-4903-ab66-95c5cba2b2bf", 2024-09-21T21:40:46.586 INFO:teuthology.orchestra.run.smithi081.stdout: "up": 1, 2024-09-21T21:40:46.586 INFO:teuthology.orchestra.run.smithi081.stdout: "in": 1, 2024-09-21T21:40:46.587 INFO:teuthology.orchestra.run.smithi081.stdout: "weight": 1, 2024-09-21T21:40:46.587 INFO:teuthology.orchestra.run.smithi081.stdout: "primary_affinity": 1, 2024-09-21T21:40:46.587 INFO:teuthology.orchestra.run.smithi081.stdout: "last_clean_begin": 0, 2024-09-21T21:40:46.587 INFO:teuthology.orchestra.run.smithi081.stdout: "last_clean_end": 0, 2024-09-21T21:40:46.587 INFO:teuthology.orchestra.run.smithi081.stdout: "up_from": 11, 2024-09-21T21:40:46.587 INFO:teuthology.orchestra.run.smithi081.stdout: "up_thru": 21, 2024-09-21T21:40:46.587 INFO:teuthology.orchestra.run.smithi081.stdout: "down_at": 0, 2024-09-21T21:40:46.587 INFO:teuthology.orchestra.run.smithi081.stdout: "lost_at": 0, 2024-09-21T21:40:46.587 INFO:teuthology.orchestra.run.smithi081.stdout: "public_addrs": { 2024-09-21T21:40:46.587 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:40:46.587 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:46.587 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:40:46.587 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6800", 2024-09-21T21:40:46.587 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 3023160932 2024-09-21T21:40:46.587 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:46.588 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:46.588 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:40:46.588 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6801", 2024-09-21T21:40:46.588 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 3023160932 2024-09-21T21:40:46.588 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:46.588 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:46.588 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:46.588 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster_addrs": { 2024-09-21T21:40:46.588 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:40:46.588 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:46.588 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:40:46.588 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6802", 2024-09-21T21:40:46.588 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 3023160932 2024-09-21T21:40:46.588 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:46.589 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:46.589 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:40:46.589 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6803", 2024-09-21T21:40:46.589 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 3023160932 2024-09-21T21:40:46.589 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:46.589 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:46.589 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:46.589 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_back_addrs": { 2024-09-21T21:40:46.589 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:40:46.589 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:46.589 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:40:46.589 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6806", 2024-09-21T21:40:46.589 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 3023160932 2024-09-21T21:40:46.589 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:46.590 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:46.590 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:40:46.590 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6807", 2024-09-21T21:40:46.590 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 3023160932 2024-09-21T21:40:46.590 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:46.590 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:46.590 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:46.590 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_front_addrs": { 2024-09-21T21:40:46.590 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:40:46.590 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:46.590 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:40:46.590 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6804", 2024-09-21T21:40:46.590 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 3023160932 2024-09-21T21:40:46.590 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:46.591 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:46.591 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:40:46.591 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6805", 2024-09-21T21:40:46.591 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 3023160932 2024-09-21T21:40:46.591 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:46.591 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:46.591 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:46.591 INFO:teuthology.orchestra.run.smithi081.stdout: "public_addr": "172.21.15.81:6801/3023160932", 2024-09-21T21:40:46.591 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster_addr": "172.21.15.81:6803/3023160932", 2024-09-21T21:40:46.591 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_back_addr": "172.21.15.81:6807/3023160932", 2024-09-21T21:40:46.591 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_front_addr": "172.21.15.81:6805/3023160932", 2024-09-21T21:40:46.591 INFO:teuthology.orchestra.run.smithi081.stdout: "state": [ 2024-09-21T21:40:46.591 INFO:teuthology.orchestra.run.smithi081.stdout: "exists", 2024-09-21T21:40:46.591 INFO:teuthology.orchestra.run.smithi081.stdout: "up" 2024-09-21T21:40:46.592 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:46.592 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:46.592 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:46.592 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": 3, 2024-09-21T21:40:46.592 INFO:teuthology.orchestra.run.smithi081.stdout: "uuid": "e8848768-c1aa-445b-9554-dc6aa4eefc8d", 2024-09-21T21:40:46.592 INFO:teuthology.orchestra.run.smithi081.stdout: "up": 1, 2024-09-21T21:40:46.592 INFO:teuthology.orchestra.run.smithi081.stdout: "in": 1, 2024-09-21T21:40:46.592 INFO:teuthology.orchestra.run.smithi081.stdout: "weight": 1, 2024-09-21T21:40:46.592 INFO:teuthology.orchestra.run.smithi081.stdout: "primary_affinity": 1, 2024-09-21T21:40:46.592 INFO:teuthology.orchestra.run.smithi081.stdout: "last_clean_begin": 0, 2024-09-21T21:40:46.592 INFO:teuthology.orchestra.run.smithi081.stdout: "last_clean_end": 0, 2024-09-21T21:40:46.592 INFO:teuthology.orchestra.run.smithi081.stdout: "up_from": 13, 2024-09-21T21:40:46.592 INFO:teuthology.orchestra.run.smithi081.stdout: "up_thru": 21, 2024-09-21T21:40:46.593 INFO:teuthology.orchestra.run.smithi081.stdout: "down_at": 0, 2024-09-21T21:40:46.593 INFO:teuthology.orchestra.run.smithi081.stdout: "lost_at": 0, 2024-09-21T21:40:46.593 INFO:teuthology.orchestra.run.smithi081.stdout: "public_addrs": { 2024-09-21T21:40:46.593 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:40:46.593 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:46.593 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:40:46.593 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6808", 2024-09-21T21:40:46.593 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2681016844 2024-09-21T21:40:46.593 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:46.593 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:46.593 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:40:46.593 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6809", 2024-09-21T21:40:46.593 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2681016844 2024-09-21T21:40:46.594 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:46.594 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:46.594 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:46.594 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster_addrs": { 2024-09-21T21:40:46.594 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:40:46.594 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:46.594 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:40:46.594 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6810", 2024-09-21T21:40:46.594 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2681016844 2024-09-21T21:40:46.594 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:46.594 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:46.594 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:40:46.594 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6811", 2024-09-21T21:40:46.595 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2681016844 2024-09-21T21:40:46.595 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:46.595 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:46.595 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:46.595 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_back_addrs": { 2024-09-21T21:40:46.595 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:40:46.595 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:46.595 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:40:46.595 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6814", 2024-09-21T21:40:46.595 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2681016844 2024-09-21T21:40:46.595 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:46.595 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:46.595 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:40:46.595 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6815", 2024-09-21T21:40:46.596 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2681016844 2024-09-21T21:40:46.596 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:46.596 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:46.596 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:46.596 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_front_addrs": { 2024-09-21T21:40:46.596 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:40:46.596 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:46.596 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:40:46.596 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6812", 2024-09-21T21:40:46.596 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2681016844 2024-09-21T21:40:46.596 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:46.596 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:46.596 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:40:46.596 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6813", 2024-09-21T21:40:46.596 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2681016844 2024-09-21T21:40:46.597 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:46.597 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:46.597 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:46.597 INFO:teuthology.orchestra.run.smithi081.stdout: "public_addr": "172.21.15.122:6809/2681016844", 2024-09-21T21:40:46.597 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster_addr": "172.21.15.122:6811/2681016844", 2024-09-21T21:40:46.597 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_back_addr": "172.21.15.122:6815/2681016844", 2024-09-21T21:40:46.597 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_front_addr": "172.21.15.122:6813/2681016844", 2024-09-21T21:40:46.597 INFO:teuthology.orchestra.run.smithi081.stdout: "state": [ 2024-09-21T21:40:46.597 INFO:teuthology.orchestra.run.smithi081.stdout: "exists", 2024-09-21T21:40:46.597 INFO:teuthology.orchestra.run.smithi081.stdout: "up" 2024-09-21T21:40:46.597 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:46.597 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:46.597 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:46.597 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": 4, 2024-09-21T21:40:46.598 INFO:teuthology.orchestra.run.smithi081.stdout: "uuid": "b1d8e85d-a207-4ba6-9322-61d07812083e", 2024-09-21T21:40:46.598 INFO:teuthology.orchestra.run.smithi081.stdout: "up": 1, 2024-09-21T21:40:46.598 INFO:teuthology.orchestra.run.smithi081.stdout: "in": 1, 2024-09-21T21:40:46.598 INFO:teuthology.orchestra.run.smithi081.stdout: "weight": 1, 2024-09-21T21:40:46.598 INFO:teuthology.orchestra.run.smithi081.stdout: "primary_affinity": 1, 2024-09-21T21:40:46.598 INFO:teuthology.orchestra.run.smithi081.stdout: "last_clean_begin": 0, 2024-09-21T21:40:46.598 INFO:teuthology.orchestra.run.smithi081.stdout: "last_clean_end": 0, 2024-09-21T21:40:46.598 INFO:teuthology.orchestra.run.smithi081.stdout: "up_from": 13, 2024-09-21T21:40:46.598 INFO:teuthology.orchestra.run.smithi081.stdout: "up_thru": 21, 2024-09-21T21:40:46.598 INFO:teuthology.orchestra.run.smithi081.stdout: "down_at": 0, 2024-09-21T21:40:46.598 INFO:teuthology.orchestra.run.smithi081.stdout: "lost_at": 0, 2024-09-21T21:40:46.598 INFO:teuthology.orchestra.run.smithi081.stdout: "public_addrs": { 2024-09-21T21:40:46.598 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:40:46.598 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:46.598 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:40:46.599 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6800", 2024-09-21T21:40:46.599 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1095023540 2024-09-21T21:40:46.599 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:46.599 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:46.599 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:40:46.599 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6801", 2024-09-21T21:40:46.599 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1095023540 2024-09-21T21:40:46.599 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:46.599 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:46.599 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:46.599 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster_addrs": { 2024-09-21T21:40:46.599 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:40:46.599 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:46.600 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:40:46.600 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6802", 2024-09-21T21:40:46.600 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1095023540 2024-09-21T21:40:46.600 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:46.600 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:46.600 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:40:46.600 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6803", 2024-09-21T21:40:46.600 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1095023540 2024-09-21T21:40:46.600 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:46.600 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:46.600 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:46.600 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_back_addrs": { 2024-09-21T21:40:46.600 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:40:46.600 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:46.600 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:40:46.601 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6806", 2024-09-21T21:40:46.601 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1095023540 2024-09-21T21:40:46.601 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:46.601 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:46.601 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:40:46.601 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6807", 2024-09-21T21:40:46.601 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1095023540 2024-09-21T21:40:46.601 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:46.601 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:46.601 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:46.601 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_front_addrs": { 2024-09-21T21:40:46.601 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:40:46.601 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:46.601 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:40:46.601 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6804", 2024-09-21T21:40:46.602 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1095023540 2024-09-21T21:40:46.602 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:46.602 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:46.602 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:40:46.602 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6805", 2024-09-21T21:40:46.602 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1095023540 2024-09-21T21:40:46.602 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:46.602 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:46.602 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:46.602 INFO:teuthology.orchestra.run.smithi081.stdout: "public_addr": "172.21.15.122:6801/1095023540", 2024-09-21T21:40:46.602 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster_addr": "172.21.15.122:6803/1095023540", 2024-09-21T21:40:46.602 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_back_addr": "172.21.15.122:6807/1095023540", 2024-09-21T21:40:46.602 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_front_addr": "172.21.15.122:6805/1095023540", 2024-09-21T21:40:46.602 INFO:teuthology.orchestra.run.smithi081.stdout: "state": [ 2024-09-21T21:40:46.603 INFO:teuthology.orchestra.run.smithi081.stdout: "exists", 2024-09-21T21:40:46.603 INFO:teuthology.orchestra.run.smithi081.stdout: "up" 2024-09-21T21:40:46.603 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:46.603 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:46.603 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:46.603 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": 5, 2024-09-21T21:40:46.603 INFO:teuthology.orchestra.run.smithi081.stdout: "uuid": "3ab0450b-d8fd-4ef2-a298-08d5aa2ea32f", 2024-09-21T21:40:46.603 INFO:teuthology.orchestra.run.smithi081.stdout: "up": 1, 2024-09-21T21:40:46.603 INFO:teuthology.orchestra.run.smithi081.stdout: "in": 1, 2024-09-21T21:40:46.603 INFO:teuthology.orchestra.run.smithi081.stdout: "weight": 1, 2024-09-21T21:40:46.603 INFO:teuthology.orchestra.run.smithi081.stdout: "primary_affinity": 1, 2024-09-21T21:40:46.603 INFO:teuthology.orchestra.run.smithi081.stdout: "last_clean_begin": 0, 2024-09-21T21:40:46.603 INFO:teuthology.orchestra.run.smithi081.stdout: "last_clean_end": 0, 2024-09-21T21:40:46.603 INFO:teuthology.orchestra.run.smithi081.stdout: "up_from": 12, 2024-09-21T21:40:46.603 INFO:teuthology.orchestra.run.smithi081.stdout: "up_thru": 21, 2024-09-21T21:40:46.604 INFO:teuthology.orchestra.run.smithi081.stdout: "down_at": 0, 2024-09-21T21:40:46.604 INFO:teuthology.orchestra.run.smithi081.stdout: "lost_at": 0, 2024-09-21T21:40:46.604 INFO:teuthology.orchestra.run.smithi081.stdout: "public_addrs": { 2024-09-21T21:40:46.604 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:40:46.604 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:46.604 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:40:46.604 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6816", 2024-09-21T21:40:46.604 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1974398333 2024-09-21T21:40:46.604 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:46.604 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:46.604 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:40:46.604 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6817", 2024-09-21T21:40:46.604 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1974398333 2024-09-21T21:40:46.604 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:46.604 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:46.605 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:46.605 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster_addrs": { 2024-09-21T21:40:46.605 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:40:46.605 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:46.605 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:40:46.605 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6818", 2024-09-21T21:40:46.605 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1974398333 2024-09-21T21:40:46.605 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:46.605 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:46.605 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:40:46.605 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6819", 2024-09-21T21:40:46.605 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1974398333 2024-09-21T21:40:46.605 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:46.605 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:46.606 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:46.606 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_back_addrs": { 2024-09-21T21:40:46.606 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:40:46.606 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:46.606 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:40:46.606 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6822", 2024-09-21T21:40:46.606 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1974398333 2024-09-21T21:40:46.606 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:46.606 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:46.606 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:40:46.606 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6823", 2024-09-21T21:40:46.606 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1974398333 2024-09-21T21:40:46.606 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:46.606 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:46.607 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:46.607 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_front_addrs": { 2024-09-21T21:40:46.607 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:40:46.607 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:46.607 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:40:46.607 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6820", 2024-09-21T21:40:46.607 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1974398333 2024-09-21T21:40:46.607 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:46.607 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:46.607 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:40:46.607 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6821", 2024-09-21T21:40:46.607 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1974398333 2024-09-21T21:40:46.607 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:46.607 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:46.608 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:46.608 INFO:teuthology.orchestra.run.smithi081.stdout: "public_addr": "172.21.15.122:6817/1974398333", 2024-09-21T21:40:46.608 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster_addr": "172.21.15.122:6819/1974398333", 2024-09-21T21:40:46.608 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_back_addr": "172.21.15.122:6823/1974398333", 2024-09-21T21:40:46.608 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_front_addr": "172.21.15.122:6821/1974398333", 2024-09-21T21:40:46.608 INFO:teuthology.orchestra.run.smithi081.stdout: "state": [ 2024-09-21T21:40:46.608 INFO:teuthology.orchestra.run.smithi081.stdout: "exists", 2024-09-21T21:40:46.608 INFO:teuthology.orchestra.run.smithi081.stdout: "up" 2024-09-21T21:40:46.608 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:46.608 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:46.608 INFO:teuthology.orchestra.run.smithi081.stdout: ], 2024-09-21T21:40:46.608 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_xinfo": [ 2024-09-21T21:40:46.608 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:46.608 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": 0, 2024-09-21T21:40:46.609 INFO:teuthology.orchestra.run.smithi081.stdout: "down_stamp": "0.000000", 2024-09-21T21:40:46.609 INFO:teuthology.orchestra.run.smithi081.stdout: "laggy_probability": 0, 2024-09-21T21:40:46.609 INFO:teuthology.orchestra.run.smithi081.stdout: "laggy_interval": 0, 2024-09-21T21:40:46.609 INFO:teuthology.orchestra.run.smithi081.stdout: "features": 4540701547738038271, 2024-09-21T21:40:46.609 INFO:teuthology.orchestra.run.smithi081.stdout: "old_weight": 0, 2024-09-21T21:40:46.609 INFO:teuthology.orchestra.run.smithi081.stdout: "last_purged_snaps_scrub": "2024-09-21T21:40:15.766648+0000", 2024-09-21T21:40:46.609 INFO:teuthology.orchestra.run.smithi081.stdout: "dead_epoch": 0 2024-09-21T21:40:46.609 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:46.609 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:46.609 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": 1, 2024-09-21T21:40:46.609 INFO:teuthology.orchestra.run.smithi081.stdout: "down_stamp": "0.000000", 2024-09-21T21:40:46.609 INFO:teuthology.orchestra.run.smithi081.stdout: "laggy_probability": 0, 2024-09-21T21:40:46.609 INFO:teuthology.orchestra.run.smithi081.stdout: "laggy_interval": 0, 2024-09-21T21:40:46.609 INFO:teuthology.orchestra.run.smithi081.stdout: "features": 4540701547738038271, 2024-09-21T21:40:46.610 INFO:teuthology.orchestra.run.smithi081.stdout: "old_weight": 0, 2024-09-21T21:40:46.610 INFO:teuthology.orchestra.run.smithi081.stdout: "last_purged_snaps_scrub": "2024-09-21T21:40:15.457496+0000", 2024-09-21T21:40:46.610 INFO:teuthology.orchestra.run.smithi081.stdout: "dead_epoch": 0 2024-09-21T21:40:46.610 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:46.610 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:46.610 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": 2, 2024-09-21T21:40:46.610 INFO:teuthology.orchestra.run.smithi081.stdout: "down_stamp": "0.000000", 2024-09-21T21:40:46.610 INFO:teuthology.orchestra.run.smithi081.stdout: "laggy_probability": 0, 2024-09-21T21:40:46.610 INFO:teuthology.orchestra.run.smithi081.stdout: "laggy_interval": 0, 2024-09-21T21:40:46.610 INFO:teuthology.orchestra.run.smithi081.stdout: "features": 4540701547738038271, 2024-09-21T21:40:46.610 INFO:teuthology.orchestra.run.smithi081.stdout: "old_weight": 0, 2024-09-21T21:40:46.610 INFO:teuthology.orchestra.run.smithi081.stdout: "last_purged_snaps_scrub": "2024-09-21T21:40:14.767039+0000", 2024-09-21T21:40:46.610 INFO:teuthology.orchestra.run.smithi081.stdout: "dead_epoch": 0 2024-09-21T21:40:46.610 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:46.610 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:46.611 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": 3, 2024-09-21T21:40:46.611 INFO:teuthology.orchestra.run.smithi081.stdout: "down_stamp": "0.000000", 2024-09-21T21:40:46.611 INFO:teuthology.orchestra.run.smithi081.stdout: "laggy_probability": 0, 2024-09-21T21:40:46.611 INFO:teuthology.orchestra.run.smithi081.stdout: "laggy_interval": 0, 2024-09-21T21:40:46.611 INFO:teuthology.orchestra.run.smithi081.stdout: "features": 4540701547738038271, 2024-09-21T21:40:46.611 INFO:teuthology.orchestra.run.smithi081.stdout: "old_weight": 0, 2024-09-21T21:40:46.611 INFO:teuthology.orchestra.run.smithi081.stdout: "last_purged_snaps_scrub": "2024-09-21T21:40:15.355570+0000", 2024-09-21T21:40:46.611 INFO:teuthology.orchestra.run.smithi081.stdout: "dead_epoch": 0 2024-09-21T21:40:46.611 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:46.611 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:46.611 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": 4, 2024-09-21T21:40:46.611 INFO:teuthology.orchestra.run.smithi081.stdout: "down_stamp": "0.000000", 2024-09-21T21:40:46.611 INFO:teuthology.orchestra.run.smithi081.stdout: "laggy_probability": 0, 2024-09-21T21:40:46.611 INFO:teuthology.orchestra.run.smithi081.stdout: "laggy_interval": 0, 2024-09-21T21:40:46.612 INFO:teuthology.orchestra.run.smithi081.stdout: "features": 4540701547738038271, 2024-09-21T21:40:46.612 INFO:teuthology.orchestra.run.smithi081.stdout: "old_weight": 0, 2024-09-21T21:40:46.612 INFO:teuthology.orchestra.run.smithi081.stdout: "last_purged_snaps_scrub": "2024-09-21T21:40:15.335729+0000", 2024-09-21T21:40:46.612 INFO:teuthology.orchestra.run.smithi081.stdout: "dead_epoch": 0 2024-09-21T21:40:46.612 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:46.612 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:46.612 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": 5, 2024-09-21T21:40:46.612 INFO:teuthology.orchestra.run.smithi081.stdout: "down_stamp": "0.000000", 2024-09-21T21:40:46.612 INFO:teuthology.orchestra.run.smithi081.stdout: "laggy_probability": 0, 2024-09-21T21:40:46.612 INFO:teuthology.orchestra.run.smithi081.stdout: "laggy_interval": 0, 2024-09-21T21:40:46.612 INFO:teuthology.orchestra.run.smithi081.stdout: "features": 4540701547738038271, 2024-09-21T21:40:46.612 INFO:teuthology.orchestra.run.smithi081.stdout: "old_weight": 0, 2024-09-21T21:40:46.612 INFO:teuthology.orchestra.run.smithi081.stdout: "last_purged_snaps_scrub": "2024-09-21T21:40:15.614733+0000", 2024-09-21T21:40:46.612 INFO:teuthology.orchestra.run.smithi081.stdout: "dead_epoch": 0 2024-09-21T21:40:46.613 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:46.613 INFO:teuthology.orchestra.run.smithi081.stdout: ], 2024-09-21T21:40:46.613 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_upmap": [], 2024-09-21T21:40:46.613 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_upmap_items": [], 2024-09-21T21:40:46.613 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_upmap_primaries": [], 2024-09-21T21:40:46.613 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_temp": [], 2024-09-21T21:40:46.613 INFO:teuthology.orchestra.run.smithi081.stdout: "primary_temp": [], 2024-09-21T21:40:46.613 INFO:teuthology.orchestra.run.smithi081.stdout: "blocklist": {}, 2024-09-21T21:40:46.613 INFO:teuthology.orchestra.run.smithi081.stdout: "range_blocklist": {}, 2024-09-21T21:40:46.613 INFO:teuthology.orchestra.run.smithi081.stdout: "erasure_code_profiles": { 2024-09-21T21:40:46.613 INFO:teuthology.orchestra.run.smithi081.stdout: "default": { 2024-09-21T21:40:46.613 INFO:teuthology.orchestra.run.smithi081.stdout: "crush-failure-domain": "osd", 2024-09-21T21:40:46.613 INFO:teuthology.orchestra.run.smithi081.stdout: "k": "2", 2024-09-21T21:40:46.613 INFO:teuthology.orchestra.run.smithi081.stdout: "m": "1", 2024-09-21T21:40:46.614 INFO:teuthology.orchestra.run.smithi081.stdout: "plugin": "jerasure", 2024-09-21T21:40:46.614 INFO:teuthology.orchestra.run.smithi081.stdout: "technique": "reed_sol_van" 2024-09-21T21:40:46.614 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:46.614 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:46.614 INFO:teuthology.orchestra.run.smithi081.stdout: "removed_snaps_queue": [ 2024-09-21T21:40:46.614 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:46.614 INFO:teuthology.orchestra.run.smithi081.stdout: "pool": 3, 2024-09-21T21:40:46.614 INFO:teuthology.orchestra.run.smithi081.stdout: "snaps": [ 2024-09-21T21:40:46.614 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:46.614 INFO:teuthology.orchestra.run.smithi081.stdout: "begin": 2, 2024-09-21T21:40:46.614 INFO:teuthology.orchestra.run.smithi081.stdout: "length": 1 2024-09-21T21:40:46.614 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:46.614 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:46.614 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:46.615 INFO:teuthology.orchestra.run.smithi081.stdout: ], 2024-09-21T21:40:46.615 INFO:teuthology.orchestra.run.smithi081.stdout: "new_removed_snaps": [], 2024-09-21T21:40:46.615 INFO:teuthology.orchestra.run.smithi081.stdout: "new_purged_snaps": [], 2024-09-21T21:40:46.615 INFO:teuthology.orchestra.run.smithi081.stdout: "crush_node_flags": {}, 2024-09-21T21:40:46.615 INFO:teuthology.orchestra.run.smithi081.stdout: "device_class_flags": {}, 2024-09-21T21:40:46.615 INFO:teuthology.orchestra.run.smithi081.stdout: "stretch_mode": { 2024-09-21T21:40:46.615 INFO:teuthology.orchestra.run.smithi081.stdout: "stretch_mode_enabled": false, 2024-09-21T21:40:46.615 INFO:teuthology.orchestra.run.smithi081.stdout: "stretch_bucket_count": 0, 2024-09-21T21:40:46.615 INFO:teuthology.orchestra.run.smithi081.stdout: "degraded_stretch_mode": 0, 2024-09-21T21:40:46.615 INFO:teuthology.orchestra.run.smithi081.stdout: "recovering_stretch_mode": 0, 2024-09-21T21:40:46.615 INFO:teuthology.orchestra.run.smithi081.stdout: "stretch_mode_bucket": 0 2024-09-21T21:40:46.615 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:46.615 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:40:46.617 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:40:46.615+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:40:46.617 INFO:teuthology.orchestra.run.smithi164.stdout:176 trunc from 0xb7c400 to 0x9c0ae00 2024-09-21T21:40:46.618 INFO:teuthology.orchestra.run.smithi164.stdout:182 read 0x624e200 thru 0x6255cbf (0x7ac0 bytes) 2024-09-21T21:40:46.618 INFO:teuthology.orchestra.run.smithi164.stdout:184 trunc from 0x9c0ae00 to 0x8298800 2024-09-21T21:40:46.618 INFO:teuthology.orchestra.run.smithi164.stdout:185 read 0x7427200 thru 0x742e6fa (0x74fb bytes) 2024-09-21T21:40:46.618 INFO:teuthology.orchestra.run.smithi164.stdout:187 write 0x4a49200 thru 0x4a4ad29 (0x1b2a bytes) 2024-09-21T21:40:46.618 INFO:teuthology.orchestra.run.smithi164.stdout:188 write 0x8d88c00 thru 0x8d8c19b (0x359c bytes) 2024-09-21T21:40:46.619 INFO:tasks.thrashosds.thrasher:j is {'epoch': 27, 'fsid': 'a0fdcfc2-ddd2-4e50-81bb-876d940f3578', 'created': '2024-09-21T21:40:07.413917+0000', 'modified': '2024-09-21T21:40:45.449182+0000', 'last_up_change': '2024-09-21T21:40:18.340430+0000', 'last_in_change': '2024-09-21T21:40:09.029953+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T21:40:18.321328+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-21T21:40:24.235658+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 19, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-21T21:40:36.093691+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 5, 'snap_epoch': 27, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': 'a9869830-8e74-4b02-963d-ab630a0ee286', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 21, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6808', 'nonce': 1657552338}, {'type': 'v1', 'addr': '172.21.15.81:6809', 'nonce': 1657552338}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6810', 'nonce': 1657552338}, {'type': 'v1', 'addr': '172.21.15.81:6811', 'nonce': 1657552338}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6814', 'nonce': 1657552338}, {'type': 'v1', 'addr': '172.21.15.81:6815', 'nonce': 1657552338}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6812', 'nonce': 1657552338}, {'type': 'v1', 'addr': '172.21.15.81:6813', 'nonce': 1657552338}]}, 'public_addr': '172.21.15.81:6809/1657552338', 'cluster_addr': '172.21.15.81:6811/1657552338', 'heartbeat_back_addr': '172.21.15.81:6815/1657552338', 'heartbeat_front_addr': '172.21.15.81:6813/1657552338', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'b189819d-69a3-4e95-96e7-96dc4809ac80', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 21, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6816', 'nonce': 2864159502}, {'type': 'v1', 'addr': '172.21.15.81:6817', 'nonce': 2864159502}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6818', 'nonce': 2864159502}, {'type': 'v1', 'addr': '172.21.15.81:6819', 'nonce': 2864159502}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6822', 'nonce': 2864159502}, {'type': 'v1', 'addr': '172.21.15.81:6823', 'nonce': 2864159502}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6820', 'nonce': 2864159502}, {'type': 'v1', 'addr': '172.21.15.81:6821', 'nonce': 2864159502}]}, 'public_addr': '172.21.15.81:6817/2864159502', 'cluster_addr': '172.21.15.81:6819/2864159502', 'heartbeat_back_addr': '172.21.15.81:6823/2864159502', 'heartbeat_front_addr': '172.21.15.81:6821/2864159502', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '8437b99a-1297-4903-ab66-95c5cba2b2bf', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 21, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6800', 'nonce': 3023160932}, {'type': 'v1', 'addr': '172.21.15.81:6801', 'nonce': 3023160932}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6802', 'nonce': 3023160932}, {'type': 'v1', 'addr': '172.21.15.81:6803', 'nonce': 3023160932}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6806', 'nonce': 3023160932}, {'type': 'v1', 'addr': '172.21.15.81:6807', 'nonce': 3023160932}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6804', 'nonce': 3023160932}, {'type': 'v1', 'addr': '172.21.15.81:6805', 'nonce': 3023160932}]}, 'public_addr': '172.21.15.81:6801/3023160932', 'cluster_addr': '172.21.15.81:6803/3023160932', 'heartbeat_back_addr': '172.21.15.81:6807/3023160932', 'heartbeat_front_addr': '172.21.15.81:6805/3023160932', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'e8848768-c1aa-445b-9554-dc6aa4eefc8d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 21, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6808', 'nonce': 2681016844}, {'type': 'v1', 'addr': '172.21.15.122:6809', 'nonce': 2681016844}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6810', 'nonce': 2681016844}, {'type': 'v1', 'addr': '172.21.15.122:6811', 'nonce': 2681016844}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6814', 'nonce': 2681016844}, {'type': 'v1', 'addr': '172.21.15.122:6815', 'nonce': 2681016844}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6812', 'nonce': 2681016844}, {'type': 'v1', 'addr': '172.21.15.122:6813', 'nonce': 2681016844}]}, 'public_addr': '172.21.15.122:6809/2681016844', 'cluster_addr': '172.21.15.122:6811/2681016844', 'heartbeat_back_addr': '172.21.15.122:6815/2681016844', 'heartbeat_front_addr': '172.21.15.122:6813/2681016844', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'b1d8e85d-a207-4ba6-9322-61d07812083e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 21, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6800', 'nonce': 1095023540}, {'type': 'v1', 'addr': '172.21.15.122:6801', 'nonce': 1095023540}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6802', 'nonce': 1095023540}, {'type': 'v1', 'addr': '172.21.15.122:6803', 'nonce': 1095023540}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6806', 'nonce': 1095023540}, {'type': 'v1', 'addr': '172.21.15.122:6807', 'nonce': 1095023540}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6804', 'nonce': 1095023540}, {'type': 'v1', 'addr': '172.21.15.122:6805', 'nonce': 1095023540}]}, 'public_addr': '172.21.15.122:6801/1095023540', 'cluster_addr': '172.21.15.122:6803/1095023540', 'heartbeat_back_addr': '172.21.15.122:6807/1095023540', 'heartbeat_front_addr': '172.21.15.122:6805/1095023540', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '3ab0450b-d8fd-4ef2-a298-08d5aa2ea32f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 21, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6816', 'nonce': 1974398333}, {'type': 'v1', 'addr': '172.21.15.122:6817', 'nonce': 1974398333}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6818', 'nonce': 1974398333}, {'type': 'v1', 'addr': '172.21.15.122:6819', 'nonce': 1974398333}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6822', 'nonce': 1974398333}, {'type': 'v1', 'addr': '172.21.15.122:6823', 'nonce': 1974398333}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6820', 'nonce': 1974398333}, {'type': 'v1', 'addr': '172.21.15.122:6821', 'nonce': 1974398333}]}, 'public_addr': '172.21.15.122:6817/1974398333', 'cluster_addr': '172.21.15.122:6819/1974398333', 'heartbeat_back_addr': '172.21.15.122:6823/1974398333', 'heartbeat_front_addr': '172.21.15.122:6821/1974398333', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T21:40:15.766648+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T21:40:15.457496+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T21:40:14.767039+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T21:40:15.355570+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T21:40:15.335729+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T21:40:15.614733+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 2, 'length': 1}]}], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-21T21:40:46.619 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2024-09-21T21:40:46.624 INFO:teuthology.orchestra.run.smithi164.stdout:189 write 0xe7de800 thru 0xe7dfb63 (0x1364 bytes) 2024-09-21T21:40:46.625 INFO:teuthology.orchestra.run.smithi164.stdout:190 write 0x2045000 thru 0x2050e4b (0xbe4c bytes) 2024-09-21T21:40:46.635 INFO:teuthology.orchestra.run.smithi164.stdout:191 trunc from 0xe7dfc00 to 0x5cb5a00 2024-09-21T21:40:46.661 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:46.662 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:46.671 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T21:40:46.683 INFO:teuthology.orchestra.run.smithi164.stdout:193 write 0xd5c4600 thru 0xd5c48f3 (0x2f4 bytes) 2024-09-21T21:40:46.684 INFO:teuthology.orchestra.run.smithi164.stdout:199 read 0x8aef400 thru 0x8afb17a (0xbd7b bytes) 2024-09-21T21:40:46.684 INFO:teuthology.orchestra.run.smithi164.stdout:200 write 0x2bf9e00 thru 0x2c0788a (0xda8b bytes) 2024-09-21T21:40:46.684 INFO:teuthology.orchestra.run.smithi164.stdout:202 read 0x2356800 thru 0x235f120 (0x8921 bytes) 2024-09-21T21:40:46.708 INFO:teuthology.orchestra.run.smithi164.stdout:203 write 0xd772a00 thru 0xd78289f (0xfea0 bytes) 2024-09-21T21:40:46.717 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:40:46.716+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:40:46.717 INFO:teuthology.orchestra.run.smithi164.stdout:204 trunc from 0xd782a00 to 0x6bed000 2024-09-21T21:40:46.742 INFO:teuthology.orchestra.run.smithi164.stdout:205 read 0x103d400 thru 0x1042b2a (0x572b bytes) 2024-09-21T21:40:46.747 INFO:teuthology.orchestra.run.smithi164.stdout:206 read 0x2769c00 thru 0x276e96a (0x4d6b bytes) 2024-09-21T21:40:46.759 INFO:teuthology.orchestra.run.smithi164.stdout:208 write 0x8677800 thru 0x867f71b (0x7f1c bytes) 2024-09-21T21:40:46.760 INFO:teuthology.orchestra.run.smithi164.stdout:210 read 0x3559e00 thru 0x3562d76 (0x8f77 bytes) 2024-09-21T21:40:46.760 INFO:teuthology.orchestra.run.smithi164.stdout:214 read 0x536b800 thru 0x537190f (0x6110 bytes) 2024-09-21T21:40:46.760 INFO:teuthology.orchestra.run.smithi164.stdout:215 read 0x3358400 thru 0x335c015 (0x3c16 bytes) 2024-09-21T21:40:46.761 INFO:teuthology.orchestra.run.smithi164.stdout:217 read 0x6217600 thru 0x621efe1 (0x79e2 bytes) 2024-09-21T21:40:46.761 INFO:teuthology.orchestra.run.smithi164.stdout:218 punch from 0x67d1e00 to 0x67e0200, (0xe400 bytes) 2024-09-21T21:40:46.791 INFO:teuthology.orchestra.run.smithi164.stdout:221 write 0xbab0c00 thru 0xbab9c71 (0x9072 bytes) 2024-09-21T21:40:46.792 INFO:teuthology.orchestra.run.smithi164.stdout:222 punch from 0x588a600 to 0x5892a00, (0x8400 bytes) 2024-09-21T21:40:46.792 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:46.792 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:46.793 INFO:teuthology.orchestra.run.smithi164.stdout:223 read 0x68e7e00 thru 0x68ea69e (0x289f bytes) 2024-09-21T21:40:46.793 INFO:teuthology.orchestra.run.smithi164.stdout:224 read 0x45dfc00 thru 0x45e74a8 (0x78a9 bytes) 2024-09-21T21:40:46.793 INFO:teuthology.orchestra.run.smithi164.stdout:225 read 0x2ff3800 thru 0x3002792 (0xef93 bytes) 2024-09-21T21:40:46.796 INFO:teuthology.orchestra.run.smithi164.stdout:228 read 0x1620200 thru 0x162dbb7 (0xd9b8 bytes) 2024-09-21T21:40:46.803 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T21:40:46.810 INFO:teuthology.orchestra.run.smithi164.stdout:229 write 0xcbc1200 thru 0xcbc1c25 (0xa26 bytes) 2024-09-21T21:40:46.811 INFO:teuthology.orchestra.run.smithi164.stdout:231 punch from 0x184ba00 to 0x1856a00, (0xb000 bytes) 2024-09-21T21:40:46.812 INFO:teuthology.orchestra.run.smithi164.stdout:232 read 0x5742e00 thru 0x574d78e (0xa98f bytes) 2024-09-21T21:40:46.813 INFO:teuthology.orchestra.run.smithi164.stdout:233 punch from 0x2162a00 to 0x2166c00, (0x4200 bytes) 2024-09-21T21:40:46.813 INFO:teuthology.orchestra.run.smithi164.stdout:truncating to largest ever: 0xb247c00 2024-09-21T21:40:46.814 INFO:teuthology.orchestra.run.smithi164.stdout:234 trunc from 0xcbc1e00 to 0xb247c00 2024-09-21T21:40:46.817 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:40:46.816+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:40:46.831 INFO:teuthology.orchestra.run.smithi164.stdout:235 punch from 0x36b9600 to 0x36c0200, (0x6c00 bytes) 2024-09-21T21:40:46.839 INFO:teuthology.orchestra.run.smithi164.stdout:236 read 0x8adf400 thru 0x8ae4233 (0x4e34 bytes) 2024-09-21T21:40:46.844 INFO:teuthology.orchestra.run.smithi164.stdout:237 write 0x6a30400 thru 0x6a324da (0x20db bytes) 2024-09-21T21:40:46.845 INFO:teuthology.orchestra.run.smithi164.stdout:239 flatten 2024-09-21T21:40:46.918 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:40:46.916+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:40:46.950 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:46.950 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:46.958 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T21:40:47.019 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:40:47.018+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:40:47.058 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T21:40:47.104 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:47.104 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:47.113 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T21:40:47.120 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:40:47.118+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:40:47.120 INFO:teuthology.orchestra.run.smithi164.stdout:241 write 0xc2dde00 thru 0xc2e3e9d (0x609e bytes) 2024-09-21T21:40:47.120 INFO:teuthology.orchestra.run.smithi164.stdout:247 punch from 0x159e200 to 0x15a7e00, (0x9c00 bytes) 2024-09-21T21:40:47.120 INFO:teuthology.orchestra.run.smithi164.stdout:248 write 0x1777400 thru 0x177b7e1 (0x43e2 bytes) 2024-09-21T21:40:47.120 INFO:teuthology.orchestra.run.smithi164.stdout:250 read 0xb632000 thru 0xb63e423 (0xc424 bytes) 2024-09-21T21:40:47.121 INFO:teuthology.orchestra.run.smithi164.stdout:251 write 0x84a9600 thru 0x84b4af1 (0xb4f2 bytes) 2024-09-21T21:40:47.123 INFO:teuthology.orchestra.run.smithi164.stdout:252 punch from 0x7b4d000 to 0x7b53400, (0x6400 bytes) 2024-09-21T21:40:47.135 INFO:teuthology.orchestra.run.smithi164.stdout:253 trunc from 0xc2e4000 to 0x30da600 2024-09-21T21:40:47.169 INFO:teuthology.orchestra.run.smithi164.stdout:254 clone 4 order 20 su 32768 sc 2 2024-09-21T21:40:47.194 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0: {} 2024-09-21T21:40:47.194 INFO:teuthology.orchestra.run.smithi081.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T21:40:47.218 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1: {} 2024-09-21T21:40:47.218 INFO:teuthology.orchestra.run.smithi081.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T21:40:47.220 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:40:47.218+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:40:47.381 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2: {} 2024-09-21T21:40:47.381 INFO:teuthology.orchestra.run.smithi081.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T21:40:47.383 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:40:47.383 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 20, 2024-09-21T21:40:47.383 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 600, 2024-09-21T21:40:47.383 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:40:47.383 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.383 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:124 3.2 3:42a46321:::rbd_data.11214be7c411.0000000000000020:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:47.383 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.534750+0000", 2024-09-21T21:40:47.383 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.7048713289999999, 2024-09-21T21:40:47.383 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0073162330000000001, 2024-09-21T21:40:47.383 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:47.383 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:47.383 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:40:47.383 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:47.384 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:47.384 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:47.384 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 124 2024-09-21T21:40:47.384 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.384 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:47.384 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.384 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:47.384 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.534750+0000", 2024-09-21T21:40:47.384 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:47.384 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.384 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.384 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:47.384 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.534750+0000", 2024-09-21T21:40:47.385 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:47.385 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.385 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.385 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:47.385 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.534751+0000", 2024-09-21T21:40:47.385 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4300000000000003e-07 2024-09-21T21:40:47.385 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.385 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.385 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:47.385 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.534753+0000", 2024-09-21T21:40:47.385 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.294e-06 2024-09-21T21:40:47.385 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.385 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.386 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:47.386 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.534759+0000", 2024-09-21T21:40:47.386 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3409999999999999e-06 2024-09-21T21:40:47.386 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.386 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.386 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:47.386 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.534764+0000", 2024-09-21T21:40:47.386 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9490000000000004e-06 2024-09-21T21:40:47.386 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.386 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.386 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:47.386 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.534817+0000", 2024-09-21T21:40:47.386 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2954000000000002e-05 2024-09-21T21:40:47.387 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.387 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.387 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:47.387 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.534924+0000", 2024-09-21T21:40:47.387 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000106884 2024-09-21T21:40:47.387 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.387 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.387 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:47.387 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.542067+0000", 2024-09-21T21:40:47.387 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0071422680000000002 2024-09-21T21:40:47.387 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:47.387 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:47.387 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:47.387 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.388 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.388 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "pg_update_log_missing(3.8 epoch 26/21 rep_tid 13 entries 26'4 (0'0) error 3:108e7c9d:::rbd_data.11214be7c411.000000000000001f:head by client.4417.0:122 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 25'2)", 2024-09-21T21:40:47.388 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.535157+0000", 2024-09-21T21:40:47.388 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.7044645950000001, 2024-09-21T21:40:47.388 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0068721269999999996, 2024-09-21T21:40:47.388 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:47.388 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:47.388 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:40:47.388 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:47.388 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.388 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:47.388 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.535157+0000", 2024-09-21T21:40:47.388 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:47.388 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.388 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.389 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:47.389 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.535157+0000", 2024-09-21T21:40:47.389 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:47.389 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.389 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.389 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:47.389 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.535158+0000", 2024-09-21T21:40:47.389 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7999999999999995e-07 2024-09-21T21:40:47.389 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.389 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.389 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:47.389 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.535165+0000", 2024-09-21T21:40:47.389 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.2869999999999999e-06 2024-09-21T21:40:47.389 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.390 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.390 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:47.390 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.535179+0000", 2024-09-21T21:40:47.390 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.4092e-05 2024-09-21T21:40:47.390 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.390 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.390 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:47.390 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.535192+0000", 2024-09-21T21:40:47.390 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.3107e-05 2024-09-21T21:40:47.390 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.390 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.390 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:47.390 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.535257+0000", 2024-09-21T21:40:47.391 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4543999999999997e-05 2024-09-21T21:40:47.391 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.391 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.391 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:47.391 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.542029+0000", 2024-09-21T21:40:47.391 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0067725169999999996 2024-09-21T21:40:47.391 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:47.391 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:47.391 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:47.391 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.391 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.391 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:147 3.19 3:9f7f40ef:::rbd_data.11214be7c411.000000000000001d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:47.391 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.536972+0000", 2024-09-21T21:40:47.391 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.7026492709999999, 2024-09-21T21:40:47.392 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0078089780000000003, 2024-09-21T21:40:47.392 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:47.392 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:47.392 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:47.392 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:47.392 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:47.392 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:47.392 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 147 2024-09-21T21:40:47.392 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.392 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:47.392 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.392 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:47.392 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.536972+0000", 2024-09-21T21:40:47.392 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:47.393 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.393 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.393 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:47.393 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.536972+0000", 2024-09-21T21:40:47.393 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:47.393 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.393 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.393 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:47.393 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.536973+0000", 2024-09-21T21:40:47.393 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.9999999999999998e-07 2024-09-21T21:40:47.393 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.393 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.393 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:47.393 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.536977+0000", 2024-09-21T21:40:47.394 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8850000000000001e-06 2024-09-21T21:40:47.394 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.394 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.394 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:47.394 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.536982+0000", 2024-09-21T21:40:47.394 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6590000000000004e-06 2024-09-21T21:40:47.394 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.394 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.394 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:47.394 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.536986+0000", 2024-09-21T21:40:47.394 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4999999999999999e-06 2024-09-21T21:40:47.395 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.395 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.395 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:47.395 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.539409+0000", 2024-09-21T21:40:47.395 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0024234259999999998 2024-09-21T21:40:47.395 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.395 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.395 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:47.395 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.539480+0000", 2024-09-21T21:40:47.395 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.1181999999999997e-05 2024-09-21T21:40:47.395 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.395 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.395 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 4", 2024-09-21T21:40:47.395 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.539597+0000", 2024-09-21T21:40:47.396 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000116239 2024-09-21T21:40:47.396 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.396 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.396 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:47.396 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.541542+0000", 2024-09-21T21:40:47.396 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.001945801 2024-09-21T21:40:47.396 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.396 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.396 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:40:47.396 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.544698+0000", 2024-09-21T21:40:47.396 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.003156042 2024-09-21T21:40:47.396 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.396 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.397 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:47.397 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.544747+0000", 2024-09-21T21:40:47.397 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8869000000000003e-05 2024-09-21T21:40:47.397 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.397 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.397 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:47.397 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.544781+0000", 2024-09-21T21:40:47.397 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3974999999999999e-05 2024-09-21T21:40:47.397 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:47.397 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:47.397 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:47.397 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.397 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.397 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "pg_update_log_missing(3.8 epoch 26/21 rep_tid 15 entries 26'5 (0'0) error 3:15b4ee09:::rbd_data.11214be7c411.0000000000000028:head by client.4417.0:154 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 25'2)", 2024-09-21T21:40:47.398 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.538536+0000", 2024-09-21T21:40:47.398 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.7010854760000003, 2024-09-21T21:40:47.398 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0061585449999999996, 2024-09-21T21:40:47.398 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:47.398 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:47.398 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:40:47.398 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:47.398 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.398 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:47.398 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.538536+0000", 2024-09-21T21:40:47.398 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:47.398 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.398 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.398 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:47.399 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.538536+0000", 2024-09-21T21:40:47.399 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:47.399 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.399 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.399 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:47.399 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.538537+0000", 2024-09-21T21:40:47.399 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9200000000000001e-07 2024-09-21T21:40:47.399 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.399 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.399 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:47.399 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.538547+0000", 2024-09-21T21:40:47.399 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0319999999999999e-05 2024-09-21T21:40:47.399 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.399 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.400 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:47.400 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.538565+0000", 2024-09-21T21:40:47.400 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.8080999999999998e-05 2024-09-21T21:40:47.400 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.400 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.400 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:47.400 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.538582+0000", 2024-09-21T21:40:47.400 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.7220000000000001e-05 2024-09-21T21:40:47.400 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.400 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.400 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:47.400 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.538644+0000", 2024-09-21T21:40:47.400 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1383999999999994e-05 2024-09-21T21:40:47.400 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.401 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.401 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:47.401 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.544695+0000", 2024-09-21T21:40:47.401 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0060509479999999996 2024-09-21T21:40:47.401 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:47.401 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:47.401 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:47.401 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.401 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.401 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:195 3.19 3:9c6b57eb:::rbd_object_map.11214be7c411.0000000000000004:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:47.401 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.546530+0000", 2024-09-21T21:40:47.401 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.6930912419999999, 2024-09-21T21:40:47.401 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0067606400000000001, 2024-09-21T21:40:47.401 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:47.402 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:47.402 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:47.402 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:47.402 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:47.402 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:47.402 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 195 2024-09-21T21:40:47.402 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.402 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:47.402 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.402 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:47.402 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.546530+0000", 2024-09-21T21:40:47.402 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:47.402 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.402 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.403 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:47.403 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.546530+0000", 2024-09-21T21:40:47.403 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:47.403 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.403 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.403 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:47.403 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.546531+0000", 2024-09-21T21:40:47.403 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1399999999999997e-07 2024-09-21T21:40:47.403 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.403 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.403 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:47.403 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.546534+0000", 2024-09-21T21:40:47.403 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.7989999999999998e-06 2024-09-21T21:40:47.403 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.404 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.404 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:47.404 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.546540+0000", 2024-09-21T21:40:47.404 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.314e-06 2024-09-21T21:40:47.404 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.404 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.404 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:47.404 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.546544+0000", 2024-09-21T21:40:47.404 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.546e-06 2024-09-21T21:40:47.404 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.404 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.404 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:47.404 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.547355+0000", 2024-09-21T21:40:47.404 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00081132899999999996 2024-09-21T21:40:47.404 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.405 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.405 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:47.405 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.547400+0000", 2024-09-21T21:40:47.405 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5627000000000002e-05 2024-09-21T21:40:47.405 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.405 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.405 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 4", 2024-09-21T21:40:47.405 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.547755+0000", 2024-09-21T21:40:47.405 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00035492899999999998 2024-09-21T21:40:47.405 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.405 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.405 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:47.405 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.549411+0000", 2024-09-21T21:40:47.406 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0016557239999999999 2024-09-21T21:40:47.406 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.406 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.406 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:40:47.406 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.553193+0000", 2024-09-21T21:40:47.406 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0037821370000000001 2024-09-21T21:40:47.406 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.406 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.406 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:47.406 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.553256+0000", 2024-09-21T21:40:47.406 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2534999999999997e-05 2024-09-21T21:40:47.406 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.406 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.406 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:47.406 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.553291+0000", 2024-09-21T21:40:47.407 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5185999999999998e-05 2024-09-21T21:40:47.407 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:47.407 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:47.407 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:47.407 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.407 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.407 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:236 3.b 3:d29f5c36:::rbd_data.11214be7c411.000000000000003b:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:47.407 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.556219+0000", 2024-09-21T21:40:47.407 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.6834029369999999, 2024-09-21T21:40:47.407 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0068065230000000001, 2024-09-21T21:40:47.407 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:47.407 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:47.407 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:40:47.407 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:47.408 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:47.408 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:47.408 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 236 2024-09-21T21:40:47.408 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.408 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:47.408 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.408 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:47.408 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556219+0000", 2024-09-21T21:40:47.408 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:47.408 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.408 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.408 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:47.408 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556219+0000", 2024-09-21T21:40:47.409 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:47.409 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.409 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.409 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:47.409 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556219+0000", 2024-09-21T21:40:47.409 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.9899999999999999e-07 2024-09-21T21:40:47.409 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.409 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.409 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:47.409 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556223+0000", 2024-09-21T21:40:47.409 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7910000000000002e-06 2024-09-21T21:40:47.409 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.409 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.409 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:47.409 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556235+0000", 2024-09-21T21:40:47.410 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1694e-05 2024-09-21T21:40:47.410 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.410 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.410 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:47.410 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556240+0000", 2024-09-21T21:40:47.410 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5199999999999997e-06 2024-09-21T21:40:47.410 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.410 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.410 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:47.410 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556300+0000", 2024-09-21T21:40:47.410 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9963000000000002e-05 2024-09-21T21:40:47.410 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.410 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.410 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:47.411 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556409+0000", 2024-09-21T21:40:47.411 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000108441 2024-09-21T21:40:47.411 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.411 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.411 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:47.411 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.563025+0000", 2024-09-21T21:40:47.411 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0066163150000000002 2024-09-21T21:40:47.411 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:47.411 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:47.411 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:47.411 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.411 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.411 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:252 3.0 3:01bf271b:::rbd_data.11214be7c411.0000000000000038:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:47.411 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.559092+0000", 2024-09-21T21:40:47.412 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.6805291750000002, 2024-09-21T21:40:47.412 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0066292950000000003, 2024-09-21T21:40:47.412 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:47.412 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:47.412 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:47.412 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:47.412 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:47.412 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:47.412 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 252 2024-09-21T21:40:47.412 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.412 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:47.412 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.412 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:47.413 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.559092+0000", 2024-09-21T21:40:47.413 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:47.413 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.413 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.413 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:47.413 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.559092+0000", 2024-09-21T21:40:47.413 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:47.413 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.413 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.413 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:47.413 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.559093+0000", 2024-09-21T21:40:47.413 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.1900000000000001e-07 2024-09-21T21:40:47.413 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.413 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.414 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:47.414 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.559098+0000", 2024-09-21T21:40:47.414 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6399999999999996e-06 2024-09-21T21:40:47.414 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.414 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.414 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:47.414 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.559108+0000", 2024-09-21T21:40:47.414 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6399999999999992e-06 2024-09-21T21:40:47.414 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.414 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.414 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:47.414 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.559121+0000", 2024-09-21T21:40:47.414 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.3687e-05 2024-09-21T21:40:47.414 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.415 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.415 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:47.415 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.559195+0000", 2024-09-21T21:40:47.415 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.3380999999999994e-05 2024-09-21T21:40:47.415 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.415 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.415 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:47.415 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.559273+0000", 2024-09-21T21:40:47.415 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.8236000000000001e-05 2024-09-21T21:40:47.415 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.415 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.415 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 2", 2024-09-21T21:40:47.415 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.559569+0000", 2024-09-21T21:40:47.415 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00029590600000000002 2024-09-21T21:40:47.415 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.416 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.416 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:40:47.416 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.563070+0000", 2024-09-21T21:40:47.416 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0035011880000000001 2024-09-21T21:40:47.416 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.416 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.416 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:47.416 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.565647+0000", 2024-09-21T21:40:47.416 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.002576877 2024-09-21T21:40:47.416 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.416 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.416 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:47.416 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.565692+0000", 2024-09-21T21:40:47.416 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.481e-05 2024-09-21T21:40:47.416 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.417 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.417 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:47.417 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.565722+0000", 2024-09-21T21:40:47.417 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0111e-05 2024-09-21T21:40:47.417 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:47.417 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:47.417 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:47.417 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.417 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.417 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:400 3.19 3:9c6b57eb:::rbd_object_map.11214be7c411.0000000000000004:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:47.417 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.586210+0000", 2024-09-21T21:40:47.417 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.6534116480000001, 2024-09-21T21:40:47.417 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0068241939999999996, 2024-09-21T21:40:47.417 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:47.418 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:47.418 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:47.418 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:47.418 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:47.418 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:47.418 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 400 2024-09-21T21:40:47.418 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.418 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:47.418 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.418 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:47.418 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.586210+0000", 2024-09-21T21:40:47.418 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:47.418 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.418 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.419 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:47.419 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.586210+0000", 2024-09-21T21:40:47.419 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:47.419 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.419 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.419 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:47.419 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.586211+0000", 2024-09-21T21:40:47.419 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.92e-07 2024-09-21T21:40:47.419 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.419 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.419 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:47.419 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.586215+0000", 2024-09-21T21:40:47.419 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2910000000000001e-06 2024-09-21T21:40:47.419 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.419 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.420 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:47.420 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.586223+0000", 2024-09-21T21:40:47.420 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.3319999999999992e-06 2024-09-21T21:40:47.420 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.420 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.420 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:47.420 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.586230+0000", 2024-09-21T21:40:47.420 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5649999999999998e-06 2024-09-21T21:40:47.420 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.420 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.420 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:47.420 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.587427+0000", 2024-09-21T21:40:47.420 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00119736 2024-09-21T21:40:47.420 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.421 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.421 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:47.421 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.587499+0000", 2024-09-21T21:40:47.421 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.1439000000000005e-05 2024-09-21T21:40:47.421 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.421 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.421 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 4", 2024-09-21T21:40:47.421 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.587824+0000", 2024-09-21T21:40:47.421 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00032502200000000001 2024-09-21T21:40:47.421 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.421 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.421 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:40:47.421 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.589401+0000", 2024-09-21T21:40:47.421 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0015776270000000001 2024-09-21T21:40:47.422 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.422 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.422 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:47.422 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.592976+0000", 2024-09-21T21:40:47.422 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.003574565 2024-09-21T21:40:47.422 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.422 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.422 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:47.422 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593004+0000", 2024-09-21T21:40:47.422 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.7937999999999999e-05 2024-09-21T21:40:47.422 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.422 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.422 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:47.422 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593034+0000", 2024-09-21T21:40:47.423 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0263000000000001e-05 2024-09-21T21:40:47.423 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:47.423 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:47.423 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:47.423 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.423 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.423 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:410 3.19 3:981b6e71:::rbd_data.11214be7c411.0000000000000065:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:47.423 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.587078+0000", 2024-09-21T21:40:47.423 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.6525439639999999, 2024-09-21T21:40:47.423 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0060338199999999996, 2024-09-21T21:40:47.423 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:47.423 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:47.423 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:40:47.423 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:47.424 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:47.424 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:47.424 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 410 2024-09-21T21:40:47.424 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.424 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:47.424 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.424 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:47.424 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.587078+0000", 2024-09-21T21:40:47.424 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:47.424 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.424 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.424 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:47.424 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.587078+0000", 2024-09-21T21:40:47.424 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:47.425 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.425 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.425 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:47.425 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.587078+0000", 2024-09-21T21:40:47.425 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6000000000000004e-07 2024-09-21T21:40:47.425 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.425 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.425 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:47.425 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.587081+0000", 2024-09-21T21:40:47.425 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.9950000000000001e-06 2024-09-21T21:40:47.425 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.425 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.425 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:47.425 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.587091+0000", 2024-09-21T21:40:47.426 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6299999999999993e-06 2024-09-21T21:40:47.426 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.426 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.426 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:47.426 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.587097+0000", 2024-09-21T21:40:47.426 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8259999999999998e-06 2024-09-21T21:40:47.426 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.426 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.426 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:47.426 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.589045+0000", 2024-09-21T21:40:47.426 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0019484179999999999 2024-09-21T21:40:47.426 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.426 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.426 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:47.427 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.589153+0000", 2024-09-21T21:40:47.427 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000107627 2024-09-21T21:40:47.427 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.427 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.427 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:47.427 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593111+0000", 2024-09-21T21:40:47.427 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.003958764 2024-09-21T21:40:47.427 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:47.427 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:47.427 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:47.427 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.427 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.427 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:459 3.4 3:22094545:::rbd_data.11214be7c411.000000000000005c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:47.427 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.593149+0000", 2024-09-21T21:40:47.428 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.646472717, 2024-09-21T21:40:47.428 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.010963733999999999, 2024-09-21T21:40:47.428 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:47.428 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:47.428 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:47.428 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:47.428 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:47.428 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:47.428 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 459 2024-09-21T21:40:47.428 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.428 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:47.428 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.428 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:47.428 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593149+0000", 2024-09-21T21:40:47.429 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:47.429 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.429 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.429 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:47.429 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593149+0000", 2024-09-21T21:40:47.429 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:47.429 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.429 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.429 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:47.429 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593150+0000", 2024-09-21T21:40:47.429 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4300000000000003e-07 2024-09-21T21:40:47.429 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.429 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.429 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:47.430 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593204+0000", 2024-09-21T21:40:47.430 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4868999999999999e-05 2024-09-21T21:40:47.430 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.430 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.430 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:47.430 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593213+0000", 2024-09-21T21:40:47.430 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.7479999999999999e-06 2024-09-21T21:40:47.430 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.430 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.430 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:47.430 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593217+0000", 2024-09-21T21:40:47.430 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.0409999999999999e-06 2024-09-21T21:40:47.430 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.431 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.431 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:47.431 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593249+0000", 2024-09-21T21:40:47.431 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.1897999999999999e-05 2024-09-21T21:40:47.431 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.431 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.431 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:47.431 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593312+0000", 2024-09-21T21:40:47.431 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2813999999999999e-05 2024-09-21T21:40:47.431 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.431 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.431 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 2", 2024-09-21T21:40:47.431 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593501+0000", 2024-09-21T21:40:47.431 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000188887 2024-09-21T21:40:47.432 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.432 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.432 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:40:47.432 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.594562+0000", 2024-09-21T21:40:47.432 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.001060975 2024-09-21T21:40:47.432 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.432 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.432 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:47.432 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.604018+0000", 2024-09-21T21:40:47.432 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0094560800000000004 2024-09-21T21:40:47.432 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.432 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.432 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:47.432 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.604074+0000", 2024-09-21T21:40:47.433 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6063999999999997e-05 2024-09-21T21:40:47.433 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.433 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.433 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:47.433 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.604113+0000", 2024-09-21T21:40:47.433 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8714999999999998e-05 2024-09-21T21:40:47.433 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:47.433 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:47.433 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:47.433 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.433 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.433 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:460 3.4 3:22094545:::rbd_data.11214be7c411.000000000000005c:head [delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:47.433 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.593227+0000", 2024-09-21T21:40:47.433 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.6463949690000002, 2024-09-21T21:40:47.434 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.01140237, 2024-09-21T21:40:47.434 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:47.434 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:47.434 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:47.434 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:47.434 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:47.434 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:47.434 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 460 2024-09-21T21:40:47.434 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.434 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:47.434 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.434 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:47.434 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593227+0000", 2024-09-21T21:40:47.434 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:47.435 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.435 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.435 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:47.435 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593227+0000", 2024-09-21T21:40:47.435 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:47.435 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.435 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.435 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:47.435 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593227+0000", 2024-09-21T21:40:47.435 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4000000000000002e-07 2024-09-21T21:40:47.435 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.435 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.435 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:47.435 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593229+0000", 2024-09-21T21:40:47.436 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1169999999999998e-06 2024-09-21T21:40:47.436 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.436 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.436 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:47.436 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593235+0000", 2024-09-21T21:40:47.436 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0510000000000002e-06 2024-09-21T21:40:47.436 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.436 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.436 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:47.436 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593248+0000", 2024-09-21T21:40:47.436 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.2403e-05 2024-09-21T21:40:47.436 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.436 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.436 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:47.436 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593895+0000", 2024-09-21T21:40:47.437 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000647358 2024-09-21T21:40:47.437 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.437 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.437 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:47.437 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593969+0000", 2024-09-21T21:40:47.437 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.4107e-05 2024-09-21T21:40:47.437 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.437 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.437 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 2", 2024-09-21T21:40:47.437 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.594139+0000", 2024-09-21T21:40:47.437 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000169457 2024-09-21T21:40:47.437 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.437 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.437 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:40:47.438 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.596005+0000", 2024-09-21T21:40:47.438 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00186608 2024-09-21T21:40:47.438 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.438 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.438 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:47.438 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.604537+0000", 2024-09-21T21:40:47.438 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.008531861 2024-09-21T21:40:47.438 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.438 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.438 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:47.438 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.604587+0000", 2024-09-21T21:40:47.438 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0544000000000003e-05 2024-09-21T21:40:47.438 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.439 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.439 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:47.439 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.604629+0000", 2024-09-21T21:40:47.439 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1851999999999998e-05 2024-09-21T21:40:47.439 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:47.439 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:47.439 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:47.439 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.439 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.439 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:463 3.19 3:981b6e71:::rbd_data.11214be7c411.0000000000000065:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:47.439 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.593716+0000", 2024-09-21T21:40:47.439 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.6459058880000002, 2024-09-21T21:40:47.439 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.006390326, 2024-09-21T21:40:47.439 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:47.439 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:47.440 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:47.440 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:47.440 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:47.440 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:47.440 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 463 2024-09-21T21:40:47.440 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.440 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:47.440 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.440 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:47.440 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593716+0000", 2024-09-21T21:40:47.440 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:47.440 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.440 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.440 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:47.441 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593716+0000", 2024-09-21T21:40:47.441 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:47.441 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.441 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.441 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:47.441 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593717+0000", 2024-09-21T21:40:47.441 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.09e-07 2024-09-21T21:40:47.441 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.441 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.441 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:47.441 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593729+0000", 2024-09-21T21:40:47.441 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.2211e-05 2024-09-21T21:40:47.441 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.441 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.442 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:47.442 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593739+0000", 2024-09-21T21:40:47.442 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.8470000000000008e-06 2024-09-21T21:40:47.442 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.442 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.442 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:47.442 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593745+0000", 2024-09-21T21:40:47.442 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5869999999999998e-06 2024-09-21T21:40:47.442 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.442 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.442 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:47.442 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.595440+0000", 2024-09-21T21:40:47.442 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.001694954 2024-09-21T21:40:47.442 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.442 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.443 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:47.443 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.595489+0000", 2024-09-21T21:40:47.443 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8782999999999997e-05 2024-09-21T21:40:47.443 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.443 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.443 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 4", 2024-09-21T21:40:47.443 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.595607+0000", 2024-09-21T21:40:47.443 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000117737 2024-09-21T21:40:47.443 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.443 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.443 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:40:47.443 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.596867+0000", 2024-09-21T21:40:47.443 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0012600210000000001 2024-09-21T21:40:47.443 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.444 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.444 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:47.444 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.600050+0000", 2024-09-21T21:40:47.444 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0031829979999999998 2024-09-21T21:40:47.444 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.444 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.444 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:47.444 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.600082+0000", 2024-09-21T21:40:47.444 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.2279e-05 2024-09-21T21:40:47.444 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.444 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.444 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:47.444 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.600106+0000", 2024-09-21T21:40:47.444 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.41e-05 2024-09-21T21:40:47.444 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:47.445 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:47.445 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:47.445 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.445 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.445 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:505 3.19 3:9c6b57eb:::rbd_object_map.11214be7c411.0000000000000004:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:47.445 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.605631+0000", 2024-09-21T21:40:47.445 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.6339904949999999, 2024-09-21T21:40:47.445 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0071505190000000001, 2024-09-21T21:40:47.445 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:47.445 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:47.445 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:47.445 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:47.445 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:47.445 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:47.445 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 505 2024-09-21T21:40:47.446 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.446 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:47.446 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.446 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:47.446 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605631+0000", 2024-09-21T21:40:47.446 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:47.446 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.446 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.446 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:47.446 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605631+0000", 2024-09-21T21:40:47.446 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:47.446 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.446 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.447 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:47.447 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605632+0000", 2024-09-21T21:40:47.447 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0900000000000001e-07 2024-09-21T21:40:47.447 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.447 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.447 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:47.447 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605636+0000", 2024-09-21T21:40:47.447 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8160000000000004e-06 2024-09-21T21:40:47.447 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.447 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.447 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:47.447 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605645+0000", 2024-09-21T21:40:47.447 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.0319999999999992e-06 2024-09-21T21:40:47.447 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.448 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.448 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:47.448 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605651+0000", 2024-09-21T21:40:47.448 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9529999999999996e-06 2024-09-21T21:40:47.448 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.448 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.448 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:47.448 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.606491+0000", 2024-09-21T21:40:47.448 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000840301 2024-09-21T21:40:47.448 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.448 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.448 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:47.448 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.606571+0000", 2024-09-21T21:40:47.448 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0507999999999996e-05 2024-09-21T21:40:47.448 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.449 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.449 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 4", 2024-09-21T21:40:47.449 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.606909+0000", 2024-09-21T21:40:47.449 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00033747300000000003 2024-09-21T21:40:47.449 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.449 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.449 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:47.449 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.608367+0000", 2024-09-21T21:40:47.449 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.001458188 2024-09-21T21:40:47.449 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.449 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.449 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:40:47.449 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.612683+0000", 2024-09-21T21:40:47.449 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0043163209999999997 2024-09-21T21:40:47.449 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.450 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.450 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:47.450 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.612732+0000", 2024-09-21T21:40:47.450 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8980999999999997e-05 2024-09-21T21:40:47.450 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.450 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.450 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:47.450 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.612782+0000", 2024-09-21T21:40:47.450 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9237000000000003e-05 2024-09-21T21:40:47.450 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:47.450 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:47.450 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:47.450 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.450 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.451 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_repop(client.4417.0:504 3.8 e26/21 3:10f57124:::rbd_object_map.11214be7c411:head v 26'13, mlcod=26'10)", 2024-09-21T21:40:47.451 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.607109+0000", 2024-09-21T21:40:47.451 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.632512282, 2024-09-21T21:40:47.451 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0085878610000000005, 2024-09-21T21:40:47.451 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:47.451 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:47.451 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:47.451 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:47.451 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.451 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:47.451 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607109+0000", 2024-09-21T21:40:47.451 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:47.451 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.451 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.451 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:47.452 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607109+0000", 2024-09-21T21:40:47.452 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:47.452 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.452 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.452 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:47.452 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607110+0000", 2024-09-21T21:40:47.452 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.82e-07 2024-09-21T21:40:47.452 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.452 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.452 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:47.452 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607115+0000", 2024-09-21T21:40:47.452 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1730000000000003e-06 2024-09-21T21:40:47.452 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.452 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.452 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:47.453 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607123+0000", 2024-09-21T21:40:47.453 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7610000000000007e-06 2024-09-21T21:40:47.453 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.453 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.453 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:47.453 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607128+0000", 2024-09-21T21:40:47.453 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6420000000000004e-06 2024-09-21T21:40:47.453 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.453 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.453 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:47.453 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607302+0000", 2024-09-21T21:40:47.453 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00017391600000000001 2024-09-21T21:40:47.453 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.453 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.454 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:47.454 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607320+0000", 2024-09-21T21:40:47.454 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.7404000000000001e-05 2024-09-21T21:40:47.454 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.454 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.454 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:47.454 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.615646+0000", 2024-09-21T21:40:47.454 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0083265130000000007 2024-09-21T21:40:47.454 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.454 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.454 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:47.454 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.615697+0000", 2024-09-21T21:40:47.454 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0869999999999999e-05 2024-09-21T21:40:47.454 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:47.455 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:47.455 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:47.455 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.455 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.455 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:513 3.19 3:9c6b57eb:::rbd_object_map.11214be7c411.0000000000000004:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:47.455 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.607160+0000", 2024-09-21T21:40:47.455 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.6324616980000002, 2024-09-21T21:40:47.455 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0086296150000000002, 2024-09-21T21:40:47.455 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:47.455 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:47.455 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:47.455 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:47.455 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:47.455 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:47.456 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 513 2024-09-21T21:40:47.456 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.456 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:47.456 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.456 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:47.456 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607160+0000", 2024-09-21T21:40:47.456 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:47.456 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.456 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.456 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:47.456 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607160+0000", 2024-09-21T21:40:47.456 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:47.456 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.456 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.456 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:47.457 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607161+0000", 2024-09-21T21:40:47.457 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.9399999999999993e-07 2024-09-21T21:40:47.457 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.457 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.457 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:47.457 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607165+0000", 2024-09-21T21:40:47.457 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3200000000000001e-06 2024-09-21T21:40:47.457 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.457 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.457 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:47.457 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607365+0000", 2024-09-21T21:40:47.457 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00019965799999999999 2024-09-21T21:40:47.457 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.457 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.458 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:47.458 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607374+0000", 2024-09-21T21:40:47.458 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.312e-06 2024-09-21T21:40:47.458 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.458 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.458 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:47.458 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607416+0000", 2024-09-21T21:40:47.458 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1962000000000002e-05 2024-09-21T21:40:47.458 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.458 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.458 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:47.458 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607485+0000", 2024-09-21T21:40:47.458 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9274000000000001e-05 2024-09-21T21:40:47.459 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.459 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.459 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 4", 2024-09-21T21:40:47.459 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607818+0000", 2024-09-21T21:40:47.459 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00033211299999999998 2024-09-21T21:40:47.459 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.459 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.459 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:47.459 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.608616+0000", 2024-09-21T21:40:47.459 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00079832700000000004 2024-09-21T21:40:47.459 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.459 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.459 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:40:47.459 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.615697+0000", 2024-09-21T21:40:47.459 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0070815849999999996 2024-09-21T21:40:47.460 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.460 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.460 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:47.460 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.615750+0000", 2024-09-21T21:40:47.460 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2778000000000002e-05 2024-09-21T21:40:47.460 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.460 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.460 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:47.460 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.615790+0000", 2024-09-21T21:40:47.460 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.9292000000000003e-05 2024-09-21T21:40:47.460 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:47.460 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:47.460 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:47.460 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.460 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.461 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_repop(client.4417.0:507 3.8 e26/21 3:10f57124:::rbd_object_map.11214be7c411:head v 26'14, mlcod=26'10)", 2024-09-21T21:40:47.461 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.607866+0000", 2024-09-21T21:40:47.461 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.6317559680000002, 2024-09-21T21:40:47.461 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0078638389999999992, 2024-09-21T21:40:47.461 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:47.461 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:47.461 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:47.461 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:47.461 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.461 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:47.461 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607866+0000", 2024-09-21T21:40:47.461 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:47.461 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.461 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.462 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:47.462 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607866+0000", 2024-09-21T21:40:47.462 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:47.462 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.462 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.462 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:47.462 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607866+0000", 2024-09-21T21:40:47.462 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6000000000000004e-07 2024-09-21T21:40:47.462 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.462 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.462 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:47.462 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607872+0000", 2024-09-21T21:40:47.462 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4820000000000002e-06 2024-09-21T21:40:47.462 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.463 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.463 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:47.463 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607882+0000", 2024-09-21T21:40:47.463 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0481e-05 2024-09-21T21:40:47.463 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.463 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.463 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:47.463 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607889+0000", 2024-09-21T21:40:47.463 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.3089999999999998e-06 2024-09-21T21:40:47.463 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.463 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.463 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:47.463 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607958+0000", 2024-09-21T21:40:47.463 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.8424999999999996e-05 2024-09-21T21:40:47.463 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.464 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.464 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:47.464 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607976+0000", 2024-09-21T21:40:47.464 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.7765e-05 2024-09-21T21:40:47.464 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.464 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.464 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:47.464 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.615700+0000", 2024-09-21T21:40:47.464 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0077241799999999998 2024-09-21T21:40:47.464 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.464 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.464 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:47.464 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.615729+0000", 2024-09-21T21:40:47.464 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.9637000000000001e-05 2024-09-21T21:40:47.465 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:47.465 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:47.465 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:47.465 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.465 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.465 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_repop(client.4417.0:508 3.8 e26/21 3:10f57124:::rbd_object_map.11214be7c411:head v 26'15, mlcod=26'10)", 2024-09-21T21:40:47.465 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.608535+0000", 2024-09-21T21:40:47.465 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.6310861459999999, 2024-09-21T21:40:47.465 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0072268009999999997, 2024-09-21T21:40:47.465 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:47.465 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:47.465 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:47.465 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:47.465 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.465 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:47.466 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.608535+0000", 2024-09-21T21:40:47.466 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:47.466 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.466 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.466 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:47.466 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.608535+0000", 2024-09-21T21:40:47.466 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:47.466 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.466 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.466 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:47.466 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.608536+0000", 2024-09-21T21:40:47.466 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2600000000000002e-07 2024-09-21T21:40:47.466 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.466 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.467 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:47.467 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.608540+0000", 2024-09-21T21:40:47.467 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4530000000000004e-06 2024-09-21T21:40:47.467 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.467 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.467 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:47.467 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.608549+0000", 2024-09-21T21:40:47.467 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.3270000000000001e-06 2024-09-21T21:40:47.467 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.467 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.467 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:47.467 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.608555+0000", 2024-09-21T21:40:47.467 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1009999999999997e-06 2024-09-21T21:40:47.467 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.468 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.468 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:47.468 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.608621+0000", 2024-09-21T21:40:47.468 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6351000000000002e-05 2024-09-21T21:40:47.468 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.468 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.468 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:47.468 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.608640+0000", 2024-09-21T21:40:47.468 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.8419000000000001e-05 2024-09-21T21:40:47.468 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.468 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.468 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:47.468 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.615731+0000", 2024-09-21T21:40:47.469 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.007091226 2024-09-21T21:40:47.469 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.469 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.469 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:47.469 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.615762+0000", 2024-09-21T21:40:47.469 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.1398e-05 2024-09-21T21:40:47.469 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:47.469 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:47.469 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:47.469 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.469 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.469 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_repop(client.4417.0:517 3.1a e26/21 3:5aa0c892:::rbd_data.11214be7c411.000000000000007a:head v 26'10, mlcod=26'8)", 2024-09-21T21:40:47.469 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.608704+0000", 2024-09-21T21:40:47.469 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.6309180210000003, 2024-09-21T21:40:47.470 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0071660500000000002, 2024-09-21T21:40:47.470 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:47.470 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:47.470 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:47.470 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:47.470 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.470 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:47.470 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.608704+0000", 2024-09-21T21:40:47.470 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:47.470 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.470 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.470 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:47.470 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.608704+0000", 2024-09-21T21:40:47.471 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:47.471 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.471 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.471 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:47.471 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.608704+0000", 2024-09-21T21:40:47.471 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2099999999999997e-07 2024-09-21T21:40:47.471 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.471 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.471 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:47.471 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.608709+0000", 2024-09-21T21:40:47.471 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8489999999999997e-06 2024-09-21T21:40:47.471 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.471 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.471 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:47.472 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.608719+0000", 2024-09-21T21:40:47.472 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0402e-05 2024-09-21T21:40:47.472 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.472 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.472 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:47.472 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.608726+0000", 2024-09-21T21:40:47.472 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.72e-06 2024-09-21T21:40:47.472 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.472 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.472 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:47.472 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.608766+0000", 2024-09-21T21:40:47.472 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.9745e-05 2024-09-21T21:40:47.472 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.472 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.472 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:47.473 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.608785+0000", 2024-09-21T21:40:47.473 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.8919e-05 2024-09-21T21:40:47.473 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.473 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.473 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:47.473 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.615831+0000", 2024-09-21T21:40:47.473 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0070464969999999997 2024-09-21T21:40:47.473 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.473 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.473 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:47.473 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.615870+0000", 2024-09-21T21:40:47.473 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8396999999999998e-05 2024-09-21T21:40:47.473 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:47.473 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:47.473 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:47.474 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.474 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.474 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_repop(client.4417.0:509 3.8 e26/21 3:10f57124:::rbd_object_map.11214be7c411:head v 26'16, mlcod=26'10)", 2024-09-21T21:40:47.474 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.609103+0000", 2024-09-21T21:40:47.474 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.6305186190000001, 2024-09-21T21:40:47.474 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0066864489999999997, 2024-09-21T21:40:47.474 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:47.474 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:47.474 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:47.474 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:47.474 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.474 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:47.474 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.609103+0000", 2024-09-21T21:40:47.474 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:47.475 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.475 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.475 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:47.475 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.609103+0000", 2024-09-21T21:40:47.475 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:47.475 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.475 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.475 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:47.475 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.609103+0000", 2024-09-21T21:40:47.475 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5999999999999999e-07 2024-09-21T21:40:47.475 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.475 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.475 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:47.475 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.609108+0000", 2024-09-21T21:40:47.476 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0100000000000003e-06 2024-09-21T21:40:47.476 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.476 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.476 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:47.476 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.609115+0000", 2024-09-21T21:40:47.476 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2319999999999998e-06 2024-09-21T21:40:47.476 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.476 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.476 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:47.476 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.609120+0000", 2024-09-21T21:40:47.476 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4199999999999998e-06 2024-09-21T21:40:47.476 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.476 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.476 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:47.476 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.609156+0000", 2024-09-21T21:40:47.476 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6269999999999997e-05 2024-09-21T21:40:47.477 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.477 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.477 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:47.477 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.609176+0000", 2024-09-21T21:40:47.477 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9666000000000001e-05 2024-09-21T21:40:47.477 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.477 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.477 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:47.477 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.615764+0000", 2024-09-21T21:40:47.477 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0065882989999999997 2024-09-21T21:40:47.477 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.477 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.477 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:47.477 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.615789+0000", 2024-09-21T21:40:47.478 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.5092e-05 2024-09-21T21:40:47.478 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:47.478 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:47.478 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:47.478 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.478 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.478 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_repop(client.4417.0:510 3.8 e26/21 3:10f57124:::rbd_object_map.11214be7c411:head v 26'17, mlcod=26'10)", 2024-09-21T21:40:47.478 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.609673+0000", 2024-09-21T21:40:47.478 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.6299485069999999, 2024-09-21T21:40:47.478 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0063224570000000001, 2024-09-21T21:40:47.478 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:47.478 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:47.478 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:47.478 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:47.479 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.479 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:47.479 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.609673+0000", 2024-09-21T21:40:47.479 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:47.479 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.479 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.479 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:47.479 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.609673+0000", 2024-09-21T21:40:47.479 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:47.479 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.479 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.479 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:47.479 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.609674+0000", 2024-09-21T21:40:47.479 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4799999999999999e-07 2024-09-21T21:40:47.479 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.480 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.480 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:47.480 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.609677+0000", 2024-09-21T21:40:47.480 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8419999999999998e-06 2024-09-21T21:40:47.480 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.480 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.480 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:47.480 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.609684+0000", 2024-09-21T21:40:47.480 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6100000000000002e-06 2024-09-21T21:40:47.480 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.480 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.480 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:47.480 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.609689+0000", 2024-09-21T21:40:47.480 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8450000000000002e-06 2024-09-21T21:40:47.481 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.481 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.481 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:47.481 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.609733+0000", 2024-09-21T21:40:47.481 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3881999999999998e-05 2024-09-21T21:40:47.481 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.481 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.481 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:47.481 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.609752+0000", 2024-09-21T21:40:47.481 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9743000000000001e-05 2024-09-21T21:40:47.481 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.481 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.481 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:47.481 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.615791+0000", 2024-09-21T21:40:47.481 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0060380570000000003 2024-09-21T21:40:47.482 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.482 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.482 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:47.482 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.615996+0000", 2024-09-21T21:40:47.482 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00020503000000000001 2024-09-21T21:40:47.482 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:47.482 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:47.482 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:47.482 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:47.482 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:47.482 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:40:47.483 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T21:40:47.483 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3: {} 2024-09-21T21:40:47.484 INFO:teuthology.orchestra.run.smithi081.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T21:40:47.484 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4: {} 2024-09-21T21:40:47.484 INFO:teuthology.orchestra.run.smithi081.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T21:40:47.484 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5: {} 2024-09-21T21:40:47.485 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T21:40:47.486 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:40:47.379+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:40:47.486 INFO:teuthology.orchestra.run.smithi081.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T21:40:47.487 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:40:47.485+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:40:47.578 INFO:teuthology.orchestra.run.smithi164.stdout:checking clone #2, image image_client.0-clone2 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent3 2024-09-21T21:40:47.586 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:40:47.585+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:40:47.636 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:40:47.636 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:40:47.636 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 0 2024-09-21T21:40:47.636 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:40:47.646 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T21:40:47.688 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:40:47.686+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:40:47.765 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:40:47.766 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:40:47.766 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:40:47.766 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:40:47.766 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:40:47.775 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T21:40:47.818 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:40:47.817+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:40:47.889 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:40:47.889 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 20, 2024-09-21T21:40:47.889 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 600, 2024-09-21T21:40:47.890 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:40:47.890 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.890 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:122 3.8 3:108e7c9d:::rbd_data.11214be7c411.000000000000001f:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:47.890 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.534714+0000", 2024-09-21T21:40:47.890 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 4.3522593690000004, 2024-09-21T21:40:47.890 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0076042100000000001, 2024-09-21T21:40:47.890 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:47.890 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:47.890 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:40:47.890 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:47.890 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:47.890 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:47.890 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 122 2024-09-21T21:40:47.890 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.891 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:47.891 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.891 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:47.891 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.534714+0000", 2024-09-21T21:40:47.891 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:47.891 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.891 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.891 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:47.891 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.534714+0000", 2024-09-21T21:40:47.891 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:47.891 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.891 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.891 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:47.891 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.534716+0000", 2024-09-21T21:40:47.892 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.624e-06 2024-09-21T21:40:47.892 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.892 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.892 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:47.892 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.534721+0000", 2024-09-21T21:40:47.892 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9069999999999998e-06 2024-09-21T21:40:47.892 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.892 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.892 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:47.892 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.534733+0000", 2024-09-21T21:40:47.892 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.2089000000000001e-05 2024-09-21T21:40:47.892 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.892 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.892 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:47.892 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.534739+0000", 2024-09-21T21:40:47.893 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4879999999999998e-06 2024-09-21T21:40:47.893 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.893 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.893 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:47.893 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.534788+0000", 2024-09-21T21:40:47.893 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9629000000000002e-05 2024-09-21T21:40:47.893 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.893 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.893 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:47.893 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.534920+0000", 2024-09-21T21:40:47.893 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00013217599999999999 2024-09-21T21:40:47.893 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.893 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.893 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:47.894 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.542319+0000", 2024-09-21T21:40:47.894 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0073982969999999999 2024-09-21T21:40:47.894 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:47.894 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:47.894 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:47.894 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.894 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.894 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:154 3.8 3:15b4ee09:::rbd_data.11214be7c411.0000000000000028:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:47.894 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.538274+0000", 2024-09-21T21:40:47.894 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 4.3487001799999998, 2024-09-21T21:40:47.894 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0067541629999999997, 2024-09-21T21:40:47.894 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:47.894 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:47.895 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:40:47.895 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:47.895 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:47.895 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:47.895 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 154 2024-09-21T21:40:47.895 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.895 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:47.895 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.895 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:47.895 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.538274+0000", 2024-09-21T21:40:47.895 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:47.895 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.895 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.895 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:47.896 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.538274+0000", 2024-09-21T21:40:47.896 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:47.896 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.896 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.896 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:47.896 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.538274+0000", 2024-09-21T21:40:47.896 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.6700000000000002e-07 2024-09-21T21:40:47.896 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.896 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.896 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:47.896 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.538278+0000", 2024-09-21T21:40:47.896 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8940000000000003e-06 2024-09-21T21:40:47.896 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.896 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.897 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:47.897 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.538288+0000", 2024-09-21T21:40:47.897 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6809999999999993e-06 2024-09-21T21:40:47.897 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.897 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.897 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:47.897 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.538293+0000", 2024-09-21T21:40:47.897 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1030000000000001e-06 2024-09-21T21:40:47.897 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.897 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.897 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:47.897 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.538331+0000", 2024-09-21T21:40:47.897 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7870999999999997e-05 2024-09-21T21:40:47.897 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.897 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.898 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:47.898 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.538408+0000", 2024-09-21T21:40:47.898 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7280000000000005e-05 2024-09-21T21:40:47.898 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.898 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.898 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:47.898 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.545028+0000", 2024-09-21T21:40:47.898 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0066194670000000004 2024-09-21T21:40:47.898 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:47.898 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:47.898 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:47.898 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.898 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.899 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:235 3.8 3:1566a55e:::rbd_data.11214be7c411.000000000000003a:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:47.899 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.556351+0000", 2024-09-21T21:40:47.899 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 4.3306226329999999, 2024-09-21T21:40:47.899 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0066768460000000002, 2024-09-21T21:40:47.899 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:47.899 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:47.899 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:40:47.899 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:47.899 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:47.899 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:47.899 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 235 2024-09-21T21:40:47.899 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.899 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:47.899 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.899 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:47.900 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556351+0000", 2024-09-21T21:40:47.900 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:47.900 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.900 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.900 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:47.900 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556351+0000", 2024-09-21T21:40:47.900 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:47.900 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.900 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.900 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:47.900 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556352+0000", 2024-09-21T21:40:47.900 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.034e-06 2024-09-21T21:40:47.900 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.900 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.901 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:47.901 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556358+0000", 2024-09-21T21:40:47.901 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7200000000000003e-06 2024-09-21T21:40:47.901 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.901 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.901 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:47.901 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556373+0000", 2024-09-21T21:40:47.901 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.4602e-05 2024-09-21T21:40:47.901 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.901 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.901 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:47.901 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556391+0000", 2024-09-21T21:40:47.901 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.8726e-05 2024-09-21T21:40:47.901 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.902 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.902 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:47.902 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556445+0000", 2024-09-21T21:40:47.902 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3687000000000003e-05 2024-09-21T21:40:47.902 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.902 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.902 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:47.902 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556587+0000", 2024-09-21T21:40:47.902 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00014181299999999999 2024-09-21T21:40:47.902 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.902 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.902 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:47.902 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.563028+0000", 2024-09-21T21:40:47.902 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0064412640000000004 2024-09-21T21:40:47.903 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:47.903 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:47.903 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:47.903 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.903 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.903 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:244 3.1e 3:7844c79d:::rbd_data.11214be7c411.0000000000000037:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:47.903 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.556699+0000", 2024-09-21T21:40:47.903 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 4.3302752260000004, 2024-09-21T21:40:47.903 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0063205409999999998, 2024-09-21T21:40:47.903 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:47.903 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:47.903 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:47.903 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:47.903 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:47.904 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:47.904 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 244 2024-09-21T21:40:47.904 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.904 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:47.904 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.904 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:47.904 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556699+0000", 2024-09-21T21:40:47.904 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:47.904 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.904 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.904 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:47.904 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556699+0000", 2024-09-21T21:40:47.904 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:47.904 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.905 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.905 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:47.905 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556699+0000", 2024-09-21T21:40:47.905 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.8899999999999998e-07 2024-09-21T21:40:47.905 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.905 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.905 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:47.905 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556704+0000", 2024-09-21T21:40:47.905 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8289999999999998e-06 2024-09-21T21:40:47.905 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.905 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.905 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:47.905 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556716+0000", 2024-09-21T21:40:47.906 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.2248e-05 2024-09-21T21:40:47.906 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.906 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.906 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:47.906 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556723+0000", 2024-09-21T21:40:47.906 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9460000000000001e-06 2024-09-21T21:40:47.906 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.906 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.906 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:47.906 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556776+0000", 2024-09-21T21:40:47.906 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2154000000000002e-05 2024-09-21T21:40:47.906 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.906 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.906 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:47.907 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556854+0000", 2024-09-21T21:40:47.907 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.8889999999999999e-05 2024-09-21T21:40:47.907 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.907 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.907 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 3", 2024-09-21T21:40:47.907 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.557063+0000", 2024-09-21T21:40:47.907 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000208314 2024-09-21T21:40:47.907 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.907 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.907 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:47.907 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.557979+0000", 2024-09-21T21:40:47.907 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00091603599999999996 2024-09-21T21:40:47.907 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.907 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.907 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:40:47.908 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.562892+0000", 2024-09-21T21:40:47.908 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0049135070000000001 2024-09-21T21:40:47.908 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.908 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.908 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:47.908 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.562963+0000", 2024-09-21T21:40:47.908 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0976999999999995e-05 2024-09-21T21:40:47.908 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.908 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.908 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:47.908 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.563019+0000", 2024-09-21T21:40:47.908 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5850999999999998e-05 2024-09-21T21:40:47.908 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:47.908 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:47.908 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:47.909 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.909 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.909 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:246 3.18 3:18a859ac:::rbd_data.11214be7c411.0000000000000036:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:47.909 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.556821+0000", 2024-09-21T21:40:47.909 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 4.3301525969999997, 2024-09-21T21:40:47.909 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0064722360000000001, 2024-09-21T21:40:47.909 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:47.909 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:47.909 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:47.909 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:47.909 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:47.909 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:47.909 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 246 2024-09-21T21:40:47.909 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.910 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:47.910 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.910 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:47.910 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556821+0000", 2024-09-21T21:40:47.910 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:47.910 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.910 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.910 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:47.910 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556821+0000", 2024-09-21T21:40:47.910 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:47.910 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.910 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.910 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:47.910 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556822+0000", 2024-09-21T21:40:47.911 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.4499999999999996e-07 2024-09-21T21:40:47.911 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.911 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.911 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:47.911 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556827+0000", 2024-09-21T21:40:47.911 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6419999999999998e-06 2024-09-21T21:40:47.911 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.911 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.911 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:47.911 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556836+0000", 2024-09-21T21:40:47.911 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2380000000000003e-06 2024-09-21T21:40:47.911 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.911 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.911 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:47.912 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556843+0000", 2024-09-21T21:40:47.912 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6930000000000001e-06 2024-09-21T21:40:47.912 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.912 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.912 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:47.912 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556881+0000", 2024-09-21T21:40:47.912 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.807e-05 2024-09-21T21:40:47.912 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.912 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.912 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:47.912 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556955+0000", 2024-09-21T21:40:47.912 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.4059000000000004e-05 2024-09-21T21:40:47.912 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.912 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.913 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 0", 2024-09-21T21:40:47.913 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.557134+0000", 2024-09-21T21:40:47.913 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000179154 2024-09-21T21:40:47.913 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.913 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.913 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:47.913 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.563029+0000", 2024-09-21T21:40:47.913 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0058954419999999999 2024-09-21T21:40:47.913 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.913 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.913 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:40:47.913 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.563188+0000", 2024-09-21T21:40:47.913 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00015865200000000001 2024-09-21T21:40:47.914 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.914 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.914 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:47.914 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.563251+0000", 2024-09-21T21:40:47.914 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3120999999999994e-05 2024-09-21T21:40:47.914 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.914 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.914 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:47.914 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.563293+0000", 2024-09-21T21:40:47.914 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2320000000000001e-05 2024-09-21T21:40:47.914 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:47.914 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:47.914 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:47.914 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.914 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.915 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_repop(client.4417.0:440 3.5 e26/21 3:a6f99a77:::rbd_data.11214be7c411.0000000000000068:head v 26'10, mlcod=26'8)", 2024-09-21T21:40:47.915 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.590756+0000", 2024-09-21T21:40:47.915 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 4.2962176779999997, 2024-09-21T21:40:47.915 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0079145159999999999, 2024-09-21T21:40:47.915 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:47.915 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:47.915 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:47.915 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:47.915 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.915 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:47.915 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.590756+0000", 2024-09-21T21:40:47.915 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:47.915 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.915 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.916 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:47.916 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.590756+0000", 2024-09-21T21:40:47.916 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:47.916 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.916 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.916 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:47.916 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.590757+0000", 2024-09-21T21:40:47.916 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9500000000000002e-07 2024-09-21T21:40:47.916 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.916 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.916 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:47.916 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.590762+0000", 2024-09-21T21:40:47.916 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.045e-06 2024-09-21T21:40:47.916 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.917 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.917 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:47.917 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.590772+0000", 2024-09-21T21:40:47.917 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0711e-05 2024-09-21T21:40:47.917 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.917 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.917 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:47.917 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.590779+0000", 2024-09-21T21:40:47.917 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2210000000000002e-06 2024-09-21T21:40:47.917 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.917 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.917 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:47.917 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.590827+0000", 2024-09-21T21:40:47.917 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8127999999999997e-05 2024-09-21T21:40:47.917 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.918 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.918 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:47.918 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.590846+0000", 2024-09-21T21:40:47.918 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.965e-05 2024-09-21T21:40:47.918 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.918 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.918 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:47.918 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.598637+0000", 2024-09-21T21:40:47.918 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0077903240000000004 2024-09-21T21:40:47.918 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.918 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.918 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:47.918 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.598671+0000", 2024-09-21T21:40:47.918 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3841999999999999e-05 2024-09-21T21:40:47.919 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:47.919 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:47.919 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:47.919 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.919 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.919 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_repop(client.4417.0:459 3.4 e26/21 3:22094545:::rbd_data.11214be7c411.000000000000005c:head v 26'4, mlcod=26'2)", 2024-09-21T21:40:47.919 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.593580+0000", 2024-09-21T21:40:47.919 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 4.2933941960000004, 2024-09-21T21:40:47.919 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.01029448, 2024-09-21T21:40:47.919 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:47.919 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:47.919 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:47.919 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:47.919 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.920 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:47.920 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593580+0000", 2024-09-21T21:40:47.920 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:47.920 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.920 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.920 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:47.920 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593580+0000", 2024-09-21T21:40:47.920 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:47.920 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.920 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.920 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:47.920 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593580+0000", 2024-09-21T21:40:47.920 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0299999999999998e-07 2024-09-21T21:40:47.920 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.920 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.921 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:47.921 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593608+0000", 2024-09-21T21:40:47.921 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.7183999999999999e-05 2024-09-21T21:40:47.921 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.921 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.921 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:47.921 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593617+0000", 2024-09-21T21:40:47.921 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.1819999999999995e-06 2024-09-21T21:40:47.921 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.921 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.921 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:47.921 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593623+0000", 2024-09-21T21:40:47.921 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.443e-06 2024-09-21T21:40:47.921 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.922 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.922 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:47.922 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593671+0000", 2024-09-21T21:40:47.922 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8213000000000001e-05 2024-09-21T21:40:47.922 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.922 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.922 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:47.922 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593691+0000", 2024-09-21T21:40:47.922 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9324999999999999e-05 2024-09-21T21:40:47.922 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.922 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.922 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:47.922 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.603832+0000", 2024-09-21T21:40:47.922 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.010141752 2024-09-21T21:40:47.922 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.923 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.923 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:47.923 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.603874+0000", 2024-09-21T21:40:47.923 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1678000000000003e-05 2024-09-21T21:40:47.923 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:47.923 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:47.923 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:47.923 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.923 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.923 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_repop(client.4417.0:460 3.4 e26/21 3:22094545:::rbd_data.11214be7c411.000000000000005c:head v 26'6, mlcod=26'2)", 2024-09-21T21:40:47.923 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.594203+0000", 2024-09-21T21:40:47.923 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 4.29277088, 2024-09-21T21:40:47.923 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.010212592, 2024-09-21T21:40:47.923 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:47.924 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:47.924 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:47.924 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:47.924 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.924 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:47.924 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.594203+0000", 2024-09-21T21:40:47.924 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:47.924 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.924 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.924 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:47.924 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.594203+0000", 2024-09-21T21:40:47.924 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:47.924 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.924 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.924 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:47.925 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.594204+0000", 2024-09-21T21:40:47.925 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5199999999999996e-07 2024-09-21T21:40:47.925 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.925 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.925 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:47.925 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.594210+0000", 2024-09-21T21:40:47.925 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.798e-06 2024-09-21T21:40:47.925 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.925 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.925 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:47.925 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.594222+0000", 2024-09-21T21:40:47.925 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1391e-05 2024-09-21T21:40:47.925 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.925 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.926 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:47.926 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.594229+0000", 2024-09-21T21:40:47.926 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.7490000000000001e-06 2024-09-21T21:40:47.926 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.926 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.926 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:47.926 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.594285+0000", 2024-09-21T21:40:47.926 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6100999999999997e-05 2024-09-21T21:40:47.926 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.926 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.926 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:47.926 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.594303+0000", 2024-09-21T21:40:47.926 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.7938999999999999e-05 2024-09-21T21:40:47.926 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.926 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.927 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:47.927 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.604374+0000", 2024-09-21T21:40:47.927 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.010071762999999999 2024-09-21T21:40:47.927 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.927 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.927 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:47.927 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.604416+0000", 2024-09-21T21:40:47.927 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1099000000000001e-05 2024-09-21T21:40:47.927 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:47.927 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:47.927 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:47.927 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.927 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.927 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_repop(client.4417.0:496 3.d e26/21 3:b065aa9d:::rbd_data.11214be7c411.0000000000000074:head v 26'21, mlcod=26'19)", 2024-09-21T21:40:47.928 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.605521+0000", 2024-09-21T21:40:47.928 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 4.281453087, 2024-09-21T21:40:47.928 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0071066059999999997, 2024-09-21T21:40:47.928 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:47.928 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:47.928 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:47.928 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:47.928 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.928 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:47.928 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605521+0000", 2024-09-21T21:40:47.928 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:47.928 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.928 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.928 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:47.928 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605521+0000", 2024-09-21T21:40:47.929 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:47.929 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.929 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.929 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:47.929 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605521+0000", 2024-09-21T21:40:47.929 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5799999999999999e-07 2024-09-21T21:40:47.929 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.929 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.929 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:47.929 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605526+0000", 2024-09-21T21:40:47.929 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.279e-06 2024-09-21T21:40:47.929 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.929 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.929 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:47.930 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605535+0000", 2024-09-21T21:40:47.930 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.594e-06 2024-09-21T21:40:47.930 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.930 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.930 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:47.930 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605540+0000", 2024-09-21T21:40:47.930 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4510000000000002e-06 2024-09-21T21:40:47.930 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.930 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.930 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:47.930 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605582+0000", 2024-09-21T21:40:47.930 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2778999999999998e-05 2024-09-21T21:40:47.930 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.930 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.931 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:47.931 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605600+0000", 2024-09-21T21:40:47.931 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.7795000000000002e-05 2024-09-21T21:40:47.931 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.931 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.931 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:47.931 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.612594+0000", 2024-09-21T21:40:47.931 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0069935509999999998 2024-09-21T21:40:47.931 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.931 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.931 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:47.931 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.612627+0000", 2024-09-21T21:40:47.931 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3599000000000001e-05 2024-09-21T21:40:47.931 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:47.932 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:47.932 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:47.932 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.932 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.932 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:503 3.8 3:10f57124:::rbd_object_map.11214be7c411:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:47.932 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.605576+0000", 2024-09-21T21:40:47.932 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 4.2813976790000003, 2024-09-21T21:40:47.932 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0071414349999999998, 2024-09-21T21:40:47.932 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:47.932 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:47.932 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:47.932 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:47.932 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:47.932 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:47.932 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 503 2024-09-21T21:40:47.933 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.933 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:47.933 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.933 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:47.933 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605576+0000", 2024-09-21T21:40:47.933 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:47.933 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.933 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.933 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:47.933 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605576+0000", 2024-09-21T21:40:47.933 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:47.933 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.933 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.933 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:47.934 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605577+0000", 2024-09-21T21:40:47.934 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1099999999999995e-07 2024-09-21T21:40:47.934 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.934 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.934 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:47.934 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605581+0000", 2024-09-21T21:40:47.934 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1489999999999996e-06 2024-09-21T21:40:47.934 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.934 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.934 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:47.934 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605587+0000", 2024-09-21T21:40:47.934 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9719999999999997e-06 2024-09-21T21:40:47.934 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.934 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.935 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:47.935 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605591+0000", 2024-09-21T21:40:47.935 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.9759999999999997e-06 2024-09-21T21:40:47.935 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.935 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.935 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:47.935 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605647+0000", 2024-09-21T21:40:47.935 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6162999999999997e-05 2024-09-21T21:40:47.935 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.935 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.935 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:47.935 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605744+0000", 2024-09-21T21:40:47.935 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6916000000000005e-05 2024-09-21T21:40:47.935 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.935 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.936 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 1", 2024-09-21T21:40:47.936 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.606412+0000", 2024-09-21T21:40:47.936 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00066832800000000004 2024-09-21T21:40:47.936 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.936 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.936 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:47.936 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.608775+0000", 2024-09-21T21:40:47.936 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0023627790000000002 2024-09-21T21:40:47.936 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.936 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.936 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:40:47.936 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.612630+0000", 2024-09-21T21:40:47.936 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0038553110000000002 2024-09-21T21:40:47.936 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.937 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.937 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:47.937 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.612671+0000", 2024-09-21T21:40:47.937 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.0451999999999998e-05 2024-09-21T21:40:47.937 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.937 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.937 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:47.937 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.612718+0000", 2024-09-21T21:40:47.937 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6777999999999998e-05 2024-09-21T21:40:47.937 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:47.937 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:47.937 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:47.937 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.937 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.938 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:504 3.8 3:10f57124:::rbd_object_map.11214be7c411:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:47.938 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.605604+0000", 2024-09-21T21:40:47.938 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 4.28136954, 2024-09-21T21:40:47.938 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.010335359000000001, 2024-09-21T21:40:47.938 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:47.938 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:47.938 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:47.938 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:47.938 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:47.938 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:47.938 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 504 2024-09-21T21:40:47.938 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.938 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:47.938 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.939 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:47.939 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605604+0000", 2024-09-21T21:40:47.939 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:47.939 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.939 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.939 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:47.939 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605604+0000", 2024-09-21T21:40:47.939 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:47.939 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.939 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.939 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:47.939 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605605+0000", 2024-09-21T21:40:47.939 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.0200000000000003e-07 2024-09-21T21:40:47.939 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.940 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.940 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:47.940 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605607+0000", 2024-09-21T21:40:47.940 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.6029999999999999e-06 2024-09-21T21:40:47.940 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.940 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.940 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:47.940 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605613+0000", 2024-09-21T21:40:47.940 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4870000000000002e-06 2024-09-21T21:40:47.940 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.940 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.940 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:47.940 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605616+0000", 2024-09-21T21:40:47.940 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4319999999999999e-06 2024-09-21T21:40:47.941 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.941 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.941 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:47.941 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.606749+0000", 2024-09-21T21:40:47.941 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.001132587 2024-09-21T21:40:47.941 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.941 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.941 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:47.941 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.606807+0000", 2024-09-21T21:40:47.941 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8356999999999997e-05 2024-09-21T21:40:47.941 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.941 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.941 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 1", 2024-09-21T21:40:47.941 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607016+0000", 2024-09-21T21:40:47.941 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00020888600000000001 2024-09-21T21:40:47.941 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.942 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.942 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:40:47.942 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.615545+0000", 2024-09-21T21:40:47.942 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0085290829999999998 2024-09-21T21:40:47.942 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.942 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.942 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:47.942 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.615842+0000", 2024-09-21T21:40:47.942 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000296951 2024-09-21T21:40:47.942 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.942 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.942 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:47.942 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.615893+0000", 2024-09-21T21:40:47.942 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0519000000000002e-05 2024-09-21T21:40:47.943 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.943 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.943 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:47.943 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.615940+0000", 2024-09-21T21:40:47.943 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7052000000000003e-05 2024-09-21T21:40:47.943 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:47.943 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:47.943 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:47.943 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.943 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.943 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:507 3.8 3:10f57124:::rbd_object_map.11214be7c411:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:47.943 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.605687+0000", 2024-09-21T21:40:47.943 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 4.2812865980000003, 2024-09-21T21:40:47.943 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.010435425, 2024-09-21T21:40:47.944 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:47.944 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:47.944 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:47.944 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:47.944 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:47.944 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:47.944 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 507 2024-09-21T21:40:47.944 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.944 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:47.944 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.944 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:47.944 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605687+0000", 2024-09-21T21:40:47.944 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:47.944 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.944 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.945 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:47.945 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605687+0000", 2024-09-21T21:40:47.945 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:47.945 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.945 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.945 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:47.945 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605688+0000", 2024-09-21T21:40:47.945 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.006e-06 2024-09-21T21:40:47.945 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.945 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.945 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:47.945 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605694+0000", 2024-09-21T21:40:47.945 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6099999999999997e-06 2024-09-21T21:40:47.945 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.946 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.946 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:47.946 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605705+0000", 2024-09-21T21:40:47.946 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.164e-05 2024-09-21T21:40:47.946 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.946 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.946 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:47.946 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605714+0000", 2024-09-21T21:40:47.946 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.4610000000000008e-06 2024-09-21T21:40:47.946 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.946 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.946 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:47.946 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607537+0000", 2024-09-21T21:40:47.946 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0018232490000000001 2024-09-21T21:40:47.947 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.947 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.947 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:47.947 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607585+0000", 2024-09-21T21:40:47.947 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7491999999999999e-05 2024-09-21T21:40:47.947 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.947 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.947 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 1", 2024-09-21T21:40:47.947 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607814+0000", 2024-09-21T21:40:47.947 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000229223 2024-09-21T21:40:47.947 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.947 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.947 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:40:47.947 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.615553+0000", 2024-09-21T21:40:47.948 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0077389700000000004 2024-09-21T21:40:47.948 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.948 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.948 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:47.948 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.615999+0000", 2024-09-21T21:40:47.948 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000446107 2024-09-21T21:40:47.948 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.948 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.948 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:47.948 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.616081+0000", 2024-09-21T21:40:47.948 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.1816000000000004e-05 2024-09-21T21:40:47.948 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.948 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.948 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:47.948 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.616123+0000", 2024-09-21T21:40:47.949 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1851000000000003e-05 2024-09-21T21:40:47.949 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:47.949 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:47.949 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:47.949 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.949 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.949 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:508 3.8 3:10f57124:::rbd_object_map.11214be7c411:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:47.949 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.605730+0000", 2024-09-21T21:40:47.949 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 4.2812440399999998, 2024-09-21T21:40:47.949 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.010553755, 2024-09-21T21:40:47.949 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:47.949 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:47.949 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:47.949 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:47.950 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:47.950 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:47.950 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 508 2024-09-21T21:40:47.950 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.950 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:47.950 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.950 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:47.950 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605730+0000", 2024-09-21T21:40:47.950 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:47.950 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.950 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.950 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:47.950 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605730+0000", 2024-09-21T21:40:47.950 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:47.951 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.951 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.951 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:47.951 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605730+0000", 2024-09-21T21:40:47.951 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5300000000000004e-07 2024-09-21T21:40:47.951 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.951 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.951 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:47.951 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605734+0000", 2024-09-21T21:40:47.951 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.2289999999999999e-06 2024-09-21T21:40:47.951 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.951 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.951 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:47.951 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605741+0000", 2024-09-21T21:40:47.952 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.4660000000000002e-06 2024-09-21T21:40:47.952 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.952 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.952 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:47.952 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605757+0000", 2024-09-21T21:40:47.952 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.5709000000000002e-05 2024-09-21T21:40:47.952 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.952 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.952 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:47.952 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.608168+0000", 2024-09-21T21:40:47.952 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.002411683 2024-09-21T21:40:47.952 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.952 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.952 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:47.952 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.608249+0000", 2024-09-21T21:40:47.953 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0903999999999997e-05 2024-09-21T21:40:47.953 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.953 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.953 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 1", 2024-09-21T21:40:47.953 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.608481+0000", 2024-09-21T21:40:47.953 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00023124800000000001 2024-09-21T21:40:47.953 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.953 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.953 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:40:47.953 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.615559+0000", 2024-09-21T21:40:47.953 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0070787009999999997 2024-09-21T21:40:47.953 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.953 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.953 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:47.953 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.616184+0000", 2024-09-21T21:40:47.954 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00062485199999999996 2024-09-21T21:40:47.954 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.954 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.954 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:47.954 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.616241+0000", 2024-09-21T21:40:47.954 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6971000000000001e-05 2024-09-21T21:40:47.954 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.954 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.954 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:47.954 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.616284+0000", 2024-09-21T21:40:47.954 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2438999999999998e-05 2024-09-21T21:40:47.954 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:47.954 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:47.954 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:47.954 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.955 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.955 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:509 3.8 3:10f57124:::rbd_object_map.11214be7c411:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:47.955 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.605831+0000", 2024-09-21T21:40:47.955 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 4.2811427230000003, 2024-09-21T21:40:47.955 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.010535957, 2024-09-21T21:40:47.955 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:47.955 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:47.955 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:47.955 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:47.955 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:47.955 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:47.955 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 509 2024-09-21T21:40:47.955 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.955 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:47.956 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.956 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:47.956 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605831+0000", 2024-09-21T21:40:47.956 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:47.956 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.956 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.956 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:47.956 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605831+0000", 2024-09-21T21:40:47.956 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:47.956 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.956 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.956 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:47.956 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605832+0000", 2024-09-21T21:40:47.956 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0999999999999997e-07 2024-09-21T21:40:47.957 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.957 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.957 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:47.957 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605837+0000", 2024-09-21T21:40:47.957 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1680000000000003e-06 2024-09-21T21:40:47.957 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.957 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.957 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:47.957 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605848+0000", 2024-09-21T21:40:47.957 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0715e-05 2024-09-21T21:40:47.957 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.957 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.957 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:47.957 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605855+0000", 2024-09-21T21:40:47.958 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.6359999999999994e-06 2024-09-21T21:40:47.958 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.958 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.958 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:47.958 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.608805+0000", 2024-09-21T21:40:47.958 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0029495430000000002 2024-09-21T21:40:47.958 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.958 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.958 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:47.958 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.608852+0000", 2024-09-21T21:40:47.958 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7258000000000001e-05 2024-09-21T21:40:47.958 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.958 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.958 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 1", 2024-09-21T21:40:47.959 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.609051+0000", 2024-09-21T21:40:47.959 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00019842300000000001 2024-09-21T21:40:47.959 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.959 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.959 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:40:47.959 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.615588+0000", 2024-09-21T21:40:47.959 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0065374049999999996 2024-09-21T21:40:47.959 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.959 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.959 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:47.959 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.616317+0000", 2024-09-21T21:40:47.960 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000729341 2024-09-21T21:40:47.960 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.960 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.960 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:47.960 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.616345+0000", 2024-09-21T21:40:47.960 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.7932999999999998e-05 2024-09-21T21:40:47.960 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.960 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.960 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:47.960 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.616367+0000", 2024-09-21T21:40:47.960 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1725e-05 2024-09-21T21:40:47.960 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:47.960 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:47.960 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:47.960 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.961 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.961 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:510 3.8 3:10f57124:::rbd_object_map.11214be7c411:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:47.961 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.606059+0000", 2024-09-21T21:40:47.961 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 4.280914653, 2024-09-21T21:40:47.961 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.010396717, 2024-09-21T21:40:47.961 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:47.961 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:47.961 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:47.961 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:47.961 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:47.961 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:47.961 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 510 2024-09-21T21:40:47.961 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.961 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:47.962 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.962 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:47.962 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.606059+0000", 2024-09-21T21:40:47.962 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:47.962 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.962 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.962 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:47.962 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.606059+0000", 2024-09-21T21:40:47.962 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:47.962 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.962 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.962 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:47.962 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.606060+0000", 2024-09-21T21:40:47.962 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.16e-07 2024-09-21T21:40:47.963 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.963 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.963 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:47.963 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.606065+0000", 2024-09-21T21:40:47.963 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2290000000000002e-06 2024-09-21T21:40:47.963 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.963 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.963 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:47.963 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.606076+0000", 2024-09-21T21:40:47.963 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0584999999999999e-05 2024-09-21T21:40:47.963 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.963 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.963 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:47.963 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.606094+0000", 2024-09-21T21:40:47.963 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.7920000000000001e-05 2024-09-21T21:40:47.964 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.964 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.964 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:47.964 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.609308+0000", 2024-09-21T21:40:47.964 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0032141980000000001 2024-09-21T21:40:47.964 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.964 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.964 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:47.964 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.609392+0000", 2024-09-21T21:40:47.964 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.4188000000000001e-05 2024-09-21T21:40:47.964 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.964 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.964 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 1", 2024-09-21T21:40:47.964 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.609627+0000", 2024-09-21T21:40:47.965 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00023437100000000001 2024-09-21T21:40:47.965 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.965 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.965 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:40:47.965 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.615594+0000", 2024-09-21T21:40:47.965 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0059679160000000002 2024-09-21T21:40:47.965 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.965 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.965 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:47.965 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.616394+0000", 2024-09-21T21:40:47.965 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00079993499999999999 2024-09-21T21:40:47.965 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.965 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.965 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:47.965 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.616419+0000", 2024-09-21T21:40:47.966 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.4834999999999999e-05 2024-09-21T21:40:47.966 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.966 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.966 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:47.966 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.616456+0000", 2024-09-21T21:40:47.966 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6624e-05 2024-09-21T21:40:47.966 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:47.966 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:47.966 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:47.966 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.966 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.966 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_repop(client.4417.0:497 3.d e26/21 3:b7efb195:::rbd_data.11214be7c411.0000000000000076:head v 26'22, mlcod=26'19)", 2024-09-21T21:40:47.966 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.606123+0000", 2024-09-21T21:40:47.967 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 4.2808507679999996, 2024-09-21T21:40:47.967 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0065331749999999996, 2024-09-21T21:40:47.967 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:47.967 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:47.967 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:47.967 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:47.967 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.967 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:47.967 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.606123+0000", 2024-09-21T21:40:47.967 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:47.967 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.967 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.967 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:47.967 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.606123+0000", 2024-09-21T21:40:47.967 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:47.968 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.968 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.968 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:47.968 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.606123+0000", 2024-09-21T21:40:47.968 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.0999999999999999e-07 2024-09-21T21:40:47.968 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.969 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.970 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:47.970 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.606128+0000", 2024-09-21T21:40:47.970 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5510000000000001e-06 2024-09-21T21:40:47.970 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.970 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.970 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:47.970 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.606133+0000", 2024-09-21T21:40:47.970 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3619999999999997e-06 2024-09-21T21:40:47.970 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.970 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.970 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:47.970 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.606138+0000", 2024-09-21T21:40:47.970 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3749999999999996e-06 2024-09-21T21:40:47.971 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.971 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.971 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:47.971 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.606195+0000", 2024-09-21T21:40:47.971 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7244999999999998e-05 2024-09-21T21:40:47.971 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.971 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.971 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:47.971 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.606213+0000", 2024-09-21T21:40:47.971 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.8116000000000001e-05 2024-09-21T21:40:47.971 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.971 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.971 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:47.971 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.612630+0000", 2024-09-21T21:40:47.971 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0064170670000000003 2024-09-21T21:40:47.972 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.972 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.972 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:47.972 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.612656+0000", 2024-09-21T21:40:47.972 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.6049000000000001e-05 2024-09-21T21:40:47.972 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:47.972 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:47.972 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:47.972 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.972 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.972 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:511 3.8 3:10f57124:::rbd_object_map.11214be7c411:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:47.972 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.606610+0000", 2024-09-21T21:40:47.972 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 4.2803642179999999, 2024-09-21T21:40:47.972 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0099483509999999994, 2024-09-21T21:40:47.973 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:47.973 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:47.973 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:47.973 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:47.973 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:47.973 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:47.973 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 511 2024-09-21T21:40:47.973 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.973 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:47.973 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.973 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:47.973 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.606610+0000", 2024-09-21T21:40:47.973 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:47.973 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.974 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.974 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:47.974 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.606610+0000", 2024-09-21T21:40:47.974 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:47.974 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.974 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.974 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:47.974 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.606610+0000", 2024-09-21T21:40:47.974 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.6000000000000002e-07 2024-09-21T21:40:47.974 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.974 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.974 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:47.974 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.606616+0000", 2024-09-21T21:40:47.975 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8429999999999998e-06 2024-09-21T21:40:47.975 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.975 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.975 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:47.975 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.606630+0000", 2024-09-21T21:40:47.975 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.3298999999999999e-05 2024-09-21T21:40:47.975 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.975 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.975 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:47.975 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.606637+0000", 2024-09-21T21:40:47.975 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5610000000000001e-06 2024-09-21T21:40:47.975 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.975 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.975 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:47.976 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.609878+0000", 2024-09-21T21:40:47.976 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0032406190000000001 2024-09-21T21:40:47.976 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.976 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.976 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:47.976 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.609922+0000", 2024-09-21T21:40:47.976 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4604000000000002e-05 2024-09-21T21:40:47.976 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.976 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.976 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 1", 2024-09-21T21:40:47.976 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.610113+0000", 2024-09-21T21:40:47.976 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00019091099999999999 2024-09-21T21:40:47.976 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.976 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.977 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:40:47.977 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.615599+0000", 2024-09-21T21:40:47.977 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0054859339999999996 2024-09-21T21:40:47.977 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.977 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.977 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:47.977 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.616497+0000", 2024-09-21T21:40:47.977 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000897908 2024-09-21T21:40:47.977 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.977 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.977 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:47.977 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.616534+0000", 2024-09-21T21:40:47.977 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6538000000000002e-05 2024-09-21T21:40:47.977 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.978 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.978 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:47.978 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.616558+0000", 2024-09-21T21:40:47.978 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.4273999999999998e-05 2024-09-21T21:40:47.978 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:47.978 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:47.978 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:47.978 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.978 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.978 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:512 3.8 3:10f57124:::rbd_object_map.11214be7c411:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:47.978 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.606653+0000", 2024-09-21T21:40:47.978 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 4.28032039, 2024-09-21T21:40:47.978 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.010056294, 2024-09-21T21:40:47.978 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:47.978 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:47.979 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:47.979 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:47.979 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:47.979 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:47.979 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 512 2024-09-21T21:40:47.979 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.979 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:47.979 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.979 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:47.979 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.606653+0000", 2024-09-21T21:40:47.979 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:47.979 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.979 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.979 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:47.979 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.606653+0000", 2024-09-21T21:40:47.980 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:47.980 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.980 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.980 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:47.980 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.606654+0000", 2024-09-21T21:40:47.980 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4600000000000005e-07 2024-09-21T21:40:47.980 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.980 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.980 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:47.980 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.606670+0000", 2024-09-21T21:40:47.980 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.6336000000000001e-05 2024-09-21T21:40:47.980 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.980 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.980 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:47.981 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.606680+0000", 2024-09-21T21:40:47.981 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.3400000000000004e-06 2024-09-21T21:40:47.981 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.981 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.981 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:47.981 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.606686+0000", 2024-09-21T21:40:47.981 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5610000000000004e-06 2024-09-21T21:40:47.981 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.981 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.981 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:47.981 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.610374+0000", 2024-09-21T21:40:47.981 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0036873330000000001 2024-09-21T21:40:47.981 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.981 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.981 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:47.982 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.610417+0000", 2024-09-21T21:40:47.982 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3872000000000003e-05 2024-09-21T21:40:47.982 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.982 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.982 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 1", 2024-09-21T21:40:47.982 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.610598+0000", 2024-09-21T21:40:47.982 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000180917 2024-09-21T21:40:47.982 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.982 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.982 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:40:47.982 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.615605+0000", 2024-09-21T21:40:47.982 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0050068179999999997 2024-09-21T21:40:47.982 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.982 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.982 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:47.983 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.616612+0000", 2024-09-21T21:40:47.983 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0010071660000000001 2024-09-21T21:40:47.983 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.983 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.983 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:47.983 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.616670+0000", 2024-09-21T21:40:47.983 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7321999999999998e-05 2024-09-21T21:40:47.983 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.983 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.983 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:47.983 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.616710+0000", 2024-09-21T21:40:47.983 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.0083000000000002e-05 2024-09-21T21:40:47.983 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:47.983 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:47.984 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:47.984 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.984 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.984 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:514 3.8 3:10f57124:::rbd_object_map.11214be7c411:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:47.984 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.607169+0000", 2024-09-21T21:40:47.984 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 4.2798045260000004, 2024-09-21T21:40:47.984 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0096953709999999995, 2024-09-21T21:40:47.984 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:47.984 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:47.984 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:47.984 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:47.984 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:47.984 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:47.984 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 514 2024-09-21T21:40:47.984 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.985 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:47.985 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.985 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:47.985 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607169+0000", 2024-09-21T21:40:47.985 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:47.985 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.985 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.985 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:47.985 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607169+0000", 2024-09-21T21:40:47.985 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:47.985 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.985 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.985 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:47.985 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607170+0000", 2024-09-21T21:40:47.985 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.9700000000000005e-07 2024-09-21T21:40:47.986 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.986 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.986 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:47.986 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607176+0000", 2024-09-21T21:40:47.986 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7529999999999998e-06 2024-09-21T21:40:47.986 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.986 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.986 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:47.986 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607188+0000", 2024-09-21T21:40:47.986 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1739999999999999e-05 2024-09-21T21:40:47.986 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.986 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.986 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:47.987 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607196+0000", 2024-09-21T21:40:47.987 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0739999999999993e-06 2024-09-21T21:40:47.987 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.987 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.987 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:47.987 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.610842+0000", 2024-09-21T21:40:47.987 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0036461750000000002 2024-09-21T21:40:47.987 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.987 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.987 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:47.987 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.610887+0000", 2024-09-21T21:40:47.987 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4953000000000001e-05 2024-09-21T21:40:47.987 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.987 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.987 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 1", 2024-09-21T21:40:47.988 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.611090+0000", 2024-09-21T21:40:47.988 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00020270100000000001 2024-09-21T21:40:47.988 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.988 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.988 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:40:47.988 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.615618+0000", 2024-09-21T21:40:47.988 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.004528486 2024-09-21T21:40:47.988 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.988 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.988 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:47.988 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.616769+0000", 2024-09-21T21:40:47.988 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.001150852 2024-09-21T21:40:47.988 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.988 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.989 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:47.989 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.616825+0000", 2024-09-21T21:40:47.989 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5686999999999997e-05 2024-09-21T21:40:47.989 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.989 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.989 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:47.989 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.616865+0000", 2024-09-21T21:40:47.989 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.0052999999999998e-05 2024-09-21T21:40:47.989 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:47.989 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:47.989 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:47.989 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.989 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.989 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:515 3.8 3:10f57124:::rbd_object_map.11214be7c411:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:47.990 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.607244+0000", 2024-09-21T21:40:47.990 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 4.2797296889999998, 2024-09-21T21:40:47.990 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0098905950000000003, 2024-09-21T21:40:47.990 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:47.990 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:47.990 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:47.990 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:47.990 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:47.990 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:47.990 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 515 2024-09-21T21:40:47.990 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.990 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:47.990 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.991 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:47.991 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607244+0000", 2024-09-21T21:40:47.991 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:47.991 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.991 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.991 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:47.991 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607244+0000", 2024-09-21T21:40:47.991 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:47.991 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.991 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.991 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:47.991 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607245+0000", 2024-09-21T21:40:47.991 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5199999999999996e-07 2024-09-21T21:40:47.991 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.991 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.992 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:47.992 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607249+0000", 2024-09-21T21:40:47.992 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3490000000000002e-06 2024-09-21T21:40:47.992 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.992 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.992 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:47.992 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607262+0000", 2024-09-21T21:40:47.992 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.3118999999999999e-05 2024-09-21T21:40:47.992 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.992 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.992 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:47.992 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607271+0000", 2024-09-21T21:40:47.992 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.8710000000000003e-06 2024-09-21T21:40:47.992 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.992 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.993 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:47.993 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.611347+0000", 2024-09-21T21:40:47.993 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0040757570000000002 2024-09-21T21:40:47.993 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.993 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.993 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:47.993 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.611432+0000", 2024-09-21T21:40:47.993 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.4670000000000001e-05 2024-09-21T21:40:47.993 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.993 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.993 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 1", 2024-09-21T21:40:47.993 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.611638+0000", 2024-09-21T21:40:47.993 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000206028 2024-09-21T21:40:47.993 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.993 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.994 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:40:47.994 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.615623+0000", 2024-09-21T21:40:47.994 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0039852960000000002 2024-09-21T21:40:47.994 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.994 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.994 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:47.994 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.617013+0000", 2024-09-21T21:40:47.994 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0013903170000000001 2024-09-21T21:40:47.994 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.994 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.994 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:47.994 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.617096+0000", 2024-09-21T21:40:47.994 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.3069e-05 2024-09-21T21:40:47.994 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:47.994 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:47.995 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:47.995 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.617135+0000", 2024-09-21T21:40:47.995 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8367e-05 2024-09-21T21:40:47.995 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:47.995 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:47.995 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:47.995 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:47.995 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:47.995 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:40:47.996 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:40:47.887+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:40:47.997 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:40:47.995+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:40:47.997 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T21:40:48.098 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:40:48.096+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:40:48.100 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:40:48.100 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:40:48.100 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-21T21:40:48.101 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:40:48.109 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T21:40:48.198 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:40:48.197+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:40:48.211 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:40:48.211 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:40:48.211 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:40:48.211 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:40:48.211 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:40:48.216 INFO:teuthology.orchestra.run.smithi164.stdout:256 read 0x2047200 thru 0x204bda9 (0x4baa bytes) 2024-09-21T21:40:48.218 INFO:teuthology.orchestra.run.smithi164.stdout:258 read 0x151c200 thru 0x152abb6 (0xe9b7 bytes) 2024-09-21T21:40:48.218 INFO:teuthology.orchestra.run.smithi164.stdout:truncating to largest ever: 0xca39600 2024-09-21T21:40:48.220 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T21:40:48.230 INFO:teuthology.orchestra.run.smithi164.stdout:259 trunc from 0x30da600 to 0xca39600 2024-09-21T21:40:48.243 INFO:teuthology.orchestra.run.smithi164.stdout:261 punch from 0x3e28800 to 0x3e2ec00, (0x6400 bytes) 2024-09-21T21:40:48.251 INFO:teuthology.orchestra.run.smithi164.stdout:263 punch from 0x31a0400 to 0x31a9200, (0x8e00 bytes) 2024-09-21T21:40:48.267 INFO:teuthology.orchestra.run.smithi164.stdout:264 write 0xd187200 thru 0xd189083 (0x1e84 bytes) 2024-09-21T21:40:48.267 INFO:teuthology.orchestra.run.smithi164.stdout:265 read 0xa8a4400 thru 0xa8a65b8 (0x21b9 bytes) 2024-09-21T21:40:48.267 INFO:teuthology.orchestra.run.smithi164.stdout:266 read 0x89b1a00 thru 0x89be4c4 (0xcac5 bytes) 2024-09-21T21:40:48.282 INFO:teuthology.orchestra.run.smithi164.stdout:267 trunc from 0xd189200 to 0x10dea00 2024-09-21T21:40:48.304 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:40:48.303+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:40:48.330 INFO:teuthology.orchestra.run.smithi164.stdout:268 write 0x9e6b400 thru 0x9e7143b (0x603c bytes) 2024-09-21T21:40:48.336 INFO:teuthology.orchestra.run.smithi164.stdout:269 punch from 0x5f50a00 to 0x5f5fc00, (0xf200 bytes) 2024-09-21T21:40:48.340 INFO:teuthology.orchestra.run.smithi164.stdout:270 trunc from 0x9e71600 to 0xc259200 2024-09-21T21:40:48.362 INFO:teuthology.orchestra.run.smithi164.stdout:272 trunc from 0xc259200 to 0x47cb000 2024-09-21T21:40:48.381 INFO:teuthology.orchestra.run.smithi164.stdout:274 punch from 0x3fee200 to 0x3ffc400, (0xe200 bytes) 2024-09-21T21:40:48.384 INFO:teuthology.orchestra.run.smithi164.stdout:275 read 0x2e7800 thru 0x2ea1c1 (0x29c2 bytes) 2024-09-21T21:40:48.384 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:40:48.384 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-21T21:40:48.384 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-21T21:40:48.385 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:40:48.385 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.385 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4183.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T21:40:48.385 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:19.643261+0000", 2024-09-21T21:40:48.385 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 28.738281702999998, 2024-09-21T21:40:48.385 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0075300310000000004, 2024-09-21T21:40:48.385 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:48.385 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:48.385 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:48.385 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:48.385 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4183", 2024-09-21T21:40:48.385 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.122:0/3913072034", 2024-09-21T21:40:48.386 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 112 2024-09-21T21:40:48.386 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.386 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:48.386 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.386 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:48.386 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643261+0000", 2024-09-21T21:40:48.386 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:48.386 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.386 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.386 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:48.386 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643261+0000", 2024-09-21T21:40:48.386 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:48.387 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.387 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.387 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:48.387 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643262+0000", 2024-09-21T21:40:48.387 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.99e-07 2024-09-21T21:40:48.387 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.387 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.387 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:48.387 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643326+0000", 2024-09-21T21:40:48.387 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4244000000000004e-05 2024-09-21T21:40:48.387 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.387 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.388 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:48.388 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643338+0000", 2024-09-21T21:40:48.388 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1807999999999999e-05 2024-09-21T21:40:48.388 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.388 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.388 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:48.388 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643344+0000", 2024-09-21T21:40:48.388 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4479999999999999e-06 2024-09-21T21:40:48.388 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.388 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.388 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:48.388 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.647517+0000", 2024-09-21T21:40:48.388 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.004172711 2024-09-21T21:40:48.389 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.389 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.389 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:48.389 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.647570+0000", 2024-09-21T21:40:48.389 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3094999999999999e-05 2024-09-21T21:40:48.389 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.389 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.389 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:40:48.389 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.647660+0000", 2024-09-21T21:40:48.389 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.0428999999999998e-05 2024-09-21T21:40:48.389 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.389 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.390 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:48.390 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.649208+0000", 2024-09-21T21:40:48.390 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0015479269999999999 2024-09-21T21:40:48.390 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.390 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.390 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:48.390 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.650733+0000", 2024-09-21T21:40:48.390 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0015247710000000001 2024-09-21T21:40:48.390 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.390 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.390 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:48.390 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.650764+0000", 2024-09-21T21:40:48.391 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0840000000000003e-05 2024-09-21T21:40:48.391 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.391 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.391 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:48.391 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.650791+0000", 2024-09-21T21:40:48.391 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7158999999999999e-05 2024-09-21T21:40:48.391 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:48.391 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:48.391 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:48.391 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.392 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.392 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4183.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T21:40:48.392 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:19.643357+0000", 2024-09-21T21:40:48.392 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 28.738185286, 2024-09-21T21:40:48.392 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.007525398, 2024-09-21T21:40:48.392 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:48.392 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:48.392 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:48.392 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:48.392 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4183", 2024-09-21T21:40:48.392 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.122:0/3913072034", 2024-09-21T21:40:48.392 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 113 2024-09-21T21:40:48.393 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.393 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:48.393 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.393 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:48.393 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643357+0000", 2024-09-21T21:40:48.393 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:48.393 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.393 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.393 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:48.393 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643357+0000", 2024-09-21T21:40:48.393 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:48.393 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.394 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.394 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:48.394 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643358+0000", 2024-09-21T21:40:48.394 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7100000000000001e-07 2024-09-21T21:40:48.394 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.394 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.394 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:48.394 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643362+0000", 2024-09-21T21:40:48.394 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2919999999999997e-06 2024-09-21T21:40:48.394 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.394 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.394 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:48.395 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643370+0000", 2024-09-21T21:40:48.395 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7190000000000001e-06 2024-09-21T21:40:48.395 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.395 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.395 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:48.395 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643376+0000", 2024-09-21T21:40:48.395 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3219999999999998e-06 2024-09-21T21:40:48.395 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.395 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.395 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:48.395 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.648180+0000", 2024-09-21T21:40:48.396 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048035619999999999 2024-09-21T21:40:48.396 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.396 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.396 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:48.396 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.648226+0000", 2024-09-21T21:40:48.396 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6303e-05 2024-09-21T21:40:48.396 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.396 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.396 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:40:48.396 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.648303+0000", 2024-09-21T21:40:48.396 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6831999999999999e-05 2024-09-21T21:40:48.396 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.396 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.397 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:48.397 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.649249+0000", 2024-09-21T21:40:48.397 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00094576299999999996 2024-09-21T21:40:48.397 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.397 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.397 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:48.397 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.650799+0000", 2024-09-21T21:40:48.397 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001549976 2024-09-21T21:40:48.397 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.397 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.397 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:48.397 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.650845+0000", 2024-09-21T21:40:48.398 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6133e-05 2024-09-21T21:40:48.398 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.398 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.398 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:48.398 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.650883+0000", 2024-09-21T21:40:48.398 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7824999999999998e-05 2024-09-21T21:40:48.398 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:48.398 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:48.398 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:48.398 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.398 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.398 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4183.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T21:40:48.399 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:19.643392+0000", 2024-09-21T21:40:48.399 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 28.738150214000001, 2024-09-21T21:40:48.399 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0075764780000000002, 2024-09-21T21:40:48.399 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:48.399 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:48.399 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:48.399 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:48.399 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4183", 2024-09-21T21:40:48.399 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.122:0/3913072034", 2024-09-21T21:40:48.399 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 114 2024-09-21T21:40:48.399 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.400 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:48.400 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.400 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:48.400 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643392+0000", 2024-09-21T21:40:48.400 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:48.400 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.400 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.400 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:48.400 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643392+0000", 2024-09-21T21:40:48.400 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:48.400 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.401 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.401 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:48.401 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643393+0000", 2024-09-21T21:40:48.401 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2900000000000003e-07 2024-09-21T21:40:48.401 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.401 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.401 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:48.401 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643398+0000", 2024-09-21T21:40:48.401 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3159999999999998e-06 2024-09-21T21:40:48.401 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.401 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.401 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:48.401 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643403+0000", 2024-09-21T21:40:48.402 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5419999999999996e-06 2024-09-21T21:40:48.402 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.402 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.402 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:48.402 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643407+0000", 2024-09-21T21:40:48.402 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4649999999999999e-06 2024-09-21T21:40:48.402 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.402 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.402 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:48.402 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.648632+0000", 2024-09-21T21:40:48.402 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0052258000000000001 2024-09-21T21:40:48.403 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.403 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.403 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:48.403 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.648734+0000", 2024-09-21T21:40:48.403 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010178099999999999 2024-09-21T21:40:48.403 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.403 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.403 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:40:48.403 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.648880+0000", 2024-09-21T21:40:48.403 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014552799999999999 2024-09-21T21:40:48.403 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.403 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.403 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:48.404 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.650398+0000", 2024-09-21T21:40:48.404 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0015188389999999999 2024-09-21T21:40:48.404 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.404 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.404 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:48.404 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.650903+0000", 2024-09-21T21:40:48.404 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00050413400000000005 2024-09-21T21:40:48.404 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.405 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.405 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:48.405 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.650940+0000", 2024-09-21T21:40:48.405 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7354999999999997e-05 2024-09-21T21:40:48.405 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.405 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.405 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:48.405 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.650969+0000", 2024-09-21T21:40:48.405 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8989e-05 2024-09-21T21:40:48.405 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:48.405 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:48.405 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:48.405 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.406 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.406 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4183.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T21:40:48.406 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:19.643416+0000", 2024-09-21T21:40:48.406 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 28.738126274999999, 2024-09-21T21:40:48.406 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0076828269999999997, 2024-09-21T21:40:48.406 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:48.406 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:48.406 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:48.406 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:48.406 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4183", 2024-09-21T21:40:48.406 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.122:0/3913072034", 2024-09-21T21:40:48.407 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 115 2024-09-21T21:40:48.407 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.407 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:48.407 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.407 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:48.407 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643416+0000", 2024-09-21T21:40:48.407 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:48.407 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.407 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.407 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:48.407 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643416+0000", 2024-09-21T21:40:48.407 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:48.408 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.408 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.408 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:48.408 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643417+0000", 2024-09-21T21:40:48.408 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.96e-07 2024-09-21T21:40:48.408 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.408 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.408 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:48.408 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643469+0000", 2024-09-21T21:40:48.408 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2293999999999998e-05 2024-09-21T21:40:48.408 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.408 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.409 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:48.409 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643474+0000", 2024-09-21T21:40:48.409 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3179999999999997e-06 2024-09-21T21:40:48.409 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.409 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.409 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:48.409 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643488+0000", 2024-09-21T21:40:48.409 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3241000000000001e-05 2024-09-21T21:40:48.409 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.409 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.409 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:48.409 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.649349+0000", 2024-09-21T21:40:48.409 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0058617690000000002 2024-09-21T21:40:48.410 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.410 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.410 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:48.410 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.649440+0000", 2024-09-21T21:40:48.410 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.1092000000000001e-05 2024-09-21T21:40:48.410 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.410 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.410 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:40:48.410 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.649583+0000", 2024-09-21T21:40:48.410 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014218499999999999 2024-09-21T21:40:48.410 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.410 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.411 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:48.411 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.650978+0000", 2024-09-21T21:40:48.411 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0013956210000000001 2024-09-21T21:40:48.411 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.411 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.411 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:48.411 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.651032+0000", 2024-09-21T21:40:48.411 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3513000000000001e-05 2024-09-21T21:40:48.411 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.411 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.411 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:48.411 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.651068+0000", 2024-09-21T21:40:48.412 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6675999999999999e-05 2024-09-21T21:40:48.412 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.412 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.412 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:48.412 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.651099+0000", 2024-09-21T21:40:48.412 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0821999999999997e-05 2024-09-21T21:40:48.412 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:48.412 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:48.412 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:48.412 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.412 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.412 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4183.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T21:40:48.413 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:19.643496+0000", 2024-09-21T21:40:48.413 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 28.738046730000001, 2024-09-21T21:40:48.413 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0077031249999999999, 2024-09-21T21:40:48.413 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:48.413 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:48.413 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:48.413 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:48.413 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4183", 2024-09-21T21:40:48.413 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.122:0/3913072034", 2024-09-21T21:40:48.413 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 116 2024-09-21T21:40:48.413 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.413 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:48.413 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.414 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:48.414 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643496+0000", 2024-09-21T21:40:48.414 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:48.414 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.414 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.414 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:48.414 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643496+0000", 2024-09-21T21:40:48.414 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:48.414 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.414 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.414 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:48.414 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643496+0000", 2024-09-21T21:40:48.415 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3599999999999999e-07 2024-09-21T21:40:48.415 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.415 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.415 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:48.415 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643500+0000", 2024-09-21T21:40:48.415 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1790000000000001e-06 2024-09-21T21:40:48.415 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.415 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.415 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:48.415 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643504+0000", 2024-09-21T21:40:48.415 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8299999999999998e-06 2024-09-21T21:40:48.415 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.415 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.416 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:48.416 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643507+0000", 2024-09-21T21:40:48.416 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.056e-06 2024-09-21T21:40:48.416 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.416 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.416 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:48.416 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.650003+0000", 2024-09-21T21:40:48.416 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0064959129999999999 2024-09-21T21:40:48.416 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.416 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.416 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:48.416 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.650051+0000", 2024-09-21T21:40:48.417 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8180999999999998e-05 2024-09-21T21:40:48.417 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.417 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.417 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:40:48.417 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.650127+0000", 2024-09-21T21:40:48.417 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5795000000000002e-05 2024-09-21T21:40:48.417 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.417 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.417 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:48.417 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.651137+0000", 2024-09-21T21:40:48.417 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0010094699999999999 2024-09-21T21:40:48.417 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.418 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.418 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:48.418 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.651144+0000", 2024-09-21T21:40:48.418 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.985e-06 2024-09-21T21:40:48.418 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.418 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.418 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:48.418 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.651169+0000", 2024-09-21T21:40:48.418 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.563e-05 2024-09-21T21:40:48.418 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.418 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.418 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:48.419 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.651199+0000", 2024-09-21T21:40:48.419 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9750000000000001e-05 2024-09-21T21:40:48.419 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:48.419 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:48.419 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:48.419 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.419 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.419 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4183.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T21:40:48.419 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:19.643512+0000", 2024-09-21T21:40:48.419 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 28.738031115999998, 2024-09-21T21:40:48.419 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0077806539999999997, 2024-09-21T21:40:48.419 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:48.420 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:48.420 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:48.420 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:48.420 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4183", 2024-09-21T21:40:48.420 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.122:0/3913072034", 2024-09-21T21:40:48.420 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 117 2024-09-21T21:40:48.420 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.420 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:48.420 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.420 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:48.420 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643512+0000", 2024-09-21T21:40:48.420 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:48.420 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.421 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.421 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:48.421 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643512+0000", 2024-09-21T21:40:48.421 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:48.421 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.421 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.421 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:48.421 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643512+0000", 2024-09-21T21:40:48.421 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.16e-07 2024-09-21T21:40:48.421 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.421 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.421 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:48.422 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643514+0000", 2024-09-21T21:40:48.422 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3709999999999998e-06 2024-09-21T21:40:48.422 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.422 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.422 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:48.422 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643517+0000", 2024-09-21T21:40:48.422 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2310000000000001e-06 2024-09-21T21:40:48.422 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.422 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.422 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:48.422 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643522+0000", 2024-09-21T21:40:48.422 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8319999999999996e-06 2024-09-21T21:40:48.423 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.423 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.423 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:48.423 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.650426+0000", 2024-09-21T21:40:48.423 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0069033380000000002 2024-09-21T21:40:48.423 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.423 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.423 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:48.423 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.650482+0000", 2024-09-21T21:40:48.423 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6468999999999998e-05 2024-09-21T21:40:48.423 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.423 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.424 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:40:48.424 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.650565+0000", 2024-09-21T21:40:48.424 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3026000000000001e-05 2024-09-21T21:40:48.424 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.424 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.424 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:48.424 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.651209+0000", 2024-09-21T21:40:48.424 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00064387199999999998 2024-09-21T21:40:48.424 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.424 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.424 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:48.424 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.651238+0000", 2024-09-21T21:40:48.425 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8799e-05 2024-09-21T21:40:48.425 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.425 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.425 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:48.425 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.651270+0000", 2024-09-21T21:40:48.425 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1934000000000003e-05 2024-09-21T21:40:48.425 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.425 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.425 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:48.425 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.651292+0000", 2024-09-21T21:40:48.425 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2566e-05 2024-09-21T21:40:48.425 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:48.425 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:48.426 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:48.426 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.426 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.426 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4183.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T21:40:48.426 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:19.916463+0000", 2024-09-21T21:40:48.426 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 28.465079632999998, 2024-09-21T21:40:48.426 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0094252929999999995, 2024-09-21T21:40:48.426 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:48.426 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:48.426 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:48.426 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:48.426 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4183", 2024-09-21T21:40:48.426 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.122:0/3913072034", 2024-09-21T21:40:48.427 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 134 2024-09-21T21:40:48.427 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.427 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:48.427 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.427 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:48.427 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916463+0000", 2024-09-21T21:40:48.427 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:48.427 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.427 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.427 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:48.427 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916463+0000", 2024-09-21T21:40:48.427 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:48.427 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.428 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.428 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:48.428 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916464+0000", 2024-09-21T21:40:48.428 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2900000000000004e-07 2024-09-21T21:40:48.428 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.428 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.428 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:48.428 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916525+0000", 2024-09-21T21:40:48.428 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1576999999999997e-05 2024-09-21T21:40:48.428 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.428 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.428 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:48.429 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916533+0000", 2024-09-21T21:40:48.429 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7510000000000008e-06 2024-09-21T21:40:48.429 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.429 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.429 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:48.429 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916537+0000", 2024-09-21T21:40:48.429 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.155e-06 2024-09-21T21:40:48.429 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.429 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.429 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:48.429 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.919134+0000", 2024-09-21T21:40:48.429 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025973469999999999 2024-09-21T21:40:48.429 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.430 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.430 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:48.430 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.919189+0000", 2024-09-21T21:40:48.430 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4339999999999998e-05 2024-09-21T21:40:48.430 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.430 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.430 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:40:48.430 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.919261+0000", 2024-09-21T21:40:48.430 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1977000000000006e-05 2024-09-21T21:40:48.430 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.430 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.430 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:48.431 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.920953+0000", 2024-09-21T21:40:48.431 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0016926109999999999 2024-09-21T21:40:48.431 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.431 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.431 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:48.431 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.925825+0000", 2024-09-21T21:40:48.431 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048719159999999996 2024-09-21T21:40:48.431 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.431 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.431 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:48.431 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.925854+0000", 2024-09-21T21:40:48.432 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8524999999999999e-05 2024-09-21T21:40:48.432 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.432 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.432 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:48.432 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.925888+0000", 2024-09-21T21:40:48.432 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4564999999999999e-05 2024-09-21T21:40:48.432 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:48.432 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:48.432 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:48.433 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.433 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.433 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4183.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T21:40:48.433 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:19.916545+0000", 2024-09-21T21:40:48.433 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 28.464997715999999, 2024-09-21T21:40:48.433 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0094280869999999999, 2024-09-21T21:40:48.433 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:48.433 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:48.433 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:48.433 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:48.433 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4183", 2024-09-21T21:40:48.433 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.122:0/3913072034", 2024-09-21T21:40:48.434 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 135 2024-09-21T21:40:48.434 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.434 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:48.434 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.434 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:48.434 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916545+0000", 2024-09-21T21:40:48.434 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:48.434 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.434 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.434 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:48.434 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916545+0000", 2024-09-21T21:40:48.434 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:48.435 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.435 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.435 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:48.435 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916545+0000", 2024-09-21T21:40:48.435 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1100000000000002e-07 2024-09-21T21:40:48.435 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.435 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.435 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:48.435 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916549+0000", 2024-09-21T21:40:48.435 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0799999999999999e-06 2024-09-21T21:40:48.435 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.435 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.436 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:48.436 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916556+0000", 2024-09-21T21:40:48.436 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0339999999999999e-06 2024-09-21T21:40:48.436 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.436 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.436 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:48.436 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916562+0000", 2024-09-21T21:40:48.436 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5439999999999998e-06 2024-09-21T21:40:48.436 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.436 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.436 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:48.436 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.919649+0000", 2024-09-21T21:40:48.436 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0030868660000000002 2024-09-21T21:40:48.437 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.437 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.437 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:48.437 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.919693+0000", 2024-09-21T21:40:48.437 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4694000000000003e-05 2024-09-21T21:40:48.437 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.437 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.437 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:40:48.437 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.919765+0000", 2024-09-21T21:40:48.437 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1427999999999994e-05 2024-09-21T21:40:48.438 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.438 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.438 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:48.438 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.921005+0000", 2024-09-21T21:40:48.438 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001240314 2024-09-21T21:40:48.438 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.438 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.438 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:48.438 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.925897+0000", 2024-09-21T21:40:48.438 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048916660000000002 2024-09-21T21:40:48.438 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.438 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.439 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:48.439 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.925942+0000", 2024-09-21T21:40:48.439 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4721000000000001e-05 2024-09-21T21:40:48.439 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.439 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.439 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:48.439 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.925973+0000", 2024-09-21T21:40:48.439 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1429e-05 2024-09-21T21:40:48.439 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:48.439 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:48.439 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:48.439 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.439 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.440 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4183.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T21:40:48.440 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:19.916568+0000", 2024-09-21T21:40:48.440 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 28.464974781999999, 2024-09-21T21:40:48.440 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0094755120000000002, 2024-09-21T21:40:48.440 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:48.440 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:48.440 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:48.440 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:48.440 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4183", 2024-09-21T21:40:48.440 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.122:0/3913072034", 2024-09-21T21:40:48.440 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 136 2024-09-21T21:40:48.440 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.440 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:48.441 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.441 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:48.441 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916568+0000", 2024-09-21T21:40:48.441 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:48.441 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.441 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.441 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:48.441 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916568+0000", 2024-09-21T21:40:48.441 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:48.441 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.441 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.441 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:48.442 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916568+0000", 2024-09-21T21:40:48.442 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9200000000000002e-07 2024-09-21T21:40:48.442 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.442 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.442 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:48.442 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916572+0000", 2024-09-21T21:40:48.442 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.428e-06 2024-09-21T21:40:48.442 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.442 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.442 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:48.442 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916576+0000", 2024-09-21T21:40:48.442 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.814e-06 2024-09-21T21:40:48.442 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.443 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.443 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:48.443 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916580+0000", 2024-09-21T21:40:48.443 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6669999999999998e-06 2024-09-21T21:40:48.443 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.443 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.443 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:48.443 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.920045+0000", 2024-09-21T21:40:48.443 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0034645209999999999 2024-09-21T21:40:48.443 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.443 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.443 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:48.443 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.920091+0000", 2024-09-21T21:40:48.444 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6146999999999997e-05 2024-09-21T21:40:48.444 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.444 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.444 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:40:48.444 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.920159+0000", 2024-09-21T21:40:48.444 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8411000000000006e-05 2024-09-21T21:40:48.444 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.444 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.444 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:48.444 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.921037+0000", 2024-09-21T21:40:48.444 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00087794700000000002 2024-09-21T21:40:48.444 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.445 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.445 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:48.445 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.925981+0000", 2024-09-21T21:40:48.445 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0049443259999999998 2024-09-21T21:40:48.445 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.445 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.445 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:48.445 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.926014+0000", 2024-09-21T21:40:48.445 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2298999999999997e-05 2024-09-21T21:40:48.445 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.445 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.445 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:48.445 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.926043+0000", 2024-09-21T21:40:48.446 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9660000000000001e-05 2024-09-21T21:40:48.446 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:48.446 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:48.446 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:48.446 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.446 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.446 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4183.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T21:40:48.446 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:19.916587+0000", 2024-09-21T21:40:48.446 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 28.464955302, 2024-09-21T21:40:48.446 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.009527265, 2024-09-21T21:40:48.446 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:48.446 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:48.446 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:48.447 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:48.447 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4183", 2024-09-21T21:40:48.447 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.122:0/3913072034", 2024-09-21T21:40:48.447 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 137 2024-09-21T21:40:48.447 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.447 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:48.447 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.447 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:48.447 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916587+0000", 2024-09-21T21:40:48.447 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:48.447 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.447 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.448 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:48.448 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916587+0000", 2024-09-21T21:40:48.448 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:48.448 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.448 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.448 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:48.448 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916588+0000", 2024-09-21T21:40:48.448 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5699999999999999e-07 2024-09-21T21:40:48.448 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.448 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.448 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:48.448 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916644+0000", 2024-09-21T21:40:48.448 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6344000000000001e-05 2024-09-21T21:40:48.449 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.449 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.449 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:48.449 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916649+0000", 2024-09-21T21:40:48.449 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0409999999999997e-06 2024-09-21T21:40:48.449 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.449 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.449 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:48.449 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916654+0000", 2024-09-21T21:40:48.449 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9919999999999998e-06 2024-09-21T21:40:48.449 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.449 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.449 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:48.450 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.920443+0000", 2024-09-21T21:40:48.450 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0037892469999999999 2024-09-21T21:40:48.450 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.450 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.450 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:48.450 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.920493+0000", 2024-09-21T21:40:48.450 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9898000000000002e-05 2024-09-21T21:40:48.450 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.450 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.450 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:40:48.450 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.920561+0000", 2024-09-21T21:40:48.450 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7481e-05 2024-09-21T21:40:48.451 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.451 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.451 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:48.451 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.925739+0000", 2024-09-21T21:40:48.451 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.005178729 2024-09-21T21:40:48.451 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.451 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.451 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:48.451 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.926052+0000", 2024-09-21T21:40:48.451 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00031252800000000001 2024-09-21T21:40:48.451 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.451 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.452 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:48.452 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.926087+0000", 2024-09-21T21:40:48.452 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4873999999999998e-05 2024-09-21T21:40:48.452 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.452 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.452 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:48.452 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.926115+0000", 2024-09-21T21:40:48.452 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7874000000000001e-05 2024-09-21T21:40:48.452 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:48.452 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:48.452 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:48.452 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.453 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.453 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4183.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T21:40:48.453 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:19.916666+0000", 2024-09-21T21:40:48.453 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 28.464876431, 2024-09-21T21:40:48.453 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0095150730000000006, 2024-09-21T21:40:48.453 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:48.453 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:48.453 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:48.453 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:48.453 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4183", 2024-09-21T21:40:48.453 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.122:0/3913072034", 2024-09-21T21:40:48.453 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 138 2024-09-21T21:40:48.454 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.454 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:48.454 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.454 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:48.454 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916666+0000", 2024-09-21T21:40:48.454 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:48.454 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.454 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.454 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:48.454 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916666+0000", 2024-09-21T21:40:48.454 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:48.454 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.454 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.455 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:48.455 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916667+0000", 2024-09-21T21:40:48.455 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3200000000000005e-07 2024-09-21T21:40:48.455 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.455 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.455 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:48.455 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916670+0000", 2024-09-21T21:40:48.455 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.427e-06 2024-09-21T21:40:48.455 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.455 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.455 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:48.455 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916684+0000", 2024-09-21T21:40:48.456 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3839000000000001e-05 2024-09-21T21:40:48.456 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.456 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.456 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:48.456 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916706+0000", 2024-09-21T21:40:48.456 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1543e-05 2024-09-21T21:40:48.456 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.456 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.456 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:48.456 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.921064+0000", 2024-09-21T21:40:48.456 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0043587119999999998 2024-09-21T21:40:48.456 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.456 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.457 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:48.457 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.921119+0000", 2024-09-21T21:40:48.457 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4619e-05 2024-09-21T21:40:48.457 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.457 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.457 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:40:48.457 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.921192+0000", 2024-09-21T21:40:48.457 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3224000000000002e-05 2024-09-21T21:40:48.457 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.457 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.457 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:48.457 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.925791+0000", 2024-09-21T21:40:48.457 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0045988670000000004 2024-09-21T21:40:48.458 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.458 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.458 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:48.458 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.926123+0000", 2024-09-21T21:40:48.458 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00033195000000000002 2024-09-21T21:40:48.458 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.458 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.458 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:48.458 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.926154+0000", 2024-09-21T21:40:48.458 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0720999999999999e-05 2024-09-21T21:40:48.458 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.458 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.459 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:48.459 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.926181+0000", 2024-09-21T21:40:48.459 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7639000000000001e-05 2024-09-21T21:40:48.459 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:48.459 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:48.459 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:48.459 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.459 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.459 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4183.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T21:40:48.459 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:19.916714+0000", 2024-09-21T21:40:48.459 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 28.464828541999999, 2024-09-21T21:40:48.459 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0095298369999999993, 2024-09-21T21:40:48.459 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:48.460 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:48.460 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:48.460 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:48.460 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4183", 2024-09-21T21:40:48.460 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.122:0/3913072034", 2024-09-21T21:40:48.460 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 139 2024-09-21T21:40:48.460 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.460 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:48.460 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.460 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:48.460 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916714+0000", 2024-09-21T21:40:48.460 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:48.460 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.461 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.461 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:48.461 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916714+0000", 2024-09-21T21:40:48.461 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:48.461 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.461 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.461 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:48.461 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916714+0000", 2024-09-21T21:40:48.461 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9799999999999999e-07 2024-09-21T21:40:48.461 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.461 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.461 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:48.462 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916718+0000", 2024-09-21T21:40:48.462 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1810000000000001e-06 2024-09-21T21:40:48.462 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.462 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.462 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:48.462 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916724+0000", 2024-09-21T21:40:48.462 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7329999999999997e-06 2024-09-21T21:40:48.462 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.462 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.462 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:48.462 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916728+0000", 2024-09-21T21:40:48.462 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2909999999999999e-06 2024-09-21T21:40:48.463 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.463 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.463 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:48.463 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.921416+0000", 2024-09-21T21:40:48.463 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0046882850000000004 2024-09-21T21:40:48.463 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.463 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.463 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:48.463 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.921461+0000", 2024-09-21T21:40:48.463 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5259000000000002e-05 2024-09-21T21:40:48.463 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.464 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.464 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:40:48.464 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.921540+0000", 2024-09-21T21:40:48.464 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8765999999999998e-05 2024-09-21T21:40:48.464 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.464 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.464 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:48.464 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.925818+0000", 2024-09-21T21:40:48.464 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0042784800000000003 2024-09-21T21:40:48.464 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.464 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.464 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:48.465 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.926190+0000", 2024-09-21T21:40:48.465 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00037112400000000002 2024-09-21T21:40:48.465 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.465 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.465 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:48.465 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.926225+0000", 2024-09-21T21:40:48.465 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5420999999999998e-05 2024-09-21T21:40:48.465 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.465 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.465 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:48.465 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.926244+0000", 2024-09-21T21:40:48.465 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.8998999999999999e-05 2024-09-21T21:40:48.466 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:48.466 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:48.466 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:48.466 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.466 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.466 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4183.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T21:40:48.466 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:19.916735+0000", 2024-09-21T21:40:48.466 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 28.464808144999999, 2024-09-21T21:40:48.466 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.010302162, 2024-09-21T21:40:48.466 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:48.466 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:48.466 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:48.467 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:48.467 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4183", 2024-09-21T21:40:48.467 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.122:0/3913072034", 2024-09-21T21:40:48.467 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 140 2024-09-21T21:40:48.467 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.467 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:48.467 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.467 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:48.467 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916735+0000", 2024-09-21T21:40:48.467 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:48.467 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.467 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.468 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:48.468 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916735+0000", 2024-09-21T21:40:48.468 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:48.468 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.468 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.468 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:48.468 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916735+0000", 2024-09-21T21:40:48.468 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5400000000000002e-07 2024-09-21T21:40:48.468 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.468 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.468 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:48.468 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916788+0000", 2024-09-21T21:40:48.469 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3074e-05 2024-09-21T21:40:48.469 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.469 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.469 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:48.469 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916795+0000", 2024-09-21T21:40:48.469 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.204e-06 2024-09-21T21:40:48.469 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.469 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.469 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:48.469 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916801+0000", 2024-09-21T21:40:48.469 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4759999999999997e-06 2024-09-21T21:40:48.469 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.469 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.470 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:48.470 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.921792+0000", 2024-09-21T21:40:48.470 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0049909749999999999 2024-09-21T21:40:48.470 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.470 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.470 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:48.470 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.921879+0000", 2024-09-21T21:40:48.470 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7361999999999995e-05 2024-09-21T21:40:48.470 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.470 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.470 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:40:48.470 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.922000+0000", 2024-09-21T21:40:48.471 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000121451 2024-09-21T21:40:48.471 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.471 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.471 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:48.471 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.926291+0000", 2024-09-21T21:40:48.471 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0042906180000000004 2024-09-21T21:40:48.471 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.471 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.471 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:48.471 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.926984+0000", 2024-09-21T21:40:48.471 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00069308599999999998 2024-09-21T21:40:48.471 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.471 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.472 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:48.472 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.927014+0000", 2024-09-21T21:40:48.472 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0222999999999998e-05 2024-09-21T21:40:48.472 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.472 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.472 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:48.472 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.927037+0000", 2024-09-21T21:40:48.472 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2439e-05 2024-09-21T21:40:48.472 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:48.472 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:48.472 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:48.472 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.472 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.473 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4183.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T21:40:48.473 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:19.916814+0000", 2024-09-21T21:40:48.473 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 28.464728955999998, 2024-09-21T21:40:48.473 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.010507598999999999, 2024-09-21T21:40:48.473 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:48.473 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:48.473 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:48.473 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:48.473 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4183", 2024-09-21T21:40:48.473 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.122:0/3913072034", 2024-09-21T21:40:48.473 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 141 2024-09-21T21:40:48.473 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.474 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:48.474 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.474 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:48.474 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916814+0000", 2024-09-21T21:40:48.474 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:48.474 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.474 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.474 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:48.474 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916814+0000", 2024-09-21T21:40:48.474 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:48.474 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.474 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.475 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:48.475 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916814+0000", 2024-09-21T21:40:48.475 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9599999999999999e-07 2024-09-21T21:40:48.475 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.475 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.475 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:48.475 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916819+0000", 2024-09-21T21:40:48.475 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7090000000000001e-06 2024-09-21T21:40:48.475 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.475 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.475 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:48.475 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916830+0000", 2024-09-21T21:40:48.475 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0977e-05 2024-09-21T21:40:48.476 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.476 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.476 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:48.476 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916835+0000", 2024-09-21T21:40:48.476 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2320000000000001e-06 2024-09-21T21:40:48.476 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.476 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.476 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:48.476 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.925249+0000", 2024-09-21T21:40:48.476 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0084135170000000006 2024-09-21T21:40:48.476 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.476 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.476 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:48.477 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.925323+0000", 2024-09-21T21:40:48.477 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3919999999999997e-05 2024-09-21T21:40:48.477 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.477 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.477 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:40:48.477 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.925442+0000", 2024-09-21T21:40:48.477 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00011944500000000001 2024-09-21T21:40:48.477 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.477 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.477 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:48.477 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.926322+0000", 2024-09-21T21:40:48.477 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00087985500000000003 2024-09-21T21:40:48.478 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.478 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.478 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:48.478 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.927245+0000", 2024-09-21T21:40:48.478 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00092342499999999996 2024-09-21T21:40:48.478 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.478 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.478 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:48.478 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.927283+0000", 2024-09-21T21:40:48.478 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8157000000000001e-05 2024-09-21T21:40:48.478 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.478 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.479 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:48.479 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.927321+0000", 2024-09-21T21:40:48.479 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7866e-05 2024-09-21T21:40:48.479 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:48.479 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:48.479 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:48.479 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.479 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.479 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4183.0:145 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T21:40:48.479 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:19.927637+0000", 2024-09-21T21:40:48.479 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 28.453905698, 2024-09-21T21:40:48.479 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.006407622, 2024-09-21T21:40:48.480 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:48.480 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:48.480 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:48.480 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:48.480 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4183", 2024-09-21T21:40:48.480 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.122:0/3913072034", 2024-09-21T21:40:48.480 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 145 2024-09-21T21:40:48.480 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.480 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:48.480 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.480 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:48.480 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.927637+0000", 2024-09-21T21:40:48.481 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:48.481 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.481 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.481 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:48.481 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.927637+0000", 2024-09-21T21:40:48.481 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:48.481 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.481 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.481 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:48.481 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.927637+0000", 2024-09-21T21:40:48.481 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5199999999999998e-07 2024-09-21T21:40:48.481 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.481 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.482 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:48.482 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.927684+0000", 2024-09-21T21:40:48.482 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6529000000000001e-05 2024-09-21T21:40:48.482 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.482 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.482 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:48.482 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.927689+0000", 2024-09-21T21:40:48.482 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6959999999999996e-06 2024-09-21T21:40:48.482 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.482 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.482 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:48.482 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.927692+0000", 2024-09-21T21:40:48.483 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1659999999999998e-06 2024-09-21T21:40:48.483 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.483 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.483 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:48.483 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.929102+0000", 2024-09-21T21:40:48.483 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0014106679999999999 2024-09-21T21:40:48.483 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.483 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.483 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:48.483 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.929147+0000", 2024-09-21T21:40:48.483 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4459999999999998e-05 2024-09-21T21:40:48.483 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.483 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.484 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:40:48.484 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.929219+0000", 2024-09-21T21:40:48.484 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2547999999999997e-05 2024-09-21T21:40:48.484 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.484 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.484 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:48.484 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.932636+0000", 2024-09-21T21:40:48.484 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0034168559999999998 2024-09-21T21:40:48.484 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.484 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.484 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:48.484 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.933989+0000", 2024-09-21T21:40:48.485 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001352699 2024-09-21T21:40:48.485 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.485 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.485 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:48.485 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.934022+0000", 2024-09-21T21:40:48.485 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2829e-05 2024-09-21T21:40:48.485 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.485 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.485 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:48.485 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.934045+0000", 2024-09-21T21:40:48.485 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2819e-05 2024-09-21T21:40:48.485 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:48.486 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:48.486 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:48.486 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.486 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.486 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4417.0:131 3.15 3:abf84a89:::rbd_data.11214be7c411.0000000000000025:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:48.486 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.535860+0000", 2024-09-21T21:40:48.486 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.8456828160000001, 2024-09-21T21:40:48.486 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0064965179999999997, 2024-09-21T21:40:48.486 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:48.486 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:48.486 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:40:48.486 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:48.487 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4417", 2024-09-21T21:40:48.487 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:48.487 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 131 2024-09-21T21:40:48.487 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.487 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:48.487 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.487 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:48.487 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.535860+0000", 2024-09-21T21:40:48.487 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:48.487 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.487 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.487 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:48.488 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.535860+0000", 2024-09-21T21:40:48.488 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:48.488 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.488 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.488 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:48.488 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.535867+0000", 2024-09-21T21:40:48.488 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2200000000000003e-06 2024-09-21T21:40:48.488 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.488 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.488 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:48.488 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.535870+0000", 2024-09-21T21:40:48.488 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4070000000000002e-06 2024-09-21T21:40:48.488 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.489 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.489 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:48.489 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.535879+0000", 2024-09-21T21:40:48.489 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1640000000000001e-06 2024-09-21T21:40:48.489 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.489 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.489 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:48.489 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.535886+0000", 2024-09-21T21:40:48.489 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.503e-06 2024-09-21T21:40:48.489 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.489 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.489 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:48.490 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.535926+0000", 2024-09-21T21:40:48.490 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0203000000000001e-05 2024-09-21T21:40:48.490 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.490 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.490 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:48.490 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.536005+0000", 2024-09-21T21:40:48.490 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8838000000000006e-05 2024-09-21T21:40:48.490 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.490 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.490 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:48.490 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.542356+0000", 2024-09-21T21:40:48.491 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0063511829999999998 2024-09-21T21:40:48.491 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:48.491 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:48.491 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:48.491 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.491 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.491 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4417.0:496 3.d 3:b065aa9d:::rbd_data.11214be7c411.0000000000000074:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:48.491 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.604764+0000", 2024-09-21T21:40:48.491 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.7767782219999999, 2024-09-21T21:40:48.491 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0080497150000000007, 2024-09-21T21:40:48.491 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:48.491 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:48.491 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:48.492 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:48.492 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4417", 2024-09-21T21:40:48.492 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:48.492 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 496 2024-09-21T21:40:48.492 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.492 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:48.492 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.492 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:48.492 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.604764+0000", 2024-09-21T21:40:48.492 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:48.492 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.492 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.492 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:48.493 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.604764+0000", 2024-09-21T21:40:48.493 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:48.493 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.493 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.493 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:48.493 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.604765+0000", 2024-09-21T21:40:48.493 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6300000000000005e-07 2024-09-21T21:40:48.493 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.493 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.493 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:48.493 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.604768+0000", 2024-09-21T21:40:48.493 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0070000000000002e-06 2024-09-21T21:40:48.494 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.494 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.494 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:48.494 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.604774+0000", 2024-09-21T21:40:48.494 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2689999999999996e-06 2024-09-21T21:40:48.494 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.494 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.494 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:48.494 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.604778+0000", 2024-09-21T21:40:48.494 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.749e-06 2024-09-21T21:40:48.494 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.494 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.495 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:48.495 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.605207+0000", 2024-09-21T21:40:48.495 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00042899099999999999 2024-09-21T21:40:48.495 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.495 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.495 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:48.495 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.605251+0000", 2024-09-21T21:40:48.495 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4258999999999998e-05 2024-09-21T21:40:48.495 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.495 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.495 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 2", 2024-09-21T21:40:48.495 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.605430+0000", 2024-09-21T21:40:48.496 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00017844700000000001 2024-09-21T21:40:48.496 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.496 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.496 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:48.496 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.606879+0000", 2024-09-21T21:40:48.496 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001449129 2024-09-21T21:40:48.496 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.496 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.496 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:48.496 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.612748+0000", 2024-09-21T21:40:48.496 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0058691510000000004 2024-09-21T21:40:48.497 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.497 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.497 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:48.497 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.612785+0000", 2024-09-21T21:40:48.497 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6730999999999998e-05 2024-09-21T21:40:48.497 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.497 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.497 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:48.497 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.612814+0000", 2024-09-21T21:40:48.497 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9419000000000001e-05 2024-09-21T21:40:48.497 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:48.498 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:48.498 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:48.498 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.498 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.498 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4417.0:497 3.d 3:b7efb195:::rbd_data.11214be7c411.0000000000000076:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:48.498 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.604904+0000", 2024-09-21T21:40:48.498 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.7766390589999999, 2024-09-21T21:40:48.498 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0080130610000000001, 2024-09-21T21:40:48.498 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:48.498 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:48.498 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:48.499 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:48.499 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4417", 2024-09-21T21:40:48.499 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:48.499 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 497 2024-09-21T21:40:48.499 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.499 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:48.499 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.499 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:48.499 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.604904+0000", 2024-09-21T21:40:48.499 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:48.499 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.499 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.500 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:48.500 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.604904+0000", 2024-09-21T21:40:48.500 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:48.500 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.500 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.500 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:48.500 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.604904+0000", 2024-09-21T21:40:48.500 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5099999999999999e-07 2024-09-21T21:40:48.500 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.500 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.500 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:48.501 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.604909+0000", 2024-09-21T21:40:48.501 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6959999999999996e-06 2024-09-21T21:40:48.501 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.501 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.501 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:48.501 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.604918+0000", 2024-09-21T21:40:48.501 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7229999999999993e-06 2024-09-21T21:40:48.501 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.501 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.501 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:48.501 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.604924+0000", 2024-09-21T21:40:48.501 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9909999999999999e-06 2024-09-21T21:40:48.502 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.502 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.502 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:48.502 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.605720+0000", 2024-09-21T21:40:48.502 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00079632800000000001 2024-09-21T21:40:48.502 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.502 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.502 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:48.502 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.605800+0000", 2024-09-21T21:40:48.502 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0302999999999993e-05 2024-09-21T21:40:48.502 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.502 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.503 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 2", 2024-09-21T21:40:48.503 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.605960+0000", 2024-09-21T21:40:48.503 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000159211 2024-09-21T21:40:48.503 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.503 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.503 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:48.503 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.606887+0000", 2024-09-21T21:40:48.503 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00092735699999999996 2024-09-21T21:40:48.503 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.503 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.503 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:48.503 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.612849+0000", 2024-09-21T21:40:48.504 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0059615450000000004 2024-09-21T21:40:48.504 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.504 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.504 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:48.504 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.612892+0000", 2024-09-21T21:40:48.504 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3683000000000002e-05 2024-09-21T21:40:48.504 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.504 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.504 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:48.504 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.612917+0000", 2024-09-21T21:40:48.504 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4473000000000001e-05 2024-09-21T21:40:48.504 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:48.505 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:48.505 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:48.505 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.505 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.505 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:636 3.13 3:ce41eea9:::rbd_data.1121dee34a14.0000000000000008:6 [read 1548288~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-21T21:40:48.505 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:48.294999+0000", 2024-09-21T21:40:48.505 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.086543901000000006, 2024-09-21T21:40:48.505 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0062603249999999997, 2024-09-21T21:40:48.505 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:48.505 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:48.505 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:40:48.505 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:48.506 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2024-09-21T21:40:48.506 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1688278210", 2024-09-21T21:40:48.506 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 636 2024-09-21T21:40:48.506 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.506 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:48.506 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.506 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:48.506 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.294999+0000", 2024-09-21T21:40:48.506 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:48.506 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.506 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.506 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:48.507 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.294999+0000", 2024-09-21T21:40:48.507 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:48.507 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.507 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.507 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:48.507 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.294999+0000", 2024-09-21T21:40:48.507 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4899999999999995e-07 2024-09-21T21:40:48.507 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.507 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.507 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:48.507 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.295002+0000", 2024-09-21T21:40:48.507 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.199e-06 2024-09-21T21:40:48.508 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.508 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.508 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:48.508 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.295008+0000", 2024-09-21T21:40:48.508 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4659999999999996e-06 2024-09-21T21:40:48.508 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.508 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.508 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:48.508 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.295013+0000", 2024-09-21T21:40:48.508 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7090000000000001e-06 2024-09-21T21:40:48.508 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.508 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.509 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:48.509 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.301102+0000", 2024-09-21T21:40:48.509 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0060893079999999999 2024-09-21T21:40:48.509 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.509 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.509 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:48.509 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.301175+0000", 2024-09-21T21:40:48.509 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2999999999999999e-05 2024-09-21T21:40:48.509 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.509 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.509 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:48.510 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.301259+0000", 2024-09-21T21:40:48.510 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.4093999999999998e-05 2024-09-21T21:40:48.510 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:48.510 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:48.510 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:48.510 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.510 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.510 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:637 3.13 3:ce41eea9:::rbd_data.1121dee34a14.0000000000000008:6 [read 1564672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-21T21:40:48.510 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:48.295020+0000", 2024-09-21T21:40:48.510 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.086523029000000001, 2024-09-21T21:40:48.510 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0063974119999999999, 2024-09-21T21:40:48.510 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:48.511 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:48.511 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:40:48.511 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:48.511 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2024-09-21T21:40:48.511 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1688278210", 2024-09-21T21:40:48.511 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 637 2024-09-21T21:40:48.511 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.511 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:48.511 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.511 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:48.511 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.295020+0000", 2024-09-21T21:40:48.511 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:48.511 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.512 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.512 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:48.512 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.295020+0000", 2024-09-21T21:40:48.512 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:48.512 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.512 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.512 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:48.512 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.295020+0000", 2024-09-21T21:40:48.512 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7899999999999999e-07 2024-09-21T21:40:48.512 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.512 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.512 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:48.513 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.295029+0000", 2024-09-21T21:40:48.513 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6710000000000005e-06 2024-09-21T21:40:48.513 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.513 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.513 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:48.513 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.295035+0000", 2024-09-21T21:40:48.513 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5080000000000002e-06 2024-09-21T21:40:48.513 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.513 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.513 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:48.513 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.295040+0000", 2024-09-21T21:40:48.513 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9189999999999998e-06 2024-09-21T21:40:48.514 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.514 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.514 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:48.514 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.301287+0000", 2024-09-21T21:40:48.514 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0062466630000000004 2024-09-21T21:40:48.514 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.514 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.514 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:48.514 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.301359+0000", 2024-09-21T21:40:48.514 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2675000000000005e-05 2024-09-21T21:40:48.514 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.514 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.515 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:48.515 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.301417+0000", 2024-09-21T21:40:48.515 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7596999999999998e-05 2024-09-21T21:40:48.515 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:48.515 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:48.515 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:48.515 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:48.515 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:48.515 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:40:48.518 INFO:teuthology.orchestra.run.smithi081.stderr:noscrub is set 2024-09-21T21:40:48.518 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T21:40:48.519 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:40:48.515+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:40:48.520 INFO:teuthology.orchestra.run.smithi164.stdout:277 write 0x139600 thru 0x13cf64 (0x3965 bytes) 2024-09-21T21:40:48.520 INFO:teuthology.orchestra.run.smithi164.stdout:280 read 0x25e1000 thru 0x25e1715 (0x716 bytes) 2024-09-21T21:40:48.521 INFO:teuthology.orchestra.run.smithi164.stdout:281 write 0xc106800 thru 0xc10ec84 (0x8485 bytes) 2024-09-21T21:40:48.521 INFO:teuthology.orchestra.run.smithi164.stdout:282 write 0xa823800 thru 0xa8321a0 (0xe9a1 bytes) 2024-09-21T21:40:48.521 INFO:teuthology.orchestra.run.smithi164.stdout:284 punch from 0x6611800 to 0x661f000, (0xd800 bytes) 2024-09-21T21:40:48.521 INFO:teuthology.orchestra.run.smithi164.stdout:285 read 0x88e2e00 thru 0x88e2e0c (0xd bytes) 2024-09-21T21:40:48.521 INFO:teuthology.orchestra.run.smithi164.stdout:286 read 0xa64bc00 thru 0xa656b7b (0xaf7c bytes) 2024-09-21T21:40:48.521 INFO:teuthology.orchestra.run.smithi164.stdout:290 punch from 0xb572000 to 0xb57aa00, (0x8a00 bytes) 2024-09-21T21:40:48.521 INFO:teuthology.orchestra.run.smithi164.stdout:295 write 0x3e18200 thru 0x3e23c89 (0xba8a bytes) 2024-09-21T21:40:48.521 INFO:teuthology.orchestra.run.smithi164.stdout:296 read 0xd3ac00 thru 0xd3af0f (0x310 bytes) 2024-09-21T21:40:48.521 INFO:teuthology.orchestra.run.smithi164.stdout:298 read 0x6df6c00 thru 0x6e028aa (0xbcab bytes) 2024-09-21T21:40:48.521 INFO:teuthology.orchestra.run.smithi164.stdout:truncating to largest ever: 0xe7e9000 2024-09-21T21:40:48.521 INFO:teuthology.orchestra.run.smithi164.stdout:299 trunc from 0xc10ee00 to 0xe7e9000 2024-09-21T21:40:48.521 INFO:teuthology.orchestra.run.smithi164.stdout:300 write 0xa418200 thru 0xa41aeed (0x2cee bytes) 2024-09-21T21:40:48.522 INFO:teuthology.orchestra.run.smithi164.stdout:302 read 0xa510000 thru 0xa51ed78 (0xed79 bytes) 2024-09-21T21:40:48.522 INFO:teuthology.orchestra.run.smithi164.stdout:303 trunc from 0xe7e9000 to 0xe3f6c00 2024-09-21T21:40:48.522 INFO:teuthology.orchestra.run.smithi164.stdout:306 write 0xce79c00 thru 0xce7fb8b (0x5f8c bytes) 2024-09-21T21:40:48.522 INFO:teuthology.orchestra.run.smithi164.stdout:307 read 0xb481400 thru 0xb486a60 (0x5661 bytes) 2024-09-21T21:40:48.522 INFO:teuthology.orchestra.run.smithi164.stdout:310 trunc from 0xe3f6c00 to 0x16bee00 2024-09-21T21:40:48.522 INFO:teuthology.orchestra.run.smithi164.stdout:312 read 0x1ab400 thru 0x1b305b (0x7c5c bytes) 2024-09-21T21:40:48.522 INFO:teuthology.orchestra.run.smithi164.stdout:313 write 0x9e02c00 thru 0x9e02fc2 (0x3c3 bytes) 2024-09-21T21:40:48.522 INFO:teuthology.orchestra.run.smithi164.stdout:316 read 0x71d600 thru 0x71f1e8 (0x1be9 bytes) 2024-09-21T21:40:48.522 INFO:teuthology.orchestra.run.smithi164.stdout:320 trunc from 0x9e03000 to 0xe0ebc00 2024-09-21T21:40:48.524 INFO:teuthology.orchestra.run.smithi164.stdout:321 read 0x7f9d400 thru 0x7fa5ac1 (0x86c2 bytes) 2024-09-21T21:40:48.525 INFO:teuthology.orchestra.run.smithi164.stdout:322 write 0xda6400 thru 0xdb0570 (0xa171 bytes) 2024-09-21T21:40:48.526 INFO:teuthology.orchestra.run.smithi164.stdout:324 write 0x76cb200 thru 0x76ce947 (0x3748 bytes) 2024-09-21T21:40:48.536 INFO:teuthology.orchestra.run.smithi164.stdout:325 trunc from 0xe0ebc00 to 0x5be3000 2024-09-21T21:40:48.568 INFO:teuthology.orchestra.run.smithi164.stdout:327 trunc from 0x5be3000 to 0x23d400 2024-09-21T21:40:48.617 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:40:48.615+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:40:48.624 INFO:teuthology.orchestra.run.smithi164.stdout:329 write 0xea35800 thru 0xea44e74 (0xf675 bytes) 2024-09-21T21:40:48.631 INFO:teuthology.orchestra.run.smithi164.stdout:331 write 0xd5a2200 thru 0xd5af497 (0xd298 bytes) 2024-09-21T21:40:48.639 INFO:teuthology.orchestra.run.smithi164.stdout:332 trunc from 0xea45000 to 0x71cb200 2024-09-21T21:40:48.667 INFO:teuthology.orchestra.run.smithi164.stdout:333 punch from 0x4467600 to 0x446aa00, (0x3400 bytes) 2024-09-21T21:40:48.668 INFO:teuthology.orchestra.run.smithi164.stdout:335 trunc from 0x71cb200 to 0x873ce00 2024-09-21T21:40:48.679 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:40:48.679 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:40:48.679 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-21T21:40:48.679 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:40:48.687 INFO:teuthology.orchestra.run.smithi164.stdout:336 trunc from 0x873ce00 to 0xacdac00 2024-09-21T21:40:48.688 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T21:40:48.700 INFO:teuthology.orchestra.run.smithi164.stdout:337 trunc from 0xacdac00 to 0xbd7a400 2024-09-21T21:40:48.710 INFO:teuthology.orchestra.run.smithi164.stdout:338 trunc from 0xbd7a400 to 0xcdaa000 2024-09-21T21:40:48.717 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:40:48.716+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:40:48.720 INFO:teuthology.orchestra.run.smithi164.stdout:342 write 0xdbda00 thru 0xdc92ed (0xb8ee bytes) 2024-09-21T21:40:48.734 INFO:teuthology.orchestra.run.smithi164.stdout:343 write 0xd38a200 thru 0xd398c56 (0xea57 bytes) 2024-09-21T21:40:48.739 INFO:teuthology.orchestra.run.smithi164.stdout:344 write 0x1196600 thru 0x11a2bca (0xc5cb bytes) 2024-09-21T21:40:48.739 INFO:teuthology.orchestra.run.smithi164.stdout:347 read 0x61f200 thru 0x622dd2 (0x3bd3 bytes) 2024-09-21T21:40:48.739 INFO:teuthology.orchestra.run.smithi164.stdout:349 punch from 0xbfa9400 to 0xbfae800, (0x5400 bytes) 2024-09-21T21:40:48.741 INFO:teuthology.orchestra.run.smithi164.stdout:350 punch from 0xa47ec00 to 0xa489000, (0xa400 bytes) 2024-09-21T21:40:48.741 INFO:teuthology.orchestra.run.smithi164.stdout:351 read 0xf86c00 thru 0xf8a5a2 (0x39a3 bytes) 2024-09-21T21:40:48.741 INFO:teuthology.orchestra.run.smithi164.stdout:352 read 0x7ca7600 thru 0x7cb4092 (0xca93 bytes) 2024-09-21T21:40:48.741 INFO:teuthology.orchestra.run.smithi164.stdout:353 read 0x69dd600 thru 0x69e37d1 (0x61d2 bytes) 2024-09-21T21:40:48.741 INFO:teuthology.orchestra.run.smithi164.stdout:355 read 0x2cb7400 thru 0x2cc3f84 (0xcb85 bytes) 2024-09-21T21:40:48.741 INFO:teuthology.orchestra.run.smithi164.stdout:357 read 0x38600 thru 0x3b77e (0x317f bytes) 2024-09-21T21:40:48.741 INFO:teuthology.orchestra.run.smithi164.stdout:358 clone 5 order 19 su 4096 sc 3 2024-09-21T21:40:48.818 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:40:48.816+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:40:48.835 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:40:48.835 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:40:48.835 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:40:48.835 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:40:48.835 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:40:48.843 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T21:40:48.919 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:40:48.918+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:40:48.947 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:40:48.947 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-21T21:40:48.947 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-21T21:40:48.947 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:40:48.947 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.947 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4417.0:117 3.9 3:92506b0a:::rbd_data.11214be7c411.000000000000001c:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:48.947 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.534624+0000", 2024-09-21T21:40:48.948 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 5.4098720130000002, 2024-09-21T21:40:48.948 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0047323440000000003, 2024-09-21T21:40:48.948 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:48.948 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:48.948 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:40:48.948 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:48.948 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4417", 2024-09-21T21:40:48.948 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:48.948 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 117 2024-09-21T21:40:48.948 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.949 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:48.949 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.949 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:48.949 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.534624+0000", 2024-09-21T21:40:48.949 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:48.949 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.949 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.949 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:48.949 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.534624+0000", 2024-09-21T21:40:48.949 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:48.949 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.949 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.949 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:48.949 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.534625+0000", 2024-09-21T21:40:48.950 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.61e-07 2024-09-21T21:40:48.950 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.950 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.950 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:48.950 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.534629+0000", 2024-09-21T21:40:48.950 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9639999999999997e-06 2024-09-21T21:40:48.950 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.950 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.950 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:48.950 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.534638+0000", 2024-09-21T21:40:48.950 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8710000000000003e-06 2024-09-21T21:40:48.950 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.950 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.950 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:48.951 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.534644+0000", 2024-09-21T21:40:48.951 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5590000000000002e-06 2024-09-21T21:40:48.951 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.951 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.951 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:48.951 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.534681+0000", 2024-09-21T21:40:48.951 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6504999999999997e-05 2024-09-21T21:40:48.951 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.951 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.951 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:48.951 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.534762+0000", 2024-09-21T21:40:48.951 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0729000000000006e-05 2024-09-21T21:40:48.951 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.951 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.951 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:48.952 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.539357+0000", 2024-09-21T21:40:48.952 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0045950549999999998 2024-09-21T21:40:48.952 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:48.952 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:48.952 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:48.952 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.952 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.952 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4417.0:162 3.9 3:92506b0a:::rbd_data.11214be7c411.000000000000001c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:48.952 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.539532+0000", 2024-09-21T21:40:48.952 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 5.404964433, 2024-09-21T21:40:48.952 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0056942770000000002, 2024-09-21T21:40:48.952 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:48.952 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:48.953 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:48.953 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:48.953 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4417", 2024-09-21T21:40:48.953 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:48.953 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 162 2024-09-21T21:40:48.953 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.953 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:48.953 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.954 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:48.954 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.539532+0000", 2024-09-21T21:40:48.954 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:48.954 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.954 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.954 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:48.954 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.539532+0000", 2024-09-21T21:40:48.954 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:48.954 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.954 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.954 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:48.954 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.539533+0000", 2024-09-21T21:40:48.954 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0610000000000001e-06 2024-09-21T21:40:48.955 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.955 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.955 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:48.955 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.539538+0000", 2024-09-21T21:40:48.955 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9949999999999996e-06 2024-09-21T21:40:48.955 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.955 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.955 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:48.955 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.539552+0000", 2024-09-21T21:40:48.955 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3823e-05 2024-09-21T21:40:48.955 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.956 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.956 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:48.956 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.539562+0000", 2024-09-21T21:40:48.956 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0011e-05 2024-09-21T21:40:48.956 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.956 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.956 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:48.956 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.539638+0000", 2024-09-21T21:40:48.956 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5912000000000001e-05 2024-09-21T21:40:48.956 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.956 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.956 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:48.956 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.539738+0000", 2024-09-21T21:40:48.956 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000100086 2024-09-21T21:40:48.957 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.957 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.957 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 2", 2024-09-21T21:40:48.957 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.539920+0000", 2024-09-21T21:40:48.957 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00018205399999999999 2024-09-21T21:40:48.957 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.957 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.957 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:48.957 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.540794+0000", 2024-09-21T21:40:48.957 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00087363399999999998 2024-09-21T21:40:48.957 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.957 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.958 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:48.958 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.545130+0000", 2024-09-21T21:40:48.958 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0043360940000000004 2024-09-21T21:40:48.958 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.958 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.958 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:48.958 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.545167+0000", 2024-09-21T21:40:48.958 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7069e-05 2024-09-21T21:40:48.958 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.958 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.958 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:48.958 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.545226+0000", 2024-09-21T21:40:48.958 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9537999999999998e-05 2024-09-21T21:40:48.959 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:48.959 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:48.959 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:48.959 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.959 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.959 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4417.0:163 3.9 3:9336cf29:::rbd_data.11214be7c411.0000000000000026:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:48.959 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.539786+0000", 2024-09-21T21:40:48.959 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 5.4047100429999997, 2024-09-21T21:40:48.959 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0055684869999999996, 2024-09-21T21:40:48.959 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:48.959 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:48.959 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:48.959 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:48.959 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4417", 2024-09-21T21:40:48.959 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:48.960 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 163 2024-09-21T21:40:48.960 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.960 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:48.960 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.960 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:48.960 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.539786+0000", 2024-09-21T21:40:48.960 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:48.960 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.960 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.960 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:48.960 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.539786+0000", 2024-09-21T21:40:48.960 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:48.960 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.960 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.960 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:48.961 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.539787+0000", 2024-09-21T21:40:48.961 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8500000000000001e-07 2024-09-21T21:40:48.961 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.961 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.961 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:48.961 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.539794+0000", 2024-09-21T21:40:48.961 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9190000000000002e-06 2024-09-21T21:40:48.961 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.961 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.961 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:48.961 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.539804+0000", 2024-09-21T21:40:48.961 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0054999999999999e-05 2024-09-21T21:40:48.961 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.961 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.961 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:48.962 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.539811+0000", 2024-09-21T21:40:48.962 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3150000000000003e-06 2024-09-21T21:40:48.962 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.962 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.962 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:48.962 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.540229+0000", 2024-09-21T21:40:48.962 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00041778099999999999 2024-09-21T21:40:48.962 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.962 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.962 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:48.962 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.540314+0000", 2024-09-21T21:40:48.962 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.4640000000000003e-05 2024-09-21T21:40:48.962 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.962 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.963 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 2", 2024-09-21T21:40:48.963 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.540528+0000", 2024-09-21T21:40:48.963 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00021472400000000001 2024-09-21T21:40:48.963 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.963 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.963 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:48.963 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.541998+0000", 2024-09-21T21:40:48.963 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001469087 2024-09-21T21:40:48.963 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.963 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.963 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:48.963 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.545266+0000", 2024-09-21T21:40:48.963 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0032689289999999998 2024-09-21T21:40:48.964 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.964 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.964 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:48.964 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.545319+0000", 2024-09-21T21:40:48.964 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2148999999999998e-05 2024-09-21T21:40:48.964 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.964 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.964 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:48.964 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.545355+0000", 2024-09-21T21:40:48.964 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6202999999999999e-05 2024-09-21T21:40:48.964 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:48.965 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:48.965 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:48.965 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.965 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.965 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4417.0:178 3.3 3:c3470000:::rbd_data.11214be7c411.000000000000002e:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:48.965 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.545144+0000", 2024-09-21T21:40:48.965 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 5.3993520290000001, 2024-09-21T21:40:48.965 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0054365009999999998, 2024-09-21T21:40:48.965 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:48.965 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:48.965 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:40:48.965 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:48.965 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4417", 2024-09-21T21:40:48.965 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:48.965 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 178 2024-09-21T21:40:48.966 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.966 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:48.966 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.966 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:48.966 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.545144+0000", 2024-09-21T21:40:48.966 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:48.966 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.966 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.966 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:48.966 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.545144+0000", 2024-09-21T21:40:48.966 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:48.966 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.966 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.967 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:48.967 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.545145+0000", 2024-09-21T21:40:48.967 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0800000000000004e-07 2024-09-21T21:40:48.967 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.967 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.967 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:48.967 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.545149+0000", 2024-09-21T21:40:48.967 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7409999999999998e-06 2024-09-21T21:40:48.967 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.967 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.967 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:48.967 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.545159+0000", 2024-09-21T21:40:48.968 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0078000000000001e-05 2024-09-21T21:40:48.968 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.968 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.968 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:48.968 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.545165+0000", 2024-09-21T21:40:48.968 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1979999999999998e-06 2024-09-21T21:40:48.968 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.968 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.968 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:48.968 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.545222+0000", 2024-09-21T21:40:48.968 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6999000000000001e-05 2024-09-21T21:40:48.968 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.968 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.968 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:48.968 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.545326+0000", 2024-09-21T21:40:48.969 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000104168 2024-09-21T21:40:48.969 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.969 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.969 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:48.969 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.550581+0000", 2024-09-21T21:40:48.969 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0052545090000000001 2024-09-21T21:40:48.969 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:48.969 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:48.969 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:48.969 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.969 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.969 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4417.0:198 3.1a 3:5ab0459a:::rbd_data.11214be7c411.0000000000000032:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:48.969 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.547904+0000", 2024-09-21T21:40:48.969 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 5.3965923050000004, 2024-09-21T21:40:48.969 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.005385679, 2024-09-21T21:40:48.970 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:48.970 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:48.970 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:40:48.970 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:48.970 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4417", 2024-09-21T21:40:48.970 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:48.971 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 198 2024-09-21T21:40:48.971 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.971 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:48.971 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.971 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:48.971 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.547904+0000", 2024-09-21T21:40:48.971 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:48.972 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.972 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.972 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:48.972 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.547904+0000", 2024-09-21T21:40:48.972 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:48.972 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.972 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.972 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:48.973 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.547905+0000", 2024-09-21T21:40:48.973 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0400000000000005e-07 2024-09-21T21:40:48.973 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.973 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.973 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:48.973 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.547909+0000", 2024-09-21T21:40:48.973 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8650000000000003e-06 2024-09-21T21:40:48.973 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.974 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.974 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:48.974 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.547919+0000", 2024-09-21T21:40:48.974 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0134e-05 2024-09-21T21:40:48.974 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.974 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.974 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:48.974 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.547926+0000", 2024-09-21T21:40:48.975 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7880000000000001e-06 2024-09-21T21:40:48.975 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.975 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.975 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:48.975 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.548144+0000", 2024-09-21T21:40:48.975 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000218598 2024-09-21T21:40:48.975 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.976 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.976 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:48.976 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.548278+0000", 2024-09-21T21:40:48.976 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000134213 2024-09-21T21:40:48.976 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.976 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.976 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:48.977 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.553290+0000", 2024-09-21T21:40:48.977 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0050112769999999997 2024-09-21T21:40:48.977 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:48.977 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:48.977 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:48.977 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.977 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.977 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4417.0:219 3.3 3:c3470000:::rbd_data.11214be7c411.000000000000002e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:48.977 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.550734+0000", 2024-09-21T21:40:48.977 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 5.3937621199999999, 2024-09-21T21:40:48.977 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0053111310000000002, 2024-09-21T21:40:48.977 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:48.977 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:48.977 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:48.978 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:48.978 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4417", 2024-09-21T21:40:48.978 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:48.978 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 219 2024-09-21T21:40:48.978 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.978 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:48.978 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.978 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:48.978 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.550734+0000", 2024-09-21T21:40:48.978 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:48.978 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.978 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.978 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:48.978 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.550734+0000", 2024-09-21T21:40:48.979 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:48.979 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.979 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.979 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:48.979 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.550736+0000", 2024-09-21T21:40:48.979 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.297e-06 2024-09-21T21:40:48.979 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.979 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.979 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:48.979 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.550741+0000", 2024-09-21T21:40:48.979 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2669999999999997e-06 2024-09-21T21:40:48.979 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.979 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.980 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:48.980 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.550754+0000", 2024-09-21T21:40:48.980 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2996000000000001e-05 2024-09-21T21:40:48.980 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.980 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.980 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:48.980 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.550761+0000", 2024-09-21T21:40:48.980 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1760000000000004e-06 2024-09-21T21:40:48.980 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.980 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.980 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:48.980 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.550801+0000", 2024-09-21T21:40:48.980 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0185999999999998e-05 2024-09-21T21:40:48.980 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.980 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.981 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:48.981 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.550871+0000", 2024-09-21T21:40:48.981 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0018999999999995e-05 2024-09-21T21:40:48.981 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.981 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.981 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:40:48.981 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.550999+0000", 2024-09-21T21:40:48.981 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012815099999999999 2024-09-21T21:40:48.981 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.981 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.981 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:48.981 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.551530+0000", 2024-09-21T21:40:48.981 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00053112500000000002 2024-09-21T21:40:48.981 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.982 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.982 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:48.982 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.555975+0000", 2024-09-21T21:40:48.982 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0044446759999999998 2024-09-21T21:40:48.982 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.982 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.982 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:48.982 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.556012+0000", 2024-09-21T21:40:48.982 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6536999999999999e-05 2024-09-21T21:40:48.982 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.982 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.982 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:48.982 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.556045+0000", 2024-09-21T21:40:48.982 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3701000000000001e-05 2024-09-21T21:40:48.983 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:48.983 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:48.983 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:48.983 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.983 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.983 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(client.4417.0:305 3.14 e26/21 3:2aa90881:::rbd_data.11214be7c411.0000000000000046:head v 26'6, mlcod=26'4)", 2024-09-21T21:40:48.983 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.569452+0000", 2024-09-21T21:40:48.983 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 5.3750439820000002, 2024-09-21T21:40:48.983 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0047599219999999998, 2024-09-21T21:40:48.983 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:48.983 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:48.983 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:48.983 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:48.983 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.984 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:48.984 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.569452+0000", 2024-09-21T21:40:48.984 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:48.984 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.984 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.984 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:48.984 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.569452+0000", 2024-09-21T21:40:48.984 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:48.984 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.984 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.984 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:48.984 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.569453+0000", 2024-09-21T21:40:48.984 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5300000000000003e-07 2024-09-21T21:40:48.984 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.984 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.985 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:48.985 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.569469+0000", 2024-09-21T21:40:48.985 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5410999999999999e-05 2024-09-21T21:40:48.985 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.985 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.985 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:48.985 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.569481+0000", 2024-09-21T21:40:48.985 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2391e-05 2024-09-21T21:40:48.985 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.985 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.985 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:48.985 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.569488+0000", 2024-09-21T21:40:48.985 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5679999999999996e-06 2024-09-21T21:40:48.985 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.985 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.985 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:48.986 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.569537+0000", 2024-09-21T21:40:48.986 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9406000000000001e-05 2024-09-21T21:40:48.986 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.986 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.986 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:48.986 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.569556+0000", 2024-09-21T21:40:48.986 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9264999999999999e-05 2024-09-21T21:40:48.986 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.986 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.986 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:48.986 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.574149+0000", 2024-09-21T21:40:48.986 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0045932780000000001 2024-09-21T21:40:48.986 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.986 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.986 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:48.989 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.574212+0000", 2024-09-21T21:40:48.989 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2849999999999996e-05 2024-09-21T21:40:48.989 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:48.989 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:48.989 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:48.990 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.990 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.990 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4417.0:318 3.3 3:c6040d19:::rbd_data.11214be7c411.000000000000004e:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:48.990 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.571797+0000", 2024-09-21T21:40:48.990 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 5.3726990199999998, 2024-09-21T21:40:48.990 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0047787790000000004, 2024-09-21T21:40:48.990 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:48.990 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:48.990 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:40:48.990 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:48.990 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4417", 2024-09-21T21:40:48.990 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:48.990 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 318 2024-09-21T21:40:48.990 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.991 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:48.991 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.991 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:48.991 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.571797+0000", 2024-09-21T21:40:48.991 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:48.991 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.991 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.991 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:48.991 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.571797+0000", 2024-09-21T21:40:48.991 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:48.991 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.991 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.991 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:48.991 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.571798+0000", 2024-09-21T21:40:48.992 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.7600000000000006e-07 2024-09-21T21:40:48.992 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.992 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.992 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:48.992 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.571803+0000", 2024-09-21T21:40:48.992 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2169999999999996e-06 2024-09-21T21:40:48.992 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.992 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.992 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:48.992 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.571811+0000", 2024-09-21T21:40:48.992 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9969999999999998e-06 2024-09-21T21:40:48.992 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.992 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.992 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:48.993 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.571815+0000", 2024-09-21T21:40:48.993 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7960000000000003e-06 2024-09-21T21:40:48.993 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.993 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.993 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:48.993 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.571867+0000", 2024-09-21T21:40:48.993 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1802000000000003e-05 2024-09-21T21:40:48.993 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.993 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.993 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:48.993 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.572003+0000", 2024-09-21T21:40:48.993 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013558600000000001 2024-09-21T21:40:48.993 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.993 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.994 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:48.994 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.576576+0000", 2024-09-21T21:40:48.994 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.004573405 2024-09-21T21:40:48.994 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:48.994 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:48.994 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:48.994 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.994 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.994 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4417.0:323 3.9 3:914d0620:::rbd_data.11214be7c411.0000000000000050:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:48.994 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.571952+0000", 2024-09-21T21:40:48.994 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 5.372544832, 2024-09-21T21:40:48.994 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.004747472, 2024-09-21T21:40:48.994 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:48.994 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:48.995 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:40:48.995 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:48.995 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4417", 2024-09-21T21:40:48.995 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:48.995 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 323 2024-09-21T21:40:48.995 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.995 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:48.995 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.995 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:48.995 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.571952+0000", 2024-09-21T21:40:48.995 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:48.995 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.995 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.995 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:48.996 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.571952+0000", 2024-09-21T21:40:48.996 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:48.996 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.996 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.996 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:48.996 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.571953+0000", 2024-09-21T21:40:48.996 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.017e-06 2024-09-21T21:40:48.996 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.996 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.996 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:48.996 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.571958+0000", 2024-09-21T21:40:48.996 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7939999999999999e-06 2024-09-21T21:40:48.996 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.996 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.997 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:48.997 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.571970+0000", 2024-09-21T21:40:48.997 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1250000000000001e-05 2024-09-21T21:40:48.997 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.997 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.997 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:48.997 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.571977+0000", 2024-09-21T21:40:48.997 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2690000000000002e-06 2024-09-21T21:40:48.997 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.997 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.997 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:48.997 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.572038+0000", 2024-09-21T21:40:48.997 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0868000000000001e-05 2024-09-21T21:40:48.997 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.997 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.998 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:48.998 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.572160+0000", 2024-09-21T21:40:48.998 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000122633 2024-09-21T21:40:48.998 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.998 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.998 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:48.998 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.576699+0000", 2024-09-21T21:40:48.998 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0045386410000000004 2024-09-21T21:40:48.998 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:48.998 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:48.998 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:48.998 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.998 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.998 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4417.0:347 3.3 3:c6040d19:::rbd_data.11214be7c411.000000000000004e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:48.998 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.576836+0000", 2024-09-21T21:40:48.999 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 5.3676604359999995, 2024-09-21T21:40:48.999 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0058561289999999998, 2024-09-21T21:40:48.999 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:48.999 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:48.999 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:48.999 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:48.999 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4417", 2024-09-21T21:40:48.999 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:48.999 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 347 2024-09-21T21:40:48.999 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:48.999 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:48.999 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:48.999 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:48.999 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.576836+0000", 2024-09-21T21:40:48.999 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:49.000 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.000 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.000 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:49.000 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.576836+0000", 2024-09-21T21:40:49.000 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:49.000 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.000 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.000 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:49.000 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.576837+0000", 2024-09-21T21:40:49.000 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.44e-07 2024-09-21T21:40:49.000 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.000 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.000 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:49.000 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.576840+0000", 2024-09-21T21:40:49.000 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1949999999999999e-06 2024-09-21T21:40:49.001 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.001 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.001 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:49.001 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.576847+0000", 2024-09-21T21:40:49.001 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1040000000000001e-06 2024-09-21T21:40:49.001 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.001 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.001 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:49.001 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.576851+0000", 2024-09-21T21:40:49.001 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.053e-06 2024-09-21T21:40:49.001 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.001 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.001 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:49.001 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.576893+0000", 2024-09-21T21:40:49.001 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1906000000000002e-05 2024-09-21T21:40:49.002 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.002 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.002 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:49.002 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.576956+0000", 2024-09-21T21:40:49.002 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3417999999999995e-05 2024-09-21T21:40:49.002 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.002 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.002 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:40:49.002 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.577175+0000", 2024-09-21T21:40:49.002 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000218381 2024-09-21T21:40:49.002 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.002 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.002 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:49.003 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.579572+0000", 2024-09-21T21:40:49.003 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0023974629999999999 2024-09-21T21:40:49.003 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.003 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.003 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:49.003 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.582549+0000", 2024-09-21T21:40:49.003 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0029770389999999999 2024-09-21T21:40:49.003 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.003 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.003 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:49.003 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.582650+0000", 2024-09-21T21:40:49.003 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010091599999999999 2024-09-21T21:40:49.003 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.003 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.004 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:49.004 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.582692+0000", 2024-09-21T21:40:49.004 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2009999999999999e-05 2024-09-21T21:40:49.004 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:49.004 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:49.004 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:49.004 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.004 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.004 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4417.0:349 3.9 3:914d0620:::rbd_data.11214be7c411.0000000000000050:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:49.004 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.576967+0000", 2024-09-21T21:40:49.004 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 5.3675295199999997, 2024-09-21T21:40:49.004 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0057303759999999997, 2024-09-21T21:40:49.004 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:49.005 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:49.005 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:49.005 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:49.005 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4417", 2024-09-21T21:40:49.005 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:49.005 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 349 2024-09-21T21:40:49.005 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.005 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:49.005 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.005 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:49.005 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.576967+0000", 2024-09-21T21:40:49.005 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:49.005 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.005 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.005 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:49.006 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.576967+0000", 2024-09-21T21:40:49.006 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:49.006 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.006 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.006 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:49.006 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.576968+0000", 2024-09-21T21:40:49.006 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.37e-07 2024-09-21T21:40:49.006 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.006 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.006 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:49.006 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.576972+0000", 2024-09-21T21:40:49.006 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9580000000000001e-06 2024-09-21T21:40:49.006 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.006 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.006 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:49.007 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.576981+0000", 2024-09-21T21:40:49.007 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9930000000000001e-06 2024-09-21T21:40:49.007 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.007 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.007 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:49.007 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.576987+0000", 2024-09-21T21:40:49.007 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7789999999999998e-06 2024-09-21T21:40:49.007 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.007 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.007 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:49.007 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.577032+0000", 2024-09-21T21:40:49.007 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4409999999999997e-05 2024-09-21T21:40:49.007 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.007 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.008 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:49.008 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.577125+0000", 2024-09-21T21:40:49.008 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3690000000000006e-05 2024-09-21T21:40:49.008 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.008 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.008 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 2", 2024-09-21T21:40:49.008 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.577359+0000", 2024-09-21T21:40:49.008 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00023399400000000001 2024-09-21T21:40:49.008 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.008 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.008 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:49.008 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.582518+0000", 2024-09-21T21:40:49.008 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0051590530000000002 2024-09-21T21:40:49.008 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.009 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.009 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:49.009 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.582556+0000", 2024-09-21T21:40:49.009 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7908999999999999e-05 2024-09-21T21:40:49.009 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.009 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.009 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:49.009 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.582646+0000", 2024-09-21T21:40:49.009 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9448000000000001e-05 2024-09-21T21:40:49.009 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.009 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.009 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:49.009 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.582697+0000", 2024-09-21T21:40:49.009 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1405e-05 2024-09-21T21:40:49.009 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:49.010 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:49.010 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:49.010 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.010 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.010 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(client.4417.0:344 3.1b e26/21 3:d9520152:::rbd_data.11214be7c411.000000000000004d:head v 26'11, mlcod=26'6)", 2024-09-21T21:40:49.010 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.577197+0000", 2024-09-21T21:40:49.010 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 5.3672994369999998, 2024-09-21T21:40:49.010 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0052244559999999997, 2024-09-21T21:40:49.010 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:49.010 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:49.010 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:49.010 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:49.010 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.010 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:49.011 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.577197+0000", 2024-09-21T21:40:49.011 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:49.011 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.011 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.011 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:49.011 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.577197+0000", 2024-09-21T21:40:49.011 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:49.011 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.011 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.011 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:49.011 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.577198+0000", 2024-09-21T21:40:49.011 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5599999999999995e-07 2024-09-21T21:40:49.011 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.011 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.012 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:49.012 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.577202+0000", 2024-09-21T21:40:49.012 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0249999999999996e-06 2024-09-21T21:40:49.012 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.012 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.012 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:49.012 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.577209+0000", 2024-09-21T21:40:49.012 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.216e-06 2024-09-21T21:40:49.012 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.012 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.012 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:49.012 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.577213+0000", 2024-09-21T21:40:49.012 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3649999999999997e-06 2024-09-21T21:40:49.013 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.013 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.013 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:49.013 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.577253+0000", 2024-09-21T21:40:49.013 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9597000000000001e-05 2024-09-21T21:40:49.013 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.013 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.013 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:49.013 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.577271+0000", 2024-09-21T21:40:49.013 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7978e-05 2024-09-21T21:40:49.013 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.013 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.013 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:49.013 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.582313+0000", 2024-09-21T21:40:49.014 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0050418449999999997 2024-09-21T21:40:49.014 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.014 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.014 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:49.014 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.582421+0000", 2024-09-21T21:40:49.014 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000108874 2024-09-21T21:40:49.014 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:49.014 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:49.014 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:49.014 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.014 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.014 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4417.0:354 3.1a 3:5c433628:::rbd_data.11214be7c411.0000000000000056:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:49.014 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.577895+0000", 2024-09-21T21:40:49.014 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 5.3666011889999998, 2024-09-21T21:40:49.015 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0047106719999999999, 2024-09-21T21:40:49.015 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:49.015 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:49.015 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:40:49.015 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:49.015 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4417", 2024-09-21T21:40:49.015 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:49.015 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 354 2024-09-21T21:40:49.015 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.015 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:49.015 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.015 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:49.015 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.577895+0000", 2024-09-21T21:40:49.015 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:49.016 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.016 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.016 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:49.016 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.577895+0000", 2024-09-21T21:40:49.016 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:49.016 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.016 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.016 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:49.016 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.577896+0000", 2024-09-21T21:40:49.016 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7899999999999998e-07 2024-09-21T21:40:49.016 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.016 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.016 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:49.016 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.577900+0000", 2024-09-21T21:40:49.017 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1180000000000002e-06 2024-09-21T21:40:49.017 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.017 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.017 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:49.017 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.577908+0000", 2024-09-21T21:40:49.017 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2460000000000003e-06 2024-09-21T21:40:49.017 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.017 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.017 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:49.017 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.577920+0000", 2024-09-21T21:40:49.017 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1425e-05 2024-09-21T21:40:49.017 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.017 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.017 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:49.018 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.577962+0000", 2024-09-21T21:40:49.018 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1885000000000003e-05 2024-09-21T21:40:49.018 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.018 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.018 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:49.018 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.578040+0000", 2024-09-21T21:40:49.018 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8274000000000003e-05 2024-09-21T21:40:49.018 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.018 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.018 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:49.018 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.582606+0000", 2024-09-21T21:40:49.018 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0045660450000000003 2024-09-21T21:40:49.018 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:49.018 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:49.019 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:49.019 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.019 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.019 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4417.0:361 3.9 3:96657d9e:::rbd_data.11214be7c411.0000000000000058:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:49.019 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.579658+0000", 2024-09-21T21:40:49.019 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 5.3648383170000002, 2024-09-21T21:40:49.019 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0053728680000000003, 2024-09-21T21:40:49.019 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:49.019 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:49.019 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:40:49.019 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:49.019 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4417", 2024-09-21T21:40:49.019 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:49.019 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 361 2024-09-21T21:40:49.020 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.020 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:49.020 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.020 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:49.020 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.579658+0000", 2024-09-21T21:40:49.020 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:49.020 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.020 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.020 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:49.020 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.579658+0000", 2024-09-21T21:40:49.020 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:49.020 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.020 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.020 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:49.020 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.579659+0000", 2024-09-21T21:40:49.021 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7700000000000004e-07 2024-09-21T21:40:49.021 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.021 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.021 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:49.021 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.579664+0000", 2024-09-21T21:40:49.021 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2379999999999997e-06 2024-09-21T21:40:49.021 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.021 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.021 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:49.021 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.579673+0000", 2024-09-21T21:40:49.021 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3079999999999997e-06 2024-09-21T21:40:49.021 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.021 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.021 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:49.022 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.579689+0000", 2024-09-21T21:40:49.022 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.522e-05 2024-09-21T21:40:49.022 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.022 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.022 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:49.022 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.579741+0000", 2024-09-21T21:40:49.022 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2329e-05 2024-09-21T21:40:49.022 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.022 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.022 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:49.022 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.579839+0000", 2024-09-21T21:40:49.022 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.8172999999999997e-05 2024-09-21T21:40:49.022 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.023 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.023 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:49.023 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.585031+0000", 2024-09-21T21:40:49.023 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0051919230000000002 2024-09-21T21:40:49.023 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:49.023 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:49.023 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:49.023 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.023 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.023 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4417.0:438 3.9 3:9170f860:::rbd_data.11214be7c411.0000000000000067:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:49.023 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.590245+0000", 2024-09-21T21:40:49.023 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 5.354251348, 2024-09-21T21:40:49.023 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0050158700000000004, 2024-09-21T21:40:49.023 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:49.023 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:49.024 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:49.024 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:49.024 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4417", 2024-09-21T21:40:49.024 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:49.024 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 438 2024-09-21T21:40:49.024 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.024 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:49.024 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.024 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:49.024 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.590245+0000", 2024-09-21T21:40:49.024 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:49.024 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.024 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.024 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:49.025 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.590245+0000", 2024-09-21T21:40:49.025 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:49.025 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.025 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.025 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:49.025 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.590246+0000", 2024-09-21T21:40:49.025 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.44e-07 2024-09-21T21:40:49.025 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.025 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.025 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:49.025 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.590249+0000", 2024-09-21T21:40:49.025 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.811e-06 2024-09-21T21:40:49.025 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.025 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.025 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:49.026 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.590258+0000", 2024-09-21T21:40:49.026 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2349999999999999e-06 2024-09-21T21:40:49.026 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.026 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.026 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:49.026 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.590262+0000", 2024-09-21T21:40:49.026 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.775e-06 2024-09-21T21:40:49.026 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.026 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.026 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:49.026 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.590304+0000", 2024-09-21T21:40:49.026 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1910999999999999e-05 2024-09-21T21:40:49.026 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.026 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.027 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:49.027 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.590372+0000", 2024-09-21T21:40:49.027 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7961999999999997e-05 2024-09-21T21:40:49.027 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.027 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.027 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 2", 2024-09-21T21:40:49.027 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.590579+0000", 2024-09-21T21:40:49.027 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00020714400000000001 2024-09-21T21:40:49.027 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.027 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.027 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:49.027 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.593958+0000", 2024-09-21T21:40:49.027 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0033788949999999998 2024-09-21T21:40:49.027 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.027 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.028 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:49.028 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.595177+0000", 2024-09-21T21:40:49.028 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012186230000000001 2024-09-21T21:40:49.028 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.028 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.028 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:49.028 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.595224+0000", 2024-09-21T21:40:49.028 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7312999999999999e-05 2024-09-21T21:40:49.028 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.028 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.028 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:49.028 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.595261+0000", 2024-09-21T21:40:49.028 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6656999999999998e-05 2024-09-21T21:40:49.028 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:49.029 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:49.029 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:49.029 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.029 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.029 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4417.0:457 3.3 3:c632f711:::rbd_data.11214be7c411.000000000000006f:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:49.029 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.593047+0000", 2024-09-21T21:40:49.029 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 5.3514497819999995, 2024-09-21T21:40:49.029 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0079810079999999995, 2024-09-21T21:40:49.029 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:49.029 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:49.029 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:40:49.029 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:49.029 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4417", 2024-09-21T21:40:49.029 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:49.029 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 457 2024-09-21T21:40:49.030 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.030 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:49.030 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.030 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:49.030 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.593047+0000", 2024-09-21T21:40:49.030 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:49.030 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.030 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.030 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:49.030 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.593047+0000", 2024-09-21T21:40:49.030 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:49.030 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.030 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.030 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:49.031 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.593047+0000", 2024-09-21T21:40:49.031 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8199999999999998e-07 2024-09-21T21:40:49.031 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.031 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.031 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:49.031 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.593052+0000", 2024-09-21T21:40:49.031 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0000000000000004e-06 2024-09-21T21:40:49.031 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.031 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.031 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:49.031 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.593066+0000", 2024-09-21T21:40:49.031 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3651e-05 2024-09-21T21:40:49.031 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.031 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.032 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:49.032 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.593074+0000", 2024-09-21T21:40:49.032 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6389999999999992e-06 2024-09-21T21:40:49.032 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.032 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.032 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:49.032 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.593149+0000", 2024-09-21T21:40:49.032 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4914999999999997e-05 2024-09-21T21:40:49.032 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.032 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.032 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:49.032 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.593286+0000", 2024-09-21T21:40:49.032 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013712800000000001 2024-09-21T21:40:49.032 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.033 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.033 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:49.033 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.601028+0000", 2024-09-21T21:40:49.033 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0077417930000000003 2024-09-21T21:40:49.033 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:49.033 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:49.033 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:49.033 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.033 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.033 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4417.0:471 3.9 3:9188e235:::rbd_data.11214be7c411.0000000000000071:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:49.033 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.597888+0000", 2024-09-21T21:40:49.033 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 5.346608679, 2024-09-21T21:40:49.033 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0061533930000000001, 2024-09-21T21:40:49.033 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:49.033 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:49.034 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:40:49.034 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:49.034 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4417", 2024-09-21T21:40:49.034 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:49.034 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 471 2024-09-21T21:40:49.034 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.034 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:49.034 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.034 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:49.034 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.597888+0000", 2024-09-21T21:40:49.034 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:49.034 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.034 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.034 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:49.035 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.597888+0000", 2024-09-21T21:40:49.035 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:49.035 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.035 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.035 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:49.035 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.597889+0000", 2024-09-21T21:40:49.035 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5199999999999995e-07 2024-09-21T21:40:49.035 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.035 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.035 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:49.035 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.597893+0000", 2024-09-21T21:40:49.035 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2189999999999998e-06 2024-09-21T21:40:49.035 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.036 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.036 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:49.036 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.597903+0000", 2024-09-21T21:40:49.036 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.8030000000000008e-06 2024-09-21T21:40:49.036 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.036 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.036 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:49.036 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.597908+0000", 2024-09-21T21:40:49.036 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3009999999999997e-06 2024-09-21T21:40:49.036 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.036 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.036 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:49.036 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.597948+0000", 2024-09-21T21:40:49.036 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9808999999999998e-05 2024-09-21T21:40:49.036 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.037 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.037 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:49.037 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.598015+0000", 2024-09-21T21:40:49.037 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7446000000000004e-05 2024-09-21T21:40:49.037 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.037 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.037 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:49.037 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.604041+0000", 2024-09-21T21:40:49.037 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0060258630000000002 2024-09-21T21:40:49.037 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:49.037 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:49.037 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:49.037 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.037 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.037 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4417.0:517 3.1a 3:5aa0c892:::rbd_data.11214be7c411.000000000000007a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:49.038 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.608399+0000", 2024-09-21T21:40:49.038 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 5.3360969159999998, 2024-09-21T21:40:49.038 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0076266900000000002, 2024-09-21T21:40:49.038 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:49.038 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:49.038 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:49.038 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:49.038 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4417", 2024-09-21T21:40:49.038 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:49.038 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 517 2024-09-21T21:40:49.039 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.039 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:49.039 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.039 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:49.039 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.608399+0000", 2024-09-21T21:40:49.039 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:49.039 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.039 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.039 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:49.039 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.608399+0000", 2024-09-21T21:40:49.039 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:49.039 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.039 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.039 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:49.040 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.608400+0000", 2024-09-21T21:40:49.040 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6899999999999996e-07 2024-09-21T21:40:49.040 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.040 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.040 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:49.040 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.608404+0000", 2024-09-21T21:40:49.040 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6959999999999998e-06 2024-09-21T21:40:49.040 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.040 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.040 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:49.040 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.608412+0000", 2024-09-21T21:40:49.040 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2660000000000001e-06 2024-09-21T21:40:49.040 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.040 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.041 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:49.041 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.608420+0000", 2024-09-21T21:40:49.041 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4789999999999999e-06 2024-09-21T21:40:49.041 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.041 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.041 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:49.041 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.608465+0000", 2024-09-21T21:40:49.041 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4886999999999999e-05 2024-09-21T21:40:49.041 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.041 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.041 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:49.041 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.608516+0000", 2024-09-21T21:40:49.041 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1693999999999997e-05 2024-09-21T21:40:49.041 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.042 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.042 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2024-09-21T21:40:49.042 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.608626+0000", 2024-09-21T21:40:49.042 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010976600000000001 2024-09-21T21:40:49.042 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.042 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.042 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:49.042 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.608962+0000", 2024-09-21T21:40:49.042 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00033614300000000001 2024-09-21T21:40:49.042 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.042 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.042 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:49.042 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.615969+0000", 2024-09-21T21:40:49.042 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0070068509999999997 2024-09-21T21:40:49.042 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.043 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.043 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:49.043 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.616002+0000", 2024-09-21T21:40:49.043 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3243e-05 2024-09-21T21:40:49.043 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.043 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.043 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:49.043 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.616026+0000", 2024-09-21T21:40:49.043 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3796000000000001e-05 2024-09-21T21:40:49.043 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:49.043 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:49.043 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:49.043 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.043 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.043 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(client.4471.0:1191 3.a e28/21 3:50c18da3:::rbd_object_map.1121bf4f5ab3:head v 28'139, mlcod=28'135)", 2024-09-21T21:40:49.044 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:48.416347+0000", 2024-09-21T21:40:49.044 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.52814940700000002, 2024-09-21T21:40:49.044 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0055930069999999997, 2024-09-21T21:40:49.044 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:49.044 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:49.044 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:49.044 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:49.044 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.044 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:49.044 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.416347+0000", 2024-09-21T21:40:49.044 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:49.044 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.044 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.044 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:49.045 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.416347+0000", 2024-09-21T21:40:49.045 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:49.045 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.045 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.045 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:49.045 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.416348+0000", 2024-09-21T21:40:49.045 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9100000000000004e-07 2024-09-21T21:40:49.045 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.045 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.045 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:49.045 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.416353+0000", 2024-09-21T21:40:49.045 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5289999999999999e-06 2024-09-21T21:40:49.045 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.045 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.046 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:49.046 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.416362+0000", 2024-09-21T21:40:49.046 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3200000000000006e-06 2024-09-21T21:40:49.046 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.046 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.046 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:49.046 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.416369+0000", 2024-09-21T21:40:49.046 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2890000000000003e-06 2024-09-21T21:40:49.046 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.046 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.046 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:49.046 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.416431+0000", 2024-09-21T21:40:49.046 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2189999999999999e-05 2024-09-21T21:40:49.046 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.046 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.047 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:49.047 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.416448+0000", 2024-09-21T21:40:49.047 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6929e-05 2024-09-21T21:40:49.047 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.047 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.047 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:49.047 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.421901+0000", 2024-09-21T21:40:49.047 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0054528909999999996 2024-09-21T21:40:49.047 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.047 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.047 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:49.047 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.421940+0000", 2024-09-21T21:40:49.047 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9267999999999997e-05 2024-09-21T21:40:49.047 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:49.048 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:49.048 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:49.048 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.048 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.048 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(client.4471.0:1192 3.a e28/21 3:50c18da3:::rbd_object_map.1121bf4f5ab3:head v 28'140, mlcod=28'136)", 2024-09-21T21:40:49.048 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:48.417340+0000", 2024-09-21T21:40:49.048 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.52715614600000005, 2024-09-21T21:40:49.048 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0046842389999999998, 2024-09-21T21:40:49.048 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:49.048 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:49.048 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:49.048 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:49.048 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.048 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:49.048 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.417340+0000", 2024-09-21T21:40:49.049 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:49.049 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.049 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.049 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:49.049 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.417340+0000", 2024-09-21T21:40:49.049 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:49.049 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.049 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.049 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:49.049 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.417341+0000", 2024-09-21T21:40:49.049 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0699999999999996e-07 2024-09-21T21:40:49.049 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.049 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.049 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:49.050 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.417347+0000", 2024-09-21T21:40:49.050 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1639999999999998e-06 2024-09-21T21:40:49.050 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.050 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.050 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:49.050 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.417370+0000", 2024-09-21T21:40:49.050 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2526000000000001e-05 2024-09-21T21:40:49.050 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.050 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.050 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:49.050 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.417377+0000", 2024-09-21T21:40:49.050 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9199999999999998e-06 2024-09-21T21:40:49.050 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.050 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.051 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:49.051 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.417430+0000", 2024-09-21T21:40:49.051 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3060999999999999e-05 2024-09-21T21:40:49.051 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.051 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.051 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:49.051 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.417448+0000", 2024-09-21T21:40:49.051 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7836e-05 2024-09-21T21:40:49.051 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.051 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.051 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:49.051 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.421943+0000", 2024-09-21T21:40:49.051 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0044949639999999997 2024-09-21T21:40:49.051 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.051 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.052 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:49.052 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.422024+0000", 2024-09-21T21:40:49.052 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1960999999999997e-05 2024-09-21T21:40:49.052 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:49.052 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:49.052 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:49.052 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:49.052 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:49.052 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:40:49.053 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:40:49.052+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:40:49.054 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T21:40:49.154 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:40:49.153+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:40:49.155 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:40:49.156 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:40:49.156 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-21T21:40:49.156 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:40:49.164 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T21:40:49.255 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:40:49.254+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:40:49.307 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:40:49.307 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:40:49.307 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:40:49.307 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:40:49.307 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:40:49.315 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T21:40:49.356 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:40:49.355+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:40:49.456 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:40:49.455+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:40:49.460 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:40:49.460 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-21T21:40:49.460 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-21T21:40:49.461 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:40:49.461 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.461 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4214.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T21:40:49.461 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:25.374665+0000", 2024-09-21T21:40:49.461 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 24.083580853000001, 2024-09-21T21:40:49.461 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0065415279999999996, 2024-09-21T21:40:49.461 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:49.461 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:49.461 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:49.461 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:49.461 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4214", 2024-09-21T21:40:49.461 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.81:0/3228106259", 2024-09-21T21:40:49.461 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1 2024-09-21T21:40:49.461 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.461 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:49.462 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.462 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:49.462 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:25.374665+0000", 2024-09-21T21:40:49.462 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:49.462 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.462 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.462 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:49.462 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:25.374665+0000", 2024-09-21T21:40:49.462 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:49.462 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.462 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.462 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:49.462 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:25.374668+0000", 2024-09-21T21:40:49.462 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4460000000000001e-06 2024-09-21T21:40:49.463 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.463 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.463 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:49.463 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:25.374674+0000", 2024-09-21T21:40:49.463 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1189999999999996e-06 2024-09-21T21:40:49.463 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.463 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.463 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:49.463 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:25.374683+0000", 2024-09-21T21:40:49.463 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.4700000000000008e-06 2024-09-21T21:40:49.463 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.463 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.463 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:49.463 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:25.374691+0000", 2024-09-21T21:40:49.463 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1140000000000006e-06 2024-09-21T21:40:49.464 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.464 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.464 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:49.464 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:25.374743+0000", 2024-09-21T21:40:49.464 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1669000000000003e-05 2024-09-21T21:40:49.464 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.464 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.464 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:49.464 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:25.374873+0000", 2024-09-21T21:40:49.464 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012997400000000001 2024-09-21T21:40:49.464 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.464 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.464 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 2", 2024-09-21T21:40:49.464 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:25.374973+0000", 2024-09-21T21:40:49.464 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.9769e-05 2024-09-21T21:40:49.465 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.465 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.465 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:49.465 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:25.375534+0000", 2024-09-21T21:40:49.465 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00056094100000000002 2024-09-21T21:40:49.465 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.465 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.465 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:49.465 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:25.381121+0000", 2024-09-21T21:40:49.465 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0055874490000000004 2024-09-21T21:40:49.465 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.465 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.465 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:49.465 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:25.381173+0000", 2024-09-21T21:40:49.466 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2447000000000001e-05 2024-09-21T21:40:49.466 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.466 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.466 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:49.466 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:25.381207+0000", 2024-09-21T21:40:49.466 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3130000000000003e-05 2024-09-21T21:40:49.466 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:49.466 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:49.466 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:49.466 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.466 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.466 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4417.0:168 3.5 3:a0cf71c3:::rbd_data.11214be7c411.000000000000002b:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:49.466 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.542298+0000", 2024-09-21T21:40:49.466 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 5.9159474950000002, 2024-09-21T21:40:49.467 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00558012, 2024-09-21T21:40:49.467 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:49.467 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:49.467 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:40:49.467 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:49.467 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4417", 2024-09-21T21:40:49.467 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:49.467 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 168 2024-09-21T21:40:49.467 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.467 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:49.467 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.467 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:49.467 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.542298+0000", 2024-09-21T21:40:49.467 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:49.468 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.468 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.468 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:49.468 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.542298+0000", 2024-09-21T21:40:49.468 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:49.468 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.468 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.468 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:49.468 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.542299+0000", 2024-09-21T21:40:49.468 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.004e-06 2024-09-21T21:40:49.468 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.468 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.468 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:49.468 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.542304+0000", 2024-09-21T21:40:49.468 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7339999999999999e-06 2024-09-21T21:40:49.469 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.469 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.469 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:49.469 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.542320+0000", 2024-09-21T21:40:49.469 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5843000000000001e-05 2024-09-21T21:40:49.469 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.469 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.469 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:49.469 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.542328+0000", 2024-09-21T21:40:49.469 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3399999999999998e-06 2024-09-21T21:40:49.469 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.469 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.470 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:49.470 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.542377+0000", 2024-09-21T21:40:49.470 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8600999999999998e-05 2024-09-21T21:40:49.470 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.470 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.470 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:49.470 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.542450+0000", 2024-09-21T21:40:49.470 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2776999999999998e-05 2024-09-21T21:40:49.470 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.470 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.470 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:49.470 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.547879+0000", 2024-09-21T21:40:49.470 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0054288210000000003 2024-09-21T21:40:49.470 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:49.471 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:49.471 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:49.471 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.471 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.471 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4417.0:296 3.5 3:a7194652:::rbd_data.11214be7c411.0000000000000041:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:49.471 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.566820+0000", 2024-09-21T21:40:49.471 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 5.8914257939999999, 2024-09-21T21:40:49.471 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0049682729999999996, 2024-09-21T21:40:49.471 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:49.471 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:49.471 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:49.471 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:49.471 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4417", 2024-09-21T21:40:49.471 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:49.471 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 296 2024-09-21T21:40:49.472 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.472 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:49.472 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.472 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:49.472 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.566820+0000", 2024-09-21T21:40:49.472 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:49.472 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.472 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.472 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:49.472 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.566820+0000", 2024-09-21T21:40:49.472 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:49.472 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.472 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.472 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:49.472 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.566821+0000", 2024-09-21T21:40:49.472 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9100000000000004e-07 2024-09-21T21:40:49.473 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.473 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.473 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:49.473 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.566825+0000", 2024-09-21T21:40:49.473 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9700000000000001e-06 2024-09-21T21:40:49.473 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.473 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.473 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:49.473 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.566833+0000", 2024-09-21T21:40:49.473 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1340000000000004e-06 2024-09-21T21:40:49.473 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.473 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.473 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:49.473 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.566838+0000", 2024-09-21T21:40:49.473 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4349999999999996e-06 2024-09-21T21:40:49.473 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.474 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.474 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:49.474 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.566888+0000", 2024-09-21T21:40:49.474 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0078999999999999e-05 2024-09-21T21:40:49.474 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.474 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.474 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:49.474 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.566941+0000", 2024-09-21T21:40:49.474 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2930000000000003e-05 2024-09-21T21:40:49.474 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.474 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.474 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 2", 2024-09-21T21:40:49.474 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.567101+0000", 2024-09-21T21:40:49.475 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00015960700000000001 2024-09-21T21:40:49.475 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.475 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.475 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:49.475 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.568575+0000", 2024-09-21T21:40:49.475 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0014742869999999999 2024-09-21T21:40:49.475 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.475 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.475 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:49.475 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.571726+0000", 2024-09-21T21:40:49.475 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0031510280000000002 2024-09-21T21:40:49.475 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.475 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.475 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:49.476 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.571764+0000", 2024-09-21T21:40:49.476 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7938999999999997e-05 2024-09-21T21:40:49.476 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.476 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.476 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:49.476 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.571788+0000", 2024-09-21T21:40:49.476 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4273e-05 2024-09-21T21:40:49.476 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:49.476 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:49.476 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:49.476 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.476 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.476 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4417.0:297 3.5 3:a7d35bab:::rbd_data.11214be7c411.0000000000000044:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:49.476 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.566899+0000", 2024-09-21T21:40:49.477 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 5.891346618, 2024-09-21T21:40:49.477 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.004944665, 2024-09-21T21:40:49.477 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:49.477 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:49.477 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:49.477 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:49.477 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4417", 2024-09-21T21:40:49.477 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:49.477 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 297 2024-09-21T21:40:49.477 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.477 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:49.477 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.477 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:49.477 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.566899+0000", 2024-09-21T21:40:49.478 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:49.478 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.478 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.478 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:49.478 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.566899+0000", 2024-09-21T21:40:49.478 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:49.478 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.478 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.478 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:49.478 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.566900+0000", 2024-09-21T21:40:49.478 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.8700000000000004e-07 2024-09-21T21:40:49.478 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.478 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.479 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:49.479 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.566907+0000", 2024-09-21T21:40:49.479 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4540000000000004e-06 2024-09-21T21:40:49.479 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.479 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.479 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:49.479 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.566918+0000", 2024-09-21T21:40:49.479 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0902e-05 2024-09-21T21:40:49.479 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.479 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.479 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:49.479 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.566925+0000", 2024-09-21T21:40:49.479 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4150000000000002e-06 2024-09-21T21:40:49.479 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.480 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.480 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:49.480 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.567351+0000", 2024-09-21T21:40:49.480 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00042601999999999999 2024-09-21T21:40:49.480 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.480 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.480 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:49.480 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.567426+0000", 2024-09-21T21:40:49.480 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5150999999999999e-05 2024-09-21T21:40:49.480 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.480 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.480 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 2", 2024-09-21T21:40:49.480 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.567630+0000", 2024-09-21T21:40:49.480 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00020410299999999999 2024-09-21T21:40:49.480 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.481 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.481 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:49.481 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.568608+0000", 2024-09-21T21:40:49.481 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00097757500000000006 2024-09-21T21:40:49.481 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.481 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.481 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:49.481 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.571794+0000", 2024-09-21T21:40:49.481 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0031861250000000002 2024-09-21T21:40:49.481 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.481 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.481 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:49.481 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.571817+0000", 2024-09-21T21:40:49.481 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2731e-05 2024-09-21T21:40:49.482 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.482 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.482 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:49.482 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.571844+0000", 2024-09-21T21:40:49.482 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7202000000000001e-05 2024-09-21T21:40:49.482 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:49.482 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:49.482 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:49.482 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.482 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.482 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(client.4417.0:306 3.13 e26/21 3:cb0f117d:::rbd_data.11214be7c411.0000000000000042:head v 26'6, mlcod=26'4)", 2024-09-21T21:40:49.482 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.569366+0000", 2024-09-21T21:40:49.482 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 5.8888799509999998, 2024-09-21T21:40:49.483 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0049738539999999998, 2024-09-21T21:40:49.483 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:49.483 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:49.483 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:49.483 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:49.483 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.483 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:49.483 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.569366+0000", 2024-09-21T21:40:49.483 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:49.483 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.483 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.483 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:49.483 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.569366+0000", 2024-09-21T21:40:49.483 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:49.484 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.484 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.484 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:49.484 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.569367+0000", 2024-09-21T21:40:49.484 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2699999999999999e-07 2024-09-21T21:40:49.484 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.484 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.484 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:49.484 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.569374+0000", 2024-09-21T21:40:49.484 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8739999999999997e-06 2024-09-21T21:40:49.484 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.484 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.484 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:49.484 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.569385+0000", 2024-09-21T21:40:49.485 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1829e-05 2024-09-21T21:40:49.485 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.485 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.485 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:49.485 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.569393+0000", 2024-09-21T21:40:49.485 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2470000000000002e-06 2024-09-21T21:40:49.485 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.485 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.485 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:49.485 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.569436+0000", 2024-09-21T21:40:49.485 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3093999999999998e-05 2024-09-21T21:40:49.485 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.485 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.485 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:49.486 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.569455+0000", 2024-09-21T21:40:49.486 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9211e-05 2024-09-21T21:40:49.486 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.486 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.486 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:49.486 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.574298+0000", 2024-09-21T21:40:49.486 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048428619999999999 2024-09-21T21:40:49.486 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.486 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.486 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:49.486 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.574340+0000", 2024-09-21T21:40:49.486 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2009999999999999e-05 2024-09-21T21:40:49.486 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:49.486 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:49.487 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:49.487 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.487 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.487 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4417.0:346 3.10 3:0db2ee8b:::rbd_data.11214be7c411.0000000000000055:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:49.487 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.576844+0000", 2024-09-21T21:40:49.487 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 5.8814021140000001, 2024-09-21T21:40:49.487 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0059696539999999996, 2024-09-21T21:40:49.487 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:49.487 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:49.487 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:40:49.487 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:49.487 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4417", 2024-09-21T21:40:49.487 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:49.487 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 346 2024-09-21T21:40:49.488 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.488 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:49.488 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.488 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:49.488 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.576844+0000", 2024-09-21T21:40:49.488 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:49.488 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.488 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.488 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:49.488 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.576844+0000", 2024-09-21T21:40:49.488 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:49.488 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.488 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.488 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:49.489 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.576845+0000", 2024-09-21T21:40:49.489 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7199999999999997e-07 2024-09-21T21:40:49.489 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.489 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.489 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:49.489 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.576848+0000", 2024-09-21T21:40:49.489 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7220000000000001e-06 2024-09-21T21:40:49.489 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.489 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.489 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:49.489 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.576858+0000", 2024-09-21T21:40:49.489 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0064999999999999e-05 2024-09-21T21:40:49.490 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.490 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.490 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:49.490 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.576873+0000", 2024-09-21T21:40:49.490 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4065e-05 2024-09-21T21:40:49.490 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.490 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.490 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:49.490 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.576918+0000", 2024-09-21T21:40:49.490 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5911999999999997e-05 2024-09-21T21:40:49.490 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.490 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.490 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:49.491 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.577286+0000", 2024-09-21T21:40:49.491 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00036716299999999999 2024-09-21T21:40:49.491 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.491 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.491 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:49.491 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.582813+0000", 2024-09-21T21:40:49.491 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0055278549999999999 2024-09-21T21:40:49.491 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:49.491 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:49.491 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:49.491 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.491 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.491 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(client.4417.0:352 3.2 e26/21 3:411f17a4:::rbd_data.11214be7c411.0000000000000051:head v 26'9, mlcod=26'6)", 2024-09-21T21:40:49.491 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.578366+0000", 2024-09-21T21:40:49.492 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 5.879880268, 2024-09-21T21:40:49.492 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0045439570000000004, 2024-09-21T21:40:49.492 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:49.492 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:49.492 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:49.492 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:49.492 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.492 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:49.492 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.578366+0000", 2024-09-21T21:40:49.492 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:49.492 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.492 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.492 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:49.492 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.578366+0000", 2024-09-21T21:40:49.493 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:49.493 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.493 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.493 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:49.493 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.578366+0000", 2024-09-21T21:40:49.493 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6000000000000004e-07 2024-09-21T21:40:49.493 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.493 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.493 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:49.493 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.578371+0000", 2024-09-21T21:40:49.493 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.254e-06 2024-09-21T21:40:49.493 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.493 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.494 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:49.494 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.578380+0000", 2024-09-21T21:40:49.494 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5760000000000006e-06 2024-09-21T21:40:49.494 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.494 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.494 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:49.494 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.578385+0000", 2024-09-21T21:40:49.494 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1939999999999997e-06 2024-09-21T21:40:49.494 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.494 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.494 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:49.494 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.578422+0000", 2024-09-21T21:40:49.494 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6723000000000001e-05 2024-09-21T21:40:49.495 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.495 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.495 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:49.495 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.578435+0000", 2024-09-21T21:40:49.495 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2690999999999999e-05 2024-09-21T21:40:49.495 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.495 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.495 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:49.495 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.582885+0000", 2024-09-21T21:40:49.495 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0044504360000000003 2024-09-21T21:40:49.495 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.495 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.495 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:49.496 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.582910+0000", 2024-09-21T21:40:49.496 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4522999999999999e-05 2024-09-21T21:40:49.496 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:49.496 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:49.496 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:49.496 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.496 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.496 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4417.0:356 3.c 3:30e79c3d:::rbd_data.11214be7c411.0000000000000057:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:49.496 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.578474+0000", 2024-09-21T21:40:49.496 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 5.8797714159999996, 2024-09-21T21:40:49.496 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0043452539999999998, 2024-09-21T21:40:49.496 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:49.496 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:49.496 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:40:49.497 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:49.497 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4417", 2024-09-21T21:40:49.497 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:49.497 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 356 2024-09-21T21:40:49.497 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.497 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:49.497 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.497 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:49.497 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.578474+0000", 2024-09-21T21:40:49.497 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:49.497 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.497 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.497 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:49.497 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.578474+0000", 2024-09-21T21:40:49.498 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:49.498 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.498 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.498 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:49.498 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.578475+0000", 2024-09-21T21:40:49.498 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6599999999999996e-07 2024-09-21T21:40:49.498 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.498 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.498 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:49.498 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.578479+0000", 2024-09-21T21:40:49.498 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9580000000000001e-06 2024-09-21T21:40:49.498 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.498 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.499 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:49.499 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.578490+0000", 2024-09-21T21:40:49.499 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0531e-05 2024-09-21T21:40:49.499 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.499 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.499 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:49.499 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.578495+0000", 2024-09-21T21:40:49.499 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3920000000000002e-06 2024-09-21T21:40:49.499 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.499 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.499 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:49.499 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.578545+0000", 2024-09-21T21:40:49.499 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9756999999999998e-05 2024-09-21T21:40:49.499 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.500 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.500 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:49.500 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.578640+0000", 2024-09-21T21:40:49.500 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5098000000000003e-05 2024-09-21T21:40:49.500 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.500 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.500 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:49.500 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.582820+0000", 2024-09-21T21:40:49.500 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0041798520000000004 2024-09-21T21:40:49.500 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:49.500 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:49.500 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:49.500 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.500 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.500 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4417.0:363 3.5 3:a6f0f50c:::rbd_data.11214be7c411.0000000000000059:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:49.501 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.579970+0000", 2024-09-21T21:40:49.501 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 5.8782758790000003, 2024-09-21T21:40:49.501 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.005099307, 2024-09-21T21:40:49.501 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:49.501 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:49.501 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:40:49.501 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:49.501 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4417", 2024-09-21T21:40:49.501 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:49.501 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 363 2024-09-21T21:40:49.501 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.501 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:49.501 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.502 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:49.502 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.579970+0000", 2024-09-21T21:40:49.502 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:49.502 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.502 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.502 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:49.502 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.579970+0000", 2024-09-21T21:40:49.502 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:49.502 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.502 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.502 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:49.502 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.579971+0000", 2024-09-21T21:40:49.502 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.61e-07 2024-09-21T21:40:49.503 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.503 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.503 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:49.503 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.579975+0000", 2024-09-21T21:40:49.503 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8670000000000004e-06 2024-09-21T21:40:49.503 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.503 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.503 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:49.503 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.579983+0000", 2024-09-21T21:40:49.503 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8100000000000004e-06 2024-09-21T21:40:49.503 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.503 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.503 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:49.503 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.579989+0000", 2024-09-21T21:40:49.503 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1370000000000001e-06 2024-09-21T21:40:49.504 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.504 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.504 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:49.504 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.580026+0000", 2024-09-21T21:40:49.504 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7421e-05 2024-09-21T21:40:49.504 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.504 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.504 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:49.504 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.580117+0000", 2024-09-21T21:40:49.504 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.0686999999999994e-05 2024-09-21T21:40:49.504 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.504 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.504 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:49.505 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.585069+0000", 2024-09-21T21:40:49.505 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0049526240000000001 2024-09-21T21:40:49.505 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:49.505 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:49.505 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:49.505 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.505 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.505 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4417.0:414 3.1c 3:3fa5f3c6:::rbd_data.11214be7c411.0000000000000066:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:49.505 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.587704+0000", 2024-09-21T21:40:49.505 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 5.8705419589999996, 2024-09-21T21:40:49.505 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0051267359999999998, 2024-09-21T21:40:49.505 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:49.505 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:49.505 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:40:49.506 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:49.506 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4417", 2024-09-21T21:40:49.506 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:49.506 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 414 2024-09-21T21:40:49.506 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.506 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:49.506 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.506 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:49.506 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.587704+0000", 2024-09-21T21:40:49.506 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:49.506 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.506 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.506 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:49.507 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.587704+0000", 2024-09-21T21:40:49.507 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:49.507 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.507 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.507 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:49.507 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.587705+0000", 2024-09-21T21:40:49.507 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5899999999999996e-07 2024-09-21T21:40:49.507 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.507 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.507 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:49.507 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.587709+0000", 2024-09-21T21:40:49.507 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9779999999999999e-06 2024-09-21T21:40:49.507 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.508 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.508 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:49.508 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.587717+0000", 2024-09-21T21:40:49.508 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8349999999999993e-06 2024-09-21T21:40:49.508 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.508 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.508 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:49.508 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.587723+0000", 2024-09-21T21:40:49.508 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1270000000000002e-06 2024-09-21T21:40:49.508 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.508 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.508 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:49.508 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.587779+0000", 2024-09-21T21:40:49.508 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6202000000000002e-05 2024-09-21T21:40:49.509 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.509 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.509 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:49.509 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.587879+0000", 2024-09-21T21:40:49.509 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.9839000000000004e-05 2024-09-21T21:40:49.509 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.509 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.509 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:49.509 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.592831+0000", 2024-09-21T21:40:49.509 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0049520959999999996 2024-09-21T21:40:49.509 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:49.509 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:49.509 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:49.509 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.509 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.509 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4417.0:439 3.1d 3:b9e79477:::rbd_data.11214be7c411.000000000000006d:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:49.510 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.590273+0000", 2024-09-21T21:40:49.510 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 5.8679730340000003, 2024-09-21T21:40:49.510 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0051529849999999997, 2024-09-21T21:40:49.510 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:49.510 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:49.510 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:40:49.510 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:49.510 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4417", 2024-09-21T21:40:49.510 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:49.510 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 439 2024-09-21T21:40:49.510 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.510 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:49.511 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.511 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:49.511 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.590273+0000", 2024-09-21T21:40:49.511 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:49.511 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.511 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.511 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:49.511 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.590273+0000", 2024-09-21T21:40:49.511 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:49.511 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.511 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.511 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:49.511 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.590273+0000", 2024-09-21T21:40:49.511 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1600000000000001e-07 2024-09-21T21:40:49.511 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.512 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.512 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:49.512 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.590277+0000", 2024-09-21T21:40:49.512 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4340000000000001e-06 2024-09-21T21:40:49.512 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.512 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.512 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:49.512 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.590284+0000", 2024-09-21T21:40:49.512 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5469999999999999e-06 2024-09-21T21:40:49.512 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.512 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.512 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:49.512 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.590297+0000", 2024-09-21T21:40:49.513 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3030999999999999e-05 2024-09-21T21:40:49.513 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.513 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.513 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:49.513 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.590348+0000", 2024-09-21T21:40:49.513 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0228000000000001e-05 2024-09-21T21:40:49.513 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.513 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.513 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:49.513 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.590465+0000", 2024-09-21T21:40:49.513 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000116847 2024-09-21T21:40:49.513 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.513 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.513 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:49.513 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.595426+0000", 2024-09-21T21:40:49.514 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.004961282 2024-09-21T21:40:49.514 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:49.514 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:49.514 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:49.514 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.514 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.514 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4417.0:440 3.5 3:a6f99a77:::rbd_data.11214be7c411.0000000000000068:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:49.514 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.590317+0000", 2024-09-21T21:40:49.514 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 5.8679287640000002, 2024-09-21T21:40:49.514 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.008514967, 2024-09-21T21:40:49.514 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:49.514 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:49.514 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:49.514 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:49.514 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4417", 2024-09-21T21:40:49.515 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:49.515 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 440 2024-09-21T21:40:49.515 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.515 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:49.515 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.515 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:49.515 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.590317+0000", 2024-09-21T21:40:49.515 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:49.515 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.515 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.515 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:49.515 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.590317+0000", 2024-09-21T21:40:49.515 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:49.515 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.516 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.516 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:49.516 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.590318+0000", 2024-09-21T21:40:49.516 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3500000000000002e-07 2024-09-21T21:40:49.516 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.516 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.516 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:49.516 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.590320+0000", 2024-09-21T21:40:49.516 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4650000000000001e-06 2024-09-21T21:40:49.516 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.516 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.516 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:49.516 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.590327+0000", 2024-09-21T21:40:49.516 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7220000000000002e-06 2024-09-21T21:40:49.516 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.517 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.517 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:49.517 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.590330+0000", 2024-09-21T21:40:49.517 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5439999999999999e-06 2024-09-21T21:40:49.517 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.517 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.517 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:49.517 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.590371+0000", 2024-09-21T21:40:49.517 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1183000000000002e-05 2024-09-21T21:40:49.517 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.517 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.517 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:49.517 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.590454+0000", 2024-09-21T21:40:49.517 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2379999999999994e-05 2024-09-21T21:40:49.517 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.518 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.518 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 2", 2024-09-21T21:40:49.518 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.590666+0000", 2024-09-21T21:40:49.518 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00021181999999999999 2024-09-21T21:40:49.518 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.518 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.518 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:49.518 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.595067+0000", 2024-09-21T21:40:49.518 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0044010830000000001 2024-09-21T21:40:49.518 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.518 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.518 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:49.518 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.598764+0000", 2024-09-21T21:40:49.518 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0036973140000000002 2024-09-21T21:40:49.518 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.519 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.519 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:49.519 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.598797+0000", 2024-09-21T21:40:49.519 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2740000000000002e-05 2024-09-21T21:40:49.519 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.519 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.519 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:49.519 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.598832+0000", 2024-09-21T21:40:49.519 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5281000000000003e-05 2024-09-21T21:40:49.519 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:49.519 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:49.519 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:49.519 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.519 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.519 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "pg_update_log_missing(3.17 epoch 26/21 rep_tid 46 entries 26'11 (0'0) error 3:ea5ebfbf:::rbd_data.11214be7c411.000000000000006c:head by client.4417.0:436 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 26'8)", 2024-09-21T21:40:49.519 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.590562+0000", 2024-09-21T21:40:49.520 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 5.8676840690000001, 2024-09-21T21:40:49.520 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00451251, 2024-09-21T21:40:49.520 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:49.520 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:49.520 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2024-09-21T21:40:49.520 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:49.520 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.520 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:49.520 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.590562+0000", 2024-09-21T21:40:49.520 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:49.520 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.520 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.520 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:49.520 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.590562+0000", 2024-09-21T21:40:49.521 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:49.521 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.521 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.521 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:49.521 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.590562+0000", 2024-09-21T21:40:49.521 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8899999999999999e-07 2024-09-21T21:40:49.521 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.521 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.521 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:49.521 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.590574+0000", 2024-09-21T21:40:49.521 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1522000000000001e-05 2024-09-21T21:40:49.521 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.521 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.521 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:49.522 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.590588+0000", 2024-09-21T21:40:49.522 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4494999999999999e-05 2024-09-21T21:40:49.522 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.522 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.522 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:49.522 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.590602+0000", 2024-09-21T21:40:49.522 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3064000000000001e-05 2024-09-21T21:40:49.522 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.522 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.522 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:49.522 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.590672+0000", 2024-09-21T21:40:49.522 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0517000000000003e-05 2024-09-21T21:40:49.522 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.522 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.522 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:49.522 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.595074+0000", 2024-09-21T21:40:49.523 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0044023229999999997 2024-09-21T21:40:49.523 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:49.523 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:49.523 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:49.523 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.523 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.523 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4417.0:455 3.1c 3:3fa5f3c6:::rbd_data.11214be7c411.0000000000000066:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:49.523 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.592971+0000", 2024-09-21T21:40:49.523 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 5.8652749140000005, 2024-09-21T21:40:49.523 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0049042829999999997, 2024-09-21T21:40:49.523 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:49.523 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:49.523 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:49.524 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:49.524 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4417", 2024-09-21T21:40:49.524 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:49.524 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 455 2024-09-21T21:40:49.524 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.524 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:49.524 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.524 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:49.524 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.592971+0000", 2024-09-21T21:40:49.524 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:49.524 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.524 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.524 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:49.524 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.592971+0000", 2024-09-21T21:40:49.524 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:49.524 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.525 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.525 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:49.525 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.592972+0000", 2024-09-21T21:40:49.525 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8000000000000005e-07 2024-09-21T21:40:49.525 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.525 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.525 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:49.525 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.592976+0000", 2024-09-21T21:40:49.525 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.121e-06 2024-09-21T21:40:49.525 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.525 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.525 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:49.525 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.592984+0000", 2024-09-21T21:40:49.525 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3270000000000001e-06 2024-09-21T21:40:49.525 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.526 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.526 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:49.526 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.592991+0000", 2024-09-21T21:40:49.526 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4130000000000003e-06 2024-09-21T21:40:49.526 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.526 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.526 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:49.526 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.593043+0000", 2024-09-21T21:40:49.526 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2546999999999997e-05 2024-09-21T21:40:49.526 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.526 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.526 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:49.526 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.593115+0000", 2024-09-21T21:40:49.527 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1763999999999999e-05 2024-09-21T21:40:49.527 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.527 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.527 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 4", 2024-09-21T21:40:49.527 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.593303+0000", 2024-09-21T21:40:49.527 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000188211 2024-09-21T21:40:49.527 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.527 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.527 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:49.527 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.597457+0000", 2024-09-21T21:40:49.527 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0041535840000000001 2024-09-21T21:40:49.527 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.527 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.527 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:49.527 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.597789+0000", 2024-09-21T21:40:49.527 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00033188100000000002 2024-09-21T21:40:49.528 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.528 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.528 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:49.528 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.597848+0000", 2024-09-21T21:40:49.528 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9119000000000001e-05 2024-09-21T21:40:49.528 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.528 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.528 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:49.528 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.597875+0000", 2024-09-21T21:40:49.528 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7535999999999999e-05 2024-09-21T21:40:49.528 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:49.528 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:49.528 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:49.528 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.528 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.529 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "pg_update_log_missing(3.2 epoch 26/21 rep_tid 112 entries 26'12 (0'0) error 3:430d3f60:::rbd_data.11214be7c411.000000000000006e:head by client.4417.0:452 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 26'10)", 2024-09-21T21:40:49.529 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.593101+0000", 2024-09-21T21:40:49.529 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 5.8651448120000005, 2024-09-21T21:40:49.529 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0043794510000000004, 2024-09-21T21:40:49.529 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:49.529 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:49.529 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2024-09-21T21:40:49.529 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:49.529 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.529 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:49.529 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.593101+0000", 2024-09-21T21:40:49.529 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:49.529 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.529 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.529 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:49.529 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.593101+0000", 2024-09-21T21:40:49.530 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:49.530 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.530 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.530 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:49.530 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.593102+0000", 2024-09-21T21:40:49.530 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8500000000000001e-07 2024-09-21T21:40:49.530 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.530 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.530 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:49.530 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.593119+0000", 2024-09-21T21:40:49.530 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6912999999999999e-05 2024-09-21T21:40:49.530 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.530 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.530 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:49.530 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.593136+0000", 2024-09-21T21:40:49.531 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7198999999999999e-05 2024-09-21T21:40:49.531 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.531 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.531 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:49.531 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.593149+0000", 2024-09-21T21:40:49.531 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3192e-05 2024-09-21T21:40:49.531 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.531 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.531 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:49.531 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.593215+0000", 2024-09-21T21:40:49.531 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6023000000000001e-05 2024-09-21T21:40:49.531 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.531 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.531 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:49.532 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.597481+0000", 2024-09-21T21:40:49.532 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0042655389999999996 2024-09-21T21:40:49.532 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:49.532 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:49.532 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:49.532 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.532 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.532 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4417.0:469 3.1d 3:b9e79477:::rbd_data.11214be7c411.000000000000006d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:49.532 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.595683+0000", 2024-09-21T21:40:49.532 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 5.8625632220000004, 2024-09-21T21:40:49.532 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0044134409999999997, 2024-09-21T21:40:49.532 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:49.532 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:49.532 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:49.532 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:49.533 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4417", 2024-09-21T21:40:49.533 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:49.533 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 469 2024-09-21T21:40:49.533 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.533 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:49.533 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.533 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:49.533 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.595683+0000", 2024-09-21T21:40:49.533 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:49.533 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.533 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.533 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:49.533 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.595683+0000", 2024-09-21T21:40:49.533 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:49.534 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.534 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.534 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:49.534 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.595683+0000", 2024-09-21T21:40:49.534 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0800000000000004e-07 2024-09-21T21:40:49.534 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.534 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.534 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:49.534 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.595688+0000", 2024-09-21T21:40:49.534 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0239999999999997e-06 2024-09-21T21:40:49.534 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.534 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.534 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:49.534 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.595697+0000", 2024-09-21T21:40:49.535 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8060000000000001e-06 2024-09-21T21:40:49.535 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.535 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.535 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:49.535 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.595702+0000", 2024-09-21T21:40:49.535 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9919999999999998e-06 2024-09-21T21:40:49.535 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.535 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.535 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:49.535 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.595748+0000", 2024-09-21T21:40:49.535 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5636000000000001e-05 2024-09-21T21:40:49.535 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.535 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.535 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:49.535 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.595829+0000", 2024-09-21T21:40:49.536 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1021999999999998e-05 2024-09-21T21:40:49.536 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.536 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.536 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 4", 2024-09-21T21:40:49.536 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.596007+0000", 2024-09-21T21:40:49.536 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000177839 2024-09-21T21:40:49.536 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.536 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.536 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:49.536 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.597759+0000", 2024-09-21T21:40:49.536 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0017523389999999999 2024-09-21T21:40:49.536 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.536 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.536 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:49.536 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.600031+0000", 2024-09-21T21:40:49.537 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022723800000000001 2024-09-21T21:40:49.537 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.537 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.537 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:49.537 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.600063+0000", 2024-09-21T21:40:49.537 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1383000000000001e-05 2024-09-21T21:40:49.537 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.537 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.537 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:49.537 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.600096+0000", 2024-09-21T21:40:49.537 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3411999999999998e-05 2024-09-21T21:40:49.537 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:49.537 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:49.537 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:49.538 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.538 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.538 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4417.0:472 3.1c 3:393bd49d:::rbd_data.11214be7c411.0000000000000072:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:49.538 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.597902+0000", 2024-09-21T21:40:49.538 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 5.8603443070000001, 2024-09-21T21:40:49.538 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048015009999999997, 2024-09-21T21:40:49.538 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:49.538 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:49.538 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:40:49.538 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:49.538 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4417", 2024-09-21T21:40:49.538 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:49.538 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 472 2024-09-21T21:40:49.539 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.539 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:49.539 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.539 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:49.539 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.597902+0000", 2024-09-21T21:40:49.539 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:49.539 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.539 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.539 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:49.539 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.597902+0000", 2024-09-21T21:40:49.539 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:49.539 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.539 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.539 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:49.540 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.597902+0000", 2024-09-21T21:40:49.540 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3399999999999999e-07 2024-09-21T21:40:49.540 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.540 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.540 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:49.540 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.597905+0000", 2024-09-21T21:40:49.540 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1829999999999998e-06 2024-09-21T21:40:49.540 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.540 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.540 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:49.540 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.597915+0000", 2024-09-21T21:40:49.540 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.1379999999999996e-06 2024-09-21T21:40:49.540 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.540 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.540 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:49.540 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.597920+0000", 2024-09-21T21:40:49.541 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.972e-06 2024-09-21T21:40:49.541 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.541 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.541 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:49.541 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.597964+0000", 2024-09-21T21:40:49.541 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4611999999999999e-05 2024-09-21T21:40:49.541 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.541 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.541 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:49.541 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.598080+0000", 2024-09-21T21:40:49.541 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000115553 2024-09-21T21:40:49.541 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.541 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.541 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:49.541 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.602703+0000", 2024-09-21T21:40:49.542 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0046234090000000002 2024-09-21T21:40:49.542 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:49.542 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:49.542 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:49.542 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.542 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.542 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4417.0:479 3.1d 3:bcbf195b:::rbd_data.11214be7c411.0000000000000077:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:49.542 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.600396+0000", 2024-09-21T21:40:49.542 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 5.857849571, 2024-09-21T21:40:49.542 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0047349260000000004, 2024-09-21T21:40:49.542 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:49.542 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:49.542 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:40:49.542 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:49.542 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4417", 2024-09-21T21:40:49.543 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:49.543 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 479 2024-09-21T21:40:49.543 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.543 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:49.543 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.543 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:49.543 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.600396+0000", 2024-09-21T21:40:49.543 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:49.543 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.543 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.543 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:49.543 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.600396+0000", 2024-09-21T21:40:49.543 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:49.544 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.544 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.544 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:49.544 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.600397+0000", 2024-09-21T21:40:49.544 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9599999999999998e-07 2024-09-21T21:40:49.544 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.544 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.544 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:49.544 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.600402+0000", 2024-09-21T21:40:49.544 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7779999999999998e-06 2024-09-21T21:40:49.544 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.544 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.544 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:49.544 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.600414+0000", 2024-09-21T21:40:49.544 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2379e-05 2024-09-21T21:40:49.545 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.545 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.545 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:49.545 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.600425+0000", 2024-09-21T21:40:49.545 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0856000000000001e-05 2024-09-21T21:40:49.545 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.545 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.545 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:49.545 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.600464+0000", 2024-09-21T21:40:49.545 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8639000000000001e-05 2024-09-21T21:40:49.545 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.545 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.545 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:49.545 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.600537+0000", 2024-09-21T21:40:49.545 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2674000000000002e-05 2024-09-21T21:40:49.546 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.546 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.546 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:49.546 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.605131+0000", 2024-09-21T21:40:49.546 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0045947039999999998 2024-09-21T21:40:49.546 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:49.546 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:49.546 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:49.546 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.546 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.546 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:1199 3.16 3:6acc33a0:::rbd_data.1121bf4f5ab3.0000000000000066:head [set-alloc-hint object_size 1048576 write_size 1048576,write 38912~26624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-21T21:40:49.546 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:48.417047+0000", 2024-09-21T21:40:49.546 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 1.041199113, 2024-09-21T21:40:49.546 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.005078061, 2024-09-21T21:40:49.547 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:49.547 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:49.547 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:49.547 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:49.547 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2024-09-21T21:40:49.547 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1688278210", 2024-09-21T21:40:49.547 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1199 2024-09-21T21:40:49.547 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.547 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:49.547 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.547 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:49.547 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.417047+0000", 2024-09-21T21:40:49.547 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:49.547 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.547 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.548 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:49.548 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.417047+0000", 2024-09-21T21:40:49.548 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:49.548 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.548 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.548 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:49.548 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.417048+0000", 2024-09-21T21:40:49.548 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1880000000000001e-06 2024-09-21T21:40:49.548 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.548 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.548 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:49.548 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.417067+0000", 2024-09-21T21:40:49.548 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9490000000000001e-05 2024-09-21T21:40:49.548 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.548 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.548 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:49.549 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.417078+0000", 2024-09-21T21:40:49.549 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0533000000000001e-05 2024-09-21T21:40:49.549 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.549 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.549 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:49.549 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.417083+0000", 2024-09-21T21:40:49.549 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0370000000000002e-06 2024-09-21T21:40:49.549 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.549 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.549 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:49.549 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.417126+0000", 2024-09-21T21:40:49.549 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3281000000000001e-05 2024-09-21T21:40:49.549 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.549 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.549 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:49.550 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.417231+0000", 2024-09-21T21:40:49.550 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010446399999999999 2024-09-21T21:40:49.550 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.550 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.550 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:40:49.550 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.417361+0000", 2024-09-21T21:40:49.550 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013056100000000001 2024-09-21T21:40:49.550 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.550 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.550 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:49.550 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.421796+0000", 2024-09-21T21:40:49.550 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0044345849999999996 2024-09-21T21:40:49.550 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.550 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.550 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:49.551 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.422035+0000", 2024-09-21T21:40:49.551 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000239134 2024-09-21T21:40:49.551 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.551 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.551 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:49.551 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.422086+0000", 2024-09-21T21:40:49.551 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1424000000000001e-05 2024-09-21T21:40:49.551 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.551 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.551 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:49.551 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.422125+0000", 2024-09-21T21:40:49.551 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8364000000000001e-05 2024-09-21T21:40:49.551 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:49.551 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:49.551 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:49.551 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.552 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.552 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:1201 3.1d 3:bfe30718:::rbd_data.1121bf4f5ab3.000000000000003f:head [write 33280~32256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-21T21:40:49.552 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:48.417097+0000", 2024-09-21T21:40:49.552 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 1.0411493439999999, 2024-09-21T21:40:49.552 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0051236809999999997, 2024-09-21T21:40:49.552 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:49.552 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:49.552 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:49.552 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:49.552 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2024-09-21T21:40:49.552 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1688278210", 2024-09-21T21:40:49.552 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1201 2024-09-21T21:40:49.552 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.552 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:49.552 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.553 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:49.553 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.417097+0000", 2024-09-21T21:40:49.553 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:49.553 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.553 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.553 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:49.553 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.417097+0000", 2024-09-21T21:40:49.553 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:49.553 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.553 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.553 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:49.553 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.417097+0000", 2024-09-21T21:40:49.553 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5600000000000001e-07 2024-09-21T21:40:49.553 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.553 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.553 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:49.554 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.417139+0000", 2024-09-21T21:40:49.554 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2252000000000001e-05 2024-09-21T21:40:49.554 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.554 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.554 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:49.554 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.417148+0000", 2024-09-21T21:40:49.554 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.422e-06 2024-09-21T21:40:49.554 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.554 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.554 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:49.554 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.417154+0000", 2024-09-21T21:40:49.554 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0950000000000001e-06 2024-09-21T21:40:49.554 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.554 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.554 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:49.554 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.417218+0000", 2024-09-21T21:40:49.555 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4140000000000006e-05 2024-09-21T21:40:49.555 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.555 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.555 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:49.555 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.417288+0000", 2024-09-21T21:40:49.555 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0229999999999997e-05 2024-09-21T21:40:49.555 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.555 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.555 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 4", 2024-09-21T21:40:49.555 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.417402+0000", 2024-09-21T21:40:49.555 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00011437100000000001 2024-09-21T21:40:49.555 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.555 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.555 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:49.555 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.418167+0000", 2024-09-21T21:40:49.556 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00076478099999999997 2024-09-21T21:40:49.556 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.556 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.556 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:49.556 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.422138+0000", 2024-09-21T21:40:49.556 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0039707759999999996 2024-09-21T21:40:49.556 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.556 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.556 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:49.556 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.422186+0000", 2024-09-21T21:40:49.556 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7626000000000001e-05 2024-09-21T21:40:49.556 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:49.556 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:49.556 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:49.556 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.422220+0000", 2024-09-21T21:40:49.556 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4632000000000003e-05 2024-09-21T21:40:49.557 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:49.557 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:49.557 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:49.557 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:49.557 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:49.557 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:40:49.557 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T21:40:49.558 INFO:teuthology.orchestra.run.smithi164.stdout:truncating image image_client.0-clone4 from 0xd398e00 (overlap 0x23d400) to 0x203e00 2024-09-21T21:40:49.559 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:40:49.557+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:40:49.559 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T21:40:49.660 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:40:49.658+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:40:49.666 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:40:49.666 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:40:49.666 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 0 2024-09-21T21:40:49.666 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:40:49.675 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T21:40:49.680 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0: {} 2024-09-21T21:40:49.680 INFO:teuthology.orchestra.run.smithi081.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T21:40:49.697 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1: {} 2024-09-21T21:40:49.697 INFO:teuthology.orchestra.run.smithi081.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T21:40:49.714 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2: {} 2024-09-21T21:40:49.715 INFO:teuthology.orchestra.run.smithi081.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T21:40:49.731 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3: {} 2024-09-21T21:40:49.731 INFO:teuthology.orchestra.run.smithi081.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T21:40:49.748 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4: {} 2024-09-21T21:40:49.748 INFO:teuthology.orchestra.run.smithi081.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T21:40:49.761 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:40:49.760+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:40:49.764 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5: {} 2024-09-21T21:40:49.765 INFO:teuthology.orchestra.run.smithi081.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T21:40:49.784 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:49.784 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:49.794 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T21:40:49.862 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:40:49.861+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:40:49.938 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:49.938 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:49.947 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T21:40:49.992 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:40:49.991+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:40:50.048 INFO:teuthology.orchestra.run.smithi164.stdout:checking clone #3, image image_client.0-clone3 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent4 2024-09-21T21:40:50.059 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:50.059 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:50.064 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:40:50.062+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:40:50.070 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T21:40:50.165 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:40:50.163+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:40:50.249 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:50.249 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:50.258 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T21:40:50.300 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:40:50.298+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:40:50.331 INFO:teuthology.orchestra.run.smithi164.stdout:360 write 0x9843200 thru 0x98509e4 (0xd7e5 bytes) 2024-09-21T21:40:50.339 INFO:teuthology.orchestra.run.smithi164.stdout:361 trunc from 0xd398e00 to 0x6e6d600 2024-09-21T21:40:50.367 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:40:50.365+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:40:50.383 INFO:teuthology.orchestra.run.smithi164.stdout:362 write 0x77c7800 thru 0x77d08f7 (0x90f8 bytes) 2024-09-21T21:40:50.402 INFO:teuthology.orchestra.run.smithi164.stdout:366 write 0x7b75e00 thru 0x7b7be38 (0x6039 bytes) 2024-09-21T21:40:50.414 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:50.414 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:50.423 INFO:teuthology.orchestra.run.smithi164.stdout:367 write 0xb764000 thru 0xb76b694 (0x7695 bytes) 2024-09-21T21:40:50.424 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T21:40:50.425 INFO:teuthology.orchestra.run.smithi164.stdout:368 write 0x4824a00 thru 0x48296bb (0x4cbc bytes) 2024-09-21T21:40:50.425 INFO:teuthology.orchestra.run.smithi164.stdout:369 read 0x9104400 thru 0x911256f (0xe170 bytes) 2024-09-21T21:40:50.425 INFO:teuthology.orchestra.run.smithi164.stdout:371 read 0x5eb8a00 thru 0x5ec561f (0xcc20 bytes) 2024-09-21T21:40:50.425 INFO:teuthology.orchestra.run.smithi164.stdout:372 write 0x27ade00 thru 0x27bcbf2 (0xedf3 bytes) 2024-09-21T21:40:50.434 INFO:teuthology.orchestra.run.smithi164.stdout:375 trunc from 0xb76b800 to 0x3c83e00 2024-09-21T21:40:50.469 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:40:50.468+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:40:50.480 INFO:teuthology.orchestra.run.smithi164.stdout:376 trunc from 0x3c83e00 to 0x7ea2200 2024-09-21T21:40:50.499 INFO:teuthology.orchestra.run.smithi164.stdout:378 write 0x599f200 thru 0x59acbbd (0xd9be bytes) 2024-09-21T21:40:50.499 INFO:teuthology.orchestra.run.smithi164.stdout:379 read 0x206800 thru 0x2079f5 (0x11f6 bytes) 2024-09-21T21:40:50.519 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T21:40:50.522 INFO:teuthology.orchestra.run.smithi164.stdout:381 write 0x96ebc00 thru 0x96f871b (0xcb1c bytes) 2024-09-21T21:40:50.541 INFO:teuthology.orchestra.run.smithi164.stdout:382 write 0xb15e800 thru 0xb16113d (0x293e bytes) 2024-09-21T21:40:50.542 INFO:teuthology.orchestra.run.smithi164.stdout:385 write 0x3d1ce00 thru 0x3d256db (0x88dc bytes) 2024-09-21T21:40:50.549 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:50.549 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:50.562 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T21:40:50.566 INFO:teuthology.orchestra.run.smithi164.stdout:386 write 0xeddf600 thru 0xede4528 (0x4f29 bytes) 2024-09-21T21:40:50.569 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:40:50.567+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:40:50.573 INFO:teuthology.orchestra.run.smithi164.stdout:387 write 0xe1c4600 thru 0xe1d114a (0xcb4b bytes) 2024-09-21T21:40:50.573 INFO:teuthology.orchestra.run.smithi164.stdout:388 read 0xa595a00 thru 0xa59dc55 (0x8256 bytes) 2024-09-21T21:40:50.573 INFO:teuthology.orchestra.run.smithi164.stdout:389 punch from 0x98c2a00 to 0x98c5800, (0x2e00 bytes) 2024-09-21T21:40:50.575 INFO:teuthology.orchestra.run.smithi164.stdout:390 punch from 0x488ce00 to 0x4898200, (0xb400 bytes) 2024-09-21T21:40:50.589 INFO:teuthology.orchestra.run.smithi164.stdout:392 trunc from 0xede4600 to 0x33d0e00 2024-09-21T21:40:50.621 INFO:teuthology.orchestra.run.smithi164.stdout:395 read 0x560a00 thru 0x562d20 (0x2321 bytes) 2024-09-21T21:40:50.627 INFO:teuthology.orchestra.run.smithi164.stdout:396 read 0x1b97e00 thru 0x1ba4d89 (0xcf8a bytes) 2024-09-21T21:40:50.628 INFO:teuthology.orchestra.run.smithi164.stdout:397 read 0x1270a00 thru 0x1278d68 (0x8369 bytes) 2024-09-21T21:40:50.628 INFO:teuthology.orchestra.run.smithi164.stdout:401 read 0x2ae2200 thru 0x2ae5faf (0x3db0 bytes) 2024-09-21T21:40:50.628 INFO:teuthology.orchestra.run.smithi164.stdout:402 read 0x29dd400 thru 0x29de9ae (0x15af bytes) 2024-09-21T21:40:50.652 INFO:teuthology.orchestra.run.smithi164.stdout:404 write 0xdb32c00 thru 0xdb394d6 (0x68d7 bytes) 2024-09-21T21:40:50.657 INFO:teuthology.orchestra.run.smithi164.stdout:405 write 0x526b400 thru 0x527b3cd (0xffce bytes) 2024-09-21T21:40:50.657 INFO:teuthology.orchestra.run.smithi164.stdout:406 read 0xc23000 thru 0xc299e7 (0x69e8 bytes) 2024-09-21T21:40:50.659 INFO:teuthology.orchestra.run.smithi164.stdout:407 read 0x877800 thru 0x87e916 (0x7117 bytes) 2024-09-21T21:40:50.659 INFO:teuthology.orchestra.run.smithi164.stdout:408 write 0x91ebe00 thru 0x91ec7d3 (0x9d4 bytes) 2024-09-21T21:40:50.659 INFO:teuthology.orchestra.run.smithi164.stdout:409 read 0xd670800 thru 0xd680039 (0xf83a bytes) 2024-09-21T21:40:50.659 INFO:teuthology.orchestra.run.smithi164.stdout:410 read 0x387e200 thru 0x387f0cc (0xecd bytes) 2024-09-21T21:40:50.659 INFO:teuthology.orchestra.run.smithi164.stdout:411 write 0xaedac00 thru 0xaee29b0 (0x7db1 bytes) 2024-09-21T21:40:50.659 INFO:teuthology.orchestra.run.smithi164.stdout:412 read 0x1dc2400 thru 0x1dc6990 (0x4591 bytes) 2024-09-21T21:40:50.659 INFO:teuthology.orchestra.run.smithi164.stdout:414 write 0x2a4d400 thru 0x2a54d70 (0x7971 bytes) 2024-09-21T21:40:50.669 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:40:50.668+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:40:50.671 INFO:teuthology.orchestra.run.smithi164.stdout:415 trunc from 0xdb39600 to 0x364fa00 2024-09-21T21:40:50.709 INFO:teuthology.orchestra.run.smithi164.stdout:417 trunc from 0x364fa00 to 0x5244000 2024-09-21T21:40:50.723 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:50.723 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:50.729 INFO:teuthology.orchestra.run.smithi164.stdout:418 read 0x1465800 thru 0x1473f18 (0xe719 bytes) 2024-09-21T21:40:50.729 INFO:teuthology.orchestra.run.smithi164.stdout:419 read 0x2a62c00 thru 0x2a64a24 (0x1e25 bytes) 2024-09-21T21:40:50.733 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T21:40:50.749 INFO:teuthology.orchestra.run.smithi164.stdout:420 write 0x9782800 thru 0x978f873 (0xd074 bytes) 2024-09-21T21:40:50.750 INFO:teuthology.orchestra.run.smithi164.stdout:422 write 0x1f3aa00 thru 0x1f41a67 (0x7068 bytes) 2024-09-21T21:40:50.750 INFO:teuthology.orchestra.run.smithi164.stdout:423 punch from 0x2437000 to 0x2443800, (0xc800 bytes) 2024-09-21T21:40:50.756 INFO:teuthology.orchestra.run.smithi164.stdout:429 trunc from 0x978fa00 to 0xc72ae00 2024-09-21T21:40:50.767 INFO:teuthology.orchestra.run.smithi164.stdout:430 write 0xf65400 thru 0xf73db6 (0xe9b7 bytes) 2024-09-21T21:40:50.773 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:40:50.772+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:40:50.781 INFO:teuthology.orchestra.run.smithi164.stdout:431 trunc from 0xc72ae00 to 0x1989800 2024-09-21T21:40:50.842 INFO:teuthology.orchestra.run.smithi164.stdout:438 write 0x94d4a00 thru 0x94de1be (0x97bf bytes) 2024-09-21T21:40:50.862 INFO:teuthology.orchestra.run.smithi164.stdout:439 write 0x9fdd800 thru 0x9fde49d (0xc9e bytes) 2024-09-21T21:40:50.863 INFO:teuthology.orchestra.run.smithi164.stdout:443 write 0xf0200 thru 0xfd538 (0xd339 bytes) 2024-09-21T21:40:50.866 INFO:teuthology.orchestra.run.smithi164.stdout:445 read 0x648c600 thru 0x648fbe8 (0x35e9 bytes) 2024-09-21T21:40:50.866 INFO:teuthology.orchestra.run.smithi164.stdout:449 read 0x60fbc00 thru 0x6104e42 (0x9243 bytes) 2024-09-21T21:40:50.866 INFO:teuthology.orchestra.run.smithi164.stdout:450 punch from 0x5905000 to 0x5905c00, (0xc00 bytes) 2024-09-21T21:40:50.870 INFO:teuthology.orchestra.run.smithi164.stdout:452 flatten 2024-09-21T21:40:50.870 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:40:50.869+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:40:50.904 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:50.904 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:50.914 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T21:40:50.971 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:40:50.970+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:40:50.998 INFO:teuthology.orchestra.run.smithi164.stdout:453 write 0x69aa800 thru 0x69ae226 (0x3a27 bytes) 2024-09-21T21:40:51.016 INFO:teuthology.orchestra.run.smithi164.stdout:457 write 0xa96e600 thru 0xa96fdaf (0x17b0 bytes) 2024-09-21T21:40:51.017 INFO:teuthology.orchestra.run.smithi164.stdout:459 read 0x8873800 thru 0x8874d84 (0x1585 bytes) 2024-09-21T21:40:51.033 INFO:teuthology.orchestra.run.smithi164.stdout:461 write 0xba7aa00 thru 0xba818f8 (0x6ef9 bytes) 2024-09-21T21:40:51.034 INFO:teuthology.orchestra.run.smithi164.stdout:463 punch from 0x773c000 to 0x7740e00, (0x4e00 bytes) 2024-09-21T21:40:51.038 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:51.039 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:51.049 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T21:40:51.054 INFO:teuthology.orchestra.run.smithi164.stdout:464 write 0xd802a00 thru 0xd806b4a (0x414b bytes) 2024-09-21T21:40:51.055 INFO:teuthology.orchestra.run.smithi164.stdout:466 punch from 0x4873000 to 0x4876a00, (0x3a00 bytes) 2024-09-21T21:40:51.056 INFO:teuthology.orchestra.run.smithi164.stdout:467 write 0x52a7e00 thru 0x52af946 (0x7b47 bytes) 2024-09-21T21:40:51.056 INFO:teuthology.orchestra.run.smithi164.stdout:468 read 0x4d6b200 thru 0x4d70bf8 (0x59f9 bytes) 2024-09-21T21:40:51.057 INFO:teuthology.orchestra.run.smithi164.stdout:470 read 0xb6a1600 thru 0xb6af767 (0xe168 bytes) 2024-09-21T21:40:51.071 INFO:teuthology.orchestra.run.smithi164.stdout:471 trunc from 0xd806c00 to 0xe8dc00 2024-09-21T21:40:51.090 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:40:51.089+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:40:51.105 INFO:teuthology.orchestra.run.smithi164.stdout:476 punch from 0x6d2a00 to 0x6dac00, (0x8200 bytes) 2024-09-21T21:40:51.110 INFO:teuthology.orchestra.run.smithi164.stdout:479 read 0x54fa00 thru 0x55cff7 (0xd5f8 bytes) 2024-09-21T21:40:51.132 INFO:teuthology.orchestra.run.smithi164.stdout:483 write 0xcb3f600 thru 0xcb448e8 (0x52e9 bytes) 2024-09-21T21:40:51.134 INFO:teuthology.orchestra.run.smithi164.stdout:485 read 0x5f1d000 thru 0x5f20166 (0x3167 bytes) 2024-09-21T21:40:51.134 INFO:teuthology.orchestra.run.smithi164.stdout:486 punch from 0x907a000 to 0x9080000, (0x6000 bytes) 2024-09-21T21:40:51.135 INFO:teuthology.orchestra.run.smithi164.stdout:489 punch from 0x9ac8400 to 0x9ac8a00, (0x600 bytes) 2024-09-21T21:40:51.135 INFO:teuthology.orchestra.run.smithi164.stdout:490 write 0x8554a00 thru 0x8555f6f (0x1570 bytes) 2024-09-21T21:40:51.135 INFO:teuthology.orchestra.run.smithi164.stdout:491 read 0x111a000 thru 0x111fafe (0x5aff bytes) 2024-09-21T21:40:51.146 INFO:teuthology.orchestra.run.smithi164.stdout:493 trunc from 0xcb44a00 to 0x36fcc00 2024-09-21T21:40:51.164 INFO:teuthology.orchestra.run.smithi164.stdout:496 read 0x3567a00 thru 0x35691c9 (0x17ca bytes) 2024-09-21T21:40:51.165 INFO:teuthology.orchestra.run.smithi164.stdout:497 punch from 0x1fe2000 to 0x1fe5a00, (0x3a00 bytes) 2024-09-21T21:40:51.170 INFO:teuthology.orchestra.run.smithi164.stdout:498 read 0x4df000 thru 0x4df07b (0x7c bytes) 2024-09-21T21:40:51.173 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:40:51.171+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:40:51.186 INFO:teuthology.orchestra.run.smithi164.stdout:500 write 0x63e5400 thru 0x63e9ed1 (0x4ad2 bytes) 2024-09-21T21:40:51.187 INFO:teuthology.orchestra.run.smithi164.stdout:501 punch from 0x292aa00 to 0x293a200, (0xf800 bytes) 2024-09-21T21:40:51.191 INFO:teuthology.orchestra.run.smithi164.stdout:503 trunc from 0x63ea000 to 0x45a3a00 2024-09-21T21:40:51.203 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:51.203 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:51.210 INFO:teuthology.orchestra.run.smithi164.stdout:504 read 0x2e98a00 thru 0x2e9b7f4 (0x2df5 bytes) 2024-09-21T21:40:51.211 INFO:teuthology.orchestra.run.smithi164.stdout:505 write 0x25eae00 thru 0x25f45eb (0x97ec bytes) 2024-09-21T21:40:51.211 INFO:teuthology.orchestra.run.smithi164.stdout:506 read 0x285e000 thru 0x286797b (0x997c bytes) 2024-09-21T21:40:51.211 INFO:teuthology.orchestra.run.smithi164.stdout:507 punch from 0x1ccfc00 to 0x1cd7200, (0x7600 bytes) 2024-09-21T21:40:51.213 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T21:40:51.213 INFO:teuthology.orchestra.run.smithi164.stdout:509 read 0x273da00 thru 0x273f46e (0x1a6f bytes) 2024-09-21T21:40:51.214 INFO:teuthology.orchestra.run.smithi164.stdout:510 read 0x3580000 thru 0x35835f9 (0x35fa bytes) 2024-09-21T21:40:51.226 INFO:teuthology.orchestra.run.smithi164.stdout:513 write 0x8605400 thru 0x86112cd (0xbece bytes) 2024-09-21T21:40:51.245 INFO:teuthology.orchestra.run.smithi164.stdout:519 write 0xd1d4800 thru 0xd1e10e1 (0xc8e2 bytes) 2024-09-21T21:40:51.252 INFO:teuthology.orchestra.run.smithi164.stdout:520 punch from 0x7212e00 to 0x7214200, (0x1400 bytes) 2024-09-21T21:40:51.254 INFO:teuthology.orchestra.run.smithi164.stdout:521 write 0xc2d800 thru 0xc2f1fa (0x19fb bytes) 2024-09-21T21:40:51.255 INFO:teuthology.orchestra.run.smithi164.stdout:523 punch from 0x626800 to 0x634e00, (0xe600 bytes) 2024-09-21T21:40:51.263 INFO:teuthology.orchestra.run.smithi164.stdout:525 trunc from 0xd1e1200 to 0x61d6800 2024-09-21T21:40:51.274 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:40:51.272+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:40:51.283 INFO:teuthology.orchestra.run.smithi164.stdout:527 trunc from 0x61d6800 to 0x2a91e00 2024-09-21T21:40:51.304 INFO:teuthology.orchestra.run.smithi164.stdout:528 trunc from 0x2a91e00 to 0xd31fe00 2024-09-21T21:40:51.321 INFO:teuthology.orchestra.run.smithi164.stdout:530 trunc from 0xd31fe00 to 0xb1f3200 2024-09-21T21:40:51.330 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:51.330 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:51.336 INFO:teuthology.orchestra.run.smithi164.stdout:531 write 0x198800 thru 0x1a4f29 (0xc72a bytes) 2024-09-21T21:40:51.336 INFO:teuthology.orchestra.run.smithi164.stdout:532 write 0xa5cea00 thru 0xa5cf4c9 (0xaca bytes) 2024-09-21T21:40:51.337 INFO:teuthology.orchestra.run.smithi164.stdout:533 punch from 0x68f6400 to 0x6902800, (0xc400 bytes) 2024-09-21T21:40:51.339 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T21:40:51.347 INFO:teuthology.orchestra.run.smithi164.stdout:534 trunc from 0xb1f3200 to 0x2328e00 2024-09-21T21:40:51.374 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:40:51.373+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:40:51.383 INFO:teuthology.orchestra.run.smithi164.stdout:536 write 0x775f600 thru 0x77627da (0x31db bytes) 2024-09-21T21:40:51.384 INFO:teuthology.orchestra.run.smithi164.stdout:537 punch from 0x3303400 to 0x3310c00, (0xd800 bytes) 2024-09-21T21:40:51.385 INFO:teuthology.orchestra.run.smithi164.stdout:538 read 0x5566c00 thru 0x5568aa3 (0x1ea4 bytes) 2024-09-21T21:40:51.402 INFO:teuthology.orchestra.run.smithi164.stdout:540 write 0xbfc6c00 thru 0xbfc9e4d (0x324e bytes) 2024-09-21T21:40:51.403 INFO:teuthology.orchestra.run.smithi164.stdout:541 punch from 0x25c6800 to 0x25c8e00, (0x2600 bytes) 2024-09-21T21:40:51.422 INFO:teuthology.orchestra.run.smithi164.stdout:542 write 0xe22c000 thru 0xe22d3d4 (0x13d5 bytes) 2024-09-21T21:40:51.424 INFO:teuthology.orchestra.run.smithi164.stdout:543 read 0x834a400 thru 0x834dcbd (0x38be bytes) 2024-09-21T21:40:51.424 INFO:teuthology.orchestra.run.smithi164.stdout:547 read 0x4cd0400 thru 0x4cdd106 (0xcd07 bytes) 2024-09-21T21:40:51.427 INFO:teuthology.orchestra.run.smithi164.stdout:549 read 0x3425e00 thru 0x3430787 (0xa988 bytes) 2024-09-21T21:40:51.427 INFO:teuthology.orchestra.run.smithi164.stdout:550 punch from 0x21eea00 to 0x21f9600, (0xac00 bytes) 2024-09-21T21:40:51.427 INFO:teuthology.orchestra.run.smithi164.stdout:551 write 0x769c000 thru 0x76a803c (0xc03d bytes) 2024-09-21T21:40:51.439 INFO:teuthology.orchestra.run.smithi164.stdout:552 trunc from 0xe22d400 to 0x2a96e00 2024-09-21T21:40:51.458 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:51.458 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:51.467 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T21:40:51.470 INFO:teuthology.orchestra.run.smithi164.stdout:555 write 0x354e000 thru 0x354eb65 (0xb66 bytes) 2024-09-21T21:40:51.479 INFO:teuthology.orchestra.run.smithi164.stdout:557 trunc from 0x354ec00 to 0x98b8800 2024-09-21T21:40:51.486 INFO:teuthology.orchestra.run.smithi081.stderr:nodeep-scrub is set 2024-09-21T21:40:51.498 INFO:teuthology.orchestra.run.smithi164.stdout:558 clone 6 order 24 su 65536 sc 3 2024-09-21T21:40:51.511 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:40:51.510+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:40:51.573 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:51.574 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:51.576 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:40:51.574+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:40:51.582 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T21:40:51.620 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-21T21:40:51.620 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T21:40:51.620 INFO:tasks.thrashosds.thrasher:inject_pause on osd.5 2024-09-21T21:40:51.620 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-21T21:40:51.620 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-21T21:40:51.621 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok config set filestore_inject_stall 3 2024-09-21T21:40:51.676 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:40:51.675+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:40:51.727 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:40:51.727 INFO:teuthology.orchestra.run.smithi122.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-21T21:40:51.727 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:40:51.728 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:51.728 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:51.737 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T21:40:51.777 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T21:40:51.780 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:40:51.779+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:40:51.840 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:51.840 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:51.848 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T21:40:51.878 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:40:51.877+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:40:51.896 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0: {} 2024-09-21T21:40:51.896 INFO:teuthology.orchestra.run.smithi081.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T21:40:51.913 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1: {} 2024-09-21T21:40:51.913 INFO:teuthology.orchestra.run.smithi081.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T21:40:51.929 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2: {} 2024-09-21T21:40:51.929 INFO:teuthology.orchestra.run.smithi081.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T21:40:51.945 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3: {} 2024-09-21T21:40:51.946 INFO:teuthology.orchestra.run.smithi081.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T21:40:51.962 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4: {} 2024-09-21T21:40:51.962 INFO:teuthology.orchestra.run.smithi081.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T21:40:51.978 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5: {} 2024-09-21T21:40:51.978 INFO:teuthology.orchestra.run.smithi081.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T21:40:51.979 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:40:51.978+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:40:51.992 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:40:51.992 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:40:51.992 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:40:51.992 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:40:51.992 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:40:52.000 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T21:40:52.080 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:40:52.078+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:40:52.144 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:40:52.145 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-21T21:40:52.145 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-21T21:40:52.145 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:40:52.145 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.145 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4214.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-21T21:40:52.145 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:25.374665+0000", 2024-09-21T21:40:52.145 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 26.767676155, 2024-09-21T21:40:52.145 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0065415279999999996, 2024-09-21T21:40:52.145 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:52.145 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:52.145 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:52.145 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:52.146 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4214", 2024-09-21T21:40:52.146 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.81:0/3228106259", 2024-09-21T21:40:52.146 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1 2024-09-21T21:40:52.146 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.146 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:52.146 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.146 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:52.146 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:25.374665+0000", 2024-09-21T21:40:52.146 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:52.146 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.146 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.146 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:52.146 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:25.374665+0000", 2024-09-21T21:40:52.147 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:52.147 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.147 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.147 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:52.147 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:25.374668+0000", 2024-09-21T21:40:52.147 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4460000000000001e-06 2024-09-21T21:40:52.147 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.147 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.147 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:52.147 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:25.374674+0000", 2024-09-21T21:40:52.147 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1189999999999996e-06 2024-09-21T21:40:52.147 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.147 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.147 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:52.148 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:25.374683+0000", 2024-09-21T21:40:52.148 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.4700000000000008e-06 2024-09-21T21:40:52.148 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.148 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.148 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:52.148 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:25.374691+0000", 2024-09-21T21:40:52.148 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1140000000000006e-06 2024-09-21T21:40:52.148 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.148 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.148 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:52.148 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:25.374743+0000", 2024-09-21T21:40:52.148 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1669000000000003e-05 2024-09-21T21:40:52.148 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.148 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.148 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:52.149 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:25.374873+0000", 2024-09-21T21:40:52.149 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012997400000000001 2024-09-21T21:40:52.149 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.149 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.149 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 2", 2024-09-21T21:40:52.149 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:25.374973+0000", 2024-09-21T21:40:52.149 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.9769e-05 2024-09-21T21:40:52.149 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.149 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.149 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:52.149 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:25.375534+0000", 2024-09-21T21:40:52.149 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00056094100000000002 2024-09-21T21:40:52.149 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.149 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.150 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:52.150 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:25.381121+0000", 2024-09-21T21:40:52.150 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0055874490000000004 2024-09-21T21:40:52.150 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.150 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.150 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:52.150 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:25.381173+0000", 2024-09-21T21:40:52.150 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2447000000000001e-05 2024-09-21T21:40:52.150 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.150 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.150 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:52.150 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:25.381207+0000", 2024-09-21T21:40:52.150 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3130000000000003e-05 2024-09-21T21:40:52.150 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:52.150 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:52.151 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:52.151 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.151 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.151 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4417.0:168 3.5 3:a0cf71c3:::rbd_data.11214be7c411.000000000000002b:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:52.151 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.542298+0000", 2024-09-21T21:40:52.151 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 8.6000427970000004, 2024-09-21T21:40:52.151 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00558012, 2024-09-21T21:40:52.151 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:52.151 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:52.151 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:40:52.151 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:52.151 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4417", 2024-09-21T21:40:52.151 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:52.151 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 168 2024-09-21T21:40:52.152 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.152 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:52.152 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.152 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:52.152 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.542298+0000", 2024-09-21T21:40:52.152 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:52.152 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.152 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.152 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:52.152 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.542298+0000", 2024-09-21T21:40:52.152 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:52.152 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.152 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.152 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:52.153 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.542299+0000", 2024-09-21T21:40:52.153 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.004e-06 2024-09-21T21:40:52.153 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.153 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.153 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:52.153 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.542304+0000", 2024-09-21T21:40:52.153 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7339999999999999e-06 2024-09-21T21:40:52.153 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.153 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.153 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:52.153 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.542320+0000", 2024-09-21T21:40:52.153 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5843000000000001e-05 2024-09-21T21:40:52.153 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.153 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.153 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:52.154 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.542328+0000", 2024-09-21T21:40:52.154 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3399999999999998e-06 2024-09-21T21:40:52.154 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.154 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.154 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:52.154 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.542377+0000", 2024-09-21T21:40:52.154 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8600999999999998e-05 2024-09-21T21:40:52.154 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.154 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.154 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:52.154 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.542450+0000", 2024-09-21T21:40:52.154 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2776999999999998e-05 2024-09-21T21:40:52.154 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.154 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.155 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:52.155 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.547879+0000", 2024-09-21T21:40:52.155 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0054288210000000003 2024-09-21T21:40:52.155 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:52.155 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:52.155 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:52.155 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.155 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.155 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4417.0:296 3.5 3:a7194652:::rbd_data.11214be7c411.0000000000000041:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:52.155 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.566820+0000", 2024-09-21T21:40:52.155 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 8.5755210959999992, 2024-09-21T21:40:52.155 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0049682729999999996, 2024-09-21T21:40:52.155 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:52.155 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:52.155 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:52.156 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:52.156 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4417", 2024-09-21T21:40:52.156 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:52.156 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 296 2024-09-21T21:40:52.156 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.156 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:52.156 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.156 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:52.156 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.566820+0000", 2024-09-21T21:40:52.156 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:52.156 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.156 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.156 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:52.156 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.566820+0000", 2024-09-21T21:40:52.156 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:52.157 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.157 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.157 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:52.157 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.566821+0000", 2024-09-21T21:40:52.157 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9100000000000004e-07 2024-09-21T21:40:52.157 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.157 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.157 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:52.157 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.566825+0000", 2024-09-21T21:40:52.157 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9700000000000001e-06 2024-09-21T21:40:52.157 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.157 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.157 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:52.157 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.566833+0000", 2024-09-21T21:40:52.157 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1340000000000004e-06 2024-09-21T21:40:52.158 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.158 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.158 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:52.158 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.566838+0000", 2024-09-21T21:40:52.158 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4349999999999996e-06 2024-09-21T21:40:52.158 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.158 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.158 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:52.158 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.566888+0000", 2024-09-21T21:40:52.158 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0078999999999999e-05 2024-09-21T21:40:52.158 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.158 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.158 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:52.159 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.566941+0000", 2024-09-21T21:40:52.159 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2930000000000003e-05 2024-09-21T21:40:52.159 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.159 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.159 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 2", 2024-09-21T21:40:52.159 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.567101+0000", 2024-09-21T21:40:52.159 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00015960700000000001 2024-09-21T21:40:52.159 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.159 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.159 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:52.159 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.568575+0000", 2024-09-21T21:40:52.159 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0014742869999999999 2024-09-21T21:40:52.159 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.159 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.160 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:52.160 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.571726+0000", 2024-09-21T21:40:52.160 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0031510280000000002 2024-09-21T21:40:52.160 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.160 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.160 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:52.160 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.571764+0000", 2024-09-21T21:40:52.160 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7938999999999997e-05 2024-09-21T21:40:52.160 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.160 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.160 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:52.161 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.571788+0000", 2024-09-21T21:40:52.161 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4273e-05 2024-09-21T21:40:52.161 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:52.161 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:52.161 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:52.161 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.161 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.161 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4417.0:297 3.5 3:a7d35bab:::rbd_data.11214be7c411.0000000000000044:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:52.161 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.566899+0000", 2024-09-21T21:40:52.161 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 8.5754419199999994, 2024-09-21T21:40:52.161 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.004944665, 2024-09-21T21:40:52.161 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:52.161 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:52.161 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:52.162 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:52.162 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4417", 2024-09-21T21:40:52.162 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:52.162 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 297 2024-09-21T21:40:52.162 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.162 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:52.162 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.162 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:52.162 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.566899+0000", 2024-09-21T21:40:52.162 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:52.162 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.162 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.162 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:52.162 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.566899+0000", 2024-09-21T21:40:52.163 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:52.163 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.163 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.163 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:52.163 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.566900+0000", 2024-09-21T21:40:52.163 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.8700000000000004e-07 2024-09-21T21:40:52.163 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.163 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.163 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:52.163 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.566907+0000", 2024-09-21T21:40:52.163 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4540000000000004e-06 2024-09-21T21:40:52.163 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.163 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.163 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:52.163 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.566918+0000", 2024-09-21T21:40:52.164 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0902e-05 2024-09-21T21:40:52.164 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.164 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.164 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:52.164 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.566925+0000", 2024-09-21T21:40:52.164 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4150000000000002e-06 2024-09-21T21:40:52.164 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.164 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.164 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:52.164 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.567351+0000", 2024-09-21T21:40:52.164 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00042601999999999999 2024-09-21T21:40:52.164 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.164 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.164 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:52.164 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.567426+0000", 2024-09-21T21:40:52.165 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5150999999999999e-05 2024-09-21T21:40:52.165 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.165 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.165 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 2", 2024-09-21T21:40:52.165 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.567630+0000", 2024-09-21T21:40:52.165 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00020410299999999999 2024-09-21T21:40:52.165 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.165 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.165 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:52.165 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.568608+0000", 2024-09-21T21:40:52.165 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00097757500000000006 2024-09-21T21:40:52.165 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.165 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.165 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:52.165 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.571794+0000", 2024-09-21T21:40:52.166 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0031861250000000002 2024-09-21T21:40:52.166 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.166 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.166 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:52.166 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.571817+0000", 2024-09-21T21:40:52.166 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2731e-05 2024-09-21T21:40:52.166 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.166 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.166 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:52.166 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.571844+0000", 2024-09-21T21:40:52.166 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7202000000000001e-05 2024-09-21T21:40:52.166 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:52.166 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:52.166 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:52.166 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.167 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.167 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(client.4417.0:306 3.13 e26/21 3:cb0f117d:::rbd_data.11214be7c411.0000000000000042:head v 26'6, mlcod=26'4)", 2024-09-21T21:40:52.167 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.569366+0000", 2024-09-21T21:40:52.167 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 8.5729752529999992, 2024-09-21T21:40:52.167 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0049738539999999998, 2024-09-21T21:40:52.167 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:52.167 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:52.167 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:52.167 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:52.167 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.167 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:52.167 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.569366+0000", 2024-09-21T21:40:52.167 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:52.167 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.168 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.168 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:52.168 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.569366+0000", 2024-09-21T21:40:52.168 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:52.168 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.168 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.168 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:52.168 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.569367+0000", 2024-09-21T21:40:52.168 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2699999999999999e-07 2024-09-21T21:40:52.168 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.168 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.168 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:52.168 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.569374+0000", 2024-09-21T21:40:52.168 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8739999999999997e-06 2024-09-21T21:40:52.168 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.169 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.169 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:52.169 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.569385+0000", 2024-09-21T21:40:52.169 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1829e-05 2024-09-21T21:40:52.169 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.169 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.169 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:52.169 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.569393+0000", 2024-09-21T21:40:52.169 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2470000000000002e-06 2024-09-21T21:40:52.169 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.169 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.169 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:52.169 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.569436+0000", 2024-09-21T21:40:52.169 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3093999999999998e-05 2024-09-21T21:40:52.170 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.170 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.170 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:52.170 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.569455+0000", 2024-09-21T21:40:52.170 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9211e-05 2024-09-21T21:40:52.170 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.170 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.170 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:52.170 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.574298+0000", 2024-09-21T21:40:52.170 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048428619999999999 2024-09-21T21:40:52.170 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.170 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.170 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:52.170 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.574340+0000", 2024-09-21T21:40:52.170 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2009999999999999e-05 2024-09-21T21:40:52.171 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:52.171 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:52.171 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:52.171 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.171 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.171 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4417.0:346 3.10 3:0db2ee8b:::rbd_data.11214be7c411.0000000000000055:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:52.171 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.576844+0000", 2024-09-21T21:40:52.171 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 8.5654974159999995, 2024-09-21T21:40:52.171 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0059696539999999996, 2024-09-21T21:40:52.171 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:52.171 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:52.171 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:40:52.171 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:52.171 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4417", 2024-09-21T21:40:52.172 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:52.172 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 346 2024-09-21T21:40:52.172 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.172 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:52.172 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.172 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:52.172 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.576844+0000", 2024-09-21T21:40:52.172 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:52.172 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.172 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.172 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:52.172 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.576844+0000", 2024-09-21T21:40:52.172 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:52.172 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.172 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.173 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:52.173 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.576845+0000", 2024-09-21T21:40:52.173 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7199999999999997e-07 2024-09-21T21:40:52.173 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.173 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.173 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:52.173 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.576848+0000", 2024-09-21T21:40:52.173 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7220000000000001e-06 2024-09-21T21:40:52.173 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.173 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.173 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:52.173 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.576858+0000", 2024-09-21T21:40:52.173 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0064999999999999e-05 2024-09-21T21:40:52.173 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.174 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.174 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:52.174 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.576873+0000", 2024-09-21T21:40:52.174 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4065e-05 2024-09-21T21:40:52.174 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.174 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.174 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:52.174 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.576918+0000", 2024-09-21T21:40:52.174 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5911999999999997e-05 2024-09-21T21:40:52.174 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.174 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.174 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:52.174 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.577286+0000", 2024-09-21T21:40:52.174 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00036716299999999999 2024-09-21T21:40:52.174 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.175 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.175 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:52.175 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.582813+0000", 2024-09-21T21:40:52.175 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0055278549999999999 2024-09-21T21:40:52.175 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:52.175 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:52.175 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:52.175 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.175 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.175 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(client.4417.0:352 3.2 e26/21 3:411f17a4:::rbd_data.11214be7c411.0000000000000051:head v 26'9, mlcod=26'6)", 2024-09-21T21:40:52.175 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.578366+0000", 2024-09-21T21:40:52.175 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 8.5639755700000002, 2024-09-21T21:40:52.175 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0045439570000000004, 2024-09-21T21:40:52.175 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:52.176 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:52.176 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:52.176 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:52.176 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.176 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:52.176 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.578366+0000", 2024-09-21T21:40:52.176 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:52.176 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.176 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.176 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:52.176 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.578366+0000", 2024-09-21T21:40:52.176 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:52.176 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.176 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.176 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:52.177 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.578366+0000", 2024-09-21T21:40:52.177 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6000000000000004e-07 2024-09-21T21:40:52.177 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.177 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.177 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:52.177 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.578371+0000", 2024-09-21T21:40:52.177 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.254e-06 2024-09-21T21:40:52.177 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.177 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.177 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:52.177 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.578380+0000", 2024-09-21T21:40:52.177 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5760000000000006e-06 2024-09-21T21:40:52.177 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.177 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.177 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:52.178 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.578385+0000", 2024-09-21T21:40:52.178 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1939999999999997e-06 2024-09-21T21:40:52.178 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.178 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.178 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:52.178 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.578422+0000", 2024-09-21T21:40:52.178 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6723000000000001e-05 2024-09-21T21:40:52.178 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.178 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.178 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:52.178 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.578435+0000", 2024-09-21T21:40:52.178 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2690999999999999e-05 2024-09-21T21:40:52.178 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.178 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.179 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:52.179 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.582885+0000", 2024-09-21T21:40:52.179 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0044504360000000003 2024-09-21T21:40:52.179 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.179 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.179 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:52.179 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.582910+0000", 2024-09-21T21:40:52.179 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4522999999999999e-05 2024-09-21T21:40:52.179 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:52.179 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:52.179 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:52.179 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.179 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.179 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4417.0:356 3.c 3:30e79c3d:::rbd_data.11214be7c411.0000000000000057:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:52.180 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.578474+0000", 2024-09-21T21:40:52.180 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 8.5638667179999999, 2024-09-21T21:40:52.180 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0043452539999999998, 2024-09-21T21:40:52.180 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:52.180 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:52.180 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:40:52.180 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:52.180 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4417", 2024-09-21T21:40:52.180 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:52.180 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 356 2024-09-21T21:40:52.180 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.180 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:52.180 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.180 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:52.181 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.578474+0000", 2024-09-21T21:40:52.181 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:52.181 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.181 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.181 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:52.181 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.578474+0000", 2024-09-21T21:40:52.181 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:52.181 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.181 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.181 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:52.181 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.578475+0000", 2024-09-21T21:40:52.181 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6599999999999996e-07 2024-09-21T21:40:52.181 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.181 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.182 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:52.182 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.578479+0000", 2024-09-21T21:40:52.182 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9580000000000001e-06 2024-09-21T21:40:52.182 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.182 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.182 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:52.182 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.578490+0000", 2024-09-21T21:40:52.182 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0531e-05 2024-09-21T21:40:52.182 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.182 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.182 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:52.182 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.578495+0000", 2024-09-21T21:40:52.182 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3920000000000002e-06 2024-09-21T21:40:52.182 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.182 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.183 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:52.183 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.578545+0000", 2024-09-21T21:40:52.183 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9756999999999998e-05 2024-09-21T21:40:52.183 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.183 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.183 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:52.183 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.578640+0000", 2024-09-21T21:40:52.183 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5098000000000003e-05 2024-09-21T21:40:52.183 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.183 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.183 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:52.183 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.582820+0000", 2024-09-21T21:40:52.183 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0041798520000000004 2024-09-21T21:40:52.183 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:52.183 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:52.184 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:52.184 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.184 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.184 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4417.0:363 3.5 3:a6f0f50c:::rbd_data.11214be7c411.0000000000000059:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:52.184 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.579970+0000", 2024-09-21T21:40:52.184 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 8.5623711809999996, 2024-09-21T21:40:52.184 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.005099307, 2024-09-21T21:40:52.184 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:52.184 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:52.184 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:40:52.184 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:52.184 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4417", 2024-09-21T21:40:52.184 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:52.184 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 363 2024-09-21T21:40:52.185 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.185 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:52.185 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.185 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:52.185 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.579970+0000", 2024-09-21T21:40:52.185 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:52.185 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.185 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.185 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:52.185 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.579970+0000", 2024-09-21T21:40:52.185 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:52.185 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.185 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.185 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:52.185 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.579971+0000", 2024-09-21T21:40:52.186 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.61e-07 2024-09-21T21:40:52.186 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.186 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.186 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:52.186 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.579975+0000", 2024-09-21T21:40:52.186 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8670000000000004e-06 2024-09-21T21:40:52.186 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.186 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.186 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:52.186 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.579983+0000", 2024-09-21T21:40:52.186 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8100000000000004e-06 2024-09-21T21:40:52.186 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.186 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.186 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:52.187 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.579989+0000", 2024-09-21T21:40:52.187 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1370000000000001e-06 2024-09-21T21:40:52.187 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.187 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.187 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:52.187 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.580026+0000", 2024-09-21T21:40:52.187 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7421e-05 2024-09-21T21:40:52.187 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.187 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.187 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:52.187 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.580117+0000", 2024-09-21T21:40:52.187 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.0686999999999994e-05 2024-09-21T21:40:52.187 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.187 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.187 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:52.188 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.585069+0000", 2024-09-21T21:40:52.188 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0049526240000000001 2024-09-21T21:40:52.188 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:52.188 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:52.188 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:52.188 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.188 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.188 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4417.0:414 3.1c 3:3fa5f3c6:::rbd_data.11214be7c411.0000000000000066:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:52.188 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.587704+0000", 2024-09-21T21:40:52.188 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 8.5546372609999999, 2024-09-21T21:40:52.188 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0051267359999999998, 2024-09-21T21:40:52.188 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:52.188 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:52.188 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:40:52.189 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:52.189 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4417", 2024-09-21T21:40:52.189 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:52.189 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 414 2024-09-21T21:40:52.189 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.189 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:52.189 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.189 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:52.189 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.587704+0000", 2024-09-21T21:40:52.189 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:52.189 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.189 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.189 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:52.189 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.587704+0000", 2024-09-21T21:40:52.190 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:52.190 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.190 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.190 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:52.190 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.587705+0000", 2024-09-21T21:40:52.190 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5899999999999996e-07 2024-09-21T21:40:52.190 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.190 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.190 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:52.190 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.587709+0000", 2024-09-21T21:40:52.190 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9779999999999999e-06 2024-09-21T21:40:52.190 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.190 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.190 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:52.191 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.587717+0000", 2024-09-21T21:40:52.191 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8349999999999993e-06 2024-09-21T21:40:52.191 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.191 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.191 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:52.191 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.587723+0000", 2024-09-21T21:40:52.191 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1270000000000002e-06 2024-09-21T21:40:52.191 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.191 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.191 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:52.191 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.587779+0000", 2024-09-21T21:40:52.191 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6202000000000002e-05 2024-09-21T21:40:52.191 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.191 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.191 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:52.192 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.587879+0000", 2024-09-21T21:40:52.192 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.9839000000000004e-05 2024-09-21T21:40:52.192 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.192 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.192 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:52.192 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.592831+0000", 2024-09-21T21:40:52.192 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0049520959999999996 2024-09-21T21:40:52.192 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:52.192 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:52.192 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:52.192 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.192 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.192 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4417.0:439 3.1d 3:b9e79477:::rbd_data.11214be7c411.000000000000006d:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:52.192 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.590273+0000", 2024-09-21T21:40:52.193 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 8.5520683359999996, 2024-09-21T21:40:52.193 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0051529849999999997, 2024-09-21T21:40:52.193 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:52.193 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:52.193 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:40:52.193 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:52.193 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4417", 2024-09-21T21:40:52.193 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:52.193 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 439 2024-09-21T21:40:52.193 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.193 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:52.193 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.193 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:52.193 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.590273+0000", 2024-09-21T21:40:52.193 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:52.194 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.194 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.194 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:52.194 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.590273+0000", 2024-09-21T21:40:52.194 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:52.194 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.194 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.194 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:52.194 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.590273+0000", 2024-09-21T21:40:52.194 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1600000000000001e-07 2024-09-21T21:40:52.194 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.194 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.194 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:52.194 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.590277+0000", 2024-09-21T21:40:52.195 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4340000000000001e-06 2024-09-21T21:40:52.195 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.195 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.195 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:52.195 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.590284+0000", 2024-09-21T21:40:52.195 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5469999999999999e-06 2024-09-21T21:40:52.195 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.195 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.195 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:52.195 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.590297+0000", 2024-09-21T21:40:52.195 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3030999999999999e-05 2024-09-21T21:40:52.195 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.195 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.195 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:52.195 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.590348+0000", 2024-09-21T21:40:52.196 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0228000000000001e-05 2024-09-21T21:40:52.196 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.196 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.196 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:52.196 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.590465+0000", 2024-09-21T21:40:52.196 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000116847 2024-09-21T21:40:52.196 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.196 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.196 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:52.196 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.595426+0000", 2024-09-21T21:40:52.196 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.004961282 2024-09-21T21:40:52.196 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:52.196 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:52.196 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:52.197 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.197 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.197 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4417.0:440 3.5 3:a6f99a77:::rbd_data.11214be7c411.0000000000000068:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:52.197 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.590317+0000", 2024-09-21T21:40:52.197 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 8.5520240659999995, 2024-09-21T21:40:52.197 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.008514967, 2024-09-21T21:40:52.197 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:52.197 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:52.197 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:52.197 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:52.197 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4417", 2024-09-21T21:40:52.197 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:52.197 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 440 2024-09-21T21:40:52.197 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.197 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:52.198 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.198 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:52.198 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.590317+0000", 2024-09-21T21:40:52.198 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:52.198 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.198 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.198 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:52.198 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.590317+0000", 2024-09-21T21:40:52.198 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:52.198 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.198 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.198 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:52.198 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.590318+0000", 2024-09-21T21:40:52.198 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3500000000000002e-07 2024-09-21T21:40:52.199 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.199 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.199 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:52.199 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.590320+0000", 2024-09-21T21:40:52.199 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4650000000000001e-06 2024-09-21T21:40:52.199 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.199 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.199 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:52.199 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.590327+0000", 2024-09-21T21:40:52.199 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7220000000000002e-06 2024-09-21T21:40:52.199 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.199 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.199 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:52.199 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.590330+0000", 2024-09-21T21:40:52.200 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5439999999999999e-06 2024-09-21T21:40:52.200 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.200 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.200 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:52.200 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.590371+0000", 2024-09-21T21:40:52.200 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1183000000000002e-05 2024-09-21T21:40:52.200 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.200 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.200 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:52.200 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.590454+0000", 2024-09-21T21:40:52.200 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2379999999999994e-05 2024-09-21T21:40:52.200 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.200 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.200 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 2", 2024-09-21T21:40:52.200 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.590666+0000", 2024-09-21T21:40:52.201 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00021181999999999999 2024-09-21T21:40:52.201 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.201 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.201 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:52.201 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.595067+0000", 2024-09-21T21:40:52.201 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0044010830000000001 2024-09-21T21:40:52.201 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.201 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.201 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:52.201 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.598764+0000", 2024-09-21T21:40:52.201 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0036973140000000002 2024-09-21T21:40:52.201 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.201 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.201 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:52.201 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.598797+0000", 2024-09-21T21:40:52.202 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2740000000000002e-05 2024-09-21T21:40:52.202 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.202 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.202 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:52.202 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.598832+0000", 2024-09-21T21:40:52.202 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5281000000000003e-05 2024-09-21T21:40:52.202 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:52.202 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:52.202 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:52.202 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.202 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.202 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "pg_update_log_missing(3.17 epoch 26/21 rep_tid 46 entries 26'11 (0'0) error 3:ea5ebfbf:::rbd_data.11214be7c411.000000000000006c:head by client.4417.0:436 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 26'8)", 2024-09-21T21:40:52.202 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.590562+0000", 2024-09-21T21:40:52.202 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 8.5517793710000003, 2024-09-21T21:40:52.203 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00451251, 2024-09-21T21:40:52.203 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:52.203 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:52.203 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2024-09-21T21:40:52.203 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:52.203 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.203 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:52.203 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.590562+0000", 2024-09-21T21:40:52.203 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:52.203 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.203 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.203 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:52.203 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.590562+0000", 2024-09-21T21:40:52.203 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:52.203 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.204 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.204 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:52.204 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.590562+0000", 2024-09-21T21:40:52.204 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8899999999999999e-07 2024-09-21T21:40:52.204 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.204 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.204 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:52.204 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.590574+0000", 2024-09-21T21:40:52.204 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1522000000000001e-05 2024-09-21T21:40:52.204 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.204 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.204 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:52.204 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.590588+0000", 2024-09-21T21:40:52.204 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4494999999999999e-05 2024-09-21T21:40:52.205 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.205 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.205 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:52.205 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.590602+0000", 2024-09-21T21:40:52.205 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3064000000000001e-05 2024-09-21T21:40:52.205 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.205 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.205 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:52.205 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.590672+0000", 2024-09-21T21:40:52.205 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0517000000000003e-05 2024-09-21T21:40:52.205 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.205 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.205 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:52.205 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.595074+0000", 2024-09-21T21:40:52.205 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0044023229999999997 2024-09-21T21:40:52.206 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:52.206 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:52.206 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:52.206 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.206 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.206 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4417.0:455 3.1c 3:3fa5f3c6:::rbd_data.11214be7c411.0000000000000066:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:52.206 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.592971+0000", 2024-09-21T21:40:52.206 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 8.5493702159999998, 2024-09-21T21:40:52.206 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0049042829999999997, 2024-09-21T21:40:52.206 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:52.206 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:52.206 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:52.206 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:52.206 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4417", 2024-09-21T21:40:52.207 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:52.207 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 455 2024-09-21T21:40:52.207 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.207 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:52.207 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.207 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:52.207 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.592971+0000", 2024-09-21T21:40:52.207 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:52.207 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.207 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.207 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:52.207 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.592971+0000", 2024-09-21T21:40:52.207 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:52.207 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.207 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.208 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:52.208 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.592972+0000", 2024-09-21T21:40:52.208 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8000000000000005e-07 2024-09-21T21:40:52.208 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.208 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.208 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:52.208 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.592976+0000", 2024-09-21T21:40:52.208 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.121e-06 2024-09-21T21:40:52.208 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.208 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.208 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:52.208 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.592984+0000", 2024-09-21T21:40:52.208 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3270000000000001e-06 2024-09-21T21:40:52.208 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.208 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.209 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:52.209 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.592991+0000", 2024-09-21T21:40:52.209 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4130000000000003e-06 2024-09-21T21:40:52.209 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.209 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.209 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:52.209 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.593043+0000", 2024-09-21T21:40:52.209 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2546999999999997e-05 2024-09-21T21:40:52.209 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.209 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.209 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:52.209 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.593115+0000", 2024-09-21T21:40:52.209 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1763999999999999e-05 2024-09-21T21:40:52.209 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.209 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.210 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 4", 2024-09-21T21:40:52.210 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.593303+0000", 2024-09-21T21:40:52.210 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000188211 2024-09-21T21:40:52.210 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.210 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.210 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:52.210 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.597457+0000", 2024-09-21T21:40:52.210 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0041535840000000001 2024-09-21T21:40:52.210 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.210 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.210 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:52.210 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.597789+0000", 2024-09-21T21:40:52.210 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00033188100000000002 2024-09-21T21:40:52.210 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.211 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.211 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:52.211 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.597848+0000", 2024-09-21T21:40:52.211 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9119000000000001e-05 2024-09-21T21:40:52.211 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.211 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.211 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:52.211 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.597875+0000", 2024-09-21T21:40:52.211 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7535999999999999e-05 2024-09-21T21:40:52.211 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:52.211 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:52.211 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:52.211 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.211 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.212 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "pg_update_log_missing(3.2 epoch 26/21 rep_tid 112 entries 26'12 (0'0) error 3:430d3f60:::rbd_data.11214be7c411.000000000000006e:head by client.4417.0:452 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 26'10)", 2024-09-21T21:40:52.212 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.593101+0000", 2024-09-21T21:40:52.212 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 8.5492401139999998, 2024-09-21T21:40:52.212 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0043794510000000004, 2024-09-21T21:40:52.212 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:52.212 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:52.212 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2024-09-21T21:40:52.212 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:52.212 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.212 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:52.212 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.593101+0000", 2024-09-21T21:40:52.212 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:52.212 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.212 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.212 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:52.213 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.593101+0000", 2024-09-21T21:40:52.213 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:52.213 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.213 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.213 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:52.213 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.593102+0000", 2024-09-21T21:40:52.213 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8500000000000001e-07 2024-09-21T21:40:52.213 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.213 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.213 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:52.213 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.593119+0000", 2024-09-21T21:40:52.213 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6912999999999999e-05 2024-09-21T21:40:52.213 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.213 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.213 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:52.214 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.593136+0000", 2024-09-21T21:40:52.214 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7198999999999999e-05 2024-09-21T21:40:52.214 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.214 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.214 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:52.214 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.593149+0000", 2024-09-21T21:40:52.214 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3192e-05 2024-09-21T21:40:52.214 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.214 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.214 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:52.214 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.593215+0000", 2024-09-21T21:40:52.214 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6023000000000001e-05 2024-09-21T21:40:52.214 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.214 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.215 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:52.215 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.597481+0000", 2024-09-21T21:40:52.215 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0042655389999999996 2024-09-21T21:40:52.215 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:52.215 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:52.215 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:52.215 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.215 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.215 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4417.0:469 3.1d 3:b9e79477:::rbd_data.11214be7c411.000000000000006d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:52.215 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.595683+0000", 2024-09-21T21:40:52.215 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 8.5466585239999997, 2024-09-21T21:40:52.215 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0044134409999999997, 2024-09-21T21:40:52.215 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:52.215 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:52.216 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:52.216 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:52.216 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4417", 2024-09-21T21:40:52.216 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:52.216 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 469 2024-09-21T21:40:52.216 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.216 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:52.216 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.216 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:52.216 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.595683+0000", 2024-09-21T21:40:52.216 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:52.216 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.216 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.216 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:52.217 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.595683+0000", 2024-09-21T21:40:52.217 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:52.217 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.217 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.217 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:52.217 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.595683+0000", 2024-09-21T21:40:52.217 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0800000000000004e-07 2024-09-21T21:40:52.217 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.217 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.217 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:52.217 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.595688+0000", 2024-09-21T21:40:52.217 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0239999999999997e-06 2024-09-21T21:40:52.217 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.217 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.217 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:52.218 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.595697+0000", 2024-09-21T21:40:52.218 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8060000000000001e-06 2024-09-21T21:40:52.218 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.218 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.218 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:52.218 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.595702+0000", 2024-09-21T21:40:52.218 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9919999999999998e-06 2024-09-21T21:40:52.218 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.218 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.218 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:52.218 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.595748+0000", 2024-09-21T21:40:52.218 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5636000000000001e-05 2024-09-21T21:40:52.218 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.218 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.219 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:52.219 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.595829+0000", 2024-09-21T21:40:52.219 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1021999999999998e-05 2024-09-21T21:40:52.219 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.219 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.219 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 4", 2024-09-21T21:40:52.219 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.596007+0000", 2024-09-21T21:40:52.219 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000177839 2024-09-21T21:40:52.219 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.219 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.219 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:52.219 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.597759+0000", 2024-09-21T21:40:52.219 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0017523389999999999 2024-09-21T21:40:52.219 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.220 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.220 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:52.220 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.600031+0000", 2024-09-21T21:40:52.220 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022723800000000001 2024-09-21T21:40:52.220 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.220 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.220 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:52.220 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.600063+0000", 2024-09-21T21:40:52.220 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1383000000000001e-05 2024-09-21T21:40:52.220 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.220 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.220 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:52.220 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.600096+0000", 2024-09-21T21:40:52.220 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3411999999999998e-05 2024-09-21T21:40:52.220 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:52.221 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:52.221 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:52.221 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.221 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.221 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4417.0:472 3.1c 3:393bd49d:::rbd_data.11214be7c411.0000000000000072:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:52.221 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.597902+0000", 2024-09-21T21:40:52.221 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 8.5444396089999994, 2024-09-21T21:40:52.221 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048015009999999997, 2024-09-21T21:40:52.221 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:52.221 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:52.221 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:40:52.221 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:52.221 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4417", 2024-09-21T21:40:52.221 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:52.222 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 472 2024-09-21T21:40:52.222 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.222 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:52.222 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.222 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:52.222 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.597902+0000", 2024-09-21T21:40:52.222 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:52.222 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.222 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.222 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:52.222 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.597902+0000", 2024-09-21T21:40:52.222 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:52.222 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.222 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.222 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:52.223 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.597902+0000", 2024-09-21T21:40:52.223 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3399999999999999e-07 2024-09-21T21:40:52.223 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.223 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.223 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:52.223 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.597905+0000", 2024-09-21T21:40:52.223 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1829999999999998e-06 2024-09-21T21:40:52.223 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.223 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.223 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:52.223 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.597915+0000", 2024-09-21T21:40:52.223 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.1379999999999996e-06 2024-09-21T21:40:52.223 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.223 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.223 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:52.224 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.597920+0000", 2024-09-21T21:40:52.224 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.972e-06 2024-09-21T21:40:52.224 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.224 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.224 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:52.224 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.597964+0000", 2024-09-21T21:40:52.224 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4611999999999999e-05 2024-09-21T21:40:52.224 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.224 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.224 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:52.224 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.598080+0000", 2024-09-21T21:40:52.224 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000115553 2024-09-21T21:40:52.224 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.224 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.225 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:52.225 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.602703+0000", 2024-09-21T21:40:52.225 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0046234090000000002 2024-09-21T21:40:52.225 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:52.225 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:52.225 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:52.225 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.225 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.225 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4417.0:479 3.1d 3:bcbf195b:::rbd_data.11214be7c411.0000000000000077:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:52.225 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.600396+0000", 2024-09-21T21:40:52.225 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 8.5419448730000003, 2024-09-21T21:40:52.225 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0047349260000000004, 2024-09-21T21:40:52.225 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:52.225 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:52.226 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:40:52.226 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:52.226 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4417", 2024-09-21T21:40:52.226 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:52.226 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 479 2024-09-21T21:40:52.226 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.226 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:52.226 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.226 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:52.226 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.600396+0000", 2024-09-21T21:40:52.226 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:52.226 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.226 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.226 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:52.227 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.600396+0000", 2024-09-21T21:40:52.227 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:52.227 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.227 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.227 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:52.227 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.600397+0000", 2024-09-21T21:40:52.227 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9599999999999998e-07 2024-09-21T21:40:52.227 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.227 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.227 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:52.227 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.600402+0000", 2024-09-21T21:40:52.227 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7779999999999998e-06 2024-09-21T21:40:52.227 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.227 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.227 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:52.228 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.600414+0000", 2024-09-21T21:40:52.228 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2379e-05 2024-09-21T21:40:52.228 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.228 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.228 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:52.228 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.600425+0000", 2024-09-21T21:40:52.228 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0856000000000001e-05 2024-09-21T21:40:52.228 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.228 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.228 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:52.228 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.600464+0000", 2024-09-21T21:40:52.228 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8639000000000001e-05 2024-09-21T21:40:52.228 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.228 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.229 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:52.229 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.600537+0000", 2024-09-21T21:40:52.229 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2674000000000002e-05 2024-09-21T21:40:52.229 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.229 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.229 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:52.229 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.605131+0000", 2024-09-21T21:40:52.229 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0045947039999999998 2024-09-21T21:40:52.229 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:52.229 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:52.229 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:52.229 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.229 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.229 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:1199 3.16 3:6acc33a0:::rbd_data.1121bf4f5ab3.0000000000000066:head [set-alloc-hint object_size 1048576 write_size 1048576,write 38912~26624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-21T21:40:52.230 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:48.417047+0000", 2024-09-21T21:40:52.230 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 3.725294415, 2024-09-21T21:40:52.230 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.005078061, 2024-09-21T21:40:52.230 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:52.230 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:52.230 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:52.230 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:52.230 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2024-09-21T21:40:52.230 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1688278210", 2024-09-21T21:40:52.230 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1199 2024-09-21T21:40:52.230 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.230 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:52.230 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.230 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:52.230 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.417047+0000", 2024-09-21T21:40:52.231 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:52.231 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.231 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.231 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:52.231 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.417047+0000", 2024-09-21T21:40:52.231 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:52.231 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.231 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.231 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:52.231 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.417048+0000", 2024-09-21T21:40:52.231 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1880000000000001e-06 2024-09-21T21:40:52.231 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.231 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.231 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:52.231 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.417067+0000", 2024-09-21T21:40:52.232 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9490000000000001e-05 2024-09-21T21:40:52.232 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.232 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.232 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:52.232 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.417078+0000", 2024-09-21T21:40:52.232 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0533000000000001e-05 2024-09-21T21:40:52.232 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.232 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.232 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:52.232 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.417083+0000", 2024-09-21T21:40:52.232 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0370000000000002e-06 2024-09-21T21:40:52.232 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.232 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.232 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:52.233 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.417126+0000", 2024-09-21T21:40:52.233 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3281000000000001e-05 2024-09-21T21:40:52.233 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.233 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.233 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:52.233 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.417231+0000", 2024-09-21T21:40:52.233 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010446399999999999 2024-09-21T21:40:52.233 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.233 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.233 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:40:52.233 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.417361+0000", 2024-09-21T21:40:52.233 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013056100000000001 2024-09-21T21:40:52.233 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.233 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.233 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:52.234 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.421796+0000", 2024-09-21T21:40:52.234 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0044345849999999996 2024-09-21T21:40:52.234 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.234 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.234 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:52.234 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.422035+0000", 2024-09-21T21:40:52.234 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000239134 2024-09-21T21:40:52.234 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.234 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.234 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:52.234 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.422086+0000", 2024-09-21T21:40:52.234 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1424000000000001e-05 2024-09-21T21:40:52.234 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.234 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.234 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:52.235 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.422125+0000", 2024-09-21T21:40:52.235 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8364000000000001e-05 2024-09-21T21:40:52.235 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:52.235 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:52.235 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:52.235 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.235 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.235 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:1201 3.1d 3:bfe30718:::rbd_data.1121bf4f5ab3.000000000000003f:head [write 33280~32256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-21T21:40:52.235 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:48.417097+0000", 2024-09-21T21:40:52.235 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 3.7252446460000002, 2024-09-21T21:40:52.235 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0051236809999999997, 2024-09-21T21:40:52.235 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:52.235 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:52.235 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:52.236 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:52.236 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2024-09-21T21:40:52.236 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1688278210", 2024-09-21T21:40:52.236 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1201 2024-09-21T21:40:52.236 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.236 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:52.236 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.236 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:52.236 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.417097+0000", 2024-09-21T21:40:52.236 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:52.236 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.236 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.236 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:52.236 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.417097+0000", 2024-09-21T21:40:52.236 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:52.237 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.237 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.237 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:52.237 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.417097+0000", 2024-09-21T21:40:52.237 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5600000000000001e-07 2024-09-21T21:40:52.237 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.237 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.237 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:52.237 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.417139+0000", 2024-09-21T21:40:52.237 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2252000000000001e-05 2024-09-21T21:40:52.237 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.237 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.238 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:52.238 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.417148+0000", 2024-09-21T21:40:52.238 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.422e-06 2024-09-21T21:40:52.238 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.238 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.238 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:52.238 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.417154+0000", 2024-09-21T21:40:52.238 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0950000000000001e-06 2024-09-21T21:40:52.238 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.238 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.238 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:52.238 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.417218+0000", 2024-09-21T21:40:52.238 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4140000000000006e-05 2024-09-21T21:40:52.238 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.239 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.239 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:52.239 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.417288+0000", 2024-09-21T21:40:52.239 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0229999999999997e-05 2024-09-21T21:40:52.239 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.239 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.239 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 4", 2024-09-21T21:40:52.239 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.417402+0000", 2024-09-21T21:40:52.239 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00011437100000000001 2024-09-21T21:40:52.239 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.239 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.239 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:52.239 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.418167+0000", 2024-09-21T21:40:52.239 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00076478099999999997 2024-09-21T21:40:52.240 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.240 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.240 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:52.240 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.422138+0000", 2024-09-21T21:40:52.240 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0039707759999999996 2024-09-21T21:40:52.240 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.240 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.240 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:52.240 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.422186+0000", 2024-09-21T21:40:52.240 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7626000000000001e-05 2024-09-21T21:40:52.240 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:52.240 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:52.240 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:52.240 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.422220+0000", 2024-09-21T21:40:52.241 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4632000000000003e-05 2024-09-21T21:40:52.241 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:52.241 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:52.241 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:52.241 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:52.241 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:52.241 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:40:52.242 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T21:40:52.243 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:40:52.241+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:40:52.343 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:40:52.341+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:40:52.345 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:40:52.345 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:40:52.345 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 0 2024-09-21T21:40:52.345 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:40:52.354 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T21:40:52.444 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:40:52.443+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:40:52.499 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:40:52.499 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:40:52.499 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:40:52.499 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:40:52.499 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:40:52.509 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T21:40:52.551 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:40:52.549+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:40:52.646 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:40:52.645+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:40:52.654 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:40:52.654 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 20, 2024-09-21T21:40:52.655 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 600, 2024-09-21T21:40:52.655 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:40:52.655 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.655 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "pg_update_log_missing(3.15 epoch 26/21 rep_tid 217 entries 26'3 (0'0) error 3:abf84a89:::rbd_data.11214be7c411.0000000000000025:head by client.4417.0:131 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 24'1)", 2024-09-21T21:40:52.655 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.536284+0000", 2024-09-21T21:40:52.655 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.1158748430000003, 2024-09-21T21:40:52.655 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0057258170000000002, 2024-09-21T21:40:52.655 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:52.655 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:52.655 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:40:52.655 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:52.655 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.655 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:52.655 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.536284+0000", 2024-09-21T21:40:52.655 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:52.656 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.656 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.656 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:52.656 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.536284+0000", 2024-09-21T21:40:52.656 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:52.656 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.656 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.656 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:52.656 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.536284+0000", 2024-09-21T21:40:52.656 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.6399999999999998e-07 2024-09-21T21:40:52.656 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.656 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.656 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:52.656 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.536290+0000", 2024-09-21T21:40:52.656 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2430000000000002e-06 2024-09-21T21:40:52.656 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.657 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.657 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:52.657 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.536309+0000", 2024-09-21T21:40:52.657 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.8572000000000001e-05 2024-09-21T21:40:52.657 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.657 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.657 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:52.657 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.536324+0000", 2024-09-21T21:40:52.657 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.4876e-05 2024-09-21T21:40:52.657 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.657 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.657 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:52.657 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.536414+0000", 2024-09-21T21:40:52.657 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.0847999999999995e-05 2024-09-21T21:40:52.657 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.658 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.658 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:52.658 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.542009+0000", 2024-09-21T21:40:52.658 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0055950139999999997 2024-09-21T21:40:52.658 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:52.658 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:52.658 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:52.658 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.658 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.658 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:157 3.6 3:678f83b3:::rbd_data.11214be7c411.000000000000002a:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:52.658 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.538947+0000", 2024-09-21T21:40:52.658 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.113211111, 2024-09-21T21:40:52.658 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.006108998, 2024-09-21T21:40:52.659 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:52.659 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:52.659 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:40:52.659 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:52.659 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:52.659 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:52.659 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 157 2024-09-21T21:40:52.659 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.659 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:52.659 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.659 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:52.659 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.538947+0000", 2024-09-21T21:40:52.659 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:52.659 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.660 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.660 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:52.660 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.538947+0000", 2024-09-21T21:40:52.660 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:52.660 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.660 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.660 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:52.660 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.538948+0000", 2024-09-21T21:40:52.660 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.7100000000000011e-07 2024-09-21T21:40:52.660 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.660 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.660 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:52.660 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.538953+0000", 2024-09-21T21:40:52.660 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7099999999999998e-06 2024-09-21T21:40:52.660 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.661 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.661 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:52.661 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.538964+0000", 2024-09-21T21:40:52.661 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0900000000000001e-05 2024-09-21T21:40:52.661 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.661 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.661 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:52.661 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.538972+0000", 2024-09-21T21:40:52.661 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7279999999999995e-06 2024-09-21T21:40:52.661 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.661 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.661 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:52.661 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.539034+0000", 2024-09-21T21:40:52.661 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2025000000000003e-05 2024-09-21T21:40:52.662 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.662 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.662 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:52.662 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.539148+0000", 2024-09-21T21:40:52.662 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000113879 2024-09-21T21:40:52.662 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.662 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.662 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:52.662 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.545056+0000", 2024-09-21T21:40:52.662 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0059087849999999997 2024-09-21T21:40:52.662 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:52.662 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:52.662 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:52.662 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.663 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.663 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:181 3.1 3:865f7811:::rbd_data.11214be7c411.000000000000002f:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:52.663 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.545151+0000", 2024-09-21T21:40:52.663 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.1070075540000008, 2024-09-21T21:40:52.663 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0053218199999999997, 2024-09-21T21:40:52.663 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:52.663 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:52.663 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:40:52.663 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:52.663 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:52.663 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:52.663 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 181 2024-09-21T21:40:52.663 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.663 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:52.663 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.663 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:52.664 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.545151+0000", 2024-09-21T21:40:52.664 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:52.664 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.664 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.664 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:52.664 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.545151+0000", 2024-09-21T21:40:52.664 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:52.664 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.664 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.664 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:52.664 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.545152+0000", 2024-09-21T21:40:52.664 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6200000000000006e-07 2024-09-21T21:40:52.664 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.664 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.664 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:52.664 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.545157+0000", 2024-09-21T21:40:52.665 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4410000000000001e-06 2024-09-21T21:40:52.665 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.665 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.665 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:52.665 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.545171+0000", 2024-09-21T21:40:52.665 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.3209e-05 2024-09-21T21:40:52.665 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.665 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.665 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:52.665 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.545178+0000", 2024-09-21T21:40:52.665 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7759999999999997e-06 2024-09-21T21:40:52.665 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.665 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.665 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:52.665 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.545230+0000", 2024-09-21T21:40:52.666 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1641000000000002e-05 2024-09-21T21:40:52.666 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.666 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.666 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:52.666 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.545357+0000", 2024-09-21T21:40:52.666 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00012734199999999999 2024-09-21T21:40:52.666 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.666 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.666 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:52.666 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.550473+0000", 2024-09-21T21:40:52.666 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0051154490000000002 2024-09-21T21:40:52.666 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:52.666 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:52.667 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:52.667 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.667 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.667 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:199 3.12 3:4fa97af7:::rbd_data.11214be7c411.0000000000000033:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:52.667 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.547897+0000", 2024-09-21T21:40:52.667 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.1042616400000007, 2024-09-21T21:40:52.667 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0052696440000000004, 2024-09-21T21:40:52.667 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:52.667 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:52.667 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:40:52.667 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:52.667 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:52.667 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:52.668 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 199 2024-09-21T21:40:52.668 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.668 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:52.668 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.668 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:52.668 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.547897+0000", 2024-09-21T21:40:52.668 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:52.668 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.668 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.668 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:52.668 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.547897+0000", 2024-09-21T21:40:52.668 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:52.669 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.669 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.669 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:52.669 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.547898+0000", 2024-09-21T21:40:52.669 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.7700000000000003e-07 2024-09-21T21:40:52.669 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.669 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.669 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:52.669 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.547902+0000", 2024-09-21T21:40:52.669 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6099999999999999e-06 2024-09-21T21:40:52.669 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.669 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.669 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:52.669 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.547915+0000", 2024-09-21T21:40:52.670 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.289e-05 2024-09-21T21:40:52.670 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.670 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.670 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:52.670 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.547922+0000", 2024-09-21T21:40:52.670 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.2099999999999996e-06 2024-09-21T21:40:52.670 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.670 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.670 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:52.670 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.547993+0000", 2024-09-21T21:40:52.670 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0290999999999995e-05 2024-09-21T21:40:52.670 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.670 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.671 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:52.671 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.548121+0000", 2024-09-21T21:40:52.671 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000128756 2024-09-21T21:40:52.671 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.671 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.671 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:52.671 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.553166+0000", 2024-09-21T21:40:52.671 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0050450099999999999 2024-09-21T21:40:52.671 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:52.671 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:52.671 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:52.671 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.671 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.671 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "pg_update_log_missing(3.18 epoch 26/21 rep_tid 19 entries 26'5 (0'0) error 3:18a859ac:::rbd_data.11214be7c411.0000000000000036:head by client.4417.0:214 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 24'3)", 2024-09-21T21:40:52.672 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.550506+0000", 2024-09-21T21:40:52.672 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.1016525500000007, 2024-09-21T21:40:52.672 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00530073, 2024-09-21T21:40:52.672 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:52.672 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:52.672 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:40:52.672 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:52.672 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.672 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:52.672 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.550506+0000", 2024-09-21T21:40:52.672 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:52.672 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.672 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.673 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:52.673 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.550506+0000", 2024-09-21T21:40:52.673 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:52.673 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.673 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.673 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:52.673 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.550506+0000", 2024-09-21T21:40:52.673 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.16e-07 2024-09-21T21:40:52.673 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.673 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.673 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:52.673 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.550512+0000", 2024-09-21T21:40:52.674 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6180000000000003e-06 2024-09-21T21:40:52.674 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.674 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.674 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:52.674 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.550529+0000", 2024-09-21T21:40:52.674 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.7325000000000001e-05 2024-09-21T21:40:52.674 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.674 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.674 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:52.674 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.550553+0000", 2024-09-21T21:40:52.674 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.4012999999999999e-05 2024-09-21T21:40:52.674 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.674 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.675 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:52.675 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.550636+0000", 2024-09-21T21:40:52.675 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.3150000000000002e-05 2024-09-21T21:40:52.675 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.675 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.675 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:52.675 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.555807+0000", 2024-09-21T21:40:52.675 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0051704079999999996 2024-09-21T21:40:52.675 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:52.675 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:52.678 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:52.678 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.678 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.678 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:217 3.1 3:865f7811:::rbd_data.11214be7c411.000000000000002f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:52.678 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.550723+0000", 2024-09-21T21:40:52.678 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.1014355519999999, 2024-09-21T21:40:52.678 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0051892220000000003, 2024-09-21T21:40:52.678 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:52.678 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:52.679 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:52.679 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:52.679 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:52.679 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:52.679 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 217 2024-09-21T21:40:52.679 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.679 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:52.679 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.679 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:52.679 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.550723+0000", 2024-09-21T21:40:52.679 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:52.679 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.679 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.679 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:52.679 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.550723+0000", 2024-09-21T21:40:52.679 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:52.680 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.680 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.680 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:52.680 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.550724+0000", 2024-09-21T21:40:52.680 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.4399999999999999e-07 2024-09-21T21:40:52.680 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.680 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.680 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:52.680 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.550728+0000", 2024-09-21T21:40:52.680 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.9179999999999996e-06 2024-09-21T21:40:52.680 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.680 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.680 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:52.680 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.550737+0000", 2024-09-21T21:40:52.680 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.9789999999999999e-06 2024-09-21T21:40:52.680 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.680 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.681 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:52.681 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.550742+0000", 2024-09-21T21:40:52.681 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9830000000000004e-06 2024-09-21T21:40:52.681 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.681 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.681 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:52.681 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.550788+0000", 2024-09-21T21:40:52.681 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6807000000000001e-05 2024-09-21T21:40:52.681 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.681 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.681 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:52.681 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.550863+0000", 2024-09-21T21:40:52.681 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.4787000000000001e-05 2024-09-21T21:40:52.681 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.681 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.681 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 4", 2024-09-21T21:40:52.682 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.551062+0000", 2024-09-21T21:40:52.682 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00019837000000000001 2024-09-21T21:40:52.682 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.682 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.682 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:52.682 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.551991+0000", 2024-09-21T21:40:52.682 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00092915800000000002 2024-09-21T21:40:52.682 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.682 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.682 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:40:52.682 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.555819+0000", 2024-09-21T21:40:52.682 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0038285160000000001 2024-09-21T21:40:52.682 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.682 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.682 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:52.683 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.555862+0000", 2024-09-21T21:40:52.683 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3158999999999998e-05 2024-09-21T21:40:52.683 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.683 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.683 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:52.683 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.555912+0000", 2024-09-21T21:40:52.683 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9801e-05 2024-09-21T21:40:52.683 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:52.683 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:52.683 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:52.683 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.683 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.683 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:218 3.12 3:48deee06:::rbd_data.11214be7c411.0000000000000031:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:52.683 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.550755+0000", 2024-09-21T21:40:52.683 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.1014036199999993, 2024-09-21T21:40:52.684 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0051449260000000002, 2024-09-21T21:40:52.684 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:52.684 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:52.684 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:52.684 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:52.684 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:52.684 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:52.684 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 218 2024-09-21T21:40:52.684 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.684 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:52.684 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.684 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:52.684 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.550755+0000", 2024-09-21T21:40:52.684 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:52.685 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.685 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.685 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:52.685 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.550755+0000", 2024-09-21T21:40:52.685 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:52.685 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.685 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.685 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:52.685 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.550755+0000", 2024-09-21T21:40:52.685 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9900000000000001e-07 2024-09-21T21:40:52.685 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.685 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.685 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:52.685 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.550758+0000", 2024-09-21T21:40:52.685 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.8030000000000001e-06 2024-09-21T21:40:52.685 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.685 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.686 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:52.686 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.550765+0000", 2024-09-21T21:40:52.686 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.567e-06 2024-09-21T21:40:52.686 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.686 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.686 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:52.686 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.550770+0000", 2024-09-21T21:40:52.686 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9180000000000002e-06 2024-09-21T21:40:52.686 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.686 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.686 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:52.686 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.550842+0000", 2024-09-21T21:40:52.686 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.2397000000000005e-05 2024-09-21T21:40:52.686 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.686 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.687 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:52.687 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.550894+0000", 2024-09-21T21:40:52.687 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1953000000000002e-05 2024-09-21T21:40:52.687 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.687 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.687 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 3", 2024-09-21T21:40:52.687 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.551019+0000", 2024-09-21T21:40:52.687 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00012530500000000001 2024-09-21T21:40:52.687 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.687 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.687 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:52.687 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.551761+0000", 2024-09-21T21:40:52.687 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00074191700000000003 2024-09-21T21:40:52.687 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.687 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.687 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:40:52.688 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.555812+0000", 2024-09-21T21:40:52.688 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0040509300000000003 2024-09-21T21:40:52.688 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.688 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.688 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:52.688 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.555861+0000", 2024-09-21T21:40:52.688 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9011000000000002e-05 2024-09-21T21:40:52.688 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.688 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.688 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:52.688 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.555900+0000", 2024-09-21T21:40:52.688 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8625999999999999e-05 2024-09-21T21:40:52.688 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:52.688 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:52.688 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:52.689 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.689 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.689 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:227 3.6 3:678f83b3:::rbd_data.11214be7c411.000000000000002a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:52.689 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.553729+0000", 2024-09-21T21:40:52.689 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.0984295500000005, 2024-09-21T21:40:52.689 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0090411439999999992, 2024-09-21T21:40:52.689 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:52.689 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:52.689 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:52.689 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:52.689 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:52.689 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:52.689 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 227 2024-09-21T21:40:52.689 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.690 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:52.690 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.690 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:52.690 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.553729+0000", 2024-09-21T21:40:52.690 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:52.690 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.690 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.690 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:52.690 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.553729+0000", 2024-09-21T21:40:52.690 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:52.690 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.690 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.690 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:52.690 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.553729+0000", 2024-09-21T21:40:52.691 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.06e-07 2024-09-21T21:40:52.691 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.691 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.691 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:52.691 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.553771+0000", 2024-09-21T21:40:52.691 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2042999999999997e-05 2024-09-21T21:40:52.691 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.691 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.691 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:52.691 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.553779+0000", 2024-09-21T21:40:52.691 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.3440000000000003e-06 2024-09-21T21:40:52.691 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.691 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.691 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:52.691 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.553784+0000", 2024-09-21T21:40:52.691 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3419999999999998e-06 2024-09-21T21:40:52.692 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.692 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.692 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:52.692 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.553831+0000", 2024-09-21T21:40:52.692 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6826000000000002e-05 2024-09-21T21:40:52.692 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.692 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.692 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:52.692 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.553914+0000", 2024-09-21T21:40:52.692 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.2816000000000001e-05 2024-09-21T21:40:52.692 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.692 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.692 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 1", 2024-09-21T21:40:52.692 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.554149+0000", 2024-09-21T21:40:52.692 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00023488900000000001 2024-09-21T21:40:52.693 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.693 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.693 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:52.693 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.558852+0000", 2024-09-21T21:40:52.693 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0047028399999999998 2024-09-21T21:40:52.693 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.693 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.693 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:40:52.693 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.562673+0000", 2024-09-21T21:40:52.693 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0038215929999999999 2024-09-21T21:40:52.693 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.693 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.693 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:52.693 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.562717+0000", 2024-09-21T21:40:52.693 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4255999999999998e-05 2024-09-21T21:40:52.693 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.694 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.694 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:52.694 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.562770+0000", 2024-09-21T21:40:52.694 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2689000000000003e-05 2024-09-21T21:40:52.694 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:52.694 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:52.694 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:52.694 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.694 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.694 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:228 3.6 3:678f83b3:::rbd_data.11214be7c411.000000000000002a:head [delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:52.694 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.553796+0000", 2024-09-21T21:40:52.694 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.0983621649999993, 2024-09-21T21:40:52.694 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0098649099999999993, 2024-09-21T21:40:52.694 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:52.694 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:52.695 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:52.695 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:52.695 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:52.695 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:52.695 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 228 2024-09-21T21:40:52.695 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.695 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:52.695 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.695 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:52.695 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.553796+0000", 2024-09-21T21:40:52.695 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:52.695 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.695 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.695 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:52.695 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.553796+0000", 2024-09-21T21:40:52.695 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:52.696 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.696 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.696 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:52.696 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.553797+0000", 2024-09-21T21:40:52.696 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6899999999999997e-07 2024-09-21T21:40:52.696 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.696 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.696 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:52.696 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.553799+0000", 2024-09-21T21:40:52.696 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3e-06 2024-09-21T21:40:52.696 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.696 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.696 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:52.696 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.553805+0000", 2024-09-21T21:40:52.697 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1310000000000003e-06 2024-09-21T21:40:52.697 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.697 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.697 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:52.697 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.553816+0000", 2024-09-21T21:40:52.697 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0506000000000001e-05 2024-09-21T21:40:52.697 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.697 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.697 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:52.697 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.555870+0000", 2024-09-21T21:40:52.697 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0020538510000000002 2024-09-21T21:40:52.697 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.697 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.697 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:52.698 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.555942+0000", 2024-09-21T21:40:52.698 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.2436999999999998e-05 2024-09-21T21:40:52.698 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.698 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.698 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 1", 2024-09-21T21:40:52.698 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556121+0000", 2024-09-21T21:40:52.698 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000178526 2024-09-21T21:40:52.698 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.698 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.698 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:52.698 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.559481+0000", 2024-09-21T21:40:52.698 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0033603019999999999 2024-09-21T21:40:52.698 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.698 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.698 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:40:52.698 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.563144+0000", 2024-09-21T21:40:52.699 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0036625799999999999 2024-09-21T21:40:52.699 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.699 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.699 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:52.699 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.563185+0000", 2024-09-21T21:40:52.699 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1084000000000002e-05 2024-09-21T21:40:52.699 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.699 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.699 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:52.699 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.563661+0000", 2024-09-21T21:40:52.699 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00047662399999999998 2024-09-21T21:40:52.699 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:52.699 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:52.699 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:52.699 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.699 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.700 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "pg_update_log_missing(3.b epoch 26/21 rep_tid 94 entries 26'3 (0'0) error 3:d29f5c36:::rbd_data.11214be7c411.000000000000003b:head by client.4417.0:236 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 26'1)", 2024-09-21T21:40:52.700 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.556653+0000", 2024-09-21T21:40:52.700 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.095505953, 2024-09-21T21:40:52.700 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.006055749, 2024-09-21T21:40:52.700 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:52.700 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:52.700 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:40:52.700 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:52.700 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.700 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:52.700 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556653+0000", 2024-09-21T21:40:52.700 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:52.700 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.700 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.700 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:52.700 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556653+0000", 2024-09-21T21:40:52.701 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:52.701 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.701 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.701 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:52.701 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556653+0000", 2024-09-21T21:40:52.701 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0900000000000001e-07 2024-09-21T21:40:52.701 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.701 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.701 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:52.701 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556661+0000", 2024-09-21T21:40:52.701 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.746e-06 2024-09-21T21:40:52.701 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.701 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.701 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:52.702 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556679+0000", 2024-09-21T21:40:52.702 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.8379000000000001e-05 2024-09-21T21:40:52.702 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.702 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.702 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:52.702 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556694+0000", 2024-09-21T21:40:52.702 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.4440000000000001e-05 2024-09-21T21:40:52.702 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.702 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.702 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:52.702 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556784+0000", 2024-09-21T21:40:52.702 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.9955000000000002e-05 2024-09-21T21:40:52.702 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.702 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.702 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:52.703 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.562708+0000", 2024-09-21T21:40:52.703 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00592452 2024-09-21T21:40:52.703 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:52.703 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:52.703 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:52.703 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.703 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.703 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_repop(client.4417.0:246 3.18 e26/21 3:18a859ac:::rbd_data.11214be7c411.0000000000000036:head v 26'6, mlcod=24'4)", 2024-09-21T21:40:52.703 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.557271+0000", 2024-09-21T21:40:52.703 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.0948871090000001, 2024-09-21T21:40:52.703 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0054572550000000003, 2024-09-21T21:40:52.703 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:52.703 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:52.704 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:52.704 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:52.704 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.704 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:52.704 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.557271+0000", 2024-09-21T21:40:52.704 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:52.704 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.704 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.704 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:52.704 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.557271+0000", 2024-09-21T21:40:52.704 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:52.704 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.704 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.704 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:52.704 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.557272+0000", 2024-09-21T21:40:52.705 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.06e-07 2024-09-21T21:40:52.705 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.705 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.705 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:52.705 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.557279+0000", 2024-09-21T21:40:52.705 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.4649999999999997e-06 2024-09-21T21:40:52.705 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.705 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.705 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:52.705 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.557289+0000", 2024-09-21T21:40:52.705 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.4190000000000008e-06 2024-09-21T21:40:52.705 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.705 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.705 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:52.705 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.557295+0000", 2024-09-21T21:40:52.705 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5950000000000004e-06 2024-09-21T21:40:52.706 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.706 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.706 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:52.706 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.557345+0000", 2024-09-21T21:40:52.706 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9208e-05 2024-09-21T21:40:52.706 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.706 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.706 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:52.706 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.557363+0000", 2024-09-21T21:40:52.706 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.8717000000000001e-05 2024-09-21T21:40:52.706 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.706 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.706 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:52.706 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.562693+0000", 2024-09-21T21:40:52.706 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0053298690000000001 2024-09-21T21:40:52.706 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.707 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.707 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:52.707 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.562729+0000", 2024-09-21T21:40:52.707 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5376000000000001e-05 2024-09-21T21:40:52.707 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:52.707 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:52.707 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:52.707 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.707 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.707 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:288 3.12 3:4db597f3:::rbd_data.11214be7c411.0000000000000048:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:52.707 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.565848+0000", 2024-09-21T21:40:52.707 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.0863101240000006, 2024-09-21T21:40:52.707 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0059913800000000001, 2024-09-21T21:40:52.707 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:52.707 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:52.707 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:40:52.708 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:52.708 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:52.708 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:52.708 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 288 2024-09-21T21:40:52.708 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.708 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:52.708 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.708 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:52.708 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.565848+0000", 2024-09-21T21:40:52.708 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:52.708 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.708 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.708 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:52.708 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.565848+0000", 2024-09-21T21:40:52.708 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:52.709 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.709 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.709 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:52.709 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.565849+0000", 2024-09-21T21:40:52.709 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0130000000000001e-06 2024-09-21T21:40:52.709 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.709 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.709 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:52.709 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.565854+0000", 2024-09-21T21:40:52.709 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.853e-06 2024-09-21T21:40:52.709 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.709 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.710 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:52.710 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.565867+0000", 2024-09-21T21:40:52.710 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.3208e-05 2024-09-21T21:40:52.710 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.710 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.710 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:52.710 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.565875+0000", 2024-09-21T21:40:52.710 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7840000000000004e-06 2024-09-21T21:40:52.710 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.710 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.710 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:52.710 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.565926+0000", 2024-09-21T21:40:52.710 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0346000000000002e-05 2024-09-21T21:40:52.710 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.710 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.710 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:52.711 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.566046+0000", 2024-09-21T21:40:52.711 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00012065000000000001 2024-09-21T21:40:52.711 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.711 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.711 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:52.711 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.571840+0000", 2024-09-21T21:40:52.711 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0057935260000000002 2024-09-21T21:40:52.711 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:52.711 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:52.711 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:52.711 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.711 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.711 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:309 3.1b 3:dd2a1785:::rbd_data.11214be7c411.000000000000004b:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:52.711 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.569250+0000", 2024-09-21T21:40:52.711 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.0829080599999994, 2024-09-21T21:40:52.712 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0051863389999999999, 2024-09-21T21:40:52.712 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:52.712 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:52.712 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:40:52.712 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:52.712 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:52.712 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:52.712 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 309 2024-09-21T21:40:52.712 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.712 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:52.712 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.712 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:52.712 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.569250+0000", 2024-09-21T21:40:52.712 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:52.712 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.712 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.713 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:52.713 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.569250+0000", 2024-09-21T21:40:52.713 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:52.713 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.713 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.713 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:52.713 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.569251+0000", 2024-09-21T21:40:52.713 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0399999999999995e-07 2024-09-21T21:40:52.713 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.713 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.713 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:52.713 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.569255+0000", 2024-09-21T21:40:52.713 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.121e-06 2024-09-21T21:40:52.713 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.714 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.714 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:52.714 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.569265+0000", 2024-09-21T21:40:52.714 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0068000000000001e-05 2024-09-21T21:40:52.714 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.714 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.714 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:52.714 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.569271+0000", 2024-09-21T21:40:52.714 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5999999999999997e-06 2024-09-21T21:40:52.714 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.714 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.714 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:52.714 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.569322+0000", 2024-09-21T21:40:52.714 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0776000000000003e-05 2024-09-21T21:40:52.714 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.715 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.715 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:52.715 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.569454+0000", 2024-09-21T21:40:52.715 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00013230699999999999 2024-09-21T21:40:52.715 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.715 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.715 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:52.715 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.574437+0000", 2024-09-21T21:40:52.715 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0049827630000000003 2024-09-21T21:40:52.715 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:52.715 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:52.715 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:52.715 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.715 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.715 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:344 3.1b 3:d9520152:::rbd_data.11214be7c411.000000000000004d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:52.716 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.576793+0000", 2024-09-21T21:40:52.716 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.0753658280000007, 2024-09-21T21:40:52.716 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0058611469999999997, 2024-09-21T21:40:52.716 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:52.716 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:52.716 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:52.716 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:52.716 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:52.716 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:52.716 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 344 2024-09-21T21:40:52.716 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.716 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:52.716 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.716 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:52.717 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.576793+0000", 2024-09-21T21:40:52.717 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:52.717 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.717 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.717 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:52.717 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.576793+0000", 2024-09-21T21:40:52.717 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:52.717 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.717 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.717 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:52.717 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.576793+0000", 2024-09-21T21:40:52.717 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.4499999999999996e-07 2024-09-21T21:40:52.717 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.717 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.717 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:52.718 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.576799+0000", 2024-09-21T21:40:52.718 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2299999999999999e-06 2024-09-21T21:40:52.718 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.718 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.718 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:52.718 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.576811+0000", 2024-09-21T21:40:52.718 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.2213e-05 2024-09-21T21:40:52.718 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.718 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.718 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:52.718 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.576819+0000", 2024-09-21T21:40:52.718 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7230000000000004e-06 2024-09-21T21:40:52.718 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.719 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.719 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:52.719 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.576865+0000", 2024-09-21T21:40:52.719 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5954000000000001e-05 2024-09-21T21:40:52.719 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.719 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.719 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:52.719 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.576946+0000", 2024-09-21T21:40:52.719 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.1689999999999999e-05 2024-09-21T21:40:52.719 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.719 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.719 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 4", 2024-09-21T21:40:52.719 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.577144+0000", 2024-09-21T21:40:52.720 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00019760599999999999 2024-09-21T21:40:52.720 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.720 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.720 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:40:52.720 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.579276+0000", 2024-09-21T21:40:52.720 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0021324009999999999 2024-09-21T21:40:52.720 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.720 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.720 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:52.720 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.582551+0000", 2024-09-21T21:40:52.720 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.003275086 2024-09-21T21:40:52.720 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.720 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.720 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:52.720 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.582610+0000", 2024-09-21T21:40:52.720 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8186999999999997e-05 2024-09-21T21:40:52.721 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.721 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.721 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:52.721 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.582654+0000", 2024-09-21T21:40:52.721 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4212000000000003e-05 2024-09-21T21:40:52.721 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:52.721 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:52.721 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:52.721 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.721 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.721 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:350 3.1b 3:db169869:::rbd_data.11214be7c411.0000000000000053:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:52.721 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.577617+0000", 2024-09-21T21:40:52.721 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.0745410230000001, 2024-09-21T21:40:52.721 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0053647620000000003, 2024-09-21T21:40:52.722 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:52.722 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:52.722 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:52.722 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:52.722 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:52.722 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:52.722 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 350 2024-09-21T21:40:52.722 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.722 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:52.722 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.722 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:52.722 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.577617+0000", 2024-09-21T21:40:52.722 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:52.722 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.722 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.723 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:52.723 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.577617+0000", 2024-09-21T21:40:52.723 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:52.723 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.723 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.723 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:52.723 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.577618+0000", 2024-09-21T21:40:52.723 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.8100000000000001e-07 2024-09-21T21:40:52.723 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.723 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.723 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:52.723 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.577623+0000", 2024-09-21T21:40:52.723 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8180000000000003e-06 2024-09-21T21:40:52.723 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.723 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.723 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:52.724 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.577641+0000", 2024-09-21T21:40:52.724 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.7986e-05 2024-09-21T21:40:52.724 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.724 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.724 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:52.724 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.577659+0000", 2024-09-21T21:40:52.724 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.8209000000000001e-05 2024-09-21T21:40:52.724 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.724 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.724 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:52.724 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.577838+0000", 2024-09-21T21:40:52.724 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00017856999999999999 2024-09-21T21:40:52.724 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.724 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.724 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:52.725 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.577886+0000", 2024-09-21T21:40:52.725 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8486999999999998e-05 2024-09-21T21:40:52.725 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.725 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.725 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 4", 2024-09-21T21:40:52.725 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.578110+0000", 2024-09-21T21:40:52.725 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000223277 2024-09-21T21:40:52.725 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.725 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.725 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:40:52.725 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.582200+0000", 2024-09-21T21:40:52.725 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0040905530000000002 2024-09-21T21:40:52.725 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.725 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.726 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:52.726 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.582886+0000", 2024-09-21T21:40:52.726 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00068611699999999998 2024-09-21T21:40:52.726 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.726 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.726 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:52.726 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.582943+0000", 2024-09-21T21:40:52.726 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6909e-05 2024-09-21T21:40:52.726 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.726 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.726 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:52.726 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.582982+0000", 2024-09-21T21:40:52.726 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8955000000000002e-05 2024-09-21T21:40:52.727 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:52.727 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:52.727 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:52.727 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.727 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.727 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:436 3.17 3:ea5ebfbf:::rbd_data.11214be7c411.000000000000006c:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:52.727 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.590210+0000", 2024-09-21T21:40:52.727 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.0619481989999997, 2024-09-21T21:40:52.727 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0083415569999999994, 2024-09-21T21:40:52.727 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:52.727 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:52.727 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:40:52.727 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:52.727 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:52.728 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:52.728 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 436 2024-09-21T21:40:52.728 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.728 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:52.728 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.728 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:52.728 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.590210+0000", 2024-09-21T21:40:52.728 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:52.728 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.728 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.728 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:52.728 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.590210+0000", 2024-09-21T21:40:52.728 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:52.728 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.728 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.729 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:52.729 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.590211+0000", 2024-09-21T21:40:52.729 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.8899999999999998e-07 2024-09-21T21:40:52.729 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.729 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.729 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:52.729 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.590222+0000", 2024-09-21T21:40:52.729 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0473e-05 2024-09-21T21:40:52.729 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.729 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.729 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:52.729 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.590235+0000", 2024-09-21T21:40:52.729 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.3003000000000001e-05 2024-09-21T21:40:52.729 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.730 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.730 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:52.730 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.590242+0000", 2024-09-21T21:40:52.730 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7749999999999993e-06 2024-09-21T21:40:52.730 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.730 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.730 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:52.730 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.590293+0000", 2024-09-21T21:40:52.730 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0133000000000003e-05 2024-09-21T21:40:52.730 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.730 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.730 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:52.730 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.590397+0000", 2024-09-21T21:40:52.730 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000104443 2024-09-21T21:40:52.731 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.731 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.731 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:52.731 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.598552+0000", 2024-09-21T21:40:52.731 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0081548409999999995 2024-09-21T21:40:52.731 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:52.731 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:52.731 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:52.731 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.731 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.731 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "pg_update_log_missing(3.3 epoch 26/21 rep_tid 23 entries 26'5 (0'0) error 3:c632f711:::rbd_data.11214be7c411.000000000000006f:head by client.4417.0:457 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 26'3)", 2024-09-21T21:40:52.731 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.593526+0000", 2024-09-21T21:40:52.731 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.0586327129999997, 2024-09-21T21:40:52.731 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.007353788, 2024-09-21T21:40:52.731 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:52.732 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:52.732 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:40:52.732 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:52.732 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.732 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:52.732 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593526+0000", 2024-09-21T21:40:52.732 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:52.732 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.732 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.732 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:52.732 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593526+0000", 2024-09-21T21:40:52.732 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:52.732 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.732 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.732 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:52.732 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593527+0000", 2024-09-21T21:40:52.733 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0389999999999999e-06 2024-09-21T21:40:52.733 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.733 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.733 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:52.733 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593535+0000", 2024-09-21T21:40:52.733 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.8909999999999995e-06 2024-09-21T21:40:52.733 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.733 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.733 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:52.733 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593556+0000", 2024-09-21T21:40:52.733 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0848000000000001e-05 2024-09-21T21:40:52.733 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.733 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.733 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:52.733 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593572+0000", 2024-09-21T21:40:52.733 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.6070999999999999e-05 2024-09-21T21:40:52.733 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.734 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.734 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:52.734 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593699+0000", 2024-09-21T21:40:52.734 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000127358 2024-09-21T21:40:52.734 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.734 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.734 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:52.734 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.600880+0000", 2024-09-21T21:40:52.734 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0071805810000000001 2024-09-21T21:40:52.734 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:52.734 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:52.734 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:52.734 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.734 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.734 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:466 3.1f 3:f854b118:::rbd_data.11214be7c411.0000000000000070:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:52.735 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.595543+0000", 2024-09-21T21:40:52.735 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.0566150000000007, 2024-09-21T21:40:52.735 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0053899910000000002, 2024-09-21T21:40:52.735 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:52.735 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:52.735 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:40:52.735 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:52.735 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:52.735 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:52.735 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 466 2024-09-21T21:40:52.735 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.735 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:52.735 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.735 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:52.735 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.595543+0000", 2024-09-21T21:40:52.736 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:52.736 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.736 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.736 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:52.736 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.595543+0000", 2024-09-21T21:40:52.736 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:52.736 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.736 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.736 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:52.736 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.595544+0000", 2024-09-21T21:40:52.736 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4700000000000001e-07 2024-09-21T21:40:52.736 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.736 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.736 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:52.736 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.595548+0000", 2024-09-21T21:40:52.737 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8759999999999998e-06 2024-09-21T21:40:52.737 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.737 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.737 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:52.737 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.595557+0000", 2024-09-21T21:40:52.737 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.833e-06 2024-09-21T21:40:52.737 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.737 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.737 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:52.737 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.595562+0000", 2024-09-21T21:40:52.737 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7520000000000004e-06 2024-09-21T21:40:52.737 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.737 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.737 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:52.738 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.595612+0000", 2024-09-21T21:40:52.738 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9960000000000003e-05 2024-09-21T21:40:52.738 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.738 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.738 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:52.738 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.595695+0000", 2024-09-21T21:40:52.738 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.3347e-05 2024-09-21T21:40:52.738 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.738 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.738 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:52.738 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.600933+0000", 2024-09-21T21:40:52.738 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.005238576 2024-09-21T21:40:52.738 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:52.738 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:52.739 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:52.739 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.739 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.739 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:475 3.17 3:ea5ebfbf:::rbd_data.11214be7c411.000000000000006c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:52.739 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.598715+0000", 2024-09-21T21:40:52.739 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.0534434350000002, 2024-09-21T21:40:52.739 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0051441300000000002, 2024-09-21T21:40:52.739 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:52.739 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:52.739 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:52.739 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:52.739 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:52.739 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:52.739 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 475 2024-09-21T21:40:52.739 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.740 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:52.740 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.740 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:52.740 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.598715+0000", 2024-09-21T21:40:52.740 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:52.740 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.740 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.740 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:52.740 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.598715+0000", 2024-09-21T21:40:52.740 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:52.740 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.740 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.740 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:52.740 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.598716+0000", 2024-09-21T21:40:52.740 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.4099999999999997e-07 2024-09-21T21:40:52.741 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.741 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.741 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:52.741 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.598721+0000", 2024-09-21T21:40:52.741 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2259999999999996e-06 2024-09-21T21:40:52.741 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.741 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.741 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:52.741 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.598735+0000", 2024-09-21T21:40:52.741 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.3838e-05 2024-09-21T21:40:52.741 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.741 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.741 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:52.741 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.598743+0000", 2024-09-21T21:40:52.741 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.4850000000000003e-06 2024-09-21T21:40:52.742 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.742 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.742 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:52.742 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.598798+0000", 2024-09-21T21:40:52.742 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5546e-05 2024-09-21T21:40:52.742 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.742 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.742 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:52.742 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.598844+0000", 2024-09-21T21:40:52.742 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5940999999999999e-05 2024-09-21T21:40:52.742 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.742 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.742 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:40:52.742 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.598957+0000", 2024-09-21T21:40:52.743 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000112871 2024-09-21T21:40:52.743 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.743 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.743 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:52.743 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.602463+0000", 2024-09-21T21:40:52.743 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0035060059999999999 2024-09-21T21:40:52.743 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.743 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.743 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:40:52.743 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.603772+0000", 2024-09-21T21:40:52.743 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.001309005 2024-09-21T21:40:52.743 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.743 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.743 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:52.743 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.603822+0000", 2024-09-21T21:40:52.744 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0365999999999999e-05 2024-09-21T21:40:52.744 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.744 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.744 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:52.744 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.603859+0000", 2024-09-21T21:40:52.744 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6905e-05 2024-09-21T21:40:52.744 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:52.744 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:52.744 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:52.744 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.744 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.744 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4471.0:2234 3.1f 3:fb6cf541:::rbd_data.1121dee34a14.000000000000000a:6 [read 335872~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-21T21:40:52.744 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:48.578066+0000", 2024-09-21T21:40:52.744 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 4.0740926599999998, 2024-09-21T21:40:52.744 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0052497209999999997, 2024-09-21T21:40:52.744 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:52.745 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:52.745 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:40:52.745 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:52.745 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4471", 2024-09-21T21:40:52.745 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1688278210", 2024-09-21T21:40:52.745 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2234 2024-09-21T21:40:52.745 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.745 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:52.745 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.745 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:52.745 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:48.578066+0000", 2024-09-21T21:40:52.745 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:52.745 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.745 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.745 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:52.745 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:48.578066+0000", 2024-09-21T21:40:52.746 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:52.746 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.746 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.746 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:52.746 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:48.578066+0000", 2024-09-21T21:40:52.746 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.2500000000000001e-07 2024-09-21T21:40:52.746 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.746 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.746 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:52.746 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:48.578068+0000", 2024-09-21T21:40:52.746 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.759e-06 2024-09-21T21:40:52.746 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.746 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.746 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:52.746 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:48.578074+0000", 2024-09-21T21:40:52.747 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6570000000000002e-06 2024-09-21T21:40:52.747 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.747 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.747 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:52.747 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:48.578078+0000", 2024-09-21T21:40:52.747 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4379999999999997e-06 2024-09-21T21:40:52.747 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.747 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.747 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:52.747 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:48.583151+0000", 2024-09-21T21:40:52.747 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0050734339999999999 2024-09-21T21:40:52.747 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.747 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.747 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:52.747 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:48.583232+0000", 2024-09-21T21:40:52.747 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0544999999999995e-05 2024-09-21T21:40:52.748 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:52.748 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:52.748 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:52.748 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:48.583316+0000", 2024-09-21T21:40:52.748 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.3562999999999999e-05 2024-09-21T21:40:52.748 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:52.748 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:52.748 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:52.748 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:52.748 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:52.748 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:40:52.749 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T21:40:52.750 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:40:52.748+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:40:52.850 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:40:52.849+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:40:52.855 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:40:52.855 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:40:52.855 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 0 2024-09-21T21:40:52.855 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:40:52.864 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T21:40:52.936 INFO:teuthology.orchestra.run.smithi164.stdout:checking clone #4, image image_client.0-clone4 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent5 2024-09-21T21:40:52.951 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:40:52.949+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:40:52.972 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:40:52.973 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:40:52.973 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:40:52.973 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:40:52.973 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:40:52.981 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T21:40:53.024 INFO:teuthology.orchestra.run.smithi164.stdout:559 write 0xdefa400 thru 0xdefade5 (0x9e6 bytes) 2024-09-21T21:40:53.039 INFO:teuthology.orchestra.run.smithi164.stdout:561 trunc from 0xdefae00 to 0x1c20200 2024-09-21T21:40:53.052 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:40:53.050+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:40:53.089 INFO:teuthology.orchestra.run.smithi164.stdout:563 read 0x133c400 thru 0x1348390 (0xbf91 bytes) 2024-09-21T21:40:53.093 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:40:53.093 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 20, 2024-09-21T21:40:53.093 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 600, 2024-09-21T21:40:53.093 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:40:53.093 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.093 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:124 3.2 3:42a46321:::rbd_data.11214be7c411.0000000000000020:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:53.093 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.534750+0000", 2024-09-21T21:40:53.093 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.5559877820000008, 2024-09-21T21:40:53.093 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0073162330000000001, 2024-09-21T21:40:53.093 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:53.094 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:53.094 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:40:53.094 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:53.094 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:53.094 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:53.094 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 124 2024-09-21T21:40:53.094 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.094 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:53.094 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.094 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:53.094 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.534750+0000", 2024-09-21T21:40:53.094 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:53.094 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.094 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.095 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:53.095 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.534750+0000", 2024-09-21T21:40:53.095 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:53.095 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.095 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.095 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:53.095 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.534751+0000", 2024-09-21T21:40:53.095 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4300000000000003e-07 2024-09-21T21:40:53.095 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.095 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.095 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:53.095 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.534753+0000", 2024-09-21T21:40:53.095 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.294e-06 2024-09-21T21:40:53.096 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.096 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.096 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:53.096 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.534759+0000", 2024-09-21T21:40:53.096 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3409999999999999e-06 2024-09-21T21:40:53.096 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.096 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.096 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:53.096 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.534764+0000", 2024-09-21T21:40:53.096 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9490000000000004e-06 2024-09-21T21:40:53.096 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.096 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.096 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:53.097 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.534817+0000", 2024-09-21T21:40:53.097 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2954000000000002e-05 2024-09-21T21:40:53.097 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.097 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.097 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:53.097 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.534924+0000", 2024-09-21T21:40:53.097 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000106884 2024-09-21T21:40:53.097 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.097 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.097 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:53.097 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.542067+0000", 2024-09-21T21:40:53.097 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0071422680000000002 2024-09-21T21:40:53.097 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:53.097 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:53.098 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:53.098 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.098 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.098 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "pg_update_log_missing(3.8 epoch 26/21 rep_tid 13 entries 26'4 (0'0) error 3:108e7c9d:::rbd_data.11214be7c411.000000000000001f:head by client.4417.0:122 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 25'2)", 2024-09-21T21:40:53.098 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.535157+0000", 2024-09-21T21:40:53.098 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.5555810480000005, 2024-09-21T21:40:53.098 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0068721269999999996, 2024-09-21T21:40:53.098 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:53.098 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:53.098 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:40:53.098 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:53.098 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.098 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:53.098 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.535157+0000", 2024-09-21T21:40:53.099 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:53.099 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.099 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.099 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:53.099 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.535157+0000", 2024-09-21T21:40:53.099 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:53.099 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.099 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.099 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:53.099 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.535158+0000", 2024-09-21T21:40:53.099 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7999999999999995e-07 2024-09-21T21:40:53.099 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.099 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.100 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:53.100 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.535165+0000", 2024-09-21T21:40:53.100 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.2869999999999999e-06 2024-09-21T21:40:53.100 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.100 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.100 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:53.100 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.535179+0000", 2024-09-21T21:40:53.100 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.4092e-05 2024-09-21T21:40:53.100 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.100 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.100 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:53.100 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.535192+0000", 2024-09-21T21:40:53.100 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.3107e-05 2024-09-21T21:40:53.100 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.100 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.101 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:53.101 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.535257+0000", 2024-09-21T21:40:53.101 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4543999999999997e-05 2024-09-21T21:40:53.101 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.101 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.101 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:53.101 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.542029+0000", 2024-09-21T21:40:53.101 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0067725169999999996 2024-09-21T21:40:53.101 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:53.101 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:53.101 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:53.101 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.101 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.101 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:147 3.19 3:9f7f40ef:::rbd_data.11214be7c411.000000000000001d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:53.102 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.536972+0000", 2024-09-21T21:40:53.102 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.5537657239999998, 2024-09-21T21:40:53.102 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0078089780000000003, 2024-09-21T21:40:53.102 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:53.102 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:53.102 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:53.102 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:53.102 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:53.102 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:53.102 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 147 2024-09-21T21:40:53.102 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.102 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:53.102 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.102 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:53.103 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.536972+0000", 2024-09-21T21:40:53.103 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:53.103 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.103 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.103 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:53.103 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.536972+0000", 2024-09-21T21:40:53.103 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:53.103 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.103 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.103 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:53.103 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.536973+0000", 2024-09-21T21:40:53.103 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.9999999999999998e-07 2024-09-21T21:40:53.103 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.103 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.104 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:53.104 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.536977+0000", 2024-09-21T21:40:53.104 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8850000000000001e-06 2024-09-21T21:40:53.104 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.104 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.104 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:53.104 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.536982+0000", 2024-09-21T21:40:53.104 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6590000000000004e-06 2024-09-21T21:40:53.104 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.104 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.104 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:53.104 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.536986+0000", 2024-09-21T21:40:53.104 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4999999999999999e-06 2024-09-21T21:40:53.104 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.105 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.105 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:53.105 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.539409+0000", 2024-09-21T21:40:53.105 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0024234259999999998 2024-09-21T21:40:53.105 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.105 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.105 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:53.105 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.539480+0000", 2024-09-21T21:40:53.105 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.1181999999999997e-05 2024-09-21T21:40:53.105 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.105 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.105 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 4", 2024-09-21T21:40:53.105 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.539597+0000", 2024-09-21T21:40:53.105 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000116239 2024-09-21T21:40:53.106 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.106 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.106 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:53.106 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.541542+0000", 2024-09-21T21:40:53.106 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.001945801 2024-09-21T21:40:53.106 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.106 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.106 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:40:53.106 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.544698+0000", 2024-09-21T21:40:53.106 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.003156042 2024-09-21T21:40:53.106 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.106 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.106 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:53.107 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.544747+0000", 2024-09-21T21:40:53.107 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8869000000000003e-05 2024-09-21T21:40:53.107 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.107 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.107 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:53.107 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.544781+0000", 2024-09-21T21:40:53.107 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3974999999999999e-05 2024-09-21T21:40:53.107 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:53.107 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:53.107 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:53.107 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.107 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.107 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "pg_update_log_missing(3.8 epoch 26/21 rep_tid 15 entries 26'5 (0'0) error 3:15b4ee09:::rbd_data.11214be7c411.0000000000000028:head by client.4417.0:154 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 25'2)", 2024-09-21T21:40:53.107 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.538536+0000", 2024-09-21T21:40:53.108 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.5522019290000006, 2024-09-21T21:40:53.108 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0061585449999999996, 2024-09-21T21:40:53.108 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:53.108 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:53.108 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:40:53.108 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:53.108 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.108 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:53.108 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.538536+0000", 2024-09-21T21:40:53.108 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:53.108 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.108 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.108 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:53.108 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.538536+0000", 2024-09-21T21:40:53.109 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:53.109 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.109 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.109 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:53.109 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.538537+0000", 2024-09-21T21:40:53.109 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9200000000000001e-07 2024-09-21T21:40:53.109 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.109 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.109 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:53.109 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.538547+0000", 2024-09-21T21:40:53.109 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0319999999999999e-05 2024-09-21T21:40:53.109 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.109 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.109 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:53.110 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.538565+0000", 2024-09-21T21:40:53.110 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.8080999999999998e-05 2024-09-21T21:40:53.110 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.110 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.110 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:53.110 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.538582+0000", 2024-09-21T21:40:53.110 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.7220000000000001e-05 2024-09-21T21:40:53.110 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.110 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.110 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:53.110 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.538644+0000", 2024-09-21T21:40:53.110 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1383999999999994e-05 2024-09-21T21:40:53.110 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.110 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.110 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:53.111 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.544695+0000", 2024-09-21T21:40:53.111 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0060509479999999996 2024-09-21T21:40:53.111 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:53.111 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:53.111 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:53.111 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.111 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.111 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:195 3.19 3:9c6b57eb:::rbd_object_map.11214be7c411.0000000000000004:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:53.111 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.546530+0000", 2024-09-21T21:40:53.111 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.5442076950000008, 2024-09-21T21:40:53.111 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0067606400000000001, 2024-09-21T21:40:53.111 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:53.111 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:53.111 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:53.112 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:53.112 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:53.112 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:53.112 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 195 2024-09-21T21:40:53.112 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.112 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:53.112 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.112 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:53.112 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.546530+0000", 2024-09-21T21:40:53.112 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:53.112 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.112 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.112 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:53.112 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.546530+0000", 2024-09-21T21:40:53.113 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:53.113 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.113 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.113 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:53.113 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.546531+0000", 2024-09-21T21:40:53.113 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1399999999999997e-07 2024-09-21T21:40:53.113 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.113 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.113 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:53.113 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.546534+0000", 2024-09-21T21:40:53.113 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.7989999999999998e-06 2024-09-21T21:40:53.113 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.113 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.113 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:53.114 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.546540+0000", 2024-09-21T21:40:53.114 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.314e-06 2024-09-21T21:40:53.114 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.114 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.114 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:53.114 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.546544+0000", 2024-09-21T21:40:53.114 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.546e-06 2024-09-21T21:40:53.114 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.114 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.114 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:53.114 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.547355+0000", 2024-09-21T21:40:53.114 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00081132899999999996 2024-09-21T21:40:53.114 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.114 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.115 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:53.115 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.547400+0000", 2024-09-21T21:40:53.115 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5627000000000002e-05 2024-09-21T21:40:53.115 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.115 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.115 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 4", 2024-09-21T21:40:53.115 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.547755+0000", 2024-09-21T21:40:53.115 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00035492899999999998 2024-09-21T21:40:53.115 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.115 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.115 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:53.115 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.549411+0000", 2024-09-21T21:40:53.115 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0016557239999999999 2024-09-21T21:40:53.115 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.116 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.116 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:40:53.116 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.553193+0000", 2024-09-21T21:40:53.116 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0037821370000000001 2024-09-21T21:40:53.116 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.116 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.116 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:53.116 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.553256+0000", 2024-09-21T21:40:53.116 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2534999999999997e-05 2024-09-21T21:40:53.116 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.116 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.116 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:53.116 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.553291+0000", 2024-09-21T21:40:53.116 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5185999999999998e-05 2024-09-21T21:40:53.117 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:53.117 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:53.117 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:53.117 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.117 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.117 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:236 3.b 3:d29f5c36:::rbd_data.11214be7c411.000000000000003b:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:53.117 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.556219+0000", 2024-09-21T21:40:53.117 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.5345193899999998, 2024-09-21T21:40:53.117 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0068065230000000001, 2024-09-21T21:40:53.117 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:53.117 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:53.117 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:40:53.117 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:53.117 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:53.118 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:53.118 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 236 2024-09-21T21:40:53.118 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.118 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:53.118 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.118 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:53.118 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556219+0000", 2024-09-21T21:40:53.118 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:53.118 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.118 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.118 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:53.118 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556219+0000", 2024-09-21T21:40:53.118 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:53.118 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.118 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.119 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:53.119 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556219+0000", 2024-09-21T21:40:53.119 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.9899999999999999e-07 2024-09-21T21:40:53.119 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.119 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.119 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:53.119 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556223+0000", 2024-09-21T21:40:53.119 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7910000000000002e-06 2024-09-21T21:40:53.119 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.119 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.119 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:53.119 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556235+0000", 2024-09-21T21:40:53.119 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1694e-05 2024-09-21T21:40:53.119 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.120 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.120 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:53.120 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556240+0000", 2024-09-21T21:40:53.120 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5199999999999997e-06 2024-09-21T21:40:53.120 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.120 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.120 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:53.120 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556300+0000", 2024-09-21T21:40:53.120 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9963000000000002e-05 2024-09-21T21:40:53.120 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.120 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.120 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:53.120 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556409+0000", 2024-09-21T21:40:53.120 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000108441 2024-09-21T21:40:53.121 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.121 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.121 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:53.121 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.563025+0000", 2024-09-21T21:40:53.121 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0066163150000000002 2024-09-21T21:40:53.121 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:53.121 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:53.121 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:53.121 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.121 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.121 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:252 3.0 3:01bf271b:::rbd_data.11214be7c411.0000000000000038:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:53.121 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.559092+0000", 2024-09-21T21:40:53.121 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.5316456279999997, 2024-09-21T21:40:53.121 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0066292950000000003, 2024-09-21T21:40:53.121 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:53.122 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:53.122 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:53.122 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:53.122 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:53.122 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:53.122 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 252 2024-09-21T21:40:53.122 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.122 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:53.122 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.122 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:53.122 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.559092+0000", 2024-09-21T21:40:53.122 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:53.122 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.122 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.123 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:53.123 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.559092+0000", 2024-09-21T21:40:53.123 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:53.123 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.123 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.123 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:53.123 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.559093+0000", 2024-09-21T21:40:53.123 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.1900000000000001e-07 2024-09-21T21:40:53.123 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.123 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.123 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:53.123 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.559098+0000", 2024-09-21T21:40:53.123 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6399999999999996e-06 2024-09-21T21:40:53.123 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.124 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.124 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:53.124 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.559108+0000", 2024-09-21T21:40:53.124 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6399999999999992e-06 2024-09-21T21:40:53.124 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.124 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.124 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:53.124 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.559121+0000", 2024-09-21T21:40:53.124 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.3687e-05 2024-09-21T21:40:53.124 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.124 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.124 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:53.124 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.559195+0000", 2024-09-21T21:40:53.124 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.3380999999999994e-05 2024-09-21T21:40:53.125 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.125 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.125 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:53.125 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.559273+0000", 2024-09-21T21:40:53.125 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.8236000000000001e-05 2024-09-21T21:40:53.125 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.125 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.125 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 2", 2024-09-21T21:40:53.125 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.559569+0000", 2024-09-21T21:40:53.125 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00029590600000000002 2024-09-21T21:40:53.125 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.125 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.125 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:40:53.125 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.563070+0000", 2024-09-21T21:40:53.126 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0035011880000000001 2024-09-21T21:40:53.126 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.126 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.126 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:53.126 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.565647+0000", 2024-09-21T21:40:53.126 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.002576877 2024-09-21T21:40:53.126 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.126 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.126 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:53.126 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.565692+0000", 2024-09-21T21:40:53.126 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.481e-05 2024-09-21T21:40:53.126 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.126 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.126 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:53.126 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.565722+0000", 2024-09-21T21:40:53.127 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0111e-05 2024-09-21T21:40:53.127 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:53.127 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:53.127 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:53.127 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.127 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.127 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:400 3.19 3:9c6b57eb:::rbd_object_map.11214be7c411.0000000000000004:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:53.127 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.586210+0000", 2024-09-21T21:40:53.127 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.504528101, 2024-09-21T21:40:53.127 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0068241939999999996, 2024-09-21T21:40:53.127 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:53.127 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:53.127 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:53.128 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:53.128 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:53.128 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:53.128 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 400 2024-09-21T21:40:53.128 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.128 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:53.128 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.128 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:53.128 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.586210+0000", 2024-09-21T21:40:53.128 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:53.128 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.128 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.128 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:53.128 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.586210+0000", 2024-09-21T21:40:53.128 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:53.129 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.129 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.129 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:53.129 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.586211+0000", 2024-09-21T21:40:53.129 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.92e-07 2024-09-21T21:40:53.129 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.129 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.129 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:53.129 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.586215+0000", 2024-09-21T21:40:53.129 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2910000000000001e-06 2024-09-21T21:40:53.129 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.129 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.129 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:53.129 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.586223+0000", 2024-09-21T21:40:53.130 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.3319999999999992e-06 2024-09-21T21:40:53.130 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.130 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.130 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:53.130 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.586230+0000", 2024-09-21T21:40:53.130 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5649999999999998e-06 2024-09-21T21:40:53.130 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.130 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.130 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:53.130 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.587427+0000", 2024-09-21T21:40:53.130 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00119736 2024-09-21T21:40:53.130 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.130 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.130 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:53.130 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.587499+0000", 2024-09-21T21:40:53.131 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.1439000000000005e-05 2024-09-21T21:40:53.131 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.131 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.131 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 4", 2024-09-21T21:40:53.131 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.587824+0000", 2024-09-21T21:40:53.131 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00032502200000000001 2024-09-21T21:40:53.131 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.131 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.131 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:40:53.131 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.589401+0000", 2024-09-21T21:40:53.131 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0015776270000000001 2024-09-21T21:40:53.131 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.131 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.131 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:53.131 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.592976+0000", 2024-09-21T21:40:53.132 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.003574565 2024-09-21T21:40:53.132 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.132 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.132 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:53.132 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593004+0000", 2024-09-21T21:40:53.132 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.7937999999999999e-05 2024-09-21T21:40:53.132 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.132 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.132 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:53.132 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593034+0000", 2024-09-21T21:40:53.132 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0263000000000001e-05 2024-09-21T21:40:53.132 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:53.132 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:53.132 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:53.133 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.133 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.133 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:410 3.19 3:981b6e71:::rbd_data.11214be7c411.0000000000000065:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:53.133 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.587078+0000", 2024-09-21T21:40:53.133 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.5036604170000007, 2024-09-21T21:40:53.133 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0060338199999999996, 2024-09-21T21:40:53.133 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:53.133 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:53.133 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:40:53.133 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:53.133 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:53.133 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:53.133 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 410 2024-09-21T21:40:53.134 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.134 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:53.134 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.134 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:53.134 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.587078+0000", 2024-09-21T21:40:53.134 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:53.134 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.134 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.134 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:53.134 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.587078+0000", 2024-09-21T21:40:53.134 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:53.134 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.134 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.134 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:53.135 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.587078+0000", 2024-09-21T21:40:53.135 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6000000000000004e-07 2024-09-21T21:40:53.135 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.135 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.135 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:53.135 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.587081+0000", 2024-09-21T21:40:53.135 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.9950000000000001e-06 2024-09-21T21:40:53.135 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.135 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.135 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:53.135 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.587091+0000", 2024-09-21T21:40:53.135 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6299999999999993e-06 2024-09-21T21:40:53.135 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.135 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.136 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:53.136 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.587097+0000", 2024-09-21T21:40:53.136 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8259999999999998e-06 2024-09-21T21:40:53.136 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.136 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.136 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:53.136 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.589045+0000", 2024-09-21T21:40:53.136 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0019484179999999999 2024-09-21T21:40:53.136 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.136 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.136 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:53.136 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.589153+0000", 2024-09-21T21:40:53.136 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000107627 2024-09-21T21:40:53.136 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.136 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.137 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:53.137 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593111+0000", 2024-09-21T21:40:53.137 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.003958764 2024-09-21T21:40:53.137 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:53.137 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:53.137 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:53.137 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.137 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.137 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:459 3.4 3:22094545:::rbd_data.11214be7c411.000000000000005c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:53.137 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.593149+0000", 2024-09-21T21:40:53.137 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.4975891699999995, 2024-09-21T21:40:53.137 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.010963733999999999, 2024-09-21T21:40:53.137 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:53.138 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:53.138 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:53.138 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:53.138 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:53.138 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:53.138 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 459 2024-09-21T21:40:53.138 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.138 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:53.138 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.138 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:53.138 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593149+0000", 2024-09-21T21:40:53.138 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:53.138 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.138 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.138 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:53.139 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593149+0000", 2024-09-21T21:40:53.139 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:53.139 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.139 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.139 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:53.139 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593150+0000", 2024-09-21T21:40:53.139 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4300000000000003e-07 2024-09-21T21:40:53.139 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.139 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.139 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:53.139 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593204+0000", 2024-09-21T21:40:53.139 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4868999999999999e-05 2024-09-21T21:40:53.139 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.139 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.140 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:53.140 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593213+0000", 2024-09-21T21:40:53.140 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.7479999999999999e-06 2024-09-21T21:40:53.140 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.140 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.140 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:53.140 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593217+0000", 2024-09-21T21:40:53.140 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.0409999999999999e-06 2024-09-21T21:40:53.140 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.140 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.140 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:53.140 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593249+0000", 2024-09-21T21:40:53.140 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.1897999999999999e-05 2024-09-21T21:40:53.140 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.141 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.141 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:53.141 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593312+0000", 2024-09-21T21:40:53.141 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2813999999999999e-05 2024-09-21T21:40:53.141 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.141 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.141 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 2", 2024-09-21T21:40:53.141 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593501+0000", 2024-09-21T21:40:53.141 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000188887 2024-09-21T21:40:53.141 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.141 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.141 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:40:53.141 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.594562+0000", 2024-09-21T21:40:53.141 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.001060975 2024-09-21T21:40:53.142 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.142 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.142 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:53.142 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.604018+0000", 2024-09-21T21:40:53.142 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0094560800000000004 2024-09-21T21:40:53.142 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.142 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.142 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:53.142 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.604074+0000", 2024-09-21T21:40:53.142 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6063999999999997e-05 2024-09-21T21:40:53.142 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.142 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.142 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:53.142 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.604113+0000", 2024-09-21T21:40:53.143 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8714999999999998e-05 2024-09-21T21:40:53.143 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:53.143 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:53.143 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:53.143 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.143 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.143 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:460 3.4 3:22094545:::rbd_data.11214be7c411.000000000000005c:head [delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:53.143 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.593227+0000", 2024-09-21T21:40:53.143 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.4975114220000005, 2024-09-21T21:40:53.143 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.01140237, 2024-09-21T21:40:53.143 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:53.143 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:53.143 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:53.143 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:53.144 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:53.144 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:53.144 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 460 2024-09-21T21:40:53.144 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.144 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:53.144 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.144 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:53.144 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593227+0000", 2024-09-21T21:40:53.144 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:53.144 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.144 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.144 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:53.144 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593227+0000", 2024-09-21T21:40:53.144 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:53.145 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.145 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.145 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:53.145 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593227+0000", 2024-09-21T21:40:53.145 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4000000000000002e-07 2024-09-21T21:40:53.145 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.145 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.145 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:53.145 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593229+0000", 2024-09-21T21:40:53.145 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1169999999999998e-06 2024-09-21T21:40:53.145 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.145 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.145 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:53.145 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593235+0000", 2024-09-21T21:40:53.145 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0510000000000002e-06 2024-09-21T21:40:53.146 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.146 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.146 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:53.146 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593248+0000", 2024-09-21T21:40:53.146 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.2403e-05 2024-09-21T21:40:53.146 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.146 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.146 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:53.146 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593895+0000", 2024-09-21T21:40:53.146 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000647358 2024-09-21T21:40:53.146 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.146 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.146 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:53.146 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593969+0000", 2024-09-21T21:40:53.147 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.4107e-05 2024-09-21T21:40:53.147 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.147 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.147 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 2", 2024-09-21T21:40:53.147 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.594139+0000", 2024-09-21T21:40:53.147 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000169457 2024-09-21T21:40:53.147 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.147 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.147 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:40:53.147 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.596005+0000", 2024-09-21T21:40:53.147 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00186608 2024-09-21T21:40:53.147 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.147 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.147 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:53.148 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.604537+0000", 2024-09-21T21:40:53.148 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.008531861 2024-09-21T21:40:53.148 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.148 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.148 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:53.148 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.604587+0000", 2024-09-21T21:40:53.148 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0544000000000003e-05 2024-09-21T21:40:53.148 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.148 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.148 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:53.148 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.604629+0000", 2024-09-21T21:40:53.148 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1851999999999998e-05 2024-09-21T21:40:53.148 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:53.148 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:53.149 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:53.149 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.149 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.149 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:463 3.19 3:981b6e71:::rbd_data.11214be7c411.0000000000000065:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:53.149 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.593716+0000", 2024-09-21T21:40:53.149 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.4970223409999992, 2024-09-21T21:40:53.149 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.006390326, 2024-09-21T21:40:53.149 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:53.149 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:53.149 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:53.149 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:53.149 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:53.149 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:53.150 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 463 2024-09-21T21:40:53.150 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.150 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:53.150 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.150 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:53.150 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593716+0000", 2024-09-21T21:40:53.150 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:53.150 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.150 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.150 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:53.150 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593716+0000", 2024-09-21T21:40:53.150 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:53.150 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.150 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.151 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:53.151 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593717+0000", 2024-09-21T21:40:53.151 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.09e-07 2024-09-21T21:40:53.151 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.151 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.151 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:53.151 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593729+0000", 2024-09-21T21:40:53.151 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.2211e-05 2024-09-21T21:40:53.151 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.151 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.151 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:53.151 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593739+0000", 2024-09-21T21:40:53.151 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.8470000000000008e-06 2024-09-21T21:40:53.151 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.152 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.152 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:53.152 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593745+0000", 2024-09-21T21:40:53.152 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5869999999999998e-06 2024-09-21T21:40:53.152 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.152 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.152 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:53.152 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.595440+0000", 2024-09-21T21:40:53.152 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.001694954 2024-09-21T21:40:53.152 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.152 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.152 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:53.152 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.595489+0000", 2024-09-21T21:40:53.152 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8782999999999997e-05 2024-09-21T21:40:53.152 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.152 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.153 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 4", 2024-09-21T21:40:53.153 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.595607+0000", 2024-09-21T21:40:53.153 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000117737 2024-09-21T21:40:53.153 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.153 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.153 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:40:53.153 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.596867+0000", 2024-09-21T21:40:53.153 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0012600210000000001 2024-09-21T21:40:53.153 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.153 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.153 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:53.153 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.600050+0000", 2024-09-21T21:40:53.153 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0031829979999999998 2024-09-21T21:40:53.153 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.154 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.154 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:53.154 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.600082+0000", 2024-09-21T21:40:53.154 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.2279e-05 2024-09-21T21:40:53.154 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.154 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.154 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:53.154 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.600106+0000", 2024-09-21T21:40:53.154 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.41e-05 2024-09-21T21:40:53.154 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:53.154 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:53.154 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:53.154 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.154 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.155 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:505 3.19 3:9c6b57eb:::rbd_object_map.11214be7c411.0000000000000004:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:53.155 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.605631+0000", 2024-09-21T21:40:53.155 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.4851069480000003, 2024-09-21T21:40:53.155 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0071505190000000001, 2024-09-21T21:40:53.155 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:53.155 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:53.155 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:53.155 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:53.155 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:53.155 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:53.155 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 505 2024-09-21T21:40:53.155 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.155 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:53.155 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.156 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:53.156 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605631+0000", 2024-09-21T21:40:53.156 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:53.156 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.156 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.156 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:53.156 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605631+0000", 2024-09-21T21:40:53.156 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:53.156 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.156 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.156 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:53.156 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605632+0000", 2024-09-21T21:40:53.156 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0900000000000001e-07 2024-09-21T21:40:53.156 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.156 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.157 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:53.157 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605636+0000", 2024-09-21T21:40:53.157 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8160000000000004e-06 2024-09-21T21:40:53.157 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.157 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.157 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:53.157 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605645+0000", 2024-09-21T21:40:53.157 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.0319999999999992e-06 2024-09-21T21:40:53.157 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.157 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.157 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:53.157 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605651+0000", 2024-09-21T21:40:53.157 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9529999999999996e-06 2024-09-21T21:40:53.157 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.157 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.158 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:53.158 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.606491+0000", 2024-09-21T21:40:53.158 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000840301 2024-09-21T21:40:53.158 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.158 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.158 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:53.158 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.606571+0000", 2024-09-21T21:40:53.158 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0507999999999996e-05 2024-09-21T21:40:53.158 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.158 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.158 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 4", 2024-09-21T21:40:53.158 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.606909+0000", 2024-09-21T21:40:53.158 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00033747300000000003 2024-09-21T21:40:53.159 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.159 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.159 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:53.159 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.608367+0000", 2024-09-21T21:40:53.159 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.001458188 2024-09-21T21:40:53.159 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.159 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.159 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:40:53.159 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.612683+0000", 2024-09-21T21:40:53.159 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0043163209999999997 2024-09-21T21:40:53.159 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.159 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.159 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:53.159 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.612732+0000", 2024-09-21T21:40:53.160 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8980999999999997e-05 2024-09-21T21:40:53.160 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.160 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.160 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:53.160 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.612782+0000", 2024-09-21T21:40:53.160 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9237000000000003e-05 2024-09-21T21:40:53.160 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:53.160 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:53.160 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:53.160 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.160 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.160 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_repop(client.4417.0:504 3.8 e26/21 3:10f57124:::rbd_object_map.11214be7c411:head v 26'13, mlcod=26'10)", 2024-09-21T21:40:53.160 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.607109+0000", 2024-09-21T21:40:53.161 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.4836287349999999, 2024-09-21T21:40:53.161 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0085878610000000005, 2024-09-21T21:40:53.161 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:53.161 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:53.161 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:53.161 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:53.161 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.161 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:53.161 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607109+0000", 2024-09-21T21:40:53.161 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:53.161 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.161 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.161 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:53.161 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607109+0000", 2024-09-21T21:40:53.161 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:53.161 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.162 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.162 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:53.162 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607110+0000", 2024-09-21T21:40:53.162 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.82e-07 2024-09-21T21:40:53.162 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.162 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.162 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:53.162 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607115+0000", 2024-09-21T21:40:53.162 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1730000000000003e-06 2024-09-21T21:40:53.162 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.162 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.162 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:53.162 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607123+0000", 2024-09-21T21:40:53.162 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7610000000000007e-06 2024-09-21T21:40:53.163 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.163 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.163 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:53.163 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607128+0000", 2024-09-21T21:40:53.163 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6420000000000004e-06 2024-09-21T21:40:53.163 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.163 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.163 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:53.163 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607302+0000", 2024-09-21T21:40:53.163 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00017391600000000001 2024-09-21T21:40:53.163 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.163 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.163 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:53.163 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607320+0000", 2024-09-21T21:40:53.164 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.7404000000000001e-05 2024-09-21T21:40:53.164 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.164 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.164 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:53.164 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.615646+0000", 2024-09-21T21:40:53.164 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0083265130000000007 2024-09-21T21:40:53.164 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.164 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.164 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:53.164 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.615697+0000", 2024-09-21T21:40:53.164 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0869999999999999e-05 2024-09-21T21:40:53.164 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:53.164 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:53.164 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:53.164 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.165 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.165 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:513 3.19 3:9c6b57eb:::rbd_object_map.11214be7c411.0000000000000004:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:53.165 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.607160+0000", 2024-09-21T21:40:53.165 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.4835781509999997, 2024-09-21T21:40:53.165 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0086296150000000002, 2024-09-21T21:40:53.165 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:53.165 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:53.165 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:53.165 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:53.165 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:53.165 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:53.165 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 513 2024-09-21T21:40:53.165 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.165 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:53.165 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.166 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:53.166 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607160+0000", 2024-09-21T21:40:53.166 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:53.166 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.166 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.166 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:53.166 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607160+0000", 2024-09-21T21:40:53.166 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:53.166 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.166 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.166 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:53.166 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607161+0000", 2024-09-21T21:40:53.166 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.9399999999999993e-07 2024-09-21T21:40:53.166 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.166 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.167 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:53.167 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607165+0000", 2024-09-21T21:40:53.167 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3200000000000001e-06 2024-09-21T21:40:53.167 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.167 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.167 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:53.167 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607365+0000", 2024-09-21T21:40:53.167 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00019965799999999999 2024-09-21T21:40:53.167 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.167 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.167 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:53.167 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607374+0000", 2024-09-21T21:40:53.167 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.312e-06 2024-09-21T21:40:53.168 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.168 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.168 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:53.168 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607416+0000", 2024-09-21T21:40:53.168 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1962000000000002e-05 2024-09-21T21:40:53.168 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.168 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.168 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:53.168 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607485+0000", 2024-09-21T21:40:53.168 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9274000000000001e-05 2024-09-21T21:40:53.168 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.168 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.168 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 4", 2024-09-21T21:40:53.168 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607818+0000", 2024-09-21T21:40:53.169 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00033211299999999998 2024-09-21T21:40:53.169 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.169 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.169 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:53.169 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.608616+0000", 2024-09-21T21:40:53.169 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00079832700000000004 2024-09-21T21:40:53.169 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.169 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.169 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:40:53.169 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.615697+0000", 2024-09-21T21:40:53.169 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0070815849999999996 2024-09-21T21:40:53.169 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.169 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.169 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:53.169 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.615750+0000", 2024-09-21T21:40:53.170 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2778000000000002e-05 2024-09-21T21:40:53.170 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.170 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.170 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:53.170 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.615790+0000", 2024-09-21T21:40:53.170 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.9292000000000003e-05 2024-09-21T21:40:53.170 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:53.170 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:53.170 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:53.170 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.170 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.170 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_repop(client.4417.0:507 3.8 e26/21 3:10f57124:::rbd_object_map.11214be7c411:head v 26'14, mlcod=26'10)", 2024-09-21T21:40:53.170 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.607866+0000", 2024-09-21T21:40:53.171 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.4828724209999997, 2024-09-21T21:40:53.171 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0078638389999999992, 2024-09-21T21:40:53.171 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:53.171 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:53.171 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:53.171 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:53.171 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.171 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:53.171 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607866+0000", 2024-09-21T21:40:53.171 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:53.171 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.171 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.171 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:53.171 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607866+0000", 2024-09-21T21:40:53.171 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:53.172 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.172 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.172 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:53.172 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607866+0000", 2024-09-21T21:40:53.172 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6000000000000004e-07 2024-09-21T21:40:53.172 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.172 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.172 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:53.172 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607872+0000", 2024-09-21T21:40:53.172 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4820000000000002e-06 2024-09-21T21:40:53.172 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.172 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.172 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:53.172 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607882+0000", 2024-09-21T21:40:53.173 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0481e-05 2024-09-21T21:40:53.173 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.173 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.173 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:53.173 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607889+0000", 2024-09-21T21:40:53.173 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.3089999999999998e-06 2024-09-21T21:40:53.173 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.173 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.173 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:53.173 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607958+0000", 2024-09-21T21:40:53.173 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.8424999999999996e-05 2024-09-21T21:40:53.173 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.173 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.173 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:53.174 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607976+0000", 2024-09-21T21:40:53.174 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.7765e-05 2024-09-21T21:40:53.174 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.174 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.174 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:53.174 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.615700+0000", 2024-09-21T21:40:53.174 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0077241799999999998 2024-09-21T21:40:53.174 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.174 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.174 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:53.174 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.615729+0000", 2024-09-21T21:40:53.174 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.9637000000000001e-05 2024-09-21T21:40:53.174 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:53.174 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:53.174 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:53.174 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.175 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.175 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_repop(client.4417.0:508 3.8 e26/21 3:10f57124:::rbd_object_map.11214be7c411:head v 26'15, mlcod=26'10)", 2024-09-21T21:40:53.175 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.608535+0000", 2024-09-21T21:40:53.175 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.4822025990000007, 2024-09-21T21:40:53.175 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0072268009999999997, 2024-09-21T21:40:53.175 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:53.175 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:53.175 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:53.175 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:53.175 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.175 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:53.175 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.608535+0000", 2024-09-21T21:40:53.175 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:53.176 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.176 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.176 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:53.176 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.608535+0000", 2024-09-21T21:40:53.176 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:53.176 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.176 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.176 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:53.176 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.608536+0000", 2024-09-21T21:40:53.176 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2600000000000002e-07 2024-09-21T21:40:53.176 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.176 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.176 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:53.177 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.608540+0000", 2024-09-21T21:40:53.177 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4530000000000004e-06 2024-09-21T21:40:53.177 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.177 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.177 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:53.177 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.608549+0000", 2024-09-21T21:40:53.177 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.3270000000000001e-06 2024-09-21T21:40:53.177 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.177 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.177 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:53.177 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.608555+0000", 2024-09-21T21:40:53.177 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1009999999999997e-06 2024-09-21T21:40:53.178 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.178 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.178 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:53.178 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.608621+0000", 2024-09-21T21:40:53.178 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6351000000000002e-05 2024-09-21T21:40:53.178 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.178 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.178 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:53.178 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.608640+0000", 2024-09-21T21:40:53.178 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.8419000000000001e-05 2024-09-21T21:40:53.178 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.178 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.178 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:53.178 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.615731+0000", 2024-09-21T21:40:53.179 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.007091226 2024-09-21T21:40:53.179 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.179 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.179 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:53.179 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.615762+0000", 2024-09-21T21:40:53.179 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.1398e-05 2024-09-21T21:40:53.179 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:53.179 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:53.179 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:53.179 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.179 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.179 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_repop(client.4417.0:517 3.1a e26/21 3:5aa0c892:::rbd_data.11214be7c411.000000000000007a:head v 26'10, mlcod=26'8)", 2024-09-21T21:40:53.179 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.608704+0000", 2024-09-21T21:40:53.179 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.4820344740000007, 2024-09-21T21:40:53.180 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0071660500000000002, 2024-09-21T21:40:53.180 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:53.180 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:53.180 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:53.180 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:53.180 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.180 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:53.180 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.608704+0000", 2024-09-21T21:40:53.180 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:53.180 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.180 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.180 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:53.180 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.608704+0000", 2024-09-21T21:40:53.180 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:53.181 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.181 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.181 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:53.181 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.608704+0000", 2024-09-21T21:40:53.181 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2099999999999997e-07 2024-09-21T21:40:53.181 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.181 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.181 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:53.181 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.608709+0000", 2024-09-21T21:40:53.181 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8489999999999997e-06 2024-09-21T21:40:53.181 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.181 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.181 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:53.181 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.608719+0000", 2024-09-21T21:40:53.182 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0402e-05 2024-09-21T21:40:53.182 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.182 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.182 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:53.182 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.608726+0000", 2024-09-21T21:40:53.182 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.72e-06 2024-09-21T21:40:53.182 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.182 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.182 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:53.182 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.608766+0000", 2024-09-21T21:40:53.182 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.9745e-05 2024-09-21T21:40:53.182 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.182 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.182 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:53.183 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.608785+0000", 2024-09-21T21:40:53.183 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.8919e-05 2024-09-21T21:40:53.183 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.183 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.183 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:53.183 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.615831+0000", 2024-09-21T21:40:53.183 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0070464969999999997 2024-09-21T21:40:53.183 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.183 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.183 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:53.183 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.615870+0000", 2024-09-21T21:40:53.183 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8396999999999998e-05 2024-09-21T21:40:53.183 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:53.183 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:53.184 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:53.184 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.184 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.184 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_repop(client.4417.0:509 3.8 e26/21 3:10f57124:::rbd_object_map.11214be7c411:head v 26'16, mlcod=26'10)", 2024-09-21T21:40:53.184 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.609103+0000", 2024-09-21T21:40:53.184 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.4816350719999996, 2024-09-21T21:40:53.184 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0066864489999999997, 2024-09-21T21:40:53.184 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:53.184 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:53.184 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:53.184 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:53.184 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.184 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:53.184 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.609103+0000", 2024-09-21T21:40:53.184 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:53.185 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.185 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.185 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:53.185 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.609103+0000", 2024-09-21T21:40:53.185 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:53.185 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.185 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.185 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:53.185 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.609103+0000", 2024-09-21T21:40:53.185 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5999999999999999e-07 2024-09-21T21:40:53.185 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.185 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.185 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:53.185 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.609108+0000", 2024-09-21T21:40:53.185 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0100000000000003e-06 2024-09-21T21:40:53.186 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.186 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.186 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:53.186 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.609115+0000", 2024-09-21T21:40:53.186 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2319999999999998e-06 2024-09-21T21:40:53.186 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.186 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.186 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:53.186 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.609120+0000", 2024-09-21T21:40:53.186 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4199999999999998e-06 2024-09-21T21:40:53.186 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.186 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.186 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:53.186 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.609156+0000", 2024-09-21T21:40:53.186 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6269999999999997e-05 2024-09-21T21:40:53.186 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.187 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.187 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:53.187 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.609176+0000", 2024-09-21T21:40:53.187 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9666000000000001e-05 2024-09-21T21:40:53.187 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.187 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.187 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:53.187 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.615764+0000", 2024-09-21T21:40:53.187 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0065882989999999997 2024-09-21T21:40:53.187 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.187 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.187 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:53.187 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.615789+0000", 2024-09-21T21:40:53.187 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.5092e-05 2024-09-21T21:40:53.187 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:53.188 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:53.188 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:53.188 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.188 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.188 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_repop(client.4417.0:510 3.8 e26/21 3:10f57124:::rbd_object_map.11214be7c411:head v 26'17, mlcod=26'10)", 2024-09-21T21:40:53.188 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.609673+0000", 2024-09-21T21:40:53.188 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.4810649599999994, 2024-09-21T21:40:53.188 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0063224570000000001, 2024-09-21T21:40:53.188 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:53.188 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:53.188 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:53.188 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:53.188 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.188 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:53.188 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.609673+0000", 2024-09-21T21:40:53.188 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:53.189 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.189 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.189 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:53.189 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.609673+0000", 2024-09-21T21:40:53.189 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:53.189 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.189 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.189 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:53.189 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.609674+0000", 2024-09-21T21:40:53.189 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4799999999999999e-07 2024-09-21T21:40:53.189 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.189 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.189 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:53.190 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.609677+0000", 2024-09-21T21:40:53.190 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8419999999999998e-06 2024-09-21T21:40:53.190 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.190 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.190 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:53.190 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.609684+0000", 2024-09-21T21:40:53.190 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6100000000000002e-06 2024-09-21T21:40:53.190 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.190 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.190 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:53.190 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.609689+0000", 2024-09-21T21:40:53.190 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8450000000000002e-06 2024-09-21T21:40:53.190 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.190 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.191 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:53.191 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.609733+0000", 2024-09-21T21:40:53.191 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3881999999999998e-05 2024-09-21T21:40:53.191 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.191 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.191 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:53.191 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.609752+0000", 2024-09-21T21:40:53.191 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9743000000000001e-05 2024-09-21T21:40:53.191 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.191 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.191 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:53.191 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.615791+0000", 2024-09-21T21:40:53.191 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0060380570000000003 2024-09-21T21:40:53.191 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.192 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.192 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:53.192 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.615996+0000", 2024-09-21T21:40:53.192 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00020503000000000001 2024-09-21T21:40:53.192 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:53.192 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:53.192 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:53.192 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:53.192 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:53.192 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:40:53.194 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T21:40:53.195 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:40:53.192+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:40:53.196 INFO:teuthology.orchestra.run.smithi164.stdout:564 write 0x76f0400 thru 0x76f208b (0x1c8c bytes) 2024-09-21T21:40:53.196 INFO:teuthology.orchestra.run.smithi164.stdout:570 write 0xe09d000 thru 0xe0a878c (0xb78d bytes) 2024-09-21T21:40:53.196 INFO:teuthology.orchestra.run.smithi164.stdout:572 punch from 0x548ce00 to 0x548dc00, (0xe00 bytes) 2024-09-21T21:40:53.196 INFO:teuthology.orchestra.run.smithi164.stdout:573 read 0xcc56200 thru 0xcc58653 (0x2454 bytes) 2024-09-21T21:40:53.196 INFO:teuthology.orchestra.run.smithi164.stdout:575 read 0xaca6800 thru 0xacaf809 (0x900a bytes) 2024-09-21T21:40:53.196 INFO:teuthology.orchestra.run.smithi164.stdout:577 read 0x62e3c00 thru 0x62eeb92 (0xaf93 bytes) 2024-09-21T21:40:53.196 INFO:teuthology.orchestra.run.smithi164.stdout:579 read 0x9043400 thru 0x9052fc8 (0xfbc9 bytes) 2024-09-21T21:40:53.196 INFO:teuthology.orchestra.run.smithi164.stdout:582 write 0x893fe00 thru 0x8940aab (0xcac bytes) 2024-09-21T21:40:53.196 INFO:teuthology.orchestra.run.smithi164.stdout:584 read 0x5c9d600 thru 0x5caa837 (0xd238 bytes) 2024-09-21T21:40:53.196 INFO:teuthology.orchestra.run.smithi164.stdout:585 read 0xca18400 thru 0xca1e120 (0x5d21 bytes) 2024-09-21T21:40:53.196 INFO:teuthology.orchestra.run.smithi164.stdout:589 read 0x3d19400 thru 0x3d195e0 (0x1e1 bytes) 2024-09-21T21:40:53.196 INFO:teuthology.orchestra.run.smithi164.stdout:591 punch from 0x20bc600 to 0x20cc000, (0xfa00 bytes) 2024-09-21T21:40:53.197 INFO:teuthology.orchestra.run.smithi164.stdout:592 write 0xd4e8200 thru 0xd4ec391 (0x4192 bytes) 2024-09-21T21:40:53.197 INFO:teuthology.orchestra.run.smithi164.stdout:593 write 0xad85600 thru 0xad8694d (0x134e bytes) 2024-09-21T21:40:53.197 INFO:teuthology.orchestra.run.smithi164.stdout:595 write 0x5565e00 thru 0x5569f14 (0x4115 bytes) 2024-09-21T21:40:53.197 INFO:teuthology.orchestra.run.smithi164.stdout:596 trunc from 0xe0a8800 to 0x1a8bc00 2024-09-21T21:40:53.197 INFO:teuthology.orchestra.run.smithi164.stdout:597 write 0x22dd600 thru 0x22e2d3a (0x573b bytes) 2024-09-21T21:40:53.205 INFO:teuthology.orchestra.run.smithi164.stdout:599 read 0x107e00 thru 0x112cb3 (0xaeb4 bytes) 2024-09-21T21:40:53.205 INFO:teuthology.orchestra.run.smithi164.stdout:601 read 0x19c6200 thru 0x19cb6ee (0x54ef bytes) 2024-09-21T21:40:53.220 INFO:teuthology.orchestra.run.smithi164.stdout:602 trunc from 0x22e2e00 to 0xdff3200 2024-09-21T21:40:53.242 INFO:teuthology.orchestra.run.smithi164.stdout:603 trunc from 0xdff3200 to 0x314a600 2024-09-21T21:40:53.262 INFO:teuthology.orchestra.run.smithi164.stdout:604 write 0x6b43000 thru 0x6b4d193 (0xa194 bytes) 2024-09-21T21:40:53.265 INFO:teuthology.orchestra.run.smithi164.stdout:606 read 0x4a28200 thru 0x4a30852 (0x8653 bytes) 2024-09-21T21:40:53.265 INFO:teuthology.orchestra.run.smithi164.stdout:607 trunc from 0x6b4d200 to 0x4a5ac00 2024-09-21T21:40:53.280 INFO:teuthology.orchestra.run.smithi164.stdout:608 read 0x3492e00 thru 0x349f84b (0xca4c bytes) 2024-09-21T21:40:53.281 INFO:teuthology.orchestra.run.smithi164.stdout:611 punch from 0x3fa6c00 to 0x3fad000, (0x6400 bytes) 2024-09-21T21:40:53.281 INFO:teuthology.orchestra.run.smithi164.stdout:613 read 0x4996c00 thru 0x49a4d4e (0xe14f bytes) 2024-09-21T21:40:53.294 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:40:53.293+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:40:53.295 INFO:teuthology.orchestra.run.smithi164.stdout:615 write 0x5aee000 thru 0x5af4975 (0x6976 bytes) 2024-09-21T21:40:53.296 INFO:teuthology.orchestra.run.smithi164.stdout:618 punch from 0x59d6c00 to 0x59de800, (0x7c00 bytes) 2024-09-21T21:40:53.305 INFO:teuthology.orchestra.run.smithi164.stdout:619 trunc from 0x5af4a00 to 0x1d72800 2024-09-21T21:40:53.322 INFO:teuthology.orchestra.run.smithi164.stdout:620 punch from 0xdc5200 to 0xdd2600, (0xd400 bytes) 2024-09-21T21:40:53.324 INFO:teuthology.orchestra.run.smithi164.stdout:621 read 0x342600 thru 0x345047 (0x2a48 bytes) 2024-09-21T21:40:53.326 INFO:teuthology.orchestra.run.smithi164.stdout:622 read 0x2ad400 thru 0x2b345e (0x605f bytes) 2024-09-21T21:40:53.326 INFO:teuthology.orchestra.run.smithi164.stdout:623 write 0x175da00 thru 0x176130a (0x390b bytes) 2024-09-21T21:40:53.343 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:40:53.343 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:40:53.343 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 0 2024-09-21T21:40:53.343 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:40:53.348 INFO:teuthology.orchestra.run.smithi164.stdout:624 write 0xc606a00 thru 0xc6168e8 (0xfee9 bytes) 2024-09-21T21:40:53.352 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T21:40:53.363 INFO:teuthology.orchestra.run.smithi164.stdout:625 write 0xcb6ea00 thru 0xcb7bfc7 (0xd5c8 bytes) 2024-09-21T21:40:53.368 INFO:teuthology.orchestra.run.smithi164.stdout:626 punch from 0x74f0000 to 0x74fbe00, (0xbe00 bytes) 2024-09-21T21:40:53.368 INFO:teuthology.orchestra.run.smithi164.stdout:627 write 0x2024400 thru 0x202a6b8 (0x62b9 bytes) 2024-09-21T21:40:53.369 INFO:teuthology.orchestra.run.smithi164.stdout:628 write 0x7139c00 thru 0x713f1a9 (0x55aa bytes) 2024-09-21T21:40:53.369 INFO:teuthology.orchestra.run.smithi164.stdout:629 punch from 0x117f600 to 0x1187000, (0x7a00 bytes) 2024-09-21T21:40:53.370 INFO:teuthology.orchestra.run.smithi164.stdout:632 write 0x7694c00 thru 0x7695e7b (0x127c bytes) 2024-09-21T21:40:53.370 INFO:teuthology.orchestra.run.smithi164.stdout:637 read 0xa9d3800 thru 0xa9d86d7 (0x4ed8 bytes) 2024-09-21T21:40:53.373 INFO:teuthology.orchestra.run.smithi164.stdout:640 trunc from 0xcb7c000 to 0x9a76c00 2024-09-21T21:40:53.388 INFO:teuthology.orchestra.run.smithi164.stdout:641 read 0x3621000 thru 0x3622c49 (0x1c4a bytes) 2024-09-21T21:40:53.389 INFO:teuthology.orchestra.run.smithi164.stdout:645 read 0x4d61c00 thru 0x4d6ca32 (0xae33 bytes) 2024-09-21T21:40:53.389 INFO:teuthology.orchestra.run.smithi164.stdout:646 write 0x8ff9000 thru 0x90048bb (0xb8bc bytes) 2024-09-21T21:40:53.389 INFO:teuthology.orchestra.run.smithi164.stdout:649 read 0x8008200 thru 0x800ef6e (0x6d6f bytes) 2024-09-21T21:40:53.389 INFO:teuthology.orchestra.run.smithi164.stdout:651 read 0x5264000 thru 0x5269310 (0x5311 bytes) 2024-09-21T21:40:53.394 INFO:teuthology.orchestra.run.smithi164.stdout:652 trunc from 0x9a76c00 to 0x56d6e00 2024-09-21T21:40:53.395 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:40:53.393+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:40:53.426 INFO:teuthology.orchestra.run.smithi164.stdout:655 write 0xbd36600 thru 0xbd37423 (0xe24 bytes) 2024-09-21T21:40:53.427 INFO:teuthology.orchestra.run.smithi164.stdout:656 write 0x17c4a00 thru 0x17d1dd3 (0xd3d4 bytes) 2024-09-21T21:40:53.428 INFO:teuthology.orchestra.run.smithi164.stdout:658 write 0x6dbec00 thru 0x6dc13fa (0x27fb bytes) 2024-09-21T21:40:53.433 INFO:teuthology.orchestra.run.smithi164.stdout:659 trunc from 0xbd37600 to 0x750b400 2024-09-21T21:40:53.457 INFO:teuthology.orchestra.run.smithi164.stdout:661 read 0xe53000 thru 0xe5e5e9 (0xb5ea bytes) 2024-09-21T21:40:53.458 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:40:53.459 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:40:53.459 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:40:53.459 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:40:53.459 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:40:53.459 INFO:teuthology.orchestra.run.smithi164.stdout:662 read 0x65bc00 thru 0x65bc2e (0x2f bytes) 2024-09-21T21:40:53.460 INFO:teuthology.orchestra.run.smithi164.stdout:663 write 0xf30a00 thru 0xf33e7b (0x347c bytes) 2024-09-21T21:40:53.460 INFO:teuthology.orchestra.run.smithi164.stdout:664 punch from 0x58a200 to 0x593000, (0x8e00 bytes) 2024-09-21T21:40:53.460 INFO:teuthology.orchestra.run.smithi164.stdout:665 read 0x12b200 thru 0x137ca7 (0xcaa8 bytes) 2024-09-21T21:40:53.468 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T21:40:53.469 INFO:teuthology.orchestra.run.smithi164.stdout:666 trunc from 0x750b400 to 0xd688600 2024-09-21T21:40:53.483 INFO:teuthology.orchestra.run.smithi164.stdout:667 trunc from 0xd688600 to 0xd504200 2024-09-21T21:40:53.493 INFO:teuthology.orchestra.run.smithi164.stdout:668 read 0x196d200 thru 0x19749df (0x77e0 bytes) 2024-09-21T21:40:53.495 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:40:53.494+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:40:53.498 INFO:teuthology.orchestra.run.smithi164.stdout:672 read 0xf98400 thru 0xfa5a4f (0xd650 bytes) 2024-09-21T21:40:53.498 INFO:teuthology.orchestra.run.smithi164.stdout:673 write 0x34c3e00 thru 0x34c8bde (0x4ddf bytes) 2024-09-21T21:40:53.498 INFO:teuthology.orchestra.run.smithi164.stdout:676 read 0x527600 thru 0x527861 (0x262 bytes) 2024-09-21T21:40:53.499 INFO:teuthology.orchestra.run.smithi164.stdout:678 read 0x3f57200 thru 0x3f5ffe8 (0x8de9 bytes) 2024-09-21T21:40:53.499 INFO:teuthology.orchestra.run.smithi164.stdout:679 write 0x326a600 thru 0x326e32d (0x3d2e bytes) 2024-09-21T21:40:53.500 INFO:teuthology.orchestra.run.smithi164.stdout:681 trunc from 0xd504200 to 0xc80ec00 2024-09-21T21:40:53.501 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T21:40:53.512 INFO:teuthology.orchestra.run.smithi164.stdout:682 trunc from 0xc80ec00 to 0xb738000 2024-09-21T21:40:53.528 INFO:teuthology.orchestra.run.smithi164.stdout:684 trunc from 0xb738000 to 0xe1f3a00 2024-09-21T21:40:53.538 INFO:teuthology.orchestra.run.smithi164.stdout:685 write 0x57e3e00 thru 0x57f0cb0 (0xceb1 bytes) 2024-09-21T21:40:53.542 INFO:teuthology.orchestra.run.smithi164.stdout:686 trunc from 0xe1f3a00 to 0xc964200 2024-09-21T21:40:53.560 INFO:teuthology.orchestra.run.smithi164.stdout:688 trunc from 0xc964200 to 0xe64cc00 2024-09-21T21:40:53.567 INFO:teuthology.orchestra.run.smithi164.stdout:691 write 0x1c3dc00 thru 0x1c3e799 (0xb9a bytes) 2024-09-21T21:40:53.568 INFO:teuthology.orchestra.run.smithi164.stdout:693 read 0xd67c400 thru 0xd6896bf (0xd2c0 bytes) 2024-09-21T21:40:53.573 INFO:teuthology.orchestra.run.smithi164.stdout:694 trunc from 0xe64cc00 to 0xa079e00 2024-09-21T21:40:53.576 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:40:53.576 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 20, 2024-09-21T21:40:53.576 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 600, 2024-09-21T21:40:53.576 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:40:53.576 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.577 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:122 3.8 3:108e7c9d:::rbd_data.11214be7c411.000000000000001f:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:53.577 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.534714+0000", 2024-09-21T21:40:53.577 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 10.039218447, 2024-09-21T21:40:53.577 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0076042100000000001, 2024-09-21T21:40:53.577 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:53.577 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:53.577 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:40:53.577 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:53.577 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:53.577 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:53.577 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 122 2024-09-21T21:40:53.577 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.577 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:53.577 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.578 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:53.578 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.534714+0000", 2024-09-21T21:40:53.578 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:53.578 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.578 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.578 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:53.578 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.534714+0000", 2024-09-21T21:40:53.578 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:53.578 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.578 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.579 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:53.579 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.534716+0000", 2024-09-21T21:40:53.579 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.624e-06 2024-09-21T21:40:53.579 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.579 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.579 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:53.579 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.534721+0000", 2024-09-21T21:40:53.579 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9069999999999998e-06 2024-09-21T21:40:53.579 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.579 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.579 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:53.580 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.534733+0000", 2024-09-21T21:40:53.580 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.2089000000000001e-05 2024-09-21T21:40:53.580 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.580 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.580 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:53.580 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.534739+0000", 2024-09-21T21:40:53.580 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4879999999999998e-06 2024-09-21T21:40:53.580 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.580 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.580 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:53.580 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.534788+0000", 2024-09-21T21:40:53.580 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9629000000000002e-05 2024-09-21T21:40:53.580 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.580 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.581 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:53.581 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.534920+0000", 2024-09-21T21:40:53.581 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00013217599999999999 2024-09-21T21:40:53.581 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.581 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.581 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:53.581 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.542319+0000", 2024-09-21T21:40:53.581 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0073982969999999999 2024-09-21T21:40:53.581 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:53.581 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:53.581 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:53.581 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.581 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.581 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:154 3.8 3:15b4ee09:::rbd_data.11214be7c411.0000000000000028:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:53.582 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.538274+0000", 2024-09-21T21:40:53.582 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 10.035659258000001, 2024-09-21T21:40:53.582 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0067541629999999997, 2024-09-21T21:40:53.582 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:53.582 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:53.582 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:40:53.582 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:53.582 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:53.582 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:53.582 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 154 2024-09-21T21:40:53.582 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.582 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:53.582 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.582 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:53.582 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.538274+0000", 2024-09-21T21:40:53.583 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:53.583 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.583 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.583 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:53.583 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.538274+0000", 2024-09-21T21:40:53.583 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:53.583 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.583 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.583 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:53.583 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.538274+0000", 2024-09-21T21:40:53.583 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.6700000000000002e-07 2024-09-21T21:40:53.583 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.583 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.583 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:53.584 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.538278+0000", 2024-09-21T21:40:53.584 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8940000000000003e-06 2024-09-21T21:40:53.584 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.584 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.584 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:53.584 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.538288+0000", 2024-09-21T21:40:53.584 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6809999999999993e-06 2024-09-21T21:40:53.584 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.584 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.584 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:53.584 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.538293+0000", 2024-09-21T21:40:53.584 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1030000000000001e-06 2024-09-21T21:40:53.584 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.584 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.585 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:53.585 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.538331+0000", 2024-09-21T21:40:53.585 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7870999999999997e-05 2024-09-21T21:40:53.585 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.585 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.585 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:53.585 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.538408+0000", 2024-09-21T21:40:53.585 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7280000000000005e-05 2024-09-21T21:40:53.585 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.585 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.585 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:53.585 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.545028+0000", 2024-09-21T21:40:53.585 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0066194670000000004 2024-09-21T21:40:53.585 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:53.585 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:53.586 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:53.586 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.586 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.586 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:235 3.8 3:1566a55e:::rbd_data.11214be7c411.000000000000003a:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:53.586 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.556351+0000", 2024-09-21T21:40:53.586 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 10.017581711, 2024-09-21T21:40:53.586 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0066768460000000002, 2024-09-21T21:40:53.586 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:53.586 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:53.586 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:40:53.586 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:53.586 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:53.586 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:53.586 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 235 2024-09-21T21:40:53.587 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.587 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:53.587 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.587 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:53.587 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556351+0000", 2024-09-21T21:40:53.587 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:53.587 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.587 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.587 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:53.587 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556351+0000", 2024-09-21T21:40:53.587 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:53.587 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.587 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.587 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:53.587 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556352+0000", 2024-09-21T21:40:53.588 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.034e-06 2024-09-21T21:40:53.588 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.588 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.588 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:53.588 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556358+0000", 2024-09-21T21:40:53.588 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7200000000000003e-06 2024-09-21T21:40:53.588 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.588 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.588 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:53.588 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556373+0000", 2024-09-21T21:40:53.588 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.4602e-05 2024-09-21T21:40:53.588 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.588 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.588 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:53.588 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556391+0000", 2024-09-21T21:40:53.589 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.8726e-05 2024-09-21T21:40:53.589 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.589 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.589 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:53.589 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556445+0000", 2024-09-21T21:40:53.589 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3687000000000003e-05 2024-09-21T21:40:53.589 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.589 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.589 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:53.589 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556587+0000", 2024-09-21T21:40:53.589 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00014181299999999999 2024-09-21T21:40:53.589 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.589 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.589 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:53.590 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.563028+0000", 2024-09-21T21:40:53.590 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0064412640000000004 2024-09-21T21:40:53.590 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:53.590 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:53.590 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:53.590 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.590 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.590 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:244 3.1e 3:7844c79d:::rbd_data.11214be7c411.0000000000000037:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:53.590 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.556699+0000", 2024-09-21T21:40:53.590 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 10.017234304, 2024-09-21T21:40:53.590 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0063205409999999998, 2024-09-21T21:40:53.590 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:53.590 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:53.590 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:53.590 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:53.591 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:53.591 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:53.591 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 244 2024-09-21T21:40:53.591 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.591 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:53.591 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.591 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:53.591 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556699+0000", 2024-09-21T21:40:53.591 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:53.591 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.591 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.591 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:53.591 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556699+0000", 2024-09-21T21:40:53.591 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:53.591 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.592 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.592 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:53.592 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556699+0000", 2024-09-21T21:40:53.592 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.8899999999999998e-07 2024-09-21T21:40:53.592 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.592 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.592 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:53.592 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556704+0000", 2024-09-21T21:40:53.592 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8289999999999998e-06 2024-09-21T21:40:53.592 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.592 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.592 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:53.592 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556716+0000", 2024-09-21T21:40:53.593 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.2248e-05 2024-09-21T21:40:53.593 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.593 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.593 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:53.593 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556723+0000", 2024-09-21T21:40:53.593 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9460000000000001e-06 2024-09-21T21:40:53.593 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.593 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.593 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:53.593 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556776+0000", 2024-09-21T21:40:53.593 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2154000000000002e-05 2024-09-21T21:40:53.593 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.593 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.593 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:53.594 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556854+0000", 2024-09-21T21:40:53.594 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.8889999999999999e-05 2024-09-21T21:40:53.594 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.594 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.594 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 3", 2024-09-21T21:40:53.594 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.557063+0000", 2024-09-21T21:40:53.594 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000208314 2024-09-21T21:40:53.594 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.594 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.594 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:53.594 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.557979+0000", 2024-09-21T21:40:53.594 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00091603599999999996 2024-09-21T21:40:53.594 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.595 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.595 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:40:53.595 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.562892+0000", 2024-09-21T21:40:53.595 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0049135070000000001 2024-09-21T21:40:53.595 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.595 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.595 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:53.595 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.562963+0000", 2024-09-21T21:40:53.595 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0976999999999995e-05 2024-09-21T21:40:53.595 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.595 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.595 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:53.595 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.563019+0000", 2024-09-21T21:40:53.596 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5850999999999998e-05 2024-09-21T21:40:53.596 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:53.596 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:53.596 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:53.596 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.596 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.596 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:246 3.18 3:18a859ac:::rbd_data.11214be7c411.0000000000000036:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:53.596 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.556821+0000", 2024-09-21T21:40:53.596 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 10.017111675000001, 2024-09-21T21:40:53.596 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0064722360000000001, 2024-09-21T21:40:53.596 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:53.596 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:53.596 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:53.596 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:53.597 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:53.597 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:53.597 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 246 2024-09-21T21:40:53.597 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.597 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:53.597 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.597 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:53.597 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556821+0000", 2024-09-21T21:40:53.597 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:53.597 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.597 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.597 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:53.597 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556821+0000", 2024-09-21T21:40:53.598 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:53.598 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.598 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.598 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:53.598 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556822+0000", 2024-09-21T21:40:53.598 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.4499999999999996e-07 2024-09-21T21:40:53.598 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.598 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.598 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:53.598 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556827+0000", 2024-09-21T21:40:53.598 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6419999999999998e-06 2024-09-21T21:40:53.598 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.598 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.598 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:53.599 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556836+0000", 2024-09-21T21:40:53.599 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2380000000000003e-06 2024-09-21T21:40:53.599 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.599 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.599 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:53.599 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556843+0000", 2024-09-21T21:40:53.599 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6930000000000001e-06 2024-09-21T21:40:53.599 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.599 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.599 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:53.599 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556881+0000", 2024-09-21T21:40:53.599 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.807e-05 2024-09-21T21:40:53.599 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.599 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.600 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:53.600 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556955+0000", 2024-09-21T21:40:53.600 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.4059000000000004e-05 2024-09-21T21:40:53.600 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.600 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.600 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 0", 2024-09-21T21:40:53.600 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.557134+0000", 2024-09-21T21:40:53.600 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000179154 2024-09-21T21:40:53.600 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.600 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.600 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:53.600 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.563029+0000", 2024-09-21T21:40:53.600 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0058954419999999999 2024-09-21T21:40:53.600 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.601 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.601 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:40:53.601 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.563188+0000", 2024-09-21T21:40:53.601 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00015865200000000001 2024-09-21T21:40:53.601 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.601 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.601 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:53.601 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.563251+0000", 2024-09-21T21:40:53.601 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3120999999999994e-05 2024-09-21T21:40:53.601 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.601 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.601 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:53.601 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.563293+0000", 2024-09-21T21:40:53.601 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2320000000000001e-05 2024-09-21T21:40:53.602 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:53.602 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:53.602 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:53.602 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.602 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.602 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_repop(client.4417.0:440 3.5 e26/21 3:a6f99a77:::rbd_data.11214be7c411.0000000000000068:head v 26'10, mlcod=26'8)", 2024-09-21T21:40:53.602 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.590756+0000", 2024-09-21T21:40:53.602 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.9831767560000007, 2024-09-21T21:40:53.602 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0079145159999999999, 2024-09-21T21:40:53.602 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:53.602 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:53.602 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:53.603 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:53.603 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.603 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:53.603 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.590756+0000", 2024-09-21T21:40:53.603 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:53.603 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.603 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.603 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:53.603 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.590756+0000", 2024-09-21T21:40:53.603 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:53.603 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.603 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.603 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:53.603 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.590757+0000", 2024-09-21T21:40:53.603 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9500000000000002e-07 2024-09-21T21:40:53.604 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.604 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.604 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:53.604 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.590762+0000", 2024-09-21T21:40:53.604 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.045e-06 2024-09-21T21:40:53.604 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.604 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.604 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:53.604 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.590772+0000", 2024-09-21T21:40:53.604 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0711e-05 2024-09-21T21:40:53.604 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.604 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.604 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:53.604 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.590779+0000", 2024-09-21T21:40:53.605 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2210000000000002e-06 2024-09-21T21:40:53.605 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.605 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.605 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:53.605 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.590827+0000", 2024-09-21T21:40:53.605 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8127999999999997e-05 2024-09-21T21:40:53.605 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.605 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.605 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:53.605 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.590846+0000", 2024-09-21T21:40:53.605 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.965e-05 2024-09-21T21:40:53.605 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.605 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.605 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:53.605 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.598637+0000", 2024-09-21T21:40:53.605 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0077903240000000004 2024-09-21T21:40:53.606 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.606 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.606 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:53.606 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.598671+0000", 2024-09-21T21:40:53.606 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3841999999999999e-05 2024-09-21T21:40:53.606 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:53.606 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:53.606 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:53.606 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.606 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.606 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_repop(client.4417.0:459 3.4 e26/21 3:22094545:::rbd_data.11214be7c411.000000000000005c:head v 26'4, mlcod=26'2)", 2024-09-21T21:40:53.606 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.593580+0000", 2024-09-21T21:40:53.606 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.9803532740000005, 2024-09-21T21:40:53.606 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.01029448, 2024-09-21T21:40:53.607 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:53.607 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:53.607 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:53.607 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:53.607 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.607 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:53.607 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593580+0000", 2024-09-21T21:40:53.607 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:53.607 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.607 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.607 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:53.607 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593580+0000", 2024-09-21T21:40:53.607 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:53.607 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.607 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.608 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:53.608 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593580+0000", 2024-09-21T21:40:53.608 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0299999999999998e-07 2024-09-21T21:40:53.608 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.608 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.608 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:53.608 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593608+0000", 2024-09-21T21:40:53.608 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.7183999999999999e-05 2024-09-21T21:40:53.608 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.608 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.608 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:53.608 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593617+0000", 2024-09-21T21:40:53.608 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.1819999999999995e-06 2024-09-21T21:40:53.608 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.609 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.609 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:53.609 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593623+0000", 2024-09-21T21:40:53.609 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.443e-06 2024-09-21T21:40:53.609 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.609 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.609 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:53.609 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593671+0000", 2024-09-21T21:40:53.609 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8213000000000001e-05 2024-09-21T21:40:53.609 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.609 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.609 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:53.609 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.593691+0000", 2024-09-21T21:40:53.609 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9324999999999999e-05 2024-09-21T21:40:53.609 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.610 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.610 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:53.610 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.603832+0000", 2024-09-21T21:40:53.610 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.010141752 2024-09-21T21:40:53.610 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.610 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.610 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:53.610 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.603874+0000", 2024-09-21T21:40:53.610 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1678000000000003e-05 2024-09-21T21:40:53.610 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:53.610 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:53.610 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:53.610 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.610 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.610 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_repop(client.4417.0:460 3.4 e26/21 3:22094545:::rbd_data.11214be7c411.000000000000005c:head v 26'6, mlcod=26'2)", 2024-09-21T21:40:53.610 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.594203+0000", 2024-09-21T21:40:53.611 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.9797299580000001, 2024-09-21T21:40:53.611 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.010212592, 2024-09-21T21:40:53.611 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:53.611 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:53.611 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:53.611 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:53.611 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.611 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:53.611 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.594203+0000", 2024-09-21T21:40:53.611 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:53.611 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.611 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.611 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:53.611 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.594203+0000", 2024-09-21T21:40:53.612 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:53.612 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.612 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.612 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:53.612 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.594204+0000", 2024-09-21T21:40:53.612 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5199999999999996e-07 2024-09-21T21:40:53.612 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.612 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.612 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:53.612 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.594210+0000", 2024-09-21T21:40:53.612 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.798e-06 2024-09-21T21:40:53.612 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.612 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.612 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:53.613 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.594222+0000", 2024-09-21T21:40:53.613 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1391e-05 2024-09-21T21:40:53.613 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.613 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.613 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:53.613 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.594229+0000", 2024-09-21T21:40:53.613 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.7490000000000001e-06 2024-09-21T21:40:53.613 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.613 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.613 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:53.613 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.594285+0000", 2024-09-21T21:40:53.613 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6100999999999997e-05 2024-09-21T21:40:53.613 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.613 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.613 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:53.614 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.594303+0000", 2024-09-21T21:40:53.614 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.7938999999999999e-05 2024-09-21T21:40:53.614 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.614 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.614 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:53.614 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.604374+0000", 2024-09-21T21:40:53.614 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.010071762999999999 2024-09-21T21:40:53.614 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.614 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.614 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:53.614 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.604416+0000", 2024-09-21T21:40:53.614 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1099000000000001e-05 2024-09-21T21:40:53.614 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:53.614 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:53.615 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:53.615 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.615 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.615 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_repop(client.4417.0:496 3.d e26/21 3:b065aa9d:::rbd_data.11214be7c411.0000000000000074:head v 26'21, mlcod=26'19)", 2024-09-21T21:40:53.615 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.605521+0000", 2024-09-21T21:40:53.615 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.9684121650000002, 2024-09-21T21:40:53.615 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0071066059999999997, 2024-09-21T21:40:53.615 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:53.615 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:53.615 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:53.615 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:53.615 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.615 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:53.615 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605521+0000", 2024-09-21T21:40:53.616 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:53.616 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.616 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.616 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:53.616 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605521+0000", 2024-09-21T21:40:53.616 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:53.616 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.616 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.616 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:53.616 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605521+0000", 2024-09-21T21:40:53.616 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5799999999999999e-07 2024-09-21T21:40:53.616 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.616 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.616 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:53.617 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605526+0000", 2024-09-21T21:40:53.617 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.279e-06 2024-09-21T21:40:53.617 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.617 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.617 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:53.617 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605535+0000", 2024-09-21T21:40:53.617 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.594e-06 2024-09-21T21:40:53.617 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.617 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.617 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:53.617 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605540+0000", 2024-09-21T21:40:53.617 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4510000000000002e-06 2024-09-21T21:40:53.617 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.617 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.618 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:53.618 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605582+0000", 2024-09-21T21:40:53.618 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2778999999999998e-05 2024-09-21T21:40:53.618 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.618 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.618 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:53.618 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605600+0000", 2024-09-21T21:40:53.618 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.7795000000000002e-05 2024-09-21T21:40:53.618 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.618 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.618 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:53.618 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.612594+0000", 2024-09-21T21:40:53.618 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0069935509999999998 2024-09-21T21:40:53.618 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.619 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.619 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:53.619 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.612627+0000", 2024-09-21T21:40:53.619 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3599000000000001e-05 2024-09-21T21:40:53.619 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:53.619 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:53.619 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:53.619 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.619 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.619 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:503 3.8 3:10f57124:::rbd_object_map.11214be7c411:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:53.619 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.605576+0000", 2024-09-21T21:40:53.619 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.9683567570000005, 2024-09-21T21:40:53.619 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0071414349999999998, 2024-09-21T21:40:53.620 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:53.620 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:53.620 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:53.620 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:53.620 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:53.620 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:53.620 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 503 2024-09-21T21:40:53.620 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.620 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:53.620 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.620 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:53.620 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605576+0000", 2024-09-21T21:40:53.620 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:53.620 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.621 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.621 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:53.621 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605576+0000", 2024-09-21T21:40:53.621 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:53.621 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.621 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.621 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:53.621 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605577+0000", 2024-09-21T21:40:53.621 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1099999999999995e-07 2024-09-21T21:40:53.621 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.621 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.621 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:53.621 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605581+0000", 2024-09-21T21:40:53.621 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1489999999999996e-06 2024-09-21T21:40:53.621 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.622 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.622 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:53.622 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605587+0000", 2024-09-21T21:40:53.622 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9719999999999997e-06 2024-09-21T21:40:53.622 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.622 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.622 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:53.622 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605591+0000", 2024-09-21T21:40:53.622 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.9759999999999997e-06 2024-09-21T21:40:53.622 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.622 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.622 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:53.622 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605647+0000", 2024-09-21T21:40:53.622 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6162999999999997e-05 2024-09-21T21:40:53.623 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.623 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.623 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:53.623 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605744+0000", 2024-09-21T21:40:53.623 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6916000000000005e-05 2024-09-21T21:40:53.623 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.623 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.623 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 1", 2024-09-21T21:40:53.623 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.606412+0000", 2024-09-21T21:40:53.623 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00066832800000000004 2024-09-21T21:40:53.623 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.623 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.623 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:53.623 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.608775+0000", 2024-09-21T21:40:53.624 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0023627790000000002 2024-09-21T21:40:53.624 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.624 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.624 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:40:53.624 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.612630+0000", 2024-09-21T21:40:53.624 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0038553110000000002 2024-09-21T21:40:53.624 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.624 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.624 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:53.624 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.612671+0000", 2024-09-21T21:40:53.624 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.0451999999999998e-05 2024-09-21T21:40:53.624 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.624 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.624 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:53.624 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.612718+0000", 2024-09-21T21:40:53.625 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6777999999999998e-05 2024-09-21T21:40:53.625 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:53.625 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:53.625 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:53.625 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.625 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.625 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:504 3.8 3:10f57124:::rbd_object_map.11214be7c411:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:53.625 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.605604+0000", 2024-09-21T21:40:53.625 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.9683286179999993, 2024-09-21T21:40:53.625 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.010335359000000001, 2024-09-21T21:40:53.625 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:53.625 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:53.625 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:53.625 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:53.625 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:53.626 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:53.626 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 504 2024-09-21T21:40:53.626 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.626 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:53.626 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.626 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:53.626 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605604+0000", 2024-09-21T21:40:53.626 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:53.626 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.626 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.626 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:53.626 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605604+0000", 2024-09-21T21:40:53.626 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:53.626 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.627 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.627 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:53.627 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605605+0000", 2024-09-21T21:40:53.627 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.0200000000000003e-07 2024-09-21T21:40:53.627 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.627 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.627 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:53.627 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605607+0000", 2024-09-21T21:40:53.627 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.6029999999999999e-06 2024-09-21T21:40:53.627 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.627 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.627 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:53.627 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605613+0000", 2024-09-21T21:40:53.627 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4870000000000002e-06 2024-09-21T21:40:53.627 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.628 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.628 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:53.628 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605616+0000", 2024-09-21T21:40:53.628 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4319999999999999e-06 2024-09-21T21:40:53.628 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.628 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.628 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:53.628 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.606749+0000", 2024-09-21T21:40:53.628 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.001132587 2024-09-21T21:40:53.628 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.628 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.628 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:53.628 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.606807+0000", 2024-09-21T21:40:53.628 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8356999999999997e-05 2024-09-21T21:40:53.629 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.629 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.629 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 1", 2024-09-21T21:40:53.629 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607016+0000", 2024-09-21T21:40:53.629 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00020888600000000001 2024-09-21T21:40:53.629 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.629 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.629 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:40:53.629 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.615545+0000", 2024-09-21T21:40:53.629 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0085290829999999998 2024-09-21T21:40:53.629 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.629 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.629 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:53.629 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.615842+0000", 2024-09-21T21:40:53.629 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000296951 2024-09-21T21:40:53.630 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.630 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.630 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:53.630 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.615893+0000", 2024-09-21T21:40:53.630 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0519000000000002e-05 2024-09-21T21:40:53.630 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.630 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.630 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:53.630 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.615940+0000", 2024-09-21T21:40:53.630 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7052000000000003e-05 2024-09-21T21:40:53.630 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:53.630 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:53.630 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:53.630 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.631 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.631 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:507 3.8 3:10f57124:::rbd_object_map.11214be7c411:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:53.631 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.605687+0000", 2024-09-21T21:40:53.631 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.9682456760000004, 2024-09-21T21:40:53.631 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.010435425, 2024-09-21T21:40:53.631 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:53.631 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:53.631 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:53.631 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:53.631 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:53.631 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:53.631 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 507 2024-09-21T21:40:53.631 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.631 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:53.631 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.632 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:53.632 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605687+0000", 2024-09-21T21:40:53.632 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:53.632 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.632 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.632 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:53.632 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605687+0000", 2024-09-21T21:40:53.632 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:53.632 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.632 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.632 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:53.632 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605688+0000", 2024-09-21T21:40:53.632 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.006e-06 2024-09-21T21:40:53.632 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.632 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.632 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:53.632 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605694+0000", 2024-09-21T21:40:53.633 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6099999999999997e-06 2024-09-21T21:40:53.633 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.633 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.633 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:53.633 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605705+0000", 2024-09-21T21:40:53.633 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.164e-05 2024-09-21T21:40:53.633 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.633 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.633 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:53.633 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605714+0000", 2024-09-21T21:40:53.633 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.4610000000000008e-06 2024-09-21T21:40:53.633 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.633 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.633 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:53.634 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607537+0000", 2024-09-21T21:40:53.634 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0018232490000000001 2024-09-21T21:40:53.634 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.634 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.634 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:53.634 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607585+0000", 2024-09-21T21:40:53.634 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7491999999999999e-05 2024-09-21T21:40:53.634 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.634 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.634 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 1", 2024-09-21T21:40:53.634 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607814+0000", 2024-09-21T21:40:53.634 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000229223 2024-09-21T21:40:53.634 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.634 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.634 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:40:53.635 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.615553+0000", 2024-09-21T21:40:53.635 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0077389700000000004 2024-09-21T21:40:53.635 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.635 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.635 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:53.635 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.615999+0000", 2024-09-21T21:40:53.635 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000446107 2024-09-21T21:40:53.635 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.635 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.635 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:53.635 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.616081+0000", 2024-09-21T21:40:53.635 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.1816000000000004e-05 2024-09-21T21:40:53.635 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.636 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.636 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:53.636 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.616123+0000", 2024-09-21T21:40:53.636 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1851000000000003e-05 2024-09-21T21:40:53.636 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:53.636 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:53.636 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:53.636 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.636 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.636 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:508 3.8 3:10f57124:::rbd_object_map.11214be7c411:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:53.636 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.605730+0000", 2024-09-21T21:40:53.636 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.9682031179999999, 2024-09-21T21:40:53.636 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.010553755, 2024-09-21T21:40:53.636 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:53.636 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:53.637 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:53.637 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:53.637 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:53.637 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:53.637 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 508 2024-09-21T21:40:53.637 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.637 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:53.637 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.637 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:53.637 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605730+0000", 2024-09-21T21:40:53.637 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:53.637 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.637 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.637 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:53.637 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605730+0000", 2024-09-21T21:40:53.637 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:53.637 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.638 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.638 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:53.638 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605730+0000", 2024-09-21T21:40:53.638 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5300000000000004e-07 2024-09-21T21:40:53.638 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.638 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.638 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:53.638 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605734+0000", 2024-09-21T21:40:53.638 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.2289999999999999e-06 2024-09-21T21:40:53.638 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.638 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.638 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:53.638 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605741+0000", 2024-09-21T21:40:53.638 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.4660000000000002e-06 2024-09-21T21:40:53.638 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.639 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.639 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:53.639 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605757+0000", 2024-09-21T21:40:53.639 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.5709000000000002e-05 2024-09-21T21:40:53.639 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.639 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.639 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:53.639 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.608168+0000", 2024-09-21T21:40:53.639 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.002411683 2024-09-21T21:40:53.639 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.639 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.639 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:53.639 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.608249+0000", 2024-09-21T21:40:53.639 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0903999999999997e-05 2024-09-21T21:40:53.640 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.640 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.640 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 1", 2024-09-21T21:40:53.640 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.608481+0000", 2024-09-21T21:40:53.640 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00023124800000000001 2024-09-21T21:40:53.640 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.640 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.640 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:40:53.640 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.615559+0000", 2024-09-21T21:40:53.640 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0070787009999999997 2024-09-21T21:40:53.640 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.640 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.640 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:53.640 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.616184+0000", 2024-09-21T21:40:53.640 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00062485199999999996 2024-09-21T21:40:53.640 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.641 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.641 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:53.641 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.616241+0000", 2024-09-21T21:40:53.641 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6971000000000001e-05 2024-09-21T21:40:53.641 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.641 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.641 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:53.641 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.616284+0000", 2024-09-21T21:40:53.641 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2438999999999998e-05 2024-09-21T21:40:53.641 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:53.641 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:53.641 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:53.641 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.641 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.641 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:509 3.8 3:10f57124:::rbd_object_map.11214be7c411:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:53.641 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.605831+0000", 2024-09-21T21:40:53.641 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.9681018009999995, 2024-09-21T21:40:53.642 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.010535957, 2024-09-21T21:40:53.642 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:53.642 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:53.642 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:53.642 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:53.642 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:53.642 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:53.642 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 509 2024-09-21T21:40:53.642 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.642 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:53.642 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.642 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:53.642 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605831+0000", 2024-09-21T21:40:53.642 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:53.642 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.642 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.643 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:53.643 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605831+0000", 2024-09-21T21:40:53.643 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:53.643 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.643 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.643 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:53.643 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605832+0000", 2024-09-21T21:40:53.643 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0999999999999997e-07 2024-09-21T21:40:53.643 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.643 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.643 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:53.643 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605837+0000", 2024-09-21T21:40:53.643 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1680000000000003e-06 2024-09-21T21:40:53.644 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.644 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.644 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:53.644 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605848+0000", 2024-09-21T21:40:53.644 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0715e-05 2024-09-21T21:40:53.644 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.644 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.644 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:53.644 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.605855+0000", 2024-09-21T21:40:53.644 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.6359999999999994e-06 2024-09-21T21:40:53.644 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.644 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.644 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:53.644 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.608805+0000", 2024-09-21T21:40:53.644 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0029495430000000002 2024-09-21T21:40:53.644 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.644 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.645 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:53.645 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.608852+0000", 2024-09-21T21:40:53.645 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7258000000000001e-05 2024-09-21T21:40:53.645 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.645 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.645 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 1", 2024-09-21T21:40:53.645 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.609051+0000", 2024-09-21T21:40:53.645 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00019842300000000001 2024-09-21T21:40:53.645 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.645 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.645 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:40:53.645 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.615588+0000", 2024-09-21T21:40:53.645 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0065374049999999996 2024-09-21T21:40:53.645 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.645 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.646 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:53.646 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.616317+0000", 2024-09-21T21:40:53.646 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000729341 2024-09-21T21:40:53.646 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.646 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.646 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:53.646 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.616345+0000", 2024-09-21T21:40:53.646 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.7932999999999998e-05 2024-09-21T21:40:53.646 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.646 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.646 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:53.646 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.616367+0000", 2024-09-21T21:40:53.646 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1725e-05 2024-09-21T21:40:53.646 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:53.647 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:53.647 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:53.647 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.647 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.647 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:510 3.8 3:10f57124:::rbd_object_map.11214be7c411:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:53.647 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.606059+0000", 2024-09-21T21:40:53.647 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.9678737309999992, 2024-09-21T21:40:53.647 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.010396717, 2024-09-21T21:40:53.647 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:53.647 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:53.647 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:53.647 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:53.647 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:53.647 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:53.647 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 510 2024-09-21T21:40:53.648 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.648 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:53.648 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.648 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:53.648 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.606059+0000", 2024-09-21T21:40:53.648 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:53.648 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.648 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.648 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:53.648 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.606059+0000", 2024-09-21T21:40:53.648 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:53.648 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.648 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.648 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:53.648 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.606060+0000", 2024-09-21T21:40:53.648 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.16e-07 2024-09-21T21:40:53.648 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.649 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.649 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:53.649 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.606065+0000", 2024-09-21T21:40:53.649 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2290000000000002e-06 2024-09-21T21:40:53.649 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.649 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.649 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:53.649 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.606076+0000", 2024-09-21T21:40:53.649 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0584999999999999e-05 2024-09-21T21:40:53.649 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.649 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.649 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:53.649 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.606094+0000", 2024-09-21T21:40:53.649 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.7920000000000001e-05 2024-09-21T21:40:53.649 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.650 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.650 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:53.650 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.609308+0000", 2024-09-21T21:40:53.650 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0032141980000000001 2024-09-21T21:40:53.650 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.650 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.650 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:53.650 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.609392+0000", 2024-09-21T21:40:53.650 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.4188000000000001e-05 2024-09-21T21:40:53.650 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.650 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.650 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 1", 2024-09-21T21:40:53.650 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.609627+0000", 2024-09-21T21:40:53.651 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00023437100000000001 2024-09-21T21:40:53.651 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.651 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.651 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:40:53.651 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.615594+0000", 2024-09-21T21:40:53.651 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0059679160000000002 2024-09-21T21:40:53.651 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.651 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.651 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:53.651 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.616394+0000", 2024-09-21T21:40:53.651 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00079993499999999999 2024-09-21T21:40:53.651 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.651 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.651 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:53.651 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.616419+0000", 2024-09-21T21:40:53.651 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.4834999999999999e-05 2024-09-21T21:40:53.651 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.652 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.652 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:53.652 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.616456+0000", 2024-09-21T21:40:53.652 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6624e-05 2024-09-21T21:40:53.652 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:53.652 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:53.652 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:53.652 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.652 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.652 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_repop(client.4417.0:497 3.d e26/21 3:b7efb195:::rbd_data.11214be7c411.0000000000000076:head v 26'22, mlcod=26'19)", 2024-09-21T21:40:53.652 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.606123+0000", 2024-09-21T21:40:53.652 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.9678098459999998, 2024-09-21T21:40:53.652 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0065331749999999996, 2024-09-21T21:40:53.653 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:53.653 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:53.653 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:53.653 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:53.653 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.653 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:53.653 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.606123+0000", 2024-09-21T21:40:53.653 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:53.653 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.653 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.653 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:53.653 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.606123+0000", 2024-09-21T21:40:53.653 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:53.653 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.653 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.653 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:53.653 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.606123+0000", 2024-09-21T21:40:53.654 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.0999999999999999e-07 2024-09-21T21:40:53.654 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.654 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.654 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:53.654 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.606128+0000", 2024-09-21T21:40:53.654 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5510000000000001e-06 2024-09-21T21:40:53.654 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.654 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.654 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:53.654 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.606133+0000", 2024-09-21T21:40:53.654 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3619999999999997e-06 2024-09-21T21:40:53.654 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.654 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.654 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:53.654 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.606138+0000", 2024-09-21T21:40:53.655 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3749999999999996e-06 2024-09-21T21:40:53.655 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.655 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.655 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:53.655 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.606195+0000", 2024-09-21T21:40:53.655 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7244999999999998e-05 2024-09-21T21:40:53.655 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.655 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.655 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:53.655 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.606213+0000", 2024-09-21T21:40:53.655 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.8116000000000001e-05 2024-09-21T21:40:53.655 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.655 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.655 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:53.655 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.612630+0000", 2024-09-21T21:40:53.655 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0064170670000000003 2024-09-21T21:40:53.656 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.656 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.656 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:53.656 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.612656+0000", 2024-09-21T21:40:53.656 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.6049000000000001e-05 2024-09-21T21:40:53.656 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:53.656 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:53.656 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:53.656 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.656 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.656 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:511 3.8 3:10f57124:::rbd_object_map.11214be7c411:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:53.656 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.606610+0000", 2024-09-21T21:40:53.656 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.967323296, 2024-09-21T21:40:53.656 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0099483509999999994, 2024-09-21T21:40:53.656 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:53.656 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:53.657 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:53.657 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:53.657 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:53.657 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:53.657 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 511 2024-09-21T21:40:53.657 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.657 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:53.657 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.657 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:53.657 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.606610+0000", 2024-09-21T21:40:53.657 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:53.657 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.657 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.657 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:53.657 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.606610+0000", 2024-09-21T21:40:53.658 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:53.658 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.658 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.658 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:53.658 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.606610+0000", 2024-09-21T21:40:53.658 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.6000000000000002e-07 2024-09-21T21:40:53.658 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.658 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.658 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:53.658 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.606616+0000", 2024-09-21T21:40:53.658 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8429999999999998e-06 2024-09-21T21:40:53.658 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.658 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.658 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:53.658 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.606630+0000", 2024-09-21T21:40:53.659 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.3298999999999999e-05 2024-09-21T21:40:53.659 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.659 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.659 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:53.659 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.606637+0000", 2024-09-21T21:40:53.659 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5610000000000001e-06 2024-09-21T21:40:53.659 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.659 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.659 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:53.659 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.609878+0000", 2024-09-21T21:40:53.659 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0032406190000000001 2024-09-21T21:40:53.659 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.659 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.659 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:53.660 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.609922+0000", 2024-09-21T21:40:53.660 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4604000000000002e-05 2024-09-21T21:40:53.660 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.660 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.660 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 1", 2024-09-21T21:40:53.660 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.610113+0000", 2024-09-21T21:40:53.660 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00019091099999999999 2024-09-21T21:40:53.660 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.660 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.660 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:40:53.660 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.615599+0000", 2024-09-21T21:40:53.660 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0054859339999999996 2024-09-21T21:40:53.660 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.660 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.660 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:53.661 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.616497+0000", 2024-09-21T21:40:53.661 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000897908 2024-09-21T21:40:53.661 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.661 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.661 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:53.661 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.616534+0000", 2024-09-21T21:40:53.661 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6538000000000002e-05 2024-09-21T21:40:53.661 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.661 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.661 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:53.661 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.616558+0000", 2024-09-21T21:40:53.661 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.4273999999999998e-05 2024-09-21T21:40:53.661 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:53.661 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:53.661 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:53.662 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.662 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.662 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:512 3.8 3:10f57124:::rbd_object_map.11214be7c411:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:53.662 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.606653+0000", 2024-09-21T21:40:53.662 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.9672794679999992, 2024-09-21T21:40:53.662 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.010056294, 2024-09-21T21:40:53.662 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:53.662 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:53.662 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:53.662 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:53.662 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:53.662 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:53.662 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 512 2024-09-21T21:40:53.662 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.662 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:53.663 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.663 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:53.663 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.606653+0000", 2024-09-21T21:40:53.663 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:53.663 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.663 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.663 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:53.663 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.606653+0000", 2024-09-21T21:40:53.663 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:53.663 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.663 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.663 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:53.663 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.606654+0000", 2024-09-21T21:40:53.663 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4600000000000005e-07 2024-09-21T21:40:53.664 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.664 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.664 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:53.664 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.606670+0000", 2024-09-21T21:40:53.664 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.6336000000000001e-05 2024-09-21T21:40:53.664 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.664 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.664 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:53.664 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.606680+0000", 2024-09-21T21:40:53.664 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.3400000000000004e-06 2024-09-21T21:40:53.664 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.664 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.664 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:53.664 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.606686+0000", 2024-09-21T21:40:53.665 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5610000000000004e-06 2024-09-21T21:40:53.665 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.665 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.665 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:53.665 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.610374+0000", 2024-09-21T21:40:53.665 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0036873330000000001 2024-09-21T21:40:53.665 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.665 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.665 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:53.665 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.610417+0000", 2024-09-21T21:40:53.665 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3872000000000003e-05 2024-09-21T21:40:53.665 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.665 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.665 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 1", 2024-09-21T21:40:53.665 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.610598+0000", 2024-09-21T21:40:53.666 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000180917 2024-09-21T21:40:53.666 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.666 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.666 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:40:53.666 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.615605+0000", 2024-09-21T21:40:53.666 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0050068179999999997 2024-09-21T21:40:53.666 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.666 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.666 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:53.666 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.616612+0000", 2024-09-21T21:40:53.666 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0010071660000000001 2024-09-21T21:40:53.666 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.666 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.666 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:53.667 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.616670+0000", 2024-09-21T21:40:53.667 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7321999999999998e-05 2024-09-21T21:40:53.667 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.667 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.667 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:53.667 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.616710+0000", 2024-09-21T21:40:53.667 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.0083000000000002e-05 2024-09-21T21:40:53.667 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:53.667 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:53.667 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:53.667 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.667 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.667 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:514 3.8 3:10f57124:::rbd_object_map.11214be7c411:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:53.667 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.607169+0000", 2024-09-21T21:40:53.668 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.9667636040000005, 2024-09-21T21:40:53.668 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0096953709999999995, 2024-09-21T21:40:53.668 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:53.668 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:53.668 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:53.668 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:53.668 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:53.668 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:53.668 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 514 2024-09-21T21:40:53.668 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.668 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:53.668 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.668 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:53.668 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607169+0000", 2024-09-21T21:40:53.669 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:53.669 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.669 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.669 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:53.669 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607169+0000", 2024-09-21T21:40:53.669 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:53.669 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.669 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.669 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:53.669 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607170+0000", 2024-09-21T21:40:53.669 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.9700000000000005e-07 2024-09-21T21:40:53.669 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.669 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.669 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:53.669 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607176+0000", 2024-09-21T21:40:53.670 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7529999999999998e-06 2024-09-21T21:40:53.670 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.670 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.670 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:53.670 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607188+0000", 2024-09-21T21:40:53.670 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1739999999999999e-05 2024-09-21T21:40:53.670 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.670 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.670 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:53.670 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607196+0000", 2024-09-21T21:40:53.670 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0739999999999993e-06 2024-09-21T21:40:53.670 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.670 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.671 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:53.671 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.610842+0000", 2024-09-21T21:40:53.671 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0036461750000000002 2024-09-21T21:40:53.671 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.671 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.671 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:53.671 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.610887+0000", 2024-09-21T21:40:53.671 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4953000000000001e-05 2024-09-21T21:40:53.671 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.671 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.671 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 1", 2024-09-21T21:40:53.671 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.611090+0000", 2024-09-21T21:40:53.671 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00020270100000000001 2024-09-21T21:40:53.671 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.671 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.672 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:40:53.672 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.615618+0000", 2024-09-21T21:40:53.672 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.004528486 2024-09-21T21:40:53.672 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.672 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.672 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:53.672 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.616769+0000", 2024-09-21T21:40:53.672 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.001150852 2024-09-21T21:40:53.672 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.672 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.672 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:53.672 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.616825+0000", 2024-09-21T21:40:53.672 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5686999999999997e-05 2024-09-21T21:40:53.672 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.672 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.672 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:53.673 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.616865+0000", 2024-09-21T21:40:53.673 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.0052999999999998e-05 2024-09-21T21:40:53.673 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:53.673 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:53.673 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:53.673 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.673 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.673 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:515 3.8 3:10f57124:::rbd_object_map.11214be7c411:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:53.673 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.607244+0000", 2024-09-21T21:40:53.673 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.9666887670000008, 2024-09-21T21:40:53.673 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0098905950000000003, 2024-09-21T21:40:53.673 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:53.673 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:53.673 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:53.673 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:53.674 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:53.674 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:53.674 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 515 2024-09-21T21:40:53.674 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.674 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:53.674 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.674 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:53.674 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607244+0000", 2024-09-21T21:40:53.674 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:53.674 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.674 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.674 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:53.674 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607244+0000", 2024-09-21T21:40:53.674 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:53.675 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.675 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.675 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:53.675 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607245+0000", 2024-09-21T21:40:53.675 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5199999999999996e-07 2024-09-21T21:40:53.675 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.675 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.675 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:53.675 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607249+0000", 2024-09-21T21:40:53.675 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3490000000000002e-06 2024-09-21T21:40:53.675 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.675 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.675 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:53.675 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607262+0000", 2024-09-21T21:40:53.675 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.3118999999999999e-05 2024-09-21T21:40:53.676 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.676 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.676 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:53.676 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.607271+0000", 2024-09-21T21:40:53.676 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.8710000000000003e-06 2024-09-21T21:40:53.676 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.676 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.676 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:53.676 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.611347+0000", 2024-09-21T21:40:53.676 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0040757570000000002 2024-09-21T21:40:53.676 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.676 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.676 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:53.676 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.611432+0000", 2024-09-21T21:40:53.676 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.4670000000000001e-05 2024-09-21T21:40:53.677 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.677 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.677 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 1", 2024-09-21T21:40:53.677 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.611638+0000", 2024-09-21T21:40:53.677 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000206028 2024-09-21T21:40:53.677 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.677 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.677 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:40:53.677 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.615623+0000", 2024-09-21T21:40:53.677 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0039852960000000002 2024-09-21T21:40:53.677 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.677 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.677 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:53.677 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.617013+0000", 2024-09-21T21:40:53.678 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0013903170000000001 2024-09-21T21:40:53.678 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.678 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.678 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:53.678 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.617096+0000", 2024-09-21T21:40:53.678 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.3069e-05 2024-09-21T21:40:53.678 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:53.678 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:53.678 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:53.678 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.617135+0000", 2024-09-21T21:40:53.678 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8367e-05 2024-09-21T21:40:53.678 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:53.678 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:53.678 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:53.678 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:53.679 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:53.679 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:40:53.682 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T21:40:53.682 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:40:53.678+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:40:53.683 INFO:teuthology.orchestra.run.smithi164.stdout:696 write 0x5d3e200 thru 0x5d42bdb (0x49dc bytes) 2024-09-21T21:40:53.684 INFO:teuthology.orchestra.run.smithi164.stdout:698 punch from 0x33f400 to 0x347600, (0x8200 bytes) 2024-09-21T21:40:53.684 INFO:teuthology.orchestra.run.smithi164.stdout:699 read 0x3777600 thru 0x377f513 (0x7f14 bytes) 2024-09-21T21:40:53.684 INFO:teuthology.orchestra.run.smithi164.stdout:703 read 0x17d4400 thru 0x17da7fb (0x63fc bytes) 2024-09-21T21:40:53.684 INFO:teuthology.orchestra.run.smithi164.stdout:705 write 0xc930c00 thru 0xc937725 (0x6b26 bytes) 2024-09-21T21:40:53.684 INFO:teuthology.orchestra.run.smithi164.stdout:709 write 0x21c3a00 thru 0x21cbc90 (0x8291 bytes) 2024-09-21T21:40:53.684 INFO:teuthology.orchestra.run.smithi164.stdout:711 write 0x2a67c00 thru 0x2a6ed16 (0x7117 bytes) 2024-09-21T21:40:53.684 INFO:teuthology.orchestra.run.smithi164.stdout:712 read 0x3696400 thru 0x369defc (0x7afd bytes) 2024-09-21T21:40:53.684 INFO:teuthology.orchestra.run.smithi164.stdout:714 read 0xb06f200 thru 0xb076845 (0x7646 bytes) 2024-09-21T21:40:53.684 INFO:teuthology.orchestra.run.smithi164.stdout:716 write 0x2fe4e00 thru 0x2ff30fb (0xe2fc bytes) 2024-09-21T21:40:53.684 INFO:teuthology.orchestra.run.smithi164.stdout:717 trunc from 0xc937800 to 0xd945c00 2024-09-21T21:40:53.684 INFO:teuthology.orchestra.run.smithi164.stdout:720 read 0x6a1c000 thru 0x6a1d276 (0x1277 bytes) 2024-09-21T21:40:53.684 INFO:teuthology.orchestra.run.smithi164.stdout:722 read 0x7807a00 thru 0x780d164 (0x5765 bytes) 2024-09-21T21:40:53.684 INFO:teuthology.orchestra.run.smithi164.stdout:725 trunc from 0xd945c00 to 0x63b5400 2024-09-21T21:40:53.684 INFO:teuthology.orchestra.run.smithi164.stdout:726 read 0x136b200 thru 0x1377fd8 (0xcdd9 bytes) 2024-09-21T21:40:53.684 INFO:teuthology.orchestra.run.smithi164.stdout:727 write 0x60e6200 thru 0x60ebf1a (0x5d1b bytes) 2024-09-21T21:40:53.686 INFO:teuthology.orchestra.run.smithi164.stdout:728 write 0xd6a4600 thru 0xd6a7f4a (0x394b bytes) 2024-09-21T21:40:53.687 INFO:teuthology.orchestra.run.smithi164.stdout:729 punch from 0x2f93e00 to 0x2f95400, (0x1600 bytes) 2024-09-21T21:40:53.691 INFO:teuthology.orchestra.run.smithi164.stdout:730 read 0xc095600 thru 0xc0a3519 (0xdf1a bytes) 2024-09-21T21:40:53.691 INFO:teuthology.orchestra.run.smithi164.stdout:732 read 0xe3f200 thru 0xe4c139 (0xcf3a bytes) 2024-09-21T21:40:53.699 INFO:teuthology.orchestra.run.smithi164.stdout:736 trunc from 0xd6a8000 to 0x7d25800 2024-09-21T21:40:53.715 INFO:teuthology.orchestra.run.smithi164.stdout:737 write 0x1cca000 thru 0x1cd50a1 (0xb0a2 bytes) 2024-09-21T21:40:53.726 INFO:teuthology.orchestra.run.smithi164.stdout:738 write 0x7edf400 thru 0x7ee6447 (0x7048 bytes) 2024-09-21T21:40:53.733 INFO:teuthology.orchestra.run.smithi164.stdout:740 punch from 0x14f2200 to 0x14f7400, (0x5200 bytes) 2024-09-21T21:40:53.733 INFO:teuthology.orchestra.run.smithi164.stdout:742 punch from 0x4590800 to 0x4591200, (0xa00 bytes) 2024-09-21T21:40:53.739 INFO:teuthology.orchestra.run.smithi164.stdout:744 trunc from 0x7ee6600 to 0xcec4800 2024-09-21T21:40:53.754 INFO:teuthology.orchestra.run.smithi164.stdout:745 read 0x2384c00 thru 0x238bafd (0x6efe bytes) 2024-09-21T21:40:53.758 INFO:teuthology.orchestra.run.smithi164.stdout:746 trunc from 0xcec4800 to 0xa291400 2024-09-21T21:40:53.766 INFO:teuthology.orchestra.run.smithi164.stdout:747 read 0x9f69800 thru 0x9f6b030 (0x1831 bytes) 2024-09-21T21:40:53.768 INFO:teuthology.orchestra.run.smithi164.stdout:749 read 0x6a10e00 thru 0x6a1b305 (0xa506 bytes) 2024-09-21T21:40:53.772 INFO:teuthology.orchestra.run.smithi164.stdout:751 read 0x3733000 thru 0x3740b9f (0xdba0 bytes) 2024-09-21T21:40:53.777 INFO:teuthology.orchestra.run.smithi164.stdout:753 trunc from 0xa291400 to 0x5edc600 2024-09-21T21:40:53.781 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:40:53.779+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:40:53.793 INFO:teuthology.orchestra.run.smithi164.stdout:754 punch from 0x12c0800 to 0x12c9400, (0x8c00 bytes) 2024-09-21T21:40:53.795 INFO:teuthology.orchestra.run.smithi164.stdout:757 read 0x3c9a400 thru 0x3c9c7c0 (0x23c1 bytes) 2024-09-21T21:40:53.801 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:40:53.801 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:40:53.802 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-21T21:40:53.802 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:40:53.810 INFO:teuthology.orchestra.run.smithi164.stdout:758 write 0xa1b9800 thru 0xa1c7d5b (0xe55c bytes) 2024-09-21T21:40:53.812 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T21:40:53.821 INFO:teuthology.orchestra.run.smithi164.stdout:760 trunc from 0xa1c7e00 to 0x1836a00 2024-09-21T21:40:53.843 INFO:teuthology.orchestra.run.smithi164.stdout:762 read 0x838000 thru 0x83ad96 (0x2d97 bytes) 2024-09-21T21:40:53.844 INFO:teuthology.orchestra.run.smithi164.stdout:764 read 0xbc8000 thru 0xbd43d3 (0xc3d4 bytes) 2024-09-21T21:40:53.848 INFO:teuthology.orchestra.run.smithi164.stdout:765 trunc from 0x1836a00 to 0x4191200 2024-09-21T21:40:53.861 INFO:teuthology.orchestra.run.smithi164.stdout:769 trunc from 0x4191200 to 0x8889e00 2024-09-21T21:40:53.882 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:40:53.880+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:40:53.892 INFO:teuthology.orchestra.run.smithi164.stdout:770 write 0xae7fc00 thru 0xae8b0cb (0xb4cc bytes) 2024-09-21T21:40:53.892 INFO:teuthology.orchestra.run.smithi164.stdout:771 write 0x3294600 thru 0x3296f21 (0x2922 bytes) 2024-09-21T21:40:53.903 INFO:teuthology.orchestra.run.smithi164.stdout:773 trunc from 0xae8b200 to 0x1a03000 2024-09-21T21:40:53.928 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:40:53.928 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:40:53.928 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:40:53.929 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:40:53.929 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:40:53.937 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T21:40:53.957 INFO:teuthology.orchestra.run.smithi164.stdout:775 write 0xe2be800 thru 0xe2ca480 (0xbc81 bytes) 2024-09-21T21:40:53.957 INFO:teuthology.orchestra.run.smithi164.stdout:776 read 0x693d800 thru 0x693fd65 (0x2566 bytes) 2024-09-21T21:40:53.965 INFO:teuthology.orchestra.run.smithi164.stdout:777 trunc from 0xe2ca600 to 0x6da5000 2024-09-21T21:40:53.982 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:40:53.980+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:40:53.990 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T21:40:53.994 INFO:teuthology.orchestra.run.smithi164.stdout:779 punch from 0x663fc00 to 0x664a800, (0xac00 bytes) 2024-09-21T21:40:53.995 INFO:teuthology.orchestra.run.smithi164.stdout:783 write 0x2433e00 thru 0x2442b85 (0xed86 bytes) 2024-09-21T21:40:53.996 INFO:teuthology.orchestra.run.smithi164.stdout:785 read 0x2734600 thru 0x273bcf1 (0x76f2 bytes) 2024-09-21T21:40:53.996 INFO:teuthology.orchestra.run.smithi164.stdout:790 punch from 0x1542400 to 0x154b600, (0x9200 bytes) 2024-09-21T21:40:53.997 INFO:teuthology.orchestra.run.smithi164.stdout:791 read 0x5be0800 thru 0x5bed001 (0xc802 bytes) 2024-09-21T21:40:53.997 INFO:teuthology.orchestra.run.smithi164.stdout:792 read 0x43cec00 thru 0x43d761b (0x8a1c bytes) 2024-09-21T21:40:53.997 INFO:teuthology.orchestra.run.smithi164.stdout:795 read 0x27b4600 thru 0x27ba908 (0x6309 bytes) 2024-09-21T21:40:53.998 INFO:teuthology.orchestra.run.smithi164.stdout:798 punch from 0x654ce00 to 0x6551e00, (0x5000 bytes) 2024-09-21T21:40:53.998 INFO:teuthology.orchestra.run.smithi164.stdout:800 write 0x1119200 thru 0x1124e35 (0xbc36 bytes) 2024-09-21T21:40:54.002 INFO:teuthology.orchestra.run.smithi164.stdout:804 read 0x25b1000 thru 0x25b1ee4 (0xee5 bytes) 2024-09-21T21:40:54.003 INFO:teuthology.orchestra.run.smithi164.stdout:813 punch from 0x2e8e200 to 0x2e8f200, (0x1000 bytes) 2024-09-21T21:40:54.003 INFO:teuthology.orchestra.run.smithi164.stdout:814 write 0x4194400 thru 0x41992f5 (0x4ef6 bytes) 2024-09-21T21:40:54.004 INFO:teuthology.orchestra.run.smithi164.stdout:815 trunc from 0x6da5000 to 0x820a400 2024-09-21T21:40:54.022 INFO:teuthology.orchestra.run.smithi164.stdout:816 write 0x6705e00 thru 0x670e531 (0x8732 bytes) 2024-09-21T21:40:54.041 INFO:teuthology.orchestra.run.smithi164.stdout:817 write 0xb831800 thru 0xb833d7e (0x257f bytes) 2024-09-21T21:40:54.041 INFO:teuthology.orchestra.run.smithi164.stdout:819 punch from 0x6782400 to 0x6792000, (0xfc00 bytes) 2024-09-21T21:40:54.041 INFO:teuthology.orchestra.run.smithi164.stdout:820 read 0x914bc00 thru 0x915779f (0xbba0 bytes) 2024-09-21T21:40:54.047 INFO:teuthology.orchestra.run.smithi164.stdout:821 trunc from 0xb833e00 to 0x67d8c00 2024-09-21T21:40:54.067 INFO:teuthology.orchestra.run.smithi164.stdout:822 write 0x2878800 thru 0x28820f5 (0x98f6 bytes) 2024-09-21T21:40:54.069 INFO:teuthology.orchestra.run.smithi164.stdout:823 read 0x22b6200 thru 0x22bbc07 (0x5a08 bytes) 2024-09-21T21:40:54.069 INFO:teuthology.orchestra.run.smithi164.stdout:824 write 0x22bf800 thru 0x22c73a2 (0x7ba3 bytes) 2024-09-21T21:40:54.082 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:40:54.081+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:40:54.088 INFO:teuthology.orchestra.run.smithi164.stdout:829 write 0x8a91c00 thru 0x8a9d896 (0xbc97 bytes) 2024-09-21T21:40:54.095 INFO:teuthology.orchestra.run.smithi164.stdout:830 trunc from 0x8a9da00 to 0xe4efc00 2024-09-21T21:40:54.101 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:40:54.101 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-21T21:40:54.101 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-21T21:40:54.101 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:40:54.101 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.101 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4183.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T21:40:54.101 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:19.643261+0000", 2024-09-21T21:40:54.101 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 34.454551985000002, 2024-09-21T21:40:54.101 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0075300310000000004, 2024-09-21T21:40:54.101 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:54.101 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:54.102 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:54.102 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:54.102 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4183", 2024-09-21T21:40:54.102 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.122:0/3913072034", 2024-09-21T21:40:54.102 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 112 2024-09-21T21:40:54.102 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.102 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:54.102 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.102 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:54.102 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643261+0000", 2024-09-21T21:40:54.102 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:54.102 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.102 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.102 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:54.103 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643261+0000", 2024-09-21T21:40:54.103 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:54.103 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.103 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.103 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:54.103 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643262+0000", 2024-09-21T21:40:54.103 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.99e-07 2024-09-21T21:40:54.103 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.103 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.103 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:54.103 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643326+0000", 2024-09-21T21:40:54.103 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4244000000000004e-05 2024-09-21T21:40:54.103 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.103 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.103 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:54.104 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643338+0000", 2024-09-21T21:40:54.104 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1807999999999999e-05 2024-09-21T21:40:54.104 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.104 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.104 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:54.105 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643344+0000", 2024-09-21T21:40:54.105 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4479999999999999e-06 2024-09-21T21:40:54.105 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.105 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.105 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:54.105 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.647517+0000", 2024-09-21T21:40:54.105 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.004172711 2024-09-21T21:40:54.106 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.106 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.106 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:54.106 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.647570+0000", 2024-09-21T21:40:54.106 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3094999999999999e-05 2024-09-21T21:40:54.106 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.106 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.106 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:40:54.106 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.647660+0000", 2024-09-21T21:40:54.106 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.0428999999999998e-05 2024-09-21T21:40:54.106 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.106 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.106 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:54.106 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.649208+0000", 2024-09-21T21:40:54.106 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0015479269999999999 2024-09-21T21:40:54.107 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.107 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.107 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:54.107 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.650733+0000", 2024-09-21T21:40:54.107 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0015247710000000001 2024-09-21T21:40:54.107 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.107 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.107 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:54.107 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.650764+0000", 2024-09-21T21:40:54.107 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0840000000000003e-05 2024-09-21T21:40:54.107 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.107 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.107 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:54.107 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.650791+0000", 2024-09-21T21:40:54.107 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7158999999999999e-05 2024-09-21T21:40:54.108 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:54.108 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:54.108 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:54.108 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.108 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.108 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4183.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T21:40:54.108 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:19.643392+0000", 2024-09-21T21:40:54.108 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 34.454420495999997, 2024-09-21T21:40:54.108 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0075764780000000002, 2024-09-21T21:40:54.108 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:54.108 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:54.108 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:54.108 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:54.108 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4183", 2024-09-21T21:40:54.109 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.122:0/3913072034", 2024-09-21T21:40:54.109 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 114 2024-09-21T21:40:54.109 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.109 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:54.109 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.109 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:54.109 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643392+0000", 2024-09-21T21:40:54.109 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:54.109 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.109 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.109 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:54.109 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643392+0000", 2024-09-21T21:40:54.109 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:54.109 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.110 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.110 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:54.110 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643393+0000", 2024-09-21T21:40:54.110 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2900000000000003e-07 2024-09-21T21:40:54.110 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.110 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.110 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:54.110 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643398+0000", 2024-09-21T21:40:54.110 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3159999999999998e-06 2024-09-21T21:40:54.110 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.110 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.110 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:54.110 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643403+0000", 2024-09-21T21:40:54.110 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5419999999999996e-06 2024-09-21T21:40:54.110 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.111 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.111 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:54.111 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643407+0000", 2024-09-21T21:40:54.111 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4649999999999999e-06 2024-09-21T21:40:54.111 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.111 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.111 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:54.111 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.648632+0000", 2024-09-21T21:40:54.111 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0052258000000000001 2024-09-21T21:40:54.111 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.111 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.111 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:54.111 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.648734+0000", 2024-09-21T21:40:54.111 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010178099999999999 2024-09-21T21:40:54.111 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.112 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.112 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:40:54.112 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.648880+0000", 2024-09-21T21:40:54.112 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014552799999999999 2024-09-21T21:40:54.112 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.112 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.112 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:54.112 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.650398+0000", 2024-09-21T21:40:54.112 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0015188389999999999 2024-09-21T21:40:54.112 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.112 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.112 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:54.112 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.650903+0000", 2024-09-21T21:40:54.112 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00050413400000000005 2024-09-21T21:40:54.112 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.113 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.113 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:54.113 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.650940+0000", 2024-09-21T21:40:54.113 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7354999999999997e-05 2024-09-21T21:40:54.113 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.113 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.113 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:54.113 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.650969+0000", 2024-09-21T21:40:54.113 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8989e-05 2024-09-21T21:40:54.113 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:54.113 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:54.113 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:54.113 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.113 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.113 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4183.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T21:40:54.113 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:19.643416+0000", 2024-09-21T21:40:54.114 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 34.454396557000003, 2024-09-21T21:40:54.114 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0076828269999999997, 2024-09-21T21:40:54.114 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:54.114 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:54.114 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:54.114 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:54.114 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4183", 2024-09-21T21:40:54.114 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.122:0/3913072034", 2024-09-21T21:40:54.114 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 115 2024-09-21T21:40:54.114 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.114 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:54.114 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.114 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:54.114 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643416+0000", 2024-09-21T21:40:54.115 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:54.115 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.115 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.115 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:54.115 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643416+0000", 2024-09-21T21:40:54.115 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:54.115 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.115 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.115 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:54.115 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643417+0000", 2024-09-21T21:40:54.115 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.96e-07 2024-09-21T21:40:54.115 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.115 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.115 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:54.116 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643469+0000", 2024-09-21T21:40:54.116 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2293999999999998e-05 2024-09-21T21:40:54.116 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.116 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.116 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:54.116 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643474+0000", 2024-09-21T21:40:54.116 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3179999999999997e-06 2024-09-21T21:40:54.116 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.116 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.116 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:54.116 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643488+0000", 2024-09-21T21:40:54.116 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3241000000000001e-05 2024-09-21T21:40:54.116 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.116 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.116 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:54.117 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.649349+0000", 2024-09-21T21:40:54.117 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0058617690000000002 2024-09-21T21:40:54.117 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.117 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.117 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:54.117 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.649440+0000", 2024-09-21T21:40:54.117 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.1092000000000001e-05 2024-09-21T21:40:54.117 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.117 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.117 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:40:54.117 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.649583+0000", 2024-09-21T21:40:54.117 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014218499999999999 2024-09-21T21:40:54.117 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.118 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.118 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:54.118 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.650978+0000", 2024-09-21T21:40:54.118 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0013956210000000001 2024-09-21T21:40:54.118 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.118 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.118 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:54.118 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.651032+0000", 2024-09-21T21:40:54.118 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3513000000000001e-05 2024-09-21T21:40:54.118 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.118 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.118 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:54.118 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.651068+0000", 2024-09-21T21:40:54.118 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6675999999999999e-05 2024-09-21T21:40:54.119 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.119 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.119 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:54.119 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.651099+0000", 2024-09-21T21:40:54.119 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0821999999999997e-05 2024-09-21T21:40:54.119 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:54.119 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:54.119 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:54.119 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.119 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.119 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4183.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T21:40:54.119 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:19.643496+0000", 2024-09-21T21:40:54.119 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 34.454317011999997, 2024-09-21T21:40:54.119 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0077031249999999999, 2024-09-21T21:40:54.120 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:54.120 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:54.120 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:54.120 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:54.120 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4183", 2024-09-21T21:40:54.120 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.122:0/3913072034", 2024-09-21T21:40:54.120 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 116 2024-09-21T21:40:54.120 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.120 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:54.120 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.120 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:54.120 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643496+0000", 2024-09-21T21:40:54.120 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:54.120 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.120 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.121 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:54.121 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643496+0000", 2024-09-21T21:40:54.121 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:54.121 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.121 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.121 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:54.121 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643496+0000", 2024-09-21T21:40:54.121 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3599999999999999e-07 2024-09-21T21:40:54.121 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.121 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.121 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:54.121 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643500+0000", 2024-09-21T21:40:54.121 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1790000000000001e-06 2024-09-21T21:40:54.121 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.121 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.122 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:54.122 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643504+0000", 2024-09-21T21:40:54.122 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8299999999999998e-06 2024-09-21T21:40:54.122 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.122 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.122 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:54.122 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643507+0000", 2024-09-21T21:40:54.122 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.056e-06 2024-09-21T21:40:54.122 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.122 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.122 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:54.122 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.650003+0000", 2024-09-21T21:40:54.122 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0064959129999999999 2024-09-21T21:40:54.122 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.122 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.123 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:54.123 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.650051+0000", 2024-09-21T21:40:54.123 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8180999999999998e-05 2024-09-21T21:40:54.123 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.123 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.123 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:40:54.123 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.650127+0000", 2024-09-21T21:40:54.123 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5795000000000002e-05 2024-09-21T21:40:54.123 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.123 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.123 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:54.123 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.651137+0000", 2024-09-21T21:40:54.123 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0010094699999999999 2024-09-21T21:40:54.123 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.123 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.123 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:54.124 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.651144+0000", 2024-09-21T21:40:54.124 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.985e-06 2024-09-21T21:40:54.124 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.124 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.124 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:54.124 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.651169+0000", 2024-09-21T21:40:54.124 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.563e-05 2024-09-21T21:40:54.124 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.124 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.124 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:54.124 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.651199+0000", 2024-09-21T21:40:54.124 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9750000000000001e-05 2024-09-21T21:40:54.124 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:54.124 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:54.124 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:54.124 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.124 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.125 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4183.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T21:40:54.125 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:19.643512+0000", 2024-09-21T21:40:54.125 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 34.454301397999998, 2024-09-21T21:40:54.125 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0077806539999999997, 2024-09-21T21:40:54.125 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:54.125 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:54.125 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:54.125 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:54.125 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4183", 2024-09-21T21:40:54.125 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.122:0/3913072034", 2024-09-21T21:40:54.125 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 117 2024-09-21T21:40:54.125 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.125 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:54.125 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.125 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:54.125 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643512+0000", 2024-09-21T21:40:54.126 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:54.126 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.126 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.126 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:54.126 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643512+0000", 2024-09-21T21:40:54.126 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:54.126 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.126 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.126 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:54.126 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643512+0000", 2024-09-21T21:40:54.126 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.16e-07 2024-09-21T21:40:54.126 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.126 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.126 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:54.126 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643514+0000", 2024-09-21T21:40:54.126 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3709999999999998e-06 2024-09-21T21:40:54.127 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.127 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.127 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:54.127 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643517+0000", 2024-09-21T21:40:54.127 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2310000000000001e-06 2024-09-21T21:40:54.127 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.127 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.127 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:54.127 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643522+0000", 2024-09-21T21:40:54.127 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8319999999999996e-06 2024-09-21T21:40:54.127 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.127 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.127 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:54.128 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.650426+0000", 2024-09-21T21:40:54.128 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0069033380000000002 2024-09-21T21:40:54.128 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.128 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.128 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:54.128 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.650482+0000", 2024-09-21T21:40:54.128 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6468999999999998e-05 2024-09-21T21:40:54.128 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.128 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.128 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:40:54.128 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.650565+0000", 2024-09-21T21:40:54.128 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3026000000000001e-05 2024-09-21T21:40:54.128 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.128 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.129 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:54.129 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.651209+0000", 2024-09-21T21:40:54.129 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00064387199999999998 2024-09-21T21:40:54.129 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.129 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.129 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:54.129 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.651238+0000", 2024-09-21T21:40:54.129 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8799e-05 2024-09-21T21:40:54.129 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.129 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.129 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:54.129 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.651270+0000", 2024-09-21T21:40:54.129 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1934000000000003e-05 2024-09-21T21:40:54.129 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.129 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.130 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:54.130 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.651292+0000", 2024-09-21T21:40:54.130 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2566e-05 2024-09-21T21:40:54.130 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:54.130 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:54.130 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:54.130 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.130 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.130 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4183.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T21:40:54.130 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:19.916463+0000", 2024-09-21T21:40:54.130 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 34.181349914999998, 2024-09-21T21:40:54.130 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0094252929999999995, 2024-09-21T21:40:54.130 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:54.130 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:54.131 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:54.131 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:54.131 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4183", 2024-09-21T21:40:54.131 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.122:0/3913072034", 2024-09-21T21:40:54.131 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 134 2024-09-21T21:40:54.131 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.131 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:54.131 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.131 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:54.131 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916463+0000", 2024-09-21T21:40:54.131 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:54.131 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.131 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.131 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:54.132 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916463+0000", 2024-09-21T21:40:54.132 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:54.132 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.132 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.132 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:54.132 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916464+0000", 2024-09-21T21:40:54.132 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2900000000000004e-07 2024-09-21T21:40:54.132 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.132 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.132 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:54.132 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916525+0000", 2024-09-21T21:40:54.132 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1576999999999997e-05 2024-09-21T21:40:54.132 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.132 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.133 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:54.133 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916533+0000", 2024-09-21T21:40:54.133 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7510000000000008e-06 2024-09-21T21:40:54.133 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.133 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.133 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:54.133 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916537+0000", 2024-09-21T21:40:54.133 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.155e-06 2024-09-21T21:40:54.133 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.133 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.133 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:54.133 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.919134+0000", 2024-09-21T21:40:54.133 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025973469999999999 2024-09-21T21:40:54.133 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.134 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.134 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:54.134 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.919189+0000", 2024-09-21T21:40:54.134 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4339999999999998e-05 2024-09-21T21:40:54.134 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.134 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.134 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:40:54.134 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.919261+0000", 2024-09-21T21:40:54.134 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1977000000000006e-05 2024-09-21T21:40:54.134 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.134 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.134 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:54.134 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.920953+0000", 2024-09-21T21:40:54.134 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0016926109999999999 2024-09-21T21:40:54.134 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.135 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.135 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:54.135 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.925825+0000", 2024-09-21T21:40:54.135 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048719159999999996 2024-09-21T21:40:54.135 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.135 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.135 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:54.135 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.925854+0000", 2024-09-21T21:40:54.135 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8524999999999999e-05 2024-09-21T21:40:54.135 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.135 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.135 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:54.135 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.925888+0000", 2024-09-21T21:40:54.136 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4564999999999999e-05 2024-09-21T21:40:54.136 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:54.136 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:54.136 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:54.136 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.136 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.136 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4183.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T21:40:54.136 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:19.916545+0000", 2024-09-21T21:40:54.136 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 34.181267998000003, 2024-09-21T21:40:54.136 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0094280869999999999, 2024-09-21T21:40:54.136 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:54.136 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:54.136 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:54.136 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:54.136 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4183", 2024-09-21T21:40:54.137 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.122:0/3913072034", 2024-09-21T21:40:54.137 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 135 2024-09-21T21:40:54.137 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.137 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:54.137 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.137 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:54.137 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916545+0000", 2024-09-21T21:40:54.137 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:54.137 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.137 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.137 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:54.137 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916545+0000", 2024-09-21T21:40:54.137 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:54.137 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.137 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.138 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:54.138 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916545+0000", 2024-09-21T21:40:54.138 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1100000000000002e-07 2024-09-21T21:40:54.138 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.138 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.138 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:54.138 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916549+0000", 2024-09-21T21:40:54.138 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0799999999999999e-06 2024-09-21T21:40:54.138 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.138 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.138 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:54.138 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916556+0000", 2024-09-21T21:40:54.138 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0339999999999999e-06 2024-09-21T21:40:54.138 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.138 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.139 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:54.139 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916562+0000", 2024-09-21T21:40:54.139 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5439999999999998e-06 2024-09-21T21:40:54.139 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.139 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.139 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:54.139 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.919649+0000", 2024-09-21T21:40:54.139 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0030868660000000002 2024-09-21T21:40:54.139 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.139 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.139 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:54.139 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.919693+0000", 2024-09-21T21:40:54.139 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4694000000000003e-05 2024-09-21T21:40:54.139 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.140 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.140 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:40:54.140 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.919765+0000", 2024-09-21T21:40:54.140 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1427999999999994e-05 2024-09-21T21:40:54.140 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.140 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.140 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:54.140 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.921005+0000", 2024-09-21T21:40:54.140 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001240314 2024-09-21T21:40:54.140 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.140 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.140 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:54.140 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.925897+0000", 2024-09-21T21:40:54.140 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048916660000000002 2024-09-21T21:40:54.140 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.141 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.141 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:54.141 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.925942+0000", 2024-09-21T21:40:54.141 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4721000000000001e-05 2024-09-21T21:40:54.141 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.141 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.141 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:54.141 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.925973+0000", 2024-09-21T21:40:54.141 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1429e-05 2024-09-21T21:40:54.141 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:54.141 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:54.141 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:54.141 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.141 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.141 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4183.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T21:40:54.142 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:19.916568+0000", 2024-09-21T21:40:54.142 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 34.181245064000002, 2024-09-21T21:40:54.142 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0094755120000000002, 2024-09-21T21:40:54.142 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:54.142 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:54.142 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:54.142 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:54.142 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4183", 2024-09-21T21:40:54.142 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.122:0/3913072034", 2024-09-21T21:40:54.142 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 136 2024-09-21T21:40:54.142 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.142 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:54.142 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.142 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:54.143 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916568+0000", 2024-09-21T21:40:54.143 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:54.143 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.143 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.143 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:54.143 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916568+0000", 2024-09-21T21:40:54.143 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:54.143 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.143 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.143 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:54.143 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916568+0000", 2024-09-21T21:40:54.143 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9200000000000002e-07 2024-09-21T21:40:54.143 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.143 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.143 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:54.143 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916572+0000", 2024-09-21T21:40:54.144 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.428e-06 2024-09-21T21:40:54.144 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.144 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.144 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:54.144 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916576+0000", 2024-09-21T21:40:54.144 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.814e-06 2024-09-21T21:40:54.144 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.144 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.144 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:54.144 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916580+0000", 2024-09-21T21:40:54.144 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6669999999999998e-06 2024-09-21T21:40:54.144 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.144 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.144 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:54.145 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.920045+0000", 2024-09-21T21:40:54.145 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0034645209999999999 2024-09-21T21:40:54.145 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.145 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.145 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:54.145 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.920091+0000", 2024-09-21T21:40:54.145 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6146999999999997e-05 2024-09-21T21:40:54.145 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.145 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.145 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:40:54.145 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.920159+0000", 2024-09-21T21:40:54.145 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8411000000000006e-05 2024-09-21T21:40:54.145 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.145 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.146 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:54.146 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.921037+0000", 2024-09-21T21:40:54.146 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00087794700000000002 2024-09-21T21:40:54.146 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.146 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.146 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:54.146 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.925981+0000", 2024-09-21T21:40:54.146 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0049443259999999998 2024-09-21T21:40:54.146 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.146 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.146 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:54.146 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.926014+0000", 2024-09-21T21:40:54.146 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2298999999999997e-05 2024-09-21T21:40:54.146 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.146 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.147 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:54.147 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.926043+0000", 2024-09-21T21:40:54.147 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9660000000000001e-05 2024-09-21T21:40:54.147 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:54.147 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:54.147 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:54.147 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.147 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.147 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4183.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T21:40:54.147 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:19.916587+0000", 2024-09-21T21:40:54.147 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 34.181225584000003, 2024-09-21T21:40:54.147 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.009527265, 2024-09-21T21:40:54.147 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:54.147 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:54.148 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:54.148 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:54.148 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4183", 2024-09-21T21:40:54.148 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.122:0/3913072034", 2024-09-21T21:40:54.148 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 137 2024-09-21T21:40:54.148 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.148 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:54.148 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.148 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:54.148 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916587+0000", 2024-09-21T21:40:54.148 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:54.148 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.148 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.148 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:54.149 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916587+0000", 2024-09-21T21:40:54.149 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:54.149 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.149 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.149 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:54.149 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916588+0000", 2024-09-21T21:40:54.149 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5699999999999999e-07 2024-09-21T21:40:54.149 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.149 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.149 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:54.149 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916644+0000", 2024-09-21T21:40:54.149 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6344000000000001e-05 2024-09-21T21:40:54.149 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.149 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.150 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:54.150 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916649+0000", 2024-09-21T21:40:54.150 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0409999999999997e-06 2024-09-21T21:40:54.150 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.150 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.150 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:54.150 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916654+0000", 2024-09-21T21:40:54.150 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9919999999999998e-06 2024-09-21T21:40:54.150 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.150 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.150 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:54.150 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.920443+0000", 2024-09-21T21:40:54.150 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0037892469999999999 2024-09-21T21:40:54.150 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.151 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.151 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:54.151 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.920493+0000", 2024-09-21T21:40:54.151 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9898000000000002e-05 2024-09-21T21:40:54.151 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.151 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.151 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:40:54.151 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.920561+0000", 2024-09-21T21:40:54.151 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7481e-05 2024-09-21T21:40:54.151 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.151 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.151 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:54.151 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.925739+0000", 2024-09-21T21:40:54.151 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.005178729 2024-09-21T21:40:54.151 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.152 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.152 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:54.152 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.926052+0000", 2024-09-21T21:40:54.152 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00031252800000000001 2024-09-21T21:40:54.152 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.152 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.152 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:54.152 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.926087+0000", 2024-09-21T21:40:54.152 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4873999999999998e-05 2024-09-21T21:40:54.152 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.152 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.152 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:54.152 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.926115+0000", 2024-09-21T21:40:54.152 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7874000000000001e-05 2024-09-21T21:40:54.153 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:54.153 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:54.153 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:54.153 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.153 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.153 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4183.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T21:40:54.153 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:19.916666+0000", 2024-09-21T21:40:54.153 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 34.181146712999997, 2024-09-21T21:40:54.153 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0095150730000000006, 2024-09-21T21:40:54.153 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:54.153 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:54.153 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:54.153 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:54.153 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4183", 2024-09-21T21:40:54.153 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.122:0/3913072034", 2024-09-21T21:40:54.154 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 138 2024-09-21T21:40:54.154 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.154 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:54.154 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.154 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:54.154 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916666+0000", 2024-09-21T21:40:54.154 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:54.154 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.154 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.154 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:54.154 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916666+0000", 2024-09-21T21:40:54.154 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:54.154 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.155 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.155 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:54.155 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916667+0000", 2024-09-21T21:40:54.155 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3200000000000005e-07 2024-09-21T21:40:54.155 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.155 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.155 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:54.155 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916670+0000", 2024-09-21T21:40:54.155 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.427e-06 2024-09-21T21:40:54.155 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.155 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.155 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:54.155 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916684+0000", 2024-09-21T21:40:54.155 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3839000000000001e-05 2024-09-21T21:40:54.155 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.156 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.156 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:54.156 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916706+0000", 2024-09-21T21:40:54.156 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1543e-05 2024-09-21T21:40:54.156 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.156 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.156 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:54.156 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.921064+0000", 2024-09-21T21:40:54.156 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0043587119999999998 2024-09-21T21:40:54.156 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.156 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.156 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:54.156 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.921119+0000", 2024-09-21T21:40:54.156 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4619e-05 2024-09-21T21:40:54.156 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.157 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.157 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:40:54.157 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.921192+0000", 2024-09-21T21:40:54.157 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3224000000000002e-05 2024-09-21T21:40:54.157 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.157 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.157 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:54.157 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.925791+0000", 2024-09-21T21:40:54.157 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0045988670000000004 2024-09-21T21:40:54.157 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.157 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.157 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:54.157 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.926123+0000", 2024-09-21T21:40:54.157 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00033195000000000002 2024-09-21T21:40:54.158 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.158 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.158 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:54.158 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.926154+0000", 2024-09-21T21:40:54.158 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0720999999999999e-05 2024-09-21T21:40:54.158 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.158 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.158 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:54.158 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.926181+0000", 2024-09-21T21:40:54.158 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7639000000000001e-05 2024-09-21T21:40:54.158 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:54.158 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:54.158 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:54.158 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.158 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.159 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4183.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T21:40:54.159 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:19.916714+0000", 2024-09-21T21:40:54.159 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 34.181098824000003, 2024-09-21T21:40:54.159 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0095298369999999993, 2024-09-21T21:40:54.159 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:54.159 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:54.159 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:54.159 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:54.159 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4183", 2024-09-21T21:40:54.159 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.122:0/3913072034", 2024-09-21T21:40:54.159 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 139 2024-09-21T21:40:54.159 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.159 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:54.159 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.159 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:54.159 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916714+0000", 2024-09-21T21:40:54.160 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:54.160 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.160 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.160 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:54.160 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916714+0000", 2024-09-21T21:40:54.160 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:54.160 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.160 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.160 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:54.160 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916714+0000", 2024-09-21T21:40:54.160 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9799999999999999e-07 2024-09-21T21:40:54.160 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.160 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.160 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:54.161 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916718+0000", 2024-09-21T21:40:54.161 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1810000000000001e-06 2024-09-21T21:40:54.161 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.161 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.161 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:54.161 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916724+0000", 2024-09-21T21:40:54.161 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7329999999999997e-06 2024-09-21T21:40:54.161 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.161 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.161 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:54.161 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916728+0000", 2024-09-21T21:40:54.161 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2909999999999999e-06 2024-09-21T21:40:54.161 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.161 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.162 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:54.162 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.921416+0000", 2024-09-21T21:40:54.162 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0046882850000000004 2024-09-21T21:40:54.162 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.162 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.162 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:54.162 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.921461+0000", 2024-09-21T21:40:54.162 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5259000000000002e-05 2024-09-21T21:40:54.162 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.162 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.162 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:40:54.162 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.921540+0000", 2024-09-21T21:40:54.162 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8765999999999998e-05 2024-09-21T21:40:54.162 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.163 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.163 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:54.163 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.925818+0000", 2024-09-21T21:40:54.163 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0042784800000000003 2024-09-21T21:40:54.163 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.163 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.163 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:54.163 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.926190+0000", 2024-09-21T21:40:54.163 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00037112400000000002 2024-09-21T21:40:54.163 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.163 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.163 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:54.163 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.926225+0000", 2024-09-21T21:40:54.163 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5420999999999998e-05 2024-09-21T21:40:54.163 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.164 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.164 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:54.164 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.926244+0000", 2024-09-21T21:40:54.164 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.8998999999999999e-05 2024-09-21T21:40:54.164 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:54.164 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:54.164 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:54.164 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.164 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.164 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4183.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T21:40:54.164 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:19.916735+0000", 2024-09-21T21:40:54.164 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 34.181078427000003, 2024-09-21T21:40:54.164 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.010302162, 2024-09-21T21:40:54.164 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:54.164 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:54.165 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:54.165 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:54.165 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4183", 2024-09-21T21:40:54.165 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.122:0/3913072034", 2024-09-21T21:40:54.165 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 140 2024-09-21T21:40:54.165 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.165 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:54.165 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.165 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:54.165 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916735+0000", 2024-09-21T21:40:54.165 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:54.165 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.165 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.165 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:54.165 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916735+0000", 2024-09-21T21:40:54.166 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:54.166 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.166 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.166 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:54.166 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916735+0000", 2024-09-21T21:40:54.166 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5400000000000002e-07 2024-09-21T21:40:54.166 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.166 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.166 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:54.166 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916788+0000", 2024-09-21T21:40:54.166 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3074e-05 2024-09-21T21:40:54.166 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.166 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.166 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:54.166 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916795+0000", 2024-09-21T21:40:54.167 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.204e-06 2024-09-21T21:40:54.167 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.167 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.167 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:54.167 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916801+0000", 2024-09-21T21:40:54.167 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4759999999999997e-06 2024-09-21T21:40:54.167 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.167 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.167 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:54.167 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.921792+0000", 2024-09-21T21:40:54.167 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0049909749999999999 2024-09-21T21:40:54.167 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.167 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.167 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:54.168 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.921879+0000", 2024-09-21T21:40:54.168 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7361999999999995e-05 2024-09-21T21:40:54.168 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.168 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.168 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:40:54.168 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.922000+0000", 2024-09-21T21:40:54.168 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000121451 2024-09-21T21:40:54.168 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.168 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.168 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:54.168 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.926291+0000", 2024-09-21T21:40:54.168 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0042906180000000004 2024-09-21T21:40:54.168 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.168 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.168 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:54.169 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.926984+0000", 2024-09-21T21:40:54.169 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00069308599999999998 2024-09-21T21:40:54.169 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.169 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.169 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:54.169 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.927014+0000", 2024-09-21T21:40:54.169 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0222999999999998e-05 2024-09-21T21:40:54.169 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.169 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.169 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:54.169 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.927037+0000", 2024-09-21T21:40:54.169 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2439e-05 2024-09-21T21:40:54.169 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:54.169 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:54.170 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:54.170 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.170 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.170 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4183.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T21:40:54.170 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:19.916814+0000", 2024-09-21T21:40:54.170 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 34.180999237999998, 2024-09-21T21:40:54.170 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.010507598999999999, 2024-09-21T21:40:54.170 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:54.170 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:54.170 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:54.170 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:54.170 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4183", 2024-09-21T21:40:54.170 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.122:0/3913072034", 2024-09-21T21:40:54.170 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 141 2024-09-21T21:40:54.171 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.171 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:54.171 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.171 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:54.171 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916814+0000", 2024-09-21T21:40:54.171 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:54.171 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.171 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.171 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:54.171 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916814+0000", 2024-09-21T21:40:54.171 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:54.171 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.171 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.171 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:54.171 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916814+0000", 2024-09-21T21:40:54.172 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9599999999999999e-07 2024-09-21T21:40:54.172 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.172 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.172 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:54.172 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916819+0000", 2024-09-21T21:40:54.172 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7090000000000001e-06 2024-09-21T21:40:54.172 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.172 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.172 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:54.172 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916830+0000", 2024-09-21T21:40:54.172 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0977e-05 2024-09-21T21:40:54.172 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.172 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.172 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:54.173 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916835+0000", 2024-09-21T21:40:54.173 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2320000000000001e-06 2024-09-21T21:40:54.173 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.173 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.173 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:54.173 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.925249+0000", 2024-09-21T21:40:54.173 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0084135170000000006 2024-09-21T21:40:54.173 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.173 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.173 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:54.173 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.925323+0000", 2024-09-21T21:40:54.173 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3919999999999997e-05 2024-09-21T21:40:54.173 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.173 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.173 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:40:54.173 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.925442+0000", 2024-09-21T21:40:54.174 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00011944500000000001 2024-09-21T21:40:54.174 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.174 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.174 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:54.174 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.926322+0000", 2024-09-21T21:40:54.174 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00087985500000000003 2024-09-21T21:40:54.174 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.174 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.174 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:54.174 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.927245+0000", 2024-09-21T21:40:54.174 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00092342499999999996 2024-09-21T21:40:54.174 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.174 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.175 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:54.175 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.927283+0000", 2024-09-21T21:40:54.175 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8157000000000001e-05 2024-09-21T21:40:54.175 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.175 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.175 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:54.175 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.927321+0000", 2024-09-21T21:40:54.175 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7866e-05 2024-09-21T21:40:54.175 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:54.175 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:54.175 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:54.175 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.175 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.175 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4417.0:496 3.d 3:b065aa9d:::rbd_data.11214be7c411.0000000000000074:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:54.176 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.604764+0000", 2024-09-21T21:40:54.176 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 10.493048504000001, 2024-09-21T21:40:54.176 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0080497150000000007, 2024-09-21T21:40:54.176 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:54.176 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:54.176 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:54.176 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:54.176 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4417", 2024-09-21T21:40:54.176 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:54.176 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 496 2024-09-21T21:40:54.176 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.176 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:54.176 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.176 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:54.177 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.604764+0000", 2024-09-21T21:40:54.177 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:54.177 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.177 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.177 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:54.177 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.604764+0000", 2024-09-21T21:40:54.177 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:54.177 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.177 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.177 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:54.177 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.604765+0000", 2024-09-21T21:40:54.177 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6300000000000005e-07 2024-09-21T21:40:54.177 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.177 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.178 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:54.178 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.604768+0000", 2024-09-21T21:40:54.178 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0070000000000002e-06 2024-09-21T21:40:54.178 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.178 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.178 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:54.178 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.604774+0000", 2024-09-21T21:40:54.178 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2689999999999996e-06 2024-09-21T21:40:54.178 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.178 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.178 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:54.178 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.604778+0000", 2024-09-21T21:40:54.178 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.749e-06 2024-09-21T21:40:54.178 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.178 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.179 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:54.179 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.605207+0000", 2024-09-21T21:40:54.179 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00042899099999999999 2024-09-21T21:40:54.179 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.179 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.179 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:54.179 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.605251+0000", 2024-09-21T21:40:54.179 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4258999999999998e-05 2024-09-21T21:40:54.179 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.179 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.179 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 2", 2024-09-21T21:40:54.179 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.605430+0000", 2024-09-21T21:40:54.179 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00017844700000000001 2024-09-21T21:40:54.179 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.180 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.180 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:54.180 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.606879+0000", 2024-09-21T21:40:54.180 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001449129 2024-09-21T21:40:54.180 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.180 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.180 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:54.180 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.612748+0000", 2024-09-21T21:40:54.180 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0058691510000000004 2024-09-21T21:40:54.180 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.180 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.180 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:54.180 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.612785+0000", 2024-09-21T21:40:54.180 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6730999999999998e-05 2024-09-21T21:40:54.180 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.181 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.181 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:54.181 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.612814+0000", 2024-09-21T21:40:54.181 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9419000000000001e-05 2024-09-21T21:40:54.181 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:54.181 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:54.181 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:54.181 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.181 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.181 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4417.0:497 3.d 3:b7efb195:::rbd_data.11214be7c411.0000000000000076:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:54.181 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.604904+0000", 2024-09-21T21:40:54.181 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 10.492909341000001, 2024-09-21T21:40:54.181 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0080130610000000001, 2024-09-21T21:40:54.181 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:54.182 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:54.182 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:54.182 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:54.182 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4417", 2024-09-21T21:40:54.182 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:54.182 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 497 2024-09-21T21:40:54.182 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.182 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:54.182 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.182 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:54.182 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.604904+0000", 2024-09-21T21:40:54.182 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:54.182 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.182 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.182 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:54.183 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.604904+0000", 2024-09-21T21:40:54.183 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:54.183 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.183 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.183 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:54.183 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.604904+0000", 2024-09-21T21:40:54.183 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5099999999999999e-07 2024-09-21T21:40:54.183 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.183 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.183 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:54.183 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.604909+0000", 2024-09-21T21:40:54.183 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6959999999999996e-06 2024-09-21T21:40:54.183 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.184 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.184 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:54.184 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.604918+0000", 2024-09-21T21:40:54.184 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7229999999999993e-06 2024-09-21T21:40:54.184 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.184 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.184 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:54.184 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.604924+0000", 2024-09-21T21:40:54.184 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9909999999999999e-06 2024-09-21T21:40:54.184 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.184 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.184 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:54.184 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.605720+0000", 2024-09-21T21:40:54.184 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00079632800000000001 2024-09-21T21:40:54.185 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.185 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.185 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:54.185 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.605800+0000", 2024-09-21T21:40:54.185 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0302999999999993e-05 2024-09-21T21:40:54.185 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.185 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.185 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 2", 2024-09-21T21:40:54.185 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.605960+0000", 2024-09-21T21:40:54.185 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000159211 2024-09-21T21:40:54.185 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.185 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.185 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:54.185 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.606887+0000", 2024-09-21T21:40:54.186 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00092735699999999996 2024-09-21T21:40:54.186 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.186 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.186 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:54.186 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.612849+0000", 2024-09-21T21:40:54.186 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0059615450000000004 2024-09-21T21:40:54.186 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.186 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.186 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:54.186 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.612892+0000", 2024-09-21T21:40:54.186 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3683000000000002e-05 2024-09-21T21:40:54.186 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.186 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.186 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:54.187 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.612917+0000", 2024-09-21T21:40:54.187 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4473000000000001e-05 2024-09-21T21:40:54.187 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:54.187 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:54.187 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:54.187 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.187 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.187 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:1191 3.a 3:50c18da3:::rbd_object_map.1121bf4f5ab3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-21T21:40:54.187 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:48.412351+0000", 2024-09-21T21:40:54.187 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 5.685461729, 2024-09-21T21:40:54.187 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0098993850000000001, 2024-09-21T21:40:54.187 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:54.187 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:54.187 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:54.187 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:54.188 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2024-09-21T21:40:54.188 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1688278210", 2024-09-21T21:40:54.188 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1191 2024-09-21T21:40:54.188 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.188 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:54.188 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.188 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:54.188 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.412351+0000", 2024-09-21T21:40:54.188 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:54.188 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.188 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.188 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:54.188 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.412351+0000", 2024-09-21T21:40:54.188 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:54.189 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.189 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.189 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:54.189 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.412352+0000", 2024-09-21T21:40:54.189 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0020000000000001e-06 2024-09-21T21:40:54.189 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.189 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.189 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:54.189 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.412358+0000", 2024-09-21T21:40:54.189 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5670000000000002e-06 2024-09-21T21:40:54.189 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.189 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.189 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:54.189 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.412368+0000", 2024-09-21T21:40:54.189 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0372e-05 2024-09-21T21:40:54.189 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.190 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.190 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:54.190 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.412376+0000", 2024-09-21T21:40:54.190 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5599999999999996e-06 2024-09-21T21:40:54.190 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.190 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.190 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:54.190 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.415940+0000", 2024-09-21T21:40:54.190 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0035641750000000002 2024-09-21T21:40:54.190 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.190 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.190 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:54.190 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.416028+0000", 2024-09-21T21:40:54.190 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8189000000000006e-05 2024-09-21T21:40:54.191 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.191 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.191 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 4", 2024-09-21T21:40:54.191 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.416277+0000", 2024-09-21T21:40:54.191 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00024857500000000001 2024-09-21T21:40:54.191 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.191 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.191 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:54.191 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.418414+0000", 2024-09-21T21:40:54.191 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002137351 2024-09-21T21:40:54.191 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.191 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.191 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:54.191 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.422109+0000", 2024-09-21T21:40:54.191 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0036947899999999999 2024-09-21T21:40:54.192 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.192 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.192 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:54.192 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.422157+0000", 2024-09-21T21:40:54.192 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7729000000000003e-05 2024-09-21T21:40:54.192 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.192 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.192 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:54.192 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.422251+0000", 2024-09-21T21:40:54.192 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.4074999999999996e-05 2024-09-21T21:40:54.192 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:54.192 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:54.192 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:54.192 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.192 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.192 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:1192 3.a 3:50c18da3:::rbd_object_map.1121bf4f5ab3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-21T21:40:54.193 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:48.413403+0000", 2024-09-21T21:40:54.193 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 5.6844100710000003, 2024-09-21T21:40:54.193 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.008976543, 2024-09-21T21:40:54.193 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:54.193 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:54.193 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:54.193 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:54.193 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2024-09-21T21:40:54.193 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1688278210", 2024-09-21T21:40:54.193 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1192 2024-09-21T21:40:54.193 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.193 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:54.193 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.193 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:54.193 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.413403+0000", 2024-09-21T21:40:54.194 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:54.194 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.194 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.194 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:54.194 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.413403+0000", 2024-09-21T21:40:54.194 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:54.194 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.194 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.194 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:54.194 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.413404+0000", 2024-09-21T21:40:54.194 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.7300000000000004e-07 2024-09-21T21:40:54.194 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.194 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.194 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:54.195 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.413409+0000", 2024-09-21T21:40:54.195 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3349999999999997e-06 2024-09-21T21:40:54.195 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.195 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.195 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:54.195 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.413419+0000", 2024-09-21T21:40:54.195 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.9690000000000006e-06 2024-09-21T21:40:54.195 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.195 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.195 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:54.195 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.413435+0000", 2024-09-21T21:40:54.195 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5811000000000002e-05 2024-09-21T21:40:54.195 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.195 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.196 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:54.196 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.416928+0000", 2024-09-21T21:40:54.196 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0034933730000000001 2024-09-21T21:40:54.196 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.196 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.196 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:54.196 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.416994+0000", 2024-09-21T21:40:54.196 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5372999999999999e-05 2024-09-21T21:40:54.196 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.196 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.196 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 4", 2024-09-21T21:40:54.196 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.417276+0000", 2024-09-21T21:40:54.196 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00028256400000000001 2024-09-21T21:40:54.196 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.197 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.197 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:54.197 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.419348+0000", 2024-09-21T21:40:54.197 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020714610000000001 2024-09-21T21:40:54.197 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.197 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.197 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:54.197 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.422320+0000", 2024-09-21T21:40:54.197 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0029726750000000001 2024-09-21T21:40:54.197 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.197 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.197 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:54.197 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.422357+0000", 2024-09-21T21:40:54.197 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6130000000000001e-05 2024-09-21T21:40:54.198 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.198 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.198 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:54.198 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.422379+0000", 2024-09-21T21:40:54.198 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2878999999999999e-05 2024-09-21T21:40:54.198 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:54.198 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:54.198 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:54.198 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.198 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.198 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:1195 3.a 3:5406e24d:::rbd_data.1121bf4f5ab3.00000000000000a8:head [set-alloc-hint object_size 1048576 write_size 1048576,write 79872~27136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-21T21:40:54.198 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:48.413478+0000", 2024-09-21T21:40:54.199 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 5.6843354079999999, 2024-09-21T21:40:54.199 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.009059246, 2024-09-21T21:40:54.199 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:54.199 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:54.199 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:54.199 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:54.199 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2024-09-21T21:40:54.199 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1688278210", 2024-09-21T21:40:54.199 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1195 2024-09-21T21:40:54.199 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.199 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:54.199 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.199 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:54.199 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.413478+0000", 2024-09-21T21:40:54.200 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:54.200 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.200 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.200 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:54.200 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.413478+0000", 2024-09-21T21:40:54.200 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:54.200 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.200 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.200 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:54.200 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.413478+0000", 2024-09-21T21:40:54.200 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7599999999999997e-07 2024-09-21T21:40:54.200 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.200 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.200 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:54.200 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.413509+0000", 2024-09-21T21:40:54.201 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1018e-05 2024-09-21T21:40:54.201 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.201 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.201 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:54.201 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.413514+0000", 2024-09-21T21:40:54.201 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2789999999999998e-06 2024-09-21T21:40:54.201 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.201 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.201 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:54.201 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.413518+0000", 2024-09-21T21:40:54.201 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5499999999999999e-06 2024-09-21T21:40:54.201 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.201 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.201 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:54.202 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.417605+0000", 2024-09-21T21:40:54.202 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0040869560000000001 2024-09-21T21:40:54.202 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.202 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.202 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:54.202 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.417705+0000", 2024-09-21T21:40:54.202 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000100095 2024-09-21T21:40:54.202 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.202 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.202 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 4", 2024-09-21T21:40:54.202 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.417882+0000", 2024-09-21T21:40:54.202 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000176544 2024-09-21T21:40:54.202 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.202 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.203 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:54.203 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.421871+0000", 2024-09-21T21:40:54.203 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0039898249999999998 2024-09-21T21:40:54.203 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.203 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.203 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:54.203 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.422454+0000", 2024-09-21T21:40:54.203 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00058244000000000004 2024-09-21T21:40:54.203 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.203 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.203 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:54.203 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.422502+0000", 2024-09-21T21:40:54.203 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7895999999999997e-05 2024-09-21T21:40:54.203 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.204 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.204 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:54.204 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.422537+0000", 2024-09-21T21:40:54.204 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5067000000000002e-05 2024-09-21T21:40:54.204 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:54.204 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:54.204 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:54.204 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.204 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.204 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:2729 3.a 3:50c18da3:::rbd_object_map.1121bf4f5ab3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T21:40:54.204 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:48.740436+0000", 2024-09-21T21:40:54.204 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 5.3573764869999998, 2024-09-21T21:40:54.204 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0080237559999999999, 2024-09-21T21:40:54.204 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:54.204 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:54.205 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:54.205 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:54.205 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2024-09-21T21:40:54.205 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1688278210", 2024-09-21T21:40:54.205 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2729 2024-09-21T21:40:54.205 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.205 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:54.205 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.205 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:54.205 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.740436+0000", 2024-09-21T21:40:54.205 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:54.205 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.205 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.205 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:54.205 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.740436+0000", 2024-09-21T21:40:54.206 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:54.206 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.206 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.206 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:54.206 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.740437+0000", 2024-09-21T21:40:54.206 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0100000000000005e-07 2024-09-21T21:40:54.206 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.206 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.206 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:54.206 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.740441+0000", 2024-09-21T21:40:54.206 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1999999999999996e-06 2024-09-21T21:40:54.206 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.206 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.206 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:54.207 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.740450+0000", 2024-09-21T21:40:54.207 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3100000000000001e-06 2024-09-21T21:40:54.207 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.207 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.207 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:54.207 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.740455+0000", 2024-09-21T21:40:54.207 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3290000000000001e-06 2024-09-21T21:40:54.207 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.207 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.207 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:54.207 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.743816+0000", 2024-09-21T21:40:54.207 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0033615289999999998 2024-09-21T21:40:54.207 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.207 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.207 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:54.208 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.743904+0000", 2024-09-21T21:40:54.208 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7959000000000003e-05 2024-09-21T21:40:54.208 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.208 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.208 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 4", 2024-09-21T21:40:54.208 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.744198+0000", 2024-09-21T21:40:54.208 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00029311499999999999 2024-09-21T21:40:54.208 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.208 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.208 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:54.208 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.748327+0000", 2024-09-21T21:40:54.208 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0041299120000000003 2024-09-21T21:40:54.208 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.208 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.209 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:54.209 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.748363+0000", 2024-09-21T21:40:54.209 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5346999999999999e-05 2024-09-21T21:40:54.209 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.209 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.209 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:54.209 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.748417+0000", 2024-09-21T21:40:54.209 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4258999999999997e-05 2024-09-21T21:40:54.209 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.209 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.209 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:54.209 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.748460+0000", 2024-09-21T21:40:54.209 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3195000000000002e-05 2024-09-21T21:40:54.209 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:54.210 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:54.210 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:54.210 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.210 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.210 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:2731 3.a 3:50c18da3:::rbd_object_map.1121bf4f5ab3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T21:40:54.210 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:48.740462+0000", 2024-09-21T21:40:54.210 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 5.3573505949999998, 2024-09-21T21:40:54.210 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0092777689999999999, 2024-09-21T21:40:54.210 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:54.210 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:54.210 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:54.210 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:54.210 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2024-09-21T21:40:54.210 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1688278210", 2024-09-21T21:40:54.210 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2731 2024-09-21T21:40:54.211 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.211 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:54.211 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.211 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:54.211 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.740462+0000", 2024-09-21T21:40:54.211 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:54.211 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.211 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.211 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:54.211 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.740462+0000", 2024-09-21T21:40:54.211 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:54.211 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.211 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.211 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:54.211 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.740463+0000", 2024-09-21T21:40:54.212 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1100000000000001e-07 2024-09-21T21:40:54.212 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.212 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.212 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:54.212 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.740465+0000", 2024-09-21T21:40:54.212 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7030000000000002e-06 2024-09-21T21:40:54.212 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.212 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.212 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:54.212 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.740471+0000", 2024-09-21T21:40:54.212 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7969999999999997e-06 2024-09-21T21:40:54.212 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.212 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.212 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:54.213 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.740476+0000", 2024-09-21T21:40:54.213 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7759999999999997e-06 2024-09-21T21:40:54.213 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.213 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.213 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:54.213 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.744841+0000", 2024-09-21T21:40:54.213 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.004364639 2024-09-21T21:40:54.213 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.213 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.213 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:54.213 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.744911+0000", 2024-09-21T21:40:54.213 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0015e-05 2024-09-21T21:40:54.213 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.213 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.213 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 4", 2024-09-21T21:40:54.214 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.745246+0000", 2024-09-21T21:40:54.214 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00033522800000000002 2024-09-21T21:40:54.214 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.214 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.214 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:54.214 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.748470+0000", 2024-09-21T21:40:54.214 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0032241589999999999 2024-09-21T21:40:54.214 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.214 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.214 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:54.214 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.749650+0000", 2024-09-21T21:40:54.214 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001180204 2024-09-21T21:40:54.214 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.214 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.215 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:54.215 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.749699+0000", 2024-09-21T21:40:54.215 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8959999999999999e-05 2024-09-21T21:40:54.215 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:54.215 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:54.215 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:54.215 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.749740+0000", 2024-09-21T21:40:54.215 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0877000000000002e-05 2024-09-21T21:40:54.215 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:54.215 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:54.215 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:54.215 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:54.215 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:54.215 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:40:54.217 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0: {} 2024-09-21T21:40:54.217 INFO:teuthology.orchestra.run.smithi081.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T21:40:54.218 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1: {} 2024-09-21T21:40:54.218 INFO:teuthology.orchestra.run.smithi081.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T21:40:54.219 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T21:40:54.219 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:40:54.215+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:40:54.220 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2: {} 2024-09-21T21:40:54.220 INFO:teuthology.orchestra.run.smithi081.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T21:40:54.220 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3: {} 2024-09-21T21:40:54.221 INFO:teuthology.orchestra.run.smithi081.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T21:40:54.221 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4: {} 2024-09-21T21:40:54.221 INFO:teuthology.orchestra.run.smithi081.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T21:40:54.222 INFO:teuthology.orchestra.run.smithi164.stdout:835 punch from 0x7102800 to 0x710a400, (0x7c00 bytes) 2024-09-21T21:40:54.222 INFO:teuthology.orchestra.run.smithi164.stdout:836 write 0x4e17400 thru 0x4e1fdef (0x89f0 bytes) 2024-09-21T21:40:54.222 INFO:teuthology.orchestra.run.smithi164.stdout:838 write 0x123600 thru 0x13288b (0xf28c bytes) 2024-09-21T21:40:54.222 INFO:teuthology.orchestra.run.smithi164.stdout:839 write 0xda6c800 thru 0xda75eb7 (0x96b8 bytes) 2024-09-21T21:40:54.222 INFO:teuthology.orchestra.run.smithi164.stdout:840 read 0x4e38a00 thru 0x4e3f7a4 (0x6da5 bytes) 2024-09-21T21:40:54.222 INFO:teuthology.orchestra.run.smithi164.stdout:841 read 0x4a25800 thru 0x4a2cc60 (0x7461 bytes) 2024-09-21T21:40:54.222 INFO:teuthology.orchestra.run.smithi164.stdout:843 read 0x848c00 thru 0x849404 (0x805 bytes) 2024-09-21T21:40:54.222 INFO:teuthology.orchestra.run.smithi164.stdout:846 trunc from 0xe4efc00 to 0x8790c00 2024-09-21T21:40:54.222 INFO:teuthology.orchestra.run.smithi164.stdout:847 read 0x7d8e00 thru 0x7e4af3 (0xbcf4 bytes) 2024-09-21T21:40:54.223 INFO:teuthology.orchestra.run.smithi164.stdout:849 read 0x71a1000 thru 0x71aed98 (0xdd99 bytes) 2024-09-21T21:40:54.223 INFO:teuthology.orchestra.run.smithi164.stdout:850 write 0x2268600 thru 0x226c8ae (0x42af bytes) 2024-09-21T21:40:54.223 INFO:teuthology.orchestra.run.smithi164.stdout:852 read 0x4383800 thru 0x438a393 (0x6b94 bytes) 2024-09-21T21:40:54.223 INFO:teuthology.orchestra.run.smithi164.stdout:854 read 0x6acac00 thru 0x6ad484b (0x9c4c bytes) 2024-09-21T21:40:54.223 INFO:teuthology.orchestra.run.smithi164.stdout:855 trunc from 0x8790c00 to 0x526b200 2024-09-21T21:40:54.223 INFO:teuthology.orchestra.run.smithi164.stdout:858 read 0x1313000 thru 0x131e55f (0xb560 bytes) 2024-09-21T21:40:54.223 INFO:teuthology.orchestra.run.smithi164.stdout:859 trunc from 0x526b200 to 0x3dad200 2024-09-21T21:40:54.223 INFO:teuthology.orchestra.run.smithi164.stdout:860 punch from 0x9800 to 0x15200, (0xba00 bytes) 2024-09-21T21:40:54.223 INFO:teuthology.orchestra.run.smithi164.stdout:863 punch from 0x3bcf400 to 0x3bcf800, (0x400 bytes) 2024-09-21T21:40:54.223 INFO:teuthology.orchestra.run.smithi164.stdout:864 write 0x53c0e00 thru 0x53c3191 (0x2392 bytes) 2024-09-21T21:40:54.223 INFO:teuthology.orchestra.run.smithi164.stdout:866 trunc from 0x53c3200 to 0x3cb2400 2024-09-21T21:40:54.223 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5: {} 2024-09-21T21:40:54.224 INFO:teuthology.orchestra.run.smithi081.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T21:40:54.230 INFO:teuthology.orchestra.run.smithi164.stdout:868 read 0x562e00 thru 0x56d271 (0xa472 bytes) 2024-09-21T21:40:54.231 INFO:teuthology.orchestra.run.smithi164.stdout:870 read 0x33b4a00 thru 0x33b8eaf (0x44b0 bytes) 2024-09-21T21:40:54.231 INFO:teuthology.orchestra.run.smithi164.stdout:875 punch from 0x26c4000 to 0x26c8c00, (0x4c00 bytes) 2024-09-21T21:40:54.231 INFO:teuthology.orchestra.run.smithi164.stdout:879 read 0x1449400 thru 0x145665c (0xd25d bytes) 2024-09-21T21:40:54.250 INFO:teuthology.orchestra.run.smithi164.stdout:881 write 0x74cf600 thru 0x74d9a19 (0xa41a bytes) 2024-09-21T21:40:54.250 INFO:teuthology.orchestra.run.smithi164.stdout:883 read 0x3505c00 thru 0x35095d8 (0x39d9 bytes) 2024-09-21T21:40:54.251 INFO:teuthology.orchestra.run.smithi164.stdout:884 punch from 0xf6b800 to 0xf6e400, (0x2c00 bytes) 2024-09-21T21:40:54.251 INFO:teuthology.orchestra.run.smithi164.stdout:885 punch from 0x54a4000 to 0x54b3800, (0xf800 bytes) 2024-09-21T21:40:54.251 INFO:teuthology.orchestra.run.smithi164.stdout:888 read 0x5aedc00 thru 0x5afd133 (0xf534 bytes) 2024-09-21T21:40:54.253 INFO:teuthology.orchestra.run.smithi164.stdout:889 write 0x6984800 thru 0x698a0bf (0x58c0 bytes) 2024-09-21T21:40:54.253 INFO:teuthology.orchestra.run.smithi164.stdout:891 read 0x25ac000 thru 0x25b8c72 (0xcc73 bytes) 2024-09-21T21:40:54.253 INFO:teuthology.orchestra.run.smithi164.stdout:892 read 0x1650e00 thru 0x165b040 (0xa241 bytes) 2024-09-21T21:40:54.254 INFO:teuthology.orchestra.run.smithi164.stdout:893 punch from 0x6e34400 to 0x6e39600, (0x5200 bytes) 2024-09-21T21:40:54.275 INFO:teuthology.orchestra.run.smithi164.stdout:894 write 0xc77f800 thru 0xc785c99 (0x649a bytes) 2024-09-21T21:40:54.284 INFO:teuthology.orchestra.run.smithi164.stdout:896 trunc from 0xc785e00 to 0x596b600 2024-09-21T21:40:54.315 INFO:teuthology.orchestra.run.smithi164.stdout:899 read 0x57fbc00 thru 0x58082c6 (0xc6c7 bytes) 2024-09-21T21:40:54.315 INFO:teuthology.orchestra.run.smithi164.stdout:900 flatten 2024-09-21T21:40:54.317 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:40:54.316+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:40:54.326 INFO:teuthology.orchestra.run.smithi164.stdout:901 read 0x176fe00 thru 0x177ed3f (0xef40 bytes) 2024-09-21T21:40:54.327 INFO:teuthology.orchestra.run.smithi164.stdout:902 read 0x3ab8c00 thru 0x3ac2e0f (0xa210 bytes) 2024-09-21T21:40:54.328 INFO:teuthology.orchestra.run.smithi164.stdout:903 trunc from 0x596b600 to 0x49b6000 2024-09-21T21:40:54.344 INFO:teuthology.orchestra.run.smithi164.stdout:904 read 0x48a0400 thru 0x48aa6b6 (0xa2b7 bytes) 2024-09-21T21:40:54.345 INFO:teuthology.orchestra.run.smithi164.stdout:907 write 0x39ee200 thru 0x39f499e (0x679f bytes) 2024-09-21T21:40:54.345 INFO:teuthology.orchestra.run.smithi164.stdout:909 read 0x1795200 thru 0x17a12a9 (0xc0aa bytes) 2024-09-21T21:40:54.345 INFO:teuthology.orchestra.run.smithi164.stdout:912 read 0x3d34e00 thru 0x3d3cd23 (0x7f24 bytes) 2024-09-21T21:40:54.346 INFO:teuthology.orchestra.run.smithi164.stdout:913 read 0x1405e00 thru 0x1414bd9 (0xedda bytes) 2024-09-21T21:40:54.346 INFO:teuthology.orchestra.run.smithi164.stdout:915 read 0x34e800 thru 0x3541c5 (0x59c6 bytes) 2024-09-21T21:40:54.347 INFO:teuthology.orchestra.run.smithi164.stdout:916 write 0x5ec400 thru 0x5f7fb4 (0xbbb5 bytes) 2024-09-21T21:40:54.351 INFO:teuthology.orchestra.run.smithi164.stdout:917 trunc from 0x49b6000 to 0x72b2400 2024-09-21T21:40:54.376 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:54.377 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:54.386 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T21:40:54.387 INFO:teuthology.orchestra.run.smithi164.stdout:923 write 0x8e70200 thru 0x8e770cf (0x6ed0 bytes) 2024-09-21T21:40:54.387 INFO:teuthology.orchestra.run.smithi164.stdout:924 read 0x1f0ce00 thru 0x1f151e1 (0x83e2 bytes) 2024-09-21T21:40:54.387 INFO:teuthology.orchestra.run.smithi164.stdout:926 read 0x73dd800 thru 0x73e149d (0x3c9e bytes) 2024-09-21T21:40:54.391 INFO:teuthology.orchestra.run.smithi164.stdout:927 trunc from 0x8e77200 to 0xd71b800 2024-09-21T21:40:54.419 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:40:54.417+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:40:54.419 INFO:teuthology.orchestra.run.smithi164.stdout:928 trunc from 0xd71b800 to 0x4b73400 2024-09-21T21:40:54.434 INFO:teuthology.orchestra.run.smithi164.stdout:929 write 0xd27200 thru 0xd2b01f (0x3e20 bytes) 2024-09-21T21:40:54.439 INFO:teuthology.orchestra.run.smithi164.stdout:930 read 0x34da600 thru 0x34dd434 (0x2e35 bytes) 2024-09-21T21:40:54.458 INFO:teuthology.orchestra.run.smithi164.stdout:933 write 0xbdeba00 thru 0xbdec1a7 (0x7a8 bytes) 2024-09-21T21:40:54.458 INFO:teuthology.orchestra.run.smithi164.stdout:934 write 0x2235c00 thru 0x223ea50 (0x8e51 bytes) 2024-09-21T21:40:54.463 INFO:teuthology.orchestra.run.smithi164.stdout:938 trunc from 0xbdec200 to 0x8b12e00 2024-09-21T21:40:54.479 INFO:teuthology.orchestra.run.smithi164.stdout:939 write 0x2585a00 thru 0x2594e68 (0xf469 bytes) 2024-09-21T21:40:54.480 INFO:teuthology.orchestra.run.smithi164.stdout:940 write 0x6d3fe00 thru 0x6d4c604 (0xc805 bytes) 2024-09-21T21:40:54.481 INFO:teuthology.orchestra.run.smithi164.stdout:942 punch from 0x2084000 to 0x2093e00, (0xfe00 bytes) 2024-09-21T21:40:54.481 INFO:teuthology.orchestra.run.smithi164.stdout:943 punch from 0x51ba800 to 0x51c1800, (0x7000 bytes) 2024-09-21T21:40:54.482 INFO:teuthology.orchestra.run.smithi164.stdout:944 trunc from 0x8b12e00 to 0x961b800 2024-09-21T21:40:54.493 INFO:teuthology.orchestra.run.smithi164.stdout:945 punch from 0x4522600 to 0x452d200, (0xac00 bytes) 2024-09-21T21:40:54.500 INFO:teuthology.orchestra.run.smithi081.stderr:noscrub is unset 2024-09-21T21:40:54.501 INFO:teuthology.orchestra.run.smithi164.stdout:947 write 0x569a00 thru 0x56f6c6 (0x5cc7 bytes) 2024-09-21T21:40:54.502 INFO:teuthology.orchestra.run.smithi164.stdout:949 write 0x9182000 thru 0x91849f3 (0x29f4 bytes) 2024-09-21T21:40:54.502 INFO:teuthology.orchestra.run.smithi164.stdout:950 read 0x246ca00 thru 0x246f643 (0x2c44 bytes) 2024-09-21T21:40:54.503 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:54.503 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:54.506 INFO:teuthology.orchestra.run.smithi164.stdout:952 trunc from 0x961b800 to 0x660aa00 2024-09-21T21:40:54.514 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T21:40:54.551 INFO:teuthology.orchestra.run.smithi164.stdout:953 write 0xc0c4600 thru 0xc0cc918 (0x8319 bytes) 2024-09-21T21:40:54.551 INFO:teuthology.orchestra.run.smithi164.stdout:954 write 0x7e7f200 thru 0x7e8c8aa (0xd6ab bytes) 2024-09-21T21:40:54.551 INFO:teuthology.orchestra.run.smithi164.stdout:957 read 0xaa57e00 thru 0xaa5b286 (0x3487 bytes) 2024-09-21T21:40:54.556 INFO:teuthology.orchestra.run.smithi164.stdout:958 trunc from 0xc0cca00 to 0x87f7e00 2024-09-21T21:40:54.556 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:40:54.555+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:40:54.583 INFO:teuthology.orchestra.run.smithi164.stdout:959 trunc from 0x87f7e00 to 0x78d7e00 2024-09-21T21:40:54.593 INFO:teuthology.orchestra.run.smithi164.stdout:961 clone 7 order 25 su 131072 sc 13 2024-09-21T21:40:54.620 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:40:54.618+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:40:54.629 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:54.629 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:54.639 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T21:40:54.722 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:40:54.721+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:40:54.781 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:54.781 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:54.790 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T21:40:54.832 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:40:54.830+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:40:54.923 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:40:54.922+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:40:54.932 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:54.933 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:54.941 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T21:40:55.023 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:40:55.022+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:40:55.044 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:55.044 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:55.053 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T21:40:55.125 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:40:55.123+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:40:55.159 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:55.160 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:55.168 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T21:40:55.226 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:40:55.224+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:40:55.271 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:55.271 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:55.280 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T21:40:55.327 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:40:55.326+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:40:55.422 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:55.422 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:55.428 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:40:55.426+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:40:55.431 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T21:40:55.528 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:40:55.527+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:40:55.577 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:55.577 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:55.585 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T21:40:55.629 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:40:55.627+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:40:55.729 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:55.729 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:55.730 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:40:55.728+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:40:55.738 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T21:40:55.830 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:40:55.829+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:40:55.884 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:55.884 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:55.893 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T21:40:55.931 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:40:55.930+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:40:55.946 INFO:teuthology.orchestra.run.smithi164.stdout:checking clone #5, image image_client.0-clone5 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent6 2024-09-21T21:40:55.998 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:55.998 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:56.008 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T21:40:56.051 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:40:56.050+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:40:56.125 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:56.126 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:56.133 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:40:56.131+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:40:56.135 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T21:40:56.225 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T21:40:56.268 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:40:56.267+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:40:56.292 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:56.292 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:56.301 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T21:40:56.346 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:40:56.345+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:40:56.351 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0: {} 2024-09-21T21:40:56.351 INFO:teuthology.orchestra.run.smithi081.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T21:40:56.368 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1: {} 2024-09-21T21:40:56.368 INFO:teuthology.orchestra.run.smithi081.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T21:40:56.383 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2: {} 2024-09-21T21:40:56.384 INFO:teuthology.orchestra.run.smithi081.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T21:40:56.399 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3: {} 2024-09-21T21:40:56.400 INFO:teuthology.orchestra.run.smithi081.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T21:40:56.405 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:40:56.405 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:40:56.405 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-21T21:40:56.406 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:40:56.414 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T21:40:56.415 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4: {} 2024-09-21T21:40:56.416 INFO:teuthology.orchestra.run.smithi081.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T21:40:56.433 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5: {} 2024-09-21T21:40:56.433 INFO:teuthology.orchestra.run.smithi081.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T21:40:56.439 INFO:teuthology.orchestra.run.smithi164.stdout:963 read 0x2f8f800 thru 0x2f994b6 (0x9cb7 bytes) 2024-09-21T21:40:56.441 INFO:teuthology.orchestra.run.smithi164.stdout:965 read 0x3947800 thru 0x394aef9 (0x36fa bytes) 2024-09-21T21:40:56.442 INFO:teuthology.orchestra.run.smithi164.stdout:966 write 0x1202000 thru 0x12097fa (0x77fb bytes) 2024-09-21T21:40:56.455 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:40:56.454+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:40:56.474 INFO:teuthology.orchestra.run.smithi164.stdout:968 write 0xec6f600 thru 0xec7edaf (0xf7b0 bytes) 2024-09-21T21:40:56.475 INFO:teuthology.orchestra.run.smithi164.stdout:969 read 0xb6ca000 thru 0xb6d8d61 (0xed62 bytes) 2024-09-21T21:40:56.486 INFO:teuthology.orchestra.run.smithi164.stdout:970 trunc from 0xec7ee00 to 0x59e4200 2024-09-21T21:40:56.518 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T21:40:56.536 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:40:56.534+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:40:56.584 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:40:56.584 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:40:56.584 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:40:56.584 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:40:56.584 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:40:56.588 INFO:teuthology.orchestra.run.smithi164.stdout:971 punch from 0x4c82e00 to 0x4c8d800, (0xaa00 bytes) 2024-09-21T21:40:56.593 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T21:40:56.613 INFO:teuthology.orchestra.run.smithi164.stdout:972 write 0xd1f8600 thru 0xd204877 (0xc278 bytes) 2024-09-21T21:40:56.614 INFO:teuthology.orchestra.run.smithi164.stdout:974 write 0xc4c8a00 thru 0xc4cf4d8 (0x6ad9 bytes) 2024-09-21T21:40:56.614 INFO:teuthology.orchestra.run.smithi164.stdout:976 punch from 0x5ee6800 to 0x5eefc00, (0x9400 bytes) 2024-09-21T21:40:56.630 INFO:teuthology.orchestra.run.smithi164.stdout:978 trunc from 0xd204a00 to 0x258a00 2024-09-21T21:40:56.677 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:40:56.676+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:40:56.737 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-21T21:40:56.737 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T21:40:56.737 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-21T21:40:56.737 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-21T21:40:56.738 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:40:56.736+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:40:56.740 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:40:56.740 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-21T21:40:56.740 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-21T21:40:56.740 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:40:56.740 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.741 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4183.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T21:40:56.741 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:19.643416+0000", 2024-09-21T21:40:56.741 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 37.094561827, 2024-09-21T21:40:56.741 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0076828269999999997, 2024-09-21T21:40:56.741 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:56.741 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:56.741 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:56.741 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:56.741 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4183", 2024-09-21T21:40:56.741 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.122:0/3913072034", 2024-09-21T21:40:56.741 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 115 2024-09-21T21:40:56.741 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.741 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:56.741 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.741 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:56.742 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643416+0000", 2024-09-21T21:40:56.742 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:56.742 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.742 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.742 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:56.742 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643416+0000", 2024-09-21T21:40:56.742 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:56.742 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.742 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.742 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:56.742 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643417+0000", 2024-09-21T21:40:56.742 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.96e-07 2024-09-21T21:40:56.742 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.742 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.742 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:56.743 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643469+0000", 2024-09-21T21:40:56.743 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2293999999999998e-05 2024-09-21T21:40:56.743 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.743 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.743 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:56.743 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643474+0000", 2024-09-21T21:40:56.743 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3179999999999997e-06 2024-09-21T21:40:56.743 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.743 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.743 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:56.743 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643488+0000", 2024-09-21T21:40:56.743 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3241000000000001e-05 2024-09-21T21:40:56.743 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.743 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.744 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:56.744 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.649349+0000", 2024-09-21T21:40:56.744 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0058617690000000002 2024-09-21T21:40:56.744 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.744 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.744 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:56.744 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.649440+0000", 2024-09-21T21:40:56.744 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.1092000000000001e-05 2024-09-21T21:40:56.744 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.744 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.744 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:40:56.744 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.649583+0000", 2024-09-21T21:40:56.744 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014218499999999999 2024-09-21T21:40:56.744 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.745 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.745 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:56.745 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.650978+0000", 2024-09-21T21:40:56.745 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0013956210000000001 2024-09-21T21:40:56.745 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.745 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.745 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:56.745 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.651032+0000", 2024-09-21T21:40:56.745 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3513000000000001e-05 2024-09-21T21:40:56.745 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.745 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.745 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:56.745 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.651068+0000", 2024-09-21T21:40:56.745 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6675999999999999e-05 2024-09-21T21:40:56.745 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.746 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.746 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:56.746 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.651099+0000", 2024-09-21T21:40:56.746 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0821999999999997e-05 2024-09-21T21:40:56.746 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:56.746 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:56.746 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:56.746 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.746 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.746 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4183.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T21:40:56.746 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:19.643496+0000", 2024-09-21T21:40:56.746 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 37.094482282000001, 2024-09-21T21:40:56.746 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0077031249999999999, 2024-09-21T21:40:56.746 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:56.747 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:56.747 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:56.747 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:56.747 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4183", 2024-09-21T21:40:56.747 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.122:0/3913072034", 2024-09-21T21:40:56.747 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 116 2024-09-21T21:40:56.747 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.747 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:56.747 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.747 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:56.747 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643496+0000", 2024-09-21T21:40:56.747 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:56.747 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.747 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.747 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:56.748 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643496+0000", 2024-09-21T21:40:56.748 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:56.748 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.748 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.748 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:56.748 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643496+0000", 2024-09-21T21:40:56.748 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3599999999999999e-07 2024-09-21T21:40:56.748 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.748 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.748 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:56.748 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643500+0000", 2024-09-21T21:40:56.748 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1790000000000001e-06 2024-09-21T21:40:56.748 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.749 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.749 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:56.749 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643504+0000", 2024-09-21T21:40:56.749 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8299999999999998e-06 2024-09-21T21:40:56.749 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.749 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.749 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:56.749 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643507+0000", 2024-09-21T21:40:56.749 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.056e-06 2024-09-21T21:40:56.749 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.749 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.749 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:56.749 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.650003+0000", 2024-09-21T21:40:56.750 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0064959129999999999 2024-09-21T21:40:56.750 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.750 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.750 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:56.750 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.650051+0000", 2024-09-21T21:40:56.750 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8180999999999998e-05 2024-09-21T21:40:56.750 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.750 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.750 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:40:56.750 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.650127+0000", 2024-09-21T21:40:56.750 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5795000000000002e-05 2024-09-21T21:40:56.750 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.750 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.750 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:56.751 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.651137+0000", 2024-09-21T21:40:56.751 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0010094699999999999 2024-09-21T21:40:56.751 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.751 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.751 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:56.751 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.651144+0000", 2024-09-21T21:40:56.751 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.985e-06 2024-09-21T21:40:56.751 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.751 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.751 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:56.751 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.651169+0000", 2024-09-21T21:40:56.751 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.563e-05 2024-09-21T21:40:56.751 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.751 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.752 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:56.752 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.651199+0000", 2024-09-21T21:40:56.752 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9750000000000001e-05 2024-09-21T21:40:56.752 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:56.752 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:56.752 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:56.752 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.752 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.752 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4183.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T21:40:56.752 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:19.643512+0000", 2024-09-21T21:40:56.752 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 37.094466668000003, 2024-09-21T21:40:56.752 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0077806539999999997, 2024-09-21T21:40:56.752 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:56.752 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:56.753 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:56.753 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:56.753 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4183", 2024-09-21T21:40:56.753 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.122:0/3913072034", 2024-09-21T21:40:56.753 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 117 2024-09-21T21:40:56.753 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.753 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:56.753 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.753 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:56.753 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643512+0000", 2024-09-21T21:40:56.753 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:56.753 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.753 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.753 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:56.754 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643512+0000", 2024-09-21T21:40:56.754 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:56.754 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.754 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.754 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:56.754 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643512+0000", 2024-09-21T21:40:56.754 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.16e-07 2024-09-21T21:40:56.754 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.754 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.754 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:56.754 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643514+0000", 2024-09-21T21:40:56.754 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3709999999999998e-06 2024-09-21T21:40:56.754 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.755 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.755 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:56.755 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643517+0000", 2024-09-21T21:40:56.755 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2310000000000001e-06 2024-09-21T21:40:56.755 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.755 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.755 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:56.755 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.643522+0000", 2024-09-21T21:40:56.755 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8319999999999996e-06 2024-09-21T21:40:56.755 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.755 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.755 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:56.755 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.650426+0000", 2024-09-21T21:40:56.755 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0069033380000000002 2024-09-21T21:40:56.755 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.756 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.756 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:56.756 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.650482+0000", 2024-09-21T21:40:56.756 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6468999999999998e-05 2024-09-21T21:40:56.756 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.756 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.756 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:40:56.756 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.650565+0000", 2024-09-21T21:40:56.756 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3026000000000001e-05 2024-09-21T21:40:56.756 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.756 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.756 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:56.756 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.651209+0000", 2024-09-21T21:40:56.756 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00064387199999999998 2024-09-21T21:40:56.757 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.757 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.757 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:56.757 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.651238+0000", 2024-09-21T21:40:56.757 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8799e-05 2024-09-21T21:40:56.757 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.757 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.757 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:56.757 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.651270+0000", 2024-09-21T21:40:56.757 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1934000000000003e-05 2024-09-21T21:40:56.757 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.757 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.757 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:56.757 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.651292+0000", 2024-09-21T21:40:56.757 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2566e-05 2024-09-21T21:40:56.758 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:56.758 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:56.758 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:56.758 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.758 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.758 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4183.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T21:40:56.758 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:19.916463+0000", 2024-09-21T21:40:56.758 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 36.821515185000003, 2024-09-21T21:40:56.758 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0094252929999999995, 2024-09-21T21:40:56.758 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:56.758 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:56.758 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:56.758 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:56.759 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4183", 2024-09-21T21:40:56.759 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.122:0/3913072034", 2024-09-21T21:40:56.759 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 134 2024-09-21T21:40:56.759 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.759 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:56.759 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.759 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:56.759 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916463+0000", 2024-09-21T21:40:56.759 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:56.759 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.759 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.759 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:56.759 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916463+0000", 2024-09-21T21:40:56.759 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:56.759 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.760 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.760 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:56.760 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916464+0000", 2024-09-21T21:40:56.760 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2900000000000004e-07 2024-09-21T21:40:56.760 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.760 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.760 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:56.760 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916525+0000", 2024-09-21T21:40:56.760 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1576999999999997e-05 2024-09-21T21:40:56.760 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.760 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.760 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:56.760 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916533+0000", 2024-09-21T21:40:56.760 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7510000000000008e-06 2024-09-21T21:40:56.760 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.761 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.761 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:56.761 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916537+0000", 2024-09-21T21:40:56.761 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.155e-06 2024-09-21T21:40:56.761 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.761 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.761 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:56.761 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.919134+0000", 2024-09-21T21:40:56.761 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025973469999999999 2024-09-21T21:40:56.761 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.761 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.761 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:56.761 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.919189+0000", 2024-09-21T21:40:56.761 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4339999999999998e-05 2024-09-21T21:40:56.761 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.762 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.762 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:40:56.762 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.919261+0000", 2024-09-21T21:40:56.762 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1977000000000006e-05 2024-09-21T21:40:56.762 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.762 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.762 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:56.762 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.920953+0000", 2024-09-21T21:40:56.762 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0016926109999999999 2024-09-21T21:40:56.762 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.762 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.762 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:56.762 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.925825+0000", 2024-09-21T21:40:56.762 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048719159999999996 2024-09-21T21:40:56.763 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.763 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.763 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:56.763 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.925854+0000", 2024-09-21T21:40:56.763 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8524999999999999e-05 2024-09-21T21:40:56.763 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.763 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.763 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:56.763 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.925888+0000", 2024-09-21T21:40:56.763 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4564999999999999e-05 2024-09-21T21:40:56.763 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:56.763 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:56.763 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:56.763 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.763 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.764 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4183.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T21:40:56.764 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:19.916545+0000", 2024-09-21T21:40:56.764 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 36.821433268, 2024-09-21T21:40:56.764 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0094280869999999999, 2024-09-21T21:40:56.764 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:56.764 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:56.764 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:56.764 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:56.764 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4183", 2024-09-21T21:40:56.764 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.122:0/3913072034", 2024-09-21T21:40:56.764 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 135 2024-09-21T21:40:56.764 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.764 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:56.764 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.764 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:56.765 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916545+0000", 2024-09-21T21:40:56.765 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:56.765 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.765 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.765 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:56.765 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916545+0000", 2024-09-21T21:40:56.765 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:56.765 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.765 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.765 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:56.765 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916545+0000", 2024-09-21T21:40:56.765 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1100000000000002e-07 2024-09-21T21:40:56.765 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.765 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.765 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:56.766 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916549+0000", 2024-09-21T21:40:56.766 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0799999999999999e-06 2024-09-21T21:40:56.766 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.766 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.766 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:56.766 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916556+0000", 2024-09-21T21:40:56.766 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0339999999999999e-06 2024-09-21T21:40:56.766 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.766 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.766 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:56.766 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916562+0000", 2024-09-21T21:40:56.766 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5439999999999998e-06 2024-09-21T21:40:56.766 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.766 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.767 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:56.767 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.919649+0000", 2024-09-21T21:40:56.767 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0030868660000000002 2024-09-21T21:40:56.767 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.767 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.767 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:56.767 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.919693+0000", 2024-09-21T21:40:56.767 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4694000000000003e-05 2024-09-21T21:40:56.767 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.767 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.767 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:40:56.767 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.919765+0000", 2024-09-21T21:40:56.767 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1427999999999994e-05 2024-09-21T21:40:56.767 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.768 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.768 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:56.768 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.921005+0000", 2024-09-21T21:40:56.768 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001240314 2024-09-21T21:40:56.768 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.768 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.768 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:56.768 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.925897+0000", 2024-09-21T21:40:56.768 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048916660000000002 2024-09-21T21:40:56.768 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.768 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.768 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:56.768 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.925942+0000", 2024-09-21T21:40:56.768 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4721000000000001e-05 2024-09-21T21:40:56.769 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.769 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.769 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:56.769 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.925973+0000", 2024-09-21T21:40:56.769 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1429e-05 2024-09-21T21:40:56.769 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:56.769 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:56.769 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:56.769 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.769 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.769 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4183.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T21:40:56.769 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:19.916568+0000", 2024-09-21T21:40:56.769 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 36.821410333999999, 2024-09-21T21:40:56.769 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0094755120000000002, 2024-09-21T21:40:56.769 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:56.770 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:56.770 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:56.770 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:56.770 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4183", 2024-09-21T21:40:56.770 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.122:0/3913072034", 2024-09-21T21:40:56.770 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 136 2024-09-21T21:40:56.770 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.770 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:56.770 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.770 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:56.770 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916568+0000", 2024-09-21T21:40:56.770 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:56.770 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.770 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.771 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:56.771 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916568+0000", 2024-09-21T21:40:56.771 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:56.771 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.771 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.771 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:56.771 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916568+0000", 2024-09-21T21:40:56.771 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9200000000000002e-07 2024-09-21T21:40:56.771 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.771 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.771 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:56.771 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916572+0000", 2024-09-21T21:40:56.771 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.428e-06 2024-09-21T21:40:56.771 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.771 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.772 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:56.772 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916576+0000", 2024-09-21T21:40:56.772 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.814e-06 2024-09-21T21:40:56.772 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.772 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.772 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:56.772 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916580+0000", 2024-09-21T21:40:56.772 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6669999999999998e-06 2024-09-21T21:40:56.772 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.772 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.772 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:56.772 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.920045+0000", 2024-09-21T21:40:56.772 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0034645209999999999 2024-09-21T21:40:56.772 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.772 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.773 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:56.773 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.920091+0000", 2024-09-21T21:40:56.773 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6146999999999997e-05 2024-09-21T21:40:56.773 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.773 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.773 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:40:56.773 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.920159+0000", 2024-09-21T21:40:56.773 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8411000000000006e-05 2024-09-21T21:40:56.773 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.773 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.773 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:56.773 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.921037+0000", 2024-09-21T21:40:56.773 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00087794700000000002 2024-09-21T21:40:56.773 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.774 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.774 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:56.774 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.925981+0000", 2024-09-21T21:40:56.774 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0049443259999999998 2024-09-21T21:40:56.774 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.774 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.774 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:56.774 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.926014+0000", 2024-09-21T21:40:56.774 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2298999999999997e-05 2024-09-21T21:40:56.774 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.774 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.774 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:56.774 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.926043+0000", 2024-09-21T21:40:56.774 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9660000000000001e-05 2024-09-21T21:40:56.775 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:56.775 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:56.775 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:56.775 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.775 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.775 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4183.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T21:40:56.775 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:19.916587+0000", 2024-09-21T21:40:56.775 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 36.821390854000001, 2024-09-21T21:40:56.775 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.009527265, 2024-09-21T21:40:56.775 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:56.775 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:56.775 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:56.775 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:56.775 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4183", 2024-09-21T21:40:56.775 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.122:0/3913072034", 2024-09-21T21:40:56.776 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 137 2024-09-21T21:40:56.776 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.776 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:56.776 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.776 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:56.776 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916587+0000", 2024-09-21T21:40:56.776 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:56.776 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.776 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.776 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:56.776 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916587+0000", 2024-09-21T21:40:56.776 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:56.776 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.776 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.777 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:56.777 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916588+0000", 2024-09-21T21:40:56.777 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5699999999999999e-07 2024-09-21T21:40:56.777 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.777 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.777 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:56.777 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916644+0000", 2024-09-21T21:40:56.777 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6344000000000001e-05 2024-09-21T21:40:56.777 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.777 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.777 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:56.777 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916649+0000", 2024-09-21T21:40:56.777 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0409999999999997e-06 2024-09-21T21:40:56.777 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.777 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.778 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:56.778 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916654+0000", 2024-09-21T21:40:56.778 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9919999999999998e-06 2024-09-21T21:40:56.778 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.778 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.778 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:56.778 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.920443+0000", 2024-09-21T21:40:56.778 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0037892469999999999 2024-09-21T21:40:56.778 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.778 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.778 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:56.778 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.920493+0000", 2024-09-21T21:40:56.778 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9898000000000002e-05 2024-09-21T21:40:56.778 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.779 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.779 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:40:56.779 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.920561+0000", 2024-09-21T21:40:56.779 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7481e-05 2024-09-21T21:40:56.779 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.779 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.779 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:56.779 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.925739+0000", 2024-09-21T21:40:56.779 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.005178729 2024-09-21T21:40:56.779 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.779 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.779 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:56.779 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.926052+0000", 2024-09-21T21:40:56.779 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00031252800000000001 2024-09-21T21:40:56.779 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.780 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.780 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:56.780 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.926087+0000", 2024-09-21T21:40:56.780 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4873999999999998e-05 2024-09-21T21:40:56.780 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.780 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.780 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:56.780 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.926115+0000", 2024-09-21T21:40:56.780 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7874000000000001e-05 2024-09-21T21:40:56.780 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:56.780 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:56.780 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:56.780 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.780 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.780 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4183.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T21:40:56.781 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:19.916666+0000", 2024-09-21T21:40:56.781 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 36.821311983000001, 2024-09-21T21:40:56.781 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0095150730000000006, 2024-09-21T21:40:56.781 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:56.781 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:56.781 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:56.781 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:56.781 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4183", 2024-09-21T21:40:56.781 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.122:0/3913072034", 2024-09-21T21:40:56.781 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 138 2024-09-21T21:40:56.781 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.781 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:56.781 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.781 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:56.782 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916666+0000", 2024-09-21T21:40:56.782 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:56.782 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.782 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.782 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:56.782 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916666+0000", 2024-09-21T21:40:56.782 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:56.782 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.782 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.782 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:56.782 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916667+0000", 2024-09-21T21:40:56.782 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3200000000000005e-07 2024-09-21T21:40:56.782 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.782 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.783 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:56.783 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916670+0000", 2024-09-21T21:40:56.783 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.427e-06 2024-09-21T21:40:56.783 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.783 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.783 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:56.783 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916684+0000", 2024-09-21T21:40:56.783 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3839000000000001e-05 2024-09-21T21:40:56.783 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.783 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.783 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:56.783 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916706+0000", 2024-09-21T21:40:56.783 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1543e-05 2024-09-21T21:40:56.783 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.784 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.784 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:56.784 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.921064+0000", 2024-09-21T21:40:56.784 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0043587119999999998 2024-09-21T21:40:56.784 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.784 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.784 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:56.784 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.921119+0000", 2024-09-21T21:40:56.784 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4619e-05 2024-09-21T21:40:56.784 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.784 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.784 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:40:56.784 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.921192+0000", 2024-09-21T21:40:56.784 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3224000000000002e-05 2024-09-21T21:40:56.785 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.785 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.785 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:56.785 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.925791+0000", 2024-09-21T21:40:56.785 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0045988670000000004 2024-09-21T21:40:56.785 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.785 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.785 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:56.785 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.926123+0000", 2024-09-21T21:40:56.785 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00033195000000000002 2024-09-21T21:40:56.785 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.785 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.785 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:56.785 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.926154+0000", 2024-09-21T21:40:56.785 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0720999999999999e-05 2024-09-21T21:40:56.786 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.786 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.786 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:56.786 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.926181+0000", 2024-09-21T21:40:56.786 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7639000000000001e-05 2024-09-21T21:40:56.786 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:56.786 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:56.786 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:56.786 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.786 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.786 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4183.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T21:40:56.786 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:19.916714+0000", 2024-09-21T21:40:56.786 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 36.821264094, 2024-09-21T21:40:56.786 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0095298369999999993, 2024-09-21T21:40:56.787 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:56.787 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:56.787 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:56.787 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:56.787 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4183", 2024-09-21T21:40:56.787 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.122:0/3913072034", 2024-09-21T21:40:56.787 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 139 2024-09-21T21:40:56.787 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.787 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:56.787 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.787 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:56.787 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916714+0000", 2024-09-21T21:40:56.787 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:56.787 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.788 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.788 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:56.788 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916714+0000", 2024-09-21T21:40:56.788 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:56.788 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.788 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.788 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:56.788 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916714+0000", 2024-09-21T21:40:56.788 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9799999999999999e-07 2024-09-21T21:40:56.788 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.788 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.788 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:56.788 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916718+0000", 2024-09-21T21:40:56.788 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1810000000000001e-06 2024-09-21T21:40:56.789 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.789 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.789 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:56.789 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916724+0000", 2024-09-21T21:40:56.789 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7329999999999997e-06 2024-09-21T21:40:56.789 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.789 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.789 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:56.789 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916728+0000", 2024-09-21T21:40:56.789 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2909999999999999e-06 2024-09-21T21:40:56.789 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.789 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.789 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:56.789 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.921416+0000", 2024-09-21T21:40:56.790 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0046882850000000004 2024-09-21T21:40:56.790 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.790 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.790 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:56.790 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.921461+0000", 2024-09-21T21:40:56.790 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5259000000000002e-05 2024-09-21T21:40:56.790 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.790 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.790 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:40:56.790 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.921540+0000", 2024-09-21T21:40:56.790 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8765999999999998e-05 2024-09-21T21:40:56.790 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.790 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.790 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:56.791 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.925818+0000", 2024-09-21T21:40:56.791 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0042784800000000003 2024-09-21T21:40:56.791 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.791 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.791 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:56.791 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.926190+0000", 2024-09-21T21:40:56.791 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00037112400000000002 2024-09-21T21:40:56.791 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.791 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.791 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:56.791 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.926225+0000", 2024-09-21T21:40:56.791 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5420999999999998e-05 2024-09-21T21:40:56.791 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.791 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.792 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:56.792 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.926244+0000", 2024-09-21T21:40:56.792 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.8998999999999999e-05 2024-09-21T21:40:56.792 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:56.792 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:56.792 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:56.792 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.792 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.792 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4183.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T21:40:56.792 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:19.916735+0000", 2024-09-21T21:40:56.792 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 36.821243697, 2024-09-21T21:40:56.792 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.010302162, 2024-09-21T21:40:56.792 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:56.792 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:56.793 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:56.793 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:56.793 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4183", 2024-09-21T21:40:56.793 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.122:0/3913072034", 2024-09-21T21:40:56.793 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 140 2024-09-21T21:40:56.793 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.793 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:56.793 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.793 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:56.793 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916735+0000", 2024-09-21T21:40:56.793 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:56.793 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.793 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.794 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:56.794 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916735+0000", 2024-09-21T21:40:56.794 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:56.794 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.794 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.794 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:56.794 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916735+0000", 2024-09-21T21:40:56.794 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5400000000000002e-07 2024-09-21T21:40:56.794 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.794 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.794 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:56.794 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916788+0000", 2024-09-21T21:40:56.794 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3074e-05 2024-09-21T21:40:56.794 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.795 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.795 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:56.795 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916795+0000", 2024-09-21T21:40:56.795 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.204e-06 2024-09-21T21:40:56.795 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.795 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.795 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:56.795 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916801+0000", 2024-09-21T21:40:56.795 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4759999999999997e-06 2024-09-21T21:40:56.795 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.795 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.795 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:56.795 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.921792+0000", 2024-09-21T21:40:56.795 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0049909749999999999 2024-09-21T21:40:56.795 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.796 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.796 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:56.796 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.921879+0000", 2024-09-21T21:40:56.796 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7361999999999995e-05 2024-09-21T21:40:56.796 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.796 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.796 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:40:56.796 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.922000+0000", 2024-09-21T21:40:56.796 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000121451 2024-09-21T21:40:56.796 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.796 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.796 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:56.796 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.926291+0000", 2024-09-21T21:40:56.796 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0042906180000000004 2024-09-21T21:40:56.796 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.797 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.797 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:56.797 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.926984+0000", 2024-09-21T21:40:56.797 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00069308599999999998 2024-09-21T21:40:56.797 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.797 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.797 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:56.797 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.927014+0000", 2024-09-21T21:40:56.797 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0222999999999998e-05 2024-09-21T21:40:56.797 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.797 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.797 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:56.797 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.927037+0000", 2024-09-21T21:40:56.797 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2439e-05 2024-09-21T21:40:56.798 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:56.798 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:56.798 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:56.798 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.798 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.798 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4183.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-21T21:40:56.798 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:19.916814+0000", 2024-09-21T21:40:56.798 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 36.821164508000003, 2024-09-21T21:40:56.798 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.010507598999999999, 2024-09-21T21:40:56.798 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:56.798 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:56.798 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:56.798 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:56.799 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4183", 2024-09-21T21:40:56.799 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.122:0/3913072034", 2024-09-21T21:40:56.799 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 141 2024-09-21T21:40:56.799 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.799 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:56.799 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.799 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:56.799 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916814+0000", 2024-09-21T21:40:56.799 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:56.799 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.799 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.799 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:56.799 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916814+0000", 2024-09-21T21:40:56.799 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:56.800 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.800 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.800 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:56.800 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916814+0000", 2024-09-21T21:40:56.800 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9599999999999999e-07 2024-09-21T21:40:56.800 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.800 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.800 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:56.800 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916819+0000", 2024-09-21T21:40:56.800 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7090000000000001e-06 2024-09-21T21:40:56.800 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.800 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.800 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:56.800 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916830+0000", 2024-09-21T21:40:56.800 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0977e-05 2024-09-21T21:40:56.801 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.801 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.801 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:56.801 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.916835+0000", 2024-09-21T21:40:56.801 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2320000000000001e-06 2024-09-21T21:40:56.801 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.801 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.801 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:56.801 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.925249+0000", 2024-09-21T21:40:56.801 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0084135170000000006 2024-09-21T21:40:56.801 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.801 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.801 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:56.801 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.925323+0000", 2024-09-21T21:40:56.802 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3919999999999997e-05 2024-09-21T21:40:56.802 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.802 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.802 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:40:56.802 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.925442+0000", 2024-09-21T21:40:56.802 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00011944500000000001 2024-09-21T21:40:56.802 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.802 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.802 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:56.802 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.926322+0000", 2024-09-21T21:40:56.802 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00087985500000000003 2024-09-21T21:40:56.802 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.802 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.803 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:56.803 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.927245+0000", 2024-09-21T21:40:56.803 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00092342499999999996 2024-09-21T21:40:56.803 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.803 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.803 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:56.803 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.927283+0000", 2024-09-21T21:40:56.803 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8157000000000001e-05 2024-09-21T21:40:56.803 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.803 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.803 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:56.803 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:19.927321+0000", 2024-09-21T21:40:56.803 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7866e-05 2024-09-21T21:40:56.803 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:56.803 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:56.804 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:56.804 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.804 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.804 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4417.0:496 3.d 3:b065aa9d:::rbd_data.11214be7c411.0000000000000074:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:56.804 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.604764+0000", 2024-09-21T21:40:56.804 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.133213774, 2024-09-21T21:40:56.804 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0080497150000000007, 2024-09-21T21:40:56.804 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:56.804 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:56.804 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:56.804 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:56.804 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4417", 2024-09-21T21:40:56.804 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:56.804 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 496 2024-09-21T21:40:56.805 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.805 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:56.805 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.805 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:56.805 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.604764+0000", 2024-09-21T21:40:56.805 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:56.805 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.805 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.805 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:56.805 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.604764+0000", 2024-09-21T21:40:56.805 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:56.805 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.805 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.805 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:56.805 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.604765+0000", 2024-09-21T21:40:56.806 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6300000000000005e-07 2024-09-21T21:40:56.806 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.806 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.806 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:56.806 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.604768+0000", 2024-09-21T21:40:56.806 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0070000000000002e-06 2024-09-21T21:40:56.806 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.806 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.806 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:56.806 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.604774+0000", 2024-09-21T21:40:56.806 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2689999999999996e-06 2024-09-21T21:40:56.806 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.806 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.807 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:56.807 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.604778+0000", 2024-09-21T21:40:56.807 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.749e-06 2024-09-21T21:40:56.807 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.807 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.807 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:56.807 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.605207+0000", 2024-09-21T21:40:56.807 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00042899099999999999 2024-09-21T21:40:56.807 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.807 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.807 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:56.807 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.605251+0000", 2024-09-21T21:40:56.807 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4258999999999998e-05 2024-09-21T21:40:56.807 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.808 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.808 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 2", 2024-09-21T21:40:56.808 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.605430+0000", 2024-09-21T21:40:56.808 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00017844700000000001 2024-09-21T21:40:56.808 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.808 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.808 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:56.808 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.606879+0000", 2024-09-21T21:40:56.808 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001449129 2024-09-21T21:40:56.808 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.808 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.808 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:56.808 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.612748+0000", 2024-09-21T21:40:56.808 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0058691510000000004 2024-09-21T21:40:56.809 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.809 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.809 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:56.809 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.612785+0000", 2024-09-21T21:40:56.809 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6730999999999998e-05 2024-09-21T21:40:56.809 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.809 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.809 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:56.809 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.612814+0000", 2024-09-21T21:40:56.809 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9419000000000001e-05 2024-09-21T21:40:56.809 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:56.809 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:56.809 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:56.809 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.810 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.810 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4417.0:497 3.d 3:b7efb195:::rbd_data.11214be7c411.0000000000000076:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:56.810 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.604904+0000", 2024-09-21T21:40:56.810 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.133074611, 2024-09-21T21:40:56.810 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0080130610000000001, 2024-09-21T21:40:56.810 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:56.810 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:56.810 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:56.810 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:56.810 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4417", 2024-09-21T21:40:56.810 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:56.810 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 497 2024-09-21T21:40:56.810 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.810 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:56.811 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.811 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:56.811 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.604904+0000", 2024-09-21T21:40:56.811 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:56.811 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.811 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.811 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:56.811 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.604904+0000", 2024-09-21T21:40:56.811 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:56.811 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.811 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.811 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:56.811 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.604904+0000", 2024-09-21T21:40:56.811 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5099999999999999e-07 2024-09-21T21:40:56.812 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.812 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.812 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:56.812 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.604909+0000", 2024-09-21T21:40:56.812 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6959999999999996e-06 2024-09-21T21:40:56.812 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.812 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.812 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:56.812 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.604918+0000", 2024-09-21T21:40:56.812 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7229999999999993e-06 2024-09-21T21:40:56.812 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.812 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.812 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:56.812 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.604924+0000", 2024-09-21T21:40:56.813 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9909999999999999e-06 2024-09-21T21:40:56.813 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.813 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.813 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:56.813 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.605720+0000", 2024-09-21T21:40:56.813 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00079632800000000001 2024-09-21T21:40:56.813 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.813 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.813 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:56.813 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.605800+0000", 2024-09-21T21:40:56.813 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0302999999999993e-05 2024-09-21T21:40:56.813 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.813 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.813 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 2", 2024-09-21T21:40:56.813 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.605960+0000", 2024-09-21T21:40:56.814 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000159211 2024-09-21T21:40:56.814 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.814 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.814 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:56.814 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.606887+0000", 2024-09-21T21:40:56.814 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00092735699999999996 2024-09-21T21:40:56.814 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.814 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.814 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:56.814 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.612849+0000", 2024-09-21T21:40:56.814 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0059615450000000004 2024-09-21T21:40:56.814 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.814 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.814 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:56.815 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.612892+0000", 2024-09-21T21:40:56.815 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3683000000000002e-05 2024-09-21T21:40:56.815 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.815 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.815 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:56.815 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.612917+0000", 2024-09-21T21:40:56.815 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4473000000000001e-05 2024-09-21T21:40:56.815 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:56.815 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:56.815 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:56.815 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.815 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.815 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:1191 3.a 3:50c18da3:::rbd_object_map.1121bf4f5ab3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-21T21:40:56.815 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:48.412351+0000", 2024-09-21T21:40:56.815 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 8.3256269990000007, 2024-09-21T21:40:56.816 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0098993850000000001, 2024-09-21T21:40:56.816 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:56.816 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:56.816 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:56.816 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:56.816 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2024-09-21T21:40:56.816 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1688278210", 2024-09-21T21:40:56.816 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1191 2024-09-21T21:40:56.816 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.816 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:56.816 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.816 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:56.816 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.412351+0000", 2024-09-21T21:40:56.816 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:56.817 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.817 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.817 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:56.817 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.412351+0000", 2024-09-21T21:40:56.817 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:56.817 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.817 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.817 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:56.817 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.412352+0000", 2024-09-21T21:40:56.817 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0020000000000001e-06 2024-09-21T21:40:56.817 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.817 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.817 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:56.817 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.412358+0000", 2024-09-21T21:40:56.818 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5670000000000002e-06 2024-09-21T21:40:56.818 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.818 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.818 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:56.818 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.412368+0000", 2024-09-21T21:40:56.818 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0372e-05 2024-09-21T21:40:56.818 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.818 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.818 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:56.818 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.412376+0000", 2024-09-21T21:40:56.818 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5599999999999996e-06 2024-09-21T21:40:56.818 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.818 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.818 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:56.819 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.415940+0000", 2024-09-21T21:40:56.819 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0035641750000000002 2024-09-21T21:40:56.819 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.819 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.819 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:56.819 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.416028+0000", 2024-09-21T21:40:56.819 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8189000000000006e-05 2024-09-21T21:40:56.819 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.819 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.819 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 4", 2024-09-21T21:40:56.819 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.416277+0000", 2024-09-21T21:40:56.819 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00024857500000000001 2024-09-21T21:40:56.819 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.819 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.820 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:56.820 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.418414+0000", 2024-09-21T21:40:56.820 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002137351 2024-09-21T21:40:56.820 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.820 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.820 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:56.820 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.422109+0000", 2024-09-21T21:40:56.820 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0036947899999999999 2024-09-21T21:40:56.820 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.820 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.820 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:56.820 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.422157+0000", 2024-09-21T21:40:56.820 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7729000000000003e-05 2024-09-21T21:40:56.820 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.821 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.821 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:56.821 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.422251+0000", 2024-09-21T21:40:56.821 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.4074999999999996e-05 2024-09-21T21:40:56.821 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:56.821 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:56.821 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:56.821 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.821 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.821 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:1192 3.a 3:50c18da3:::rbd_object_map.1121bf4f5ab3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-21T21:40:56.821 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:48.413403+0000", 2024-09-21T21:40:56.821 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 8.3245753409999992, 2024-09-21T21:40:56.821 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.008976543, 2024-09-21T21:40:56.821 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:56.822 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:56.822 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:56.822 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:56.822 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2024-09-21T21:40:56.822 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1688278210", 2024-09-21T21:40:56.822 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1192 2024-09-21T21:40:56.822 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.822 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:56.822 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.822 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:56.822 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.413403+0000", 2024-09-21T21:40:56.822 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:56.822 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.822 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.822 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:56.823 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.413403+0000", 2024-09-21T21:40:56.823 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:56.823 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.823 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.823 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:56.823 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.413404+0000", 2024-09-21T21:40:56.823 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.7300000000000004e-07 2024-09-21T21:40:56.823 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.823 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.823 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:56.823 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.413409+0000", 2024-09-21T21:40:56.823 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3349999999999997e-06 2024-09-21T21:40:56.823 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.823 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.823 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:56.824 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.413419+0000", 2024-09-21T21:40:56.824 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.9690000000000006e-06 2024-09-21T21:40:56.824 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.824 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.824 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:56.824 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.413435+0000", 2024-09-21T21:40:56.824 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5811000000000002e-05 2024-09-21T21:40:56.824 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.824 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.824 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:56.824 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.416928+0000", 2024-09-21T21:40:56.824 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0034933730000000001 2024-09-21T21:40:56.824 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.824 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.825 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:56.825 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.416994+0000", 2024-09-21T21:40:56.825 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5372999999999999e-05 2024-09-21T21:40:56.825 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.825 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.825 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 4", 2024-09-21T21:40:56.825 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.417276+0000", 2024-09-21T21:40:56.825 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00028256400000000001 2024-09-21T21:40:56.825 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.825 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.825 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:56.825 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.419348+0000", 2024-09-21T21:40:56.825 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020714610000000001 2024-09-21T21:40:56.825 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.825 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.826 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:56.826 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.422320+0000", 2024-09-21T21:40:56.826 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0029726750000000001 2024-09-21T21:40:56.826 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.826 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.826 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:56.826 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.422357+0000", 2024-09-21T21:40:56.826 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6130000000000001e-05 2024-09-21T21:40:56.826 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.826 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.826 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:56.826 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.422379+0000", 2024-09-21T21:40:56.826 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2878999999999999e-05 2024-09-21T21:40:56.826 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:56.826 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:56.827 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:56.827 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.827 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.827 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:1195 3.a 3:5406e24d:::rbd_data.1121bf4f5ab3.00000000000000a8:head [set-alloc-hint object_size 1048576 write_size 1048576,write 79872~27136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-21T21:40:56.827 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:48.413478+0000", 2024-09-21T21:40:56.827 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 8.3245006779999997, 2024-09-21T21:40:56.827 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.009059246, 2024-09-21T21:40:56.827 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:56.827 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:56.827 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:56.827 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:56.827 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2024-09-21T21:40:56.827 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1688278210", 2024-09-21T21:40:56.827 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1195 2024-09-21T21:40:56.827 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.828 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:56.828 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.828 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:56.828 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.413478+0000", 2024-09-21T21:40:56.828 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:56.828 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.828 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.828 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:56.828 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.413478+0000", 2024-09-21T21:40:56.828 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:56.828 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.828 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.828 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:56.828 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.413478+0000", 2024-09-21T21:40:56.829 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7599999999999997e-07 2024-09-21T21:40:56.829 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.829 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.829 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:56.829 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.413509+0000", 2024-09-21T21:40:56.829 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1018e-05 2024-09-21T21:40:56.829 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.829 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.829 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:56.829 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.413514+0000", 2024-09-21T21:40:56.829 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2789999999999998e-06 2024-09-21T21:40:56.829 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.829 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.829 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:56.829 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.413518+0000", 2024-09-21T21:40:56.830 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5499999999999999e-06 2024-09-21T21:40:56.830 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.830 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.830 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:56.830 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.417605+0000", 2024-09-21T21:40:56.830 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0040869560000000001 2024-09-21T21:40:56.830 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.830 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.830 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:56.830 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.417705+0000", 2024-09-21T21:40:56.830 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000100095 2024-09-21T21:40:56.830 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.830 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.830 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 4", 2024-09-21T21:40:56.831 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.417882+0000", 2024-09-21T21:40:56.831 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000176544 2024-09-21T21:40:56.831 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.831 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.831 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:56.831 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.421871+0000", 2024-09-21T21:40:56.831 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0039898249999999998 2024-09-21T21:40:56.831 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.831 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.831 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:56.831 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.422454+0000", 2024-09-21T21:40:56.831 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00058244000000000004 2024-09-21T21:40:56.831 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.831 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.831 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:56.832 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.422502+0000", 2024-09-21T21:40:56.832 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7895999999999997e-05 2024-09-21T21:40:56.832 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.832 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.832 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:56.832 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.422537+0000", 2024-09-21T21:40:56.832 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5067000000000002e-05 2024-09-21T21:40:56.832 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:56.832 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:56.832 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:56.832 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.832 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.832 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:2729 3.a 3:50c18da3:::rbd_object_map.1121bf4f5ab3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T21:40:56.832 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:48.740436+0000", 2024-09-21T21:40:56.832 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 7.9975417570000005, 2024-09-21T21:40:56.833 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0080237559999999999, 2024-09-21T21:40:56.833 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:56.833 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:56.833 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:56.833 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:56.833 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2024-09-21T21:40:56.833 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1688278210", 2024-09-21T21:40:56.833 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2729 2024-09-21T21:40:56.833 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.833 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:56.833 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.833 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:56.833 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.740436+0000", 2024-09-21T21:40:56.833 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:56.833 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.834 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.834 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:56.834 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.740436+0000", 2024-09-21T21:40:56.834 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:56.834 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.834 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.834 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:56.834 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.740437+0000", 2024-09-21T21:40:56.834 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0100000000000005e-07 2024-09-21T21:40:56.834 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.834 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.834 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:56.834 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.740441+0000", 2024-09-21T21:40:56.834 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1999999999999996e-06 2024-09-21T21:40:56.835 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.835 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.835 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:56.835 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.740450+0000", 2024-09-21T21:40:56.835 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3100000000000001e-06 2024-09-21T21:40:56.835 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.835 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.835 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:56.835 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.740455+0000", 2024-09-21T21:40:56.835 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3290000000000001e-06 2024-09-21T21:40:56.835 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.835 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.835 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:56.835 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.743816+0000", 2024-09-21T21:40:56.835 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0033615289999999998 2024-09-21T21:40:56.836 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.836 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.836 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:56.836 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.743904+0000", 2024-09-21T21:40:56.836 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7959000000000003e-05 2024-09-21T21:40:56.836 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.836 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.836 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 4", 2024-09-21T21:40:56.836 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.744198+0000", 2024-09-21T21:40:56.836 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00029311499999999999 2024-09-21T21:40:56.836 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.836 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.836 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:56.836 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.748327+0000", 2024-09-21T21:40:56.837 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0041299120000000003 2024-09-21T21:40:56.837 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.837 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.837 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:56.837 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.748363+0000", 2024-09-21T21:40:56.837 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5346999999999999e-05 2024-09-21T21:40:56.837 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.837 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.837 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:56.837 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.748417+0000", 2024-09-21T21:40:56.837 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4258999999999997e-05 2024-09-21T21:40:56.837 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.837 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.837 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:56.837 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.748460+0000", 2024-09-21T21:40:56.838 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3195000000000002e-05 2024-09-21T21:40:56.838 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:56.838 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:56.838 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:56.838 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.838 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.838 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4471.0:2731 3.a 3:50c18da3:::rbd_object_map.1121bf4f5ab3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-21T21:40:56.838 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:48.740462+0000", 2024-09-21T21:40:56.838 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 7.9975158650000004, 2024-09-21T21:40:56.838 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0092777689999999999, 2024-09-21T21:40:56.838 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:56.838 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:56.838 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:56.838 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:56.839 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4471", 2024-09-21T21:40:56.839 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1688278210", 2024-09-21T21:40:56.839 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2731 2024-09-21T21:40:56.839 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.839 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:56.839 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.839 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:56.839 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.740462+0000", 2024-09-21T21:40:56.839 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:56.839 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.839 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.839 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:56.839 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.740462+0000", 2024-09-21T21:40:56.839 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:56.839 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.840 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.840 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:56.840 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.740463+0000", 2024-09-21T21:40:56.840 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1100000000000001e-07 2024-09-21T21:40:56.840 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.840 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.840 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:56.840 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.740465+0000", 2024-09-21T21:40:56.840 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7030000000000002e-06 2024-09-21T21:40:56.840 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.840 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.840 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:56.840 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.740471+0000", 2024-09-21T21:40:56.840 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7969999999999997e-06 2024-09-21T21:40:56.841 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.841 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.841 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:56.841 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.740476+0000", 2024-09-21T21:40:56.841 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7759999999999997e-06 2024-09-21T21:40:56.841 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.841 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.841 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:56.841 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.744841+0000", 2024-09-21T21:40:56.841 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.004364639 2024-09-21T21:40:56.841 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.841 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.841 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:56.841 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.744911+0000", 2024-09-21T21:40:56.842 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0015e-05 2024-09-21T21:40:56.842 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.842 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.842 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 4", 2024-09-21T21:40:56.842 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.745246+0000", 2024-09-21T21:40:56.842 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00033522800000000002 2024-09-21T21:40:56.842 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.842 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.842 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:56.842 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.748470+0000", 2024-09-21T21:40:56.842 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0032241589999999999 2024-09-21T21:40:56.842 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.842 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.842 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:56.842 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.749650+0000", 2024-09-21T21:40:56.843 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001180204 2024-09-21T21:40:56.843 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.843 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.843 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:56.843 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.749699+0000", 2024-09-21T21:40:56.843 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8959999999999999e-05 2024-09-21T21:40:56.843 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.843 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.843 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:56.843 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.749740+0000", 2024-09-21T21:40:56.843 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0877000000000002e-05 2024-09-21T21:40:56.843 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:56.843 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:56.843 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:56.844 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.844 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.844 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4555.0:1737 3.13 3:cf584aeb:::rbd_data.1121889d185d.0000000000000002:9 [sparse-read 16318464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-21T21:40:56.844 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:56.558049+0000", 2024-09-21T21:40:56.844 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.179929491, 2024-09-21T21:40:56.844 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.007635335, 2024-09-21T21:40:56.844 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:56.844 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:56.844 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:40:56.844 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:56.844 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4555", 2024-09-21T21:40:56.844 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3402748328", 2024-09-21T21:40:56.844 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1737 2024-09-21T21:40:56.844 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.844 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:56.845 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.845 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:56.845 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:56.558049+0000", 2024-09-21T21:40:56.845 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:56.845 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.845 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.845 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:56.845 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:56.558049+0000", 2024-09-21T21:40:56.845 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:56.845 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.845 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.845 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:56.845 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:56.558049+0000", 2024-09-21T21:40:56.845 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7799999999999997e-07 2024-09-21T21:40:56.845 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.846 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.846 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:56.846 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:56.558051+0000", 2024-09-21T21:40:56.846 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.804e-06 2024-09-21T21:40:56.846 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.846 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.846 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:56.846 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:56.558057+0000", 2024-09-21T21:40:56.846 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5300000000000002e-06 2024-09-21T21:40:56.846 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.846 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.846 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:56.846 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:56.558062+0000", 2024-09-21T21:40:56.846 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.408e-06 2024-09-21T21:40:56.847 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.847 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.847 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:56.847 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:56.565592+0000", 2024-09-21T21:40:56.847 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.007530368 2024-09-21T21:40:56.847 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.847 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.847 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:56.847 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:56.565635+0000", 2024-09-21T21:40:56.847 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3186999999999998e-05 2024-09-21T21:40:56.847 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.847 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.847 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:56.847 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:56.565684+0000", 2024-09-21T21:40:56.848 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8760000000000001e-05 2024-09-21T21:40:56.848 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:56.848 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:56.848 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:56.848 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.848 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.848 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4555.0:1738 3.13 3:cf584aeb:::rbd_data.1121889d185d.0000000000000002:9 [sparse-read 15794176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-21T21:40:56.848 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:56.558067+0000", 2024-09-21T21:40:56.848 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.179910867, 2024-09-21T21:40:56.848 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0077095510000000003, 2024-09-21T21:40:56.848 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:56.848 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:56.848 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:40:56.848 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:56.848 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4555", 2024-09-21T21:40:56.849 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3402748328", 2024-09-21T21:40:56.849 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1738 2024-09-21T21:40:56.849 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.849 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:56.849 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.849 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:56.849 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:56.558067+0000", 2024-09-21T21:40:56.849 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:56.849 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.849 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.849 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:56.849 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:56.558067+0000", 2024-09-21T21:40:56.849 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:56.849 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.850 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.850 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:56.850 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:56.558068+0000", 2024-09-21T21:40:56.850 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4999999999999998e-07 2024-09-21T21:40:56.850 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.850 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.850 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:56.850 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:56.558072+0000", 2024-09-21T21:40:56.850 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4309999999999996e-06 2024-09-21T21:40:56.850 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.850 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.850 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:56.850 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:56.558076+0000", 2024-09-21T21:40:56.850 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8800000000000001e-06 2024-09-21T21:40:56.850 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.851 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.851 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:56.851 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:56.558079+0000", 2024-09-21T21:40:56.851 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.588e-06 2024-09-21T21:40:56.851 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.851 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.851 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:56.851 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:56.565699+0000", 2024-09-21T21:40:56.851 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0076201860000000001 2024-09-21T21:40:56.851 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.851 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.851 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:56.851 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:56.565732+0000", 2024-09-21T21:40:56.851 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3259000000000001e-05 2024-09-21T21:40:56.852 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:56.852 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:56.852 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:56.852 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:56.565777+0000", 2024-09-21T21:40:56.852 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4857000000000001e-05 2024-09-21T21:40:56.852 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:56.852 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:56.852 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:56.852 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:56.852 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:56.852 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:40:56.854 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T21:40:56.854 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:40:56.852+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:40:56.909 INFO:teuthology.orchestra.run.smithi164.stdout:985 read 0x1f3400 thru 0x1f9811 (0x6412 bytes) 2024-09-21T21:40:56.909 INFO:teuthology.orchestra.run.smithi164.stdout:987 read 0x18fa00 thru 0x1902fb (0x8fc bytes) 2024-09-21T21:40:56.923 INFO:teuthology.orchestra.run.smithi164.stdout:988 write 0x276c200 thru 0x276f54b (0x334c bytes) 2024-09-21T21:40:56.954 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:40:56.953+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:40:57.000 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:40:57.000 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:40:57.000 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-21T21:40:57.000 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:40:57.009 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T21:40:57.016 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:40:57.016 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:40:57.016 INFO:teuthology.orchestra.run.smithi081.stdout: "epoch": 34, 2024-09-21T21:40:57.016 INFO:teuthology.orchestra.run.smithi081.stdout: "fsid": "a0fdcfc2-ddd2-4e50-81bb-876d940f3578", 2024-09-21T21:40:57.016 INFO:teuthology.orchestra.run.smithi081.stdout: "created": "2024-09-21T21:40:07.413917+0000", 2024-09-21T21:40:57.016 INFO:teuthology.orchestra.run.smithi081.stdout: "modified": "2024-09-21T21:40:55.499815+0000", 2024-09-21T21:40:57.016 INFO:teuthology.orchestra.run.smithi081.stdout: "last_up_change": "2024-09-21T21:40:18.340430+0000", 2024-09-21T21:40:57.016 INFO:teuthology.orchestra.run.smithi081.stdout: "last_in_change": "2024-09-21T21:40:09.029953+0000", 2024-09-21T21:40:57.016 INFO:teuthology.orchestra.run.smithi081.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-21T21:40:57.016 INFO:teuthology.orchestra.run.smithi081.stdout: "flags_num": 5804032, 2024-09-21T21:40:57.016 INFO:teuthology.orchestra.run.smithi081.stdout: "flags_set": [ 2024-09-21T21:40:57.016 INFO:teuthology.orchestra.run.smithi081.stdout: "nodeep-scrub", 2024-09-21T21:40:57.017 INFO:teuthology.orchestra.run.smithi081.stdout: "pglog_hardlimit", 2024-09-21T21:40:57.017 INFO:teuthology.orchestra.run.smithi081.stdout: "purged_snapdirs", 2024-09-21T21:40:57.017 INFO:teuthology.orchestra.run.smithi081.stdout: "recovery_deletes", 2024-09-21T21:40:57.017 INFO:teuthology.orchestra.run.smithi081.stdout: "sortbitwise" 2024-09-21T21:40:57.017 INFO:teuthology.orchestra.run.smithi081.stdout: ], 2024-09-21T21:40:57.017 INFO:teuthology.orchestra.run.smithi081.stdout: "crush_version": 5, 2024-09-21T21:40:57.017 INFO:teuthology.orchestra.run.smithi081.stdout: "full_ratio": 0.94999998807907104, 2024-09-21T21:40:57.017 INFO:teuthology.orchestra.run.smithi081.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-21T21:40:57.017 INFO:teuthology.orchestra.run.smithi081.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-21T21:40:57.017 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster_snapshot": "", 2024-09-21T21:40:57.017 INFO:teuthology.orchestra.run.smithi081.stdout: "pool_max": 3, 2024-09-21T21:40:57.017 INFO:teuthology.orchestra.run.smithi081.stdout: "max_osd": 6, 2024-09-21T21:40:57.018 INFO:teuthology.orchestra.run.smithi081.stdout: "require_min_compat_client": "luminous", 2024-09-21T21:40:57.018 INFO:teuthology.orchestra.run.smithi081.stdout: "min_compat_client": "jewel", 2024-09-21T21:40:57.018 INFO:teuthology.orchestra.run.smithi081.stdout: "require_osd_release": "squid", 2024-09-21T21:40:57.018 INFO:teuthology.orchestra.run.smithi081.stdout: "allow_crimson": false, 2024-09-21T21:40:57.018 INFO:teuthology.orchestra.run.smithi081.stdout: "pools": [ 2024-09-21T21:40:57.018 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:57.018 INFO:teuthology.orchestra.run.smithi081.stdout: "pool": 1, 2024-09-21T21:40:57.018 INFO:teuthology.orchestra.run.smithi081.stdout: "pool_name": ".mgr", 2024-09-21T21:40:57.018 INFO:teuthology.orchestra.run.smithi081.stdout: "create_time": "2024-09-21T21:40:18.321328+0000", 2024-09-21T21:40:57.018 INFO:teuthology.orchestra.run.smithi081.stdout: "flags": 1, 2024-09-21T21:40:57.018 INFO:teuthology.orchestra.run.smithi081.stdout: "flags_names": "hashpspool", 2024-09-21T21:40:57.018 INFO:teuthology.orchestra.run.smithi081.stdout: "type": 1, 2024-09-21T21:40:57.018 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 2, 2024-09-21T21:40:57.019 INFO:teuthology.orchestra.run.smithi081.stdout: "min_size": 1, 2024-09-21T21:40:57.019 INFO:teuthology.orchestra.run.smithi081.stdout: "crush_rule": 0, 2024-09-21T21:40:57.019 INFO:teuthology.orchestra.run.smithi081.stdout: "peering_crush_bucket_count": 0, 2024-09-21T21:40:57.019 INFO:teuthology.orchestra.run.smithi081.stdout: "peering_crush_bucket_target": 0, 2024-09-21T21:40:57.019 INFO:teuthology.orchestra.run.smithi081.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T21:40:57.019 INFO:teuthology.orchestra.run.smithi081.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T21:40:57.019 INFO:teuthology.orchestra.run.smithi081.stdout: "is_stretch_pool": false, 2024-09-21T21:40:57.019 INFO:teuthology.orchestra.run.smithi081.stdout: "object_hash": 2, 2024-09-21T21:40:57.019 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_autoscale_mode": "off", 2024-09-21T21:40:57.019 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_num": 1, 2024-09-21T21:40:57.019 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_placement_num": 1, 2024-09-21T21:40:57.019 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_placement_num_target": 1, 2024-09-21T21:40:57.019 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_num_target": 1, 2024-09-21T21:40:57.019 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_num_pending": 1, 2024-09-21T21:40:57.020 INFO:teuthology.orchestra.run.smithi081.stdout: "last_pg_merge_meta": { 2024-09-21T21:40:57.020 INFO:teuthology.orchestra.run.smithi081.stdout: "source_pgid": "0.0", 2024-09-21T21:40:57.020 INFO:teuthology.orchestra.run.smithi081.stdout: "ready_epoch": 0, 2024-09-21T21:40:57.020 INFO:teuthology.orchestra.run.smithi081.stdout: "last_epoch_started": 0, 2024-09-21T21:40:57.020 INFO:teuthology.orchestra.run.smithi081.stdout: "last_epoch_clean": 0, 2024-09-21T21:40:57.020 INFO:teuthology.orchestra.run.smithi081.stdout: "source_version": "0'0", 2024-09-21T21:40:57.020 INFO:teuthology.orchestra.run.smithi081.stdout: "target_version": "0'0" 2024-09-21T21:40:57.020 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:57.020 INFO:teuthology.orchestra.run.smithi081.stdout: "last_change": "15", 2024-09-21T21:40:57.020 INFO:teuthology.orchestra.run.smithi081.stdout: "last_force_op_resend": "0", 2024-09-21T21:40:57.020 INFO:teuthology.orchestra.run.smithi081.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T21:40:57.020 INFO:teuthology.orchestra.run.smithi081.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T21:40:57.020 INFO:teuthology.orchestra.run.smithi081.stdout: "auid": 0, 2024-09-21T21:40:57.021 INFO:teuthology.orchestra.run.smithi081.stdout: "snap_mode": "selfmanaged", 2024-09-21T21:40:57.021 INFO:teuthology.orchestra.run.smithi081.stdout: "snap_seq": 0, 2024-09-21T21:40:57.021 INFO:teuthology.orchestra.run.smithi081.stdout: "snap_epoch": 0, 2024-09-21T21:40:57.021 INFO:teuthology.orchestra.run.smithi081.stdout: "pool_snaps": [], 2024-09-21T21:40:57.021 INFO:teuthology.orchestra.run.smithi081.stdout: "removed_snaps": "[]", 2024-09-21T21:40:57.021 INFO:teuthology.orchestra.run.smithi081.stdout: "quota_max_bytes": 0, 2024-09-21T21:40:57.021 INFO:teuthology.orchestra.run.smithi081.stdout: "quota_max_objects": 0, 2024-09-21T21:40:57.021 INFO:teuthology.orchestra.run.smithi081.stdout: "tiers": [], 2024-09-21T21:40:57.021 INFO:teuthology.orchestra.run.smithi081.stdout: "tier_of": -1, 2024-09-21T21:40:57.021 INFO:teuthology.orchestra.run.smithi081.stdout: "read_tier": -1, 2024-09-21T21:40:57.021 INFO:teuthology.orchestra.run.smithi081.stdout: "write_tier": -1, 2024-09-21T21:40:57.021 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_mode": "none", 2024-09-21T21:40:57.021 INFO:teuthology.orchestra.run.smithi081.stdout: "target_max_bytes": 0, 2024-09-21T21:40:57.021 INFO:teuthology.orchestra.run.smithi081.stdout: "target_max_objects": 0, 2024-09-21T21:40:57.021 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T21:40:57.022 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T21:40:57.022 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T21:40:57.022 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_min_flush_age": 0, 2024-09-21T21:40:57.022 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_min_evict_age": 0, 2024-09-21T21:40:57.022 INFO:teuthology.orchestra.run.smithi081.stdout: "erasure_code_profile": "", 2024-09-21T21:40:57.022 INFO:teuthology.orchestra.run.smithi081.stdout: "hit_set_params": { 2024-09-21T21:40:57.022 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "none" 2024-09-21T21:40:57.022 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:57.022 INFO:teuthology.orchestra.run.smithi081.stdout: "hit_set_period": 0, 2024-09-21T21:40:57.022 INFO:teuthology.orchestra.run.smithi081.stdout: "hit_set_count": 0, 2024-09-21T21:40:57.022 INFO:teuthology.orchestra.run.smithi081.stdout: "use_gmt_hitset": true, 2024-09-21T21:40:57.022 INFO:teuthology.orchestra.run.smithi081.stdout: "min_read_recency_for_promote": 0, 2024-09-21T21:40:57.022 INFO:teuthology.orchestra.run.smithi081.stdout: "min_write_recency_for_promote": 0, 2024-09-21T21:40:57.023 INFO:teuthology.orchestra.run.smithi081.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T21:40:57.023 INFO:teuthology.orchestra.run.smithi081.stdout: "hit_set_search_last_n": 0, 2024-09-21T21:40:57.023 INFO:teuthology.orchestra.run.smithi081.stdout: "grade_table": [], 2024-09-21T21:40:57.023 INFO:teuthology.orchestra.run.smithi081.stdout: "stripe_width": 0, 2024-09-21T21:40:57.023 INFO:teuthology.orchestra.run.smithi081.stdout: "expected_num_objects": 0, 2024-09-21T21:40:57.023 INFO:teuthology.orchestra.run.smithi081.stdout: "fast_read": false, 2024-09-21T21:40:57.023 INFO:teuthology.orchestra.run.smithi081.stdout: "options": { 2024-09-21T21:40:57.023 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_num_max": 32, 2024-09-21T21:40:57.023 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_num_min": 1 2024-09-21T21:40:57.023 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:57.023 INFO:teuthology.orchestra.run.smithi081.stdout: "application_metadata": { 2024-09-21T21:40:57.023 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": {} 2024-09-21T21:40:57.023 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:57.023 INFO:teuthology.orchestra.run.smithi081.stdout: "read_balance": { 2024-09-21T21:40:57.024 INFO:teuthology.orchestra.run.smithi081.stdout: "score_type": "Fair distribution", 2024-09-21T21:40:57.024 INFO:teuthology.orchestra.run.smithi081.stdout: "score_acting": 6.059999942779541, 2024-09-21T21:40:57.024 INFO:teuthology.orchestra.run.smithi081.stdout: "score_stable": 6.059999942779541, 2024-09-21T21:40:57.024 INFO:teuthology.orchestra.run.smithi081.stdout: "optimal_score": 0.33000001311302185, 2024-09-21T21:40:57.024 INFO:teuthology.orchestra.run.smithi081.stdout: "raw_score_acting": 2, 2024-09-21T21:40:57.024 INFO:teuthology.orchestra.run.smithi081.stdout: "raw_score_stable": 2, 2024-09-21T21:40:57.024 INFO:teuthology.orchestra.run.smithi081.stdout: "primary_affinity_weighted": 1, 2024-09-21T21:40:57.024 INFO:teuthology.orchestra.run.smithi081.stdout: "average_primary_affinity": 1, 2024-09-21T21:40:57.024 INFO:teuthology.orchestra.run.smithi081.stdout: "average_primary_affinity_weighted": 1 2024-09-21T21:40:57.024 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:57.024 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:57.024 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:57.024 INFO:teuthology.orchestra.run.smithi081.stdout: "pool": 2, 2024-09-21T21:40:57.024 INFO:teuthology.orchestra.run.smithi081.stdout: "pool_name": "rbd", 2024-09-21T21:40:57.025 INFO:teuthology.orchestra.run.smithi081.stdout: "create_time": "2024-09-21T21:40:24.235658+0000", 2024-09-21T21:40:57.025 INFO:teuthology.orchestra.run.smithi081.stdout: "flags": 8193, 2024-09-21T21:40:57.025 INFO:teuthology.orchestra.run.smithi081.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T21:40:57.025 INFO:teuthology.orchestra.run.smithi081.stdout: "type": 1, 2024-09-21T21:40:57.025 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 2, 2024-09-21T21:40:57.025 INFO:teuthology.orchestra.run.smithi081.stdout: "min_size": 1, 2024-09-21T21:40:57.025 INFO:teuthology.orchestra.run.smithi081.stdout: "crush_rule": 0, 2024-09-21T21:40:57.025 INFO:teuthology.orchestra.run.smithi081.stdout: "peering_crush_bucket_count": 0, 2024-09-21T21:40:57.025 INFO:teuthology.orchestra.run.smithi081.stdout: "peering_crush_bucket_target": 0, 2024-09-21T21:40:57.025 INFO:teuthology.orchestra.run.smithi081.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T21:40:57.025 INFO:teuthology.orchestra.run.smithi081.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T21:40:57.025 INFO:teuthology.orchestra.run.smithi081.stdout: "is_stretch_pool": false, 2024-09-21T21:40:57.025 INFO:teuthology.orchestra.run.smithi081.stdout: "object_hash": 2, 2024-09-21T21:40:57.025 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_autoscale_mode": "off", 2024-09-21T21:40:57.026 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_num": 8, 2024-09-21T21:40:57.026 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_placement_num": 8, 2024-09-21T21:40:57.026 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_placement_num_target": 8, 2024-09-21T21:40:57.026 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_num_target": 8, 2024-09-21T21:40:57.026 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_num_pending": 8, 2024-09-21T21:40:57.026 INFO:teuthology.orchestra.run.smithi081.stdout: "last_pg_merge_meta": { 2024-09-21T21:40:57.026 INFO:teuthology.orchestra.run.smithi081.stdout: "source_pgid": "0.0", 2024-09-21T21:40:57.026 INFO:teuthology.orchestra.run.smithi081.stdout: "ready_epoch": 0, 2024-09-21T21:40:57.026 INFO:teuthology.orchestra.run.smithi081.stdout: "last_epoch_started": 0, 2024-09-21T21:40:57.026 INFO:teuthology.orchestra.run.smithi081.stdout: "last_epoch_clean": 0, 2024-09-21T21:40:57.026 INFO:teuthology.orchestra.run.smithi081.stdout: "source_version": "0'0", 2024-09-21T21:40:57.026 INFO:teuthology.orchestra.run.smithi081.stdout: "target_version": "0'0" 2024-09-21T21:40:57.026 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:57.026 INFO:teuthology.orchestra.run.smithi081.stdout: "last_change": "19", 2024-09-21T21:40:57.027 INFO:teuthology.orchestra.run.smithi081.stdout: "last_force_op_resend": "0", 2024-09-21T21:40:57.027 INFO:teuthology.orchestra.run.smithi081.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T21:40:57.027 INFO:teuthology.orchestra.run.smithi081.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T21:40:57.027 INFO:teuthology.orchestra.run.smithi081.stdout: "auid": 0, 2024-09-21T21:40:57.027 INFO:teuthology.orchestra.run.smithi081.stdout: "snap_mode": "selfmanaged", 2024-09-21T21:40:57.027 INFO:teuthology.orchestra.run.smithi081.stdout: "snap_seq": 2, 2024-09-21T21:40:57.027 INFO:teuthology.orchestra.run.smithi081.stdout: "snap_epoch": 19, 2024-09-21T21:40:57.027 INFO:teuthology.orchestra.run.smithi081.stdout: "pool_snaps": [], 2024-09-21T21:40:57.027 INFO:teuthology.orchestra.run.smithi081.stdout: "removed_snaps": "[]", 2024-09-21T21:40:57.027 INFO:teuthology.orchestra.run.smithi081.stdout: "quota_max_bytes": 0, 2024-09-21T21:40:57.027 INFO:teuthology.orchestra.run.smithi081.stdout: "quota_max_objects": 0, 2024-09-21T21:40:57.027 INFO:teuthology.orchestra.run.smithi081.stdout: "tiers": [], 2024-09-21T21:40:57.027 INFO:teuthology.orchestra.run.smithi081.stdout: "tier_of": -1, 2024-09-21T21:40:57.027 INFO:teuthology.orchestra.run.smithi081.stdout: "read_tier": -1, 2024-09-21T21:40:57.027 INFO:teuthology.orchestra.run.smithi081.stdout: "write_tier": -1, 2024-09-21T21:40:57.028 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_mode": "none", 2024-09-21T21:40:57.028 INFO:teuthology.orchestra.run.smithi081.stdout: "target_max_bytes": 0, 2024-09-21T21:40:57.028 INFO:teuthology.orchestra.run.smithi081.stdout: "target_max_objects": 0, 2024-09-21T21:40:57.028 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T21:40:57.028 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T21:40:57.028 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T21:40:57.028 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_min_flush_age": 0, 2024-09-21T21:40:57.028 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_min_evict_age": 0, 2024-09-21T21:40:57.028 INFO:teuthology.orchestra.run.smithi081.stdout: "erasure_code_profile": "", 2024-09-21T21:40:57.028 INFO:teuthology.orchestra.run.smithi081.stdout: "hit_set_params": { 2024-09-21T21:40:57.028 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "none" 2024-09-21T21:40:57.028 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:57.028 INFO:teuthology.orchestra.run.smithi081.stdout: "hit_set_period": 0, 2024-09-21T21:40:57.028 INFO:teuthology.orchestra.run.smithi081.stdout: "hit_set_count": 0, 2024-09-21T21:40:57.029 INFO:teuthology.orchestra.run.smithi081.stdout: "use_gmt_hitset": true, 2024-09-21T21:40:57.029 INFO:teuthology.orchestra.run.smithi081.stdout: "min_read_recency_for_promote": 0, 2024-09-21T21:40:57.029 INFO:teuthology.orchestra.run.smithi081.stdout: "min_write_recency_for_promote": 0, 2024-09-21T21:40:57.029 INFO:teuthology.orchestra.run.smithi081.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T21:40:57.029 INFO:teuthology.orchestra.run.smithi081.stdout: "hit_set_search_last_n": 0, 2024-09-21T21:40:57.029 INFO:teuthology.orchestra.run.smithi081.stdout: "grade_table": [], 2024-09-21T21:40:57.029 INFO:teuthology.orchestra.run.smithi081.stdout: "stripe_width": 0, 2024-09-21T21:40:57.029 INFO:teuthology.orchestra.run.smithi081.stdout: "expected_num_objects": 0, 2024-09-21T21:40:57.029 INFO:teuthology.orchestra.run.smithi081.stdout: "fast_read": false, 2024-09-21T21:40:57.029 INFO:teuthology.orchestra.run.smithi081.stdout: "options": {}, 2024-09-21T21:40:57.029 INFO:teuthology.orchestra.run.smithi081.stdout: "application_metadata": { 2024-09-21T21:40:57.029 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd": {} 2024-09-21T21:40:57.029 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:57.029 INFO:teuthology.orchestra.run.smithi081.stdout: "read_balance": { 2024-09-21T21:40:57.030 INFO:teuthology.orchestra.run.smithi081.stdout: "score_type": "Fair distribution", 2024-09-21T21:40:57.030 INFO:teuthology.orchestra.run.smithi081.stdout: "score_acting": 2.25, 2024-09-21T21:40:57.030 INFO:teuthology.orchestra.run.smithi081.stdout: "score_stable": 2.25, 2024-09-21T21:40:57.030 INFO:teuthology.orchestra.run.smithi081.stdout: "optimal_score": 1, 2024-09-21T21:40:57.030 INFO:teuthology.orchestra.run.smithi081.stdout: "raw_score_acting": 2.25, 2024-09-21T21:40:57.030 INFO:teuthology.orchestra.run.smithi081.stdout: "raw_score_stable": 2.25, 2024-09-21T21:40:57.030 INFO:teuthology.orchestra.run.smithi081.stdout: "primary_affinity_weighted": 1, 2024-09-21T21:40:57.030 INFO:teuthology.orchestra.run.smithi081.stdout: "average_primary_affinity": 1, 2024-09-21T21:40:57.030 INFO:teuthology.orchestra.run.smithi081.stdout: "average_primary_affinity_weighted": 1 2024-09-21T21:40:57.030 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:57.030 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:57.030 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:57.030 INFO:teuthology.orchestra.run.smithi081.stdout: "pool": 3, 2024-09-21T21:40:57.030 INFO:teuthology.orchestra.run.smithi081.stdout: "pool_name": "pool_client.0", 2024-09-21T21:40:57.031 INFO:teuthology.orchestra.run.smithi081.stdout: "create_time": "2024-09-21T21:40:36.093691+0000", 2024-09-21T21:40:57.031 INFO:teuthology.orchestra.run.smithi081.stdout: "flags": 8193, 2024-09-21T21:40:57.031 INFO:teuthology.orchestra.run.smithi081.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T21:40:57.031 INFO:teuthology.orchestra.run.smithi081.stdout: "type": 1, 2024-09-21T21:40:57.031 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 2, 2024-09-21T21:40:57.031 INFO:teuthology.orchestra.run.smithi081.stdout: "min_size": 1, 2024-09-21T21:40:57.031 INFO:teuthology.orchestra.run.smithi081.stdout: "crush_rule": 0, 2024-09-21T21:40:57.031 INFO:teuthology.orchestra.run.smithi081.stdout: "peering_crush_bucket_count": 0, 2024-09-21T21:40:57.031 INFO:teuthology.orchestra.run.smithi081.stdout: "peering_crush_bucket_target": 0, 2024-09-21T21:40:57.031 INFO:teuthology.orchestra.run.smithi081.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T21:40:57.031 INFO:teuthology.orchestra.run.smithi081.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T21:40:57.031 INFO:teuthology.orchestra.run.smithi081.stdout: "is_stretch_pool": false, 2024-09-21T21:40:57.031 INFO:teuthology.orchestra.run.smithi081.stdout: "object_hash": 2, 2024-09-21T21:40:57.031 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_autoscale_mode": "off", 2024-09-21T21:40:57.031 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_num": 32, 2024-09-21T21:40:57.032 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_placement_num": 32, 2024-09-21T21:40:57.032 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_placement_num_target": 32, 2024-09-21T21:40:57.032 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_num_target": 32, 2024-09-21T21:40:57.032 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_num_pending": 32, 2024-09-21T21:40:57.032 INFO:teuthology.orchestra.run.smithi081.stdout: "last_pg_merge_meta": { 2024-09-21T21:40:57.032 INFO:teuthology.orchestra.run.smithi081.stdout: "source_pgid": "0.0", 2024-09-21T21:40:57.032 INFO:teuthology.orchestra.run.smithi081.stdout: "ready_epoch": 0, 2024-09-21T21:40:57.032 INFO:teuthology.orchestra.run.smithi081.stdout: "last_epoch_started": 0, 2024-09-21T21:40:57.032 INFO:teuthology.orchestra.run.smithi081.stdout: "last_epoch_clean": 0, 2024-09-21T21:40:57.032 INFO:teuthology.orchestra.run.smithi081.stdout: "source_version": "0'0", 2024-09-21T21:40:57.032 INFO:teuthology.orchestra.run.smithi081.stdout: "target_version": "0'0" 2024-09-21T21:40:57.032 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:57.032 INFO:teuthology.orchestra.run.smithi081.stdout: "last_change": "34", 2024-09-21T21:40:57.032 INFO:teuthology.orchestra.run.smithi081.stdout: "last_force_op_resend": "0", 2024-09-21T21:40:57.033 INFO:teuthology.orchestra.run.smithi081.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T21:40:57.033 INFO:teuthology.orchestra.run.smithi081.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T21:40:57.033 INFO:teuthology.orchestra.run.smithi081.stdout: "auid": 0, 2024-09-21T21:40:57.033 INFO:teuthology.orchestra.run.smithi081.stdout: "snap_mode": "selfmanaged", 2024-09-21T21:40:57.033 INFO:teuthology.orchestra.run.smithi081.stdout: "snap_seq": 9, 2024-09-21T21:40:57.033 INFO:teuthology.orchestra.run.smithi081.stdout: "snap_epoch": 34, 2024-09-21T21:40:57.033 INFO:teuthology.orchestra.run.smithi081.stdout: "pool_snaps": [], 2024-09-21T21:40:57.033 INFO:teuthology.orchestra.run.smithi081.stdout: "removed_snaps": "[]", 2024-09-21T21:40:57.033 INFO:teuthology.orchestra.run.smithi081.stdout: "quota_max_bytes": 0, 2024-09-21T21:40:57.033 INFO:teuthology.orchestra.run.smithi081.stdout: "quota_max_objects": 0, 2024-09-21T21:40:57.033 INFO:teuthology.orchestra.run.smithi081.stdout: "tiers": [], 2024-09-21T21:40:57.033 INFO:teuthology.orchestra.run.smithi081.stdout: "tier_of": -1, 2024-09-21T21:40:57.033 INFO:teuthology.orchestra.run.smithi081.stdout: "read_tier": -1, 2024-09-21T21:40:57.033 INFO:teuthology.orchestra.run.smithi081.stdout: "write_tier": -1, 2024-09-21T21:40:57.034 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_mode": "none", 2024-09-21T21:40:57.034 INFO:teuthology.orchestra.run.smithi081.stdout: "target_max_bytes": 0, 2024-09-21T21:40:57.034 INFO:teuthology.orchestra.run.smithi081.stdout: "target_max_objects": 0, 2024-09-21T21:40:57.034 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T21:40:57.034 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T21:40:57.034 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T21:40:57.034 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_min_flush_age": 0, 2024-09-21T21:40:57.034 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_min_evict_age": 0, 2024-09-21T21:40:57.034 INFO:teuthology.orchestra.run.smithi081.stdout: "erasure_code_profile": "", 2024-09-21T21:40:57.034 INFO:teuthology.orchestra.run.smithi081.stdout: "hit_set_params": { 2024-09-21T21:40:57.034 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "none" 2024-09-21T21:40:57.034 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:57.034 INFO:teuthology.orchestra.run.smithi081.stdout: "hit_set_period": 0, 2024-09-21T21:40:57.034 INFO:teuthology.orchestra.run.smithi081.stdout: "hit_set_count": 0, 2024-09-21T21:40:57.035 INFO:teuthology.orchestra.run.smithi081.stdout: "use_gmt_hitset": true, 2024-09-21T21:40:57.035 INFO:teuthology.orchestra.run.smithi081.stdout: "min_read_recency_for_promote": 0, 2024-09-21T21:40:57.035 INFO:teuthology.orchestra.run.smithi081.stdout: "min_write_recency_for_promote": 0, 2024-09-21T21:40:57.035 INFO:teuthology.orchestra.run.smithi081.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T21:40:57.035 INFO:teuthology.orchestra.run.smithi081.stdout: "hit_set_search_last_n": 0, 2024-09-21T21:40:57.035 INFO:teuthology.orchestra.run.smithi081.stdout: "grade_table": [], 2024-09-21T21:40:57.035 INFO:teuthology.orchestra.run.smithi081.stdout: "stripe_width": 0, 2024-09-21T21:40:57.035 INFO:teuthology.orchestra.run.smithi081.stdout: "expected_num_objects": 0, 2024-09-21T21:40:57.035 INFO:teuthology.orchestra.run.smithi081.stdout: "fast_read": false, 2024-09-21T21:40:57.035 INFO:teuthology.orchestra.run.smithi081.stdout: "options": {}, 2024-09-21T21:40:57.035 INFO:teuthology.orchestra.run.smithi081.stdout: "application_metadata": { 2024-09-21T21:40:57.035 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd": {} 2024-09-21T21:40:57.035 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:57.035 INFO:teuthology.orchestra.run.smithi081.stdout: "read_balance": { 2024-09-21T21:40:57.036 INFO:teuthology.orchestra.run.smithi081.stdout: "score_type": "Fair distribution", 2024-09-21T21:40:57.036 INFO:teuthology.orchestra.run.smithi081.stdout: "score_acting": 1.5, 2024-09-21T21:40:57.036 INFO:teuthology.orchestra.run.smithi081.stdout: "score_stable": 1.5, 2024-09-21T21:40:57.036 INFO:teuthology.orchestra.run.smithi081.stdout: "optimal_score": 1, 2024-09-21T21:40:57.036 INFO:teuthology.orchestra.run.smithi081.stdout: "raw_score_acting": 1.5, 2024-09-21T21:40:57.036 INFO:teuthology.orchestra.run.smithi081.stdout: "raw_score_stable": 1.5, 2024-09-21T21:40:57.036 INFO:teuthology.orchestra.run.smithi081.stdout: "primary_affinity_weighted": 1, 2024-09-21T21:40:57.036 INFO:teuthology.orchestra.run.smithi081.stdout: "average_primary_affinity": 1, 2024-09-21T21:40:57.036 INFO:teuthology.orchestra.run.smithi081.stdout: "average_primary_affinity_weighted": 1 2024-09-21T21:40:57.036 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:57.036 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:57.036 INFO:teuthology.orchestra.run.smithi081.stdout: ], 2024-09-21T21:40:57.036 INFO:teuthology.orchestra.run.smithi081.stdout: "osds": [ 2024-09-21T21:40:57.036 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:57.037 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": 0, 2024-09-21T21:40:57.037 INFO:teuthology.orchestra.run.smithi081.stdout: "uuid": "a9869830-8e74-4b02-963d-ab630a0ee286", 2024-09-21T21:40:57.037 INFO:teuthology.orchestra.run.smithi081.stdout: "up": 1, 2024-09-21T21:40:57.037 INFO:teuthology.orchestra.run.smithi081.stdout: "in": 1, 2024-09-21T21:40:57.037 INFO:teuthology.orchestra.run.smithi081.stdout: "weight": 1, 2024-09-21T21:40:57.037 INFO:teuthology.orchestra.run.smithi081.stdout: "primary_affinity": 1, 2024-09-21T21:40:57.037 INFO:teuthology.orchestra.run.smithi081.stdout: "last_clean_begin": 0, 2024-09-21T21:40:57.037 INFO:teuthology.orchestra.run.smithi081.stdout: "last_clean_end": 0, 2024-09-21T21:40:57.037 INFO:teuthology.orchestra.run.smithi081.stdout: "up_from": 12, 2024-09-21T21:40:57.037 INFO:teuthology.orchestra.run.smithi081.stdout: "up_thru": 21, 2024-09-21T21:40:57.037 INFO:teuthology.orchestra.run.smithi081.stdout: "down_at": 0, 2024-09-21T21:40:57.037 INFO:teuthology.orchestra.run.smithi081.stdout: "lost_at": 0, 2024-09-21T21:40:57.037 INFO:teuthology.orchestra.run.smithi081.stdout: "public_addrs": { 2024-09-21T21:40:57.037 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:40:57.038 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:57.038 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:40:57.038 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6808", 2024-09-21T21:40:57.038 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1657552338 2024-09-21T21:40:57.038 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:57.038 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:57.038 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:40:57.038 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6809", 2024-09-21T21:40:57.038 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1657552338 2024-09-21T21:40:57.038 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:57.038 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:57.038 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:57.038 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster_addrs": { 2024-09-21T21:40:57.038 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:40:57.039 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:57.039 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:40:57.039 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6810", 2024-09-21T21:40:57.039 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1657552338 2024-09-21T21:40:57.039 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:57.039 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:57.039 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:40:57.039 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6811", 2024-09-21T21:40:57.039 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1657552338 2024-09-21T21:40:57.039 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:57.039 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:57.039 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:57.039 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_back_addrs": { 2024-09-21T21:40:57.039 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:40:57.039 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:57.040 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:40:57.040 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6814", 2024-09-21T21:40:57.040 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1657552338 2024-09-21T21:40:57.040 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:57.040 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:57.040 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:40:57.040 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6815", 2024-09-21T21:40:57.040 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1657552338 2024-09-21T21:40:57.040 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:57.040 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:57.040 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:57.040 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_front_addrs": { 2024-09-21T21:40:57.040 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:40:57.040 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:57.041 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:40:57.041 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6812", 2024-09-21T21:40:57.041 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1657552338 2024-09-21T21:40:57.041 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:57.041 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:57.041 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:40:57.041 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6813", 2024-09-21T21:40:57.041 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1657552338 2024-09-21T21:40:57.041 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:57.041 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:57.041 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:57.041 INFO:teuthology.orchestra.run.smithi081.stdout: "public_addr": "172.21.15.81:6809/1657552338", 2024-09-21T21:40:57.041 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster_addr": "172.21.15.81:6811/1657552338", 2024-09-21T21:40:57.041 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_back_addr": "172.21.15.81:6815/1657552338", 2024-09-21T21:40:57.041 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_front_addr": "172.21.15.81:6813/1657552338", 2024-09-21T21:40:57.041 INFO:teuthology.orchestra.run.smithi081.stdout: "state": [ 2024-09-21T21:40:57.042 INFO:teuthology.orchestra.run.smithi081.stdout: "exists", 2024-09-21T21:40:57.042 INFO:teuthology.orchestra.run.smithi081.stdout: "up" 2024-09-21T21:40:57.042 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:57.042 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:57.042 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:57.042 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": 1, 2024-09-21T21:40:57.042 INFO:teuthology.orchestra.run.smithi081.stdout: "uuid": "b189819d-69a3-4e95-96e7-96dc4809ac80", 2024-09-21T21:40:57.042 INFO:teuthology.orchestra.run.smithi081.stdout: "up": 1, 2024-09-21T21:40:57.042 INFO:teuthology.orchestra.run.smithi081.stdout: "in": 1, 2024-09-21T21:40:57.042 INFO:teuthology.orchestra.run.smithi081.stdout: "weight": 1, 2024-09-21T21:40:57.042 INFO:teuthology.orchestra.run.smithi081.stdout: "primary_affinity": 1, 2024-09-21T21:40:57.042 INFO:teuthology.orchestra.run.smithi081.stdout: "last_clean_begin": 0, 2024-09-21T21:40:57.042 INFO:teuthology.orchestra.run.smithi081.stdout: "last_clean_end": 0, 2024-09-21T21:40:57.042 INFO:teuthology.orchestra.run.smithi081.stdout: "up_from": 12, 2024-09-21T21:40:57.043 INFO:teuthology.orchestra.run.smithi081.stdout: "up_thru": 21, 2024-09-21T21:40:57.043 INFO:teuthology.orchestra.run.smithi081.stdout: "down_at": 0, 2024-09-21T21:40:57.043 INFO:teuthology.orchestra.run.smithi081.stdout: "lost_at": 0, 2024-09-21T21:40:57.043 INFO:teuthology.orchestra.run.smithi081.stdout: "public_addrs": { 2024-09-21T21:40:57.043 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:40:57.043 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:57.043 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:40:57.043 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6816", 2024-09-21T21:40:57.043 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2864159502 2024-09-21T21:40:57.043 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:57.043 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:57.043 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:40:57.043 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6817", 2024-09-21T21:40:57.043 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2864159502 2024-09-21T21:40:57.044 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:57.044 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:57.044 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:57.044 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster_addrs": { 2024-09-21T21:40:57.044 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:40:57.044 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:57.044 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:40:57.044 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6818", 2024-09-21T21:40:57.044 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2864159502 2024-09-21T21:40:57.044 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:57.044 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:57.044 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:40:57.044 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6819", 2024-09-21T21:40:57.044 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2864159502 2024-09-21T21:40:57.045 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:57.045 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:57.045 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:57.045 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_back_addrs": { 2024-09-21T21:40:57.045 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:40:57.045 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:57.045 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:40:57.045 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6822", 2024-09-21T21:40:57.045 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2864159502 2024-09-21T21:40:57.045 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:57.045 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:57.045 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:40:57.045 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6823", 2024-09-21T21:40:57.045 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2864159502 2024-09-21T21:40:57.046 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:57.046 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:57.046 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:57.046 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_front_addrs": { 2024-09-21T21:40:57.046 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:40:57.046 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:57.046 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:40:57.046 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6820", 2024-09-21T21:40:57.046 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2864159502 2024-09-21T21:40:57.046 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:57.046 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:57.046 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:40:57.046 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6821", 2024-09-21T21:40:57.046 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2864159502 2024-09-21T21:40:57.047 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:57.047 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:57.047 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:57.047 INFO:teuthology.orchestra.run.smithi081.stdout: "public_addr": "172.21.15.81:6817/2864159502", 2024-09-21T21:40:57.047 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster_addr": "172.21.15.81:6819/2864159502", 2024-09-21T21:40:57.047 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_back_addr": "172.21.15.81:6823/2864159502", 2024-09-21T21:40:57.047 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_front_addr": "172.21.15.81:6821/2864159502", 2024-09-21T21:40:57.047 INFO:teuthology.orchestra.run.smithi081.stdout: "state": [ 2024-09-21T21:40:57.047 INFO:teuthology.orchestra.run.smithi081.stdout: "exists", 2024-09-21T21:40:57.047 INFO:teuthology.orchestra.run.smithi081.stdout: "up" 2024-09-21T21:40:57.047 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:57.047 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:57.047 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:57.047 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": 2, 2024-09-21T21:40:57.048 INFO:teuthology.orchestra.run.smithi081.stdout: "uuid": "8437b99a-1297-4903-ab66-95c5cba2b2bf", 2024-09-21T21:40:57.048 INFO:teuthology.orchestra.run.smithi081.stdout: "up": 1, 2024-09-21T21:40:57.048 INFO:teuthology.orchestra.run.smithi081.stdout: "in": 1, 2024-09-21T21:40:57.048 INFO:teuthology.orchestra.run.smithi081.stdout: "weight": 1, 2024-09-21T21:40:57.048 INFO:teuthology.orchestra.run.smithi081.stdout: "primary_affinity": 1, 2024-09-21T21:40:57.048 INFO:teuthology.orchestra.run.smithi081.stdout: "last_clean_begin": 0, 2024-09-21T21:40:57.048 INFO:teuthology.orchestra.run.smithi081.stdout: "last_clean_end": 0, 2024-09-21T21:40:57.048 INFO:teuthology.orchestra.run.smithi081.stdout: "up_from": 11, 2024-09-21T21:40:57.048 INFO:teuthology.orchestra.run.smithi081.stdout: "up_thru": 21, 2024-09-21T21:40:57.048 INFO:teuthology.orchestra.run.smithi081.stdout: "down_at": 0, 2024-09-21T21:40:57.048 INFO:teuthology.orchestra.run.smithi081.stdout: "lost_at": 0, 2024-09-21T21:40:57.048 INFO:teuthology.orchestra.run.smithi081.stdout: "public_addrs": { 2024-09-21T21:40:57.048 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:40:57.049 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:57.049 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:40:57.049 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6800", 2024-09-21T21:40:57.049 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 3023160932 2024-09-21T21:40:57.049 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:57.049 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:57.049 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:40:57.049 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6801", 2024-09-21T21:40:57.049 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 3023160932 2024-09-21T21:40:57.049 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:57.049 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:57.049 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:57.049 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster_addrs": { 2024-09-21T21:40:57.049 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:40:57.049 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:57.050 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:40:57.050 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6802", 2024-09-21T21:40:57.050 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 3023160932 2024-09-21T21:40:57.050 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:57.050 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:57.050 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:40:57.050 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6803", 2024-09-21T21:40:57.050 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 3023160932 2024-09-21T21:40:57.050 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:57.050 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:57.050 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:57.050 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_back_addrs": { 2024-09-21T21:40:57.050 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:40:57.051 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:57.051 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:40:57.051 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6806", 2024-09-21T21:40:57.051 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 3023160932 2024-09-21T21:40:57.051 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:57.051 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:57.051 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:40:57.051 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6807", 2024-09-21T21:40:57.051 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 3023160932 2024-09-21T21:40:57.051 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:57.051 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:57.051 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:57.051 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_front_addrs": { 2024-09-21T21:40:57.051 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:40:57.052 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:57.052 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:40:57.052 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6804", 2024-09-21T21:40:57.052 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 3023160932 2024-09-21T21:40:57.052 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:57.052 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:57.052 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:40:57.052 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6805", 2024-09-21T21:40:57.052 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 3023160932 2024-09-21T21:40:57.052 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:57.052 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:57.052 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:57.052 INFO:teuthology.orchestra.run.smithi081.stdout: "public_addr": "172.21.15.81:6801/3023160932", 2024-09-21T21:40:57.052 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster_addr": "172.21.15.81:6803/3023160932", 2024-09-21T21:40:57.052 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_back_addr": "172.21.15.81:6807/3023160932", 2024-09-21T21:40:57.053 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_front_addr": "172.21.15.81:6805/3023160932", 2024-09-21T21:40:57.053 INFO:teuthology.orchestra.run.smithi081.stdout: "state": [ 2024-09-21T21:40:57.053 INFO:teuthology.orchestra.run.smithi081.stdout: "exists", 2024-09-21T21:40:57.053 INFO:teuthology.orchestra.run.smithi081.stdout: "up" 2024-09-21T21:40:57.053 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:57.053 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:57.053 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:57.053 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": 3, 2024-09-21T21:40:57.053 INFO:teuthology.orchestra.run.smithi081.stdout: "uuid": "e8848768-c1aa-445b-9554-dc6aa4eefc8d", 2024-09-21T21:40:57.053 INFO:teuthology.orchestra.run.smithi081.stdout: "up": 1, 2024-09-21T21:40:57.053 INFO:teuthology.orchestra.run.smithi081.stdout: "in": 1, 2024-09-21T21:40:57.053 INFO:teuthology.orchestra.run.smithi081.stdout: "weight": 1, 2024-09-21T21:40:57.053 INFO:teuthology.orchestra.run.smithi081.stdout: "primary_affinity": 1, 2024-09-21T21:40:57.053 INFO:teuthology.orchestra.run.smithi081.stdout: "last_clean_begin": 0, 2024-09-21T21:40:57.053 INFO:teuthology.orchestra.run.smithi081.stdout: "last_clean_end": 0, 2024-09-21T21:40:57.054 INFO:teuthology.orchestra.run.smithi081.stdout: "up_from": 13, 2024-09-21T21:40:57.054 INFO:teuthology.orchestra.run.smithi081.stdout: "up_thru": 21, 2024-09-21T21:40:57.054 INFO:teuthology.orchestra.run.smithi081.stdout: "down_at": 0, 2024-09-21T21:40:57.054 INFO:teuthology.orchestra.run.smithi081.stdout: "lost_at": 0, 2024-09-21T21:40:57.054 INFO:teuthology.orchestra.run.smithi081.stdout: "public_addrs": { 2024-09-21T21:40:57.054 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:40:57.054 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:57.054 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:40:57.054 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6808", 2024-09-21T21:40:57.054 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2681016844 2024-09-21T21:40:57.054 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:57.054 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:57.054 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:40:57.054 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6809", 2024-09-21T21:40:57.055 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2681016844 2024-09-21T21:40:57.055 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:57.055 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:57.055 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:57.055 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster_addrs": { 2024-09-21T21:40:57.055 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:40:57.055 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:57.055 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:40:57.055 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6810", 2024-09-21T21:40:57.055 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2681016844 2024-09-21T21:40:57.055 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:57.055 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:57.055 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:40:57.055 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6811", 2024-09-21T21:40:57.055 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2681016844 2024-09-21T21:40:57.056 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:57.056 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:57.056 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:57.056 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_back_addrs": { 2024-09-21T21:40:57.056 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:40:57.056 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:57.056 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:40:57.056 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6814", 2024-09-21T21:40:57.056 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2681016844 2024-09-21T21:40:57.056 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:57.056 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:57.056 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:40:57.056 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6815", 2024-09-21T21:40:57.056 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2681016844 2024-09-21T21:40:57.056 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:57.056 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:57.057 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:57.057 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_front_addrs": { 2024-09-21T21:40:57.057 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:40:57.057 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:57.057 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:40:57.057 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6812", 2024-09-21T21:40:57.057 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2681016844 2024-09-21T21:40:57.057 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:57.057 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:57.057 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:40:57.057 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6813", 2024-09-21T21:40:57.057 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2681016844 2024-09-21T21:40:57.057 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:57.057 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:57.058 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:57.058 INFO:teuthology.orchestra.run.smithi081.stdout: "public_addr": "172.21.15.122:6809/2681016844", 2024-09-21T21:40:57.058 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster_addr": "172.21.15.122:6811/2681016844", 2024-09-21T21:40:57.058 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_back_addr": "172.21.15.122:6815/2681016844", 2024-09-21T21:40:57.058 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_front_addr": "172.21.15.122:6813/2681016844", 2024-09-21T21:40:57.058 INFO:teuthology.orchestra.run.smithi081.stdout: "state": [ 2024-09-21T21:40:57.058 INFO:teuthology.orchestra.run.smithi081.stdout: "exists", 2024-09-21T21:40:57.058 INFO:teuthology.orchestra.run.smithi081.stdout: "up" 2024-09-21T21:40:57.058 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:57.058 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:57.058 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:57.058 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": 4, 2024-09-21T21:40:57.058 INFO:teuthology.orchestra.run.smithi081.stdout: "uuid": "b1d8e85d-a207-4ba6-9322-61d07812083e", 2024-09-21T21:40:57.059 INFO:teuthology.orchestra.run.smithi081.stdout: "up": 1, 2024-09-21T21:40:57.059 INFO:teuthology.orchestra.run.smithi081.stdout: "in": 1, 2024-09-21T21:40:57.059 INFO:teuthology.orchestra.run.smithi081.stdout: "weight": 1, 2024-09-21T21:40:57.059 INFO:teuthology.orchestra.run.smithi081.stdout: "primary_affinity": 1, 2024-09-21T21:40:57.059 INFO:teuthology.orchestra.run.smithi081.stdout: "last_clean_begin": 0, 2024-09-21T21:40:57.059 INFO:teuthology.orchestra.run.smithi081.stdout: "last_clean_end": 0, 2024-09-21T21:40:57.059 INFO:teuthology.orchestra.run.smithi081.stdout: "up_from": 13, 2024-09-21T21:40:57.059 INFO:teuthology.orchestra.run.smithi081.stdout: "up_thru": 21, 2024-09-21T21:40:57.059 INFO:teuthology.orchestra.run.smithi081.stdout: "down_at": 0, 2024-09-21T21:40:57.059 INFO:teuthology.orchestra.run.smithi081.stdout: "lost_at": 0, 2024-09-21T21:40:57.059 INFO:teuthology.orchestra.run.smithi081.stdout: "public_addrs": { 2024-09-21T21:40:57.059 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:40:57.059 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:57.059 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:40:57.060 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6800", 2024-09-21T21:40:57.060 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1095023540 2024-09-21T21:40:57.060 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:57.060 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:57.060 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:40:57.060 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6801", 2024-09-21T21:40:57.060 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1095023540 2024-09-21T21:40:57.060 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:57.060 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:57.060 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:57.060 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster_addrs": { 2024-09-21T21:40:57.060 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:40:57.060 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:57.060 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:40:57.061 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6802", 2024-09-21T21:40:57.061 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1095023540 2024-09-21T21:40:57.061 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:57.061 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:57.061 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:40:57.061 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6803", 2024-09-21T21:40:57.061 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1095023540 2024-09-21T21:40:57.061 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:57.061 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:57.061 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:57.061 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_back_addrs": { 2024-09-21T21:40:57.061 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:40:57.061 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:57.061 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:40:57.062 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6806", 2024-09-21T21:40:57.062 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1095023540 2024-09-21T21:40:57.062 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:57.062 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:57.062 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:40:57.062 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6807", 2024-09-21T21:40:57.062 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1095023540 2024-09-21T21:40:57.062 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:57.062 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:57.062 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:57.062 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_front_addrs": { 2024-09-21T21:40:57.062 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:40:57.062 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:57.062 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:40:57.063 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6804", 2024-09-21T21:40:57.063 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1095023540 2024-09-21T21:40:57.063 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:57.063 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:57.063 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:40:57.063 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6805", 2024-09-21T21:40:57.063 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1095023540 2024-09-21T21:40:57.063 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:57.063 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:57.063 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:57.063 INFO:teuthology.orchestra.run.smithi081.stdout: "public_addr": "172.21.15.122:6801/1095023540", 2024-09-21T21:40:57.063 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster_addr": "172.21.15.122:6803/1095023540", 2024-09-21T21:40:57.063 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_back_addr": "172.21.15.122:6807/1095023540", 2024-09-21T21:40:57.064 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_front_addr": "172.21.15.122:6805/1095023540", 2024-09-21T21:40:57.064 INFO:teuthology.orchestra.run.smithi081.stdout: "state": [ 2024-09-21T21:40:57.064 INFO:teuthology.orchestra.run.smithi081.stdout: "exists", 2024-09-21T21:40:57.064 INFO:teuthology.orchestra.run.smithi081.stdout: "up" 2024-09-21T21:40:57.064 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:57.064 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:57.064 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:57.064 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": 5, 2024-09-21T21:40:57.064 INFO:teuthology.orchestra.run.smithi081.stdout: "uuid": "3ab0450b-d8fd-4ef2-a298-08d5aa2ea32f", 2024-09-21T21:40:57.064 INFO:teuthology.orchestra.run.smithi081.stdout: "up": 1, 2024-09-21T21:40:57.064 INFO:teuthology.orchestra.run.smithi081.stdout: "in": 1, 2024-09-21T21:40:57.064 INFO:teuthology.orchestra.run.smithi081.stdout: "weight": 1, 2024-09-21T21:40:57.064 INFO:teuthology.orchestra.run.smithi081.stdout: "primary_affinity": 1, 2024-09-21T21:40:57.065 INFO:teuthology.orchestra.run.smithi081.stdout: "last_clean_begin": 0, 2024-09-21T21:40:57.065 INFO:teuthology.orchestra.run.smithi081.stdout: "last_clean_end": 0, 2024-09-21T21:40:57.065 INFO:teuthology.orchestra.run.smithi081.stdout: "up_from": 12, 2024-09-21T21:40:57.065 INFO:teuthology.orchestra.run.smithi081.stdout: "up_thru": 21, 2024-09-21T21:40:57.065 INFO:teuthology.orchestra.run.smithi081.stdout: "down_at": 0, 2024-09-21T21:40:57.065 INFO:teuthology.orchestra.run.smithi081.stdout: "lost_at": 0, 2024-09-21T21:40:57.065 INFO:teuthology.orchestra.run.smithi081.stdout: "public_addrs": { 2024-09-21T21:40:57.065 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:40:57.065 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:57.065 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:40:57.065 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6816", 2024-09-21T21:40:57.065 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1974398333 2024-09-21T21:40:57.065 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:57.065 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:57.066 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:40:57.066 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6817", 2024-09-21T21:40:57.066 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1974398333 2024-09-21T21:40:57.066 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:57.066 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:57.066 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:57.066 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster_addrs": { 2024-09-21T21:40:57.066 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:40:57.066 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:57.066 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:40:57.066 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6818", 2024-09-21T21:40:57.066 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1974398333 2024-09-21T21:40:57.066 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:57.066 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:57.067 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:40:57.067 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6819", 2024-09-21T21:40:57.067 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1974398333 2024-09-21T21:40:57.067 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:57.067 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:57.067 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:57.067 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_back_addrs": { 2024-09-21T21:40:57.067 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:40:57.067 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:57.067 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:40:57.067 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6822", 2024-09-21T21:40:57.067 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1974398333 2024-09-21T21:40:57.067 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:57.067 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:57.067 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:40:57.068 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6823", 2024-09-21T21:40:57.068 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1974398333 2024-09-21T21:40:57.068 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:57.068 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:57.068 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:57.068 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_front_addrs": { 2024-09-21T21:40:57.068 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:40:57.068 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:57.068 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:40:57.068 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6820", 2024-09-21T21:40:57.068 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1974398333 2024-09-21T21:40:57.068 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:57.068 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:57.068 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:40:57.068 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6821", 2024-09-21T21:40:57.069 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1974398333 2024-09-21T21:40:57.069 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:57.069 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:57.069 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:57.069 INFO:teuthology.orchestra.run.smithi081.stdout: "public_addr": "172.21.15.122:6817/1974398333", 2024-09-21T21:40:57.069 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster_addr": "172.21.15.122:6819/1974398333", 2024-09-21T21:40:57.069 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_back_addr": "172.21.15.122:6823/1974398333", 2024-09-21T21:40:57.069 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_front_addr": "172.21.15.122:6821/1974398333", 2024-09-21T21:40:57.069 INFO:teuthology.orchestra.run.smithi081.stdout: "state": [ 2024-09-21T21:40:57.069 INFO:teuthology.orchestra.run.smithi081.stdout: "exists", 2024-09-21T21:40:57.069 INFO:teuthology.orchestra.run.smithi081.stdout: "up" 2024-09-21T21:40:57.069 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:57.069 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:57.069 INFO:teuthology.orchestra.run.smithi081.stdout: ], 2024-09-21T21:40:57.070 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_xinfo": [ 2024-09-21T21:40:57.070 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:57.070 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": 0, 2024-09-21T21:40:57.070 INFO:teuthology.orchestra.run.smithi081.stdout: "down_stamp": "0.000000", 2024-09-21T21:40:57.070 INFO:teuthology.orchestra.run.smithi081.stdout: "laggy_probability": 0, 2024-09-21T21:40:57.070 INFO:teuthology.orchestra.run.smithi081.stdout: "laggy_interval": 0, 2024-09-21T21:40:57.070 INFO:teuthology.orchestra.run.smithi081.stdout: "features": 4540701547738038271, 2024-09-21T21:40:57.070 INFO:teuthology.orchestra.run.smithi081.stdout: "old_weight": 0, 2024-09-21T21:40:57.070 INFO:teuthology.orchestra.run.smithi081.stdout: "last_purged_snaps_scrub": "2024-09-21T21:40:15.766648+0000", 2024-09-21T21:40:57.070 INFO:teuthology.orchestra.run.smithi081.stdout: "dead_epoch": 0 2024-09-21T21:40:57.070 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:57.070 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:57.070 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": 1, 2024-09-21T21:40:57.070 INFO:teuthology.orchestra.run.smithi081.stdout: "down_stamp": "0.000000", 2024-09-21T21:40:57.070 INFO:teuthology.orchestra.run.smithi081.stdout: "laggy_probability": 0, 2024-09-21T21:40:57.071 INFO:teuthology.orchestra.run.smithi081.stdout: "laggy_interval": 0, 2024-09-21T21:40:57.071 INFO:teuthology.orchestra.run.smithi081.stdout: "features": 4540701547738038271, 2024-09-21T21:40:57.071 INFO:teuthology.orchestra.run.smithi081.stdout: "old_weight": 0, 2024-09-21T21:40:57.071 INFO:teuthology.orchestra.run.smithi081.stdout: "last_purged_snaps_scrub": "2024-09-21T21:40:15.457496+0000", 2024-09-21T21:40:57.071 INFO:teuthology.orchestra.run.smithi081.stdout: "dead_epoch": 0 2024-09-21T21:40:57.071 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:57.071 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:57.071 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": 2, 2024-09-21T21:40:57.071 INFO:teuthology.orchestra.run.smithi081.stdout: "down_stamp": "0.000000", 2024-09-21T21:40:57.071 INFO:teuthology.orchestra.run.smithi081.stdout: "laggy_probability": 0, 2024-09-21T21:40:57.071 INFO:teuthology.orchestra.run.smithi081.stdout: "laggy_interval": 0, 2024-09-21T21:40:57.071 INFO:teuthology.orchestra.run.smithi081.stdout: "features": 4540701547738038271, 2024-09-21T21:40:57.071 INFO:teuthology.orchestra.run.smithi081.stdout: "old_weight": 0, 2024-09-21T21:40:57.071 INFO:teuthology.orchestra.run.smithi081.stdout: "last_purged_snaps_scrub": "2024-09-21T21:40:14.767039+0000", 2024-09-21T21:40:57.071 INFO:teuthology.orchestra.run.smithi081.stdout: "dead_epoch": 0 2024-09-21T21:40:57.072 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:57.072 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:57.072 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": 3, 2024-09-21T21:40:57.072 INFO:teuthology.orchestra.run.smithi081.stdout: "down_stamp": "0.000000", 2024-09-21T21:40:57.072 INFO:teuthology.orchestra.run.smithi081.stdout: "laggy_probability": 0, 2024-09-21T21:40:57.072 INFO:teuthology.orchestra.run.smithi081.stdout: "laggy_interval": 0, 2024-09-21T21:40:57.072 INFO:teuthology.orchestra.run.smithi081.stdout: "features": 4540701547738038271, 2024-09-21T21:40:57.072 INFO:teuthology.orchestra.run.smithi081.stdout: "old_weight": 0, 2024-09-21T21:40:57.072 INFO:teuthology.orchestra.run.smithi081.stdout: "last_purged_snaps_scrub": "2024-09-21T21:40:15.355570+0000", 2024-09-21T21:40:57.072 INFO:teuthology.orchestra.run.smithi081.stdout: "dead_epoch": 0 2024-09-21T21:40:57.072 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:57.072 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:57.072 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": 4, 2024-09-21T21:40:57.072 INFO:teuthology.orchestra.run.smithi081.stdout: "down_stamp": "0.000000", 2024-09-21T21:40:57.073 INFO:teuthology.orchestra.run.smithi081.stdout: "laggy_probability": 0, 2024-09-21T21:40:57.073 INFO:teuthology.orchestra.run.smithi081.stdout: "laggy_interval": 0, 2024-09-21T21:40:57.073 INFO:teuthology.orchestra.run.smithi081.stdout: "features": 4540701547738038271, 2024-09-21T21:40:57.073 INFO:teuthology.orchestra.run.smithi081.stdout: "old_weight": 0, 2024-09-21T21:40:57.073 INFO:teuthology.orchestra.run.smithi081.stdout: "last_purged_snaps_scrub": "2024-09-21T21:40:15.335729+0000", 2024-09-21T21:40:57.073 INFO:teuthology.orchestra.run.smithi081.stdout: "dead_epoch": 0 2024-09-21T21:40:57.073 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:57.073 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:57.073 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": 5, 2024-09-21T21:40:57.073 INFO:teuthology.orchestra.run.smithi081.stdout: "down_stamp": "0.000000", 2024-09-21T21:40:57.073 INFO:teuthology.orchestra.run.smithi081.stdout: "laggy_probability": 0, 2024-09-21T21:40:57.073 INFO:teuthology.orchestra.run.smithi081.stdout: "laggy_interval": 0, 2024-09-21T21:40:57.073 INFO:teuthology.orchestra.run.smithi081.stdout: "features": 4540701547738038271, 2024-09-21T21:40:57.073 INFO:teuthology.orchestra.run.smithi081.stdout: "old_weight": 0, 2024-09-21T21:40:57.073 INFO:teuthology.orchestra.run.smithi081.stdout: "last_purged_snaps_scrub": "2024-09-21T21:40:15.614733+0000", 2024-09-21T21:40:57.073 INFO:teuthology.orchestra.run.smithi081.stdout: "dead_epoch": 0 2024-09-21T21:40:57.074 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:57.074 INFO:teuthology.orchestra.run.smithi081.stdout: ], 2024-09-21T21:40:57.074 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_upmap": [], 2024-09-21T21:40:57.074 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_upmap_items": [], 2024-09-21T21:40:57.074 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_upmap_primaries": [], 2024-09-21T21:40:57.074 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_temp": [], 2024-09-21T21:40:57.074 INFO:teuthology.orchestra.run.smithi081.stdout: "primary_temp": [], 2024-09-21T21:40:57.074 INFO:teuthology.orchestra.run.smithi081.stdout: "blocklist": {}, 2024-09-21T21:40:57.074 INFO:teuthology.orchestra.run.smithi081.stdout: "range_blocklist": {}, 2024-09-21T21:40:57.074 INFO:teuthology.orchestra.run.smithi081.stdout: "erasure_code_profiles": { 2024-09-21T21:40:57.074 INFO:teuthology.orchestra.run.smithi081.stdout: "default": { 2024-09-21T21:40:57.074 INFO:teuthology.orchestra.run.smithi081.stdout: "crush-failure-domain": "osd", 2024-09-21T21:40:57.074 INFO:teuthology.orchestra.run.smithi081.stdout: "k": "2", 2024-09-21T21:40:57.074 INFO:teuthology.orchestra.run.smithi081.stdout: "m": "1", 2024-09-21T21:40:57.074 INFO:teuthology.orchestra.run.smithi081.stdout: "plugin": "jerasure", 2024-09-21T21:40:57.075 INFO:teuthology.orchestra.run.smithi081.stdout: "technique": "reed_sol_van" 2024-09-21T21:40:57.075 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:57.075 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:57.075 INFO:teuthology.orchestra.run.smithi081.stdout: "removed_snaps_queue": [], 2024-09-21T21:40:57.075 INFO:teuthology.orchestra.run.smithi081.stdout: "new_removed_snaps": [], 2024-09-21T21:40:57.075 INFO:teuthology.orchestra.run.smithi081.stdout: "new_purged_snaps": [], 2024-09-21T21:40:57.075 INFO:teuthology.orchestra.run.smithi081.stdout: "crush_node_flags": {}, 2024-09-21T21:40:57.075 INFO:teuthology.orchestra.run.smithi081.stdout: "device_class_flags": {}, 2024-09-21T21:40:57.075 INFO:teuthology.orchestra.run.smithi081.stdout: "stretch_mode": { 2024-09-21T21:40:57.075 INFO:teuthology.orchestra.run.smithi081.stdout: "stretch_mode_enabled": false, 2024-09-21T21:40:57.075 INFO:teuthology.orchestra.run.smithi081.stdout: "stretch_bucket_count": 0, 2024-09-21T21:40:57.075 INFO:teuthology.orchestra.run.smithi081.stdout: "degraded_stretch_mode": 0, 2024-09-21T21:40:57.075 INFO:teuthology.orchestra.run.smithi081.stdout: "recovering_stretch_mode": 0, 2024-09-21T21:40:57.075 INFO:teuthology.orchestra.run.smithi081.stdout: "stretch_mode_bucket": 0 2024-09-21T21:40:57.076 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:57.076 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:40:57.077 INFO:tasks.thrashosds.thrasher:j is {'epoch': 34, 'fsid': 'a0fdcfc2-ddd2-4e50-81bb-876d940f3578', 'created': '2024-09-21T21:40:07.413917+0000', 'modified': '2024-09-21T21:40:55.499815+0000', 'last_up_change': '2024-09-21T21:40:18.340430+0000', 'last_in_change': '2024-09-21T21:40:09.029953+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T21:40:18.321328+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-21T21:40:24.235658+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 19, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-21T21:40:36.093691+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 9, 'snap_epoch': 34, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': 'a9869830-8e74-4b02-963d-ab630a0ee286', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 21, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6808', 'nonce': 1657552338}, {'type': 'v1', 'addr': '172.21.15.81:6809', 'nonce': 1657552338}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6810', 'nonce': 1657552338}, {'type': 'v1', 'addr': '172.21.15.81:6811', 'nonce': 1657552338}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6814', 'nonce': 1657552338}, {'type': 'v1', 'addr': '172.21.15.81:6815', 'nonce': 1657552338}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6812', 'nonce': 1657552338}, {'type': 'v1', 'addr': '172.21.15.81:6813', 'nonce': 1657552338}]}, 'public_addr': '172.21.15.81:6809/1657552338', 'cluster_addr': '172.21.15.81:6811/1657552338', 'heartbeat_back_addr': '172.21.15.81:6815/1657552338', 'heartbeat_front_addr': '172.21.15.81:6813/1657552338', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'b189819d-69a3-4e95-96e7-96dc4809ac80', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 21, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6816', 'nonce': 2864159502}, {'type': 'v1', 'addr': '172.21.15.81:6817', 'nonce': 2864159502}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6818', 'nonce': 2864159502}, {'type': 'v1', 'addr': '172.21.15.81:6819', 'nonce': 2864159502}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6822', 'nonce': 2864159502}, {'type': 'v1', 'addr': '172.21.15.81:6823', 'nonce': 2864159502}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6820', 'nonce': 2864159502}, {'type': 'v1', 'addr': '172.21.15.81:6821', 'nonce': 2864159502}]}, 'public_addr': '172.21.15.81:6817/2864159502', 'cluster_addr': '172.21.15.81:6819/2864159502', 'heartbeat_back_addr': '172.21.15.81:6823/2864159502', 'heartbeat_front_addr': '172.21.15.81:6821/2864159502', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '8437b99a-1297-4903-ab66-95c5cba2b2bf', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 21, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6800', 'nonce': 3023160932}, {'type': 'v1', 'addr': '172.21.15.81:6801', 'nonce': 3023160932}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6802', 'nonce': 3023160932}, {'type': 'v1', 'addr': '172.21.15.81:6803', 'nonce': 3023160932}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6806', 'nonce': 3023160932}, {'type': 'v1', 'addr': '172.21.15.81:6807', 'nonce': 3023160932}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6804', 'nonce': 3023160932}, {'type': 'v1', 'addr': '172.21.15.81:6805', 'nonce': 3023160932}]}, 'public_addr': '172.21.15.81:6801/3023160932', 'cluster_addr': '172.21.15.81:6803/3023160932', 'heartbeat_back_addr': '172.21.15.81:6807/3023160932', 'heartbeat_front_addr': '172.21.15.81:6805/3023160932', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'e8848768-c1aa-445b-9554-dc6aa4eefc8d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 21, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6808', 'nonce': 2681016844}, {'type': 'v1', 'addr': '172.21.15.122:6809', 'nonce': 2681016844}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6810', 'nonce': 2681016844}, {'type': 'v1', 'addr': '172.21.15.122:6811', 'nonce': 2681016844}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6814', 'nonce': 2681016844}, {'type': 'v1', 'addr': '172.21.15.122:6815', 'nonce': 2681016844}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6812', 'nonce': 2681016844}, {'type': 'v1', 'addr': '172.21.15.122:6813', 'nonce': 2681016844}]}, 'public_addr': '172.21.15.122:6809/2681016844', 'cluster_addr': '172.21.15.122:6811/2681016844', 'heartbeat_back_addr': '172.21.15.122:6815/2681016844', 'heartbeat_front_addr': '172.21.15.122:6813/2681016844', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'b1d8e85d-a207-4ba6-9322-61d07812083e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 21, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6800', 'nonce': 1095023540}, {'type': 'v1', 'addr': '172.21.15.122:6801', 'nonce': 1095023540}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6802', 'nonce': 1095023540}, {'type': 'v1', 'addr': '172.21.15.122:6803', 'nonce': 1095023540}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6806', 'nonce': 1095023540}, {'type': 'v1', 'addr': '172.21.15.122:6807', 'nonce': 1095023540}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6804', 'nonce': 1095023540}, {'type': 'v1', 'addr': '172.21.15.122:6805', 'nonce': 1095023540}]}, 'public_addr': '172.21.15.122:6801/1095023540', 'cluster_addr': '172.21.15.122:6803/1095023540', 'heartbeat_back_addr': '172.21.15.122:6807/1095023540', 'heartbeat_front_addr': '172.21.15.122:6805/1095023540', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '3ab0450b-d8fd-4ef2-a298-08d5aa2ea32f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 21, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6816', 'nonce': 1974398333}, {'type': 'v1', 'addr': '172.21.15.122:6817', 'nonce': 1974398333}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6818', 'nonce': 1974398333}, {'type': 'v1', 'addr': '172.21.15.122:6819', 'nonce': 1974398333}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6822', 'nonce': 1974398333}, {'type': 'v1', 'addr': '172.21.15.122:6823', 'nonce': 1974398333}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6820', 'nonce': 1974398333}, {'type': 'v1', 'addr': '172.21.15.122:6821', 'nonce': 1974398333}]}, 'public_addr': '172.21.15.122:6817/1974398333', 'cluster_addr': '172.21.15.122:6819/1974398333', 'heartbeat_back_addr': '172.21.15.122:6823/1974398333', 'heartbeat_front_addr': '172.21.15.122:6821/1974398333', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T21:40:15.766648+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T21:40:15.457496+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T21:40:14.767039+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T21:40:15.355570+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T21:40:15.335729+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T21:40:15.614733+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-21T21:40:57.078 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T21:40:57.078 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:40:57.075+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:40:57.111 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:40:57.111 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:40:57.111 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:40:57.111 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:40:57.111 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:40:57.120 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T21:40:57.149 INFO:teuthology.orchestra.run.smithi164.stdout:989 write 0x751d400 thru 0x752a2e9 (0xceea bytes) 2024-09-21T21:40:57.150 INFO:teuthology.orchestra.run.smithi164.stdout:990 punch from 0x6b04e00 to 0x6b0cc00, (0x7e00 bytes) 2024-09-21T21:40:57.151 INFO:teuthology.orchestra.run.smithi164.stdout:991 read 0x377c00 thru 0x383420 (0xb821 bytes) 2024-09-21T21:40:57.153 INFO:teuthology.orchestra.run.smithi164.stdout:994 write 0xa34400 thru 0xa3c28e (0x7e8f bytes) 2024-09-21T21:40:57.156 INFO:teuthology.orchestra.run.smithi164.stdout:997 trunc from 0x752a400 to 0x55c1a00 2024-09-21T21:40:57.170 INFO:teuthology.orchestra.run.smithi164.stdout:998 punch from 0x2743200 to 0x274a000, (0x6e00 bytes) 2024-09-21T21:40:57.175 INFO:teuthology.orchestra.run.smithi164.stdout:1002 trunc from 0x55c1a00 to 0x3266c00 2024-09-21T21:40:57.178 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:40:57.176+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:40:57.189 INFO:teuthology.orchestra.run.smithi164.stdout:1003 read 0x139e800 thru 0x13a4185 (0x5986 bytes) 2024-09-21T21:40:57.191 INFO:teuthology.orchestra.run.smithi164.stdout:1011 read 0x2b9600 thru 0x2bf825 (0x6226 bytes) 2024-09-21T21:40:57.200 INFO:teuthology.orchestra.run.smithi164.stdout:1012 trunc from 0x3266c00 to 0xa5a7a00 2024-09-21T21:40:57.215 INFO:teuthology.orchestra.run.smithi164.stdout:1013 write 0x2efcc00 thru 0x2f093ca (0xc7cb bytes) 2024-09-21T21:40:57.226 INFO:teuthology.orchestra.run.smithi164.stdout:1014 trunc from 0xa5a7a00 to 0x52d400 2024-09-21T21:40:57.239 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:40:57.239 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-21T21:40:57.239 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-21T21:40:57.239 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:40:57.239 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.239 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4417.0:117 3.9 3:92506b0a:::rbd_data.11214be7c411.000000000000001c:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:57.240 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.534624+0000", 2024-09-21T21:40:57.240 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.702071831, 2024-09-21T21:40:57.240 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0047323440000000003, 2024-09-21T21:40:57.240 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:57.240 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:57.240 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:40:57.240 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:57.240 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4417", 2024-09-21T21:40:57.240 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:57.240 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 117 2024-09-21T21:40:57.240 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.240 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:57.240 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.240 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:57.240 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.534624+0000", 2024-09-21T21:40:57.241 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:57.241 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.241 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.241 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:57.241 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.534624+0000", 2024-09-21T21:40:57.241 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:57.241 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.241 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.241 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:57.241 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.534625+0000", 2024-09-21T21:40:57.241 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.61e-07 2024-09-21T21:40:57.241 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.241 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.241 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:57.241 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.534629+0000", 2024-09-21T21:40:57.242 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9639999999999997e-06 2024-09-21T21:40:57.242 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.242 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.242 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:57.242 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.534638+0000", 2024-09-21T21:40:57.242 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8710000000000003e-06 2024-09-21T21:40:57.242 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.242 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.242 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:57.242 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.534644+0000", 2024-09-21T21:40:57.242 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5590000000000002e-06 2024-09-21T21:40:57.242 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.242 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.242 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:57.243 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.534681+0000", 2024-09-21T21:40:57.243 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6504999999999997e-05 2024-09-21T21:40:57.243 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.243 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.243 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:57.243 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.534762+0000", 2024-09-21T21:40:57.243 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0729000000000006e-05 2024-09-21T21:40:57.243 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.243 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.243 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:57.243 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.539357+0000", 2024-09-21T21:40:57.243 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0045950549999999998 2024-09-21T21:40:57.243 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:57.243 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:57.243 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:57.244 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.244 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.244 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4417.0:162 3.9 3:92506b0a:::rbd_data.11214be7c411.000000000000001c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:57.244 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.539532+0000", 2024-09-21T21:40:57.244 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.697164251, 2024-09-21T21:40:57.244 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0056942770000000002, 2024-09-21T21:40:57.244 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:57.244 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:57.244 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:57.244 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:57.244 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4417", 2024-09-21T21:40:57.244 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:57.244 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 162 2024-09-21T21:40:57.244 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.245 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:57.245 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.245 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:57.245 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.539532+0000", 2024-09-21T21:40:57.245 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:57.245 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.245 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.245 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:57.245 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.539532+0000", 2024-09-21T21:40:57.245 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:57.245 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.245 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.246 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:57.246 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.539533+0000", 2024-09-21T21:40:57.246 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0610000000000001e-06 2024-09-21T21:40:57.246 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.246 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.246 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:57.246 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.539538+0000", 2024-09-21T21:40:57.246 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9949999999999996e-06 2024-09-21T21:40:57.246 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.246 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.246 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:57.246 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.539552+0000", 2024-09-21T21:40:57.246 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3823e-05 2024-09-21T21:40:57.246 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.246 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.246 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:57.247 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.539562+0000", 2024-09-21T21:40:57.247 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0011e-05 2024-09-21T21:40:57.247 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.247 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.247 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:57.247 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.539638+0000", 2024-09-21T21:40:57.247 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5912000000000001e-05 2024-09-21T21:40:57.247 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.247 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.247 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:57.247 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.539738+0000", 2024-09-21T21:40:57.247 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000100086 2024-09-21T21:40:57.247 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.247 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.247 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 2", 2024-09-21T21:40:57.248 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.539920+0000", 2024-09-21T21:40:57.248 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00018205399999999999 2024-09-21T21:40:57.248 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.248 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.248 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:57.248 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.540794+0000", 2024-09-21T21:40:57.248 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00087363399999999998 2024-09-21T21:40:57.248 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.248 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.248 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:57.248 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.545130+0000", 2024-09-21T21:40:57.248 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0043360940000000004 2024-09-21T21:40:57.248 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.248 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.248 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:57.249 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.545167+0000", 2024-09-21T21:40:57.249 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7069e-05 2024-09-21T21:40:57.249 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.249 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.249 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:57.249 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.545226+0000", 2024-09-21T21:40:57.249 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9537999999999998e-05 2024-09-21T21:40:57.249 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:57.249 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:57.249 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:57.249 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.249 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.249 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4417.0:163 3.9 3:9336cf29:::rbd_data.11214be7c411.0000000000000026:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:57.249 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.539786+0000", 2024-09-21T21:40:57.250 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.696909861, 2024-09-21T21:40:57.250 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0055684869999999996, 2024-09-21T21:40:57.250 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:57.250 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:57.250 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:57.250 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:57.250 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4417", 2024-09-21T21:40:57.250 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:57.250 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 163 2024-09-21T21:40:57.250 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.250 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:57.250 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.250 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:57.250 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.539786+0000", 2024-09-21T21:40:57.251 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:57.251 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.251 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.251 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:57.251 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.539786+0000", 2024-09-21T21:40:57.251 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:57.251 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.251 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.251 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:57.251 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.539787+0000", 2024-09-21T21:40:57.251 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8500000000000001e-07 2024-09-21T21:40:57.252 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.252 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.252 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:57.252 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.539794+0000", 2024-09-21T21:40:57.252 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9190000000000002e-06 2024-09-21T21:40:57.252 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.252 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.252 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:57.252 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.539804+0000", 2024-09-21T21:40:57.252 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0054999999999999e-05 2024-09-21T21:40:57.252 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.252 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.252 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:57.252 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.539811+0000", 2024-09-21T21:40:57.252 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3150000000000003e-06 2024-09-21T21:40:57.253 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.253 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.253 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:57.253 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.540229+0000", 2024-09-21T21:40:57.253 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00041778099999999999 2024-09-21T21:40:57.253 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.253 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.253 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:57.253 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.540314+0000", 2024-09-21T21:40:57.253 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.4640000000000003e-05 2024-09-21T21:40:57.253 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.253 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.253 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 2", 2024-09-21T21:40:57.254 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.540528+0000", 2024-09-21T21:40:57.254 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00021472400000000001 2024-09-21T21:40:57.254 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.254 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.254 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:57.254 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.541998+0000", 2024-09-21T21:40:57.254 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001469087 2024-09-21T21:40:57.254 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.254 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.254 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:57.254 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.545266+0000", 2024-09-21T21:40:57.254 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0032689289999999998 2024-09-21T21:40:57.254 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.254 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.255 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:57.255 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.545319+0000", 2024-09-21T21:40:57.255 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2148999999999998e-05 2024-09-21T21:40:57.255 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.255 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.255 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:57.255 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.545355+0000", 2024-09-21T21:40:57.255 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6202999999999999e-05 2024-09-21T21:40:57.255 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:57.255 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:57.255 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:57.255 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.255 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.255 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4417.0:178 3.3 3:c3470000:::rbd_data.11214be7c411.000000000000002e:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:57.256 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.545144+0000", 2024-09-21T21:40:57.256 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.691551846999999, 2024-09-21T21:40:57.256 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0054365009999999998, 2024-09-21T21:40:57.256 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:57.256 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:57.256 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:40:57.256 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:57.256 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4417", 2024-09-21T21:40:57.256 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:57.256 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 178 2024-09-21T21:40:57.256 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.256 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:57.256 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.256 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:57.256 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.545144+0000", 2024-09-21T21:40:57.256 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:57.257 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.257 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.257 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:57.257 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.545144+0000", 2024-09-21T21:40:57.257 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:57.257 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.257 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.257 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:57.257 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.545145+0000", 2024-09-21T21:40:57.257 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0800000000000004e-07 2024-09-21T21:40:57.257 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.257 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.257 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:57.257 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.545149+0000", 2024-09-21T21:40:57.257 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7409999999999998e-06 2024-09-21T21:40:57.258 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.258 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.258 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:57.258 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.545159+0000", 2024-09-21T21:40:57.258 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0078000000000001e-05 2024-09-21T21:40:57.258 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.258 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.258 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:57.258 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.545165+0000", 2024-09-21T21:40:57.258 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1979999999999998e-06 2024-09-21T21:40:57.258 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.258 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.258 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:57.259 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.545222+0000", 2024-09-21T21:40:57.259 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6999000000000001e-05 2024-09-21T21:40:57.259 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.259 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.259 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:57.259 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.545326+0000", 2024-09-21T21:40:57.259 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000104168 2024-09-21T21:40:57.259 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.259 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.259 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:57.259 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.550581+0000", 2024-09-21T21:40:57.259 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0052545090000000001 2024-09-21T21:40:57.259 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:57.259 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:57.259 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:57.260 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.260 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.260 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4417.0:198 3.1a 3:5ab0459a:::rbd_data.11214be7c411.0000000000000032:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:57.260 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.547904+0000", 2024-09-21T21:40:57.260 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.688792123000001, 2024-09-21T21:40:57.260 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.005385679, 2024-09-21T21:40:57.260 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:57.260 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:57.260 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:40:57.260 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:57.260 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4417", 2024-09-21T21:40:57.260 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:57.260 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 198 2024-09-21T21:40:57.260 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.261 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:57.261 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.261 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:57.261 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.547904+0000", 2024-09-21T21:40:57.261 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:57.261 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.261 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.261 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:57.261 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.547904+0000", 2024-09-21T21:40:57.261 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:57.261 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.261 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.261 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:57.261 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.547905+0000", 2024-09-21T21:40:57.261 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0400000000000005e-07 2024-09-21T21:40:57.261 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.262 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.262 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:57.262 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.547909+0000", 2024-09-21T21:40:57.262 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8650000000000003e-06 2024-09-21T21:40:57.262 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.262 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.262 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:57.262 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.547919+0000", 2024-09-21T21:40:57.262 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0134e-05 2024-09-21T21:40:57.262 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.262 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.262 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:57.262 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.547926+0000", 2024-09-21T21:40:57.263 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7880000000000001e-06 2024-09-21T21:40:57.263 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.263 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.263 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:57.263 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.548144+0000", 2024-09-21T21:40:57.263 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000218598 2024-09-21T21:40:57.263 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.263 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.263 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:57.263 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.548278+0000", 2024-09-21T21:40:57.263 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000134213 2024-09-21T21:40:57.263 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.263 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.263 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:57.264 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.553290+0000", 2024-09-21T21:40:57.264 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0050112769999999997 2024-09-21T21:40:57.264 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:57.264 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:57.264 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:57.264 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.264 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.264 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4417.0:219 3.3 3:c3470000:::rbd_data.11214be7c411.000000000000002e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:57.264 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.550734+0000", 2024-09-21T21:40:57.264 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.685961938, 2024-09-21T21:40:57.264 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0053111310000000002, 2024-09-21T21:40:57.264 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:57.264 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:57.264 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:57.264 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:57.265 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4417", 2024-09-21T21:40:57.265 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:57.265 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 219 2024-09-21T21:40:57.265 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.265 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:57.265 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.265 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:57.265 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.550734+0000", 2024-09-21T21:40:57.265 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:57.265 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.265 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.265 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:57.265 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.550734+0000", 2024-09-21T21:40:57.265 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:57.265 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.265 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.266 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:57.266 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.550736+0000", 2024-09-21T21:40:57.266 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.297e-06 2024-09-21T21:40:57.266 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.266 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.266 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:57.266 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.550741+0000", 2024-09-21T21:40:57.266 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2669999999999997e-06 2024-09-21T21:40:57.266 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.266 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.266 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:57.266 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.550754+0000", 2024-09-21T21:40:57.266 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2996000000000001e-05 2024-09-21T21:40:57.266 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.266 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.267 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:57.267 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.550761+0000", 2024-09-21T21:40:57.267 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1760000000000004e-06 2024-09-21T21:40:57.267 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.267 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.267 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:57.267 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.550801+0000", 2024-09-21T21:40:57.267 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0185999999999998e-05 2024-09-21T21:40:57.267 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.267 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.267 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:57.267 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.550871+0000", 2024-09-21T21:40:57.267 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0018999999999995e-05 2024-09-21T21:40:57.268 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.268 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.268 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:40:57.268 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.550999+0000", 2024-09-21T21:40:57.268 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012815099999999999 2024-09-21T21:40:57.268 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.268 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.268 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:57.268 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.551530+0000", 2024-09-21T21:40:57.268 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00053112500000000002 2024-09-21T21:40:57.268 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.268 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.268 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:57.268 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.555975+0000", 2024-09-21T21:40:57.268 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0044446759999999998 2024-09-21T21:40:57.268 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.269 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.269 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:57.269 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.556012+0000", 2024-09-21T21:40:57.269 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6536999999999999e-05 2024-09-21T21:40:57.269 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.269 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.269 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:57.269 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.556045+0000", 2024-09-21T21:40:57.269 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3701000000000001e-05 2024-09-21T21:40:57.269 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:57.269 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:57.269 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:57.269 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.269 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.270 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(client.4417.0:305 3.14 e26/21 3:2aa90881:::rbd_data.11214be7c411.0000000000000046:head v 26'6, mlcod=26'4)", 2024-09-21T21:40:57.270 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.569452+0000", 2024-09-21T21:40:57.270 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.6672438, 2024-09-21T21:40:57.270 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0047599219999999998, 2024-09-21T21:40:57.270 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:57.270 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:57.270 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:57.270 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:57.270 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.270 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:57.270 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.569452+0000", 2024-09-21T21:40:57.270 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:57.270 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.270 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.270 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:57.271 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.569452+0000", 2024-09-21T21:40:57.271 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:57.271 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.271 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.271 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:57.271 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.569453+0000", 2024-09-21T21:40:57.271 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5300000000000003e-07 2024-09-21T21:40:57.271 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.271 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.271 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:57.271 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.569469+0000", 2024-09-21T21:40:57.271 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5410999999999999e-05 2024-09-21T21:40:57.271 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.271 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.271 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:57.272 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.569481+0000", 2024-09-21T21:40:57.272 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2391e-05 2024-09-21T21:40:57.272 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.272 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.272 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:57.272 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.569488+0000", 2024-09-21T21:40:57.272 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5679999999999996e-06 2024-09-21T21:40:57.272 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.272 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.272 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:57.272 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.569537+0000", 2024-09-21T21:40:57.272 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9406000000000001e-05 2024-09-21T21:40:57.272 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.272 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.272 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:57.273 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.569556+0000", 2024-09-21T21:40:57.273 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9264999999999999e-05 2024-09-21T21:40:57.273 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.273 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.273 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:57.273 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.574149+0000", 2024-09-21T21:40:57.273 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0045932780000000001 2024-09-21T21:40:57.273 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.273 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.273 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:57.273 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.574212+0000", 2024-09-21T21:40:57.273 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2849999999999996e-05 2024-09-21T21:40:57.273 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:57.273 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:57.273 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:57.273 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.274 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.274 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4417.0:318 3.3 3:c6040d19:::rbd_data.11214be7c411.000000000000004e:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:57.274 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.571797+0000", 2024-09-21T21:40:57.274 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.664898837999999, 2024-09-21T21:40:57.274 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0047787790000000004, 2024-09-21T21:40:57.274 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:57.274 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:57.274 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:40:57.274 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:57.274 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4417", 2024-09-21T21:40:57.274 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:57.274 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 318 2024-09-21T21:40:57.274 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.274 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:57.274 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.274 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:57.275 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.571797+0000", 2024-09-21T21:40:57.275 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:57.275 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.275 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.275 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:57.275 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.571797+0000", 2024-09-21T21:40:57.275 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:57.275 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.275 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.275 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:57.275 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.571798+0000", 2024-09-21T21:40:57.275 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.7600000000000006e-07 2024-09-21T21:40:57.275 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.275 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.276 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:57.276 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.571803+0000", 2024-09-21T21:40:57.276 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2169999999999996e-06 2024-09-21T21:40:57.276 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.276 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.276 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:57.276 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.571811+0000", 2024-09-21T21:40:57.276 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9969999999999998e-06 2024-09-21T21:40:57.276 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.276 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.276 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:57.276 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.571815+0000", 2024-09-21T21:40:57.276 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7960000000000003e-06 2024-09-21T21:40:57.276 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.276 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.276 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:57.277 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.571867+0000", 2024-09-21T21:40:57.277 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1802000000000003e-05 2024-09-21T21:40:57.277 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.277 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.277 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:57.277 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.572003+0000", 2024-09-21T21:40:57.277 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013558600000000001 2024-09-21T21:40:57.277 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.277 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.277 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:57.277 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.576576+0000", 2024-09-21T21:40:57.277 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.004573405 2024-09-21T21:40:57.277 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:57.277 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:57.277 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:57.277 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.278 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.278 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4417.0:323 3.9 3:914d0620:::rbd_data.11214be7c411.0000000000000050:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:57.278 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.571952+0000", 2024-09-21T21:40:57.278 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.664744649999999, 2024-09-21T21:40:57.278 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.004747472, 2024-09-21T21:40:57.278 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:57.278 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:57.278 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:40:57.278 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:57.278 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4417", 2024-09-21T21:40:57.278 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:57.278 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 323 2024-09-21T21:40:57.278 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.278 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:57.278 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.279 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:57.279 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.571952+0000", 2024-09-21T21:40:57.279 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:57.279 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.279 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.279 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:57.279 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.571952+0000", 2024-09-21T21:40:57.279 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:57.279 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.279 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.279 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:57.279 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.571953+0000", 2024-09-21T21:40:57.279 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.017e-06 2024-09-21T21:40:57.279 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.280 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.280 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:57.280 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.571958+0000", 2024-09-21T21:40:57.280 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7939999999999999e-06 2024-09-21T21:40:57.280 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.280 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.280 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:57.280 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.571970+0000", 2024-09-21T21:40:57.280 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1250000000000001e-05 2024-09-21T21:40:57.280 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.280 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.280 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:57.280 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.571977+0000", 2024-09-21T21:40:57.280 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2690000000000002e-06 2024-09-21T21:40:57.280 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.281 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.281 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:57.281 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.572038+0000", 2024-09-21T21:40:57.281 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0868000000000001e-05 2024-09-21T21:40:57.281 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.281 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.281 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:57.281 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.572160+0000", 2024-09-21T21:40:57.281 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000122633 2024-09-21T21:40:57.281 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.281 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.281 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:57.281 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.576699+0000", 2024-09-21T21:40:57.281 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0045386410000000004 2024-09-21T21:40:57.281 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:57.281 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:57.281 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:57.282 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.282 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.282 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4417.0:347 3.3 3:c6040d19:::rbd_data.11214be7c411.000000000000004e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:57.282 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.576836+0000", 2024-09-21T21:40:57.282 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.659860254, 2024-09-21T21:40:57.282 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0058561289999999998, 2024-09-21T21:40:57.282 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:57.282 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:57.282 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:57.282 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:57.282 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4417", 2024-09-21T21:40:57.282 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:57.282 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 347 2024-09-21T21:40:57.282 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.283 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:57.283 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.283 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:57.283 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.576836+0000", 2024-09-21T21:40:57.283 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:57.283 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.283 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.283 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:57.283 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.576836+0000", 2024-09-21T21:40:57.283 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:57.283 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.283 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.283 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:57.283 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.576837+0000", 2024-09-21T21:40:57.283 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.44e-07 2024-09-21T21:40:57.283 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.283 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.284 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:57.284 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.576840+0000", 2024-09-21T21:40:57.284 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1949999999999999e-06 2024-09-21T21:40:57.284 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.284 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.284 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:57.284 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.576847+0000", 2024-09-21T21:40:57.284 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1040000000000001e-06 2024-09-21T21:40:57.284 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.284 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.284 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:57.284 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.576851+0000", 2024-09-21T21:40:57.284 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.053e-06 2024-09-21T21:40:57.284 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.284 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.285 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:57.285 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.576893+0000", 2024-09-21T21:40:57.285 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1906000000000002e-05 2024-09-21T21:40:57.285 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.285 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.285 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:57.285 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.576956+0000", 2024-09-21T21:40:57.285 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3417999999999995e-05 2024-09-21T21:40:57.285 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.285 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.285 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:40:57.285 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.577175+0000", 2024-09-21T21:40:57.285 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000218381 2024-09-21T21:40:57.285 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.285 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.286 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:57.286 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.579572+0000", 2024-09-21T21:40:57.286 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0023974629999999999 2024-09-21T21:40:57.286 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.286 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.286 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:57.286 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.582549+0000", 2024-09-21T21:40:57.286 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0029770389999999999 2024-09-21T21:40:57.286 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.286 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.286 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:57.286 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.582650+0000", 2024-09-21T21:40:57.286 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010091599999999999 2024-09-21T21:40:57.287 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.287 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.287 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:57.287 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.582692+0000", 2024-09-21T21:40:57.287 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2009999999999999e-05 2024-09-21T21:40:57.287 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:57.287 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:57.287 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:57.287 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.287 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.287 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4417.0:349 3.9 3:914d0620:::rbd_data.11214be7c411.0000000000000050:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:57.287 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.576967+0000", 2024-09-21T21:40:57.287 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.659729338, 2024-09-21T21:40:57.287 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0057303759999999997, 2024-09-21T21:40:57.288 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:57.288 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:57.288 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:57.288 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:57.288 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4417", 2024-09-21T21:40:57.288 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:57.288 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 349 2024-09-21T21:40:57.288 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.288 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:57.288 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.288 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:57.288 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.576967+0000", 2024-09-21T21:40:57.288 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:57.288 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.288 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.289 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:57.289 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.576967+0000", 2024-09-21T21:40:57.289 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:57.289 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.289 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.289 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:57.289 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.576968+0000", 2024-09-21T21:40:57.289 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.37e-07 2024-09-21T21:40:57.289 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.289 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.289 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:57.289 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.576972+0000", 2024-09-21T21:40:57.289 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9580000000000001e-06 2024-09-21T21:40:57.289 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.289 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.290 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:57.290 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.576981+0000", 2024-09-21T21:40:57.290 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9930000000000001e-06 2024-09-21T21:40:57.290 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.290 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.290 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:57.290 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.576987+0000", 2024-09-21T21:40:57.290 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7789999999999998e-06 2024-09-21T21:40:57.290 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.290 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.290 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:57.290 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.577032+0000", 2024-09-21T21:40:57.290 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4409999999999997e-05 2024-09-21T21:40:57.291 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.291 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.291 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:57.291 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.577125+0000", 2024-09-21T21:40:57.291 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3690000000000006e-05 2024-09-21T21:40:57.291 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.291 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.291 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 2", 2024-09-21T21:40:57.291 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.577359+0000", 2024-09-21T21:40:57.291 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00023399400000000001 2024-09-21T21:40:57.291 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.291 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.291 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:57.292 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.582518+0000", 2024-09-21T21:40:57.292 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0051590530000000002 2024-09-21T21:40:57.292 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.292 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.292 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:57.292 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.582556+0000", 2024-09-21T21:40:57.292 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7908999999999999e-05 2024-09-21T21:40:57.292 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.292 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.292 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:57.292 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.582646+0000", 2024-09-21T21:40:57.292 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9448000000000001e-05 2024-09-21T21:40:57.293 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.293 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.293 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:57.293 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.582697+0000", 2024-09-21T21:40:57.293 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1405e-05 2024-09-21T21:40:57.293 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:57.293 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:57.293 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:57.293 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.293 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.293 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(client.4417.0:344 3.1b e26/21 3:d9520152:::rbd_data.11214be7c411.000000000000004d:head v 26'11, mlcod=26'6)", 2024-09-21T21:40:57.293 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.577197+0000", 2024-09-21T21:40:57.293 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.659499255, 2024-09-21T21:40:57.294 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0052244559999999997, 2024-09-21T21:40:57.294 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:57.294 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:57.294 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:57.294 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:57.294 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.294 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:57.294 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.577197+0000", 2024-09-21T21:40:57.294 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:57.294 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.294 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.294 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:57.295 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.577197+0000", 2024-09-21T21:40:57.295 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:57.295 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.295 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.295 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:57.295 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.577198+0000", 2024-09-21T21:40:57.295 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5599999999999995e-07 2024-09-21T21:40:57.295 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.295 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.295 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:57.295 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.577202+0000", 2024-09-21T21:40:57.295 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0249999999999996e-06 2024-09-21T21:40:57.295 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.296 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.296 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:57.296 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.577209+0000", 2024-09-21T21:40:57.296 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.216e-06 2024-09-21T21:40:57.296 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.296 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.296 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:57.296 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.577213+0000", 2024-09-21T21:40:57.296 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3649999999999997e-06 2024-09-21T21:40:57.296 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.296 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.296 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:57.296 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.577253+0000", 2024-09-21T21:40:57.297 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9597000000000001e-05 2024-09-21T21:40:57.297 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.297 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.297 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:57.297 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.577271+0000", 2024-09-21T21:40:57.297 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7978e-05 2024-09-21T21:40:57.297 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.297 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.297 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:57.297 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.582313+0000", 2024-09-21T21:40:57.297 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0050418449999999997 2024-09-21T21:40:57.297 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.297 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.298 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:57.298 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.582421+0000", 2024-09-21T21:40:57.298 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000108874 2024-09-21T21:40:57.298 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:57.298 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:57.298 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:57.298 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.298 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.298 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4417.0:354 3.1a 3:5c433628:::rbd_data.11214be7c411.0000000000000056:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:57.298 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.577895+0000", 2024-09-21T21:40:57.298 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.658801006999999, 2024-09-21T21:40:57.298 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0047106719999999999, 2024-09-21T21:40:57.298 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:57.298 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:57.299 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:40:57.299 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:57.299 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4417", 2024-09-21T21:40:57.299 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:57.299 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 354 2024-09-21T21:40:57.299 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.299 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:57.299 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.299 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:57.299 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.577895+0000", 2024-09-21T21:40:57.299 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:57.299 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.299 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.299 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:57.300 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.577895+0000", 2024-09-21T21:40:57.300 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:57.300 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.300 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.300 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:57.300 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.577896+0000", 2024-09-21T21:40:57.300 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7899999999999998e-07 2024-09-21T21:40:57.300 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.300 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.300 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:57.300 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.577900+0000", 2024-09-21T21:40:57.300 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1180000000000002e-06 2024-09-21T21:40:57.300 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.300 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.301 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:57.301 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.577908+0000", 2024-09-21T21:40:57.301 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2460000000000003e-06 2024-09-21T21:40:57.301 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.301 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.301 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:57.301 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.577920+0000", 2024-09-21T21:40:57.301 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1425e-05 2024-09-21T21:40:57.301 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.301 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.301 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:57.301 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.577962+0000", 2024-09-21T21:40:57.301 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1885000000000003e-05 2024-09-21T21:40:57.301 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.302 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.302 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:57.302 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.578040+0000", 2024-09-21T21:40:57.302 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8274000000000003e-05 2024-09-21T21:40:57.302 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.302 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.302 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:57.302 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.582606+0000", 2024-09-21T21:40:57.302 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0045660450000000003 2024-09-21T21:40:57.302 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:57.302 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:57.302 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:57.302 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.303 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.303 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4417.0:361 3.9 3:96657d9e:::rbd_data.11214be7c411.0000000000000058:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:57.303 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.579658+0000", 2024-09-21T21:40:57.303 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.657038135000001, 2024-09-21T21:40:57.303 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0053728680000000003, 2024-09-21T21:40:57.303 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:57.303 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:57.303 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:40:57.303 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:57.303 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4417", 2024-09-21T21:40:57.303 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:57.303 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 361 2024-09-21T21:40:57.303 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.303 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:57.304 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.304 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:57.304 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.579658+0000", 2024-09-21T21:40:57.304 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:57.304 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.304 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.304 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:57.304 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.579658+0000", 2024-09-21T21:40:57.304 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:57.304 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.304 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.304 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:57.304 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.579659+0000", 2024-09-21T21:40:57.304 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7700000000000004e-07 2024-09-21T21:40:57.305 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.305 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.305 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:57.305 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.579664+0000", 2024-09-21T21:40:57.305 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2379999999999997e-06 2024-09-21T21:40:57.305 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.305 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.305 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:57.305 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.579673+0000", 2024-09-21T21:40:57.305 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3079999999999997e-06 2024-09-21T21:40:57.305 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.305 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.305 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:57.305 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.579689+0000", 2024-09-21T21:40:57.306 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.522e-05 2024-09-21T21:40:57.306 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.306 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.306 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:57.306 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.579741+0000", 2024-09-21T21:40:57.306 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2329e-05 2024-09-21T21:40:57.306 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.306 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.306 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:57.306 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.579839+0000", 2024-09-21T21:40:57.306 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.8172999999999997e-05 2024-09-21T21:40:57.306 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.306 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.306 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:57.307 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.585031+0000", 2024-09-21T21:40:57.307 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0051919230000000002 2024-09-21T21:40:57.307 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:57.307 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:57.307 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:57.307 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.307 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.307 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4417.0:438 3.9 3:9170f860:::rbd_data.11214be7c411.0000000000000067:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:57.307 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.590245+0000", 2024-09-21T21:40:57.307 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.646451166, 2024-09-21T21:40:57.307 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0050158700000000004, 2024-09-21T21:40:57.307 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:57.307 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:57.307 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:57.308 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:57.308 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4417", 2024-09-21T21:40:57.308 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:57.308 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 438 2024-09-21T21:40:57.308 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.308 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:57.308 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.308 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:57.308 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.590245+0000", 2024-09-21T21:40:57.308 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:57.308 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.308 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.308 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:57.308 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.590245+0000", 2024-09-21T21:40:57.309 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:57.309 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.309 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.309 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:57.309 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.590246+0000", 2024-09-21T21:40:57.309 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.44e-07 2024-09-21T21:40:57.309 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.309 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.309 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:57.309 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.590249+0000", 2024-09-21T21:40:57.309 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.811e-06 2024-09-21T21:40:57.309 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.309 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.309 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:57.309 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.590258+0000", 2024-09-21T21:40:57.310 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2349999999999999e-06 2024-09-21T21:40:57.310 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.310 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.310 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:57.310 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.590262+0000", 2024-09-21T21:40:57.310 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.775e-06 2024-09-21T21:40:57.310 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.310 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.310 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:57.310 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.590304+0000", 2024-09-21T21:40:57.310 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1910999999999999e-05 2024-09-21T21:40:57.310 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.310 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.310 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:57.310 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.590372+0000", 2024-09-21T21:40:57.311 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7961999999999997e-05 2024-09-21T21:40:57.311 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.311 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.311 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 2", 2024-09-21T21:40:57.311 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.590579+0000", 2024-09-21T21:40:57.311 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00020714400000000001 2024-09-21T21:40:57.311 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.311 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.311 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:57.311 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.593958+0000", 2024-09-21T21:40:57.311 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0033788949999999998 2024-09-21T21:40:57.311 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.311 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.311 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:57.311 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.595177+0000", 2024-09-21T21:40:57.311 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012186230000000001 2024-09-21T21:40:57.312 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.312 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.312 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:57.312 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.595224+0000", 2024-09-21T21:40:57.312 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7312999999999999e-05 2024-09-21T21:40:57.312 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.312 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.312 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:57.312 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.595261+0000", 2024-09-21T21:40:57.312 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6656999999999998e-05 2024-09-21T21:40:57.312 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:57.312 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:57.312 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:57.312 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.312 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.313 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4417.0:457 3.3 3:c632f711:::rbd_data.11214be7c411.000000000000006f:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:57.313 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.593047+0000", 2024-09-21T21:40:57.313 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.6436496, 2024-09-21T21:40:57.313 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0079810079999999995, 2024-09-21T21:40:57.313 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:57.313 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:57.313 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:40:57.313 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:57.313 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4417", 2024-09-21T21:40:57.313 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:57.313 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 457 2024-09-21T21:40:57.313 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.313 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:57.313 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.313 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:57.314 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.593047+0000", 2024-09-21T21:40:57.314 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:57.314 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.314 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.314 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:57.314 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.593047+0000", 2024-09-21T21:40:57.314 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:57.314 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.314 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.314 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:57.314 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.593047+0000", 2024-09-21T21:40:57.314 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8199999999999998e-07 2024-09-21T21:40:57.314 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.314 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.314 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:57.314 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.593052+0000", 2024-09-21T21:40:57.315 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0000000000000004e-06 2024-09-21T21:40:57.315 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.315 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.315 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:57.315 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.593066+0000", 2024-09-21T21:40:57.315 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3651e-05 2024-09-21T21:40:57.315 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.315 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.315 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:57.315 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.593074+0000", 2024-09-21T21:40:57.315 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6389999999999992e-06 2024-09-21T21:40:57.315 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.315 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.315 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:57.316 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.593149+0000", 2024-09-21T21:40:57.316 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4914999999999997e-05 2024-09-21T21:40:57.316 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.316 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.316 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:57.316 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.593286+0000", 2024-09-21T21:40:57.316 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013712800000000001 2024-09-21T21:40:57.316 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.316 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.316 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:57.316 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.601028+0000", 2024-09-21T21:40:57.316 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0077417930000000003 2024-09-21T21:40:57.316 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:57.316 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:57.316 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:57.317 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.317 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.317 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4417.0:471 3.9 3:9188e235:::rbd_data.11214be7c411.0000000000000071:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:57.317 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.597888+0000", 2024-09-21T21:40:57.317 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.638808496999999, 2024-09-21T21:40:57.317 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0061533930000000001, 2024-09-21T21:40:57.317 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:57.317 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:57.317 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:40:57.317 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:57.317 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4417", 2024-09-21T21:40:57.317 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:57.317 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 471 2024-09-21T21:40:57.318 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.318 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:57.318 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.318 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:57.318 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.597888+0000", 2024-09-21T21:40:57.318 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:57.318 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.318 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.318 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:57.318 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.597888+0000", 2024-09-21T21:40:57.318 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:57.318 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.318 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.318 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:57.319 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.597889+0000", 2024-09-21T21:40:57.319 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5199999999999995e-07 2024-09-21T21:40:57.319 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.319 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.319 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:57.319 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.597893+0000", 2024-09-21T21:40:57.319 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2189999999999998e-06 2024-09-21T21:40:57.319 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.319 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.319 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:57.319 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.597903+0000", 2024-09-21T21:40:57.319 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.8030000000000008e-06 2024-09-21T21:40:57.319 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.319 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.320 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:57.320 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.597908+0000", 2024-09-21T21:40:57.320 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3009999999999997e-06 2024-09-21T21:40:57.320 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.320 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.320 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:57.320 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.597948+0000", 2024-09-21T21:40:57.320 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9808999999999998e-05 2024-09-21T21:40:57.320 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.320 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.320 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:57.320 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.598015+0000", 2024-09-21T21:40:57.320 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7446000000000004e-05 2024-09-21T21:40:57.320 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.321 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.321 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:57.321 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.604041+0000", 2024-09-21T21:40:57.321 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0060258630000000002 2024-09-21T21:40:57.321 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:57.321 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:57.321 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:57.321 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.321 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.321 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4417.0:517 3.1a 3:5aa0c892:::rbd_data.11214be7c411.000000000000007a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:57.321 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:43.608399+0000", 2024-09-21T21:40:57.321 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.628296733999999, 2024-09-21T21:40:57.321 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0076266900000000002, 2024-09-21T21:40:57.321 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:57.321 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:57.321 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:57.322 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:57.322 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4417", 2024-09-21T21:40:57.322 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:57.322 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 517 2024-09-21T21:40:57.322 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.322 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:57.322 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.322 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:57.322 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.608399+0000", 2024-09-21T21:40:57.322 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:57.322 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.322 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.322 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:57.322 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.608399+0000", 2024-09-21T21:40:57.323 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:57.323 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.323 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.323 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:57.323 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.608400+0000", 2024-09-21T21:40:57.323 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6899999999999996e-07 2024-09-21T21:40:57.323 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.323 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.323 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:57.323 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.608404+0000", 2024-09-21T21:40:57.323 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6959999999999998e-06 2024-09-21T21:40:57.323 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.323 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.324 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:57.324 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.608412+0000", 2024-09-21T21:40:57.324 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2660000000000001e-06 2024-09-21T21:40:57.324 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.324 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.324 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:57.324 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.608420+0000", 2024-09-21T21:40:57.324 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4789999999999999e-06 2024-09-21T21:40:57.324 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.324 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.324 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:57.324 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.608465+0000", 2024-09-21T21:40:57.324 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4886999999999999e-05 2024-09-21T21:40:57.324 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.325 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.325 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:57.325 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.608516+0000", 2024-09-21T21:40:57.325 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1693999999999997e-05 2024-09-21T21:40:57.325 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.325 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.325 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2024-09-21T21:40:57.325 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.608626+0000", 2024-09-21T21:40:57.325 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010976600000000001 2024-09-21T21:40:57.325 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.325 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.325 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:40:57.325 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.608962+0000", 2024-09-21T21:40:57.325 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00033614300000000001 2024-09-21T21:40:57.325 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.325 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.326 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:57.326 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.615969+0000", 2024-09-21T21:40:57.326 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0070068509999999997 2024-09-21T21:40:57.326 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.326 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.326 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:57.326 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.616002+0000", 2024-09-21T21:40:57.326 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3243e-05 2024-09-21T21:40:57.326 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.326 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.326 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:57.326 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:43.616026+0000", 2024-09-21T21:40:57.326 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3796000000000001e-05 2024-09-21T21:40:57.326 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:57.326 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:57.327 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:57.327 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.327 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.327 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(client.4471.0:1191 3.a e28/21 3:50c18da3:::rbd_object_map.1121bf4f5ab3:head v 28'139, mlcod=28'135)", 2024-09-21T21:40:57.327 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:48.416347+0000", 2024-09-21T21:40:57.327 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 8.8203492249999993, 2024-09-21T21:40:57.327 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0055930069999999997, 2024-09-21T21:40:57.327 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:57.327 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:57.327 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:57.327 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:57.327 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.327 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:57.327 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.416347+0000", 2024-09-21T21:40:57.327 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:57.328 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.328 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.328 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:57.328 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.416347+0000", 2024-09-21T21:40:57.328 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:57.328 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.328 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.328 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:57.328 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.416348+0000", 2024-09-21T21:40:57.328 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9100000000000004e-07 2024-09-21T21:40:57.328 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.328 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.328 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:57.328 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.416353+0000", 2024-09-21T21:40:57.328 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5289999999999999e-06 2024-09-21T21:40:57.329 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.329 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.329 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:57.329 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.416362+0000", 2024-09-21T21:40:57.329 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3200000000000006e-06 2024-09-21T21:40:57.329 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.329 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.329 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:57.329 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.416369+0000", 2024-09-21T21:40:57.329 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2890000000000003e-06 2024-09-21T21:40:57.329 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.329 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.329 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:57.329 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.416431+0000", 2024-09-21T21:40:57.329 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2189999999999999e-05 2024-09-21T21:40:57.330 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.330 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.330 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:57.330 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.416448+0000", 2024-09-21T21:40:57.330 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6929e-05 2024-09-21T21:40:57.330 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.330 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.330 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:57.330 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.421901+0000", 2024-09-21T21:40:57.330 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0054528909999999996 2024-09-21T21:40:57.330 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.330 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.330 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:57.330 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.421940+0000", 2024-09-21T21:40:57.331 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9267999999999997e-05 2024-09-21T21:40:57.331 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:57.331 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:57.331 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:57.331 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.331 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.331 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(client.4471.0:1192 3.a e28/21 3:50c18da3:::rbd_object_map.1121bf4f5ab3:head v 28'140, mlcod=28'136)", 2024-09-21T21:40:57.331 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:48.417340+0000", 2024-09-21T21:40:57.331 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 8.8193559639999997, 2024-09-21T21:40:57.331 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0046842389999999998, 2024-09-21T21:40:57.331 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:57.331 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:57.331 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:57.331 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:57.331 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.332 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:57.332 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.417340+0000", 2024-09-21T21:40:57.332 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:57.332 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.332 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.332 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:57.332 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.417340+0000", 2024-09-21T21:40:57.332 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:57.332 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.332 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.332 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:57.332 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.417341+0000", 2024-09-21T21:40:57.332 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0699999999999996e-07 2024-09-21T21:40:57.333 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.333 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.333 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:57.333 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.417347+0000", 2024-09-21T21:40:57.333 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1639999999999998e-06 2024-09-21T21:40:57.333 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.333 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.333 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:57.333 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.417370+0000", 2024-09-21T21:40:57.333 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2526000000000001e-05 2024-09-21T21:40:57.333 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.333 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.334 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:57.334 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.417377+0000", 2024-09-21T21:40:57.334 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9199999999999998e-06 2024-09-21T21:40:57.334 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.334 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.334 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:57.334 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.417430+0000", 2024-09-21T21:40:57.334 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3060999999999999e-05 2024-09-21T21:40:57.334 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.334 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.334 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:57.334 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.417448+0000", 2024-09-21T21:40:57.334 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7836e-05 2024-09-21T21:40:57.335 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.335 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.335 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:40:57.335 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.421943+0000", 2024-09-21T21:40:57.335 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0044949639999999997 2024-09-21T21:40:57.335 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.335 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.335 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:57.335 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:48.422024+0000", 2024-09-21T21:40:57.335 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1960999999999997e-05 2024-09-21T21:40:57.335 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:57.335 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:57.335 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:57.335 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:57.336 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:57.336 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:40:57.338 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T21:40:57.338 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:40:57.335+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:40:57.338 INFO:teuthology.orchestra.run.smithi164.stdout:1017 read 0x1cce00 thru 0x1cdd27 (0xf28 bytes) 2024-09-21T21:40:57.338 INFO:teuthology.orchestra.run.smithi164.stdout:1018 trunc from 0x52d400 to 0x4589400 2024-09-21T21:40:57.338 INFO:teuthology.orchestra.run.smithi164.stdout:1019 write 0x99ed600 thru 0x99faf01 (0xd902 bytes) 2024-09-21T21:40:57.338 INFO:teuthology.orchestra.run.smithi164.stdout:1021 punch from 0x1fd6200 to 0x1fdd400, (0x7200 bytes) 2024-09-21T21:40:57.338 INFO:teuthology.orchestra.run.smithi164.stdout:1024 trunc from 0x99fb000 to 0x8f44800 2024-09-21T21:40:57.338 INFO:teuthology.orchestra.run.smithi164.stdout:1026 write 0x923d600 thru 0x9245d14 (0x8715 bytes) 2024-09-21T21:40:57.339 INFO:teuthology.orchestra.run.smithi164.stdout:1027 write 0xa70e000 thru 0xa711093 (0x3094 bytes) 2024-09-21T21:40:57.341 INFO:teuthology.orchestra.run.smithi164.stdout:1030 read 0x10ae800 thru 0x10b2b0d (0x430e bytes) 2024-09-21T21:40:57.342 INFO:teuthology.orchestra.run.smithi164.stdout:1031 clone 8 order 19 su 4096 sc 14 2024-09-21T21:40:57.407 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:40:57.408 INFO:teuthology.orchestra.run.smithi081.stderr:dumped all 2024-09-21T21:40:57.418 INFO:teuthology.orchestra.run.smithi081.stdout:{"pg_ready":true,"pg_map":{"version":49,"stamp":"2024-09-21T21:40:56.297679+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":142252456,"num_objects":241,"num_object_clones":19,"num_object_copies":482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":241,"num_whiteouts":107,"num_read":31100,"num_read_kb":125102,"num_write":2475,"num_write_kb":34421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2202,"ondisk_log_size":2202,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":192388,"kb_used_data":5216,"kb_used_omap":9,"kb_used_meta":186998,"kb_avail":566038652,"statfs":{"total":579820584960,"available":579623579648,"internally_reserved":0,"allocated":5341184,"data_stored":9061921,"data_compressed":900921,"data_compressed_allocated":1228800,"data_compressed_original":6564352,"omap_allocated":9568,"internal_metadata":191486624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":67653957,"num_objects":57,"num_object_clones":10,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":9,"num_read":15758,"num_read_kb":80981,"num_write":1111,"num_write_kb":10496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.811669"},"pg_stats":[{"pgid":"3.1f","version":"30'31","reported_seq":693,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T21:40:53.061617+0000","last_change":"2024-09-21T21:40:39.430005+0000","last_active":"2024-09-21T21:40:53.061617+0000","last_peered":"2024-09-21T21:40:53.061617+0000","last_clean":"2024-09-21T21:40:53.061617+0000","last_became_active":"2024-09-21T21:40:37.424213+0000","last_became_peered":"2024-09-21T21:40:37.424213+0000","last_unstale":"2024-09-21T21:40:53.061617+0000","last_undegraded":"2024-09-21T21:40:53.061617+0000","last_fullsized":"2024-09-21T21:40:53.061617+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:07:31.993538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010985019999999999,"stat_sum":{"num_bytes":3708432,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":2,"num_read":634,"num_read_kb":4345,"num_write":35,"num_write_kb":4174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"30'179","reported_seq":1321,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T21:40:51.081988+0000","last_change":"2024-09-21T21:40:39.416688+0000","last_active":"2024-09-21T21:40:51.081988+0000","last_peered":"2024-09-21T21:40:51.081988+0000","last_clean":"2024-09-21T21:40:51.081988+0000","last_became_active":"2024-09-21T21:40:37.417190+0000","last_became_peered":"2024-09-21T21:40:37.417190+0000","last_unstale":"2024-09-21T21:40:51.081988+0000","last_undegraded":"2024-09-21T21:40:51.081988+0000","last_fullsized":"2024-09-21T21:40:51.081988+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":179,"log_dups_size":0,"ondisk_log_size":179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:53:06.068056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042608099999999997,"stat_sum":{"num_bytes":989739,"num_objects":8,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":4,"num_read":1923,"num_read_kb":5393,"num_write":299,"num_write_kb":353,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"30'43","reported_seq":676,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T21:40:53.065328+0000","last_change":"2024-09-21T21:40:39.426039+0000","last_active":"2024-09-21T21:40:53.065328+0000","last_peered":"2024-09-21T21:40:53.065328+0000","last_clean":"2024-09-21T21:40:53.065328+0000","last_became_active":"2024-09-21T21:40:37.426441+0000","last_became_peered":"2024-09-21T21:40:37.426441+0000","last_unstale":"2024-09-21T21:40:53.065328+0000","last_undegraded":"2024-09-21T21:40:53.065328+0000","last_fullsized":"2024-09-21T21:40:53.065328+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:13:54.799527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001986561,"stat_sum":{"num_bytes":2158080,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":4,"num_read":600,"num_read_kb":4656,"num_write":39,"num_write_kb":315,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"32'107","reported_seq":1157,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T21:40:53.305681+0000","last_change":"2024-09-21T21:40:39.424788+0000","last_active":"2024-09-21T21:40:53.305681+0000","last_peered":"2024-09-21T21:40:53.305681+0000","last_clean":"2024-09-21T21:40:53.305681+0000","last_became_active":"2024-09-21T21:40:37.426690+0000","last_became_peered":"2024-09-21T21:40:37.426690+0000","last_unstale":"2024-09-21T21:40:53.305681+0000","last_undegraded":"2024-09-21T21:40:53.305681+0000","last_fullsized":"2024-09-21T21:40:53.305681+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:02:41.760104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011105570000000001,"stat_sum":{"num_bytes":3958820,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":2,"num_read":1512,"num_read_kb":3720,"num_write":114,"num_write_kb":335,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"32'33","reported_seq":859,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T21:40:53.166008+0000","last_change":"2024-09-21T21:40:39.429478+0000","last_active":"2024-09-21T21:40:53.166008+0000","last_peered":"2024-09-21T21:40:53.166008+0000","last_clean":"2024-09-21T21:40:53.166008+0000","last_became_active":"2024-09-21T21:40:37.425925+0000","last_became_peered":"2024-09-21T21:40:37.425925+0000","last_unstale":"2024-09-21T21:40:53.166008+0000","last_undegraded":"2024-09-21T21:40:53.166008+0000","last_fullsized":"2024-09-21T21:40:53.166008+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:03:49.066906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069216500000000001,"stat_sum":{"num_bytes":4174378,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":6,"num_read":797,"num_read_kb":934,"num_write":28,"num_write_kb":1218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"30'46","reported_seq":761,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T21:40:54.269334+0000","last_change":"2024-09-21T21:40:39.420213+0000","last_active":"2024-09-21T21:40:54.269334+0000","last_peered":"2024-09-21T21:40:54.269334+0000","last_clean":"2024-09-21T21:40:54.269334+0000","last_became_active":"2024-09-21T21:40:37.424316+0000","last_became_peered":"2024-09-21T21:40:37.424316+0000","last_unstale":"2024-09-21T21:40:54.269334+0000","last_undegraded":"2024-09-21T21:40:54.269334+0000","last_fullsized":"2024-09-21T21:40:54.269334+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:26:31.548147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095031900000000003,"stat_sum":{"num_bytes":1347603,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":4,"num_read":682,"num_read_kb":1152,"num_write":42,"num_write_kb":196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"32'71","reported_seq":1101,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T21:40:53.306427+0000","last_change":"2024-09-21T21:40:39.414575+0000","last_active":"2024-09-21T21:40:53.306427+0000","last_peered":"2024-09-21T21:40:53.306427+0000","last_clean":"2024-09-21T21:40:53.306427+0000","last_became_active":"2024-09-21T21:40:37.426715+0000","last_became_peered":"2024-09-21T21:40:37.426715+0000","last_unstale":"2024-09-21T21:40:53.306427+0000","last_undegraded":"2024-09-21T21:40:53.306427+0000","last_fullsized":"2024-09-21T21:40:53.306427+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:02:57.174627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066263800000000001,"stat_sum":{"num_bytes":9218625,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":4,"num_read":1230,"num_read_kb":5128,"num_write":64,"num_write_kb":2255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"30'29","reported_seq":1291,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T21:40:50.943986+0000","last_change":"2024-09-21T21:40:39.416634+0000","last_active":"2024-09-21T21:40:50.943986+0000","last_peered":"2024-09-21T21:40:50.943986+0000","last_clean":"2024-09-21T21:40:50.943986+0000","last_became_active":"2024-09-21T21:40:37.421603+0000","last_became_peered":"2024-09-21T21:40:37.421603+0000","last_unstale":"2024-09-21T21:40:50.943986+0000","last_undegraded":"2024-09-21T21:40:50.943986+0000","last_fullsized":"2024-09-21T21:40:50.943986+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:41:16.516173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042576799999999998,"stat_sum":{"num_bytes":1201152,"num_objects":6,"num_object_clones":1,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":3,"num_read":1237,"num_read_kb":3610,"num_write":26,"num_write_kb":134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"30'80","reported_seq":1687,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T21:40:53.060760+0000","last_change":"2024-09-21T21:40:39.430190+0000","last_active":"2024-09-21T21:40:53.060760+0000","last_peered":"2024-09-21T21:40:53.060760+0000","last_clean":"2024-09-21T21:40:53.060760+0000","last_became_active":"2024-09-21T21:40:37.424911+0000","last_became_peered":"2024-09-21T21:40:37.424911+0000","last_unstale":"2024-09-21T21:40:53.060760+0000","last_undegraded":"2024-09-21T21:40:53.060760+0000","last_fullsized":"2024-09-21T21:40:53.060760+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:03:55.851377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001258211,"stat_sum":{"num_bytes":2913329,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":4,"num_read":2290,"num_read_kb":4692,"num_write":47,"num_write_kb":180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"32'101","reported_seq":2045,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T21:40:53.075629+0000","last_change":"2024-09-21T21:40:39.425366+0000","last_active":"2024-09-21T21:40:53.075629+0000","last_peered":"2024-09-21T21:40:53.075629+0000","last_clean":"2024-09-21T21:40:53.075629+0000","last_became_active":"2024-09-21T21:40:37.424364+0000","last_became_peered":"2024-09-21T21:40:37.424364+0000","last_unstale":"2024-09-21T21:40:53.075629+0000","last_undegraded":"2024-09-21T21:40:53.075629+0000","last_fullsized":"2024-09-21T21:40:53.075629+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:22:10.654563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015121799999999999,"stat_sum":{"num_bytes":8563200,"num_objects":11,"num_object_clones":1,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":2,"num_read":2630,"num_read_kb":9130,"num_write":73,"num_write_kb":720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"30'22","reported_seq":309,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T21:40:53.058997+0000","last_change":"2024-09-21T21:40:39.415411+0000","last_active":"2024-09-21T21:40:53.058997+0000","last_peered":"2024-09-21T21:40:53.058997+0000","last_clean":"2024-09-21T21:40:53.058997+0000","last_became_active":"2024-09-21T21:40:37.421701+0000","last_became_peered":"2024-09-21T21:40:37.421701+0000","last_unstale":"2024-09-21T21:40:53.058997+0000","last_undegraded":"2024-09-21T21:40:53.058997+0000","last_fullsized":"2024-09-21T21:40:53.058997+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:39:35.215696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001006961,"stat_sum":{"num_bytes":2939392,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":2,"num_read":252,"num_read_kb":2135,"num_write":20,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"32'49","reported_seq":409,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T21:40:54.256883+0000","last_change":"2024-09-21T21:40:39.415100+0000","last_active":"2024-09-21T21:40:54.256883+0000","last_peered":"2024-09-21T21:40:54.256883+0000","last_clean":"2024-09-21T21:40:54.256883+0000","last_became_active":"2024-09-21T21:40:37.426062+0000","last_became_peered":"2024-09-21T21:40:37.426062+0000","last_unstale":"2024-09-21T21:40:54.256883+0000","last_undegraded":"2024-09-21T21:40:54.256883+0000","last_fullsized":"2024-09-21T21:40:54.256883+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:45:53.498259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093907599999999995,"stat_sum":{"num_bytes":901247,"num_objects":8,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":4,"num_read":335,"num_read_kb":1259,"num_write":45,"num_write_kb":443,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":43,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T21:40:52.496763+0000","last_change":"2024-09-21T21:40:27.380563+0000","last_active":"2024-09-21T21:40:52.496763+0000","last_peered":"2024-09-21T21:40:52.496763+0000","last_clean":"2024-09-21T21:40:52.496763+0000","last_became_active":"2024-09-21T21:40:25.501200+0000","last_became_peered":"2024-09-21T21:40:25.501200+0000","last_unstale":"2024-09-21T21:40:52.496763+0000","last_undegraded":"2024-09-21T21:40:52.496763+0000","last_fullsized":"2024-09-21T21:40:52.496763+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:03:24.900063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072643899999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"32'29","reported_seq":422,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T21:40:53.296163+0000","last_change":"2024-09-21T21:40:39.430054+0000","last_active":"2024-09-21T21:40:53.296163+0000","last_peered":"2024-09-21T21:40:53.296163+0000","last_clean":"2024-09-21T21:40:53.296163+0000","last_became_active":"2024-09-21T21:40:37.422487+0000","last_became_peered":"2024-09-21T21:40:37.422487+0000","last_unstale":"2024-09-21T21:40:53.296163+0000","last_undegraded":"2024-09-21T21:40:53.296163+0000","last_fullsized":"2024-09-21T21:40:53.296163+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:28:05.006081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011470160000000001,"stat_sum":{"num_bytes":12721664,"num_objects":5,"num_object_clones":1,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":2,"num_read":360,"num_read_kb":1817,"num_write":30,"num_write_kb":1497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":45,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T21:40:52.502187+0000","last_change":"2024-09-21T21:40:27.379903+0000","last_active":"2024-09-21T21:40:52.502187+0000","last_peered":"2024-09-21T21:40:52.502187+0000","last_clean":"2024-09-21T21:40:52.502187+0000","last_became_active":"2024-09-21T21:40:25.371095+0000","last_became_peered":"2024-09-21T21:40:25.371095+0000","last_unstale":"2024-09-21T21:40:52.502187+0000","last_undegraded":"2024-09-21T21:40:52.502187+0000","last_fullsized":"2024-09-21T21:40:52.502187+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:08:09.434308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062280799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"30'22","reported_seq":75,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T21:40:51.151254+0000","last_change":"2024-09-21T21:40:39.416965+0000","last_active":"2024-09-21T21:40:51.151254+0000","last_peered":"2024-09-21T21:40:51.151254+0000","last_clean":"2024-09-21T21:40:51.151254+0000","last_became_active":"2024-09-21T21:40:37.418428+0000","last_became_peered":"2024-09-21T21:40:37.418428+0000","last_unstale":"2024-09-21T21:40:51.151254+0000","last_undegraded":"2024-09-21T21:40:51.151254+0000","last_fullsized":"2024-09-21T21:40:51.151254+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:41:29.229503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066113800000000003,"stat_sum":{"num_bytes":753664,"num_objects":5,"num_object_clones":1,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":4,"num_read":23,"num_read_kb":104,"num_write":17,"num_write_kb":200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":43,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T21:40:52.496740+0000","last_change":"2024-09-21T21:40:27.380218+0000","last_active":"2024-09-21T21:40:52.496740+0000","last_peered":"2024-09-21T21:40:52.496740+0000","last_clean":"2024-09-21T21:40:52.496740+0000","last_became_active":"2024-09-21T21:40:25.500143+0000","last_became_peered":"2024-09-21T21:40:25.500143+0000","last_unstale":"2024-09-21T21:40:52.496740+0000","last_undegraded":"2024-09-21T21:40:52.496740+0000","last_fullsized":"2024-09-21T21:40:52.496740+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:11:42.747388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042715899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"30'26","reported_seq":150,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T21:40:52.501881+0000","last_change":"2024-09-21T21:40:39.414839+0000","last_active":"2024-09-21T21:40:52.501881+0000","last_peered":"2024-09-21T21:40:52.501881+0000","last_clean":"2024-09-21T21:40:52.501881+0000","last_became_active":"2024-09-21T21:40:37.419944+0000","last_became_peered":"2024-09-21T21:40:37.419944+0000","last_unstale":"2024-09-21T21:40:52.501881+0000","last_undegraded":"2024-09-21T21:40:52.501881+0000","last_fullsized":"2024-09-21T21:40:52.501881+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:47:51.000689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085769100000000005,"stat_sum":{"num_bytes":1605632,"num_objects":6,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":3,"num_read":91,"num_read_kb":396,"num_write":23,"num_write_kb":447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":45,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T21:40:52.501993+0000","last_change":"2024-09-21T21:40:27.380005+0000","last_active":"2024-09-21T21:40:52.501993+0000","last_peered":"2024-09-21T21:40:52.501993+0000","last_clean":"2024-09-21T21:40:52.501993+0000","last_became_active":"2024-09-21T21:40:25.500109+0000","last_became_peered":"2024-09-21T21:40:25.500109+0000","last_unstale":"2024-09-21T21:40:52.501993+0000","last_undegraded":"2024-09-21T21:40:52.501993+0000","last_fullsized":"2024-09-21T21:40:52.501993+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:21:10.631127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070669899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"32'38","reported_seq":599,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T21:40:53.066149+0000","last_change":"2024-09-21T21:40:39.424843+0000","last_active":"2024-09-21T21:40:53.066149+0000","last_peered":"2024-09-21T21:40:53.066149+0000","last_clean":"2024-09-21T21:40:53.066149+0000","last_became_active":"2024-09-21T21:40:37.423874+0000","last_became_peered":"2024-09-21T21:40:37.423874+0000","last_unstale":"2024-09-21T21:40:53.066149+0000","last_undegraded":"2024-09-21T21:40:53.066149+0000","last_fullsized":"2024-09-21T21:40:53.066149+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:47:50.927973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010561050000000001,"stat_sum":{"num_bytes":2437152,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":5,"num_read":543,"num_read_kb":5990,"num_write":39,"num_write_kb":560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"30'25","reported_seq":114,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T21:40:53.058526+0000","last_change":"2024-09-21T21:40:39.419600+0000","last_active":"2024-09-21T21:40:53.058526+0000","last_peered":"2024-09-21T21:40:53.058526+0000","last_clean":"2024-09-21T21:40:53.058526+0000","last_became_active":"2024-09-21T21:40:37.424903+0000","last_became_peered":"2024-09-21T21:40:37.424903+0000","last_unstale":"2024-09-21T21:40:53.058526+0000","last_undegraded":"2024-09-21T21:40:53.058526+0000","last_fullsized":"2024-09-21T21:40:53.058526+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:56:26.566893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041679500000000002,"stat_sum":{"num_bytes":119296,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":3,"num_read":52,"num_read_kb":295,"num_write":22,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":47,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T21:40:52.506234+0000","last_change":"2024-09-21T21:40:27.382736+0000","last_active":"2024-09-21T21:40:52.506234+0000","last_peered":"2024-09-21T21:40:52.506234+0000","last_clean":"2024-09-21T21:40:52.506234+0000","last_became_active":"2024-09-21T21:40:25.371972+0000","last_became_peered":"2024-09-21T21:40:25.371972+0000","last_unstale":"2024-09-21T21:40:52.506234+0000","last_undegraded":"2024-09-21T21:40:52.506234+0000","last_fullsized":"2024-09-21T21:40:52.506234+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:45:30.417508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000906268,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"30'90","reported_seq":1158,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T21:40:52.501684+0000","last_change":"2024-09-21T21:40:39.414972+0000","last_active":"2024-09-21T21:40:52.501684+0000","last_peered":"2024-09-21T21:40:52.501684+0000","last_clean":"2024-09-21T21:40:52.501684+0000","last_became_active":"2024-09-21T21:40:37.418973+0000","last_became_peered":"2024-09-21T21:40:37.418973+0000","last_unstale":"2024-09-21T21:40:52.501684+0000","last_undegraded":"2024-09-21T21:40:52.501684+0000","last_fullsized":"2024-09-21T21:40:52.501684+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:05:06.513064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001017923,"stat_sum":{"num_bytes":1863696,"num_objects":6,"num_object_clones":1,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":2,"num_read":2097,"num_read_kb":2943,"num_write":45,"num_write_kb":120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":39,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T21:40:49.481859+0000","last_change":"2024-09-21T21:40:27.380792+0000","last_active":"2024-09-21T21:40:49.481859+0000","last_peered":"2024-09-21T21:40:49.481859+0000","last_clean":"2024-09-21T21:40:49.481859+0000","last_became_active":"2024-09-21T21:40:25.371578+0000","last_became_peered":"2024-09-21T21:40:25.371578+0000","last_unstale":"2024-09-21T21:40:49.481859+0000","last_undegraded":"2024-09-21T21:40:49.481859+0000","last_fullsized":"2024-09-21T21:40:49.481859+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:16:44.813324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000459858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"30'28","reported_seq":791,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T21:40:52.945634+0000","last_change":"2024-09-21T21:40:39.429211+0000","last_active":"2024-09-21T21:40:52.945634+0000","last_peered":"2024-09-21T21:40:52.945634+0000","last_clean":"2024-09-21T21:40:52.945634+0000","last_became_active":"2024-09-21T21:40:37.428542+0000","last_became_peered":"2024-09-21T21:40:37.428542+0000","last_unstale":"2024-09-21T21:40:52.945634+0000","last_undegraded":"2024-09-21T21:40:52.945634+0000","last_fullsized":"2024-09-21T21:40:52.945634+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:35:42.978872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045982299999999999,"stat_sum":{"num_bytes":2755627,"num_objects":7,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":4,"num_read":741,"num_read_kb":4812,"num_write":24,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":43,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T21:40:52.496381+0000","last_change":"2024-09-21T21:40:27.380219+0000","last_active":"2024-09-21T21:40:52.496381+0000","last_peered":"2024-09-21T21:40:52.496381+0000","last_clean":"2024-09-21T21:40:52.496381+0000","last_became_active":"2024-09-21T21:40:25.371451+0000","last_became_peered":"2024-09-21T21:40:25.371451+0000","last_unstale":"2024-09-21T21:40:52.496381+0000","last_undegraded":"2024-09-21T21:40:52.496381+0000","last_fullsized":"2024-09-21T21:40:52.496381+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:05:41.445313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000438863,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"32'39","reported_seq":1095,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T21:40:53.305767+0000","last_change":"2024-09-21T21:40:39.415337+0000","last_active":"2024-09-21T21:40:53.305767+0000","last_peered":"2024-09-21T21:40:53.305767+0000","last_clean":"2024-09-21T21:40:53.305767+0000","last_became_active":"2024-09-21T21:40:37.421655+0000","last_became_peered":"2024-09-21T21:40:37.421655+0000","last_unstale":"2024-09-21T21:40:53.305767+0000","last_undegraded":"2024-09-21T21:40:53.305767+0000","last_fullsized":"2024-09-21T21:40:53.305767+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:21:40.036520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000835322,"stat_sum":{"num_bytes":19742224,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":5,"num_read":1030,"num_read_kb":5082,"num_write":36,"num_write_kb":2370,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":46,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T21:40:52.506276+0000","last_change":"2024-09-21T21:40:27.382253+0000","last_active":"2024-09-21T21:40:52.506276+0000","last_peered":"2024-09-21T21:40:52.506276+0000","last_clean":"2024-09-21T21:40:52.506276+0000","last_became_active":"2024-09-21T21:40:25.372202+0000","last_became_peered":"2024-09-21T21:40:25.372202+0000","last_unstale":"2024-09-21T21:40:52.506276+0000","last_undegraded":"2024-09-21T21:40:52.506276+0000","last_fullsized":"2024-09-21T21:40:52.506276+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:19:18.518621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036803999999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":195,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T21:40:52.496388+0000","last_change":"2024-09-21T21:40:19.352502+0000","last_active":"2024-09-21T21:40:52.496388+0000","last_peered":"2024-09-21T21:40:52.496388+0000","last_clean":"2024-09-21T21:40:52.496388+0000","last_became_active":"2024-09-21T21:40:19.352227+0000","last_became_peered":"2024-09-21T21:40:19.352227+0000","last_unstale":"2024-09-21T21:40:52.496388+0000","last_undegraded":"2024-09-21T21:40:52.496388+0000","last_fullsized":"2024-09-21T21:40:52.496388+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:18.340430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:18.340430+0000","last_clean_scrub_stamp":"2024-09-21T21:40:18.340430+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:19:13.902196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"30'45","reported_seq":770,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-21T21:40:51.240541+0000","last_change":"2024-09-21T21:40:39.416961+0000","last_active":"2024-09-21T21:40:51.240541+0000","last_peered":"2024-09-21T21:40:51.240541+0000","last_clean":"2024-09-21T21:40:51.240541+0000","last_became_active":"2024-09-21T21:40:37.418416+0000","last_became_peered":"2024-09-21T21:40:37.418416+0000","last_unstale":"2024-09-21T21:40:51.240541+0000","last_undegraded":"2024-09-21T21:40:51.240541+0000","last_fullsized":"2024-09-21T21:40:51.240541+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:08:11.330244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056255200000000002,"stat_sum":{"num_bytes":4030009,"num_objects":11,"num_object_clones":1,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":5,"num_read":772,"num_read_kb":5764,"num_write":50,"num_write_kb":1234,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"30'50","reported_seq":390,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T21:40:53.061104+0000","last_change":"2024-09-21T21:40:39.419805+0000","last_active":"2024-09-21T21:40:53.061104+0000","last_peered":"2024-09-21T21:40:53.061104+0000","last_clean":"2024-09-21T21:40:53.061104+0000","last_became_active":"2024-09-21T21:40:37.425495+0000","last_became_peered":"2024-09-21T21:40:37.425495+0000","last_unstale":"2024-09-21T21:40:53.061104+0000","last_undegraded":"2024-09-21T21:40:53.061104+0000","last_fullsized":"2024-09-21T21:40:53.061104+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:19:27.577647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035788200000000001,"stat_sum":{"num_bytes":3611735,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":6,"num_read":316,"num_read_kb":2869,"num_write":47,"num_write_kb":1479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"32'208","reported_seq":308,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T21:40:53.442036+0000","last_change":"2024-09-21T21:40:39.414797+0000","last_active":"2024-09-21T21:40:53.442036+0000","last_peered":"2024-09-21T21:40:53.442036+0000","last_clean":"2024-09-21T21:40:53.442036+0000","last_became_active":"2024-09-21T21:40:37.426458+0000","last_became_peered":"2024-09-21T21:40:37.426458+0000","last_unstale":"2024-09-21T21:40:53.442036+0000","last_undegraded":"2024-09-21T21:40:53.442036+0000","last_fullsized":"2024-09-21T21:40:53.442036+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:54:05.950419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065146799999999999,"stat_sum":{"num_bytes":126,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":1,"num_read":896,"num_read_kb":1221,"num_write":318,"num_write_kb":415,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"30'26","reported_seq":87,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T21:40:52.501502+0000","last_change":"2024-09-21T21:40:39.415091+0000","last_active":"2024-09-21T21:40:52.501502+0000","last_peered":"2024-09-21T21:40:52.501502+0000","last_clean":"2024-09-21T21:40:52.501502+0000","last_became_active":"2024-09-21T21:40:37.421700+0000","last_became_peered":"2024-09-21T21:40:37.421700+0000","last_unstale":"2024-09-21T21:40:52.501502+0000","last_undegraded":"2024-09-21T21:40:52.501502+0000","last_fullsized":"2024-09-21T21:40:52.501502+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:40:54.472961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001083334,"stat_sum":{"num_bytes":1048576,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":2,"num_read":27,"num_read_kb":56,"num_write":27,"num_write_kb":2510,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"32'271","reported_seq":1279,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T21:40:53.064140+0000","last_change":"2024-09-21T21:40:39.425143+0000","last_active":"2024-09-21T21:40:53.064140+0000","last_peered":"2024-09-21T21:40:53.064140+0000","last_clean":"2024-09-21T21:40:53.064140+0000","last_became_active":"2024-09-21T21:40:37.423078+0000","last_became_peered":"2024-09-21T21:40:37.423078+0000","last_unstale":"2024-09-21T21:40:53.064140+0000","last_undegraded":"2024-09-21T21:40:53.064140+0000","last_fullsized":"2024-09-21T21:40:53.064140+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:11:23.164431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001302274,"stat_sum":{"num_bytes":4759663,"num_objects":7,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":2,"num_read":2308,"num_read_kb":8136,"num_write":472,"num_write_kb":1721,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"32'57","reported_seq":281,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T21:40:54.257291+0000","last_change":"2024-09-21T21:40:39.415498+0000","last_active":"2024-09-21T21:40:54.257291+0000","last_peered":"2024-09-21T21:40:54.257291+0000","last_clean":"2024-09-21T21:40:54.257291+0000","last_became_active":"2024-09-21T21:40:37.416021+0000","last_became_peered":"2024-09-21T21:40:37.416021+0000","last_unstale":"2024-09-21T21:40:54.257291+0000","last_undegraded":"2024-09-21T21:40:54.257291+0000","last_fullsized":"2024-09-21T21:40:54.257291+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:39:16.111896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098736399999999991,"stat_sum":{"num_bytes":2912256,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":5,"num_read":190,"num_read_kb":1033,"num_write":52,"num_write_kb":610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"32'26","reported_seq":572,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T21:40:53.142716+0000","last_change":"2024-09-21T21:40:39.425937+0000","last_active":"2024-09-21T21:40:53.142716+0000","last_peered":"2024-09-21T21:40:53.142716+0000","last_clean":"2024-09-21T21:40:53.142716+0000","last_became_active":"2024-09-21T21:40:37.425683+0000","last_became_peered":"2024-09-21T21:40:37.425683+0000","last_unstale":"2024-09-21T21:40:53.142716+0000","last_undegraded":"2024-09-21T21:40:53.142716+0000","last_fullsized":"2024-09-21T21:40:53.142716+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:45:48.509457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020524470000000002,"stat_sum":{"num_bytes":2562048,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":2,"num_read":511,"num_read_kb":6239,"num_write":31,"num_write_kb":1697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"32'46","reported_seq":880,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T21:40:53.162046+0000","last_change":"2024-09-21T21:40:39.425496+0000","last_active":"2024-09-21T21:40:53.162046+0000","last_peered":"2024-09-21T21:40:53.162046+0000","last_clean":"2024-09-21T21:40:53.162046+0000","last_became_active":"2024-09-21T21:40:37.424987+0000","last_became_peered":"2024-09-21T21:40:37.424987+0000","last_unstale":"2024-09-21T21:40:53.162046+0000","last_undegraded":"2024-09-21T21:40:53.162046+0000","last_fullsized":"2024-09-21T21:40:53.162046+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:01:36.751398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016364540000000001,"stat_sum":{"num_bytes":2544128,"num_objects":10,"num_object_clones":3,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":3,"num_read":1071,"num_read_kb":3176,"num_write":33,"num_write_kb":261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"32'27","reported_seq":390,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T21:40:53.290374+0000","last_change":"2024-09-21T21:40:39.424287+0000","last_active":"2024-09-21T21:40:53.290374+0000","last_peered":"2024-09-21T21:40:53.290374+0000","last_clean":"2024-09-21T21:40:53.290374+0000","last_became_active":"2024-09-21T21:40:37.423011+0000","last_became_peered":"2024-09-21T21:40:37.423011+0000","last_unstale":"2024-09-21T21:40:53.290374+0000","last_undegraded":"2024-09-21T21:40:53.290374+0000","last_fullsized":"2024-09-21T21:40:53.290374+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:28:31.042223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065325099999999998,"stat_sum":{"num_bytes":10533888,"num_objects":7,"num_object_clones":2,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":4,"num_read":331,"num_read_kb":2490,"num_write":27,"num_write_kb":1512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"30'37","reported_seq":983,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T21:40:52.946681+0000","last_change":"2024-09-21T21:40:39.415203+0000","last_active":"2024-09-21T21:40:52.946681+0000","last_peered":"2024-09-21T21:40:52.946681+0000","last_clean":"2024-09-21T21:40:52.946681+0000","last_became_active":"2024-09-21T21:40:37.426148+0000","last_became_peered":"2024-09-21T21:40:37.426148+0000","last_unstale":"2024-09-21T21:40:52.946681+0000","last_undegraded":"2024-09-21T21:40:52.946681+0000","last_fullsized":"2024-09-21T21:40:52.946681+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:57:01.139847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087239000000000001,"stat_sum":{"num_bytes":5398528,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":910,"num_read_kb":7664,"num_write":38,"num_write_kb":2377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"32'130","reported_seq":2501,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T21:40:53.059121+0000","last_change":"2024-09-21T21:40:39.429216+0000","last_active":"2024-09-21T21:40:53.059121+0000","last_peered":"2024-09-21T21:40:53.059121+0000","last_clean":"2024-09-21T21:40:53.059121+0000","last_became_active":"2024-09-21T21:40:37.424011+0000","last_became_peered":"2024-09-21T21:40:37.424011+0000","last_unstale":"2024-09-21T21:40:53.059121+0000","last_undegraded":"2024-09-21T21:40:53.059121+0000","last_fullsized":"2024-09-21T21:40:53.059121+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:26:38.948417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046261000000000002,"stat_sum":{"num_bytes":2792448,"num_objects":9,"num_object_clones":2,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":4,"num_read":2966,"num_read_kb":4373,"num_write":78,"num_write_kb":613,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"32'71","reported_seq":1713,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T21:40:54.287726+0000","last_change":"2024-09-21T21:40:39.414779+0000","last_active":"2024-09-21T21:40:54.287726+0000","last_peered":"2024-09-21T21:40:54.287726+0000","last_clean":"2024-09-21T21:40:54.287726+0000","last_became_active":"2024-09-21T21:40:37.421627+0000","last_became_peered":"2024-09-21T21:40:37.421627+0000","last_unstale":"2024-09-21T21:40:54.287726+0000","last_undegraded":"2024-09-21T21:40:54.287726+0000","last_fullsized":"2024-09-21T21:40:54.287726+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:29:37.534850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070986300000000003,"stat_sum":{"num_bytes":17395712,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":3,"num_read":1627,"num_read_kb":14388,"num_write":73,"num_write_kb":1832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":141662069,"num_objects":237,"num_object_clones":19,"num_object_copies":474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":237,"num_whiteouts":107,"num_read":30984,"num_read_kb":125002,"num_write":2314,"num_write_kb":32003,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4108288,"data_stored":14516442,"data_compressed":1526871,"data_compressed_allocated":2007040,"data_compressed_original":12782080,"omap_allocated":0,"internal_metadata":0},"log_size":2082,"ondisk_log_size":2082,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64790,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32064,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339776,"statfs":{"total":96636764160,"available":96603930624,"internally_reserved":0,"allocated":901120,"data_stored":1281208,"data_compressed":135444,"data_compressed_allocated":212992,"data_compressed_original":880640,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32276,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339564,"statfs":{"total":96636764160,"available":96603713536,"internally_reserved":0,"allocated":1114112,"data_stored":3088879,"data_compressed":292816,"data_compressed_allocated":376832,"data_compressed_original":2644480,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":12,"seq":51539607562,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32248,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339592,"statfs":{"total":96636764160,"available":96603742208,"internally_reserved":0,"allocated":1081344,"data_stored":2364319,"data_compressed":167604,"data_compressed_allocated":290816,"data_compressed_original":1859584,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607562,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31884,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339956,"statfs":{"total":96636764160,"available":96604114944,"internally_reserved":0,"allocated":712704,"data_stored":559320,"data_compressed":50980,"data_compressed_allocated":73728,"data_compressed_original":176128,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607563,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32036,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339804,"statfs":{"total":96636764160,"available":96603959296,"internally_reserved":0,"allocated":868352,"data_stored":1283901,"data_compressed":160294,"data_compressed_allocated":155648,"data_compressed_original":823296,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640266,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31880,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339960,"statfs":{"total":96636764160,"available":96604119040,"internally_reserved":0,"allocated":663552,"data_stored":484294,"data_compressed":93783,"data_compressed_allocated":118784,"data_compressed_original":180224,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32395,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32395,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":815104,"data_stored":3306245,"data_compressed":399683,"data_compressed_allocated":438272,"data_compressed_original":2973696,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":540672,"data_stored":1826477,"data_compressed":178358,"data_compressed_allocated":229376,"data_compressed_original":1560576,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":421888,"data_stored":968411,"data_compressed":174593,"data_compressed_allocated":225280,"data_compressed_original":815104,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":753664,"data_stored":2437432,"data_compressed":261991,"data_compressed_allocated":339968,"data_compressed_original":2103808,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":528384,"data_stored":1167393,"data_compressed":137765,"data_compressed_allocated":217088,"data_compressed_original":929792,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1048576,"data_stored":4810484,"data_compressed":374481,"data_compressed_allocated":557056,"data_compressed_original":4399104,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T21:40:57.421 INFO:tasks.thrashosds.thrasher:Setting 3.4 to [0, 4, 5, 3] 2024-09-21T21:40:57.421 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.4', '0', '4', '5', '3'] 2024-09-21T21:40:57.421 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.4 0 4 5 3 2024-09-21T21:40:57.437 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:40:57.436+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:40:57.481 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:40:57.481 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:40:57.481 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-21T21:40:57.481 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:40:57.490 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T21:40:57.516 INFO:teuthology.orchestra.run.smithi081.stderr:nodeep-scrub is unset 2024-09-21T21:40:57.538 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:40:57.537+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:40:57.585 INFO:teuthology.orchestra.run.smithi164.stdout:truncating image image_client.0-clone7 from 0xa711200 (overlap 0x258a00) to 0x1e0800 2024-09-21T21:40:57.640 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:40:57.638+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:40:57.642 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:40:57.642 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:40:57.642 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:40:57.642 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:40:57.642 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:40:57.651 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T21:40:57.741 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:40:57.740+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:40:57.800 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:40:57.800 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-21T21:40:57.800 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-21T21:40:57.800 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:40:57.800 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.800 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4710.0:435 3.e 3:75e0c9d8:::rbd_data.1121343e7d74.0000000000000001:8 [read 348160~24576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T21:40:57.800 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:53.053197+0000", 2024-09-21T21:40:57.800 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.7445437400000001, 2024-09-21T21:40:57.800 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.01258082, 2024-09-21T21:40:57.801 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:57.801 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:57.801 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:40:57.801 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:57.801 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4710", 2024-09-21T21:40:57.801 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1566290539", 2024-09-21T21:40:57.801 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 435 2024-09-21T21:40:57.801 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.801 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:57.801 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.801 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:57.801 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.053197+0000", 2024-09-21T21:40:57.801 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:57.801 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.801 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.802 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:57.802 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.053197+0000", 2024-09-21T21:40:57.802 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:57.802 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.802 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.802 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:57.802 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.053197+0000", 2024-09-21T21:40:57.802 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7300000000000001e-07 2024-09-21T21:40:57.802 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.802 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.802 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:57.802 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.053199+0000", 2024-09-21T21:40:57.802 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0169999999999999e-06 2024-09-21T21:40:57.802 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.802 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.803 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:57.803 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.053206+0000", 2024-09-21T21:40:57.803 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3069999999999999e-06 2024-09-21T21:40:57.803 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.803 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.803 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:57.803 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.053210+0000", 2024-09-21T21:40:57.803 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.853e-06 2024-09-21T21:40:57.803 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.803 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.803 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:57.803 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.065683+0000", 2024-09-21T21:40:57.803 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.012472492 2024-09-21T21:40:57.803 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.804 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.804 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:57.804 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.065728+0000", 2024-09-21T21:40:57.804 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5198999999999999e-05 2024-09-21T21:40:57.804 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.804 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.804 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:57.804 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.065778+0000", 2024-09-21T21:40:57.804 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9478999999999998e-05 2024-09-21T21:40:57.804 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:57.804 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:57.804 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:57.804 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.804 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.804 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4710.0:436 3.e 3:75e0c9d8:::rbd_data.1121343e7d74.0000000000000001:8 [read 413696~24576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T21:40:57.805 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:53.053217+0000", 2024-09-21T21:40:57.805 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.7445236140000002, 2024-09-21T21:40:57.805 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.012678024, 2024-09-21T21:40:57.805 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:57.805 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:57.805 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:40:57.805 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:57.805 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4710", 2024-09-21T21:40:57.805 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1566290539", 2024-09-21T21:40:57.805 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 436 2024-09-21T21:40:57.805 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.805 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:57.805 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.805 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:57.805 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.053217+0000", 2024-09-21T21:40:57.806 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:57.806 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.806 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.806 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:57.806 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.053217+0000", 2024-09-21T21:40:57.806 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:57.806 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.806 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.806 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:57.806 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.053217+0000", 2024-09-21T21:40:57.806 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3900000000000005e-07 2024-09-21T21:40:57.806 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.806 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.806 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:57.807 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.053220+0000", 2024-09-21T21:40:57.807 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1550000000000001e-06 2024-09-21T21:40:57.807 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.807 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.807 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:57.807 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.053226+0000", 2024-09-21T21:40:57.807 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5849999999999996e-06 2024-09-21T21:40:57.807 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.807 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.807 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:57.807 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.053231+0000", 2024-09-21T21:40:57.807 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8960000000000002e-06 2024-09-21T21:40:57.807 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.807 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.807 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:57.808 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.065797+0000", 2024-09-21T21:40:57.808 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.012566216 2024-09-21T21:40:57.808 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.808 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.808 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:57.808 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.065835+0000", 2024-09-21T21:40:57.808 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7740000000000001e-05 2024-09-21T21:40:57.808 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.808 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.808 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:57.808 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.065895+0000", 2024-09-21T21:40:57.808 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9892999999999997e-05 2024-09-21T21:40:57.808 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:57.808 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:57.808 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:57.809 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.809 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.809 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4710.0:437 3.e 3:75e0c9d8:::rbd_data.1121343e7d74.0000000000000001:8 [read 479232~24576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T21:40:57.809 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:53.053237+0000", 2024-09-21T21:40:57.809 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.7445035090000003, 2024-09-21T21:40:57.809 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.012913330000000001, 2024-09-21T21:40:57.809 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:57.809 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:57.809 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:40:57.809 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:57.809 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4710", 2024-09-21T21:40:57.809 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1566290539", 2024-09-21T21:40:57.809 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 437 2024-09-21T21:40:57.809 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.809 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:57.809 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.810 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:57.810 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.053237+0000", 2024-09-21T21:40:57.810 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:57.810 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.810 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.810 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:57.810 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.053237+0000", 2024-09-21T21:40:57.810 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:57.810 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.810 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.810 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:57.810 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.053237+0000", 2024-09-21T21:40:57.810 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9299999999999999e-07 2024-09-21T21:40:57.810 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.811 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.811 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:57.811 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.053239+0000", 2024-09-21T21:40:57.811 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.007e-06 2024-09-21T21:40:57.811 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.811 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.811 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:57.811 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.053245+0000", 2024-09-21T21:40:57.811 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1240000000000002e-06 2024-09-21T21:40:57.811 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.811 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.811 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:57.811 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.053250+0000", 2024-09-21T21:40:57.811 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8749999999999999e-06 2024-09-21T21:40:57.811 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.812 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.812 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:57.812 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.065924+0000", 2024-09-21T21:40:57.812 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.012673637 2024-09-21T21:40:57.812 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.812 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.812 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:57.812 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.065970+0000", 2024-09-21T21:40:57.812 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6347999999999997e-05 2024-09-21T21:40:57.812 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.812 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.812 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:57.812 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.066150+0000", 2024-09-21T21:40:57.812 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000179946 2024-09-21T21:40:57.812 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:57.813 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:57.813 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:57.813 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.813 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.813 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4710.0:491 3.e 3:71ab2529:::rbd_data.1121343e7d74.000000000000000e:8 [read 348160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T21:40:57.813 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:53.054683+0000", 2024-09-21T21:40:57.813 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.743057576, 2024-09-21T21:40:57.813 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.012616554, 2024-09-21T21:40:57.813 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:57.813 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:57.813 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:40:57.813 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:57.813 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4710", 2024-09-21T21:40:57.813 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1566290539", 2024-09-21T21:40:57.814 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 491 2024-09-21T21:40:57.814 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.814 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:57.814 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.814 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:57.814 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.054683+0000", 2024-09-21T21:40:57.814 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:57.814 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.814 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.814 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:57.814 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.054683+0000", 2024-09-21T21:40:57.814 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:57.814 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.814 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.815 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:57.815 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.054683+0000", 2024-09-21T21:40:57.815 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5999999999999999e-07 2024-09-21T21:40:57.815 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.815 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.815 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:57.815 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.054686+0000", 2024-09-21T21:40:57.815 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2840000000000001e-06 2024-09-21T21:40:57.815 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.815 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.815 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:57.815 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.054692+0000", 2024-09-21T21:40:57.815 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.443e-06 2024-09-21T21:40:57.815 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.816 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.816 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:57.816 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.054697+0000", 2024-09-21T21:40:57.816 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9429999999999999e-06 2024-09-21T21:40:57.816 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.816 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.816 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:57.816 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.067208+0000", 2024-09-21T21:40:57.816 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.012511034000000001 2024-09-21T21:40:57.816 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.816 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.816 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:57.816 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.067254+0000", 2024-09-21T21:40:57.816 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6310999999999998e-05 2024-09-21T21:40:57.816 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.817 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.817 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:57.817 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.067299+0000", 2024-09-21T21:40:57.817 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5079e-05 2024-09-21T21:40:57.817 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:57.817 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:57.817 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:57.817 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.817 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.817 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4710.0:492 3.e 3:71ab2529:::rbd_data.1121343e7d74.000000000000000e:8 [read 413696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T21:40:57.817 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:53.054703+0000", 2024-09-21T21:40:57.817 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.7430373770000003, 2024-09-21T21:40:57.817 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.012791482999999999, 2024-09-21T21:40:57.817 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:57.817 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:57.818 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:40:57.818 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:57.818 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4710", 2024-09-21T21:40:57.818 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1566290539", 2024-09-21T21:40:57.818 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 492 2024-09-21T21:40:57.818 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.818 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:57.818 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.818 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:57.818 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.054703+0000", 2024-09-21T21:40:57.818 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:57.818 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.818 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.818 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:57.819 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.054703+0000", 2024-09-21T21:40:57.819 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:57.819 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.819 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.819 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:57.819 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.054704+0000", 2024-09-21T21:40:57.819 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6499999999999999e-07 2024-09-21T21:40:57.819 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.819 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.819 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:57.819 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.054706+0000", 2024-09-21T21:40:57.819 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1430000000000001e-06 2024-09-21T21:40:57.819 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.819 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.819 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:57.819 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.054712+0000", 2024-09-21T21:40:57.820 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0870000000000003e-06 2024-09-21T21:40:57.820 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.820 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.820 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:57.820 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.054718+0000", 2024-09-21T21:40:57.820 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4919999999999998e-06 2024-09-21T21:40:57.820 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.820 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.820 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:57.820 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.067338+0000", 2024-09-21T21:40:57.820 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.012619798 2024-09-21T21:40:57.820 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.820 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.820 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:57.820 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.067414+0000", 2024-09-21T21:40:57.821 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5785999999999996e-05 2024-09-21T21:40:57.821 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.821 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.821 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:57.821 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.067495+0000", 2024-09-21T21:40:57.821 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0611999999999993e-05 2024-09-21T21:40:57.821 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:57.821 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:57.821 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:57.821 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.821 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.821 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4710.0:493 3.e 3:71ab2529:::rbd_data.1121343e7d74.000000000000000e:8 [read 479232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T21:40:57.821 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:53.054725+0000", 2024-09-21T21:40:57.821 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.7430157260000003, 2024-09-21T21:40:57.821 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.012957733000000001, 2024-09-21T21:40:57.822 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:57.822 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:57.822 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:40:57.822 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:57.822 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4710", 2024-09-21T21:40:57.822 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1566290539", 2024-09-21T21:40:57.822 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 493 2024-09-21T21:40:57.822 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.822 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:57.822 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.822 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:57.822 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.054725+0000", 2024-09-21T21:40:57.822 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:57.822 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.822 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.823 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:57.823 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.054725+0000", 2024-09-21T21:40:57.823 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:57.823 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.823 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.823 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:57.823 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.054725+0000", 2024-09-21T21:40:57.823 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3499999999999996e-07 2024-09-21T21:40:57.823 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.823 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.823 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:57.823 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.054728+0000", 2024-09-21T21:40:57.823 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2520000000000002e-06 2024-09-21T21:40:57.823 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.823 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.824 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:57.824 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.054734+0000", 2024-09-21T21:40:57.824 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2280000000000003e-06 2024-09-21T21:40:57.824 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.824 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.824 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:57.824 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.054739+0000", 2024-09-21T21:40:57.824 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0220000000000004e-06 2024-09-21T21:40:57.824 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.824 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.824 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:57.824 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.067535+0000", 2024-09-21T21:40:57.824 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.012796569000000001 2024-09-21T21:40:57.824 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.824 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.824 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:57.825 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.067608+0000", 2024-09-21T21:40:57.825 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2199000000000004e-05 2024-09-21T21:40:57.825 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.825 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.825 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:57.825 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.067682+0000", 2024-09-21T21:40:57.825 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4927999999999998e-05 2024-09-21T21:40:57.825 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:57.825 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:57.825 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:57.825 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.825 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.825 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4710.0:494 3.16 3:69888811:::rbd_data.1121343e7d74.0000000000000018:8 [read 24576~20480] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T21:40:57.825 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:53.054745+0000", 2024-09-21T21:40:57.825 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.7429953989999998, 2024-09-21T21:40:57.826 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.01295819, 2024-09-21T21:40:57.826 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:57.826 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:57.826 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:40:57.826 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:57.826 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4710", 2024-09-21T21:40:57.826 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1566290539", 2024-09-21T21:40:57.826 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 494 2024-09-21T21:40:57.826 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.826 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:57.826 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.826 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:57.826 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.054745+0000", 2024-09-21T21:40:57.826 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:57.827 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.827 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.827 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:57.827 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.054745+0000", 2024-09-21T21:40:57.827 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:57.827 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.827 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.827 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:57.827 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.054746+0000", 2024-09-21T21:40:57.827 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2799999999999996e-07 2024-09-21T21:40:57.827 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.827 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.827 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:57.827 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.054748+0000", 2024-09-21T21:40:57.827 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1409999999999999e-06 2024-09-21T21:40:57.828 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.828 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.828 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:57.828 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.054754+0000", 2024-09-21T21:40:57.828 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3369999999999996e-06 2024-09-21T21:40:57.828 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.828 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.828 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:57.828 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.054759+0000", 2024-09-21T21:40:57.828 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9189999999999998e-06 2024-09-21T21:40:57.828 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.828 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.829 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:57.829 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.067523+0000", 2024-09-21T21:40:57.829 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.012763752999999999 2024-09-21T21:40:57.829 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.829 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.829 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:57.829 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.067600+0000", 2024-09-21T21:40:57.829 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7353000000000003e-05 2024-09-21T21:40:57.829 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.829 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.829 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:57.829 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.067703+0000", 2024-09-21T21:40:57.829 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010315900000000001 2024-09-21T21:40:57.829 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:57.830 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:57.830 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:57.830 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.830 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.830 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4710.0:495 3.16 3:69888811:::rbd_data.1121343e7d74.0000000000000018:8 [read 90112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T21:40:57.830 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:53.054765+0000", 2024-09-21T21:40:57.830 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.7429753220000004, 2024-09-21T21:40:57.830 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.013039179, 2024-09-21T21:40:57.830 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:57.830 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:57.830 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:40:57.830 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:57.830 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4710", 2024-09-21T21:40:57.830 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1566290539", 2024-09-21T21:40:57.831 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 495 2024-09-21T21:40:57.831 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.831 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:57.831 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.831 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:57.831 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.054765+0000", 2024-09-21T21:40:57.831 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:57.831 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.831 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.831 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:57.831 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.054765+0000", 2024-09-21T21:40:57.831 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:57.831 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.831 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.832 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:57.832 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.054766+0000", 2024-09-21T21:40:57.832 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1399999999999997e-07 2024-09-21T21:40:57.832 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.832 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.832 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:57.832 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.054768+0000", 2024-09-21T21:40:57.832 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9269999999999999e-06 2024-09-21T21:40:57.832 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.832 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.832 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:57.832 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.054773+0000", 2024-09-21T21:40:57.832 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5609999999999998e-06 2024-09-21T21:40:57.832 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.832 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.832 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:57.833 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.054778+0000", 2024-09-21T21:40:57.833 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5959999999999997e-06 2024-09-21T21:40:57.833 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.833 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.833 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:57.833 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.067725+0000", 2024-09-21T21:40:57.833 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.012947264 2024-09-21T21:40:57.833 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.833 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.833 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:57.833 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.067767+0000", 2024-09-21T21:40:57.833 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2426999999999999e-05 2024-09-21T21:40:57.833 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.833 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.833 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:57.834 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.067804+0000", 2024-09-21T21:40:57.834 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6990000000000003e-05 2024-09-21T21:40:57.834 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:57.834 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:57.834 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:57.834 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.834 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.834 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4710.0:496 3.16 3:69888811:::rbd_data.1121343e7d74.0000000000000018:8 [read 155648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T21:40:57.834 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:53.054784+0000", 2024-09-21T21:40:57.834 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.7429564629999996, 2024-09-21T21:40:57.834 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.013131597, 2024-09-21T21:40:57.834 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:57.834 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:57.834 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:40:57.834 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:57.835 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4710", 2024-09-21T21:40:57.835 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1566290539", 2024-09-21T21:40:57.835 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 496 2024-09-21T21:40:57.835 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.835 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:57.835 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.835 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:57.835 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.054784+0000", 2024-09-21T21:40:57.835 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:57.835 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.835 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.835 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:57.835 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.054784+0000", 2024-09-21T21:40:57.835 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:57.836 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.836 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.836 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:57.836 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.054784+0000", 2024-09-21T21:40:57.836 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.51e-07 2024-09-21T21:40:57.836 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.836 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.836 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:57.836 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.054786+0000", 2024-09-21T21:40:57.836 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.8640000000000001e-06 2024-09-21T21:40:57.836 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.836 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.836 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:57.836 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.054792+0000", 2024-09-21T21:40:57.836 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1500000000000004e-06 2024-09-21T21:40:57.837 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.837 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.837 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:57.837 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.054797+0000", 2024-09-21T21:40:57.837 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.685e-06 2024-09-21T21:40:57.837 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.837 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.837 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:57.837 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.067830+0000", 2024-09-21T21:40:57.837 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.013032386999999999 2024-09-21T21:40:57.837 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.837 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.837 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:57.837 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.067876+0000", 2024-09-21T21:40:57.837 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6468000000000003e-05 2024-09-21T21:40:57.838 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.838 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.838 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:57.838 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.067916+0000", 2024-09-21T21:40:57.838 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9592000000000003e-05 2024-09-21T21:40:57.838 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:57.838 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:57.838 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:57.838 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.838 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.838 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4710.0:497 3.16 3:69888811:::rbd_data.1121343e7d74.0000000000000018:8 [read 221184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T21:40:57.838 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:53.054803+0000", 2024-09-21T21:40:57.838 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.7429374089999996, 2024-09-21T21:40:57.838 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.013213723, 2024-09-21T21:40:57.839 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:57.839 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:57.839 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:40:57.839 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:57.839 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4710", 2024-09-21T21:40:57.839 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1566290539", 2024-09-21T21:40:57.839 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 497 2024-09-21T21:40:57.839 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.839 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:57.839 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.839 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:57.839 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.054803+0000", 2024-09-21T21:40:57.839 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:57.839 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.840 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.840 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:57.840 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.054803+0000", 2024-09-21T21:40:57.840 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:57.840 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.840 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.840 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:57.840 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.054803+0000", 2024-09-21T21:40:57.840 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6600000000000002e-07 2024-09-21T21:40:57.840 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.840 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.840 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:57.840 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.054811+0000", 2024-09-21T21:40:57.840 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7889999999999995e-06 2024-09-21T21:40:57.841 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.841 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.841 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:57.841 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.054818+0000", 2024-09-21T21:40:57.841 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2759999999999997e-06 2024-09-21T21:40:57.841 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.841 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.841 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:57.841 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.055527+0000", 2024-09-21T21:40:57.841 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00070927000000000002 2024-09-21T21:40:57.841 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.841 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.841 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:57.841 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.067940+0000", 2024-09-21T21:40:57.841 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.012413621999999999 2024-09-21T21:40:57.842 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.842 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.842 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:57.842 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.067979+0000", 2024-09-21T21:40:57.842 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8247000000000002e-05 2024-09-21T21:40:57.842 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.842 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.842 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:57.842 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.068017+0000", 2024-09-21T21:40:57.842 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8152999999999999e-05 2024-09-21T21:40:57.842 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:57.842 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:57.842 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:57.842 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.842 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.843 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4710.0:499 3.16 3:69888811:::rbd_data.1121343e7d74.0000000000000018:8 [read 352256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T21:40:57.843 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:53.055578+0000", 2024-09-21T21:40:57.843 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.7421622780000003, 2024-09-21T21:40:57.843 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.012638095, 2024-09-21T21:40:57.843 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:57.843 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:57.843 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:40:57.843 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:57.843 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4710", 2024-09-21T21:40:57.843 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1566290539", 2024-09-21T21:40:57.843 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 499 2024-09-21T21:40:57.843 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.843 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:57.843 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.844 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:57.844 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.055578+0000", 2024-09-21T21:40:57.844 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:57.844 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.844 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.844 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:57.844 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.055578+0000", 2024-09-21T21:40:57.844 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:57.844 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.844 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.844 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:57.844 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.056232+0000", 2024-09-21T21:40:57.844 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00065383599999999998 2024-09-21T21:40:57.844 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.844 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.845 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:57.845 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.056237+0000", 2024-09-21T21:40:57.845 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9930000000000003e-06 2024-09-21T21:40:57.845 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.845 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.845 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:57.845 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.056248+0000", 2024-09-21T21:40:57.845 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0604e-05 2024-09-21T21:40:57.845 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.845 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.845 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:57.845 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.056255+0000", 2024-09-21T21:40:57.845 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5689999999999999e-06 2024-09-21T21:40:57.845 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.845 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.845 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:57.846 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.068140+0000", 2024-09-21T21:40:57.846 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.011884937999999999 2024-09-21T21:40:57.846 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.846 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.846 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:57.846 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.068177+0000", 2024-09-21T21:40:57.846 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7184999999999997e-05 2024-09-21T21:40:57.846 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.846 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.846 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:57.846 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.068216+0000", 2024-09-21T21:40:57.846 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8970000000000001e-05 2024-09-21T21:40:57.846 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:57.846 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:57.847 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:57.847 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.847 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.847 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4710.0:716 3.16 3:69888811:::rbd_data.1121343e7d74.0000000000000018:8 [read 438272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T21:40:57.847 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:53.061609+0000", 2024-09-21T21:40:57.847 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.7361318680000002, 2024-09-21T21:40:57.847 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.012602906000000001, 2024-09-21T21:40:57.847 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:57.847 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:57.847 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:40:57.847 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:57.847 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4710", 2024-09-21T21:40:57.847 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1566290539", 2024-09-21T21:40:57.847 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 716 2024-09-21T21:40:57.847 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.848 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:57.848 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.848 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:57.848 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.061609+0000", 2024-09-21T21:40:57.848 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:57.848 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.848 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.848 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:57.848 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.061609+0000", 2024-09-21T21:40:57.848 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:57.848 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.848 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.848 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:57.848 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.061609+0000", 2024-09-21T21:40:57.848 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3900000000000005e-07 2024-09-21T21:40:57.849 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.849 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.849 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:57.849 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.061611+0000", 2024-09-21T21:40:57.849 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.8670000000000001e-06 2024-09-21T21:40:57.849 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.849 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.849 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:57.849 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.061617+0000", 2024-09-21T21:40:57.849 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.694e-06 2024-09-21T21:40:57.849 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.849 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.849 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:57.849 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.061621+0000", 2024-09-21T21:40:57.849 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3220000000000003e-06 2024-09-21T21:40:57.850 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.850 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.850 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:57.850 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.074136+0000", 2024-09-21T21:40:57.850 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.012515024 2024-09-21T21:40:57.850 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.850 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.850 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:57.850 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.074176+0000", 2024-09-21T21:40:57.850 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9759000000000003e-05 2024-09-21T21:40:57.850 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.850 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.850 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:57.850 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.074212+0000", 2024-09-21T21:40:57.850 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5701000000000003e-05 2024-09-21T21:40:57.851 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:57.851 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:57.851 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:57.851 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.851 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.851 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4710.0:717 3.16 3:69888811:::rbd_data.1121343e7d74.0000000000000018:8 [read 503808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T21:40:57.851 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:53.061627+0000", 2024-09-21T21:40:57.851 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.7361139100000003, 2024-09-21T21:40:57.851 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.012786051, 2024-09-21T21:40:57.851 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:57.851 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:57.851 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:40:57.851 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:57.851 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4710", 2024-09-21T21:40:57.852 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1566290539", 2024-09-21T21:40:57.852 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 717 2024-09-21T21:40:57.852 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.852 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:57.852 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.852 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:57.852 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.061627+0000", 2024-09-21T21:40:57.852 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:57.852 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.852 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.852 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:57.852 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.061627+0000", 2024-09-21T21:40:57.852 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:57.852 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.853 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.853 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:57.853 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.061627+0000", 2024-09-21T21:40:57.853 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5200000000000002e-07 2024-09-21T21:40:57.853 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.853 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.853 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:57.853 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.061629+0000", 2024-09-21T21:40:57.853 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2819999999999999e-06 2024-09-21T21:40:57.853 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.853 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.853 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:57.853 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.061636+0000", 2024-09-21T21:40:57.853 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.126e-06 2024-09-21T21:40:57.853 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.854 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.854 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:57.854 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.061641+0000", 2024-09-21T21:40:57.854 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8579999999999999e-06 2024-09-21T21:40:57.854 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.854 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.854 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:57.854 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.074243+0000", 2024-09-21T21:40:57.854 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.012601311 2024-09-21T21:40:57.854 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.854 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.854 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:57.854 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.074343+0000", 2024-09-21T21:40:57.854 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.9902999999999995e-05 2024-09-21T21:40:57.855 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.855 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.855 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:57.855 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.074413+0000", 2024-09-21T21:40:57.855 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0118999999999997e-05 2024-09-21T21:40:57.855 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:57.855 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:57.855 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:57.855 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.855 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.855 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4710.0:815 3.16 3:6f3307d0:::rbd_data.1121343e7d74.0000000000000043:8 [read 110592~20480] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T21:40:57.855 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:53.062045+0000", 2024-09-21T21:40:57.855 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.7356950390000003, 2024-09-21T21:40:57.855 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.012729496, 2024-09-21T21:40:57.855 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:57.856 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:57.856 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:40:57.856 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:57.856 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4710", 2024-09-21T21:40:57.856 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1566290539", 2024-09-21T21:40:57.856 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 815 2024-09-21T21:40:57.856 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.856 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:57.856 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.856 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:57.856 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.062045+0000", 2024-09-21T21:40:57.856 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:57.856 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.856 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.856 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:57.857 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.062045+0000", 2024-09-21T21:40:57.857 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:57.857 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.857 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.857 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:57.857 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.062046+0000", 2024-09-21T21:40:57.857 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4000000000000001e-07 2024-09-21T21:40:57.857 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.857 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.857 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:57.857 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.062049+0000", 2024-09-21T21:40:57.857 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6230000000000001e-06 2024-09-21T21:40:57.857 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.857 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.857 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:57.858 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.062056+0000", 2024-09-21T21:40:57.858 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9500000000000004e-06 2024-09-21T21:40:57.858 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.858 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.858 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:57.858 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.062065+0000", 2024-09-21T21:40:57.858 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3749999999999992e-06 2024-09-21T21:40:57.858 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.858 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.858 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:57.858 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.074584+0000", 2024-09-21T21:40:57.858 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.012518692 2024-09-21T21:40:57.858 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.858 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.858 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:57.859 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.074656+0000", 2024-09-21T21:40:57.859 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1945000000000003e-05 2024-09-21T21:40:57.859 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.859 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.859 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:57.859 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.074775+0000", 2024-09-21T21:40:57.859 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000119271 2024-09-21T21:40:57.859 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:57.859 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:57.859 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:57.859 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.859 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.859 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4710.0:816 3.16 3:6f3307d0:::rbd_data.1121343e7d74.0000000000000043:8 [read 176128~20480] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T21:40:57.859 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:53.062072+0000", 2024-09-21T21:40:57.860 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.7356682699999997, 2024-09-21T21:40:57.860 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.012808775, 2024-09-21T21:40:57.860 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:57.860 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:57.860 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:40:57.860 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:57.860 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4710", 2024-09-21T21:40:57.860 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1566290539", 2024-09-21T21:40:57.860 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 816 2024-09-21T21:40:57.860 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.860 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:57.860 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.860 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:57.860 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.062072+0000", 2024-09-21T21:40:57.860 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:57.861 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.861 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.861 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:57.861 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.062072+0000", 2024-09-21T21:40:57.861 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:57.861 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.861 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.861 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:57.861 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.062073+0000", 2024-09-21T21:40:57.861 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8699999999999995e-07 2024-09-21T21:40:57.861 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.861 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.861 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:57.861 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.062078+0000", 2024-09-21T21:40:57.861 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6720000000000001e-06 2024-09-21T21:40:57.862 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.862 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.862 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:57.862 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.062086+0000", 2024-09-21T21:40:57.862 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.011e-06 2024-09-21T21:40:57.862 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.862 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.862 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:57.862 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.062098+0000", 2024-09-21T21:40:57.862 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1722999999999999e-05 2024-09-21T21:40:57.862 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.862 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.862 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:57.862 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.074790+0000", 2024-09-21T21:40:57.862 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.012692195 2024-09-21T21:40:57.863 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.863 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.863 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:57.863 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.074825+0000", 2024-09-21T21:40:57.863 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4474999999999998e-05 2024-09-21T21:40:57.863 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.863 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.863 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:57.863 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.074881+0000", 2024-09-21T21:40:57.863 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6212000000000003e-05 2024-09-21T21:40:57.863 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:57.863 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:57.863 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:57.863 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.863 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.864 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4710.0:817 3.16 3:6f3307d0:::rbd_data.1121343e7d74.0000000000000043:8 [read 241664~20480] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T21:40:57.864 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:53.062107+0000", 2024-09-21T21:40:57.864 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.735633097, 2024-09-21T21:40:57.864 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.013004898000000001, 2024-09-21T21:40:57.864 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:57.864 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:57.864 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:40:57.864 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:57.864 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4710", 2024-09-21T21:40:57.864 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1566290539", 2024-09-21T21:40:57.864 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 817 2024-09-21T21:40:57.864 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.864 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:57.864 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.864 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:57.864 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.062107+0000", 2024-09-21T21:40:57.865 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:57.865 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.865 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.865 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:57.865 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.062107+0000", 2024-09-21T21:40:57.865 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:57.865 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.865 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.865 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:57.865 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.062108+0000", 2024-09-21T21:40:57.865 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6600000000000002e-07 2024-09-21T21:40:57.865 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.865 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.865 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:57.865 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.062110+0000", 2024-09-21T21:40:57.866 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1710000000000001e-06 2024-09-21T21:40:57.866 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.866 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.866 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:57.866 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.062117+0000", 2024-09-21T21:40:57.866 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8990000000000004e-06 2024-09-21T21:40:57.866 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.866 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.866 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:57.866 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.062123+0000", 2024-09-21T21:40:57.866 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7440000000000004e-06 2024-09-21T21:40:57.866 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.866 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.866 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:57.867 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.074913+0000", 2024-09-21T21:40:57.867 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0127899 2024-09-21T21:40:57.867 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.867 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.867 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:57.867 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.075009+0000", 2024-09-21T21:40:57.867 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.6095999999999996e-05 2024-09-21T21:40:57.867 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.867 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.867 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:57.867 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.075112+0000", 2024-09-21T21:40:57.867 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010362200000000001 2024-09-21T21:40:57.867 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:57.867 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:57.867 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:57.868 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.868 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.868 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4710.0:818 3.16 3:6f3307d0:::rbd_data.1121343e7d74.0000000000000043:8 [read 307200~20480] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T21:40:57.868 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:53.062128+0000", 2024-09-21T21:40:57.868 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.735612132, 2024-09-21T21:40:57.868 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.013226366, 2024-09-21T21:40:57.868 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:57.868 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:57.868 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:40:57.868 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:57.868 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4710", 2024-09-21T21:40:57.868 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1566290539", 2024-09-21T21:40:57.868 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 818 2024-09-21T21:40:57.868 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.868 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:57.869 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.869 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:57.869 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.062128+0000", 2024-09-21T21:40:57.869 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:57.869 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.869 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.869 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:57.869 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.062128+0000", 2024-09-21T21:40:57.869 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:57.869 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.869 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.869 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:57.869 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.062129+0000", 2024-09-21T21:40:57.869 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5900000000000003e-07 2024-09-21T21:40:57.869 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.870 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.870 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:57.870 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.062131+0000", 2024-09-21T21:40:57.870 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9400000000000001e-06 2024-09-21T21:40:57.870 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.870 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.870 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:57.870 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.062136+0000", 2024-09-21T21:40:57.870 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6250000000000004e-06 2024-09-21T21:40:57.870 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.870 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.870 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:57.870 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.062141+0000", 2024-09-21T21:40:57.870 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4610000000000001e-06 2024-09-21T21:40:57.871 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.871 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.871 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:57.871 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.075138+0000", 2024-09-21T21:40:57.871 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.012997234 2024-09-21T21:40:57.871 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.871 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.871 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:57.871 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.075177+0000", 2024-09-21T21:40:57.871 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9283000000000003e-05 2024-09-21T21:40:57.871 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.871 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.871 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:57.871 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.075355+0000", 2024-09-21T21:40:57.871 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00017746400000000001 2024-09-21T21:40:57.872 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:57.872 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:57.872 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:57.872 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.872 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.872 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4710.0:819 3.16 3:6f3307d0:::rbd_data.1121343e7d74.0000000000000043:8 [read 372736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T21:40:57.872 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:53.062146+0000", 2024-09-21T21:40:57.872 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.7355947480000005, 2024-09-21T21:40:57.872 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.013401115999999999, 2024-09-21T21:40:57.872 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:57.872 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:57.872 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:40:57.872 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:57.872 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4710", 2024-09-21T21:40:57.872 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1566290539", 2024-09-21T21:40:57.873 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 819 2024-09-21T21:40:57.873 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.873 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:57.873 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.873 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:57.873 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.062146+0000", 2024-09-21T21:40:57.873 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:57.873 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.873 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.873 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:57.873 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.062146+0000", 2024-09-21T21:40:57.873 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:57.873 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.873 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.873 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:57.873 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.062146+0000", 2024-09-21T21:40:57.874 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6399999999999998e-07 2024-09-21T21:40:57.874 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.874 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.874 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:57.874 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.062148+0000", 2024-09-21T21:40:57.874 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.8810000000000001e-06 2024-09-21T21:40:57.874 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.874 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.874 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:57.874 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.062154+0000", 2024-09-21T21:40:57.874 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6400000000000002e-06 2024-09-21T21:40:57.874 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.874 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.874 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:57.874 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.062158+0000", 2024-09-21T21:40:57.875 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3499999999999999e-06 2024-09-21T21:40:57.875 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.875 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.875 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:57.875 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.075388+0000", 2024-09-21T21:40:57.875 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.013230398000000001 2024-09-21T21:40:57.875 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.875 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.875 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:57.875 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.075480+0000", 2024-09-21T21:40:57.875 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.1137999999999994e-05 2024-09-21T21:40:57.875 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.875 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.875 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:57.875 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.075547+0000", 2024-09-21T21:40:57.876 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7345e-05 2024-09-21T21:40:57.876 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:57.876 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:57.876 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:57.876 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.876 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.876 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4710.0:820 3.16 3:6f3307d0:::rbd_data.1121343e7d74.0000000000000043:8 [read 438272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T21:40:57.876 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:53.062163+0000", 2024-09-21T21:40:57.876 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.7355770790000005, 2024-09-21T21:40:57.876 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.013481887999999999, 2024-09-21T21:40:57.876 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:57.876 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:57.876 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:40:57.876 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:57.876 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4710", 2024-09-21T21:40:57.877 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1566290539", 2024-09-21T21:40:57.877 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 820 2024-09-21T21:40:57.877 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.877 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:57.877 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.877 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:57.877 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.062163+0000", 2024-09-21T21:40:57.877 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:57.877 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.877 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.877 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:57.877 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.062163+0000", 2024-09-21T21:40:57.877 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:57.877 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.877 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.877 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:57.878 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.062164+0000", 2024-09-21T21:40:57.878 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9099999999999999e-07 2024-09-21T21:40:57.878 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.878 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.878 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:57.878 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.062166+0000", 2024-09-21T21:40:57.878 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.939e-06 2024-09-21T21:40:57.878 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.878 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.878 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:57.878 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.062173+0000", 2024-09-21T21:40:57.878 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4549999999999998e-06 2024-09-21T21:40:57.878 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.878 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.879 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:57.879 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.062181+0000", 2024-09-21T21:40:57.879 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4710000000000001e-06 2024-09-21T21:40:57.879 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.879 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.879 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:57.879 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.075572+0000", 2024-09-21T21:40:57.879 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.013390868 2024-09-21T21:40:57.879 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.879 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.879 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:57.879 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.075608+0000", 2024-09-21T21:40:57.879 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6433000000000002e-05 2024-09-21T21:40:57.879 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.879 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.880 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:57.880 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.075645+0000", 2024-09-21T21:40:57.880 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7330999999999999e-05 2024-09-21T21:40:57.880 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:57.880 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:57.880 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:57.880 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.880 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.880 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4710.0:821 3.16 3:6f3307d0:::rbd_data.1121343e7d74.0000000000000043:8 [read 503808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-21T21:40:57.880 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:53.062186+0000", 2024-09-21T21:40:57.880 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.7355541480000003, 2024-09-21T21:40:57.880 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.01362532, 2024-09-21T21:40:57.880 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:40:57.880 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:40:57.880 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:40:57.881 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:40:57.881 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4710", 2024-09-21T21:40:57.881 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1566290539", 2024-09-21T21:40:57.881 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 821 2024-09-21T21:40:57.881 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.881 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:40:57.881 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.881 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:40:57.881 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.062186+0000", 2024-09-21T21:40:57.881 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:57.881 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.881 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.881 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:40:57.881 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.062186+0000", 2024-09-21T21:40:57.881 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:40:57.882 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.882 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.882 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:40:57.882 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.062187+0000", 2024-09-21T21:40:57.882 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5899999999999996e-07 2024-09-21T21:40:57.882 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.882 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.882 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:40:57.882 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.062189+0000", 2024-09-21T21:40:57.882 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9369999999999998e-06 2024-09-21T21:40:57.882 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.882 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.882 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:40:57.882 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.062197+0000", 2024-09-21T21:40:57.882 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7919999999999993e-06 2024-09-21T21:40:57.883 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.883 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.883 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:40:57.883 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.062202+0000", 2024-09-21T21:40:57.883 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9679999999999997e-06 2024-09-21T21:40:57.883 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.883 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.883 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:40:57.883 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.075670+0000", 2024-09-21T21:40:57.883 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.013468826999999999 2024-09-21T21:40:57.883 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.883 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.883 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:40:57.883 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.075743+0000", 2024-09-21T21:40:57.883 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2543e-05 2024-09-21T21:40:57.884 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:40:57.884 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:40:57.884 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:40:57.884 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:53.075812+0000", 2024-09-21T21:40:57.884 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8694000000000003e-05 2024-09-21T21:40:57.884 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:57.884 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:57.884 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:57.884 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:40:57.884 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:40:57.884 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:40:57.885 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T21:40:57.886 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:40:57.884+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:40:57.912 INFO:teuthology.orchestra.run.smithi164.stdout:checking clone #6, image image_client.0-clone6 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent7 2024-09-21T21:40:57.986 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:40:57.984+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:40:57.993 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:40:57.993 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:40:57.994 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 0 2024-09-21T21:40:57.994 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:40:58.001 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T21:40:58.087 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:40:58.085+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:40:58.172 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:40:58.172 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:40:58.172 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:40:58.172 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:40:58.172 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:40:58.180 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T21:40:58.222 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:40:58.220+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:40:58.289 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:40:58.288+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:40:58.326 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:40:58.326 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 20, 2024-09-21T21:40:58.326 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 600, 2024-09-21T21:40:58.326 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:40:58.327 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.327 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:227 3.6 3:678f83b3:::rbd_data.11214be7c411.000000000000002a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:58.327 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.553729+0000", 2024-09-21T21:40:58.327 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 14.770397881999999, 2024-09-21T21:40:58.327 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0090411439999999992, 2024-09-21T21:40:58.327 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:58.327 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:58.327 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:58.327 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:58.327 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:58.327 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:58.327 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 227 2024-09-21T21:40:58.327 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.327 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:58.328 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.328 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:58.328 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.553729+0000", 2024-09-21T21:40:58.328 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:58.328 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.328 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.328 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:58.328 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.553729+0000", 2024-09-21T21:40:58.328 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:58.328 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.328 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.328 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:58.328 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.553729+0000", 2024-09-21T21:40:58.328 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.06e-07 2024-09-21T21:40:58.328 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.329 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.329 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:58.329 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.553771+0000", 2024-09-21T21:40:58.329 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2042999999999997e-05 2024-09-21T21:40:58.329 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.329 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.329 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:58.329 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.553779+0000", 2024-09-21T21:40:58.329 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.3440000000000003e-06 2024-09-21T21:40:58.329 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.329 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.329 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:58.329 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.553784+0000", 2024-09-21T21:40:58.330 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3419999999999998e-06 2024-09-21T21:40:58.330 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.330 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.330 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:58.330 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.553831+0000", 2024-09-21T21:40:58.330 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6826000000000002e-05 2024-09-21T21:40:58.330 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.330 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.330 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:58.330 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.553914+0000", 2024-09-21T21:40:58.330 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.2816000000000001e-05 2024-09-21T21:40:58.330 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.330 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.330 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 1", 2024-09-21T21:40:58.330 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.554149+0000", 2024-09-21T21:40:58.331 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00023488900000000001 2024-09-21T21:40:58.331 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.331 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.331 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:58.331 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.558852+0000", 2024-09-21T21:40:58.331 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0047028399999999998 2024-09-21T21:40:58.331 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.331 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.331 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:40:58.331 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.562673+0000", 2024-09-21T21:40:58.331 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0038215929999999999 2024-09-21T21:40:58.331 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.331 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.331 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:58.332 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.562717+0000", 2024-09-21T21:40:58.332 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4255999999999998e-05 2024-09-21T21:40:58.332 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.332 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.332 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:58.332 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.562770+0000", 2024-09-21T21:40:58.332 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2689000000000003e-05 2024-09-21T21:40:58.332 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:58.332 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:58.332 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:58.332 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.332 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.332 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:228 3.6 3:678f83b3:::rbd_data.11214be7c411.000000000000002a:head [delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:58.332 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.553796+0000", 2024-09-21T21:40:58.333 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 14.770330497, 2024-09-21T21:40:58.333 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0098649099999999993, 2024-09-21T21:40:58.333 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:58.333 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:58.333 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:40:58.333 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:58.333 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:58.333 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:58.333 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 228 2024-09-21T21:40:58.333 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.333 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:58.333 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.333 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:58.333 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.553796+0000", 2024-09-21T21:40:58.333 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:58.333 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.334 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.334 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:58.334 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.553796+0000", 2024-09-21T21:40:58.334 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:58.334 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.334 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.334 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:58.334 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.553797+0000", 2024-09-21T21:40:58.334 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6899999999999997e-07 2024-09-21T21:40:58.334 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.334 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.334 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:58.334 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.553799+0000", 2024-09-21T21:40:58.334 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3e-06 2024-09-21T21:40:58.335 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.335 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.335 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:58.335 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.553805+0000", 2024-09-21T21:40:58.335 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1310000000000003e-06 2024-09-21T21:40:58.335 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.335 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.335 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:58.335 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.553816+0000", 2024-09-21T21:40:58.335 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0506000000000001e-05 2024-09-21T21:40:58.335 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.335 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.335 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:58.335 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.555870+0000", 2024-09-21T21:40:58.335 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0020538510000000002 2024-09-21T21:40:58.336 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.336 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.336 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:58.336 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.555942+0000", 2024-09-21T21:40:58.336 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.2436999999999998e-05 2024-09-21T21:40:58.336 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.336 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.336 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 1", 2024-09-21T21:40:58.336 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556121+0000", 2024-09-21T21:40:58.336 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000178526 2024-09-21T21:40:58.336 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.336 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.336 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:40:58.336 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.559481+0000", 2024-09-21T21:40:58.336 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0033603019999999999 2024-09-21T21:40:58.336 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.337 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.337 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:40:58.337 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.563144+0000", 2024-09-21T21:40:58.337 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0036625799999999999 2024-09-21T21:40:58.337 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.337 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.337 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:40:58.337 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.563185+0000", 2024-09-21T21:40:58.337 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1084000000000002e-05 2024-09-21T21:40:58.337 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.337 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.337 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:58.337 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.563661+0000", 2024-09-21T21:40:58.337 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00047662399999999998 2024-09-21T21:40:58.337 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:58.338 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:58.338 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:58.338 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.338 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.338 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:436 3.17 3:ea5ebfbf:::rbd_data.11214be7c411.000000000000006c:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:40:58.338 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.590210+0000", 2024-09-21T21:40:58.338 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 14.733916531, 2024-09-21T21:40:58.338 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0083415569999999994, 2024-09-21T21:40:58.338 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:58.338 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:58.338 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:40:58.338 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:58.338 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:40:58.338 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:40:58.338 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 436 2024-09-21T21:40:58.339 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.339 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:58.339 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.339 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:58.339 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.590210+0000", 2024-09-21T21:40:58.339 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:58.339 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.339 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.339 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:58.339 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.590210+0000", 2024-09-21T21:40:58.339 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:58.339 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.339 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.340 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:58.340 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.590211+0000", 2024-09-21T21:40:58.340 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.8899999999999998e-07 2024-09-21T21:40:58.340 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.340 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.340 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:58.340 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.590222+0000", 2024-09-21T21:40:58.340 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0473e-05 2024-09-21T21:40:58.340 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.340 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.340 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:58.340 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.590235+0000", 2024-09-21T21:40:58.340 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.3003000000000001e-05 2024-09-21T21:40:58.340 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.340 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.341 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:58.341 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.590242+0000", 2024-09-21T21:40:58.341 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7749999999999993e-06 2024-09-21T21:40:58.341 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.341 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.341 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:58.341 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.590293+0000", 2024-09-21T21:40:58.341 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0133000000000003e-05 2024-09-21T21:40:58.341 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.341 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.341 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:58.341 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.590397+0000", 2024-09-21T21:40:58.341 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000104443 2024-09-21T21:40:58.341 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.342 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.342 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:58.342 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.598552+0000", 2024-09-21T21:40:58.342 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0081548409999999995 2024-09-21T21:40:58.342 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:58.342 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:58.342 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:58.342 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.342 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.342 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4555.0:819 3.6 3:66e91e08:::rbd_data.1121889d185d.0000000000000000:9 [sparse-read 16056320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-21T21:40:58.342 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:56.513732+0000", 2024-09-21T21:40:58.342 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 1.8103943139999998, 2024-09-21T21:40:58.342 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0073956619999999999, 2024-09-21T21:40:58.342 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:58.342 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:58.343 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:40:58.343 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:58.343 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4555", 2024-09-21T21:40:58.343 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3402748328", 2024-09-21T21:40:58.343 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 819 2024-09-21T21:40:58.343 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.343 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:58.343 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.343 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:58.343 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.513732+0000", 2024-09-21T21:40:58.343 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:58.343 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.343 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.343 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:58.343 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.513732+0000", 2024-09-21T21:40:58.344 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:58.344 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.344 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.344 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:58.344 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.513733+0000", 2024-09-21T21:40:58.344 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0900000000000002e-07 2024-09-21T21:40:58.344 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.344 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.344 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:58.344 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.513735+0000", 2024-09-21T21:40:58.344 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2239999999999998e-06 2024-09-21T21:40:58.344 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.344 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.344 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:58.344 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.513741+0000", 2024-09-21T21:40:58.345 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0260000000000004e-06 2024-09-21T21:40:58.345 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.345 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.345 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:58.345 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.513746+0000", 2024-09-21T21:40:58.345 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.493e-06 2024-09-21T21:40:58.345 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.345 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.345 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:58.345 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.521003+0000", 2024-09-21T21:40:58.345 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0072573669999999998 2024-09-21T21:40:58.345 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.345 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.345 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:58.345 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.521058+0000", 2024-09-21T21:40:58.346 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4815999999999998e-05 2024-09-21T21:40:58.346 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.346 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.346 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:58.346 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.521128+0000", 2024-09-21T21:40:58.346 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0227000000000004e-05 2024-09-21T21:40:58.346 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:58.346 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:58.346 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:58.346 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.346 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.346 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4555.0:1372 3.6 3:66e91e08:::rbd_data.1121889d185d.0000000000000000:9 [sparse-read 6094848~2560] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-21T21:40:58.346 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:56.542275+0000", 2024-09-21T21:40:58.346 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 1.781851769, 2024-09-21T21:40:58.346 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0074430920000000001, 2024-09-21T21:40:58.347 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:58.347 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:58.347 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:40:58.347 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:58.347 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4555", 2024-09-21T21:40:58.347 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3402748328", 2024-09-21T21:40:58.347 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1372 2024-09-21T21:40:58.347 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.347 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:58.347 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.347 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:58.347 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.542275+0000", 2024-09-21T21:40:58.347 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:58.347 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.347 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.348 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:58.348 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.542275+0000", 2024-09-21T21:40:58.348 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:58.348 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.348 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.348 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:58.348 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.542275+0000", 2024-09-21T21:40:58.348 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0100000000000001e-07 2024-09-21T21:40:58.348 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.348 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.348 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:58.348 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.542276+0000", 2024-09-21T21:40:58.348 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.049e-06 2024-09-21T21:40:58.348 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.348 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.348 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:58.349 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.542279+0000", 2024-09-21T21:40:58.349 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.9900000000000002e-06 2024-09-21T21:40:58.349 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.349 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.349 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:58.349 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.542282+0000", 2024-09-21T21:40:58.349 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.531e-06 2024-09-21T21:40:58.349 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.349 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.349 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:58.349 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.549338+0000", 2024-09-21T21:40:58.349 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0070564479999999999 2024-09-21T21:40:58.349 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.349 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.349 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:58.350 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.549408+0000", 2024-09-21T21:40:58.350 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0007999999999998e-05 2024-09-21T21:40:58.350 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.350 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.350 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:58.350 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.549718+0000", 2024-09-21T21:40:58.350 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00030986499999999999 2024-09-21T21:40:58.350 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:58.350 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:58.350 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:58.350 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.350 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.350 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4555.0:1374 3.6 3:66e91e08:::rbd_data.1121889d185d.0000000000000000:9 [sparse-read 7208960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-21T21:40:58.350 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:56.542285+0000", 2024-09-21T21:40:58.350 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 1.781842014, 2024-09-21T21:40:58.351 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0075585130000000002, 2024-09-21T21:40:58.351 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:58.351 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:58.351 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:40:58.351 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:58.351 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4555", 2024-09-21T21:40:58.351 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3402748328", 2024-09-21T21:40:58.351 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1374 2024-09-21T21:40:58.351 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.351 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:58.351 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.351 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:58.351 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.542285+0000", 2024-09-21T21:40:58.351 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:58.352 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.352 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.352 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:58.352 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.542285+0000", 2024-09-21T21:40:58.352 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:58.352 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.352 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.352 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:58.352 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.542285+0000", 2024-09-21T21:40:58.352 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.05e-07 2024-09-21T21:40:58.352 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.352 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.352 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:58.352 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.542286+0000", 2024-09-21T21:40:58.352 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0189999999999999e-06 2024-09-21T21:40:58.353 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.353 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.353 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:58.353 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.542290+0000", 2024-09-21T21:40:58.353 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8249999999999998e-06 2024-09-21T21:40:58.353 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.353 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.353 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:58.353 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.542293+0000", 2024-09-21T21:40:58.353 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.54e-06 2024-09-21T21:40:58.353 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.353 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.353 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:58.353 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.549746+0000", 2024-09-21T21:40:58.353 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0074526799999999997 2024-09-21T21:40:58.354 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.354 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.354 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:58.354 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.549795+0000", 2024-09-21T21:40:58.354 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9203000000000003e-05 2024-09-21T21:40:58.354 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.354 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.354 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:58.354 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.549843+0000", 2024-09-21T21:40:58.354 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8041000000000003e-05 2024-09-21T21:40:58.354 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:58.354 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:58.354 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:58.354 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.355 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.355 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4555.0:1376 3.6 3:66e91e08:::rbd_data.1121889d185d.0000000000000000:9 [sparse-read 7798784~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-21T21:40:58.355 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:56.542297+0000", 2024-09-21T21:40:58.355 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 1.7818302990000001, 2024-09-21T21:40:58.355 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0076504800000000003, 2024-09-21T21:40:58.355 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:58.355 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:58.355 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:40:58.355 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:58.355 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4555", 2024-09-21T21:40:58.355 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3402748328", 2024-09-21T21:40:58.355 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1376 2024-09-21T21:40:58.355 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.355 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:58.355 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.356 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:58.356 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.542297+0000", 2024-09-21T21:40:58.356 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:58.356 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.356 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.356 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:58.356 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.542297+0000", 2024-09-21T21:40:58.356 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:58.356 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.356 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.356 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:58.356 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.542297+0000", 2024-09-21T21:40:58.356 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.16e-07 2024-09-21T21:40:58.356 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.356 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.357 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:58.357 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.542298+0000", 2024-09-21T21:40:58.357 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.3689999999999999e-06 2024-09-21T21:40:58.357 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.357 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.357 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:58.357 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.542301+0000", 2024-09-21T21:40:58.357 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0869999999999998e-06 2024-09-21T21:40:58.357 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.357 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.357 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:58.357 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.542304+0000", 2024-09-21T21:40:58.357 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.5339999999999998e-06 2024-09-21T21:40:58.357 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.357 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.358 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:58.358 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.549861+0000", 2024-09-21T21:40:58.358 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0075574860000000004 2024-09-21T21:40:58.358 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.358 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.358 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:58.358 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.549895+0000", 2024-09-21T21:40:58.358 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3664999999999997e-05 2024-09-21T21:40:58.358 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.358 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.358 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:58.358 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.549947+0000", 2024-09-21T21:40:58.358 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2123000000000002e-05 2024-09-21T21:40:58.358 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:58.359 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:58.359 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:58.359 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.359 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.359 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4555.0:1378 3.6 3:66e91e08:::rbd_data.1121889d185d.0000000000000000:9 [sparse-read 8912896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-21T21:40:58.359 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:56.542307+0000", 2024-09-21T21:40:58.359 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 1.781820095, 2024-09-21T21:40:58.359 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0078200690000000007, 2024-09-21T21:40:58.359 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:58.359 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:58.359 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:40:58.359 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:58.359 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4555", 2024-09-21T21:40:58.359 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3402748328", 2024-09-21T21:40:58.360 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1378 2024-09-21T21:40:58.360 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.360 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:58.360 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.360 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:58.360 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.542307+0000", 2024-09-21T21:40:58.360 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:58.360 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.360 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.360 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:58.360 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.542307+0000", 2024-09-21T21:40:58.360 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:58.360 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.360 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.360 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:58.361 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.542307+0000", 2024-09-21T21:40:58.361 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0699999999999999e-07 2024-09-21T21:40:58.361 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.361 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.361 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:58.361 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.542311+0000", 2024-09-21T21:40:58.361 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4449999999999998e-06 2024-09-21T21:40:58.361 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.361 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.361 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:58.361 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.542315+0000", 2024-09-21T21:40:58.361 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3060000000000002e-06 2024-09-21T21:40:58.361 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.361 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.361 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:58.362 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.542317+0000", 2024-09-21T21:40:58.362 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.5509999999999998e-06 2024-09-21T21:40:58.362 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.362 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.362 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:58.362 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.549976+0000", 2024-09-21T21:40:58.362 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0076583440000000001 2024-09-21T21:40:58.362 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.362 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.362 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:58.362 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.550042+0000", 2024-09-21T21:40:58.362 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6878000000000006e-05 2024-09-21T21:40:58.362 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.362 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.363 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:58.363 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.550127+0000", 2024-09-21T21:40:58.363 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.4338000000000005e-05 2024-09-21T21:40:58.363 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:58.363 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:58.363 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:58.363 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.363 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.363 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4555.0:1380 3.6 3:66e91e08:::rbd_data.1121889d185d.0000000000000000:9 [sparse-read 9502720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-21T21:40:58.363 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:56.542321+0000", 2024-09-21T21:40:58.363 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 1.7818063, 2024-09-21T21:40:58.363 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0080062320000000003, 2024-09-21T21:40:58.363 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:58.363 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:58.363 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:40:58.364 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:58.364 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4555", 2024-09-21T21:40:58.364 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3402748328", 2024-09-21T21:40:58.364 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1380 2024-09-21T21:40:58.364 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.364 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:58.364 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.364 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:58.364 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.542321+0000", 2024-09-21T21:40:58.364 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:58.364 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.364 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.364 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:58.364 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.542321+0000", 2024-09-21T21:40:58.364 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:58.364 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.365 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.365 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:58.365 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.542321+0000", 2024-09-21T21:40:58.365 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2499999999999999e-07 2024-09-21T21:40:58.365 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.365 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.365 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:58.365 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.542322+0000", 2024-09-21T21:40:58.365 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1689999999999999e-06 2024-09-21T21:40:58.365 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.365 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.365 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:58.365 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.542325+0000", 2024-09-21T21:40:58.365 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5669999999999999e-06 2024-09-21T21:40:58.365 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.366 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.366 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:58.366 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.542328+0000", 2024-09-21T21:40:58.366 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.6010000000000002e-06 2024-09-21T21:40:58.366 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.366 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.366 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:58.366 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.550157+0000", 2024-09-21T21:40:58.366 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0078292629999999995 2024-09-21T21:40:58.366 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.366 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.366 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:58.366 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.550230+0000", 2024-09-21T21:40:58.366 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.2480999999999999e-05 2024-09-21T21:40:58.367 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.367 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.367 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:58.367 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.550327+0000", 2024-09-21T21:40:58.367 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6926e-05 2024-09-21T21:40:58.367 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:58.367 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:58.367 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:58.367 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.367 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.367 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4555.0:1382 3.6 3:66e91e08:::rbd_data.1121889d185d.0000000000000000:9 [sparse-read 10616832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-21T21:40:58.367 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:56.542332+0000", 2024-09-21T21:40:58.367 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 1.7817952639999999, 2024-09-21T21:40:58.367 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0081892930000000003, 2024-09-21T21:40:58.368 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:58.368 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:58.368 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:40:58.368 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:58.368 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4555", 2024-09-21T21:40:58.368 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3402748328", 2024-09-21T21:40:58.368 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1382 2024-09-21T21:40:58.368 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.368 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:58.368 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.368 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:58.368 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.542332+0000", 2024-09-21T21:40:58.368 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:58.368 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.368 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.369 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:58.369 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.542332+0000", 2024-09-21T21:40:58.369 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:58.369 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.369 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.369 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:58.369 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.542332+0000", 2024-09-21T21:40:58.369 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0699999999999999e-07 2024-09-21T21:40:58.369 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.369 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.369 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:58.369 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.542333+0000", 2024-09-21T21:40:58.369 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.023e-06 2024-09-21T21:40:58.369 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.369 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.370 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:58.370 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.542337+0000", 2024-09-21T21:40:58.370 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.907e-06 2024-09-21T21:40:58.370 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.370 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.370 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:58.370 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.542340+0000", 2024-09-21T21:40:58.370 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5410000000000001e-06 2024-09-21T21:40:58.370 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.370 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.370 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:58.370 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.550355+0000", 2024-09-21T21:40:58.370 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0080145400000000006 2024-09-21T21:40:58.370 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.370 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.371 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:58.371 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.550430+0000", 2024-09-21T21:40:58.371 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5346000000000007e-05 2024-09-21T21:40:58.371 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.371 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.371 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:58.371 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.550521+0000", 2024-09-21T21:40:58.371 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.0729000000000005e-05 2024-09-21T21:40:58.371 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:58.371 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:58.371 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:58.371 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.371 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.371 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4555.0:1383 3.6 3:66e91e08:::rbd_data.1121889d185d.0000000000000000:9 [sparse-read 11206656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-21T21:40:58.371 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:56.542343+0000", 2024-09-21T21:40:58.372 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 1.78178357, 2024-09-21T21:40:58.372 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0083518519999999999, 2024-09-21T21:40:58.372 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:58.372 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:58.372 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:40:58.372 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:58.372 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4555", 2024-09-21T21:40:58.372 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3402748328", 2024-09-21T21:40:58.372 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1383 2024-09-21T21:40:58.372 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.372 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:58.372 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.372 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:58.372 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.542343+0000", 2024-09-21T21:40:58.373 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:58.373 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.373 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.373 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:58.373 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.542343+0000", 2024-09-21T21:40:58.373 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:58.373 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.373 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.373 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:58.373 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.542343+0000", 2024-09-21T21:40:58.373 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.22e-07 2024-09-21T21:40:58.373 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.373 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.373 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:58.373 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.542345+0000", 2024-09-21T21:40:58.373 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.198e-06 2024-09-21T21:40:58.374 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.374 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.374 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:58.374 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.542348+0000", 2024-09-21T21:40:58.374 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.095e-06 2024-09-21T21:40:58.374 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.374 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.374 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:58.374 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.542350+0000", 2024-09-21T21:40:58.374 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.57e-06 2024-09-21T21:40:58.374 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.374 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.374 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:58.374 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.550568+0000", 2024-09-21T21:40:58.375 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0082177799999999992 2024-09-21T21:40:58.375 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.375 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.375 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:58.375 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.550633+0000", 2024-09-21T21:40:58.375 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4733999999999994e-05 2024-09-21T21:40:58.375 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.375 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.375 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:58.375 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.550695+0000", 2024-09-21T21:40:58.375 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2253000000000002e-05 2024-09-21T21:40:58.375 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:58.375 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:58.375 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:58.375 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.375 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.376 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4555.0:1384 3.6 3:66e91e08:::rbd_data.1121889d185d.0000000000000000:9 [sparse-read 12320768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-21T21:40:58.376 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:56.542353+0000", 2024-09-21T21:40:58.376 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 1.7817736310000001, 2024-09-21T21:40:58.376 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0085612150000000005, 2024-09-21T21:40:58.376 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:58.376 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:58.376 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:40:58.376 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:58.376 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4555", 2024-09-21T21:40:58.376 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3402748328", 2024-09-21T21:40:58.376 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1384 2024-09-21T21:40:58.376 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.376 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:58.376 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.376 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:58.377 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.542353+0000", 2024-09-21T21:40:58.377 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:58.377 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.377 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.377 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:58.377 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.542353+0000", 2024-09-21T21:40:58.377 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:58.377 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.377 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.377 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:58.377 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.542353+0000", 2024-09-21T21:40:58.377 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.05e-07 2024-09-21T21:40:58.377 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.377 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.377 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:58.378 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.542354+0000", 2024-09-21T21:40:58.378 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0780000000000001e-06 2024-09-21T21:40:58.378 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.378 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.378 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:58.378 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.542357+0000", 2024-09-21T21:40:58.378 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.9730000000000002e-06 2024-09-21T21:40:58.378 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.378 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.378 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:58.378 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.542360+0000", 2024-09-21T21:40:58.378 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.4870000000000001e-06 2024-09-21T21:40:58.378 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.378 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.378 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:58.379 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.550744+0000", 2024-09-21T21:40:58.379 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0083841190000000006 2024-09-21T21:40:58.379 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.379 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.379 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:58.379 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.550817+0000", 2024-09-21T21:40:58.379 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.2723000000000002e-05 2024-09-21T21:40:58.379 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.379 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.379 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:58.379 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.550914+0000", 2024-09-21T21:40:58.379 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.7629999999999999e-05 2024-09-21T21:40:58.379 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:58.379 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:58.379 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:58.380 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.380 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.380 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4555.0:1386 3.6 3:66e91e08:::rbd_data.1121889d185d.0000000000000000:9 [sparse-read 12910592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-21T21:40:58.380 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:56.542363+0000", 2024-09-21T21:40:58.380 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 1.78176398, 2024-09-21T21:40:58.380 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0086866800000000004, 2024-09-21T21:40:58.380 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:58.380 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:58.380 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:40:58.380 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:58.380 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4555", 2024-09-21T21:40:58.380 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3402748328", 2024-09-21T21:40:58.380 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1386 2024-09-21T21:40:58.380 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.380 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:58.381 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.381 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:58.381 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.542363+0000", 2024-09-21T21:40:58.381 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:58.381 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.381 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.381 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:58.381 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.542363+0000", 2024-09-21T21:40:58.381 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:58.381 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.381 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.381 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:58.381 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.542363+0000", 2024-09-21T21:40:58.381 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0200000000000001e-07 2024-09-21T21:40:58.381 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.382 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.382 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:58.382 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.542364+0000", 2024-09-21T21:40:58.382 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1510000000000001e-06 2024-09-21T21:40:58.382 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.382 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.382 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:58.382 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.542367+0000", 2024-09-21T21:40:58.382 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.8310000000000001e-06 2024-09-21T21:40:58.382 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.382 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.382 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:58.382 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.542370+0000", 2024-09-21T21:40:58.382 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.5059999999999998e-06 2024-09-21T21:40:58.382 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.383 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.383 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:58.383 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.550954+0000", 2024-09-21T21:40:58.383 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0085849010000000007 2024-09-21T21:40:58.383 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.383 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.383 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:58.383 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.550995+0000", 2024-09-21T21:40:58.383 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.0349000000000003e-05 2024-09-21T21:40:58.383 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.383 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.383 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:58.383 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.551050+0000", 2024-09-21T21:40:58.383 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4740000000000001e-05 2024-09-21T21:40:58.383 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:58.384 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:58.384 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:58.384 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.384 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.384 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4555.0:1388 3.6 3:66e91e08:::rbd_data.1121889d185d.0000000000000000:9 [sparse-read 14024704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-21T21:40:58.384 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:56.542372+0000", 2024-09-21T21:40:58.384 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 1.781754348, 2024-09-21T21:40:58.384 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0088536650000000001, 2024-09-21T21:40:58.384 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:58.384 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:58.384 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:40:58.384 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:58.384 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4555", 2024-09-21T21:40:58.384 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3402748328", 2024-09-21T21:40:58.384 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1388 2024-09-21T21:40:58.385 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.385 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:58.385 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.385 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:58.385 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.542372+0000", 2024-09-21T21:40:58.385 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:58.385 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.385 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.385 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:58.385 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.542372+0000", 2024-09-21T21:40:58.385 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:58.385 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.385 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.385 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:58.385 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.542373+0000", 2024-09-21T21:40:58.385 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1899999999999999e-07 2024-09-21T21:40:58.386 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.386 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.386 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:58.386 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.542374+0000", 2024-09-21T21:40:58.386 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.083e-06 2024-09-21T21:40:58.386 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.386 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.386 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:58.386 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.542377+0000", 2024-09-21T21:40:58.386 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.8509999999999999e-06 2024-09-21T21:40:58.386 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.386 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.386 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:58.386 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.542379+0000", 2024-09-21T21:40:58.387 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.4360000000000001e-06 2024-09-21T21:40:58.387 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.387 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.387 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:58.387 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.551094+0000", 2024-09-21T21:40:58.387 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0087149389999999997 2024-09-21T21:40:58.387 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.387 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.387 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:58.387 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.551167+0000", 2024-09-21T21:40:58.387 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.2844999999999998e-05 2024-09-21T21:40:58.387 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.387 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.387 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:58.387 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.551226+0000", 2024-09-21T21:40:58.388 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9292000000000001e-05 2024-09-21T21:40:58.388 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:58.388 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:58.388 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:58.388 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.388 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.388 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4555.0:1389 3.6 3:66e91e08:::rbd_data.1121889d185d.0000000000000000:9 [sparse-read 14614528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-21T21:40:58.388 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:56.542382+0000", 2024-09-21T21:40:58.388 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 1.7817447770000001, 2024-09-21T21:40:58.388 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0090268789999999998, 2024-09-21T21:40:58.388 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:58.388 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:58.388 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:40:58.388 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:58.388 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4555", 2024-09-21T21:40:58.389 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3402748328", 2024-09-21T21:40:58.389 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1389 2024-09-21T21:40:58.389 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.389 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:58.389 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.389 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:58.389 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.542382+0000", 2024-09-21T21:40:58.389 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:58.389 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.389 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.389 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:58.389 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.542382+0000", 2024-09-21T21:40:58.389 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:58.389 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.389 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.390 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:58.390 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.542382+0000", 2024-09-21T21:40:58.390 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3999999999999998e-07 2024-09-21T21:40:58.390 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.390 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.390 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:58.390 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.542383+0000", 2024-09-21T21:40:58.390 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.023e-06 2024-09-21T21:40:58.390 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.390 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.390 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:58.390 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.542386+0000", 2024-09-21T21:40:58.390 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.8389999999999998e-06 2024-09-21T21:40:58.390 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.390 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.391 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:58.391 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.542389+0000", 2024-09-21T21:40:58.391 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.9459999999999998e-06 2024-09-21T21:40:58.391 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.391 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.391 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:58.391 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.551271+0000", 2024-09-21T21:40:58.391 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0088821060000000007 2024-09-21T21:40:58.391 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.391 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.391 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:58.391 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.551344+0000", 2024-09-21T21:40:58.391 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.2527000000000005e-05 2024-09-21T21:40:58.391 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.391 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.392 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:58.392 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.551409+0000", 2024-09-21T21:40:58.392 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5197999999999995e-05 2024-09-21T21:40:58.392 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:58.392 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:58.392 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:58.392 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.392 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.392 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4555.0:1390 3.6 3:66e91e08:::rbd_data.1121889d185d.0000000000000000:9 [sparse-read 15728640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-21T21:40:58.392 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:56.542421+0000", 2024-09-21T21:40:58.392 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 1.781706265, 2024-09-21T21:40:58.392 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0091987730000000004, 2024-09-21T21:40:58.392 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:58.392 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:58.392 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:40:58.393 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:58.393 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4555", 2024-09-21T21:40:58.393 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3402748328", 2024-09-21T21:40:58.393 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1390 2024-09-21T21:40:58.393 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.393 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:58.393 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.393 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:58.393 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.542421+0000", 2024-09-21T21:40:58.393 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:58.393 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.393 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.393 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:58.393 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.542421+0000", 2024-09-21T21:40:58.393 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:58.394 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.394 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.394 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:58.394 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.542421+0000", 2024-09-21T21:40:58.394 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.7599999999999996e-07 2024-09-21T21:40:58.394 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.394 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.394 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:58.394 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.542425+0000", 2024-09-21T21:40:58.394 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7519999999999998e-06 2024-09-21T21:40:58.394 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.394 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.394 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:58.394 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.542436+0000", 2024-09-21T21:40:58.394 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1109e-05 2024-09-21T21:40:58.395 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.395 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.395 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:58.395 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.542443+0000", 2024-09-21T21:40:58.395 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9759999999999998e-06 2024-09-21T21:40:58.395 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.395 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.395 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:58.395 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.551451+0000", 2024-09-21T21:40:58.395 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0090076210000000004 2024-09-21T21:40:58.395 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.395 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.395 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:58.395 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.551526+0000", 2024-09-21T21:40:58.395 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5082999999999999e-05 2024-09-21T21:40:58.396 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.396 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.396 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:58.396 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.551619+0000", 2024-09-21T21:40:58.396 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.3356000000000005e-05 2024-09-21T21:40:58.396 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:58.396 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:58.396 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:58.396 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.396 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.396 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4555.0:1392 3.6 3:66e91e08:::rbd_data.1121889d185d.0000000000000000:9 [sparse-read 16318464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-21T21:40:58.396 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:56.542454+0000", 2024-09-21T21:40:58.396 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 1.7816730870000002, 2024-09-21T21:40:58.396 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0093699349999999994, 2024-09-21T21:40:58.396 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:58.397 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:58.397 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:40:58.397 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:58.397 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4555", 2024-09-21T21:40:58.397 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3402748328", 2024-09-21T21:40:58.397 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1392 2024-09-21T21:40:58.397 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.397 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:58.397 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.397 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:58.397 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.542454+0000", 2024-09-21T21:40:58.397 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:58.397 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.397 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.397 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:58.398 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.542454+0000", 2024-09-21T21:40:58.398 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:58.398 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.398 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.398 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:58.398 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.542454+0000", 2024-09-21T21:40:58.398 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.51e-07 2024-09-21T21:40:58.398 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.398 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.398 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:58.398 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.542457+0000", 2024-09-21T21:40:58.398 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.6929999999999999e-06 2024-09-21T21:40:58.398 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.398 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.398 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:58.399 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.542464+0000", 2024-09-21T21:40:58.399 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.8140000000000003e-06 2024-09-21T21:40:58.399 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.399 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.399 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:58.399 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.542469+0000", 2024-09-21T21:40:58.399 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6210000000000001e-06 2024-09-21T21:40:58.399 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.399 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.399 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:58.399 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.551641+0000", 2024-09-21T21:40:58.399 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0091716009999999997 2024-09-21T21:40:58.399 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.399 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.399 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:58.400 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.551726+0000", 2024-09-21T21:40:58.400 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.5204000000000006e-05 2024-09-21T21:40:58.400 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.400 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.400 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:58.400 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.551824+0000", 2024-09-21T21:40:58.400 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.7450999999999999e-05 2024-09-21T21:40:58.400 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:58.400 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:58.400 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:58.400 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.400 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.400 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4555.0:1658 3.6 3:66e91e08:::rbd_data.1121889d185d.0000000000000000:9 [sparse-read 16384000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-21T21:40:58.400 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:56.557526+0000", 2024-09-21T21:40:58.400 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 1.766600886, 2024-09-21T21:40:58.401 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.007427599, 2024-09-21T21:40:58.401 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:58.401 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:58.401 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:40:58.401 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:58.401 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4555", 2024-09-21T21:40:58.401 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3402748328", 2024-09-21T21:40:58.401 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1658 2024-09-21T21:40:58.401 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.401 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:58.401 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.401 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:58.401 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.557526+0000", 2024-09-21T21:40:58.401 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:58.401 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.401 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.402 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:58.402 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.557526+0000", 2024-09-21T21:40:58.402 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:58.402 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.402 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.402 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:58.402 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.557526+0000", 2024-09-21T21:40:58.402 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.72e-07 2024-09-21T21:40:58.402 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.402 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.402 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:58.402 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.557528+0000", 2024-09-21T21:40:58.402 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.99e-06 2024-09-21T21:40:58.402 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.402 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.403 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:58.403 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.557540+0000", 2024-09-21T21:40:58.403 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1472e-05 2024-09-21T21:40:58.403 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.403 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.403 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:58.405 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.557545+0000", 2024-09-21T21:40:58.405 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8500000000000002e-06 2024-09-21T21:40:58.405 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.406 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.406 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:58.406 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.564832+0000", 2024-09-21T21:40:58.406 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0072876590000000002 2024-09-21T21:40:58.406 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.406 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.406 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:58.406 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.564874+0000", 2024-09-21T21:40:58.406 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2111999999999999e-05 2024-09-21T21:40:58.406 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.406 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.406 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:58.406 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.564954+0000", 2024-09-21T21:40:58.406 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.9144e-05 2024-09-21T21:40:58.406 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:58.407 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:58.407 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:58.407 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.407 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.407 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4555.0:1659 3.6 3:66e91e08:::rbd_data.1121889d185d.0000000000000000:9 [sparse-read 15204352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-21T21:40:58.407 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:56.557572+0000", 2024-09-21T21:40:58.407 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 1.7665543779999999, 2024-09-21T21:40:58.407 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0076059789999999997, 2024-09-21T21:40:58.407 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:58.407 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:58.407 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:40:58.407 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:58.407 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4555", 2024-09-21T21:40:58.407 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3402748328", 2024-09-21T21:40:58.407 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1659 2024-09-21T21:40:58.408 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.408 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:58.408 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.408 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:58.408 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.557572+0000", 2024-09-21T21:40:58.408 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:58.408 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.408 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.408 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:58.408 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.557572+0000", 2024-09-21T21:40:58.408 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:58.408 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.408 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.408 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:58.408 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.557573+0000", 2024-09-21T21:40:58.409 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2599999999999998e-07 2024-09-21T21:40:58.409 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.409 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.409 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:58.409 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.557576+0000", 2024-09-21T21:40:58.409 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.757e-06 2024-09-21T21:40:58.409 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.409 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.409 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:58.409 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.557589+0000", 2024-09-21T21:40:58.409 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.3261000000000001e-05 2024-09-21T21:40:58.409 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.409 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.409 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:58.410 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.557610+0000", 2024-09-21T21:40:58.410 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0805000000000002e-05 2024-09-21T21:40:58.410 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.410 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.410 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:58.410 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.564997+0000", 2024-09-21T21:40:58.410 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0073869890000000001 2024-09-21T21:40:58.410 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.410 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.410 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:58.410 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.565080+0000", 2024-09-21T21:40:58.410 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.3733999999999995e-05 2024-09-21T21:40:58.410 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.410 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.411 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:58.411 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.565178+0000", 2024-09-21T21:40:58.411 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.8006999999999999e-05 2024-09-21T21:40:58.411 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:58.411 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:58.411 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:58.411 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.411 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.411 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4555.0:1661 3.6 3:66e91e08:::rbd_data.1121889d185d.0000000000000000:9 [sparse-read 15794176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-21T21:40:58.411 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:56.557618+0000", 2024-09-21T21:40:58.411 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 1.766509254, 2024-09-21T21:40:58.411 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0077992840000000001, 2024-09-21T21:40:58.411 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:58.411 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:58.411 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:40:58.412 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:58.412 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4555", 2024-09-21T21:40:58.412 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3402748328", 2024-09-21T21:40:58.412 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1661 2024-09-21T21:40:58.412 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.412 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:58.412 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.412 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:58.412 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.557618+0000", 2024-09-21T21:40:58.412 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:58.412 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.412 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.412 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:58.412 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.557618+0000", 2024-09-21T21:40:58.412 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:58.413 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.413 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.413 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:58.413 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.557618+0000", 2024-09-21T21:40:58.413 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0100000000000005e-07 2024-09-21T21:40:58.413 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.413 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.413 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:58.413 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.557620+0000", 2024-09-21T21:40:58.413 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0269999999999998e-06 2024-09-21T21:40:58.413 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.413 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.413 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:58.413 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.557636+0000", 2024-09-21T21:40:58.413 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.5763000000000001e-05 2024-09-21T21:40:58.413 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.414 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.414 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:58.414 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.557641+0000", 2024-09-21T21:40:58.414 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4349999999999996e-06 2024-09-21T21:40:58.414 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.414 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.414 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:58.414 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.565222+0000", 2024-09-21T21:40:58.414 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0075805289999999999 2024-09-21T21:40:58.414 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.414 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.414 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:58.414 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.565303+0000", 2024-09-21T21:40:58.414 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0771000000000003e-05 2024-09-21T21:40:58.415 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.415 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.415 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:40:58.415 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:56.565417+0000", 2024-09-21T21:40:58.415 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000114258 2024-09-21T21:40:58.415 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:58.415 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:58.415 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:58.415 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:58.415 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:58.415 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:40:58.417 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T21:40:58.417 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:40:58.415+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:40:58.418 INFO:teuthology.orchestra.run.smithi164.stdout:1032 read 0x1386000 thru 0x138a50b (0x450c bytes) 2024-09-21T21:40:58.418 INFO:teuthology.orchestra.run.smithi164.stdout:1034 read 0x1441800 thru 0x144cd32 (0xb533 bytes) 2024-09-21T21:40:58.418 INFO:teuthology.orchestra.run.smithi164.stdout:1036 read 0x17ad000 thru 0x17b6c51 (0x9c52 bytes) 2024-09-21T21:40:58.418 INFO:teuthology.orchestra.run.smithi164.stdout:1040 read 0x6613600 thru 0x66207de (0xd1df bytes) 2024-09-21T21:40:58.418 INFO:teuthology.orchestra.run.smithi164.stdout:1043 read 0x2418800 thru 0x2424d2c (0xc52d bytes) 2024-09-21T21:40:58.419 INFO:teuthology.orchestra.run.smithi164.stdout:1044 punch from 0x2f95e00 to 0x2f96000, (0x200 bytes) 2024-09-21T21:40:58.419 INFO:teuthology.orchestra.run.smithi164.stdout:1045 write 0x2d01400 thru 0x2d03a1e (0x261f bytes) 2024-09-21T21:40:58.419 INFO:teuthology.orchestra.run.smithi164.stdout:1046 write 0x80a9c00 thru 0x80b4f70 (0xb371 bytes) 2024-09-21T21:40:58.419 INFO:teuthology.orchestra.run.smithi164.stdout:1047 write 0x8023c00 thru 0x8030578 (0xc979 bytes) 2024-09-21T21:40:58.419 INFO:teuthology.orchestra.run.smithi164.stdout:1048 write 0x8153a00 thru 0x816359e (0xfb9f bytes) 2024-09-21T21:40:58.419 INFO:teuthology.orchestra.run.smithi164.stdout:1049 write 0x8470400 thru 0x84711a2 (0xda3 bytes) 2024-09-21T21:40:58.419 INFO:teuthology.orchestra.run.smithi164.stdout:1050 write 0x8335a00 thru 0x833ce74 (0x7475 bytes) 2024-09-21T21:40:58.419 INFO:teuthology.orchestra.run.smithi164.stdout:1051 trunc from 0xa711200 to 0xe06a400 2024-09-21T21:40:58.443 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T21:40:58.476 INFO:teuthology.orchestra.run.smithi164.stdout:1055 read 0x12dce00 thru 0x12e3622 (0x6823 bytes) 2024-09-21T21:40:58.478 INFO:teuthology.orchestra.run.smithi164.stdout:1060 write 0xe027000 thru 0xe032927 (0xb928 bytes) 2024-09-21T21:40:58.478 INFO:teuthology.orchestra.run.smithi164.stdout:1061 punch from 0x58b8a00 to 0x58bc000, (0x3600 bytes) 2024-09-21T21:40:58.481 INFO:teuthology.orchestra.run.smithi164.stdout:1063 read 0x6ae6200 thru 0x6ae6c2d (0xa2e bytes) 2024-09-21T21:40:58.503 INFO:teuthology.orchestra.run.smithi164.stdout:1064 write 0x1996800 thru 0x199de83 (0x7684 bytes) 2024-09-21T21:40:58.503 INFO:teuthology.orchestra.run.smithi164.stdout:1065 punch from 0x17e8400 to 0x17f4c00, (0xc800 bytes) 2024-09-21T21:40:58.517 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:40:58.516+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:40:58.529 INFO:teuthology.orchestra.run.smithi081.stderr:set 3.4 pg_upmap_items mapping to [0->4,5->3] 2024-09-21T21:40:58.537 INFO:teuthology.orchestra.run.smithi164.stdout:1066 write 0xa826000 thru 0xa828ce5 (0x2ce6 bytes) 2024-09-21T21:40:58.537 INFO:teuthology.orchestra.run.smithi164.stdout:1070 read 0x82c0c00 thru 0x82cb34f (0xa750 bytes) 2024-09-21T21:40:58.585 INFO:teuthology.orchestra.run.smithi164.stdout:1071 read 0xa4bc000 thru 0xa4c282f (0x6830 bytes) 2024-09-21T21:40:58.586 INFO:teuthology.orchestra.run.smithi164.stdout:1074 write 0x9443200 thru 0x9450bc2 (0xd9c3 bytes) 2024-09-21T21:40:58.587 INFO:teuthology.orchestra.run.smithi164.stdout:1077 punch from 0x84ec600 to 0x84f9600, (0xd000 bytes) 2024-09-21T21:40:58.605 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:40:58.605 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:40:58.605 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.605 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:6301 3.0 3.eaff32e0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:40:58.605 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.593713+0000", 2024-09-21T21:40:58.605 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.0057600100000000003, 2024-09-21T21:40:58.605 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0057815289999999997, 2024-09-21T21:40:58.605 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:58.605 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:58.606 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:40:58.606 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:58.606 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:40:58.606 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:40:58.606 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 6301 2024-09-21T21:40:58.606 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.606 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:58.606 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.606 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:58.606 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.593713+0000", 2024-09-21T21:40:58.606 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:58.606 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.606 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.607 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:58.607 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.593713+0000", 2024-09-21T21:40:58.607 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:58.607 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.607 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.607 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:58.607 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.593714+0000", 2024-09-21T21:40:58.607 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3900000000000005e-07 2024-09-21T21:40:58.607 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.607 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.607 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:58.607 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.593716+0000", 2024-09-21T21:40:58.607 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.8059999999999999e-06 2024-09-21T21:40:58.608 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.608 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.608 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:58.608 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.593723+0000", 2024-09-21T21:40:58.608 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9040000000000003e-06 2024-09-21T21:40:58.609 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.609 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.609 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:58.610 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.593728+0000", 2024-09-21T21:40:58.610 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1780000000000002e-06 2024-09-21T21:40:58.610 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:58.610 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:58.610 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:58.610 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.610 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.610 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:6302 3.0 3.eaff32e0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:40:58.610 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.593735+0000", 2024-09-21T21:40:58.610 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.005738005, 2024-09-21T21:40:58.610 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0058233970000000001, 2024-09-21T21:40:58.610 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:58.610 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:58.611 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:40:58.611 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:58.611 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:40:58.611 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:40:58.611 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 6302 2024-09-21T21:40:58.611 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.611 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:58.611 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.611 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:58.612 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.593735+0000", 2024-09-21T21:40:58.612 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:58.612 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.612 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.612 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:58.612 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.593735+0000", 2024-09-21T21:40:58.612 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:58.612 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.612 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.612 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:58.612 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.593735+0000", 2024-09-21T21:40:58.612 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4200000000000002e-07 2024-09-21T21:40:58.612 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.612 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.613 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:58.613 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.593742+0000", 2024-09-21T21:40:58.613 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4330000000000001e-06 2024-09-21T21:40:58.613 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.613 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.613 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:58.613 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.593748+0000", 2024-09-21T21:40:58.613 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1199999999999999e-06 2024-09-21T21:40:58.613 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.613 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.613 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:58.613 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.593753+0000", 2024-09-21T21:40:58.613 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7910000000000004e-06 2024-09-21T21:40:58.613 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:58.613 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:58.614 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:58.614 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.614 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.614 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:6303 3.0 3.eaff32e0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:40:58.614 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.593759+0000", 2024-09-21T21:40:58.614 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.0057143000000000003, 2024-09-21T21:40:58.614 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0058533780000000002, 2024-09-21T21:40:58.614 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:58.614 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:58.614 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:40:58.614 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:58.614 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:40:58.614 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:40:58.614 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 6303 2024-09-21T21:40:58.614 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.615 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:58.615 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.615 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:58.615 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.593759+0000", 2024-09-21T21:40:58.615 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:58.615 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.615 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.615 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:58.615 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.593759+0000", 2024-09-21T21:40:58.615 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:58.615 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.615 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.615 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:58.616 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.593759+0000", 2024-09-21T21:40:58.616 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7e-07 2024-09-21T21:40:58.616 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.616 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.616 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:58.616 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.593761+0000", 2024-09-21T21:40:58.616 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0650000000000001e-06 2024-09-21T21:40:58.616 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.616 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.616 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:58.616 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.593767+0000", 2024-09-21T21:40:58.616 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7819999999999999e-06 2024-09-21T21:40:58.616 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.616 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.617 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:58.617 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.593771+0000", 2024-09-21T21:40:58.617 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4970000000000003e-06 2024-09-21T21:40:58.617 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:58.617 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:58.617 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:58.617 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.617 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.617 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:6304 3.0 3.eaff32e0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:40:58.617 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.593777+0000", 2024-09-21T21:40:58.617 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.0056960300000000004, 2024-09-21T21:40:58.617 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.005889554, 2024-09-21T21:40:58.617 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:58.617 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:58.618 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:40:58.618 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:58.618 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:40:58.618 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:40:58.618 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 6304 2024-09-21T21:40:58.618 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.618 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:58.618 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.618 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:58.618 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.593777+0000", 2024-09-21T21:40:58.618 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:58.618 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.618 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.618 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:58.619 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.593777+0000", 2024-09-21T21:40:58.619 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:58.619 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.619 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.619 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:58.619 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.593777+0000", 2024-09-21T21:40:58.619 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4700000000000002e-07 2024-09-21T21:40:58.619 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.619 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.619 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:58.619 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.593779+0000", 2024-09-21T21:40:58.619 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.978e-06 2024-09-21T21:40:58.619 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.619 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.619 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:58.620 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.593785+0000", 2024-09-21T21:40:58.620 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8649999999999998e-06 2024-09-21T21:40:58.620 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.620 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.620 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:58.620 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.593790+0000", 2024-09-21T21:40:58.620 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4479999999999996e-06 2024-09-21T21:40:58.620 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:58.620 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:58.620 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:58.620 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.620 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.620 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:6305 3.0 3.eaff32e0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:40:58.620 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.593795+0000", 2024-09-21T21:40:58.620 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.00567783, 2024-09-21T21:40:58.621 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0059256600000000001, 2024-09-21T21:40:58.621 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:58.621 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:58.621 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:40:58.621 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:58.621 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:40:58.621 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:40:58.621 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 6305 2024-09-21T21:40:58.621 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.621 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:58.621 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.621 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:58.621 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.593795+0000", 2024-09-21T21:40:58.621 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:58.621 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.621 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.622 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:58.622 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.593795+0000", 2024-09-21T21:40:58.622 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:58.622 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.622 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.622 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:58.622 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.593796+0000", 2024-09-21T21:40:58.622 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4999999999999998e-07 2024-09-21T21:40:58.622 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.622 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.622 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:58.622 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.593798+0000", 2024-09-21T21:40:58.622 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9539999999999998e-06 2024-09-21T21:40:58.622 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.622 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.623 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:58.623 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.593803+0000", 2024-09-21T21:40:58.623 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6509999999999998e-06 2024-09-21T21:40:58.623 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.623 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.623 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:58.623 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.593808+0000", 2024-09-21T21:40:58.623 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.532e-06 2024-09-21T21:40:58.623 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:58.623 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:58.623 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:58.623 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.623 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.623 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:6306 3.0 3.eaff32e0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:40:58.623 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.593886+0000", 2024-09-21T21:40:58.624 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.0055869429999999996, 2024-09-21T21:40:58.624 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0058904359999999998, 2024-09-21T21:40:58.624 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:58.624 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:58.624 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:40:58.624 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:58.624 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:40:58.624 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:40:58.624 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 6306 2024-09-21T21:40:58.624 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.624 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:58.624 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.624 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:58.624 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.593886+0000", 2024-09-21T21:40:58.625 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:58.625 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.625 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.625 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:58.625 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.593886+0000", 2024-09-21T21:40:58.625 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:58.625 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.625 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.625 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:58.625 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.593887+0000", 2024-09-21T21:40:58.625 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.8599999999999998e-07 2024-09-21T21:40:58.625 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.625 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.625 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:58.625 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.593891+0000", 2024-09-21T21:40:58.626 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.9330000000000003e-06 2024-09-21T21:40:58.626 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.626 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.626 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:58.626 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.593899+0000", 2024-09-21T21:40:58.626 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.6920000000000008e-06 2024-09-21T21:40:58.626 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.626 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.626 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:58.626 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.593906+0000", 2024-09-21T21:40:58.626 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6159999999999998e-06 2024-09-21T21:40:58.626 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:58.627 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:58.627 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:58.627 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.627 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.627 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:6307 3.0 3.eaff32e0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:40:58.627 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.593917+0000", 2024-09-21T21:40:58.627 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.0055562110000000001, 2024-09-21T21:40:58.627 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0059129209999999998, 2024-09-21T21:40:58.627 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:58.627 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:58.627 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:40:58.627 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:58.627 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:40:58.627 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:40:58.627 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 6307 2024-09-21T21:40:58.628 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.628 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:58.628 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.628 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:58.628 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.593917+0000", 2024-09-21T21:40:58.628 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:58.628 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.628 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.628 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:58.628 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.593917+0000", 2024-09-21T21:40:58.628 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:58.628 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.628 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.628 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:58.629 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.593917+0000", 2024-09-21T21:40:58.629 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.58e-07 2024-09-21T21:40:58.629 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.629 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.629 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:58.629 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.593920+0000", 2024-09-21T21:40:58.629 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3709999999999998e-06 2024-09-21T21:40:58.629 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.629 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.629 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:58.629 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.593926+0000", 2024-09-21T21:40:58.629 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1380000000000004e-06 2024-09-21T21:40:58.629 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.629 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.630 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:58.630 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.593931+0000", 2024-09-21T21:40:58.630 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9060000000000001e-06 2024-09-21T21:40:58.630 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:58.630 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:58.630 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:58.630 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.630 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.630 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:6308 3.0 3.eaff32e0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:40:58.630 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.593948+0000", 2024-09-21T21:40:58.630 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.0055255349999999998, 2024-09-21T21:40:58.630 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0059364229999999997, 2024-09-21T21:40:58.630 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:58.630 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:58.631 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:40:58.631 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:58.631 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:40:58.631 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:40:58.631 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 6308 2024-09-21T21:40:58.631 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.631 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:58.631 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.631 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:58.631 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.593948+0000", 2024-09-21T21:40:58.631 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:58.631 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.631 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.632 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:58.632 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.593948+0000", 2024-09-21T21:40:58.632 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:58.632 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.632 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.632 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:58.632 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.593948+0000", 2024-09-21T21:40:58.632 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.39e-07 2024-09-21T21:40:58.632 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.632 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.632 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:58.632 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.593950+0000", 2024-09-21T21:40:58.632 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.4329999999999998e-06 2024-09-21T21:40:58.632 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.633 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.633 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:58.633 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.593957+0000", 2024-09-21T21:40:58.633 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3770000000000001e-06 2024-09-21T21:40:58.633 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.633 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.633 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:58.633 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.593962+0000", 2024-09-21T21:40:58.633 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0000000000000004e-06 2024-09-21T21:40:58.633 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:58.633 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:58.633 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:58.633 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.633 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.634 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:6431 3.0 3.eaff32e0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:40:58.634 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.595759+0000", 2024-09-21T21:40:58.634 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.0037138800000000001, 2024-09-21T21:40:58.634 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0041834200000000002, 2024-09-21T21:40:58.634 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:58.634 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:58.634 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:40:58.634 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:58.634 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:40:58.634 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:40:58.634 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 6431 2024-09-21T21:40:58.634 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.634 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:58.635 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.635 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:58.635 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.595759+0000", 2024-09-21T21:40:58.635 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:58.635 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.635 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.635 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:58.635 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.595759+0000", 2024-09-21T21:40:58.635 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:58.635 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.635 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.635 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:58.635 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.595760+0000", 2024-09-21T21:40:58.635 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.6499999999999998e-07 2024-09-21T21:40:58.635 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.636 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.636 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:58.636 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.595764+0000", 2024-09-21T21:40:58.636 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3760000000000001e-06 2024-09-21T21:40:58.636 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.636 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.636 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:58.636 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.595776+0000", 2024-09-21T21:40:58.636 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1433e-05 2024-09-21T21:40:58.636 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.636 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.636 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:58.636 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.595786+0000", 2024-09-21T21:40:58.637 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0018e-05 2024-09-21T21:40:58.637 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:58.637 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:58.637 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:58.637 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.637 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.637 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:6432 3.0 3.eaff32e0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:40:58.637 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.595798+0000", 2024-09-21T21:40:58.637 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.0036746650000000001, 2024-09-21T21:40:58.637 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0042179549999999998, 2024-09-21T21:40:58.637 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:58.637 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:58.637 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:40:58.637 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:58.638 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:40:58.638 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:40:58.638 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 6432 2024-09-21T21:40:58.638 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.638 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:58.638 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.638 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:58.638 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.595798+0000", 2024-09-21T21:40:58.638 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:58.638 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.638 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.638 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:58.638 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.595798+0000", 2024-09-21T21:40:58.639 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:58.639 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.639 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.639 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:58.639 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.595799+0000", 2024-09-21T21:40:58.639 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.0400000000000002e-07 2024-09-21T21:40:58.639 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.639 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.639 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:58.639 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.595801+0000", 2024-09-21T21:40:58.639 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.554e-06 2024-09-21T21:40:58.639 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.639 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.639 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:58.640 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.595810+0000", 2024-09-21T21:40:58.640 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.2199999999999992e-06 2024-09-21T21:40:58.640 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.640 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.640 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:58.640 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.595815+0000", 2024-09-21T21:40:58.640 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6099999999999997e-06 2024-09-21T21:40:58.640 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:58.640 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:58.640 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:58.640 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.640 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.640 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:6433 3.0 3.eaff32e0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:40:58.641 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.595822+0000", 2024-09-21T21:40:58.641 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.0036510370000000002, 2024-09-21T21:40:58.641 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.004250541, 2024-09-21T21:40:58.641 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:58.641 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:58.641 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:40:58.641 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:58.641 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:40:58.641 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:40:58.641 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 6433 2024-09-21T21:40:58.641 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.641 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:58.641 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.641 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:58.642 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.595822+0000", 2024-09-21T21:40:58.642 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:58.642 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.642 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.642 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:58.642 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.595822+0000", 2024-09-21T21:40:58.642 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:58.642 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.642 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.642 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:58.642 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.595822+0000", 2024-09-21T21:40:58.642 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6699999999999999e-07 2024-09-21T21:40:58.642 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.642 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.643 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:58.643 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.595824+0000", 2024-09-21T21:40:58.643 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0849999999999999e-06 2024-09-21T21:40:58.643 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.643 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.643 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:58.643 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.595831+0000", 2024-09-21T21:40:58.643 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9070000000000001e-06 2024-09-21T21:40:58.643 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.643 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.643 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:58.643 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.595835+0000", 2024-09-21T21:40:58.643 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.121e-06 2024-09-21T21:40:58.644 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:58.644 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:58.644 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:58.644 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.644 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.644 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:6434 3.0 3.eaff32e0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:40:58.644 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.595841+0000", 2024-09-21T21:40:58.644 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.0036324650000000001, 2024-09-21T21:40:58.644 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0042861490000000004, 2024-09-21T21:40:58.644 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:58.644 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:58.644 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:40:58.644 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:58.644 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:40:58.645 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:40:58.645 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 6434 2024-09-21T21:40:58.645 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.645 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:58.645 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.645 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:58.645 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.595841+0000", 2024-09-21T21:40:58.645 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:58.645 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.645 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.645 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:58.646 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.595841+0000", 2024-09-21T21:40:58.646 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:58.646 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.646 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.646 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:58.646 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.595841+0000", 2024-09-21T21:40:58.646 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.2899999999999999e-07 2024-09-21T21:40:58.646 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.646 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.646 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:58.646 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.595843+0000", 2024-09-21T21:40:58.646 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.7549999999999999e-06 2024-09-21T21:40:58.646 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.646 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.647 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:58.647 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.595848+0000", 2024-09-21T21:40:58.647 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.096e-06 2024-09-21T21:40:58.647 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.647 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.647 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:58.647 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.595852+0000", 2024-09-21T21:40:58.647 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.9040000000000002e-06 2024-09-21T21:40:58.647 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:58.647 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:58.647 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:58.647 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.647 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.647 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:6435 3.0 3.eaff32e0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:40:58.648 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.595864+0000", 2024-09-21T21:40:58.648 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.0036091819999999998, 2024-09-21T21:40:58.648 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0043161359999999999, 2024-09-21T21:40:58.648 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:58.648 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:58.648 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:40:58.648 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:58.648 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:40:58.648 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:40:58.648 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 6435 2024-09-21T21:40:58.648 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.648 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:58.648 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.648 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:58.649 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.595864+0000", 2024-09-21T21:40:58.649 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:58.649 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.649 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.649 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:58.649 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.595864+0000", 2024-09-21T21:40:58.649 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:58.649 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.649 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.649 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:58.649 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.595864+0000", 2024-09-21T21:40:58.649 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6400000000000003e-07 2024-09-21T21:40:58.649 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.649 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.650 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:58.650 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.595867+0000", 2024-09-21T21:40:58.650 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.5289999999999998e-06 2024-09-21T21:40:58.650 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.650 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.650 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:58.650 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.595875+0000", 2024-09-21T21:40:58.650 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.8790000000000003e-06 2024-09-21T21:40:58.650 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.650 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.650 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:58.650 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.595880+0000", 2024-09-21T21:40:58.650 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7720000000000002e-06 2024-09-21T21:40:58.650 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:58.651 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:58.651 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:58.651 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.651 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.651 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:6436 3.0 3.eaff32e0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:40:58.651 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.595887+0000", 2024-09-21T21:40:58.651 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.0035862289999999998, 2024-09-21T21:40:58.651 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0043460929999999997, 2024-09-21T21:40:58.651 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:58.651 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:58.651 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:40:58.651 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:58.651 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:40:58.651 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:40:58.651 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 6436 2024-09-21T21:40:58.652 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.652 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:58.652 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.652 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:58.652 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.595887+0000", 2024-09-21T21:40:58.652 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:58.652 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.652 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.652 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:58.652 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.595887+0000", 2024-09-21T21:40:58.652 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:58.652 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.652 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.652 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:58.652 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.595887+0000", 2024-09-21T21:40:58.653 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1199999999999998e-07 2024-09-21T21:40:58.653 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.653 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.653 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:58.653 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.595889+0000", 2024-09-21T21:40:58.653 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.159e-06 2024-09-21T21:40:58.653 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.653 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.653 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:58.653 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.595898+0000", 2024-09-21T21:40:58.653 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.5990000000000002e-06 2024-09-21T21:40:58.653 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.653 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.654 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:58.654 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.595903+0000", 2024-09-21T21:40:58.654 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7330000000000003e-06 2024-09-21T21:40:58.654 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:58.654 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:58.654 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:58.654 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.654 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.654 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:6437 3.0 3.eaff32e0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:40:58.654 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.595909+0000", 2024-09-21T21:40:58.654 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.003563798, 2024-09-21T21:40:58.654 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0043776079999999998, 2024-09-21T21:40:58.654 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:58.654 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:58.655 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:40:58.655 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:58.655 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:40:58.655 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:40:58.655 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 6437 2024-09-21T21:40:58.655 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.655 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:58.655 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.655 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:58.655 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.595909+0000", 2024-09-21T21:40:58.655 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:58.655 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.655 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.655 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:58.655 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.595909+0000", 2024-09-21T21:40:58.656 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:58.656 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.656 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.656 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:58.656 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.595910+0000", 2024-09-21T21:40:58.656 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5900000000000003e-07 2024-09-21T21:40:58.656 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.656 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.656 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:58.656 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.595912+0000", 2024-09-21T21:40:58.656 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1919999999999999e-06 2024-09-21T21:40:58.656 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.656 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.656 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:58.656 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.595918+0000", 2024-09-21T21:40:58.657 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7459999999999997e-06 2024-09-21T21:40:58.657 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.657 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.657 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:58.657 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.595925+0000", 2024-09-21T21:40:58.657 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.2479999999999999e-06 2024-09-21T21:40:58.657 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:58.657 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:58.657 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:58.657 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.657 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.657 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:6438 3.0 3.eaff32e0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:40:58.657 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.595931+0000", 2024-09-21T21:40:58.657 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.0035423999999999998, 2024-09-21T21:40:58.658 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0044102519999999999, 2024-09-21T21:40:58.658 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:58.658 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:58.658 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:40:58.658 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:58.658 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:40:58.658 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:40:58.658 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 6438 2024-09-21T21:40:58.658 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.658 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:58.658 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.658 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:58.658 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.595931+0000", 2024-09-21T21:40:58.659 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:58.659 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.659 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.659 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:58.659 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.595931+0000", 2024-09-21T21:40:58.659 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:58.659 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.659 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.659 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:58.659 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.595931+0000", 2024-09-21T21:40:58.659 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7599999999999998e-07 2024-09-21T21:40:58.659 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.659 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.659 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:58.660 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.595933+0000", 2024-09-21T21:40:58.660 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2019999999999998e-06 2024-09-21T21:40:58.660 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.660 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.660 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:58.660 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.595949+0000", 2024-09-21T21:40:58.660 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.6222000000000002e-05 2024-09-21T21:40:58.660 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.660 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.660 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:58.660 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.595954+0000", 2024-09-21T21:40:58.660 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9790000000000001e-06 2024-09-21T21:40:58.660 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:58.661 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:58.661 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:58.661 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.661 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.661 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:6439 3.0 3.eaff32e0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:40:58.661 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.595961+0000", 2024-09-21T21:40:58.661 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.0035122489999999998, 2024-09-21T21:40:58.661 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0044371009999999997, 2024-09-21T21:40:58.661 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:58.661 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:58.661 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:40:58.661 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:58.661 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:40:58.661 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:40:58.661 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 6439 2024-09-21T21:40:58.661 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.662 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:58.662 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.662 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:58.662 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.595961+0000", 2024-09-21T21:40:58.662 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:58.662 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.662 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.662 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:58.662 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.595961+0000", 2024-09-21T21:40:58.662 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:58.662 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.662 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.662 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:58.663 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.595961+0000", 2024-09-21T21:40:58.663 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7100000000000002e-07 2024-09-21T21:40:58.663 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.663 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.663 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:58.663 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.595972+0000", 2024-09-21T21:40:58.663 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0514e-05 2024-09-21T21:40:58.663 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.663 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.663 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:58.663 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.595982+0000", 2024-09-21T21:40:58.663 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.7829999999999993e-06 2024-09-21T21:40:58.663 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.663 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.663 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:58.664 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.595987+0000", 2024-09-21T21:40:58.664 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0780000000000003e-06 2024-09-21T21:40:58.664 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:58.664 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:58.664 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:58.664 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.664 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.664 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:6179 3.0 3:074cff57:::rbd_data.112145fae779.000000000000000c:a [read 12222464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:40:58.664 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.593055+0000", 2024-09-21T21:40:58.664 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.0064175710000000004, 2024-09-21T21:40:58.664 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0074185020000000004, 2024-09-21T21:40:58.664 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:58.665 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:58.665 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:40:58.665 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:58.665 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:40:58.665 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:40:58.665 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 6179 2024-09-21T21:40:58.665 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.665 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:58.665 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.665 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:58.665 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.593055+0000", 2024-09-21T21:40:58.665 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:58.665 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.665 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.666 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:58.666 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.593055+0000", 2024-09-21T21:40:58.666 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:58.666 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.666 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.666 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:58.666 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.593056+0000", 2024-09-21T21:40:58.666 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6400000000000002e-07 2024-09-21T21:40:58.666 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.666 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.666 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:58.666 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.593058+0000", 2024-09-21T21:40:58.666 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2060000000000001e-06 2024-09-21T21:40:58.666 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.667 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.667 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:58.667 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.593065+0000", 2024-09-21T21:40:58.667 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.1919999999999999e-06 2024-09-21T21:40:58.667 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.667 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.667 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:58.667 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.593071+0000", 2024-09-21T21:40:58.667 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2599999999999996e-06 2024-09-21T21:40:58.667 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.667 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.667 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:58.667 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.600339+0000", 2024-09-21T21:40:58.667 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.007268081 2024-09-21T21:40:58.667 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.668 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.668 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:40:58.668 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.600415+0000", 2024-09-21T21:40:58.668 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.6489999999999994e-05 2024-09-21T21:40:58.668 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:58.668 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:58.668 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:58.668 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.668 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.668 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:6180 3.0 3:074cff57:::rbd_data.112145fae779.000000000000000c:a [read 12279808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:40:58.668 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.593078+0000", 2024-09-21T21:40:58.668 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.0063948829999999996, 2024-09-21T21:40:58.668 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0074635179999999997, 2024-09-21T21:40:58.668 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:58.668 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:58.669 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:40:58.669 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:58.669 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:40:58.669 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:40:58.669 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 6180 2024-09-21T21:40:58.669 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.669 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:58.669 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.669 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:58.669 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.593078+0000", 2024-09-21T21:40:58.669 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:58.669 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.669 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.669 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:58.670 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.593078+0000", 2024-09-21T21:40:58.670 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:58.670 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.670 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.670 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:58.670 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.593079+0000", 2024-09-21T21:40:58.670 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5999999999999999e-07 2024-09-21T21:40:58.670 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.670 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.670 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:58.670 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.593081+0000", 2024-09-21T21:40:58.670 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.441e-06 2024-09-21T21:40:58.670 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.670 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.671 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:58.671 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.593088+0000", 2024-09-21T21:40:58.671 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.883e-06 2024-09-21T21:40:58.671 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.671 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.671 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:58.671 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.593093+0000", 2024-09-21T21:40:58.671 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4820000000000002e-06 2024-09-21T21:40:58.671 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.671 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.671 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:40:58.671 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.600510+0000", 2024-09-21T21:40:58.671 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0074165380000000003 2024-09-21T21:40:58.672 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:58.672 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:58.672 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:58.672 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.672 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.672 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:6300 3.0 3.eaff32e0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:40:58.672 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.593677+0000", 2024-09-21T21:40:58.672 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.0057964840000000002, 2024-09-21T21:40:58.672 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0069260520000000002, 2024-09-21T21:40:58.672 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:40:58.672 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:40:58.672 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:40:58.672 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:40:58.672 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:40:58.672 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:40:58.673 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 6300 2024-09-21T21:40:58.673 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.673 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:40:58.673 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.673 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:40:58.673 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.593677+0000", 2024-09-21T21:40:58.673 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:58.673 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.673 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.673 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:40:58.673 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.593677+0000", 2024-09-21T21:40:58.673 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:40:58.673 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.673 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.673 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:40:58.674 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.593677+0000", 2024-09-21T21:40:58.674 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2399999999999996e-07 2024-09-21T21:40:58.674 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.674 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.674 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:40:58.674 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.593682+0000", 2024-09-21T21:40:58.674 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3460000000000004e-06 2024-09-21T21:40:58.674 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.674 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.674 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:40:58.674 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.593694+0000", 2024-09-21T21:40:58.674 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.167e-05 2024-09-21T21:40:58.674 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:40:58.674 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:40:58.675 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:40:58.675 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.593701+0000", 2024-09-21T21:40:58.675 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.2910000000000002e-06 2024-09-21T21:40:58.675 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:58.675 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:40:58.675 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:58.675 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:40:58.675 INFO:teuthology.orchestra.run.smithi081.stdout: ], 2024-09-21T21:40:58.675 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 20 2024-09-21T21:40:58.675 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:40:58.676 INFO:teuthology.orchestra.run.smithi164.stdout:1078 read 0x604d200 thru 0x60599ff (0xc800 bytes) 2024-09-21T21:40:58.676 INFO:teuthology.orchestra.run.smithi164.stdout:1079 trunc from 0xe06a400 to 0x7225000 2024-09-21T21:40:58.677 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T21:40:58.677 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0: {} 2024-09-21T21:40:58.678 INFO:teuthology.orchestra.run.smithi081.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T21:40:58.678 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1: {} 2024-09-21T21:40:58.678 INFO:teuthology.orchestra.run.smithi081.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T21:40:58.679 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2: {} 2024-09-21T21:40:58.679 INFO:teuthology.orchestra.run.smithi081.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T21:40:58.679 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:40:58.675+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:40:58.723 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3: {} 2024-09-21T21:40:58.724 INFO:teuthology.orchestra.run.smithi081.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T21:40:58.745 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4: {} 2024-09-21T21:40:58.746 INFO:teuthology.orchestra.run.smithi081.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T21:40:58.765 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5: {} 2024-09-21T21:40:58.765 INFO:teuthology.orchestra.run.smithi081.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T21:40:58.772 INFO:teuthology.orchestra.run.smithi164.stdout:1080 read 0x650600 thru 0x653fb0 (0x39b1 bytes) 2024-09-21T21:40:58.773 INFO:teuthology.orchestra.run.smithi164.stdout:1081 read 0x47ad000 thru 0x47b80c1 (0xb0c2 bytes) 2024-09-21T21:40:58.775 INFO:teuthology.orchestra.run.smithi164.stdout:1082 read 0x5a62400 thru 0x5a6b3fd (0x8ffe bytes) 2024-09-21T21:40:58.777 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:40:58.775+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:40:58.777 INFO:teuthology.orchestra.run.smithi164.stdout:1085 read 0x879400 thru 0x8836f0 (0xa2f1 bytes) 2024-09-21T21:40:58.778 INFO:teuthology.orchestra.run.smithi164.stdout:1086 read 0x271fe00 thru 0x2723048 (0x3249 bytes) 2024-09-21T21:40:58.779 INFO:teuthology.orchestra.run.smithi164.stdout:1088 read 0x48d1a00 thru 0x48d43f3 (0x29f4 bytes) 2024-09-21T21:40:58.780 INFO:teuthology.orchestra.run.smithi164.stdout:1090 write 0x3707a00 thru 0x370eeea (0x74eb bytes) 2024-09-21T21:40:58.780 INFO:teuthology.orchestra.run.smithi164.stdout:1091 write 0x515b800 thru 0x5164907 (0x9108 bytes) 2024-09-21T21:40:58.780 INFO:teuthology.orchestra.run.smithi164.stdout:1092 write 0x5709e00 thru 0x570bfa0 (0x21a1 bytes) 2024-09-21T21:40:58.781 INFO:teuthology.orchestra.run.smithi164.stdout:1094 read 0x6e600 thru 0x7cb28 (0xe529 bytes) 2024-09-21T21:40:58.784 INFO:teuthology.orchestra.run.smithi164.stdout:1095 read 0x36e6400 thru 0x36eae4e (0x4a4f bytes) 2024-09-21T21:40:58.785 INFO:teuthology.orchestra.run.smithi164.stdout:1096 punch from 0x3b2dc00 to 0x3b34a00, (0x6e00 bytes) 2024-09-21T21:40:58.852 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:58.852 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:58.862 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T21:40:58.863 INFO:teuthology.orchestra.run.smithi164.stdout:1097 trunc from 0x7225000 to 0xc4a9800 2024-09-21T21:40:58.906 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:40:58.905+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:40:58.941 INFO:teuthology.orchestra.run.smithi164.stdout:1100 read 0xc08c00 thru 0xc131f0 (0xa5f1 bytes) 2024-09-21T21:40:58.943 INFO:teuthology.orchestra.run.smithi164.stdout:1101 punch from 0x1474e00 to 0x1476800, (0x1a00 bytes) 2024-09-21T21:40:58.950 INFO:teuthology.orchestra.run.smithi164.stdout:1102 trunc from 0xc4a9800 to 0xe563800 2024-09-21T21:40:58.976 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:58.976 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:58.976 INFO:teuthology.orchestra.run.smithi164.stdout:1103 read 0xcfca000 thru 0xcfce8d6 (0x48d7 bytes) 2024-09-21T21:40:58.977 INFO:teuthology.orchestra.run.smithi164.stdout:1105 punch from 0xb849000 to 0xb84ac00, (0x1c00 bytes) 2024-09-21T21:40:58.978 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:40:58.977+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:40:58.978 INFO:teuthology.orchestra.run.smithi164.stdout:1106 read 0xc504a00 thru 0xc50ddd1 (0x93d2 bytes) 2024-09-21T21:40:58.985 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T21:40:58.986 INFO:teuthology.orchestra.run.smithi164.stdout:1108 trunc from 0xe563800 to 0x81eda00 2024-09-21T21:40:59.019 INFO:teuthology.orchestra.run.smithi164.stdout:1109 write 0x9f3a200 thru 0x9f405d3 (0x63d4 bytes) 2024-09-21T21:40:59.019 INFO:teuthology.orchestra.run.smithi164.stdout:1111 read 0x437a400 thru 0x4384305 (0x9f06 bytes) 2024-09-21T21:40:59.031 INFO:teuthology.orchestra.run.smithi164.stdout:1113 trunc from 0x9f40600 to 0xdbfe00 2024-09-21T21:40:59.079 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:40:59.078+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:40:59.151 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:59.151 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:59.161 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T21:40:59.185 INFO:teuthology.orchestra.run.smithi164.stdout:1114 write 0x6119e00 thru 0x611d014 (0x3215 bytes) 2024-09-21T21:40:59.186 INFO:teuthology.orchestra.run.smithi164.stdout:1115 read 0x5a11e00 thru 0x5a20b02 (0xed03 bytes) 2024-09-21T21:40:59.186 INFO:teuthology.orchestra.run.smithi164.stdout:1117 write 0x13fec00 thru 0x14061f4 (0x75f5 bytes) 2024-09-21T21:40:59.186 INFO:teuthology.orchestra.run.smithi164.stdout:1118 punch from 0x2e45600 to 0x2e52e00, (0xd800 bytes) 2024-09-21T21:40:59.189 INFO:teuthology.orchestra.run.smithi164.stdout:1120 trunc from 0x611d200 to 0x4bbfe00 2024-09-21T21:40:59.201 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:40:59.200+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:40:59.230 INFO:teuthology.orchestra.run.smithi164.stdout:1122 punch from 0x3494e00 to 0x349a000, (0x5200 bytes) 2024-09-21T21:40:59.233 INFO:teuthology.orchestra.run.smithi164.stdout:1124 read 0x48e6600 thru 0x48ec4d4 (0x5ed5 bytes) 2024-09-21T21:40:59.233 INFO:teuthology.orchestra.run.smithi164.stdout:1126 write 0x3f4ca00 thru 0x3f52814 (0x5e15 bytes) 2024-09-21T21:40:59.243 INFO:teuthology.orchestra.run.smithi164.stdout:1128 trunc from 0x4bbfe00 to 0xd9ff800 2024-09-21T21:40:59.278 INFO:teuthology.orchestra.run.smithi164.stdout:1129 trunc from 0xd9ff800 to 0x2e9ca00 2024-09-21T21:40:59.281 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:40:59.280+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:40:59.318 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:59.319 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:59.322 INFO:teuthology.orchestra.run.smithi164.stdout:1130 write 0xa72e200 thru 0xa7383b4 (0xa1b5 bytes) 2024-09-21T21:40:59.323 INFO:teuthology.orchestra.run.smithi164.stdout:1131 write 0x4fde800 thru 0x4fe89e2 (0xa1e3 bytes) 2024-09-21T21:40:59.323 INFO:teuthology.orchestra.run.smithi164.stdout:1132 read 0x1ac1e00 thru 0x1aca3ae (0x85af bytes) 2024-09-21T21:40:59.324 INFO:teuthology.orchestra.run.smithi164.stdout:1134 trunc from 0xa738400 to 0xbaa7000 2024-09-21T21:40:59.328 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T21:40:59.355 INFO:teuthology.orchestra.run.smithi164.stdout:1140 read 0x1930a00 thru 0x19398f9 (0x8efa bytes) 2024-09-21T21:40:59.357 INFO:teuthology.orchestra.run.smithi164.stdout:1142 write 0xa6f5e00 thru 0xa703492 (0xd693 bytes) 2024-09-21T21:40:59.358 INFO:teuthology.orchestra.run.smithi164.stdout:1143 read 0x146800 thru 0x14ea6f (0x8270 bytes) 2024-09-21T21:40:59.360 INFO:teuthology.orchestra.run.smithi164.stdout:1145 write 0x3a97000 thru 0x3a9821d (0x121e bytes) 2024-09-21T21:40:59.360 INFO:teuthology.orchestra.run.smithi164.stdout:1146 read 0x321800 thru 0x324123 (0x2924 bytes) 2024-09-21T21:40:59.365 INFO:teuthology.orchestra.run.smithi164.stdout:1147 trunc from 0xbaa7000 to 0x86f3600 2024-09-21T21:40:59.382 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:40:59.380+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:40:59.475 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:59.475 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:59.484 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T21:40:59.484 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:40:59.482+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:40:59.533 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T21:40:59.584 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:40:59.582+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:40:59.588 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:59.588 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:59.597 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T21:40:59.646 INFO:teuthology.orchestra.run.smithi164.stdout:1150 read 0x2bd1600 thru 0x2bd38b8 (0x22b9 bytes) 2024-09-21T21:40:59.661 INFO:teuthology.orchestra.run.smithi164.stdout:1151 write 0xcfa2a00 thru 0xcfa7952 (0x4f53 bytes) 2024-09-21T21:40:59.662 INFO:teuthology.orchestra.run.smithi164.stdout:1152 write 0x535000 thru 0x53d1be (0x81bf bytes) 2024-09-21T21:40:59.662 INFO:teuthology.orchestra.run.smithi164.stdout:1153 read 0x989e200 thru 0x98a7bf7 (0x99f8 bytes) 2024-09-21T21:40:59.686 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:40:59.683+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:40:59.724 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:59.724 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:59.735 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T21:40:59.785 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:40:59.784+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:40:59.877 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:40:59.877 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:40:59.886 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T21:40:59.887 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:40:59.884+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:40:59.927 INFO:teuthology.orchestra.run.smithi164.stdout:1154 write 0xee17000 thru 0xee2583d (0xe83e bytes) 2024-09-21T21:40:59.944 INFO:teuthology.orchestra.run.smithi164.stdout:1155 trunc from 0xee25a00 to 0x1289000 2024-09-21T21:40:59.987 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:40:59.986+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:00.007 INFO:teuthology.orchestra.run.smithi164.stdout:1159 write 0x4df800 thru 0x4e742d (0x7c2e bytes) 2024-09-21T21:41:00.038 INFO:teuthology.orchestra.run.smithi164.stdout:1160 write 0x4fa5e00 thru 0x4fb2447 (0xc648 bytes) 2024-09-21T21:41:00.039 INFO:teuthology.orchestra.run.smithi164.stdout:1161 read 0x2160200 thru 0x216d823 (0xd624 bytes) 2024-09-21T21:41:00.039 INFO:teuthology.orchestra.run.smithi164.stdout:1162 punch from 0xa5c800 to 0xa5d000, (0x800 bytes) 2024-09-21T21:41:00.049 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:00.049 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:00.059 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T21:41:00.065 INFO:teuthology.orchestra.run.smithi164.stdout:1164 write 0xb84b800 thru 0xb851d79 (0x657a bytes) 2024-09-21T21:41:00.066 INFO:teuthology.orchestra.run.smithi164.stdout:1165 read 0x155e000 thru 0x1562c77 (0x4c78 bytes) 2024-09-21T21:41:00.067 INFO:teuthology.orchestra.run.smithi164.stdout:1167 punch from 0x340fa00 to 0x341a400, (0xaa00 bytes) 2024-09-21T21:41:00.068 INFO:teuthology.orchestra.run.smithi164.stdout:1168 read 0x7ff5600 thru 0x8002865 (0xd266 bytes) 2024-09-21T21:41:00.068 INFO:teuthology.orchestra.run.smithi164.stdout:1170 read 0x9c41e00 thru 0x9c4bf2a (0xa12b bytes) 2024-09-21T21:41:00.068 INFO:teuthology.orchestra.run.smithi164.stdout:1175 write 0x9c74e00 thru 0x9c84600 (0xf801 bytes) 2024-09-21T21:41:00.068 INFO:teuthology.orchestra.run.smithi164.stdout:1176 read 0x9990e00 thru 0x999c1f6 (0xb3f7 bytes) 2024-09-21T21:41:00.069 INFO:teuthology.orchestra.run.smithi164.stdout:1178 punch from 0x8d64000 to 0x8d64800, (0x800 bytes) 2024-09-21T21:41:00.073 INFO:teuthology.orchestra.run.smithi164.stdout:1179 trunc from 0xb851e00 to 0x809b200 2024-09-21T21:41:00.102 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:00.101+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:00.123 INFO:teuthology.orchestra.run.smithi164.stdout:1180 read 0x4bca400 thru 0x4bcb006 (0xc07 bytes) 2024-09-21T21:41:00.129 INFO:teuthology.orchestra.run.smithi164.stdout:1181 clone 9 order 22 su 524288 sc 9 2024-09-21T21:41:00.176 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:00.176 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:00.185 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T21:41:00.225 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:00.224+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:00.290 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:00.289+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:00.328 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:00.328 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:00.336 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T21:41:00.421 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:00.420+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:00.480 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:00.480 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:00.488 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T21:41:00.529 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:00.528+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:00.532 INFO:teuthology.orchestra.run.smithi081.stderr:noscrub is set 2024-09-21T21:41:00.592 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:00.590+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:00.607 INFO:teuthology.orchestra.run.smithi164.stdout:leaving image image_client.0-clone8 intact 2024-09-21T21:41:00.636 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:00.636 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:00.645 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T21:41:00.692 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:00.691+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:00.751 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:00.751 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:00.760 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T21:41:00.777 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T21:41:00.804 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:00.803+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:00.867 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:00.867 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:00.877 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T21:41:00.895 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:00.893+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:00.928 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0: {} 2024-09-21T21:41:00.929 INFO:teuthology.orchestra.run.smithi081.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T21:41:00.946 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1: {} 2024-09-21T21:41:00.946 INFO:teuthology.orchestra.run.smithi081.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T21:41:00.962 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2: {} 2024-09-21T21:41:00.963 INFO:teuthology.orchestra.run.smithi081.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T21:41:00.965 INFO:teuthology.orchestra.run.smithi164.stdout:checking clone #7, image image_client.0-clone7 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent8 2024-09-21T21:41:00.979 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3: {} 2024-09-21T21:41:00.980 INFO:teuthology.orchestra.run.smithi081.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T21:41:00.985 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:00.985 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:41:00.985 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:41:00.985 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:41:00.985 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:00.994 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T21:41:00.995 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:00.994+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:00.997 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4: {} 2024-09-21T21:41:00.997 INFO:teuthology.orchestra.run.smithi081.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T21:41:01.014 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5: {} 2024-09-21T21:41:01.014 INFO:teuthology.orchestra.run.smithi081.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T21:41:01.097 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:01.095+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:01.100 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:01.100 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 20, 2024-09-21T21:41:01.100 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 600, 2024-09-21T21:41:01.100 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:41:01.100 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.100 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:228 3.6 3:678f83b3:::rbd_data.11214be7c411.000000000000002a:head [delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:41:01.100 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.553796+0000", 2024-09-21T21:41:01.100 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 17.543928841, 2024-09-21T21:41:01.100 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0098649099999999993, 2024-09-21T21:41:01.100 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:01.100 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:01.100 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:41:01.101 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:01.101 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:41:01.101 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:41:01.101 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 228 2024-09-21T21:41:01.101 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.101 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:01.101 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.101 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:01.101 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.553796+0000", 2024-09-21T21:41:01.101 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:01.101 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.101 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.101 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:01.101 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.553796+0000", 2024-09-21T21:41:01.101 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:01.102 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.102 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.102 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:01.102 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.553797+0000", 2024-09-21T21:41:01.102 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6899999999999997e-07 2024-09-21T21:41:01.102 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.102 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.102 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:01.102 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.553799+0000", 2024-09-21T21:41:01.102 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3e-06 2024-09-21T21:41:01.102 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.102 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.102 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:01.103 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.553805+0000", 2024-09-21T21:41:01.103 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1310000000000003e-06 2024-09-21T21:41:01.103 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.103 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.103 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:01.103 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.553816+0000", 2024-09-21T21:41:01.103 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0506000000000001e-05 2024-09-21T21:41:01.103 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.103 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.103 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:01.103 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.555870+0000", 2024-09-21T21:41:01.103 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0020538510000000002 2024-09-21T21:41:01.103 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.103 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.104 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:01.104 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.555942+0000", 2024-09-21T21:41:01.104 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.2436999999999998e-05 2024-09-21T21:41:01.104 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.104 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.104 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 1", 2024-09-21T21:41:01.104 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556121+0000", 2024-09-21T21:41:01.104 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000178526 2024-09-21T21:41:01.104 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.104 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.104 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:41:01.104 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.559481+0000", 2024-09-21T21:41:01.104 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0033603019999999999 2024-09-21T21:41:01.104 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.104 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.105 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:41:01.105 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.563144+0000", 2024-09-21T21:41:01.105 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0036625799999999999 2024-09-21T21:41:01.105 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.105 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.105 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:41:01.105 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.563185+0000", 2024-09-21T21:41:01.105 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1084000000000002e-05 2024-09-21T21:41:01.105 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.105 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.105 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:01.105 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.563661+0000", 2024-09-21T21:41:01.105 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00047662399999999998 2024-09-21T21:41:01.106 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.106 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:01.106 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.106 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.106 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.106 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:1608 3.12 3:4bfa5158:::rbd_data.112145fae779.0000000000000002:a [read 10678272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:01.106 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.381876+0000", 2024-09-21T21:41:01.106 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 2.7158488890000001, 2024-09-21T21:41:01.106 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0096257990000000009, 2024-09-21T21:41:01.106 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:01.106 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:01.106 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:01.106 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:01.107 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:01.107 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:01.107 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1608 2024-09-21T21:41:01.107 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.107 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:01.107 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.107 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:01.107 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.381876+0000", 2024-09-21T21:41:01.107 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:01.107 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.107 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.107 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:01.107 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.381876+0000", 2024-09-21T21:41:01.107 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:01.107 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.108 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.108 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:01.108 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.381876+0000", 2024-09-21T21:41:01.108 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1e-07 2024-09-21T21:41:01.108 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.108 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.108 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:01.108 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.381878+0000", 2024-09-21T21:41:01.108 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.111e-06 2024-09-21T21:41:01.108 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.108 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.108 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:01.108 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.381881+0000", 2024-09-21T21:41:01.108 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0570000000000001e-06 2024-09-21T21:41:01.108 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.108 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.109 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:01.109 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.381883+0000", 2024-09-21T21:41:01.109 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3429999999999998e-06 2024-09-21T21:41:01.109 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.109 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.109 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:01.109 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.391392+0000", 2024-09-21T21:41:01.109 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.009509221 2024-09-21T21:41:01.109 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.109 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.109 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:01.109 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.391448+0000", 2024-09-21T21:41:01.109 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5961999999999997e-05 2024-09-21T21:41:01.109 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.110 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.110 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:01.110 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.391502+0000", 2024-09-21T21:41:01.110 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3894999999999998e-05 2024-09-21T21:41:01.110 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.110 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:01.110 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.110 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.110 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.110 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:1610 3.12 3:4bfa5158:::rbd_data.112145fae779.0000000000000002:a [read 10735616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:01.110 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.381886+0000", 2024-09-21T21:41:01.110 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 2.715839103, 2024-09-21T21:41:01.110 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0097775250000000005, 2024-09-21T21:41:01.110 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:01.111 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:01.111 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:01.111 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:01.111 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:01.111 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:01.111 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1610 2024-09-21T21:41:01.111 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.111 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:01.111 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.111 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:01.111 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.381886+0000", 2024-09-21T21:41:01.111 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:01.111 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.111 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.112 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:01.112 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.381886+0000", 2024-09-21T21:41:01.112 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:01.112 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.112 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.112 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:01.112 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.381886+0000", 2024-09-21T21:41:01.112 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.05e-07 2024-09-21T21:41:01.112 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.112 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.112 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:01.112 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.381887+0000", 2024-09-21T21:41:01.112 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1960000000000001e-06 2024-09-21T21:41:01.112 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.113 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.113 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:01.113 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.381891+0000", 2024-09-21T21:41:01.113 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.061e-06 2024-09-21T21:41:01.113 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.113 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.113 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:01.113 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.381893+0000", 2024-09-21T21:41:01.113 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.463e-06 2024-09-21T21:41:01.113 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.113 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.113 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:01.113 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.391533+0000", 2024-09-21T21:41:01.113 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0096403979999999997 2024-09-21T21:41:01.113 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.114 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.114 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:01.114 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.391593+0000", 2024-09-21T21:41:01.114 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9778999999999998e-05 2024-09-21T21:41:01.114 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.114 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.114 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:01.114 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.391664+0000", 2024-09-21T21:41:01.114 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0423e-05 2024-09-21T21:41:01.114 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.114 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:01.114 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.114 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.114 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.114 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:1800 3.12 3:4bfa5158:::rbd_data.112145fae779.0000000000000002:a [read 10391552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:01.114 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.382760+0000", 2024-09-21T21:41:01.115 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 2.7149649650000001, 2024-09-21T21:41:01.115 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0094873609999999997, 2024-09-21T21:41:01.115 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:01.115 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:01.115 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:01.115 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:01.115 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:01.115 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:01.115 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1800 2024-09-21T21:41:01.115 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.115 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:01.115 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.115 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:01.115 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382760+0000", 2024-09-21T21:41:01.115 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:01.115 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.116 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.116 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:01.116 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382760+0000", 2024-09-21T21:41:01.116 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:01.116 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.116 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.116 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:01.116 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382761+0000", 2024-09-21T21:41:01.116 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.63e-07 2024-09-21T21:41:01.116 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.116 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.116 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:01.116 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382763+0000", 2024-09-21T21:41:01.116 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.4080000000000001e-06 2024-09-21T21:41:01.116 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.116 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.117 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:01.117 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382770+0000", 2024-09-21T21:41:01.117 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.528e-06 2024-09-21T21:41:01.117 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.117 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.117 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:01.117 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382775+0000", 2024-09-21T21:41:01.117 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0540000000000002e-06 2024-09-21T21:41:01.117 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.117 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.117 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:01.117 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.392092+0000", 2024-09-21T21:41:01.117 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0093168999999999995 2024-09-21T21:41:01.117 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.118 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.118 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:01.118 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.392173+0000", 2024-09-21T21:41:01.118 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.1897999999999994e-05 2024-09-21T21:41:01.118 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.118 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.118 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:01.118 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.392248+0000", 2024-09-21T21:41:01.118 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.4109999999999993e-05 2024-09-21T21:41:01.118 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.118 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:01.118 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.118 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.118 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.118 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:1802 3.12 3:4bfa5158:::rbd_data.112145fae779.0000000000000002:a [read 10448896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:01.119 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.382781+0000", 2024-09-21T21:41:01.119 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 2.7149437179999998, 2024-09-21T21:41:01.119 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0096511759999999992, 2024-09-21T21:41:01.119 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:01.119 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:01.119 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:01.119 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:01.119 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:01.119 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:01.119 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1802 2024-09-21T21:41:01.119 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.119 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:01.119 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.119 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:01.119 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382781+0000", 2024-09-21T21:41:01.120 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:01.120 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.120 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.120 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:01.120 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382781+0000", 2024-09-21T21:41:01.120 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:01.120 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.120 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.120 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:01.120 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382782+0000", 2024-09-21T21:41:01.120 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.41e-07 2024-09-21T21:41:01.120 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.120 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.120 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:01.120 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382784+0000", 2024-09-21T21:41:01.120 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1509999999999998e-06 2024-09-21T21:41:01.120 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.121 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.121 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:01.121 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382799+0000", 2024-09-21T21:41:01.121 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.4711e-05 2024-09-21T21:41:01.121 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.121 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.121 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:01.121 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382803+0000", 2024-09-21T21:41:01.121 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8130000000000003e-06 2024-09-21T21:41:01.121 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.121 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.121 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:01.121 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.392304+0000", 2024-09-21T21:41:01.121 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0095000510000000007 2024-09-21T21:41:01.121 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.121 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.122 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:01.122 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.392383+0000", 2024-09-21T21:41:01.122 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.9666999999999995e-05 2024-09-21T21:41:01.122 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.122 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.122 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:01.122 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.392433+0000", 2024-09-21T21:41:01.122 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9441999999999999e-05 2024-09-21T21:41:01.122 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.122 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:01.122 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.122 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.122 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.122 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:1804 3.12 3:4bfa5158:::rbd_data.112145fae779.0000000000000002:a [read 10539008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:01.122 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.382815+0000", 2024-09-21T21:41:01.123 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 2.7149098619999998, 2024-09-21T21:41:01.123 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.009793263, 2024-09-21T21:41:01.123 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:01.123 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:01.123 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:01.123 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:01.123 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:01.123 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:01.123 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1804 2024-09-21T21:41:01.123 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.123 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:01.123 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.123 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:01.123 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382815+0000", 2024-09-21T21:41:01.123 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:01.123 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.124 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.124 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:01.124 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382815+0000", 2024-09-21T21:41:01.124 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:01.124 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.124 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.124 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:01.124 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382816+0000", 2024-09-21T21:41:01.124 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.06e-07 2024-09-21T21:41:01.124 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.124 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.124 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:01.124 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382818+0000", 2024-09-21T21:41:01.124 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.5069999999999999e-06 2024-09-21T21:41:01.124 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.124 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.125 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:01.125 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382825+0000", 2024-09-21T21:41:01.125 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0899999999999999e-06 2024-09-21T21:41:01.125 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.125 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.125 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:01.125 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382831+0000", 2024-09-21T21:41:01.125 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1830000000000002e-06 2024-09-21T21:41:01.125 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.125 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.125 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:01.125 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.392460+0000", 2024-09-21T21:41:01.125 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0096296730000000001 2024-09-21T21:41:01.125 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.125 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.126 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:01.126 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.392530+0000", 2024-09-21T21:41:01.126 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9590000000000003e-05 2024-09-21T21:41:01.126 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.126 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.126 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:01.126 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.392609+0000", 2024-09-21T21:41:01.126 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.8714000000000006e-05 2024-09-21T21:41:01.126 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.126 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:01.126 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.126 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.126 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.126 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:1806 3.12 3:4bfa5158:::rbd_data.112145fae779.0000000000000002:a [read 10596352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:01.126 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.382838+0000", 2024-09-21T21:41:01.126 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 2.714887059, 2024-09-21T21:41:01.127 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0099573270000000002, 2024-09-21T21:41:01.127 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:01.127 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:01.127 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:01.127 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:01.127 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:01.127 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:01.127 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1806 2024-09-21T21:41:01.127 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.127 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:01.127 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.127 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:01.127 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382838+0000", 2024-09-21T21:41:01.127 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:01.127 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.127 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.128 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:01.128 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382838+0000", 2024-09-21T21:41:01.128 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:01.128 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.128 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.128 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:01.128 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382839+0000", 2024-09-21T21:41:01.128 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7599999999999997e-07 2024-09-21T21:41:01.128 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.128 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.128 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:01.128 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382841+0000", 2024-09-21T21:41:01.128 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3070000000000001e-06 2024-09-21T21:41:01.128 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.128 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.128 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:01.129 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382847+0000", 2024-09-21T21:41:01.129 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2870000000000001e-06 2024-09-21T21:41:01.129 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.129 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.129 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:01.129 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382852+0000", 2024-09-21T21:41:01.129 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5950000000000001e-06 2024-09-21T21:41:01.129 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.129 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.129 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:01.129 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.392651+0000", 2024-09-21T21:41:01.129 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0097987419999999992 2024-09-21T21:41:01.129 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.129 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.129 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:01.129 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.392731+0000", 2024-09-21T21:41:01.130 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0154999999999994e-05 2024-09-21T21:41:01.130 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.130 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.130 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:01.130 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.392795+0000", 2024-09-21T21:41:01.130 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4665000000000005e-05 2024-09-21T21:41:01.130 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.130 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:01.130 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.130 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.130 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.130 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:1807 3.12 3:4bfa5158:::rbd_data.112145fae779.0000000000000002:a [read 10629120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:01.130 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.382858+0000", 2024-09-21T21:41:01.130 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 2.7148670400000001, 2024-09-21T21:41:01.130 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.010090127000000001, 2024-09-21T21:41:01.130 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:01.131 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:01.131 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:01.131 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:01.131 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:01.131 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:01.131 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1807 2024-09-21T21:41:01.131 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.131 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:01.131 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.131 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:01.131 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382858+0000", 2024-09-21T21:41:01.131 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:01.131 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.131 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.131 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:01.131 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382858+0000", 2024-09-21T21:41:01.131 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:01.132 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.132 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.132 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:01.132 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382859+0000", 2024-09-21T21:41:01.132 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7599999999999997e-07 2024-09-21T21:41:01.132 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.132 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.132 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:01.132 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382861+0000", 2024-09-21T21:41:01.132 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.176e-06 2024-09-21T21:41:01.132 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.132 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.132 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:01.132 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382867+0000", 2024-09-21T21:41:01.132 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0990000000000004e-06 2024-09-21T21:41:01.132 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.133 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.133 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:01.133 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382872+0000", 2024-09-21T21:41:01.133 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8230000000000002e-06 2024-09-21T21:41:01.133 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.133 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.133 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:01.133 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.392829+0000", 2024-09-21T21:41:01.133 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0099573869999999998 2024-09-21T21:41:01.133 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.133 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.133 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:01.133 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.392888+0000", 2024-09-21T21:41:01.133 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8585000000000002e-05 2024-09-21T21:41:01.133 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.133 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.134 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:01.134 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.392948+0000", 2024-09-21T21:41:01.134 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0581000000000002e-05 2024-09-21T21:41:01.134 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.134 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:01.134 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.134 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.134 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.134 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:1809 3.12 3:4bfa5158:::rbd_data.112145fae779.0000000000000002:a [read 10686464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:01.134 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.382878+0000", 2024-09-21T21:41:01.134 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 2.714847147, 2024-09-21T21:41:01.134 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.010264758000000001, 2024-09-21T21:41:01.134 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:01.135 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:01.135 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:01.135 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:01.135 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:01.135 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:01.135 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1809 2024-09-21T21:41:01.135 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.135 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:01.135 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.135 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:01.135 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382878+0000", 2024-09-21T21:41:01.135 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:01.135 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.135 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.135 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:01.135 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382878+0000", 2024-09-21T21:41:01.136 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:01.136 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.136 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.136 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:01.136 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382878+0000", 2024-09-21T21:41:01.136 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8000000000000001e-07 2024-09-21T21:41:01.136 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.136 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.136 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:01.136 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382880+0000", 2024-09-21T21:41:01.136 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0470000000000001e-06 2024-09-21T21:41:01.136 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.136 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.136 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:01.136 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382887+0000", 2024-09-21T21:41:01.136 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2899999999999999e-06 2024-09-21T21:41:01.137 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.137 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.137 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:01.137 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382891+0000", 2024-09-21T21:41:01.137 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.566e-06 2024-09-21T21:41:01.137 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.137 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.137 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:01.137 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.392978+0000", 2024-09-21T21:41:01.137 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.010086603 2024-09-21T21:41:01.137 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.137 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.137 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:01.137 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.393063+0000", 2024-09-21T21:41:01.137 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.5197000000000004e-05 2024-09-21T21:41:01.137 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.137 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.138 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:01.138 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.393143+0000", 2024-09-21T21:41:01.138 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.9674999999999999e-05 2024-09-21T21:41:01.138 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.138 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:01.138 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.138 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.138 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.138 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:1810 3.12 3:4bfa5158:::rbd_data.112145fae779.0000000000000002:a [read 10743808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:01.138 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.382898+0000", 2024-09-21T21:41:01.138 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 2.7148272589999998, 2024-09-21T21:41:01.138 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.010450245, 2024-09-21T21:41:01.138 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:01.138 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:01.138 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:01.139 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:01.139 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:01.139 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:01.139 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1810 2024-09-21T21:41:01.139 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.139 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:01.139 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.139 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:01.139 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382898+0000", 2024-09-21T21:41:01.139 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:01.139 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.139 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.139 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:01.139 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382898+0000", 2024-09-21T21:41:01.139 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:01.139 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.140 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.140 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:01.140 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382898+0000", 2024-09-21T21:41:01.140 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2e-07 2024-09-21T21:41:01.140 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.140 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.140 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:01.140 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382900+0000", 2024-09-21T21:41:01.140 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9439999999999999e-06 2024-09-21T21:41:01.140 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.140 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.140 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:01.140 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382906+0000", 2024-09-21T21:41:01.140 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.981e-06 2024-09-21T21:41:01.140 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.141 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.141 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:01.141 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382912+0000", 2024-09-21T21:41:01.141 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5389999999999998e-06 2024-09-21T21:41:01.141 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.141 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.141 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:01.141 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.393190+0000", 2024-09-21T21:41:01.141 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.010277846 2024-09-21T21:41:01.141 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.141 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.141 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:01.141 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.393271+0000", 2024-09-21T21:41:01.141 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.1666e-05 2024-09-21T21:41:01.141 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.142 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.142 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:01.142 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.393348+0000", 2024-09-21T21:41:01.142 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.6848999999999996e-05 2024-09-21T21:41:01.142 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.142 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:01.142 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.142 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.142 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.142 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:1856 3.12 3:4bfa5158:::rbd_data.112145fae779.0000000000000002:a [read 10240000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:01.142 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.382998+0000", 2024-09-21T21:41:01.142 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 2.7147267689999999, 2024-09-21T21:41:01.142 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.010571644, 2024-09-21T21:41:01.142 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:01.143 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:01.143 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:01.143 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:01.143 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:01.143 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:01.143 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1856 2024-09-21T21:41:01.143 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.143 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:01.143 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.143 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:01.143 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382998+0000", 2024-09-21T21:41:01.143 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:01.143 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.143 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.143 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:01.143 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382998+0000", 2024-09-21T21:41:01.144 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:01.144 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.144 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.144 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:01.144 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382999+0000", 2024-09-21T21:41:01.144 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7700000000000004e-07 2024-09-21T21:41:01.144 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.144 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.144 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:01.144 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383003+0000", 2024-09-21T21:41:01.144 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.822e-06 2024-09-21T21:41:01.144 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.144 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.144 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:01.144 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383014+0000", 2024-09-21T21:41:01.144 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0713000000000001e-05 2024-09-21T21:41:01.145 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.145 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.145 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:01.145 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383020+0000", 2024-09-21T21:41:01.145 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3160000000000002e-06 2024-09-21T21:41:01.145 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.145 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.145 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:01.145 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.393394+0000", 2024-09-21T21:41:01.145 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.010374362 2024-09-21T21:41:01.145 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.145 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.145 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:01.145 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.393484+0000", 2024-09-21T21:41:01.145 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.9975000000000005e-05 2024-09-21T21:41:01.145 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.146 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.146 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:01.146 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.393570+0000", 2024-09-21T21:41:01.146 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.5679000000000004e-05 2024-09-21T21:41:01.146 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.146 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:01.146 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.146 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.146 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.146 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:1858 3.12 3:4bfa5158:::rbd_data.112145fae779.0000000000000002:a [read 10297344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:01.146 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.383036+0000", 2024-09-21T21:41:01.146 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 2.714689592, 2024-09-21T21:41:01.146 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.010660364, 2024-09-21T21:41:01.146 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:01.146 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:01.147 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:01.147 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:01.147 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:01.147 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:01.147 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1858 2024-09-21T21:41:01.147 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.147 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:01.147 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.147 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:01.147 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383036+0000", 2024-09-21T21:41:01.147 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:01.147 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.147 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.147 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:01.148 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383036+0000", 2024-09-21T21:41:01.148 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:01.148 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.148 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.148 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:01.148 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383036+0000", 2024-09-21T21:41:01.148 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8899999999999999e-07 2024-09-21T21:41:01.148 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.148 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.148 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:01.148 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383039+0000", 2024-09-21T21:41:01.148 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.6510000000000001e-06 2024-09-21T21:41:01.148 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.148 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.148 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:01.149 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383048+0000", 2024-09-21T21:41:01.149 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6530000000000006e-06 2024-09-21T21:41:01.149 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.149 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.149 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:01.149 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383053+0000", 2024-09-21T21:41:01.149 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.933e-06 2024-09-21T21:41:01.149 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.149 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.149 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:01.149 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.393608+0000", 2024-09-21T21:41:01.149 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.01055442 2024-09-21T21:41:01.149 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.149 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.149 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:01.149 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.393650+0000", 2024-09-21T21:41:01.149 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2586999999999997e-05 2024-09-21T21:41:01.150 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.150 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.150 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:01.150 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.393696+0000", 2024-09-21T21:41:01.150 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5531000000000002e-05 2024-09-21T21:41:01.150 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.150 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:01.150 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.150 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.150 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.150 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:1861 3.12 3:4bfa5158:::rbd_data.112145fae779.0000000000000002:a [read 10387456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:01.150 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.383063+0000", 2024-09-21T21:41:01.150 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 2.7146616720000001, 2024-09-21T21:41:01.150 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.010754104, 2024-09-21T21:41:01.150 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:01.151 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:01.151 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:01.151 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:01.151 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:01.151 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:01.151 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1861 2024-09-21T21:41:01.151 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.151 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:01.151 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.151 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:01.151 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383063+0000", 2024-09-21T21:41:01.151 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:01.151 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.151 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.151 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:01.152 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383063+0000", 2024-09-21T21:41:01.152 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:01.152 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.152 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.152 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:01.152 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383064+0000", 2024-09-21T21:41:01.152 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9399999999999995e-07 2024-09-21T21:41:01.152 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.152 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.152 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:01.152 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383067+0000", 2024-09-21T21:41:01.152 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.6019999999999998e-06 2024-09-21T21:41:01.152 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.153 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.153 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:01.153 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383075+0000", 2024-09-21T21:41:01.153 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.1850000000000004e-06 2024-09-21T21:41:01.153 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.153 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.153 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:01.153 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383080+0000", 2024-09-21T21:41:01.153 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3909999999999997e-06 2024-09-21T21:41:01.153 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.153 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.153 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:01.153 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.393727+0000", 2024-09-21T21:41:01.153 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.010646631 2024-09-21T21:41:01.153 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.154 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.154 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:01.154 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.393777+0000", 2024-09-21T21:41:01.154 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0356999999999999e-05 2024-09-21T21:41:01.154 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.154 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.154 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:01.154 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.393818+0000", 2024-09-21T21:41:01.154 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.0444000000000001e-05 2024-09-21T21:41:01.154 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.154 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:01.154 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.154 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.154 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.155 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:1863 3.12 3:4bfa5158:::rbd_data.112145fae779.0000000000000002:a [read 10444800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:01.155 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.383088+0000", 2024-09-21T21:41:01.155 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 2.7146370279999998, 2024-09-21T21:41:01.155 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.010836438, 2024-09-21T21:41:01.155 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:01.155 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:01.155 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:01.155 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:01.155 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:01.155 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:01.155 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1863 2024-09-21T21:41:01.155 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.155 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:01.155 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.156 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:01.156 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383088+0000", 2024-09-21T21:41:01.156 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:01.156 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.156 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.156 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:01.156 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383088+0000", 2024-09-21T21:41:01.156 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:01.156 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.156 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.156 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:01.156 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383089+0000", 2024-09-21T21:41:01.156 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2500000000000001e-07 2024-09-21T21:41:01.156 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.157 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.157 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:01.157 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383091+0000", 2024-09-21T21:41:01.157 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.283e-06 2024-09-21T21:41:01.157 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.157 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.157 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:01.157 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383097+0000", 2024-09-21T21:41:01.157 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1279999999999996e-06 2024-09-21T21:41:01.157 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.157 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.157 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:01.157 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383103+0000", 2024-09-21T21:41:01.157 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6219999999999997e-06 2024-09-21T21:41:01.158 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.158 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.158 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:01.158 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.393845+0000", 2024-09-21T21:41:01.158 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.010742188999999999 2024-09-21T21:41:01.158 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.158 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.158 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:01.158 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.393886+0000", 2024-09-21T21:41:01.158 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1119999999999999e-05 2024-09-21T21:41:01.158 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.158 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.158 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:01.158 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.393925+0000", 2024-09-21T21:41:01.159 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8671000000000003e-05 2024-09-21T21:41:01.159 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.159 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:01.159 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.159 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.159 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.159 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:1865 3.12 3:4bfa5158:::rbd_data.112145fae779.0000000000000002:a [read 10534912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:01.159 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.383134+0000", 2024-09-21T21:41:01.159 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 2.7145912409999999, 2024-09-21T21:41:01.159 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.01092018, 2024-09-21T21:41:01.159 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:01.159 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:01.159 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:01.159 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:01.159 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:01.160 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:01.160 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1865 2024-09-21T21:41:01.160 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.160 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:01.160 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.160 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:01.160 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383134+0000", 2024-09-21T21:41:01.160 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:01.160 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.160 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.160 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:01.160 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383134+0000", 2024-09-21T21:41:01.160 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:01.160 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.160 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.160 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:01.161 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383135+0000", 2024-09-21T21:41:01.161 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.9800000000000002e-07 2024-09-21T21:41:01.161 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.161 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.161 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:01.161 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383140+0000", 2024-09-21T21:41:01.161 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7099999999999998e-06 2024-09-21T21:41:01.161 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.161 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.161 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:01.161 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383150+0000", 2024-09-21T21:41:01.161 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0472000000000001e-05 2024-09-21T21:41:01.161 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.161 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.161 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:01.161 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383157+0000", 2024-09-21T21:41:01.162 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9029999999999998e-06 2024-09-21T21:41:01.162 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.162 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.162 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:01.162 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.393973+0000", 2024-09-21T21:41:01.162 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.010816299 2024-09-21T21:41:01.162 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.162 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.162 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:01.162 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.394015+0000", 2024-09-21T21:41:01.162 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1467000000000001e-05 2024-09-21T21:41:01.162 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.162 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.162 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:01.163 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.394054+0000", 2024-09-21T21:41:01.163 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.9331e-05 2024-09-21T21:41:01.163 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.163 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:01.163 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.163 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.163 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.163 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:1867 3.12 3:4bfa5158:::rbd_data.112145fae779.0000000000000002:a [read 10592256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:01.163 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.383287+0000", 2024-09-21T21:41:01.163 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 2.7144379160000001, 2024-09-21T21:41:01.163 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.01089415, 2024-09-21T21:41:01.163 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:01.163 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:01.163 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:01.163 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:01.163 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:01.164 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:01.164 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1867 2024-09-21T21:41:01.164 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.164 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:01.164 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.164 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:01.164 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383287+0000", 2024-09-21T21:41:01.164 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:01.164 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.164 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.164 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:01.164 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383287+0000", 2024-09-21T21:41:01.164 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:01.164 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.164 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.165 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:01.165 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383288+0000", 2024-09-21T21:41:01.165 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.02e-07 2024-09-21T21:41:01.165 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.165 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.165 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:01.165 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383293+0000", 2024-09-21T21:41:01.165 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8339999999999998e-06 2024-09-21T21:41:01.165 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.165 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.165 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:01.165 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383305+0000", 2024-09-21T21:41:01.165 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1677e-05 2024-09-21T21:41:01.165 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.165 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.166 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:01.166 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383312+0000", 2024-09-21T21:41:01.166 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9079999999999998e-06 2024-09-21T21:41:01.166 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.166 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.166 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:01.166 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.394080+0000", 2024-09-21T21:41:01.166 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.010768402 2024-09-21T21:41:01.166 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.166 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.166 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:01.166 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.394131+0000", 2024-09-21T21:41:01.166 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0926e-05 2024-09-21T21:41:01.166 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.166 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.167 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:01.167 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.394181+0000", 2024-09-21T21:41:01.167 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0501000000000003e-05 2024-09-21T21:41:01.167 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.167 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:01.167 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.167 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.167 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.167 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:1869 3.12 3:4bfa5158:::rbd_data.112145fae779.0000000000000002:a [read 10625024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:01.167 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.383348+0000", 2024-09-21T21:41:01.167 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 2.7143773649999998, 2024-09-21T21:41:01.167 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.011004575000000001, 2024-09-21T21:41:01.167 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:01.167 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:01.168 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:01.168 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:01.168 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:01.168 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:01.168 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1869 2024-09-21T21:41:01.168 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.168 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:01.168 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.168 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:01.168 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383348+0000", 2024-09-21T21:41:01.168 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:01.168 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.168 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.168 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:01.168 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383348+0000", 2024-09-21T21:41:01.168 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:01.168 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.169 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.169 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:01.169 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383348+0000", 2024-09-21T21:41:01.169 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5400000000000001e-07 2024-09-21T21:41:01.169 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.169 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.169 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:01.169 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383352+0000", 2024-09-21T21:41:01.169 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4089999999999999e-06 2024-09-21T21:41:01.169 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.169 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.169 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:01.169 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383360+0000", 2024-09-21T21:41:01.169 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.2430000000000005e-06 2024-09-21T21:41:01.169 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.170 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.170 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:01.170 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383366+0000", 2024-09-21T21:41:01.170 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4909999999999996e-06 2024-09-21T21:41:01.170 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.170 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.170 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:01.170 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.394212+0000", 2024-09-21T21:41:01.170 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.010846000999999999 2024-09-21T21:41:01.170 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.170 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.170 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:01.170 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.394279+0000", 2024-09-21T21:41:01.170 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.7873000000000006e-05 2024-09-21T21:41:01.171 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.171 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.171 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:01.171 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.394352+0000", 2024-09-21T21:41:01.171 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.2904000000000005e-05 2024-09-21T21:41:01.171 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.171 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:01.171 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.171 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.171 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.171 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:1871 3.12 3:4bfa5158:::rbd_data.112145fae779.0000000000000002:a [read 10682368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:01.171 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.383497+0000", 2024-09-21T21:41:01.171 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 2.7142283850000002, 2024-09-21T21:41:01.171 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.010980227, 2024-09-21T21:41:01.172 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:01.172 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:01.172 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:01.172 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:01.172 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:01.172 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:01.172 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1871 2024-09-21T21:41:01.172 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.172 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:01.172 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.172 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:01.172 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383497+0000", 2024-09-21T21:41:01.172 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:01.172 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.172 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.173 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:01.173 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383497+0000", 2024-09-21T21:41:01.173 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:01.173 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.173 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.173 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:01.173 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383498+0000", 2024-09-21T21:41:01.173 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2500000000000004e-07 2024-09-21T21:41:01.173 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.173 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.173 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:01.173 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383502+0000", 2024-09-21T21:41:01.173 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7049999999999998e-06 2024-09-21T21:41:01.173 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.173 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.174 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:01.174 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383512+0000", 2024-09-21T21:41:01.174 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.3239999999999992e-06 2024-09-21T21:41:01.174 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.174 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.174 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:01.174 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383518+0000", 2024-09-21T21:41:01.174 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5370000000000003e-06 2024-09-21T21:41:01.174 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.174 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.174 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:01.174 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.394378+0000", 2024-09-21T21:41:01.174 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.010859897 2024-09-21T21:41:01.174 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.175 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.175 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:01.175 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.394434+0000", 2024-09-21T21:41:01.175 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6198e-05 2024-09-21T21:41:01.175 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.175 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.175 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:01.175 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.394477+0000", 2024-09-21T21:41:01.175 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2641000000000001e-05 2024-09-21T21:41:01.175 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.175 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:01.176 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.176 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.176 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.176 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:1873 3.12 3:4bfa5158:::rbd_data.112145fae779.0000000000000002:a [read 10739712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:01.176 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.383639+0000", 2024-09-21T21:41:01.176 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 2.7140866309999998, 2024-09-21T21:41:01.176 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.011039125, 2024-09-21T21:41:01.176 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:01.176 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:01.176 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:01.176 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:01.176 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:01.176 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:01.176 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1873 2024-09-21T21:41:01.176 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.177 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:01.177 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.177 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:01.177 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383639+0000", 2024-09-21T21:41:01.177 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:01.177 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.177 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.177 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:01.177 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383639+0000", 2024-09-21T21:41:01.177 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:01.177 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.177 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.177 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:01.177 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383639+0000", 2024-09-21T21:41:01.178 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.8000000000000004e-07 2024-09-21T21:41:01.178 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.178 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.178 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:01.178 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383644+0000", 2024-09-21T21:41:01.178 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3699999999999997e-06 2024-09-21T21:41:01.178 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.178 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.178 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:01.178 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383653+0000", 2024-09-21T21:41:01.178 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.3330000000000003e-06 2024-09-21T21:41:01.178 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.178 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.178 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:01.179 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383659+0000", 2024-09-21T21:41:01.179 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7060000000000001e-06 2024-09-21T21:41:01.179 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.179 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.179 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:01.179 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.394518+0000", 2024-09-21T21:41:01.179 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.010859647 2024-09-21T21:41:01.179 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.179 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.179 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:01.179 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.394599+0000", 2024-09-21T21:41:01.179 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0770000000000001e-05 2024-09-21T21:41:01.179 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.179 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.180 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:01.180 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.394678+0000", 2024-09-21T21:41:01.180 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.8418999999999996e-05 2024-09-21T21:41:01.180 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.180 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:01.180 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.180 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.180 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.180 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4491 3.12 3:4bfa5158:::rbd_data.112145fae779.0000000000000002:a [read 2269184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:01.180 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.513914+0000", 2024-09-21T21:41:01.180 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 2.5838109789999999, 2024-09-21T21:41:01.180 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0098656990000000003, 2024-09-21T21:41:01.180 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:01.180 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:01.180 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:01.181 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:01.181 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:01.181 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:01.181 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4491 2024-09-21T21:41:01.181 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.181 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:01.181 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.181 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:01.181 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.513914+0000", 2024-09-21T21:41:01.181 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:01.181 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.181 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.181 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:01.181 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.513914+0000", 2024-09-21T21:41:01.181 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:01.181 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.182 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.182 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:01.182 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.513914+0000", 2024-09-21T21:41:01.182 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.11e-07 2024-09-21T21:41:01.182 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.182 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.182 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:01.182 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.513916+0000", 2024-09-21T21:41:01.182 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1799999999999999e-06 2024-09-21T21:41:01.182 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.182 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.182 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:01.182 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.513919+0000", 2024-09-21T21:41:01.182 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5099999999999999e-06 2024-09-21T21:41:01.182 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.183 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.183 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:01.183 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.513922+0000", 2024-09-21T21:41:01.183 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.429e-06 2024-09-21T21:41:01.183 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.183 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.183 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:01.183 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.521609+0000", 2024-09-21T21:41:01.183 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0076873979999999998 2024-09-21T21:41:01.183 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.183 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.183 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:01.183 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.521700+0000", 2024-09-21T21:41:01.184 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.0675999999999997e-05 2024-09-21T21:41:01.184 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.184 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.184 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:01.184 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.523780+0000", 2024-09-21T21:41:01.184 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0020802949999999998 2024-09-21T21:41:01.184 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.184 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:01.184 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.184 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.184 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:01.184 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:01.185 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T21:41:01.198 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:01.196+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:01.235 INFO:teuthology.orchestra.run.smithi164.stdout:1183 write 0x58ae400 thru 0x58bbe1a (0xda1b bytes) 2024-09-21T21:41:01.239 INFO:teuthology.orchestra.run.smithi164.stdout:1185 read 0x1460400 thru 0x1467ac0 (0x76c1 bytes) 2024-09-21T21:41:01.244 INFO:teuthology.orchestra.run.smithi164.stdout:1190 trunc from 0x809b200 to 0x40fe200 2024-09-21T21:41:01.288 INFO:teuthology.orchestra.run.smithi164.stdout:1191 write 0x2444800 thru 0x2453e89 (0xf68a bytes) 2024-09-21T21:41:01.298 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:01.297+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:01.319 INFO:teuthology.orchestra.run.smithi164.stdout:1193 write 0xb023200 thru 0xb032d69 (0xfb6a bytes) 2024-09-21T21:41:01.319 INFO:teuthology.orchestra.run.smithi164.stdout:1194 write 0x3e87600 thru 0x3e9149e (0x9e9f bytes) 2024-09-21T21:41:01.320 INFO:teuthology.orchestra.run.smithi164.stdout:1195 punch from 0x33eca00 to 0x33f7a00, (0xb000 bytes) 2024-09-21T21:41:01.320 INFO:teuthology.orchestra.run.smithi164.stdout:1199 punch from 0x1871600 to 0x1875000, (0x3a00 bytes) 2024-09-21T21:41:01.342 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:01.342 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:41:01.342 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.343 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4824.0:731 3.4 3:2296f847:::rbd_data.1121e540c096.0000000000000003:b [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-21T21:41:01.343 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:01.341482+0000", 2024-09-21T21:41:01.343 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.000122031, 2024-09-21T21:41:01.343 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000145928, 2024-09-21T21:41:01.343 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:01.343 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:01.343 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:41:01.343 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:01.343 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4824", 2024-09-21T21:41:01.343 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:01.343 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 731 2024-09-21T21:41:01.343 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.343 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:01.343 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.343 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:01.344 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.341482+0000", 2024-09-21T21:41:01.344 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:01.344 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.344 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.344 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:01.344 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.341482+0000", 2024-09-21T21:41:01.344 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:01.344 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.344 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.344 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:01.344 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.341483+0000", 2024-09-21T21:41:01.344 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0279999999999999e-06 2024-09-21T21:41:01.344 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.344 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.344 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:01.344 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.341488+0000", 2024-09-21T21:41:01.345 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1930000000000001e-06 2024-09-21T21:41:01.345 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.345 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.345 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:01.345 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.341502+0000", 2024-09-21T21:41:01.345 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.4151999999999999e-05 2024-09-21T21:41:01.345 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.345 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.345 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:01.345 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.341509+0000", 2024-09-21T21:41:01.345 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0380000000000002e-06 2024-09-21T21:41:01.345 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.345 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.345 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:01.345 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.341565+0000", 2024-09-21T21:41:01.345 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5727999999999999e-05 2024-09-21T21:41:01.345 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.346 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:01.346 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.346 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.346 INFO:teuthology.orchestra.run.smithi081.stdout: ], 2024-09-21T21:41:01.346 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 1 2024-09-21T21:41:01.346 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:01.353 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T21:41:01.357 INFO:teuthology.orchestra.run.smithi164.stdout:1200 write 0xe65f800 thru 0xe66c48c (0xcc8d bytes) 2024-09-21T21:41:01.357 INFO:teuthology.orchestra.run.smithi164.stdout:1204 punch from 0x1d0a00 to 0x1d9000, (0x8600 bytes) 2024-09-21T21:41:01.364 INFO:teuthology.orchestra.run.smithi164.stdout:1207 trunc from 0xe66c600 to 0x920da00 2024-09-21T21:41:01.378 INFO:teuthology.orchestra.run.smithi164.stdout:1211 write 0x84d1200 thru 0x84e063c (0xf43d bytes) 2024-09-21T21:41:01.399 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:01.398+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:01.402 INFO:teuthology.orchestra.run.smithi164.stdout:1212 write 0xd438200 thru 0xd44305d (0xae5e bytes) 2024-09-21T21:41:01.407 INFO:teuthology.orchestra.run.smithi164.stdout:1215 read 0x29d4400 thru 0x29db9f1 (0x75f2 bytes) 2024-09-21T21:41:01.408 INFO:teuthology.orchestra.run.smithi164.stdout:1218 write 0xbc6f200 thru 0xbc6f300 (0x101 bytes) 2024-09-21T21:41:01.408 INFO:teuthology.orchestra.run.smithi164.stdout:1219 punch from 0x96e5e00 to 0x96e8600, (0x2800 bytes) 2024-09-21T21:41:01.409 INFO:teuthology.orchestra.run.smithi164.stdout:1223 clone 10 order 22 su 1048576 sc 11 2024-09-21T21:41:01.465 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:01.465 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:41:01.465 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:41:01.465 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:41:01.465 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:01.474 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T21:41:01.500 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:01.499+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:01.601 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:01.599+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:01.624 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:01.624 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 20, 2024-09-21T21:41:01.624 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 600, 2024-09-21T21:41:01.624 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:41:01.624 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.624 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:2885 3.b 3:d49b370f:::rbd_object_map.1121e540c096:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:01.624 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.478228+0000", 2024-09-21T21:41:01.624 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.1431157980000002, 2024-09-21T21:41:01.624 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.015327835999999999, 2024-09-21T21:41:01.625 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:01.625 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:01.625 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:41:01.625 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:01.625 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:01.625 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:01.625 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2885 2024-09-21T21:41:01.625 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.625 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:01.625 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.625 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:01.625 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478228+0000", 2024-09-21T21:41:01.625 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:01.625 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.626 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.626 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:01.626 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478228+0000", 2024-09-21T21:41:01.626 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:01.626 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.626 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.626 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:01.626 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478229+0000", 2024-09-21T21:41:01.626 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8000000000000001e-07 2024-09-21T21:41:01.626 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.626 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.626 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:01.626 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478231+0000", 2024-09-21T21:41:01.626 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.5069999999999999e-06 2024-09-21T21:41:01.627 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.627 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.627 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:01.627 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478238+0000", 2024-09-21T21:41:01.627 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.8820000000000003e-06 2024-09-21T21:41:01.627 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.627 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.627 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:01.627 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478243+0000", 2024-09-21T21:41:01.627 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2209999999999996e-06 2024-09-21T21:41:01.627 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.627 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.627 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:01.628 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.488160+0000", 2024-09-21T21:41:01.628 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0099169849999999997 2024-09-21T21:41:01.628 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.628 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.628 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:01.628 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.488231+0000", 2024-09-21T21:41:01.628 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0909999999999997e-05 2024-09-21T21:41:01.628 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.628 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.628 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 0", 2024-09-21T21:41:01.628 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.488538+0000", 2024-09-21T21:41:01.628 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000306736 2024-09-21T21:41:01.628 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.628 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.629 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:41:01.629 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.493210+0000", 2024-09-21T21:41:01.629 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0046722120000000002 2024-09-21T21:41:01.629 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.629 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.629 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:41:01.629 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.493471+0000", 2024-09-21T21:41:01.629 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00026132899999999998 2024-09-21T21:41:01.629 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.629 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.629 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:41:01.629 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.493517+0000", 2024-09-21T21:41:01.629 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5309000000000003e-05 2024-09-21T21:41:01.629 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.630 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.630 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:01.630 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.493556+0000", 2024-09-21T21:41:01.630 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.9365e-05 2024-09-21T21:41:01.630 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.630 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:01.630 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.630 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.630 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.630 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:2886 3.b 3:d49b370f:::rbd_object_map.1121e540c096:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:01.631 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.478249+0000", 2024-09-21T21:41:01.631 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.1430950179999999, 2024-09-21T21:41:01.631 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.015426120999999999, 2024-09-21T21:41:01.631 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:01.631 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:01.631 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:41:01.631 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:01.631 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:01.631 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:01.631 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2886 2024-09-21T21:41:01.631 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.631 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:01.631 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.631 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:01.632 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478249+0000", 2024-09-21T21:41:01.632 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:01.632 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.632 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.632 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:01.632 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478249+0000", 2024-09-21T21:41:01.632 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:01.632 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.632 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.632 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:01.632 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478249+0000", 2024-09-21T21:41:01.632 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.9999999999999998e-07 2024-09-21T21:41:01.632 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.632 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.632 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:01.633 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478252+0000", 2024-09-21T21:41:01.633 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.5500000000000001e-06 2024-09-21T21:41:01.633 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.633 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.633 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:01.633 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478258+0000", 2024-09-21T21:41:01.633 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7080000000000002e-06 2024-09-21T21:41:01.633 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.633 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.633 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:01.633 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478265+0000", 2024-09-21T21:41:01.633 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9110000000000004e-06 2024-09-21T21:41:01.633 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.633 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.633 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:01.634 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.488889+0000", 2024-09-21T21:41:01.634 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.010624662 2024-09-21T21:41:01.634 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.634 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.634 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:01.634 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.488965+0000", 2024-09-21T21:41:01.634 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5949000000000001e-05 2024-09-21T21:41:01.634 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.634 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.634 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 0", 2024-09-21T21:41:01.634 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.489235+0000", 2024-09-21T21:41:01.634 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00027019499999999998 2024-09-21T21:41:01.635 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.635 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.635 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:41:01.635 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.493233+0000", 2024-09-21T21:41:01.635 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0039978799999999997 2024-09-21T21:41:01.635 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.635 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.635 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:41:01.635 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.493601+0000", 2024-09-21T21:41:01.635 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00036728200000000003 2024-09-21T21:41:01.636 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.636 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.636 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:41:01.636 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.493637+0000", 2024-09-21T21:41:01.636 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6263000000000002e-05 2024-09-21T21:41:01.636 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.636 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.637 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:01.637 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.493675+0000", 2024-09-21T21:41:01.637 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8321000000000001e-05 2024-09-21T21:41:01.637 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.637 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:01.637 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.637 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.638 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.638 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:2887 3.b 3:d49b370f:::rbd_object_map.1121e540c096:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:01.638 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.478271+0000", 2024-09-21T21:41:01.638 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.1430735059999999, 2024-09-21T21:41:01.638 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.015551966, 2024-09-21T21:41:01.638 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:01.638 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:01.639 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:41:01.639 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:01.639 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:01.639 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:01.640 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2887 2024-09-21T21:41:01.640 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.640 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:01.640 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.640 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:01.640 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478271+0000", 2024-09-21T21:41:01.640 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:01.641 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.641 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.641 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:01.641 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478271+0000", 2024-09-21T21:41:01.641 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:01.642 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.642 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.642 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:01.642 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478271+0000", 2024-09-21T21:41:01.642 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.0600000000000001e-07 2024-09-21T21:41:01.642 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.642 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.643 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:01.643 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478274+0000", 2024-09-21T21:41:01.643 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.6390000000000001e-06 2024-09-21T21:41:01.643 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.643 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.643 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:01.644 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478280+0000", 2024-09-21T21:41:01.644 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3219999999999998e-06 2024-09-21T21:41:01.644 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.644 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.644 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:01.644 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478285+0000", 2024-09-21T21:41:01.644 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6589999999999998e-06 2024-09-21T21:41:01.645 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.645 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.645 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:01.645 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.489524+0000", 2024-09-21T21:41:01.645 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.01123979 2024-09-21T21:41:01.645 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.646 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.646 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:01.646 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.489567+0000", 2024-09-21T21:41:01.646 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2673000000000003e-05 2024-09-21T21:41:01.646 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.646 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.647 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 0", 2024-09-21T21:41:01.647 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.489834+0000", 2024-09-21T21:41:01.647 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00026672799999999998 2024-09-21T21:41:01.647 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.647 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.647 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:41:01.648 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.493290+0000", 2024-09-21T21:41:01.648 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0034557429999999998 2024-09-21T21:41:01.648 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.648 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.649 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:41:01.649 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.493737+0000", 2024-09-21T21:41:01.649 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00044731799999999999 2024-09-21T21:41:01.649 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.649 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.649 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:41:01.649 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.493784+0000", 2024-09-21T21:41:01.650 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7178999999999997e-05 2024-09-21T21:41:01.650 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.650 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.650 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:01.650 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.493823+0000", 2024-09-21T21:41:01.650 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8509e-05 2024-09-21T21:41:01.651 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.651 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:01.651 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.651 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.651 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.651 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4324 3.0 3:074cff57:::rbd_data.112145fae779.000000000000000c:a [read 2007040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:01.651 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.510116+0000", 2024-09-21T21:41:01.651 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.1112276969999999, 2024-09-21T21:41:01.652 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013165949, 2024-09-21T21:41:01.652 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:01.652 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:01.652 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:01.652 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:01.652 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:01.653 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:01.653 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4324 2024-09-21T21:41:01.653 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.653 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:01.653 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.653 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:01.653 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510116+0000", 2024-09-21T21:41:01.653 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:01.653 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.653 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.653 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:01.654 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510116+0000", 2024-09-21T21:41:01.654 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:01.654 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.654 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.654 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:01.654 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510117+0000", 2024-09-21T21:41:01.654 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5999999999999999e-07 2024-09-21T21:41:01.654 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.654 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.654 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:01.654 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510119+0000", 2024-09-21T21:41:01.654 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0600000000000002e-06 2024-09-21T21:41:01.655 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.655 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.655 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:01.655 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510129+0000", 2024-09-21T21:41:01.655 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0485999999999999e-05 2024-09-21T21:41:01.655 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.655 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.655 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:01.655 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510134+0000", 2024-09-21T21:41:01.655 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.028e-06 2024-09-21T21:41:01.655 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.655 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.656 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:01.656 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.523118+0000", 2024-09-21T21:41:01.656 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.012983877 2024-09-21T21:41:01.656 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.656 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.656 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:01.656 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.523195+0000", 2024-09-21T21:41:01.656 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.6456000000000001e-05 2024-09-21T21:41:01.656 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.657 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.657 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:01.657 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.523282+0000", 2024-09-21T21:41:01.657 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.7682000000000005e-05 2024-09-21T21:41:01.657 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.657 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:01.657 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.657 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.657 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.658 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4325 3.0 3:074cff57:::rbd_data.112145fae779.000000000000000c:a [read 2064384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:01.658 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.510141+0000", 2024-09-21T21:41:01.658 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.1112033440000002, 2024-09-21T21:41:01.658 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013330887, 2024-09-21T21:41:01.658 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:01.658 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:01.658 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:01.658 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:01.658 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:01.658 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:01.658 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4325 2024-09-21T21:41:01.658 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.659 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:01.659 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.659 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:01.659 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510141+0000", 2024-09-21T21:41:01.659 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:01.659 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.659 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.659 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:01.659 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510141+0000", 2024-09-21T21:41:01.659 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:01.659 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.659 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.660 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:01.660 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510141+0000", 2024-09-21T21:41:01.660 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.72e-07 2024-09-21T21:41:01.660 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.660 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.660 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:01.660 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510143+0000", 2024-09-21T21:41:01.660 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2409999999999998e-06 2024-09-21T21:41:01.660 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.660 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.660 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:01.660 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510151+0000", 2024-09-21T21:41:01.660 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.3390000000000004e-06 2024-09-21T21:41:01.661 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.661 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.661 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:01.661 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510155+0000", 2024-09-21T21:41:01.661 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5630000000000002e-06 2024-09-21T21:41:01.661 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.661 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.661 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:01.661 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.523310+0000", 2024-09-21T21:41:01.661 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013154658 2024-09-21T21:41:01.661 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.661 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.661 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:01.661 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.523386+0000", 2024-09-21T21:41:01.661 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.6553999999999999e-05 2024-09-21T21:41:01.661 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.661 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.662 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:01.662 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.523472+0000", 2024-09-21T21:41:01.662 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.5160000000000005e-05 2024-09-21T21:41:01.662 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.662 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:01.662 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.662 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.662 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.662 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4326 3.0 3:074cff57:::rbd_data.112145fae779.000000000000000c:a [read 2097152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:01.662 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.510162+0000", 2024-09-21T21:41:01.662 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.111182329, 2024-09-21T21:41:01.662 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013537466, 2024-09-21T21:41:01.662 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:01.662 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:01.663 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:01.663 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:01.663 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:01.663 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:01.663 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4326 2024-09-21T21:41:01.663 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.663 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:01.663 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.663 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:01.663 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510162+0000", 2024-09-21T21:41:01.663 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:01.663 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.663 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.663 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:01.663 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510162+0000", 2024-09-21T21:41:01.664 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:01.664 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.664 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.664 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:01.664 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510162+0000", 2024-09-21T21:41:01.664 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.0699999999999998e-07 2024-09-21T21:41:01.664 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.664 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.664 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:01.664 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510164+0000", 2024-09-21T21:41:01.664 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.17e-06 2024-09-21T21:41:01.664 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.664 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.665 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:01.665 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510174+0000", 2024-09-21T21:41:01.665 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2439999999999999e-06 2024-09-21T21:41:01.665 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.665 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.665 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:01.665 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510179+0000", 2024-09-21T21:41:01.665 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2159999999999997e-06 2024-09-21T21:41:01.665 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.665 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.665 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:01.665 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.523502+0000", 2024-09-21T21:41:01.665 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013322779 2024-09-21T21:41:01.666 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.666 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.666 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:01.666 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.523579+0000", 2024-09-21T21:41:01.666 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7402000000000002e-05 2024-09-21T21:41:01.666 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.666 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.666 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:01.666 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.523699+0000", 2024-09-21T21:41:01.666 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000120248 2024-09-21T21:41:01.666 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.666 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:01.667 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.667 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.667 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.667 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4327 3.0 3:074cff57:::rbd_data.112145fae779.000000000000000c:a [read 2154496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:01.667 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.510418+0000", 2024-09-21T21:41:01.667 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.1109258940000002, 2024-09-21T21:41:01.667 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013505039999999999, 2024-09-21T21:41:01.667 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:01.667 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:01.667 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:01.667 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:01.667 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:01.667 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:01.667 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4327 2024-09-21T21:41:01.668 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.668 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:01.668 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.668 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:01.668 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510418+0000", 2024-09-21T21:41:01.668 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:01.668 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.668 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.668 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:01.668 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510418+0000", 2024-09-21T21:41:01.668 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:01.668 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.668 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.669 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:01.669 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510419+0000", 2024-09-21T21:41:01.669 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8999999999999997e-07 2024-09-21T21:41:01.669 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.669 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.669 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:01.669 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510421+0000", 2024-09-21T21:41:01.669 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.4779999999999998e-06 2024-09-21T21:41:01.669 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.669 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.669 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:01.669 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510427+0000", 2024-09-21T21:41:01.670 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5609999999999998e-06 2024-09-21T21:41:01.670 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.670 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.670 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:01.670 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510432+0000", 2024-09-21T21:41:01.670 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5389999999999998e-06 2024-09-21T21:41:01.670 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.670 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.670 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:01.670 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.523743+0000", 2024-09-21T21:41:01.670 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013310963 2024-09-21T21:41:01.670 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.670 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.671 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:01.671 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.523824+0000", 2024-09-21T21:41:01.671 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0463000000000005e-05 2024-09-21T21:41:01.671 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.671 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.671 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:01.671 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.523923+0000", 2024-09-21T21:41:01.671 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.9545999999999999e-05 2024-09-21T21:41:01.671 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.671 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:01.671 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.671 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.671 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.672 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4328 3.0 3:074cff57:::rbd_data.112145fae779.000000000000000c:a [read 2211840~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:01.672 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.510443+0000", 2024-09-21T21:41:01.672 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.110901594, 2024-09-21T21:41:01.672 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013684564, 2024-09-21T21:41:01.672 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:01.672 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:01.672 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:01.672 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:01.672 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:01.672 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:01.672 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4328 2024-09-21T21:41:01.672 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.672 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:01.673 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.673 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:01.673 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510443+0000", 2024-09-21T21:41:01.673 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:01.673 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.673 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.673 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:01.673 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510443+0000", 2024-09-21T21:41:01.673 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:01.673 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.673 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.673 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:01.673 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510443+0000", 2024-09-21T21:41:01.674 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2199999999999999e-07 2024-09-21T21:41:01.674 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.674 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.674 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:01.674 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510445+0000", 2024-09-21T21:41:01.674 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1679999999999998e-06 2024-09-21T21:41:01.674 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.674 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.674 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:01.674 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510452+0000", 2024-09-21T21:41:01.674 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6259999999999997e-06 2024-09-21T21:41:01.674 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.674 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.674 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:01.675 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510457+0000", 2024-09-21T21:41:01.675 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8110000000000002e-06 2024-09-21T21:41:01.675 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.675 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.675 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:01.675 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.523968+0000", 2024-09-21T21:41:01.675 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013510996000000001 2024-09-21T21:41:01.675 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.675 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.675 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:01.675 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.524044+0000", 2024-09-21T21:41:01.675 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.6259000000000003e-05 2024-09-21T21:41:01.675 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.676 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.676 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:01.676 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.524127+0000", 2024-09-21T21:41:01.676 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.3282000000000007e-05 2024-09-21T21:41:01.676 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.676 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:01.676 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.676 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.676 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.676 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4577 3.0 3:074cff57:::rbd_data.112145fae779.000000000000000c:a [read 1744896~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:01.676 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.514327+0000", 2024-09-21T21:41:01.676 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.107016937, 2024-09-21T21:41:01.676 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013451424999999999, 2024-09-21T21:41:01.676 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:01.677 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:01.677 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:01.677 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:01.677 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:01.677 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:01.677 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4577 2024-09-21T21:41:01.677 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.677 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:01.677 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.677 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:01.677 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514327+0000", 2024-09-21T21:41:01.677 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:01.677 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.677 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.677 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:01.678 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514327+0000", 2024-09-21T21:41:01.678 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:01.678 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.678 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.678 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:01.678 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514328+0000", 2024-09-21T21:41:01.678 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4900000000000001e-07 2024-09-21T21:41:01.678 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.678 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.678 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:01.678 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514330+0000", 2024-09-21T21:41:01.678 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.227e-06 2024-09-21T21:41:01.678 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.679 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.679 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:01.679 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514336+0000", 2024-09-21T21:41:01.679 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1940000000000003e-06 2024-09-21T21:41:01.679 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.679 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.679 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:01.679 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514341+0000", 2024-09-21T21:41:01.679 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7319999999999998e-06 2024-09-21T21:41:01.679 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.679 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.679 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:01.679 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.527415+0000", 2024-09-21T21:41:01.679 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013074657 2024-09-21T21:41:01.679 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.680 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.680 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:01.680 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.527491+0000", 2024-09-21T21:41:01.680 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5899e-05 2024-09-21T21:41:01.680 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.680 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.680 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:01.680 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.527779+0000", 2024-09-21T21:41:01.680 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00028736700000000002 2024-09-21T21:41:01.680 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.680 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:01.680 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.680 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.680 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.680 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4578 3.0 3:074cff57:::rbd_data.112145fae779.000000000000000c:a [read 1802240~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:01.680 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.514347+0000", 2024-09-21T21:41:01.681 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.1069974820000001, 2024-09-21T21:41:01.681 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.014025592999999999, 2024-09-21T21:41:01.681 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:01.681 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:01.681 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:01.681 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:01.681 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:01.681 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:01.681 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4578 2024-09-21T21:41:01.681 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.681 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:01.681 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.681 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:01.681 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514347+0000", 2024-09-21T21:41:01.681 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:01.681 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.682 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.682 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:01.682 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514347+0000", 2024-09-21T21:41:01.682 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:01.682 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.682 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.682 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:01.682 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514347+0000", 2024-09-21T21:41:01.682 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6699999999999999e-07 2024-09-21T21:41:01.682 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.682 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.682 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:01.682 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514349+0000", 2024-09-21T21:41:01.683 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0109999999999999e-06 2024-09-21T21:41:01.683 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.683 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.683 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:01.683 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514355+0000", 2024-09-21T21:41:01.683 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1090000000000003e-06 2024-09-21T21:41:01.683 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.683 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.683 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:01.683 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514360+0000", 2024-09-21T21:41:01.683 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5900000000000001e-06 2024-09-21T21:41:01.683 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.683 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.684 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:01.684 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.527862+0000", 2024-09-21T21:41:01.684 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013501812 2024-09-21T21:41:01.684 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.684 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.684 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:01.684 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.527933+0000", 2024-09-21T21:41:01.684 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.1322000000000006e-05 2024-09-21T21:41:01.684 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.684 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.685 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:01.685 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.528372+0000", 2024-09-21T21:41:01.685 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00043938199999999999 2024-09-21T21:41:01.685 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.685 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:01.685 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.685 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.685 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.685 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4579 3.0 3:074cff57:::rbd_data.112145fae779.000000000000000c:a [read 1835008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:01.685 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.514365+0000", 2024-09-21T21:41:01.685 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.106978625, 2024-09-21T21:41:01.685 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.014158828, 2024-09-21T21:41:01.685 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:01.685 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:01.686 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:01.686 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:01.686 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:01.686 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:01.686 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4579 2024-09-21T21:41:01.686 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.686 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:01.686 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.686 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:01.686 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514365+0000", 2024-09-21T21:41:01.686 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:01.686 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.687 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.687 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:01.687 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514365+0000", 2024-09-21T21:41:01.687 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:01.687 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.687 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.687 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:01.687 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514366+0000", 2024-09-21T21:41:01.687 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7800000000000002e-07 2024-09-21T21:41:01.687 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.687 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.687 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:01.688 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514368+0000", 2024-09-21T21:41:01.688 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9920000000000002e-06 2024-09-21T21:41:01.688 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.688 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.688 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:01.688 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514374+0000", 2024-09-21T21:41:01.688 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9370000000000001e-06 2024-09-21T21:41:01.688 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.688 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.688 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:01.688 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514379+0000", 2024-09-21T21:41:01.688 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7149999999999997e-06 2024-09-21T21:41:01.688 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.689 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.689 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:01.689 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.528303+0000", 2024-09-21T21:41:01.689 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013924076000000001 2024-09-21T21:41:01.689 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.689 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.689 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:01.689 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.528429+0000", 2024-09-21T21:41:01.689 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00012643100000000001 2024-09-21T21:41:01.689 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.689 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.689 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:01.689 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.528524+0000", 2024-09-21T21:41:01.690 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.5298999999999997e-05 2024-09-21T21:41:01.690 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.690 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:01.690 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.690 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.690 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.690 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4580 3.0 3:074cff57:::rbd_data.112145fae779.000000000000000c:a [read 1892352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:01.690 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.514384+0000", 2024-09-21T21:41:01.690 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.1069600290000001, 2024-09-21T21:41:01.691 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.014365243, 2024-09-21T21:41:01.691 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:01.691 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:01.691 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:01.691 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:01.691 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:01.691 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:01.691 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4580 2024-09-21T21:41:01.691 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.691 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:01.691 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.691 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:01.691 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514384+0000", 2024-09-21T21:41:01.691 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:01.692 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.692 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.692 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:01.692 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514384+0000", 2024-09-21T21:41:01.692 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:01.692 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.692 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.692 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:01.692 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514384+0000", 2024-09-21T21:41:01.692 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1100000000000001e-07 2024-09-21T21:41:01.692 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.693 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.693 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:01.693 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514387+0000", 2024-09-21T21:41:01.693 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1150000000000001e-06 2024-09-21T21:41:01.693 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.693 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.693 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:01.693 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514393+0000", 2024-09-21T21:41:01.693 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1140000000000002e-06 2024-09-21T21:41:01.693 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.693 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.693 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:01.693 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514398+0000", 2024-09-21T21:41:01.694 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6330000000000001e-06 2024-09-21T21:41:01.694 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.694 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.694 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:01.694 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.528569+0000", 2024-09-21T21:41:01.694 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.014171011000000001 2024-09-21T21:41:01.694 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.694 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.694 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:01.694 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.528642+0000", 2024-09-21T21:41:01.694 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.2336000000000007e-05 2024-09-21T21:41:01.694 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.694 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.694 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:01.695 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.528749+0000", 2024-09-21T21:41:01.695 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000107623 2024-09-21T21:41:01.695 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.695 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:01.695 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.695 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.695 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.695 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4581 3.0 3:074cff57:::rbd_data.112145fae779.000000000000000c:a [read 1949696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:01.695 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.514404+0000", 2024-09-21T21:41:01.695 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.1069398619999999, 2024-09-21T21:41:01.695 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.014551563999999999, 2024-09-21T21:41:01.695 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:01.695 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:01.696 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:01.696 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:01.696 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:01.696 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:01.696 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4581 2024-09-21T21:41:01.696 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.696 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:01.696 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.696 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:01.696 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514404+0000", 2024-09-21T21:41:01.696 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:01.696 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.696 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.696 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:01.696 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514404+0000", 2024-09-21T21:41:01.697 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:01.697 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.697 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.697 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:01.697 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514405+0000", 2024-09-21T21:41:01.697 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7599999999999998e-07 2024-09-21T21:41:01.697 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.697 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.697 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:01.697 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514407+0000", 2024-09-21T21:41:01.697 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0169999999999999e-06 2024-09-21T21:41:01.697 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.697 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.697 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:01.697 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514413+0000", 2024-09-21T21:41:01.697 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0020000000000003e-06 2024-09-21T21:41:01.698 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.698 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.698 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:01.698 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514417+0000", 2024-09-21T21:41:01.698 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7319999999999998e-06 2024-09-21T21:41:01.698 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.698 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.698 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:01.698 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.528781+0000", 2024-09-21T21:41:01.698 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.014363433 2024-09-21T21:41:01.698 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.698 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.698 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:01.698 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.528861+0000", 2024-09-21T21:41:01.699 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0408000000000007e-05 2024-09-21T21:41:01.699 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.699 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.699 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:01.699 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.528956+0000", 2024-09-21T21:41:01.699 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.4596e-05 2024-09-21T21:41:01.699 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.699 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:01.699 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.699 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.699 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.699 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4582 3.0 3:074cff57:::rbd_data.112145fae779.000000000000000c:a [read 1982464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:01.700 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.514423+0000", 2024-09-21T21:41:01.700 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.1069209779999998, 2024-09-21T21:41:01.700 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.014719965, 2024-09-21T21:41:01.700 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:01.700 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:01.700 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:01.700 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:01.700 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:01.700 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:01.700 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4582 2024-09-21T21:41:01.700 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.701 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:01.701 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.701 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:01.701 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514423+0000", 2024-09-21T21:41:01.701 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:01.701 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.701 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.701 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:01.701 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514423+0000", 2024-09-21T21:41:01.701 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:01.701 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.701 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.701 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:01.701 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514423+0000", 2024-09-21T21:41:01.702 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.53e-07 2024-09-21T21:41:01.702 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.702 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.702 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:01.702 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514426+0000", 2024-09-21T21:41:01.702 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2000000000000001e-06 2024-09-21T21:41:01.702 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.702 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.702 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:01.702 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514433+0000", 2024-09-21T21:41:01.702 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.3170000000000004e-06 2024-09-21T21:41:01.702 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.702 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.702 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:01.703 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514438+0000", 2024-09-21T21:41:01.703 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3730000000000001e-06 2024-09-21T21:41:01.703 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.703 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.703 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:01.703 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.528984+0000", 2024-09-21T21:41:01.703 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.014545773 2024-09-21T21:41:01.703 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.703 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.703 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:01.703 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.529056+0000", 2024-09-21T21:41:01.703 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.2067e-05 2024-09-21T21:41:01.704 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.704 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.704 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:01.704 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.529143+0000", 2024-09-21T21:41:01.704 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.6881999999999999e-05 2024-09-21T21:41:01.704 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.704 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:01.704 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.704 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.704 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.704 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4583 3.0 3:074cff57:::rbd_data.112145fae779.000000000000000c:a [read 2039808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:01.704 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.514446+0000", 2024-09-21T21:41:01.704 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.1068982059999999, 2024-09-21T21:41:01.704 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.016072466000000001, 2024-09-21T21:41:01.704 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:01.704 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:01.705 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:01.705 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:01.705 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:01.705 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:01.705 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4583 2024-09-21T21:41:01.705 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.705 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:01.705 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.705 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:01.705 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514446+0000", 2024-09-21T21:41:01.705 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:01.705 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.705 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.705 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:01.706 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514446+0000", 2024-09-21T21:41:01.706 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:01.706 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.706 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.706 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:01.706 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514446+0000", 2024-09-21T21:41:01.706 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.9700000000000002e-07 2024-09-21T21:41:01.706 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.706 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.706 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:01.706 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514448+0000", 2024-09-21T21:41:01.706 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1849999999999998e-06 2024-09-21T21:41:01.706 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.706 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.707 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:01.707 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514454+0000", 2024-09-21T21:41:01.707 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.015e-06 2024-09-21T21:41:01.707 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.707 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.707 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:01.707 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514459+0000", 2024-09-21T21:41:01.707 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7639999999999996e-06 2024-09-21T21:41:01.707 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.707 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.707 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:01.707 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.529173+0000", 2024-09-21T21:41:01.707 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.014713688000000001 2024-09-21T21:41:01.707 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.708 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.708 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:01.708 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.529248+0000", 2024-09-21T21:41:01.708 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5073000000000004e-05 2024-09-21T21:41:01.708 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.708 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.708 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:01.708 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.530518+0000", 2024-09-21T21:41:01.708 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0012703440000000001 2024-09-21T21:41:01.708 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.708 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:01.708 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.708 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.708 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.708 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4584 3.0 3:074cff57:::rbd_data.112145fae779.000000000000000c:a [read 2129920~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:01.709 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.514465+0000", 2024-09-21T21:41:01.709 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.106878767, 2024-09-21T21:41:01.709 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.016252026999999999, 2024-09-21T21:41:01.709 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:01.709 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:01.709 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:01.709 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:01.709 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:01.709 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:01.709 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4584 2024-09-21T21:41:01.709 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.709 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:01.709 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.709 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:01.709 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514465+0000", 2024-09-21T21:41:01.710 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:01.710 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.710 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.710 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:01.710 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514465+0000", 2024-09-21T21:41:01.710 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:01.710 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.710 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.710 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:01.710 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514466+0000", 2024-09-21T21:41:01.710 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.0400000000000002e-07 2024-09-21T21:41:01.710 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.711 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.711 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:01.711 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514468+0000", 2024-09-21T21:41:01.711 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3159999999999999e-06 2024-09-21T21:41:01.711 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.711 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.711 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:01.711 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514474+0000", 2024-09-21T21:41:01.711 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.224e-06 2024-09-21T21:41:01.711 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.711 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.712 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:01.712 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514479+0000", 2024-09-21T21:41:01.712 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8099999999999997e-06 2024-09-21T21:41:01.712 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.712 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.712 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:01.712 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.530552+0000", 2024-09-21T21:41:01.712 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.016072622000000002 2024-09-21T21:41:01.712 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.712 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.712 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:01.712 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.530630+0000", 2024-09-21T21:41:01.712 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.8448000000000005e-05 2024-09-21T21:41:01.712 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.713 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.713 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:01.713 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.530717+0000", 2024-09-21T21:41:01.713 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.7202999999999998e-05 2024-09-21T21:41:01.713 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.713 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:01.713 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.713 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.713 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.713 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4585 3.0 3:074cff57:::rbd_data.112145fae779.000000000000000c:a [read 2187264~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:01.713 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.514485+0000", 2024-09-21T21:41:01.713 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.1068590999999999, 2024-09-21T21:41:01.714 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.016420605000000001, 2024-09-21T21:41:01.714 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:01.714 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:01.714 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:01.714 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:01.714 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:01.714 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:01.714 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4585 2024-09-21T21:41:01.714 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.714 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:01.714 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.714 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:01.715 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514485+0000", 2024-09-21T21:41:01.715 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:01.715 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.715 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.715 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:01.715 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514485+0000", 2024-09-21T21:41:01.715 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:01.715 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.715 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.715 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:01.715 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514485+0000", 2024-09-21T21:41:01.715 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3599999999999999e-07 2024-09-21T21:41:01.715 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.715 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.715 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:01.716 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514487+0000", 2024-09-21T21:41:01.716 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0870000000000001e-06 2024-09-21T21:41:01.716 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.716 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.716 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:01.716 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514495+0000", 2024-09-21T21:41:01.716 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7589999999999997e-06 2024-09-21T21:41:01.716 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.716 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.716 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:01.716 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514501+0000", 2024-09-21T21:41:01.716 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4e-06 2024-09-21T21:41:01.716 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.716 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.716 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:01.716 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.530747+0000", 2024-09-21T21:41:01.717 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.016246564000000002 2024-09-21T21:41:01.717 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.717 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.717 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:01.717 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.530824+0000", 2024-09-21T21:41:01.717 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7109999999999999e-05 2024-09-21T21:41:01.717 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.717 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.717 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:01.717 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.530906+0000", 2024-09-21T21:41:01.717 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.1348999999999997e-05 2024-09-21T21:41:01.717 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.717 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:01.717 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.717 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.718 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.718 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4711 3.0 3:074cff57:::rbd_data.112145fae779.000000000000000c:a [read 2043904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:01.718 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.519604+0000", 2024-09-21T21:41:01.718 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.1017396110000002, 2024-09-21T21:41:01.718 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013270252999999999, 2024-09-21T21:41:01.718 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:01.718 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:01.718 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:01.718 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:01.718 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:01.718 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:01.718 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4711 2024-09-21T21:41:01.718 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.718 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:01.719 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.719 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:01.719 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519604+0000", 2024-09-21T21:41:01.719 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:01.719 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.719 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.719 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:01.719 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519604+0000", 2024-09-21T21:41:01.719 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:01.719 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.719 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.719 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:01.719 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519605+0000", 2024-09-21T21:41:01.719 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8500000000000002e-07 2024-09-21T21:41:01.720 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.720 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.720 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:01.720 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519607+0000", 2024-09-21T21:41:01.720 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1789999999999998e-06 2024-09-21T21:41:01.720 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.720 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.720 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:01.720 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519613+0000", 2024-09-21T21:41:01.720 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2609999999999999e-06 2024-09-21T21:41:01.720 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.720 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.720 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:01.720 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519618+0000", 2024-09-21T21:41:01.720 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9579999999999998e-06 2024-09-21T21:41:01.720 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.721 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.721 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:01.721 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.532678+0000", 2024-09-21T21:41:01.721 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013059822 2024-09-21T21:41:01.721 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.721 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.721 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:01.721 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.532756+0000", 2024-09-21T21:41:01.721 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7467999999999998e-05 2024-09-21T21:41:01.721 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.721 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.722 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:01.722 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.532875+0000", 2024-09-21T21:41:01.722 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011917999999999999 2024-09-21T21:41:01.722 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.722 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:01.722 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.722 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.722 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.722 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4712 3.0 3:074cff57:::rbd_data.112145fae779.000000000000000c:a [read 2134016~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:01.722 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.519632+0000", 2024-09-21T21:41:01.722 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.101712128, 2024-09-21T21:41:01.722 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013480002, 2024-09-21T21:41:01.722 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:01.723 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:01.723 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:01.723 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:01.723 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:01.723 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:01.723 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4712 2024-09-21T21:41:01.723 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.723 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:01.723 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.723 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:01.723 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519632+0000", 2024-09-21T21:41:01.723 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:01.723 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.723 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.723 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:01.723 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519632+0000", 2024-09-21T21:41:01.724 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:01.724 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.724 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.724 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:01.724 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519632+0000", 2024-09-21T21:41:01.724 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2899999999999999e-07 2024-09-21T21:41:01.724 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.724 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.724 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:01.724 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519635+0000", 2024-09-21T21:41:01.724 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.6910000000000002e-06 2024-09-21T21:41:01.725 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.725 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.725 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:01.725 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519761+0000", 2024-09-21T21:41:01.725 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000125988 2024-09-21T21:41:01.725 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.725 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.725 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:01.725 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519767+0000", 2024-09-21T21:41:01.725 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9190000000000004e-06 2024-09-21T21:41:01.725 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.725 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.725 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:01.725 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.532905+0000", 2024-09-21T21:41:01.725 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013137970000000001 2024-09-21T21:41:01.725 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.725 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.726 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:01.726 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.532984+0000", 2024-09-21T21:41:01.726 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.8616999999999996e-05 2024-09-21T21:41:01.726 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.726 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.726 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:01.726 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.533112+0000", 2024-09-21T21:41:01.726 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000128388 2024-09-21T21:41:01.726 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.726 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:01.726 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.726 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.726 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.726 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4713 3.0 3:074cff57:::rbd_data.112145fae779.000000000000000c:a [read 2191360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:01.727 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.519777+0000", 2024-09-21T21:41:01.727 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.1015674280000001, 2024-09-21T21:41:01.727 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.01355251, 2024-09-21T21:41:01.727 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:01.727 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:01.727 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:01.727 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:01.727 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:01.727 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:01.727 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4713 2024-09-21T21:41:01.727 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.727 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:01.727 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.728 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:01.728 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519777+0000", 2024-09-21T21:41:01.728 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:01.728 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.728 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.728 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:01.728 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519777+0000", 2024-09-21T21:41:01.728 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:01.728 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.728 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.728 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:01.728 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519777+0000", 2024-09-21T21:41:01.728 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9900000000000001e-07 2024-09-21T21:41:01.728 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.729 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.729 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:01.729 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519780+0000", 2024-09-21T21:41:01.729 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.886e-06 2024-09-21T21:41:01.729 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.729 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.729 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:01.729 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519786+0000", 2024-09-21T21:41:01.729 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0859999999999998e-06 2024-09-21T21:41:01.729 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.729 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.729 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:01.729 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519790+0000", 2024-09-21T21:41:01.729 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2400000000000001e-06 2024-09-21T21:41:01.729 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.730 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.730 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:01.730 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.533143+0000", 2024-09-21T21:41:01.730 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013352711 2024-09-21T21:41:01.730 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.730 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.730 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:01.730 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.533224+0000", 2024-09-21T21:41:01.730 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0675999999999998e-05 2024-09-21T21:41:01.730 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:01.730 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:01.730 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:01.730 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.533329+0000", 2024-09-21T21:41:01.731 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000105412 2024-09-21T21:41:01.731 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.731 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:01.731 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.731 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:01.731 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:01.731 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:01.732 INFO:teuthology.orchestra.run.smithi164.stdout:truncating image image_client.0-clone9 from 0xd443200 (overlap 0x40fe200) to 0x0 2024-09-21T21:41:01.733 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T21:41:01.733 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:01.731+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:01.833 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:01.831+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:01.878 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:01.878 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:41:01.878 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 0 2024-09-21T21:41:01.878 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:01.887 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T21:41:01.934 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:01.932+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:01.992 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:01.992 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:41:01.993 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:41:01.993 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:41:01.993 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:02.001 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T21:41:02.034 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:02.033+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:02.066 INFO:teuthology.orchestra.run.smithi164.stdout:checking clone #8, image image_client.0-clone8 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent9 2024-09-21T21:41:02.136 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:02.134+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:02.172 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:02.172 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 20, 2024-09-21T21:41:02.172 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 600, 2024-09-21T21:41:02.172 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:41:02.172 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.172 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:1704 3.8 3:1339e2f0:::rbd_data.112145fae779.0000000000000007:a [read 10653696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:02.172 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.382652+0000", 2024-09-21T21:41:02.172 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.7864259009999999, 2024-09-21T21:41:02.172 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.012841594, 2024-09-21T21:41:02.172 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:02.173 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:02.173 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:02.173 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:02.173 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:02.173 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:02.173 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1704 2024-09-21T21:41:02.173 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.173 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:02.173 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.173 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:02.173 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382652+0000", 2024-09-21T21:41:02.173 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:02.173 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.174 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.174 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:02.174 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382652+0000", 2024-09-21T21:41:02.174 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:02.174 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.174 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.174 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:02.174 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382653+0000", 2024-09-21T21:41:02.174 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1600000000000002e-07 2024-09-21T21:41:02.174 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.174 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.174 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:02.174 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382655+0000", 2024-09-21T21:41:02.174 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1449999999999998e-06 2024-09-21T21:41:02.175 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.175 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.175 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:02.175 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382661+0000", 2024-09-21T21:41:02.175 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3779999999999998e-06 2024-09-21T21:41:02.175 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.175 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.175 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:02.175 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382667+0000", 2024-09-21T21:41:02.175 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1329999999999998e-06 2024-09-21T21:41:02.175 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.175 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.175 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:02.176 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.395334+0000", 2024-09-21T21:41:02.176 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.012667928 2024-09-21T21:41:02.176 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.176 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.176 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:02.176 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.395418+0000", 2024-09-21T21:41:02.176 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.4046000000000001e-05 2024-09-21T21:41:02.176 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.176 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.176 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:02.176 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.395494+0000", 2024-09-21T21:41:02.176 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5548000000000002e-05 2024-09-21T21:41:02.176 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:02.176 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:02.176 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:02.177 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.177 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.177 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:1706 3.8 3:1339e2f0:::rbd_data.112145fae779.0000000000000007:a [read 10711040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:02.177 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.382673+0000", 2024-09-21T21:41:02.177 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.7864057190000002, 2024-09-21T21:41:02.177 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013021824, 2024-09-21T21:41:02.177 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:02.177 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:02.177 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:02.177 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:02.177 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:02.177 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:02.177 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1706 2024-09-21T21:41:02.177 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.177 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:02.178 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.178 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:02.178 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382673+0000", 2024-09-21T21:41:02.178 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:02.178 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.178 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.178 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:02.178 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382673+0000", 2024-09-21T21:41:02.178 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:02.178 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.178 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.178 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:02.178 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382673+0000", 2024-09-21T21:41:02.179 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.01e-07 2024-09-21T21:41:02.179 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.179 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.179 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:02.179 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382675+0000", 2024-09-21T21:41:02.179 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.8959999999999999e-06 2024-09-21T21:41:02.179 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.179 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.179 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:02.179 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382681+0000", 2024-09-21T21:41:02.179 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4309999999999999e-06 2024-09-21T21:41:02.179 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.179 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.179 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:02.180 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382687+0000", 2024-09-21T21:41:02.180 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2229999999999998e-06 2024-09-21T21:41:02.180 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.180 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.180 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:02.180 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.395537+0000", 2024-09-21T21:41:02.180 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.012850225 2024-09-21T21:41:02.180 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.180 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.180 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:02.180 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.395622+0000", 2024-09-21T21:41:02.180 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.4856000000000002e-05 2024-09-21T21:41:02.180 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.180 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.180 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:02.180 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.395694+0000", 2024-09-21T21:41:02.181 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.2792000000000004e-05 2024-09-21T21:41:02.181 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:02.181 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:02.181 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:02.181 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.181 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.181 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:1886 3.8 3:1339e2f0:::rbd_data.112145fae779.0000000000000007:a [read 10424320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:02.181 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.383956+0000", 2024-09-21T21:41:02.181 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.7851224320000001, 2024-09-21T21:41:02.181 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.012878921, 2024-09-21T21:41:02.181 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:02.181 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:02.181 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:02.181 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:02.181 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:02.182 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:02.182 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1886 2024-09-21T21:41:02.182 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.182 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:02.182 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.182 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:02.182 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383956+0000", 2024-09-21T21:41:02.182 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:02.182 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.182 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.182 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:02.182 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383956+0000", 2024-09-21T21:41:02.182 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:02.182 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.183 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.183 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:02.183 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383956+0000", 2024-09-21T21:41:02.183 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.27e-07 2024-09-21T21:41:02.183 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.183 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.183 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:02.183 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383959+0000", 2024-09-21T21:41:02.183 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.5170000000000002e-06 2024-09-21T21:41:02.183 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.183 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.183 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:02.183 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383965+0000", 2024-09-21T21:41:02.183 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3149999999999997e-06 2024-09-21T21:41:02.183 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.183 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.184 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:02.184 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383971+0000", 2024-09-21T21:41:02.184 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3900000000000001e-06 2024-09-21T21:41:02.184 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.184 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.184 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:02.184 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.396727+0000", 2024-09-21T21:41:02.184 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.012756226000000001 2024-09-21T21:41:02.184 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.184 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.184 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:02.184 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.396777+0000", 2024-09-21T21:41:02.184 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0419e-05 2024-09-21T21:41:02.184 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.184 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.184 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:02.184 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.396835+0000", 2024-09-21T21:41:02.185 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7627000000000002e-05 2024-09-21T21:41:02.185 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:02.185 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:02.185 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:02.185 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.185 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.185 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:1888 3.8 3:1339e2f0:::rbd_data.112145fae779.0000000000000007:a [read 10481664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:02.185 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.383992+0000", 2024-09-21T21:41:02.185 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.7850863119999998, 2024-09-21T21:41:02.185 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013052779, 2024-09-21T21:41:02.185 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:02.185 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:02.185 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:02.185 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:02.185 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:02.185 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:02.186 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1888 2024-09-21T21:41:02.186 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.186 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:02.186 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.186 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:02.186 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383992+0000", 2024-09-21T21:41:02.186 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:02.186 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.186 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.186 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:02.186 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383992+0000", 2024-09-21T21:41:02.186 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:02.186 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.186 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.187 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:02.187 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383992+0000", 2024-09-21T21:41:02.187 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4200000000000002e-07 2024-09-21T21:41:02.187 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.187 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.187 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:02.187 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383994+0000", 2024-09-21T21:41:02.187 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9e-06 2024-09-21T21:41:02.187 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.187 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.187 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:02.187 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.384001+0000", 2024-09-21T21:41:02.187 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9439999999999999e-06 2024-09-21T21:41:02.187 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.187 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.188 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:02.188 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.384005+0000", 2024-09-21T21:41:02.188 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4620000000000001e-06 2024-09-21T21:41:02.188 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.188 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.188 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:02.188 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.396876+0000", 2024-09-21T21:41:02.188 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.012871825999999999 2024-09-21T21:41:02.188 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.188 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.188 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:02.188 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.396971+0000", 2024-09-21T21:41:02.188 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.4378999999999999e-05 2024-09-21T21:41:02.188 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.188 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.189 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:02.189 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.397045+0000", 2024-09-21T21:41:02.189 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.3925999999999997e-05 2024-09-21T21:41:02.189 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:02.189 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:02.189 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:02.189 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.189 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.189 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:1890 3.8 3:1339e2f0:::rbd_data.112145fae779.0000000000000007:a [read 10514432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:02.189 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.384011+0000", 2024-09-21T21:41:02.189 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.7850678000000002, 2024-09-21T21:41:02.189 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013260717, 2024-09-21T21:41:02.189 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:02.189 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:02.189 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:02.190 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:02.190 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:02.190 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:02.190 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1890 2024-09-21T21:41:02.190 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.190 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:02.190 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.190 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:02.190 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.384011+0000", 2024-09-21T21:41:02.190 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:02.190 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.190 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.190 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:02.190 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.384011+0000", 2024-09-21T21:41:02.190 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:02.191 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.191 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.191 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:02.191 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.384011+0000", 2024-09-21T21:41:02.191 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.7700000000000001e-07 2024-09-21T21:41:02.191 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.191 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.191 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:02.191 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.384012+0000", 2024-09-21T21:41:02.191 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.3740000000000001e-06 2024-09-21T21:41:02.191 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.191 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.191 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:02.192 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.384015+0000", 2024-09-21T21:41:02.192 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3129999999999999e-06 2024-09-21T21:41:02.192 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.192 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.192 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:02.192 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.384018+0000", 2024-09-21T21:41:02.192 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.6620000000000001e-06 2024-09-21T21:41:02.192 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.192 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.192 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:02.192 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.397089+0000", 2024-09-21T21:41:02.192 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013070724000000001 2024-09-21T21:41:02.192 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.192 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.193 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:02.193 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.397191+0000", 2024-09-21T21:41:02.193 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010187399999999999 2024-09-21T21:41:02.193 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.193 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.193 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:02.193 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.397271+0000", 2024-09-21T21:41:02.193 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0493000000000003e-05 2024-09-21T21:41:02.193 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:02.193 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:02.193 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:02.193 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.193 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.193 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:1892 3.8 3:1339e2f0:::rbd_data.112145fae779.0000000000000007:a [read 10571776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:02.194 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.384023+0000", 2024-09-21T21:41:02.194 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.7850555620000002, 2024-09-21T21:41:02.194 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013380702, 2024-09-21T21:41:02.194 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:02.194 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:02.194 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:02.194 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:02.194 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:02.194 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:02.194 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1892 2024-09-21T21:41:02.194 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.194 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:02.194 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.195 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:02.195 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.384023+0000", 2024-09-21T21:41:02.195 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:02.195 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.195 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.195 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:02.195 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.384023+0000", 2024-09-21T21:41:02.195 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:02.195 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.195 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.195 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:02.195 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.384023+0000", 2024-09-21T21:41:02.195 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.1800000000000002e-07 2024-09-21T21:41:02.195 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.196 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.196 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:02.196 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.384024+0000", 2024-09-21T21:41:02.196 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.2300000000000001e-06 2024-09-21T21:41:02.196 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.196 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.196 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:02.196 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.384028+0000", 2024-09-21T21:41:02.196 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.2990000000000001e-06 2024-09-21T21:41:02.196 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.196 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.196 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:02.196 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.384030+0000", 2024-09-21T21:41:02.196 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.5000000000000002e-06 2024-09-21T21:41:02.197 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.197 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.197 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:02.197 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.397311+0000", 2024-09-21T21:41:02.197 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013280760000000001 2024-09-21T21:41:02.197 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.197 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.197 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:02.197 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.397359+0000", 2024-09-21T21:41:02.197 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7707999999999998e-05 2024-09-21T21:41:02.197 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.197 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.197 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:02.197 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.397403+0000", 2024-09-21T21:41:02.198 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4886999999999999e-05 2024-09-21T21:41:02.198 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:02.198 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:02.198 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:02.198 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.198 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.198 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:1894 3.8 3:1339e2f0:::rbd_data.112145fae779.0000000000000007:a [read 10661888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:02.198 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.384157+0000", 2024-09-21T21:41:02.198 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.7849213079999999, 2024-09-21T21:41:02.198 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013377534999999999, 2024-09-21T21:41:02.198 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:02.198 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:02.199 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:02.199 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:02.199 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:02.199 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:02.199 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1894 2024-09-21T21:41:02.199 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.199 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:02.199 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.199 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:02.199 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.384157+0000", 2024-09-21T21:41:02.199 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:02.199 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.199 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.200 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:02.200 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.384157+0000", 2024-09-21T21:41:02.200 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:02.200 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.200 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.200 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:02.200 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.384158+0000", 2024-09-21T21:41:02.200 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2099999999999995e-07 2024-09-21T21:41:02.200 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.200 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.200 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:02.200 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.384163+0000", 2024-09-21T21:41:02.200 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8350000000000003e-06 2024-09-21T21:41:02.200 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.201 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.201 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:02.201 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.384176+0000", 2024-09-21T21:41:02.201 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.3286e-05 2024-09-21T21:41:02.201 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.201 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.201 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:02.201 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.384183+0000", 2024-09-21T21:41:02.201 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.357e-06 2024-09-21T21:41:02.201 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.201 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.201 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:02.201 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.397431+0000", 2024-09-21T21:41:02.201 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013247649 2024-09-21T21:41:02.201 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.201 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.202 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:02.202 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.397486+0000", 2024-09-21T21:41:02.202 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4747000000000003e-05 2024-09-21T21:41:02.202 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.202 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.202 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:02.202 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.397535+0000", 2024-09-21T21:41:02.202 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8739999999999998e-05 2024-09-21T21:41:02.202 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:02.202 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:02.202 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:02.202 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.202 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.203 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:1897 3.8 3:1339e2f0:::rbd_data.112145fae779.0000000000000007:a [read 10719232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:02.203 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.384298+0000", 2024-09-21T21:41:02.203 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.784780037, 2024-09-21T21:41:02.203 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013351722, 2024-09-21T21:41:02.203 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:02.203 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:02.203 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:02.203 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:02.203 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:02.203 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:02.203 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1897 2024-09-21T21:41:02.203 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.203 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:02.203 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.204 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:02.204 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.384298+0000", 2024-09-21T21:41:02.204 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:02.204 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.204 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.204 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:02.204 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.384298+0000", 2024-09-21T21:41:02.204 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:02.204 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.204 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.204 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:02.204 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.384299+0000", 2024-09-21T21:41:02.204 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.9999999999999996e-07 2024-09-21T21:41:02.204 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.204 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.204 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:02.205 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.384303+0000", 2024-09-21T21:41:02.205 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2270000000000004e-06 2024-09-21T21:41:02.205 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.205 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.205 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:02.205 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.384313+0000", 2024-09-21T21:41:02.205 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.5489999999999995e-06 2024-09-21T21:41:02.205 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.205 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.205 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:02.205 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.384319+0000", 2024-09-21T21:41:02.205 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4930000000000003e-06 2024-09-21T21:41:02.205 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.205 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.205 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:02.205 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.397561+0000", 2024-09-21T21:41:02.205 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013241493999999999 2024-09-21T21:41:02.206 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.206 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.206 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:02.206 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.397606+0000", 2024-09-21T21:41:02.206 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5473000000000003e-05 2024-09-21T21:41:02.206 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.206 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.206 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:02.206 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.397650+0000", 2024-09-21T21:41:02.206 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3686000000000002e-05 2024-09-21T21:41:02.206 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:02.206 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:02.206 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:02.206 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.207 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.207 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:6258 3.8 3:1339e2f0:::rbd_data.112145fae779.0000000000000007:a [read 12398592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:41:02.207 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.593604+0000", 2024-09-21T21:41:02.207 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.5754742999999998, 2024-09-21T21:41:02.207 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.012967548000000001, 2024-09-21T21:41:02.207 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:02.207 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:02.207 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:02.207 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:02.207 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:02.207 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:02.207 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 6258 2024-09-21T21:41:02.207 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.207 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:02.207 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.208 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:02.208 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.593604+0000", 2024-09-21T21:41:02.208 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:02.208 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.208 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.208 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:02.208 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.593604+0000", 2024-09-21T21:41:02.208 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:02.208 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.208 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.208 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:02.208 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.593604+0000", 2024-09-21T21:41:02.208 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.9200000000000002e-07 2024-09-21T21:41:02.208 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.209 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.209 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:02.209 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.593611+0000", 2024-09-21T21:41:02.209 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2249999999999997e-06 2024-09-21T21:41:02.209 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.209 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.209 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:02.209 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.593617+0000", 2024-09-21T21:41:02.209 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2129999999999996e-06 2024-09-21T21:41:02.209 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.209 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.209 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:02.209 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.593622+0000", 2024-09-21T21:41:02.209 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7430000000000002e-06 2024-09-21T21:41:02.210 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.210 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.210 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:02.210 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.606399+0000", 2024-09-21T21:41:02.210 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.012776984999999999 2024-09-21T21:41:02.210 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.210 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.210 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:02.210 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.606488+0000", 2024-09-21T21:41:02.210 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.8918000000000005e-05 2024-09-21T21:41:02.210 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.210 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.210 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:02.210 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.606572+0000", 2024-09-21T21:41:02.210 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.4072000000000004e-05 2024-09-21T21:41:02.211 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:02.211 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:02.211 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:02.211 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.211 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.211 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:6378 3.8 3:1339e2f0:::rbd_data.112145fae779.0000000000000007:a [read 11960320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:41:02.211 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.594157+0000", 2024-09-21T21:41:02.211 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.574921158, 2024-09-21T21:41:02.211 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.012907642, 2024-09-21T21:41:02.211 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:02.211 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:02.211 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:02.211 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:02.211 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:02.211 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:02.212 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 6378 2024-09-21T21:41:02.212 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.212 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:02.212 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.212 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:02.212 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594157+0000", 2024-09-21T21:41:02.212 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:02.212 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.212 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.212 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:02.212 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594157+0000", 2024-09-21T21:41:02.212 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:02.212 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.212 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.212 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:02.212 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594159+0000", 2024-09-21T21:41:02.212 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.939e-06 2024-09-21T21:41:02.213 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.213 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.213 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:02.213 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594162+0000", 2024-09-21T21:41:02.213 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.8849999999999999e-06 2024-09-21T21:41:02.213 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.213 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.213 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:02.213 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594169+0000", 2024-09-21T21:41:02.213 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9190000000000002e-06 2024-09-21T21:41:02.213 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.213 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.213 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:02.213 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594174+0000", 2024-09-21T21:41:02.213 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.22e-06 2024-09-21T21:41:02.214 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.214 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.214 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:02.214 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.606959+0000", 2024-09-21T21:41:02.214 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.012784644 2024-09-21T21:41:02.214 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.214 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.214 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:02.214 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.607003+0000", 2024-09-21T21:41:02.214 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4067999999999999e-05 2024-09-21T21:41:02.214 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.214 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.214 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:02.214 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.607065+0000", 2024-09-21T21:41:02.214 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1966999999999998e-05 2024-09-21T21:41:02.214 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:02.215 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:02.215 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:02.215 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.215 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.215 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:6379 3.8 3:1339e2f0:::rbd_data.112145fae779.0000000000000007:a [read 12017664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:41:02.215 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.594186+0000", 2024-09-21T21:41:02.215 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.5748927589999999, 2024-09-21T21:41:02.215 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013082819000000001, 2024-09-21T21:41:02.215 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:02.215 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:02.215 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:02.215 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:02.215 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:02.215 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:02.215 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 6379 2024-09-21T21:41:02.215 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.216 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:02.216 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.216 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:02.216 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594186+0000", 2024-09-21T21:41:02.216 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:02.216 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.216 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.216 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:02.216 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594186+0000", 2024-09-21T21:41:02.216 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:02.216 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.216 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.216 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:02.216 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594186+0000", 2024-09-21T21:41:02.216 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.39e-07 2024-09-21T21:41:02.217 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.217 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.217 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:02.217 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594188+0000", 2024-09-21T21:41:02.217 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3159999999999999e-06 2024-09-21T21:41:02.217 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.217 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.217 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:02.217 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594194+0000", 2024-09-21T21:41:02.217 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.088e-06 2024-09-21T21:41:02.217 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.217 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.217 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:02.217 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594199+0000", 2024-09-21T21:41:02.217 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8729999999999998e-06 2024-09-21T21:41:02.217 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.218 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.218 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:02.218 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.607102+0000", 2024-09-21T21:41:02.218 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.012902521 2024-09-21T21:41:02.218 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.218 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.218 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:02.218 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.607185+0000", 2024-09-21T21:41:02.218 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.2779000000000001e-05 2024-09-21T21:41:02.218 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.218 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.218 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:02.218 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.607268+0000", 2024-09-21T21:41:02.219 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.3802999999999997e-05 2024-09-21T21:41:02.219 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:02.219 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:02.219 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:02.219 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.219 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.219 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:6380 3.8 3:1339e2f0:::rbd_data.112145fae779.0000000000000007:a [read 12107776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:41:02.219 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.594206+0000", 2024-09-21T21:41:02.219 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.5748724969999999, 2024-09-21T21:41:02.219 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.01319447, 2024-09-21T21:41:02.219 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:02.219 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:02.219 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:02.219 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:02.219 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:02.220 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:02.220 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 6380 2024-09-21T21:41:02.220 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.220 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:02.220 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.220 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:02.220 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594206+0000", 2024-09-21T21:41:02.220 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:02.220 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.220 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.220 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:02.220 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594206+0000", 2024-09-21T21:41:02.220 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:02.220 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.220 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.221 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:02.221 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594206+0000", 2024-09-21T21:41:02.221 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8099999999999998e-07 2024-09-21T21:41:02.221 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.221 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.221 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:02.221 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594208+0000", 2024-09-21T21:41:02.221 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9479999999999998e-06 2024-09-21T21:41:02.221 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.221 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.221 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:02.221 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594214+0000", 2024-09-21T21:41:02.221 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7409999999999998e-06 2024-09-21T21:41:02.222 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.222 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.222 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:02.222 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594218+0000", 2024-09-21T21:41:02.222 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3830000000000002e-06 2024-09-21T21:41:02.222 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.222 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.222 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:02.222 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.607298+0000", 2024-09-21T21:41:02.222 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013079856000000001 2024-09-21T21:41:02.222 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.222 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.222 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:02.222 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.607341+0000", 2024-09-21T21:41:02.223 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3118999999999998e-05 2024-09-21T21:41:02.223 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.223 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.223 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:02.223 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.607400+0000", 2024-09-21T21:41:02.223 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9042000000000002e-05 2024-09-21T21:41:02.223 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:02.223 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:02.223 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:02.223 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.223 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.223 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:6381 3.8 3:1339e2f0:::rbd_data.112145fae779.0000000000000007:a [read 12165120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:41:02.223 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.594224+0000", 2024-09-21T21:41:02.223 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.5748539250000002, 2024-09-21T21:41:02.223 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013387049999999999, 2024-09-21T21:41:02.224 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:02.224 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:02.224 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:02.224 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:02.224 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:02.224 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:02.224 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 6381 2024-09-21T21:41:02.224 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.224 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:02.224 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.224 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:02.224 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594224+0000", 2024-09-21T21:41:02.224 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:02.224 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.225 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.225 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:02.225 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594224+0000", 2024-09-21T21:41:02.225 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:02.225 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.225 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.225 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:02.225 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594225+0000", 2024-09-21T21:41:02.225 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1699999999999999e-07 2024-09-21T21:41:02.225 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.225 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.225 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:02.225 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594227+0000", 2024-09-21T21:41:02.225 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1710000000000001e-06 2024-09-21T21:41:02.225 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.226 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.226 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:02.226 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594233+0000", 2024-09-21T21:41:02.226 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.631e-06 2024-09-21T21:41:02.226 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.226 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.226 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:02.226 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594237+0000", 2024-09-21T21:41:02.226 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4989999999999996e-06 2024-09-21T21:41:02.226 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.226 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.226 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:02.226 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.607438+0000", 2024-09-21T21:41:02.227 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013200788999999999 2024-09-21T21:41:02.227 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.227 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.227 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:02.227 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.607526+0000", 2024-09-21T21:41:02.227 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.7975999999999999e-05 2024-09-21T21:41:02.227 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.227 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.227 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:02.227 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.607611+0000", 2024-09-21T21:41:02.227 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.5567000000000003e-05 2024-09-21T21:41:02.227 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:02.227 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:02.227 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:02.227 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.228 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.228 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:6382 3.8 3:1339e2f0:::rbd_data.112145fae779.0000000000000007:a [read 12197888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:41:02.228 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.594251+0000", 2024-09-21T21:41:02.228 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.5748276849999998, 2024-09-21T21:41:02.228 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.01350056, 2024-09-21T21:41:02.228 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:02.228 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:02.228 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:02.228 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:02.228 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:02.228 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:02.228 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 6382 2024-09-21T21:41:02.228 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.229 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:02.229 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.229 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:02.229 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594251+0000", 2024-09-21T21:41:02.229 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:02.229 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.229 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.229 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:02.229 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594251+0000", 2024-09-21T21:41:02.229 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:02.229 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.229 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.229 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:02.229 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594251+0000", 2024-09-21T21:41:02.230 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4000000000000002e-07 2024-09-21T21:41:02.230 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.230 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.230 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:02.230 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594253+0000", 2024-09-21T21:41:02.230 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2500000000000001e-06 2024-09-21T21:41:02.230 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.230 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.230 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:02.230 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594259+0000", 2024-09-21T21:41:02.230 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7559999999999996e-06 2024-09-21T21:41:02.230 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.230 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.230 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:02.231 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594264+0000", 2024-09-21T21:41:02.231 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5719999999999996e-06 2024-09-21T21:41:02.231 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.231 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.231 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:02.231 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.607642+0000", 2024-09-21T21:41:02.231 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013378750999999999 2024-09-21T21:41:02.231 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.231 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.231 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:02.231 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.607690+0000", 2024-09-21T21:41:02.231 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7632000000000001e-05 2024-09-21T21:41:02.231 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.231 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.232 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:02.232 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.607751+0000", 2024-09-21T21:41:02.232 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1159000000000002e-05 2024-09-21T21:41:02.232 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:02.232 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:02.232 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:02.232 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.232 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.232 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:6383 3.8 3:1339e2f0:::rbd_data.112145fae779.0000000000000007:a [read 12255232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:41:02.232 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.594270+0000", 2024-09-21T21:41:02.232 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.5748083180000001, 2024-09-21T21:41:02.232 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013702288999999999, 2024-09-21T21:41:02.232 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:02.232 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:02.232 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:02.233 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:02.233 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:02.233 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:02.233 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 6383 2024-09-21T21:41:02.233 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.233 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:02.233 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.233 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:02.233 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594270+0000", 2024-09-21T21:41:02.233 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:02.233 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.233 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.233 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:02.233 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594270+0000", 2024-09-21T21:41:02.233 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:02.234 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.234 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.234 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:02.234 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594270+0000", 2024-09-21T21:41:02.234 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8799999999999998e-07 2024-09-21T21:41:02.234 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.234 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.234 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:02.234 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594272+0000", 2024-09-21T21:41:02.234 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.995e-06 2024-09-21T21:41:02.234 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.234 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.234 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:02.234 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594278+0000", 2024-09-21T21:41:02.235 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5459999999999999e-06 2024-09-21T21:41:02.235 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.235 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.235 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:02.235 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594283+0000", 2024-09-21T21:41:02.235 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3739999999999997e-06 2024-09-21T21:41:02.235 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.235 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.235 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:02.235 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.607790+0000", 2024-09-21T21:41:02.235 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013506827000000001 2024-09-21T21:41:02.235 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.235 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.235 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:02.235 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.607880+0000", 2024-09-21T21:41:02.236 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.9647000000000004e-05 2024-09-21T21:41:02.236 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.236 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.236 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:02.236 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.607972+0000", 2024-09-21T21:41:02.236 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2511999999999998e-05 2024-09-21T21:41:02.236 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:02.236 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:02.236 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:02.236 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.236 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.236 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:6384 3.8 3:1339e2f0:::rbd_data.112145fae779.0000000000000007:a [read 12312576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:41:02.236 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.594289+0000", 2024-09-21T21:41:02.236 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.574789258, 2024-09-21T21:41:02.237 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.01381668, 2024-09-21T21:41:02.237 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:02.237 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:02.237 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:02.237 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:02.237 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:02.237 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:02.237 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 6384 2024-09-21T21:41:02.237 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.237 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:02.237 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.237 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:02.237 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594289+0000", 2024-09-21T21:41:02.237 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:02.238 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.238 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.238 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:02.238 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594289+0000", 2024-09-21T21:41:02.238 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:02.238 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.238 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.238 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:02.238 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594289+0000", 2024-09-21T21:41:02.238 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8000000000000001e-07 2024-09-21T21:41:02.238 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.238 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.238 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:02.238 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594291+0000", 2024-09-21T21:41:02.238 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.8899999999999999e-06 2024-09-21T21:41:02.239 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.239 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.239 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:02.239 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594297+0000", 2024-09-21T21:41:02.239 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7230000000000001e-06 2024-09-21T21:41:02.239 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.239 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.239 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:02.239 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594302+0000", 2024-09-21T21:41:02.239 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4560000000000002e-06 2024-09-21T21:41:02.239 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.239 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.239 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:02.239 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.608001+0000", 2024-09-21T21:41:02.239 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013699742000000001 2024-09-21T21:41:02.240 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.240 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.240 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:02.240 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.608046+0000", 2024-09-21T21:41:02.240 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4925000000000001e-05 2024-09-21T21:41:02.240 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.240 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.240 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:02.240 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.608106+0000", 2024-09-21T21:41:02.240 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9564000000000001e-05 2024-09-21T21:41:02.240 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:02.240 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:02.240 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:02.240 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.240 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.240 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:6385 3.8 3:1339e2f0:::rbd_data.112145fae779.0000000000000007:a [read 12345344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:41:02.241 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.594307+0000", 2024-09-21T21:41:02.241 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.57477109, 2024-09-21T21:41:02.241 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.014004925, 2024-09-21T21:41:02.241 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:02.241 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:02.241 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:02.241 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:02.241 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:02.241 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:02.241 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 6385 2024-09-21T21:41:02.241 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.241 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:02.241 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.241 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:02.242 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594307+0000", 2024-09-21T21:41:02.242 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:02.242 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.242 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.242 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:02.242 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594307+0000", 2024-09-21T21:41:02.242 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:02.242 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.242 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.242 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:02.242 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594308+0000", 2024-09-21T21:41:02.242 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8500000000000002e-07 2024-09-21T21:41:02.242 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.242 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.243 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:02.243 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594310+0000", 2024-09-21T21:41:02.243 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9300000000000002e-06 2024-09-21T21:41:02.243 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.243 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.243 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:02.243 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594315+0000", 2024-09-21T21:41:02.243 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.626e-06 2024-09-21T21:41:02.243 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.243 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.243 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:02.243 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594320+0000", 2024-09-21T21:41:02.243 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5079999999999999e-06 2024-09-21T21:41:02.243 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.243 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.244 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:02.244 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.608141+0000", 2024-09-21T21:41:02.244 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013821787 2024-09-21T21:41:02.244 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.244 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.244 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:02.244 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.608229+0000", 2024-09-21T21:41:02.244 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.7670000000000006e-05 2024-09-21T21:41:02.244 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.244 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.244 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:02.244 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.608312+0000", 2024-09-21T21:41:02.244 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.3018999999999999e-05 2024-09-21T21:41:02.244 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:02.245 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:02.245 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:02.245 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.245 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.245 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:6386 3.8 3:1339e2f0:::rbd_data.112145fae779.0000000000000007:a [read 12402688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:41:02.245 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.594325+0000", 2024-09-21T21:41:02.245 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.574753152, 2024-09-21T21:41:02.245 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.014122307000000001, 2024-09-21T21:41:02.245 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:02.245 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:02.245 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:02.245 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:02.245 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:02.245 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:02.245 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 6386 2024-09-21T21:41:02.246 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.246 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:02.246 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.246 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:02.246 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594325+0000", 2024-09-21T21:41:02.246 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:02.246 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.246 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.246 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:02.246 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594325+0000", 2024-09-21T21:41:02.246 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:02.246 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.246 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.247 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:02.247 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594326+0000", 2024-09-21T21:41:02.247 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8099999999999998e-07 2024-09-21T21:41:02.247 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.247 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.247 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:02.247 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594328+0000", 2024-09-21T21:41:02.247 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0200000000000001e-06 2024-09-21T21:41:02.247 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.247 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.247 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:02.247 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594333+0000", 2024-09-21T21:41:02.247 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6119999999999998e-06 2024-09-21T21:41:02.247 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.247 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.248 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:02.248 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594338+0000", 2024-09-21T21:41:02.248 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3359999999999997e-06 2024-09-21T21:41:02.248 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.248 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.248 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:02.248 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.608346+0000", 2024-09-21T21:41:02.248 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.014008909999999999 2024-09-21T21:41:02.248 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.248 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.248 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:02.248 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.608388+0000", 2024-09-21T21:41:02.248 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1943000000000001e-05 2024-09-21T21:41:02.249 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.249 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.249 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:02.249 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.608447+0000", 2024-09-21T21:41:02.249 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9104999999999997e-05 2024-09-21T21:41:02.249 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:02.249 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:02.249 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:02.249 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.249 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.249 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:6391 3.18 3:199efa5f:::rbd_data.1121e540c096.000000000000015c:head [set-alloc-hint object_size 524288 write_size 524288,write 8192~3584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:41:02.249 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.594343+0000", 2024-09-21T21:41:02.249 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.574735113, 2024-09-21T21:41:02.250 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.016432473, 2024-09-21T21:41:02.250 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:02.250 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:02.250 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:41:02.250 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:02.250 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:02.250 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:02.250 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 6391 2024-09-21T21:41:02.250 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.250 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:02.250 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.250 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:02.250 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594343+0000", 2024-09-21T21:41:02.250 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:02.251 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.251 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.251 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:02.251 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594343+0000", 2024-09-21T21:41:02.251 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:02.251 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.251 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.251 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:02.251 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594344+0000", 2024-09-21T21:41:02.251 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4499999999999997e-07 2024-09-21T21:41:02.251 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.251 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.251 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:02.251 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594363+0000", 2024-09-21T21:41:02.252 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9048000000000001e-05 2024-09-21T21:41:02.252 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.252 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.252 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:02.252 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594370+0000", 2024-09-21T21:41:02.252 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.7989999999999996e-06 2024-09-21T21:41:02.252 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.252 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.252 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:02.252 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594376+0000", 2024-09-21T21:41:02.252 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9829999999999999e-06 2024-09-21T21:41:02.252 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.252 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.252 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:02.252 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.608344+0000", 2024-09-21T21:41:02.253 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013967319000000001 2024-09-21T21:41:02.253 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.253 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.253 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:02.253 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.608490+0000", 2024-09-21T21:41:02.253 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000145802 2024-09-21T21:41:02.253 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.253 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.253 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 0", 2024-09-21T21:41:02.253 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.608660+0000", 2024-09-21T21:41:02.253 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000170194 2024-09-21T21:41:02.253 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.253 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.253 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:41:02.254 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.610174+0000", 2024-09-21T21:41:02.254 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0015143279999999999 2024-09-21T21:41:02.254 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.254 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.254 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:41:02.254 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.610701+0000", 2024-09-21T21:41:02.254 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00052702499999999998 2024-09-21T21:41:02.254 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.254 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.254 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:41:02.254 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.610740+0000", 2024-09-21T21:41:02.254 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.9168000000000002e-05 2024-09-21T21:41:02.254 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.254 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.255 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:02.255 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.610776+0000", 2024-09-21T21:41:02.255 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5361999999999998e-05 2024-09-21T21:41:02.255 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:02.255 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:02.255 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:02.255 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.255 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.255 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4824.0:1854 3.1e 3:7cbf3d9a:::rbd_data.1121b97f2925.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:02.255 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:01.700058+0000", 2024-09-21T21:41:02.255 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.469020626, 2024-09-21T21:41:02.255 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013123351, 2024-09-21T21:41:02.255 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:02.255 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:02.255 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:41:02.256 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:02.256 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4824", 2024-09-21T21:41:02.256 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:02.256 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1854 2024-09-21T21:41:02.256 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.256 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:02.256 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.256 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:02.256 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.700058+0000", 2024-09-21T21:41:02.256 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:02.256 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.256 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.256 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:02.256 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.700058+0000", 2024-09-21T21:41:02.256 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:02.257 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.257 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.257 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:02.257 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.700060+0000", 2024-09-21T21:41:02.257 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.7199999999999998e-06 2024-09-21T21:41:02.257 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.257 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.257 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:02.257 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.704571+0000", 2024-09-21T21:41:02.257 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0045108259999999999 2024-09-21T21:41:02.257 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.257 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.257 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:02.257 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.704584+0000", 2024-09-21T21:41:02.257 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.2337000000000001e-05 2024-09-21T21:41:02.258 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.258 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.258 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:02.258 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.704590+0000", 2024-09-21T21:41:02.258 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0870000000000003e-06 2024-09-21T21:41:02.258 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.258 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.258 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:02.258 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.704636+0000", 2024-09-21T21:41:02.258 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6483000000000002e-05 2024-09-21T21:41:02.258 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.258 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.258 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:02.258 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.704690+0000", 2024-09-21T21:41:02.259 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3634999999999997e-05 2024-09-21T21:41:02.259 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.259 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.259 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 3", 2024-09-21T21:41:02.259 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.705008+0000", 2024-09-21T21:41:02.259 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00031831600000000002 2024-09-21T21:41:02.259 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.259 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.259 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:41:02.259 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.710076+0000", 2024-09-21T21:41:02.259 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0050678119999999997 2024-09-21T21:41:02.259 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.259 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.259 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:41:02.260 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.713082+0000", 2024-09-21T21:41:02.260 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0030060389999999998 2024-09-21T21:41:02.260 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.260 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.260 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:41:02.260 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.713140+0000", 2024-09-21T21:41:02.260 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7615000000000003e-05 2024-09-21T21:41:02.260 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:02.260 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:02.260 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:02.260 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.713181+0000", 2024-09-21T21:41:02.260 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1480999999999998e-05 2024-09-21T21:41:02.260 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:02.260 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:02.260 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:02.261 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:02.261 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:02.261 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:02.262 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T21:41:02.262 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:02.260+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:02.362 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:02.361+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:02.416 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:02.416 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:41:02.417 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.417 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4854.0:9551 3.11 3:8c2fb66e:::rbd_data.1121e540c096.000000000000009c:head [read 225280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:02.417 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:02.415069+0000", 2024-09-21T21:41:02.417 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.00041001200000000002, 2024-09-21T21:41:02.417 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00043184000000000001, 2024-09-21T21:41:02.417 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:02.417 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:02.417 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:02.417 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:02.417 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4854", 2024-09-21T21:41:02.417 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4118528424", 2024-09-21T21:41:02.417 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 9551 2024-09-21T21:41:02.417 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.417 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:02.418 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.418 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:02.418 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:02.415069+0000", 2024-09-21T21:41:02.418 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:02.418 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.418 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.418 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:02.418 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:02.415069+0000", 2024-09-21T21:41:02.418 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:02.418 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.418 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.418 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:02.418 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:02.415069+0000", 2024-09-21T21:41:02.418 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1200000000000003e-07 2024-09-21T21:41:02.419 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.419 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.419 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:02.419 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:02.415072+0000", 2024-09-21T21:41:02.419 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4889999999999998e-06 2024-09-21T21:41:02.419 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.419 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.419 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:02.419 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:02.415079+0000", 2024-09-21T21:41:02.419 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1849999999999998e-06 2024-09-21T21:41:02.419 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.419 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.419 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:02.419 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:02.415084+0000", 2024-09-21T21:41:02.419 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4240000000000001e-06 2024-09-21T21:41:02.420 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.420 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.420 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:02.420 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:02.415374+0000", 2024-09-21T21:41:02.420 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00028966099999999998 2024-09-21T21:41:02.420 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.420 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.420 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:02.420 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:02.415440+0000", 2024-09-21T21:41:02.420 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6229000000000006e-05 2024-09-21T21:41:02.420 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:02.420 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:02.420 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:02.420 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.420 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.421 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4854.0:9563 3.11 3.2b520311 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:02.421 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:02.415479+0000", 2024-09-21T21:41:02.421 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4294967295.999999, 2024-09-21T21:41:02.421 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7342999999999994e-05, 2024-09-21T21:41:02.421 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:02.421 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:02.421 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2024-09-21T21:41:02.421 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:02.421 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4854", 2024-09-21T21:41:02.421 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4118528424", 2024-09-21T21:41:02.421 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 9563 2024-09-21T21:41:02.421 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.422 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:02.422 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.422 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:02.422 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:02.415479+0000", 2024-09-21T21:41:02.423 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:02.423 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.423 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.423 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:02.423 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:02.415479+0000", 2024-09-21T21:41:02.423 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:02.423 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.423 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.423 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:02.423 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:02.415480+0000", 2024-09-21T21:41:02.423 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4900000000000005e-07 2024-09-21T21:41:02.423 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.423 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.423 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:02.424 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:02.415484+0000", 2024-09-21T21:41:02.424 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9210000000000002e-06 2024-09-21T21:41:02.424 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.424 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.424 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:02.424 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:02.415495+0000", 2024-09-21T21:41:02.424 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0553e-05 2024-09-21T21:41:02.424 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.424 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.424 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:02.424 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:02.415501+0000", 2024-09-21T21:41:02.424 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5910000000000001e-06 2024-09-21T21:41:02.424 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.424 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.424 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:02.425 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:02.415565+0000", 2024-09-21T21:41:02.425 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3249000000000004e-05 2024-09-21T21:41:02.425 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:02.425 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:02.425 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:02.425 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.425 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.425 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4854.0:9564 3.15 3:ad3dbba4:::rbd_data.1121e540c096.000000000000009b:head [read 237568~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:02.425 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:02.415515+0000", 2024-09-21T21:41:02.425 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4294967295.9999638, 2024-09-21T21:41:02.425 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000119647, 2024-09-21T21:41:02.425 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:02.425 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:02.426 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2024-09-21T21:41:02.426 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:02.426 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4854", 2024-09-21T21:41:02.426 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4118528424", 2024-09-21T21:41:02.426 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 9564 2024-09-21T21:41:02.426 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.426 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:02.426 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.426 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:02.426 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:02.415515+0000", 2024-09-21T21:41:02.426 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:02.426 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.426 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.426 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:02.427 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:02.415515+0000", 2024-09-21T21:41:02.427 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:02.427 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.427 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.427 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:02.427 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:02.415515+0000", 2024-09-21T21:41:02.427 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2e-07 2024-09-21T21:41:02.427 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.427 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.427 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:02.427 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:02.415518+0000", 2024-09-21T21:41:02.427 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6800000000000002e-06 2024-09-21T21:41:02.427 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.428 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.428 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:02.428 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:02.415525+0000", 2024-09-21T21:41:02.428 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7000000000000002e-06 2024-09-21T21:41:02.428 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.428 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.428 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:02.428 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:02.415530+0000", 2024-09-21T21:41:02.428 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9019999999999998e-06 2024-09-21T21:41:02.428 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.428 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.428 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:02.428 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:02.415601+0000", 2024-09-21T21:41:02.428 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0839000000000004e-05 2024-09-21T21:41:02.428 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:02.429 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:02.429 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:02.429 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.429 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.429 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4854.0:9565 3.11 3:8c2fb66e:::rbd_data.1121e540c096.000000000000009c:head [read 237568~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:02.429 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:02.415542+0000", 2024-09-21T21:41:02.429 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4294967295.9999366, 2024-09-21T21:41:02.429 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00015947199999999999, 2024-09-21T21:41:02.429 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:02.429 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:02.429 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2024-09-21T21:41:02.429 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:02.429 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4854", 2024-09-21T21:41:02.429 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4118528424", 2024-09-21T21:41:02.429 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 9565 2024-09-21T21:41:02.430 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.430 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:02.430 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.430 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:02.430 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:02.415542+0000", 2024-09-21T21:41:02.430 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:02.430 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.430 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.430 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:02.430 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:02.415542+0000", 2024-09-21T21:41:02.430 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:02.430 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.430 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.430 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:02.431 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:02.415542+0000", 2024-09-21T21:41:02.431 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1600000000000001e-07 2024-09-21T21:41:02.431 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.431 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.431 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:02.431 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:02.415545+0000", 2024-09-21T21:41:02.431 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4949999999999998e-06 2024-09-21T21:41:02.431 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.431 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.431 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:02.431 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:02.415560+0000", 2024-09-21T21:41:02.431 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.543e-05 2024-09-21T21:41:02.431 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.431 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.432 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:02.432 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:02.415566+0000", 2024-09-21T21:41:02.432 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.784e-06 2024-09-21T21:41:02.432 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.432 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.432 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:02.432 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:02.415681+0000", 2024-09-21T21:41:02.432 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000114883 2024-09-21T21:41:02.432 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.432 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.432 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:02.432 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:02.415722+0000", 2024-09-21T21:41:02.432 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0818000000000001e-05 2024-09-21T21:41:02.432 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:02.432 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:02.433 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:02.433 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:02.433 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-21T21:41:02.433 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 4 2024-09-21T21:41:02.433 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:02.434 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T21:41:02.476 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:02.475+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:02.548 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T21:41:02.564 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:02.562+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:02.615 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:02.615 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:41:02.615 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:41:02.615 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:41:02.615 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:02.624 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T21:41:02.666 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:02.665+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:02.741 INFO:teuthology.orchestra.run.smithi164.stdout:1224 write 0x585f400 thru 0x5869b45 (0xa746 bytes) 2024-09-21T21:41:02.742 INFO:teuthology.orchestra.run.smithi164.stdout:1225 write 0x35f3800 thru 0x35fcad0 (0x92d1 bytes) 2024-09-21T21:41:02.743 INFO:teuthology.orchestra.run.smithi164.stdout:1226 punch from 0xdc4c00 to 0xdc7c00, (0x3000 bytes) 2024-09-21T21:41:02.759 INFO:teuthology.orchestra.run.smithi164.stdout:1227 read 0x12ec000 thru 0x12f11ed (0x51ee bytes) 2024-09-21T21:41:02.759 INFO:teuthology.orchestra.run.smithi164.stdout:1229 read 0xb8e7400 thru 0xb8f1481 (0xa082 bytes) 2024-09-21T21:41:02.759 INFO:teuthology.orchestra.run.smithi164.stdout:1232 read 0x466ac00 thru 0x4671282 (0x6683 bytes) 2024-09-21T21:41:02.760 INFO:teuthology.orchestra.run.smithi164.stdout:1233 read 0x143b400 thru 0x144709b (0xbc9c bytes) 2024-09-21T21:41:02.761 INFO:teuthology.orchestra.run.smithi164.stdout:1235 trunc from 0xd443200 to 0xdf6f800 2024-09-21T21:41:02.767 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:02.765+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:02.773 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:02.773 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-21T21:41:02.773 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-21T21:41:02.773 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:41:02.773 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.773 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2415 3.d 3:b26310db:::rbd_data.112145fae779.0000000000000006:a [read 10690560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:02.773 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.406907+0000", 2024-09-21T21:41:02.774 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.3640172210000001, 2024-09-21T21:41:02.774 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044424119999999997, 2024-09-21T21:41:02.774 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:02.774 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:02.774 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:02.774 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:02.774 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:02.774 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:02.774 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2415 2024-09-21T21:41:02.774 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.774 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:02.774 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.775 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:02.775 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.406907+0000", 2024-09-21T21:41:02.775 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:02.775 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.775 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.775 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:02.775 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.406907+0000", 2024-09-21T21:41:02.775 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:02.775 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.775 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.775 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:02.775 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.406908+0000", 2024-09-21T21:41:02.775 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7599999999999998e-07 2024-09-21T21:41:02.775 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.776 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.776 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:02.776 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.406915+0000", 2024-09-21T21:41:02.776 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6729999999999992e-06 2024-09-21T21:41:02.776 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.776 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.776 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:02.776 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.406922+0000", 2024-09-21T21:41:02.776 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7139999999999996e-06 2024-09-21T21:41:02.776 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.776 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.776 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:02.776 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.406927+0000", 2024-09-21T21:41:02.777 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.327e-06 2024-09-21T21:41:02.777 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.777 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.777 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:02.777 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.451179+0000", 2024-09-21T21:41:02.777 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044251984000000001 2024-09-21T21:41:02.777 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.777 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.777 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:02.777 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.451254+0000", 2024-09-21T21:41:02.777 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4703999999999995e-05 2024-09-21T21:41:02.777 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.777 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.777 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:02.777 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.451331+0000", 2024-09-21T21:41:02.778 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7342000000000006e-05 2024-09-21T21:41:02.778 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:02.778 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:02.778 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:02.778 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.778 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.778 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2417 3.d 3:b26310db:::rbd_data.112145fae779.0000000000000006:a [read 10252288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:02.778 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.406934+0000", 2024-09-21T21:41:02.778 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.3639904769999998, 2024-09-21T21:41:02.778 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044511163999999999, 2024-09-21T21:41:02.778 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:02.778 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:02.778 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:02.778 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:02.778 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:02.779 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:02.779 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2417 2024-09-21T21:41:02.779 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.779 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:02.779 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.779 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:02.779 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.406934+0000", 2024-09-21T21:41:02.779 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:02.779 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.779 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.779 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:02.779 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.406934+0000", 2024-09-21T21:41:02.779 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:02.780 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.780 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.780 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:02.780 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.406935+0000", 2024-09-21T21:41:02.780 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6499999999999999e-07 2024-09-21T21:41:02.780 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.780 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.780 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:02.780 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.406937+0000", 2024-09-21T21:41:02.780 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1299999999999999e-06 2024-09-21T21:41:02.780 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.780 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.780 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:02.780 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.406943+0000", 2024-09-21T21:41:02.780 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9989999999999997e-06 2024-09-21T21:41:02.781 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.781 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.781 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:02.781 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.406948+0000", 2024-09-21T21:41:02.781 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9729999999999997e-06 2024-09-21T21:41:02.781 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.781 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.781 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:02.781 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.451365+0000", 2024-09-21T21:41:02.781 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044417808000000003 2024-09-21T21:41:02.781 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.781 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.781 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:02.781 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.451403+0000", 2024-09-21T21:41:02.782 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7944999999999997e-05 2024-09-21T21:41:02.782 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.782 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.782 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:02.782 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.451445+0000", 2024-09-21T21:41:02.782 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1844000000000001e-05 2024-09-21T21:41:02.782 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:02.782 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:02.782 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:02.783 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.783 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.783 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2425 3.d 3:b26310db:::rbd_data.112145fae779.0000000000000006:a [read 10457088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:02.783 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.407639+0000", 2024-09-21T21:41:02.783 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.3632857759999997, 2024-09-21T21:41:02.783 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044288292999999999, 2024-09-21T21:41:02.783 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:02.783 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:02.783 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:02.783 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:02.783 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:02.783 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:02.783 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2425 2024-09-21T21:41:02.783 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.784 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:02.784 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.784 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:02.784 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407639+0000", 2024-09-21T21:41:02.784 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:02.785 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.785 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.785 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:02.785 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407639+0000", 2024-09-21T21:41:02.785 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:02.785 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.785 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.785 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:02.785 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407639+0000", 2024-09-21T21:41:02.785 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8299999999999998e-07 2024-09-21T21:41:02.786 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.786 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.786 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:02.786 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407641+0000", 2024-09-21T21:41:02.786 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9149999999999999e-06 2024-09-21T21:41:02.786 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.786 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.786 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:02.786 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407647+0000", 2024-09-21T21:41:02.786 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3110000000000002e-06 2024-09-21T21:41:02.786 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.786 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.786 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:02.786 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407653+0000", 2024-09-21T21:41:02.787 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1309999999999997e-06 2024-09-21T21:41:02.787 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.787 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.787 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:02.787 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.451778+0000", 2024-09-21T21:41:02.787 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044125782000000002 2024-09-21T21:41:02.787 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.787 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.787 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:02.787 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.451850+0000", 2024-09-21T21:41:02.787 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1211999999999995e-05 2024-09-21T21:41:02.787 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.787 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.787 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:02.788 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.451927+0000", 2024-09-21T21:41:02.788 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7558999999999994e-05 2024-09-21T21:41:02.788 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:02.788 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:02.788 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:02.788 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.788 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.788 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2427 3.d 3:b26310db:::rbd_data.112145fae779.0000000000000006:a [read 10489856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:02.788 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.407659+0000", 2024-09-21T21:41:02.788 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.3632654630000003, 2024-09-21T21:41:02.788 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044463186000000002, 2024-09-21T21:41:02.788 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:02.788 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:02.788 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:02.789 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:02.789 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:02.789 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:02.789 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2427 2024-09-21T21:41:02.789 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.789 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:02.789 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.789 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:02.789 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407659+0000", 2024-09-21T21:41:02.789 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:02.789 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.789 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.789 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:02.789 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407659+0000", 2024-09-21T21:41:02.790 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:02.790 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.790 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.790 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:02.790 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407659+0000", 2024-09-21T21:41:02.790 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3000000000000002e-07 2024-09-21T21:41:02.790 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.790 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.790 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:02.790 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407662+0000", 2024-09-21T21:41:02.790 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0980000000000001e-06 2024-09-21T21:41:02.790 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.790 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.790 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:02.791 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407668+0000", 2024-09-21T21:41:02.791 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5209999999999999e-06 2024-09-21T21:41:02.791 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.791 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.791 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:02.791 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407674+0000", 2024-09-21T21:41:02.791 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7590000000000003e-06 2024-09-21T21:41:02.791 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.791 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.791 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:02.791 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.451969+0000", 2024-09-21T21:41:02.791 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044294965999999998 2024-09-21T21:41:02.791 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.791 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.792 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:02.792 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.452045+0000", 2024-09-21T21:41:02.792 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6271000000000002e-05 2024-09-21T21:41:02.792 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.792 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.792 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:02.792 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.452122+0000", 2024-09-21T21:41:02.792 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7241000000000001e-05 2024-09-21T21:41:02.792 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:02.792 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:02.792 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:02.792 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.793 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.793 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2429 3.d 3:b26310db:::rbd_data.112145fae779.0000000000000006:a [read 10547200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:02.793 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.407681+0000", 2024-09-21T21:41:02.793 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.3632437890000002, 2024-09-21T21:41:02.793 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.04455829, 2024-09-21T21:41:02.793 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:02.793 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:02.793 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:02.793 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:02.793 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:02.793 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:02.793 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2429 2024-09-21T21:41:02.793 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.794 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:02.794 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.794 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:02.794 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407681+0000", 2024-09-21T21:41:02.794 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:02.794 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.794 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.794 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:02.794 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407681+0000", 2024-09-21T21:41:02.794 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:02.794 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.794 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.794 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:02.795 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407681+0000", 2024-09-21T21:41:02.795 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2300000000000001e-07 2024-09-21T21:41:02.795 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.795 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.795 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:02.795 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407683+0000", 2024-09-21T21:41:02.795 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0020000000000001e-06 2024-09-21T21:41:02.795 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.795 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.795 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:02.795 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407690+0000", 2024-09-21T21:41:02.795 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4269999999999996e-06 2024-09-21T21:41:02.795 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.795 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.796 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:02.796 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407694+0000", 2024-09-21T21:41:02.796 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6029999999999998e-06 2024-09-21T21:41:02.796 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.796 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.796 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:02.796 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.452153+0000", 2024-09-21T21:41:02.796 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044458232 2024-09-21T21:41:02.796 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.796 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.796 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:02.796 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.452196+0000", 2024-09-21T21:41:02.797 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3248999999999999e-05 2024-09-21T21:41:02.797 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.797 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.797 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:02.797 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.452239+0000", 2024-09-21T21:41:02.797 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3254000000000003e-05 2024-09-21T21:41:02.797 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:02.797 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:02.797 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:02.797 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.797 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.797 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2431 3.d 3:b26310db:::rbd_data.112145fae779.0000000000000006:a [read 10604544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:02.797 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.407701+0000", 2024-09-21T21:41:02.798 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.3632240060000003, 2024-09-21T21:41:02.798 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044639389000000002, 2024-09-21T21:41:02.798 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:02.798 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:02.798 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:02.798 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:02.798 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:02.798 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:02.798 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2431 2024-09-21T21:41:02.798 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.798 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:02.798 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.798 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:02.798 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407701+0000", 2024-09-21T21:41:02.799 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:02.799 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.799 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.799 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:02.799 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407701+0000", 2024-09-21T21:41:02.799 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:02.799 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.799 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.799 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:02.799 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407701+0000", 2024-09-21T21:41:02.799 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.63e-07 2024-09-21T21:41:02.799 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.799 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.799 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:02.800 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407703+0000", 2024-09-21T21:41:02.800 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.979e-06 2024-09-21T21:41:02.800 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.800 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.800 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:02.800 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407710+0000", 2024-09-21T21:41:02.800 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5470000000000002e-06 2024-09-21T21:41:02.800 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.800 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.800 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:02.800 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407715+0000", 2024-09-21T21:41:02.800 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.972e-06 2024-09-21T21:41:02.800 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.800 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.800 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:02.801 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.452266+0000", 2024-09-21T21:41:02.801 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044551201999999998 2024-09-21T21:41:02.801 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.801 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.801 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:02.801 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.452302+0000", 2024-09-21T21:41:02.801 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6516000000000001e-05 2024-09-21T21:41:02.801 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.801 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.801 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:02.801 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.452340+0000", 2024-09-21T21:41:02.801 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7710000000000003e-05 2024-09-21T21:41:02.801 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:02.801 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:02.802 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:02.802 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.802 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.802 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2434 3.d 3:b26310db:::rbd_data.112145fae779.0000000000000006:a [read 10637312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:02.802 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.407720+0000", 2024-09-21T21:41:02.802 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.3632041660000001, 2024-09-21T21:41:02.802 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.04471874, 2024-09-21T21:41:02.802 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:02.802 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:02.802 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:02.802 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:02.802 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:02.802 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:02.803 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2434 2024-09-21T21:41:02.803 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.803 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:02.803 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.803 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:02.803 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407720+0000", 2024-09-21T21:41:02.803 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:02.803 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.803 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.803 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:02.803 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407720+0000", 2024-09-21T21:41:02.803 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:02.803 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.804 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.804 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:02.804 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407721+0000", 2024-09-21T21:41:02.804 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5199999999999998e-07 2024-09-21T21:41:02.804 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.804 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.804 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:02.804 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407723+0000", 2024-09-21T21:41:02.804 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9139999999999998e-06 2024-09-21T21:41:02.804 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.804 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.804 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:02.804 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407729+0000", 2024-09-21T21:41:02.804 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2009999999999996e-06 2024-09-21T21:41:02.804 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.805 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.805 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:02.805 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407734+0000", 2024-09-21T21:41:02.805 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.006e-06 2024-09-21T21:41:02.805 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.805 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.805 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:02.805 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.452364+0000", 2024-09-21T21:41:02.805 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044630625 2024-09-21T21:41:02.805 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.805 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.805 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:02.805 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.452402+0000", 2024-09-21T21:41:02.805 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7518999999999998e-05 2024-09-21T21:41:02.805 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.806 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.806 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:02.806 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.452439+0000", 2024-09-21T21:41:02.806 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7122999999999997e-05 2024-09-21T21:41:02.806 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:02.806 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:02.806 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:02.806 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.806 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.806 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2436 3.d 3:b26310db:::rbd_data.112145fae779.0000000000000006:a [read 10694656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:02.806 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.407740+0000", 2024-09-21T21:41:02.806 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.3631848230000001, 2024-09-21T21:41:02.806 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044797624000000001, 2024-09-21T21:41:02.806 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:02.807 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:02.807 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:02.807 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:02.807 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:02.807 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:02.807 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2436 2024-09-21T21:41:02.807 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.807 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:02.807 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.807 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:02.807 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407740+0000", 2024-09-21T21:41:02.807 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:02.807 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.807 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.807 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:02.808 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407740+0000", 2024-09-21T21:41:02.808 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:02.808 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.808 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.808 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:02.808 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407740+0000", 2024-09-21T21:41:02.808 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9299999999999999e-07 2024-09-21T21:41:02.808 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.808 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.808 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:02.808 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407742+0000", 2024-09-21T21:41:02.808 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.826e-06 2024-09-21T21:41:02.808 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.808 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.808 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:02.809 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407748+0000", 2024-09-21T21:41:02.809 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2149999999999998e-06 2024-09-21T21:41:02.809 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.809 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.809 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:02.809 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407753+0000", 2024-09-21T21:41:02.809 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.848e-06 2024-09-21T21:41:02.809 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.809 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.809 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:02.809 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.452463+0000", 2024-09-21T21:41:02.809 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044709596999999997 2024-09-21T21:41:02.809 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.809 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.809 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:02.810 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.452500+0000", 2024-09-21T21:41:02.810 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7058999999999999e-05 2024-09-21T21:41:02.810 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.810 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.810 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:02.810 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.452537+0000", 2024-09-21T21:41:02.810 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7685999999999998e-05 2024-09-21T21:41:02.810 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:02.810 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:02.810 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:02.810 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.810 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.810 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2602 3.d 3:b154679f:::rbd_data.112145fae779.0000000000000004:a [read 3923968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:02.810 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.411479+0000", 2024-09-21T21:41:02.811 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.359445365, 2024-09-21T21:41:02.811 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044281854000000002, 2024-09-21T21:41:02.811 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:02.811 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:02.811 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:02.811 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:02.811 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:02.811 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:02.811 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2602 2024-09-21T21:41:02.811 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.811 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:02.811 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.811 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:02.811 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411479+0000", 2024-09-21T21:41:02.812 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:02.812 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.812 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.812 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:02.812 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411479+0000", 2024-09-21T21:41:02.812 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:02.812 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.812 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.812 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:02.812 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411480+0000", 2024-09-21T21:41:02.812 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.37e-07 2024-09-21T21:41:02.812 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.812 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.812 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:02.812 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411482+0000", 2024-09-21T21:41:02.813 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9829999999999999e-06 2024-09-21T21:41:02.813 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.813 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.813 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:02.813 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411488+0000", 2024-09-21T21:41:02.813 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.947e-06 2024-09-21T21:41:02.813 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.813 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.813 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:02.813 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411492+0000", 2024-09-21T21:41:02.813 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.357e-06 2024-09-21T21:41:02.813 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.813 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.813 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:02.813 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.455581+0000", 2024-09-21T21:41:02.813 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044089077999999997 2024-09-21T21:41:02.814 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.814 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.814 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:02.814 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.455675+0000", 2024-09-21T21:41:02.814 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3757000000000004e-05 2024-09-21T21:41:02.814 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.814 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.814 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:02.814 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.455761+0000", 2024-09-21T21:41:02.814 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6194999999999997e-05 2024-09-21T21:41:02.814 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:02.814 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:02.814 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:02.814 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.815 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.815 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2612 3.d 3:b26310db:::rbd_data.112145fae779.0000000000000006:a [read 3448832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:02.815 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.411498+0000", 2024-09-21T21:41:02.815 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.3594265969999997, 2024-09-21T21:41:02.815 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044369585000000003, 2024-09-21T21:41:02.815 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:02.815 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:02.815 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:02.815 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:02.815 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:02.815 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:02.815 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2612 2024-09-21T21:41:02.815 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.815 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:02.815 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.816 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:02.816 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411498+0000", 2024-09-21T21:41:02.816 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:02.816 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.816 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.816 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:02.816 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411498+0000", 2024-09-21T21:41:02.816 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:02.816 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.816 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.816 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:02.816 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411498+0000", 2024-09-21T21:41:02.816 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6899999999999998e-07 2024-09-21T21:41:02.816 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.817 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.817 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:02.817 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411500+0000", 2024-09-21T21:41:02.817 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.841e-06 2024-09-21T21:41:02.817 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.817 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.817 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:02.817 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411506+0000", 2024-09-21T21:41:02.817 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7350000000000001e-06 2024-09-21T21:41:02.817 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.817 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.817 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:02.817 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411510+0000", 2024-09-21T21:41:02.817 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4830000000000001e-06 2024-09-21T21:41:02.817 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.818 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.818 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:02.818 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.455789+0000", 2024-09-21T21:41:02.818 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044278359000000003 2024-09-21T21:41:02.818 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.818 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.818 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:02.818 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.455824+0000", 2024-09-21T21:41:02.818 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5719000000000001e-05 2024-09-21T21:41:02.818 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.818 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.818 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:02.818 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.455868+0000", 2024-09-21T21:41:02.818 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3078999999999999e-05 2024-09-21T21:41:02.818 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:02.819 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:02.819 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:02.819 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.819 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.819 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2613 3.d 3:b26310db:::rbd_data.112145fae779.0000000000000006:a [read 3506176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:02.819 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.411516+0000", 2024-09-21T21:41:02.819 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.3594089199999999, 2024-09-21T21:41:02.819 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044562615, 2024-09-21T21:41:02.819 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:02.819 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:02.819 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:02.820 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:02.820 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:02.820 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:02.820 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2613 2024-09-21T21:41:02.820 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.820 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:02.820 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.820 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:02.820 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411516+0000", 2024-09-21T21:41:02.820 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:02.820 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.820 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.820 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:02.820 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411516+0000", 2024-09-21T21:41:02.820 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:02.821 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.821 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.821 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:02.821 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411516+0000", 2024-09-21T21:41:02.821 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4400000000000001e-07 2024-09-21T21:41:02.821 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.821 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.821 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:02.821 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411522+0000", 2024-09-21T21:41:02.821 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2600000000000002e-06 2024-09-21T21:41:02.821 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.821 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.821 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:02.821 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411528+0000", 2024-09-21T21:41:02.822 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5759999999999996e-06 2024-09-21T21:41:02.822 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.822 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.822 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:02.822 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411534+0000", 2024-09-21T21:41:02.822 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3010000000000003e-06 2024-09-21T21:41:02.822 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.822 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.822 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:02.822 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.455911+0000", 2024-09-21T21:41:02.822 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044376546000000003 2024-09-21T21:41:02.822 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.822 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.822 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:02.822 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.455996+0000", 2024-09-21T21:41:02.823 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.4980000000000003e-05 2024-09-21T21:41:02.823 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.823 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.823 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:02.823 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.456078+0000", 2024-09-21T21:41:02.823 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2608000000000006e-05 2024-09-21T21:41:02.823 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:02.823 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:02.823 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:02.823 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.823 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.823 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2614 3.d 3:b26310db:::rbd_data.112145fae779.0000000000000006:a [read 3538944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:02.823 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.411540+0000", 2024-09-21T21:41:02.823 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.3593841060000003, 2024-09-21T21:41:02.823 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044644049999999998, 2024-09-21T21:41:02.824 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:02.824 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:02.824 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:02.824 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:02.824 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:02.824 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:02.824 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2614 2024-09-21T21:41:02.824 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.824 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:02.824 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.824 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:02.824 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411540+0000", 2024-09-21T21:41:02.824 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:02.824 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.824 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.825 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:02.825 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411540+0000", 2024-09-21T21:41:02.825 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:02.825 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.825 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.825 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:02.825 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411541+0000", 2024-09-21T21:41:02.825 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3799999999999997e-07 2024-09-21T21:41:02.825 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.825 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.825 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:02.825 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411543+0000", 2024-09-21T21:41:02.825 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1100000000000001e-06 2024-09-21T21:41:02.825 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.825 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.826 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:02.826 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411549+0000", 2024-09-21T21:41:02.826 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8300000000000001e-06 2024-09-21T21:41:02.826 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.826 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.826 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:02.826 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411554+0000", 2024-09-21T21:41:02.826 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8169999999999998e-06 2024-09-21T21:41:02.826 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.826 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.826 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:02.826 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.456107+0000", 2024-09-21T21:41:02.826 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044553552000000003 2024-09-21T21:41:02.826 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.827 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.827 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:02.827 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.456144+0000", 2024-09-21T21:41:02.827 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6560000000000002e-05 2024-09-21T21:41:02.827 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.827 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.827 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:02.827 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.456185+0000", 2024-09-21T21:41:02.827 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0642999999999997e-05 2024-09-21T21:41:02.827 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:02.827 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:02.827 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:02.827 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.827 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.827 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2615 3.d 3:b26310db:::rbd_data.112145fae779.0000000000000006:a [read 3596288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:02.828 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.411560+0000", 2024-09-21T21:41:02.828 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.3593645509999996, 2024-09-21T21:41:02.828 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044828397999999998, 2024-09-21T21:41:02.828 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:02.828 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:02.828 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:02.828 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:02.828 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:02.828 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:02.828 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2615 2024-09-21T21:41:02.828 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.828 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:02.828 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.828 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:02.828 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411560+0000", 2024-09-21T21:41:02.829 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:02.829 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.829 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.829 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:02.829 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411560+0000", 2024-09-21T21:41:02.829 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:02.829 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.829 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.829 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:02.829 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411560+0000", 2024-09-21T21:41:02.829 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7399999999999999e-07 2024-09-21T21:41:02.829 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.829 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.829 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:02.830 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411562+0000", 2024-09-21T21:41:02.830 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.854e-06 2024-09-21T21:41:02.830 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.830 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.830 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:02.830 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411568+0000", 2024-09-21T21:41:02.830 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6760000000000004e-06 2024-09-21T21:41:02.830 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.830 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.830 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:02.830 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411573+0000", 2024-09-21T21:41:02.830 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6269999999999999e-06 2024-09-21T21:41:02.830 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.831 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.831 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:02.831 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.456226+0000", 2024-09-21T21:41:02.831 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044653088000000001 2024-09-21T21:41:02.831 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.831 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.831 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:02.831 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.456311+0000", 2024-09-21T21:41:02.831 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5685000000000004e-05 2024-09-21T21:41:02.831 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.831 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.831 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:02.831 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.456388+0000", 2024-09-21T21:41:02.831 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7094000000000004e-05 2024-09-21T21:41:02.831 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:02.832 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:02.832 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:02.832 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.832 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.832 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2616 3.d 3:b26310db:::rbd_data.112145fae779.0000000000000006:a [read 3653632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:02.832 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.411578+0000", 2024-09-21T21:41:02.832 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.3593466469999997, 2024-09-21T21:41:02.832 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044914731999999999, 2024-09-21T21:41:02.832 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:02.832 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:02.832 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:02.832 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:02.832 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:02.832 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:02.832 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2616 2024-09-21T21:41:02.833 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.833 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:02.833 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.833 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:02.833 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411578+0000", 2024-09-21T21:41:02.833 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:02.833 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.833 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.833 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:02.833 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411578+0000", 2024-09-21T21:41:02.833 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:02.833 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.833 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.833 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:02.834 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411578+0000", 2024-09-21T21:41:02.834 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.03e-07 2024-09-21T21:41:02.834 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.834 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.834 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:02.834 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411580+0000", 2024-09-21T21:41:02.834 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.8199999999999999e-06 2024-09-21T21:41:02.834 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.834 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.834 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:02.834 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411586+0000", 2024-09-21T21:41:02.834 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6319999999999996e-06 2024-09-21T21:41:02.834 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.834 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.835 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:02.835 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411590+0000", 2024-09-21T21:41:02.835 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5979999999999999e-06 2024-09-21T21:41:02.835 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.835 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.835 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:02.835 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.456418+0000", 2024-09-21T21:41:02.835 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044827504999999997 2024-09-21T21:41:02.835 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.835 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.835 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:02.835 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.456454+0000", 2024-09-21T21:41:02.835 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5883999999999997e-05 2024-09-21T21:41:02.835 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.835 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.836 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:02.836 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.456493+0000", 2024-09-21T21:41:02.836 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8890000000000002e-05 2024-09-21T21:41:02.836 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:02.836 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:02.836 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:02.836 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.836 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.836 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2617 3.d 3:b26310db:::rbd_data.112145fae779.0000000000000006:a [read 3686400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:02.836 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.411596+0000", 2024-09-21T21:41:02.836 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.3593290570000001, 2024-09-21T21:41:02.836 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.045094515000000002, 2024-09-21T21:41:02.836 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:02.836 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:02.836 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:02.837 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:02.837 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:02.837 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:02.837 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2617 2024-09-21T21:41:02.837 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.837 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:02.837 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.837 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:02.837 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411596+0000", 2024-09-21T21:41:02.837 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:02.837 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.837 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.837 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:02.837 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411596+0000", 2024-09-21T21:41:02.837 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:02.838 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.838 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.838 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:02.838 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411596+0000", 2024-09-21T21:41:02.838 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3500000000000002e-07 2024-09-21T21:41:02.838 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.838 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.838 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:02.838 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411598+0000", 2024-09-21T21:41:02.838 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.871e-06 2024-09-21T21:41:02.838 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.838 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.838 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:02.838 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411603+0000", 2024-09-21T21:41:02.839 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6420000000000004e-06 2024-09-21T21:41:02.839 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.839 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.839 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:02.839 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411608+0000", 2024-09-21T21:41:02.839 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4780000000000002e-06 2024-09-21T21:41:02.839 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.839 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.839 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:02.839 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.456529+0000", 2024-09-21T21:41:02.839 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044921232999999998 2024-09-21T21:41:02.839 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.839 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.839 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:02.839 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.456616+0000", 2024-09-21T21:41:02.840 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6712000000000006e-05 2024-09-21T21:41:02.840 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.840 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.840 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:02.840 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.456690+0000", 2024-09-21T21:41:02.840 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4244000000000003e-05 2024-09-21T21:41:02.840 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:02.840 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:02.840 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:02.840 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.840 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.840 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2618 3.d 3:b26310db:::rbd_data.112145fae779.0000000000000006:a [read 3743744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:02.840 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.411613+0000", 2024-09-21T21:41:02.840 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.3593114780000004, 2024-09-21T21:41:02.840 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.045180423999999997, 2024-09-21T21:41:02.841 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:02.841 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:02.841 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:02.841 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:02.841 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:02.841 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:02.841 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2618 2024-09-21T21:41:02.841 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.841 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:02.841 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.841 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:02.841 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411613+0000", 2024-09-21T21:41:02.841 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:02.841 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.841 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.842 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:02.842 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411613+0000", 2024-09-21T21:41:02.842 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:02.842 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.842 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.842 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:02.842 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.412868+0000", 2024-09-21T21:41:02.842 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012549 2024-09-21T21:41:02.842 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.842 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.842 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:02.842 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.412873+0000", 2024-09-21T21:41:02.842 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9060000000000001e-06 2024-09-21T21:41:02.842 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.842 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.843 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:02.843 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.412884+0000", 2024-09-21T21:41:02.843 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1164e-05 2024-09-21T21:41:02.843 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.843 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.843 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:02.843 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.412892+0000", 2024-09-21T21:41:02.843 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5719999999999997e-06 2024-09-21T21:41:02.843 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.843 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.843 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:02.843 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.456717+0000", 2024-09-21T21:41:02.843 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043825559 2024-09-21T21:41:02.843 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.844 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.844 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:02.844 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.456753+0000", 2024-09-21T21:41:02.844 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6158000000000002e-05 2024-09-21T21:41:02.844 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.844 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.844 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:02.844 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.456794+0000", 2024-09-21T21:41:02.844 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0164999999999999e-05 2024-09-21T21:41:02.844 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:02.844 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:02.844 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:02.844 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.844 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.845 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2667 3.d 3:b2093d43:::rbd_data.112145fae779.000000000000000b:a [read 3809280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:02.845 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.414362+0000", 2024-09-21T21:41:02.845 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.3565627999999998, 2024-09-21T21:41:02.845 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044335620999999999, 2024-09-21T21:41:02.845 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:02.845 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:02.845 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:02.845 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:02.845 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:02.845 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:02.845 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2667 2024-09-21T21:41:02.845 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.845 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:02.845 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.846 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:02.846 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.414362+0000", 2024-09-21T21:41:02.846 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:02.846 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.846 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.846 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:02.846 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.414362+0000", 2024-09-21T21:41:02.846 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:02.846 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.846 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.846 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:02.846 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.414867+0000", 2024-09-21T21:41:02.846 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00050531999999999997 2024-09-21T21:41:02.846 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.847 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.847 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:02.847 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.414871+0000", 2024-09-21T21:41:02.847 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2520000000000001e-06 2024-09-21T21:41:02.847 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.847 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.847 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:02.847 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.414879+0000", 2024-09-21T21:41:02.847 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0369999999999995e-06 2024-09-21T21:41:02.847 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.847 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.847 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:02.847 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.414886+0000", 2024-09-21T21:41:02.847 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.19e-06 2024-09-21T21:41:02.848 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.848 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.848 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:02.848 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.458613+0000", 2024-09-21T21:41:02.848 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043727184000000002 2024-09-21T21:41:02.848 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.848 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.848 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:02.848 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.458649+0000", 2024-09-21T21:41:02.848 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6418999999999998e-05 2024-09-21T21:41:02.848 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.848 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.848 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:02.848 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.458697+0000", 2024-09-21T21:41:02.848 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8219e-05 2024-09-21T21:41:02.849 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:02.849 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:02.849 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:02.849 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.849 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.849 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2725 3.d 3:b154679f:::rbd_data.112145fae779.0000000000000004:a [read 3534848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:02.849 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.414975+0000", 2024-09-21T21:41:02.849 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.3559498740000002, 2024-09-21T21:41:02.849 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044402427000000001, 2024-09-21T21:41:02.849 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:02.849 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:02.849 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:02.849 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:02.849 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:02.849 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:02.850 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2725 2024-09-21T21:41:02.850 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.850 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:02.850 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.850 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:02.850 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.414975+0000", 2024-09-21T21:41:02.850 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:02.850 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.850 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.850 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:02.850 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.414975+0000", 2024-09-21T21:41:02.850 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:02.850 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.850 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.851 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:02.851 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.414975+0000", 2024-09-21T21:41:02.851 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1899999999999999e-07 2024-09-21T21:41:02.851 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.851 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.851 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:02.851 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.414976+0000", 2024-09-21T21:41:02.851 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1689999999999999e-06 2024-09-21T21:41:02.851 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.851 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.851 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:02.851 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.414979+0000", 2024-09-21T21:41:02.851 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9900000000000002e-06 2024-09-21T21:41:02.851 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.851 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.852 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:02.852 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.414982+0000", 2024-09-21T21:41:02.852 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.621e-06 2024-09-21T21:41:02.852 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.852 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.852 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:02.852 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.459221+0000", 2024-09-21T21:41:02.852 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044239602000000003 2024-09-21T21:41:02.852 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.852 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.852 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:02.852 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.459297+0000", 2024-09-21T21:41:02.852 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5768999999999999e-05 2024-09-21T21:41:02.852 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.853 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.853 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:02.853 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.459377+0000", 2024-09-21T21:41:02.853 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0056999999999996e-05 2024-09-21T21:41:02.853 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:02.853 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:02.853 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:02.853 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.853 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.853 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2726 3.d 3:b154679f:::rbd_data.112145fae779.0000000000000004:a [read 3567616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:02.853 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.414985+0000", 2024-09-21T21:41:02.853 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.3559398070000004, 2024-09-21T21:41:02.853 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044526137, 2024-09-21T21:41:02.853 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:02.853 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:02.854 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:02.854 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:02.854 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:02.854 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:02.854 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2726 2024-09-21T21:41:02.854 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.854 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:02.854 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.854 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:02.854 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.414985+0000", 2024-09-21T21:41:02.854 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:02.854 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.854 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.854 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:02.855 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.414985+0000", 2024-09-21T21:41:02.855 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:02.855 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.855 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.855 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:02.855 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.414985+0000", 2024-09-21T21:41:02.855 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0600000000000001e-07 2024-09-21T21:41:02.855 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.855 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.855 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:02.855 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.414986+0000", 2024-09-21T21:41:02.855 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.175e-06 2024-09-21T21:41:02.855 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.855 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.855 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:02.856 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.414989+0000", 2024-09-21T21:41:02.856 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8720000000000002e-06 2024-09-21T21:41:02.856 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.856 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.856 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:02.856 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.414992+0000", 2024-09-21T21:41:02.856 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3680000000000002e-06 2024-09-21T21:41:02.856 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.856 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.856 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:02.856 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.459415+0000", 2024-09-21T21:41:02.856 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044422167999999998 2024-09-21T21:41:02.856 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.856 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.856 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:02.857 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.459466+0000", 2024-09-21T21:41:02.857 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1177000000000001e-05 2024-09-21T21:41:02.857 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.857 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.857 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:02.857 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.459511+0000", 2024-09-21T21:41:02.857 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5071000000000003e-05 2024-09-21T21:41:02.857 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:02.857 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:02.857 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:02.857 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.857 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.857 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2727 3.d 3:b154679f:::rbd_data.112145fae779.0000000000000004:a [read 3624960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:02.857 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.414997+0000", 2024-09-21T21:41:02.857 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.3559272340000001, 2024-09-21T21:41:02.858 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044651328999999997, 2024-09-21T21:41:02.858 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:02.858 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:02.858 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:02.858 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:02.858 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:02.858 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:02.858 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2727 2024-09-21T21:41:02.858 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.858 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:02.858 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.858 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:02.858 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.414997+0000", 2024-09-21T21:41:02.858 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:02.858 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.859 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.859 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:02.859 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.414997+0000", 2024-09-21T21:41:02.859 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:02.859 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.859 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.859 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:02.859 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.416868+0000", 2024-09-21T21:41:02.859 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001871019 2024-09-21T21:41:02.859 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.859 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.859 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:02.859 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.416873+0000", 2024-09-21T21:41:02.859 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6299999999999997e-06 2024-09-21T21:41:02.860 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.860 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.860 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:02.860 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.416883+0000", 2024-09-21T21:41:02.860 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0172e-05 2024-09-21T21:41:02.860 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.860 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.860 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:02.860 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.416890+0000", 2024-09-21T21:41:02.860 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8689999999999998e-06 2024-09-21T21:41:02.860 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.860 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.861 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:02.861 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.459541+0000", 2024-09-21T21:41:02.861 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042650592000000001 2024-09-21T21:41:02.861 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.861 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.861 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:02.861 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.459601+0000", 2024-09-21T21:41:02.861 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9978000000000001e-05 2024-09-21T21:41:02.861 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:02.861 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:02.861 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:02.861 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.459649+0000", 2024-09-21T21:41:02.861 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8068999999999997e-05 2024-09-21T21:41:02.861 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:02.861 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:02.861 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:02.862 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:02.862 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:02.862 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:02.864 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T21:41:02.864 INFO:teuthology.orchestra.run.smithi164.stdout:1236 write 0xee5b800 thru 0xee65ce1 (0xa4e2 bytes) 2024-09-21T21:41:02.865 INFO:teuthology.orchestra.run.smithi164.stdout:truncating to largest ever: 0xe8d6000 2024-09-21T21:41:02.865 INFO:teuthology.orchestra.run.smithi164.stdout:1237 trunc from 0xee65e00 to 0xe8d6000 2024-09-21T21:41:02.865 INFO:teuthology.orchestra.run.smithi164.stdout:1238 read 0x5aa6600 thru 0x5ab3ec3 (0xd8c4 bytes) 2024-09-21T21:41:02.865 INFO:teuthology.orchestra.run.smithi164.stdout:1239 punch from 0x1e2ea00 to 0x1e34a00, (0x6000 bytes) 2024-09-21T21:41:02.865 INFO:teuthology.orchestra.run.smithi164.stdout:1240 trunc from 0xe8d6000 to 0xce66c00 2024-09-21T21:41:02.865 INFO:teuthology.orchestra.run.smithi164.stdout:1243 write 0x15d9800 thru 0x15dd328 (0x3b29 bytes) 2024-09-21T21:41:02.865 INFO:teuthology.orchestra.run.smithi164.stdout:1244 write 0xa13fa00 thru 0xa1425be (0x2bbf bytes) 2024-09-21T21:41:02.865 INFO:teuthology.orchestra.run.smithi164.stdout:1246 read 0x78dde00 thru 0x78e5b7d (0x7d7e bytes) 2024-09-21T21:41:02.865 INFO:teuthology.orchestra.run.smithi164.stdout:1247 write 0x54a9a00 thru 0x54acb17 (0x3118 bytes) 2024-09-21T21:41:02.865 INFO:teuthology.orchestra.run.smithi164.stdout:1248 trunc from 0xce66c00 to 0xcd09600 2024-09-21T21:41:02.867 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:02.866+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:02.868 INFO:teuthology.orchestra.run.smithi164.stdout:1249 read 0x5d0ca00 thru 0x5d0fb7b (0x317c bytes) 2024-09-21T21:41:02.868 INFO:teuthology.orchestra.run.smithi164.stdout:1254 write 0x5cf3400 thru 0x5cfc8b8 (0x94b9 bytes) 2024-09-21T21:41:02.868 INFO:teuthology.orchestra.run.smithi164.stdout:1256 read 0x478ee00 thru 0x47991cf (0xa3d0 bytes) 2024-09-21T21:41:02.871 INFO:teuthology.orchestra.run.smithi164.stdout:1260 read 0x27e8800 thru 0x27f37d2 (0xafd3 bytes) 2024-09-21T21:41:02.872 INFO:teuthology.orchestra.run.smithi164.stdout:1261 write 0x10e2400 thru 0x10e3807 (0x1408 bytes) 2024-09-21T21:41:02.872 INFO:teuthology.orchestra.run.smithi164.stdout:1265 write 0x297e000 thru 0x298467a (0x667b bytes) 2024-09-21T21:41:02.873 INFO:teuthology.orchestra.run.smithi164.stdout:1267 read 0xb7fa600 thru 0xb7fc85a (0x225b bytes) 2024-09-21T21:41:02.873 INFO:teuthology.orchestra.run.smithi164.stdout:1270 read 0x397ec00 thru 0x3987a61 (0x8e62 bytes) 2024-09-21T21:41:02.874 INFO:teuthology.orchestra.run.smithi164.stdout:1272 write 0xab45400 thru 0xab4d664 (0x8265 bytes) 2024-09-21T21:41:02.874 INFO:teuthology.orchestra.run.smithi164.stdout:1273 write 0x7f6cc00 thru 0x7f76826 (0x9c27 bytes) 2024-09-21T21:41:02.879 INFO:teuthology.orchestra.run.smithi164.stdout:1274 trunc from 0xcd09600 to 0x8ea9800 2024-09-21T21:41:02.919 INFO:teuthology.orchestra.run.smithi164.stdout:1275 punch from 0x7000e00 to 0x7001400, (0x600 bytes) 2024-09-21T21:41:02.927 INFO:teuthology.orchestra.run.smithi164.stdout:1277 write 0x6e04800 thru 0x6e120a1 (0xd8a2 bytes) 2024-09-21T21:41:02.934 INFO:teuthology.orchestra.run.smithi164.stdout:1278 trunc from 0x8ea9800 to 0x37fb000 2024-09-21T21:41:02.968 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:02.967+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:02.980 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:02.980 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:41:02.980 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-21T21:41:02.980 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:02.989 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T21:41:02.989 INFO:teuthology.orchestra.run.smithi164.stdout:1280 write 0x5cd7e00 thru 0x5ce71fc (0xf3fd bytes) 2024-09-21T21:41:02.992 INFO:teuthology.orchestra.run.smithi164.stdout:1283 write 0x5942c00 thru 0x594368f (0xa90 bytes) 2024-09-21T21:41:02.992 INFO:teuthology.orchestra.run.smithi164.stdout:1284 punch from 0xadc00 to 0xafe00, (0x2200 bytes) 2024-09-21T21:41:02.994 INFO:teuthology.orchestra.run.smithi164.stdout:1285 punch from 0x4e34400 to 0x4e43c00, (0xf800 bytes) 2024-09-21T21:41:02.994 INFO:teuthology.orchestra.run.smithi164.stdout:1289 punch from 0x355c200 to 0x3568800, (0xc600 bytes) 2024-09-21T21:41:02.996 INFO:teuthology.orchestra.run.smithi164.stdout:1293 trunc from 0x5ce7200 to 0x436aa00 2024-09-21T21:41:03.026 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T21:41:03.027 INFO:teuthology.orchestra.run.smithi164.stdout:1294 read 0x26b6400 thru 0x26b92f1 (0x2ef2 bytes) 2024-09-21T21:41:03.028 INFO:teuthology.orchestra.run.smithi164.stdout:1295 punch from 0x3ae7a00 to 0x3af0000, (0x8600 bytes) 2024-09-21T21:41:03.031 INFO:teuthology.orchestra.run.smithi164.stdout:1297 trunc from 0x436aa00 to 0x2ad0c00 2024-09-21T21:41:03.057 INFO:teuthology.orchestra.run.smithi164.stdout:1298 read 0x3a0000 thru 0x3a5b56 (0x5b57 bytes) 2024-09-21T21:41:03.059 INFO:teuthology.orchestra.run.smithi164.stdout:1299 read 0x15c1200 thru 0x15cd771 (0xc572 bytes) 2024-09-21T21:41:03.060 INFO:teuthology.orchestra.run.smithi164.stdout:1300 read 0xdd4c00 thru 0xddec84 (0xa085 bytes) 2024-09-21T21:41:03.060 INFO:teuthology.orchestra.run.smithi164.stdout:1302 write 0x857000 thru 0x85cd61 (0x5d62 bytes) 2024-09-21T21:41:03.069 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:03.067+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:03.081 INFO:teuthology.orchestra.run.smithi164.stdout:1303 write 0xc4ebc00 thru 0xc4ee31e (0x271f bytes) 2024-09-21T21:41:03.092 INFO:teuthology.orchestra.run.smithi164.stdout:1305 trunc from 0xc4ee400 to 0x849be00 2024-09-21T21:41:03.105 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:03.105 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:41:03.105 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:41:03.105 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:41:03.105 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:03.108 INFO:teuthology.orchestra.run.smithi164.stdout:1311 read 0x4412800 thru 0x441d8e7 (0xb0e8 bytes) 2024-09-21T21:41:03.114 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T21:41:03.116 INFO:teuthology.orchestra.run.smithi164.stdout:1312 trunc from 0x849be00 to 0x9218800 2024-09-21T21:41:03.124 INFO:teuthology.orchestra.run.smithi164.stdout:1313 read 0x370f600 thru 0x3711680 (0x2081 bytes) 2024-09-21T21:41:03.131 INFO:teuthology.orchestra.run.smithi164.stdout:1314 read 0x34b1600 thru 0x34b4c1c (0x361d bytes) 2024-09-21T21:41:03.145 INFO:teuthology.orchestra.run.smithi164.stdout:1315 write 0xdbf2e00 thru 0xdbfbdc3 (0x8fc4 bytes) 2024-09-21T21:41:03.145 INFO:teuthology.orchestra.run.smithi164.stdout:1316 write 0xb8aa200 thru 0xb8b53e4 (0xb1e5 bytes) 2024-09-21T21:41:03.150 INFO:teuthology.orchestra.run.smithi164.stdout:1318 trunc from 0xdbfbe00 to 0x9c31a00 2024-09-21T21:41:03.153 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0: {} 2024-09-21T21:41:03.153 INFO:teuthology.orchestra.run.smithi081.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T21:41:03.168 INFO:teuthology.orchestra.run.smithi164.stdout:1319 write 0x566e00 thru 0x5700ec (0x92ed bytes) 2024-09-21T21:41:03.169 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:03.168+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:03.170 INFO:teuthology.orchestra.run.smithi164.stdout:1321 write 0x4ac7400 thru 0x4aca870 (0x3471 bytes) 2024-09-21T21:41:03.171 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1: {} 2024-09-21T21:41:03.171 INFO:teuthology.orchestra.run.smithi081.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T21:41:03.184 INFO:teuthology.orchestra.run.smithi164.stdout:1322 write 0xc1b4200 thru 0xc1b5243 (0x1044 bytes) 2024-09-21T21:41:03.185 INFO:teuthology.orchestra.run.smithi164.stdout:1323 write 0xd7fa00 thru 0xd8e05e (0xe65f bytes) 2024-09-21T21:41:03.188 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2: {} 2024-09-21T21:41:03.188 INFO:teuthology.orchestra.run.smithi081.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T21:41:03.189 INFO:teuthology.orchestra.run.smithi164.stdout:1327 read 0x6642400 thru 0x6643203 (0xe04 bytes) 2024-09-21T21:41:03.189 INFO:teuthology.orchestra.run.smithi164.stdout:1330 clone 11 order 25 su 1048576 sc 12 2024-09-21T21:41:03.207 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3: {} 2024-09-21T21:41:03.207 INFO:teuthology.orchestra.run.smithi081.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T21:41:03.223 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4: {} 2024-09-21T21:41:03.223 INFO:teuthology.orchestra.run.smithi081.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T21:41:03.239 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5: {} 2024-09-21T21:41:03.239 INFO:teuthology.orchestra.run.smithi081.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T21:41:03.263 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:03.263 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:03.270 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:03.268+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:03.271 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T21:41:03.371 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:03.370+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:03.414 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:03.415 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:03.423 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T21:41:03.472 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:03.471+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:03.541 INFO:teuthology.orchestra.run.smithi081.stderr:nodeep-scrub is set 2024-09-21T21:41:03.547 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-21T21:41:03.547 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T21:41:03.547 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-21T21:41:03.547 INFO:tasks.thrashosds.thrasher:Setting osd 5 primary_affinity to 0.328062 2024-09-21T21:41:03.547 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 0.3280622549535088 2024-09-21T21:41:03.568 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:03.568 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:03.573 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:03.572+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:03.577 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T21:41:03.635 INFO:teuthology.orchestra.run.smithi164.stdout:truncating image image_client.0-clone10 from 0xc1b5400 (overlap 0x2ad0c00) to 0xcd8200 2024-09-21T21:41:03.673 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:03.672+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:03.734 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:03.734 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:03.743 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T21:41:03.787 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:03.786+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:03.848 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:03.849 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:03.858 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T21:41:03.875 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:03.873+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:03.975 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:03.974+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:04.001 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:04.001 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:04.010 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T21:41:04.094 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:04.093+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:04.158 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:04.158 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:04.167 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T21:41:04.199 INFO:teuthology.orchestra.run.smithi164.stdout:checking clone #9, image image_client.0-clone9 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent10 2024-09-21T21:41:04.207 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:04.206+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:04.255 INFO:teuthology.orchestra.run.smithi164.stdout:1333 write 0xa72ee00 thru 0xa739bdb (0xaddc bytes) 2024-09-21T21:41:04.256 INFO:teuthology.orchestra.run.smithi164.stdout:1334 write 0x8338000 thru 0x8339da7 (0x1da8 bytes) 2024-09-21T21:41:04.256 INFO:teuthology.orchestra.run.smithi164.stdout:1335 punch from 0xc07de00 to 0xc083e00, (0x6000 bytes) 2024-09-21T21:41:04.278 INFO:teuthology.orchestra.run.smithi164.stdout:1338 read 0x44d2600 thru 0x44db2e2 (0x8ce3 bytes) 2024-09-21T21:41:04.278 INFO:teuthology.orchestra.run.smithi164.stdout:1339 punch from 0x7e25400 to 0x7e2e000, (0x8c00 bytes) 2024-09-21T21:41:04.279 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:04.277+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:04.283 INFO:teuthology.orchestra.run.smithi164.stdout:1341 write 0x924d800 thru 0x925aea0 (0xd6a1 bytes) 2024-09-21T21:41:04.283 INFO:teuthology.orchestra.run.smithi164.stdout:1342 read 0x260f000 thru 0x2615ff8 (0x6ff9 bytes) 2024-09-21T21:41:04.284 INFO:teuthology.orchestra.run.smithi164.stdout:1343 read 0x9bcc000 thru 0x9bcc6ae (0x6af bytes) 2024-09-21T21:41:04.284 INFO:teuthology.orchestra.run.smithi164.stdout:1344 write 0x4c72e00 thru 0x4c829fa (0xfbfb bytes) 2024-09-21T21:41:04.284 INFO:teuthology.orchestra.run.smithi164.stdout:1345 punch from 0x90b5200 to 0x90bac00, (0x5a00 bytes) 2024-09-21T21:41:04.308 INFO:teuthology.orchestra.run.smithi164.stdout:1346 write 0x4f1e000 thru 0x4f22315 (0x4316 bytes) 2024-09-21T21:41:04.308 INFO:teuthology.orchestra.run.smithi164.stdout:1347 read 0x1543e00 thru 0x15470e8 (0x32e9 bytes) 2024-09-21T21:41:04.309 INFO:teuthology.orchestra.run.smithi164.stdout:1349 read 0xb1dd000 thru 0xb1ecbe5 (0xfbe6 bytes) 2024-09-21T21:41:04.309 INFO:teuthology.orchestra.run.smithi164.stdout:1351 read 0x6df3400 thru 0x6dfdb41 (0xa742 bytes) 2024-09-21T21:41:04.314 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:04.314 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:04.322 INFO:teuthology.orchestra.run.smithi164.stdout:1352 trunc from 0xc1b5400 to 0xd98800 2024-09-21T21:41:04.324 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T21:41:04.379 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:04.378+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:04.382 INFO:teuthology.orchestra.run.smithi164.stdout:1353 read 0x6ee400 thru 0x6f6221 (0x7e22 bytes) 2024-09-21T21:41:04.408 INFO:teuthology.orchestra.run.smithi164.stdout:1354 write 0xeb7f800 thru 0xeb825c2 (0x2dc3 bytes) 2024-09-21T21:41:04.409 INFO:teuthology.orchestra.run.smithi164.stdout:1355 write 0x36f5a00 thru 0x36f9822 (0x3e23 bytes) 2024-09-21T21:41:04.409 INFO:teuthology.orchestra.run.smithi164.stdout:1361 punch from 0x7506400 to 0x750a800, (0x4400 bytes) 2024-09-21T21:41:04.410 INFO:teuthology.orchestra.run.smithi164.stdout:1362 punch from 0x1677000 to 0x167d800, (0x6800 bytes) 2024-09-21T21:41:04.410 INFO:teuthology.orchestra.run.smithi164.stdout:1363 write 0x51cec00 thru 0x51d5376 (0x6777 bytes) 2024-09-21T21:41:04.429 INFO:teuthology.orchestra.run.smithi164.stdout:1365 trunc from 0xeb82600 to 0x33de00 2024-09-21T21:41:04.433 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:04.433 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:04.443 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T21:41:04.480 INFO:teuthology.orchestra.run.smithi164.stdout:1368 read 0x266400 thru 0x268a88 (0x2689 bytes) 2024-09-21T21:41:04.481 INFO:teuthology.orchestra.run.smithi164.stdout:1371 punch from 0x26be00 to 0x26fe00, (0x4000 bytes) 2024-09-21T21:41:04.482 INFO:teuthology.orchestra.run.smithi164.stdout:1372 read 0xa3e00 thru 0xae052 (0xa253 bytes) 2024-09-21T21:41:04.484 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:04.483+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:04.492 INFO:teuthology.orchestra.run.smithi164.stdout:1373 write 0xc80400 thru 0xc8821b (0x7e1c bytes) 2024-09-21T21:41:04.494 INFO:teuthology.orchestra.run.smithi164.stdout:1374 punch from 0x840400 to 0x84fe00, (0xfa00 bytes) 2024-09-21T21:41:04.494 INFO:teuthology.orchestra.run.smithi164.stdout:1376 punch from 0x581c00 to 0x58a400, (0x8800 bytes) 2024-09-21T21:41:04.495 INFO:teuthology.orchestra.run.smithi164.stdout:1378 read 0x60e600 thru 0x61061a (0x201b bytes) 2024-09-21T21:41:04.495 INFO:teuthology.orchestra.run.smithi164.stdout:1381 read 0xb47e00 thru 0xb4eaa5 (0x6ca6 bytes) 2024-09-21T21:41:04.509 INFO:teuthology.orchestra.run.smithi164.stdout:1382 write 0x4d54400 thru 0x4d5841e (0x401f bytes) 2024-09-21T21:41:04.522 INFO:teuthology.orchestra.run.smithi164.stdout:1385 write 0x751b400 thru 0x75207a5 (0x53a6 bytes) 2024-09-21T21:41:04.523 INFO:teuthology.orchestra.run.smithi164.stdout:1386 read 0x70ca800 thru 0x70d92ae (0xeaaf bytes) 2024-09-21T21:41:04.523 INFO:teuthology.orchestra.run.smithi164.stdout:1387 read 0x2a6ac00 thru 0x2a738ce (0x8ccf bytes) 2024-09-21T21:41:04.531 INFO:teuthology.orchestra.run.smithi164.stdout:1388 trunc from 0x7520800 to 0x121200 2024-09-21T21:41:04.545 INFO:teuthology.orchestra.run.smithi081.stderr:set osd.5 primary-affinity to 0.328062 (8214992) 2024-09-21T21:41:04.582 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:04.580+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:04.595 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:04.595 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:04.604 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T21:41:04.682 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:04.681+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:04.748 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:04.748 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:04.756 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T21:41:04.783 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:04.782+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:04.858 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:04.858 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:04.866 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T21:41:04.909 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:04.908+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:04.969 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:04.969 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:04.978 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T21:41:05.021 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:05.020+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:05.081 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:05.081 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:05.085 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:05.085+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:05.090 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T21:41:05.187 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:05.186+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:05.232 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:05.232 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:05.240 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T21:41:05.251 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T21:41:05.288 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:05.286+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:05.343 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:05.343 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:05.352 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T21:41:05.370 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0: {} 2024-09-21T21:41:05.370 INFO:teuthology.orchestra.run.smithi081.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T21:41:05.386 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1: {} 2024-09-21T21:41:05.386 INFO:teuthology.orchestra.run.smithi081.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T21:41:05.393 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:05.391+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:05.402 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2: {} 2024-09-21T21:41:05.402 INFO:teuthology.orchestra.run.smithi081.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T21:41:05.418 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3: {} 2024-09-21T21:41:05.418 INFO:teuthology.orchestra.run.smithi081.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T21:41:05.434 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4: {} 2024-09-21T21:41:05.435 INFO:teuthology.orchestra.run.smithi081.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T21:41:05.451 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5: {} 2024-09-21T21:41:05.451 INFO:teuthology.orchestra.run.smithi081.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T21:41:05.489 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:05.487+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:05.495 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:05.495 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:41:05.495 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-21T21:41:05.495 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:05.504 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T21:41:05.558 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T21:41:05.574 INFO:teuthology.orchestra.run.smithi164.stdout:1391 read 0xa7c00 thru 0xb5c56 (0xe057 bytes) 2024-09-21T21:41:05.604 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:05.602+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:05.609 INFO:teuthology.orchestra.run.smithi164.stdout:1394 write 0xe4c4200 thru 0xe4c820b (0x400c bytes) 2024-09-21T21:41:05.612 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:05.613 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:41:05.613 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:41:05.613 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:41:05.613 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:05.614 INFO:teuthology.orchestra.run.smithi164.stdout:1395 read 0x7ab0e00 thru 0x7ab19c8 (0xbc9 bytes) 2024-09-21T21:41:05.621 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T21:41:05.623 INFO:teuthology.orchestra.run.smithi164.stdout:1396 trunc from 0xe4c8400 to 0x6120e00 2024-09-21T21:41:05.635 INFO:teuthology.orchestra.run.smithi164.stdout:1397 read 0x4ee6e00 thru 0x4ef5ab2 (0xecb3 bytes) 2024-09-21T21:41:05.645 INFO:teuthology.orchestra.run.smithi164.stdout:1398 read 0x581f400 thru 0x582a258 (0xae59 bytes) 2024-09-21T21:41:05.662 INFO:teuthology.orchestra.run.smithi164.stdout:1400 write 0x7d4e000 thru 0x7d538ff (0x5900 bytes) 2024-09-21T21:41:05.663 INFO:teuthology.orchestra.run.smithi164.stdout:1401 read 0x4ad2e00 thru 0x4ad9e8b (0x708c bytes) 2024-09-21T21:41:05.663 INFO:teuthology.orchestra.run.smithi164.stdout:1404 trunc from 0x7d53a00 to 0x7707000 2024-09-21T21:41:05.682 INFO:teuthology.orchestra.run.smithi164.stdout:1405 punch from 0x49dba00 to 0x49e2800, (0x6e00 bytes) 2024-09-21T21:41:05.684 INFO:teuthology.orchestra.run.smithi164.stdout:1406 write 0x3681c00 thru 0x368c3bb (0xa7bc bytes) 2024-09-21T21:41:05.684 INFO:teuthology.orchestra.run.smithi164.stdout:1407 read 0x7392600 thru 0x7397937 (0x5338 bytes) 2024-09-21T21:41:05.684 INFO:teuthology.orchestra.run.smithi164.stdout:1409 write 0x6c34c00 thru 0x6c37bbc (0x2fbd bytes) 2024-09-21T21:41:05.685 INFO:teuthology.orchestra.run.smithi164.stdout:1411 read 0x4dfc00 thru 0x4e7ab4 (0x7eb5 bytes) 2024-09-21T21:41:05.690 INFO:teuthology.orchestra.run.smithi164.stdout:1414 trunc from 0x7707000 to 0xc184a00 2024-09-21T21:41:05.691 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:05.689+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:05.709 INFO:teuthology.orchestra.run.smithi164.stdout:1415 trunc from 0xc184a00 to 0xb513c00 2024-09-21T21:41:05.726 INFO:teuthology.orchestra.run.smithi164.stdout:1417 read 0x66d9e00 thru 0x66e4e22 (0xb023 bytes) 2024-09-21T21:41:05.726 INFO:teuthology.orchestra.run.smithi164.stdout:1421 read 0x9fc1400 thru 0x9fc1ca5 (0x8a6 bytes) 2024-09-21T21:41:05.726 INFO:teuthology.orchestra.run.smithi164.stdout:1422 read 0x2508a00 thru 0x2509fcc (0x15cd bytes) 2024-09-21T21:41:05.727 INFO:teuthology.orchestra.run.smithi164.stdout:1424 write 0x747c000 thru 0x74896cf (0xd6d0 bytes) 2024-09-21T21:41:05.727 INFO:teuthology.orchestra.run.smithi164.stdout:1426 read 0x2d0b800 thru 0x2d0f737 (0x3f38 bytes) 2024-09-21T21:41:05.727 INFO:teuthology.orchestra.run.smithi164.stdout:1428 write 0x75eb200 thru 0x75edacb (0x28cc bytes) 2024-09-21T21:41:05.728 INFO:teuthology.orchestra.run.smithi164.stdout:1431 read 0x903400 thru 0x90714f (0x3d50 bytes) 2024-09-21T21:41:05.728 INFO:teuthology.orchestra.run.smithi164.stdout:1432 read 0x2ea9e00 thru 0x2eaabfd (0xdfe bytes) 2024-09-21T21:41:05.731 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:05.731 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-21T21:41:05.731 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-21T21:41:05.731 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:41:05.731 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.732 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:8196 3.3 3:c030d67f:::rbd_data.112145fae779.0000000000000003:a [read 9502720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:41:05.732 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.688149+0000", 2024-09-21T21:41:05.732 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 7.0409658009999996, 2024-09-21T21:41:05.732 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.018375909999999999, 2024-09-21T21:41:05.732 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:05.732 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:05.732 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:05.732 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:05.732 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:05.732 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:05.732 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 8196 2024-09-21T21:41:05.733 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.733 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:05.733 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.733 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:05.733 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688149+0000", 2024-09-21T21:41:05.733 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:05.733 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.733 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.733 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:05.733 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688149+0000", 2024-09-21T21:41:05.733 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:05.733 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.733 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.733 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:05.734 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688150+0000", 2024-09-21T21:41:05.734 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5900000000000003e-07 2024-09-21T21:41:05.734 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.734 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.734 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:05.734 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688156+0000", 2024-09-21T21:41:05.734 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3160000000000002e-06 2024-09-21T21:41:05.734 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.734 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.734 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:05.734 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688163+0000", 2024-09-21T21:41:05.734 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7920000000000004e-06 2024-09-21T21:41:05.734 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.735 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.735 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:05.735 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688167+0000", 2024-09-21T21:41:05.735 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4360000000000004e-06 2024-09-21T21:41:05.735 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.735 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.735 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:05.735 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.706376+0000", 2024-09-21T21:41:05.735 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.018208762 2024-09-21T21:41:05.735 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.736 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.736 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:05.736 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.706456+0000", 2024-09-21T21:41:05.736 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0065e-05 2024-09-21T21:41:05.737 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.737 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.737 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:05.737 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.706525+0000", 2024-09-21T21:41:05.738 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9179999999999998e-05 2024-09-21T21:41:05.738 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:05.738 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:05.738 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:05.738 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.738 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.739 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:8197 3.3 3:c030d67f:::rbd_data.112145fae779.0000000000000003:a [read 9560064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:41:05.739 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.688173+0000", 2024-09-21T21:41:05.739 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 7.0409423679999996, 2024-09-21T21:41:05.739 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.018539605000000001, 2024-09-21T21:41:05.739 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:05.739 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:05.740 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:05.740 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:05.740 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:05.740 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:05.740 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 8197 2024-09-21T21:41:05.740 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.741 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:05.741 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.741 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:05.741 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688173+0000", 2024-09-21T21:41:05.741 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:05.741 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.742 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.742 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:05.742 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688173+0000", 2024-09-21T21:41:05.742 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:05.742 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.742 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.743 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:05.743 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688173+0000", 2024-09-21T21:41:05.743 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2300000000000002e-07 2024-09-21T21:41:05.743 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.743 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.743 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:05.744 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688175+0000", 2024-09-21T21:41:05.744 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9589999999999998e-06 2024-09-21T21:41:05.744 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.744 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.744 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:05.744 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688182+0000", 2024-09-21T21:41:05.745 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4239999999999998e-06 2024-09-21T21:41:05.745 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.745 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.745 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:05.745 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688186+0000", 2024-09-21T21:41:05.745 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4939999999999997e-06 2024-09-21T21:41:05.746 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.746 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.746 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:05.746 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.706559+0000", 2024-09-21T21:41:05.746 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.018373308000000001 2024-09-21T21:41:05.747 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.747 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.747 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:05.747 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.706642+0000", 2024-09-21T21:41:05.747 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2964000000000001e-05 2024-09-21T21:41:05.748 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.748 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.748 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:05.748 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.706712+0000", 2024-09-21T21:41:05.748 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0032999999999998e-05 2024-09-21T21:41:05.748 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:05.749 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:05.749 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:05.749 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.749 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.749 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:8317 3.3 3:c030d67f:::rbd_data.112145fae779.0000000000000003:a [read 9064448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:41:05.749 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.688286+0000", 2024-09-21T21:41:05.749 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 7.0408286970000002, 2024-09-21T21:41:05.750 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.018621219000000001, 2024-09-21T21:41:05.750 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:05.750 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:05.750 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:05.750 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:05.751 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:05.751 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:05.751 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 8317 2024-09-21T21:41:05.751 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.751 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:05.752 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.752 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:05.752 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688286+0000", 2024-09-21T21:41:05.752 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:05.752 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.752 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.753 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:05.753 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688286+0000", 2024-09-21T21:41:05.753 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:05.753 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.754 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.754 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:05.754 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688287+0000", 2024-09-21T21:41:05.754 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6499999999999999e-07 2024-09-21T21:41:05.754 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.754 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.755 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:05.755 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688290+0000", 2024-09-21T21:41:05.755 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1879999999999998e-06 2024-09-21T21:41:05.755 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.755 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.756 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:05.756 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688317+0000", 2024-09-21T21:41:05.756 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7353999999999999e-05 2024-09-21T21:41:05.756 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.756 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.756 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:05.757 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688324+0000", 2024-09-21T21:41:05.757 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2910000000000004e-06 2024-09-21T21:41:05.757 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.757 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.758 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:05.758 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.706744+0000", 2024-09-21T21:41:05.758 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.018420414 2024-09-21T21:41:05.758 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.758 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.758 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:05.759 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.706826+0000", 2024-09-21T21:41:05.759 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2286999999999993e-05 2024-09-21T21:41:05.759 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.759 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.759 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:05.759 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.706908+0000", 2024-09-21T21:41:05.760 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1119999999999996e-05 2024-09-21T21:41:05.760 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:05.760 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:05.760 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:05.760 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.760 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.761 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:8318 3.3 3:c030d67f:::rbd_data.112145fae779.0000000000000003:a [read 9121792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:41:05.761 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.688335+0000", 2024-09-21T21:41:05.761 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 7.0407799840000003, 2024-09-21T21:41:05.761 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.018752181999999999, 2024-09-21T21:41:05.761 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:05.762 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:05.762 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:05.762 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:05.762 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:05.763 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:05.763 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 8318 2024-09-21T21:41:05.763 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.763 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:05.763 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.764 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:05.764 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688335+0000", 2024-09-21T21:41:05.764 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:05.764 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.764 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.764 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:05.765 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688335+0000", 2024-09-21T21:41:05.765 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:05.765 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.765 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.765 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:05.765 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688336+0000", 2024-09-21T21:41:05.766 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5000000000000003e-07 2024-09-21T21:41:05.766 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.766 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.766 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:05.766 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688339+0000", 2024-09-21T21:41:05.766 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.128e-06 2024-09-21T21:41:05.767 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.767 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.767 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:05.767 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688347+0000", 2024-09-21T21:41:05.767 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5920000000000001e-06 2024-09-21T21:41:05.767 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.768 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.768 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:05.768 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688353+0000", 2024-09-21T21:41:05.768 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6570000000000002e-06 2024-09-21T21:41:05.768 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.768 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.769 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:05.769 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.706940+0000", 2024-09-21T21:41:05.769 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.018586689 2024-09-21T21:41:05.769 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.769 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.769 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:05.770 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.707015+0000", 2024-09-21T21:41:05.770 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4888999999999994e-05 2024-09-21T21:41:05.770 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.770 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.770 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:05.770 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.707087+0000", 2024-09-21T21:41:05.771 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2676999999999996e-05 2024-09-21T21:41:05.771 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:05.771 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:05.771 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:05.771 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.771 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.772 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:8319 3.3 3:c030d67f:::rbd_data.112145fae779.0000000000000003:a [read 9211904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:41:05.772 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.688360+0000", 2024-09-21T21:41:05.772 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 7.0407547050000003, 2024-09-21T21:41:05.772 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.018899836, 2024-09-21T21:41:05.772 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:05.772 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:05.773 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:05.773 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:05.773 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:05.773 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:05.773 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 8319 2024-09-21T21:41:05.773 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.774 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:05.774 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.774 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:05.774 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688360+0000", 2024-09-21T21:41:05.774 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:05.774 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.775 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.775 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:05.775 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688360+0000", 2024-09-21T21:41:05.775 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:05.775 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.775 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.776 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:05.776 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688361+0000", 2024-09-21T21:41:05.777 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5499999999999998e-07 2024-09-21T21:41:05.778 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.778 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.778 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:05.778 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688363+0000", 2024-09-21T21:41:05.779 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.21e-06 2024-09-21T21:41:05.779 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.779 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.779 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:05.779 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688371+0000", 2024-09-21T21:41:05.779 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2539999999999992e-06 2024-09-21T21:41:05.780 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.780 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.780 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:05.780 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688377+0000", 2024-09-21T21:41:05.780 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.468e-06 2024-09-21T21:41:05.780 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.781 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.781 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:05.781 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.707121+0000", 2024-09-21T21:41:05.781 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.018743774000000001 2024-09-21T21:41:05.781 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.782 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.782 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:05.782 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.707195+0000", 2024-09-21T21:41:05.782 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4745000000000003e-05 2024-09-21T21:41:05.782 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.782 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.782 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:05.783 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.707260+0000", 2024-09-21T21:41:05.783 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4930000000000003e-05 2024-09-21T21:41:05.783 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:05.783 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:05.783 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:05.784 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.784 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.784 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:8320 3.3 3:c030d67f:::rbd_data.112145fae779.0000000000000003:a [read 9269248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:41:05.784 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.688383+0000", 2024-09-21T21:41:05.784 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 7.0407316050000004, 2024-09-21T21:41:05.784 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.019040664999999998, 2024-09-21T21:41:05.785 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:05.785 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:05.785 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:05.785 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:05.785 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:05.785 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:05.786 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 8320 2024-09-21T21:41:05.786 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.786 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:05.786 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.786 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:05.787 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688383+0000", 2024-09-21T21:41:05.787 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:05.787 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.787 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.787 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:05.787 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688383+0000", 2024-09-21T21:41:05.787 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:05.788 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.788 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.788 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:05.788 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688384+0000", 2024-09-21T21:41:05.788 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.27e-07 2024-09-21T21:41:05.788 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.788 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.789 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:05.789 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688386+0000", 2024-09-21T21:41:05.789 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2299999999999998e-06 2024-09-21T21:41:05.789 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.789 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.789 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:05.789 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688395+0000", 2024-09-21T21:41:05.789 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.0410000000000003e-06 2024-09-21T21:41:05.789 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.789 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.789 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:05.789 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688400+0000", 2024-09-21T21:41:05.790 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.198e-06 2024-09-21T21:41:05.790 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.790 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.790 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:05.790 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.707289+0000", 2024-09-21T21:41:05.790 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.018888820000000001 2024-09-21T21:41:05.790 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.790 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.790 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:05.790 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.707358+0000", 2024-09-21T21:41:05.790 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9219000000000003e-05 2024-09-21T21:41:05.791 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.791 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.791 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:05.791 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.707424+0000", 2024-09-21T21:41:05.791 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5729999999999996e-05 2024-09-21T21:41:05.791 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:05.791 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:05.791 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:05.791 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.791 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.791 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:8321 3.3 3:c030d67f:::rbd_data.112145fae779.0000000000000003:a [read 9359360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:41:05.791 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.688407+0000", 2024-09-21T21:41:05.791 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 7.0407084170000003, 2024-09-21T21:41:05.791 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.019217557999999999, 2024-09-21T21:41:05.792 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:05.792 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:05.792 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:05.792 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:05.792 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:05.792 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:05.792 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 8321 2024-09-21T21:41:05.792 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.792 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:05.792 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.792 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:05.792 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688407+0000", 2024-09-21T21:41:05.792 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:05.792 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.793 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.793 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:05.793 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688407+0000", 2024-09-21T21:41:05.793 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:05.793 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.793 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.793 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:05.793 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688407+0000", 2024-09-21T21:41:05.793 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2500000000000001e-07 2024-09-21T21:41:05.793 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.793 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.793 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:05.793 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688409+0000", 2024-09-21T21:41:05.793 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1299999999999999e-06 2024-09-21T21:41:05.793 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.794 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.794 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:05.794 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688416+0000", 2024-09-21T21:41:05.794 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9539999999999998e-06 2024-09-21T21:41:05.794 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.794 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.794 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:05.794 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688421+0000", 2024-09-21T21:41:05.794 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2610000000000001e-06 2024-09-21T21:41:05.794 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.794 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.794 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:05.794 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.707456+0000", 2024-09-21T21:41:05.794 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.019035024000000001 2024-09-21T21:41:05.795 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.795 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.795 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:05.795 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.707550+0000", 2024-09-21T21:41:05.795 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3696000000000006e-05 2024-09-21T21:41:05.795 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.795 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.795 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:05.795 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.707624+0000", 2024-09-21T21:41:05.795 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4067999999999996e-05 2024-09-21T21:41:05.795 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:05.795 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:05.795 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:05.795 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.795 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.796 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:8322 3.3 3:c030d67f:::rbd_data.112145fae779.0000000000000003:a [read 9416704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:41:05.796 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.688427+0000", 2024-09-21T21:41:05.796 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 7.040687599, 2024-09-21T21:41:05.796 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.019393840999999998, 2024-09-21T21:41:05.796 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:05.796 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:05.796 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:05.796 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:05.796 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:05.796 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:05.796 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 8322 2024-09-21T21:41:05.796 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.796 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:05.796 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.796 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:05.797 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688427+0000", 2024-09-21T21:41:05.797 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:05.797 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.797 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.797 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:05.797 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688427+0000", 2024-09-21T21:41:05.797 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:05.797 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.797 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.797 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:05.797 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688428+0000", 2024-09-21T21:41:05.797 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7e-07 2024-09-21T21:41:05.797 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.797 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.797 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:05.798 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688430+0000", 2024-09-21T21:41:05.798 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2189999999999998e-06 2024-09-21T21:41:05.798 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.798 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.798 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:05.798 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688436+0000", 2024-09-21T21:41:05.798 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4309999999999999e-06 2024-09-21T21:41:05.798 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.798 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.798 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:05.798 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688441+0000", 2024-09-21T21:41:05.798 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7029999999999997e-06 2024-09-21T21:41:05.798 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.798 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.799 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:05.799 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.707659+0000", 2024-09-21T21:41:05.799 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.019217998999999999 2024-09-21T21:41:05.799 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.799 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.799 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:05.799 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.707749+0000", 2024-09-21T21:41:05.799 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.0229999999999995e-05 2024-09-21T21:41:05.799 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.799 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.799 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:05.799 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.707821+0000", 2024-09-21T21:41:05.799 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1889000000000002e-05 2024-09-21T21:41:05.799 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:05.800 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:05.801 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:05.801 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.801 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.801 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:8323 3.3 3:c030d67f:::rbd_data.112145fae779.0000000000000003:a [read 9449472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:41:05.801 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.688451+0000", 2024-09-21T21:41:05.801 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 7.0406639789999996, 2024-09-21T21:41:05.801 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.019572622000000001, 2024-09-21T21:41:05.801 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:05.801 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:05.801 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:05.801 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:05.801 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:05.801 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:05.802 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 8323 2024-09-21T21:41:05.802 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.802 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:05.802 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.802 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:05.802 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688451+0000", 2024-09-21T21:41:05.802 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:05.802 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.802 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.802 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:05.802 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688451+0000", 2024-09-21T21:41:05.802 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:05.802 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.802 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.802 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:05.803 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688452+0000", 2024-09-21T21:41:05.803 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.39e-07 2024-09-21T21:41:05.803 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.803 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.803 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:05.803 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688454+0000", 2024-09-21T21:41:05.803 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.272e-06 2024-09-21T21:41:05.803 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.803 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.803 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:05.803 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688460+0000", 2024-09-21T21:41:05.803 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5270000000000004e-06 2024-09-21T21:41:05.803 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.803 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.804 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:05.804 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688466+0000", 2024-09-21T21:41:05.804 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1909999999999999e-06 2024-09-21T21:41:05.804 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.804 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.804 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:05.804 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.707854+0000", 2024-09-21T21:41:05.804 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.019388216999999999 2024-09-21T21:41:05.804 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.804 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.804 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:05.804 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.707952+0000", 2024-09-21T21:41:05.804 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.8136999999999999e-05 2024-09-21T21:41:05.804 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.805 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.805 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:05.805 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.708024+0000", 2024-09-21T21:41:05.805 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1839000000000001e-05 2024-09-21T21:41:05.805 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:05.805 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:05.805 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:05.805 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.805 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.805 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:8324 3.3 3:c030d67f:::rbd_data.112145fae779.0000000000000003:a [read 9506816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:41:05.805 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.688472+0000", 2024-09-21T21:41:05.805 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 7.040642847, 2024-09-21T21:41:05.805 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.019741218000000001, 2024-09-21T21:41:05.805 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:05.805 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:05.806 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:05.806 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:05.806 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:05.806 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:05.806 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 8324 2024-09-21T21:41:05.806 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.806 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:05.806 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.806 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:05.806 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688472+0000", 2024-09-21T21:41:05.806 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:05.806 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.806 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.806 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:05.806 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688472+0000", 2024-09-21T21:41:05.807 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:05.807 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.807 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.807 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:05.807 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688473+0000", 2024-09-21T21:41:05.807 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4299999999999999e-07 2024-09-21T21:41:05.807 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.807 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.807 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:05.807 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688475+0000", 2024-09-21T21:41:05.807 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1409999999999999e-06 2024-09-21T21:41:05.807 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.807 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.807 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:05.808 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688481+0000", 2024-09-21T21:41:05.808 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5529999999999998e-06 2024-09-21T21:41:05.808 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.808 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.808 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:05.808 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688486+0000", 2024-09-21T21:41:05.808 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9459999999999997e-06 2024-09-21T21:41:05.808 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.808 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.808 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:05.808 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.708064+0000", 2024-09-21T21:41:05.808 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.019577721999999999 2024-09-21T21:41:05.809 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.809 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.809 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:05.809 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.708144+0000", 2024-09-21T21:41:05.809 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0554000000000002e-05 2024-09-21T21:41:05.809 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.809 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.809 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:05.809 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.708213+0000", 2024-09-21T21:41:05.809 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8959000000000002e-05 2024-09-21T21:41:05.809 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:05.809 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:05.809 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:05.809 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.809 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.810 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:8325 3.3 3:c030d67f:::rbd_data.112145fae779.0000000000000003:a [read 9564160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:41:05.810 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.688493+0000", 2024-09-21T21:41:05.810 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 7.040622087, 2024-09-21T21:41:05.810 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.019899284999999999, 2024-09-21T21:41:05.810 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:05.810 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:05.810 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:05.810 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:05.810 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:05.810 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:05.810 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 8325 2024-09-21T21:41:05.810 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.810 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:05.811 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.811 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:05.811 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688493+0000", 2024-09-21T21:41:05.811 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:05.811 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.811 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.811 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:05.811 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688493+0000", 2024-09-21T21:41:05.811 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:05.811 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.811 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.811 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:05.811 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688493+0000", 2024-09-21T21:41:05.811 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4999999999999998e-07 2024-09-21T21:41:05.812 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.812 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.812 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:05.812 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688496+0000", 2024-09-21T21:41:05.812 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0949999999999998e-06 2024-09-21T21:41:05.812 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.812 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.812 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:05.812 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688502+0000", 2024-09-21T21:41:05.812 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4409999999999998e-06 2024-09-21T21:41:05.812 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.812 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.812 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:05.812 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688507+0000", 2024-09-21T21:41:05.812 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6229999999999996e-06 2024-09-21T21:41:05.813 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.813 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.813 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:05.813 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.708246+0000", 2024-09-21T21:41:05.813 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.019739626 2024-09-21T21:41:05.813 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.813 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.813 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:05.813 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.708321+0000", 2024-09-21T21:41:05.813 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4803999999999997e-05 2024-09-21T21:41:05.813 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.813 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.813 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:05.813 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.708392+0000", 2024-09-21T21:41:05.814 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1246000000000002e-05 2024-09-21T21:41:05.814 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:05.814 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:05.814 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:05.814 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.814 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.814 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:8444 3.3 3:c030d67f:::rbd_data.112145fae779.0000000000000003:a [read 9068544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:41:05.814 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.689763+0000", 2024-09-21T21:41:05.814 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 7.0393525080000003, 2024-09-21T21:41:05.814 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.018821181999999999, 2024-09-21T21:41:05.814 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:05.814 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:05.815 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:05.815 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:05.815 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:05.815 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:05.815 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 8444 2024-09-21T21:41:05.815 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.815 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:05.815 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.815 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:05.815 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.689763+0000", 2024-09-21T21:41:05.815 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:05.815 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.815 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.815 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:05.816 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.689763+0000", 2024-09-21T21:41:05.816 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:05.816 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.816 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.816 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:05.816 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.689764+0000", 2024-09-21T21:41:05.816 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.9300000000000006e-07 2024-09-21T21:41:05.816 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.816 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.816 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:05.816 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.689769+0000", 2024-09-21T21:41:05.816 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.994e-06 2024-09-21T21:41:05.816 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.817 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.817 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:05.817 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.689781+0000", 2024-09-21T21:41:05.817 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.198e-05 2024-09-21T21:41:05.817 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.817 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.817 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:05.817 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.689789+0000", 2024-09-21T21:41:05.817 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2800000000000003e-06 2024-09-21T21:41:05.817 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.818 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.818 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:05.818 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.708426+0000", 2024-09-21T21:41:05.818 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.018637628 2024-09-21T21:41:05.818 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.818 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.818 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:05.818 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.708517+0000", 2024-09-21T21:41:05.818 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.0227000000000002e-05 2024-09-21T21:41:05.818 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.818 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.818 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:05.818 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.708584+0000", 2024-09-21T21:41:05.819 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7080000000000001e-05 2024-09-21T21:41:05.819 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:05.819 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:05.819 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:05.819 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.819 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.819 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:8445 3.3 3:c030d67f:::rbd_data.112145fae779.0000000000000003:a [read 9125888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:41:05.819 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.689842+0000", 2024-09-21T21:41:05.819 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 7.039273562, 2024-09-21T21:41:05.819 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.019029801999999998, 2024-09-21T21:41:05.819 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:05.819 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:05.819 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:05.819 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:05.819 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:05.820 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:05.820 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 8445 2024-09-21T21:41:05.820 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.820 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:05.820 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.820 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:05.820 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.689842+0000", 2024-09-21T21:41:05.820 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:05.820 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.820 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.820 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:05.820 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.689842+0000", 2024-09-21T21:41:05.820 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:05.820 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.821 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.821 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:05.821 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.689842+0000", 2024-09-21T21:41:05.821 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4399999999999999e-07 2024-09-21T21:41:05.821 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.821 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.821 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:05.821 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.689847+0000", 2024-09-21T21:41:05.821 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.746e-06 2024-09-21T21:41:05.821 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.822 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.822 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:05.822 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.689856+0000", 2024-09-21T21:41:05.822 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.2909999999999996e-06 2024-09-21T21:41:05.822 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.822 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.822 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:05.822 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.689868+0000", 2024-09-21T21:41:05.822 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1498000000000001e-05 2024-09-21T21:41:05.822 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.822 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.822 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:05.822 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.708618+0000", 2024-09-21T21:41:05.823 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.018750032 2024-09-21T21:41:05.823 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.823 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.823 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:05.823 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.708726+0000", 2024-09-21T21:41:05.823 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010836399999999999 2024-09-21T21:41:05.823 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.823 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.823 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:05.823 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.708871+0000", 2024-09-21T21:41:05.823 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014512699999999999 2024-09-21T21:41:05.823 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:05.823 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:05.823 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:05.824 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.824 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.824 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:8446 3.3 3:c030d67f:::rbd_data.112145fae779.0000000000000003:a [read 9216000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:41:05.824 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.689911+0000", 2024-09-21T21:41:05.824 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 7.0392042310000003, 2024-09-21T21:41:05.824 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.019300134, 2024-09-21T21:41:05.824 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:05.824 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:05.824 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:05.824 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:05.824 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:05.824 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:05.825 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 8446 2024-09-21T21:41:05.825 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.825 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:05.825 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.825 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:05.825 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.689911+0000", 2024-09-21T21:41:05.825 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:05.825 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.826 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.826 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:05.826 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.689911+0000", 2024-09-21T21:41:05.826 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:05.826 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.826 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.826 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:05.826 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.689912+0000", 2024-09-21T21:41:05.826 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8899999999999999e-07 2024-09-21T21:41:05.826 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.826 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.826 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:05.826 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.689915+0000", 2024-09-21T21:41:05.827 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1839999999999999e-06 2024-09-21T21:41:05.827 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.827 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.827 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:05.827 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.689924+0000", 2024-09-21T21:41:05.827 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.0170000000000002e-06 2024-09-21T21:41:05.827 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.827 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.827 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:05.827 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.689930+0000", 2024-09-21T21:41:05.827 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4930000000000003e-06 2024-09-21T21:41:05.827 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.827 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.827 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:05.827 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.708850+0000", 2024-09-21T21:41:05.828 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.018919703 2024-09-21T21:41:05.828 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.828 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.828 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:05.828 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.709104+0000", 2024-09-21T21:41:05.828 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00025365899999999999 2024-09-21T21:41:05.828 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.828 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.828 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:05.828 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.709211+0000", 2024-09-21T21:41:05.828 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000107389 2024-09-21T21:41:05.828 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:05.828 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:05.829 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:05.829 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.829 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.829 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:8447 3.3 3:c030d67f:::rbd_data.112145fae779.0000000000000003:a [read 9273344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:41:05.829 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.689970+0000", 2024-09-21T21:41:05.829 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 7.0391451460000001, 2024-09-21T21:41:05.829 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.019358220999999998, 2024-09-21T21:41:05.829 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:05.829 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:05.829 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:05.829 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:05.829 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:05.829 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:05.829 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 8447 2024-09-21T21:41:05.830 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.830 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:05.830 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.830 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:05.830 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.689970+0000", 2024-09-21T21:41:05.830 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:05.830 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.830 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.830 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:05.830 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.689970+0000", 2024-09-21T21:41:05.830 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:05.830 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.830 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.830 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:05.830 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.689971+0000", 2024-09-21T21:41:05.831 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3200000000000005e-07 2024-09-21T21:41:05.831 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.831 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.831 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:05.831 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.689974+0000", 2024-09-21T21:41:05.831 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1820000000000002e-06 2024-09-21T21:41:05.831 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.831 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.831 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:05.831 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.689982+0000", 2024-09-21T21:41:05.831 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7320000000000004e-06 2024-09-21T21:41:05.831 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.831 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.831 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:05.832 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.689989+0000", 2024-09-21T21:41:05.832 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2129999999999996e-06 2024-09-21T21:41:05.832 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.832 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.832 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:05.832 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.709241+0000", 2024-09-21T21:41:05.832 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.019252612999999998 2024-09-21T21:41:05.832 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.832 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.832 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:05.832 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.709285+0000", 2024-09-21T21:41:05.832 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3925999999999999e-05 2024-09-21T21:41:05.832 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.832 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.832 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:05.833 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.709328+0000", 2024-09-21T21:41:05.833 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2923000000000002e-05 2024-09-21T21:41:05.833 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:05.833 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:05.833 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:05.833 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.833 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.833 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:8448 3.3 3:c030d67f:::rbd_data.112145fae779.0000000000000003:a [read 9306112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:41:05.833 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.689998+0000", 2024-09-21T21:41:05.833 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 7.0391173120000001, 2024-09-21T21:41:05.833 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.019538304999999999, 2024-09-21T21:41:05.833 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:05.833 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:05.833 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:05.833 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:05.834 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:05.834 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:05.834 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 8448 2024-09-21T21:41:05.834 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.834 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:05.834 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.834 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:05.834 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.689998+0000", 2024-09-21T21:41:05.834 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:05.834 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.834 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.834 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:05.834 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.689998+0000", 2024-09-21T21:41:05.834 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:05.835 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.835 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.835 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:05.835 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.689998+0000", 2024-09-21T21:41:05.835 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4499999999999997e-07 2024-09-21T21:41:05.835 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.835 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.835 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:05.835 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.690001+0000", 2024-09-21T21:41:05.835 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5849999999999998e-06 2024-09-21T21:41:05.835 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.835 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.835 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:05.835 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.690007+0000", 2024-09-21T21:41:05.836 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4620000000000001e-06 2024-09-21T21:41:05.836 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.836 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.836 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:05.836 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.690012+0000", 2024-09-21T21:41:05.836 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9559999999999996e-06 2024-09-21T21:41:05.836 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.836 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.836 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:05.836 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.709359+0000", 2024-09-21T21:41:05.836 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.019346550000000001 2024-09-21T21:41:05.836 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.836 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.837 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:05.837 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.709435+0000", 2024-09-21T21:41:05.837 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6556000000000004e-05 2024-09-21T21:41:05.837 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.837 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.837 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:05.837 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.709536+0000", 2024-09-21T21:41:05.837 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000100751 2024-09-21T21:41:05.837 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:05.837 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:05.837 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:05.837 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.838 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.838 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:8449 3.3 3:c030d67f:::rbd_data.112145fae779.0000000000000003:a [read 9363456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:41:05.838 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.690019+0000", 2024-09-21T21:41:05.838 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 7.0390955770000003, 2024-09-21T21:41:05.838 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.019725633999999999, 2024-09-21T21:41:05.838 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:05.838 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:05.838 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:05.838 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:05.838 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:05.838 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:05.838 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 8449 2024-09-21T21:41:05.838 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.839 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:05.839 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.839 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:05.839 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.690019+0000", 2024-09-21T21:41:05.839 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:05.839 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.839 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.839 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:05.839 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.690019+0000", 2024-09-21T21:41:05.839 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:05.839 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.839 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.839 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:05.839 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.690020+0000", 2024-09-21T21:41:05.840 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3000000000000001e-07 2024-09-21T21:41:05.840 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.840 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.840 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:05.840 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.690022+0000", 2024-09-21T21:41:05.840 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2000000000000001e-06 2024-09-21T21:41:05.840 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.840 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.840 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:05.840 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.690029+0000", 2024-09-21T21:41:05.840 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5860000000000001e-06 2024-09-21T21:41:05.840 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.840 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.840 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:05.840 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.690034+0000", 2024-09-21T21:41:05.841 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9350000000000002e-06 2024-09-21T21:41:05.841 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.841 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.841 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:05.841 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.709566+0000", 2024-09-21T21:41:05.841 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.019532445999999998 2024-09-21T21:41:05.841 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.841 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.841 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:05.841 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.709647+0000", 2024-09-21T21:41:05.841 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0931000000000002e-05 2024-09-21T21:41:05.841 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.841 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.841 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:05.842 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.709745+0000", 2024-09-21T21:41:05.842 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.8105999999999999e-05 2024-09-21T21:41:05.842 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:05.842 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:05.842 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:05.842 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.842 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.842 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:8450 3.3 3:c030d67f:::rbd_data.112145fae779.0000000000000003:a [read 9420800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:41:05.842 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.690040+0000", 2024-09-21T21:41:05.842 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 7.0390751030000001, 2024-09-21T21:41:05.842 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.019843738999999999, 2024-09-21T21:41:05.842 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:05.842 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:05.842 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:05.843 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:05.843 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:05.843 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:05.843 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 8450 2024-09-21T21:41:05.843 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.843 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:05.843 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.843 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:05.843 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.690040+0000", 2024-09-21T21:41:05.843 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:05.843 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.843 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.843 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:05.843 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.690040+0000", 2024-09-21T21:41:05.843 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:05.844 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.844 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.844 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:05.844 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.690040+0000", 2024-09-21T21:41:05.844 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1699999999999999e-07 2024-09-21T21:41:05.844 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.844 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.844 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:05.844 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.690042+0000", 2024-09-21T21:41:05.844 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9240000000000001e-06 2024-09-21T21:41:05.844 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.844 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.844 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:05.844 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.690049+0000", 2024-09-21T21:41:05.844 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3069999999999999e-06 2024-09-21T21:41:05.845 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.845 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.845 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:05.845 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.690053+0000", 2024-09-21T21:41:05.845 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8239999999999999e-06 2024-09-21T21:41:05.845 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.845 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.845 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:05.845 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.709790+0000", 2024-09-21T21:41:05.845 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.019736942 2024-09-21T21:41:05.845 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.845 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.845 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:05.845 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.709834+0000", 2024-09-21T21:41:05.845 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3744e-05 2024-09-21T21:41:05.846 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.846 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.846 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:05.846 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.709884+0000", 2024-09-21T21:41:05.846 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9580999999999998e-05 2024-09-21T21:41:05.846 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:05.846 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:05.846 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:05.846 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.846 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.846 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:8451 3.3 3:c030d67f:::rbd_data.112145fae779.0000000000000003:a [read 9453568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:41:05.846 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.690077+0000", 2024-09-21T21:41:05.846 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 7.039037617, 2024-09-21T21:41:05.846 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.020018285, 2024-09-21T21:41:05.847 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:05.847 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:05.847 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:05.847 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:05.847 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:05.847 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:05.847 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 8451 2024-09-21T21:41:05.847 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.847 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:05.847 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.847 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:05.847 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.690077+0000", 2024-09-21T21:41:05.847 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:05.847 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.848 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.848 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:05.848 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.690077+0000", 2024-09-21T21:41:05.848 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:05.848 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.848 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.848 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:05.848 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.690078+0000", 2024-09-21T21:41:05.848 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1999999999999998e-07 2024-09-21T21:41:05.848 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.848 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.848 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:05.848 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.690082+0000", 2024-09-21T21:41:05.849 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0879999999999997e-06 2024-09-21T21:41:05.849 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.849 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.849 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:05.849 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.690091+0000", 2024-09-21T21:41:05.849 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6549999999999993e-06 2024-09-21T21:41:05.849 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.849 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.849 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:05.849 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.690098+0000", 2024-09-21T21:41:05.849 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7710000000000001e-06 2024-09-21T21:41:05.849 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.849 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.849 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:05.850 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.709913+0000", 2024-09-21T21:41:05.850 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.019815510000000001 2024-09-21T21:41:05.850 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.850 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.850 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:05.850 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.709991+0000", 2024-09-21T21:41:05.850 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7296e-05 2024-09-21T21:41:05.850 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.850 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.850 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:05.850 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.710096+0000", 2024-09-21T21:41:05.850 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000105145 2024-09-21T21:41:05.850 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:05.851 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:05.851 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:05.851 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.851 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.851 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:8452 3.3 3:c030d67f:::rbd_data.112145fae779.0000000000000003:a [read 9510912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:41:05.851 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.690109+0000", 2024-09-21T21:41:05.851 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 7.0390064020000001, 2024-09-21T21:41:05.851 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.020107368, 2024-09-21T21:41:05.851 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:05.851 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:05.851 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:05.851 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:05.851 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:05.851 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:05.852 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 8452 2024-09-21T21:41:05.852 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.852 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:05.852 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.852 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:05.852 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.690109+0000", 2024-09-21T21:41:05.852 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:05.852 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.852 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.852 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:05.852 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.690109+0000", 2024-09-21T21:41:05.852 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:05.852 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.852 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.853 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:05.853 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.690109+0000", 2024-09-21T21:41:05.853 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2899999999999999e-07 2024-09-21T21:41:05.853 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.853 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.853 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:05.853 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.690111+0000", 2024-09-21T21:41:05.853 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2409999999999998e-06 2024-09-21T21:41:05.853 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.853 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.853 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:05.853 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.690118+0000", 2024-09-21T21:41:05.853 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8850000000000001e-06 2024-09-21T21:41:05.853 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.853 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.854 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:05.854 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.690124+0000", 2024-09-21T21:41:05.854 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5589999999999996e-06 2024-09-21T21:41:05.854 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.854 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.854 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:05.854 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.710126+0000", 2024-09-21T21:41:05.854 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.020002217999999999 2024-09-21T21:41:05.854 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.854 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.854 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:05.854 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.710177+0000", 2024-09-21T21:41:05.854 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0587999999999997e-05 2024-09-21T21:41:05.854 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:05.855 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:05.855 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:05.855 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.710216+0000", 2024-09-21T21:41:05.855 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9447999999999999e-05 2024-09-21T21:41:05.855 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:05.855 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:05.855 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:05.855 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:05.855 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:05.855 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:05.857 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:05.854+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:05.858 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T21:41:05.858 INFO:teuthology.orchestra.run.smithi164.stdout:1434 trunc from 0xb513c00 to 0x362b200 2024-09-21T21:41:05.859 INFO:teuthology.orchestra.run.smithi164.stdout:1437 trunc from 0x362b200 to 0xd73b600 2024-09-21T21:41:05.859 INFO:teuthology.orchestra.run.smithi164.stdout:1438 trunc from 0xd73b600 to 0x53d7a00 2024-09-21T21:41:05.859 INFO:teuthology.orchestra.run.smithi164.stdout:1439 write 0x9cfae00 thru 0x9d048ab (0x9aac bytes) 2024-09-21T21:41:05.859 INFO:teuthology.orchestra.run.smithi164.stdout:1440 read 0x82a2600 thru 0x82a8515 (0x5f16 bytes) 2024-09-21T21:41:05.859 INFO:teuthology.orchestra.run.smithi164.stdout:1441 write 0x9402400 thru 0x9406f9b (0x4b9c bytes) 2024-09-21T21:41:05.859 INFO:teuthology.orchestra.run.smithi164.stdout:1448 write 0x3fb0400 thru 0x3fb786d (0x746e bytes) 2024-09-21T21:41:05.859 INFO:teuthology.orchestra.run.smithi164.stdout:1449 read 0x91f000 thru 0x92d686 (0xe687 bytes) 2024-09-21T21:41:05.859 INFO:teuthology.orchestra.run.smithi164.stdout:1450 read 0x3382e00 thru 0x3383055 (0x256 bytes) 2024-09-21T21:41:05.859 INFO:teuthology.orchestra.run.smithi164.stdout:1451 trunc from 0x9d04a00 to 0xc88400 2024-09-21T21:41:05.888 INFO:teuthology.orchestra.run.smithi164.stdout:1452 trunc from 0xc88400 to 0x7f01c00 2024-09-21T21:41:05.901 INFO:teuthology.orchestra.run.smithi164.stdout:1453 punch from 0x1a45400 to 0x1a52a00, (0xd600 bytes) 2024-09-21T21:41:05.902 INFO:teuthology.orchestra.run.smithi164.stdout:1454 trunc from 0x7f01c00 to 0x70c1000 2024-09-21T21:41:05.920 INFO:teuthology.orchestra.run.smithi164.stdout:1455 punch from 0x5002600 to 0x5011c00, (0xf600 bytes) 2024-09-21T21:41:05.921 INFO:teuthology.orchestra.run.smithi164.stdout:1456 write 0x2fbe00 thru 0x303d16 (0x7f17 bytes) 2024-09-21T21:41:05.923 INFO:teuthology.orchestra.run.smithi164.stdout:1457 punch from 0x98ea00 to 0x991e00, (0x3400 bytes) 2024-09-21T21:41:05.923 INFO:teuthology.orchestra.run.smithi164.stdout:1458 read 0x5d5aa00 thru 0x5d68591 (0xdb92 bytes) 2024-09-21T21:41:05.932 INFO:teuthology.orchestra.run.smithi164.stdout:1459 trunc from 0x70c1000 to 0xda3c200 2024-09-21T21:41:05.947 INFO:teuthology.orchestra.run.smithi164.stdout:1465 read 0xd6e1600 thru 0xd6e5659 (0x405a bytes) 2024-09-21T21:41:05.947 INFO:teuthology.orchestra.run.smithi164.stdout:1466 read 0x8e33e00 thru 0x8e35d30 (0x1f31 bytes) 2024-09-21T21:41:05.957 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:05.956+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:05.961 INFO:teuthology.orchestra.run.smithi164.stdout:1468 trunc from 0xda3c200 to 0xf81000 2024-09-21T21:41:06.005 INFO:teuthology.orchestra.run.smithi164.stdout:1470 write 0x891c800 thru 0x8922098 (0x5899 bytes) 2024-09-21T21:41:06.006 INFO:teuthology.orchestra.run.smithi164.stdout:1472 write 0x74ed400 thru 0x74ef5e0 (0x21e1 bytes) 2024-09-21T21:41:06.006 INFO:teuthology.orchestra.run.smithi164.stdout:1473 write 0x2677c00 thru 0x2684e8b (0xd28c bytes) 2024-09-21T21:41:06.007 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:06.007 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:41:06.007 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-21T21:41:06.007 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:06.010 INFO:teuthology.orchestra.run.smithi164.stdout:1474 trunc from 0x8922200 to 0x5a45e00 2024-09-21T21:41:06.017 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T21:41:06.058 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:06.056+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:06.159 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:06.158+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:06.159 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:06.160 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:41:06.160 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:41:06.160 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:41:06.160 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:06.168 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T21:41:06.260 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:06.259+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:06.290 INFO:teuthology.orchestra.run.smithi164.stdout:1475 write 0xcd8fa00 thru 0xcd95739 (0x5d3a bytes) 2024-09-21T21:41:06.291 INFO:teuthology.orchestra.run.smithi164.stdout:1477 read 0x8f21000 thru 0x8f3044f (0xf450 bytes) 2024-09-21T21:41:06.291 INFO:teuthology.orchestra.run.smithi164.stdout:1483 read 0x8b2b000 thru 0x8b2e0cd (0x30ce bytes) 2024-09-21T21:41:06.291 INFO:teuthology.orchestra.run.smithi164.stdout:1484 read 0x41f4c00 thru 0x41fde87 (0x9288 bytes) 2024-09-21T21:41:06.292 INFO:teuthology.orchestra.run.smithi164.stdout:1487 write 0x221b000 thru 0x222818f (0xd190 bytes) 2024-09-21T21:41:06.292 INFO:teuthology.orchestra.run.smithi164.stdout:1488 write 0xbf26600 thru 0xbf2c207 (0x5c08 bytes) 2024-09-21T21:41:06.292 INFO:teuthology.orchestra.run.smithi164.stdout:1489 write 0x6be9800 thru 0x6bea5be (0xdbf bytes) 2024-09-21T21:41:06.298 INFO:teuthology.orchestra.run.smithi164.stdout:1491 trunc from 0xcd95800 to 0x790c800 2024-09-21T21:41:06.316 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:06.316 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-21T21:41:06.316 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-21T21:41:06.316 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:41:06.316 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.316 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1484 3.16 3:6a41a830:::rbd_data.112145fae779.000000000000000a:a [read 303104~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:06.317 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634086+0000", 2024-09-21T21:41:06.317 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.6800354569999998, 2024-09-21T21:41:06.317 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.058257055000000002, 2024-09-21T21:41:06.317 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:06.317 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:06.317 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:06.317 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:06.317 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:06.317 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:06.317 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1484 2024-09-21T21:41:06.317 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.317 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:06.317 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.317 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:06.318 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634086+0000", 2024-09-21T21:41:06.318 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:06.318 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.318 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.318 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:06.318 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634086+0000", 2024-09-21T21:41:06.318 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:06.318 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.318 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.318 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:06.318 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634086+0000", 2024-09-21T21:41:06.318 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6100000000000001e-07 2024-09-21T21:41:06.318 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.319 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.319 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:06.319 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634088+0000", 2024-09-21T21:41:06.319 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1100000000000001e-06 2024-09-21T21:41:06.319 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.319 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.319 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:06.319 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634095+0000", 2024-09-21T21:41:06.319 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4080000000000003e-06 2024-09-21T21:41:06.319 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.319 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.319 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:06.319 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634100+0000", 2024-09-21T21:41:06.320 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0730000000000004e-06 2024-09-21T21:41:06.320 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.320 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.320 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:06.320 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.692197+0000", 2024-09-21T21:41:06.320 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.058097331000000002 2024-09-21T21:41:06.320 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.320 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.320 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:06.320 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.692271+0000", 2024-09-21T21:41:06.320 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3423999999999994e-05 2024-09-21T21:41:06.321 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.321 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.321 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:06.321 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.692343+0000", 2024-09-21T21:41:06.321 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2248000000000004e-05 2024-09-21T21:41:06.321 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:06.321 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:06.321 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:06.321 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.321 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.321 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1485 3.16 3:695c582e:::rbd_data.1121e540c096.0000000000000008:b [read 335872~36864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:06.321 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634106+0000", 2024-09-21T21:41:06.321 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.6800156600000005, 2024-09-21T21:41:06.321 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059223685999999998, 2024-09-21T21:41:06.321 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:06.322 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:06.322 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:06.322 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:06.322 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:06.322 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:06.322 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1485 2024-09-21T21:41:06.322 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.322 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:06.322 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.322 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:06.322 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634106+0000", 2024-09-21T21:41:06.322 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:06.322 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.323 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.323 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:06.323 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634106+0000", 2024-09-21T21:41:06.323 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:06.323 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.323 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.323 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:06.323 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634106+0000", 2024-09-21T21:41:06.323 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3300000000000002e-07 2024-09-21T21:41:06.323 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.323 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.323 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:06.323 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634108+0000", 2024-09-21T21:41:06.323 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.996e-06 2024-09-21T21:41:06.324 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.324 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.324 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:06.324 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634115+0000", 2024-09-21T21:41:06.324 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5139999999999998e-06 2024-09-21T21:41:06.324 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.324 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.324 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:06.324 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634120+0000", 2024-09-21T21:41:06.324 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1050000000000003e-06 2024-09-21T21:41:06.324 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.324 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.325 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:06.325 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.692374+0000", 2024-09-21T21:41:06.325 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.058254729999999998 2024-09-21T21:41:06.325 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.325 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.325 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:06.325 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.692440+0000", 2024-09-21T21:41:06.325 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6076999999999998e-05 2024-09-21T21:41:06.325 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.325 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.325 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:06.325 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693329+0000", 2024-09-21T21:41:06.325 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00088873100000000003 2024-09-21T21:41:06.325 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:06.326 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:06.326 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:06.326 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.326 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.326 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1486 3.16 3:6a41a830:::rbd_data.112145fae779.000000000000000a:a [read 360448~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:06.326 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634126+0000", 2024-09-21T21:41:06.326 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.6799953680000002, 2024-09-21T21:41:06.326 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059399528, 2024-09-21T21:41:06.326 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:06.326 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:06.326 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:06.326 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:06.326 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:06.327 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:06.327 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1486 2024-09-21T21:41:06.327 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.327 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:06.327 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.327 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:06.327 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634126+0000", 2024-09-21T21:41:06.327 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:06.327 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.327 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.327 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:06.327 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634126+0000", 2024-09-21T21:41:06.327 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:06.327 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.328 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.328 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:06.328 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634126+0000", 2024-09-21T21:41:06.328 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4099999999999999e-07 2024-09-21T21:41:06.328 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.328 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.328 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:06.328 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634128+0000", 2024-09-21T21:41:06.328 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1069999999999999e-06 2024-09-21T21:41:06.328 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.328 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.328 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:06.328 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634135+0000", 2024-09-21T21:41:06.328 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.212e-06 2024-09-21T21:41:06.328 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.329 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.329 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:06.329 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634140+0000", 2024-09-21T21:41:06.329 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1429999999999997e-06 2024-09-21T21:41:06.329 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.329 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.329 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:06.329 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693363+0000", 2024-09-21T21:41:06.329 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059223614000000001 2024-09-21T21:41:06.329 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.329 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.329 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:06.329 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693431+0000", 2024-09-21T21:41:06.329 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7294999999999999e-05 2024-09-21T21:41:06.330 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.330 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.330 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:06.330 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693525+0000", 2024-09-21T21:41:06.330 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.4716000000000006e-05 2024-09-21T21:41:06.330 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:06.330 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:06.330 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:06.330 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.330 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.330 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1499 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 49152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:06.330 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634186+0000", 2024-09-21T21:41:06.330 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.6799354910000002, 2024-09-21T21:41:06.331 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059339256999999999, 2024-09-21T21:41:06.331 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:06.331 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:06.331 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:06.331 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:06.331 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:06.331 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:06.331 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1499 2024-09-21T21:41:06.331 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.331 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:06.331 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.331 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:06.331 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634186+0000", 2024-09-21T21:41:06.331 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:06.331 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.332 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.332 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:06.332 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634186+0000", 2024-09-21T21:41:06.332 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:06.332 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.332 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.332 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:06.332 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634186+0000", 2024-09-21T21:41:06.332 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0999999999999999e-07 2024-09-21T21:41:06.332 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.332 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.332 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:06.332 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634188+0000", 2024-09-21T21:41:06.332 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.86e-06 2024-09-21T21:41:06.332 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.333 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.333 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:06.333 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634194+0000", 2024-09-21T21:41:06.333 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1109999999999996e-06 2024-09-21T21:41:06.333 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.333 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.333 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:06.333 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634199+0000", 2024-09-21T21:41:06.333 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7419999999999997e-06 2024-09-21T21:41:06.333 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.333 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.333 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:06.333 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693357+0000", 2024-09-21T21:41:06.333 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059157847 2024-09-21T21:41:06.333 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.334 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.334 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:06.334 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693430+0000", 2024-09-21T21:41:06.334 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3203999999999999e-05 2024-09-21T21:41:06.334 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.334 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.334 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:06.334 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693525+0000", 2024-09-21T21:41:06.334 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5082999999999997e-05 2024-09-21T21:41:06.334 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:06.334 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:06.334 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:06.334 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.335 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.335 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1501 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 106496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:06.335 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634205+0000", 2024-09-21T21:41:06.335 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.6799164009999998, 2024-09-21T21:41:06.335 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059482394000000001, 2024-09-21T21:41:06.335 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:06.335 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:06.335 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:06.335 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:06.335 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:06.335 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:06.335 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1501 2024-09-21T21:41:06.335 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.335 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:06.336 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.336 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:06.336 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634205+0000", 2024-09-21T21:41:06.336 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:06.336 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.336 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.336 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:06.336 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634205+0000", 2024-09-21T21:41:06.336 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:06.336 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.336 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.336 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:06.336 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634205+0000", 2024-09-21T21:41:06.336 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.03e-07 2024-09-21T21:41:06.336 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.337 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.337 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:06.337 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634207+0000", 2024-09-21T21:41:06.337 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0389999999999999e-06 2024-09-21T21:41:06.337 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.337 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.337 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:06.337 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634213+0000", 2024-09-21T21:41:06.337 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2600000000000002e-06 2024-09-21T21:41:06.337 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.337 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.337 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:06.338 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634218+0000", 2024-09-21T21:41:06.338 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8570000000000003e-06 2024-09-21T21:41:06.338 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.338 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.338 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:06.338 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693552+0000", 2024-09-21T21:41:06.338 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059333636000000002 2024-09-21T21:41:06.338 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.338 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.338 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:06.338 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693613+0000", 2024-09-21T21:41:06.338 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0560000000000003e-05 2024-09-21T21:41:06.338 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.338 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.339 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:06.339 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693687+0000", 2024-09-21T21:41:06.339 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4639000000000001e-05 2024-09-21T21:41:06.339 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:06.339 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:06.339 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:06.339 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.339 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.339 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1515 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 434176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:06.339 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634318+0000", 2024-09-21T21:41:06.339 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.679802714, 2024-09-21T21:41:06.339 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059565812000000003, 2024-09-21T21:41:06.339 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:06.339 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:06.339 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:06.340 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:06.340 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:06.340 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:06.340 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1515 2024-09-21T21:41:06.340 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.340 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:06.340 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.340 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:06.340 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634318+0000", 2024-09-21T21:41:06.341 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:06.341 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.341 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.341 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:06.342 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634318+0000", 2024-09-21T21:41:06.342 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:06.342 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.342 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.342 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:06.342 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634319+0000", 2024-09-21T21:41:06.342 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9400000000000001e-07 2024-09-21T21:41:06.342 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.342 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.342 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:06.342 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634321+0000", 2024-09-21T21:41:06.342 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.154e-06 2024-09-21T21:41:06.343 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.343 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.343 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:06.343 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634327+0000", 2024-09-21T21:41:06.343 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9830000000000001e-06 2024-09-21T21:41:06.343 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.343 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.343 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:06.343 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634332+0000", 2024-09-21T21:41:06.343 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9080000000000003e-06 2024-09-21T21:41:06.343 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.343 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.344 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:06.344 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693717+0000", 2024-09-21T21:41:06.344 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059385447000000001 2024-09-21T21:41:06.344 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.344 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.344 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:06.344 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693795+0000", 2024-09-21T21:41:06.344 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7267000000000004e-05 2024-09-21T21:41:06.344 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.344 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.344 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:06.344 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693884+0000", 2024-09-21T21:41:06.344 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9659000000000003e-05 2024-09-21T21:41:06.344 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:06.345 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:06.345 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:06.345 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.345 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.345 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1517 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 491520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:06.345 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634338+0000", 2024-09-21T21:41:06.345 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.679782941, 2024-09-21T21:41:06.345 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060100745999999997, 2024-09-21T21:41:06.345 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:06.345 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:06.345 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:06.345 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:06.345 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:06.345 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:06.346 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1517 2024-09-21T21:41:06.346 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.346 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:06.346 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.346 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:06.346 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634338+0000", 2024-09-21T21:41:06.346 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:06.346 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.346 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.346 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:06.346 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634338+0000", 2024-09-21T21:41:06.346 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:06.347 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.347 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.347 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:06.347 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634339+0000", 2024-09-21T21:41:06.347 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2899999999999999e-07 2024-09-21T21:41:06.347 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.347 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.347 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:06.347 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634341+0000", 2024-09-21T21:41:06.347 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.8959999999999999e-06 2024-09-21T21:41:06.347 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.347 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.347 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:06.347 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634347+0000", 2024-09-21T21:41:06.348 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.302e-06 2024-09-21T21:41:06.348 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.348 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.348 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:06.348 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634351+0000", 2024-09-21T21:41:06.348 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6070000000000001e-06 2024-09-21T21:41:06.348 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.348 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.348 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:06.348 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694279+0000", 2024-09-21T21:41:06.348 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059927046999999997 2024-09-21T21:41:06.348 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.348 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.349 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:06.349 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694355+0000", 2024-09-21T21:41:06.349 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6483999999999995e-05 2024-09-21T21:41:06.349 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.349 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.349 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:06.349 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694439+0000", 2024-09-21T21:41:06.349 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3980999999999994e-05 2024-09-21T21:41:06.349 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:06.349 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:06.349 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:06.349 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.349 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.349 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1518 3.16 3:6a41a830:::rbd_data.112145fae779.000000000000000a:a [read 294912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:06.350 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634357+0000", 2024-09-21T21:41:06.350 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.6797640349999998, 2024-09-21T21:41:06.350 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059712318, 2024-09-21T21:41:06.350 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:06.350 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:06.350 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:06.350 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:06.350 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:06.350 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:06.350 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1518 2024-09-21T21:41:06.350 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.350 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:06.350 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.350 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:06.350 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634357+0000", 2024-09-21T21:41:06.351 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:06.351 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.351 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.351 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:06.351 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634357+0000", 2024-09-21T21:41:06.351 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:06.351 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.351 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.351 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:06.351 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634358+0000", 2024-09-21T21:41:06.351 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2100000000000002e-07 2024-09-21T21:41:06.351 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.351 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.351 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:06.352 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634360+0000", 2024-09-21T21:41:06.352 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0940000000000002e-06 2024-09-21T21:41:06.352 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.352 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.352 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:06.352 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634366+0000", 2024-09-21T21:41:06.352 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3389999999999998e-06 2024-09-21T21:41:06.352 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.352 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.352 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:06.352 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634371+0000", 2024-09-21T21:41:06.352 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6670000000000004e-06 2024-09-21T21:41:06.352 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.353 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.353 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:06.353 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693910+0000", 2024-09-21T21:41:06.353 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059538973000000002 2024-09-21T21:41:06.353 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.353 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.353 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:06.353 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693982+0000", 2024-09-21T21:41:06.353 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2806999999999996e-05 2024-09-21T21:41:06.353 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.353 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.353 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:06.353 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694069+0000", 2024-09-21T21:41:06.353 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7016999999999997e-05 2024-09-21T21:41:06.354 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:06.354 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:06.354 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:06.354 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.354 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.354 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1519 3.16 3:6a41a830:::rbd_data.112145fae779.000000000000000a:a [read 352256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:06.354 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634377+0000", 2024-09-21T21:41:06.354 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.6797443330000004, 2024-09-21T21:41:06.354 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059872213000000001, 2024-09-21T21:41:06.354 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:06.354 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:06.354 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:06.354 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:06.355 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:06.355 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:06.355 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1519 2024-09-21T21:41:06.355 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.355 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:06.355 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.355 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:06.355 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634377+0000", 2024-09-21T21:41:06.355 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:06.355 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.355 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.355 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:06.356 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634377+0000", 2024-09-21T21:41:06.356 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:06.356 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.356 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.356 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:06.356 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634377+0000", 2024-09-21T21:41:06.356 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2600000000000002e-07 2024-09-21T21:41:06.356 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.356 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.356 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:06.356 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634379+0000", 2024-09-21T21:41:06.356 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0729999999999999e-06 2024-09-21T21:41:06.356 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.357 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.357 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:06.357 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634386+0000", 2024-09-21T21:41:06.357 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4690000000000002e-06 2024-09-21T21:41:06.357 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.357 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.357 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:06.357 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634390+0000", 2024-09-21T21:41:06.357 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4580000000000003e-06 2024-09-21T21:41:06.357 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.357 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.357 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:06.357 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694094+0000", 2024-09-21T21:41:06.357 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059703830999999999 2024-09-21T21:41:06.358 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.358 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.358 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:06.358 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694168+0000", 2024-09-21T21:41:06.358 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4222999999999997e-05 2024-09-21T21:41:06.358 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.358 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.358 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:06.358 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694249+0000", 2024-09-21T21:41:06.358 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0632999999999999e-05 2024-09-21T21:41:06.358 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:06.358 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:06.358 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:06.359 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.359 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.359 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1529 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 53248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:06.359 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634397+0000", 2024-09-21T21:41:06.359 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.6797241820000002, 2024-09-21T21:41:06.359 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060189919000000001, 2024-09-21T21:41:06.359 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:06.359 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:06.359 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:06.359 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:06.359 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:06.359 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:06.359 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1529 2024-09-21T21:41:06.359 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.359 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:06.360 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.360 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:06.360 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634397+0000", 2024-09-21T21:41:06.360 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:06.360 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.360 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.360 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:06.360 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634397+0000", 2024-09-21T21:41:06.360 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:06.360 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.360 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.360 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:06.360 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634397+0000", 2024-09-21T21:41:06.361 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2e-07 2024-09-21T21:41:06.361 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.361 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.361 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:06.361 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634399+0000", 2024-09-21T21:41:06.361 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0609999999999998e-06 2024-09-21T21:41:06.361 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.361 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.361 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:06.361 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634406+0000", 2024-09-21T21:41:06.361 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7259999999999996e-06 2024-09-21T21:41:06.361 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.361 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.361 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:06.362 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634411+0000", 2024-09-21T21:41:06.362 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9339999999999997e-06 2024-09-21T21:41:06.362 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.362 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.362 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:06.362 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694480+0000", 2024-09-21T21:41:06.362 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060069105999999997 2024-09-21T21:41:06.362 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.362 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.362 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:06.362 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694527+0000", 2024-09-21T21:41:06.362 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7120000000000003e-05 2024-09-21T21:41:06.362 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.362 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.362 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:06.363 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694587+0000", 2024-09-21T21:41:06.363 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9552000000000002e-05 2024-09-21T21:41:06.363 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:06.363 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:06.363 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:06.363 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.363 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.363 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1531 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 110592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:06.363 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634417+0000", 2024-09-21T21:41:06.363 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.6797042480000002, 2024-09-21T21:41:06.363 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060347287999999999, 2024-09-21T21:41:06.363 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:06.363 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:06.364 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:06.364 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:06.364 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:06.364 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:06.364 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1531 2024-09-21T21:41:06.364 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.364 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:06.364 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.364 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:06.364 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634417+0000", 2024-09-21T21:41:06.364 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:06.364 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.364 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.364 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:06.364 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634417+0000", 2024-09-21T21:41:06.365 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:06.365 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.365 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.365 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:06.365 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634417+0000", 2024-09-21T21:41:06.365 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1699999999999999e-07 2024-09-21T21:41:06.365 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.365 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.365 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:06.365 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634420+0000", 2024-09-21T21:41:06.365 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.243e-06 2024-09-21T21:41:06.365 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.365 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.366 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:06.366 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634426+0000", 2024-09-21T21:41:06.366 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2550000000000003e-06 2024-09-21T21:41:06.366 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.366 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.366 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:06.366 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634431+0000", 2024-09-21T21:41:06.366 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0150000000000003e-06 2024-09-21T21:41:06.366 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.366 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.366 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:06.366 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694618+0000", 2024-09-21T21:41:06.366 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060186836000000001 2024-09-21T21:41:06.367 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.367 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.367 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:06.367 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694685+0000", 2024-09-21T21:41:06.367 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6970000000000004e-05 2024-09-21T21:41:06.367 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.367 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.367 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:06.367 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694764+0000", 2024-09-21T21:41:06.367 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9552e-05 2024-09-21T21:41:06.367 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:06.367 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:06.367 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:06.367 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.368 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.368 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1532 3.e 3:724b31a3:::rbd_data.1121e540c096.0000000000000015:b [read 151552~36864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:06.368 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634437+0000", 2024-09-21T21:41:06.368 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.6796844520000001, 2024-09-21T21:41:06.368 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060474314000000001, 2024-09-21T21:41:06.368 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:06.368 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:06.368 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:06.368 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:06.368 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:06.368 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:06.369 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1532 2024-09-21T21:41:06.369 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.369 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:06.369 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.369 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:06.369 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634437+0000", 2024-09-21T21:41:06.369 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:06.369 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.369 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.369 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:06.369 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634437+0000", 2024-09-21T21:41:06.369 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:06.369 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.370 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.370 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:06.370 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634437+0000", 2024-09-21T21:41:06.370 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9400000000000001e-07 2024-09-21T21:41:06.370 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.370 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.370 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:06.370 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634439+0000", 2024-09-21T21:41:06.370 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.114e-06 2024-09-21T21:41:06.370 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.370 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.370 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:06.370 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634446+0000", 2024-09-21T21:41:06.370 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1389999999999998e-06 2024-09-21T21:41:06.370 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.371 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.371 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:06.371 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634451+0000", 2024-09-21T21:41:06.371 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0010000000000001e-06 2024-09-21T21:41:06.371 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.371 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.371 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:06.371 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694794+0000", 2024-09-21T21:41:06.371 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060342509000000003 2024-09-21T21:41:06.371 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.371 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.371 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:06.371 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694835+0000", 2024-09-21T21:41:06.372 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0728e-05 2024-09-21T21:41:06.372 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.372 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.372 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:06.372 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694911+0000", 2024-09-21T21:41:06.372 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6428999999999996e-05 2024-09-21T21:41:06.372 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:06.372 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:06.372 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:06.372 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.372 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.372 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1534 3.e 3:724b31a3:::rbd_data.1121e540c096.0000000000000015:b [read 487424~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:06.372 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634477+0000", 2024-09-21T21:41:06.372 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.6796440490000002, 2024-09-21T21:41:06.373 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060624009, 2024-09-21T21:41:06.373 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:06.373 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:06.373 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:06.373 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:06.373 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:06.373 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:06.373 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1534 2024-09-21T21:41:06.373 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.373 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:06.373 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.373 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:06.373 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634477+0000", 2024-09-21T21:41:06.374 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:06.374 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.374 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.374 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:06.374 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634477+0000", 2024-09-21T21:41:06.374 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:06.374 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.374 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.374 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:06.374 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634478+0000", 2024-09-21T21:41:06.374 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1399999999999997e-07 2024-09-21T21:41:06.374 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.374 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.375 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:06.375 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634480+0000", 2024-09-21T21:41:06.375 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1009999999999999e-06 2024-09-21T21:41:06.375 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.375 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.375 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:06.375 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634486+0000", 2024-09-21T21:41:06.375 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3289999999999999e-06 2024-09-21T21:41:06.375 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.375 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.375 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:06.375 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634491+0000", 2024-09-21T21:41:06.375 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6260000000000003e-06 2024-09-21T21:41:06.376 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.376 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.376 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:06.376 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694943+0000", 2024-09-21T21:41:06.376 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060451960999999999 2024-09-21T21:41:06.376 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.376 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.376 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:06.376 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695014+0000", 2024-09-21T21:41:06.376 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1587000000000004e-05 2024-09-21T21:41:06.376 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.376 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.376 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:06.376 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695101+0000", 2024-09-21T21:41:06.377 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6990999999999994e-05 2024-09-21T21:41:06.377 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:06.377 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:06.377 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:06.377 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.377 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.377 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1545 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 438272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:06.377 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634515+0000", 2024-09-21T21:41:06.377 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.6796058489999997, 2024-09-21T21:41:06.377 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060707944, 2024-09-21T21:41:06.377 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:06.377 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:06.377 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:06.377 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:06.378 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:06.378 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:06.378 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1545 2024-09-21T21:41:06.378 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.378 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:06.378 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.378 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:06.378 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634515+0000", 2024-09-21T21:41:06.378 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:06.378 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.378 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.378 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:06.378 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634515+0000", 2024-09-21T21:41:06.378 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:06.379 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.379 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.379 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:06.379 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634516+0000", 2024-09-21T21:41:06.379 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0999999999999999e-07 2024-09-21T21:41:06.379 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.379 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.379 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:06.379 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634518+0000", 2024-09-21T21:41:06.379 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.846e-06 2024-09-21T21:41:06.379 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.379 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.379 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:06.379 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634524+0000", 2024-09-21T21:41:06.380 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2550000000000003e-06 2024-09-21T21:41:06.380 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.380 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.380 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:06.381 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634529+0000", 2024-09-21T21:41:06.381 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6940000000000003e-06 2024-09-21T21:41:06.381 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.381 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.382 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:06.382 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695127+0000", 2024-09-21T21:41:06.382 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060598921 2024-09-21T21:41:06.382 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.382 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.382 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:06.382 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695171+0000", 2024-09-21T21:41:06.382 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3588999999999999e-05 2024-09-21T21:41:06.382 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.382 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.382 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:06.382 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695223+0000", 2024-09-21T21:41:06.382 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2228999999999997e-05 2024-09-21T21:41:06.383 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:06.383 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:06.383 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:06.383 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.383 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.383 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1547 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 495616~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:06.383 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634534+0000", 2024-09-21T21:41:06.383 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.6795870580000001, 2024-09-21T21:41:06.383 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060881131999999998, 2024-09-21T21:41:06.383 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:06.383 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:06.383 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:06.383 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:06.383 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:06.384 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:06.384 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1547 2024-09-21T21:41:06.384 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.384 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:06.384 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.384 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:06.384 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634534+0000", 2024-09-21T21:41:06.384 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:06.384 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.384 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.384 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:06.384 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634534+0000", 2024-09-21T21:41:06.384 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:06.384 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.384 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.385 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:06.385 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634535+0000", 2024-09-21T21:41:06.385 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.77e-07 2024-09-21T21:41:06.385 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.385 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.385 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:06.385 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634542+0000", 2024-09-21T21:41:06.385 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9000000000000006e-06 2024-09-21T21:41:06.385 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.385 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.386 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:06.386 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634549+0000", 2024-09-21T21:41:06.386 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5910000000000001e-06 2024-09-21T21:41:06.386 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.386 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.386 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:06.386 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634554+0000", 2024-09-21T21:41:06.386 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7940000000000002e-06 2024-09-21T21:41:06.386 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.386 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.386 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:06.386 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695256+0000", 2024-09-21T21:41:06.386 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.06070246 2024-09-21T21:41:06.387 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.387 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.387 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:06.387 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695332+0000", 2024-09-21T21:41:06.387 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6248000000000006e-05 2024-09-21T21:41:06.387 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.387 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.387 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:06.387 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695415+0000", 2024-09-21T21:41:06.387 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2762000000000005e-05 2024-09-21T21:41:06.387 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:06.387 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:06.387 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:06.387 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.387 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.388 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1548 3.16 3:6a41a830:::rbd_data.112145fae779.000000000000000a:a [read 299008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:06.388 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634561+0000", 2024-09-21T21:41:06.388 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.6795604810000002, 2024-09-21T21:41:06.388 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060786743999999997, 2024-09-21T21:41:06.388 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:06.388 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:06.388 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:06.388 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:06.388 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:06.388 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:06.388 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1548 2024-09-21T21:41:06.388 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.388 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:06.388 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.389 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:06.389 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634561+0000", 2024-09-21T21:41:06.389 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:06.389 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.389 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.389 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:06.389 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634561+0000", 2024-09-21T21:41:06.389 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:06.389 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.389 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.389 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:06.389 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634561+0000", 2024-09-21T21:41:06.389 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5199999999999997e-07 2024-09-21T21:41:06.389 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.389 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.390 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:06.390 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634564+0000", 2024-09-21T21:41:06.390 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4040000000000002e-06 2024-09-21T21:41:06.390 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.390 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.390 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:06.390 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634571+0000", 2024-09-21T21:41:06.390 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4379999999999998e-06 2024-09-21T21:41:06.390 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.390 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.390 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:06.390 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634576+0000", 2024-09-21T21:41:06.390 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1399999999999999e-06 2024-09-21T21:41:06.390 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.391 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.391 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:06.391 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695247+0000", 2024-09-21T21:41:06.391 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060670397000000001 2024-09-21T21:41:06.391 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.391 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.391 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:06.391 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695299+0000", 2024-09-21T21:41:06.391 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2564e-05 2024-09-21T21:41:06.391 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.391 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.391 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:06.391 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695347+0000", 2024-09-21T21:41:06.391 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8248999999999998e-05 2024-09-21T21:41:06.391 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:06.392 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:06.392 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:06.392 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.392 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.392 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1549 3.16 3:6a41a830:::rbd_data.112145fae779.000000000000000a:a [read 356352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:06.392 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634583+0000", 2024-09-21T21:41:06.392 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.6795385679999999, 2024-09-21T21:41:06.392 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060889459, 2024-09-21T21:41:06.392 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:06.392 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:06.392 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:06.392 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:06.392 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:06.392 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:06.392 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1549 2024-09-21T21:41:06.392 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.393 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:06.393 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.393 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:06.393 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634583+0000", 2024-09-21T21:41:06.393 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:06.393 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.393 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.393 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:06.393 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634583+0000", 2024-09-21T21:41:06.393 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:06.393 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.393 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.393 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:06.393 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634583+0000", 2024-09-21T21:41:06.394 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1899999999999998e-07 2024-09-21T21:41:06.394 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.394 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.394 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:06.394 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634585+0000", 2024-09-21T21:41:06.394 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2129999999999998e-06 2024-09-21T21:41:06.394 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.394 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.394 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:06.394 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634592+0000", 2024-09-21T21:41:06.394 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2840000000000003e-06 2024-09-21T21:41:06.395 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.395 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.395 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:06.395 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634596+0000", 2024-09-21T21:41:06.395 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9309999999999999e-06 2024-09-21T21:41:06.395 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.395 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.395 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:06.395 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695371+0000", 2024-09-21T21:41:06.395 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060774425999999999 2024-09-21T21:41:06.395 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.395 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.395 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:06.395 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695419+0000", 2024-09-21T21:41:06.396 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8607999999999999e-05 2024-09-21T21:41:06.396 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.396 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.396 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:06.396 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695472+0000", 2024-09-21T21:41:06.396 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2577999999999997e-05 2024-09-21T21:41:06.396 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:06.396 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:06.396 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:06.396 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.396 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.396 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1588 3.16 3:695c582e:::rbd_data.1121e540c096.0000000000000008:b [read 147456~40960] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:06.396 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.636942+0000", 2024-09-21T21:41:06.396 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.6771787910000002, 2024-09-21T21:41:06.397 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.058643811999999997, 2024-09-21T21:41:06.397 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:06.397 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:06.397 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:06.397 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:06.397 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:06.397 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:06.397 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1588 2024-09-21T21:41:06.397 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.397 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:06.397 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.397 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:06.397 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636942+0000", 2024-09-21T21:41:06.398 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:06.398 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.398 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.398 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:06.398 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636942+0000", 2024-09-21T21:41:06.398 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:06.398 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.398 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.398 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:06.398 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636943+0000", 2024-09-21T21:41:06.398 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4199999999999996e-07 2024-09-21T21:41:06.398 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.399 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.399 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:06.399 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636945+0000", 2024-09-21T21:41:06.399 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.401e-06 2024-09-21T21:41:06.399 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.399 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.399 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:06.399 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636952+0000", 2024-09-21T21:41:06.399 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5379999999999999e-06 2024-09-21T21:41:06.399 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.399 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.399 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:06.399 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636958+0000", 2024-09-21T21:41:06.399 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.489e-06 2024-09-21T21:41:06.399 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.400 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.400 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:06.400 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695497+0000", 2024-09-21T21:41:06.400 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.058538504999999998 2024-09-21T21:41:06.400 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.400 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.400 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:06.400 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695533+0000", 2024-09-21T21:41:06.400 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6118000000000002e-05 2024-09-21T21:41:06.400 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.400 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.400 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:06.400 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695586+0000", 2024-09-21T21:41:06.400 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3219e-05 2024-09-21T21:41:06.401 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:06.401 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:06.401 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:06.401 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.401 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.401 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1589 3.16 3:695c582e:::rbd_data.1121e540c096.0000000000000008:b [read 487424~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:06.401 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.636965+0000", 2024-09-21T21:41:06.401 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.6771557000000001, 2024-09-21T21:41:06.401 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.058728152999999998, 2024-09-21T21:41:06.401 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:06.401 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:06.401 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:06.401 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:06.401 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:06.402 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:06.402 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1589 2024-09-21T21:41:06.402 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.402 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:06.402 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.402 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:06.402 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636965+0000", 2024-09-21T21:41:06.402 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:06.402 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.402 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.402 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:06.402 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636965+0000", 2024-09-21T21:41:06.402 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:06.402 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.403 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.403 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:06.403 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636966+0000", 2024-09-21T21:41:06.403 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3799999999999998e-07 2024-09-21T21:41:06.403 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.403 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.403 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:06.403 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636968+0000", 2024-09-21T21:41:06.403 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2400000000000002e-06 2024-09-21T21:41:06.403 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.403 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.403 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:06.403 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636976+0000", 2024-09-21T21:41:06.403 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5050000000000001e-06 2024-09-21T21:41:06.404 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.404 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.404 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:06.404 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636981+0000", 2024-09-21T21:41:06.404 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2170000000000002e-06 2024-09-21T21:41:06.404 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.404 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.404 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:06.404 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695610+0000", 2024-09-21T21:41:06.404 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.058628998000000002 2024-09-21T21:41:06.404 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.404 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.404 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:06.404 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695647+0000", 2024-09-21T21:41:06.405 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6752000000000003e-05 2024-09-21T21:41:06.405 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.405 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.405 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:06.405 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695694+0000", 2024-09-21T21:41:06.405 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7002999999999997e-05 2024-09-21T21:41:06.405 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:06.405 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:06.405 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:06.405 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.405 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.405 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1607 3.e 3:724b31a3:::rbd_data.1121e540c096.0000000000000015:b [read 299008~36864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:06.405 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.637062+0000", 2024-09-21T21:41:06.405 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.6770596089999996, 2024-09-21T21:41:06.406 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.058623078000000002, 2024-09-21T21:41:06.406 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:06.406 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:06.406 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:06.406 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:06.406 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:06.406 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:06.406 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1607 2024-09-21T21:41:06.406 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.406 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:06.406 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.406 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:06.406 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.637062+0000", 2024-09-21T21:41:06.406 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:06.407 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.407 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.407 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:06.407 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.637062+0000", 2024-09-21T21:41:06.407 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:06.407 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.407 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.407 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:06.407 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.637062+0000", 2024-09-21T21:41:06.407 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0900000000000002e-07 2024-09-21T21:41:06.407 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.407 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.407 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:06.407 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.637069+0000", 2024-09-21T21:41:06.407 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.267e-06 2024-09-21T21:41:06.408 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.408 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.408 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:06.408 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.637076+0000", 2024-09-21T21:41:06.408 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7320000000000001e-06 2024-09-21T21:41:06.408 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.408 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.408 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:06.408 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.637082+0000", 2024-09-21T21:41:06.408 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4760000000000003e-06 2024-09-21T21:41:06.408 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.408 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.408 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:06.408 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695604+0000", 2024-09-21T21:41:06.409 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.058521261999999998 2024-09-21T21:41:06.409 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.409 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.409 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:06.409 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695638+0000", 2024-09-21T21:41:06.409 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4292999999999998e-05 2024-09-21T21:41:06.409 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:06.409 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:06.409 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:06.409 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695685+0000", 2024-09-21T21:41:06.409 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6638999999999998e-05 2024-09-21T21:41:06.409 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:06.409 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:06.409 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:06.409 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:06.410 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:06.410 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:06.411 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:06.408+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:06.413 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T21:41:06.413 INFO:teuthology.orchestra.run.smithi164.stdout:1492 read 0x214f400 thru 0x2158ced (0x98ee bytes) 2024-09-21T21:41:06.413 INFO:teuthology.orchestra.run.smithi164.stdout:1494 read 0x7502400 thru 0x750bd99 (0x999a bytes) 2024-09-21T21:41:06.414 INFO:teuthology.orchestra.run.smithi164.stdout:1496 write 0x5d95400 thru 0x5da365d (0xe25e bytes) 2024-09-21T21:41:06.414 INFO:teuthology.orchestra.run.smithi164.stdout:1499 trunc from 0x790c800 to 0x78eca00 2024-09-21T21:41:06.414 INFO:teuthology.orchestra.run.smithi164.stdout:1500 read 0x2a6c00 thru 0x2a73e3 (0x7e4 bytes) 2024-09-21T21:41:06.414 INFO:teuthology.orchestra.run.smithi164.stdout:1503 read 0x6cb8200 thru 0x6cbc68e (0x448f bytes) 2024-09-21T21:41:06.414 INFO:teuthology.orchestra.run.smithi164.stdout:1504 write 0x43f8000 thru 0x440783d (0xf83e bytes) 2024-09-21T21:41:06.414 INFO:teuthology.orchestra.run.smithi164.stdout:1505 punch from 0x67ce400 to 0x67d6800, (0x8400 bytes) 2024-09-21T21:41:06.414 INFO:teuthology.orchestra.run.smithi164.stdout:1506 trunc from 0x78eca00 to 0x11f9200 2024-09-21T21:41:06.414 INFO:teuthology.orchestra.run.smithi164.stdout:1509 punch from 0x254200 to 0x262000, (0xde00 bytes) 2024-09-21T21:41:06.414 INFO:teuthology.orchestra.run.smithi164.stdout:1510 trunc from 0x11f9200 to 0x70d1800 2024-09-21T21:41:06.414 INFO:teuthology.orchestra.run.smithi164.stdout:1511 write 0x49d2e00 thru 0x49dee43 (0xc044 bytes) 2024-09-21T21:41:06.414 INFO:teuthology.orchestra.run.smithi164.stdout:1512 read 0x355800 thru 0x35d43c (0x7c3d bytes) 2024-09-21T21:41:06.414 INFO:teuthology.orchestra.run.smithi164.stdout:1513 write 0x76d7600 thru 0x76e033e (0x8d3f bytes) 2024-09-21T21:41:06.429 INFO:teuthology.orchestra.run.smithi164.stdout:1514 write 0xe562600 thru 0xe565cc2 (0x36c3 bytes) 2024-09-21T21:41:06.448 INFO:teuthology.orchestra.run.smithi164.stdout:1516 trunc from 0xe565e00 to 0x3c7f000 2024-09-21T21:41:06.472 INFO:teuthology.orchestra.run.smithi164.stdout:1519 read 0x2ee6e00 thru 0x2ef063b (0x983c bytes) 2024-09-21T21:41:06.495 INFO:teuthology.orchestra.run.smithi164.stdout:1521 write 0x529dc00 thru 0x52a2b86 (0x4f87 bytes) 2024-09-21T21:41:06.495 INFO:teuthology.orchestra.run.smithi164.stdout:1522 read 0x1e21000 thru 0x1e28db8 (0x7db9 bytes) 2024-09-21T21:41:06.496 INFO:teuthology.orchestra.run.smithi164.stdout:1523 read 0xdbaa00 thru 0xdc1461 (0x6a62 bytes) 2024-09-21T21:41:06.496 INFO:teuthology.orchestra.run.smithi164.stdout:1525 read 0x8f8c00 thru 0x904aac (0xbead bytes) 2024-09-21T21:41:06.497 INFO:teuthology.orchestra.run.smithi164.stdout:1526 punch from 0x29a1a00 to 0x29af800, (0xde00 bytes) 2024-09-21T21:41:06.512 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:06.509+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:06.521 INFO:teuthology.orchestra.run.smithi164.stdout:1528 write 0xd5ba000 thru 0xd5bbe0a (0x1e0b bytes) 2024-09-21T21:41:06.522 INFO:teuthology.orchestra.run.smithi164.stdout:1529 punch from 0xd51d600 to 0xd51f400, (0x1e00 bytes) 2024-09-21T21:41:06.523 INFO:teuthology.orchestra.run.smithi164.stdout:1532 read 0x1c47200 thru 0x1c55b3c (0xe93d bytes) 2024-09-21T21:41:06.523 INFO:teuthology.orchestra.run.smithi164.stdout:1533 write 0x4f1f600 thru 0x4f20f89 (0x198a bytes) 2024-09-21T21:41:06.524 INFO:teuthology.orchestra.run.smithi164.stdout:1536 read 0x7d3800 thru 0x7d5159 (0x195a bytes) 2024-09-21T21:41:06.530 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:06.530 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:41:06.531 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 0 2024-09-21T21:41:06.531 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:06.540 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T21:41:06.540 INFO:teuthology.orchestra.run.smithi164.stdout:1538 trunc from 0xd5bc000 to 0x4d400 2024-09-21T21:41:06.559 INFO:teuthology.orchestra.run.smithi081.stderr:noscrub is unset 2024-09-21T21:41:06.613 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:06.611+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:06.619 INFO:teuthology.orchestra.run.smithi164.stdout:1539 write 0x61cc00 thru 0x62b9a0 (0xeda1 bytes) 2024-09-21T21:41:06.642 INFO:teuthology.orchestra.run.smithi164.stdout:1540 write 0x36c1000 thru 0x36cac57 (0x9c58 bytes) 2024-09-21T21:41:06.642 INFO:teuthology.orchestra.run.smithi164.stdout:1544 write 0xa4b200 thru 0xa5801e (0xce1f bytes) 2024-09-21T21:41:06.642 INFO:teuthology.orchestra.run.smithi164.stdout:1545 punch from 0x564000 to 0x56c400, (0x8400 bytes) 2024-09-21T21:41:06.664 INFO:teuthology.orchestra.run.smithi164.stdout:1549 write 0xeaeb800 thru 0xeaf2eca (0x76cb bytes) 2024-09-21T21:41:06.669 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:06.669 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:41:06.669 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:41:06.669 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:41:06.669 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:06.671 INFO:teuthology.orchestra.run.smithi164.stdout:1550 write 0x8cf3a00 thru 0x8cfd35e (0x995f bytes) 2024-09-21T21:41:06.671 INFO:teuthology.orchestra.run.smithi164.stdout:1552 punch from 0x26c1600 to 0x26cf400, (0xde00 bytes) 2024-09-21T21:41:06.671 INFO:teuthology.orchestra.run.smithi164.stdout:1554 read 0x52c600 thru 0x52dc56 (0x1657 bytes) 2024-09-21T21:41:06.671 INFO:teuthology.orchestra.run.smithi164.stdout:1555 write 0x7f49800 thru 0x7f5734e (0xdb4f bytes) 2024-09-21T21:41:06.678 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T21:41:06.687 INFO:teuthology.orchestra.run.smithi164.stdout:1556 trunc from 0xeaf3000 to 0x1c9c000 2024-09-21T21:41:06.707 INFO:teuthology.orchestra.run.smithi164.stdout:1557 punch from 0x1ad6000 to 0x1ae2200, (0xc200 bytes) 2024-09-21T21:41:06.712 INFO:teuthology.orchestra.run.smithi164.stdout:1562 punch from 0x94600 to 0x9bc00, (0x7600 bytes) 2024-09-21T21:41:06.712 INFO:teuthology.orchestra.run.smithi164.stdout:1564 read 0x3bd400 thru 0x3c86b2 (0xb2b3 bytes) 2024-09-21T21:41:06.713 INFO:teuthology.orchestra.run.smithi164.stdout:1565 read 0x638200 thru 0x63e5ba (0x63bb bytes) 2024-09-21T21:41:06.713 INFO:teuthology.orchestra.run.smithi164.stdout:1566 read 0xe2b400 thru 0xe375ac (0xc1ad bytes) 2024-09-21T21:41:06.714 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:06.712+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:06.714 INFO:teuthology.orchestra.run.smithi164.stdout:1569 punch from 0x10b5800 to 0x10bc800, (0x7000 bytes) 2024-09-21T21:41:06.714 INFO:teuthology.orchestra.run.smithi164.stdout:1570 punch from 0x18e7800 to 0x18f3400, (0xbc00 bytes) 2024-09-21T21:41:06.715 INFO:teuthology.orchestra.run.smithi164.stdout:1572 trunc from 0x1c9c000 to 0x107be00 2024-09-21T21:41:06.727 INFO:teuthology.orchestra.run.smithi164.stdout:1579 read 0xab8a00 thru 0xac5255 (0xc856 bytes) 2024-09-21T21:41:06.728 INFO:teuthology.orchestra.run.smithi164.stdout:1580 read 0x606000 thru 0x60f057 (0x9058 bytes) 2024-09-21T21:41:06.729 INFO:teuthology.orchestra.run.smithi164.stdout:1584 punch from 0x30fc00 to 0x31aa00, (0xae00 bytes) 2024-09-21T21:41:06.729 INFO:teuthology.orchestra.run.smithi164.stdout:1589 punch from 0x5b200 to 0x5ce00, (0x1c00 bytes) 2024-09-21T21:41:06.729 INFO:teuthology.orchestra.run.smithi164.stdout:1591 read 0xe5000 thru 0xec651 (0x7652 bytes) 2024-09-21T21:41:06.743 INFO:teuthology.orchestra.run.smithi164.stdout:1592 write 0x4abfe00 thru 0x4ac131a (0x151b bytes) 2024-09-21T21:41:06.759 INFO:teuthology.orchestra.run.smithi164.stdout:1593 write 0x72c4400 thru 0x72ca387 (0x5f88 bytes) 2024-09-21T21:41:06.761 INFO:teuthology.orchestra.run.smithi164.stdout:1594 trunc from 0x72ca400 to 0x5c06600 2024-09-21T21:41:06.773 INFO:teuthology.orchestra.run.smithi164.stdout:1596 write 0x2883e00 thru 0x28859d4 (0x1bd5 bytes) 2024-09-21T21:41:06.774 INFO:teuthology.orchestra.run.smithi164.stdout:1599 punch from 0x11e9200 to 0x11f6800, (0xd600 bytes) 2024-09-21T21:41:06.789 INFO:teuthology.orchestra.run.smithi164.stdout:1603 write 0xa412800 thru 0xa4204b2 (0xdcb3 bytes) 2024-09-21T21:41:06.790 INFO:teuthology.orchestra.run.smithi164.stdout:1604 read 0x8e5d600 thru 0x8e63b5a (0x655b bytes) 2024-09-21T21:41:06.790 INFO:teuthology.orchestra.run.smithi164.stdout:1605 write 0x5465a00 thru 0x546aa08 (0x5009 bytes) 2024-09-21T21:41:06.791 INFO:teuthology.orchestra.run.smithi164.stdout:1608 punch from 0x25c5e00 to 0x25c7c00, (0x1e00 bytes) 2024-09-21T21:41:06.791 INFO:teuthology.orchestra.run.smithi164.stdout:1609 punch from 0x82e0000 to 0x82e8600, (0x8600 bytes) 2024-09-21T21:41:06.791 INFO:teuthology.orchestra.run.smithi164.stdout:1610 punch from 0x5d47800 to 0x5d4da00, (0x6200 bytes) 2024-09-21T21:41:06.803 INFO:teuthology.orchestra.run.smithi164.stdout:1611 trunc from 0xa420600 to 0x658400 2024-09-21T21:41:06.814 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:06.811+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:06.833 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:06.833 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 20, 2024-09-21T21:41:06.833 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 600, 2024-09-21T21:41:06.833 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:41:06.834 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.834 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:228 3.6 3:678f83b3:::rbd_data.11214be7c411.000000000000002a:head [delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:41:06.834 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.553796+0000", 2024-09-21T21:41:06.834 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 23.277464239, 2024-09-21T21:41:06.834 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0098649099999999993, 2024-09-21T21:41:06.834 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:06.834 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:06.834 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:41:06.834 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:06.834 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:41:06.834 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:41:06.834 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 228 2024-09-21T21:41:06.834 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.834 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:06.835 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.835 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:06.835 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.553796+0000", 2024-09-21T21:41:06.835 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:06.835 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.835 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.835 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:06.835 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.553796+0000", 2024-09-21T21:41:06.835 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:06.835 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.835 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.835 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:06.835 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.553797+0000", 2024-09-21T21:41:06.835 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6899999999999997e-07 2024-09-21T21:41:06.836 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.836 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.836 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:06.836 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.553799+0000", 2024-09-21T21:41:06.836 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3e-06 2024-09-21T21:41:06.836 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.836 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.836 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:06.836 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.553805+0000", 2024-09-21T21:41:06.836 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1310000000000003e-06 2024-09-21T21:41:06.836 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.836 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.836 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:06.836 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.553816+0000", 2024-09-21T21:41:06.837 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0506000000000001e-05 2024-09-21T21:41:06.837 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.837 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.837 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:06.837 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.555870+0000", 2024-09-21T21:41:06.837 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0020538510000000002 2024-09-21T21:41:06.837 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.837 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.837 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:06.837 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.555942+0000", 2024-09-21T21:41:06.837 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.2436999999999998e-05 2024-09-21T21:41:06.837 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.837 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.837 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 1", 2024-09-21T21:41:06.838 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556121+0000", 2024-09-21T21:41:06.838 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000178526 2024-09-21T21:41:06.838 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.838 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.838 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:41:06.838 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.559481+0000", 2024-09-21T21:41:06.838 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0033603019999999999 2024-09-21T21:41:06.838 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.838 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.838 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:41:06.838 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.563144+0000", 2024-09-21T21:41:06.838 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0036625799999999999 2024-09-21T21:41:06.838 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.838 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.839 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:41:06.839 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.563185+0000", 2024-09-21T21:41:06.839 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1084000000000002e-05 2024-09-21T21:41:06.839 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.839 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.839 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:06.839 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.563661+0000", 2024-09-21T21:41:06.839 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00047662399999999998 2024-09-21T21:41:06.839 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:06.839 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:06.839 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:06.839 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.839 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.839 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:1608 3.12 3:4bfa5158:::rbd_data.112145fae779.0000000000000002:a [read 10678272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:06.839 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.381876+0000", 2024-09-21T21:41:06.840 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 8.4493842870000009, 2024-09-21T21:41:06.840 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0096257990000000009, 2024-09-21T21:41:06.840 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:06.840 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:06.840 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:06.840 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:06.840 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:06.840 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:06.840 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1608 2024-09-21T21:41:06.840 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.840 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:06.840 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.840 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:06.840 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.381876+0000", 2024-09-21T21:41:06.841 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:06.841 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.841 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.841 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:06.841 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.381876+0000", 2024-09-21T21:41:06.841 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:06.841 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.841 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.841 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:06.841 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.381876+0000", 2024-09-21T21:41:06.841 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1e-07 2024-09-21T21:41:06.841 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.841 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.841 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:06.841 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.381878+0000", 2024-09-21T21:41:06.842 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.111e-06 2024-09-21T21:41:06.842 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.842 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.842 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:06.842 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.381881+0000", 2024-09-21T21:41:06.842 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0570000000000001e-06 2024-09-21T21:41:06.842 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.842 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.842 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:06.842 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.381883+0000", 2024-09-21T21:41:06.842 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3429999999999998e-06 2024-09-21T21:41:06.842 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.842 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.843 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:06.843 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.391392+0000", 2024-09-21T21:41:06.843 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.009509221 2024-09-21T21:41:06.843 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.843 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.843 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:06.843 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.391448+0000", 2024-09-21T21:41:06.843 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5961999999999997e-05 2024-09-21T21:41:06.843 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.843 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.843 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:06.843 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.391502+0000", 2024-09-21T21:41:06.843 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3894999999999998e-05 2024-09-21T21:41:06.843 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:06.844 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:06.844 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:06.844 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.844 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.844 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:1610 3.12 3:4bfa5158:::rbd_data.112145fae779.0000000000000002:a [read 10735616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:06.844 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.381886+0000", 2024-09-21T21:41:06.844 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 8.4493745009999994, 2024-09-21T21:41:06.844 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0097775250000000005, 2024-09-21T21:41:06.844 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:06.844 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:06.844 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:06.844 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:06.844 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:06.844 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:06.844 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1610 2024-09-21T21:41:06.845 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.845 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:06.845 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.845 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:06.845 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.381886+0000", 2024-09-21T21:41:06.845 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:06.845 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.845 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.845 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:06.845 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.381886+0000", 2024-09-21T21:41:06.845 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:06.846 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.846 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.846 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:06.846 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.381886+0000", 2024-09-21T21:41:06.846 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.05e-07 2024-09-21T21:41:06.846 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.846 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.846 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:06.846 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.381887+0000", 2024-09-21T21:41:06.846 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1960000000000001e-06 2024-09-21T21:41:06.846 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.846 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.846 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:06.846 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.381891+0000", 2024-09-21T21:41:06.847 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.061e-06 2024-09-21T21:41:06.847 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.847 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.847 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:06.847 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.381893+0000", 2024-09-21T21:41:06.847 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.463e-06 2024-09-21T21:41:06.847 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.847 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.847 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:06.847 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.391533+0000", 2024-09-21T21:41:06.847 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0096403979999999997 2024-09-21T21:41:06.847 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.847 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.847 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:06.848 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.391593+0000", 2024-09-21T21:41:06.848 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9778999999999998e-05 2024-09-21T21:41:06.848 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.848 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.848 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:06.848 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.391664+0000", 2024-09-21T21:41:06.848 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0423e-05 2024-09-21T21:41:06.848 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:06.848 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:06.848 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:06.848 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.848 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.848 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:1800 3.12 3:4bfa5158:::rbd_data.112145fae779.0000000000000002:a [read 10391552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:06.848 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.382760+0000", 2024-09-21T21:41:06.848 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 8.4485003630000008, 2024-09-21T21:41:06.849 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0094873609999999997, 2024-09-21T21:41:06.849 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:06.849 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:06.849 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:06.849 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:06.849 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:06.849 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:06.849 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1800 2024-09-21T21:41:06.849 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.849 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:06.849 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.849 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:06.849 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382760+0000", 2024-09-21T21:41:06.849 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:06.850 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.850 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.850 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:06.850 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382760+0000", 2024-09-21T21:41:06.850 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:06.850 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.850 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.850 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:06.850 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382761+0000", 2024-09-21T21:41:06.850 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.63e-07 2024-09-21T21:41:06.850 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.850 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.850 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:06.850 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382763+0000", 2024-09-21T21:41:06.851 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.4080000000000001e-06 2024-09-21T21:41:06.851 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.851 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.851 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:06.851 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382770+0000", 2024-09-21T21:41:06.851 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.528e-06 2024-09-21T21:41:06.851 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.851 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.851 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:06.851 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382775+0000", 2024-09-21T21:41:06.851 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0540000000000002e-06 2024-09-21T21:41:06.851 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.851 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.851 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:06.851 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.392092+0000", 2024-09-21T21:41:06.852 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0093168999999999995 2024-09-21T21:41:06.852 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.852 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.852 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:06.852 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.392173+0000", 2024-09-21T21:41:06.852 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.1897999999999994e-05 2024-09-21T21:41:06.852 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.852 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.852 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:06.852 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.392248+0000", 2024-09-21T21:41:06.852 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.4109999999999993e-05 2024-09-21T21:41:06.852 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:06.852 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:06.852 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:06.852 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.853 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.853 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:1802 3.12 3:4bfa5158:::rbd_data.112145fae779.0000000000000002:a [read 10448896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:06.853 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.382781+0000", 2024-09-21T21:41:06.853 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 8.4484791159999997, 2024-09-21T21:41:06.853 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0096511759999999992, 2024-09-21T21:41:06.853 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:06.853 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:06.853 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:06.853 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:06.853 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:06.853 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:06.853 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1802 2024-09-21T21:41:06.853 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.853 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:06.853 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.854 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:06.854 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382781+0000", 2024-09-21T21:41:06.854 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:06.854 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.854 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.854 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:06.854 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382781+0000", 2024-09-21T21:41:06.854 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:06.854 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.854 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.854 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:06.854 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382782+0000", 2024-09-21T21:41:06.854 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.41e-07 2024-09-21T21:41:06.854 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.855 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.855 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:06.855 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382784+0000", 2024-09-21T21:41:06.855 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1509999999999998e-06 2024-09-21T21:41:06.855 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.855 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.855 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:06.855 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382799+0000", 2024-09-21T21:41:06.855 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.4711e-05 2024-09-21T21:41:06.855 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.855 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.855 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:06.855 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382803+0000", 2024-09-21T21:41:06.855 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8130000000000003e-06 2024-09-21T21:41:06.856 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.856 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.856 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:06.856 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.392304+0000", 2024-09-21T21:41:06.856 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0095000510000000007 2024-09-21T21:41:06.856 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.856 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.856 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:06.856 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.392383+0000", 2024-09-21T21:41:06.856 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.9666999999999995e-05 2024-09-21T21:41:06.856 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.856 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.856 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:06.856 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.392433+0000", 2024-09-21T21:41:06.856 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9441999999999999e-05 2024-09-21T21:41:06.857 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:06.857 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:06.857 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:06.857 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.857 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.857 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:1804 3.12 3:4bfa5158:::rbd_data.112145fae779.0000000000000002:a [read 10539008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:06.857 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.382815+0000", 2024-09-21T21:41:06.857 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 8.4484452599999997, 2024-09-21T21:41:06.857 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.009793263, 2024-09-21T21:41:06.857 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:06.857 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:06.857 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:06.857 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:06.857 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:06.858 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:06.858 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1804 2024-09-21T21:41:06.858 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.858 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:06.858 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.858 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:06.858 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382815+0000", 2024-09-21T21:41:06.858 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:06.858 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.858 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.858 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:06.858 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382815+0000", 2024-09-21T21:41:06.858 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:06.858 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.859 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.859 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:06.859 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382816+0000", 2024-09-21T21:41:06.859 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.06e-07 2024-09-21T21:41:06.859 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.859 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.859 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:06.859 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382818+0000", 2024-09-21T21:41:06.859 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.5069999999999999e-06 2024-09-21T21:41:06.859 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.859 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.859 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:06.859 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382825+0000", 2024-09-21T21:41:06.859 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0899999999999999e-06 2024-09-21T21:41:06.859 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.860 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.860 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:06.860 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382831+0000", 2024-09-21T21:41:06.860 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1830000000000002e-06 2024-09-21T21:41:06.860 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.860 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.860 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:06.860 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.392460+0000", 2024-09-21T21:41:06.860 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0096296730000000001 2024-09-21T21:41:06.860 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.860 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.860 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:06.860 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.392530+0000", 2024-09-21T21:41:06.860 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9590000000000003e-05 2024-09-21T21:41:06.861 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.861 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.861 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:06.861 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.392609+0000", 2024-09-21T21:41:06.861 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.8714000000000006e-05 2024-09-21T21:41:06.861 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:06.861 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:06.861 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:06.861 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.861 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.861 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:1806 3.12 3:4bfa5158:::rbd_data.112145fae779.0000000000000002:a [read 10596352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:06.861 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.382838+0000", 2024-09-21T21:41:06.861 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 8.4484224569999995, 2024-09-21T21:41:06.862 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0099573270000000002, 2024-09-21T21:41:06.862 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:06.862 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:06.862 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:06.862 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:06.862 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:06.862 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:06.862 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1806 2024-09-21T21:41:06.862 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.862 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:06.862 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.862 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:06.862 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382838+0000", 2024-09-21T21:41:06.862 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:06.863 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.863 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.863 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:06.863 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382838+0000", 2024-09-21T21:41:06.863 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:06.863 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.863 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.863 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:06.863 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382839+0000", 2024-09-21T21:41:06.863 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7599999999999997e-07 2024-09-21T21:41:06.863 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.863 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.863 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:06.863 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382841+0000", 2024-09-21T21:41:06.863 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3070000000000001e-06 2024-09-21T21:41:06.864 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.864 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.864 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:06.864 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382847+0000", 2024-09-21T21:41:06.864 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2870000000000001e-06 2024-09-21T21:41:06.864 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.864 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.864 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:06.864 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382852+0000", 2024-09-21T21:41:06.864 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5950000000000001e-06 2024-09-21T21:41:06.864 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.864 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.864 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:06.864 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.392651+0000", 2024-09-21T21:41:06.864 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0097987419999999992 2024-09-21T21:41:06.865 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.865 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.865 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:06.865 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.392731+0000", 2024-09-21T21:41:06.865 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0154999999999994e-05 2024-09-21T21:41:06.865 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.865 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.865 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:06.865 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.392795+0000", 2024-09-21T21:41:06.865 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4665000000000005e-05 2024-09-21T21:41:06.865 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:06.865 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:06.865 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:06.865 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.865 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.866 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:1807 3.12 3:4bfa5158:::rbd_data.112145fae779.0000000000000002:a [read 10629120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:06.866 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.382858+0000", 2024-09-21T21:41:06.866 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 8.4484024380000005, 2024-09-21T21:41:06.866 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.010090127000000001, 2024-09-21T21:41:06.866 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:06.866 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:06.866 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:06.866 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:06.866 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:06.866 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:06.866 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1807 2024-09-21T21:41:06.866 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.866 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:06.866 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.867 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:06.867 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382858+0000", 2024-09-21T21:41:06.867 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:06.867 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.867 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.867 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:06.867 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382858+0000", 2024-09-21T21:41:06.867 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:06.867 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.867 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.867 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:06.867 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382859+0000", 2024-09-21T21:41:06.867 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7599999999999997e-07 2024-09-21T21:41:06.867 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.867 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.867 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:06.868 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382861+0000", 2024-09-21T21:41:06.868 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.176e-06 2024-09-21T21:41:06.868 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.868 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.868 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:06.868 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382867+0000", 2024-09-21T21:41:06.868 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0990000000000004e-06 2024-09-21T21:41:06.868 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.868 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.868 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:06.868 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382872+0000", 2024-09-21T21:41:06.868 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8230000000000002e-06 2024-09-21T21:41:06.868 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.868 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.868 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:06.869 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.392829+0000", 2024-09-21T21:41:06.869 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0099573869999999998 2024-09-21T21:41:06.869 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.869 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.869 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:06.869 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.392888+0000", 2024-09-21T21:41:06.869 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8585000000000002e-05 2024-09-21T21:41:06.869 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.869 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.869 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:06.869 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.392948+0000", 2024-09-21T21:41:06.869 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0581000000000002e-05 2024-09-21T21:41:06.869 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:06.869 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:06.869 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:06.870 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.870 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.870 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:1809 3.12 3:4bfa5158:::rbd_data.112145fae779.0000000000000002:a [read 10686464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:06.870 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.382878+0000", 2024-09-21T21:41:06.870 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 8.4483825449999994, 2024-09-21T21:41:06.870 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.010264758000000001, 2024-09-21T21:41:06.870 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:06.870 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:06.870 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:06.870 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:06.870 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:06.870 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:06.870 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1809 2024-09-21T21:41:06.870 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.870 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:06.871 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.871 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:06.871 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382878+0000", 2024-09-21T21:41:06.871 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:06.871 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.871 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.871 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:06.871 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382878+0000", 2024-09-21T21:41:06.871 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:06.871 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.871 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.871 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:06.871 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382878+0000", 2024-09-21T21:41:06.871 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8000000000000001e-07 2024-09-21T21:41:06.871 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.872 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.872 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:06.872 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382880+0000", 2024-09-21T21:41:06.872 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0470000000000001e-06 2024-09-21T21:41:06.872 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.872 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.872 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:06.872 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382887+0000", 2024-09-21T21:41:06.872 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2899999999999999e-06 2024-09-21T21:41:06.872 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.872 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.872 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:06.872 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382891+0000", 2024-09-21T21:41:06.872 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.566e-06 2024-09-21T21:41:06.872 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.873 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.873 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:06.873 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.392978+0000", 2024-09-21T21:41:06.873 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.010086603 2024-09-21T21:41:06.873 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.873 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.873 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:06.873 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.393063+0000", 2024-09-21T21:41:06.873 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.5197000000000004e-05 2024-09-21T21:41:06.873 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.873 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.873 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:06.873 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.393143+0000", 2024-09-21T21:41:06.873 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.9674999999999999e-05 2024-09-21T21:41:06.873 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:06.874 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:06.874 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:06.874 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.874 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.874 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:1810 3.12 3:4bfa5158:::rbd_data.112145fae779.0000000000000002:a [read 10743808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:06.874 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.382898+0000", 2024-09-21T21:41:06.874 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 8.4483626570000006, 2024-09-21T21:41:06.874 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.010450245, 2024-09-21T21:41:06.874 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:06.874 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:06.874 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:06.874 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:06.874 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:06.874 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:06.875 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1810 2024-09-21T21:41:06.875 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.875 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:06.875 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.875 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:06.875 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382898+0000", 2024-09-21T21:41:06.875 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:06.875 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.875 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.875 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:06.875 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382898+0000", 2024-09-21T21:41:06.875 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:06.875 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.875 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.876 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:06.876 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382898+0000", 2024-09-21T21:41:06.876 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2e-07 2024-09-21T21:41:06.876 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.876 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.876 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:06.876 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382900+0000", 2024-09-21T21:41:06.876 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9439999999999999e-06 2024-09-21T21:41:06.876 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.876 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.876 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:06.876 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382906+0000", 2024-09-21T21:41:06.876 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.981e-06 2024-09-21T21:41:06.876 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.876 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.877 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:06.877 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382912+0000", 2024-09-21T21:41:06.877 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5389999999999998e-06 2024-09-21T21:41:06.877 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.877 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.877 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:06.877 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.393190+0000", 2024-09-21T21:41:06.877 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.010277846 2024-09-21T21:41:06.877 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.877 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.877 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:06.877 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.393271+0000", 2024-09-21T21:41:06.877 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.1666e-05 2024-09-21T21:41:06.877 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.878 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.878 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:06.878 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.393348+0000", 2024-09-21T21:41:06.878 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.6848999999999996e-05 2024-09-21T21:41:06.878 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:06.878 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:06.878 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:06.878 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.878 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.878 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:1856 3.12 3:4bfa5158:::rbd_data.112145fae779.0000000000000002:a [read 10240000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:06.878 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.382998+0000", 2024-09-21T21:41:06.878 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 8.4482621669999993, 2024-09-21T21:41:06.878 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.010571644, 2024-09-21T21:41:06.878 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:06.879 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:06.879 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:06.879 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:06.879 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:06.879 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:06.879 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1856 2024-09-21T21:41:06.879 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.879 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:06.879 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.879 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:06.879 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382998+0000", 2024-09-21T21:41:06.879 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:06.879 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.879 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.879 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:06.880 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382998+0000", 2024-09-21T21:41:06.880 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:06.880 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.880 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.880 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:06.880 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382999+0000", 2024-09-21T21:41:06.880 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7700000000000004e-07 2024-09-21T21:41:06.880 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.880 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.880 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:06.880 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383003+0000", 2024-09-21T21:41:06.880 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.822e-06 2024-09-21T21:41:06.880 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.880 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.880 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:06.881 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383014+0000", 2024-09-21T21:41:06.881 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0713000000000001e-05 2024-09-21T21:41:06.881 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.881 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.881 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:06.881 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383020+0000", 2024-09-21T21:41:06.881 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3160000000000002e-06 2024-09-21T21:41:06.881 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.881 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.881 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:06.881 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.393394+0000", 2024-09-21T21:41:06.881 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.010374362 2024-09-21T21:41:06.881 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.881 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.881 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:06.882 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.393484+0000", 2024-09-21T21:41:06.882 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.9975000000000005e-05 2024-09-21T21:41:06.882 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.882 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.882 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:06.882 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.393570+0000", 2024-09-21T21:41:06.882 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.5679000000000004e-05 2024-09-21T21:41:06.882 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:06.882 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:06.882 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:06.882 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.882 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.882 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:1858 3.12 3:4bfa5158:::rbd_data.112145fae779.0000000000000002:a [read 10297344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:06.883 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.383036+0000", 2024-09-21T21:41:06.883 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 8.4482249899999999, 2024-09-21T21:41:06.883 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.010660364, 2024-09-21T21:41:06.883 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:06.883 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:06.883 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:06.883 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:06.883 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:06.883 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:06.883 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1858 2024-09-21T21:41:06.883 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.883 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:06.883 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.883 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:06.884 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383036+0000", 2024-09-21T21:41:06.884 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:06.884 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.884 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.884 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:06.884 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383036+0000", 2024-09-21T21:41:06.884 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:06.884 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.884 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.884 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:06.884 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383036+0000", 2024-09-21T21:41:06.884 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8899999999999999e-07 2024-09-21T21:41:06.884 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.885 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.885 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:06.885 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383039+0000", 2024-09-21T21:41:06.885 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.6510000000000001e-06 2024-09-21T21:41:06.885 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.885 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.885 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:06.885 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383048+0000", 2024-09-21T21:41:06.885 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6530000000000006e-06 2024-09-21T21:41:06.885 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.885 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.885 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:06.885 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383053+0000", 2024-09-21T21:41:06.885 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.933e-06 2024-09-21T21:41:06.885 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.886 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.886 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:06.886 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.393608+0000", 2024-09-21T21:41:06.886 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.01055442 2024-09-21T21:41:06.886 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.886 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.886 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:06.886 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.393650+0000", 2024-09-21T21:41:06.886 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2586999999999997e-05 2024-09-21T21:41:06.886 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.886 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.886 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:06.886 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.393696+0000", 2024-09-21T21:41:06.886 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5531000000000002e-05 2024-09-21T21:41:06.887 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:06.887 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:06.887 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:06.887 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.887 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.887 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:1861 3.12 3:4bfa5158:::rbd_data.112145fae779.0000000000000002:a [read 10387456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:06.887 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.383063+0000", 2024-09-21T21:41:06.887 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 8.4481970699999991, 2024-09-21T21:41:06.887 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.010754104, 2024-09-21T21:41:06.887 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:06.887 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:06.887 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:06.887 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:06.887 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:06.888 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:06.888 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1861 2024-09-21T21:41:06.888 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.888 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:06.888 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.888 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:06.888 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383063+0000", 2024-09-21T21:41:06.888 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:06.888 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.888 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.888 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:06.888 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383063+0000", 2024-09-21T21:41:06.888 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:06.888 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.889 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.889 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:06.889 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383064+0000", 2024-09-21T21:41:06.889 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9399999999999995e-07 2024-09-21T21:41:06.889 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.889 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.889 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:06.889 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383067+0000", 2024-09-21T21:41:06.889 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.6019999999999998e-06 2024-09-21T21:41:06.889 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.889 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.889 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:06.889 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383075+0000", 2024-09-21T21:41:06.889 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.1850000000000004e-06 2024-09-21T21:41:06.890 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.890 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.890 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:06.890 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383080+0000", 2024-09-21T21:41:06.890 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3909999999999997e-06 2024-09-21T21:41:06.890 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.890 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.890 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:06.890 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.393727+0000", 2024-09-21T21:41:06.890 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.010646631 2024-09-21T21:41:06.890 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.890 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.890 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:06.890 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.393777+0000", 2024-09-21T21:41:06.891 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0356999999999999e-05 2024-09-21T21:41:06.891 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.891 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.891 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:06.891 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.393818+0000", 2024-09-21T21:41:06.891 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.0444000000000001e-05 2024-09-21T21:41:06.891 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:06.891 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:06.891 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:06.891 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.891 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.891 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:1863 3.12 3:4bfa5158:::rbd_data.112145fae779.0000000000000002:a [read 10444800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:06.891 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.383088+0000", 2024-09-21T21:41:06.891 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 8.4481724259999993, 2024-09-21T21:41:06.891 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.010836438, 2024-09-21T21:41:06.892 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:06.892 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:06.892 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:06.892 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:06.892 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:06.892 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:06.892 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1863 2024-09-21T21:41:06.892 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.892 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:06.892 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.892 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:06.892 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383088+0000", 2024-09-21T21:41:06.892 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:06.892 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.892 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.893 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:06.893 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383088+0000", 2024-09-21T21:41:06.893 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:06.893 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.893 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.893 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:06.893 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383089+0000", 2024-09-21T21:41:06.893 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2500000000000001e-07 2024-09-21T21:41:06.893 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.893 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.893 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:06.893 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383091+0000", 2024-09-21T21:41:06.893 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.283e-06 2024-09-21T21:41:06.893 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.893 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.894 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:06.894 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383097+0000", 2024-09-21T21:41:06.894 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1279999999999996e-06 2024-09-21T21:41:06.894 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.894 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.894 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:06.894 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383103+0000", 2024-09-21T21:41:06.894 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6219999999999997e-06 2024-09-21T21:41:06.894 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.894 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.894 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:06.894 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.393845+0000", 2024-09-21T21:41:06.894 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.010742188999999999 2024-09-21T21:41:06.895 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.895 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.895 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:06.895 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.393886+0000", 2024-09-21T21:41:06.895 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1119999999999999e-05 2024-09-21T21:41:06.895 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.895 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.895 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:06.895 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.393925+0000", 2024-09-21T21:41:06.895 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8671000000000003e-05 2024-09-21T21:41:06.895 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:06.895 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:06.895 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:06.895 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.896 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.896 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:1865 3.12 3:4bfa5158:::rbd_data.112145fae779.0000000000000002:a [read 10534912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:06.896 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.383134+0000", 2024-09-21T21:41:06.896 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 8.4481266389999998, 2024-09-21T21:41:06.896 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.01092018, 2024-09-21T21:41:06.896 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:06.896 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:06.896 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:06.896 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:06.896 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:06.896 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:06.896 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1865 2024-09-21T21:41:06.896 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.896 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:06.896 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.897 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:06.897 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383134+0000", 2024-09-21T21:41:06.897 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:06.897 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.897 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.897 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:06.897 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383134+0000", 2024-09-21T21:41:06.897 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:06.897 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.897 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.897 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:06.897 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383135+0000", 2024-09-21T21:41:06.897 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.9800000000000002e-07 2024-09-21T21:41:06.898 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.898 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.898 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:06.898 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383140+0000", 2024-09-21T21:41:06.898 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7099999999999998e-06 2024-09-21T21:41:06.898 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.898 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.898 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:06.898 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383150+0000", 2024-09-21T21:41:06.898 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0472000000000001e-05 2024-09-21T21:41:06.898 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.898 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.898 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:06.898 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383157+0000", 2024-09-21T21:41:06.899 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9029999999999998e-06 2024-09-21T21:41:06.899 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.899 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.899 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:06.899 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.393973+0000", 2024-09-21T21:41:06.899 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.010816299 2024-09-21T21:41:06.899 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.899 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.899 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:06.899 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.394015+0000", 2024-09-21T21:41:06.899 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1467000000000001e-05 2024-09-21T21:41:06.899 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.899 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.899 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:06.900 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.394054+0000", 2024-09-21T21:41:06.900 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.9331e-05 2024-09-21T21:41:06.900 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:06.900 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:06.900 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:06.900 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.900 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.900 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:1867 3.12 3:4bfa5158:::rbd_data.112145fae779.0000000000000002:a [read 10592256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:06.900 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.383287+0000", 2024-09-21T21:41:06.900 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 8.4479733140000004, 2024-09-21T21:41:06.900 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.01089415, 2024-09-21T21:41:06.900 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:06.900 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:06.900 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:06.901 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:06.901 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:06.901 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:06.901 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1867 2024-09-21T21:41:06.901 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.901 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:06.901 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.901 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:06.901 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383287+0000", 2024-09-21T21:41:06.901 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:06.901 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.901 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.901 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:06.901 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383287+0000", 2024-09-21T21:41:06.901 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:06.902 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.902 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.902 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:06.902 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383288+0000", 2024-09-21T21:41:06.902 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.02e-07 2024-09-21T21:41:06.902 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.902 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.902 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:06.902 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383293+0000", 2024-09-21T21:41:06.902 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8339999999999998e-06 2024-09-21T21:41:06.902 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.902 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.902 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:06.902 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383305+0000", 2024-09-21T21:41:06.903 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1677e-05 2024-09-21T21:41:06.903 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.903 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.903 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:06.903 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383312+0000", 2024-09-21T21:41:06.903 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9079999999999998e-06 2024-09-21T21:41:06.903 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.903 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.903 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:06.903 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.394080+0000", 2024-09-21T21:41:06.903 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.010768402 2024-09-21T21:41:06.903 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.903 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.903 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:06.903 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.394131+0000", 2024-09-21T21:41:06.904 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0926e-05 2024-09-21T21:41:06.904 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.904 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.904 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:06.904 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.394181+0000", 2024-09-21T21:41:06.904 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0501000000000003e-05 2024-09-21T21:41:06.904 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:06.904 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:06.904 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:06.904 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.904 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.904 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:1869 3.12 3:4bfa5158:::rbd_data.112145fae779.0000000000000002:a [read 10625024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:06.904 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.383348+0000", 2024-09-21T21:41:06.904 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 8.4479127629999997, 2024-09-21T21:41:06.905 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.011004575000000001, 2024-09-21T21:41:06.905 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:06.905 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:06.905 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:06.905 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:06.905 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:06.905 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:06.905 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1869 2024-09-21T21:41:06.905 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.905 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:06.905 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.905 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:06.905 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383348+0000", 2024-09-21T21:41:06.905 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:06.905 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.906 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.906 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:06.906 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383348+0000", 2024-09-21T21:41:06.906 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:06.906 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.906 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.906 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:06.906 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383348+0000", 2024-09-21T21:41:06.906 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5400000000000001e-07 2024-09-21T21:41:06.906 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.906 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.906 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:06.906 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383352+0000", 2024-09-21T21:41:06.906 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4089999999999999e-06 2024-09-21T21:41:06.907 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.907 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.907 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:06.907 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383360+0000", 2024-09-21T21:41:06.907 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.2430000000000005e-06 2024-09-21T21:41:06.907 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.907 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.907 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:06.907 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383366+0000", 2024-09-21T21:41:06.907 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4909999999999996e-06 2024-09-21T21:41:06.907 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.907 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.907 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:06.907 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.394212+0000", 2024-09-21T21:41:06.908 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.010846000999999999 2024-09-21T21:41:06.908 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.908 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.908 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:06.908 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.394279+0000", 2024-09-21T21:41:06.908 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.7873000000000006e-05 2024-09-21T21:41:06.908 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.908 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.908 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:06.908 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.394352+0000", 2024-09-21T21:41:06.908 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.2904000000000005e-05 2024-09-21T21:41:06.908 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:06.908 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:06.908 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:06.909 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.909 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.909 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:1871 3.12 3:4bfa5158:::rbd_data.112145fae779.0000000000000002:a [read 10682368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:06.909 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.383497+0000", 2024-09-21T21:41:06.909 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 8.4477637829999992, 2024-09-21T21:41:06.909 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.010980227, 2024-09-21T21:41:06.909 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:06.909 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:06.909 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:06.909 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:06.909 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:06.909 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:06.909 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1871 2024-09-21T21:41:06.909 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.909 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:06.910 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.910 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:06.910 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383497+0000", 2024-09-21T21:41:06.910 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:06.910 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.910 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.910 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:06.910 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383497+0000", 2024-09-21T21:41:06.910 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:06.910 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.910 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.910 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:06.910 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383498+0000", 2024-09-21T21:41:06.911 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2500000000000004e-07 2024-09-21T21:41:06.911 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.911 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.911 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:06.911 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383502+0000", 2024-09-21T21:41:06.911 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7049999999999998e-06 2024-09-21T21:41:06.911 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.911 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.911 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:06.911 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383512+0000", 2024-09-21T21:41:06.911 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.3239999999999992e-06 2024-09-21T21:41:06.911 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.911 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.911 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:06.911 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383518+0000", 2024-09-21T21:41:06.912 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5370000000000003e-06 2024-09-21T21:41:06.912 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.912 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.912 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:06.912 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.394378+0000", 2024-09-21T21:41:06.912 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.010859897 2024-09-21T21:41:06.912 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.912 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.912 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:06.912 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.394434+0000", 2024-09-21T21:41:06.912 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6198e-05 2024-09-21T21:41:06.912 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.912 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.912 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:06.913 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.394477+0000", 2024-09-21T21:41:06.913 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2641000000000001e-05 2024-09-21T21:41:06.913 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:06.913 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:06.913 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:06.913 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.913 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.913 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:1873 3.12 3:4bfa5158:::rbd_data.112145fae779.0000000000000002:a [read 10739712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:06.913 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.383639+0000", 2024-09-21T21:41:06.913 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 8.4476220289999997, 2024-09-21T21:41:06.913 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.011039125, 2024-09-21T21:41:06.913 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:06.913 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:06.913 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:06.913 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:06.914 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:06.914 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:06.914 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1873 2024-09-21T21:41:06.914 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.914 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:06.914 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.914 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:06.914 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383639+0000", 2024-09-21T21:41:06.914 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:06.914 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.914 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.914 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:06.914 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383639+0000", 2024-09-21T21:41:06.914 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:06.915 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.915 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.915 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:06.915 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383639+0000", 2024-09-21T21:41:06.915 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.8000000000000004e-07 2024-09-21T21:41:06.915 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.915 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.915 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:06.915 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383644+0000", 2024-09-21T21:41:06.915 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3699999999999997e-06 2024-09-21T21:41:06.915 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.915 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.915 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:06.915 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383653+0000", 2024-09-21T21:41:06.916 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.3330000000000003e-06 2024-09-21T21:41:06.916 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.916 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.916 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:06.916 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383659+0000", 2024-09-21T21:41:06.916 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7060000000000001e-06 2024-09-21T21:41:06.916 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.916 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.916 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:06.916 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.394518+0000", 2024-09-21T21:41:06.916 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.010859647 2024-09-21T21:41:06.916 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.916 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.916 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:06.916 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.394599+0000", 2024-09-21T21:41:06.917 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0770000000000001e-05 2024-09-21T21:41:06.917 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.917 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.917 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:06.917 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.394678+0000", 2024-09-21T21:41:06.917 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.8418999999999996e-05 2024-09-21T21:41:06.917 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:06.917 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:06.917 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:06.917 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.917 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.917 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4491 3.12 3:4bfa5158:::rbd_data.112145fae779.0000000000000002:a [read 2269184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:06.917 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.513914+0000", 2024-09-21T21:41:06.917 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 8.3173463769999998, 2024-09-21T21:41:06.918 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0098656990000000003, 2024-09-21T21:41:06.918 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:06.918 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:06.918 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:06.918 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:06.918 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:06.918 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:06.918 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4491 2024-09-21T21:41:06.918 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.918 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:06.918 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.918 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:06.918 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.513914+0000", 2024-09-21T21:41:06.918 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:06.919 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.919 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.919 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:06.919 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.513914+0000", 2024-09-21T21:41:06.919 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:06.919 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.919 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.919 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:06.919 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.513914+0000", 2024-09-21T21:41:06.919 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.11e-07 2024-09-21T21:41:06.919 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.919 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.919 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:06.919 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.513916+0000", 2024-09-21T21:41:06.920 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1799999999999999e-06 2024-09-21T21:41:06.920 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.920 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.920 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:06.920 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.513919+0000", 2024-09-21T21:41:06.920 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5099999999999999e-06 2024-09-21T21:41:06.920 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.920 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.920 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:06.920 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.513922+0000", 2024-09-21T21:41:06.920 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.429e-06 2024-09-21T21:41:06.920 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.920 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.920 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:06.920 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.521609+0000", 2024-09-21T21:41:06.921 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0076873979999999998 2024-09-21T21:41:06.921 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.921 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.921 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:06.921 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.521700+0000", 2024-09-21T21:41:06.921 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.0675999999999997e-05 2024-09-21T21:41:06.921 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:06.921 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:06.921 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:06.921 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.523780+0000", 2024-09-21T21:41:06.921 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0020802949999999998 2024-09-21T21:41:06.921 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:06.921 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:06.921 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:06.921 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:06.922 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:06.922 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:06.924 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T21:41:06.924 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:06.921+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:06.924 INFO:teuthology.orchestra.run.smithi164.stdout:1617 punch from 0x5f9000 to 0x5fd400, (0x4400 bytes) 2024-09-21T21:41:06.924 INFO:teuthology.orchestra.run.smithi164.stdout:1618 punch from 0x58cc00 to 0x58de00, (0x1200 bytes) 2024-09-21T21:41:06.924 INFO:teuthology.orchestra.run.smithi164.stdout:1619 punch from 0x56800 to 0x64a00, (0xe200 bytes) 2024-09-21T21:41:06.924 INFO:teuthology.orchestra.run.smithi164.stdout:1621 trunc from 0x658400 to 0x3c90600 2024-09-21T21:41:06.925 INFO:teuthology.orchestra.run.smithi164.stdout:1622 read 0xd98a00 thru 0xda3345 (0xa946 bytes) 2024-09-21T21:41:06.925 INFO:teuthology.orchestra.run.smithi164.stdout:1623 read 0xcf3600 thru 0xcf8896 (0x5297 bytes) 2024-09-21T21:41:06.925 INFO:teuthology.orchestra.run.smithi164.stdout:1625 write 0x2207000 thru 0x220872f (0x1730 bytes) 2024-09-21T21:41:06.925 INFO:teuthology.orchestra.run.smithi164.stdout:1626 clone 12 order 19 su 32768 sc 15 2024-09-21T21:41:07.024 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:07.021+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:07.067 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:07.068 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:41:07.068 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 0 2024-09-21T21:41:07.068 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:07.076 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T21:41:07.125 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:07.124+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:07.219 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:07.219 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:41:07.220 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:41:07.220 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:41:07.220 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:07.226 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:07.225+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:07.228 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T21:41:07.327 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:07.325+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:07.332 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:07.332 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 20, 2024-09-21T21:41:07.332 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 600, 2024-09-21T21:41:07.332 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:41:07.332 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.332 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:2885 3.b 3:d49b370f:::rbd_object_map.1121e540c096:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:07.332 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.478228+0000", 2024-09-21T21:41:07.332 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 8.8517732450000004, 2024-09-21T21:41:07.332 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.015327835999999999, 2024-09-21T21:41:07.333 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:07.333 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:07.333 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:41:07.333 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:07.333 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:07.333 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:07.333 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2885 2024-09-21T21:41:07.333 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.333 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:07.333 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.333 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:07.333 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478228+0000", 2024-09-21T21:41:07.333 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:07.333 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.334 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.334 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:07.334 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478228+0000", 2024-09-21T21:41:07.334 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:07.334 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.334 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.334 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:07.334 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478229+0000", 2024-09-21T21:41:07.334 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8000000000000001e-07 2024-09-21T21:41:07.334 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.334 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.334 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:07.334 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478231+0000", 2024-09-21T21:41:07.334 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.5069999999999999e-06 2024-09-21T21:41:07.335 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.335 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.335 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:07.335 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478238+0000", 2024-09-21T21:41:07.335 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.8820000000000003e-06 2024-09-21T21:41:07.335 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.335 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.335 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:07.335 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478243+0000", 2024-09-21T21:41:07.335 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2209999999999996e-06 2024-09-21T21:41:07.335 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.335 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.335 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:07.335 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.488160+0000", 2024-09-21T21:41:07.336 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0099169849999999997 2024-09-21T21:41:07.336 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.336 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.336 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:07.336 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.488231+0000", 2024-09-21T21:41:07.336 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0909999999999997e-05 2024-09-21T21:41:07.336 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.336 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.336 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 0", 2024-09-21T21:41:07.336 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.488538+0000", 2024-09-21T21:41:07.336 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000306736 2024-09-21T21:41:07.336 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.336 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.336 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:41:07.337 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.493210+0000", 2024-09-21T21:41:07.337 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0046722120000000002 2024-09-21T21:41:07.337 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.337 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.337 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:41:07.337 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.493471+0000", 2024-09-21T21:41:07.337 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00026132899999999998 2024-09-21T21:41:07.337 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.337 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.337 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:41:07.337 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.493517+0000", 2024-09-21T21:41:07.337 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5309000000000003e-05 2024-09-21T21:41:07.337 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.337 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.337 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:07.338 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.493556+0000", 2024-09-21T21:41:07.338 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.9365e-05 2024-09-21T21:41:07.338 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:07.338 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:07.338 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:07.338 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.338 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.338 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:2886 3.b 3:d49b370f:::rbd_object_map.1121e540c096:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:07.338 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.478249+0000", 2024-09-21T21:41:07.338 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 8.8517524650000006, 2024-09-21T21:41:07.338 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.015426120999999999, 2024-09-21T21:41:07.338 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:07.338 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:07.339 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:41:07.339 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:07.339 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:07.339 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:07.339 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2886 2024-09-21T21:41:07.339 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.339 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:07.339 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.339 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:07.339 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478249+0000", 2024-09-21T21:41:07.339 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:07.339 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.339 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.339 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:07.339 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478249+0000", 2024-09-21T21:41:07.340 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:07.340 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.340 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.340 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:07.340 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478249+0000", 2024-09-21T21:41:07.340 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.9999999999999998e-07 2024-09-21T21:41:07.340 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.340 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.340 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:07.341 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478252+0000", 2024-09-21T21:41:07.341 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.5500000000000001e-06 2024-09-21T21:41:07.341 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.341 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.341 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:07.341 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478258+0000", 2024-09-21T21:41:07.341 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7080000000000002e-06 2024-09-21T21:41:07.341 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.341 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.341 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:07.341 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478265+0000", 2024-09-21T21:41:07.341 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9110000000000004e-06 2024-09-21T21:41:07.341 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.341 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.342 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:07.342 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.488889+0000", 2024-09-21T21:41:07.342 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.010624662 2024-09-21T21:41:07.342 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.342 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.342 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:07.342 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.488965+0000", 2024-09-21T21:41:07.342 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5949000000000001e-05 2024-09-21T21:41:07.342 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.342 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.343 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 0", 2024-09-21T21:41:07.343 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.489235+0000", 2024-09-21T21:41:07.343 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00027019499999999998 2024-09-21T21:41:07.343 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.343 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.343 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:41:07.343 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.493233+0000", 2024-09-21T21:41:07.343 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0039978799999999997 2024-09-21T21:41:07.343 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.343 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.343 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:41:07.344 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.493601+0000", 2024-09-21T21:41:07.344 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00036728200000000003 2024-09-21T21:41:07.344 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.344 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.344 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:41:07.344 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.493637+0000", 2024-09-21T21:41:07.344 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6263000000000002e-05 2024-09-21T21:41:07.344 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.344 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.344 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:07.344 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.493675+0000", 2024-09-21T21:41:07.344 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8321000000000001e-05 2024-09-21T21:41:07.345 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:07.345 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:07.345 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:07.345 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.345 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.345 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:2887 3.b 3:d49b370f:::rbd_object_map.1121e540c096:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:07.345 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.478271+0000", 2024-09-21T21:41:07.345 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 8.8517309530000006, 2024-09-21T21:41:07.345 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.015551966, 2024-09-21T21:41:07.345 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:07.345 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:07.345 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:41:07.345 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:07.345 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:07.345 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:07.346 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2887 2024-09-21T21:41:07.346 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.346 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:07.346 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.346 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:07.346 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478271+0000", 2024-09-21T21:41:07.346 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:07.346 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.346 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.346 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:07.346 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478271+0000", 2024-09-21T21:41:07.346 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:07.346 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.346 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.347 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:07.347 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478271+0000", 2024-09-21T21:41:07.347 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.0600000000000001e-07 2024-09-21T21:41:07.347 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.347 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.347 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:07.347 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478274+0000", 2024-09-21T21:41:07.347 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.6390000000000001e-06 2024-09-21T21:41:07.347 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.347 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.348 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:07.348 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478280+0000", 2024-09-21T21:41:07.348 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3219999999999998e-06 2024-09-21T21:41:07.348 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.348 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.348 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:07.348 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478285+0000", 2024-09-21T21:41:07.348 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6589999999999998e-06 2024-09-21T21:41:07.348 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.348 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.348 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:07.348 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.489524+0000", 2024-09-21T21:41:07.348 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.01123979 2024-09-21T21:41:07.348 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.349 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.349 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:07.349 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.489567+0000", 2024-09-21T21:41:07.349 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2673000000000003e-05 2024-09-21T21:41:07.349 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.349 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.349 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 0", 2024-09-21T21:41:07.349 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.489834+0000", 2024-09-21T21:41:07.349 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00026672799999999998 2024-09-21T21:41:07.349 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.349 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.349 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:41:07.349 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.493290+0000", 2024-09-21T21:41:07.350 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0034557429999999998 2024-09-21T21:41:07.350 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.350 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.350 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:41:07.350 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.493737+0000", 2024-09-21T21:41:07.350 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00044731799999999999 2024-09-21T21:41:07.350 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.350 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.350 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:41:07.351 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.493784+0000", 2024-09-21T21:41:07.351 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7178999999999997e-05 2024-09-21T21:41:07.351 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.351 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.351 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:07.351 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.493823+0000", 2024-09-21T21:41:07.351 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8509e-05 2024-09-21T21:41:07.351 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:07.351 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:07.351 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:07.351 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.351 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.351 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4325 3.0 3:074cff57:::rbd_data.112145fae779.000000000000000c:a [read 2064384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:07.352 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.510141+0000", 2024-09-21T21:41:07.352 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 8.819860791, 2024-09-21T21:41:07.352 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013330887, 2024-09-21T21:41:07.352 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:07.352 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:07.352 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:07.352 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:07.352 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:07.352 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:07.352 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4325 2024-09-21T21:41:07.352 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.352 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:07.352 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.352 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:07.353 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510141+0000", 2024-09-21T21:41:07.353 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:07.353 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.353 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.353 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:07.353 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510141+0000", 2024-09-21T21:41:07.353 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:07.353 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.353 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.353 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:07.353 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510141+0000", 2024-09-21T21:41:07.353 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.72e-07 2024-09-21T21:41:07.353 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.353 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.354 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:07.354 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510143+0000", 2024-09-21T21:41:07.354 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2409999999999998e-06 2024-09-21T21:41:07.354 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.354 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.354 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:07.354 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510151+0000", 2024-09-21T21:41:07.354 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.3390000000000004e-06 2024-09-21T21:41:07.354 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.354 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.354 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:07.354 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510155+0000", 2024-09-21T21:41:07.354 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5630000000000002e-06 2024-09-21T21:41:07.354 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.355 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.355 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:07.355 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.523310+0000", 2024-09-21T21:41:07.355 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013154658 2024-09-21T21:41:07.355 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.355 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.355 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:07.355 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.523386+0000", 2024-09-21T21:41:07.355 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.6553999999999999e-05 2024-09-21T21:41:07.355 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.355 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.355 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:07.355 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.523472+0000", 2024-09-21T21:41:07.356 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.5160000000000005e-05 2024-09-21T21:41:07.356 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:07.356 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:07.356 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:07.356 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.356 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.356 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4326 3.0 3:074cff57:::rbd_data.112145fae779.000000000000000c:a [read 2097152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:07.356 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.510162+0000", 2024-09-21T21:41:07.356 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 8.8198397760000002, 2024-09-21T21:41:07.356 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013537466, 2024-09-21T21:41:07.356 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:07.356 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:07.356 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:07.356 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:07.357 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:07.357 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:07.357 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4326 2024-09-21T21:41:07.357 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.357 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:07.357 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.357 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:07.357 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510162+0000", 2024-09-21T21:41:07.357 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:07.357 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.357 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.357 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:07.357 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510162+0000", 2024-09-21T21:41:07.357 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:07.358 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.358 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.358 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:07.358 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510162+0000", 2024-09-21T21:41:07.358 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.0699999999999998e-07 2024-09-21T21:41:07.358 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.358 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.358 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:07.358 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510164+0000", 2024-09-21T21:41:07.358 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.17e-06 2024-09-21T21:41:07.358 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.358 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.358 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:07.358 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510174+0000", 2024-09-21T21:41:07.359 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2439999999999999e-06 2024-09-21T21:41:07.359 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.359 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.359 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:07.359 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510179+0000", 2024-09-21T21:41:07.359 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2159999999999997e-06 2024-09-21T21:41:07.359 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.359 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.359 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:07.359 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.523502+0000", 2024-09-21T21:41:07.359 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013322779 2024-09-21T21:41:07.359 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.359 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.359 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:07.360 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.523579+0000", 2024-09-21T21:41:07.360 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7402000000000002e-05 2024-09-21T21:41:07.360 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.360 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.360 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:07.360 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.523699+0000", 2024-09-21T21:41:07.360 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000120248 2024-09-21T21:41:07.360 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:07.360 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:07.360 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:07.360 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.360 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.360 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4327 3.0 3:074cff57:::rbd_data.112145fae779.000000000000000c:a [read 2154496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:07.360 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.510418+0000", 2024-09-21T21:41:07.361 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 8.8195833409999995, 2024-09-21T21:41:07.361 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013505039999999999, 2024-09-21T21:41:07.361 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:07.361 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:07.361 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:07.361 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:07.361 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:07.361 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:07.361 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4327 2024-09-21T21:41:07.361 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.361 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:07.361 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.361 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:07.362 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510418+0000", 2024-09-21T21:41:07.362 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:07.362 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.362 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.362 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:07.362 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510418+0000", 2024-09-21T21:41:07.362 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:07.362 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.362 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.362 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:07.362 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510419+0000", 2024-09-21T21:41:07.362 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8999999999999997e-07 2024-09-21T21:41:07.362 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.362 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.363 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:07.363 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510421+0000", 2024-09-21T21:41:07.363 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.4779999999999998e-06 2024-09-21T21:41:07.363 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.363 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.363 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:07.363 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510427+0000", 2024-09-21T21:41:07.363 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5609999999999998e-06 2024-09-21T21:41:07.363 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.363 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.363 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:07.363 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510432+0000", 2024-09-21T21:41:07.363 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5389999999999998e-06 2024-09-21T21:41:07.363 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.364 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.364 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:07.364 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.523743+0000", 2024-09-21T21:41:07.364 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013310963 2024-09-21T21:41:07.364 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.364 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.364 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:07.364 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.523824+0000", 2024-09-21T21:41:07.364 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0463000000000005e-05 2024-09-21T21:41:07.364 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.364 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.364 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:07.364 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.523923+0000", 2024-09-21T21:41:07.365 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.9545999999999999e-05 2024-09-21T21:41:07.365 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:07.365 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:07.365 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:07.365 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.365 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.365 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4328 3.0 3:074cff57:::rbd_data.112145fae779.000000000000000c:a [read 2211840~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:07.365 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.510443+0000", 2024-09-21T21:41:07.365 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 8.8195590409999998, 2024-09-21T21:41:07.365 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013684564, 2024-09-21T21:41:07.365 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:07.365 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:07.365 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:07.365 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:07.366 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:07.366 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:07.366 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4328 2024-09-21T21:41:07.366 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.366 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:07.366 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.366 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:07.366 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510443+0000", 2024-09-21T21:41:07.366 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:07.366 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.366 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.366 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:07.366 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510443+0000", 2024-09-21T21:41:07.366 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:07.366 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.367 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.367 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:07.367 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510443+0000", 2024-09-21T21:41:07.367 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2199999999999999e-07 2024-09-21T21:41:07.367 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.367 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.367 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:07.367 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510445+0000", 2024-09-21T21:41:07.367 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1679999999999998e-06 2024-09-21T21:41:07.367 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.367 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.367 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:07.367 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510452+0000", 2024-09-21T21:41:07.367 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6259999999999997e-06 2024-09-21T21:41:07.368 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.368 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.368 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:07.368 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510457+0000", 2024-09-21T21:41:07.368 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8110000000000002e-06 2024-09-21T21:41:07.368 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.368 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.368 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:07.368 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.523968+0000", 2024-09-21T21:41:07.368 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013510996000000001 2024-09-21T21:41:07.368 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.368 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.368 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:07.368 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.524044+0000", 2024-09-21T21:41:07.368 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.6259000000000003e-05 2024-09-21T21:41:07.369 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.369 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.369 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:07.369 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.524127+0000", 2024-09-21T21:41:07.369 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.3282000000000007e-05 2024-09-21T21:41:07.369 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:07.369 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:07.369 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:07.369 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.369 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.369 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4577 3.0 3:074cff57:::rbd_data.112145fae779.000000000000000c:a [read 1744896~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:07.369 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.514327+0000", 2024-09-21T21:41:07.369 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 8.8156743839999994, 2024-09-21T21:41:07.369 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013451424999999999, 2024-09-21T21:41:07.369 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:07.370 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:07.370 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:07.370 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:07.370 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:07.370 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:07.370 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4577 2024-09-21T21:41:07.370 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.370 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:07.370 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.370 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:07.370 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514327+0000", 2024-09-21T21:41:07.370 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:07.370 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.371 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.371 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:07.371 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514327+0000", 2024-09-21T21:41:07.371 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:07.371 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.371 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.371 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:07.371 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514328+0000", 2024-09-21T21:41:07.371 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4900000000000001e-07 2024-09-21T21:41:07.371 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.371 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.371 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:07.371 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514330+0000", 2024-09-21T21:41:07.371 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.227e-06 2024-09-21T21:41:07.372 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.372 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.372 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:07.372 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514336+0000", 2024-09-21T21:41:07.372 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1940000000000003e-06 2024-09-21T21:41:07.372 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.372 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.372 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:07.372 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514341+0000", 2024-09-21T21:41:07.372 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7319999999999998e-06 2024-09-21T21:41:07.372 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.372 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.372 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:07.372 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.527415+0000", 2024-09-21T21:41:07.373 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013074657 2024-09-21T21:41:07.373 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.373 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.373 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:07.373 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.527491+0000", 2024-09-21T21:41:07.373 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5899e-05 2024-09-21T21:41:07.373 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.373 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.373 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:07.373 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.527779+0000", 2024-09-21T21:41:07.373 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00028736700000000002 2024-09-21T21:41:07.373 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:07.373 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:07.373 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:07.373 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.374 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.374 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4578 3.0 3:074cff57:::rbd_data.112145fae779.000000000000000c:a [read 1802240~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:07.374 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.514347+0000", 2024-09-21T21:41:07.374 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 8.8156549290000008, 2024-09-21T21:41:07.374 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.014025592999999999, 2024-09-21T21:41:07.374 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:07.374 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:07.374 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:07.374 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:07.374 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:07.374 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:07.374 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4578 2024-09-21T21:41:07.374 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.374 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:07.375 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.375 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:07.375 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514347+0000", 2024-09-21T21:41:07.375 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:07.375 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.375 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.375 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:07.375 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514347+0000", 2024-09-21T21:41:07.375 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:07.375 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.375 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.375 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:07.375 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514347+0000", 2024-09-21T21:41:07.376 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6699999999999999e-07 2024-09-21T21:41:07.376 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.376 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.376 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:07.376 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514349+0000", 2024-09-21T21:41:07.376 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0109999999999999e-06 2024-09-21T21:41:07.376 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.376 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.376 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:07.376 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514355+0000", 2024-09-21T21:41:07.376 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1090000000000003e-06 2024-09-21T21:41:07.376 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.376 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.376 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:07.377 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514360+0000", 2024-09-21T21:41:07.377 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5900000000000001e-06 2024-09-21T21:41:07.377 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.377 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.377 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:07.377 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.527862+0000", 2024-09-21T21:41:07.377 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013501812 2024-09-21T21:41:07.377 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.377 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.377 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:07.377 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.527933+0000", 2024-09-21T21:41:07.377 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.1322000000000006e-05 2024-09-21T21:41:07.377 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.377 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.377 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:07.378 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.528372+0000", 2024-09-21T21:41:07.378 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00043938199999999999 2024-09-21T21:41:07.378 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:07.378 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:07.378 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:07.378 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.378 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.378 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4579 3.0 3:074cff57:::rbd_data.112145fae779.000000000000000c:a [read 1835008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:07.378 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.514365+0000", 2024-09-21T21:41:07.378 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 8.8156360720000002, 2024-09-21T21:41:07.378 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.014158828, 2024-09-21T21:41:07.378 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:07.378 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:07.379 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:07.379 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:07.379 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:07.379 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:07.379 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4579 2024-09-21T21:41:07.379 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.379 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:07.379 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.379 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:07.379 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514365+0000", 2024-09-21T21:41:07.379 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:07.379 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.379 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.379 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:07.380 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514365+0000", 2024-09-21T21:41:07.380 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:07.380 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.380 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.380 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:07.380 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514366+0000", 2024-09-21T21:41:07.380 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7800000000000002e-07 2024-09-21T21:41:07.380 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.380 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.380 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:07.380 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514368+0000", 2024-09-21T21:41:07.380 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9920000000000002e-06 2024-09-21T21:41:07.380 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.380 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.380 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:07.381 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514374+0000", 2024-09-21T21:41:07.381 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9370000000000001e-06 2024-09-21T21:41:07.381 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.381 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.381 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:07.381 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514379+0000", 2024-09-21T21:41:07.381 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7149999999999997e-06 2024-09-21T21:41:07.381 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.381 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.381 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:07.381 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.528303+0000", 2024-09-21T21:41:07.381 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013924076000000001 2024-09-21T21:41:07.381 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.382 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.382 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:07.382 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.528429+0000", 2024-09-21T21:41:07.382 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00012643100000000001 2024-09-21T21:41:07.382 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.382 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.382 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:07.382 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.528524+0000", 2024-09-21T21:41:07.382 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.5298999999999997e-05 2024-09-21T21:41:07.382 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:07.382 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:07.382 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:07.382 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.382 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.383 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4580 3.0 3:074cff57:::rbd_data.112145fae779.000000000000000c:a [read 1892352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:07.383 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.514384+0000", 2024-09-21T21:41:07.383 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 8.8156174759999999, 2024-09-21T21:41:07.383 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.014365243, 2024-09-21T21:41:07.383 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:07.383 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:07.383 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:07.383 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:07.383 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:07.383 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:07.383 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4580 2024-09-21T21:41:07.383 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.383 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:07.384 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.384 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:07.384 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514384+0000", 2024-09-21T21:41:07.384 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:07.384 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.384 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.384 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:07.384 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514384+0000", 2024-09-21T21:41:07.384 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:07.384 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.384 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.384 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:07.384 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514384+0000", 2024-09-21T21:41:07.384 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1100000000000001e-07 2024-09-21T21:41:07.385 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.385 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.385 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:07.385 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514387+0000", 2024-09-21T21:41:07.385 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1150000000000001e-06 2024-09-21T21:41:07.385 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.385 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.385 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:07.385 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514393+0000", 2024-09-21T21:41:07.385 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1140000000000002e-06 2024-09-21T21:41:07.385 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.385 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.385 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:07.385 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514398+0000", 2024-09-21T21:41:07.386 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6330000000000001e-06 2024-09-21T21:41:07.386 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.386 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.386 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:07.386 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.528569+0000", 2024-09-21T21:41:07.386 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.014171011000000001 2024-09-21T21:41:07.386 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.386 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.386 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:07.386 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.528642+0000", 2024-09-21T21:41:07.386 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.2336000000000007e-05 2024-09-21T21:41:07.386 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.386 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.386 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:07.387 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.528749+0000", 2024-09-21T21:41:07.387 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000107623 2024-09-21T21:41:07.387 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:07.387 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:07.387 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:07.387 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.387 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.387 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4581 3.0 3:074cff57:::rbd_data.112145fae779.000000000000000c:a [read 1949696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:07.387 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.514404+0000", 2024-09-21T21:41:07.387 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 8.8155973089999993, 2024-09-21T21:41:07.387 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.014551563999999999, 2024-09-21T21:41:07.387 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:07.387 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:07.387 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:07.388 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:07.388 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:07.388 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:07.388 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4581 2024-09-21T21:41:07.388 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.388 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:07.388 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.388 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:07.388 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514404+0000", 2024-09-21T21:41:07.388 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:07.388 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.388 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.388 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:07.388 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514404+0000", 2024-09-21T21:41:07.389 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:07.389 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.389 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.389 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:07.389 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514405+0000", 2024-09-21T21:41:07.389 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7599999999999998e-07 2024-09-21T21:41:07.389 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.389 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.389 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:07.389 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514407+0000", 2024-09-21T21:41:07.389 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0169999999999999e-06 2024-09-21T21:41:07.389 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.389 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.390 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:07.390 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514413+0000", 2024-09-21T21:41:07.390 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0020000000000003e-06 2024-09-21T21:41:07.390 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.390 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.390 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:07.390 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514417+0000", 2024-09-21T21:41:07.390 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7319999999999998e-06 2024-09-21T21:41:07.390 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.390 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.390 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:07.390 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.528781+0000", 2024-09-21T21:41:07.390 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.014363433 2024-09-21T21:41:07.390 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.391 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.391 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:07.391 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.528861+0000", 2024-09-21T21:41:07.391 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0408000000000007e-05 2024-09-21T21:41:07.391 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.391 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.391 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:07.391 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.528956+0000", 2024-09-21T21:41:07.391 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.4596e-05 2024-09-21T21:41:07.391 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:07.391 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:07.391 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:07.391 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.391 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.392 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4582 3.0 3:074cff57:::rbd_data.112145fae779.000000000000000c:a [read 1982464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:07.392 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.514423+0000", 2024-09-21T21:41:07.392 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 8.815578425, 2024-09-21T21:41:07.392 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.014719965, 2024-09-21T21:41:07.392 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:07.392 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:07.392 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:07.392 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:07.392 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:07.392 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:07.392 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4582 2024-09-21T21:41:07.392 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.392 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:07.392 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.392 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:07.393 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514423+0000", 2024-09-21T21:41:07.393 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:07.393 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.393 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.393 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:07.393 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514423+0000", 2024-09-21T21:41:07.393 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:07.393 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.393 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.393 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:07.393 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514423+0000", 2024-09-21T21:41:07.393 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.53e-07 2024-09-21T21:41:07.393 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.393 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.394 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:07.394 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514426+0000", 2024-09-21T21:41:07.394 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2000000000000001e-06 2024-09-21T21:41:07.394 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.394 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.394 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:07.394 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514433+0000", 2024-09-21T21:41:07.394 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.3170000000000004e-06 2024-09-21T21:41:07.394 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.394 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.394 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:07.394 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514438+0000", 2024-09-21T21:41:07.394 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3730000000000001e-06 2024-09-21T21:41:07.395 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.395 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.395 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:07.395 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.528984+0000", 2024-09-21T21:41:07.395 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.014545773 2024-09-21T21:41:07.395 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.395 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.395 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:07.395 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.529056+0000", 2024-09-21T21:41:07.395 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.2067e-05 2024-09-21T21:41:07.395 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.395 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.395 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:07.395 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.529143+0000", 2024-09-21T21:41:07.396 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.6881999999999999e-05 2024-09-21T21:41:07.396 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:07.396 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:07.396 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:07.396 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.396 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.396 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4583 3.0 3:074cff57:::rbd_data.112145fae779.000000000000000c:a [read 2039808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:07.396 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.514446+0000", 2024-09-21T21:41:07.396 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 8.8155556530000005, 2024-09-21T21:41:07.396 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.016072466000000001, 2024-09-21T21:41:07.396 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:07.396 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:07.396 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:07.396 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:07.397 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:07.397 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:07.397 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4583 2024-09-21T21:41:07.397 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.397 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:07.397 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.397 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:07.397 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514446+0000", 2024-09-21T21:41:07.397 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:07.397 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.397 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.397 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:07.397 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514446+0000", 2024-09-21T21:41:07.397 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:07.398 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.398 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.398 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:07.398 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514446+0000", 2024-09-21T21:41:07.398 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.9700000000000002e-07 2024-09-21T21:41:07.398 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.398 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.398 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:07.398 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514448+0000", 2024-09-21T21:41:07.398 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1849999999999998e-06 2024-09-21T21:41:07.398 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.398 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.398 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:07.398 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514454+0000", 2024-09-21T21:41:07.399 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.015e-06 2024-09-21T21:41:07.399 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.399 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.399 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:07.399 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514459+0000", 2024-09-21T21:41:07.399 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7639999999999996e-06 2024-09-21T21:41:07.399 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.399 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.399 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:07.399 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.529173+0000", 2024-09-21T21:41:07.399 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.014713688000000001 2024-09-21T21:41:07.399 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.399 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.399 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:07.400 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.529248+0000", 2024-09-21T21:41:07.400 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5073000000000004e-05 2024-09-21T21:41:07.400 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.400 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.400 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:07.400 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.530518+0000", 2024-09-21T21:41:07.400 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0012703440000000001 2024-09-21T21:41:07.400 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:07.400 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:07.400 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:07.400 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.400 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.400 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4584 3.0 3:074cff57:::rbd_data.112145fae779.000000000000000c:a [read 2129920~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:07.400 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.514465+0000", 2024-09-21T21:41:07.401 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 8.8155362139999998, 2024-09-21T21:41:07.401 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.016252026999999999, 2024-09-21T21:41:07.401 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:07.401 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:07.401 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:07.401 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:07.401 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:07.401 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:07.401 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4584 2024-09-21T21:41:07.401 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.401 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:07.401 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.401 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:07.401 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514465+0000", 2024-09-21T21:41:07.402 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:07.402 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.402 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.402 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:07.402 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514465+0000", 2024-09-21T21:41:07.402 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:07.402 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.402 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.402 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:07.402 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514466+0000", 2024-09-21T21:41:07.402 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.0400000000000002e-07 2024-09-21T21:41:07.402 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.402 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.402 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:07.403 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514468+0000", 2024-09-21T21:41:07.403 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3159999999999999e-06 2024-09-21T21:41:07.403 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.403 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.403 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:07.403 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514474+0000", 2024-09-21T21:41:07.403 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.224e-06 2024-09-21T21:41:07.403 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.403 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.403 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:07.403 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514479+0000", 2024-09-21T21:41:07.403 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8099999999999997e-06 2024-09-21T21:41:07.403 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.403 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.404 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:07.404 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.530552+0000", 2024-09-21T21:41:07.404 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.016072622000000002 2024-09-21T21:41:07.404 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.404 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.404 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:07.404 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.530630+0000", 2024-09-21T21:41:07.404 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.8448000000000005e-05 2024-09-21T21:41:07.404 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.404 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.404 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:07.404 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.530717+0000", 2024-09-21T21:41:07.404 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.7202999999999998e-05 2024-09-21T21:41:07.405 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:07.405 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:07.405 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:07.405 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.405 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.405 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4585 3.0 3:074cff57:::rbd_data.112145fae779.000000000000000c:a [read 2187264~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:07.405 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.514485+0000", 2024-09-21T21:41:07.405 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 8.8155165469999996, 2024-09-21T21:41:07.405 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.016420605000000001, 2024-09-21T21:41:07.405 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:07.405 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:07.405 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:07.405 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:07.405 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:07.406 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:07.406 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4585 2024-09-21T21:41:07.406 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.406 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:07.406 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.406 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:07.406 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514485+0000", 2024-09-21T21:41:07.406 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:07.406 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.406 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.406 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:07.406 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514485+0000", 2024-09-21T21:41:07.406 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:07.407 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.407 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.407 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:07.407 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514485+0000", 2024-09-21T21:41:07.407 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3599999999999999e-07 2024-09-21T21:41:07.407 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.407 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.407 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:07.407 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514487+0000", 2024-09-21T21:41:07.407 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0870000000000001e-06 2024-09-21T21:41:07.407 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.407 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.407 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:07.407 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514495+0000", 2024-09-21T21:41:07.408 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7589999999999997e-06 2024-09-21T21:41:07.408 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.408 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.408 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:07.408 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514501+0000", 2024-09-21T21:41:07.408 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4e-06 2024-09-21T21:41:07.409 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.409 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.409 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:07.410 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.530747+0000", 2024-09-21T21:41:07.410 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.016246564000000002 2024-09-21T21:41:07.410 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.410 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.410 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:07.410 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.530824+0000", 2024-09-21T21:41:07.410 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7109999999999999e-05 2024-09-21T21:41:07.410 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.410 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.410 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:07.410 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.530906+0000", 2024-09-21T21:41:07.410 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.1348999999999997e-05 2024-09-21T21:41:07.410 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:07.410 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:07.411 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:07.411 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.411 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.411 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4711 3.0 3:074cff57:::rbd_data.112145fae779.000000000000000c:a [read 2043904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:07.411 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.519604+0000", 2024-09-21T21:41:07.411 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 8.8103970579999995, 2024-09-21T21:41:07.411 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013270252999999999, 2024-09-21T21:41:07.411 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:07.411 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:07.411 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:07.411 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:07.411 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:07.411 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:07.412 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4711 2024-09-21T21:41:07.412 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.412 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:07.412 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.412 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:07.412 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519604+0000", 2024-09-21T21:41:07.412 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:07.412 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.412 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.412 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:07.412 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519604+0000", 2024-09-21T21:41:07.412 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:07.412 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.412 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.413 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:07.413 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519605+0000", 2024-09-21T21:41:07.413 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8500000000000002e-07 2024-09-21T21:41:07.413 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.413 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.413 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:07.413 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519607+0000", 2024-09-21T21:41:07.413 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1789999999999998e-06 2024-09-21T21:41:07.413 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.413 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.413 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:07.413 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519613+0000", 2024-09-21T21:41:07.413 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2609999999999999e-06 2024-09-21T21:41:07.413 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.414 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.414 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:07.414 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519618+0000", 2024-09-21T21:41:07.414 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9579999999999998e-06 2024-09-21T21:41:07.414 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.414 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.414 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:07.414 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.532678+0000", 2024-09-21T21:41:07.414 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013059822 2024-09-21T21:41:07.414 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.414 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.414 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:07.414 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.532756+0000", 2024-09-21T21:41:07.414 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7467999999999998e-05 2024-09-21T21:41:07.414 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.415 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.415 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:07.415 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.532875+0000", 2024-09-21T21:41:07.415 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011917999999999999 2024-09-21T21:41:07.415 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:07.415 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:07.415 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:07.415 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.415 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.415 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4712 3.0 3:074cff57:::rbd_data.112145fae779.000000000000000c:a [read 2134016~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:07.415 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.519632+0000", 2024-09-21T21:41:07.415 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 8.8103695749999993, 2024-09-21T21:41:07.415 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013480002, 2024-09-21T21:41:07.415 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:07.415 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:07.415 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:07.416 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:07.416 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:07.416 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:07.416 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4712 2024-09-21T21:41:07.416 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.416 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:07.416 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.416 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:07.416 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519632+0000", 2024-09-21T21:41:07.416 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:07.416 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.416 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.416 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:07.416 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519632+0000", 2024-09-21T21:41:07.416 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:07.417 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.417 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.417 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:07.417 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519632+0000", 2024-09-21T21:41:07.417 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2899999999999999e-07 2024-09-21T21:41:07.417 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.417 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.417 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:07.417 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519635+0000", 2024-09-21T21:41:07.417 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.6910000000000002e-06 2024-09-21T21:41:07.417 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.417 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.417 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:07.417 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519761+0000", 2024-09-21T21:41:07.417 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000125988 2024-09-21T21:41:07.418 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.418 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.418 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:07.418 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519767+0000", 2024-09-21T21:41:07.418 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9190000000000004e-06 2024-09-21T21:41:07.418 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.418 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.418 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:07.418 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.532905+0000", 2024-09-21T21:41:07.418 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013137970000000001 2024-09-21T21:41:07.418 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.418 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.418 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:07.418 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.532984+0000", 2024-09-21T21:41:07.418 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.8616999999999996e-05 2024-09-21T21:41:07.419 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.419 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.419 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:07.419 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.533112+0000", 2024-09-21T21:41:07.419 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000128388 2024-09-21T21:41:07.419 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:07.419 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:07.419 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:07.419 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.419 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.419 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4713 3.0 3:074cff57:::rbd_data.112145fae779.000000000000000c:a [read 2191360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:07.419 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.519777+0000", 2024-09-21T21:41:07.419 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 8.8102248749999994, 2024-09-21T21:41:07.419 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.01355251, 2024-09-21T21:41:07.419 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:07.419 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:07.420 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:07.420 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:07.420 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:07.420 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:07.420 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4713 2024-09-21T21:41:07.420 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.420 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:07.420 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.420 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:07.420 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519777+0000", 2024-09-21T21:41:07.420 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:07.420 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.420 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.420 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:07.420 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519777+0000", 2024-09-21T21:41:07.421 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:07.421 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.421 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.421 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:07.421 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519777+0000", 2024-09-21T21:41:07.421 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9900000000000001e-07 2024-09-21T21:41:07.421 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.421 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.421 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:07.421 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519780+0000", 2024-09-21T21:41:07.421 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.886e-06 2024-09-21T21:41:07.421 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.421 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.421 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:07.422 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519786+0000", 2024-09-21T21:41:07.422 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0859999999999998e-06 2024-09-21T21:41:07.422 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.422 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.422 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:07.422 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519790+0000", 2024-09-21T21:41:07.422 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2400000000000001e-06 2024-09-21T21:41:07.422 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.422 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.422 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:07.422 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.533143+0000", 2024-09-21T21:41:07.422 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013352711 2024-09-21T21:41:07.422 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.422 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.423 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:07.423 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.533224+0000", 2024-09-21T21:41:07.423 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0675999999999998e-05 2024-09-21T21:41:07.423 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.423 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.423 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:07.423 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.533329+0000", 2024-09-21T21:41:07.423 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000105412 2024-09-21T21:41:07.423 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:07.423 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:07.423 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:07.423 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.423 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.424 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4824.0:1564 3.2 3:4647520c:::rbd_data.1121b97f2925.0000000000000006:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:07.424 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:41:07.424 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 5.6939370010000001, 2024-09-21T21:41:07.424 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056097347999999998, 2024-09-21T21:41:07.424 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:07.424 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:07.424 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:07.424 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:07.424 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4824", 2024-09-21T21:41:07.424 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:07.424 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1564 2024-09-21T21:41:07.424 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.424 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:07.425 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.425 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:07.425 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:41:07.425 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:07.425 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.425 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.425 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:07.425 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:41:07.425 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:07.425 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.425 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.425 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:07.425 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:41:07.425 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6499999999999999e-07 2024-09-21T21:41:07.426 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.426 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.426 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:07.426 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636067+0000", 2024-09-21T21:41:07.426 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3879999999999998e-06 2024-09-21T21:41:07.426 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.426 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.426 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:07.426 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636075+0000", 2024-09-21T21:41:07.426 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5480000000000004e-06 2024-09-21T21:41:07.426 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.426 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.426 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:07.426 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636080+0000", 2024-09-21T21:41:07.427 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.339e-06 2024-09-21T21:41:07.427 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.427 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.427 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:07.427 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636133+0000", 2024-09-21T21:41:07.427 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2930000000000003e-05 2024-09-21T21:41:07.427 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.427 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.427 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:07.427 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636265+0000", 2024-09-21T21:41:07.427 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000132058 2024-09-21T21:41:07.427 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:07.427 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:07.427 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:07.428 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.692162+0000", 2024-09-21T21:41:07.428 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055896620000000001 2024-09-21T21:41:07.428 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:07.428 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:07.428 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:07.428 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:07.428 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:07.428 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:07.429 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T21:41:07.430 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:07.428+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:07.463 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T21:41:07.530 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:07.527+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:07.581 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:07.581 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:41:07.582 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 0 2024-09-21T21:41:07.582 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:07.591 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T21:41:07.593 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0: {} 2024-09-21T21:41:07.594 INFO:teuthology.orchestra.run.smithi081.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T21:41:07.613 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1: {} 2024-09-21T21:41:07.613 INFO:teuthology.orchestra.run.smithi081.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T21:41:07.629 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2: {} 2024-09-21T21:41:07.629 INFO:teuthology.orchestra.run.smithi081.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T21:41:07.630 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:07.629+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:07.647 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3: {} 2024-09-21T21:41:07.647 INFO:teuthology.orchestra.run.smithi081.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T21:41:07.665 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4: {} 2024-09-21T21:41:07.665 INFO:teuthology.orchestra.run.smithi081.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T21:41:07.669 INFO:teuthology.orchestra.run.smithi164.stdout:leaving image image_client.0-clone11 intact 2024-09-21T21:41:07.682 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5: {} 2024-09-21T21:41:07.682 INFO:teuthology.orchestra.run.smithi081.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T21:41:07.701 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:07.701 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:07.710 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T21:41:07.752 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:07.749+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:07.831 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:07.830+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:07.856 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:07.856 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:07.865 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T21:41:07.932 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:07.930+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:07.975 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:07.976 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:07.984 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T21:41:08.068 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:08.067+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:08.101 INFO:teuthology.orchestra.run.smithi164.stdout:checking clone #10, image image_client.0-clone10 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent11 2024-09-21T21:41:08.130 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:08.131 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:08.134 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:08.132+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:08.140 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T21:41:08.214 INFO:teuthology.orchestra.run.smithi164.stdout:1627 read 0xfa5a00 thru 0xfa8aba (0x30bb bytes) 2024-09-21T21:41:08.215 INFO:teuthology.orchestra.run.smithi164.stdout:1631 read 0x2880c00 thru 0x2885df2 (0x51f3 bytes) 2024-09-21T21:41:08.234 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:08.232+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:08.239 INFO:teuthology.orchestra.run.smithi164.stdout:1632 write 0xb496000 thru 0xb4a0e90 (0xae91 bytes) 2024-09-21T21:41:08.257 INFO:teuthology.orchestra.run.smithi164.stdout:1633 trunc from 0xb4a1000 to 0x1738400 2024-09-21T21:41:08.286 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:08.286 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:08.296 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T21:41:08.299 INFO:teuthology.orchestra.run.smithi164.stdout:1636 read 0xe82600 thru 0xe87f72 (0x5973 bytes) 2024-09-21T21:41:08.307 INFO:teuthology.orchestra.run.smithi164.stdout:1638 punch from 0xa66800 to 0xa69a00, (0x3200 bytes) 2024-09-21T21:41:08.328 INFO:teuthology.orchestra.run.smithi164.stdout:1639 write 0xbb21400 thru 0xbb25727 (0x4328 bytes) 2024-09-21T21:41:08.334 INFO:teuthology.orchestra.run.smithi164.stdout:1640 read 0x2143c00 thru 0x21481bc (0x45bd bytes) 2024-09-21T21:41:08.334 INFO:teuthology.orchestra.run.smithi164.stdout:1642 read 0x48f800 thru 0x490e77 (0x1678 bytes) 2024-09-21T21:41:08.335 INFO:teuthology.orchestra.run.smithi164.stdout:1643 read 0x2a69200 thru 0x2a71392 (0x8193 bytes) 2024-09-21T21:41:08.335 INFO:teuthology.orchestra.run.smithi164.stdout:1647 write 0x27d7000 thru 0x27dc799 (0x579a bytes) 2024-09-21T21:41:08.335 INFO:teuthology.orchestra.run.smithi164.stdout:1648 write 0x5376e00 thru 0x5383a28 (0xcc29 bytes) 2024-09-21T21:41:08.335 INFO:teuthology.orchestra.run.smithi164.stdout:1650 read 0x4d84a00 thru 0x4d897ac (0x4dad bytes) 2024-09-21T21:41:08.335 INFO:teuthology.orchestra.run.smithi164.stdout:1651 write 0x87d8200 thru 0x87df7da (0x75db bytes) 2024-09-21T21:41:08.339 INFO:teuthology.orchestra.run.smithi164.stdout:1652 trunc from 0xbb25800 to 0x7f8f800 2024-09-21T21:41:08.340 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:08.339+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:08.379 INFO:teuthology.orchestra.run.smithi164.stdout:1653 write 0x8974a00 thru 0x897f55c (0xab5d bytes) 2024-09-21T21:41:08.401 INFO:teuthology.orchestra.run.smithi164.stdout:1654 write 0xa354600 thru 0xa356f58 (0x2959 bytes) 2024-09-21T21:41:08.404 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:08.404 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:08.411 INFO:teuthology.orchestra.run.smithi164.stdout:1655 trunc from 0xa357000 to 0x57b5c00 2024-09-21T21:41:08.412 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T21:41:08.444 INFO:teuthology.orchestra.run.smithi164.stdout:1656 write 0x768e600 thru 0x7697c28 (0x9629 bytes) 2024-09-21T21:41:08.449 INFO:teuthology.orchestra.run.smithi164.stdout:1657 write 0x5d86200 thru 0x5d9031d (0xa11e bytes) 2024-09-21T21:41:08.449 INFO:teuthology.orchestra.run.smithi164.stdout:1658 write 0x30acc00 thru 0x30aec37 (0x2038 bytes) 2024-09-21T21:41:08.449 INFO:teuthology.orchestra.run.smithi164.stdout:1659 read 0x6b56800 thru 0x6b5b45e (0x4c5f bytes) 2024-09-21T21:41:08.449 INFO:teuthology.orchestra.run.smithi164.stdout:1661 write 0x3a6a00 thru 0x3ab390 (0x4991 bytes) 2024-09-21T21:41:08.450 INFO:teuthology.orchestra.run.smithi164.stdout:1664 read 0x2204c00 thru 0x220be17 (0x7218 bytes) 2024-09-21T21:41:08.450 INFO:teuthology.orchestra.run.smithi164.stdout:1665 read 0x48a2400 thru 0x48a54ec (0x30ed bytes) 2024-09-21T21:41:08.450 INFO:teuthology.orchestra.run.smithi164.stdout:1668 read 0x1adc400 thru 0x1ae772f (0xb330 bytes) 2024-09-21T21:41:08.450 INFO:teuthology.orchestra.run.smithi164.stdout:1671 punch from 0x1162c00 to 0x116e200, (0xb600 bytes) 2024-09-21T21:41:08.451 INFO:teuthology.orchestra.run.smithi164.stdout:1673 read 0x490a800 thru 0x49147dd (0x9fde bytes) 2024-09-21T21:41:08.451 INFO:teuthology.orchestra.run.smithi164.stdout:1674 read 0x3bbc000 thru 0x3bbfdad (0x3dae bytes) 2024-09-21T21:41:08.451 INFO:teuthology.orchestra.run.smithi164.stdout:1677 read 0x6663800 thru 0x666746b (0x3c6c bytes) 2024-09-21T21:41:08.451 INFO:teuthology.orchestra.run.smithi164.stdout:1678 write 0x2668000 thru 0x2670ccc (0x8ccd bytes) 2024-09-21T21:41:08.453 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:08.452+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:08.454 INFO:teuthology.orchestra.run.smithi164.stdout:1680 trunc from 0x7697e00 to 0x5518000 2024-09-21T21:41:08.479 INFO:teuthology.orchestra.run.smithi164.stdout:1684 read 0x9e6000 thru 0x9ecee7 (0x6ee8 bytes) 2024-09-21T21:41:08.487 INFO:teuthology.orchestra.run.smithi164.stdout:1686 trunc from 0x5518000 to 0x413d200 2024-09-21T21:41:08.509 INFO:teuthology.orchestra.run.smithi164.stdout:1688 trunc from 0x413d200 to 0xc0a8000 2024-09-21T21:41:08.524 INFO:teuthology.orchestra.run.smithi164.stdout:1689 write 0x4e22e00 thru 0x4e2c254 (0x9455 bytes) 2024-09-21T21:41:08.526 INFO:teuthology.orchestra.run.smithi164.stdout:1690 trunc from 0xc0a8000 to 0xd8bb600 2024-09-21T21:41:08.537 INFO:teuthology.orchestra.run.smithi164.stdout:1692 read 0x5493800 thru 0x549a117 (0x6918 bytes) 2024-09-21T21:41:08.537 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:08.536+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:08.538 INFO:teuthology.orchestra.run.smithi164.stdout:1693 punch from 0x58b1600 to 0x58b6000, (0x4a00 bytes) 2024-09-21T21:41:08.540 INFO:teuthology.orchestra.run.smithi164.stdout:1695 write 0xcf04e00 thru 0xcf10429 (0xb62a bytes) 2024-09-21T21:41:08.540 INFO:teuthology.orchestra.run.smithi164.stdout:1696 punch from 0x2153c00 to 0x215e000, (0xa400 bytes) 2024-09-21T21:41:08.548 INFO:teuthology.orchestra.run.smithi164.stdout:1698 trunc from 0xd8bb600 to 0x64da600 2024-09-21T21:41:08.560 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:08.560 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:08.570 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T21:41:08.571 INFO:teuthology.orchestra.run.smithi164.stdout:1701 trunc from 0x64da600 to 0x5d4a00 2024-09-21T21:41:08.573 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T21:41:08.620 INFO:teuthology.orchestra.run.smithi164.stdout:1705 read 0x5a6400 thru 0x5ad78d (0x738e bytes) 2024-09-21T21:41:08.620 INFO:teuthology.orchestra.run.smithi164.stdout:1706 read 0x224800 thru 0x2328fd (0xe0fe bytes) 2024-09-21T21:41:08.638 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:08.635+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:08.644 INFO:teuthology.orchestra.run.smithi164.stdout:1708 write 0x6dfd800 thru 0x6e031eb (0x59ec bytes) 2024-09-21T21:41:08.644 INFO:teuthology.orchestra.run.smithi164.stdout:1711 read 0x4af5000 thru 0x4afc028 (0x7029 bytes) 2024-09-21T21:41:08.649 INFO:teuthology.orchestra.run.smithi164.stdout:1715 trunc from 0x6e03200 to 0xb490a00 2024-09-21T21:41:08.669 INFO:teuthology.orchestra.run.smithi164.stdout:1718 write 0x9d2600 thru 0x9d9529 (0x6f2a bytes) 2024-09-21T21:41:08.669 INFO:teuthology.orchestra.run.smithi164.stdout:1719 read 0x639c000 thru 0x63a1f34 (0x5f35 bytes) 2024-09-21T21:41:08.670 INFO:teuthology.orchestra.run.smithi164.stdout:1721 punch from 0x1c08000 to 0x1c12600, (0xa600 bytes) 2024-09-21T21:41:08.670 INFO:teuthology.orchestra.run.smithi164.stdout:1722 write 0x4be6600 thru 0x4becc87 (0x6688 bytes) 2024-09-21T21:41:08.692 INFO:teuthology.orchestra.run.smithi164.stdout:1723 write 0xcacfc00 thru 0xcadca27 (0xce28 bytes) 2024-09-21T21:41:08.692 INFO:teuthology.orchestra.run.smithi164.stdout:1724 write 0xc23b400 thru 0xc2422da (0x6edb bytes) 2024-09-21T21:41:08.692 INFO:teuthology.orchestra.run.smithi164.stdout:1729 read 0x9bd2200 thru 0x9bdb1c6 (0x8fc7 bytes) 2024-09-21T21:41:08.692 INFO:teuthology.orchestra.run.smithi164.stdout:1730 punch from 0x22b7c00 to 0x22c6800, (0xec00 bytes) 2024-09-21T21:41:08.692 INFO:teuthology.orchestra.run.smithi164.stdout:1731 punch from 0xb652a00 to 0xb661400, (0xea00 bytes) 2024-09-21T21:41:08.693 INFO:teuthology.orchestra.run.smithi164.stdout:1732 punch from 0x4684600 to 0x4687a00, (0x3400 bytes) 2024-09-21T21:41:08.698 INFO:teuthology.orchestra.run.smithi164.stdout:1733 trunc from 0xcadcc00 to 0x73f4a00 2024-09-21T21:41:08.720 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:08.720 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:08.724 INFO:teuthology.orchestra.run.smithi164.stdout:1734 read 0x5221800 thru 0x5226894 (0x5095 bytes) 2024-09-21T21:41:08.729 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T21:41:08.740 INFO:teuthology.orchestra.run.smithi164.stdout:1735 write 0x987b200 thru 0x9881407 (0x6208 bytes) 2024-09-21T21:41:08.746 INFO:teuthology.orchestra.run.smithi164.stdout:1736 punch from 0x521cc00 to 0x522be00, (0xf200 bytes) 2024-09-21T21:41:08.747 INFO:teuthology.orchestra.run.smithi164.stdout:1737 read 0x10ac00 thru 0x10df05 (0x3306 bytes) 2024-09-21T21:41:08.748 INFO:teuthology.orchestra.run.smithi164.stdout:1741 punch from 0x38d5a00 to 0x38dfe00, (0xa400 bytes) 2024-09-21T21:41:08.763 INFO:teuthology.orchestra.run.smithi164.stdout:1745 write 0xbe19600 thru 0xbe1deda (0x48db bytes) 2024-09-21T21:41:08.770 INFO:teuthology.orchestra.run.smithi164.stdout:1752 write 0x411a600 thru 0x4125f98 (0xb999 bytes) 2024-09-21T21:41:08.773 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:08.771+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:08.776 INFO:teuthology.orchestra.run.smithi164.stdout:1754 trunc from 0xbe1e000 to 0x6160e00 2024-09-21T21:41:08.798 INFO:teuthology.orchestra.run.smithi164.stdout:1755 punch from 0x1e36800 to 0x1e3b400, (0x4c00 bytes) 2024-09-21T21:41:08.800 INFO:teuthology.orchestra.run.smithi164.stdout:1758 punch from 0x2ec8000 to 0x2ed3c00, (0xbc00 bytes) 2024-09-21T21:41:08.806 INFO:teuthology.orchestra.run.smithi164.stdout:1760 trunc from 0x6160e00 to 0xb400600 2024-09-21T21:41:08.817 INFO:teuthology.orchestra.run.smithi164.stdout:1761 read 0x3f62600 thru 0x3f71f74 (0xf975 bytes) 2024-09-21T21:41:08.822 INFO:teuthology.orchestra.run.smithi164.stdout:1762 write 0x24d9000 thru 0x24e73cb (0xe3cc bytes) 2024-09-21T21:41:08.822 INFO:teuthology.orchestra.run.smithi164.stdout:1765 read 0x2c6e800 thru 0x2c7bd11 (0xd512 bytes) 2024-09-21T21:41:08.822 INFO:teuthology.orchestra.run.smithi164.stdout:1766 write 0x421c600 thru 0x421fbe5 (0x35e6 bytes) 2024-09-21T21:41:08.822 INFO:teuthology.orchestra.run.smithi164.stdout:1768 read 0x975d200 thru 0x9760288 (0x3089 bytes) 2024-09-21T21:41:08.838 INFO:teuthology.orchestra.run.smithi164.stdout:1771 write 0xe806000 thru 0xe80b135 (0x5136 bytes) 2024-09-21T21:41:08.839 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:08.839 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:08.839 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:08.836+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:08.844 INFO:teuthology.orchestra.run.smithi164.stdout:1773 write 0xad72000 thru 0xad73c65 (0x1c66 bytes) 2024-09-21T21:41:08.845 INFO:teuthology.orchestra.run.smithi164.stdout:1776 write 0x7e50800 thru 0x7e542e4 (0x3ae5 bytes) 2024-09-21T21:41:08.845 INFO:teuthology.orchestra.run.smithi164.stdout:1777 write 0x7256600 thru 0x72594fe (0x2eff bytes) 2024-09-21T21:41:08.845 INFO:teuthology.orchestra.run.smithi164.stdout:1778 write 0xc9f8400 thru 0xca03610 (0xb211 bytes) 2024-09-21T21:41:08.845 INFO:teuthology.orchestra.run.smithi164.stdout:1780 write 0x1c12e00 thru 0x1c14e6a (0x206b bytes) 2024-09-21T21:41:08.847 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T21:41:08.854 INFO:teuthology.orchestra.run.smithi164.stdout:1781 trunc from 0xe80b200 to 0x782a000 2024-09-21T21:41:08.904 INFO:teuthology.orchestra.run.smithi164.stdout:1782 write 0xb092600 thru 0xb095aaf (0x34b0 bytes) 2024-09-21T21:41:08.911 INFO:teuthology.orchestra.run.smithi164.stdout:1783 write 0x3dffe00 thru 0x3e0b70f (0xb910 bytes) 2024-09-21T21:41:08.912 INFO:teuthology.orchestra.run.smithi164.stdout:1784 write 0x72d4e00 thru 0x72d788c (0x2a8d bytes) 2024-09-21T21:41:08.916 INFO:teuthology.orchestra.run.smithi164.stdout:1786 trunc from 0xb095c00 to 0xe835400 2024-09-21T21:41:08.931 INFO:teuthology.orchestra.run.smithi164.stdout:1788 trunc from 0xe835400 to 0xce68400 2024-09-21T21:41:08.940 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:08.938+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:08.947 INFO:teuthology.orchestra.run.smithi164.stdout:1789 read 0x6539e00 thru 0x653f77c (0x597d bytes) 2024-09-21T21:41:08.948 INFO:teuthology.orchestra.run.smithi164.stdout:1790 punch from 0x4326200 to 0x4328000, (0x1e00 bytes) 2024-09-21T21:41:08.949 INFO:teuthology.orchestra.run.smithi164.stdout:1791 write 0x93dd000 thru 0x93df586 (0x2587 bytes) 2024-09-21T21:41:08.963 INFO:teuthology.orchestra.run.smithi164.stdout:1792 trunc from 0xce68400 to 0x1747400 2024-09-21T21:41:08.996 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:08.996 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:09.005 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T21:41:09.024 INFO:teuthology.orchestra.run.smithi164.stdout:1798 write 0xe166a00 thru 0xe17229e (0xb89f bytes) 2024-09-21T21:41:09.029 INFO:teuthology.orchestra.run.smithi164.stdout:1799 read 0x9225000 thru 0x922ec95 (0x9c96 bytes) 2024-09-21T21:41:09.029 INFO:teuthology.orchestra.run.smithi164.stdout:1800 trunc from 0xe172400 to 0xe823200 2024-09-21T21:41:09.040 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:09.039+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:09.043 INFO:teuthology.orchestra.run.smithi164.stdout:1801 read 0x45a6a00 thru 0x45b5795 (0xed96 bytes) 2024-09-21T21:41:09.045 INFO:teuthology.orchestra.run.smithi164.stdout:1802 read 0x1c3c000 thru 0x1c490ad (0xd0ae bytes) 2024-09-21T21:41:09.045 INFO:teuthology.orchestra.run.smithi164.stdout:1806 read 0x341a000 thru 0x34219fa (0x79fb bytes) 2024-09-21T21:41:09.061 INFO:teuthology.orchestra.run.smithi164.stdout:1808 trunc from 0xe823200 to 0xd3ee00 2024-09-21T21:41:09.100 INFO:teuthology.orchestra.run.smithi164.stdout:1809 write 0xb0f8e00 thru 0xb0fa787 (0x1988 bytes) 2024-09-21T21:41:09.101 INFO:teuthology.orchestra.run.smithi164.stdout:1810 read 0x8d9b600 thru 0x8da13df (0x5de0 bytes) 2024-09-21T21:41:09.101 INFO:teuthology.orchestra.run.smithi164.stdout:1811 write 0xa667800 thru 0xa66d331 (0x5b32 bytes) 2024-09-21T21:41:09.101 INFO:teuthology.orchestra.run.smithi164.stdout:1812 read 0x124b800 thru 0x125ada4 (0xf5a5 bytes) 2024-09-21T21:41:09.102 INFO:teuthology.orchestra.run.smithi164.stdout:1813 read 0x37e7c00 thru 0x37f77cb (0xfbcc bytes) 2024-09-21T21:41:09.102 INFO:teuthology.orchestra.run.smithi164.stdout:1815 write 0x8ac4600 thru 0x8ac678b (0x218c bytes) 2024-09-21T21:41:09.102 INFO:teuthology.orchestra.run.smithi164.stdout:1816 write 0x8312200 thru 0x831ba6f (0x9870 bytes) 2024-09-21T21:41:09.102 INFO:teuthology.orchestra.run.smithi164.stdout:1817 punch from 0x140600 to 0x140e00, (0x800 bytes) 2024-09-21T21:41:09.102 INFO:teuthology.orchestra.run.smithi164.stdout:1819 read 0x2e75000 thru 0x2e7aaf0 (0x5af1 bytes) 2024-09-21T21:41:09.103 INFO:teuthology.orchestra.run.smithi164.stdout:1820 write 0x481d000 thru 0x482416c (0x716d bytes) 2024-09-21T21:41:09.113 INFO:teuthology.orchestra.run.smithi164.stdout:1822 trunc from 0xb0fa800 to 0x1b59200 2024-09-21T21:41:09.133 INFO:teuthology.orchestra.run.smithi164.stdout:1823 read 0x11d3200 thru 0x11db35d (0x815e bytes) 2024-09-21T21:41:09.141 INFO:teuthology.orchestra.run.smithi164.stdout:1825 punch from 0x6f4800 to 0x6f8200, (0x3a00 bytes) 2024-09-21T21:41:09.141 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:09.139+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:09.151 INFO:teuthology.orchestra.run.smithi164.stdout:1827 write 0x33c8800 thru 0x33cc2db (0x3adc bytes) 2024-09-21T21:41:09.155 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:09.155 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:09.161 INFO:teuthology.orchestra.run.smithi164.stdout:1828 trunc from 0x33cc400 to 0xbe5e00 2024-09-21T21:41:09.163 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T21:41:09.179 INFO:teuthology.orchestra.run.smithi164.stdout:1830 read 0xa60000 thru 0xa63615 (0x3616 bytes) 2024-09-21T21:41:09.183 INFO:teuthology.orchestra.run.smithi164.stdout:1832 trunc from 0xbe5e00 to 0x342a600 2024-09-21T21:41:09.209 INFO:teuthology.orchestra.run.smithi164.stdout:1833 write 0x951aa00 thru 0x95265b1 (0xbbb2 bytes) 2024-09-21T21:41:09.210 INFO:teuthology.orchestra.run.smithi164.stdout:1835 read 0x87aa400 thru 0x87b6a63 (0xc664 bytes) 2024-09-21T21:41:09.210 INFO:teuthology.orchestra.run.smithi164.stdout:1837 write 0x45df000 thru 0x45ec5b2 (0xd5b3 bytes) 2024-09-21T21:41:09.238 INFO:teuthology.orchestra.run.smithi164.stdout:1838 write 0xe84a400 thru 0xe8593ba (0xefbb bytes) 2024-09-21T21:41:09.238 INFO:teuthology.orchestra.run.smithi164.stdout:1839 write 0x73b7600 thru 0x73c62c9 (0xecca bytes) 2024-09-21T21:41:09.238 INFO:teuthology.orchestra.run.smithi164.stdout:1841 write 0x276f000 thru 0x277b900 (0xc901 bytes) 2024-09-21T21:41:09.238 INFO:teuthology.orchestra.run.smithi164.stdout:1842 read 0xa3af400 thru 0xa3baeca (0xbacb bytes) 2024-09-21T21:41:09.238 INFO:teuthology.orchestra.run.smithi164.stdout:1843 punch from 0xafd3800 to 0xafd9800, (0x6000 bytes) 2024-09-21T21:41:09.239 INFO:teuthology.orchestra.run.smithi164.stdout:1844 write 0x3e47000 thru 0x3e4f1fd (0x81fe bytes) 2024-09-21T21:41:09.239 INFO:teuthology.orchestra.run.smithi164.stdout:1849 read 0xd549800 thru 0xd54f9d7 (0x61d8 bytes) 2024-09-21T21:41:09.239 INFO:teuthology.orchestra.run.smithi164.stdout:1851 read 0x392ec00 thru 0x3930c1c (0x201d bytes) 2024-09-21T21:41:09.242 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:09.240+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:09.248 INFO:teuthology.orchestra.run.smithi164.stdout:1854 trunc from 0xe859400 to 0x6798400 2024-09-21T21:41:09.275 INFO:teuthology.orchestra.run.smithi164.stdout:1855 read 0x57f1400 thru 0x57f7595 (0x6196 bytes) 2024-09-21T21:41:09.275 INFO:teuthology.orchestra.run.smithi164.stdout:1858 read 0x3337e00 thru 0x3344f20 (0xd121 bytes) 2024-09-21T21:41:09.276 INFO:teuthology.orchestra.run.smithi164.stdout:1861 flatten 2024-09-21T21:41:09.283 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:09.283 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:09.286 INFO:teuthology.orchestra.run.smithi164.stdout:1863 write 0x3f15c00 thru 0x3f18140 (0x2541 bytes) 2024-09-21T21:41:09.286 INFO:teuthology.orchestra.run.smithi164.stdout:1866 read 0x3ae8000 thru 0x3af1420 (0x9421 bytes) 2024-09-21T21:41:09.288 INFO:teuthology.orchestra.run.smithi164.stdout:1867 trunc from 0x6798400 to 0x7e12c00 2024-09-21T21:41:09.292 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T21:41:09.301 INFO:teuthology.orchestra.run.smithi164.stdout:1868 read 0x2da5600 thru 0x2daa85d (0x525e bytes) 2024-09-21T21:41:09.302 INFO:teuthology.orchestra.run.smithi164.stdout:1870 read 0x6645800 thru 0x664d25f (0x7a60 bytes) 2024-09-21T21:41:09.302 INFO:teuthology.orchestra.run.smithi164.stdout:1873 trunc from 0x7e12c00 to 0x7c7be00 2024-09-21T21:41:09.311 INFO:teuthology.orchestra.run.smithi164.stdout:1874 write 0x70a600 thru 0x7124a4 (0x7ea5 bytes) 2024-09-21T21:41:09.326 INFO:teuthology.orchestra.run.smithi164.stdout:1875 write 0xb264800 thru 0xb270293 (0xba94 bytes) 2024-09-21T21:41:09.326 INFO:teuthology.orchestra.run.smithi164.stdout:1876 read 0x31e1800 thru 0x31e75fa (0x5dfb bytes) 2024-09-21T21:41:09.327 INFO:teuthology.orchestra.run.smithi164.stdout:1877 read 0xb020c00 thru 0xb02b6f1 (0xaaf2 bytes) 2024-09-21T21:41:09.339 INFO:teuthology.orchestra.run.smithi164.stdout:1878 write 0xb919400 thru 0xb91faec (0x66ed bytes) 2024-09-21T21:41:09.339 INFO:teuthology.orchestra.run.smithi164.stdout:1879 punch from 0xb00d200 to 0xb01b000, (0xde00 bytes) 2024-09-21T21:41:09.341 INFO:teuthology.orchestra.run.smithi164.stdout:1881 write 0x43dcc00 thru 0x43e4c5e (0x805f bytes) 2024-09-21T21:41:09.341 INFO:teuthology.orchestra.run.smithi164.stdout:1883 write 0x9bbb400 thru 0x9bbe8f9 (0x34fa bytes) 2024-09-21T21:41:09.341 INFO:teuthology.orchestra.run.smithi164.stdout:1884 read 0x9285c00 thru 0x928fa82 (0x9e83 bytes) 2024-09-21T21:41:09.341 INFO:teuthology.orchestra.run.smithi164.stdout:1887 read 0x882fa00 thru 0x883f0cd (0xf6ce bytes) 2024-09-21T21:41:09.341 INFO:teuthology.orchestra.run.smithi164.stdout:1888 read 0x4907a00 thru 0x4907b89 (0x18a bytes) 2024-09-21T21:41:09.341 INFO:teuthology.orchestra.run.smithi164.stdout:1891 read 0xb871400 thru 0xb8787c6 (0x73c7 bytes) 2024-09-21T21:41:09.342 INFO:teuthology.orchestra.run.smithi164.stdout:1894 punch from 0xb7a4a00 to 0xb7b3200, (0xe800 bytes) 2024-09-21T21:41:09.342 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:09.341+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:09.358 INFO:teuthology.orchestra.run.smithi164.stdout:1895 write 0xea91e00 thru 0xea999d5 (0x7bd6 bytes) 2024-09-21T21:41:09.365 INFO:teuthology.orchestra.run.smithi164.stdout:1901 write 0x7488c00 thru 0x748eeec (0x62ed bytes) 2024-09-21T21:41:09.370 INFO:teuthology.orchestra.run.smithi164.stdout:1902 trunc from 0xea99a00 to 0xa7e9e00 2024-09-21T21:41:09.399 INFO:teuthology.orchestra.run.smithi164.stdout:1903 write 0x858f400 thru 0x8591256 (0x1e57 bytes) 2024-09-21T21:41:09.422 INFO:teuthology.orchestra.run.smithi164.stdout:1904 write 0xe714400 thru 0xe720e8a (0xca8b bytes) 2024-09-21T21:41:09.424 INFO:teuthology.orchestra.run.smithi164.stdout:1905 trunc from 0xe721000 to 0xcaf8c00 2024-09-21T21:41:09.443 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:09.442+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:09.451 INFO:teuthology.orchestra.run.smithi164.stdout:1907 punch from 0x2e45600 to 0x2e54600, (0xf000 bytes) 2024-09-21T21:41:09.452 INFO:teuthology.orchestra.run.smithi164.stdout:1908 punch from 0x9aa600 to 0x9b3800, (0x9200 bytes) 2024-09-21T21:41:09.452 INFO:teuthology.orchestra.run.smithi164.stdout:1909 read 0x4e9b200 thru 0x4e9ee9a (0x3c9b bytes) 2024-09-21T21:41:09.452 INFO:teuthology.orchestra.run.smithi164.stdout:1910 read 0xb4f1800 thru 0xb4f8bb9 (0x73ba bytes) 2024-09-21T21:41:09.452 INFO:teuthology.orchestra.run.smithi164.stdout:1913 punch from 0xa88e00 to 0xa98600, (0xf800 bytes) 2024-09-21T21:41:09.456 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:09.456 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:09.466 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T21:41:09.484 INFO:teuthology.orchestra.run.smithi164.stdout:1916 write 0xe518c00 thru 0xe51dae2 (0x4ee3 bytes) 2024-09-21T21:41:09.496 INFO:teuthology.orchestra.run.smithi164.stdout:1917 trunc from 0xe51dc00 to 0x4d3ec00 2024-09-21T21:41:09.515 INFO:teuthology.orchestra.run.smithi164.stdout:1918 punch from 0x1b63c00 to 0x1b70e00, (0xd200 bytes) 2024-09-21T21:41:09.517 INFO:teuthology.orchestra.run.smithi164.stdout:1919 punch from 0x11d9c00 to 0x11e6000, (0xc400 bytes) 2024-09-21T21:41:09.518 INFO:teuthology.orchestra.run.smithi164.stdout:1921 trunc from 0x4d3ec00 to 0x57c9800 2024-09-21T21:41:09.535 INFO:teuthology.orchestra.run.smithi164.stdout:1923 punch from 0x3936a00 to 0x3941e00, (0xb400 bytes) 2024-09-21T21:41:09.537 INFO:teuthology.orchestra.run.smithi164.stdout:1925 punch from 0x4a33a00 to 0x4a42400, (0xea00 bytes) 2024-09-21T21:41:09.544 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:09.543+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:09.554 INFO:teuthology.orchestra.run.smithi164.stdout:1928 write 0x6ea0400 thru 0x6ea69fb (0x65fc bytes) 2024-09-21T21:41:09.560 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-21T21:41:09.560 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T21:41:09.560 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-21T21:41:09.560 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-21T21:41:09.576 INFO:teuthology.orchestra.run.smithi081.stderr:nodeep-scrub is unset 2024-09-21T21:41:09.579 INFO:teuthology.orchestra.run.smithi164.stdout:1929 write 0xdba6200 thru 0xdbb1a80 (0xb881 bytes) 2024-09-21T21:41:09.591 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:09.591 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:09.596 INFO:teuthology.orchestra.run.smithi164.stdout:1932 trunc from 0xdbb1c00 to 0x6661a00 2024-09-21T21:41:09.599 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T21:41:09.622 INFO:teuthology.orchestra.run.smithi164.stdout:1936 trunc from 0x6661a00 to 0xcbcfc00 2024-09-21T21:41:09.639 INFO:teuthology.orchestra.run.smithi164.stdout:1937 write 0x545b800 thru 0x54635af (0x7db0 bytes) 2024-09-21T21:41:09.640 INFO:teuthology.orchestra.run.smithi164.stdout:1940 write 0x5bffa00 thru 0x5c0cd5a (0xd35b bytes) 2024-09-21T21:41:09.641 INFO:teuthology.orchestra.run.smithi164.stdout:1941 write 0x4d07e00 thru 0x4d1342c (0xb62d bytes) 2024-09-21T21:41:09.641 INFO:teuthology.orchestra.run.smithi164.stdout:1944 write 0x97fc400 thru 0x97ff326 (0x2f27 bytes) 2024-09-21T21:41:09.641 INFO:teuthology.orchestra.run.smithi164.stdout:1945 read 0xafdfa00 thru 0xafe521c (0x581d bytes) 2024-09-21T21:41:09.669 INFO:teuthology.orchestra.run.smithi164.stdout:1946 write 0xdbbd400 thru 0xdbc6224 (0x8e25 bytes) 2024-09-21T21:41:09.671 INFO:teuthology.orchestra.run.smithi164.stdout:1948 write 0xd424a00 thru 0xd42b6a9 (0x6caa bytes) 2024-09-21T21:41:09.671 INFO:teuthology.orchestra.run.smithi164.stdout:1949 read 0x3f3200 thru 0x40161f (0xe420 bytes) 2024-09-21T21:41:09.672 INFO:teuthology.orchestra.run.smithi164.stdout:1950 read 0x973e800 thru 0x974d724 (0xef25 bytes) 2024-09-21T21:41:09.672 INFO:teuthology.orchestra.run.smithi164.stdout:1952 read 0x89ab200 thru 0x89ab8db (0x6dc bytes) 2024-09-21T21:41:09.672 INFO:teuthology.orchestra.run.smithi164.stdout:1954 read 0x47c0c00 thru 0x47cfcac (0xf0ad bytes) 2024-09-21T21:41:09.676 INFO:teuthology.orchestra.run.smithi164.stdout:1958 trunc from 0xdbc6400 to 0xa057a00 2024-09-21T21:41:09.685 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:09.684+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:09.693 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T21:41:09.710 INFO:teuthology.orchestra.run.smithi164.stdout:1959 punch from 0x55f9200 to 0x5603800, (0xa600 bytes) 2024-09-21T21:41:09.711 INFO:teuthology.orchestra.run.smithi164.stdout:1960 punch from 0x9c3aa00 to 0x9c48200, (0xd800 bytes) 2024-09-21T21:41:09.713 INFO:teuthology.orchestra.run.smithi164.stdout:1961 trunc from 0xa057a00 to 0x885bc00 2024-09-21T21:41:09.745 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:09.744+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:09.748 INFO:teuthology.orchestra.run.smithi164.stdout:1962 read 0x3771200 thru 0x37793e6 (0x81e7 bytes) 2024-09-21T21:41:09.748 INFO:teuthology.orchestra.run.smithi164.stdout:1963 read 0x14a6e00 thru 0x14b051f (0x9720 bytes) 2024-09-21T21:41:09.749 INFO:teuthology.orchestra.run.smithi164.stdout:1964 punch from 0xc92a00 to 0xc9b800, (0x8e00 bytes) 2024-09-21T21:41:09.749 INFO:teuthology.orchestra.run.smithi164.stdout:1965 write 0x68a4200 thru 0x68a92e5 (0x50e6 bytes) 2024-09-21T21:41:09.749 INFO:teuthology.orchestra.run.smithi164.stdout:1966 punch from 0x43e7400 to 0x43f5800, (0xe400 bytes) 2024-09-21T21:41:09.749 INFO:teuthology.orchestra.run.smithi164.stdout:1967 read 0x5626200 thru 0x56308f0 (0xa6f1 bytes) 2024-09-21T21:41:09.754 INFO:teuthology.orchestra.run.smithi164.stdout:1969 trunc from 0x885bc00 to 0x4741e00 2024-09-21T21:41:09.776 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:09.776 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:09.786 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T21:41:09.834 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0: {} 2024-09-21T21:41:09.834 INFO:teuthology.orchestra.run.smithi081.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T21:41:09.846 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:09.845+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:09.852 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1: {} 2024-09-21T21:41:09.852 INFO:teuthology.orchestra.run.smithi081.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T21:41:09.868 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2: {} 2024-09-21T21:41:09.868 INFO:teuthology.orchestra.run.smithi081.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T21:41:09.886 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3: {} 2024-09-21T21:41:09.886 INFO:teuthology.orchestra.run.smithi081.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T21:41:09.893 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:09.893 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:41:09.894 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:41:09.894 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:41:09.894 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:09.902 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:41:09.903 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:09.903 INFO:teuthology.orchestra.run.smithi081.stdout: "epoch": 44, 2024-09-21T21:41:09.903 INFO:teuthology.orchestra.run.smithi081.stdout: "fsid": "a0fdcfc2-ddd2-4e50-81bb-876d940f3578", 2024-09-21T21:41:09.903 INFO:teuthology.orchestra.run.smithi081.stdout: "created": "2024-09-21T21:40:07.413917+0000", 2024-09-21T21:41:09.903 INFO:teuthology.orchestra.run.smithi081.stdout: "modified": "2024-09-21T21:41:09.564472+0000", 2024-09-21T21:41:09.903 INFO:teuthology.orchestra.run.smithi081.stdout: "last_up_change": "2024-09-21T21:40:18.340430+0000", 2024-09-21T21:41:09.903 INFO:teuthology.orchestra.run.smithi081.stdout: "last_in_change": "2024-09-21T21:40:09.029953+0000", 2024-09-21T21:41:09.903 INFO:teuthology.orchestra.run.smithi081.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-21T21:41:09.903 INFO:teuthology.orchestra.run.smithi081.stdout: "flags_num": 5799936, 2024-09-21T21:41:09.903 INFO:teuthology.orchestra.run.smithi081.stdout: "flags_set": [ 2024-09-21T21:41:09.903 INFO:teuthology.orchestra.run.smithi081.stdout: "pglog_hardlimit", 2024-09-21T21:41:09.903 INFO:teuthology.orchestra.run.smithi081.stdout: "purged_snapdirs", 2024-09-21T21:41:09.903 INFO:teuthology.orchestra.run.smithi081.stdout: "recovery_deletes", 2024-09-21T21:41:09.904 INFO:teuthology.orchestra.run.smithi081.stdout: "sortbitwise" 2024-09-21T21:41:09.904 INFO:teuthology.orchestra.run.smithi081.stdout: ], 2024-09-21T21:41:09.904 INFO:teuthology.orchestra.run.smithi081.stdout: "crush_version": 5, 2024-09-21T21:41:09.904 INFO:teuthology.orchestra.run.smithi081.stdout: "full_ratio": 0.94999998807907104, 2024-09-21T21:41:09.904 INFO:teuthology.orchestra.run.smithi081.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-21T21:41:09.904 INFO:teuthology.orchestra.run.smithi081.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-21T21:41:09.904 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster_snapshot": "", 2024-09-21T21:41:09.904 INFO:teuthology.orchestra.run.smithi081.stdout: "pool_max": 3, 2024-09-21T21:41:09.904 INFO:teuthology.orchestra.run.smithi081.stdout: "max_osd": 6, 2024-09-21T21:41:09.904 INFO:teuthology.orchestra.run.smithi081.stdout: "require_min_compat_client": "luminous", 2024-09-21T21:41:09.904 INFO:teuthology.orchestra.run.smithi081.stdout: "min_compat_client": "jewel", 2024-09-21T21:41:09.904 INFO:teuthology.orchestra.run.smithi081.stdout: "require_osd_release": "squid", 2024-09-21T21:41:09.904 INFO:teuthology.orchestra.run.smithi081.stdout: "allow_crimson": false, 2024-09-21T21:41:09.905 INFO:teuthology.orchestra.run.smithi081.stdout: "pools": [ 2024-09-21T21:41:09.905 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:09.905 INFO:teuthology.orchestra.run.smithi081.stdout: "pool": 1, 2024-09-21T21:41:09.905 INFO:teuthology.orchestra.run.smithi081.stdout: "pool_name": ".mgr", 2024-09-21T21:41:09.905 INFO:teuthology.orchestra.run.smithi081.stdout: "create_time": "2024-09-21T21:40:18.321328+0000", 2024-09-21T21:41:09.905 INFO:teuthology.orchestra.run.smithi081.stdout: "flags": 1, 2024-09-21T21:41:09.905 INFO:teuthology.orchestra.run.smithi081.stdout: "flags_names": "hashpspool", 2024-09-21T21:41:09.905 INFO:teuthology.orchestra.run.smithi081.stdout: "type": 1, 2024-09-21T21:41:09.905 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 2, 2024-09-21T21:41:09.905 INFO:teuthology.orchestra.run.smithi081.stdout: "min_size": 1, 2024-09-21T21:41:09.905 INFO:teuthology.orchestra.run.smithi081.stdout: "crush_rule": 0, 2024-09-21T21:41:09.905 INFO:teuthology.orchestra.run.smithi081.stdout: "peering_crush_bucket_count": 0, 2024-09-21T21:41:09.905 INFO:teuthology.orchestra.run.smithi081.stdout: "peering_crush_bucket_target": 0, 2024-09-21T21:41:09.905 INFO:teuthology.orchestra.run.smithi081.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T21:41:09.906 INFO:teuthology.orchestra.run.smithi081.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T21:41:09.906 INFO:teuthology.orchestra.run.smithi081.stdout: "is_stretch_pool": false, 2024-09-21T21:41:09.906 INFO:teuthology.orchestra.run.smithi081.stdout: "object_hash": 2, 2024-09-21T21:41:09.906 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_autoscale_mode": "off", 2024-09-21T21:41:09.906 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_num": 1, 2024-09-21T21:41:09.906 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_placement_num": 1, 2024-09-21T21:41:09.906 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_placement_num_target": 1, 2024-09-21T21:41:09.906 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_num_target": 1, 2024-09-21T21:41:09.906 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_num_pending": 1, 2024-09-21T21:41:09.906 INFO:teuthology.orchestra.run.smithi081.stdout: "last_pg_merge_meta": { 2024-09-21T21:41:09.906 INFO:teuthology.orchestra.run.smithi081.stdout: "source_pgid": "0.0", 2024-09-21T21:41:09.906 INFO:teuthology.orchestra.run.smithi081.stdout: "ready_epoch": 0, 2024-09-21T21:41:09.906 INFO:teuthology.orchestra.run.smithi081.stdout: "last_epoch_started": 0, 2024-09-21T21:41:09.906 INFO:teuthology.orchestra.run.smithi081.stdout: "last_epoch_clean": 0, 2024-09-21T21:41:09.907 INFO:teuthology.orchestra.run.smithi081.stdout: "source_version": "0'0", 2024-09-21T21:41:09.907 INFO:teuthology.orchestra.run.smithi081.stdout: "target_version": "0'0" 2024-09-21T21:41:09.907 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:09.907 INFO:teuthology.orchestra.run.smithi081.stdout: "last_change": "15", 2024-09-21T21:41:09.907 INFO:teuthology.orchestra.run.smithi081.stdout: "last_force_op_resend": "0", 2024-09-21T21:41:09.907 INFO:teuthology.orchestra.run.smithi081.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T21:41:09.907 INFO:teuthology.orchestra.run.smithi081.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T21:41:09.907 INFO:teuthology.orchestra.run.smithi081.stdout: "auid": 0, 2024-09-21T21:41:09.907 INFO:teuthology.orchestra.run.smithi081.stdout: "snap_mode": "selfmanaged", 2024-09-21T21:41:09.907 INFO:teuthology.orchestra.run.smithi081.stdout: "snap_seq": 0, 2024-09-21T21:41:09.907 INFO:teuthology.orchestra.run.smithi081.stdout: "snap_epoch": 0, 2024-09-21T21:41:09.907 INFO:teuthology.orchestra.run.smithi081.stdout: "pool_snaps": [], 2024-09-21T21:41:09.907 INFO:teuthology.orchestra.run.smithi081.stdout: "removed_snaps": "[]", 2024-09-21T21:41:09.907 INFO:teuthology.orchestra.run.smithi081.stdout: "quota_max_bytes": 0, 2024-09-21T21:41:09.907 INFO:teuthology.orchestra.run.smithi081.stdout: "quota_max_objects": 0, 2024-09-21T21:41:09.908 INFO:teuthology.orchestra.run.smithi081.stdout: "tiers": [], 2024-09-21T21:41:09.908 INFO:teuthology.orchestra.run.smithi081.stdout: "tier_of": -1, 2024-09-21T21:41:09.908 INFO:teuthology.orchestra.run.smithi081.stdout: "read_tier": -1, 2024-09-21T21:41:09.908 INFO:teuthology.orchestra.run.smithi081.stdout: "write_tier": -1, 2024-09-21T21:41:09.908 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_mode": "none", 2024-09-21T21:41:09.908 INFO:teuthology.orchestra.run.smithi081.stdout: "target_max_bytes": 0, 2024-09-21T21:41:09.908 INFO:teuthology.orchestra.run.smithi081.stdout: "target_max_objects": 0, 2024-09-21T21:41:09.908 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T21:41:09.908 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T21:41:09.908 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T21:41:09.908 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_min_flush_age": 0, 2024-09-21T21:41:09.908 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_min_evict_age": 0, 2024-09-21T21:41:09.908 INFO:teuthology.orchestra.run.smithi081.stdout: "erasure_code_profile": "", 2024-09-21T21:41:09.908 INFO:teuthology.orchestra.run.smithi081.stdout: "hit_set_params": { 2024-09-21T21:41:09.909 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "none" 2024-09-21T21:41:09.909 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:09.909 INFO:teuthology.orchestra.run.smithi081.stdout: "hit_set_period": 0, 2024-09-21T21:41:09.909 INFO:teuthology.orchestra.run.smithi081.stdout: "hit_set_count": 0, 2024-09-21T21:41:09.909 INFO:teuthology.orchestra.run.smithi081.stdout: "use_gmt_hitset": true, 2024-09-21T21:41:09.909 INFO:teuthology.orchestra.run.smithi081.stdout: "min_read_recency_for_promote": 0, 2024-09-21T21:41:09.909 INFO:teuthology.orchestra.run.smithi081.stdout: "min_write_recency_for_promote": 0, 2024-09-21T21:41:09.909 INFO:teuthology.orchestra.run.smithi081.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T21:41:09.909 INFO:teuthology.orchestra.run.smithi081.stdout: "hit_set_search_last_n": 0, 2024-09-21T21:41:09.909 INFO:teuthology.orchestra.run.smithi081.stdout: "grade_table": [], 2024-09-21T21:41:09.909 INFO:teuthology.orchestra.run.smithi081.stdout: "stripe_width": 0, 2024-09-21T21:41:09.909 INFO:teuthology.orchestra.run.smithi081.stdout: "expected_num_objects": 0, 2024-09-21T21:41:09.909 INFO:teuthology.orchestra.run.smithi081.stdout: "fast_read": false, 2024-09-21T21:41:09.909 INFO:teuthology.orchestra.run.smithi081.stdout: "options": { 2024-09-21T21:41:09.910 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_num_max": 32, 2024-09-21T21:41:09.910 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_num_min": 1 2024-09-21T21:41:09.910 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:09.910 INFO:teuthology.orchestra.run.smithi081.stdout: "application_metadata": { 2024-09-21T21:41:09.910 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": {} 2024-09-21T21:41:09.910 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:09.910 INFO:teuthology.orchestra.run.smithi081.stdout: "read_balance": { 2024-09-21T21:41:09.910 INFO:teuthology.orchestra.run.smithi081.stdout: "score_type": "Fair distribution", 2024-09-21T21:41:09.910 INFO:teuthology.orchestra.run.smithi081.stdout: "score_acting": 6.059999942779541, 2024-09-21T21:41:09.910 INFO:teuthology.orchestra.run.smithi081.stdout: "score_stable": 6.059999942779541, 2024-09-21T21:41:09.910 INFO:teuthology.orchestra.run.smithi081.stdout: "optimal_score": 0.33000001311302185, 2024-09-21T21:41:09.910 INFO:teuthology.orchestra.run.smithi081.stdout: "raw_score_acting": 2, 2024-09-21T21:41:09.910 INFO:teuthology.orchestra.run.smithi081.stdout: "raw_score_stable": 2, 2024-09-21T21:41:09.910 INFO:teuthology.orchestra.run.smithi081.stdout: "primary_affinity_weighted": 0.88800817728042603, 2024-09-21T21:41:09.911 INFO:teuthology.orchestra.run.smithi081.stdout: "average_primary_affinity": 0.88999998569488525, 2024-09-21T21:41:09.911 INFO:teuthology.orchestra.run.smithi081.stdout: "average_primary_affinity_weighted": 0.88999998569488525 2024-09-21T21:41:09.911 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:09.911 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:09.911 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:09.911 INFO:teuthology.orchestra.run.smithi081.stdout: "pool": 2, 2024-09-21T21:41:09.911 INFO:teuthology.orchestra.run.smithi081.stdout: "pool_name": "rbd", 2024-09-21T21:41:09.911 INFO:teuthology.orchestra.run.smithi081.stdout: "create_time": "2024-09-21T21:40:24.235658+0000", 2024-09-21T21:41:09.911 INFO:teuthology.orchestra.run.smithi081.stdout: "flags": 8193, 2024-09-21T21:41:09.911 INFO:teuthology.orchestra.run.smithi081.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T21:41:09.911 INFO:teuthology.orchestra.run.smithi081.stdout: "type": 1, 2024-09-21T21:41:09.911 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 2, 2024-09-21T21:41:09.911 INFO:teuthology.orchestra.run.smithi081.stdout: "min_size": 1, 2024-09-21T21:41:09.911 INFO:teuthology.orchestra.run.smithi081.stdout: "crush_rule": 0, 2024-09-21T21:41:09.912 INFO:teuthology.orchestra.run.smithi081.stdout: "peering_crush_bucket_count": 0, 2024-09-21T21:41:09.912 INFO:teuthology.orchestra.run.smithi081.stdout: "peering_crush_bucket_target": 0, 2024-09-21T21:41:09.912 INFO:teuthology.orchestra.run.smithi081.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T21:41:09.912 INFO:teuthology.orchestra.run.smithi081.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T21:41:09.912 INFO:teuthology.orchestra.run.smithi081.stdout: "is_stretch_pool": false, 2024-09-21T21:41:09.912 INFO:teuthology.orchestra.run.smithi081.stdout: "object_hash": 2, 2024-09-21T21:41:09.912 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_autoscale_mode": "off", 2024-09-21T21:41:09.912 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_num": 8, 2024-09-21T21:41:09.912 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_placement_num": 8, 2024-09-21T21:41:09.912 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_placement_num_target": 8, 2024-09-21T21:41:09.912 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_num_target": 8, 2024-09-21T21:41:09.912 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_num_pending": 8, 2024-09-21T21:41:09.912 INFO:teuthology.orchestra.run.smithi081.stdout: "last_pg_merge_meta": { 2024-09-21T21:41:09.912 INFO:teuthology.orchestra.run.smithi081.stdout: "source_pgid": "0.0", 2024-09-21T21:41:09.913 INFO:teuthology.orchestra.run.smithi081.stdout: "ready_epoch": 0, 2024-09-21T21:41:09.913 INFO:teuthology.orchestra.run.smithi081.stdout: "last_epoch_started": 0, 2024-09-21T21:41:09.913 INFO:teuthology.orchestra.run.smithi081.stdout: "last_epoch_clean": 0, 2024-09-21T21:41:09.913 INFO:teuthology.orchestra.run.smithi081.stdout: "source_version": "0'0", 2024-09-21T21:41:09.913 INFO:teuthology.orchestra.run.smithi081.stdout: "target_version": "0'0" 2024-09-21T21:41:09.913 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:09.913 INFO:teuthology.orchestra.run.smithi081.stdout: "last_change": "19", 2024-09-21T21:41:09.913 INFO:teuthology.orchestra.run.smithi081.stdout: "last_force_op_resend": "0", 2024-09-21T21:41:09.913 INFO:teuthology.orchestra.run.smithi081.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T21:41:09.913 INFO:teuthology.orchestra.run.smithi081.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T21:41:09.913 INFO:teuthology.orchestra.run.smithi081.stdout: "auid": 0, 2024-09-21T21:41:09.913 INFO:teuthology.orchestra.run.smithi081.stdout: "snap_mode": "selfmanaged", 2024-09-21T21:41:09.913 INFO:teuthology.orchestra.run.smithi081.stdout: "snap_seq": 2, 2024-09-21T21:41:09.913 INFO:teuthology.orchestra.run.smithi081.stdout: "snap_epoch": 19, 2024-09-21T21:41:09.914 INFO:teuthology.orchestra.run.smithi081.stdout: "pool_snaps": [], 2024-09-21T21:41:09.914 INFO:teuthology.orchestra.run.smithi081.stdout: "removed_snaps": "[]", 2024-09-21T21:41:09.914 INFO:teuthology.orchestra.run.smithi081.stdout: "quota_max_bytes": 0, 2024-09-21T21:41:09.914 INFO:teuthology.orchestra.run.smithi081.stdout: "quota_max_objects": 0, 2024-09-21T21:41:09.914 INFO:teuthology.orchestra.run.smithi081.stdout: "tiers": [], 2024-09-21T21:41:09.914 INFO:teuthology.orchestra.run.smithi081.stdout: "tier_of": -1, 2024-09-21T21:41:09.914 INFO:teuthology.orchestra.run.smithi081.stdout: "read_tier": -1, 2024-09-21T21:41:09.914 INFO:teuthology.orchestra.run.smithi081.stdout: "write_tier": -1, 2024-09-21T21:41:09.914 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_mode": "none", 2024-09-21T21:41:09.914 INFO:teuthology.orchestra.run.smithi081.stdout: "target_max_bytes": 0, 2024-09-21T21:41:09.914 INFO:teuthology.orchestra.run.smithi081.stdout: "target_max_objects": 0, 2024-09-21T21:41:09.914 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T21:41:09.914 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T21:41:09.914 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T21:41:09.915 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_min_flush_age": 0, 2024-09-21T21:41:09.915 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_min_evict_age": 0, 2024-09-21T21:41:09.915 INFO:teuthology.orchestra.run.smithi081.stdout: "erasure_code_profile": "", 2024-09-21T21:41:09.915 INFO:teuthology.orchestra.run.smithi081.stdout: "hit_set_params": { 2024-09-21T21:41:09.915 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "none" 2024-09-21T21:41:09.915 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:09.915 INFO:teuthology.orchestra.run.smithi081.stdout: "hit_set_period": 0, 2024-09-21T21:41:09.915 INFO:teuthology.orchestra.run.smithi081.stdout: "hit_set_count": 0, 2024-09-21T21:41:09.915 INFO:teuthology.orchestra.run.smithi081.stdout: "use_gmt_hitset": true, 2024-09-21T21:41:09.915 INFO:teuthology.orchestra.run.smithi081.stdout: "min_read_recency_for_promote": 0, 2024-09-21T21:41:09.915 INFO:teuthology.orchestra.run.smithi081.stdout: "min_write_recency_for_promote": 0, 2024-09-21T21:41:09.915 INFO:teuthology.orchestra.run.smithi081.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T21:41:09.915 INFO:teuthology.orchestra.run.smithi081.stdout: "hit_set_search_last_n": 0, 2024-09-21T21:41:09.915 INFO:teuthology.orchestra.run.smithi081.stdout: "grade_table": [], 2024-09-21T21:41:09.915 INFO:teuthology.orchestra.run.smithi081.stdout: "stripe_width": 0, 2024-09-21T21:41:09.915 INFO:teuthology.orchestra.run.smithi081.stdout: "expected_num_objects": 0, 2024-09-21T21:41:09.916 INFO:teuthology.orchestra.run.smithi081.stdout: "fast_read": false, 2024-09-21T21:41:09.916 INFO:teuthology.orchestra.run.smithi081.stdout: "options": {}, 2024-09-21T21:41:09.916 INFO:teuthology.orchestra.run.smithi081.stdout: "application_metadata": { 2024-09-21T21:41:09.916 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd": {} 2024-09-21T21:41:09.916 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:09.916 INFO:teuthology.orchestra.run.smithi081.stdout: "read_balance": { 2024-09-21T21:41:09.916 INFO:teuthology.orchestra.run.smithi081.stdout: "score_type": "Fair distribution", 2024-09-21T21:41:09.916 INFO:teuthology.orchestra.run.smithi081.stdout: "score_acting": 2.25, 2024-09-21T21:41:09.916 INFO:teuthology.orchestra.run.smithi081.stdout: "score_stable": 2.25, 2024-09-21T21:41:09.916 INFO:teuthology.orchestra.run.smithi081.stdout: "optimal_score": 1, 2024-09-21T21:41:09.916 INFO:teuthology.orchestra.run.smithi081.stdout: "raw_score_acting": 2.25, 2024-09-21T21:41:09.916 INFO:teuthology.orchestra.run.smithi081.stdout: "raw_score_stable": 2.25, 2024-09-21T21:41:09.916 INFO:teuthology.orchestra.run.smithi081.stdout: "primary_affinity_weighted": 0.88800817728042603, 2024-09-21T21:41:09.916 INFO:teuthology.orchestra.run.smithi081.stdout: "average_primary_affinity": 0.88999998569488525, 2024-09-21T21:41:09.916 INFO:teuthology.orchestra.run.smithi081.stdout: "average_primary_affinity_weighted": 0.88999998569488525 2024-09-21T21:41:09.917 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:09.917 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:09.917 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:09.917 INFO:teuthology.orchestra.run.smithi081.stdout: "pool": 3, 2024-09-21T21:41:09.917 INFO:teuthology.orchestra.run.smithi081.stdout: "pool_name": "pool_client.0", 2024-09-21T21:41:09.917 INFO:teuthology.orchestra.run.smithi081.stdout: "create_time": "2024-09-21T21:40:36.093691+0000", 2024-09-21T21:41:09.917 INFO:teuthology.orchestra.run.smithi081.stdout: "flags": 8193, 2024-09-21T21:41:09.917 INFO:teuthology.orchestra.run.smithi081.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T21:41:09.917 INFO:teuthology.orchestra.run.smithi081.stdout: "type": 1, 2024-09-21T21:41:09.917 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 2, 2024-09-21T21:41:09.917 INFO:teuthology.orchestra.run.smithi081.stdout: "min_size": 1, 2024-09-21T21:41:09.917 INFO:teuthology.orchestra.run.smithi081.stdout: "crush_rule": 0, 2024-09-21T21:41:09.917 INFO:teuthology.orchestra.run.smithi081.stdout: "peering_crush_bucket_count": 0, 2024-09-21T21:41:09.917 INFO:teuthology.orchestra.run.smithi081.stdout: "peering_crush_bucket_target": 0, 2024-09-21T21:41:09.918 INFO:teuthology.orchestra.run.smithi081.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T21:41:09.918 INFO:teuthology.orchestra.run.smithi081.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T21:41:09.918 INFO:teuthology.orchestra.run.smithi081.stdout: "is_stretch_pool": false, 2024-09-21T21:41:09.918 INFO:teuthology.orchestra.run.smithi081.stdout: "object_hash": 2, 2024-09-21T21:41:09.918 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_autoscale_mode": "off", 2024-09-21T21:41:09.918 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_num": 32, 2024-09-21T21:41:09.918 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_placement_num": 32, 2024-09-21T21:41:09.918 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_placement_num_target": 32, 2024-09-21T21:41:09.918 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_num_target": 32, 2024-09-21T21:41:09.918 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_num_pending": 32, 2024-09-21T21:41:09.918 INFO:teuthology.orchestra.run.smithi081.stdout: "last_pg_merge_meta": { 2024-09-21T21:41:09.918 INFO:teuthology.orchestra.run.smithi081.stdout: "source_pgid": "0.0", 2024-09-21T21:41:09.918 INFO:teuthology.orchestra.run.smithi081.stdout: "ready_epoch": 0, 2024-09-21T21:41:09.919 INFO:teuthology.orchestra.run.smithi081.stdout: "last_epoch_started": 0, 2024-09-21T21:41:09.919 INFO:teuthology.orchestra.run.smithi081.stdout: "last_epoch_clean": 0, 2024-09-21T21:41:09.919 INFO:teuthology.orchestra.run.smithi081.stdout: "source_version": "0'0", 2024-09-21T21:41:09.919 INFO:teuthology.orchestra.run.smithi081.stdout: "target_version": "0'0" 2024-09-21T21:41:09.919 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:09.919 INFO:teuthology.orchestra.run.smithi081.stdout: "last_change": "43", 2024-09-21T21:41:09.919 INFO:teuthology.orchestra.run.smithi081.stdout: "last_force_op_resend": "0", 2024-09-21T21:41:09.919 INFO:teuthology.orchestra.run.smithi081.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T21:41:09.919 INFO:teuthology.orchestra.run.smithi081.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T21:41:09.919 INFO:teuthology.orchestra.run.smithi081.stdout: "auid": 0, 2024-09-21T21:41:09.919 INFO:teuthology.orchestra.run.smithi081.stdout: "snap_mode": "selfmanaged", 2024-09-21T21:41:09.919 INFO:teuthology.orchestra.run.smithi081.stdout: "snap_seq": 14, 2024-09-21T21:41:09.919 INFO:teuthology.orchestra.run.smithi081.stdout: "snap_epoch": 43, 2024-09-21T21:41:09.920 INFO:teuthology.orchestra.run.smithi081.stdout: "pool_snaps": [], 2024-09-21T21:41:09.920 INFO:teuthology.orchestra.run.smithi081.stdout: "removed_snaps": "[]", 2024-09-21T21:41:09.920 INFO:teuthology.orchestra.run.smithi081.stdout: "quota_max_bytes": 0, 2024-09-21T21:41:09.920 INFO:teuthology.orchestra.run.smithi081.stdout: "quota_max_objects": 0, 2024-09-21T21:41:09.920 INFO:teuthology.orchestra.run.smithi081.stdout: "tiers": [], 2024-09-21T21:41:09.920 INFO:teuthology.orchestra.run.smithi081.stdout: "tier_of": -1, 2024-09-21T21:41:09.920 INFO:teuthology.orchestra.run.smithi081.stdout: "read_tier": -1, 2024-09-21T21:41:09.920 INFO:teuthology.orchestra.run.smithi081.stdout: "write_tier": -1, 2024-09-21T21:41:09.920 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_mode": "none", 2024-09-21T21:41:09.920 INFO:teuthology.orchestra.run.smithi081.stdout: "target_max_bytes": 0, 2024-09-21T21:41:09.920 INFO:teuthology.orchestra.run.smithi081.stdout: "target_max_objects": 0, 2024-09-21T21:41:09.920 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T21:41:09.920 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T21:41:09.920 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T21:41:09.921 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_min_flush_age": 0, 2024-09-21T21:41:09.921 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_min_evict_age": 0, 2024-09-21T21:41:09.921 INFO:teuthology.orchestra.run.smithi081.stdout: "erasure_code_profile": "", 2024-09-21T21:41:09.921 INFO:teuthology.orchestra.run.smithi081.stdout: "hit_set_params": { 2024-09-21T21:41:09.921 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "none" 2024-09-21T21:41:09.921 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:09.921 INFO:teuthology.orchestra.run.smithi081.stdout: "hit_set_period": 0, 2024-09-21T21:41:09.921 INFO:teuthology.orchestra.run.smithi081.stdout: "hit_set_count": 0, 2024-09-21T21:41:09.921 INFO:teuthology.orchestra.run.smithi081.stdout: "use_gmt_hitset": true, 2024-09-21T21:41:09.921 INFO:teuthology.orchestra.run.smithi081.stdout: "min_read_recency_for_promote": 0, 2024-09-21T21:41:09.921 INFO:teuthology.orchestra.run.smithi081.stdout: "min_write_recency_for_promote": 0, 2024-09-21T21:41:09.921 INFO:teuthology.orchestra.run.smithi081.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T21:41:09.921 INFO:teuthology.orchestra.run.smithi081.stdout: "hit_set_search_last_n": 0, 2024-09-21T21:41:09.921 INFO:teuthology.orchestra.run.smithi081.stdout: "grade_table": [], 2024-09-21T21:41:09.922 INFO:teuthology.orchestra.run.smithi081.stdout: "stripe_width": 0, 2024-09-21T21:41:09.922 INFO:teuthology.orchestra.run.smithi081.stdout: "expected_num_objects": 0, 2024-09-21T21:41:09.922 INFO:teuthology.orchestra.run.smithi081.stdout: "fast_read": false, 2024-09-21T21:41:09.922 INFO:teuthology.orchestra.run.smithi081.stdout: "options": {}, 2024-09-21T21:41:09.922 INFO:teuthology.orchestra.run.smithi081.stdout: "application_metadata": { 2024-09-21T21:41:09.922 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd": {} 2024-09-21T21:41:09.922 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:09.922 INFO:teuthology.orchestra.run.smithi081.stdout: "read_balance": { 2024-09-21T21:41:09.922 INFO:teuthology.orchestra.run.smithi081.stdout: "score_type": "Fair distribution", 2024-09-21T21:41:09.922 INFO:teuthology.orchestra.run.smithi081.stdout: "score_acting": 1.5, 2024-09-21T21:41:09.922 INFO:teuthology.orchestra.run.smithi081.stdout: "score_stable": 1.5, 2024-09-21T21:41:09.922 INFO:teuthology.orchestra.run.smithi081.stdout: "optimal_score": 1, 2024-09-21T21:41:09.922 INFO:teuthology.orchestra.run.smithi081.stdout: "raw_score_acting": 1.5, 2024-09-21T21:41:09.922 INFO:teuthology.orchestra.run.smithi081.stdout: "raw_score_stable": 1.5, 2024-09-21T21:41:09.923 INFO:teuthology.orchestra.run.smithi081.stdout: "primary_affinity_weighted": 0.88800817728042603, 2024-09-21T21:41:09.923 INFO:teuthology.orchestra.run.smithi081.stdout: "average_primary_affinity": 0.88999998569488525, 2024-09-21T21:41:09.923 INFO:teuthology.orchestra.run.smithi081.stdout: "average_primary_affinity_weighted": 0.88999998569488525 2024-09-21T21:41:09.923 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:09.923 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:09.923 INFO:teuthology.orchestra.run.smithi081.stdout: ], 2024-09-21T21:41:09.923 INFO:teuthology.orchestra.run.smithi081.stdout: "osds": [ 2024-09-21T21:41:09.923 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:09.923 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": 0, 2024-09-21T21:41:09.923 INFO:teuthology.orchestra.run.smithi081.stdout: "uuid": "a9869830-8e74-4b02-963d-ab630a0ee286", 2024-09-21T21:41:09.923 INFO:teuthology.orchestra.run.smithi081.stdout: "up": 1, 2024-09-21T21:41:09.923 INFO:teuthology.orchestra.run.smithi081.stdout: "in": 1, 2024-09-21T21:41:09.923 INFO:teuthology.orchestra.run.smithi081.stdout: "weight": 1, 2024-09-21T21:41:09.923 INFO:teuthology.orchestra.run.smithi081.stdout: "primary_affinity": 1, 2024-09-21T21:41:09.924 INFO:teuthology.orchestra.run.smithi081.stdout: "last_clean_begin": 0, 2024-09-21T21:41:09.924 INFO:teuthology.orchestra.run.smithi081.stdout: "last_clean_end": 0, 2024-09-21T21:41:09.924 INFO:teuthology.orchestra.run.smithi081.stdout: "up_from": 12, 2024-09-21T21:41:09.924 INFO:teuthology.orchestra.run.smithi081.stdout: "up_thru": 40, 2024-09-21T21:41:09.924 INFO:teuthology.orchestra.run.smithi081.stdout: "down_at": 0, 2024-09-21T21:41:09.924 INFO:teuthology.orchestra.run.smithi081.stdout: "lost_at": 0, 2024-09-21T21:41:09.924 INFO:teuthology.orchestra.run.smithi081.stdout: "public_addrs": { 2024-09-21T21:41:09.924 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:41:09.924 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:09.924 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:41:09.924 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6808", 2024-09-21T21:41:09.924 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1657552338 2024-09-21T21:41:09.924 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:09.924 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:09.925 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:41:09.925 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6809", 2024-09-21T21:41:09.925 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1657552338 2024-09-21T21:41:09.925 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:09.925 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:09.925 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:09.925 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster_addrs": { 2024-09-21T21:41:09.925 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:41:09.925 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:09.925 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:41:09.925 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6810", 2024-09-21T21:41:09.925 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1657552338 2024-09-21T21:41:09.925 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:09.925 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:09.925 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:41:09.926 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6811", 2024-09-21T21:41:09.926 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1657552338 2024-09-21T21:41:09.926 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:09.926 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:09.926 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:09.926 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_back_addrs": { 2024-09-21T21:41:09.926 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:41:09.926 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:09.926 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:41:09.926 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6814", 2024-09-21T21:41:09.926 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1657552338 2024-09-21T21:41:09.926 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:09.926 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:09.926 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:41:09.927 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6815", 2024-09-21T21:41:09.927 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1657552338 2024-09-21T21:41:09.927 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:09.927 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:09.927 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:09.927 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_front_addrs": { 2024-09-21T21:41:09.927 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:41:09.927 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:09.927 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:41:09.927 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6812", 2024-09-21T21:41:09.927 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1657552338 2024-09-21T21:41:09.927 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:09.927 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:09.927 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:41:09.927 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6813", 2024-09-21T21:41:09.928 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1657552338 2024-09-21T21:41:09.928 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:09.928 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:09.928 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:09.928 INFO:teuthology.orchestra.run.smithi081.stdout: "public_addr": "172.21.15.81:6809/1657552338", 2024-09-21T21:41:09.928 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster_addr": "172.21.15.81:6811/1657552338", 2024-09-21T21:41:09.928 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_back_addr": "172.21.15.81:6815/1657552338", 2024-09-21T21:41:09.928 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_front_addr": "172.21.15.81:6813/1657552338", 2024-09-21T21:41:09.928 INFO:teuthology.orchestra.run.smithi081.stdout: "state": [ 2024-09-21T21:41:09.928 INFO:teuthology.orchestra.run.smithi081.stdout: "exists", 2024-09-21T21:41:09.928 INFO:teuthology.orchestra.run.smithi081.stdout: "up" 2024-09-21T21:41:09.928 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:09.928 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:09.928 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:09.929 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": 1, 2024-09-21T21:41:09.929 INFO:teuthology.orchestra.run.smithi081.stdout: "uuid": "b189819d-69a3-4e95-96e7-96dc4809ac80", 2024-09-21T21:41:09.929 INFO:teuthology.orchestra.run.smithi081.stdout: "up": 1, 2024-09-21T21:41:09.929 INFO:teuthology.orchestra.run.smithi081.stdout: "in": 1, 2024-09-21T21:41:09.929 INFO:teuthology.orchestra.run.smithi081.stdout: "weight": 1, 2024-09-21T21:41:09.929 INFO:teuthology.orchestra.run.smithi081.stdout: "primary_affinity": 1, 2024-09-21T21:41:09.929 INFO:teuthology.orchestra.run.smithi081.stdout: "last_clean_begin": 0, 2024-09-21T21:41:09.929 INFO:teuthology.orchestra.run.smithi081.stdout: "last_clean_end": 0, 2024-09-21T21:41:09.929 INFO:teuthology.orchestra.run.smithi081.stdout: "up_from": 12, 2024-09-21T21:41:09.929 INFO:teuthology.orchestra.run.smithi081.stdout: "up_thru": 40, 2024-09-21T21:41:09.929 INFO:teuthology.orchestra.run.smithi081.stdout: "down_at": 0, 2024-09-21T21:41:09.929 INFO:teuthology.orchestra.run.smithi081.stdout: "lost_at": 0, 2024-09-21T21:41:09.929 INFO:teuthology.orchestra.run.smithi081.stdout: "public_addrs": { 2024-09-21T21:41:09.929 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:41:09.929 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:09.930 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:41:09.930 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6816", 2024-09-21T21:41:09.930 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2864159502 2024-09-21T21:41:09.930 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:09.930 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:09.930 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:41:09.930 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6817", 2024-09-21T21:41:09.930 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2864159502 2024-09-21T21:41:09.930 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:09.930 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:09.930 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:09.930 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster_addrs": { 2024-09-21T21:41:09.930 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:41:09.930 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:09.930 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:41:09.931 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6818", 2024-09-21T21:41:09.931 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2864159502 2024-09-21T21:41:09.931 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:09.931 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:09.931 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:41:09.931 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6819", 2024-09-21T21:41:09.931 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2864159502 2024-09-21T21:41:09.931 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:09.931 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:09.931 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:09.931 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_back_addrs": { 2024-09-21T21:41:09.931 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:41:09.931 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:09.931 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:41:09.931 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6822", 2024-09-21T21:41:09.932 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2864159502 2024-09-21T21:41:09.932 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:09.932 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:09.932 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:41:09.932 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6823", 2024-09-21T21:41:09.932 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2864159502 2024-09-21T21:41:09.932 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:09.932 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:09.932 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:09.932 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_front_addrs": { 2024-09-21T21:41:09.932 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:41:09.932 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:09.932 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:41:09.933 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6820", 2024-09-21T21:41:09.933 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2864159502 2024-09-21T21:41:09.933 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:09.933 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:09.933 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:41:09.933 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6821", 2024-09-21T21:41:09.933 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2864159502 2024-09-21T21:41:09.933 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:09.933 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:09.933 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:09.933 INFO:teuthology.orchestra.run.smithi081.stdout: "public_addr": "172.21.15.81:6817/2864159502", 2024-09-21T21:41:09.933 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster_addr": "172.21.15.81:6819/2864159502", 2024-09-21T21:41:09.933 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_back_addr": "172.21.15.81:6823/2864159502", 2024-09-21T21:41:09.933 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_front_addr": "172.21.15.81:6821/2864159502", 2024-09-21T21:41:09.934 INFO:teuthology.orchestra.run.smithi081.stdout: "state": [ 2024-09-21T21:41:09.934 INFO:teuthology.orchestra.run.smithi081.stdout: "exists", 2024-09-21T21:41:09.934 INFO:teuthology.orchestra.run.smithi081.stdout: "up" 2024-09-21T21:41:09.934 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:09.934 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:09.934 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:09.934 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": 2, 2024-09-21T21:41:09.934 INFO:teuthology.orchestra.run.smithi081.stdout: "uuid": "8437b99a-1297-4903-ab66-95c5cba2b2bf", 2024-09-21T21:41:09.934 INFO:teuthology.orchestra.run.smithi081.stdout: "up": 1, 2024-09-21T21:41:09.934 INFO:teuthology.orchestra.run.smithi081.stdout: "in": 1, 2024-09-21T21:41:09.934 INFO:teuthology.orchestra.run.smithi081.stdout: "weight": 1, 2024-09-21T21:41:09.934 INFO:teuthology.orchestra.run.smithi081.stdout: "primary_affinity": 1, 2024-09-21T21:41:09.934 INFO:teuthology.orchestra.run.smithi081.stdout: "last_clean_begin": 0, 2024-09-21T21:41:09.934 INFO:teuthology.orchestra.run.smithi081.stdout: "last_clean_end": 0, 2024-09-21T21:41:09.935 INFO:teuthology.orchestra.run.smithi081.stdout: "up_from": 11, 2024-09-21T21:41:09.935 INFO:teuthology.orchestra.run.smithi081.stdout: "up_thru": 40, 2024-09-21T21:41:09.935 INFO:teuthology.orchestra.run.smithi081.stdout: "down_at": 0, 2024-09-21T21:41:09.935 INFO:teuthology.orchestra.run.smithi081.stdout: "lost_at": 0, 2024-09-21T21:41:09.935 INFO:teuthology.orchestra.run.smithi081.stdout: "public_addrs": { 2024-09-21T21:41:09.935 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:41:09.935 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:09.935 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:41:09.935 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6800", 2024-09-21T21:41:09.935 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 3023160932 2024-09-21T21:41:09.935 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:09.935 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:09.935 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:41:09.935 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6801", 2024-09-21T21:41:09.936 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 3023160932 2024-09-21T21:41:09.936 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:09.936 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:09.936 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:09.936 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster_addrs": { 2024-09-21T21:41:09.936 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:41:09.936 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:09.936 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:41:09.936 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6802", 2024-09-21T21:41:09.936 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 3023160932 2024-09-21T21:41:09.936 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:09.936 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:09.936 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:41:09.936 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6803", 2024-09-21T21:41:09.937 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 3023160932 2024-09-21T21:41:09.937 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:09.937 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:09.937 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:09.937 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_back_addrs": { 2024-09-21T21:41:09.937 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:41:09.937 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:09.937 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:41:09.937 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6806", 2024-09-21T21:41:09.937 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 3023160932 2024-09-21T21:41:09.937 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:09.937 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:09.937 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:41:09.937 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6807", 2024-09-21T21:41:09.937 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 3023160932 2024-09-21T21:41:09.938 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:09.938 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:09.938 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:09.938 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_front_addrs": { 2024-09-21T21:41:09.938 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:41:09.938 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:09.938 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:41:09.938 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6804", 2024-09-21T21:41:09.938 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 3023160932 2024-09-21T21:41:09.938 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:09.938 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:09.938 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:41:09.938 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6805", 2024-09-21T21:41:09.939 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 3023160932 2024-09-21T21:41:09.939 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:09.939 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:09.939 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:09.939 INFO:teuthology.orchestra.run.smithi081.stdout: "public_addr": "172.21.15.81:6801/3023160932", 2024-09-21T21:41:09.939 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster_addr": "172.21.15.81:6803/3023160932", 2024-09-21T21:41:09.939 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_back_addr": "172.21.15.81:6807/3023160932", 2024-09-21T21:41:09.939 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_front_addr": "172.21.15.81:6805/3023160932", 2024-09-21T21:41:09.939 INFO:teuthology.orchestra.run.smithi081.stdout: "state": [ 2024-09-21T21:41:09.939 INFO:teuthology.orchestra.run.smithi081.stdout: "exists", 2024-09-21T21:41:09.939 INFO:teuthology.orchestra.run.smithi081.stdout: "up" 2024-09-21T21:41:09.939 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:09.939 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:09.939 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:09.939 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": 3, 2024-09-21T21:41:09.940 INFO:teuthology.orchestra.run.smithi081.stdout: "uuid": "e8848768-c1aa-445b-9554-dc6aa4eefc8d", 2024-09-21T21:41:09.940 INFO:teuthology.orchestra.run.smithi081.stdout: "up": 1, 2024-09-21T21:41:09.940 INFO:teuthology.orchestra.run.smithi081.stdout: "in": 1, 2024-09-21T21:41:09.940 INFO:teuthology.orchestra.run.smithi081.stdout: "weight": 1, 2024-09-21T21:41:09.940 INFO:teuthology.orchestra.run.smithi081.stdout: "primary_affinity": 1, 2024-09-21T21:41:09.940 INFO:teuthology.orchestra.run.smithi081.stdout: "last_clean_begin": 0, 2024-09-21T21:41:09.940 INFO:teuthology.orchestra.run.smithi081.stdout: "last_clean_end": 0, 2024-09-21T21:41:09.940 INFO:teuthology.orchestra.run.smithi081.stdout: "up_from": 13, 2024-09-21T21:41:09.940 INFO:teuthology.orchestra.run.smithi081.stdout: "up_thru": 21, 2024-09-21T21:41:09.940 INFO:teuthology.orchestra.run.smithi081.stdout: "down_at": 0, 2024-09-21T21:41:09.940 INFO:teuthology.orchestra.run.smithi081.stdout: "lost_at": 0, 2024-09-21T21:41:09.940 INFO:teuthology.orchestra.run.smithi081.stdout: "public_addrs": { 2024-09-21T21:41:09.940 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:41:09.941 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:09.941 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:41:09.941 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6808", 2024-09-21T21:41:09.941 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2681016844 2024-09-21T21:41:09.941 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:09.941 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:09.941 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:41:09.941 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6809", 2024-09-21T21:41:09.941 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2681016844 2024-09-21T21:41:09.941 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:09.941 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:09.941 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:09.941 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster_addrs": { 2024-09-21T21:41:09.942 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:41:09.942 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:09.942 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:41:09.942 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6810", 2024-09-21T21:41:09.942 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2681016844 2024-09-21T21:41:09.942 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:09.942 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:09.942 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:41:09.942 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6811", 2024-09-21T21:41:09.942 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2681016844 2024-09-21T21:41:09.942 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:09.942 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:09.943 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:09.943 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_back_addrs": { 2024-09-21T21:41:09.943 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:41:09.943 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:09.943 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:41:09.943 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6814", 2024-09-21T21:41:09.943 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2681016844 2024-09-21T21:41:09.943 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:09.943 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:09.943 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:41:09.943 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6815", 2024-09-21T21:41:09.943 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2681016844 2024-09-21T21:41:09.943 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:09.944 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:09.944 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:09.944 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_front_addrs": { 2024-09-21T21:41:09.944 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:41:09.944 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:09.944 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:41:09.944 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6812", 2024-09-21T21:41:09.944 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2681016844 2024-09-21T21:41:09.944 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:09.944 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:09.944 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:41:09.944 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6813", 2024-09-21T21:41:09.945 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2681016844 2024-09-21T21:41:09.945 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:09.945 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:09.945 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:09.945 INFO:teuthology.orchestra.run.smithi081.stdout: "public_addr": "172.21.15.122:6809/2681016844", 2024-09-21T21:41:09.945 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster_addr": "172.21.15.122:6811/2681016844", 2024-09-21T21:41:09.945 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_back_addr": "172.21.15.122:6815/2681016844", 2024-09-21T21:41:09.945 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_front_addr": "172.21.15.122:6813/2681016844", 2024-09-21T21:41:09.945 INFO:teuthology.orchestra.run.smithi081.stdout: "state": [ 2024-09-21T21:41:09.945 INFO:teuthology.orchestra.run.smithi081.stdout: "exists", 2024-09-21T21:41:09.945 INFO:teuthology.orchestra.run.smithi081.stdout: "up" 2024-09-21T21:41:09.945 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:09.946 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:09.946 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:09.946 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": 4, 2024-09-21T21:41:09.946 INFO:teuthology.orchestra.run.smithi081.stdout: "uuid": "b1d8e85d-a207-4ba6-9322-61d07812083e", 2024-09-21T21:41:09.946 INFO:teuthology.orchestra.run.smithi081.stdout: "up": 1, 2024-09-21T21:41:09.946 INFO:teuthology.orchestra.run.smithi081.stdout: "in": 1, 2024-09-21T21:41:09.946 INFO:teuthology.orchestra.run.smithi081.stdout: "weight": 1, 2024-09-21T21:41:09.946 INFO:teuthology.orchestra.run.smithi081.stdout: "primary_affinity": 1, 2024-09-21T21:41:09.946 INFO:teuthology.orchestra.run.smithi081.stdout: "last_clean_begin": 0, 2024-09-21T21:41:09.946 INFO:teuthology.orchestra.run.smithi081.stdout: "last_clean_end": 0, 2024-09-21T21:41:09.946 INFO:teuthology.orchestra.run.smithi081.stdout: "up_from": 13, 2024-09-21T21:41:09.946 INFO:teuthology.orchestra.run.smithi081.stdout: "up_thru": 40, 2024-09-21T21:41:09.946 INFO:teuthology.orchestra.run.smithi081.stdout: "down_at": 0, 2024-09-21T21:41:09.946 INFO:teuthology.orchestra.run.smithi081.stdout: "lost_at": 0, 2024-09-21T21:41:09.946 INFO:teuthology.orchestra.run.smithi081.stdout: "public_addrs": { 2024-09-21T21:41:09.947 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:41:09.947 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:09.947 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:41:09.947 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6800", 2024-09-21T21:41:09.947 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1095023540 2024-09-21T21:41:09.947 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:09.947 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:09.947 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:41:09.947 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6801", 2024-09-21T21:41:09.947 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1095023540 2024-09-21T21:41:09.947 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:09.947 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:09.947 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:09.948 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster_addrs": { 2024-09-21T21:41:09.948 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:41:09.948 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:09.948 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:41:09.948 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6802", 2024-09-21T21:41:09.948 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1095023540 2024-09-21T21:41:09.948 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:09.948 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:09.948 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:41:09.948 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6803", 2024-09-21T21:41:09.948 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1095023540 2024-09-21T21:41:09.948 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:09.948 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:09.948 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:09.949 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_back_addrs": { 2024-09-21T21:41:09.949 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:41:09.949 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:09.949 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:41:09.949 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6806", 2024-09-21T21:41:09.949 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1095023540 2024-09-21T21:41:09.949 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:09.949 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:09.949 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:41:09.949 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6807", 2024-09-21T21:41:09.949 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1095023540 2024-09-21T21:41:09.949 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:09.949 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:09.950 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:09.950 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_front_addrs": { 2024-09-21T21:41:09.950 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:41:09.950 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:09.950 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:41:09.950 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6804", 2024-09-21T21:41:09.950 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1095023540 2024-09-21T21:41:09.950 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:09.950 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:09.950 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:41:09.950 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6805", 2024-09-21T21:41:09.950 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1095023540 2024-09-21T21:41:09.950 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:09.950 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:09.950 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:09.951 INFO:teuthology.orchestra.run.smithi081.stdout: "public_addr": "172.21.15.122:6801/1095023540", 2024-09-21T21:41:09.951 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster_addr": "172.21.15.122:6803/1095023540", 2024-09-21T21:41:09.951 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_back_addr": "172.21.15.122:6807/1095023540", 2024-09-21T21:41:09.951 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_front_addr": "172.21.15.122:6805/1095023540", 2024-09-21T21:41:09.951 INFO:teuthology.orchestra.run.smithi081.stdout: "state": [ 2024-09-21T21:41:09.951 INFO:teuthology.orchestra.run.smithi081.stdout: "exists", 2024-09-21T21:41:09.951 INFO:teuthology.orchestra.run.smithi081.stdout: "up" 2024-09-21T21:41:09.951 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:09.951 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:09.951 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:09.951 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": 5, 2024-09-21T21:41:09.951 INFO:teuthology.orchestra.run.smithi081.stdout: "uuid": "3ab0450b-d8fd-4ef2-a298-08d5aa2ea32f", 2024-09-21T21:41:09.951 INFO:teuthology.orchestra.run.smithi081.stdout: "up": 1, 2024-09-21T21:41:09.951 INFO:teuthology.orchestra.run.smithi081.stdout: "in": 1, 2024-09-21T21:41:09.951 INFO:teuthology.orchestra.run.smithi081.stdout: "weight": 1, 2024-09-21T21:41:09.952 INFO:teuthology.orchestra.run.smithi081.stdout: "primary_affinity": 0.3280487060546875, 2024-09-21T21:41:09.952 INFO:teuthology.orchestra.run.smithi081.stdout: "last_clean_begin": 0, 2024-09-21T21:41:09.952 INFO:teuthology.orchestra.run.smithi081.stdout: "last_clean_end": 0, 2024-09-21T21:41:09.952 INFO:teuthology.orchestra.run.smithi081.stdout: "up_from": 12, 2024-09-21T21:41:09.952 INFO:teuthology.orchestra.run.smithi081.stdout: "up_thru": 21, 2024-09-21T21:41:09.952 INFO:teuthology.orchestra.run.smithi081.stdout: "down_at": 0, 2024-09-21T21:41:09.952 INFO:teuthology.orchestra.run.smithi081.stdout: "lost_at": 0, 2024-09-21T21:41:09.952 INFO:teuthology.orchestra.run.smithi081.stdout: "public_addrs": { 2024-09-21T21:41:09.952 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:41:09.952 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:09.952 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:41:09.952 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6816", 2024-09-21T21:41:09.952 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1974398333 2024-09-21T21:41:09.952 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:09.952 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:09.952 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:41:09.953 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6817", 2024-09-21T21:41:09.953 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1974398333 2024-09-21T21:41:09.953 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:09.953 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:09.953 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:09.953 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster_addrs": { 2024-09-21T21:41:09.953 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:41:09.953 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:09.953 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:41:09.953 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6818", 2024-09-21T21:41:09.953 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1974398333 2024-09-21T21:41:09.953 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:09.953 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:09.953 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:41:09.953 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6819", 2024-09-21T21:41:09.954 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1974398333 2024-09-21T21:41:09.954 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:09.954 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:09.954 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:09.954 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_back_addrs": { 2024-09-21T21:41:09.954 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:41:09.954 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:09.954 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:41:09.954 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6822", 2024-09-21T21:41:09.954 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1974398333 2024-09-21T21:41:09.954 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:09.954 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:09.954 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:41:09.954 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6823", 2024-09-21T21:41:09.954 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1974398333 2024-09-21T21:41:09.955 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:09.955 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:09.955 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:09.955 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_front_addrs": { 2024-09-21T21:41:09.955 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:41:09.955 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:09.955 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:41:09.955 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6820", 2024-09-21T21:41:09.955 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1974398333 2024-09-21T21:41:09.955 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:09.955 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:09.955 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:41:09.955 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6821", 2024-09-21T21:41:09.955 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1974398333 2024-09-21T21:41:09.955 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:09.956 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:09.956 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:09.956 INFO:teuthology.orchestra.run.smithi081.stdout: "public_addr": "172.21.15.122:6817/1974398333", 2024-09-21T21:41:09.956 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster_addr": "172.21.15.122:6819/1974398333", 2024-09-21T21:41:09.956 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_back_addr": "172.21.15.122:6823/1974398333", 2024-09-21T21:41:09.956 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_front_addr": "172.21.15.122:6821/1974398333", 2024-09-21T21:41:09.956 INFO:teuthology.orchestra.run.smithi081.stdout: "state": [ 2024-09-21T21:41:09.956 INFO:teuthology.orchestra.run.smithi081.stdout: "exists", 2024-09-21T21:41:09.956 INFO:teuthology.orchestra.run.smithi081.stdout: "up" 2024-09-21T21:41:09.956 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:09.956 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:09.956 INFO:teuthology.orchestra.run.smithi081.stdout: ], 2024-09-21T21:41:09.956 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_xinfo": [ 2024-09-21T21:41:09.956 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:09.956 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": 0, 2024-09-21T21:41:09.957 INFO:teuthology.orchestra.run.smithi081.stdout: "down_stamp": "0.000000", 2024-09-21T21:41:09.957 INFO:teuthology.orchestra.run.smithi081.stdout: "laggy_probability": 0, 2024-09-21T21:41:09.957 INFO:teuthology.orchestra.run.smithi081.stdout: "laggy_interval": 0, 2024-09-21T21:41:09.957 INFO:teuthology.orchestra.run.smithi081.stdout: "features": 4540701547738038271, 2024-09-21T21:41:09.957 INFO:teuthology.orchestra.run.smithi081.stdout: "old_weight": 0, 2024-09-21T21:41:09.957 INFO:teuthology.orchestra.run.smithi081.stdout: "last_purged_snaps_scrub": "2024-09-21T21:40:15.766648+0000", 2024-09-21T21:41:09.957 INFO:teuthology.orchestra.run.smithi081.stdout: "dead_epoch": 0 2024-09-21T21:41:09.957 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:09.957 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:09.957 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": 1, 2024-09-21T21:41:09.957 INFO:teuthology.orchestra.run.smithi081.stdout: "down_stamp": "0.000000", 2024-09-21T21:41:09.957 INFO:teuthology.orchestra.run.smithi081.stdout: "laggy_probability": 0, 2024-09-21T21:41:09.957 INFO:teuthology.orchestra.run.smithi081.stdout: "laggy_interval": 0, 2024-09-21T21:41:09.957 INFO:teuthology.orchestra.run.smithi081.stdout: "features": 4540701547738038271, 2024-09-21T21:41:09.957 INFO:teuthology.orchestra.run.smithi081.stdout: "old_weight": 0, 2024-09-21T21:41:09.958 INFO:teuthology.orchestra.run.smithi081.stdout: "last_purged_snaps_scrub": "2024-09-21T21:40:15.457496+0000", 2024-09-21T21:41:09.958 INFO:teuthology.orchestra.run.smithi081.stdout: "dead_epoch": 0 2024-09-21T21:41:09.958 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:09.958 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:09.958 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": 2, 2024-09-21T21:41:09.958 INFO:teuthology.orchestra.run.smithi081.stdout: "down_stamp": "0.000000", 2024-09-21T21:41:09.958 INFO:teuthology.orchestra.run.smithi081.stdout: "laggy_probability": 0, 2024-09-21T21:41:09.958 INFO:teuthology.orchestra.run.smithi081.stdout: "laggy_interval": 0, 2024-09-21T21:41:09.958 INFO:teuthology.orchestra.run.smithi081.stdout: "features": 4540701547738038271, 2024-09-21T21:41:09.958 INFO:teuthology.orchestra.run.smithi081.stdout: "old_weight": 0, 2024-09-21T21:41:09.958 INFO:teuthology.orchestra.run.smithi081.stdout: "last_purged_snaps_scrub": "2024-09-21T21:40:14.767039+0000", 2024-09-21T21:41:09.958 INFO:teuthology.orchestra.run.smithi081.stdout: "dead_epoch": 0 2024-09-21T21:41:09.958 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:09.958 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:09.959 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": 3, 2024-09-21T21:41:09.959 INFO:teuthology.orchestra.run.smithi081.stdout: "down_stamp": "0.000000", 2024-09-21T21:41:09.959 INFO:teuthology.orchestra.run.smithi081.stdout: "laggy_probability": 0, 2024-09-21T21:41:09.959 INFO:teuthology.orchestra.run.smithi081.stdout: "laggy_interval": 0, 2024-09-21T21:41:09.959 INFO:teuthology.orchestra.run.smithi081.stdout: "features": 4540701547738038271, 2024-09-21T21:41:09.959 INFO:teuthology.orchestra.run.smithi081.stdout: "old_weight": 0, 2024-09-21T21:41:09.959 INFO:teuthology.orchestra.run.smithi081.stdout: "last_purged_snaps_scrub": "2024-09-21T21:40:15.355570+0000", 2024-09-21T21:41:09.959 INFO:teuthology.orchestra.run.smithi081.stdout: "dead_epoch": 0 2024-09-21T21:41:09.959 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:09.959 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:09.959 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": 4, 2024-09-21T21:41:09.959 INFO:teuthology.orchestra.run.smithi081.stdout: "down_stamp": "0.000000", 2024-09-21T21:41:09.959 INFO:teuthology.orchestra.run.smithi081.stdout: "laggy_probability": 0, 2024-09-21T21:41:09.959 INFO:teuthology.orchestra.run.smithi081.stdout: "laggy_interval": 0, 2024-09-21T21:41:09.960 INFO:teuthology.orchestra.run.smithi081.stdout: "features": 4540701547738038271, 2024-09-21T21:41:09.960 INFO:teuthology.orchestra.run.smithi081.stdout: "old_weight": 0, 2024-09-21T21:41:09.960 INFO:teuthology.orchestra.run.smithi081.stdout: "last_purged_snaps_scrub": "2024-09-21T21:40:15.335729+0000", 2024-09-21T21:41:09.960 INFO:teuthology.orchestra.run.smithi081.stdout: "dead_epoch": 0 2024-09-21T21:41:09.960 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:09.960 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:09.960 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": 5, 2024-09-21T21:41:09.960 INFO:teuthology.orchestra.run.smithi081.stdout: "down_stamp": "0.000000", 2024-09-21T21:41:09.960 INFO:teuthology.orchestra.run.smithi081.stdout: "laggy_probability": 0, 2024-09-21T21:41:09.960 INFO:teuthology.orchestra.run.smithi081.stdout: "laggy_interval": 0, 2024-09-21T21:41:09.960 INFO:teuthology.orchestra.run.smithi081.stdout: "features": 4540701547738038271, 2024-09-21T21:41:09.960 INFO:teuthology.orchestra.run.smithi081.stdout: "old_weight": 0, 2024-09-21T21:41:09.960 INFO:teuthology.orchestra.run.smithi081.stdout: "last_purged_snaps_scrub": "2024-09-21T21:40:15.614733+0000", 2024-09-21T21:41:09.960 INFO:teuthology.orchestra.run.smithi081.stdout: "dead_epoch": 0 2024-09-21T21:41:09.961 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:09.961 INFO:teuthology.orchestra.run.smithi081.stdout: ], 2024-09-21T21:41:09.961 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_upmap": [], 2024-09-21T21:41:09.961 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_upmap_items": [], 2024-09-21T21:41:09.961 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_upmap_primaries": [], 2024-09-21T21:41:09.961 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_temp": [], 2024-09-21T21:41:09.961 INFO:teuthology.orchestra.run.smithi081.stdout: "primary_temp": [], 2024-09-21T21:41:09.961 INFO:teuthology.orchestra.run.smithi081.stdout: "blocklist": {}, 2024-09-21T21:41:09.961 INFO:teuthology.orchestra.run.smithi081.stdout: "range_blocklist": {}, 2024-09-21T21:41:09.961 INFO:teuthology.orchestra.run.smithi081.stdout: "erasure_code_profiles": { 2024-09-21T21:41:09.961 INFO:teuthology.orchestra.run.smithi081.stdout: "default": { 2024-09-21T21:41:09.961 INFO:teuthology.orchestra.run.smithi081.stdout: "crush-failure-domain": "osd", 2024-09-21T21:41:09.961 INFO:teuthology.orchestra.run.smithi081.stdout: "k": "2", 2024-09-21T21:41:09.961 INFO:teuthology.orchestra.run.smithi081.stdout: "m": "1", 2024-09-21T21:41:09.962 INFO:teuthology.orchestra.run.smithi081.stdout: "plugin": "jerasure", 2024-09-21T21:41:09.962 INFO:teuthology.orchestra.run.smithi081.stdout: "technique": "reed_sol_van" 2024-09-21T21:41:09.962 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:09.962 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:09.962 INFO:teuthology.orchestra.run.smithi081.stdout: "removed_snaps_queue": [], 2024-09-21T21:41:09.962 INFO:teuthology.orchestra.run.smithi081.stdout: "new_removed_snaps": [], 2024-09-21T21:41:09.962 INFO:teuthology.orchestra.run.smithi081.stdout: "new_purged_snaps": [], 2024-09-21T21:41:09.962 INFO:teuthology.orchestra.run.smithi081.stdout: "crush_node_flags": {}, 2024-09-21T21:41:09.962 INFO:teuthology.orchestra.run.smithi081.stdout: "device_class_flags": {}, 2024-09-21T21:41:09.962 INFO:teuthology.orchestra.run.smithi081.stdout: "stretch_mode": { 2024-09-21T21:41:09.962 INFO:teuthology.orchestra.run.smithi081.stdout: "stretch_mode_enabled": false, 2024-09-21T21:41:09.962 INFO:teuthology.orchestra.run.smithi081.stdout: "stretch_bucket_count": 0, 2024-09-21T21:41:09.962 INFO:teuthology.orchestra.run.smithi081.stdout: "degraded_stretch_mode": 0, 2024-09-21T21:41:09.962 INFO:teuthology.orchestra.run.smithi081.stdout: "recovering_stretch_mode": 0, 2024-09-21T21:41:09.962 INFO:teuthology.orchestra.run.smithi081.stdout: "stretch_mode_bucket": 0 2024-09-21T21:41:09.963 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:09.963 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:09.964 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T21:41:09.964 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4: {} 2024-09-21T21:41:09.964 INFO:teuthology.orchestra.run.smithi081.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T21:41:09.966 INFO:tasks.thrashosds.thrasher:j is {'epoch': 44, 'fsid': 'a0fdcfc2-ddd2-4e50-81bb-876d940f3578', 'created': '2024-09-21T21:40:07.413917+0000', 'modified': '2024-09-21T21:41:09.564472+0000', 'last_up_change': '2024-09-21T21:40:18.340430+0000', 'last_in_change': '2024-09-21T21:40:09.029953+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T21:40:18.321328+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.888008177280426, 'average_primary_affinity': 0.8899999856948853, 'average_primary_affinity_weighted': 0.8899999856948853}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-21T21:40:24.235658+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 19, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.888008177280426, 'average_primary_affinity': 0.8899999856948853, 'average_primary_affinity_weighted': 0.8899999856948853}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-21T21:40:36.093691+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '43', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 14, 'snap_epoch': 43, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.888008177280426, 'average_primary_affinity': 0.8899999856948853, 'average_primary_affinity_weighted': 0.8899999856948853}}], 'osds': [{'osd': 0, 'uuid': 'a9869830-8e74-4b02-963d-ab630a0ee286', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 40, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6808', 'nonce': 1657552338}, {'type': 'v1', 'addr': '172.21.15.81:6809', 'nonce': 1657552338}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6810', 'nonce': 1657552338}, {'type': 'v1', 'addr': '172.21.15.81:6811', 'nonce': 1657552338}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6814', 'nonce': 1657552338}, {'type': 'v1', 'addr': '172.21.15.81:6815', 'nonce': 1657552338}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6812', 'nonce': 1657552338}, {'type': 'v1', 'addr': '172.21.15.81:6813', 'nonce': 1657552338}]}, 'public_addr': '172.21.15.81:6809/1657552338', 'cluster_addr': '172.21.15.81:6811/1657552338', 'heartbeat_back_addr': '172.21.15.81:6815/1657552338', 'heartbeat_front_addr': '172.21.15.81:6813/1657552338', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'b189819d-69a3-4e95-96e7-96dc4809ac80', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 40, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6816', 'nonce': 2864159502}, {'type': 'v1', 'addr': '172.21.15.81:6817', 'nonce': 2864159502}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6818', 'nonce': 2864159502}, {'type': 'v1', 'addr': '172.21.15.81:6819', 'nonce': 2864159502}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6822', 'nonce': 2864159502}, {'type': 'v1', 'addr': '172.21.15.81:6823', 'nonce': 2864159502}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6820', 'nonce': 2864159502}, {'type': 'v1', 'addr': '172.21.15.81:6821', 'nonce': 2864159502}]}, 'public_addr': '172.21.15.81:6817/2864159502', 'cluster_addr': '172.21.15.81:6819/2864159502', 'heartbeat_back_addr': '172.21.15.81:6823/2864159502', 'heartbeat_front_addr': '172.21.15.81:6821/2864159502', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '8437b99a-1297-4903-ab66-95c5cba2b2bf', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 40, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6800', 'nonce': 3023160932}, {'type': 'v1', 'addr': '172.21.15.81:6801', 'nonce': 3023160932}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6802', 'nonce': 3023160932}, {'type': 'v1', 'addr': '172.21.15.81:6803', 'nonce': 3023160932}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6806', 'nonce': 3023160932}, {'type': 'v1', 'addr': '172.21.15.81:6807', 'nonce': 3023160932}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6804', 'nonce': 3023160932}, {'type': 'v1', 'addr': '172.21.15.81:6805', 'nonce': 3023160932}]}, 'public_addr': '172.21.15.81:6801/3023160932', 'cluster_addr': '172.21.15.81:6803/3023160932', 'heartbeat_back_addr': '172.21.15.81:6807/3023160932', 'heartbeat_front_addr': '172.21.15.81:6805/3023160932', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'e8848768-c1aa-445b-9554-dc6aa4eefc8d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 21, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6808', 'nonce': 2681016844}, {'type': 'v1', 'addr': '172.21.15.122:6809', 'nonce': 2681016844}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6810', 'nonce': 2681016844}, {'type': 'v1', 'addr': '172.21.15.122:6811', 'nonce': 2681016844}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6814', 'nonce': 2681016844}, {'type': 'v1', 'addr': '172.21.15.122:6815', 'nonce': 2681016844}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6812', 'nonce': 2681016844}, {'type': 'v1', 'addr': '172.21.15.122:6813', 'nonce': 2681016844}]}, 'public_addr': '172.21.15.122:6809/2681016844', 'cluster_addr': '172.21.15.122:6811/2681016844', 'heartbeat_back_addr': '172.21.15.122:6815/2681016844', 'heartbeat_front_addr': '172.21.15.122:6813/2681016844', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'b1d8e85d-a207-4ba6-9322-61d07812083e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 40, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6800', 'nonce': 1095023540}, {'type': 'v1', 'addr': '172.21.15.122:6801', 'nonce': 1095023540}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6802', 'nonce': 1095023540}, {'type': 'v1', 'addr': '172.21.15.122:6803', 'nonce': 1095023540}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6806', 'nonce': 1095023540}, {'type': 'v1', 'addr': '172.21.15.122:6807', 'nonce': 1095023540}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6804', 'nonce': 1095023540}, {'type': 'v1', 'addr': '172.21.15.122:6805', 'nonce': 1095023540}]}, 'public_addr': '172.21.15.122:6801/1095023540', 'cluster_addr': '172.21.15.122:6803/1095023540', 'heartbeat_back_addr': '172.21.15.122:6807/1095023540', 'heartbeat_front_addr': '172.21.15.122:6805/1095023540', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '3ab0450b-d8fd-4ef2-a298-08d5aa2ea32f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.3280487060546875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 21, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6816', 'nonce': 1974398333}, {'type': 'v1', 'addr': '172.21.15.122:6817', 'nonce': 1974398333}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6818', 'nonce': 1974398333}, {'type': 'v1', 'addr': '172.21.15.122:6819', 'nonce': 1974398333}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6822', 'nonce': 1974398333}, {'type': 'v1', 'addr': '172.21.15.122:6823', 'nonce': 1974398333}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6820', 'nonce': 1974398333}, {'type': 'v1', 'addr': '172.21.15.122:6821', 'nonce': 1974398333}]}, 'public_addr': '172.21.15.122:6817/1974398333', 'cluster_addr': '172.21.15.122:6819/1974398333', 'heartbeat_back_addr': '172.21.15.122:6823/1974398333', 'heartbeat_front_addr': '172.21.15.122:6821/1974398333', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T21:40:15.766648+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T21:40:15.457496+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T21:40:14.767039+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T21:40:15.355570+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T21:40:15.335729+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T21:40:15.614733+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-21T21:41:09.966 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T21:41:09.966 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5: {} 2024-09-21T21:41:09.967 INFO:teuthology.orchestra.run.smithi081.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T21:41:10.008 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:10.006+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:10.052 INFO:teuthology.orchestra.run.smithi164.stdout:1971 write 0x4baac00 thru 0x4bb3537 (0x8938 bytes) 2024-09-21T21:41:10.058 INFO:teuthology.orchestra.run.smithi164.stdout:1972 trunc from 0x4bb3600 to 0x9dde000 2024-09-21T21:41:10.069 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:10.067+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:10.079 INFO:teuthology.orchestra.run.smithi164.stdout:1974 read 0x564ba00 thru 0x5655fa1 (0xa5a2 bytes) 2024-09-21T21:41:10.079 INFO:teuthology.orchestra.run.smithi164.stdout:1975 read 0x83c5000 thru 0x83d2b6d (0xdb6e bytes) 2024-09-21T21:41:10.079 INFO:teuthology.orchestra.run.smithi164.stdout:1976 read 0x726000 thru 0x72f00e (0x900f bytes) 2024-09-21T21:41:10.080 INFO:teuthology.orchestra.run.smithi164.stdout:1977 read 0x7db0200 thru 0x7dbd8f1 (0xd6f2 bytes) 2024-09-21T21:41:10.080 INFO:teuthology.orchestra.run.smithi164.stdout:1979 write 0x758a200 thru 0x7590140 (0x5f41 bytes) 2024-09-21T21:41:10.080 INFO:teuthology.orchestra.run.smithi164.stdout:1985 write 0x215c600 thru 0x2165d7d (0x977e bytes) 2024-09-21T21:41:10.088 INFO:teuthology.orchestra.run.smithi164.stdout:1986 trunc from 0x9dde000 to 0x2c5b400 2024-09-21T21:41:10.126 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:10.126 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 20, 2024-09-21T21:41:10.126 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 600, 2024-09-21T21:41:10.127 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:41:10.127 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.127 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:2885 3.b 3:d49b370f:::rbd_object_map.1121e540c096:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:10.127 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.478228+0000", 2024-09-21T21:41:10.127 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 11.646087781, 2024-09-21T21:41:10.127 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.015327835999999999, 2024-09-21T21:41:10.127 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:10.127 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:10.127 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:41:10.127 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:10.127 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:10.127 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:10.127 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2885 2024-09-21T21:41:10.127 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.128 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:10.128 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.128 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:10.128 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478228+0000", 2024-09-21T21:41:10.128 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:10.128 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.128 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.128 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:10.128 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478228+0000", 2024-09-21T21:41:10.128 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:10.128 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.128 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.128 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:10.128 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478229+0000", 2024-09-21T21:41:10.129 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8000000000000001e-07 2024-09-21T21:41:10.129 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.129 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.129 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:10.129 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478231+0000", 2024-09-21T21:41:10.129 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.5069999999999999e-06 2024-09-21T21:41:10.129 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.129 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.129 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:10.129 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478238+0000", 2024-09-21T21:41:10.129 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.8820000000000003e-06 2024-09-21T21:41:10.129 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.129 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.130 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:10.130 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478243+0000", 2024-09-21T21:41:10.130 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2209999999999996e-06 2024-09-21T21:41:10.130 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.130 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.130 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:10.130 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.488160+0000", 2024-09-21T21:41:10.130 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0099169849999999997 2024-09-21T21:41:10.130 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.130 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.130 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:10.130 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.488231+0000", 2024-09-21T21:41:10.130 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0909999999999997e-05 2024-09-21T21:41:10.130 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.130 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.131 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 0", 2024-09-21T21:41:10.131 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.488538+0000", 2024-09-21T21:41:10.131 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000306736 2024-09-21T21:41:10.131 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.131 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.131 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:41:10.131 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.493210+0000", 2024-09-21T21:41:10.131 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0046722120000000002 2024-09-21T21:41:10.131 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.131 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.131 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:41:10.131 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.493471+0000", 2024-09-21T21:41:10.131 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00026132899999999998 2024-09-21T21:41:10.131 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.132 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.132 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:41:10.132 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.493517+0000", 2024-09-21T21:41:10.132 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5309000000000003e-05 2024-09-21T21:41:10.132 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.132 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.132 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:10.132 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.493556+0000", 2024-09-21T21:41:10.132 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.9365e-05 2024-09-21T21:41:10.132 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:10.132 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:10.132 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:10.132 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.132 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.133 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:2886 3.b 3:d49b370f:::rbd_object_map.1121e540c096:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:10.133 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.478249+0000", 2024-09-21T21:41:10.133 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 11.646067001, 2024-09-21T21:41:10.133 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.015426120999999999, 2024-09-21T21:41:10.133 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:10.133 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:10.133 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:41:10.133 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:10.133 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:10.133 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:10.133 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2886 2024-09-21T21:41:10.133 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.133 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:10.133 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.133 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:10.134 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478249+0000", 2024-09-21T21:41:10.134 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:10.134 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.134 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.134 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:10.134 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478249+0000", 2024-09-21T21:41:10.134 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:10.134 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.134 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.134 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:10.134 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478249+0000", 2024-09-21T21:41:10.134 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.9999999999999998e-07 2024-09-21T21:41:10.134 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.134 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.135 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:10.135 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478252+0000", 2024-09-21T21:41:10.135 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.5500000000000001e-06 2024-09-21T21:41:10.135 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.135 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.135 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:10.135 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478258+0000", 2024-09-21T21:41:10.135 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7080000000000002e-06 2024-09-21T21:41:10.135 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.135 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.135 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:10.135 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478265+0000", 2024-09-21T21:41:10.135 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9110000000000004e-06 2024-09-21T21:41:10.135 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.135 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.136 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:10.136 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.488889+0000", 2024-09-21T21:41:10.136 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.010624662 2024-09-21T21:41:10.136 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.136 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.136 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:10.136 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.488965+0000", 2024-09-21T21:41:10.136 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5949000000000001e-05 2024-09-21T21:41:10.136 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.136 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.136 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 0", 2024-09-21T21:41:10.136 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.489235+0000", 2024-09-21T21:41:10.136 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00027019499999999998 2024-09-21T21:41:10.136 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.136 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.137 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:41:10.137 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.493233+0000", 2024-09-21T21:41:10.137 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0039978799999999997 2024-09-21T21:41:10.137 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.137 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.137 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:41:10.137 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.493601+0000", 2024-09-21T21:41:10.137 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00036728200000000003 2024-09-21T21:41:10.137 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.137 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.137 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:41:10.137 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.493637+0000", 2024-09-21T21:41:10.137 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6263000000000002e-05 2024-09-21T21:41:10.137 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.137 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.138 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:10.138 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.493675+0000", 2024-09-21T21:41:10.138 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8321000000000001e-05 2024-09-21T21:41:10.138 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:10.138 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:10.138 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:10.138 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.138 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.138 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:2887 3.b 3:d49b370f:::rbd_object_map.1121e540c096:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:10.138 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.478271+0000", 2024-09-21T21:41:10.138 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 11.646045489, 2024-09-21T21:41:10.138 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.015551966, 2024-09-21T21:41:10.138 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:10.138 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:10.138 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:41:10.139 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:10.139 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:10.139 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:10.139 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2887 2024-09-21T21:41:10.139 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.139 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:10.139 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.139 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:10.139 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478271+0000", 2024-09-21T21:41:10.139 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:10.139 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.139 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.139 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:10.139 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478271+0000", 2024-09-21T21:41:10.140 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:10.140 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.140 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.140 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:10.140 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478271+0000", 2024-09-21T21:41:10.140 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.0600000000000001e-07 2024-09-21T21:41:10.140 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.140 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.140 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:10.140 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478274+0000", 2024-09-21T21:41:10.140 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.6390000000000001e-06 2024-09-21T21:41:10.140 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.140 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.140 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:10.140 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478280+0000", 2024-09-21T21:41:10.141 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3219999999999998e-06 2024-09-21T21:41:10.141 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.141 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.141 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:10.141 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478285+0000", 2024-09-21T21:41:10.141 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6589999999999998e-06 2024-09-21T21:41:10.141 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.141 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.141 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:10.141 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.489524+0000", 2024-09-21T21:41:10.141 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.01123979 2024-09-21T21:41:10.141 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.141 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.141 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:10.141 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.489567+0000", 2024-09-21T21:41:10.142 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2673000000000003e-05 2024-09-21T21:41:10.142 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.142 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.142 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 0", 2024-09-21T21:41:10.142 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.489834+0000", 2024-09-21T21:41:10.142 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00026672799999999998 2024-09-21T21:41:10.142 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.142 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.142 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:41:10.142 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.493290+0000", 2024-09-21T21:41:10.142 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0034557429999999998 2024-09-21T21:41:10.142 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.142 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.142 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:41:10.142 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.493737+0000", 2024-09-21T21:41:10.142 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00044731799999999999 2024-09-21T21:41:10.143 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.143 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.143 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:41:10.143 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.493784+0000", 2024-09-21T21:41:10.143 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7178999999999997e-05 2024-09-21T21:41:10.143 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.143 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.143 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:10.143 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.493823+0000", 2024-09-21T21:41:10.143 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8509e-05 2024-09-21T21:41:10.143 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:10.143 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:10.143 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:10.143 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.143 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.144 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4325 3.0 3:074cff57:::rbd_data.112145fae779.000000000000000c:a [read 2064384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:10.144 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.510141+0000", 2024-09-21T21:41:10.144 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 11.614175327, 2024-09-21T21:41:10.144 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013330887, 2024-09-21T21:41:10.144 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:10.144 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:10.144 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:10.144 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:10.144 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:10.144 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:10.144 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4325 2024-09-21T21:41:10.144 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.144 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:10.144 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.144 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:10.144 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510141+0000", 2024-09-21T21:41:10.144 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:10.145 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.145 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.145 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:10.145 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510141+0000", 2024-09-21T21:41:10.145 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:10.145 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.145 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.145 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:10.145 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510141+0000", 2024-09-21T21:41:10.145 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.72e-07 2024-09-21T21:41:10.145 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.145 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.145 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:10.145 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510143+0000", 2024-09-21T21:41:10.145 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2409999999999998e-06 2024-09-21T21:41:10.145 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.146 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.146 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:10.146 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510151+0000", 2024-09-21T21:41:10.146 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.3390000000000004e-06 2024-09-21T21:41:10.146 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.146 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.146 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:10.146 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510155+0000", 2024-09-21T21:41:10.146 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5630000000000002e-06 2024-09-21T21:41:10.146 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.146 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.146 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:10.146 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.523310+0000", 2024-09-21T21:41:10.146 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013154658 2024-09-21T21:41:10.147 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.147 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.147 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:10.147 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.523386+0000", 2024-09-21T21:41:10.147 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.6553999999999999e-05 2024-09-21T21:41:10.147 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.147 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.147 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:10.147 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.523472+0000", 2024-09-21T21:41:10.147 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.5160000000000005e-05 2024-09-21T21:41:10.147 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:10.147 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:10.147 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:10.147 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.147 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.148 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4326 3.0 3:074cff57:::rbd_data.112145fae779.000000000000000c:a [read 2097152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:10.148 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.510162+0000", 2024-09-21T21:41:10.148 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 11.614154312, 2024-09-21T21:41:10.148 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013537466, 2024-09-21T21:41:10.148 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:10.148 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:10.148 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:10.148 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:10.148 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:10.148 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:10.148 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4326 2024-09-21T21:41:10.148 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.148 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:10.148 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.148 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:10.148 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510162+0000", 2024-09-21T21:41:10.149 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:10.149 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.149 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.149 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:10.149 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510162+0000", 2024-09-21T21:41:10.149 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:10.149 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.149 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.149 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:10.149 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510162+0000", 2024-09-21T21:41:10.149 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.0699999999999998e-07 2024-09-21T21:41:10.149 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.149 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.149 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:10.150 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510164+0000", 2024-09-21T21:41:10.150 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.17e-06 2024-09-21T21:41:10.150 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.150 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.150 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:10.150 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510174+0000", 2024-09-21T21:41:10.150 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2439999999999999e-06 2024-09-21T21:41:10.150 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.150 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.150 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:10.150 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510179+0000", 2024-09-21T21:41:10.150 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2159999999999997e-06 2024-09-21T21:41:10.150 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.150 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.151 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:10.151 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.523502+0000", 2024-09-21T21:41:10.151 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013322779 2024-09-21T21:41:10.151 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.151 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.151 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:10.151 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.523579+0000", 2024-09-21T21:41:10.151 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7402000000000002e-05 2024-09-21T21:41:10.151 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.151 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.151 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:10.151 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.523699+0000", 2024-09-21T21:41:10.151 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000120248 2024-09-21T21:41:10.151 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:10.151 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:10.151 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:10.152 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.152 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.152 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4327 3.0 3:074cff57:::rbd_data.112145fae779.000000000000000c:a [read 2154496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:10.152 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.510418+0000", 2024-09-21T21:41:10.152 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 11.613897876999999, 2024-09-21T21:41:10.152 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013505039999999999, 2024-09-21T21:41:10.152 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:10.152 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:10.152 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:10.152 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:10.152 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:10.152 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:10.152 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4327 2024-09-21T21:41:10.152 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.153 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:10.153 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.153 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:10.153 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510418+0000", 2024-09-21T21:41:10.153 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:10.153 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.153 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.153 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:10.153 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510418+0000", 2024-09-21T21:41:10.153 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:10.153 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.153 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.153 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:10.153 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510419+0000", 2024-09-21T21:41:10.153 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8999999999999997e-07 2024-09-21T21:41:10.153 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.154 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.154 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:10.154 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510421+0000", 2024-09-21T21:41:10.154 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.4779999999999998e-06 2024-09-21T21:41:10.154 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.154 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.154 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:10.154 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510427+0000", 2024-09-21T21:41:10.154 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5609999999999998e-06 2024-09-21T21:41:10.154 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.154 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.154 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:10.154 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510432+0000", 2024-09-21T21:41:10.154 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5389999999999998e-06 2024-09-21T21:41:10.155 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.155 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.155 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:10.155 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.523743+0000", 2024-09-21T21:41:10.155 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013310963 2024-09-21T21:41:10.155 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.155 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.155 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:10.155 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.523824+0000", 2024-09-21T21:41:10.155 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0463000000000005e-05 2024-09-21T21:41:10.155 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.155 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.155 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:10.155 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.523923+0000", 2024-09-21T21:41:10.155 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.9545999999999999e-05 2024-09-21T21:41:10.156 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:10.156 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:10.156 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:10.156 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.156 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.156 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4328 3.0 3:074cff57:::rbd_data.112145fae779.000000000000000c:a [read 2211840~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:10.156 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.510443+0000", 2024-09-21T21:41:10.156 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 11.613873577, 2024-09-21T21:41:10.156 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013684564, 2024-09-21T21:41:10.156 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:10.156 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:10.156 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:10.156 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:10.156 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:10.157 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:10.157 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4328 2024-09-21T21:41:10.157 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.157 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:10.157 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.157 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:10.157 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510443+0000", 2024-09-21T21:41:10.157 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:10.157 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.157 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.157 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:10.157 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510443+0000", 2024-09-21T21:41:10.157 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:10.157 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.157 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.157 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:10.158 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510443+0000", 2024-09-21T21:41:10.158 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2199999999999999e-07 2024-09-21T21:41:10.158 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.158 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.158 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:10.158 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510445+0000", 2024-09-21T21:41:10.158 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1679999999999998e-06 2024-09-21T21:41:10.158 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.158 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.158 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:10.158 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510452+0000", 2024-09-21T21:41:10.158 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6259999999999997e-06 2024-09-21T21:41:10.158 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.158 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.159 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:10.159 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510457+0000", 2024-09-21T21:41:10.159 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8110000000000002e-06 2024-09-21T21:41:10.159 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.159 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.159 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:10.159 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.523968+0000", 2024-09-21T21:41:10.159 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013510996000000001 2024-09-21T21:41:10.159 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.159 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.159 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:10.159 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.524044+0000", 2024-09-21T21:41:10.159 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.6259000000000003e-05 2024-09-21T21:41:10.159 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.159 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.160 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:10.160 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.524127+0000", 2024-09-21T21:41:10.160 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.3282000000000007e-05 2024-09-21T21:41:10.160 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:10.160 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:10.160 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:10.160 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.160 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.160 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4577 3.0 3:074cff57:::rbd_data.112145fae779.000000000000000c:a [read 1744896~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:10.160 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.514327+0000", 2024-09-21T21:41:10.160 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 11.609988919999999, 2024-09-21T21:41:10.160 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013451424999999999, 2024-09-21T21:41:10.160 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:10.160 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:10.161 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:10.161 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:10.161 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:10.161 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:10.161 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4577 2024-09-21T21:41:10.161 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.161 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:10.161 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.161 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:10.161 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514327+0000", 2024-09-21T21:41:10.161 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:10.161 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.161 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.161 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:10.161 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514327+0000", 2024-09-21T21:41:10.161 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:10.162 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.162 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.162 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:10.162 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514328+0000", 2024-09-21T21:41:10.162 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4900000000000001e-07 2024-09-21T21:41:10.162 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.162 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.162 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:10.162 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514330+0000", 2024-09-21T21:41:10.162 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.227e-06 2024-09-21T21:41:10.162 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.162 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.162 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:10.162 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514336+0000", 2024-09-21T21:41:10.163 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1940000000000003e-06 2024-09-21T21:41:10.163 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.163 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.163 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:10.163 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514341+0000", 2024-09-21T21:41:10.163 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7319999999999998e-06 2024-09-21T21:41:10.163 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.163 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.163 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:10.163 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.527415+0000", 2024-09-21T21:41:10.163 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013074657 2024-09-21T21:41:10.163 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.163 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.163 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:10.163 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.527491+0000", 2024-09-21T21:41:10.164 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5899e-05 2024-09-21T21:41:10.164 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.164 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.164 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:10.164 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.527779+0000", 2024-09-21T21:41:10.164 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00028736700000000002 2024-09-21T21:41:10.164 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:10.164 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:10.164 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:10.164 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.164 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.164 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4578 3.0 3:074cff57:::rbd_data.112145fae779.000000000000000c:a [read 1802240~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:10.164 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.514347+0000", 2024-09-21T21:41:10.164 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 11.609969465000001, 2024-09-21T21:41:10.164 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.014025592999999999, 2024-09-21T21:41:10.164 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:10.165 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:10.165 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:10.165 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:10.165 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:10.165 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:10.165 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4578 2024-09-21T21:41:10.165 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.165 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:10.165 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.165 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:10.165 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514347+0000", 2024-09-21T21:41:10.165 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:10.165 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.165 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.165 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:10.166 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514347+0000", 2024-09-21T21:41:10.166 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:10.166 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.166 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.166 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:10.166 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514347+0000", 2024-09-21T21:41:10.166 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6699999999999999e-07 2024-09-21T21:41:10.166 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.166 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.166 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:10.166 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514349+0000", 2024-09-21T21:41:10.166 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0109999999999999e-06 2024-09-21T21:41:10.166 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.166 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.167 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:10.167 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514355+0000", 2024-09-21T21:41:10.167 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1090000000000003e-06 2024-09-21T21:41:10.167 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.167 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.167 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:10.167 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514360+0000", 2024-09-21T21:41:10.167 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5900000000000001e-06 2024-09-21T21:41:10.167 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.167 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.167 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:10.167 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.527862+0000", 2024-09-21T21:41:10.167 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013501812 2024-09-21T21:41:10.167 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.167 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.168 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:10.168 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.527933+0000", 2024-09-21T21:41:10.168 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.1322000000000006e-05 2024-09-21T21:41:10.168 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.168 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.168 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:10.168 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.528372+0000", 2024-09-21T21:41:10.168 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00043938199999999999 2024-09-21T21:41:10.168 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:10.168 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:10.168 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:10.168 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.168 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.168 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4579 3.0 3:074cff57:::rbd_data.112145fae779.000000000000000c:a [read 1835008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:10.169 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.514365+0000", 2024-09-21T21:41:10.169 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 11.609950608, 2024-09-21T21:41:10.169 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.014158828, 2024-09-21T21:41:10.169 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:10.169 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:10.169 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:10.169 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:10.169 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:10.169 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:10.169 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4579 2024-09-21T21:41:10.169 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.169 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:10.169 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.169 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:10.169 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514365+0000", 2024-09-21T21:41:10.170 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:10.170 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.170 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.170 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:10.170 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514365+0000", 2024-09-21T21:41:10.170 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:10.170 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.170 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.170 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:10.170 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514366+0000", 2024-09-21T21:41:10.170 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7800000000000002e-07 2024-09-21T21:41:10.170 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.170 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.170 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:10.170 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514368+0000", 2024-09-21T21:41:10.171 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9920000000000002e-06 2024-09-21T21:41:10.171 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.171 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.171 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:10.171 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514374+0000", 2024-09-21T21:41:10.171 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9370000000000001e-06 2024-09-21T21:41:10.171 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.171 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.171 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:10.171 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514379+0000", 2024-09-21T21:41:10.171 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7149999999999997e-06 2024-09-21T21:41:10.171 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.171 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.171 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:10.171 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.528303+0000", 2024-09-21T21:41:10.172 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013924076000000001 2024-09-21T21:41:10.172 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.172 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.172 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:10.172 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.528429+0000", 2024-09-21T21:41:10.172 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00012643100000000001 2024-09-21T21:41:10.172 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.172 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.172 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:10.172 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.528524+0000", 2024-09-21T21:41:10.172 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.5298999999999997e-05 2024-09-21T21:41:10.172 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:10.172 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:10.172 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:10.172 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.173 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.173 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4580 3.0 3:074cff57:::rbd_data.112145fae779.000000000000000c:a [read 1892352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:10.173 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.514384+0000", 2024-09-21T21:41:10.173 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 11.609932012, 2024-09-21T21:41:10.173 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.014365243, 2024-09-21T21:41:10.173 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:10.173 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:10.173 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:10.173 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:10.173 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:10.173 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:10.173 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4580 2024-09-21T21:41:10.173 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.173 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:10.174 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.174 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:10.174 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514384+0000", 2024-09-21T21:41:10.174 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:10.174 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.174 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.174 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:10.174 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514384+0000", 2024-09-21T21:41:10.174 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:10.174 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.174 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.174 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:10.174 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514384+0000", 2024-09-21T21:41:10.174 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1100000000000001e-07 2024-09-21T21:41:10.174 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.175 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.175 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:10.175 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514387+0000", 2024-09-21T21:41:10.175 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1150000000000001e-06 2024-09-21T21:41:10.175 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.175 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.175 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:10.175 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514393+0000", 2024-09-21T21:41:10.175 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1140000000000002e-06 2024-09-21T21:41:10.175 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.175 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.175 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:10.175 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514398+0000", 2024-09-21T21:41:10.176 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6330000000000001e-06 2024-09-21T21:41:10.176 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.176 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.176 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:10.176 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.528569+0000", 2024-09-21T21:41:10.176 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.014171011000000001 2024-09-21T21:41:10.176 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.176 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.176 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:10.176 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.528642+0000", 2024-09-21T21:41:10.176 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.2336000000000007e-05 2024-09-21T21:41:10.176 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.176 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.176 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:10.176 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.528749+0000", 2024-09-21T21:41:10.177 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000107623 2024-09-21T21:41:10.177 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:10.177 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:10.177 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:10.177 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.177 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.177 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4581 3.0 3:074cff57:::rbd_data.112145fae779.000000000000000c:a [read 1949696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:10.177 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.514404+0000", 2024-09-21T21:41:10.177 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 11.609911844999999, 2024-09-21T21:41:10.177 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.014551563999999999, 2024-09-21T21:41:10.177 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:10.177 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:10.177 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:10.177 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:10.177 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:10.177 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:10.178 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4581 2024-09-21T21:41:10.178 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.178 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:10.178 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.178 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:10.178 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514404+0000", 2024-09-21T21:41:10.178 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:10.178 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.178 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.178 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:10.178 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514404+0000", 2024-09-21T21:41:10.178 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:10.178 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.178 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.178 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:10.179 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514405+0000", 2024-09-21T21:41:10.179 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7599999999999998e-07 2024-09-21T21:41:10.179 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.179 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.179 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:10.179 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514407+0000", 2024-09-21T21:41:10.179 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0169999999999999e-06 2024-09-21T21:41:10.179 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.179 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.179 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:10.179 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514413+0000", 2024-09-21T21:41:10.179 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0020000000000003e-06 2024-09-21T21:41:10.179 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.179 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.179 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:10.179 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514417+0000", 2024-09-21T21:41:10.180 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7319999999999998e-06 2024-09-21T21:41:10.180 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.180 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.180 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:10.180 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.528781+0000", 2024-09-21T21:41:10.180 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.014363433 2024-09-21T21:41:10.180 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.180 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.180 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:10.180 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.528861+0000", 2024-09-21T21:41:10.180 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0408000000000007e-05 2024-09-21T21:41:10.180 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.180 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.180 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:10.180 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.528956+0000", 2024-09-21T21:41:10.181 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.4596e-05 2024-09-21T21:41:10.181 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:10.181 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:10.181 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:10.181 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.181 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.181 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4582 3.0 3:074cff57:::rbd_data.112145fae779.000000000000000c:a [read 1982464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:10.181 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.514423+0000", 2024-09-21T21:41:10.181 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 11.609892961, 2024-09-21T21:41:10.181 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.014719965, 2024-09-21T21:41:10.181 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:10.181 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:10.181 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:10.181 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:10.182 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:10.182 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:10.182 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4582 2024-09-21T21:41:10.182 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.182 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:10.182 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.182 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:10.182 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514423+0000", 2024-09-21T21:41:10.182 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:10.182 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.182 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.182 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:10.182 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514423+0000", 2024-09-21T21:41:10.182 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:10.183 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.183 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.183 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:10.183 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514423+0000", 2024-09-21T21:41:10.183 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.53e-07 2024-09-21T21:41:10.183 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.183 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.183 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:10.183 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514426+0000", 2024-09-21T21:41:10.183 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2000000000000001e-06 2024-09-21T21:41:10.183 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.183 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.183 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:10.183 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514433+0000", 2024-09-21T21:41:10.184 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.3170000000000004e-06 2024-09-21T21:41:10.184 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.184 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.184 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:10.184 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514438+0000", 2024-09-21T21:41:10.184 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3730000000000001e-06 2024-09-21T21:41:10.184 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.184 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.184 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:10.184 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.528984+0000", 2024-09-21T21:41:10.184 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.014545773 2024-09-21T21:41:10.184 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.184 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.184 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:10.184 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.529056+0000", 2024-09-21T21:41:10.185 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.2067e-05 2024-09-21T21:41:10.185 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.185 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.185 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:10.185 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.529143+0000", 2024-09-21T21:41:10.185 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.6881999999999999e-05 2024-09-21T21:41:10.185 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:10.185 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:10.185 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:10.185 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.185 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.185 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4583 3.0 3:074cff57:::rbd_data.112145fae779.000000000000000c:a [read 2039808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:10.185 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.514446+0000", 2024-09-21T21:41:10.185 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 11.609870189, 2024-09-21T21:41:10.185 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.016072466000000001, 2024-09-21T21:41:10.185 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:10.186 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:10.186 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:10.186 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:10.186 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:10.186 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:10.186 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4583 2024-09-21T21:41:10.186 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.186 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:10.186 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.186 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:10.186 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514446+0000", 2024-09-21T21:41:10.186 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:10.186 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.187 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.187 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:10.187 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514446+0000", 2024-09-21T21:41:10.187 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:10.187 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.187 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.187 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:10.187 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514446+0000", 2024-09-21T21:41:10.187 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.9700000000000002e-07 2024-09-21T21:41:10.187 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.187 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.187 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:10.187 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514448+0000", 2024-09-21T21:41:10.188 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1849999999999998e-06 2024-09-21T21:41:10.188 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.188 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.188 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:10.188 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514454+0000", 2024-09-21T21:41:10.188 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.015e-06 2024-09-21T21:41:10.188 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.188 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.188 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:10.188 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514459+0000", 2024-09-21T21:41:10.188 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7639999999999996e-06 2024-09-21T21:41:10.188 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.188 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.188 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:10.188 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.529173+0000", 2024-09-21T21:41:10.189 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.014713688000000001 2024-09-21T21:41:10.189 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.189 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.189 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:10.189 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.529248+0000", 2024-09-21T21:41:10.189 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5073000000000004e-05 2024-09-21T21:41:10.189 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.189 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.189 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:10.189 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.530518+0000", 2024-09-21T21:41:10.189 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0012703440000000001 2024-09-21T21:41:10.189 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:10.189 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:10.189 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:10.190 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.190 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.190 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4584 3.0 3:074cff57:::rbd_data.112145fae779.000000000000000c:a [read 2129920~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:10.190 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.514465+0000", 2024-09-21T21:41:10.190 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 11.60985075, 2024-09-21T21:41:10.190 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.016252026999999999, 2024-09-21T21:41:10.190 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:10.190 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:10.190 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:10.190 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:10.190 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:10.190 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:10.190 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4584 2024-09-21T21:41:10.190 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.190 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:10.191 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.191 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:10.191 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514465+0000", 2024-09-21T21:41:10.191 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:10.191 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.191 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.191 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:10.191 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514465+0000", 2024-09-21T21:41:10.191 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:10.191 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.191 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.191 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:10.191 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514466+0000", 2024-09-21T21:41:10.191 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.0400000000000002e-07 2024-09-21T21:41:10.192 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.192 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.192 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:10.192 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514468+0000", 2024-09-21T21:41:10.192 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3159999999999999e-06 2024-09-21T21:41:10.192 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.192 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.192 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:10.192 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514474+0000", 2024-09-21T21:41:10.192 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.224e-06 2024-09-21T21:41:10.192 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.192 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.192 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:10.192 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514479+0000", 2024-09-21T21:41:10.193 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8099999999999997e-06 2024-09-21T21:41:10.193 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.193 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.193 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:10.193 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.530552+0000", 2024-09-21T21:41:10.193 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.016072622000000002 2024-09-21T21:41:10.193 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.193 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.193 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:10.193 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.530630+0000", 2024-09-21T21:41:10.193 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.8448000000000005e-05 2024-09-21T21:41:10.193 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.193 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.193 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:10.193 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.530717+0000", 2024-09-21T21:41:10.194 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.7202999999999998e-05 2024-09-21T21:41:10.194 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:10.194 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:10.194 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:10.194 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.194 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.194 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4585 3.0 3:074cff57:::rbd_data.112145fae779.000000000000000c:a [read 2187264~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:10.194 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.514485+0000", 2024-09-21T21:41:10.194 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 11.609831083, 2024-09-21T21:41:10.194 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.016420605000000001, 2024-09-21T21:41:10.194 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:10.194 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:10.194 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:10.194 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:10.195 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:10.195 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:10.195 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4585 2024-09-21T21:41:10.195 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.195 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:10.195 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.195 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:10.195 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514485+0000", 2024-09-21T21:41:10.195 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:10.195 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.195 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.195 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:10.195 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514485+0000", 2024-09-21T21:41:10.195 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:10.196 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.196 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.196 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:10.196 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514485+0000", 2024-09-21T21:41:10.196 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3599999999999999e-07 2024-09-21T21:41:10.196 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.196 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.196 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:10.196 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514487+0000", 2024-09-21T21:41:10.196 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0870000000000001e-06 2024-09-21T21:41:10.196 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.196 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.196 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:10.196 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514495+0000", 2024-09-21T21:41:10.197 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7589999999999997e-06 2024-09-21T21:41:10.197 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.197 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.197 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:10.197 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514501+0000", 2024-09-21T21:41:10.197 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4e-06 2024-09-21T21:41:10.197 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.197 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.197 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:10.197 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.530747+0000", 2024-09-21T21:41:10.197 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.016246564000000002 2024-09-21T21:41:10.197 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.197 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.197 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:10.198 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.530824+0000", 2024-09-21T21:41:10.198 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7109999999999999e-05 2024-09-21T21:41:10.198 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.198 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.198 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:10.198 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.530906+0000", 2024-09-21T21:41:10.198 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.1348999999999997e-05 2024-09-21T21:41:10.198 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:10.198 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:10.198 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:10.198 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.198 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.198 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4711 3.0 3:074cff57:::rbd_data.112145fae779.000000000000000c:a [read 2043904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:10.198 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.519604+0000", 2024-09-21T21:41:10.199 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 11.604711593999999, 2024-09-21T21:41:10.199 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013270252999999999, 2024-09-21T21:41:10.199 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:10.199 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:10.199 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:10.199 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:10.199 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:10.199 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:10.199 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4711 2024-09-21T21:41:10.199 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.199 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:10.199 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.199 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:10.199 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519604+0000", 2024-09-21T21:41:10.200 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:10.200 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.200 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.200 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:10.200 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519604+0000", 2024-09-21T21:41:10.200 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:10.200 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.200 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.200 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:10.200 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519605+0000", 2024-09-21T21:41:10.200 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8500000000000002e-07 2024-09-21T21:41:10.200 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.200 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.200 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:10.200 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519607+0000", 2024-09-21T21:41:10.201 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1789999999999998e-06 2024-09-21T21:41:10.201 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.201 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.201 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:10.201 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519613+0000", 2024-09-21T21:41:10.201 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2609999999999999e-06 2024-09-21T21:41:10.201 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.201 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.201 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:10.201 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519618+0000", 2024-09-21T21:41:10.201 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9579999999999998e-06 2024-09-21T21:41:10.201 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.201 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.201 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:10.201 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.532678+0000", 2024-09-21T21:41:10.202 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013059822 2024-09-21T21:41:10.202 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.202 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.202 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:10.202 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.532756+0000", 2024-09-21T21:41:10.202 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7467999999999998e-05 2024-09-21T21:41:10.202 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.202 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.202 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:10.202 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.532875+0000", 2024-09-21T21:41:10.202 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011917999999999999 2024-09-21T21:41:10.202 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:10.202 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:10.202 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:10.202 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.203 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.203 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4712 3.0 3:074cff57:::rbd_data.112145fae779.000000000000000c:a [read 2134016~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:10.203 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.519632+0000", 2024-09-21T21:41:10.203 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 11.604684110999999, 2024-09-21T21:41:10.203 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013480002, 2024-09-21T21:41:10.203 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:10.203 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:10.203 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:10.203 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:10.203 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:10.203 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:10.203 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4712 2024-09-21T21:41:10.203 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.203 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:10.203 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.204 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:10.204 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519632+0000", 2024-09-21T21:41:10.204 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:10.204 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.204 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.204 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:10.204 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519632+0000", 2024-09-21T21:41:10.204 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:10.204 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.204 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.204 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:10.204 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519632+0000", 2024-09-21T21:41:10.204 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2899999999999999e-07 2024-09-21T21:41:10.204 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.205 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.205 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:10.205 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519635+0000", 2024-09-21T21:41:10.205 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.6910000000000002e-06 2024-09-21T21:41:10.205 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.205 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.205 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:10.205 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519761+0000", 2024-09-21T21:41:10.205 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000125988 2024-09-21T21:41:10.205 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.205 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.205 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:10.205 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519767+0000", 2024-09-21T21:41:10.205 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9190000000000004e-06 2024-09-21T21:41:10.205 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.205 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.205 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:10.206 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.532905+0000", 2024-09-21T21:41:10.206 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013137970000000001 2024-09-21T21:41:10.206 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.206 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.206 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:10.206 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.532984+0000", 2024-09-21T21:41:10.206 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.8616999999999996e-05 2024-09-21T21:41:10.206 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.206 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.206 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:10.206 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.533112+0000", 2024-09-21T21:41:10.206 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000128388 2024-09-21T21:41:10.206 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:10.206 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:10.206 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:10.207 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.207 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.207 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4713 3.0 3:074cff57:::rbd_data.112145fae779.000000000000000c:a [read 2191360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:10.207 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.519777+0000", 2024-09-21T21:41:10.207 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 11.604539410999999, 2024-09-21T21:41:10.207 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.01355251, 2024-09-21T21:41:10.207 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:10.207 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:10.207 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:10.207 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:10.207 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:10.207 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:10.207 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4713 2024-09-21T21:41:10.207 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.208 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:10.208 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.208 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:10.208 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519777+0000", 2024-09-21T21:41:10.208 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:10.208 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.208 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.208 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:10.208 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519777+0000", 2024-09-21T21:41:10.208 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:10.208 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.208 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.208 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:10.208 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519777+0000", 2024-09-21T21:41:10.208 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9900000000000001e-07 2024-09-21T21:41:10.208 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.209 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.209 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:10.209 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519780+0000", 2024-09-21T21:41:10.209 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.886e-06 2024-09-21T21:41:10.209 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.209 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.209 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:10.209 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519786+0000", 2024-09-21T21:41:10.209 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0859999999999998e-06 2024-09-21T21:41:10.209 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.209 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.209 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:10.209 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519790+0000", 2024-09-21T21:41:10.209 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2400000000000001e-06 2024-09-21T21:41:10.210 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.210 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.210 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:10.210 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.533143+0000", 2024-09-21T21:41:10.210 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013352711 2024-09-21T21:41:10.210 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.210 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.210 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:10.210 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.533224+0000", 2024-09-21T21:41:10.210 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0675999999999998e-05 2024-09-21T21:41:10.210 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.210 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.210 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:10.210 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.533329+0000", 2024-09-21T21:41:10.211 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000105412 2024-09-21T21:41:10.211 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:10.211 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:10.211 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:10.211 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.211 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.211 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4824.0:1564 3.2 3:4647520c:::rbd_data.1121b97f2925.0000000000000006:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:10.211 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:41:10.211 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 8.488251537, 2024-09-21T21:41:10.211 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056097347999999998, 2024-09-21T21:41:10.211 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:10.211 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:10.211 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:10.211 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:10.211 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4824", 2024-09-21T21:41:10.212 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:10.212 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1564 2024-09-21T21:41:10.212 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.212 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:10.212 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.212 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:10.212 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:41:10.212 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:10.212 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.212 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.212 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:10.212 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:41:10.212 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:10.212 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.213 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.213 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:10.213 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:41:10.213 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6499999999999999e-07 2024-09-21T21:41:10.213 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.213 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.213 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:10.213 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636067+0000", 2024-09-21T21:41:10.213 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3879999999999998e-06 2024-09-21T21:41:10.213 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.213 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.213 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:10.213 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636075+0000", 2024-09-21T21:41:10.213 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5480000000000004e-06 2024-09-21T21:41:10.214 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.214 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.214 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:10.214 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636080+0000", 2024-09-21T21:41:10.214 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.339e-06 2024-09-21T21:41:10.214 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.214 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.214 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:10.214 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636133+0000", 2024-09-21T21:41:10.214 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2930000000000003e-05 2024-09-21T21:41:10.214 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.214 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.214 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:10.214 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636265+0000", 2024-09-21T21:41:10.214 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000132058 2024-09-21T21:41:10.215 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.215 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.215 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:10.215 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.692162+0000", 2024-09-21T21:41:10.215 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055896620000000001 2024-09-21T21:41:10.215 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:10.215 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:10.215 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:10.215 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:10.215 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:10.215 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:10.217 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:10.215+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:10.218 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T21:41:10.218 INFO:teuthology.orchestra.run.smithi164.stdout:1988 punch from 0x30400 to 0x37800, (0x7400 bytes) 2024-09-21T21:41:10.218 INFO:teuthology.orchestra.run.smithi164.stdout:1990 write 0xe88000 thru 0xe8b9e4 (0x39e5 bytes) 2024-09-21T21:41:10.218 INFO:teuthology.orchestra.run.smithi164.stdout:1991 write 0xc2e6e00 thru 0xc2edc77 (0x6e78 bytes) 2024-09-21T21:41:10.218 INFO:teuthology.orchestra.run.smithi164.stdout:1992 read 0x1203a00 thru 0x120b8b7 (0x7eb8 bytes) 2024-09-21T21:41:10.219 INFO:teuthology.orchestra.run.smithi164.stdout:1993 read 0x5d2cc00 thru 0x5d33160 (0x6561 bytes) 2024-09-21T21:41:10.219 INFO:teuthology.orchestra.run.smithi164.stdout:1994 write 0xcf7c400 thru 0xcf85711 (0x9312 bytes) 2024-09-21T21:41:10.219 INFO:teuthology.orchestra.run.smithi164.stdout:1996 write 0x47a6000 thru 0x47ae959 (0x895a bytes) 2024-09-21T21:41:10.219 INFO:teuthology.orchestra.run.smithi164.stdout:1997 read 0x3423400 thru 0x3428a05 (0x5606 bytes) 2024-09-21T21:41:10.219 INFO:teuthology.orchestra.run.smithi164.stdout:1999 write 0xec03a00 thru 0xec0b6d2 (0x7cd3 bytes) 2024-09-21T21:41:10.219 INFO:teuthology.orchestra.run.smithi164.stdout:2001 trunc from 0xec0b800 to 0xcc1f000 2024-09-21T21:41:10.226 INFO:teuthology.orchestra.run.smithi164.stdout:2002 write 0x13a1400 thru 0x13a7061 (0x5c62 bytes) 2024-09-21T21:41:10.238 INFO:teuthology.orchestra.run.smithi164.stdout:2004 trunc from 0xcc1f000 to 0x1ff9000 2024-09-21T21:41:10.284 INFO:teuthology.orchestra.run.smithi164.stdout:2005 write 0x6661a00 thru 0x666499e (0x2f9f bytes) 2024-09-21T21:41:10.309 INFO:teuthology.orchestra.run.smithi164.stdout:2006 write 0xb604200 thru 0xb60558c (0x138d bytes) 2024-09-21T21:41:10.310 INFO:teuthology.orchestra.run.smithi164.stdout:2007 trunc from 0xb605600 to 0xc77d200 2024-09-21T21:41:10.317 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:10.316+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:10.328 INFO:teuthology.orchestra.run.smithi164.stdout:2008 read 0xa1cfe00 thru 0xa1d3de6 (0x3fe7 bytes) 2024-09-21T21:41:10.328 INFO:teuthology.orchestra.run.smithi164.stdout:2009 punch from 0x9ffe00 to 0xa04200, (0x4400 bytes) 2024-09-21T21:41:10.329 INFO:teuthology.orchestra.run.smithi164.stdout:2010 trunc from 0xc77d200 to 0xc43b800 2024-09-21T21:41:10.331 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:41:10.331 INFO:teuthology.orchestra.run.smithi081.stderr:dumped all 2024-09-21T21:41:10.337 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:10.337 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:41:10.337 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 0 2024-09-21T21:41:10.337 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:10.346 INFO:teuthology.orchestra.run.smithi081.stdout:{"pg_ready":true,"pg_map":{"version":66,"stamp":"2024-09-21T21:41:10.299470+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":355079266,"num_objects":446,"num_object_clones":59,"num_object_copies":892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":446,"num_whiteouts":132,"num_read":85371,"num_read_kb":384732,"num_write":5676,"num_write_kb":84012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4907,"ondisk_log_size":4907,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":224024,"kb_used_data":15884,"kb_used_omap":9,"kb_used_meta":207798,"kb_avail":566007016,"statfs":{"total":579820584960,"available":579591184384,"internally_reserved":0,"allocated":16265216,"data_stored":94126384,"data_compressed":7528193,"data_compressed_allocated":8232960,"data_compressed_original":88571904,"omap_allocated":9568,"internal_metadata":212785824},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-3388325,"num_objects":45,"num_object_clones":8,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":4,"num_read":9007,"num_read_kb":58432,"num_write":713,"num_write_kb":13182,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.753504"},"pg_stats":[{"pgid":"3.1f","version":"43'114","reported_seq":1741,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T21:41:08.193622+0000","last_change":"2024-09-21T21:40:39.430005+0000","last_active":"2024-09-21T21:41:08.193622+0000","last_peered":"2024-09-21T21:41:08.193622+0000","last_clean":"2024-09-21T21:41:08.193622+0000","last_became_active":"2024-09-21T21:40:37.424213+0000","last_became_peered":"2024-09-21T21:40:37.424213+0000","last_unstale":"2024-09-21T21:41:08.193622+0000","last_undegraded":"2024-09-21T21:41:08.193622+0000","last_fullsized":"2024-09-21T21:41:08.193622+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:06:18.329638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010985019999999999,"stat_sum":{"num_bytes":10915392,"num_objects":15,"num_object_clones":2,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":3,"num_read":2307,"num_read_kb":8450,"num_write":97,"num_write_kb":4322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"41'224","reported_seq":2269,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T21:41:06.058035+0000","last_change":"2024-09-21T21:40:39.416688+0000","last_active":"2024-09-21T21:41:06.058035+0000","last_peered":"2024-09-21T21:41:06.058035+0000","last_clean":"2024-09-21T21:41:06.058035+0000","last_became_active":"2024-09-21T21:40:37.417190+0000","last_became_peered":"2024-09-21T21:40:37.417190+0000","last_unstale":"2024-09-21T21:41:06.058035+0000","last_undegraded":"2024-09-21T21:41:06.058035+0000","last_fullsized":"2024-09-21T21:41:06.058035+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":224,"log_dups_size":0,"ondisk_log_size":224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:40:58.121639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042608099999999997,"stat_sum":{"num_bytes":11241515,"num_objects":15,"num_object_clones":4,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":7,"num_read":2828,"num_read_kb":22801,"num_write":355,"num_write_kb":6653,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"43'113","reported_seq":821,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T21:41:09.246614+0000","last_change":"2024-09-21T21:41:05.551822+0000","last_active":"2024-09-21T21:41:09.246614+0000","last_peered":"2024-09-21T21:41:09.246614+0000","last_clean":"2024-09-21T21:41:09.246614+0000","last_became_active":"2024-09-21T21:41:05.551289+0000","last_became_peered":"2024-09-21T21:41:05.551289+0000","last_unstale":"2024-09-21T21:41:09.246614+0000","last_undegraded":"2024-09-21T21:41:09.246614+0000","last_fullsized":"2024-09-21T21:41:09.246614+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":113,"log_dups_size":0,"ondisk_log_size":113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:05:51.887240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001986561,"stat_sum":{"num_bytes":2631710,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":4,"num_read":821,"num_read_kb":5053,"num_write":136,"num_write_kb":601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"43'472","reported_seq":5087,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T21:41:08.581704+0000","last_change":"2024-09-21T21:41:05.551891+0000","last_active":"2024-09-21T21:41:08.581704+0000","last_peered":"2024-09-21T21:41:08.581704+0000","last_clean":"2024-09-21T21:41:08.581704+0000","last_became_active":"2024-09-21T21:41:05.551346+0000","last_became_peered":"2024-09-21T21:41:05.551346+0000","last_unstale":"2024-09-21T21:41:08.581704+0000","last_undegraded":"2024-09-21T21:41:08.581704+0000","last_fullsized":"2024-09-21T21:41:08.581704+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":472,"log_dups_size":0,"ondisk_log_size":472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:28:38.277910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011105570000000001,"stat_sum":{"num_bytes":4454437,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":2,"num_read":7859,"num_read_kb":10629,"num_write":452,"num_write_kb":1132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"43'70","reported_seq":1105,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T21:41:07.560376+0000","last_change":"2024-09-21T21:40:39.429478+0000","last_active":"2024-09-21T21:41:07.560376+0000","last_peered":"2024-09-21T21:41:07.560376+0000","last_clean":"2024-09-21T21:41:07.560376+0000","last_became_active":"2024-09-21T21:40:37.425925+0000","last_became_peered":"2024-09-21T21:40:37.425925+0000","last_unstale":"2024-09-21T21:41:07.560376+0000","last_undegraded":"2024-09-21T21:41:07.560376+0000","last_fullsized":"2024-09-21T21:41:07.560376+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:10:14.175605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069216500000000001,"stat_sum":{"num_bytes":5046826,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":6,"num_read":989,"num_read_kb":3319,"num_write":67,"num_write_kb":1402,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"43'65","reported_seq":1734,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T21:41:09.306901+0000","last_change":"2024-09-21T21:40:39.420213+0000","last_active":"2024-09-21T21:41:09.306901+0000","last_peered":"2024-09-21T21:41:09.306901+0000","last_clean":"2024-09-21T21:41:09.306901+0000","last_became_active":"2024-09-21T21:40:37.424316+0000","last_became_peered":"2024-09-21T21:40:37.424316+0000","last_unstale":"2024-09-21T21:41:09.306901+0000","last_undegraded":"2024-09-21T21:41:09.306901+0000","last_fullsized":"2024-09-21T21:41:09.306901+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:41:10.080952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095031900000000003,"stat_sum":{"num_bytes":2793555,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":4,"num_read":1642,"num_read_kb":4569,"num_write":64,"num_write_kb":230,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"43'296","reported_seq":3711,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T21:41:08.309211+0000","last_change":"2024-09-21T21:40:39.414575+0000","last_active":"2024-09-21T21:41:08.309211+0000","last_peered":"2024-09-21T21:41:08.309211+0000","last_clean":"2024-09-21T21:41:08.309211+0000","last_became_active":"2024-09-21T21:40:37.426715+0000","last_became_peered":"2024-09-21T21:40:37.426715+0000","last_unstale":"2024-09-21T21:41:08.309211+0000","last_undegraded":"2024-09-21T21:41:08.309211+0000","last_fullsized":"2024-09-21T21:41:08.309211+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":296,"log_dups_size":0,"ondisk_log_size":296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:19:27.583642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066263800000000001,"stat_sum":{"num_bytes":15991399,"num_objects":21,"num_object_clones":1,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":5,"num_read":5214,"num_read_kb":16285,"num_write":259,"num_write_kb":3873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"38'52","reported_seq":1587,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T21:41:05.545040+0000","last_change":"2024-09-21T21:40:39.416634+0000","last_active":"2024-09-21T21:41:05.545040+0000","last_peered":"2024-09-21T21:41:05.545040+0000","last_clean":"2024-09-21T21:41:05.545040+0000","last_became_active":"2024-09-21T21:40:37.421603+0000","last_became_peered":"2024-09-21T21:40:37.421603+0000","last_unstale":"2024-09-21T21:41:05.545040+0000","last_undegraded":"2024-09-21T21:41:05.545040+0000","last_fullsized":"2024-09-21T21:41:05.545040+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:45:46.972148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042576799999999998,"stat_sum":{"num_bytes":1897472,"num_objects":8,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":3,"num_read":1486,"num_read_kb":5564,"num_write":57,"num_write_kb":548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"43'101","reported_seq":1838,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T21:41:08.261287+0000","last_change":"2024-09-21T21:40:39.430190+0000","last_active":"2024-09-21T21:41:08.261287+0000","last_peered":"2024-09-21T21:41:08.261287+0000","last_clean":"2024-09-21T21:41:08.261287+0000","last_became_active":"2024-09-21T21:40:37.424911+0000","last_became_peered":"2024-09-21T21:40:37.424911+0000","last_unstale":"2024-09-21T21:41:08.261287+0000","last_undegraded":"2024-09-21T21:41:08.261287+0000","last_fullsized":"2024-09-21T21:41:08.261287+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:39:35.909461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001258211,"stat_sum":{"num_bytes":14714929,"num_objects":17,"num_object_clones":3,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":7,"num_read":2404,"num_read_kb":5355,"num_write":65,"num_write_kb":246,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"36'126","reported_seq":3816,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T21:41:07.990097+0000","last_change":"2024-09-21T21:40:39.425366+0000","last_active":"2024-09-21T21:41:07.990097+0000","last_peered":"2024-09-21T21:41:07.990097+0000","last_clean":"2024-09-21T21:41:07.990097+0000","last_became_active":"2024-09-21T21:40:37.424364+0000","last_became_peered":"2024-09-21T21:40:37.424364+0000","last_unstale":"2024-09-21T21:41:07.990097+0000","last_undegraded":"2024-09-21T21:41:07.990097+0000","last_fullsized":"2024-09-21T21:41:07.990097+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:58:38.272465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015121799999999999,"stat_sum":{"num_bytes":22699046,"num_objects":16,"num_object_clones":2,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":2,"num_read":4383,"num_read_kb":17739,"num_write":99,"num_write_kb":1129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"43'69","reported_seq":499,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T21:41:09.269363+0000","last_change":"2024-09-21T21:40:39.415411+0000","last_active":"2024-09-21T21:41:09.269363+0000","last_peered":"2024-09-21T21:41:09.269363+0000","last_clean":"2024-09-21T21:41:09.269363+0000","last_became_active":"2024-09-21T21:40:37.421701+0000","last_became_peered":"2024-09-21T21:40:37.421701+0000","last_unstale":"2024-09-21T21:41:09.269363+0000","last_undegraded":"2024-09-21T21:41:09.269363+0000","last_fullsized":"2024-09-21T21:41:09.269363+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:40:49.348069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001006961,"stat_sum":{"num_bytes":7207424,"num_objects":9,"num_object_clones":1,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":3,"num_read":379,"num_read_kb":3249,"num_write":66,"num_write_kb":325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"43'117","reported_seq":2742,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T21:41:09.057907+0000","last_change":"2024-09-21T21:40:39.415100+0000","last_active":"2024-09-21T21:41:09.057907+0000","last_peered":"2024-09-21T21:41:09.057907+0000","last_clean":"2024-09-21T21:41:09.057907+0000","last_became_active":"2024-09-21T21:40:37.426062+0000","last_became_peered":"2024-09-21T21:40:37.426062+0000","last_unstale":"2024-09-21T21:41:09.057907+0000","last_undegraded":"2024-09-21T21:41:09.057907+0000","last_fullsized":"2024-09-21T21:41:09.057907+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:09:47.738847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093907599999999995,"stat_sum":{"num_bytes":14954623,"num_objects":15,"num_object_clones":2,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":4,"num_read":2590,"num_read_kb":15169,"num_write":108,"num_write_kb":4504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T21:41:07.564898+0000","last_change":"2024-09-21T21:40:27.380563+0000","last_active":"2024-09-21T21:41:07.564898+0000","last_peered":"2024-09-21T21:41:07.564898+0000","last_clean":"2024-09-21T21:41:07.564898+0000","last_became_active":"2024-09-21T21:40:25.501200+0000","last_became_peered":"2024-09-21T21:40:25.501200+0000","last_unstale":"2024-09-21T21:41:07.564898+0000","last_undegraded":"2024-09-21T21:41:07.564898+0000","last_fullsized":"2024-09-21T21:41:07.564898+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:03:24.900063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072643899999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"43'110","reported_seq":2691,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T21:41:08.060889+0000","last_change":"2024-09-21T21:40:39.430054+0000","last_active":"2024-09-21T21:41:08.060889+0000","last_peered":"2024-09-21T21:41:08.060889+0000","last_clean":"2024-09-21T21:41:08.060889+0000","last_became_active":"2024-09-21T21:40:37.422487+0000","last_became_peered":"2024-09-21T21:40:37.422487+0000","last_unstale":"2024-09-21T21:41:08.060889+0000","last_undegraded":"2024-09-21T21:41:08.060889+0000","last_fullsized":"2024-09-21T21:41:08.060889+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:12:58.966396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011470160000000001,"stat_sum":{"num_bytes":17904128,"num_objects":9,"num_object_clones":1,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":2,"num_read":3136,"num_read_kb":27254,"num_write":107,"num_write_kb":2346,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T21:41:07.560630+0000","last_change":"2024-09-21T21:40:27.379903+0000","last_active":"2024-09-21T21:41:07.560630+0000","last_peered":"2024-09-21T21:41:07.560630+0000","last_clean":"2024-09-21T21:41:07.560630+0000","last_became_active":"2024-09-21T21:40:25.371095+0000","last_became_peered":"2024-09-21T21:40:25.371095+0000","last_unstale":"2024-09-21T21:41:07.560630+0000","last_undegraded":"2024-09-21T21:41:07.560630+0000","last_fullsized":"2024-09-21T21:41:07.560630+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:08:09.434308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062280799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"38'70","reported_seq":1126,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T21:41:05.545139+0000","last_change":"2024-09-21T21:40:39.416965+0000","last_active":"2024-09-21T21:41:05.545139+0000","last_peered":"2024-09-21T21:41:05.545139+0000","last_clean":"2024-09-21T21:41:05.545139+0000","last_became_active":"2024-09-21T21:40:37.418428+0000","last_became_peered":"2024-09-21T21:40:37.418428+0000","last_unstale":"2024-09-21T21:41:05.545139+0000","last_undegraded":"2024-09-21T21:41:05.545139+0000","last_fullsized":"2024-09-21T21:41:05.545139+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:30:28.192627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066113800000000003,"stat_sum":{"num_bytes":13553664,"num_objects":8,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":4,"num_read":1005,"num_read_kb":9620,"num_write":73,"num_write_kb":828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T21:41:07.564841+0000","last_change":"2024-09-21T21:40:27.380218+0000","last_active":"2024-09-21T21:41:07.564841+0000","last_peered":"2024-09-21T21:41:07.564841+0000","last_clean":"2024-09-21T21:41:07.564841+0000","last_became_active":"2024-09-21T21:40:25.500143+0000","last_became_peered":"2024-09-21T21:40:25.500143+0000","last_unstale":"2024-09-21T21:41:07.564841+0000","last_undegraded":"2024-09-21T21:41:07.564841+0000","last_fullsized":"2024-09-21T21:41:07.564841+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:11:42.747388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042715899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"43'79","reported_seq":598,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T21:41:08.295875+0000","last_change":"2024-09-21T21:40:39.414839+0000","last_active":"2024-09-21T21:41:08.295875+0000","last_peered":"2024-09-21T21:41:08.295875+0000","last_clean":"2024-09-21T21:41:08.295875+0000","last_became_active":"2024-09-21T21:40:37.419944+0000","last_became_peered":"2024-09-21T21:40:37.419944+0000","last_unstale":"2024-09-21T21:41:08.295875+0000","last_undegraded":"2024-09-21T21:41:08.295875+0000","last_fullsized":"2024-09-21T21:41:08.295875+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:53:22.209416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085769100000000005,"stat_sum":{"num_bytes":3850326,"num_objects":13,"num_object_clones":2,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":4,"num_read":614,"num_read_kb":4015,"num_write":116,"num_write_kb":1278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":67,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T21:41:07.561030+0000","last_change":"2024-09-21T21:40:27.380005+0000","last_active":"2024-09-21T21:41:07.561030+0000","last_peered":"2024-09-21T21:41:07.561030+0000","last_clean":"2024-09-21T21:41:07.561030+0000","last_became_active":"2024-09-21T21:40:25.500109+0000","last_became_peered":"2024-09-21T21:40:25.500109+0000","last_unstale":"2024-09-21T21:41:07.561030+0000","last_undegraded":"2024-09-21T21:41:07.561030+0000","last_fullsized":"2024-09-21T21:41:07.561030+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:21:10.631127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070669899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"39'56","reported_seq":881,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T21:41:05.550283+0000","last_change":"2024-09-21T21:41:05.549770+0000","last_active":"2024-09-21T21:41:05.550283+0000","last_peered":"2024-09-21T21:41:05.550283+0000","last_clean":"2024-09-21T21:41:05.550283+0000","last_became_active":"2024-09-21T21:41:05.549265+0000","last_became_peered":"2024-09-21T21:41:05.549265+0000","last_unstale":"2024-09-21T21:41:05.550283+0000","last_undegraded":"2024-09-21T21:41:05.550283+0000","last_fullsized":"2024-09-21T21:41:05.550283+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:21:43.892073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010561050000000001,"stat_sum":{"num_bytes":5076000,"num_objects":15,"num_object_clones":1,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":6,"num_read":786,"num_read_kb":9244,"num_write":56,"num_write_kb":581,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"43'76","reported_seq":2115,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T21:41:09.288143+0000","last_change":"2024-09-21T21:40:39.419600+0000","last_active":"2024-09-21T21:41:09.288143+0000","last_peered":"2024-09-21T21:41:09.288143+0000","last_clean":"2024-09-21T21:41:09.288143+0000","last_became_active":"2024-09-21T21:40:37.424903+0000","last_became_peered":"2024-09-21T21:40:37.424903+0000","last_unstale":"2024-09-21T21:41:09.288143+0000","last_undegraded":"2024-09-21T21:41:09.288143+0000","last_fullsized":"2024-09-21T21:41:09.288143+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:52:35.815117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041679500000000002,"stat_sum":{"num_bytes":4444672,"num_objects":10,"num_object_clones":2,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":3,"num_read":2000,"num_read_kb":9475,"num_write":83,"num_write_kb":3386,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":32,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T21:41:07.560979+0000","last_change":"2024-09-21T21:41:05.549766+0000","last_active":"2024-09-21T21:41:07.560979+0000","last_peered":"2024-09-21T21:41:07.560979+0000","last_clean":"2024-09-21T21:41:07.560979+0000","last_became_active":"2024-09-21T21:41:05.549439+0000","last_became_peered":"2024-09-21T21:41:05.549439+0000","last_unstale":"2024-09-21T21:41:07.560979+0000","last_undegraded":"2024-09-21T21:41:07.560979+0000","last_fullsized":"2024-09-21T21:41:07.560979+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:49:30.301108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"43'167","reported_seq":2891,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T21:41:08.305304+0000","last_change":"2024-09-21T21:40:39.414972+0000","last_active":"2024-09-21T21:41:08.305304+0000","last_peered":"2024-09-21T21:41:08.305304+0000","last_clean":"2024-09-21T21:41:08.305304+0000","last_became_active":"2024-09-21T21:40:37.418973+0000","last_became_peered":"2024-09-21T21:40:37.418973+0000","last_unstale":"2024-09-21T21:41:08.305304+0000","last_undegraded":"2024-09-21T21:41:08.305304+0000","last_fullsized":"2024-09-21T21:41:08.305304+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:28:22.690678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001017923,"stat_sum":{"num_bytes":20886048,"num_objects":18,"num_object_clones":5,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":4,"num_read":3850,"num_read_kb":10009,"num_write":114,"num_write_kb":912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":61,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T21:41:05.545085+0000","last_change":"2024-09-21T21:40:27.380792+0000","last_active":"2024-09-21T21:41:05.545085+0000","last_peered":"2024-09-21T21:41:05.545085+0000","last_clean":"2024-09-21T21:41:05.545085+0000","last_became_active":"2024-09-21T21:40:25.371578+0000","last_became_peered":"2024-09-21T21:40:25.371578+0000","last_unstale":"2024-09-21T21:41:05.545085+0000","last_undegraded":"2024-09-21T21:41:05.545085+0000","last_fullsized":"2024-09-21T21:41:05.545085+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:16:44.813324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000459858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"38'38","reported_seq":838,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T21:41:07.560126+0000","last_change":"2024-09-21T21:40:39.429211+0000","last_active":"2024-09-21T21:41:07.560126+0000","last_peered":"2024-09-21T21:41:07.560126+0000","last_clean":"2024-09-21T21:41:07.560126+0000","last_became_active":"2024-09-21T21:40:37.428542+0000","last_became_peered":"2024-09-21T21:40:37.428542+0000","last_unstale":"2024-09-21T21:41:07.560126+0000","last_undegraded":"2024-09-21T21:41:07.560126+0000","last_fullsized":"2024-09-21T21:41:07.560126+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:57:20.820527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045982299999999999,"stat_sum":{"num_bytes":2755627,"num_objects":8,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":5,"num_read":753,"num_read_kb":4856,"num_write":33,"num_write_kb":94,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T21:41:07.564570+0000","last_change":"2024-09-21T21:40:27.380219+0000","last_active":"2024-09-21T21:41:07.564570+0000","last_peered":"2024-09-21T21:41:07.564570+0000","last_clean":"2024-09-21T21:41:07.564570+0000","last_became_active":"2024-09-21T21:40:25.371451+0000","last_became_peered":"2024-09-21T21:40:25.371451+0000","last_unstale":"2024-09-21T21:41:07.564570+0000","last_undegraded":"2024-09-21T21:41:07.564570+0000","last_fullsized":"2024-09-21T21:41:07.564570+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:05:41.445313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000438863,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"42'202","reported_seq":2223,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T21:41:07.560982+0000","last_change":"2024-09-21T21:40:39.415337+0000","last_active":"2024-09-21T21:41:07.560982+0000","last_peered":"2024-09-21T21:41:07.560982+0000","last_clean":"2024-09-21T21:41:07.560982+0000","last_became_active":"2024-09-21T21:40:37.421655+0000","last_became_peered":"2024-09-21T21:40:37.421655+0000","last_unstale":"2024-09-21T21:41:07.560982+0000","last_undegraded":"2024-09-21T21:41:07.560982+0000","last_fullsized":"2024-09-21T21:41:07.560982+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":202,"log_dups_size":0,"ondisk_log_size":202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:54:17.125307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000835322,"stat_sum":{"num_bytes":20331573,"num_objects":17,"num_object_clones":1,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":7,"num_read":2513,"num_read_kb":13241,"num_write":273,"num_write_kb":2846,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":18,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T21:41:05.548995+0000","last_change":"2024-09-21T21:41:05.548635+0000","last_active":"2024-09-21T21:41:05.548995+0000","last_peered":"2024-09-21T21:41:05.548995+0000","last_clean":"2024-09-21T21:41:05.548995+0000","last_became_active":"2024-09-21T21:41:05.548420+0000","last_became_peered":"2024-09-21T21:41:05.548420+0000","last_unstale":"2024-09-21T21:41:05.548995+0000","last_undegraded":"2024-09-21T21:41:05.548995+0000","last_fullsized":"2024-09-21T21:41:05.548995+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:36:23.320735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":217,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T21:41:07.564585+0000","last_change":"2024-09-21T21:40:19.352502+0000","last_active":"2024-09-21T21:41:07.564585+0000","last_peered":"2024-09-21T21:41:07.564585+0000","last_clean":"2024-09-21T21:41:07.564585+0000","last_became_active":"2024-09-21T21:40:19.352227+0000","last_became_peered":"2024-09-21T21:40:19.352227+0000","last_unstale":"2024-09-21T21:41:07.564585+0000","last_undegraded":"2024-09-21T21:41:07.564585+0000","last_fullsized":"2024-09-21T21:41:07.564585+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:18.340430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:18.340430+0000","last_clean_scrub_stamp":"2024-09-21T21:40:18.340430+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:19:13.902196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"39'70","reported_seq":2405,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T21:41:05.545153+0000","last_change":"2024-09-21T21:40:39.416961+0000","last_active":"2024-09-21T21:41:05.545153+0000","last_peered":"2024-09-21T21:41:05.545153+0000","last_clean":"2024-09-21T21:41:05.545153+0000","last_became_active":"2024-09-21T21:40:37.418416+0000","last_became_peered":"2024-09-21T21:40:37.418416+0000","last_unstale":"2024-09-21T21:41:05.545153+0000","last_undegraded":"2024-09-21T21:41:05.545153+0000","last_fullsized":"2024-09-21T21:41:05.545153+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:41:38.644877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056255200000000002,"stat_sum":{"num_bytes":10902073,"num_objects":17,"num_object_clones":3,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":5,"num_read":2372,"num_read_kb":9733,"num_write":73,"num_write_kb":1719,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"43'107","reported_seq":1159,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T21:41:09.303545+0000","last_change":"2024-09-21T21:40:39.419805+0000","last_active":"2024-09-21T21:41:09.303545+0000","last_peered":"2024-09-21T21:41:09.303545+0000","last_clean":"2024-09-21T21:41:09.303545+0000","last_became_active":"2024-09-21T21:40:37.425495+0000","last_became_peered":"2024-09-21T21:40:37.425495+0000","last_unstale":"2024-09-21T21:41:09.303545+0000","last_undegraded":"2024-09-21T21:41:09.303545+0000","last_fullsized":"2024-09-21T21:41:09.303545+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:13:47.333541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035788200000000001,"stat_sum":{"num_bytes":18729047,"num_objects":23,"num_object_clones":4,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":9,"num_read":1027,"num_read_kb":7352,"num_write":116,"num_write_kb":2046,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"43'257","reported_seq":584,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T21:41:09.280021+0000","last_change":"2024-09-21T21:40:39.414797+0000","last_active":"2024-09-21T21:41:09.280021+0000","last_peered":"2024-09-21T21:41:09.280021+0000","last_clean":"2024-09-21T21:41:09.280021+0000","last_became_active":"2024-09-21T21:40:37.426458+0000","last_became_peered":"2024-09-21T21:40:37.426458+0000","last_unstale":"2024-09-21T21:41:09.280021+0000","last_undegraded":"2024-09-21T21:41:09.280021+0000","last_fullsized":"2024-09-21T21:41:09.280021+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":257,"log_dups_size":0,"ondisk_log_size":257,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:13:11.924778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065146799999999999,"stat_sum":{"num_bytes":3125886,"num_objects":11,"num_object_clones":1,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":3,"num_read":1117,"num_read_kb":5106,"num_write":376,"num_write_kb":2459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"43'414","reported_seq":1079,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T21:41:08.308853+0000","last_change":"2024-09-21T21:40:39.415091+0000","last_active":"2024-09-21T21:41:08.308853+0000","last_peered":"2024-09-21T21:41:08.308853+0000","last_clean":"2024-09-21T21:41:08.308853+0000","last_became_active":"2024-09-21T21:40:37.421700+0000","last_became_peered":"2024-09-21T21:40:37.421700+0000","last_unstale":"2024-09-21T21:41:08.308853+0000","last_undegraded":"2024-09-21T21:41:08.308853+0000","last_fullsized":"2024-09-21T21:41:08.308853+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":414,"log_dups_size":0,"ondisk_log_size":414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:52:09.645290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001083334,"stat_sum":{"num_bytes":5662954,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":2,"num_read":2481,"num_read_kb":6742,"num_write":730,"num_write_kb":5837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"42'350","reported_seq":3765,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T21:41:08.277276+0000","last_change":"2024-09-21T21:41:05.551020+0000","last_active":"2024-09-21T21:41:08.277276+0000","last_peered":"2024-09-21T21:41:08.277276+0000","last_clean":"2024-09-21T21:41:08.277276+0000","last_became_active":"2024-09-21T21:41:05.549644+0000","last_became_peered":"2024-09-21T21:41:05.549644+0000","last_unstale":"2024-09-21T21:41:08.277276+0000","last_undegraded":"2024-09-21T21:41:08.277276+0000","last_fullsized":"2024-09-21T21:41:08.277276+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":350,"log_dups_size":0,"ondisk_log_size":350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:24:59.178416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001302274,"stat_sum":{"num_bytes":12312687,"num_objects":16,"num_object_clones":4,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":4,"num_read":4709,"num_read_kb":16576,"num_write":559,"num_write_kb":5892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"43'121","reported_seq":4770,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T21:41:09.289305+0000","last_change":"2024-09-21T21:40:39.415498+0000","last_active":"2024-09-21T21:41:09.289305+0000","last_peered":"2024-09-21T21:41:09.289305+0000","last_clean":"2024-09-21T21:41:09.289305+0000","last_became_active":"2024-09-21T21:40:37.416021+0000","last_became_peered":"2024-09-21T21:40:37.416021+0000","last_unstale":"2024-09-21T21:41:09.289305+0000","last_undegraded":"2024-09-21T21:41:09.289305+0000","last_fullsized":"2024-09-21T21:41:09.289305+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":121,"log_dups_size":0,"ondisk_log_size":121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:51:49.009075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098736399999999991,"stat_sum":{"num_bytes":6451200,"num_objects":16,"num_object_clones":2,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":5,"num_read":4622,"num_read_kb":7505,"num_write":115,"num_write_kb":6171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"43'90","reported_seq":3764,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T21:41:09.254993+0000","last_change":"2024-09-21T21:41:05.551767+0000","last_active":"2024-09-21T21:41:09.254993+0000","last_peered":"2024-09-21T21:41:09.254993+0000","last_clean":"2024-09-21T21:41:09.254993+0000","last_became_active":"2024-09-21T21:41:05.551305+0000","last_became_peered":"2024-09-21T21:41:05.551305+0000","last_unstale":"2024-09-21T21:41:09.254993+0000","last_undegraded":"2024-09-21T21:41:09.254993+0000","last_fullsized":"2024-09-21T21:41:09.254993+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:01:54.187236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020524470000000002,"stat_sum":{"num_bytes":23915045,"num_objects":12,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":2,"num_read":3644,"num_read_kb":22943,"num_write":97,"num_write_kb":2370,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"43'154","reported_seq":1884,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T21:41:08.276800+0000","last_change":"2024-09-21T21:40:39.425496+0000","last_active":"2024-09-21T21:41:08.276800+0000","last_peered":"2024-09-21T21:41:08.276800+0000","last_clean":"2024-09-21T21:41:08.276800+0000","last_became_active":"2024-09-21T21:40:37.424987+0000","last_became_peered":"2024-09-21T21:40:37.424987+0000","last_unstale":"2024-09-21T21:41:08.276800+0000","last_undegraded":"2024-09-21T21:41:08.276800+0000","last_fullsized":"2024-09-21T21:41:08.276800+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:21:31.674193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016364540000000001,"stat_sum":{"num_bytes":3058704,"num_objects":16,"num_object_clones":3,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":4,"num_read":3028,"num_read_kb":6351,"num_write":103,"num_write_kb":616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"43'90","reported_seq":4484,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T21:41:08.289377+0000","last_change":"2024-09-21T21:41:05.549466+0000","last_active":"2024-09-21T21:41:08.289377+0000","last_peered":"2024-09-21T21:41:08.289377+0000","last_clean":"2024-09-21T21:41:08.289377+0000","last_became_active":"2024-09-21T21:41:05.549078+0000","last_became_peered":"2024-09-21T21:41:05.549078+0000","last_unstale":"2024-09-21T21:41:08.289377+0000","last_undegraded":"2024-09-21T21:41:08.289377+0000","last_fullsized":"2024-09-21T21:41:08.289377+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:32:54.331133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065325099999999998,"stat_sum":{"num_bytes":26439680,"num_objects":14,"num_object_clones":4,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":4,"num_read":4380,"num_read_kb":28362,"num_write":90,"num_write_kb":3095,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"43'231","reported_seq":1714,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T21:41:08.589652+0000","last_change":"2024-09-21T21:40:39.415203+0000","last_active":"2024-09-21T21:41:08.589652+0000","last_peered":"2024-09-21T21:41:08.589652+0000","last_clean":"2024-09-21T21:41:08.589652+0000","last_became_active":"2024-09-21T21:40:37.426148+0000","last_became_peered":"2024-09-21T21:40:37.426148+0000","last_unstale":"2024-09-21T21:41:08.589652+0000","last_undegraded":"2024-09-21T21:41:08.589652+0000","last_fullsized":"2024-09-21T21:41:08.589652+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":231,"log_dups_size":0,"ondisk_log_size":231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:05:16.187844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087239000000000001,"stat_sum":{"num_bytes":8222757,"num_objects":13,"num_object_clones":1,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":2,"num_read":2126,"num_read_kb":12313,"num_write":315,"num_write_kb":2899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"39'163","reported_seq":4750,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T21:41:08.124542+0000","last_change":"2024-09-21T21:40:39.429216+0000","last_active":"2024-09-21T21:41:08.124542+0000","last_peered":"2024-09-21T21:41:08.124542+0000","last_clean":"2024-09-21T21:41:08.124542+0000","last_became_active":"2024-09-21T21:40:37.424011+0000","last_became_peered":"2024-09-21T21:40:37.424011+0000","last_unstale":"2024-09-21T21:41:08.124542+0000","last_undegraded":"2024-09-21T21:41:08.124542+0000","last_fullsized":"2024-09-21T21:41:08.124542+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":163,"log_dups_size":0,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:27:06.828117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046261000000000002,"stat_sum":{"num_bytes":6296576,"num_objects":16,"num_object_clones":4,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":4,"num_read":5221,"num_read_kb":13833,"num_write":114,"num_write_kb":3404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"39'123","reported_seq":3115,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T21:41:08.154030+0000","last_change":"2024-09-21T21:40:39.414779+0000","last_active":"2024-09-21T21:41:08.154030+0000","last_peered":"2024-09-21T21:41:08.154030+0000","last_clean":"2024-09-21T21:41:08.154030+0000","last_became_active":"2024-09-21T21:40:37.421627+0000","last_became_peered":"2024-09-21T21:40:37.421627+0000","last_unstale":"2024-09-21T21:41:08.154030+0000","last_undegraded":"2024-09-21T21:41:08.154030+0000","last_fullsized":"2024-09-21T21:41:08.154030+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:31:01.648496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070986300000000003,"stat_sum":{"num_bytes":26021904,"num_objects":15,"num_object_clones":2,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":3,"num_read":2969,"num_read_kb":41920,"num_write":147,"num_write_kb":7850,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":354488879,"num_objects":442,"num_object_clones":59,"num_object_copies":884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":442,"num_whiteouts":132,"num_read":85255,"num_read_kb":384632,"num_write":5515,"num_write_kb":81594,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":13447168,"data_stored":91922952,"data_compressed":7460873,"data_compressed_allocated":8134656,"data_compressed_original":87371776,"omap_allocated":0,"internal_metadata":0},"log_size":4787,"ondisk_log_size":4787,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64790,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":4,"up_from":13,"seq":55834574861,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37392,"kb_used_data":2000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334448,"statfs":{"total":96636764160,"available":96598474752,"internally_reserved":0,"allocated":2048000,"data_stored":13872891,"data_compressed":1148073,"data_compressed_allocated":1138688,"data_compressed_original":13353472,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574861,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38856,"kb_used_data":3512,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94332984,"statfs":{"total":96636764160,"available":96596975616,"internally_reserved":0,"allocated":3596288,"data_stored":29222910,"data_compressed":2115801,"data_compressed_allocated":2146304,"data_compressed_original":28201472,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":12,"seq":51539607565,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38488,"kb_used_data":3140,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333352,"statfs":{"total":96636764160,"available":96597352448,"internally_reserved":0,"allocated":3215360,"data_stored":19099073,"data_compressed":1368383,"data_compressed_allocated":1564672,"data_compressed_original":17901568,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607565,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37496,"kb_used_data":1996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334344,"statfs":{"total":96636764160,"available":96598368256,"internally_reserved":0,"allocated":2043904,"data_stored":5061033,"data_compressed":530496,"data_compressed_allocated":663552,"data_compressed_original":4112896,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607566,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38544,"kb_used_data":3184,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333296,"statfs":{"total":96636764160,"available":96597295104,"internally_reserved":0,"allocated":3260416,"data_stored":17243043,"data_compressed":1423576,"data_compressed_allocated":1536000,"data_compressed_original":15931392,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640269,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33248,"kb_used_data":2052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338592,"statfs":{"total":96636764160,"available":96602718208,"internally_reserved":0,"allocated":2101248,"data_stored":9627434,"data_compressed":941864,"data_compressed_allocated":1183744,"data_compressed_original":9071104,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32395,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32395,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2736128,"data_stored":16475679,"data_compressed":1391181,"data_compressed_allocated":1490944,"data_compressed_original":15357952,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1597440,"data_stored":4890248,"data_compressed":530496,"data_compressed_allocated":663552,"data_compressed_original":4112896,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1646592,"data_stored":9456379,"data_compressed":942264,"data_compressed_allocated":1187840,"data_compressed_original":9079296,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":3080192,"data_stored":28419539,"data_compressed":2080476,"data_compressed_allocated":2088960,"data_compressed_original":27566592,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1650688,"data_stored":13764336,"data_compressed":1148073,"data_compressed_allocated":1138688,"data_compressed_original":13353472,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":2736128,"data_stored":18916771,"data_compressed":1368383,"data_compressed_allocated":1564672,"data_compressed_original":17901568,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T21:41:10.348 INFO:tasks.thrashosds.thrasher:Setting 3.8 to [2, 3] 2024-09-21T21:41:10.348 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.8', '2', '3'] 2024-09-21T21:41:10.348 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.8 2 3 2024-09-21T21:41:10.349 INFO:teuthology.orchestra.run.smithi164.stdout:2011 punch from 0x207ac00 to 0x2087c00, (0xd000 bytes) 2024-09-21T21:41:10.349 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T21:41:10.352 INFO:teuthology.orchestra.run.smithi164.stdout:2013 write 0xa69c00 thru 0xa74317 (0xa718 bytes) 2024-09-21T21:41:10.353 INFO:teuthology.orchestra.run.smithi164.stdout:2014 write 0x1e67600 thru 0x1e67c49 (0x64a bytes) 2024-09-21T21:41:10.353 INFO:teuthology.orchestra.run.smithi164.stdout:2021 read 0x8ab0a00 thru 0x8ab5b60 (0x5161 bytes) 2024-09-21T21:41:10.353 INFO:teuthology.orchestra.run.smithi164.stdout:2022 read 0x6de6c00 thru 0x6dec77c (0x5b7d bytes) 2024-09-21T21:41:10.353 INFO:teuthology.orchestra.run.smithi164.stdout:2023 write 0x9e0e400 thru 0x9e14d39 (0x693a bytes) 2024-09-21T21:41:10.353 INFO:teuthology.orchestra.run.smithi164.stdout:2026 read 0x5225400 thru 0x522dbc9 (0x87ca bytes) 2024-09-21T21:41:10.359 INFO:teuthology.orchestra.run.smithi164.stdout:2027 trunc from 0xc43b800 to 0x718cc00 2024-09-21T21:41:10.380 INFO:teuthology.orchestra.run.smithi164.stdout:2028 read 0x5808000 thru 0x580b9e1 (0x39e2 bytes) 2024-09-21T21:41:10.385 INFO:teuthology.orchestra.run.smithi164.stdout:2033 trunc from 0x718cc00 to 0x47a5a00 2024-09-21T21:41:10.398 INFO:teuthology.orchestra.run.smithi164.stdout:2037 read 0x30f5800 thru 0x30fe103 (0x8904 bytes) 2024-09-21T21:41:10.406 INFO:teuthology.orchestra.run.smithi164.stdout:2039 trunc from 0x47a5a00 to 0x5e63800 2024-09-21T21:41:10.419 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:10.417+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:10.421 INFO:teuthology.orchestra.run.smithi164.stdout:2040 read 0x400c00 thru 0x40bf43 (0xb344 bytes) 2024-09-21T21:41:10.422 INFO:teuthology.orchestra.run.smithi164.stdout:2041 punch from 0x3220800 to 0x3228600, (0x7e00 bytes) 2024-09-21T21:41:10.422 INFO:teuthology.orchestra.run.smithi164.stdout:2042 read 0x57f7200 thru 0x57fecc8 (0x7ac9 bytes) 2024-09-21T21:41:10.441 INFO:teuthology.orchestra.run.smithi164.stdout:2043 write 0x6566a00 thru 0x65721e0 (0xb7e1 bytes) 2024-09-21T21:41:10.441 INFO:teuthology.orchestra.run.smithi164.stdout:2044 read 0x2406000 thru 0x24125c3 (0xc5c4 bytes) 2024-09-21T21:41:10.469 INFO:teuthology.orchestra.run.smithi164.stdout:2048 write 0xc9fcc00 thru 0xc9ff921 (0x2d22 bytes) 2024-09-21T21:41:10.478 INFO:teuthology.orchestra.run.smithi164.stdout:2051 trunc from 0xc9ffa00 to 0x5222000 2024-09-21T21:41:10.510 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:10.510 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:41:10.510 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:41:10.510 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:41:10.510 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:10.519 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:10.518+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:10.520 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T21:41:10.529 INFO:teuthology.orchestra.run.smithi164.stdout:2052 write 0xdec0a00 thru 0xdeca28b (0x988c bytes) 2024-09-21T21:41:10.529 INFO:teuthology.orchestra.run.smithi164.stdout:2053 read 0x21c6000 thru 0x21d4d23 (0xed24 bytes) 2024-09-21T21:41:10.529 INFO:teuthology.orchestra.run.smithi164.stdout:2054 write 0x19ddc00 thru 0x19dddf8 (0x1f9 bytes) 2024-09-21T21:41:10.529 INFO:teuthology.orchestra.run.smithi164.stdout:2058 write 0x4d2be00 thru 0x4d36060 (0xa261 bytes) 2024-09-21T21:41:10.532 INFO:teuthology.orchestra.run.smithi164.stdout:2061 trunc from 0xdeca400 to 0xb34f000 2024-09-21T21:41:10.579 INFO:teuthology.orchestra.run.smithi164.stdout:2062 write 0xe464400 thru 0xe46a5f9 (0x61fa bytes) 2024-09-21T21:41:10.590 INFO:teuthology.orchestra.run.smithi164.stdout:2063 trunc from 0xe46a600 to 0x4fa6e00 2024-09-21T21:41:10.620 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:10.618+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:10.621 INFO:teuthology.orchestra.run.smithi164.stdout:2067 write 0x5f7de00 thru 0x5f84e88 (0x7089 bytes) 2024-09-21T21:41:10.622 INFO:teuthology.orchestra.run.smithi164.stdout:2068 read 0x2981400 thru 0x298aad4 (0x96d5 bytes) 2024-09-21T21:41:10.645 INFO:teuthology.orchestra.run.smithi164.stdout:2069 write 0xc239e00 thru 0xc240331 (0x6532 bytes) 2024-09-21T21:41:10.646 INFO:teuthology.orchestra.run.smithi164.stdout:2070 read 0x6582e00 thru 0x6589aeb (0x6cec bytes) 2024-09-21T21:41:10.646 INFO:teuthology.orchestra.run.smithi164.stdout:2071 write 0xf5d000 thru 0xf6c558 (0xf559 bytes) 2024-09-21T21:41:10.653 INFO:teuthology.orchestra.run.smithi164.stdout:2072 trunc from 0xc240400 to 0x66e3800 2024-09-21T21:41:10.681 INFO:teuthology.orchestra.run.smithi164.stdout:2073 trunc from 0x66e3800 to 0x3866400 2024-09-21T21:41:10.686 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:10.686 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 20, 2024-09-21T21:41:10.686 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 600, 2024-09-21T21:41:10.686 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:41:10.686 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.686 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:1704 3.8 3:1339e2f0:::rbd_data.112145fae779.0000000000000007:a [read 10653696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:10.686 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.382652+0000", 2024-09-21T21:41:10.686 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 12.301059474000001, 2024-09-21T21:41:10.686 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.012841594, 2024-09-21T21:41:10.687 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:10.687 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:10.687 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:10.687 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:10.687 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:10.687 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:10.687 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1704 2024-09-21T21:41:10.687 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.687 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:10.687 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.687 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:10.687 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382652+0000", 2024-09-21T21:41:10.687 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:10.688 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.688 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.688 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:10.688 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382652+0000", 2024-09-21T21:41:10.688 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:10.688 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.688 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.688 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:10.688 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382653+0000", 2024-09-21T21:41:10.688 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1600000000000002e-07 2024-09-21T21:41:10.688 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.688 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.688 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:10.688 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382655+0000", 2024-09-21T21:41:10.688 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1449999999999998e-06 2024-09-21T21:41:10.688 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.689 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.689 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:10.689 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382661+0000", 2024-09-21T21:41:10.689 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3779999999999998e-06 2024-09-21T21:41:10.689 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.689 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.689 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:10.689 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382667+0000", 2024-09-21T21:41:10.689 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1329999999999998e-06 2024-09-21T21:41:10.689 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.689 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.689 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:10.689 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.395334+0000", 2024-09-21T21:41:10.689 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.012667928 2024-09-21T21:41:10.689 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.690 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.690 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:10.690 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.395418+0000", 2024-09-21T21:41:10.690 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.4046000000000001e-05 2024-09-21T21:41:10.690 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.690 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.690 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:10.690 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.395494+0000", 2024-09-21T21:41:10.690 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5548000000000002e-05 2024-09-21T21:41:10.690 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:10.690 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:10.690 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:10.690 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.690 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.690 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:1706 3.8 3:1339e2f0:::rbd_data.112145fae779.0000000000000007:a [read 10711040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:10.691 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.382673+0000", 2024-09-21T21:41:10.691 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 12.301039292, 2024-09-21T21:41:10.691 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013021824, 2024-09-21T21:41:10.691 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:10.691 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:10.691 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:10.691 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:10.691 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:10.691 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:10.691 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1706 2024-09-21T21:41:10.691 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.691 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:10.691 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.692 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:10.692 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382673+0000", 2024-09-21T21:41:10.692 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:10.692 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.692 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.692 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:10.692 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382673+0000", 2024-09-21T21:41:10.692 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:10.692 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.692 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.692 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:10.692 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382673+0000", 2024-09-21T21:41:10.692 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.01e-07 2024-09-21T21:41:10.692 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.692 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.693 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:10.693 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382675+0000", 2024-09-21T21:41:10.693 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.8959999999999999e-06 2024-09-21T21:41:10.693 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.693 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.693 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:10.693 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382681+0000", 2024-09-21T21:41:10.693 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4309999999999999e-06 2024-09-21T21:41:10.693 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.693 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.693 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:10.693 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382687+0000", 2024-09-21T21:41:10.693 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2229999999999998e-06 2024-09-21T21:41:10.693 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.693 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.693 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:10.694 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.395537+0000", 2024-09-21T21:41:10.694 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.012850225 2024-09-21T21:41:10.694 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.694 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.694 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:10.694 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.395622+0000", 2024-09-21T21:41:10.694 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.4856000000000002e-05 2024-09-21T21:41:10.694 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.694 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.694 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:10.694 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.395694+0000", 2024-09-21T21:41:10.694 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.2792000000000004e-05 2024-09-21T21:41:10.694 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:10.694 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:10.695 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:10.695 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.695 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.695 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:1886 3.8 3:1339e2f0:::rbd_data.112145fae779.0000000000000007:a [read 10424320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:10.695 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.383956+0000", 2024-09-21T21:41:10.695 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 12.299756005000001, 2024-09-21T21:41:10.695 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.012878921, 2024-09-21T21:41:10.695 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:10.695 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:10.695 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:10.695 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:10.695 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:10.695 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:10.695 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1886 2024-09-21T21:41:10.695 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.696 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:10.696 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.696 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:10.696 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383956+0000", 2024-09-21T21:41:10.696 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:10.696 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.696 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.696 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:10.696 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383956+0000", 2024-09-21T21:41:10.696 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:10.696 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.696 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.696 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:10.696 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383956+0000", 2024-09-21T21:41:10.696 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.27e-07 2024-09-21T21:41:10.697 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.697 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.697 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:10.697 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383959+0000", 2024-09-21T21:41:10.697 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.5170000000000002e-06 2024-09-21T21:41:10.697 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.697 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.697 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:10.697 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383965+0000", 2024-09-21T21:41:10.697 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3149999999999997e-06 2024-09-21T21:41:10.697 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.697 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.697 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:10.698 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383971+0000", 2024-09-21T21:41:10.698 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3900000000000001e-06 2024-09-21T21:41:10.698 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.698 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.698 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:10.698 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.396727+0000", 2024-09-21T21:41:10.698 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.012756226000000001 2024-09-21T21:41:10.698 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.698 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.698 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:10.698 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.396777+0000", 2024-09-21T21:41:10.698 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0419e-05 2024-09-21T21:41:10.698 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.699 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.699 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:10.699 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.396835+0000", 2024-09-21T21:41:10.699 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7627000000000002e-05 2024-09-21T21:41:10.699 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:10.699 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:10.699 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:10.699 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.699 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.699 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:1888 3.8 3:1339e2f0:::rbd_data.112145fae779.0000000000000007:a [read 10481664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:10.699 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.383992+0000", 2024-09-21T21:41:10.699 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 12.299719885, 2024-09-21T21:41:10.699 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013052779, 2024-09-21T21:41:10.699 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:10.700 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:10.700 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:10.700 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:10.700 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:10.700 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:10.700 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1888 2024-09-21T21:41:10.700 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.700 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:10.700 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.700 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:10.700 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383992+0000", 2024-09-21T21:41:10.700 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:10.700 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.701 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.701 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:10.701 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383992+0000", 2024-09-21T21:41:10.701 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:10.701 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.701 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.701 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:10.701 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383992+0000", 2024-09-21T21:41:10.701 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4200000000000002e-07 2024-09-21T21:41:10.701 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.701 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.701 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:10.701 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383994+0000", 2024-09-21T21:41:10.701 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9e-06 2024-09-21T21:41:10.702 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.702 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.702 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:10.702 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.384001+0000", 2024-09-21T21:41:10.702 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9439999999999999e-06 2024-09-21T21:41:10.702 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.702 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.702 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:10.702 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.384005+0000", 2024-09-21T21:41:10.702 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4620000000000001e-06 2024-09-21T21:41:10.702 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.702 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.702 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:10.702 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.396876+0000", 2024-09-21T21:41:10.703 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.012871825999999999 2024-09-21T21:41:10.703 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.703 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.703 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:10.703 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.396971+0000", 2024-09-21T21:41:10.703 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.4378999999999999e-05 2024-09-21T21:41:10.703 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.703 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.703 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:10.703 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.397045+0000", 2024-09-21T21:41:10.703 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.3925999999999997e-05 2024-09-21T21:41:10.703 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:10.703 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:10.704 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:10.704 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.704 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.704 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:1890 3.8 3:1339e2f0:::rbd_data.112145fae779.0000000000000007:a [read 10514432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:10.704 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.384011+0000", 2024-09-21T21:41:10.704 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 12.299701373, 2024-09-21T21:41:10.704 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013260717, 2024-09-21T21:41:10.704 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:10.704 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:10.704 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:10.704 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:10.704 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:10.704 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:10.704 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1890 2024-09-21T21:41:10.704 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.705 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:10.705 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.705 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:10.705 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.384011+0000", 2024-09-21T21:41:10.705 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:10.705 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.705 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.705 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:10.705 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.384011+0000", 2024-09-21T21:41:10.705 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:10.705 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.705 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.705 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:10.705 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.384011+0000", 2024-09-21T21:41:10.706 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.7700000000000001e-07 2024-09-21T21:41:10.706 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.706 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.706 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:10.706 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.384012+0000", 2024-09-21T21:41:10.706 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.3740000000000001e-06 2024-09-21T21:41:10.706 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.706 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.706 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:10.706 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.384015+0000", 2024-09-21T21:41:10.706 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3129999999999999e-06 2024-09-21T21:41:10.706 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.706 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.706 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:10.707 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.384018+0000", 2024-09-21T21:41:10.707 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.6620000000000001e-06 2024-09-21T21:41:10.707 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.707 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.707 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:10.707 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.397089+0000", 2024-09-21T21:41:10.707 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013070724000000001 2024-09-21T21:41:10.707 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.707 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.707 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:10.707 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.397191+0000", 2024-09-21T21:41:10.707 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010187399999999999 2024-09-21T21:41:10.707 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.708 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.708 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:10.708 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.397271+0000", 2024-09-21T21:41:10.708 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0493000000000003e-05 2024-09-21T21:41:10.708 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:10.708 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:10.708 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:10.708 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.708 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.708 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:1892 3.8 3:1339e2f0:::rbd_data.112145fae779.0000000000000007:a [read 10571776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:10.708 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.384023+0000", 2024-09-21T21:41:10.708 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 12.299689134999999, 2024-09-21T21:41:10.708 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013380702, 2024-09-21T21:41:10.708 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:10.708 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:10.709 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:10.709 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:10.709 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:10.709 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:10.709 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1892 2024-09-21T21:41:10.709 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.709 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:10.709 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.709 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:10.709 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.384023+0000", 2024-09-21T21:41:10.709 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:10.709 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.709 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.709 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:10.709 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.384023+0000", 2024-09-21T21:41:10.710 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:10.710 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.710 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.710 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:10.710 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.384023+0000", 2024-09-21T21:41:10.710 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.1800000000000002e-07 2024-09-21T21:41:10.710 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.710 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.710 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:10.710 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.384024+0000", 2024-09-21T21:41:10.710 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.2300000000000001e-06 2024-09-21T21:41:10.710 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.710 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.710 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:10.710 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.384028+0000", 2024-09-21T21:41:10.711 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.2990000000000001e-06 2024-09-21T21:41:10.711 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.711 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.711 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:10.711 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.384030+0000", 2024-09-21T21:41:10.711 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.5000000000000002e-06 2024-09-21T21:41:10.711 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.711 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.711 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:10.711 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.397311+0000", 2024-09-21T21:41:10.711 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013280760000000001 2024-09-21T21:41:10.711 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.711 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.711 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:10.711 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.397359+0000", 2024-09-21T21:41:10.711 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7707999999999998e-05 2024-09-21T21:41:10.712 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.712 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.712 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:10.712 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.397403+0000", 2024-09-21T21:41:10.712 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4886999999999999e-05 2024-09-21T21:41:10.712 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:10.712 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:10.712 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:10.712 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.712 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.712 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:1894 3.8 3:1339e2f0:::rbd_data.112145fae779.0000000000000007:a [read 10661888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:10.712 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.384157+0000", 2024-09-21T21:41:10.712 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 12.299554881000001, 2024-09-21T21:41:10.712 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013377534999999999, 2024-09-21T21:41:10.713 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:10.713 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:10.713 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:10.713 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:10.713 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:10.713 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:10.713 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1894 2024-09-21T21:41:10.713 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.713 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:10.713 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.713 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:10.713 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.384157+0000", 2024-09-21T21:41:10.713 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:10.713 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.714 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.714 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:10.714 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.384157+0000", 2024-09-21T21:41:10.714 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:10.714 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.714 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.714 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:10.714 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.384158+0000", 2024-09-21T21:41:10.714 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2099999999999995e-07 2024-09-21T21:41:10.714 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.714 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.714 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:10.714 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.384163+0000", 2024-09-21T21:41:10.714 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8350000000000003e-06 2024-09-21T21:41:10.715 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.715 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.715 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:10.715 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.384176+0000", 2024-09-21T21:41:10.715 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.3286e-05 2024-09-21T21:41:10.715 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.715 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.715 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:10.715 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.384183+0000", 2024-09-21T21:41:10.715 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.357e-06 2024-09-21T21:41:10.715 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.715 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.715 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:10.715 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.397431+0000", 2024-09-21T21:41:10.716 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013247649 2024-09-21T21:41:10.716 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.716 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.716 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:10.716 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.397486+0000", 2024-09-21T21:41:10.716 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4747000000000003e-05 2024-09-21T21:41:10.716 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.716 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.716 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:10.716 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.397535+0000", 2024-09-21T21:41:10.716 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8739999999999998e-05 2024-09-21T21:41:10.716 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:10.716 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:10.716 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:10.716 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.717 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.717 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:1897 3.8 3:1339e2f0:::rbd_data.112145fae779.0000000000000007:a [read 10719232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:10.717 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.384298+0000", 2024-09-21T21:41:10.717 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 12.29941361, 2024-09-21T21:41:10.717 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013351722, 2024-09-21T21:41:10.717 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:10.717 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:10.717 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:10.717 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:10.717 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:10.717 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:10.717 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1897 2024-09-21T21:41:10.717 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.717 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:10.718 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.718 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:10.718 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.384298+0000", 2024-09-21T21:41:10.718 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:10.718 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.718 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.718 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:10.718 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.384298+0000", 2024-09-21T21:41:10.718 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:10.718 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.718 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.718 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:10.718 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.384299+0000", 2024-09-21T21:41:10.718 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.9999999999999996e-07 2024-09-21T21:41:10.719 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.719 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.719 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:10.719 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.384303+0000", 2024-09-21T21:41:10.719 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2270000000000004e-06 2024-09-21T21:41:10.719 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.719 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.719 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:10.719 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.384313+0000", 2024-09-21T21:41:10.719 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.5489999999999995e-06 2024-09-21T21:41:10.719 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.719 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.719 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:10.719 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.384319+0000", 2024-09-21T21:41:10.719 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4930000000000003e-06 2024-09-21T21:41:10.720 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.720 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.720 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:10.720 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.397561+0000", 2024-09-21T21:41:10.720 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013241493999999999 2024-09-21T21:41:10.720 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.720 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.720 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:10.720 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.397606+0000", 2024-09-21T21:41:10.720 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5473000000000003e-05 2024-09-21T21:41:10.720 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.720 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.720 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:10.720 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.397650+0000", 2024-09-21T21:41:10.720 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3686000000000002e-05 2024-09-21T21:41:10.720 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:10.721 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:10.721 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:10.721 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.721 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.721 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:6258 3.8 3:1339e2f0:::rbd_data.112145fae779.0000000000000007:a [read 12398592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:41:10.721 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.593604+0000", 2024-09-21T21:41:10.721 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 12.090107873000001, 2024-09-21T21:41:10.721 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.012967548000000001, 2024-09-21T21:41:10.721 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:10.721 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:10.721 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:10.721 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:10.721 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:10.721 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:10.722 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 6258 2024-09-21T21:41:10.722 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.722 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:10.722 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.722 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:10.722 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.593604+0000", 2024-09-21T21:41:10.722 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:10.722 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.722 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.722 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:10.722 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.593604+0000", 2024-09-21T21:41:10.722 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:10.722 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.722 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.722 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:10.723 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.593604+0000", 2024-09-21T21:41:10.723 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.9200000000000002e-07 2024-09-21T21:41:10.723 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.723 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.723 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:10.723 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.593611+0000", 2024-09-21T21:41:10.723 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2249999999999997e-06 2024-09-21T21:41:10.723 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.723 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.723 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:10.723 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.593617+0000", 2024-09-21T21:41:10.723 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2129999999999996e-06 2024-09-21T21:41:10.723 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.723 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.724 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:10.724 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.593622+0000", 2024-09-21T21:41:10.724 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7430000000000002e-06 2024-09-21T21:41:10.724 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.724 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.724 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:10.724 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.606399+0000", 2024-09-21T21:41:10.724 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.012776984999999999 2024-09-21T21:41:10.724 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.724 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.724 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:10.724 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.606488+0000", 2024-09-21T21:41:10.724 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.8918000000000005e-05 2024-09-21T21:41:10.724 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.724 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.725 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:10.725 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.606572+0000", 2024-09-21T21:41:10.725 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.4072000000000004e-05 2024-09-21T21:41:10.725 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:10.725 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:10.725 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:10.725 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.725 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.725 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:6378 3.8 3:1339e2f0:::rbd_data.112145fae779.0000000000000007:a [read 11960320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:41:10.725 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.594157+0000", 2024-09-21T21:41:10.725 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 12.089554731, 2024-09-21T21:41:10.725 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.012907642, 2024-09-21T21:41:10.725 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:10.725 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:10.725 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:10.725 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:10.725 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:10.726 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:10.726 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 6378 2024-09-21T21:41:10.726 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.726 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:10.726 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.726 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:10.726 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594157+0000", 2024-09-21T21:41:10.726 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:10.726 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.726 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.726 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:10.726 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594157+0000", 2024-09-21T21:41:10.726 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:10.726 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.726 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.726 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:10.727 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594159+0000", 2024-09-21T21:41:10.727 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.939e-06 2024-09-21T21:41:10.727 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.727 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.727 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:10.727 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594162+0000", 2024-09-21T21:41:10.727 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.8849999999999999e-06 2024-09-21T21:41:10.727 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.727 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.727 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:10.727 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594169+0000", 2024-09-21T21:41:10.727 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9190000000000002e-06 2024-09-21T21:41:10.727 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.727 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.728 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:10.728 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594174+0000", 2024-09-21T21:41:10.728 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.22e-06 2024-09-21T21:41:10.728 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.728 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.728 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:10.728 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.606959+0000", 2024-09-21T21:41:10.728 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.012784644 2024-09-21T21:41:10.728 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.728 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.728 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:10.728 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.607003+0000", 2024-09-21T21:41:10.728 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4067999999999999e-05 2024-09-21T21:41:10.728 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.728 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.729 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:10.729 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.607065+0000", 2024-09-21T21:41:10.729 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1966999999999998e-05 2024-09-21T21:41:10.729 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:10.729 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:10.729 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:10.729 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.729 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.729 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:6379 3.8 3:1339e2f0:::rbd_data.112145fae779.0000000000000007:a [read 12017664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:41:10.729 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.594186+0000", 2024-09-21T21:41:10.729 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 12.089526332, 2024-09-21T21:41:10.729 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013082819000000001, 2024-09-21T21:41:10.730 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:10.730 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:10.730 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:10.730 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:10.730 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:10.730 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:10.730 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 6379 2024-09-21T21:41:10.730 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.730 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:10.730 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.730 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:10.730 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594186+0000", 2024-09-21T21:41:10.730 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:10.730 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.731 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.731 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:10.731 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594186+0000", 2024-09-21T21:41:10.731 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:10.731 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.731 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.731 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:10.731 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594186+0000", 2024-09-21T21:41:10.731 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.39e-07 2024-09-21T21:41:10.731 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.731 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.731 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:10.731 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594188+0000", 2024-09-21T21:41:10.731 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3159999999999999e-06 2024-09-21T21:41:10.731 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.732 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.732 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:10.732 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594194+0000", 2024-09-21T21:41:10.732 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.088e-06 2024-09-21T21:41:10.732 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.732 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.732 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:10.732 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594199+0000", 2024-09-21T21:41:10.732 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8729999999999998e-06 2024-09-21T21:41:10.732 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.732 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.732 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:10.732 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.607102+0000", 2024-09-21T21:41:10.732 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.012902521 2024-09-21T21:41:10.733 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.733 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.733 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:10.733 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.607185+0000", 2024-09-21T21:41:10.733 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.2779000000000001e-05 2024-09-21T21:41:10.733 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.733 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.733 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:10.733 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.607268+0000", 2024-09-21T21:41:10.733 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.3802999999999997e-05 2024-09-21T21:41:10.733 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:10.733 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:10.733 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:10.733 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.733 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.734 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:6380 3.8 3:1339e2f0:::rbd_data.112145fae779.0000000000000007:a [read 12107776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:41:10.734 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.594206+0000", 2024-09-21T21:41:10.734 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 12.089506070000001, 2024-09-21T21:41:10.734 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.01319447, 2024-09-21T21:41:10.734 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:10.734 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:10.734 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:10.734 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:10.734 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:10.734 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:10.734 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 6380 2024-09-21T21:41:10.734 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.734 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:10.734 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.734 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:10.735 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594206+0000", 2024-09-21T21:41:10.735 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:10.735 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.735 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.735 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:10.735 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594206+0000", 2024-09-21T21:41:10.735 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:10.735 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.735 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.735 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:10.735 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594206+0000", 2024-09-21T21:41:10.735 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8099999999999998e-07 2024-09-21T21:41:10.735 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.735 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.736 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:10.736 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594208+0000", 2024-09-21T21:41:10.736 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9479999999999998e-06 2024-09-21T21:41:10.736 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.736 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.736 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:10.736 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594214+0000", 2024-09-21T21:41:10.736 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7409999999999998e-06 2024-09-21T21:41:10.736 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.736 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.736 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:10.736 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594218+0000", 2024-09-21T21:41:10.736 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3830000000000002e-06 2024-09-21T21:41:10.736 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.736 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.737 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:10.737 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.607298+0000", 2024-09-21T21:41:10.737 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013079856000000001 2024-09-21T21:41:10.737 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.737 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.737 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:10.737 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.607341+0000", 2024-09-21T21:41:10.737 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3118999999999998e-05 2024-09-21T21:41:10.737 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.737 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.737 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:10.737 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.607400+0000", 2024-09-21T21:41:10.737 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9042000000000002e-05 2024-09-21T21:41:10.737 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:10.738 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:10.738 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:10.738 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.738 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.738 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:6381 3.8 3:1339e2f0:::rbd_data.112145fae779.0000000000000007:a [read 12165120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:41:10.738 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.594224+0000", 2024-09-21T21:41:10.738 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 12.089487498, 2024-09-21T21:41:10.738 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013387049999999999, 2024-09-21T21:41:10.738 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:10.738 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:10.738 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:10.738 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:10.738 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:10.738 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:10.738 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 6381 2024-09-21T21:41:10.739 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.739 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:10.739 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.739 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:10.739 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594224+0000", 2024-09-21T21:41:10.739 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:10.739 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.739 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.739 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:10.739 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594224+0000", 2024-09-21T21:41:10.739 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:10.739 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.739 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.739 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:10.740 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594225+0000", 2024-09-21T21:41:10.740 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1699999999999999e-07 2024-09-21T21:41:10.740 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.740 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.740 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:10.740 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594227+0000", 2024-09-21T21:41:10.740 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1710000000000001e-06 2024-09-21T21:41:10.740 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.740 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.740 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:10.740 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594233+0000", 2024-09-21T21:41:10.740 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.631e-06 2024-09-21T21:41:10.740 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.740 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.741 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:10.741 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594237+0000", 2024-09-21T21:41:10.741 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4989999999999996e-06 2024-09-21T21:41:10.741 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.741 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.741 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:10.741 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.607438+0000", 2024-09-21T21:41:10.741 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013200788999999999 2024-09-21T21:41:10.741 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.741 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.741 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:10.741 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.607526+0000", 2024-09-21T21:41:10.741 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.7975999999999999e-05 2024-09-21T21:41:10.741 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.742 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.742 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:10.742 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.607611+0000", 2024-09-21T21:41:10.742 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.5567000000000003e-05 2024-09-21T21:41:10.742 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:10.742 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:10.742 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:10.742 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.742 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.742 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:6382 3.8 3:1339e2f0:::rbd_data.112145fae779.0000000000000007:a [read 12197888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:41:10.742 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.594251+0000", 2024-09-21T21:41:10.742 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 12.089461258, 2024-09-21T21:41:10.742 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.01350056, 2024-09-21T21:41:10.742 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:10.743 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:10.743 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:10.743 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:10.743 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:10.743 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:10.743 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 6382 2024-09-21T21:41:10.743 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.743 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:10.743 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.743 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:10.743 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594251+0000", 2024-09-21T21:41:10.743 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:10.743 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.743 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.744 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:10.744 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594251+0000", 2024-09-21T21:41:10.744 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:10.744 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.744 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.744 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:10.744 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594251+0000", 2024-09-21T21:41:10.744 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4000000000000002e-07 2024-09-21T21:41:10.744 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.744 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.744 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:10.744 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594253+0000", 2024-09-21T21:41:10.744 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2500000000000001e-06 2024-09-21T21:41:10.745 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.745 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.745 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:10.745 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594259+0000", 2024-09-21T21:41:10.745 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7559999999999996e-06 2024-09-21T21:41:10.745 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.745 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.745 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:10.745 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594264+0000", 2024-09-21T21:41:10.745 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5719999999999996e-06 2024-09-21T21:41:10.745 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.745 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.745 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:10.745 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.607642+0000", 2024-09-21T21:41:10.745 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013378750999999999 2024-09-21T21:41:10.746 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.746 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.746 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:10.746 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.607690+0000", 2024-09-21T21:41:10.746 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7632000000000001e-05 2024-09-21T21:41:10.746 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.746 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.746 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:10.746 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.607751+0000", 2024-09-21T21:41:10.746 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1159000000000002e-05 2024-09-21T21:41:10.746 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:10.746 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:10.746 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:10.746 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.747 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.747 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:6383 3.8 3:1339e2f0:::rbd_data.112145fae779.0000000000000007:a [read 12255232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:41:10.747 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.594270+0000", 2024-09-21T21:41:10.747 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 12.089441891, 2024-09-21T21:41:10.747 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013702288999999999, 2024-09-21T21:41:10.747 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:10.747 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:10.747 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:10.747 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:10.747 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:10.747 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:10.747 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 6383 2024-09-21T21:41:10.747 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.748 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:10.748 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.748 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:10.748 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594270+0000", 2024-09-21T21:41:10.748 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:10.748 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.748 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.748 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:10.748 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594270+0000", 2024-09-21T21:41:10.748 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:10.748 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.748 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.748 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:10.748 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594270+0000", 2024-09-21T21:41:10.749 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8799999999999998e-07 2024-09-21T21:41:10.749 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.749 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.749 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:10.749 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594272+0000", 2024-09-21T21:41:10.749 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.995e-06 2024-09-21T21:41:10.749 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.749 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.749 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:10.749 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594278+0000", 2024-09-21T21:41:10.749 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5459999999999999e-06 2024-09-21T21:41:10.749 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.749 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.749 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:10.750 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594283+0000", 2024-09-21T21:41:10.750 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3739999999999997e-06 2024-09-21T21:41:10.750 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.750 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.750 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:10.750 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.607790+0000", 2024-09-21T21:41:10.750 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013506827000000001 2024-09-21T21:41:10.750 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.750 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.750 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:10.750 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.607880+0000", 2024-09-21T21:41:10.750 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.9647000000000004e-05 2024-09-21T21:41:10.750 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.751 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.751 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:10.751 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.607972+0000", 2024-09-21T21:41:10.751 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2511999999999998e-05 2024-09-21T21:41:10.751 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:10.751 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:10.751 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:10.751 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.751 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.751 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:6384 3.8 3:1339e2f0:::rbd_data.112145fae779.0000000000000007:a [read 12312576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:41:10.751 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.594289+0000", 2024-09-21T21:41:10.751 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 12.089422831, 2024-09-21T21:41:10.751 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.01381668, 2024-09-21T21:41:10.751 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:10.751 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:10.752 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:10.752 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:10.752 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:10.752 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:10.752 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 6384 2024-09-21T21:41:10.752 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.752 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:10.752 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.752 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:10.752 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594289+0000", 2024-09-21T21:41:10.752 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:10.752 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.752 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.752 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:10.753 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594289+0000", 2024-09-21T21:41:10.753 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:10.753 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.753 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.753 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:10.753 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594289+0000", 2024-09-21T21:41:10.753 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8000000000000001e-07 2024-09-21T21:41:10.753 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.753 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.753 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:10.753 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594291+0000", 2024-09-21T21:41:10.753 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.8899999999999999e-06 2024-09-21T21:41:10.753 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.753 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.753 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:10.754 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594297+0000", 2024-09-21T21:41:10.754 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7230000000000001e-06 2024-09-21T21:41:10.754 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.754 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.754 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:10.754 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594302+0000", 2024-09-21T21:41:10.754 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4560000000000002e-06 2024-09-21T21:41:10.754 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.754 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.754 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:10.754 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.608001+0000", 2024-09-21T21:41:10.754 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013699742000000001 2024-09-21T21:41:10.754 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.755 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.755 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:10.755 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.608046+0000", 2024-09-21T21:41:10.755 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4925000000000001e-05 2024-09-21T21:41:10.755 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.755 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.755 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:10.755 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.608106+0000", 2024-09-21T21:41:10.755 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9564000000000001e-05 2024-09-21T21:41:10.755 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:10.755 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:10.755 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:10.755 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.756 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.756 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:6385 3.8 3:1339e2f0:::rbd_data.112145fae779.0000000000000007:a [read 12345344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:41:10.756 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.594307+0000", 2024-09-21T21:41:10.756 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 12.089404663, 2024-09-21T21:41:10.756 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.014004925, 2024-09-21T21:41:10.756 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:10.756 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:10.756 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:10.756 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:10.756 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:10.756 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:10.756 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 6385 2024-09-21T21:41:10.756 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.756 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:10.757 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.757 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:10.757 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594307+0000", 2024-09-21T21:41:10.757 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:10.757 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.757 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.757 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:10.757 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594307+0000", 2024-09-21T21:41:10.757 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:10.757 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.757 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.757 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:10.757 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594308+0000", 2024-09-21T21:41:10.757 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8500000000000002e-07 2024-09-21T21:41:10.758 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.758 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.758 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:10.758 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594310+0000", 2024-09-21T21:41:10.758 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9300000000000002e-06 2024-09-21T21:41:10.758 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.758 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.758 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:10.758 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594315+0000", 2024-09-21T21:41:10.758 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.626e-06 2024-09-21T21:41:10.758 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.758 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.758 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:10.758 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594320+0000", 2024-09-21T21:41:10.758 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5079999999999999e-06 2024-09-21T21:41:10.759 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.759 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.759 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:10.759 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.608141+0000", 2024-09-21T21:41:10.759 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013821787 2024-09-21T21:41:10.759 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.759 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.759 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:10.759 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.608229+0000", 2024-09-21T21:41:10.759 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.7670000000000006e-05 2024-09-21T21:41:10.759 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.759 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.759 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:10.759 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.608312+0000", 2024-09-21T21:41:10.760 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.3018999999999999e-05 2024-09-21T21:41:10.760 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:10.760 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:10.760 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:10.760 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.760 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.760 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:6386 3.8 3:1339e2f0:::rbd_data.112145fae779.0000000000000007:a [read 12402688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:41:10.760 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.594325+0000", 2024-09-21T21:41:10.760 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 12.089386725000001, 2024-09-21T21:41:10.760 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.014122307000000001, 2024-09-21T21:41:10.760 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:10.760 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:10.760 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:10.761 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:10.761 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:10.761 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:10.761 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 6386 2024-09-21T21:41:10.761 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.761 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:10.761 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.761 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:10.761 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594325+0000", 2024-09-21T21:41:10.761 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:10.761 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.761 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.761 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:10.761 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594325+0000", 2024-09-21T21:41:10.761 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:10.762 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.762 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.762 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:10.762 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594326+0000", 2024-09-21T21:41:10.762 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8099999999999998e-07 2024-09-21T21:41:10.762 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.762 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.762 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:10.762 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594328+0000", 2024-09-21T21:41:10.762 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0200000000000001e-06 2024-09-21T21:41:10.762 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.762 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.762 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:10.762 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594333+0000", 2024-09-21T21:41:10.762 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6119999999999998e-06 2024-09-21T21:41:10.763 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.763 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.763 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:10.763 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594338+0000", 2024-09-21T21:41:10.763 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3359999999999997e-06 2024-09-21T21:41:10.763 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.763 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.763 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:10.763 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.608346+0000", 2024-09-21T21:41:10.763 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.014008909999999999 2024-09-21T21:41:10.763 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.763 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.763 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:10.763 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.608388+0000", 2024-09-21T21:41:10.764 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1943000000000001e-05 2024-09-21T21:41:10.764 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.764 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.764 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:10.764 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.608447+0000", 2024-09-21T21:41:10.764 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9104999999999997e-05 2024-09-21T21:41:10.764 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:10.764 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:10.764 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:10.764 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.764 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.764 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:6391 3.18 3:199efa5f:::rbd_data.1121e540c096.000000000000015c:head [set-alloc-hint object_size 524288 write_size 524288,write 8192~3584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:41:10.764 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.594343+0000", 2024-09-21T21:41:10.764 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 12.089368686, 2024-09-21T21:41:10.765 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.016432473, 2024-09-21T21:41:10.765 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:10.765 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:10.765 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:41:10.765 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:10.765 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:10.765 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:10.765 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 6391 2024-09-21T21:41:10.765 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.765 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:10.765 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.765 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:10.765 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594343+0000", 2024-09-21T21:41:10.765 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:10.765 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.766 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.766 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:10.766 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594343+0000", 2024-09-21T21:41:10.766 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:10.766 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.766 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.766 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:10.766 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594344+0000", 2024-09-21T21:41:10.766 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4499999999999997e-07 2024-09-21T21:41:10.766 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.766 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.766 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:10.766 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594363+0000", 2024-09-21T21:41:10.766 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9048000000000001e-05 2024-09-21T21:41:10.767 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.767 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.767 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:10.767 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594370+0000", 2024-09-21T21:41:10.767 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.7989999999999996e-06 2024-09-21T21:41:10.767 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.767 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.767 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:10.767 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.594376+0000", 2024-09-21T21:41:10.767 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9829999999999999e-06 2024-09-21T21:41:10.767 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.767 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.767 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:10.767 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.608344+0000", 2024-09-21T21:41:10.767 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013967319000000001 2024-09-21T21:41:10.768 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.768 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.768 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:10.768 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.608490+0000", 2024-09-21T21:41:10.768 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000145802 2024-09-21T21:41:10.768 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.768 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.768 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 0", 2024-09-21T21:41:10.768 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.608660+0000", 2024-09-21T21:41:10.768 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000170194 2024-09-21T21:41:10.768 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.768 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.768 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:41:10.768 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.610174+0000", 2024-09-21T21:41:10.769 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0015143279999999999 2024-09-21T21:41:10.769 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.769 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.769 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:41:10.769 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.610701+0000", 2024-09-21T21:41:10.769 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00052702499999999998 2024-09-21T21:41:10.769 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.769 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.769 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:41:10.769 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.610740+0000", 2024-09-21T21:41:10.769 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.9168000000000002e-05 2024-09-21T21:41:10.769 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.769 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.769 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:10.769 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.610776+0000", 2024-09-21T21:41:10.770 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5361999999999998e-05 2024-09-21T21:41:10.770 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:10.770 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:10.770 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:10.770 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.770 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.770 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4824.0:1854 3.1e 3:7cbf3d9a:::rbd_data.1121b97f2925.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 4194304 write_size 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:10.770 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:01.700058+0000", 2024-09-21T21:41:10.770 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 8.9836541990000001, 2024-09-21T21:41:10.770 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013123351, 2024-09-21T21:41:10.770 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:10.770 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:10.770 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:41:10.770 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:10.771 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4824", 2024-09-21T21:41:10.771 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:10.771 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1854 2024-09-21T21:41:10.771 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.771 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:10.771 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.771 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:10.771 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.700058+0000", 2024-09-21T21:41:10.771 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:10.771 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.771 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.771 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:10.771 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.700058+0000", 2024-09-21T21:41:10.771 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:10.772 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.772 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.772 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:10.772 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.700060+0000", 2024-09-21T21:41:10.772 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.7199999999999998e-06 2024-09-21T21:41:10.772 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.772 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.772 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:10.772 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.704571+0000", 2024-09-21T21:41:10.772 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0045108259999999999 2024-09-21T21:41:10.772 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.772 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.772 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:10.772 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.704584+0000", 2024-09-21T21:41:10.773 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.2337000000000001e-05 2024-09-21T21:41:10.773 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.773 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.773 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:10.773 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.704590+0000", 2024-09-21T21:41:10.773 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0870000000000003e-06 2024-09-21T21:41:10.773 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.773 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.773 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:10.773 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.704636+0000", 2024-09-21T21:41:10.773 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6483000000000002e-05 2024-09-21T21:41:10.773 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.773 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.773 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:10.773 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.704690+0000", 2024-09-21T21:41:10.774 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3634999999999997e-05 2024-09-21T21:41:10.774 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.774 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.774 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 3", 2024-09-21T21:41:10.774 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.705008+0000", 2024-09-21T21:41:10.774 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00031831600000000002 2024-09-21T21:41:10.774 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.774 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.774 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:41:10.774 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.710076+0000", 2024-09-21T21:41:10.774 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0050678119999999997 2024-09-21T21:41:10.774 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.774 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.774 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:41:10.775 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.713082+0000", 2024-09-21T21:41:10.775 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0030060389999999998 2024-09-21T21:41:10.775 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.775 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.775 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:41:10.775 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.713140+0000", 2024-09-21T21:41:10.775 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7615000000000003e-05 2024-09-21T21:41:10.775 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:10.775 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:10.775 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:10.775 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.713181+0000", 2024-09-21T21:41:10.775 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1480999999999998e-05 2024-09-21T21:41:10.775 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:10.775 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:10.776 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:10.776 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:10.776 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:10.776 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:10.777 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:10.775+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:10.778 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T21:41:10.779 INFO:teuthology.orchestra.run.smithi164.stdout:2074 read 0x5fee00 thru 0x60dc6f (0xee70 bytes) 2024-09-21T21:41:10.779 INFO:teuthology.orchestra.run.smithi164.stdout:2075 trunc from 0x3866400 to 0x6c81400 2024-09-21T21:41:10.779 INFO:teuthology.orchestra.run.smithi164.stdout:2076 write 0xe11400 thru 0xe1b0f3 (0x9cf4 bytes) 2024-09-21T21:41:10.779 INFO:teuthology.orchestra.run.smithi164.stdout:2077 write 0x92d0400 thru 0x92d5fe0 (0x5be1 bytes) 2024-09-21T21:41:10.779 INFO:teuthology.orchestra.run.smithi164.stdout:2079 punch from 0x352bc00 to 0x3532a00, (0x6e00 bytes) 2024-09-21T21:41:10.779 INFO:teuthology.orchestra.run.smithi164.stdout:2082 write 0x9c6f200 thru 0x9c7d383 (0xe184 bytes) 2024-09-21T21:41:10.779 INFO:teuthology.orchestra.run.smithi164.stdout:2084 write 0x3298a00 thru 0x329ee76 (0x6477 bytes) 2024-09-21T21:41:10.779 INFO:teuthology.orchestra.run.smithi164.stdout:2085 trunc from 0x9c7d400 to 0x41cb800 2024-09-21T21:41:10.779 INFO:teuthology.orchestra.run.smithi164.stdout:2088 read 0x104bc00 thru 0x1059615 (0xda16 bytes) 2024-09-21T21:41:10.795 INFO:teuthology.orchestra.run.smithi164.stdout:2090 write 0xc7efe00 thru 0xc7f2d83 (0x2f84 bytes) 2024-09-21T21:41:10.800 INFO:teuthology.orchestra.run.smithi164.stdout:2093 punch from 0x970e400 to 0x971e200, (0xfe00 bytes) 2024-09-21T21:41:10.802 INFO:teuthology.orchestra.run.smithi164.stdout:2094 punch from 0x7039600 to 0x7048e00, (0xf800 bytes) 2024-09-21T21:41:10.802 INFO:teuthology.orchestra.run.smithi164.stdout:2095 read 0xbbe4800 thru 0xbbef9ac (0xb1ad bytes) 2024-09-21T21:41:10.802 INFO:teuthology.orchestra.run.smithi164.stdout:2097 read 0xbf5b600 thru 0xbf5c3aa (0xdab bytes) 2024-09-21T21:41:10.802 INFO:teuthology.orchestra.run.smithi164.stdout:2099 clone 13 order 18 su 1024 sc 13 2024-09-21T21:41:10.878 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:10.876+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:10.884 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:10.884 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:41:10.884 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-21T21:41:10.884 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:10.892 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T21:41:10.979 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:10.977+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:10.995 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:10.995 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:41:10.995 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:41:10.995 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:41:10.995 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:11.004 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T21:41:11.079 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:11.078+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:11.108 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:11.108 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-21T21:41:11.108 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-21T21:41:11.108 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:41:11.108 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.108 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2415 3.d 3:b26310db:::rbd_data.112145fae779.0000000000000006:a [read 10690560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:11.108 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.406907+0000", 2024-09-21T21:41:11.108 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.699026169, 2024-09-21T21:41:11.108 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044424119999999997, 2024-09-21T21:41:11.108 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:11.108 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:11.109 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:11.109 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:11.109 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:11.109 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:11.109 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2415 2024-09-21T21:41:11.109 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.109 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:11.109 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.109 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:11.109 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.406907+0000", 2024-09-21T21:41:11.109 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:11.109 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.109 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.109 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:11.109 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.406907+0000", 2024-09-21T21:41:11.109 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:11.110 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.110 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.110 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:11.110 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.406908+0000", 2024-09-21T21:41:11.110 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7599999999999998e-07 2024-09-21T21:41:11.110 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.110 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.110 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:11.110 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.406915+0000", 2024-09-21T21:41:11.110 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6729999999999992e-06 2024-09-21T21:41:11.110 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.110 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.110 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:11.110 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.406922+0000", 2024-09-21T21:41:11.111 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7139999999999996e-06 2024-09-21T21:41:11.111 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.111 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.111 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:11.111 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.406927+0000", 2024-09-21T21:41:11.111 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.327e-06 2024-09-21T21:41:11.111 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.111 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.111 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:11.111 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.451179+0000", 2024-09-21T21:41:11.111 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044251984000000001 2024-09-21T21:41:11.111 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.111 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.111 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:11.111 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.451254+0000", 2024-09-21T21:41:11.111 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4703999999999995e-05 2024-09-21T21:41:11.112 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.112 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.112 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:11.112 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.451331+0000", 2024-09-21T21:41:11.112 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7342000000000006e-05 2024-09-21T21:41:11.112 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:11.112 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:11.112 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:11.112 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.112 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.112 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2417 3.d 3:b26310db:::rbd_data.112145fae779.0000000000000006:a [read 10252288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:11.112 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.406934+0000", 2024-09-21T21:41:11.112 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.698999425, 2024-09-21T21:41:11.112 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044511163999999999, 2024-09-21T21:41:11.113 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:11.113 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:11.113 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:11.113 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:11.113 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:11.113 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:11.113 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2417 2024-09-21T21:41:11.113 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.113 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:11.113 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.113 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:11.113 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.406934+0000", 2024-09-21T21:41:11.113 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:11.114 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.114 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.114 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:11.114 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.406934+0000", 2024-09-21T21:41:11.114 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:11.114 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.114 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.114 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:11.114 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.406935+0000", 2024-09-21T21:41:11.114 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6499999999999999e-07 2024-09-21T21:41:11.114 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.114 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.114 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:11.114 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.406937+0000", 2024-09-21T21:41:11.115 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1299999999999999e-06 2024-09-21T21:41:11.115 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.115 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.115 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:11.115 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.406943+0000", 2024-09-21T21:41:11.115 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9989999999999997e-06 2024-09-21T21:41:11.115 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.115 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.115 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:11.115 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.406948+0000", 2024-09-21T21:41:11.115 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9729999999999997e-06 2024-09-21T21:41:11.115 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.115 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.115 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:11.115 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.451365+0000", 2024-09-21T21:41:11.116 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044417808000000003 2024-09-21T21:41:11.116 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.116 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.116 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:11.116 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.451403+0000", 2024-09-21T21:41:11.116 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7944999999999997e-05 2024-09-21T21:41:11.116 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.116 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.116 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:11.116 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.451445+0000", 2024-09-21T21:41:11.116 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1844000000000001e-05 2024-09-21T21:41:11.116 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:11.116 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:11.116 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:11.116 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.117 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.117 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2425 3.d 3:b26310db:::rbd_data.112145fae779.0000000000000006:a [read 10457088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:11.117 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.407639+0000", 2024-09-21T21:41:11.117 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.698294724, 2024-09-21T21:41:11.117 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044288292999999999, 2024-09-21T21:41:11.117 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:11.117 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:11.117 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:11.117 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:11.117 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:11.117 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:11.117 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2425 2024-09-21T21:41:11.117 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.118 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:11.118 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.118 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:11.118 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407639+0000", 2024-09-21T21:41:11.118 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:11.118 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.118 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.118 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:11.118 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407639+0000", 2024-09-21T21:41:11.118 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:11.118 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.118 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.118 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:11.118 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407639+0000", 2024-09-21T21:41:11.119 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8299999999999998e-07 2024-09-21T21:41:11.119 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.119 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.119 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:11.119 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407641+0000", 2024-09-21T21:41:11.119 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9149999999999999e-06 2024-09-21T21:41:11.119 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.119 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.119 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:11.119 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407647+0000", 2024-09-21T21:41:11.119 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3110000000000002e-06 2024-09-21T21:41:11.119 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.119 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.119 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:11.120 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407653+0000", 2024-09-21T21:41:11.120 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1309999999999997e-06 2024-09-21T21:41:11.120 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.120 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.120 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:11.120 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.451778+0000", 2024-09-21T21:41:11.120 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044125782000000002 2024-09-21T21:41:11.120 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.120 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.120 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:11.120 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.451850+0000", 2024-09-21T21:41:11.120 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1211999999999995e-05 2024-09-21T21:41:11.120 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.120 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.120 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:11.121 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.451927+0000", 2024-09-21T21:41:11.121 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7558999999999994e-05 2024-09-21T21:41:11.121 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:11.121 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:11.121 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:11.121 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.121 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.121 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2427 3.d 3:b26310db:::rbd_data.112145fae779.0000000000000006:a [read 10489856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:11.121 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.407659+0000", 2024-09-21T21:41:11.121 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.698274411, 2024-09-21T21:41:11.121 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044463186000000002, 2024-09-21T21:41:11.121 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:11.121 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:11.121 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:11.121 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:11.122 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:11.122 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:11.122 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2427 2024-09-21T21:41:11.122 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.122 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:11.122 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.122 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:11.122 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407659+0000", 2024-09-21T21:41:11.122 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:11.122 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.122 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.122 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:11.122 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407659+0000", 2024-09-21T21:41:11.123 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:11.123 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.123 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.123 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:11.123 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407659+0000", 2024-09-21T21:41:11.123 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3000000000000002e-07 2024-09-21T21:41:11.123 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.123 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.123 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:11.123 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407662+0000", 2024-09-21T21:41:11.123 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0980000000000001e-06 2024-09-21T21:41:11.123 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.123 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.123 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:11.123 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407668+0000", 2024-09-21T21:41:11.123 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5209999999999999e-06 2024-09-21T21:41:11.124 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.124 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.124 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:11.124 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407674+0000", 2024-09-21T21:41:11.124 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7590000000000003e-06 2024-09-21T21:41:11.124 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.124 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.124 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:11.124 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.451969+0000", 2024-09-21T21:41:11.124 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044294965999999998 2024-09-21T21:41:11.124 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.124 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.124 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:11.124 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.452045+0000", 2024-09-21T21:41:11.125 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6271000000000002e-05 2024-09-21T21:41:11.125 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.125 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.125 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:11.125 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.452122+0000", 2024-09-21T21:41:11.125 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7241000000000001e-05 2024-09-21T21:41:11.125 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:11.125 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:11.125 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:11.125 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.125 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.125 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2429 3.d 3:b26310db:::rbd_data.112145fae779.0000000000000006:a [read 10547200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:11.125 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.407681+0000", 2024-09-21T21:41:11.125 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.698252737000001, 2024-09-21T21:41:11.126 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.04455829, 2024-09-21T21:41:11.126 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:11.126 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:11.126 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:11.126 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:11.126 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:11.126 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:11.126 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2429 2024-09-21T21:41:11.126 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.126 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:11.126 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.126 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:11.126 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407681+0000", 2024-09-21T21:41:11.127 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:11.127 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.127 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.127 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:11.127 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407681+0000", 2024-09-21T21:41:11.127 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:11.127 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.127 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.127 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:11.127 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407681+0000", 2024-09-21T21:41:11.127 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2300000000000001e-07 2024-09-21T21:41:11.127 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.127 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.127 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:11.128 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407683+0000", 2024-09-21T21:41:11.128 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0020000000000001e-06 2024-09-21T21:41:11.128 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.128 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.128 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:11.128 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407690+0000", 2024-09-21T21:41:11.128 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4269999999999996e-06 2024-09-21T21:41:11.128 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.128 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.128 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:11.128 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407694+0000", 2024-09-21T21:41:11.128 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6029999999999998e-06 2024-09-21T21:41:11.128 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.128 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.128 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:11.128 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.452153+0000", 2024-09-21T21:41:11.129 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044458232 2024-09-21T21:41:11.129 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.129 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.129 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:11.129 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.452196+0000", 2024-09-21T21:41:11.129 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3248999999999999e-05 2024-09-21T21:41:11.129 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.129 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.129 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:11.129 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.452239+0000", 2024-09-21T21:41:11.129 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3254000000000003e-05 2024-09-21T21:41:11.129 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:11.129 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:11.130 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:11.130 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.130 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.130 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2431 3.d 3:b26310db:::rbd_data.112145fae779.0000000000000006:a [read 10604544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:11.130 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.407701+0000", 2024-09-21T21:41:11.130 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.698232954, 2024-09-21T21:41:11.130 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044639389000000002, 2024-09-21T21:41:11.130 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:11.130 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:11.130 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:11.130 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:11.130 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:11.130 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:11.130 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2431 2024-09-21T21:41:11.131 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.131 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:11.131 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.131 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:11.131 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407701+0000", 2024-09-21T21:41:11.131 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:11.131 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.131 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.131 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:11.131 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407701+0000", 2024-09-21T21:41:11.131 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:11.131 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.131 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.131 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:11.132 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407701+0000", 2024-09-21T21:41:11.132 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.63e-07 2024-09-21T21:41:11.132 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.132 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.132 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:11.132 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407703+0000", 2024-09-21T21:41:11.132 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.979e-06 2024-09-21T21:41:11.132 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.132 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.132 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:11.132 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407710+0000", 2024-09-21T21:41:11.132 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5470000000000002e-06 2024-09-21T21:41:11.132 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.132 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.132 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:11.133 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407715+0000", 2024-09-21T21:41:11.133 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.972e-06 2024-09-21T21:41:11.133 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.133 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.133 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:11.133 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.452266+0000", 2024-09-21T21:41:11.133 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044551201999999998 2024-09-21T21:41:11.133 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.133 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.133 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:11.133 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.452302+0000", 2024-09-21T21:41:11.133 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6516000000000001e-05 2024-09-21T21:41:11.133 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.133 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.134 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:11.134 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.452340+0000", 2024-09-21T21:41:11.134 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7710000000000003e-05 2024-09-21T21:41:11.134 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:11.134 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:11.134 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:11.134 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.134 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.134 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2434 3.d 3:b26310db:::rbd_data.112145fae779.0000000000000006:a [read 10637312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:11.134 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.407720+0000", 2024-09-21T21:41:11.134 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.698213114, 2024-09-21T21:41:11.134 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.04471874, 2024-09-21T21:41:11.134 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:11.134 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:11.134 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:11.135 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:11.135 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:11.135 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:11.135 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2434 2024-09-21T21:41:11.135 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.135 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:11.135 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.135 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:11.135 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407720+0000", 2024-09-21T21:41:11.135 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:11.135 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.135 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.135 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:11.135 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407720+0000", 2024-09-21T21:41:11.136 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:11.136 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.136 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.136 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:11.136 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407721+0000", 2024-09-21T21:41:11.136 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5199999999999998e-07 2024-09-21T21:41:11.136 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.136 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.136 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:11.136 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407723+0000", 2024-09-21T21:41:11.136 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9139999999999998e-06 2024-09-21T21:41:11.136 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.136 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.136 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:11.136 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407729+0000", 2024-09-21T21:41:11.136 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2009999999999996e-06 2024-09-21T21:41:11.137 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.137 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.137 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:11.137 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407734+0000", 2024-09-21T21:41:11.137 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.006e-06 2024-09-21T21:41:11.137 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.137 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.137 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:11.137 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.452364+0000", 2024-09-21T21:41:11.137 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044630625 2024-09-21T21:41:11.137 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.137 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.137 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:11.137 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.452402+0000", 2024-09-21T21:41:11.137 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7518999999999998e-05 2024-09-21T21:41:11.137 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.138 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.138 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:11.138 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.452439+0000", 2024-09-21T21:41:11.138 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7122999999999997e-05 2024-09-21T21:41:11.138 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:11.138 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:11.138 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:11.138 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.138 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.138 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2436 3.d 3:b26310db:::rbd_data.112145fae779.0000000000000006:a [read 10694656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:11.138 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.407740+0000", 2024-09-21T21:41:11.138 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.698193771, 2024-09-21T21:41:11.138 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044797624000000001, 2024-09-21T21:41:11.138 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:11.139 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:11.139 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:11.139 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:11.139 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:11.139 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:11.139 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2436 2024-09-21T21:41:11.139 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.139 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:11.139 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.139 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:11.139 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407740+0000", 2024-09-21T21:41:11.139 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:11.139 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.139 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.140 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:11.140 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407740+0000", 2024-09-21T21:41:11.140 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:11.140 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.140 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.140 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:11.140 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407740+0000", 2024-09-21T21:41:11.140 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9299999999999999e-07 2024-09-21T21:41:11.140 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.140 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.140 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:11.140 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407742+0000", 2024-09-21T21:41:11.140 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.826e-06 2024-09-21T21:41:11.141 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.141 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.141 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:11.141 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407748+0000", 2024-09-21T21:41:11.141 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2149999999999998e-06 2024-09-21T21:41:11.141 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.141 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.141 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:11.141 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407753+0000", 2024-09-21T21:41:11.141 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.848e-06 2024-09-21T21:41:11.141 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.141 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.141 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:11.142 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.452463+0000", 2024-09-21T21:41:11.142 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044709596999999997 2024-09-21T21:41:11.142 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.142 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.142 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:11.142 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.452500+0000", 2024-09-21T21:41:11.142 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7058999999999999e-05 2024-09-21T21:41:11.142 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.142 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.142 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:11.142 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.452537+0000", 2024-09-21T21:41:11.142 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7685999999999998e-05 2024-09-21T21:41:11.142 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:11.142 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:11.143 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:11.143 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.143 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.143 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2602 3.d 3:b154679f:::rbd_data.112145fae779.0000000000000004:a [read 3923968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:11.143 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.411479+0000", 2024-09-21T21:41:11.143 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.694454313, 2024-09-21T21:41:11.143 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044281854000000002, 2024-09-21T21:41:11.143 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:11.143 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:11.143 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:11.143 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:11.143 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:11.143 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:11.143 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2602 2024-09-21T21:41:11.143 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.143 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:11.144 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.144 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:11.144 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411479+0000", 2024-09-21T21:41:11.144 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:11.144 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.144 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.144 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:11.144 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411479+0000", 2024-09-21T21:41:11.144 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:11.144 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.144 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.144 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:11.144 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411480+0000", 2024-09-21T21:41:11.144 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.37e-07 2024-09-21T21:41:11.144 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.144 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.145 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:11.145 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411482+0000", 2024-09-21T21:41:11.145 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9829999999999999e-06 2024-09-21T21:41:11.145 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.145 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.145 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:11.145 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411488+0000", 2024-09-21T21:41:11.145 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.947e-06 2024-09-21T21:41:11.145 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.145 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.145 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:11.145 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411492+0000", 2024-09-21T21:41:11.145 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.357e-06 2024-09-21T21:41:11.145 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.145 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.146 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:11.146 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.455581+0000", 2024-09-21T21:41:11.146 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044089077999999997 2024-09-21T21:41:11.146 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.146 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.146 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:11.146 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.455675+0000", 2024-09-21T21:41:11.146 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3757000000000004e-05 2024-09-21T21:41:11.146 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.146 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.146 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:11.146 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.455761+0000", 2024-09-21T21:41:11.146 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6194999999999997e-05 2024-09-21T21:41:11.146 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:11.146 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:11.147 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:11.147 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.147 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.147 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2612 3.d 3:b26310db:::rbd_data.112145fae779.0000000000000006:a [read 3448832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:11.147 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.411498+0000", 2024-09-21T21:41:11.147 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.694435544999999, 2024-09-21T21:41:11.147 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044369585000000003, 2024-09-21T21:41:11.147 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:11.147 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:11.147 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:11.147 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:11.147 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:11.148 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:11.148 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2612 2024-09-21T21:41:11.148 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.148 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:11.148 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.148 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:11.148 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411498+0000", 2024-09-21T21:41:11.148 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:11.148 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.148 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.148 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:11.148 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411498+0000", 2024-09-21T21:41:11.148 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:11.148 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.149 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.149 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:11.149 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411498+0000", 2024-09-21T21:41:11.149 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6899999999999998e-07 2024-09-21T21:41:11.149 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.149 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.149 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:11.149 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411500+0000", 2024-09-21T21:41:11.149 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.841e-06 2024-09-21T21:41:11.149 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.149 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.149 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:11.149 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411506+0000", 2024-09-21T21:41:11.149 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7350000000000001e-06 2024-09-21T21:41:11.149 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.149 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.150 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:11.150 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411510+0000", 2024-09-21T21:41:11.150 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4830000000000001e-06 2024-09-21T21:41:11.150 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.150 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.150 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:11.150 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.455789+0000", 2024-09-21T21:41:11.150 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044278359000000003 2024-09-21T21:41:11.150 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.150 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.150 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:11.150 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.455824+0000", 2024-09-21T21:41:11.150 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5719000000000001e-05 2024-09-21T21:41:11.151 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.151 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.151 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:11.151 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.455868+0000", 2024-09-21T21:41:11.151 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3078999999999999e-05 2024-09-21T21:41:11.151 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:11.151 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:11.151 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:11.151 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.151 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.151 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2613 3.d 3:b26310db:::rbd_data.112145fae779.0000000000000006:a [read 3506176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:11.151 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.411516+0000", 2024-09-21T21:41:11.151 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.694417868, 2024-09-21T21:41:11.151 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044562615, 2024-09-21T21:41:11.152 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:11.152 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:11.152 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:11.152 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:11.152 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:11.152 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:11.152 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2613 2024-09-21T21:41:11.152 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.152 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:11.152 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.152 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:11.152 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411516+0000", 2024-09-21T21:41:11.152 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:11.153 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.153 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.153 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:11.153 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411516+0000", 2024-09-21T21:41:11.153 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:11.153 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.153 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.153 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:11.153 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411516+0000", 2024-09-21T21:41:11.153 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4400000000000001e-07 2024-09-21T21:41:11.153 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.153 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.153 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:11.154 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411522+0000", 2024-09-21T21:41:11.154 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2600000000000002e-06 2024-09-21T21:41:11.154 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.154 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.154 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:11.154 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411528+0000", 2024-09-21T21:41:11.154 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5759999999999996e-06 2024-09-21T21:41:11.154 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.154 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.154 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:11.154 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411534+0000", 2024-09-21T21:41:11.154 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3010000000000003e-06 2024-09-21T21:41:11.154 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.154 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.155 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:11.155 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.455911+0000", 2024-09-21T21:41:11.155 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044376546000000003 2024-09-21T21:41:11.155 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.155 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.155 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:11.155 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.455996+0000", 2024-09-21T21:41:11.155 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.4980000000000003e-05 2024-09-21T21:41:11.155 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.155 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.155 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:11.155 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.456078+0000", 2024-09-21T21:41:11.155 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2608000000000006e-05 2024-09-21T21:41:11.155 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:11.156 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:11.156 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:11.156 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.156 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.156 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2614 3.d 3:b26310db:::rbd_data.112145fae779.0000000000000006:a [read 3538944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:11.156 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.411540+0000", 2024-09-21T21:41:11.156 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.694393054000001, 2024-09-21T21:41:11.156 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044644049999999998, 2024-09-21T21:41:11.156 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:11.156 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:11.156 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:11.156 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:11.156 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:11.156 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:11.157 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2614 2024-09-21T21:41:11.157 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.157 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:11.157 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.157 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:11.157 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411540+0000", 2024-09-21T21:41:11.157 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:11.157 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.157 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.157 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:11.157 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411540+0000", 2024-09-21T21:41:11.157 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:11.157 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.158 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.158 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:11.158 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411541+0000", 2024-09-21T21:41:11.158 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3799999999999997e-07 2024-09-21T21:41:11.158 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.158 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.158 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:11.158 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411543+0000", 2024-09-21T21:41:11.158 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1100000000000001e-06 2024-09-21T21:41:11.158 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.158 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.158 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:11.158 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411549+0000", 2024-09-21T21:41:11.158 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8300000000000001e-06 2024-09-21T21:41:11.159 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.159 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.159 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:11.159 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411554+0000", 2024-09-21T21:41:11.159 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8169999999999998e-06 2024-09-21T21:41:11.159 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.159 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.159 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:11.159 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.456107+0000", 2024-09-21T21:41:11.159 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044553552000000003 2024-09-21T21:41:11.159 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.159 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.159 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:11.160 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.456144+0000", 2024-09-21T21:41:11.160 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6560000000000002e-05 2024-09-21T21:41:11.160 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.160 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.160 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:11.160 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.456185+0000", 2024-09-21T21:41:11.160 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0642999999999997e-05 2024-09-21T21:41:11.160 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:11.160 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:11.160 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:11.160 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.160 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.160 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2615 3.d 3:b26310db:::rbd_data.112145fae779.0000000000000006:a [read 3596288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:11.160 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.411560+0000", 2024-09-21T21:41:11.160 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.694373498999999, 2024-09-21T21:41:11.161 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044828397999999998, 2024-09-21T21:41:11.161 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:11.161 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:11.161 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:11.161 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:11.161 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:11.161 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:11.161 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2615 2024-09-21T21:41:11.161 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.161 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:11.161 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.161 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:11.161 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411560+0000", 2024-09-21T21:41:11.161 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:11.161 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.161 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.162 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:11.162 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411560+0000", 2024-09-21T21:41:11.162 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:11.162 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.162 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.162 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:11.162 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411560+0000", 2024-09-21T21:41:11.162 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7399999999999999e-07 2024-09-21T21:41:11.162 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.162 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.162 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:11.162 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411562+0000", 2024-09-21T21:41:11.162 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.854e-06 2024-09-21T21:41:11.162 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.162 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.163 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:11.163 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411568+0000", 2024-09-21T21:41:11.163 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6760000000000004e-06 2024-09-21T21:41:11.163 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.163 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.163 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:11.163 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411573+0000", 2024-09-21T21:41:11.163 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6269999999999999e-06 2024-09-21T21:41:11.163 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.163 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.163 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:11.163 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.456226+0000", 2024-09-21T21:41:11.163 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044653088000000001 2024-09-21T21:41:11.163 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.163 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.164 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:11.164 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.456311+0000", 2024-09-21T21:41:11.164 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5685000000000004e-05 2024-09-21T21:41:11.164 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.164 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.164 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:11.164 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.456388+0000", 2024-09-21T21:41:11.164 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7094000000000004e-05 2024-09-21T21:41:11.164 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:11.164 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:11.164 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:11.164 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.164 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.164 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2616 3.d 3:b26310db:::rbd_data.112145fae779.0000000000000006:a [read 3653632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:11.164 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.411578+0000", 2024-09-21T21:41:11.164 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.694355594999999, 2024-09-21T21:41:11.164 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044914731999999999, 2024-09-21T21:41:11.165 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:11.165 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:11.165 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:11.165 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:11.165 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:11.165 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:11.165 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2616 2024-09-21T21:41:11.165 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.165 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:11.165 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.165 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:11.165 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411578+0000", 2024-09-21T21:41:11.165 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:11.166 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.166 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.166 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:11.166 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411578+0000", 2024-09-21T21:41:11.166 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:11.166 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.166 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.166 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:11.166 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411578+0000", 2024-09-21T21:41:11.166 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.03e-07 2024-09-21T21:41:11.166 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.166 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.166 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:11.167 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411580+0000", 2024-09-21T21:41:11.167 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.8199999999999999e-06 2024-09-21T21:41:11.167 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.167 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.167 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:11.167 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411586+0000", 2024-09-21T21:41:11.167 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6319999999999996e-06 2024-09-21T21:41:11.167 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.167 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.167 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:11.167 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411590+0000", 2024-09-21T21:41:11.167 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5979999999999999e-06 2024-09-21T21:41:11.167 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.167 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.168 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:11.168 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.456418+0000", 2024-09-21T21:41:11.168 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044827504999999997 2024-09-21T21:41:11.168 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.168 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.168 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:11.168 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.456454+0000", 2024-09-21T21:41:11.168 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5883999999999997e-05 2024-09-21T21:41:11.168 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.168 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.168 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:11.169 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.456493+0000", 2024-09-21T21:41:11.169 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8890000000000002e-05 2024-09-21T21:41:11.169 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:11.169 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:11.169 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:11.169 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.169 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.169 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2617 3.d 3:b26310db:::rbd_data.112145fae779.0000000000000006:a [read 3686400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:11.169 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.411596+0000", 2024-09-21T21:41:11.169 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.694338005000001, 2024-09-21T21:41:11.169 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.045094515000000002, 2024-09-21T21:41:11.169 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:11.169 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:11.170 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:11.170 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:11.170 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:11.170 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:11.170 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2617 2024-09-21T21:41:11.170 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.170 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:11.170 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.170 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:11.170 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411596+0000", 2024-09-21T21:41:11.170 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:11.170 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.170 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.170 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:11.171 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411596+0000", 2024-09-21T21:41:11.171 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:11.171 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.171 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.171 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:11.171 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411596+0000", 2024-09-21T21:41:11.171 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3500000000000002e-07 2024-09-21T21:41:11.171 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.171 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.171 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:11.171 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411598+0000", 2024-09-21T21:41:11.171 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.871e-06 2024-09-21T21:41:11.171 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.171 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.172 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:11.172 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411603+0000", 2024-09-21T21:41:11.172 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6420000000000004e-06 2024-09-21T21:41:11.172 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.172 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.172 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:11.172 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411608+0000", 2024-09-21T21:41:11.172 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4780000000000002e-06 2024-09-21T21:41:11.172 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.172 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.172 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:11.172 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.456529+0000", 2024-09-21T21:41:11.172 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044921232999999998 2024-09-21T21:41:11.172 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.173 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.173 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:11.173 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.456616+0000", 2024-09-21T21:41:11.173 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6712000000000006e-05 2024-09-21T21:41:11.173 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.173 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.173 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:11.173 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.456690+0000", 2024-09-21T21:41:11.173 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4244000000000003e-05 2024-09-21T21:41:11.173 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:11.173 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:11.173 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:11.173 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.173 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.174 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2618 3.d 3:b26310db:::rbd_data.112145fae779.0000000000000006:a [read 3743744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:11.174 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.411613+0000", 2024-09-21T21:41:11.174 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.694320426000001, 2024-09-21T21:41:11.174 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.045180423999999997, 2024-09-21T21:41:11.174 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:11.174 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:11.174 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:11.174 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:11.174 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:11.174 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:11.174 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2618 2024-09-21T21:41:11.174 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.174 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:11.175 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.175 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:11.175 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411613+0000", 2024-09-21T21:41:11.175 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:11.175 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.175 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.175 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:11.175 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411613+0000", 2024-09-21T21:41:11.175 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:11.175 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.175 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.175 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:11.175 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.412868+0000", 2024-09-21T21:41:11.175 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012549 2024-09-21T21:41:11.175 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.176 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.176 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:11.176 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.412873+0000", 2024-09-21T21:41:11.176 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9060000000000001e-06 2024-09-21T21:41:11.176 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.176 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.176 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:11.176 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.412884+0000", 2024-09-21T21:41:11.176 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1164e-05 2024-09-21T21:41:11.176 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.176 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.176 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:11.176 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.412892+0000", 2024-09-21T21:41:11.176 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5719999999999997e-06 2024-09-21T21:41:11.177 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.177 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.177 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:11.177 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.456717+0000", 2024-09-21T21:41:11.177 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043825559 2024-09-21T21:41:11.177 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.177 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.177 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:11.177 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.456753+0000", 2024-09-21T21:41:11.177 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6158000000000002e-05 2024-09-21T21:41:11.177 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.177 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.177 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:11.178 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.456794+0000", 2024-09-21T21:41:11.178 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0164999999999999e-05 2024-09-21T21:41:11.178 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:11.178 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:11.178 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:11.178 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.178 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.178 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2667 3.d 3:b2093d43:::rbd_data.112145fae779.000000000000000b:a [read 3809280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:11.178 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.414362+0000", 2024-09-21T21:41:11.178 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.691571747999999, 2024-09-21T21:41:11.178 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044335620999999999, 2024-09-21T21:41:11.178 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:11.178 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:11.179 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:11.179 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:11.179 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:11.179 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:11.179 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2667 2024-09-21T21:41:11.179 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.179 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:11.179 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.179 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:11.179 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.414362+0000", 2024-09-21T21:41:11.179 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:11.179 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.179 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.179 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:11.180 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.414362+0000", 2024-09-21T21:41:11.180 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:11.180 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.180 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.180 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:11.180 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.414867+0000", 2024-09-21T21:41:11.180 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00050531999999999997 2024-09-21T21:41:11.180 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.180 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.180 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:11.180 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.414871+0000", 2024-09-21T21:41:11.180 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2520000000000001e-06 2024-09-21T21:41:11.180 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.180 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.181 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:11.181 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.414879+0000", 2024-09-21T21:41:11.181 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0369999999999995e-06 2024-09-21T21:41:11.181 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.181 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.181 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:11.181 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.414886+0000", 2024-09-21T21:41:11.181 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.19e-06 2024-09-21T21:41:11.181 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.181 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.181 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:11.181 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.458613+0000", 2024-09-21T21:41:11.181 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043727184000000002 2024-09-21T21:41:11.181 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.182 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.182 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:11.182 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.458649+0000", 2024-09-21T21:41:11.182 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6418999999999998e-05 2024-09-21T21:41:11.182 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.182 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.182 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:11.182 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.458697+0000", 2024-09-21T21:41:11.182 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8219e-05 2024-09-21T21:41:11.182 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:11.182 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:11.182 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:11.182 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.183 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.183 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2725 3.d 3:b154679f:::rbd_data.112145fae779.0000000000000004:a [read 3534848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:11.183 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.414975+0000", 2024-09-21T21:41:11.183 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.690958822000001, 2024-09-21T21:41:11.183 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044402427000000001, 2024-09-21T21:41:11.183 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:11.183 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:11.183 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:11.183 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:11.183 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:11.183 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:11.183 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2725 2024-09-21T21:41:11.183 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.183 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:11.183 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.184 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:11.184 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.414975+0000", 2024-09-21T21:41:11.184 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:11.184 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.184 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.184 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:11.184 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.414975+0000", 2024-09-21T21:41:11.184 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:11.184 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.184 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.184 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:11.184 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.414975+0000", 2024-09-21T21:41:11.184 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1899999999999999e-07 2024-09-21T21:41:11.184 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.184 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.185 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:11.185 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.414976+0000", 2024-09-21T21:41:11.185 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1689999999999999e-06 2024-09-21T21:41:11.185 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.185 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.185 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:11.185 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.414979+0000", 2024-09-21T21:41:11.185 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9900000000000002e-06 2024-09-21T21:41:11.185 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.185 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.185 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:11.185 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.414982+0000", 2024-09-21T21:41:11.185 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.621e-06 2024-09-21T21:41:11.185 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.185 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.186 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:11.186 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.459221+0000", 2024-09-21T21:41:11.186 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044239602000000003 2024-09-21T21:41:11.186 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.186 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.186 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:11.186 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.459297+0000", 2024-09-21T21:41:11.186 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5768999999999999e-05 2024-09-21T21:41:11.186 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.186 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.186 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:11.186 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.459377+0000", 2024-09-21T21:41:11.186 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0056999999999996e-05 2024-09-21T21:41:11.186 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:11.186 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:11.187 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:11.187 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.187 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.187 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2726 3.d 3:b154679f:::rbd_data.112145fae779.0000000000000004:a [read 3567616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:11.187 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.414985+0000", 2024-09-21T21:41:11.187 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.690948755000001, 2024-09-21T21:41:11.187 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044526137, 2024-09-21T21:41:11.187 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:11.187 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:11.187 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:11.187 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:11.187 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:11.187 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:11.187 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2726 2024-09-21T21:41:11.187 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.188 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:11.188 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.188 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:11.188 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.414985+0000", 2024-09-21T21:41:11.188 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:11.188 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.188 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.188 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:11.188 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.414985+0000", 2024-09-21T21:41:11.188 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:11.188 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.188 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.188 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:11.188 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.414985+0000", 2024-09-21T21:41:11.189 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0600000000000001e-07 2024-09-21T21:41:11.189 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.189 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.189 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:11.189 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.414986+0000", 2024-09-21T21:41:11.189 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.175e-06 2024-09-21T21:41:11.189 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.189 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.189 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:11.189 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.414989+0000", 2024-09-21T21:41:11.189 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8720000000000002e-06 2024-09-21T21:41:11.189 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.189 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.189 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:11.189 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.414992+0000", 2024-09-21T21:41:11.190 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3680000000000002e-06 2024-09-21T21:41:11.190 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.190 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.190 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:11.190 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.459415+0000", 2024-09-21T21:41:11.190 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044422167999999998 2024-09-21T21:41:11.190 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.190 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.190 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:11.190 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.459466+0000", 2024-09-21T21:41:11.190 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1177000000000001e-05 2024-09-21T21:41:11.190 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.190 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.191 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:11.191 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.459511+0000", 2024-09-21T21:41:11.191 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5071000000000003e-05 2024-09-21T21:41:11.191 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:11.191 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:11.191 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:11.191 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.191 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.191 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2727 3.d 3:b154679f:::rbd_data.112145fae779.0000000000000004:a [read 3624960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:11.191 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.414997+0000", 2024-09-21T21:41:11.191 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.690936182, 2024-09-21T21:41:11.191 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044651328999999997, 2024-09-21T21:41:11.191 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:11.192 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:11.192 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:11.192 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:11.192 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:11.192 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:11.192 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2727 2024-09-21T21:41:11.192 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.192 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:11.192 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.192 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:11.192 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.414997+0000", 2024-09-21T21:41:11.192 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:11.192 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.192 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.192 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:11.193 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.414997+0000", 2024-09-21T21:41:11.193 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:11.193 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.193 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.193 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:11.193 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.416868+0000", 2024-09-21T21:41:11.193 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001871019 2024-09-21T21:41:11.193 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.193 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.193 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:11.193 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.416873+0000", 2024-09-21T21:41:11.193 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6299999999999997e-06 2024-09-21T21:41:11.193 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.193 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.193 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:11.194 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.416883+0000", 2024-09-21T21:41:11.194 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0172e-05 2024-09-21T21:41:11.194 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.194 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.194 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:11.194 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.416890+0000", 2024-09-21T21:41:11.194 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8689999999999998e-06 2024-09-21T21:41:11.194 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.194 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.194 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:11.194 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.459541+0000", 2024-09-21T21:41:11.194 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042650592000000001 2024-09-21T21:41:11.194 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.194 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.195 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:11.195 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.459601+0000", 2024-09-21T21:41:11.195 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9978000000000001e-05 2024-09-21T21:41:11.195 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.195 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.195 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:11.195 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.459649+0000", 2024-09-21T21:41:11.195 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8068999999999997e-05 2024-09-21T21:41:11.195 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:11.195 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:11.195 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:11.195 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:11.195 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:11.195 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:11.197 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T21:41:11.197 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:11.195+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:11.297 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:11.296+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:11.341 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:11.341 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:41:11.341 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-21T21:41:11.341 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:11.349 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T21:41:11.398 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:11.396+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:11.492 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:11.492 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:41:11.492 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:41:11.492 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:41:11.492 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:11.499 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:11.498+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:11.500 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T21:41:11.583 INFO:teuthology.orchestra.run.smithi081.stderr:set 3.8 pg_upmap mapping to [2,3] 2024-09-21T21:41:11.591 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T21:41:11.600 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:11.598+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:11.607 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:11.607 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-21T21:41:11.607 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-21T21:41:11.608 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:41:11.608 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.608 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:8196 3.3 3:c030d67f:::rbd_data.112145fae779.0000000000000003:a [read 9502720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:41:11.608 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.688149+0000", 2024-09-21T21:41:11.608 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.917406081999999, 2024-09-21T21:41:11.608 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.018375909999999999, 2024-09-21T21:41:11.608 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:11.608 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:11.608 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:11.608 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:11.608 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:11.608 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:11.609 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 8196 2024-09-21T21:41:11.609 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.609 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:11.609 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.609 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:11.609 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688149+0000", 2024-09-21T21:41:11.609 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:11.609 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.609 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.609 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:11.609 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688149+0000", 2024-09-21T21:41:11.609 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:11.609 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.610 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.610 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:11.610 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688150+0000", 2024-09-21T21:41:11.610 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5900000000000003e-07 2024-09-21T21:41:11.610 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.610 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.610 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:11.610 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688156+0000", 2024-09-21T21:41:11.610 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3160000000000002e-06 2024-09-21T21:41:11.610 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.610 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.610 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:11.610 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688163+0000", 2024-09-21T21:41:11.611 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7920000000000004e-06 2024-09-21T21:41:11.611 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.611 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.611 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:11.611 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688167+0000", 2024-09-21T21:41:11.611 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4360000000000004e-06 2024-09-21T21:41:11.611 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.611 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.611 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:11.611 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.706376+0000", 2024-09-21T21:41:11.611 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.018208762 2024-09-21T21:41:11.611 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.611 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.611 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:11.612 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.706456+0000", 2024-09-21T21:41:11.612 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0065e-05 2024-09-21T21:41:11.612 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.612 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.612 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:11.612 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.706525+0000", 2024-09-21T21:41:11.612 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9179999999999998e-05 2024-09-21T21:41:11.612 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:11.612 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:11.612 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:11.612 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.612 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.612 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:8197 3.3 3:c030d67f:::rbd_data.112145fae779.0000000000000003:a [read 9560064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:41:11.612 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.688173+0000", 2024-09-21T21:41:11.613 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.917382649, 2024-09-21T21:41:11.613 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.018539605000000001, 2024-09-21T21:41:11.613 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:11.613 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:11.613 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:11.613 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:11.613 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:11.613 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:11.613 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 8197 2024-09-21T21:41:11.613 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.613 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:11.613 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.613 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:11.613 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688173+0000", 2024-09-21T21:41:11.613 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:11.614 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.614 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.614 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:11.614 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688173+0000", 2024-09-21T21:41:11.614 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:11.614 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.614 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.614 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:11.614 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688173+0000", 2024-09-21T21:41:11.614 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2300000000000002e-07 2024-09-21T21:41:11.614 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.614 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.614 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:11.615 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688175+0000", 2024-09-21T21:41:11.615 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9589999999999998e-06 2024-09-21T21:41:11.615 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.615 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.615 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:11.615 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688182+0000", 2024-09-21T21:41:11.615 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4239999999999998e-06 2024-09-21T21:41:11.615 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.615 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.615 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:11.615 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688186+0000", 2024-09-21T21:41:11.615 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4939999999999997e-06 2024-09-21T21:41:11.615 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.616 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.616 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:11.616 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.706559+0000", 2024-09-21T21:41:11.616 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.018373308000000001 2024-09-21T21:41:11.616 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.616 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.616 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:11.616 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.706642+0000", 2024-09-21T21:41:11.616 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2964000000000001e-05 2024-09-21T21:41:11.616 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.616 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.616 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:11.616 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.706712+0000", 2024-09-21T21:41:11.616 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0032999999999998e-05 2024-09-21T21:41:11.617 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:11.617 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:11.617 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:11.617 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.617 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.617 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:8317 3.3 3:c030d67f:::rbd_data.112145fae779.0000000000000003:a [read 9064448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:41:11.617 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.688286+0000", 2024-09-21T21:41:11.617 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.917268977999999, 2024-09-21T21:41:11.617 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.018621219000000001, 2024-09-21T21:41:11.617 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:11.617 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:11.617 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:11.617 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:11.617 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:11.617 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:11.618 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 8317 2024-09-21T21:41:11.618 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.618 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:11.618 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.618 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:11.618 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688286+0000", 2024-09-21T21:41:11.618 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:11.618 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.618 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.618 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:11.618 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688286+0000", 2024-09-21T21:41:11.618 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:11.618 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.619 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.619 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:11.619 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688287+0000", 2024-09-21T21:41:11.619 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6499999999999999e-07 2024-09-21T21:41:11.619 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.619 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.619 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:11.619 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688290+0000", 2024-09-21T21:41:11.619 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1879999999999998e-06 2024-09-21T21:41:11.619 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.619 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.619 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:11.619 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688317+0000", 2024-09-21T21:41:11.619 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7353999999999999e-05 2024-09-21T21:41:11.619 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.620 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.620 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:11.620 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688324+0000", 2024-09-21T21:41:11.620 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2910000000000004e-06 2024-09-21T21:41:11.620 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.620 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.620 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:11.620 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.706744+0000", 2024-09-21T21:41:11.620 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.018420414 2024-09-21T21:41:11.620 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.620 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.620 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:11.620 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.706826+0000", 2024-09-21T21:41:11.620 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2286999999999993e-05 2024-09-21T21:41:11.621 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.621 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.621 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:11.621 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.706908+0000", 2024-09-21T21:41:11.621 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1119999999999996e-05 2024-09-21T21:41:11.621 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:11.621 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:11.621 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:11.621 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.621 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.621 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:8318 3.3 3:c030d67f:::rbd_data.112145fae779.0000000000000003:a [read 9121792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:41:11.621 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.688335+0000", 2024-09-21T21:41:11.621 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.917220264999999, 2024-09-21T21:41:11.621 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.018752181999999999, 2024-09-21T21:41:11.622 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:11.622 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:11.622 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:11.622 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:11.622 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:11.622 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:11.622 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 8318 2024-09-21T21:41:11.622 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.622 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:11.622 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.622 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:11.622 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688335+0000", 2024-09-21T21:41:11.622 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:11.622 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.623 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.623 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:11.623 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688335+0000", 2024-09-21T21:41:11.623 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:11.623 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.623 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.623 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:11.623 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688336+0000", 2024-09-21T21:41:11.623 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5000000000000003e-07 2024-09-21T21:41:11.623 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.623 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.623 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:11.623 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688339+0000", 2024-09-21T21:41:11.623 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.128e-06 2024-09-21T21:41:11.624 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.624 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.624 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:11.624 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688347+0000", 2024-09-21T21:41:11.624 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5920000000000001e-06 2024-09-21T21:41:11.624 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.624 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.624 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:11.624 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688353+0000", 2024-09-21T21:41:11.624 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6570000000000002e-06 2024-09-21T21:41:11.624 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.624 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.624 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:11.624 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.706940+0000", 2024-09-21T21:41:11.624 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.018586689 2024-09-21T21:41:11.625 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.625 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.625 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:11.625 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.707015+0000", 2024-09-21T21:41:11.625 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4888999999999994e-05 2024-09-21T21:41:11.625 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.625 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.625 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:11.625 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.707087+0000", 2024-09-21T21:41:11.625 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2676999999999996e-05 2024-09-21T21:41:11.625 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:11.625 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:11.625 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:11.625 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.626 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.626 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:8319 3.3 3:c030d67f:::rbd_data.112145fae779.0000000000000003:a [read 9211904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:41:11.626 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.688360+0000", 2024-09-21T21:41:11.626 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.917194986, 2024-09-21T21:41:11.626 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.018899836, 2024-09-21T21:41:11.626 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:11.626 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:11.626 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:11.626 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:11.626 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:11.626 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:11.626 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 8319 2024-09-21T21:41:11.626 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.626 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:11.627 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.627 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:11.627 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688360+0000", 2024-09-21T21:41:11.627 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:11.627 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.627 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.627 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:11.627 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688360+0000", 2024-09-21T21:41:11.627 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:11.627 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.627 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.627 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:11.627 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688361+0000", 2024-09-21T21:41:11.627 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5499999999999998e-07 2024-09-21T21:41:11.628 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.628 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.628 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:11.628 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688363+0000", 2024-09-21T21:41:11.628 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.21e-06 2024-09-21T21:41:11.628 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.628 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.628 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:11.628 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688371+0000", 2024-09-21T21:41:11.628 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2539999999999992e-06 2024-09-21T21:41:11.628 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.628 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.628 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:11.629 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688377+0000", 2024-09-21T21:41:11.629 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.468e-06 2024-09-21T21:41:11.629 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.629 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.629 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:11.629 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.707121+0000", 2024-09-21T21:41:11.629 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.018743774000000001 2024-09-21T21:41:11.629 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.629 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.629 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:11.629 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.707195+0000", 2024-09-21T21:41:11.629 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4745000000000003e-05 2024-09-21T21:41:11.629 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.629 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.630 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:11.630 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.707260+0000", 2024-09-21T21:41:11.630 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4930000000000003e-05 2024-09-21T21:41:11.630 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:11.630 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:11.630 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:11.630 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.630 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.630 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:8320 3.3 3:c030d67f:::rbd_data.112145fae779.0000000000000003:a [read 9269248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:41:11.630 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.688383+0000", 2024-09-21T21:41:11.630 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.917171886, 2024-09-21T21:41:11.630 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.019040664999999998, 2024-09-21T21:41:11.630 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:11.631 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:11.631 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:11.631 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:11.631 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:11.631 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:11.631 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 8320 2024-09-21T21:41:11.631 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.631 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:11.631 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.631 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:11.631 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688383+0000", 2024-09-21T21:41:11.631 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:11.631 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.631 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.632 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:11.632 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688383+0000", 2024-09-21T21:41:11.632 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:11.632 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.632 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.632 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:11.632 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688384+0000", 2024-09-21T21:41:11.632 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.27e-07 2024-09-21T21:41:11.632 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.632 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.632 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:11.632 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688386+0000", 2024-09-21T21:41:11.632 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2299999999999998e-06 2024-09-21T21:41:11.633 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.633 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.633 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:11.633 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688395+0000", 2024-09-21T21:41:11.633 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.0410000000000003e-06 2024-09-21T21:41:11.633 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.633 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.633 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:11.633 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688400+0000", 2024-09-21T21:41:11.633 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.198e-06 2024-09-21T21:41:11.633 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.633 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.633 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:11.633 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.707289+0000", 2024-09-21T21:41:11.633 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.018888820000000001 2024-09-21T21:41:11.634 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.634 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.634 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:11.634 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.707358+0000", 2024-09-21T21:41:11.634 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9219000000000003e-05 2024-09-21T21:41:11.634 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.634 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.634 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:11.634 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.707424+0000", 2024-09-21T21:41:11.634 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5729999999999996e-05 2024-09-21T21:41:11.634 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:11.634 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:11.634 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:11.634 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.635 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.635 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:8321 3.3 3:c030d67f:::rbd_data.112145fae779.0000000000000003:a [read 9359360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:41:11.635 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.688407+0000", 2024-09-21T21:41:11.635 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.917148698, 2024-09-21T21:41:11.635 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.019217557999999999, 2024-09-21T21:41:11.635 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:11.635 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:11.635 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:11.635 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:11.635 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:11.635 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:11.635 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 8321 2024-09-21T21:41:11.635 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.635 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:11.636 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.636 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:11.636 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688407+0000", 2024-09-21T21:41:11.636 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:11.636 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.636 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.636 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:11.636 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688407+0000", 2024-09-21T21:41:11.636 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:11.636 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.636 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.636 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:11.636 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688407+0000", 2024-09-21T21:41:11.636 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2500000000000001e-07 2024-09-21T21:41:11.636 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.636 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.637 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:11.637 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688409+0000", 2024-09-21T21:41:11.637 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1299999999999999e-06 2024-09-21T21:41:11.637 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.637 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.637 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:11.637 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688416+0000", 2024-09-21T21:41:11.637 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9539999999999998e-06 2024-09-21T21:41:11.637 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.637 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.637 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:11.637 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688421+0000", 2024-09-21T21:41:11.637 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2610000000000001e-06 2024-09-21T21:41:11.637 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.637 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.637 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:11.638 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.707456+0000", 2024-09-21T21:41:11.638 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.019035024000000001 2024-09-21T21:41:11.638 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.638 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.638 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:11.638 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.707550+0000", 2024-09-21T21:41:11.638 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3696000000000006e-05 2024-09-21T21:41:11.638 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.638 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.638 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:11.638 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.707624+0000", 2024-09-21T21:41:11.638 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4067999999999996e-05 2024-09-21T21:41:11.638 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:11.638 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:11.638 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:11.639 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.639 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.639 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:8322 3.3 3:c030d67f:::rbd_data.112145fae779.0000000000000003:a [read 9416704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:41:11.639 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.688427+0000", 2024-09-21T21:41:11.639 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.917127880000001, 2024-09-21T21:41:11.639 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.019393840999999998, 2024-09-21T21:41:11.639 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:11.639 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:11.639 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:11.639 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:11.639 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:11.639 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:11.639 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 8322 2024-09-21T21:41:11.639 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.640 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:11.640 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.640 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:11.640 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688427+0000", 2024-09-21T21:41:11.640 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:11.640 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.640 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.640 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:11.640 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688427+0000", 2024-09-21T21:41:11.640 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:11.640 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.640 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.640 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:11.640 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688428+0000", 2024-09-21T21:41:11.641 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7e-07 2024-09-21T21:41:11.641 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.641 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.641 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:11.641 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688430+0000", 2024-09-21T21:41:11.641 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2189999999999998e-06 2024-09-21T21:41:11.641 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.641 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.641 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:11.641 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688436+0000", 2024-09-21T21:41:11.641 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4309999999999999e-06 2024-09-21T21:41:11.641 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.641 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.641 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:11.641 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688441+0000", 2024-09-21T21:41:11.642 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7029999999999997e-06 2024-09-21T21:41:11.642 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.642 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.642 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:11.642 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.707659+0000", 2024-09-21T21:41:11.642 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.019217998999999999 2024-09-21T21:41:11.642 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.642 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.642 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:11.642 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.707749+0000", 2024-09-21T21:41:11.642 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.0229999999999995e-05 2024-09-21T21:41:11.642 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.642 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.642 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:11.643 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.707821+0000", 2024-09-21T21:41:11.643 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1889000000000002e-05 2024-09-21T21:41:11.643 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:11.643 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:11.643 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:11.643 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.643 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.643 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:8323 3.3 3:c030d67f:::rbd_data.112145fae779.0000000000000003:a [read 9449472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:41:11.643 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.688451+0000", 2024-09-21T21:41:11.643 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.91710426, 2024-09-21T21:41:11.643 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.019572622000000001, 2024-09-21T21:41:11.643 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:11.643 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:11.644 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:11.644 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:11.644 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:11.644 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:11.644 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 8323 2024-09-21T21:41:11.644 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.644 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:11.644 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.644 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:11.644 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688451+0000", 2024-09-21T21:41:11.644 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:11.644 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.644 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.644 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:11.645 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688451+0000", 2024-09-21T21:41:11.645 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:11.645 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.645 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.645 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:11.645 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688452+0000", 2024-09-21T21:41:11.645 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.39e-07 2024-09-21T21:41:11.645 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.645 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.645 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:11.645 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688454+0000", 2024-09-21T21:41:11.645 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.272e-06 2024-09-21T21:41:11.645 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.645 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.645 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:11.645 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688460+0000", 2024-09-21T21:41:11.646 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5270000000000004e-06 2024-09-21T21:41:11.646 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.646 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.646 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:11.646 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688466+0000", 2024-09-21T21:41:11.646 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1909999999999999e-06 2024-09-21T21:41:11.646 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.646 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.646 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:11.646 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.707854+0000", 2024-09-21T21:41:11.646 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.019388216999999999 2024-09-21T21:41:11.646 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.646 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.646 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:11.647 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.707952+0000", 2024-09-21T21:41:11.647 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.8136999999999999e-05 2024-09-21T21:41:11.647 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.647 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.647 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:11.647 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.708024+0000", 2024-09-21T21:41:11.647 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1839000000000001e-05 2024-09-21T21:41:11.647 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:11.647 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:11.647 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:11.647 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.647 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.647 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:8324 3.3 3:c030d67f:::rbd_data.112145fae779.0000000000000003:a [read 9506816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:41:11.647 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.688472+0000", 2024-09-21T21:41:11.647 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.917083128, 2024-09-21T21:41:11.648 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.019741218000000001, 2024-09-21T21:41:11.648 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:11.648 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:11.648 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:11.648 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:11.648 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:11.648 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:11.648 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 8324 2024-09-21T21:41:11.648 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.648 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:11.648 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.648 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:11.648 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688472+0000", 2024-09-21T21:41:11.648 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:11.648 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.649 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.649 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:11.649 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688472+0000", 2024-09-21T21:41:11.649 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:11.649 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.649 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.649 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:11.649 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688473+0000", 2024-09-21T21:41:11.649 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4299999999999999e-07 2024-09-21T21:41:11.649 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.649 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.649 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:11.649 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688475+0000", 2024-09-21T21:41:11.650 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1409999999999999e-06 2024-09-21T21:41:11.650 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.650 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.650 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:11.650 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688481+0000", 2024-09-21T21:41:11.650 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5529999999999998e-06 2024-09-21T21:41:11.650 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.650 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.650 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:11.650 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688486+0000", 2024-09-21T21:41:11.650 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9459999999999997e-06 2024-09-21T21:41:11.650 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.650 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.651 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:11.651 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.708064+0000", 2024-09-21T21:41:11.651 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.019577721999999999 2024-09-21T21:41:11.651 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.651 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.651 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:11.651 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.708144+0000", 2024-09-21T21:41:11.651 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0554000000000002e-05 2024-09-21T21:41:11.651 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.651 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.651 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:11.651 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.708213+0000", 2024-09-21T21:41:11.651 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8959000000000002e-05 2024-09-21T21:41:11.652 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:11.652 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:11.652 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:11.652 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.652 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.652 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:8325 3.3 3:c030d67f:::rbd_data.112145fae779.0000000000000003:a [read 9564160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:41:11.652 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.688493+0000", 2024-09-21T21:41:11.652 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.917062368, 2024-09-21T21:41:11.652 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.019899284999999999, 2024-09-21T21:41:11.652 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:11.652 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:11.652 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:11.652 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:11.652 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:11.653 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:11.653 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 8325 2024-09-21T21:41:11.653 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.653 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:11.653 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.653 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:11.653 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688493+0000", 2024-09-21T21:41:11.653 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:11.653 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.653 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.653 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:11.653 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688493+0000", 2024-09-21T21:41:11.653 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:11.654 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.654 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.654 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:11.654 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688493+0000", 2024-09-21T21:41:11.654 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4999999999999998e-07 2024-09-21T21:41:11.654 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.654 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.654 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:11.654 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688496+0000", 2024-09-21T21:41:11.654 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0949999999999998e-06 2024-09-21T21:41:11.654 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.654 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.654 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:11.655 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688502+0000", 2024-09-21T21:41:11.655 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4409999999999998e-06 2024-09-21T21:41:11.655 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.655 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.655 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:11.655 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688507+0000", 2024-09-21T21:41:11.655 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6229999999999996e-06 2024-09-21T21:41:11.655 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.655 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.655 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:11.655 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.708246+0000", 2024-09-21T21:41:11.655 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.019739626 2024-09-21T21:41:11.655 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.655 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.656 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:11.656 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.708321+0000", 2024-09-21T21:41:11.656 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4803999999999997e-05 2024-09-21T21:41:11.656 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.656 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.656 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:11.656 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.708392+0000", 2024-09-21T21:41:11.656 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1246000000000002e-05 2024-09-21T21:41:11.656 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:11.656 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:11.656 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:11.656 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.656 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.657 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:8444 3.3 3:c030d67f:::rbd_data.112145fae779.0000000000000003:a [read 9068544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:41:11.657 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.689763+0000", 2024-09-21T21:41:11.657 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.915792788999999, 2024-09-21T21:41:11.657 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.018821181999999999, 2024-09-21T21:41:11.657 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:11.657 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:11.657 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:11.657 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:11.657 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:11.657 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:11.657 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 8444 2024-09-21T21:41:11.657 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.657 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:11.657 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.658 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:11.658 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.689763+0000", 2024-09-21T21:41:11.658 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:11.658 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.658 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.658 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:11.658 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.689763+0000", 2024-09-21T21:41:11.658 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:11.658 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.658 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.658 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:11.658 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.689764+0000", 2024-09-21T21:41:11.658 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.9300000000000006e-07 2024-09-21T21:41:11.658 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.659 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.659 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:11.659 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.689769+0000", 2024-09-21T21:41:11.659 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.994e-06 2024-09-21T21:41:11.659 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.659 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.659 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:11.659 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.689781+0000", 2024-09-21T21:41:11.659 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.198e-05 2024-09-21T21:41:11.659 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.659 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.659 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:11.659 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.689789+0000", 2024-09-21T21:41:11.659 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2800000000000003e-06 2024-09-21T21:41:11.660 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.660 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.660 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:11.660 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.708426+0000", 2024-09-21T21:41:11.660 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.018637628 2024-09-21T21:41:11.660 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.660 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.660 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:11.660 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.708517+0000", 2024-09-21T21:41:11.660 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.0227000000000002e-05 2024-09-21T21:41:11.660 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.660 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.660 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:11.660 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.708584+0000", 2024-09-21T21:41:11.661 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7080000000000001e-05 2024-09-21T21:41:11.661 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:11.661 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:11.661 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:11.661 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.661 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.661 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:8445 3.3 3:c030d67f:::rbd_data.112145fae779.0000000000000003:a [read 9125888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:41:11.661 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.689842+0000", 2024-09-21T21:41:11.661 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.915713843000001, 2024-09-21T21:41:11.661 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.019029801999999998, 2024-09-21T21:41:11.661 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:11.661 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:11.661 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:11.661 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:11.662 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:11.662 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:11.662 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 8445 2024-09-21T21:41:11.662 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.662 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:11.662 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.662 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:11.662 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.689842+0000", 2024-09-21T21:41:11.662 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:11.662 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.662 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.662 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:11.662 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.689842+0000", 2024-09-21T21:41:11.662 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:11.663 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.663 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.663 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:11.663 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.689842+0000", 2024-09-21T21:41:11.663 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4399999999999999e-07 2024-09-21T21:41:11.663 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.663 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.663 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:11.663 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.689847+0000", 2024-09-21T21:41:11.663 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.746e-06 2024-09-21T21:41:11.663 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.663 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.663 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:11.664 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.689856+0000", 2024-09-21T21:41:11.664 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.2909999999999996e-06 2024-09-21T21:41:11.664 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.664 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.664 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:11.664 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.689868+0000", 2024-09-21T21:41:11.664 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1498000000000001e-05 2024-09-21T21:41:11.664 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.664 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.664 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:11.664 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.708618+0000", 2024-09-21T21:41:11.664 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.018750032 2024-09-21T21:41:11.664 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.664 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.664 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:11.665 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.708726+0000", 2024-09-21T21:41:11.665 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010836399999999999 2024-09-21T21:41:11.665 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.665 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.665 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:11.665 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.708871+0000", 2024-09-21T21:41:11.665 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014512699999999999 2024-09-21T21:41:11.665 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:11.665 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:11.665 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:11.665 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.665 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.665 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:8446 3.3 3:c030d67f:::rbd_data.112145fae779.0000000000000003:a [read 9216000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:41:11.666 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.689911+0000", 2024-09-21T21:41:11.666 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.915644512, 2024-09-21T21:41:11.666 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.019300134, 2024-09-21T21:41:11.666 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:11.666 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:11.666 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:11.666 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:11.666 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:11.666 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:11.666 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 8446 2024-09-21T21:41:11.666 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.666 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:11.666 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.666 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:11.667 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.689911+0000", 2024-09-21T21:41:11.667 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:11.667 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.667 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.667 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:11.667 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.689911+0000", 2024-09-21T21:41:11.667 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:11.667 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.667 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.667 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:11.667 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.689912+0000", 2024-09-21T21:41:11.667 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8899999999999999e-07 2024-09-21T21:41:11.667 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.667 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.667 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:11.668 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.689915+0000", 2024-09-21T21:41:11.668 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1839999999999999e-06 2024-09-21T21:41:11.668 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.668 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.668 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:11.668 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.689924+0000", 2024-09-21T21:41:11.668 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.0170000000000002e-06 2024-09-21T21:41:11.668 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.668 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.668 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:11.668 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.689930+0000", 2024-09-21T21:41:11.668 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4930000000000003e-06 2024-09-21T21:41:11.668 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.668 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.668 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:11.668 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.708850+0000", 2024-09-21T21:41:11.669 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.018919703 2024-09-21T21:41:11.669 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.669 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.669 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:11.669 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.709104+0000", 2024-09-21T21:41:11.669 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00025365899999999999 2024-09-21T21:41:11.669 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.669 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.669 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:11.669 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.709211+0000", 2024-09-21T21:41:11.669 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000107389 2024-09-21T21:41:11.669 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:11.669 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:11.669 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:11.670 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.670 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.670 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:8447 3.3 3:c030d67f:::rbd_data.112145fae779.0000000000000003:a [read 9273344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:41:11.670 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.689970+0000", 2024-09-21T21:41:11.670 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.915585427, 2024-09-21T21:41:11.670 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.019358220999999998, 2024-09-21T21:41:11.670 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:11.670 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:11.670 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:11.670 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:11.670 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:11.670 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:11.670 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 8447 2024-09-21T21:41:11.670 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.670 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:11.671 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.671 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:11.671 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.689970+0000", 2024-09-21T21:41:11.671 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:11.671 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.671 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.671 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:11.671 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.689970+0000", 2024-09-21T21:41:11.671 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:11.671 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.671 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.671 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:11.671 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.689971+0000", 2024-09-21T21:41:11.671 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3200000000000005e-07 2024-09-21T21:41:11.671 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.672 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.672 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:11.672 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.689974+0000", 2024-09-21T21:41:11.672 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1820000000000002e-06 2024-09-21T21:41:11.672 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.672 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.672 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:11.672 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.689982+0000", 2024-09-21T21:41:11.672 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7320000000000004e-06 2024-09-21T21:41:11.672 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.672 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.672 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:11.673 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.689989+0000", 2024-09-21T21:41:11.673 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2129999999999996e-06 2024-09-21T21:41:11.673 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.673 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.673 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:11.673 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.709241+0000", 2024-09-21T21:41:11.673 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.019252612999999998 2024-09-21T21:41:11.673 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.673 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.673 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:11.673 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.709285+0000", 2024-09-21T21:41:11.673 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3925999999999999e-05 2024-09-21T21:41:11.673 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.673 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.673 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:11.673 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.709328+0000", 2024-09-21T21:41:11.674 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2923000000000002e-05 2024-09-21T21:41:11.674 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:11.674 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:11.674 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:11.674 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.674 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.674 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:8448 3.3 3:c030d67f:::rbd_data.112145fae779.0000000000000003:a [read 9306112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:41:11.674 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.689998+0000", 2024-09-21T21:41:11.674 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.915557592999999, 2024-09-21T21:41:11.674 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.019538304999999999, 2024-09-21T21:41:11.674 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:11.674 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:11.674 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:11.674 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:11.674 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:11.675 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:11.675 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 8448 2024-09-21T21:41:11.675 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.675 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:11.675 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.675 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:11.675 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.689998+0000", 2024-09-21T21:41:11.675 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:11.675 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.675 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.675 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:11.675 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.689998+0000", 2024-09-21T21:41:11.675 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:11.675 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.675 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.675 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:11.676 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.689998+0000", 2024-09-21T21:41:11.676 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4499999999999997e-07 2024-09-21T21:41:11.676 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.676 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.676 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:11.676 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.690001+0000", 2024-09-21T21:41:11.676 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5849999999999998e-06 2024-09-21T21:41:11.676 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.676 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.676 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:11.676 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.690007+0000", 2024-09-21T21:41:11.676 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4620000000000001e-06 2024-09-21T21:41:11.676 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.676 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.677 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:11.677 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.690012+0000", 2024-09-21T21:41:11.677 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9559999999999996e-06 2024-09-21T21:41:11.677 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.677 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.677 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:11.677 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.709359+0000", 2024-09-21T21:41:11.677 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.019346550000000001 2024-09-21T21:41:11.677 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.677 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.677 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:11.677 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.709435+0000", 2024-09-21T21:41:11.677 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6556000000000004e-05 2024-09-21T21:41:11.677 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.677 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.678 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:11.678 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.709536+0000", 2024-09-21T21:41:11.678 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000100751 2024-09-21T21:41:11.678 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:11.678 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:11.678 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:11.678 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.678 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.678 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:8449 3.3 3:c030d67f:::rbd_data.112145fae779.0000000000000003:a [read 9363456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:41:11.678 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.690019+0000", 2024-09-21T21:41:11.678 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.915535858, 2024-09-21T21:41:11.678 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.019725633999999999, 2024-09-21T21:41:11.678 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:11.678 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:11.679 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:11.679 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:11.679 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:11.679 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:11.679 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 8449 2024-09-21T21:41:11.679 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.679 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:11.679 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.679 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:11.679 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.690019+0000", 2024-09-21T21:41:11.679 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:11.679 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.679 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.679 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:11.679 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.690019+0000", 2024-09-21T21:41:11.680 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:11.680 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.680 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.680 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:11.680 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.690020+0000", 2024-09-21T21:41:11.680 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3000000000000001e-07 2024-09-21T21:41:11.680 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.680 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.680 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:11.680 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.690022+0000", 2024-09-21T21:41:11.680 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2000000000000001e-06 2024-09-21T21:41:11.680 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.680 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.680 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:11.680 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.690029+0000", 2024-09-21T21:41:11.681 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5860000000000001e-06 2024-09-21T21:41:11.681 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.681 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.681 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:11.681 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.690034+0000", 2024-09-21T21:41:11.681 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9350000000000002e-06 2024-09-21T21:41:11.681 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.681 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.681 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:11.681 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.709566+0000", 2024-09-21T21:41:11.681 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.019532445999999998 2024-09-21T21:41:11.681 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.681 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.681 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:11.681 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.709647+0000", 2024-09-21T21:41:11.682 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0931000000000002e-05 2024-09-21T21:41:11.682 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.682 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.682 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:11.682 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.709745+0000", 2024-09-21T21:41:11.682 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.8105999999999999e-05 2024-09-21T21:41:11.682 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:11.682 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:11.682 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:11.682 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.682 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.682 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:8450 3.3 3:c030d67f:::rbd_data.112145fae779.0000000000000003:a [read 9420800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:41:11.682 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.690040+0000", 2024-09-21T21:41:11.682 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.915515384000001, 2024-09-21T21:41:11.683 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.019843738999999999, 2024-09-21T21:41:11.683 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:11.683 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:11.683 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:11.683 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:11.683 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:11.683 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:11.683 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 8450 2024-09-21T21:41:11.683 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.683 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:11.683 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.683 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:11.683 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.690040+0000", 2024-09-21T21:41:11.683 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:11.683 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.684 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.684 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:11.684 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.690040+0000", 2024-09-21T21:41:11.684 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:11.684 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.684 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.684 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:11.684 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.690040+0000", 2024-09-21T21:41:11.684 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1699999999999999e-07 2024-09-21T21:41:11.684 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.684 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.684 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:11.684 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.690042+0000", 2024-09-21T21:41:11.684 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9240000000000001e-06 2024-09-21T21:41:11.685 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.685 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.685 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:11.685 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.690049+0000", 2024-09-21T21:41:11.685 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3069999999999999e-06 2024-09-21T21:41:11.685 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.685 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.685 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:11.685 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.690053+0000", 2024-09-21T21:41:11.685 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8239999999999999e-06 2024-09-21T21:41:11.685 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.685 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.685 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:11.685 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.709790+0000", 2024-09-21T21:41:11.686 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.019736942 2024-09-21T21:41:11.686 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.686 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.686 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:11.686 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.709834+0000", 2024-09-21T21:41:11.686 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3744e-05 2024-09-21T21:41:11.686 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.686 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.686 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:11.686 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.709884+0000", 2024-09-21T21:41:11.686 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9580999999999998e-05 2024-09-21T21:41:11.686 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:11.686 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:11.686 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:11.687 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.687 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.687 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:8451 3.3 3:c030d67f:::rbd_data.112145fae779.0000000000000003:a [read 9453568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:41:11.687 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.690077+0000", 2024-09-21T21:41:11.687 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.915477898000001, 2024-09-21T21:41:11.687 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.020018285, 2024-09-21T21:41:11.687 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:11.687 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:11.687 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:11.687 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:11.687 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:11.687 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:11.687 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 8451 2024-09-21T21:41:11.687 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.687 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:11.687 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.688 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:11.688 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.690077+0000", 2024-09-21T21:41:11.688 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:11.688 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.688 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.688 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:11.688 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.690077+0000", 2024-09-21T21:41:11.688 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:11.688 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.688 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.688 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:11.688 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.690078+0000", 2024-09-21T21:41:11.688 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1999999999999998e-07 2024-09-21T21:41:11.689 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.689 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.689 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:11.689 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.690082+0000", 2024-09-21T21:41:11.689 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0879999999999997e-06 2024-09-21T21:41:11.689 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.689 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.689 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:11.689 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.690091+0000", 2024-09-21T21:41:11.689 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6549999999999993e-06 2024-09-21T21:41:11.689 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.689 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.689 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:11.689 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.690098+0000", 2024-09-21T21:41:11.689 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7710000000000001e-06 2024-09-21T21:41:11.690 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.690 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.690 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:11.690 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.709913+0000", 2024-09-21T21:41:11.690 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.019815510000000001 2024-09-21T21:41:11.690 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.690 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.690 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:11.690 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.709991+0000", 2024-09-21T21:41:11.690 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7296e-05 2024-09-21T21:41:11.690 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.690 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.690 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:11.691 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.710096+0000", 2024-09-21T21:41:11.691 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000105145 2024-09-21T21:41:11.691 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:11.691 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:11.691 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:11.691 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.691 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.691 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:8452 3.3 3:c030d67f:::rbd_data.112145fae779.0000000000000003:a [read 9510912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:41:11.691 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.690109+0000", 2024-09-21T21:41:11.691 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.915446683000001, 2024-09-21T21:41:11.691 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.020107368, 2024-09-21T21:41:11.691 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:11.691 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:11.691 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:11.692 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:11.692 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:11.692 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:11.692 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 8452 2024-09-21T21:41:11.692 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.692 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:11.692 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.692 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:11.692 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.690109+0000", 2024-09-21T21:41:11.692 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:11.692 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.692 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.692 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:11.693 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.690109+0000", 2024-09-21T21:41:11.693 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:11.693 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.693 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.693 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:11.693 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.690109+0000", 2024-09-21T21:41:11.693 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2899999999999999e-07 2024-09-21T21:41:11.693 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.693 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.693 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:11.693 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.690111+0000", 2024-09-21T21:41:11.693 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2409999999999998e-06 2024-09-21T21:41:11.693 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.693 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.694 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:11.694 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.690118+0000", 2024-09-21T21:41:11.694 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8850000000000001e-06 2024-09-21T21:41:11.694 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.694 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.694 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:11.694 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.690124+0000", 2024-09-21T21:41:11.694 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5589999999999996e-06 2024-09-21T21:41:11.694 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.694 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.694 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:11.694 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.710126+0000", 2024-09-21T21:41:11.694 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.020002217999999999 2024-09-21T21:41:11.694 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.694 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.695 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:11.695 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.710177+0000", 2024-09-21T21:41:11.695 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0587999999999997e-05 2024-09-21T21:41:11.695 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:11.695 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:11.695 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:11.695 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.710216+0000", 2024-09-21T21:41:11.695 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9447999999999999e-05 2024-09-21T21:41:11.695 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:11.695 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:11.695 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:11.695 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:11.695 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:11.695 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:11.697 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T21:41:11.701 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:11.699+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:11.802 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:11.801+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:11.842 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:11.842 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:41:11.842 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-21T21:41:11.842 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:11.850 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T21:41:11.903 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:11.902+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:11.969 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T21:41:11.969 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:11.969 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:41:11.970 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:41:11.970 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:41:11.970 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:11.979 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T21:41:12.021 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:12.020+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:12.090 INFO:teuthology.orchestra.run.smithi164.stdout:checking clone #11, image image_client.0-clone11 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent12 2024-09-21T21:41:12.091 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0: {} 2024-09-21T21:41:12.092 INFO:teuthology.orchestra.run.smithi081.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T21:41:12.105 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:12.103+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:12.108 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1: {} 2024-09-21T21:41:12.109 INFO:teuthology.orchestra.run.smithi081.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T21:41:12.125 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2: {} 2024-09-21T21:41:12.126 INFO:teuthology.orchestra.run.smithi081.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T21:41:12.129 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:12.129 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-21T21:41:12.129 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-21T21:41:12.129 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:41:12.130 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.130 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1484 3.16 3:6a41a830:::rbd_data.112145fae779.000000000000000a:a [read 303104~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:12.130 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634086+0000", 2024-09-21T21:41:12.130 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 10.491418187000001, 2024-09-21T21:41:12.130 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.058257055000000002, 2024-09-21T21:41:12.130 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:12.130 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:12.130 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:12.130 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:12.130 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:12.130 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:12.130 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1484 2024-09-21T21:41:12.130 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.131 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:12.131 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.131 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:12.131 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634086+0000", 2024-09-21T21:41:12.131 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:12.131 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.131 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.131 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:12.131 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634086+0000", 2024-09-21T21:41:12.131 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:12.131 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.131 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.131 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:12.131 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634086+0000", 2024-09-21T21:41:12.132 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6100000000000001e-07 2024-09-21T21:41:12.132 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.132 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.132 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:12.132 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634088+0000", 2024-09-21T21:41:12.132 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1100000000000001e-06 2024-09-21T21:41:12.132 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.132 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.132 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:12.132 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634095+0000", 2024-09-21T21:41:12.132 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4080000000000003e-06 2024-09-21T21:41:12.132 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.132 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.132 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:12.132 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634100+0000", 2024-09-21T21:41:12.133 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0730000000000004e-06 2024-09-21T21:41:12.133 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.133 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.133 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:12.133 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.692197+0000", 2024-09-21T21:41:12.133 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.058097331000000002 2024-09-21T21:41:12.133 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.133 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.133 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:12.133 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.692271+0000", 2024-09-21T21:41:12.133 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3423999999999994e-05 2024-09-21T21:41:12.133 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.133 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.134 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:12.134 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.692343+0000", 2024-09-21T21:41:12.134 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2248000000000004e-05 2024-09-21T21:41:12.134 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:12.134 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:12.134 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:12.134 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.134 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.134 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1485 3.16 3:695c582e:::rbd_data.1121e540c096.0000000000000008:b [read 335872~36864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:12.134 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634106+0000", 2024-09-21T21:41:12.134 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 10.491398390000001, 2024-09-21T21:41:12.134 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059223685999999998, 2024-09-21T21:41:12.134 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:12.134 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:12.135 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:12.135 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:12.135 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:12.135 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:12.135 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1485 2024-09-21T21:41:12.135 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.135 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:12.135 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.135 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:12.135 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634106+0000", 2024-09-21T21:41:12.135 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:12.135 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.135 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.136 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:12.136 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634106+0000", 2024-09-21T21:41:12.136 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:12.136 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.136 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.136 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:12.136 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634106+0000", 2024-09-21T21:41:12.136 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3300000000000002e-07 2024-09-21T21:41:12.136 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.136 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.136 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:12.136 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634108+0000", 2024-09-21T21:41:12.136 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.996e-06 2024-09-21T21:41:12.136 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.136 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.137 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:12.137 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634115+0000", 2024-09-21T21:41:12.137 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5139999999999998e-06 2024-09-21T21:41:12.137 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.137 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.137 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:12.137 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634120+0000", 2024-09-21T21:41:12.137 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1050000000000003e-06 2024-09-21T21:41:12.137 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.137 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.137 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:12.137 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.692374+0000", 2024-09-21T21:41:12.137 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.058254729999999998 2024-09-21T21:41:12.137 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.137 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.138 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:12.138 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.692440+0000", 2024-09-21T21:41:12.138 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6076999999999998e-05 2024-09-21T21:41:12.138 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.138 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.138 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:12.138 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693329+0000", 2024-09-21T21:41:12.138 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00088873100000000003 2024-09-21T21:41:12.138 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:12.138 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:12.138 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:12.138 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.138 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.138 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1486 3.16 3:6a41a830:::rbd_data.112145fae779.000000000000000a:a [read 360448~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:12.138 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634126+0000", 2024-09-21T21:41:12.139 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 10.491378098, 2024-09-21T21:41:12.139 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059399528, 2024-09-21T21:41:12.139 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:12.139 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:12.139 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:12.139 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:12.139 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:12.139 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:12.139 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1486 2024-09-21T21:41:12.139 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.139 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:12.139 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.139 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:12.139 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634126+0000", 2024-09-21T21:41:12.139 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:12.139 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.140 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.140 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:12.140 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634126+0000", 2024-09-21T21:41:12.140 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:12.140 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.140 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.140 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:12.140 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634126+0000", 2024-09-21T21:41:12.140 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4099999999999999e-07 2024-09-21T21:41:12.140 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.140 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.140 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:12.140 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634128+0000", 2024-09-21T21:41:12.140 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1069999999999999e-06 2024-09-21T21:41:12.141 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.141 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.141 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:12.141 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634135+0000", 2024-09-21T21:41:12.141 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.212e-06 2024-09-21T21:41:12.141 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.141 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.141 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:12.141 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634140+0000", 2024-09-21T21:41:12.141 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1429999999999997e-06 2024-09-21T21:41:12.141 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.141 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.141 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:12.141 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693363+0000", 2024-09-21T21:41:12.142 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059223614000000001 2024-09-21T21:41:12.142 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.142 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.142 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:12.142 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693431+0000", 2024-09-21T21:41:12.142 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7294999999999999e-05 2024-09-21T21:41:12.142 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.142 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.142 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:12.142 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693525+0000", 2024-09-21T21:41:12.142 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.4716000000000006e-05 2024-09-21T21:41:12.142 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:12.142 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:12.142 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:12.142 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.142 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.143 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1499 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 49152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:12.143 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634186+0000", 2024-09-21T21:41:12.143 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 10.491318221, 2024-09-21T21:41:12.143 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059339256999999999, 2024-09-21T21:41:12.143 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:12.143 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:12.143 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:12.143 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:12.143 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:12.143 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:12.143 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1499 2024-09-21T21:41:12.143 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.143 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:12.143 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.144 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:12.144 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634186+0000", 2024-09-21T21:41:12.144 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:12.144 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.144 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.144 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:12.144 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634186+0000", 2024-09-21T21:41:12.144 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:12.144 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.144 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.144 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:12.144 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634186+0000", 2024-09-21T21:41:12.144 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0999999999999999e-07 2024-09-21T21:41:12.144 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.144 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.145 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:12.145 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634188+0000", 2024-09-21T21:41:12.145 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.86e-06 2024-09-21T21:41:12.145 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.145 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.145 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:12.145 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634194+0000", 2024-09-21T21:41:12.145 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1109999999999996e-06 2024-09-21T21:41:12.145 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.145 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.145 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:12.145 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634199+0000", 2024-09-21T21:41:12.145 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7419999999999997e-06 2024-09-21T21:41:12.145 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.145 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.146 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:12.146 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693357+0000", 2024-09-21T21:41:12.146 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059157847 2024-09-21T21:41:12.146 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.146 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.146 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:12.146 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693430+0000", 2024-09-21T21:41:12.146 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3203999999999999e-05 2024-09-21T21:41:12.146 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.146 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.146 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:12.146 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693525+0000", 2024-09-21T21:41:12.147 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5082999999999997e-05 2024-09-21T21:41:12.147 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:12.147 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:12.147 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:12.147 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.147 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.147 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1501 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 106496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:12.147 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634205+0000", 2024-09-21T21:41:12.147 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 10.491299131, 2024-09-21T21:41:12.147 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059482394000000001, 2024-09-21T21:41:12.147 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:12.147 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:12.147 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:12.147 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:12.147 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:12.148 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:12.148 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1501 2024-09-21T21:41:12.148 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.148 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:12.148 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.148 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:12.148 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634205+0000", 2024-09-21T21:41:12.148 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:12.148 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.148 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.148 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:12.148 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634205+0000", 2024-09-21T21:41:12.148 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:12.148 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.148 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.149 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:12.149 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634205+0000", 2024-09-21T21:41:12.149 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.03e-07 2024-09-21T21:41:12.149 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.149 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.149 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:12.149 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634207+0000", 2024-09-21T21:41:12.149 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0389999999999999e-06 2024-09-21T21:41:12.149 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.149 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.149 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:12.149 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634213+0000", 2024-09-21T21:41:12.149 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2600000000000002e-06 2024-09-21T21:41:12.149 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.149 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.150 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:12.150 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634218+0000", 2024-09-21T21:41:12.150 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8570000000000003e-06 2024-09-21T21:41:12.150 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.150 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.150 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:12.150 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693552+0000", 2024-09-21T21:41:12.150 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059333636000000002 2024-09-21T21:41:12.150 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.150 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.150 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:12.150 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693613+0000", 2024-09-21T21:41:12.150 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0560000000000003e-05 2024-09-21T21:41:12.150 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.150 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.151 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:12.151 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693687+0000", 2024-09-21T21:41:12.151 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4639000000000001e-05 2024-09-21T21:41:12.151 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:12.151 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:12.151 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:12.151 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.151 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.151 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1515 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 434176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:12.151 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634318+0000", 2024-09-21T21:41:12.151 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 10.491185443999999, 2024-09-21T21:41:12.151 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059565812000000003, 2024-09-21T21:41:12.151 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:12.151 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:12.152 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:12.152 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:12.152 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:12.152 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:12.152 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1515 2024-09-21T21:41:12.152 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.152 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:12.152 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.152 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:12.152 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634318+0000", 2024-09-21T21:41:12.152 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:12.152 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.152 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.152 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:12.152 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634318+0000", 2024-09-21T21:41:12.153 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:12.153 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.153 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.153 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:12.153 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634319+0000", 2024-09-21T21:41:12.153 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9400000000000001e-07 2024-09-21T21:41:12.153 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.153 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.153 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:12.153 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634321+0000", 2024-09-21T21:41:12.153 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.154e-06 2024-09-21T21:41:12.153 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.153 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.153 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:12.153 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634327+0000", 2024-09-21T21:41:12.153 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9830000000000001e-06 2024-09-21T21:41:12.154 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.154 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.154 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:12.154 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634332+0000", 2024-09-21T21:41:12.154 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9080000000000003e-06 2024-09-21T21:41:12.154 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.154 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.154 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:12.154 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693717+0000", 2024-09-21T21:41:12.154 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059385447000000001 2024-09-21T21:41:12.154 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.154 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.154 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:12.154 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693795+0000", 2024-09-21T21:41:12.155 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7267000000000004e-05 2024-09-21T21:41:12.155 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.155 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.155 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:12.155 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693884+0000", 2024-09-21T21:41:12.155 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9659000000000003e-05 2024-09-21T21:41:12.155 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:12.155 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:12.155 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:12.155 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.155 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.155 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1517 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 491520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:12.155 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634338+0000", 2024-09-21T21:41:12.155 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 10.491165670999999, 2024-09-21T21:41:12.156 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060100745999999997, 2024-09-21T21:41:12.156 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:12.156 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:12.156 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:12.156 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:12.156 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:12.156 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:12.156 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1517 2024-09-21T21:41:12.156 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.156 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:12.156 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.156 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:12.156 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634338+0000", 2024-09-21T21:41:12.156 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:12.156 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.157 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.157 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:12.157 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634338+0000", 2024-09-21T21:41:12.157 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:12.157 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.157 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.157 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:12.157 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634339+0000", 2024-09-21T21:41:12.157 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2899999999999999e-07 2024-09-21T21:41:12.157 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.157 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.157 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:12.157 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634341+0000", 2024-09-21T21:41:12.158 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.8959999999999999e-06 2024-09-21T21:41:12.158 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.158 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.158 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:12.158 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634347+0000", 2024-09-21T21:41:12.158 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.302e-06 2024-09-21T21:41:12.158 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.158 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.158 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:12.158 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634351+0000", 2024-09-21T21:41:12.158 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6070000000000001e-06 2024-09-21T21:41:12.158 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.158 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.158 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:12.158 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694279+0000", 2024-09-21T21:41:12.159 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059927046999999997 2024-09-21T21:41:12.159 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.159 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.159 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:12.159 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694355+0000", 2024-09-21T21:41:12.159 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6483999999999995e-05 2024-09-21T21:41:12.159 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.159 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.159 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:12.159 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694439+0000", 2024-09-21T21:41:12.159 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3980999999999994e-05 2024-09-21T21:41:12.159 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:12.159 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:12.159 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:12.160 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.160 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.160 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1518 3.16 3:6a41a830:::rbd_data.112145fae779.000000000000000a:a [read 294912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:12.160 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634357+0000", 2024-09-21T21:41:12.160 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 10.491146765, 2024-09-21T21:41:12.160 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059712318, 2024-09-21T21:41:12.160 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:12.160 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:12.160 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:12.160 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:12.160 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:12.160 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:12.160 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1518 2024-09-21T21:41:12.160 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.160 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:12.160 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.161 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:12.161 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634357+0000", 2024-09-21T21:41:12.161 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:12.161 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.161 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.161 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:12.161 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634357+0000", 2024-09-21T21:41:12.161 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:12.161 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.161 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.161 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:12.161 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634358+0000", 2024-09-21T21:41:12.161 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2100000000000002e-07 2024-09-21T21:41:12.161 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.162 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.162 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:12.162 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634360+0000", 2024-09-21T21:41:12.162 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0940000000000002e-06 2024-09-21T21:41:12.162 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.162 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.162 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:12.162 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634366+0000", 2024-09-21T21:41:12.162 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3389999999999998e-06 2024-09-21T21:41:12.162 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.162 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.162 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:12.162 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634371+0000", 2024-09-21T21:41:12.162 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6670000000000004e-06 2024-09-21T21:41:12.163 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.163 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.163 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:12.163 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693910+0000", 2024-09-21T21:41:12.163 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059538973000000002 2024-09-21T21:41:12.163 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.163 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.163 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:12.163 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693982+0000", 2024-09-21T21:41:12.163 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2806999999999996e-05 2024-09-21T21:41:12.163 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.163 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.163 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:12.163 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694069+0000", 2024-09-21T21:41:12.163 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7016999999999997e-05 2024-09-21T21:41:12.164 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:12.164 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:12.164 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:12.164 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.164 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.164 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1519 3.16 3:6a41a830:::rbd_data.112145fae779.000000000000000a:a [read 352256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:12.164 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634377+0000", 2024-09-21T21:41:12.164 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 10.491127063, 2024-09-21T21:41:12.164 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059872213000000001, 2024-09-21T21:41:12.164 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:12.164 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:12.164 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:12.164 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:12.165 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:12.165 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:12.165 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1519 2024-09-21T21:41:12.165 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.165 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:12.165 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.165 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:12.165 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634377+0000", 2024-09-21T21:41:12.165 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:12.165 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.165 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.165 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:12.165 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634377+0000", 2024-09-21T21:41:12.165 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:12.166 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.166 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.166 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:12.166 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634377+0000", 2024-09-21T21:41:12.166 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2600000000000002e-07 2024-09-21T21:41:12.166 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.166 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.166 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:12.166 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634379+0000", 2024-09-21T21:41:12.166 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0729999999999999e-06 2024-09-21T21:41:12.166 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.166 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.166 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:12.166 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634386+0000", 2024-09-21T21:41:12.167 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4690000000000002e-06 2024-09-21T21:41:12.167 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.167 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.167 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:12.167 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634390+0000", 2024-09-21T21:41:12.167 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4580000000000003e-06 2024-09-21T21:41:12.167 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.167 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.167 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:12.167 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694094+0000", 2024-09-21T21:41:12.167 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059703830999999999 2024-09-21T21:41:12.167 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.167 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.167 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:12.168 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694168+0000", 2024-09-21T21:41:12.168 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4222999999999997e-05 2024-09-21T21:41:12.168 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.168 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.168 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:12.168 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694249+0000", 2024-09-21T21:41:12.168 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0632999999999999e-05 2024-09-21T21:41:12.168 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:12.168 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:12.168 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:12.168 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.168 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.168 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1529 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 53248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:12.168 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634397+0000", 2024-09-21T21:41:12.168 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 10.491106911999999, 2024-09-21T21:41:12.169 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060189919000000001, 2024-09-21T21:41:12.169 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:12.169 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:12.169 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:12.169 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:12.169 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:12.169 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:12.169 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1529 2024-09-21T21:41:12.169 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.169 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:12.169 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.169 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:12.169 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634397+0000", 2024-09-21T21:41:12.169 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:12.169 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.170 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.170 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:12.170 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634397+0000", 2024-09-21T21:41:12.170 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:12.170 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.170 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.170 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:12.170 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634397+0000", 2024-09-21T21:41:12.170 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2e-07 2024-09-21T21:41:12.170 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.170 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.170 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:12.170 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634399+0000", 2024-09-21T21:41:12.171 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0609999999999998e-06 2024-09-21T21:41:12.171 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.171 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.171 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:12.171 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634406+0000", 2024-09-21T21:41:12.171 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7259999999999996e-06 2024-09-21T21:41:12.171 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.171 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.171 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:12.171 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634411+0000", 2024-09-21T21:41:12.171 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9339999999999997e-06 2024-09-21T21:41:12.171 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.171 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.171 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:12.171 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694480+0000", 2024-09-21T21:41:12.172 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060069105999999997 2024-09-21T21:41:12.172 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.172 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.172 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:12.172 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694527+0000", 2024-09-21T21:41:12.172 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7120000000000003e-05 2024-09-21T21:41:12.172 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.172 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.172 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:12.172 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694587+0000", 2024-09-21T21:41:12.172 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9552000000000002e-05 2024-09-21T21:41:12.172 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:12.172 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:12.172 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:12.173 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.173 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.173 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1531 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 110592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:12.173 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634417+0000", 2024-09-21T21:41:12.173 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 10.491086978, 2024-09-21T21:41:12.173 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060347287999999999, 2024-09-21T21:41:12.173 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:12.173 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:12.173 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:12.173 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:12.173 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:12.173 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:12.173 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1531 2024-09-21T21:41:12.173 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.174 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:12.174 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.174 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:12.174 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634417+0000", 2024-09-21T21:41:12.174 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:12.174 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.174 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.174 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:12.174 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634417+0000", 2024-09-21T21:41:12.174 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:12.174 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.174 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.174 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:12.174 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634417+0000", 2024-09-21T21:41:12.175 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1699999999999999e-07 2024-09-21T21:41:12.175 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.175 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.175 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:12.175 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634420+0000", 2024-09-21T21:41:12.175 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.243e-06 2024-09-21T21:41:12.175 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.175 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.175 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:12.175 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634426+0000", 2024-09-21T21:41:12.175 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2550000000000003e-06 2024-09-21T21:41:12.175 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.175 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.175 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:12.175 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634431+0000", 2024-09-21T21:41:12.176 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0150000000000003e-06 2024-09-21T21:41:12.176 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.176 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.176 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:12.176 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694618+0000", 2024-09-21T21:41:12.176 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060186836000000001 2024-09-21T21:41:12.176 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.176 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.176 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:12.176 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694685+0000", 2024-09-21T21:41:12.176 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6970000000000004e-05 2024-09-21T21:41:12.176 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.176 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.176 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:12.177 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694764+0000", 2024-09-21T21:41:12.177 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9552e-05 2024-09-21T21:41:12.177 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:12.177 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:12.177 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:12.177 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.177 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.177 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1532 3.e 3:724b31a3:::rbd_data.1121e540c096.0000000000000015:b [read 151552~36864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:12.177 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634437+0000", 2024-09-21T21:41:12.177 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 10.491067182, 2024-09-21T21:41:12.177 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060474314000000001, 2024-09-21T21:41:12.177 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:12.177 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:12.177 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:12.178 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:12.178 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:12.178 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:12.178 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1532 2024-09-21T21:41:12.178 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.178 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:12.178 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.178 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:12.178 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634437+0000", 2024-09-21T21:41:12.178 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:12.178 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.178 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.178 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:12.178 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634437+0000", 2024-09-21T21:41:12.179 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:12.179 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.179 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.179 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:12.179 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634437+0000", 2024-09-21T21:41:12.179 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9400000000000001e-07 2024-09-21T21:41:12.179 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.179 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.179 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:12.179 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634439+0000", 2024-09-21T21:41:12.179 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.114e-06 2024-09-21T21:41:12.179 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.179 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.179 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:12.180 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634446+0000", 2024-09-21T21:41:12.180 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1389999999999998e-06 2024-09-21T21:41:12.180 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.180 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.180 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:12.180 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634451+0000", 2024-09-21T21:41:12.180 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0010000000000001e-06 2024-09-21T21:41:12.180 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.180 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.180 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:12.180 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694794+0000", 2024-09-21T21:41:12.180 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060342509000000003 2024-09-21T21:41:12.180 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.180 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.181 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:12.181 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694835+0000", 2024-09-21T21:41:12.181 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0728e-05 2024-09-21T21:41:12.181 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.181 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.181 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:12.181 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694911+0000", 2024-09-21T21:41:12.181 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6428999999999996e-05 2024-09-21T21:41:12.181 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:12.181 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:12.181 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:12.181 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.181 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.181 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1534 3.e 3:724b31a3:::rbd_data.1121e540c096.0000000000000015:b [read 487424~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:12.182 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634477+0000", 2024-09-21T21:41:12.182 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 10.491026779, 2024-09-21T21:41:12.182 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060624009, 2024-09-21T21:41:12.182 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:12.182 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:12.182 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:12.182 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:12.182 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:12.182 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:12.182 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1534 2024-09-21T21:41:12.182 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.182 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:12.182 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.182 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:12.183 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634477+0000", 2024-09-21T21:41:12.183 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:12.183 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.183 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.183 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:12.183 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634477+0000", 2024-09-21T21:41:12.183 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:12.183 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.183 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.183 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:12.183 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634478+0000", 2024-09-21T21:41:12.183 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1399999999999997e-07 2024-09-21T21:41:12.183 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.183 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.184 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:12.184 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634480+0000", 2024-09-21T21:41:12.184 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1009999999999999e-06 2024-09-21T21:41:12.184 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.184 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.184 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:12.184 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634486+0000", 2024-09-21T21:41:12.184 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3289999999999999e-06 2024-09-21T21:41:12.184 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.184 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.184 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:12.184 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634491+0000", 2024-09-21T21:41:12.184 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6260000000000003e-06 2024-09-21T21:41:12.184 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.184 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.185 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:12.185 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694943+0000", 2024-09-21T21:41:12.185 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060451960999999999 2024-09-21T21:41:12.185 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.185 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.185 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:12.185 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695014+0000", 2024-09-21T21:41:12.185 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1587000000000004e-05 2024-09-21T21:41:12.185 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.185 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.185 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:12.185 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695101+0000", 2024-09-21T21:41:12.185 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6990999999999994e-05 2024-09-21T21:41:12.185 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:12.186 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:12.186 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:12.186 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.186 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.186 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1545 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 438272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:12.186 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634515+0000", 2024-09-21T21:41:12.186 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 10.490988579, 2024-09-21T21:41:12.186 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060707944, 2024-09-21T21:41:12.186 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:12.186 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:12.186 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:12.186 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:12.186 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:12.186 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:12.187 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1545 2024-09-21T21:41:12.187 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.187 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:12.187 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.187 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:12.187 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634515+0000", 2024-09-21T21:41:12.187 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:12.187 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.187 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.187 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:12.187 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634515+0000", 2024-09-21T21:41:12.187 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:12.187 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.188 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.188 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:12.188 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634516+0000", 2024-09-21T21:41:12.188 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0999999999999999e-07 2024-09-21T21:41:12.188 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.188 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.188 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:12.188 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634518+0000", 2024-09-21T21:41:12.188 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.846e-06 2024-09-21T21:41:12.188 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.188 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.188 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:12.188 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634524+0000", 2024-09-21T21:41:12.189 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2550000000000003e-06 2024-09-21T21:41:12.189 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.189 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.189 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:12.189 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634529+0000", 2024-09-21T21:41:12.189 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6940000000000003e-06 2024-09-21T21:41:12.189 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.189 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.189 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:12.189 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695127+0000", 2024-09-21T21:41:12.189 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060598921 2024-09-21T21:41:12.189 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.189 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.189 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:12.190 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695171+0000", 2024-09-21T21:41:12.190 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3588999999999999e-05 2024-09-21T21:41:12.190 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.190 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.190 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:12.190 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695223+0000", 2024-09-21T21:41:12.190 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2228999999999997e-05 2024-09-21T21:41:12.190 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:12.190 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:12.190 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:12.190 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.190 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.190 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1547 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 495616~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:12.190 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634534+0000", 2024-09-21T21:41:12.191 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 10.490969787999999, 2024-09-21T21:41:12.191 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060881131999999998, 2024-09-21T21:41:12.191 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:12.191 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:12.191 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:12.191 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:12.191 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:12.191 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:12.191 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1547 2024-09-21T21:41:12.191 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.191 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:12.191 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.191 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:12.191 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634534+0000", 2024-09-21T21:41:12.192 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:12.192 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.192 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.192 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:12.192 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634534+0000", 2024-09-21T21:41:12.192 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:12.192 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.192 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.192 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:12.192 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634535+0000", 2024-09-21T21:41:12.192 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.77e-07 2024-09-21T21:41:12.192 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.192 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.192 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:12.193 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634542+0000", 2024-09-21T21:41:12.193 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9000000000000006e-06 2024-09-21T21:41:12.193 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.193 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.193 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:12.193 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634549+0000", 2024-09-21T21:41:12.193 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5910000000000001e-06 2024-09-21T21:41:12.193 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.193 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.193 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:12.193 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634554+0000", 2024-09-21T21:41:12.193 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7940000000000002e-06 2024-09-21T21:41:12.193 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.193 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.194 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:12.194 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695256+0000", 2024-09-21T21:41:12.194 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.06070246 2024-09-21T21:41:12.194 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.194 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.194 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:12.194 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695332+0000", 2024-09-21T21:41:12.194 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6248000000000006e-05 2024-09-21T21:41:12.194 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.194 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.194 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:12.194 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695415+0000", 2024-09-21T21:41:12.194 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2762000000000005e-05 2024-09-21T21:41:12.195 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:12.195 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:12.195 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:12.195 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.195 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.195 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1548 3.16 3:6a41a830:::rbd_data.112145fae779.000000000000000a:a [read 299008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:12.195 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634561+0000", 2024-09-21T21:41:12.195 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 10.490943210999999, 2024-09-21T21:41:12.195 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060786743999999997, 2024-09-21T21:41:12.195 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:12.195 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:12.195 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:12.195 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:12.195 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:12.196 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:12.196 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1548 2024-09-21T21:41:12.196 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.196 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:12.196 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.196 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:12.196 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634561+0000", 2024-09-21T21:41:12.196 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:12.196 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.196 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.196 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:12.196 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634561+0000", 2024-09-21T21:41:12.196 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:12.196 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.197 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.197 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:12.197 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634561+0000", 2024-09-21T21:41:12.197 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5199999999999997e-07 2024-09-21T21:41:12.197 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.197 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.197 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:12.197 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634564+0000", 2024-09-21T21:41:12.197 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4040000000000002e-06 2024-09-21T21:41:12.197 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.197 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.197 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:12.197 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634571+0000", 2024-09-21T21:41:12.197 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4379999999999998e-06 2024-09-21T21:41:12.197 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.198 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.198 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:12.198 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634576+0000", 2024-09-21T21:41:12.198 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1399999999999999e-06 2024-09-21T21:41:12.198 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.198 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.198 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:12.198 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695247+0000", 2024-09-21T21:41:12.198 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060670397000000001 2024-09-21T21:41:12.198 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.198 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.198 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:12.198 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695299+0000", 2024-09-21T21:41:12.198 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2564e-05 2024-09-21T21:41:12.199 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.199 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.199 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:12.199 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695347+0000", 2024-09-21T21:41:12.199 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8248999999999998e-05 2024-09-21T21:41:12.199 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:12.199 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:12.199 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:12.199 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.199 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.199 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1549 3.16 3:6a41a830:::rbd_data.112145fae779.000000000000000a:a [read 356352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:12.199 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634583+0000", 2024-09-21T21:41:12.199 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 10.490921298, 2024-09-21T21:41:12.199 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060889459, 2024-09-21T21:41:12.200 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:12.200 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:12.200 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:12.200 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:12.200 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:12.200 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:12.200 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1549 2024-09-21T21:41:12.200 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.200 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:12.200 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.200 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:12.200 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634583+0000", 2024-09-21T21:41:12.200 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:12.200 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.201 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.201 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:12.201 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634583+0000", 2024-09-21T21:41:12.201 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:12.201 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.201 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.201 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:12.201 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634583+0000", 2024-09-21T21:41:12.201 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1899999999999998e-07 2024-09-21T21:41:12.201 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.201 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.201 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:12.201 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634585+0000", 2024-09-21T21:41:12.202 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2129999999999998e-06 2024-09-21T21:41:12.202 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.202 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.202 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:12.202 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634592+0000", 2024-09-21T21:41:12.202 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2840000000000003e-06 2024-09-21T21:41:12.202 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.202 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.202 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:12.202 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634596+0000", 2024-09-21T21:41:12.202 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9309999999999999e-06 2024-09-21T21:41:12.202 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.202 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.202 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:12.202 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695371+0000", 2024-09-21T21:41:12.203 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060774425999999999 2024-09-21T21:41:12.203 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.203 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.203 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:12.203 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695419+0000", 2024-09-21T21:41:12.203 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8607999999999999e-05 2024-09-21T21:41:12.203 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.203 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.203 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:12.203 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695472+0000", 2024-09-21T21:41:12.203 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2577999999999997e-05 2024-09-21T21:41:12.203 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:12.203 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:12.203 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:12.204 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.204 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.204 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1588 3.16 3:695c582e:::rbd_data.1121e540c096.0000000000000008:b [read 147456~40960] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:12.204 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.636942+0000", 2024-09-21T21:41:12.204 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 10.488561520999999, 2024-09-21T21:41:12.204 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.058643811999999997, 2024-09-21T21:41:12.204 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:12.204 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:12.204 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:12.204 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:12.204 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:12.204 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:12.204 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1588 2024-09-21T21:41:12.204 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.204 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:12.205 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.205 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:12.205 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636942+0000", 2024-09-21T21:41:12.205 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:12.205 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.205 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.205 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:12.205 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636942+0000", 2024-09-21T21:41:12.205 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:12.205 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.205 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.205 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:12.205 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636943+0000", 2024-09-21T21:41:12.205 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4199999999999996e-07 2024-09-21T21:41:12.205 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.206 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.206 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:12.206 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636945+0000", 2024-09-21T21:41:12.206 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.401e-06 2024-09-21T21:41:12.206 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.206 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.206 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:12.206 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636952+0000", 2024-09-21T21:41:12.206 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5379999999999999e-06 2024-09-21T21:41:12.206 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.206 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.206 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:12.206 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636958+0000", 2024-09-21T21:41:12.207 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.489e-06 2024-09-21T21:41:12.207 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.207 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.207 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:12.207 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695497+0000", 2024-09-21T21:41:12.207 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.058538504999999998 2024-09-21T21:41:12.207 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.207 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.207 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:12.207 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695533+0000", 2024-09-21T21:41:12.207 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6118000000000002e-05 2024-09-21T21:41:12.207 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.207 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.207 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:12.208 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695586+0000", 2024-09-21T21:41:12.208 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3219e-05 2024-09-21T21:41:12.208 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:12.208 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:12.208 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:12.208 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.208 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.208 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1589 3.16 3:695c582e:::rbd_data.1121e540c096.0000000000000008:b [read 487424~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:12.208 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.636965+0000", 2024-09-21T21:41:12.208 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 10.48853843, 2024-09-21T21:41:12.208 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.058728152999999998, 2024-09-21T21:41:12.208 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:12.208 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:12.208 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:12.208 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:12.209 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:12.209 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:12.209 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1589 2024-09-21T21:41:12.209 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.209 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:12.209 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.209 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:12.209 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636965+0000", 2024-09-21T21:41:12.209 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:12.209 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.209 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.209 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:12.209 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636965+0000", 2024-09-21T21:41:12.210 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:12.210 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.210 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.210 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:12.210 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636966+0000", 2024-09-21T21:41:12.210 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3799999999999998e-07 2024-09-21T21:41:12.210 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.210 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.210 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:12.210 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636968+0000", 2024-09-21T21:41:12.210 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2400000000000002e-06 2024-09-21T21:41:12.210 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.210 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.210 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:12.211 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636976+0000", 2024-09-21T21:41:12.211 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5050000000000001e-06 2024-09-21T21:41:12.211 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.211 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.211 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:12.211 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636981+0000", 2024-09-21T21:41:12.211 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2170000000000002e-06 2024-09-21T21:41:12.211 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.211 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.211 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:12.211 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695610+0000", 2024-09-21T21:41:12.211 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.058628998000000002 2024-09-21T21:41:12.211 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.211 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.211 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:12.212 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695647+0000", 2024-09-21T21:41:12.212 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6752000000000003e-05 2024-09-21T21:41:12.212 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.212 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.212 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:12.212 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695694+0000", 2024-09-21T21:41:12.212 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7002999999999997e-05 2024-09-21T21:41:12.212 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:12.212 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:12.212 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:12.212 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.212 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.212 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1607 3.e 3:724b31a3:::rbd_data.1121e540c096.0000000000000015:b [read 299008~36864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:12.212 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.637062+0000", 2024-09-21T21:41:12.212 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 10.488442339000001, 2024-09-21T21:41:12.213 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.058623078000000002, 2024-09-21T21:41:12.213 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:12.213 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:12.213 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:12.213 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:12.213 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:12.213 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:12.213 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1607 2024-09-21T21:41:12.213 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.213 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:12.213 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.213 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:12.213 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.637062+0000", 2024-09-21T21:41:12.213 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:12.213 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.214 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.214 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:12.214 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.637062+0000", 2024-09-21T21:41:12.214 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:12.214 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.214 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.214 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:12.214 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.637062+0000", 2024-09-21T21:41:12.214 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0900000000000002e-07 2024-09-21T21:41:12.214 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.214 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.214 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:12.214 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.637069+0000", 2024-09-21T21:41:12.215 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.267e-06 2024-09-21T21:41:12.215 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.215 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.215 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:12.215 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.637076+0000", 2024-09-21T21:41:12.215 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7320000000000001e-06 2024-09-21T21:41:12.215 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.215 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.215 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:12.215 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.637082+0000", 2024-09-21T21:41:12.215 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4760000000000003e-06 2024-09-21T21:41:12.215 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.215 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.215 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:12.215 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695604+0000", 2024-09-21T21:41:12.216 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.058521261999999998 2024-09-21T21:41:12.216 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.216 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.216 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:12.216 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695638+0000", 2024-09-21T21:41:12.216 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4292999999999998e-05 2024-09-21T21:41:12.216 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:12.216 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:12.216 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:12.216 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695685+0000", 2024-09-21T21:41:12.216 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6638999999999998e-05 2024-09-21T21:41:12.216 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:12.216 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:12.216 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:12.217 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:12.217 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:12.217 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:12.218 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3: {} 2024-09-21T21:41:12.218 INFO:teuthology.orchestra.run.smithi081.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T21:41:12.219 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4: {} 2024-09-21T21:41:12.219 INFO:teuthology.orchestra.run.smithi081.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T21:41:12.219 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:12.216+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:12.220 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T21:41:12.220 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5: {} 2024-09-21T21:41:12.221 INFO:teuthology.orchestra.run.smithi081.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T21:41:12.308 INFO:teuthology.orchestra.run.smithi164.stdout:2105 write 0xa3d7c00 thru 0xa3d7d3e (0x13f bytes) 2024-09-21T21:41:12.308 INFO:teuthology.orchestra.run.smithi164.stdout:2106 read 0x4ec6e00 thru 0x4ece9bc (0x7bbd bytes) 2024-09-21T21:41:12.309 INFO:teuthology.orchestra.run.smithi164.stdout:2107 read 0x409d600 thru 0x409dbb0 (0x5b1 bytes) 2024-09-21T21:41:12.309 INFO:teuthology.orchestra.run.smithi164.stdout:2108 read 0x127de00 thru 0x1286ea7 (0x90a8 bytes) 2024-09-21T21:41:12.319 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:12.317+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:12.365 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:12.366 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:12.374 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T21:41:12.420 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:12.418+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:12.477 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:12.477 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:12.486 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T21:41:12.520 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:12.519+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:12.588 INFO:teuthology.orchestra.run.smithi081.stderr:noscrub is set 2024-09-21T21:41:12.592 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:12.592 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:12.602 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T21:41:12.621 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:12.620+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:12.722 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:12.720+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:12.745 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:12.745 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:12.754 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T21:41:12.823 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:12.821+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:12.879 INFO:teuthology.orchestra.run.smithi164.stdout:2116 read 0x79d0000 thru 0x79dd32d (0xd32e bytes) 2024-09-21T21:41:12.882 INFO:teuthology.orchestra.run.smithi164.stdout:2120 trunc from 0xc7f2e00 to 0xc287a00 2024-09-21T21:41:12.901 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:12.901 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:12.912 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T21:41:12.918 INFO:teuthology.orchestra.run.smithi164.stdout:2121 punch from 0x8a2e200 to 0x8a3b400, (0xd200 bytes) 2024-09-21T21:41:12.924 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:12.922+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:12.930 INFO:teuthology.orchestra.run.smithi164.stdout:2123 punch from 0x4a6b000 to 0x4a6b800, (0x800 bytes) 2024-09-21T21:41:12.932 INFO:teuthology.orchestra.run.smithi164.stdout:2124 write 0xaf5e400 thru 0xaf68675 (0xa276 bytes) 2024-09-21T21:41:12.932 INFO:teuthology.orchestra.run.smithi164.stdout:2133 punch from 0x67e4000 to 0x67ea600, (0x6600 bytes) 2024-09-21T21:41:12.952 INFO:teuthology.orchestra.run.smithi164.stdout:2136 write 0x195d800 thru 0x19693cd (0xbbce bytes) 2024-09-21T21:41:12.952 INFO:teuthology.orchestra.run.smithi164.stdout:2137 read 0x129ce00 thru 0x129da13 (0xc14 bytes) 2024-09-21T21:41:12.952 INFO:teuthology.orchestra.run.smithi164.stdout:2139 read 0x95fee00 thru 0x9601319 (0x251a bytes) 2024-09-21T21:41:12.952 INFO:teuthology.orchestra.run.smithi164.stdout:2141 write 0x643a200 thru 0x6449846 (0xf647 bytes) 2024-09-21T21:41:12.952 INFO:teuthology.orchestra.run.smithi164.stdout:2142 read 0xacfe600 thru 0xad041a6 (0x5ba7 bytes) 2024-09-21T21:41:12.969 INFO:teuthology.orchestra.run.smithi164.stdout:2143 trunc from 0xc287a00 to 0x1f2aa00 2024-09-21T21:41:13.025 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:13.024+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:13.057 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:13.057 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:13.068 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T21:41:13.126 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:13.124+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:13.143 INFO:teuthology.orchestra.run.smithi164.stdout:2144 write 0x1503000 thru 0x1505002 (0x2003 bytes) 2024-09-21T21:41:13.145 INFO:teuthology.orchestra.run.smithi164.stdout:2145 write 0x1ac7000 thru 0x1ad1e86 (0xae87 bytes) 2024-09-21T21:41:13.145 INFO:teuthology.orchestra.run.smithi164.stdout:2146 trunc from 0x1f2aa00 to 0x1f85600 2024-09-21T21:41:13.212 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:13.212 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:13.223 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T21:41:13.227 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:13.225+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:13.238 INFO:teuthology.orchestra.run.smithi164.stdout:2147 write 0x8d03800 thru 0x8d0e39f (0xaba0 bytes) 2024-09-21T21:41:13.238 INFO:teuthology.orchestra.run.smithi164.stdout:2148 punch from 0x79fa400 to 0x7a06400, (0xc000 bytes) 2024-09-21T21:41:13.243 INFO:teuthology.orchestra.run.smithi164.stdout:2151 trunc from 0x8d0e400 to 0xc528a00 2024-09-21T21:41:13.291 INFO:teuthology.orchestra.run.smithi164.stdout:2152 trunc from 0xc528a00 to 0x3128800 2024-09-21T21:41:13.327 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:13.326+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:13.342 INFO:teuthology.orchestra.run.smithi164.stdout:2154 write 0xadca800 thru 0xadd70ae (0xc8af bytes) 2024-09-21T21:41:13.343 INFO:teuthology.orchestra.run.smithi164.stdout:2155 read 0x4e61600 thru 0x4e677f7 (0x61f8 bytes) 2024-09-21T21:41:13.345 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:13.346 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:13.346 INFO:teuthology.orchestra.run.smithi164.stdout:2156 trunc from 0xadd7200 to 0xd49d600 2024-09-21T21:41:13.358 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T21:41:13.379 INFO:teuthology.orchestra.run.smithi164.stdout:2157 write 0xd6ade00 thru 0xd6afe97 (0x2098 bytes) 2024-09-21T21:41:13.380 INFO:teuthology.orchestra.run.smithi164.stdout:2158 read 0x5425800 thru 0x542687d (0x107e bytes) 2024-09-21T21:41:13.383 INFO:teuthology.orchestra.run.smithi164.stdout:2161 trunc from 0xd6b0000 to 0xb3c0a00 2024-09-21T21:41:13.421 INFO:teuthology.orchestra.run.smithi164.stdout:2164 write 0x5b55800 thru 0x5b6374d (0xdf4e bytes) 2024-09-21T21:41:13.421 INFO:teuthology.orchestra.run.smithi164.stdout:2165 flatten 2024-09-21T21:41:13.444 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:13.442+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:13.528 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:13.526+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:13.541 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:13.541 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:13.552 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T21:41:13.629 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:13.627+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:13.716 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:13.716 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:13.726 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T21:41:13.730 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:13.728+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:13.831 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:13.829+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:13.931 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:13.930+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:13.939 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:13.940 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:13.953 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T21:41:14.011 INFO:teuthology.orchestra.run.smithi164.stdout:2166 write 0xe511800 thru 0xe512b5f (0x1360 bytes) 2024-09-21T21:41:14.013 INFO:teuthology.orchestra.run.smithi164.stdout:2167 write 0xb5ae000 thru 0xb5b0745 (0x2746 bytes) 2024-09-21T21:41:14.013 INFO:teuthology.orchestra.run.smithi164.stdout:2168 write 0x8832e00 thru 0x883b1f5 (0x83f6 bytes) 2024-09-21T21:41:14.013 INFO:teuthology.orchestra.run.smithi164.stdout:2169 punch from 0x85ba800 to 0x85c4000, (0x9800 bytes) 2024-09-21T21:41:14.013 INFO:teuthology.orchestra.run.smithi164.stdout:2170 write 0xc33c400 thru 0xc349093 (0xcc94 bytes) 2024-09-21T21:41:14.013 INFO:teuthology.orchestra.run.smithi164.stdout:2172 read 0x5402000 thru 0x5410b0f (0xeb10 bytes) 2024-09-21T21:41:14.014 INFO:teuthology.orchestra.run.smithi164.stdout:2173 write 0x564a200 thru 0x564ead8 (0x48d9 bytes) 2024-09-21T21:41:14.030 INFO:teuthology.orchestra.run.smithi164.stdout:2175 trunc from 0xe512c00 to 0x881c00 2024-09-21T21:41:14.032 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:14.030+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:14.121 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:14.122 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:14.131 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T21:41:14.132 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:14.130+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:14.217 INFO:teuthology.orchestra.run.smithi164.stdout:2177 read 0x128400 thru 0x133d4e (0xb94f bytes) 2024-09-21T21:41:14.219 INFO:teuthology.orchestra.run.smithi164.stdout:2178 read 0x310e00 thru 0x31785f (0x6a60 bytes) 2024-09-21T21:41:14.221 INFO:teuthology.orchestra.run.smithi164.stdout:2180 trunc from 0x881c00 to 0x2566200 2024-09-21T21:41:14.222 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T21:41:14.232 INFO:teuthology.orchestra.run.smithi164.stdout:2181 read 0x2ebc00 thru 0x2f7fb4 (0xc3b5 bytes) 2024-09-21T21:41:14.232 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:14.231+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:14.236 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:14.236 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:14.246 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T21:41:14.246 INFO:teuthology.orchestra.run.smithi164.stdout:2182 trunc from 0x2566200 to 0x7e7de00 2024-09-21T21:41:14.260 INFO:teuthology.orchestra.run.smithi164.stdout:2186 trunc from 0x7e7de00 to 0xc5e7a00 2024-09-21T21:41:14.270 INFO:teuthology.orchestra.run.smithi164.stdout:2187 read 0xb476200 thru 0xb4791a8 (0x2fa9 bytes) 2024-09-21T21:41:14.277 INFO:teuthology.orchestra.run.smithi164.stdout:2188 read 0x5e0a00 thru 0x5e2376 (0x1977 bytes) 2024-09-21T21:41:14.278 INFO:teuthology.orchestra.run.smithi164.stdout:2189 punch from 0x3691200 to 0x369b000, (0x9e00 bytes) 2024-09-21T21:41:14.287 INFO:teuthology.orchestra.run.smithi164.stdout:2191 trunc from 0xc5e7a00 to 0x4ee7e00 2024-09-21T21:41:14.301 INFO:teuthology.orchestra.run.smithi164.stdout:2192 read 0x1de7400 thru 0x1df52e4 (0xdee5 bytes) 2024-09-21T21:41:14.302 INFO:teuthology.orchestra.run.smithi164.stdout:2195 punch from 0x4441800 to 0x4446200, (0x4a00 bytes) 2024-09-21T21:41:14.302 INFO:teuthology.orchestra.run.smithi164.stdout:2196 read 0x1d6fa00 thru 0x1d732b8 (0x38b9 bytes) 2024-09-21T21:41:14.302 INFO:teuthology.orchestra.run.smithi164.stdout:2197 clone 14 order 18 su 2048 sc 6 2024-09-21T21:41:14.333 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:14.331+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:14.355 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0: {} 2024-09-21T21:41:14.355 INFO:teuthology.orchestra.run.smithi081.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T21:41:14.371 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1: {} 2024-09-21T21:41:14.372 INFO:teuthology.orchestra.run.smithi081.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T21:41:14.390 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2: {} 2024-09-21T21:41:14.390 INFO:teuthology.orchestra.run.smithi081.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T21:41:14.392 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:14.392 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:14.401 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T21:41:14.404 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3: {} 2024-09-21T21:41:14.404 INFO:teuthology.orchestra.run.smithi081.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T21:41:14.420 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4: {} 2024-09-21T21:41:14.421 INFO:teuthology.orchestra.run.smithi081.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T21:41:14.437 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5: {} 2024-09-21T21:41:14.437 INFO:teuthology.orchestra.run.smithi081.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T21:41:14.443 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:14.441+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:14.535 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:14.533+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:14.544 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:14.545 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-21T21:41:14.545 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-21T21:41:14.545 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:41:14.545 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.545 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:8196 3.3 3:c030d67f:::rbd_data.112145fae779.0000000000000003:a [read 9502720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:41:14.545 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.688149+0000", 2024-09-21T21:41:14.545 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 15.854537006999999, 2024-09-21T21:41:14.545 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.018375909999999999, 2024-09-21T21:41:14.545 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:14.545 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:14.545 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:14.545 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:14.545 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:14.545 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:14.546 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 8196 2024-09-21T21:41:14.546 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.546 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:14.546 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.546 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:14.546 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688149+0000", 2024-09-21T21:41:14.546 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:14.546 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.546 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.546 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:14.546 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688149+0000", 2024-09-21T21:41:14.546 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:14.546 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.546 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.547 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:14.547 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688150+0000", 2024-09-21T21:41:14.547 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5900000000000003e-07 2024-09-21T21:41:14.547 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.547 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.547 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:14.547 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688156+0000", 2024-09-21T21:41:14.547 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3160000000000002e-06 2024-09-21T21:41:14.547 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.547 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.547 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:14.547 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688163+0000", 2024-09-21T21:41:14.547 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7920000000000004e-06 2024-09-21T21:41:14.547 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.548 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.548 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:14.548 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688167+0000", 2024-09-21T21:41:14.548 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4360000000000004e-06 2024-09-21T21:41:14.548 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.548 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.548 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:14.548 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.706376+0000", 2024-09-21T21:41:14.548 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.018208762 2024-09-21T21:41:14.548 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.548 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.548 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:14.548 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.706456+0000", 2024-09-21T21:41:14.548 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0065e-05 2024-09-21T21:41:14.549 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.549 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.549 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:14.549 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.706525+0000", 2024-09-21T21:41:14.549 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9179999999999998e-05 2024-09-21T21:41:14.549 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:14.549 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:14.549 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:14.549 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.549 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.549 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:8197 3.3 3:c030d67f:::rbd_data.112145fae779.0000000000000003:a [read 9560064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:41:14.549 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.688173+0000", 2024-09-21T21:41:14.549 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 15.854513574, 2024-09-21T21:41:14.549 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.018539605000000001, 2024-09-21T21:41:14.550 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:14.550 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:14.550 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:14.550 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:14.550 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:14.550 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:14.550 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 8197 2024-09-21T21:41:14.550 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.550 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:14.550 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.550 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:14.550 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688173+0000", 2024-09-21T21:41:14.550 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:14.550 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.550 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.551 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:14.551 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688173+0000", 2024-09-21T21:41:14.551 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:14.551 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.551 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.551 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:14.551 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688173+0000", 2024-09-21T21:41:14.551 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2300000000000002e-07 2024-09-21T21:41:14.551 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.551 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.551 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:14.551 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688175+0000", 2024-09-21T21:41:14.551 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9589999999999998e-06 2024-09-21T21:41:14.552 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.552 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.552 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:14.552 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688182+0000", 2024-09-21T21:41:14.552 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4239999999999998e-06 2024-09-21T21:41:14.552 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.552 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.552 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:14.552 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688186+0000", 2024-09-21T21:41:14.552 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4939999999999997e-06 2024-09-21T21:41:14.552 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.552 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.552 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:14.552 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.706559+0000", 2024-09-21T21:41:14.552 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.018373308000000001 2024-09-21T21:41:14.553 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.553 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.553 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:14.553 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.706642+0000", 2024-09-21T21:41:14.553 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2964000000000001e-05 2024-09-21T21:41:14.553 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.553 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.553 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:14.553 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.706712+0000", 2024-09-21T21:41:14.553 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0032999999999998e-05 2024-09-21T21:41:14.553 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:14.553 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:14.553 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:14.553 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.554 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.554 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:8317 3.3 3:c030d67f:::rbd_data.112145fae779.0000000000000003:a [read 9064448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:41:14.554 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.688286+0000", 2024-09-21T21:41:14.554 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 15.854399903000001, 2024-09-21T21:41:14.554 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.018621219000000001, 2024-09-21T21:41:14.554 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:14.554 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:14.554 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:14.554 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:14.554 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:14.554 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:14.554 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 8317 2024-09-21T21:41:14.554 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.554 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:14.554 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.555 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:14.555 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688286+0000", 2024-09-21T21:41:14.555 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:14.555 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.555 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.555 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:14.555 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688286+0000", 2024-09-21T21:41:14.555 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:14.555 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.556 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.556 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:14.556 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688287+0000", 2024-09-21T21:41:14.556 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6499999999999999e-07 2024-09-21T21:41:14.556 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.556 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.556 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:14.556 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688290+0000", 2024-09-21T21:41:14.556 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1879999999999998e-06 2024-09-21T21:41:14.557 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.557 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.557 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:14.557 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688317+0000", 2024-09-21T21:41:14.557 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7353999999999999e-05 2024-09-21T21:41:14.557 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.557 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.557 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:14.557 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688324+0000", 2024-09-21T21:41:14.557 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2910000000000004e-06 2024-09-21T21:41:14.557 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.557 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.557 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:14.557 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.706744+0000", 2024-09-21T21:41:14.558 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.018420414 2024-09-21T21:41:14.558 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.558 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.558 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:14.558 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.706826+0000", 2024-09-21T21:41:14.558 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2286999999999993e-05 2024-09-21T21:41:14.558 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.558 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.558 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:14.558 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.706908+0000", 2024-09-21T21:41:14.558 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1119999999999996e-05 2024-09-21T21:41:14.558 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:14.558 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:14.558 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:14.559 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.559 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.559 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:8318 3.3 3:c030d67f:::rbd_data.112145fae779.0000000000000003:a [read 9121792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:41:14.559 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.688335+0000", 2024-09-21T21:41:14.559 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 15.854351189999999, 2024-09-21T21:41:14.559 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.018752181999999999, 2024-09-21T21:41:14.559 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:14.559 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:14.559 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:14.559 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:14.559 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:14.559 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:14.559 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 8318 2024-09-21T21:41:14.559 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.560 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:14.560 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.560 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:14.560 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688335+0000", 2024-09-21T21:41:14.560 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:14.560 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.560 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.560 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:14.560 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688335+0000", 2024-09-21T21:41:14.560 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:14.560 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.560 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.560 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:14.560 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688336+0000", 2024-09-21T21:41:14.561 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5000000000000003e-07 2024-09-21T21:41:14.561 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.561 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.561 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:14.561 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688339+0000", 2024-09-21T21:41:14.561 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.128e-06 2024-09-21T21:41:14.561 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.561 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.561 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:14.561 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688347+0000", 2024-09-21T21:41:14.561 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5920000000000001e-06 2024-09-21T21:41:14.561 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.561 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.561 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:14.562 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688353+0000", 2024-09-21T21:41:14.562 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6570000000000002e-06 2024-09-21T21:41:14.562 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.562 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.562 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:14.562 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.706940+0000", 2024-09-21T21:41:14.562 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.018586689 2024-09-21T21:41:14.562 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.562 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.562 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:14.562 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.707015+0000", 2024-09-21T21:41:14.562 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4888999999999994e-05 2024-09-21T21:41:14.562 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.562 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.563 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:14.563 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.707087+0000", 2024-09-21T21:41:14.563 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2676999999999996e-05 2024-09-21T21:41:14.563 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:14.563 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:14.563 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:14.563 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.563 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.563 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:8319 3.3 3:c030d67f:::rbd_data.112145fae779.0000000000000003:a [read 9211904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:41:14.563 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.688360+0000", 2024-09-21T21:41:14.563 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 15.854325911, 2024-09-21T21:41:14.563 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.018899836, 2024-09-21T21:41:14.563 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:14.563 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:14.564 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:14.564 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:14.564 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:14.564 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:14.564 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 8319 2024-09-21T21:41:14.564 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.564 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:14.564 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.564 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:14.564 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688360+0000", 2024-09-21T21:41:14.564 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:14.564 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.564 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.564 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:14.565 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688360+0000", 2024-09-21T21:41:14.565 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:14.565 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.565 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.565 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:14.565 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688361+0000", 2024-09-21T21:41:14.565 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5499999999999998e-07 2024-09-21T21:41:14.565 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.565 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.565 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:14.565 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688363+0000", 2024-09-21T21:41:14.565 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.21e-06 2024-09-21T21:41:14.565 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.565 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.566 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:14.566 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688371+0000", 2024-09-21T21:41:14.566 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2539999999999992e-06 2024-09-21T21:41:14.566 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.566 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.566 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:14.566 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688377+0000", 2024-09-21T21:41:14.566 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.468e-06 2024-09-21T21:41:14.566 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.566 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.566 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:14.566 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.707121+0000", 2024-09-21T21:41:14.566 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.018743774000000001 2024-09-21T21:41:14.566 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.567 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.567 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:14.567 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.707195+0000", 2024-09-21T21:41:14.567 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4745000000000003e-05 2024-09-21T21:41:14.567 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.567 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.567 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:14.567 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.707260+0000", 2024-09-21T21:41:14.567 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4930000000000003e-05 2024-09-21T21:41:14.567 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:14.567 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:14.567 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:14.567 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.567 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.567 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:8320 3.3 3:c030d67f:::rbd_data.112145fae779.0000000000000003:a [read 9269248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:41:14.568 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.688383+0000", 2024-09-21T21:41:14.568 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 15.854302811, 2024-09-21T21:41:14.568 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.019040664999999998, 2024-09-21T21:41:14.568 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:14.568 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:14.568 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:14.568 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:14.568 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:14.568 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:14.568 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 8320 2024-09-21T21:41:14.568 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.568 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:14.568 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.568 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:14.569 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688383+0000", 2024-09-21T21:41:14.569 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:14.569 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.569 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.569 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:14.569 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688383+0000", 2024-09-21T21:41:14.569 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:14.569 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.569 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.569 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:14.569 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688384+0000", 2024-09-21T21:41:14.569 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.27e-07 2024-09-21T21:41:14.569 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.569 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.570 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:14.570 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688386+0000", 2024-09-21T21:41:14.570 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2299999999999998e-06 2024-09-21T21:41:14.570 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.570 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.570 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:14.570 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688395+0000", 2024-09-21T21:41:14.570 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.0410000000000003e-06 2024-09-21T21:41:14.570 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.570 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.570 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:14.570 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688400+0000", 2024-09-21T21:41:14.570 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.198e-06 2024-09-21T21:41:14.570 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.571 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.571 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:14.571 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.707289+0000", 2024-09-21T21:41:14.571 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.018888820000000001 2024-09-21T21:41:14.571 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.571 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.571 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:14.571 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.707358+0000", 2024-09-21T21:41:14.571 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9219000000000003e-05 2024-09-21T21:41:14.571 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.571 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.571 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:14.571 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.707424+0000", 2024-09-21T21:41:14.571 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5729999999999996e-05 2024-09-21T21:41:14.572 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:14.572 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:14.572 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:14.572 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.572 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.572 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:8321 3.3 3:c030d67f:::rbd_data.112145fae779.0000000000000003:a [read 9359360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:41:14.572 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.688407+0000", 2024-09-21T21:41:14.572 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 15.854279623, 2024-09-21T21:41:14.572 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.019217557999999999, 2024-09-21T21:41:14.572 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:14.572 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:14.572 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:14.572 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:14.572 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:14.573 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:14.573 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 8321 2024-09-21T21:41:14.573 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.573 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:14.573 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.573 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:14.573 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688407+0000", 2024-09-21T21:41:14.573 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:14.573 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.573 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.573 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:14.573 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688407+0000", 2024-09-21T21:41:14.573 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:14.574 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.574 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.574 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:14.574 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688407+0000", 2024-09-21T21:41:14.574 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2500000000000001e-07 2024-09-21T21:41:14.574 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.574 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.574 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:14.574 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688409+0000", 2024-09-21T21:41:14.574 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1299999999999999e-06 2024-09-21T21:41:14.574 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.574 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.574 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:14.574 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688416+0000", 2024-09-21T21:41:14.575 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9539999999999998e-06 2024-09-21T21:41:14.575 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.575 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.575 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:14.575 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688421+0000", 2024-09-21T21:41:14.575 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2610000000000001e-06 2024-09-21T21:41:14.575 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.575 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.575 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:14.575 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.707456+0000", 2024-09-21T21:41:14.575 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.019035024000000001 2024-09-21T21:41:14.575 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.575 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.575 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:14.576 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.707550+0000", 2024-09-21T21:41:14.576 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3696000000000006e-05 2024-09-21T21:41:14.576 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.576 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.576 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:14.576 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.707624+0000", 2024-09-21T21:41:14.576 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4067999999999996e-05 2024-09-21T21:41:14.576 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:14.576 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:14.576 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:14.576 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.576 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.576 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:8322 3.3 3:c030d67f:::rbd_data.112145fae779.0000000000000003:a [read 9416704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:41:14.576 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.688427+0000", 2024-09-21T21:41:14.577 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 15.854258805000001, 2024-09-21T21:41:14.577 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.019393840999999998, 2024-09-21T21:41:14.577 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:14.577 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:14.577 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:14.577 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:14.577 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:14.577 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:14.577 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 8322 2024-09-21T21:41:14.577 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.577 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:14.577 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.577 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:14.577 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688427+0000", 2024-09-21T21:41:14.578 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:14.578 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.578 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.578 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:14.578 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688427+0000", 2024-09-21T21:41:14.578 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:14.578 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.578 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.578 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:14.578 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688428+0000", 2024-09-21T21:41:14.578 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7e-07 2024-09-21T21:41:14.578 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.578 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.578 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:14.579 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688430+0000", 2024-09-21T21:41:14.579 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2189999999999998e-06 2024-09-21T21:41:14.579 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.579 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.579 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:14.579 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688436+0000", 2024-09-21T21:41:14.579 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4309999999999999e-06 2024-09-21T21:41:14.579 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.579 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.579 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:14.579 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688441+0000", 2024-09-21T21:41:14.579 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7029999999999997e-06 2024-09-21T21:41:14.579 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.579 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.580 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:14.580 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.707659+0000", 2024-09-21T21:41:14.580 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.019217998999999999 2024-09-21T21:41:14.580 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.580 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.580 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:14.580 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.707749+0000", 2024-09-21T21:41:14.580 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.0229999999999995e-05 2024-09-21T21:41:14.580 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.580 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.580 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:14.580 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.707821+0000", 2024-09-21T21:41:14.580 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1889000000000002e-05 2024-09-21T21:41:14.580 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:14.580 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:14.581 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:14.581 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.581 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.581 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:8323 3.3 3:c030d67f:::rbd_data.112145fae779.0000000000000003:a [read 9449472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:41:14.581 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.688451+0000", 2024-09-21T21:41:14.581 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 15.854235185, 2024-09-21T21:41:14.581 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.019572622000000001, 2024-09-21T21:41:14.581 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:14.581 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:14.581 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:14.581 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:14.581 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:14.581 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:14.581 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 8323 2024-09-21T21:41:14.582 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.582 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:14.582 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.582 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:14.582 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688451+0000", 2024-09-21T21:41:14.582 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:14.582 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.582 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.582 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:14.582 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688451+0000", 2024-09-21T21:41:14.582 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:14.582 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.582 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.582 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:14.583 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688452+0000", 2024-09-21T21:41:14.583 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.39e-07 2024-09-21T21:41:14.583 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.583 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.583 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:14.583 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688454+0000", 2024-09-21T21:41:14.583 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.272e-06 2024-09-21T21:41:14.583 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.583 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.583 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:14.583 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688460+0000", 2024-09-21T21:41:14.583 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5270000000000004e-06 2024-09-21T21:41:14.583 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.583 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.584 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:14.584 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688466+0000", 2024-09-21T21:41:14.584 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1909999999999999e-06 2024-09-21T21:41:14.584 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.584 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.584 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:14.584 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.707854+0000", 2024-09-21T21:41:14.584 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.019388216999999999 2024-09-21T21:41:14.584 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.584 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.584 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:14.584 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.707952+0000", 2024-09-21T21:41:14.584 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.8136999999999999e-05 2024-09-21T21:41:14.584 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.585 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.585 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:14.585 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.708024+0000", 2024-09-21T21:41:14.585 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1839000000000001e-05 2024-09-21T21:41:14.585 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:14.585 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:14.585 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:14.585 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.585 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.585 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:8324 3.3 3:c030d67f:::rbd_data.112145fae779.0000000000000003:a [read 9506816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:41:14.585 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.688472+0000", 2024-09-21T21:41:14.585 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 15.854214053, 2024-09-21T21:41:14.585 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.019741218000000001, 2024-09-21T21:41:14.585 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:14.585 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:14.586 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:14.586 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:14.586 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:14.586 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:14.586 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 8324 2024-09-21T21:41:14.586 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.586 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:14.586 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.586 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:14.586 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688472+0000", 2024-09-21T21:41:14.586 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:14.586 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.586 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.586 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:14.587 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688472+0000", 2024-09-21T21:41:14.587 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:14.587 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.587 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.587 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:14.587 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688473+0000", 2024-09-21T21:41:14.587 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4299999999999999e-07 2024-09-21T21:41:14.587 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.587 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.587 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:14.587 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688475+0000", 2024-09-21T21:41:14.587 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1409999999999999e-06 2024-09-21T21:41:14.587 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.587 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.588 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:14.588 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688481+0000", 2024-09-21T21:41:14.588 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5529999999999998e-06 2024-09-21T21:41:14.588 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.588 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.588 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:14.588 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688486+0000", 2024-09-21T21:41:14.588 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9459999999999997e-06 2024-09-21T21:41:14.588 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.588 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.588 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:14.588 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.708064+0000", 2024-09-21T21:41:14.588 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.019577721999999999 2024-09-21T21:41:14.588 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.589 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.589 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:14.589 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.708144+0000", 2024-09-21T21:41:14.589 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0554000000000002e-05 2024-09-21T21:41:14.589 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.589 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.589 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:14.589 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.708213+0000", 2024-09-21T21:41:14.589 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8959000000000002e-05 2024-09-21T21:41:14.589 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:14.589 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:14.589 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:14.589 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.589 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.589 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:8325 3.3 3:c030d67f:::rbd_data.112145fae779.0000000000000003:a [read 9564160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:41:14.590 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.688493+0000", 2024-09-21T21:41:14.590 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 15.854193293, 2024-09-21T21:41:14.590 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.019899284999999999, 2024-09-21T21:41:14.590 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:14.590 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:14.590 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:14.590 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:14.590 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:14.590 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:14.590 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 8325 2024-09-21T21:41:14.590 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.590 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:14.590 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.590 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:14.591 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688493+0000", 2024-09-21T21:41:14.591 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:14.591 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.591 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.591 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:14.591 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688493+0000", 2024-09-21T21:41:14.591 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:14.591 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.591 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.591 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:14.591 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688493+0000", 2024-09-21T21:41:14.591 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4999999999999998e-07 2024-09-21T21:41:14.591 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.591 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.592 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:14.593 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688496+0000", 2024-09-21T21:41:14.593 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0949999999999998e-06 2024-09-21T21:41:14.593 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.593 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.593 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:14.593 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688502+0000", 2024-09-21T21:41:14.594 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4409999999999998e-06 2024-09-21T21:41:14.594 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.594 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.594 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:14.594 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.688507+0000", 2024-09-21T21:41:14.594 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6229999999999996e-06 2024-09-21T21:41:14.594 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.594 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.594 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:14.594 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.708246+0000", 2024-09-21T21:41:14.594 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.019739626 2024-09-21T21:41:14.594 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.594 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.595 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:14.595 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.708321+0000", 2024-09-21T21:41:14.595 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4803999999999997e-05 2024-09-21T21:41:14.595 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.595 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.595 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:14.595 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.708392+0000", 2024-09-21T21:41:14.595 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1246000000000002e-05 2024-09-21T21:41:14.595 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:14.595 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:14.595 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:14.595 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.595 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.595 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:8444 3.3 3:c030d67f:::rbd_data.112145fae779.0000000000000003:a [read 9068544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:41:14.596 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.689763+0000", 2024-09-21T21:41:14.596 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 15.852923713999999, 2024-09-21T21:41:14.596 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.018821181999999999, 2024-09-21T21:41:14.596 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:14.596 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:14.596 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:14.596 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:14.596 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:14.596 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:14.596 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 8444 2024-09-21T21:41:14.596 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.596 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:14.596 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.596 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:14.597 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.689763+0000", 2024-09-21T21:41:14.597 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:14.597 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.597 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.597 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:14.597 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.689763+0000", 2024-09-21T21:41:14.597 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:14.597 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.597 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.597 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:14.597 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.689764+0000", 2024-09-21T21:41:14.597 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.9300000000000006e-07 2024-09-21T21:41:14.597 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.598 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.598 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:14.598 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.689769+0000", 2024-09-21T21:41:14.598 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.994e-06 2024-09-21T21:41:14.598 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.598 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.598 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:14.598 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.689781+0000", 2024-09-21T21:41:14.598 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.198e-05 2024-09-21T21:41:14.598 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.598 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.598 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:14.598 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.689789+0000", 2024-09-21T21:41:14.598 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2800000000000003e-06 2024-09-21T21:41:14.599 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.599 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.599 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:14.599 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.708426+0000", 2024-09-21T21:41:14.599 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.018637628 2024-09-21T21:41:14.599 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.599 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.599 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:14.599 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.708517+0000", 2024-09-21T21:41:14.599 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.0227000000000002e-05 2024-09-21T21:41:14.599 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.599 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.599 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:14.600 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.708584+0000", 2024-09-21T21:41:14.600 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7080000000000001e-05 2024-09-21T21:41:14.600 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:14.600 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:14.600 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:14.600 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.600 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.600 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:8445 3.3 3:c030d67f:::rbd_data.112145fae779.0000000000000003:a [read 9125888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:41:14.600 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.689842+0000", 2024-09-21T21:41:14.600 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 15.852844768000001, 2024-09-21T21:41:14.600 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.019029801999999998, 2024-09-21T21:41:14.600 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:14.600 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:14.600 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:14.600 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:14.601 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:14.601 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:14.601 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 8445 2024-09-21T21:41:14.601 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.601 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:14.601 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.601 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:14.601 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.689842+0000", 2024-09-21T21:41:14.601 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:14.601 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.601 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.601 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:14.601 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.689842+0000", 2024-09-21T21:41:14.601 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:14.602 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.602 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.602 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:14.602 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.689842+0000", 2024-09-21T21:41:14.602 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4399999999999999e-07 2024-09-21T21:41:14.602 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.602 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.602 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:14.602 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.689847+0000", 2024-09-21T21:41:14.602 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.746e-06 2024-09-21T21:41:14.602 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.602 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.602 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:14.603 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.689856+0000", 2024-09-21T21:41:14.603 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.2909999999999996e-06 2024-09-21T21:41:14.603 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.603 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.603 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:14.603 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.689868+0000", 2024-09-21T21:41:14.603 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1498000000000001e-05 2024-09-21T21:41:14.603 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.603 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.603 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:14.603 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.708618+0000", 2024-09-21T21:41:14.603 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.018750032 2024-09-21T21:41:14.603 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.603 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.604 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:14.604 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.708726+0000", 2024-09-21T21:41:14.604 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010836399999999999 2024-09-21T21:41:14.604 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.604 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.604 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:14.604 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.708871+0000", 2024-09-21T21:41:14.604 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014512699999999999 2024-09-21T21:41:14.604 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:14.604 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:14.604 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:14.604 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.604 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.605 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:8446 3.3 3:c030d67f:::rbd_data.112145fae779.0000000000000003:a [read 9216000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:41:14.605 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.689911+0000", 2024-09-21T21:41:14.605 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 15.852775437, 2024-09-21T21:41:14.605 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.019300134, 2024-09-21T21:41:14.605 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:14.605 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:14.605 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:14.605 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:14.605 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:14.605 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:14.605 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 8446 2024-09-21T21:41:14.605 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.605 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:14.605 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.606 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:14.606 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.689911+0000", 2024-09-21T21:41:14.606 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:14.606 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.606 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.606 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:14.606 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.689911+0000", 2024-09-21T21:41:14.606 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:14.606 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.606 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.606 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:14.606 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.689912+0000", 2024-09-21T21:41:14.606 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8899999999999999e-07 2024-09-21T21:41:14.606 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.607 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.607 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:14.607 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.689915+0000", 2024-09-21T21:41:14.607 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1839999999999999e-06 2024-09-21T21:41:14.607 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.607 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.607 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:14.607 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.689924+0000", 2024-09-21T21:41:14.607 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.0170000000000002e-06 2024-09-21T21:41:14.607 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.607 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.607 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:14.607 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.689930+0000", 2024-09-21T21:41:14.607 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4930000000000003e-06 2024-09-21T21:41:14.608 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.608 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.608 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:14.608 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.708850+0000", 2024-09-21T21:41:14.608 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.018919703 2024-09-21T21:41:14.608 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.608 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.608 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:14.608 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.709104+0000", 2024-09-21T21:41:14.608 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00025365899999999999 2024-09-21T21:41:14.608 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.608 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.608 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:14.608 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.709211+0000", 2024-09-21T21:41:14.608 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000107389 2024-09-21T21:41:14.609 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:14.609 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:14.609 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:14.609 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.609 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.609 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:8447 3.3 3:c030d67f:::rbd_data.112145fae779.0000000000000003:a [read 9273344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:41:14.609 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.689970+0000", 2024-09-21T21:41:14.609 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 15.852716352, 2024-09-21T21:41:14.609 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.019358220999999998, 2024-09-21T21:41:14.609 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:14.609 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:14.609 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:14.609 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:14.609 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:14.610 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:14.610 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 8447 2024-09-21T21:41:14.610 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.610 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:14.610 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.610 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:14.610 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.689970+0000", 2024-09-21T21:41:14.610 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:14.610 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.610 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.610 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:14.610 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.689970+0000", 2024-09-21T21:41:14.610 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:14.611 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.611 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.611 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:14.611 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.689971+0000", 2024-09-21T21:41:14.611 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3200000000000005e-07 2024-09-21T21:41:14.611 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.611 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.611 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:14.611 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.689974+0000", 2024-09-21T21:41:14.611 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1820000000000002e-06 2024-09-21T21:41:14.611 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.611 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.611 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:14.611 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.689982+0000", 2024-09-21T21:41:14.612 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7320000000000004e-06 2024-09-21T21:41:14.612 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.612 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.612 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:14.612 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.689989+0000", 2024-09-21T21:41:14.612 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2129999999999996e-06 2024-09-21T21:41:14.612 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.612 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.612 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:14.612 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.709241+0000", 2024-09-21T21:41:14.612 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.019252612999999998 2024-09-21T21:41:14.612 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.612 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.613 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:14.613 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.709285+0000", 2024-09-21T21:41:14.613 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3925999999999999e-05 2024-09-21T21:41:14.613 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.613 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.613 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:14.613 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.709328+0000", 2024-09-21T21:41:14.613 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2923000000000002e-05 2024-09-21T21:41:14.613 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:14.613 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:14.613 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:14.613 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.613 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.613 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:8448 3.3 3:c030d67f:::rbd_data.112145fae779.0000000000000003:a [read 9306112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:41:14.613 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.689998+0000", 2024-09-21T21:41:14.614 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 15.852688518000001, 2024-09-21T21:41:14.614 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.019538304999999999, 2024-09-21T21:41:14.614 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:14.614 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:14.614 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:14.614 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:14.614 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:14.614 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:14.614 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 8448 2024-09-21T21:41:14.614 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.614 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:14.614 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.614 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:14.614 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.689998+0000", 2024-09-21T21:41:14.614 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:14.615 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.615 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.615 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:14.615 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.689998+0000", 2024-09-21T21:41:14.615 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:14.615 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.615 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.615 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:14.615 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.689998+0000", 2024-09-21T21:41:14.615 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4499999999999997e-07 2024-09-21T21:41:14.615 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.615 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.615 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:14.615 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.690001+0000", 2024-09-21T21:41:14.616 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5849999999999998e-06 2024-09-21T21:41:14.616 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.616 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.616 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:14.616 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.690007+0000", 2024-09-21T21:41:14.616 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4620000000000001e-06 2024-09-21T21:41:14.616 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.616 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.616 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:14.616 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.690012+0000", 2024-09-21T21:41:14.616 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9559999999999996e-06 2024-09-21T21:41:14.616 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.616 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.616 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:14.617 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.709359+0000", 2024-09-21T21:41:14.617 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.019346550000000001 2024-09-21T21:41:14.617 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.617 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.617 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:14.617 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.709435+0000", 2024-09-21T21:41:14.617 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6556000000000004e-05 2024-09-21T21:41:14.617 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.617 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.617 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:14.617 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.709536+0000", 2024-09-21T21:41:14.617 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000100751 2024-09-21T21:41:14.617 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:14.617 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:14.618 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:14.618 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.618 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.618 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:8449 3.3 3:c030d67f:::rbd_data.112145fae779.0000000000000003:a [read 9363456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:41:14.618 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.690019+0000", 2024-09-21T21:41:14.618 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 15.852666783, 2024-09-21T21:41:14.618 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.019725633999999999, 2024-09-21T21:41:14.618 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:14.618 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:14.618 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:14.618 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:14.618 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:14.618 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:14.618 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 8449 2024-09-21T21:41:14.619 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.619 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:14.619 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.619 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:14.619 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.690019+0000", 2024-09-21T21:41:14.619 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:14.619 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.619 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.619 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:14.619 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.690019+0000", 2024-09-21T21:41:14.619 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:14.619 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.619 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.619 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:14.620 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.690020+0000", 2024-09-21T21:41:14.620 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3000000000000001e-07 2024-09-21T21:41:14.620 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.620 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.620 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:14.620 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.690022+0000", 2024-09-21T21:41:14.620 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2000000000000001e-06 2024-09-21T21:41:14.620 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.620 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.620 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:14.620 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.690029+0000", 2024-09-21T21:41:14.620 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5860000000000001e-06 2024-09-21T21:41:14.620 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.620 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.620 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:14.621 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.690034+0000", 2024-09-21T21:41:14.621 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9350000000000002e-06 2024-09-21T21:41:14.621 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.621 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.621 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:14.621 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.709566+0000", 2024-09-21T21:41:14.621 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.019532445999999998 2024-09-21T21:41:14.621 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.621 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.621 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:14.621 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.709647+0000", 2024-09-21T21:41:14.621 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0931000000000002e-05 2024-09-21T21:41:14.621 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.621 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.622 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:14.622 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.709745+0000", 2024-09-21T21:41:14.622 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.8105999999999999e-05 2024-09-21T21:41:14.622 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:14.622 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:14.622 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:14.622 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.622 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.622 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:8450 3.3 3:c030d67f:::rbd_data.112145fae779.0000000000000003:a [read 9420800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:41:14.622 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.690040+0000", 2024-09-21T21:41:14.622 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 15.852646309000001, 2024-09-21T21:41:14.622 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.019843738999999999, 2024-09-21T21:41:14.622 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:14.622 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:14.623 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:14.623 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:14.623 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:14.623 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:14.623 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 8450 2024-09-21T21:41:14.623 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.623 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:14.623 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.623 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:14.623 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.690040+0000", 2024-09-21T21:41:14.623 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:14.623 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.623 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.623 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:14.623 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.690040+0000", 2024-09-21T21:41:14.624 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:14.624 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.624 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.624 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:14.624 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.690040+0000", 2024-09-21T21:41:14.624 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1699999999999999e-07 2024-09-21T21:41:14.624 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.624 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.624 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:14.624 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.690042+0000", 2024-09-21T21:41:14.624 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9240000000000001e-06 2024-09-21T21:41:14.624 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.624 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.624 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:14.624 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.690049+0000", 2024-09-21T21:41:14.625 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3069999999999999e-06 2024-09-21T21:41:14.625 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.625 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.625 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:14.625 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.690053+0000", 2024-09-21T21:41:14.625 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8239999999999999e-06 2024-09-21T21:41:14.625 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.625 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.625 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:14.625 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.709790+0000", 2024-09-21T21:41:14.625 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.019736942 2024-09-21T21:41:14.625 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.625 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.625 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:14.626 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.709834+0000", 2024-09-21T21:41:14.626 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3744e-05 2024-09-21T21:41:14.626 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.626 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.626 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:14.626 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.709884+0000", 2024-09-21T21:41:14.626 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9580999999999998e-05 2024-09-21T21:41:14.626 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:14.626 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:14.626 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:14.626 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.626 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.626 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:8451 3.3 3:c030d67f:::rbd_data.112145fae779.0000000000000003:a [read 9453568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:41:14.626 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.690077+0000", 2024-09-21T21:41:14.627 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 15.852608823000001, 2024-09-21T21:41:14.627 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.020018285, 2024-09-21T21:41:14.627 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:14.627 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:14.627 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:14.627 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:14.627 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:14.627 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:14.627 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 8451 2024-09-21T21:41:14.627 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.627 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:14.627 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.627 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:14.627 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.690077+0000", 2024-09-21T21:41:14.627 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:14.627 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.628 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.628 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:14.628 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.690077+0000", 2024-09-21T21:41:14.628 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:14.628 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.628 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.628 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:14.628 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.690078+0000", 2024-09-21T21:41:14.628 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1999999999999998e-07 2024-09-21T21:41:14.628 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.628 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.628 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:14.628 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.690082+0000", 2024-09-21T21:41:14.628 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0879999999999997e-06 2024-09-21T21:41:14.629 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.629 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.629 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:14.629 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.690091+0000", 2024-09-21T21:41:14.629 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6549999999999993e-06 2024-09-21T21:41:14.629 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.629 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.629 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:14.629 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.690098+0000", 2024-09-21T21:41:14.629 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7710000000000001e-06 2024-09-21T21:41:14.629 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.629 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.629 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:14.629 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.709913+0000", 2024-09-21T21:41:14.630 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.019815510000000001 2024-09-21T21:41:14.630 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.630 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.630 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:14.630 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.709991+0000", 2024-09-21T21:41:14.630 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7296e-05 2024-09-21T21:41:14.630 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.630 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.630 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:14.630 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.710096+0000", 2024-09-21T21:41:14.630 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000105145 2024-09-21T21:41:14.630 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:14.630 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:14.631 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:14.631 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.631 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.631 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:8452 3.3 3:c030d67f:::rbd_data.112145fae779.0000000000000003:a [read 9510912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-21T21:41:14.631 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.690109+0000", 2024-09-21T21:41:14.631 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 15.852577608000001, 2024-09-21T21:41:14.631 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.020107368, 2024-09-21T21:41:14.631 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:14.631 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:14.631 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:14.631 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:14.631 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:14.631 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:14.631 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 8452 2024-09-21T21:41:14.632 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.632 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:14.632 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.632 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:14.632 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.690109+0000", 2024-09-21T21:41:14.632 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:14.632 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.632 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.632 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:14.632 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.690109+0000", 2024-09-21T21:41:14.632 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:14.632 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.632 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.632 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:14.633 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.690109+0000", 2024-09-21T21:41:14.633 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2899999999999999e-07 2024-09-21T21:41:14.633 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.633 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.633 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:14.633 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.690111+0000", 2024-09-21T21:41:14.633 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2409999999999998e-06 2024-09-21T21:41:14.633 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.633 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.633 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:14.633 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.690118+0000", 2024-09-21T21:41:14.633 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8850000000000001e-06 2024-09-21T21:41:14.633 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.633 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.633 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:14.634 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.690124+0000", 2024-09-21T21:41:14.634 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5589999999999996e-06 2024-09-21T21:41:14.634 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.634 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.634 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:14.634 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.710126+0000", 2024-09-21T21:41:14.634 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.020002217999999999 2024-09-21T21:41:14.634 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.634 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.634 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:14.634 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.710177+0000", 2024-09-21T21:41:14.634 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0587999999999997e-05 2024-09-21T21:41:14.634 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:14.634 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:14.635 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:14.635 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.710216+0000", 2024-09-21T21:41:14.635 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9447999999999999e-05 2024-09-21T21:41:14.635 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:14.635 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:14.635 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:14.635 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:14.635 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:14.635 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:14.635 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T21:41:14.637 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:14.635+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:14.637 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T21:41:14.737 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:14.736+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:14.741 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:14.741 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:41:14.741 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-21T21:41:14.741 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:14.750 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T21:41:14.838 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:14.837+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:14.896 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:14.896 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:41:14.897 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:41:14.897 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:41:14.897 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:14.905 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T21:41:14.933 INFO:teuthology.orchestra.run.smithi164.stdout:checking clone #12, image image_client.0-clone12 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent13 2024-09-21T21:41:14.938 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:14.937+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:15.039 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:15.038+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:15.074 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:15.074 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-21T21:41:15.074 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-21T21:41:15.074 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:41:15.074 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.074 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1484 3.16 3:6a41a830:::rbd_data.112145fae779.000000000000000a:a [read 303104~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:15.074 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634086+0000", 2024-09-21T21:41:15.074 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.437198349999999, 2024-09-21T21:41:15.074 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.058257055000000002, 2024-09-21T21:41:15.074 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:15.075 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:15.075 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:15.075 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:15.075 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:15.075 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:15.075 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1484 2024-09-21T21:41:15.075 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.075 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:15.075 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.075 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:15.075 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634086+0000", 2024-09-21T21:41:15.075 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:15.075 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.075 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.076 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:15.076 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634086+0000", 2024-09-21T21:41:15.076 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:15.076 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.076 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.076 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:15.076 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634086+0000", 2024-09-21T21:41:15.076 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6100000000000001e-07 2024-09-21T21:41:15.076 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.076 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.076 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:15.076 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634088+0000", 2024-09-21T21:41:15.076 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1100000000000001e-06 2024-09-21T21:41:15.076 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.077 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.077 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:15.077 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634095+0000", 2024-09-21T21:41:15.077 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4080000000000003e-06 2024-09-21T21:41:15.077 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.077 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.077 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:15.077 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634100+0000", 2024-09-21T21:41:15.077 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0730000000000004e-06 2024-09-21T21:41:15.077 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.077 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.077 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:15.077 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.692197+0000", 2024-09-21T21:41:15.077 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.058097331000000002 2024-09-21T21:41:15.078 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.078 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.078 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:15.078 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.692271+0000", 2024-09-21T21:41:15.078 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3423999999999994e-05 2024-09-21T21:41:15.078 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.078 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.078 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:15.078 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.692343+0000", 2024-09-21T21:41:15.078 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2248000000000004e-05 2024-09-21T21:41:15.078 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:15.078 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:15.078 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:15.079 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.079 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.079 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1485 3.16 3:695c582e:::rbd_data.1121e540c096.0000000000000008:b [read 335872~36864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:15.079 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634106+0000", 2024-09-21T21:41:15.079 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.437178553000001, 2024-09-21T21:41:15.079 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059223685999999998, 2024-09-21T21:41:15.079 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:15.079 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:15.079 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:15.079 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:15.079 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:15.079 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:15.079 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1485 2024-09-21T21:41:15.079 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.079 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:15.080 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.080 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:15.080 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634106+0000", 2024-09-21T21:41:15.080 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:15.080 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.080 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.080 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:15.080 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634106+0000", 2024-09-21T21:41:15.080 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:15.080 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.080 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.080 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:15.080 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634106+0000", 2024-09-21T21:41:15.080 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3300000000000002e-07 2024-09-21T21:41:15.080 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.081 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.081 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:15.081 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634108+0000", 2024-09-21T21:41:15.081 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.996e-06 2024-09-21T21:41:15.081 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.081 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.081 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:15.081 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634115+0000", 2024-09-21T21:41:15.081 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5139999999999998e-06 2024-09-21T21:41:15.081 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.081 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.081 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:15.081 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634120+0000", 2024-09-21T21:41:15.081 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1050000000000003e-06 2024-09-21T21:41:15.082 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.082 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.082 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:15.082 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.692374+0000", 2024-09-21T21:41:15.082 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.058254729999999998 2024-09-21T21:41:15.082 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.082 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.082 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:15.082 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.692440+0000", 2024-09-21T21:41:15.082 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6076999999999998e-05 2024-09-21T21:41:15.082 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.082 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.082 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:15.082 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693329+0000", 2024-09-21T21:41:15.083 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00088873100000000003 2024-09-21T21:41:15.083 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:15.083 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:15.083 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:15.083 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.083 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.083 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1486 3.16 3:6a41a830:::rbd_data.112145fae779.000000000000000a:a [read 360448~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:15.083 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634126+0000", 2024-09-21T21:41:15.083 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.437158261, 2024-09-21T21:41:15.083 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059399528, 2024-09-21T21:41:15.083 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:15.083 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:15.083 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:15.083 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:15.084 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:15.084 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:15.084 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1486 2024-09-21T21:41:15.084 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.084 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:15.084 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.084 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:15.084 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634126+0000", 2024-09-21T21:41:15.084 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:15.084 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.084 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.084 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:15.084 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634126+0000", 2024-09-21T21:41:15.084 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:15.085 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.085 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.085 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:15.085 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634126+0000", 2024-09-21T21:41:15.085 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4099999999999999e-07 2024-09-21T21:41:15.085 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.085 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.085 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:15.085 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634128+0000", 2024-09-21T21:41:15.085 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1069999999999999e-06 2024-09-21T21:41:15.085 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.085 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.085 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:15.085 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634135+0000", 2024-09-21T21:41:15.086 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.212e-06 2024-09-21T21:41:15.086 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.086 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.086 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:15.086 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634140+0000", 2024-09-21T21:41:15.086 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1429999999999997e-06 2024-09-21T21:41:15.086 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.086 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.086 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:15.086 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693363+0000", 2024-09-21T21:41:15.086 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059223614000000001 2024-09-21T21:41:15.086 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.086 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.086 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:15.087 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693431+0000", 2024-09-21T21:41:15.087 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7294999999999999e-05 2024-09-21T21:41:15.087 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.087 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.087 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:15.087 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693525+0000", 2024-09-21T21:41:15.087 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.4716000000000006e-05 2024-09-21T21:41:15.087 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:15.087 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:15.087 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:15.087 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.087 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.087 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1499 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 49152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:15.087 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634186+0000", 2024-09-21T21:41:15.087 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.437098384, 2024-09-21T21:41:15.088 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059339256999999999, 2024-09-21T21:41:15.088 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:15.088 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:15.088 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:15.088 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:15.088 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:15.088 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:15.088 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1499 2024-09-21T21:41:15.088 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.088 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:15.088 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.088 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:15.088 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634186+0000", 2024-09-21T21:41:15.088 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:15.089 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.089 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.089 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:15.089 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634186+0000", 2024-09-21T21:41:15.089 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:15.089 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.089 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.089 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:15.089 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634186+0000", 2024-09-21T21:41:15.089 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0999999999999999e-07 2024-09-21T21:41:15.089 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.089 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.089 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:15.089 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634188+0000", 2024-09-21T21:41:15.089 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.86e-06 2024-09-21T21:41:15.090 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.090 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.090 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:15.090 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634194+0000", 2024-09-21T21:41:15.090 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1109999999999996e-06 2024-09-21T21:41:15.090 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.090 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.090 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:15.090 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634199+0000", 2024-09-21T21:41:15.090 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7419999999999997e-06 2024-09-21T21:41:15.090 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.090 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.090 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:15.090 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693357+0000", 2024-09-21T21:41:15.091 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059157847 2024-09-21T21:41:15.091 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.091 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.091 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:15.091 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693430+0000", 2024-09-21T21:41:15.091 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3203999999999999e-05 2024-09-21T21:41:15.091 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.091 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.091 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:15.091 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693525+0000", 2024-09-21T21:41:15.091 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5082999999999997e-05 2024-09-21T21:41:15.091 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:15.091 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:15.091 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:15.091 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.092 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.092 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1501 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 106496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:15.092 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634205+0000", 2024-09-21T21:41:15.092 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.437079294, 2024-09-21T21:41:15.092 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059482394000000001, 2024-09-21T21:41:15.092 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:15.092 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:15.092 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:15.092 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:15.092 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:15.092 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:15.092 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1501 2024-09-21T21:41:15.092 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.092 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:15.092 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.093 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:15.093 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634205+0000", 2024-09-21T21:41:15.093 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:15.093 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.093 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.093 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:15.093 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634205+0000", 2024-09-21T21:41:15.093 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:15.093 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.093 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.093 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:15.093 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634205+0000", 2024-09-21T21:41:15.093 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.03e-07 2024-09-21T21:41:15.094 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.094 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.094 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:15.094 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634207+0000", 2024-09-21T21:41:15.094 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0389999999999999e-06 2024-09-21T21:41:15.094 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.094 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.094 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:15.094 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634213+0000", 2024-09-21T21:41:15.094 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2600000000000002e-06 2024-09-21T21:41:15.094 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.094 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.094 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:15.094 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634218+0000", 2024-09-21T21:41:15.095 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8570000000000003e-06 2024-09-21T21:41:15.095 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.095 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.095 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:15.095 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693552+0000", 2024-09-21T21:41:15.095 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059333636000000002 2024-09-21T21:41:15.095 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.095 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.095 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:15.095 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693613+0000", 2024-09-21T21:41:15.095 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0560000000000003e-05 2024-09-21T21:41:15.095 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.095 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.095 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:15.096 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693687+0000", 2024-09-21T21:41:15.096 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4639000000000001e-05 2024-09-21T21:41:15.096 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:15.096 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:15.096 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:15.096 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.096 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.096 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1515 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 434176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:15.096 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634318+0000", 2024-09-21T21:41:15.096 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.436965606999999, 2024-09-21T21:41:15.096 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059565812000000003, 2024-09-21T21:41:15.096 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:15.096 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:15.097 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:15.097 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:15.097 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:15.097 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:15.097 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1515 2024-09-21T21:41:15.097 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.097 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:15.097 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.097 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:15.097 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634318+0000", 2024-09-21T21:41:15.097 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:15.097 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.097 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.097 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:15.097 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634318+0000", 2024-09-21T21:41:15.098 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:15.098 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.098 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.098 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:15.098 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634319+0000", 2024-09-21T21:41:15.098 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9400000000000001e-07 2024-09-21T21:41:15.098 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.098 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.098 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:15.098 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634321+0000", 2024-09-21T21:41:15.098 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.154e-06 2024-09-21T21:41:15.098 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.098 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.098 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:15.099 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634327+0000", 2024-09-21T21:41:15.099 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9830000000000001e-06 2024-09-21T21:41:15.099 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.099 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.099 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:15.099 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634332+0000", 2024-09-21T21:41:15.099 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9080000000000003e-06 2024-09-21T21:41:15.099 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.099 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.099 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:15.099 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693717+0000", 2024-09-21T21:41:15.099 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059385447000000001 2024-09-21T21:41:15.099 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.099 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.100 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:15.100 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693795+0000", 2024-09-21T21:41:15.100 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7267000000000004e-05 2024-09-21T21:41:15.100 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.100 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.100 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:15.100 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693884+0000", 2024-09-21T21:41:15.100 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9659000000000003e-05 2024-09-21T21:41:15.100 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:15.100 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:15.100 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:15.100 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.100 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.100 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1517 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 491520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:15.100 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634338+0000", 2024-09-21T21:41:15.101 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.436945833999999, 2024-09-21T21:41:15.101 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060100745999999997, 2024-09-21T21:41:15.101 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:15.101 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:15.101 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:15.101 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:15.101 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:15.101 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:15.101 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1517 2024-09-21T21:41:15.101 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.101 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:15.101 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.101 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:15.102 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634338+0000", 2024-09-21T21:41:15.102 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:15.102 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.102 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.102 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:15.102 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634338+0000", 2024-09-21T21:41:15.102 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:15.102 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.102 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.102 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:15.102 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634339+0000", 2024-09-21T21:41:15.102 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2899999999999999e-07 2024-09-21T21:41:15.102 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.103 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.103 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:15.103 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634341+0000", 2024-09-21T21:41:15.103 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.8959999999999999e-06 2024-09-21T21:41:15.103 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.103 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.103 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:15.103 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634347+0000", 2024-09-21T21:41:15.103 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.302e-06 2024-09-21T21:41:15.103 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.103 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.103 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:15.103 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634351+0000", 2024-09-21T21:41:15.103 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6070000000000001e-06 2024-09-21T21:41:15.104 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.104 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.104 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:15.104 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694279+0000", 2024-09-21T21:41:15.104 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059927046999999997 2024-09-21T21:41:15.104 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.104 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.104 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:15.104 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694355+0000", 2024-09-21T21:41:15.104 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6483999999999995e-05 2024-09-21T21:41:15.104 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.104 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.104 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:15.104 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694439+0000", 2024-09-21T21:41:15.104 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3980999999999994e-05 2024-09-21T21:41:15.105 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:15.105 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:15.105 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:15.105 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.105 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.105 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1518 3.16 3:6a41a830:::rbd_data.112145fae779.000000000000000a:a [read 294912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:15.105 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634357+0000", 2024-09-21T21:41:15.105 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.436926928, 2024-09-21T21:41:15.105 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059712318, 2024-09-21T21:41:15.105 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:15.105 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:15.105 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:15.105 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:15.105 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:15.106 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:15.106 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1518 2024-09-21T21:41:15.106 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.106 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:15.106 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.106 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:15.106 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634357+0000", 2024-09-21T21:41:15.106 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:15.106 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.106 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.106 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:15.106 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634357+0000", 2024-09-21T21:41:15.106 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:15.106 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.107 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.107 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:15.107 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634358+0000", 2024-09-21T21:41:15.107 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2100000000000002e-07 2024-09-21T21:41:15.107 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.107 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.107 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:15.107 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634360+0000", 2024-09-21T21:41:15.107 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0940000000000002e-06 2024-09-21T21:41:15.107 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.107 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.107 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:15.107 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634366+0000", 2024-09-21T21:41:15.107 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3389999999999998e-06 2024-09-21T21:41:15.108 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.108 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.108 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:15.108 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634371+0000", 2024-09-21T21:41:15.108 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6670000000000004e-06 2024-09-21T21:41:15.108 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.108 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.108 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:15.108 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693910+0000", 2024-09-21T21:41:15.108 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059538973000000002 2024-09-21T21:41:15.108 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.108 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.108 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:15.108 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693982+0000", 2024-09-21T21:41:15.108 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2806999999999996e-05 2024-09-21T21:41:15.109 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.109 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.109 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:15.109 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694069+0000", 2024-09-21T21:41:15.109 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7016999999999997e-05 2024-09-21T21:41:15.109 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:15.109 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:15.109 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:15.109 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.109 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.109 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1519 3.16 3:6a41a830:::rbd_data.112145fae779.000000000000000a:a [read 352256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:15.109 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634377+0000", 2024-09-21T21:41:15.109 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.436907226000001, 2024-09-21T21:41:15.109 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059872213000000001, 2024-09-21T21:41:15.109 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:15.110 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:15.110 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:15.110 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:15.110 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:15.110 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:15.110 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1519 2024-09-21T21:41:15.110 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.110 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:15.110 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.110 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:15.110 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634377+0000", 2024-09-21T21:41:15.110 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:15.111 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.111 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.111 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:15.111 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634377+0000", 2024-09-21T21:41:15.111 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:15.111 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.111 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.111 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:15.111 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634377+0000", 2024-09-21T21:41:15.111 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2600000000000002e-07 2024-09-21T21:41:15.111 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.111 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.111 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:15.111 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634379+0000", 2024-09-21T21:41:15.112 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0729999999999999e-06 2024-09-21T21:41:15.112 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.112 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.112 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:15.112 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634386+0000", 2024-09-21T21:41:15.112 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4690000000000002e-06 2024-09-21T21:41:15.112 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.112 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.112 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:15.112 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634390+0000", 2024-09-21T21:41:15.112 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4580000000000003e-06 2024-09-21T21:41:15.112 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.112 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.112 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:15.113 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694094+0000", 2024-09-21T21:41:15.113 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059703830999999999 2024-09-21T21:41:15.113 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.113 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.113 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:15.113 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694168+0000", 2024-09-21T21:41:15.113 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4222999999999997e-05 2024-09-21T21:41:15.113 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.113 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.113 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:15.113 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694249+0000", 2024-09-21T21:41:15.113 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0632999999999999e-05 2024-09-21T21:41:15.113 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:15.113 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:15.113 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:15.114 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.114 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.114 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1529 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 53248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:15.114 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634397+0000", 2024-09-21T21:41:15.114 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.436887075, 2024-09-21T21:41:15.114 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060189919000000001, 2024-09-21T21:41:15.114 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:15.114 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:15.114 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:15.114 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:15.114 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:15.114 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:15.114 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1529 2024-09-21T21:41:15.114 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.115 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:15.115 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.115 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:15.115 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634397+0000", 2024-09-21T21:41:15.115 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:15.115 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.115 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.115 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:15.115 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634397+0000", 2024-09-21T21:41:15.115 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:15.115 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.115 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.115 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:15.115 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634397+0000", 2024-09-21T21:41:15.116 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2e-07 2024-09-21T21:41:15.116 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.116 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.116 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:15.116 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634399+0000", 2024-09-21T21:41:15.116 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0609999999999998e-06 2024-09-21T21:41:15.116 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.116 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.116 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:15.116 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634406+0000", 2024-09-21T21:41:15.116 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7259999999999996e-06 2024-09-21T21:41:15.116 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.116 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.116 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:15.116 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634411+0000", 2024-09-21T21:41:15.117 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9339999999999997e-06 2024-09-21T21:41:15.117 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.117 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.117 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:15.117 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694480+0000", 2024-09-21T21:41:15.117 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060069105999999997 2024-09-21T21:41:15.117 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.117 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.117 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:15.117 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694527+0000", 2024-09-21T21:41:15.117 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7120000000000003e-05 2024-09-21T21:41:15.117 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.117 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.117 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:15.118 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694587+0000", 2024-09-21T21:41:15.118 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9552000000000002e-05 2024-09-21T21:41:15.118 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:15.118 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:15.118 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:15.118 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.118 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.118 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1531 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 110592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:15.118 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634417+0000", 2024-09-21T21:41:15.118 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.436867141, 2024-09-21T21:41:15.118 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060347287999999999, 2024-09-21T21:41:15.118 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:15.118 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:15.118 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:15.118 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:15.119 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:15.119 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:15.119 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1531 2024-09-21T21:41:15.119 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.119 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:15.119 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.119 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:15.119 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634417+0000", 2024-09-21T21:41:15.119 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:15.119 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.119 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.119 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:15.119 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634417+0000", 2024-09-21T21:41:15.120 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:15.120 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.120 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.120 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:15.120 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634417+0000", 2024-09-21T21:41:15.120 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1699999999999999e-07 2024-09-21T21:41:15.120 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.120 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.120 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:15.120 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634420+0000", 2024-09-21T21:41:15.120 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.243e-06 2024-09-21T21:41:15.120 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.120 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.120 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:15.121 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634426+0000", 2024-09-21T21:41:15.121 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2550000000000003e-06 2024-09-21T21:41:15.121 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.121 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.121 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:15.121 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634431+0000", 2024-09-21T21:41:15.121 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0150000000000003e-06 2024-09-21T21:41:15.121 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.121 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.121 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:15.121 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694618+0000", 2024-09-21T21:41:15.121 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060186836000000001 2024-09-21T21:41:15.121 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.122 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.122 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:15.122 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694685+0000", 2024-09-21T21:41:15.122 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6970000000000004e-05 2024-09-21T21:41:15.122 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.122 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.122 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:15.122 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694764+0000", 2024-09-21T21:41:15.122 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9552e-05 2024-09-21T21:41:15.122 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:15.122 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:15.122 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:15.122 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.122 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.122 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1532 3.e 3:724b31a3:::rbd_data.1121e540c096.0000000000000015:b [read 151552~36864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:15.123 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634437+0000", 2024-09-21T21:41:15.123 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.436847345, 2024-09-21T21:41:15.123 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060474314000000001, 2024-09-21T21:41:15.123 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:15.123 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:15.123 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:15.123 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:15.123 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:15.123 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:15.123 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1532 2024-09-21T21:41:15.123 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.123 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:15.123 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.123 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:15.124 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634437+0000", 2024-09-21T21:41:15.124 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:15.124 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.124 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.124 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:15.124 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634437+0000", 2024-09-21T21:41:15.124 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:15.124 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.124 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.124 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:15.124 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634437+0000", 2024-09-21T21:41:15.124 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9400000000000001e-07 2024-09-21T21:41:15.124 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.124 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.124 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:15.125 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634439+0000", 2024-09-21T21:41:15.125 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.114e-06 2024-09-21T21:41:15.125 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.125 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.125 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:15.125 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634446+0000", 2024-09-21T21:41:15.125 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1389999999999998e-06 2024-09-21T21:41:15.125 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.125 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.126 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:15.126 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634451+0000", 2024-09-21T21:41:15.126 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0010000000000001e-06 2024-09-21T21:41:15.126 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.126 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.126 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:15.126 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694794+0000", 2024-09-21T21:41:15.126 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060342509000000003 2024-09-21T21:41:15.126 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.127 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.127 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:15.127 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694835+0000", 2024-09-21T21:41:15.127 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0728e-05 2024-09-21T21:41:15.127 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.127 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.127 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:15.127 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694911+0000", 2024-09-21T21:41:15.127 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6428999999999996e-05 2024-09-21T21:41:15.127 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:15.127 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:15.127 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:15.127 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.127 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.128 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1534 3.e 3:724b31a3:::rbd_data.1121e540c096.0000000000000015:b [read 487424~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:15.128 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634477+0000", 2024-09-21T21:41:15.128 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.436806942, 2024-09-21T21:41:15.128 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060624009, 2024-09-21T21:41:15.128 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:15.128 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:15.128 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:15.128 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:15.128 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:15.128 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:15.128 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1534 2024-09-21T21:41:15.128 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.128 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:15.128 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.128 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:15.129 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634477+0000", 2024-09-21T21:41:15.129 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:15.129 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.129 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.129 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:15.129 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634477+0000", 2024-09-21T21:41:15.129 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:15.129 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.129 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.129 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:15.129 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634478+0000", 2024-09-21T21:41:15.129 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1399999999999997e-07 2024-09-21T21:41:15.129 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.129 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.130 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:15.130 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634480+0000", 2024-09-21T21:41:15.130 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1009999999999999e-06 2024-09-21T21:41:15.130 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.130 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.130 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:15.130 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634486+0000", 2024-09-21T21:41:15.130 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3289999999999999e-06 2024-09-21T21:41:15.130 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.130 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.130 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:15.130 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634491+0000", 2024-09-21T21:41:15.130 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6260000000000003e-06 2024-09-21T21:41:15.131 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.131 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.131 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:15.131 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694943+0000", 2024-09-21T21:41:15.131 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060451960999999999 2024-09-21T21:41:15.131 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.131 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.131 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:15.131 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695014+0000", 2024-09-21T21:41:15.131 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1587000000000004e-05 2024-09-21T21:41:15.131 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.131 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.131 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:15.131 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695101+0000", 2024-09-21T21:41:15.131 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6990999999999994e-05 2024-09-21T21:41:15.132 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:15.132 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:15.132 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:15.132 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.132 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.132 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1545 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 438272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:15.132 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634515+0000", 2024-09-21T21:41:15.132 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.436768742, 2024-09-21T21:41:15.132 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060707944, 2024-09-21T21:41:15.132 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:15.132 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:15.132 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:15.132 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:15.132 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:15.132 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:15.133 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1545 2024-09-21T21:41:15.133 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.133 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:15.133 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.133 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:15.133 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634515+0000", 2024-09-21T21:41:15.133 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:15.133 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.133 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.133 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:15.133 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634515+0000", 2024-09-21T21:41:15.133 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:15.133 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.133 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.133 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:15.134 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634516+0000", 2024-09-21T21:41:15.134 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0999999999999999e-07 2024-09-21T21:41:15.134 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.134 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.134 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:15.134 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634518+0000", 2024-09-21T21:41:15.134 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.846e-06 2024-09-21T21:41:15.134 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.134 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.134 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:15.134 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634524+0000", 2024-09-21T21:41:15.134 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2550000000000003e-06 2024-09-21T21:41:15.134 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.135 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.135 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:15.135 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634529+0000", 2024-09-21T21:41:15.135 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6940000000000003e-06 2024-09-21T21:41:15.135 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.135 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.135 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:15.135 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695127+0000", 2024-09-21T21:41:15.135 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060598921 2024-09-21T21:41:15.135 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.135 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.135 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:15.135 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695171+0000", 2024-09-21T21:41:15.135 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3588999999999999e-05 2024-09-21T21:41:15.135 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.136 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.136 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:15.136 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695223+0000", 2024-09-21T21:41:15.136 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2228999999999997e-05 2024-09-21T21:41:15.136 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:15.136 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:15.136 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:15.136 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.136 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.136 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1547 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 495616~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:15.136 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634534+0000", 2024-09-21T21:41:15.136 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.436749950999999, 2024-09-21T21:41:15.136 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060881131999999998, 2024-09-21T21:41:15.136 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:15.137 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:15.137 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:15.137 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:15.137 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:15.137 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:15.137 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1547 2024-09-21T21:41:15.137 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.137 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:15.137 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.137 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:15.137 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634534+0000", 2024-09-21T21:41:15.137 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:15.137 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.137 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.137 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:15.138 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634534+0000", 2024-09-21T21:41:15.138 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:15.138 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.138 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.138 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:15.138 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634535+0000", 2024-09-21T21:41:15.138 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.77e-07 2024-09-21T21:41:15.138 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.138 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.138 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:15.138 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634542+0000", 2024-09-21T21:41:15.138 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9000000000000006e-06 2024-09-21T21:41:15.138 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.139 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.139 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:15.139 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634549+0000", 2024-09-21T21:41:15.139 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5910000000000001e-06 2024-09-21T21:41:15.139 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.139 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.139 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:15.139 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634554+0000", 2024-09-21T21:41:15.139 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7940000000000002e-06 2024-09-21T21:41:15.139 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.139 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.139 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:15.139 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695256+0000", 2024-09-21T21:41:15.140 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.06070246 2024-09-21T21:41:15.140 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.140 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.140 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:15.140 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695332+0000", 2024-09-21T21:41:15.140 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6248000000000006e-05 2024-09-21T21:41:15.140 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.140 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.140 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:15.140 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695415+0000", 2024-09-21T21:41:15.140 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2762000000000005e-05 2024-09-21T21:41:15.140 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:15.140 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:15.140 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:15.140 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.141 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.141 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1548 3.16 3:6a41a830:::rbd_data.112145fae779.000000000000000a:a [read 299008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:15.141 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634561+0000", 2024-09-21T21:41:15.141 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.436723374, 2024-09-21T21:41:15.141 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060786743999999997, 2024-09-21T21:41:15.141 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:15.141 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:15.141 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:15.141 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:15.141 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:15.141 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:15.141 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1548 2024-09-21T21:41:15.141 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.141 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:15.142 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.142 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:15.142 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634561+0000", 2024-09-21T21:41:15.142 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:15.142 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.142 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.142 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:15.142 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634561+0000", 2024-09-21T21:41:15.142 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:15.142 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.142 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.142 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:15.142 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634561+0000", 2024-09-21T21:41:15.142 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5199999999999997e-07 2024-09-21T21:41:15.143 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.143 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.143 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:15.143 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634564+0000", 2024-09-21T21:41:15.143 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4040000000000002e-06 2024-09-21T21:41:15.143 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.143 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.143 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:15.143 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634571+0000", 2024-09-21T21:41:15.143 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4379999999999998e-06 2024-09-21T21:41:15.143 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.143 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.143 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:15.143 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634576+0000", 2024-09-21T21:41:15.144 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1399999999999999e-06 2024-09-21T21:41:15.144 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.144 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.144 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:15.144 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695247+0000", 2024-09-21T21:41:15.144 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060670397000000001 2024-09-21T21:41:15.144 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.144 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.144 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:15.144 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695299+0000", 2024-09-21T21:41:15.144 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2564e-05 2024-09-21T21:41:15.144 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.144 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.144 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:15.144 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695347+0000", 2024-09-21T21:41:15.145 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8248999999999998e-05 2024-09-21T21:41:15.145 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:15.145 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:15.145 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:15.145 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.145 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.145 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1549 3.16 3:6a41a830:::rbd_data.112145fae779.000000000000000a:a [read 356352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:15.145 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634583+0000", 2024-09-21T21:41:15.145 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.436701461, 2024-09-21T21:41:15.145 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060889459, 2024-09-21T21:41:15.145 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:15.145 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:15.145 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:15.146 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:15.146 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:15.146 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:15.146 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1549 2024-09-21T21:41:15.146 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.146 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:15.146 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.146 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:15.146 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634583+0000", 2024-09-21T21:41:15.146 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:15.146 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.146 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.146 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:15.146 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634583+0000", 2024-09-21T21:41:15.147 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:15.147 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.147 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.147 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:15.147 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634583+0000", 2024-09-21T21:41:15.147 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1899999999999998e-07 2024-09-21T21:41:15.147 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.147 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.147 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:15.147 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634585+0000", 2024-09-21T21:41:15.147 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2129999999999998e-06 2024-09-21T21:41:15.147 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.147 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.147 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:15.148 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634592+0000", 2024-09-21T21:41:15.148 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2840000000000003e-06 2024-09-21T21:41:15.148 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.148 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.148 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:15.148 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634596+0000", 2024-09-21T21:41:15.148 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9309999999999999e-06 2024-09-21T21:41:15.148 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.148 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.148 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:15.148 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695371+0000", 2024-09-21T21:41:15.148 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060774425999999999 2024-09-21T21:41:15.148 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.149 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.149 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:15.149 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695419+0000", 2024-09-21T21:41:15.149 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8607999999999999e-05 2024-09-21T21:41:15.149 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.149 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.149 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:15.149 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695472+0000", 2024-09-21T21:41:15.149 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2577999999999997e-05 2024-09-21T21:41:15.149 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:15.149 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:15.149 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:15.149 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.149 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.150 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1588 3.16 3:695c582e:::rbd_data.1121e540c096.0000000000000008:b [read 147456~40960] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:15.150 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.636942+0000", 2024-09-21T21:41:15.150 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.434341684, 2024-09-21T21:41:15.150 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.058643811999999997, 2024-09-21T21:41:15.150 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:15.150 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:15.150 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:15.150 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:15.150 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:15.150 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:15.150 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1588 2024-09-21T21:41:15.150 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.151 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:15.151 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.151 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:15.151 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636942+0000", 2024-09-21T21:41:15.151 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:15.151 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.151 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.151 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:15.151 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636942+0000", 2024-09-21T21:41:15.151 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:15.151 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.151 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.151 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:15.151 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636943+0000", 2024-09-21T21:41:15.152 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4199999999999996e-07 2024-09-21T21:41:15.152 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.152 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.152 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:15.152 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636945+0000", 2024-09-21T21:41:15.152 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.401e-06 2024-09-21T21:41:15.152 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.152 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.152 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:15.152 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636952+0000", 2024-09-21T21:41:15.152 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5379999999999999e-06 2024-09-21T21:41:15.152 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.152 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.153 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:15.153 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636958+0000", 2024-09-21T21:41:15.153 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.489e-06 2024-09-21T21:41:15.153 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.153 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.153 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:15.153 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695497+0000", 2024-09-21T21:41:15.153 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.058538504999999998 2024-09-21T21:41:15.153 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.153 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.153 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:15.153 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695533+0000", 2024-09-21T21:41:15.153 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6118000000000002e-05 2024-09-21T21:41:15.153 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.154 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.154 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:15.154 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695586+0000", 2024-09-21T21:41:15.154 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3219e-05 2024-09-21T21:41:15.154 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:15.154 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:15.154 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:15.154 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.154 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.154 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1589 3.16 3:695c582e:::rbd_data.1121e540c096.0000000000000008:b [read 487424~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:15.154 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.636965+0000", 2024-09-21T21:41:15.154 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.434318593, 2024-09-21T21:41:15.154 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.058728152999999998, 2024-09-21T21:41:15.154 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:15.154 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:15.155 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:15.155 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:15.155 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:15.155 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:15.155 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1589 2024-09-21T21:41:15.155 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.155 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:15.155 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.155 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:15.155 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636965+0000", 2024-09-21T21:41:15.155 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:15.155 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.155 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.155 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:15.156 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636965+0000", 2024-09-21T21:41:15.156 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:15.156 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.156 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.156 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:15.156 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636966+0000", 2024-09-21T21:41:15.156 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3799999999999998e-07 2024-09-21T21:41:15.156 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.156 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.156 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:15.156 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636968+0000", 2024-09-21T21:41:15.156 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2400000000000002e-06 2024-09-21T21:41:15.156 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.156 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.156 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:15.157 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636976+0000", 2024-09-21T21:41:15.157 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5050000000000001e-06 2024-09-21T21:41:15.157 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.157 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.157 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:15.157 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636981+0000", 2024-09-21T21:41:15.157 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2170000000000002e-06 2024-09-21T21:41:15.157 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.157 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.157 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:15.157 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695610+0000", 2024-09-21T21:41:15.157 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.058628998000000002 2024-09-21T21:41:15.157 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.157 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.158 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:15.158 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695647+0000", 2024-09-21T21:41:15.158 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6752000000000003e-05 2024-09-21T21:41:15.158 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.158 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.158 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:15.158 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695694+0000", 2024-09-21T21:41:15.158 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7002999999999997e-05 2024-09-21T21:41:15.158 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:15.158 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:15.158 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:15.158 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.159 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.159 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1607 3.e 3:724b31a3:::rbd_data.1121e540c096.0000000000000015:b [read 299008~36864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:15.159 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.637062+0000", 2024-09-21T21:41:15.159 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.434222502000001, 2024-09-21T21:41:15.159 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.058623078000000002, 2024-09-21T21:41:15.159 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:15.159 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:15.159 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:15.159 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:15.159 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:15.159 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:15.159 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1607 2024-09-21T21:41:15.159 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.159 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:15.160 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.160 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:15.160 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.637062+0000", 2024-09-21T21:41:15.160 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:15.160 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.160 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.160 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:15.160 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.637062+0000", 2024-09-21T21:41:15.160 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:15.160 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.160 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.160 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:15.160 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.637062+0000", 2024-09-21T21:41:15.160 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0900000000000002e-07 2024-09-21T21:41:15.160 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.161 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.161 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:15.161 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.637069+0000", 2024-09-21T21:41:15.161 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.267e-06 2024-09-21T21:41:15.161 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.161 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.161 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:15.161 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.637076+0000", 2024-09-21T21:41:15.161 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7320000000000001e-06 2024-09-21T21:41:15.161 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.161 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.161 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:15.161 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.637082+0000", 2024-09-21T21:41:15.161 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4760000000000003e-06 2024-09-21T21:41:15.161 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.162 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.162 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:15.162 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695604+0000", 2024-09-21T21:41:15.162 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.058521261999999998 2024-09-21T21:41:15.162 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.162 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.162 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:15.162 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695638+0000", 2024-09-21T21:41:15.162 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4292999999999998e-05 2024-09-21T21:41:15.162 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:15.162 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:15.162 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:15.162 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695685+0000", 2024-09-21T21:41:15.163 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6638999999999998e-05 2024-09-21T21:41:15.163 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:15.163 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:15.163 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:15.163 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:15.163 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:15.163 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:15.164 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:15.162+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:15.165 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T21:41:15.265 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:15.264+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:15.302 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:15.302 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:41:15.302 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 0 2024-09-21T21:41:15.302 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:15.313 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T21:41:15.366 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:15.364+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:15.467 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:15.465+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:15.472 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:15.473 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:41:15.473 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:41:15.473 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:41:15.473 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:15.481 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T21:41:15.568 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:15.566+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:15.586 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:15.586 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 20, 2024-09-21T21:41:15.586 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 600, 2024-09-21T21:41:15.586 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:41:15.586 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.586 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4417.0:228 3.6 3:678f83b3:::rbd_data.11214be7c411.000000000000002a:head [delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-21T21:41:15.586 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:43.553796+0000", 2024-09-21T21:41:15.586 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 32.029943617999997, 2024-09-21T21:41:15.586 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0098649099999999993, 2024-09-21T21:41:15.586 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:15.587 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:15.587 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:41:15.587 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:15.587 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4417", 2024-09-21T21:41:15.587 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3507586822", 2024-09-21T21:41:15.587 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 228 2024-09-21T21:41:15.587 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.587 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:15.587 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.587 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:15.587 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.553796+0000", 2024-09-21T21:41:15.587 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:15.587 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.587 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.588 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:15.588 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.553796+0000", 2024-09-21T21:41:15.588 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:15.588 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.588 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.588 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:15.588 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.553797+0000", 2024-09-21T21:41:15.588 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6899999999999997e-07 2024-09-21T21:41:15.588 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.588 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.588 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:15.588 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.553799+0000", 2024-09-21T21:41:15.588 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3e-06 2024-09-21T21:41:15.589 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.589 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.589 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:15.589 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.553805+0000", 2024-09-21T21:41:15.589 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1310000000000003e-06 2024-09-21T21:41:15.589 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.589 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.589 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:15.589 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.553816+0000", 2024-09-21T21:41:15.589 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0506000000000001e-05 2024-09-21T21:41:15.589 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.589 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.589 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:15.590 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.555870+0000", 2024-09-21T21:41:15.590 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0020538510000000002 2024-09-21T21:41:15.590 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.590 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.590 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:15.590 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.555942+0000", 2024-09-21T21:41:15.590 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.2436999999999998e-05 2024-09-21T21:41:15.590 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.590 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.590 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 1", 2024-09-21T21:41:15.590 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.556121+0000", 2024-09-21T21:41:15.590 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000178526 2024-09-21T21:41:15.590 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.591 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.591 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:41:15.591 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.559481+0000", 2024-09-21T21:41:15.591 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0033603019999999999 2024-09-21T21:41:15.591 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.591 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.591 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:41:15.591 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.563144+0000", 2024-09-21T21:41:15.591 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0036625799999999999 2024-09-21T21:41:15.591 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.591 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.591 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:41:15.591 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.563185+0000", 2024-09-21T21:41:15.591 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1084000000000002e-05 2024-09-21T21:41:15.592 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.592 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.592 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:15.592 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:43.563661+0000", 2024-09-21T21:41:15.592 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00047662399999999998 2024-09-21T21:41:15.592 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:15.592 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:15.592 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:15.592 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.592 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.592 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:1608 3.12 3:4bfa5158:::rbd_data.112145fae779.0000000000000002:a [read 10678272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:15.592 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.381876+0000", 2024-09-21T21:41:15.592 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 17.201863666000001, 2024-09-21T21:41:15.592 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0096257990000000009, 2024-09-21T21:41:15.593 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:15.593 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:15.593 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:15.593 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:15.593 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:15.593 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:15.593 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1608 2024-09-21T21:41:15.593 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.593 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:15.593 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.593 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:15.593 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.381876+0000", 2024-09-21T21:41:15.593 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:15.593 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.594 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.594 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:15.594 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.381876+0000", 2024-09-21T21:41:15.594 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:15.594 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.594 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.594 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:15.594 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.381876+0000", 2024-09-21T21:41:15.594 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1e-07 2024-09-21T21:41:15.594 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.594 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.594 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:15.594 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.381878+0000", 2024-09-21T21:41:15.594 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.111e-06 2024-09-21T21:41:15.595 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.595 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.595 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:15.595 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.381881+0000", 2024-09-21T21:41:15.595 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0570000000000001e-06 2024-09-21T21:41:15.595 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.595 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.595 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:15.595 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.381883+0000", 2024-09-21T21:41:15.595 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3429999999999998e-06 2024-09-21T21:41:15.595 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.595 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.595 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:15.596 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.391392+0000", 2024-09-21T21:41:15.596 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.009509221 2024-09-21T21:41:15.596 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.596 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.596 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:15.596 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.391448+0000", 2024-09-21T21:41:15.596 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5961999999999997e-05 2024-09-21T21:41:15.596 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.596 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.596 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:15.596 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.391502+0000", 2024-09-21T21:41:15.596 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3894999999999998e-05 2024-09-21T21:41:15.597 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:15.597 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:15.597 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:15.597 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.597 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.597 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:1610 3.12 3:4bfa5158:::rbd_data.112145fae779.0000000000000002:a [read 10735616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:15.597 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.381886+0000", 2024-09-21T21:41:15.597 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 17.201853880000002, 2024-09-21T21:41:15.597 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0097775250000000005, 2024-09-21T21:41:15.597 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:15.597 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:15.597 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:15.597 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:15.598 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:15.598 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:15.598 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1610 2024-09-21T21:41:15.598 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.598 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:15.598 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.598 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:15.598 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.381886+0000", 2024-09-21T21:41:15.598 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:15.598 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.598 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.598 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:15.598 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.381886+0000", 2024-09-21T21:41:15.599 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:15.599 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.599 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.599 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:15.599 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.381886+0000", 2024-09-21T21:41:15.599 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.05e-07 2024-09-21T21:41:15.599 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.599 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.599 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:15.599 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.381887+0000", 2024-09-21T21:41:15.599 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1960000000000001e-06 2024-09-21T21:41:15.599 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.599 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.599 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:15.600 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.381891+0000", 2024-09-21T21:41:15.600 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.061e-06 2024-09-21T21:41:15.600 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.600 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.600 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:15.600 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.381893+0000", 2024-09-21T21:41:15.600 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.463e-06 2024-09-21T21:41:15.600 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.600 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.600 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:15.600 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.391533+0000", 2024-09-21T21:41:15.600 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0096403979999999997 2024-09-21T21:41:15.600 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.600 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.601 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:15.601 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.391593+0000", 2024-09-21T21:41:15.601 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9778999999999998e-05 2024-09-21T21:41:15.601 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.601 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.601 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:15.601 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.391664+0000", 2024-09-21T21:41:15.601 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0423e-05 2024-09-21T21:41:15.601 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:15.601 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:15.601 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:15.601 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.601 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.601 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:1800 3.12 3:4bfa5158:::rbd_data.112145fae779.0000000000000002:a [read 10391552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:15.602 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.382760+0000", 2024-09-21T21:41:15.602 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 17.200979742000001, 2024-09-21T21:41:15.602 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0094873609999999997, 2024-09-21T21:41:15.602 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:15.602 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:15.602 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:15.602 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:15.602 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:15.602 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:15.602 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1800 2024-09-21T21:41:15.602 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.602 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:15.602 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.602 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:15.603 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382760+0000", 2024-09-21T21:41:15.603 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:15.603 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.603 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.603 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:15.603 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382760+0000", 2024-09-21T21:41:15.603 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:15.603 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.603 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.603 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:15.603 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382761+0000", 2024-09-21T21:41:15.603 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.63e-07 2024-09-21T21:41:15.603 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.604 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.604 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:15.604 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382763+0000", 2024-09-21T21:41:15.604 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.4080000000000001e-06 2024-09-21T21:41:15.604 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.604 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.604 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:15.604 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382770+0000", 2024-09-21T21:41:15.604 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.528e-06 2024-09-21T21:41:15.604 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.604 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.604 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:15.604 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382775+0000", 2024-09-21T21:41:15.604 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0540000000000002e-06 2024-09-21T21:41:15.604 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.605 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.605 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:15.605 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.392092+0000", 2024-09-21T21:41:15.605 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0093168999999999995 2024-09-21T21:41:15.605 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.605 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.605 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:15.605 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.392173+0000", 2024-09-21T21:41:15.605 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.1897999999999994e-05 2024-09-21T21:41:15.605 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.605 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.605 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:15.605 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.392248+0000", 2024-09-21T21:41:15.606 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.4109999999999993e-05 2024-09-21T21:41:15.606 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:15.606 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:15.606 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:15.606 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.606 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.606 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:1802 3.12 3:4bfa5158:::rbd_data.112145fae779.0000000000000002:a [read 10448896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:15.606 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.382781+0000", 2024-09-21T21:41:15.606 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 17.200958494999998, 2024-09-21T21:41:15.606 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0096511759999999992, 2024-09-21T21:41:15.606 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:15.606 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:15.606 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:15.606 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:15.607 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:15.607 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:15.607 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1802 2024-09-21T21:41:15.607 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.607 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:15.607 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.607 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:15.607 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382781+0000", 2024-09-21T21:41:15.607 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:15.607 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.607 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.607 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:15.607 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382781+0000", 2024-09-21T21:41:15.607 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:15.608 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.608 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.608 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:15.608 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382782+0000", 2024-09-21T21:41:15.608 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.41e-07 2024-09-21T21:41:15.608 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.608 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.608 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:15.608 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382784+0000", 2024-09-21T21:41:15.608 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1509999999999998e-06 2024-09-21T21:41:15.608 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.608 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.608 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:15.608 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382799+0000", 2024-09-21T21:41:15.609 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.4711e-05 2024-09-21T21:41:15.609 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.609 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.609 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:15.609 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382803+0000", 2024-09-21T21:41:15.609 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8130000000000003e-06 2024-09-21T21:41:15.609 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.609 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.609 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:15.609 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.392304+0000", 2024-09-21T21:41:15.609 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0095000510000000007 2024-09-21T21:41:15.609 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.609 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.609 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:15.610 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.392383+0000", 2024-09-21T21:41:15.610 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.9666999999999995e-05 2024-09-21T21:41:15.610 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.610 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.610 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:15.610 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.392433+0000", 2024-09-21T21:41:15.610 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9441999999999999e-05 2024-09-21T21:41:15.610 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:15.610 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:15.610 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:15.610 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.610 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.610 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:1804 3.12 3:4bfa5158:::rbd_data.112145fae779.0000000000000002:a [read 10539008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:15.610 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.382815+0000", 2024-09-21T21:41:15.611 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 17.200924639, 2024-09-21T21:41:15.611 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.009793263, 2024-09-21T21:41:15.611 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:15.611 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:15.611 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:15.611 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:15.611 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:15.611 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:15.611 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1804 2024-09-21T21:41:15.611 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.611 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:15.611 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.611 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:15.611 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382815+0000", 2024-09-21T21:41:15.612 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:15.612 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.612 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.612 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:15.612 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382815+0000", 2024-09-21T21:41:15.612 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:15.612 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.612 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.612 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:15.612 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382816+0000", 2024-09-21T21:41:15.612 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.06e-07 2024-09-21T21:41:15.612 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.612 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.613 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:15.613 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382818+0000", 2024-09-21T21:41:15.613 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.5069999999999999e-06 2024-09-21T21:41:15.613 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.613 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.613 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:15.613 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382825+0000", 2024-09-21T21:41:15.613 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0899999999999999e-06 2024-09-21T21:41:15.613 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.613 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.613 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:15.613 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382831+0000", 2024-09-21T21:41:15.613 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1830000000000002e-06 2024-09-21T21:41:15.613 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.613 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.614 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:15.614 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.392460+0000", 2024-09-21T21:41:15.614 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0096296730000000001 2024-09-21T21:41:15.614 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.614 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.614 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:15.614 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.392530+0000", 2024-09-21T21:41:15.614 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9590000000000003e-05 2024-09-21T21:41:15.614 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.614 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.614 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:15.614 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.392609+0000", 2024-09-21T21:41:15.614 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.8714000000000006e-05 2024-09-21T21:41:15.614 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:15.615 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:15.615 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:15.615 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.615 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.615 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:1806 3.12 3:4bfa5158:::rbd_data.112145fae779.0000000000000002:a [read 10596352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:15.615 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.382838+0000", 2024-09-21T21:41:15.615 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 17.200901836, 2024-09-21T21:41:15.615 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0099573270000000002, 2024-09-21T21:41:15.615 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:15.615 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:15.615 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:15.615 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:15.615 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:15.615 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:15.616 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1806 2024-09-21T21:41:15.616 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.616 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:15.616 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.616 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:15.616 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382838+0000", 2024-09-21T21:41:15.616 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:15.616 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.616 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.617 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:15.617 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382838+0000", 2024-09-21T21:41:15.618 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:15.618 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.618 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.618 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:15.618 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382839+0000", 2024-09-21T21:41:15.618 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7599999999999997e-07 2024-09-21T21:41:15.618 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.618 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.618 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:15.618 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382841+0000", 2024-09-21T21:41:15.618 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3070000000000001e-06 2024-09-21T21:41:15.619 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.619 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.619 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:15.619 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382847+0000", 2024-09-21T21:41:15.619 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2870000000000001e-06 2024-09-21T21:41:15.619 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.619 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.619 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:15.619 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382852+0000", 2024-09-21T21:41:15.619 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5950000000000001e-06 2024-09-21T21:41:15.619 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.619 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.619 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:15.619 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.392651+0000", 2024-09-21T21:41:15.620 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0097987419999999992 2024-09-21T21:41:15.620 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.620 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.620 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:15.620 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.392731+0000", 2024-09-21T21:41:15.620 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0154999999999994e-05 2024-09-21T21:41:15.620 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.620 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.620 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:15.620 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.392795+0000", 2024-09-21T21:41:15.620 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4665000000000005e-05 2024-09-21T21:41:15.620 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:15.620 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:15.620 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:15.621 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.621 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.621 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:1807 3.12 3:4bfa5158:::rbd_data.112145fae779.0000000000000002:a [read 10629120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:15.621 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.382858+0000", 2024-09-21T21:41:15.621 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 17.200881816999999, 2024-09-21T21:41:15.621 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.010090127000000001, 2024-09-21T21:41:15.621 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:15.621 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:15.621 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:15.621 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:15.621 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:15.621 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:15.621 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1807 2024-09-21T21:41:15.621 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.621 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:15.622 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.622 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:15.622 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382858+0000", 2024-09-21T21:41:15.622 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:15.622 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.622 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.622 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:15.622 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382858+0000", 2024-09-21T21:41:15.622 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:15.622 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.622 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.622 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:15.622 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382859+0000", 2024-09-21T21:41:15.623 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7599999999999997e-07 2024-09-21T21:41:15.623 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.623 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.623 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:15.623 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382861+0000", 2024-09-21T21:41:15.623 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.176e-06 2024-09-21T21:41:15.623 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.623 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.623 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:15.623 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382867+0000", 2024-09-21T21:41:15.623 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0990000000000004e-06 2024-09-21T21:41:15.623 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.623 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.623 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:15.624 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382872+0000", 2024-09-21T21:41:15.624 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8230000000000002e-06 2024-09-21T21:41:15.624 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.624 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.624 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:15.624 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.392829+0000", 2024-09-21T21:41:15.624 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0099573869999999998 2024-09-21T21:41:15.624 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.624 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.624 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:15.624 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.392888+0000", 2024-09-21T21:41:15.624 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8585000000000002e-05 2024-09-21T21:41:15.624 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.624 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.625 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:15.625 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.392948+0000", 2024-09-21T21:41:15.625 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0581000000000002e-05 2024-09-21T21:41:15.625 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:15.625 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:15.625 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:15.625 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.625 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.625 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:1809 3.12 3:4bfa5158:::rbd_data.112145fae779.0000000000000002:a [read 10686464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:15.625 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.382878+0000", 2024-09-21T21:41:15.625 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 17.200861924000002, 2024-09-21T21:41:15.625 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.010264758000000001, 2024-09-21T21:41:15.625 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:15.625 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:15.626 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:15.626 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:15.626 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:15.626 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:15.626 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1809 2024-09-21T21:41:15.626 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.626 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:15.626 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.626 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:15.626 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382878+0000", 2024-09-21T21:41:15.626 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:15.626 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.626 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.626 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:15.627 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382878+0000", 2024-09-21T21:41:15.627 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:15.627 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.627 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.627 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:15.627 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382878+0000", 2024-09-21T21:41:15.627 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8000000000000001e-07 2024-09-21T21:41:15.627 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.627 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.627 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:15.627 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382880+0000", 2024-09-21T21:41:15.627 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0470000000000001e-06 2024-09-21T21:41:15.628 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.628 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.628 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:15.628 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382887+0000", 2024-09-21T21:41:15.628 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2899999999999999e-06 2024-09-21T21:41:15.628 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.628 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.628 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:15.628 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382891+0000", 2024-09-21T21:41:15.628 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.566e-06 2024-09-21T21:41:15.628 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.628 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.628 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:15.628 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.392978+0000", 2024-09-21T21:41:15.629 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.010086603 2024-09-21T21:41:15.629 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.629 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.629 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:15.629 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.393063+0000", 2024-09-21T21:41:15.629 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.5197000000000004e-05 2024-09-21T21:41:15.629 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.629 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.629 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:15.629 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.393143+0000", 2024-09-21T21:41:15.629 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.9674999999999999e-05 2024-09-21T21:41:15.629 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:15.629 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:15.629 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:15.630 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.630 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.630 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:1810 3.12 3:4bfa5158:::rbd_data.112145fae779.0000000000000002:a [read 10743808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:15.630 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.382898+0000", 2024-09-21T21:41:15.630 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 17.200842036000001, 2024-09-21T21:41:15.630 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.010450245, 2024-09-21T21:41:15.630 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:15.630 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:15.630 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:15.630 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:15.630 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:15.630 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:15.630 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1810 2024-09-21T21:41:15.630 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.630 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:15.631 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.631 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:15.631 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382898+0000", 2024-09-21T21:41:15.631 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:15.631 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.631 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.631 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:15.631 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382898+0000", 2024-09-21T21:41:15.631 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:15.631 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.631 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.631 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:15.632 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382898+0000", 2024-09-21T21:41:15.632 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2e-07 2024-09-21T21:41:15.632 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.632 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.632 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:15.632 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382900+0000", 2024-09-21T21:41:15.632 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9439999999999999e-06 2024-09-21T21:41:15.632 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.632 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.632 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:15.632 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382906+0000", 2024-09-21T21:41:15.632 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.981e-06 2024-09-21T21:41:15.632 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.632 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.633 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:15.633 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382912+0000", 2024-09-21T21:41:15.633 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5389999999999998e-06 2024-09-21T21:41:15.633 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.633 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.633 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:15.633 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.393190+0000", 2024-09-21T21:41:15.633 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.010277846 2024-09-21T21:41:15.633 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.633 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.633 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:15.633 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.393271+0000", 2024-09-21T21:41:15.633 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.1666e-05 2024-09-21T21:41:15.634 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.634 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.634 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:15.634 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.393348+0000", 2024-09-21T21:41:15.634 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.6848999999999996e-05 2024-09-21T21:41:15.634 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:15.634 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:15.634 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:15.634 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.634 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.634 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:1856 3.12 3:4bfa5158:::rbd_data.112145fae779.0000000000000002:a [read 10240000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:15.634 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.382998+0000", 2024-09-21T21:41:15.634 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 17.200741546, 2024-09-21T21:41:15.635 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.010571644, 2024-09-21T21:41:15.635 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:15.635 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:15.635 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:15.635 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:15.635 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:15.635 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:15.635 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1856 2024-09-21T21:41:15.635 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.635 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:15.635 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.635 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:15.635 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382998+0000", 2024-09-21T21:41:15.635 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:15.636 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.636 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.636 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:15.636 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382998+0000", 2024-09-21T21:41:15.636 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:15.636 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.636 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.636 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:15.636 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.382999+0000", 2024-09-21T21:41:15.636 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7700000000000004e-07 2024-09-21T21:41:15.636 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.636 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.636 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:15.637 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383003+0000", 2024-09-21T21:41:15.637 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.822e-06 2024-09-21T21:41:15.637 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.637 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.637 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:15.637 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383014+0000", 2024-09-21T21:41:15.637 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0713000000000001e-05 2024-09-21T21:41:15.637 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.637 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.637 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:15.637 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383020+0000", 2024-09-21T21:41:15.637 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3160000000000002e-06 2024-09-21T21:41:15.637 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.637 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.638 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:15.638 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.393394+0000", 2024-09-21T21:41:15.638 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.010374362 2024-09-21T21:41:15.638 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.638 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.638 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:15.638 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.393484+0000", 2024-09-21T21:41:15.638 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.9975000000000005e-05 2024-09-21T21:41:15.638 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.638 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.638 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:15.638 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.393570+0000", 2024-09-21T21:41:15.638 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.5679000000000004e-05 2024-09-21T21:41:15.638 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:15.639 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:15.639 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:15.639 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.639 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.639 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:1858 3.12 3:4bfa5158:::rbd_data.112145fae779.0000000000000002:a [read 10297344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:15.639 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.383036+0000", 2024-09-21T21:41:15.639 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 17.200704369, 2024-09-21T21:41:15.639 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.010660364, 2024-09-21T21:41:15.639 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:15.639 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:15.639 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:15.639 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:15.639 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:15.640 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:15.640 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1858 2024-09-21T21:41:15.640 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.640 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:15.640 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.640 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:15.640 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383036+0000", 2024-09-21T21:41:15.640 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:15.640 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.640 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.640 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:15.640 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383036+0000", 2024-09-21T21:41:15.640 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:15.640 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.641 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.641 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:15.641 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383036+0000", 2024-09-21T21:41:15.641 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8899999999999999e-07 2024-09-21T21:41:15.641 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.641 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.641 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:15.641 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383039+0000", 2024-09-21T21:41:15.641 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.6510000000000001e-06 2024-09-21T21:41:15.641 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.641 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.641 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:15.641 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383048+0000", 2024-09-21T21:41:15.641 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6530000000000006e-06 2024-09-21T21:41:15.642 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.642 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.642 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:15.642 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383053+0000", 2024-09-21T21:41:15.642 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.933e-06 2024-09-21T21:41:15.642 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.642 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.642 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:15.642 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.393608+0000", 2024-09-21T21:41:15.642 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.01055442 2024-09-21T21:41:15.642 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.642 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.642 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:15.642 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.393650+0000", 2024-09-21T21:41:15.643 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2586999999999997e-05 2024-09-21T21:41:15.643 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.643 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.643 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:15.643 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.393696+0000", 2024-09-21T21:41:15.643 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5531000000000002e-05 2024-09-21T21:41:15.643 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:15.643 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:15.643 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:15.643 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.643 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.643 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:1861 3.12 3:4bfa5158:::rbd_data.112145fae779.0000000000000002:a [read 10387456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:15.643 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.383063+0000", 2024-09-21T21:41:15.643 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 17.200676448999999, 2024-09-21T21:41:15.644 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.010754104, 2024-09-21T21:41:15.644 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:15.644 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:15.644 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:15.644 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:15.644 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:15.644 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:15.644 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1861 2024-09-21T21:41:15.644 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.644 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:15.644 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.644 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:15.644 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383063+0000", 2024-09-21T21:41:15.645 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:15.645 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.645 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.645 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:15.645 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383063+0000", 2024-09-21T21:41:15.645 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:15.645 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.645 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.645 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:15.645 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383064+0000", 2024-09-21T21:41:15.645 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9399999999999995e-07 2024-09-21T21:41:15.645 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.645 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.645 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:15.646 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383067+0000", 2024-09-21T21:41:15.646 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.6019999999999998e-06 2024-09-21T21:41:15.646 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.646 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.646 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:15.646 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383075+0000", 2024-09-21T21:41:15.646 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.1850000000000004e-06 2024-09-21T21:41:15.646 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.646 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.646 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:15.646 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383080+0000", 2024-09-21T21:41:15.646 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3909999999999997e-06 2024-09-21T21:41:15.646 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.646 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.647 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:15.647 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.393727+0000", 2024-09-21T21:41:15.647 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.010646631 2024-09-21T21:41:15.647 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.647 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.647 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:15.647 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.393777+0000", 2024-09-21T21:41:15.647 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0356999999999999e-05 2024-09-21T21:41:15.647 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.647 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.647 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:15.647 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.393818+0000", 2024-09-21T21:41:15.647 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.0444000000000001e-05 2024-09-21T21:41:15.647 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:15.648 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:15.648 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:15.648 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.648 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.648 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:1863 3.12 3:4bfa5158:::rbd_data.112145fae779.0000000000000002:a [read 10444800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:15.648 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.383088+0000", 2024-09-21T21:41:15.648 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 17.200651805, 2024-09-21T21:41:15.648 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.010836438, 2024-09-21T21:41:15.648 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:15.648 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:15.648 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:15.648 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:15.648 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:15.649 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:15.649 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1863 2024-09-21T21:41:15.649 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.649 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:15.649 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.649 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:15.649 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383088+0000", 2024-09-21T21:41:15.649 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:15.649 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.649 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.649 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:15.649 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383088+0000", 2024-09-21T21:41:15.649 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:15.650 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.650 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.650 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:15.650 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383089+0000", 2024-09-21T21:41:15.650 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2500000000000001e-07 2024-09-21T21:41:15.650 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.650 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.650 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:15.650 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383091+0000", 2024-09-21T21:41:15.650 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.283e-06 2024-09-21T21:41:15.650 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.650 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.650 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:15.650 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383097+0000", 2024-09-21T21:41:15.651 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1279999999999996e-06 2024-09-21T21:41:15.651 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.651 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.651 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:15.651 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383103+0000", 2024-09-21T21:41:15.651 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6219999999999997e-06 2024-09-21T21:41:15.651 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.651 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.651 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:15.651 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.393845+0000", 2024-09-21T21:41:15.651 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.010742188999999999 2024-09-21T21:41:15.651 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.651 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.651 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:15.652 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.393886+0000", 2024-09-21T21:41:15.652 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1119999999999999e-05 2024-09-21T21:41:15.652 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.652 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.652 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:15.652 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.393925+0000", 2024-09-21T21:41:15.652 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8671000000000003e-05 2024-09-21T21:41:15.652 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:15.652 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:15.652 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:15.652 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.652 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.652 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:1865 3.12 3:4bfa5158:::rbd_data.112145fae779.0000000000000002:a [read 10534912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:15.652 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.383134+0000", 2024-09-21T21:41:15.653 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 17.200606017999998, 2024-09-21T21:41:15.653 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.01092018, 2024-09-21T21:41:15.653 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:15.653 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:15.653 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:15.653 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:15.653 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:15.653 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:15.653 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1865 2024-09-21T21:41:15.653 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.653 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:15.653 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.653 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:15.653 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383134+0000", 2024-09-21T21:41:15.653 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:15.654 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.654 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.654 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:15.654 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383134+0000", 2024-09-21T21:41:15.654 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:15.654 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.654 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.654 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:15.654 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383135+0000", 2024-09-21T21:41:15.654 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.9800000000000002e-07 2024-09-21T21:41:15.654 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.654 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.654 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:15.655 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383140+0000", 2024-09-21T21:41:15.655 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7099999999999998e-06 2024-09-21T21:41:15.655 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.655 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.655 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:15.655 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383150+0000", 2024-09-21T21:41:15.655 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0472000000000001e-05 2024-09-21T21:41:15.655 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.655 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.655 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:15.655 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383157+0000", 2024-09-21T21:41:15.655 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9029999999999998e-06 2024-09-21T21:41:15.655 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.655 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.655 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:15.656 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.393973+0000", 2024-09-21T21:41:15.656 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.010816299 2024-09-21T21:41:15.656 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.656 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.656 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:15.656 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.394015+0000", 2024-09-21T21:41:15.656 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1467000000000001e-05 2024-09-21T21:41:15.656 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.656 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.656 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:15.656 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.394054+0000", 2024-09-21T21:41:15.656 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.9331e-05 2024-09-21T21:41:15.656 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:15.656 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:15.656 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:15.657 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.657 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.657 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:1867 3.12 3:4bfa5158:::rbd_data.112145fae779.0000000000000002:a [read 10592256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:15.657 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.383287+0000", 2024-09-21T21:41:15.657 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 17.200452692999999, 2024-09-21T21:41:15.657 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.01089415, 2024-09-21T21:41:15.657 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:15.657 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:15.657 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:15.657 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:15.657 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:15.657 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:15.657 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1867 2024-09-21T21:41:15.657 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.658 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:15.658 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.658 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:15.658 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383287+0000", 2024-09-21T21:41:15.658 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:15.658 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.658 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.658 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:15.658 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383287+0000", 2024-09-21T21:41:15.658 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:15.658 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.658 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.658 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:15.659 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383288+0000", 2024-09-21T21:41:15.659 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.02e-07 2024-09-21T21:41:15.659 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.659 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.659 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:15.659 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383293+0000", 2024-09-21T21:41:15.659 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8339999999999998e-06 2024-09-21T21:41:15.659 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.659 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.659 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:15.659 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383305+0000", 2024-09-21T21:41:15.659 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1677e-05 2024-09-21T21:41:15.659 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.659 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.660 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:15.660 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383312+0000", 2024-09-21T21:41:15.660 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9079999999999998e-06 2024-09-21T21:41:15.660 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.660 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.660 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:15.660 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.394080+0000", 2024-09-21T21:41:15.660 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.010768402 2024-09-21T21:41:15.660 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.660 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.660 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:15.660 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.394131+0000", 2024-09-21T21:41:15.660 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0926e-05 2024-09-21T21:41:15.660 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.661 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.661 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:15.661 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.394181+0000", 2024-09-21T21:41:15.661 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0501000000000003e-05 2024-09-21T21:41:15.661 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:15.661 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:15.661 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:15.661 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.661 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.661 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:1869 3.12 3:4bfa5158:::rbd_data.112145fae779.0000000000000002:a [read 10625024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:15.661 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.383348+0000", 2024-09-21T21:41:15.661 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 17.200392141999998, 2024-09-21T21:41:15.661 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.011004575000000001, 2024-09-21T21:41:15.661 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:15.662 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:15.662 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:15.662 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:15.662 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:15.662 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:15.662 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1869 2024-09-21T21:41:15.662 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.662 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:15.662 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.662 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:15.662 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383348+0000", 2024-09-21T21:41:15.662 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:15.662 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.662 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.663 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:15.663 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383348+0000", 2024-09-21T21:41:15.663 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:15.663 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.663 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.663 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:15.663 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383348+0000", 2024-09-21T21:41:15.663 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5400000000000001e-07 2024-09-21T21:41:15.663 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.663 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.663 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:15.663 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383352+0000", 2024-09-21T21:41:15.663 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4089999999999999e-06 2024-09-21T21:41:15.663 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.664 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.664 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:15.664 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383360+0000", 2024-09-21T21:41:15.664 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.2430000000000005e-06 2024-09-21T21:41:15.664 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.664 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.664 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:15.664 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383366+0000", 2024-09-21T21:41:15.664 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4909999999999996e-06 2024-09-21T21:41:15.664 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.664 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.664 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:15.664 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.394212+0000", 2024-09-21T21:41:15.664 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.010846000999999999 2024-09-21T21:41:15.665 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.665 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.665 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:15.665 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.394279+0000", 2024-09-21T21:41:15.665 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.7873000000000006e-05 2024-09-21T21:41:15.665 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.665 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.665 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:15.665 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.394352+0000", 2024-09-21T21:41:15.665 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.2904000000000005e-05 2024-09-21T21:41:15.665 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:15.665 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:15.665 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:15.665 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.665 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.666 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:1871 3.12 3:4bfa5158:::rbd_data.112145fae779.0000000000000002:a [read 10682368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:15.666 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.383497+0000", 2024-09-21T21:41:15.666 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 17.200243162, 2024-09-21T21:41:15.666 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.010980227, 2024-09-21T21:41:15.666 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:15.666 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:15.666 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:15.666 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:15.666 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:15.666 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:15.666 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1871 2024-09-21T21:41:15.666 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.666 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:15.666 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.667 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:15.667 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383497+0000", 2024-09-21T21:41:15.667 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:15.667 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.667 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.667 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:15.667 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383497+0000", 2024-09-21T21:41:15.667 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:15.667 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.667 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.667 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:15.667 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383498+0000", 2024-09-21T21:41:15.667 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2500000000000004e-07 2024-09-21T21:41:15.668 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.668 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.668 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:15.668 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383502+0000", 2024-09-21T21:41:15.668 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7049999999999998e-06 2024-09-21T21:41:15.668 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.668 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.668 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:15.668 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383512+0000", 2024-09-21T21:41:15.668 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.3239999999999992e-06 2024-09-21T21:41:15.668 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.668 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.668 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:15.668 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383518+0000", 2024-09-21T21:41:15.668 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5370000000000003e-06 2024-09-21T21:41:15.669 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.669 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.669 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:15.669 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.394378+0000", 2024-09-21T21:41:15.669 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.010859897 2024-09-21T21:41:15.669 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.669 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.669 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:15.669 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.394434+0000", 2024-09-21T21:41:15.669 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6198e-05 2024-09-21T21:41:15.669 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.670 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.670 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:15.670 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.394477+0000", 2024-09-21T21:41:15.670 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2641000000000001e-05 2024-09-21T21:41:15.670 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:15.670 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:15.670 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:15.670 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.670 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.670 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:1873 3.12 3:4bfa5158:::rbd_data.112145fae779.0000000000000002:a [read 10739712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:15.670 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.383639+0000", 2024-09-21T21:41:15.670 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 17.200101407999998, 2024-09-21T21:41:15.671 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.011039125, 2024-09-21T21:41:15.671 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:15.671 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:15.671 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:15.671 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:15.671 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:15.671 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:15.671 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1873 2024-09-21T21:41:15.671 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.671 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:15.671 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.671 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:15.671 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383639+0000", 2024-09-21T21:41:15.671 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:15.672 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.672 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.672 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:15.672 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383639+0000", 2024-09-21T21:41:15.672 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:15.672 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.672 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.672 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:15.672 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383639+0000", 2024-09-21T21:41:15.672 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.8000000000000004e-07 2024-09-21T21:41:15.672 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.672 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.672 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:15.672 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383644+0000", 2024-09-21T21:41:15.673 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3699999999999997e-06 2024-09-21T21:41:15.673 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.673 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.673 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:15.673 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383653+0000", 2024-09-21T21:41:15.673 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.3330000000000003e-06 2024-09-21T21:41:15.673 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.673 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.673 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:15.673 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.383659+0000", 2024-09-21T21:41:15.673 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7060000000000001e-06 2024-09-21T21:41:15.673 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.673 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.673 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:15.673 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.394518+0000", 2024-09-21T21:41:15.674 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.010859647 2024-09-21T21:41:15.674 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.674 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.674 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:15.674 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.394599+0000", 2024-09-21T21:41:15.674 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0770000000000001e-05 2024-09-21T21:41:15.674 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.674 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.674 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:15.674 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.394678+0000", 2024-09-21T21:41:15.674 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.8418999999999996e-05 2024-09-21T21:41:15.674 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:15.674 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:15.675 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:15.675 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.675 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.675 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4491 3.12 3:4bfa5158:::rbd_data.112145fae779.0000000000000002:a [read 2269184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:15.675 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.513914+0000", 2024-09-21T21:41:15.675 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 17.069825756, 2024-09-21T21:41:15.675 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0098656990000000003, 2024-09-21T21:41:15.675 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:15.675 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:15.675 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:15.675 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:15.675 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:15.675 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:15.675 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4491 2024-09-21T21:41:15.676 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.676 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:15.676 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.676 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:15.676 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.513914+0000", 2024-09-21T21:41:15.676 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:15.676 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.676 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.676 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:15.676 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.513914+0000", 2024-09-21T21:41:15.676 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:15.676 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.676 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.676 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:15.677 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.513914+0000", 2024-09-21T21:41:15.677 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.11e-07 2024-09-21T21:41:15.677 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.677 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.677 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:15.677 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.513916+0000", 2024-09-21T21:41:15.677 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1799999999999999e-06 2024-09-21T21:41:15.677 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.677 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.677 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:15.677 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.513919+0000", 2024-09-21T21:41:15.677 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5099999999999999e-06 2024-09-21T21:41:15.677 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.677 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.678 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:15.678 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.513922+0000", 2024-09-21T21:41:15.678 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.429e-06 2024-09-21T21:41:15.678 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.678 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.678 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:15.678 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.521609+0000", 2024-09-21T21:41:15.678 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0076873979999999998 2024-09-21T21:41:15.678 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.678 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.678 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:15.678 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.521700+0000", 2024-09-21T21:41:15.678 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.0675999999999997e-05 2024-09-21T21:41:15.678 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:15.679 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:15.679 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:15.679 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.523780+0000", 2024-09-21T21:41:15.679 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0020802949999999998 2024-09-21T21:41:15.679 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:15.679 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:15.679 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:15.679 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:15.679 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:15.679 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:15.680 INFO:teuthology.orchestra.run.smithi081.stderr:nodeep-scrub is set 2024-09-21T21:41:15.681 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T21:41:15.682 INFO:teuthology.orchestra.run.smithi164.stdout:2199 read 0x4474800 thru 0x447a78c (0x5f8d bytes) 2024-09-21T21:41:15.682 INFO:teuthology.orchestra.run.smithi164.stdout:2201 trunc from 0x4ee7e00 to 0xd2cb200 2024-09-21T21:41:15.682 INFO:teuthology.orchestra.run.smithi164.stdout:2202 write 0xb74e400 thru 0xb752bfa (0x47fb bytes) 2024-09-21T21:41:15.682 INFO:teuthology.orchestra.run.smithi164.stdout:2203 write 0x4268e00 thru 0x426fc56 (0x6e57 bytes) 2024-09-21T21:41:15.682 INFO:teuthology.orchestra.run.smithi164.stdout:2207 read 0xce79200 thru 0xce7f2f0 (0x60f1 bytes) 2024-09-21T21:41:15.682 INFO:teuthology.orchestra.run.smithi164.stdout:2208 read 0x8b78800 thru 0x8b845c5 (0xbdc6 bytes) 2024-09-21T21:41:15.683 INFO:teuthology.orchestra.run.smithi164.stdout:2209 trunc from 0xd2cb200 to 0x7eb8000 2024-09-21T21:41:15.683 INFO:teuthology.orchestra.run.smithi164.stdout:2210 write 0x6cd7800 thru 0x6ce0c04 (0x9405 bytes) 2024-09-21T21:41:15.683 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:15.679+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:15.710 INFO:teuthology.orchestra.run.smithi164.stdout:2213 write 0xbb90800 thru 0xbb95f2d (0x572e bytes) 2024-09-21T21:41:15.710 INFO:teuthology.orchestra.run.smithi164.stdout:2214 write 0x6b57600 thru 0x6b63609 (0xc00a bytes) 2024-09-21T21:41:15.710 INFO:teuthology.orchestra.run.smithi164.stdout:2216 read 0xa4ce200 thru 0xa4da842 (0xc643 bytes) 2024-09-21T21:41:15.710 INFO:teuthology.orchestra.run.smithi164.stdout:2217 write 0xa9e2e00 thru 0xa9f0b50 (0xdd51 bytes) 2024-09-21T21:41:15.710 INFO:teuthology.orchestra.run.smithi164.stdout:2219 punch from 0x96adc00 to 0x96b1800, (0x3c00 bytes) 2024-09-21T21:41:15.711 INFO:teuthology.orchestra.run.smithi164.stdout:2220 write 0x19f7800 thru 0x1a04a8e (0xd28f bytes) 2024-09-21T21:41:15.716 INFO:teuthology.orchestra.run.smithi164.stdout:2221 trunc from 0xbb96000 to 0x73fd600 2024-09-21T21:41:15.755 INFO:teuthology.orchestra.run.smithi164.stdout:2222 read 0x4b4ae00 thru 0x4b58f89 (0xe18a bytes) 2024-09-21T21:41:15.756 INFO:teuthology.orchestra.run.smithi164.stdout:2223 read 0x2311600 thru 0x231435a (0x2d5b bytes) 2024-09-21T21:41:15.756 INFO:teuthology.orchestra.run.smithi164.stdout:2225 read 0x5766a00 thru 0x5775cb4 (0xf2b5 bytes) 2024-09-21T21:41:15.756 INFO:teuthology.orchestra.run.smithi164.stdout:2226 punch from 0x4484a00 to 0x448c000, (0x7600 bytes) 2024-09-21T21:41:15.781 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:15.780+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:15.786 INFO:teuthology.orchestra.run.smithi164.stdout:2229 write 0xab84200 thru 0xab89a5c (0x585d bytes) 2024-09-21T21:41:15.793 INFO:teuthology.orchestra.run.smithi164.stdout:2230 write 0x72e4400 thru 0x72ec66d (0x826e bytes) 2024-09-21T21:41:15.794 INFO:teuthology.orchestra.run.smithi164.stdout:2231 trunc from 0xab89c00 to 0xb3ea800 2024-09-21T21:41:15.815 INFO:teuthology.orchestra.run.smithi164.stdout:2232 write 0x739d400 thru 0x73a957c (0xc17d bytes) 2024-09-21T21:41:15.834 INFO:teuthology.orchestra.run.smithi164.stdout:2237 write 0xe295c00 thru 0xe29c5fa (0x69fb bytes) 2024-09-21T21:41:15.835 INFO:teuthology.orchestra.run.smithi164.stdout:2238 read 0xcd7e200 thru 0xcd8a3b1 (0xc1b2 bytes) 2024-09-21T21:41:15.835 INFO:teuthology.orchestra.run.smithi164.stdout:2239 punch from 0x32bec00 to 0x32c2400, (0x3800 bytes) 2024-09-21T21:41:15.837 INFO:teuthology.orchestra.run.smithi164.stdout:2240 write 0x1506a00 thru 0x150ed52 (0x8353 bytes) 2024-09-21T21:41:15.838 INFO:teuthology.orchestra.run.smithi164.stdout:2241 read 0x9212200 thru 0x921ce28 (0xac29 bytes) 2024-09-21T21:41:15.838 INFO:teuthology.orchestra.run.smithi164.stdout:2242 punch from 0x15c1a00 to 0x15c8a00, (0x7000 bytes) 2024-09-21T21:41:15.838 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:15.838 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:41:15.838 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 0 2024-09-21T21:41:15.838 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:15.847 INFO:teuthology.orchestra.run.smithi164.stdout:2246 trunc from 0xe29c600 to 0x7aeec00 2024-09-21T21:41:15.849 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T21:41:15.873 INFO:teuthology.orchestra.run.smithi164.stdout:2247 read 0xb82200 thru 0xb86f6b (0x4d6c bytes) 2024-09-21T21:41:15.877 INFO:teuthology.orchestra.run.smithi164.stdout:2248 trunc from 0x7aeec00 to 0xa387200 2024-09-21T21:41:15.885 INFO:teuthology.orchestra.run.smithi164.stdout:2249 write 0x999be00 thru 0x99a48ca (0x8acb bytes) 2024-09-21T21:41:15.887 INFO:teuthology.orchestra.run.smithi164.stdout:2250 read 0x5c50400 thru 0x5c515fc (0x11fd bytes) 2024-09-21T21:41:15.887 INFO:teuthology.orchestra.run.smithi164.stdout:2251 write 0x8727400 thru 0x872b430 (0x4031 bytes) 2024-09-21T21:41:15.891 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:15.889+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:15.892 INFO:teuthology.orchestra.run.smithi164.stdout:2252 trunc from 0xa387200 to 0x624b000 2024-09-21T21:41:15.936 INFO:teuthology.orchestra.run.smithi164.stdout:2253 trunc from 0x624b000 to 0x50f5a00 2024-09-21T21:41:15.945 INFO:teuthology.orchestra.run.smithi164.stdout:2254 write 0x3767a00 thru 0x376e299 (0x689a bytes) 2024-09-21T21:41:15.952 INFO:teuthology.orchestra.run.smithi164.stdout:2256 trunc from 0x50f5a00 to 0xa697e00 2024-09-21T21:41:15.983 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:15.982+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:15.992 INFO:teuthology.orchestra.run.smithi164.stdout:2257 trunc from 0xa697e00 to 0xbbb6600 2024-09-21T21:41:15.999 INFO:teuthology.orchestra.run.smithi164.stdout:2258 write 0x7eda000 thru 0x7eddef6 (0x3ef7 bytes) 2024-09-21T21:41:16.000 INFO:teuthology.orchestra.run.smithi164.stdout:2259 read 0x77b4000 thru 0x77bc2f8 (0x82f9 bytes) 2024-09-21T21:41:16.000 INFO:teuthology.orchestra.run.smithi164.stdout:2260 read 0x1915200 thru 0x1923102 (0xdf03 bytes) 2024-09-21T21:41:16.001 INFO:teuthology.orchestra.run.smithi164.stdout:2262 read 0x9a46a00 thru 0x9a5097b (0x9f7c bytes) 2024-09-21T21:41:16.001 INFO:teuthology.orchestra.run.smithi164.stdout:2267 read 0xb072000 thru 0xb080be2 (0xebe3 bytes) 2024-09-21T21:41:16.001 INFO:teuthology.orchestra.run.smithi164.stdout:2268 punch from 0x9c7cc00 to 0x9c88400, (0xb800 bytes) 2024-09-21T21:41:16.002 INFO:teuthology.orchestra.run.smithi164.stdout:2273 read 0x48f9200 thru 0x48fe0ed (0x4eee bytes) 2024-09-21T21:41:16.002 INFO:teuthology.orchestra.run.smithi164.stdout:2274 read 0xa005a00 thru 0xa005ae6 (0xe7 bytes) 2024-09-21T21:41:16.002 INFO:teuthology.orchestra.run.smithi164.stdout:2275 read 0x2a21000 thru 0x2a2bf38 (0xaf39 bytes) 2024-09-21T21:41:16.002 INFO:teuthology.orchestra.run.smithi164.stdout:2277 read 0x9b93400 thru 0x9b95e4d (0x2a4e bytes) 2024-09-21T21:41:16.002 INFO:teuthology.orchestra.run.smithi164.stdout:2279 write 0x4383000 thru 0x43859b8 (0x29b9 bytes) 2024-09-21T21:41:16.002 INFO:teuthology.orchestra.run.smithi164.stdout:2280 punch from 0x5e81200 to 0x5e81c00, (0xa00 bytes) 2024-09-21T21:41:16.005 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:16.005 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:41:16.005 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:41:16.005 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:41:16.005 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:16.015 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T21:41:16.026 INFO:teuthology.orchestra.run.smithi164.stdout:2281 write 0xd9d3a00 thru 0xd9d67fe (0x2dff bytes) 2024-09-21T21:41:16.027 INFO:teuthology.orchestra.run.smithi164.stdout:2283 write 0x3065000 thru 0x306d5e4 (0x85e5 bytes) 2024-09-21T21:41:16.027 INFO:teuthology.orchestra.run.smithi164.stdout:2284 punch from 0x8e3e400 to 0x8e43a00, (0x5600 bytes) 2024-09-21T21:41:16.029 INFO:teuthology.orchestra.run.smithi164.stdout:2285 write 0x2184c00 thru 0x2185323 (0x724 bytes) 2024-09-21T21:41:16.033 INFO:teuthology.orchestra.run.smithi164.stdout:2287 trunc from 0xd9d6800 to 0x9fe7800 2024-09-21T21:41:16.067 INFO:teuthology.orchestra.run.smithi164.stdout:2288 punch from 0x82b8e00 to 0x82c2800, (0x9a00 bytes) 2024-09-21T21:41:16.067 INFO:teuthology.orchestra.run.smithi164.stdout:2290 read 0x1e68800 thru 0x1e75b6b (0xd36c bytes) 2024-09-21T21:41:16.068 INFO:teuthology.orchestra.run.smithi164.stdout:2292 write 0x58e0a00 thru 0x58e781b (0x6e1c bytes) 2024-09-21T21:41:16.068 INFO:teuthology.orchestra.run.smithi164.stdout:2294 write 0x8c6e600 thru 0x8c767ad (0x81ae bytes) 2024-09-21T21:41:16.069 INFO:teuthology.orchestra.run.smithi164.stdout:2295 trunc from 0x9fe7800 to 0xaf2aa00 2024-09-21T21:41:16.084 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:16.082+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:16.106 INFO:teuthology.orchestra.run.smithi164.stdout:2297 trunc from 0xaf2aa00 to 0x6832a00 2024-09-21T21:41:16.131 INFO:teuthology.orchestra.run.smithi164.stdout:2300 write 0x188fe00 thru 0x1898c60 (0x8e61 bytes) 2024-09-21T21:41:16.132 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:16.132 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 20, 2024-09-21T21:41:16.132 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 600, 2024-09-21T21:41:16.132 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:41:16.132 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.132 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:2885 3.b 3:d49b370f:::rbd_object_map.1121e540c096:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:16.132 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.478228+0000", 2024-09-21T21:41:16.132 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 17.6512484, 2024-09-21T21:41:16.132 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.015327835999999999, 2024-09-21T21:41:16.133 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:16.133 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:16.133 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:41:16.133 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:16.133 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:16.133 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:16.133 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2885 2024-09-21T21:41:16.133 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.133 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:16.133 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.133 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:16.133 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478228+0000", 2024-09-21T21:41:16.133 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:16.133 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.134 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.134 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:16.134 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478228+0000", 2024-09-21T21:41:16.134 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:16.134 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.134 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.134 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:16.134 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478229+0000", 2024-09-21T21:41:16.134 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8000000000000001e-07 2024-09-21T21:41:16.134 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.134 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.134 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:16.134 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478231+0000", 2024-09-21T21:41:16.135 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.5069999999999999e-06 2024-09-21T21:41:16.135 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.135 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.135 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:16.135 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478238+0000", 2024-09-21T21:41:16.135 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.8820000000000003e-06 2024-09-21T21:41:16.135 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.135 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.135 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:16.135 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478243+0000", 2024-09-21T21:41:16.135 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2209999999999996e-06 2024-09-21T21:41:16.135 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.135 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.135 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:16.136 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.488160+0000", 2024-09-21T21:41:16.136 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0099169849999999997 2024-09-21T21:41:16.136 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.136 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.136 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:16.136 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.488231+0000", 2024-09-21T21:41:16.136 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0909999999999997e-05 2024-09-21T21:41:16.136 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.136 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.136 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 0", 2024-09-21T21:41:16.136 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.488538+0000", 2024-09-21T21:41:16.136 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000306736 2024-09-21T21:41:16.136 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.136 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.136 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:41:16.137 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.493210+0000", 2024-09-21T21:41:16.137 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0046722120000000002 2024-09-21T21:41:16.137 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.137 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.137 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:41:16.137 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.493471+0000", 2024-09-21T21:41:16.137 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00026132899999999998 2024-09-21T21:41:16.137 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.137 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.137 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:41:16.137 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.493517+0000", 2024-09-21T21:41:16.137 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5309000000000003e-05 2024-09-21T21:41:16.137 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.137 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.137 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:16.137 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.493556+0000", 2024-09-21T21:41:16.138 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.9365e-05 2024-09-21T21:41:16.138 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:16.138 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:16.138 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:16.138 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.138 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.138 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:2886 3.b 3:d49b370f:::rbd_object_map.1121e540c096:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:16.138 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.478249+0000", 2024-09-21T21:41:16.138 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 17.65122762, 2024-09-21T21:41:16.138 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.015426120999999999, 2024-09-21T21:41:16.138 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:16.138 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:16.138 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:41:16.138 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:16.139 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:16.139 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:16.139 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2886 2024-09-21T21:41:16.139 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.139 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:16.139 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.139 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:16.139 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478249+0000", 2024-09-21T21:41:16.139 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:16.139 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.139 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.139 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:16.139 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478249+0000", 2024-09-21T21:41:16.140 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:16.140 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.140 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.140 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:16.140 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478249+0000", 2024-09-21T21:41:16.140 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.9999999999999998e-07 2024-09-21T21:41:16.140 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.140 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.140 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:16.140 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478252+0000", 2024-09-21T21:41:16.140 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.5500000000000001e-06 2024-09-21T21:41:16.140 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.140 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.140 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:16.140 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478258+0000", 2024-09-21T21:41:16.141 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7080000000000002e-06 2024-09-21T21:41:16.141 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.141 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.141 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:16.141 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478265+0000", 2024-09-21T21:41:16.141 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9110000000000004e-06 2024-09-21T21:41:16.141 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.141 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.141 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:16.141 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.488889+0000", 2024-09-21T21:41:16.141 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.010624662 2024-09-21T21:41:16.141 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.141 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.141 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:16.141 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.488965+0000", 2024-09-21T21:41:16.142 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5949000000000001e-05 2024-09-21T21:41:16.142 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.142 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.142 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 0", 2024-09-21T21:41:16.142 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.489235+0000", 2024-09-21T21:41:16.142 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00027019499999999998 2024-09-21T21:41:16.142 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.142 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.142 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:41:16.142 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.493233+0000", 2024-09-21T21:41:16.142 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0039978799999999997 2024-09-21T21:41:16.142 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.142 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.142 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:41:16.143 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.493601+0000", 2024-09-21T21:41:16.143 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00036728200000000003 2024-09-21T21:41:16.143 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.143 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.143 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:41:16.143 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.493637+0000", 2024-09-21T21:41:16.143 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6263000000000002e-05 2024-09-21T21:41:16.143 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.143 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.143 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:16.143 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.493675+0000", 2024-09-21T21:41:16.143 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8321000000000001e-05 2024-09-21T21:41:16.143 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:16.143 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:16.143 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:16.144 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.144 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.144 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:2887 3.b 3:d49b370f:::rbd_object_map.1121e540c096:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:16.144 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.478271+0000", 2024-09-21T21:41:16.144 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 17.651206108, 2024-09-21T21:41:16.144 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.015551966, 2024-09-21T21:41:16.144 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:16.144 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:16.144 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:41:16.144 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:16.144 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:16.144 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:16.144 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2887 2024-09-21T21:41:16.144 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.144 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:16.145 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.145 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:16.145 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478271+0000", 2024-09-21T21:41:16.145 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:16.145 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.145 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.145 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:16.145 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478271+0000", 2024-09-21T21:41:16.145 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:16.145 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.145 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.145 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:16.145 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478271+0000", 2024-09-21T21:41:16.145 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.0600000000000001e-07 2024-09-21T21:41:16.145 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.146 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.146 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:16.146 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478274+0000", 2024-09-21T21:41:16.146 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.6390000000000001e-06 2024-09-21T21:41:16.146 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.146 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.146 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:16.146 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478280+0000", 2024-09-21T21:41:16.146 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3219999999999998e-06 2024-09-21T21:41:16.146 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.146 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.146 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:16.146 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478285+0000", 2024-09-21T21:41:16.146 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6589999999999998e-06 2024-09-21T21:41:16.147 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.147 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.147 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:16.147 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.489524+0000", 2024-09-21T21:41:16.147 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.01123979 2024-09-21T21:41:16.147 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.147 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.147 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:16.147 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.489567+0000", 2024-09-21T21:41:16.147 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2673000000000003e-05 2024-09-21T21:41:16.147 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.147 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.147 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 0", 2024-09-21T21:41:16.147 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.489834+0000", 2024-09-21T21:41:16.148 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00026672799999999998 2024-09-21T21:41:16.148 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.148 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.148 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:41:16.148 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.493290+0000", 2024-09-21T21:41:16.148 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0034557429999999998 2024-09-21T21:41:16.148 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.148 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.148 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:41:16.148 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.493737+0000", 2024-09-21T21:41:16.148 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00044731799999999999 2024-09-21T21:41:16.148 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.148 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.148 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:41:16.149 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.493784+0000", 2024-09-21T21:41:16.149 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7178999999999997e-05 2024-09-21T21:41:16.149 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.149 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.149 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:16.149 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.493823+0000", 2024-09-21T21:41:16.149 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8509e-05 2024-09-21T21:41:16.149 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:16.149 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:16.149 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:16.149 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.149 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.149 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4325 3.0 3:074cff57:::rbd_data.112145fae779.000000000000000c:a [read 2064384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:16.149 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.510141+0000", 2024-09-21T21:41:16.149 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 17.619335946, 2024-09-21T21:41:16.150 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013330887, 2024-09-21T21:41:16.150 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:16.150 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:16.150 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:16.150 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:16.150 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:16.150 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:16.150 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4325 2024-09-21T21:41:16.150 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.150 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:16.150 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.150 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:16.150 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510141+0000", 2024-09-21T21:41:16.150 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:16.151 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.151 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.151 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:16.151 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510141+0000", 2024-09-21T21:41:16.151 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:16.151 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.151 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.151 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:16.151 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510141+0000", 2024-09-21T21:41:16.151 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.72e-07 2024-09-21T21:41:16.151 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.151 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.151 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:16.151 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510143+0000", 2024-09-21T21:41:16.151 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2409999999999998e-06 2024-09-21T21:41:16.152 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.152 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.152 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:16.152 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510151+0000", 2024-09-21T21:41:16.152 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.3390000000000004e-06 2024-09-21T21:41:16.152 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.152 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.152 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:16.152 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510155+0000", 2024-09-21T21:41:16.152 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5630000000000002e-06 2024-09-21T21:41:16.152 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.152 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.152 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:16.152 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.523310+0000", 2024-09-21T21:41:16.152 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013154658 2024-09-21T21:41:16.152 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.153 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.153 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:16.153 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.523386+0000", 2024-09-21T21:41:16.153 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.6553999999999999e-05 2024-09-21T21:41:16.153 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.153 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.153 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:16.153 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.523472+0000", 2024-09-21T21:41:16.153 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.5160000000000005e-05 2024-09-21T21:41:16.153 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:16.153 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:16.153 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:16.153 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.153 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.154 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4326 3.0 3:074cff57:::rbd_data.112145fae779.000000000000000c:a [read 2097152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:16.154 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.510162+0000", 2024-09-21T21:41:16.154 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 17.619314931000002, 2024-09-21T21:41:16.154 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013537466, 2024-09-21T21:41:16.154 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:16.154 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:16.154 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:16.154 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:16.154 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:16.154 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:16.154 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4326 2024-09-21T21:41:16.154 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.154 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:16.154 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.155 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:16.155 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510162+0000", 2024-09-21T21:41:16.155 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:16.155 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.155 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.155 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:16.155 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510162+0000", 2024-09-21T21:41:16.155 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:16.155 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.155 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.155 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:16.155 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510162+0000", 2024-09-21T21:41:16.155 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.0699999999999998e-07 2024-09-21T21:41:16.155 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.155 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.156 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:16.156 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510164+0000", 2024-09-21T21:41:16.156 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.17e-06 2024-09-21T21:41:16.156 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.156 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.156 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:16.156 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510174+0000", 2024-09-21T21:41:16.156 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2439999999999999e-06 2024-09-21T21:41:16.156 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.156 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.156 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:16.156 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510179+0000", 2024-09-21T21:41:16.156 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2159999999999997e-06 2024-09-21T21:41:16.156 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.156 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.157 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:16.157 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.523502+0000", 2024-09-21T21:41:16.157 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013322779 2024-09-21T21:41:16.157 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.157 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.157 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:16.157 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.523579+0000", 2024-09-21T21:41:16.157 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7402000000000002e-05 2024-09-21T21:41:16.157 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.157 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.157 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:16.157 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.523699+0000", 2024-09-21T21:41:16.157 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000120248 2024-09-21T21:41:16.157 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:16.157 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:16.158 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:16.158 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.158 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.158 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4327 3.0 3:074cff57:::rbd_data.112145fae779.000000000000000c:a [read 2154496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:16.158 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.510418+0000", 2024-09-21T21:41:16.158 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 17.619058496000001, 2024-09-21T21:41:16.158 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013505039999999999, 2024-09-21T21:41:16.158 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:16.158 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:16.158 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:16.158 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:16.158 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:16.158 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:16.158 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4327 2024-09-21T21:41:16.159 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.159 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:16.159 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.159 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:16.159 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510418+0000", 2024-09-21T21:41:16.159 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:16.159 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.159 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.159 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:16.159 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510418+0000", 2024-09-21T21:41:16.159 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:16.159 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.159 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.160 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:16.160 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510419+0000", 2024-09-21T21:41:16.160 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8999999999999997e-07 2024-09-21T21:41:16.160 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.160 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.160 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:16.160 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510421+0000", 2024-09-21T21:41:16.160 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.4779999999999998e-06 2024-09-21T21:41:16.160 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.160 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.160 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:16.160 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510427+0000", 2024-09-21T21:41:16.160 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5609999999999998e-06 2024-09-21T21:41:16.160 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.160 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.161 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:16.161 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510432+0000", 2024-09-21T21:41:16.161 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5389999999999998e-06 2024-09-21T21:41:16.161 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.161 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.161 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:16.161 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.523743+0000", 2024-09-21T21:41:16.161 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013310963 2024-09-21T21:41:16.161 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.161 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.161 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:16.161 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.523824+0000", 2024-09-21T21:41:16.161 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0463000000000005e-05 2024-09-21T21:41:16.161 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.162 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.162 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:16.162 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.523923+0000", 2024-09-21T21:41:16.162 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.9545999999999999e-05 2024-09-21T21:41:16.162 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:16.162 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:16.162 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:16.162 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.162 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.162 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4328 3.0 3:074cff57:::rbd_data.112145fae779.000000000000000c:a [read 2211840~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:16.162 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.510443+0000", 2024-09-21T21:41:16.162 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 17.619034196000001, 2024-09-21T21:41:16.162 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013684564, 2024-09-21T21:41:16.162 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:16.163 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:16.163 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:16.163 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:16.163 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:16.163 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:16.163 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4328 2024-09-21T21:41:16.163 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.163 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:16.163 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.163 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:16.163 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510443+0000", 2024-09-21T21:41:16.163 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:16.163 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.163 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.163 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:16.164 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510443+0000", 2024-09-21T21:41:16.164 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:16.164 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.164 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.164 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:16.164 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510443+0000", 2024-09-21T21:41:16.164 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2199999999999999e-07 2024-09-21T21:41:16.164 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.164 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.164 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:16.164 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510445+0000", 2024-09-21T21:41:16.164 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1679999999999998e-06 2024-09-21T21:41:16.164 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.165 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.165 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:16.165 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510452+0000", 2024-09-21T21:41:16.165 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6259999999999997e-06 2024-09-21T21:41:16.165 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.165 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.165 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:16.165 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510457+0000", 2024-09-21T21:41:16.165 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8110000000000002e-06 2024-09-21T21:41:16.165 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.165 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.165 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:16.165 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.523968+0000", 2024-09-21T21:41:16.165 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013510996000000001 2024-09-21T21:41:16.165 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.166 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.166 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:16.166 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.524044+0000", 2024-09-21T21:41:16.166 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.6259000000000003e-05 2024-09-21T21:41:16.166 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.166 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.166 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:16.166 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.524127+0000", 2024-09-21T21:41:16.166 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.3282000000000007e-05 2024-09-21T21:41:16.166 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:16.166 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:16.166 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:16.166 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.166 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.166 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4577 3.0 3:074cff57:::rbd_data.112145fae779.000000000000000c:a [read 1744896~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:16.167 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.514327+0000", 2024-09-21T21:41:16.167 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 17.615149539000001, 2024-09-21T21:41:16.167 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013451424999999999, 2024-09-21T21:41:16.167 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:16.167 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:16.167 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:16.167 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:16.167 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:16.167 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:16.167 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4577 2024-09-21T21:41:16.167 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.167 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:16.167 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.167 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:16.168 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514327+0000", 2024-09-21T21:41:16.168 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:16.168 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.168 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.168 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:16.168 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514327+0000", 2024-09-21T21:41:16.168 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:16.168 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.168 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.168 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:16.168 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514328+0000", 2024-09-21T21:41:16.168 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4900000000000001e-07 2024-09-21T21:41:16.168 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.168 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.168 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:16.168 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514330+0000", 2024-09-21T21:41:16.169 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.227e-06 2024-09-21T21:41:16.169 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.169 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.169 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:16.169 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514336+0000", 2024-09-21T21:41:16.169 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1940000000000003e-06 2024-09-21T21:41:16.169 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.169 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.169 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:16.169 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514341+0000", 2024-09-21T21:41:16.169 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7319999999999998e-06 2024-09-21T21:41:16.169 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.169 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.169 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:16.170 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.527415+0000", 2024-09-21T21:41:16.170 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013074657 2024-09-21T21:41:16.170 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.170 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.170 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:16.170 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.527491+0000", 2024-09-21T21:41:16.170 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5899e-05 2024-09-21T21:41:16.170 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.170 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.170 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:16.170 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.527779+0000", 2024-09-21T21:41:16.170 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00028736700000000002 2024-09-21T21:41:16.170 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:16.170 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:16.170 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:16.171 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.171 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.171 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4578 3.0 3:074cff57:::rbd_data.112145fae779.000000000000000c:a [read 1802240~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:16.171 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.514347+0000", 2024-09-21T21:41:16.171 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 17.615130084, 2024-09-21T21:41:16.171 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.014025592999999999, 2024-09-21T21:41:16.171 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:16.171 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:16.171 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:16.171 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:16.171 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:16.171 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:16.171 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4578 2024-09-21T21:41:16.172 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.172 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:16.172 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.172 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:16.172 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514347+0000", 2024-09-21T21:41:16.172 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:16.172 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.172 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.172 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:16.172 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514347+0000", 2024-09-21T21:41:16.172 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:16.172 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.172 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.172 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:16.173 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514347+0000", 2024-09-21T21:41:16.173 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6699999999999999e-07 2024-09-21T21:41:16.173 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.173 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.173 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:16.173 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514349+0000", 2024-09-21T21:41:16.173 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0109999999999999e-06 2024-09-21T21:41:16.173 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.173 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.173 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:16.173 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514355+0000", 2024-09-21T21:41:16.173 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1090000000000003e-06 2024-09-21T21:41:16.173 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.173 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.173 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:16.173 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514360+0000", 2024-09-21T21:41:16.174 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5900000000000001e-06 2024-09-21T21:41:16.174 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.174 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.174 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:16.174 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.527862+0000", 2024-09-21T21:41:16.174 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013501812 2024-09-21T21:41:16.174 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.174 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.174 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:16.174 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.527933+0000", 2024-09-21T21:41:16.174 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.1322000000000006e-05 2024-09-21T21:41:16.174 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.174 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.174 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:16.175 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.528372+0000", 2024-09-21T21:41:16.175 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00043938199999999999 2024-09-21T21:41:16.175 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:16.175 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:16.175 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:16.175 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.175 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.175 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4579 3.0 3:074cff57:::rbd_data.112145fae779.000000000000000c:a [read 1835008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:16.175 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.514365+0000", 2024-09-21T21:41:16.175 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 17.615111227, 2024-09-21T21:41:16.175 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.014158828, 2024-09-21T21:41:16.175 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:16.175 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:16.175 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:16.176 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:16.176 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:16.176 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:16.176 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4579 2024-09-21T21:41:16.176 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.176 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:16.176 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.176 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:16.176 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514365+0000", 2024-09-21T21:41:16.176 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:16.176 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.176 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.176 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:16.176 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514365+0000", 2024-09-21T21:41:16.177 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:16.177 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.177 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.177 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:16.177 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514366+0000", 2024-09-21T21:41:16.177 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7800000000000002e-07 2024-09-21T21:41:16.177 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.177 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.177 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:16.177 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514368+0000", 2024-09-21T21:41:16.177 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9920000000000002e-06 2024-09-21T21:41:16.177 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.177 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.177 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:16.177 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514374+0000", 2024-09-21T21:41:16.178 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9370000000000001e-06 2024-09-21T21:41:16.178 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.178 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.178 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:16.178 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514379+0000", 2024-09-21T21:41:16.178 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7149999999999997e-06 2024-09-21T21:41:16.178 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.178 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.178 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:16.178 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.528303+0000", 2024-09-21T21:41:16.178 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013924076000000001 2024-09-21T21:41:16.178 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.178 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.178 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:16.179 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.528429+0000", 2024-09-21T21:41:16.179 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00012643100000000001 2024-09-21T21:41:16.179 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.179 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.179 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:16.179 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.528524+0000", 2024-09-21T21:41:16.179 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.5298999999999997e-05 2024-09-21T21:41:16.179 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:16.179 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:16.179 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:16.179 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.179 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.179 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4580 3.0 3:074cff57:::rbd_data.112145fae779.000000000000000c:a [read 1892352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:16.179 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.514384+0000", 2024-09-21T21:41:16.179 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 17.615092631, 2024-09-21T21:41:16.180 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.014365243, 2024-09-21T21:41:16.180 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:16.180 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:16.180 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:16.180 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:16.180 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:16.180 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:16.180 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4580 2024-09-21T21:41:16.180 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.180 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:16.180 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.180 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:16.180 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514384+0000", 2024-09-21T21:41:16.180 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:16.181 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.181 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.181 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:16.181 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514384+0000", 2024-09-21T21:41:16.181 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:16.181 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.181 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.181 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:16.181 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514384+0000", 2024-09-21T21:41:16.181 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1100000000000001e-07 2024-09-21T21:41:16.181 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.181 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.181 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:16.181 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514387+0000", 2024-09-21T21:41:16.181 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1150000000000001e-06 2024-09-21T21:41:16.182 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.182 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.182 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:16.182 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514393+0000", 2024-09-21T21:41:16.182 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1140000000000002e-06 2024-09-21T21:41:16.182 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.182 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.182 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:16.182 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514398+0000", 2024-09-21T21:41:16.182 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6330000000000001e-06 2024-09-21T21:41:16.182 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.182 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.182 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:16.182 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.528569+0000", 2024-09-21T21:41:16.183 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.014171011000000001 2024-09-21T21:41:16.183 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.183 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.183 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:16.183 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.528642+0000", 2024-09-21T21:41:16.183 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.2336000000000007e-05 2024-09-21T21:41:16.183 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.183 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.183 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:16.183 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.528749+0000", 2024-09-21T21:41:16.183 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000107623 2024-09-21T21:41:16.183 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:16.183 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:16.183 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:16.184 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.184 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.184 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4581 3.0 3:074cff57:::rbd_data.112145fae779.000000000000000c:a [read 1949696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:16.184 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.514404+0000", 2024-09-21T21:41:16.184 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 17.615072464000001, 2024-09-21T21:41:16.184 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.014551563999999999, 2024-09-21T21:41:16.184 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:16.184 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:16.184 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:16.184 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:16.184 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:16.184 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:16.184 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4581 2024-09-21T21:41:16.184 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.184 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:16.185 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.185 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:16.185 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514404+0000", 2024-09-21T21:41:16.185 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:16.185 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.185 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.185 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:16.185 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514404+0000", 2024-09-21T21:41:16.185 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:16.185 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.185 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.185 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:16.185 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514405+0000", 2024-09-21T21:41:16.185 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7599999999999998e-07 2024-09-21T21:41:16.185 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.185 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.186 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:16.186 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514407+0000", 2024-09-21T21:41:16.186 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0169999999999999e-06 2024-09-21T21:41:16.186 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.186 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.186 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:16.186 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514413+0000", 2024-09-21T21:41:16.186 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0020000000000003e-06 2024-09-21T21:41:16.186 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.186 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.186 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:16.186 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514417+0000", 2024-09-21T21:41:16.186 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7319999999999998e-06 2024-09-21T21:41:16.186 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.186 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.187 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:16.187 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.528781+0000", 2024-09-21T21:41:16.187 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.014363433 2024-09-21T21:41:16.187 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.187 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.187 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:16.187 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.528861+0000", 2024-09-21T21:41:16.187 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0408000000000007e-05 2024-09-21T21:41:16.187 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.187 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.187 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:16.187 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.528956+0000", 2024-09-21T21:41:16.187 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.4596e-05 2024-09-21T21:41:16.187 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:16.188 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:16.188 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:16.188 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.188 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.188 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4582 3.0 3:074cff57:::rbd_data.112145fae779.000000000000000c:a [read 1982464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:16.188 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.514423+0000", 2024-09-21T21:41:16.188 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 17.615053580000001, 2024-09-21T21:41:16.188 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.014719965, 2024-09-21T21:41:16.188 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:16.188 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:16.188 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:16.188 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:16.188 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:16.188 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:16.189 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4582 2024-09-21T21:41:16.189 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.189 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:16.189 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.189 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:16.189 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514423+0000", 2024-09-21T21:41:16.189 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:16.189 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.189 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.189 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:16.189 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514423+0000", 2024-09-21T21:41:16.189 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:16.189 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.189 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.190 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:16.190 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514423+0000", 2024-09-21T21:41:16.190 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.53e-07 2024-09-21T21:41:16.190 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.190 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.190 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:16.190 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514426+0000", 2024-09-21T21:41:16.190 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2000000000000001e-06 2024-09-21T21:41:16.190 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.190 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.190 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:16.190 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514433+0000", 2024-09-21T21:41:16.190 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.3170000000000004e-06 2024-09-21T21:41:16.190 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.191 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.191 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:16.191 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514438+0000", 2024-09-21T21:41:16.191 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3730000000000001e-06 2024-09-21T21:41:16.191 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.191 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.191 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:16.191 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.528984+0000", 2024-09-21T21:41:16.191 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.014545773 2024-09-21T21:41:16.191 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.191 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.191 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:16.191 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.529056+0000", 2024-09-21T21:41:16.191 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.2067e-05 2024-09-21T21:41:16.192 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.192 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.192 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:16.192 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.529143+0000", 2024-09-21T21:41:16.192 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.6881999999999999e-05 2024-09-21T21:41:16.192 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:16.192 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:16.192 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:16.192 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.192 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.192 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4583 3.0 3:074cff57:::rbd_data.112145fae779.000000000000000c:a [read 2039808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:16.192 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.514446+0000", 2024-09-21T21:41:16.192 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 17.615030808, 2024-09-21T21:41:16.192 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.016072466000000001, 2024-09-21T21:41:16.193 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:16.193 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:16.193 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:16.193 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:16.193 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:16.193 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:16.193 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4583 2024-09-21T21:41:16.193 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.193 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:16.193 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.193 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:16.193 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514446+0000", 2024-09-21T21:41:16.193 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:16.193 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.193 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.194 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:16.194 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514446+0000", 2024-09-21T21:41:16.194 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:16.194 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.194 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.194 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:16.194 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514446+0000", 2024-09-21T21:41:16.194 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.9700000000000002e-07 2024-09-21T21:41:16.194 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.194 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.194 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:16.194 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514448+0000", 2024-09-21T21:41:16.194 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1849999999999998e-06 2024-09-21T21:41:16.194 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.195 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.195 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:16.195 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514454+0000", 2024-09-21T21:41:16.195 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.015e-06 2024-09-21T21:41:16.195 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.195 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.195 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:16.195 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514459+0000", 2024-09-21T21:41:16.195 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7639999999999996e-06 2024-09-21T21:41:16.195 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.195 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.195 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:16.195 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.529173+0000", 2024-09-21T21:41:16.195 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.014713688000000001 2024-09-21T21:41:16.196 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.196 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.196 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:16.196 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.529248+0000", 2024-09-21T21:41:16.196 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5073000000000004e-05 2024-09-21T21:41:16.196 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.196 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.196 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:16.196 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.530518+0000", 2024-09-21T21:41:16.196 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0012703440000000001 2024-09-21T21:41:16.196 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:16.196 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:16.196 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:16.196 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.197 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.197 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4584 3.0 3:074cff57:::rbd_data.112145fae779.000000000000000c:a [read 2129920~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:16.197 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.514465+0000", 2024-09-21T21:41:16.197 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 17.615011369000001, 2024-09-21T21:41:16.197 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.016252026999999999, 2024-09-21T21:41:16.197 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:16.197 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:16.197 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:16.197 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:16.197 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:16.197 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:16.197 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4584 2024-09-21T21:41:16.197 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.197 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:16.197 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.198 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:16.198 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514465+0000", 2024-09-21T21:41:16.198 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:16.198 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.198 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.198 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:16.198 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514465+0000", 2024-09-21T21:41:16.198 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:16.198 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.198 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.198 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:16.198 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514466+0000", 2024-09-21T21:41:16.198 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.0400000000000002e-07 2024-09-21T21:41:16.198 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.199 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.199 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:16.199 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514468+0000", 2024-09-21T21:41:16.199 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3159999999999999e-06 2024-09-21T21:41:16.199 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.199 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.199 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:16.199 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514474+0000", 2024-09-21T21:41:16.199 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.224e-06 2024-09-21T21:41:16.199 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.199 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.199 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:16.199 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514479+0000", 2024-09-21T21:41:16.199 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8099999999999997e-06 2024-09-21T21:41:16.200 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.200 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.200 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:16.200 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.530552+0000", 2024-09-21T21:41:16.200 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.016072622000000002 2024-09-21T21:41:16.200 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.200 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.200 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:16.200 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.530630+0000", 2024-09-21T21:41:16.200 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.8448000000000005e-05 2024-09-21T21:41:16.200 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.200 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.200 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:16.200 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.530717+0000", 2024-09-21T21:41:16.200 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.7202999999999998e-05 2024-09-21T21:41:16.201 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:16.201 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:16.201 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:16.201 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.201 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.201 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4585 3.0 3:074cff57:::rbd_data.112145fae779.000000000000000c:a [read 2187264~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:16.201 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.514485+0000", 2024-09-21T21:41:16.201 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 17.614991702000001, 2024-09-21T21:41:16.201 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.016420605000000001, 2024-09-21T21:41:16.201 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:16.201 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:16.201 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:16.201 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:16.201 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:16.202 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:16.202 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4585 2024-09-21T21:41:16.202 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.202 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:16.202 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.202 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:16.202 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514485+0000", 2024-09-21T21:41:16.202 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:16.202 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.202 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.202 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:16.202 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514485+0000", 2024-09-21T21:41:16.202 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:16.202 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.203 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.203 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:16.203 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514485+0000", 2024-09-21T21:41:16.203 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3599999999999999e-07 2024-09-21T21:41:16.203 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.203 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.203 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:16.203 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514487+0000", 2024-09-21T21:41:16.203 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0870000000000001e-06 2024-09-21T21:41:16.203 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.203 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.203 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:16.203 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514495+0000", 2024-09-21T21:41:16.203 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7589999999999997e-06 2024-09-21T21:41:16.204 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.204 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.204 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:16.204 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514501+0000", 2024-09-21T21:41:16.204 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4e-06 2024-09-21T21:41:16.204 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.204 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.204 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:16.204 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.530747+0000", 2024-09-21T21:41:16.204 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.016246564000000002 2024-09-21T21:41:16.204 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.204 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.204 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:16.204 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.530824+0000", 2024-09-21T21:41:16.205 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7109999999999999e-05 2024-09-21T21:41:16.205 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.205 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.205 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:16.205 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.530906+0000", 2024-09-21T21:41:16.205 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.1348999999999997e-05 2024-09-21T21:41:16.205 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:16.205 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:16.205 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:16.205 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.205 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.205 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4711 3.0 3:074cff57:::rbd_data.112145fae779.000000000000000c:a [read 2043904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:16.205 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.519604+0000", 2024-09-21T21:41:16.205 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 17.609872212999999, 2024-09-21T21:41:16.206 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013270252999999999, 2024-09-21T21:41:16.206 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:16.206 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:16.206 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:16.206 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:16.206 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:16.206 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:16.206 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4711 2024-09-21T21:41:16.206 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.206 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:16.207 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.207 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:16.207 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519604+0000", 2024-09-21T21:41:16.207 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:16.207 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.207 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.207 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:16.207 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519604+0000", 2024-09-21T21:41:16.207 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:16.207 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.207 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.207 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:16.207 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519605+0000", 2024-09-21T21:41:16.207 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8500000000000002e-07 2024-09-21T21:41:16.208 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.208 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.208 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:16.208 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519607+0000", 2024-09-21T21:41:16.208 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1789999999999998e-06 2024-09-21T21:41:16.208 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.208 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.208 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:16.208 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519613+0000", 2024-09-21T21:41:16.208 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2609999999999999e-06 2024-09-21T21:41:16.208 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.208 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.208 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:16.208 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519618+0000", 2024-09-21T21:41:16.209 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9579999999999998e-06 2024-09-21T21:41:16.209 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.209 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.209 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:16.209 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.532678+0000", 2024-09-21T21:41:16.209 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013059822 2024-09-21T21:41:16.209 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.209 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.209 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:16.209 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.532756+0000", 2024-09-21T21:41:16.209 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7467999999999998e-05 2024-09-21T21:41:16.209 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.209 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.209 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:16.209 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.532875+0000", 2024-09-21T21:41:16.210 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011917999999999999 2024-09-21T21:41:16.210 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:16.210 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:16.210 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:16.210 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.210 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.210 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4712 3.0 3:074cff57:::rbd_data.112145fae779.000000000000000c:a [read 2134016~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:16.210 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.519632+0000", 2024-09-21T21:41:16.210 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 17.609844729999999, 2024-09-21T21:41:16.210 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013480002, 2024-09-21T21:41:16.210 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:16.210 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:16.210 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:16.210 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:16.210 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:16.211 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:16.211 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4712 2024-09-21T21:41:16.211 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.211 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:16.211 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.211 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:16.211 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519632+0000", 2024-09-21T21:41:16.211 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:16.211 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.211 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.211 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:16.211 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519632+0000", 2024-09-21T21:41:16.211 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:16.211 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.212 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.212 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:16.212 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519632+0000", 2024-09-21T21:41:16.212 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2899999999999999e-07 2024-09-21T21:41:16.212 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.212 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.212 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:16.212 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519635+0000", 2024-09-21T21:41:16.212 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.6910000000000002e-06 2024-09-21T21:41:16.212 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.212 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.212 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:16.212 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519761+0000", 2024-09-21T21:41:16.212 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000125988 2024-09-21T21:41:16.212 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.213 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.213 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:16.213 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519767+0000", 2024-09-21T21:41:16.213 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9190000000000004e-06 2024-09-21T21:41:16.213 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.213 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.213 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:16.213 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.532905+0000", 2024-09-21T21:41:16.213 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013137970000000001 2024-09-21T21:41:16.213 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.213 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.213 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:16.213 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.532984+0000", 2024-09-21T21:41:16.213 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.8616999999999996e-05 2024-09-21T21:41:16.213 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.214 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.214 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:16.214 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.533112+0000", 2024-09-21T21:41:16.214 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000128388 2024-09-21T21:41:16.214 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:16.214 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:16.214 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:16.214 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.214 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.214 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4713 3.0 3:074cff57:::rbd_data.112145fae779.000000000000000c:a [read 2191360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:16.214 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.519777+0000", 2024-09-21T21:41:16.214 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 17.609700029999999, 2024-09-21T21:41:16.214 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.01355251, 2024-09-21T21:41:16.214 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:16.215 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:16.215 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:16.215 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:16.215 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:16.215 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:16.215 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4713 2024-09-21T21:41:16.215 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.215 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:16.215 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.215 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:16.215 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519777+0000", 2024-09-21T21:41:16.215 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:16.215 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.215 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.215 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:16.216 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519777+0000", 2024-09-21T21:41:16.216 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:16.216 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.216 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.216 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:16.216 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519777+0000", 2024-09-21T21:41:16.216 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9900000000000001e-07 2024-09-21T21:41:16.216 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.216 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.216 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:16.216 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519780+0000", 2024-09-21T21:41:16.216 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.886e-06 2024-09-21T21:41:16.216 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.216 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.216 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:16.217 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519786+0000", 2024-09-21T21:41:16.217 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0859999999999998e-06 2024-09-21T21:41:16.217 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.217 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.217 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:16.217 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519790+0000", 2024-09-21T21:41:16.217 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2400000000000001e-06 2024-09-21T21:41:16.217 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.217 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.217 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:16.217 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.533143+0000", 2024-09-21T21:41:16.217 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013352711 2024-09-21T21:41:16.217 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.217 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.218 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:16.218 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.533224+0000", 2024-09-21T21:41:16.218 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0675999999999998e-05 2024-09-21T21:41:16.218 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.218 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.218 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:16.218 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.533329+0000", 2024-09-21T21:41:16.218 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000105412 2024-09-21T21:41:16.218 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:16.218 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:16.218 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:16.218 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.218 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.218 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4824.0:1564 3.2 3:4647520c:::rbd_data.1121b97f2925.0000000000000006:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:16.219 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:41:16.219 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 14.493412156, 2024-09-21T21:41:16.219 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056097347999999998, 2024-09-21T21:41:16.219 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:16.219 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:16.219 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:16.219 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:16.219 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4824", 2024-09-21T21:41:16.219 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:16.219 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1564 2024-09-21T21:41:16.219 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.219 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:16.219 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.219 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:16.220 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:41:16.220 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:16.220 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.220 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.220 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:16.220 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:41:16.220 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:16.220 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.220 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.220 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:16.220 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:41:16.220 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6499999999999999e-07 2024-09-21T21:41:16.220 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.220 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.220 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:16.221 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636067+0000", 2024-09-21T21:41:16.221 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3879999999999998e-06 2024-09-21T21:41:16.221 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.221 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.221 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:16.221 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636075+0000", 2024-09-21T21:41:16.221 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5480000000000004e-06 2024-09-21T21:41:16.221 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.221 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.221 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:16.221 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636080+0000", 2024-09-21T21:41:16.221 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.339e-06 2024-09-21T21:41:16.221 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.221 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.222 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:16.222 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636133+0000", 2024-09-21T21:41:16.222 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2930000000000003e-05 2024-09-21T21:41:16.222 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.222 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.222 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:16.222 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636265+0000", 2024-09-21T21:41:16.222 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000132058 2024-09-21T21:41:16.222 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.222 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.222 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:16.222 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.692162+0000", 2024-09-21T21:41:16.222 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055896620000000001 2024-09-21T21:41:16.222 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:16.222 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:16.223 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:16.223 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:16.223 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:16.223 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:16.225 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T21:41:16.225 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:16.222+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:16.225 INFO:teuthology.orchestra.run.smithi164.stdout:2301 read 0x1a36400 thru 0x1a45754 (0xf355 bytes) 2024-09-21T21:41:16.225 INFO:teuthology.orchestra.run.smithi164.stdout:2303 write 0x78b2800 thru 0x78bfbbf (0xd3c0 bytes) 2024-09-21T21:41:16.226 INFO:teuthology.orchestra.run.smithi164.stdout:2304 punch from 0x5256c00 to 0x525e600, (0x7a00 bytes) 2024-09-21T21:41:16.226 INFO:teuthology.orchestra.run.smithi164.stdout:2305 trunc from 0x78bfc00 to 0x3104800 2024-09-21T21:41:16.226 INFO:teuthology.orchestra.run.smithi164.stdout:2306 write 0x3d46200 thru 0x3d498c2 (0x36c3 bytes) 2024-09-21T21:41:16.226 INFO:teuthology.orchestra.run.smithi164.stdout:2308 read 0x3901400 thru 0x390df99 (0xcb9a bytes) 2024-09-21T21:41:16.226 INFO:teuthology.orchestra.run.smithi164.stdout:2313 trunc from 0x3d49a00 to 0xd9b0200 2024-09-21T21:41:16.237 INFO:teuthology.orchestra.run.smithi164.stdout:2316 trunc from 0xd9b0200 to 0xc217e00 2024-09-21T21:41:16.246 INFO:teuthology.orchestra.run.smithi164.stdout:2317 read 0x9abd400 thru 0x9abef1c (0x1b1d bytes) 2024-09-21T21:41:16.247 INFO:teuthology.orchestra.run.smithi164.stdout:2320 read 0x429c200 thru 0x42a7a98 (0xb899 bytes) 2024-09-21T21:41:16.247 INFO:teuthology.orchestra.run.smithi164.stdout:2321 punch from 0x580e00 to 0x590800, (0xfa00 bytes) 2024-09-21T21:41:16.256 INFO:teuthology.orchestra.run.smithi164.stdout:2323 read 0x90f9600 thru 0x9101c50 (0x8651 bytes) 2024-09-21T21:41:16.256 INFO:teuthology.orchestra.run.smithi164.stdout:2324 write 0x3d36a00 thru 0x3d41fd4 (0xb5d5 bytes) 2024-09-21T21:41:16.280 INFO:teuthology.orchestra.run.smithi164.stdout:2325 trunc from 0xc217e00 to 0xe328a00 2024-09-21T21:41:16.303 INFO:teuthology.orchestra.run.smithi164.stdout:2327 punch from 0x6769800 to 0x6769c00, (0x400 bytes) 2024-09-21T21:41:16.304 INFO:teuthology.orchestra.run.smithi164.stdout:2329 write 0x6fb3600 thru 0x6fbd62f (0xa030 bytes) 2024-09-21T21:41:16.305 INFO:teuthology.orchestra.run.smithi164.stdout:2330 write 0xa3ad600 thru 0xa3aedb0 (0x17b1 bytes) 2024-09-21T21:41:16.305 INFO:teuthology.orchestra.run.smithi164.stdout:2333 punch from 0x7984a00 to 0x798c800, (0x7e00 bytes) 2024-09-21T21:41:16.305 INFO:teuthology.orchestra.run.smithi164.stdout:2337 write 0xcd91800 thru 0xcd9b2b2 (0x9ab3 bytes) 2024-09-21T21:41:16.306 INFO:teuthology.orchestra.run.smithi164.stdout:2339 trunc from 0xe328a00 to 0xd843c00 2024-09-21T21:41:16.325 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:16.323+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:16.343 INFO:teuthology.orchestra.run.smithi164.stdout:2344 write 0xdbc7200 thru 0xdbd25d0 (0xb3d1 bytes) 2024-09-21T21:41:16.344 INFO:teuthology.orchestra.run.smithi164.stdout:2346 punch from 0x693de00 to 0x6944200, (0x6400 bytes) 2024-09-21T21:41:16.345 INFO:teuthology.orchestra.run.smithi164.stdout:2349 write 0x3a32600 thru 0x3a3b898 (0x9299 bytes) 2024-09-21T21:41:16.345 INFO:teuthology.orchestra.run.smithi164.stdout:2350 read 0x1cde200 thru 0x1ce4560 (0x6361 bytes) 2024-09-21T21:41:16.345 INFO:teuthology.orchestra.run.smithi164.stdout:2351 read 0x4201400 thru 0x4204e1e (0x3a1f bytes) 2024-09-21T21:41:16.345 INFO:teuthology.orchestra.run.smithi164.stdout:2352 read 0x34c8e00 thru 0x34ca0a2 (0x12a3 bytes) 2024-09-21T21:41:16.345 INFO:teuthology.orchestra.run.smithi164.stdout:2355 punch from 0x5479400 to 0x5484000, (0xac00 bytes) 2024-09-21T21:41:16.346 INFO:teuthology.orchestra.run.smithi164.stdout:2358 write 0x9ab6200 thru 0x9ab7e74 (0x1c75 bytes) 2024-09-21T21:41:16.378 INFO:teuthology.orchestra.run.smithi164.stdout:2359 write 0xde72400 thru 0xde81a6e (0xf66f bytes) 2024-09-21T21:41:16.384 INFO:teuthology.orchestra.run.smithi164.stdout:2361 punch from 0x26f1800 to 0x26fba00, (0xa200 bytes) 2024-09-21T21:41:16.386 INFO:teuthology.orchestra.run.smithi164.stdout:2362 read 0xb49ec00 thru 0xb49fd34 (0x1135 bytes) 2024-09-21T21:41:16.386 INFO:teuthology.orchestra.run.smithi164.stdout:2366 write 0x8bd2400 thru 0x8bdb73d (0x933e bytes) 2024-09-21T21:41:16.386 INFO:teuthology.orchestra.run.smithi164.stdout:2367 punch from 0x65a4600 to 0x65ad000, (0x8a00 bytes) 2024-09-21T21:41:16.388 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:16.388 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:41:16.388 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.389 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_repop(client.4840.0:3742 3.d e48/21 3:b457f831:::rbd_object_map.1121e0174ab1:head v 48'443, mlcod=48'439)", 2024-09-21T21:41:16.389 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:16.387654+0000", 2024-09-21T21:41:16.389 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.00014610900000000001, 2024-09-21T21:41:16.389 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00016208499999999999, 2024-09-21T21:41:16.389 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:16.389 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:16.389 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:16.389 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:16.389 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.389 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:16.390 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.387654+0000", 2024-09-21T21:41:16.390 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:16.390 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.390 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.390 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:16.390 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.387654+0000", 2024-09-21T21:41:16.390 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:16.390 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.390 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.390 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:16.390 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.387654+0000", 2024-09-21T21:41:16.390 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0200000000000002e-07 2024-09-21T21:41:16.390 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.391 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.391 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:16.391 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.387659+0000", 2024-09-21T21:41:16.391 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.967e-06 2024-09-21T21:41:16.391 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.391 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.391 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:16.391 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.387670+0000", 2024-09-21T21:41:16.391 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0161e-05 2024-09-21T21:41:16.391 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.392 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.392 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:16.392 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.387677+0000", 2024-09-21T21:41:16.392 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.824e-06 2024-09-21T21:41:16.392 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.392 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.392 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:16.392 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.387719+0000", 2024-09-21T21:41:16.392 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1596e-05 2024-09-21T21:41:16.392 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.392 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.392 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:16.392 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.387730+0000", 2024-09-21T21:41:16.392 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1192999999999999e-05 2024-09-21T21:41:16.393 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:16.393 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:16.393 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:16.393 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:16.393 INFO:teuthology.orchestra.run.smithi081.stdout: ], 2024-09-21T21:41:16.393 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 1 2024-09-21T21:41:16.393 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:16.400 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T21:41:16.402 INFO:teuthology.orchestra.run.smithi164.stdout:2368 trunc from 0xde81c00 to 0x75b200 2024-09-21T21:41:16.446 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:16.442+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:16.448 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T21:41:16.526 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:16.525+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:16.536 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:16.536 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:41:16.536 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:41:16.536 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:41:16.536 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:16.538 INFO:teuthology.orchestra.run.smithi164.stdout:2369 write 0x45cba00 thru 0x45d75fc (0xbbfd bytes) 2024-09-21T21:41:16.539 INFO:teuthology.orchestra.run.smithi164.stdout:2374 write 0x4516400 thru 0x4520512 (0xa113 bytes) 2024-09-21T21:41:16.546 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T21:41:16.565 INFO:teuthology.orchestra.run.smithi164.stdout:2377 write 0xb598000 thru 0xb5a0a49 (0x8a4a bytes) 2024-09-21T21:41:16.572 INFO:teuthology.orchestra.run.smithi164.stdout:2380 read 0x1bc3400 thru 0x1bce5a5 (0xb1a6 bytes) 2024-09-21T21:41:16.578 INFO:teuthology.orchestra.run.smithi164.stdout:2381 trunc from 0xb5a0c00 to 0x61ca400 2024-09-21T21:41:16.589 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0: {} 2024-09-21T21:41:16.589 INFO:teuthology.orchestra.run.smithi081.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T21:41:16.600 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-21T21:41:16.601 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T21:41:16.601 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-21T21:41:16.601 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-21T21:41:16.602 INFO:teuthology.orchestra.run.smithi164.stdout:2382 punch from 0x49bd600 to 0x49cb800, (0xe200 bytes) 2024-09-21T21:41:16.609 INFO:teuthology.orchestra.run.smithi164.stdout:2384 trunc from 0x61ca400 to 0x1102600 2024-09-21T21:41:16.611 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1: {} 2024-09-21T21:41:16.612 INFO:teuthology.orchestra.run.smithi081.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T21:41:16.627 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:16.626+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:16.632 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2: {} 2024-09-21T21:41:16.632 INFO:teuthology.orchestra.run.smithi081.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T21:41:16.644 INFO:teuthology.orchestra.run.smithi164.stdout:2387 write 0x160ce00 thru 0x16145ea (0x77eb bytes) 2024-09-21T21:41:16.649 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3: {} 2024-09-21T21:41:16.649 INFO:teuthology.orchestra.run.smithi081.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T21:41:16.651 INFO:teuthology.orchestra.run.smithi164.stdout:2388 read 0xdb5000 thru 0xdba4fa (0x54fb bytes) 2024-09-21T21:41:16.651 INFO:teuthology.orchestra.run.smithi164.stdout:2392 read 0x86f400 thru 0x870bb5 (0x17b6 bytes) 2024-09-21T21:41:16.651 INFO:teuthology.orchestra.run.smithi164.stdout:2393 read 0xf00e00 thru 0xf0b0c1 (0xa2c2 bytes) 2024-09-21T21:41:16.651 INFO:teuthology.orchestra.run.smithi164.stdout:2396 punch from 0x192e00 to 0x195800, (0x2a00 bytes) 2024-09-21T21:41:16.667 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:16.667 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:16.670 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4: {} 2024-09-21T21:41:16.670 INFO:teuthology.orchestra.run.smithi081.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T21:41:16.674 INFO:teuthology.orchestra.run.smithi164.stdout:2400 write 0xbe3de00 thru 0xbe3effe (0x11ff bytes) 2024-09-21T21:41:16.676 INFO:teuthology.orchestra.run.smithi164.stdout:2401 read 0x9e72600 thru 0x9e75bd3 (0x35d4 bytes) 2024-09-21T21:41:16.676 INFO:teuthology.orchestra.run.smithi164.stdout:2403 write 0x84ae200 thru 0x84ba618 (0xc419 bytes) 2024-09-21T21:41:16.677 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T21:41:16.679 INFO:teuthology.orchestra.run.smithi164.stdout:2407 trunc from 0xbe3f000 to 0x8c61e00 2024-09-21T21:41:16.689 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5: {} 2024-09-21T21:41:16.689 INFO:teuthology.orchestra.run.smithi081.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T21:41:16.721 INFO:teuthology.orchestra.run.smithi164.stdout:2409 write 0xd4b7c00 thru 0xd4baf06 (0x3307 bytes) 2024-09-21T21:41:16.722 INFO:teuthology.orchestra.run.smithi164.stdout:2410 read 0x2c7a800 thru 0x2c7af24 (0x725 bytes) 2024-09-21T21:41:16.723 INFO:teuthology.orchestra.run.smithi164.stdout:2411 punch from 0x6786a00 to 0x678fa00, (0x9000 bytes) 2024-09-21T21:41:16.723 INFO:teuthology.orchestra.run.smithi164.stdout:2413 clone 15 order 24 su 8388608 sc 11 2024-09-21T21:41:16.743 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:16.742+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:16.791 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:16.791 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:16.800 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T21:41:16.829 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:16.827+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:16.896 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:41:16.896 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:16.896 INFO:teuthology.orchestra.run.smithi081.stdout: "epoch": 48, 2024-09-21T21:41:16.896 INFO:teuthology.orchestra.run.smithi081.stdout: "fsid": "a0fdcfc2-ddd2-4e50-81bb-876d940f3578", 2024-09-21T21:41:16.896 INFO:teuthology.orchestra.run.smithi081.stdout: "created": "2024-09-21T21:40:07.413917+0000", 2024-09-21T21:41:16.896 INFO:teuthology.orchestra.run.smithi081.stdout: "modified": "2024-09-21T21:41:15.589007+0000", 2024-09-21T21:41:16.896 INFO:teuthology.orchestra.run.smithi081.stdout: "last_up_change": "2024-09-21T21:40:18.340430+0000", 2024-09-21T21:41:16.896 INFO:teuthology.orchestra.run.smithi081.stdout: "last_in_change": "2024-09-21T21:40:09.029953+0000", 2024-09-21T21:41:16.896 INFO:teuthology.orchestra.run.smithi081.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-21T21:41:16.896 INFO:teuthology.orchestra.run.smithi081.stdout: "flags_num": 5806080, 2024-09-21T21:41:16.897 INFO:teuthology.orchestra.run.smithi081.stdout: "flags_set": [ 2024-09-21T21:41:16.897 INFO:teuthology.orchestra.run.smithi081.stdout: "nodeep-scrub", 2024-09-21T21:41:16.897 INFO:teuthology.orchestra.run.smithi081.stdout: "noscrub", 2024-09-21T21:41:16.897 INFO:teuthology.orchestra.run.smithi081.stdout: "pglog_hardlimit", 2024-09-21T21:41:16.897 INFO:teuthology.orchestra.run.smithi081.stdout: "purged_snapdirs", 2024-09-21T21:41:16.897 INFO:teuthology.orchestra.run.smithi081.stdout: "recovery_deletes", 2024-09-21T21:41:16.897 INFO:teuthology.orchestra.run.smithi081.stdout: "sortbitwise" 2024-09-21T21:41:16.897 INFO:teuthology.orchestra.run.smithi081.stdout: ], 2024-09-21T21:41:16.897 INFO:teuthology.orchestra.run.smithi081.stdout: "crush_version": 5, 2024-09-21T21:41:16.897 INFO:teuthology.orchestra.run.smithi081.stdout: "full_ratio": 0.94999998807907104, 2024-09-21T21:41:16.897 INFO:teuthology.orchestra.run.smithi081.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-21T21:41:16.897 INFO:teuthology.orchestra.run.smithi081.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-21T21:41:16.897 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster_snapshot": "", 2024-09-21T21:41:16.897 INFO:teuthology.orchestra.run.smithi081.stdout: "pool_max": 3, 2024-09-21T21:41:16.897 INFO:teuthology.orchestra.run.smithi081.stdout: "max_osd": 6, 2024-09-21T21:41:16.898 INFO:teuthology.orchestra.run.smithi081.stdout: "require_min_compat_client": "luminous", 2024-09-21T21:41:16.898 INFO:teuthology.orchestra.run.smithi081.stdout: "min_compat_client": "luminous", 2024-09-21T21:41:16.898 INFO:teuthology.orchestra.run.smithi081.stdout: "require_osd_release": "squid", 2024-09-21T21:41:16.898 INFO:teuthology.orchestra.run.smithi081.stdout: "allow_crimson": false, 2024-09-21T21:41:16.898 INFO:teuthology.orchestra.run.smithi081.stdout: "pools": [ 2024-09-21T21:41:16.898 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.898 INFO:teuthology.orchestra.run.smithi081.stdout: "pool": 1, 2024-09-21T21:41:16.898 INFO:teuthology.orchestra.run.smithi081.stdout: "pool_name": ".mgr", 2024-09-21T21:41:16.898 INFO:teuthology.orchestra.run.smithi081.stdout: "create_time": "2024-09-21T21:40:18.321328+0000", 2024-09-21T21:41:16.898 INFO:teuthology.orchestra.run.smithi081.stdout: "flags": 1, 2024-09-21T21:41:16.898 INFO:teuthology.orchestra.run.smithi081.stdout: "flags_names": "hashpspool", 2024-09-21T21:41:16.898 INFO:teuthology.orchestra.run.smithi081.stdout: "type": 1, 2024-09-21T21:41:16.898 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 2, 2024-09-21T21:41:16.898 INFO:teuthology.orchestra.run.smithi081.stdout: "min_size": 1, 2024-09-21T21:41:16.898 INFO:teuthology.orchestra.run.smithi081.stdout: "crush_rule": 0, 2024-09-21T21:41:16.899 INFO:teuthology.orchestra.run.smithi081.stdout: "peering_crush_bucket_count": 0, 2024-09-21T21:41:16.899 INFO:teuthology.orchestra.run.smithi081.stdout: "peering_crush_bucket_target": 0, 2024-09-21T21:41:16.899 INFO:teuthology.orchestra.run.smithi081.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T21:41:16.899 INFO:teuthology.orchestra.run.smithi081.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T21:41:16.899 INFO:teuthology.orchestra.run.smithi081.stdout: "is_stretch_pool": false, 2024-09-21T21:41:16.899 INFO:teuthology.orchestra.run.smithi081.stdout: "object_hash": 2, 2024-09-21T21:41:16.899 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_autoscale_mode": "off", 2024-09-21T21:41:16.899 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_num": 1, 2024-09-21T21:41:16.899 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_placement_num": 1, 2024-09-21T21:41:16.899 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_placement_num_target": 1, 2024-09-21T21:41:16.899 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_num_target": 1, 2024-09-21T21:41:16.899 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_num_pending": 1, 2024-09-21T21:41:16.899 INFO:teuthology.orchestra.run.smithi081.stdout: "last_pg_merge_meta": { 2024-09-21T21:41:16.899 INFO:teuthology.orchestra.run.smithi081.stdout: "source_pgid": "0.0", 2024-09-21T21:41:16.900 INFO:teuthology.orchestra.run.smithi081.stdout: "ready_epoch": 0, 2024-09-21T21:41:16.900 INFO:teuthology.orchestra.run.smithi081.stdout: "last_epoch_started": 0, 2024-09-21T21:41:16.900 INFO:teuthology.orchestra.run.smithi081.stdout: "last_epoch_clean": 0, 2024-09-21T21:41:16.900 INFO:teuthology.orchestra.run.smithi081.stdout: "source_version": "0'0", 2024-09-21T21:41:16.900 INFO:teuthology.orchestra.run.smithi081.stdout: "target_version": "0'0" 2024-09-21T21:41:16.900 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.900 INFO:teuthology.orchestra.run.smithi081.stdout: "last_change": "15", 2024-09-21T21:41:16.900 INFO:teuthology.orchestra.run.smithi081.stdout: "last_force_op_resend": "0", 2024-09-21T21:41:16.900 INFO:teuthology.orchestra.run.smithi081.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T21:41:16.900 INFO:teuthology.orchestra.run.smithi081.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T21:41:16.900 INFO:teuthology.orchestra.run.smithi081.stdout: "auid": 0, 2024-09-21T21:41:16.900 INFO:teuthology.orchestra.run.smithi081.stdout: "snap_mode": "selfmanaged", 2024-09-21T21:41:16.900 INFO:teuthology.orchestra.run.smithi081.stdout: "snap_seq": 0, 2024-09-21T21:41:16.900 INFO:teuthology.orchestra.run.smithi081.stdout: "snap_epoch": 0, 2024-09-21T21:41:16.900 INFO:teuthology.orchestra.run.smithi081.stdout: "pool_snaps": [], 2024-09-21T21:41:16.900 INFO:teuthology.orchestra.run.smithi081.stdout: "removed_snaps": "[]", 2024-09-21T21:41:16.901 INFO:teuthology.orchestra.run.smithi081.stdout: "quota_max_bytes": 0, 2024-09-21T21:41:16.901 INFO:teuthology.orchestra.run.smithi081.stdout: "quota_max_objects": 0, 2024-09-21T21:41:16.901 INFO:teuthology.orchestra.run.smithi081.stdout: "tiers": [], 2024-09-21T21:41:16.901 INFO:teuthology.orchestra.run.smithi081.stdout: "tier_of": -1, 2024-09-21T21:41:16.901 INFO:teuthology.orchestra.run.smithi081.stdout: "read_tier": -1, 2024-09-21T21:41:16.901 INFO:teuthology.orchestra.run.smithi081.stdout: "write_tier": -1, 2024-09-21T21:41:16.901 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_mode": "none", 2024-09-21T21:41:16.901 INFO:teuthology.orchestra.run.smithi081.stdout: "target_max_bytes": 0, 2024-09-21T21:41:16.901 INFO:teuthology.orchestra.run.smithi081.stdout: "target_max_objects": 0, 2024-09-21T21:41:16.901 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T21:41:16.901 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T21:41:16.901 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T21:41:16.901 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_min_flush_age": 0, 2024-09-21T21:41:16.901 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_min_evict_age": 0, 2024-09-21T21:41:16.902 INFO:teuthology.orchestra.run.smithi081.stdout: "erasure_code_profile": "", 2024-09-21T21:41:16.902 INFO:teuthology.orchestra.run.smithi081.stdout: "hit_set_params": { 2024-09-21T21:41:16.902 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "none" 2024-09-21T21:41:16.902 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.902 INFO:teuthology.orchestra.run.smithi081.stdout: "hit_set_period": 0, 2024-09-21T21:41:16.902 INFO:teuthology.orchestra.run.smithi081.stdout: "hit_set_count": 0, 2024-09-21T21:41:16.902 INFO:teuthology.orchestra.run.smithi081.stdout: "use_gmt_hitset": true, 2024-09-21T21:41:16.902 INFO:teuthology.orchestra.run.smithi081.stdout: "min_read_recency_for_promote": 0, 2024-09-21T21:41:16.902 INFO:teuthology.orchestra.run.smithi081.stdout: "min_write_recency_for_promote": 0, 2024-09-21T21:41:16.902 INFO:teuthology.orchestra.run.smithi081.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T21:41:16.902 INFO:teuthology.orchestra.run.smithi081.stdout: "hit_set_search_last_n": 0, 2024-09-21T21:41:16.902 INFO:teuthology.orchestra.run.smithi081.stdout: "grade_table": [], 2024-09-21T21:41:16.902 INFO:teuthology.orchestra.run.smithi081.stdout: "stripe_width": 0, 2024-09-21T21:41:16.902 INFO:teuthology.orchestra.run.smithi081.stdout: "expected_num_objects": 0, 2024-09-21T21:41:16.903 INFO:teuthology.orchestra.run.smithi081.stdout: "fast_read": false, 2024-09-21T21:41:16.903 INFO:teuthology.orchestra.run.smithi081.stdout: "options": { 2024-09-21T21:41:16.903 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_num_max": 32, 2024-09-21T21:41:16.903 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_num_min": 1 2024-09-21T21:41:16.903 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.903 INFO:teuthology.orchestra.run.smithi081.stdout: "application_metadata": { 2024-09-21T21:41:16.903 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": {} 2024-09-21T21:41:16.903 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.903 INFO:teuthology.orchestra.run.smithi081.stdout: "read_balance": { 2024-09-21T21:41:16.903 INFO:teuthology.orchestra.run.smithi081.stdout: "score_type": "Fair distribution", 2024-09-21T21:41:16.903 INFO:teuthology.orchestra.run.smithi081.stdout: "score_acting": 6.059999942779541, 2024-09-21T21:41:16.903 INFO:teuthology.orchestra.run.smithi081.stdout: "score_stable": 6.059999942779541, 2024-09-21T21:41:16.903 INFO:teuthology.orchestra.run.smithi081.stdout: "optimal_score": 0.33000001311302185, 2024-09-21T21:41:16.903 INFO:teuthology.orchestra.run.smithi081.stdout: "raw_score_acting": 2, 2024-09-21T21:41:16.904 INFO:teuthology.orchestra.run.smithi081.stdout: "raw_score_stable": 2, 2024-09-21T21:41:16.904 INFO:teuthology.orchestra.run.smithi081.stdout: "primary_affinity_weighted": 0.88800817728042603, 2024-09-21T21:41:16.904 INFO:teuthology.orchestra.run.smithi081.stdout: "average_primary_affinity": 0.88999998569488525, 2024-09-21T21:41:16.904 INFO:teuthology.orchestra.run.smithi081.stdout: "average_primary_affinity_weighted": 0.88999998569488525 2024-09-21T21:41:16.904 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:16.904 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.904 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.904 INFO:teuthology.orchestra.run.smithi081.stdout: "pool": 2, 2024-09-21T21:41:16.904 INFO:teuthology.orchestra.run.smithi081.stdout: "pool_name": "rbd", 2024-09-21T21:41:16.904 INFO:teuthology.orchestra.run.smithi081.stdout: "create_time": "2024-09-21T21:40:24.235658+0000", 2024-09-21T21:41:16.904 INFO:teuthology.orchestra.run.smithi081.stdout: "flags": 8193, 2024-09-21T21:41:16.904 INFO:teuthology.orchestra.run.smithi081.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T21:41:16.904 INFO:teuthology.orchestra.run.smithi081.stdout: "type": 1, 2024-09-21T21:41:16.904 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 2, 2024-09-21T21:41:16.904 INFO:teuthology.orchestra.run.smithi081.stdout: "min_size": 1, 2024-09-21T21:41:16.905 INFO:teuthology.orchestra.run.smithi081.stdout: "crush_rule": 0, 2024-09-21T21:41:16.905 INFO:teuthology.orchestra.run.smithi081.stdout: "peering_crush_bucket_count": 0, 2024-09-21T21:41:16.905 INFO:teuthology.orchestra.run.smithi081.stdout: "peering_crush_bucket_target": 0, 2024-09-21T21:41:16.905 INFO:teuthology.orchestra.run.smithi081.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T21:41:16.905 INFO:teuthology.orchestra.run.smithi081.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T21:41:16.905 INFO:teuthology.orchestra.run.smithi081.stdout: "is_stretch_pool": false, 2024-09-21T21:41:16.905 INFO:teuthology.orchestra.run.smithi081.stdout: "object_hash": 2, 2024-09-21T21:41:16.905 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_autoscale_mode": "off", 2024-09-21T21:41:16.905 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_num": 8, 2024-09-21T21:41:16.905 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_placement_num": 8, 2024-09-21T21:41:16.905 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_placement_num_target": 8, 2024-09-21T21:41:16.905 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_num_target": 8, 2024-09-21T21:41:16.905 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_num_pending": 8, 2024-09-21T21:41:16.905 INFO:teuthology.orchestra.run.smithi081.stdout: "last_pg_merge_meta": { 2024-09-21T21:41:16.906 INFO:teuthology.orchestra.run.smithi081.stdout: "source_pgid": "0.0", 2024-09-21T21:41:16.906 INFO:teuthology.orchestra.run.smithi081.stdout: "ready_epoch": 0, 2024-09-21T21:41:16.906 INFO:teuthology.orchestra.run.smithi081.stdout: "last_epoch_started": 0, 2024-09-21T21:41:16.906 INFO:teuthology.orchestra.run.smithi081.stdout: "last_epoch_clean": 0, 2024-09-21T21:41:16.906 INFO:teuthology.orchestra.run.smithi081.stdout: "source_version": "0'0", 2024-09-21T21:41:16.906 INFO:teuthology.orchestra.run.smithi081.stdout: "target_version": "0'0" 2024-09-21T21:41:16.906 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.906 INFO:teuthology.orchestra.run.smithi081.stdout: "last_change": "19", 2024-09-21T21:41:16.906 INFO:teuthology.orchestra.run.smithi081.stdout: "last_force_op_resend": "0", 2024-09-21T21:41:16.906 INFO:teuthology.orchestra.run.smithi081.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T21:41:16.906 INFO:teuthology.orchestra.run.smithi081.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T21:41:16.906 INFO:teuthology.orchestra.run.smithi081.stdout: "auid": 0, 2024-09-21T21:41:16.906 INFO:teuthology.orchestra.run.smithi081.stdout: "snap_mode": "selfmanaged", 2024-09-21T21:41:16.906 INFO:teuthology.orchestra.run.smithi081.stdout: "snap_seq": 2, 2024-09-21T21:41:16.906 INFO:teuthology.orchestra.run.smithi081.stdout: "snap_epoch": 19, 2024-09-21T21:41:16.907 INFO:teuthology.orchestra.run.smithi081.stdout: "pool_snaps": [], 2024-09-21T21:41:16.907 INFO:teuthology.orchestra.run.smithi081.stdout: "removed_snaps": "[]", 2024-09-21T21:41:16.907 INFO:teuthology.orchestra.run.smithi081.stdout: "quota_max_bytes": 0, 2024-09-21T21:41:16.907 INFO:teuthology.orchestra.run.smithi081.stdout: "quota_max_objects": 0, 2024-09-21T21:41:16.907 INFO:teuthology.orchestra.run.smithi081.stdout: "tiers": [], 2024-09-21T21:41:16.907 INFO:teuthology.orchestra.run.smithi081.stdout: "tier_of": -1, 2024-09-21T21:41:16.907 INFO:teuthology.orchestra.run.smithi081.stdout: "read_tier": -1, 2024-09-21T21:41:16.907 INFO:teuthology.orchestra.run.smithi081.stdout: "write_tier": -1, 2024-09-21T21:41:16.907 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_mode": "none", 2024-09-21T21:41:16.907 INFO:teuthology.orchestra.run.smithi081.stdout: "target_max_bytes": 0, 2024-09-21T21:41:16.907 INFO:teuthology.orchestra.run.smithi081.stdout: "target_max_objects": 0, 2024-09-21T21:41:16.907 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T21:41:16.907 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T21:41:16.907 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T21:41:16.908 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_min_flush_age": 0, 2024-09-21T21:41:16.908 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_min_evict_age": 0, 2024-09-21T21:41:16.908 INFO:teuthology.orchestra.run.smithi081.stdout: "erasure_code_profile": "", 2024-09-21T21:41:16.908 INFO:teuthology.orchestra.run.smithi081.stdout: "hit_set_params": { 2024-09-21T21:41:16.908 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "none" 2024-09-21T21:41:16.908 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.908 INFO:teuthology.orchestra.run.smithi081.stdout: "hit_set_period": 0, 2024-09-21T21:41:16.908 INFO:teuthology.orchestra.run.smithi081.stdout: "hit_set_count": 0, 2024-09-21T21:41:16.908 INFO:teuthology.orchestra.run.smithi081.stdout: "use_gmt_hitset": true, 2024-09-21T21:41:16.908 INFO:teuthology.orchestra.run.smithi081.stdout: "min_read_recency_for_promote": 0, 2024-09-21T21:41:16.908 INFO:teuthology.orchestra.run.smithi081.stdout: "min_write_recency_for_promote": 0, 2024-09-21T21:41:16.908 INFO:teuthology.orchestra.run.smithi081.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T21:41:16.908 INFO:teuthology.orchestra.run.smithi081.stdout: "hit_set_search_last_n": 0, 2024-09-21T21:41:16.908 INFO:teuthology.orchestra.run.smithi081.stdout: "grade_table": [], 2024-09-21T21:41:16.908 INFO:teuthology.orchestra.run.smithi081.stdout: "stripe_width": 0, 2024-09-21T21:41:16.909 INFO:teuthology.orchestra.run.smithi081.stdout: "expected_num_objects": 0, 2024-09-21T21:41:16.909 INFO:teuthology.orchestra.run.smithi081.stdout: "fast_read": false, 2024-09-21T21:41:16.909 INFO:teuthology.orchestra.run.smithi081.stdout: "options": {}, 2024-09-21T21:41:16.909 INFO:teuthology.orchestra.run.smithi081.stdout: "application_metadata": { 2024-09-21T21:41:16.909 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd": {} 2024-09-21T21:41:16.909 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.909 INFO:teuthology.orchestra.run.smithi081.stdout: "read_balance": { 2024-09-21T21:41:16.909 INFO:teuthology.orchestra.run.smithi081.stdout: "score_type": "Fair distribution", 2024-09-21T21:41:16.909 INFO:teuthology.orchestra.run.smithi081.stdout: "score_acting": 2.25, 2024-09-21T21:41:16.909 INFO:teuthology.orchestra.run.smithi081.stdout: "score_stable": 2.25, 2024-09-21T21:41:16.909 INFO:teuthology.orchestra.run.smithi081.stdout: "optimal_score": 1, 2024-09-21T21:41:16.909 INFO:teuthology.orchestra.run.smithi081.stdout: "raw_score_acting": 2.25, 2024-09-21T21:41:16.909 INFO:teuthology.orchestra.run.smithi081.stdout: "raw_score_stable": 2.25, 2024-09-21T21:41:16.909 INFO:teuthology.orchestra.run.smithi081.stdout: "primary_affinity_weighted": 0.88800817728042603, 2024-09-21T21:41:16.909 INFO:teuthology.orchestra.run.smithi081.stdout: "average_primary_affinity": 0.88999998569488525, 2024-09-21T21:41:16.909 INFO:teuthology.orchestra.run.smithi081.stdout: "average_primary_affinity_weighted": 0.88999998569488525 2024-09-21T21:41:16.910 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:16.910 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.910 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.910 INFO:teuthology.orchestra.run.smithi081.stdout: "pool": 3, 2024-09-21T21:41:16.910 INFO:teuthology.orchestra.run.smithi081.stdout: "pool_name": "pool_client.0", 2024-09-21T21:41:16.910 INFO:teuthology.orchestra.run.smithi081.stdout: "create_time": "2024-09-21T21:40:36.093691+0000", 2024-09-21T21:41:16.910 INFO:teuthology.orchestra.run.smithi081.stdout: "flags": 8193, 2024-09-21T21:41:16.910 INFO:teuthology.orchestra.run.smithi081.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T21:41:16.910 INFO:teuthology.orchestra.run.smithi081.stdout: "type": 1, 2024-09-21T21:41:16.910 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 2, 2024-09-21T21:41:16.910 INFO:teuthology.orchestra.run.smithi081.stdout: "min_size": 1, 2024-09-21T21:41:16.910 INFO:teuthology.orchestra.run.smithi081.stdout: "crush_rule": 0, 2024-09-21T21:41:16.910 INFO:teuthology.orchestra.run.smithi081.stdout: "peering_crush_bucket_count": 0, 2024-09-21T21:41:16.910 INFO:teuthology.orchestra.run.smithi081.stdout: "peering_crush_bucket_target": 0, 2024-09-21T21:41:16.911 INFO:teuthology.orchestra.run.smithi081.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T21:41:16.911 INFO:teuthology.orchestra.run.smithi081.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T21:41:16.911 INFO:teuthology.orchestra.run.smithi081.stdout: "is_stretch_pool": false, 2024-09-21T21:41:16.911 INFO:teuthology.orchestra.run.smithi081.stdout: "object_hash": 2, 2024-09-21T21:41:16.911 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_autoscale_mode": "off", 2024-09-21T21:41:16.911 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_num": 32, 2024-09-21T21:41:16.911 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_placement_num": 32, 2024-09-21T21:41:16.911 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_placement_num_target": 32, 2024-09-21T21:41:16.911 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_num_target": 32, 2024-09-21T21:41:16.911 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_num_pending": 32, 2024-09-21T21:41:16.911 INFO:teuthology.orchestra.run.smithi081.stdout: "last_pg_merge_meta": { 2024-09-21T21:41:16.911 INFO:teuthology.orchestra.run.smithi081.stdout: "source_pgid": "0.0", 2024-09-21T21:41:16.911 INFO:teuthology.orchestra.run.smithi081.stdout: "ready_epoch": 0, 2024-09-21T21:41:16.911 INFO:teuthology.orchestra.run.smithi081.stdout: "last_epoch_started": 0, 2024-09-21T21:41:16.911 INFO:teuthology.orchestra.run.smithi081.stdout: "last_epoch_clean": 0, 2024-09-21T21:41:16.911 INFO:teuthology.orchestra.run.smithi081.stdout: "source_version": "0'0", 2024-09-21T21:41:16.912 INFO:teuthology.orchestra.run.smithi081.stdout: "target_version": "0'0" 2024-09-21T21:41:16.912 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.912 INFO:teuthology.orchestra.run.smithi081.stdout: "last_change": "47", 2024-09-21T21:41:16.912 INFO:teuthology.orchestra.run.smithi081.stdout: "last_force_op_resend": "0", 2024-09-21T21:41:16.912 INFO:teuthology.orchestra.run.smithi081.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T21:41:16.912 INFO:teuthology.orchestra.run.smithi081.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T21:41:16.912 INFO:teuthology.orchestra.run.smithi081.stdout: "auid": 0, 2024-09-21T21:41:16.912 INFO:teuthology.orchestra.run.smithi081.stdout: "snap_mode": "selfmanaged", 2024-09-21T21:41:16.912 INFO:teuthology.orchestra.run.smithi081.stdout: "snap_seq": 16, 2024-09-21T21:41:16.912 INFO:teuthology.orchestra.run.smithi081.stdout: "snap_epoch": 47, 2024-09-21T21:41:16.912 INFO:teuthology.orchestra.run.smithi081.stdout: "pool_snaps": [], 2024-09-21T21:41:16.912 INFO:teuthology.orchestra.run.smithi081.stdout: "removed_snaps": "[]", 2024-09-21T21:41:16.912 INFO:teuthology.orchestra.run.smithi081.stdout: "quota_max_bytes": 0, 2024-09-21T21:41:16.913 INFO:teuthology.orchestra.run.smithi081.stdout: "quota_max_objects": 0, 2024-09-21T21:41:16.913 INFO:teuthology.orchestra.run.smithi081.stdout: "tiers": [], 2024-09-21T21:41:16.913 INFO:teuthology.orchestra.run.smithi081.stdout: "tier_of": -1, 2024-09-21T21:41:16.913 INFO:teuthology.orchestra.run.smithi081.stdout: "read_tier": -1, 2024-09-21T21:41:16.913 INFO:teuthology.orchestra.run.smithi081.stdout: "write_tier": -1, 2024-09-21T21:41:16.913 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_mode": "none", 2024-09-21T21:41:16.913 INFO:teuthology.orchestra.run.smithi081.stdout: "target_max_bytes": 0, 2024-09-21T21:41:16.913 INFO:teuthology.orchestra.run.smithi081.stdout: "target_max_objects": 0, 2024-09-21T21:41:16.913 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T21:41:16.913 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T21:41:16.913 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T21:41:16.913 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_min_flush_age": 0, 2024-09-21T21:41:16.913 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_min_evict_age": 0, 2024-09-21T21:41:16.913 INFO:teuthology.orchestra.run.smithi081.stdout: "erasure_code_profile": "", 2024-09-21T21:41:16.913 INFO:teuthology.orchestra.run.smithi081.stdout: "hit_set_params": { 2024-09-21T21:41:16.913 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "none" 2024-09-21T21:41:16.914 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.914 INFO:teuthology.orchestra.run.smithi081.stdout: "hit_set_period": 0, 2024-09-21T21:41:16.914 INFO:teuthology.orchestra.run.smithi081.stdout: "hit_set_count": 0, 2024-09-21T21:41:16.914 INFO:teuthology.orchestra.run.smithi081.stdout: "use_gmt_hitset": true, 2024-09-21T21:41:16.914 INFO:teuthology.orchestra.run.smithi081.stdout: "min_read_recency_for_promote": 0, 2024-09-21T21:41:16.914 INFO:teuthology.orchestra.run.smithi081.stdout: "min_write_recency_for_promote": 0, 2024-09-21T21:41:16.914 INFO:teuthology.orchestra.run.smithi081.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T21:41:16.914 INFO:teuthology.orchestra.run.smithi081.stdout: "hit_set_search_last_n": 0, 2024-09-21T21:41:16.914 INFO:teuthology.orchestra.run.smithi081.stdout: "grade_table": [], 2024-09-21T21:41:16.914 INFO:teuthology.orchestra.run.smithi081.stdout: "stripe_width": 0, 2024-09-21T21:41:16.914 INFO:teuthology.orchestra.run.smithi081.stdout: "expected_num_objects": 0, 2024-09-21T21:41:16.914 INFO:teuthology.orchestra.run.smithi081.stdout: "fast_read": false, 2024-09-21T21:41:16.914 INFO:teuthology.orchestra.run.smithi081.stdout: "options": {}, 2024-09-21T21:41:16.914 INFO:teuthology.orchestra.run.smithi081.stdout: "application_metadata": { 2024-09-21T21:41:16.915 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd": {} 2024-09-21T21:41:16.915 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.915 INFO:teuthology.orchestra.run.smithi081.stdout: "read_balance": { 2024-09-21T21:41:16.915 INFO:teuthology.orchestra.run.smithi081.stdout: "score_type": "Fair distribution", 2024-09-21T21:41:16.915 INFO:teuthology.orchestra.run.smithi081.stdout: "score_acting": 1.5, 2024-09-21T21:41:16.915 INFO:teuthology.orchestra.run.smithi081.stdout: "score_stable": 1.5, 2024-09-21T21:41:16.915 INFO:teuthology.orchestra.run.smithi081.stdout: "optimal_score": 1, 2024-09-21T21:41:16.915 INFO:teuthology.orchestra.run.smithi081.stdout: "raw_score_acting": 1.5, 2024-09-21T21:41:16.915 INFO:teuthology.orchestra.run.smithi081.stdout: "raw_score_stable": 1.5, 2024-09-21T21:41:16.915 INFO:teuthology.orchestra.run.smithi081.stdout: "primary_affinity_weighted": 0.88800817728042603, 2024-09-21T21:41:16.915 INFO:teuthology.orchestra.run.smithi081.stdout: "average_primary_affinity": 0.88999998569488525, 2024-09-21T21:41:16.915 INFO:teuthology.orchestra.run.smithi081.stdout: "average_primary_affinity_weighted": 0.88999998569488525 2024-09-21T21:41:16.915 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:16.916 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:16.916 INFO:teuthology.orchestra.run.smithi081.stdout: ], 2024-09-21T21:41:16.916 INFO:teuthology.orchestra.run.smithi081.stdout: "osds": [ 2024-09-21T21:41:16.916 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.916 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": 0, 2024-09-21T21:41:16.916 INFO:teuthology.orchestra.run.smithi081.stdout: "uuid": "a9869830-8e74-4b02-963d-ab630a0ee286", 2024-09-21T21:41:16.916 INFO:teuthology.orchestra.run.smithi081.stdout: "up": 1, 2024-09-21T21:41:16.916 INFO:teuthology.orchestra.run.smithi081.stdout: "in": 1, 2024-09-21T21:41:16.916 INFO:teuthology.orchestra.run.smithi081.stdout: "weight": 1, 2024-09-21T21:41:16.916 INFO:teuthology.orchestra.run.smithi081.stdout: "primary_affinity": 1, 2024-09-21T21:41:16.916 INFO:teuthology.orchestra.run.smithi081.stdout: "last_clean_begin": 0, 2024-09-21T21:41:16.916 INFO:teuthology.orchestra.run.smithi081.stdout: "last_clean_end": 0, 2024-09-21T21:41:16.916 INFO:teuthology.orchestra.run.smithi081.stdout: "up_from": 12, 2024-09-21T21:41:16.917 INFO:teuthology.orchestra.run.smithi081.stdout: "up_thru": 40, 2024-09-21T21:41:16.917 INFO:teuthology.orchestra.run.smithi081.stdout: "down_at": 0, 2024-09-21T21:41:16.917 INFO:teuthology.orchestra.run.smithi081.stdout: "lost_at": 0, 2024-09-21T21:41:16.917 INFO:teuthology.orchestra.run.smithi081.stdout: "public_addrs": { 2024-09-21T21:41:16.917 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:41:16.917 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.917 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:41:16.917 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6808", 2024-09-21T21:41:16.917 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1657552338 2024-09-21T21:41:16.917 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.917 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.917 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:41:16.917 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6809", 2024-09-21T21:41:16.917 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1657552338 2024-09-21T21:41:16.918 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:16.918 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:16.918 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.918 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster_addrs": { 2024-09-21T21:41:16.918 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:41:16.918 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.918 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:41:16.918 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6810", 2024-09-21T21:41:16.918 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1657552338 2024-09-21T21:41:16.918 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.918 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.918 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:41:16.918 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6811", 2024-09-21T21:41:16.919 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1657552338 2024-09-21T21:41:16.919 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:16.919 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:16.919 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.919 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_back_addrs": { 2024-09-21T21:41:16.919 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:41:16.919 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.919 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:41:16.919 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6814", 2024-09-21T21:41:16.919 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1657552338 2024-09-21T21:41:16.919 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.919 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.919 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:41:16.919 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6815", 2024-09-21T21:41:16.920 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1657552338 2024-09-21T21:41:16.920 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:16.920 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:16.920 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.920 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_front_addrs": { 2024-09-21T21:41:16.920 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:41:16.920 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.920 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:41:16.920 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6812", 2024-09-21T21:41:16.920 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1657552338 2024-09-21T21:41:16.920 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.920 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.920 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:41:16.920 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6813", 2024-09-21T21:41:16.921 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1657552338 2024-09-21T21:41:16.921 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:16.921 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:16.921 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.921 INFO:teuthology.orchestra.run.smithi081.stdout: "public_addr": "172.21.15.81:6809/1657552338", 2024-09-21T21:41:16.921 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster_addr": "172.21.15.81:6811/1657552338", 2024-09-21T21:41:16.921 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_back_addr": "172.21.15.81:6815/1657552338", 2024-09-21T21:41:16.921 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_front_addr": "172.21.15.81:6813/1657552338", 2024-09-21T21:41:16.921 INFO:teuthology.orchestra.run.smithi081.stdout: "state": [ 2024-09-21T21:41:16.921 INFO:teuthology.orchestra.run.smithi081.stdout: "exists", 2024-09-21T21:41:16.921 INFO:teuthology.orchestra.run.smithi081.stdout: "up" 2024-09-21T21:41:16.921 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:16.921 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.921 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.921 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": 1, 2024-09-21T21:41:16.922 INFO:teuthology.orchestra.run.smithi081.stdout: "uuid": "b189819d-69a3-4e95-96e7-96dc4809ac80", 2024-09-21T21:41:16.922 INFO:teuthology.orchestra.run.smithi081.stdout: "up": 1, 2024-09-21T21:41:16.922 INFO:teuthology.orchestra.run.smithi081.stdout: "in": 1, 2024-09-21T21:41:16.922 INFO:teuthology.orchestra.run.smithi081.stdout: "weight": 1, 2024-09-21T21:41:16.922 INFO:teuthology.orchestra.run.smithi081.stdout: "primary_affinity": 1, 2024-09-21T21:41:16.922 INFO:teuthology.orchestra.run.smithi081.stdout: "last_clean_begin": 0, 2024-09-21T21:41:16.922 INFO:teuthology.orchestra.run.smithi081.stdout: "last_clean_end": 0, 2024-09-21T21:41:16.922 INFO:teuthology.orchestra.run.smithi081.stdout: "up_from": 12, 2024-09-21T21:41:16.922 INFO:teuthology.orchestra.run.smithi081.stdout: "up_thru": 40, 2024-09-21T21:41:16.922 INFO:teuthology.orchestra.run.smithi081.stdout: "down_at": 0, 2024-09-21T21:41:16.922 INFO:teuthology.orchestra.run.smithi081.stdout: "lost_at": 0, 2024-09-21T21:41:16.922 INFO:teuthology.orchestra.run.smithi081.stdout: "public_addrs": { 2024-09-21T21:41:16.922 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:41:16.923 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.923 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:41:16.923 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6816", 2024-09-21T21:41:16.923 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2864159502 2024-09-21T21:41:16.923 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.923 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.923 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:41:16.923 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6817", 2024-09-21T21:41:16.923 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2864159502 2024-09-21T21:41:16.923 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:16.923 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:16.923 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.923 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster_addrs": { 2024-09-21T21:41:16.923 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:41:16.923 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.923 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:41:16.924 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6818", 2024-09-21T21:41:16.924 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2864159502 2024-09-21T21:41:16.924 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.924 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.924 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:41:16.924 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6819", 2024-09-21T21:41:16.924 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2864159502 2024-09-21T21:41:16.924 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:16.924 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:16.924 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.924 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_back_addrs": { 2024-09-21T21:41:16.924 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:41:16.924 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.924 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:41:16.924 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6822", 2024-09-21T21:41:16.925 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2864159502 2024-09-21T21:41:16.925 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.925 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.925 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:41:16.925 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6823", 2024-09-21T21:41:16.925 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2864159502 2024-09-21T21:41:16.925 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:16.925 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:16.925 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.925 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_front_addrs": { 2024-09-21T21:41:16.925 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:41:16.925 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.925 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:41:16.925 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6820", 2024-09-21T21:41:16.925 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2864159502 2024-09-21T21:41:16.926 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.926 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.926 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:41:16.926 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6821", 2024-09-21T21:41:16.926 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2864159502 2024-09-21T21:41:16.926 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:16.926 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:16.926 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.926 INFO:teuthology.orchestra.run.smithi081.stdout: "public_addr": "172.21.15.81:6817/2864159502", 2024-09-21T21:41:16.926 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster_addr": "172.21.15.81:6819/2864159502", 2024-09-21T21:41:16.926 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_back_addr": "172.21.15.81:6823/2864159502", 2024-09-21T21:41:16.926 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_front_addr": "172.21.15.81:6821/2864159502", 2024-09-21T21:41:16.926 INFO:teuthology.orchestra.run.smithi081.stdout: "state": [ 2024-09-21T21:41:16.926 INFO:teuthology.orchestra.run.smithi081.stdout: "exists", 2024-09-21T21:41:16.927 INFO:teuthology.orchestra.run.smithi081.stdout: "up" 2024-09-21T21:41:16.927 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:16.927 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.927 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.927 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": 2, 2024-09-21T21:41:16.927 INFO:teuthology.orchestra.run.smithi081.stdout: "uuid": "8437b99a-1297-4903-ab66-95c5cba2b2bf", 2024-09-21T21:41:16.927 INFO:teuthology.orchestra.run.smithi081.stdout: "up": 1, 2024-09-21T21:41:16.927 INFO:teuthology.orchestra.run.smithi081.stdout: "in": 1, 2024-09-21T21:41:16.927 INFO:teuthology.orchestra.run.smithi081.stdout: "weight": 1, 2024-09-21T21:41:16.927 INFO:teuthology.orchestra.run.smithi081.stdout: "primary_affinity": 1, 2024-09-21T21:41:16.927 INFO:teuthology.orchestra.run.smithi081.stdout: "last_clean_begin": 0, 2024-09-21T21:41:16.927 INFO:teuthology.orchestra.run.smithi081.stdout: "last_clean_end": 0, 2024-09-21T21:41:16.927 INFO:teuthology.orchestra.run.smithi081.stdout: "up_from": 11, 2024-09-21T21:41:16.928 INFO:teuthology.orchestra.run.smithi081.stdout: "up_thru": 45, 2024-09-21T21:41:16.928 INFO:teuthology.orchestra.run.smithi081.stdout: "down_at": 0, 2024-09-21T21:41:16.928 INFO:teuthology.orchestra.run.smithi081.stdout: "lost_at": 0, 2024-09-21T21:41:16.928 INFO:teuthology.orchestra.run.smithi081.stdout: "public_addrs": { 2024-09-21T21:41:16.928 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:41:16.928 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.928 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:41:16.928 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6800", 2024-09-21T21:41:16.928 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 3023160932 2024-09-21T21:41:16.928 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.928 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.928 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:41:16.928 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6801", 2024-09-21T21:41:16.928 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 3023160932 2024-09-21T21:41:16.929 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:16.929 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:16.929 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.929 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster_addrs": { 2024-09-21T21:41:16.929 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:41:16.929 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.929 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:41:16.929 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6802", 2024-09-21T21:41:16.929 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 3023160932 2024-09-21T21:41:16.929 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.929 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.929 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:41:16.929 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6803", 2024-09-21T21:41:16.929 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 3023160932 2024-09-21T21:41:16.930 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:16.930 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:16.930 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.930 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_back_addrs": { 2024-09-21T21:41:16.930 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:41:16.930 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.930 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:41:16.930 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6806", 2024-09-21T21:41:16.930 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 3023160932 2024-09-21T21:41:16.930 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.930 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.930 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:41:16.930 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6807", 2024-09-21T21:41:16.930 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 3023160932 2024-09-21T21:41:16.931 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:16.931 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:16.931 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.931 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_front_addrs": { 2024-09-21T21:41:16.931 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:41:16.931 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.931 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:41:16.931 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6804", 2024-09-21T21:41:16.931 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 3023160932 2024-09-21T21:41:16.931 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.931 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.931 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:41:16.931 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6805", 2024-09-21T21:41:16.931 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 3023160932 2024-09-21T21:41:16.932 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:16.932 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:16.932 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.932 INFO:teuthology.orchestra.run.smithi081.stdout: "public_addr": "172.21.15.81:6801/3023160932", 2024-09-21T21:41:16.932 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster_addr": "172.21.15.81:6803/3023160932", 2024-09-21T21:41:16.932 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_back_addr": "172.21.15.81:6807/3023160932", 2024-09-21T21:41:16.932 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_front_addr": "172.21.15.81:6805/3023160932", 2024-09-21T21:41:16.932 INFO:teuthology.orchestra.run.smithi081.stdout: "state": [ 2024-09-21T21:41:16.932 INFO:teuthology.orchestra.run.smithi081.stdout: "exists", 2024-09-21T21:41:16.932 INFO:teuthology.orchestra.run.smithi081.stdout: "up" 2024-09-21T21:41:16.932 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:16.932 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.932 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.932 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": 3, 2024-09-21T21:41:16.932 INFO:teuthology.orchestra.run.smithi081.stdout: "uuid": "e8848768-c1aa-445b-9554-dc6aa4eefc8d", 2024-09-21T21:41:16.933 INFO:teuthology.orchestra.run.smithi081.stdout: "up": 1, 2024-09-21T21:41:16.933 INFO:teuthology.orchestra.run.smithi081.stdout: "in": 1, 2024-09-21T21:41:16.933 INFO:teuthology.orchestra.run.smithi081.stdout: "weight": 1, 2024-09-21T21:41:16.933 INFO:teuthology.orchestra.run.smithi081.stdout: "primary_affinity": 1, 2024-09-21T21:41:16.933 INFO:teuthology.orchestra.run.smithi081.stdout: "last_clean_begin": 0, 2024-09-21T21:41:16.933 INFO:teuthology.orchestra.run.smithi081.stdout: "last_clean_end": 0, 2024-09-21T21:41:16.933 INFO:teuthology.orchestra.run.smithi081.stdout: "up_from": 13, 2024-09-21T21:41:16.933 INFO:teuthology.orchestra.run.smithi081.stdout: "up_thru": 21, 2024-09-21T21:41:16.933 INFO:teuthology.orchestra.run.smithi081.stdout: "down_at": 0, 2024-09-21T21:41:16.933 INFO:teuthology.orchestra.run.smithi081.stdout: "lost_at": 0, 2024-09-21T21:41:16.933 INFO:teuthology.orchestra.run.smithi081.stdout: "public_addrs": { 2024-09-21T21:41:16.934 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:41:16.934 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.934 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:41:16.934 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6808", 2024-09-21T21:41:16.934 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2681016844 2024-09-21T21:41:16.934 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.934 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.934 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:41:16.934 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6809", 2024-09-21T21:41:16.934 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2681016844 2024-09-21T21:41:16.934 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:16.934 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:16.934 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.934 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster_addrs": { 2024-09-21T21:41:16.934 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:41:16.935 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.935 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:41:16.935 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6810", 2024-09-21T21:41:16.935 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2681016844 2024-09-21T21:41:16.935 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.935 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.935 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:41:16.935 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6811", 2024-09-21T21:41:16.935 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2681016844 2024-09-21T21:41:16.935 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:16.935 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:16.935 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.935 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_back_addrs": { 2024-09-21T21:41:16.935 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:41:16.936 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.936 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:41:16.936 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6814", 2024-09-21T21:41:16.936 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2681016844 2024-09-21T21:41:16.936 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.936 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.936 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:41:16.936 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6815", 2024-09-21T21:41:16.936 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2681016844 2024-09-21T21:41:16.936 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:16.936 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:16.936 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.936 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_front_addrs": { 2024-09-21T21:41:16.936 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:41:16.936 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.936 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:41:16.937 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6812", 2024-09-21T21:41:16.937 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2681016844 2024-09-21T21:41:16.937 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.937 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.937 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:41:16.937 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6813", 2024-09-21T21:41:16.937 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2681016844 2024-09-21T21:41:16.937 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:16.937 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:16.937 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.937 INFO:teuthology.orchestra.run.smithi081.stdout: "public_addr": "172.21.15.122:6809/2681016844", 2024-09-21T21:41:16.937 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster_addr": "172.21.15.122:6811/2681016844", 2024-09-21T21:41:16.937 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_back_addr": "172.21.15.122:6815/2681016844", 2024-09-21T21:41:16.937 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_front_addr": "172.21.15.122:6813/2681016844", 2024-09-21T21:41:16.938 INFO:teuthology.orchestra.run.smithi081.stdout: "state": [ 2024-09-21T21:41:16.938 INFO:teuthology.orchestra.run.smithi081.stdout: "exists", 2024-09-21T21:41:16.938 INFO:teuthology.orchestra.run.smithi081.stdout: "up" 2024-09-21T21:41:16.938 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:16.938 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.938 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.938 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": 4, 2024-09-21T21:41:16.938 INFO:teuthology.orchestra.run.smithi081.stdout: "uuid": "b1d8e85d-a207-4ba6-9322-61d07812083e", 2024-09-21T21:41:16.938 INFO:teuthology.orchestra.run.smithi081.stdout: "up": 1, 2024-09-21T21:41:16.938 INFO:teuthology.orchestra.run.smithi081.stdout: "in": 1, 2024-09-21T21:41:16.938 INFO:teuthology.orchestra.run.smithi081.stdout: "weight": 1, 2024-09-21T21:41:16.938 INFO:teuthology.orchestra.run.smithi081.stdout: "primary_affinity": 1, 2024-09-21T21:41:16.938 INFO:teuthology.orchestra.run.smithi081.stdout: "last_clean_begin": 0, 2024-09-21T21:41:16.938 INFO:teuthology.orchestra.run.smithi081.stdout: "last_clean_end": 0, 2024-09-21T21:41:16.939 INFO:teuthology.orchestra.run.smithi081.stdout: "up_from": 13, 2024-09-21T21:41:16.939 INFO:teuthology.orchestra.run.smithi081.stdout: "up_thru": 40, 2024-09-21T21:41:16.939 INFO:teuthology.orchestra.run.smithi081.stdout: "down_at": 0, 2024-09-21T21:41:16.939 INFO:teuthology.orchestra.run.smithi081.stdout: "lost_at": 0, 2024-09-21T21:41:16.939 INFO:teuthology.orchestra.run.smithi081.stdout: "public_addrs": { 2024-09-21T21:41:16.939 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:41:16.939 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.939 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:41:16.939 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6800", 2024-09-21T21:41:16.939 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1095023540 2024-09-21T21:41:16.939 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.939 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.939 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:41:16.940 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6801", 2024-09-21T21:41:16.940 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1095023540 2024-09-21T21:41:16.940 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:16.940 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:16.940 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.940 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster_addrs": { 2024-09-21T21:41:16.940 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:41:16.940 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.940 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:41:16.940 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6802", 2024-09-21T21:41:16.940 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1095023540 2024-09-21T21:41:16.940 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.940 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.940 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:41:16.941 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6803", 2024-09-21T21:41:16.941 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1095023540 2024-09-21T21:41:16.941 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:16.941 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:16.941 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.941 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_back_addrs": { 2024-09-21T21:41:16.941 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:41:16.941 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.941 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:41:16.941 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6806", 2024-09-21T21:41:16.941 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1095023540 2024-09-21T21:41:16.941 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.941 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.941 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:41:16.941 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6807", 2024-09-21T21:41:16.942 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1095023540 2024-09-21T21:41:16.942 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:16.942 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:16.942 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.942 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_front_addrs": { 2024-09-21T21:41:16.942 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:41:16.942 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.942 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:41:16.942 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6804", 2024-09-21T21:41:16.942 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1095023540 2024-09-21T21:41:16.942 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.942 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.942 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:41:16.942 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6805", 2024-09-21T21:41:16.943 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1095023540 2024-09-21T21:41:16.943 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:16.943 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:16.943 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.943 INFO:teuthology.orchestra.run.smithi081.stdout: "public_addr": "172.21.15.122:6801/1095023540", 2024-09-21T21:41:16.943 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster_addr": "172.21.15.122:6803/1095023540", 2024-09-21T21:41:16.943 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_back_addr": "172.21.15.122:6807/1095023540", 2024-09-21T21:41:16.943 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_front_addr": "172.21.15.122:6805/1095023540", 2024-09-21T21:41:16.944 INFO:teuthology.orchestra.run.smithi081.stdout: "state": [ 2024-09-21T21:41:16.944 INFO:teuthology.orchestra.run.smithi081.stdout: "exists", 2024-09-21T21:41:16.944 INFO:teuthology.orchestra.run.smithi081.stdout: "up" 2024-09-21T21:41:16.944 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:16.944 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.944 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.944 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": 5, 2024-09-21T21:41:16.944 INFO:teuthology.orchestra.run.smithi081.stdout: "uuid": "3ab0450b-d8fd-4ef2-a298-08d5aa2ea32f", 2024-09-21T21:41:16.944 INFO:teuthology.orchestra.run.smithi081.stdout: "up": 1, 2024-09-21T21:41:16.944 INFO:teuthology.orchestra.run.smithi081.stdout: "in": 1, 2024-09-21T21:41:16.944 INFO:teuthology.orchestra.run.smithi081.stdout: "weight": 1, 2024-09-21T21:41:16.944 INFO:teuthology.orchestra.run.smithi081.stdout: "primary_affinity": 0.3280487060546875, 2024-09-21T21:41:16.944 INFO:teuthology.orchestra.run.smithi081.stdout: "last_clean_begin": 0, 2024-09-21T21:41:16.945 INFO:teuthology.orchestra.run.smithi081.stdout: "last_clean_end": 0, 2024-09-21T21:41:16.945 INFO:teuthology.orchestra.run.smithi081.stdout: "up_from": 12, 2024-09-21T21:41:16.945 INFO:teuthology.orchestra.run.smithi081.stdout: "up_thru": 21, 2024-09-21T21:41:16.945 INFO:teuthology.orchestra.run.smithi081.stdout: "down_at": 0, 2024-09-21T21:41:16.945 INFO:teuthology.orchestra.run.smithi081.stdout: "lost_at": 0, 2024-09-21T21:41:16.945 INFO:teuthology.orchestra.run.smithi081.stdout: "public_addrs": { 2024-09-21T21:41:16.945 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:41:16.945 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.945 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:41:16.945 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6816", 2024-09-21T21:41:16.945 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1974398333 2024-09-21T21:41:16.945 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.945 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.945 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:41:16.946 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6817", 2024-09-21T21:41:16.946 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1974398333 2024-09-21T21:41:16.946 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:16.946 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:16.946 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.946 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster_addrs": { 2024-09-21T21:41:16.946 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:41:16.946 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.946 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:41:16.946 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6818", 2024-09-21T21:41:16.946 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1974398333 2024-09-21T21:41:16.946 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.946 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.946 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:41:16.947 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6819", 2024-09-21T21:41:16.947 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1974398333 2024-09-21T21:41:16.947 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:16.947 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:16.947 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.947 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_back_addrs": { 2024-09-21T21:41:16.947 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:41:16.947 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.947 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:41:16.947 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6822", 2024-09-21T21:41:16.947 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1974398333 2024-09-21T21:41:16.947 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.947 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.948 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:41:16.948 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6823", 2024-09-21T21:41:16.948 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1974398333 2024-09-21T21:41:16.948 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:16.948 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:16.948 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.948 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_front_addrs": { 2024-09-21T21:41:16.948 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:41:16.948 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.948 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:41:16.948 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6820", 2024-09-21T21:41:16.948 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1974398333 2024-09-21T21:41:16.948 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.948 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.949 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:41:16.949 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6821", 2024-09-21T21:41:16.949 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1974398333 2024-09-21T21:41:16.949 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:16.949 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:16.949 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.949 INFO:teuthology.orchestra.run.smithi081.stdout: "public_addr": "172.21.15.122:6817/1974398333", 2024-09-21T21:41:16.949 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster_addr": "172.21.15.122:6819/1974398333", 2024-09-21T21:41:16.949 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_back_addr": "172.21.15.122:6823/1974398333", 2024-09-21T21:41:16.949 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_front_addr": "172.21.15.122:6821/1974398333", 2024-09-21T21:41:16.949 INFO:teuthology.orchestra.run.smithi081.stdout: "state": [ 2024-09-21T21:41:16.949 INFO:teuthology.orchestra.run.smithi081.stdout: "exists", 2024-09-21T21:41:16.949 INFO:teuthology.orchestra.run.smithi081.stdout: "up" 2024-09-21T21:41:16.949 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:16.950 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:16.950 INFO:teuthology.orchestra.run.smithi081.stdout: ], 2024-09-21T21:41:16.950 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_xinfo": [ 2024-09-21T21:41:16.950 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.950 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": 0, 2024-09-21T21:41:16.950 INFO:teuthology.orchestra.run.smithi081.stdout: "down_stamp": "0.000000", 2024-09-21T21:41:16.950 INFO:teuthology.orchestra.run.smithi081.stdout: "laggy_probability": 0, 2024-09-21T21:41:16.950 INFO:teuthology.orchestra.run.smithi081.stdout: "laggy_interval": 0, 2024-09-21T21:41:16.950 INFO:teuthology.orchestra.run.smithi081.stdout: "features": 4540701547738038271, 2024-09-21T21:41:16.950 INFO:teuthology.orchestra.run.smithi081.stdout: "old_weight": 0, 2024-09-21T21:41:16.950 INFO:teuthology.orchestra.run.smithi081.stdout: "last_purged_snaps_scrub": "2024-09-21T21:40:15.766648+0000", 2024-09-21T21:41:16.950 INFO:teuthology.orchestra.run.smithi081.stdout: "dead_epoch": 0 2024-09-21T21:41:16.950 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.950 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.951 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": 1, 2024-09-21T21:41:16.951 INFO:teuthology.orchestra.run.smithi081.stdout: "down_stamp": "0.000000", 2024-09-21T21:41:16.951 INFO:teuthology.orchestra.run.smithi081.stdout: "laggy_probability": 0, 2024-09-21T21:41:16.951 INFO:teuthology.orchestra.run.smithi081.stdout: "laggy_interval": 0, 2024-09-21T21:41:16.951 INFO:teuthology.orchestra.run.smithi081.stdout: "features": 4540701547738038271, 2024-09-21T21:41:16.951 INFO:teuthology.orchestra.run.smithi081.stdout: "old_weight": 0, 2024-09-21T21:41:16.951 INFO:teuthology.orchestra.run.smithi081.stdout: "last_purged_snaps_scrub": "2024-09-21T21:40:15.457496+0000", 2024-09-21T21:41:16.951 INFO:teuthology.orchestra.run.smithi081.stdout: "dead_epoch": 0 2024-09-21T21:41:16.951 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.951 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.951 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": 2, 2024-09-21T21:41:16.951 INFO:teuthology.orchestra.run.smithi081.stdout: "down_stamp": "0.000000", 2024-09-21T21:41:16.951 INFO:teuthology.orchestra.run.smithi081.stdout: "laggy_probability": 0, 2024-09-21T21:41:16.951 INFO:teuthology.orchestra.run.smithi081.stdout: "laggy_interval": 0, 2024-09-21T21:41:16.952 INFO:teuthology.orchestra.run.smithi081.stdout: "features": 4540701547738038271, 2024-09-21T21:41:16.952 INFO:teuthology.orchestra.run.smithi081.stdout: "old_weight": 0, 2024-09-21T21:41:16.952 INFO:teuthology.orchestra.run.smithi081.stdout: "last_purged_snaps_scrub": "2024-09-21T21:40:14.767039+0000", 2024-09-21T21:41:16.952 INFO:teuthology.orchestra.run.smithi081.stdout: "dead_epoch": 0 2024-09-21T21:41:16.952 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.952 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.952 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": 3, 2024-09-21T21:41:16.952 INFO:teuthology.orchestra.run.smithi081.stdout: "down_stamp": "0.000000", 2024-09-21T21:41:16.952 INFO:teuthology.orchestra.run.smithi081.stdout: "laggy_probability": 0, 2024-09-21T21:41:16.952 INFO:teuthology.orchestra.run.smithi081.stdout: "laggy_interval": 0, 2024-09-21T21:41:16.952 INFO:teuthology.orchestra.run.smithi081.stdout: "features": 4540701547738038271, 2024-09-21T21:41:16.952 INFO:teuthology.orchestra.run.smithi081.stdout: "old_weight": 0, 2024-09-21T21:41:16.952 INFO:teuthology.orchestra.run.smithi081.stdout: "last_purged_snaps_scrub": "2024-09-21T21:40:15.355570+0000", 2024-09-21T21:41:16.952 INFO:teuthology.orchestra.run.smithi081.stdout: "dead_epoch": 0 2024-09-21T21:41:16.953 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.953 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.953 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": 4, 2024-09-21T21:41:16.953 INFO:teuthology.orchestra.run.smithi081.stdout: "down_stamp": "0.000000", 2024-09-21T21:41:16.953 INFO:teuthology.orchestra.run.smithi081.stdout: "laggy_probability": 0, 2024-09-21T21:41:16.953 INFO:teuthology.orchestra.run.smithi081.stdout: "laggy_interval": 0, 2024-09-21T21:41:16.953 INFO:teuthology.orchestra.run.smithi081.stdout: "features": 4540701547738038271, 2024-09-21T21:41:16.953 INFO:teuthology.orchestra.run.smithi081.stdout: "old_weight": 0, 2024-09-21T21:41:16.953 INFO:teuthology.orchestra.run.smithi081.stdout: "last_purged_snaps_scrub": "2024-09-21T21:40:15.335729+0000", 2024-09-21T21:41:16.953 INFO:teuthology.orchestra.run.smithi081.stdout: "dead_epoch": 0 2024-09-21T21:41:16.953 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.953 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.953 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": 5, 2024-09-21T21:41:16.953 INFO:teuthology.orchestra.run.smithi081.stdout: "down_stamp": "0.000000", 2024-09-21T21:41:16.953 INFO:teuthology.orchestra.run.smithi081.stdout: "laggy_probability": 0, 2024-09-21T21:41:16.954 INFO:teuthology.orchestra.run.smithi081.stdout: "laggy_interval": 0, 2024-09-21T21:41:16.954 INFO:teuthology.orchestra.run.smithi081.stdout: "features": 4540701547738038271, 2024-09-21T21:41:16.954 INFO:teuthology.orchestra.run.smithi081.stdout: "old_weight": 0, 2024-09-21T21:41:16.954 INFO:teuthology.orchestra.run.smithi081.stdout: "last_purged_snaps_scrub": "2024-09-21T21:40:15.614733+0000", 2024-09-21T21:41:16.954 INFO:teuthology.orchestra.run.smithi081.stdout: "dead_epoch": 0 2024-09-21T21:41:16.954 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:16.954 INFO:teuthology.orchestra.run.smithi081.stdout: ], 2024-09-21T21:41:16.954 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_upmap": [ 2024-09-21T21:41:16.954 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:16.954 INFO:teuthology.orchestra.run.smithi081.stdout: "pgid": "3.8", 2024-09-21T21:41:16.954 INFO:teuthology.orchestra.run.smithi081.stdout: "osds": [ 2024-09-21T21:41:16.954 INFO:teuthology.orchestra.run.smithi081.stdout: 2, 2024-09-21T21:41:16.954 INFO:teuthology.orchestra.run.smithi081.stdout: 3 2024-09-21T21:41:16.954 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:16.955 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:16.955 INFO:teuthology.orchestra.run.smithi081.stdout: ], 2024-09-21T21:41:16.955 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_upmap_items": [], 2024-09-21T21:41:16.955 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_upmap_primaries": [], 2024-09-21T21:41:16.955 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_temp": [], 2024-09-21T21:41:16.955 INFO:teuthology.orchestra.run.smithi081.stdout: "primary_temp": [], 2024-09-21T21:41:16.955 INFO:teuthology.orchestra.run.smithi081.stdout: "blocklist": {}, 2024-09-21T21:41:16.955 INFO:teuthology.orchestra.run.smithi081.stdout: "range_blocklist": {}, 2024-09-21T21:41:16.955 INFO:teuthology.orchestra.run.smithi081.stdout: "erasure_code_profiles": { 2024-09-21T21:41:16.955 INFO:teuthology.orchestra.run.smithi081.stdout: "default": { 2024-09-21T21:41:16.955 INFO:teuthology.orchestra.run.smithi081.stdout: "crush-failure-domain": "osd", 2024-09-21T21:41:16.955 INFO:teuthology.orchestra.run.smithi081.stdout: "k": "2", 2024-09-21T21:41:16.955 INFO:teuthology.orchestra.run.smithi081.stdout: "m": "1", 2024-09-21T21:41:16.956 INFO:teuthology.orchestra.run.smithi081.stdout: "plugin": "jerasure", 2024-09-21T21:41:16.956 INFO:teuthology.orchestra.run.smithi081.stdout: "technique": "reed_sol_van" 2024-09-21T21:41:16.956 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:16.956 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:16.956 INFO:teuthology.orchestra.run.smithi081.stdout: "removed_snaps_queue": [], 2024-09-21T21:41:16.956 INFO:teuthology.orchestra.run.smithi081.stdout: "new_removed_snaps": [], 2024-09-21T21:41:16.956 INFO:teuthology.orchestra.run.smithi081.stdout: "new_purged_snaps": [], 2024-09-21T21:41:16.956 INFO:teuthology.orchestra.run.smithi081.stdout: "crush_node_flags": {}, 2024-09-21T21:41:16.956 INFO:teuthology.orchestra.run.smithi081.stdout: "device_class_flags": {}, 2024-09-21T21:41:16.956 INFO:teuthology.orchestra.run.smithi081.stdout: "stretch_mode": { 2024-09-21T21:41:16.956 INFO:teuthology.orchestra.run.smithi081.stdout: "stretch_mode_enabled": false, 2024-09-21T21:41:16.956 INFO:teuthology.orchestra.run.smithi081.stdout: "stretch_bucket_count": 0, 2024-09-21T21:41:16.956 INFO:teuthology.orchestra.run.smithi081.stdout: "degraded_stretch_mode": 0, 2024-09-21T21:41:16.957 INFO:teuthology.orchestra.run.smithi081.stdout: "recovering_stretch_mode": 0, 2024-09-21T21:41:16.957 INFO:teuthology.orchestra.run.smithi081.stdout: "stretch_mode_bucket": 0 2024-09-21T21:41:16.957 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:16.957 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:16.958 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:16.958 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:16.959 INFO:tasks.thrashosds.thrasher:j is {'epoch': 48, 'fsid': 'a0fdcfc2-ddd2-4e50-81bb-876d940f3578', 'created': '2024-09-21T21:40:07.413917+0000', 'modified': '2024-09-21T21:41:15.589007+0000', 'last_up_change': '2024-09-21T21:40:18.340430+0000', 'last_in_change': '2024-09-21T21:40:09.029953+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T21:40:18.321328+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.888008177280426, 'average_primary_affinity': 0.8899999856948853, 'average_primary_affinity_weighted': 0.8899999856948853}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-21T21:40:24.235658+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 19, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.888008177280426, 'average_primary_affinity': 0.8899999856948853, 'average_primary_affinity_weighted': 0.8899999856948853}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-21T21:40:36.093691+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '47', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 16, 'snap_epoch': 47, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.888008177280426, 'average_primary_affinity': 0.8899999856948853, 'average_primary_affinity_weighted': 0.8899999856948853}}], 'osds': [{'osd': 0, 'uuid': 'a9869830-8e74-4b02-963d-ab630a0ee286', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 40, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6808', 'nonce': 1657552338}, {'type': 'v1', 'addr': '172.21.15.81:6809', 'nonce': 1657552338}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6810', 'nonce': 1657552338}, {'type': 'v1', 'addr': '172.21.15.81:6811', 'nonce': 1657552338}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6814', 'nonce': 1657552338}, {'type': 'v1', 'addr': '172.21.15.81:6815', 'nonce': 1657552338}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6812', 'nonce': 1657552338}, {'type': 'v1', 'addr': '172.21.15.81:6813', 'nonce': 1657552338}]}, 'public_addr': '172.21.15.81:6809/1657552338', 'cluster_addr': '172.21.15.81:6811/1657552338', 'heartbeat_back_addr': '172.21.15.81:6815/1657552338', 'heartbeat_front_addr': '172.21.15.81:6813/1657552338', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'b189819d-69a3-4e95-96e7-96dc4809ac80', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 40, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6816', 'nonce': 2864159502}, {'type': 'v1', 'addr': '172.21.15.81:6817', 'nonce': 2864159502}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6818', 'nonce': 2864159502}, {'type': 'v1', 'addr': '172.21.15.81:6819', 'nonce': 2864159502}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6822', 'nonce': 2864159502}, {'type': 'v1', 'addr': '172.21.15.81:6823', 'nonce': 2864159502}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6820', 'nonce': 2864159502}, {'type': 'v1', 'addr': '172.21.15.81:6821', 'nonce': 2864159502}]}, 'public_addr': '172.21.15.81:6817/2864159502', 'cluster_addr': '172.21.15.81:6819/2864159502', 'heartbeat_back_addr': '172.21.15.81:6823/2864159502', 'heartbeat_front_addr': '172.21.15.81:6821/2864159502', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '8437b99a-1297-4903-ab66-95c5cba2b2bf', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 45, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6800', 'nonce': 3023160932}, {'type': 'v1', 'addr': '172.21.15.81:6801', 'nonce': 3023160932}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6802', 'nonce': 3023160932}, {'type': 'v1', 'addr': '172.21.15.81:6803', 'nonce': 3023160932}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6806', 'nonce': 3023160932}, {'type': 'v1', 'addr': '172.21.15.81:6807', 'nonce': 3023160932}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6804', 'nonce': 3023160932}, {'type': 'v1', 'addr': '172.21.15.81:6805', 'nonce': 3023160932}]}, 'public_addr': '172.21.15.81:6801/3023160932', 'cluster_addr': '172.21.15.81:6803/3023160932', 'heartbeat_back_addr': '172.21.15.81:6807/3023160932', 'heartbeat_front_addr': '172.21.15.81:6805/3023160932', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'e8848768-c1aa-445b-9554-dc6aa4eefc8d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 21, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6808', 'nonce': 2681016844}, {'type': 'v1', 'addr': '172.21.15.122:6809', 'nonce': 2681016844}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6810', 'nonce': 2681016844}, {'type': 'v1', 'addr': '172.21.15.122:6811', 'nonce': 2681016844}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6814', 'nonce': 2681016844}, {'type': 'v1', 'addr': '172.21.15.122:6815', 'nonce': 2681016844}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6812', 'nonce': 2681016844}, {'type': 'v1', 'addr': '172.21.15.122:6813', 'nonce': 2681016844}]}, 'public_addr': '172.21.15.122:6809/2681016844', 'cluster_addr': '172.21.15.122:6811/2681016844', 'heartbeat_back_addr': '172.21.15.122:6815/2681016844', 'heartbeat_front_addr': '172.21.15.122:6813/2681016844', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'b1d8e85d-a207-4ba6-9322-61d07812083e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 40, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6800', 'nonce': 1095023540}, {'type': 'v1', 'addr': '172.21.15.122:6801', 'nonce': 1095023540}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6802', 'nonce': 1095023540}, {'type': 'v1', 'addr': '172.21.15.122:6803', 'nonce': 1095023540}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6806', 'nonce': 1095023540}, {'type': 'v1', 'addr': '172.21.15.122:6807', 'nonce': 1095023540}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6804', 'nonce': 1095023540}, {'type': 'v1', 'addr': '172.21.15.122:6805', 'nonce': 1095023540}]}, 'public_addr': '172.21.15.122:6801/1095023540', 'cluster_addr': '172.21.15.122:6803/1095023540', 'heartbeat_back_addr': '172.21.15.122:6807/1095023540', 'heartbeat_front_addr': '172.21.15.122:6805/1095023540', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '3ab0450b-d8fd-4ef2-a298-08d5aa2ea32f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.3280487060546875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 21, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6816', 'nonce': 1974398333}, {'type': 'v1', 'addr': '172.21.15.122:6817', 'nonce': 1974398333}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6818', 'nonce': 1974398333}, {'type': 'v1', 'addr': '172.21.15.122:6819', 'nonce': 1974398333}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6822', 'nonce': 1974398333}, {'type': 'v1', 'addr': '172.21.15.122:6823', 'nonce': 1974398333}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6820', 'nonce': 1974398333}, {'type': 'v1', 'addr': '172.21.15.122:6821', 'nonce': 1974398333}]}, 'public_addr': '172.21.15.122:6817/1974398333', 'cluster_addr': '172.21.15.122:6819/1974398333', 'heartbeat_back_addr': '172.21.15.122:6823/1974398333', 'heartbeat_front_addr': '172.21.15.122:6821/1974398333', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T21:40:15.766648+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T21:40:15.457496+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T21:40:14.767039+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T21:40:15.355570+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T21:40:15.335729+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T21:40:15.614733+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.8', 'osds': [2, 3]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-21T21:41:16.960 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T21:41:16.960 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T21:41:16.960 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:16.957+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:17.058 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:17.057+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:17.101 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:17.102 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:17.110 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T21:41:17.159 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:17.158+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:17.213 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:17.213 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:17.221 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T21:41:17.260 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:17.259+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:17.277 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:41:17.278 INFO:teuthology.orchestra.run.smithi081.stderr:dumped all 2024-09-21T21:41:17.289 INFO:teuthology.orchestra.run.smithi081.stdout:{"pg_ready":true,"pg_map":{"version":73,"stamp":"2024-09-21T21:41:16.300396+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":372273353,"num_objects":514,"num_object_clones":59,"num_object_copies":1028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":514,"num_whiteouts":132,"num_read":111469,"num_read_kb":447739,"num_write":7488,"num_write_kb":91127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6278,"ondisk_log_size":6278,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":230628,"kb_used_data":18544,"kb_used_omap":9,"kb_used_meta":211958,"kb_avail":566000412,"statfs":{"total":579820584960,"available":579584421888,"internally_reserved":0,"allocated":18989056,"data_stored":97128436,"data_compressed":7656520,"data_compressed_allocated":8572928,"data_compressed_original":90035200,"omap_allocated":9568,"internal_metadata":217045664},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[7,2],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":16441447,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":24875,"num_read_kb":48505,"num_write":1719,"num_write_kb":6260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.719152"},"pg_stats":[{"pgid":"3.1f","version":"46'146","reported_seq":1824,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T21:41:13.296916+0000","last_change":"2024-09-21T21:40:39.430005+0000","last_active":"2024-09-21T21:41:13.296916+0000","last_peered":"2024-09-21T21:41:13.296916+0000","last_clean":"2024-09-21T21:41:13.296916+0000","last_became_active":"2024-09-21T21:40:37.424213+0000","last_became_peered":"2024-09-21T21:40:37.424213+0000","last_unstale":"2024-09-21T21:41:13.296916+0000","last_undegraded":"2024-09-21T21:41:13.296916+0000","last_fullsized":"2024-09-21T21:41:13.296916+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:30:14.243340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010985019999999999,"stat_sum":{"num_bytes":10981952,"num_objects":16,"num_object_clones":2,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":3,"num_read":2405,"num_read_kb":8678,"num_write":129,"num_write_kb":4508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"44'252","reported_seq":3465,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T21:41:10.812969+0000","last_change":"2024-09-21T21:40:39.416688+0000","last_active":"2024-09-21T21:41:10.812969+0000","last_peered":"2024-09-21T21:41:10.812969+0000","last_clean":"2024-09-21T21:41:10.812969+0000","last_became_active":"2024-09-21T21:40:37.417190+0000","last_became_peered":"2024-09-21T21:40:37.417190+0000","last_unstale":"2024-09-21T21:41:10.812969+0000","last_undegraded":"2024-09-21T21:41:10.812969+0000","last_fullsized":"2024-09-21T21:41:10.812969+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":252,"log_dups_size":0,"ondisk_log_size":252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:31:12.460164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042608099999999997,"stat_sum":{"num_bytes":11469867,"num_objects":18,"num_object_clones":4,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":7,"num_read":4003,"num_read_kb":36982,"num_write":378,"num_write_kb":7005,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"46'140","reported_seq":1915,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T21:41:14.239924+0000","last_change":"2024-09-21T21:41:05.551822+0000","last_active":"2024-09-21T21:41:14.239924+0000","last_peered":"2024-09-21T21:41:14.239924+0000","last_clean":"2024-09-21T21:41:14.239924+0000","last_became_active":"2024-09-21T21:41:05.551289+0000","last_became_peered":"2024-09-21T21:41:05.551289+0000","last_unstale":"2024-09-21T21:41:14.239924+0000","last_undegraded":"2024-09-21T21:41:14.239924+0000","last_fullsized":"2024-09-21T21:41:14.239924+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":140,"log_dups_size":0,"ondisk_log_size":140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:27:17.333944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001986561,"stat_sum":{"num_bytes":3209246,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":4,"num_read":1888,"num_read_kb":5835,"num_write":223,"num_write_kb":1168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"46'489","reported_seq":5734,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T21:41:14.089427+0000","last_change":"2024-09-21T21:41:05.551891+0000","last_active":"2024-09-21T21:41:14.089427+0000","last_peered":"2024-09-21T21:41:14.089427+0000","last_clean":"2024-09-21T21:41:14.089427+0000","last_became_active":"2024-09-21T21:41:05.551346+0000","last_became_peered":"2024-09-21T21:41:05.551346+0000","last_unstale":"2024-09-21T21:41:14.089427+0000","last_undegraded":"2024-09-21T21:41:14.089427+0000","last_fullsized":"2024-09-21T21:41:14.089427+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":489,"log_dups_size":0,"ondisk_log_size":489,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:04:09.093423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011105570000000001,"stat_sum":{"num_bytes":5188133,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":2,"num_read":8588,"num_read_kb":11278,"num_write":490,"num_write_kb":1513,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"46'103","reported_seq":1294,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T21:41:13.232624+0000","last_change":"2024-09-21T21:40:39.429478+0000","last_active":"2024-09-21T21:41:13.232624+0000","last_peered":"2024-09-21T21:41:13.232624+0000","last_clean":"2024-09-21T21:41:13.232624+0000","last_became_active":"2024-09-21T21:40:37.425925+0000","last_became_peered":"2024-09-21T21:40:37.425925+0000","last_unstale":"2024-09-21T21:41:13.232624+0000","last_undegraded":"2024-09-21T21:41:13.232624+0000","last_fullsized":"2024-09-21T21:41:13.232624+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:32:10.645024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069216500000000001,"stat_sum":{"num_bytes":5922346,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":6,"num_read":1145,"num_read_kb":3357,"num_write":106,"num_write_kb":1629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"46'86","reported_seq":5281,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T21:41:14.300433+0000","last_change":"2024-09-21T21:40:39.420213+0000","last_active":"2024-09-21T21:41:14.300433+0000","last_peered":"2024-09-21T21:41:14.300433+0000","last_clean":"2024-09-21T21:41:14.300433+0000","last_became_active":"2024-09-21T21:40:37.424316+0000","last_became_peered":"2024-09-21T21:40:37.424316+0000","last_unstale":"2024-09-21T21:41:14.300433+0000","last_undegraded":"2024-09-21T21:41:14.300433+0000","last_fullsized":"2024-09-21T21:41:14.300433+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:08:40.702506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095031900000000003,"stat_sum":{"num_bytes":2826323,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":4,"num_read":5168,"num_read_kb":11684,"num_write":85,"num_write_kb":376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"46'326","reported_seq":4394,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T21:41:13.228165+0000","last_change":"2024-09-21T21:40:39.414575+0000","last_active":"2024-09-21T21:41:13.228165+0000","last_peered":"2024-09-21T21:41:13.228165+0000","last_clean":"2024-09-21T21:41:13.228165+0000","last_became_active":"2024-09-21T21:40:37.426715+0000","last_became_peered":"2024-09-21T21:40:37.426715+0000","last_unstale":"2024-09-21T21:41:13.228165+0000","last_undegraded":"2024-09-21T21:41:13.228165+0000","last_fullsized":"2024-09-21T21:41:13.228165+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":326,"log_dups_size":0,"ondisk_log_size":326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:13:32.429037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066263800000000001,"stat_sum":{"num_bytes":17152103,"num_objects":25,"num_object_clones":1,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":5,"num_read":5913,"num_read_kb":17789,"num_write":285,"num_write_kb":4028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"44'65","reported_seq":1624,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T21:41:10.736917+0000","last_change":"2024-09-21T21:40:39.416634+0000","last_active":"2024-09-21T21:41:10.736917+0000","last_peered":"2024-09-21T21:41:10.736917+0000","last_clean":"2024-09-21T21:41:10.736917+0000","last_became_active":"2024-09-21T21:40:37.421603+0000","last_became_peered":"2024-09-21T21:40:37.421603+0000","last_unstale":"2024-09-21T21:41:10.736917+0000","last_undegraded":"2024-09-21T21:41:10.736917+0000","last_fullsized":"2024-09-21T21:41:10.736917+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:11:25.646408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042576799999999998,"stat_sum":{"num_bytes":2421760,"num_objects":11,"num_object_clones":1,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":3,"num_read":1503,"num_read_kb":5683,"num_write":70,"num_write_kb":720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"46'122","reported_seq":2079,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T21:41:13.249732+0000","last_change":"2024-09-21T21:40:39.430190+0000","last_active":"2024-09-21T21:41:13.249732+0000","last_peered":"2024-09-21T21:41:13.249732+0000","last_clean":"2024-09-21T21:41:13.249732+0000","last_became_active":"2024-09-21T21:40:37.424911+0000","last_became_peered":"2024-09-21T21:40:37.424911+0000","last_unstale":"2024-09-21T21:41:13.249732+0000","last_undegraded":"2024-09-21T21:41:13.249732+0000","last_fullsized":"2024-09-21T21:41:13.249732+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:59:54.144300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001258211,"stat_sum":{"num_bytes":15475761,"num_objects":22,"num_object_clones":3,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":7,"num_read":2620,"num_read_kb":5584,"num_write":86,"num_write_kb":365,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"46'141","reported_seq":3845,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T21:41:13.042268+0000","last_change":"2024-09-21T21:40:39.425366+0000","last_active":"2024-09-21T21:41:13.042268+0000","last_peered":"2024-09-21T21:41:13.042268+0000","last_clean":"2024-09-21T21:41:13.042268+0000","last_became_active":"2024-09-21T21:40:37.424364+0000","last_became_peered":"2024-09-21T21:40:37.424364+0000","last_unstale":"2024-09-21T21:41:13.042268+0000","last_undegraded":"2024-09-21T21:41:13.042268+0000","last_fullsized":"2024-09-21T21:41:13.042268+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:21:57.095889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015121799999999999,"stat_sum":{"num_bytes":22993958,"num_objects":17,"num_object_clones":2,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":2,"num_read":4394,"num_read_kb":17757,"num_write":114,"num_write_kb":1235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"46'107","reported_seq":1786,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T21:41:14.100607+0000","last_change":"2024-09-21T21:40:39.415411+0000","last_active":"2024-09-21T21:41:14.100607+0000","last_peered":"2024-09-21T21:41:14.100607+0000","last_clean":"2024-09-21T21:41:14.100607+0000","last_became_active":"2024-09-21T21:40:37.421701+0000","last_became_peered":"2024-09-21T21:40:37.421701+0000","last_unstale":"2024-09-21T21:41:14.100607+0000","last_undegraded":"2024-09-21T21:41:14.100607+0000","last_fullsized":"2024-09-21T21:41:14.100607+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:03:36.118399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001006961,"stat_sum":{"num_bytes":7777792,"num_objects":11,"num_object_clones":1,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":3,"num_read":1628,"num_read_kb":5594,"num_write":104,"num_write_kb":452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"46'137","reported_seq":2769,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T21:41:14.097543+0000","last_change":"2024-09-21T21:40:39.415100+0000","last_active":"2024-09-21T21:41:14.097543+0000","last_peered":"2024-09-21T21:41:14.097543+0000","last_clean":"2024-09-21T21:41:14.097543+0000","last_became_active":"2024-09-21T21:40:37.426062+0000","last_became_peered":"2024-09-21T21:40:37.426062+0000","last_unstale":"2024-09-21T21:41:14.097543+0000","last_undegraded":"2024-09-21T21:41:14.097543+0000","last_fullsized":"2024-09-21T21:41:14.097543+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:37:46.549601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093907599999999995,"stat_sum":{"num_bytes":14954623,"num_objects":15,"num_object_clones":2,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":4,"num_read":2595,"num_read_kb":15169,"num_write":153,"num_write_kb":4673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":71,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T21:41:12.876055+0000","last_change":"2024-09-21T21:40:27.380563+0000","last_active":"2024-09-21T21:41:12.876055+0000","last_peered":"2024-09-21T21:41:12.876055+0000","last_clean":"2024-09-21T21:41:12.876055+0000","last_became_active":"2024-09-21T21:40:25.501200+0000","last_became_peered":"2024-09-21T21:40:25.501200+0000","last_unstale":"2024-09-21T21:41:12.876055+0000","last_undegraded":"2024-09-21T21:41:12.876055+0000","last_fullsized":"2024-09-21T21:41:12.876055+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:03:24.900063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072643899999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"46'126","reported_seq":3446,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T21:41:13.274004+0000","last_change":"2024-09-21T21:40:39.430054+0000","last_active":"2024-09-21T21:41:13.274004+0000","last_peered":"2024-09-21T21:41:13.274004+0000","last_clean":"2024-09-21T21:41:13.274004+0000","last_became_active":"2024-09-21T21:40:37.422487+0000","last_became_peered":"2024-09-21T21:40:37.422487+0000","last_unstale":"2024-09-21T21:41:13.274004+0000","last_undegraded":"2024-09-21T21:41:13.274004+0000","last_fullsized":"2024-09-21T21:41:13.274004+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:04:58.698277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011470160000000001,"stat_sum":{"num_bytes":18912768,"num_objects":12,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":2,"num_read":3920,"num_read_kb":29895,"num_write":123,"num_write_kb":2437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T21:41:12.587235+0000","last_change":"2024-09-21T21:40:27.379903+0000","last_active":"2024-09-21T21:41:12.587235+0000","last_peered":"2024-09-21T21:41:12.587235+0000","last_clean":"2024-09-21T21:41:12.587235+0000","last_became_active":"2024-09-21T21:40:25.371095+0000","last_became_peered":"2024-09-21T21:40:25.371095+0000","last_unstale":"2024-09-21T21:41:12.587235+0000","last_undegraded":"2024-09-21T21:41:12.587235+0000","last_fullsized":"2024-09-21T21:41:12.587235+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:08:09.434308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062280799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"44'90","reported_seq":1156,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T21:41:10.102963+0000","last_change":"2024-09-21T21:40:39.416965+0000","last_active":"2024-09-21T21:41:10.102963+0000","last_peered":"2024-09-21T21:41:10.102963+0000","last_clean":"2024-09-21T21:41:10.102963+0000","last_became_active":"2024-09-21T21:40:37.418428+0000","last_became_peered":"2024-09-21T21:40:37.418428+0000","last_unstale":"2024-09-21T21:41:10.102963+0000","last_undegraded":"2024-09-21T21:41:10.102963+0000","last_fullsized":"2024-09-21T21:41:10.102963+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:25:19.430654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066113800000000003,"stat_sum":{"num_bytes":13553664,"num_objects":8,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":4,"num_read":1009,"num_read_kb":9620,"num_write":93,"num_write_kb":976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":71,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T21:41:12.876139+0000","last_change":"2024-09-21T21:40:27.380218+0000","last_active":"2024-09-21T21:41:12.876139+0000","last_peered":"2024-09-21T21:41:12.876139+0000","last_clean":"2024-09-21T21:41:12.876139+0000","last_became_active":"2024-09-21T21:40:25.500143+0000","last_became_peered":"2024-09-21T21:40:25.500143+0000","last_unstale":"2024-09-21T21:41:12.876139+0000","last_undegraded":"2024-09-21T21:41:12.876139+0000","last_fullsized":"2024-09-21T21:41:12.876139+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:11:42.747388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042715899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"46'508","reported_seq":1406,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T21:41:13.281880+0000","last_change":"2024-09-21T21:40:39.414839+0000","last_active":"2024-09-21T21:41:13.281880+0000","last_peered":"2024-09-21T21:41:13.281880+0000","last_clean":"2024-09-21T21:41:13.281880+0000","last_became_active":"2024-09-21T21:40:37.419944+0000","last_became_peered":"2024-09-21T21:40:37.419944+0000","last_unstale":"2024-09-21T21:41:13.281880+0000","last_undegraded":"2024-09-21T21:41:13.281880+0000","last_fullsized":"2024-09-21T21:41:13.281880+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":508,"log_dups_size":0,"ondisk_log_size":508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:52:14.622878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085769100000000005,"stat_sum":{"num_bytes":4725421,"num_objects":17,"num_object_clones":2,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":4,"num_read":3128,"num_read_kb":8182,"num_write":857,"num_write_kb":2108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T21:41:12.587483+0000","last_change":"2024-09-21T21:40:27.380005+0000","last_active":"2024-09-21T21:41:12.587483+0000","last_peered":"2024-09-21T21:41:12.587483+0000","last_clean":"2024-09-21T21:41:12.587483+0000","last_became_active":"2024-09-21T21:40:25.500109+0000","last_became_peered":"2024-09-21T21:40:25.500109+0000","last_unstale":"2024-09-21T21:41:12.587483+0000","last_undegraded":"2024-09-21T21:41:12.587483+0000","last_fullsized":"2024-09-21T21:41:12.587483+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:21:10.631127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070669899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"44'66","reported_seq":899,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T21:41:10.759989+0000","last_change":"2024-09-21T21:41:05.549770+0000","last_active":"2024-09-21T21:41:10.759989+0000","last_peered":"2024-09-21T21:41:10.759989+0000","last_clean":"2024-09-21T21:41:10.759989+0000","last_became_active":"2024-09-21T21:41:05.549265+0000","last_became_peered":"2024-09-21T21:41:05.549265+0000","last_unstale":"2024-09-21T21:41:10.759989+0000","last_undegraded":"2024-09-21T21:41:10.759989+0000","last_fullsized":"2024-09-21T21:41:10.759989+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:27:26.314512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010561050000000001,"stat_sum":{"num_bytes":5076000,"num_objects":15,"num_object_clones":1,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":6,"num_read":787,"num_read_kb":9244,"num_write":66,"num_write_kb":637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"46'104","reported_seq":3740,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T21:41:14.088573+0000","last_change":"2024-09-21T21:40:39.419600+0000","last_active":"2024-09-21T21:41:14.088573+0000","last_peered":"2024-09-21T21:41:14.088573+0000","last_clean":"2024-09-21T21:41:14.088573+0000","last_became_active":"2024-09-21T21:40:37.424903+0000","last_became_peered":"2024-09-21T21:40:37.424903+0000","last_unstale":"2024-09-21T21:41:14.088573+0000","last_undegraded":"2024-09-21T21:41:14.088573+0000","last_fullsized":"2024-09-21T21:41:14.088573+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:01:24.476665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041679500000000002,"stat_sum":{"num_bytes":4535296,"num_objects":11,"num_object_clones":2,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":3,"num_read":3595,"num_read_kb":13655,"num_write":150,"num_write_kb":3770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":38,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T21:41:12.587978+0000","last_change":"2024-09-21T21:41:05.549766+0000","last_active":"2024-09-21T21:41:12.587978+0000","last_peered":"2024-09-21T21:41:12.587978+0000","last_clean":"2024-09-21T21:41:12.587978+0000","last_became_active":"2024-09-21T21:41:05.549439+0000","last_became_peered":"2024-09-21T21:41:05.549439+0000","last_unstale":"2024-09-21T21:41:12.587978+0000","last_undegraded":"2024-09-21T21:41:12.587978+0000","last_fullsized":"2024-09-21T21:41:12.587978+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:49:30.301108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"46'393","reported_seq":6474,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T21:41:13.302983+0000","last_change":"2024-09-21T21:40:39.414972+0000","last_active":"2024-09-21T21:41:13.302983+0000","last_peered":"2024-09-21T21:41:13.302983+0000","last_clean":"2024-09-21T21:41:13.302983+0000","last_became_active":"2024-09-21T21:40:37.418973+0000","last_became_peered":"2024-09-21T21:40:37.418973+0000","last_unstale":"2024-09-21T21:41:13.302983+0000","last_undegraded":"2024-09-21T21:41:13.302983+0000","last_fullsized":"2024-09-21T21:41:13.302983+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":393,"log_dups_size":0,"ondisk_log_size":393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:30:04.600359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001017923,"stat_sum":{"num_bytes":22428192,"num_objects":23,"num_object_clones":5,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":4,"num_read":8872,"num_read_kb":16391,"num_write":239,"num_write_kb":1227,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":67,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T21:41:09.583931+0000","last_change":"2024-09-21T21:40:27.380792+0000","last_active":"2024-09-21T21:41:09.583931+0000","last_peered":"2024-09-21T21:41:09.583931+0000","last_clean":"2024-09-21T21:41:09.583931+0000","last_became_active":"2024-09-21T21:40:25.371578+0000","last_became_peered":"2024-09-21T21:40:25.371578+0000","last_unstale":"2024-09-21T21:41:09.583931+0000","last_undegraded":"2024-09-21T21:41:09.583931+0000","last_fullsized":"2024-09-21T21:41:09.583931+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:16:44.813324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000459858,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"46'62","reported_seq":945,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T21:41:13.261141+0000","last_change":"2024-09-21T21:40:39.429211+0000","last_active":"2024-09-21T21:41:13.261141+0000","last_peered":"2024-09-21T21:41:13.261141+0000","last_clean":"2024-09-21T21:41:13.261141+0000","last_became_active":"2024-09-21T21:40:37.428542+0000","last_became_peered":"2024-09-21T21:40:37.428542+0000","last_unstale":"2024-09-21T21:41:13.261141+0000","last_undegraded":"2024-09-21T21:41:13.261141+0000","last_fullsized":"2024-09-21T21:41:13.261141+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:45:51.552117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045982299999999999,"stat_sum":{"num_bytes":3511851,"num_objects":11,"num_object_clones":1,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":5,"num_read":833,"num_read_kb":5021,"num_write":60,"num_write_kb":216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":71,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T21:41:12.877295+0000","last_change":"2024-09-21T21:40:27.380219+0000","last_active":"2024-09-21T21:41:12.877295+0000","last_peered":"2024-09-21T21:41:12.877295+0000","last_clean":"2024-09-21T21:41:12.877295+0000","last_became_active":"2024-09-21T21:40:25.371451+0000","last_became_peered":"2024-09-21T21:40:25.371451+0000","last_unstale":"2024-09-21T21:41:12.877295+0000","last_undegraded":"2024-09-21T21:41:12.877295+0000","last_fullsized":"2024-09-21T21:41:12.877295+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:05:41.445313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000438863,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"46'239","reported_seq":2280,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T21:41:13.264317+0000","last_change":"2024-09-21T21:40:39.415337+0000","last_active":"2024-09-21T21:41:13.264317+0000","last_peered":"2024-09-21T21:41:13.264317+0000","last_clean":"2024-09-21T21:41:13.264317+0000","last_became_active":"2024-09-21T21:40:37.421655+0000","last_became_peered":"2024-09-21T21:40:37.421655+0000","last_unstale":"2024-09-21T21:41:13.264317+0000","last_undegraded":"2024-09-21T21:41:13.264317+0000","last_fullsized":"2024-09-21T21:41:13.264317+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":239,"log_dups_size":0,"ondisk_log_size":239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:40:46.561266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000835322,"stat_sum":{"num_bytes":20855861,"num_objects":19,"num_object_clones":1,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":7,"num_read":2529,"num_read_kb":13387,"num_write":316,"num_write_kb":3153,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":24,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T21:41:09.584022+0000","last_change":"2024-09-21T21:41:05.548635+0000","last_active":"2024-09-21T21:41:09.584022+0000","last_peered":"2024-09-21T21:41:09.584022+0000","last_clean":"2024-09-21T21:41:09.584022+0000","last_became_active":"2024-09-21T21:41:05.548420+0000","last_became_peered":"2024-09-21T21:41:05.548420+0000","last_unstale":"2024-09-21T21:41:09.584022+0000","last_undegraded":"2024-09-21T21:41:09.584022+0000","last_fullsized":"2024-09-21T21:41:09.584022+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:36:23.320735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":223,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T21:41:12.877320+0000","last_change":"2024-09-21T21:40:19.352502+0000","last_active":"2024-09-21T21:41:12.877320+0000","last_peered":"2024-09-21T21:41:12.877320+0000","last_clean":"2024-09-21T21:41:12.877320+0000","last_became_active":"2024-09-21T21:40:19.352227+0000","last_became_peered":"2024-09-21T21:40:19.352227+0000","last_unstale":"2024-09-21T21:41:12.877320+0000","last_undegraded":"2024-09-21T21:41:12.877320+0000","last_fullsized":"2024-09-21T21:41:12.877320+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:18.340430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:18.340430+0000","last_clean_scrub_stamp":"2024-09-21T21:40:18.340430+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:19:13.902196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"44'97","reported_seq":2463,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T21:41:10.659350+0000","last_change":"2024-09-21T21:40:39.416961+0000","last_active":"2024-09-21T21:41:10.659350+0000","last_peered":"2024-09-21T21:41:10.659350+0000","last_clean":"2024-09-21T21:41:10.659350+0000","last_became_active":"2024-09-21T21:40:37.418416+0000","last_became_peered":"2024-09-21T21:40:37.418416+0000","last_unstale":"2024-09-21T21:41:10.659350+0000","last_undegraded":"2024-09-21T21:41:10.659350+0000","last_fullsized":"2024-09-21T21:41:10.659350+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:14:32.383386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056255200000000002,"stat_sum":{"num_bytes":10902073,"num_objects":17,"num_object_clones":3,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":5,"num_read":2398,"num_read_kb":9935,"num_write":100,"num_write_kb":1846,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"46'133","reported_seq":3035,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T21:41:14.278857+0000","last_change":"2024-09-21T21:40:39.419805+0000","last_active":"2024-09-21T21:41:14.278857+0000","last_peered":"2024-09-21T21:41:14.278857+0000","last_clean":"2024-09-21T21:41:14.278857+0000","last_became_active":"2024-09-21T21:40:37.425495+0000","last_became_peered":"2024-09-21T21:40:37.425495+0000","last_unstale":"2024-09-21T21:41:14.278857+0000","last_undegraded":"2024-09-21T21:41:14.278857+0000","last_fullsized":"2024-09-21T21:41:14.278857+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:33:45.924005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035788200000000001,"stat_sum":{"num_bytes":19089495,"num_objects":24,"num_object_clones":4,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":9,"num_read":2873,"num_read_kb":11705,"num_write":146,"num_write_kb":2393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"46'282","reported_seq":1873,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T21:41:14.278832+0000","last_change":"2024-09-21T21:40:39.414797+0000","last_active":"2024-09-21T21:41:14.278832+0000","last_peered":"2024-09-21T21:41:14.278832+0000","last_clean":"2024-09-21T21:41:14.278832+0000","last_became_active":"2024-09-21T21:40:37.426458+0000","last_became_peered":"2024-09-21T21:40:37.426458+0000","last_unstale":"2024-09-21T21:41:14.278832+0000","last_undegraded":"2024-09-21T21:41:14.278832+0000","last_fullsized":"2024-09-21T21:41:14.278832+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:47:50.121347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065146799999999999,"stat_sum":{"num_bytes":4006542,"num_objects":15,"num_object_clones":1,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":3,"num_read":2388,"num_read_kb":7386,"num_write":426,"num_write_kb":2873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"46'436","reported_seq":1448,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T21:41:13.279952+0000","last_change":"2024-09-21T21:40:39.415091+0000","last_active":"2024-09-21T21:41:13.279952+0000","last_peered":"2024-09-21T21:41:13.279952+0000","last_clean":"2024-09-21T21:41:13.279952+0000","last_became_active":"2024-09-21T21:40:37.421700+0000","last_became_peered":"2024-09-21T21:40:37.421700+0000","last_unstale":"2024-09-21T21:41:13.279952+0000","last_undegraded":"2024-09-21T21:41:13.279952+0000","last_fullsized":"2024-09-21T21:41:13.279952+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":436,"log_dups_size":0,"ondisk_log_size":436,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:58:05.773218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001083334,"stat_sum":{"num_bytes":7108330,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":2,"num_read":2825,"num_read_kb":10218,"num_write":752,"num_write_kb":5970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"46'371","reported_seq":3930,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T21:41:13.298241+0000","last_change":"2024-09-21T21:41:05.551020+0000","last_active":"2024-09-21T21:41:13.298241+0000","last_peered":"2024-09-21T21:41:13.298241+0000","last_clean":"2024-09-21T21:41:13.298241+0000","last_became_active":"2024-09-21T21:41:05.549644+0000","last_became_peered":"2024-09-21T21:41:05.549644+0000","last_unstale":"2024-09-21T21:41:13.298241+0000","last_undegraded":"2024-09-21T21:41:13.298241+0000","last_fullsized":"2024-09-21T21:41:13.298241+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:10:28.927617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001302274,"stat_sum":{"num_bytes":12871279,"num_objects":18,"num_object_clones":4,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":4,"num_read":4849,"num_read_kb":17802,"num_write":580,"num_write_kb":6006,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"46'144","reported_seq":6128,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T21:41:14.289368+0000","last_change":"2024-09-21T21:40:39.415498+0000","last_active":"2024-09-21T21:41:14.289368+0000","last_peered":"2024-09-21T21:41:14.289368+0000","last_clean":"2024-09-21T21:41:14.289368+0000","last_became_active":"2024-09-21T21:40:37.416021+0000","last_became_peered":"2024-09-21T21:40:37.416021+0000","last_unstale":"2024-09-21T21:41:14.289368+0000","last_undegraded":"2024-09-21T21:41:14.289368+0000","last_fullsized":"2024-09-21T21:41:14.289368+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:39:55.536033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098736399999999991,"stat_sum":{"num_bytes":6821376,"num_objects":19,"num_object_clones":2,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":5,"num_read":5954,"num_read_kb":9801,"num_write":147,"num_write_kb":6264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"46'123","reported_seq":4365,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T21:41:14.093523+0000","last_change":"2024-09-21T21:41:05.551767+0000","last_active":"2024-09-21T21:41:14.093523+0000","last_peered":"2024-09-21T21:41:14.093523+0000","last_clean":"2024-09-21T21:41:14.093523+0000","last_became_active":"2024-09-21T21:41:05.551305+0000","last_became_peered":"2024-09-21T21:41:05.551305+0000","last_unstale":"2024-09-21T21:41:14.093523+0000","last_undegraded":"2024-09-21T21:41:14.093523+0000","last_fullsized":"2024-09-21T21:41:14.093523+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:42:41.176672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020524470000000002,"stat_sum":{"num_bytes":24104485,"num_objects":12,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":2,"num_read":4214,"num_read_kb":23437,"num_write":134,"num_write_kb":2504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"46'183","reported_seq":2474,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T21:41:13.290413+0000","last_change":"2024-09-21T21:40:39.425496+0000","last_active":"2024-09-21T21:41:13.290413+0000","last_peered":"2024-09-21T21:41:13.290413+0000","last_clean":"2024-09-21T21:41:13.290413+0000","last_became_active":"2024-09-21T21:40:37.424987+0000","last_became_peered":"2024-09-21T21:40:37.424987+0000","last_unstale":"2024-09-21T21:41:13.290413+0000","last_undegraded":"2024-09-21T21:41:13.290413+0000","last_fullsized":"2024-09-21T21:41:13.290413+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:14:05.247449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016364540000000001,"stat_sum":{"num_bytes":3723792,"num_objects":17,"num_object_clones":3,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":4,"num_read":3632,"num_read_kb":9344,"num_write":132,"num_write_kb":857,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"46'105","reported_seq":4523,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T21:41:13.232656+0000","last_change":"2024-09-21T21:41:05.549466+0000","last_active":"2024-09-21T21:41:13.232656+0000","last_peered":"2024-09-21T21:41:13.232656+0000","last_clean":"2024-09-21T21:41:13.232656+0000","last_became_active":"2024-09-21T21:41:05.549078+0000","last_became_peered":"2024-09-21T21:41:05.549078+0000","last_unstale":"2024-09-21T21:41:13.232656+0000","last_undegraded":"2024-09-21T21:41:13.232656+0000","last_fullsized":"2024-09-21T21:41:13.232656+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:32:30.626009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065325099999999998,"stat_sum":{"num_bytes":26496000,"num_objects":13,"num_object_clones":4,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":4,"num_read":4400,"num_read_kb":28378,"num_write":105,"num_write_kb":3190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"46'254","reported_seq":2682,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T21:41:14.283170+0000","last_change":"2024-09-21T21:40:39.415203+0000","last_active":"2024-09-21T21:41:14.283170+0000","last_peered":"2024-09-21T21:41:14.283170+0000","last_clean":"2024-09-21T21:41:14.283170+0000","last_became_active":"2024-09-21T21:40:37.426148+0000","last_became_peered":"2024-09-21T21:40:37.426148+0000","last_unstale":"2024-09-21T21:41:14.283170+0000","last_undegraded":"2024-09-21T21:41:14.283170+0000","last_fullsized":"2024-09-21T21:41:14.283170+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":254,"log_dups_size":0,"ondisk_log_size":254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:24:38.666774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087239000000000001,"stat_sum":{"num_bytes":9232933,"num_objects":18,"num_object_clones":1,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":2,"num_read":3071,"num_read_kb":13001,"num_write":369,"num_write_kb":3156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"46'187","reported_seq":4799,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T21:41:13.233213+0000","last_change":"2024-09-21T21:40:39.429216+0000","last_active":"2024-09-21T21:41:13.233213+0000","last_peered":"2024-09-21T21:41:13.233213+0000","last_clean":"2024-09-21T21:41:13.233213+0000","last_became_active":"2024-09-21T21:40:37.424011+0000","last_became_peered":"2024-09-21T21:40:37.424011+0000","last_unstale":"2024-09-21T21:41:13.233213+0000","last_undegraded":"2024-09-21T21:41:13.233213+0000","last_fullsized":"2024-09-21T21:41:13.233213+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":187,"log_dups_size":0,"ondisk_log_size":187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:45:45.114393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046261000000000002,"stat_sum":{"num_bytes":7121920,"num_objects":20,"num_object_clones":4,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":4,"num_read":5245,"num_read_kb":13882,"num_write":141,"num_write_kb":3567,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"46'141","reported_seq":3149,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T21:41:14.099343+0000","last_change":"2024-09-21T21:40:39.414779+0000","last_active":"2024-09-21T21:41:14.099343+0000","last_peered":"2024-09-21T21:41:14.099343+0000","last_clean":"2024-09-21T21:41:14.099343+0000","last_became_active":"2024-09-21T21:40:37.421627+0000","last_became_peered":"2024-09-21T21:40:37.421627+0000","last_unstale":"2024-09-21T21:41:14.099343+0000","last_undegraded":"2024-09-21T21:41:14.099343+0000","last_fullsized":"2024-09-21T21:41:14.099343+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:03:25.067944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070986300000000003,"stat_sum":{"num_bytes":26231824,"num_objects":16,"num_object_clones":2,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":3,"num_read":2981,"num_read_kb":41965,"num_write":168,"num_write_kb":7887,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":371682966,"num_objects":510,"num_object_clones":59,"num_object_copies":1020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":510,"num_whiteouts":132,"num_read":111353,"num_read_kb":447639,"num_write":7327,"num_write_kb":88709,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":15925248,"data_stored":95643176,"data_compressed":7534129,"data_compressed_allocated":8482816,"data_compressed_original":89773056,"omap_allocated":0,"internal_metadata":0},"log_size":6158,"ondisk_log_size":6158,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64790,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":4,"up_from":13,"seq":55834574862,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38092,"kb_used_data":2740,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333748,"statfs":{"total":96636764160,"available":96597757952,"internally_reserved":0,"allocated":2805760,"data_stored":14547595,"data_compressed":1157071,"data_compressed_allocated":1179648,"data_compressed_original":13541888,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574862,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40128,"kb_used_data":4784,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94331712,"statfs":{"total":96636764160,"available":96595673088,"internally_reserved":0,"allocated":4898816,"data_stored":31207207,"data_compressed":2290143,"data_compressed_allocated":2420736,"data_compressed_original":29540864,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":12,"seq":51539607566,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38704,"kb_used_data":3356,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333136,"statfs":{"total":96636764160,"available":96597131264,"internally_reserved":0,"allocated":3436544,"data_stored":19336151,"data_compressed":1372290,"data_compressed_allocated":1581056,"data_compressed_original":17983488,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607566,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37532,"kb_used_data":2184,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334308,"statfs":{"total":96636764160,"available":96598331392,"internally_reserved":0,"allocated":2236416,"data_stored":5300499,"data_compressed":536533,"data_compressed_allocated":684032,"data_compressed_original":4239872,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,2],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607567,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38704,"kb_used_data":3356,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333136,"statfs":{"total":96636764160,"available":96597131264,"internally_reserved":0,"allocated":3436544,"data_stored":17534048,"data_compressed":1434721,"data_compressed_allocated":1585152,"data_compressed_original":16164864,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640270,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37468,"kb_used_data":2124,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334372,"statfs":{"total":96636764160,"available":96598396928,"internally_reserved":0,"allocated":2174976,"data_stored":9202936,"data_compressed":865762,"data_compressed_allocated":1122304,"data_compressed_original":8564224,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32395,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32395,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3125248,"data_stored":16890399,"data_compressed":1402326,"data_compressed_allocated":1540096,"data_compressed_original":15591424,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1650688,"data_stored":4717734,"data_compressed":415163,"data_compressed_allocated":602112,"data_compressed_original":3908096,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1699840,"data_stored":9075624,"data_compressed":868484,"data_compressed_allocated":1130496,"data_compressed_original":8621568,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":3874816,"data_stored":30010648,"data_compressed":2247155,"data_compressed_allocated":2293760,"data_compressed_original":28791296,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":2449408,"data_stored":15659264,"data_compressed":1226711,"data_compressed_allocated":1314816,"data_compressed_original":14836224,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":3125248,"data_stored":19289507,"data_compressed":1374290,"data_compressed_allocated":1601536,"data_compressed_original":18024448,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T21:41:17.291 INFO:tasks.thrashosds.thrasher:Setting 2.2 to [1, 4, 3, 2] 2024-09-21T21:41:17.291 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '2.2', '1', '4', '3', '2'] 2024-09-21T21:41:17.291 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 2.2 1 4 3 2 2024-09-21T21:41:17.325 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:17.325 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:17.334 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T21:41:17.378 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:17.377+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:17.438 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:17.438 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:17.447 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T21:41:17.462 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:17.461+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:17.563 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:17.561+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:17.590 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:17.591 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:17.599 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T21:41:17.682 INFO:teuthology.orchestra.run.smithi164.stdout:flattening image image_client.0-clone14 2024-09-21T21:41:17.683 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:17.682+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:17.684 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T21:41:17.754 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:17.754 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:17.763 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T21:41:17.764 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:17.763+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:17.865 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:17.864+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:17.910 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:17.911 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:17.920 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T21:41:17.966 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:17.964+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:18.028 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:18.028 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:18.037 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T21:41:18.078 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:18.077+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:18.095 INFO:teuthology.orchestra.run.smithi164.stdout:checking clone #13, image image_client.0-clone13 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent14 2024-09-21T21:41:18.167 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:18.165+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:18.221 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:18.221 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:18.232 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T21:41:18.267 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:18.266+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:18.368 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:18.367+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:18.382 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:18.382 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:18.394 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T21:41:18.477 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:18.476+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:18.537 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:18.537 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:18.539 INFO:teuthology.orchestra.run.smithi164.stdout:2415 trunc from 0xd4bb000 to 0x54f9200 2024-09-21T21:41:18.546 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T21:41:18.570 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:18.569+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:18.594 INFO:teuthology.orchestra.run.smithi164.stdout:2417 read 0x11b9400 thru 0x11be7bd (0x53be bytes) 2024-09-21T21:41:18.596 INFO:teuthology.orchestra.run.smithi164.stdout:2418 read 0x3f5ba00 thru 0x3f5e792 (0x2d93 bytes) 2024-09-21T21:41:18.596 INFO:teuthology.orchestra.run.smithi164.stdout:2419 read 0x238e000 thru 0x239ae95 (0xce96 bytes) 2024-09-21T21:41:18.608 INFO:teuthology.orchestra.run.smithi164.stdout:2420 write 0x7f8ac00 thru 0x7f8c8ea (0x1ceb bytes) 2024-09-21T21:41:18.613 INFO:teuthology.orchestra.run.smithi164.stdout:2422 trunc from 0x7f8ca00 to 0xafc2a00 2024-09-21T21:41:18.615 INFO:teuthology.orchestra.run.smithi081.stderr:set 2.2 pg_upmap_items mapping to [1->4,3->2] 2024-09-21T21:41:18.615 INFO:teuthology.orchestra.run.smithi081.stderr:noscrub is unset 2024-09-21T21:41:18.630 INFO:teuthology.orchestra.run.smithi164.stdout:2425 read 0x83db200 thru 0x83e1c13 (0x6a14 bytes) 2024-09-21T21:41:18.633 INFO:teuthology.orchestra.run.smithi164.stdout:2426 read 0x7ab7200 thru 0x7ac5e09 (0xec0a bytes) 2024-09-21T21:41:18.633 INFO:teuthology.orchestra.run.smithi164.stdout:2428 read 0x1751a00 thru 0x175c542 (0xab43 bytes) 2024-09-21T21:41:18.635 INFO:teuthology.orchestra.run.smithi164.stdout:2429 write 0x6f8aa00 thru 0x6f970e7 (0xc6e8 bytes) 2024-09-21T21:41:18.635 INFO:teuthology.orchestra.run.smithi164.stdout:2430 write 0x4de9a00 thru 0x4dea2ab (0x8ac bytes) 2024-09-21T21:41:18.635 INFO:teuthology.orchestra.run.smithi164.stdout:2432 read 0x23f8000 thru 0x2406cbc (0xecbd bytes) 2024-09-21T21:41:18.636 INFO:teuthology.orchestra.run.smithi164.stdout:2434 write 0x530800 thru 0x53e4cb (0xdccc bytes) 2024-09-21T21:41:18.652 INFO:teuthology.orchestra.run.smithi164.stdout:2436 write 0xdcbc600 thru 0xdcc09c4 (0x43c5 bytes) 2024-09-21T21:41:18.658 INFO:teuthology.orchestra.run.smithi164.stdout:2440 trunc from 0xdcc0a00 to 0xd24c400 2024-09-21T21:41:18.671 INFO:teuthology.orchestra.run.smithi164.stdout:2441 read 0xab99e00 thru 0xaba63ce (0xc5cf bytes) 2024-09-21T21:41:18.672 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:18.670+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:18.685 INFO:teuthology.orchestra.run.smithi164.stdout:2442 write 0xd40f200 thru 0xd410ffc (0x1dfd bytes) 2024-09-21T21:41:18.698 INFO:teuthology.orchestra.run.smithi164.stdout:2443 trunc from 0xd411000 to 0x2ecce00 2024-09-21T21:41:18.702 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T21:41:18.707 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:18.707 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:18.717 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T21:41:18.738 INFO:teuthology.orchestra.run.smithi164.stdout:2444 punch from 0x1ef2a00 to 0x1ef8e00, (0x6400 bytes) 2024-09-21T21:41:18.747 INFO:teuthology.orchestra.run.smithi164.stdout:2445 write 0x3450600 thru 0x345a1cc (0x9bcd bytes) 2024-09-21T21:41:18.773 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:18.771+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:18.773 INFO:teuthology.orchestra.run.smithi164.stdout:2446 write 0xe861000 thru 0xe870dee (0xfdef bytes) 2024-09-21T21:41:18.774 INFO:teuthology.orchestra.run.smithi164.stdout:2448 write 0x6e62800 thru 0x6e6cf1a (0xa71b bytes) 2024-09-21T21:41:18.774 INFO:teuthology.orchestra.run.smithi164.stdout:2451 write 0x6884800 thru 0x6888dcf (0x45d0 bytes) 2024-09-21T21:41:18.775 INFO:teuthology.orchestra.run.smithi164.stdout:2453 write 0xae56e00 thru 0xae5f0bd (0x82be bytes) 2024-09-21T21:41:18.775 INFO:teuthology.orchestra.run.smithi164.stdout:2454 write 0x2fcac00 thru 0x2fcc37f (0x1780 bytes) 2024-09-21T21:41:18.776 INFO:teuthology.orchestra.run.smithi164.stdout:2456 write 0x20e00 thru 0x23520 (0x2721 bytes) 2024-09-21T21:41:18.793 INFO:teuthology.orchestra.run.smithi164.stdout:2457 trunc from 0xe870e00 to 0x95d000 2024-09-21T21:41:18.833 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0: {} 2024-09-21T21:41:18.833 INFO:teuthology.orchestra.run.smithi081.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T21:41:18.844 INFO:teuthology.orchestra.run.smithi164.stdout:2458 trunc from 0x95d000 to 0x6b23c00 2024-09-21T21:41:18.854 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1: {} 2024-09-21T21:41:18.855 INFO:teuthology.orchestra.run.smithi081.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T21:41:18.861 INFO:teuthology.orchestra.run.smithi164.stdout:2459 trunc from 0x6b23c00 to 0x174a400 2024-09-21T21:41:18.873 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2: {} 2024-09-21T21:41:18.873 INFO:teuthology.orchestra.run.smithi081.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T21:41:18.873 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:18.872+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:18.874 INFO:teuthology.orchestra.run.smithi164.stdout:2460 read 0x401800 thru 0x40cfc0 (0xb7c1 bytes) 2024-09-21T21:41:18.876 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:18.876 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 20, 2024-09-21T21:41:18.876 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 600, 2024-09-21T21:41:18.876 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:41:18.877 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.877 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:2885 3.b 3:d49b370f:::rbd_object_map.1121e540c096:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:18.877 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.478228+0000", 2024-09-21T21:41:18.877 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 20.395906555, 2024-09-21T21:41:18.877 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.015327835999999999, 2024-09-21T21:41:18.877 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:18.877 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:18.877 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:41:18.877 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:18.877 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:18.877 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:18.877 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2885 2024-09-21T21:41:18.877 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.877 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:18.878 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.878 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:18.878 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478228+0000", 2024-09-21T21:41:18.878 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:18.878 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.878 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.878 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:18.878 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478228+0000", 2024-09-21T21:41:18.878 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:18.878 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.878 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.878 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:18.878 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478229+0000", 2024-09-21T21:41:18.878 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8000000000000001e-07 2024-09-21T21:41:18.879 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.879 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.879 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:18.879 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478231+0000", 2024-09-21T21:41:18.879 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.5069999999999999e-06 2024-09-21T21:41:18.879 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.879 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.879 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:18.879 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478238+0000", 2024-09-21T21:41:18.879 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.8820000000000003e-06 2024-09-21T21:41:18.879 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.879 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.880 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:18.880 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478243+0000", 2024-09-21T21:41:18.880 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2209999999999996e-06 2024-09-21T21:41:18.880 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.880 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.880 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:18.880 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.488160+0000", 2024-09-21T21:41:18.880 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0099169849999999997 2024-09-21T21:41:18.880 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.880 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.880 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:18.880 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.488231+0000", 2024-09-21T21:41:18.880 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0909999999999997e-05 2024-09-21T21:41:18.880 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.880 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.881 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 0", 2024-09-21T21:41:18.881 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.488538+0000", 2024-09-21T21:41:18.881 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000306736 2024-09-21T21:41:18.881 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.881 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.881 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:41:18.881 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.493210+0000", 2024-09-21T21:41:18.881 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0046722120000000002 2024-09-21T21:41:18.881 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.881 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.881 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:41:18.881 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.493471+0000", 2024-09-21T21:41:18.881 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00026132899999999998 2024-09-21T21:41:18.881 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.882 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.882 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:41:18.882 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.493517+0000", 2024-09-21T21:41:18.882 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5309000000000003e-05 2024-09-21T21:41:18.882 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.882 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.882 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:18.882 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.493556+0000", 2024-09-21T21:41:18.882 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.9365e-05 2024-09-21T21:41:18.882 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:18.882 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:18.882 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:18.882 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.882 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.882 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:2886 3.b 3:d49b370f:::rbd_object_map.1121e540c096:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:18.883 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.478249+0000", 2024-09-21T21:41:18.883 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 20.395885775, 2024-09-21T21:41:18.883 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.015426120999999999, 2024-09-21T21:41:18.883 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:18.883 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:18.883 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:41:18.883 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:18.883 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:18.883 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:18.883 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2886 2024-09-21T21:41:18.883 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.883 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:18.883 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.883 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:18.883 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478249+0000", 2024-09-21T21:41:18.884 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:18.884 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.884 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.884 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:18.884 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478249+0000", 2024-09-21T21:41:18.884 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:18.884 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.884 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.884 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:18.884 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478249+0000", 2024-09-21T21:41:18.884 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.9999999999999998e-07 2024-09-21T21:41:18.884 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.884 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.884 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:18.885 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478252+0000", 2024-09-21T21:41:18.885 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.5500000000000001e-06 2024-09-21T21:41:18.885 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.885 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.885 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:18.885 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478258+0000", 2024-09-21T21:41:18.885 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7080000000000002e-06 2024-09-21T21:41:18.885 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.885 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.885 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:18.885 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478265+0000", 2024-09-21T21:41:18.886 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9110000000000004e-06 2024-09-21T21:41:18.886 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.886 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.886 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:18.886 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.488889+0000", 2024-09-21T21:41:18.886 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.010624662 2024-09-21T21:41:18.886 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.886 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.886 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:18.886 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.488965+0000", 2024-09-21T21:41:18.887 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5949000000000001e-05 2024-09-21T21:41:18.887 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.887 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.887 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 0", 2024-09-21T21:41:18.887 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.489235+0000", 2024-09-21T21:41:18.887 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00027019499999999998 2024-09-21T21:41:18.887 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.887 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.887 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:41:18.887 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.493233+0000", 2024-09-21T21:41:18.887 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0039978799999999997 2024-09-21T21:41:18.887 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.887 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.887 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:41:18.888 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.493601+0000", 2024-09-21T21:41:18.888 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00036728200000000003 2024-09-21T21:41:18.888 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.888 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.888 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:41:18.888 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.493637+0000", 2024-09-21T21:41:18.888 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6263000000000002e-05 2024-09-21T21:41:18.888 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.888 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.888 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:18.888 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.493675+0000", 2024-09-21T21:41:18.888 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8321000000000001e-05 2024-09-21T21:41:18.888 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:18.888 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:18.888 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:18.889 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.889 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.889 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:2887 3.b 3:d49b370f:::rbd_object_map.1121e540c096:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:18.889 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.478271+0000", 2024-09-21T21:41:18.889 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 20.395864263, 2024-09-21T21:41:18.889 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.015551966, 2024-09-21T21:41:18.889 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:18.889 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:18.889 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:41:18.889 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:18.889 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:18.889 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:18.889 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2887 2024-09-21T21:41:18.889 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.889 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:18.890 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.890 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:18.890 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478271+0000", 2024-09-21T21:41:18.890 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:18.890 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.890 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.890 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:18.890 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478271+0000", 2024-09-21T21:41:18.890 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:18.890 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.890 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.890 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:18.890 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478271+0000", 2024-09-21T21:41:18.890 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.0600000000000001e-07 2024-09-21T21:41:18.891 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.891 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.891 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:18.891 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478274+0000", 2024-09-21T21:41:18.891 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.6390000000000001e-06 2024-09-21T21:41:18.891 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.891 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.891 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:18.891 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478280+0000", 2024-09-21T21:41:18.891 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3219999999999998e-06 2024-09-21T21:41:18.891 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.891 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.891 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:18.891 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.478285+0000", 2024-09-21T21:41:18.891 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6589999999999998e-06 2024-09-21T21:41:18.892 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.892 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.892 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:18.892 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.489524+0000", 2024-09-21T21:41:18.892 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.01123979 2024-09-21T21:41:18.892 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.892 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.892 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:18.892 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.489567+0000", 2024-09-21T21:41:18.892 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2673000000000003e-05 2024-09-21T21:41:18.892 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.892 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.892 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 0", 2024-09-21T21:41:18.892 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.489834+0000", 2024-09-21T21:41:18.893 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00026672799999999998 2024-09-21T21:41:18.893 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.893 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.893 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:41:18.893 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.493290+0000", 2024-09-21T21:41:18.893 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0034557429999999998 2024-09-21T21:41:18.893 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.893 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.893 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:41:18.893 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.493737+0000", 2024-09-21T21:41:18.893 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00044731799999999999 2024-09-21T21:41:18.893 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.893 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.893 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:41:18.894 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.493784+0000", 2024-09-21T21:41:18.894 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7178999999999997e-05 2024-09-21T21:41:18.894 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.894 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.894 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:18.894 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.493823+0000", 2024-09-21T21:41:18.894 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8509e-05 2024-09-21T21:41:18.894 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:18.894 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:18.894 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:18.894 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.894 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.894 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4325 3.0 3:074cff57:::rbd_data.112145fae779.000000000000000c:a [read 2064384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:18.895 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.510141+0000", 2024-09-21T21:41:18.895 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 20.363994100999999, 2024-09-21T21:41:18.895 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013330887, 2024-09-21T21:41:18.895 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:18.895 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:18.896 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:18.896 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:18.896 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:18.896 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:18.896 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4325 2024-09-21T21:41:18.896 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.896 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:18.896 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.896 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:18.896 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510141+0000", 2024-09-21T21:41:18.896 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:18.897 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.897 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.897 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:18.897 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510141+0000", 2024-09-21T21:41:18.897 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:18.897 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.897 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.898 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:18.898 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510141+0000", 2024-09-21T21:41:18.898 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.72e-07 2024-09-21T21:41:18.898 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.898 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.898 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:18.898 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510143+0000", 2024-09-21T21:41:18.898 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2409999999999998e-06 2024-09-21T21:41:18.898 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.899 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.899 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:18.899 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510151+0000", 2024-09-21T21:41:18.899 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.3390000000000004e-06 2024-09-21T21:41:18.899 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.899 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.899 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:18.899 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510155+0000", 2024-09-21T21:41:18.899 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5630000000000002e-06 2024-09-21T21:41:18.899 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.899 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.899 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:18.899 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.523310+0000", 2024-09-21T21:41:18.899 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013154658 2024-09-21T21:41:18.900 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.900 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.900 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:18.900 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.523386+0000", 2024-09-21T21:41:18.900 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.6553999999999999e-05 2024-09-21T21:41:18.900 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.900 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.900 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:18.900 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.523472+0000", 2024-09-21T21:41:18.900 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.5160000000000005e-05 2024-09-21T21:41:18.900 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:18.900 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:18.900 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:18.900 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.901 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.901 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4326 3.0 3:074cff57:::rbd_data.112145fae779.000000000000000c:a [read 2097152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:18.901 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.510162+0000", 2024-09-21T21:41:18.901 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 20.363973086000001, 2024-09-21T21:41:18.901 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013537466, 2024-09-21T21:41:18.901 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:18.901 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:18.901 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:18.901 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:18.901 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:18.901 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:18.901 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4326 2024-09-21T21:41:18.901 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.902 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:18.902 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.902 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:18.902 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510162+0000", 2024-09-21T21:41:18.902 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:18.902 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.902 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.902 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:18.902 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510162+0000", 2024-09-21T21:41:18.902 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:18.902 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.902 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.902 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:18.902 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510162+0000", 2024-09-21T21:41:18.903 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.0699999999999998e-07 2024-09-21T21:41:18.903 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.903 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.903 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:18.903 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510164+0000", 2024-09-21T21:41:18.903 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.17e-06 2024-09-21T21:41:18.903 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.903 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.903 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:18.903 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510174+0000", 2024-09-21T21:41:18.903 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2439999999999999e-06 2024-09-21T21:41:18.903 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.903 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.903 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:18.903 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510179+0000", 2024-09-21T21:41:18.904 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2159999999999997e-06 2024-09-21T21:41:18.904 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.904 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.904 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:18.904 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.523502+0000", 2024-09-21T21:41:18.904 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013322779 2024-09-21T21:41:18.904 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.904 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.904 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:18.904 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.523579+0000", 2024-09-21T21:41:18.904 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7402000000000002e-05 2024-09-21T21:41:18.904 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.904 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.905 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:18.905 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.523699+0000", 2024-09-21T21:41:18.905 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000120248 2024-09-21T21:41:18.905 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:18.905 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:18.905 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:18.905 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.905 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.905 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4327 3.0 3:074cff57:::rbd_data.112145fae779.000000000000000c:a [read 2154496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:18.905 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.510418+0000", 2024-09-21T21:41:18.905 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 20.363716651000001, 2024-09-21T21:41:18.905 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013505039999999999, 2024-09-21T21:41:18.905 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:18.905 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:18.905 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:18.906 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:18.906 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:18.906 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:18.906 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4327 2024-09-21T21:41:18.906 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.906 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:18.906 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.906 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:18.906 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510418+0000", 2024-09-21T21:41:18.906 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:18.906 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.906 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.906 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:18.907 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510418+0000", 2024-09-21T21:41:18.907 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:18.907 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.907 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.907 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:18.907 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510419+0000", 2024-09-21T21:41:18.907 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8999999999999997e-07 2024-09-21T21:41:18.907 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.907 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.907 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:18.907 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510421+0000", 2024-09-21T21:41:18.907 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.4779999999999998e-06 2024-09-21T21:41:18.907 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.907 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.907 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:18.908 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510427+0000", 2024-09-21T21:41:18.908 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5609999999999998e-06 2024-09-21T21:41:18.908 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.908 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.908 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:18.908 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510432+0000", 2024-09-21T21:41:18.908 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5389999999999998e-06 2024-09-21T21:41:18.908 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.908 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.908 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:18.908 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.523743+0000", 2024-09-21T21:41:18.909 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013310963 2024-09-21T21:41:18.909 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.909 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.909 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:18.909 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.523824+0000", 2024-09-21T21:41:18.909 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0463000000000005e-05 2024-09-21T21:41:18.909 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.909 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.909 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:18.909 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.523923+0000", 2024-09-21T21:41:18.909 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.9545999999999999e-05 2024-09-21T21:41:18.909 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:18.909 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:18.909 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:18.910 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.910 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.910 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4328 3.0 3:074cff57:::rbd_data.112145fae779.000000000000000c:a [read 2211840~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:18.910 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.510443+0000", 2024-09-21T21:41:18.910 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 20.363692351000001, 2024-09-21T21:41:18.910 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013684564, 2024-09-21T21:41:18.910 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:18.910 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:18.910 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:18.910 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:18.910 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:18.910 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:18.911 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4328 2024-09-21T21:41:18.911 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.911 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:18.911 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.911 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:18.911 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510443+0000", 2024-09-21T21:41:18.911 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:18.911 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.911 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.912 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:18.912 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510443+0000", 2024-09-21T21:41:18.912 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:18.912 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.912 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.912 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:18.912 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510443+0000", 2024-09-21T21:41:18.912 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2199999999999999e-07 2024-09-21T21:41:18.912 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.912 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.912 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:18.912 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510445+0000", 2024-09-21T21:41:18.912 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1679999999999998e-06 2024-09-21T21:41:18.913 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.913 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.913 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:18.913 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510452+0000", 2024-09-21T21:41:18.913 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6259999999999997e-06 2024-09-21T21:41:18.913 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.913 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.913 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:18.913 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.510457+0000", 2024-09-21T21:41:18.913 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8110000000000002e-06 2024-09-21T21:41:18.913 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.913 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.913 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:18.913 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.523968+0000", 2024-09-21T21:41:18.914 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013510996000000001 2024-09-21T21:41:18.914 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.914 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.914 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:18.914 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.524044+0000", 2024-09-21T21:41:18.914 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.6259000000000003e-05 2024-09-21T21:41:18.914 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.914 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.914 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:18.914 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.524127+0000", 2024-09-21T21:41:18.914 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.3282000000000007e-05 2024-09-21T21:41:18.915 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:18.915 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:18.915 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:18.915 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.915 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.915 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4577 3.0 3:074cff57:::rbd_data.112145fae779.000000000000000c:a [read 1744896~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:18.915 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.514327+0000", 2024-09-21T21:41:18.915 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 20.359807694000001, 2024-09-21T21:41:18.915 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013451424999999999, 2024-09-21T21:41:18.915 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:18.915 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:18.915 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:18.915 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:18.915 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:18.916 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:18.916 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4577 2024-09-21T21:41:18.916 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.916 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:18.916 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.916 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:18.916 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514327+0000", 2024-09-21T21:41:18.916 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:18.916 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.916 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.916 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:18.916 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514327+0000", 2024-09-21T21:41:18.916 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:18.916 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.917 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.917 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:18.917 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514328+0000", 2024-09-21T21:41:18.917 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4900000000000001e-07 2024-09-21T21:41:18.917 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.917 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.917 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:18.917 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514330+0000", 2024-09-21T21:41:18.917 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.227e-06 2024-09-21T21:41:18.917 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.917 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.917 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:18.917 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514336+0000", 2024-09-21T21:41:18.917 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1940000000000003e-06 2024-09-21T21:41:18.918 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.918 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.918 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:18.918 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514341+0000", 2024-09-21T21:41:18.918 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7319999999999998e-06 2024-09-21T21:41:18.918 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.918 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.918 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:18.918 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.527415+0000", 2024-09-21T21:41:18.918 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013074657 2024-09-21T21:41:18.918 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.918 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.918 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:18.919 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.527491+0000", 2024-09-21T21:41:18.919 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5899e-05 2024-09-21T21:41:18.919 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.919 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.919 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:18.919 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.527779+0000", 2024-09-21T21:41:18.919 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00028736700000000002 2024-09-21T21:41:18.919 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:18.919 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:18.919 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:18.919 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.920 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.920 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4578 3.0 3:074cff57:::rbd_data.112145fae779.000000000000000c:a [read 1802240~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:18.920 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.514347+0000", 2024-09-21T21:41:18.920 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 20.359788239, 2024-09-21T21:41:18.920 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.014025592999999999, 2024-09-21T21:41:18.920 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:18.920 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:18.920 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:18.920 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:18.920 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:18.920 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:18.920 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4578 2024-09-21T21:41:18.920 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.921 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:18.921 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.921 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:18.921 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514347+0000", 2024-09-21T21:41:18.921 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:18.921 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.921 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.921 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:18.921 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514347+0000", 2024-09-21T21:41:18.921 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:18.921 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.921 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.922 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:18.922 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514347+0000", 2024-09-21T21:41:18.922 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6699999999999999e-07 2024-09-21T21:41:18.922 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.922 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.922 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:18.922 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514349+0000", 2024-09-21T21:41:18.922 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0109999999999999e-06 2024-09-21T21:41:18.922 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.922 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.922 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:18.922 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514355+0000", 2024-09-21T21:41:18.923 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1090000000000003e-06 2024-09-21T21:41:18.923 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.923 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.923 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:18.923 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514360+0000", 2024-09-21T21:41:18.923 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5900000000000001e-06 2024-09-21T21:41:18.923 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.923 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.923 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:18.923 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.527862+0000", 2024-09-21T21:41:18.923 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013501812 2024-09-21T21:41:18.923 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.923 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.924 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:18.924 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.527933+0000", 2024-09-21T21:41:18.924 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.1322000000000006e-05 2024-09-21T21:41:18.924 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.924 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.924 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:18.924 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.528372+0000", 2024-09-21T21:41:18.924 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00043938199999999999 2024-09-21T21:41:18.924 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:18.924 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:18.924 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:18.924 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.924 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.925 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4579 3.0 3:074cff57:::rbd_data.112145fae779.000000000000000c:a [read 1835008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:18.925 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.514365+0000", 2024-09-21T21:41:18.925 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 20.359769382, 2024-09-21T21:41:18.925 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.014158828, 2024-09-21T21:41:18.925 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:18.925 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:18.925 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:18.925 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:18.925 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:18.925 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:18.926 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4579 2024-09-21T21:41:18.926 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.926 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:18.926 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.926 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:18.926 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514365+0000", 2024-09-21T21:41:18.926 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:18.926 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.926 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.926 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:18.926 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514365+0000", 2024-09-21T21:41:18.926 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:18.926 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.927 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.927 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:18.927 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514366+0000", 2024-09-21T21:41:18.927 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7800000000000002e-07 2024-09-21T21:41:18.927 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.927 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.927 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:18.927 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514368+0000", 2024-09-21T21:41:18.927 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9920000000000002e-06 2024-09-21T21:41:18.927 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.927 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.927 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:18.927 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514374+0000", 2024-09-21T21:41:18.927 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9370000000000001e-06 2024-09-21T21:41:18.928 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.928 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.928 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:18.928 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514379+0000", 2024-09-21T21:41:18.928 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7149999999999997e-06 2024-09-21T21:41:18.928 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.928 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.928 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:18.928 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.528303+0000", 2024-09-21T21:41:18.928 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013924076000000001 2024-09-21T21:41:18.928 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.928 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.929 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:18.929 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.528429+0000", 2024-09-21T21:41:18.929 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00012643100000000001 2024-09-21T21:41:18.929 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.929 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.929 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:18.929 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.528524+0000", 2024-09-21T21:41:18.929 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.5298999999999997e-05 2024-09-21T21:41:18.929 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:18.929 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:18.929 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:18.929 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.929 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.929 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4580 3.0 3:074cff57:::rbd_data.112145fae779.000000000000000c:a [read 1892352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:18.929 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.514384+0000", 2024-09-21T21:41:18.930 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 20.359750785999999, 2024-09-21T21:41:18.930 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.014365243, 2024-09-21T21:41:18.930 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:18.930 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:18.930 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:18.930 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:18.930 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:18.930 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:18.930 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4580 2024-09-21T21:41:18.930 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.930 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:18.930 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.930 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:18.930 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514384+0000", 2024-09-21T21:41:18.931 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:18.931 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.931 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.931 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:18.931 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514384+0000", 2024-09-21T21:41:18.931 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:18.931 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.931 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.931 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:18.931 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514384+0000", 2024-09-21T21:41:18.931 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1100000000000001e-07 2024-09-21T21:41:18.931 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.931 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.931 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:18.932 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514387+0000", 2024-09-21T21:41:18.932 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1150000000000001e-06 2024-09-21T21:41:18.932 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.932 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.932 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:18.932 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514393+0000", 2024-09-21T21:41:18.932 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1140000000000002e-06 2024-09-21T21:41:18.932 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.932 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.932 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:18.932 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514398+0000", 2024-09-21T21:41:18.932 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6330000000000001e-06 2024-09-21T21:41:18.932 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.932 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.933 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:18.933 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.528569+0000", 2024-09-21T21:41:18.933 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.014171011000000001 2024-09-21T21:41:18.933 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.933 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.933 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:18.933 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.528642+0000", 2024-09-21T21:41:18.933 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.2336000000000007e-05 2024-09-21T21:41:18.933 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.933 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.933 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:18.933 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.528749+0000", 2024-09-21T21:41:18.933 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000107623 2024-09-21T21:41:18.933 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:18.933 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:18.934 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:18.934 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.934 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.934 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4581 3.0 3:074cff57:::rbd_data.112145fae779.000000000000000c:a [read 1949696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:18.934 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.514404+0000", 2024-09-21T21:41:18.934 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 20.359730619, 2024-09-21T21:41:18.934 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.014551563999999999, 2024-09-21T21:41:18.934 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:18.934 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:18.934 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:18.934 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:18.934 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:18.934 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:18.935 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4581 2024-09-21T21:41:18.935 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.935 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:18.935 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.935 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:18.935 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514404+0000", 2024-09-21T21:41:18.935 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:18.935 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.935 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.935 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:18.935 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514404+0000", 2024-09-21T21:41:18.935 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:18.935 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.935 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.935 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:18.936 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514405+0000", 2024-09-21T21:41:18.936 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7599999999999998e-07 2024-09-21T21:41:18.936 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.936 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.936 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:18.936 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514407+0000", 2024-09-21T21:41:18.936 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0169999999999999e-06 2024-09-21T21:41:18.936 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.936 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.936 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:18.936 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514413+0000", 2024-09-21T21:41:18.937 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0020000000000003e-06 2024-09-21T21:41:18.937 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.937 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.937 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:18.937 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514417+0000", 2024-09-21T21:41:18.937 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7319999999999998e-06 2024-09-21T21:41:18.937 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.937 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.937 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:18.937 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.528781+0000", 2024-09-21T21:41:18.937 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.014363433 2024-09-21T21:41:18.937 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.937 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.937 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:18.937 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.528861+0000", 2024-09-21T21:41:18.938 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0408000000000007e-05 2024-09-21T21:41:18.938 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.938 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.938 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:18.938 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.528956+0000", 2024-09-21T21:41:18.938 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.4596e-05 2024-09-21T21:41:18.938 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:18.938 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:18.938 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:18.938 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.938 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.938 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4582 3.0 3:074cff57:::rbd_data.112145fae779.000000000000000c:a [read 1982464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:18.939 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.514423+0000", 2024-09-21T21:41:18.939 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 20.359711735000001, 2024-09-21T21:41:18.939 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.014719965, 2024-09-21T21:41:18.939 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:18.939 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:18.939 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:18.939 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:18.939 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:18.940 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:18.940 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4582 2024-09-21T21:41:18.940 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.940 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:18.940 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.940 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:18.940 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514423+0000", 2024-09-21T21:41:18.940 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:18.941 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.941 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.941 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:18.941 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514423+0000", 2024-09-21T21:41:18.941 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:18.941 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.941 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.941 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:18.941 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514423+0000", 2024-09-21T21:41:18.941 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.53e-07 2024-09-21T21:41:18.942 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.942 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.942 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:18.942 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514426+0000", 2024-09-21T21:41:18.942 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2000000000000001e-06 2024-09-21T21:41:18.942 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.942 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.942 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:18.943 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514433+0000", 2024-09-21T21:41:18.943 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.3170000000000004e-06 2024-09-21T21:41:18.943 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.943 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.943 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:18.943 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514438+0000", 2024-09-21T21:41:18.943 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3730000000000001e-06 2024-09-21T21:41:18.943 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.943 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.943 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:18.944 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.528984+0000", 2024-09-21T21:41:18.944 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.014545773 2024-09-21T21:41:18.944 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.944 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.944 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:18.944 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.529056+0000", 2024-09-21T21:41:18.945 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.2067e-05 2024-09-21T21:41:18.945 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.945 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.945 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:18.945 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.529143+0000", 2024-09-21T21:41:18.945 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.6881999999999999e-05 2024-09-21T21:41:18.946 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:18.946 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:18.946 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:18.946 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.946 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.946 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4583 3.0 3:074cff57:::rbd_data.112145fae779.000000000000000c:a [read 2039808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:18.947 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.514446+0000", 2024-09-21T21:41:18.947 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 20.359688963, 2024-09-21T21:41:18.947 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.016072466000000001, 2024-09-21T21:41:18.947 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:18.947 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:18.947 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:18.948 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:18.948 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:18.948 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:18.948 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4583 2024-09-21T21:41:18.948 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.948 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:18.949 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.949 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:18.949 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514446+0000", 2024-09-21T21:41:18.949 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:18.949 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.949 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.949 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:18.950 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514446+0000", 2024-09-21T21:41:18.950 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:18.950 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.950 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.950 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:18.951 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514446+0000", 2024-09-21T21:41:18.951 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.9700000000000002e-07 2024-09-21T21:41:18.951 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.951 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.951 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:18.951 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514448+0000", 2024-09-21T21:41:18.952 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1849999999999998e-06 2024-09-21T21:41:18.952 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.952 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.952 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:18.952 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514454+0000", 2024-09-21T21:41:18.952 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.015e-06 2024-09-21T21:41:18.952 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.953 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.953 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:18.953 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514459+0000", 2024-09-21T21:41:18.953 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7639999999999996e-06 2024-09-21T21:41:18.953 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.953 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.954 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:18.954 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.529173+0000", 2024-09-21T21:41:18.954 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.014713688000000001 2024-09-21T21:41:18.954 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.954 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.954 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:18.954 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.529248+0000", 2024-09-21T21:41:18.955 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5073000000000004e-05 2024-09-21T21:41:18.955 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.955 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.955 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:18.955 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.530518+0000", 2024-09-21T21:41:18.955 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0012703440000000001 2024-09-21T21:41:18.955 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:18.955 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:18.955 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:18.956 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.956 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.956 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4584 3.0 3:074cff57:::rbd_data.112145fae779.000000000000000c:a [read 2129920~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:18.956 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.514465+0000", 2024-09-21T21:41:18.956 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 20.359669524000001, 2024-09-21T21:41:18.956 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.016252026999999999, 2024-09-21T21:41:18.956 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:18.957 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:18.957 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:18.957 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:18.957 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:18.957 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:18.957 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4584 2024-09-21T21:41:18.957 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.957 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:18.957 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.957 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:18.958 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514465+0000", 2024-09-21T21:41:18.958 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:18.958 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.958 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.958 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:18.958 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514465+0000", 2024-09-21T21:41:18.958 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:18.958 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.959 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.959 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:18.959 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514466+0000", 2024-09-21T21:41:18.959 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.0400000000000002e-07 2024-09-21T21:41:18.959 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.959 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.959 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:18.959 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514468+0000", 2024-09-21T21:41:18.960 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3159999999999999e-06 2024-09-21T21:41:18.960 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.960 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.960 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:18.960 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514474+0000", 2024-09-21T21:41:18.960 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.224e-06 2024-09-21T21:41:18.960 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.960 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.960 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:18.960 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514479+0000", 2024-09-21T21:41:18.960 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8099999999999997e-06 2024-09-21T21:41:18.960 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.960 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.961 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:18.961 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.530552+0000", 2024-09-21T21:41:18.961 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.016072622000000002 2024-09-21T21:41:18.961 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.961 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.961 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:18.961 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.530630+0000", 2024-09-21T21:41:18.961 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.8448000000000005e-05 2024-09-21T21:41:18.961 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.961 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.961 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:18.961 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.530717+0000", 2024-09-21T21:41:18.961 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.7202999999999998e-05 2024-09-21T21:41:18.961 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:18.962 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:18.962 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:18.962 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.962 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.962 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4585 3.0 3:074cff57:::rbd_data.112145fae779.000000000000000c:a [read 2187264~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:18.962 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.514485+0000", 2024-09-21T21:41:18.962 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 20.359649857000001, 2024-09-21T21:41:18.962 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.016420605000000001, 2024-09-21T21:41:18.962 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:18.962 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:18.962 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:18.962 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:18.962 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:18.963 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:18.963 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4585 2024-09-21T21:41:18.963 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.963 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:18.963 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.963 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:18.963 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514485+0000", 2024-09-21T21:41:18.963 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:18.963 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.963 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.964 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:18.964 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514485+0000", 2024-09-21T21:41:18.964 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:18.964 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.964 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.964 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:18.964 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514485+0000", 2024-09-21T21:41:18.964 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3599999999999999e-07 2024-09-21T21:41:18.964 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.964 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.964 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:18.964 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514487+0000", 2024-09-21T21:41:18.964 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0870000000000001e-06 2024-09-21T21:41:18.964 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.965 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.965 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:18.965 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514495+0000", 2024-09-21T21:41:18.965 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7589999999999997e-06 2024-09-21T21:41:18.965 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.965 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.965 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:18.965 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.514501+0000", 2024-09-21T21:41:18.965 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4e-06 2024-09-21T21:41:18.965 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.965 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.966 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:18.966 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.530747+0000", 2024-09-21T21:41:18.966 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.016246564000000002 2024-09-21T21:41:18.966 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.966 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.966 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:18.966 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.530824+0000", 2024-09-21T21:41:18.966 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7109999999999999e-05 2024-09-21T21:41:18.966 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.966 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.966 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:18.966 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.530906+0000", 2024-09-21T21:41:18.966 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.1348999999999997e-05 2024-09-21T21:41:18.966 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:18.967 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:18.967 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:18.967 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.967 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.967 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4711 3.0 3:074cff57:::rbd_data.112145fae779.000000000000000c:a [read 2043904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:18.967 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.519604+0000", 2024-09-21T21:41:18.967 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 20.354530367999999, 2024-09-21T21:41:18.967 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013270252999999999, 2024-09-21T21:41:18.967 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:18.967 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:18.967 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:18.967 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:18.967 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:18.968 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:18.968 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4711 2024-09-21T21:41:18.968 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.968 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:18.968 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.968 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:18.968 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519604+0000", 2024-09-21T21:41:18.968 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:18.968 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.968 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.968 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:18.968 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519604+0000", 2024-09-21T21:41:18.968 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:18.969 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.969 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.969 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:18.969 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519605+0000", 2024-09-21T21:41:18.969 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8500000000000002e-07 2024-09-21T21:41:18.969 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.969 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.969 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:18.969 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519607+0000", 2024-09-21T21:41:18.969 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1789999999999998e-06 2024-09-21T21:41:18.969 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.969 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.969 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:18.969 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519613+0000", 2024-09-21T21:41:18.970 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2609999999999999e-06 2024-09-21T21:41:18.970 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.970 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.970 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:18.970 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519618+0000", 2024-09-21T21:41:18.970 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9579999999999998e-06 2024-09-21T21:41:18.970 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.970 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.970 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:18.970 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.532678+0000", 2024-09-21T21:41:18.970 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013059822 2024-09-21T21:41:18.970 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.970 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.971 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:18.971 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.532756+0000", 2024-09-21T21:41:18.971 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7467999999999998e-05 2024-09-21T21:41:18.971 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.971 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.971 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:18.971 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.532875+0000", 2024-09-21T21:41:18.971 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011917999999999999 2024-09-21T21:41:18.971 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:18.971 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:18.971 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:18.971 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.971 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.972 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4712 3.0 3:074cff57:::rbd_data.112145fae779.000000000000000c:a [read 2134016~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:18.972 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.519632+0000", 2024-09-21T21:41:18.972 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 20.354502884999999, 2024-09-21T21:41:18.972 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013480002, 2024-09-21T21:41:18.972 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:18.972 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:18.972 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:18.972 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:18.972 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:18.972 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:18.972 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4712 2024-09-21T21:41:18.972 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.972 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:18.972 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.972 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:18.973 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519632+0000", 2024-09-21T21:41:18.973 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:18.973 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.973 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.973 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:18.973 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519632+0000", 2024-09-21T21:41:18.973 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:18.973 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.973 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.973 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:18.973 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519632+0000", 2024-09-21T21:41:18.973 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2899999999999999e-07 2024-09-21T21:41:18.973 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.974 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.974 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:18.974 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519635+0000", 2024-09-21T21:41:18.974 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.6910000000000002e-06 2024-09-21T21:41:18.974 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.974 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.974 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:18.974 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519761+0000", 2024-09-21T21:41:18.974 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000125988 2024-09-21T21:41:18.974 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.974 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.974 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:18.974 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519767+0000", 2024-09-21T21:41:18.974 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9190000000000004e-06 2024-09-21T21:41:18.975 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.975 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.975 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:18.975 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.532905+0000", 2024-09-21T21:41:18.975 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013137970000000001 2024-09-21T21:41:18.975 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.975 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.975 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:18.975 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.532984+0000", 2024-09-21T21:41:18.975 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.8616999999999996e-05 2024-09-21T21:41:18.975 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.975 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.975 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:18.975 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.533112+0000", 2024-09-21T21:41:18.976 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000128388 2024-09-21T21:41:18.976 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:18.976 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:18.976 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:18.976 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.976 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.976 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4797.0:4713 3.0 3:074cff57:::rbd_data.112145fae779.000000000000000c:a [read 2191360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:18.976 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:40:58.519777+0000", 2024-09-21T21:41:18.976 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 20.354358184999999, 2024-09-21T21:41:18.976 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.01355251, 2024-09-21T21:41:18.976 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:18.976 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:18.976 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:18.976 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:18.976 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4797", 2024-09-21T21:41:18.977 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:18.977 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4713 2024-09-21T21:41:18.977 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.977 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:18.977 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.977 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:18.977 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519777+0000", 2024-09-21T21:41:18.977 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:18.977 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.977 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.977 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:18.977 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519777+0000", 2024-09-21T21:41:18.977 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:18.977 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.977 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.978 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:18.978 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519777+0000", 2024-09-21T21:41:18.978 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9900000000000001e-07 2024-09-21T21:41:18.978 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.978 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.978 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:18.978 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519780+0000", 2024-09-21T21:41:18.978 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.886e-06 2024-09-21T21:41:18.978 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.978 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.978 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:18.978 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519786+0000", 2024-09-21T21:41:18.978 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0859999999999998e-06 2024-09-21T21:41:18.978 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.979 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.979 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:18.979 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.519790+0000", 2024-09-21T21:41:18.979 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2400000000000001e-06 2024-09-21T21:41:18.979 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.979 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.979 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:18.979 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.533143+0000", 2024-09-21T21:41:18.979 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013352711 2024-09-21T21:41:18.979 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.979 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.979 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:18.979 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.533224+0000", 2024-09-21T21:41:18.979 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0675999999999998e-05 2024-09-21T21:41:18.979 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.980 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.980 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:18.980 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:40:58.533329+0000", 2024-09-21T21:41:18.980 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000105412 2024-09-21T21:41:18.980 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:18.980 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:18.980 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:18.980 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.980 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.980 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4824.0:1564 3.2 3:4647520c:::rbd_data.1121b97f2925.0000000000000006:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:18.980 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:41:18.980 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 17.238070311000001, 2024-09-21T21:41:18.980 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056097347999999998, 2024-09-21T21:41:18.980 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:18.980 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:18.981 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:18.981 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:18.981 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4824", 2024-09-21T21:41:18.981 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:18.981 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1564 2024-09-21T21:41:18.981 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.981 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:18.981 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.981 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:18.981 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:41:18.981 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:18.981 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.981 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.981 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:18.981 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:41:18.982 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:18.982 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.982 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.982 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:18.982 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:41:18.982 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6499999999999999e-07 2024-09-21T21:41:18.982 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.982 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.982 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:18.982 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636067+0000", 2024-09-21T21:41:18.982 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3879999999999998e-06 2024-09-21T21:41:18.982 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.982 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.982 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:18.982 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636075+0000", 2024-09-21T21:41:18.983 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5480000000000004e-06 2024-09-21T21:41:18.983 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.983 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.983 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:18.983 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636080+0000", 2024-09-21T21:41:18.983 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.339e-06 2024-09-21T21:41:18.983 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.983 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.983 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:18.983 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636133+0000", 2024-09-21T21:41:18.983 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2930000000000003e-05 2024-09-21T21:41:18.983 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.984 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.984 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:18.984 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636265+0000", 2024-09-21T21:41:18.984 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000132058 2024-09-21T21:41:18.984 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:18.984 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:18.984 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:18.984 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.692162+0000", 2024-09-21T21:41:18.984 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055896620000000001 2024-09-21T21:41:18.984 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:18.984 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:18.984 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:18.984 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:18.984 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:18.984 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:18.987 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T21:41:18.987 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3: {} 2024-09-21T21:41:18.987 INFO:teuthology.orchestra.run.smithi081.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T21:41:18.988 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4: {} 2024-09-21T21:41:18.988 INFO:teuthology.orchestra.run.smithi081.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T21:41:18.989 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5: {} 2024-09-21T21:41:18.990 INFO:teuthology.orchestra.run.smithi081.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T21:41:18.991 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:18.984+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:18.992 INFO:teuthology.orchestra.run.smithi164.stdout:2461 write 0xd2d1200 thru 0xd2d993c (0x873d bytes) 2024-09-21T21:41:18.992 INFO:teuthology.orchestra.run.smithi164.stdout:2466 read 0xdaaa00 thru 0xdadd86 (0x3387 bytes) 2024-09-21T21:41:18.992 INFO:teuthology.orchestra.run.smithi164.stdout:2468 write 0xca2a400 thru 0xca2e552 (0x4153 bytes) 2024-09-21T21:41:18.992 INFO:teuthology.orchestra.run.smithi164.stdout:2469 read 0x65ce00 thru 0x669c02 (0xce03 bytes) 2024-09-21T21:41:18.992 INFO:teuthology.orchestra.run.smithi164.stdout:2475 read 0x554be00 thru 0x5551a08 (0x5c09 bytes) 2024-09-21T21:41:18.992 INFO:teuthology.orchestra.run.smithi164.stdout:2476 punch from 0x7ebc800 to 0x7ec7000, (0xa800 bytes) 2024-09-21T21:41:18.992 INFO:teuthology.orchestra.run.smithi164.stdout:2480 punch from 0x7b5ae00 to 0x7b67200, (0xc400 bytes) 2024-09-21T21:41:18.993 INFO:teuthology.orchestra.run.smithi164.stdout:2482 trunc from 0xd2d9a00 to 0x4617600 2024-09-21T21:41:18.993 INFO:teuthology.orchestra.run.smithi164.stdout:2483 read 0x2125200 thru 0x212af92 (0x5d93 bytes) 2024-09-21T21:41:18.993 INFO:teuthology.orchestra.run.smithi164.stdout:2487 trunc from 0x4617600 to 0x5d0b800 2024-09-21T21:41:18.993 INFO:teuthology.orchestra.run.smithi164.stdout:2492 read 0x5151000 thru 0x515312f (0x2130 bytes) 2024-09-21T21:41:18.993 INFO:teuthology.orchestra.run.smithi164.stdout:2494 write 0xcde0a00 thru 0xcdef86d (0xee6e bytes) 2024-09-21T21:41:18.993 INFO:teuthology.orchestra.run.smithi164.stdout:2495 write 0x60db800 thru 0x60dc55c (0xd5d bytes) 2024-09-21T21:41:18.993 INFO:teuthology.orchestra.run.smithi164.stdout:2498 write 0xdcb800 thru 0xdd9c23 (0xe424 bytes) 2024-09-21T21:41:18.993 INFO:teuthology.orchestra.run.smithi164.stdout:2502 write 0x200fe00 thru 0x2013a58 (0x3c59 bytes) 2024-09-21T21:41:18.993 INFO:teuthology.orchestra.run.smithi164.stdout:2503 write 0x93f6e00 thru 0x93fc3ba (0x55bb bytes) 2024-09-21T21:41:18.993 INFO:teuthology.orchestra.run.smithi164.stdout:2506 write 0x7d81400 thru 0x7d8f7d7 (0xe3d8 bytes) 2024-09-21T21:41:18.993 INFO:teuthology.orchestra.run.smithi164.stdout:2507 read 0xcbd8200 thru 0xcbe6885 (0xe686 bytes) 2024-09-21T21:41:18.993 INFO:teuthology.orchestra.run.smithi164.stdout:2508 write 0xabd5800 thru 0xabdb3b1 (0x5bb2 bytes) 2024-09-21T21:41:18.993 INFO:teuthology.orchestra.run.smithi164.stdout:2509 read 0x85e8800 thru 0x85f4b12 (0xc313 bytes) 2024-09-21T21:41:18.993 INFO:teuthology.orchestra.run.smithi164.stdout:2511 punch from 0x62c3800 to 0x62cd200, (0x9a00 bytes) 2024-09-21T21:41:18.993 INFO:teuthology.orchestra.run.smithi164.stdout:2513 write 0x1073200 thru 0x108099b (0xd79c bytes) 2024-09-21T21:41:18.994 INFO:teuthology.orchestra.run.smithi164.stdout:2519 punch from 0x4293800 to 0x4297a00, (0x4200 bytes) 2024-09-21T21:41:18.994 INFO:teuthology.orchestra.run.smithi164.stdout:2520 write 0x7875800 thru 0x7879048 (0x3849 bytes) 2024-09-21T21:41:18.994 INFO:teuthology.orchestra.run.smithi164.stdout:2521 write 0x7b48c00 thru 0x7b5702f (0xe430 bytes) 2024-09-21T21:41:18.994 INFO:teuthology.orchestra.run.smithi164.stdout:2524 read 0x911b200 thru 0x9124216 (0x9017 bytes) 2024-09-21T21:41:18.994 INFO:teuthology.orchestra.run.smithi164.stdout:2526 write 0x5ace000 thru 0x5ad3eb1 (0x5eb2 bytes) 2024-09-21T21:41:18.994 INFO:teuthology.orchestra.run.smithi164.stdout:2529 read 0x59d0e00 thru 0x59d571d (0x491e bytes) 2024-09-21T21:41:18.994 INFO:teuthology.orchestra.run.smithi164.stdout:2530 read 0x453b400 thru 0x45492c4 (0xdec5 bytes) 2024-09-21T21:41:18.994 INFO:teuthology.orchestra.run.smithi164.stdout:2531 write 0xe036200 thru 0xe03df39 (0x7d3a bytes) 2024-09-21T21:41:18.994 INFO:teuthology.orchestra.run.smithi164.stdout:2533 write 0x9c92600 thru 0x9c9c774 (0xa175 bytes) 2024-09-21T21:41:18.994 INFO:teuthology.orchestra.run.smithi164.stdout:2534 punch from 0x854d000 to 0x8553600, (0x6600 bytes) 2024-09-21T21:41:18.994 INFO:teuthology.orchestra.run.smithi164.stdout:2537 punch from 0x853e200 to 0x8545a00, (0x7800 bytes) 2024-09-21T21:41:18.994 INFO:teuthology.orchestra.run.smithi164.stdout:2540 write 0x54eb200 thru 0x54f8d5b (0xdb5c bytes) 2024-09-21T21:41:18.994 INFO:teuthology.orchestra.run.smithi164.stdout:2541 read 0x9a97800 thru 0x9a9b6bd (0x3ebe bytes) 2024-09-21T21:41:18.994 INFO:teuthology.orchestra.run.smithi164.stdout:2543 read 0x655de00 thru 0x65608df (0x2ae0 bytes) 2024-09-21T21:41:18.995 INFO:teuthology.orchestra.run.smithi164.stdout:2545 punch from 0x6505c00 to 0x650fa00, (0x9e00 bytes) 2024-09-21T21:41:18.995 INFO:teuthology.orchestra.run.smithi164.stdout:2547 read 0xc5a0000 thru 0xc5a48a8 (0x48a9 bytes) 2024-09-21T21:41:18.995 INFO:teuthology.orchestra.run.smithi164.stdout:2548 read 0x1553400 thru 0x15599e6 (0x65e7 bytes) 2024-09-21T21:41:18.995 INFO:teuthology.orchestra.run.smithi164.stdout:2549 punch from 0xc1cd800 to 0xc1d6000, (0x8800 bytes) 2024-09-21T21:41:18.995 INFO:teuthology.orchestra.run.smithi164.stdout:2550 read 0x35ac600 thru 0x35b0311 (0x3d12 bytes) 2024-09-21T21:41:18.995 INFO:teuthology.orchestra.run.smithi164.stdout:2551 trunc from 0xe03e000 to 0x9fda800 2024-09-21T21:41:19.013 INFO:teuthology.orchestra.run.smithi164.stdout:2552 punch from 0x52a6c00 to 0x52a9400, (0x2800 bytes) 2024-09-21T21:41:19.014 INFO:teuthology.orchestra.run.smithi164.stdout:2553 write 0x89f3800 thru 0x89f60e3 (0x28e4 bytes) 2024-09-21T21:41:19.024 INFO:teuthology.orchestra.run.smithi164.stdout:2554 trunc from 0x9fda800 to 0x1516c00 2024-09-21T21:41:19.074 INFO:teuthology.orchestra.run.smithi164.stdout:2555 punch from 0x1f2000 to 0x1f2200, (0x200 bytes) 2024-09-21T21:41:19.080 INFO:teuthology.orchestra.run.smithi164.stdout:2556 trunc from 0x1516c00 to 0x58a4a00 2024-09-21T21:41:19.086 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:19.085+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:19.099 INFO:teuthology.orchestra.run.smithi164.stdout:2558 write 0x6149600 thru 0x614b15c (0x1b5d bytes) 2024-09-21T21:41:19.100 INFO:teuthology.orchestra.run.smithi164.stdout:2563 punch from 0x4443a00 to 0x444ac00, (0x7200 bytes) 2024-09-21T21:41:19.100 INFO:teuthology.orchestra.run.smithi164.stdout:2564 write 0xebde00 thru 0xeca84e (0xca4f bytes) 2024-09-21T21:41:19.102 INFO:teuthology.orchestra.run.smithi164.stdout:2565 read 0x5039800 thru 0x503d944 (0x4145 bytes) 2024-09-21T21:41:19.109 INFO:teuthology.orchestra.run.smithi164.stdout:2566 trunc from 0x614b200 to 0xc0fa200 2024-09-21T21:41:19.120 INFO:teuthology.orchestra.run.smithi164.stdout:2567 read 0xb59ae00 thru 0xb5a706b (0xc26c bytes) 2024-09-21T21:41:19.120 INFO:teuthology.orchestra.run.smithi164.stdout:2568 write 0x1a0c400 thru 0x1a0e8b9 (0x24ba bytes) 2024-09-21T21:41:19.120 INFO:teuthology.orchestra.run.smithi164.stdout:2569 punch from 0x655b800 to 0x655fe00, (0x4600 bytes) 2024-09-21T21:41:19.132 INFO:teuthology.orchestra.run.smithi164.stdout:2570 write 0xe756600 thru 0xe75883c (0x223d bytes) 2024-09-21T21:41:19.132 INFO:teuthology.orchestra.run.smithi164.stdout:2572 write 0x76a6600 thru 0x76b5e68 (0xf869 bytes) 2024-09-21T21:41:19.133 INFO:teuthology.orchestra.run.smithi164.stdout:2573 punch from 0xb0f3200 to 0xb0f5a00, (0x2800 bytes) 2024-09-21T21:41:19.133 INFO:teuthology.orchestra.run.smithi164.stdout:2574 read 0xa249a00 thru 0xa251961 (0x7f62 bytes) 2024-09-21T21:41:19.133 INFO:teuthology.orchestra.run.smithi164.stdout:2575 read 0x1a4bc00 thru 0x1a56d01 (0xb102 bytes) 2024-09-21T21:41:19.135 INFO:teuthology.orchestra.run.smithi164.stdout:2576 read 0x4797c00 thru 0x479c5dd (0x49de bytes) 2024-09-21T21:41:19.135 INFO:teuthology.orchestra.run.smithi164.stdout:2577 write 0x8f1f000 thru 0x8f2b0a1 (0xc0a2 bytes) 2024-09-21T21:41:19.135 INFO:teuthology.orchestra.run.smithi164.stdout:2579 read 0x8ae5a00 thru 0x8aed30f (0x7910 bytes) 2024-09-21T21:41:19.136 INFO:teuthology.orchestra.run.smithi164.stdout:2582 punch from 0xb8b800 to 0xb9b200, (0xfa00 bytes) 2024-09-21T21:41:19.136 INFO:teuthology.orchestra.run.smithi164.stdout:2583 write 0xb33d600 thru 0xb34c0dd (0xeade bytes) 2024-09-21T21:41:19.146 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:19.146 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:41:19.146 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 0 2024-09-21T21:41:19.146 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:19.149 INFO:teuthology.orchestra.run.smithi164.stdout:2585 trunc from 0xe758a00 to 0x3d17a00 2024-09-21T21:41:19.156 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T21:41:19.172 INFO:teuthology.orchestra.run.smithi164.stdout:2590 punch from 0x1dbb400 to 0x1dc1200, (0x5e00 bytes) 2024-09-21T21:41:19.176 INFO:teuthology.orchestra.run.smithi164.stdout:2591 trunc from 0x3d17a00 to 0x18d9c00 2024-09-21T21:41:19.191 INFO:teuthology.orchestra.run.smithi164.stdout:2594 punch from 0x87a000 to 0x887400, (0xd400 bytes) 2024-09-21T21:41:19.193 INFO:teuthology.orchestra.run.smithi164.stdout:2595 read 0x82e000 thru 0x82e2b2 (0x2b3 bytes) 2024-09-21T21:41:19.196 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:19.195+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:19.211 INFO:teuthology.orchestra.run.smithi164.stdout:2596 write 0x7e4fe00 thru 0x7e59c54 (0x9e55 bytes) 2024-09-21T21:41:19.212 INFO:teuthology.orchestra.run.smithi164.stdout:2597 trunc from 0x7e59e00 to 0x76a3c00 2024-09-21T21:41:19.225 INFO:teuthology.orchestra.run.smithi164.stdout:2598 punch from 0x3a73a00 to 0x3a7ae00, (0x7400 bytes) 2024-09-21T21:41:19.233 INFO:teuthology.orchestra.run.smithi164.stdout:2599 trunc from 0x76a3c00 to 0x87d9400 2024-09-21T21:41:19.243 INFO:teuthology.orchestra.run.smithi164.stdout:2601 punch from 0x4b49000 to 0x4b53800, (0xa800 bytes) 2024-09-21T21:41:19.262 INFO:teuthology.orchestra.run.smithi164.stdout:2604 write 0x8feb400 thru 0x8fec525 (0x1126 bytes) 2024-09-21T21:41:19.269 INFO:teuthology.orchestra.run.smithi164.stdout:2607 read 0x14de00 thru 0x15cde5 (0xefe6 bytes) 2024-09-21T21:41:19.270 INFO:teuthology.orchestra.run.smithi164.stdout:2608 read 0x15af400 thru 0x15ba40c (0xb00d bytes) 2024-09-21T21:41:19.270 INFO:teuthology.orchestra.run.smithi164.stdout:2609 read 0x74ec000 thru 0x74f1361 (0x5362 bytes) 2024-09-21T21:41:19.271 INFO:teuthology.orchestra.run.smithi164.stdout:2610 clone 16 order 25 su 131072 sc 13 2024-09-21T21:41:19.288 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:19.287+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:19.305 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:19.305 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:41:19.305 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:41:19.305 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:41:19.306 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:19.314 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T21:41:19.390 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:19.389+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:19.460 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:19.461 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 20, 2024-09-21T21:41:19.461 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 600, 2024-09-21T21:41:19.461 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:41:19.461 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.461 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4840.0:4906 3.1e 3:7e975c4e:::rbd_data.11214ff8bd9b.0000000000000024:10 [read 51200~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T21:41:19.461 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:16.447629+0000", 2024-09-21T21:41:19.462 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.010195526, 2024-09-21T21:41:19.462 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.016683802000000001, 2024-09-21T21:41:19.462 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:19.462 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:19.462 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:19.462 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:19.463 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4840", 2024-09-21T21:41:19.463 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1293784138", 2024-09-21T21:41:19.463 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4906 2024-09-21T21:41:19.463 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.463 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:19.463 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.464 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:19.464 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.447629+0000", 2024-09-21T21:41:19.464 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:19.464 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.464 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.464 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:19.464 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.447629+0000", 2024-09-21T21:41:19.465 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:19.465 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.465 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.465 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:19.465 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.447629+0000", 2024-09-21T21:41:19.465 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.2899999999999999e-07 2024-09-21T21:41:19.466 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.466 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.466 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:19.466 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.447631+0000", 2024-09-21T21:41:19.466 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0760000000000001e-06 2024-09-21T21:41:19.466 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.467 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.467 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:19.467 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.447637+0000", 2024-09-21T21:41:19.467 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1079999999999998e-06 2024-09-21T21:41:19.467 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.467 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.467 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:19.467 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.447642+0000", 2024-09-21T21:41:19.468 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6179999999999997e-06 2024-09-21T21:41:19.468 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.468 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.468 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:19.468 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.464212+0000", 2024-09-21T21:41:19.469 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.016570465999999999 2024-09-21T21:41:19.469 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.469 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.469 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:19.469 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.464261+0000", 2024-09-21T21:41:19.469 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9070999999999998e-05 2024-09-21T21:41:19.469 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.469 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.469 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:19.470 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.464312+0000", 2024-09-21T21:41:19.470 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1134000000000002e-05 2024-09-21T21:41:19.470 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:19.470 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:19.470 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:19.470 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.470 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.470 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4840.0:4907 3.1e 3:7e975c4e:::rbd_data.11214ff8bd9b.0000000000000024:10 [read 63488~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T21:41:19.470 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:16.447648+0000", 2024-09-21T21:41:19.470 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.0101760460000002, 2024-09-21T21:41:19.470 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.016792049, 2024-09-21T21:41:19.470 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:19.470 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:19.471 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:19.471 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:19.471 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4840", 2024-09-21T21:41:19.471 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1293784138", 2024-09-21T21:41:19.471 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4907 2024-09-21T21:41:19.471 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.471 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:19.471 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.471 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:19.471 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.447648+0000", 2024-09-21T21:41:19.471 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:19.471 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.471 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.471 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:19.472 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.447648+0000", 2024-09-21T21:41:19.472 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:19.472 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.472 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.472 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:19.472 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.447648+0000", 2024-09-21T21:41:19.472 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.01e-07 2024-09-21T21:41:19.472 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.472 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.472 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:19.472 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.447651+0000", 2024-09-21T21:41:19.472 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1019999999999999e-06 2024-09-21T21:41:19.472 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.472 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.473 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:19.473 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.447656+0000", 2024-09-21T21:41:19.473 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5419999999999996e-06 2024-09-21T21:41:19.473 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.473 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.473 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:19.473 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.447660+0000", 2024-09-21T21:41:19.473 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.206e-06 2024-09-21T21:41:19.473 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.473 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.473 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:19.473 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.464339+0000", 2024-09-21T21:41:19.473 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.016678407999999999 2024-09-21T21:41:19.473 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.474 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.474 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:19.474 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.464376+0000", 2024-09-21T21:41:19.474 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7058000000000003e-05 2024-09-21T21:41:19.474 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.474 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.474 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:19.474 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.464440+0000", 2024-09-21T21:41:19.474 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4331999999999993e-05 2024-09-21T21:41:19.474 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:19.474 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:19.474 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:19.474 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.475 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.475 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4840.0:4908 3.1e 3:7e975c4e:::rbd_data.11214ff8bd9b.0000000000000024:10 [read 75776~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T21:41:19.475 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:16.447666+0000", 2024-09-21T21:41:19.475 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.0101576319999999, 2024-09-21T21:41:19.475 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.016920369000000001, 2024-09-21T21:41:19.475 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:19.475 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:19.475 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:19.475 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:19.475 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4840", 2024-09-21T21:41:19.475 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1293784138", 2024-09-21T21:41:19.475 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4908 2024-09-21T21:41:19.475 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.475 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:19.476 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.476 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:19.476 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.447666+0000", 2024-09-21T21:41:19.476 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:19.476 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.476 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.476 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:19.476 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.447666+0000", 2024-09-21T21:41:19.476 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:19.476 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.476 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.476 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:19.476 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.447667+0000", 2024-09-21T21:41:19.476 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8799999999999998e-07 2024-09-21T21:41:19.477 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.477 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.477 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:19.477 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.447669+0000", 2024-09-21T21:41:19.477 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.103e-06 2024-09-21T21:41:19.477 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.477 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.477 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:19.477 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.447675+0000", 2024-09-21T21:41:19.477 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1870000000000002e-06 2024-09-21T21:41:19.477 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.477 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.477 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:19.477 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.447680+0000", 2024-09-21T21:41:19.478 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7389999999999999e-06 2024-09-21T21:41:19.478 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.478 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.478 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:19.478 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.464473+0000", 2024-09-21T21:41:19.478 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.016792677999999998 2024-09-21T21:41:19.478 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.478 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.478 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:19.478 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.464515+0000", 2024-09-21T21:41:19.478 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1968999999999997e-05 2024-09-21T21:41:19.478 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.478 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.479 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:19.479 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.464587+0000", 2024-09-21T21:41:19.479 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.2305000000000007e-05 2024-09-21T21:41:19.479 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:19.479 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:19.479 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:19.479 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.479 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.479 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4840.0:4936 3.1e 3:788f7146:::rbd_data.11214ff8bd9b.0000000000000011:10 [read 223232~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T21:41:19.479 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:16.447831+0000", 2024-09-21T21:41:19.479 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.009993347, 2024-09-21T21:41:19.479 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.016935091999999999, 2024-09-21T21:41:19.480 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:19.480 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:19.480 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:19.480 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:19.480 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4840", 2024-09-21T21:41:19.480 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1293784138", 2024-09-21T21:41:19.480 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4936 2024-09-21T21:41:19.480 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.480 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:19.480 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.480 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:19.480 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.447831+0000", 2024-09-21T21:41:19.481 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:19.481 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.481 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.481 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:19.481 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.447831+0000", 2024-09-21T21:41:19.481 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:19.481 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.481 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.481 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:19.481 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.447831+0000", 2024-09-21T21:41:19.481 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8799999999999998e-07 2024-09-21T21:41:19.481 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.481 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.481 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:19.481 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.447833+0000", 2024-09-21T21:41:19.482 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0980000000000001e-06 2024-09-21T21:41:19.482 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.482 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.482 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:19.482 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.447839+0000", 2024-09-21T21:41:19.482 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2829999999999998e-06 2024-09-21T21:41:19.482 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.482 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.482 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:19.482 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.447845+0000", 2024-09-21T21:41:19.482 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0919999999999997e-06 2024-09-21T21:41:19.482 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.483 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.483 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:19.483 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.464614+0000", 2024-09-21T21:41:19.483 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.016769017000000001 2024-09-21T21:41:19.483 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.483 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.483 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:19.483 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.464676+0000", 2024-09-21T21:41:19.483 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2261999999999994e-05 2024-09-21T21:41:19.483 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.483 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.483 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:19.484 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.464766+0000", 2024-09-21T21:41:19.484 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.9951999999999995e-05 2024-09-21T21:41:19.484 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:19.484 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:19.484 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:19.484 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.484 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.484 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4840.0:4937 3.1e 3:788f7146:::rbd_data.11214ff8bd9b.0000000000000011:10 [read 235520~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T21:41:19.484 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:16.447850+0000", 2024-09-21T21:41:19.484 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.009973751, 2024-09-21T21:41:19.484 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.017027231, 2024-09-21T21:41:19.484 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:19.485 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:19.485 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:19.485 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:19.485 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4840", 2024-09-21T21:41:19.485 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1293784138", 2024-09-21T21:41:19.485 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4937 2024-09-21T21:41:19.485 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.485 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:19.485 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.485 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:19.485 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.447850+0000", 2024-09-21T21:41:19.485 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:19.485 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.486 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.486 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:19.486 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.447850+0000", 2024-09-21T21:41:19.486 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:19.486 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.486 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.486 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:19.486 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.447851+0000", 2024-09-21T21:41:19.487 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2399999999999999e-07 2024-09-21T21:41:19.487 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.487 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.487 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:19.487 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.447853+0000", 2024-09-21T21:41:19.487 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.12e-06 2024-09-21T21:41:19.487 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.487 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.487 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:19.487 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.447859+0000", 2024-09-21T21:41:19.487 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0510000000000002e-06 2024-09-21T21:41:19.487 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.488 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.488 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:19.488 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.447864+0000", 2024-09-21T21:41:19.488 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6580000000000001e-06 2024-09-21T21:41:19.488 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.488 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.488 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:19.488 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.464793+0000", 2024-09-21T21:41:19.488 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.016929036000000001 2024-09-21T21:41:19.488 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.488 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.488 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:19.488 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.464829+0000", 2024-09-21T21:41:19.489 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6121999999999997e-05 2024-09-21T21:41:19.489 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.489 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.489 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:19.489 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.464878+0000", 2024-09-21T21:41:19.489 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8819999999999997e-05 2024-09-21T21:41:19.489 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:19.489 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:19.489 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:19.489 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.489 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.489 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4840.0:4938 3.1e 3:788f7146:::rbd_data.11214ff8bd9b.0000000000000011:10 [read 247808~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T21:41:19.489 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:16.447870+0000", 2024-09-21T21:41:19.489 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.0099544549999999, 2024-09-21T21:41:19.490 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.017157932000000001, 2024-09-21T21:41:19.490 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:19.490 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:19.490 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:19.490 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:19.490 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4840", 2024-09-21T21:41:19.490 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1293784138", 2024-09-21T21:41:19.490 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4938 2024-09-21T21:41:19.490 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.490 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:19.490 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.490 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:19.490 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.447870+0000", 2024-09-21T21:41:19.491 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:19.491 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.491 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.491 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:19.491 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.447870+0000", 2024-09-21T21:41:19.491 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:19.491 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.491 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.491 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:19.491 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.447870+0000", 2024-09-21T21:41:19.491 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7599999999999998e-07 2024-09-21T21:41:19.491 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.492 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.492 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:19.492 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.447872+0000", 2024-09-21T21:41:19.492 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0899999999999999e-06 2024-09-21T21:41:19.492 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.492 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.492 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:19.492 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.447879+0000", 2024-09-21T21:41:19.492 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5919999999999997e-06 2024-09-21T21:41:19.492 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.492 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.492 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:19.492 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.447883+0000", 2024-09-21T21:41:19.492 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6859999999999997e-06 2024-09-21T21:41:19.493 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.493 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.493 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:19.493 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.464903+0000", 2024-09-21T21:41:19.493 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.017020094 2024-09-21T21:41:19.493 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.493 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.493 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:19.493 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.464945+0000", 2024-09-21T21:41:19.493 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1628000000000002e-05 2024-09-21T21:41:19.493 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.493 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.493 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:19.494 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.465028+0000", 2024-09-21T21:41:19.494 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.2466000000000006e-05 2024-09-21T21:41:19.494 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:19.494 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:19.494 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:19.494 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.494 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.494 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4840.0:4939 3.1e 3:788f7146:::rbd_data.11214ff8bd9b.0000000000000011:10 [read 260096~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T21:41:19.494 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:16.447890+0000", 2024-09-21T21:41:19.494 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.0099344280000002, 2024-09-21T21:41:19.494 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.017476058999999999, 2024-09-21T21:41:19.495 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:19.495 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:19.495 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:19.495 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:19.495 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4840", 2024-09-21T21:41:19.495 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1293784138", 2024-09-21T21:41:19.495 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4939 2024-09-21T21:41:19.495 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.495 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:19.495 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.495 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:19.495 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.447890+0000", 2024-09-21T21:41:19.495 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:19.495 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.496 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.496 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:19.496 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.447890+0000", 2024-09-21T21:41:19.496 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:19.496 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.496 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.496 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:19.496 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.447890+0000", 2024-09-21T21:41:19.496 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1100000000000001e-07 2024-09-21T21:41:19.496 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.496 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.497 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:19.497 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.447892+0000", 2024-09-21T21:41:19.497 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.4109999999999999e-06 2024-09-21T21:41:19.497 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.497 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.497 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:19.497 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.447898+0000", 2024-09-21T21:41:19.497 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.767e-06 2024-09-21T21:41:19.497 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.497 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.497 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:19.497 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.447903+0000", 2024-09-21T21:41:19.497 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3630000000000004e-06 2024-09-21T21:41:19.498 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.498 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.498 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:19.498 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.465057+0000", 2024-09-21T21:41:19.498 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.017154106999999998 2024-09-21T21:41:19.498 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.498 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.498 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:19.498 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.465120+0000", 2024-09-21T21:41:19.498 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3748e-05 2024-09-21T21:41:19.498 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.498 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.498 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:19.499 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.465366+0000", 2024-09-21T21:41:19.499 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000245252 2024-09-21T21:41:19.499 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:19.499 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:19.499 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:19.499 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.499 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.499 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4840.0:4994 3.1e 3:7fca165a:::rbd_data.11214ff8bd9b.000000000000001e:10 [read 10240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T21:41:19.499 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:16.448139+0000", 2024-09-21T21:41:19.499 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.0096848089999999, 2024-09-21T21:41:19.499 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.017419482999999999, 2024-09-21T21:41:19.499 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:19.499 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:19.500 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:19.500 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:19.500 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4840", 2024-09-21T21:41:19.500 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1293784138", 2024-09-21T21:41:19.500 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4994 2024-09-21T21:41:19.500 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.500 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:19.500 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.500 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:19.501 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.448139+0000", 2024-09-21T21:41:19.501 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:19.501 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.501 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.501 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:19.501 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.448139+0000", 2024-09-21T21:41:19.501 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:19.501 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.501 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.502 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:19.502 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.448140+0000", 2024-09-21T21:41:19.502 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6699999999999999e-07 2024-09-21T21:41:19.502 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.502 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.502 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:19.502 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.448142+0000", 2024-09-21T21:41:19.502 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1909999999999999e-06 2024-09-21T21:41:19.502 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.503 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.503 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:19.503 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.448148+0000", 2024-09-21T21:41:19.503 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4459999999999998e-06 2024-09-21T21:41:19.503 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.503 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.503 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:19.503 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.448153+0000", 2024-09-21T21:41:19.504 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.933e-06 2024-09-21T21:41:19.504 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.504 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.504 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:19.504 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.465398+0000", 2024-09-21T21:41:19.504 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.017245084000000001 2024-09-21T21:41:19.504 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.504 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.504 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:19.504 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.465472+0000", 2024-09-21T21:41:19.505 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.4010999999999993e-05 2024-09-21T21:41:19.505 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.505 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.505 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:19.505 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.465559+0000", 2024-09-21T21:41:19.505 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.6451000000000003e-05 2024-09-21T21:41:19.505 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:19.506 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:19.506 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:19.506 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.506 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.506 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4840.0:4995 3.1e 3:7fca165a:::rbd_data.11214ff8bd9b.000000000000001e:10 [read 22528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T21:41:19.506 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:16.448159+0000", 2024-09-21T21:41:19.506 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.009664779, 2024-09-21T21:41:19.506 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.017571962999999999, 2024-09-21T21:41:19.506 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:19.507 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:19.507 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:19.507 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:19.507 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4840", 2024-09-21T21:41:19.507 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1293784138", 2024-09-21T21:41:19.507 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4995 2024-09-21T21:41:19.507 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.508 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:19.508 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.508 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:19.509 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.448159+0000", 2024-09-21T21:41:19.509 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:19.509 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.510 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.510 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:19.510 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.448159+0000", 2024-09-21T21:41:19.510 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:19.510 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.510 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.510 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:19.510 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.448160+0000", 2024-09-21T21:41:19.510 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1699999999999999e-07 2024-09-21T21:41:19.510 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.510 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.510 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:19.510 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.448162+0000", 2024-09-21T21:41:19.511 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9769999999999999e-06 2024-09-21T21:41:19.511 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.511 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.511 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:19.511 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.448168+0000", 2024-09-21T21:41:19.511 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2090000000000002e-06 2024-09-21T21:41:19.511 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.511 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.511 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:19.511 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.448173+0000", 2024-09-21T21:41:19.511 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8500000000000002e-06 2024-09-21T21:41:19.511 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.511 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.511 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:19.511 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.465586+0000", 2024-09-21T21:41:19.512 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.017412928000000001 2024-09-21T21:41:19.512 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.512 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.512 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:19.512 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.465649+0000", 2024-09-21T21:41:19.512 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3058999999999994e-05 2024-09-21T21:41:19.512 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.512 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.512 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:19.512 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.465731+0000", 2024-09-21T21:41:19.512 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.2522999999999996e-05 2024-09-21T21:41:19.512 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:19.512 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:19.512 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:19.512 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.513 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.513 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4840.0:4996 3.1e 3:7fca165a:::rbd_data.11214ff8bd9b.000000000000001e:10 [read 34816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T21:41:19.513 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:16.448179+0000", 2024-09-21T21:41:19.513 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.009645227, 2024-09-21T21:41:19.513 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.017833287999999999, 2024-09-21T21:41:19.513 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:19.513 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:19.513 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:19.513 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:19.513 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4840", 2024-09-21T21:41:19.513 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1293784138", 2024-09-21T21:41:19.513 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4996 2024-09-21T21:41:19.513 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.513 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:19.513 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.514 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:19.514 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.448179+0000", 2024-09-21T21:41:19.514 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:19.514 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.514 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.514 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:19.514 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.448179+0000", 2024-09-21T21:41:19.514 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:19.514 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.514 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.514 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:19.514 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.448179+0000", 2024-09-21T21:41:19.514 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1699999999999999e-07 2024-09-21T21:41:19.514 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.515 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.515 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:19.515 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.448181+0000", 2024-09-21T21:41:19.515 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2230000000000001e-06 2024-09-21T21:41:19.515 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.515 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.515 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:19.515 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.448188+0000", 2024-09-21T21:41:19.515 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1840000000000004e-06 2024-09-21T21:41:19.515 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.515 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.515 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:19.515 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.448193+0000", 2024-09-21T21:41:19.515 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8459999999999999e-06 2024-09-21T21:41:19.516 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.516 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.516 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:19.516 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.465759+0000", 2024-09-21T21:41:19.516 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.017566892000000001 2024-09-21T21:41:19.516 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.516 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.516 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:19.516 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.465825+0000", 2024-09-21T21:41:19.516 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5683000000000001e-05 2024-09-21T21:41:19.516 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.516 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.516 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:19.516 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.466012+0000", 2024-09-21T21:41:19.516 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00018704299999999999 2024-09-21T21:41:19.517 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:19.517 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:19.517 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:19.517 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.517 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.517 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4840.0:4997 3.1e 3:7fca165a:::rbd_data.11214ff8bd9b.000000000000001e:10 [read 47104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T21:41:19.517 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:16.448198+0000", 2024-09-21T21:41:19.517 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.0096258059999998, 2024-09-21T21:41:19.517 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.017993156, 2024-09-21T21:41:19.517 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:19.517 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:19.518 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:19.518 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:19.518 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4840", 2024-09-21T21:41:19.518 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1293784138", 2024-09-21T21:41:19.518 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4997 2024-09-21T21:41:19.518 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.518 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:19.518 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.518 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:19.518 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.448198+0000", 2024-09-21T21:41:19.518 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:19.518 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.518 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.518 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:19.519 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.448198+0000", 2024-09-21T21:41:19.519 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:19.519 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.519 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.519 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:19.519 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.448199+0000", 2024-09-21T21:41:19.519 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.0499999999999999e-07 2024-09-21T21:41:19.519 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.519 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.519 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:19.519 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.448201+0000", 2024-09-21T21:41:19.519 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.889e-06 2024-09-21T21:41:19.519 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.519 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.520 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:19.520 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.448206+0000", 2024-09-21T21:41:19.520 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6860000000000003e-06 2024-09-21T21:41:19.520 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.520 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.520 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:19.520 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.448211+0000", 2024-09-21T21:41:19.520 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7559999999999999e-06 2024-09-21T21:41:19.520 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.520 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.520 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:19.520 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.466040+0000", 2024-09-21T21:41:19.520 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.017829126000000001 2024-09-21T21:41:19.520 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.520 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.521 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:19.521 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.466105+0000", 2024-09-21T21:41:19.521 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4764999999999994e-05 2024-09-21T21:41:19.521 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.521 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.521 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:19.521 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.466191+0000", 2024-09-21T21:41:19.521 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.6528999999999998e-05 2024-09-21T21:41:19.521 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:19.521 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:19.521 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:19.521 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.521 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.521 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4840.0:4998 3.1e 3:7fca165a:::rbd_data.11214ff8bd9b.000000000000001e:10 [read 59392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T21:41:19.522 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:16.448217+0000", 2024-09-21T21:41:19.522 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.0096072330000001, 2024-09-21T21:41:19.522 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.018148931, 2024-09-21T21:41:19.522 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:19.522 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:19.522 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:19.522 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:19.522 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4840", 2024-09-21T21:41:19.522 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1293784138", 2024-09-21T21:41:19.523 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4998 2024-09-21T21:41:19.523 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.523 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:19.523 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.523 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:19.523 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.448217+0000", 2024-09-21T21:41:19.523 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:19.523 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.524 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.524 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:19.524 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.448217+0000", 2024-09-21T21:41:19.524 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:19.524 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.524 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.524 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:19.524 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.448217+0000", 2024-09-21T21:41:19.524 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.9099999999999999e-07 2024-09-21T21:41:19.524 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.524 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.524 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:19.524 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.448220+0000", 2024-09-21T21:41:19.524 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2800000000000002e-06 2024-09-21T21:41:19.525 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.525 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.525 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:19.525 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.448226+0000", 2024-09-21T21:41:19.525 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0410000000000003e-06 2024-09-21T21:41:19.525 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.525 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.525 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:19.525 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.448230+0000", 2024-09-21T21:41:19.525 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7489999999999998e-06 2024-09-21T21:41:19.525 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.525 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.525 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:19.526 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.466219+0000", 2024-09-21T21:41:19.526 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.017988839999999999 2024-09-21T21:41:19.526 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.526 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.526 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:19.526 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.466284+0000", 2024-09-21T21:41:19.526 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4890999999999999e-05 2024-09-21T21:41:19.526 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.526 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.526 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:19.526 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.466366+0000", 2024-09-21T21:41:19.526 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.1738999999999998e-05 2024-09-21T21:41:19.526 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:19.527 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:19.527 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:19.527 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.527 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.527 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4840.0:4999 3.1e 3:7fca165a:::rbd_data.11214ff8bd9b.000000000000001e:10 [read 71680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T21:41:19.527 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:16.448236+0000", 2024-09-21T21:41:19.527 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.0095880460000002, 2024-09-21T21:41:19.527 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.018299929, 2024-09-21T21:41:19.527 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:19.527 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:19.527 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:19.527 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:19.527 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4840", 2024-09-21T21:41:19.528 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1293784138", 2024-09-21T21:41:19.528 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4999 2024-09-21T21:41:19.528 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.528 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:19.528 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.528 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:19.528 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.448236+0000", 2024-09-21T21:41:19.528 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:19.528 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.528 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.529 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:19.529 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.448236+0000", 2024-09-21T21:41:19.529 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:19.529 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.529 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.529 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:19.529 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.448236+0000", 2024-09-21T21:41:19.529 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.65e-07 2024-09-21T21:41:19.529 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.529 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.529 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:19.529 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.448243+0000", 2024-09-21T21:41:19.529 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1199999999999999e-06 2024-09-21T21:41:19.529 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.529 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.530 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:19.530 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.448249+0000", 2024-09-21T21:41:19.530 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4080000000000003e-06 2024-09-21T21:41:19.530 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.530 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.530 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:19.530 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.448254+0000", 2024-09-21T21:41:19.530 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7890000000000002e-06 2024-09-21T21:41:19.530 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.530 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.530 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:19.530 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.466393+0000", 2024-09-21T21:41:19.530 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.018139702000000001 2024-09-21T21:41:19.530 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.531 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.531 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:19.531 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.466459+0000", 2024-09-21T21:41:19.531 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5894000000000003e-05 2024-09-21T21:41:19.531 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.531 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.531 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:19.531 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.466536+0000", 2024-09-21T21:41:19.531 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.6650999999999995e-05 2024-09-21T21:41:19.531 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:19.531 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:19.531 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:19.531 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.531 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.532 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4840.0:5032 3.1e 3:7e975c4e:::rbd_data.11214ff8bd9b.0000000000000024:10 [read 4096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T21:41:19.532 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:16.448261+0000", 2024-09-21T21:41:19.532 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.0095633479999999, 2024-09-21T21:41:19.532 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.018455336999999999, 2024-09-21T21:41:19.532 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:19.532 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:19.532 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:19.532 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:19.532 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4840", 2024-09-21T21:41:19.532 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1293784138", 2024-09-21T21:41:19.532 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 5032 2024-09-21T21:41:19.533 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.533 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:19.533 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.533 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:19.533 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.448261+0000", 2024-09-21T21:41:19.533 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:19.533 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.533 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.533 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:19.533 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.448261+0000", 2024-09-21T21:41:19.533 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:19.533 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.533 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.533 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:19.533 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.448261+0000", 2024-09-21T21:41:19.534 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3099999999999998e-07 2024-09-21T21:41:19.534 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.534 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.534 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:19.534 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.448263+0000", 2024-09-21T21:41:19.534 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3020000000000002e-06 2024-09-21T21:41:19.534 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.534 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.534 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:19.534 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.448270+0000", 2024-09-21T21:41:19.534 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5069999999999997e-06 2024-09-21T21:41:19.534 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.534 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.535 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:19.535 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.448275+0000", 2024-09-21T21:41:19.535 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0259999999999998e-06 2024-09-21T21:41:19.535 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.535 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.535 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:19.535 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.466563+0000", 2024-09-21T21:41:19.535 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.018288087000000001 2024-09-21T21:41:19.535 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.535 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.535 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:19.535 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.466625+0000", 2024-09-21T21:41:19.535 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1632999999999998e-05 2024-09-21T21:41:19.535 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.536 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.536 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:19.536 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.466716+0000", 2024-09-21T21:41:19.536 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.1351e-05 2024-09-21T21:41:19.536 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:19.536 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:19.536 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:19.536 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.536 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.536 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4840.0:5033 3.1e 3:7e975c4e:::rbd_data.11214ff8bd9b.0000000000000024:10 [read 16384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T21:41:19.536 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:16.448281+0000", 2024-09-21T21:41:19.536 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.009543028, 2024-09-21T21:41:19.536 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.018610323000000002, 2024-09-21T21:41:19.536 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:19.537 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:19.537 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:19.537 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:19.537 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4840", 2024-09-21T21:41:19.537 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1293784138", 2024-09-21T21:41:19.537 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 5033 2024-09-21T21:41:19.537 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.537 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:19.537 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.537 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:19.537 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.448281+0000", 2024-09-21T21:41:19.537 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:19.537 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.538 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.538 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:19.538 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.448281+0000", 2024-09-21T21:41:19.538 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:19.538 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.538 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.538 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:19.539 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.448281+0000", 2024-09-21T21:41:19.539 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7599999999999998e-07 2024-09-21T21:41:19.539 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.539 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.539 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:19.539 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.448283+0000", 2024-09-21T21:41:19.539 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0540000000000002e-06 2024-09-21T21:41:19.539 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.539 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.539 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:19.540 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.448290+0000", 2024-09-21T21:41:19.540 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3459999999999999e-06 2024-09-21T21:41:19.540 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.540 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.540 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:19.540 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.448295+0000", 2024-09-21T21:41:19.540 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1240000000000004e-06 2024-09-21T21:41:19.540 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.541 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.541 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:19.541 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.466743+0000", 2024-09-21T21:41:19.541 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.018448506 2024-09-21T21:41:19.541 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.541 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.541 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:19.542 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.466807+0000", 2024-09-21T21:41:19.542 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3391000000000003e-05 2024-09-21T21:41:19.542 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.542 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.542 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:19.542 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.466891+0000", 2024-09-21T21:41:19.542 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.4525999999999997e-05 2024-09-21T21:41:19.542 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:19.542 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:19.542 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:19.542 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.542 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.542 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4840.0:5034 3.1e 3:7e975c4e:::rbd_data.11214ff8bd9b.0000000000000024:10 [read 28672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T21:41:19.543 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:16.448301+0000", 2024-09-21T21:41:19.543 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.0095232040000002, 2024-09-21T21:41:19.543 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.018772560000000001, 2024-09-21T21:41:19.543 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:19.543 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:19.543 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:19.543 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:19.543 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4840", 2024-09-21T21:41:19.543 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1293784138", 2024-09-21T21:41:19.543 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 5034 2024-09-21T21:41:19.543 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.544 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:19.544 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.544 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:19.544 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.448301+0000", 2024-09-21T21:41:19.544 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:19.544 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.544 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.544 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:19.544 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.448301+0000", 2024-09-21T21:41:19.544 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:19.544 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.544 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.544 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:19.544 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.448301+0000", 2024-09-21T21:41:19.544 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1199999999999998e-07 2024-09-21T21:41:19.545 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.545 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.545 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:19.545 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.448303+0000", 2024-09-21T21:41:19.545 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1280000000000002e-06 2024-09-21T21:41:19.545 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.545 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.545 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:19.545 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.448342+0000", 2024-09-21T21:41:19.545 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8665999999999999e-05 2024-09-21T21:41:19.545 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.545 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.545 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:19.545 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.448348+0000", 2024-09-21T21:41:19.546 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9089999999999997e-06 2024-09-21T21:41:19.546 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.546 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.546 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:19.546 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.466919+0000", 2024-09-21T21:41:19.546 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.018570927000000001 2024-09-21T21:41:19.546 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.546 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.546 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:19.546 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.466990+0000", 2024-09-21T21:41:19.546 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.1180000000000006e-05 2024-09-21T21:41:19.546 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.546 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.546 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:19.547 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.467073+0000", 2024-09-21T21:41:19.547 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.3337999999999994e-05 2024-09-21T21:41:19.547 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:19.547 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:19.547 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:19.547 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.547 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.547 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4840.0:5035 3.1e 3:7e975c4e:::rbd_data.11214ff8bd9b.0000000000000024:10 [read 40960~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T21:41:19.547 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:16.448356+0000", 2024-09-21T21:41:19.547 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.009467603, 2024-09-21T21:41:19.547 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.018891946, 2024-09-21T21:41:19.547 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:19.547 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:19.547 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:19.548 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:19.548 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4840", 2024-09-21T21:41:19.548 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1293784138", 2024-09-21T21:41:19.548 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 5035 2024-09-21T21:41:19.548 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.548 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:19.548 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.548 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:19.548 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.448356+0000", 2024-09-21T21:41:19.548 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:19.548 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.548 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.548 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:19.549 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.448356+0000", 2024-09-21T21:41:19.549 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:19.549 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.549 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.549 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:19.549 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.448357+0000", 2024-09-21T21:41:19.549 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3300000000000002e-07 2024-09-21T21:41:19.549 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.549 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.549 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:19.549 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.448360+0000", 2024-09-21T21:41:19.549 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.785e-06 2024-09-21T21:41:19.550 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.550 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.550 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:19.550 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.448367+0000", 2024-09-21T21:41:19.550 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.8889999999999996e-06 2024-09-21T21:41:19.550 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.550 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.550 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:19.550 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.448371+0000", 2024-09-21T21:41:19.550 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4800000000000003e-06 2024-09-21T21:41:19.550 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.550 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.550 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:19.550 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.467104+0000", 2024-09-21T21:41:19.551 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.018733191 2024-09-21T21:41:19.551 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.551 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.551 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:19.551 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.467173+0000", 2024-09-21T21:41:19.551 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.8208999999999997e-05 2024-09-21T21:41:19.551 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.551 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.551 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:19.551 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.467248+0000", 2024-09-21T21:41:19.551 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5859000000000007e-05 2024-09-21T21:41:19.551 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:19.551 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:19.551 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:19.551 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.552 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.552 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4840.0:5036 3.1e 3:7e975c4e:::rbd_data.11214ff8bd9b.0000000000000024:10 [read 53248~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T21:41:19.552 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:16.448377+0000", 2024-09-21T21:41:19.552 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.0094467890000001, 2024-09-21T21:41:19.552 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.019006918000000001, 2024-09-21T21:41:19.552 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:19.552 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:19.552 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:19.552 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:19.552 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4840", 2024-09-21T21:41:19.552 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1293784138", 2024-09-21T21:41:19.552 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 5036 2024-09-21T21:41:19.552 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.552 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:19.553 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.553 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:19.553 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.448377+0000", 2024-09-21T21:41:19.553 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:19.553 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.553 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.553 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:19.553 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.448377+0000", 2024-09-21T21:41:19.553 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:19.553 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.553 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.553 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:19.553 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.448378+0000", 2024-09-21T21:41:19.553 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6399999999999998e-07 2024-09-21T21:41:19.553 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.554 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.554 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:19.554 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.448380+0000", 2024-09-21T21:41:19.554 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2869999999999999e-06 2024-09-21T21:41:19.554 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.554 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.554 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:19.554 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.448386+0000", 2024-09-21T21:41:19.554 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2450000000000003e-06 2024-09-21T21:41:19.554 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.554 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.554 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:19.554 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.448391+0000", 2024-09-21T21:41:19.554 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.057e-06 2024-09-21T21:41:19.555 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.555 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.555 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:19.555 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.467272+0000", 2024-09-21T21:41:19.555 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.018880788999999999 2024-09-21T21:41:19.555 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.555 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.555 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:19.555 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.467323+0000", 2024-09-21T21:41:19.555 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0769999999999997e-05 2024-09-21T21:41:19.555 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.555 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.555 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:19.555 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.467384+0000", 2024-09-21T21:41:19.555 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1406000000000001e-05 2024-09-21T21:41:19.556 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:19.556 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:19.556 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:19.556 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.556 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.556 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4840.0:5037 3.1e 3:7e975c4e:::rbd_data.11214ff8bd9b.0000000000000024:10 [read 65536~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T21:41:19.556 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:16.448398+0000", 2024-09-21T21:41:19.556 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.0094260949999998, 2024-09-21T21:41:19.556 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.01914619, 2024-09-21T21:41:19.556 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:19.556 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:19.556 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:19.556 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:19.556 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4840", 2024-09-21T21:41:19.557 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1293784138", 2024-09-21T21:41:19.557 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 5037 2024-09-21T21:41:19.557 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.557 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:19.557 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.557 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:19.557 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.448398+0000", 2024-09-21T21:41:19.557 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:19.557 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.557 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.557 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:19.557 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.448398+0000", 2024-09-21T21:41:19.557 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:19.557 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.558 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.558 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:19.558 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.448398+0000", 2024-09-21T21:41:19.558 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.01e-07 2024-09-21T21:41:19.558 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.558 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.558 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:19.558 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.448401+0000", 2024-09-21T21:41:19.558 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2079999999999999e-06 2024-09-21T21:41:19.558 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.558 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.559 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:19.559 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.448407+0000", 2024-09-21T21:41:19.559 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.297e-06 2024-09-21T21:41:19.559 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.559 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.559 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:19.559 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.448412+0000", 2024-09-21T21:41:19.559 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.865e-06 2024-09-21T21:41:19.559 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.559 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.560 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:19.560 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.467407+0000", 2024-09-21T21:41:19.560 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.018995128999999999 2024-09-21T21:41:19.560 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.560 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.560 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:19.560 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.467463+0000", 2024-09-21T21:41:19.560 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5958999999999997e-05 2024-09-21T21:41:19.560 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.560 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.560 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:19.560 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.467544+0000", 2024-09-21T21:41:19.560 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.1330999999999998e-05 2024-09-21T21:41:19.560 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:19.560 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:19.561 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:19.561 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.561 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.561 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4840.0:5038 3.1e 3:7e975c4e:::rbd_data.11214ff8bd9b.0000000000000024:10 [read 77824~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T21:41:19.561 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:16.448433+0000", 2024-09-21T21:41:19.561 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.0093906499999998, 2024-09-21T21:41:19.561 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.019281691, 2024-09-21T21:41:19.561 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:19.561 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:19.561 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:19.561 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:19.561 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4840", 2024-09-21T21:41:19.561 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1293784138", 2024-09-21T21:41:19.561 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 5038 2024-09-21T21:41:19.562 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.562 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:19.562 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.562 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:19.562 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.448433+0000", 2024-09-21T21:41:19.562 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:19.562 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.563 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.563 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:19.563 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.448433+0000", 2024-09-21T21:41:19.563 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:19.563 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.563 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.563 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:19.563 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.448434+0000", 2024-09-21T21:41:19.563 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8999999999999997e-07 2024-09-21T21:41:19.563 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.563 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.563 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:19.564 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.448436+0000", 2024-09-21T21:41:19.564 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.5100000000000001e-06 2024-09-21T21:41:19.564 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.564 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.564 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:19.564 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.448444+0000", 2024-09-21T21:41:19.564 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.4429999999999997e-06 2024-09-21T21:41:19.564 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.564 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.564 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:19.564 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.448457+0000", 2024-09-21T21:41:19.564 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.2830000000000001e-05 2024-09-21T21:41:19.564 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.565 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.565 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:19.565 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.467570+0000", 2024-09-21T21:41:19.565 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.019113356000000001 2024-09-21T21:41:19.565 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.565 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.565 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:19.565 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.467636+0000", 2024-09-21T21:41:19.565 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5598000000000005e-05 2024-09-21T21:41:19.565 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:19.565 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:19.565 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:19.565 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.467715+0000", 2024-09-21T21:41:19.565 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.9463999999999997e-05 2024-09-21T21:41:19.565 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:19.566 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:19.566 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:19.566 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:19.566 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:19.566 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:19.567 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T21:41:19.568 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:19.565+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:19.668 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:19.667+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:19.677 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:19.678 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:41:19.678 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:19.678 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4385.0:10055 3.a 3:535385f0:::rbd_children:head [call rbd.add_child in=59b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-21T21:41:19.678 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:19.676628+0000", 2024-09-21T21:41:19.678 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.000143845, 2024-09-21T21:41:19.678 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000159748, 2024-09-21T21:41:19.678 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:19.678 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:19.678 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:19.678 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:19.679 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4385", 2024-09-21T21:41:19.679 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:41:19.679 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 10055 2024-09-21T21:41:19.679 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:19.679 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:19.679 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:19.679 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:19.679 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:19.676628+0000", 2024-09-21T21:41:19.679 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:19.679 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:19.679 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:19.679 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:19.679 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:19.676628+0000", 2024-09-21T21:41:19.679 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:19.680 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:19.680 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:19.680 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:19.680 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:19.676629+0000", 2024-09-21T21:41:19.680 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8800000000000001e-07 2024-09-21T21:41:19.680 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:19.680 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:19.680 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:19.680 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:19.676635+0000", 2024-09-21T21:41:19.680 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3350000000000003e-06 2024-09-21T21:41:19.680 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:19.680 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:19.680 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:19.680 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:19.676648+0000", 2024-09-21T21:41:19.680 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3268000000000001e-05 2024-09-21T21:41:19.681 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:19.681 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:19.681 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:19.681 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:19.676657+0000", 2024-09-21T21:41:19.681 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5050000000000007e-06 2024-09-21T21:41:19.681 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:19.682 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:19.682 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:19.682 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:19.676703+0000", 2024-09-21T21:41:19.682 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6623999999999999e-05 2024-09-21T21:41:19.682 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:19.683 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:19.683 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:19.683 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:19.676752+0000", 2024-09-21T21:41:19.683 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8418999999999998e-05 2024-09-21T21:41:19.683 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:19.683 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:19.683 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:19.683 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:19.683 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-21T21:41:19.683 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 1 2024-09-21T21:41:19.683 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:19.686 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T21:41:19.687 INFO:teuthology.orchestra.run.smithi164.stdout:truncating image image_client.0-clone15 from 0x8fec600 (overlap 0x95d000) to 0x0 2024-09-21T21:41:19.770 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:19.769+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:19.831 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:19.831 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:41:19.831 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:41:19.831 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:41:19.831 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:19.839 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T21:41:19.870 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:19.868+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:19.971 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:19.969+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:19.980 INFO:teuthology.orchestra.run.smithi164.stdout:checking clone #14, image image_client.0-clone14 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent15 2024-09-21T21:41:19.987 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:19.988 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-21T21:41:19.988 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-21T21:41:19.988 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:41:19.988 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:19.988 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2415 3.d 3:b26310db:::rbd_data.112145fae779.0000000000000006:a [read 10690560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:19.988 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.406907+0000", 2024-09-21T21:41:19.988 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 21.578570744, 2024-09-21T21:41:19.988 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044424119999999997, 2024-09-21T21:41:19.988 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:19.988 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:19.988 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:19.988 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:19.988 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:19.988 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:19.989 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2415 2024-09-21T21:41:19.989 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:19.989 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:19.989 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:19.989 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:19.989 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.406907+0000", 2024-09-21T21:41:19.989 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:19.989 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:19.989 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:19.989 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:19.989 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.406907+0000", 2024-09-21T21:41:19.989 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:19.989 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:19.989 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:19.990 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:19.990 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.406908+0000", 2024-09-21T21:41:19.990 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7599999999999998e-07 2024-09-21T21:41:19.990 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:19.990 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:19.990 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:19.990 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.406915+0000", 2024-09-21T21:41:19.990 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6729999999999992e-06 2024-09-21T21:41:19.990 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:19.990 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:19.990 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:19.990 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.406922+0000", 2024-09-21T21:41:19.990 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7139999999999996e-06 2024-09-21T21:41:19.990 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:19.991 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:19.991 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:19.991 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.406927+0000", 2024-09-21T21:41:19.991 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.327e-06 2024-09-21T21:41:19.991 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:19.991 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:19.991 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:19.991 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.451179+0000", 2024-09-21T21:41:19.991 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044251984000000001 2024-09-21T21:41:19.991 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:19.991 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:19.991 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:19.991 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.451254+0000", 2024-09-21T21:41:19.992 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4703999999999995e-05 2024-09-21T21:41:19.992 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:19.992 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:19.992 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:19.992 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.451331+0000", 2024-09-21T21:41:19.992 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7342000000000006e-05 2024-09-21T21:41:19.992 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:19.992 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:19.992 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:19.992 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:19.992 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:19.992 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2417 3.d 3:b26310db:::rbd_data.112145fae779.0000000000000006:a [read 10252288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:19.992 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.406934+0000", 2024-09-21T21:41:19.992 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 21.578544000000001, 2024-09-21T21:41:19.992 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044511163999999999, 2024-09-21T21:41:19.993 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:19.993 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:19.993 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:19.993 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:19.993 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:19.993 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:19.993 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2417 2024-09-21T21:41:19.993 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:19.993 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:19.993 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:19.993 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:19.993 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.406934+0000", 2024-09-21T21:41:19.993 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:19.993 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:19.993 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:19.994 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:19.994 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.406934+0000", 2024-09-21T21:41:19.994 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:19.994 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:19.994 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:19.994 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:19.994 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.406935+0000", 2024-09-21T21:41:19.994 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6499999999999999e-07 2024-09-21T21:41:19.994 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:19.994 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:19.994 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:19.994 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.406937+0000", 2024-09-21T21:41:19.994 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1299999999999999e-06 2024-09-21T21:41:19.994 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:19.995 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:19.995 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:19.995 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.406943+0000", 2024-09-21T21:41:19.995 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9989999999999997e-06 2024-09-21T21:41:19.995 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:19.995 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:19.995 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:19.996 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.406948+0000", 2024-09-21T21:41:19.996 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9729999999999997e-06 2024-09-21T21:41:19.996 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:19.996 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:19.996 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:19.996 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.451365+0000", 2024-09-21T21:41:19.996 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044417808000000003 2024-09-21T21:41:19.996 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:19.996 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:19.996 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:19.996 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.451403+0000", 2024-09-21T21:41:19.996 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7944999999999997e-05 2024-09-21T21:41:19.996 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:19.997 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:19.997 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:19.997 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.451445+0000", 2024-09-21T21:41:19.997 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1844000000000001e-05 2024-09-21T21:41:19.997 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:19.997 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:19.997 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:19.997 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:19.997 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:19.997 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2425 3.d 3:b26310db:::rbd_data.112145fae779.0000000000000006:a [read 10457088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:19.997 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.407639+0000", 2024-09-21T21:41:19.997 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 21.577839299000001, 2024-09-21T21:41:19.997 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044288292999999999, 2024-09-21T21:41:19.997 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:19.998 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:19.998 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:19.998 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:19.998 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:19.998 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:19.998 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2425 2024-09-21T21:41:19.998 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:19.998 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:19.998 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:19.998 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:19.998 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407639+0000", 2024-09-21T21:41:19.998 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:19.999 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:19.999 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:19.999 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:19.999 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407639+0000", 2024-09-21T21:41:19.999 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:19.999 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:19.999 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:19.999 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:19.999 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407639+0000", 2024-09-21T21:41:19.999 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8299999999999998e-07 2024-09-21T21:41:19.999 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:19.999 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.000 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:20.000 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407641+0000", 2024-09-21T21:41:20.000 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9149999999999999e-06 2024-09-21T21:41:20.000 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.000 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.000 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:20.000 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407647+0000", 2024-09-21T21:41:20.001 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3110000000000002e-06 2024-09-21T21:41:20.001 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.001 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.001 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:20.001 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407653+0000", 2024-09-21T21:41:20.001 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1309999999999997e-06 2024-09-21T21:41:20.001 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.002 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.002 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:20.002 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.451778+0000", 2024-09-21T21:41:20.002 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044125782000000002 2024-09-21T21:41:20.002 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.002 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.002 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:20.003 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.451850+0000", 2024-09-21T21:41:20.003 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1211999999999995e-05 2024-09-21T21:41:20.003 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.003 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.003 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:20.003 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.451927+0000", 2024-09-21T21:41:20.003 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7558999999999994e-05 2024-09-21T21:41:20.003 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:20.003 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:20.004 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:20.004 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.004 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.004 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2427 3.d 3:b26310db:::rbd_data.112145fae779.0000000000000006:a [read 10489856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:20.004 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.407659+0000", 2024-09-21T21:41:20.004 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 21.577818986, 2024-09-21T21:41:20.004 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044463186000000002, 2024-09-21T21:41:20.005 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:20.005 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:20.005 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:20.005 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:20.006 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:20.006 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:20.006 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2427 2024-09-21T21:41:20.006 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.006 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:20.006 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.006 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:20.006 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407659+0000", 2024-09-21T21:41:20.006 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:20.006 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.006 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.006 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:20.006 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407659+0000", 2024-09-21T21:41:20.007 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:20.007 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.007 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.007 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:20.007 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407659+0000", 2024-09-21T21:41:20.007 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3000000000000002e-07 2024-09-21T21:41:20.007 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.007 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.007 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:20.007 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407662+0000", 2024-09-21T21:41:20.007 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0980000000000001e-06 2024-09-21T21:41:20.007 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.007 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.007 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:20.008 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407668+0000", 2024-09-21T21:41:20.008 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5209999999999999e-06 2024-09-21T21:41:20.008 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.008 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.008 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:20.008 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407674+0000", 2024-09-21T21:41:20.008 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7590000000000003e-06 2024-09-21T21:41:20.008 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.008 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.008 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:20.008 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.451969+0000", 2024-09-21T21:41:20.008 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044294965999999998 2024-09-21T21:41:20.008 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.008 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.009 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:20.009 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.452045+0000", 2024-09-21T21:41:20.009 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6271000000000002e-05 2024-09-21T21:41:20.009 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.009 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.009 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:20.009 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.452122+0000", 2024-09-21T21:41:20.009 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7241000000000001e-05 2024-09-21T21:41:20.009 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:20.009 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:20.009 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:20.009 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.009 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.009 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2429 3.d 3:b26310db:::rbd_data.112145fae779.0000000000000006:a [read 10547200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:20.010 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.407681+0000", 2024-09-21T21:41:20.010 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 21.577797312000001, 2024-09-21T21:41:20.010 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.04455829, 2024-09-21T21:41:20.010 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:20.010 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:20.010 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:20.010 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:20.010 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:20.010 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:20.010 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2429 2024-09-21T21:41:20.010 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.010 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:20.010 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.010 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:20.011 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407681+0000", 2024-09-21T21:41:20.011 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:20.011 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.011 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.011 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:20.011 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407681+0000", 2024-09-21T21:41:20.011 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:20.011 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.011 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.011 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:20.011 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407681+0000", 2024-09-21T21:41:20.011 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2300000000000001e-07 2024-09-21T21:41:20.011 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.011 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.012 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:20.012 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407683+0000", 2024-09-21T21:41:20.012 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0020000000000001e-06 2024-09-21T21:41:20.012 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.012 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.012 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:20.012 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407690+0000", 2024-09-21T21:41:20.012 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4269999999999996e-06 2024-09-21T21:41:20.012 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.012 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.012 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:20.012 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407694+0000", 2024-09-21T21:41:20.012 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6029999999999998e-06 2024-09-21T21:41:20.012 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.012 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.013 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:20.013 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.452153+0000", 2024-09-21T21:41:20.013 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044458232 2024-09-21T21:41:20.013 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.013 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.013 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:20.013 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.452196+0000", 2024-09-21T21:41:20.013 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3248999999999999e-05 2024-09-21T21:41:20.013 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.013 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.013 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:20.013 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.452239+0000", 2024-09-21T21:41:20.013 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3254000000000003e-05 2024-09-21T21:41:20.013 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:20.013 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:20.014 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:20.014 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.014 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.014 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2431 3.d 3:b26310db:::rbd_data.112145fae779.0000000000000006:a [read 10604544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:20.014 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.407701+0000", 2024-09-21T21:41:20.014 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 21.577777528999999, 2024-09-21T21:41:20.014 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044639389000000002, 2024-09-21T21:41:20.014 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:20.014 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:20.014 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:20.014 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:20.014 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:20.014 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:20.014 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2431 2024-09-21T21:41:20.015 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.015 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:20.015 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.015 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:20.015 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407701+0000", 2024-09-21T21:41:20.015 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:20.015 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.015 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.015 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:20.015 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407701+0000", 2024-09-21T21:41:20.015 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:20.015 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.015 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.015 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:20.015 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407701+0000", 2024-09-21T21:41:20.016 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.63e-07 2024-09-21T21:41:20.016 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.016 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.016 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:20.016 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407703+0000", 2024-09-21T21:41:20.016 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.979e-06 2024-09-21T21:41:20.016 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.016 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.016 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:20.016 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407710+0000", 2024-09-21T21:41:20.016 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5470000000000002e-06 2024-09-21T21:41:20.016 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.016 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.016 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:20.016 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407715+0000", 2024-09-21T21:41:20.017 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.972e-06 2024-09-21T21:41:20.017 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.017 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.017 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:20.017 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.452266+0000", 2024-09-21T21:41:20.017 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044551201999999998 2024-09-21T21:41:20.017 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.017 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.017 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:20.017 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.452302+0000", 2024-09-21T21:41:20.017 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6516000000000001e-05 2024-09-21T21:41:20.017 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.017 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.017 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:20.018 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.452340+0000", 2024-09-21T21:41:20.018 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7710000000000003e-05 2024-09-21T21:41:20.018 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:20.018 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:20.018 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:20.018 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.018 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.018 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2434 3.d 3:b26310db:::rbd_data.112145fae779.0000000000000006:a [read 10637312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:20.018 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.407720+0000", 2024-09-21T21:41:20.018 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 21.577757688999998, 2024-09-21T21:41:20.019 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.04471874, 2024-09-21T21:41:20.019 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:20.019 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:20.019 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:20.019 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:20.019 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:20.019 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:20.019 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2434 2024-09-21T21:41:20.019 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.019 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:20.019 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.019 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:20.019 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407720+0000", 2024-09-21T21:41:20.019 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:20.020 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.020 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.020 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:20.020 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407720+0000", 2024-09-21T21:41:20.020 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:20.020 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.020 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.020 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:20.020 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407721+0000", 2024-09-21T21:41:20.020 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5199999999999998e-07 2024-09-21T21:41:20.020 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.020 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.020 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:20.020 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407723+0000", 2024-09-21T21:41:20.020 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9139999999999998e-06 2024-09-21T21:41:20.021 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.021 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.021 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:20.021 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407729+0000", 2024-09-21T21:41:20.021 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2009999999999996e-06 2024-09-21T21:41:20.021 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.021 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.021 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:20.021 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407734+0000", 2024-09-21T21:41:20.021 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.006e-06 2024-09-21T21:41:20.021 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.021 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.021 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:20.021 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.452364+0000", 2024-09-21T21:41:20.022 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044630625 2024-09-21T21:41:20.022 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.022 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.022 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:20.022 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.452402+0000", 2024-09-21T21:41:20.022 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7518999999999998e-05 2024-09-21T21:41:20.022 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.022 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.022 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:20.022 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.452439+0000", 2024-09-21T21:41:20.022 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7122999999999997e-05 2024-09-21T21:41:20.022 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:20.022 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:20.022 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:20.023 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.023 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.023 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2436 3.d 3:b26310db:::rbd_data.112145fae779.0000000000000006:a [read 10694656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:20.023 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.407740+0000", 2024-09-21T21:41:20.023 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 21.577738346, 2024-09-21T21:41:20.023 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044797624000000001, 2024-09-21T21:41:20.023 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:20.023 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:20.023 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:20.023 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:20.023 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:20.023 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:20.023 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2436 2024-09-21T21:41:20.023 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.024 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:20.024 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.024 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:20.024 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407740+0000", 2024-09-21T21:41:20.024 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:20.024 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.024 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.024 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:20.024 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407740+0000", 2024-09-21T21:41:20.024 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:20.024 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.024 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.024 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:20.024 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407740+0000", 2024-09-21T21:41:20.024 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9299999999999999e-07 2024-09-21T21:41:20.025 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.025 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.025 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:20.025 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407742+0000", 2024-09-21T21:41:20.025 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.826e-06 2024-09-21T21:41:20.025 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.025 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.025 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:20.025 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407748+0000", 2024-09-21T21:41:20.025 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2149999999999998e-06 2024-09-21T21:41:20.025 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.025 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.025 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:20.025 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407753+0000", 2024-09-21T21:41:20.026 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.848e-06 2024-09-21T21:41:20.026 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.026 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.026 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:20.026 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.452463+0000", 2024-09-21T21:41:20.026 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044709596999999997 2024-09-21T21:41:20.026 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.026 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.026 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:20.026 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.452500+0000", 2024-09-21T21:41:20.026 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7058999999999999e-05 2024-09-21T21:41:20.026 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.026 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.026 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:20.027 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.452537+0000", 2024-09-21T21:41:20.027 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7685999999999998e-05 2024-09-21T21:41:20.027 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:20.027 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:20.027 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:20.027 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.027 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.027 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2602 3.d 3:b154679f:::rbd_data.112145fae779.0000000000000004:a [read 3923968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:20.027 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.411479+0000", 2024-09-21T21:41:20.027 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 21.573998887999998, 2024-09-21T21:41:20.027 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044281854000000002, 2024-09-21T21:41:20.027 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:20.027 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:20.027 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:20.028 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:20.028 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:20.028 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:20.028 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2602 2024-09-21T21:41:20.028 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.028 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:20.028 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.028 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:20.028 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411479+0000", 2024-09-21T21:41:20.028 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:20.028 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.028 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.028 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:20.028 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411479+0000", 2024-09-21T21:41:20.029 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:20.029 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.029 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.029 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:20.029 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411480+0000", 2024-09-21T21:41:20.029 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.37e-07 2024-09-21T21:41:20.029 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.029 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.029 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:20.029 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411482+0000", 2024-09-21T21:41:20.029 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9829999999999999e-06 2024-09-21T21:41:20.029 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.029 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.029 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:20.030 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411488+0000", 2024-09-21T21:41:20.030 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.947e-06 2024-09-21T21:41:20.030 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.030 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.030 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:20.030 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411492+0000", 2024-09-21T21:41:20.030 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.357e-06 2024-09-21T21:41:20.030 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.030 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.030 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:20.030 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.455581+0000", 2024-09-21T21:41:20.030 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044089077999999997 2024-09-21T21:41:20.030 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.030 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.031 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:20.031 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.455675+0000", 2024-09-21T21:41:20.031 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3757000000000004e-05 2024-09-21T21:41:20.031 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.031 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.031 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:20.031 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.455761+0000", 2024-09-21T21:41:20.031 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6194999999999997e-05 2024-09-21T21:41:20.031 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:20.031 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:20.031 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:20.031 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.031 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.031 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2612 3.d 3:b26310db:::rbd_data.112145fae779.0000000000000006:a [read 3448832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:20.031 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.411498+0000", 2024-09-21T21:41:20.032 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 21.573980120000002, 2024-09-21T21:41:20.032 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044369585000000003, 2024-09-21T21:41:20.032 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:20.032 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:20.032 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:20.032 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:20.032 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:20.032 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:20.032 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2612 2024-09-21T21:41:20.032 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.032 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:20.032 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.032 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:20.032 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411498+0000", 2024-09-21T21:41:20.032 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:20.033 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.033 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.033 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:20.033 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411498+0000", 2024-09-21T21:41:20.033 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:20.033 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.033 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.033 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:20.033 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411498+0000", 2024-09-21T21:41:20.033 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6899999999999998e-07 2024-09-21T21:41:20.033 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.033 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.033 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:20.033 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411500+0000", 2024-09-21T21:41:20.034 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.841e-06 2024-09-21T21:41:20.034 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.034 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.034 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:20.034 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411506+0000", 2024-09-21T21:41:20.034 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7350000000000001e-06 2024-09-21T21:41:20.034 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.034 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.034 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:20.034 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411510+0000", 2024-09-21T21:41:20.034 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4830000000000001e-06 2024-09-21T21:41:20.034 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.034 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.034 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:20.034 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.455789+0000", 2024-09-21T21:41:20.035 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044278359000000003 2024-09-21T21:41:20.035 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.035 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.035 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:20.035 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.455824+0000", 2024-09-21T21:41:20.035 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5719000000000001e-05 2024-09-21T21:41:20.035 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.035 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.035 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:20.035 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.455868+0000", 2024-09-21T21:41:20.035 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3078999999999999e-05 2024-09-21T21:41:20.035 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:20.035 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:20.036 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:20.036 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.036 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.036 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2613 3.d 3:b26310db:::rbd_data.112145fae779.0000000000000006:a [read 3506176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:20.036 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.411516+0000", 2024-09-21T21:41:20.036 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 21.573962442999999, 2024-09-21T21:41:20.036 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044562615, 2024-09-21T21:41:20.036 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:20.036 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:20.036 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:20.036 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:20.037 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:20.037 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:20.037 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2613 2024-09-21T21:41:20.037 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.037 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:20.037 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.037 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:20.037 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411516+0000", 2024-09-21T21:41:20.037 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:20.037 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.037 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.037 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:20.037 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411516+0000", 2024-09-21T21:41:20.038 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:20.038 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.038 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.038 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:20.038 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411516+0000", 2024-09-21T21:41:20.038 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4400000000000001e-07 2024-09-21T21:41:20.038 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.038 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.038 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:20.038 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411522+0000", 2024-09-21T21:41:20.038 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2600000000000002e-06 2024-09-21T21:41:20.038 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.038 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.038 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:20.039 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411528+0000", 2024-09-21T21:41:20.039 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5759999999999996e-06 2024-09-21T21:41:20.039 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.039 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.039 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:20.039 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411534+0000", 2024-09-21T21:41:20.039 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3010000000000003e-06 2024-09-21T21:41:20.039 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.039 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.039 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:20.039 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.455911+0000", 2024-09-21T21:41:20.039 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044376546000000003 2024-09-21T21:41:20.040 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.040 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.040 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:20.040 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.455996+0000", 2024-09-21T21:41:20.040 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.4980000000000003e-05 2024-09-21T21:41:20.040 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.040 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.040 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:20.040 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.456078+0000", 2024-09-21T21:41:20.040 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2608000000000006e-05 2024-09-21T21:41:20.040 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:20.040 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:20.040 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:20.040 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.041 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.041 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2614 3.d 3:b26310db:::rbd_data.112145fae779.0000000000000006:a [read 3538944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:20.041 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.411540+0000", 2024-09-21T21:41:20.041 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 21.573937629, 2024-09-21T21:41:20.041 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044644049999999998, 2024-09-21T21:41:20.041 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:20.041 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:20.041 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:20.041 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:20.041 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:20.041 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:20.041 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2614 2024-09-21T21:41:20.041 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.042 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:20.042 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.042 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:20.042 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411540+0000", 2024-09-21T21:41:20.042 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:20.042 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.042 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.042 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:20.042 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411540+0000", 2024-09-21T21:41:20.042 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:20.042 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.042 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.042 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:20.042 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411541+0000", 2024-09-21T21:41:20.043 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3799999999999997e-07 2024-09-21T21:41:20.043 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.043 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.043 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:20.043 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411543+0000", 2024-09-21T21:41:20.043 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1100000000000001e-06 2024-09-21T21:41:20.043 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.043 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.043 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:20.043 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411549+0000", 2024-09-21T21:41:20.043 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8300000000000001e-06 2024-09-21T21:41:20.043 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.043 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.043 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:20.044 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411554+0000", 2024-09-21T21:41:20.044 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8169999999999998e-06 2024-09-21T21:41:20.044 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.044 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.044 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:20.044 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.456107+0000", 2024-09-21T21:41:20.044 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044553552000000003 2024-09-21T21:41:20.044 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.044 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.044 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:20.044 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.456144+0000", 2024-09-21T21:41:20.044 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6560000000000002e-05 2024-09-21T21:41:20.044 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.044 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.045 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:20.045 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.456185+0000", 2024-09-21T21:41:20.045 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0642999999999997e-05 2024-09-21T21:41:20.045 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:20.045 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:20.045 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:20.045 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.045 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.045 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2615 3.d 3:b26310db:::rbd_data.112145fae779.0000000000000006:a [read 3596288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:20.045 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.411560+0000", 2024-09-21T21:41:20.045 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 21.573918074000002, 2024-09-21T21:41:20.045 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044828397999999998, 2024-09-21T21:41:20.045 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:20.045 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:20.046 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:20.046 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:20.046 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:20.046 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:20.046 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2615 2024-09-21T21:41:20.046 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.046 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:20.046 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.046 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:20.046 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411560+0000", 2024-09-21T21:41:20.046 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:20.046 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.046 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.047 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:20.047 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411560+0000", 2024-09-21T21:41:20.047 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:20.047 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.047 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.047 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:20.047 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411560+0000", 2024-09-21T21:41:20.047 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7399999999999999e-07 2024-09-21T21:41:20.047 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.047 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.047 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:20.047 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411562+0000", 2024-09-21T21:41:20.047 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.854e-06 2024-09-21T21:41:20.047 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.048 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.048 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:20.048 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411568+0000", 2024-09-21T21:41:20.048 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6760000000000004e-06 2024-09-21T21:41:20.048 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.048 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.048 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:20.048 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411573+0000", 2024-09-21T21:41:20.048 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6269999999999999e-06 2024-09-21T21:41:20.048 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.048 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.048 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:20.048 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.456226+0000", 2024-09-21T21:41:20.048 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044653088000000001 2024-09-21T21:41:20.048 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.049 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.049 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:20.049 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.456311+0000", 2024-09-21T21:41:20.049 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5685000000000004e-05 2024-09-21T21:41:20.049 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.049 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.049 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:20.049 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.456388+0000", 2024-09-21T21:41:20.049 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7094000000000004e-05 2024-09-21T21:41:20.049 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:20.049 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:20.049 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:20.049 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.049 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.050 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2616 3.d 3:b26310db:::rbd_data.112145fae779.0000000000000006:a [read 3653632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:20.050 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.411578+0000", 2024-09-21T21:41:20.050 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 21.573900170000002, 2024-09-21T21:41:20.050 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044914731999999999, 2024-09-21T21:41:20.050 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:20.050 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:20.050 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:20.050 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:20.050 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:20.050 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:20.050 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2616 2024-09-21T21:41:20.050 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.050 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:20.050 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.051 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:20.051 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411578+0000", 2024-09-21T21:41:20.051 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:20.051 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.051 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.051 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:20.051 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411578+0000", 2024-09-21T21:41:20.051 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:20.051 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.051 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.051 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:20.051 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411578+0000", 2024-09-21T21:41:20.051 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.03e-07 2024-09-21T21:41:20.051 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.051 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.052 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:20.052 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411580+0000", 2024-09-21T21:41:20.052 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.8199999999999999e-06 2024-09-21T21:41:20.052 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.052 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.052 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:20.052 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411586+0000", 2024-09-21T21:41:20.052 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6319999999999996e-06 2024-09-21T21:41:20.052 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.052 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.052 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:20.052 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411590+0000", 2024-09-21T21:41:20.052 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5979999999999999e-06 2024-09-21T21:41:20.052 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.052 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.053 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:20.053 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.456418+0000", 2024-09-21T21:41:20.053 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044827504999999997 2024-09-21T21:41:20.053 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.053 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.053 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:20.053 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.456454+0000", 2024-09-21T21:41:20.053 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5883999999999997e-05 2024-09-21T21:41:20.053 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.053 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.053 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:20.053 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.456493+0000", 2024-09-21T21:41:20.053 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8890000000000002e-05 2024-09-21T21:41:20.054 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:20.054 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:20.054 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:20.054 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.054 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.054 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2617 3.d 3:b26310db:::rbd_data.112145fae779.0000000000000006:a [read 3686400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:20.054 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.411596+0000", 2024-09-21T21:41:20.054 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 21.573882579999999, 2024-09-21T21:41:20.054 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.045094515000000002, 2024-09-21T21:41:20.054 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:20.054 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:20.054 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:20.054 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:20.054 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:20.055 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:20.055 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2617 2024-09-21T21:41:20.055 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.055 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:20.055 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.055 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:20.055 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411596+0000", 2024-09-21T21:41:20.055 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:20.055 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.055 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.055 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:20.055 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411596+0000", 2024-09-21T21:41:20.055 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:20.055 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.056 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.056 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:20.056 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411596+0000", 2024-09-21T21:41:20.056 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3500000000000002e-07 2024-09-21T21:41:20.056 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.056 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.056 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:20.056 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411598+0000", 2024-09-21T21:41:20.056 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.871e-06 2024-09-21T21:41:20.056 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.056 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.056 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:20.056 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411603+0000", 2024-09-21T21:41:20.056 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6420000000000004e-06 2024-09-21T21:41:20.057 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.057 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.057 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:20.057 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411608+0000", 2024-09-21T21:41:20.057 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4780000000000002e-06 2024-09-21T21:41:20.057 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.057 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.057 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:20.057 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.456529+0000", 2024-09-21T21:41:20.057 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044921232999999998 2024-09-21T21:41:20.057 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.057 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.057 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:20.057 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.456616+0000", 2024-09-21T21:41:20.057 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6712000000000006e-05 2024-09-21T21:41:20.058 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.058 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.058 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:20.058 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.456690+0000", 2024-09-21T21:41:20.058 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4244000000000003e-05 2024-09-21T21:41:20.058 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:20.058 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:20.058 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:20.058 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.058 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.058 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2618 3.d 3:b26310db:::rbd_data.112145fae779.0000000000000006:a [read 3743744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:20.058 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.411613+0000", 2024-09-21T21:41:20.058 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 21.573865001000001, 2024-09-21T21:41:20.058 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.045180423999999997, 2024-09-21T21:41:20.059 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:20.059 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:20.059 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:20.059 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:20.059 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:20.059 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:20.059 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2618 2024-09-21T21:41:20.059 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.059 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:20.059 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.059 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:20.059 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411613+0000", 2024-09-21T21:41:20.059 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:20.059 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.060 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.060 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:20.060 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411613+0000", 2024-09-21T21:41:20.060 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:20.060 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.060 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.060 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:20.060 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.412868+0000", 2024-09-21T21:41:20.060 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012549 2024-09-21T21:41:20.060 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.060 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.060 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:20.060 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.412873+0000", 2024-09-21T21:41:20.060 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9060000000000001e-06 2024-09-21T21:41:20.060 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.061 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.061 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:20.061 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.412884+0000", 2024-09-21T21:41:20.061 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1164e-05 2024-09-21T21:41:20.061 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.061 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.061 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:20.061 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.412892+0000", 2024-09-21T21:41:20.061 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5719999999999997e-06 2024-09-21T21:41:20.061 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.061 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.061 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:20.061 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.456717+0000", 2024-09-21T21:41:20.061 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043825559 2024-09-21T21:41:20.062 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.062 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.062 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:20.062 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.456753+0000", 2024-09-21T21:41:20.062 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6158000000000002e-05 2024-09-21T21:41:20.062 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.062 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.062 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:20.062 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.456794+0000", 2024-09-21T21:41:20.062 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0164999999999999e-05 2024-09-21T21:41:20.062 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:20.062 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:20.062 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:20.062 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.063 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.063 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2667 3.d 3:b2093d43:::rbd_data.112145fae779.000000000000000b:a [read 3809280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:20.063 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.414362+0000", 2024-09-21T21:41:20.063 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 21.571116322999998, 2024-09-21T21:41:20.063 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044335620999999999, 2024-09-21T21:41:20.063 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:20.063 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:20.063 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:20.063 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:20.063 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:20.063 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:20.063 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2667 2024-09-21T21:41:20.063 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.063 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:20.063 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.064 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:20.064 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.414362+0000", 2024-09-21T21:41:20.064 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:20.064 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.064 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.064 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:20.064 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.414362+0000", 2024-09-21T21:41:20.064 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:20.064 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.064 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.064 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:20.064 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.414867+0000", 2024-09-21T21:41:20.064 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00050531999999999997 2024-09-21T21:41:20.064 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.065 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.065 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:20.065 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.414871+0000", 2024-09-21T21:41:20.065 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2520000000000001e-06 2024-09-21T21:41:20.065 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.065 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.065 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:20.065 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.414879+0000", 2024-09-21T21:41:20.065 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0369999999999995e-06 2024-09-21T21:41:20.065 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.065 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.065 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:20.065 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.414886+0000", 2024-09-21T21:41:20.065 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.19e-06 2024-09-21T21:41:20.065 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.066 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.066 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:20.066 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.458613+0000", 2024-09-21T21:41:20.066 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043727184000000002 2024-09-21T21:41:20.066 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.066 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.066 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:20.066 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.458649+0000", 2024-09-21T21:41:20.066 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6418999999999998e-05 2024-09-21T21:41:20.066 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.066 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.066 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:20.066 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.458697+0000", 2024-09-21T21:41:20.066 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8219e-05 2024-09-21T21:41:20.067 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:20.067 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:20.067 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:20.067 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.067 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.067 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2725 3.d 3:b154679f:::rbd_data.112145fae779.0000000000000004:a [read 3534848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:20.067 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.414975+0000", 2024-09-21T21:41:20.067 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 21.570503397, 2024-09-21T21:41:20.067 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044402427000000001, 2024-09-21T21:41:20.067 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:20.067 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:20.067 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:20.067 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:20.067 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:20.067 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:20.068 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2725 2024-09-21T21:41:20.068 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.068 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:20.068 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.068 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:20.068 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.414975+0000", 2024-09-21T21:41:20.068 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:20.068 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.068 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.068 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:20.068 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.414975+0000", 2024-09-21T21:41:20.068 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:20.068 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.068 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.068 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:20.069 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.414975+0000", 2024-09-21T21:41:20.069 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1899999999999999e-07 2024-09-21T21:41:20.069 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.069 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.069 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:20.069 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.414976+0000", 2024-09-21T21:41:20.069 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1689999999999999e-06 2024-09-21T21:41:20.069 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.069 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.069 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:20.069 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.414979+0000", 2024-09-21T21:41:20.069 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9900000000000002e-06 2024-09-21T21:41:20.069 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.069 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.070 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:20.070 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.414982+0000", 2024-09-21T21:41:20.070 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.621e-06 2024-09-21T21:41:20.070 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.070 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.070 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:20.070 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.459221+0000", 2024-09-21T21:41:20.070 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044239602000000003 2024-09-21T21:41:20.070 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.070 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.070 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:20.070 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.459297+0000", 2024-09-21T21:41:20.070 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5768999999999999e-05 2024-09-21T21:41:20.071 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.071 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.071 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:20.071 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.459377+0000", 2024-09-21T21:41:20.071 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0056999999999996e-05 2024-09-21T21:41:20.071 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:20.071 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:20.071 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:20.071 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.071 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.071 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2726 3.d 3:b154679f:::rbd_data.112145fae779.0000000000000004:a [read 3567616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:20.071 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.414985+0000", 2024-09-21T21:41:20.071 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 21.570493330000001, 2024-09-21T21:41:20.071 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044526137, 2024-09-21T21:41:20.072 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:20.072 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:20.072 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:20.072 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:20.072 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:20.072 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:20.072 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2726 2024-09-21T21:41:20.072 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.072 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:20.072 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.072 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:20.072 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.414985+0000", 2024-09-21T21:41:20.072 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:20.072 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.073 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.073 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:20.073 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.414985+0000", 2024-09-21T21:41:20.073 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:20.073 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.073 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.073 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:20.073 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.414985+0000", 2024-09-21T21:41:20.073 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0600000000000001e-07 2024-09-21T21:41:20.073 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.073 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.073 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:20.073 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.414986+0000", 2024-09-21T21:41:20.073 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.175e-06 2024-09-21T21:41:20.073 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.074 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.074 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:20.074 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.414989+0000", 2024-09-21T21:41:20.074 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8720000000000002e-06 2024-09-21T21:41:20.074 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.074 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.074 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:20.074 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.414992+0000", 2024-09-21T21:41:20.074 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3680000000000002e-06 2024-09-21T21:41:20.074 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.074 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.074 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:20.074 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.459415+0000", 2024-09-21T21:41:20.075 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044422167999999998 2024-09-21T21:41:20.075 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.075 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.075 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:20.075 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.459466+0000", 2024-09-21T21:41:20.075 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1177000000000001e-05 2024-09-21T21:41:20.075 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.075 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.075 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:20.075 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.459511+0000", 2024-09-21T21:41:20.075 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5071000000000003e-05 2024-09-21T21:41:20.075 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:20.075 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:20.075 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:20.075 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.076 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.076 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2727 3.d 3:b154679f:::rbd_data.112145fae779.0000000000000004:a [read 3624960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:20.076 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.414997+0000", 2024-09-21T21:41:20.076 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 21.570480756999999, 2024-09-21T21:41:20.076 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044651328999999997, 2024-09-21T21:41:20.076 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:20.076 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:20.076 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:20.076 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:20.076 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:20.076 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:20.076 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2727 2024-09-21T21:41:20.076 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.076 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:20.076 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.077 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:20.077 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.414997+0000", 2024-09-21T21:41:20.077 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:20.077 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.077 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.077 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:20.077 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.414997+0000", 2024-09-21T21:41:20.077 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:20.077 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.077 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.077 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:20.077 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.416868+0000", 2024-09-21T21:41:20.077 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001871019 2024-09-21T21:41:20.077 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.078 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.078 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:20.078 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.416873+0000", 2024-09-21T21:41:20.078 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6299999999999997e-06 2024-09-21T21:41:20.078 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.078 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.078 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:20.078 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.416883+0000", 2024-09-21T21:41:20.078 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0172e-05 2024-09-21T21:41:20.078 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.078 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.078 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:20.078 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.416890+0000", 2024-09-21T21:41:20.078 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8689999999999998e-06 2024-09-21T21:41:20.078 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.079 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.079 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:20.079 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.459541+0000", 2024-09-21T21:41:20.079 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042650592000000001 2024-09-21T21:41:20.079 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.079 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.079 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:20.079 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.459601+0000", 2024-09-21T21:41:20.079 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9978000000000001e-05 2024-09-21T21:41:20.079 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.079 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.079 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:20.079 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.459649+0000", 2024-09-21T21:41:20.079 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8068999999999997e-05 2024-09-21T21:41:20.079 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:20.080 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:20.080 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:20.080 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:20.080 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:20.080 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:20.081 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T21:41:20.082 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:20.079+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:20.182 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:20.180+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:20.252 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:20.253 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:41:20.253 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-21T21:41:20.253 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:20.263 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T21:41:20.306 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:20.305+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:20.383 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:20.381+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:20.394 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:20.395 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:41:20.395 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:41:20.395 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:41:20.395 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:20.405 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T21:41:20.489 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:20.488+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:20.562 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:20.562 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-21T21:41:20.562 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-21T21:41:20.562 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:41:20.562 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.562 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4840.0:5190 3.1a 3:5fa6cc52:::rbd_header.1121e0174ab1:head [notify cookie 139800381081568] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T21:41:20.562 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:16.463940+0000", 2024-09-21T21:41:20.562 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.0957649629999997, 2024-09-21T21:41:20.562 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.026118738999999998, 2024-09-21T21:41:20.562 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:20.562 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:20.562 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:20.563 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:20.563 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4840", 2024-09-21T21:41:20.563 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1293784138", 2024-09-21T21:41:20.563 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 5190 2024-09-21T21:41:20.563 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.563 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:20.563 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.563 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:20.563 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.463940+0000", 2024-09-21T21:41:20.563 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:20.563 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.563 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.563 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:20.563 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.463940+0000", 2024-09-21T21:41:20.563 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:20.564 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.564 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.564 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:20.564 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.463940+0000", 2024-09-21T21:41:20.564 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5699999999999998e-07 2024-09-21T21:41:20.564 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.564 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.564 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:20.564 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.463942+0000", 2024-09-21T21:41:20.564 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.491e-06 2024-09-21T21:41:20.564 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.564 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.564 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:20.564 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.463951+0000", 2024-09-21T21:41:20.564 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2689999999999999e-06 2024-09-21T21:41:20.565 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.565 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.565 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:20.565 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.463955+0000", 2024-09-21T21:41:20.565 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8070000000000001e-06 2024-09-21T21:41:20.565 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.565 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.565 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:20.565 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.489958+0000", 2024-09-21T21:41:20.565 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.026003480999999998 2024-09-21T21:41:20.565 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.565 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.565 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:20.565 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.489995+0000", 2024-09-21T21:41:20.566 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7265000000000003e-05 2024-09-21T21:41:20.566 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.566 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.566 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:20.566 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.490058+0000", 2024-09-21T21:41:20.566 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3069000000000002e-05 2024-09-21T21:41:20.566 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:20.566 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:20.566 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:20.566 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.566 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.566 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4840.0:5191 3.1a 3:5fa6cc52:::rbd_header.1121e0174ab1:head [notify cookie 139800381093248] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T21:41:20.566 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:16.463965+0000", 2024-09-21T21:41:20.566 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.0957396729999997, 2024-09-21T21:41:20.567 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.026300389, 2024-09-21T21:41:20.567 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:20.567 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:20.567 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:20.567 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:20.567 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4840", 2024-09-21T21:41:20.567 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1293784138", 2024-09-21T21:41:20.567 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 5191 2024-09-21T21:41:20.567 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.567 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:20.567 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.567 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:20.567 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.463965+0000", 2024-09-21T21:41:20.567 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:20.568 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.568 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.568 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:20.568 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.463965+0000", 2024-09-21T21:41:20.568 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:20.568 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.568 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.568 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:20.568 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.463965+0000", 2024-09-21T21:41:20.568 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7800000000000001e-07 2024-09-21T21:41:20.568 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.568 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.568 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:20.568 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.463969+0000", 2024-09-21T21:41:20.568 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1669999999999999e-06 2024-09-21T21:41:20.569 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.569 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.569 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:20.569 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.463975+0000", 2024-09-21T21:41:20.569 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2920000000000001e-06 2024-09-21T21:41:20.569 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.569 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.569 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:20.569 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.463980+0000", 2024-09-21T21:41:20.569 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3580000000000002e-06 2024-09-21T21:41:20.569 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.569 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.569 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:20.569 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.490090+0000", 2024-09-21T21:41:20.569 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.026109614 2024-09-21T21:41:20.570 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.570 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.570 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:20.570 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.490155+0000", 2024-09-21T21:41:20.570 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5542000000000004e-05 2024-09-21T21:41:20.570 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.570 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.570 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:20.570 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.490265+0000", 2024-09-21T21:41:20.570 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000109838 2024-09-21T21:41:20.570 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:20.570 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:20.570 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:20.570 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.570 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.571 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4840.0:5192 3.1a 3:5fa6cc52:::rbd_header.1121e0174ab1:head [notify cookie 139800381104400] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T21:41:20.571 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:16.463987+0000", 2024-09-21T21:41:20.571 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.095717177, 2024-09-21T21:41:20.571 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.026422074, 2024-09-21T21:41:20.571 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:20.571 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:20.571 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:20.571 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:20.571 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4840", 2024-09-21T21:41:20.571 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1293784138", 2024-09-21T21:41:20.571 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 5192 2024-09-21T21:41:20.571 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.571 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:20.571 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.572 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:20.572 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.463987+0000", 2024-09-21T21:41:20.572 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:20.572 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.572 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.572 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:20.572 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.463987+0000", 2024-09-21T21:41:20.572 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:20.572 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.572 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.572 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:20.572 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.463988+0000", 2024-09-21T21:41:20.572 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2e-07 2024-09-21T21:41:20.572 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.572 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.573 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:20.573 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.463990+0000", 2024-09-21T21:41:20.573 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5490000000000001e-06 2024-09-21T21:41:20.573 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.573 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.573 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:20.573 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.463998+0000", 2024-09-21T21:41:20.573 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0190000000000006e-06 2024-09-21T21:41:20.573 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.573 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.573 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:20.573 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464003+0000", 2024-09-21T21:41:20.573 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3810000000000001e-06 2024-09-21T21:41:20.573 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.573 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.574 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:20.574 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.490279+0000", 2024-09-21T21:41:20.574 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.026276520000000001 2024-09-21T21:41:20.574 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.574 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.574 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:20.574 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.490331+0000", 2024-09-21T21:41:20.574 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1999999999999997e-05 2024-09-21T21:41:20.574 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.574 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.574 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:20.574 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.490409+0000", 2024-09-21T21:41:20.574 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8184999999999998e-05 2024-09-21T21:41:20.574 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:20.575 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:20.575 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:20.575 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.575 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.575 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4840.0:5193 3.1a 3:5fa6cc52:::rbd_header.1121e0174ab1:head [notify cookie 139800380193680] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T21:41:20.575 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:16.464011+0000", 2024-09-21T21:41:20.575 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.0956935760000004, 2024-09-21T21:41:20.575 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.026606253, 2024-09-21T21:41:20.575 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:20.575 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:20.575 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:20.575 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:20.575 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4840", 2024-09-21T21:41:20.575 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1293784138", 2024-09-21T21:41:20.576 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 5193 2024-09-21T21:41:20.576 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.576 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:20.576 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.576 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:20.576 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464011+0000", 2024-09-21T21:41:20.576 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:20.576 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.576 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.576 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:20.576 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464011+0000", 2024-09-21T21:41:20.576 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:20.576 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.576 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.577 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:20.577 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464011+0000", 2024-09-21T21:41:20.577 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5600000000000001e-07 2024-09-21T21:41:20.577 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.577 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.577 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:20.577 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464014+0000", 2024-09-21T21:41:20.577 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4619999999999999e-06 2024-09-21T21:41:20.577 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.577 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.577 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:20.577 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464020+0000", 2024-09-21T21:41:20.577 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7819999999999999e-06 2024-09-21T21:41:20.577 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.578 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.578 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:20.578 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464024+0000", 2024-09-21T21:41:20.578 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7820000000000001e-06 2024-09-21T21:41:20.578 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.578 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.578 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:20.578 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.490444+0000", 2024-09-21T21:41:20.578 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.026419257000000002 2024-09-21T21:41:20.578 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.578 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.578 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:20.578 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.490508+0000", 2024-09-21T21:41:20.578 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3961999999999995e-05 2024-09-21T21:41:20.579 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.579 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.579 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:20.579 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.490617+0000", 2024-09-21T21:41:20.579 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000109652 2024-09-21T21:41:20.579 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:20.579 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:20.579 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:20.579 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.579 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.579 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4840.0:5194 3.1a 3:5fa6cc52:::rbd_header.1121e0174ab1:head [notify cookie 139800381118288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T21:41:20.579 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:16.464032+0000", 2024-09-21T21:41:20.579 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.0956727879999999, 2024-09-21T21:41:20.579 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.026691425000000001, 2024-09-21T21:41:20.579 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:20.580 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:20.580 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:20.580 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:20.580 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4840", 2024-09-21T21:41:20.580 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1293784138", 2024-09-21T21:41:20.580 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 5194 2024-09-21T21:41:20.580 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.580 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:20.580 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.580 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:20.580 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464032+0000", 2024-09-21T21:41:20.580 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:20.580 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.580 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.580 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:20.581 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464032+0000", 2024-09-21T21:41:20.581 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:20.581 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.581 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.581 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:20.581 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464032+0000", 2024-09-21T21:41:20.581 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8799999999999998e-07 2024-09-21T21:41:20.581 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.581 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.581 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:20.581 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464034+0000", 2024-09-21T21:41:20.581 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.295e-06 2024-09-21T21:41:20.581 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.581 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.581 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:20.582 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464042+0000", 2024-09-21T21:41:20.582 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5650000000000004e-06 2024-09-21T21:41:20.582 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.582 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.582 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:20.582 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464047+0000", 2024-09-21T21:41:20.582 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.04e-06 2024-09-21T21:41:20.582 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.582 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.582 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:20.582 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.490632+0000", 2024-09-21T21:41:20.582 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.026585233999999999 2024-09-21T21:41:20.582 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.582 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.582 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:20.583 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.490665+0000", 2024-09-21T21:41:20.583 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2203999999999999e-05 2024-09-21T21:41:20.583 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.583 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.583 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:20.583 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.490723+0000", 2024-09-21T21:41:20.583 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8699000000000002e-05 2024-09-21T21:41:20.583 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:20.583 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:20.583 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:20.583 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.583 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.583 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4840.0:5195 3.1a 3:5fa6cc52:::rbd_header.1121e0174ab1:head [notify cookie 139800380588912] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T21:41:20.583 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:16.464057+0000", 2024-09-21T21:41:20.584 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.0956478010000001, 2024-09-21T21:41:20.584 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.026886305999999999, 2024-09-21T21:41:20.584 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:20.584 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:20.584 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:20.584 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:20.584 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4840", 2024-09-21T21:41:20.584 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1293784138", 2024-09-21T21:41:20.584 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 5195 2024-09-21T21:41:20.584 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.584 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:20.584 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.584 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:20.584 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464057+0000", 2024-09-21T21:41:20.585 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:20.585 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.585 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.585 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:20.585 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464057+0000", 2024-09-21T21:41:20.585 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:20.585 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.585 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.585 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:20.585 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464057+0000", 2024-09-21T21:41:20.585 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2599999999999998e-07 2024-09-21T21:41:20.585 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.585 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.585 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:20.585 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464061+0000", 2024-09-21T21:41:20.586 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3330000000000001e-06 2024-09-21T21:41:20.586 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.586 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.586 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:20.586 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464071+0000", 2024-09-21T21:41:20.586 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.9809999999999998e-06 2024-09-21T21:41:20.586 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.586 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.586 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:20.586 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464077+0000", 2024-09-21T21:41:20.586 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0430000000000004e-06 2024-09-21T21:41:20.586 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.586 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.586 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:20.587 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.490754+0000", 2024-09-21T21:41:20.587 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.026677881000000001 2024-09-21T21:41:20.587 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.587 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.587 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:20.587 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.490822+0000", 2024-09-21T21:41:20.587 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8021999999999993e-05 2024-09-21T21:41:20.587 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.587 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.587 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:20.587 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.490943+0000", 2024-09-21T21:41:20.587 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012061999999999999 2024-09-21T21:41:20.588 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:20.588 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:20.588 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:20.588 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.588 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.588 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4840.0:5196 3.1a 3:5fa6cc52:::rbd_header.1121e0174ab1:head [notify cookie 139800380477648] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T21:41:20.588 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:16.464083+0000", 2024-09-21T21:41:20.588 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.095621446, 2024-09-21T21:41:20.588 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.026990353000000002, 2024-09-21T21:41:20.588 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:20.588 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:20.588 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:20.588 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:20.589 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4840", 2024-09-21T21:41:20.589 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1293784138", 2024-09-21T21:41:20.589 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 5196 2024-09-21T21:41:20.589 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.589 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:20.589 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.589 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:20.589 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464083+0000", 2024-09-21T21:41:20.589 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:20.589 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.589 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.589 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:20.589 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464083+0000", 2024-09-21T21:41:20.590 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:20.590 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.590 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.590 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:20.590 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464084+0000", 2024-09-21T21:41:20.590 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9700000000000002e-07 2024-09-21T21:41:20.590 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.590 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.590 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:20.590 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464086+0000", 2024-09-21T21:41:20.590 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4600000000000002e-06 2024-09-21T21:41:20.590 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.590 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.590 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:20.591 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464094+0000", 2024-09-21T21:41:20.591 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0220000000000004e-06 2024-09-21T21:41:20.591 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.591 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.591 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:20.591 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464100+0000", 2024-09-21T21:41:20.591 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8089999999999998e-06 2024-09-21T21:41:20.591 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.591 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.591 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:20.591 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.490971+0000", 2024-09-21T21:41:20.591 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.026871229 2024-09-21T21:41:20.591 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.591 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.592 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:20.592 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.491013+0000", 2024-09-21T21:41:20.592 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1723e-05 2024-09-21T21:41:20.592 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.592 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.592 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:20.592 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.491074+0000", 2024-09-21T21:41:20.592 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0713e-05 2024-09-21T21:41:20.592 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:20.592 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:20.592 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:20.592 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.592 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.592 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4840.0:5197 3.1a 3:5fa6cc52:::rbd_header.1121e0174ab1:head [notify cookie 139800380481024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T21:41:20.592 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:16.464109+0000", 2024-09-21T21:41:20.593 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.095595157, 2024-09-21T21:41:20.593 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.027188467000000001, 2024-09-21T21:41:20.593 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:20.593 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:20.593 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:20.593 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:20.593 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4840", 2024-09-21T21:41:20.593 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1293784138", 2024-09-21T21:41:20.593 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 5197 2024-09-21T21:41:20.593 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.593 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:20.593 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.593 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:20.593 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464109+0000", 2024-09-21T21:41:20.593 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:20.594 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.594 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.594 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:20.594 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464109+0000", 2024-09-21T21:41:20.594 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:20.594 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.594 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.594 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:20.594 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464110+0000", 2024-09-21T21:41:20.594 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.58e-07 2024-09-21T21:41:20.594 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.594 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.594 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:20.594 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464112+0000", 2024-09-21T21:41:20.594 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.553e-06 2024-09-21T21:41:20.595 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.595 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.595 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:20.595 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464119+0000", 2024-09-21T21:41:20.595 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4160000000000001e-06 2024-09-21T21:41:20.595 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.595 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.595 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:20.595 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464125+0000", 2024-09-21T21:41:20.595 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7769999999999999e-06 2024-09-21T21:41:20.595 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.595 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.595 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:20.596 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.491105+0000", 2024-09-21T21:41:20.596 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.026980481000000001 2024-09-21T21:41:20.596 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.596 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.596 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:20.596 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.491176+0000", 2024-09-21T21:41:20.596 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0807000000000002e-05 2024-09-21T21:41:20.596 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.596 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.596 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:20.596 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.491298+0000", 2024-09-21T21:41:20.596 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000122075 2024-09-21T21:41:20.596 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:20.596 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:20.596 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:20.597 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.597 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.597 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4840.0:5198 3.1a 3:5fa6cc52:::rbd_header.1121e0174ab1:head [notify cookie 139800381142928] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T21:41:20.597 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:16.464131+0000", 2024-09-21T21:41:20.597 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.0955740629999999, 2024-09-21T21:41:20.597 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.027266123, 2024-09-21T21:41:20.597 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:20.597 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:20.597 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:20.597 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:20.597 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4840", 2024-09-21T21:41:20.597 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1293784138", 2024-09-21T21:41:20.597 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 5198 2024-09-21T21:41:20.597 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.597 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:20.598 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.598 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:20.598 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464131+0000", 2024-09-21T21:41:20.598 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:20.598 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.598 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.598 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:20.598 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464131+0000", 2024-09-21T21:41:20.598 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:20.598 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.598 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.598 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:20.598 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464131+0000", 2024-09-21T21:41:20.598 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4299999999999999e-07 2024-09-21T21:41:20.599 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.599 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.599 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:20.599 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464134+0000", 2024-09-21T21:41:20.599 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3289999999999998e-06 2024-09-21T21:41:20.599 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.599 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.599 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:20.599 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464140+0000", 2024-09-21T21:41:20.599 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6699999999999999e-06 2024-09-21T21:41:20.599 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.599 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.599 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:20.600 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464144+0000", 2024-09-21T21:41:20.600 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4440000000000001e-06 2024-09-21T21:41:20.600 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.600 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.600 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:20.600 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.491310+0000", 2024-09-21T21:41:20.600 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.027165827 2024-09-21T21:41:20.600 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.600 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.600 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:20.600 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.491344+0000", 2024-09-21T21:41:20.600 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3382e-05 2024-09-21T21:41:20.600 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.600 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.600 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:20.601 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.491397+0000", 2024-09-21T21:41:20.601 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3127999999999997e-05 2024-09-21T21:41:20.601 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:20.601 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:20.601 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:20.601 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.601 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.601 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4840.0:5199 3.1a 3:5fa6cc52:::rbd_header.1121e0174ab1:head [notify cookie 139800381153680] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T21:41:20.601 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:16.464152+0000", 2024-09-21T21:41:20.601 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.0955529799999999, 2024-09-21T21:41:20.601 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.027530745999999998, 2024-09-21T21:41:20.601 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:20.601 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:20.601 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:20.602 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:20.602 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4840", 2024-09-21T21:41:20.602 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1293784138", 2024-09-21T21:41:20.602 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 5199 2024-09-21T21:41:20.602 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.602 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:20.602 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.602 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:20.602 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464152+0000", 2024-09-21T21:41:20.602 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:20.602 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.602 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.602 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:20.602 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464152+0000", 2024-09-21T21:41:20.602 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:20.603 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.603 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.603 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:20.603 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464152+0000", 2024-09-21T21:41:20.603 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8299999999999998e-07 2024-09-21T21:41:20.603 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.603 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.603 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:20.603 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464155+0000", 2024-09-21T21:41:20.603 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.349e-06 2024-09-21T21:41:20.603 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.603 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.603 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:20.603 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464160+0000", 2024-09-21T21:41:20.603 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4150000000000001e-06 2024-09-21T21:41:20.604 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.604 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.604 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:20.604 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464163+0000", 2024-09-21T21:41:20.604 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2499999999999998e-06 2024-09-21T21:41:20.604 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.604 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.604 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:20.604 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.491448+0000", 2024-09-21T21:41:20.604 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.027284503000000002 2024-09-21T21:41:20.604 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.604 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.604 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:20.604 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.491558+0000", 2024-09-21T21:41:20.604 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00011007500000000001 2024-09-21T21:41:20.605 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.605 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.605 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:20.605 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.491682+0000", 2024-09-21T21:41:20.605 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000124771 2024-09-21T21:41:20.605 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:20.605 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:20.605 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:20.605 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.605 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.605 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4840.0:5200 3.1a 3:5fa6cc52:::rbd_header.1121e0174ab1:head [notify cookie 139800381164992] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T21:41:20.605 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:16.464192+0000", 2024-09-21T21:41:20.605 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.0955122900000003, 2024-09-21T21:41:20.605 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.027596809999999999, 2024-09-21T21:41:20.605 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:20.606 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:20.606 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:20.606 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:20.606 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4840", 2024-09-21T21:41:20.606 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1293784138", 2024-09-21T21:41:20.606 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 5200 2024-09-21T21:41:20.606 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.606 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:20.606 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.606 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:20.606 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464192+0000", 2024-09-21T21:41:20.606 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:20.606 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.606 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.607 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:20.607 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464192+0000", 2024-09-21T21:41:20.607 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:20.607 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.607 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.607 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:20.607 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464193+0000", 2024-09-21T21:41:20.607 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5799999999999999e-07 2024-09-21T21:41:20.607 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.607 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.607 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:20.607 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464199+0000", 2024-09-21T21:41:20.607 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5309999999999998e-06 2024-09-21T21:41:20.607 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.608 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.608 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:20.608 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464207+0000", 2024-09-21T21:41:20.608 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2620000000000001e-06 2024-09-21T21:41:20.608 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.608 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.608 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:20.608 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464211+0000", 2024-09-21T21:41:20.608 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5510000000000001e-06 2024-09-21T21:41:20.608 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.608 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.608 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:20.608 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.491696+0000", 2024-09-21T21:41:20.608 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.027484463000000001 2024-09-21T21:41:20.608 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.609 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.609 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:20.609 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.491729+0000", 2024-09-21T21:41:20.609 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3396999999999999e-05 2024-09-21T21:41:20.609 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.609 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.609 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:20.609 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.491789+0000", 2024-09-21T21:41:20.609 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0047999999999998e-05 2024-09-21T21:41:20.609 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:20.609 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:20.609 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:20.609 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.609 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.610 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4840.0:5201 3.1a 3:5fa6cc52:::rbd_header.1121e0174ab1:head [notify cookie 139800381176640] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T21:41:20.610 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:16.464225+0000", 2024-09-21T21:41:20.610 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.0954792680000001, 2024-09-21T21:41:20.610 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.027776392, 2024-09-21T21:41:20.610 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:20.610 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:20.610 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:20.610 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:20.610 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4840", 2024-09-21T21:41:20.610 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1293784138", 2024-09-21T21:41:20.610 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 5201 2024-09-21T21:41:20.610 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.610 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:20.610 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.610 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:20.610 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464225+0000", 2024-09-21T21:41:20.611 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:20.611 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.611 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.611 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:20.611 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464225+0000", 2024-09-21T21:41:20.611 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:20.611 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.611 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.611 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:20.611 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464226+0000", 2024-09-21T21:41:20.611 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8299999999999998e-07 2024-09-21T21:41:20.611 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.611 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.611 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:20.611 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464228+0000", 2024-09-21T21:41:20.612 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.846e-06 2024-09-21T21:41:20.612 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.612 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.612 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:20.612 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464231+0000", 2024-09-21T21:41:20.612 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.675e-06 2024-09-21T21:41:20.612 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.612 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.612 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:20.612 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464235+0000", 2024-09-21T21:41:20.612 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7249999999999999e-06 2024-09-21T21:41:20.612 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.612 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.612 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:20.612 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.491822+0000", 2024-09-21T21:41:20.613 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.027586626999999999 2024-09-21T21:41:20.613 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.613 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.613 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:20.613 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.491894+0000", 2024-09-21T21:41:20.613 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2353000000000003e-05 2024-09-21T21:41:20.613 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.613 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.613 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:20.613 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.492002+0000", 2024-09-21T21:41:20.613 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010778299999999999 2024-09-21T21:41:20.613 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:20.613 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:20.613 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:20.613 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.613 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.614 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4840.0:5202 3.1a 3:5fa6cc52:::rbd_header.1121e0174ab1:head [notify cookie 139800380311792] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T21:41:20.614 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:16.464239+0000", 2024-09-21T21:41:20.614 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.0954659119999999, 2024-09-21T21:41:20.614 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.027979713, 2024-09-21T21:41:20.614 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:20.614 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:20.614 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:20.614 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:20.614 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4840", 2024-09-21T21:41:20.614 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1293784138", 2024-09-21T21:41:20.614 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 5202 2024-09-21T21:41:20.614 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.614 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:20.614 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.614 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:20.615 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464239+0000", 2024-09-21T21:41:20.615 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:20.615 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.615 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.615 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:20.615 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464239+0000", 2024-09-21T21:41:20.615 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:20.615 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.615 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.615 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:20.615 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464239+0000", 2024-09-21T21:41:20.615 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4699999999999998e-07 2024-09-21T21:41:20.615 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.615 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.616 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:20.616 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464240+0000", 2024-09-21T21:41:20.616 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4959999999999999e-06 2024-09-21T21:41:20.616 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.616 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.616 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:20.616 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464244+0000", 2024-09-21T21:41:20.616 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.118e-06 2024-09-21T21:41:20.616 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.616 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.616 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:20.616 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464246+0000", 2024-09-21T21:41:20.616 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4789999999999999e-06 2024-09-21T21:41:20.616 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.617 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.617 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:20.617 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.492033+0000", 2024-09-21T21:41:20.617 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.02778692 2024-09-21T21:41:20.617 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.617 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.617 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:20.617 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.492107+0000", 2024-09-21T21:41:20.617 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3724000000000001e-05 2024-09-21T21:41:20.617 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.617 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.617 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:20.617 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.492218+0000", 2024-09-21T21:41:20.618 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000111729 2024-09-21T21:41:20.618 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:20.618 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:20.618 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:20.618 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.618 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.618 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4840.0:5203 3.1a 3:5fa6cc52:::rbd_header.1121e0174ab1:head [notify cookie 139800380609168] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T21:41:20.618 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:16.464249+0000", 2024-09-21T21:41:20.618 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.095455812, 2024-09-21T21:41:20.618 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.028186418000000001, 2024-09-21T21:41:20.618 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:20.618 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:20.618 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:20.619 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:20.619 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4840", 2024-09-21T21:41:20.619 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1293784138", 2024-09-21T21:41:20.619 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 5203 2024-09-21T21:41:20.619 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.619 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:20.619 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.619 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:20.619 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464249+0000", 2024-09-21T21:41:20.619 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:20.619 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.620 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.620 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:20.620 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464249+0000", 2024-09-21T21:41:20.620 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:20.620 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.620 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.620 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:20.620 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464249+0000", 2024-09-21T21:41:20.620 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.16e-07 2024-09-21T21:41:20.620 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.620 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.620 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:20.620 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464251+0000", 2024-09-21T21:41:20.621 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1859999999999999e-06 2024-09-21T21:41:20.621 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.621 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.621 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:20.621 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464257+0000", 2024-09-21T21:41:20.621 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1340000000000001e-06 2024-09-21T21:41:20.621 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.621 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.621 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:20.621 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464260+0000", 2024-09-21T21:41:20.621 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6240000000000002e-06 2024-09-21T21:41:20.621 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.621 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.621 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:20.622 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.492255+0000", 2024-09-21T21:41:20.622 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.027995181000000001 2024-09-21T21:41:20.622 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.622 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.622 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:20.622 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.492322+0000", 2024-09-21T21:41:20.622 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7139999999999998e-05 2024-09-21T21:41:20.622 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.622 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.622 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:20.622 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.492435+0000", 2024-09-21T21:41:20.622 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00011293700000000001 2024-09-21T21:41:20.622 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:20.622 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:20.623 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:20.623 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.623 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.623 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4840.0:5204 3.1a 3:5fa6cc52:::rbd_header.1121e0174ab1:head [notify cookie 139800381199072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T21:41:20.623 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:16.464263+0000", 2024-09-21T21:41:20.623 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.095441385, 2024-09-21T21:41:20.623 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.028339177, 2024-09-21T21:41:20.623 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:20.623 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:20.623 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:20.623 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:20.623 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4840", 2024-09-21T21:41:20.623 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1293784138", 2024-09-21T21:41:20.624 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 5204 2024-09-21T21:41:20.624 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.624 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:20.624 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.624 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:20.624 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464263+0000", 2024-09-21T21:41:20.624 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:20.624 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.624 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.624 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:20.624 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464263+0000", 2024-09-21T21:41:20.624 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:20.624 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.624 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.624 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:20.625 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464263+0000", 2024-09-21T21:41:20.625 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.22e-07 2024-09-21T21:41:20.625 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.625 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.625 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:20.625 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464265+0000", 2024-09-21T21:41:20.625 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.437e-06 2024-09-21T21:41:20.625 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.625 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.625 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:20.625 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464268+0000", 2024-09-21T21:41:20.625 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.965e-06 2024-09-21T21:41:20.625 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.625 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.625 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:20.626 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464270+0000", 2024-09-21T21:41:20.626 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.565e-06 2024-09-21T21:41:20.626 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.626 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.626 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:20.626 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.492457+0000", 2024-09-21T21:41:20.626 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.028186178999999999 2024-09-21T21:41:20.626 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.626 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.626 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:20.626 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.492520+0000", 2024-09-21T21:41:20.626 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3490000000000004e-05 2024-09-21T21:41:20.626 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.626 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.627 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:20.627 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.492602+0000", 2024-09-21T21:41:20.627 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2318999999999996e-05 2024-09-21T21:41:20.627 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:20.627 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:20.627 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:20.627 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.627 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.627 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4840.0:5205 3.1a 3:5fa6cc52:::rbd_header.1121e0174ab1:head [notify cookie 139800381211120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T21:41:20.627 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:16.464274+0000", 2024-09-21T21:41:20.627 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.0954307520000004, 2024-09-21T21:41:20.627 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.028530798, 2024-09-21T21:41:20.627 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:20.627 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:20.628 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:20.628 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:20.628 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4840", 2024-09-21T21:41:20.628 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1293784138", 2024-09-21T21:41:20.628 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 5205 2024-09-21T21:41:20.628 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.628 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:20.628 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.628 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:20.628 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464274+0000", 2024-09-21T21:41:20.628 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:20.628 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.628 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.629 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:20.629 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464274+0000", 2024-09-21T21:41:20.629 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:20.629 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.629 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.629 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:20.629 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464274+0000", 2024-09-21T21:41:20.629 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1799999999999999e-07 2024-09-21T21:41:20.629 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.629 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.629 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:20.629 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464275+0000", 2024-09-21T21:41:20.629 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3510000000000001e-06 2024-09-21T21:41:20.629 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.629 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.630 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:20.630 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464279+0000", 2024-09-21T21:41:20.630 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1219999999999999e-06 2024-09-21T21:41:20.630 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.630 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.630 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:20.630 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464281+0000", 2024-09-21T21:41:20.630 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5359999999999999e-06 2024-09-21T21:41:20.630 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.630 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.631 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:20.631 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.492652+0000", 2024-09-21T21:41:20.631 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.028371400000000001 2024-09-21T21:41:20.631 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.631 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.631 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:20.631 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.492738+0000", 2024-09-21T21:41:20.631 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5155999999999996e-05 2024-09-21T21:41:20.631 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.631 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.631 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:20.631 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.492805+0000", 2024-09-21T21:41:20.631 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7014999999999994e-05 2024-09-21T21:41:20.631 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:20.631 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:20.632 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:20.632 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.632 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.632 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4840.0:5206 3.1a 3:5fa6cc52:::rbd_header.1121e0174ab1:head [notify cookie 139800380621360] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T21:41:20.632 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:16.464285+0000", 2024-09-21T21:41:20.632 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.0954198609999999, 2024-09-21T21:41:20.632 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.028730286000000001, 2024-09-21T21:41:20.632 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:20.632 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:20.632 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:20.632 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:20.632 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4840", 2024-09-21T21:41:20.632 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1293784138", 2024-09-21T21:41:20.632 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 5206 2024-09-21T21:41:20.632 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.632 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:20.633 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.633 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:20.633 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464285+0000", 2024-09-21T21:41:20.633 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:20.633 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.633 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.633 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:20.633 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464285+0000", 2024-09-21T21:41:20.633 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:20.633 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.633 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.633 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:20.633 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464285+0000", 2024-09-21T21:41:20.633 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2600000000000001e-07 2024-09-21T21:41:20.633 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.634 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.634 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:20.634 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464286+0000", 2024-09-21T21:41:20.634 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.345e-06 2024-09-21T21:41:20.634 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.634 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.634 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:20.634 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464289+0000", 2024-09-21T21:41:20.634 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9340000000000002e-06 2024-09-21T21:41:20.634 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.634 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.634 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:20.634 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464292+0000", 2024-09-21T21:41:20.634 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5220000000000002e-06 2024-09-21T21:41:20.634 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.634 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.635 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:20.635 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.492837+0000", 2024-09-21T21:41:20.635 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.028544773999999998 2024-09-21T21:41:20.635 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.635 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.635 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:20.635 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.492903+0000", 2024-09-21T21:41:20.635 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6086000000000004e-05 2024-09-21T21:41:20.635 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.636 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.636 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:20.636 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.493015+0000", 2024-09-21T21:41:20.636 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00011239900000000001 2024-09-21T21:41:20.636 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:20.636 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:20.636 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:20.636 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.636 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.636 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4840.0:5207 3.1a 3:5fa6cc52:::rbd_header.1121e0174ab1:head [notify cookie 139800380633968] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T21:41:20.636 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:16.464294+0000", 2024-09-21T21:41:20.636 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.0954101270000001, 2024-09-21T21:41:20.636 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.028879564999999999, 2024-09-21T21:41:20.636 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:20.637 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:20.637 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:20.637 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:20.637 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4840", 2024-09-21T21:41:20.637 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1293784138", 2024-09-21T21:41:20.637 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 5207 2024-09-21T21:41:20.637 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.637 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:20.637 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.637 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:20.637 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464294+0000", 2024-09-21T21:41:20.637 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:20.637 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.637 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.638 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:20.638 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464294+0000", 2024-09-21T21:41:20.638 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:20.638 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.638 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.638 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:20.638 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464295+0000", 2024-09-21T21:41:20.638 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.05e-07 2024-09-21T21:41:20.638 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.638 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.638 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:20.639 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464297+0000", 2024-09-21T21:41:20.639 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9910000000000001e-06 2024-09-21T21:41:20.639 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.639 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.639 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:20.639 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464301+0000", 2024-09-21T21:41:20.639 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8909999999999997e-06 2024-09-21T21:41:20.639 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.639 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.639 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:20.639 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464306+0000", 2024-09-21T21:41:20.639 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2469999999999999e-06 2024-09-21T21:41:20.639 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.639 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.639 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:20.640 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.493047+0000", 2024-09-21T21:41:20.640 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.028740875999999999 2024-09-21T21:41:20.640 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.640 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.640 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:20.640 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.493103+0000", 2024-09-21T21:41:20.640 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6651999999999999e-05 2024-09-21T21:41:20.640 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.640 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.640 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:20.640 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.493174+0000", 2024-09-21T21:41:20.640 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0703000000000004e-05 2024-09-21T21:41:20.641 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:20.641 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:20.641 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:20.641 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.641 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.641 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4840.0:5208 3.1a 3:5fa6cc52:::rbd_header.1121e0174ab1:head [notify cookie 139800381241120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T21:41:20.641 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:16.464325+0000", 2024-09-21T21:41:20.641 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.0953799999999996, 2024-09-21T21:41:20.641 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.028998797999999999, 2024-09-21T21:41:20.641 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:20.641 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:20.641 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:20.642 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:20.642 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4840", 2024-09-21T21:41:20.642 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1293784138", 2024-09-21T21:41:20.642 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 5208 2024-09-21T21:41:20.642 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.642 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:20.642 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.642 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:20.642 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464325+0000", 2024-09-21T21:41:20.642 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:20.642 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.642 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.642 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:20.642 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464325+0000", 2024-09-21T21:41:20.643 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:20.643 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.643 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.643 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:20.643 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464325+0000", 2024-09-21T21:41:20.643 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9900000000000001e-07 2024-09-21T21:41:20.643 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.643 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.643 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:20.643 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464328+0000", 2024-09-21T21:41:20.643 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4399999999999999e-06 2024-09-21T21:41:20.643 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.643 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.643 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:20.643 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464335+0000", 2024-09-21T21:41:20.644 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8839999999999994e-06 2024-09-21T21:41:20.644 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.644 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.644 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:20.644 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464340+0000", 2024-09-21T21:41:20.644 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5959999999999997e-06 2024-09-21T21:41:20.644 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.644 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.644 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:20.644 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.493210+0000", 2024-09-21T21:41:20.644 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.028869789999999999 2024-09-21T21:41:20.644 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.644 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.644 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:20.645 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.493264+0000", 2024-09-21T21:41:20.645 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3947999999999999e-05 2024-09-21T21:41:20.645 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.645 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.645 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:20.645 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.493323+0000", 2024-09-21T21:41:20.645 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9641e-05 2024-09-21T21:41:20.645 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:20.645 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:20.645 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:20.645 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.645 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.645 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4840.0:5209 3.1a 3:5fa6cc52:::rbd_header.1121e0174ab1:head [notify cookie 139800380495120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T21:41:20.646 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:16.464364+0000", 2024-09-21T21:41:20.646 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.095340738, 2024-09-21T21:41:20.646 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.029077913, 2024-09-21T21:41:20.646 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:20.646 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:20.646 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:20.646 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:20.646 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4840", 2024-09-21T21:41:20.646 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1293784138", 2024-09-21T21:41:20.646 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 5209 2024-09-21T21:41:20.646 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.646 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:20.646 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.647 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:20.647 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464364+0000", 2024-09-21T21:41:20.647 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:20.647 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.647 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.647 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:20.647 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464364+0000", 2024-09-21T21:41:20.647 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:20.647 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.647 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.647 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:20.647 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464364+0000", 2024-09-21T21:41:20.647 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4099999999999999e-07 2024-09-21T21:41:20.647 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.648 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.648 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:20.648 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464367+0000", 2024-09-21T21:41:20.648 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.379e-06 2024-09-21T21:41:20.648 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.648 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.648 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:20.648 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464371+0000", 2024-09-21T21:41:20.648 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.403e-06 2024-09-21T21:41:20.648 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.648 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.648 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:20.648 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464374+0000", 2024-09-21T21:41:20.648 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9579999999999999e-06 2024-09-21T21:41:20.649 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.649 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.649 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:20.649 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.493352+0000", 2024-09-21T21:41:20.649 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.028977889999999999 2024-09-21T21:41:20.649 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.649 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.649 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:20.649 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.493387+0000", 2024-09-21T21:41:20.649 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4613000000000002e-05 2024-09-21T21:41:20.649 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:20.649 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:20.649 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:20.649 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.493442+0000", 2024-09-21T21:41:20.649 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5129e-05 2024-09-21T21:41:20.649 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:20.649 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:20.650 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:20.650 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:20.650 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:20.650 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:20.650 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T21:41:20.651 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:20.649+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:20.652 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T21:41:20.696 INFO:teuthology.orchestra.run.smithi164.stdout:2612 write 0x5139800 thru 0x514183b (0x803c bytes) 2024-09-21T21:41:20.697 INFO:teuthology.orchestra.run.smithi164.stdout:2613 read 0x54e000 thru 0x5507b5 (0x27b6 bytes) 2024-09-21T21:41:20.699 INFO:teuthology.orchestra.run.smithi164.stdout:2616 trunc from 0x8fec600 to 0x7fb3e00 2024-09-21T21:41:20.752 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:20.751+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:20.764 INFO:teuthology.orchestra.run.smithi164.stdout:2617 read 0x5bb0400 thru 0x5bbe4da (0xe0db bytes) 2024-09-21T21:41:20.771 INFO:teuthology.orchestra.run.smithi164.stdout:2618 read 0x7725a00 thru 0x772f57f (0x9b80 bytes) 2024-09-21T21:41:20.772 INFO:teuthology.orchestra.run.smithi164.stdout:2619 punch from 0x6061200 to 0x6066400, (0x5200 bytes) 2024-09-21T21:41:20.772 INFO:teuthology.orchestra.run.smithi164.stdout:2622 write 0x7102a00 thru 0x7105b34 (0x3135 bytes) 2024-09-21T21:41:20.773 INFO:teuthology.orchestra.run.smithi164.stdout:2623 read 0x5927c00 thru 0x592a3dd (0x27de bytes) 2024-09-21T21:41:20.773 INFO:teuthology.orchestra.run.smithi164.stdout:2625 read 0x7529600 thru 0x752e72d (0x512e bytes) 2024-09-21T21:41:20.774 INFO:teuthology.orchestra.run.smithi164.stdout:2626 read 0x51f9600 thru 0x5207e16 (0xe817 bytes) 2024-09-21T21:41:20.774 INFO:teuthology.orchestra.run.smithi164.stdout:2630 read 0xe31400 thru 0xe32e4c (0x1a4d bytes) 2024-09-21T21:41:20.774 INFO:teuthology.orchestra.run.smithi164.stdout:2634 write 0x7091000 thru 0x70957f1 (0x47f2 bytes) 2024-09-21T21:41:20.775 INFO:teuthology.orchestra.run.smithi164.stdout:2635 read 0x4e4fc00 thru 0x4e537aa (0x3bab bytes) 2024-09-21T21:41:20.775 INFO:teuthology.orchestra.run.smithi164.stdout:2638 write 0x151b600 thru 0x1526f87 (0xb988 bytes) 2024-09-21T21:41:20.776 INFO:teuthology.orchestra.run.smithi164.stdout:2639 write 0x3f62400 thru 0x3f6eb10 (0xc711 bytes) 2024-09-21T21:41:20.776 INFO:teuthology.orchestra.run.smithi164.stdout:2643 write 0x67f3e00 thru 0x67fc338 (0x8539 bytes) 2024-09-21T21:41:20.776 INFO:teuthology.orchestra.run.smithi164.stdout:2644 read 0x269a600 thru 0x269c4c3 (0x1ec4 bytes) 2024-09-21T21:41:20.777 INFO:teuthology.orchestra.run.smithi164.stdout:2645 punch from 0x244f000 to 0x245b600, (0xc600 bytes) 2024-09-21T21:41:20.777 INFO:teuthology.orchestra.run.smithi164.stdout:2646 write 0x330200 thru 0x3340fd (0x3efe bytes) 2024-09-21T21:41:20.798 INFO:teuthology.orchestra.run.smithi164.stdout:2647 write 0xd666000 thru 0xd66c123 (0x6124 bytes) 2024-09-21T21:41:20.804 INFO:teuthology.orchestra.run.smithi164.stdout:2648 write 0x4fbd600 thru 0x4fcb32e (0xdd2f bytes) 2024-09-21T21:41:20.810 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:20.811 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:41:20.811 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-21T21:41:20.811 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:20.816 INFO:teuthology.orchestra.run.smithi164.stdout:2649 trunc from 0xd66c200 to 0x2fdda00 2024-09-21T21:41:20.820 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T21:41:20.845 INFO:teuthology.orchestra.run.smithi164.stdout:2655 read 0x2856c00 thru 0x28627b3 (0xbbb4 bytes) 2024-09-21T21:41:20.852 INFO:teuthology.orchestra.run.smithi164.stdout:2657 read 0x220b200 thru 0x220bd7c (0xb7d bytes) 2024-09-21T21:41:20.853 INFO:teuthology.orchestra.run.smithi164.stdout:2658 read 0x1412200 thru 0x1420807 (0xe608 bytes) 2024-09-21T21:41:20.853 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:20.851+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:20.875 INFO:teuthology.orchestra.run.smithi164.stdout:2659 write 0xde75600 thru 0xde80a24 (0xb425 bytes) 2024-09-21T21:41:20.880 INFO:teuthology.orchestra.run.smithi164.stdout:2660 write 0xd833200 thru 0xd83d740 (0xa541 bytes) 2024-09-21T21:41:20.896 INFO:teuthology.orchestra.run.smithi164.stdout:2662 trunc from 0xde80c00 to 0x10aec00 2024-09-21T21:41:20.939 INFO:teuthology.orchestra.run.smithi164.stdout:2664 trunc from 0x10aec00 to 0xc581a00 2024-09-21T21:41:20.954 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:20.952+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:20.959 INFO:teuthology.orchestra.run.smithi164.stdout:2666 write 0xd25c200 thru 0xd25ff10 (0x3d11 bytes) 2024-09-21T21:41:20.960 INFO:teuthology.orchestra.run.smithi164.stdout:2669 read 0x20cf800 thru 0x20dd219 (0xda1a bytes) 2024-09-21T21:41:20.960 INFO:teuthology.orchestra.run.smithi164.stdout:2671 read 0x938f200 thru 0x939c894 (0xd695 bytes) 2024-09-21T21:41:20.961 INFO:teuthology.orchestra.run.smithi164.stdout:2672 read 0xc0f2c00 thru 0xc0f9deb (0x71ec bytes) 2024-09-21T21:41:20.961 INFO:teuthology.orchestra.run.smithi164.stdout:2674 read 0x1289400 thru 0x129936a (0xff6b bytes) 2024-09-21T21:41:20.961 INFO:teuthology.orchestra.run.smithi164.stdout:2675 read 0x8e23400 thru 0x8e2e31a (0xaf1b bytes) 2024-09-21T21:41:20.971 INFO:teuthology.orchestra.run.smithi164.stdout:2676 trunc from 0xd260000 to 0x6856000 2024-09-21T21:41:20.973 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:20.973 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:41:20.973 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:41:20.973 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:41:20.973 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:20.982 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T21:41:20.991 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T21:41:20.991 INFO:teuthology.orchestra.run.smithi164.stdout:2678 punch from 0xf8f000 to 0xf9d200, (0xe200 bytes) 2024-09-21T21:41:20.994 INFO:teuthology.orchestra.run.smithi164.stdout:2680 punch from 0x3b4d800 to 0x3b56600, (0x8e00 bytes) 2024-09-21T21:41:21.009 INFO:teuthology.orchestra.run.smithi164.stdout:2681 write 0xc643600 thru 0xc648967 (0x5368 bytes) 2024-09-21T21:41:21.025 INFO:teuthology.orchestra.run.smithi164.stdout:2682 trunc from 0xc648a00 to 0x5630e00 2024-09-21T21:41:21.044 INFO:teuthology.orchestra.run.smithi164.stdout:2684 punch from 0xc47000 to 0xc54800, (0xd800 bytes) 2024-09-21T21:41:21.055 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:21.053+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:21.061 INFO:teuthology.orchestra.run.smithi164.stdout:2686 trunc from 0x5630e00 to 0xcbb5600 2024-09-21T21:41:21.070 INFO:teuthology.orchestra.run.smithi164.stdout:2687 write 0x5569a00 thru 0x5571878 (0x7e79 bytes) 2024-09-21T21:41:21.083 INFO:teuthology.orchestra.run.smithi164.stdout:2689 trunc from 0xcbb5600 to 0x7984e00 2024-09-21T21:41:21.106 INFO:teuthology.orchestra.run.smithi164.stdout:2690 punch from 0x5c1a200 to 0x5c21c00, (0x7a00 bytes) 2024-09-21T21:41:21.119 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0: {} 2024-09-21T21:41:21.119 INFO:teuthology.orchestra.run.smithi081.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T21:41:21.123 INFO:teuthology.orchestra.run.smithi164.stdout:2691 write 0xe80b600 thru 0xe812c44 (0x7645 bytes) 2024-09-21T21:41:21.124 INFO:teuthology.orchestra.run.smithi164.stdout:2693 punch from 0x106c800 to 0x1077200, (0xaa00 bytes) 2024-09-21T21:41:21.129 INFO:teuthology.orchestra.run.smithi164.stdout:2695 write 0xe0d3600 thru 0xe0dec24 (0xb625 bytes) 2024-09-21T21:41:21.135 INFO:teuthology.orchestra.run.smithi164.stdout:2696 trunc from 0xe812e00 to 0x9587800 2024-09-21T21:41:21.137 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1: {} 2024-09-21T21:41:21.137 INFO:teuthology.orchestra.run.smithi081.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T21:41:21.138 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:21.138 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-21T21:41:21.138 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-21T21:41:21.138 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:41:21.138 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.139 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1484 3.16 3:6a41a830:::rbd_data.112145fae779.000000000000000a:a [read 303104~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:21.139 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634086+0000", 2024-09-21T21:41:21.139 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 19.502032944, 2024-09-21T21:41:21.139 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.058257055000000002, 2024-09-21T21:41:21.139 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:21.139 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:21.139 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:21.139 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:21.139 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:21.139 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:21.139 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1484 2024-09-21T21:41:21.139 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.139 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:21.139 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.140 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:21.140 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634086+0000", 2024-09-21T21:41:21.140 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:21.140 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.140 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.140 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:21.140 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634086+0000", 2024-09-21T21:41:21.140 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:21.140 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.140 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.140 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:21.140 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634086+0000", 2024-09-21T21:41:21.140 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6100000000000001e-07 2024-09-21T21:41:21.140 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.141 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.141 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:21.141 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634088+0000", 2024-09-21T21:41:21.141 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1100000000000001e-06 2024-09-21T21:41:21.141 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.141 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.141 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:21.141 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634095+0000", 2024-09-21T21:41:21.141 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4080000000000003e-06 2024-09-21T21:41:21.141 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.141 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.141 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:21.141 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634100+0000", 2024-09-21T21:41:21.141 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0730000000000004e-06 2024-09-21T21:41:21.141 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.142 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.142 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:21.142 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.692197+0000", 2024-09-21T21:41:21.142 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.058097331000000002 2024-09-21T21:41:21.142 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.142 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.142 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:21.142 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.692271+0000", 2024-09-21T21:41:21.142 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3423999999999994e-05 2024-09-21T21:41:21.142 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.142 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.142 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:21.142 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.692343+0000", 2024-09-21T21:41:21.142 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2248000000000004e-05 2024-09-21T21:41:21.143 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:21.143 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:21.143 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:21.143 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.143 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.143 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1485 3.16 3:695c582e:::rbd_data.1121e540c096.0000000000000008:b [read 335872~36864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:21.143 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634106+0000", 2024-09-21T21:41:21.143 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 19.502013147, 2024-09-21T21:41:21.143 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059223685999999998, 2024-09-21T21:41:21.143 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:21.143 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:21.143 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:21.143 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:21.144 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:21.144 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:21.144 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1485 2024-09-21T21:41:21.144 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.144 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:21.144 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.144 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:21.144 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634106+0000", 2024-09-21T21:41:21.144 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:21.144 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.144 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.144 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:21.144 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634106+0000", 2024-09-21T21:41:21.144 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:21.145 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.145 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.145 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:21.145 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634106+0000", 2024-09-21T21:41:21.145 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3300000000000002e-07 2024-09-21T21:41:21.145 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.145 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.145 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:21.145 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634108+0000", 2024-09-21T21:41:21.145 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.996e-06 2024-09-21T21:41:21.145 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.145 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.145 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:21.145 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634115+0000", 2024-09-21T21:41:21.145 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5139999999999998e-06 2024-09-21T21:41:21.146 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.146 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.146 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:21.146 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634120+0000", 2024-09-21T21:41:21.146 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1050000000000003e-06 2024-09-21T21:41:21.146 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.146 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.146 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:21.146 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.692374+0000", 2024-09-21T21:41:21.146 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.058254729999999998 2024-09-21T21:41:21.146 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.146 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.146 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:21.146 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.692440+0000", 2024-09-21T21:41:21.147 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6076999999999998e-05 2024-09-21T21:41:21.147 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.147 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.147 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:21.147 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693329+0000", 2024-09-21T21:41:21.147 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00088873100000000003 2024-09-21T21:41:21.147 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:21.147 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:21.147 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:21.147 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.147 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.147 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1486 3.16 3:6a41a830:::rbd_data.112145fae779.000000000000000a:a [read 360448~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:21.147 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634126+0000", 2024-09-21T21:41:21.147 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 19.501992855000001, 2024-09-21T21:41:21.148 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059399528, 2024-09-21T21:41:21.148 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:21.148 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:21.148 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:21.148 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:21.148 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:21.148 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:21.148 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1486 2024-09-21T21:41:21.148 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.148 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:21.148 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.148 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:21.148 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634126+0000", 2024-09-21T21:41:21.148 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:21.149 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.149 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.149 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:21.149 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634126+0000", 2024-09-21T21:41:21.149 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:21.149 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.149 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.149 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:21.149 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634126+0000", 2024-09-21T21:41:21.149 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4099999999999999e-07 2024-09-21T21:41:21.149 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.149 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.149 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:21.149 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634128+0000", 2024-09-21T21:41:21.149 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1069999999999999e-06 2024-09-21T21:41:21.150 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.150 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.150 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:21.150 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634135+0000", 2024-09-21T21:41:21.150 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.212e-06 2024-09-21T21:41:21.150 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.150 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.150 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:21.150 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634140+0000", 2024-09-21T21:41:21.150 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1429999999999997e-06 2024-09-21T21:41:21.150 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.150 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.150 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:21.150 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693363+0000", 2024-09-21T21:41:21.150 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059223614000000001 2024-09-21T21:41:21.151 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.151 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.151 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:21.151 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693431+0000", 2024-09-21T21:41:21.151 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7294999999999999e-05 2024-09-21T21:41:21.151 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.151 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.151 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:21.151 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693525+0000", 2024-09-21T21:41:21.151 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.4716000000000006e-05 2024-09-21T21:41:21.151 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:21.151 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:21.151 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:21.151 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.151 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.152 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1499 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 49152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:21.152 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634186+0000", 2024-09-21T21:41:21.152 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 19.501932977999999, 2024-09-21T21:41:21.152 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059339256999999999, 2024-09-21T21:41:21.152 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:21.152 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:21.152 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:21.152 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:21.152 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:21.152 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:21.152 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1499 2024-09-21T21:41:21.152 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.152 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:21.153 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.153 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:21.153 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634186+0000", 2024-09-21T21:41:21.153 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:21.153 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.153 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.153 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:21.153 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634186+0000", 2024-09-21T21:41:21.153 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:21.153 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.153 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.153 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:21.153 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634186+0000", 2024-09-21T21:41:21.153 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0999999999999999e-07 2024-09-21T21:41:21.153 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.154 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.154 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:21.154 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634188+0000", 2024-09-21T21:41:21.154 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.86e-06 2024-09-21T21:41:21.154 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.154 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.154 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:21.154 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634194+0000", 2024-09-21T21:41:21.154 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1109999999999996e-06 2024-09-21T21:41:21.154 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.154 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.154 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:21.154 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634199+0000", 2024-09-21T21:41:21.154 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7419999999999997e-06 2024-09-21T21:41:21.155 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.155 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.155 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:21.155 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693357+0000", 2024-09-21T21:41:21.155 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059157847 2024-09-21T21:41:21.155 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.155 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.155 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:21.155 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693430+0000", 2024-09-21T21:41:21.155 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3203999999999999e-05 2024-09-21T21:41:21.155 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.155 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.155 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:21.155 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693525+0000", 2024-09-21T21:41:21.155 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5082999999999997e-05 2024-09-21T21:41:21.156 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:21.156 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:21.156 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:21.156 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.156 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.156 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1501 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 106496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:21.156 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634205+0000", 2024-09-21T21:41:21.156 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 19.501913888000001, 2024-09-21T21:41:21.156 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059482394000000001, 2024-09-21T21:41:21.156 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:21.156 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:21.156 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:21.156 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:21.156 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:21.156 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:21.157 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1501 2024-09-21T21:41:21.157 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.157 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:21.157 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.157 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:21.157 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634205+0000", 2024-09-21T21:41:21.157 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:21.157 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.157 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.157 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:21.157 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634205+0000", 2024-09-21T21:41:21.157 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:21.157 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.157 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.158 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:21.158 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634205+0000", 2024-09-21T21:41:21.158 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.03e-07 2024-09-21T21:41:21.158 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.158 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.158 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:21.158 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634207+0000", 2024-09-21T21:41:21.158 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0389999999999999e-06 2024-09-21T21:41:21.158 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.158 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.158 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:21.158 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634213+0000", 2024-09-21T21:41:21.158 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2600000000000002e-06 2024-09-21T21:41:21.159 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.159 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.159 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:21.159 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634218+0000", 2024-09-21T21:41:21.159 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8570000000000003e-06 2024-09-21T21:41:21.159 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.159 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.159 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:21.159 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693552+0000", 2024-09-21T21:41:21.159 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059333636000000002 2024-09-21T21:41:21.159 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.159 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.159 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:21.159 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693613+0000", 2024-09-21T21:41:21.159 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0560000000000003e-05 2024-09-21T21:41:21.160 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.160 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.160 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:21.160 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693687+0000", 2024-09-21T21:41:21.160 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4639000000000001e-05 2024-09-21T21:41:21.160 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:21.160 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:21.160 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:21.160 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.160 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.160 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1515 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 434176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:21.160 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634318+0000", 2024-09-21T21:41:21.160 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 19.501800201000002, 2024-09-21T21:41:21.160 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059565812000000003, 2024-09-21T21:41:21.161 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:21.161 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:21.161 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:21.161 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:21.161 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:21.161 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:21.161 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1515 2024-09-21T21:41:21.161 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.161 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:21.161 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.161 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:21.161 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634318+0000", 2024-09-21T21:41:21.161 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:21.161 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.162 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.162 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:21.162 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634318+0000", 2024-09-21T21:41:21.162 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:21.162 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.162 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.162 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:21.162 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634319+0000", 2024-09-21T21:41:21.162 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9400000000000001e-07 2024-09-21T21:41:21.162 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.162 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.162 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:21.162 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634321+0000", 2024-09-21T21:41:21.162 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.154e-06 2024-09-21T21:41:21.163 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.163 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.163 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:21.163 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634327+0000", 2024-09-21T21:41:21.163 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9830000000000001e-06 2024-09-21T21:41:21.163 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.163 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.163 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:21.163 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634332+0000", 2024-09-21T21:41:21.163 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9080000000000003e-06 2024-09-21T21:41:21.163 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.163 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.163 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:21.163 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693717+0000", 2024-09-21T21:41:21.163 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059385447000000001 2024-09-21T21:41:21.164 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.164 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.164 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:21.164 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693795+0000", 2024-09-21T21:41:21.164 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7267000000000004e-05 2024-09-21T21:41:21.164 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.164 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.164 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:21.164 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693884+0000", 2024-09-21T21:41:21.164 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9659000000000003e-05 2024-09-21T21:41:21.164 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:21.164 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:21.165 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:21.165 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.165 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.165 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1517 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 491520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:21.165 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634338+0000", 2024-09-21T21:41:21.165 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 19.501780428, 2024-09-21T21:41:21.165 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060100745999999997, 2024-09-21T21:41:21.165 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:21.165 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:21.165 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:21.165 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:21.165 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:21.165 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:21.165 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1517 2024-09-21T21:41:21.166 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.166 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:21.166 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.166 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:21.166 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634338+0000", 2024-09-21T21:41:21.166 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:21.166 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.166 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.166 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:21.166 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634338+0000", 2024-09-21T21:41:21.166 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:21.166 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.166 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.166 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:21.166 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634339+0000", 2024-09-21T21:41:21.167 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2899999999999999e-07 2024-09-21T21:41:21.167 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.167 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.167 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:21.167 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634341+0000", 2024-09-21T21:41:21.167 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.8959999999999999e-06 2024-09-21T21:41:21.167 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.167 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.167 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:21.167 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634347+0000", 2024-09-21T21:41:21.167 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.302e-06 2024-09-21T21:41:21.167 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.167 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.167 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:21.167 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634351+0000", 2024-09-21T21:41:21.168 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6070000000000001e-06 2024-09-21T21:41:21.168 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.168 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.168 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:21.168 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694279+0000", 2024-09-21T21:41:21.168 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059927046999999997 2024-09-21T21:41:21.168 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.168 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.168 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:21.168 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694355+0000", 2024-09-21T21:41:21.168 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6483999999999995e-05 2024-09-21T21:41:21.168 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.168 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.168 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:21.169 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694439+0000", 2024-09-21T21:41:21.169 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3980999999999994e-05 2024-09-21T21:41:21.169 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:21.169 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:21.169 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:21.169 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.169 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.169 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1518 3.16 3:6a41a830:::rbd_data.112145fae779.000000000000000a:a [read 294912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:21.169 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634357+0000", 2024-09-21T21:41:21.169 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 19.501761521999999, 2024-09-21T21:41:21.169 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059712318, 2024-09-21T21:41:21.169 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:21.169 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:21.169 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:21.169 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:21.170 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:21.170 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:21.170 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1518 2024-09-21T21:41:21.170 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.170 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:21.170 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.170 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:21.170 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634357+0000", 2024-09-21T21:41:21.170 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:21.170 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.170 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.170 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:21.170 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634357+0000", 2024-09-21T21:41:21.170 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:21.171 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.171 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.171 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:21.171 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634358+0000", 2024-09-21T21:41:21.171 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2100000000000002e-07 2024-09-21T21:41:21.171 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.171 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.171 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:21.171 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634360+0000", 2024-09-21T21:41:21.171 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0940000000000002e-06 2024-09-21T21:41:21.171 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.171 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.171 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:21.171 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634366+0000", 2024-09-21T21:41:21.172 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3389999999999998e-06 2024-09-21T21:41:21.172 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.172 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.172 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:21.172 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634371+0000", 2024-09-21T21:41:21.172 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6670000000000004e-06 2024-09-21T21:41:21.172 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.172 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.172 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:21.172 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693910+0000", 2024-09-21T21:41:21.172 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059538973000000002 2024-09-21T21:41:21.172 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.172 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.172 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:21.172 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693982+0000", 2024-09-21T21:41:21.173 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2806999999999996e-05 2024-09-21T21:41:21.173 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.173 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.173 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:21.173 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694069+0000", 2024-09-21T21:41:21.173 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7016999999999997e-05 2024-09-21T21:41:21.173 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:21.173 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:21.173 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:21.173 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.173 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.173 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1519 3.16 3:6a41a830:::rbd_data.112145fae779.000000000000000a:a [read 352256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:21.173 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634377+0000", 2024-09-21T21:41:21.173 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 19.501741819999999, 2024-09-21T21:41:21.173 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059872213000000001, 2024-09-21T21:41:21.174 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:21.174 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:21.174 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:21.174 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:21.174 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:21.174 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:21.174 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1519 2024-09-21T21:41:21.174 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.174 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:21.174 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.174 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:21.174 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634377+0000", 2024-09-21T21:41:21.174 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:21.174 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.174 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.175 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:21.175 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634377+0000", 2024-09-21T21:41:21.175 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:21.175 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.175 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.175 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:21.175 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634377+0000", 2024-09-21T21:41:21.175 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2600000000000002e-07 2024-09-21T21:41:21.175 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.175 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.175 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:21.175 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634379+0000", 2024-09-21T21:41:21.175 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0729999999999999e-06 2024-09-21T21:41:21.175 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.176 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.176 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:21.176 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634386+0000", 2024-09-21T21:41:21.176 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4690000000000002e-06 2024-09-21T21:41:21.176 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.176 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.176 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:21.176 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634390+0000", 2024-09-21T21:41:21.176 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4580000000000003e-06 2024-09-21T21:41:21.176 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.176 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.176 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:21.176 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694094+0000", 2024-09-21T21:41:21.176 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059703830999999999 2024-09-21T21:41:21.176 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.176 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.177 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:21.177 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694168+0000", 2024-09-21T21:41:21.177 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4222999999999997e-05 2024-09-21T21:41:21.177 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.177 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.177 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:21.177 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694249+0000", 2024-09-21T21:41:21.177 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0632999999999999e-05 2024-09-21T21:41:21.177 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:21.177 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:21.177 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:21.177 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.177 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.177 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1529 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 53248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:21.178 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634397+0000", 2024-09-21T21:41:21.178 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 19.501721668999998, 2024-09-21T21:41:21.178 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060189919000000001, 2024-09-21T21:41:21.178 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:21.178 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:21.178 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:21.178 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:21.178 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:21.178 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:21.178 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1529 2024-09-21T21:41:21.178 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.178 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:21.178 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.178 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:21.178 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634397+0000", 2024-09-21T21:41:21.179 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:21.179 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.179 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.179 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:21.179 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634397+0000", 2024-09-21T21:41:21.179 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:21.179 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.179 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.179 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:21.179 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634397+0000", 2024-09-21T21:41:21.179 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2e-07 2024-09-21T21:41:21.179 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.179 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.179 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:21.180 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634399+0000", 2024-09-21T21:41:21.180 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0609999999999998e-06 2024-09-21T21:41:21.180 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.180 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.180 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:21.180 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634406+0000", 2024-09-21T21:41:21.180 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7259999999999996e-06 2024-09-21T21:41:21.180 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.180 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.180 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:21.180 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634411+0000", 2024-09-21T21:41:21.180 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9339999999999997e-06 2024-09-21T21:41:21.180 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.180 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.181 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:21.181 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694480+0000", 2024-09-21T21:41:21.181 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060069105999999997 2024-09-21T21:41:21.181 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.181 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.181 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:21.181 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694527+0000", 2024-09-21T21:41:21.181 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7120000000000003e-05 2024-09-21T21:41:21.181 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.181 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.181 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:21.181 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694587+0000", 2024-09-21T21:41:21.181 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9552000000000002e-05 2024-09-21T21:41:21.181 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:21.181 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:21.181 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:21.182 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.182 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.182 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1531 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 110592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:21.182 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634417+0000", 2024-09-21T21:41:21.182 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 19.501701735000001, 2024-09-21T21:41:21.182 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060347287999999999, 2024-09-21T21:41:21.182 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:21.182 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:21.182 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:21.182 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:21.182 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:21.182 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:21.182 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1531 2024-09-21T21:41:21.183 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.183 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:21.183 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.183 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:21.183 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634417+0000", 2024-09-21T21:41:21.183 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:21.183 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.183 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.183 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:21.183 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634417+0000", 2024-09-21T21:41:21.183 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:21.183 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.183 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.183 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:21.183 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634417+0000", 2024-09-21T21:41:21.184 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1699999999999999e-07 2024-09-21T21:41:21.184 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.184 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.184 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:21.184 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634420+0000", 2024-09-21T21:41:21.184 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.243e-06 2024-09-21T21:41:21.184 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.184 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.184 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:21.184 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634426+0000", 2024-09-21T21:41:21.184 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2550000000000003e-06 2024-09-21T21:41:21.184 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.184 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.184 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:21.184 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634431+0000", 2024-09-21T21:41:21.184 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0150000000000003e-06 2024-09-21T21:41:21.185 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.185 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.185 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:21.185 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694618+0000", 2024-09-21T21:41:21.185 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060186836000000001 2024-09-21T21:41:21.185 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.185 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.185 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:21.185 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694685+0000", 2024-09-21T21:41:21.185 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6970000000000004e-05 2024-09-21T21:41:21.185 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.185 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.185 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:21.185 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694764+0000", 2024-09-21T21:41:21.185 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9552e-05 2024-09-21T21:41:21.185 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:21.186 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:21.186 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:21.186 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.186 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.186 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1532 3.e 3:724b31a3:::rbd_data.1121e540c096.0000000000000015:b [read 151552~36864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:21.186 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634437+0000", 2024-09-21T21:41:21.186 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 19.501681939000001, 2024-09-21T21:41:21.186 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060474314000000001, 2024-09-21T21:41:21.186 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:21.186 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:21.186 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:21.186 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:21.186 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:21.187 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:21.187 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1532 2024-09-21T21:41:21.187 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.187 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:21.187 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.187 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:21.187 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634437+0000", 2024-09-21T21:41:21.187 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:21.187 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.187 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.187 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:21.187 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634437+0000", 2024-09-21T21:41:21.187 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:21.187 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.187 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.187 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:21.188 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634437+0000", 2024-09-21T21:41:21.188 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9400000000000001e-07 2024-09-21T21:41:21.188 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.188 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.188 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:21.188 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634439+0000", 2024-09-21T21:41:21.188 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.114e-06 2024-09-21T21:41:21.188 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.188 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.188 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:21.188 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634446+0000", 2024-09-21T21:41:21.188 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1389999999999998e-06 2024-09-21T21:41:21.188 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.188 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.189 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:21.189 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634451+0000", 2024-09-21T21:41:21.189 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0010000000000001e-06 2024-09-21T21:41:21.189 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.189 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.189 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:21.189 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694794+0000", 2024-09-21T21:41:21.189 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060342509000000003 2024-09-21T21:41:21.189 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.189 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.189 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:21.189 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694835+0000", 2024-09-21T21:41:21.189 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0728e-05 2024-09-21T21:41:21.189 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.190 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.190 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:21.190 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694911+0000", 2024-09-21T21:41:21.190 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6428999999999996e-05 2024-09-21T21:41:21.190 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:21.190 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:21.190 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:21.190 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.190 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.190 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1534 3.e 3:724b31a3:::rbd_data.1121e540c096.0000000000000015:b [read 487424~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:21.190 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634477+0000", 2024-09-21T21:41:21.190 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 19.501641536000001, 2024-09-21T21:41:21.190 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060624009, 2024-09-21T21:41:21.190 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:21.191 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:21.191 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:21.191 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:21.191 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:21.191 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:21.191 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1534 2024-09-21T21:41:21.191 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.191 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:21.191 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.191 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:21.191 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634477+0000", 2024-09-21T21:41:21.191 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:21.191 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.191 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.191 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:21.192 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634477+0000", 2024-09-21T21:41:21.192 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:21.192 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.192 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.192 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:21.192 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634478+0000", 2024-09-21T21:41:21.192 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1399999999999997e-07 2024-09-21T21:41:21.192 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.192 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.192 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:21.192 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634480+0000", 2024-09-21T21:41:21.192 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1009999999999999e-06 2024-09-21T21:41:21.192 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.192 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.193 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:21.193 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634486+0000", 2024-09-21T21:41:21.193 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3289999999999999e-06 2024-09-21T21:41:21.193 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.193 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.193 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:21.193 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634491+0000", 2024-09-21T21:41:21.193 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6260000000000003e-06 2024-09-21T21:41:21.193 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.193 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.193 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:21.193 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694943+0000", 2024-09-21T21:41:21.193 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060451960999999999 2024-09-21T21:41:21.194 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.194 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.194 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:21.194 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695014+0000", 2024-09-21T21:41:21.194 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1587000000000004e-05 2024-09-21T21:41:21.194 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.194 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.194 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:21.194 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695101+0000", 2024-09-21T21:41:21.194 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6990999999999994e-05 2024-09-21T21:41:21.194 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:21.194 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:21.194 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:21.194 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.195 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.195 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1545 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 438272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:21.195 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634515+0000", 2024-09-21T21:41:21.195 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 19.501603335999999, 2024-09-21T21:41:21.195 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060707944, 2024-09-21T21:41:21.195 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:21.195 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:21.195 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:21.195 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:21.195 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:21.195 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:21.195 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1545 2024-09-21T21:41:21.195 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.195 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:21.196 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.196 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:21.196 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634515+0000", 2024-09-21T21:41:21.196 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:21.196 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.196 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.196 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:21.196 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634515+0000", 2024-09-21T21:41:21.196 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:21.196 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.196 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.196 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:21.196 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634516+0000", 2024-09-21T21:41:21.197 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0999999999999999e-07 2024-09-21T21:41:21.197 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.197 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.197 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:21.197 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634518+0000", 2024-09-21T21:41:21.197 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.846e-06 2024-09-21T21:41:21.197 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.197 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.197 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:21.197 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634524+0000", 2024-09-21T21:41:21.197 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2550000000000003e-06 2024-09-21T21:41:21.197 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.197 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.197 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:21.198 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634529+0000", 2024-09-21T21:41:21.198 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6940000000000003e-06 2024-09-21T21:41:21.198 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.198 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.198 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:21.198 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695127+0000", 2024-09-21T21:41:21.198 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060598921 2024-09-21T21:41:21.198 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.198 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.198 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:21.198 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695171+0000", 2024-09-21T21:41:21.198 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3588999999999999e-05 2024-09-21T21:41:21.198 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.199 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.199 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:21.199 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695223+0000", 2024-09-21T21:41:21.199 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2228999999999997e-05 2024-09-21T21:41:21.199 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:21.199 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:21.199 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:21.199 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.199 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.199 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1547 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 495616~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:21.199 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634534+0000", 2024-09-21T21:41:21.199 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 19.501584545, 2024-09-21T21:41:21.199 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060881131999999998, 2024-09-21T21:41:21.199 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:21.200 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:21.200 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:21.200 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:21.200 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:21.200 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:21.200 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1547 2024-09-21T21:41:21.200 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.200 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:21.200 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.200 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:21.200 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634534+0000", 2024-09-21T21:41:21.200 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:21.200 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.200 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.201 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:21.201 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634534+0000", 2024-09-21T21:41:21.201 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:21.201 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.201 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.201 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:21.201 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634535+0000", 2024-09-21T21:41:21.201 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.77e-07 2024-09-21T21:41:21.201 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.201 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.201 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:21.201 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634542+0000", 2024-09-21T21:41:21.201 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9000000000000006e-06 2024-09-21T21:41:21.201 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.201 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.202 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:21.202 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634549+0000", 2024-09-21T21:41:21.202 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5910000000000001e-06 2024-09-21T21:41:21.202 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.202 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.202 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:21.202 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634554+0000", 2024-09-21T21:41:21.202 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7940000000000002e-06 2024-09-21T21:41:21.202 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.202 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.202 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:21.202 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695256+0000", 2024-09-21T21:41:21.202 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.06070246 2024-09-21T21:41:21.203 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.203 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.203 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:21.203 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695332+0000", 2024-09-21T21:41:21.203 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6248000000000006e-05 2024-09-21T21:41:21.203 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.203 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.203 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:21.203 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695415+0000", 2024-09-21T21:41:21.203 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2762000000000005e-05 2024-09-21T21:41:21.203 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:21.203 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:21.203 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:21.203 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.204 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.204 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1548 3.16 3:6a41a830:::rbd_data.112145fae779.000000000000000a:a [read 299008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:21.204 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634561+0000", 2024-09-21T21:41:21.204 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 19.501557968, 2024-09-21T21:41:21.204 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060786743999999997, 2024-09-21T21:41:21.204 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:21.204 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:21.204 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:21.204 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:21.204 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:21.204 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:21.204 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1548 2024-09-21T21:41:21.204 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.204 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:21.204 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.205 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:21.205 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634561+0000", 2024-09-21T21:41:21.205 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:21.205 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.205 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.205 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:21.205 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634561+0000", 2024-09-21T21:41:21.205 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:21.205 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.205 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.205 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:21.205 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634561+0000", 2024-09-21T21:41:21.205 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5199999999999997e-07 2024-09-21T21:41:21.205 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.206 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.206 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:21.206 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634564+0000", 2024-09-21T21:41:21.206 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4040000000000002e-06 2024-09-21T21:41:21.206 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.206 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.206 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:21.206 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634571+0000", 2024-09-21T21:41:21.206 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4379999999999998e-06 2024-09-21T21:41:21.206 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.206 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.206 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:21.206 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634576+0000", 2024-09-21T21:41:21.206 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1399999999999999e-06 2024-09-21T21:41:21.206 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.207 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.207 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:21.207 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695247+0000", 2024-09-21T21:41:21.207 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060670397000000001 2024-09-21T21:41:21.207 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.207 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.207 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:21.207 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695299+0000", 2024-09-21T21:41:21.207 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2564e-05 2024-09-21T21:41:21.207 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.207 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.207 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:21.207 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695347+0000", 2024-09-21T21:41:21.207 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8248999999999998e-05 2024-09-21T21:41:21.207 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:21.208 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:21.208 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:21.208 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.208 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.208 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1549 3.16 3:6a41a830:::rbd_data.112145fae779.000000000000000a:a [read 356352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:21.208 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634583+0000", 2024-09-21T21:41:21.208 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 19.501536054999999, 2024-09-21T21:41:21.208 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060889459, 2024-09-21T21:41:21.208 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:21.208 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:21.208 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:21.208 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:21.208 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:21.208 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:21.209 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1549 2024-09-21T21:41:21.209 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.209 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:21.209 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.209 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:21.209 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634583+0000", 2024-09-21T21:41:21.209 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:21.209 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.209 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.209 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:21.209 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634583+0000", 2024-09-21T21:41:21.209 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:21.209 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.209 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.210 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:21.210 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634583+0000", 2024-09-21T21:41:21.210 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1899999999999998e-07 2024-09-21T21:41:21.210 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.210 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.210 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:21.210 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634585+0000", 2024-09-21T21:41:21.210 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2129999999999998e-06 2024-09-21T21:41:21.210 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.210 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.210 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:21.210 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634592+0000", 2024-09-21T21:41:21.210 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2840000000000003e-06 2024-09-21T21:41:21.210 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.210 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.211 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:21.211 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634596+0000", 2024-09-21T21:41:21.211 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9309999999999999e-06 2024-09-21T21:41:21.211 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.211 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.211 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:21.211 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695371+0000", 2024-09-21T21:41:21.211 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060774425999999999 2024-09-21T21:41:21.211 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.211 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.211 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:21.211 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695419+0000", 2024-09-21T21:41:21.211 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8607999999999999e-05 2024-09-21T21:41:21.212 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.212 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.212 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:21.212 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695472+0000", 2024-09-21T21:41:21.212 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2577999999999997e-05 2024-09-21T21:41:21.212 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:21.212 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:21.212 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:21.212 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.212 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.212 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1588 3.16 3:695c582e:::rbd_data.1121e540c096.0000000000000008:b [read 147456~40960] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:21.212 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.636942+0000", 2024-09-21T21:41:21.212 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 19.499176278, 2024-09-21T21:41:21.212 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.058643811999999997, 2024-09-21T21:41:21.213 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:21.213 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:21.213 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:21.213 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:21.213 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:21.213 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:21.213 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1588 2024-09-21T21:41:21.213 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.213 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:21.213 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.213 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:21.213 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636942+0000", 2024-09-21T21:41:21.213 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:21.213 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.213 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.214 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:21.214 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636942+0000", 2024-09-21T21:41:21.214 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:21.214 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.214 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.214 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:21.214 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636943+0000", 2024-09-21T21:41:21.214 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4199999999999996e-07 2024-09-21T21:41:21.214 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.214 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.214 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:21.214 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636945+0000", 2024-09-21T21:41:21.214 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.401e-06 2024-09-21T21:41:21.215 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.215 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.215 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:21.215 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636952+0000", 2024-09-21T21:41:21.215 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5379999999999999e-06 2024-09-21T21:41:21.215 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.215 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.215 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:21.215 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636958+0000", 2024-09-21T21:41:21.215 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.489e-06 2024-09-21T21:41:21.215 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.215 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.215 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:21.215 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695497+0000", 2024-09-21T21:41:21.215 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.058538504999999998 2024-09-21T21:41:21.216 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.216 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.216 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:21.216 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695533+0000", 2024-09-21T21:41:21.216 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6118000000000002e-05 2024-09-21T21:41:21.216 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.216 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.216 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:21.216 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695586+0000", 2024-09-21T21:41:21.216 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3219e-05 2024-09-21T21:41:21.216 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:21.216 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:21.216 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:21.216 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.216 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.217 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1589 3.16 3:695c582e:::rbd_data.1121e540c096.0000000000000008:b [read 487424~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:21.217 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.636965+0000", 2024-09-21T21:41:21.217 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 19.499153187000001, 2024-09-21T21:41:21.217 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.058728152999999998, 2024-09-21T21:41:21.217 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:21.217 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:21.217 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:21.217 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:21.217 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:21.217 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:21.217 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1589 2024-09-21T21:41:21.217 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.217 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:21.217 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.217 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:21.218 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636965+0000", 2024-09-21T21:41:21.218 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:21.218 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.218 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.218 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:21.218 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636965+0000", 2024-09-21T21:41:21.218 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:21.218 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.218 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.218 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:21.218 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636966+0000", 2024-09-21T21:41:21.218 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3799999999999998e-07 2024-09-21T21:41:21.218 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.218 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.219 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:21.219 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636968+0000", 2024-09-21T21:41:21.219 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2400000000000002e-06 2024-09-21T21:41:21.219 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.219 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.219 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:21.219 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636976+0000", 2024-09-21T21:41:21.219 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5050000000000001e-06 2024-09-21T21:41:21.219 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.219 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.219 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:21.219 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636981+0000", 2024-09-21T21:41:21.219 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2170000000000002e-06 2024-09-21T21:41:21.219 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.220 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.220 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:21.220 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695610+0000", 2024-09-21T21:41:21.220 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.058628998000000002 2024-09-21T21:41:21.220 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.220 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.220 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:21.220 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695647+0000", 2024-09-21T21:41:21.220 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6752000000000003e-05 2024-09-21T21:41:21.220 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.220 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.220 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:21.220 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695694+0000", 2024-09-21T21:41:21.220 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7002999999999997e-05 2024-09-21T21:41:21.221 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:21.221 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:21.221 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:21.221 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.221 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.221 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1607 3.e 3:724b31a3:::rbd_data.1121e540c096.0000000000000015:b [read 299008~36864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:21.221 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.637062+0000", 2024-09-21T21:41:21.221 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 19.499057096000001, 2024-09-21T21:41:21.221 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.058623078000000002, 2024-09-21T21:41:21.221 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:21.221 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:21.221 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:21.221 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:21.221 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:21.221 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:21.222 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1607 2024-09-21T21:41:21.222 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.222 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:21.222 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.222 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:21.222 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.637062+0000", 2024-09-21T21:41:21.222 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:21.222 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.222 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.222 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:21.222 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.637062+0000", 2024-09-21T21:41:21.222 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:21.222 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.223 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.223 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:21.223 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.637062+0000", 2024-09-21T21:41:21.223 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0900000000000002e-07 2024-09-21T21:41:21.223 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.223 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.223 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:21.223 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.637069+0000", 2024-09-21T21:41:21.223 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.267e-06 2024-09-21T21:41:21.223 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.223 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.223 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:21.223 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.637076+0000", 2024-09-21T21:41:21.223 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7320000000000001e-06 2024-09-21T21:41:21.223 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.224 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.224 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:21.224 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.637082+0000", 2024-09-21T21:41:21.224 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4760000000000003e-06 2024-09-21T21:41:21.224 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.224 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.224 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:21.224 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695604+0000", 2024-09-21T21:41:21.224 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.058521261999999998 2024-09-21T21:41:21.224 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.224 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.224 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:21.224 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695638+0000", 2024-09-21T21:41:21.224 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4292999999999998e-05 2024-09-21T21:41:21.225 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:21.225 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:21.225 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:21.225 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695685+0000", 2024-09-21T21:41:21.225 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6638999999999998e-05 2024-09-21T21:41:21.225 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:21.225 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:21.225 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:21.225 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:21.225 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:21.225 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:21.226 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2: {} 2024-09-21T21:41:21.227 INFO:teuthology.orchestra.run.smithi081.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T21:41:21.228 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3: {} 2024-09-21T21:41:21.228 INFO:teuthology.orchestra.run.smithi081.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T21:41:21.229 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T21:41:21.229 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4: {} 2024-09-21T21:41:21.230 INFO:teuthology.orchestra.run.smithi081.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T21:41:21.230 INFO:teuthology.orchestra.run.smithi164.stdout:2697 read 0x934b400 thru 0x934c9b0 (0x15b1 bytes) 2024-09-21T21:41:21.230 INFO:teuthology.orchestra.run.smithi164.stdout:2698 write 0x3215a00 thru 0x3218823 (0x2e24 bytes) 2024-09-21T21:41:21.230 INFO:teuthology.orchestra.run.smithi164.stdout:2699 trunc from 0x9587800 to 0x3e8d600 2024-09-21T21:41:21.230 INFO:teuthology.orchestra.run.smithi164.stdout:2700 write 0x115bc00 thru 0x1164b17 (0x8f18 bytes) 2024-09-21T21:41:21.230 INFO:teuthology.orchestra.run.smithi164.stdout:2701 read 0x875200 thru 0x87ee6b (0x9c6c bytes) 2024-09-21T21:41:21.230 INFO:teuthology.orchestra.run.smithi164.stdout:2702 punch from 0x2706c00 to 0x270b600, (0x4a00 bytes) 2024-09-21T21:41:21.230 INFO:teuthology.orchestra.run.smithi164.stdout:2705 write 0xc44a200 thru 0xc456c80 (0xca81 bytes) 2024-09-21T21:41:21.231 INFO:teuthology.orchestra.run.smithi164.stdout:2706 read 0x7731400 thru 0x773c2ad (0xaeae bytes) 2024-09-21T21:41:21.231 INFO:teuthology.orchestra.run.smithi164.stdout:2707 read 0x8587a00 thru 0x8597104 (0xf705 bytes) 2024-09-21T21:41:21.231 INFO:teuthology.orchestra.run.smithi164.stdout:2708 read 0x7af6a00 thru 0x7afe945 (0x7f46 bytes) 2024-09-21T21:41:21.231 INFO:teuthology.orchestra.run.smithi164.stdout:2709 trunc from 0xc456e00 to 0xe2a7800 2024-09-21T21:41:21.231 INFO:teuthology.orchestra.run.smithi164.stdout:2710 write 0x8bbd000 thru 0x8bc9372 (0xc373 bytes) 2024-09-21T21:41:21.231 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5: {} 2024-09-21T21:41:21.232 INFO:teuthology.orchestra.run.smithi081.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T21:41:21.232 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:21.225+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:21.235 INFO:teuthology.orchestra.run.smithi164.stdout:2713 trunc from 0xe2a7800 to 0x28b0e00 2024-09-21T21:41:21.261 INFO:teuthology.orchestra.run.smithi164.stdout:2714 read 0x2810a00 thru 0x281b932 (0xaf33 bytes) 2024-09-21T21:41:21.263 INFO:teuthology.orchestra.run.smithi164.stdout:2718 trunc from 0x28b0e00 to 0x294f400 2024-09-21T21:41:21.271 INFO:teuthology.orchestra.run.smithi164.stdout:2719 read 0x1bc9a00 thru 0x1bd77d1 (0xddd2 bytes) 2024-09-21T21:41:21.272 INFO:teuthology.orchestra.run.smithi164.stdout:2720 punch from 0x2340e00 to 0x2345600, (0x4800 bytes) 2024-09-21T21:41:21.272 INFO:teuthology.orchestra.run.smithi164.stdout:2724 read 0x2662e00 thru 0x266c80b (0x9a0c bytes) 2024-09-21T21:41:21.273 INFO:teuthology.orchestra.run.smithi164.stdout:2725 read 0x168ba00 thru 0x1691e5c (0x645d bytes) 2024-09-21T21:41:21.287 INFO:teuthology.orchestra.run.smithi164.stdout:2727 write 0x69ef400 thru 0x69fb22e (0xbe2f bytes) 2024-09-21T21:41:21.288 INFO:teuthology.orchestra.run.smithi164.stdout:2729 read 0x61d5400 thru 0x61db6af (0x62b0 bytes) 2024-09-21T21:41:21.288 INFO:teuthology.orchestra.run.smithi164.stdout:2730 read 0x692b000 thru 0x692f530 (0x4531 bytes) 2024-09-21T21:41:21.288 INFO:teuthology.orchestra.run.smithi164.stdout:2732 write 0x50efc00 thru 0x50ffb3a (0xff3b bytes) 2024-09-21T21:41:21.300 INFO:teuthology.orchestra.run.smithi164.stdout:2733 write 0x72f8400 thru 0x72fa50a (0x210b bytes) 2024-09-21T21:41:21.301 INFO:teuthology.orchestra.run.smithi164.stdout:2736 read 0x34db000 thru 0x34e138e (0x638f bytes) 2024-09-21T21:41:21.301 INFO:teuthology.orchestra.run.smithi164.stdout:2738 write 0x39dd800 thru 0x39e8d68 (0xb569 bytes) 2024-09-21T21:41:21.302 INFO:teuthology.orchestra.run.smithi164.stdout:2739 write 0x3bbe800 thru 0x3bc8bf0 (0xa3f1 bytes) 2024-09-21T21:41:21.316 INFO:teuthology.orchestra.run.smithi164.stdout:2741 write 0x958c600 thru 0x95996d8 (0xd0d9 bytes) 2024-09-21T21:41:21.318 INFO:teuthology.orchestra.run.smithi164.stdout:2742 read 0x63cd800 thru 0x63d05d3 (0x2dd4 bytes) 2024-09-21T21:41:21.327 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:21.326+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:21.327 INFO:teuthology.orchestra.run.smithi164.stdout:2743 trunc from 0x9599800 to 0x1d7a800 2024-09-21T21:41:21.348 INFO:teuthology.orchestra.run.smithi164.stdout:2744 clone 17 order 22 su 2097152 sc 7 2024-09-21T21:41:21.383 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:21.383 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:21.392 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T21:41:21.432 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:21.431+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:21.529 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:21.527+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:21.534 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:21.534 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:21.543 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T21:41:21.630 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:21.629+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:21.634 INFO:teuthology.orchestra.run.smithi081.stderr:nodeep-scrub is unset 2024-09-21T21:41:21.692 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:21.692 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:21.701 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T21:41:21.717 INFO:teuthology.orchestra.run.smithi164.stdout:truncating image image_client.0-clone16 from 0x1d7a800 (overlap 0x10aec00) to 0x0 2024-09-21T21:41:21.744 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:21.743+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:21.806 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:21.806 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:21.815 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T21:41:21.857 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:21.855+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:21.933 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:21.931+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:21.965 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:21.965 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:21.975 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T21:41:22.033 INFO:teuthology.orchestra.run.smithi164.stdout:checking clone #15, image image_client.0-clone15 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent16 2024-09-21T21:41:22.058 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:22.057+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:22.101 INFO:teuthology.orchestra.run.smithi164.stdout:2745 trunc from 0x1d7a800 to 0xbcab400 2024-09-21T21:41:22.113 INFO:teuthology.orchestra.run.smithi164.stdout:2746 write 0x2fbd200 thru 0x2fc12f0 (0x40f1 bytes) 2024-09-21T21:41:22.117 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:22.117 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:22.122 INFO:teuthology.orchestra.run.smithi164.stdout:2748 trunc from 0xbcab400 to 0xa194e00 2024-09-21T21:41:22.127 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T21:41:22.133 INFO:teuthology.orchestra.run.smithi164.stdout:2749 trunc from 0xa194e00 to 0xa18b000 2024-09-21T21:41:22.134 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:22.132+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:22.148 INFO:teuthology.orchestra.run.smithi164.stdout:2750 read 0x5e55000 thru 0x5e601a0 (0xb1a1 bytes) 2024-09-21T21:41:22.155 INFO:teuthology.orchestra.run.smithi164.stdout:2751 read 0x316c400 thru 0x3178ae4 (0xc6e5 bytes) 2024-09-21T21:41:22.155 INFO:teuthology.orchestra.run.smithi164.stdout:2752 write 0x6ab9800 thru 0x6abac78 (0x1479 bytes) 2024-09-21T21:41:22.155 INFO:teuthology.orchestra.run.smithi164.stdout:2753 read 0x99f4a00 thru 0x99f6f1e (0x251f bytes) 2024-09-21T21:41:22.170 INFO:teuthology.orchestra.run.smithi164.stdout:2757 write 0xced9800 thru 0xcee3118 (0x9919 bytes) 2024-09-21T21:41:22.171 INFO:teuthology.orchestra.run.smithi164.stdout:2760 read 0x1810e00 thru 0x1818915 (0x7b16 bytes) 2024-09-21T21:41:22.171 INFO:teuthology.orchestra.run.smithi164.stdout:2761 write 0x3fd7e00 thru 0x3fd83e1 (0x5e2 bytes) 2024-09-21T21:41:22.183 INFO:teuthology.orchestra.run.smithi164.stdout:2763 trunc from 0xcee3200 to 0x22e7600 2024-09-21T21:41:22.204 INFO:teuthology.orchestra.run.smithi164.stdout:2764 punch from 0x8bc600 to 0x8c8200, (0xbc00 bytes) 2024-09-21T21:41:22.209 INFO:teuthology.orchestra.run.smithi164.stdout:2765 read 0x1dfd200 thru 0x1e04991 (0x7792 bytes) 2024-09-21T21:41:22.219 INFO:teuthology.orchestra.run.smithi164.stdout:2766 trunc from 0x22e7600 to 0x9fa5800 2024-09-21T21:41:22.227 INFO:teuthology.orchestra.run.smithi164.stdout:2767 clone 18 order 24 su 65536 sc 8 2024-09-21T21:41:22.234 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:22.233+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:22.273 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:22.273 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:22.281 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T21:41:22.335 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:22.334+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:22.383 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:22.384 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:22.392 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T21:41:22.476 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:22.475+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:22.535 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:22.535 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:22.538 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:22.537+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:22.544 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T21:41:22.639 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:22.637+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:22.652 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:22.652 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:22.661 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T21:41:22.722 INFO:teuthology.orchestra.run.smithi164.stdout:truncating image image_client.0-clone17 from 0x9fa5800 (overlap 0x1d7a800) to 0x11afe00 2024-09-21T21:41:22.745 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:22.744+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:22.813 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:22.813 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:22.823 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T21:41:22.840 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:22.839+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:22.941 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:22.939+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:22.969 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:22.969 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:22.978 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T21:41:23.041 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:23.040+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:23.058 INFO:teuthology.orchestra.run.smithi164.stdout:checking clone #16, image image_client.0-clone16 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent17 2024-09-21T21:41:23.082 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:23.082 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:23.090 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T21:41:23.131 INFO:teuthology.orchestra.run.smithi164.stdout:2768 write 0xc19b800 thru 0xc1ab16c (0xf96d bytes) 2024-09-21T21:41:23.136 INFO:teuthology.orchestra.run.smithi164.stdout:2773 write 0x2e9d200 thru 0x2ea7258 (0xa059 bytes) 2024-09-21T21:41:23.138 INFO:teuthology.orchestra.run.smithi164.stdout:2774 trunc from 0xc1ab200 to 0x724ce00 2024-09-21T21:41:23.142 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:23.141+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:23.199 INFO:teuthology.orchestra.run.smithi164.stdout:2778 punch from 0x3d3d000 to 0x3d42c00, (0x5c00 bytes) 2024-09-21T21:41:23.200 INFO:teuthology.orchestra.run.smithi164.stdout:2780 read 0x5777c00 thru 0x5786846 (0xec47 bytes) 2024-09-21T21:41:23.201 INFO:teuthology.orchestra.run.smithi164.stdout:2783 punch from 0x4ff2e00 to 0x4ff9600, (0x6800 bytes) 2024-09-21T21:41:23.201 INFO:teuthology.orchestra.run.smithi164.stdout:2784 write 0x6e37200 thru 0x6e3a505 (0x3306 bytes) 2024-09-21T21:41:23.209 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:23.210 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:23.219 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T21:41:23.220 INFO:teuthology.orchestra.run.smithi164.stdout:2785 write 0xee09000 thru 0xee161f3 (0xd1f4 bytes) 2024-09-21T21:41:23.221 INFO:teuthology.orchestra.run.smithi164.stdout:2786 read 0xb5e9000 thru 0xb5f0da7 (0x7da8 bytes) 2024-09-21T21:41:23.233 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T21:41:23.236 INFO:teuthology.orchestra.run.smithi164.stdout:2787 trunc from 0xee16200 to 0x2107400 2024-09-21T21:41:23.262 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:23.261+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:23.283 INFO:teuthology.orchestra.run.smithi164.stdout:2791 write 0xe6fc000 thru 0xe6fd99a (0x199b bytes) 2024-09-21T21:41:23.283 INFO:teuthology.orchestra.run.smithi164.stdout:2793 read 0xb70d800 thru 0xb7196f8 (0xbef9 bytes) 2024-09-21T21:41:23.283 INFO:teuthology.orchestra.run.smithi164.stdout:2794 read 0x5c6a200 thru 0x5c72208 (0x8009 bytes) 2024-09-21T21:41:23.284 INFO:teuthology.orchestra.run.smithi164.stdout:2795 punch from 0x2ad6000 to 0x2ad9e00, (0x3e00 bytes) 2024-09-21T21:41:23.293 INFO:teuthology.orchestra.run.smithi164.stdout:2797 trunc from 0xe6fda00 to 0x65a1e00 2024-09-21T21:41:23.326 INFO:teuthology.orchestra.run.smithi164.stdout:2798 write 0xb5ede00 thru 0xb5f545b (0x765c bytes) 2024-09-21T21:41:23.326 INFO:teuthology.orchestra.run.smithi164.stdout:2801 write 0x6b44200 thru 0x6b4c3f3 (0x81f4 bytes) 2024-09-21T21:41:23.327 INFO:teuthology.orchestra.run.smithi164.stdout:2804 read 0x6606c00 thru 0x660eac3 (0x7ec4 bytes) 2024-09-21T21:41:23.327 INFO:teuthology.orchestra.run.smithi164.stdout:2806 flatten 2024-09-21T21:41:23.330 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:23.330 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:23.340 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T21:41:23.342 INFO:teuthology.orchestra.run.smithi164.stdout:2808 read 0xb31a800 thru 0xb32055f (0x5d60 bytes) 2024-09-21T21:41:23.348 INFO:teuthology.orchestra.run.smithi164.stdout:2809 trunc from 0xb5f5600 to 0xe6ff800 2024-09-21T21:41:23.357 INFO:teuthology.orchestra.run.smithi164.stdout:2810 read 0x5896400 thru 0x589ecff (0x8900 bytes) 2024-09-21T21:41:23.358 INFO:teuthology.orchestra.run.smithi164.stdout:2811 write 0x80cf400 thru 0x80db3ba (0xbfbb bytes) 2024-09-21T21:41:23.358 INFO:teuthology.orchestra.run.smithi164.stdout:2813 punch from 0x48c5200 to 0x48d4a00, (0xf800 bytes) 2024-09-21T21:41:23.358 INFO:teuthology.orchestra.run.smithi164.stdout:2814 write 0x87d6c00 thru 0x87d78fd (0xcfe bytes) 2024-09-21T21:41:23.368 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0: {} 2024-09-21T21:41:23.368 INFO:teuthology.orchestra.run.smithi081.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T21:41:23.371 INFO:teuthology.orchestra.run.smithi164.stdout:2815 trunc from 0xe6ff800 to 0x303b600 2024-09-21T21:41:23.382 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:23.381+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:23.387 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1: {} 2024-09-21T21:41:23.387 INFO:teuthology.orchestra.run.smithi081.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T21:41:23.406 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2: {} 2024-09-21T21:41:23.407 INFO:teuthology.orchestra.run.smithi081.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T21:41:23.409 INFO:teuthology.orchestra.run.smithi164.stdout:2817 write 0xbbc5000 thru 0xbbc8d19 (0x3d1a bytes) 2024-09-21T21:41:23.410 INFO:teuthology.orchestra.run.smithi164.stdout:2819 read 0xb77000 thru 0xb7c408 (0x5409 bytes) 2024-09-21T21:41:23.416 INFO:teuthology.orchestra.run.smithi164.stdout:2820 trunc from 0xbbc8e00 to 0x66fd400 2024-09-21T21:41:23.425 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3: {} 2024-09-21T21:41:23.425 INFO:teuthology.orchestra.run.smithi081.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T21:41:23.438 INFO:teuthology.orchestra.run.smithi164.stdout:2823 punch from 0xf31c00 to 0xf3ce00, (0xb200 bytes) 2024-09-21T21:41:23.445 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:23.444+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:23.446 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4: {} 2024-09-21T21:41:23.446 INFO:teuthology.orchestra.run.smithi081.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T21:41:23.446 INFO:teuthology.orchestra.run.smithi164.stdout:2825 read 0x2122600 thru 0x212a717 (0x8118 bytes) 2024-09-21T21:41:23.446 INFO:teuthology.orchestra.run.smithi164.stdout:2826 read 0x569d400 thru 0x56aa9e2 (0xd5e3 bytes) 2024-09-21T21:41:23.447 INFO:teuthology.orchestra.run.smithi164.stdout:2827 read 0x336ba00 thru 0x3374dc0 (0x93c1 bytes) 2024-09-21T21:41:23.447 INFO:teuthology.orchestra.run.smithi164.stdout:2828 punch from 0x1b4a400 to 0x1b52c00, (0x8800 bytes) 2024-09-21T21:41:23.447 INFO:teuthology.orchestra.run.smithi164.stdout:2831 punch from 0x53d2c00 to 0x53dec00, (0xc000 bytes) 2024-09-21T21:41:23.447 INFO:teuthology.orchestra.run.smithi164.stdout:2832 read 0x263bc00 thru 0x26474a7 (0xb8a8 bytes) 2024-09-21T21:41:23.449 INFO:teuthology.orchestra.run.smithi164.stdout:2834 punch from 0x1fc9200 to 0x1fc9400, (0x200 bytes) 2024-09-21T21:41:23.449 INFO:teuthology.orchestra.run.smithi164.stdout:2835 punch from 0x11d5200 to 0x11e4a00, (0xf800 bytes) 2024-09-21T21:41:23.449 INFO:teuthology.orchestra.run.smithi164.stdout:2837 write 0x5172600 thru 0x51773a5 (0x4da6 bytes) 2024-09-21T21:41:23.450 INFO:teuthology.orchestra.run.smithi164.stdout:2839 write 0x2436000 thru 0x2445a2e (0xfa2f bytes) 2024-09-21T21:41:23.451 INFO:teuthology.orchestra.run.smithi164.stdout:2841 read 0x5d51000 thru 0x5d51bba (0xbbb bytes) 2024-09-21T21:41:23.452 INFO:teuthology.orchestra.run.smithi164.stdout:2843 read 0x1f98c00 thru 0x1f998e4 (0xce5 bytes) 2024-09-21T21:41:23.452 INFO:teuthology.orchestra.run.smithi164.stdout:2844 punch from 0x3cd8a00 to 0x3ce4400, (0xba00 bytes) 2024-09-21T21:41:23.452 INFO:teuthology.orchestra.run.smithi164.stdout:2846 write 0x4721a00 thru 0x4730f10 (0xf511 bytes) 2024-09-21T21:41:23.454 INFO:teuthology.orchestra.run.smithi164.stdout:2847 read 0x2c1ec00 thru 0x2c2d59f (0xe9a0 bytes) 2024-09-21T21:41:23.465 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5: {} 2024-09-21T21:41:23.465 INFO:teuthology.orchestra.run.smithi081.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T21:41:23.474 INFO:teuthology.orchestra.run.smithi164.stdout:2848 write 0xdf7a200 thru 0xdf81fb3 (0x7db4 bytes) 2024-09-21T21:41:23.475 INFO:teuthology.orchestra.run.smithi164.stdout:2849 punch from 0x511b800 to 0x511c600, (0xe00 bytes) 2024-09-21T21:41:23.475 INFO:teuthology.orchestra.run.smithi164.stdout:2851 read 0xc9a0a00 thru 0xc9a29b3 (0x1fb4 bytes) 2024-09-21T21:41:23.475 INFO:teuthology.orchestra.run.smithi164.stdout:2852 clone 19 order 24 su 131072 sc 14 2024-09-21T21:41:23.494 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:23.495 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:41:23.495 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-21T21:41:23.495 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:23.503 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T21:41:23.546 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:23.545+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:23.633 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-21T21:41:23.634 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T21:41:23.634 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-21T21:41:23.634 INFO:tasks.thrashosds.thrasher:Setting osd 2 primary_affinity to 0.000000 2024-09-21T21:41:23.634 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 0 2024-09-21T21:41:23.647 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:23.647 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:41:23.647 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:41:23.647 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:41:23.647 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:23.647 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:23.646+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:23.650 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T21:41:23.655 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T21:41:23.748 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:23.746+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:23.800 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:23.801 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-21T21:41:23.801 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-21T21:41:23.801 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:41:23.801 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.801 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1484 3.16 3:6a41a830:::rbd_data.112145fae779.000000000000000a:a [read 303104~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:23.801 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634086+0000", 2024-09-21T21:41:23.801 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 22.164460278, 2024-09-21T21:41:23.801 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.058257055000000002, 2024-09-21T21:41:23.801 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:23.801 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:23.801 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:23.801 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:23.801 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:23.802 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:23.802 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1484 2024-09-21T21:41:23.802 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.802 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:23.802 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.802 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:23.802 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634086+0000", 2024-09-21T21:41:23.802 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:23.802 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.802 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.802 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:23.802 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634086+0000", 2024-09-21T21:41:23.802 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:23.802 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.803 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.803 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:23.803 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634086+0000", 2024-09-21T21:41:23.803 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6100000000000001e-07 2024-09-21T21:41:23.803 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.803 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.803 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:23.803 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634088+0000", 2024-09-21T21:41:23.803 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1100000000000001e-06 2024-09-21T21:41:23.803 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.803 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.803 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:23.803 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634095+0000", 2024-09-21T21:41:23.803 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4080000000000003e-06 2024-09-21T21:41:23.803 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.804 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.804 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:23.804 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634100+0000", 2024-09-21T21:41:23.804 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0730000000000004e-06 2024-09-21T21:41:23.804 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.804 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.804 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:23.804 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.692197+0000", 2024-09-21T21:41:23.804 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.058097331000000002 2024-09-21T21:41:23.804 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.804 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.804 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:23.804 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.692271+0000", 2024-09-21T21:41:23.804 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3423999999999994e-05 2024-09-21T21:41:23.804 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.805 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.805 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:23.805 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.692343+0000", 2024-09-21T21:41:23.805 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2248000000000004e-05 2024-09-21T21:41:23.805 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:23.805 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:23.805 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:23.805 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.805 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.805 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1485 3.16 3:695c582e:::rbd_data.1121e540c096.0000000000000008:b [read 335872~36864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:23.805 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634106+0000", 2024-09-21T21:41:23.805 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 22.164440481, 2024-09-21T21:41:23.805 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059223685999999998, 2024-09-21T21:41:23.805 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:23.805 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:23.806 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:23.806 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:23.806 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:23.806 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:23.806 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1485 2024-09-21T21:41:23.806 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.806 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:23.806 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.806 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:23.806 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634106+0000", 2024-09-21T21:41:23.806 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:23.806 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.806 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.806 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:23.807 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634106+0000", 2024-09-21T21:41:23.807 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:23.807 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.807 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.807 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:23.807 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634106+0000", 2024-09-21T21:41:23.807 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3300000000000002e-07 2024-09-21T21:41:23.807 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.807 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.807 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:23.807 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634108+0000", 2024-09-21T21:41:23.807 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.996e-06 2024-09-21T21:41:23.807 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.807 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.808 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:23.808 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634115+0000", 2024-09-21T21:41:23.808 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5139999999999998e-06 2024-09-21T21:41:23.808 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.808 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.808 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:23.808 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634120+0000", 2024-09-21T21:41:23.808 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1050000000000003e-06 2024-09-21T21:41:23.808 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.808 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.808 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:23.808 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.692374+0000", 2024-09-21T21:41:23.808 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.058254729999999998 2024-09-21T21:41:23.808 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.808 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.809 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:23.809 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.692440+0000", 2024-09-21T21:41:23.809 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6076999999999998e-05 2024-09-21T21:41:23.809 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.809 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.809 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:23.809 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693329+0000", 2024-09-21T21:41:23.809 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00088873100000000003 2024-09-21T21:41:23.809 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:23.809 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:23.809 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:23.809 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.809 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.809 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1486 3.16 3:6a41a830:::rbd_data.112145fae779.000000000000000a:a [read 360448~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:23.810 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634126+0000", 2024-09-21T21:41:23.810 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 22.164420189000001, 2024-09-21T21:41:23.810 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059399528, 2024-09-21T21:41:23.810 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:23.810 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:23.810 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:23.810 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:23.810 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:23.810 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:23.810 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1486 2024-09-21T21:41:23.810 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.810 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:23.810 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.810 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:23.810 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634126+0000", 2024-09-21T21:41:23.811 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:23.811 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.811 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.811 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:23.811 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634126+0000", 2024-09-21T21:41:23.811 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:23.811 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.811 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.811 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:23.811 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634126+0000", 2024-09-21T21:41:23.811 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4099999999999999e-07 2024-09-21T21:41:23.811 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.811 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.811 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:23.811 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634128+0000", 2024-09-21T21:41:23.812 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1069999999999999e-06 2024-09-21T21:41:23.812 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.812 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.812 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:23.812 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634135+0000", 2024-09-21T21:41:23.812 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.212e-06 2024-09-21T21:41:23.812 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.812 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.812 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:23.812 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634140+0000", 2024-09-21T21:41:23.812 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1429999999999997e-06 2024-09-21T21:41:23.812 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.812 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.812 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:23.812 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693363+0000", 2024-09-21T21:41:23.813 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059223614000000001 2024-09-21T21:41:23.813 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.813 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.813 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:23.813 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693431+0000", 2024-09-21T21:41:23.813 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7294999999999999e-05 2024-09-21T21:41:23.813 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.813 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.813 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:23.813 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693525+0000", 2024-09-21T21:41:23.813 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.4716000000000006e-05 2024-09-21T21:41:23.813 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:23.813 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:23.813 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:23.814 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.814 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.814 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1499 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 49152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:23.814 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634186+0000", 2024-09-21T21:41:23.814 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 22.164360311999999, 2024-09-21T21:41:23.814 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059339256999999999, 2024-09-21T21:41:23.814 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:23.814 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:23.814 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:23.814 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:23.814 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:23.814 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:23.814 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1499 2024-09-21T21:41:23.814 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.815 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:23.815 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.815 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:23.815 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634186+0000", 2024-09-21T21:41:23.815 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:23.815 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.815 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.815 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:23.815 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634186+0000", 2024-09-21T21:41:23.815 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:23.815 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.815 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.815 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:23.815 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634186+0000", 2024-09-21T21:41:23.815 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0999999999999999e-07 2024-09-21T21:41:23.816 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.816 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.816 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:23.816 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634188+0000", 2024-09-21T21:41:23.816 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.86e-06 2024-09-21T21:41:23.816 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.816 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.816 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:23.816 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634194+0000", 2024-09-21T21:41:23.816 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1109999999999996e-06 2024-09-21T21:41:23.816 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.816 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.816 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:23.816 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634199+0000", 2024-09-21T21:41:23.816 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7419999999999997e-06 2024-09-21T21:41:23.817 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.817 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.817 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:23.817 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693357+0000", 2024-09-21T21:41:23.817 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059157847 2024-09-21T21:41:23.817 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.817 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.817 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:23.817 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693430+0000", 2024-09-21T21:41:23.817 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3203999999999999e-05 2024-09-21T21:41:23.817 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.817 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.817 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:23.817 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693525+0000", 2024-09-21T21:41:23.817 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5082999999999997e-05 2024-09-21T21:41:23.818 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:23.818 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:23.818 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:23.818 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.818 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.818 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1501 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 106496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:23.818 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634205+0000", 2024-09-21T21:41:23.818 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 22.164341222000001, 2024-09-21T21:41:23.818 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059482394000000001, 2024-09-21T21:41:23.818 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:23.818 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:23.818 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:23.818 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:23.818 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:23.819 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:23.819 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1501 2024-09-21T21:41:23.819 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.819 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:23.819 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.819 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:23.819 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634205+0000", 2024-09-21T21:41:23.819 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:23.819 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.819 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.819 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:23.819 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634205+0000", 2024-09-21T21:41:23.819 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:23.819 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.819 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.820 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:23.820 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634205+0000", 2024-09-21T21:41:23.820 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.03e-07 2024-09-21T21:41:23.820 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.820 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.820 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:23.820 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634207+0000", 2024-09-21T21:41:23.820 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0389999999999999e-06 2024-09-21T21:41:23.820 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.820 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.820 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:23.820 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634213+0000", 2024-09-21T21:41:23.820 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2600000000000002e-06 2024-09-21T21:41:23.820 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.820 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.821 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:23.821 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634218+0000", 2024-09-21T21:41:23.821 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8570000000000003e-06 2024-09-21T21:41:23.821 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.821 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.821 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:23.821 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693552+0000", 2024-09-21T21:41:23.821 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059333636000000002 2024-09-21T21:41:23.821 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.821 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.821 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:23.821 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693613+0000", 2024-09-21T21:41:23.821 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0560000000000003e-05 2024-09-21T21:41:23.821 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.822 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.822 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:23.822 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693687+0000", 2024-09-21T21:41:23.822 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4639000000000001e-05 2024-09-21T21:41:23.822 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:23.822 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:23.822 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:23.822 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.822 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.822 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1515 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 434176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:23.822 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634318+0000", 2024-09-21T21:41:23.823 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 22.164227534999998, 2024-09-21T21:41:23.823 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059565812000000003, 2024-09-21T21:41:23.823 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:23.823 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:23.823 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:23.823 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:23.823 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:23.823 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:23.823 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1515 2024-09-21T21:41:23.823 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.823 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:23.824 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.824 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:23.824 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634318+0000", 2024-09-21T21:41:23.824 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:23.824 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.824 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.824 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:23.824 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634318+0000", 2024-09-21T21:41:23.824 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:23.824 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.824 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.824 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:23.824 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634319+0000", 2024-09-21T21:41:23.824 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9400000000000001e-07 2024-09-21T21:41:23.824 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.825 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.825 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:23.825 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634321+0000", 2024-09-21T21:41:23.825 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.154e-06 2024-09-21T21:41:23.825 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.825 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.825 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:23.825 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634327+0000", 2024-09-21T21:41:23.825 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9830000000000001e-06 2024-09-21T21:41:23.825 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.825 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.825 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:23.825 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634332+0000", 2024-09-21T21:41:23.826 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9080000000000003e-06 2024-09-21T21:41:23.826 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.826 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.826 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:23.826 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693717+0000", 2024-09-21T21:41:23.826 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059385447000000001 2024-09-21T21:41:23.826 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.826 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.826 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:23.826 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693795+0000", 2024-09-21T21:41:23.826 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7267000000000004e-05 2024-09-21T21:41:23.826 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.826 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.826 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:23.826 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693884+0000", 2024-09-21T21:41:23.827 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9659000000000003e-05 2024-09-21T21:41:23.827 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:23.827 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:23.827 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:23.827 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.827 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.827 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1517 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 491520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:23.827 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634338+0000", 2024-09-21T21:41:23.827 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 22.164207762, 2024-09-21T21:41:23.827 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060100745999999997, 2024-09-21T21:41:23.827 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:23.827 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:23.827 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:23.827 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:23.828 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:23.828 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:23.828 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1517 2024-09-21T21:41:23.828 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.828 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:23.828 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.828 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:23.828 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634338+0000", 2024-09-21T21:41:23.828 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:23.828 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.828 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.828 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:23.828 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634338+0000", 2024-09-21T21:41:23.829 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:23.829 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.829 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.829 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:23.829 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634339+0000", 2024-09-21T21:41:23.829 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2899999999999999e-07 2024-09-21T21:41:23.829 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.829 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.829 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:23.829 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634341+0000", 2024-09-21T21:41:23.829 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.8959999999999999e-06 2024-09-21T21:41:23.829 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.829 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.829 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:23.830 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634347+0000", 2024-09-21T21:41:23.830 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.302e-06 2024-09-21T21:41:23.830 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.830 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.830 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:23.830 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634351+0000", 2024-09-21T21:41:23.830 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6070000000000001e-06 2024-09-21T21:41:23.830 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.830 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.830 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:23.830 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694279+0000", 2024-09-21T21:41:23.830 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059927046999999997 2024-09-21T21:41:23.830 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.830 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.831 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:23.831 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694355+0000", 2024-09-21T21:41:23.831 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6483999999999995e-05 2024-09-21T21:41:23.831 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.831 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.831 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:23.831 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694439+0000", 2024-09-21T21:41:23.831 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3980999999999994e-05 2024-09-21T21:41:23.831 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:23.831 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:23.831 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:23.831 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.831 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.832 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1518 3.16 3:6a41a830:::rbd_data.112145fae779.000000000000000a:a [read 294912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:23.832 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634357+0000", 2024-09-21T21:41:23.832 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 22.164188855999999, 2024-09-21T21:41:23.832 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059712318, 2024-09-21T21:41:23.832 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:23.832 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:23.832 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:23.832 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:23.832 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:23.832 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:23.832 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1518 2024-09-21T21:41:23.832 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.832 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:23.832 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.833 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:23.833 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634357+0000", 2024-09-21T21:41:23.833 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:23.833 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.833 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.833 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:23.833 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634357+0000", 2024-09-21T21:41:23.833 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:23.833 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.833 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.833 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:23.833 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634358+0000", 2024-09-21T21:41:23.833 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2100000000000002e-07 2024-09-21T21:41:23.833 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.833 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.834 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:23.834 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634360+0000", 2024-09-21T21:41:23.834 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0940000000000002e-06 2024-09-21T21:41:23.834 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.834 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.834 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:23.834 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634366+0000", 2024-09-21T21:41:23.834 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3389999999999998e-06 2024-09-21T21:41:23.834 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.834 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.834 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:23.834 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634371+0000", 2024-09-21T21:41:23.834 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6670000000000004e-06 2024-09-21T21:41:23.834 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.834 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.835 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:23.835 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693910+0000", 2024-09-21T21:41:23.835 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059538973000000002 2024-09-21T21:41:23.835 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.835 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.835 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:23.835 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693982+0000", 2024-09-21T21:41:23.835 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2806999999999996e-05 2024-09-21T21:41:23.835 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.835 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.835 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:23.835 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694069+0000", 2024-09-21T21:41:23.835 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7016999999999997e-05 2024-09-21T21:41:23.835 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:23.836 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:23.836 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:23.836 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.836 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.836 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1519 3.16 3:6a41a830:::rbd_data.112145fae779.000000000000000a:a [read 352256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:23.836 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634377+0000", 2024-09-21T21:41:23.836 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 22.164169154, 2024-09-21T21:41:23.836 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059872213000000001, 2024-09-21T21:41:23.836 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:23.836 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:23.836 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:23.836 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:23.836 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:23.836 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:23.837 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1519 2024-09-21T21:41:23.837 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.837 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:23.837 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.837 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:23.837 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634377+0000", 2024-09-21T21:41:23.837 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:23.837 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.837 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.837 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:23.837 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634377+0000", 2024-09-21T21:41:23.837 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:23.837 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.838 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.838 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:23.838 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634377+0000", 2024-09-21T21:41:23.838 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2600000000000002e-07 2024-09-21T21:41:23.838 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.838 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.838 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:23.838 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634379+0000", 2024-09-21T21:41:23.838 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0729999999999999e-06 2024-09-21T21:41:23.838 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.838 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.838 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:23.838 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634386+0000", 2024-09-21T21:41:23.839 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4690000000000002e-06 2024-09-21T21:41:23.839 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.839 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.839 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:23.839 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634390+0000", 2024-09-21T21:41:23.839 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4580000000000003e-06 2024-09-21T21:41:23.839 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.839 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.839 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:23.839 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694094+0000", 2024-09-21T21:41:23.839 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059703830999999999 2024-09-21T21:41:23.839 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.839 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.840 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:23.840 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694168+0000", 2024-09-21T21:41:23.840 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4222999999999997e-05 2024-09-21T21:41:23.840 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.840 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.840 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:23.840 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694249+0000", 2024-09-21T21:41:23.840 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0632999999999999e-05 2024-09-21T21:41:23.840 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:23.840 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:23.840 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:23.840 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.840 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.841 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1529 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 53248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:23.841 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634397+0000", 2024-09-21T21:41:23.841 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 22.164149002999999, 2024-09-21T21:41:23.841 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060189919000000001, 2024-09-21T21:41:23.841 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:23.841 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:23.841 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:23.841 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:23.841 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:23.841 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:23.841 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1529 2024-09-21T21:41:23.841 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.841 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:23.841 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.842 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:23.842 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634397+0000", 2024-09-21T21:41:23.842 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:23.842 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.842 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.842 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:23.842 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634397+0000", 2024-09-21T21:41:23.842 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:23.842 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.842 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.842 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:23.842 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634397+0000", 2024-09-21T21:41:23.843 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2e-07 2024-09-21T21:41:23.843 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.843 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.843 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:23.843 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634399+0000", 2024-09-21T21:41:23.843 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0609999999999998e-06 2024-09-21T21:41:23.843 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.843 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.843 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:23.843 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634406+0000", 2024-09-21T21:41:23.843 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7259999999999996e-06 2024-09-21T21:41:23.843 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.843 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.843 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:23.843 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634411+0000", 2024-09-21T21:41:23.844 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9339999999999997e-06 2024-09-21T21:41:23.844 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.844 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.844 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:23.844 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694480+0000", 2024-09-21T21:41:23.844 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060069105999999997 2024-09-21T21:41:23.844 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.844 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.844 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:23.844 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694527+0000", 2024-09-21T21:41:23.844 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7120000000000003e-05 2024-09-21T21:41:23.844 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.844 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.844 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:23.844 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694587+0000", 2024-09-21T21:41:23.844 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9552000000000002e-05 2024-09-21T21:41:23.845 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:23.845 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:23.845 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:23.845 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.845 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.845 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1531 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 110592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:23.845 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634417+0000", 2024-09-21T21:41:23.845 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 22.164129069000001, 2024-09-21T21:41:23.845 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060347287999999999, 2024-09-21T21:41:23.845 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:23.845 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:23.845 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:23.845 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:23.845 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:23.845 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:23.846 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1531 2024-09-21T21:41:23.846 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.846 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:23.846 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.846 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:23.846 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634417+0000", 2024-09-21T21:41:23.846 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:23.846 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.846 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.846 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:23.846 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634417+0000", 2024-09-21T21:41:23.846 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:23.846 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.847 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.847 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:23.847 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634417+0000", 2024-09-21T21:41:23.847 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1699999999999999e-07 2024-09-21T21:41:23.847 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.847 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.847 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:23.847 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634420+0000", 2024-09-21T21:41:23.847 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.243e-06 2024-09-21T21:41:23.847 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.847 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.847 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:23.847 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634426+0000", 2024-09-21T21:41:23.847 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2550000000000003e-06 2024-09-21T21:41:23.847 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.847 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.848 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:23.848 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634431+0000", 2024-09-21T21:41:23.848 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0150000000000003e-06 2024-09-21T21:41:23.848 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.848 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.848 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:23.848 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694618+0000", 2024-09-21T21:41:23.848 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060186836000000001 2024-09-21T21:41:23.848 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.848 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.848 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:23.848 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694685+0000", 2024-09-21T21:41:23.849 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6970000000000004e-05 2024-09-21T21:41:23.849 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.849 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.849 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:23.849 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694764+0000", 2024-09-21T21:41:23.849 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9552e-05 2024-09-21T21:41:23.849 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:23.849 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:23.849 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:23.849 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.849 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.849 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1532 3.e 3:724b31a3:::rbd_data.1121e540c096.0000000000000015:b [read 151552~36864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:23.849 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634437+0000", 2024-09-21T21:41:23.849 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 22.164109273000001, 2024-09-21T21:41:23.850 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060474314000000001, 2024-09-21T21:41:23.850 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:23.850 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:23.850 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:23.850 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:23.850 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:23.850 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:23.850 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1532 2024-09-21T21:41:23.850 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.850 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:23.850 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.850 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:23.850 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634437+0000", 2024-09-21T21:41:23.851 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:23.851 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.851 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.851 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:23.851 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634437+0000", 2024-09-21T21:41:23.851 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:23.851 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.851 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.851 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:23.851 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634437+0000", 2024-09-21T21:41:23.851 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9400000000000001e-07 2024-09-21T21:41:23.851 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.851 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.851 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:23.852 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634439+0000", 2024-09-21T21:41:23.852 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.114e-06 2024-09-21T21:41:23.852 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.852 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.852 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:23.852 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634446+0000", 2024-09-21T21:41:23.852 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1389999999999998e-06 2024-09-21T21:41:23.852 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.852 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.852 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:23.852 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634451+0000", 2024-09-21T21:41:23.852 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0010000000000001e-06 2024-09-21T21:41:23.852 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.852 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.852 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:23.853 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694794+0000", 2024-09-21T21:41:23.853 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060342509000000003 2024-09-21T21:41:23.853 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.853 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.853 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:23.853 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694835+0000", 2024-09-21T21:41:23.853 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0728e-05 2024-09-21T21:41:23.853 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.853 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.853 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:23.853 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694911+0000", 2024-09-21T21:41:23.853 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6428999999999996e-05 2024-09-21T21:41:23.853 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:23.854 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:23.854 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:23.854 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.854 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.854 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1534 3.e 3:724b31a3:::rbd_data.1121e540c096.0000000000000015:b [read 487424~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:23.854 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634477+0000", 2024-09-21T21:41:23.854 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 22.164068870000001, 2024-09-21T21:41:23.854 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060624009, 2024-09-21T21:41:23.854 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:23.854 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:23.854 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:23.854 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:23.855 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:23.855 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:23.855 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1534 2024-09-21T21:41:23.855 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.855 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:23.855 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.855 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:23.855 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634477+0000", 2024-09-21T21:41:23.855 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:23.855 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.855 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.855 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:23.855 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634477+0000", 2024-09-21T21:41:23.855 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:23.856 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.856 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.856 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:23.856 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634478+0000", 2024-09-21T21:41:23.856 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1399999999999997e-07 2024-09-21T21:41:23.856 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.856 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.856 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:23.856 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634480+0000", 2024-09-21T21:41:23.856 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1009999999999999e-06 2024-09-21T21:41:23.856 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.856 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.856 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:23.857 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634486+0000", 2024-09-21T21:41:23.857 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3289999999999999e-06 2024-09-21T21:41:23.857 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.857 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.857 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:23.857 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634491+0000", 2024-09-21T21:41:23.857 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6260000000000003e-06 2024-09-21T21:41:23.857 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.857 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.857 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:23.858 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694943+0000", 2024-09-21T21:41:23.858 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060451960999999999 2024-09-21T21:41:23.858 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.858 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.858 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:23.858 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695014+0000", 2024-09-21T21:41:23.858 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1587000000000004e-05 2024-09-21T21:41:23.858 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.858 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.858 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:23.858 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695101+0000", 2024-09-21T21:41:23.858 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6990999999999994e-05 2024-09-21T21:41:23.858 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:23.859 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:23.859 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:23.859 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.859 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.859 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1545 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 438272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:23.859 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634515+0000", 2024-09-21T21:41:23.859 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 22.164030669999999, 2024-09-21T21:41:23.859 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060707944, 2024-09-21T21:41:23.859 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:23.859 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:23.859 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:23.859 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:23.859 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:23.860 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:23.860 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1545 2024-09-21T21:41:23.860 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.860 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:23.860 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.860 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:23.860 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634515+0000", 2024-09-21T21:41:23.860 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:23.860 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.860 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.860 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:23.860 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634515+0000", 2024-09-21T21:41:23.860 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:23.860 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.861 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.861 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:23.861 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634516+0000", 2024-09-21T21:41:23.861 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0999999999999999e-07 2024-09-21T21:41:23.861 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.861 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.861 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:23.861 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634518+0000", 2024-09-21T21:41:23.861 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.846e-06 2024-09-21T21:41:23.861 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.861 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.861 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:23.862 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634524+0000", 2024-09-21T21:41:23.862 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2550000000000003e-06 2024-09-21T21:41:23.862 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.862 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.862 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:23.862 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634529+0000", 2024-09-21T21:41:23.862 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6940000000000003e-06 2024-09-21T21:41:23.862 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.862 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.862 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:23.862 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695127+0000", 2024-09-21T21:41:23.862 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060598921 2024-09-21T21:41:23.862 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.862 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.863 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:23.863 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695171+0000", 2024-09-21T21:41:23.863 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3588999999999999e-05 2024-09-21T21:41:23.863 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.863 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.863 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:23.863 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695223+0000", 2024-09-21T21:41:23.863 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2228999999999997e-05 2024-09-21T21:41:23.863 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:23.863 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:23.863 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:23.863 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.863 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.863 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1547 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 495616~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:23.863 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634534+0000", 2024-09-21T21:41:23.864 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 22.164011879, 2024-09-21T21:41:23.864 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060881131999999998, 2024-09-21T21:41:23.864 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:23.864 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:23.864 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:23.864 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:23.864 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:23.864 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:23.864 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1547 2024-09-21T21:41:23.864 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.864 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:23.864 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.864 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:23.864 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634534+0000", 2024-09-21T21:41:23.865 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:23.865 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.865 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.865 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:23.865 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634534+0000", 2024-09-21T21:41:23.865 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:23.865 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.865 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.865 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:23.865 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634535+0000", 2024-09-21T21:41:23.865 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.77e-07 2024-09-21T21:41:23.865 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.865 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.866 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:23.866 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634542+0000", 2024-09-21T21:41:23.866 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9000000000000006e-06 2024-09-21T21:41:23.866 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.866 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.866 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:23.866 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634549+0000", 2024-09-21T21:41:23.866 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5910000000000001e-06 2024-09-21T21:41:23.866 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.866 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.866 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:23.866 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634554+0000", 2024-09-21T21:41:23.866 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7940000000000002e-06 2024-09-21T21:41:23.866 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.867 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.867 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:23.867 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695256+0000", 2024-09-21T21:41:23.867 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.06070246 2024-09-21T21:41:23.867 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.867 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.867 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:23.867 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695332+0000", 2024-09-21T21:41:23.867 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6248000000000006e-05 2024-09-21T21:41:23.867 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.867 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.867 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:23.867 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695415+0000", 2024-09-21T21:41:23.867 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2762000000000005e-05 2024-09-21T21:41:23.868 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:23.868 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:23.868 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:23.868 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.868 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.868 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1548 3.16 3:6a41a830:::rbd_data.112145fae779.000000000000000a:a [read 299008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:23.868 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634561+0000", 2024-09-21T21:41:23.868 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 22.163985302, 2024-09-21T21:41:23.868 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060786743999999997, 2024-09-21T21:41:23.868 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:23.868 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:23.868 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:23.868 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:23.868 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:23.869 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:23.869 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1548 2024-09-21T21:41:23.869 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.869 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:23.869 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.869 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:23.869 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634561+0000", 2024-09-21T21:41:23.869 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:23.869 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.869 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.869 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:23.869 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634561+0000", 2024-09-21T21:41:23.869 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:23.869 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.870 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.870 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:23.870 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634561+0000", 2024-09-21T21:41:23.870 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5199999999999997e-07 2024-09-21T21:41:23.870 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.870 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.870 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:23.870 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634564+0000", 2024-09-21T21:41:23.870 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4040000000000002e-06 2024-09-21T21:41:23.870 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.870 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.870 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:23.870 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634571+0000", 2024-09-21T21:41:23.870 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4379999999999998e-06 2024-09-21T21:41:23.871 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.871 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.871 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:23.871 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634576+0000", 2024-09-21T21:41:23.871 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1399999999999999e-06 2024-09-21T21:41:23.871 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.871 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.871 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:23.871 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695247+0000", 2024-09-21T21:41:23.871 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060670397000000001 2024-09-21T21:41:23.871 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.871 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.871 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:23.871 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695299+0000", 2024-09-21T21:41:23.872 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2564e-05 2024-09-21T21:41:23.872 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.872 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.872 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:23.872 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695347+0000", 2024-09-21T21:41:23.872 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8248999999999998e-05 2024-09-21T21:41:23.872 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:23.872 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:23.872 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:23.872 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.872 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.872 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1549 3.16 3:6a41a830:::rbd_data.112145fae779.000000000000000a:a [read 356352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:23.872 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634583+0000", 2024-09-21T21:41:23.872 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 22.163963388999999, 2024-09-21T21:41:23.873 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060889459, 2024-09-21T21:41:23.873 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:23.873 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:23.873 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:23.873 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:23.873 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:23.873 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:23.873 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1549 2024-09-21T21:41:23.873 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.873 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:23.873 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.873 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:23.873 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634583+0000", 2024-09-21T21:41:23.873 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:23.874 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.874 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.874 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:23.874 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634583+0000", 2024-09-21T21:41:23.874 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:23.874 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.874 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.874 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:23.874 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634583+0000", 2024-09-21T21:41:23.874 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1899999999999998e-07 2024-09-21T21:41:23.874 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.874 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.874 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:23.875 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634585+0000", 2024-09-21T21:41:23.875 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2129999999999998e-06 2024-09-21T21:41:23.875 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.875 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.875 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:23.875 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634592+0000", 2024-09-21T21:41:23.875 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2840000000000003e-06 2024-09-21T21:41:23.875 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.875 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.875 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:23.875 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634596+0000", 2024-09-21T21:41:23.875 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9309999999999999e-06 2024-09-21T21:41:23.875 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.875 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.875 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:23.876 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695371+0000", 2024-09-21T21:41:23.876 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060774425999999999 2024-09-21T21:41:23.876 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.876 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.876 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:23.876 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695419+0000", 2024-09-21T21:41:23.876 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8607999999999999e-05 2024-09-21T21:41:23.876 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.876 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.876 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:23.876 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695472+0000", 2024-09-21T21:41:23.876 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2577999999999997e-05 2024-09-21T21:41:23.876 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:23.876 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:23.877 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:23.877 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.877 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.877 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1588 3.16 3:695c582e:::rbd_data.1121e540c096.0000000000000008:b [read 147456~40960] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:23.877 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.636942+0000", 2024-09-21T21:41:23.877 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 22.161603612, 2024-09-21T21:41:23.877 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.058643811999999997, 2024-09-21T21:41:23.877 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:23.877 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:23.877 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:23.877 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:23.877 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:23.877 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:23.877 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1588 2024-09-21T21:41:23.878 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.878 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:23.878 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.878 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:23.878 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636942+0000", 2024-09-21T21:41:23.878 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:23.878 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.878 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.878 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:23.878 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636942+0000", 2024-09-21T21:41:23.878 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:23.878 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.878 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.878 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:23.879 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636943+0000", 2024-09-21T21:41:23.879 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4199999999999996e-07 2024-09-21T21:41:23.879 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.879 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.879 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:23.879 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636945+0000", 2024-09-21T21:41:23.879 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.401e-06 2024-09-21T21:41:23.879 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.879 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.879 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:23.879 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636952+0000", 2024-09-21T21:41:23.879 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5379999999999999e-06 2024-09-21T21:41:23.879 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.880 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.880 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:23.880 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636958+0000", 2024-09-21T21:41:23.880 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.489e-06 2024-09-21T21:41:23.880 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.880 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.880 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:23.880 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695497+0000", 2024-09-21T21:41:23.880 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.058538504999999998 2024-09-21T21:41:23.880 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.880 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.880 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:23.880 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695533+0000", 2024-09-21T21:41:23.880 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6118000000000002e-05 2024-09-21T21:41:23.881 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.881 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.881 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:23.881 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695586+0000", 2024-09-21T21:41:23.881 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3219e-05 2024-09-21T21:41:23.881 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:23.881 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:23.881 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:23.881 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.881 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.881 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1589 3.16 3:695c582e:::rbd_data.1121e540c096.0000000000000008:b [read 487424~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:23.881 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.636965+0000", 2024-09-21T21:41:23.881 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 22.161580521000001, 2024-09-21T21:41:23.881 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.058728152999999998, 2024-09-21T21:41:23.882 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:23.882 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:23.882 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:23.882 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:23.882 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:23.882 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:23.882 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1589 2024-09-21T21:41:23.882 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.882 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:23.882 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.882 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:23.882 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636965+0000", 2024-09-21T21:41:23.882 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:23.882 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.883 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.883 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:23.883 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636965+0000", 2024-09-21T21:41:23.883 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:23.883 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.883 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.883 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:23.883 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636966+0000", 2024-09-21T21:41:23.883 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3799999999999998e-07 2024-09-21T21:41:23.883 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.883 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.883 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:23.883 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636968+0000", 2024-09-21T21:41:23.883 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2400000000000002e-06 2024-09-21T21:41:23.884 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.884 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.884 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:23.884 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636976+0000", 2024-09-21T21:41:23.884 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5050000000000001e-06 2024-09-21T21:41:23.884 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.884 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.884 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:23.884 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636981+0000", 2024-09-21T21:41:23.884 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2170000000000002e-06 2024-09-21T21:41:23.884 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.884 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.884 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:23.885 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695610+0000", 2024-09-21T21:41:23.885 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.058628998000000002 2024-09-21T21:41:23.885 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.885 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.885 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:23.885 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695647+0000", 2024-09-21T21:41:23.885 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6752000000000003e-05 2024-09-21T21:41:23.885 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.885 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.885 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:23.885 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695694+0000", 2024-09-21T21:41:23.885 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7002999999999997e-05 2024-09-21T21:41:23.886 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:23.886 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:23.886 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:23.886 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.886 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.886 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1607 3.e 3:724b31a3:::rbd_data.1121e540c096.0000000000000015:b [read 299008~36864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:23.886 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.637062+0000", 2024-09-21T21:41:23.886 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 22.161484430000002, 2024-09-21T21:41:23.886 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.058623078000000002, 2024-09-21T21:41:23.886 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:23.886 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:23.886 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:23.886 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:23.887 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:23.887 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:23.887 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1607 2024-09-21T21:41:23.887 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.887 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:23.887 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.887 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:23.887 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.637062+0000", 2024-09-21T21:41:23.887 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:23.887 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.887 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.887 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:23.887 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.637062+0000", 2024-09-21T21:41:23.887 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:23.888 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.888 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.888 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:23.888 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.637062+0000", 2024-09-21T21:41:23.888 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0900000000000002e-07 2024-09-21T21:41:23.888 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.888 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.888 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:23.888 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.637069+0000", 2024-09-21T21:41:23.888 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.267e-06 2024-09-21T21:41:23.888 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.889 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.889 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:23.889 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.637076+0000", 2024-09-21T21:41:23.889 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7320000000000001e-06 2024-09-21T21:41:23.889 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.889 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.889 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:23.889 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.637082+0000", 2024-09-21T21:41:23.889 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4760000000000003e-06 2024-09-21T21:41:23.890 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.890 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.890 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:23.890 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695604+0000", 2024-09-21T21:41:23.890 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.058521261999999998 2024-09-21T21:41:23.890 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.890 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.890 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:23.890 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695638+0000", 2024-09-21T21:41:23.890 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4292999999999998e-05 2024-09-21T21:41:23.890 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:23.890 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:23.890 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:23.890 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695685+0000", 2024-09-21T21:41:23.891 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6638999999999998e-05 2024-09-21T21:41:23.891 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:23.891 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:23.891 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:23.891 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:23.891 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:23.891 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:23.892 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:23.891+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:23.893 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T21:41:23.993 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:23.991+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:24.002 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:24.002 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:41:24.002 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 0 2024-09-21T21:41:24.002 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:24.011 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T21:41:24.097 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:24.096+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:24.100 INFO:teuthology.orchestra.run.smithi164.stdout:checking clone #17, image image_client.0-clone17 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent18 2024-09-21T21:41:24.161 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:24.161 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:41:24.162 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:41:24.162 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:41:24.162 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:24.170 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T21:41:24.212 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:24.210+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:24.218 INFO:teuthology.orchestra.run.smithi164.stdout:2853 read 0x31a6e00 thru 0x31a920e (0x240f bytes) 2024-09-21T21:41:24.219 INFO:teuthology.orchestra.run.smithi164.stdout:2854 punch from 0x70fdc00 to 0x7103c00, (0x6000 bytes) 2024-09-21T21:41:24.242 INFO:teuthology.orchestra.run.smithi164.stdout:2855 read 0xae7e000 thru 0xae81a7b (0x3a7c bytes) 2024-09-21T21:41:24.242 INFO:teuthology.orchestra.run.smithi164.stdout:2856 write 0x6567200 thru 0x657682c (0xf62d bytes) 2024-09-21T21:41:24.243 INFO:teuthology.orchestra.run.smithi164.stdout:2857 punch from 0xbcb1a00 to 0xbcbd400, (0xba00 bytes) 2024-09-21T21:41:24.272 INFO:teuthology.orchestra.run.smithi164.stdout:2859 write 0xb830c00 thru 0xb83fd23 (0xf124 bytes) 2024-09-21T21:41:24.275 INFO:teuthology.orchestra.run.smithi164.stdout:2860 trunc from 0xdf82000 to 0xb5fe000 2024-09-21T21:41:24.295 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:24.293+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:24.358 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:24.358 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 20, 2024-09-21T21:41:24.358 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 600, 2024-09-21T21:41:24.358 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:41:24.358 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.358 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4840.0:4841 3.10 3:0d540e35:::rbd_data.11214ff8bd9b.000000000000001a:10 [read 61440~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T21:41:24.358 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:16.470965+0000", 2024-09-21T21:41:24.358 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 7.8843738949999995, 2024-09-21T21:41:24.359 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.016531272999999999, 2024-09-21T21:41:24.359 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:24.359 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:24.359 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:24.359 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:24.359 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4840", 2024-09-21T21:41:24.359 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1293784138", 2024-09-21T21:41:24.359 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4841 2024-09-21T21:41:24.359 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.359 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:24.359 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.359 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:24.359 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.470965+0000", 2024-09-21T21:41:24.360 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:24.360 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.360 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.360 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:24.360 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.470965+0000", 2024-09-21T21:41:24.360 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:24.360 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.360 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.360 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:24.360 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.470966+0000", 2024-09-21T21:41:24.360 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4900000000000001e-07 2024-09-21T21:41:24.361 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.361 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.361 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:24.361 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.470968+0000", 2024-09-21T21:41:24.361 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9929999999999998e-06 2024-09-21T21:41:24.361 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.361 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.361 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:24.361 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.470974+0000", 2024-09-21T21:41:24.361 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7860000000000002e-06 2024-09-21T21:41:24.361 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.361 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.361 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:24.362 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.470979+0000", 2024-09-21T21:41:24.362 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4609999999999999e-06 2024-09-21T21:41:24.362 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.362 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.362 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:24.362 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.487343+0000", 2024-09-21T21:41:24.362 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.016363453 2024-09-21T21:41:24.362 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.362 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.362 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:24.363 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.487411+0000", 2024-09-21T21:41:24.363 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.8072999999999997e-05 2024-09-21T21:41:24.363 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.363 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.363 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:24.363 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.487497+0000", 2024-09-21T21:41:24.363 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.6057999999999995e-05 2024-09-21T21:41:24.363 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:24.363 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:24.363 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:24.363 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.363 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.363 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4840.0:4842 3.10 3:0d540e35:::rbd_data.11214ff8bd9b.000000000000001a:10 [read 73728~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T21:41:24.364 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:16.470985+0000", 2024-09-21T21:41:24.364 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 7.884354396, 2024-09-21T21:41:24.364 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.016696136, 2024-09-21T21:41:24.364 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:24.364 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:24.364 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:24.364 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:24.364 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4840", 2024-09-21T21:41:24.364 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1293784138", 2024-09-21T21:41:24.364 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4842 2024-09-21T21:41:24.364 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.364 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:24.364 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.364 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:24.365 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.470985+0000", 2024-09-21T21:41:24.365 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:24.365 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.365 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.365 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:24.365 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.470985+0000", 2024-09-21T21:41:24.365 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:24.365 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.365 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.365 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:24.365 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.470985+0000", 2024-09-21T21:41:24.365 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5900000000000003e-07 2024-09-21T21:41:24.365 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.366 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.366 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:24.366 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.470987+0000", 2024-09-21T21:41:24.366 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0200000000000001e-06 2024-09-21T21:41:24.366 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.366 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.366 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:24.366 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.470994+0000", 2024-09-21T21:41:24.366 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4300000000000003e-06 2024-09-21T21:41:24.366 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.366 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.366 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:24.366 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.471000+0000", 2024-09-21T21:41:24.367 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0260000000000004e-06 2024-09-21T21:41:24.367 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.367 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.367 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:24.367 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.487526+0000", 2024-09-21T21:41:24.367 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.016526279000000001 2024-09-21T21:41:24.367 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.367 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.367 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:24.367 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.487592+0000", 2024-09-21T21:41:24.367 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6187999999999997e-05 2024-09-21T21:41:24.367 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.367 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.367 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:24.368 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.487681+0000", 2024-09-21T21:41:24.368 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.8833999999999997e-05 2024-09-21T21:41:24.368 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:24.368 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:24.368 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:24.368 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.368 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.368 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4840.0:4850 3.10 3:0d9e118e:::rbd_data.11214ff8bd9b.000000000000001c:10 [read 10240~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T21:41:24.368 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:16.471006+0000", 2024-09-21T21:41:24.368 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 7.8843335210000003, 2024-09-21T21:41:24.368 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.01680357, 2024-09-21T21:41:24.368 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:24.368 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:24.368 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:24.369 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:24.369 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4840", 2024-09-21T21:41:24.369 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1293784138", 2024-09-21T21:41:24.369 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4850 2024-09-21T21:41:24.369 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.369 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:24.369 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.369 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:24.369 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.471006+0000", 2024-09-21T21:41:24.369 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:24.369 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.369 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.369 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:24.369 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.471006+0000", 2024-09-21T21:41:24.370 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:24.370 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.370 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.370 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:24.370 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.471006+0000", 2024-09-21T21:41:24.370 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.77e-07 2024-09-21T21:41:24.370 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.370 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.370 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:24.370 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.471008+0000", 2024-09-21T21:41:24.370 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.187e-06 2024-09-21T21:41:24.370 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.370 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.370 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:24.370 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.471015+0000", 2024-09-21T21:41:24.371 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.08e-06 2024-09-21T21:41:24.371 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.371 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.371 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:24.371 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.471020+0000", 2024-09-21T21:41:24.371 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8609999999999997e-06 2024-09-21T21:41:24.371 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.371 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.371 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:24.371 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.487708+0000", 2024-09-21T21:41:24.371 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.016687898999999999 2024-09-21T21:41:24.371 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.371 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.371 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:24.372 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.487756+0000", 2024-09-21T21:41:24.372 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7654999999999997e-05 2024-09-21T21:41:24.372 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.372 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.372 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:24.372 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.487809+0000", 2024-09-21T21:41:24.372 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3511000000000003e-05 2024-09-21T21:41:24.372 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:24.372 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:24.372 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:24.372 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.372 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.372 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4840.0:4851 3.10 3:0d9e118e:::rbd_data.11214ff8bd9b.000000000000001c:10 [read 22528~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T21:41:24.373 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:16.471026+0000", 2024-09-21T21:41:24.373 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 7.8843137260000002, 2024-09-21T21:41:24.373 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.016986455000000001, 2024-09-21T21:41:24.373 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:24.373 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:24.373 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:24.373 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:24.373 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4840", 2024-09-21T21:41:24.373 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1293784138", 2024-09-21T21:41:24.373 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4851 2024-09-21T21:41:24.373 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.373 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:24.373 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.373 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:24.373 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.471026+0000", 2024-09-21T21:41:24.374 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:24.374 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.374 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.374 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:24.374 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.471026+0000", 2024-09-21T21:41:24.374 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:24.374 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.374 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.374 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:24.374 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.471026+0000", 2024-09-21T21:41:24.374 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.1899999999999998e-07 2024-09-21T21:41:24.374 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.374 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.374 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:24.374 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.471028+0000", 2024-09-21T21:41:24.375 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9230000000000001e-06 2024-09-21T21:41:24.375 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.375 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.375 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:24.375 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.471033+0000", 2024-09-21T21:41:24.375 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1259999999999997e-06 2024-09-21T21:41:24.375 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.375 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.375 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:24.375 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.471037+0000", 2024-09-21T21:41:24.375 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.9330000000000003e-06 2024-09-21T21:41:24.375 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.375 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.376 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:24.376 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.487846+0000", 2024-09-21T21:41:24.376 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.016808640999999999 2024-09-21T21:41:24.376 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.376 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.376 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:24.376 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.487915+0000", 2024-09-21T21:41:24.376 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9715000000000006e-05 2024-09-21T21:41:24.376 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.376 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.376 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:24.376 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.488012+0000", 2024-09-21T21:41:24.376 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6798000000000004e-05 2024-09-21T21:41:24.376 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:24.376 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:24.377 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:24.377 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.377 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.377 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4840.0:4852 3.10 3:0d9e118e:::rbd_data.11214ff8bd9b.000000000000001c:10 [read 34816~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T21:41:24.377 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:16.471254+0000", 2024-09-21T21:41:24.377 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 7.8840850900000001, 2024-09-21T21:41:24.377 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.017002172999999999, 2024-09-21T21:41:24.377 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:24.377 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:24.377 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:24.377 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:24.377 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4840", 2024-09-21T21:41:24.377 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1293784138", 2024-09-21T21:41:24.377 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4852 2024-09-21T21:41:24.378 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.378 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:24.378 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.378 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:24.378 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.471254+0000", 2024-09-21T21:41:24.378 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:24.378 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.378 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.378 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:24.378 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.471254+0000", 2024-09-21T21:41:24.378 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:24.378 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.378 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.378 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:24.379 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.471255+0000", 2024-09-21T21:41:24.379 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.7400000000000003e-07 2024-09-21T21:41:24.379 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.379 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.379 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:24.379 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.471259+0000", 2024-09-21T21:41:24.379 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6260000000000001e-06 2024-09-21T21:41:24.379 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.379 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.379 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:24.379 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.471267+0000", 2024-09-21T21:41:24.379 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.9910000000000002e-06 2024-09-21T21:41:24.379 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.379 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.379 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:24.380 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.471273+0000", 2024-09-21T21:41:24.380 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4300000000000003e-06 2024-09-21T21:41:24.380 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.380 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.380 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:24.380 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.488041+0000", 2024-09-21T21:41:24.380 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.016767707999999999 2024-09-21T21:41:24.380 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.380 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.380 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:24.380 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.488083+0000", 2024-09-21T21:41:24.380 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2206000000000002e-05 2024-09-21T21:41:24.380 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.380 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.381 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:24.381 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.488256+0000", 2024-09-21T21:41:24.381 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00017353799999999999 2024-09-21T21:41:24.381 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:24.381 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:24.381 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:24.381 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.381 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.381 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4840.0:4853 3.10 3:0d9e118e:::rbd_data.11214ff8bd9b.000000000000001c:10 [read 47104~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T21:41:24.381 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:16.471284+0000", 2024-09-21T21:41:24.381 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 7.884055418, 2024-09-21T21:41:24.381 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.017177549, 2024-09-21T21:41:24.381 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:24.381 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:24.382 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:24.382 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:24.382 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4840", 2024-09-21T21:41:24.382 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1293784138", 2024-09-21T21:41:24.382 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4853 2024-09-21T21:41:24.382 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.382 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:24.382 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.382 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:24.382 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.471284+0000", 2024-09-21T21:41:24.382 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:24.382 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.382 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.382 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:24.383 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.471284+0000", 2024-09-21T21:41:24.383 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:24.383 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.383 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.383 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:24.383 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.471284+0000", 2024-09-21T21:41:24.383 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0999999999999999e-07 2024-09-21T21:41:24.383 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.383 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.383 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:24.383 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.471287+0000", 2024-09-21T21:41:24.383 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2800000000000002e-06 2024-09-21T21:41:24.383 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.383 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.383 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:24.384 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.471293+0000", 2024-09-21T21:41:24.384 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.7270000000000001e-06 2024-09-21T21:41:24.384 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.384 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.384 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:24.384 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.471299+0000", 2024-09-21T21:41:24.384 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.091e-06 2024-09-21T21:41:24.384 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.384 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.384 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:24.384 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.488287+0000", 2024-09-21T21:41:24.384 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.016988429999999999 2024-09-21T21:41:24.384 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.384 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.385 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:24.385 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.488362+0000", 2024-09-21T21:41:24.385 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.4690999999999994e-05 2024-09-21T21:41:24.385 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.385 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.385 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:24.385 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.488462+0000", 2024-09-21T21:41:24.385 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.9820000000000003e-05 2024-09-21T21:41:24.385 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:24.385 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:24.385 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:24.385 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.385 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.385 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4840.0:4854 3.10 3:0d9e118e:::rbd_data.11214ff8bd9b.000000000000001c:10 [read 59392~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T21:41:24.386 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:16.471305+0000", 2024-09-21T21:41:24.386 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 7.8840340800000002, 2024-09-21T21:41:24.386 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.017362687000000002, 2024-09-21T21:41:24.386 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:24.386 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:24.386 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:24.386 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:24.386 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4840", 2024-09-21T21:41:24.386 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1293784138", 2024-09-21T21:41:24.386 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4854 2024-09-21T21:41:24.386 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.386 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:24.386 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.387 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:24.387 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.471305+0000", 2024-09-21T21:41:24.387 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:24.387 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.387 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.387 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:24.387 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.471305+0000", 2024-09-21T21:41:24.387 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:24.387 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.387 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.387 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:24.387 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.471306+0000", 2024-09-21T21:41:24.387 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7099999999999997e-07 2024-09-21T21:41:24.387 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.388 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.388 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:24.388 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.471313+0000", 2024-09-21T21:41:24.388 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0299999999999996e-06 2024-09-21T21:41:24.388 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.388 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.388 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:24.388 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.471319+0000", 2024-09-21T21:41:24.388 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.776e-06 2024-09-21T21:41:24.388 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.388 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.388 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:24.388 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.471325+0000", 2024-09-21T21:41:24.388 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0599999999999998e-06 2024-09-21T21:41:24.389 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.389 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.389 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:24.389 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.488496+0000", 2024-09-21T21:41:24.389 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.017171158999999998 2024-09-21T21:41:24.389 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.389 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.389 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:24.389 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.488570+0000", 2024-09-21T21:41:24.389 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.4621000000000003e-05 2024-09-21T21:41:24.389 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.389 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.389 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:24.389 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.488668+0000", 2024-09-21T21:41:24.389 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.7670000000000005e-05 2024-09-21T21:41:24.390 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:24.390 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:24.390 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:24.390 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.390 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.390 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4840.0:4855 3.10 3:0d9e118e:::rbd_data.11214ff8bd9b.000000000000001c:10 [read 71680~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T21:41:24.390 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:16.471331+0000", 2024-09-21T21:41:24.390 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 7.8840080810000002, 2024-09-21T21:41:24.390 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.017549258000000002, 2024-09-21T21:41:24.390 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:24.390 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:24.390 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:24.390 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:24.391 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4840", 2024-09-21T21:41:24.391 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1293784138", 2024-09-21T21:41:24.391 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4855 2024-09-21T21:41:24.391 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.391 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:24.391 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.391 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:24.391 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.471331+0000", 2024-09-21T21:41:24.391 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:24.391 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.391 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.391 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:24.391 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.471331+0000", 2024-09-21T21:41:24.392 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:24.392 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.392 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.392 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:24.392 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.471332+0000", 2024-09-21T21:41:24.392 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6300000000000005e-07 2024-09-21T21:41:24.392 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.392 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.392 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:24.392 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.471334+0000", 2024-09-21T21:41:24.392 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1840000000000002e-06 2024-09-21T21:41:24.392 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.392 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.392 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:24.392 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.471340+0000", 2024-09-21T21:41:24.392 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3929999999999996e-06 2024-09-21T21:41:24.393 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.393 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.393 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:24.393 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.471346+0000", 2024-09-21T21:41:24.393 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8200000000000002e-06 2024-09-21T21:41:24.393 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.393 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.393 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:24.393 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.488701+0000", 2024-09-21T21:41:24.393 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.017354517999999999 2024-09-21T21:41:24.393 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.393 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.393 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:24.393 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.488783+0000", 2024-09-21T21:41:24.394 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.2175999999999994e-05 2024-09-21T21:41:24.394 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.394 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.394 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:24.394 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.488881+0000", 2024-09-21T21:41:24.394 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.7603999999999996e-05 2024-09-21T21:41:24.394 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:24.394 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:24.394 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:24.394 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.394 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.394 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4840.0:4969 3.10 3:0d540e35:::rbd_data.11214ff8bd9b.000000000000001a:10 [read 38912~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T21:41:24.394 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:16.474361+0000", 2024-09-21T21:41:24.394 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 7.8809786610000003, 2024-09-21T21:41:24.395 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.016443787000000001, 2024-09-21T21:41:24.395 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:24.395 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:24.395 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:24.395 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:24.395 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4840", 2024-09-21T21:41:24.395 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1293784138", 2024-09-21T21:41:24.395 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4969 2024-09-21T21:41:24.395 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.395 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:24.395 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.395 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:24.395 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.474361+0000", 2024-09-21T21:41:24.395 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:24.396 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.396 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.396 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:24.396 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.474361+0000", 2024-09-21T21:41:24.396 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:24.396 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.396 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.396 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:24.396 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.474361+0000", 2024-09-21T21:41:24.396 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.9700000000000003e-07 2024-09-21T21:41:24.396 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.396 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.396 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:24.396 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.474363+0000", 2024-09-21T21:41:24.396 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.8840000000000001e-06 2024-09-21T21:41:24.397 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.397 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.397 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:24.397 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.474368+0000", 2024-09-21T21:41:24.397 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3550000000000004e-06 2024-09-21T21:41:24.397 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.397 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.397 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:24.397 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.474372+0000", 2024-09-21T21:41:24.397 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8340000000000001e-06 2024-09-21T21:41:24.397 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.397 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.397 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:24.397 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.490648+0000", 2024-09-21T21:41:24.398 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.016275540000000002 2024-09-21T21:41:24.398 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.398 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.398 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:24.398 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.490714+0000", 2024-09-21T21:41:24.398 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6173000000000005e-05 2024-09-21T21:41:24.398 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.398 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.398 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:24.398 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.490804+0000", 2024-09-21T21:41:24.398 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.0704000000000004e-05 2024-09-21T21:41:24.398 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:24.398 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:24.398 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:24.398 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.399 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.399 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4840.0:4970 3.10 3:0d540e35:::rbd_data.11214ff8bd9b.000000000000001a:10 [read 51200~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T21:41:24.399 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:16.474378+0000", 2024-09-21T21:41:24.399 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 7.8809611019999997, 2024-09-21T21:41:24.399 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.016554359000000001, 2024-09-21T21:41:24.399 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:24.399 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:24.399 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:24.399 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:24.399 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4840", 2024-09-21T21:41:24.399 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1293784138", 2024-09-21T21:41:24.399 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4970 2024-09-21T21:41:24.399 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.399 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:24.400 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.400 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:24.400 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.474378+0000", 2024-09-21T21:41:24.400 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:24.400 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.400 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.400 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:24.400 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.474378+0000", 2024-09-21T21:41:24.400 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:24.400 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.400 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.400 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:24.400 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.474379+0000", 2024-09-21T21:41:24.400 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1699999999999999e-07 2024-09-21T21:41:24.400 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.401 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.401 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:24.401 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.474381+0000", 2024-09-21T21:41:24.401 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.289e-06 2024-09-21T21:41:24.401 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.401 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.401 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:24.401 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.474387+0000", 2024-09-21T21:41:24.401 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2790000000000004e-06 2024-09-21T21:41:24.401 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.401 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.401 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:24.401 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.474392+0000", 2024-09-21T21:41:24.401 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.904e-06 2024-09-21T21:41:24.402 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.402 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.402 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:24.402 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.490837+0000", 2024-09-21T21:41:24.402 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.016444437999999999 2024-09-21T21:41:24.402 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.402 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.402 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:24.402 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.490876+0000", 2024-09-21T21:41:24.403 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.9578e-05 2024-09-21T21:41:24.403 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.403 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.403 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:24.403 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.490933+0000", 2024-09-21T21:41:24.403 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6453999999999999e-05 2024-09-21T21:41:24.403 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:24.403 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:24.403 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:24.403 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.403 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.403 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4840.0:4971 3.10 3:0d540e35:::rbd_data.11214ff8bd9b.000000000000001a:10 [read 63488~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T21:41:24.403 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:16.474398+0000", 2024-09-21T21:41:24.404 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 7.8809412859999997, 2024-09-21T21:41:24.404 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.016653682, 2024-09-21T21:41:24.404 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:24.404 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:24.404 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:24.404 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:24.404 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4840", 2024-09-21T21:41:24.404 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1293784138", 2024-09-21T21:41:24.404 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4971 2024-09-21T21:41:24.404 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.404 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:24.404 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.404 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:24.404 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.474398+0000", 2024-09-21T21:41:24.404 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:24.405 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.405 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.405 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:24.405 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.474398+0000", 2024-09-21T21:41:24.405 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:24.405 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.405 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.405 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:24.405 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.474399+0000", 2024-09-21T21:41:24.405 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6100000000000001e-07 2024-09-21T21:41:24.405 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.405 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.405 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:24.405 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.474401+0000", 2024-09-21T21:41:24.405 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1610000000000001e-06 2024-09-21T21:41:24.406 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.406 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.406 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:24.406 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.474407+0000", 2024-09-21T21:41:24.406 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.139e-06 2024-09-21T21:41:24.406 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.406 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.406 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:24.406 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.474412+0000", 2024-09-21T21:41:24.406 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8770000000000001e-06 2024-09-21T21:41:24.406 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.406 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.406 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:24.406 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.490965+0000", 2024-09-21T21:41:24.406 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.016553674000000001 2024-09-21T21:41:24.407 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.407 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.407 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:24.407 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.491001+0000", 2024-09-21T21:41:24.407 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5858999999999997e-05 2024-09-21T21:41:24.407 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.407 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.407 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:24.407 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.491052+0000", 2024-09-21T21:41:24.407 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0510999999999998e-05 2024-09-21T21:41:24.407 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:24.407 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:24.407 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:24.407 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.407 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.408 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4840.0:4972 3.10 3:0d540e35:::rbd_data.11214ff8bd9b.000000000000001a:10 [read 75776~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T21:41:24.408 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:16.474418+0000", 2024-09-21T21:41:24.408 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 7.8809217729999999, 2024-09-21T21:41:24.408 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.016839878999999999, 2024-09-21T21:41:24.408 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:24.408 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:24.408 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:24.408 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:24.408 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4840", 2024-09-21T21:41:24.409 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1293784138", 2024-09-21T21:41:24.409 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4972 2024-09-21T21:41:24.409 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.409 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:24.409 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.410 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:24.410 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.474418+0000", 2024-09-21T21:41:24.410 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:24.410 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.410 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.410 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:24.410 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.474418+0000", 2024-09-21T21:41:24.410 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:24.410 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.410 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.410 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:24.410 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.474418+0000", 2024-09-21T21:41:24.410 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6800000000000001e-07 2024-09-21T21:41:24.411 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.411 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.411 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:24.411 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.474424+0000", 2024-09-21T21:41:24.411 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8730000000000004e-06 2024-09-21T21:41:24.411 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.411 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.411 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:24.411 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.474430+0000", 2024-09-21T21:41:24.411 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.409e-06 2024-09-21T21:41:24.411 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.411 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.412 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:24.412 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.474435+0000", 2024-09-21T21:41:24.412 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7659999999999998e-06 2024-09-21T21:41:24.412 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.412 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.412 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:24.412 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.491091+0000", 2024-09-21T21:41:24.412 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.016656098000000001 2024-09-21T21:41:24.412 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.412 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.412 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:24.412 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.491165+0000", 2024-09-21T21:41:24.412 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.3832999999999996e-05 2024-09-21T21:41:24.413 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.413 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.413 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:24.413 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.491257+0000", 2024-09-21T21:41:24.413 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2532000000000001e-05 2024-09-21T21:41:24.413 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:24.413 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:24.413 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:24.413 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.413 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.413 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4840.0:4980 3.10 3:0d9e118e:::rbd_data.11214ff8bd9b.000000000000001c:10 [read 0~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T21:41:24.413 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:16.474441+0000", 2024-09-21T21:41:24.413 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 7.8808984850000003, 2024-09-21T21:41:24.413 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.016936824999999999, 2024-09-21T21:41:24.413 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:24.414 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:24.414 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:24.414 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:24.414 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4840", 2024-09-21T21:41:24.414 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1293784138", 2024-09-21T21:41:24.414 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4980 2024-09-21T21:41:24.414 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.414 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:24.414 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.414 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:24.414 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.474441+0000", 2024-09-21T21:41:24.414 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:24.414 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.414 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.414 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:24.415 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.474441+0000", 2024-09-21T21:41:24.415 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:24.415 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.415 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.415 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:24.415 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.474441+0000", 2024-09-21T21:41:24.415 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.77e-07 2024-09-21T21:41:24.415 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.415 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.415 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:24.415 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.474443+0000", 2024-09-21T21:41:24.415 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.885e-06 2024-09-21T21:41:24.415 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.415 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.415 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:24.416 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.474449+0000", 2024-09-21T21:41:24.416 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6180000000000003e-06 2024-09-21T21:41:24.416 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.416 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.416 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:24.416 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.474453+0000", 2024-09-21T21:41:24.416 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3420000000000001e-06 2024-09-21T21:41:24.416 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.416 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.416 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:24.416 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.491285+0000", 2024-09-21T21:41:24.416 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.016831723 2024-09-21T21:41:24.416 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.417 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.417 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:24.417 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.491320+0000", 2024-09-21T21:41:24.417 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5366e-05 2024-09-21T21:41:24.417 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.417 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.417 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:24.417 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.491378+0000", 2024-09-21T21:41:24.417 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7513999999999999e-05 2024-09-21T21:41:24.417 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:24.417 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:24.417 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:24.417 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.418 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.418 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4840.0:4981 3.10 3:0d9e118e:::rbd_data.11214ff8bd9b.000000000000001c:10 [read 12288~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T21:41:24.418 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:16.474458+0000", 2024-09-21T21:41:24.418 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 7.880881273, 2024-09-21T21:41:24.418 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.017120076000000001, 2024-09-21T21:41:24.418 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:24.418 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:24.418 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:24.418 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:24.418 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4840", 2024-09-21T21:41:24.418 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1293784138", 2024-09-21T21:41:24.418 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4981 2024-09-21T21:41:24.418 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.419 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:24.419 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.419 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:24.419 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.474458+0000", 2024-09-21T21:41:24.419 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:24.419 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.419 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.419 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:24.419 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.474458+0000", 2024-09-21T21:41:24.419 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:24.419 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.419 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.419 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:24.420 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.474458+0000", 2024-09-21T21:41:24.420 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.2899999999999999e-07 2024-09-21T21:41:24.420 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.420 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.420 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:24.420 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.474460+0000", 2024-09-21T21:41:24.420 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.75e-06 2024-09-21T21:41:24.420 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.420 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.420 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:24.420 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.474466+0000", 2024-09-21T21:41:24.420 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5790000000000003e-06 2024-09-21T21:41:24.420 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.420 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.421 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:24.421 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.474470+0000", 2024-09-21T21:41:24.421 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4479999999999996e-06 2024-09-21T21:41:24.421 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.421 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.421 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:24.421 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.491414+0000", 2024-09-21T21:41:24.421 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.016943940000000001 2024-09-21T21:41:24.421 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.421 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.421 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:24.421 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.491486+0000", 2024-09-21T21:41:24.421 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.1849999999999998e-05 2024-09-21T21:41:24.422 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.422 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.422 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:24.422 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.491578+0000", 2024-09-21T21:41:24.422 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2180000000000002e-05 2024-09-21T21:41:24.422 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:24.422 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:24.422 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:24.422 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.422 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.422 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4840.0:4982 3.10 3:0d9e118e:::rbd_data.11214ff8bd9b.000000000000001c:10 [read 24576~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T21:41:24.422 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:16.474475+0000", 2024-09-21T21:41:24.422 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 7.8808642149999999, 2024-09-21T21:41:24.422 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.017216813000000001, 2024-09-21T21:41:24.422 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:24.423 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:24.423 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:24.423 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:24.423 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4840", 2024-09-21T21:41:24.423 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1293784138", 2024-09-21T21:41:24.423 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4982 2024-09-21T21:41:24.423 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.423 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:24.423 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.423 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:24.423 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.474475+0000", 2024-09-21T21:41:24.423 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:24.423 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.423 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.424 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:24.424 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.474475+0000", 2024-09-21T21:41:24.424 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:24.424 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.424 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.424 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:24.424 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.474475+0000", 2024-09-21T21:41:24.424 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.2899999999999999e-07 2024-09-21T21:41:24.424 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.424 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.424 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:24.424 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.474477+0000", 2024-09-21T21:41:24.424 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.86e-06 2024-09-21T21:41:24.424 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.425 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.425 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:24.425 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.474483+0000", 2024-09-21T21:41:24.425 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7880000000000003e-06 2024-09-21T21:41:24.425 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.425 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.425 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:24.425 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.474487+0000", 2024-09-21T21:41:24.425 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2919999999999997e-06 2024-09-21T21:41:24.425 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.425 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.425 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:24.425 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.491607+0000", 2024-09-21T21:41:24.425 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0171193 2024-09-21T21:41:24.426 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.426 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.426 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:24.426 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.491642+0000", 2024-09-21T21:41:24.426 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5120000000000003e-05 2024-09-21T21:41:24.426 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.426 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.426 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:24.426 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.491692+0000", 2024-09-21T21:41:24.426 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0124000000000003e-05 2024-09-21T21:41:24.426 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:24.426 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:24.426 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:24.426 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.427 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.427 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4840.0:4983 3.10 3:0d9e118e:::rbd_data.11214ff8bd9b.000000000000001c:10 [read 36864~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T21:41:24.427 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:16.474493+0000", 2024-09-21T21:41:24.427 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 7.880846869, 2024-09-21T21:41:24.427 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.017492733999999999, 2024-09-21T21:41:24.427 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:24.427 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:24.427 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:24.427 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:24.427 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4840", 2024-09-21T21:41:24.427 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1293784138", 2024-09-21T21:41:24.427 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4983 2024-09-21T21:41:24.427 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.427 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:24.428 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.428 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:24.428 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.474493+0000", 2024-09-21T21:41:24.428 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:24.428 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.428 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.428 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:24.428 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.474493+0000", 2024-09-21T21:41:24.428 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:24.428 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.428 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.428 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:24.428 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.474493+0000", 2024-09-21T21:41:24.428 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.7599999999999998e-07 2024-09-21T21:41:24.429 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.429 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.429 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:24.429 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.474495+0000", 2024-09-21T21:41:24.429 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.72e-06 2024-09-21T21:41:24.429 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.429 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.429 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:24.429 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.474527+0000", 2024-09-21T21:41:24.429 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.2756999999999998e-05 2024-09-21T21:41:24.429 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.429 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.429 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:24.430 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.474532+0000", 2024-09-21T21:41:24.430 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7790000000000003e-06 2024-09-21T21:41:24.430 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.430 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.430 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:24.430 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.491729+0000", 2024-09-21T21:41:24.430 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.017196626999999999 2024-09-21T21:41:24.430 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.430 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.430 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:24.430 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.491802+0000", 2024-09-21T21:41:24.430 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.3180000000000001e-05 2024-09-21T21:41:24.430 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.430 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.430 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:24.431 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.491985+0000", 2024-09-21T21:41:24.431 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00018339499999999999 2024-09-21T21:41:24.431 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:24.431 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:24.431 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:24.431 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.431 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.431 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4840.0:4984 3.10 3:0d9e118e:::rbd_data.11214ff8bd9b.000000000000001c:10 [read 49152~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T21:41:24.431 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:16.474538+0000", 2024-09-21T21:41:24.431 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 7.8808009089999995, 2024-09-21T21:41:24.431 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.017558496, 2024-09-21T21:41:24.431 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:24.431 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:24.431 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:24.432 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:24.432 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4840", 2024-09-21T21:41:24.432 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1293784138", 2024-09-21T21:41:24.432 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4984 2024-09-21T21:41:24.432 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.432 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:24.432 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.432 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:24.432 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.474538+0000", 2024-09-21T21:41:24.432 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:24.432 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.432 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.432 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:24.432 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.474538+0000", 2024-09-21T21:41:24.433 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:24.433 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.433 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.433 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:24.433 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.474539+0000", 2024-09-21T21:41:24.433 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8599999999999999e-07 2024-09-21T21:41:24.433 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.433 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.433 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:24.433 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.474541+0000", 2024-09-21T21:41:24.433 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.385e-06 2024-09-21T21:41:24.433 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.433 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.434 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:24.434 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.474547+0000", 2024-09-21T21:41:24.434 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7649999999999999e-06 2024-09-21T21:41:24.434 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.434 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.434 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:24.434 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.474551+0000", 2024-09-21T21:41:24.434 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2370000000000003e-06 2024-09-21T21:41:24.434 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.434 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.434 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:24.434 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.492010+0000", 2024-09-21T21:41:24.434 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.017458530999999999 2024-09-21T21:41:24.434 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.435 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.435 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:24.435 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.492047+0000", 2024-09-21T21:41:24.435 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6922000000000003e-05 2024-09-21T21:41:24.435 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.435 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.435 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:24.435 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.492097+0000", 2024-09-21T21:41:24.435 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0269999999999998e-05 2024-09-21T21:41:24.435 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:24.435 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:24.435 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:24.435 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.435 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.436 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4840.0:4985 3.10 3:0d9e118e:::rbd_data.11214ff8bd9b.000000000000001c:10 [read 61440~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T21:41:24.436 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:16.474556+0000", 2024-09-21T21:41:24.436 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 7.8807829080000005, 2024-09-21T21:41:24.436 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.017654330999999999, 2024-09-21T21:41:24.436 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:24.436 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:24.436 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:24.436 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:24.436 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4840", 2024-09-21T21:41:24.436 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1293784138", 2024-09-21T21:41:24.436 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4985 2024-09-21T21:41:24.436 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.436 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:24.436 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.436 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:24.437 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.474556+0000", 2024-09-21T21:41:24.437 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:24.437 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.437 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.437 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:24.437 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.474556+0000", 2024-09-21T21:41:24.437 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:24.437 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.437 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.437 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:24.437 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.474557+0000", 2024-09-21T21:41:24.437 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3099999999999999e-07 2024-09-21T21:41:24.437 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.437 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.438 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:24.438 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.474559+0000", 2024-09-21T21:41:24.438 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.902e-06 2024-09-21T21:41:24.438 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.438 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.438 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:24.438 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.474564+0000", 2024-09-21T21:41:24.438 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5310000000000001e-06 2024-09-21T21:41:24.438 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.438 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.438 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:24.438 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.474569+0000", 2024-09-21T21:41:24.438 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3080000000000001e-06 2024-09-21T21:41:24.438 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.438 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.439 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:24.439 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.492121+0000", 2024-09-21T21:41:24.439 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.017552697999999999 2024-09-21T21:41:24.439 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.439 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.439 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:24.439 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.492160+0000", 2024-09-21T21:41:24.439 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8603999999999998e-05 2024-09-21T21:41:24.439 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.439 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.439 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:24.439 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.492211+0000", 2024-09-21T21:41:24.439 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0957e-05 2024-09-21T21:41:24.439 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:24.440 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:24.440 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:24.440 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.440 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.440 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4840.0:4986 3.10 3:0d9e118e:::rbd_data.11214ff8bd9b.000000000000001c:10 [read 73728~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T21:41:24.440 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:16.474574+0000", 2024-09-21T21:41:24.440 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 7.8807657970000005, 2024-09-21T21:41:24.440 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.017743294, 2024-09-21T21:41:24.440 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:24.440 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:24.440 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:24.440 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:24.440 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4840", 2024-09-21T21:41:24.440 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1293784138", 2024-09-21T21:41:24.441 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4986 2024-09-21T21:41:24.441 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.441 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:24.441 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.441 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:24.441 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.474574+0000", 2024-09-21T21:41:24.441 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:24.441 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.441 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.441 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:24.441 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.474574+0000", 2024-09-21T21:41:24.441 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:24.441 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.441 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.441 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:24.442 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.474574+0000", 2024-09-21T21:41:24.442 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.1699999999999999e-07 2024-09-21T21:41:24.442 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.442 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.442 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:24.442 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.474576+0000", 2024-09-21T21:41:24.442 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.933e-06 2024-09-21T21:41:24.442 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.442 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.442 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:24.442 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.474581+0000", 2024-09-21T21:41:24.442 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6200000000000004e-06 2024-09-21T21:41:24.442 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.442 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.443 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:24.443 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.474586+0000", 2024-09-21T21:41:24.443 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3919999999999996e-06 2024-09-21T21:41:24.443 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.443 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.443 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:24.443 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.492233+0000", 2024-09-21T21:41:24.443 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.017647164999999999 2024-09-21T21:41:24.443 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.443 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.443 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:24.443 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.492268+0000", 2024-09-21T21:41:24.443 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4662000000000001e-05 2024-09-21T21:41:24.443 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.444 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.444 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:24.444 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:16.492317+0000", 2024-09-21T21:41:24.444 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9205000000000001e-05 2024-09-21T21:41:24.444 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:24.444 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:24.444 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:24.444 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.444 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.444 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4912.0:1056 3.1 3:828a773c:::rbd_data.1121b3a54d7e.0000000000000006:12 [sparse-read 15466496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-21T21:41:24.444 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:20.723563+0000", 2024-09-21T21:41:24.444 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.6317766200000001, 2024-09-21T21:41:24.444 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.016467641000000002, 2024-09-21T21:41:24.444 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:24.445 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:24.445 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:24.445 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:24.445 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4912", 2024-09-21T21:41:24.445 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4204709190", 2024-09-21T21:41:24.445 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1056 2024-09-21T21:41:24.445 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.445 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:24.445 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.445 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:24.445 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723563+0000", 2024-09-21T21:41:24.445 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:24.445 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.445 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.445 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:24.446 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723563+0000", 2024-09-21T21:41:24.446 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:24.446 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.446 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.446 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:24.446 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723563+0000", 2024-09-21T21:41:24.446 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2300000000000002e-07 2024-09-21T21:41:24.446 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.446 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.446 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:24.446 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723565+0000", 2024-09-21T21:41:24.446 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9570000000000001e-06 2024-09-21T21:41:24.446 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.446 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.446 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:24.447 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723572+0000", 2024-09-21T21:41:24.447 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.8020000000000003e-06 2024-09-21T21:41:24.447 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.447 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.447 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:24.447 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723577+0000", 2024-09-21T21:41:24.447 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0529999999999997e-06 2024-09-21T21:41:24.447 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.447 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.447 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:24.447 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.739864+0000", 2024-09-21T21:41:24.447 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.016287329 2024-09-21T21:41:24.447 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.448 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.448 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:24.448 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.739944+0000", 2024-09-21T21:41:24.448 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0038999999999997e-05 2024-09-21T21:41:24.448 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.448 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.448 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:24.448 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.740030+0000", 2024-09-21T21:41:24.448 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.6038000000000005e-05 2024-09-21T21:41:24.448 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:24.448 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:24.448 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:24.448 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:24.448 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:24.448 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:24.450 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:24.447+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:24.451 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T21:41:24.452 INFO:teuthology.orchestra.run.smithi164.stdout:2862 read 0x6a78c00 thru 0x6a7bb7d (0x2f7e bytes) 2024-09-21T21:41:24.452 INFO:teuthology.orchestra.run.smithi164.stdout:2867 read 0x8985200 thru 0x8989aa2 (0x48a3 bytes) 2024-09-21T21:41:24.452 INFO:teuthology.orchestra.run.smithi164.stdout:2868 write 0x4137a00 thru 0x413f1b3 (0x77b4 bytes) 2024-09-21T21:41:24.452 INFO:teuthology.orchestra.run.smithi164.stdout:2871 punch from 0x9b49400 to 0x9b4c200, (0x2e00 bytes) 2024-09-21T21:41:24.452 INFO:teuthology.orchestra.run.smithi164.stdout:2874 punch from 0x5ff1800 to 0x5ff8e00, (0x7600 bytes) 2024-09-21T21:41:24.452 INFO:teuthology.orchestra.run.smithi164.stdout:2875 punch from 0x4fb7400 to 0x4fbe400, (0x7000 bytes) 2024-09-21T21:41:24.452 INFO:teuthology.orchestra.run.smithi164.stdout:2876 read 0xab1c00 thru 0xabb29e (0x969f bytes) 2024-09-21T21:41:24.452 INFO:teuthology.orchestra.run.smithi164.stdout:2878 read 0x814fe00 thru 0x81566ca (0x68cb bytes) 2024-09-21T21:41:24.452 INFO:teuthology.orchestra.run.smithi164.stdout:2880 write 0xc184c00 thru 0xc1919d7 (0xcdd8 bytes) 2024-09-21T21:41:24.453 INFO:teuthology.orchestra.run.smithi164.stdout:2881 write 0xe177a00 thru 0xe18315e (0xb75f bytes) 2024-09-21T21:41:24.453 INFO:teuthology.orchestra.run.smithi164.stdout:2882 punch from 0x6076800 to 0x6077400, (0xc00 bytes) 2024-09-21T21:41:24.453 INFO:teuthology.orchestra.run.smithi164.stdout:2883 trunc from 0xe183200 to 0xd1b1600 2024-09-21T21:41:24.453 INFO:teuthology.orchestra.run.smithi164.stdout:2885 read 0x7f8400 thru 0x7f9840 (0x1441 bytes) 2024-09-21T21:41:24.453 INFO:teuthology.orchestra.run.smithi164.stdout:2886 write 0xd8b1200 thru 0xd8be42d (0xd22e bytes) 2024-09-21T21:41:24.461 INFO:teuthology.orchestra.run.smithi164.stdout:2890 write 0x6931a00 thru 0x6934d26 (0x3327 bytes) 2024-09-21T21:41:24.461 INFO:teuthology.orchestra.run.smithi164.stdout:2891 read 0x653d600 thru 0x6546d4d (0x974e bytes) 2024-09-21T21:41:24.462 INFO:teuthology.orchestra.run.smithi164.stdout:2892 write 0x2c7a800 thru 0x2c7a9c4 (0x1c5 bytes) 2024-09-21T21:41:24.462 INFO:teuthology.orchestra.run.smithi164.stdout:2893 read 0x457c400 thru 0x458563a (0x923b bytes) 2024-09-21T21:41:24.463 INFO:teuthology.orchestra.run.smithi164.stdout:2894 write 0xa2db600 thru 0xa2dce35 (0x1836 bytes) 2024-09-21T21:41:24.463 INFO:teuthology.orchestra.run.smithi164.stdout:2896 write 0xb933e00 thru 0xb93480c (0xa0d bytes) 2024-09-21T21:41:24.464 INFO:teuthology.orchestra.run.smithi164.stdout:2897 write 0x226b400 thru 0x2277b87 (0xc788 bytes) 2024-09-21T21:41:24.465 INFO:teuthology.orchestra.run.smithi164.stdout:2899 punch from 0x2764a00 to 0x2771600, (0xcc00 bytes) 2024-09-21T21:41:24.466 INFO:teuthology.orchestra.run.smithi164.stdout:2900 read 0x983c200 thru 0x9843cda (0x7adb bytes) 2024-09-21T21:41:24.469 INFO:teuthology.orchestra.run.smithi164.stdout:2901 read 0x6bf1c00 thru 0x6bfef9b (0xd39c bytes) 2024-09-21T21:41:24.470 INFO:teuthology.orchestra.run.smithi164.stdout:2904 write 0x394c600 thru 0x394ca06 (0x407 bytes) 2024-09-21T21:41:24.471 INFO:teuthology.orchestra.run.smithi164.stdout:2905 read 0x74bb800 thru 0x74c81eb (0xc9ec bytes) 2024-09-21T21:41:24.471 INFO:teuthology.orchestra.run.smithi164.stdout:2906 read 0x3e75a00 thru 0x3e79a70 (0x4071 bytes) 2024-09-21T21:41:24.484 INFO:teuthology.orchestra.run.smithi164.stdout:2907 trunc from 0xd8be600 to 0x37a4800 2024-09-21T21:41:24.502 INFO:teuthology.orchestra.run.smithi164.stdout:2908 read 0x17da200 thru 0x17df313 (0x5114 bytes) 2024-09-21T21:41:24.510 INFO:teuthology.orchestra.run.smithi164.stdout:2909 read 0x2bfa800 thru 0x2c08dbc (0xe5bd bytes) 2024-09-21T21:41:24.515 INFO:teuthology.orchestra.run.smithi164.stdout:2911 trunc from 0x37a4800 to 0x722a800 2024-09-21T21:41:24.547 INFO:teuthology.orchestra.run.smithi164.stdout:2912 punch from 0x6daec00 to 0x6db5a00, (0x6e00 bytes) 2024-09-21T21:41:24.550 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:24.549+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:24.559 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:24.559 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:41:24.559 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 0 2024-09-21T21:41:24.559 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:24.560 INFO:teuthology.orchestra.run.smithi164.stdout:2914 trunc from 0x722a800 to 0x17c4800 2024-09-21T21:41:24.568 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T21:41:24.597 INFO:teuthology.orchestra.run.smithi164.stdout:2915 punch from 0x505400 to 0x505a00, (0x600 bytes) 2024-09-21T21:41:24.618 INFO:teuthology.orchestra.run.smithi164.stdout:2917 write 0x5fd0800 thru 0x5fd4691 (0x3e92 bytes) 2024-09-21T21:41:24.625 INFO:teuthology.orchestra.run.smithi164.stdout:2918 read 0x1935200 thru 0x193c615 (0x7416 bytes) 2024-09-21T21:41:24.642 INFO:teuthology.orchestra.run.smithi164.stdout:2922 write 0xccac200 thru 0xccb38e9 (0x76ea bytes) 2024-09-21T21:41:24.644 INFO:teuthology.orchestra.run.smithi081.stderr:noscrub is set 2024-09-21T21:41:24.648 INFO:teuthology.orchestra.run.smithi081.stderr:set osd.2 primary-affinity to 0 (802) 2024-09-21T21:41:24.652 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:24.651+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:24.722 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:24.722 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:41:24.722 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:41:24.722 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:41:24.722 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:24.730 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T21:41:24.775 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:24.774+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:24.835 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:24.835 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 20, 2024-09-21T21:41:24.835 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 600, 2024-09-21T21:41:24.835 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:41:24.835 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.835 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4824.0:1564 3.2 3:4647520c:::rbd_data.1121b97f2925.0000000000000006:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:24.835 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:41:24.835 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 23.196828742000001, 2024-09-21T21:41:24.835 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056097347999999998, 2024-09-21T21:41:24.836 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:24.836 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:24.836 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:24.836 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:24.836 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4824", 2024-09-21T21:41:24.836 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:24.836 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1564 2024-09-21T21:41:24.836 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.836 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:24.836 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.836 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:24.836 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:41:24.836 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:24.836 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.837 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.837 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:24.837 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:41:24.837 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:24.837 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.837 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.837 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:24.837 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:41:24.837 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6499999999999999e-07 2024-09-21T21:41:24.837 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.837 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.837 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:24.837 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636067+0000", 2024-09-21T21:41:24.837 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3879999999999998e-06 2024-09-21T21:41:24.837 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.837 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.838 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:24.838 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636075+0000", 2024-09-21T21:41:24.838 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5480000000000004e-06 2024-09-21T21:41:24.838 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.838 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.838 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:24.838 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636080+0000", 2024-09-21T21:41:24.838 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.339e-06 2024-09-21T21:41:24.838 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.838 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.838 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:24.838 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636133+0000", 2024-09-21T21:41:24.838 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2930000000000003e-05 2024-09-21T21:41:24.838 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.839 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.839 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:24.839 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636265+0000", 2024-09-21T21:41:24.839 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000132058 2024-09-21T21:41:24.839 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.839 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.839 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:24.839 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.692162+0000", 2024-09-21T21:41:24.839 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055896620000000001 2024-09-21T21:41:24.839 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:24.839 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:24.839 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:24.839 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.839 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.839 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4912.0:869 3.19 3:9ddcaed2:::rbd_data.1121b3a54d7e.0000000000000000:12 [sparse-read 12713984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-21T21:41:24.840 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:20.721256+0000", 2024-09-21T21:41:24.840 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 4.11163743, 2024-09-21T21:41:24.840 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.026073025, 2024-09-21T21:41:24.840 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:24.840 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:24.840 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:24.840 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:24.840 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4912", 2024-09-21T21:41:24.840 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4204709190", 2024-09-21T21:41:24.840 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 869 2024-09-21T21:41:24.840 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.840 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:24.840 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.840 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:24.840 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.721256+0000", 2024-09-21T21:41:24.840 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:24.841 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.841 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.841 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:24.841 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.721256+0000", 2024-09-21T21:41:24.841 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:24.841 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.841 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.841 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:24.841 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.721256+0000", 2024-09-21T21:41:24.841 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5999999999999999e-07 2024-09-21T21:41:24.841 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.841 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.841 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:24.841 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.721258+0000", 2024-09-21T21:41:24.841 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.8679999999999999e-06 2024-09-21T21:41:24.842 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.842 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.842 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:24.842 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.721264+0000", 2024-09-21T21:41:24.842 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.57e-06 2024-09-21T21:41:24.842 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.842 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.842 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:24.842 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.721268+0000", 2024-09-21T21:41:24.842 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6190000000000002e-06 2024-09-21T21:41:24.842 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.842 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.842 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:24.842 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.747160+0000", 2024-09-21T21:41:24.842 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.025892043999999999 2024-09-21T21:41:24.843 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.843 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.843 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:24.843 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.747237+0000", 2024-09-21T21:41:24.843 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7130000000000002e-05 2024-09-21T21:41:24.843 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.843 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.843 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:24.843 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.747329+0000", 2024-09-21T21:41:24.843 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.1434000000000006e-05 2024-09-21T21:41:24.843 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:24.843 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:24.843 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:24.843 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.843 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.844 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4912.0:870 3.19 3:9ddcaed2:::rbd_data.1121b3a54d7e.0000000000000000:12 [sparse-read 14417920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-21T21:41:24.844 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:20.721274+0000", 2024-09-21T21:41:24.844 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 4.111619342, 2024-09-21T21:41:24.844 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.026299247000000001, 2024-09-21T21:41:24.844 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:24.844 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:24.844 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:24.844 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:24.844 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4912", 2024-09-21T21:41:24.844 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4204709190", 2024-09-21T21:41:24.844 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 870 2024-09-21T21:41:24.844 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.844 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:24.845 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.845 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:24.845 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.721274+0000", 2024-09-21T21:41:24.845 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:24.845 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.845 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.845 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:24.845 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.721274+0000", 2024-09-21T21:41:24.845 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:24.845 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.845 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.845 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:24.845 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.721274+0000", 2024-09-21T21:41:24.846 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5499999999999998e-07 2024-09-21T21:41:24.846 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.846 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.846 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:24.846 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.721276+0000", 2024-09-21T21:41:24.846 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.871e-06 2024-09-21T21:41:24.846 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.846 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.846 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:24.846 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.721282+0000", 2024-09-21T21:41:24.846 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9549999999999997e-06 2024-09-21T21:41:24.846 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.846 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.846 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:24.847 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.721287+0000", 2024-09-21T21:41:24.847 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7729999999999999e-06 2024-09-21T21:41:24.847 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.847 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.847 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:24.847 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.747372+0000", 2024-09-21T21:41:24.847 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.026084746999999998 2024-09-21T21:41:24.847 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.847 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.847 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:24.847 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.747464+0000", 2024-09-21T21:41:24.847 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2326999999999999e-05 2024-09-21T21:41:24.847 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.847 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.847 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:24.848 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.747573+0000", 2024-09-21T21:41:24.848 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000109119 2024-09-21T21:41:24.848 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:24.848 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:24.848 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:24.848 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.848 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.848 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4912.0:871 3.19 3:9ddcaed2:::rbd_data.1121b3a54d7e.0000000000000000:12 [sparse-read 16121856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-21T21:41:24.848 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:20.721293+0000", 2024-09-21T21:41:24.848 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 4.1116007870000004, 2024-09-21T21:41:24.848 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.026485022, 2024-09-21T21:41:24.848 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:24.848 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:24.848 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:24.848 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:24.849 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4912", 2024-09-21T21:41:24.849 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4204709190", 2024-09-21T21:41:24.849 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 871 2024-09-21T21:41:24.849 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.849 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:24.849 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.849 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:24.849 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.721293+0000", 2024-09-21T21:41:24.849 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:24.849 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.849 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.849 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:24.849 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.721293+0000", 2024-09-21T21:41:24.849 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:24.850 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.850 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.850 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:24.850 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.721293+0000", 2024-09-21T21:41:24.850 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3000000000000002e-07 2024-09-21T21:41:24.850 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.850 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.850 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:24.850 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.721295+0000", 2024-09-21T21:41:24.850 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.7749999999999999e-06 2024-09-21T21:41:24.850 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.850 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.850 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:24.850 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.721300+0000", 2024-09-21T21:41:24.851 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3319999999999999e-06 2024-09-21T21:41:24.851 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.851 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.851 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:24.851 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.721305+0000", 2024-09-21T21:41:24.851 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.578e-06 2024-09-21T21:41:24.851 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.851 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.851 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:24.851 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.747610+0000", 2024-09-21T21:41:24.851 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.026305469000000001 2024-09-21T21:41:24.852 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.852 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.852 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:24.852 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.747691+0000", 2024-09-21T21:41:24.852 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.1390999999999994e-05 2024-09-21T21:41:24.852 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.852 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.852 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:24.852 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.747778+0000", 2024-09-21T21:41:24.852 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.6147e-05 2024-09-21T21:41:24.852 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:24.852 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:24.852 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:24.852 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.853 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.853 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4912.0:935 3.19 3:9ddcaed2:::rbd_data.1121b3a54d7e.0000000000000000:12 [sparse-read 8126464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-21T21:41:24.853 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:20.723242+0000", 2024-09-21T21:41:24.853 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 4.1096517710000002, 2024-09-21T21:41:24.853 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.025925357999999999, 2024-09-21T21:41:24.853 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:24.853 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:24.853 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:24.853 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:24.853 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4912", 2024-09-21T21:41:24.853 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4204709190", 2024-09-21T21:41:24.853 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 935 2024-09-21T21:41:24.853 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.853 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:24.854 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.854 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:24.854 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723242+0000", 2024-09-21T21:41:24.854 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:24.854 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.854 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.854 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:24.854 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723242+0000", 2024-09-21T21:41:24.854 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:24.854 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.854 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.854 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:24.854 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723242+0000", 2024-09-21T21:41:24.854 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4900000000000001e-07 2024-09-21T21:41:24.854 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.855 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.855 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:24.855 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723244+0000", 2024-09-21T21:41:24.855 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2189999999999998e-06 2024-09-21T21:41:24.855 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.855 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.855 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:24.855 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723250+0000", 2024-09-21T21:41:24.855 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9120000000000003e-06 2024-09-21T21:41:24.855 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.855 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.855 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:24.855 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723256+0000", 2024-09-21T21:41:24.855 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9200000000000001e-06 2024-09-21T21:41:24.856 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.856 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.856 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:24.856 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.748952+0000", 2024-09-21T21:41:24.856 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.025696363 2024-09-21T21:41:24.856 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.856 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.856 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:24.856 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.749056+0000", 2024-09-21T21:41:24.856 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000104071 2024-09-21T21:41:24.856 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.856 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.856 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:24.856 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.749167+0000", 2024-09-21T21:41:24.856 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000110524 2024-09-21T21:41:24.856 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:24.857 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:24.857 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:24.857 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.857 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.857 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4912.0:936 3.19 3:9ddcaed2:::rbd_data.1121b3a54d7e.0000000000000000:12 [sparse-read 9437184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-21T21:41:24.857 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:20.723262+0000", 2024-09-21T21:41:24.857 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 4.1096309050000004, 2024-09-21T21:41:24.857 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.026110020000000001, 2024-09-21T21:41:24.857 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:24.857 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:24.857 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:24.858 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:24.858 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4912", 2024-09-21T21:41:24.858 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4204709190", 2024-09-21T21:41:24.858 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 936 2024-09-21T21:41:24.858 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.858 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:24.858 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.858 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:24.858 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723262+0000", 2024-09-21T21:41:24.858 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:24.858 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.858 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.858 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:24.858 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723262+0000", 2024-09-21T21:41:24.859 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:24.859 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.859 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.859 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:24.859 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723263+0000", 2024-09-21T21:41:24.859 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.9000000000000002e-07 2024-09-21T21:41:24.859 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.859 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.859 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:24.859 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723265+0000", 2024-09-21T21:41:24.859 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1100000000000001e-06 2024-09-21T21:41:24.859 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.859 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.859 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:24.859 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723271+0000", 2024-09-21T21:41:24.860 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1020000000000002e-06 2024-09-21T21:41:24.860 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.860 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.860 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:24.860 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723276+0000", 2024-09-21T21:41:24.860 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8400000000000002e-06 2024-09-21T21:41:24.860 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.860 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.860 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:24.860 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.749196+0000", 2024-09-21T21:41:24.860 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.025920505 2024-09-21T21:41:24.860 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.860 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.861 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:24.861 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.749277+0000", 2024-09-21T21:41:24.861 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0765000000000003e-05 2024-09-21T21:41:24.861 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.861 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.861 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:24.861 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.749372+0000", 2024-09-21T21:41:24.861 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.5308000000000003e-05 2024-09-21T21:41:24.861 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:24.861 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:24.861 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:24.861 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.861 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.861 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4912.0:937 3.19 3:9ddcaed2:::rbd_data.1121b3a54d7e.0000000000000000:12 [sparse-read 11141120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-21T21:41:24.862 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:20.723291+0000", 2024-09-21T21:41:24.862 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 4.1096021159999996, 2024-09-21T21:41:24.862 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.026232482000000001, 2024-09-21T21:41:24.862 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:24.862 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:24.862 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:24.862 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:24.862 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4912", 2024-09-21T21:41:24.862 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4204709190", 2024-09-21T21:41:24.862 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 937 2024-09-21T21:41:24.862 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.862 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:24.862 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.862 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:24.863 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723291+0000", 2024-09-21T21:41:24.863 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:24.863 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.863 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.863 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:24.863 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723291+0000", 2024-09-21T21:41:24.863 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:24.863 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.863 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.863 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:24.863 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723292+0000", 2024-09-21T21:41:24.863 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7399999999999999e-07 2024-09-21T21:41:24.863 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.863 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.864 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:24.864 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723303+0000", 2024-09-21T21:41:24.864 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1559000000000001e-05 2024-09-21T21:41:24.864 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.864 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.864 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:24.864 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723310+0000", 2024-09-21T21:41:24.864 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.878e-06 2024-09-21T21:41:24.864 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.864 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.864 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:24.864 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723315+0000", 2024-09-21T21:41:24.865 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3009999999999997e-06 2024-09-21T21:41:24.865 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.865 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.865 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:24.865 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.749408+0000", 2024-09-21T21:41:24.865 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.026092256000000001 2024-09-21T21:41:24.865 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.865 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.865 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:24.865 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.749461+0000", 2024-09-21T21:41:24.865 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3099000000000001e-05 2024-09-21T21:41:24.865 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.865 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.865 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:24.866 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.749524+0000", 2024-09-21T21:41:24.866 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3015000000000006e-05 2024-09-21T21:41:24.866 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:24.866 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:24.866 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:24.866 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.866 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.866 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4912.0:938 3.19 3:9ddcaed2:::rbd_data.1121b3a54d7e.0000000000000000:12 [sparse-read 12845056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-21T21:41:24.866 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:20.723322+0000", 2024-09-21T21:41:24.866 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 4.1095711760000002, 2024-09-21T21:41:24.866 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.026387079000000001, 2024-09-21T21:41:24.866 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:24.866 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:24.867 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:24.867 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:24.867 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4912", 2024-09-21T21:41:24.867 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4204709190", 2024-09-21T21:41:24.867 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 938 2024-09-21T21:41:24.867 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.867 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:24.867 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.867 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:24.867 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723322+0000", 2024-09-21T21:41:24.867 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:24.867 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.867 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.867 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:24.868 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723322+0000", 2024-09-21T21:41:24.868 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:24.868 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.868 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.868 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:24.868 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723322+0000", 2024-09-21T21:41:24.868 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.72e-07 2024-09-21T21:41:24.868 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.868 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.868 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:24.869 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723324+0000", 2024-09-21T21:41:24.869 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9250000000000002e-06 2024-09-21T21:41:24.869 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.869 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.869 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:24.869 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723330+0000", 2024-09-21T21:41:24.869 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6799999999999998e-06 2024-09-21T21:41:24.869 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.869 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.869 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:24.869 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723334+0000", 2024-09-21T21:41:24.869 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3640000000000001e-06 2024-09-21T21:41:24.869 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.870 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.870 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:24.870 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.749545+0000", 2024-09-21T21:41:24.870 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.026210715999999998 2024-09-21T21:41:24.870 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.870 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.870 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:24.870 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.749623+0000", 2024-09-21T21:41:24.870 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7602999999999995e-05 2024-09-21T21:41:24.870 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.870 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.870 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:24.870 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.749709+0000", 2024-09-21T21:41:24.870 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.6419e-05 2024-09-21T21:41:24.870 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:24.871 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:24.871 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:24.871 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.871 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.871 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4912.0:939 3.19 3:9ddcaed2:::rbd_data.1121b3a54d7e.0000000000000000:12 [sparse-read 14548992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-21T21:41:24.871 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:20.723340+0000", 2024-09-21T21:41:24.871 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 4.109552978, 2024-09-21T21:41:24.871 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.026505974000000002, 2024-09-21T21:41:24.871 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:24.871 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:24.871 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:24.871 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:24.871 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4912", 2024-09-21T21:41:24.871 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4204709190", 2024-09-21T21:41:24.872 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 939 2024-09-21T21:41:24.872 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.872 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:24.872 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.872 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:24.872 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723340+0000", 2024-09-21T21:41:24.872 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:24.872 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.872 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.872 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:24.872 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723340+0000", 2024-09-21T21:41:24.872 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:24.872 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.872 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.872 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:24.873 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723341+0000", 2024-09-21T21:41:24.873 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6100000000000002e-07 2024-09-21T21:41:24.873 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.873 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.873 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:24.873 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723342+0000", 2024-09-21T21:41:24.873 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.782e-06 2024-09-21T21:41:24.873 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.873 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.873 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:24.873 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723348+0000", 2024-09-21T21:41:24.873 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3990000000000003e-06 2024-09-21T21:41:24.873 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.873 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.873 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:24.874 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723353+0000", 2024-09-21T21:41:24.874 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3009999999999997e-06 2024-09-21T21:41:24.874 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.874 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.874 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:24.874 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.749741+0000", 2024-09-21T21:41:24.874 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.026387911999999999 2024-09-21T21:41:24.874 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.874 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.874 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:24.874 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.749788+0000", 2024-09-21T21:41:24.874 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7354000000000001e-05 2024-09-21T21:41:24.874 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.874 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.874 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:24.875 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.749846+0000", 2024-09-21T21:41:24.875 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7865000000000002e-05 2024-09-21T21:41:24.875 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:24.875 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:24.875 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:24.875 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.875 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.875 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4912.0:940 3.19 3:9ddcaed2:::rbd_data.1121b3a54d7e.0000000000000000:12 [sparse-read 16252928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-21T21:41:24.875 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:20.723360+0000", 2024-09-21T21:41:24.875 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 4.109533763, 2024-09-21T21:41:24.875 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.026697166000000001, 2024-09-21T21:41:24.876 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:24.876 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:24.876 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:24.876 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:24.876 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4912", 2024-09-21T21:41:24.876 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4204709190", 2024-09-21T21:41:24.876 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 940 2024-09-21T21:41:24.876 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.876 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:24.876 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.876 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:24.876 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723360+0000", 2024-09-21T21:41:24.876 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:24.877 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.877 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.877 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:24.877 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723360+0000", 2024-09-21T21:41:24.877 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:24.877 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.877 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.877 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:24.877 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723360+0000", 2024-09-21T21:41:24.877 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.9900000000000001e-07 2024-09-21T21:41:24.877 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.877 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.877 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:24.877 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723362+0000", 2024-09-21T21:41:24.878 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1289999999999999e-06 2024-09-21T21:41:24.878 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.878 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.878 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:24.878 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723368+0000", 2024-09-21T21:41:24.878 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.75e-06 2024-09-21T21:41:24.878 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.878 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.878 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:24.878 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723373+0000", 2024-09-21T21:41:24.878 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6959999999999996e-06 2024-09-21T21:41:24.878 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.879 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.879 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:24.879 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.749882+0000", 2024-09-21T21:41:24.879 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.026509893999999999 2024-09-21T21:41:24.879 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.879 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.879 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:24.879 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.749966+0000", 2024-09-21T21:41:24.879 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.3449999999999996e-05 2024-09-21T21:41:24.879 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.879 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.879 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:24.879 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.750057+0000", 2024-09-21T21:41:24.880 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.0847999999999995e-05 2024-09-21T21:41:24.880 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:24.880 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:24.880 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:24.880 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.880 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.880 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4912.0:999 3.19 3:9ddcaed2:::rbd_data.1121b3a54d7e.0000000000000000:12 [sparse-read 1441792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-21T21:41:24.880 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:20.723784+0000", 2024-09-21T21:41:24.880 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 4.1091088239999998, 2024-09-21T21:41:24.880 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.026500717, 2024-09-21T21:41:24.880 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:24.880 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:24.880 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:24.880 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:24.880 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4912", 2024-09-21T21:41:24.881 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4204709190", 2024-09-21T21:41:24.881 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 999 2024-09-21T21:41:24.881 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.881 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:24.881 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.881 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:24.881 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723784+0000", 2024-09-21T21:41:24.881 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:24.881 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.881 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.881 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:24.881 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723784+0000", 2024-09-21T21:41:24.881 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:24.881 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.882 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.882 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:24.882 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723785+0000", 2024-09-21T21:41:24.882 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4900000000000001e-07 2024-09-21T21:41:24.882 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.882 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.882 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:24.882 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723787+0000", 2024-09-21T21:41:24.882 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9819999999999998e-06 2024-09-21T21:41:24.882 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.882 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.882 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:24.882 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723793+0000", 2024-09-21T21:41:24.882 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.032e-06 2024-09-21T21:41:24.883 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.883 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.883 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:24.883 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723798+0000", 2024-09-21T21:41:24.883 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7099999999999998e-06 2024-09-21T21:41:24.883 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.883 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.883 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:24.883 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.750104+0000", 2024-09-21T21:41:24.883 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.02630658 2024-09-21T21:41:24.883 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.883 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.883 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:24.883 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.750185+0000", 2024-09-21T21:41:24.884 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0345000000000004e-05 2024-09-21T21:41:24.884 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.884 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.884 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:24.884 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.750285+0000", 2024-09-21T21:41:24.884 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010061899999999999 2024-09-21T21:41:24.884 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:24.884 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:24.884 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:24.884 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.884 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.884 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4912.0:1000 3.19 3:9ddcaed2:::rbd_data.1121b3a54d7e.0000000000000000:12 [sparse-read 3145728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-21T21:41:24.884 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:20.723804+0000", 2024-09-21T21:41:24.884 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 4.1090896419999998, 2024-09-21T21:41:24.885 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.026694361, 2024-09-21T21:41:24.885 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:24.885 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:24.885 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:24.885 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:24.885 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4912", 2024-09-21T21:41:24.885 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4204709190", 2024-09-21T21:41:24.885 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1000 2024-09-21T21:41:24.885 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.885 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:24.885 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.885 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:24.885 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723804+0000", 2024-09-21T21:41:24.885 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:24.886 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.886 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.886 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:24.886 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723804+0000", 2024-09-21T21:41:24.886 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:24.886 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.886 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.886 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:24.886 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723804+0000", 2024-09-21T21:41:24.886 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7399999999999999e-07 2024-09-21T21:41:24.886 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.886 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.887 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:24.887 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723811+0000", 2024-09-21T21:41:24.887 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.1149999999999996e-06 2024-09-21T21:41:24.887 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.887 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.887 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:24.887 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723818+0000", 2024-09-21T21:41:24.887 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9779999999999999e-06 2024-09-21T21:41:24.887 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.887 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.887 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:24.887 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723826+0000", 2024-09-21T21:41:24.887 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0600000000000008e-06 2024-09-21T21:41:24.888 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.888 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.888 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:24.888 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.750308+0000", 2024-09-21T21:41:24.888 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.026482316999999998 2024-09-21T21:41:24.888 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.888 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.888 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:24.888 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.750403+0000", 2024-09-21T21:41:24.888 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.4775e-05 2024-09-21T21:41:24.888 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.888 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.889 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:24.889 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.750498+0000", 2024-09-21T21:41:24.889 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.4741999999999995e-05 2024-09-21T21:41:24.889 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:24.889 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:24.889 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:24.889 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.889 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.889 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4912.0:1001 3.19 3:9ddcaed2:::rbd_data.1121b3a54d7e.0000000000000000:12 [sparse-read 4849664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-21T21:41:24.889 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:20.723833+0000", 2024-09-21T21:41:24.889 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 4.1090607849999996, 2024-09-21T21:41:24.889 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.026886792999999999, 2024-09-21T21:41:24.889 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:24.890 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:24.890 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:24.890 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:24.890 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4912", 2024-09-21T21:41:24.890 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4204709190", 2024-09-21T21:41:24.890 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1001 2024-09-21T21:41:24.890 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.890 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:24.890 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.890 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:24.890 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723833+0000", 2024-09-21T21:41:24.890 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:24.890 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.890 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.891 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:24.891 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723833+0000", 2024-09-21T21:41:24.891 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:24.891 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.891 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.891 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:24.891 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723833+0000", 2024-09-21T21:41:24.891 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4099999999999999e-07 2024-09-21T21:41:24.891 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.891 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.891 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:24.891 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723835+0000", 2024-09-21T21:41:24.891 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.968e-06 2024-09-21T21:41:24.892 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.892 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.892 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:24.892 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723841+0000", 2024-09-21T21:41:24.892 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5209999999999999e-06 2024-09-21T21:41:24.892 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.892 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.892 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:24.892 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723846+0000", 2024-09-21T21:41:24.892 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9810000000000003e-06 2024-09-21T21:41:24.892 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.892 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.892 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:24.893 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.750538+0000", 2024-09-21T21:41:24.893 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.026692041 2024-09-21T21:41:24.893 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.893 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.893 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:24.893 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.750625+0000", 2024-09-21T21:41:24.893 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.6706000000000006e-05 2024-09-21T21:41:24.893 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.893 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.893 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:24.893 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.750719+0000", 2024-09-21T21:41:24.893 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.4135000000000006e-05 2024-09-21T21:41:24.893 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:24.894 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:24.894 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:24.894 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.894 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.894 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4912.0:1002 3.19 3:9ddcaed2:::rbd_data.1121b3a54d7e.0000000000000000:12 [sparse-read 6553600~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-21T21:41:24.894 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:20.723853+0000", 2024-09-21T21:41:24.894 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 4.1090406819999998, 2024-09-21T21:41:24.894 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.027244385999999999, 2024-09-21T21:41:24.894 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:24.894 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:24.894 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:24.894 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:24.894 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4912", 2024-09-21T21:41:24.895 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4204709190", 2024-09-21T21:41:24.895 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1002 2024-09-21T21:41:24.895 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.895 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:24.895 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.895 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:24.895 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723853+0000", 2024-09-21T21:41:24.895 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:24.895 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.895 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.895 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:24.895 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723853+0000", 2024-09-21T21:41:24.895 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:24.895 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.896 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.896 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:24.896 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723853+0000", 2024-09-21T21:41:24.896 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6899999999999998e-07 2024-09-21T21:41:24.896 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.896 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.896 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:24.896 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723858+0000", 2024-09-21T21:41:24.896 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0710000000000002e-06 2024-09-21T21:41:24.896 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.896 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.896 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:24.896 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723865+0000", 2024-09-21T21:41:24.896 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.613e-06 2024-09-21T21:41:24.897 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.897 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.897 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:24.897 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723870+0000", 2024-09-21T21:41:24.897 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0350000000000001e-06 2024-09-21T21:41:24.897 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.897 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.897 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:24.897 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.750749+0000", 2024-09-21T21:41:24.897 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.026878808000000001 2024-09-21T21:41:24.897 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.897 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.897 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:24.897 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.750830+0000", 2024-09-21T21:41:24.898 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.1817999999999995e-05 2024-09-21T21:41:24.898 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.898 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.898 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:24.898 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.751097+0000", 2024-09-21T21:41:24.898 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00026667200000000002 2024-09-21T21:41:24.898 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:24.898 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:24.898 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:24.898 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.898 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.898 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4912.0:1003 3.19 3:9ddcaed2:::rbd_data.1121b3a54d7e.0000000000000000:12 [sparse-read 8257536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-21T21:41:24.898 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:20.723876+0000", 2024-09-21T21:41:24.898 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 4.1090169049999998, 2024-09-21T21:41:24.898 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.027396406000000002, 2024-09-21T21:41:24.899 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:24.899 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:24.899 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:24.899 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:24.899 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4912", 2024-09-21T21:41:24.899 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4204709190", 2024-09-21T21:41:24.899 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1003 2024-09-21T21:41:24.899 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.899 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:24.899 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.899 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:24.899 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723876+0000", 2024-09-21T21:41:24.899 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:24.899 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.900 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.900 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:24.900 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723876+0000", 2024-09-21T21:41:24.900 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:24.900 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.900 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.900 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:24.900 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723877+0000", 2024-09-21T21:41:24.900 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5999999999999999e-07 2024-09-21T21:41:24.900 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.900 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.900 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:24.900 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723879+0000", 2024-09-21T21:41:24.900 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1299999999999999e-06 2024-09-21T21:41:24.900 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.901 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.901 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:24.901 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723885+0000", 2024-09-21T21:41:24.901 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9580000000000004e-06 2024-09-21T21:41:24.901 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.901 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.901 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:24.901 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723889+0000", 2024-09-21T21:41:24.901 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4850000000000003e-06 2024-09-21T21:41:24.901 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.901 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.901 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:24.901 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.751125+0000", 2024-09-21T21:41:24.901 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.027236136000000001 2024-09-21T21:41:24.902 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.902 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.902 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:24.902 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.751171+0000", 2024-09-21T21:41:24.902 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5708999999999999e-05 2024-09-21T21:41:24.902 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.902 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.902 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:24.902 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.751273+0000", 2024-09-21T21:41:24.902 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000101628 2024-09-21T21:41:24.902 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:24.902 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:24.902 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:24.902 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.903 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.903 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4912.0:1004 3.19 3:9ddcaed2:::rbd_data.1121b3a54d7e.0000000000000000:12 [sparse-read 9568256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-21T21:41:24.903 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:20.723895+0000", 2024-09-21T21:41:24.903 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 4.1089981959999999, 2024-09-21T21:41:24.903 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.027611696000000002, 2024-09-21T21:41:24.903 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:24.903 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:24.903 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:24.903 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:24.903 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4912", 2024-09-21T21:41:24.903 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4204709190", 2024-09-21T21:41:24.903 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1004 2024-09-21T21:41:24.903 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.903 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:24.904 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.904 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:24.904 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723895+0000", 2024-09-21T21:41:24.904 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:24.904 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.904 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.904 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:24.904 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723895+0000", 2024-09-21T21:41:24.904 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:24.904 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.904 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.904 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:24.904 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723895+0000", 2024-09-21T21:41:24.904 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.1199999999999999e-07 2024-09-21T21:41:24.904 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.905 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.905 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:24.905 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723897+0000", 2024-09-21T21:41:24.905 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.8059999999999999e-06 2024-09-21T21:41:24.905 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.905 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.905 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:24.905 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723903+0000", 2024-09-21T21:41:24.905 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9939999999999997e-06 2024-09-21T21:41:24.905 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.905 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.905 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:24.905 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723908+0000", 2024-09-21T21:41:24.905 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5260000000000004e-06 2024-09-21T21:41:24.906 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.906 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.906 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:24.906 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.751316+0000", 2024-09-21T21:41:24.906 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.027408377000000001 2024-09-21T21:41:24.906 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.906 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.906 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:24.906 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.751402+0000", 2024-09-21T21:41:24.906 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.5830999999999999e-05 2024-09-21T21:41:24.906 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.906 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.906 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:24.906 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.751507+0000", 2024-09-21T21:41:24.907 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010485 2024-09-21T21:41:24.907 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:24.907 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:24.907 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:24.907 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.907 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.907 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4912.0:1005 3.19 3:9ddcaed2:::rbd_data.1121b3a54d7e.0000000000000000:12 [sparse-read 11272192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-21T21:41:24.907 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:20.723914+0000", 2024-09-21T21:41:24.907 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 4.1089794250000002, 2024-09-21T21:41:24.907 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.027789695999999999, 2024-09-21T21:41:24.907 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:24.907 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:24.907 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:24.907 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:24.907 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4912", 2024-09-21T21:41:24.908 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4204709190", 2024-09-21T21:41:24.908 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1005 2024-09-21T21:41:24.908 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.908 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:24.908 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.908 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:24.908 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723914+0000", 2024-09-21T21:41:24.908 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:24.908 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.908 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.908 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:24.908 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723914+0000", 2024-09-21T21:41:24.908 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:24.908 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.909 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.909 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:24.909 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723914+0000", 2024-09-21T21:41:24.909 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4099999999999999e-07 2024-09-21T21:41:24.909 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.909 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.909 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:24.909 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723916+0000", 2024-09-21T21:41:24.909 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0559999999999999e-06 2024-09-21T21:41:24.909 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.909 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.909 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:24.909 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723923+0000", 2024-09-21T21:41:24.909 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.139e-06 2024-09-21T21:41:24.909 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.910 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.910 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:24.910 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723927+0000", 2024-09-21T21:41:24.910 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.826e-06 2024-09-21T21:41:24.910 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.910 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.910 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:24.910 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.751550+0000", 2024-09-21T21:41:24.910 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.027622357 2024-09-21T21:41:24.910 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.910 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.910 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:24.910 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.751636+0000", 2024-09-21T21:41:24.911 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.6604e-05 2024-09-21T21:41:24.911 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.911 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.911 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:24.911 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.751704+0000", 2024-09-21T21:41:24.911 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.7173000000000002e-05 2024-09-21T21:41:24.911 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:24.911 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:24.911 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:24.911 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.911 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.911 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4912.0:1006 3.19 3:9ddcaed2:::rbd_data.1121b3a54d7e.0000000000000000:12 [sparse-read 12976128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-21T21:41:24.911 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:20.723939+0000", 2024-09-21T21:41:24.911 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 4.1089539439999996, 2024-09-21T21:41:24.911 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.028010094999999999, 2024-09-21T21:41:24.912 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:24.912 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:24.912 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:24.912 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:24.912 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4912", 2024-09-21T21:41:24.912 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4204709190", 2024-09-21T21:41:24.912 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1006 2024-09-21T21:41:24.912 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.912 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:24.912 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.912 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:24.912 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723939+0000", 2024-09-21T21:41:24.912 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:24.912 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.913 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.913 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:24.913 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723939+0000", 2024-09-21T21:41:24.913 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:24.913 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.913 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.913 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:24.913 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723940+0000", 2024-09-21T21:41:24.913 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4999999999999998e-07 2024-09-21T21:41:24.913 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.913 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.913 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:24.913 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723942+0000", 2024-09-21T21:41:24.913 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1660000000000001e-06 2024-09-21T21:41:24.913 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.914 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.914 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:24.914 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723949+0000", 2024-09-21T21:41:24.914 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0949999999999998e-06 2024-09-21T21:41:24.914 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.914 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.914 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:24.914 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723954+0000", 2024-09-21T21:41:24.914 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0250000000000002e-06 2024-09-21T21:41:24.914 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.914 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.914 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:24.914 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.751755+0000", 2024-09-21T21:41:24.914 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.027801228000000001 2024-09-21T21:41:24.915 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.915 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.915 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:24.915 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.751841+0000", 2024-09-21T21:41:24.915 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.5829999999999996e-05 2024-09-21T21:41:24.915 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.915 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.915 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:24.915 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.751949+0000", 2024-09-21T21:41:24.915 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010830100000000001 2024-09-21T21:41:24.915 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:24.915 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:24.915 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:24.915 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.915 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.916 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4912.0:1007 3.19 3:9ddcaed2:::rbd_data.1121b3a54d7e.0000000000000000:12 [sparse-read 14680064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-21T21:41:24.916 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:20.723961+0000", 2024-09-21T21:41:24.916 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 4.1089327689999999, 2024-09-21T21:41:24.916 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.028245155000000001, 2024-09-21T21:41:24.916 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:24.916 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:24.916 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:24.916 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:24.916 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4912", 2024-09-21T21:41:24.916 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4204709190", 2024-09-21T21:41:24.916 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1007 2024-09-21T21:41:24.916 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.916 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:24.916 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.916 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:24.917 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723961+0000", 2024-09-21T21:41:24.917 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:24.917 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.917 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.917 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:24.917 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723961+0000", 2024-09-21T21:41:24.917 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:24.917 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.917 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.917 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:24.917 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723961+0000", 2024-09-21T21:41:24.917 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4900000000000001e-07 2024-09-21T21:41:24.917 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.917 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.917 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:24.918 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723963+0000", 2024-09-21T21:41:24.918 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.787e-06 2024-09-21T21:41:24.918 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.918 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.918 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:24.918 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723969+0000", 2024-09-21T21:41:24.918 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8289999999999996e-06 2024-09-21T21:41:24.918 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.918 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.918 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:24.918 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723973+0000", 2024-09-21T21:41:24.918 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5499999999999996e-06 2024-09-21T21:41:24.918 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.918 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.919 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:24.919 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.751994+0000", 2024-09-21T21:41:24.919 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.028021146 2024-09-21T21:41:24.919 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.919 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.919 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:24.919 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.752101+0000", 2024-09-21T21:41:24.919 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010621899999999999 2024-09-21T21:41:24.919 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.919 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.919 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:24.919 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.752206+0000", 2024-09-21T21:41:24.919 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000105175 2024-09-21T21:41:24.919 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:24.920 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:24.920 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:24.920 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.920 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.920 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4912.0:1008 3.19 3:9ddcaed2:::rbd_data.1121b3a54d7e.0000000000000000:12 [sparse-read 16384000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-21T21:41:24.920 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:20.723978+0000", 2024-09-21T21:41:24.920 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 4.1089149479999998, 2024-09-21T21:41:24.920 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.028444661999999999, 2024-09-21T21:41:24.920 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:24.920 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:24.920 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:24.920 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:24.920 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4912", 2024-09-21T21:41:24.920 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4204709190", 2024-09-21T21:41:24.921 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1008 2024-09-21T21:41:24.921 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.921 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:24.921 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.921 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:24.921 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723978+0000", 2024-09-21T21:41:24.921 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:24.921 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.921 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.921 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:24.921 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723978+0000", 2024-09-21T21:41:24.921 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:24.921 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.921 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.921 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:24.922 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.724409+0000", 2024-09-21T21:41:24.922 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00043058300000000001 2024-09-21T21:41:24.922 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.922 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.922 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:24.922 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.724412+0000", 2024-09-21T21:41:24.922 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3730000000000001e-06 2024-09-21T21:41:24.922 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.922 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.922 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:24.922 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.724421+0000", 2024-09-21T21:41:24.922 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.9840000000000007e-06 2024-09-21T21:41:24.922 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.922 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.923 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:24.923 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.724427+0000", 2024-09-21T21:41:24.923 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2029999999999997e-06 2024-09-21T21:41:24.923 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.923 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.923 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:24.923 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.752238+0000", 2024-09-21T21:41:24.923 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.027810577999999999 2024-09-21T21:41:24.923 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.923 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.923 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:24.923 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.752323+0000", 2024-09-21T21:41:24.923 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.4652000000000002e-05 2024-09-21T21:41:24.923 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:24.923 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:24.924 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:24.924 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.752423+0000", 2024-09-21T21:41:24.924 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000100289 2024-09-21T21:41:24.924 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:24.924 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:24.924 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:24.924 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:24.924 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:24.924 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:24.926 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T21:41:24.926 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:24.924+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:25.027 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:25.025+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:25.069 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:25.069 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:41:25.069 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 0 2024-09-21T21:41:25.069 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:25.077 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T21:41:25.128 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:25.126+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:25.180 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:25.180 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:41:25.180 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:41:25.180 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:41:25.180 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:25.189 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T21:41:25.229 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:25.228+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:25.330 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:25.328+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:25.335 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:25.335 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 20, 2024-09-21T21:41:25.335 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 600, 2024-09-21T21:41:25.335 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:41:25.335 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.335 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:1678 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 14483456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:25.335 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.299288+0000", 2024-09-21T21:41:25.335 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 1.0334876150000001, 2024-09-21T21:41:25.335 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031376253, 2024-09-21T21:41:25.335 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:25.335 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:25.336 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:25.336 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:25.336 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:25.336 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:25.336 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1678 2024-09-21T21:41:25.336 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.336 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:25.336 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.336 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:25.336 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299288+0000", 2024-09-21T21:41:25.336 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:25.336 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.336 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.336 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:25.337 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299288+0000", 2024-09-21T21:41:25.337 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:25.337 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.337 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.337 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:25.337 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299288+0000", 2024-09-21T21:41:25.337 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.2500000000000001e-07 2024-09-21T21:41:25.337 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.337 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.337 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:25.337 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299290+0000", 2024-09-21T21:41:25.337 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.945e-06 2024-09-21T21:41:25.337 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.338 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.338 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:25.338 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299295+0000", 2024-09-21T21:41:25.338 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0379999999999999e-06 2024-09-21T21:41:25.338 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.338 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.338 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:25.338 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299299+0000", 2024-09-21T21:41:25.338 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7960000000000002e-06 2024-09-21T21:41:25.338 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.338 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.338 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:25.338 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.330586+0000", 2024-09-21T21:41:25.338 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031287540000000003 2024-09-21T21:41:25.339 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.339 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.339 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:25.339 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.330620+0000", 2024-09-21T21:41:25.339 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3492999999999999e-05 2024-09-21T21:41:25.339 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.339 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.339 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:25.339 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.330664+0000", 2024-09-21T21:41:25.339 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4116000000000002e-05 2024-09-21T21:41:25.339 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:25.339 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:25.339 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:25.339 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.340 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.340 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:1679 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 14942208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:25.340 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.299333+0000", 2024-09-21T21:41:25.340 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 1.03344196, 2024-09-21T21:41:25.340 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031433567000000003, 2024-09-21T21:41:25.340 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:25.340 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:25.340 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:25.340 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:25.340 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:25.341 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:25.341 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1679 2024-09-21T21:41:25.341 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.341 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:25.341 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.341 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:25.341 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299333+0000", 2024-09-21T21:41:25.341 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:25.341 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.341 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.341 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:25.341 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299333+0000", 2024-09-21T21:41:25.341 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:25.341 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.342 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.342 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:25.342 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299334+0000", 2024-09-21T21:41:25.342 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0100000000000005e-07 2024-09-21T21:41:25.342 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.342 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.342 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:25.342 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299336+0000", 2024-09-21T21:41:25.342 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3439999999999999e-06 2024-09-21T21:41:25.342 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.342 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.342 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:25.342 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299343+0000", 2024-09-21T21:41:25.342 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0299999999999996e-06 2024-09-21T21:41:25.343 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.343 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.343 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:25.343 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299348+0000", 2024-09-21T21:41:25.343 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2340000000000002e-06 2024-09-21T21:41:25.343 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.343 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.343 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:25.343 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.330687+0000", 2024-09-21T21:41:25.343 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031338132999999997 2024-09-21T21:41:25.343 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.343 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.343 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:25.343 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.330721+0000", 2024-09-21T21:41:25.344 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4156999999999998e-05 2024-09-21T21:41:25.344 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.344 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.344 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:25.344 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.330767+0000", 2024-09-21T21:41:25.344 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6168000000000002e-05 2024-09-21T21:41:25.344 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:25.344 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:25.344 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:25.344 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.344 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.344 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:1680 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 15400960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:25.344 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.299356+0000", 2024-09-21T21:41:25.344 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 1.0334196099999999, 2024-09-21T21:41:25.344 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031510096000000001, 2024-09-21T21:41:25.345 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:25.345 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:25.345 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:25.345 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:25.345 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:25.345 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:25.345 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1680 2024-09-21T21:41:25.345 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.345 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:25.345 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.345 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:25.345 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299356+0000", 2024-09-21T21:41:25.345 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:25.345 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.345 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.346 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:25.346 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299356+0000", 2024-09-21T21:41:25.346 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:25.346 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.346 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.346 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:25.346 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299356+0000", 2024-09-21T21:41:25.346 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.63e-07 2024-09-21T21:41:25.346 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.346 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.346 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:25.346 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299358+0000", 2024-09-21T21:41:25.346 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1399999999999998e-06 2024-09-21T21:41:25.346 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.347 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.347 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:25.347 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299364+0000", 2024-09-21T21:41:25.347 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1650000000000003e-06 2024-09-21T21:41:25.347 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.347 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.347 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:25.347 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299369+0000", 2024-09-21T21:41:25.347 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8400000000000002e-06 2024-09-21T21:41:25.347 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.347 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.347 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:25.347 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.330788+0000", 2024-09-21T21:41:25.347 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031418399 2024-09-21T21:41:25.348 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.348 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.348 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:25.348 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.330822+0000", 2024-09-21T21:41:25.348 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4218999999999999e-05 2024-09-21T21:41:25.348 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.348 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.348 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:25.348 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.330866+0000", 2024-09-21T21:41:25.348 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3869999999999998e-05 2024-09-21T21:41:25.348 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:25.348 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:25.348 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:25.348 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.348 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.349 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:1681 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 15859712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:25.349 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.299375+0000", 2024-09-21T21:41:25.349 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 1.0333999060000001, 2024-09-21T21:41:25.349 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031600251000000003, 2024-09-21T21:41:25.349 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:25.349 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:25.349 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:25.349 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:25.349 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:25.349 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:25.349 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1681 2024-09-21T21:41:25.349 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.349 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:25.350 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.350 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:25.350 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299375+0000", 2024-09-21T21:41:25.350 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:25.350 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.350 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.350 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:25.350 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299375+0000", 2024-09-21T21:41:25.350 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:25.351 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.351 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.351 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:25.351 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299376+0000", 2024-09-21T21:41:25.351 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8700000000000001e-07 2024-09-21T21:41:25.351 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.351 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.351 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:25.351 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299382+0000", 2024-09-21T21:41:25.351 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3060000000000003e-06 2024-09-21T21:41:25.351 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.352 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.352 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:25.352 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299389+0000", 2024-09-21T21:41:25.352 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.3629999999999996e-06 2024-09-21T21:41:25.352 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.352 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.352 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:25.352 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299395+0000", 2024-09-21T21:41:25.352 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1279999999999999e-06 2024-09-21T21:41:25.352 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.352 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.352 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:25.352 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.330891+0000", 2024-09-21T21:41:25.352 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031496108000000002 2024-09-21T21:41:25.353 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.353 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.353 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:25.353 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.330925+0000", 2024-09-21T21:41:25.353 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4150000000000003e-05 2024-09-21T21:41:25.353 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.353 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.353 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:25.353 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.330976+0000", 2024-09-21T21:41:25.353 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0809000000000001e-05 2024-09-21T21:41:25.353 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:25.354 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:25.354 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:25.354 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.354 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.354 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:1682 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 16318464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:25.354 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.299401+0000", 2024-09-21T21:41:25.354 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 1.0333742619999999, 2024-09-21T21:41:25.354 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031676932999999997, 2024-09-21T21:41:25.354 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:25.354 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:25.354 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:25.354 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:25.354 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:25.354 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:25.355 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1682 2024-09-21T21:41:25.355 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.355 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:25.355 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.355 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:25.355 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299401+0000", 2024-09-21T21:41:25.355 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:25.355 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.355 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.355 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:25.355 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299401+0000", 2024-09-21T21:41:25.355 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:25.355 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.355 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.356 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:25.356 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299401+0000", 2024-09-21T21:41:25.356 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.9299999999999999e-07 2024-09-21T21:41:25.356 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.356 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.356 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:25.356 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299404+0000", 2024-09-21T21:41:25.356 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1720000000000001e-06 2024-09-21T21:41:25.356 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.356 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.356 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:25.356 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299410+0000", 2024-09-21T21:41:25.356 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2480000000000001e-06 2024-09-21T21:41:25.356 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.357 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.357 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:25.357 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299414+0000", 2024-09-21T21:41:25.357 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6480000000000002e-06 2024-09-21T21:41:25.357 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.357 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.357 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:25.357 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.330998+0000", 2024-09-21T21:41:25.357 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031583364000000003 2024-09-21T21:41:25.357 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.357 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.357 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:25.357 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.331033+0000", 2024-09-21T21:41:25.357 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4759999999999999e-05 2024-09-21T21:41:25.357 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.358 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.358 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:25.358 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.331078+0000", 2024-09-21T21:41:25.358 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5348e-05 2024-09-21T21:41:25.358 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:25.358 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:25.358 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:25.358 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.358 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.358 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:2010 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 7864320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:25.358 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.306462+0000", 2024-09-21T21:41:25.358 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 1.0263131759999999, 2024-09-21T21:41:25.358 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031317032000000002, 2024-09-21T21:41:25.358 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:25.359 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:25.359 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:25.359 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:25.359 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:25.359 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:25.359 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2010 2024-09-21T21:41:25.359 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.359 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:25.359 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.359 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:25.359 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306462+0000", 2024-09-21T21:41:25.359 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:25.359 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.359 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.360 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:25.360 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306462+0000", 2024-09-21T21:41:25.360 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:25.360 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.360 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.360 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:25.360 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306463+0000", 2024-09-21T21:41:25.360 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2500000000000001e-07 2024-09-21T21:41:25.360 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.360 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.360 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:25.360 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306465+0000", 2024-09-21T21:41:25.360 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9740000000000001e-06 2024-09-21T21:41:25.360 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.361 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.361 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:25.361 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306470+0000", 2024-09-21T21:41:25.361 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9689999999999999e-06 2024-09-21T21:41:25.361 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.361 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.361 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:25.361 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306475+0000", 2024-09-21T21:41:25.361 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7199999999999997e-06 2024-09-21T21:41:25.361 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.361 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.361 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:25.361 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.337701+0000", 2024-09-21T21:41:25.361 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031225796 2024-09-21T21:41:25.362 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.362 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.362 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:25.362 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.337735+0000", 2024-09-21T21:41:25.362 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4399e-05 2024-09-21T21:41:25.362 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.362 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.362 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:25.362 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.337779+0000", 2024-09-21T21:41:25.362 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3748999999999997e-05 2024-09-21T21:41:25.362 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:25.362 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:25.362 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:25.362 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.363 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.363 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:2011 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 8323072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:25.363 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.306481+0000", 2024-09-21T21:41:25.363 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 1.026294072, 2024-09-21T21:41:25.363 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031404840000000003, 2024-09-21T21:41:25.363 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:25.363 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:25.363 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:25.363 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:25.363 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:25.363 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:25.363 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2011 2024-09-21T21:41:25.363 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.363 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:25.364 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.364 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:25.364 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306481+0000", 2024-09-21T21:41:25.364 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:25.364 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.364 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.364 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:25.364 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306481+0000", 2024-09-21T21:41:25.364 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:25.364 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.364 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.364 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:25.364 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306482+0000", 2024-09-21T21:41:25.364 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9100000000000004e-07 2024-09-21T21:41:25.364 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.365 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.365 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:25.365 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306484+0000", 2024-09-21T21:41:25.365 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.855e-06 2024-09-21T21:41:25.365 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.365 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.365 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:25.365 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306489+0000", 2024-09-21T21:41:25.365 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7799999999999997e-06 2024-09-21T21:41:25.365 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.365 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.365 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:25.365 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306494+0000", 2024-09-21T21:41:25.365 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7539999999999997e-06 2024-09-21T21:41:25.366 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.366 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.366 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:25.366 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.337802+0000", 2024-09-21T21:41:25.366 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031307684000000002 2024-09-21T21:41:25.366 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.366 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.366 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:25.366 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.337836+0000", 2024-09-21T21:41:25.366 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4560000000000001e-05 2024-09-21T21:41:25.366 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.366 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.366 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:25.366 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.337886+0000", 2024-09-21T21:41:25.367 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9716000000000003e-05 2024-09-21T21:41:25.367 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:25.367 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:25.367 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:25.367 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.367 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.367 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:2012 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 8781824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:25.367 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.306500+0000", 2024-09-21T21:41:25.367 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 1.026275257, 2024-09-21T21:41:25.367 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031483165, 2024-09-21T21:41:25.367 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:25.367 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:25.367 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:25.367 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:25.367 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:25.368 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:25.368 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2012 2024-09-21T21:41:25.368 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.368 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:25.368 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.368 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:25.368 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306500+0000", 2024-09-21T21:41:25.368 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:25.368 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.368 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.368 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:25.368 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306500+0000", 2024-09-21T21:41:25.368 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:25.368 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.368 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.369 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:25.369 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306500+0000", 2024-09-21T21:41:25.369 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7099999999999997e-07 2024-09-21T21:41:25.369 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.369 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.369 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:25.369 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306502+0000", 2024-09-21T21:41:25.369 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1019999999999999e-06 2024-09-21T21:41:25.369 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.369 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.369 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:25.369 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306508+0000", 2024-09-21T21:41:25.369 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8390000000000003e-06 2024-09-21T21:41:25.369 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.370 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.370 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:25.370 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306513+0000", 2024-09-21T21:41:25.370 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4909999999999999e-06 2024-09-21T21:41:25.370 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.370 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.370 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:25.370 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.337898+0000", 2024-09-21T21:41:25.370 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031385437000000002 2024-09-21T21:41:25.370 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.370 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.370 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:25.370 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.337931+0000", 2024-09-21T21:41:25.370 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.2656e-05 2024-09-21T21:41:25.371 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.371 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.371 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:25.371 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.337983+0000", 2024-09-21T21:41:25.371 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2268999999999997e-05 2024-09-21T21:41:25.371 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:25.371 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:25.371 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:25.371 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.371 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.371 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:2013 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 9240576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:25.371 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.306520+0000", 2024-09-21T21:41:25.371 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 1.0262555289999999, 2024-09-21T21:41:25.371 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031570864999999997, 2024-09-21T21:41:25.371 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:25.372 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:25.372 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:25.372 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:25.372 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:25.372 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:25.372 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2013 2024-09-21T21:41:25.372 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.372 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:25.372 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.372 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:25.372 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306520+0000", 2024-09-21T21:41:25.372 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:25.372 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.372 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.373 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:25.373 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306520+0000", 2024-09-21T21:41:25.373 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:25.373 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.373 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.373 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:25.373 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306520+0000", 2024-09-21T21:41:25.373 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3700000000000001e-07 2024-09-21T21:41:25.373 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.373 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.373 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:25.373 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306530+0000", 2024-09-21T21:41:25.373 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0336000000000001e-05 2024-09-21T21:41:25.374 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.374 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.374 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:25.374 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306538+0000", 2024-09-21T21:41:25.374 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.3690000000000001e-06 2024-09-21T21:41:25.374 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.374 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.374 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:25.374 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306543+0000", 2024-09-21T21:41:25.374 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8930000000000004e-06 2024-09-21T21:41:25.374 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.374 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.374 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:25.374 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338005+0000", 2024-09-21T21:41:25.375 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031461973999999997 2024-09-21T21:41:25.375 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.375 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.375 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:25.375 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338039+0000", 2024-09-21T21:41:25.375 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4545999999999998e-05 2024-09-21T21:41:25.375 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.375 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.375 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:25.375 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338091+0000", 2024-09-21T21:41:25.375 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1409999999999997e-05 2024-09-21T21:41:25.375 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:25.375 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:25.376 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:25.376 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.376 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.376 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:2014 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 9699328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:25.376 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.306553+0000", 2024-09-21T21:41:25.376 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 1.0262225760000001, 2024-09-21T21:41:25.376 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031642225000000003, 2024-09-21T21:41:25.376 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:25.376 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:25.376 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:25.376 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:25.376 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:25.376 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:25.376 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2014 2024-09-21T21:41:25.376 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.377 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:25.377 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.377 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:25.377 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306553+0000", 2024-09-21T21:41:25.377 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:25.377 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.377 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.377 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:25.377 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306553+0000", 2024-09-21T21:41:25.377 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:25.377 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.377 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.377 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:25.377 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306553+0000", 2024-09-21T21:41:25.377 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5600000000000001e-07 2024-09-21T21:41:25.378 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.378 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.378 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:25.378 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306555+0000", 2024-09-21T21:41:25.378 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3070000000000001e-06 2024-09-21T21:41:25.378 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.378 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.378 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:25.378 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306561+0000", 2024-09-21T21:41:25.378 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8860000000000001e-06 2024-09-21T21:41:25.378 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.378 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.379 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:25.379 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306566+0000", 2024-09-21T21:41:25.379 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3420000000000001e-06 2024-09-21T21:41:25.379 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.379 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.379 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:25.379 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338115+0000", 2024-09-21T21:41:25.379 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031549333999999998 2024-09-21T21:41:25.379 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.379 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.380 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:25.380 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338150+0000", 2024-09-21T21:41:25.380 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4649e-05 2024-09-21T21:41:25.380 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.380 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.380 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:25.380 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338195+0000", 2024-09-21T21:41:25.380 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5250999999999998e-05 2024-09-21T21:41:25.380 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:25.380 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:25.380 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:25.380 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.381 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.381 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:2015 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 10158080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:25.381 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.306572+0000", 2024-09-21T21:41:25.381 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 1.026203658, 2024-09-21T21:41:25.381 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031729593, 2024-09-21T21:41:25.381 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:25.381 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:25.381 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:25.381 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:25.381 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:25.381 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:25.381 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2015 2024-09-21T21:41:25.381 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.382 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:25.382 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.382 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:25.382 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306572+0000", 2024-09-21T21:41:25.382 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:25.382 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.382 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.382 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:25.382 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306572+0000", 2024-09-21T21:41:25.382 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:25.382 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.382 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.382 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:25.383 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306572+0000", 2024-09-21T21:41:25.383 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7500000000000001e-07 2024-09-21T21:41:25.383 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.383 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.383 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:25.383 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306574+0000", 2024-09-21T21:41:25.383 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0669999999999999e-06 2024-09-21T21:41:25.383 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.383 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.383 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:25.383 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306579+0000", 2024-09-21T21:41:25.384 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4009999999999996e-06 2024-09-21T21:41:25.384 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.384 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.384 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:25.384 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306585+0000", 2024-09-21T21:41:25.384 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3349999999999997e-06 2024-09-21T21:41:25.384 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.384 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.384 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:25.384 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338217+0000", 2024-09-21T21:41:25.384 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031631805999999998 2024-09-21T21:41:25.384 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.385 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.385 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:25.385 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338251+0000", 2024-09-21T21:41:25.385 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4813e-05 2024-09-21T21:41:25.385 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.385 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.385 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:25.385 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338301+0000", 2024-09-21T21:41:25.385 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9796000000000002e-05 2024-09-21T21:41:25.385 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:25.385 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:25.385 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:25.385 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.386 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.386 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:2016 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 10616832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:25.386 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.306601+0000", 2024-09-21T21:41:25.386 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 1.0261744850000001, 2024-09-21T21:41:25.386 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031801840999999997, 2024-09-21T21:41:25.386 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:25.386 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:25.386 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:25.386 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:25.386 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:25.386 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:25.386 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2016 2024-09-21T21:41:25.387 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.387 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:25.387 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.387 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:25.387 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306601+0000", 2024-09-21T21:41:25.387 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:25.387 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.387 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.387 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:25.387 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306601+0000", 2024-09-21T21:41:25.387 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:25.387 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.387 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.388 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:25.388 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306601+0000", 2024-09-21T21:41:25.388 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1900000000000003e-07 2024-09-21T21:41:25.388 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.388 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.388 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:25.388 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306603+0000", 2024-09-21T21:41:25.388 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.17e-06 2024-09-21T21:41:25.388 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.388 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.388 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:25.388 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306609+0000", 2024-09-21T21:41:25.389 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8069999999999996e-06 2024-09-21T21:41:25.389 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.389 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.389 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:25.389 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306614+0000", 2024-09-21T21:41:25.389 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5090000000000004e-06 2024-09-21T21:41:25.389 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.389 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.389 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:25.389 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338325+0000", 2024-09-21T21:41:25.389 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031711237000000003 2024-09-21T21:41:25.389 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.389 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.390 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:25.390 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338359+0000", 2024-09-21T21:41:25.390 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3810000000000003e-05 2024-09-21T21:41:25.390 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.390 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.390 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:25.390 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338403+0000", 2024-09-21T21:41:25.390 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3788999999999997e-05 2024-09-21T21:41:25.390 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:25.390 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:25.390 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:25.391 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.391 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.391 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:2017 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 11075584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:25.391 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.306621+0000", 2024-09-21T21:41:25.391 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 1.026154719, 2024-09-21T21:41:25.391 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031891271999999998, 2024-09-21T21:41:25.391 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:25.391 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:25.391 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:25.391 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:25.391 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:25.391 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:25.392 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2017 2024-09-21T21:41:25.392 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.392 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:25.392 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.392 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:25.392 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306621+0000", 2024-09-21T21:41:25.392 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:25.392 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.392 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.392 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:25.392 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306621+0000", 2024-09-21T21:41:25.392 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:25.392 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.393 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.393 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:25.393 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306621+0000", 2024-09-21T21:41:25.393 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5400000000000002e-07 2024-09-21T21:41:25.393 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.393 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.393 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:25.393 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306623+0000", 2024-09-21T21:41:25.393 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1100000000000001e-06 2024-09-21T21:41:25.393 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.393 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.393 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:25.394 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306629+0000", 2024-09-21T21:41:25.394 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7810000000000002e-06 2024-09-21T21:41:25.394 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.394 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.394 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:25.394 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306633+0000", 2024-09-21T21:41:25.394 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5589999999999999e-06 2024-09-21T21:41:25.394 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.394 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.394 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:25.395 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338427+0000", 2024-09-21T21:41:25.395 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031793948000000002 2024-09-21T21:41:25.395 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.395 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.395 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:25.395 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338462+0000", 2024-09-21T21:41:25.395 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4785e-05 2024-09-21T21:41:25.395 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.395 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.395 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:25.395 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338512+0000", 2024-09-21T21:41:25.395 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9734999999999997e-05 2024-09-21T21:41:25.396 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:25.396 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:25.396 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:25.396 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.396 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.396 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:2018 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 11534336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:25.396 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.306639+0000", 2024-09-21T21:41:25.396 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 1.0261360900000001, 2024-09-21T21:41:25.396 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031963189000000003, 2024-09-21T21:41:25.396 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:25.396 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:25.396 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:25.396 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:25.397 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:25.397 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:25.397 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2018 2024-09-21T21:41:25.397 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.397 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:25.397 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.397 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:25.397 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306639+0000", 2024-09-21T21:41:25.397 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:25.397 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.397 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.397 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:25.397 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306639+0000", 2024-09-21T21:41:25.398 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:25.398 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.398 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.398 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:25.398 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306640+0000", 2024-09-21T21:41:25.398 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4600000000000005e-07 2024-09-21T21:41:25.398 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.398 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.399 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:25.399 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306642+0000", 2024-09-21T21:41:25.399 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.916e-06 2024-09-21T21:41:25.399 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.399 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.399 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:25.399 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306647+0000", 2024-09-21T21:41:25.399 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7590000000000003e-06 2024-09-21T21:41:25.399 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.399 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.399 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:25.399 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306652+0000", 2024-09-21T21:41:25.399 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4580000000000003e-06 2024-09-21T21:41:25.400 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.400 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.400 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:25.400 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338524+0000", 2024-09-21T21:41:25.400 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031872172999999997 2024-09-21T21:41:25.400 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.400 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.400 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:25.400 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338557+0000", 2024-09-21T21:41:25.400 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3136000000000003e-05 2024-09-21T21:41:25.400 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.400 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.400 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:25.401 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338602+0000", 2024-09-21T21:41:25.401 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5201000000000003e-05 2024-09-21T21:41:25.401 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:25.401 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:25.401 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:25.401 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.401 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.401 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:2019 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 11993088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:25.401 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.306658+0000", 2024-09-21T21:41:25.401 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 1.026117696, 2024-09-21T21:41:25.401 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.032059007, 2024-09-21T21:41:25.401 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:25.401 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:25.401 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:25.402 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:25.402 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:25.402 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:25.402 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2019 2024-09-21T21:41:25.402 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.402 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:25.402 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.402 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:25.402 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306658+0000", 2024-09-21T21:41:25.402 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:25.402 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.402 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.402 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:25.403 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306658+0000", 2024-09-21T21:41:25.403 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:25.403 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.403 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.403 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:25.403 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.307950+0000", 2024-09-21T21:41:25.403 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.001292199 2024-09-21T21:41:25.403 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.403 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.403 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:25.403 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.307954+0000", 2024-09-21T21:41:25.404 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5689999999999998e-06 2024-09-21T21:41:25.404 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.404 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.404 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:25.404 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.307964+0000", 2024-09-21T21:41:25.404 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6749999999999997e-06 2024-09-21T21:41:25.404 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.404 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.404 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:25.404 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.307971+0000", 2024-09-21T21:41:25.404 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5189999999999998e-06 2024-09-21T21:41:25.404 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.404 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.405 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:25.405 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338625+0000", 2024-09-21T21:41:25.405 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.030654569 2024-09-21T21:41:25.405 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.405 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.405 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:25.405 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338660+0000", 2024-09-21T21:41:25.405 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4656999999999997e-05 2024-09-21T21:41:25.405 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.405 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.405 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:25.405 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338717+0000", 2024-09-21T21:41:25.405 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6818999999999999e-05 2024-09-21T21:41:25.406 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:25.406 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:25.406 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:25.406 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.406 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.406 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:2026 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 15204352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:25.406 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.308109+0000", 2024-09-21T21:41:25.406 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 1.024666096, 2024-09-21T21:41:25.406 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031315883000000003, 2024-09-21T21:41:25.406 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:25.406 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:25.406 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:25.407 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:25.407 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:25.407 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:25.407 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2026 2024-09-21T21:41:25.407 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.407 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:25.407 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.407 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:25.407 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308109+0000", 2024-09-21T21:41:25.407 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:25.407 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.407 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.408 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:25.408 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308109+0000", 2024-09-21T21:41:25.408 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:25.408 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.408 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.408 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:25.408 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308110+0000", 2024-09-21T21:41:25.408 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.96e-07 2024-09-21T21:41:25.408 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.408 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.408 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:25.409 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308112+0000", 2024-09-21T21:41:25.409 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0269999999999998e-06 2024-09-21T21:41:25.409 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.409 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.409 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:25.409 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308117+0000", 2024-09-21T21:41:25.409 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.631e-06 2024-09-21T21:41:25.409 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.409 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.409 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:25.409 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308122+0000", 2024-09-21T21:41:25.409 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7310000000000001e-06 2024-09-21T21:41:25.410 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.410 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.410 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:25.410 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.339341+0000", 2024-09-21T21:41:25.410 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031218610000000001 2024-09-21T21:41:25.410 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.410 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.410 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:25.410 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.339376+0000", 2024-09-21T21:41:25.410 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5305000000000002e-05 2024-09-21T21:41:25.410 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.411 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.411 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:25.411 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.339425+0000", 2024-09-21T21:41:25.411 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9183e-05 2024-09-21T21:41:25.411 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:25.411 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:25.411 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:25.411 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.411 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.411 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:2027 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 15663104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:25.411 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.308128+0000", 2024-09-21T21:41:25.411 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 1.02464756, 2024-09-21T21:41:25.411 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031387452000000003, 2024-09-21T21:41:25.412 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:25.412 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:25.412 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:25.412 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:25.412 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:25.412 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:25.412 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2027 2024-09-21T21:41:25.412 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.412 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:25.412 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.412 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:25.412 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308128+0000", 2024-09-21T21:41:25.412 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:25.413 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.413 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.413 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:25.413 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308128+0000", 2024-09-21T21:41:25.413 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:25.413 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.413 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.413 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:25.413 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308128+0000", 2024-09-21T21:41:25.413 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4700000000000002e-07 2024-09-21T21:41:25.413 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.413 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.414 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:25.414 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308130+0000", 2024-09-21T21:41:25.414 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.8670000000000001e-06 2024-09-21T21:41:25.414 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.414 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.414 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:25.414 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308136+0000", 2024-09-21T21:41:25.414 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0299999999999999e-06 2024-09-21T21:41:25.414 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.414 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.414 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:25.414 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308140+0000", 2024-09-21T21:41:25.415 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3830000000000002e-06 2024-09-21T21:41:25.415 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.415 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.415 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:25.415 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.339437+0000", 2024-09-21T21:41:25.415 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031296864000000001 2024-09-21T21:41:25.415 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.415 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.415 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:25.415 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.339471+0000", 2024-09-21T21:41:25.415 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3302000000000001e-05 2024-09-21T21:41:25.416 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.416 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.416 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:25.416 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.339515+0000", 2024-09-21T21:41:25.416 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4659000000000001e-05 2024-09-21T21:41:25.416 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:25.416 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:25.416 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:25.416 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.416 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.416 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:2028 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 16121856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:25.416 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.308147+0000", 2024-09-21T21:41:25.417 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 1.0246286330000001, 2024-09-21T21:41:25.417 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031474477000000001, 2024-09-21T21:41:25.417 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:25.417 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:25.417 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:25.417 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:25.417 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:25.417 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:25.417 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2028 2024-09-21T21:41:25.417 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.417 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:25.417 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.418 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:25.418 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308147+0000", 2024-09-21T21:41:25.418 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:25.418 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.418 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.418 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:25.418 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308147+0000", 2024-09-21T21:41:25.418 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:25.418 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.418 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.418 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:25.418 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308147+0000", 2024-09-21T21:41:25.419 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1900000000000003e-07 2024-09-21T21:41:25.419 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.419 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.419 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:25.419 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308149+0000", 2024-09-21T21:41:25.419 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9479999999999998e-06 2024-09-21T21:41:25.419 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.419 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.419 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:25.419 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308155+0000", 2024-09-21T21:41:25.419 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6200000000000004e-06 2024-09-21T21:41:25.419 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.420 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.420 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:25.420 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308160+0000", 2024-09-21T21:41:25.420 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0799999999999996e-06 2024-09-21T21:41:25.420 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.420 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.420 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:25.420 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.339539+0000", 2024-09-21T21:41:25.420 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031378902 2024-09-21T21:41:25.420 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.420 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.421 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:25.421 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.339574+0000", 2024-09-21T21:41:25.421 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5176999999999999e-05 2024-09-21T21:41:25.421 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.421 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.421 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:25.421 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.339621+0000", 2024-09-21T21:41:25.421 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7231000000000002e-05 2024-09-21T21:41:25.421 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:25.421 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:25.421 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:25.421 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.422 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.422 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:2029 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 16580608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:25.422 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.308167+0000", 2024-09-21T21:41:25.422 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 1.0246082439999999, 2024-09-21T21:41:25.422 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031555582999999998, 2024-09-21T21:41:25.422 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:25.422 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:25.422 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:25.422 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:25.422 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:25.422 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:25.422 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2029 2024-09-21T21:41:25.422 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.423 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:25.423 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.423 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:25.423 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308167+0000", 2024-09-21T21:41:25.423 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:25.423 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.423 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.423 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:25.423 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308167+0000", 2024-09-21T21:41:25.423 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:25.423 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.423 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.424 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:25.424 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308167+0000", 2024-09-21T21:41:25.424 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6600000000000002e-07 2024-09-21T21:41:25.424 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.424 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.424 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:25.424 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308169+0000", 2024-09-21T21:41:25.424 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9819999999999998e-06 2024-09-21T21:41:25.424 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.424 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.424 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:25.424 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308175+0000", 2024-09-21T21:41:25.425 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8289999999999996e-06 2024-09-21T21:41:25.425 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.425 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.425 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:25.425 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308180+0000", 2024-09-21T21:41:25.425 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6439999999999999e-06 2024-09-21T21:41:25.425 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.425 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.425 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:25.425 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.339645+0000", 2024-09-21T21:41:25.425 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031465083999999997 2024-09-21T21:41:25.425 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.426 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.426 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:25.426 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.339679+0000", 2024-09-21T21:41:25.426 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3572999999999999e-05 2024-09-21T21:41:25.426 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.426 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.426 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:25.426 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.339723+0000", 2024-09-21T21:41:25.426 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4104999999999999e-05 2024-09-21T21:41:25.426 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:25.426 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:25.426 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:25.427 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.427 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.427 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:2059 3.5 3:a6b41377:::rbd_data.11213654c685.0000000000000008:head [stat,truncate 13631488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:25.427 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.308186+0000", 2024-09-21T21:41:25.427 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 1.024589666, 2024-09-21T21:41:25.427 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.032997110000000003, 2024-09-21T21:41:25.427 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:25.427 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:25.427 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:41:25.427 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:25.427 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:25.427 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:25.428 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2059 2024-09-21T21:41:25.428 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.428 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:25.429 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.429 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:25.429 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308186+0000", 2024-09-21T21:41:25.429 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:25.430 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.430 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.430 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:25.430 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308186+0000", 2024-09-21T21:41:25.430 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:25.430 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.430 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.430 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:25.430 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308186+0000", 2024-09-21T21:41:25.430 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1899999999999998e-07 2024-09-21T21:41:25.430 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.430 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.431 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:25.431 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308188+0000", 2024-09-21T21:41:25.431 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0099999999999998e-06 2024-09-21T21:41:25.431 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.431 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.431 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:25.431 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308195+0000", 2024-09-21T21:41:25.431 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0589999999999997e-06 2024-09-21T21:41:25.431 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.431 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.431 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:25.431 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308199+0000", 2024-09-21T21:41:25.432 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2930000000000002e-06 2024-09-21T21:41:25.432 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.432 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.432 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:25.432 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.339745+0000", 2024-09-21T21:41:25.432 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031546091999999998 2024-09-21T21:41:25.432 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.432 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.432 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:25.432 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.339782+0000", 2024-09-21T21:41:25.432 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6847999999999997e-05 2024-09-21T21:41:25.432 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.433 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.433 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:41:25.433 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.339856+0000", 2024-09-21T21:41:25.433 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.3992999999999994e-05 2024-09-21T21:41:25.433 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.433 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.433 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:41:25.433 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.341013+0000", 2024-09-21T21:41:25.433 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00115642 2024-09-21T21:41:25.433 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.433 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.433 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:41:25.433 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.341138+0000", 2024-09-21T21:41:25.434 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00012540699999999999 2024-09-21T21:41:25.434 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.434 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.434 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:41:25.434 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.341164+0000", 2024-09-21T21:41:25.434 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.6168000000000001e-05 2024-09-21T21:41:25.434 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:25.434 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:25.434 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:25.434 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.341183+0000", 2024-09-21T21:41:25.434 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.8400999999999999e-05 2024-09-21T21:41:25.434 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:25.435 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:25.435 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:25.435 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:25.435 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:25.435 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:25.436 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T21:41:25.437 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:25.434+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:25.478 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T21:41:25.537 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:25.536+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:25.543 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:25.543 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:41:25.543 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-21T21:41:25.543 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:25.552 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T21:41:25.598 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0: {} 2024-09-21T21:41:25.598 INFO:teuthology.orchestra.run.smithi081.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T21:41:25.614 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1: {} 2024-09-21T21:41:25.615 INFO:teuthology.orchestra.run.smithi081.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T21:41:25.630 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2: {} 2024-09-21T21:41:25.631 INFO:teuthology.orchestra.run.smithi081.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T21:41:25.638 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:25.635+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:25.647 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3: {} 2024-09-21T21:41:25.647 INFO:teuthology.orchestra.run.smithi081.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T21:41:25.654 INFO:teuthology.orchestra.run.smithi164.stdout:2924 punch from 0x55df600 to 0x55ed800, (0xe200 bytes) 2024-09-21T21:41:25.655 INFO:teuthology.orchestra.run.smithi164.stdout:2932 write 0xb921200 thru 0xb925345 (0x4146 bytes) 2024-09-21T21:41:25.658 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:25.658 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:25.660 INFO:teuthology.orchestra.run.smithi164.stdout:skipping zero length punch hole 2024-09-21T21:41:25.660 INFO:teuthology.orchestra.run.smithi164.stdout:2936 punch from 0x2dc5e00 to 0x2dd2a00, (0xcc00 bytes) 2024-09-21T21:41:25.661 INFO:teuthology.orchestra.run.smithi164.stdout:2938 punch from 0xa3c8e00 to 0xa3cb600, (0x2800 bytes) 2024-09-21T21:41:25.661 INFO:teuthology.orchestra.run.smithi164.stdout:2940 read 0x5af6600 thru 0x5afe476 (0x7e77 bytes) 2024-09-21T21:41:25.668 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T21:41:25.669 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4: {} 2024-09-21T21:41:25.669 INFO:teuthology.orchestra.run.smithi081.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T21:41:25.674 INFO:teuthology.orchestra.run.smithi164.stdout:2941 trunc from 0xccb3a00 to 0x67d3200 2024-09-21T21:41:25.686 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5: {} 2024-09-21T21:41:25.686 INFO:teuthology.orchestra.run.smithi081.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T21:41:25.694 INFO:teuthology.orchestra.run.smithi164.stdout:2944 punch from 0x1e35e00 to 0x1e3ea00, (0x8c00 bytes) 2024-09-21T21:41:25.701 INFO:teuthology.orchestra.run.smithi164.stdout:2945 write 0xde0000 thru 0xdeb067 (0xb068 bytes) 2024-09-21T21:41:25.711 INFO:teuthology.orchestra.run.smithi164.stdout:2947 trunc from 0x67d3200 to 0xb57e00 2024-09-21T21:41:25.745 INFO:teuthology.orchestra.run.smithi164.stdout:2948 write 0x41ee800 thru 0x41f53fe (0x6bff bytes) 2024-09-21T21:41:25.746 INFO:teuthology.orchestra.run.smithi164.stdout:2949 punch from 0x2466200 to 0x2467a00, (0x1800 bytes) 2024-09-21T21:41:25.747 INFO:teuthology.orchestra.run.smithi164.stdout:2950 read 0x27a2800 thru 0x27ab69a (0x8e9b bytes) 2024-09-21T21:41:25.748 INFO:teuthology.orchestra.run.smithi164.stdout:2953 read 0x13ab600 thru 0x13aecff (0x3700 bytes) 2024-09-21T21:41:25.752 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:25.750+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:25.776 INFO:teuthology.orchestra.run.smithi164.stdout:2954 write 0xccb5e00 thru 0xccb7223 (0x1424 bytes) 2024-09-21T21:41:25.776 INFO:teuthology.orchestra.run.smithi164.stdout:2955 punch from 0x3196800 to 0x3197000, (0x800 bytes) 2024-09-21T21:41:25.776 INFO:teuthology.orchestra.run.smithi164.stdout:2959 read 0x486be00 thru 0x4872c36 (0x6e37 bytes) 2024-09-21T21:41:25.776 INFO:teuthology.orchestra.run.smithi164.stdout:2960 punch from 0x592c400 to 0x592e600, (0x2200 bytes) 2024-09-21T21:41:25.788 INFO:teuthology.orchestra.run.smithi164.stdout:2962 trunc from 0xccb7400 to 0x37b4e00 2024-09-21T21:41:25.808 INFO:teuthology.orchestra.run.smithi164.stdout:2963 trunc from 0x37b4e00 to 0x15e2800 2024-09-21T21:41:25.820 INFO:teuthology.orchestra.run.smithi164.stdout:2964 punch from 0xf5f400 to 0xf65400, (0x6000 bytes) 2024-09-21T21:41:25.825 INFO:teuthology.orchestra.run.smithi164.stdout:2966 read 0x1af600 thru 0x1b74e4 (0x7ee5 bytes) 2024-09-21T21:41:25.828 INFO:teuthology.orchestra.run.smithi164.stdout:2968 trunc from 0x15e2800 to 0x358ee00 2024-09-21T21:41:25.829 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:25.829 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:25.839 INFO:teuthology.orchestra.run.smithi164.stdout:2971 read 0x148a00 thru 0x153f3b (0xb53c bytes) 2024-09-21T21:41:25.839 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:25.838+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:25.840 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T21:41:25.841 INFO:teuthology.orchestra.run.smithi164.stdout:2972 read 0x108f400 thru 0x1098b63 (0x9764 bytes) 2024-09-21T21:41:25.850 INFO:teuthology.orchestra.run.smithi164.stdout:2973 write 0x4348a00 thru 0x434de11 (0x5412 bytes) 2024-09-21T21:41:25.857 INFO:teuthology.orchestra.run.smithi164.stdout:2974 punch from 0x2aa6000 to 0x2ab0000, (0xa000 bytes) 2024-09-21T21:41:25.857 INFO:teuthology.orchestra.run.smithi164.stdout:2975 write 0x182b000 thru 0x182d71b (0x271c bytes) 2024-09-21T21:41:25.859 INFO:teuthology.orchestra.run.smithi164.stdout:2976 trunc from 0x434e000 to 0x5c9fe00 2024-09-21T21:41:25.869 INFO:teuthology.orchestra.run.smithi164.stdout:2979 punch from 0x4ff4a00 to 0x4ffd400, (0x8a00 bytes) 2024-09-21T21:41:25.870 INFO:teuthology.orchestra.run.smithi164.stdout:2981 read 0x40e600 thru 0x41b0b0 (0xcab1 bytes) 2024-09-21T21:41:25.880 INFO:teuthology.orchestra.run.smithi164.stdout:2983 write 0x7979400 thru 0x7982ed3 (0x9ad4 bytes) 2024-09-21T21:41:25.882 INFO:teuthology.orchestra.run.smithi164.stdout:2984 punch from 0x5241c00 to 0x524b000, (0x9400 bytes) 2024-09-21T21:41:25.882 INFO:teuthology.orchestra.run.smithi164.stdout:2985 punch from 0x58ae800 to 0x58b1800, (0x3000 bytes) 2024-09-21T21:41:25.898 INFO:teuthology.orchestra.run.smithi164.stdout:2988 write 0xd60b000 thru 0xd60b4cd (0x4ce bytes) 2024-09-21T21:41:25.899 INFO:teuthology.orchestra.run.smithi164.stdout:2989 read 0x564ae00 thru 0x564f031 (0x4232 bytes) 2024-09-21T21:41:25.900 INFO:teuthology.orchestra.run.smithi164.stdout:2990 punch from 0xcb14c00 to 0xcb1c600, (0x7a00 bytes) 2024-09-21T21:41:25.900 INFO:teuthology.orchestra.run.smithi164.stdout:2991 write 0xbe59a00 thru 0xbe620c2 (0x86c3 bytes) 2024-09-21T21:41:25.901 INFO:teuthology.orchestra.run.smithi164.stdout:2997 punch from 0x2f6f200 to 0x2f7e200, (0xf000 bytes) 2024-09-21T21:41:25.901 INFO:teuthology.orchestra.run.smithi164.stdout:2998 write 0x12eb400 thru 0x12f1bff (0x6800 bytes) 2024-09-21T21:41:25.903 INFO:teuthology.orchestra.run.smithi164.stdout:3001 read 0x95b2c00 thru 0x95b4a76 (0x1e77 bytes) 2024-09-21T21:41:25.903 INFO:teuthology.orchestra.run.smithi164.stdout:3003 read 0x2ccbc00 thru 0x2ccfb23 (0x3f24 bytes) 2024-09-21T21:41:25.909 INFO:teuthology.orchestra.run.smithi164.stdout:3004 trunc from 0xd60b600 to 0x80c6e00 2024-09-21T21:41:25.938 INFO:teuthology.orchestra.run.smithi164.stdout:3007 write 0x812ec00 thru 0x813205e (0x345f bytes) 2024-09-21T21:41:25.939 INFO:teuthology.orchestra.run.smithi164.stdout:3008 read 0x3cdae00 thru 0x3ce8ebf (0xe0c0 bytes) 2024-09-21T21:41:25.939 INFO:teuthology.orchestra.run.smithi164.stdout:3010 write 0x781a400 thru 0x781eeb4 (0x4ab5 bytes) 2024-09-21T21:41:25.939 INFO:teuthology.orchestra.run.smithi164.stdout:3011 read 0x7890a00 thru 0x7899b34 (0x9135 bytes) 2024-09-21T21:41:25.939 INFO:teuthology.orchestra.run.smithi164.stdout:3012 write 0x27a2000 thru 0x27a3208 (0x1209 bytes) 2024-09-21T21:41:25.940 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:25.939+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:25.946 INFO:teuthology.orchestra.run.smithi164.stdout:3013 trunc from 0x8132200 to 0x11e7800 2024-09-21T21:41:25.965 INFO:teuthology.orchestra.run.smithi164.stdout:3016 read 0xa70e00 thru 0xa7a8a0 (0x9aa1 bytes) 2024-09-21T21:41:25.976 INFO:teuthology.orchestra.run.smithi164.stdout:3018 write 0x14a3400 thru 0x14b1dcd (0xe9ce bytes) 2024-09-21T21:41:25.982 INFO:teuthology.orchestra.run.smithi164.stdout:3020 trunc from 0x14b1e00 to 0x5c75c00 2024-09-21T21:41:25.996 INFO:teuthology.orchestra.run.smithi164.stdout:3021 trunc from 0x5c75c00 to 0x3c9de00 2024-09-21T21:41:26.007 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:26.008 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:26.015 INFO:teuthology.orchestra.run.smithi164.stdout:3022 read 0x16a4600 thru 0x16b0feb (0xc9ec bytes) 2024-09-21T21:41:26.017 INFO:teuthology.orchestra.run.smithi164.stdout:3023 read 0x532600 thru 0x536d37 (0x4738 bytes) 2024-09-21T21:41:26.017 INFO:teuthology.orchestra.run.smithi164.stdout:3024 read 0x24b0800 thru 0x24b3796 (0x2f97 bytes) 2024-09-21T21:41:26.017 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T21:41:26.026 INFO:teuthology.orchestra.run.smithi164.stdout:3025 trunc from 0x3c9de00 to 0xb79ba00 2024-09-21T21:41:26.041 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:26.040+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:26.054 INFO:teuthology.orchestra.run.smithi164.stdout:3026 trunc from 0xb79ba00 to 0xedd400 2024-09-21T21:41:26.082 INFO:teuthology.orchestra.run.smithi164.stdout:3027 read 0xa0400 thru 0xa415d (0x3d5e bytes) 2024-09-21T21:41:26.082 INFO:teuthology.orchestra.run.smithi164.stdout:3028 read 0x602c00 thru 0x606207 (0x3608 bytes) 2024-09-21T21:41:26.082 INFO:teuthology.orchestra.run.smithi164.stdout:3029 write 0xaa2a00 thru 0xaab877 (0x8e78 bytes) 2024-09-21T21:41:26.083 INFO:teuthology.orchestra.run.smithi164.stdout:3036 read 0xaf5e00 thru 0xafa206 (0x4407 bytes) 2024-09-21T21:41:26.099 INFO:teuthology.orchestra.run.smithi164.stdout:3041 trunc from 0xedd400 to 0xde79000 2024-09-21T21:41:26.124 INFO:teuthology.orchestra.run.smithi164.stdout:3044 trunc from 0xde79000 to 0x26f4000 2024-09-21T21:41:26.130 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:26.131 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:26.140 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T21:41:26.141 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:26.140+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:26.156 INFO:teuthology.orchestra.run.smithi164.stdout:3046 trunc from 0x26f4000 to 0x708400 2024-09-21T21:41:26.176 INFO:teuthology.orchestra.run.smithi164.stdout:3050 punch from 0x529c00 to 0x52ea00, (0x4e00 bytes) 2024-09-21T21:41:26.176 INFO:teuthology.orchestra.run.smithi164.stdout:3053 read 0x57f800 thru 0x589d91 (0xa592 bytes) 2024-09-21T21:41:26.176 INFO:teuthology.orchestra.run.smithi164.stdout:3055 read 0x48de00 thru 0x4979e0 (0x9be1 bytes) 2024-09-21T21:41:26.189 INFO:teuthology.orchestra.run.smithi164.stdout:3058 write 0x1f7f600 thru 0x1f89b7c (0xa57d bytes) 2024-09-21T21:41:26.202 INFO:teuthology.orchestra.run.smithi164.stdout:3059 write 0x40b1200 thru 0x40be5b6 (0xd3b7 bytes) 2024-09-21T21:41:26.203 INFO:teuthology.orchestra.run.smithi164.stdout:3061 clone 20 order 25 su 131072 sc 4 2024-09-21T21:41:26.242 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:26.240+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:26.255 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:26.255 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:26.263 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T21:41:26.347 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:26.345+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:26.407 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:26.407 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:26.416 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T21:41:26.457 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:26.454+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:26.544 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:26.542+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:26.557 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:26.557 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:26.566 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T21:41:26.646 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:26.644+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:26.663 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T21:41:26.715 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:26.716 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:26.724 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T21:41:26.738 INFO:teuthology.orchestra.run.smithi164.stdout:truncating image image_client.0-clone19 from 0x40be600 (overlap 0x708400) to 0x654200 2024-09-21T21:41:26.746 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:26.744+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:26.837 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:26.838 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:26.847 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T21:41:26.848 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:26.844+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:26.948 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:26.945+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:26.954 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:26.955 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:26.964 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T21:41:27.033 INFO:teuthology.orchestra.run.smithi164.stdout:checking clone #18, image image_client.0-clone18 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent19 2024-09-21T21:41:27.049 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:27.047+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:27.069 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:27.069 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:27.078 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T21:41:27.161 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:27.160+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:27.225 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:27.225 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:27.234 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T21:41:27.276 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:27.274+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:27.352 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:27.350+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:27.383 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:27.383 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:27.392 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T21:41:27.452 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:27.450+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:27.543 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:27.543 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:27.553 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T21:41:27.554 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:27.551+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:27.654 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:27.652+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:27.662 INFO:teuthology.orchestra.run.smithi081.stderr:nodeep-scrub is set 2024-09-21T21:41:27.697 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T21:41:27.699 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:27.700 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:27.709 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T21:41:27.792 INFO:teuthology.orchestra.run.smithi164.stdout:3062 read 0x217a800 thru 0x2188721 (0xdf22 bytes) 2024-09-21T21:41:27.793 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:27.791+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:27.796 INFO:teuthology.orchestra.run.smithi164.stdout:3064 read 0x3ac0e00 thru 0x3ac3f40 (0x3141 bytes) 2024-09-21T21:41:27.797 INFO:teuthology.orchestra.run.smithi164.stdout:3065 read 0x3231c00 thru 0x323b134 (0x9535 bytes) 2024-09-21T21:41:27.798 INFO:teuthology.orchestra.run.smithi164.stdout:3066 write 0x964800 thru 0x96c68c (0x7e8d bytes) 2024-09-21T21:41:27.799 INFO:teuthology.orchestra.run.smithi164.stdout:3067 punch from 0x1e8e200 to 0x1e96400, (0x8200 bytes) 2024-09-21T21:41:27.820 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0: {} 2024-09-21T21:41:27.820 INFO:teuthology.orchestra.run.smithi081.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T21:41:27.824 INFO:teuthology.orchestra.run.smithi164.stdout:3069 flatten 2024-09-21T21:41:27.842 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1: {} 2024-09-21T21:41:27.842 INFO:teuthology.orchestra.run.smithi081.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T21:41:27.852 INFO:teuthology.orchestra.run.smithi164.stdout:3070 trunc from 0x40be600 to 0x5cc3200 2024-09-21T21:41:27.856 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:27.854+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:27.860 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2: {} 2024-09-21T21:41:27.860 INFO:teuthology.orchestra.run.smithi081.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T21:41:27.862 INFO:teuthology.orchestra.run.smithi164.stdout:3072 write 0x4a70c00 thru 0x4a72bf7 (0x1ff8 bytes) 2024-09-21T21:41:27.863 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:27.863 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:41:27.864 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:41:27.864 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:41:27.864 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:27.873 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T21:41:27.881 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3: {} 2024-09-21T21:41:27.881 INFO:teuthology.orchestra.run.smithi081.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T21:41:27.884 INFO:teuthology.orchestra.run.smithi164.stdout:3075 write 0x979d800 thru 0x979fb17 (0x2318 bytes) 2024-09-21T21:41:27.889 INFO:teuthology.orchestra.run.smithi164.stdout:3076 punch from 0x284200 to 0x286800, (0x2600 bytes) 2024-09-21T21:41:27.890 INFO:teuthology.orchestra.run.smithi164.stdout:3077 read 0x2f11a00 thru 0x2f14063 (0x2664 bytes) 2024-09-21T21:41:27.893 INFO:teuthology.orchestra.run.smithi164.stdout:3078 write 0x96b8000 thru 0x96bce61 (0x4e62 bytes) 2024-09-21T21:41:27.899 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4: {} 2024-09-21T21:41:27.899 INFO:teuthology.orchestra.run.smithi081.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T21:41:27.910 INFO:teuthology.orchestra.run.smithi164.stdout:3081 write 0xcb17800 thru 0xcb20578 (0x8d79 bytes) 2024-09-21T21:41:27.913 INFO:teuthology.orchestra.run.smithi164.stdout:3082 trunc from 0xcb20600 to 0xb18d400 2024-09-21T21:41:27.920 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5: {} 2024-09-21T21:41:27.920 INFO:teuthology.orchestra.run.smithi081.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T21:41:27.925 INFO:teuthology.orchestra.run.smithi164.stdout:3083 punch from 0x2fe8400 to 0x2ff0200, (0x7e00 bytes) 2024-09-21T21:41:27.928 INFO:teuthology.orchestra.run.smithi164.stdout:3084 read 0x1e66200 thru 0x1e737e8 (0xd5e9 bytes) 2024-09-21T21:41:27.929 INFO:teuthology.orchestra.run.smithi164.stdout:3085 read 0x19ff200 thru 0x1a08c19 (0x9a1a bytes) 2024-09-21T21:41:27.931 INFO:teuthology.orchestra.run.smithi164.stdout:3087 trunc from 0xb18d400 to 0x9d08c00 2024-09-21T21:41:27.952 INFO:teuthology.orchestra.run.smithi164.stdout:3088 trunc from 0x9d08c00 to 0x8f6600 2024-09-21T21:41:27.973 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:27.972+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:27.999 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:27.999 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 20, 2024-09-21T21:41:27.999 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 600, 2024-09-21T21:41:27.999 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:41:27.999 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:27.999 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:1678 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 14483456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:27.999 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.299288+0000", 2024-09-21T21:41:28.000 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.6975213689999999, 2024-09-21T21:41:28.000 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031376253, 2024-09-21T21:41:28.000 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:28.000 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:28.000 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:28.000 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:28.000 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:28.000 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:28.000 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1678 2024-09-21T21:41:28.000 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.000 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:28.000 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.000 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:28.000 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299288+0000", 2024-09-21T21:41:28.001 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:28.001 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.001 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.001 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:28.001 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299288+0000", 2024-09-21T21:41:28.001 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:28.001 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.001 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.001 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:28.001 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299288+0000", 2024-09-21T21:41:28.001 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.2500000000000001e-07 2024-09-21T21:41:28.001 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.001 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.001 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:28.002 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299290+0000", 2024-09-21T21:41:28.002 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.945e-06 2024-09-21T21:41:28.002 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.002 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.002 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:28.002 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299295+0000", 2024-09-21T21:41:28.002 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0379999999999999e-06 2024-09-21T21:41:28.002 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.002 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.002 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:28.002 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299299+0000", 2024-09-21T21:41:28.002 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7960000000000002e-06 2024-09-21T21:41:28.002 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.002 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.002 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:28.003 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.330586+0000", 2024-09-21T21:41:28.003 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031287540000000003 2024-09-21T21:41:28.003 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.003 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.003 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:28.003 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.330620+0000", 2024-09-21T21:41:28.003 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3492999999999999e-05 2024-09-21T21:41:28.003 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.003 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.003 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:28.003 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.330664+0000", 2024-09-21T21:41:28.003 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4116000000000002e-05 2024-09-21T21:41:28.003 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:28.003 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:28.003 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:28.004 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.004 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.004 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:1679 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 14942208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:28.004 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.299333+0000", 2024-09-21T21:41:28.004 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.6974757139999999, 2024-09-21T21:41:28.004 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031433567000000003, 2024-09-21T21:41:28.004 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:28.004 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:28.004 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:28.004 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:28.004 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:28.004 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:28.004 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1679 2024-09-21T21:41:28.004 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.004 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:28.005 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.005 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:28.005 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299333+0000", 2024-09-21T21:41:28.005 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:28.005 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.005 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.005 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:28.005 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299333+0000", 2024-09-21T21:41:28.005 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:28.005 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.005 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.005 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:28.005 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299334+0000", 2024-09-21T21:41:28.006 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0100000000000005e-07 2024-09-21T21:41:28.006 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.006 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.006 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:28.006 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299336+0000", 2024-09-21T21:41:28.006 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3439999999999999e-06 2024-09-21T21:41:28.006 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.006 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.006 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:28.006 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299343+0000", 2024-09-21T21:41:28.006 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0299999999999996e-06 2024-09-21T21:41:28.006 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.006 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.007 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:28.007 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299348+0000", 2024-09-21T21:41:28.007 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2340000000000002e-06 2024-09-21T21:41:28.007 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.007 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.007 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:28.007 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.330687+0000", 2024-09-21T21:41:28.007 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031338132999999997 2024-09-21T21:41:28.007 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.007 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.007 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:28.007 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.330721+0000", 2024-09-21T21:41:28.007 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4156999999999998e-05 2024-09-21T21:41:28.008 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.008 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.008 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:28.008 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.330767+0000", 2024-09-21T21:41:28.008 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6168000000000002e-05 2024-09-21T21:41:28.008 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:28.008 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:28.008 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:28.008 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.008 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.008 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:1680 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 15400960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:28.008 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.299356+0000", 2024-09-21T21:41:28.008 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.6974533640000002, 2024-09-21T21:41:28.008 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031510096000000001, 2024-09-21T21:41:28.009 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:28.009 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:28.009 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:28.009 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:28.009 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:28.009 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:28.009 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1680 2024-09-21T21:41:28.009 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.009 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:28.009 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.009 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:28.009 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299356+0000", 2024-09-21T21:41:28.010 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:28.010 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.010 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.010 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:28.010 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299356+0000", 2024-09-21T21:41:28.010 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:28.010 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.010 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.010 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:28.010 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299356+0000", 2024-09-21T21:41:28.010 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.63e-07 2024-09-21T21:41:28.010 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.011 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.011 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:28.011 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299358+0000", 2024-09-21T21:41:28.011 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1399999999999998e-06 2024-09-21T21:41:28.011 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.011 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.011 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:28.011 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299364+0000", 2024-09-21T21:41:28.011 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1650000000000003e-06 2024-09-21T21:41:28.011 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.011 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.011 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:28.011 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299369+0000", 2024-09-21T21:41:28.011 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8400000000000002e-06 2024-09-21T21:41:28.012 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.012 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.012 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:28.012 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.330788+0000", 2024-09-21T21:41:28.012 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031418399 2024-09-21T21:41:28.012 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.012 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.012 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:28.012 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.330822+0000", 2024-09-21T21:41:28.012 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4218999999999999e-05 2024-09-21T21:41:28.012 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.012 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.012 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:28.013 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.330866+0000", 2024-09-21T21:41:28.013 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3869999999999998e-05 2024-09-21T21:41:28.013 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:28.013 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:28.013 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:28.013 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.013 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.013 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:1681 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 15859712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:28.013 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.299375+0000", 2024-09-21T21:41:28.013 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.6974336599999997, 2024-09-21T21:41:28.013 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031600251000000003, 2024-09-21T21:41:28.013 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:28.013 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:28.014 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:28.014 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:28.014 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:28.014 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:28.014 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1681 2024-09-21T21:41:28.014 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.014 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:28.014 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.014 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:28.014 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299375+0000", 2024-09-21T21:41:28.014 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:28.014 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.014 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.015 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:28.015 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299375+0000", 2024-09-21T21:41:28.015 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:28.015 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.015 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.015 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:28.015 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299376+0000", 2024-09-21T21:41:28.015 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8700000000000001e-07 2024-09-21T21:41:28.015 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.015 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.015 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:28.015 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299382+0000", 2024-09-21T21:41:28.015 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3060000000000003e-06 2024-09-21T21:41:28.016 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.016 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.016 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:28.016 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299389+0000", 2024-09-21T21:41:28.016 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.3629999999999996e-06 2024-09-21T21:41:28.016 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.016 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.016 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:28.016 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299395+0000", 2024-09-21T21:41:28.016 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1279999999999999e-06 2024-09-21T21:41:28.016 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.016 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.016 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:28.016 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.330891+0000", 2024-09-21T21:41:28.016 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031496108000000002 2024-09-21T21:41:28.017 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.017 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.017 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:28.017 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.330925+0000", 2024-09-21T21:41:28.017 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4150000000000003e-05 2024-09-21T21:41:28.017 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.017 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.017 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:28.017 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.330976+0000", 2024-09-21T21:41:28.017 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0809000000000001e-05 2024-09-21T21:41:28.017 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:28.017 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:28.017 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:28.018 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.018 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.018 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:1682 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 16318464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:28.018 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.299401+0000", 2024-09-21T21:41:28.018 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.6974080159999998, 2024-09-21T21:41:28.018 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031676932999999997, 2024-09-21T21:41:28.018 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:28.018 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:28.018 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:28.018 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:28.018 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:28.018 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:28.018 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1682 2024-09-21T21:41:28.019 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.019 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:28.019 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.019 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:28.019 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299401+0000", 2024-09-21T21:41:28.019 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:28.019 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.019 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.019 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:28.019 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299401+0000", 2024-09-21T21:41:28.019 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:28.019 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.019 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.019 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:28.020 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299401+0000", 2024-09-21T21:41:28.020 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.9299999999999999e-07 2024-09-21T21:41:28.020 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.020 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.020 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:28.020 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299404+0000", 2024-09-21T21:41:28.020 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1720000000000001e-06 2024-09-21T21:41:28.020 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.020 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.020 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:28.020 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299410+0000", 2024-09-21T21:41:28.020 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2480000000000001e-06 2024-09-21T21:41:28.020 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.020 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.020 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:28.020 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299414+0000", 2024-09-21T21:41:28.021 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6480000000000002e-06 2024-09-21T21:41:28.021 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.021 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.021 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:28.021 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.330998+0000", 2024-09-21T21:41:28.021 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031583364000000003 2024-09-21T21:41:28.021 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.021 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.021 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:28.021 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.331033+0000", 2024-09-21T21:41:28.021 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4759999999999999e-05 2024-09-21T21:41:28.021 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.021 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.022 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:28.022 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.331078+0000", 2024-09-21T21:41:28.022 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5348e-05 2024-09-21T21:41:28.022 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:28.022 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:28.022 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:28.022 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.022 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.022 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:2010 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 7864320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:28.022 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.306462+0000", 2024-09-21T21:41:28.022 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.69034693, 2024-09-21T21:41:28.022 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031317032000000002, 2024-09-21T21:41:28.022 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:28.022 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:28.023 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:28.023 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:28.023 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:28.023 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:28.023 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2010 2024-09-21T21:41:28.023 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.023 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:28.023 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.023 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:28.023 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306462+0000", 2024-09-21T21:41:28.023 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:28.023 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.023 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.024 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:28.024 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306462+0000", 2024-09-21T21:41:28.024 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:28.024 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.024 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.024 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:28.024 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306463+0000", 2024-09-21T21:41:28.024 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2500000000000001e-07 2024-09-21T21:41:28.024 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.024 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.024 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:28.024 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306465+0000", 2024-09-21T21:41:28.024 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9740000000000001e-06 2024-09-21T21:41:28.025 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.025 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.025 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:28.025 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306470+0000", 2024-09-21T21:41:28.025 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9689999999999999e-06 2024-09-21T21:41:28.025 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.025 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.025 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:28.025 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306475+0000", 2024-09-21T21:41:28.025 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7199999999999997e-06 2024-09-21T21:41:28.025 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.025 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.025 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:28.025 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.337701+0000", 2024-09-21T21:41:28.026 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031225796 2024-09-21T21:41:28.026 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.026 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.026 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:28.026 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.337735+0000", 2024-09-21T21:41:28.026 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4399e-05 2024-09-21T21:41:28.026 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.026 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.026 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:28.026 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.337779+0000", 2024-09-21T21:41:28.027 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3748999999999997e-05 2024-09-21T21:41:28.027 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:28.027 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:28.027 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:28.027 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.027 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.027 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:2011 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 8323072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:28.027 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.306481+0000", 2024-09-21T21:41:28.027 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.6903278259999999, 2024-09-21T21:41:28.027 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031404840000000003, 2024-09-21T21:41:28.027 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:28.027 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:28.028 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:28.028 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:28.028 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:28.028 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:28.028 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2011 2024-09-21T21:41:28.028 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.028 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:28.028 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.028 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:28.028 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306481+0000", 2024-09-21T21:41:28.028 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:28.028 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.028 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.029 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:28.029 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306481+0000", 2024-09-21T21:41:28.029 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:28.029 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.029 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.029 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:28.029 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306482+0000", 2024-09-21T21:41:28.029 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9100000000000004e-07 2024-09-21T21:41:28.029 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.029 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.029 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:28.029 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306484+0000", 2024-09-21T21:41:28.029 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.855e-06 2024-09-21T21:41:28.030 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.030 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.030 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:28.030 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306489+0000", 2024-09-21T21:41:28.030 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7799999999999997e-06 2024-09-21T21:41:28.030 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.030 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.030 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:28.030 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306494+0000", 2024-09-21T21:41:28.030 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7539999999999997e-06 2024-09-21T21:41:28.030 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.030 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.030 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:28.031 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.337802+0000", 2024-09-21T21:41:28.031 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031307684000000002 2024-09-21T21:41:28.031 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.031 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.031 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:28.031 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.337836+0000", 2024-09-21T21:41:28.031 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4560000000000001e-05 2024-09-21T21:41:28.031 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.031 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.031 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:28.031 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.337886+0000", 2024-09-21T21:41:28.031 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9716000000000003e-05 2024-09-21T21:41:28.031 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:28.031 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:28.032 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:28.032 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.032 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.032 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:2012 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 8781824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:28.032 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.306500+0000", 2024-09-21T21:41:28.032 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.6903090110000001, 2024-09-21T21:41:28.032 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031483165, 2024-09-21T21:41:28.032 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:28.032 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:28.032 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:28.032 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:28.032 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:28.032 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:28.032 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2012 2024-09-21T21:41:28.032 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.033 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:28.033 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.033 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:28.033 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306500+0000", 2024-09-21T21:41:28.033 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:28.033 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.033 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.033 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:28.033 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306500+0000", 2024-09-21T21:41:28.033 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:28.033 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.033 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.033 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:28.033 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306500+0000", 2024-09-21T21:41:28.033 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7099999999999997e-07 2024-09-21T21:41:28.033 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.034 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.034 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:28.034 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306502+0000", 2024-09-21T21:41:28.034 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1019999999999999e-06 2024-09-21T21:41:28.034 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.034 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.034 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:28.034 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306508+0000", 2024-09-21T21:41:28.034 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8390000000000003e-06 2024-09-21T21:41:28.034 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.034 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.034 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:28.034 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306513+0000", 2024-09-21T21:41:28.034 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4909999999999999e-06 2024-09-21T21:41:28.035 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.035 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.035 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:28.035 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.337898+0000", 2024-09-21T21:41:28.035 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031385437000000002 2024-09-21T21:41:28.035 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.035 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.035 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:28.035 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.337931+0000", 2024-09-21T21:41:28.035 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.2656e-05 2024-09-21T21:41:28.035 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.035 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.035 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:28.035 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.337983+0000", 2024-09-21T21:41:28.035 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2268999999999997e-05 2024-09-21T21:41:28.036 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:28.036 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:28.036 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:28.036 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.036 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.036 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:2013 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 9240576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:28.036 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.306520+0000", 2024-09-21T21:41:28.036 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.6902892830000003, 2024-09-21T21:41:28.036 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031570864999999997, 2024-09-21T21:41:28.036 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:28.036 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:28.036 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:28.036 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:28.036 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:28.036 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:28.037 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2013 2024-09-21T21:41:28.037 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.037 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:28.037 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.037 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:28.037 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306520+0000", 2024-09-21T21:41:28.037 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:28.037 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.037 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.037 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:28.037 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306520+0000", 2024-09-21T21:41:28.037 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:28.037 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.037 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.037 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:28.037 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306520+0000", 2024-09-21T21:41:28.038 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3700000000000001e-07 2024-09-21T21:41:28.038 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.038 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.038 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:28.038 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306530+0000", 2024-09-21T21:41:28.038 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0336000000000001e-05 2024-09-21T21:41:28.038 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.038 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.038 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:28.038 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306538+0000", 2024-09-21T21:41:28.038 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.3690000000000001e-06 2024-09-21T21:41:28.038 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.038 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.039 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:28.039 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306543+0000", 2024-09-21T21:41:28.039 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8930000000000004e-06 2024-09-21T21:41:28.039 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.039 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.039 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:28.039 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338005+0000", 2024-09-21T21:41:28.039 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031461973999999997 2024-09-21T21:41:28.039 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.039 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.039 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:28.039 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338039+0000", 2024-09-21T21:41:28.039 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4545999999999998e-05 2024-09-21T21:41:28.039 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.040 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.040 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:28.040 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338091+0000", 2024-09-21T21:41:28.040 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1409999999999997e-05 2024-09-21T21:41:28.040 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:28.040 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:28.040 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:28.040 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.040 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.040 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:2014 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 9699328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:28.040 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.306553+0000", 2024-09-21T21:41:28.040 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.69025633, 2024-09-21T21:41:28.040 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031642225000000003, 2024-09-21T21:41:28.040 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:28.041 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:28.041 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:28.041 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:28.041 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:28.041 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:28.041 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2014 2024-09-21T21:41:28.041 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.041 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:28.041 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.041 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:28.041 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306553+0000", 2024-09-21T21:41:28.041 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:28.041 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.041 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.042 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:28.042 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306553+0000", 2024-09-21T21:41:28.042 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:28.042 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.042 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.042 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:28.042 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306553+0000", 2024-09-21T21:41:28.042 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5600000000000001e-07 2024-09-21T21:41:28.042 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.042 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.042 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:28.042 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306555+0000", 2024-09-21T21:41:28.043 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3070000000000001e-06 2024-09-21T21:41:28.043 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.043 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.043 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:28.043 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306561+0000", 2024-09-21T21:41:28.043 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8860000000000001e-06 2024-09-21T21:41:28.043 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.043 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.043 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:28.043 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306566+0000", 2024-09-21T21:41:28.043 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3420000000000001e-06 2024-09-21T21:41:28.043 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.043 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.043 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:28.044 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338115+0000", 2024-09-21T21:41:28.044 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031549333999999998 2024-09-21T21:41:28.044 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.044 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.044 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:28.044 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338150+0000", 2024-09-21T21:41:28.044 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4649e-05 2024-09-21T21:41:28.044 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.044 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.044 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:28.044 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338195+0000", 2024-09-21T21:41:28.044 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5250999999999998e-05 2024-09-21T21:41:28.044 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:28.044 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:28.044 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:28.045 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.045 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.045 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:2015 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 10158080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:28.045 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.306572+0000", 2024-09-21T21:41:28.045 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.6902374120000001, 2024-09-21T21:41:28.045 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031729593, 2024-09-21T21:41:28.045 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:28.045 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:28.045 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:28.045 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:28.045 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:28.045 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:28.045 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2015 2024-09-21T21:41:28.046 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.046 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:28.046 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.046 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:28.046 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306572+0000", 2024-09-21T21:41:28.046 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:28.046 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.046 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.046 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:28.046 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306572+0000", 2024-09-21T21:41:28.046 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:28.046 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.046 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.046 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:28.047 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306572+0000", 2024-09-21T21:41:28.047 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7500000000000001e-07 2024-09-21T21:41:28.047 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.047 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.047 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:28.047 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306574+0000", 2024-09-21T21:41:28.047 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0669999999999999e-06 2024-09-21T21:41:28.047 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.047 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.047 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:28.047 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306579+0000", 2024-09-21T21:41:28.047 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4009999999999996e-06 2024-09-21T21:41:28.047 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.047 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.048 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:28.048 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306585+0000", 2024-09-21T21:41:28.048 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3349999999999997e-06 2024-09-21T21:41:28.048 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.048 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.048 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:28.048 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338217+0000", 2024-09-21T21:41:28.048 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031631805999999998 2024-09-21T21:41:28.048 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.048 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.048 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:28.048 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338251+0000", 2024-09-21T21:41:28.048 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4813e-05 2024-09-21T21:41:28.048 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.049 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.049 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:28.049 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338301+0000", 2024-09-21T21:41:28.049 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9796000000000002e-05 2024-09-21T21:41:28.049 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:28.049 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:28.049 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:28.049 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.049 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.049 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:2016 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 10616832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:28.049 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.306601+0000", 2024-09-21T21:41:28.049 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.6902082389999999, 2024-09-21T21:41:28.049 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031801840999999997, 2024-09-21T21:41:28.049 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:28.049 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:28.050 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:28.050 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:28.050 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:28.050 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:28.050 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2016 2024-09-21T21:41:28.050 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.050 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:28.050 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.050 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:28.050 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306601+0000", 2024-09-21T21:41:28.050 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:28.050 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.050 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.051 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:28.051 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306601+0000", 2024-09-21T21:41:28.051 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:28.051 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.051 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.051 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:28.051 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306601+0000", 2024-09-21T21:41:28.051 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1900000000000003e-07 2024-09-21T21:41:28.051 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.051 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.051 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:28.051 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306603+0000", 2024-09-21T21:41:28.051 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.17e-06 2024-09-21T21:41:28.052 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.052 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.052 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:28.052 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306609+0000", 2024-09-21T21:41:28.052 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8069999999999996e-06 2024-09-21T21:41:28.052 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.052 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.052 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:28.052 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306614+0000", 2024-09-21T21:41:28.052 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5090000000000004e-06 2024-09-21T21:41:28.052 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.052 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.052 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:28.052 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338325+0000", 2024-09-21T21:41:28.053 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031711237000000003 2024-09-21T21:41:28.053 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.053 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.053 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:28.053 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338359+0000", 2024-09-21T21:41:28.053 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3810000000000003e-05 2024-09-21T21:41:28.053 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.053 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.053 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:28.053 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338403+0000", 2024-09-21T21:41:28.053 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3788999999999997e-05 2024-09-21T21:41:28.053 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:28.053 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:28.053 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:28.053 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.054 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.054 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:2017 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 11075584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:28.054 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.306621+0000", 2024-09-21T21:41:28.054 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.6901884730000001, 2024-09-21T21:41:28.054 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031891271999999998, 2024-09-21T21:41:28.054 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:28.054 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:28.054 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:28.054 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:28.054 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:28.054 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:28.054 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2017 2024-09-21T21:41:28.055 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.055 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:28.055 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.055 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:28.055 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306621+0000", 2024-09-21T21:41:28.055 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:28.055 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.055 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.055 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:28.055 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306621+0000", 2024-09-21T21:41:28.055 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:28.055 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.055 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.055 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:28.056 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306621+0000", 2024-09-21T21:41:28.056 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5400000000000002e-07 2024-09-21T21:41:28.056 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.056 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.056 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:28.056 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306623+0000", 2024-09-21T21:41:28.056 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1100000000000001e-06 2024-09-21T21:41:28.056 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.056 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.056 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:28.056 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306629+0000", 2024-09-21T21:41:28.056 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7810000000000002e-06 2024-09-21T21:41:28.056 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.056 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.057 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:28.057 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306633+0000", 2024-09-21T21:41:28.057 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5589999999999999e-06 2024-09-21T21:41:28.057 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.057 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.057 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:28.057 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338427+0000", 2024-09-21T21:41:28.057 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031793948000000002 2024-09-21T21:41:28.057 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.057 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.057 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:28.057 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338462+0000", 2024-09-21T21:41:28.057 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4785e-05 2024-09-21T21:41:28.058 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.058 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.058 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:28.058 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338512+0000", 2024-09-21T21:41:28.058 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9734999999999997e-05 2024-09-21T21:41:28.058 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:28.058 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:28.058 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:28.058 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.058 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.058 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:2018 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 11534336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:28.058 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.306639+0000", 2024-09-21T21:41:28.058 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.6901698440000001, 2024-09-21T21:41:28.058 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031963189000000003, 2024-09-21T21:41:28.059 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:28.059 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:28.059 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:28.059 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:28.059 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:28.059 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:28.059 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2018 2024-09-21T21:41:28.059 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.059 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:28.059 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.059 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:28.059 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306639+0000", 2024-09-21T21:41:28.059 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:28.059 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.060 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.060 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:28.060 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306639+0000", 2024-09-21T21:41:28.060 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:28.060 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.060 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.060 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:28.060 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306640+0000", 2024-09-21T21:41:28.060 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4600000000000005e-07 2024-09-21T21:41:28.060 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.060 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.060 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:28.060 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306642+0000", 2024-09-21T21:41:28.060 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.916e-06 2024-09-21T21:41:28.061 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.061 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.061 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:28.061 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306647+0000", 2024-09-21T21:41:28.061 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7590000000000003e-06 2024-09-21T21:41:28.061 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.061 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.061 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:28.061 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306652+0000", 2024-09-21T21:41:28.061 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4580000000000003e-06 2024-09-21T21:41:28.061 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.061 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.061 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:28.061 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338524+0000", 2024-09-21T21:41:28.062 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031872172999999997 2024-09-21T21:41:28.062 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.062 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.062 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:28.062 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338557+0000", 2024-09-21T21:41:28.062 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3136000000000003e-05 2024-09-21T21:41:28.062 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.062 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.062 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:28.062 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338602+0000", 2024-09-21T21:41:28.062 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5201000000000003e-05 2024-09-21T21:41:28.062 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:28.062 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:28.062 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:28.063 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.063 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.063 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:2019 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 11993088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:28.063 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.306658+0000", 2024-09-21T21:41:28.063 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.6901514500000001, 2024-09-21T21:41:28.063 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.032059007, 2024-09-21T21:41:28.063 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:28.063 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:28.063 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:28.063 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:28.063 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:28.063 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:28.063 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2019 2024-09-21T21:41:28.063 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.064 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:28.064 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.064 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:28.064 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306658+0000", 2024-09-21T21:41:28.064 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:28.064 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.064 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.064 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:28.064 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306658+0000", 2024-09-21T21:41:28.064 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:28.064 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.064 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.064 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:28.064 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.307950+0000", 2024-09-21T21:41:28.065 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.001292199 2024-09-21T21:41:28.065 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.065 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.065 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:28.065 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.307954+0000", 2024-09-21T21:41:28.065 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5689999999999998e-06 2024-09-21T21:41:28.065 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.065 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.065 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:28.065 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.307964+0000", 2024-09-21T21:41:28.065 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6749999999999997e-06 2024-09-21T21:41:28.065 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.065 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.065 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:28.066 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.307971+0000", 2024-09-21T21:41:28.066 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5189999999999998e-06 2024-09-21T21:41:28.066 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.066 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.066 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:28.066 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338625+0000", 2024-09-21T21:41:28.066 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.030654569 2024-09-21T21:41:28.066 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.066 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.066 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:28.066 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338660+0000", 2024-09-21T21:41:28.066 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4656999999999997e-05 2024-09-21T21:41:28.066 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.067 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.067 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:28.067 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338717+0000", 2024-09-21T21:41:28.067 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6818999999999999e-05 2024-09-21T21:41:28.067 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:28.067 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:28.067 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:28.067 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.067 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.067 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:2026 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 15204352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:28.067 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.308109+0000", 2024-09-21T21:41:28.067 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.6886998499999999, 2024-09-21T21:41:28.067 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031315883000000003, 2024-09-21T21:41:28.067 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:28.068 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:28.068 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:28.068 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:28.068 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:28.068 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:28.068 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2026 2024-09-21T21:41:28.068 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.068 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:28.068 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.068 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:28.068 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308109+0000", 2024-09-21T21:41:28.068 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:28.068 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.069 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.069 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:28.069 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308109+0000", 2024-09-21T21:41:28.069 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:28.069 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.069 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.069 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:28.069 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308110+0000", 2024-09-21T21:41:28.069 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.96e-07 2024-09-21T21:41:28.069 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.069 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.069 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:28.069 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308112+0000", 2024-09-21T21:41:28.069 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0269999999999998e-06 2024-09-21T21:41:28.070 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.070 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.070 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:28.070 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308117+0000", 2024-09-21T21:41:28.070 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.631e-06 2024-09-21T21:41:28.070 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.070 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.070 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:28.070 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308122+0000", 2024-09-21T21:41:28.070 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7310000000000001e-06 2024-09-21T21:41:28.070 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.070 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.070 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:28.071 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.339341+0000", 2024-09-21T21:41:28.071 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031218610000000001 2024-09-21T21:41:28.071 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.071 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.071 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:28.071 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.339376+0000", 2024-09-21T21:41:28.071 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5305000000000002e-05 2024-09-21T21:41:28.071 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.071 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.071 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:28.071 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.339425+0000", 2024-09-21T21:41:28.071 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9183e-05 2024-09-21T21:41:28.071 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:28.072 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:28.072 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:28.072 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.072 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.072 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:2027 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 15663104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:28.072 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.308128+0000", 2024-09-21T21:41:28.072 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.6886813140000001, 2024-09-21T21:41:28.072 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031387452000000003, 2024-09-21T21:41:28.072 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:28.072 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:28.072 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:28.072 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:28.072 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:28.072 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:28.073 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2027 2024-09-21T21:41:28.073 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.073 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:28.073 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.073 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:28.073 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308128+0000", 2024-09-21T21:41:28.073 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:28.073 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.073 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.073 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:28.073 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308128+0000", 2024-09-21T21:41:28.073 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:28.073 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.073 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.074 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:28.074 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308128+0000", 2024-09-21T21:41:28.074 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4700000000000002e-07 2024-09-21T21:41:28.074 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.074 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.074 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:28.074 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308130+0000", 2024-09-21T21:41:28.074 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.8670000000000001e-06 2024-09-21T21:41:28.074 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.074 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.074 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:28.074 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308136+0000", 2024-09-21T21:41:28.074 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0299999999999999e-06 2024-09-21T21:41:28.075 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.075 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.075 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:28.075 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308140+0000", 2024-09-21T21:41:28.075 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3830000000000002e-06 2024-09-21T21:41:28.075 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.075 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.075 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:28.075 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.339437+0000", 2024-09-21T21:41:28.075 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031296864000000001 2024-09-21T21:41:28.075 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.075 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.075 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:28.075 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.339471+0000", 2024-09-21T21:41:28.075 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3302000000000001e-05 2024-09-21T21:41:28.076 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.076 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.076 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:28.076 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.339515+0000", 2024-09-21T21:41:28.076 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4659000000000001e-05 2024-09-21T21:41:28.076 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:28.076 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:28.076 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:28.076 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.076 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.076 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:2028 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 16121856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:28.076 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.308147+0000", 2024-09-21T21:41:28.076 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.6886623869999999, 2024-09-21T21:41:28.076 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031474477000000001, 2024-09-21T21:41:28.077 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:28.077 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:28.077 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:28.077 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:28.077 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:28.077 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:28.077 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2028 2024-09-21T21:41:28.077 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.077 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:28.077 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.077 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:28.077 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308147+0000", 2024-09-21T21:41:28.077 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:28.077 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.077 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.078 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:28.078 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308147+0000", 2024-09-21T21:41:28.078 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:28.078 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.078 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.078 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:28.078 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308147+0000", 2024-09-21T21:41:28.078 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1900000000000003e-07 2024-09-21T21:41:28.078 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.078 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.078 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:28.078 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308149+0000", 2024-09-21T21:41:28.078 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9479999999999998e-06 2024-09-21T21:41:28.078 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.079 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.079 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:28.079 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308155+0000", 2024-09-21T21:41:28.079 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6200000000000004e-06 2024-09-21T21:41:28.079 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.079 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.079 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:28.079 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308160+0000", 2024-09-21T21:41:28.079 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0799999999999996e-06 2024-09-21T21:41:28.079 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.079 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.079 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:28.079 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.339539+0000", 2024-09-21T21:41:28.079 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031378902 2024-09-21T21:41:28.080 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.080 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.080 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:28.080 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.339574+0000", 2024-09-21T21:41:28.080 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5176999999999999e-05 2024-09-21T21:41:28.080 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.080 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.080 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:28.080 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.339621+0000", 2024-09-21T21:41:28.080 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7231000000000002e-05 2024-09-21T21:41:28.080 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:28.080 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:28.080 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:28.081 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.081 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.081 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:2029 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 16580608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:28.081 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.308167+0000", 2024-09-21T21:41:28.081 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.688641998, 2024-09-21T21:41:28.081 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031555582999999998, 2024-09-21T21:41:28.081 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:28.081 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:28.081 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:28.081 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:28.081 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:28.081 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:28.081 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2029 2024-09-21T21:41:28.081 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.082 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:28.082 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.082 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:28.082 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308167+0000", 2024-09-21T21:41:28.082 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:28.082 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.082 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.082 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:28.082 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308167+0000", 2024-09-21T21:41:28.082 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:28.082 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.082 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.082 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:28.082 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308167+0000", 2024-09-21T21:41:28.083 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6600000000000002e-07 2024-09-21T21:41:28.083 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.083 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.083 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:28.083 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308169+0000", 2024-09-21T21:41:28.083 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9819999999999998e-06 2024-09-21T21:41:28.083 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.083 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.083 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:28.083 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308175+0000", 2024-09-21T21:41:28.083 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8289999999999996e-06 2024-09-21T21:41:28.083 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.083 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.083 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:28.084 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308180+0000", 2024-09-21T21:41:28.084 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6439999999999999e-06 2024-09-21T21:41:28.084 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.084 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.084 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:28.084 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.339645+0000", 2024-09-21T21:41:28.084 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031465083999999997 2024-09-21T21:41:28.084 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.084 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.084 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:28.084 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.339679+0000", 2024-09-21T21:41:28.084 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3572999999999999e-05 2024-09-21T21:41:28.085 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.085 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.085 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:28.085 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.339723+0000", 2024-09-21T21:41:28.085 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4104999999999999e-05 2024-09-21T21:41:28.085 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:28.085 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:28.085 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:28.085 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.085 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.085 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:2059 3.5 3:a6b41377:::rbd_data.11213654c685.0000000000000008:head [stat,truncate 13631488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:28.085 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.308186+0000", 2024-09-21T21:41:28.085 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.6886234199999999, 2024-09-21T21:41:28.085 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.032997110000000003, 2024-09-21T21:41:28.086 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:28.086 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:28.086 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:41:28.086 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:28.086 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:28.086 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:28.086 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2059 2024-09-21T21:41:28.086 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.086 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:28.086 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.086 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:28.086 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308186+0000", 2024-09-21T21:41:28.086 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:28.086 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.087 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.087 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:28.087 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308186+0000", 2024-09-21T21:41:28.087 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:28.087 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.087 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.087 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:28.087 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308186+0000", 2024-09-21T21:41:28.087 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1899999999999998e-07 2024-09-21T21:41:28.087 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.087 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.087 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:28.087 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308188+0000", 2024-09-21T21:41:28.087 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0099999999999998e-06 2024-09-21T21:41:28.088 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.088 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.088 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:28.088 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308195+0000", 2024-09-21T21:41:28.088 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0589999999999997e-06 2024-09-21T21:41:28.088 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.088 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.088 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:28.088 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308199+0000", 2024-09-21T21:41:28.088 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2930000000000002e-06 2024-09-21T21:41:28.088 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.088 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.088 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:28.088 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.339745+0000", 2024-09-21T21:41:28.089 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031546091999999998 2024-09-21T21:41:28.089 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.089 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.089 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:28.089 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.339782+0000", 2024-09-21T21:41:28.089 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6847999999999997e-05 2024-09-21T21:41:28.089 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.089 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.089 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:41:28.089 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.339856+0000", 2024-09-21T21:41:28.089 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.3992999999999994e-05 2024-09-21T21:41:28.089 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.089 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.089 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:41:28.090 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.341013+0000", 2024-09-21T21:41:28.090 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00115642 2024-09-21T21:41:28.090 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.090 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.090 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:41:28.090 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.341138+0000", 2024-09-21T21:41:28.090 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00012540699999999999 2024-09-21T21:41:28.090 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.090 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.090 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:41:28.090 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.341164+0000", 2024-09-21T21:41:28.090 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.6168000000000001e-05 2024-09-21T21:41:28.090 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:28.090 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:28.091 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:28.091 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.341183+0000", 2024-09-21T21:41:28.091 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.8400999999999999e-05 2024-09-21T21:41:28.091 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:28.091 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:28.091 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:28.091 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:28.091 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:28.091 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:28.093 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T21:41:28.094 INFO:teuthology.orchestra.run.smithi164.stdout:3089 punch from 0x9ae00 to 0xa4400, (0x9600 bytes) 2024-09-21T21:41:28.094 INFO:teuthology.orchestra.run.smithi164.stdout:3090 read 0x109400 thru 0x1100ee (0x6cef bytes) 2024-09-21T21:41:28.094 INFO:teuthology.orchestra.run.smithi164.stdout:3091 read 0x2a8400 thru 0x2ad454 (0x5055 bytes) 2024-09-21T21:41:28.094 INFO:teuthology.orchestra.run.smithi164.stdout:3092 read 0x593e00 thru 0x59af3e (0x713f bytes) 2024-09-21T21:41:28.094 INFO:teuthology.orchestra.run.smithi164.stdout:3093 read 0x187a00 thru 0x196b99 (0xf19a bytes) 2024-09-21T21:41:28.095 INFO:teuthology.orchestra.run.smithi164.stdout:3096 trunc from 0x8f6600 to 0xe3f9400 2024-09-21T21:41:28.095 INFO:teuthology.orchestra.run.smithi164.stdout:3097 write 0x981fa00 thru 0x982b55c (0xbb5d bytes) 2024-09-21T21:41:28.095 INFO:teuthology.orchestra.run.smithi164.stdout:3098 write 0xe6ede00 thru 0xe6f788c (0x9a8d bytes) 2024-09-21T21:41:28.095 INFO:teuthology.orchestra.run.smithi164.stdout:3099 trunc from 0xe6f7a00 to 0x957da00 2024-09-21T21:41:28.095 INFO:teuthology.orchestra.run.smithi164.stdout:3100 read 0x51e8400 thru 0x51f4a76 (0xc677 bytes) 2024-09-21T21:41:28.095 INFO:teuthology.orchestra.run.smithi164.stdout:3101 trunc from 0x957da00 to 0x427fe00 2024-09-21T21:41:28.095 INFO:teuthology.orchestra.run.smithi164.stdout:3102 write 0xf06200 thru 0xf07ad3 (0x18d4 bytes) 2024-09-21T21:41:28.095 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:28.091+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:28.098 INFO:teuthology.orchestra.run.smithi164.stdout:3103 write 0xa88800 thru 0xa8da69 (0x526a bytes) 2024-09-21T21:41:28.099 INFO:teuthology.orchestra.run.smithi164.stdout:3104 clone 21 order 19 su 2048 sc 7 2024-09-21T21:41:28.193 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:28.192+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:28.197 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:28.197 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:41:28.197 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-21T21:41:28.197 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:28.205 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T21:41:28.294 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:28.291+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:28.309 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:28.309 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:41:28.309 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:41:28.309 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:41:28.309 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:28.318 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T21:41:28.394 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:28.392+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:28.462 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:28.463 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-21T21:41:28.463 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-21T21:41:28.463 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:41:28.463 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.463 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2415 3.d 3:b26310db:::rbd_data.112145fae779.0000000000000006:a [read 10690560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:28.463 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.406907+0000", 2024-09-21T21:41:28.463 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 30.053460818000001, 2024-09-21T21:41:28.463 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044424119999999997, 2024-09-21T21:41:28.463 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:28.463 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:28.463 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:28.463 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:28.463 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:28.464 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:28.464 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2415 2024-09-21T21:41:28.464 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.464 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:28.464 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.464 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:28.464 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.406907+0000", 2024-09-21T21:41:28.464 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:28.464 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.464 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.464 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:28.464 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.406907+0000", 2024-09-21T21:41:28.464 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:28.464 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.465 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.465 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:28.465 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.406908+0000", 2024-09-21T21:41:28.465 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7599999999999998e-07 2024-09-21T21:41:28.465 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.465 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.465 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:28.465 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.406915+0000", 2024-09-21T21:41:28.465 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6729999999999992e-06 2024-09-21T21:41:28.465 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.465 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.465 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:28.465 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.406922+0000", 2024-09-21T21:41:28.465 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7139999999999996e-06 2024-09-21T21:41:28.466 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.466 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.466 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:28.466 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.406927+0000", 2024-09-21T21:41:28.466 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.327e-06 2024-09-21T21:41:28.466 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.466 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.466 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:28.466 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.451179+0000", 2024-09-21T21:41:28.466 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044251984000000001 2024-09-21T21:41:28.466 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.466 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.466 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:28.466 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.451254+0000", 2024-09-21T21:41:28.466 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4703999999999995e-05 2024-09-21T21:41:28.467 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.467 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.467 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:28.467 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.451331+0000", 2024-09-21T21:41:28.467 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7342000000000006e-05 2024-09-21T21:41:28.467 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:28.467 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:28.467 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:28.467 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.467 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.467 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2417 3.d 3:b26310db:::rbd_data.112145fae779.0000000000000006:a [read 10252288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:28.467 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.406934+0000", 2024-09-21T21:41:28.467 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 30.053434073999998, 2024-09-21T21:41:28.467 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044511163999999999, 2024-09-21T21:41:28.468 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:28.468 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:28.468 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:28.468 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:28.468 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:28.468 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:28.468 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2417 2024-09-21T21:41:28.468 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.468 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:28.468 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.468 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:28.468 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.406934+0000", 2024-09-21T21:41:28.468 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:28.468 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.468 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.469 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:28.469 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.406934+0000", 2024-09-21T21:41:28.469 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:28.469 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.469 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.469 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:28.469 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.406935+0000", 2024-09-21T21:41:28.469 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6499999999999999e-07 2024-09-21T21:41:28.469 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.469 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.469 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:28.469 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.406937+0000", 2024-09-21T21:41:28.469 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1299999999999999e-06 2024-09-21T21:41:28.469 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.470 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.470 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:28.470 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.406943+0000", 2024-09-21T21:41:28.470 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9989999999999997e-06 2024-09-21T21:41:28.470 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.470 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.470 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:28.470 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.406948+0000", 2024-09-21T21:41:28.470 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9729999999999997e-06 2024-09-21T21:41:28.470 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.470 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.470 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:28.470 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.451365+0000", 2024-09-21T21:41:28.470 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044417808000000003 2024-09-21T21:41:28.471 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.471 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.471 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:28.471 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.451403+0000", 2024-09-21T21:41:28.471 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7944999999999997e-05 2024-09-21T21:41:28.471 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.471 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.471 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:28.471 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.451445+0000", 2024-09-21T21:41:28.471 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1844000000000001e-05 2024-09-21T21:41:28.471 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:28.471 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:28.471 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:28.472 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.472 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.472 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2425 3.d 3:b26310db:::rbd_data.112145fae779.0000000000000006:a [read 10457088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:28.472 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.407639+0000", 2024-09-21T21:41:28.472 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 30.052729372999998, 2024-09-21T21:41:28.472 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044288292999999999, 2024-09-21T21:41:28.472 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:28.472 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:28.472 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:28.472 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:28.472 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:28.472 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:28.472 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2425 2024-09-21T21:41:28.472 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.473 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:28.473 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.473 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:28.473 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407639+0000", 2024-09-21T21:41:28.473 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:28.473 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.473 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.473 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:28.473 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407639+0000", 2024-09-21T21:41:28.473 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:28.473 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.473 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.473 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:28.473 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407639+0000", 2024-09-21T21:41:28.474 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8299999999999998e-07 2024-09-21T21:41:28.474 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.474 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.474 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:28.474 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407641+0000", 2024-09-21T21:41:28.474 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9149999999999999e-06 2024-09-21T21:41:28.474 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.474 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.474 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:28.474 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407647+0000", 2024-09-21T21:41:28.474 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3110000000000002e-06 2024-09-21T21:41:28.474 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.474 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.474 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:28.475 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407653+0000", 2024-09-21T21:41:28.475 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1309999999999997e-06 2024-09-21T21:41:28.475 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.475 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.475 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:28.475 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.451778+0000", 2024-09-21T21:41:28.475 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044125782000000002 2024-09-21T21:41:28.475 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.475 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.475 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:28.475 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.451850+0000", 2024-09-21T21:41:28.475 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1211999999999995e-05 2024-09-21T21:41:28.475 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.475 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.476 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:28.476 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.451927+0000", 2024-09-21T21:41:28.476 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7558999999999994e-05 2024-09-21T21:41:28.476 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:28.476 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:28.476 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:28.476 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.476 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.476 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2427 3.d 3:b26310db:::rbd_data.112145fae779.0000000000000006:a [read 10489856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:28.476 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.407659+0000", 2024-09-21T21:41:28.476 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 30.052709060000002, 2024-09-21T21:41:28.476 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044463186000000002, 2024-09-21T21:41:28.476 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:28.476 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:28.477 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:28.477 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:28.477 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:28.477 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:28.477 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2427 2024-09-21T21:41:28.477 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.477 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:28.477 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.477 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:28.477 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407659+0000", 2024-09-21T21:41:28.477 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:28.477 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.477 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.477 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:28.477 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407659+0000", 2024-09-21T21:41:28.478 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:28.478 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.478 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.478 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:28.478 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407659+0000", 2024-09-21T21:41:28.478 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3000000000000002e-07 2024-09-21T21:41:28.478 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.478 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.478 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:28.478 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407662+0000", 2024-09-21T21:41:28.478 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0980000000000001e-06 2024-09-21T21:41:28.478 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.478 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.478 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:28.478 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407668+0000", 2024-09-21T21:41:28.479 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5209999999999999e-06 2024-09-21T21:41:28.479 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.479 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.479 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:28.479 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407674+0000", 2024-09-21T21:41:28.479 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7590000000000003e-06 2024-09-21T21:41:28.479 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.479 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.479 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:28.479 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.451969+0000", 2024-09-21T21:41:28.479 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044294965999999998 2024-09-21T21:41:28.479 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.480 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.480 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:28.480 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.452045+0000", 2024-09-21T21:41:28.480 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6271000000000002e-05 2024-09-21T21:41:28.480 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.480 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.480 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:28.480 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.452122+0000", 2024-09-21T21:41:28.480 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7241000000000001e-05 2024-09-21T21:41:28.480 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:28.480 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:28.480 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:28.480 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.481 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.481 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2429 3.d 3:b26310db:::rbd_data.112145fae779.0000000000000006:a [read 10547200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:28.481 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.407681+0000", 2024-09-21T21:41:28.481 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 30.052687385999999, 2024-09-21T21:41:28.481 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.04455829, 2024-09-21T21:41:28.481 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:28.481 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:28.481 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:28.481 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:28.481 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:28.481 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:28.481 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2429 2024-09-21T21:41:28.481 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.482 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:28.482 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.482 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:28.482 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407681+0000", 2024-09-21T21:41:28.482 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:28.482 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.482 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.482 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:28.482 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407681+0000", 2024-09-21T21:41:28.482 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:28.482 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.482 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.482 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:28.482 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407681+0000", 2024-09-21T21:41:28.483 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2300000000000001e-07 2024-09-21T21:41:28.483 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.483 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.483 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:28.483 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407683+0000", 2024-09-21T21:41:28.483 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0020000000000001e-06 2024-09-21T21:41:28.483 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.483 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.483 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:28.483 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407690+0000", 2024-09-21T21:41:28.483 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4269999999999996e-06 2024-09-21T21:41:28.483 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.483 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.483 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:28.484 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407694+0000", 2024-09-21T21:41:28.484 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6029999999999998e-06 2024-09-21T21:41:28.484 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.484 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.484 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:28.484 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.452153+0000", 2024-09-21T21:41:28.484 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044458232 2024-09-21T21:41:28.484 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.484 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.484 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:28.484 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.452196+0000", 2024-09-21T21:41:28.484 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3248999999999999e-05 2024-09-21T21:41:28.484 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.484 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.485 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:28.485 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.452239+0000", 2024-09-21T21:41:28.485 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3254000000000003e-05 2024-09-21T21:41:28.485 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:28.485 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:28.485 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:28.485 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.485 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.485 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2431 3.d 3:b26310db:::rbd_data.112145fae779.0000000000000006:a [read 10604544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:28.485 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.407701+0000", 2024-09-21T21:41:28.485 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 30.052667603, 2024-09-21T21:41:28.485 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044639389000000002, 2024-09-21T21:41:28.485 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:28.486 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:28.486 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:28.486 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:28.486 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:28.486 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:28.486 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2431 2024-09-21T21:41:28.486 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.486 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:28.486 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.486 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:28.486 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407701+0000", 2024-09-21T21:41:28.486 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:28.486 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.486 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.487 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:28.487 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407701+0000", 2024-09-21T21:41:28.487 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:28.487 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.487 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.487 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:28.487 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407701+0000", 2024-09-21T21:41:28.487 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.63e-07 2024-09-21T21:41:28.487 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.487 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.487 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:28.487 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407703+0000", 2024-09-21T21:41:28.487 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.979e-06 2024-09-21T21:41:28.487 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.487 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.488 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:28.488 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407710+0000", 2024-09-21T21:41:28.488 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5470000000000002e-06 2024-09-21T21:41:28.488 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.488 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.488 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:28.488 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407715+0000", 2024-09-21T21:41:28.488 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.972e-06 2024-09-21T21:41:28.488 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.488 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.488 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:28.488 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.452266+0000", 2024-09-21T21:41:28.488 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044551201999999998 2024-09-21T21:41:28.489 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.489 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.489 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:28.489 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.452302+0000", 2024-09-21T21:41:28.489 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6516000000000001e-05 2024-09-21T21:41:28.489 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.489 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.489 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:28.489 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.452340+0000", 2024-09-21T21:41:28.489 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7710000000000003e-05 2024-09-21T21:41:28.489 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:28.489 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:28.489 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:28.489 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.490 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.490 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2434 3.d 3:b26310db:::rbd_data.112145fae779.0000000000000006:a [read 10637312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:28.490 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.407720+0000", 2024-09-21T21:41:28.490 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 30.052647763, 2024-09-21T21:41:28.490 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.04471874, 2024-09-21T21:41:28.490 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:28.490 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:28.490 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:28.490 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:28.490 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:28.490 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:28.490 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2434 2024-09-21T21:41:28.491 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.491 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:28.491 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.491 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:28.491 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407720+0000", 2024-09-21T21:41:28.491 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:28.491 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.491 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.491 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:28.491 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407720+0000", 2024-09-21T21:41:28.491 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:28.492 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.492 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.492 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:28.492 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407721+0000", 2024-09-21T21:41:28.492 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5199999999999998e-07 2024-09-21T21:41:28.492 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.492 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.492 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:28.492 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407723+0000", 2024-09-21T21:41:28.492 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9139999999999998e-06 2024-09-21T21:41:28.492 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.492 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.492 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:28.493 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407729+0000", 2024-09-21T21:41:28.493 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2009999999999996e-06 2024-09-21T21:41:28.493 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.493 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.493 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:28.493 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407734+0000", 2024-09-21T21:41:28.493 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.006e-06 2024-09-21T21:41:28.493 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.493 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.493 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:28.493 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.452364+0000", 2024-09-21T21:41:28.493 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044630625 2024-09-21T21:41:28.493 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.493 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.494 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:28.494 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.452402+0000", 2024-09-21T21:41:28.494 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7518999999999998e-05 2024-09-21T21:41:28.494 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.494 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.494 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:28.494 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.452439+0000", 2024-09-21T21:41:28.494 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7122999999999997e-05 2024-09-21T21:41:28.494 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:28.494 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:28.494 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:28.495 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.495 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.495 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2436 3.d 3:b26310db:::rbd_data.112145fae779.0000000000000006:a [read 10694656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:28.495 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.407740+0000", 2024-09-21T21:41:28.495 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 30.052628420000001, 2024-09-21T21:41:28.495 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044797624000000001, 2024-09-21T21:41:28.495 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:28.495 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:28.495 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:28.495 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:28.495 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:28.495 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:28.495 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2436 2024-09-21T21:41:28.496 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.496 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:28.496 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.496 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:28.496 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407740+0000", 2024-09-21T21:41:28.496 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:28.496 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.496 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.496 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:28.496 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407740+0000", 2024-09-21T21:41:28.496 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:28.496 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.496 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.496 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:28.497 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407740+0000", 2024-09-21T21:41:28.497 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9299999999999999e-07 2024-09-21T21:41:28.497 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.497 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.497 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:28.497 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407742+0000", 2024-09-21T21:41:28.497 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.826e-06 2024-09-21T21:41:28.497 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.497 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.497 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:28.497 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407748+0000", 2024-09-21T21:41:28.497 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2149999999999998e-06 2024-09-21T21:41:28.497 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.497 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.497 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:28.498 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.407753+0000", 2024-09-21T21:41:28.498 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.848e-06 2024-09-21T21:41:28.498 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.498 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.498 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:28.498 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.452463+0000", 2024-09-21T21:41:28.498 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044709596999999997 2024-09-21T21:41:28.498 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.498 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.498 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:28.498 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.452500+0000", 2024-09-21T21:41:28.498 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7058999999999999e-05 2024-09-21T21:41:28.498 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.498 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.499 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:28.499 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.452537+0000", 2024-09-21T21:41:28.499 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7685999999999998e-05 2024-09-21T21:41:28.499 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:28.499 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:28.499 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:28.499 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.499 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.501 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2602 3.d 3:b154679f:::rbd_data.112145fae779.0000000000000004:a [read 3923968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:28.501 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.411479+0000", 2024-09-21T21:41:28.501 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 30.048888961999999, 2024-09-21T21:41:28.501 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044281854000000002, 2024-09-21T21:41:28.501 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:28.502 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:28.502 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:28.502 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:28.502 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:28.502 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:28.502 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2602 2024-09-21T21:41:28.502 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.502 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:28.502 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.502 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:28.502 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411479+0000", 2024-09-21T21:41:28.502 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:28.502 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.502 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.502 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:28.503 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411479+0000", 2024-09-21T21:41:28.503 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:28.503 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.503 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.503 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:28.503 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411480+0000", 2024-09-21T21:41:28.503 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.37e-07 2024-09-21T21:41:28.503 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.503 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.503 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:28.503 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411482+0000", 2024-09-21T21:41:28.503 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9829999999999999e-06 2024-09-21T21:41:28.503 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.503 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.504 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:28.504 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411488+0000", 2024-09-21T21:41:28.504 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.947e-06 2024-09-21T21:41:28.504 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.504 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.504 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:28.504 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411492+0000", 2024-09-21T21:41:28.504 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.357e-06 2024-09-21T21:41:28.504 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.504 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.504 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:28.504 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.455581+0000", 2024-09-21T21:41:28.504 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044089077999999997 2024-09-21T21:41:28.504 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.504 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.505 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:28.505 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.455675+0000", 2024-09-21T21:41:28.505 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3757000000000004e-05 2024-09-21T21:41:28.505 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.505 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.505 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:28.505 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.455761+0000", 2024-09-21T21:41:28.505 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6194999999999997e-05 2024-09-21T21:41:28.505 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:28.505 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:28.505 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:28.505 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.505 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.506 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2612 3.d 3:b26310db:::rbd_data.112145fae779.0000000000000006:a [read 3448832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:28.506 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.411498+0000", 2024-09-21T21:41:28.506 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 30.048870193999999, 2024-09-21T21:41:28.506 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044369585000000003, 2024-09-21T21:41:28.506 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:28.506 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:28.506 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:28.506 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:28.506 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:28.506 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:28.506 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2612 2024-09-21T21:41:28.506 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.506 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:28.506 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.507 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:28.507 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411498+0000", 2024-09-21T21:41:28.507 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:28.507 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.507 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.507 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:28.507 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411498+0000", 2024-09-21T21:41:28.507 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:28.507 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.507 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.507 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:28.507 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411498+0000", 2024-09-21T21:41:28.507 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6899999999999998e-07 2024-09-21T21:41:28.507 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.508 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.508 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:28.508 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411500+0000", 2024-09-21T21:41:28.508 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.841e-06 2024-09-21T21:41:28.508 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.508 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.508 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:28.508 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411506+0000", 2024-09-21T21:41:28.508 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7350000000000001e-06 2024-09-21T21:41:28.508 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.508 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.508 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:28.508 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411510+0000", 2024-09-21T21:41:28.508 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4830000000000001e-06 2024-09-21T21:41:28.509 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.509 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.509 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:28.509 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.455789+0000", 2024-09-21T21:41:28.509 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044278359000000003 2024-09-21T21:41:28.509 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.509 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.509 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:28.509 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.455824+0000", 2024-09-21T21:41:28.509 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5719000000000001e-05 2024-09-21T21:41:28.509 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.509 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.510 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:28.510 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.455868+0000", 2024-09-21T21:41:28.510 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3078999999999999e-05 2024-09-21T21:41:28.510 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:28.510 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:28.510 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:28.510 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.510 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.510 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2613 3.d 3:b26310db:::rbd_data.112145fae779.0000000000000006:a [read 3506176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:28.510 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.411516+0000", 2024-09-21T21:41:28.510 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 30.048852517, 2024-09-21T21:41:28.510 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044562615, 2024-09-21T21:41:28.510 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:28.511 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:28.511 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:28.511 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:28.511 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:28.511 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:28.511 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2613 2024-09-21T21:41:28.511 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.511 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:28.511 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.511 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:28.511 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411516+0000", 2024-09-21T21:41:28.511 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:28.511 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.511 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.512 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:28.512 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411516+0000", 2024-09-21T21:41:28.512 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:28.512 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.512 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.512 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:28.512 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411516+0000", 2024-09-21T21:41:28.512 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4400000000000001e-07 2024-09-21T21:41:28.512 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.512 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.512 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:28.512 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411522+0000", 2024-09-21T21:41:28.512 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2600000000000002e-06 2024-09-21T21:41:28.512 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.513 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.513 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:28.513 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411528+0000", 2024-09-21T21:41:28.513 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5759999999999996e-06 2024-09-21T21:41:28.513 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.513 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.513 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:28.513 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411534+0000", 2024-09-21T21:41:28.513 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3010000000000003e-06 2024-09-21T21:41:28.513 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.513 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.513 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:28.513 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.455911+0000", 2024-09-21T21:41:28.513 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044376546000000003 2024-09-21T21:41:28.514 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.514 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.514 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:28.514 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.455996+0000", 2024-09-21T21:41:28.514 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.4980000000000003e-05 2024-09-21T21:41:28.514 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.514 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.514 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:28.514 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.456078+0000", 2024-09-21T21:41:28.514 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2608000000000006e-05 2024-09-21T21:41:28.514 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:28.514 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:28.514 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:28.514 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.514 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.515 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2614 3.d 3:b26310db:::rbd_data.112145fae779.0000000000000006:a [read 3538944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:28.515 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.411540+0000", 2024-09-21T21:41:28.515 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 30.048827703000001, 2024-09-21T21:41:28.515 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044644049999999998, 2024-09-21T21:41:28.515 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:28.515 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:28.515 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:28.515 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:28.515 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:28.515 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:28.515 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2614 2024-09-21T21:41:28.515 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.515 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:28.515 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.515 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:28.516 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411540+0000", 2024-09-21T21:41:28.516 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:28.516 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.516 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.516 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:28.516 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411540+0000", 2024-09-21T21:41:28.516 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:28.516 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.516 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.516 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:28.516 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411541+0000", 2024-09-21T21:41:28.516 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3799999999999997e-07 2024-09-21T21:41:28.516 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.516 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.516 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:28.517 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411543+0000", 2024-09-21T21:41:28.517 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1100000000000001e-06 2024-09-21T21:41:28.517 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.517 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.517 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:28.517 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411549+0000", 2024-09-21T21:41:28.517 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8300000000000001e-06 2024-09-21T21:41:28.517 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.517 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.517 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:28.517 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411554+0000", 2024-09-21T21:41:28.517 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8169999999999998e-06 2024-09-21T21:41:28.517 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.517 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.517 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:28.518 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.456107+0000", 2024-09-21T21:41:28.518 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044553552000000003 2024-09-21T21:41:28.518 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.518 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.518 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:28.518 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.456144+0000", 2024-09-21T21:41:28.518 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6560000000000002e-05 2024-09-21T21:41:28.518 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.518 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.518 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:28.518 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.456185+0000", 2024-09-21T21:41:28.518 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0642999999999997e-05 2024-09-21T21:41:28.518 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:28.518 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:28.519 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:28.519 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.519 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.519 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2615 3.d 3:b26310db:::rbd_data.112145fae779.0000000000000006:a [read 3596288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:28.519 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.411560+0000", 2024-09-21T21:41:28.519 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 30.048808147999999, 2024-09-21T21:41:28.519 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044828397999999998, 2024-09-21T21:41:28.519 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:28.519 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:28.519 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:28.519 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:28.519 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:28.519 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:28.519 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2615 2024-09-21T21:41:28.519 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.520 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:28.520 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.520 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:28.520 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411560+0000", 2024-09-21T21:41:28.520 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:28.520 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.520 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.520 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:28.520 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411560+0000", 2024-09-21T21:41:28.520 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:28.520 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.520 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.520 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:28.520 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411560+0000", 2024-09-21T21:41:28.520 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7399999999999999e-07 2024-09-21T21:41:28.521 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.521 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.521 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:28.521 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411562+0000", 2024-09-21T21:41:28.521 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.854e-06 2024-09-21T21:41:28.521 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.521 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.521 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:28.521 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411568+0000", 2024-09-21T21:41:28.521 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6760000000000004e-06 2024-09-21T21:41:28.521 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.521 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.521 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:28.521 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411573+0000", 2024-09-21T21:41:28.522 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6269999999999999e-06 2024-09-21T21:41:28.522 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.522 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.522 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:28.522 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.456226+0000", 2024-09-21T21:41:28.522 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044653088000000001 2024-09-21T21:41:28.522 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.522 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.522 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:28.522 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.456311+0000", 2024-09-21T21:41:28.522 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5685000000000004e-05 2024-09-21T21:41:28.522 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.522 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.523 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:28.523 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.456388+0000", 2024-09-21T21:41:28.523 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7094000000000004e-05 2024-09-21T21:41:28.523 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:28.523 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:28.523 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:28.523 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.523 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.523 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2616 3.d 3:b26310db:::rbd_data.112145fae779.0000000000000006:a [read 3653632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:28.523 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.411578+0000", 2024-09-21T21:41:28.523 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 30.048790243999999, 2024-09-21T21:41:28.523 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044914731999999999, 2024-09-21T21:41:28.523 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:28.523 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:28.523 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:28.524 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:28.524 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:28.524 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:28.524 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2616 2024-09-21T21:41:28.524 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.524 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:28.524 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.524 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:28.524 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411578+0000", 2024-09-21T21:41:28.524 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:28.524 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.524 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.524 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:28.524 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411578+0000", 2024-09-21T21:41:28.524 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:28.525 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.525 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.525 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:28.525 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411578+0000", 2024-09-21T21:41:28.525 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.03e-07 2024-09-21T21:41:28.525 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.525 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.525 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:28.525 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411580+0000", 2024-09-21T21:41:28.525 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.8199999999999999e-06 2024-09-21T21:41:28.525 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.525 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.525 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:28.525 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411586+0000", 2024-09-21T21:41:28.526 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6319999999999996e-06 2024-09-21T21:41:28.526 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.526 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.526 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:28.526 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411590+0000", 2024-09-21T21:41:28.526 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5979999999999999e-06 2024-09-21T21:41:28.526 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.526 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.526 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:28.526 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.456418+0000", 2024-09-21T21:41:28.526 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044827504999999997 2024-09-21T21:41:28.526 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.526 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.526 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:28.527 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.456454+0000", 2024-09-21T21:41:28.527 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5883999999999997e-05 2024-09-21T21:41:28.527 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.527 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.527 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:28.527 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.456493+0000", 2024-09-21T21:41:28.527 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8890000000000002e-05 2024-09-21T21:41:28.527 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:28.527 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:28.527 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:28.527 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.527 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.527 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2617 3.d 3:b26310db:::rbd_data.112145fae779.0000000000000006:a [read 3686400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:28.527 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.411596+0000", 2024-09-21T21:41:28.527 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 30.048772654, 2024-09-21T21:41:28.528 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.045094515000000002, 2024-09-21T21:41:28.528 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:28.528 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:28.528 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:28.528 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:28.528 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:28.528 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:28.528 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2617 2024-09-21T21:41:28.528 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.528 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:28.528 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.528 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:28.528 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411596+0000", 2024-09-21T21:41:28.528 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:28.528 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.529 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.529 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:28.529 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411596+0000", 2024-09-21T21:41:28.529 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:28.529 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.529 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.529 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:28.529 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411596+0000", 2024-09-21T21:41:28.529 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3500000000000002e-07 2024-09-21T21:41:28.529 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.529 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.529 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:28.529 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411598+0000", 2024-09-21T21:41:28.529 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.871e-06 2024-09-21T21:41:28.529 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.530 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.530 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:28.530 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411603+0000", 2024-09-21T21:41:28.530 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6420000000000004e-06 2024-09-21T21:41:28.530 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.530 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.530 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:28.530 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411608+0000", 2024-09-21T21:41:28.530 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4780000000000002e-06 2024-09-21T21:41:28.530 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.530 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.530 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:28.530 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.456529+0000", 2024-09-21T21:41:28.530 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044921232999999998 2024-09-21T21:41:28.530 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.531 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.531 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:28.531 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.456616+0000", 2024-09-21T21:41:28.531 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6712000000000006e-05 2024-09-21T21:41:28.531 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.531 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.531 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:28.531 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.456690+0000", 2024-09-21T21:41:28.531 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4244000000000003e-05 2024-09-21T21:41:28.531 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:28.531 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:28.531 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:28.531 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.531 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.531 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2618 3.d 3:b26310db:::rbd_data.112145fae779.0000000000000006:a [read 3743744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:28.532 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.411613+0000", 2024-09-21T21:41:28.532 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 30.048755074999999, 2024-09-21T21:41:28.532 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.045180423999999997, 2024-09-21T21:41:28.532 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:28.532 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:28.532 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:28.532 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:28.532 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:28.532 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:28.532 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2618 2024-09-21T21:41:28.532 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.532 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:28.532 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.532 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:28.532 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411613+0000", 2024-09-21T21:41:28.533 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:28.533 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.533 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.533 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:28.533 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411613+0000", 2024-09-21T21:41:28.533 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:28.533 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.533 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.533 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:28.533 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.412868+0000", 2024-09-21T21:41:28.533 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012549 2024-09-21T21:41:28.533 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.533 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.533 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:28.533 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.412873+0000", 2024-09-21T21:41:28.534 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9060000000000001e-06 2024-09-21T21:41:28.534 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.534 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.534 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:28.534 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.412884+0000", 2024-09-21T21:41:28.534 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1164e-05 2024-09-21T21:41:28.534 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.534 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.534 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:28.534 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.412892+0000", 2024-09-21T21:41:28.534 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5719999999999997e-06 2024-09-21T21:41:28.534 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.534 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.534 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:28.535 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.456717+0000", 2024-09-21T21:41:28.535 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043825559 2024-09-21T21:41:28.535 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.535 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.535 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:28.535 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.456753+0000", 2024-09-21T21:41:28.535 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6158000000000002e-05 2024-09-21T21:41:28.535 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.535 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.535 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:28.535 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.456794+0000", 2024-09-21T21:41:28.535 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0164999999999999e-05 2024-09-21T21:41:28.535 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:28.535 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:28.535 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:28.536 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.536 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.536 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2667 3.d 3:b2093d43:::rbd_data.112145fae779.000000000000000b:a [read 3809280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:28.536 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.414362+0000", 2024-09-21T21:41:28.536 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 30.046006396999999, 2024-09-21T21:41:28.536 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044335620999999999, 2024-09-21T21:41:28.536 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:28.536 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:28.536 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:28.536 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:28.536 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:28.536 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:28.536 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2667 2024-09-21T21:41:28.536 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.536 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:28.537 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.537 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:28.537 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.414362+0000", 2024-09-21T21:41:28.537 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:28.537 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.537 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.537 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:28.537 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.414362+0000", 2024-09-21T21:41:28.537 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:28.537 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.537 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.537 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:28.537 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.414867+0000", 2024-09-21T21:41:28.537 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00050531999999999997 2024-09-21T21:41:28.537 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.538 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.538 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:28.538 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.414871+0000", 2024-09-21T21:41:28.538 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2520000000000001e-06 2024-09-21T21:41:28.538 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.538 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.538 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:28.538 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.414879+0000", 2024-09-21T21:41:28.538 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0369999999999995e-06 2024-09-21T21:41:28.538 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.538 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.538 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:28.538 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.414886+0000", 2024-09-21T21:41:28.538 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.19e-06 2024-09-21T21:41:28.538 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.539 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.539 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:28.539 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.458613+0000", 2024-09-21T21:41:28.539 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043727184000000002 2024-09-21T21:41:28.539 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.539 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.539 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:28.539 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.458649+0000", 2024-09-21T21:41:28.539 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6418999999999998e-05 2024-09-21T21:41:28.539 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.539 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.539 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:28.539 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.458697+0000", 2024-09-21T21:41:28.539 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8219e-05 2024-09-21T21:41:28.539 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:28.540 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:28.540 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:28.540 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.540 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.540 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2725 3.d 3:b154679f:::rbd_data.112145fae779.0000000000000004:a [read 3534848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:28.540 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.414975+0000", 2024-09-21T21:41:28.540 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 30.045393471000001, 2024-09-21T21:41:28.540 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044402427000000001, 2024-09-21T21:41:28.540 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:28.540 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:28.540 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:28.540 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:28.540 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:28.540 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:28.541 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2725 2024-09-21T21:41:28.541 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.541 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:28.541 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.541 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:28.541 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.414975+0000", 2024-09-21T21:41:28.541 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:28.541 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.541 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.541 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:28.541 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.414975+0000", 2024-09-21T21:41:28.541 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:28.541 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.541 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.541 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:28.542 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.414975+0000", 2024-09-21T21:41:28.542 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1899999999999999e-07 2024-09-21T21:41:28.542 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.542 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.542 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:28.542 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.414976+0000", 2024-09-21T21:41:28.542 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1689999999999999e-06 2024-09-21T21:41:28.542 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.542 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.542 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:28.542 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.414979+0000", 2024-09-21T21:41:28.542 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9900000000000002e-06 2024-09-21T21:41:28.542 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.542 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.543 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:28.543 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.414982+0000", 2024-09-21T21:41:28.543 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.621e-06 2024-09-21T21:41:28.543 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.543 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.543 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:28.543 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.459221+0000", 2024-09-21T21:41:28.543 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044239602000000003 2024-09-21T21:41:28.543 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.543 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.543 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:28.543 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.459297+0000", 2024-09-21T21:41:28.543 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5768999999999999e-05 2024-09-21T21:41:28.543 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.543 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.544 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:28.544 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.459377+0000", 2024-09-21T21:41:28.544 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0056999999999996e-05 2024-09-21T21:41:28.544 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:28.544 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:28.544 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:28.544 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.544 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.544 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2726 3.d 3:b154679f:::rbd_data.112145fae779.0000000000000004:a [read 3567616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:28.544 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.414985+0000", 2024-09-21T21:41:28.544 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 30.045383403999999, 2024-09-21T21:41:28.544 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044526137, 2024-09-21T21:41:28.544 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:28.544 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:28.545 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:28.545 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:28.545 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:28.545 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:28.545 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2726 2024-09-21T21:41:28.545 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.545 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:28.545 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.545 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:28.545 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.414985+0000", 2024-09-21T21:41:28.545 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:28.545 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.545 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.545 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:28.546 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.414985+0000", 2024-09-21T21:41:28.546 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:28.546 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.546 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.546 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:28.546 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.414985+0000", 2024-09-21T21:41:28.546 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0600000000000001e-07 2024-09-21T21:41:28.546 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.546 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.546 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:28.546 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.414986+0000", 2024-09-21T21:41:28.546 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.175e-06 2024-09-21T21:41:28.546 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.546 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.546 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:28.547 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.414989+0000", 2024-09-21T21:41:28.547 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8720000000000002e-06 2024-09-21T21:41:28.547 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.547 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.547 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:28.547 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.414992+0000", 2024-09-21T21:41:28.547 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3680000000000002e-06 2024-09-21T21:41:28.547 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.547 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.547 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:28.547 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.459415+0000", 2024-09-21T21:41:28.547 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044422167999999998 2024-09-21T21:41:28.547 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.548 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.548 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:28.548 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.459466+0000", 2024-09-21T21:41:28.548 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1177000000000001e-05 2024-09-21T21:41:28.548 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.548 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.548 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:28.548 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.459511+0000", 2024-09-21T21:41:28.548 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5071000000000003e-05 2024-09-21T21:41:28.548 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:28.548 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:28.548 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:28.548 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.548 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.548 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2727 3.d 3:b154679f:::rbd_data.112145fae779.0000000000000004:a [read 3624960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:28.549 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.414997+0000", 2024-09-21T21:41:28.549 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 30.045370831, 2024-09-21T21:41:28.549 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044651328999999997, 2024-09-21T21:41:28.549 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:28.549 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:28.549 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:28.549 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:28.549 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:28.549 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:28.549 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2727 2024-09-21T21:41:28.549 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.549 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:28.549 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.550 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:28.550 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.414997+0000", 2024-09-21T21:41:28.550 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:28.550 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.550 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.550 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:28.550 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.414997+0000", 2024-09-21T21:41:28.550 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:28.550 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.550 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.550 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:28.550 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.416868+0000", 2024-09-21T21:41:28.550 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001871019 2024-09-21T21:41:28.550 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.551 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.551 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:28.551 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.416873+0000", 2024-09-21T21:41:28.551 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6299999999999997e-06 2024-09-21T21:41:28.551 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.551 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.551 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:28.551 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.416883+0000", 2024-09-21T21:41:28.551 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0172e-05 2024-09-21T21:41:28.551 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.551 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.551 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:28.551 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.416890+0000", 2024-09-21T21:41:28.551 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8689999999999998e-06 2024-09-21T21:41:28.551 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.552 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.552 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:28.552 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.459541+0000", 2024-09-21T21:41:28.552 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042650592000000001 2024-09-21T21:41:28.552 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.552 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.552 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:28.552 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.459601+0000", 2024-09-21T21:41:28.552 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9978000000000001e-05 2024-09-21T21:41:28.552 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.552 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.552 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:28.552 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.459649+0000", 2024-09-21T21:41:28.552 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8068999999999997e-05 2024-09-21T21:41:28.553 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:28.553 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:28.553 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:28.553 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:28.553 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:28.553 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:28.554 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T21:41:28.555 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:28.553+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:28.655 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:28.654+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:28.657 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:28.657 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:41:28.657 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-21T21:41:28.657 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:28.666 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T21:41:28.756 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:28.753+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:28.771 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:28.772 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:41:28.772 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:41:28.772 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:41:28.772 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:28.780 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T21:41:28.864 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:28.862+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:28.926 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:28.927 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-21T21:41:28.927 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-21T21:41:28.927 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:41:28.927 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.927 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4840.0:5190 3.1a 3:5fa6cc52:::rbd_header.1121e0174ab1:head [notify cookie 139800381081568] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T21:41:28.927 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:16.463940+0000", 2024-09-21T21:41:28.927 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.460552815, 2024-09-21T21:41:28.927 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.026118738999999998, 2024-09-21T21:41:28.927 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:28.927 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:28.928 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:28.928 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:28.928 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4840", 2024-09-21T21:41:28.928 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1293784138", 2024-09-21T21:41:28.928 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 5190 2024-09-21T21:41:28.928 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.928 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:28.928 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.928 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:28.928 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.463940+0000", 2024-09-21T21:41:28.928 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:28.928 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.928 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.928 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:28.929 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.463940+0000", 2024-09-21T21:41:28.929 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:28.929 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.929 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.929 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:28.929 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.463940+0000", 2024-09-21T21:41:28.929 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5699999999999998e-07 2024-09-21T21:41:28.929 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.929 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.929 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:28.929 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.463942+0000", 2024-09-21T21:41:28.929 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.491e-06 2024-09-21T21:41:28.929 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.929 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.929 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:28.930 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.463951+0000", 2024-09-21T21:41:28.930 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2689999999999999e-06 2024-09-21T21:41:28.930 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.930 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.930 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:28.930 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.463955+0000", 2024-09-21T21:41:28.930 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8070000000000001e-06 2024-09-21T21:41:28.930 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.930 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.930 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:28.930 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.489958+0000", 2024-09-21T21:41:28.930 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.026003480999999998 2024-09-21T21:41:28.930 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.930 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.931 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:28.931 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.489995+0000", 2024-09-21T21:41:28.931 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7265000000000003e-05 2024-09-21T21:41:28.931 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.931 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.931 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:28.931 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.490058+0000", 2024-09-21T21:41:28.931 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3069000000000002e-05 2024-09-21T21:41:28.931 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:28.931 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:28.931 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:28.931 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.931 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.931 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4840.0:5191 3.1a 3:5fa6cc52:::rbd_header.1121e0174ab1:head [notify cookie 139800381093248] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T21:41:28.932 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:16.463965+0000", 2024-09-21T21:41:28.932 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.460527525, 2024-09-21T21:41:28.932 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.026300389, 2024-09-21T21:41:28.932 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:28.932 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:28.932 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:28.932 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:28.932 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4840", 2024-09-21T21:41:28.932 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1293784138", 2024-09-21T21:41:28.932 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 5191 2024-09-21T21:41:28.932 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.932 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:28.932 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.932 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:28.932 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.463965+0000", 2024-09-21T21:41:28.933 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:28.933 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.933 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.933 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:28.933 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.463965+0000", 2024-09-21T21:41:28.933 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:28.933 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.933 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.933 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:28.933 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.463965+0000", 2024-09-21T21:41:28.933 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7800000000000001e-07 2024-09-21T21:41:28.933 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.933 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.933 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:28.933 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.463969+0000", 2024-09-21T21:41:28.934 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1669999999999999e-06 2024-09-21T21:41:28.934 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.934 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.934 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:28.934 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.463975+0000", 2024-09-21T21:41:28.934 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2920000000000001e-06 2024-09-21T21:41:28.934 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.934 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.934 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:28.934 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.463980+0000", 2024-09-21T21:41:28.934 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3580000000000002e-06 2024-09-21T21:41:28.934 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.934 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.934 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:28.934 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.490090+0000", 2024-09-21T21:41:28.935 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.026109614 2024-09-21T21:41:28.935 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.935 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.935 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:28.935 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.490155+0000", 2024-09-21T21:41:28.935 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5542000000000004e-05 2024-09-21T21:41:28.935 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.935 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.935 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:28.935 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.490265+0000", 2024-09-21T21:41:28.936 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000109838 2024-09-21T21:41:28.936 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:28.936 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:28.936 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:28.936 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.936 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.936 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4840.0:5192 3.1a 3:5fa6cc52:::rbd_header.1121e0174ab1:head [notify cookie 139800381104400] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T21:41:28.936 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:16.463987+0000", 2024-09-21T21:41:28.936 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.460505029, 2024-09-21T21:41:28.936 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.026422074, 2024-09-21T21:41:28.936 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:28.936 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:28.937 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:28.937 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:28.937 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4840", 2024-09-21T21:41:28.937 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1293784138", 2024-09-21T21:41:28.937 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 5192 2024-09-21T21:41:28.937 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.937 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:28.937 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.937 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:28.937 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.463987+0000", 2024-09-21T21:41:28.937 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:28.937 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.937 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.937 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:28.938 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.463987+0000", 2024-09-21T21:41:28.938 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:28.938 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.938 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.938 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:28.938 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.463988+0000", 2024-09-21T21:41:28.938 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2e-07 2024-09-21T21:41:28.938 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.938 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.938 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:28.938 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.463990+0000", 2024-09-21T21:41:28.938 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5490000000000001e-06 2024-09-21T21:41:28.938 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.938 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.939 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:28.939 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.463998+0000", 2024-09-21T21:41:28.939 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0190000000000006e-06 2024-09-21T21:41:28.939 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.939 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.939 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:28.939 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464003+0000", 2024-09-21T21:41:28.939 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3810000000000001e-06 2024-09-21T21:41:28.939 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.939 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.939 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:28.939 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.490279+0000", 2024-09-21T21:41:28.939 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.026276520000000001 2024-09-21T21:41:28.939 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.939 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.940 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:28.940 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.490331+0000", 2024-09-21T21:41:28.940 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1999999999999997e-05 2024-09-21T21:41:28.940 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.940 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.940 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:28.940 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.490409+0000", 2024-09-21T21:41:28.940 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8184999999999998e-05 2024-09-21T21:41:28.940 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:28.941 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:28.941 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:28.941 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.941 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.941 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4840.0:5193 3.1a 3:5fa6cc52:::rbd_header.1121e0174ab1:head [notify cookie 139800380193680] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T21:41:28.941 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:16.464011+0000", 2024-09-21T21:41:28.941 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.460481428, 2024-09-21T21:41:28.942 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.026606253, 2024-09-21T21:41:28.942 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:28.942 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:28.942 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:28.942 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:28.942 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4840", 2024-09-21T21:41:28.942 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1293784138", 2024-09-21T21:41:28.942 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 5193 2024-09-21T21:41:28.942 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.942 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:28.942 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.942 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:28.943 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464011+0000", 2024-09-21T21:41:28.943 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:28.943 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.943 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.943 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:28.943 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464011+0000", 2024-09-21T21:41:28.943 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:28.943 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.943 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.943 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:28.943 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464011+0000", 2024-09-21T21:41:28.943 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5600000000000001e-07 2024-09-21T21:41:28.943 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.943 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.943 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:28.944 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464014+0000", 2024-09-21T21:41:28.944 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4619999999999999e-06 2024-09-21T21:41:28.944 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.944 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.944 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:28.944 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464020+0000", 2024-09-21T21:41:28.944 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7819999999999999e-06 2024-09-21T21:41:28.944 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.944 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.944 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:28.944 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464024+0000", 2024-09-21T21:41:28.944 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7820000000000001e-06 2024-09-21T21:41:28.944 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.945 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.945 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:28.945 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.490444+0000", 2024-09-21T21:41:28.945 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.026419257000000002 2024-09-21T21:41:28.945 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.945 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.945 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:28.945 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.490508+0000", 2024-09-21T21:41:28.945 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3961999999999995e-05 2024-09-21T21:41:28.945 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.945 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.945 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:28.945 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.490617+0000", 2024-09-21T21:41:28.945 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000109652 2024-09-21T21:41:28.946 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:28.946 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:28.946 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:28.946 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.946 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.946 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4840.0:5194 3.1a 3:5fa6cc52:::rbd_header.1121e0174ab1:head [notify cookie 139800381118288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T21:41:28.946 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:16.464032+0000", 2024-09-21T21:41:28.946 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.460460640000001, 2024-09-21T21:41:28.946 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.026691425000000001, 2024-09-21T21:41:28.946 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:28.946 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:28.946 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:28.946 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:28.947 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4840", 2024-09-21T21:41:28.947 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1293784138", 2024-09-21T21:41:28.947 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 5194 2024-09-21T21:41:28.947 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.947 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:28.947 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.947 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:28.947 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464032+0000", 2024-09-21T21:41:28.947 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:28.947 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.947 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.947 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:28.947 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464032+0000", 2024-09-21T21:41:28.947 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:28.948 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.948 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.948 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:28.948 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464032+0000", 2024-09-21T21:41:28.948 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8799999999999998e-07 2024-09-21T21:41:28.948 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.948 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.948 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:28.948 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464034+0000", 2024-09-21T21:41:28.948 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.295e-06 2024-09-21T21:41:28.948 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.948 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.948 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:28.948 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464042+0000", 2024-09-21T21:41:28.949 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5650000000000004e-06 2024-09-21T21:41:28.949 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.949 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.949 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:28.949 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464047+0000", 2024-09-21T21:41:28.949 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.04e-06 2024-09-21T21:41:28.949 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.949 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.949 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:28.949 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.490632+0000", 2024-09-21T21:41:28.949 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.026585233999999999 2024-09-21T21:41:28.950 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.950 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.950 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:28.950 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.490665+0000", 2024-09-21T21:41:28.950 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2203999999999999e-05 2024-09-21T21:41:28.950 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.950 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.950 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:28.950 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.490723+0000", 2024-09-21T21:41:28.950 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8699000000000002e-05 2024-09-21T21:41:28.950 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:28.950 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:28.950 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:28.951 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.951 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.951 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4840.0:5195 3.1a 3:5fa6cc52:::rbd_header.1121e0174ab1:head [notify cookie 139800380588912] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T21:41:28.951 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:16.464057+0000", 2024-09-21T21:41:28.951 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.460435652999999, 2024-09-21T21:41:28.951 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.026886305999999999, 2024-09-21T21:41:28.951 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:28.951 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:28.951 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:28.951 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:28.951 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4840", 2024-09-21T21:41:28.951 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1293784138", 2024-09-21T21:41:28.952 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 5195 2024-09-21T21:41:28.952 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.952 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:28.952 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.952 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:28.952 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464057+0000", 2024-09-21T21:41:28.952 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:28.952 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.952 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.952 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:28.952 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464057+0000", 2024-09-21T21:41:28.953 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:28.953 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.953 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.953 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:28.953 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464057+0000", 2024-09-21T21:41:28.953 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2599999999999998e-07 2024-09-21T21:41:28.953 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.953 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.953 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:28.953 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464061+0000", 2024-09-21T21:41:28.953 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3330000000000001e-06 2024-09-21T21:41:28.953 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.953 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.953 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:28.954 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464071+0000", 2024-09-21T21:41:28.954 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.9809999999999998e-06 2024-09-21T21:41:28.954 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.954 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.954 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:28.954 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464077+0000", 2024-09-21T21:41:28.954 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0430000000000004e-06 2024-09-21T21:41:28.954 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.954 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.954 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:28.954 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.490754+0000", 2024-09-21T21:41:28.954 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.026677881000000001 2024-09-21T21:41:28.955 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.955 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.955 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:28.955 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.490822+0000", 2024-09-21T21:41:28.955 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8021999999999993e-05 2024-09-21T21:41:28.955 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.955 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.955 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:28.955 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.490943+0000", 2024-09-21T21:41:28.955 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012061999999999999 2024-09-21T21:41:28.955 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:28.955 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:28.955 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:28.956 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.956 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.956 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4840.0:5196 3.1a 3:5fa6cc52:::rbd_header.1121e0174ab1:head [notify cookie 139800380477648] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T21:41:28.956 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:16.464083+0000", 2024-09-21T21:41:28.956 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.460409298, 2024-09-21T21:41:28.956 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.026990353000000002, 2024-09-21T21:41:28.956 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:28.956 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:28.956 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:28.956 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:28.956 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4840", 2024-09-21T21:41:28.956 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1293784138", 2024-09-21T21:41:28.956 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 5196 2024-09-21T21:41:28.956 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.957 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:28.957 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.957 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:28.957 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464083+0000", 2024-09-21T21:41:28.957 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:28.957 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.957 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.957 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:28.957 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464083+0000", 2024-09-21T21:41:28.957 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:28.957 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.957 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.957 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:28.958 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464084+0000", 2024-09-21T21:41:28.958 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9700000000000002e-07 2024-09-21T21:41:28.958 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.958 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.958 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:28.958 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464086+0000", 2024-09-21T21:41:28.958 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4600000000000002e-06 2024-09-21T21:41:28.958 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.958 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.958 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:28.958 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464094+0000", 2024-09-21T21:41:28.958 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0220000000000004e-06 2024-09-21T21:41:28.958 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.959 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.959 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:28.959 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464100+0000", 2024-09-21T21:41:28.959 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8089999999999998e-06 2024-09-21T21:41:28.959 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.959 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.959 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:28.959 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.490971+0000", 2024-09-21T21:41:28.959 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.026871229 2024-09-21T21:41:28.959 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.959 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.959 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:28.959 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.491013+0000", 2024-09-21T21:41:28.959 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1723e-05 2024-09-21T21:41:28.960 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.960 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.960 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:28.960 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.491074+0000", 2024-09-21T21:41:28.960 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0713e-05 2024-09-21T21:41:28.960 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:28.960 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:28.960 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:28.960 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.960 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.960 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4840.0:5197 3.1a 3:5fa6cc52:::rbd_header.1121e0174ab1:head [notify cookie 139800380481024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T21:41:28.960 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:16.464109+0000", 2024-09-21T21:41:28.960 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.460383008999999, 2024-09-21T21:41:28.960 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.027188467000000001, 2024-09-21T21:41:28.961 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:28.961 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:28.961 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:28.961 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:28.961 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4840", 2024-09-21T21:41:28.961 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1293784138", 2024-09-21T21:41:28.961 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 5197 2024-09-21T21:41:28.961 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.961 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:28.962 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.962 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:28.962 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464109+0000", 2024-09-21T21:41:28.962 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:28.962 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.962 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.962 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:28.962 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464109+0000", 2024-09-21T21:41:28.962 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:28.962 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.962 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.962 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:28.962 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464110+0000", 2024-09-21T21:41:28.963 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.58e-07 2024-09-21T21:41:28.963 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.963 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.963 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:28.963 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464112+0000", 2024-09-21T21:41:28.963 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.553e-06 2024-09-21T21:41:28.963 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.963 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.963 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:28.963 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464119+0000", 2024-09-21T21:41:28.963 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4160000000000001e-06 2024-09-21T21:41:28.963 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.963 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.963 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:28.964 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464125+0000", 2024-09-21T21:41:28.964 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7769999999999999e-06 2024-09-21T21:41:28.964 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.964 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.964 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:28.964 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.491105+0000", 2024-09-21T21:41:28.964 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.026980481000000001 2024-09-21T21:41:28.964 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.964 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.964 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:28.964 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.491176+0000", 2024-09-21T21:41:28.964 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0807000000000002e-05 2024-09-21T21:41:28.964 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.964 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.965 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:28.965 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.491298+0000", 2024-09-21T21:41:28.965 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000122075 2024-09-21T21:41:28.965 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:28.965 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:28.965 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:28.965 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.965 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.965 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4840.0:5198 3.1a 3:5fa6cc52:::rbd_header.1121e0174ab1:head [notify cookie 139800381142928] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T21:41:28.965 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:16.464131+0000", 2024-09-21T21:41:28.965 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.460361915, 2024-09-21T21:41:28.965 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.027266123, 2024-09-21T21:41:28.965 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:28.965 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:28.965 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:28.966 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:28.966 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4840", 2024-09-21T21:41:28.966 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1293784138", 2024-09-21T21:41:28.966 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 5198 2024-09-21T21:41:28.966 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.966 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:28.966 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.966 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:28.966 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464131+0000", 2024-09-21T21:41:28.966 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:28.966 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.966 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.966 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:28.966 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464131+0000", 2024-09-21T21:41:28.967 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:28.967 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.967 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.967 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:28.967 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464131+0000", 2024-09-21T21:41:28.967 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4299999999999999e-07 2024-09-21T21:41:28.967 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.967 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.967 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:28.967 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464134+0000", 2024-09-21T21:41:28.967 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3289999999999998e-06 2024-09-21T21:41:28.967 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.967 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.967 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:28.968 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464140+0000", 2024-09-21T21:41:28.968 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6699999999999999e-06 2024-09-21T21:41:28.968 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.968 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.968 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:28.968 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464144+0000", 2024-09-21T21:41:28.968 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4440000000000001e-06 2024-09-21T21:41:28.968 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.968 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.968 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:28.968 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.491310+0000", 2024-09-21T21:41:28.968 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.027165827 2024-09-21T21:41:28.969 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.969 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.969 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:28.969 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.491344+0000", 2024-09-21T21:41:28.969 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3382e-05 2024-09-21T21:41:28.969 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.969 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.969 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:28.969 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.491397+0000", 2024-09-21T21:41:28.969 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3127999999999997e-05 2024-09-21T21:41:28.969 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:28.969 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:28.969 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:28.969 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.970 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.970 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4840.0:5199 3.1a 3:5fa6cc52:::rbd_header.1121e0174ab1:head [notify cookie 139800381153680] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T21:41:28.970 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:16.464152+0000", 2024-09-21T21:41:28.970 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.460340832, 2024-09-21T21:41:28.970 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.027530745999999998, 2024-09-21T21:41:28.970 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:28.970 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:28.970 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:28.970 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:28.970 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4840", 2024-09-21T21:41:28.970 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1293784138", 2024-09-21T21:41:28.970 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 5199 2024-09-21T21:41:28.970 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.970 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:28.971 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.971 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:28.971 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464152+0000", 2024-09-21T21:41:28.971 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:28.971 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.971 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.971 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:28.971 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464152+0000", 2024-09-21T21:41:28.971 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:28.971 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.971 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.971 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:28.971 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464152+0000", 2024-09-21T21:41:28.972 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8299999999999998e-07 2024-09-21T21:41:28.972 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.972 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.972 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:28.972 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464155+0000", 2024-09-21T21:41:28.972 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.349e-06 2024-09-21T21:41:28.972 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.972 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.972 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:28.972 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464160+0000", 2024-09-21T21:41:28.972 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4150000000000001e-06 2024-09-21T21:41:28.972 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.972 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.972 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:28.973 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464163+0000", 2024-09-21T21:41:28.973 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2499999999999998e-06 2024-09-21T21:41:28.973 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.973 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.973 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:28.973 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.491448+0000", 2024-09-21T21:41:28.973 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.027284503000000002 2024-09-21T21:41:28.973 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.973 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.973 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:28.973 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.491558+0000", 2024-09-21T21:41:28.973 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00011007500000000001 2024-09-21T21:41:28.973 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.973 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.973 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:28.974 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.491682+0000", 2024-09-21T21:41:28.974 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000124771 2024-09-21T21:41:28.974 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:28.974 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:28.974 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:28.974 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.974 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.974 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4840.0:5200 3.1a 3:5fa6cc52:::rbd_header.1121e0174ab1:head [notify cookie 139800381164992] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T21:41:28.974 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:16.464192+0000", 2024-09-21T21:41:28.974 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.460300141999999, 2024-09-21T21:41:28.974 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.027596809999999999, 2024-09-21T21:41:28.974 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:28.974 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:28.974 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:28.975 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:28.975 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4840", 2024-09-21T21:41:28.975 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1293784138", 2024-09-21T21:41:28.975 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 5200 2024-09-21T21:41:28.975 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.975 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:28.975 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.975 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:28.975 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464192+0000", 2024-09-21T21:41:28.975 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:28.975 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.975 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.975 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:28.975 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464192+0000", 2024-09-21T21:41:28.976 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:28.976 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.976 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.976 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:28.976 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464193+0000", 2024-09-21T21:41:28.976 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5799999999999999e-07 2024-09-21T21:41:28.976 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.976 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.976 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:28.976 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464199+0000", 2024-09-21T21:41:28.976 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5309999999999998e-06 2024-09-21T21:41:28.976 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.976 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.976 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:28.977 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464207+0000", 2024-09-21T21:41:28.977 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2620000000000001e-06 2024-09-21T21:41:28.977 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.977 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.977 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:28.977 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464211+0000", 2024-09-21T21:41:28.977 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5510000000000001e-06 2024-09-21T21:41:28.977 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.977 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.977 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:28.977 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.491696+0000", 2024-09-21T21:41:28.977 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.027484463000000001 2024-09-21T21:41:28.977 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.978 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.978 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:28.978 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.491729+0000", 2024-09-21T21:41:28.978 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3396999999999999e-05 2024-09-21T21:41:28.978 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.978 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.978 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:28.978 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.491789+0000", 2024-09-21T21:41:28.978 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0047999999999998e-05 2024-09-21T21:41:28.978 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:28.978 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:28.978 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:28.978 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.978 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.979 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4840.0:5201 3.1a 3:5fa6cc52:::rbd_header.1121e0174ab1:head [notify cookie 139800381176640] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T21:41:28.979 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:16.464225+0000", 2024-09-21T21:41:28.979 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.460267119999999, 2024-09-21T21:41:28.979 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.027776392, 2024-09-21T21:41:28.979 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:28.979 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:28.979 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:28.979 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:28.979 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4840", 2024-09-21T21:41:28.979 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1293784138", 2024-09-21T21:41:28.979 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 5201 2024-09-21T21:41:28.979 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.979 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:28.980 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.980 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:28.980 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464225+0000", 2024-09-21T21:41:28.980 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:28.980 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.980 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.980 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:28.980 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464225+0000", 2024-09-21T21:41:28.980 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:28.980 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.980 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.980 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:28.980 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464226+0000", 2024-09-21T21:41:28.980 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8299999999999998e-07 2024-09-21T21:41:28.981 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.981 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.981 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:28.981 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464228+0000", 2024-09-21T21:41:28.981 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.846e-06 2024-09-21T21:41:28.981 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.981 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.981 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:28.981 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464231+0000", 2024-09-21T21:41:28.981 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.675e-06 2024-09-21T21:41:28.981 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.981 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.981 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:28.982 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464235+0000", 2024-09-21T21:41:28.982 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7249999999999999e-06 2024-09-21T21:41:28.982 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.982 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.982 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:28.982 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.491822+0000", 2024-09-21T21:41:28.982 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.027586626999999999 2024-09-21T21:41:28.982 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.982 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.982 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:28.982 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.491894+0000", 2024-09-21T21:41:28.982 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2353000000000003e-05 2024-09-21T21:41:28.982 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.983 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.983 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:28.983 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.492002+0000", 2024-09-21T21:41:28.983 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010778299999999999 2024-09-21T21:41:28.983 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:28.983 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:28.983 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:28.983 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.983 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.983 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4840.0:5202 3.1a 3:5fa6cc52:::rbd_header.1121e0174ab1:head [notify cookie 139800380311792] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T21:41:28.983 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:16.464239+0000", 2024-09-21T21:41:28.983 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.460253764000001, 2024-09-21T21:41:28.983 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.027979713, 2024-09-21T21:41:28.984 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:28.984 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:28.984 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:28.984 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:28.984 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4840", 2024-09-21T21:41:28.984 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1293784138", 2024-09-21T21:41:28.984 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 5202 2024-09-21T21:41:28.984 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.984 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:28.984 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.984 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:28.984 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464239+0000", 2024-09-21T21:41:28.984 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:28.984 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.985 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.985 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:28.985 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464239+0000", 2024-09-21T21:41:28.985 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:28.985 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.985 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.985 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:28.985 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464239+0000", 2024-09-21T21:41:28.985 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4699999999999998e-07 2024-09-21T21:41:28.985 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.985 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.985 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:28.985 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464240+0000", 2024-09-21T21:41:28.986 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4959999999999999e-06 2024-09-21T21:41:28.986 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.986 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.986 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:28.986 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464244+0000", 2024-09-21T21:41:28.986 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.118e-06 2024-09-21T21:41:28.986 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.986 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.986 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:28.986 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464246+0000", 2024-09-21T21:41:28.986 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4789999999999999e-06 2024-09-21T21:41:28.986 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.986 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.987 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:28.987 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.492033+0000", 2024-09-21T21:41:28.987 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.02778692 2024-09-21T21:41:28.987 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.987 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.987 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:28.987 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.492107+0000", 2024-09-21T21:41:28.987 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3724000000000001e-05 2024-09-21T21:41:28.987 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.987 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.987 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:28.987 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.492218+0000", 2024-09-21T21:41:28.988 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000111729 2024-09-21T21:41:28.988 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:28.988 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:28.988 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:28.988 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.988 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.988 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4840.0:5203 3.1a 3:5fa6cc52:::rbd_header.1121e0174ab1:head [notify cookie 139800380609168] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T21:41:28.988 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:16.464249+0000", 2024-09-21T21:41:28.988 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.460243664, 2024-09-21T21:41:28.988 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.028186418000000001, 2024-09-21T21:41:28.988 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:28.988 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:28.988 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:28.989 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:28.989 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4840", 2024-09-21T21:41:28.989 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1293784138", 2024-09-21T21:41:28.989 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 5203 2024-09-21T21:41:28.989 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.989 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:28.989 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.989 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:28.989 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464249+0000", 2024-09-21T21:41:28.989 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:28.989 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.989 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.989 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:28.990 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464249+0000", 2024-09-21T21:41:28.990 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:28.990 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.990 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.990 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:28.990 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464249+0000", 2024-09-21T21:41:28.990 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.16e-07 2024-09-21T21:41:28.990 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.990 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.990 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:28.990 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464251+0000", 2024-09-21T21:41:28.990 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1859999999999999e-06 2024-09-21T21:41:28.990 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.991 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.991 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:28.991 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464257+0000", 2024-09-21T21:41:28.991 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1340000000000001e-06 2024-09-21T21:41:28.991 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.991 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.991 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:28.991 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464260+0000", 2024-09-21T21:41:28.991 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6240000000000002e-06 2024-09-21T21:41:28.991 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.991 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.991 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:28.991 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.492255+0000", 2024-09-21T21:41:28.991 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.027995181000000001 2024-09-21T21:41:28.991 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.991 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.992 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:28.992 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.492322+0000", 2024-09-21T21:41:28.992 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7139999999999998e-05 2024-09-21T21:41:28.992 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.992 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.992 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:28.992 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.492435+0000", 2024-09-21T21:41:28.992 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00011293700000000001 2024-09-21T21:41:28.992 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:28.992 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:28.992 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:28.992 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.992 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.992 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4840.0:5204 3.1a 3:5fa6cc52:::rbd_header.1121e0174ab1:head [notify cookie 139800381199072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T21:41:28.992 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:16.464263+0000", 2024-09-21T21:41:28.992 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.460229237, 2024-09-21T21:41:28.993 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.028339177, 2024-09-21T21:41:28.993 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:28.993 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:28.993 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:28.993 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:28.993 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4840", 2024-09-21T21:41:28.993 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1293784138", 2024-09-21T21:41:28.993 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 5204 2024-09-21T21:41:28.993 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.993 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:28.993 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.993 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:28.993 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464263+0000", 2024-09-21T21:41:28.993 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:28.993 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.994 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.994 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:28.994 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464263+0000", 2024-09-21T21:41:28.994 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:28.994 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.994 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.994 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:28.994 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464263+0000", 2024-09-21T21:41:28.994 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.22e-07 2024-09-21T21:41:28.994 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.994 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.994 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:28.994 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464265+0000", 2024-09-21T21:41:28.994 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.437e-06 2024-09-21T21:41:28.995 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.995 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.995 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:28.995 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464268+0000", 2024-09-21T21:41:28.995 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.965e-06 2024-09-21T21:41:28.995 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.995 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.995 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:28.995 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464270+0000", 2024-09-21T21:41:28.995 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.565e-06 2024-09-21T21:41:28.995 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.995 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.995 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:28.995 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.492457+0000", 2024-09-21T21:41:28.995 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.028186178999999999 2024-09-21T21:41:28.996 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.996 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.996 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:28.996 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.492520+0000", 2024-09-21T21:41:28.996 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3490000000000004e-05 2024-09-21T21:41:28.996 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.996 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.996 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:28.996 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.492602+0000", 2024-09-21T21:41:28.996 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2318999999999996e-05 2024-09-21T21:41:28.996 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:28.996 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:28.996 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:28.997 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.997 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.997 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4840.0:5205 3.1a 3:5fa6cc52:::rbd_header.1121e0174ab1:head [notify cookie 139800381211120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T21:41:28.997 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:16.464274+0000", 2024-09-21T21:41:28.997 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.460218604, 2024-09-21T21:41:28.997 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.028530798, 2024-09-21T21:41:28.997 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:28.997 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:28.997 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:28.997 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:28.997 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4840", 2024-09-21T21:41:28.997 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1293784138", 2024-09-21T21:41:28.997 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 5205 2024-09-21T21:41:28.997 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.997 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:28.998 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.998 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:28.998 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464274+0000", 2024-09-21T21:41:28.998 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:28.998 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.998 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.998 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:28.998 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464274+0000", 2024-09-21T21:41:28.998 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:28.998 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.998 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.998 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:28.998 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464274+0000", 2024-09-21T21:41:28.999 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1799999999999999e-07 2024-09-21T21:41:28.999 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.999 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.999 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:28.999 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464275+0000", 2024-09-21T21:41:28.999 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3510000000000001e-06 2024-09-21T21:41:28.999 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.999 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:28.999 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:28.999 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464279+0000", 2024-09-21T21:41:28.999 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1219999999999999e-06 2024-09-21T21:41:28.999 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:28.999 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.000 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:29.000 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464281+0000", 2024-09-21T21:41:29.000 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5359999999999999e-06 2024-09-21T21:41:29.000 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.000 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.000 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:29.000 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.492652+0000", 2024-09-21T21:41:29.000 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.028371400000000001 2024-09-21T21:41:29.000 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.000 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.000 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:29.000 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.492738+0000", 2024-09-21T21:41:29.000 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5155999999999996e-05 2024-09-21T21:41:29.001 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.001 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.001 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:29.001 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.492805+0000", 2024-09-21T21:41:29.001 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7014999999999994e-05 2024-09-21T21:41:29.001 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:29.001 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:29.001 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:29.001 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.001 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.001 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4840.0:5206 3.1a 3:5fa6cc52:::rbd_header.1121e0174ab1:head [notify cookie 139800380621360] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T21:41:29.001 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:16.464285+0000", 2024-09-21T21:41:29.001 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.460207713000001, 2024-09-21T21:41:29.001 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.028730286000000001, 2024-09-21T21:41:29.002 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:29.002 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:29.002 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:29.002 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:29.002 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4840", 2024-09-21T21:41:29.002 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1293784138", 2024-09-21T21:41:29.002 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 5206 2024-09-21T21:41:29.002 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.002 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:29.002 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.002 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:29.002 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464285+0000", 2024-09-21T21:41:29.002 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:29.002 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.003 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.003 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:29.003 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464285+0000", 2024-09-21T21:41:29.003 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:29.003 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.003 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.003 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:29.003 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464285+0000", 2024-09-21T21:41:29.003 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2600000000000001e-07 2024-09-21T21:41:29.003 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.003 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.003 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:29.003 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464286+0000", 2024-09-21T21:41:29.003 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.345e-06 2024-09-21T21:41:29.003 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.004 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.004 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:29.004 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464289+0000", 2024-09-21T21:41:29.004 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9340000000000002e-06 2024-09-21T21:41:29.004 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.004 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.004 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:29.004 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464292+0000", 2024-09-21T21:41:29.004 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5220000000000002e-06 2024-09-21T21:41:29.004 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.004 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.004 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:29.004 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.492837+0000", 2024-09-21T21:41:29.005 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.028544773999999998 2024-09-21T21:41:29.005 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.005 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.005 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:29.005 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.492903+0000", 2024-09-21T21:41:29.005 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6086000000000004e-05 2024-09-21T21:41:29.005 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.005 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.005 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:29.005 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.493015+0000", 2024-09-21T21:41:29.005 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00011239900000000001 2024-09-21T21:41:29.005 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:29.005 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:29.006 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:29.006 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.006 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.006 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4840.0:5207 3.1a 3:5fa6cc52:::rbd_header.1121e0174ab1:head [notify cookie 139800380633968] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T21:41:29.006 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:16.464294+0000", 2024-09-21T21:41:29.006 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.460197979, 2024-09-21T21:41:29.006 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.028879564999999999, 2024-09-21T21:41:29.006 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:29.006 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:29.006 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:29.006 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:29.006 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4840", 2024-09-21T21:41:29.006 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1293784138", 2024-09-21T21:41:29.007 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 5207 2024-09-21T21:41:29.007 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.007 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:29.007 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.007 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:29.007 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464294+0000", 2024-09-21T21:41:29.007 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:29.007 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.007 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.007 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:29.007 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464294+0000", 2024-09-21T21:41:29.007 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:29.007 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.008 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.008 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:29.008 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464295+0000", 2024-09-21T21:41:29.008 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.05e-07 2024-09-21T21:41:29.008 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.008 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.008 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:29.008 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464297+0000", 2024-09-21T21:41:29.008 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9910000000000001e-06 2024-09-21T21:41:29.008 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.008 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.008 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:29.008 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464301+0000", 2024-09-21T21:41:29.008 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8909999999999997e-06 2024-09-21T21:41:29.008 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.009 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.009 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:29.009 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464306+0000", 2024-09-21T21:41:29.009 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2469999999999999e-06 2024-09-21T21:41:29.009 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.009 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.009 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:29.009 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.493047+0000", 2024-09-21T21:41:29.009 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.028740875999999999 2024-09-21T21:41:29.009 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.009 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.009 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:29.009 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.493103+0000", 2024-09-21T21:41:29.009 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6651999999999999e-05 2024-09-21T21:41:29.010 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.010 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.010 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:29.010 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.493174+0000", 2024-09-21T21:41:29.010 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0703000000000004e-05 2024-09-21T21:41:29.010 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:29.010 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:29.010 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:29.010 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.010 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.010 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4840.0:5208 3.1a 3:5fa6cc52:::rbd_header.1121e0174ab1:head [notify cookie 139800381241120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T21:41:29.010 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:16.464325+0000", 2024-09-21T21:41:29.010 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.460167852, 2024-09-21T21:41:29.010 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.028998797999999999, 2024-09-21T21:41:29.011 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:29.011 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:29.011 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:29.011 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:29.011 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4840", 2024-09-21T21:41:29.011 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1293784138", 2024-09-21T21:41:29.011 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 5208 2024-09-21T21:41:29.011 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.011 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:29.011 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.011 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:29.011 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464325+0000", 2024-09-21T21:41:29.011 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:29.011 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.012 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.012 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:29.012 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464325+0000", 2024-09-21T21:41:29.012 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:29.012 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.012 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.012 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:29.012 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464325+0000", 2024-09-21T21:41:29.012 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9900000000000001e-07 2024-09-21T21:41:29.012 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.012 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.012 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:29.012 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464328+0000", 2024-09-21T21:41:29.012 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4399999999999999e-06 2024-09-21T21:41:29.012 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.013 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.013 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:29.013 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464335+0000", 2024-09-21T21:41:29.013 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8839999999999994e-06 2024-09-21T21:41:29.013 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.013 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.013 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:29.013 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464340+0000", 2024-09-21T21:41:29.013 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5959999999999997e-06 2024-09-21T21:41:29.013 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.013 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.013 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:29.013 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.493210+0000", 2024-09-21T21:41:29.013 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.028869789999999999 2024-09-21T21:41:29.013 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.014 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.014 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:29.014 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.493264+0000", 2024-09-21T21:41:29.014 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3947999999999999e-05 2024-09-21T21:41:29.014 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.014 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.014 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:29.014 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.493323+0000", 2024-09-21T21:41:29.014 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9641e-05 2024-09-21T21:41:29.014 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:29.014 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:29.014 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:29.014 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.014 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.014 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4840.0:5209 3.1a 3:5fa6cc52:::rbd_header.1121e0174ab1:head [notify cookie 139800380495120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-21T21:41:29.015 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:16.464364+0000", 2024-09-21T21:41:29.015 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.46012859, 2024-09-21T21:41:29.015 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.029077913, 2024-09-21T21:41:29.015 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:29.015 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:29.015 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:29.015 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:29.015 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4840", 2024-09-21T21:41:29.015 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1293784138", 2024-09-21T21:41:29.015 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 5209 2024-09-21T21:41:29.015 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.015 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:29.015 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.016 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:29.016 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464364+0000", 2024-09-21T21:41:29.016 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:29.016 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.016 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.016 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:29.016 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464364+0000", 2024-09-21T21:41:29.016 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:29.016 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.016 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.016 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:29.016 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464364+0000", 2024-09-21T21:41:29.016 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4099999999999999e-07 2024-09-21T21:41:29.016 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.016 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.017 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:29.017 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464367+0000", 2024-09-21T21:41:29.017 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.379e-06 2024-09-21T21:41:29.017 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.017 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.017 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:29.017 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464371+0000", 2024-09-21T21:41:29.017 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.403e-06 2024-09-21T21:41:29.017 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.017 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.017 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:29.017 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.464374+0000", 2024-09-21T21:41:29.017 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9579999999999999e-06 2024-09-21T21:41:29.017 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.018 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.018 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:29.018 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.493352+0000", 2024-09-21T21:41:29.018 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.028977889999999999 2024-09-21T21:41:29.018 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.018 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.018 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:29.018 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.493387+0000", 2024-09-21T21:41:29.018 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4613000000000002e-05 2024-09-21T21:41:29.018 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.018 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.018 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:29.018 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:16.493442+0000", 2024-09-21T21:41:29.018 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5129e-05 2024-09-21T21:41:29.019 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:29.019 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:29.019 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:29.019 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:29.019 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:29.019 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:29.020 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T21:41:29.021 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:29.018+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:29.039 INFO:teuthology.orchestra.run.smithi164.stdout:checking clone #19, image image_client.0-clone19 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent20 2024-09-21T21:41:29.121 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:29.119+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:29.123 INFO:teuthology.orchestra.run.smithi164.stdout:3105 read 0x39cc800 thru 0x39cff6f (0x3770 bytes) 2024-09-21T21:41:29.123 INFO:teuthology.orchestra.run.smithi164.stdout:3106 read 0x84e600 thru 0x85d923 (0xf324 bytes) 2024-09-21T21:41:29.147 INFO:teuthology.orchestra.run.smithi164.stdout:3107 write 0xcc9f800 thru 0xccaf7b7 (0xffb8 bytes) 2024-09-21T21:41:29.148 INFO:teuthology.orchestra.run.smithi164.stdout:3108 write 0xa734400 thru 0xa73c01d (0x7c1e bytes) 2024-09-21T21:41:29.148 INFO:teuthology.orchestra.run.smithi164.stdout:3110 read 0x3309400 thru 0x33181e3 (0xede4 bytes) 2024-09-21T21:41:29.153 INFO:teuthology.orchestra.run.smithi164.stdout:3111 read 0x9f3200 thru 0x9f4849 (0x164a bytes) 2024-09-21T21:41:29.153 INFO:teuthology.orchestra.run.smithi164.stdout:3113 write 0x518a000 thru 0x51927cd (0x87ce bytes) 2024-09-21T21:41:29.154 INFO:teuthology.orchestra.run.smithi164.stdout:3116 read 0x1f69200 thru 0x1f721bc (0x8fbd bytes) 2024-09-21T21:41:29.157 INFO:teuthology.orchestra.run.smithi164.stdout:3118 read 0x5e74a00 thru 0x5e84576 (0xfb77 bytes) 2024-09-21T21:41:29.157 INFO:teuthology.orchestra.run.smithi164.stdout:3119 read 0x10fd200 thru 0x1108c95 (0xba96 bytes) 2024-09-21T21:41:29.160 INFO:teuthology.orchestra.run.smithi164.stdout:3120 read 0x279e00 thru 0x286062 (0xc263 bytes) 2024-09-21T21:41:29.163 INFO:teuthology.orchestra.run.smithi164.stdout:3122 write 0x6bb5e00 thru 0x6bc19d9 (0xbbda bytes) 2024-09-21T21:41:29.163 INFO:teuthology.orchestra.run.smithi164.stdout:3123 read 0x92d5800 thru 0x92e042c (0xac2d bytes) 2024-09-21T21:41:29.168 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:29.168 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:41:29.168 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-21T21:41:29.168 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:29.176 INFO:teuthology.orchestra.run.smithi164.stdout:3124 trunc from 0xccaf800 to 0x1b1c600 2024-09-21T21:41:29.179 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T21:41:29.222 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:29.221+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:29.297 INFO:teuthology.orchestra.run.smithi164.stdout:3125 read 0xab0000 thru 0xab38d8 (0x38d9 bytes) 2024-09-21T21:41:29.301 INFO:teuthology.orchestra.run.smithi164.stdout:3126 clone 22 order 20 su 524288 sc 13 2024-09-21T21:41:29.323 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:29.321+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:29.340 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:29.340 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:41:29.340 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:41:29.340 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:41:29.340 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:29.349 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T21:41:29.423 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:29.421+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:29.452 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:29.452 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-21T21:41:29.452 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-21T21:41:29.452 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:41:29.453 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.453 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1484 3.16 3:6a41a830:::rbd_data.112145fae779.000000000000000a:a [read 303104~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:29.453 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634086+0000", 2024-09-21T21:41:29.453 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 27.816225257999999, 2024-09-21T21:41:29.453 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.058257055000000002, 2024-09-21T21:41:29.453 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:29.453 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:29.453 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:29.453 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:29.453 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:29.453 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:29.453 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1484 2024-09-21T21:41:29.453 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.453 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:29.454 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.454 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:29.454 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634086+0000", 2024-09-21T21:41:29.454 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:29.454 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.454 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.454 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:29.454 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634086+0000", 2024-09-21T21:41:29.454 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:29.454 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.454 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.454 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:29.454 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634086+0000", 2024-09-21T21:41:29.454 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6100000000000001e-07 2024-09-21T21:41:29.455 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.455 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.455 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:29.455 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634088+0000", 2024-09-21T21:41:29.455 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1100000000000001e-06 2024-09-21T21:41:29.455 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.455 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.455 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:29.455 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634095+0000", 2024-09-21T21:41:29.455 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4080000000000003e-06 2024-09-21T21:41:29.455 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.455 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.455 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:29.455 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634100+0000", 2024-09-21T21:41:29.455 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0730000000000004e-06 2024-09-21T21:41:29.456 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.456 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.456 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:29.456 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.692197+0000", 2024-09-21T21:41:29.456 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.058097331000000002 2024-09-21T21:41:29.456 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.456 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.456 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:29.456 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.692271+0000", 2024-09-21T21:41:29.456 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3423999999999994e-05 2024-09-21T21:41:29.456 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.456 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.456 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:29.456 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.692343+0000", 2024-09-21T21:41:29.456 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2248000000000004e-05 2024-09-21T21:41:29.457 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:29.457 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:29.457 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:29.457 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.457 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.457 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1485 3.16 3:695c582e:::rbd_data.1121e540c096.0000000000000008:b [read 335872~36864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:29.457 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634106+0000", 2024-09-21T21:41:29.457 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 27.816205460999999, 2024-09-21T21:41:29.457 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059223685999999998, 2024-09-21T21:41:29.457 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:29.457 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:29.457 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:29.457 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:29.457 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:29.457 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:29.458 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1485 2024-09-21T21:41:29.458 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.458 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:29.458 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.458 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:29.458 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634106+0000", 2024-09-21T21:41:29.458 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:29.458 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.458 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.458 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:29.458 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634106+0000", 2024-09-21T21:41:29.458 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:29.458 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.458 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.459 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:29.459 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634106+0000", 2024-09-21T21:41:29.459 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3300000000000002e-07 2024-09-21T21:41:29.459 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.459 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.459 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:29.459 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634108+0000", 2024-09-21T21:41:29.459 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.996e-06 2024-09-21T21:41:29.459 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.459 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.460 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:29.460 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634115+0000", 2024-09-21T21:41:29.460 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5139999999999998e-06 2024-09-21T21:41:29.460 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.460 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.460 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:29.460 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634120+0000", 2024-09-21T21:41:29.460 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1050000000000003e-06 2024-09-21T21:41:29.460 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.460 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.460 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:29.460 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.692374+0000", 2024-09-21T21:41:29.460 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.058254729999999998 2024-09-21T21:41:29.460 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.460 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.461 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:29.461 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.692440+0000", 2024-09-21T21:41:29.461 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6076999999999998e-05 2024-09-21T21:41:29.461 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.461 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.461 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:29.461 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693329+0000", 2024-09-21T21:41:29.461 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00088873100000000003 2024-09-21T21:41:29.461 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:29.461 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:29.461 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:29.461 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.461 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.461 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1486 3.16 3:6a41a830:::rbd_data.112145fae779.000000000000000a:a [read 360448~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:29.461 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634126+0000", 2024-09-21T21:41:29.462 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 27.816185169000001, 2024-09-21T21:41:29.462 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059399528, 2024-09-21T21:41:29.462 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:29.462 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:29.462 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:29.462 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:29.462 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:29.462 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:29.462 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1486 2024-09-21T21:41:29.462 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.462 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:29.462 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.462 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:29.462 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634126+0000", 2024-09-21T21:41:29.463 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:29.463 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.463 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.463 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:29.463 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634126+0000", 2024-09-21T21:41:29.463 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:29.463 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.463 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.463 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:29.463 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634126+0000", 2024-09-21T21:41:29.463 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4099999999999999e-07 2024-09-21T21:41:29.464 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.464 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.464 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:29.464 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634128+0000", 2024-09-21T21:41:29.464 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1069999999999999e-06 2024-09-21T21:41:29.464 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.464 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.464 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:29.464 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634135+0000", 2024-09-21T21:41:29.464 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.212e-06 2024-09-21T21:41:29.464 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.464 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.464 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:29.464 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634140+0000", 2024-09-21T21:41:29.464 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1429999999999997e-06 2024-09-21T21:41:29.465 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.465 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.465 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:29.465 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693363+0000", 2024-09-21T21:41:29.465 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059223614000000001 2024-09-21T21:41:29.465 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.465 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.465 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:29.465 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693431+0000", 2024-09-21T21:41:29.465 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7294999999999999e-05 2024-09-21T21:41:29.465 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.465 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.465 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:29.465 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693525+0000", 2024-09-21T21:41:29.465 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.4716000000000006e-05 2024-09-21T21:41:29.466 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:29.466 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:29.466 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:29.466 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.466 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.466 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1499 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 49152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:29.466 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634186+0000", 2024-09-21T21:41:29.466 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 27.816125291999999, 2024-09-21T21:41:29.466 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059339256999999999, 2024-09-21T21:41:29.466 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:29.466 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:29.466 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:29.466 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:29.466 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:29.467 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:29.467 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1499 2024-09-21T21:41:29.467 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.467 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:29.467 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.467 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:29.467 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634186+0000", 2024-09-21T21:41:29.467 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:29.467 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.467 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.467 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:29.467 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634186+0000", 2024-09-21T21:41:29.467 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:29.467 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.467 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.468 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:29.468 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634186+0000", 2024-09-21T21:41:29.468 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0999999999999999e-07 2024-09-21T21:41:29.468 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.468 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.468 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:29.468 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634188+0000", 2024-09-21T21:41:29.468 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.86e-06 2024-09-21T21:41:29.468 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.468 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.468 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:29.468 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634194+0000", 2024-09-21T21:41:29.468 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1109999999999996e-06 2024-09-21T21:41:29.468 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.468 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.469 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:29.469 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634199+0000", 2024-09-21T21:41:29.469 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7419999999999997e-06 2024-09-21T21:41:29.469 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.469 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.469 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:29.469 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693357+0000", 2024-09-21T21:41:29.469 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059157847 2024-09-21T21:41:29.469 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.469 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.469 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:29.469 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693430+0000", 2024-09-21T21:41:29.469 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3203999999999999e-05 2024-09-21T21:41:29.469 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.469 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.470 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:29.470 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693525+0000", 2024-09-21T21:41:29.470 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5082999999999997e-05 2024-09-21T21:41:29.470 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:29.470 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:29.470 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:29.470 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.470 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.470 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1501 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 106496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:29.470 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634205+0000", 2024-09-21T21:41:29.470 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 27.816106202, 2024-09-21T21:41:29.470 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059482394000000001, 2024-09-21T21:41:29.470 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:29.470 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:29.471 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:29.471 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:29.471 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:29.471 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:29.471 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1501 2024-09-21T21:41:29.471 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.471 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:29.471 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.471 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:29.471 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634205+0000", 2024-09-21T21:41:29.471 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:29.471 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.471 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.471 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:29.471 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634205+0000", 2024-09-21T21:41:29.472 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:29.472 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.472 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.472 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:29.472 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634205+0000", 2024-09-21T21:41:29.472 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.03e-07 2024-09-21T21:41:29.472 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.472 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.472 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:29.472 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634207+0000", 2024-09-21T21:41:29.472 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0389999999999999e-06 2024-09-21T21:41:29.472 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.472 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.472 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:29.472 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634213+0000", 2024-09-21T21:41:29.473 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2600000000000002e-06 2024-09-21T21:41:29.473 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.473 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.473 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:29.473 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634218+0000", 2024-09-21T21:41:29.473 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8570000000000003e-06 2024-09-21T21:41:29.473 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.473 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.473 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:29.473 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693552+0000", 2024-09-21T21:41:29.473 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059333636000000002 2024-09-21T21:41:29.473 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.473 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.473 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:29.473 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693613+0000", 2024-09-21T21:41:29.474 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0560000000000003e-05 2024-09-21T21:41:29.474 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.474 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.474 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:29.474 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693687+0000", 2024-09-21T21:41:29.474 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4639000000000001e-05 2024-09-21T21:41:29.474 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:29.474 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:29.474 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:29.474 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.474 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.474 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1515 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 434176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:29.474 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634318+0000", 2024-09-21T21:41:29.474 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 27.815992515000001, 2024-09-21T21:41:29.474 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059565812000000003, 2024-09-21T21:41:29.475 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:29.475 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:29.475 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:29.475 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:29.475 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:29.475 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:29.475 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1515 2024-09-21T21:41:29.475 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.475 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:29.475 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.475 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:29.475 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634318+0000", 2024-09-21T21:41:29.475 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:29.475 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.475 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.476 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:29.476 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634318+0000", 2024-09-21T21:41:29.476 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:29.476 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.476 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.476 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:29.476 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634319+0000", 2024-09-21T21:41:29.476 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9400000000000001e-07 2024-09-21T21:41:29.476 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.476 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.476 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:29.476 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634321+0000", 2024-09-21T21:41:29.476 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.154e-06 2024-09-21T21:41:29.476 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.476 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.477 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:29.477 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634327+0000", 2024-09-21T21:41:29.477 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9830000000000001e-06 2024-09-21T21:41:29.477 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.477 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.477 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:29.477 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634332+0000", 2024-09-21T21:41:29.477 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9080000000000003e-06 2024-09-21T21:41:29.477 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.477 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.477 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:29.477 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693717+0000", 2024-09-21T21:41:29.477 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059385447000000001 2024-09-21T21:41:29.477 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.477 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.478 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:29.478 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693795+0000", 2024-09-21T21:41:29.478 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7267000000000004e-05 2024-09-21T21:41:29.478 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.478 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.478 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:29.478 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693884+0000", 2024-09-21T21:41:29.478 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9659000000000003e-05 2024-09-21T21:41:29.478 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:29.478 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:29.478 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:29.478 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.478 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.478 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1517 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 491520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:29.478 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634338+0000", 2024-09-21T21:41:29.479 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 27.815972742, 2024-09-21T21:41:29.479 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060100745999999997, 2024-09-21T21:41:29.479 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:29.479 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:29.479 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:29.479 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:29.479 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:29.479 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:29.479 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1517 2024-09-21T21:41:29.479 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.479 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:29.479 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.479 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:29.480 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634338+0000", 2024-09-21T21:41:29.480 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:29.480 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.480 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.480 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:29.480 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634338+0000", 2024-09-21T21:41:29.480 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:29.480 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.480 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.480 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:29.480 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634339+0000", 2024-09-21T21:41:29.480 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2899999999999999e-07 2024-09-21T21:41:29.480 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.480 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.480 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:29.481 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634341+0000", 2024-09-21T21:41:29.481 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.8959999999999999e-06 2024-09-21T21:41:29.481 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.481 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.481 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:29.481 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634347+0000", 2024-09-21T21:41:29.481 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.302e-06 2024-09-21T21:41:29.481 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.481 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.481 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:29.481 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634351+0000", 2024-09-21T21:41:29.481 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6070000000000001e-06 2024-09-21T21:41:29.481 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.481 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.481 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:29.482 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694279+0000", 2024-09-21T21:41:29.482 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059927046999999997 2024-09-21T21:41:29.482 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.482 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.482 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:29.482 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694355+0000", 2024-09-21T21:41:29.482 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6483999999999995e-05 2024-09-21T21:41:29.482 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.482 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.482 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:29.482 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694439+0000", 2024-09-21T21:41:29.482 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3980999999999994e-05 2024-09-21T21:41:29.482 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:29.482 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:29.482 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:29.483 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.483 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.483 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1518 3.16 3:6a41a830:::rbd_data.112145fae779.000000000000000a:a [read 294912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:29.483 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634357+0000", 2024-09-21T21:41:29.483 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 27.815953835999998, 2024-09-21T21:41:29.483 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059712318, 2024-09-21T21:41:29.483 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:29.483 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:29.483 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:29.483 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:29.483 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:29.483 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:29.483 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1518 2024-09-21T21:41:29.483 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.484 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:29.484 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.484 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:29.484 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634357+0000", 2024-09-21T21:41:29.484 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:29.484 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.484 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.484 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:29.484 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634357+0000", 2024-09-21T21:41:29.484 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:29.484 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.484 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.484 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:29.484 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634358+0000", 2024-09-21T21:41:29.484 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2100000000000002e-07 2024-09-21T21:41:29.485 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.485 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.485 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:29.485 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634360+0000", 2024-09-21T21:41:29.485 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0940000000000002e-06 2024-09-21T21:41:29.485 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.485 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.485 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:29.485 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634366+0000", 2024-09-21T21:41:29.485 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3389999999999998e-06 2024-09-21T21:41:29.485 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.485 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.485 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:29.485 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634371+0000", 2024-09-21T21:41:29.485 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6670000000000004e-06 2024-09-21T21:41:29.486 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.486 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.486 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:29.486 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693910+0000", 2024-09-21T21:41:29.486 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059538973000000002 2024-09-21T21:41:29.486 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.486 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.486 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:29.486 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693982+0000", 2024-09-21T21:41:29.486 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2806999999999996e-05 2024-09-21T21:41:29.486 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.486 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.486 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:29.486 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694069+0000", 2024-09-21T21:41:29.487 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7016999999999997e-05 2024-09-21T21:41:29.487 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:29.487 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:29.487 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:29.487 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.487 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.487 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1519 3.16 3:6a41a830:::rbd_data.112145fae779.000000000000000a:a [read 352256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:29.487 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634377+0000", 2024-09-21T21:41:29.487 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 27.815934133999999, 2024-09-21T21:41:29.487 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059872213000000001, 2024-09-21T21:41:29.487 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:29.487 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:29.487 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:29.487 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:29.487 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:29.488 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:29.488 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1519 2024-09-21T21:41:29.488 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.488 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:29.488 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.488 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:29.488 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634377+0000", 2024-09-21T21:41:29.488 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:29.488 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.488 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.488 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:29.488 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634377+0000", 2024-09-21T21:41:29.488 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:29.488 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.488 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.489 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:29.489 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634377+0000", 2024-09-21T21:41:29.489 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2600000000000002e-07 2024-09-21T21:41:29.489 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.489 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.489 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:29.489 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634379+0000", 2024-09-21T21:41:29.489 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0729999999999999e-06 2024-09-21T21:41:29.489 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.489 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.489 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:29.489 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634386+0000", 2024-09-21T21:41:29.489 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4690000000000002e-06 2024-09-21T21:41:29.489 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.489 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.490 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:29.490 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634390+0000", 2024-09-21T21:41:29.490 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4580000000000003e-06 2024-09-21T21:41:29.490 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.490 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.490 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:29.490 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694094+0000", 2024-09-21T21:41:29.490 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059703830999999999 2024-09-21T21:41:29.490 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.490 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.490 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:29.490 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694168+0000", 2024-09-21T21:41:29.490 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4222999999999997e-05 2024-09-21T21:41:29.490 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.491 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.491 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:29.491 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694249+0000", 2024-09-21T21:41:29.491 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0632999999999999e-05 2024-09-21T21:41:29.491 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:29.491 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:29.491 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:29.491 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.491 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.491 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1529 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 53248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:29.491 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634397+0000", 2024-09-21T21:41:29.491 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 27.815913983000002, 2024-09-21T21:41:29.491 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060189919000000001, 2024-09-21T21:41:29.491 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:29.491 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:29.492 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:29.492 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:29.492 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:29.492 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:29.492 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1529 2024-09-21T21:41:29.492 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.492 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:29.492 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.492 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:29.492 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634397+0000", 2024-09-21T21:41:29.492 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:29.492 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.492 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.492 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:29.492 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634397+0000", 2024-09-21T21:41:29.493 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:29.493 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.493 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.493 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:29.493 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634397+0000", 2024-09-21T21:41:29.493 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2e-07 2024-09-21T21:41:29.493 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.493 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.493 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:29.493 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634399+0000", 2024-09-21T21:41:29.493 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0609999999999998e-06 2024-09-21T21:41:29.493 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.493 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.493 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:29.494 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634406+0000", 2024-09-21T21:41:29.494 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7259999999999996e-06 2024-09-21T21:41:29.494 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.494 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.494 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:29.494 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634411+0000", 2024-09-21T21:41:29.494 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9339999999999997e-06 2024-09-21T21:41:29.494 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.494 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.494 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:29.494 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694480+0000", 2024-09-21T21:41:29.494 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060069105999999997 2024-09-21T21:41:29.494 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.494 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.495 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:29.495 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694527+0000", 2024-09-21T21:41:29.495 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7120000000000003e-05 2024-09-21T21:41:29.495 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.495 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.495 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:29.495 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694587+0000", 2024-09-21T21:41:29.495 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9552000000000002e-05 2024-09-21T21:41:29.495 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:29.495 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:29.495 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:29.495 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.495 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.495 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1531 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 110592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:29.495 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634417+0000", 2024-09-21T21:41:29.496 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 27.815894049000001, 2024-09-21T21:41:29.496 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060347287999999999, 2024-09-21T21:41:29.496 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:29.496 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:29.496 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:29.496 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:29.496 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:29.496 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:29.496 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1531 2024-09-21T21:41:29.496 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.496 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:29.496 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.496 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:29.496 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634417+0000", 2024-09-21T21:41:29.496 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:29.497 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.497 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.497 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:29.497 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634417+0000", 2024-09-21T21:41:29.497 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:29.497 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.497 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.497 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:29.497 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634417+0000", 2024-09-21T21:41:29.497 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1699999999999999e-07 2024-09-21T21:41:29.497 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.497 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.497 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:29.497 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634420+0000", 2024-09-21T21:41:29.497 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.243e-06 2024-09-21T21:41:29.498 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.498 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.498 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:29.498 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634426+0000", 2024-09-21T21:41:29.498 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2550000000000003e-06 2024-09-21T21:41:29.498 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.498 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.498 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:29.498 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634431+0000", 2024-09-21T21:41:29.498 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0150000000000003e-06 2024-09-21T21:41:29.498 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.498 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.498 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:29.498 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694618+0000", 2024-09-21T21:41:29.499 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060186836000000001 2024-09-21T21:41:29.499 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.499 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.499 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:29.499 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694685+0000", 2024-09-21T21:41:29.499 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6970000000000004e-05 2024-09-21T21:41:29.499 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.499 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.499 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:29.499 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694764+0000", 2024-09-21T21:41:29.499 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9552e-05 2024-09-21T21:41:29.499 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:29.499 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:29.499 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:29.499 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.500 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.500 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1532 3.e 3:724b31a3:::rbd_data.1121e540c096.0000000000000015:b [read 151552~36864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:29.500 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634437+0000", 2024-09-21T21:41:29.500 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 27.815874253, 2024-09-21T21:41:29.500 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060474314000000001, 2024-09-21T21:41:29.500 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:29.500 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:29.500 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:29.500 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:29.500 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:29.500 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:29.500 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1532 2024-09-21T21:41:29.500 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.500 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:29.500 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.501 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:29.501 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634437+0000", 2024-09-21T21:41:29.501 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:29.501 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.501 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.501 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:29.501 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634437+0000", 2024-09-21T21:41:29.501 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:29.501 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.501 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.501 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:29.501 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634437+0000", 2024-09-21T21:41:29.501 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9400000000000001e-07 2024-09-21T21:41:29.501 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.502 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.502 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:29.502 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634439+0000", 2024-09-21T21:41:29.502 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.114e-06 2024-09-21T21:41:29.502 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.502 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.502 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:29.502 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634446+0000", 2024-09-21T21:41:29.502 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1389999999999998e-06 2024-09-21T21:41:29.502 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.502 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.502 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:29.502 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634451+0000", 2024-09-21T21:41:29.502 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0010000000000001e-06 2024-09-21T21:41:29.502 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.503 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.503 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:29.503 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694794+0000", 2024-09-21T21:41:29.503 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060342509000000003 2024-09-21T21:41:29.503 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.503 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.503 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:29.503 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694835+0000", 2024-09-21T21:41:29.503 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0728e-05 2024-09-21T21:41:29.503 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.503 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.503 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:29.503 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694911+0000", 2024-09-21T21:41:29.503 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6428999999999996e-05 2024-09-21T21:41:29.504 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:29.504 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:29.504 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:29.504 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.504 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.504 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1534 3.e 3:724b31a3:::rbd_data.1121e540c096.0000000000000015:b [read 487424~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:29.504 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634477+0000", 2024-09-21T21:41:29.504 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 27.815833850000001, 2024-09-21T21:41:29.504 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060624009, 2024-09-21T21:41:29.504 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:29.504 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:29.504 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:29.504 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:29.504 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:29.504 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:29.504 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1534 2024-09-21T21:41:29.505 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.505 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:29.505 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.505 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:29.505 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634477+0000", 2024-09-21T21:41:29.505 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:29.505 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.505 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.505 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:29.505 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634477+0000", 2024-09-21T21:41:29.505 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:29.505 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.505 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.505 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:29.505 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634478+0000", 2024-09-21T21:41:29.506 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1399999999999997e-07 2024-09-21T21:41:29.506 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.506 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.506 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:29.506 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634480+0000", 2024-09-21T21:41:29.506 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1009999999999999e-06 2024-09-21T21:41:29.506 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.506 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.506 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:29.506 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634486+0000", 2024-09-21T21:41:29.506 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3289999999999999e-06 2024-09-21T21:41:29.506 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.506 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.506 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:29.507 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634491+0000", 2024-09-21T21:41:29.507 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6260000000000003e-06 2024-09-21T21:41:29.507 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.507 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.507 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:29.507 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694943+0000", 2024-09-21T21:41:29.507 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060451960999999999 2024-09-21T21:41:29.507 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.507 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.507 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:29.507 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695014+0000", 2024-09-21T21:41:29.507 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1587000000000004e-05 2024-09-21T21:41:29.507 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.507 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.507 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:29.508 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695101+0000", 2024-09-21T21:41:29.508 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6990999999999994e-05 2024-09-21T21:41:29.508 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:29.508 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:29.508 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:29.508 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.508 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.508 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1545 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 438272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:29.508 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634515+0000", 2024-09-21T21:41:29.508 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 27.815795649999998, 2024-09-21T21:41:29.508 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060707944, 2024-09-21T21:41:29.508 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:29.508 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:29.508 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:29.508 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:29.509 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:29.509 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:29.509 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1545 2024-09-21T21:41:29.509 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.509 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:29.509 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.509 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:29.509 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634515+0000", 2024-09-21T21:41:29.509 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:29.509 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.509 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.509 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:29.509 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634515+0000", 2024-09-21T21:41:29.509 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:29.509 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.510 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.510 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:29.510 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634516+0000", 2024-09-21T21:41:29.510 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0999999999999999e-07 2024-09-21T21:41:29.510 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.510 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.510 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:29.510 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634518+0000", 2024-09-21T21:41:29.510 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.846e-06 2024-09-21T21:41:29.510 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.510 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.510 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:29.510 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634524+0000", 2024-09-21T21:41:29.510 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2550000000000003e-06 2024-09-21T21:41:29.510 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.511 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.511 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:29.511 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634529+0000", 2024-09-21T21:41:29.511 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6940000000000003e-06 2024-09-21T21:41:29.511 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.511 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.511 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:29.511 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695127+0000", 2024-09-21T21:41:29.511 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060598921 2024-09-21T21:41:29.511 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.511 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.511 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:29.511 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695171+0000", 2024-09-21T21:41:29.511 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3588999999999999e-05 2024-09-21T21:41:29.512 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.512 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.512 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:29.512 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695223+0000", 2024-09-21T21:41:29.512 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2228999999999997e-05 2024-09-21T21:41:29.512 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:29.512 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:29.512 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:29.512 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.512 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.512 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1547 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 495616~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:29.512 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634534+0000", 2024-09-21T21:41:29.512 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 27.815776859, 2024-09-21T21:41:29.512 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060881131999999998, 2024-09-21T21:41:29.512 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:29.512 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:29.513 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:29.513 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:29.513 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:29.513 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:29.513 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1547 2024-09-21T21:41:29.513 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.513 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:29.513 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.513 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:29.513 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634534+0000", 2024-09-21T21:41:29.513 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:29.513 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.513 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.513 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:29.514 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634534+0000", 2024-09-21T21:41:29.514 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:29.514 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.514 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.514 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:29.514 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634535+0000", 2024-09-21T21:41:29.514 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.77e-07 2024-09-21T21:41:29.514 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.514 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.514 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:29.514 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634542+0000", 2024-09-21T21:41:29.514 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9000000000000006e-06 2024-09-21T21:41:29.514 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.514 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.514 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:29.515 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634549+0000", 2024-09-21T21:41:29.515 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5910000000000001e-06 2024-09-21T21:41:29.515 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.515 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.515 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:29.515 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634554+0000", 2024-09-21T21:41:29.515 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7940000000000002e-06 2024-09-21T21:41:29.515 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.515 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.515 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:29.515 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695256+0000", 2024-09-21T21:41:29.515 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.06070246 2024-09-21T21:41:29.515 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.515 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.515 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:29.516 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695332+0000", 2024-09-21T21:41:29.516 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6248000000000006e-05 2024-09-21T21:41:29.516 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.516 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.516 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:29.516 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695415+0000", 2024-09-21T21:41:29.516 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2762000000000005e-05 2024-09-21T21:41:29.516 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:29.516 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:29.516 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:29.516 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.516 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.516 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1548 3.16 3:6a41a830:::rbd_data.112145fae779.000000000000000a:a [read 299008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:29.517 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634561+0000", 2024-09-21T21:41:29.517 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 27.815750282, 2024-09-21T21:41:29.517 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060786743999999997, 2024-09-21T21:41:29.517 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:29.517 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:29.517 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:29.517 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:29.517 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:29.517 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:29.517 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1548 2024-09-21T21:41:29.517 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.517 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:29.517 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.517 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:29.517 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634561+0000", 2024-09-21T21:41:29.518 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:29.518 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.518 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.518 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:29.518 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634561+0000", 2024-09-21T21:41:29.518 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:29.518 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.518 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.518 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:29.518 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634561+0000", 2024-09-21T21:41:29.518 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5199999999999997e-07 2024-09-21T21:41:29.518 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.518 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.518 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:29.519 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634564+0000", 2024-09-21T21:41:29.519 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4040000000000002e-06 2024-09-21T21:41:29.519 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.519 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.519 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:29.519 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634571+0000", 2024-09-21T21:41:29.519 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4379999999999998e-06 2024-09-21T21:41:29.519 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.519 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.519 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:29.519 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634576+0000", 2024-09-21T21:41:29.519 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1399999999999999e-06 2024-09-21T21:41:29.519 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.519 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.519 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:29.520 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695247+0000", 2024-09-21T21:41:29.520 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060670397000000001 2024-09-21T21:41:29.520 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.520 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.520 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:29.520 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695299+0000", 2024-09-21T21:41:29.520 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2564e-05 2024-09-21T21:41:29.520 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.520 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.520 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:29.520 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695347+0000", 2024-09-21T21:41:29.520 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8248999999999998e-05 2024-09-21T21:41:29.520 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:29.520 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:29.520 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:29.521 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.521 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.521 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1549 3.16 3:6a41a830:::rbd_data.112145fae779.000000000000000a:a [read 356352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:29.521 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634583+0000", 2024-09-21T21:41:29.521 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 27.815728368999999, 2024-09-21T21:41:29.521 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060889459, 2024-09-21T21:41:29.521 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:29.521 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:29.521 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:29.521 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:29.521 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:29.521 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:29.521 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1549 2024-09-21T21:41:29.521 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.521 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:29.522 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.522 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:29.522 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634583+0000", 2024-09-21T21:41:29.522 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:29.522 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.522 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.522 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:29.522 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634583+0000", 2024-09-21T21:41:29.522 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:29.522 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.522 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.522 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:29.522 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634583+0000", 2024-09-21T21:41:29.522 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1899999999999998e-07 2024-09-21T21:41:29.523 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.523 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.523 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:29.523 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634585+0000", 2024-09-21T21:41:29.523 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2129999999999998e-06 2024-09-21T21:41:29.523 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.523 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.523 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:29.523 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634592+0000", 2024-09-21T21:41:29.523 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2840000000000003e-06 2024-09-21T21:41:29.523 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.523 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.523 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:29.523 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634596+0000", 2024-09-21T21:41:29.523 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9309999999999999e-06 2024-09-21T21:41:29.524 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.524 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.524 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:29.524 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695371+0000", 2024-09-21T21:41:29.524 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060774425999999999 2024-09-21T21:41:29.524 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.524 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.524 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:29.524 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695419+0000", 2024-09-21T21:41:29.524 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8607999999999999e-05 2024-09-21T21:41:29.524 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.524 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.524 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:29.524 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695472+0000", 2024-09-21T21:41:29.525 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2577999999999997e-05 2024-09-21T21:41:29.525 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:29.525 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:29.525 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:29.525 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.525 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.525 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1588 3.16 3:695c582e:::rbd_data.1121e540c096.0000000000000008:b [read 147456~40960] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:29.525 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.636942+0000", 2024-09-21T21:41:29.525 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 27.813368592, 2024-09-21T21:41:29.525 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.058643811999999997, 2024-09-21T21:41:29.525 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:29.525 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:29.525 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:29.525 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:29.525 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:29.525 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:29.526 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1588 2024-09-21T21:41:29.526 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.526 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:29.526 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.526 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:29.526 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636942+0000", 2024-09-21T21:41:29.526 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:29.526 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.526 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.526 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:29.526 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636942+0000", 2024-09-21T21:41:29.526 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:29.526 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.526 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.527 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:29.527 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636943+0000", 2024-09-21T21:41:29.527 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4199999999999996e-07 2024-09-21T21:41:29.527 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.527 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.527 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:29.527 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636945+0000", 2024-09-21T21:41:29.527 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.401e-06 2024-09-21T21:41:29.527 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.527 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.527 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:29.527 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636952+0000", 2024-09-21T21:41:29.527 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5379999999999999e-06 2024-09-21T21:41:29.527 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.527 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.528 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:29.528 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636958+0000", 2024-09-21T21:41:29.528 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.489e-06 2024-09-21T21:41:29.528 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.528 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.528 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:29.528 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695497+0000", 2024-09-21T21:41:29.528 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.058538504999999998 2024-09-21T21:41:29.528 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.528 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.528 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:29.528 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695533+0000", 2024-09-21T21:41:29.528 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6118000000000002e-05 2024-09-21T21:41:29.528 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.528 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.529 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:29.529 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695586+0000", 2024-09-21T21:41:29.529 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3219e-05 2024-09-21T21:41:29.529 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:29.529 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:29.529 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:29.529 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.529 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.529 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1589 3.16 3:695c582e:::rbd_data.1121e540c096.0000000000000008:b [read 487424~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:29.529 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.636965+0000", 2024-09-21T21:41:29.529 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 27.813345501000001, 2024-09-21T21:41:29.529 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.058728152999999998, 2024-09-21T21:41:29.529 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:29.529 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:29.529 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:29.530 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:29.530 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:29.530 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:29.530 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1589 2024-09-21T21:41:29.530 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.530 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:29.530 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.530 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:29.530 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636965+0000", 2024-09-21T21:41:29.530 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:29.530 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.530 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.530 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:29.530 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636965+0000", 2024-09-21T21:41:29.530 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:29.531 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.531 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.531 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:29.531 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636966+0000", 2024-09-21T21:41:29.531 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3799999999999998e-07 2024-09-21T21:41:29.531 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.531 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.531 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:29.531 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636968+0000", 2024-09-21T21:41:29.531 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2400000000000002e-06 2024-09-21T21:41:29.531 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.531 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.531 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:29.531 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636976+0000", 2024-09-21T21:41:29.531 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5050000000000001e-06 2024-09-21T21:41:29.532 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.532 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.532 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:29.532 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636981+0000", 2024-09-21T21:41:29.532 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2170000000000002e-06 2024-09-21T21:41:29.532 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.532 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.532 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:29.532 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695610+0000", 2024-09-21T21:41:29.532 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.058628998000000002 2024-09-21T21:41:29.532 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.532 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.532 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:29.532 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695647+0000", 2024-09-21T21:41:29.532 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6752000000000003e-05 2024-09-21T21:41:29.533 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.533 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.533 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:29.533 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695694+0000", 2024-09-21T21:41:29.533 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7002999999999997e-05 2024-09-21T21:41:29.533 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:29.533 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:29.533 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:29.533 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.533 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.533 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1607 3.e 3:724b31a3:::rbd_data.1121e540c096.0000000000000015:b [read 299008~36864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:29.533 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.637062+0000", 2024-09-21T21:41:29.533 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 27.813249410000001, 2024-09-21T21:41:29.533 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.058623078000000002, 2024-09-21T21:41:29.533 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:29.534 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:29.534 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:29.534 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:29.534 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:29.534 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:29.534 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1607 2024-09-21T21:41:29.534 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.534 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:29.534 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.534 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:29.534 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.637062+0000", 2024-09-21T21:41:29.534 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:29.534 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.534 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.534 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:29.535 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.637062+0000", 2024-09-21T21:41:29.535 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:29.535 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.535 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.535 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:29.535 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.637062+0000", 2024-09-21T21:41:29.535 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0900000000000002e-07 2024-09-21T21:41:29.535 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.535 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.535 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:29.535 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.637069+0000", 2024-09-21T21:41:29.535 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.267e-06 2024-09-21T21:41:29.535 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.535 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.535 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:29.536 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.637076+0000", 2024-09-21T21:41:29.536 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7320000000000001e-06 2024-09-21T21:41:29.536 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.536 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.536 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:29.536 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.637082+0000", 2024-09-21T21:41:29.536 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4760000000000003e-06 2024-09-21T21:41:29.536 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.536 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.536 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:29.536 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695604+0000", 2024-09-21T21:41:29.536 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.058521261999999998 2024-09-21T21:41:29.536 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.536 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.536 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:29.537 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695638+0000", 2024-09-21T21:41:29.537 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4292999999999998e-05 2024-09-21T21:41:29.537 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:29.537 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:29.537 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:29.537 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695685+0000", 2024-09-21T21:41:29.537 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6638999999999998e-05 2024-09-21T21:41:29.537 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:29.537 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:29.537 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:29.537 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:29.537 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:29.537 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:29.539 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:29.536+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:29.539 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T21:41:29.640 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:29.638+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:29.642 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:29.642 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:41:29.642 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 0 2024-09-21T21:41:29.642 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:29.651 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T21:41:29.669 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-21T21:41:29.669 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T21:41:29.669 INFO:tasks.thrashosds.thrasher:inject_pause on osd.3 2024-09-21T21:41:29.670 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-21T21:41:29.670 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-21T21:41:29.670 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok config set filestore_inject_stall 3 2024-09-21T21:41:29.677 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T21:41:29.741 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:29.738+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:29.749 INFO:teuthology.orchestra.run.smithi164.stdout:truncating image image_client.0-clone21 from 0x1b1c600 (overlap 0x1b1c600) to 0x2b6000 2024-09-21T21:41:29.761 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:29.761 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:41:29.762 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:41:29.762 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:41:29.762 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:29.774 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T21:41:29.800 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:29.801 INFO:teuthology.orchestra.run.smithi122.stdout: "success": "" 2024-09-21T21:41:29.801 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:29.853 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:29.851+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:29.933 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T21:41:29.937 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:29.937 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 20, 2024-09-21T21:41:29.937 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 600, 2024-09-21T21:41:29.937 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:41:29.937 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.938 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4952.0:3526 3.10 3:086eb8a4:::rbd_data.1121a3eabc61.0000000000000000:17 [read 2410496~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:29.938 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:29.795290+0000", 2024-09-21T21:41:29.938 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.138628628, 2024-09-21T21:41:29.938 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.032825482000000003, 2024-09-21T21:41:29.938 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:29.938 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:29.938 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:29.938 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:29.938 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4952", 2024-09-21T21:41:29.938 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:29.938 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3526 2024-09-21T21:41:29.938 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.938 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:29.939 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.939 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:29.939 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795290+0000", 2024-09-21T21:41:29.939 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:29.939 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.939 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.939 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:29.939 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795290+0000", 2024-09-21T21:41:29.939 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:29.939 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.939 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.939 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:29.939 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795291+0000", 2024-09-21T21:41:29.939 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3499999999999996e-07 2024-09-21T21:41:29.939 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.940 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.940 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:29.940 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795293+0000", 2024-09-21T21:41:29.940 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0779999999999998e-06 2024-09-21T21:41:29.940 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.940 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.940 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:29.940 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795299+0000", 2024-09-21T21:41:29.940 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7270000000000004e-06 2024-09-21T21:41:29.940 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.940 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.940 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:29.940 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795303+0000", 2024-09-21T21:41:29.940 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3579999999999996e-06 2024-09-21T21:41:29.941 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.941 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.941 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:29.941 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.827920+0000", 2024-09-21T21:41:29.941 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.032617118 2024-09-21T21:41:29.941 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.941 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.941 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:29.941 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.828011+0000", 2024-09-21T21:41:29.941 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.0676999999999999e-05 2024-09-21T21:41:29.941 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.942 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.942 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:29.943 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.828116+0000", 2024-09-21T21:41:29.943 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010498899999999999 2024-09-21T21:41:29.943 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:29.943 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:29.943 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:29.943 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.943 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.943 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4952.0:3528 3.10 3:086eb8a4:::rbd_data.1121a3eabc61.0000000000000000:17 [read 2424832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:29.943 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:29.795308+0000", 2024-09-21T21:41:29.943 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.13861090600000001, 2024-09-21T21:41:29.943 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033028465999999999, 2024-09-21T21:41:29.943 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:29.943 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:29.944 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:29.944 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:29.944 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4952", 2024-09-21T21:41:29.944 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:29.944 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3528 2024-09-21T21:41:29.944 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.944 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:29.944 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.944 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:29.944 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795308+0000", 2024-09-21T21:41:29.944 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:29.944 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.944 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.944 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:29.945 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795308+0000", 2024-09-21T21:41:29.945 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:29.945 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.945 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.945 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:29.945 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795308+0000", 2024-09-21T21:41:29.945 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.22e-07 2024-09-21T21:41:29.945 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.945 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.945 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:29.945 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795310+0000", 2024-09-21T21:41:29.945 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.7940000000000001e-06 2024-09-21T21:41:29.945 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.945 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.946 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:29.946 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795316+0000", 2024-09-21T21:41:29.946 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9410000000000004e-06 2024-09-21T21:41:29.946 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.946 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.946 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:29.946 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795321+0000", 2024-09-21T21:41:29.946 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4800000000000003e-06 2024-09-21T21:41:29.946 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.946 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.947 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:29.947 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.828149+0000", 2024-09-21T21:41:29.947 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.032828001000000002 2024-09-21T21:41:29.947 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.947 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.947 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:29.947 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.828223+0000", 2024-09-21T21:41:29.947 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.4048000000000007e-05 2024-09-21T21:41:29.947 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.947 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.947 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:29.947 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.828337+0000", 2024-09-21T21:41:29.947 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011388 2024-09-21T21:41:29.947 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:29.948 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:29.948 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:29.948 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.948 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.948 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4952.0:3529 3.10 3:086eb8a4:::rbd_data.1121a3eabc61.0000000000000000:17 [read 2439168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:29.948 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:29.795328+0000", 2024-09-21T21:41:29.948 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.13859081000000001, 2024-09-21T21:41:29.948 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033236716999999999, 2024-09-21T21:41:29.948 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:29.948 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:29.948 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:29.948 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:29.948 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4952", 2024-09-21T21:41:29.948 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:29.949 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3529 2024-09-21T21:41:29.949 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.949 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:29.949 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.949 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:29.949 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795328+0000", 2024-09-21T21:41:29.949 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:29.949 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.949 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.949 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:29.949 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795328+0000", 2024-09-21T21:41:29.949 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:29.949 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.949 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.949 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:29.950 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795329+0000", 2024-09-21T21:41:29.950 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.44e-07 2024-09-21T21:41:29.950 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.950 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.950 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:29.950 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795335+0000", 2024-09-21T21:41:29.950 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7400000000000001e-06 2024-09-21T21:41:29.950 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.950 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.950 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:29.951 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795340+0000", 2024-09-21T21:41:29.951 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9009999999999999e-06 2024-09-21T21:41:29.951 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.951 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.951 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:29.951 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795345+0000", 2024-09-21T21:41:29.951 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4839999999999998e-06 2024-09-21T21:41:29.951 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.951 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.951 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:29.951 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.828382+0000", 2024-09-21T21:41:29.951 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033037113 2024-09-21T21:41:29.952 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.952 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.952 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:29.952 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.828473+0000", 2024-09-21T21:41:29.952 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.1056000000000004e-05 2024-09-21T21:41:29.952 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.952 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.952 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:29.952 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.828565+0000", 2024-09-21T21:41:29.952 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.1879000000000006e-05 2024-09-21T21:41:29.952 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:29.952 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:29.952 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:29.952 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.952 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.953 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4952.0:3531 3.10 3:086eb8a4:::rbd_data.1121a3eabc61.0000000000000000:17 [read 2453504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:29.953 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:29.795353+0000", 2024-09-21T21:41:29.953 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.13856608100000001, 2024-09-21T21:41:29.953 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033394973000000001, 2024-09-21T21:41:29.953 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:29.953 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:29.953 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:29.953 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:29.953 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4952", 2024-09-21T21:41:29.953 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:29.954 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3531 2024-09-21T21:41:29.954 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.954 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:29.954 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.954 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:29.954 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795353+0000", 2024-09-21T21:41:29.954 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:29.954 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.954 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.954 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:29.954 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795353+0000", 2024-09-21T21:41:29.954 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:29.954 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.954 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.955 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:29.955 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795353+0000", 2024-09-21T21:41:29.955 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7300000000000001e-07 2024-09-21T21:41:29.955 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.955 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.955 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:29.955 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795360+0000", 2024-09-21T21:41:29.955 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2829999999999998e-06 2024-09-21T21:41:29.955 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.955 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.955 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:29.955 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795366+0000", 2024-09-21T21:41:29.955 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1260000000000003e-06 2024-09-21T21:41:29.955 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.955 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.956 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:29.956 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795370+0000", 2024-09-21T21:41:29.956 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1899999999999997e-06 2024-09-21T21:41:29.956 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.956 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.956 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:29.956 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.828587+0000", 2024-09-21T21:41:29.956 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033216516000000001 2024-09-21T21:41:29.956 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.956 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.956 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:29.956 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.828662+0000", 2024-09-21T21:41:29.956 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5316999999999997e-05 2024-09-21T21:41:29.956 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.957 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.957 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:29.957 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.828748+0000", 2024-09-21T21:41:29.957 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.6068000000000003e-05 2024-09-21T21:41:29.957 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:29.957 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:29.957 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:29.957 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.957 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.957 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4952.0:3533 3.10 3:086eb8a4:::rbd_data.1121a3eabc61.0000000000000000:17 [read 2467840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:29.957 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:29.795376+0000", 2024-09-21T21:41:29.957 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.13854288000000001, 2024-09-21T21:41:29.957 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033574701999999998, 2024-09-21T21:41:29.957 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:29.957 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:29.958 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:29.958 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:29.958 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4952", 2024-09-21T21:41:29.958 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:29.958 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3533 2024-09-21T21:41:29.958 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.958 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:29.958 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.958 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:29.958 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795376+0000", 2024-09-21T21:41:29.958 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:29.958 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.958 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.958 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:29.959 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795376+0000", 2024-09-21T21:41:29.959 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:29.959 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.959 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.959 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:29.959 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795377+0000", 2024-09-21T21:41:29.959 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5400000000000002e-07 2024-09-21T21:41:29.959 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.959 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.959 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:29.959 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795379+0000", 2024-09-21T21:41:29.959 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9659999999999999e-06 2024-09-21T21:41:29.959 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.959 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.960 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:29.960 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795385+0000", 2024-09-21T21:41:29.960 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1519999999999997e-06 2024-09-21T21:41:29.960 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.960 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.960 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:29.960 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795389+0000", 2024-09-21T21:41:29.960 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4399999999999998e-06 2024-09-21T21:41:29.960 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.960 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.960 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:29.960 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.828777+0000", 2024-09-21T21:41:29.960 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033388099999999997 2024-09-21T21:41:29.960 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.960 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.961 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:29.961 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.828856+0000", 2024-09-21T21:41:29.961 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.8770999999999995e-05 2024-09-21T21:41:29.961 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.961 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.961 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:29.961 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.828951+0000", 2024-09-21T21:41:29.961 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.4919000000000004e-05 2024-09-21T21:41:29.961 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:29.961 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:29.961 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:29.961 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.961 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.961 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4952.0:3535 3.10 3:086eb8a4:::rbd_data.1121a3eabc61.0000000000000000:17 [read 2482176~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:29.962 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:29.795394+0000", 2024-09-21T21:41:29.962 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.138524751, 2024-09-21T21:41:29.962 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033738335000000001, 2024-09-21T21:41:29.962 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:29.962 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:29.962 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:29.962 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:29.962 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4952", 2024-09-21T21:41:29.962 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:29.962 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3535 2024-09-21T21:41:29.962 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.962 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:29.962 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.962 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:29.963 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795394+0000", 2024-09-21T21:41:29.963 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:29.963 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.963 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.963 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:29.963 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795394+0000", 2024-09-21T21:41:29.963 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:29.963 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.963 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.963 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:29.963 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.796945+0000", 2024-09-21T21:41:29.963 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.001551097 2024-09-21T21:41:29.963 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.963 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.963 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:29.964 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.796949+0000", 2024-09-21T21:41:29.964 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5499999999999999e-06 2024-09-21T21:41:29.964 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.964 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.964 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:29.964 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.796958+0000", 2024-09-21T21:41:29.964 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.1649999999999995e-06 2024-09-21T21:41:29.964 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.964 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.964 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:29.964 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.796964+0000", 2024-09-21T21:41:29.964 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1199999999999999e-06 2024-09-21T21:41:29.964 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.964 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.964 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:29.965 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.828983+0000", 2024-09-21T21:41:29.965 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.032018396999999997 2024-09-21T21:41:29.965 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.965 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.965 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:29.965 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.829050+0000", 2024-09-21T21:41:29.965 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.7020000000000005e-05 2024-09-21T21:41:29.965 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.965 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.965 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:29.965 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.829133+0000", 2024-09-21T21:41:29.965 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.2985999999999995e-05 2024-09-21T21:41:29.965 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:29.966 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:29.966 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:29.966 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.966 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.966 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4952.0:3544 3.10 3:086eb8a4:::rbd_data.1121a3eabc61.0000000000000000:17 [read 2562048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:29.966 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:29.797097+0000", 2024-09-21T21:41:29.966 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.13682194, 2024-09-21T21:41:29.966 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.032772161000000001, 2024-09-21T21:41:29.966 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:29.966 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:29.966 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:29.966 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:29.966 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4952", 2024-09-21T21:41:29.966 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:29.967 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3544 2024-09-21T21:41:29.967 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.967 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:29.967 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.967 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:29.967 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797097+0000", 2024-09-21T21:41:29.967 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:29.967 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.967 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.967 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:29.967 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797097+0000", 2024-09-21T21:41:29.967 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:29.967 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.967 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.968 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:29.968 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797097+0000", 2024-09-21T21:41:29.968 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.7799999999999997e-07 2024-09-21T21:41:29.968 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.968 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.968 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:29.968 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797099+0000", 2024-09-21T21:41:29.968 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.844e-06 2024-09-21T21:41:29.968 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.968 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.969 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:29.969 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797107+0000", 2024-09-21T21:41:29.969 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.4399999999999999e-06 2024-09-21T21:41:29.969 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.969 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.969 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:29.969 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797111+0000", 2024-09-21T21:41:29.969 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5220000000000001e-06 2024-09-21T21:41:29.969 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.969 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.969 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:29.969 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.829785+0000", 2024-09-21T21:41:29.969 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.032673319999999999 2024-09-21T21:41:29.970 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.970 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.970 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:29.970 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.829822+0000", 2024-09-21T21:41:29.970 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7376999999999998e-05 2024-09-21T21:41:29.970 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.970 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.970 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:29.970 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.829869+0000", 2024-09-21T21:41:29.970 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7379999999999997e-05 2024-09-21T21:41:29.970 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:29.971 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:29.971 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:29.971 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.971 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.971 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4952.0:3546 3.10 3:086eb8a4:::rbd_data.1121a3eabc61.0000000000000000:17 [read 2576384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:29.971 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:29.797117+0000", 2024-09-21T21:41:29.971 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.13680189700000001, 2024-09-21T21:41:29.971 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.03288253, 2024-09-21T21:41:29.971 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:29.971 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:29.971 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:29.971 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:29.971 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4952", 2024-09-21T21:41:29.971 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:29.972 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3546 2024-09-21T21:41:29.972 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.972 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:29.972 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.972 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:29.972 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797117+0000", 2024-09-21T21:41:29.972 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:29.972 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.972 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.972 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:29.972 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797117+0000", 2024-09-21T21:41:29.972 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:29.972 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.972 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.973 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:29.973 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797118+0000", 2024-09-21T21:41:29.973 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.39e-07 2024-09-21T21:41:29.973 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.973 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.973 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:29.973 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797120+0000", 2024-09-21T21:41:29.973 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9980000000000002e-06 2024-09-21T21:41:29.973 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.973 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.973 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:29.973 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797125+0000", 2024-09-21T21:41:29.973 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7409999999999998e-06 2024-09-21T21:41:29.973 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.973 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.974 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:29.974 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797130+0000", 2024-09-21T21:41:29.974 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5820000000000003e-06 2024-09-21T21:41:29.974 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.974 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.974 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:29.974 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.829898+0000", 2024-09-21T21:41:29.974 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.032768600000000002 2024-09-21T21:41:29.974 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.974 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.974 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:29.974 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.829934+0000", 2024-09-21T21:41:29.974 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5342000000000001e-05 2024-09-21T21:41:29.974 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.975 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.975 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:29.975 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830000+0000", 2024-09-21T21:41:29.975 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5927999999999996e-05 2024-09-21T21:41:29.975 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:29.975 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:29.975 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:29.975 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.975 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.975 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4952.0:3548 3.10 3:086eb8a4:::rbd_data.1121a3eabc61.0000000000000000:17 [read 2590720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:29.975 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:29.797136+0000", 2024-09-21T21:41:29.975 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.136783457, 2024-09-21T21:41:29.975 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033006742999999998, 2024-09-21T21:41:29.975 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:29.975 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:29.976 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:29.976 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:29.976 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4952", 2024-09-21T21:41:29.976 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:29.976 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3548 2024-09-21T21:41:29.976 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.976 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:29.976 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.976 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:29.976 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797136+0000", 2024-09-21T21:41:29.976 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:29.976 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.976 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.976 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:29.976 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797136+0000", 2024-09-21T21:41:29.977 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:29.977 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.977 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.977 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:29.977 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797136+0000", 2024-09-21T21:41:29.977 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3600000000000004e-07 2024-09-21T21:41:29.977 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.977 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.977 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:29.977 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797138+0000", 2024-09-21T21:41:29.977 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.933e-06 2024-09-21T21:41:29.977 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.977 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.977 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:29.978 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797144+0000", 2024-09-21T21:41:29.978 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6099999999999997e-06 2024-09-21T21:41:29.978 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.978 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.978 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:29.978 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797149+0000", 2024-09-21T21:41:29.978 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0690000000000001e-06 2024-09-21T21:41:29.978 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.978 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.978 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:29.978 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830038+0000", 2024-09-21T21:41:29.978 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.032889535999999997 2024-09-21T21:41:29.978 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.978 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.978 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:29.979 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830087+0000", 2024-09-21T21:41:29.979 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9159999999999997e-05 2024-09-21T21:41:29.979 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.979 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.979 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:29.979 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830142+0000", 2024-09-21T21:41:29.979 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4898999999999997e-05 2024-09-21T21:41:29.979 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:29.979 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:29.979 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:29.979 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.979 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.979 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4952.0:3550 3.10 3:086eb8a4:::rbd_data.1121a3eabc61.0000000000000000:17 [read 2605056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:29.979 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:29.797154+0000", 2024-09-21T21:41:29.979 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.136764899, 2024-09-21T21:41:29.980 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033113318000000003, 2024-09-21T21:41:29.980 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:29.980 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:29.980 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:29.980 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:29.980 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4952", 2024-09-21T21:41:29.980 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:29.980 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3550 2024-09-21T21:41:29.980 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.980 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:29.980 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.980 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:29.980 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797154+0000", 2024-09-21T21:41:29.980 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:29.980 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.981 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.981 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:29.981 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797154+0000", 2024-09-21T21:41:29.981 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:29.981 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.981 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.981 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:29.981 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797155+0000", 2024-09-21T21:41:29.981 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1199999999999998e-07 2024-09-21T21:41:29.981 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.981 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.981 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:29.981 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797165+0000", 2024-09-21T21:41:29.981 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0635000000000001e-05 2024-09-21T21:41:29.982 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.982 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.982 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:29.982 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797172+0000", 2024-09-21T21:41:29.982 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5300000000000002e-06 2024-09-21T21:41:29.982 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.982 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.982 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:29.982 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797176+0000", 2024-09-21T21:41:29.982 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7419999999999997e-06 2024-09-21T21:41:29.982 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.982 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.982 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:29.982 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830167+0000", 2024-09-21T21:41:29.982 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.032990468000000002 2024-09-21T21:41:29.983 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.983 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.983 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:29.983 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830202+0000", 2024-09-21T21:41:29.983 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5382000000000001e-05 2024-09-21T21:41:29.983 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.983 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.983 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:29.983 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830267+0000", 2024-09-21T21:41:29.983 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5148999999999996e-05 2024-09-21T21:41:29.983 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:29.983 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:29.983 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:29.983 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.984 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.984 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4952.0:3551 3.10 3:086eb8a4:::rbd_data.1121a3eabc61.0000000000000000:17 [read 2619392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:29.984 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:29.797183+0000", 2024-09-21T21:41:29.985 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.13673628700000001, 2024-09-21T21:41:29.985 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033198722, 2024-09-21T21:41:29.985 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:29.985 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:29.985 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:29.985 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:29.985 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4952", 2024-09-21T21:41:29.985 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:29.985 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3551 2024-09-21T21:41:29.985 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.985 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:29.986 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.986 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:29.986 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797183+0000", 2024-09-21T21:41:29.986 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:29.986 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.986 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.986 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:29.986 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797183+0000", 2024-09-21T21:41:29.986 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:29.986 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.986 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.986 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:29.986 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797183+0000", 2024-09-21T21:41:29.987 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8000000000000001e-07 2024-09-21T21:41:29.987 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.987 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.987 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:29.987 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797185+0000", 2024-09-21T21:41:29.987 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0940000000000002e-06 2024-09-21T21:41:29.987 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.987 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.987 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:29.987 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797191+0000", 2024-09-21T21:41:29.987 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8300000000000001e-06 2024-09-21T21:41:29.987 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.987 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.987 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:29.988 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797196+0000", 2024-09-21T21:41:29.988 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4530000000000004e-06 2024-09-21T21:41:29.988 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.988 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.988 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:29.988 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830295+0000", 2024-09-21T21:41:29.988 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033099669999999998 2024-09-21T21:41:29.988 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.988 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.988 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:29.988 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830332+0000", 2024-09-21T21:41:29.988 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7156999999999997e-05 2024-09-21T21:41:29.988 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.988 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.988 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:29.989 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830381+0000", 2024-09-21T21:41:29.989 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9138000000000003e-05 2024-09-21T21:41:29.989 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:29.989 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:29.989 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:29.989 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.989 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.989 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4952.0:3553 3.10 3:086eb8a4:::rbd_data.1121a3eabc61.0000000000000000:17 [read 2627584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:29.989 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:29.797201+0000", 2024-09-21T21:41:29.989 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.136718067, 2024-09-21T21:41:29.989 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033313262000000003, 2024-09-21T21:41:29.989 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:29.990 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:29.990 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:29.990 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:29.990 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4952", 2024-09-21T21:41:29.990 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:29.990 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3553 2024-09-21T21:41:29.990 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.990 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:29.990 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.990 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:29.990 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797201+0000", 2024-09-21T21:41:29.990 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:29.991 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.991 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.991 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:29.991 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797201+0000", 2024-09-21T21:41:29.991 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:29.991 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.991 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.991 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:29.991 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797201+0000", 2024-09-21T21:41:29.991 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6600000000000002e-07 2024-09-21T21:41:29.991 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.992 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.992 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:29.992 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797203+0000", 2024-09-21T21:41:29.992 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.8810000000000001e-06 2024-09-21T21:41:29.992 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.992 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.992 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:29.992 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797209+0000", 2024-09-21T21:41:29.992 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.037e-06 2024-09-21T21:41:29.992 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.992 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.993 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:29.993 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797214+0000", 2024-09-21T21:41:29.993 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4920000000000004e-06 2024-09-21T21:41:29.993 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.993 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.993 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:29.993 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830408+0000", 2024-09-21T21:41:29.993 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033194199000000001 2024-09-21T21:41:29.993 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.993 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.993 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:29.993 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830455+0000", 2024-09-21T21:41:29.994 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6547e-05 2024-09-21T21:41:29.994 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.994 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.994 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:29.994 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830514+0000", 2024-09-21T21:41:29.994 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9740000000000001e-05 2024-09-21T21:41:29.994 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:29.994 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:29.994 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:29.994 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.994 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.995 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4952.0:3554 3.10 3:086eb8a4:::rbd_data.1121a3eabc61.0000000000000000:17 [read 2641920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:29.995 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:29.797220+0000", 2024-09-21T21:41:29.995 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.136698773, 2024-09-21T21:41:29.995 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033396381000000003, 2024-09-21T21:41:29.995 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:29.995 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:29.995 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:29.995 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:29.995 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4952", 2024-09-21T21:41:29.995 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:29.995 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3554 2024-09-21T21:41:29.996 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.996 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:29.996 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.996 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:29.996 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797220+0000", 2024-09-21T21:41:29.996 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:29.996 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.996 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.996 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:29.996 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797220+0000", 2024-09-21T21:41:29.996 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:29.996 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.996 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.997 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:29.997 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797221+0000", 2024-09-21T21:41:29.997 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4099999999999999e-07 2024-09-21T21:41:29.997 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.997 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.997 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:29.997 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797223+0000", 2024-09-21T21:41:29.997 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.95e-06 2024-09-21T21:41:29.997 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.997 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.997 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:29.997 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797228+0000", 2024-09-21T21:41:29.997 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6919999999999999e-06 2024-09-21T21:41:29.998 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.998 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.998 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:29.998 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797238+0000", 2024-09-21T21:41:29.998 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6509999999999996e-06 2024-09-21T21:41:29.998 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.998 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.998 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:29.998 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830532+0000", 2024-09-21T21:41:29.998 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033294473999999998 2024-09-21T21:41:29.998 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.998 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.998 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:29.999 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830569+0000", 2024-09-21T21:41:29.999 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6723000000000001e-05 2024-09-21T21:41:29.999 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.999 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.999 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:29.999 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830617+0000", 2024-09-21T21:41:29.999 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7450000000000001e-05 2024-09-21T21:41:29.999 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:29.999 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:29.999 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:29.999 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:29.999 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:29.999 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4952.0:3555 3.10 3:086eb8a4:::rbd_data.1121a3eabc61.0000000000000000:17 [read 2656256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:29.999 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:29.797244+0000", 2024-09-21T21:41:30.000 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.13667490099999999, 2024-09-21T21:41:30.000 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033483065999999999, 2024-09-21T21:41:30.000 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:30.000 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:30.000 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:30.000 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:30.000 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4952", 2024-09-21T21:41:30.000 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:30.000 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3555 2024-09-21T21:41:30.000 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:30.000 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:30.000 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:30.000 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:30.000 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797244+0000", 2024-09-21T21:41:30.001 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:30.001 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:30.001 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:30.001 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:30.001 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797244+0000", 2024-09-21T21:41:30.001 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:30.001 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:30.001 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:30.001 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:30.001 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797245+0000", 2024-09-21T21:41:30.001 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9299999999999998e-07 2024-09-21T21:41:30.001 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:30.001 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:30.001 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:30.002 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797247+0000", 2024-09-21T21:41:30.002 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0370000000000001e-06 2024-09-21T21:41:30.002 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:30.002 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:30.002 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:30.002 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797253+0000", 2024-09-21T21:41:30.002 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8189999999999997e-06 2024-09-21T21:41:30.002 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:30.002 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:30.002 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:30.002 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797257+0000", 2024-09-21T21:41:30.002 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.33e-06 2024-09-21T21:41:30.002 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:30.003 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:30.003 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:30.003 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830642+0000", 2024-09-21T21:41:30.003 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033385629999999999 2024-09-21T21:41:30.003 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:30.003 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:30.003 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:30.003 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830679+0000", 2024-09-21T21:41:30.003 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6100999999999999e-05 2024-09-21T21:41:30.003 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:30.004 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:30.004 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:30.004 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830727+0000", 2024-09-21T21:41:30.004 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8656000000000003e-05 2024-09-21T21:41:30.004 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:30.004 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:30.004 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:30.004 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:30.004 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:30.004 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4952.0:3557 3.10 3:086eb8a4:::rbd_data.1121a3eabc61.0000000000000000:17 [read 2670592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:30.004 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:29.797263+0000", 2024-09-21T21:41:30.004 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.13665603900000001, 2024-09-21T21:41:30.004 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033580213999999997, 2024-09-21T21:41:30.004 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:30.005 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:30.005 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:30.005 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:30.005 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4952", 2024-09-21T21:41:30.005 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:30.005 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3557 2024-09-21T21:41:30.005 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:30.005 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:30.005 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:30.005 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:30.005 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797263+0000", 2024-09-21T21:41:30.005 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:30.006 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:30.006 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:30.006 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:30.006 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797263+0000", 2024-09-21T21:41:30.006 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:30.006 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:30.007 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:30.007 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:30.007 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797263+0000", 2024-09-21T21:41:30.007 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4799999999999999e-07 2024-09-21T21:41:30.007 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:30.007 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:30.007 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:30.008 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797265+0000", 2024-09-21T21:41:30.008 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.029e-06 2024-09-21T21:41:30.008 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:30.008 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:30.008 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:30.008 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797271+0000", 2024-09-21T21:41:30.008 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0000000000000002e-06 2024-09-21T21:41:30.009 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:30.009 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:30.009 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:30.009 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797276+0000", 2024-09-21T21:41:30.009 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5310000000000003e-06 2024-09-21T21:41:30.009 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:30.010 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:30.010 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:30.010 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830756+0000", 2024-09-21T21:41:30.010 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033479636 2024-09-21T21:41:30.010 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:30.010 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:30.010 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:30.010 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830792+0000", 2024-09-21T21:41:30.011 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6584000000000001e-05 2024-09-21T21:41:30.011 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:30.011 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:30.011 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:30.011 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830843+0000", 2024-09-21T21:41:30.011 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0986000000000003e-05 2024-09-21T21:41:30.011 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:30.011 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:30.011 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:30.011 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:30.012 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:30.012 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4952.0:3558 3.10 3:086eb8a4:::rbd_data.1121a3eabc61.0000000000000000:17 [read 2684928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:30.012 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:29.797281+0000", 2024-09-21T21:41:30.012 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.13663774300000001, 2024-09-21T21:41:30.012 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033692174999999998, 2024-09-21T21:41:30.012 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:30.012 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:30.013 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:30.013 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:30.013 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4952", 2024-09-21T21:41:30.013 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:30.013 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3558 2024-09-21T21:41:30.013 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:30.013 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:30.014 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:30.014 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:30.014 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797281+0000", 2024-09-21T21:41:30.014 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:30.014 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:30.014 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:30.014 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:30.014 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797281+0000", 2024-09-21T21:41:30.015 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:30.015 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:30.015 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:30.015 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:30.015 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797282+0000", 2024-09-21T21:41:30.015 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.41e-07 2024-09-21T21:41:30.015 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:30.015 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:30.016 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:30.016 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797283+0000", 2024-09-21T21:41:30.016 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.8390000000000001e-06 2024-09-21T21:41:30.016 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:30.016 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:30.016 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:30.016 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797289+0000", 2024-09-21T21:41:30.016 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3839999999999996e-06 2024-09-21T21:41:30.017 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:30.017 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:30.017 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:30.017 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797294+0000", 2024-09-21T21:41:30.017 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1900000000000003e-06 2024-09-21T21:41:30.017 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:30.017 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:30.018 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:30.018 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830874+0000", 2024-09-21T21:41:30.018 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033580384999999997 2024-09-21T21:41:30.018 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:30.018 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:30.018 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:30.018 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830912+0000", 2024-09-21T21:41:30.019 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7367000000000003e-05 2024-09-21T21:41:30.019 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:30.019 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:30.019 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:30.019 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830973+0000", 2024-09-21T21:41:30.019 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1668999999999995e-05 2024-09-21T21:41:30.019 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:30.019 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:30.020 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:30.020 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:30.020 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:30.020 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4952.0:3560 3.10 3:086eb8a4:::rbd_data.1121a3eabc61.0000000000000000:17 [read 2699264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:30.020 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:29.797300+0000", 2024-09-21T21:41:30.020 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.13661905999999999, 2024-09-21T21:41:30.020 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033777109, 2024-09-21T21:41:30.020 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:30.020 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:30.020 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:30.020 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:30.020 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4952", 2024-09-21T21:41:30.020 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:30.021 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3560 2024-09-21T21:41:30.021 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:30.021 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:30.021 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:30.021 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:30.021 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797300+0000", 2024-09-21T21:41:30.021 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:30.021 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:30.021 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:30.021 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:30.021 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797300+0000", 2024-09-21T21:41:30.021 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:30.021 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:30.021 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:30.021 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:30.022 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797300+0000", 2024-09-21T21:41:30.022 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.2399999999999999e-07 2024-09-21T21:41:30.022 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:30.022 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:30.022 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:30.022 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797302+0000", 2024-09-21T21:41:30.022 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0260000000000002e-06 2024-09-21T21:41:30.022 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:30.022 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:30.022 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:30.022 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797308+0000", 2024-09-21T21:41:30.022 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8839999999999999e-06 2024-09-21T21:41:30.022 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:30.022 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:30.023 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:30.023 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797313+0000", 2024-09-21T21:41:30.023 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3959999999999999e-06 2024-09-21T21:41:30.023 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:30.023 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:30.023 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:30.023 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830997+0000", 2024-09-21T21:41:30.023 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033684506000000003 2024-09-21T21:41:30.023 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:30.023 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:30.023 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:30.023 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.831033+0000", 2024-09-21T21:41:30.023 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5964000000000003e-05 2024-09-21T21:41:30.023 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:30.024 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:30.024 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:30.024 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.831077+0000", 2024-09-21T21:41:30.024 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4008999999999998e-05 2024-09-21T21:41:30.024 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:30.024 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:30.024 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:30.024 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:30.024 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:30.024 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4952.0:3562 3.10 3:086eb8a4:::rbd_data.1121a3eabc61.0000000000000000:17 [read 2713600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:30.024 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:29.797318+0000", 2024-09-21T21:41:30.025 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.136601203, 2024-09-21T21:41:30.025 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033877712999999997, 2024-09-21T21:41:30.025 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:30.025 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:30.025 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:30.025 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:30.025 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4952", 2024-09-21T21:41:30.025 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:30.025 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3562 2024-09-21T21:41:30.025 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:30.025 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:30.025 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:30.025 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:30.025 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797318+0000", 2024-09-21T21:41:30.026 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:30.026 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:30.026 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:30.026 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:30.026 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797318+0000", 2024-09-21T21:41:30.026 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:30.026 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:30.026 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:30.027 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:30.027 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797318+0000", 2024-09-21T21:41:30.027 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.2000000000000001e-07 2024-09-21T21:41:30.027 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:30.027 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:30.027 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:30.027 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797320+0000", 2024-09-21T21:41:30.027 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9700000000000002e-06 2024-09-21T21:41:30.027 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:30.027 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:30.027 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:30.027 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797326+0000", 2024-09-21T21:41:30.028 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.212e-06 2024-09-21T21:41:30.028 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:30.028 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:30.028 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:30.028 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797331+0000", 2024-09-21T21:41:30.028 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9409999999999998e-06 2024-09-21T21:41:30.028 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:30.028 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:30.028 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:30.028 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.831104+0000", 2024-09-21T21:41:30.028 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033772976000000003 2024-09-21T21:41:30.028 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:30.028 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:30.028 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:30.029 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.831144+0000", 2024-09-21T21:41:30.029 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.0209000000000001e-05 2024-09-21T21:41:30.029 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:30.029 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:30.029 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:30.029 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.831196+0000", 2024-09-21T21:41:30.029 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1085000000000003e-05 2024-09-21T21:41:30.029 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:30.029 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:30.029 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:30.029 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:30.029 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:30.029 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4952.0:3563 3.10 3:086eb8a4:::rbd_data.1121a3eabc61.0000000000000000:17 [read 2727936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:30.029 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:29.797350+0000", 2024-09-21T21:41:30.030 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.13656895099999999, 2024-09-21T21:41:30.030 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033952501000000003, 2024-09-21T21:41:30.030 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:30.030 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:30.030 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:30.030 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:30.030 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4952", 2024-09-21T21:41:30.030 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:30.030 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3563 2024-09-21T21:41:30.030 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:30.030 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:30.030 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:30.030 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:30.030 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797350+0000", 2024-09-21T21:41:30.031 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:30.031 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:30.031 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:30.031 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:30.031 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797350+0000", 2024-09-21T21:41:30.031 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:30.031 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:30.031 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:30.031 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:30.031 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797351+0000", 2024-09-21T21:41:30.031 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5499999999999998e-07 2024-09-21T21:41:30.031 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:30.031 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:30.031 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:30.032 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797353+0000", 2024-09-21T21:41:30.032 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2299999999999998e-06 2024-09-21T21:41:30.032 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:30.032 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:30.032 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:30.032 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797359+0000", 2024-09-21T21:41:30.032 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3330000000000002e-06 2024-09-21T21:41:30.032 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:30.032 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:30.032 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:30.032 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797364+0000", 2024-09-21T21:41:30.032 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4410000000000003e-06 2024-09-21T21:41:30.032 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:30.032 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:30.032 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:30.033 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.831223+0000", 2024-09-21T21:41:30.033 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033859002999999999 2024-09-21T21:41:30.033 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:30.033 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:30.033 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:30.033 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.831258+0000", 2024-09-21T21:41:30.033 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5417000000000003e-05 2024-09-21T21:41:30.033 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:30.033 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:30.033 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:30.033 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.831303+0000", 2024-09-21T21:41:30.033 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4521999999999998e-05 2024-09-21T21:41:30.033 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:30.033 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:30.034 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:30.034 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:30.034 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:30.034 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4952.0:3564 3.10 3:086eb8a4:::rbd_data.1121a3eabc61.0000000000000000:17 [read 2742272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:30.034 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:29.797370+0000", 2024-09-21T21:41:30.034 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.13654910000000001, 2024-09-21T21:41:30.034 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.034051217000000002, 2024-09-21T21:41:30.034 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:30.034 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:30.034 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:30.034 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:30.034 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4952", 2024-09-21T21:41:30.034 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:30.034 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3564 2024-09-21T21:41:30.035 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:30.035 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:30.035 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:30.035 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:30.035 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797370+0000", 2024-09-21T21:41:30.035 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:30.035 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:30.035 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:30.035 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:30.035 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797370+0000", 2024-09-21T21:41:30.035 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:30.035 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:30.036 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:30.036 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:30.036 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797370+0000", 2024-09-21T21:41:30.036 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5499999999999998e-07 2024-09-21T21:41:30.036 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:30.036 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:30.036 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:30.036 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797372+0000", 2024-09-21T21:41:30.036 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9290000000000001e-06 2024-09-21T21:41:30.036 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:30.036 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:30.037 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:30.037 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797378+0000", 2024-09-21T21:41:30.037 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9599999999999997e-06 2024-09-21T21:41:30.037 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:30.037 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:30.037 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:30.037 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797388+0000", 2024-09-21T21:41:30.037 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.5040000000000008e-06 2024-09-21T21:41:30.038 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:30.038 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:30.038 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:30.038 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.831335+0000", 2024-09-21T21:41:30.038 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033947374000000002 2024-09-21T21:41:30.038 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:30.038 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:30.039 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:30.039 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.831371+0000", 2024-09-21T21:41:30.039 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5868999999999998e-05 2024-09-21T21:41:30.039 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:30.039 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:30.039 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:30.039 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.831421+0000", 2024-09-21T21:41:30.039 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0126000000000001e-05 2024-09-21T21:41:30.039 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:30.039 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:30.039 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:30.039 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:30.039 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:30.039 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:30.041 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:30.038+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:30.041 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T21:41:30.103 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0: {} 2024-09-21T21:41:30.103 INFO:teuthology.orchestra.run.smithi081.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T21:41:30.133 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1: {} 2024-09-21T21:41:30.133 INFO:teuthology.orchestra.run.smithi081.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T21:41:30.141 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:30.140+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:30.168 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2: {} 2024-09-21T21:41:30.168 INFO:teuthology.orchestra.run.smithi081.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T21:41:30.194 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3: {} 2024-09-21T21:41:30.194 INFO:teuthology.orchestra.run.smithi081.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T21:41:30.219 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4: {} 2024-09-21T21:41:30.219 INFO:teuthology.orchestra.run.smithi081.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T21:41:30.242 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:30.241+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:30.243 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5: {} 2024-09-21T21:41:30.243 INFO:teuthology.orchestra.run.smithi081.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T21:41:30.244 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:30.244 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:30.259 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T21:41:30.343 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:30.341+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:30.420 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:30.421 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:30.430 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T21:41:30.473 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:30.472+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:30.535 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:30.535 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:30.543 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T21:41:30.545 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:30.543+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:30.633 INFO:teuthology.orchestra.run.smithi164.stdout:checking clone #20, image image_client.0-clone20 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent21 2024-09-21T21:41:30.645 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:30.644+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:30.684 INFO:teuthology.orchestra.run.smithi081.stderr:noscrub is unset 2024-09-21T21:41:30.696 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:30.696 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:30.705 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T21:41:30.748 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:30.746+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:30.847 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:30.845+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:30.853 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:30.853 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:30.862 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T21:41:30.897 INFO:teuthology.orchestra.run.smithi164.stdout:3128 write 0x9bda600 thru 0x9bde825 (0x4226 bytes) 2024-09-21T21:41:30.898 INFO:teuthology.orchestra.run.smithi164.stdout:3132 read 0x1778200 thru 0x1778cab (0xaac bytes) 2024-09-21T21:41:30.898 INFO:teuthology.orchestra.run.smithi164.stdout:3133 read 0x6b4f200 thru 0x6b5b7ed (0xc5ee bytes) 2024-09-21T21:41:30.903 INFO:teuthology.orchestra.run.smithi164.stdout:3135 trunc from 0x9bdea00 to 0x5ed6400 2024-09-21T21:41:30.925 INFO:teuthology.orchestra.run.smithi164.stdout:3138 read 0x3f8bc00 thru 0x3f94dad (0x91ae bytes) 2024-09-21T21:41:30.930 INFO:teuthology.orchestra.run.smithi164.stdout:3148 trunc from 0x5ed6400 to 0x1936400 2024-09-21T21:41:30.948 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:30.947+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:30.980 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:30.980 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:30.990 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T21:41:30.990 INFO:teuthology.orchestra.run.smithi164.stdout:3149 trunc from 0x1936400 to 0xb340400 2024-09-21T21:41:31.005 INFO:teuthology.orchestra.run.smithi164.stdout:3151 read 0x4054600 thru 0x405bdcb (0x77cc bytes) 2024-09-21T21:41:31.005 INFO:teuthology.orchestra.run.smithi164.stdout:3152 read 0x4704c00 thru 0x4709d14 (0x5115 bytes) 2024-09-21T21:41:31.005 INFO:teuthology.orchestra.run.smithi164.stdout:3153 punch from 0x9ffc400 to 0xa004600, (0x8200 bytes) 2024-09-21T21:41:31.005 INFO:teuthology.orchestra.run.smithi164.stdout:3154 write 0x2631400 thru 0x2638ca9 (0x78aa bytes) 2024-09-21T21:41:31.005 INFO:teuthology.orchestra.run.smithi164.stdout:3157 write 0x3770000 thru 0x3779da2 (0x9da3 bytes) 2024-09-21T21:41:31.005 INFO:teuthology.orchestra.run.smithi164.stdout:3158 write 0x3a65e00 thru 0x3a713fd (0xb5fe bytes) 2024-09-21T21:41:31.005 INFO:teuthology.orchestra.run.smithi164.stdout:3159 read 0x11e2400 thru 0x11ecdd8 (0xa9d9 bytes) 2024-09-21T21:41:31.008 INFO:teuthology.orchestra.run.smithi164.stdout:3164 write 0x8a79200 thru 0x8a82f21 (0x9d22 bytes) 2024-09-21T21:41:31.029 INFO:teuthology.orchestra.run.smithi164.stdout:3165 write 0xe71cc00 thru 0xe720112 (0x3513 bytes) 2024-09-21T21:41:31.035 INFO:teuthology.orchestra.run.smithi164.stdout:3167 read 0xe394400 thru 0xe39bdf8 (0x79f9 bytes) 2024-09-21T21:41:31.035 INFO:teuthology.orchestra.run.smithi164.stdout:3169 write 0xcc5c000 thru 0xcc609cf (0x49d0 bytes) 2024-09-21T21:41:31.035 INFO:teuthology.orchestra.run.smithi164.stdout:3172 read 0x3064c00 thru 0x30678d2 (0x2cd3 bytes) 2024-09-21T21:41:31.035 INFO:teuthology.orchestra.run.smithi164.stdout:3173 write 0x2994600 thru 0x299a169 (0x5b6a bytes) 2024-09-21T21:41:31.046 INFO:teuthology.orchestra.run.smithi164.stdout:3174 trunc from 0xe720200 to 0x5142a00 2024-09-21T21:41:31.049 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:31.048+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:31.070 INFO:teuthology.orchestra.run.smithi164.stdout:3175 write 0x1233000 thru 0x1233069 (0x6a bytes) 2024-09-21T21:41:31.101 INFO:teuthology.orchestra.run.smithi164.stdout:3177 write 0x7b75c00 thru 0x7b7632b (0x72c bytes) 2024-09-21T21:41:31.105 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:31.106 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:31.115 INFO:teuthology.orchestra.run.smithi164.stdout:3180 write 0xa2a9000 thru 0xa2b5821 (0xc822 bytes) 2024-09-21T21:41:31.116 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T21:41:31.128 INFO:teuthology.orchestra.run.smithi164.stdout:3181 trunc from 0xa2b5a00 to 0x280e00 2024-09-21T21:41:31.160 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:31.159+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:31.229 INFO:teuthology.orchestra.run.smithi164.stdout:3182 write 0x5bba200 thru 0x5bc6e24 (0xcc25 bytes) 2024-09-21T21:41:31.230 INFO:teuthology.orchestra.run.smithi164.stdout:3183 write 0x474000 thru 0x47ac6a (0x6c6b bytes) 2024-09-21T21:41:31.236 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:31.236 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:31.246 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T21:41:31.247 INFO:teuthology.orchestra.run.smithi164.stdout:3184 write 0x8502e00 thru 0x8509e2c (0x702d bytes) 2024-09-21T21:41:31.248 INFO:teuthology.orchestra.run.smithi164.stdout:3185 write 0x2731000 thru 0x2736aa0 (0x5aa1 bytes) 2024-09-21T21:41:31.248 INFO:teuthology.orchestra.run.smithi164.stdout:3186 read 0x66a5600 thru 0x66a981c (0x421d bytes) 2024-09-21T21:41:31.248 INFO:teuthology.orchestra.run.smithi164.stdout:3187 punch from 0x1657800 to 0x1665600, (0xde00 bytes) 2024-09-21T21:41:31.255 INFO:teuthology.orchestra.run.smithi164.stdout:3188 trunc from 0x850a000 to 0xcfbde00 2024-09-21T21:41:31.264 INFO:teuthology.orchestra.run.smithi164.stdout:3189 write 0x33ee200 thru 0x33fc98f (0xe790 bytes) 2024-09-21T21:41:31.265 INFO:teuthology.orchestra.run.smithi164.stdout:3190 read 0xbbe8c00 thru 0xbbf6d03 (0xe104 bytes) 2024-09-21T21:41:31.265 INFO:teuthology.orchestra.run.smithi164.stdout:3191 clone 23 order 19 su 131072 sc 2 2024-09-21T21:41:31.288 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:31.286+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:31.351 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:31.350+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:31.390 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:31.390 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:31.399 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T21:41:31.452 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:31.451+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:31.501 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:31.501 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:31.510 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T21:41:31.553 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:31.552+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:31.612 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:31.613 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:31.621 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T21:41:31.663 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:31.661+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:31.754 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:31.753+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:31.764 INFO:teuthology.orchestra.run.smithi164.stdout:truncating image image_client.0-clone22 from 0xcfbde00 (overlap 0x280e00) to 0x40000 2024-09-21T21:41:31.772 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:31.772 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:31.782 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T21:41:31.866 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:31.865+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:31.927 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:31.927 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:31.935 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T21:41:31.976 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:31.975+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:32.056 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:32.055+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:32.082 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:32.082 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:32.091 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T21:41:32.097 INFO:teuthology.orchestra.run.smithi164.stdout:checking clone #21, image image_client.0-clone21 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent22 2024-09-21T21:41:32.157 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:32.156+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:32.172 INFO:teuthology.orchestra.run.smithi164.stdout:3192 read 0x2521200 thru 0x2530474 (0xf275 bytes) 2024-09-21T21:41:32.172 INFO:teuthology.orchestra.run.smithi164.stdout:3199 punch from 0x7937e00 to 0x793ae00, (0x3000 bytes) 2024-09-21T21:41:32.179 INFO:teuthology.orchestra.run.smithi164.stdout:3200 read 0xcf41800 thru 0xcf4d29e (0xba9f bytes) 2024-09-21T21:41:32.192 INFO:teuthology.orchestra.run.smithi164.stdout:3202 trunc from 0xcfbde00 to 0x2650000 2024-09-21T21:41:32.201 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:32.201 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:32.211 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T21:41:32.212 INFO:teuthology.orchestra.run.smithi164.stdout:3203 read 0x8ac400 thru 0x8b7117 (0xad18 bytes) 2024-09-21T21:41:32.213 INFO:teuthology.orchestra.run.smithi164.stdout:3204 read 0x233f200 thru 0x233f72b (0x52c bytes) 2024-09-21T21:41:32.221 INFO:teuthology.orchestra.run.smithi164.stdout:3205 trunc from 0x2650000 to 0x98aba00 2024-09-21T21:41:32.231 INFO:teuthology.orchestra.run.smithi164.stdout:3208 write 0x36bbc00 thru 0x36c637f (0xa780 bytes) 2024-09-21T21:41:32.232 INFO:teuthology.orchestra.run.smithi164.stdout:3210 read 0x379dc00 thru 0x37aa379 (0xc77a bytes) 2024-09-21T21:41:32.245 INFO:teuthology.orchestra.run.smithi164.stdout:3211 write 0xa8dec00 thru 0xa8e4738 (0x5b39 bytes) 2024-09-21T21:41:32.258 INFO:teuthology.orchestra.run.smithi164.stdout:3212 trunc from 0xa8e4800 to 0x9ade00 2024-09-21T21:41:32.258 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:32.257+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:32.266 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T21:41:32.279 INFO:teuthology.orchestra.run.smithi164.stdout:3213 read 0x35c00 thru 0x36ea7 (0x12a8 bytes) 2024-09-21T21:41:32.305 INFO:teuthology.orchestra.run.smithi164.stdout:3215 write 0x8f0e000 thru 0x8f12746 (0x4747 bytes) 2024-09-21T21:41:32.312 INFO:teuthology.orchestra.run.smithi164.stdout:3216 trunc from 0x8f12800 to 0x3d78a00 2024-09-21T21:41:32.320 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:32.320 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:32.329 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T21:41:32.333 INFO:teuthology.orchestra.run.smithi164.stdout:3222 read 0xc7b000 thru 0xc857a1 (0xa7a2 bytes) 2024-09-21T21:41:32.360 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:32.357+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:32.361 INFO:teuthology.orchestra.run.smithi164.stdout:3224 write 0xec29e00 thru 0xec36f23 (0xd124 bytes) 2024-09-21T21:41:32.362 INFO:teuthology.orchestra.run.smithi164.stdout:3225 read 0xc7ea800 thru 0xc7f14e8 (0x6ce9 bytes) 2024-09-21T21:41:32.362 INFO:teuthology.orchestra.run.smithi164.stdout:3228 read 0x60d0400 thru 0x60dd071 (0xcc72 bytes) 2024-09-21T21:41:32.362 INFO:teuthology.orchestra.run.smithi164.stdout:3229 read 0x7807800 thru 0x78080c9 (0x8ca bytes) 2024-09-21T21:41:32.362 INFO:teuthology.orchestra.run.smithi164.stdout:3230 read 0xa24fa00 thru 0xa25e18e (0xe78f bytes) 2024-09-21T21:41:32.362 INFO:teuthology.orchestra.run.smithi164.stdout:3231 read 0xb18de00 thru 0xb18fd6e (0x1f6f bytes) 2024-09-21T21:41:32.362 INFO:teuthology.orchestra.run.smithi164.stdout:3232 write 0xa838e00 thru 0xa83d5c6 (0x47c7 bytes) 2024-09-21T21:41:32.362 INFO:teuthology.orchestra.run.smithi164.stdout:3233 write 0xd101000 thru 0xd10f812 (0xe813 bytes) 2024-09-21T21:41:32.362 INFO:teuthology.orchestra.run.smithi164.stdout:3234 write 0xdb86600 thru 0xdb90adf (0xa4e0 bytes) 2024-09-21T21:41:32.362 INFO:teuthology.orchestra.run.smithi164.stdout:3235 read 0xa508000 thru 0xa50c642 (0x4643 bytes) 2024-09-21T21:41:32.362 INFO:teuthology.orchestra.run.smithi164.stdout:3238 punch from 0x4912c00 to 0x4921200, (0xe600 bytes) 2024-09-21T21:41:32.363 INFO:teuthology.orchestra.run.smithi164.stdout:3239 read 0xd2a5000 thru 0xd2ad2e4 (0x82e5 bytes) 2024-09-21T21:41:32.363 INFO:teuthology.orchestra.run.smithi164.stdout:3240 write 0x812ce00 thru 0x812fdb3 (0x2fb4 bytes) 2024-09-21T21:41:32.368 INFO:teuthology.orchestra.run.smithi164.stdout:3243 trunc from 0xec37000 to 0xa2d5400 2024-09-21T21:41:32.390 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0: {} 2024-09-21T21:41:32.390 INFO:teuthology.orchestra.run.smithi081.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T21:41:32.398 INFO:teuthology.orchestra.run.smithi164.stdout:3249 write 0x4d6ae00 thru 0x4d730ce (0x82cf bytes) 2024-09-21T21:41:32.398 INFO:teuthology.orchestra.run.smithi164.stdout:3251 write 0x1c96800 thru 0x1ca1c0e (0xb40f bytes) 2024-09-21T21:41:32.398 INFO:teuthology.orchestra.run.smithi164.stdout:3252 write 0x249c800 thru 0x24a5045 (0x8846 bytes) 2024-09-21T21:41:32.398 INFO:teuthology.orchestra.run.smithi164.stdout:3253 trunc from 0xa2d5400 to 0xa715e00 2024-09-21T21:41:32.408 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1: {} 2024-09-21T21:41:32.408 INFO:teuthology.orchestra.run.smithi081.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T21:41:32.419 INFO:teuthology.orchestra.run.smithi164.stdout:3257 read 0x3c14200 thru 0x3c18d6b (0x4b6c bytes) 2024-09-21T21:41:32.420 INFO:teuthology.orchestra.run.smithi164.stdout:3259 read 0x5fca200 thru 0x5fcbafa (0x18fb bytes) 2024-09-21T21:41:32.420 INFO:teuthology.orchestra.run.smithi164.stdout:3260 read 0x92b8e00 thru 0x92c8559 (0xf75a bytes) 2024-09-21T21:41:32.420 INFO:teuthology.orchestra.run.smithi164.stdout:3261 write 0x27b6200 thru 0x27c41fb (0xdffc bytes) 2024-09-21T21:41:32.420 INFO:teuthology.orchestra.run.smithi164.stdout:3262 write 0x9931600 thru 0x993a91e (0x931f bytes) 2024-09-21T21:41:32.420 INFO:teuthology.orchestra.run.smithi164.stdout:3263 write 0x2cba00 thru 0x2d1141 (0x5742 bytes) 2024-09-21T21:41:32.420 INFO:teuthology.orchestra.run.smithi164.stdout:3265 write 0x5fe6800 thru 0x5fec97c (0x617d bytes) 2024-09-21T21:41:32.420 INFO:teuthology.orchestra.run.smithi164.stdout:3267 read 0x8d90600 thru 0x8d9f476 (0xee77 bytes) 2024-09-21T21:41:32.424 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2: {} 2024-09-21T21:41:32.424 INFO:teuthology.orchestra.run.smithi081.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T21:41:32.429 INFO:teuthology.orchestra.run.smithi164.stdout:3269 trunc from 0xa715e00 to 0x2015200 2024-09-21T21:41:32.439 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:32.439 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:41:32.439 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:41:32.440 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:41:32.440 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:32.441 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3: {} 2024-09-21T21:41:32.441 INFO:teuthology.orchestra.run.smithi081.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T21:41:32.449 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T21:41:32.455 INFO:teuthology.orchestra.run.smithi164.stdout:3270 read 0xbd5400 thru 0xbe43e8 (0xefe9 bytes) 2024-09-21T21:41:32.459 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4: {} 2024-09-21T21:41:32.459 INFO:teuthology.orchestra.run.smithi081.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T21:41:32.459 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:32.458+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:32.460 INFO:teuthology.orchestra.run.smithi164.stdout:3272 read 0x1149800 thru 0x1149f21 (0x722 bytes) 2024-09-21T21:41:32.474 INFO:teuthology.orchestra.run.smithi164.stdout:3274 write 0x4621200 thru 0x46229a1 (0x17a2 bytes) 2024-09-21T21:41:32.477 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5: {} 2024-09-21T21:41:32.477 INFO:teuthology.orchestra.run.smithi081.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T21:41:32.483 INFO:teuthology.orchestra.run.smithi164.stdout:3275 trunc from 0x4622a00 to 0xc4daa00 2024-09-21T21:41:32.495 INFO:teuthology.orchestra.run.smithi164.stdout:3276 read 0x9b52000 thru 0x9b5ad8d (0x8d8e bytes) 2024-09-21T21:41:32.496 INFO:teuthology.orchestra.run.smithi164.stdout:3277 write 0x6eb3c00 thru 0x6ebabce (0x6fcf bytes) 2024-09-21T21:41:32.496 INFO:teuthology.orchestra.run.smithi164.stdout:3279 write 0x4138e00 thru 0x4145172 (0xc373 bytes) 2024-09-21T21:41:32.502 INFO:teuthology.orchestra.run.smithi164.stdout:3280 trunc from 0xc4daa00 to 0x6ecb600 2024-09-21T21:41:32.524 INFO:teuthology.orchestra.run.smithi164.stdout:3281 trunc from 0x6ecb600 to 0xf92e00 2024-09-21T21:41:32.538 INFO:teuthology.orchestra.run.smithi164.stdout:3282 write 0x4c6e00 thru 0x4d5b59 (0xed5a bytes) 2024-09-21T21:41:32.553 INFO:teuthology.orchestra.run.smithi164.stdout:3283 trunc from 0xf92e00 to 0xc94a800 2024-09-21T21:41:32.558 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:32.558 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 20, 2024-09-21T21:41:32.558 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 600, 2024-09-21T21:41:32.558 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:41:32.558 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.558 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4952.0:3526 3.10 3:086eb8a4:::rbd_data.1121a3eabc61.0000000000000000:17 [read 2410496~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:32.558 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:29.795290+0000", 2024-09-21T21:41:32.558 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 2.7602123120000002, 2024-09-21T21:41:32.558 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.032825482000000003, 2024-09-21T21:41:32.559 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:32.559 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:32.559 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:32.559 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:32.559 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4952", 2024-09-21T21:41:32.559 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:32.559 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3526 2024-09-21T21:41:32.559 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.559 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:32.559 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.559 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:32.559 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795290+0000", 2024-09-21T21:41:32.560 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:32.560 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.560 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.560 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:32.560 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795290+0000", 2024-09-21T21:41:32.560 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:32.560 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.560 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.560 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:32.560 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795291+0000", 2024-09-21T21:41:32.560 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3499999999999996e-07 2024-09-21T21:41:32.560 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.560 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.561 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:32.561 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795293+0000", 2024-09-21T21:41:32.561 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0779999999999998e-06 2024-09-21T21:41:32.561 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.561 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.561 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:32.561 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795299+0000", 2024-09-21T21:41:32.561 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7270000000000004e-06 2024-09-21T21:41:32.561 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.561 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.561 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:32.562 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795303+0000", 2024-09-21T21:41:32.562 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3579999999999996e-06 2024-09-21T21:41:32.562 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.562 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.562 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:32.562 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.827920+0000", 2024-09-21T21:41:32.562 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.032617118 2024-09-21T21:41:32.562 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.562 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.562 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:32.562 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.828011+0000", 2024-09-21T21:41:32.562 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.0676999999999999e-05 2024-09-21T21:41:32.563 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.563 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.563 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:32.563 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.828116+0000", 2024-09-21T21:41:32.563 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010498899999999999 2024-09-21T21:41:32.563 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:32.563 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:32.563 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:32.563 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.563 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.563 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4952.0:3528 3.10 3:086eb8a4:::rbd_data.1121a3eabc61.0000000000000000:17 [read 2424832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:32.563 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:29.795308+0000", 2024-09-21T21:41:32.564 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 2.7601945900000002, 2024-09-21T21:41:32.564 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033028465999999999, 2024-09-21T21:41:32.564 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:32.564 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:32.564 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:32.564 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:32.564 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4952", 2024-09-21T21:41:32.564 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:32.564 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3528 2024-09-21T21:41:32.564 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.564 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:32.564 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.564 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:32.565 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795308+0000", 2024-09-21T21:41:32.565 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:32.565 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.565 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.565 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:32.565 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795308+0000", 2024-09-21T21:41:32.565 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:32.565 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.565 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.565 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:32.565 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795308+0000", 2024-09-21T21:41:32.565 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.22e-07 2024-09-21T21:41:32.566 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.566 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.566 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:32.566 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795310+0000", 2024-09-21T21:41:32.566 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.7940000000000001e-06 2024-09-21T21:41:32.566 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.566 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.566 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:32.566 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795316+0000", 2024-09-21T21:41:32.566 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9410000000000004e-06 2024-09-21T21:41:32.566 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.566 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.567 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:32.567 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795321+0000", 2024-09-21T21:41:32.567 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4800000000000003e-06 2024-09-21T21:41:32.567 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.567 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.567 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:32.567 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.828149+0000", 2024-09-21T21:41:32.567 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.032828001000000002 2024-09-21T21:41:32.567 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.567 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.567 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:32.567 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.828223+0000", 2024-09-21T21:41:32.567 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.4048000000000007e-05 2024-09-21T21:41:32.568 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.568 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.568 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:32.568 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.828337+0000", 2024-09-21T21:41:32.568 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011388 2024-09-21T21:41:32.568 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:32.568 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:32.568 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:32.568 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.568 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.568 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4952.0:3529 3.10 3:086eb8a4:::rbd_data.1121a3eabc61.0000000000000000:17 [read 2439168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:32.568 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:29.795328+0000", 2024-09-21T21:41:32.569 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 2.7601744940000001, 2024-09-21T21:41:32.569 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033236716999999999, 2024-09-21T21:41:32.569 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:32.569 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:32.569 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:32.569 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:32.569 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4952", 2024-09-21T21:41:32.569 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:32.569 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3529 2024-09-21T21:41:32.569 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.569 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:32.569 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.570 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:32.570 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795328+0000", 2024-09-21T21:41:32.570 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:32.570 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.570 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.570 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:32.570 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795328+0000", 2024-09-21T21:41:32.570 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:32.570 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.570 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.570 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:32.570 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795329+0000", 2024-09-21T21:41:32.570 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.44e-07 2024-09-21T21:41:32.571 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.571 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.571 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:32.571 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795335+0000", 2024-09-21T21:41:32.571 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7400000000000001e-06 2024-09-21T21:41:32.571 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.571 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.571 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:32.571 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795340+0000", 2024-09-21T21:41:32.571 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9009999999999999e-06 2024-09-21T21:41:32.571 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.571 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.572 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:32.572 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795345+0000", 2024-09-21T21:41:32.572 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4839999999999998e-06 2024-09-21T21:41:32.572 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.572 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.572 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:32.572 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.828382+0000", 2024-09-21T21:41:32.572 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033037113 2024-09-21T21:41:32.572 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.572 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.572 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:32.573 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.828473+0000", 2024-09-21T21:41:32.573 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.1056000000000004e-05 2024-09-21T21:41:32.573 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.573 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.573 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:32.573 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.828565+0000", 2024-09-21T21:41:32.573 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.1879000000000006e-05 2024-09-21T21:41:32.573 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:32.573 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:32.573 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:32.573 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.573 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.574 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4952.0:3531 3.10 3:086eb8a4:::rbd_data.1121a3eabc61.0000000000000000:17 [read 2453504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:32.574 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:29.795353+0000", 2024-09-21T21:41:32.574 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 2.760149765, 2024-09-21T21:41:32.574 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033394973000000001, 2024-09-21T21:41:32.574 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:32.574 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:32.574 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:32.574 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:32.574 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4952", 2024-09-21T21:41:32.574 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:32.574 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3531 2024-09-21T21:41:32.574 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.575 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:32.575 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.575 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:32.575 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795353+0000", 2024-09-21T21:41:32.575 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:32.575 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.575 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.575 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:32.575 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795353+0000", 2024-09-21T21:41:32.575 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:32.575 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.575 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.576 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:32.576 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795353+0000", 2024-09-21T21:41:32.576 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7300000000000001e-07 2024-09-21T21:41:32.576 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.576 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.576 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:32.576 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795360+0000", 2024-09-21T21:41:32.576 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2829999999999998e-06 2024-09-21T21:41:32.576 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.576 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.576 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:32.576 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795366+0000", 2024-09-21T21:41:32.577 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1260000000000003e-06 2024-09-21T21:41:32.577 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.577 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.577 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:32.577 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795370+0000", 2024-09-21T21:41:32.577 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1899999999999997e-06 2024-09-21T21:41:32.577 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.577 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.577 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:32.577 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.828587+0000", 2024-09-21T21:41:32.577 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033216516000000001 2024-09-21T21:41:32.577 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.578 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.578 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:32.578 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.828662+0000", 2024-09-21T21:41:32.578 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5316999999999997e-05 2024-09-21T21:41:32.578 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.578 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.578 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:32.578 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.828748+0000", 2024-09-21T21:41:32.578 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.6068000000000003e-05 2024-09-21T21:41:32.578 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:32.578 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:32.578 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:32.579 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.579 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.579 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4952.0:3533 3.10 3:086eb8a4:::rbd_data.1121a3eabc61.0000000000000000:17 [read 2467840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:32.579 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:29.795376+0000", 2024-09-21T21:41:32.579 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 2.7601265640000001, 2024-09-21T21:41:32.579 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033574701999999998, 2024-09-21T21:41:32.579 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:32.579 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:32.579 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:32.579 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:32.579 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4952", 2024-09-21T21:41:32.579 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:32.580 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3533 2024-09-21T21:41:32.580 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.580 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:32.580 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.580 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:32.580 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795376+0000", 2024-09-21T21:41:32.580 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:32.580 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.580 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.580 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:32.580 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795376+0000", 2024-09-21T21:41:32.580 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:32.580 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.581 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.581 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:32.581 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795377+0000", 2024-09-21T21:41:32.581 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5400000000000002e-07 2024-09-21T21:41:32.581 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.581 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.581 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:32.581 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795379+0000", 2024-09-21T21:41:32.581 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9659999999999999e-06 2024-09-21T21:41:32.581 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.581 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.581 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:32.582 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795385+0000", 2024-09-21T21:41:32.582 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1519999999999997e-06 2024-09-21T21:41:32.582 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.582 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.582 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:32.582 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795389+0000", 2024-09-21T21:41:32.582 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4399999999999998e-06 2024-09-21T21:41:32.582 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.582 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.582 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:32.582 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.828777+0000", 2024-09-21T21:41:32.582 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033388099999999997 2024-09-21T21:41:32.582 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.583 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.583 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:32.583 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.828856+0000", 2024-09-21T21:41:32.583 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.8770999999999995e-05 2024-09-21T21:41:32.583 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.583 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.583 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:32.583 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.828951+0000", 2024-09-21T21:41:32.583 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.4919000000000004e-05 2024-09-21T21:41:32.583 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:32.583 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:32.583 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:32.584 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.584 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.584 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4952.0:3535 3.10 3:086eb8a4:::rbd_data.1121a3eabc61.0000000000000000:17 [read 2482176~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:32.584 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:29.795394+0000", 2024-09-21T21:41:32.584 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 2.7601084350000002, 2024-09-21T21:41:32.584 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033738335000000001, 2024-09-21T21:41:32.584 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:32.584 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:32.584 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:32.584 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:32.584 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4952", 2024-09-21T21:41:32.584 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:32.584 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3535 2024-09-21T21:41:32.585 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.585 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:32.585 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.585 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:32.585 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795394+0000", 2024-09-21T21:41:32.585 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:32.585 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.585 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.585 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:32.585 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795394+0000", 2024-09-21T21:41:32.585 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:32.585 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.586 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.586 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:32.586 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.796945+0000", 2024-09-21T21:41:32.586 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.001551097 2024-09-21T21:41:32.586 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.586 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.586 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:32.586 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.796949+0000", 2024-09-21T21:41:32.586 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5499999999999999e-06 2024-09-21T21:41:32.586 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.586 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.586 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:32.586 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.796958+0000", 2024-09-21T21:41:32.587 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.1649999999999995e-06 2024-09-21T21:41:32.587 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.587 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.587 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:32.587 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.796964+0000", 2024-09-21T21:41:32.587 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1199999999999999e-06 2024-09-21T21:41:32.587 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.587 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.587 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:32.587 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.828983+0000", 2024-09-21T21:41:32.587 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.032018396999999997 2024-09-21T21:41:32.587 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.588 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.588 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:32.588 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.829050+0000", 2024-09-21T21:41:32.588 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.7020000000000005e-05 2024-09-21T21:41:32.588 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.588 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.588 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:32.588 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.829133+0000", 2024-09-21T21:41:32.588 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.2985999999999995e-05 2024-09-21T21:41:32.588 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:32.588 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:32.588 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:32.588 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.589 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.589 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4952.0:3544 3.10 3:086eb8a4:::rbd_data.1121a3eabc61.0000000000000000:17 [read 2562048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:32.589 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:29.797097+0000", 2024-09-21T21:41:32.589 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 2.7584056239999999, 2024-09-21T21:41:32.589 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.032772161000000001, 2024-09-21T21:41:32.589 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:32.589 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:32.589 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:32.589 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:32.589 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4952", 2024-09-21T21:41:32.589 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:32.589 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3544 2024-09-21T21:41:32.590 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.590 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:32.590 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.590 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:32.590 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797097+0000", 2024-09-21T21:41:32.590 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:32.590 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.590 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.590 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:32.590 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797097+0000", 2024-09-21T21:41:32.590 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:32.590 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.591 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.591 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:32.591 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797097+0000", 2024-09-21T21:41:32.591 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.7799999999999997e-07 2024-09-21T21:41:32.591 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.591 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.591 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:32.591 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797099+0000", 2024-09-21T21:41:32.591 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.844e-06 2024-09-21T21:41:32.591 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.591 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.591 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:32.591 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797107+0000", 2024-09-21T21:41:32.592 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.4399999999999999e-06 2024-09-21T21:41:32.592 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.592 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.592 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:32.592 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797111+0000", 2024-09-21T21:41:32.592 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5220000000000001e-06 2024-09-21T21:41:32.592 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.592 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.592 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:32.592 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.829785+0000", 2024-09-21T21:41:32.592 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.032673319999999999 2024-09-21T21:41:32.592 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.592 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.593 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:32.593 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.829822+0000", 2024-09-21T21:41:32.593 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7376999999999998e-05 2024-09-21T21:41:32.593 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.593 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.593 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:32.593 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.829869+0000", 2024-09-21T21:41:32.593 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7379999999999997e-05 2024-09-21T21:41:32.593 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:32.593 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:32.593 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:32.593 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.594 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.594 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4952.0:3546 3.10 3:086eb8a4:::rbd_data.1121a3eabc61.0000000000000000:17 [read 2576384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:32.594 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:29.797117+0000", 2024-09-21T21:41:32.594 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 2.7583855809999998, 2024-09-21T21:41:32.594 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.03288253, 2024-09-21T21:41:32.594 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:32.594 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:32.594 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:32.594 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:32.594 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4952", 2024-09-21T21:41:32.594 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:32.594 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3546 2024-09-21T21:41:32.595 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.595 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:32.595 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.595 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:32.595 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797117+0000", 2024-09-21T21:41:32.595 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:32.595 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.595 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.595 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:32.595 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797117+0000", 2024-09-21T21:41:32.595 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:32.595 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.595 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.596 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:32.596 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797118+0000", 2024-09-21T21:41:32.596 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.39e-07 2024-09-21T21:41:32.596 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.596 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.596 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:32.596 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797120+0000", 2024-09-21T21:41:32.596 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9980000000000002e-06 2024-09-21T21:41:32.596 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.596 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.596 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:32.596 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797125+0000", 2024-09-21T21:41:32.597 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7409999999999998e-06 2024-09-21T21:41:32.597 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.597 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.597 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:32.597 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797130+0000", 2024-09-21T21:41:32.597 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5820000000000003e-06 2024-09-21T21:41:32.597 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.597 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.597 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:32.597 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.829898+0000", 2024-09-21T21:41:32.597 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.032768600000000002 2024-09-21T21:41:32.597 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.597 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.598 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:32.598 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.829934+0000", 2024-09-21T21:41:32.598 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5342000000000001e-05 2024-09-21T21:41:32.598 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.598 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.598 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:32.598 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830000+0000", 2024-09-21T21:41:32.598 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5927999999999996e-05 2024-09-21T21:41:32.598 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:32.598 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:32.599 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:32.599 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.599 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.599 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4952.0:3548 3.10 3:086eb8a4:::rbd_data.1121a3eabc61.0000000000000000:17 [read 2590720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:32.599 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:29.797136+0000", 2024-09-21T21:41:32.599 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 2.7583671409999999, 2024-09-21T21:41:32.601 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033006742999999998, 2024-09-21T21:41:32.601 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:32.602 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:32.602 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:32.602 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:32.602 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4952", 2024-09-21T21:41:32.602 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:32.602 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3548 2024-09-21T21:41:32.602 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.602 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:32.602 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.602 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:32.602 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797136+0000", 2024-09-21T21:41:32.602 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:32.603 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.603 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.603 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:32.603 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797136+0000", 2024-09-21T21:41:32.603 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:32.603 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.603 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.603 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:32.603 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797136+0000", 2024-09-21T21:41:32.603 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3600000000000004e-07 2024-09-21T21:41:32.603 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.603 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.604 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:32.604 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797138+0000", 2024-09-21T21:41:32.604 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.933e-06 2024-09-21T21:41:32.604 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.604 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.604 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:32.604 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797144+0000", 2024-09-21T21:41:32.604 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6099999999999997e-06 2024-09-21T21:41:32.604 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.604 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.604 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:32.604 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797149+0000", 2024-09-21T21:41:32.604 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0690000000000001e-06 2024-09-21T21:41:32.605 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.605 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.605 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:32.605 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830038+0000", 2024-09-21T21:41:32.605 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.032889535999999997 2024-09-21T21:41:32.605 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.605 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.605 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:32.605 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830087+0000", 2024-09-21T21:41:32.605 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9159999999999997e-05 2024-09-21T21:41:32.605 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.605 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.606 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:32.606 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830142+0000", 2024-09-21T21:41:32.606 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4898999999999997e-05 2024-09-21T21:41:32.606 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:32.606 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:32.606 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:32.606 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.606 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.606 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4952.0:3550 3.10 3:086eb8a4:::rbd_data.1121a3eabc61.0000000000000000:17 [read 2605056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:32.606 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:29.797154+0000", 2024-09-21T21:41:32.606 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 2.7583485830000001, 2024-09-21T21:41:32.606 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033113318000000003, 2024-09-21T21:41:32.607 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:32.607 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:32.607 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:32.607 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:32.607 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4952", 2024-09-21T21:41:32.607 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:32.607 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3550 2024-09-21T21:41:32.607 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.607 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:32.607 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.607 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:32.607 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797154+0000", 2024-09-21T21:41:32.607 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:32.608 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.608 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.608 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:32.608 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797154+0000", 2024-09-21T21:41:32.608 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:32.608 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.608 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.608 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:32.608 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797155+0000", 2024-09-21T21:41:32.608 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1199999999999998e-07 2024-09-21T21:41:32.608 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.609 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.609 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:32.609 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797165+0000", 2024-09-21T21:41:32.609 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0635000000000001e-05 2024-09-21T21:41:32.609 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.609 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.609 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:32.609 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797172+0000", 2024-09-21T21:41:32.609 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5300000000000002e-06 2024-09-21T21:41:32.609 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.609 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.609 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:32.609 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797176+0000", 2024-09-21T21:41:32.610 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7419999999999997e-06 2024-09-21T21:41:32.610 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.610 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.610 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:32.610 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830167+0000", 2024-09-21T21:41:32.610 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.032990468000000002 2024-09-21T21:41:32.610 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.610 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.610 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:32.610 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830202+0000", 2024-09-21T21:41:32.610 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5382000000000001e-05 2024-09-21T21:41:32.610 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.611 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.611 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:32.611 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830267+0000", 2024-09-21T21:41:32.611 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5148999999999996e-05 2024-09-21T21:41:32.611 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:32.611 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:32.611 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:32.611 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.611 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.611 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4952.0:3551 3.10 3:086eb8a4:::rbd_data.1121a3eabc61.0000000000000000:17 [read 2619392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:32.611 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:29.797183+0000", 2024-09-21T21:41:32.611 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 2.7583199710000001, 2024-09-21T21:41:32.612 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033198722, 2024-09-21T21:41:32.612 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:32.612 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:32.612 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:32.612 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:32.612 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4952", 2024-09-21T21:41:32.612 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:32.612 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3551 2024-09-21T21:41:32.612 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.612 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:32.612 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.612 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:32.613 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797183+0000", 2024-09-21T21:41:32.613 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:32.613 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.613 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.613 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:32.613 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797183+0000", 2024-09-21T21:41:32.613 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:32.613 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.613 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.613 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:32.613 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797183+0000", 2024-09-21T21:41:32.613 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8000000000000001e-07 2024-09-21T21:41:32.613 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.614 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.614 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:32.614 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797185+0000", 2024-09-21T21:41:32.614 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0940000000000002e-06 2024-09-21T21:41:32.614 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.614 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.614 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:32.614 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797191+0000", 2024-09-21T21:41:32.614 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8300000000000001e-06 2024-09-21T21:41:32.614 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.614 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.614 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:32.615 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797196+0000", 2024-09-21T21:41:32.615 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4530000000000004e-06 2024-09-21T21:41:32.615 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.615 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.615 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:32.615 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830295+0000", 2024-09-21T21:41:32.615 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033099669999999998 2024-09-21T21:41:32.615 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.615 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.615 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:32.615 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830332+0000", 2024-09-21T21:41:32.615 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7156999999999997e-05 2024-09-21T21:41:32.615 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.616 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.616 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:32.616 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830381+0000", 2024-09-21T21:41:32.616 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9138000000000003e-05 2024-09-21T21:41:32.616 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:32.616 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:32.616 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:32.616 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.616 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.616 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4952.0:3553 3.10 3:086eb8a4:::rbd_data.1121a3eabc61.0000000000000000:17 [read 2627584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:32.616 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:29.797201+0000", 2024-09-21T21:41:32.616 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 2.7583017509999999, 2024-09-21T21:41:32.617 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033313262000000003, 2024-09-21T21:41:32.617 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:32.617 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:32.617 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:32.617 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:32.617 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4952", 2024-09-21T21:41:32.617 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:32.617 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3553 2024-09-21T21:41:32.617 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.617 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:32.617 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.617 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:32.617 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797201+0000", 2024-09-21T21:41:32.618 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:32.618 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.618 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.618 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:32.618 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797201+0000", 2024-09-21T21:41:32.618 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:32.618 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.618 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.618 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:32.618 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797201+0000", 2024-09-21T21:41:32.618 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6600000000000002e-07 2024-09-21T21:41:32.618 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.619 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.619 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:32.619 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797203+0000", 2024-09-21T21:41:32.619 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.8810000000000001e-06 2024-09-21T21:41:32.619 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.619 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.619 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:32.619 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797209+0000", 2024-09-21T21:41:32.619 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.037e-06 2024-09-21T21:41:32.619 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.619 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.619 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:32.620 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797214+0000", 2024-09-21T21:41:32.620 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4920000000000004e-06 2024-09-21T21:41:32.620 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.620 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.620 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:32.620 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830408+0000", 2024-09-21T21:41:32.620 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033194199000000001 2024-09-21T21:41:32.620 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.620 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.620 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:32.620 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830455+0000", 2024-09-21T21:41:32.620 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6547e-05 2024-09-21T21:41:32.620 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.621 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.621 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:32.621 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830514+0000", 2024-09-21T21:41:32.621 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9740000000000001e-05 2024-09-21T21:41:32.621 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:32.621 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:32.621 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:32.621 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.621 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.621 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4952.0:3554 3.10 3:086eb8a4:::rbd_data.1121a3eabc61.0000000000000000:17 [read 2641920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:32.621 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:29.797220+0000", 2024-09-21T21:41:32.621 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 2.758282457, 2024-09-21T21:41:32.621 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033396381000000003, 2024-09-21T21:41:32.622 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:32.622 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:32.622 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:32.622 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:32.622 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4952", 2024-09-21T21:41:32.622 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:32.622 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3554 2024-09-21T21:41:32.622 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.622 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:32.622 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.622 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:32.622 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797220+0000", 2024-09-21T21:41:32.622 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:32.623 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.623 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.623 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:32.623 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797220+0000", 2024-09-21T21:41:32.623 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:32.623 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.623 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.623 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:32.623 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797221+0000", 2024-09-21T21:41:32.623 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4099999999999999e-07 2024-09-21T21:41:32.623 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.623 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.624 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:32.624 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797223+0000", 2024-09-21T21:41:32.624 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.95e-06 2024-09-21T21:41:32.624 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.624 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.624 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:32.624 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797228+0000", 2024-09-21T21:41:32.624 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6919999999999999e-06 2024-09-21T21:41:32.624 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.624 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.624 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:32.624 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797238+0000", 2024-09-21T21:41:32.624 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6509999999999996e-06 2024-09-21T21:41:32.625 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.625 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.625 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:32.625 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830532+0000", 2024-09-21T21:41:32.625 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033294473999999998 2024-09-21T21:41:32.625 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.625 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.625 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:32.625 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830569+0000", 2024-09-21T21:41:32.625 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6723000000000001e-05 2024-09-21T21:41:32.625 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.625 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.625 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:32.626 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830617+0000", 2024-09-21T21:41:32.626 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7450000000000001e-05 2024-09-21T21:41:32.626 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:32.626 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:32.626 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:32.626 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.626 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.626 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4952.0:3555 3.10 3:086eb8a4:::rbd_data.1121a3eabc61.0000000000000000:17 [read 2656256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:32.626 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:29.797244+0000", 2024-09-21T21:41:32.626 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 2.7582585850000001, 2024-09-21T21:41:32.626 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033483065999999999, 2024-09-21T21:41:32.626 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:32.626 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:32.627 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:32.627 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:32.627 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4952", 2024-09-21T21:41:32.627 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:32.627 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3555 2024-09-21T21:41:32.627 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.627 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:32.627 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.627 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:32.627 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797244+0000", 2024-09-21T21:41:32.627 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:32.627 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.628 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.628 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:32.628 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797244+0000", 2024-09-21T21:41:32.628 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:32.628 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.628 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.628 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:32.628 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797245+0000", 2024-09-21T21:41:32.628 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9299999999999998e-07 2024-09-21T21:41:32.628 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.628 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.628 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:32.628 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797247+0000", 2024-09-21T21:41:32.629 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0370000000000001e-06 2024-09-21T21:41:32.629 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.629 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.629 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:32.629 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797253+0000", 2024-09-21T21:41:32.629 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8189999999999997e-06 2024-09-21T21:41:32.629 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.629 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.629 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:32.629 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797257+0000", 2024-09-21T21:41:32.629 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.33e-06 2024-09-21T21:41:32.630 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.630 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.630 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:32.630 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830642+0000", 2024-09-21T21:41:32.630 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033385629999999999 2024-09-21T21:41:32.630 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.630 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.630 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:32.630 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830679+0000", 2024-09-21T21:41:32.630 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6100999999999999e-05 2024-09-21T21:41:32.630 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.630 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.631 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:32.631 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830727+0000", 2024-09-21T21:41:32.631 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8656000000000003e-05 2024-09-21T21:41:32.631 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:32.631 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:32.631 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:32.631 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.631 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.631 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4952.0:3557 3.10 3:086eb8a4:::rbd_data.1121a3eabc61.0000000000000000:17 [read 2670592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:32.631 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:29.797263+0000", 2024-09-21T21:41:32.631 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 2.758239723, 2024-09-21T21:41:32.631 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033580213999999997, 2024-09-21T21:41:32.631 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:32.631 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:32.632 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:32.632 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:32.632 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4952", 2024-09-21T21:41:32.632 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:32.632 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3557 2024-09-21T21:41:32.632 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.632 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:32.632 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.632 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:32.632 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797263+0000", 2024-09-21T21:41:32.632 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:32.632 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.632 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.633 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:32.633 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797263+0000", 2024-09-21T21:41:32.633 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:32.633 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.633 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.633 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:32.633 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797263+0000", 2024-09-21T21:41:32.633 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4799999999999999e-07 2024-09-21T21:41:32.633 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.633 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.633 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:32.633 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797265+0000", 2024-09-21T21:41:32.633 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.029e-06 2024-09-21T21:41:32.634 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.634 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.634 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:32.634 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797271+0000", 2024-09-21T21:41:32.634 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0000000000000002e-06 2024-09-21T21:41:32.634 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.634 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.634 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:32.634 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797276+0000", 2024-09-21T21:41:32.634 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5310000000000003e-06 2024-09-21T21:41:32.634 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.634 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.635 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:32.635 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830756+0000", 2024-09-21T21:41:32.635 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033479636 2024-09-21T21:41:32.635 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.635 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.635 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:32.635 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830792+0000", 2024-09-21T21:41:32.635 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6584000000000001e-05 2024-09-21T21:41:32.635 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.635 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.635 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:32.635 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830843+0000", 2024-09-21T21:41:32.635 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0986000000000003e-05 2024-09-21T21:41:32.635 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:32.636 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:32.636 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:32.636 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.636 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.636 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4952.0:3558 3.10 3:086eb8a4:::rbd_data.1121a3eabc61.0000000000000000:17 [read 2684928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:32.636 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:29.797281+0000", 2024-09-21T21:41:32.636 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 2.7582214270000001, 2024-09-21T21:41:32.636 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033692174999999998, 2024-09-21T21:41:32.636 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:32.636 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:32.636 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:32.636 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:32.636 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4952", 2024-09-21T21:41:32.636 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:32.637 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3558 2024-09-21T21:41:32.637 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.637 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:32.637 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.637 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:32.637 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797281+0000", 2024-09-21T21:41:32.637 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:32.637 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.637 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.637 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:32.637 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797281+0000", 2024-09-21T21:41:32.637 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:32.637 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.638 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.638 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:32.638 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797282+0000", 2024-09-21T21:41:32.638 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.41e-07 2024-09-21T21:41:32.638 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.638 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.638 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:32.638 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797283+0000", 2024-09-21T21:41:32.638 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.8390000000000001e-06 2024-09-21T21:41:32.638 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.638 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.638 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:32.639 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797289+0000", 2024-09-21T21:41:32.639 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3839999999999996e-06 2024-09-21T21:41:32.639 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.639 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.639 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:32.639 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797294+0000", 2024-09-21T21:41:32.639 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1900000000000003e-06 2024-09-21T21:41:32.639 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.639 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.639 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:32.639 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830874+0000", 2024-09-21T21:41:32.639 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033580384999999997 2024-09-21T21:41:32.640 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.640 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.640 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:32.640 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830912+0000", 2024-09-21T21:41:32.640 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7367000000000003e-05 2024-09-21T21:41:32.640 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.640 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.640 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:32.640 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830973+0000", 2024-09-21T21:41:32.640 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1668999999999995e-05 2024-09-21T21:41:32.640 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:32.640 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:32.641 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:32.641 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.641 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.641 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4952.0:3560 3.10 3:086eb8a4:::rbd_data.1121a3eabc61.0000000000000000:17 [read 2699264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:32.641 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:29.797300+0000", 2024-09-21T21:41:32.641 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 2.7582027440000001, 2024-09-21T21:41:32.641 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033777109, 2024-09-21T21:41:32.641 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:32.641 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:32.641 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:32.641 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:32.641 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4952", 2024-09-21T21:41:32.641 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:32.642 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3560 2024-09-21T21:41:32.642 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.642 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:32.642 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.642 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:32.642 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797300+0000", 2024-09-21T21:41:32.642 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:32.642 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.642 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.642 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:32.642 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797300+0000", 2024-09-21T21:41:32.642 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:32.643 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.643 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.643 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:32.643 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797300+0000", 2024-09-21T21:41:32.643 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.2399999999999999e-07 2024-09-21T21:41:32.643 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.643 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.643 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:32.643 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797302+0000", 2024-09-21T21:41:32.643 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0260000000000002e-06 2024-09-21T21:41:32.643 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.643 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.644 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:32.644 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797308+0000", 2024-09-21T21:41:32.644 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8839999999999999e-06 2024-09-21T21:41:32.644 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.644 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.644 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:32.644 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797313+0000", 2024-09-21T21:41:32.644 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3959999999999999e-06 2024-09-21T21:41:32.644 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.644 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.644 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:32.644 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830997+0000", 2024-09-21T21:41:32.644 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033684506000000003 2024-09-21T21:41:32.645 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.645 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.645 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:32.645 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.831033+0000", 2024-09-21T21:41:32.645 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5964000000000003e-05 2024-09-21T21:41:32.645 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.645 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.645 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:32.645 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.831077+0000", 2024-09-21T21:41:32.645 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4008999999999998e-05 2024-09-21T21:41:32.645 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:32.646 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:32.646 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:32.646 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.646 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.646 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4952.0:3562 3.10 3:086eb8a4:::rbd_data.1121a3eabc61.0000000000000000:17 [read 2713600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:32.646 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:29.797318+0000", 2024-09-21T21:41:32.646 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 2.7581848870000001, 2024-09-21T21:41:32.646 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033877712999999997, 2024-09-21T21:41:32.646 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:32.646 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:32.646 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:32.646 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:32.647 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4952", 2024-09-21T21:41:32.647 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:32.647 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3562 2024-09-21T21:41:32.647 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.647 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:32.647 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.647 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:32.647 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797318+0000", 2024-09-21T21:41:32.647 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:32.647 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.647 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.647 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:32.648 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797318+0000", 2024-09-21T21:41:32.648 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:32.648 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.648 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.648 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:32.648 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797318+0000", 2024-09-21T21:41:32.648 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.2000000000000001e-07 2024-09-21T21:41:32.648 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.648 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.648 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:32.648 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797320+0000", 2024-09-21T21:41:32.648 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9700000000000002e-06 2024-09-21T21:41:32.649 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.649 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.649 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:32.649 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797326+0000", 2024-09-21T21:41:32.649 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.212e-06 2024-09-21T21:41:32.649 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.649 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.649 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:32.649 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797331+0000", 2024-09-21T21:41:32.649 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9409999999999998e-06 2024-09-21T21:41:32.649 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.649 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.649 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:32.650 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.831104+0000", 2024-09-21T21:41:32.650 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033772976000000003 2024-09-21T21:41:32.650 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.650 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.650 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:32.650 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.831144+0000", 2024-09-21T21:41:32.650 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.0209000000000001e-05 2024-09-21T21:41:32.650 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.650 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.650 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:32.650 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.831196+0000", 2024-09-21T21:41:32.650 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1085000000000003e-05 2024-09-21T21:41:32.651 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:32.651 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:32.651 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:32.651 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.651 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.651 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4952.0:3563 3.10 3:086eb8a4:::rbd_data.1121a3eabc61.0000000000000000:17 [read 2727936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:32.651 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:29.797350+0000", 2024-09-21T21:41:32.651 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 2.7581526350000001, 2024-09-21T21:41:32.651 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033952501000000003, 2024-09-21T21:41:32.651 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:32.651 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:32.651 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:32.652 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:32.652 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4952", 2024-09-21T21:41:32.652 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:32.652 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3563 2024-09-21T21:41:32.652 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.652 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:32.652 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.652 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:32.652 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797350+0000", 2024-09-21T21:41:32.652 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:32.652 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.652 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.652 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:32.653 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797350+0000", 2024-09-21T21:41:32.653 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:32.653 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.653 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.653 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:32.653 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797351+0000", 2024-09-21T21:41:32.653 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5499999999999998e-07 2024-09-21T21:41:32.653 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.653 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.653 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:32.653 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797353+0000", 2024-09-21T21:41:32.653 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2299999999999998e-06 2024-09-21T21:41:32.654 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.654 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.654 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:32.654 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797359+0000", 2024-09-21T21:41:32.654 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3330000000000002e-06 2024-09-21T21:41:32.654 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.654 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.654 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:32.654 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797364+0000", 2024-09-21T21:41:32.654 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4410000000000003e-06 2024-09-21T21:41:32.654 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.654 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.655 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:32.655 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.831223+0000", 2024-09-21T21:41:32.655 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033859002999999999 2024-09-21T21:41:32.655 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.655 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.655 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:32.655 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.831258+0000", 2024-09-21T21:41:32.655 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5417000000000003e-05 2024-09-21T21:41:32.655 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.655 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.655 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:32.655 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.831303+0000", 2024-09-21T21:41:32.656 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4521999999999998e-05 2024-09-21T21:41:32.656 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:32.656 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:32.656 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:32.656 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.656 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.656 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4952.0:3564 3.10 3:086eb8a4:::rbd_data.1121a3eabc61.0000000000000000:17 [read 2742272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:32.656 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:29.797370+0000", 2024-09-21T21:41:32.656 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 2.7581327839999998, 2024-09-21T21:41:32.656 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.034051217000000002, 2024-09-21T21:41:32.656 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:32.656 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:32.657 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:32.657 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:32.657 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4952", 2024-09-21T21:41:32.657 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:32.657 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3564 2024-09-21T21:41:32.657 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.657 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:32.657 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.657 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:32.657 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797370+0000", 2024-09-21T21:41:32.657 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:32.657 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.658 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.658 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:32.658 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797370+0000", 2024-09-21T21:41:32.658 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:32.658 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.658 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.658 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:32.658 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797370+0000", 2024-09-21T21:41:32.658 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5499999999999998e-07 2024-09-21T21:41:32.658 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.658 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.658 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:32.659 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797372+0000", 2024-09-21T21:41:32.659 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9290000000000001e-06 2024-09-21T21:41:32.659 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.659 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.659 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:32.659 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797378+0000", 2024-09-21T21:41:32.659 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9599999999999997e-06 2024-09-21T21:41:32.659 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.659 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.659 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:32.659 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797388+0000", 2024-09-21T21:41:32.660 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.5040000000000008e-06 2024-09-21T21:41:32.660 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.660 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.660 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:32.660 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.831335+0000", 2024-09-21T21:41:32.660 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033947374000000002 2024-09-21T21:41:32.660 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.660 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.660 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:32.660 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.831371+0000", 2024-09-21T21:41:32.660 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5868999999999998e-05 2024-09-21T21:41:32.660 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:32.661 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:32.661 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:32.661 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.831421+0000", 2024-09-21T21:41:32.661 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0126000000000001e-05 2024-09-21T21:41:32.661 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:32.661 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:32.661 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:32.661 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:32.661 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:32.661 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:32.664 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T21:41:32.664 INFO:teuthology.orchestra.run.smithi164.stdout:3295 write 0x797b000 thru 0x7985728 (0xa729 bytes) 2024-09-21T21:41:32.665 INFO:teuthology.orchestra.run.smithi164.stdout:3296 read 0x94b1c00 thru 0x94bd6b4 (0xbab5 bytes) 2024-09-21T21:41:32.665 INFO:teuthology.orchestra.run.smithi164.stdout:3297 write 0x9111a00 thru 0x911a6ea (0x8ceb bytes) 2024-09-21T21:41:32.665 INFO:teuthology.orchestra.run.smithi164.stdout:3299 punch from 0xba06a00 to 0xba0f600, (0x8c00 bytes) 2024-09-21T21:41:32.665 INFO:teuthology.orchestra.run.smithi164.stdout:3300 punch from 0x849e200 to 0x84acc00, (0xea00 bytes) 2024-09-21T21:41:32.665 INFO:teuthology.orchestra.run.smithi164.stdout:3301 read 0xad1600 thru 0xaddb50 (0xc551 bytes) 2024-09-21T21:41:32.665 INFO:teuthology.orchestra.run.smithi164.stdout:3302 read 0x8c66400 thru 0x8c6f3f2 (0x8ff3 bytes) 2024-09-21T21:41:32.665 INFO:teuthology.orchestra.run.smithi164.stdout:3303 read 0x7bd5200 thru 0x7bde3db (0x91dc bytes) 2024-09-21T21:41:32.665 INFO:teuthology.orchestra.run.smithi164.stdout:3304 write 0xa4ce000 thru 0xa4d165f (0x3660 bytes) 2024-09-21T21:41:32.665 INFO:teuthology.orchestra.run.smithi164.stdout:3305 read 0x3cac00 thru 0x3d2cac (0x80ad bytes) 2024-09-21T21:41:32.665 INFO:teuthology.orchestra.run.smithi164.stdout:3307 clone 24 order 18 su 8192 sc 7 2024-09-21T21:41:32.666 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:32.661+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:32.698 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T21:41:32.763 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:32.761+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:32.775 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:32.775 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:41:32.775 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 0 2024-09-21T21:41:32.775 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:32.777 INFO:teuthology.orchestra.run.smithi164.stdout:leaving image image_client.0-clone23 intact 2024-09-21T21:41:32.784 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T21:41:32.864 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:32.863+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:32.892 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:32.892 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:41:32.892 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:41:32.892 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:41:32.892 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:32.901 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T21:41:32.966 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:32.964+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:33.007 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:33.007 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 20, 2024-09-21T21:41:33.007 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 600, 2024-09-21T21:41:33.007 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:41:33.007 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.007 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4824.0:1564 3.2 3:4647520c:::rbd_data.1121b97f2925.0000000000000006:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:33.007 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:41:33.007 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 31.368817346, 2024-09-21T21:41:33.008 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056097347999999998, 2024-09-21T21:41:33.008 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:33.008 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:33.008 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:33.008 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:33.008 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4824", 2024-09-21T21:41:33.008 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:33.008 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1564 2024-09-21T21:41:33.008 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.008 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:33.008 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.008 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:33.008 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:41:33.008 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:33.009 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.009 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.009 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:33.009 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:41:33.009 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:33.009 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.009 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.009 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:33.009 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:41:33.009 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6499999999999999e-07 2024-09-21T21:41:33.009 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.009 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.009 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:33.009 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636067+0000", 2024-09-21T21:41:33.010 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3879999999999998e-06 2024-09-21T21:41:33.010 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.010 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.010 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:33.010 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636075+0000", 2024-09-21T21:41:33.010 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5480000000000004e-06 2024-09-21T21:41:33.010 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.010 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.010 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:33.010 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636080+0000", 2024-09-21T21:41:33.010 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.339e-06 2024-09-21T21:41:33.010 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.010 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.010 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:33.011 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636133+0000", 2024-09-21T21:41:33.011 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2930000000000003e-05 2024-09-21T21:41:33.011 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.011 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.011 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:33.011 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636265+0000", 2024-09-21T21:41:33.011 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000132058 2024-09-21T21:41:33.011 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.011 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.011 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:33.011 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.692162+0000", 2024-09-21T21:41:33.011 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055896620000000001 2024-09-21T21:41:33.011 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:33.011 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:33.012 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:33.012 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.012 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.012 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4912.0:871 3.19 3:9ddcaed2:::rbd_data.1121b3a54d7e.0000000000000000:12 [sparse-read 16121856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-21T21:41:33.012 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:20.721293+0000", 2024-09-21T21:41:33.012 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 12.283589391, 2024-09-21T21:41:33.012 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.026485022, 2024-09-21T21:41:33.012 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:33.012 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:33.012 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:33.012 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:33.012 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4912", 2024-09-21T21:41:33.012 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4204709190", 2024-09-21T21:41:33.012 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 871 2024-09-21T21:41:33.012 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.013 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:33.013 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.013 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:33.013 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.721293+0000", 2024-09-21T21:41:33.013 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:33.013 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.013 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.013 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:33.013 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.721293+0000", 2024-09-21T21:41:33.013 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:33.014 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.014 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.014 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:33.014 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.721293+0000", 2024-09-21T21:41:33.014 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3000000000000002e-07 2024-09-21T21:41:33.014 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.014 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.014 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:33.014 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.721295+0000", 2024-09-21T21:41:33.014 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.7749999999999999e-06 2024-09-21T21:41:33.015 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.015 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.015 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:33.015 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.721300+0000", 2024-09-21T21:41:33.015 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3319999999999999e-06 2024-09-21T21:41:33.015 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.015 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.015 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:33.015 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.721305+0000", 2024-09-21T21:41:33.015 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.578e-06 2024-09-21T21:41:33.015 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.015 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.015 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:33.015 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.747610+0000", 2024-09-21T21:41:33.016 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.026305469000000001 2024-09-21T21:41:33.016 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.016 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.016 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:33.016 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.747691+0000", 2024-09-21T21:41:33.016 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.1390999999999994e-05 2024-09-21T21:41:33.016 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.016 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.016 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:33.016 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.747778+0000", 2024-09-21T21:41:33.016 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.6147e-05 2024-09-21T21:41:33.016 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:33.016 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:33.016 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:33.016 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.017 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.017 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4912.0:938 3.19 3:9ddcaed2:::rbd_data.1121b3a54d7e.0000000000000000:12 [sparse-read 12845056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-21T21:41:33.017 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:20.723322+0000", 2024-09-21T21:41:33.017 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 12.28155978, 2024-09-21T21:41:33.017 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.026387079000000001, 2024-09-21T21:41:33.017 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:33.017 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:33.017 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:33.017 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:33.017 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4912", 2024-09-21T21:41:33.017 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4204709190", 2024-09-21T21:41:33.017 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 938 2024-09-21T21:41:33.017 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.017 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:33.018 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.018 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:33.018 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723322+0000", 2024-09-21T21:41:33.018 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:33.018 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.018 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.018 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:33.018 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723322+0000", 2024-09-21T21:41:33.018 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:33.018 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.018 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.018 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:33.018 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723322+0000", 2024-09-21T21:41:33.018 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.72e-07 2024-09-21T21:41:33.018 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.019 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.019 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:33.019 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723324+0000", 2024-09-21T21:41:33.019 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9250000000000002e-06 2024-09-21T21:41:33.019 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.019 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.019 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:33.019 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723330+0000", 2024-09-21T21:41:33.019 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6799999999999998e-06 2024-09-21T21:41:33.019 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.019 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.019 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:33.019 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723334+0000", 2024-09-21T21:41:33.020 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3640000000000001e-06 2024-09-21T21:41:33.020 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.020 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.020 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:33.020 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.749545+0000", 2024-09-21T21:41:33.020 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.026210715999999998 2024-09-21T21:41:33.020 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.020 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.020 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:33.020 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.749623+0000", 2024-09-21T21:41:33.020 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7602999999999995e-05 2024-09-21T21:41:33.020 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.020 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.020 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:33.020 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.749709+0000", 2024-09-21T21:41:33.021 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.6419e-05 2024-09-21T21:41:33.021 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:33.021 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:33.021 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:33.021 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.021 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.021 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4912.0:939 3.19 3:9ddcaed2:::rbd_data.1121b3a54d7e.0000000000000000:12 [sparse-read 14548992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-21T21:41:33.021 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:20.723340+0000", 2024-09-21T21:41:33.021 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 12.281541581999999, 2024-09-21T21:41:33.021 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.026505974000000002, 2024-09-21T21:41:33.021 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:33.021 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:33.021 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:33.022 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:33.022 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4912", 2024-09-21T21:41:33.022 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4204709190", 2024-09-21T21:41:33.022 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 939 2024-09-21T21:41:33.022 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.022 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:33.022 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.022 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:33.022 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723340+0000", 2024-09-21T21:41:33.022 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:33.022 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.022 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.022 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:33.022 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723340+0000", 2024-09-21T21:41:33.023 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:33.023 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.023 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.023 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:33.023 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723341+0000", 2024-09-21T21:41:33.023 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6100000000000002e-07 2024-09-21T21:41:33.023 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.023 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.023 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:33.023 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723342+0000", 2024-09-21T21:41:33.023 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.782e-06 2024-09-21T21:41:33.023 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.023 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.023 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:33.024 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723348+0000", 2024-09-21T21:41:33.024 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3990000000000003e-06 2024-09-21T21:41:33.024 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.024 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.024 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:33.024 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723353+0000", 2024-09-21T21:41:33.024 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3009999999999997e-06 2024-09-21T21:41:33.024 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.024 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.024 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:33.025 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.749741+0000", 2024-09-21T21:41:33.025 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.026387911999999999 2024-09-21T21:41:33.025 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.025 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.025 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:33.025 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.749788+0000", 2024-09-21T21:41:33.025 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7354000000000001e-05 2024-09-21T21:41:33.025 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.025 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.025 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:33.025 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.749846+0000", 2024-09-21T21:41:33.025 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7865000000000002e-05 2024-09-21T21:41:33.025 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:33.026 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:33.026 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:33.026 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.026 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.026 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4912.0:940 3.19 3:9ddcaed2:::rbd_data.1121b3a54d7e.0000000000000000:12 [sparse-read 16252928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-21T21:41:33.026 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:20.723360+0000", 2024-09-21T21:41:33.026 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 12.281522367000001, 2024-09-21T21:41:33.026 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.026697166000000001, 2024-09-21T21:41:33.026 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:33.026 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:33.026 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:33.026 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:33.026 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4912", 2024-09-21T21:41:33.027 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4204709190", 2024-09-21T21:41:33.027 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 940 2024-09-21T21:41:33.027 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.027 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:33.027 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.027 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:33.027 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723360+0000", 2024-09-21T21:41:33.027 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:33.027 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.027 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.027 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:33.027 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723360+0000", 2024-09-21T21:41:33.027 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:33.027 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.027 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.028 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:33.028 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723360+0000", 2024-09-21T21:41:33.028 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.9900000000000001e-07 2024-09-21T21:41:33.028 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.028 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.028 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:33.028 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723362+0000", 2024-09-21T21:41:33.028 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1289999999999999e-06 2024-09-21T21:41:33.028 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.028 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.028 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:33.028 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723368+0000", 2024-09-21T21:41:33.028 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.75e-06 2024-09-21T21:41:33.029 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.029 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.029 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:33.029 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723373+0000", 2024-09-21T21:41:33.029 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6959999999999996e-06 2024-09-21T21:41:33.029 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.029 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.029 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:33.029 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.749882+0000", 2024-09-21T21:41:33.029 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.026509893999999999 2024-09-21T21:41:33.029 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.029 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.029 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:33.029 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.749966+0000", 2024-09-21T21:41:33.029 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.3449999999999996e-05 2024-09-21T21:41:33.030 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.030 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.030 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:33.030 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.750057+0000", 2024-09-21T21:41:33.030 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.0847999999999995e-05 2024-09-21T21:41:33.030 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:33.030 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:33.030 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:33.030 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.030 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.030 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4912.0:999 3.19 3:9ddcaed2:::rbd_data.1121b3a54d7e.0000000000000000:12 [sparse-read 1441792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-21T21:41:33.030 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:20.723784+0000", 2024-09-21T21:41:33.030 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 12.281097428000001, 2024-09-21T21:41:33.031 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.026500717, 2024-09-21T21:41:33.031 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:33.031 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:33.031 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:33.031 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:33.031 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4912", 2024-09-21T21:41:33.031 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4204709190", 2024-09-21T21:41:33.031 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 999 2024-09-21T21:41:33.031 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.031 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:33.031 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.031 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:33.031 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723784+0000", 2024-09-21T21:41:33.031 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:33.031 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.032 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.032 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:33.032 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723784+0000", 2024-09-21T21:41:33.032 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:33.032 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.032 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.032 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:33.032 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723785+0000", 2024-09-21T21:41:33.032 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4900000000000001e-07 2024-09-21T21:41:33.032 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.032 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.032 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:33.032 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723787+0000", 2024-09-21T21:41:33.032 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9819999999999998e-06 2024-09-21T21:41:33.033 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.033 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.033 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:33.033 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723793+0000", 2024-09-21T21:41:33.033 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.032e-06 2024-09-21T21:41:33.033 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.033 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.033 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:33.033 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723798+0000", 2024-09-21T21:41:33.033 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7099999999999998e-06 2024-09-21T21:41:33.033 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.033 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.033 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:33.033 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.750104+0000", 2024-09-21T21:41:33.034 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.02630658 2024-09-21T21:41:33.034 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.034 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.034 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:33.034 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.750185+0000", 2024-09-21T21:41:33.034 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0345000000000004e-05 2024-09-21T21:41:33.034 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.034 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.034 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:33.034 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.750285+0000", 2024-09-21T21:41:33.034 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010061899999999999 2024-09-21T21:41:33.034 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:33.034 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:33.035 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:33.035 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.035 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.035 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4912.0:1000 3.19 3:9ddcaed2:::rbd_data.1121b3a54d7e.0000000000000000:12 [sparse-read 3145728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-21T21:41:33.035 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:20.723804+0000", 2024-09-21T21:41:33.035 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 12.281078246, 2024-09-21T21:41:33.035 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.026694361, 2024-09-21T21:41:33.035 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:33.035 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:33.035 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:33.035 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:33.035 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4912", 2024-09-21T21:41:33.035 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4204709190", 2024-09-21T21:41:33.035 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1000 2024-09-21T21:41:33.036 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.036 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:33.036 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.036 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:33.036 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723804+0000", 2024-09-21T21:41:33.036 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:33.036 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.036 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.036 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:33.036 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723804+0000", 2024-09-21T21:41:33.036 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:33.036 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.037 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.037 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:33.037 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723804+0000", 2024-09-21T21:41:33.037 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7399999999999999e-07 2024-09-21T21:41:33.037 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.037 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.037 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:33.037 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723811+0000", 2024-09-21T21:41:33.037 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.1149999999999996e-06 2024-09-21T21:41:33.037 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.037 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.037 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:33.037 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723818+0000", 2024-09-21T21:41:33.037 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9779999999999999e-06 2024-09-21T21:41:33.038 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.038 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.038 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:33.038 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723826+0000", 2024-09-21T21:41:33.038 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0600000000000008e-06 2024-09-21T21:41:33.038 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.038 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.038 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:33.038 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.750308+0000", 2024-09-21T21:41:33.038 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.026482316999999998 2024-09-21T21:41:33.038 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.038 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.038 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:33.038 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.750403+0000", 2024-09-21T21:41:33.039 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.4775e-05 2024-09-21T21:41:33.039 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.039 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.039 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:33.039 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.750498+0000", 2024-09-21T21:41:33.039 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.4741999999999995e-05 2024-09-21T21:41:33.039 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:33.039 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:33.039 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:33.039 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.039 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.039 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4912.0:1001 3.19 3:9ddcaed2:::rbd_data.1121b3a54d7e.0000000000000000:12 [sparse-read 4849664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-21T21:41:33.039 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:20.723833+0000", 2024-09-21T21:41:33.040 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 12.281049389, 2024-09-21T21:41:33.040 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.026886792999999999, 2024-09-21T21:41:33.040 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:33.040 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:33.040 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:33.040 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:33.040 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4912", 2024-09-21T21:41:33.040 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4204709190", 2024-09-21T21:41:33.040 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1001 2024-09-21T21:41:33.040 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.040 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:33.040 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.040 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:33.041 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723833+0000", 2024-09-21T21:41:33.041 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:33.041 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.041 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.041 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:33.041 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723833+0000", 2024-09-21T21:41:33.041 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:33.041 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.041 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.041 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:33.041 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723833+0000", 2024-09-21T21:41:33.041 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4099999999999999e-07 2024-09-21T21:41:33.041 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.041 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.042 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:33.042 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723835+0000", 2024-09-21T21:41:33.042 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.968e-06 2024-09-21T21:41:33.042 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.042 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.042 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:33.042 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723841+0000", 2024-09-21T21:41:33.042 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5209999999999999e-06 2024-09-21T21:41:33.042 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.042 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.042 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:33.042 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723846+0000", 2024-09-21T21:41:33.042 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9810000000000003e-06 2024-09-21T21:41:33.042 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.043 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.043 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:33.043 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.750538+0000", 2024-09-21T21:41:33.043 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.026692041 2024-09-21T21:41:33.043 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.043 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.043 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:33.043 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.750625+0000", 2024-09-21T21:41:33.043 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.6706000000000006e-05 2024-09-21T21:41:33.043 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.043 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.043 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:33.043 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.750719+0000", 2024-09-21T21:41:33.043 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.4135000000000006e-05 2024-09-21T21:41:33.044 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:33.044 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:33.044 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:33.044 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.044 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.044 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4912.0:1002 3.19 3:9ddcaed2:::rbd_data.1121b3a54d7e.0000000000000000:12 [sparse-read 6553600~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-21T21:41:33.044 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:20.723853+0000", 2024-09-21T21:41:33.044 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 12.281029286000001, 2024-09-21T21:41:33.044 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.027244385999999999, 2024-09-21T21:41:33.044 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:33.044 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:33.044 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:33.044 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:33.044 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4912", 2024-09-21T21:41:33.045 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4204709190", 2024-09-21T21:41:33.045 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1002 2024-09-21T21:41:33.045 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.045 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:33.045 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.045 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:33.045 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723853+0000", 2024-09-21T21:41:33.045 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:33.045 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.045 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.045 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:33.045 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723853+0000", 2024-09-21T21:41:33.045 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:33.045 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.046 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.046 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:33.046 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723853+0000", 2024-09-21T21:41:33.046 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6899999999999998e-07 2024-09-21T21:41:33.046 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.046 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.046 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:33.046 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723858+0000", 2024-09-21T21:41:33.046 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0710000000000002e-06 2024-09-21T21:41:33.046 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.046 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.046 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:33.046 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723865+0000", 2024-09-21T21:41:33.046 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.613e-06 2024-09-21T21:41:33.047 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.047 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.047 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:33.047 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723870+0000", 2024-09-21T21:41:33.047 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0350000000000001e-06 2024-09-21T21:41:33.047 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.047 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.047 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:33.047 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.750749+0000", 2024-09-21T21:41:33.047 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.026878808000000001 2024-09-21T21:41:33.047 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.047 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.047 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:33.047 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.750830+0000", 2024-09-21T21:41:33.048 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.1817999999999995e-05 2024-09-21T21:41:33.048 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.048 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.048 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:33.048 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.751097+0000", 2024-09-21T21:41:33.048 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00026667200000000002 2024-09-21T21:41:33.048 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:33.048 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:33.048 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:33.048 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.048 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.048 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4912.0:1003 3.19 3:9ddcaed2:::rbd_data.1121b3a54d7e.0000000000000000:12 [sparse-read 8257536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-21T21:41:33.048 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:20.723876+0000", 2024-09-21T21:41:33.048 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 12.281005509, 2024-09-21T21:41:33.048 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.027396406000000002, 2024-09-21T21:41:33.049 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:33.049 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:33.049 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:33.049 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:33.049 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4912", 2024-09-21T21:41:33.049 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4204709190", 2024-09-21T21:41:33.049 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1003 2024-09-21T21:41:33.049 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.049 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:33.049 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.049 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:33.049 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723876+0000", 2024-09-21T21:41:33.049 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:33.049 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.049 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.050 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:33.050 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723876+0000", 2024-09-21T21:41:33.050 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:33.050 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.050 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.050 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:33.050 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723877+0000", 2024-09-21T21:41:33.050 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5999999999999999e-07 2024-09-21T21:41:33.050 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.050 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.050 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:33.050 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723879+0000", 2024-09-21T21:41:33.050 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1299999999999999e-06 2024-09-21T21:41:33.050 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.051 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.051 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:33.051 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723885+0000", 2024-09-21T21:41:33.051 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9580000000000004e-06 2024-09-21T21:41:33.051 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.051 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.051 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:33.051 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723889+0000", 2024-09-21T21:41:33.051 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4850000000000003e-06 2024-09-21T21:41:33.051 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.051 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.051 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:33.051 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.751125+0000", 2024-09-21T21:41:33.051 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.027236136000000001 2024-09-21T21:41:33.052 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.052 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.052 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:33.052 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.751171+0000", 2024-09-21T21:41:33.052 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5708999999999999e-05 2024-09-21T21:41:33.052 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.052 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.052 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:33.052 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.751273+0000", 2024-09-21T21:41:33.052 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000101628 2024-09-21T21:41:33.052 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:33.052 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:33.052 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:33.052 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.053 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.053 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4912.0:1004 3.19 3:9ddcaed2:::rbd_data.1121b3a54d7e.0000000000000000:12 [sparse-read 9568256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-21T21:41:33.053 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:20.723895+0000", 2024-09-21T21:41:33.053 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 12.280986800000001, 2024-09-21T21:41:33.053 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.027611696000000002, 2024-09-21T21:41:33.053 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:33.053 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:33.053 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:33.053 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:33.053 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4912", 2024-09-21T21:41:33.053 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4204709190", 2024-09-21T21:41:33.053 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1004 2024-09-21T21:41:33.053 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.053 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:33.053 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.054 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:33.054 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723895+0000", 2024-09-21T21:41:33.054 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:33.054 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.054 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.054 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:33.054 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723895+0000", 2024-09-21T21:41:33.054 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:33.054 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.054 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.054 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:33.054 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723895+0000", 2024-09-21T21:41:33.054 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.1199999999999999e-07 2024-09-21T21:41:33.054 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.055 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.055 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:33.055 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723897+0000", 2024-09-21T21:41:33.055 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.8059999999999999e-06 2024-09-21T21:41:33.055 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.055 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.055 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:33.055 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723903+0000", 2024-09-21T21:41:33.055 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9939999999999997e-06 2024-09-21T21:41:33.055 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.055 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.055 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:33.055 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723908+0000", 2024-09-21T21:41:33.055 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5260000000000004e-06 2024-09-21T21:41:33.056 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.056 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.056 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:33.056 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.751316+0000", 2024-09-21T21:41:33.056 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.027408377000000001 2024-09-21T21:41:33.056 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.056 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.056 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:33.056 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.751402+0000", 2024-09-21T21:41:33.056 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.5830999999999999e-05 2024-09-21T21:41:33.056 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.056 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.056 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:33.056 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.751507+0000", 2024-09-21T21:41:33.056 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010485 2024-09-21T21:41:33.057 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:33.057 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:33.057 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:33.057 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.057 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.057 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4912.0:1005 3.19 3:9ddcaed2:::rbd_data.1121b3a54d7e.0000000000000000:12 [sparse-read 11272192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-21T21:41:33.057 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:20.723914+0000", 2024-09-21T21:41:33.057 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 12.280968029, 2024-09-21T21:41:33.057 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.027789695999999999, 2024-09-21T21:41:33.057 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:33.057 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:33.057 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:33.057 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:33.057 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4912", 2024-09-21T21:41:33.057 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4204709190", 2024-09-21T21:41:33.058 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1005 2024-09-21T21:41:33.058 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.058 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:33.058 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.058 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:33.058 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723914+0000", 2024-09-21T21:41:33.058 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:33.058 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.058 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.058 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:33.058 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723914+0000", 2024-09-21T21:41:33.058 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:33.058 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.059 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.059 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:33.059 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723914+0000", 2024-09-21T21:41:33.059 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4099999999999999e-07 2024-09-21T21:41:33.059 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.059 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.059 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:33.059 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723916+0000", 2024-09-21T21:41:33.059 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0559999999999999e-06 2024-09-21T21:41:33.059 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.059 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.059 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:33.059 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723923+0000", 2024-09-21T21:41:33.059 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.139e-06 2024-09-21T21:41:33.059 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.060 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.060 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:33.060 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723927+0000", 2024-09-21T21:41:33.060 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.826e-06 2024-09-21T21:41:33.060 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.060 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.060 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:33.060 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.751550+0000", 2024-09-21T21:41:33.060 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.027622357 2024-09-21T21:41:33.060 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.060 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.060 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:33.060 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.751636+0000", 2024-09-21T21:41:33.060 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.6604e-05 2024-09-21T21:41:33.060 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.061 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.061 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:33.061 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.751704+0000", 2024-09-21T21:41:33.061 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.7173000000000002e-05 2024-09-21T21:41:33.061 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:33.061 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:33.061 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:33.061 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.061 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.061 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4912.0:1006 3.19 3:9ddcaed2:::rbd_data.1121b3a54d7e.0000000000000000:12 [sparse-read 12976128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-21T21:41:33.061 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:20.723939+0000", 2024-09-21T21:41:33.061 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 12.280942548000001, 2024-09-21T21:41:33.061 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.028010094999999999, 2024-09-21T21:41:33.061 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:33.061 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:33.062 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:33.062 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:33.062 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4912", 2024-09-21T21:41:33.062 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4204709190", 2024-09-21T21:41:33.062 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1006 2024-09-21T21:41:33.062 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.062 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:33.062 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.062 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:33.062 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723939+0000", 2024-09-21T21:41:33.062 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:33.062 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.062 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.062 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:33.063 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723939+0000", 2024-09-21T21:41:33.063 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:33.063 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.063 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.063 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:33.063 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723940+0000", 2024-09-21T21:41:33.063 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4999999999999998e-07 2024-09-21T21:41:33.063 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.063 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.063 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:33.063 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723942+0000", 2024-09-21T21:41:33.063 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1660000000000001e-06 2024-09-21T21:41:33.063 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.063 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.064 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:33.064 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723949+0000", 2024-09-21T21:41:33.064 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0949999999999998e-06 2024-09-21T21:41:33.064 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.064 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.064 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:33.064 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723954+0000", 2024-09-21T21:41:33.064 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0250000000000002e-06 2024-09-21T21:41:33.064 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.064 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.064 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:33.064 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.751755+0000", 2024-09-21T21:41:33.064 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.027801228000000001 2024-09-21T21:41:33.064 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.064 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.065 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:33.065 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.751841+0000", 2024-09-21T21:41:33.065 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.5829999999999996e-05 2024-09-21T21:41:33.065 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.065 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.065 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:33.065 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.751949+0000", 2024-09-21T21:41:33.065 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010830100000000001 2024-09-21T21:41:33.065 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:33.065 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:33.065 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:33.065 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.065 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.066 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4912.0:1007 3.19 3:9ddcaed2:::rbd_data.1121b3a54d7e.0000000000000000:12 [sparse-read 14680064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-21T21:41:33.066 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:20.723961+0000", 2024-09-21T21:41:33.066 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 12.280921373, 2024-09-21T21:41:33.066 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.028245155000000001, 2024-09-21T21:41:33.066 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:33.066 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:33.066 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:33.066 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:33.066 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4912", 2024-09-21T21:41:33.066 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4204709190", 2024-09-21T21:41:33.066 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1007 2024-09-21T21:41:33.066 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.066 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:33.066 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.067 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:33.067 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723961+0000", 2024-09-21T21:41:33.067 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:33.067 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.067 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.067 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:33.067 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723961+0000", 2024-09-21T21:41:33.067 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:33.067 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.067 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.067 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:33.067 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723961+0000", 2024-09-21T21:41:33.067 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4900000000000001e-07 2024-09-21T21:41:33.067 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.067 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.068 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:33.068 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723963+0000", 2024-09-21T21:41:33.068 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.787e-06 2024-09-21T21:41:33.068 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.068 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.068 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:33.068 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723969+0000", 2024-09-21T21:41:33.068 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8289999999999996e-06 2024-09-21T21:41:33.068 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.068 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.068 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:33.068 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723973+0000", 2024-09-21T21:41:33.068 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5499999999999996e-06 2024-09-21T21:41:33.068 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.069 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.069 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:33.069 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.751994+0000", 2024-09-21T21:41:33.069 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.028021146 2024-09-21T21:41:33.069 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.069 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.069 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:33.069 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.752101+0000", 2024-09-21T21:41:33.069 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010621899999999999 2024-09-21T21:41:33.069 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.069 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.069 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:33.069 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.752206+0000", 2024-09-21T21:41:33.069 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000105175 2024-09-21T21:41:33.069 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:33.070 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:33.070 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:33.070 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.070 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.070 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4912.0:1008 3.19 3:9ddcaed2:::rbd_data.1121b3a54d7e.0000000000000000:12 [sparse-read 16384000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-21T21:41:33.070 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:20.723978+0000", 2024-09-21T21:41:33.070 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 12.280903552, 2024-09-21T21:41:33.070 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.028444661999999999, 2024-09-21T21:41:33.070 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:33.070 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:33.070 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:33.070 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:33.070 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4912", 2024-09-21T21:41:33.070 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4204709190", 2024-09-21T21:41:33.071 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1008 2024-09-21T21:41:33.071 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.071 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:33.071 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.071 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:33.071 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723978+0000", 2024-09-21T21:41:33.071 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:33.071 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.071 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.071 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:33.071 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723978+0000", 2024-09-21T21:41:33.071 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:33.071 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.071 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.072 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:33.072 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.724409+0000", 2024-09-21T21:41:33.072 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00043058300000000001 2024-09-21T21:41:33.072 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.072 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.072 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:33.072 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.724412+0000", 2024-09-21T21:41:33.072 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3730000000000001e-06 2024-09-21T21:41:33.072 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.072 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.072 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:33.072 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.724421+0000", 2024-09-21T21:41:33.072 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.9840000000000007e-06 2024-09-21T21:41:33.072 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.073 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.073 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:33.073 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.724427+0000", 2024-09-21T21:41:33.073 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2029999999999997e-06 2024-09-21T21:41:33.073 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.073 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.073 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:33.073 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.752238+0000", 2024-09-21T21:41:33.073 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.027810577999999999 2024-09-21T21:41:33.073 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.073 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.073 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:33.073 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.752323+0000", 2024-09-21T21:41:33.073 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.4652000000000002e-05 2024-09-21T21:41:33.073 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.074 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.074 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:33.074 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.752423+0000", 2024-09-21T21:41:33.074 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000100289 2024-09-21T21:41:33.074 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:33.074 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:33.074 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:33.074 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.074 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.074 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5196.0:984 3.2 3:404ccd4e:::rbd_data.1121a3eabc61.0000000000000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-21T21:41:33.074 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:27.956266+0000", 2024-09-21T21:41:33.074 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 5.0486157340000002, 2024-09-21T21:41:33.074 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.032332125000000003, 2024-09-21T21:41:33.074 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:33.075 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:33.075 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:41:33.075 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:33.075 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5196", 2024-09-21T21:41:33.075 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3707540331", 2024-09-21T21:41:33.075 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 984 2024-09-21T21:41:33.075 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.075 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:33.075 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.075 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:33.075 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.956266+0000", 2024-09-21T21:41:33.075 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:33.075 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.075 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.076 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:33.076 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.956266+0000", 2024-09-21T21:41:33.076 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:33.076 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.076 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.076 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:33.076 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.956267+0000", 2024-09-21T21:41:33.076 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0800000000000004e-07 2024-09-21T21:41:33.076 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.076 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.076 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:33.076 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.956271+0000", 2024-09-21T21:41:33.076 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.0199999999999996e-06 2024-09-21T21:41:33.076 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.076 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.077 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:33.077 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.956281+0000", 2024-09-21T21:41:33.077 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0669000000000001e-05 2024-09-21T21:41:33.077 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.077 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.077 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:33.077 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.956288+0000", 2024-09-21T21:41:33.077 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9639999999999997e-06 2024-09-21T21:41:33.077 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.077 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.077 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:33.077 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.956337+0000", 2024-09-21T21:41:33.077 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8538999999999997e-05 2024-09-21T21:41:33.077 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.077 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.078 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:33.078 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.956413+0000", 2024-09-21T21:41:33.078 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5652e-05 2024-09-21T21:41:33.078 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.078 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.078 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:41:33.078 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.956519+0000", 2024-09-21T21:41:33.078 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010645 2024-09-21T21:41:33.078 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.078 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.078 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:41:33.078 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.988330+0000", 2024-09-21T21:41:33.078 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031810488999999997 2024-09-21T21:41:33.078 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.079 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.079 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:41:33.079 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.988532+0000", 2024-09-21T21:41:33.079 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00020196999999999999 2024-09-21T21:41:33.079 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.079 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.079 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:41:33.079 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.988563+0000", 2024-09-21T21:41:33.079 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.1951e-05 2024-09-21T21:41:33.079 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.079 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.079 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:33.079 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.988598+0000", 2024-09-21T21:41:33.079 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4813e-05 2024-09-21T21:41:33.080 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:33.080 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:33.080 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:33.080 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.080 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.080 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5196.0:991 3.7 3:e594ca4e:::rbd_header.1121a3eabc61:head [notify cookie 139700387993488] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-21T21:41:33.080 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:27.956339+0000", 2024-09-21T21:41:33.080 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 5.0485425069999996, 2024-09-21T21:41:33.080 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031791316, 2024-09-21T21:41:33.080 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:33.080 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:33.080 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:33.080 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:33.080 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5196", 2024-09-21T21:41:33.080 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3707540331", 2024-09-21T21:41:33.081 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 991 2024-09-21T21:41:33.081 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.081 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:33.081 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.081 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:33.081 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.956339+0000", 2024-09-21T21:41:33.081 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:33.081 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.081 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.081 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:33.081 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.956339+0000", 2024-09-21T21:41:33.081 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:33.081 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.082 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.082 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:33.082 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.956340+0000", 2024-09-21T21:41:33.082 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7999999999999996e-07 2024-09-21T21:41:33.082 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.082 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.082 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:33.082 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.956343+0000", 2024-09-21T21:41:33.082 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.8260000000000001e-06 2024-09-21T21:41:33.082 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.082 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.082 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:33.082 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.956350+0000", 2024-09-21T21:41:33.082 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.1019999999999999e-06 2024-09-21T21:41:33.083 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.083 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.083 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:33.083 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.956355+0000", 2024-09-21T21:41:33.083 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6149999999999996e-06 2024-09-21T21:41:33.083 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.083 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.083 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:33.083 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.956507+0000", 2024-09-21T21:41:33.083 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000151963 2024-09-21T21:41:33.083 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.083 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.083 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:33.083 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.988010+0000", 2024-09-21T21:41:33.083 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031502636000000001 2024-09-21T21:41:33.084 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.084 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.084 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:33.084 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.988131+0000", 2024-09-21T21:41:33.084 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00012069399999999999 2024-09-21T21:41:33.084 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:33.084 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:33.084 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:33.084 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.084 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.084 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5196.0:992 3.7 3:e594ca4e:::rbd_header.1121a3eabc61:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-21T21:41:33.084 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:27.956555+0000", 2024-09-21T21:41:33.084 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 5.0483272799999996, 2024-09-21T21:41:33.084 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031712787999999999, 2024-09-21T21:41:33.084 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:33.085 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:33.085 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:33.085 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:33.085 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5196", 2024-09-21T21:41:33.085 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3707540331", 2024-09-21T21:41:33.085 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 992 2024-09-21T21:41:33.085 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.085 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:33.085 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.085 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:33.085 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.956555+0000", 2024-09-21T21:41:33.085 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:33.085 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.085 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.085 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:33.086 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.956555+0000", 2024-09-21T21:41:33.086 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:33.086 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.086 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.086 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:33.086 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.956555+0000", 2024-09-21T21:41:33.086 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.2900000000000002e-07 2024-09-21T21:41:33.086 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.086 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.086 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:33.086 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.956561+0000", 2024-09-21T21:41:33.086 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1699999999999996e-06 2024-09-21T21:41:33.086 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.086 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.087 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:33.087 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.987985+0000", 2024-09-21T21:41:33.087 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031424872 2024-09-21T21:41:33.087 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.087 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.087 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:33.087 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.987996+0000", 2024-09-21T21:41:33.087 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0742e-05 2024-09-21T21:41:33.087 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.087 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.087 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:33.087 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.988161+0000", 2024-09-21T21:41:33.087 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00016449800000000001 2024-09-21T21:41:33.087 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.087 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.088 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:33.088 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.988214+0000", 2024-09-21T21:41:33.088 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3528e-05 2024-09-21T21:41:33.088 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.088 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.088 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:33.088 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.988267+0000", 2024-09-21T21:41:33.088 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3149000000000002e-05 2024-09-21T21:41:33.088 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:33.088 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:33.088 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:33.088 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.088 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.088 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4385.0:11863 3.7 3:e594ca4e:::rbd_header.1121a3eabc61:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-21T21:41:33.089 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:27.956588+0000", 2024-09-21T21:41:33.089 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 5.048293642, 2024-09-21T21:41:33.089 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031871075999999998, 2024-09-21T21:41:33.089 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:33.089 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:33.089 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:33.089 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:33.089 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4385", 2024-09-21T21:41:33.089 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:41:33.089 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11863 2024-09-21T21:41:33.089 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.089 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:33.089 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.089 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:33.090 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.956588+0000", 2024-09-21T21:41:33.090 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:33.090 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.090 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.090 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:33.090 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.956588+0000", 2024-09-21T21:41:33.090 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:33.090 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.090 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.090 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:33.090 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.956589+0000", 2024-09-21T21:41:33.090 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.1699999999999997e-07 2024-09-21T21:41:33.090 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.090 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.090 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:33.091 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.956593+0000", 2024-09-21T21:41:33.091 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1219999999999997e-06 2024-09-21T21:41:33.091 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.091 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.091 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:33.091 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.987976+0000", 2024-09-21T21:41:33.091 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031382366000000002 2024-09-21T21:41:33.091 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.091 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.091 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:33.091 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.987996+0000", 2024-09-21T21:41:33.091 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0010999999999999e-05 2024-09-21T21:41:33.091 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.091 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.092 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:33.092 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.988296+0000", 2024-09-21T21:41:33.092 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00030068500000000003 2024-09-21T21:41:33.092 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.092 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.092 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:33.092 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.988365+0000", 2024-09-21T21:41:33.092 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.8986999999999995e-05 2024-09-21T21:41:33.092 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.092 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.092 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:33.092 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.988459+0000", 2024-09-21T21:41:33.092 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.4087999999999998e-05 2024-09-21T21:41:33.092 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:33.092 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:33.093 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:33.093 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.093 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.093 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_repop_reply(client.5196.0:984 3.2 e58/21 ondisk, result = 0)", 2024-09-21T21:41:33.093 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:27.958905+0000", 2024-09-21T21:41:33.093 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 5.0459765340000002, 2024-09-21T21:41:33.093 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.029433239, 2024-09-21T21:41:33.093 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:33.093 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:33.093 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:33.093 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:33.093 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.093 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:33.093 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.958905+0000", 2024-09-21T21:41:33.093 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:33.094 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.094 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.094 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:33.094 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.958905+0000", 2024-09-21T21:41:33.094 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:33.094 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.094 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.094 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:33.094 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.958906+0000", 2024-09-21T21:41:33.094 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.8199999999999999e-07 2024-09-21T21:41:33.094 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.094 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.094 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:33.094 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.958914+0000", 2024-09-21T21:41:33.095 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0180000000000001e-06 2024-09-21T21:41:33.095 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.095 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.095 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:33.095 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.987982+0000", 2024-09-21T21:41:33.095 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.029067829999999999 2024-09-21T21:41:33.095 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.095 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.095 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:33.095 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.987995+0000", 2024-09-21T21:41:33.095 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.3196999999999999e-05 2024-09-21T21:41:33.095 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.095 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.095 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:33.096 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.988312+0000", 2024-09-21T21:41:33.096 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00031657400000000002 2024-09-21T21:41:33.096 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.096 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.096 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:33.096 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.988323+0000", 2024-09-21T21:41:33.096 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1344e-05 2024-09-21T21:41:33.096 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.096 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.096 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:33.096 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.988339+0000", 2024-09-21T21:41:33.096 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.5594e-05 2024-09-21T21:41:33.096 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:33.096 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:33.097 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:33.097 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:33.097 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:33.097 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:33.098 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T21:41:33.099 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:33.096+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:33.187 INFO:teuthology.orchestra.run.smithi164.stdout:checking clone #22, image image_client.0-clone22 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent23 2024-09-21T21:41:33.199 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:33.197+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:33.242 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:33.242 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:41:33.242 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 0 2024-09-21T21:41:33.243 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:33.245 INFO:teuthology.orchestra.run.smithi164.stdout:3309 read 0xafeca00 thru 0xaff9c2c (0xd22d bytes) 2024-09-21T21:41:33.251 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T21:41:33.259 INFO:teuthology.orchestra.run.smithi164.stdout:3310 write 0xda9b000 thru 0xda9b7c9 (0x7ca bytes) 2024-09-21T21:41:33.260 INFO:teuthology.orchestra.run.smithi164.stdout:3311 write 0xa0a0800 thru 0xa0a15fb (0xdfc bytes) 2024-09-21T21:41:33.260 INFO:teuthology.orchestra.run.smithi164.stdout:3312 read 0x314bc00 thru 0x314d9f9 (0x1dfa bytes) 2024-09-21T21:41:33.260 INFO:teuthology.orchestra.run.smithi164.stdout:3313 read 0x3fd7000 thru 0x3fd952d (0x252e bytes) 2024-09-21T21:41:33.274 INFO:teuthology.orchestra.run.smithi164.stdout:3317 write 0xe555e00 thru 0xe55c117 (0x6318 bytes) 2024-09-21T21:41:33.275 INFO:teuthology.orchestra.run.smithi164.stdout:3318 read 0x3ec1c00 thru 0x3ec8ca2 (0x70a3 bytes) 2024-09-21T21:41:33.275 INFO:teuthology.orchestra.run.smithi164.stdout:3320 read 0xb5da400 thru 0xb5e75da (0xd1db bytes) 2024-09-21T21:41:33.275 INFO:teuthology.orchestra.run.smithi164.stdout:3321 read 0x87a9000 thru 0x87b452b (0xb52c bytes) 2024-09-21T21:41:33.275 INFO:teuthology.orchestra.run.smithi164.stdout:3322 write 0xb8c200 thru 0xb942fa (0x80fb bytes) 2024-09-21T21:41:33.277 INFO:teuthology.orchestra.run.smithi164.stdout:3324 trunc from 0xe55c200 to 0xc1aea00 2024-09-21T21:41:33.313 INFO:teuthology.orchestra.run.smithi164.stdout:3325 read 0x626b600 thru 0x627aece (0xf8cf bytes) 2024-09-21T21:41:33.314 INFO:teuthology.orchestra.run.smithi164.stdout:3330 write 0xad60400 thru 0xad63f67 (0x3b68 bytes) 2024-09-21T21:41:33.314 INFO:teuthology.orchestra.run.smithi164.stdout:3333 write 0xdd0200 thru 0xdde4b7 (0xe2b8 bytes) 2024-09-21T21:41:33.314 INFO:teuthology.orchestra.run.smithi164.stdout:3334 punch from 0x3007000 to 0x3007200, (0x200 bytes) 2024-09-21T21:41:33.315 INFO:teuthology.orchestra.run.smithi164.stdout:3335 write 0x5c9ae00 thru 0x5ca5a09 (0xac0a bytes) 2024-09-21T21:41:33.315 INFO:teuthology.orchestra.run.smithi164.stdout:3336 read 0x38fd000 thru 0x39042eb (0x72ec bytes) 2024-09-21T21:41:33.315 INFO:teuthology.orchestra.run.smithi164.stdout:3338 read 0x301800 thru 0x30a337 (0x8b38 bytes) 2024-09-21T21:41:33.335 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:33.334+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:33.342 INFO:teuthology.orchestra.run.smithi164.stdout:3340 write 0xd402400 thru 0xd409e62 (0x7a63 bytes) 2024-09-21T21:41:33.343 INFO:teuthology.orchestra.run.smithi164.stdout:3343 read 0x68cc00 thru 0x68eb4a (0x1f4b bytes) 2024-09-21T21:41:33.347 INFO:teuthology.orchestra.run.smithi164.stdout:3344 punch from 0x6218000 to 0x6219a00, (0x1a00 bytes) 2024-09-21T21:41:33.348 INFO:teuthology.orchestra.run.smithi164.stdout:3346 read 0x2021600 thru 0x202ea68 (0xd469 bytes) 2024-09-21T21:41:33.348 INFO:teuthology.orchestra.run.smithi164.stdout:3350 read 0xbe9fc00 thru 0xbea3d7f (0x4180 bytes) 2024-09-21T21:41:33.356 INFO:teuthology.orchestra.run.smithi164.stdout:3351 trunc from 0xd40a000 to 0x607aa00 2024-09-21T21:41:33.388 INFO:teuthology.orchestra.run.smithi164.stdout:3354 punch from 0x4e9c400 to 0x4e9dc00, (0x1800 bytes) 2024-09-21T21:41:33.400 INFO:teuthology.orchestra.run.smithi164.stdout:3355 write 0x62ec000 thru 0x62fa170 (0xe171 bytes) 2024-09-21T21:41:33.400 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:33.399+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:33.403 INFO:teuthology.orchestra.run.smithi164.stdout:3357 trunc from 0x62fa200 to 0x4601800 2024-09-21T21:41:33.405 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:33.405 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:41:33.405 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:41:33.405 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:41:33.405 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:33.415 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T21:41:33.451 INFO:teuthology.orchestra.run.smithi164.stdout:3358 write 0xc35bc00 thru 0xc35f1a1 (0x35a2 bytes) 2024-09-21T21:41:33.456 INFO:teuthology.orchestra.run.smithi164.stdout:3359 trunc from 0xc35f200 to 0x8250200 2024-09-21T21:41:33.480 INFO:teuthology.orchestra.run.smithi164.stdout:3360 write 0x1fd4400 thru 0x1fd9dfc (0x59fd bytes) 2024-09-21T21:41:33.488 INFO:teuthology.orchestra.run.smithi164.stdout:3362 read 0x28d2200 thru 0x28d8e27 (0x6c28 bytes) 2024-09-21T21:41:33.501 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:33.499+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:33.505 INFO:teuthology.orchestra.run.smithi164.stdout:3363 write 0xa3f4000 thru 0xa3f7c4e (0x3c4f bytes) 2024-09-21T21:41:33.511 INFO:teuthology.orchestra.run.smithi164.stdout:3364 punch from 0x413b400 to 0x4141800, (0x6400 bytes) 2024-09-21T21:41:33.515 INFO:teuthology.orchestra.run.smithi164.stdout:3368 write 0x8425400 thru 0x842bc60 (0x6861 bytes) 2024-09-21T21:41:33.515 INFO:teuthology.orchestra.run.smithi164.stdout:3369 read 0x8b14000 thru 0x8b19c73 (0x5c74 bytes) 2024-09-21T21:41:33.515 INFO:teuthology.orchestra.run.smithi164.stdout:3370 read 0x107600 thru 0x111bc3 (0xa5c4 bytes) 2024-09-21T21:41:33.519 INFO:teuthology.orchestra.run.smithi164.stdout:3371 trunc from 0xa3f7e00 to 0x7d38e00 2024-09-21T21:41:33.542 INFO:teuthology.orchestra.run.smithi164.stdout:3372 read 0x6e97800 thru 0x6ea1214 (0x9a15 bytes) 2024-09-21T21:41:33.543 INFO:teuthology.orchestra.run.smithi164.stdout:3373 read 0x3708600 thru 0x370e279 (0x5c7a bytes) 2024-09-21T21:41:33.543 INFO:teuthology.orchestra.run.smithi164.stdout:3374 write 0x7432800 thru 0x743d664 (0xae65 bytes) 2024-09-21T21:41:33.545 INFO:teuthology.orchestra.run.smithi164.stdout:3375 trunc from 0x7d38e00 to 0x6307e00 2024-09-21T21:41:33.565 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:33.565 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 20, 2024-09-21T21:41:33.565 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 600, 2024-09-21T21:41:33.565 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:41:33.565 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.565 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:1678 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 14483456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:33.565 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.299288+0000", 2024-09-21T21:41:33.566 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.2637274119999997, 2024-09-21T21:41:33.566 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031376253, 2024-09-21T21:41:33.566 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:33.566 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:33.566 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:33.566 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:33.566 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:33.566 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:33.566 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1678 2024-09-21T21:41:33.566 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.566 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:33.566 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.566 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:33.566 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299288+0000", 2024-09-21T21:41:33.566 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:33.567 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.567 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.567 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:33.567 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299288+0000", 2024-09-21T21:41:33.567 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:33.567 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.567 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.567 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:33.567 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299288+0000", 2024-09-21T21:41:33.567 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.2500000000000001e-07 2024-09-21T21:41:33.567 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.567 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.567 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:33.567 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299290+0000", 2024-09-21T21:41:33.568 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.945e-06 2024-09-21T21:41:33.568 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.568 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.568 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:33.568 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299295+0000", 2024-09-21T21:41:33.568 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0379999999999999e-06 2024-09-21T21:41:33.568 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.568 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.568 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:33.568 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299299+0000", 2024-09-21T21:41:33.568 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7960000000000002e-06 2024-09-21T21:41:33.568 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.568 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.568 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:33.569 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.330586+0000", 2024-09-21T21:41:33.569 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031287540000000003 2024-09-21T21:41:33.569 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.569 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.569 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:33.569 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.330620+0000", 2024-09-21T21:41:33.569 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3492999999999999e-05 2024-09-21T21:41:33.569 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.569 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.569 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:33.569 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.330664+0000", 2024-09-21T21:41:33.569 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4116000000000002e-05 2024-09-21T21:41:33.569 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:33.569 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:33.569 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:33.570 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.570 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.570 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:1679 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 14942208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:33.570 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.299333+0000", 2024-09-21T21:41:33.570 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.2636817570000005, 2024-09-21T21:41:33.570 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031433567000000003, 2024-09-21T21:41:33.570 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:33.570 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:33.570 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:33.570 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:33.570 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:33.570 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:33.570 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1679 2024-09-21T21:41:33.570 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.571 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:33.571 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.571 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:33.571 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299333+0000", 2024-09-21T21:41:33.571 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:33.571 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.571 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.571 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:33.571 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299333+0000", 2024-09-21T21:41:33.571 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:33.571 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.571 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.571 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:33.571 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299334+0000", 2024-09-21T21:41:33.571 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0100000000000005e-07 2024-09-21T21:41:33.572 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.572 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.572 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:33.572 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299336+0000", 2024-09-21T21:41:33.572 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3439999999999999e-06 2024-09-21T21:41:33.572 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.572 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.572 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:33.572 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299343+0000", 2024-09-21T21:41:33.572 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0299999999999996e-06 2024-09-21T21:41:33.572 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.572 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.572 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:33.572 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299348+0000", 2024-09-21T21:41:33.572 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2340000000000002e-06 2024-09-21T21:41:33.573 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.573 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.573 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:33.573 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.330687+0000", 2024-09-21T21:41:33.573 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031338132999999997 2024-09-21T21:41:33.573 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.573 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.573 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:33.573 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.330721+0000", 2024-09-21T21:41:33.573 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4156999999999998e-05 2024-09-21T21:41:33.573 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.573 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.573 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:33.573 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.330767+0000", 2024-09-21T21:41:33.574 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6168000000000002e-05 2024-09-21T21:41:33.574 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:33.574 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:33.574 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:33.574 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.574 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.574 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:1680 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 15400960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:33.574 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.299356+0000", 2024-09-21T21:41:33.574 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.2636594070000005, 2024-09-21T21:41:33.574 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031510096000000001, 2024-09-21T21:41:33.574 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:33.574 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:33.574 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:33.574 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:33.574 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:33.575 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:33.575 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1680 2024-09-21T21:41:33.575 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.575 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:33.575 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.575 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:33.575 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299356+0000", 2024-09-21T21:41:33.575 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:33.575 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.575 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.575 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:33.575 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299356+0000", 2024-09-21T21:41:33.575 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:33.575 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.576 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.576 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:33.576 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299356+0000", 2024-09-21T21:41:33.576 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.63e-07 2024-09-21T21:41:33.576 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.576 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.576 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:33.576 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299358+0000", 2024-09-21T21:41:33.576 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1399999999999998e-06 2024-09-21T21:41:33.576 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.576 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.576 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:33.577 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299364+0000", 2024-09-21T21:41:33.577 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1650000000000003e-06 2024-09-21T21:41:33.577 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.577 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.577 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:33.577 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299369+0000", 2024-09-21T21:41:33.577 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8400000000000002e-06 2024-09-21T21:41:33.577 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.577 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.577 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:33.577 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.330788+0000", 2024-09-21T21:41:33.577 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031418399 2024-09-21T21:41:33.577 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.577 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.577 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:33.577 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.330822+0000", 2024-09-21T21:41:33.578 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4218999999999999e-05 2024-09-21T21:41:33.578 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.578 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.578 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:33.578 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.330866+0000", 2024-09-21T21:41:33.578 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3869999999999998e-05 2024-09-21T21:41:33.578 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:33.578 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:33.578 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:33.578 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.578 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.578 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:1681 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 15859712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:33.578 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.299375+0000", 2024-09-21T21:41:33.578 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.2636397030000008, 2024-09-21T21:41:33.579 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031600251000000003, 2024-09-21T21:41:33.579 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:33.579 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:33.579 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:33.579 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:33.579 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:33.579 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:33.579 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1681 2024-09-21T21:41:33.579 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.579 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:33.579 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.579 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:33.579 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299375+0000", 2024-09-21T21:41:33.579 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:33.580 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.580 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.580 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:33.580 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299375+0000", 2024-09-21T21:41:33.580 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:33.580 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.580 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.580 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:33.580 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299376+0000", 2024-09-21T21:41:33.580 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8700000000000001e-07 2024-09-21T21:41:33.580 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.580 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.580 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:33.580 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299382+0000", 2024-09-21T21:41:33.580 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3060000000000003e-06 2024-09-21T21:41:33.581 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.581 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.581 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:33.581 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299389+0000", 2024-09-21T21:41:33.581 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.3629999999999996e-06 2024-09-21T21:41:33.581 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.581 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.581 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:33.581 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299395+0000", 2024-09-21T21:41:33.581 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1279999999999999e-06 2024-09-21T21:41:33.581 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.581 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.581 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:33.581 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.330891+0000", 2024-09-21T21:41:33.581 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031496108000000002 2024-09-21T21:41:33.581 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.582 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.582 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:33.582 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.330925+0000", 2024-09-21T21:41:33.582 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4150000000000003e-05 2024-09-21T21:41:33.582 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.582 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.582 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:33.582 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.330976+0000", 2024-09-21T21:41:33.582 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0809000000000001e-05 2024-09-21T21:41:33.582 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:33.582 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:33.582 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:33.582 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.582 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.583 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:1682 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 16318464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:33.583 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.299401+0000", 2024-09-21T21:41:33.583 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.263614059, 2024-09-21T21:41:33.583 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031676932999999997, 2024-09-21T21:41:33.583 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:33.583 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:33.583 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:33.583 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:33.583 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:33.583 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:33.583 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1682 2024-09-21T21:41:33.583 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.583 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:33.583 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.584 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:33.584 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299401+0000", 2024-09-21T21:41:33.584 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:33.584 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.584 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.584 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:33.584 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299401+0000", 2024-09-21T21:41:33.584 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:33.584 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.584 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.584 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:33.584 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299401+0000", 2024-09-21T21:41:33.584 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.9299999999999999e-07 2024-09-21T21:41:33.584 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.584 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.585 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:33.585 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299404+0000", 2024-09-21T21:41:33.585 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1720000000000001e-06 2024-09-21T21:41:33.585 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.585 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.585 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:33.585 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299410+0000", 2024-09-21T21:41:33.585 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2480000000000001e-06 2024-09-21T21:41:33.585 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.585 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.585 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:33.585 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299414+0000", 2024-09-21T21:41:33.585 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6480000000000002e-06 2024-09-21T21:41:33.586 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.586 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.586 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:33.586 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.330998+0000", 2024-09-21T21:41:33.586 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031583364000000003 2024-09-21T21:41:33.586 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.586 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.586 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:33.586 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.331033+0000", 2024-09-21T21:41:33.586 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4759999999999999e-05 2024-09-21T21:41:33.586 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.586 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.586 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:33.587 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.331078+0000", 2024-09-21T21:41:33.587 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5348e-05 2024-09-21T21:41:33.587 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:33.587 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:33.587 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:33.587 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.587 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.587 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:2010 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 7864320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:33.587 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.306462+0000", 2024-09-21T21:41:33.587 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.2565529729999998, 2024-09-21T21:41:33.587 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031317032000000002, 2024-09-21T21:41:33.587 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:33.587 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:33.587 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:33.587 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:33.588 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:33.588 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:33.588 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2010 2024-09-21T21:41:33.588 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.588 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:33.588 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.588 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:33.588 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306462+0000", 2024-09-21T21:41:33.588 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:33.588 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.588 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.588 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:33.588 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306462+0000", 2024-09-21T21:41:33.588 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:33.588 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.589 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.589 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:33.589 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306463+0000", 2024-09-21T21:41:33.589 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2500000000000001e-07 2024-09-21T21:41:33.589 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.589 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.589 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:33.589 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306465+0000", 2024-09-21T21:41:33.589 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9740000000000001e-06 2024-09-21T21:41:33.589 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.589 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.589 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:33.589 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306470+0000", 2024-09-21T21:41:33.589 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9689999999999999e-06 2024-09-21T21:41:33.589 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.590 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.590 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:33.590 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306475+0000", 2024-09-21T21:41:33.590 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7199999999999997e-06 2024-09-21T21:41:33.590 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.590 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.590 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:33.590 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.337701+0000", 2024-09-21T21:41:33.590 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031225796 2024-09-21T21:41:33.590 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.590 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.590 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:33.590 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.337735+0000", 2024-09-21T21:41:33.590 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4399e-05 2024-09-21T21:41:33.591 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.591 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.591 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:33.591 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.337779+0000", 2024-09-21T21:41:33.591 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3748999999999997e-05 2024-09-21T21:41:33.591 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:33.591 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:33.591 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:33.591 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.591 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.591 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:2011 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 8323072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:33.591 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.306481+0000", 2024-09-21T21:41:33.591 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.2565338690000001, 2024-09-21T21:41:33.592 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031404840000000003, 2024-09-21T21:41:33.592 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:33.592 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:33.592 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:33.592 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:33.592 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:33.592 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:33.592 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2011 2024-09-21T21:41:33.592 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.592 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:33.592 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.592 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:33.592 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306481+0000", 2024-09-21T21:41:33.592 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:33.593 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.593 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.593 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:33.593 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306481+0000", 2024-09-21T21:41:33.593 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:33.593 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.593 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.593 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:33.593 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306482+0000", 2024-09-21T21:41:33.593 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9100000000000004e-07 2024-09-21T21:41:33.593 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.593 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.593 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:33.593 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306484+0000", 2024-09-21T21:41:33.593 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.855e-06 2024-09-21T21:41:33.594 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.594 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.594 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:33.594 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306489+0000", 2024-09-21T21:41:33.594 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7799999999999997e-06 2024-09-21T21:41:33.594 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.594 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.594 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:33.594 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306494+0000", 2024-09-21T21:41:33.594 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7539999999999997e-06 2024-09-21T21:41:33.594 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.594 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.594 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:33.594 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.337802+0000", 2024-09-21T21:41:33.594 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031307684000000002 2024-09-21T21:41:33.595 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.595 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.595 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:33.595 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.337836+0000", 2024-09-21T21:41:33.595 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4560000000000001e-05 2024-09-21T21:41:33.595 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.595 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.595 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:33.595 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.337886+0000", 2024-09-21T21:41:33.595 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9716000000000003e-05 2024-09-21T21:41:33.595 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:33.595 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:33.595 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:33.595 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.596 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.596 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:2012 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 8781824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:33.596 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.306500+0000", 2024-09-21T21:41:33.596 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.2565150539999994, 2024-09-21T21:41:33.596 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031483165, 2024-09-21T21:41:33.596 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:33.596 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:33.596 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:33.596 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:33.596 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:33.596 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:33.596 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2012 2024-09-21T21:41:33.596 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.596 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:33.597 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.597 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:33.597 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306500+0000", 2024-09-21T21:41:33.597 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:33.597 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.597 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.597 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:33.597 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306500+0000", 2024-09-21T21:41:33.597 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:33.597 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.597 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.597 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:33.597 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306500+0000", 2024-09-21T21:41:33.597 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7099999999999997e-07 2024-09-21T21:41:33.597 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.598 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.598 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:33.598 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306502+0000", 2024-09-21T21:41:33.598 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1019999999999999e-06 2024-09-21T21:41:33.598 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.598 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.598 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:33.598 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306508+0000", 2024-09-21T21:41:33.598 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8390000000000003e-06 2024-09-21T21:41:33.598 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.598 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.598 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:33.598 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306513+0000", 2024-09-21T21:41:33.598 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4909999999999999e-06 2024-09-21T21:41:33.599 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.599 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.599 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:33.599 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.337898+0000", 2024-09-21T21:41:33.599 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031385437000000002 2024-09-21T21:41:33.599 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.599 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.599 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:33.599 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.337931+0000", 2024-09-21T21:41:33.599 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.2656e-05 2024-09-21T21:41:33.599 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.599 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.599 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:33.599 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.337983+0000", 2024-09-21T21:41:33.599 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2268999999999997e-05 2024-09-21T21:41:33.600 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:33.600 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:33.600 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:33.600 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.600 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.600 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:2013 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 9240576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:33.600 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.306520+0000", 2024-09-21T21:41:33.600 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.2564953259999996, 2024-09-21T21:41:33.600 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031570864999999997, 2024-09-21T21:41:33.600 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:33.600 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:33.600 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:33.600 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:33.600 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:33.600 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:33.601 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2013 2024-09-21T21:41:33.601 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.601 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:33.601 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.601 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:33.601 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306520+0000", 2024-09-21T21:41:33.601 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:33.601 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.601 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.601 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:33.601 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306520+0000", 2024-09-21T21:41:33.601 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:33.601 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.601 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.601 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:33.602 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306520+0000", 2024-09-21T21:41:33.602 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3700000000000001e-07 2024-09-21T21:41:33.602 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.602 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.602 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:33.602 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306530+0000", 2024-09-21T21:41:33.602 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0336000000000001e-05 2024-09-21T21:41:33.602 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.602 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.602 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:33.602 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306538+0000", 2024-09-21T21:41:33.602 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.3690000000000001e-06 2024-09-21T21:41:33.602 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.602 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.602 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:33.603 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306543+0000", 2024-09-21T21:41:33.603 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8930000000000004e-06 2024-09-21T21:41:33.603 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.603 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.603 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:33.603 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338005+0000", 2024-09-21T21:41:33.603 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031461973999999997 2024-09-21T21:41:33.603 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.603 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.603 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:33.603 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338039+0000", 2024-09-21T21:41:33.603 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4545999999999998e-05 2024-09-21T21:41:33.603 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.603 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.603 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:33.604 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338091+0000", 2024-09-21T21:41:33.604 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1409999999999997e-05 2024-09-21T21:41:33.604 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:33.604 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:33.604 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:33.604 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.604 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.604 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:2014 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 9699328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:33.604 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.306553+0000", 2024-09-21T21:41:33.604 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.2564623729999997, 2024-09-21T21:41:33.604 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031642225000000003, 2024-09-21T21:41:33.604 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:33.604 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:33.604 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:33.604 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:33.605 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:33.605 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:33.605 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2014 2024-09-21T21:41:33.605 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.605 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:33.605 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.605 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:33.605 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306553+0000", 2024-09-21T21:41:33.605 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:33.605 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.605 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.605 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:33.605 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306553+0000", 2024-09-21T21:41:33.605 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:33.605 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.606 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.606 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:33.606 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306553+0000", 2024-09-21T21:41:33.606 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5600000000000001e-07 2024-09-21T21:41:33.606 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.606 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.606 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:33.606 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306555+0000", 2024-09-21T21:41:33.606 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3070000000000001e-06 2024-09-21T21:41:33.606 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.606 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.606 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:33.606 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306561+0000", 2024-09-21T21:41:33.606 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8860000000000001e-06 2024-09-21T21:41:33.607 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.607 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.607 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:33.607 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306566+0000", 2024-09-21T21:41:33.607 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3420000000000001e-06 2024-09-21T21:41:33.607 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.607 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.607 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:33.607 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338115+0000", 2024-09-21T21:41:33.607 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031549333999999998 2024-09-21T21:41:33.607 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.607 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.607 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:33.607 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338150+0000", 2024-09-21T21:41:33.607 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4649e-05 2024-09-21T21:41:33.608 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.608 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.608 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:33.608 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338195+0000", 2024-09-21T21:41:33.608 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5250999999999998e-05 2024-09-21T21:41:33.608 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:33.608 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:33.608 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:33.608 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.608 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.608 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:2015 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 10158080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:33.608 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.306572+0000", 2024-09-21T21:41:33.608 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.2564434549999994, 2024-09-21T21:41:33.608 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031729593, 2024-09-21T21:41:33.608 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:33.609 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:33.609 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:33.609 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:33.609 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:33.609 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:33.609 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2015 2024-09-21T21:41:33.609 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.609 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:33.609 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.609 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:33.609 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306572+0000", 2024-09-21T21:41:33.609 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:33.609 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.609 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.609 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:33.610 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306572+0000", 2024-09-21T21:41:33.610 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:33.610 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.610 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.610 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:33.610 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306572+0000", 2024-09-21T21:41:33.610 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7500000000000001e-07 2024-09-21T21:41:33.610 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.610 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.610 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:33.610 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306574+0000", 2024-09-21T21:41:33.610 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0669999999999999e-06 2024-09-21T21:41:33.610 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.610 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.611 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:33.611 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306579+0000", 2024-09-21T21:41:33.611 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4009999999999996e-06 2024-09-21T21:41:33.611 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.611 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.611 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:33.611 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306585+0000", 2024-09-21T21:41:33.611 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3349999999999997e-06 2024-09-21T21:41:33.611 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.611 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.611 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:33.611 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338217+0000", 2024-09-21T21:41:33.611 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031631805999999998 2024-09-21T21:41:33.611 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.611 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.612 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:33.612 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338251+0000", 2024-09-21T21:41:33.612 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4813e-05 2024-09-21T21:41:33.612 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.612 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.612 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:33.612 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338301+0000", 2024-09-21T21:41:33.612 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9796000000000002e-05 2024-09-21T21:41:33.612 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:33.612 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:33.612 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:33.612 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.612 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.612 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:2016 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 10616832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:33.612 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.306601+0000", 2024-09-21T21:41:33.613 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.2564142819999997, 2024-09-21T21:41:33.613 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031801840999999997, 2024-09-21T21:41:33.613 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:33.613 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:33.613 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:33.613 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:33.613 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:33.613 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:33.613 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2016 2024-09-21T21:41:33.613 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.613 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:33.613 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.613 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:33.613 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306601+0000", 2024-09-21T21:41:33.613 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:33.614 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.614 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.614 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:33.614 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306601+0000", 2024-09-21T21:41:33.614 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:33.614 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.614 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.614 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:33.614 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306601+0000", 2024-09-21T21:41:33.614 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1900000000000003e-07 2024-09-21T21:41:33.614 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.614 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.614 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:33.614 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306603+0000", 2024-09-21T21:41:33.614 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.17e-06 2024-09-21T21:41:33.615 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.615 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.615 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:33.615 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306609+0000", 2024-09-21T21:41:33.615 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8069999999999996e-06 2024-09-21T21:41:33.615 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.615 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.615 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:33.615 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306614+0000", 2024-09-21T21:41:33.615 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5090000000000004e-06 2024-09-21T21:41:33.615 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.615 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.615 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:33.615 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338325+0000", 2024-09-21T21:41:33.615 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031711237000000003 2024-09-21T21:41:33.616 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.616 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.616 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:33.616 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338359+0000", 2024-09-21T21:41:33.616 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3810000000000003e-05 2024-09-21T21:41:33.616 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.616 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.616 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:33.616 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338403+0000", 2024-09-21T21:41:33.616 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3788999999999997e-05 2024-09-21T21:41:33.616 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:33.616 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:33.616 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:33.616 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.616 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.617 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:2017 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 11075584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:33.617 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.306621+0000", 2024-09-21T21:41:33.617 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.2563945160000003, 2024-09-21T21:41:33.617 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031891271999999998, 2024-09-21T21:41:33.617 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:33.617 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:33.617 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:33.617 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:33.617 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:33.617 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:33.617 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2017 2024-09-21T21:41:33.617 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.617 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:33.617 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.618 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:33.618 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306621+0000", 2024-09-21T21:41:33.618 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:33.618 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.618 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.618 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:33.618 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306621+0000", 2024-09-21T21:41:33.618 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:33.618 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.618 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.618 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:33.618 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306621+0000", 2024-09-21T21:41:33.618 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5400000000000002e-07 2024-09-21T21:41:33.618 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.619 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.619 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:33.619 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306623+0000", 2024-09-21T21:41:33.619 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1100000000000001e-06 2024-09-21T21:41:33.619 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.619 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.621 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:33.622 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306629+0000", 2024-09-21T21:41:33.622 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7810000000000002e-06 2024-09-21T21:41:33.622 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.622 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.622 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:33.622 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306633+0000", 2024-09-21T21:41:33.622 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5589999999999999e-06 2024-09-21T21:41:33.622 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.622 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.622 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:33.622 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338427+0000", 2024-09-21T21:41:33.622 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031793948000000002 2024-09-21T21:41:33.622 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.622 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.623 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:33.623 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338462+0000", 2024-09-21T21:41:33.623 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4785e-05 2024-09-21T21:41:33.623 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.623 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.623 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:33.623 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338512+0000", 2024-09-21T21:41:33.623 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9734999999999997e-05 2024-09-21T21:41:33.623 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:33.623 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:33.623 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:33.623 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.623 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.624 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:2018 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 11534336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:33.624 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.306639+0000", 2024-09-21T21:41:33.624 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.2563758870000008, 2024-09-21T21:41:33.624 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031963189000000003, 2024-09-21T21:41:33.624 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:33.624 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:33.624 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:33.624 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:33.624 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:33.624 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:33.624 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2018 2024-09-21T21:41:33.624 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.624 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:33.624 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.625 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:33.625 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306639+0000", 2024-09-21T21:41:33.625 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:33.625 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.625 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.625 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:33.625 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306639+0000", 2024-09-21T21:41:33.625 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:33.625 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.625 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.625 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:33.625 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306640+0000", 2024-09-21T21:41:33.625 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4600000000000005e-07 2024-09-21T21:41:33.625 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.625 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.626 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:33.626 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306642+0000", 2024-09-21T21:41:33.626 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.916e-06 2024-09-21T21:41:33.626 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.626 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.626 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:33.626 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306647+0000", 2024-09-21T21:41:33.626 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7590000000000003e-06 2024-09-21T21:41:33.626 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.626 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.626 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:33.626 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306652+0000", 2024-09-21T21:41:33.626 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4580000000000003e-06 2024-09-21T21:41:33.626 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.627 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.627 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:33.627 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338524+0000", 2024-09-21T21:41:33.627 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031872172999999997 2024-09-21T21:41:33.627 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.627 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.627 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:33.627 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338557+0000", 2024-09-21T21:41:33.627 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3136000000000003e-05 2024-09-21T21:41:33.627 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.627 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.627 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:33.627 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338602+0000", 2024-09-21T21:41:33.627 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5201000000000003e-05 2024-09-21T21:41:33.627 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:33.628 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:33.628 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:33.628 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.628 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.628 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:2019 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 11993088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:33.628 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.306658+0000", 2024-09-21T21:41:33.628 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.2563574929999994, 2024-09-21T21:41:33.628 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.032059007, 2024-09-21T21:41:33.628 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:33.628 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:33.628 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:33.628 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:33.628 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:33.628 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:33.629 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2019 2024-09-21T21:41:33.629 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.629 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:33.629 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.629 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:33.629 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306658+0000", 2024-09-21T21:41:33.629 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:33.629 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.629 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.629 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:33.629 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306658+0000", 2024-09-21T21:41:33.629 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:33.629 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.629 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.629 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:33.630 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.307950+0000", 2024-09-21T21:41:33.630 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.001292199 2024-09-21T21:41:33.630 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.630 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.630 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:33.630 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.307954+0000", 2024-09-21T21:41:33.630 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5689999999999998e-06 2024-09-21T21:41:33.630 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.630 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.630 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:33.630 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.307964+0000", 2024-09-21T21:41:33.630 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6749999999999997e-06 2024-09-21T21:41:33.630 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.630 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.631 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:33.631 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.307971+0000", 2024-09-21T21:41:33.631 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5189999999999998e-06 2024-09-21T21:41:33.631 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.631 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.631 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:33.631 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338625+0000", 2024-09-21T21:41:33.631 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.030654569 2024-09-21T21:41:33.631 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.631 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.631 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:33.631 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338660+0000", 2024-09-21T21:41:33.631 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4656999999999997e-05 2024-09-21T21:41:33.631 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.631 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.632 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:33.632 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338717+0000", 2024-09-21T21:41:33.632 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6818999999999999e-05 2024-09-21T21:41:33.632 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:33.632 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:33.632 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:33.632 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.632 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.632 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:2026 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 15204352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:33.632 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.308109+0000", 2024-09-21T21:41:33.632 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.2549058930000001, 2024-09-21T21:41:33.632 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031315883000000003, 2024-09-21T21:41:33.632 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:33.632 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:33.632 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:33.633 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:33.633 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:33.633 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:33.633 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2026 2024-09-21T21:41:33.633 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.633 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:33.633 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.633 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:33.633 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308109+0000", 2024-09-21T21:41:33.633 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:33.633 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.633 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.633 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:33.633 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308109+0000", 2024-09-21T21:41:33.633 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:33.634 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.634 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.634 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:33.634 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308110+0000", 2024-09-21T21:41:33.634 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.96e-07 2024-09-21T21:41:33.634 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.634 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.634 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:33.634 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308112+0000", 2024-09-21T21:41:33.634 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0269999999999998e-06 2024-09-21T21:41:33.634 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.634 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.634 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:33.634 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308117+0000", 2024-09-21T21:41:33.635 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.631e-06 2024-09-21T21:41:33.635 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.635 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.635 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:33.635 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308122+0000", 2024-09-21T21:41:33.635 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7310000000000001e-06 2024-09-21T21:41:33.635 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.635 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.635 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:33.635 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.339341+0000", 2024-09-21T21:41:33.635 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031218610000000001 2024-09-21T21:41:33.635 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.635 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.635 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:33.636 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.339376+0000", 2024-09-21T21:41:33.636 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5305000000000002e-05 2024-09-21T21:41:33.636 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.636 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.636 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:33.636 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.339425+0000", 2024-09-21T21:41:33.636 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9183e-05 2024-09-21T21:41:33.636 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:33.636 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:33.636 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:33.636 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.636 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.636 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:2027 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 15663104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:33.636 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.308128+0000", 2024-09-21T21:41:33.636 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.2548873569999994, 2024-09-21T21:41:33.637 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031387452000000003, 2024-09-21T21:41:33.637 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:33.637 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:33.637 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:33.637 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:33.637 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:33.637 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:33.637 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2027 2024-09-21T21:41:33.637 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.637 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:33.637 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.637 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:33.637 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308128+0000", 2024-09-21T21:41:33.637 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:33.637 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.638 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.638 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:33.638 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308128+0000", 2024-09-21T21:41:33.638 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:33.638 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.638 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.638 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:33.638 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308128+0000", 2024-09-21T21:41:33.638 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4700000000000002e-07 2024-09-21T21:41:33.638 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.638 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.638 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:33.638 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308130+0000", 2024-09-21T21:41:33.638 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.8670000000000001e-06 2024-09-21T21:41:33.639 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.639 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.639 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:33.639 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308136+0000", 2024-09-21T21:41:33.639 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0299999999999999e-06 2024-09-21T21:41:33.639 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.639 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.639 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:33.639 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308140+0000", 2024-09-21T21:41:33.639 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3830000000000002e-06 2024-09-21T21:41:33.639 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.639 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.639 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:33.639 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.339437+0000", 2024-09-21T21:41:33.640 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031296864000000001 2024-09-21T21:41:33.640 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.640 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.640 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:33.640 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.339471+0000", 2024-09-21T21:41:33.640 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3302000000000001e-05 2024-09-21T21:41:33.640 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.640 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.640 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:33.640 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.339515+0000", 2024-09-21T21:41:33.640 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4659000000000001e-05 2024-09-21T21:41:33.640 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:33.640 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:33.640 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:33.640 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.640 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.641 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:2028 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 16121856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:33.641 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.308147+0000", 2024-09-21T21:41:33.641 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.2548684300000001, 2024-09-21T21:41:33.641 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031474477000000001, 2024-09-21T21:41:33.641 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:33.641 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:33.641 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:33.641 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:33.641 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:33.641 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:33.641 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2028 2024-09-21T21:41:33.641 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.641 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:33.641 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.641 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:33.642 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308147+0000", 2024-09-21T21:41:33.642 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:33.642 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.642 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.642 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:33.642 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308147+0000", 2024-09-21T21:41:33.642 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:33.642 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.642 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.642 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:33.642 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308147+0000", 2024-09-21T21:41:33.642 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1900000000000003e-07 2024-09-21T21:41:33.642 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.642 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.643 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:33.643 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308149+0000", 2024-09-21T21:41:33.643 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9479999999999998e-06 2024-09-21T21:41:33.643 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.643 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.643 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:33.643 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308155+0000", 2024-09-21T21:41:33.643 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6200000000000004e-06 2024-09-21T21:41:33.643 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.643 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.643 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:33.643 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308160+0000", 2024-09-21T21:41:33.643 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0799999999999996e-06 2024-09-21T21:41:33.643 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.644 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.644 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:33.644 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.339539+0000", 2024-09-21T21:41:33.644 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031378902 2024-09-21T21:41:33.644 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.644 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.644 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:33.644 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.339574+0000", 2024-09-21T21:41:33.644 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5176999999999999e-05 2024-09-21T21:41:33.644 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.644 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.644 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:33.644 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.339621+0000", 2024-09-21T21:41:33.644 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7231000000000002e-05 2024-09-21T21:41:33.644 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:33.645 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:33.645 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:33.645 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.645 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.645 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:2029 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 16580608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:33.645 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.308167+0000", 2024-09-21T21:41:33.645 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.2548480410000007, 2024-09-21T21:41:33.645 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031555582999999998, 2024-09-21T21:41:33.645 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:33.645 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:33.645 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:33.645 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:33.645 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:33.645 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:33.645 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2029 2024-09-21T21:41:33.646 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.646 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:33.646 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.646 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:33.646 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308167+0000", 2024-09-21T21:41:33.646 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:33.646 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.646 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.646 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:33.646 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308167+0000", 2024-09-21T21:41:33.646 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:33.646 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.646 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.646 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:33.647 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308167+0000", 2024-09-21T21:41:33.647 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6600000000000002e-07 2024-09-21T21:41:33.647 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.647 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.647 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:33.647 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308169+0000", 2024-09-21T21:41:33.647 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9819999999999998e-06 2024-09-21T21:41:33.647 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.647 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.647 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:33.647 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308175+0000", 2024-09-21T21:41:33.647 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8289999999999996e-06 2024-09-21T21:41:33.647 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.647 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.648 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:33.648 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308180+0000", 2024-09-21T21:41:33.648 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6439999999999999e-06 2024-09-21T21:41:33.648 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.648 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.648 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:33.648 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.339645+0000", 2024-09-21T21:41:33.648 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031465083999999997 2024-09-21T21:41:33.648 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.648 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.648 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:33.648 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.339679+0000", 2024-09-21T21:41:33.648 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3572999999999999e-05 2024-09-21T21:41:33.648 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.649 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.649 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:33.649 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.339723+0000", 2024-09-21T21:41:33.649 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4104999999999999e-05 2024-09-21T21:41:33.649 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:33.649 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:33.649 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:33.649 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.649 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.649 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:2059 3.5 3:a6b41377:::rbd_data.11213654c685.0000000000000008:head [stat,truncate 13631488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:33.649 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.308186+0000", 2024-09-21T21:41:33.649 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.2548294630000001, 2024-09-21T21:41:33.649 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.032997110000000003, 2024-09-21T21:41:33.649 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:33.649 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:33.650 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:41:33.650 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:33.650 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:33.650 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:33.650 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2059 2024-09-21T21:41:33.650 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.650 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:33.650 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.650 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:33.650 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308186+0000", 2024-09-21T21:41:33.650 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:33.650 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.650 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.651 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:33.651 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308186+0000", 2024-09-21T21:41:33.651 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:33.651 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.651 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.651 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:33.651 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308186+0000", 2024-09-21T21:41:33.651 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1899999999999998e-07 2024-09-21T21:41:33.651 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.651 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.651 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:33.651 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308188+0000", 2024-09-21T21:41:33.651 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0099999999999998e-06 2024-09-21T21:41:33.651 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.652 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.652 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:33.652 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308195+0000", 2024-09-21T21:41:33.652 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0589999999999997e-06 2024-09-21T21:41:33.652 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.652 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.652 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:33.652 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308199+0000", 2024-09-21T21:41:33.652 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2930000000000002e-06 2024-09-21T21:41:33.652 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.652 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.652 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:33.652 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.339745+0000", 2024-09-21T21:41:33.652 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031546091999999998 2024-09-21T21:41:33.652 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.653 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.653 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:33.653 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.339782+0000", 2024-09-21T21:41:33.653 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6847999999999997e-05 2024-09-21T21:41:33.653 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.653 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.653 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:41:33.653 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.339856+0000", 2024-09-21T21:41:33.653 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.3992999999999994e-05 2024-09-21T21:41:33.653 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.653 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.653 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:41:33.653 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.341013+0000", 2024-09-21T21:41:33.653 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00115642 2024-09-21T21:41:33.654 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.654 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.654 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:41:33.654 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.341138+0000", 2024-09-21T21:41:33.654 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00012540699999999999 2024-09-21T21:41:33.654 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.654 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.654 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:41:33.654 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.341164+0000", 2024-09-21T21:41:33.654 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.6168000000000001e-05 2024-09-21T21:41:33.654 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:33.654 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:33.654 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:33.654 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.341183+0000", 2024-09-21T21:41:33.655 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.8400999999999999e-05 2024-09-21T21:41:33.655 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:33.655 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:33.655 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:33.655 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:33.655 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:33.655 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:33.657 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T21:41:33.658 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:33.654+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:33.658 INFO:teuthology.orchestra.run.smithi164.stdout:3376 trunc from 0x6307e00 to 0x497b400 2024-09-21T21:41:33.658 INFO:teuthology.orchestra.run.smithi164.stdout:3380 read 0x1483a00 thru 0x1490545 (0xcb46 bytes) 2024-09-21T21:41:33.658 INFO:teuthology.orchestra.run.smithi164.stdout:3382 write 0x85d6400 thru 0x85db5b1 (0x51b2 bytes) 2024-09-21T21:41:33.658 INFO:teuthology.orchestra.run.smithi164.stdout:3383 read 0x68e9c00 thru 0x68ea8c0 (0xcc1 bytes) 2024-09-21T21:41:33.658 INFO:teuthology.orchestra.run.smithi164.stdout:3385 write 0x976f000 thru 0x977af2a (0xbf2b bytes) 2024-09-21T21:41:33.659 INFO:teuthology.orchestra.run.smithi164.stdout:3386 trunc from 0x977b000 to 0xb258a00 2024-09-21T21:41:33.659 INFO:teuthology.orchestra.run.smithi164.stdout:3387 write 0x9cc4c00 thru 0x9cd3303 (0xe704 bytes) 2024-09-21T21:41:33.659 INFO:teuthology.orchestra.run.smithi164.stdout:3388 write 0x76be800 thru 0x76cb204 (0xca05 bytes) 2024-09-21T21:41:33.659 INFO:teuthology.orchestra.run.smithi164.stdout:3389 read 0xa0b5c00 thru 0xa0b732d (0x172e bytes) 2024-09-21T21:41:33.659 INFO:teuthology.orchestra.run.smithi164.stdout:3391 write 0xa25e200 thru 0xa260d64 (0x2b65 bytes) 2024-09-21T21:41:33.659 INFO:teuthology.orchestra.run.smithi164.stdout:3392 write 0x6f2e400 thru 0x6f3bd1e (0xd91f bytes) 2024-09-21T21:41:33.659 INFO:teuthology.orchestra.run.smithi164.stdout:3393 trunc from 0xb258a00 to 0x26ce600 2024-09-21T21:41:33.695 INFO:teuthology.orchestra.run.smithi081.stderr:nodeep-scrub is unset 2024-09-21T21:41:33.736 INFO:teuthology.orchestra.run.smithi164.stdout:3394 write 0xcf7ac00 thru 0xcf83b86 (0x8f87 bytes) 2024-09-21T21:41:33.751 INFO:teuthology.orchestra.run.smithi164.stdout:3395 trunc from 0xcf83c00 to 0x10e9c00 2024-09-21T21:41:33.757 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:33.756+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:33.795 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:33.795 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:41:33.795 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-21T21:41:33.795 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:33.798 INFO:teuthology.orchestra.run.smithi164.stdout:3401 write 0x2c06a00 thru 0x2c08056 (0x1657 bytes) 2024-09-21T21:41:33.804 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T21:41:33.805 INFO:teuthology.orchestra.run.smithi164.stdout:3402 trunc from 0x2c08200 to 0x7ab3a00 2024-09-21T21:41:33.831 INFO:teuthology.orchestra.run.smithi164.stdout:3403 write 0xa8c9c00 thru 0xa8d418d (0xa58e bytes) 2024-09-21T21:41:33.838 INFO:teuthology.orchestra.run.smithi164.stdout:3407 flatten 2024-09-21T21:41:33.859 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:33.857+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:33.960 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:33.958+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:33.961 INFO:teuthology.orchestra.run.smithi164.stdout:3408 trunc from 0xa8d4200 to 0xd0b8e00 2024-09-21T21:41:33.972 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:33.972 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:41:33.972 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:41:33.972 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:41:33.972 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:33.975 INFO:teuthology.orchestra.run.smithi164.stdout:3410 write 0x2883200 thru 0x28922fa (0xf0fb bytes) 2024-09-21T21:41:33.975 INFO:teuthology.orchestra.run.smithi164.stdout:3411 read 0x4ba0400 thru 0x4badb0b (0xd70c bytes) 2024-09-21T21:41:33.975 INFO:teuthology.orchestra.run.smithi164.stdout:3414 write 0x36c3200 thru 0x36d20e0 (0xeee1 bytes) 2024-09-21T21:41:33.975 INFO:teuthology.orchestra.run.smithi164.stdout:3417 read 0x6ca6000 thru 0x6cadb94 (0x7b95 bytes) 2024-09-21T21:41:33.976 INFO:teuthology.orchestra.run.smithi164.stdout:3419 write 0x96f1000 thru 0x96f4d77 (0x3d78 bytes) 2024-09-21T21:41:33.976 INFO:teuthology.orchestra.run.smithi164.stdout:3422 read 0x4d10800 thru 0x4d15598 (0x4d99 bytes) 2024-09-21T21:41:33.976 INFO:teuthology.orchestra.run.smithi164.stdout:3424 write 0x57eae00 thru 0x57f3550 (0x8751 bytes) 2024-09-21T21:41:33.976 INFO:teuthology.orchestra.run.smithi164.stdout:3426 punch from 0x5ad3200 to 0x5adcc00, (0x9a00 bytes) 2024-09-21T21:41:33.976 INFO:teuthology.orchestra.run.smithi164.stdout:3428 read 0x655c800 thru 0x655ce33 (0x634 bytes) 2024-09-21T21:41:33.976 INFO:teuthology.orchestra.run.smithi164.stdout:3433 read 0x6b1b600 thru 0x6b213c9 (0x5dca bytes) 2024-09-21T21:41:33.976 INFO:teuthology.orchestra.run.smithi164.stdout:3434 read 0xbc0a600 thru 0xbc1a158 (0xfb59 bytes) 2024-09-21T21:41:33.977 INFO:teuthology.orchestra.run.smithi164.stdout:3435 clone 25 order 24 su 1048576 sc 9 2024-09-21T21:41:33.983 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T21:41:34.061 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:34.059+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:34.091 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:34.091 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-21T21:41:34.091 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-21T21:41:34.091 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:41:34.091 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.091 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2617 3.d 3:b26310db:::rbd_data.112145fae779.0000000000000006:a [read 3686400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:34.091 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.411596+0000", 2024-09-21T21:41:34.092 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 35.676893579000001, 2024-09-21T21:41:34.092 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.045094515000000002, 2024-09-21T21:41:34.092 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:34.092 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:34.092 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:34.092 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:34.092 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:34.092 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:34.092 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2617 2024-09-21T21:41:34.092 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.092 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:34.092 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.092 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:34.093 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411596+0000", 2024-09-21T21:41:34.093 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:34.093 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.093 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.093 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:34.093 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411596+0000", 2024-09-21T21:41:34.093 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:34.093 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.093 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.093 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:34.093 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411596+0000", 2024-09-21T21:41:34.093 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3500000000000002e-07 2024-09-21T21:41:34.093 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.093 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.094 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:34.094 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411598+0000", 2024-09-21T21:41:34.094 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.871e-06 2024-09-21T21:41:34.094 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.094 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.094 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:34.094 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411603+0000", 2024-09-21T21:41:34.094 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6420000000000004e-06 2024-09-21T21:41:34.094 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.094 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.094 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:34.094 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411608+0000", 2024-09-21T21:41:34.094 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4780000000000002e-06 2024-09-21T21:41:34.094 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.095 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.095 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:34.095 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.456529+0000", 2024-09-21T21:41:34.095 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044921232999999998 2024-09-21T21:41:34.095 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.095 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.095 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:34.095 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.456616+0000", 2024-09-21T21:41:34.095 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6712000000000006e-05 2024-09-21T21:41:34.095 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.095 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.095 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:34.095 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.456690+0000", 2024-09-21T21:41:34.096 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4244000000000003e-05 2024-09-21T21:41:34.096 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:34.096 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:34.096 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:34.096 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.096 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.096 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2618 3.d 3:b26310db:::rbd_data.112145fae779.0000000000000006:a [read 3743744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:34.096 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.411613+0000", 2024-09-21T21:41:34.096 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 35.676876, 2024-09-21T21:41:34.096 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.045180423999999997, 2024-09-21T21:41:34.096 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:34.096 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:34.096 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:34.096 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:34.096 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:34.097 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:34.097 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2618 2024-09-21T21:41:34.097 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.097 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:34.097 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.097 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:34.097 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411613+0000", 2024-09-21T21:41:34.097 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:34.097 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.097 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.097 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:34.097 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411613+0000", 2024-09-21T21:41:34.097 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:34.097 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.098 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.098 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:34.098 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.412868+0000", 2024-09-21T21:41:34.098 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012549 2024-09-21T21:41:34.098 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.098 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.098 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:34.098 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.412873+0000", 2024-09-21T21:41:34.098 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9060000000000001e-06 2024-09-21T21:41:34.098 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.098 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.098 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:34.098 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.412884+0000", 2024-09-21T21:41:34.098 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1164e-05 2024-09-21T21:41:34.099 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.099 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.099 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:34.099 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.412892+0000", 2024-09-21T21:41:34.099 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5719999999999997e-06 2024-09-21T21:41:34.099 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.099 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.099 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:34.099 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.456717+0000", 2024-09-21T21:41:34.099 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043825559 2024-09-21T21:41:34.099 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.099 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.099 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:34.099 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.456753+0000", 2024-09-21T21:41:34.100 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6158000000000002e-05 2024-09-21T21:41:34.100 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.100 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.100 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:34.100 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.456794+0000", 2024-09-21T21:41:34.100 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0164999999999999e-05 2024-09-21T21:41:34.100 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:34.100 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:34.100 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:34.100 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.101 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.101 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5018.0:2937 3.15 3:af2e3e9a:::rbd_data.11213fc02c9d.0000000000000000:19 [read 475136~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T21:41:34.101 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:33.865486+0000", 2024-09-21T21:41:34.101 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.22300318399999999, 2024-09-21T21:41:34.101 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047419950000000002, 2024-09-21T21:41:34.101 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:34.101 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:34.101 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:34.101 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:34.101 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5018", 2024-09-21T21:41:34.102 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3046875242", 2024-09-21T21:41:34.102 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2937 2024-09-21T21:41:34.102 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.102 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:34.102 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.102 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:34.102 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865486+0000", 2024-09-21T21:41:34.102 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:34.102 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.102 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.102 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:34.102 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865486+0000", 2024-09-21T21:41:34.103 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:34.103 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.103 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.103 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:34.103 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865486+0000", 2024-09-21T21:41:34.103 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1800000000000001e-07 2024-09-21T21:41:34.103 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.103 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.103 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:34.103 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865488+0000", 2024-09-21T21:41:34.103 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9230000000000001e-06 2024-09-21T21:41:34.103 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.103 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.104 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:34.104 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865496+0000", 2024-09-21T21:41:34.104 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6260000000000003e-06 2024-09-21T21:41:34.104 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.104 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.104 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:34.104 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865501+0000", 2024-09-21T21:41:34.104 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7620000000000003e-06 2024-09-21T21:41:34.104 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.104 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.104 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:34.104 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.912653+0000", 2024-09-21T21:41:34.104 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047152816 2024-09-21T21:41:34.104 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.104 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.104 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:34.105 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.912768+0000", 2024-09-21T21:41:34.105 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000114558 2024-09-21T21:41:34.105 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.105 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.105 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:34.105 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.912906+0000", 2024-09-21T21:41:34.105 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000137847 2024-09-21T21:41:34.105 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:34.105 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:34.105 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:34.105 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.105 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.105 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5018.0:2938 3.15 3:af2e3e9a:::rbd_data.11213fc02c9d.0000000000000000:19 [read 417792~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T21:41:34.105 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:33.865505+0000", 2024-09-21T21:41:34.106 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.222984448, 2024-09-21T21:41:34.106 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047654654999999997, 2024-09-21T21:41:34.106 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:34.106 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:34.106 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:34.106 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:34.106 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5018", 2024-09-21T21:41:34.106 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3046875242", 2024-09-21T21:41:34.106 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2938 2024-09-21T21:41:34.106 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.106 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:34.106 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.106 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:34.106 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865505+0000", 2024-09-21T21:41:34.107 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:34.107 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.107 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.107 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:34.107 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865505+0000", 2024-09-21T21:41:34.107 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:34.107 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.107 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.107 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:34.107 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865505+0000", 2024-09-21T21:41:34.107 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2999999999999999e-07 2024-09-21T21:41:34.107 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.107 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.108 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:34.108 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865506+0000", 2024-09-21T21:41:34.108 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2979999999999999e-06 2024-09-21T21:41:34.108 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.108 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.108 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:34.108 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865509+0000", 2024-09-21T21:41:34.108 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.163e-06 2024-09-21T21:41:34.108 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.108 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.108 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:34.108 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865512+0000", 2024-09-21T21:41:34.108 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7429999999999998e-06 2024-09-21T21:41:34.108 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.109 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.109 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:34.109 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.912962+0000", 2024-09-21T21:41:34.109 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047450349000000003 2024-09-21T21:41:34.109 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.109 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.109 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:34.109 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.913054+0000", 2024-09-21T21:41:34.109 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.1384999999999993e-05 2024-09-21T21:41:34.109 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.110 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.110 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:34.110 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.913159+0000", 2024-09-21T21:41:34.110 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000105487 2024-09-21T21:41:34.110 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:34.110 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:34.110 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:34.110 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.110 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.110 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5018.0:2939 3.15 3:af2e3e9a:::rbd_data.11213fc02c9d.0000000000000000:19 [read 425984~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T21:41:34.111 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:33.865515+0000", 2024-09-21T21:41:34.111 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.22297394800000001, 2024-09-21T21:41:34.111 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047853433000000001, 2024-09-21T21:41:34.111 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:34.111 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:34.111 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:34.111 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:34.111 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5018", 2024-09-21T21:41:34.111 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3046875242", 2024-09-21T21:41:34.111 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2939 2024-09-21T21:41:34.111 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.112 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:34.112 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.112 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:34.112 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865515+0000", 2024-09-21T21:41:34.112 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:34.112 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.112 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.112 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:34.112 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865515+0000", 2024-09-21T21:41:34.112 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:34.112 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.112 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.112 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:34.112 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865516+0000", 2024-09-21T21:41:34.113 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6800000000000001e-07 2024-09-21T21:41:34.113 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.113 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.113 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:34.113 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865517+0000", 2024-09-21T21:41:34.113 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.083e-06 2024-09-21T21:41:34.113 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.113 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.113 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:34.113 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865520+0000", 2024-09-21T21:41:34.113 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0699999999999998e-06 2024-09-21T21:41:34.113 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.113 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.113 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:34.114 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865523+0000", 2024-09-21T21:41:34.114 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.959e-06 2024-09-21T21:41:34.114 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.114 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.114 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:34.114 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.913199+0000", 2024-09-21T21:41:34.114 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047676653999999999 2024-09-21T21:41:34.114 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.114 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.114 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:34.114 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.913275+0000", 2024-09-21T21:41:34.114 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6154000000000003e-05 2024-09-21T21:41:34.115 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.115 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.115 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:34.115 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.913369+0000", 2024-09-21T21:41:34.115 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3145000000000004e-05 2024-09-21T21:41:34.115 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:34.115 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:34.115 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:34.115 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.115 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.115 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5018.0:2940 3.15 3:af2e3e9a:::rbd_data.11213fc02c9d.0000000000000000:19 [read 434176~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T21:41:34.115 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:33.865528+0000", 2024-09-21T21:41:34.116 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.22296149400000001, 2024-09-21T21:41:34.116 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.048052481000000001, 2024-09-21T21:41:34.116 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:34.116 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:34.116 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:34.116 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:34.116 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5018", 2024-09-21T21:41:34.116 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3046875242", 2024-09-21T21:41:34.116 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2940 2024-09-21T21:41:34.116 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.116 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:34.116 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.116 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:34.116 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865528+0000", 2024-09-21T21:41:34.117 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:34.117 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.117 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.117 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:34.117 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865528+0000", 2024-09-21T21:41:34.117 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:34.117 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.117 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.117 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:34.117 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865528+0000", 2024-09-21T21:41:34.117 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3500000000000002e-07 2024-09-21T21:41:34.117 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.117 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.117 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:34.117 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865530+0000", 2024-09-21T21:41:34.118 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1689999999999999e-06 2024-09-21T21:41:34.118 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.118 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.118 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:34.118 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865536+0000", 2024-09-21T21:41:34.118 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8560000000000003e-06 2024-09-21T21:41:34.118 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.118 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.118 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:34.118 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865541+0000", 2024-09-21T21:41:34.118 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.899e-06 2024-09-21T21:41:34.118 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.118 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.119 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:34.119 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.913406+0000", 2024-09-21T21:41:34.119 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047865216000000002 2024-09-21T21:41:34.119 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.119 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.119 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:34.119 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.913483+0000", 2024-09-21T21:41:34.119 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6525000000000003e-05 2024-09-21T21:41:34.119 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.119 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.120 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:34.120 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.913580+0000", 2024-09-21T21:41:34.120 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.7480999999999997e-05 2024-09-21T21:41:34.120 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:34.120 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:34.120 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:34.120 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.120 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.120 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5018.0:2941 3.15 3:af2e3e9a:::rbd_data.11213fc02c9d.0000000000000000:19 [read 483328~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T21:41:34.120 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:33.865558+0000", 2024-09-21T21:41:34.120 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.22293122500000001, 2024-09-21T21:41:34.120 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.048220161999999997, 2024-09-21T21:41:34.120 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:34.120 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:34.121 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:34.121 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:34.121 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5018", 2024-09-21T21:41:34.121 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3046875242", 2024-09-21T21:41:34.121 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2941 2024-09-21T21:41:34.121 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.121 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:34.121 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.121 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:34.121 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865558+0000", 2024-09-21T21:41:34.121 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:34.121 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.121 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.121 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:34.122 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865558+0000", 2024-09-21T21:41:34.122 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:34.122 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.122 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.122 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:34.122 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865558+0000", 2024-09-21T21:41:34.122 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1799999999999995e-07 2024-09-21T21:41:34.122 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.122 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.122 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:34.122 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865561+0000", 2024-09-21T21:41:34.122 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.317e-06 2024-09-21T21:41:34.122 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.122 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.123 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:34.123 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865568+0000", 2024-09-21T21:41:34.123 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8510000000000001e-06 2024-09-21T21:41:34.123 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.123 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.123 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:34.123 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865573+0000", 2024-09-21T21:41:34.123 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1730000000000003e-06 2024-09-21T21:41:34.123 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.123 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.123 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:34.123 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.913617+0000", 2024-09-21T21:41:34.123 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.048044302999999997 2024-09-21T21:41:34.123 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.124 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.124 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:34.124 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.913689+0000", 2024-09-21T21:41:34.124 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1658999999999999e-05 2024-09-21T21:41:34.124 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.124 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.124 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:34.124 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.913778+0000", 2024-09-21T21:41:34.124 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9340999999999997e-05 2024-09-21T21:41:34.124 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:34.124 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:34.124 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:34.124 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.124 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.124 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5018.0:2942 3.15 3:af2e3e9a:::rbd_data.11213fc02c9d.0000000000000000:19 [read 491520~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T21:41:34.125 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:33.865580+0000", 2024-09-21T21:41:34.125 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.222908726, 2024-09-21T21:41:34.125 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.048410436000000001, 2024-09-21T21:41:34.125 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:34.125 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:34.125 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:34.125 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:34.125 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5018", 2024-09-21T21:41:34.125 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3046875242", 2024-09-21T21:41:34.125 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2942 2024-09-21T21:41:34.125 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.125 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:34.125 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.125 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:34.126 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865580+0000", 2024-09-21T21:41:34.126 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:34.126 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.126 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.126 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:34.126 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865580+0000", 2024-09-21T21:41:34.126 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:34.126 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.126 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.126 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:34.126 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865581+0000", 2024-09-21T21:41:34.126 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.51e-07 2024-09-21T21:41:34.126 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.126 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.127 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:34.127 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865583+0000", 2024-09-21T21:41:34.127 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.12e-06 2024-09-21T21:41:34.127 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.127 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.127 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:34.127 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865589+0000", 2024-09-21T21:41:34.127 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3080000000000004e-06 2024-09-21T21:41:34.127 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.127 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.127 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:34.127 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865594+0000", 2024-09-21T21:41:34.127 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9300000000000002e-06 2024-09-21T21:41:34.127 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.128 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.128 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:34.128 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.913813+0000", 2024-09-21T21:41:34.128 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.048218924000000003 2024-09-21T21:41:34.128 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.128 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.128 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:34.128 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.913891+0000", 2024-09-21T21:41:34.128 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7523999999999998e-05 2024-09-21T21:41:34.128 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.128 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.128 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:34.128 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.913991+0000", 2024-09-21T21:41:34.128 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000100179 2024-09-21T21:41:34.129 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:34.129 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:34.129 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:34.129 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.129 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.129 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5018.0:2943 3.15 3:af2e3e9a:::rbd_data.11213fc02c9d.0000000000000000:19 [read 393216~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T21:41:34.129 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:33.865612+0000", 2024-09-21T21:41:34.129 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.22287659100000001, 2024-09-21T21:41:34.129 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.048585616999999998, 2024-09-21T21:41:34.129 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:34.129 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:34.129 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:34.129 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:34.129 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5018", 2024-09-21T21:41:34.129 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3046875242", 2024-09-21T21:41:34.130 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2943 2024-09-21T21:41:34.130 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.130 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:34.130 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.130 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:34.130 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865612+0000", 2024-09-21T21:41:34.130 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:34.130 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.130 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.130 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:34.130 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865612+0000", 2024-09-21T21:41:34.130 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:34.131 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.131 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.131 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:34.131 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865613+0000", 2024-09-21T21:41:34.131 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9200000000000002e-07 2024-09-21T21:41:34.131 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.131 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.131 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:34.131 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865618+0000", 2024-09-21T21:41:34.131 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9729999999999997e-06 2024-09-21T21:41:34.131 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.131 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.131 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:34.131 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865623+0000", 2024-09-21T21:41:34.131 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6970000000000001e-06 2024-09-21T21:41:34.132 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.132 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.132 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:34.132 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865626+0000", 2024-09-21T21:41:34.132 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0879999999999999e-06 2024-09-21T21:41:34.132 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.132 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.132 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:34.132 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.914028+0000", 2024-09-21T21:41:34.132 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.048401921000000001 2024-09-21T21:41:34.132 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.132 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.132 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:34.132 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.914104+0000", 2024-09-21T21:41:34.133 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6312999999999999e-05 2024-09-21T21:41:34.133 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.133 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.133 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:34.133 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.914198+0000", 2024-09-21T21:41:34.133 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.4233000000000004e-05 2024-09-21T21:41:34.133 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:34.133 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:34.133 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:34.133 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.133 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.133 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5018.0:2944 3.15 3:af2e3e9a:::rbd_data.11213fc02c9d.0000000000000000:19 [read 442368~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T21:41:34.133 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:33.865654+0000", 2024-09-21T21:41:34.133 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.22283508799999999, 2024-09-21T21:41:34.133 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.048771901999999999, 2024-09-21T21:41:34.134 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:34.134 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:34.134 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:34.134 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:34.134 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5018", 2024-09-21T21:41:34.134 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3046875242", 2024-09-21T21:41:34.134 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2944 2024-09-21T21:41:34.134 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.134 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:34.134 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.134 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:34.134 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865654+0000", 2024-09-21T21:41:34.134 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:34.134 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.135 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.135 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:34.135 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865654+0000", 2024-09-21T21:41:34.135 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:34.135 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.135 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.135 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:34.135 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865655+0000", 2024-09-21T21:41:34.135 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4199999999999995e-07 2024-09-21T21:41:34.135 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.135 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.135 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:34.135 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865656+0000", 2024-09-21T21:41:34.135 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.8139999999999999e-06 2024-09-21T21:41:34.136 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.136 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.136 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:34.136 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865661+0000", 2024-09-21T21:41:34.136 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6489999999999999e-06 2024-09-21T21:41:34.136 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.136 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.136 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:34.136 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865664+0000", 2024-09-21T21:41:34.136 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2119999999999999e-06 2024-09-21T21:41:34.136 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.136 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.136 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:34.136 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.914236+0000", 2024-09-21T21:41:34.137 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.048571707999999998 2024-09-21T21:41:34.137 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.137 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.137 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:34.137 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.914319+0000", 2024-09-21T21:41:34.137 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2689999999999996e-05 2024-09-21T21:41:34.137 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.137 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.137 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:34.137 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.914426+0000", 2024-09-21T21:41:34.137 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000107187 2024-09-21T21:41:34.137 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:34.137 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:34.137 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:34.137 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.138 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.138 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5018.0:2945 3.15 3:af2e3e9a:::rbd_data.11213fc02c9d.0000000000000000:19 [read 507904~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T21:41:34.138 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:33.865670+0000", 2024-09-21T21:41:34.138 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.22281958199999999, 2024-09-21T21:41:34.138 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.048961808000000002, 2024-09-21T21:41:34.138 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:34.138 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:34.138 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:34.138 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:34.138 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5018", 2024-09-21T21:41:34.138 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3046875242", 2024-09-21T21:41:34.138 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2945 2024-09-21T21:41:34.138 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.138 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:34.139 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.139 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:34.139 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865670+0000", 2024-09-21T21:41:34.139 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:34.139 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.139 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.139 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:34.139 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865670+0000", 2024-09-21T21:41:34.139 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:34.139 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.139 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.139 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:34.139 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865670+0000", 2024-09-21T21:41:34.139 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8400000000000005e-07 2024-09-21T21:41:34.140 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.140 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.140 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:34.140 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865672+0000", 2024-09-21T21:41:34.140 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.691e-06 2024-09-21T21:41:34.140 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.140 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.140 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:34.140 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865676+0000", 2024-09-21T21:41:34.140 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5390000000000001e-06 2024-09-21T21:41:34.140 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.140 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.140 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:34.141 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865680+0000", 2024-09-21T21:41:34.141 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.67e-06 2024-09-21T21:41:34.141 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.141 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.141 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:34.141 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.914460+0000", 2024-09-21T21:41:34.141 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.048780221999999998 2024-09-21T21:41:34.141 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.141 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.141 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:34.141 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.914539+0000", 2024-09-21T21:41:34.141 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8634000000000006e-05 2024-09-21T21:41:34.141 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.141 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.142 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:34.142 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.914631+0000", 2024-09-21T21:41:34.142 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.2567999999999999e-05 2024-09-21T21:41:34.142 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:34.142 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:34.142 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:34.142 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.142 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.142 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5018.0:2946 3.15 3:af2e3e9a:::rbd_data.11213fc02c9d.0000000000000000:19 [read 450560~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T21:41:34.142 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:33.865694+0000", 2024-09-21T21:41:34.142 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.22279465400000001, 2024-09-21T21:41:34.142 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.049163473999999999, 2024-09-21T21:41:34.142 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:34.143 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:34.143 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:34.143 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:34.143 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5018", 2024-09-21T21:41:34.143 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3046875242", 2024-09-21T21:41:34.143 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2946 2024-09-21T21:41:34.143 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.143 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:34.143 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.143 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:34.143 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865694+0000", 2024-09-21T21:41:34.143 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:34.143 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.144 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.144 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:34.144 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865694+0000", 2024-09-21T21:41:34.144 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:34.144 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.144 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.144 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:34.144 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865695+0000", 2024-09-21T21:41:34.144 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3299999999999998e-07 2024-09-21T21:41:34.144 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.144 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.144 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:34.144 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865697+0000", 2024-09-21T21:41:34.144 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.08e-06 2024-09-21T21:41:34.145 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.145 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.145 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:34.145 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865703+0000", 2024-09-21T21:41:34.145 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6840000000000001e-06 2024-09-21T21:41:34.145 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.145 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.145 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:34.145 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865708+0000", 2024-09-21T21:41:34.145 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7690000000000002e-06 2024-09-21T21:41:34.145 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.145 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.145 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:34.146 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.914670+0000", 2024-09-21T21:41:34.146 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.048961849000000002 2024-09-21T21:41:34.146 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.146 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.146 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:34.146 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.914753+0000", 2024-09-21T21:41:34.146 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2596000000000007e-05 2024-09-21T21:41:34.146 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.146 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.146 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:34.146 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.914858+0000", 2024-09-21T21:41:34.146 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000105163 2024-09-21T21:41:34.146 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:34.146 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:34.147 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:34.147 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.147 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.147 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5018.0:2947 3.15 3:af2e3e9a:::rbd_data.11213fc02c9d.0000000000000000:19 [read 499712~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T21:41:34.147 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:33.865714+0000", 2024-09-21T21:41:34.147 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.22277497700000001, 2024-09-21T21:41:34.147 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.049362488000000003, 2024-09-21T21:41:34.147 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:34.147 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:34.147 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:34.147 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:34.147 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5018", 2024-09-21T21:41:34.147 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3046875242", 2024-09-21T21:41:34.147 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2947 2024-09-21T21:41:34.148 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.148 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:34.148 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.148 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:34.148 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865714+0000", 2024-09-21T21:41:34.148 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:34.148 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.148 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.148 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:34.148 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865714+0000", 2024-09-21T21:41:34.148 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:34.148 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.148 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.148 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:34.149 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865714+0000", 2024-09-21T21:41:34.149 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.27e-07 2024-09-21T21:41:34.149 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.149 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.149 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:34.149 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865716+0000", 2024-09-21T21:41:34.149 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.8789999999999999e-06 2024-09-21T21:41:34.149 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.149 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.149 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:34.149 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865722+0000", 2024-09-21T21:41:34.149 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7930000000000003e-06 2024-09-21T21:41:34.149 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.149 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.150 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:34.150 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865728+0000", 2024-09-21T21:41:34.150 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7960000000000001e-06 2024-09-21T21:41:34.150 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.150 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.150 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:34.150 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.914902+0000", 2024-09-21T21:41:34.150 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.049174021999999998 2024-09-21T21:41:34.150 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.150 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.150 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:34.150 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.914983+0000", 2024-09-21T21:41:34.150 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1446999999999995e-05 2024-09-21T21:41:34.150 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.151 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.151 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:34.151 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.915077+0000", 2024-09-21T21:41:34.151 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3224000000000001e-05 2024-09-21T21:41:34.151 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:34.151 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:34.151 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:34.151 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.151 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.151 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5018.0:3004 3.15 3:aa7edf89:::rbd_object_map.1121c78ac9e5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e64)", 2024-09-21T21:41:34.151 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:33.865734+0000", 2024-09-21T21:41:34.151 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.222755435, 2024-09-21T21:41:34.151 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.052869125000000003, 2024-09-21T21:41:34.151 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:34.152 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:34.152 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:41:34.152 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:34.152 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5018", 2024-09-21T21:41:34.152 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3046875242", 2024-09-21T21:41:34.152 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 3004 2024-09-21T21:41:34.152 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.152 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:34.152 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.152 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:34.152 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865734+0000", 2024-09-21T21:41:34.152 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:34.152 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.152 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.153 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:34.153 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865734+0000", 2024-09-21T21:41:34.153 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:34.153 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.153 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.153 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:34.153 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865734+0000", 2024-09-21T21:41:34.153 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0999999999999999e-07 2024-09-21T21:41:34.153 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.153 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.153 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:34.153 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865737+0000", 2024-09-21T21:41:34.153 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6709999999999999e-06 2024-09-21T21:41:34.154 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.154 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.154 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:34.154 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865743+0000", 2024-09-21T21:41:34.154 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9109999999999998e-06 2024-09-21T21:41:34.154 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.154 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.154 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:34.154 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865747+0000", 2024-09-21T21:41:34.154 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4819999999999996e-06 2024-09-21T21:41:34.154 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.154 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.154 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:34.154 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.915114+0000", 2024-09-21T21:41:34.155 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.049367220000000003 2024-09-21T21:41:34.155 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.155 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.155 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:34.155 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.915203+0000", 2024-09-21T21:41:34.155 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9068999999999997e-05 2024-09-21T21:41:34.155 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.155 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.155 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:41:34.155 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.915603+0000", 2024-09-21T21:41:34.155 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00039933999999999998 2024-09-21T21:41:34.155 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.155 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.156 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:41:34.156 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.918432+0000", 2024-09-21T21:41:34.156 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0028295590000000002 2024-09-21T21:41:34.156 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.156 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.156 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:41:34.156 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.918489+0000", 2024-09-21T21:41:34.156 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6866000000000001e-05 2024-09-21T21:41:34.156 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.156 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.156 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:41:34.156 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.918550+0000", 2024-09-21T21:41:34.156 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0557000000000003e-05 2024-09-21T21:41:34.156 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.157 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.157 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:34.157 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.918603+0000", 2024-09-21T21:41:34.157 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.304e-05 2024-09-21T21:41:34.157 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:34.157 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:34.157 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:34.157 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.157 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.157 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5018.0:3005 3.15 3:aa7edf89:::rbd_object_map.1121c78ac9e5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e64)", 2024-09-21T21:41:34.157 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:33.865763+0000", 2024-09-21T21:41:34.157 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.22272571099999999, 2024-09-21T21:41:34.157 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.052947052000000001, 2024-09-21T21:41:34.157 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:34.158 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:34.158 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:41:34.158 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:34.158 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5018", 2024-09-21T21:41:34.158 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3046875242", 2024-09-21T21:41:34.158 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 3005 2024-09-21T21:41:34.158 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.158 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:34.158 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.158 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:34.158 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865763+0000", 2024-09-21T21:41:34.158 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:34.158 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.158 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.159 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:34.159 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865763+0000", 2024-09-21T21:41:34.159 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:34.159 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.159 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.159 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:34.159 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865764+0000", 2024-09-21T21:41:34.159 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8099999999999998e-07 2024-09-21T21:41:34.159 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.159 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.159 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:34.159 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865766+0000", 2024-09-21T21:41:34.159 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6579999999999998e-06 2024-09-21T21:41:34.159 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.160 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.160 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:34.160 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865774+0000", 2024-09-21T21:41:34.160 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5660000000000001e-06 2024-09-21T21:41:34.160 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.160 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.160 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:34.160 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865779+0000", 2024-09-21T21:41:34.160 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9860000000000002e-06 2024-09-21T21:41:34.160 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.160 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.160 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:34.160 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.916225+0000", 2024-09-21T21:41:34.160 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.050445613 2024-09-21T21:41:34.161 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.161 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.161 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:34.161 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.916277+0000", 2024-09-21T21:41:34.161 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2380000000000003e-05 2024-09-21T21:41:34.161 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.161 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.161 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:41:34.161 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.916482+0000", 2024-09-21T21:41:34.161 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00020543699999999999 2024-09-21T21:41:34.161 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.161 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.161 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:41:34.161 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.918649+0000", 2024-09-21T21:41:34.161 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021668070000000002 2024-09-21T21:41:34.162 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.162 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.162 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:41:34.162 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.918658+0000", 2024-09-21T21:41:34.162 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3960000000000006e-06 2024-09-21T21:41:34.162 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.162 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.162 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:41:34.162 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.918686+0000", 2024-09-21T21:41:34.162 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8464000000000001e-05 2024-09-21T21:41:34.162 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.162 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.162 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:34.162 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.918710+0000", 2024-09-21T21:41:34.163 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4363999999999999e-05 2024-09-21T21:41:34.163 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:34.163 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:34.163 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:34.163 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.163 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.163 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5018.0:3006 3.15 3:aa7edf89:::rbd_object_map.1121c78ac9e5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e64)", 2024-09-21T21:41:34.163 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:33.865787+0000", 2024-09-21T21:41:34.163 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.22270206200000001, 2024-09-21T21:41:34.163 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.055611471000000003, 2024-09-21T21:41:34.163 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:34.163 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:34.163 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:41:34.163 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:34.164 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5018", 2024-09-21T21:41:34.164 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3046875242", 2024-09-21T21:41:34.164 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 3006 2024-09-21T21:41:34.164 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.164 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:34.164 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.164 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:34.164 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865787+0000", 2024-09-21T21:41:34.164 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:34.164 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.164 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.164 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:34.164 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865787+0000", 2024-09-21T21:41:34.164 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:34.164 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.165 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.165 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:34.165 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865787+0000", 2024-09-21T21:41:34.165 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6600000000000002e-07 2024-09-21T21:41:34.165 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.165 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.165 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:34.165 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865790+0000", 2024-09-21T21:41:34.165 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4899999999999999e-06 2024-09-21T21:41:34.165 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.165 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.165 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:34.165 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865795+0000", 2024-09-21T21:41:34.165 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.045e-06 2024-09-21T21:41:34.166 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.166 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.166 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:34.166 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865799+0000", 2024-09-21T21:41:34.166 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7890000000000001e-06 2024-09-21T21:41:34.166 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.166 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.166 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:34.166 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.916851+0000", 2024-09-21T21:41:34.166 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.051052202999999997 2024-09-21T21:41:34.166 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.166 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.166 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:34.166 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.916938+0000", 2024-09-21T21:41:34.167 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7083999999999995e-05 2024-09-21T21:41:34.167 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.167 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.167 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:41:34.167 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.917291+0000", 2024-09-21T21:41:34.167 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00035321400000000002 2024-09-21T21:41:34.167 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.167 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.167 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:41:34.167 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.918717+0000", 2024-09-21T21:41:34.167 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001425548 2024-09-21T21:41:34.167 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.167 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.167 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:41:34.168 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.921291+0000", 2024-09-21T21:41:34.168 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025744460000000002 2024-09-21T21:41:34.168 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.168 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.168 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:41:34.168 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.921359+0000", 2024-09-21T21:41:34.168 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8267000000000002e-05 2024-09-21T21:41:34.168 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.168 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.168 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:34.168 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.921398+0000", 2024-09-21T21:41:34.168 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9019e-05 2024-09-21T21:41:34.168 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:34.168 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:34.169 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:34.169 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.169 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.169 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5018.0:3007 3.15 3:aa7edf89:::rbd_object_map.1121c78ac9e5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e64)", 2024-09-21T21:41:34.169 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:33.865803+0000", 2024-09-21T21:41:34.169 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.22268608200000001, 2024-09-21T21:41:34.169 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.055713580999999998, 2024-09-21T21:41:34.169 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:34.169 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:34.169 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:41:34.169 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:34.169 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5018", 2024-09-21T21:41:34.169 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3046875242", 2024-09-21T21:41:34.169 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 3007 2024-09-21T21:41:34.170 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.170 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:34.170 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.170 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:34.170 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865803+0000", 2024-09-21T21:41:34.170 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:34.170 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.170 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.170 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:34.170 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865803+0000", 2024-09-21T21:41:34.170 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:34.170 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.170 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.170 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:34.171 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865803+0000", 2024-09-21T21:41:34.171 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.22e-07 2024-09-21T21:41:34.171 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.171 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.171 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:34.171 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865806+0000", 2024-09-21T21:41:34.171 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3099999999999999e-06 2024-09-21T21:41:34.171 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.171 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.171 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:34.171 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865811+0000", 2024-09-21T21:41:34.171 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3669999999999996e-06 2024-09-21T21:41:34.171 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.171 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.172 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:34.172 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865815+0000", 2024-09-21T21:41:34.172 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2180000000000001e-06 2024-09-21T21:41:34.172 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.172 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.172 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:34.172 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.917759+0000", 2024-09-21T21:41:34.172 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.051943277000000003 2024-09-21T21:41:34.172 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.172 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.172 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:34.172 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.917843+0000", 2024-09-21T21:41:34.172 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.4202999999999993e-05 2024-09-21T21:41:34.172 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.173 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.173 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:41:34.173 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.918148+0000", 2024-09-21T21:41:34.173 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000305524 2024-09-21T21:41:34.173 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.173 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.173 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:41:34.173 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.920498+0000", 2024-09-21T21:41:34.173 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0023499250000000001 2024-09-21T21:41:34.173 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.173 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.173 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:41:34.173 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.921455+0000", 2024-09-21T21:41:34.173 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00095657900000000004 2024-09-21T21:41:34.173 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.174 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.174 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:41:34.174 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.921484+0000", 2024-09-21T21:41:34.174 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9479999999999999e-05 2024-09-21T21:41:34.174 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.174 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.174 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:34.174 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.921517+0000", 2024-09-21T21:41:34.174 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2376000000000003e-05 2024-09-21T21:41:34.174 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:34.174 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:34.174 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:34.174 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.174 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.175 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5018.0:3009 3.15 3:aa7edf89:::rbd_object_map.1121c78ac9e5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-21T21:41:34.175 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:33.865859+0000", 2024-09-21T21:41:34.175 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.22262980099999999, 2024-09-21T21:41:34.175 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.055778333999999999, 2024-09-21T21:41:34.175 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:34.175 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:34.175 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:41:34.175 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:34.175 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5018", 2024-09-21T21:41:34.175 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3046875242", 2024-09-21T21:41:34.175 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 3009 2024-09-21T21:41:34.175 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.175 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:34.175 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.175 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:34.176 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865859+0000", 2024-09-21T21:41:34.176 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:34.176 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.176 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.176 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:34.176 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865859+0000", 2024-09-21T21:41:34.176 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:34.176 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.176 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.176 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:34.176 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865860+0000", 2024-09-21T21:41:34.176 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5600000000000001e-07 2024-09-21T21:41:34.176 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.176 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.177 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:34.177 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865867+0000", 2024-09-21T21:41:34.177 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6890000000000004e-06 2024-09-21T21:41:34.177 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.177 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.177 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:34.177 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865874+0000", 2024-09-21T21:41:34.177 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1e-06 2024-09-21T21:41:34.177 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.177 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.177 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:34.177 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865878+0000", 2024-09-21T21:41:34.177 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6929999999999998e-06 2024-09-21T21:41:34.178 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.178 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.178 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:34.178 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.918751+0000", 2024-09-21T21:41:34.178 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.052873113999999999 2024-09-21T21:41:34.178 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.178 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.178 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:34.178 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.918830+0000", 2024-09-21T21:41:34.178 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8862999999999994e-05 2024-09-21T21:41:34.178 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.178 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.178 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:41:34.178 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.919165+0000", 2024-09-21T21:41:34.179 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00033459599999999999 2024-09-21T21:41:34.179 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.179 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.179 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:41:34.179 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.921232+0000", 2024-09-21T21:41:34.179 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002067255 2024-09-21T21:41:34.179 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.179 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.179 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:41:34.179 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.921560+0000", 2024-09-21T21:41:34.179 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00032807300000000001 2024-09-21T21:41:34.179 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.179 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.179 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:41:34.180 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.921600+0000", 2024-09-21T21:41:34.180 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9421000000000001e-05 2024-09-21T21:41:34.180 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.180 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.180 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:34.180 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.921638+0000", 2024-09-21T21:41:34.180 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8074000000000002e-05 2024-09-21T21:41:34.180 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:34.180 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:34.180 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:34.180 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.180 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.180 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5018.0:3036 3.15 3:aa7edf89:::rbd_object_map.1121c78ac9e5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-21T21:41:34.180 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:33.866205+0000", 2024-09-21T21:41:34.180 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.22228409499999999, 2024-09-21T21:41:34.181 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.057529386000000002, 2024-09-21T21:41:34.181 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:34.181 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:34.181 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:41:34.181 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:34.181 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5018", 2024-09-21T21:41:34.181 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3046875242", 2024-09-21T21:41:34.181 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 3036 2024-09-21T21:41:34.181 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.181 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:34.181 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.181 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:34.181 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.866205+0000", 2024-09-21T21:41:34.181 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:34.181 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.182 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.182 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:34.182 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.866205+0000", 2024-09-21T21:41:34.182 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:34.182 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.182 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.182 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:34.182 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.866206+0000", 2024-09-21T21:41:34.182 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5499999999999997e-07 2024-09-21T21:41:34.182 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.182 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.182 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:34.182 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.866211+0000", 2024-09-21T21:41:34.182 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8389999999999997e-06 2024-09-21T21:41:34.183 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.183 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.183 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:34.183 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.866223+0000", 2024-09-21T21:41:34.183 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2061e-05 2024-09-21T21:41:34.183 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.183 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.183 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:34.183 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.866230+0000", 2024-09-21T21:41:34.183 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7560000000000002e-06 2024-09-21T21:41:34.183 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.183 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.183 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:34.183 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.919617+0000", 2024-09-21T21:41:34.183 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.053387807000000002 2024-09-21T21:41:34.184 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.184 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.184 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:34.184 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.919700+0000", 2024-09-21T21:41:34.184 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2682999999999994e-05 2024-09-21T21:41:34.184 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.184 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.184 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:41:34.184 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.920067+0000", 2024-09-21T21:41:34.184 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000367408 2024-09-21T21:41:34.184 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.184 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.184 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:41:34.184 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.921529+0000", 2024-09-21T21:41:34.184 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0014611839999999999 2024-09-21T21:41:34.185 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.185 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.185 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:41:34.185 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.923641+0000", 2024-09-21T21:41:34.185 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021122670000000001 2024-09-21T21:41:34.185 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.185 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.185 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:41:34.185 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.923696+0000", 2024-09-21T21:41:34.185 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4756000000000002e-05 2024-09-21T21:41:34.185 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.185 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.185 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:34.185 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.923734+0000", 2024-09-21T21:41:34.185 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8770000000000003e-05 2024-09-21T21:41:34.186 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:34.186 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:34.186 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:34.186 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.186 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.186 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5018.0:3125 3.15 3:aa7edf89:::rbd_object_map.1121c78ac9e5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-21T21:41:34.186 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:33.871152+0000", 2024-09-21T21:41:34.186 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.217337319, 2024-09-21T21:41:34.186 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.052772614000000002, 2024-09-21T21:41:34.186 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:34.186 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:34.186 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:41:34.186 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:34.186 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5018", 2024-09-21T21:41:34.186 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3046875242", 2024-09-21T21:41:34.187 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 3125 2024-09-21T21:41:34.187 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.187 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:34.187 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.187 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:34.187 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.871152+0000", 2024-09-21T21:41:34.187 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:34.187 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.187 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.187 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:34.187 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.871152+0000", 2024-09-21T21:41:34.187 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:34.187 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.187 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.187 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:34.188 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.871153+0000", 2024-09-21T21:41:34.188 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.4e-07 2024-09-21T21:41:34.188 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.188 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.188 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:34.188 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.871157+0000", 2024-09-21T21:41:34.188 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1069999999999998e-06 2024-09-21T21:41:34.188 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.188 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.188 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:34.188 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.912625+0000", 2024-09-21T21:41:34.188 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041468659999999997 2024-09-21T21:41:34.188 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.188 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.188 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:34.189 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.912644+0000", 2024-09-21T21:41:34.189 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.8652e-05 2024-09-21T21:41:34.189 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.189 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.189 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:34.189 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.920531+0000", 2024-09-21T21:41:34.189 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0078870910000000006 2024-09-21T21:41:34.189 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.189 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.189 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:34.189 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.920612+0000", 2024-09-21T21:41:34.189 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0933999999999995e-05 2024-09-21T21:41:34.189 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.190 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.190 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:41:34.190 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.920926+0000", 2024-09-21T21:41:34.190 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00031363700000000002 2024-09-21T21:41:34.190 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.190 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.190 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:41:34.190 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.923748+0000", 2024-09-21T21:41:34.190 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0028222479999999999 2024-09-21T21:41:34.190 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.190 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.190 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:41:34.190 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.923802+0000", 2024-09-21T21:41:34.190 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3912999999999997e-05 2024-09-21T21:41:34.190 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.191 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.191 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:41:34.191 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.923857+0000", 2024-09-21T21:41:34.191 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5312000000000002e-05 2024-09-21T21:41:34.191 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.191 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.191 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:34.191 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.923924+0000", 2024-09-21T21:41:34.191 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7219999999999997e-05 2024-09-21T21:41:34.191 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:34.191 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:34.191 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:34.191 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:34.191 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:34.191 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:34.193 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T21:41:34.193 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:34.191+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:34.294 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:34.293+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:34.339 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:34.339 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:41:34.339 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-21T21:41:34.339 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:34.348 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T21:41:34.395 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:34.393+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:34.451 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:34.451 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:41:34.451 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:41:34.451 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:41:34.451 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:34.459 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T21:41:34.489 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T21:41:34.534 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:34.533+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:34.597 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:34.595+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:34.604 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:34.604 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-21T21:41:34.604 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-21T21:41:34.604 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:41:34.604 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.604 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6972 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5238784~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:34.604 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.922911+0000", 2024-09-21T21:41:34.604 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.6791014080000002, 2024-09-21T21:41:34.604 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041605798999999999, 2024-09-21T21:41:34.605 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:34.605 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:34.605 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:34.605 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:34.605 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:34.605 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:34.605 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6972 2024-09-21T21:41:34.605 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.605 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:34.605 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.605 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:34.605 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922911+0000", 2024-09-21T21:41:34.605 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:34.605 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.606 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.606 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:34.606 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922911+0000", 2024-09-21T21:41:34.606 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:34.606 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.606 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.606 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:34.606 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922912+0000", 2024-09-21T21:41:34.606 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8999999999999996e-07 2024-09-21T21:41:34.606 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.606 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.606 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:34.606 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922914+0000", 2024-09-21T21:41:34.606 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1050000000000002e-06 2024-09-21T21:41:34.607 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.607 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.607 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:34.607 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922920+0000", 2024-09-21T21:41:34.607 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4380000000000003e-06 2024-09-21T21:41:34.607 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.607 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.607 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:34.607 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922925+0000", 2024-09-21T21:41:34.607 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6300000000000003e-06 2024-09-21T21:41:34.607 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.607 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.607 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:34.607 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964424+0000", 2024-09-21T21:41:34.608 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041498342000000001 2024-09-21T21:41:34.608 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.608 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.608 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:34.608 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964466+0000", 2024-09-21T21:41:34.608 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2245e-05 2024-09-21T21:41:34.608 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.608 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.608 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:34.608 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964517+0000", 2024-09-21T21:41:34.608 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1449000000000002e-05 2024-09-21T21:41:34.608 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:34.608 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:34.609 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:34.609 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.609 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.609 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6974 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5246976~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:34.609 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.922932+0000", 2024-09-21T21:41:34.609 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.679080377, 2024-09-21T21:41:34.609 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041773352999999999, 2024-09-21T21:41:34.609 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:34.609 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:34.609 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:34.609 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:34.609 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:34.609 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:34.609 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6974 2024-09-21T21:41:34.610 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.610 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:34.610 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.610 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:34.610 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922932+0000", 2024-09-21T21:41:34.610 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:34.610 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.610 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.610 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:34.610 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922932+0000", 2024-09-21T21:41:34.610 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:34.610 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.610 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.611 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:34.611 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922933+0000", 2024-09-21T21:41:34.611 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.46e-07 2024-09-21T21:41:34.611 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.611 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.611 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:34.611 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922940+0000", 2024-09-21T21:41:34.611 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6140000000000003e-06 2024-09-21T21:41:34.611 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.611 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.611 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:34.611 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922949+0000", 2024-09-21T21:41:34.611 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6899999999999998e-06 2024-09-21T21:41:34.611 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.611 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.612 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:34.612 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922955+0000", 2024-09-21T21:41:34.612 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5140000000000001e-06 2024-09-21T21:41:34.612 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.612 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.612 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:34.612 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964547+0000", 2024-09-21T21:41:34.612 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041592854999999998 2024-09-21T21:41:34.612 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.612 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.612 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:34.612 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964617+0000", 2024-09-21T21:41:34.612 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9058000000000002e-05 2024-09-21T21:41:34.612 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.613 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.613 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:34.613 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964706+0000", 2024-09-21T21:41:34.613 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9276000000000004e-05 2024-09-21T21:41:34.613 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:34.613 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:34.613 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:34.613 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.613 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.613 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6976 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5261312~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:34.613 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.922961+0000", 2024-09-21T21:41:34.613 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.6790513320000002, 2024-09-21T21:41:34.613 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041863648000000003, 2024-09-21T21:41:34.613 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:34.614 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:34.614 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:34.614 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:34.614 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:34.614 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:34.614 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6976 2024-09-21T21:41:34.614 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.614 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:34.614 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.614 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:34.614 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922961+0000", 2024-09-21T21:41:34.614 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:34.614 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.614 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.614 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:34.615 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922961+0000", 2024-09-21T21:41:34.615 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:34.615 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.615 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.615 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:34.615 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922962+0000", 2024-09-21T21:41:34.615 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.89e-07 2024-09-21T21:41:34.615 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.615 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.615 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:34.615 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922964+0000", 2024-09-21T21:41:34.615 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1270000000000001e-06 2024-09-21T21:41:34.615 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.615 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.616 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:34.616 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922970+0000", 2024-09-21T21:41:34.616 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8030000000000002e-06 2024-09-21T21:41:34.616 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.616 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.616 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:34.616 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922979+0000", 2024-09-21T21:41:34.616 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5559999999999996e-06 2024-09-21T21:41:34.616 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.616 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.616 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:34.616 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964734+0000", 2024-09-21T21:41:34.616 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041754415000000003 2024-09-21T21:41:34.616 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.616 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.617 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:34.617 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964775+0000", 2024-09-21T21:41:34.617 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0806999999999997e-05 2024-09-21T21:41:34.617 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.617 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.617 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:34.617 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964825+0000", 2024-09-21T21:41:34.617 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0550999999999998e-05 2024-09-21T21:41:34.617 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:34.617 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:34.617 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:34.617 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.617 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.617 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6978 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5275648~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:34.618 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.922987+0000", 2024-09-21T21:41:34.618 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.6790255470000002, 2024-09-21T21:41:34.618 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042043876000000001, 2024-09-21T21:41:34.618 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:34.618 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:34.618 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:34.618 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:34.618 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:34.618 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:34.618 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6978 2024-09-21T21:41:34.618 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.618 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:34.618 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.618 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:34.619 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922987+0000", 2024-09-21T21:41:34.619 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:34.619 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.619 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.619 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:34.619 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922987+0000", 2024-09-21T21:41:34.619 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:34.619 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.619 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.619 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:34.619 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922988+0000", 2024-09-21T21:41:34.619 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1199999999999998e-07 2024-09-21T21:41:34.619 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.619 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.620 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:34.620 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922990+0000", 2024-09-21T21:41:34.620 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.142e-06 2024-09-21T21:41:34.620 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.620 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.620 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:34.620 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922997+0000", 2024-09-21T21:41:34.620 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.199e-06 2024-09-21T21:41:34.620 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.620 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.620 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:34.620 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923004+0000", 2024-09-21T21:41:34.620 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3599999999999998e-06 2024-09-21T21:41:34.620 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.621 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.621 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:34.621 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964856+0000", 2024-09-21T21:41:34.621 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041851521000000003 2024-09-21T21:41:34.621 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.621 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.621 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:34.621 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964944+0000", 2024-09-21T21:41:34.621 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7859e-05 2024-09-21T21:41:34.621 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.621 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.621 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:34.621 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965031+0000", 2024-09-21T21:41:34.621 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7383e-05 2024-09-21T21:41:34.621 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:34.622 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:34.622 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:34.622 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.622 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.622 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6980 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5289984~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:34.622 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923011+0000", 2024-09-21T21:41:34.622 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.6790022100000002, 2024-09-21T21:41:34.622 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042139337999999998, 2024-09-21T21:41:34.622 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:34.622 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:34.622 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:34.622 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:34.622 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:34.622 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:34.623 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6980 2024-09-21T21:41:34.623 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.623 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:34.623 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.623 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:34.623 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923011+0000", 2024-09-21T21:41:34.623 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:34.623 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.623 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.623 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:34.623 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923011+0000", 2024-09-21T21:41:34.623 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:34.623 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.623 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.624 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:34.624 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923011+0000", 2024-09-21T21:41:34.624 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6699999999999999e-07 2024-09-21T21:41:34.624 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.624 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.624 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:34.624 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923013+0000", 2024-09-21T21:41:34.624 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0279999999999999e-06 2024-09-21T21:41:34.624 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.624 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.624 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:34.624 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923018+0000", 2024-09-21T21:41:34.624 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4790000000000004e-06 2024-09-21T21:41:34.624 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.625 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.625 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:34.625 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923029+0000", 2024-09-21T21:41:34.625 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0601999999999999e-05 2024-09-21T21:41:34.625 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.625 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.625 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:34.625 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965059+0000", 2024-09-21T21:41:34.625 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042029947999999998 2024-09-21T21:41:34.625 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.625 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.625 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:34.625 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965100+0000", 2024-09-21T21:41:34.625 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1170000000000001e-05 2024-09-21T21:41:34.626 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.626 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.626 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:34.626 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965150+0000", 2024-09-21T21:41:34.626 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9743999999999997e-05 2024-09-21T21:41:34.626 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:34.626 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:34.626 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:34.626 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.626 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.626 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6982 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5304320~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:34.626 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923036+0000", 2024-09-21T21:41:34.626 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.6789763889999998, 2024-09-21T21:41:34.627 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042225666000000002, 2024-09-21T21:41:34.627 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:34.627 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:34.627 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:34.627 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:34.627 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:34.627 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:34.627 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6982 2024-09-21T21:41:34.627 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.627 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:34.627 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.627 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:34.627 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923036+0000", 2024-09-21T21:41:34.627 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:34.628 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.628 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.628 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:34.628 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923036+0000", 2024-09-21T21:41:34.628 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:34.628 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.628 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.628 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:34.628 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923037+0000", 2024-09-21T21:41:34.628 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2800000000000002e-07 2024-09-21T21:41:34.628 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.628 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.628 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:34.628 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923039+0000", 2024-09-21T21:41:34.629 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1500000000000002e-06 2024-09-21T21:41:34.629 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.629 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.629 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:34.629 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923049+0000", 2024-09-21T21:41:34.629 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0236e-05 2024-09-21T21:41:34.629 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.629 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.629 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:34.629 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923056+0000", 2024-09-21T21:41:34.629 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2540000000000003e-06 2024-09-21T21:41:34.629 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.629 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.629 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:34.630 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965173+0000", 2024-09-21T21:41:34.630 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042116686 2024-09-21T21:41:34.630 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.630 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.630 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:34.630 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965215+0000", 2024-09-21T21:41:34.630 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1696000000000002e-05 2024-09-21T21:41:34.630 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.630 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.630 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:34.630 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965262+0000", 2024-09-21T21:41:34.630 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7216000000000003e-05 2024-09-21T21:41:34.630 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:34.630 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:34.631 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:34.631 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.631 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.631 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6984 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5318656~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:34.631 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923063+0000", 2024-09-21T21:41:34.631 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.6789497349999998, 2024-09-21T21:41:34.631 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042318651999999998, 2024-09-21T21:41:34.631 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:34.631 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:34.631 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:34.631 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:34.631 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:34.631 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:34.631 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6984 2024-09-21T21:41:34.632 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.632 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:34.632 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.632 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:34.632 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923063+0000", 2024-09-21T21:41:34.632 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:34.632 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.632 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.632 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:34.632 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923063+0000", 2024-09-21T21:41:34.632 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:34.632 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.632 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.632 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:34.632 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923064+0000", 2024-09-21T21:41:34.633 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2e-07 2024-09-21T21:41:34.633 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.633 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.633 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:34.633 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923066+0000", 2024-09-21T21:41:34.633 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.057e-06 2024-09-21T21:41:34.633 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.633 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.633 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:34.633 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923071+0000", 2024-09-21T21:41:34.633 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8810000000000001e-06 2024-09-21T21:41:34.633 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.633 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.634 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:34.634 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923078+0000", 2024-09-21T21:41:34.634 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5479999999999998e-06 2024-09-21T21:41:34.634 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.634 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.634 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:34.634 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965288+0000", 2024-09-21T21:41:34.634 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042209576999999998 2024-09-21T21:41:34.634 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.634 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.634 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:34.634 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965332+0000", 2024-09-21T21:41:34.634 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4366999999999997e-05 2024-09-21T21:41:34.634 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.635 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.635 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:34.635 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965382+0000", 2024-09-21T21:41:34.635 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9802000000000002e-05 2024-09-21T21:41:34.635 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:34.635 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:34.635 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:34.635 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.635 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.635 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6987 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5332992~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:34.635 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923085+0000", 2024-09-21T21:41:34.635 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.6789282100000005, 2024-09-21T21:41:34.635 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042412103999999999, 2024-09-21T21:41:34.636 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:34.636 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:34.636 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:34.636 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:34.636 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:34.636 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:34.636 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6987 2024-09-21T21:41:34.636 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.636 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:34.636 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.636 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:34.636 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923085+0000", 2024-09-21T21:41:34.636 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:34.636 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.637 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.637 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:34.637 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923085+0000", 2024-09-21T21:41:34.637 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:34.637 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.637 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.637 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:34.637 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923085+0000", 2024-09-21T21:41:34.637 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.89e-07 2024-09-21T21:41:34.637 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.637 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.637 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:34.637 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923087+0000", 2024-09-21T21:41:34.637 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0810000000000001e-06 2024-09-21T21:41:34.638 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.638 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.638 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:34.638 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923098+0000", 2024-09-21T21:41:34.638 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1299e-05 2024-09-21T21:41:34.638 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.638 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.638 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:34.638 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923104+0000", 2024-09-21T21:41:34.638 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9100000000000002e-06 2024-09-21T21:41:34.638 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.638 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.638 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:34.638 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965404+0000", 2024-09-21T21:41:34.639 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042299622000000002 2024-09-21T21:41:34.639 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.639 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.639 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:34.639 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965445+0000", 2024-09-21T21:41:34.639 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0701000000000002e-05 2024-09-21T21:41:34.639 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.639 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.639 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:34.639 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965497+0000", 2024-09-21T21:41:34.639 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2101999999999997e-05 2024-09-21T21:41:34.639 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:34.640 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:34.640 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:34.640 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.640 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.640 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6989 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5347328~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:34.640 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923111+0000", 2024-09-21T21:41:34.640 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.6789018699999998, 2024-09-21T21:41:34.640 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042505188999999999, 2024-09-21T21:41:34.640 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:34.640 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:34.640 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:34.640 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:34.641 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:34.641 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:34.641 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6989 2024-09-21T21:41:34.641 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.641 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:34.641 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.641 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:34.641 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923111+0000", 2024-09-21T21:41:34.641 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:34.641 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.641 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.641 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:34.641 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923111+0000", 2024-09-21T21:41:34.641 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:34.642 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.642 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.642 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:34.642 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923111+0000", 2024-09-21T21:41:34.642 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4499999999999997e-07 2024-09-21T21:41:34.642 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.642 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.642 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:34.642 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923114+0000", 2024-09-21T21:41:34.642 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1569999999999998e-06 2024-09-21T21:41:34.642 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.642 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.642 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:34.642 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923120+0000", 2024-09-21T21:41:34.643 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1759999999999998e-06 2024-09-21T21:41:34.643 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.643 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.643 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:34.643 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923125+0000", 2024-09-21T21:41:34.643 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2560000000000002e-06 2024-09-21T21:41:34.643 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.643 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.643 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:34.643 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965523+0000", 2024-09-21T21:41:34.643 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042398094999999997 2024-09-21T21:41:34.643 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.643 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.643 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:34.644 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965566+0000", 2024-09-21T21:41:34.644 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3393999999999998e-05 2024-09-21T21:41:34.644 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.644 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.644 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:34.644 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965616+0000", 2024-09-21T21:41:34.644 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9666000000000002e-05 2024-09-21T21:41:34.644 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:34.644 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:34.644 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:34.644 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.644 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.644 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6991 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5361664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:34.644 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923132+0000", 2024-09-21T21:41:34.644 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.6788805809999996, 2024-09-21T21:41:34.645 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042600779999999998, 2024-09-21T21:41:34.645 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:34.645 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:34.645 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:34.645 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:34.645 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:34.645 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:34.645 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6991 2024-09-21T21:41:34.645 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.645 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:34.645 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.645 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:34.645 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923132+0000", 2024-09-21T21:41:34.645 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:34.646 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.646 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.646 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:34.646 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923132+0000", 2024-09-21T21:41:34.646 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:34.646 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.646 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.646 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:34.646 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923132+0000", 2024-09-21T21:41:34.646 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4200000000000002e-07 2024-09-21T21:41:34.646 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.646 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.646 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:34.646 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923134+0000", 2024-09-21T21:41:34.647 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2270000000000001e-06 2024-09-21T21:41:34.647 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.647 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.647 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:34.647 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923137+0000", 2024-09-21T21:41:34.647 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1930000000000002e-06 2024-09-21T21:41:34.647 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.647 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.647 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:34.647 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923139+0000", 2024-09-21T21:41:34.647 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4710000000000001e-06 2024-09-21T21:41:34.647 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.647 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.647 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:34.647 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965639+0000", 2024-09-21T21:41:34.648 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.04249965 2024-09-21T21:41:34.648 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.648 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.648 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:34.648 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965679+0000", 2024-09-21T21:41:34.648 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9852999999999999e-05 2024-09-21T21:41:34.648 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.648 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.648 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:34.648 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965733+0000", 2024-09-21T21:41:34.648 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4144000000000002e-05 2024-09-21T21:41:34.648 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:34.648 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:34.648 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:34.649 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.649 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.649 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6993 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5384192~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:34.649 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923143+0000", 2024-09-21T21:41:34.649 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.678869787, 2024-09-21T21:41:34.649 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042708182999999997, 2024-09-21T21:41:34.649 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:34.649 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:34.649 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:34.649 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:34.649 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:34.649 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:34.649 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6993 2024-09-21T21:41:34.649 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.650 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:34.650 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.650 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:34.650 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923143+0000", 2024-09-21T21:41:34.650 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:34.650 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.650 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.650 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:34.650 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923143+0000", 2024-09-21T21:41:34.650 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:34.650 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.650 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.650 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:34.650 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923143+0000", 2024-09-21T21:41:34.651 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0800000000000001e-07 2024-09-21T21:41:34.651 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.651 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.651 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:34.651 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923144+0000", 2024-09-21T21:41:34.651 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.009e-06 2024-09-21T21:41:34.651 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.651 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.651 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:34.651 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923148+0000", 2024-09-21T21:41:34.651 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.737e-06 2024-09-21T21:41:34.651 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.651 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.651 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:34.652 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923150+0000", 2024-09-21T21:41:34.652 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4669999999999998e-06 2024-09-21T21:41:34.652 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.652 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.652 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:34.652 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965760+0000", 2024-09-21T21:41:34.652 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042609399999999999 2024-09-21T21:41:34.652 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.652 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.652 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:34.652 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965801+0000", 2024-09-21T21:41:34.652 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1399999999999997e-05 2024-09-21T21:41:34.652 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.652 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.652 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:34.653 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965851+0000", 2024-09-21T21:41:34.653 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9962e-05 2024-09-21T21:41:34.653 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:34.653 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:34.653 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:34.653 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.653 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.653 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6995 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5398528~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:34.653 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923153+0000", 2024-09-21T21:41:34.653 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.6788594230000005, 2024-09-21T21:41:34.653 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042828034000000001, 2024-09-21T21:41:34.653 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:34.653 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:34.653 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:34.654 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:34.654 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:34.654 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:34.654 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6995 2024-09-21T21:41:34.654 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.654 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:34.654 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.654 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:34.654 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923153+0000", 2024-09-21T21:41:34.654 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:34.654 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.654 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.654 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:34.654 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923153+0000", 2024-09-21T21:41:34.655 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:34.655 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.655 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.655 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:34.655 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923154+0000", 2024-09-21T21:41:34.655 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2600000000000001e-07 2024-09-21T21:41:34.655 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.655 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.655 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:34.655 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923158+0000", 2024-09-21T21:41:34.655 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0559999999999998e-06 2024-09-21T21:41:34.655 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.655 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.655 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:34.655 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923161+0000", 2024-09-21T21:41:34.656 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6519999999999999e-06 2024-09-21T21:41:34.656 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.656 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.656 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:34.656 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923170+0000", 2024-09-21T21:41:34.656 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5720000000000003e-06 2024-09-21T21:41:34.656 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.656 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.656 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:34.656 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965887+0000", 2024-09-21T21:41:34.656 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042716863000000001 2024-09-21T21:41:34.656 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.656 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.656 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:34.657 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965927+0000", 2024-09-21T21:41:34.657 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0346999999999998e-05 2024-09-21T21:41:34.657 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.657 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.657 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:34.657 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965981+0000", 2024-09-21T21:41:34.657 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4317999999999997e-05 2024-09-21T21:41:34.657 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:34.657 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:34.657 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:34.657 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.657 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.657 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6997 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5412864~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:34.657 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923174+0000", 2024-09-21T21:41:34.657 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.6788392679999999, 2024-09-21T21:41:34.658 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042928296999999997, 2024-09-21T21:41:34.658 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:34.658 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:34.658 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:34.658 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:34.658 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:34.658 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:34.658 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6997 2024-09-21T21:41:34.658 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.658 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:34.658 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.658 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:34.658 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923174+0000", 2024-09-21T21:41:34.658 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:34.658 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.659 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.659 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:34.659 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923174+0000", 2024-09-21T21:41:34.659 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:34.659 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.659 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.659 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:34.659 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923174+0000", 2024-09-21T21:41:34.659 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.67e-07 2024-09-21T21:41:34.659 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.659 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.659 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:34.659 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923178+0000", 2024-09-21T21:41:34.659 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1330000000000001e-06 2024-09-21T21:41:34.660 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.660 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.660 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:34.660 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923182+0000", 2024-09-21T21:41:34.660 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.794e-06 2024-09-21T21:41:34.660 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.660 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.660 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:34.660 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923184+0000", 2024-09-21T21:41:34.660 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.435e-06 2024-09-21T21:41:34.660 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.660 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.660 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:34.660 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966009+0000", 2024-09-21T21:41:34.661 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042824327000000002 2024-09-21T21:41:34.661 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.661 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.661 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:34.661 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966052+0000", 2024-09-21T21:41:34.661 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3034000000000002e-05 2024-09-21T21:41:34.661 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.661 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.661 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:34.661 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966102+0000", 2024-09-21T21:41:34.661 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0306999999999998e-05 2024-09-21T21:41:34.661 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:34.661 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:34.661 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:34.661 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.662 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.662 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6999 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5427200~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:34.662 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923187+0000", 2024-09-21T21:41:34.662 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.6788255400000001, 2024-09-21T21:41:34.662 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043104267000000002, 2024-09-21T21:41:34.662 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:34.662 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:34.662 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:34.662 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:34.662 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:34.662 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:34.662 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6999 2024-09-21T21:41:34.662 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.662 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:34.663 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.663 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:34.663 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923187+0000", 2024-09-21T21:41:34.663 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:34.663 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.663 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.663 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:34.663 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923187+0000", 2024-09-21T21:41:34.663 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:34.663 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.663 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.663 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:34.663 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923188+0000", 2024-09-21T21:41:34.663 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3999999999999998e-07 2024-09-21T21:41:34.663 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.664 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.664 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:34.664 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923189+0000", 2024-09-21T21:41:34.664 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4169999999999999e-06 2024-09-21T21:41:34.664 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.664 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.664 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:34.664 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923194+0000", 2024-09-21T21:41:34.664 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1819999999999997e-06 2024-09-21T21:41:34.664 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.664 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.664 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:34.664 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923198+0000", 2024-09-21T21:41:34.664 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.143e-06 2024-09-21T21:41:34.665 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.665 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.665 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:34.665 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966135+0000", 2024-09-21T21:41:34.665 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042936297999999998 2024-09-21T21:41:34.665 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.665 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.665 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:34.665 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966204+0000", 2024-09-21T21:41:34.665 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8989e-05 2024-09-21T21:41:34.665 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.665 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.665 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:34.665 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966292+0000", 2024-09-21T21:41:34.665 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7997999999999993e-05 2024-09-21T21:41:34.666 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:34.666 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:34.666 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:34.666 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.666 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.666 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7001 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5441536~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:34.666 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923211+0000", 2024-09-21T21:41:34.666 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.6788017489999998, 2024-09-21T21:41:34.666 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043196518000000003, 2024-09-21T21:41:34.666 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:34.666 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:34.666 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:34.667 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:34.667 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:34.667 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:34.667 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7001 2024-09-21T21:41:34.667 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.667 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:34.667 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.667 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:34.667 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923211+0000", 2024-09-21T21:41:34.667 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:34.667 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.667 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.667 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:34.667 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923211+0000", 2024-09-21T21:41:34.668 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:34.668 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.668 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.668 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:34.668 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923212+0000", 2024-09-21T21:41:34.668 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3900000000000005e-07 2024-09-21T21:41:34.668 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.668 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.668 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:34.668 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923214+0000", 2024-09-21T21:41:34.668 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9970000000000001e-06 2024-09-21T21:41:34.668 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.668 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.668 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:34.668 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923219+0000", 2024-09-21T21:41:34.669 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4310000000000002e-06 2024-09-21T21:41:34.669 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.669 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.669 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:34.669 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923224+0000", 2024-09-21T21:41:34.669 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6410000000000001e-06 2024-09-21T21:41:34.669 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.669 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.669 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:34.669 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966318+0000", 2024-09-21T21:41:34.669 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043093842 2024-09-21T21:41:34.669 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.669 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.669 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:34.669 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966359+0000", 2024-09-21T21:41:34.670 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1260999999999997e-05 2024-09-21T21:41:34.670 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.670 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.670 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:34.670 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966408+0000", 2024-09-21T21:41:34.670 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8807000000000002e-05 2024-09-21T21:41:34.670 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:34.670 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:34.670 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:34.670 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.670 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.670 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7002 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5455872~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:34.670 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923229+0000", 2024-09-21T21:41:34.670 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.6787833389999998, 2024-09-21T21:41:34.671 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043364568999999999, 2024-09-21T21:41:34.671 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:34.671 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:34.671 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:34.671 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:34.671 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:34.671 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:34.671 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7002 2024-09-21T21:41:34.671 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.671 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:34.671 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.671 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:34.671 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923229+0000", 2024-09-21T21:41:34.672 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:34.672 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.672 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.672 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:34.672 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923229+0000", 2024-09-21T21:41:34.672 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:34.672 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.672 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.672 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:34.672 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923230+0000", 2024-09-21T21:41:34.672 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4700000000000002e-07 2024-09-21T21:41:34.672 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.672 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.672 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:34.672 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923232+0000", 2024-09-21T21:41:34.673 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0719999999999998e-06 2024-09-21T21:41:34.673 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.673 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.673 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:34.673 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923237+0000", 2024-09-21T21:41:34.673 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2290000000000002e-06 2024-09-21T21:41:34.673 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.673 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.673 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:34.673 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923245+0000", 2024-09-21T21:41:34.673 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2959999999999999e-06 2024-09-21T21:41:34.673 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.673 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.673 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:34.674 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966438+0000", 2024-09-21T21:41:34.674 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043193027000000002 2024-09-21T21:41:34.674 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.674 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.674 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:34.674 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966506+0000", 2024-09-21T21:41:34.674 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7707999999999996e-05 2024-09-21T21:41:34.674 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.674 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.674 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:34.674 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966594+0000", 2024-09-21T21:41:34.674 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.789e-05 2024-09-21T21:41:34.674 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:34.674 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:34.675 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:34.675 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.675 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.675 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7005 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5470208~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:34.675 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923254+0000", 2024-09-21T21:41:34.675 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.6787591080000004, 2024-09-21T21:41:34.675 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043457176, 2024-09-21T21:41:34.675 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:34.675 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:34.675 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:34.675 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:34.675 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:34.675 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:34.675 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7005 2024-09-21T21:41:34.676 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.676 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:34.676 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.676 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:34.676 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923254+0000", 2024-09-21T21:41:34.676 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:34.676 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.676 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.676 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:34.676 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923254+0000", 2024-09-21T21:41:34.676 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:34.676 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.676 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.676 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:34.677 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923254+0000", 2024-09-21T21:41:34.677 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4700000000000002e-07 2024-09-21T21:41:34.677 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.677 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.677 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:34.677 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923256+0000", 2024-09-21T21:41:34.677 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.007e-06 2024-09-21T21:41:34.677 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.677 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.677 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:34.677 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923266+0000", 2024-09-21T21:41:34.677 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.4700000000000008e-06 2024-09-21T21:41:34.677 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.678 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.678 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:34.678 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923271+0000", 2024-09-21T21:41:34.678 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2809999999999999e-06 2024-09-21T21:41:34.678 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.678 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.678 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:34.678 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966619+0000", 2024-09-21T21:41:34.678 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043347951000000003 2024-09-21T21:41:34.678 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.678 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.678 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:34.678 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966661+0000", 2024-09-21T21:41:34.678 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2614999999999998e-05 2024-09-21T21:41:34.679 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.679 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.679 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:34.679 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966711+0000", 2024-09-21T21:41:34.679 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9505000000000001e-05 2024-09-21T21:41:34.679 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:34.679 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:34.679 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:34.679 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.679 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.679 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7008 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5484544~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:34.679 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923278+0000", 2024-09-21T21:41:34.679 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.6787344610000003, 2024-09-21T21:41:34.679 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043631071, 2024-09-21T21:41:34.680 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:34.680 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:34.680 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:34.680 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:34.680 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:34.680 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:34.680 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7008 2024-09-21T21:41:34.680 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.680 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:34.680 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.680 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:34.680 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923278+0000", 2024-09-21T21:41:34.680 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:34.681 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.681 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.681 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:34.681 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923278+0000", 2024-09-21T21:41:34.681 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:34.681 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.681 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.681 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:34.681 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923279+0000", 2024-09-21T21:41:34.681 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.77e-07 2024-09-21T21:41:34.681 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.681 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.681 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:34.681 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923281+0000", 2024-09-21T21:41:34.681 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0219999999999999e-06 2024-09-21T21:41:34.682 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.682 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.682 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:34.682 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923287+0000", 2024-09-21T21:41:34.682 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7799999999999997e-06 2024-09-21T21:41:34.682 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.682 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.682 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:34.682 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923291+0000", 2024-09-21T21:41:34.682 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1699999999999999e-06 2024-09-21T21:41:34.682 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.682 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.682 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:34.682 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966742+0000", 2024-09-21T21:41:34.683 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043451344000000003 2024-09-21T21:41:34.683 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.683 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.683 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:34.683 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966809+0000", 2024-09-21T21:41:34.683 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7070999999999995e-05 2024-09-21T21:41:34.683 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.683 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.683 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:34.683 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966909+0000", 2024-09-21T21:41:34.683 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000100307 2024-09-21T21:41:34.683 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:34.683 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:34.684 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:34.684 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.684 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.684 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7010 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5498880~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:34.684 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923301+0000", 2024-09-21T21:41:34.684 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.6787115660000005, 2024-09-21T21:41:34.684 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043717830999999999, 2024-09-21T21:41:34.684 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:34.684 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:34.684 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:34.684 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:34.684 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:34.684 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:34.684 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7010 2024-09-21T21:41:34.685 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.685 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:34.685 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.685 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:34.685 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923301+0000", 2024-09-21T21:41:34.685 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:34.685 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.685 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.685 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:34.685 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923301+0000", 2024-09-21T21:41:34.685 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:34.685 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.685 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.685 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:34.685 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923302+0000", 2024-09-21T21:41:34.686 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.41e-07 2024-09-21T21:41:34.686 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.686 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.686 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:34.686 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923304+0000", 2024-09-21T21:41:34.686 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0990000000000001e-06 2024-09-21T21:41:34.686 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.686 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.686 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:34.686 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923310+0000", 2024-09-21T21:41:34.686 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3860000000000004e-06 2024-09-21T21:41:34.686 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.686 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.687 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:34.687 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923315+0000", 2024-09-21T21:41:34.687 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8459999999999999e-06 2024-09-21T21:41:34.687 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.687 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.687 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:34.687 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966938+0000", 2024-09-21T21:41:34.687 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043622693999999997 2024-09-21T21:41:34.687 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.687 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.687 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:34.687 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966975+0000", 2024-09-21T21:41:34.687 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7608999999999999e-05 2024-09-21T21:41:34.687 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.688 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.688 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:34.688 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.967019+0000", 2024-09-21T21:41:34.688 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3856000000000002e-05 2024-09-21T21:41:34.688 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:34.688 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:34.688 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:34.688 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.688 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.688 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7354 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 2107392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:34.688 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.936932+0000", 2024-09-21T21:41:34.688 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.6650807109999999, 2024-09-21T21:41:34.688 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041607708, 2024-09-21T21:41:34.688 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:34.689 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:34.689 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:34.689 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:34.689 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:34.689 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:34.689 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7354 2024-09-21T21:41:34.689 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.689 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:34.689 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.689 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:34.689 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.936932+0000", 2024-09-21T21:41:34.689 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:34.689 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.689 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.690 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:34.690 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.936932+0000", 2024-09-21T21:41:34.690 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:34.690 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.690 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.690 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:34.690 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.937276+0000", 2024-09-21T21:41:34.690 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00034344200000000001 2024-09-21T21:41:34.690 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.690 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.690 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:34.690 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.937279+0000", 2024-09-21T21:41:34.690 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.844e-06 2024-09-21T21:41:34.690 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.691 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.691 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:34.691 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.937288+0000", 2024-09-21T21:41:34.691 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2409999999999995e-06 2024-09-21T21:41:34.691 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.691 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.691 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:34.691 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.937293+0000", 2024-09-21T21:41:34.691 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6559999999999997e-06 2024-09-21T21:41:34.691 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.691 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.691 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:34.691 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.978448+0000", 2024-09-21T21:41:34.692 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041154621000000002 2024-09-21T21:41:34.692 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.692 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.692 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:34.692 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.978487+0000", 2024-09-21T21:41:34.692 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9254000000000001e-05 2024-09-21T21:41:34.692 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:34.692 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:34.692 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:34.692 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.978540+0000", 2024-09-21T21:41:34.692 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2649999999999999e-05 2024-09-21T21:41:34.692 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:34.692 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:34.692 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:34.693 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:34.693 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:34.693 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:34.693 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0: {} 2024-09-21T21:41:34.694 INFO:teuthology.orchestra.run.smithi081.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T21:41:34.694 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1: {} 2024-09-21T21:41:34.695 INFO:teuthology.orchestra.run.smithi081.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T21:41:34.695 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T21:41:34.695 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2: {} 2024-09-21T21:41:34.696 INFO:teuthology.orchestra.run.smithi081.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T21:41:34.696 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3: {} 2024-09-21T21:41:34.696 INFO:teuthology.orchestra.run.smithi081.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T21:41:34.696 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4: {} 2024-09-21T21:41:34.697 INFO:teuthology.orchestra.run.smithi081.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T21:41:34.700 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5: {} 2024-09-21T21:41:34.700 INFO:teuthology.orchestra.run.smithi081.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T21:41:34.735 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:34.734+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:34.799 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:34.798+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:34.813 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-21T21:41:34.813 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T21:41:34.813 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-21T21:41:34.813 INFO:tasks.thrashosds.thrasher:Setting osd 2 primary_affinity to 0.000000 2024-09-21T21:41:34.814 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 0 2024-09-21T21:41:34.839 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:34.840 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:34.848 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T21:41:34.899 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:34.898+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:34.954 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:34.955 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:34.963 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T21:41:35.004 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:35.003+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:35.102 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:35.101+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:35.109 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:35.109 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:35.118 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T21:41:35.167 INFO:teuthology.orchestra.run.smithi164.stdout:checking clone #23, image image_client.0-clone23 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent24 2024-09-21T21:41:35.203 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:35.202+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:35.223 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:35.224 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:35.233 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T21:41:35.304 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:35.303+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:35.387 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:35.387 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:35.397 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T21:41:35.405 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:35.404+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:35.506 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:35.505+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:35.510 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:35.510 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:35.519 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T21:41:35.607 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:35.606+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:35.662 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:35.662 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:35.670 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T21:41:35.705 INFO:teuthology.orchestra.run.smithi081.stderr:set osd.2 primary-affinity to 0 (802) 2024-09-21T21:41:35.708 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:35.706+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:35.711 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T21:41:35.721 INFO:teuthology.orchestra.run.smithi164.stdout:3436 read 0x619600 thru 0x6255ca (0xbfcb bytes) 2024-09-21T21:41:35.723 INFO:teuthology.orchestra.run.smithi164.stdout:3437 write 0x9d97800 thru 0x9d9cb10 (0x5311 bytes) 2024-09-21T21:41:35.724 INFO:teuthology.orchestra.run.smithi164.stdout:3438 read 0x6eea000 thru 0x6eecd04 (0x2d05 bytes) 2024-09-21T21:41:35.726 INFO:teuthology.orchestra.run.smithi164.stdout:3440 read 0xc359400 thru 0xc35c38b (0x2f8c bytes) 2024-09-21T21:41:35.726 INFO:teuthology.orchestra.run.smithi164.stdout:3441 read 0x2a48400 thru 0x2a4fdcc (0x79cd bytes) 2024-09-21T21:41:35.727 INFO:teuthology.orchestra.run.smithi164.stdout:3442 read 0x93cd200 thru 0x93cea78 (0x1879 bytes) 2024-09-21T21:41:35.727 INFO:teuthology.orchestra.run.smithi164.stdout:3443 write 0xb0a8c00 thru 0xb0b6fb8 (0xe3b9 bytes) 2024-09-21T21:41:35.730 INFO:teuthology.orchestra.run.smithi164.stdout:3444 write 0x39b1000 thru 0x39b1092 (0x93 bytes) 2024-09-21T21:41:35.730 INFO:teuthology.orchestra.run.smithi164.stdout:3445 write 0x3f18e00 thru 0x3f25e45 (0xd046 bytes) 2024-09-21T21:41:35.731 INFO:teuthology.orchestra.run.smithi164.stdout:3446 write 0x5366c00 thru 0x5372374 (0xb775 bytes) 2024-09-21T21:41:35.731 INFO:teuthology.orchestra.run.smithi164.stdout:3449 punch from 0xb465200 to 0xb475000, (0xfe00 bytes) 2024-09-21T21:41:35.755 INFO:teuthology.orchestra.run.smithi164.stdout:3452 trunc from 0xd0b8e00 to 0x9f46600 2024-09-21T21:41:35.775 INFO:teuthology.orchestra.run.smithi164.stdout:3455 write 0x7a45800 thru 0x7a46b91 (0x1392 bytes) 2024-09-21T21:41:35.798 INFO:teuthology.orchestra.run.smithi164.stdout:3457 write 0xc797000 thru 0xc7a0840 (0x9841 bytes) 2024-09-21T21:41:35.799 INFO:teuthology.orchestra.run.smithi164.stdout:3458 write 0x1186c00 thru 0x1189916 (0x2d17 bytes) 2024-09-21T21:41:35.799 INFO:teuthology.orchestra.run.smithi164.stdout:3459 punch from 0x777200 to 0x77da00, (0x6800 bytes) 2024-09-21T21:41:35.800 INFO:teuthology.orchestra.run.smithi164.stdout:3460 write 0x50f4600 thru 0x50fc066 (0x7a67 bytes) 2024-09-21T21:41:35.804 INFO:teuthology.orchestra.run.smithi164.stdout:3461 trunc from 0xc7a0a00 to 0x9403e00 2024-09-21T21:41:35.808 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:35.807+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:35.818 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:35.819 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:35.828 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T21:41:35.846 INFO:teuthology.orchestra.run.smithi164.stdout:3463 trunc from 0x9403e00 to 0x38dc600 2024-09-21T21:41:35.879 INFO:teuthology.orchestra.run.smithi164.stdout:3465 punch from 0x1230a00 to 0x1232c00, (0x2200 bytes) 2024-09-21T21:41:35.881 INFO:teuthology.orchestra.run.smithi164.stdout:3467 trunc from 0x38dc600 to 0x37fbe00 2024-09-21T21:41:35.892 INFO:teuthology.orchestra.run.smithi164.stdout:3469 read 0x2608400 thru 0x2613884 (0xb485 bytes) 2024-09-21T21:41:35.893 INFO:teuthology.orchestra.run.smithi164.stdout:3470 read 0xfffa00 thru 0x1002ca3 (0x32a4 bytes) 2024-09-21T21:41:35.894 INFO:teuthology.orchestra.run.smithi164.stdout:3472 write 0x1b08000 thru 0x1b0bf20 (0x3f21 bytes) 2024-09-21T21:41:35.895 INFO:teuthology.orchestra.run.smithi164.stdout:3474 punch from 0x2e54c00 to 0x2e64a00, (0xfe00 bytes) 2024-09-21T21:41:35.895 INFO:teuthology.orchestra.run.smithi164.stdout:3476 read 0xfe0400 thru 0xfe2603 (0x2204 bytes) 2024-09-21T21:41:35.895 INFO:teuthology.orchestra.run.smithi164.stdout:3480 punch from 0x36df200 to 0x36e3a00, (0x4800 bytes) 2024-09-21T21:41:35.895 INFO:teuthology.orchestra.run.smithi164.stdout:3482 punch from 0x165ec00 to 0x1667e00, (0x9200 bytes) 2024-09-21T21:41:35.903 INFO:teuthology.orchestra.run.smithi164.stdout:3483 trunc from 0x37fbe00 to 0xa495800 2024-09-21T21:41:35.909 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:35.908+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:35.927 INFO:teuthology.orchestra.run.smithi164.stdout:3485 write 0xe764400 thru 0xe769874 (0x5475 bytes) 2024-09-21T21:41:35.928 INFO:teuthology.orchestra.run.smithi164.stdout:3487 read 0x7ee9600 thru 0x7ef23ab (0x8dac bytes) 2024-09-21T21:41:35.929 INFO:teuthology.orchestra.run.smithi164.stdout:3488 read 0x8ef6800 thru 0x8efd811 (0x7012 bytes) 2024-09-21T21:41:35.930 INFO:teuthology.orchestra.run.smithi164.stdout:3491 write 0xbfacc00 thru 0xbfb986a (0xcc6b bytes) 2024-09-21T21:41:35.930 INFO:teuthology.orchestra.run.smithi164.stdout:3494 write 0x2180600 thru 0x218ad48 (0xa749 bytes) 2024-09-21T21:41:35.930 INFO:teuthology.orchestra.run.smithi164.stdout:3496 write 0x90b0600 thru 0x90b6de8 (0x67e9 bytes) 2024-09-21T21:41:35.930 INFO:teuthology.orchestra.run.smithi164.stdout:3497 punch from 0x2108800 to 0x2110e00, (0x8600 bytes) 2024-09-21T21:41:35.933 INFO:teuthology.orchestra.run.smithi164.stdout:3498 read 0x6df5c00 thru 0x6df679c (0xb9d bytes) 2024-09-21T21:41:35.933 INFO:teuthology.orchestra.run.smithi164.stdout:3500 write 0xc544e00 thru 0xc546361 (0x1562 bytes) 2024-09-21T21:41:35.933 INFO:teuthology.orchestra.run.smithi164.stdout:3502 write 0xd256200 thru 0xd25a7d7 (0x45d8 bytes) 2024-09-21T21:41:35.933 INFO:teuthology.orchestra.run.smithi164.stdout:3503 write 0x4bf3800 thru 0x4bf940e (0x5c0f bytes) 2024-09-21T21:41:35.934 INFO:teuthology.orchestra.run.smithi164.stdout:3504 read 0xc1d0400 thru 0xc1d0f4c (0xb4d bytes) 2024-09-21T21:41:35.934 INFO:teuthology.orchestra.run.smithi164.stdout:3505 read 0x8a26a00 thru 0x8a26d57 (0x358 bytes) 2024-09-21T21:41:35.934 INFO:teuthology.orchestra.run.smithi164.stdout:3506 write 0x1f15800 thru 0x1f1ebb3 (0x93b4 bytes) 2024-09-21T21:41:35.934 INFO:teuthology.orchestra.run.smithi164.stdout:3507 read 0x933dc00 thru 0x934b0f9 (0xd4fa bytes) 2024-09-21T21:41:35.934 INFO:teuthology.orchestra.run.smithi164.stdout:3508 write 0x3006a00 thru 0x300eb01 (0x8102 bytes) 2024-09-21T21:41:35.945 INFO:teuthology.orchestra.run.smithi164.stdout:3513 trunc from 0xe769a00 to 0x3a0e800 2024-09-21T21:41:35.970 INFO:teuthology.orchestra.run.smithi164.stdout:3514 punch from 0x3787200 to 0x3796600, (0xf400 bytes) 2024-09-21T21:41:35.979 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:35.979 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:35.988 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T21:41:35.991 INFO:teuthology.orchestra.run.smithi164.stdout:3515 write 0xae2d800 thru 0xae36b22 (0x9323 bytes) 2024-09-21T21:41:35.995 INFO:teuthology.orchestra.run.smithi164.stdout:3516 trunc from 0xae36c00 to 0xda90200 2024-09-21T21:41:36.007 INFO:teuthology.orchestra.run.smithi164.stdout:3517 write 0x3219600 thru 0x321a730 (0x1131 bytes) 2024-09-21T21:41:36.008 INFO:teuthology.orchestra.run.smithi164.stdout:3518 write 0x1d9f800 thru 0x1da2b4e (0x334f bytes) 2024-09-21T21:41:36.009 INFO:teuthology.orchestra.run.smithi164.stdout:3520 read 0x96f1a00 thru 0x96f2d4d (0x134e bytes) 2024-09-21T21:41:36.009 INFO:teuthology.orchestra.run.smithi164.stdout:3523 write 0x6e52800 thru 0x6e5802a (0x582b bytes) 2024-09-21T21:41:36.010 INFO:teuthology.orchestra.run.smithi164.stdout:3526 write 0x6a9600 thru 0x6b7f2f (0xe930 bytes) 2024-09-21T21:41:36.010 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:36.009+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:36.024 INFO:teuthology.orchestra.run.smithi164.stdout:3527 trunc from 0xda90200 to 0x1be7e00 2024-09-21T21:41:36.051 INFO:teuthology.orchestra.run.smithi164.stdout:3528 read 0x12d0000 thru 0x12dd4ed (0xd4ee bytes) 2024-09-21T21:41:36.058 INFO:teuthology.orchestra.run.smithi164.stdout:3532 read 0x1555200 thru 0x1558ede (0x3cdf bytes) 2024-09-21T21:41:36.058 INFO:teuthology.orchestra.run.smithi164.stdout:3533 read 0x11be00 thru 0x12169b (0x589c bytes) 2024-09-21T21:41:36.066 INFO:teuthology.orchestra.run.smithi164.stdout:3534 trunc from 0x1be7e00 to 0x872b400 2024-09-21T21:41:36.087 INFO:teuthology.orchestra.run.smithi164.stdout:3535 write 0xba2f800 thru 0xba3918d (0x998e bytes) 2024-09-21T21:41:36.087 INFO:teuthology.orchestra.run.smithi164.stdout:3537 read 0x261800 thru 0x263752 (0x1f53 bytes) 2024-09-21T21:41:36.088 INFO:teuthology.orchestra.run.smithi164.stdout:3538 read 0x2678400 thru 0x26820ab (0x9cac bytes) 2024-09-21T21:41:36.092 INFO:teuthology.orchestra.run.smithi164.stdout:3540 read 0x1a55800 thru 0x1a5bf72 (0x6773 bytes) 2024-09-21T21:41:36.092 INFO:teuthology.orchestra.run.smithi164.stdout:3543 write 0x7943c00 thru 0x794a9de (0x6ddf bytes) 2024-09-21T21:41:36.093 INFO:teuthology.orchestra.run.smithi164.stdout:3544 punch from 0x6472400 to 0x6480200, (0xde00 bytes) 2024-09-21T21:41:36.093 INFO:teuthology.orchestra.run.smithi164.stdout:3545 punch from 0x23fa00 to 0x24c800, (0xce00 bytes) 2024-09-21T21:41:36.093 INFO:teuthology.orchestra.run.smithi164.stdout:3548 write 0x31df000 thru 0x31ee345 (0xf346 bytes) 2024-09-21T21:41:36.095 INFO:teuthology.orchestra.run.smithi164.stdout:3549 read 0x13ef400 thru 0x13fa054 (0xac55 bytes) 2024-09-21T21:41:36.095 INFO:teuthology.orchestra.run.smithi164.stdout:3550 read 0x604e600 thru 0x605037f (0x1d80 bytes) 2024-09-21T21:41:36.099 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:36.099 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:36.108 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T21:41:36.110 INFO:teuthology.orchestra.run.smithi164.stdout:3552 write 0xe377e00 thru 0xe380b20 (0x8d21 bytes) 2024-09-21T21:41:36.110 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:36.109+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:36.115 INFO:teuthology.orchestra.run.smithi164.stdout:3555 write 0x38c2000 thru 0x38d1ff1 (0xfff2 bytes) 2024-09-21T21:41:36.116 INFO:teuthology.orchestra.run.smithi164.stdout:3557 punch from 0x805c000 to 0x8060200, (0x4200 bytes) 2024-09-21T21:41:36.129 INFO:teuthology.orchestra.run.smithi164.stdout:3559 trunc from 0xe380c00 to 0x3f59c00 2024-09-21T21:41:36.152 INFO:teuthology.orchestra.run.smithi164.stdout:3560 punch from 0x3991e00 to 0x3999600, (0x7800 bytes) 2024-09-21T21:41:36.169 INFO:teuthology.orchestra.run.smithi164.stdout:3561 trunc from 0x3f59c00 to 0xbf62400 2024-09-21T21:41:36.178 INFO:teuthology.orchestra.run.smithi164.stdout:3564 punch from 0x3de5400 to 0x3deba00, (0x6600 bytes) 2024-09-21T21:41:36.187 INFO:teuthology.orchestra.run.smithi164.stdout:3566 write 0x2a9c600 thru 0x2aaafea (0xe9eb bytes) 2024-09-21T21:41:36.187 INFO:teuthology.orchestra.run.smithi164.stdout:3571 read 0x284c00 thru 0x28b746 (0x6b47 bytes) 2024-09-21T21:41:36.187 INFO:teuthology.orchestra.run.smithi164.stdout:3574 write 0x3defc00 thru 0x3dffa6f (0xfe70 bytes) 2024-09-21T21:41:36.188 INFO:teuthology.orchestra.run.smithi164.stdout:3575 read 0x533e000 thru 0x534b3bc (0xd3bd bytes) 2024-09-21T21:41:36.201 INFO:teuthology.orchestra.run.smithi164.stdout:3576 write 0xd553a00 thru 0xd560019 (0xc61a bytes) 2024-09-21T21:41:36.206 INFO:teuthology.orchestra.run.smithi164.stdout:3577 read 0x1a0ec00 thru 0x1a12185 (0x3586 bytes) 2024-09-21T21:41:36.207 INFO:teuthology.orchestra.run.smithi164.stdout:3578 punch from 0x1d0dc00 to 0x1d19000, (0xb400 bytes) 2024-09-21T21:41:36.211 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:36.209+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:36.215 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:36.216 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:36.222 INFO:teuthology.orchestra.run.smithi164.stdout:3579 trunc from 0xd560200 to 0xdaea00 2024-09-21T21:41:36.224 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T21:41:36.262 INFO:teuthology.orchestra.run.smithi164.stdout:3580 write 0x4a93600 thru 0x4aa279b (0xf19c bytes) 2024-09-21T21:41:36.278 INFO:teuthology.orchestra.run.smithi164.stdout:3581 write 0x917d400 thru 0x91870e6 (0x9ce7 bytes) 2024-09-21T21:41:36.284 INFO:teuthology.orchestra.run.smithi164.stdout:3582 read 0x81a1600 thru 0x81b0272 (0xec73 bytes) 2024-09-21T21:41:36.284 INFO:teuthology.orchestra.run.smithi164.stdout:3584 read 0x67d6e00 thru 0x67d77b0 (0x9b1 bytes) 2024-09-21T21:41:36.284 INFO:teuthology.orchestra.run.smithi164.stdout:3585 write 0x232c800 thru 0x2334665 (0x7e66 bytes) 2024-09-21T21:41:36.285 INFO:teuthology.orchestra.run.smithi164.stdout:3589 write 0x1e1c200 thru 0x1e2159e (0x539f bytes) 2024-09-21T21:41:36.289 INFO:teuthology.orchestra.run.smithi164.stdout:3590 trunc from 0x9187200 to 0xc533c00 2024-09-21T21:41:36.302 INFO:teuthology.orchestra.run.smithi164.stdout:3591 write 0x6c0da00 thru 0x6c1b214 (0xd815 bytes) 2024-09-21T21:41:36.307 INFO:teuthology.orchestra.run.smithi164.stdout:3592 read 0xa37ec00 thru 0xa386799 (0x7b9a bytes) 2024-09-21T21:41:36.307 INFO:teuthology.orchestra.run.smithi164.stdout:3594 read 0xaebd800 thru 0xaecd5e3 (0xfde4 bytes) 2024-09-21T21:41:36.307 INFO:teuthology.orchestra.run.smithi164.stdout:3595 punch from 0x48bf400 to 0x48c8c00, (0x9800 bytes) 2024-09-21T21:41:36.308 INFO:teuthology.orchestra.run.smithi164.stdout:3596 read 0x426f000 thru 0x427501a (0x601b bytes) 2024-09-21T21:41:36.309 INFO:teuthology.orchestra.run.smithi164.stdout:3597 write 0x3a90400 thru 0x3a9bb8e (0xb78f bytes) 2024-09-21T21:41:36.309 INFO:teuthology.orchestra.run.smithi164.stdout:3598 read 0x186c200 thru 0x186d6fc (0x14fd bytes) 2024-09-21T21:41:36.312 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:36.311+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:36.321 INFO:teuthology.orchestra.run.smithi164.stdout:3599 write 0xccdd200 thru 0xcce9cb5 (0xcab6 bytes) 2024-09-21T21:41:36.321 INFO:teuthology.orchestra.run.smithi164.stdout:3602 write 0x89ae00 thru 0x8a1754 (0x6955 bytes) 2024-09-21T21:41:36.325 INFO:teuthology.orchestra.run.smithi164.stdout:3603 write 0x7c8b000 thru 0x7c8c230 (0x1231 bytes) 2024-09-21T21:41:36.337 INFO:teuthology.orchestra.run.smithi164.stdout:3604 write 0xcf30000 thru 0xcf37b9a (0x7b9b bytes) 2024-09-21T21:41:36.337 INFO:teuthology.orchestra.run.smithi164.stdout:3610 read 0x5f90a00 thru 0x5f93563 (0x2b64 bytes) 2024-09-21T21:41:36.348 INFO:teuthology.orchestra.run.smithi164.stdout:3611 trunc from 0xcf37c00 to 0x4186200 2024-09-21T21:41:36.370 INFO:teuthology.orchestra.run.smithi164.stdout:3612 read 0x2e3aa00 thru 0x2e41ce9 (0x72ea bytes) 2024-09-21T21:41:36.375 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:36.375 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:36.384 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T21:41:36.393 INFO:teuthology.orchestra.run.smithi164.stdout:3615 write 0xd1f4a00 thru 0xd1f8254 (0x3855 bytes) 2024-09-21T21:41:36.394 INFO:teuthology.orchestra.run.smithi164.stdout:3616 read 0x9bf4e00 thru 0x9bf62ea (0x14eb bytes) 2024-09-21T21:41:36.394 INFO:teuthology.orchestra.run.smithi164.stdout:3617 punch from 0x763fc00 to 0x7646800, (0x6c00 bytes) 2024-09-21T21:41:36.396 INFO:teuthology.orchestra.run.smithi164.stdout:3618 trunc from 0xd1f8400 to 0xcfbde00 2024-09-21T21:41:36.409 INFO:teuthology.orchestra.run.smithi164.stdout:3622 punch from 0x9986a00 to 0x9987e00, (0x1400 bytes) 2024-09-21T21:41:36.411 INFO:teuthology.orchestra.run.smithi164.stdout:3624 punch from 0xba88e00 to 0xba91c00, (0x8e00 bytes) 2024-09-21T21:41:36.411 INFO:teuthology.orchestra.run.smithi164.stdout:3625 punch from 0xc281c00 to 0xc290800, (0xec00 bytes) 2024-09-21T21:41:36.411 INFO:teuthology.orchestra.run.smithi164.stdout:3626 read 0x154ce00 thru 0x155a7fa (0xd9fb bytes) 2024-09-21T21:41:36.412 INFO:teuthology.orchestra.run.smithi164.stdout:3628 write 0x1b2a000 thru 0x1b2bfd2 (0x1fd3 bytes) 2024-09-21T21:41:36.422 INFO:teuthology.orchestra.run.smithi164.stdout:3629 trunc from 0xcfbde00 to 0x4d7c800 2024-09-21T21:41:36.428 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:36.427+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:36.444 INFO:teuthology.orchestra.run.smithi164.stdout:3630 clone 26 order 24 su 4194304 sc 7 2024-09-21T21:41:36.498 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:36.499 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:36.507 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T21:41:36.513 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:36.512+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:36.614 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:36.613+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:36.650 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:36.651 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:36.660 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T21:41:36.712 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T21:41:36.718 INFO:teuthology.orchestra.run.smithi081.stderr:noscrub is set 2024-09-21T21:41:36.718 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:36.716+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:36.786 INFO:teuthology.orchestra.run.smithi164.stdout:truncating image image_client.0-clone25 from 0x4d7c800 (overlap 0xdaea00) to 0xc50400 2024-09-21T21:41:36.814 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:36.814 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:36.815 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:36.814+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:36.824 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T21:41:36.836 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0: {} 2024-09-21T21:41:36.837 INFO:teuthology.orchestra.run.smithi081.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T21:41:36.854 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1: {} 2024-09-21T21:41:36.855 INFO:teuthology.orchestra.run.smithi081.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T21:41:36.870 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2: {} 2024-09-21T21:41:36.870 INFO:teuthology.orchestra.run.smithi081.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T21:41:36.887 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3: {} 2024-09-21T21:41:36.887 INFO:teuthology.orchestra.run.smithi081.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T21:41:36.903 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4: {} 2024-09-21T21:41:36.903 INFO:teuthology.orchestra.run.smithi081.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T21:41:36.916 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:36.915+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:36.921 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5: {} 2024-09-21T21:41:36.922 INFO:teuthology.orchestra.run.smithi081.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T21:41:36.973 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:36.973 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:41:36.973 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-21T21:41:36.973 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:36.982 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T21:41:37.026 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:37.025+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:37.089 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:37.089 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:41:37.089 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:41:37.089 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:41:37.089 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:37.094 INFO:teuthology.orchestra.run.smithi164.stdout:checking clone #24, image image_client.0-clone24 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent25 2024-09-21T21:41:37.098 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T21:41:37.139 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:37.138+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:37.219 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:37.218+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:37.285 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:37.285 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-21T21:41:37.285 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-21T21:41:37.285 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:41:37.285 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.285 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6972 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5238784~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:37.286 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.922911+0000", 2024-09-21T21:41:37.286 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 7.3596416820000004, 2024-09-21T21:41:37.286 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041605798999999999, 2024-09-21T21:41:37.286 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:37.286 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:37.286 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:37.286 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:37.286 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:37.286 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:37.286 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6972 2024-09-21T21:41:37.286 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.286 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:37.286 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.286 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:37.287 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922911+0000", 2024-09-21T21:41:37.287 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:37.287 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.287 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.287 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:37.287 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922911+0000", 2024-09-21T21:41:37.287 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:37.287 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.287 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.287 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:37.287 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922912+0000", 2024-09-21T21:41:37.287 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8999999999999996e-07 2024-09-21T21:41:37.287 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.287 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.287 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:37.288 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922914+0000", 2024-09-21T21:41:37.288 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1050000000000002e-06 2024-09-21T21:41:37.288 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.288 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.288 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:37.288 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922920+0000", 2024-09-21T21:41:37.288 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4380000000000003e-06 2024-09-21T21:41:37.288 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.288 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.288 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:37.288 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922925+0000", 2024-09-21T21:41:37.288 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6300000000000003e-06 2024-09-21T21:41:37.288 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.288 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.288 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:37.289 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964424+0000", 2024-09-21T21:41:37.289 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041498342000000001 2024-09-21T21:41:37.289 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.289 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.289 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:37.289 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964466+0000", 2024-09-21T21:41:37.289 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2245e-05 2024-09-21T21:41:37.289 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.289 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.289 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:37.289 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964517+0000", 2024-09-21T21:41:37.289 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1449000000000002e-05 2024-09-21T21:41:37.289 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.289 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:37.289 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.290 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.290 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.290 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6974 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5246976~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:37.290 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.922932+0000", 2024-09-21T21:41:37.290 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 7.3596206510000002, 2024-09-21T21:41:37.290 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041773352999999999, 2024-09-21T21:41:37.290 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:37.290 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:37.290 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:37.290 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:37.290 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:37.290 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:37.290 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6974 2024-09-21T21:41:37.290 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.291 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:37.291 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.291 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:37.291 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922932+0000", 2024-09-21T21:41:37.291 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:37.291 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.291 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.291 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:37.291 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922932+0000", 2024-09-21T21:41:37.291 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:37.291 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.291 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.291 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:37.291 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922933+0000", 2024-09-21T21:41:37.291 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.46e-07 2024-09-21T21:41:37.292 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.292 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.292 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:37.292 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922940+0000", 2024-09-21T21:41:37.292 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6140000000000003e-06 2024-09-21T21:41:37.292 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.292 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.292 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:37.292 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922949+0000", 2024-09-21T21:41:37.292 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6899999999999998e-06 2024-09-21T21:41:37.292 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.292 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.292 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:37.292 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922955+0000", 2024-09-21T21:41:37.292 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5140000000000001e-06 2024-09-21T21:41:37.293 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.293 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.293 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:37.293 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964547+0000", 2024-09-21T21:41:37.293 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041592854999999998 2024-09-21T21:41:37.293 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.293 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.293 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:37.293 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964617+0000", 2024-09-21T21:41:37.293 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9058000000000002e-05 2024-09-21T21:41:37.293 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.293 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.293 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:37.293 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964706+0000", 2024-09-21T21:41:37.293 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9276000000000004e-05 2024-09-21T21:41:37.294 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.294 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:37.294 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.294 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.294 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.294 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6976 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5261312~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:37.294 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.922961+0000", 2024-09-21T21:41:37.294 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 7.3595916060000004, 2024-09-21T21:41:37.294 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041863648000000003, 2024-09-21T21:41:37.294 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:37.294 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:37.294 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:37.294 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:37.294 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:37.295 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:37.295 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6976 2024-09-21T21:41:37.295 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.295 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:37.295 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.295 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:37.295 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922961+0000", 2024-09-21T21:41:37.295 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:37.295 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.295 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.295 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:37.295 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922961+0000", 2024-09-21T21:41:37.295 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:37.295 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.295 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.296 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:37.296 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922962+0000", 2024-09-21T21:41:37.296 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.89e-07 2024-09-21T21:41:37.296 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.296 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.296 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:37.296 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922964+0000", 2024-09-21T21:41:37.296 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1270000000000001e-06 2024-09-21T21:41:37.296 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.296 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.296 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:37.296 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922970+0000", 2024-09-21T21:41:37.296 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8030000000000002e-06 2024-09-21T21:41:37.296 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.296 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.297 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:37.297 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922979+0000", 2024-09-21T21:41:37.297 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5559999999999996e-06 2024-09-21T21:41:37.297 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.297 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.297 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:37.297 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964734+0000", 2024-09-21T21:41:37.297 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041754415000000003 2024-09-21T21:41:37.297 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.297 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.297 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:37.297 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964775+0000", 2024-09-21T21:41:37.297 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0806999999999997e-05 2024-09-21T21:41:37.297 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.297 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.298 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:37.298 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964825+0000", 2024-09-21T21:41:37.298 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0550999999999998e-05 2024-09-21T21:41:37.298 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.298 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:37.298 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.298 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.298 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.298 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6978 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5275648~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:37.298 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.922987+0000", 2024-09-21T21:41:37.298 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 7.3595658210000003, 2024-09-21T21:41:37.298 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042043876000000001, 2024-09-21T21:41:37.298 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:37.298 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:37.299 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:37.299 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:37.299 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:37.299 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:37.299 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6978 2024-09-21T21:41:37.299 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.299 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:37.299 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.299 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:37.299 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922987+0000", 2024-09-21T21:41:37.299 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:37.299 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.299 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.299 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:37.299 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922987+0000", 2024-09-21T21:41:37.300 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:37.300 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.300 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.300 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:37.300 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922988+0000", 2024-09-21T21:41:37.300 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1199999999999998e-07 2024-09-21T21:41:37.300 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.300 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.300 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:37.300 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922990+0000", 2024-09-21T21:41:37.300 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.142e-06 2024-09-21T21:41:37.300 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.300 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.300 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:37.300 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922997+0000", 2024-09-21T21:41:37.301 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.199e-06 2024-09-21T21:41:37.301 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.301 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.301 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:37.301 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923004+0000", 2024-09-21T21:41:37.301 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3599999999999998e-06 2024-09-21T21:41:37.301 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.301 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.301 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:37.301 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964856+0000", 2024-09-21T21:41:37.301 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041851521000000003 2024-09-21T21:41:37.301 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.301 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.301 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:37.301 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964944+0000", 2024-09-21T21:41:37.302 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7859e-05 2024-09-21T21:41:37.302 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.302 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.302 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:37.302 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965031+0000", 2024-09-21T21:41:37.302 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7383e-05 2024-09-21T21:41:37.302 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.302 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:37.302 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.302 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.302 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.302 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6980 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5289984~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:37.302 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923011+0000", 2024-09-21T21:41:37.302 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 7.3595424840000003, 2024-09-21T21:41:37.303 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042139337999999998, 2024-09-21T21:41:37.303 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:37.303 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:37.303 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:37.303 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:37.303 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:37.303 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:37.303 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6980 2024-09-21T21:41:37.303 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.303 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:37.303 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.303 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:37.303 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923011+0000", 2024-09-21T21:41:37.303 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:37.303 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.304 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.304 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:37.304 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923011+0000", 2024-09-21T21:41:37.304 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:37.304 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.304 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.304 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:37.304 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923011+0000", 2024-09-21T21:41:37.304 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6699999999999999e-07 2024-09-21T21:41:37.304 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.304 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.304 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:37.304 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923013+0000", 2024-09-21T21:41:37.304 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0279999999999999e-06 2024-09-21T21:41:37.304 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.305 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.305 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:37.305 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923018+0000", 2024-09-21T21:41:37.305 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4790000000000004e-06 2024-09-21T21:41:37.305 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.305 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.305 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:37.305 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923029+0000", 2024-09-21T21:41:37.305 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0601999999999999e-05 2024-09-21T21:41:37.305 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.305 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.305 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:37.305 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965059+0000", 2024-09-21T21:41:37.305 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042029947999999998 2024-09-21T21:41:37.305 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.306 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.306 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:37.306 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965100+0000", 2024-09-21T21:41:37.306 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1170000000000001e-05 2024-09-21T21:41:37.306 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.306 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.306 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:37.306 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965150+0000", 2024-09-21T21:41:37.306 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9743999999999997e-05 2024-09-21T21:41:37.306 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.306 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:37.306 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.306 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.306 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.306 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6982 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5304320~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:37.307 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923036+0000", 2024-09-21T21:41:37.307 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 7.359516663, 2024-09-21T21:41:37.307 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042225666000000002, 2024-09-21T21:41:37.307 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:37.307 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:37.307 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:37.307 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:37.307 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:37.307 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:37.307 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6982 2024-09-21T21:41:37.307 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.307 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:37.307 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.307 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:37.307 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923036+0000", 2024-09-21T21:41:37.308 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:37.308 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.308 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.308 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:37.308 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923036+0000", 2024-09-21T21:41:37.308 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:37.308 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.308 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.308 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:37.308 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923037+0000", 2024-09-21T21:41:37.308 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2800000000000002e-07 2024-09-21T21:41:37.308 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.308 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.308 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:37.309 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923039+0000", 2024-09-21T21:41:37.309 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1500000000000002e-06 2024-09-21T21:41:37.309 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.309 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.309 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:37.309 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923049+0000", 2024-09-21T21:41:37.309 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0236e-05 2024-09-21T21:41:37.309 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.309 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.309 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:37.309 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923056+0000", 2024-09-21T21:41:37.309 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2540000000000003e-06 2024-09-21T21:41:37.309 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.309 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.309 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:37.310 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965173+0000", 2024-09-21T21:41:37.310 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042116686 2024-09-21T21:41:37.310 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.310 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.310 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:37.310 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965215+0000", 2024-09-21T21:41:37.310 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1696000000000002e-05 2024-09-21T21:41:37.310 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.310 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.310 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:37.310 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965262+0000", 2024-09-21T21:41:37.310 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7216000000000003e-05 2024-09-21T21:41:37.310 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.310 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:37.311 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.311 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.311 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.311 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6984 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5318656~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:37.311 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923063+0000", 2024-09-21T21:41:37.311 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 7.3594900089999999, 2024-09-21T21:41:37.311 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042318651999999998, 2024-09-21T21:41:37.311 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:37.311 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:37.311 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:37.311 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:37.311 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:37.311 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:37.311 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6984 2024-09-21T21:41:37.312 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.312 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:37.312 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.312 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:37.312 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923063+0000", 2024-09-21T21:41:37.312 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:37.312 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.312 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.312 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:37.312 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923063+0000", 2024-09-21T21:41:37.312 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:37.312 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.312 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.312 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:37.312 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923064+0000", 2024-09-21T21:41:37.313 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2e-07 2024-09-21T21:41:37.313 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.313 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.313 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:37.313 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923066+0000", 2024-09-21T21:41:37.313 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.057e-06 2024-09-21T21:41:37.313 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.313 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.313 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:37.313 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923071+0000", 2024-09-21T21:41:37.313 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8810000000000001e-06 2024-09-21T21:41:37.313 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.313 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.313 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:37.314 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923078+0000", 2024-09-21T21:41:37.314 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5479999999999998e-06 2024-09-21T21:41:37.314 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.314 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.314 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:37.314 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965288+0000", 2024-09-21T21:41:37.314 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042209576999999998 2024-09-21T21:41:37.314 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.314 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.314 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:37.314 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965332+0000", 2024-09-21T21:41:37.314 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4366999999999997e-05 2024-09-21T21:41:37.314 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.314 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.314 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:37.315 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965382+0000", 2024-09-21T21:41:37.315 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9802000000000002e-05 2024-09-21T21:41:37.315 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.315 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:37.315 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.315 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.315 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.315 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6987 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5332992~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:37.315 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923085+0000", 2024-09-21T21:41:37.315 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 7.3594684839999998, 2024-09-21T21:41:37.315 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042412103999999999, 2024-09-21T21:41:37.315 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:37.315 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:37.315 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:37.315 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:37.316 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:37.316 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:37.316 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6987 2024-09-21T21:41:37.316 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.316 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:37.316 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.316 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:37.316 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923085+0000", 2024-09-21T21:41:37.316 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:37.316 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.316 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.316 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:37.316 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923085+0000", 2024-09-21T21:41:37.316 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:37.316 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.316 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.317 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:37.317 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923085+0000", 2024-09-21T21:41:37.317 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.89e-07 2024-09-21T21:41:37.317 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.317 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.317 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:37.317 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923087+0000", 2024-09-21T21:41:37.317 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0810000000000001e-06 2024-09-21T21:41:37.317 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.317 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.318 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:37.318 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923098+0000", 2024-09-21T21:41:37.318 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1299e-05 2024-09-21T21:41:37.318 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.318 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.318 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:37.318 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923104+0000", 2024-09-21T21:41:37.318 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9100000000000002e-06 2024-09-21T21:41:37.318 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.318 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.318 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:37.318 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965404+0000", 2024-09-21T21:41:37.318 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042299622000000002 2024-09-21T21:41:37.318 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.319 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.319 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:37.319 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965445+0000", 2024-09-21T21:41:37.319 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0701000000000002e-05 2024-09-21T21:41:37.319 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.319 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.319 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:37.319 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965497+0000", 2024-09-21T21:41:37.319 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2101999999999997e-05 2024-09-21T21:41:37.319 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.319 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:37.319 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.319 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.319 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.320 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6989 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5347328~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:37.320 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923111+0000", 2024-09-21T21:41:37.320 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 7.359442144, 2024-09-21T21:41:37.320 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042505188999999999, 2024-09-21T21:41:37.320 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:37.320 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:37.320 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:37.320 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:37.320 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:37.320 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:37.320 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6989 2024-09-21T21:41:37.320 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.320 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:37.320 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.320 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:37.321 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923111+0000", 2024-09-21T21:41:37.321 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:37.321 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.321 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.321 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:37.321 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923111+0000", 2024-09-21T21:41:37.321 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:37.321 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.321 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.321 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:37.321 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923111+0000", 2024-09-21T21:41:37.321 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4499999999999997e-07 2024-09-21T21:41:37.321 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.321 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.321 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:37.322 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923114+0000", 2024-09-21T21:41:37.322 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1569999999999998e-06 2024-09-21T21:41:37.322 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.322 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.322 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:37.322 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923120+0000", 2024-09-21T21:41:37.322 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1759999999999998e-06 2024-09-21T21:41:37.322 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.322 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.322 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:37.322 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923125+0000", 2024-09-21T21:41:37.322 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2560000000000002e-06 2024-09-21T21:41:37.322 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.322 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.322 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:37.323 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965523+0000", 2024-09-21T21:41:37.323 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042398094999999997 2024-09-21T21:41:37.323 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.323 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.323 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:37.323 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965566+0000", 2024-09-21T21:41:37.323 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3393999999999998e-05 2024-09-21T21:41:37.323 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.323 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.323 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:37.323 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965616+0000", 2024-09-21T21:41:37.323 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9666000000000002e-05 2024-09-21T21:41:37.323 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.323 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:37.324 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.324 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.324 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.324 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6991 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5361664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:37.324 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923132+0000", 2024-09-21T21:41:37.324 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 7.3594208549999998, 2024-09-21T21:41:37.324 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042600779999999998, 2024-09-21T21:41:37.324 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:37.324 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:37.324 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:37.324 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:37.324 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:37.324 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:37.324 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6991 2024-09-21T21:41:37.324 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.325 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:37.325 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.325 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:37.325 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923132+0000", 2024-09-21T21:41:37.325 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:37.325 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.325 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.325 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:37.325 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923132+0000", 2024-09-21T21:41:37.325 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:37.325 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.325 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.325 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:37.325 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923132+0000", 2024-09-21T21:41:37.325 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4200000000000002e-07 2024-09-21T21:41:37.326 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.326 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.326 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:37.326 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923134+0000", 2024-09-21T21:41:37.326 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2270000000000001e-06 2024-09-21T21:41:37.326 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.326 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.326 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:37.326 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923137+0000", 2024-09-21T21:41:37.326 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1930000000000002e-06 2024-09-21T21:41:37.326 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.326 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.326 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:37.326 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923139+0000", 2024-09-21T21:41:37.326 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4710000000000001e-06 2024-09-21T21:41:37.327 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.327 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.327 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:37.327 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965639+0000", 2024-09-21T21:41:37.327 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.04249965 2024-09-21T21:41:37.327 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.327 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.327 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:37.327 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965679+0000", 2024-09-21T21:41:37.327 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9852999999999999e-05 2024-09-21T21:41:37.327 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.327 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.327 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:37.327 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965733+0000", 2024-09-21T21:41:37.327 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4144000000000002e-05 2024-09-21T21:41:37.328 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.328 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:37.328 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.328 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.328 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.328 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6993 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5384192~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:37.328 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923143+0000", 2024-09-21T21:41:37.328 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 7.3594100610000002, 2024-09-21T21:41:37.328 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042708182999999997, 2024-09-21T21:41:37.328 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:37.328 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:37.328 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:37.328 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:37.328 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:37.328 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:37.329 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6993 2024-09-21T21:41:37.329 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.329 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:37.329 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.329 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:37.329 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923143+0000", 2024-09-21T21:41:37.329 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:37.329 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.329 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.329 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:37.329 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923143+0000", 2024-09-21T21:41:37.329 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:37.329 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.329 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.329 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:37.330 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923143+0000", 2024-09-21T21:41:37.330 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0800000000000001e-07 2024-09-21T21:41:37.330 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.330 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.330 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:37.330 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923144+0000", 2024-09-21T21:41:37.330 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.009e-06 2024-09-21T21:41:37.330 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.330 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.330 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:37.330 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923148+0000", 2024-09-21T21:41:37.330 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.737e-06 2024-09-21T21:41:37.331 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.331 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.331 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:37.331 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923150+0000", 2024-09-21T21:41:37.331 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4669999999999998e-06 2024-09-21T21:41:37.331 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.331 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.331 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:37.331 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965760+0000", 2024-09-21T21:41:37.331 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042609399999999999 2024-09-21T21:41:37.331 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.331 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.331 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:37.332 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965801+0000", 2024-09-21T21:41:37.332 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1399999999999997e-05 2024-09-21T21:41:37.332 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.332 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.332 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:37.332 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965851+0000", 2024-09-21T21:41:37.332 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9962e-05 2024-09-21T21:41:37.332 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.332 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:37.332 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.332 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.332 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.333 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6995 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5398528~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:37.333 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923153+0000", 2024-09-21T21:41:37.333 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 7.3593996969999997, 2024-09-21T21:41:37.333 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042828034000000001, 2024-09-21T21:41:37.333 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:37.333 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:37.333 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:37.333 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:37.333 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:37.333 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:37.333 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6995 2024-09-21T21:41:37.333 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.333 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:37.333 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.334 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:37.334 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923153+0000", 2024-09-21T21:41:37.334 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:37.334 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.334 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.334 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:37.334 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923153+0000", 2024-09-21T21:41:37.334 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:37.334 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.334 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.334 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:37.334 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923154+0000", 2024-09-21T21:41:37.334 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2600000000000001e-07 2024-09-21T21:41:37.334 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.334 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.335 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:37.335 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923158+0000", 2024-09-21T21:41:37.335 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0559999999999998e-06 2024-09-21T21:41:37.335 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.335 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.335 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:37.335 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923161+0000", 2024-09-21T21:41:37.335 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6519999999999999e-06 2024-09-21T21:41:37.335 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.335 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.335 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:37.335 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923170+0000", 2024-09-21T21:41:37.335 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5720000000000003e-06 2024-09-21T21:41:37.335 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.335 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.336 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:37.336 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965887+0000", 2024-09-21T21:41:37.336 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042716863000000001 2024-09-21T21:41:37.336 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.336 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.336 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:37.336 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965927+0000", 2024-09-21T21:41:37.336 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0346999999999998e-05 2024-09-21T21:41:37.336 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.336 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.336 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:37.336 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965981+0000", 2024-09-21T21:41:37.336 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4317999999999997e-05 2024-09-21T21:41:37.336 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.337 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:37.337 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.337 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.337 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.337 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6997 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5412864~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:37.337 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923174+0000", 2024-09-21T21:41:37.337 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 7.3593795420000001, 2024-09-21T21:41:37.337 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042928296999999997, 2024-09-21T21:41:37.337 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:37.337 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:37.337 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:37.337 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:37.337 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:37.337 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:37.337 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6997 2024-09-21T21:41:37.338 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.338 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:37.338 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.338 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:37.338 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923174+0000", 2024-09-21T21:41:37.338 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:37.338 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.338 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.338 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:37.338 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923174+0000", 2024-09-21T21:41:37.338 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:37.338 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.338 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.338 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:37.338 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923174+0000", 2024-09-21T21:41:37.339 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.67e-07 2024-09-21T21:41:37.339 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.339 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.339 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:37.339 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923178+0000", 2024-09-21T21:41:37.339 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1330000000000001e-06 2024-09-21T21:41:37.339 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.339 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.339 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:37.339 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923182+0000", 2024-09-21T21:41:37.339 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.794e-06 2024-09-21T21:41:37.339 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.339 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.339 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:37.339 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923184+0000", 2024-09-21T21:41:37.340 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.435e-06 2024-09-21T21:41:37.340 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.340 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.340 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:37.340 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966009+0000", 2024-09-21T21:41:37.340 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042824327000000002 2024-09-21T21:41:37.340 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.340 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.340 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:37.340 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966052+0000", 2024-09-21T21:41:37.340 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3034000000000002e-05 2024-09-21T21:41:37.340 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.340 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.340 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:37.340 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966102+0000", 2024-09-21T21:41:37.341 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0306999999999998e-05 2024-09-21T21:41:37.341 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.341 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:37.341 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.341 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.341 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.341 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6999 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5427200~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:37.341 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923187+0000", 2024-09-21T21:41:37.341 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 7.3593658140000002, 2024-09-21T21:41:37.341 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043104267000000002, 2024-09-21T21:41:37.341 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:37.341 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:37.341 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:37.341 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:37.341 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:37.342 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:37.342 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6999 2024-09-21T21:41:37.342 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.342 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:37.342 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.342 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:37.342 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923187+0000", 2024-09-21T21:41:37.342 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:37.342 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.342 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.342 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:37.342 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923187+0000", 2024-09-21T21:41:37.342 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:37.342 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.343 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.343 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:37.343 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923188+0000", 2024-09-21T21:41:37.343 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3999999999999998e-07 2024-09-21T21:41:37.343 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.343 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.343 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:37.343 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923189+0000", 2024-09-21T21:41:37.343 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4169999999999999e-06 2024-09-21T21:41:37.343 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.343 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.343 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:37.343 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923194+0000", 2024-09-21T21:41:37.343 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1819999999999997e-06 2024-09-21T21:41:37.343 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.343 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.344 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:37.344 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923198+0000", 2024-09-21T21:41:37.344 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.143e-06 2024-09-21T21:41:37.344 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.344 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.344 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:37.344 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966135+0000", 2024-09-21T21:41:37.344 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042936297999999998 2024-09-21T21:41:37.344 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.344 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.344 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:37.344 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966204+0000", 2024-09-21T21:41:37.344 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8989e-05 2024-09-21T21:41:37.344 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.344 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.345 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:37.345 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966292+0000", 2024-09-21T21:41:37.345 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7997999999999993e-05 2024-09-21T21:41:37.345 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.345 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:37.345 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.345 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.345 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.345 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7001 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5441536~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:37.345 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923211+0000", 2024-09-21T21:41:37.345 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 7.359342023, 2024-09-21T21:41:37.345 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043196518000000003, 2024-09-21T21:41:37.345 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:37.345 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:37.346 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:37.346 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:37.346 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:37.346 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:37.346 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7001 2024-09-21T21:41:37.346 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.346 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:37.346 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.346 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:37.346 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923211+0000", 2024-09-21T21:41:37.346 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:37.346 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.346 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.346 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:37.346 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923211+0000", 2024-09-21T21:41:37.347 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:37.347 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.347 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.347 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:37.347 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923212+0000", 2024-09-21T21:41:37.347 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3900000000000005e-07 2024-09-21T21:41:37.347 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.347 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.347 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:37.347 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923214+0000", 2024-09-21T21:41:37.347 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9970000000000001e-06 2024-09-21T21:41:37.347 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.347 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.347 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:37.347 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923219+0000", 2024-09-21T21:41:37.348 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4310000000000002e-06 2024-09-21T21:41:37.348 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.348 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.348 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:37.348 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923224+0000", 2024-09-21T21:41:37.348 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6410000000000001e-06 2024-09-21T21:41:37.348 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.348 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.348 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:37.348 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966318+0000", 2024-09-21T21:41:37.348 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043093842 2024-09-21T21:41:37.348 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.348 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.348 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:37.349 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966359+0000", 2024-09-21T21:41:37.349 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1260999999999997e-05 2024-09-21T21:41:37.349 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.349 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.349 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:37.349 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966408+0000", 2024-09-21T21:41:37.349 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8807000000000002e-05 2024-09-21T21:41:37.349 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.349 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:37.349 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.349 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.349 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.349 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7002 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5455872~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:37.349 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923229+0000", 2024-09-21T21:41:37.349 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 7.3593236129999999, 2024-09-21T21:41:37.350 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043364568999999999, 2024-09-21T21:41:37.350 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:37.350 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:37.350 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:37.350 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:37.350 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:37.350 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:37.350 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7002 2024-09-21T21:41:37.350 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.350 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:37.350 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.350 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:37.350 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923229+0000", 2024-09-21T21:41:37.350 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:37.351 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.351 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.351 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:37.351 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923229+0000", 2024-09-21T21:41:37.351 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:37.351 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.351 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.351 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:37.351 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923230+0000", 2024-09-21T21:41:37.351 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4700000000000002e-07 2024-09-21T21:41:37.351 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.351 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.351 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:37.351 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923232+0000", 2024-09-21T21:41:37.352 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0719999999999998e-06 2024-09-21T21:41:37.352 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.352 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.352 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:37.352 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923237+0000", 2024-09-21T21:41:37.352 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2290000000000002e-06 2024-09-21T21:41:37.352 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.352 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.352 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:37.352 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923245+0000", 2024-09-21T21:41:37.352 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2959999999999999e-06 2024-09-21T21:41:37.352 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.352 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.352 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:37.352 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966438+0000", 2024-09-21T21:41:37.353 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043193027000000002 2024-09-21T21:41:37.353 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.353 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.353 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:37.353 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966506+0000", 2024-09-21T21:41:37.353 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7707999999999996e-05 2024-09-21T21:41:37.353 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.353 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.353 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:37.353 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966594+0000", 2024-09-21T21:41:37.353 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.789e-05 2024-09-21T21:41:37.353 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.353 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:37.353 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.353 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.354 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.354 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7005 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5470208~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:37.354 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923254+0000", 2024-09-21T21:41:37.354 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 7.3592993819999997, 2024-09-21T21:41:37.354 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043457176, 2024-09-21T21:41:37.354 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:37.354 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:37.354 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:37.354 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:37.354 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:37.354 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:37.354 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7005 2024-09-21T21:41:37.354 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.354 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:37.354 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.355 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:37.355 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923254+0000", 2024-09-21T21:41:37.355 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:37.355 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.355 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.355 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:37.355 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923254+0000", 2024-09-21T21:41:37.355 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:37.355 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.355 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.355 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:37.355 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923254+0000", 2024-09-21T21:41:37.355 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4700000000000002e-07 2024-09-21T21:41:37.355 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.355 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.356 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:37.356 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923256+0000", 2024-09-21T21:41:37.356 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.007e-06 2024-09-21T21:41:37.356 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.356 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.356 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:37.356 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923266+0000", 2024-09-21T21:41:37.356 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.4700000000000008e-06 2024-09-21T21:41:37.356 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.356 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.356 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:37.356 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923271+0000", 2024-09-21T21:41:37.356 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2809999999999999e-06 2024-09-21T21:41:37.356 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.356 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.357 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:37.357 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966619+0000", 2024-09-21T21:41:37.357 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043347951000000003 2024-09-21T21:41:37.357 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.357 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.357 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:37.357 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966661+0000", 2024-09-21T21:41:37.357 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2614999999999998e-05 2024-09-21T21:41:37.357 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.357 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.357 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:37.357 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966711+0000", 2024-09-21T21:41:37.357 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9505000000000001e-05 2024-09-21T21:41:37.357 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.357 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:37.358 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.358 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.358 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.358 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7008 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5484544~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:37.358 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923278+0000", 2024-09-21T21:41:37.358 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 7.3592747349999996, 2024-09-21T21:41:37.358 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043631071, 2024-09-21T21:41:37.358 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:37.358 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:37.358 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:37.358 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:37.358 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:37.358 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:37.358 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7008 2024-09-21T21:41:37.359 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.359 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:37.359 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.359 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:37.359 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923278+0000", 2024-09-21T21:41:37.359 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:37.359 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.359 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.359 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:37.359 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923278+0000", 2024-09-21T21:41:37.359 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:37.359 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.359 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.359 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:37.359 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923279+0000", 2024-09-21T21:41:37.360 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.77e-07 2024-09-21T21:41:37.360 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.360 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.360 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:37.360 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923281+0000", 2024-09-21T21:41:37.360 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0219999999999999e-06 2024-09-21T21:41:37.360 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.360 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.360 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:37.360 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923287+0000", 2024-09-21T21:41:37.360 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7799999999999997e-06 2024-09-21T21:41:37.360 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.360 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.360 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:37.360 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923291+0000", 2024-09-21T21:41:37.361 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1699999999999999e-06 2024-09-21T21:41:37.361 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.361 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.361 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:37.361 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966742+0000", 2024-09-21T21:41:37.361 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043451344000000003 2024-09-21T21:41:37.361 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.361 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.361 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:37.361 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966809+0000", 2024-09-21T21:41:37.361 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7070999999999995e-05 2024-09-21T21:41:37.361 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.361 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.361 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:37.362 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966909+0000", 2024-09-21T21:41:37.362 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000100307 2024-09-21T21:41:37.362 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.362 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:37.362 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.362 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.362 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.362 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7010 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5498880~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:37.362 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923301+0000", 2024-09-21T21:41:37.362 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 7.3592518399999998, 2024-09-21T21:41:37.362 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043717830999999999, 2024-09-21T21:41:37.362 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:37.362 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:37.362 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:37.362 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:37.363 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:37.363 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:37.363 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7010 2024-09-21T21:41:37.363 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.363 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:37.363 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.363 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:37.363 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923301+0000", 2024-09-21T21:41:37.363 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:37.363 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.363 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.363 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:37.363 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923301+0000", 2024-09-21T21:41:37.363 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:37.364 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.364 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.364 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:37.364 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923302+0000", 2024-09-21T21:41:37.364 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.41e-07 2024-09-21T21:41:37.364 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.364 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.364 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:37.364 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923304+0000", 2024-09-21T21:41:37.364 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0990000000000001e-06 2024-09-21T21:41:37.364 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.364 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.364 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:37.364 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923310+0000", 2024-09-21T21:41:37.364 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3860000000000004e-06 2024-09-21T21:41:37.364 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.365 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.365 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:37.365 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923315+0000", 2024-09-21T21:41:37.365 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8459999999999999e-06 2024-09-21T21:41:37.365 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.365 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.365 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:37.365 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966938+0000", 2024-09-21T21:41:37.365 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043622693999999997 2024-09-21T21:41:37.365 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.365 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.365 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:37.365 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966975+0000", 2024-09-21T21:41:37.365 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7608999999999999e-05 2024-09-21T21:41:37.366 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.366 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.366 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:37.366 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.967019+0000", 2024-09-21T21:41:37.366 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3856000000000002e-05 2024-09-21T21:41:37.366 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.366 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:37.366 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.366 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.366 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.366 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7354 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 2107392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:37.366 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.936932+0000", 2024-09-21T21:41:37.366 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 7.345620985, 2024-09-21T21:41:37.366 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041607708, 2024-09-21T21:41:37.366 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:37.367 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:37.367 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:37.367 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:37.367 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:37.367 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:37.367 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7354 2024-09-21T21:41:37.367 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.367 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:37.367 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.367 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:37.367 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.936932+0000", 2024-09-21T21:41:37.367 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:37.367 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.367 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.367 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:37.368 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.936932+0000", 2024-09-21T21:41:37.368 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:37.368 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.368 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.368 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:37.368 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.937276+0000", 2024-09-21T21:41:37.368 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00034344200000000001 2024-09-21T21:41:37.368 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.368 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.368 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:37.368 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.937279+0000", 2024-09-21T21:41:37.368 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.844e-06 2024-09-21T21:41:37.368 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.368 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.368 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:37.369 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.937288+0000", 2024-09-21T21:41:37.369 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2409999999999995e-06 2024-09-21T21:41:37.369 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.369 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.369 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:37.369 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.937293+0000", 2024-09-21T21:41:37.369 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6559999999999997e-06 2024-09-21T21:41:37.369 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.369 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.369 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:37.369 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.978448+0000", 2024-09-21T21:41:37.369 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041154621000000002 2024-09-21T21:41:37.369 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.369 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.369 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:37.370 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.978487+0000", 2024-09-21T21:41:37.370 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9254000000000001e-05 2024-09-21T21:41:37.370 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.370 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.370 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:37.370 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.978540+0000", 2024-09-21T21:41:37.370 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2649999999999999e-05 2024-09-21T21:41:37.370 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.370 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:37.370 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.370 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.370 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:37.370 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:37.372 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T21:41:37.372 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:37.370+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:37.472 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:37.470+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:37.552 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:37.553 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:41:37.553 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.553 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5224.0:10199 3.16 3.7f76da96 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-21T21:41:37.553 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:37.551491+0000", 2024-09-21T21:41:37.553 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 5.8693999999999997e-05, 2024-09-21T21:41:37.553 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3159999999999997e-05, 2024-09-21T21:41:37.553 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:37.553 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:37.553 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:37.553 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:37.553 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5224", 2024-09-21T21:41:37.553 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1601827218", 2024-09-21T21:41:37.553 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 10199 2024-09-21T21:41:37.553 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.554 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:37.554 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.554 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:37.554 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:37.551491+0000", 2024-09-21T21:41:37.554 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:37.554 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.554 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.554 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:37.554 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:37.551491+0000", 2024-09-21T21:41:37.554 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:37.554 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.554 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.554 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:37.554 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:37.551491+0000", 2024-09-21T21:41:37.554 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.92e-07 2024-09-21T21:41:37.555 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.555 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.555 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:37.555 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:37.551495+0000", 2024-09-21T21:41:37.555 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8639999999999998e-06 2024-09-21T21:41:37.555 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.555 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.555 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:37.555 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:37.551503+0000", 2024-09-21T21:41:37.555 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0299999999999994e-06 2024-09-21T21:41:37.555 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.555 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.555 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:37.555 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:37.551510+0000", 2024-09-21T21:41:37.556 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2299999999999996e-06 2024-09-21T21:41:37.556 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.556 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:37.556 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.556 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.556 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-21T21:41:37.556 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 1 2024-09-21T21:41:37.556 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:37.563 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T21:41:37.573 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:37.571+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:37.674 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:37.672+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:37.710 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:37.710 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:41:37.710 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:41:37.710 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:41:37.710 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:37.719 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T21:41:37.775 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:37.774+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:37.823 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:37.823 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-21T21:41:37.823 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-21T21:41:37.823 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:41:37.823 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.823 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1484 3.16 3:6a41a830:::rbd_data.112145fae779.000000000000000a:a [read 303104~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:37.823 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634086+0000", 2024-09-21T21:41:37.823 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 36.18684975, 2024-09-21T21:41:37.823 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.058257055000000002, 2024-09-21T21:41:37.824 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:37.824 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:37.824 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:37.824 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:37.824 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:37.824 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:37.824 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1484 2024-09-21T21:41:37.824 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.824 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:37.824 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.824 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:37.824 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634086+0000", 2024-09-21T21:41:37.824 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:37.824 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.825 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.825 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:37.825 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634086+0000", 2024-09-21T21:41:37.825 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:37.825 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.825 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.825 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:37.825 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634086+0000", 2024-09-21T21:41:37.825 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6100000000000001e-07 2024-09-21T21:41:37.825 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.825 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.825 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:37.825 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634088+0000", 2024-09-21T21:41:37.825 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1100000000000001e-06 2024-09-21T21:41:37.825 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.826 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.826 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:37.826 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634095+0000", 2024-09-21T21:41:37.826 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4080000000000003e-06 2024-09-21T21:41:37.826 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.826 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.826 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:37.826 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634100+0000", 2024-09-21T21:41:37.826 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0730000000000004e-06 2024-09-21T21:41:37.826 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.826 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.826 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:37.826 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.692197+0000", 2024-09-21T21:41:37.826 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.058097331000000002 2024-09-21T21:41:37.827 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.827 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.827 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:37.827 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.692271+0000", 2024-09-21T21:41:37.827 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3423999999999994e-05 2024-09-21T21:41:37.827 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.827 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.827 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:37.827 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.692343+0000", 2024-09-21T21:41:37.827 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2248000000000004e-05 2024-09-21T21:41:37.827 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.827 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:37.827 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.827 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.827 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.828 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1485 3.16 3:695c582e:::rbd_data.1121e540c096.0000000000000008:b [read 335872~36864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:37.828 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634106+0000", 2024-09-21T21:41:37.828 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 36.186829953, 2024-09-21T21:41:37.828 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059223685999999998, 2024-09-21T21:41:37.828 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:37.828 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:37.828 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:37.828 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:37.828 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:37.828 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:37.828 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1485 2024-09-21T21:41:37.828 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.828 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:37.828 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.829 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:37.829 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634106+0000", 2024-09-21T21:41:37.829 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:37.829 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.829 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.829 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:37.829 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634106+0000", 2024-09-21T21:41:37.829 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:37.829 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.829 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.829 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:37.829 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634106+0000", 2024-09-21T21:41:37.829 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3300000000000002e-07 2024-09-21T21:41:37.829 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.830 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.830 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:37.830 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634108+0000", 2024-09-21T21:41:37.830 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.996e-06 2024-09-21T21:41:37.830 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.830 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.830 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:37.830 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634115+0000", 2024-09-21T21:41:37.830 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5139999999999998e-06 2024-09-21T21:41:37.830 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.830 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.830 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:37.830 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634120+0000", 2024-09-21T21:41:37.830 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1050000000000003e-06 2024-09-21T21:41:37.830 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.831 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.831 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:37.831 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.692374+0000", 2024-09-21T21:41:37.831 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.058254729999999998 2024-09-21T21:41:37.831 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.831 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.831 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:37.831 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.692440+0000", 2024-09-21T21:41:37.831 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6076999999999998e-05 2024-09-21T21:41:37.831 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.831 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.831 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:37.831 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693329+0000", 2024-09-21T21:41:37.831 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00088873100000000003 2024-09-21T21:41:37.832 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.832 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:37.832 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.832 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.832 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.832 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1486 3.16 3:6a41a830:::rbd_data.112145fae779.000000000000000a:a [read 360448~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:37.832 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634126+0000", 2024-09-21T21:41:37.832 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 36.186809660999998, 2024-09-21T21:41:37.832 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059399528, 2024-09-21T21:41:37.832 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:37.832 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:37.832 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:37.832 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:37.832 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:37.833 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:37.833 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1486 2024-09-21T21:41:37.833 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.833 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:37.833 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.833 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:37.833 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634126+0000", 2024-09-21T21:41:37.833 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:37.833 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.833 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.833 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:37.833 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634126+0000", 2024-09-21T21:41:37.833 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:37.833 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.834 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.834 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:37.834 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634126+0000", 2024-09-21T21:41:37.834 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4099999999999999e-07 2024-09-21T21:41:37.834 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.834 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.834 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:37.834 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634128+0000", 2024-09-21T21:41:37.834 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1069999999999999e-06 2024-09-21T21:41:37.834 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.834 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.834 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:37.834 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634135+0000", 2024-09-21T21:41:37.834 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.212e-06 2024-09-21T21:41:37.834 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.835 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.835 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:37.835 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634140+0000", 2024-09-21T21:41:37.835 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1429999999999997e-06 2024-09-21T21:41:37.835 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.835 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.835 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:37.835 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693363+0000", 2024-09-21T21:41:37.835 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059223614000000001 2024-09-21T21:41:37.835 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.835 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.835 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:37.835 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693431+0000", 2024-09-21T21:41:37.835 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7294999999999999e-05 2024-09-21T21:41:37.836 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.836 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.836 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:37.836 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693525+0000", 2024-09-21T21:41:37.836 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.4716000000000006e-05 2024-09-21T21:41:37.836 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.836 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:37.836 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.836 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.836 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.836 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1499 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 49152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:37.836 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634186+0000", 2024-09-21T21:41:37.836 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 36.186749784, 2024-09-21T21:41:37.836 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059339256999999999, 2024-09-21T21:41:37.836 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:37.837 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:37.837 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:37.837 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:37.837 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:37.837 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:37.837 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1499 2024-09-21T21:41:37.837 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.837 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:37.837 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.837 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:37.837 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634186+0000", 2024-09-21T21:41:37.837 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:37.837 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.837 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.837 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:37.838 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634186+0000", 2024-09-21T21:41:37.838 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:37.838 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.838 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.838 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:37.838 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634186+0000", 2024-09-21T21:41:37.838 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0999999999999999e-07 2024-09-21T21:41:37.838 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.838 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.838 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:37.838 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634188+0000", 2024-09-21T21:41:37.838 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.86e-06 2024-09-21T21:41:37.838 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.838 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.839 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:37.839 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634194+0000", 2024-09-21T21:41:37.839 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1109999999999996e-06 2024-09-21T21:41:37.839 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.839 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.839 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:37.839 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634199+0000", 2024-09-21T21:41:37.839 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7419999999999997e-06 2024-09-21T21:41:37.839 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.839 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.839 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:37.839 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693357+0000", 2024-09-21T21:41:37.839 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059157847 2024-09-21T21:41:37.839 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.840 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.840 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:37.840 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693430+0000", 2024-09-21T21:41:37.840 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3203999999999999e-05 2024-09-21T21:41:37.840 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.840 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.840 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:37.840 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693525+0000", 2024-09-21T21:41:37.840 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5082999999999997e-05 2024-09-21T21:41:37.840 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.840 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:37.840 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.840 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.840 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.841 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1501 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 106496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:37.841 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634205+0000", 2024-09-21T21:41:37.841 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 36.186730693999998, 2024-09-21T21:41:37.841 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059482394000000001, 2024-09-21T21:41:37.841 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:37.841 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:37.841 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:37.841 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:37.841 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:37.841 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:37.841 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1501 2024-09-21T21:41:37.841 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.841 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:37.841 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.841 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:37.842 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634205+0000", 2024-09-21T21:41:37.842 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:37.842 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.842 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.842 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:37.842 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634205+0000", 2024-09-21T21:41:37.842 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:37.842 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.842 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.842 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:37.842 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634205+0000", 2024-09-21T21:41:37.842 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.03e-07 2024-09-21T21:41:37.842 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.843 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.843 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:37.843 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634207+0000", 2024-09-21T21:41:37.843 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0389999999999999e-06 2024-09-21T21:41:37.843 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.843 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.843 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:37.843 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634213+0000", 2024-09-21T21:41:37.843 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2600000000000002e-06 2024-09-21T21:41:37.843 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.843 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.843 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:37.843 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634218+0000", 2024-09-21T21:41:37.843 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8570000000000003e-06 2024-09-21T21:41:37.843 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.844 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.844 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:37.844 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693552+0000", 2024-09-21T21:41:37.844 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059333636000000002 2024-09-21T21:41:37.844 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.844 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.844 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:37.844 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693613+0000", 2024-09-21T21:41:37.844 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0560000000000003e-05 2024-09-21T21:41:37.844 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.844 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.844 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:37.844 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693687+0000", 2024-09-21T21:41:37.844 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4639000000000001e-05 2024-09-21T21:41:37.844 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.845 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:37.845 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.845 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.845 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.845 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1515 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 434176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:37.845 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634318+0000", 2024-09-21T21:41:37.845 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 36.186617007000002, 2024-09-21T21:41:37.845 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059565812000000003, 2024-09-21T21:41:37.845 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:37.845 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:37.845 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:37.845 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:37.845 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:37.845 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:37.846 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1515 2024-09-21T21:41:37.846 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.846 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:37.846 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.846 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:37.846 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634318+0000", 2024-09-21T21:41:37.846 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:37.846 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.846 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.846 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:37.846 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634318+0000", 2024-09-21T21:41:37.846 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:37.846 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.846 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.846 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:37.847 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634319+0000", 2024-09-21T21:41:37.847 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9400000000000001e-07 2024-09-21T21:41:37.847 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.847 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.847 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:37.847 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634321+0000", 2024-09-21T21:41:37.847 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.154e-06 2024-09-21T21:41:37.847 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.847 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.847 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:37.847 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634327+0000", 2024-09-21T21:41:37.847 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9830000000000001e-06 2024-09-21T21:41:37.847 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.847 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.848 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:37.848 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634332+0000", 2024-09-21T21:41:37.848 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9080000000000003e-06 2024-09-21T21:41:37.848 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.848 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.848 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:37.848 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693717+0000", 2024-09-21T21:41:37.848 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059385447000000001 2024-09-21T21:41:37.848 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.848 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.848 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:37.848 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693795+0000", 2024-09-21T21:41:37.848 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7267000000000004e-05 2024-09-21T21:41:37.848 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.848 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.849 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:37.849 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693884+0000", 2024-09-21T21:41:37.849 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9659000000000003e-05 2024-09-21T21:41:37.849 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.849 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:37.849 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.849 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.849 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.849 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1517 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 491520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:37.849 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634338+0000", 2024-09-21T21:41:37.849 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 36.186597233999997, 2024-09-21T21:41:37.849 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060100745999999997, 2024-09-21T21:41:37.849 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:37.849 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:37.850 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:37.850 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:37.850 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:37.850 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:37.850 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1517 2024-09-21T21:41:37.850 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.850 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:37.850 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.850 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:37.850 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634338+0000", 2024-09-21T21:41:37.850 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:37.850 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.850 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.850 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:37.851 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634338+0000", 2024-09-21T21:41:37.851 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:37.851 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.851 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.851 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:37.851 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634339+0000", 2024-09-21T21:41:37.851 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2899999999999999e-07 2024-09-21T21:41:37.851 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.851 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.851 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:37.851 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634341+0000", 2024-09-21T21:41:37.851 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.8959999999999999e-06 2024-09-21T21:41:37.851 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.851 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.851 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:37.851 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634347+0000", 2024-09-21T21:41:37.852 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.302e-06 2024-09-21T21:41:37.852 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.852 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.852 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:37.852 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634351+0000", 2024-09-21T21:41:37.852 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6070000000000001e-06 2024-09-21T21:41:37.852 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.852 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.852 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:37.852 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694279+0000", 2024-09-21T21:41:37.852 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059927046999999997 2024-09-21T21:41:37.852 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.852 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.852 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:37.852 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694355+0000", 2024-09-21T21:41:37.852 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6483999999999995e-05 2024-09-21T21:41:37.853 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.853 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.853 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:37.853 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694439+0000", 2024-09-21T21:41:37.853 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3980999999999994e-05 2024-09-21T21:41:37.853 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.853 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:37.853 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.853 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.853 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.853 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1518 3.16 3:6a41a830:::rbd_data.112145fae779.000000000000000a:a [read 294912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:37.853 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634357+0000", 2024-09-21T21:41:37.853 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 36.186578328000003, 2024-09-21T21:41:37.853 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059712318, 2024-09-21T21:41:37.853 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:37.854 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:37.854 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:37.854 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:37.854 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:37.854 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:37.854 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1518 2024-09-21T21:41:37.854 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.854 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:37.854 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.854 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:37.854 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634357+0000", 2024-09-21T21:41:37.854 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:37.854 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.855 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.855 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:37.855 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634357+0000", 2024-09-21T21:41:37.855 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:37.855 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.855 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.855 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:37.855 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634358+0000", 2024-09-21T21:41:37.855 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2100000000000002e-07 2024-09-21T21:41:37.855 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.855 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.855 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:37.855 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634360+0000", 2024-09-21T21:41:37.855 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0940000000000002e-06 2024-09-21T21:41:37.856 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.856 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.856 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:37.856 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634366+0000", 2024-09-21T21:41:37.856 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3389999999999998e-06 2024-09-21T21:41:37.856 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.856 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.856 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:37.856 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634371+0000", 2024-09-21T21:41:37.856 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6670000000000004e-06 2024-09-21T21:41:37.856 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.856 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.856 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:37.856 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693910+0000", 2024-09-21T21:41:37.856 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059538973000000002 2024-09-21T21:41:37.857 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.857 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.857 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:37.857 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693982+0000", 2024-09-21T21:41:37.857 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2806999999999996e-05 2024-09-21T21:41:37.857 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.857 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.857 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:37.857 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694069+0000", 2024-09-21T21:41:37.857 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7016999999999997e-05 2024-09-21T21:41:37.857 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.857 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:37.857 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.857 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.857 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.858 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1519 3.16 3:6a41a830:::rbd_data.112145fae779.000000000000000a:a [read 352256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:37.858 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634377+0000", 2024-09-21T21:41:37.858 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 36.186558626, 2024-09-21T21:41:37.858 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059872213000000001, 2024-09-21T21:41:37.858 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:37.858 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:37.858 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:37.858 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:37.858 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:37.858 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:37.858 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1519 2024-09-21T21:41:37.858 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.858 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:37.858 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.858 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:37.859 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634377+0000", 2024-09-21T21:41:37.859 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:37.859 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.859 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.859 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:37.859 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634377+0000", 2024-09-21T21:41:37.859 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:37.859 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.859 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.859 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:37.859 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634377+0000", 2024-09-21T21:41:37.859 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2600000000000002e-07 2024-09-21T21:41:37.859 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.859 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.859 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:37.860 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634379+0000", 2024-09-21T21:41:37.860 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0729999999999999e-06 2024-09-21T21:41:37.860 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.860 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.860 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:37.860 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634386+0000", 2024-09-21T21:41:37.860 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4690000000000002e-06 2024-09-21T21:41:37.860 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.860 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.860 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:37.860 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634390+0000", 2024-09-21T21:41:37.860 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4580000000000003e-06 2024-09-21T21:41:37.860 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.860 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.860 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:37.861 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694094+0000", 2024-09-21T21:41:37.861 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059703830999999999 2024-09-21T21:41:37.861 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.861 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.861 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:37.861 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694168+0000", 2024-09-21T21:41:37.861 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4222999999999997e-05 2024-09-21T21:41:37.861 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.861 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.861 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:37.861 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694249+0000", 2024-09-21T21:41:37.861 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0632999999999999e-05 2024-09-21T21:41:37.861 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.861 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:37.861 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.861 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.862 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.862 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1529 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 53248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:37.862 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634397+0000", 2024-09-21T21:41:37.862 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 36.186538474999999, 2024-09-21T21:41:37.862 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060189919000000001, 2024-09-21T21:41:37.862 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:37.862 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:37.862 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:37.862 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:37.862 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:37.862 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:37.862 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1529 2024-09-21T21:41:37.862 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.862 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:37.863 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.863 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:37.863 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634397+0000", 2024-09-21T21:41:37.863 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:37.863 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.863 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.863 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:37.863 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634397+0000", 2024-09-21T21:41:37.863 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:37.863 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.863 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.863 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:37.863 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634397+0000", 2024-09-21T21:41:37.863 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2e-07 2024-09-21T21:41:37.863 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.864 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.864 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:37.864 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634399+0000", 2024-09-21T21:41:37.864 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0609999999999998e-06 2024-09-21T21:41:37.864 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.864 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.864 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:37.864 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634406+0000", 2024-09-21T21:41:37.864 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7259999999999996e-06 2024-09-21T21:41:37.864 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.864 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.864 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:37.864 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634411+0000", 2024-09-21T21:41:37.864 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9339999999999997e-06 2024-09-21T21:41:37.864 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.865 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.865 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:37.865 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694480+0000", 2024-09-21T21:41:37.865 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060069105999999997 2024-09-21T21:41:37.865 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.865 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.865 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:37.865 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694527+0000", 2024-09-21T21:41:37.865 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7120000000000003e-05 2024-09-21T21:41:37.865 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.865 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.865 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:37.865 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694587+0000", 2024-09-21T21:41:37.865 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9552000000000002e-05 2024-09-21T21:41:37.866 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.866 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:37.866 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.866 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.866 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.866 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1531 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 110592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:37.866 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634417+0000", 2024-09-21T21:41:37.866 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 36.186518540999998, 2024-09-21T21:41:37.866 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060347287999999999, 2024-09-21T21:41:37.866 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:37.866 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:37.866 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:37.866 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:37.866 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:37.866 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:37.867 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1531 2024-09-21T21:41:37.867 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.867 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:37.867 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.867 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:37.867 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634417+0000", 2024-09-21T21:41:37.867 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:37.867 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.867 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.867 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:37.867 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634417+0000", 2024-09-21T21:41:37.867 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:37.867 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.867 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.867 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:37.868 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634417+0000", 2024-09-21T21:41:37.868 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1699999999999999e-07 2024-09-21T21:41:37.868 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.868 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.868 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:37.868 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634420+0000", 2024-09-21T21:41:37.868 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.243e-06 2024-09-21T21:41:37.868 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.868 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.868 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:37.868 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634426+0000", 2024-09-21T21:41:37.868 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2550000000000003e-06 2024-09-21T21:41:37.868 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.868 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.868 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:37.869 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634431+0000", 2024-09-21T21:41:37.869 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0150000000000003e-06 2024-09-21T21:41:37.869 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.869 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.869 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:37.869 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694618+0000", 2024-09-21T21:41:37.869 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060186836000000001 2024-09-21T21:41:37.869 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.869 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.869 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:37.869 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694685+0000", 2024-09-21T21:41:37.869 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6970000000000004e-05 2024-09-21T21:41:37.869 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.869 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.870 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:37.870 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694764+0000", 2024-09-21T21:41:37.870 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9552e-05 2024-09-21T21:41:37.870 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.870 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:37.870 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.870 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.870 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.870 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1532 3.e 3:724b31a3:::rbd_data.1121e540c096.0000000000000015:b [read 151552~36864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:37.870 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634437+0000", 2024-09-21T21:41:37.870 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 36.186498745000002, 2024-09-21T21:41:37.870 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060474314000000001, 2024-09-21T21:41:37.870 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:37.870 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:37.871 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:37.871 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:37.871 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:37.871 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:37.871 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1532 2024-09-21T21:41:37.871 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.871 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:37.871 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.871 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:37.871 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634437+0000", 2024-09-21T21:41:37.871 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:37.871 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.871 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.871 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:37.871 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634437+0000", 2024-09-21T21:41:37.872 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:37.872 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.872 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.872 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:37.872 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634437+0000", 2024-09-21T21:41:37.872 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9400000000000001e-07 2024-09-21T21:41:37.872 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.872 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.872 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:37.872 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634439+0000", 2024-09-21T21:41:37.872 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.114e-06 2024-09-21T21:41:37.872 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.872 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.872 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:37.872 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634446+0000", 2024-09-21T21:41:37.873 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1389999999999998e-06 2024-09-21T21:41:37.873 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.873 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.873 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:37.873 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634451+0000", 2024-09-21T21:41:37.873 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0010000000000001e-06 2024-09-21T21:41:37.873 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.873 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.873 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:37.873 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694794+0000", 2024-09-21T21:41:37.873 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060342509000000003 2024-09-21T21:41:37.873 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.873 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.873 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:37.873 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694835+0000", 2024-09-21T21:41:37.874 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0728e-05 2024-09-21T21:41:37.874 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.874 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.874 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:37.874 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694911+0000", 2024-09-21T21:41:37.874 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6428999999999996e-05 2024-09-21T21:41:37.874 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.874 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:37.874 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.874 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.874 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.874 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1534 3.e 3:724b31a3:::rbd_data.1121e540c096.0000000000000015:b [read 487424~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:37.874 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634477+0000", 2024-09-21T21:41:37.874 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 36.186458342000002, 2024-09-21T21:41:37.874 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060624009, 2024-09-21T21:41:37.875 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:37.875 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:37.875 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:37.875 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:37.875 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:37.875 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:37.875 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1534 2024-09-21T21:41:37.875 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.875 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:37.875 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.875 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:37.875 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634477+0000", 2024-09-21T21:41:37.875 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:37.875 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.876 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.876 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:37.876 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634477+0000", 2024-09-21T21:41:37.876 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:37.876 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.876 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.876 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:37.876 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634478+0000", 2024-09-21T21:41:37.876 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1399999999999997e-07 2024-09-21T21:41:37.876 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.876 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.876 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:37.876 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634480+0000", 2024-09-21T21:41:37.876 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1009999999999999e-06 2024-09-21T21:41:37.877 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.877 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.877 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:37.877 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634486+0000", 2024-09-21T21:41:37.877 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3289999999999999e-06 2024-09-21T21:41:37.877 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.877 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.877 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:37.877 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634491+0000", 2024-09-21T21:41:37.877 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6260000000000003e-06 2024-09-21T21:41:37.877 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.877 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.877 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:37.877 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694943+0000", 2024-09-21T21:41:37.877 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060451960999999999 2024-09-21T21:41:37.878 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.878 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.878 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:37.878 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695014+0000", 2024-09-21T21:41:37.878 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1587000000000004e-05 2024-09-21T21:41:37.878 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.878 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.878 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:37.878 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695101+0000", 2024-09-21T21:41:37.878 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6990999999999994e-05 2024-09-21T21:41:37.878 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.878 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:37.878 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.878 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.879 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.879 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1545 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 438272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:37.879 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634515+0000", 2024-09-21T21:41:37.879 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 36.186420142000003, 2024-09-21T21:41:37.879 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060707944, 2024-09-21T21:41:37.879 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:37.879 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:37.879 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:37.879 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:37.879 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:37.879 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:37.879 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1545 2024-09-21T21:41:37.879 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.879 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:37.880 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.880 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:37.880 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634515+0000", 2024-09-21T21:41:37.880 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:37.880 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.880 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.880 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:37.880 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634515+0000", 2024-09-21T21:41:37.880 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:37.880 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.880 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.880 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:37.880 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634516+0000", 2024-09-21T21:41:37.880 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0999999999999999e-07 2024-09-21T21:41:37.881 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.881 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.881 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:37.881 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634518+0000", 2024-09-21T21:41:37.881 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.846e-06 2024-09-21T21:41:37.881 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.881 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.881 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:37.881 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634524+0000", 2024-09-21T21:41:37.881 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2550000000000003e-06 2024-09-21T21:41:37.881 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.881 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.881 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:37.881 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634529+0000", 2024-09-21T21:41:37.881 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6940000000000003e-06 2024-09-21T21:41:37.882 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.882 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.882 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:37.882 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695127+0000", 2024-09-21T21:41:37.882 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060598921 2024-09-21T21:41:37.882 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.882 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.882 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:37.882 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695171+0000", 2024-09-21T21:41:37.882 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3588999999999999e-05 2024-09-21T21:41:37.882 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.882 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.882 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:37.883 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695223+0000", 2024-09-21T21:41:37.883 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2228999999999997e-05 2024-09-21T21:41:37.883 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.883 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:37.883 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.883 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.883 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.883 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1547 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 495616~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:37.883 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634534+0000", 2024-09-21T21:41:37.883 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 36.186401351000001, 2024-09-21T21:41:37.883 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060881131999999998, 2024-09-21T21:41:37.883 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:37.883 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:37.883 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:37.883 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:37.884 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:37.884 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:37.884 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1547 2024-09-21T21:41:37.884 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.884 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:37.884 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.884 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:37.884 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634534+0000", 2024-09-21T21:41:37.884 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:37.884 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.884 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.884 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:37.884 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634534+0000", 2024-09-21T21:41:37.884 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:37.885 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.885 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.885 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:37.885 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634535+0000", 2024-09-21T21:41:37.885 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.77e-07 2024-09-21T21:41:37.885 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.885 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.885 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:37.885 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634542+0000", 2024-09-21T21:41:37.885 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9000000000000006e-06 2024-09-21T21:41:37.885 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.885 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.885 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:37.885 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634549+0000", 2024-09-21T21:41:37.885 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5910000000000001e-06 2024-09-21T21:41:37.886 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.886 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.886 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:37.886 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634554+0000", 2024-09-21T21:41:37.886 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7940000000000002e-06 2024-09-21T21:41:37.886 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.886 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.886 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:37.886 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695256+0000", 2024-09-21T21:41:37.886 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.06070246 2024-09-21T21:41:37.886 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.886 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.886 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:37.886 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695332+0000", 2024-09-21T21:41:37.887 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6248000000000006e-05 2024-09-21T21:41:37.887 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.887 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.887 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:37.887 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695415+0000", 2024-09-21T21:41:37.887 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2762000000000005e-05 2024-09-21T21:41:37.887 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.887 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:37.887 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.887 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.887 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.887 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1548 3.16 3:6a41a830:::rbd_data.112145fae779.000000000000000a:a [read 299008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:37.887 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634561+0000", 2024-09-21T21:41:37.887 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 36.186374774000001, 2024-09-21T21:41:37.887 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060786743999999997, 2024-09-21T21:41:37.888 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:37.888 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:37.888 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:37.888 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:37.888 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:37.888 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:37.888 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1548 2024-09-21T21:41:37.888 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.888 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:37.888 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.888 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:37.888 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634561+0000", 2024-09-21T21:41:37.888 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:37.888 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.888 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.889 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:37.889 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634561+0000", 2024-09-21T21:41:37.889 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:37.889 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.889 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.889 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:37.889 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634561+0000", 2024-09-21T21:41:37.889 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5199999999999997e-07 2024-09-21T21:41:37.889 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.889 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.889 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:37.889 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634564+0000", 2024-09-21T21:41:37.889 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4040000000000002e-06 2024-09-21T21:41:37.889 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.889 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.890 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:37.890 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634571+0000", 2024-09-21T21:41:37.890 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4379999999999998e-06 2024-09-21T21:41:37.890 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.890 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.890 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:37.890 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634576+0000", 2024-09-21T21:41:37.890 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1399999999999999e-06 2024-09-21T21:41:37.890 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.890 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.890 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:37.890 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695247+0000", 2024-09-21T21:41:37.890 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060670397000000001 2024-09-21T21:41:37.890 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.891 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.891 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:37.891 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695299+0000", 2024-09-21T21:41:37.891 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2564e-05 2024-09-21T21:41:37.891 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.891 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.891 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:37.891 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695347+0000", 2024-09-21T21:41:37.891 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8248999999999998e-05 2024-09-21T21:41:37.891 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.891 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:37.891 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.891 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.891 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.891 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1549 3.16 3:6a41a830:::rbd_data.112145fae779.000000000000000a:a [read 356352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:37.892 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634583+0000", 2024-09-21T21:41:37.892 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 36.186352861000003, 2024-09-21T21:41:37.892 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060889459, 2024-09-21T21:41:37.892 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:37.892 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:37.892 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:37.892 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:37.892 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:37.892 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:37.892 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1549 2024-09-21T21:41:37.892 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.892 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:37.892 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.892 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:37.892 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634583+0000", 2024-09-21T21:41:37.893 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:37.893 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.893 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.893 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:37.893 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634583+0000", 2024-09-21T21:41:37.893 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:37.893 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.893 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.893 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:37.893 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634583+0000", 2024-09-21T21:41:37.893 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1899999999999998e-07 2024-09-21T21:41:37.893 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.893 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.893 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:37.893 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634585+0000", 2024-09-21T21:41:37.894 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2129999999999998e-06 2024-09-21T21:41:37.894 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.894 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.894 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:37.894 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634592+0000", 2024-09-21T21:41:37.894 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2840000000000003e-06 2024-09-21T21:41:37.894 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.894 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.894 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:37.894 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634596+0000", 2024-09-21T21:41:37.894 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9309999999999999e-06 2024-09-21T21:41:37.894 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.894 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.894 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:37.895 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695371+0000", 2024-09-21T21:41:37.895 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060774425999999999 2024-09-21T21:41:37.895 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.895 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.895 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:37.895 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695419+0000", 2024-09-21T21:41:37.895 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8607999999999999e-05 2024-09-21T21:41:37.895 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.895 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.895 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:37.895 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695472+0000", 2024-09-21T21:41:37.895 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2577999999999997e-05 2024-09-21T21:41:37.895 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.895 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:37.895 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.896 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.896 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.896 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1588 3.16 3:695c582e:::rbd_data.1121e540c096.0000000000000008:b [read 147456~40960] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:37.896 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.636942+0000", 2024-09-21T21:41:37.896 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 36.183993084000001, 2024-09-21T21:41:37.896 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.058643811999999997, 2024-09-21T21:41:37.896 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:37.896 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:37.896 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:37.896 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:37.896 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:37.896 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:37.896 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1588 2024-09-21T21:41:37.896 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.896 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:37.897 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.897 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:37.897 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636942+0000", 2024-09-21T21:41:37.897 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:37.897 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.897 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.897 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:37.897 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636942+0000", 2024-09-21T21:41:37.897 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:37.897 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.897 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.897 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:37.897 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636943+0000", 2024-09-21T21:41:37.897 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4199999999999996e-07 2024-09-21T21:41:37.897 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.898 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.898 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:37.898 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636945+0000", 2024-09-21T21:41:37.898 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.401e-06 2024-09-21T21:41:37.898 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.898 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.898 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:37.898 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636952+0000", 2024-09-21T21:41:37.898 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5379999999999999e-06 2024-09-21T21:41:37.898 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.898 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.898 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:37.898 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636958+0000", 2024-09-21T21:41:37.898 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.489e-06 2024-09-21T21:41:37.899 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.899 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.899 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:37.899 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695497+0000", 2024-09-21T21:41:37.899 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.058538504999999998 2024-09-21T21:41:37.899 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.899 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.899 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:37.899 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695533+0000", 2024-09-21T21:41:37.899 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6118000000000002e-05 2024-09-21T21:41:37.899 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.899 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.899 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:37.899 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695586+0000", 2024-09-21T21:41:37.899 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3219e-05 2024-09-21T21:41:37.900 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.900 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:37.900 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.900 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.900 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.900 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1589 3.16 3:695c582e:::rbd_data.1121e540c096.0000000000000008:b [read 487424~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:37.900 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.636965+0000", 2024-09-21T21:41:37.900 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 36.183969992999998, 2024-09-21T21:41:37.900 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.058728152999999998, 2024-09-21T21:41:37.900 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:37.900 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:37.900 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:37.900 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:37.900 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:37.900 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:37.901 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1589 2024-09-21T21:41:37.901 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.901 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:37.901 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.901 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:37.901 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636965+0000", 2024-09-21T21:41:37.901 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:37.901 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.901 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.901 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:37.901 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636965+0000", 2024-09-21T21:41:37.901 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:37.901 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.901 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.901 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:37.902 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636966+0000", 2024-09-21T21:41:37.902 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3799999999999998e-07 2024-09-21T21:41:37.902 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.902 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.902 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:37.902 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636968+0000", 2024-09-21T21:41:37.902 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2400000000000002e-06 2024-09-21T21:41:37.902 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.902 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.902 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:37.902 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636976+0000", 2024-09-21T21:41:37.902 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5050000000000001e-06 2024-09-21T21:41:37.902 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.902 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.903 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:37.903 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636981+0000", 2024-09-21T21:41:37.903 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2170000000000002e-06 2024-09-21T21:41:37.903 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.903 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.903 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:37.903 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695610+0000", 2024-09-21T21:41:37.903 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.058628998000000002 2024-09-21T21:41:37.903 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.903 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.903 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:37.903 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695647+0000", 2024-09-21T21:41:37.903 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6752000000000003e-05 2024-09-21T21:41:37.903 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.903 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.904 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:37.904 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695694+0000", 2024-09-21T21:41:37.904 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7002999999999997e-05 2024-09-21T21:41:37.904 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.904 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:37.904 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.904 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.904 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.904 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1607 3.e 3:724b31a3:::rbd_data.1121e540c096.0000000000000015:b [read 299008~36864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:37.904 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.637062+0000", 2024-09-21T21:41:37.904 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 36.183873902000002, 2024-09-21T21:41:37.904 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.058623078000000002, 2024-09-21T21:41:37.904 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:37.904 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:37.904 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:37.904 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:37.905 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:37.905 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:37.905 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1607 2024-09-21T21:41:37.905 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.905 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:37.905 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.905 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:37.905 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.637062+0000", 2024-09-21T21:41:37.905 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:37.905 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.905 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.905 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:37.905 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.637062+0000", 2024-09-21T21:41:37.905 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:37.906 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.906 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.906 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:37.906 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.637062+0000", 2024-09-21T21:41:37.906 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0900000000000002e-07 2024-09-21T21:41:37.906 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.906 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.906 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:37.906 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.637069+0000", 2024-09-21T21:41:37.906 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.267e-06 2024-09-21T21:41:37.906 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.906 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.906 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:37.906 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.637076+0000", 2024-09-21T21:41:37.906 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7320000000000001e-06 2024-09-21T21:41:37.907 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.907 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.907 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:37.907 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.637082+0000", 2024-09-21T21:41:37.907 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4760000000000003e-06 2024-09-21T21:41:37.907 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.907 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.907 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:37.907 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695604+0000", 2024-09-21T21:41:37.907 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.058521261999999998 2024-09-21T21:41:37.907 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.907 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.907 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:37.907 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695638+0000", 2024-09-21T21:41:37.907 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4292999999999998e-05 2024-09-21T21:41:37.908 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:37.908 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:37.908 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:37.908 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695685+0000", 2024-09-21T21:41:37.908 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6638999999999998e-05 2024-09-21T21:41:37.908 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.908 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:37.908 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.908 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:37.908 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:37.908 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:37.909 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:37.908+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:37.911 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T21:41:37.912 INFO:teuthology.orchestra.run.smithi164.stdout:3632 trunc from 0x4d7c800 to 0x665d800 2024-09-21T21:41:37.912 INFO:teuthology.orchestra.run.smithi164.stdout:3633 trunc from 0x665d800 to 0xbbfc200 2024-09-21T21:41:37.912 INFO:teuthology.orchestra.run.smithi164.stdout:3636 punch from 0xaca000 to 0xacd000, (0x3000 bytes) 2024-09-21T21:41:37.912 INFO:teuthology.orchestra.run.smithi164.stdout:3637 read 0x90bbc00 thru 0x90c9313 (0xd714 bytes) 2024-09-21T21:41:37.912 INFO:teuthology.orchestra.run.smithi164.stdout:3639 write 0x21b9e00 thru 0x21c6d94 (0xcf95 bytes) 2024-09-21T21:41:37.912 INFO:teuthology.orchestra.run.smithi164.stdout:3640 trunc from 0xbbfc200 to 0xe11ea00 2024-09-21T21:41:37.912 INFO:teuthology.orchestra.run.smithi164.stdout:3641 write 0x2440600 thru 0x244d6a3 (0xd0a4 bytes) 2024-09-21T21:41:37.912 INFO:teuthology.orchestra.run.smithi164.stdout:3642 write 0x9850600 thru 0x985a8c7 (0xa2c8 bytes) 2024-09-21T21:41:37.912 INFO:teuthology.orchestra.run.smithi164.stdout:3647 write 0x813a00 thru 0x8238ac (0xfead bytes) 2024-09-21T21:41:37.912 INFO:teuthology.orchestra.run.smithi164.stdout:3648 punch from 0xa9c3400 to 0xa9ce200, (0xae00 bytes) 2024-09-21T21:41:37.912 INFO:teuthology.orchestra.run.smithi164.stdout:3649 read 0xa54ae00 thru 0xa5513d8 (0x65d9 bytes) 2024-09-21T21:41:37.912 INFO:teuthology.orchestra.run.smithi164.stdout:3650 write 0x39a200 thru 0x39aac9 (0x8ca bytes) 2024-09-21T21:41:37.913 INFO:teuthology.orchestra.run.smithi164.stdout:3651 write 0x4609400 thru 0x460b75d (0x235e bytes) 2024-09-21T21:41:37.913 INFO:teuthology.orchestra.run.smithi164.stdout:3654 write 0x7a2400 thru 0x7adc96 (0xb897 bytes) 2024-09-21T21:41:37.913 INFO:teuthology.orchestra.run.smithi164.stdout:3655 write 0x2711000 thru 0x2714f0b (0x3f0c bytes) 2024-09-21T21:41:37.913 INFO:teuthology.orchestra.run.smithi164.stdout:3657 write 0xea9b800 thru 0xeaab700 (0xff01 bytes) 2024-09-21T21:41:37.918 INFO:teuthology.orchestra.run.smithi164.stdout:3658 trunc from 0xeaab800 to 0x6266000 2024-09-21T21:41:37.945 INFO:teuthology.orchestra.run.smithi164.stdout:3659 read 0x1cb1c00 thru 0x1cb561e (0x3a1f bytes) 2024-09-21T21:41:37.959 INFO:teuthology.orchestra.run.smithi164.stdout:3661 write 0x9d08200 thru 0x9d0b0ef (0x2ef0 bytes) 2024-09-21T21:41:37.960 INFO:teuthology.orchestra.run.smithi164.stdout:3662 write 0x95bd000 thru 0x95caff9 (0xdffa bytes) 2024-09-21T21:41:37.960 INFO:teuthology.orchestra.run.smithi164.stdout:3663 punch from 0x6993c00 to 0x69a3400, (0xf800 bytes) 2024-09-21T21:41:37.963 INFO:teuthology.orchestra.run.smithi164.stdout:3666 trunc from 0x9d0b200 to 0x79ea000 2024-09-21T21:41:37.982 INFO:teuthology.orchestra.run.smithi164.stdout:3667 read 0x43c6600 thru 0x43d2efc (0xc8fd bytes) 2024-09-21T21:41:37.983 INFO:teuthology.orchestra.run.smithi164.stdout:3668 write 0x7851600 thru 0x7859b23 (0x8524 bytes) 2024-09-21T21:41:37.989 INFO:teuthology.orchestra.run.smithi164.stdout:3669 trunc from 0x79ea000 to 0x40f5200 2024-09-21T21:41:38.005 INFO:teuthology.orchestra.run.smithi164.stdout:3670 write 0x39faa00 thru 0x3a0315d (0x875e bytes) 2024-09-21T21:41:38.010 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:38.008+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:38.022 INFO:teuthology.orchestra.run.smithi164.stdout:3672 write 0x7eb2a00 thru 0x7ec02be (0xd8bf bytes) 2024-09-21T21:41:38.030 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:38.030 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:41:38.030 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.030 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5215.0:694 3.12 3.cd224e52 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-21T21:41:38.030 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:38.029472+0000", 2024-09-21T21:41:38.030 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.7391000000000002e-05, 2024-09-21T21:41:38.031 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3007999999999998e-05, 2024-09-21T21:41:38.031 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:38.031 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:38.031 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:41:38.031 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:38.031 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5215", 2024-09-21T21:41:38.031 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1580282168", 2024-09-21T21:41:38.031 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 694 2024-09-21T21:41:38.031 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.031 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:38.031 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.031 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:38.031 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:38.029472+0000", 2024-09-21T21:41:38.031 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:38.031 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.032 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.032 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:38.032 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:38.029472+0000", 2024-09-21T21:41:38.032 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:38.032 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.032 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.032 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:38.032 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:38.029473+0000", 2024-09-21T21:41:38.032 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1900000000000002e-07 2024-09-21T21:41:38.032 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.032 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.032 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:38.032 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:38.029476+0000", 2024-09-21T21:41:38.032 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.9210000000000001e-06 2024-09-21T21:41:38.033 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.033 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.033 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:38.033 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:38.029482+0000", 2024-09-21T21:41:38.033 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.528e-06 2024-09-21T21:41:38.033 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.033 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.033 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:38.033 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:38.029486+0000", 2024-09-21T21:41:38.033 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1740000000000002e-06 2024-09-21T21:41:38.033 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.033 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.033 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:38.034 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:38.029518+0000", 2024-09-21T21:41:38.034 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.1919999999999999e-05 2024-09-21T21:41:38.034 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.034 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:38.034 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.034 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.034 INFO:teuthology.orchestra.run.smithi081.stdout: ], 2024-09-21T21:41:38.034 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 1 2024-09-21T21:41:38.035 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:38.041 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T21:41:38.041 INFO:teuthology.orchestra.run.smithi164.stdout:3673 write 0xce16200 thru 0xce1a71d (0x451e bytes) 2024-09-21T21:41:38.042 INFO:teuthology.orchestra.run.smithi164.stdout:3674 punch from 0xcc49a00 to 0xcc52200, (0x8800 bytes) 2024-09-21T21:41:38.042 INFO:teuthology.orchestra.run.smithi164.stdout:3676 punch from 0x8402200 to 0x840cc00, (0xaa00 bytes) 2024-09-21T21:41:38.046 INFO:teuthology.orchestra.run.smithi164.stdout:3677 read 0x22d1800 thru 0x22d6a8b (0x528c bytes) 2024-09-21T21:41:38.046 INFO:teuthology.orchestra.run.smithi164.stdout:3678 punch from 0x4831000 to 0x4832a00, (0x1a00 bytes) 2024-09-21T21:41:38.046 INFO:teuthology.orchestra.run.smithi164.stdout:3683 write 0x36bd800 thru 0x36be949 (0x114a bytes) 2024-09-21T21:41:38.046 INFO:teuthology.orchestra.run.smithi164.stdout:3684 read 0xa308800 thru 0xa313063 (0xa864 bytes) 2024-09-21T21:41:38.046 INFO:teuthology.orchestra.run.smithi164.stdout:3685 read 0x9d9b000 thru 0x9d9c0e8 (0x10e9 bytes) 2024-09-21T21:41:38.047 INFO:teuthology.orchestra.run.smithi164.stdout:3687 trunc from 0xce1a800 to 0x968a600 2024-09-21T21:41:38.060 INFO:teuthology.orchestra.run.smithi164.stdout:3688 write 0x58f0600 thru 0x58f9eb1 (0x98b2 bytes) 2024-09-21T21:41:38.062 INFO:teuthology.orchestra.run.smithi164.stdout:3689 read 0x4d33400 thru 0x4d406de (0xd2df bytes) 2024-09-21T21:41:38.062 INFO:teuthology.orchestra.run.smithi164.stdout:3691 read 0x5c4ac00 thru 0x5c590c8 (0xe4c9 bytes) 2024-09-21T21:41:38.064 INFO:teuthology.orchestra.run.smithi164.stdout:3692 write 0x7994000 thru 0x799bc9b (0x7c9c bytes) 2024-09-21T21:41:38.065 INFO:teuthology.orchestra.run.smithi164.stdout:3693 read 0x85c6c00 thru 0x85d1f48 (0xb349 bytes) 2024-09-21T21:41:38.065 INFO:teuthology.orchestra.run.smithi164.stdout:3694 flatten 2024-09-21T21:41:38.080 INFO:teuthology.orchestra.run.smithi164.stdout:3697 trunc from 0x968a600 to 0x4a14c00 2024-09-21T21:41:38.111 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:38.109+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:38.117 INFO:teuthology.orchestra.run.smithi164.stdout:3698 write 0xc0bae00 thru 0xc0c81e1 (0xd3e2 bytes) 2024-09-21T21:41:38.117 INFO:teuthology.orchestra.run.smithi164.stdout:3700 read 0x464a00 thru 0x47212c (0xd72d bytes) 2024-09-21T21:41:38.118 INFO:teuthology.orchestra.run.smithi164.stdout:3701 punch from 0x32dce00 to 0x32e5800, (0x8a00 bytes) 2024-09-21T21:41:38.118 INFO:teuthology.orchestra.run.smithi164.stdout:3702 write 0xb5a8400 thru 0xb5a9718 (0x1319 bytes) 2024-09-21T21:41:38.118 INFO:teuthology.orchestra.run.smithi164.stdout:3703 read 0x2c93600 thru 0x2c9ea21 (0xb422 bytes) 2024-09-21T21:41:38.119 INFO:teuthology.orchestra.run.smithi164.stdout:3704 trunc from 0xc0c8200 to 0xbd68000 2024-09-21T21:41:38.131 INFO:teuthology.orchestra.run.smithi164.stdout:3705 punch from 0x2ec4e00 to 0x2ed2c00, (0xde00 bytes) 2024-09-21T21:41:38.135 INFO:teuthology.orchestra.run.smithi164.stdout:3706 trunc from 0xbd68000 to 0xe6c2c00 2024-09-21T21:41:38.145 INFO:teuthology.orchestra.run.smithi164.stdout:3707 write 0xd926600 thru 0xd92e374 (0x7d75 bytes) 2024-09-21T21:41:38.145 INFO:teuthology.orchestra.run.smithi164.stdout:3709 punch from 0xdcafe00 to 0xdcb5200, (0x5400 bytes) 2024-09-21T21:41:38.146 INFO:teuthology.orchestra.run.smithi164.stdout:3710 read 0xa76d200 thru 0xa76e7e0 (0x15e1 bytes) 2024-09-21T21:41:38.148 INFO:teuthology.orchestra.run.smithi164.stdout:3711 read 0xb852c00 thru 0xb85f0c7 (0xc4c8 bytes) 2024-09-21T21:41:38.148 INFO:teuthology.orchestra.run.smithi164.stdout:3712 punch from 0x4f0cc00 to 0x4f0d400, (0x800 bytes) 2024-09-21T21:41:38.148 INFO:teuthology.orchestra.run.smithi164.stdout:3713 write 0x9e4f200 thru 0x9e5d1a6 (0xdfa7 bytes) 2024-09-21T21:41:38.148 INFO:teuthology.orchestra.run.smithi164.stdout:3714 punch from 0x639fa00 to 0x63a2000, (0x2600 bytes) 2024-09-21T21:41:38.148 INFO:teuthology.orchestra.run.smithi164.stdout:3716 write 0x6d9ae00 thru 0x6da10c8 (0x62c9 bytes) 2024-09-21T21:41:38.149 INFO:teuthology.orchestra.run.smithi164.stdout:3720 write 0x86fc00 thru 0x87e9b6 (0xedb7 bytes) 2024-09-21T21:41:38.158 INFO:teuthology.orchestra.run.smithi164.stdout:3721 trunc from 0xe6c2c00 to 0x72fa000 2024-09-21T21:41:38.191 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:38.191 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:41:38.191 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:41:38.191 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:41:38.191 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:38.200 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T21:41:38.223 INFO:teuthology.orchestra.run.smithi164.stdout:3722 write 0x633ec00 thru 0x6349746 (0xab47 bytes) 2024-09-21T21:41:38.225 INFO:teuthology.orchestra.run.smithi164.stdout:3725 punch from 0x4bf4e00 to 0x4c04a00, (0xfc00 bytes) 2024-09-21T21:41:38.233 INFO:teuthology.orchestra.run.smithi164.stdout:3726 trunc from 0x72fa000 to 0x96d200 2024-09-21T21:41:38.243 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:38.242+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:38.262 INFO:teuthology.orchestra.run.smithi164.stdout:3730 read 0x78a200 thru 0x796dd4 (0xcbd5 bytes) 2024-09-21T21:41:38.264 INFO:teuthology.orchestra.run.smithi164.stdout:3731 read 0x777400 thru 0x779d26 (0x2927 bytes) 2024-09-21T21:41:38.264 INFO:teuthology.orchestra.run.smithi164.stdout:3732 read 0x31e400 thru 0x3297fd (0xb3fe bytes) 2024-09-21T21:41:38.265 INFO:teuthology.orchestra.run.smithi164.stdout:3735 read 0x281600 thru 0x2821a5 (0xba6 bytes) 2024-09-21T21:41:38.270 INFO:teuthology.orchestra.run.smithi164.stdout:3736 trunc from 0x96d200 to 0x4945400 2024-09-21T21:41:38.299 INFO:teuthology.orchestra.run.smithi164.stdout:3739 write 0xdb5ee00 thru 0xdb6e241 (0xf442 bytes) 2024-09-21T21:41:38.300 INFO:teuthology.orchestra.run.smithi164.stdout:3740 write 0x5312a00 thru 0x5319376 (0x6977 bytes) 2024-09-21T21:41:38.300 INFO:teuthology.orchestra.run.smithi164.stdout:3742 punch from 0x7b8c00 to 0x7c1a00, (0x8e00 bytes) 2024-09-21T21:41:38.300 INFO:teuthology.orchestra.run.smithi164.stdout:3743 read 0x1f03400 thru 0x1f05ab0 (0x26b1 bytes) 2024-09-21T21:41:38.301 INFO:teuthology.orchestra.run.smithi164.stdout:3745 read 0x710b800 thru 0x711abec (0xf3ed bytes) 2024-09-21T21:41:38.310 INFO:teuthology.orchestra.run.smithi164.stdout:3746 trunc from 0xdb6e400 to 0x63f5c00 2024-09-21T21:41:38.312 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:38.311+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:38.314 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:38.314 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 20, 2024-09-21T21:41:38.314 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 600, 2024-09-21T21:41:38.315 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:41:38.315 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.315 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4952.0:3526 3.10 3:086eb8a4:::rbd_data.1121a3eabc61.0000000000000000:17 [read 2410496~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:38.315 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:29.795290+0000", 2024-09-21T21:41:38.315 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 8.5167470020000007, 2024-09-21T21:41:38.315 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.032825482000000003, 2024-09-21T21:41:38.315 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:38.315 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:38.315 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:38.315 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:38.315 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4952", 2024-09-21T21:41:38.315 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:38.315 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3526 2024-09-21T21:41:38.316 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.316 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:38.316 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.316 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:38.316 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795290+0000", 2024-09-21T21:41:38.316 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:38.316 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.316 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.316 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:38.316 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795290+0000", 2024-09-21T21:41:38.316 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:38.316 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.316 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.316 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:38.317 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795291+0000", 2024-09-21T21:41:38.317 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3499999999999996e-07 2024-09-21T21:41:38.317 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.317 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.317 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:38.317 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795293+0000", 2024-09-21T21:41:38.317 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0779999999999998e-06 2024-09-21T21:41:38.317 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.317 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.317 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:38.317 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795299+0000", 2024-09-21T21:41:38.317 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7270000000000004e-06 2024-09-21T21:41:38.317 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.317 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.318 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:38.318 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795303+0000", 2024-09-21T21:41:38.318 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3579999999999996e-06 2024-09-21T21:41:38.318 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.318 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.318 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:38.318 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.827920+0000", 2024-09-21T21:41:38.318 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.032617118 2024-09-21T21:41:38.318 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.318 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.318 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:38.318 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.828011+0000", 2024-09-21T21:41:38.318 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.0676999999999999e-05 2024-09-21T21:41:38.318 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.319 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.319 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:38.319 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.828116+0000", 2024-09-21T21:41:38.319 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010498899999999999 2024-09-21T21:41:38.319 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.319 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:38.319 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.319 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.319 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.319 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4952.0:3528 3.10 3:086eb8a4:::rbd_data.1121a3eabc61.0000000000000000:17 [read 2424832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:38.319 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:29.795308+0000", 2024-09-21T21:41:38.319 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 8.5167292799999998, 2024-09-21T21:41:38.319 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033028465999999999, 2024-09-21T21:41:38.319 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:38.320 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:38.320 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:38.320 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:38.320 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4952", 2024-09-21T21:41:38.320 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:38.320 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3528 2024-09-21T21:41:38.320 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.320 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:38.320 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.320 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:38.320 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795308+0000", 2024-09-21T21:41:38.320 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:38.320 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.320 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.320 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:38.321 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795308+0000", 2024-09-21T21:41:38.321 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:38.321 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.321 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.321 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:38.321 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795308+0000", 2024-09-21T21:41:38.321 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.22e-07 2024-09-21T21:41:38.321 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.321 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.321 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:38.321 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795310+0000", 2024-09-21T21:41:38.321 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.7940000000000001e-06 2024-09-21T21:41:38.321 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.321 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.322 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:38.322 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795316+0000", 2024-09-21T21:41:38.322 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9410000000000004e-06 2024-09-21T21:41:38.322 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.322 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.322 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:38.322 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795321+0000", 2024-09-21T21:41:38.322 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4800000000000003e-06 2024-09-21T21:41:38.322 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.322 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.322 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:38.322 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.828149+0000", 2024-09-21T21:41:38.322 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.032828001000000002 2024-09-21T21:41:38.322 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.322 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.323 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:38.323 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.828223+0000", 2024-09-21T21:41:38.323 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.4048000000000007e-05 2024-09-21T21:41:38.323 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.323 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.323 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:38.323 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.828337+0000", 2024-09-21T21:41:38.323 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011388 2024-09-21T21:41:38.323 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.323 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:38.323 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.323 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.323 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.323 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4952.0:3529 3.10 3:086eb8a4:::rbd_data.1121a3eabc61.0000000000000000:17 [read 2439168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:38.323 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:29.795328+0000", 2024-09-21T21:41:38.324 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 8.5167091839999998, 2024-09-21T21:41:38.324 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033236716999999999, 2024-09-21T21:41:38.324 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:38.324 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:38.324 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:38.324 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:38.324 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4952", 2024-09-21T21:41:38.324 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:38.324 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3529 2024-09-21T21:41:38.324 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.324 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:38.324 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.324 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:38.324 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795328+0000", 2024-09-21T21:41:38.325 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:38.325 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.325 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.325 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:38.325 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795328+0000", 2024-09-21T21:41:38.325 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:38.325 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.325 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.325 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:38.325 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795329+0000", 2024-09-21T21:41:38.325 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.44e-07 2024-09-21T21:41:38.325 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.325 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.325 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:38.325 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795335+0000", 2024-09-21T21:41:38.326 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7400000000000001e-06 2024-09-21T21:41:38.326 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.326 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.326 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:38.326 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795340+0000", 2024-09-21T21:41:38.326 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9009999999999999e-06 2024-09-21T21:41:38.326 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.326 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.326 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:38.326 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795345+0000", 2024-09-21T21:41:38.326 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4839999999999998e-06 2024-09-21T21:41:38.326 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.326 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.326 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:38.327 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.828382+0000", 2024-09-21T21:41:38.327 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033037113 2024-09-21T21:41:38.327 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.327 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.327 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:38.327 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.828473+0000", 2024-09-21T21:41:38.327 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.1056000000000004e-05 2024-09-21T21:41:38.327 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.327 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.327 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:38.327 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.828565+0000", 2024-09-21T21:41:38.327 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.1879000000000006e-05 2024-09-21T21:41:38.327 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.327 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:38.327 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.328 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.328 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.328 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4952.0:3531 3.10 3:086eb8a4:::rbd_data.1121a3eabc61.0000000000000000:17 [read 2453504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:38.328 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:29.795353+0000", 2024-09-21T21:41:38.328 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 8.516684455, 2024-09-21T21:41:38.328 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033394973000000001, 2024-09-21T21:41:38.328 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:38.328 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:38.328 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:38.328 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:38.328 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4952", 2024-09-21T21:41:38.328 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:38.328 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3531 2024-09-21T21:41:38.328 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.329 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:38.329 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.329 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:38.329 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795353+0000", 2024-09-21T21:41:38.329 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:38.329 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.329 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.329 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:38.329 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795353+0000", 2024-09-21T21:41:38.329 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:38.329 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.329 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.329 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:38.329 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795353+0000", 2024-09-21T21:41:38.329 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7300000000000001e-07 2024-09-21T21:41:38.330 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.330 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.330 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:38.330 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795360+0000", 2024-09-21T21:41:38.330 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2829999999999998e-06 2024-09-21T21:41:38.330 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.330 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.330 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:38.330 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795366+0000", 2024-09-21T21:41:38.330 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1260000000000003e-06 2024-09-21T21:41:38.330 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.330 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.330 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:38.330 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795370+0000", 2024-09-21T21:41:38.331 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1899999999999997e-06 2024-09-21T21:41:38.331 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.331 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.331 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:38.331 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.828587+0000", 2024-09-21T21:41:38.331 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033216516000000001 2024-09-21T21:41:38.331 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.331 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.331 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:38.331 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.828662+0000", 2024-09-21T21:41:38.331 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5316999999999997e-05 2024-09-21T21:41:38.331 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.331 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.331 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:38.331 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.828748+0000", 2024-09-21T21:41:38.332 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.6068000000000003e-05 2024-09-21T21:41:38.332 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.332 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:38.332 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.332 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.332 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.332 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4952.0:3533 3.10 3:086eb8a4:::rbd_data.1121a3eabc61.0000000000000000:17 [read 2467840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:38.332 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:29.795376+0000", 2024-09-21T21:41:38.332 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 8.5166612540000006, 2024-09-21T21:41:38.332 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033574701999999998, 2024-09-21T21:41:38.332 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:38.332 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:38.332 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:38.332 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:38.333 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4952", 2024-09-21T21:41:38.333 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:38.333 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3533 2024-09-21T21:41:38.333 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.333 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:38.333 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.333 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:38.333 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795376+0000", 2024-09-21T21:41:38.333 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:38.333 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.333 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.333 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:38.333 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795376+0000", 2024-09-21T21:41:38.333 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:38.334 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.334 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.334 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:38.334 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795377+0000", 2024-09-21T21:41:38.334 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5400000000000002e-07 2024-09-21T21:41:38.334 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.334 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.334 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:38.334 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795379+0000", 2024-09-21T21:41:38.334 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9659999999999999e-06 2024-09-21T21:41:38.334 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.334 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.334 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:38.334 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795385+0000", 2024-09-21T21:41:38.335 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1519999999999997e-06 2024-09-21T21:41:38.335 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.335 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.335 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:38.335 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795389+0000", 2024-09-21T21:41:38.335 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4399999999999998e-06 2024-09-21T21:41:38.335 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.335 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.335 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:38.335 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.828777+0000", 2024-09-21T21:41:38.335 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033388099999999997 2024-09-21T21:41:38.335 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.335 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.335 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:38.335 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.828856+0000", 2024-09-21T21:41:38.336 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.8770999999999995e-05 2024-09-21T21:41:38.336 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.336 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.336 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:38.336 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.828951+0000", 2024-09-21T21:41:38.336 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.4919000000000004e-05 2024-09-21T21:41:38.336 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.336 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:38.336 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.336 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.336 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.336 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4952.0:3535 3.10 3:086eb8a4:::rbd_data.1121a3eabc61.0000000000000000:17 [read 2482176~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:38.336 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:29.795394+0000", 2024-09-21T21:41:38.336 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 8.5166431249999999, 2024-09-21T21:41:38.337 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033738335000000001, 2024-09-21T21:41:38.337 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:38.337 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:38.337 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:38.337 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:38.337 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4952", 2024-09-21T21:41:38.337 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:38.337 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3535 2024-09-21T21:41:38.337 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.337 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:38.337 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.337 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:38.337 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795394+0000", 2024-09-21T21:41:38.337 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:38.338 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.338 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.338 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:38.338 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795394+0000", 2024-09-21T21:41:38.338 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:38.338 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.338 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.338 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:38.338 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.796945+0000", 2024-09-21T21:41:38.338 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.001551097 2024-09-21T21:41:38.338 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.338 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.338 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:38.338 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.796949+0000", 2024-09-21T21:41:38.338 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5499999999999999e-06 2024-09-21T21:41:38.339 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.339 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.339 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:38.339 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.796958+0000", 2024-09-21T21:41:38.339 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.1649999999999995e-06 2024-09-21T21:41:38.339 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.339 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.339 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:38.339 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.796964+0000", 2024-09-21T21:41:38.339 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1199999999999999e-06 2024-09-21T21:41:38.339 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.339 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.339 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:38.339 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.828983+0000", 2024-09-21T21:41:38.340 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.032018396999999997 2024-09-21T21:41:38.340 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.340 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.340 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:38.340 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.829050+0000", 2024-09-21T21:41:38.340 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.7020000000000005e-05 2024-09-21T21:41:38.340 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.340 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.340 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:38.340 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.829133+0000", 2024-09-21T21:41:38.340 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.2985999999999995e-05 2024-09-21T21:41:38.340 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.340 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:38.340 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.340 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.341 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.341 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4952.0:3546 3.10 3:086eb8a4:::rbd_data.1121a3eabc61.0000000000000000:17 [read 2576384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:38.341 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:29.797117+0000", 2024-09-21T21:41:38.341 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 8.5149202709999994, 2024-09-21T21:41:38.341 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.03288253, 2024-09-21T21:41:38.341 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:38.341 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:38.341 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:38.341 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:38.341 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4952", 2024-09-21T21:41:38.341 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:38.341 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3546 2024-09-21T21:41:38.341 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.341 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:38.342 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.342 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:38.342 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797117+0000", 2024-09-21T21:41:38.342 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:38.342 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.342 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.342 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:38.342 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797117+0000", 2024-09-21T21:41:38.342 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:38.342 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.342 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.342 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:38.342 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797118+0000", 2024-09-21T21:41:38.342 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.39e-07 2024-09-21T21:41:38.342 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.343 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.343 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:38.343 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797120+0000", 2024-09-21T21:41:38.343 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9980000000000002e-06 2024-09-21T21:41:38.343 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.343 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.343 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:38.343 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797125+0000", 2024-09-21T21:41:38.343 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7409999999999998e-06 2024-09-21T21:41:38.343 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.343 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.343 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:38.343 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797130+0000", 2024-09-21T21:41:38.343 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5820000000000003e-06 2024-09-21T21:41:38.344 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.344 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.344 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:38.344 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.829898+0000", 2024-09-21T21:41:38.344 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.032768600000000002 2024-09-21T21:41:38.344 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.344 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.344 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:38.344 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.829934+0000", 2024-09-21T21:41:38.344 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5342000000000001e-05 2024-09-21T21:41:38.344 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.344 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.344 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:38.344 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830000+0000", 2024-09-21T21:41:38.344 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5927999999999996e-05 2024-09-21T21:41:38.345 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.345 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:38.345 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.345 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.345 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.345 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4952.0:3548 3.10 3:086eb8a4:::rbd_data.1121a3eabc61.0000000000000000:17 [read 2590720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:38.345 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:29.797136+0000", 2024-09-21T21:41:38.345 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 8.5149018309999995, 2024-09-21T21:41:38.345 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033006742999999998, 2024-09-21T21:41:38.345 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:38.345 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:38.345 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:38.345 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:38.345 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4952", 2024-09-21T21:41:38.346 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:38.346 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3548 2024-09-21T21:41:38.346 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.346 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:38.346 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.346 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:38.346 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797136+0000", 2024-09-21T21:41:38.346 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:38.346 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.346 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.346 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:38.346 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797136+0000", 2024-09-21T21:41:38.346 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:38.347 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.347 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.347 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:38.347 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797136+0000", 2024-09-21T21:41:38.347 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3600000000000004e-07 2024-09-21T21:41:38.347 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.347 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.347 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:38.347 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797138+0000", 2024-09-21T21:41:38.347 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.933e-06 2024-09-21T21:41:38.347 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.347 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.347 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:38.347 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797144+0000", 2024-09-21T21:41:38.347 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6099999999999997e-06 2024-09-21T21:41:38.348 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.348 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.348 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:38.348 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797149+0000", 2024-09-21T21:41:38.348 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0690000000000001e-06 2024-09-21T21:41:38.348 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.348 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.348 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:38.348 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830038+0000", 2024-09-21T21:41:38.348 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.032889535999999997 2024-09-21T21:41:38.348 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.348 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.348 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:38.348 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830087+0000", 2024-09-21T21:41:38.348 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9159999999999997e-05 2024-09-21T21:41:38.349 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.349 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.349 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:38.349 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830142+0000", 2024-09-21T21:41:38.349 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4898999999999997e-05 2024-09-21T21:41:38.349 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.349 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:38.349 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.349 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.349 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.349 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4952.0:3550 3.10 3:086eb8a4:::rbd_data.1121a3eabc61.0000000000000000:17 [read 2605056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:38.349 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:29.797154+0000", 2024-09-21T21:41:38.349 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 8.5148832730000006, 2024-09-21T21:41:38.349 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033113318000000003, 2024-09-21T21:41:38.349 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:38.350 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:38.350 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:38.350 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:38.350 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4952", 2024-09-21T21:41:38.350 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:38.350 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3550 2024-09-21T21:41:38.350 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.350 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:38.350 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.350 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:38.350 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797154+0000", 2024-09-21T21:41:38.350 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:38.350 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.350 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.351 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:38.351 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797154+0000", 2024-09-21T21:41:38.351 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:38.351 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.351 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.351 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:38.351 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797155+0000", 2024-09-21T21:41:38.351 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1199999999999998e-07 2024-09-21T21:41:38.351 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.351 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.351 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:38.351 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797165+0000", 2024-09-21T21:41:38.351 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0635000000000001e-05 2024-09-21T21:41:38.351 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.352 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.352 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:38.352 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797172+0000", 2024-09-21T21:41:38.352 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5300000000000002e-06 2024-09-21T21:41:38.352 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.352 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.352 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:38.352 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797176+0000", 2024-09-21T21:41:38.352 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7419999999999997e-06 2024-09-21T21:41:38.352 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.352 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.352 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:38.352 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830167+0000", 2024-09-21T21:41:38.352 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.032990468000000002 2024-09-21T21:41:38.352 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.353 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.353 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:38.353 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830202+0000", 2024-09-21T21:41:38.353 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5382000000000001e-05 2024-09-21T21:41:38.353 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.353 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.353 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:38.353 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830267+0000", 2024-09-21T21:41:38.353 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5148999999999996e-05 2024-09-21T21:41:38.353 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.353 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:38.353 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.353 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.353 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.353 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4952.0:3551 3.10 3:086eb8a4:::rbd_data.1121a3eabc61.0000000000000000:17 [read 2619392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:38.354 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:29.797183+0000", 2024-09-21T21:41:38.354 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 8.5148546609999993, 2024-09-21T21:41:38.354 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033198722, 2024-09-21T21:41:38.354 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:38.354 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:38.354 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:38.354 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:38.354 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4952", 2024-09-21T21:41:38.354 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:38.354 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3551 2024-09-21T21:41:38.354 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.354 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:38.354 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.354 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:38.354 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797183+0000", 2024-09-21T21:41:38.355 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:38.355 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.355 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.355 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:38.355 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797183+0000", 2024-09-21T21:41:38.355 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:38.355 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.355 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.355 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:38.355 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797183+0000", 2024-09-21T21:41:38.355 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8000000000000001e-07 2024-09-21T21:41:38.355 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.355 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.355 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:38.356 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797185+0000", 2024-09-21T21:41:38.356 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0940000000000002e-06 2024-09-21T21:41:38.356 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.356 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.356 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:38.356 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797191+0000", 2024-09-21T21:41:38.356 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8300000000000001e-06 2024-09-21T21:41:38.356 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.356 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.356 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:38.356 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797196+0000", 2024-09-21T21:41:38.356 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4530000000000004e-06 2024-09-21T21:41:38.356 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.356 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.356 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:38.357 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830295+0000", 2024-09-21T21:41:38.357 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033099669999999998 2024-09-21T21:41:38.357 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.357 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.357 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:38.357 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830332+0000", 2024-09-21T21:41:38.357 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7156999999999997e-05 2024-09-21T21:41:38.357 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.357 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.357 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:38.357 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830381+0000", 2024-09-21T21:41:38.357 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9138000000000003e-05 2024-09-21T21:41:38.357 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.357 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:38.357 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.358 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.358 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.358 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4952.0:3553 3.10 3:086eb8a4:::rbd_data.1121a3eabc61.0000000000000000:17 [read 2627584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:38.358 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:29.797201+0000", 2024-09-21T21:41:38.358 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 8.5148364409999999, 2024-09-21T21:41:38.358 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033313262000000003, 2024-09-21T21:41:38.358 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:38.358 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:38.358 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:38.358 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:38.358 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4952", 2024-09-21T21:41:38.358 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:38.358 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3553 2024-09-21T21:41:38.358 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.359 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:38.359 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.359 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:38.359 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797201+0000", 2024-09-21T21:41:38.359 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:38.359 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.359 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.359 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:38.359 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797201+0000", 2024-09-21T21:41:38.359 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:38.359 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.359 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.359 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:38.360 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797201+0000", 2024-09-21T21:41:38.360 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6600000000000002e-07 2024-09-21T21:41:38.360 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.360 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.360 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:38.360 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797203+0000", 2024-09-21T21:41:38.360 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.8810000000000001e-06 2024-09-21T21:41:38.360 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.360 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.360 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:38.360 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797209+0000", 2024-09-21T21:41:38.360 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.037e-06 2024-09-21T21:41:38.360 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.360 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.361 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:38.361 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797214+0000", 2024-09-21T21:41:38.361 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4920000000000004e-06 2024-09-21T21:41:38.361 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.361 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.361 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:38.361 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830408+0000", 2024-09-21T21:41:38.361 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033194199000000001 2024-09-21T21:41:38.361 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.361 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.361 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:38.361 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830455+0000", 2024-09-21T21:41:38.361 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6547e-05 2024-09-21T21:41:38.361 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.361 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.362 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:38.362 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830514+0000", 2024-09-21T21:41:38.362 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9740000000000001e-05 2024-09-21T21:41:38.362 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.362 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:38.362 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.362 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.362 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.362 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4952.0:3554 3.10 3:086eb8a4:::rbd_data.1121a3eabc61.0000000000000000:17 [read 2641920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:38.362 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:29.797220+0000", 2024-09-21T21:41:38.362 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 8.5148171470000005, 2024-09-21T21:41:38.362 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033396381000000003, 2024-09-21T21:41:38.362 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:38.362 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:38.363 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:38.363 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:38.363 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4952", 2024-09-21T21:41:38.363 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:38.363 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3554 2024-09-21T21:41:38.363 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.363 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:38.363 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.363 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:38.363 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797220+0000", 2024-09-21T21:41:38.363 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:38.363 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.363 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.363 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:38.363 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797220+0000", 2024-09-21T21:41:38.364 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:38.364 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.364 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.364 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:38.364 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797221+0000", 2024-09-21T21:41:38.364 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4099999999999999e-07 2024-09-21T21:41:38.364 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.364 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.364 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:38.364 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797223+0000", 2024-09-21T21:41:38.364 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.95e-06 2024-09-21T21:41:38.364 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.364 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.364 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:38.364 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797228+0000", 2024-09-21T21:41:38.365 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6919999999999999e-06 2024-09-21T21:41:38.365 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.365 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.365 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:38.365 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797238+0000", 2024-09-21T21:41:38.365 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6509999999999996e-06 2024-09-21T21:41:38.365 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.365 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.365 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:38.365 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830532+0000", 2024-09-21T21:41:38.365 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033294473999999998 2024-09-21T21:41:38.365 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.365 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.365 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:38.366 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830569+0000", 2024-09-21T21:41:38.366 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6723000000000001e-05 2024-09-21T21:41:38.366 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.366 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.366 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:38.366 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830617+0000", 2024-09-21T21:41:38.366 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7450000000000001e-05 2024-09-21T21:41:38.366 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.366 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:38.366 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.366 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.366 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.366 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4952.0:3555 3.10 3:086eb8a4:::rbd_data.1121a3eabc61.0000000000000000:17 [read 2656256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:38.366 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:29.797244+0000", 2024-09-21T21:41:38.366 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 8.5147932750000006, 2024-09-21T21:41:38.367 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033483065999999999, 2024-09-21T21:41:38.367 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:38.367 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:38.367 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:38.367 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:38.367 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4952", 2024-09-21T21:41:38.367 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:38.367 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3555 2024-09-21T21:41:38.367 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.367 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:38.367 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.367 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:38.367 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797244+0000", 2024-09-21T21:41:38.367 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:38.367 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.368 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.368 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:38.368 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797244+0000", 2024-09-21T21:41:38.368 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:38.368 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.368 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.368 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:38.368 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797245+0000", 2024-09-21T21:41:38.368 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9299999999999998e-07 2024-09-21T21:41:38.368 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.368 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.368 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:38.368 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797247+0000", 2024-09-21T21:41:38.368 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0370000000000001e-06 2024-09-21T21:41:38.369 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.369 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.369 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:38.369 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797253+0000", 2024-09-21T21:41:38.369 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8189999999999997e-06 2024-09-21T21:41:38.369 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.369 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.369 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:38.369 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797257+0000", 2024-09-21T21:41:38.369 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.33e-06 2024-09-21T21:41:38.369 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.369 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.369 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:38.369 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830642+0000", 2024-09-21T21:41:38.370 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033385629999999999 2024-09-21T21:41:38.370 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.370 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.370 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:38.370 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830679+0000", 2024-09-21T21:41:38.370 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6100999999999999e-05 2024-09-21T21:41:38.370 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.370 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.370 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:38.370 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830727+0000", 2024-09-21T21:41:38.370 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8656000000000003e-05 2024-09-21T21:41:38.370 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.370 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:38.370 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.370 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.371 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.371 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4952.0:3557 3.10 3:086eb8a4:::rbd_data.1121a3eabc61.0000000000000000:17 [read 2670592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:38.371 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:29.797263+0000", 2024-09-21T21:41:38.371 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 8.5147744129999996, 2024-09-21T21:41:38.371 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033580213999999997, 2024-09-21T21:41:38.371 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:38.371 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:38.371 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:38.371 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:38.371 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4952", 2024-09-21T21:41:38.371 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:38.371 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3557 2024-09-21T21:41:38.371 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.371 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:38.372 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.372 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:38.372 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797263+0000", 2024-09-21T21:41:38.372 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:38.372 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.372 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.372 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:38.372 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797263+0000", 2024-09-21T21:41:38.372 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:38.372 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.372 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.372 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:38.372 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797263+0000", 2024-09-21T21:41:38.372 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4799999999999999e-07 2024-09-21T21:41:38.372 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.373 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.373 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:38.373 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797265+0000", 2024-09-21T21:41:38.373 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.029e-06 2024-09-21T21:41:38.373 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.373 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.373 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:38.373 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797271+0000", 2024-09-21T21:41:38.373 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0000000000000002e-06 2024-09-21T21:41:38.373 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.373 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.373 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:38.373 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797276+0000", 2024-09-21T21:41:38.373 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5310000000000003e-06 2024-09-21T21:41:38.373 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.374 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.374 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:38.374 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830756+0000", 2024-09-21T21:41:38.374 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033479636 2024-09-21T21:41:38.374 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.374 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.374 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:38.374 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830792+0000", 2024-09-21T21:41:38.374 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6584000000000001e-05 2024-09-21T21:41:38.374 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.374 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.374 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:38.374 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830843+0000", 2024-09-21T21:41:38.374 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0986000000000003e-05 2024-09-21T21:41:38.375 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.375 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:38.375 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.375 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.375 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.375 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4952.0:3558 3.10 3:086eb8a4:::rbd_data.1121a3eabc61.0000000000000000:17 [read 2684928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:38.375 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:29.797281+0000", 2024-09-21T21:41:38.375 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 8.5147561169999992, 2024-09-21T21:41:38.375 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033692174999999998, 2024-09-21T21:41:38.375 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:38.375 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:38.375 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:38.375 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:38.375 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4952", 2024-09-21T21:41:38.375 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:38.376 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3558 2024-09-21T21:41:38.376 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.376 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:38.376 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.376 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:38.376 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797281+0000", 2024-09-21T21:41:38.376 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:38.376 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.376 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.376 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:38.376 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797281+0000", 2024-09-21T21:41:38.376 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:38.376 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.376 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.376 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:38.377 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797282+0000", 2024-09-21T21:41:38.377 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.41e-07 2024-09-21T21:41:38.377 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.377 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.377 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:38.377 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797283+0000", 2024-09-21T21:41:38.377 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.8390000000000001e-06 2024-09-21T21:41:38.377 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.377 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.377 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:38.377 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797289+0000", 2024-09-21T21:41:38.377 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3839999999999996e-06 2024-09-21T21:41:38.377 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.377 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.377 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:38.378 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797294+0000", 2024-09-21T21:41:38.378 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1900000000000003e-06 2024-09-21T21:41:38.378 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.378 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.378 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:38.378 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830874+0000", 2024-09-21T21:41:38.378 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033580384999999997 2024-09-21T21:41:38.378 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.378 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.378 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:38.378 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830912+0000", 2024-09-21T21:41:38.378 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7367000000000003e-05 2024-09-21T21:41:38.378 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.378 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.378 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:38.379 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830973+0000", 2024-09-21T21:41:38.379 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1668999999999995e-05 2024-09-21T21:41:38.379 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.379 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:38.379 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.379 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.379 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.379 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4952.0:3560 3.10 3:086eb8a4:::rbd_data.1121a3eabc61.0000000000000000:17 [read 2699264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:38.379 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:29.797300+0000", 2024-09-21T21:41:38.379 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 8.5147374340000006, 2024-09-21T21:41:38.379 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033777109, 2024-09-21T21:41:38.379 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:38.379 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:38.379 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:38.379 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:38.380 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4952", 2024-09-21T21:41:38.380 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:38.380 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3560 2024-09-21T21:41:38.380 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.380 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:38.380 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.380 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:38.380 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797300+0000", 2024-09-21T21:41:38.380 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:38.380 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.380 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.380 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:38.380 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797300+0000", 2024-09-21T21:41:38.380 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:38.380 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.381 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.381 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:38.381 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797300+0000", 2024-09-21T21:41:38.381 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.2399999999999999e-07 2024-09-21T21:41:38.381 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.381 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.381 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:38.381 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797302+0000", 2024-09-21T21:41:38.381 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0260000000000002e-06 2024-09-21T21:41:38.381 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.381 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.381 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:38.381 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797308+0000", 2024-09-21T21:41:38.381 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8839999999999999e-06 2024-09-21T21:41:38.382 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.382 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.382 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:38.382 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797313+0000", 2024-09-21T21:41:38.382 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3959999999999999e-06 2024-09-21T21:41:38.382 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.382 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.382 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:38.382 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830997+0000", 2024-09-21T21:41:38.382 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033684506000000003 2024-09-21T21:41:38.382 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.382 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.382 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:38.382 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.831033+0000", 2024-09-21T21:41:38.383 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5964000000000003e-05 2024-09-21T21:41:38.383 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.383 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.383 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:38.383 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.831077+0000", 2024-09-21T21:41:38.383 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4008999999999998e-05 2024-09-21T21:41:38.383 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.383 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:38.383 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.383 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.383 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.383 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4952.0:3562 3.10 3:086eb8a4:::rbd_data.1121a3eabc61.0000000000000000:17 [read 2713600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:38.383 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:29.797318+0000", 2024-09-21T21:41:38.383 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 8.5147195769999993, 2024-09-21T21:41:38.384 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033877712999999997, 2024-09-21T21:41:38.384 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:38.384 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:38.384 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:38.384 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:38.384 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4952", 2024-09-21T21:41:38.384 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:38.384 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3562 2024-09-21T21:41:38.384 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.384 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:38.384 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.384 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:38.384 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797318+0000", 2024-09-21T21:41:38.384 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:38.385 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.385 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.385 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:38.385 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797318+0000", 2024-09-21T21:41:38.385 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:38.385 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.385 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.385 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:38.385 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797318+0000", 2024-09-21T21:41:38.385 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.2000000000000001e-07 2024-09-21T21:41:38.385 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.385 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.385 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:38.385 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797320+0000", 2024-09-21T21:41:38.386 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9700000000000002e-06 2024-09-21T21:41:38.386 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.386 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.386 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:38.386 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797326+0000", 2024-09-21T21:41:38.386 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.212e-06 2024-09-21T21:41:38.386 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.386 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.386 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:38.386 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797331+0000", 2024-09-21T21:41:38.386 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9409999999999998e-06 2024-09-21T21:41:38.386 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.386 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.386 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:38.387 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.831104+0000", 2024-09-21T21:41:38.387 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033772976000000003 2024-09-21T21:41:38.387 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.387 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.387 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:38.387 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.831144+0000", 2024-09-21T21:41:38.387 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.0209000000000001e-05 2024-09-21T21:41:38.387 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.387 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.387 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:38.387 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.831196+0000", 2024-09-21T21:41:38.387 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1085000000000003e-05 2024-09-21T21:41:38.387 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.387 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:38.388 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.388 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.388 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.388 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4952.0:3563 3.10 3:086eb8a4:::rbd_data.1121a3eabc61.0000000000000000:17 [read 2727936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:38.388 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:29.797350+0000", 2024-09-21T21:41:38.388 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 8.5146873250000006, 2024-09-21T21:41:38.388 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033952501000000003, 2024-09-21T21:41:38.388 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:38.388 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:38.388 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:38.388 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:38.388 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4952", 2024-09-21T21:41:38.388 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:38.388 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3563 2024-09-21T21:41:38.388 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.389 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:38.389 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.389 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:38.389 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797350+0000", 2024-09-21T21:41:38.389 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:38.389 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.389 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.389 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:38.389 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797350+0000", 2024-09-21T21:41:38.389 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:38.389 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.389 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.389 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:38.389 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797351+0000", 2024-09-21T21:41:38.390 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5499999999999998e-07 2024-09-21T21:41:38.390 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.390 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.390 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:38.390 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797353+0000", 2024-09-21T21:41:38.390 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2299999999999998e-06 2024-09-21T21:41:38.390 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.390 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.390 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:38.390 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797359+0000", 2024-09-21T21:41:38.390 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3330000000000002e-06 2024-09-21T21:41:38.390 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.390 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.390 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:38.390 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797364+0000", 2024-09-21T21:41:38.391 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4410000000000003e-06 2024-09-21T21:41:38.391 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.391 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.391 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:38.391 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.831223+0000", 2024-09-21T21:41:38.391 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033859002999999999 2024-09-21T21:41:38.391 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.391 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.391 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:38.391 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.831258+0000", 2024-09-21T21:41:38.391 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5417000000000003e-05 2024-09-21T21:41:38.391 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.391 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.391 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:38.391 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.831303+0000", 2024-09-21T21:41:38.392 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4521999999999998e-05 2024-09-21T21:41:38.392 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.392 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:38.392 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.392 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.392 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.392 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4952.0:3564 3.10 3:086eb8a4:::rbd_data.1121a3eabc61.0000000000000000:17 [read 2742272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:38.392 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:29.797370+0000", 2024-09-21T21:41:38.392 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 8.5146674739999995, 2024-09-21T21:41:38.392 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.034051217000000002, 2024-09-21T21:41:38.392 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:38.393 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:38.393 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:38.393 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:38.393 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4952", 2024-09-21T21:41:38.393 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:38.393 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3564 2024-09-21T21:41:38.393 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.393 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:38.393 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.393 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:38.393 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797370+0000", 2024-09-21T21:41:38.393 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:38.393 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.393 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.394 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:38.394 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797370+0000", 2024-09-21T21:41:38.394 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:38.394 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.394 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.394 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:38.394 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797370+0000", 2024-09-21T21:41:38.394 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5499999999999998e-07 2024-09-21T21:41:38.394 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.394 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.394 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:38.395 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797372+0000", 2024-09-21T21:41:38.395 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9290000000000001e-06 2024-09-21T21:41:38.395 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.395 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.395 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:38.395 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797378+0000", 2024-09-21T21:41:38.395 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9599999999999997e-06 2024-09-21T21:41:38.395 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.395 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.395 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:38.395 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797388+0000", 2024-09-21T21:41:38.395 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.5040000000000008e-06 2024-09-21T21:41:38.395 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.395 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.396 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:38.396 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.831335+0000", 2024-09-21T21:41:38.396 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033947374000000002 2024-09-21T21:41:38.396 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.396 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.396 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:38.396 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.831371+0000", 2024-09-21T21:41:38.396 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5868999999999998e-05 2024-09-21T21:41:38.396 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.396 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.396 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:38.396 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.831421+0000", 2024-09-21T21:41:38.396 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0126000000000001e-05 2024-09-21T21:41:38.396 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.397 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:38.397 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.397 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.397 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.397 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5215.0:1073 3.1b 3:db41b1e4:::rbd_data.1121524d2522.0000000000000002:head [write 456704~63488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-21T21:41:38.397 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:38.158066+0000", 2024-09-21T21:41:38.397 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.15397190799999999, 2024-09-21T21:41:38.397 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.038046675000000002, 2024-09-21T21:41:38.397 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:38.397 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:38.397 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:41:38.397 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:38.397 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5215", 2024-09-21T21:41:38.397 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1580282168", 2024-09-21T21:41:38.398 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1073 2024-09-21T21:41:38.398 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.398 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:38.398 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.398 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:38.398 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:38.158066+0000", 2024-09-21T21:41:38.398 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:38.398 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.398 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.398 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:38.398 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:38.158066+0000", 2024-09-21T21:41:38.399 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:38.399 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.399 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.399 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:38.399 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:38.158066+0000", 2024-09-21T21:41:38.399 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.3600000000000002e-07 2024-09-21T21:41:38.399 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.399 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.399 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:38.399 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:38.158109+0000", 2024-09-21T21:41:38.399 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2731000000000001e-05 2024-09-21T21:41:38.399 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.399 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.399 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:38.399 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:38.158121+0000", 2024-09-21T21:41:38.400 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1562999999999999e-05 2024-09-21T21:41:38.400 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.400 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.400 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:38.400 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:38.158127+0000", 2024-09-21T21:41:38.400 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.348e-06 2024-09-21T21:41:38.400 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.400 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.400 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:38.400 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:38.158192+0000", 2024-09-21T21:41:38.400 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5246000000000005e-05 2024-09-21T21:41:38.400 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.400 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.400 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:38.400 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:38.158269+0000", 2024-09-21T21:41:38.401 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7103999999999999e-05 2024-09-21T21:41:38.401 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.401 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.401 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 4", 2024-09-21T21:41:38.401 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:38.158393+0000", 2024-09-21T21:41:38.401 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00012402600000000001 2024-09-21T21:41:38.401 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.401 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.401 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:41:38.401 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:38.159240+0000", 2024-09-21T21:41:38.401 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00084709399999999995 2024-09-21T21:41:38.401 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.401 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.401 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:41:38.402 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:38.196002+0000", 2024-09-21T21:41:38.402 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.036761794 2024-09-21T21:41:38.402 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.402 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.402 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:41:38.402 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:38.196064+0000", 2024-09-21T21:41:38.402 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1247000000000005e-05 2024-09-21T21:41:38.402 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.402 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.402 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:38.402 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:38.196112+0000", 2024-09-21T21:41:38.402 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8686000000000001e-05 2024-09-21T21:41:38.402 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.402 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:38.402 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.403 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.403 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:38.403 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:38.404 INFO:teuthology.orchestra.run.smithi164.stdout:3747 trunc from 0x63f5c00 to 0x9e0200 2024-09-21T21:41:38.404 INFO:teuthology.orchestra.run.smithi164.stdout:3748 write 0xea5c800 thru 0xea65be3 (0x93e4 bytes) 2024-09-21T21:41:38.404 INFO:teuthology.orchestra.run.smithi164.stdout:3750 trunc from 0xea65c00 to 0x38af400 2024-09-21T21:41:38.404 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T21:41:38.414 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:38.412+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:38.422 INFO:teuthology.orchestra.run.smithi164.stdout:3751 trunc from 0x38af400 to 0xc6f0800 2024-09-21T21:41:38.432 INFO:teuthology.orchestra.run.smithi164.stdout:3752 read 0x8704a00 thru 0x870ed74 (0xa375 bytes) 2024-09-21T21:41:38.433 INFO:teuthology.orchestra.run.smithi164.stdout:3755 read 0x891ac00 thru 0x891dcce (0x30cf bytes) 2024-09-21T21:41:38.433 INFO:teuthology.orchestra.run.smithi164.stdout:3756 read 0x6ecc600 thru 0x6ed88ce (0xc2cf bytes) 2024-09-21T21:41:38.433 INFO:teuthology.orchestra.run.smithi164.stdout:3760 read 0x7915600 thru 0x7915a7b (0x47c bytes) 2024-09-21T21:41:38.433 INFO:teuthology.orchestra.run.smithi164.stdout:3764 punch from 0xaea9a00 to 0xaeaa000, (0x600 bytes) 2024-09-21T21:41:38.434 INFO:teuthology.orchestra.run.smithi164.stdout:3766 trunc from 0xc6f0800 to 0xd306800 2024-09-21T21:41:38.446 INFO:teuthology.orchestra.run.smithi164.stdout:3767 read 0x21adc00 thru 0x21b888f (0xac90 bytes) 2024-09-21T21:41:38.448 INFO:teuthology.orchestra.run.smithi164.stdout:3770 write 0x88c3200 thru 0x88d266f (0xf470 bytes) 2024-09-21T21:41:38.452 INFO:teuthology.orchestra.run.smithi164.stdout:3771 trunc from 0xd306800 to 0xa7ad400 2024-09-21T21:41:38.465 INFO:teuthology.orchestra.run.smithi164.stdout:3772 write 0x7d38e00 thru 0x7d446f7 (0xb8f8 bytes) 2024-09-21T21:41:38.489 INFO:teuthology.orchestra.run.smithi164.stdout:3773 write 0xc7ec000 thru 0xc7f0639 (0x463a bytes) 2024-09-21T21:41:38.489 INFO:teuthology.orchestra.run.smithi164.stdout:3777 read 0x3f3b600 thru 0x3f46336 (0xad37 bytes) 2024-09-21T21:41:38.490 INFO:teuthology.orchestra.run.smithi164.stdout:3778 write 0x84fd000 thru 0x8503226 (0x6227 bytes) 2024-09-21T21:41:38.490 INFO:teuthology.orchestra.run.smithi164.stdout:3779 read 0xaa81000 thru 0xaa8c345 (0xb346 bytes) 2024-09-21T21:41:38.490 INFO:teuthology.orchestra.run.smithi164.stdout:3780 read 0xaa58400 thru 0xaa66973 (0xe574 bytes) 2024-09-21T21:41:38.490 INFO:teuthology.orchestra.run.smithi164.stdout:3783 punch from 0xc166e00 to 0xc171600, (0xa800 bytes) 2024-09-21T21:41:38.491 INFO:teuthology.orchestra.run.smithi164.stdout:3784 punch from 0xb5dbc00 to 0xb5dfa00, (0x3e00 bytes) 2024-09-21T21:41:38.491 INFO:teuthology.orchestra.run.smithi164.stdout:3785 write 0x78aae00 thru 0x78b0e87 (0x6088 bytes) 2024-09-21T21:41:38.491 INFO:teuthology.orchestra.run.smithi164.stdout:3787 read 0xc34c400 thru 0xc359e55 (0xda56 bytes) 2024-09-21T21:41:38.492 INFO:teuthology.orchestra.run.smithi164.stdout:3788 punch from 0x4a6fc00 to 0x4a78600, (0x8a00 bytes) 2024-09-21T21:41:38.503 INFO:teuthology.orchestra.run.smithi164.stdout:3790 trunc from 0xc7f0800 to 0x277ce00 2024-09-21T21:41:38.515 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:38.513+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:38.523 INFO:teuthology.orchestra.run.smithi164.stdout:3791 trunc from 0x277ce00 to 0x3428a00 2024-09-21T21:41:38.551 INFO:teuthology.orchestra.run.smithi164.stdout:3792 write 0x4facc00 thru 0x4fb536d (0x876e bytes) 2024-09-21T21:41:38.551 INFO:teuthology.orchestra.run.smithi164.stdout:3794 write 0x18e9c00 thru 0x18f8737 (0xeb38 bytes) 2024-09-21T21:41:38.551 INFO:teuthology.orchestra.run.smithi164.stdout:3797 punch from 0x15cf000 to 0x15d4000, (0x5000 bytes) 2024-09-21T21:41:38.551 INFO:teuthology.orchestra.run.smithi164.stdout:3798 write 0x151d800 thru 0x1529fda (0xc7db bytes) 2024-09-21T21:41:38.557 INFO:teuthology.orchestra.run.smithi164.stdout:3800 trunc from 0x4fb5400 to 0xa985400 2024-09-21T21:41:38.563 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:38.563 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:41:38.563 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 0 2024-09-21T21:41:38.563 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:38.574 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T21:41:38.575 INFO:teuthology.orchestra.run.smithi164.stdout:3801 write 0xb40600 thru 0xb4c607 (0xc008 bytes) 2024-09-21T21:41:38.580 INFO:teuthology.orchestra.run.smithi164.stdout:3803 trunc from 0xa985400 to 0x6944400 2024-09-21T21:41:38.591 INFO:teuthology.orchestra.run.smithi164.stdout:3805 punch from 0x1a5a200 to 0x1a5fe00, (0x5c00 bytes) 2024-09-21T21:41:38.618 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:38.617+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:38.619 INFO:teuthology.orchestra.run.smithi164.stdout:3806 write 0xe03c800 thru 0xe045b80 (0x9381 bytes) 2024-09-21T21:41:38.620 INFO:teuthology.orchestra.run.smithi164.stdout:3808 read 0xdb5600 thru 0xdc0911 (0xb312 bytes) 2024-09-21T21:41:38.620 INFO:teuthology.orchestra.run.smithi164.stdout:3809 write 0x3f65e00 thru 0x3f6bc30 (0x5e31 bytes) 2024-09-21T21:41:38.624 INFO:teuthology.orchestra.run.smithi164.stdout:3812 read 0x88f400 thru 0x895c3e (0x683f bytes) 2024-09-21T21:41:38.624 INFO:teuthology.orchestra.run.smithi164.stdout:3814 read 0x8722600 thru 0x87310a1 (0xeaa2 bytes) 2024-09-21T21:41:38.624 INFO:teuthology.orchestra.run.smithi164.stdout:3816 punch from 0x85ddc00 to 0x85e5000, (0x7400 bytes) 2024-09-21T21:41:38.625 INFO:teuthology.orchestra.run.smithi164.stdout:3817 write 0x8730400 thru 0x873dc5b (0xd85c bytes) 2024-09-21T21:41:38.625 INFO:teuthology.orchestra.run.smithi164.stdout:3818 read 0x5338600 thru 0x533bf9e (0x399f bytes) 2024-09-21T21:41:38.625 INFO:teuthology.orchestra.run.smithi164.stdout:3821 punch from 0x3cca800 to 0x3cd3200, (0x8a00 bytes) 2024-09-21T21:41:38.625 INFO:teuthology.orchestra.run.smithi164.stdout:3824 write 0xb5e0800 thru 0xb5ee219 (0xda1a bytes) 2024-09-21T21:41:38.625 INFO:teuthology.orchestra.run.smithi164.stdout:3825 punch from 0x6feb600 to 0x6ff9600, (0xe000 bytes) 2024-09-21T21:41:38.626 INFO:teuthology.orchestra.run.smithi164.stdout:3827 write 0x5397400 thru 0x5397de5 (0x9e6 bytes) 2024-09-21T21:41:38.644 INFO:teuthology.orchestra.run.smithi164.stdout:3829 trunc from 0xe045c00 to 0x411a00 2024-09-21T21:41:38.683 INFO:teuthology.orchestra.run.smithi164.stdout:3833 read 0x3baa00 thru 0x3c224d (0x784e bytes) 2024-09-21T21:41:38.684 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:38.685 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:41:38.685 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:41:38.685 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:41:38.685 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:38.694 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T21:41:38.714 INFO:teuthology.orchestra.run.smithi164.stdout:3834 write 0xbb2d800 thru 0xbb3998a (0xc18b bytes) 2024-09-21T21:41:38.730 INFO:teuthology.orchestra.run.smithi164.stdout:3835 write 0xdc1e200 thru 0xdc208a8 (0x26a9 bytes) 2024-09-21T21:41:38.733 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T21:41:38.734 INFO:teuthology.orchestra.run.smithi164.stdout:3837 write 0xcb9f200 thru 0xcbabeae (0xccaf bytes) 2024-09-21T21:41:38.734 INFO:teuthology.orchestra.run.smithi164.stdout:3839 write 0xcad7e00 thru 0xcadd6d2 (0x58d3 bytes) 2024-09-21T21:41:38.735 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:38.733+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:38.744 INFO:teuthology.orchestra.run.smithi164.stdout:3841 trunc from 0xdc20a00 to 0x2ce3400 2024-09-21T21:41:38.763 INFO:teuthology.orchestra.run.smithi164.stdout:3842 punch from 0xab8400 to 0xac6200, (0xde00 bytes) 2024-09-21T21:41:38.771 INFO:teuthology.orchestra.run.smithi164.stdout:3845 read 0x7b1e00 thru 0x7c17ee (0xf9ef bytes) 2024-09-21T21:41:38.784 INFO:teuthology.orchestra.run.smithi164.stdout:3846 trunc from 0x2ce3400 to 0xde68e00 2024-09-21T21:41:38.795 INFO:teuthology.orchestra.run.smithi164.stdout:3847 punch from 0xc65de00 to 0xc66cc00, (0xee00 bytes) 2024-09-21T21:41:38.802 INFO:teuthology.orchestra.run.smithi164.stdout:3849 write 0x3f81800 thru 0x3f9083d (0xf03e bytes) 2024-09-21T21:41:38.813 INFO:teuthology.orchestra.run.smithi164.stdout:3851 trunc from 0xde68e00 to 0x5015000 2024-09-21T21:41:38.818 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:38.816+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:38.849 INFO:teuthology.orchestra.run.smithi164.stdout:3852 write 0x822a800 thru 0x8239553 (0xed54 bytes) 2024-09-21T21:41:38.850 INFO:teuthology.orchestra.run.smithi164.stdout:3854 read 0x7101e00 thru 0x710d027 (0xb228 bytes) 2024-09-21T21:41:38.850 INFO:teuthology.orchestra.run.smithi164.stdout:3856 read 0x6d73800 thru 0x6d823f6 (0xebf7 bytes) 2024-09-21T21:41:38.850 INFO:teuthology.orchestra.run.smithi164.stdout:3858 read 0x49dd600 thru 0x49ebc13 (0xe614 bytes) 2024-09-21T21:41:38.850 INFO:teuthology.orchestra.run.smithi164.stdout:3859 clone 27 order 24 su 4194304 sc 10 2024-09-21T21:41:38.852 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:38.852 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 20, 2024-09-21T21:41:38.852 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 600, 2024-09-21T21:41:38.852 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:41:38.852 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.852 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4824.0:1564 3.2 3:4647520c:::rbd_data.1121b97f2925.0000000000000006:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:38.852 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:41:38.852 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 37.213400774, 2024-09-21T21:41:38.852 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056097347999999998, 2024-09-21T21:41:38.852 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:38.852 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:38.852 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:38.853 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:38.853 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4824", 2024-09-21T21:41:38.853 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:38.853 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1564 2024-09-21T21:41:38.853 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.853 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:38.853 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.853 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:38.853 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:41:38.853 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:38.853 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.853 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.853 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:38.853 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:41:38.854 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:38.854 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.854 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.854 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:38.854 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:41:38.854 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6499999999999999e-07 2024-09-21T21:41:38.854 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.854 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.854 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:38.854 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636067+0000", 2024-09-21T21:41:38.854 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3879999999999998e-06 2024-09-21T21:41:38.854 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.854 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.854 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:38.855 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636075+0000", 2024-09-21T21:41:38.855 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5480000000000004e-06 2024-09-21T21:41:38.855 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.855 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.855 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:38.855 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636080+0000", 2024-09-21T21:41:38.855 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.339e-06 2024-09-21T21:41:38.855 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.855 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.855 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:38.855 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636133+0000", 2024-09-21T21:41:38.855 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2930000000000003e-05 2024-09-21T21:41:38.855 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.855 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.856 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:38.856 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636265+0000", 2024-09-21T21:41:38.856 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000132058 2024-09-21T21:41:38.856 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.856 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.856 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:38.856 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.692162+0000", 2024-09-21T21:41:38.856 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055896620000000001 2024-09-21T21:41:38.856 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.856 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:38.856 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.856 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.856 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.856 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4912.0:871 3.19 3:9ddcaed2:::rbd_data.1121b3a54d7e.0000000000000000:12 [sparse-read 16121856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-21T21:41:38.857 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:20.721293+0000", 2024-09-21T21:41:38.857 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 18.128172819, 2024-09-21T21:41:38.857 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.026485022, 2024-09-21T21:41:38.857 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:38.857 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:38.857 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:38.857 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:38.857 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4912", 2024-09-21T21:41:38.857 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4204709190", 2024-09-21T21:41:38.857 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 871 2024-09-21T21:41:38.857 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.858 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:38.858 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.858 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:38.858 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.721293+0000", 2024-09-21T21:41:38.858 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:38.858 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.858 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.858 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:38.858 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.721293+0000", 2024-09-21T21:41:38.858 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:38.858 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.858 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.858 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:38.858 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.721293+0000", 2024-09-21T21:41:38.859 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3000000000000002e-07 2024-09-21T21:41:38.859 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.859 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.859 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:38.859 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.721295+0000", 2024-09-21T21:41:38.859 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.7749999999999999e-06 2024-09-21T21:41:38.859 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.859 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.859 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:38.859 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.721300+0000", 2024-09-21T21:41:38.859 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3319999999999999e-06 2024-09-21T21:41:38.859 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.859 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.860 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:38.860 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.721305+0000", 2024-09-21T21:41:38.860 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.578e-06 2024-09-21T21:41:38.860 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.860 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.860 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:38.860 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.747610+0000", 2024-09-21T21:41:38.860 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.026305469000000001 2024-09-21T21:41:38.860 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.860 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.860 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:38.860 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.747691+0000", 2024-09-21T21:41:38.861 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.1390999999999994e-05 2024-09-21T21:41:38.861 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.861 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.861 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:38.861 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.747778+0000", 2024-09-21T21:41:38.861 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.6147e-05 2024-09-21T21:41:38.861 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.861 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:38.861 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.861 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.861 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.862 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4912.0:938 3.19 3:9ddcaed2:::rbd_data.1121b3a54d7e.0000000000000000:12 [sparse-read 12845056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-21T21:41:38.862 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:20.723322+0000", 2024-09-21T21:41:38.862 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 18.126143207999998, 2024-09-21T21:41:38.862 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.026387079000000001, 2024-09-21T21:41:38.862 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:38.862 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:38.862 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:38.862 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:38.862 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4912", 2024-09-21T21:41:38.862 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4204709190", 2024-09-21T21:41:38.862 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 938 2024-09-21T21:41:38.862 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.862 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:38.862 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.863 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:38.863 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723322+0000", 2024-09-21T21:41:38.863 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:38.863 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.863 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.863 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:38.863 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723322+0000", 2024-09-21T21:41:38.863 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:38.863 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.863 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.863 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:38.864 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723322+0000", 2024-09-21T21:41:38.864 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.72e-07 2024-09-21T21:41:38.864 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.864 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.864 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:38.864 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723324+0000", 2024-09-21T21:41:38.864 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9250000000000002e-06 2024-09-21T21:41:38.864 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.864 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.864 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:38.864 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723330+0000", 2024-09-21T21:41:38.864 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6799999999999998e-06 2024-09-21T21:41:38.864 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.864 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.865 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:38.865 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723334+0000", 2024-09-21T21:41:38.865 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3640000000000001e-06 2024-09-21T21:41:38.865 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.865 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.865 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:38.865 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.749545+0000", 2024-09-21T21:41:38.865 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.026210715999999998 2024-09-21T21:41:38.865 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.865 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.865 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:38.865 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.749623+0000", 2024-09-21T21:41:38.865 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7602999999999995e-05 2024-09-21T21:41:38.865 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.866 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.866 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:38.866 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.749709+0000", 2024-09-21T21:41:38.866 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.6419e-05 2024-09-21T21:41:38.866 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.866 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:38.866 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.866 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.866 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.866 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4912.0:939 3.19 3:9ddcaed2:::rbd_data.1121b3a54d7e.0000000000000000:12 [sparse-read 14548992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-21T21:41:38.866 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:20.723340+0000", 2024-09-21T21:41:38.866 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 18.126125009999999, 2024-09-21T21:41:38.866 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.026505974000000002, 2024-09-21T21:41:38.867 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:38.867 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:38.867 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:38.867 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:38.867 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4912", 2024-09-21T21:41:38.867 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4204709190", 2024-09-21T21:41:38.867 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 939 2024-09-21T21:41:38.867 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.867 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:38.867 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.867 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:38.867 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723340+0000", 2024-09-21T21:41:38.867 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:38.867 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.868 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.868 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:38.868 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723340+0000", 2024-09-21T21:41:38.868 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:38.868 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.868 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.868 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:38.868 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723341+0000", 2024-09-21T21:41:38.868 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6100000000000002e-07 2024-09-21T21:41:38.868 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.868 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.868 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:38.868 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723342+0000", 2024-09-21T21:41:38.868 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.782e-06 2024-09-21T21:41:38.869 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.869 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.869 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:38.869 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723348+0000", 2024-09-21T21:41:38.869 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3990000000000003e-06 2024-09-21T21:41:38.869 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.869 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.869 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:38.869 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723353+0000", 2024-09-21T21:41:38.869 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3009999999999997e-06 2024-09-21T21:41:38.869 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.869 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.870 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:38.870 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.749741+0000", 2024-09-21T21:41:38.870 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.026387911999999999 2024-09-21T21:41:38.870 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.870 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.870 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:38.870 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.749788+0000", 2024-09-21T21:41:38.870 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7354000000000001e-05 2024-09-21T21:41:38.870 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.870 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.870 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:38.870 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.749846+0000", 2024-09-21T21:41:38.870 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7865000000000002e-05 2024-09-21T21:41:38.871 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.871 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:38.871 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.871 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.871 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.871 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4912.0:940 3.19 3:9ddcaed2:::rbd_data.1121b3a54d7e.0000000000000000:12 [sparse-read 16252928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-21T21:41:38.871 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:20.723360+0000", 2024-09-21T21:41:38.871 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 18.126105795000001, 2024-09-21T21:41:38.871 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.026697166000000001, 2024-09-21T21:41:38.871 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:38.872 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:38.872 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:38.872 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:38.872 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4912", 2024-09-21T21:41:38.872 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4204709190", 2024-09-21T21:41:38.872 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 940 2024-09-21T21:41:38.872 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.872 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:38.872 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.872 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:38.872 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723360+0000", 2024-09-21T21:41:38.872 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:38.872 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.873 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.873 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:38.873 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723360+0000", 2024-09-21T21:41:38.873 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:38.873 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.873 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.873 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:38.873 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723360+0000", 2024-09-21T21:41:38.873 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.9900000000000001e-07 2024-09-21T21:41:38.873 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.873 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.874 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:38.874 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723362+0000", 2024-09-21T21:41:38.874 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1289999999999999e-06 2024-09-21T21:41:38.874 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.874 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.874 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:38.874 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723368+0000", 2024-09-21T21:41:38.874 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.75e-06 2024-09-21T21:41:38.874 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.874 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.874 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:38.874 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723373+0000", 2024-09-21T21:41:38.875 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6959999999999996e-06 2024-09-21T21:41:38.875 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.875 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.875 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:38.875 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.749882+0000", 2024-09-21T21:41:38.875 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.026509893999999999 2024-09-21T21:41:38.875 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.875 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.875 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:38.875 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.749966+0000", 2024-09-21T21:41:38.876 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.3449999999999996e-05 2024-09-21T21:41:38.876 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.876 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.876 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:38.876 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.750057+0000", 2024-09-21T21:41:38.876 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.0847999999999995e-05 2024-09-21T21:41:38.876 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.876 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:38.876 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.876 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.876 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.876 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4912.0:999 3.19 3:9ddcaed2:::rbd_data.1121b3a54d7e.0000000000000000:12 [sparse-read 1441792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-21T21:41:38.876 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:20.723784+0000", 2024-09-21T21:41:38.876 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 18.125680855999999, 2024-09-21T21:41:38.877 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.026500717, 2024-09-21T21:41:38.877 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:38.877 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:38.877 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:38.877 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:38.877 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4912", 2024-09-21T21:41:38.877 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4204709190", 2024-09-21T21:41:38.877 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 999 2024-09-21T21:41:38.877 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.877 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:38.877 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.878 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:38.878 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723784+0000", 2024-09-21T21:41:38.878 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:38.878 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.878 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.878 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:38.878 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723784+0000", 2024-09-21T21:41:38.878 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:38.878 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.878 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.878 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:38.879 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723785+0000", 2024-09-21T21:41:38.879 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4900000000000001e-07 2024-09-21T21:41:38.879 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.879 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.879 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:38.879 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723787+0000", 2024-09-21T21:41:38.879 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9819999999999998e-06 2024-09-21T21:41:38.879 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.879 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.879 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:38.879 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723793+0000", 2024-09-21T21:41:38.880 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.032e-06 2024-09-21T21:41:38.880 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.880 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.880 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:38.880 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723798+0000", 2024-09-21T21:41:38.880 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7099999999999998e-06 2024-09-21T21:41:38.880 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.880 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.880 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:38.880 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.750104+0000", 2024-09-21T21:41:38.880 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.02630658 2024-09-21T21:41:38.881 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.881 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.881 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:38.881 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.750185+0000", 2024-09-21T21:41:38.881 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0345000000000004e-05 2024-09-21T21:41:38.881 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.881 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.881 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:38.881 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.750285+0000", 2024-09-21T21:41:38.881 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010061899999999999 2024-09-21T21:41:38.881 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.882 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:38.882 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.882 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.882 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.882 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4912.0:1000 3.19 3:9ddcaed2:::rbd_data.1121b3a54d7e.0000000000000000:12 [sparse-read 3145728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-21T21:41:38.882 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:20.723804+0000", 2024-09-21T21:41:38.882 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 18.125661674, 2024-09-21T21:41:38.882 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.026694361, 2024-09-21T21:41:38.882 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:38.882 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:38.882 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:38.882 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:38.882 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4912", 2024-09-21T21:41:38.883 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4204709190", 2024-09-21T21:41:38.883 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1000 2024-09-21T21:41:38.883 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.883 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:38.883 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.883 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:38.883 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723804+0000", 2024-09-21T21:41:38.883 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:38.883 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.883 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.883 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:38.883 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723804+0000", 2024-09-21T21:41:38.883 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:38.884 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.884 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.884 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:38.884 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723804+0000", 2024-09-21T21:41:38.884 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7399999999999999e-07 2024-09-21T21:41:38.884 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.884 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.884 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:38.884 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723811+0000", 2024-09-21T21:41:38.884 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.1149999999999996e-06 2024-09-21T21:41:38.884 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.884 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.884 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:38.884 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723818+0000", 2024-09-21T21:41:38.885 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9779999999999999e-06 2024-09-21T21:41:38.885 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.885 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.885 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:38.885 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723826+0000", 2024-09-21T21:41:38.885 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0600000000000008e-06 2024-09-21T21:41:38.885 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.885 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.885 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:38.885 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.750308+0000", 2024-09-21T21:41:38.885 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.026482316999999998 2024-09-21T21:41:38.885 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.885 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.885 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:38.885 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.750403+0000", 2024-09-21T21:41:38.886 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.4775e-05 2024-09-21T21:41:38.886 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.886 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.886 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:38.886 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.750498+0000", 2024-09-21T21:41:38.886 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.4741999999999995e-05 2024-09-21T21:41:38.886 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.886 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:38.886 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.886 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.886 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.886 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4912.0:1001 3.19 3:9ddcaed2:::rbd_data.1121b3a54d7e.0000000000000000:12 [sparse-read 4849664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-21T21:41:38.886 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:20.723833+0000", 2024-09-21T21:41:38.886 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 18.125632817, 2024-09-21T21:41:38.887 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.026886792999999999, 2024-09-21T21:41:38.887 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:38.887 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:38.887 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:38.887 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:38.887 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4912", 2024-09-21T21:41:38.887 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4204709190", 2024-09-21T21:41:38.887 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1001 2024-09-21T21:41:38.887 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.887 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:38.887 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.887 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:38.887 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723833+0000", 2024-09-21T21:41:38.887 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:38.888 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.888 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.888 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:38.888 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723833+0000", 2024-09-21T21:41:38.888 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:38.888 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.888 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.888 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:38.888 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723833+0000", 2024-09-21T21:41:38.888 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4099999999999999e-07 2024-09-21T21:41:38.888 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.888 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.888 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:38.888 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723835+0000", 2024-09-21T21:41:38.889 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.968e-06 2024-09-21T21:41:38.889 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.889 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.889 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:38.889 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723841+0000", 2024-09-21T21:41:38.889 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5209999999999999e-06 2024-09-21T21:41:38.889 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.889 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.889 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:38.889 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723846+0000", 2024-09-21T21:41:38.889 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9810000000000003e-06 2024-09-21T21:41:38.889 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.889 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.890 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:38.890 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.750538+0000", 2024-09-21T21:41:38.890 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.026692041 2024-09-21T21:41:38.890 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.890 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.890 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:38.890 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.750625+0000", 2024-09-21T21:41:38.890 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.6706000000000006e-05 2024-09-21T21:41:38.890 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.890 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.890 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:38.890 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.750719+0000", 2024-09-21T21:41:38.890 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.4135000000000006e-05 2024-09-21T21:41:38.890 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.890 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:38.891 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.891 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.891 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.891 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4912.0:1002 3.19 3:9ddcaed2:::rbd_data.1121b3a54d7e.0000000000000000:12 [sparse-read 6553600~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-21T21:41:38.891 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:20.723853+0000", 2024-09-21T21:41:38.891 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 18.125612713999999, 2024-09-21T21:41:38.891 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.027244385999999999, 2024-09-21T21:41:38.891 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:38.891 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:38.891 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:38.891 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:38.891 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4912", 2024-09-21T21:41:38.891 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4204709190", 2024-09-21T21:41:38.891 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1002 2024-09-21T21:41:38.891 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.892 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:38.892 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.892 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:38.892 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723853+0000", 2024-09-21T21:41:38.892 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:38.892 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.892 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.892 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:38.892 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723853+0000", 2024-09-21T21:41:38.892 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:38.892 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.892 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.892 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:38.892 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723853+0000", 2024-09-21T21:41:38.893 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6899999999999998e-07 2024-09-21T21:41:38.893 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.893 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.893 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:38.893 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723858+0000", 2024-09-21T21:41:38.893 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0710000000000002e-06 2024-09-21T21:41:38.893 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.893 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.893 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:38.893 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723865+0000", 2024-09-21T21:41:38.893 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.613e-06 2024-09-21T21:41:38.893 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.893 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.893 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:38.893 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723870+0000", 2024-09-21T21:41:38.894 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0350000000000001e-06 2024-09-21T21:41:38.894 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.894 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.894 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:38.894 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.750749+0000", 2024-09-21T21:41:38.894 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.026878808000000001 2024-09-21T21:41:38.894 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.894 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.894 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:38.894 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.750830+0000", 2024-09-21T21:41:38.894 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.1817999999999995e-05 2024-09-21T21:41:38.894 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.894 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.894 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:38.895 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.751097+0000", 2024-09-21T21:41:38.895 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00026667200000000002 2024-09-21T21:41:38.895 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.895 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:38.895 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.895 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.895 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.895 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4912.0:1003 3.19 3:9ddcaed2:::rbd_data.1121b3a54d7e.0000000000000000:12 [sparse-read 8257536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-21T21:41:38.895 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:20.723876+0000", 2024-09-21T21:41:38.895 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 18.125588937, 2024-09-21T21:41:38.895 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.027396406000000002, 2024-09-21T21:41:38.895 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:38.895 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:38.895 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:38.896 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:38.896 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4912", 2024-09-21T21:41:38.896 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4204709190", 2024-09-21T21:41:38.896 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1003 2024-09-21T21:41:38.896 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.896 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:38.896 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.896 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:38.896 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723876+0000", 2024-09-21T21:41:38.896 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:38.896 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.896 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.896 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:38.896 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723876+0000", 2024-09-21T21:41:38.897 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:38.897 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.897 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.897 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:38.897 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723877+0000", 2024-09-21T21:41:38.897 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5999999999999999e-07 2024-09-21T21:41:38.897 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.897 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.897 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:38.897 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723879+0000", 2024-09-21T21:41:38.897 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1299999999999999e-06 2024-09-21T21:41:38.897 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.897 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.897 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:38.897 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723885+0000", 2024-09-21T21:41:38.898 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9580000000000004e-06 2024-09-21T21:41:38.898 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.898 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.898 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:38.898 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723889+0000", 2024-09-21T21:41:38.898 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4850000000000003e-06 2024-09-21T21:41:38.898 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.898 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.898 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:38.898 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.751125+0000", 2024-09-21T21:41:38.898 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.027236136000000001 2024-09-21T21:41:38.898 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.898 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.898 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:38.899 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.751171+0000", 2024-09-21T21:41:38.899 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5708999999999999e-05 2024-09-21T21:41:38.899 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.899 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.899 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:38.899 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.751273+0000", 2024-09-21T21:41:38.899 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000101628 2024-09-21T21:41:38.899 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.899 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:38.899 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.899 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.899 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.899 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4912.0:1004 3.19 3:9ddcaed2:::rbd_data.1121b3a54d7e.0000000000000000:12 [sparse-read 9568256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-21T21:41:38.899 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:20.723895+0000", 2024-09-21T21:41:38.900 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 18.125570228000001, 2024-09-21T21:41:38.900 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.027611696000000002, 2024-09-21T21:41:38.900 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:38.900 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:38.900 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:38.900 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:38.900 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4912", 2024-09-21T21:41:38.900 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4204709190", 2024-09-21T21:41:38.900 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1004 2024-09-21T21:41:38.900 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.900 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:38.900 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.900 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:38.900 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723895+0000", 2024-09-21T21:41:38.900 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:38.901 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.901 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.901 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:38.901 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723895+0000", 2024-09-21T21:41:38.901 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:38.901 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.901 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.901 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:38.901 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723895+0000", 2024-09-21T21:41:38.901 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.1199999999999999e-07 2024-09-21T21:41:38.901 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.901 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.901 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:38.901 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723897+0000", 2024-09-21T21:41:38.901 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.8059999999999999e-06 2024-09-21T21:41:38.902 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.902 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.902 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:38.902 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723903+0000", 2024-09-21T21:41:38.902 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9939999999999997e-06 2024-09-21T21:41:38.902 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.902 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.902 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:38.902 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723908+0000", 2024-09-21T21:41:38.902 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5260000000000004e-06 2024-09-21T21:41:38.902 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.902 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.902 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:38.903 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.751316+0000", 2024-09-21T21:41:38.903 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.027408377000000001 2024-09-21T21:41:38.903 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.903 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.903 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:38.903 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.751402+0000", 2024-09-21T21:41:38.903 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.5830999999999999e-05 2024-09-21T21:41:38.903 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.903 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.903 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:38.903 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.751507+0000", 2024-09-21T21:41:38.903 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010485 2024-09-21T21:41:38.903 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.903 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:38.904 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.904 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.904 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.904 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4912.0:1005 3.19 3:9ddcaed2:::rbd_data.1121b3a54d7e.0000000000000000:12 [sparse-read 11272192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-21T21:41:38.904 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:20.723914+0000", 2024-09-21T21:41:38.904 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 18.125551457, 2024-09-21T21:41:38.904 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.027789695999999999, 2024-09-21T21:41:38.904 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:38.904 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:38.904 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:38.904 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:38.904 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4912", 2024-09-21T21:41:38.904 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4204709190", 2024-09-21T21:41:38.904 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1005 2024-09-21T21:41:38.904 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.905 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:38.905 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.905 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:38.905 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723914+0000", 2024-09-21T21:41:38.905 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:38.905 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.905 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.905 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:38.905 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723914+0000", 2024-09-21T21:41:38.905 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:38.905 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.905 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.905 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:38.905 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723914+0000", 2024-09-21T21:41:38.905 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4099999999999999e-07 2024-09-21T21:41:38.906 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.906 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.906 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:38.906 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723916+0000", 2024-09-21T21:41:38.906 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0559999999999999e-06 2024-09-21T21:41:38.906 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.906 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.906 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:38.906 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723923+0000", 2024-09-21T21:41:38.906 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.139e-06 2024-09-21T21:41:38.906 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.906 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.906 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:38.906 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723927+0000", 2024-09-21T21:41:38.907 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.826e-06 2024-09-21T21:41:38.907 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.907 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.907 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:38.907 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.751550+0000", 2024-09-21T21:41:38.907 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.027622357 2024-09-21T21:41:38.907 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.907 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.907 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:38.907 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.751636+0000", 2024-09-21T21:41:38.908 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.6604e-05 2024-09-21T21:41:38.908 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.908 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.908 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:38.908 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.751704+0000", 2024-09-21T21:41:38.908 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.7173000000000002e-05 2024-09-21T21:41:38.908 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.908 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:38.908 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.908 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.908 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.909 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4912.0:1006 3.19 3:9ddcaed2:::rbd_data.1121b3a54d7e.0000000000000000:12 [sparse-read 12976128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-21T21:41:38.909 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:20.723939+0000", 2024-09-21T21:41:38.909 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 18.125525975999999, 2024-09-21T21:41:38.909 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.028010094999999999, 2024-09-21T21:41:38.909 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:38.909 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:38.909 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:38.909 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:38.909 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4912", 2024-09-21T21:41:38.909 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4204709190", 2024-09-21T21:41:38.910 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1006 2024-09-21T21:41:38.910 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.910 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:38.910 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.910 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:38.910 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723939+0000", 2024-09-21T21:41:38.910 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:38.910 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.910 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.910 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:38.910 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723939+0000", 2024-09-21T21:41:38.910 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:38.911 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.911 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.911 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:38.911 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723940+0000", 2024-09-21T21:41:38.911 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4999999999999998e-07 2024-09-21T21:41:38.911 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.911 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.911 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:38.911 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723942+0000", 2024-09-21T21:41:38.911 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1660000000000001e-06 2024-09-21T21:41:38.912 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.912 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.912 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:38.912 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723949+0000", 2024-09-21T21:41:38.912 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0949999999999998e-06 2024-09-21T21:41:38.912 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.912 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.912 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:38.912 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723954+0000", 2024-09-21T21:41:38.912 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0250000000000002e-06 2024-09-21T21:41:38.912 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.912 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.913 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:38.913 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.751755+0000", 2024-09-21T21:41:38.913 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.027801228000000001 2024-09-21T21:41:38.913 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.913 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.913 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:38.913 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.751841+0000", 2024-09-21T21:41:38.913 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.5829999999999996e-05 2024-09-21T21:41:38.913 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.913 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.913 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:38.914 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.751949+0000", 2024-09-21T21:41:38.914 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010830100000000001 2024-09-21T21:41:38.914 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.914 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:38.914 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.914 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.914 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.914 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4912.0:1007 3.19 3:9ddcaed2:::rbd_data.1121b3a54d7e.0000000000000000:12 [sparse-read 14680064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-21T21:41:38.914 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:20.723961+0000", 2024-09-21T21:41:38.914 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 18.125504801000002, 2024-09-21T21:41:38.915 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.028245155000000001, 2024-09-21T21:41:38.915 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:38.915 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:38.915 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:38.915 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:38.915 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4912", 2024-09-21T21:41:38.915 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4204709190", 2024-09-21T21:41:38.915 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1007 2024-09-21T21:41:38.915 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.915 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:38.915 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.916 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:38.916 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723961+0000", 2024-09-21T21:41:38.916 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:38.916 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.916 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.916 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:38.916 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723961+0000", 2024-09-21T21:41:38.916 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:38.916 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.917 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.917 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:38.917 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723961+0000", 2024-09-21T21:41:38.917 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4900000000000001e-07 2024-09-21T21:41:38.917 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.917 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.917 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:38.917 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723963+0000", 2024-09-21T21:41:38.917 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.787e-06 2024-09-21T21:41:38.917 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.917 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.918 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:38.918 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723969+0000", 2024-09-21T21:41:38.918 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8289999999999996e-06 2024-09-21T21:41:38.918 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.918 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.918 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:38.918 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723973+0000", 2024-09-21T21:41:38.918 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5499999999999996e-06 2024-09-21T21:41:38.918 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.918 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.918 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:38.918 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.751994+0000", 2024-09-21T21:41:38.918 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.028021146 2024-09-21T21:41:38.918 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.919 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.919 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:38.919 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.752101+0000", 2024-09-21T21:41:38.919 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010621899999999999 2024-09-21T21:41:38.919 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.919 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.919 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:38.919 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.752206+0000", 2024-09-21T21:41:38.919 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000105175 2024-09-21T21:41:38.919 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.919 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:38.919 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.919 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.920 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.920 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4912.0:1008 3.19 3:9ddcaed2:::rbd_data.1121b3a54d7e.0000000000000000:12 [sparse-read 16384000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-21T21:41:38.920 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:20.723978+0000", 2024-09-21T21:41:38.920 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 18.125486980000002, 2024-09-21T21:41:38.920 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.028444661999999999, 2024-09-21T21:41:38.920 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:38.920 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:38.920 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:38.920 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:38.920 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4912", 2024-09-21T21:41:38.920 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4204709190", 2024-09-21T21:41:38.920 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1008 2024-09-21T21:41:38.920 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.920 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:38.920 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.921 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:38.921 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723978+0000", 2024-09-21T21:41:38.921 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:38.921 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.921 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.921 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:38.921 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723978+0000", 2024-09-21T21:41:38.921 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:38.921 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.921 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.921 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:38.922 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.724409+0000", 2024-09-21T21:41:38.922 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00043058300000000001 2024-09-21T21:41:38.922 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.922 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.922 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:38.922 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.724412+0000", 2024-09-21T21:41:38.922 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3730000000000001e-06 2024-09-21T21:41:38.922 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.922 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.922 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:38.922 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.724421+0000", 2024-09-21T21:41:38.922 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.9840000000000007e-06 2024-09-21T21:41:38.922 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.923 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.923 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:38.923 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.724427+0000", 2024-09-21T21:41:38.923 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2029999999999997e-06 2024-09-21T21:41:38.923 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.923 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.923 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:38.923 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.752238+0000", 2024-09-21T21:41:38.923 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.027810577999999999 2024-09-21T21:41:38.923 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.923 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.923 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:38.923 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.752323+0000", 2024-09-21T21:41:38.923 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.4652000000000002e-05 2024-09-21T21:41:38.924 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.924 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.924 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:38.924 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.752423+0000", 2024-09-21T21:41:38.924 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000100289 2024-09-21T21:41:38.924 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.924 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:38.924 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.924 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.924 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.924 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5196.0:984 3.2 3:404ccd4e:::rbd_data.1121a3eabc61.0000000000000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-21T21:41:38.924 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:27.956266+0000", 2024-09-21T21:41:38.924 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 10.893199162, 2024-09-21T21:41:38.924 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.032332125000000003, 2024-09-21T21:41:38.924 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:38.925 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:38.925 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:41:38.925 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:38.925 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5196", 2024-09-21T21:41:38.925 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3707540331", 2024-09-21T21:41:38.925 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 984 2024-09-21T21:41:38.925 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.925 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:38.925 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.925 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:38.925 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.956266+0000", 2024-09-21T21:41:38.925 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:38.925 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.925 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.926 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:38.926 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.956266+0000", 2024-09-21T21:41:38.926 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:38.926 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.926 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.926 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:38.926 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.956267+0000", 2024-09-21T21:41:38.926 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0800000000000004e-07 2024-09-21T21:41:38.926 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.926 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.926 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:38.926 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.956271+0000", 2024-09-21T21:41:38.926 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.0199999999999996e-06 2024-09-21T21:41:38.926 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.926 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.927 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:38.927 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.956281+0000", 2024-09-21T21:41:38.927 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0669000000000001e-05 2024-09-21T21:41:38.927 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.927 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.927 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:38.927 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.956288+0000", 2024-09-21T21:41:38.927 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9639999999999997e-06 2024-09-21T21:41:38.927 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.927 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.927 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:38.927 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.956337+0000", 2024-09-21T21:41:38.927 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8538999999999997e-05 2024-09-21T21:41:38.927 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.928 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.928 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:38.928 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.956413+0000", 2024-09-21T21:41:38.928 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5652e-05 2024-09-21T21:41:38.928 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.928 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.928 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:41:38.928 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.956519+0000", 2024-09-21T21:41:38.928 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010645 2024-09-21T21:41:38.928 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.928 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.928 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:41:38.928 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.988330+0000", 2024-09-21T21:41:38.928 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031810488999999997 2024-09-21T21:41:38.929 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.929 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.929 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:41:38.929 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.988532+0000", 2024-09-21T21:41:38.929 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00020196999999999999 2024-09-21T21:41:38.929 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.929 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.929 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:41:38.929 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.988563+0000", 2024-09-21T21:41:38.929 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.1951e-05 2024-09-21T21:41:38.929 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.929 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.929 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:38.929 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.988598+0000", 2024-09-21T21:41:38.930 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4813e-05 2024-09-21T21:41:38.930 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.930 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:38.930 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.930 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.930 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.930 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5196.0:991 3.7 3:e594ca4e:::rbd_header.1121a3eabc61:head [notify cookie 139700387993488] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-21T21:41:38.930 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:27.956339+0000", 2024-09-21T21:41:38.930 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 10.893125935, 2024-09-21T21:41:38.930 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031791316, 2024-09-21T21:41:38.930 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:38.930 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:38.930 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:38.930 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:38.931 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5196", 2024-09-21T21:41:38.931 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3707540331", 2024-09-21T21:41:38.931 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 991 2024-09-21T21:41:38.931 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.931 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:38.931 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.931 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:38.931 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.956339+0000", 2024-09-21T21:41:38.931 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:38.931 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.931 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.931 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:38.931 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.956339+0000", 2024-09-21T21:41:38.931 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:38.931 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.932 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.932 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:38.932 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.956340+0000", 2024-09-21T21:41:38.932 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7999999999999996e-07 2024-09-21T21:41:38.932 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.932 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.932 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:38.932 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.956343+0000", 2024-09-21T21:41:38.932 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.8260000000000001e-06 2024-09-21T21:41:38.932 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.932 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.932 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:38.933 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.956350+0000", 2024-09-21T21:41:38.933 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.1019999999999999e-06 2024-09-21T21:41:38.933 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.933 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.933 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:38.933 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.956355+0000", 2024-09-21T21:41:38.933 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6149999999999996e-06 2024-09-21T21:41:38.933 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.933 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.933 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:38.933 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.956507+0000", 2024-09-21T21:41:38.933 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000151963 2024-09-21T21:41:38.933 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.933 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.933 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:38.934 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.988010+0000", 2024-09-21T21:41:38.934 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031502636000000001 2024-09-21T21:41:38.934 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.934 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.934 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:38.934 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.988131+0000", 2024-09-21T21:41:38.934 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00012069399999999999 2024-09-21T21:41:38.934 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.934 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:38.934 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.934 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.934 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.934 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5196.0:992 3.7 3:e594ca4e:::rbd_header.1121a3eabc61:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-21T21:41:38.935 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:27.956555+0000", 2024-09-21T21:41:38.935 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 10.892910708, 2024-09-21T21:41:38.935 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031712787999999999, 2024-09-21T21:41:38.935 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:38.935 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:38.935 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:38.935 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:38.935 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5196", 2024-09-21T21:41:38.935 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3707540331", 2024-09-21T21:41:38.935 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 992 2024-09-21T21:41:38.935 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.935 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:38.935 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.935 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:38.935 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.956555+0000", 2024-09-21T21:41:38.936 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:38.936 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.936 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.936 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:38.936 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.956555+0000", 2024-09-21T21:41:38.936 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:38.936 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.936 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.936 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:38.936 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.956555+0000", 2024-09-21T21:41:38.936 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.2900000000000002e-07 2024-09-21T21:41:38.936 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.936 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.936 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:38.936 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.956561+0000", 2024-09-21T21:41:38.937 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1699999999999996e-06 2024-09-21T21:41:38.937 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.937 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.937 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:38.937 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.987985+0000", 2024-09-21T21:41:38.937 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031424872 2024-09-21T21:41:38.937 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.937 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.937 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:38.937 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.987996+0000", 2024-09-21T21:41:38.937 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0742e-05 2024-09-21T21:41:38.938 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.938 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.938 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:38.938 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.988161+0000", 2024-09-21T21:41:38.938 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00016449800000000001 2024-09-21T21:41:38.938 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.938 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.938 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:38.938 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.988214+0000", 2024-09-21T21:41:38.938 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3528e-05 2024-09-21T21:41:38.938 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.938 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.939 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:38.939 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.988267+0000", 2024-09-21T21:41:38.939 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3149000000000002e-05 2024-09-21T21:41:38.939 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.939 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:38.939 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.939 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.939 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.939 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4385.0:11863 3.7 3:e594ca4e:::rbd_header.1121a3eabc61:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-21T21:41:38.939 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:27.956588+0000", 2024-09-21T21:41:38.939 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 10.892877070000001, 2024-09-21T21:41:38.940 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031871075999999998, 2024-09-21T21:41:38.940 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:38.940 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:38.940 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:38.940 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:38.940 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4385", 2024-09-21T21:41:38.940 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:41:38.940 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11863 2024-09-21T21:41:38.940 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.940 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:38.940 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.940 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:38.941 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.956588+0000", 2024-09-21T21:41:38.941 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:38.941 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.941 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.941 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:38.941 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.956588+0000", 2024-09-21T21:41:38.941 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:38.941 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.941 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.941 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:38.941 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.956589+0000", 2024-09-21T21:41:38.941 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.1699999999999997e-07 2024-09-21T21:41:38.941 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.942 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.942 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:38.942 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.956593+0000", 2024-09-21T21:41:38.942 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1219999999999997e-06 2024-09-21T21:41:38.942 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.942 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.942 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:38.942 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.987976+0000", 2024-09-21T21:41:38.942 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031382366000000002 2024-09-21T21:41:38.942 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.943 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.943 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:38.943 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.987996+0000", 2024-09-21T21:41:38.943 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0010999999999999e-05 2024-09-21T21:41:38.943 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.943 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.943 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:38.943 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.988296+0000", 2024-09-21T21:41:38.943 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00030068500000000003 2024-09-21T21:41:38.943 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.943 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.944 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:38.944 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.988365+0000", 2024-09-21T21:41:38.944 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.8986999999999995e-05 2024-09-21T21:41:38.944 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.944 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.944 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:38.944 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.988459+0000", 2024-09-21T21:41:38.944 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.4087999999999998e-05 2024-09-21T21:41:38.944 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.944 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:38.944 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.944 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.944 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.945 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_repop_reply(client.5196.0:984 3.2 e58/21 ondisk, result = 0)", 2024-09-21T21:41:38.945 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:27.958905+0000", 2024-09-21T21:41:38.945 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 10.890559961999999, 2024-09-21T21:41:38.945 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.029433239, 2024-09-21T21:41:38.945 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:38.945 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:38.945 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:38.945 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:38.945 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.945 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:38.946 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.958905+0000", 2024-09-21T21:41:38.946 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:38.946 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.946 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.946 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:38.946 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.958905+0000", 2024-09-21T21:41:38.946 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:38.946 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.946 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.946 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:38.946 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.958906+0000", 2024-09-21T21:41:38.946 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.8199999999999999e-07 2024-09-21T21:41:38.947 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.947 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.947 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:38.947 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.958914+0000", 2024-09-21T21:41:38.947 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0180000000000001e-06 2024-09-21T21:41:38.947 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.947 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.947 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:38.947 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.987982+0000", 2024-09-21T21:41:38.948 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.029067829999999999 2024-09-21T21:41:38.948 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.948 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.948 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:38.948 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.987995+0000", 2024-09-21T21:41:38.948 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.3196999999999999e-05 2024-09-21T21:41:38.948 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.948 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.948 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:38.949 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.988312+0000", 2024-09-21T21:41:38.949 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00031657400000000002 2024-09-21T21:41:38.949 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.949 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.949 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:38.949 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.988323+0000", 2024-09-21T21:41:38.949 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1344e-05 2024-09-21T21:41:38.949 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:38.949 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:38.949 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:38.950 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.988339+0000", 2024-09-21T21:41:38.950 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.5594e-05 2024-09-21T21:41:38.950 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.950 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:38.950 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.950 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:38.950 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:38.950 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:38.951 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T21:41:38.952 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:38.950+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:38.952 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T21:41:39.052 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:39.050+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:39.099 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:39.099 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:41:39.099 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 0 2024-09-21T21:41:39.099 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:39.108 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T21:41:39.112 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0: {} 2024-09-21T21:41:39.113 INFO:teuthology.orchestra.run.smithi081.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T21:41:39.130 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1: {} 2024-09-21T21:41:39.131 INFO:teuthology.orchestra.run.smithi081.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T21:41:39.147 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2: {} 2024-09-21T21:41:39.147 INFO:teuthology.orchestra.run.smithi081.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T21:41:39.152 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:39.151+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:39.164 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3: {} 2024-09-21T21:41:39.164 INFO:teuthology.orchestra.run.smithi081.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T21:41:39.181 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4: {} 2024-09-21T21:41:39.181 INFO:teuthology.orchestra.run.smithi081.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T21:41:39.198 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5: {} 2024-09-21T21:41:39.198 INFO:teuthology.orchestra.run.smithi081.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T21:41:39.218 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:39.219 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:39.228 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T21:41:39.253 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:39.252+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:39.354 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:39.352+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:39.371 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:39.371 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:39.380 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T21:41:39.455 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:39.454+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:39.485 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:39.485 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:39.493 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T21:41:39.556 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:39.555+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:39.636 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:39.636 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:39.645 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T21:41:39.657 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:39.656+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:39.726 INFO:teuthology.orchestra.run.smithi081.stderr:nodeep-scrub is set 2024-09-21T21:41:39.749 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:39.749 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:39.758 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T21:41:39.781 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:39.780+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:39.858 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:39.857+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:39.860 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:39.860 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:39.869 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T21:41:39.959 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:39.958+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:39.974 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:39.974 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:39.983 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T21:41:40.060 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:40.058+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:40.089 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:40.089 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:40.098 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T21:41:40.136 INFO:teuthology.orchestra.run.smithi164.stdout:checking clone #25, image image_client.0-clone25 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent26 2024-09-21T21:41:40.161 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:40.159+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:40.206 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:40.206 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:40.215 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T21:41:40.237 INFO:teuthology.orchestra.run.smithi164.stdout:3860 punch from 0x29f5800 to 0x29fbc00, (0x6400 bytes) 2024-09-21T21:41:40.252 INFO:teuthology.orchestra.run.smithi164.stdout:3861 read 0x5010200 thru 0x501c9bd (0xc7be bytes) 2024-09-21T21:41:40.253 INFO:teuthology.orchestra.run.smithi164.stdout:3864 write 0x4d29000 thru 0x4d3473e (0xb73f bytes) 2024-09-21T21:41:40.262 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:40.260+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:40.269 INFO:teuthology.orchestra.run.smithi164.stdout:3865 write 0x93b2800 thru 0x93b3aab (0x12ac bytes) 2024-09-21T21:41:40.276 INFO:teuthology.orchestra.run.smithi164.stdout:3866 clone 28 order 22 su 131072 sc 10 2024-09-21T21:41:40.322 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:40.322 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:40.331 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T21:41:40.374 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:40.373+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:40.464 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:40.463+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:40.476 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:40.476 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:40.484 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T21:41:40.565 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:40.563+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:40.586 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:40.586 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:40.594 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T21:41:40.679 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:40.677+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:40.722 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-21T21:41:40.722 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T21:41:40.722 INFO:tasks.thrashosds.thrasher:Removing osd 3, in_osds are: [0, 1, 2, 3, 4, 5] 2024-09-21T21:41:40.722 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 3 2024-09-21T21:41:40.741 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:40.741 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:40.750 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T21:41:40.767 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:40.765+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:40.868 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:40.867+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:40.895 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:40.895 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:40.904 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T21:41:40.969 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:40.968+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:41.007 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:41.007 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:41.014 INFO:teuthology.orchestra.run.smithi164.stdout:truncating image image_client.0-clone27 from 0x93b3c00 (overlap 0x8239600) to 0x1a0b600 2024-09-21T21:41:41.016 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T21:41:41.022 INFO:tasks.ceph.mon.a.smithi081.stderr:2024-09-21T21:41:41.020+0000 7f5d67cb4640 -1 mon.a@0(leader).osd e69 definitely_dead 0 2024-09-21T21:41:41.071 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:41.069+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:41.127 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:41.127 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:41.136 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T21:41:41.172 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:41.171+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:41.212 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T21:41:41.241 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:41.242 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:41.251 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T21:41:41.272 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:41.271+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:41.327 INFO:teuthology.orchestra.run.smithi164.stdout:checking clone #26, image image_client.0-clone26 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent27 2024-09-21T21:41:41.336 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0: {} 2024-09-21T21:41:41.337 INFO:teuthology.orchestra.run.smithi081.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T21:41:41.354 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1: {} 2024-09-21T21:41:41.354 INFO:teuthology.orchestra.run.smithi081.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T21:41:41.373 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2: {} 2024-09-21T21:41:41.373 INFO:teuthology.orchestra.run.smithi081.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T21:41:41.373 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:41.371+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:41.393 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3: {} 2024-09-21T21:41:41.393 INFO:teuthology.orchestra.run.smithi081.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T21:41:41.408 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:41.408 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:41:41.408 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 0 2024-09-21T21:41:41.408 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:41.414 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4: {} 2024-09-21T21:41:41.415 INFO:teuthology.orchestra.run.smithi081.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T21:41:41.418 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T21:41:41.434 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5: {} 2024-09-21T21:41:41.434 INFO:teuthology.orchestra.run.smithi081.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T21:41:41.486 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:41.485+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:41.546 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:41.546 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:41:41.546 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:41:41.546 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:41:41.546 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:41.555 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T21:41:41.575 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:41.573+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:41.676 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:41.674+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:41.693 INFO:teuthology.orchestra.run.smithi164.stdout:3867 read 0x5bdea00 thru 0x5be2f94 (0x4595 bytes) 2024-09-21T21:41:41.694 INFO:teuthology.orchestra.run.smithi164.stdout:3868 write 0x2bfa800 thru 0x2c06710 (0xbf11 bytes) 2024-09-21T21:41:41.700 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:41.700 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 20, 2024-09-21T21:41:41.700 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 600, 2024-09-21T21:41:41.701 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:41:41.701 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.701 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:1678 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 14483456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:41.701 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.299288+0000", 2024-09-21T21:41:41.701 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 17.399187409, 2024-09-21T21:41:41.701 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031376253, 2024-09-21T21:41:41.701 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:41.701 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:41.701 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:41.701 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:41.701 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:41.701 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:41.701 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1678 2024-09-21T21:41:41.702 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.702 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:41.702 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.702 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:41.702 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299288+0000", 2024-09-21T21:41:41.702 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:41.702 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.702 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.702 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:41.703 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299288+0000", 2024-09-21T21:41:41.703 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:41.703 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.703 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.703 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:41.703 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299288+0000", 2024-09-21T21:41:41.703 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.2500000000000001e-07 2024-09-21T21:41:41.703 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.703 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.703 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:41.703 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299290+0000", 2024-09-21T21:41:41.703 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.945e-06 2024-09-21T21:41:41.703 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.703 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.704 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:41.704 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299295+0000", 2024-09-21T21:41:41.704 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0379999999999999e-06 2024-09-21T21:41:41.704 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.704 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.704 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:41.704 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299299+0000", 2024-09-21T21:41:41.704 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7960000000000002e-06 2024-09-21T21:41:41.704 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.704 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.704 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:41.704 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.330586+0000", 2024-09-21T21:41:41.705 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031287540000000003 2024-09-21T21:41:41.705 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.705 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.705 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:41.705 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.330620+0000", 2024-09-21T21:41:41.705 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3492999999999999e-05 2024-09-21T21:41:41.705 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.705 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.705 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:41.705 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.330664+0000", 2024-09-21T21:41:41.705 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4116000000000002e-05 2024-09-21T21:41:41.705 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:41.705 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:41.705 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:41.706 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.706 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.706 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:1679 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 14942208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:41.706 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.299333+0000", 2024-09-21T21:41:41.706 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 17.399141753999999, 2024-09-21T21:41:41.706 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031433567000000003, 2024-09-21T21:41:41.706 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:41.706 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:41.706 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:41.706 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:41.706 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:41.706 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:41.706 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1679 2024-09-21T21:41:41.706 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.707 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:41.707 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.707 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:41.707 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299333+0000", 2024-09-21T21:41:41.707 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:41.707 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.707 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.707 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:41.707 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299333+0000", 2024-09-21T21:41:41.707 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:41.707 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.707 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.707 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:41.707 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299334+0000", 2024-09-21T21:41:41.708 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0100000000000005e-07 2024-09-21T21:41:41.708 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.708 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.708 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:41.708 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299336+0000", 2024-09-21T21:41:41.708 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3439999999999999e-06 2024-09-21T21:41:41.708 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.708 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.708 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:41.708 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299343+0000", 2024-09-21T21:41:41.708 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0299999999999996e-06 2024-09-21T21:41:41.708 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.708 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.708 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:41.708 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299348+0000", 2024-09-21T21:41:41.709 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2340000000000002e-06 2024-09-21T21:41:41.709 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.709 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.709 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:41.709 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.330687+0000", 2024-09-21T21:41:41.709 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031338132999999997 2024-09-21T21:41:41.709 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.709 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.709 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:41.709 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.330721+0000", 2024-09-21T21:41:41.709 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4156999999999998e-05 2024-09-21T21:41:41.709 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.709 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.709 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:41.709 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.330767+0000", 2024-09-21T21:41:41.710 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6168000000000002e-05 2024-09-21T21:41:41.710 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:41.710 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:41.710 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:41.710 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.710 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.710 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:1680 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 15400960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:41.710 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.299356+0000", 2024-09-21T21:41:41.710 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 17.399119404, 2024-09-21T21:41:41.710 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031510096000000001, 2024-09-21T21:41:41.710 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:41.710 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:41.710 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:41.710 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:41.711 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:41.711 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:41.711 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1680 2024-09-21T21:41:41.711 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.711 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:41.711 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.711 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:41.711 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299356+0000", 2024-09-21T21:41:41.711 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:41.711 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.711 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.711 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:41.711 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299356+0000", 2024-09-21T21:41:41.711 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:41.712 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.712 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.712 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:41.712 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299356+0000", 2024-09-21T21:41:41.712 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.63e-07 2024-09-21T21:41:41.712 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.712 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.712 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:41.712 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299358+0000", 2024-09-21T21:41:41.712 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1399999999999998e-06 2024-09-21T21:41:41.712 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.712 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.712 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:41.712 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299364+0000", 2024-09-21T21:41:41.713 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1650000000000003e-06 2024-09-21T21:41:41.713 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.713 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.713 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:41.713 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299369+0000", 2024-09-21T21:41:41.713 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8400000000000002e-06 2024-09-21T21:41:41.713 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.713 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.713 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:41.713 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.330788+0000", 2024-09-21T21:41:41.713 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031418399 2024-09-21T21:41:41.713 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.713 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.713 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:41.713 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.330822+0000", 2024-09-21T21:41:41.714 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4218999999999999e-05 2024-09-21T21:41:41.714 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.714 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.714 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:41.714 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.330866+0000", 2024-09-21T21:41:41.714 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3869999999999998e-05 2024-09-21T21:41:41.714 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:41.714 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:41.714 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:41.714 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.714 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.714 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:1681 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 15859712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:41.714 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.299375+0000", 2024-09-21T21:41:41.715 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 17.399099700000001, 2024-09-21T21:41:41.715 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031600251000000003, 2024-09-21T21:41:41.715 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:41.715 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:41.715 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:41.715 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:41.715 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:41.715 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:41.715 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1681 2024-09-21T21:41:41.715 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.715 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:41.715 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.715 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:41.715 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299375+0000", 2024-09-21T21:41:41.715 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:41.716 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.716 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.716 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:41.716 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299375+0000", 2024-09-21T21:41:41.716 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:41.716 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.716 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.716 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:41.716 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299376+0000", 2024-09-21T21:41:41.717 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8700000000000001e-07 2024-09-21T21:41:41.717 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.717 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.717 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:41.717 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299382+0000", 2024-09-21T21:41:41.717 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3060000000000003e-06 2024-09-21T21:41:41.717 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.717 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.717 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:41.717 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299389+0000", 2024-09-21T21:41:41.717 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.3629999999999996e-06 2024-09-21T21:41:41.717 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.717 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.717 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:41.717 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299395+0000", 2024-09-21T21:41:41.718 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1279999999999999e-06 2024-09-21T21:41:41.718 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.718 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.718 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:41.718 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.330891+0000", 2024-09-21T21:41:41.718 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031496108000000002 2024-09-21T21:41:41.718 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.718 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.718 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:41.718 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.330925+0000", 2024-09-21T21:41:41.718 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4150000000000003e-05 2024-09-21T21:41:41.719 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.719 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.719 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:41.719 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.330976+0000", 2024-09-21T21:41:41.719 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0809000000000001e-05 2024-09-21T21:41:41.719 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:41.719 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:41.719 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:41.719 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.719 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.719 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:1682 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 16318464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:41.719 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.299401+0000", 2024-09-21T21:41:41.720 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 17.399074056, 2024-09-21T21:41:41.720 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031676932999999997, 2024-09-21T21:41:41.720 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:41.720 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:41.720 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:41.720 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:41.720 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:41.720 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:41.720 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1682 2024-09-21T21:41:41.720 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.720 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:41.720 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.721 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:41.721 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299401+0000", 2024-09-21T21:41:41.721 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:41.721 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.721 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.721 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:41.721 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299401+0000", 2024-09-21T21:41:41.721 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:41.721 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.721 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.721 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:41.721 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299401+0000", 2024-09-21T21:41:41.721 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.9299999999999999e-07 2024-09-21T21:41:41.722 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.722 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.722 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:41.722 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299404+0000", 2024-09-21T21:41:41.722 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1720000000000001e-06 2024-09-21T21:41:41.722 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.722 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.722 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:41.722 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299410+0000", 2024-09-21T21:41:41.722 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2480000000000001e-06 2024-09-21T21:41:41.723 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.723 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.723 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:41.723 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299414+0000", 2024-09-21T21:41:41.723 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6480000000000002e-06 2024-09-21T21:41:41.723 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.723 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.723 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:41.723 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.330998+0000", 2024-09-21T21:41:41.723 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031583364000000003 2024-09-21T21:41:41.723 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.723 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.723 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:41.723 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.331033+0000", 2024-09-21T21:41:41.724 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4759999999999999e-05 2024-09-21T21:41:41.724 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.724 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.724 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:41.724 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.331078+0000", 2024-09-21T21:41:41.724 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5348e-05 2024-09-21T21:41:41.724 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:41.724 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:41.724 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:41.724 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.724 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.724 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:2010 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 7864320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:41.724 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.306462+0000", 2024-09-21T21:41:41.724 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 17.39201297, 2024-09-21T21:41:41.725 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031317032000000002, 2024-09-21T21:41:41.725 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:41.725 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:41.725 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:41.725 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:41.725 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:41.725 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:41.725 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2010 2024-09-21T21:41:41.725 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.725 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:41.725 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.725 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:41.725 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306462+0000", 2024-09-21T21:41:41.725 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:41.726 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.726 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.726 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:41.726 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306462+0000", 2024-09-21T21:41:41.726 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:41.726 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.726 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.726 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:41.726 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306463+0000", 2024-09-21T21:41:41.726 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2500000000000001e-07 2024-09-21T21:41:41.726 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.726 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.726 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:41.727 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306465+0000", 2024-09-21T21:41:41.727 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9740000000000001e-06 2024-09-21T21:41:41.727 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.727 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.727 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:41.727 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306470+0000", 2024-09-21T21:41:41.727 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9689999999999999e-06 2024-09-21T21:41:41.727 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.727 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.727 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:41.727 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306475+0000", 2024-09-21T21:41:41.728 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7199999999999997e-06 2024-09-21T21:41:41.728 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.728 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.728 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:41.728 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.337701+0000", 2024-09-21T21:41:41.728 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031225796 2024-09-21T21:41:41.728 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.728 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.728 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:41.728 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.337735+0000", 2024-09-21T21:41:41.728 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4399e-05 2024-09-21T21:41:41.729 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.729 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.729 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:41.729 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.337779+0000", 2024-09-21T21:41:41.729 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3748999999999997e-05 2024-09-21T21:41:41.729 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:41.729 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:41.729 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:41.729 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.729 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.730 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:2011 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 8323072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:41.730 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.306481+0000", 2024-09-21T21:41:41.730 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 17.391993866, 2024-09-21T21:41:41.730 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031404840000000003, 2024-09-21T21:41:41.730 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:41.730 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:41.730 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:41.730 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:41.730 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:41.730 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:41.731 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2011 2024-09-21T21:41:41.731 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.731 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:41.731 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.731 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:41.731 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306481+0000", 2024-09-21T21:41:41.731 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:41.731 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.731 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.731 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:41.732 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306481+0000", 2024-09-21T21:41:41.732 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:41.732 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.732 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.732 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:41.732 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306482+0000", 2024-09-21T21:41:41.732 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9100000000000004e-07 2024-09-21T21:41:41.732 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.732 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.732 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:41.732 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306484+0000", 2024-09-21T21:41:41.732 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.855e-06 2024-09-21T21:41:41.732 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.732 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.732 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:41.733 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306489+0000", 2024-09-21T21:41:41.733 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7799999999999997e-06 2024-09-21T21:41:41.733 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.733 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.733 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:41.733 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306494+0000", 2024-09-21T21:41:41.733 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7539999999999997e-06 2024-09-21T21:41:41.733 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.733 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.733 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:41.733 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.337802+0000", 2024-09-21T21:41:41.733 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031307684000000002 2024-09-21T21:41:41.733 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.733 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.733 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:41.734 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.337836+0000", 2024-09-21T21:41:41.734 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4560000000000001e-05 2024-09-21T21:41:41.734 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.734 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.734 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:41.734 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.337886+0000", 2024-09-21T21:41:41.734 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9716000000000003e-05 2024-09-21T21:41:41.734 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:41.734 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:41.734 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:41.734 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.734 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.734 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:2012 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 8781824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:41.735 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.306500+0000", 2024-09-21T21:41:41.735 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 17.391975050999999, 2024-09-21T21:41:41.735 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031483165, 2024-09-21T21:41:41.735 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:41.735 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:41.735 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:41.735 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:41.735 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:41.735 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:41.735 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2012 2024-09-21T21:41:41.735 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.735 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:41.735 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.735 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:41.735 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306500+0000", 2024-09-21T21:41:41.736 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:41.736 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.736 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.736 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:41.736 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306500+0000", 2024-09-21T21:41:41.736 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:41.736 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.736 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.736 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:41.736 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306500+0000", 2024-09-21T21:41:41.736 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7099999999999997e-07 2024-09-21T21:41:41.736 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.736 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.736 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:41.736 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306502+0000", 2024-09-21T21:41:41.737 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1019999999999999e-06 2024-09-21T21:41:41.737 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.737 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.737 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:41.737 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306508+0000", 2024-09-21T21:41:41.737 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8390000000000003e-06 2024-09-21T21:41:41.737 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.737 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.737 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:41.737 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306513+0000", 2024-09-21T21:41:41.737 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4909999999999999e-06 2024-09-21T21:41:41.737 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.737 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.737 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:41.738 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.337898+0000", 2024-09-21T21:41:41.738 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031385437000000002 2024-09-21T21:41:41.738 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.738 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.738 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:41.738 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.337931+0000", 2024-09-21T21:41:41.738 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.2656e-05 2024-09-21T21:41:41.738 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.738 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.738 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:41.738 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.337983+0000", 2024-09-21T21:41:41.738 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2268999999999997e-05 2024-09-21T21:41:41.738 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:41.738 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:41.739 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:41.739 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.739 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.739 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:2013 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 9240576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:41.739 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.306520+0000", 2024-09-21T21:41:41.739 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 17.391955323000001, 2024-09-21T21:41:41.739 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031570864999999997, 2024-09-21T21:41:41.739 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:41.739 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:41.739 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:41.739 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:41.739 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:41.739 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:41.739 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2013 2024-09-21T21:41:41.740 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.740 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:41.740 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.740 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:41.740 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306520+0000", 2024-09-21T21:41:41.740 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:41.740 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.740 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.740 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:41.740 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306520+0000", 2024-09-21T21:41:41.740 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:41.740 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.740 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.740 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:41.741 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306520+0000", 2024-09-21T21:41:41.741 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3700000000000001e-07 2024-09-21T21:41:41.741 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.741 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.741 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:41.741 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306530+0000", 2024-09-21T21:41:41.741 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0336000000000001e-05 2024-09-21T21:41:41.741 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.741 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.741 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:41.741 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306538+0000", 2024-09-21T21:41:41.741 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.3690000000000001e-06 2024-09-21T21:41:41.741 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.741 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.741 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:41.742 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306543+0000", 2024-09-21T21:41:41.742 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8930000000000004e-06 2024-09-21T21:41:41.742 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.742 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.742 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:41.742 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338005+0000", 2024-09-21T21:41:41.742 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031461973999999997 2024-09-21T21:41:41.742 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.742 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.742 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:41.742 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338039+0000", 2024-09-21T21:41:41.742 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4545999999999998e-05 2024-09-21T21:41:41.743 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.743 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.743 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:41.743 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338091+0000", 2024-09-21T21:41:41.743 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1409999999999997e-05 2024-09-21T21:41:41.743 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:41.743 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:41.743 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:41.743 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.743 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.743 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:2014 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 9699328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:41.743 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.306553+0000", 2024-09-21T21:41:41.743 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 17.39192237, 2024-09-21T21:41:41.743 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031642225000000003, 2024-09-21T21:41:41.743 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:41.744 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:41.744 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:41.744 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:41.744 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:41.744 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:41.744 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2014 2024-09-21T21:41:41.744 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.744 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:41.744 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.744 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:41.744 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306553+0000", 2024-09-21T21:41:41.744 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:41.744 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.744 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.745 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:41.745 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306553+0000", 2024-09-21T21:41:41.745 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:41.745 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.745 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.745 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:41.745 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306553+0000", 2024-09-21T21:41:41.745 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5600000000000001e-07 2024-09-21T21:41:41.745 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.745 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.745 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:41.745 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306555+0000", 2024-09-21T21:41:41.745 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3070000000000001e-06 2024-09-21T21:41:41.745 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.745 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.746 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:41.746 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306561+0000", 2024-09-21T21:41:41.746 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8860000000000001e-06 2024-09-21T21:41:41.746 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.746 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.746 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:41.746 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306566+0000", 2024-09-21T21:41:41.746 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3420000000000001e-06 2024-09-21T21:41:41.746 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.746 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.746 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:41.746 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338115+0000", 2024-09-21T21:41:41.746 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031549333999999998 2024-09-21T21:41:41.746 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.747 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.747 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:41.747 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338150+0000", 2024-09-21T21:41:41.747 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4649e-05 2024-09-21T21:41:41.747 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.747 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.747 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:41.747 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338195+0000", 2024-09-21T21:41:41.747 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5250999999999998e-05 2024-09-21T21:41:41.747 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:41.747 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:41.747 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:41.747 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.747 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.748 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:2015 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 10158080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:41.748 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.306572+0000", 2024-09-21T21:41:41.748 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 17.391903452000001, 2024-09-21T21:41:41.748 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031729593, 2024-09-21T21:41:41.748 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:41.748 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:41.748 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:41.748 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:41.748 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:41.748 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:41.748 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2015 2024-09-21T21:41:41.748 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.748 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:41.748 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.748 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:41.749 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306572+0000", 2024-09-21T21:41:41.749 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:41.749 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.749 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.749 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:41.749 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306572+0000", 2024-09-21T21:41:41.749 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:41.749 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.749 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.749 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:41.749 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306572+0000", 2024-09-21T21:41:41.749 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7500000000000001e-07 2024-09-21T21:41:41.749 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.749 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.750 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:41.750 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306574+0000", 2024-09-21T21:41:41.750 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0669999999999999e-06 2024-09-21T21:41:41.750 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.750 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.750 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:41.750 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306579+0000", 2024-09-21T21:41:41.750 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4009999999999996e-06 2024-09-21T21:41:41.750 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.750 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.750 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:41.750 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306585+0000", 2024-09-21T21:41:41.750 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3349999999999997e-06 2024-09-21T21:41:41.750 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.751 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.751 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:41.751 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338217+0000", 2024-09-21T21:41:41.751 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031631805999999998 2024-09-21T21:41:41.751 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.751 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.751 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:41.751 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338251+0000", 2024-09-21T21:41:41.751 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4813e-05 2024-09-21T21:41:41.751 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.751 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.751 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:41.751 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338301+0000", 2024-09-21T21:41:41.751 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9796000000000002e-05 2024-09-21T21:41:41.752 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:41.752 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:41.752 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:41.752 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.752 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.752 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:2016 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 10616832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:41.752 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.306601+0000", 2024-09-21T21:41:41.752 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 17.391874279, 2024-09-21T21:41:41.752 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031801840999999997, 2024-09-21T21:41:41.752 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:41.753 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:41.753 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:41.753 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:41.753 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:41.753 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:41.753 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2016 2024-09-21T21:41:41.753 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.753 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:41.753 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.753 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:41.753 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306601+0000", 2024-09-21T21:41:41.753 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:41.753 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.753 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.754 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:41.754 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306601+0000", 2024-09-21T21:41:41.754 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:41.754 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.754 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.754 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:41.754 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306601+0000", 2024-09-21T21:41:41.754 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1900000000000003e-07 2024-09-21T21:41:41.754 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.754 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.755 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:41.755 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306603+0000", 2024-09-21T21:41:41.755 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.17e-06 2024-09-21T21:41:41.755 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.755 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.755 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:41.755 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306609+0000", 2024-09-21T21:41:41.755 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8069999999999996e-06 2024-09-21T21:41:41.755 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.755 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.755 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:41.755 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306614+0000", 2024-09-21T21:41:41.755 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5090000000000004e-06 2024-09-21T21:41:41.755 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.756 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.756 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:41.756 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338325+0000", 2024-09-21T21:41:41.756 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031711237000000003 2024-09-21T21:41:41.756 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.756 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.756 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:41.756 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338359+0000", 2024-09-21T21:41:41.756 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3810000000000003e-05 2024-09-21T21:41:41.756 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.756 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.757 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:41.757 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338403+0000", 2024-09-21T21:41:41.757 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3788999999999997e-05 2024-09-21T21:41:41.757 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:41.757 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:41.757 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:41.757 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.757 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.757 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:2017 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 11075584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:41.757 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.306621+0000", 2024-09-21T21:41:41.757 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 17.391854512999998, 2024-09-21T21:41:41.757 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031891271999999998, 2024-09-21T21:41:41.757 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:41.757 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:41.758 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:41.758 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:41.758 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:41.758 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:41.758 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2017 2024-09-21T21:41:41.758 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.758 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:41.758 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.758 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:41.758 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306621+0000", 2024-09-21T21:41:41.758 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:41.759 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.759 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.759 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:41.759 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306621+0000", 2024-09-21T21:41:41.759 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:41.759 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.759 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.759 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:41.759 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306621+0000", 2024-09-21T21:41:41.759 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5400000000000002e-07 2024-09-21T21:41:41.759 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.759 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.759 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:41.760 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306623+0000", 2024-09-21T21:41:41.760 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1100000000000001e-06 2024-09-21T21:41:41.760 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.760 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.760 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:41.760 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306629+0000", 2024-09-21T21:41:41.760 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7810000000000002e-06 2024-09-21T21:41:41.760 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.760 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.760 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:41.760 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306633+0000", 2024-09-21T21:41:41.760 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5589999999999999e-06 2024-09-21T21:41:41.761 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.761 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.761 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:41.761 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338427+0000", 2024-09-21T21:41:41.761 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031793948000000002 2024-09-21T21:41:41.761 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.761 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.761 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:41.761 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338462+0000", 2024-09-21T21:41:41.761 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4785e-05 2024-09-21T21:41:41.762 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.762 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.762 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:41.762 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338512+0000", 2024-09-21T21:41:41.762 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9734999999999997e-05 2024-09-21T21:41:41.762 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:41.762 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:41.762 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:41.762 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.762 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.762 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:2018 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 11534336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:41.763 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.306639+0000", 2024-09-21T21:41:41.763 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 17.391835883999999, 2024-09-21T21:41:41.763 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031963189000000003, 2024-09-21T21:41:41.763 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:41.763 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:41.763 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:41.763 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:41.763 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:41.763 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:41.763 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2018 2024-09-21T21:41:41.763 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.764 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:41.764 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.764 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:41.764 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306639+0000", 2024-09-21T21:41:41.764 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:41.764 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.764 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.764 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:41.764 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306639+0000", 2024-09-21T21:41:41.764 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:41.765 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.765 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.765 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:41.765 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306640+0000", 2024-09-21T21:41:41.765 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4600000000000005e-07 2024-09-21T21:41:41.765 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.765 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.765 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:41.765 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306642+0000", 2024-09-21T21:41:41.765 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.916e-06 2024-09-21T21:41:41.765 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.765 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.765 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:41.766 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306647+0000", 2024-09-21T21:41:41.766 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7590000000000003e-06 2024-09-21T21:41:41.766 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.766 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.766 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:41.766 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306652+0000", 2024-09-21T21:41:41.766 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4580000000000003e-06 2024-09-21T21:41:41.766 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.766 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.766 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:41.766 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338524+0000", 2024-09-21T21:41:41.766 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031872172999999997 2024-09-21T21:41:41.766 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.766 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.767 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:41.767 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338557+0000", 2024-09-21T21:41:41.767 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3136000000000003e-05 2024-09-21T21:41:41.767 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.767 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.767 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:41.767 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338602+0000", 2024-09-21T21:41:41.767 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5201000000000003e-05 2024-09-21T21:41:41.767 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:41.767 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:41.767 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:41.767 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.767 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.767 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:2019 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 11993088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:41.768 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.306658+0000", 2024-09-21T21:41:41.768 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 17.391817490000001, 2024-09-21T21:41:41.768 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.032059007, 2024-09-21T21:41:41.768 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:41.768 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:41.768 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:41.768 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:41.768 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:41.768 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:41.768 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2019 2024-09-21T21:41:41.768 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.768 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:41.768 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.768 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:41.768 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306658+0000", 2024-09-21T21:41:41.769 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:41.769 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.769 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.769 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:41.769 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306658+0000", 2024-09-21T21:41:41.769 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:41.769 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.769 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.769 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:41.769 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.307950+0000", 2024-09-21T21:41:41.769 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.001292199 2024-09-21T21:41:41.769 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.769 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.769 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:41.770 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.307954+0000", 2024-09-21T21:41:41.770 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5689999999999998e-06 2024-09-21T21:41:41.770 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.770 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.770 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:41.770 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.307964+0000", 2024-09-21T21:41:41.770 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6749999999999997e-06 2024-09-21T21:41:41.770 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.770 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.770 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:41.770 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.307971+0000", 2024-09-21T21:41:41.770 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5189999999999998e-06 2024-09-21T21:41:41.770 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.770 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.771 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:41.771 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338625+0000", 2024-09-21T21:41:41.771 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.030654569 2024-09-21T21:41:41.771 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.771 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.771 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:41.771 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338660+0000", 2024-09-21T21:41:41.771 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4656999999999997e-05 2024-09-21T21:41:41.771 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.771 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.771 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:41.771 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338717+0000", 2024-09-21T21:41:41.771 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6818999999999999e-05 2024-09-21T21:41:41.771 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:41.772 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:41.772 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:41.772 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.772 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.772 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:2026 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 15204352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:41.772 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.308109+0000", 2024-09-21T21:41:41.772 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 17.390365889999998, 2024-09-21T21:41:41.772 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031315883000000003, 2024-09-21T21:41:41.772 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:41.772 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:41.772 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:41.772 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:41.772 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:41.772 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:41.772 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2026 2024-09-21T21:41:41.773 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.773 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:41.773 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.773 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:41.773 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308109+0000", 2024-09-21T21:41:41.773 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:41.773 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.773 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.773 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:41.773 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308109+0000", 2024-09-21T21:41:41.773 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:41.773 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.773 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.773 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:41.774 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308110+0000", 2024-09-21T21:41:41.774 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.96e-07 2024-09-21T21:41:41.774 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.774 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.774 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:41.774 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308112+0000", 2024-09-21T21:41:41.774 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0269999999999998e-06 2024-09-21T21:41:41.774 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.774 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.774 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:41.774 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308117+0000", 2024-09-21T21:41:41.774 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.631e-06 2024-09-21T21:41:41.774 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.775 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.775 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:41.775 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308122+0000", 2024-09-21T21:41:41.775 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7310000000000001e-06 2024-09-21T21:41:41.775 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.775 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.775 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:41.775 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.339341+0000", 2024-09-21T21:41:41.775 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031218610000000001 2024-09-21T21:41:41.775 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.775 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.775 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:41.775 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.339376+0000", 2024-09-21T21:41:41.775 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5305000000000002e-05 2024-09-21T21:41:41.776 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.776 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.776 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:41.776 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.339425+0000", 2024-09-21T21:41:41.776 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9183e-05 2024-09-21T21:41:41.776 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:41.776 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:41.776 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:41.776 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.776 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.776 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:2027 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 15663104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:41.776 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.308128+0000", 2024-09-21T21:41:41.776 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 17.390347353999999, 2024-09-21T21:41:41.776 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031387452000000003, 2024-09-21T21:41:41.776 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:41.777 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:41.777 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:41.777 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:41.777 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:41.777 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:41.777 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2027 2024-09-21T21:41:41.777 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.777 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:41.777 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.777 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:41.777 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308128+0000", 2024-09-21T21:41:41.777 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:41.777 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.777 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.777 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:41.778 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308128+0000", 2024-09-21T21:41:41.778 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:41.778 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.778 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.778 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:41.778 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308128+0000", 2024-09-21T21:41:41.778 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4700000000000002e-07 2024-09-21T21:41:41.778 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.778 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.778 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:41.778 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308130+0000", 2024-09-21T21:41:41.778 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.8670000000000001e-06 2024-09-21T21:41:41.778 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.778 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.779 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:41.779 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308136+0000", 2024-09-21T21:41:41.779 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0299999999999999e-06 2024-09-21T21:41:41.779 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.779 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.779 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:41.779 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308140+0000", 2024-09-21T21:41:41.779 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3830000000000002e-06 2024-09-21T21:41:41.779 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.779 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.779 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:41.779 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.339437+0000", 2024-09-21T21:41:41.779 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031296864000000001 2024-09-21T21:41:41.779 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.780 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.780 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:41.780 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.339471+0000", 2024-09-21T21:41:41.780 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3302000000000001e-05 2024-09-21T21:41:41.780 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.780 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.780 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:41.780 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.339515+0000", 2024-09-21T21:41:41.780 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4659000000000001e-05 2024-09-21T21:41:41.780 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:41.780 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:41.780 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:41.780 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.780 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.780 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:2028 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 16121856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:41.781 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.308147+0000", 2024-09-21T21:41:41.781 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 17.390328427, 2024-09-21T21:41:41.781 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031474477000000001, 2024-09-21T21:41:41.781 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:41.781 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:41.781 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:41.781 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:41.781 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:41.781 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:41.781 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2028 2024-09-21T21:41:41.781 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.781 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:41.781 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.781 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:41.782 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308147+0000", 2024-09-21T21:41:41.782 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:41.782 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.782 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.782 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:41.782 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308147+0000", 2024-09-21T21:41:41.782 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:41.782 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.782 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.782 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:41.782 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308147+0000", 2024-09-21T21:41:41.782 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1900000000000003e-07 2024-09-21T21:41:41.782 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.782 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.782 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:41.783 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308149+0000", 2024-09-21T21:41:41.783 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9479999999999998e-06 2024-09-21T21:41:41.783 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.783 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.783 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:41.783 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308155+0000", 2024-09-21T21:41:41.783 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6200000000000004e-06 2024-09-21T21:41:41.783 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.783 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.783 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:41.783 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308160+0000", 2024-09-21T21:41:41.783 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0799999999999996e-06 2024-09-21T21:41:41.783 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.783 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.784 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:41.784 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.339539+0000", 2024-09-21T21:41:41.784 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031378902 2024-09-21T21:41:41.784 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.784 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.784 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:41.784 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.339574+0000", 2024-09-21T21:41:41.784 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5176999999999999e-05 2024-09-21T21:41:41.784 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.784 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.784 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:41.784 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.339621+0000", 2024-09-21T21:41:41.784 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7231000000000002e-05 2024-09-21T21:41:41.784 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:41.784 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:41.785 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:41.785 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.785 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.785 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:2029 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 16580608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:41.785 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.308167+0000", 2024-09-21T21:41:41.785 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 17.390308038000001, 2024-09-21T21:41:41.785 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031555582999999998, 2024-09-21T21:41:41.785 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:41.785 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:41.785 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:41.785 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:41.786 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:41.786 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:41.786 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2029 2024-09-21T21:41:41.786 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.786 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:41.786 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.786 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:41.786 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308167+0000", 2024-09-21T21:41:41.786 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:41.786 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.786 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.786 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:41.786 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308167+0000", 2024-09-21T21:41:41.787 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:41.787 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.787 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.787 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:41.787 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308167+0000", 2024-09-21T21:41:41.787 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6600000000000002e-07 2024-09-21T21:41:41.787 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.787 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.787 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:41.787 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308169+0000", 2024-09-21T21:41:41.787 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9819999999999998e-06 2024-09-21T21:41:41.787 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.787 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.787 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:41.788 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308175+0000", 2024-09-21T21:41:41.788 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8289999999999996e-06 2024-09-21T21:41:41.788 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.788 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.788 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:41.788 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308180+0000", 2024-09-21T21:41:41.788 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6439999999999999e-06 2024-09-21T21:41:41.788 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.788 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.788 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:41.788 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.339645+0000", 2024-09-21T21:41:41.789 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031465083999999997 2024-09-21T21:41:41.789 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.789 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.789 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:41.789 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.339679+0000", 2024-09-21T21:41:41.789 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3572999999999999e-05 2024-09-21T21:41:41.789 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.789 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.789 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:41.789 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.339723+0000", 2024-09-21T21:41:41.789 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4104999999999999e-05 2024-09-21T21:41:41.790 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:41.790 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:41.790 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:41.790 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.790 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.790 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:2059 3.5 3:a6b41377:::rbd_data.11213654c685.0000000000000008:head [stat,truncate 13631488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:41.790 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.308186+0000", 2024-09-21T21:41:41.790 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 17.390289459999998, 2024-09-21T21:41:41.791 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.032997110000000003, 2024-09-21T21:41:41.791 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:41.791 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:41.791 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:41:41.791 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:41.791 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:41.791 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:41.791 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2059 2024-09-21T21:41:41.791 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.791 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:41.791 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.791 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:41.792 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308186+0000", 2024-09-21T21:41:41.792 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:41.792 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.792 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.792 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:41.792 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308186+0000", 2024-09-21T21:41:41.792 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:41.792 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.792 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.792 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:41.792 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308186+0000", 2024-09-21T21:41:41.792 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1899999999999998e-07 2024-09-21T21:41:41.793 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.793 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.793 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:41.793 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308188+0000", 2024-09-21T21:41:41.793 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0099999999999998e-06 2024-09-21T21:41:41.793 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.793 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.793 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:41.793 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308195+0000", 2024-09-21T21:41:41.793 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0589999999999997e-06 2024-09-21T21:41:41.793 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.794 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.794 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:41.794 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308199+0000", 2024-09-21T21:41:41.794 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2930000000000002e-06 2024-09-21T21:41:41.794 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.794 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.794 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:41.794 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.339745+0000", 2024-09-21T21:41:41.794 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031546091999999998 2024-09-21T21:41:41.794 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.794 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.794 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:41.794 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.339782+0000", 2024-09-21T21:41:41.795 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6847999999999997e-05 2024-09-21T21:41:41.795 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.795 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.795 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:41:41.795 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.339856+0000", 2024-09-21T21:41:41.795 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.3992999999999994e-05 2024-09-21T21:41:41.795 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.795 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.795 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:41:41.795 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.341013+0000", 2024-09-21T21:41:41.795 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00115642 2024-09-21T21:41:41.796 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.796 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.796 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:41:41.796 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.341138+0000", 2024-09-21T21:41:41.796 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00012540699999999999 2024-09-21T21:41:41.796 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.796 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.796 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:41:41.796 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.341164+0000", 2024-09-21T21:41:41.796 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.6168000000000001e-05 2024-09-21T21:41:41.796 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:41.796 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:41.796 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:41.796 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.341183+0000", 2024-09-21T21:41:41.797 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.8400999999999999e-05 2024-09-21T21:41:41.797 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:41.797 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:41.797 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:41.797 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:41.797 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:41.797 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:41.797 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T21:41:41.799 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:41.797+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:41.800 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T21:41:41.801 INFO:teuthology.orchestra.run.smithi164.stdout:3869 write 0xc0a1000 thru 0xc0a8015 (0x7016 bytes) 2024-09-21T21:41:41.801 INFO:teuthology.orchestra.run.smithi164.stdout:3871 punch from 0x5755800 to 0x5763a00, (0xe200 bytes) 2024-09-21T21:41:41.801 INFO:teuthology.orchestra.run.smithi164.stdout:3876 write 0x6759400 thru 0x6765ec0 (0xcac1 bytes) 2024-09-21T21:41:41.801 INFO:teuthology.orchestra.run.smithi164.stdout:3877 write 0x34b800 thru 0x353d8f (0x8590 bytes) 2024-09-21T21:41:41.801 INFO:teuthology.orchestra.run.smithi164.stdout:3880 write 0x9e89400 thru 0x9e93b8e (0xa78f bytes) 2024-09-21T21:41:41.801 INFO:teuthology.orchestra.run.smithi164.stdout:3881 read 0x9b2f400 thru 0x9b3bbc4 (0xc7c5 bytes) 2024-09-21T21:41:41.801 INFO:teuthology.orchestra.run.smithi164.stdout:3882 clone 29 order 24 su 65536 sc 11 2024-09-21T21:41:41.802 INFO:teuthology.orchestra.run.smithi081.stderr:marked out osd.3. 2024-09-21T21:41:41.899 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:41.898+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:41.905 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:41.905 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:41:41.905 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-21T21:41:41.905 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:41.914 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T21:41:42.000 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:41.998+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:42.057 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:42.057 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:41:42.057 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:41:42.057 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:41:42.057 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:42.065 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T21:41:42.106 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:42.105+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:42.201 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:42.200+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:42.204 INFO:teuthology.orchestra.run.smithi081.stderr:noscrub is unset 2024-09-21T21:41:42.213 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:42.213 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-21T21:41:42.213 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-21T21:41:42.213 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:41:42.213 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.213 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2617 3.d 3:b26310db:::rbd_data.112145fae779.0000000000000006:a [read 3686400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:42.213 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.411596+0000", 2024-09-21T21:41:42.213 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 43.798559050000001, 2024-09-21T21:41:42.213 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.045094515000000002, 2024-09-21T21:41:42.214 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:42.214 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:42.214 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:42.214 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:42.214 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:42.214 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:42.214 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2617 2024-09-21T21:41:42.214 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.214 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:42.214 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.214 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:42.214 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411596+0000", 2024-09-21T21:41:42.214 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.214 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.215 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.215 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:42.215 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411596+0000", 2024-09-21T21:41:42.215 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.215 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.215 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.215 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:42.215 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411596+0000", 2024-09-21T21:41:42.215 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3500000000000002e-07 2024-09-21T21:41:42.215 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.215 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.215 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:42.215 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411598+0000", 2024-09-21T21:41:42.215 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.871e-06 2024-09-21T21:41:42.215 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.216 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.216 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:42.216 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411603+0000", 2024-09-21T21:41:42.216 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6420000000000004e-06 2024-09-21T21:41:42.216 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.216 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.216 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:42.216 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411608+0000", 2024-09-21T21:41:42.216 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4780000000000002e-06 2024-09-21T21:41:42.216 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.216 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.216 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:42.216 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.456529+0000", 2024-09-21T21:41:42.217 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044921232999999998 2024-09-21T21:41:42.217 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.217 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.217 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:42.217 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.456616+0000", 2024-09-21T21:41:42.217 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6712000000000006e-05 2024-09-21T21:41:42.217 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.217 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.217 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:42.217 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.456690+0000", 2024-09-21T21:41:42.217 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4244000000000003e-05 2024-09-21T21:41:42.218 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.218 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:42.218 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.218 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.218 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.218 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2618 3.d 3:b26310db:::rbd_data.112145fae779.0000000000000006:a [read 3743744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:42.218 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.411613+0000", 2024-09-21T21:41:42.218 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 43.798541471, 2024-09-21T21:41:42.219 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.045180423999999997, 2024-09-21T21:41:42.219 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:42.219 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:42.219 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:42.219 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:42.219 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:42.219 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:42.219 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2618 2024-09-21T21:41:42.220 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.220 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:42.220 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.220 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:42.220 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411613+0000", 2024-09-21T21:41:42.220 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.220 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.220 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.220 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:42.220 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411613+0000", 2024-09-21T21:41:42.220 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.220 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.220 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.220 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:42.221 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.412868+0000", 2024-09-21T21:41:42.221 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012549 2024-09-21T21:41:42.221 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.221 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.221 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:42.221 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.412873+0000", 2024-09-21T21:41:42.221 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9060000000000001e-06 2024-09-21T21:41:42.221 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.221 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.221 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:42.221 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.412884+0000", 2024-09-21T21:41:42.222 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1164e-05 2024-09-21T21:41:42.222 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.222 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.222 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:42.222 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.412892+0000", 2024-09-21T21:41:42.222 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5719999999999997e-06 2024-09-21T21:41:42.222 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.222 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.222 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:42.222 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.456717+0000", 2024-09-21T21:41:42.222 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043825559 2024-09-21T21:41:42.223 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.223 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.223 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:42.223 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.456753+0000", 2024-09-21T21:41:42.223 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6158000000000002e-05 2024-09-21T21:41:42.223 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.223 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.223 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:42.223 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.456794+0000", 2024-09-21T21:41:42.223 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0164999999999999e-05 2024-09-21T21:41:42.223 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.223 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:42.223 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.223 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.224 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.224 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5018.0:2937 3.15 3:af2e3e9a:::rbd_data.11213fc02c9d.0000000000000000:19 [read 475136~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T21:41:42.224 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:33.865486+0000", 2024-09-21T21:41:42.224 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 8.3446686549999995, 2024-09-21T21:41:42.224 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047419950000000002, 2024-09-21T21:41:42.224 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:42.224 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:42.224 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:42.224 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:42.224 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5018", 2024-09-21T21:41:42.224 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3046875242", 2024-09-21T21:41:42.224 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2937 2024-09-21T21:41:42.224 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.224 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:42.225 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.225 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:42.225 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865486+0000", 2024-09-21T21:41:42.225 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.225 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.225 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.225 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:42.225 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865486+0000", 2024-09-21T21:41:42.225 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.225 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.225 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.225 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:42.225 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865486+0000", 2024-09-21T21:41:42.225 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1800000000000001e-07 2024-09-21T21:41:42.225 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.226 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.226 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:42.226 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865488+0000", 2024-09-21T21:41:42.226 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9230000000000001e-06 2024-09-21T21:41:42.226 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.226 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.226 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:42.226 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865496+0000", 2024-09-21T21:41:42.226 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6260000000000003e-06 2024-09-21T21:41:42.226 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.226 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.226 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:42.226 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865501+0000", 2024-09-21T21:41:42.227 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7620000000000003e-06 2024-09-21T21:41:42.227 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.227 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.227 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:42.227 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.912653+0000", 2024-09-21T21:41:42.227 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047152816 2024-09-21T21:41:42.227 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.227 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.227 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:42.227 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.912768+0000", 2024-09-21T21:41:42.227 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000114558 2024-09-21T21:41:42.227 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.228 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.228 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:42.228 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.912906+0000", 2024-09-21T21:41:42.228 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000137847 2024-09-21T21:41:42.228 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.228 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:42.228 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.228 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.228 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.228 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5018.0:2938 3.15 3:af2e3e9a:::rbd_data.11213fc02c9d.0000000000000000:19 [read 417792~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T21:41:42.229 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:33.865505+0000", 2024-09-21T21:41:42.229 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 8.3446499190000001, 2024-09-21T21:41:42.229 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047654654999999997, 2024-09-21T21:41:42.229 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:42.229 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:42.229 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:42.229 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:42.229 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5018", 2024-09-21T21:41:42.229 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3046875242", 2024-09-21T21:41:42.229 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2938 2024-09-21T21:41:42.229 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.230 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:42.230 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.230 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:42.230 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865505+0000", 2024-09-21T21:41:42.230 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.230 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.230 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.230 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:42.230 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865505+0000", 2024-09-21T21:41:42.230 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.230 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.231 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.231 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:42.231 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865505+0000", 2024-09-21T21:41:42.231 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2999999999999999e-07 2024-09-21T21:41:42.231 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.231 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.231 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:42.231 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865506+0000", 2024-09-21T21:41:42.231 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2979999999999999e-06 2024-09-21T21:41:42.231 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.231 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.231 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:42.231 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865509+0000", 2024-09-21T21:41:42.232 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.163e-06 2024-09-21T21:41:42.232 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.232 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.232 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:42.232 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865512+0000", 2024-09-21T21:41:42.232 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7429999999999998e-06 2024-09-21T21:41:42.232 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.232 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.232 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:42.233 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.912962+0000", 2024-09-21T21:41:42.233 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047450349000000003 2024-09-21T21:41:42.233 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.233 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.233 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:42.233 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.913054+0000", 2024-09-21T21:41:42.233 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.1384999999999993e-05 2024-09-21T21:41:42.233 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.233 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.233 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:42.233 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.913159+0000", 2024-09-21T21:41:42.234 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000105487 2024-09-21T21:41:42.234 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.234 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:42.234 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.234 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.234 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.234 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5018.0:2939 3.15 3:af2e3e9a:::rbd_data.11213fc02c9d.0000000000000000:19 [read 425984~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T21:41:42.234 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:33.865515+0000", 2024-09-21T21:41:42.234 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 8.3446394189999999, 2024-09-21T21:41:42.234 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047853433000000001, 2024-09-21T21:41:42.234 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:42.234 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:42.234 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:42.235 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:42.235 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5018", 2024-09-21T21:41:42.235 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3046875242", 2024-09-21T21:41:42.235 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2939 2024-09-21T21:41:42.235 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.235 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:42.235 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.235 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:42.235 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865515+0000", 2024-09-21T21:41:42.235 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.235 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.235 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.235 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:42.236 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865515+0000", 2024-09-21T21:41:42.236 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.236 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.236 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.236 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:42.236 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865516+0000", 2024-09-21T21:41:42.236 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6800000000000001e-07 2024-09-21T21:41:42.236 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.236 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.236 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:42.236 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865517+0000", 2024-09-21T21:41:42.236 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.083e-06 2024-09-21T21:41:42.236 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.236 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.236 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:42.237 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865520+0000", 2024-09-21T21:41:42.237 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0699999999999998e-06 2024-09-21T21:41:42.237 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.237 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.237 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:42.237 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865523+0000", 2024-09-21T21:41:42.237 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.959e-06 2024-09-21T21:41:42.237 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.237 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.237 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:42.237 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.913199+0000", 2024-09-21T21:41:42.237 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047676653999999999 2024-09-21T21:41:42.237 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.238 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.238 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:42.238 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.913275+0000", 2024-09-21T21:41:42.238 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6154000000000003e-05 2024-09-21T21:41:42.238 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.238 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.238 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:42.238 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.913369+0000", 2024-09-21T21:41:42.238 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3145000000000004e-05 2024-09-21T21:41:42.238 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.239 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:42.239 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.239 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.239 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.239 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5018.0:2940 3.15 3:af2e3e9a:::rbd_data.11213fc02c9d.0000000000000000:19 [read 434176~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T21:41:42.239 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:33.865528+0000", 2024-09-21T21:41:42.239 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 8.3446269649999998, 2024-09-21T21:41:42.239 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.048052481000000001, 2024-09-21T21:41:42.239 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:42.239 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:42.240 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:42.240 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:42.240 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5018", 2024-09-21T21:41:42.240 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3046875242", 2024-09-21T21:41:42.240 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2940 2024-09-21T21:41:42.240 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.240 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:42.240 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.240 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:42.240 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865528+0000", 2024-09-21T21:41:42.240 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.240 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.240 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.241 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:42.241 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865528+0000", 2024-09-21T21:41:42.241 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.241 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.241 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.241 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:42.241 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865528+0000", 2024-09-21T21:41:42.241 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3500000000000002e-07 2024-09-21T21:41:42.241 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.241 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.241 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:42.241 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865530+0000", 2024-09-21T21:41:42.241 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1689999999999999e-06 2024-09-21T21:41:42.241 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.242 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.242 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:42.242 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865536+0000", 2024-09-21T21:41:42.242 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8560000000000003e-06 2024-09-21T21:41:42.242 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.242 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.242 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:42.242 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865541+0000", 2024-09-21T21:41:42.242 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.899e-06 2024-09-21T21:41:42.242 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.242 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.243 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:42.243 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.913406+0000", 2024-09-21T21:41:42.243 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047865216000000002 2024-09-21T21:41:42.243 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.243 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.243 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:42.243 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.913483+0000", 2024-09-21T21:41:42.243 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6525000000000003e-05 2024-09-21T21:41:42.243 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.243 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.243 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:42.243 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.913580+0000", 2024-09-21T21:41:42.243 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.7480999999999997e-05 2024-09-21T21:41:42.243 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.243 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:42.244 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.244 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.244 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.244 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5018.0:2941 3.15 3:af2e3e9a:::rbd_data.11213fc02c9d.0000000000000000:19 [read 483328~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T21:41:42.244 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:33.865558+0000", 2024-09-21T21:41:42.244 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 8.344596696, 2024-09-21T21:41:42.244 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.048220161999999997, 2024-09-21T21:41:42.244 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:42.244 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:42.244 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:42.244 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:42.244 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5018", 2024-09-21T21:41:42.244 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3046875242", 2024-09-21T21:41:42.244 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2941 2024-09-21T21:41:42.245 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.245 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:42.245 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.245 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:42.245 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865558+0000", 2024-09-21T21:41:42.245 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.245 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.245 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.245 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:42.245 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865558+0000", 2024-09-21T21:41:42.245 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.245 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.245 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.245 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:42.246 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865558+0000", 2024-09-21T21:41:42.246 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1799999999999995e-07 2024-09-21T21:41:42.246 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.246 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.246 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:42.246 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865561+0000", 2024-09-21T21:41:42.247 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.317e-06 2024-09-21T21:41:42.247 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.247 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.247 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:42.247 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865568+0000", 2024-09-21T21:41:42.247 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8510000000000001e-06 2024-09-21T21:41:42.247 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.248 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.248 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:42.248 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865573+0000", 2024-09-21T21:41:42.248 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1730000000000003e-06 2024-09-21T21:41:42.248 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.248 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.248 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:42.249 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.913617+0000", 2024-09-21T21:41:42.249 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.048044302999999997 2024-09-21T21:41:42.249 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.249 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.249 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:42.249 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.913689+0000", 2024-09-21T21:41:42.249 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1658999999999999e-05 2024-09-21T21:41:42.249 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.250 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.250 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:42.250 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.913778+0000", 2024-09-21T21:41:42.250 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9340999999999997e-05 2024-09-21T21:41:42.250 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.250 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:42.250 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.251 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.251 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.251 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5018.0:2942 3.15 3:af2e3e9a:::rbd_data.11213fc02c9d.0000000000000000:19 [read 491520~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T21:41:42.251 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:33.865580+0000", 2024-09-21T21:41:42.251 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 8.344574197, 2024-09-21T21:41:42.251 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.048410436000000001, 2024-09-21T21:41:42.251 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:42.252 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:42.252 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:42.252 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:42.252 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5018", 2024-09-21T21:41:42.252 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3046875242", 2024-09-21T21:41:42.252 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2942 2024-09-21T21:41:42.252 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.253 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:42.253 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.253 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:42.253 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865580+0000", 2024-09-21T21:41:42.253 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.253 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.254 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.254 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:42.254 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865580+0000", 2024-09-21T21:41:42.254 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.254 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.254 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.254 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:42.254 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865581+0000", 2024-09-21T21:41:42.255 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.51e-07 2024-09-21T21:41:42.255 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.255 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.255 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:42.255 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865583+0000", 2024-09-21T21:41:42.255 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.12e-06 2024-09-21T21:41:42.255 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.255 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.255 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:42.255 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865589+0000", 2024-09-21T21:41:42.255 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3080000000000004e-06 2024-09-21T21:41:42.255 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.256 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.256 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:42.256 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865594+0000", 2024-09-21T21:41:42.256 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9300000000000002e-06 2024-09-21T21:41:42.256 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.256 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.256 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:42.256 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.913813+0000", 2024-09-21T21:41:42.256 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.048218924000000003 2024-09-21T21:41:42.256 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.256 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.256 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:42.256 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.913891+0000", 2024-09-21T21:41:42.256 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7523999999999998e-05 2024-09-21T21:41:42.257 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.257 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.257 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:42.257 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.913991+0000", 2024-09-21T21:41:42.257 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000100179 2024-09-21T21:41:42.257 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.257 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:42.257 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.257 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.257 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.257 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5018.0:2943 3.15 3:af2e3e9a:::rbd_data.11213fc02c9d.0000000000000000:19 [read 393216~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T21:41:42.257 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:33.865612+0000", 2024-09-21T21:41:42.257 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 8.3445420620000004, 2024-09-21T21:41:42.258 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.048585616999999998, 2024-09-21T21:41:42.258 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:42.258 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:42.258 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:42.258 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:42.258 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5018", 2024-09-21T21:41:42.258 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3046875242", 2024-09-21T21:41:42.258 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2943 2024-09-21T21:41:42.258 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.258 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:42.258 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.258 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:42.258 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865612+0000", 2024-09-21T21:41:42.259 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.259 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.259 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.259 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:42.259 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865612+0000", 2024-09-21T21:41:42.259 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.259 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.259 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.259 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:42.259 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865613+0000", 2024-09-21T21:41:42.259 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9200000000000002e-07 2024-09-21T21:41:42.260 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.260 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.260 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:42.260 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865618+0000", 2024-09-21T21:41:42.260 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9729999999999997e-06 2024-09-21T21:41:42.260 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.260 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.260 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:42.260 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865623+0000", 2024-09-21T21:41:42.260 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6970000000000001e-06 2024-09-21T21:41:42.260 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.261 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.261 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:42.261 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865626+0000", 2024-09-21T21:41:42.261 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0879999999999999e-06 2024-09-21T21:41:42.261 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.261 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.261 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:42.261 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.914028+0000", 2024-09-21T21:41:42.261 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.048401921000000001 2024-09-21T21:41:42.261 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.261 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.261 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:42.261 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.914104+0000", 2024-09-21T21:41:42.262 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6312999999999999e-05 2024-09-21T21:41:42.262 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.262 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.262 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:42.262 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.914198+0000", 2024-09-21T21:41:42.262 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.4233000000000004e-05 2024-09-21T21:41:42.262 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.262 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:42.262 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.262 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.262 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.262 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5018.0:2944 3.15 3:af2e3e9a:::rbd_data.11213fc02c9d.0000000000000000:19 [read 442368~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T21:41:42.262 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:33.865654+0000", 2024-09-21T21:41:42.263 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 8.3445005590000001, 2024-09-21T21:41:42.263 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.048771901999999999, 2024-09-21T21:41:42.263 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:42.263 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:42.263 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:42.263 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:42.263 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5018", 2024-09-21T21:41:42.263 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3046875242", 2024-09-21T21:41:42.263 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2944 2024-09-21T21:41:42.263 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.264 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:42.264 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.264 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:42.264 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865654+0000", 2024-09-21T21:41:42.264 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.264 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.264 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.264 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:42.264 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865654+0000", 2024-09-21T21:41:42.264 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.264 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.264 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.264 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:42.264 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865655+0000", 2024-09-21T21:41:42.265 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4199999999999995e-07 2024-09-21T21:41:42.265 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.265 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.265 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:42.265 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865656+0000", 2024-09-21T21:41:42.265 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.8139999999999999e-06 2024-09-21T21:41:42.265 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.265 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.265 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:42.265 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865661+0000", 2024-09-21T21:41:42.265 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6489999999999999e-06 2024-09-21T21:41:42.265 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.265 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.265 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:42.265 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865664+0000", 2024-09-21T21:41:42.266 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2119999999999999e-06 2024-09-21T21:41:42.266 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.266 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.266 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:42.266 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.914236+0000", 2024-09-21T21:41:42.266 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.048571707999999998 2024-09-21T21:41:42.266 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.266 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.266 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:42.266 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.914319+0000", 2024-09-21T21:41:42.266 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2689999999999996e-05 2024-09-21T21:41:42.266 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.266 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.266 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:42.267 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.914426+0000", 2024-09-21T21:41:42.267 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000107187 2024-09-21T21:41:42.267 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.267 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:42.267 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.267 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.267 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.267 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5018.0:2945 3.15 3:af2e3e9a:::rbd_data.11213fc02c9d.0000000000000000:19 [read 507904~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T21:41:42.267 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:33.865670+0000", 2024-09-21T21:41:42.267 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 8.3444850529999997, 2024-09-21T21:41:42.267 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.048961808000000002, 2024-09-21T21:41:42.267 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:42.267 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:42.267 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:42.267 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:42.268 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5018", 2024-09-21T21:41:42.268 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3046875242", 2024-09-21T21:41:42.268 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2945 2024-09-21T21:41:42.268 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.268 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:42.268 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.268 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:42.268 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865670+0000", 2024-09-21T21:41:42.268 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.268 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.268 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.268 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:42.269 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865670+0000", 2024-09-21T21:41:42.269 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.269 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.269 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.269 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:42.269 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865670+0000", 2024-09-21T21:41:42.269 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8400000000000005e-07 2024-09-21T21:41:42.269 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.269 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.269 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:42.269 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865672+0000", 2024-09-21T21:41:42.269 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.691e-06 2024-09-21T21:41:42.269 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.270 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.270 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:42.270 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865676+0000", 2024-09-21T21:41:42.270 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5390000000000001e-06 2024-09-21T21:41:42.270 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.270 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.270 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:42.270 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865680+0000", 2024-09-21T21:41:42.270 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.67e-06 2024-09-21T21:41:42.270 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.270 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.270 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:42.271 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.914460+0000", 2024-09-21T21:41:42.271 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.048780221999999998 2024-09-21T21:41:42.271 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.271 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.271 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:42.271 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.914539+0000", 2024-09-21T21:41:42.271 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8634000000000006e-05 2024-09-21T21:41:42.271 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.271 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.272 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:42.272 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.914631+0000", 2024-09-21T21:41:42.272 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.2567999999999999e-05 2024-09-21T21:41:42.272 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.272 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:42.272 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.272 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.272 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.272 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5018.0:2946 3.15 3:af2e3e9a:::rbd_data.11213fc02c9d.0000000000000000:19 [read 450560~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T21:41:42.272 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:33.865694+0000", 2024-09-21T21:41:42.272 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 8.3444601249999995, 2024-09-21T21:41:42.272 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.049163473999999999, 2024-09-21T21:41:42.272 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:42.272 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:42.272 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:42.273 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:42.273 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5018", 2024-09-21T21:41:42.273 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3046875242", 2024-09-21T21:41:42.273 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2946 2024-09-21T21:41:42.273 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.273 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:42.273 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.273 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:42.273 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865694+0000", 2024-09-21T21:41:42.273 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.273 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.273 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.274 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:42.274 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865694+0000", 2024-09-21T21:41:42.274 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.274 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.274 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.274 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:42.274 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865695+0000", 2024-09-21T21:41:42.274 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3299999999999998e-07 2024-09-21T21:41:42.274 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.274 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.274 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:42.275 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865697+0000", 2024-09-21T21:41:42.275 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.08e-06 2024-09-21T21:41:42.275 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.275 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.275 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:42.275 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865703+0000", 2024-09-21T21:41:42.275 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6840000000000001e-06 2024-09-21T21:41:42.275 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.275 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.275 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:42.275 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865708+0000", 2024-09-21T21:41:42.275 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7690000000000002e-06 2024-09-21T21:41:42.275 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.275 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.275 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:42.276 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.914670+0000", 2024-09-21T21:41:42.276 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.048961849000000002 2024-09-21T21:41:42.276 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.276 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.276 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:42.276 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.914753+0000", 2024-09-21T21:41:42.276 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2596000000000007e-05 2024-09-21T21:41:42.276 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.276 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.276 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:42.276 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.914858+0000", 2024-09-21T21:41:42.277 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000105163 2024-09-21T21:41:42.277 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.277 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:42.277 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.277 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.277 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.277 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5018.0:2947 3.15 3:af2e3e9a:::rbd_data.11213fc02c9d.0000000000000000:19 [read 499712~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T21:41:42.277 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:33.865714+0000", 2024-09-21T21:41:42.277 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 8.3444404480000003, 2024-09-21T21:41:42.277 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.049362488000000003, 2024-09-21T21:41:42.277 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:42.277 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:42.277 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:42.278 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:42.278 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5018", 2024-09-21T21:41:42.278 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3046875242", 2024-09-21T21:41:42.278 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2947 2024-09-21T21:41:42.278 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.278 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:42.278 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.278 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:42.278 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865714+0000", 2024-09-21T21:41:42.278 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.278 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.278 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.279 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:42.279 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865714+0000", 2024-09-21T21:41:42.279 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.279 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.279 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.279 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:42.279 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865714+0000", 2024-09-21T21:41:42.279 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.27e-07 2024-09-21T21:41:42.279 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.279 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.279 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:42.279 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865716+0000", 2024-09-21T21:41:42.279 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.8789999999999999e-06 2024-09-21T21:41:42.280 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.280 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.280 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:42.280 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865722+0000", 2024-09-21T21:41:42.280 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7930000000000003e-06 2024-09-21T21:41:42.280 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.280 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.280 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:42.280 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865728+0000", 2024-09-21T21:41:42.280 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7960000000000001e-06 2024-09-21T21:41:42.280 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.281 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.281 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:42.281 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.914902+0000", 2024-09-21T21:41:42.281 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.049174021999999998 2024-09-21T21:41:42.281 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.281 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.281 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:42.281 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.914983+0000", 2024-09-21T21:41:42.281 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1446999999999995e-05 2024-09-21T21:41:42.281 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.281 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.282 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:42.282 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.915077+0000", 2024-09-21T21:41:42.282 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3224000000000001e-05 2024-09-21T21:41:42.282 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.282 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:42.282 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.282 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.282 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.282 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5018.0:3004 3.15 3:aa7edf89:::rbd_object_map.1121c78ac9e5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e64)", 2024-09-21T21:41:42.282 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:33.865734+0000", 2024-09-21T21:41:42.282 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 8.3444209059999999, 2024-09-21T21:41:42.282 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.052869125000000003, 2024-09-21T21:41:42.283 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:42.283 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:42.283 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:41:42.283 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:42.283 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5018", 2024-09-21T21:41:42.283 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3046875242", 2024-09-21T21:41:42.283 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 3004 2024-09-21T21:41:42.283 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.283 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:42.283 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.283 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:42.283 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865734+0000", 2024-09-21T21:41:42.283 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.283 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.284 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.284 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:42.284 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865734+0000", 2024-09-21T21:41:42.284 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.284 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.284 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.284 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:42.284 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865734+0000", 2024-09-21T21:41:42.284 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0999999999999999e-07 2024-09-21T21:41:42.284 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.285 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.285 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:42.285 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865737+0000", 2024-09-21T21:41:42.285 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6709999999999999e-06 2024-09-21T21:41:42.285 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.285 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.285 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:42.285 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865743+0000", 2024-09-21T21:41:42.285 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9109999999999998e-06 2024-09-21T21:41:42.285 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.285 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.285 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:42.285 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865747+0000", 2024-09-21T21:41:42.285 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4819999999999996e-06 2024-09-21T21:41:42.286 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.286 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.286 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:42.286 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.915114+0000", 2024-09-21T21:41:42.286 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.049367220000000003 2024-09-21T21:41:42.286 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.286 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.286 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:42.286 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.915203+0000", 2024-09-21T21:41:42.286 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9068999999999997e-05 2024-09-21T21:41:42.286 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.286 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.287 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:41:42.287 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.915603+0000", 2024-09-21T21:41:42.287 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00039933999999999998 2024-09-21T21:41:42.287 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.287 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.287 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:41:42.287 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.918432+0000", 2024-09-21T21:41:42.287 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0028295590000000002 2024-09-21T21:41:42.287 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.287 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.287 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:41:42.287 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.918489+0000", 2024-09-21T21:41:42.287 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6866000000000001e-05 2024-09-21T21:41:42.288 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.288 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.288 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:41:42.288 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.918550+0000", 2024-09-21T21:41:42.288 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0557000000000003e-05 2024-09-21T21:41:42.288 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.288 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.288 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:42.288 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.918603+0000", 2024-09-21T21:41:42.288 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.304e-05 2024-09-21T21:41:42.288 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.288 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:42.289 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.289 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.289 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.289 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5018.0:3005 3.15 3:aa7edf89:::rbd_object_map.1121c78ac9e5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e64)", 2024-09-21T21:41:42.289 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:33.865763+0000", 2024-09-21T21:41:42.289 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 8.3443911820000007, 2024-09-21T21:41:42.289 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.052947052000000001, 2024-09-21T21:41:42.289 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:42.289 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:42.289 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:41:42.289 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:42.289 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5018", 2024-09-21T21:41:42.289 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3046875242", 2024-09-21T21:41:42.290 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 3005 2024-09-21T21:41:42.290 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.290 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:42.290 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.290 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:42.290 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865763+0000", 2024-09-21T21:41:42.290 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.290 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.290 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.290 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:42.290 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865763+0000", 2024-09-21T21:41:42.291 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.291 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.291 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.291 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:42.291 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865764+0000", 2024-09-21T21:41:42.291 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8099999999999998e-07 2024-09-21T21:41:42.291 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.291 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.291 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:42.291 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865766+0000", 2024-09-21T21:41:42.291 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6579999999999998e-06 2024-09-21T21:41:42.291 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.291 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.291 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:42.291 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865774+0000", 2024-09-21T21:41:42.292 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5660000000000001e-06 2024-09-21T21:41:42.292 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.292 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.292 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:42.292 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865779+0000", 2024-09-21T21:41:42.292 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9860000000000002e-06 2024-09-21T21:41:42.292 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.292 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.292 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:42.293 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.916225+0000", 2024-09-21T21:41:42.293 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.050445613 2024-09-21T21:41:42.293 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.293 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.293 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:42.293 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.916277+0000", 2024-09-21T21:41:42.293 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2380000000000003e-05 2024-09-21T21:41:42.293 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.294 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.294 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:41:42.294 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.916482+0000", 2024-09-21T21:41:42.294 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00020543699999999999 2024-09-21T21:41:42.294 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.294 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.294 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:41:42.294 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.918649+0000", 2024-09-21T21:41:42.294 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021668070000000002 2024-09-21T21:41:42.294 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.294 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.294 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:41:42.295 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.918658+0000", 2024-09-21T21:41:42.295 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3960000000000006e-06 2024-09-21T21:41:42.295 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.295 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.295 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:41:42.295 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.918686+0000", 2024-09-21T21:41:42.295 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8464000000000001e-05 2024-09-21T21:41:42.295 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.295 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.295 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:42.295 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.918710+0000", 2024-09-21T21:41:42.296 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4363999999999999e-05 2024-09-21T21:41:42.296 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.296 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:42.296 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.296 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.296 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.296 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5018.0:3006 3.15 3:aa7edf89:::rbd_object_map.1121c78ac9e5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e64)", 2024-09-21T21:41:42.296 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:33.865787+0000", 2024-09-21T21:41:42.296 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 8.3443675329999998, 2024-09-21T21:41:42.296 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.055611471000000003, 2024-09-21T21:41:42.296 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:42.296 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:42.297 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:41:42.297 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:42.297 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5018", 2024-09-21T21:41:42.297 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3046875242", 2024-09-21T21:41:42.297 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 3006 2024-09-21T21:41:42.297 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.297 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:42.297 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.297 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:42.297 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865787+0000", 2024-09-21T21:41:42.297 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.298 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.298 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.298 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:42.298 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865787+0000", 2024-09-21T21:41:42.298 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.298 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.298 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.298 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:42.298 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865787+0000", 2024-09-21T21:41:42.298 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6600000000000002e-07 2024-09-21T21:41:42.298 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.298 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.299 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:42.299 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865790+0000", 2024-09-21T21:41:42.299 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4899999999999999e-06 2024-09-21T21:41:42.299 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.299 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.299 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:42.299 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865795+0000", 2024-09-21T21:41:42.299 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.045e-06 2024-09-21T21:41:42.299 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.299 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.299 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:42.299 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865799+0000", 2024-09-21T21:41:42.299 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7890000000000001e-06 2024-09-21T21:41:42.299 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.300 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.300 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:42.300 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.916851+0000", 2024-09-21T21:41:42.300 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.051052202999999997 2024-09-21T21:41:42.300 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.300 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.300 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:42.300 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.916938+0000", 2024-09-21T21:41:42.300 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7083999999999995e-05 2024-09-21T21:41:42.300 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.300 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.300 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:41:42.300 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.917291+0000", 2024-09-21T21:41:42.300 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00035321400000000002 2024-09-21T21:41:42.301 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.301 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.301 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:41:42.301 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.918717+0000", 2024-09-21T21:41:42.301 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001425548 2024-09-21T21:41:42.301 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.301 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.302 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:41:42.302 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.921291+0000", 2024-09-21T21:41:42.302 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025744460000000002 2024-09-21T21:41:42.302 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.302 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.302 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:41:42.303 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.921359+0000", 2024-09-21T21:41:42.303 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8267000000000002e-05 2024-09-21T21:41:42.303 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.303 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.303 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:42.303 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.921398+0000", 2024-09-21T21:41:42.304 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9019e-05 2024-09-21T21:41:42.304 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.304 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:42.304 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.304 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.304 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.304 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5018.0:3007 3.15 3:aa7edf89:::rbd_object_map.1121c78ac9e5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e64)", 2024-09-21T21:41:42.305 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:33.865803+0000", 2024-09-21T21:41:42.305 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 8.3443515529999992, 2024-09-21T21:41:42.305 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.055713580999999998, 2024-09-21T21:41:42.305 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:42.305 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:42.305 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:41:42.306 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:42.306 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5018", 2024-09-21T21:41:42.306 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3046875242", 2024-09-21T21:41:42.306 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 3007 2024-09-21T21:41:42.306 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.306 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:42.306 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.306 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:42.307 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865803+0000", 2024-09-21T21:41:42.307 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.307 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.307 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.307 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:42.307 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865803+0000", 2024-09-21T21:41:42.307 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.308 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.308 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.308 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:42.308 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865803+0000", 2024-09-21T21:41:42.308 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.22e-07 2024-09-21T21:41:42.308 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.308 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.309 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:42.309 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865806+0000", 2024-09-21T21:41:42.309 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3099999999999999e-06 2024-09-21T21:41:42.309 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.309 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.309 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:42.309 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865811+0000", 2024-09-21T21:41:42.310 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3669999999999996e-06 2024-09-21T21:41:42.310 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.310 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.310 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:42.310 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865815+0000", 2024-09-21T21:41:42.310 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2180000000000001e-06 2024-09-21T21:41:42.310 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.310 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.310 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:42.310 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.917759+0000", 2024-09-21T21:41:42.310 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.051943277000000003 2024-09-21T21:41:42.310 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.311 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.311 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:42.311 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.917843+0000", 2024-09-21T21:41:42.311 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.4202999999999993e-05 2024-09-21T21:41:42.311 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.311 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.311 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:41:42.311 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.918148+0000", 2024-09-21T21:41:42.311 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000305524 2024-09-21T21:41:42.311 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.311 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.312 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:41:42.312 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.920498+0000", 2024-09-21T21:41:42.312 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0023499250000000001 2024-09-21T21:41:42.312 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.312 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.312 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:41:42.312 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.921455+0000", 2024-09-21T21:41:42.312 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00095657900000000004 2024-09-21T21:41:42.312 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.312 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.312 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:41:42.312 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.921484+0000", 2024-09-21T21:41:42.312 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9479999999999999e-05 2024-09-21T21:41:42.312 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.313 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.313 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:42.313 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.921517+0000", 2024-09-21T21:41:42.313 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2376000000000003e-05 2024-09-21T21:41:42.313 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.313 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:42.313 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.313 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.313 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.313 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5018.0:3009 3.15 3:aa7edf89:::rbd_object_map.1121c78ac9e5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-21T21:41:42.313 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:33.865859+0000", 2024-09-21T21:41:42.314 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 8.3442952720000001, 2024-09-21T21:41:42.314 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.055778333999999999, 2024-09-21T21:41:42.314 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:42.314 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:42.314 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:41:42.314 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:42.314 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5018", 2024-09-21T21:41:42.314 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3046875242", 2024-09-21T21:41:42.314 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 3009 2024-09-21T21:41:42.314 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.314 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:42.314 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.314 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:42.314 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865859+0000", 2024-09-21T21:41:42.315 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.315 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.315 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.315 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:42.315 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865859+0000", 2024-09-21T21:41:42.315 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.315 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.315 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.315 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:42.315 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865860+0000", 2024-09-21T21:41:42.315 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5600000000000001e-07 2024-09-21T21:41:42.316 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.316 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.316 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:42.316 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865867+0000", 2024-09-21T21:41:42.316 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6890000000000004e-06 2024-09-21T21:41:42.316 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.316 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.316 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:42.316 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865874+0000", 2024-09-21T21:41:42.316 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1e-06 2024-09-21T21:41:42.317 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.317 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.317 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:42.317 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865878+0000", 2024-09-21T21:41:42.317 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6929999999999998e-06 2024-09-21T21:41:42.317 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.317 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.317 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:42.317 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.918751+0000", 2024-09-21T21:41:42.317 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.052873113999999999 2024-09-21T21:41:42.317 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.317 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.317 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:42.317 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.918830+0000", 2024-09-21T21:41:42.318 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8862999999999994e-05 2024-09-21T21:41:42.318 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.318 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.318 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:41:42.318 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.919165+0000", 2024-09-21T21:41:42.318 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00033459599999999999 2024-09-21T21:41:42.318 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.318 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.318 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:41:42.318 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.921232+0000", 2024-09-21T21:41:42.318 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002067255 2024-09-21T21:41:42.318 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.319 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.319 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:41:42.319 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.921560+0000", 2024-09-21T21:41:42.319 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00032807300000000001 2024-09-21T21:41:42.319 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.319 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.319 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:41:42.319 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.921600+0000", 2024-09-21T21:41:42.319 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9421000000000001e-05 2024-09-21T21:41:42.319 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.319 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.319 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:42.319 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.921638+0000", 2024-09-21T21:41:42.320 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8074000000000002e-05 2024-09-21T21:41:42.320 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.320 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:42.320 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.320 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.320 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.320 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5018.0:3036 3.15 3:aa7edf89:::rbd_object_map.1121c78ac9e5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-21T21:41:42.320 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:33.866205+0000", 2024-09-21T21:41:42.320 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 8.3439495659999992, 2024-09-21T21:41:42.320 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.057529386000000002, 2024-09-21T21:41:42.320 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:42.320 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:42.320 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:41:42.320 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:42.320 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5018", 2024-09-21T21:41:42.321 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3046875242", 2024-09-21T21:41:42.321 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 3036 2024-09-21T21:41:42.321 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.321 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:42.321 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.321 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:42.321 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.866205+0000", 2024-09-21T21:41:42.321 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.321 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.321 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.321 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:42.322 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.866205+0000", 2024-09-21T21:41:42.322 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.322 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.322 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.322 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:42.322 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.866206+0000", 2024-09-21T21:41:42.322 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5499999999999997e-07 2024-09-21T21:41:42.322 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.322 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.322 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:42.322 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.866211+0000", 2024-09-21T21:41:42.323 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8389999999999997e-06 2024-09-21T21:41:42.323 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.323 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.323 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:42.323 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.866223+0000", 2024-09-21T21:41:42.323 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2061e-05 2024-09-21T21:41:42.323 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.323 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.323 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:42.324 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.866230+0000", 2024-09-21T21:41:42.324 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7560000000000002e-06 2024-09-21T21:41:42.324 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.324 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.324 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:42.324 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.919617+0000", 2024-09-21T21:41:42.324 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.053387807000000002 2024-09-21T21:41:42.324 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.324 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.325 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:42.325 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.919700+0000", 2024-09-21T21:41:42.325 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2682999999999994e-05 2024-09-21T21:41:42.325 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.325 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.325 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:41:42.325 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.920067+0000", 2024-09-21T21:41:42.325 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000367408 2024-09-21T21:41:42.325 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.325 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.325 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:41:42.325 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.921529+0000", 2024-09-21T21:41:42.325 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0014611839999999999 2024-09-21T21:41:42.325 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.326 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.326 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:41:42.326 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.923641+0000", 2024-09-21T21:41:42.326 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021122670000000001 2024-09-21T21:41:42.326 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.326 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.326 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:41:42.326 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.923696+0000", 2024-09-21T21:41:42.326 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4756000000000002e-05 2024-09-21T21:41:42.326 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.326 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.326 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:42.326 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.923734+0000", 2024-09-21T21:41:42.326 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8770000000000003e-05 2024-09-21T21:41:42.327 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.327 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:42.327 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.327 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.327 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.327 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5018.0:3125 3.15 3:aa7edf89:::rbd_object_map.1121c78ac9e5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-21T21:41:42.327 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:33.871152+0000", 2024-09-21T21:41:42.327 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 8.3390027900000003, 2024-09-21T21:41:42.327 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.052772614000000002, 2024-09-21T21:41:42.327 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:42.327 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:42.327 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:41:42.327 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:42.327 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5018", 2024-09-21T21:41:42.327 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3046875242", 2024-09-21T21:41:42.328 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 3125 2024-09-21T21:41:42.328 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.328 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:42.328 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.328 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:42.328 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.871152+0000", 2024-09-21T21:41:42.328 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.328 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.328 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.328 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:42.328 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.871152+0000", 2024-09-21T21:41:42.328 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.328 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.328 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.329 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:42.329 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.871153+0000", 2024-09-21T21:41:42.329 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.4e-07 2024-09-21T21:41:42.329 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.329 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.329 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:42.329 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.871157+0000", 2024-09-21T21:41:42.329 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1069999999999998e-06 2024-09-21T21:41:42.329 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.329 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.329 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:42.329 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.912625+0000", 2024-09-21T21:41:42.329 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041468659999999997 2024-09-21T21:41:42.330 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.330 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.330 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:42.330 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.912644+0000", 2024-09-21T21:41:42.330 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.8652e-05 2024-09-21T21:41:42.330 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.330 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.330 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:42.330 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.920531+0000", 2024-09-21T21:41:42.330 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0078870910000000006 2024-09-21T21:41:42.330 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.330 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.330 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:42.331 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.920612+0000", 2024-09-21T21:41:42.331 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0933999999999995e-05 2024-09-21T21:41:42.331 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.331 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.331 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:41:42.331 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.920926+0000", 2024-09-21T21:41:42.331 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00031363700000000002 2024-09-21T21:41:42.331 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.331 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.331 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:41:42.331 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.923748+0000", 2024-09-21T21:41:42.331 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0028222479999999999 2024-09-21T21:41:42.332 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.332 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.332 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:41:42.332 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.923802+0000", 2024-09-21T21:41:42.332 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3912999999999997e-05 2024-09-21T21:41:42.332 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.332 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.332 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:41:42.332 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.923857+0000", 2024-09-21T21:41:42.332 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5312000000000002e-05 2024-09-21T21:41:42.332 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.332 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.333 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:42.333 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.923924+0000", 2024-09-21T21:41:42.333 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7219999999999997e-05 2024-09-21T21:41:42.333 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.333 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:42.333 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.333 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.333 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:42.333 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:42.335 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T21:41:42.335 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:42.333+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:42.436 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:42.434+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:42.479 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:42.479 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:41:42.480 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-21T21:41:42.480 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:42.488 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T21:41:42.537 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:42.536+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:42.629 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:42.629 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:41:42.630 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:41:42.630 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:41:42.630 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:42.638 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T21:41:42.639 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:42.637+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:42.739 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:42.738+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:42.743 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:42.743 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-21T21:41:42.743 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-21T21:41:42.743 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:41:42.743 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.743 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6974 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5246976~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:42.743 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.922932+0000", 2024-09-21T21:41:42.744 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.817956434999999, 2024-09-21T21:41:42.744 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041773352999999999, 2024-09-21T21:41:42.744 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:42.744 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:42.744 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:42.744 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:42.744 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:42.744 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:42.744 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6974 2024-09-21T21:41:42.744 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.744 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:42.744 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.744 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:42.744 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922932+0000", 2024-09-21T21:41:42.745 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.745 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.745 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.745 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:42.745 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922932+0000", 2024-09-21T21:41:42.745 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.745 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.745 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.745 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:42.745 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922933+0000", 2024-09-21T21:41:42.745 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.46e-07 2024-09-21T21:41:42.745 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.745 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.745 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:42.745 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922940+0000", 2024-09-21T21:41:42.746 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6140000000000003e-06 2024-09-21T21:41:42.746 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.746 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.746 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:42.746 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922949+0000", 2024-09-21T21:41:42.746 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6899999999999998e-06 2024-09-21T21:41:42.746 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.746 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.746 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:42.746 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922955+0000", 2024-09-21T21:41:42.746 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5140000000000001e-06 2024-09-21T21:41:42.746 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.746 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.747 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:42.747 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964547+0000", 2024-09-21T21:41:42.747 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041592854999999998 2024-09-21T21:41:42.747 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.747 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.747 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:42.747 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964617+0000", 2024-09-21T21:41:42.747 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9058000000000002e-05 2024-09-21T21:41:42.747 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.747 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.747 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:42.747 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964706+0000", 2024-09-21T21:41:42.747 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9276000000000004e-05 2024-09-21T21:41:42.747 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.747 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:42.748 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.748 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.748 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.748 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6976 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5261312~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:42.748 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.922961+0000", 2024-09-21T21:41:42.748 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.817927389999999, 2024-09-21T21:41:42.748 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041863648000000003, 2024-09-21T21:41:42.748 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:42.748 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:42.748 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:42.748 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:42.748 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:42.748 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:42.748 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6976 2024-09-21T21:41:42.748 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.749 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:42.749 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.749 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:42.749 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922961+0000", 2024-09-21T21:41:42.749 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.749 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.749 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.749 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:42.749 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922961+0000", 2024-09-21T21:41:42.749 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.749 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.749 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.749 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:42.749 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922962+0000", 2024-09-21T21:41:42.750 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.89e-07 2024-09-21T21:41:42.750 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.750 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.750 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:42.750 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922964+0000", 2024-09-21T21:41:42.750 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1270000000000001e-06 2024-09-21T21:41:42.750 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.750 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.750 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:42.750 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922970+0000", 2024-09-21T21:41:42.750 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8030000000000002e-06 2024-09-21T21:41:42.750 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.751 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.751 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:42.751 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922979+0000", 2024-09-21T21:41:42.751 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5559999999999996e-06 2024-09-21T21:41:42.751 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.751 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.751 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:42.751 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964734+0000", 2024-09-21T21:41:42.751 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041754415000000003 2024-09-21T21:41:42.751 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.751 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.751 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:42.751 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964775+0000", 2024-09-21T21:41:42.752 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0806999999999997e-05 2024-09-21T21:41:42.752 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.752 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.752 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:42.752 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964825+0000", 2024-09-21T21:41:42.752 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0550999999999998e-05 2024-09-21T21:41:42.752 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.752 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:42.752 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.752 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.752 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.752 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6978 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5275648~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:42.752 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.922987+0000", 2024-09-21T21:41:42.752 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.817901604999999, 2024-09-21T21:41:42.753 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042043876000000001, 2024-09-21T21:41:42.753 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:42.753 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:42.753 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:42.753 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:42.753 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:42.753 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:42.753 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6978 2024-09-21T21:41:42.753 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.753 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:42.753 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.753 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:42.754 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922987+0000", 2024-09-21T21:41:42.754 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.754 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.754 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.754 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:42.754 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922987+0000", 2024-09-21T21:41:42.754 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.754 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.754 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.754 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:42.754 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922988+0000", 2024-09-21T21:41:42.755 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1199999999999998e-07 2024-09-21T21:41:42.755 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.755 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.755 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:42.755 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922990+0000", 2024-09-21T21:41:42.755 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.142e-06 2024-09-21T21:41:42.755 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.755 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.755 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:42.755 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922997+0000", 2024-09-21T21:41:42.755 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.199e-06 2024-09-21T21:41:42.755 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.755 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.756 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:42.756 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923004+0000", 2024-09-21T21:41:42.756 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3599999999999998e-06 2024-09-21T21:41:42.756 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.756 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.756 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:42.756 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964856+0000", 2024-09-21T21:41:42.756 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041851521000000003 2024-09-21T21:41:42.756 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.756 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.757 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:42.757 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964944+0000", 2024-09-21T21:41:42.757 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7859e-05 2024-09-21T21:41:42.757 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.757 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.757 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:42.757 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965031+0000", 2024-09-21T21:41:42.757 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7383e-05 2024-09-21T21:41:42.757 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.757 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:42.757 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.757 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.758 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.758 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6980 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5289984~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:42.758 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923011+0000", 2024-09-21T21:41:42.758 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.817878267999999, 2024-09-21T21:41:42.758 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042139337999999998, 2024-09-21T21:41:42.758 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:42.758 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:42.758 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:42.758 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:42.758 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:42.758 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:42.758 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6980 2024-09-21T21:41:42.759 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.759 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:42.759 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.759 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:42.759 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923011+0000", 2024-09-21T21:41:42.759 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.759 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.759 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.759 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:42.759 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923011+0000", 2024-09-21T21:41:42.759 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.759 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.760 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.760 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:42.760 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923011+0000", 2024-09-21T21:41:42.760 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6699999999999999e-07 2024-09-21T21:41:42.760 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.760 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.760 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:42.760 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923013+0000", 2024-09-21T21:41:42.760 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0279999999999999e-06 2024-09-21T21:41:42.760 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.760 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.760 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:42.760 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923018+0000", 2024-09-21T21:41:42.761 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4790000000000004e-06 2024-09-21T21:41:42.761 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.761 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.761 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:42.761 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923029+0000", 2024-09-21T21:41:42.761 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0601999999999999e-05 2024-09-21T21:41:42.761 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.761 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.761 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:42.761 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965059+0000", 2024-09-21T21:41:42.761 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042029947999999998 2024-09-21T21:41:42.761 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.761 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.762 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:42.762 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965100+0000", 2024-09-21T21:41:42.762 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1170000000000001e-05 2024-09-21T21:41:42.762 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.762 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.762 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:42.762 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965150+0000", 2024-09-21T21:41:42.762 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9743999999999997e-05 2024-09-21T21:41:42.762 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.762 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:42.762 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.762 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.762 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.762 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6982 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5304320~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:42.763 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923036+0000", 2024-09-21T21:41:42.763 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.817852447, 2024-09-21T21:41:42.763 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042225666000000002, 2024-09-21T21:41:42.763 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:42.763 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:42.763 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:42.763 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:42.763 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:42.763 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:42.763 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6982 2024-09-21T21:41:42.763 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.763 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:42.763 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.763 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:42.764 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923036+0000", 2024-09-21T21:41:42.764 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.764 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.764 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.764 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:42.764 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923036+0000", 2024-09-21T21:41:42.764 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.764 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.764 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.764 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:42.765 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923037+0000", 2024-09-21T21:41:42.765 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2800000000000002e-07 2024-09-21T21:41:42.765 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.765 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.765 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:42.765 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923039+0000", 2024-09-21T21:41:42.765 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1500000000000002e-06 2024-09-21T21:41:42.765 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.765 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.765 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:42.765 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923049+0000", 2024-09-21T21:41:42.765 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0236e-05 2024-09-21T21:41:42.765 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.766 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.766 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:42.766 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923056+0000", 2024-09-21T21:41:42.766 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2540000000000003e-06 2024-09-21T21:41:42.766 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.766 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.766 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:42.766 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965173+0000", 2024-09-21T21:41:42.766 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042116686 2024-09-21T21:41:42.766 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.766 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.766 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:42.766 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965215+0000", 2024-09-21T21:41:42.766 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1696000000000002e-05 2024-09-21T21:41:42.767 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.767 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.767 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:42.767 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965262+0000", 2024-09-21T21:41:42.767 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7216000000000003e-05 2024-09-21T21:41:42.767 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.767 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:42.767 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.767 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.767 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.767 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6984 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5318656~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:42.767 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923063+0000", 2024-09-21T21:41:42.767 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.817825793000001, 2024-09-21T21:41:42.767 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042318651999999998, 2024-09-21T21:41:42.768 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:42.768 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:42.768 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:42.768 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:42.768 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:42.768 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:42.768 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6984 2024-09-21T21:41:42.768 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.768 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:42.768 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.768 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:42.768 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923063+0000", 2024-09-21T21:41:42.768 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.768 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.768 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.769 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:42.769 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923063+0000", 2024-09-21T21:41:42.769 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.769 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.769 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.769 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:42.769 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923064+0000", 2024-09-21T21:41:42.769 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2e-07 2024-09-21T21:41:42.769 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.769 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.769 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:42.769 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923066+0000", 2024-09-21T21:41:42.769 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.057e-06 2024-09-21T21:41:42.769 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.770 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.770 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:42.770 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923071+0000", 2024-09-21T21:41:42.770 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8810000000000001e-06 2024-09-21T21:41:42.770 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.770 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.770 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:42.770 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923078+0000", 2024-09-21T21:41:42.770 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5479999999999998e-06 2024-09-21T21:41:42.770 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.770 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.770 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:42.770 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965288+0000", 2024-09-21T21:41:42.770 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042209576999999998 2024-09-21T21:41:42.771 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.771 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.771 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:42.771 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965332+0000", 2024-09-21T21:41:42.771 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4366999999999997e-05 2024-09-21T21:41:42.771 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.771 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.771 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:42.771 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965382+0000", 2024-09-21T21:41:42.771 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9802000000000002e-05 2024-09-21T21:41:42.772 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.772 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:42.772 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.772 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.772 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.772 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6987 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5332992~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:42.772 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923085+0000", 2024-09-21T21:41:42.772 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.817804268, 2024-09-21T21:41:42.772 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042412103999999999, 2024-09-21T21:41:42.772 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:42.772 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:42.772 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:42.772 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:42.773 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:42.773 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:42.773 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6987 2024-09-21T21:41:42.773 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.773 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:42.773 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.773 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:42.773 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923085+0000", 2024-09-21T21:41:42.773 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.773 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.773 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.773 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:42.774 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923085+0000", 2024-09-21T21:41:42.774 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.774 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.774 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.774 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:42.774 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923085+0000", 2024-09-21T21:41:42.774 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.89e-07 2024-09-21T21:41:42.774 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.774 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.775 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:42.775 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923087+0000", 2024-09-21T21:41:42.775 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0810000000000001e-06 2024-09-21T21:41:42.775 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.775 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.775 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:42.775 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923098+0000", 2024-09-21T21:41:42.775 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1299e-05 2024-09-21T21:41:42.775 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.775 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.775 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:42.775 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923104+0000", 2024-09-21T21:41:42.775 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9100000000000002e-06 2024-09-21T21:41:42.776 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.776 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.776 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:42.776 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965404+0000", 2024-09-21T21:41:42.776 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042299622000000002 2024-09-21T21:41:42.776 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.776 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.776 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:42.776 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965445+0000", 2024-09-21T21:41:42.776 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0701000000000002e-05 2024-09-21T21:41:42.776 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.776 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.776 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:42.776 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965497+0000", 2024-09-21T21:41:42.777 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2101999999999997e-05 2024-09-21T21:41:42.777 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.777 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:42.777 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.777 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.777 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.777 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6989 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5347328~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:42.777 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923111+0000", 2024-09-21T21:41:42.777 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.817777928, 2024-09-21T21:41:42.777 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042505188999999999, 2024-09-21T21:41:42.777 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:42.777 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:42.777 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:42.777 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:42.777 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:42.778 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:42.778 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6989 2024-09-21T21:41:42.778 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.778 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:42.778 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.778 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:42.778 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923111+0000", 2024-09-21T21:41:42.778 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.778 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.778 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.778 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:42.779 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923111+0000", 2024-09-21T21:41:42.779 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.779 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.779 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.779 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:42.779 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923111+0000", 2024-09-21T21:41:42.779 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4499999999999997e-07 2024-09-21T21:41:42.779 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.780 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.780 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:42.780 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923114+0000", 2024-09-21T21:41:42.780 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1569999999999998e-06 2024-09-21T21:41:42.780 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.780 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.780 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:42.780 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923120+0000", 2024-09-21T21:41:42.780 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1759999999999998e-06 2024-09-21T21:41:42.780 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.780 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.780 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:42.780 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923125+0000", 2024-09-21T21:41:42.781 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2560000000000002e-06 2024-09-21T21:41:42.781 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.781 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.781 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:42.781 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965523+0000", 2024-09-21T21:41:42.781 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042398094999999997 2024-09-21T21:41:42.781 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.781 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.781 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:42.781 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965566+0000", 2024-09-21T21:41:42.781 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3393999999999998e-05 2024-09-21T21:41:42.781 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.781 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.781 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:42.782 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965616+0000", 2024-09-21T21:41:42.782 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9666000000000002e-05 2024-09-21T21:41:42.782 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.782 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:42.782 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.782 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.782 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.782 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6991 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5361664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:42.782 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923132+0000", 2024-09-21T21:41:42.782 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.817756639000001, 2024-09-21T21:41:42.782 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042600779999999998, 2024-09-21T21:41:42.782 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:42.782 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:42.783 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:42.783 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:42.783 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:42.783 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:42.783 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6991 2024-09-21T21:41:42.783 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.783 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:42.783 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.783 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:42.783 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923132+0000", 2024-09-21T21:41:42.783 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.783 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.783 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.783 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:42.784 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923132+0000", 2024-09-21T21:41:42.784 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.784 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.784 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.784 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:42.784 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923132+0000", 2024-09-21T21:41:42.784 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4200000000000002e-07 2024-09-21T21:41:42.784 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.784 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.784 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:42.784 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923134+0000", 2024-09-21T21:41:42.784 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2270000000000001e-06 2024-09-21T21:41:42.784 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.784 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.784 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:42.785 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923137+0000", 2024-09-21T21:41:42.785 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1930000000000002e-06 2024-09-21T21:41:42.785 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.785 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.785 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:42.785 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923139+0000", 2024-09-21T21:41:42.785 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4710000000000001e-06 2024-09-21T21:41:42.785 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.785 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.785 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:42.785 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965639+0000", 2024-09-21T21:41:42.785 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.04249965 2024-09-21T21:41:42.786 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.786 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.786 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:42.786 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965679+0000", 2024-09-21T21:41:42.786 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9852999999999999e-05 2024-09-21T21:41:42.786 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.786 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.786 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:42.786 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965733+0000", 2024-09-21T21:41:42.786 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4144000000000002e-05 2024-09-21T21:41:42.786 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.786 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:42.786 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.787 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.787 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.787 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6993 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5384192~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:42.787 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923143+0000", 2024-09-21T21:41:42.787 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.817745844999999, 2024-09-21T21:41:42.787 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042708182999999997, 2024-09-21T21:41:42.787 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:42.787 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:42.787 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:42.787 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:42.787 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:42.787 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:42.787 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6993 2024-09-21T21:41:42.787 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.788 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:42.788 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.788 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:42.788 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923143+0000", 2024-09-21T21:41:42.788 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.788 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.788 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.788 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:42.788 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923143+0000", 2024-09-21T21:41:42.788 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.788 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.788 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.788 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:42.789 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923143+0000", 2024-09-21T21:41:42.789 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0800000000000001e-07 2024-09-21T21:41:42.789 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.789 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.789 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:42.789 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923144+0000", 2024-09-21T21:41:42.789 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.009e-06 2024-09-21T21:41:42.789 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.789 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.789 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:42.789 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923148+0000", 2024-09-21T21:41:42.789 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.737e-06 2024-09-21T21:41:42.789 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.789 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.790 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:42.790 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923150+0000", 2024-09-21T21:41:42.790 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4669999999999998e-06 2024-09-21T21:41:42.790 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.790 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.790 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:42.790 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965760+0000", 2024-09-21T21:41:42.790 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042609399999999999 2024-09-21T21:41:42.790 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.790 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.790 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:42.790 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965801+0000", 2024-09-21T21:41:42.791 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1399999999999997e-05 2024-09-21T21:41:42.791 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.791 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.791 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:42.791 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965851+0000", 2024-09-21T21:41:42.791 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9962e-05 2024-09-21T21:41:42.791 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.791 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:42.791 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.792 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.792 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.792 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6995 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5398528~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:42.792 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923153+0000", 2024-09-21T21:41:42.792 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.817735481, 2024-09-21T21:41:42.792 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042828034000000001, 2024-09-21T21:41:42.792 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:42.792 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:42.792 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:42.792 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:42.792 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:42.792 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:42.792 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6995 2024-09-21T21:41:42.792 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.792 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:42.793 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.793 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:42.793 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923153+0000", 2024-09-21T21:41:42.793 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.793 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.793 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.793 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:42.793 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923153+0000", 2024-09-21T21:41:42.793 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.793 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.793 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.793 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:42.793 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923154+0000", 2024-09-21T21:41:42.794 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2600000000000001e-07 2024-09-21T21:41:42.794 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.794 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.794 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:42.794 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923158+0000", 2024-09-21T21:41:42.794 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0559999999999998e-06 2024-09-21T21:41:42.794 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.794 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.794 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:42.794 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923161+0000", 2024-09-21T21:41:42.794 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6519999999999999e-06 2024-09-21T21:41:42.794 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.795 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.795 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:42.795 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923170+0000", 2024-09-21T21:41:42.795 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5720000000000003e-06 2024-09-21T21:41:42.795 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.795 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.795 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:42.795 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965887+0000", 2024-09-21T21:41:42.795 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042716863000000001 2024-09-21T21:41:42.795 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.795 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.796 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:42.796 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965927+0000", 2024-09-21T21:41:42.796 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0346999999999998e-05 2024-09-21T21:41:42.796 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.796 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.796 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:42.796 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965981+0000", 2024-09-21T21:41:42.796 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4317999999999997e-05 2024-09-21T21:41:42.796 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.796 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:42.796 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.796 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.796 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.796 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6997 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5412864~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:42.797 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923174+0000", 2024-09-21T21:41:42.797 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.817715326, 2024-09-21T21:41:42.797 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042928296999999997, 2024-09-21T21:41:42.797 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:42.797 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:42.797 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:42.797 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:42.797 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:42.797 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:42.797 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6997 2024-09-21T21:41:42.797 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.797 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:42.798 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.798 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:42.798 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923174+0000", 2024-09-21T21:41:42.798 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.798 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.798 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.798 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:42.798 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923174+0000", 2024-09-21T21:41:42.798 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.799 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.799 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.799 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:42.799 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923174+0000", 2024-09-21T21:41:42.799 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.67e-07 2024-09-21T21:41:42.799 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.799 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.799 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:42.800 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923178+0000", 2024-09-21T21:41:42.800 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1330000000000001e-06 2024-09-21T21:41:42.800 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.800 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.800 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:42.800 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923182+0000", 2024-09-21T21:41:42.800 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.794e-06 2024-09-21T21:41:42.800 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.800 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.800 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:42.800 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923184+0000", 2024-09-21T21:41:42.801 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.435e-06 2024-09-21T21:41:42.801 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.801 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.801 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:42.801 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966009+0000", 2024-09-21T21:41:42.801 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042824327000000002 2024-09-21T21:41:42.801 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.801 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.801 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:42.801 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966052+0000", 2024-09-21T21:41:42.801 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3034000000000002e-05 2024-09-21T21:41:42.802 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.802 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.802 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:42.802 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966102+0000", 2024-09-21T21:41:42.802 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0306999999999998e-05 2024-09-21T21:41:42.802 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.802 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:42.802 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.802 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.803 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.803 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6999 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5427200~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:42.803 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923187+0000", 2024-09-21T21:41:42.803 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.817701597999999, 2024-09-21T21:41:42.803 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043104267000000002, 2024-09-21T21:41:42.803 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:42.803 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:42.803 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:42.803 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:42.803 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:42.803 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:42.803 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6999 2024-09-21T21:41:42.803 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.803 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:42.804 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.804 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:42.804 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923187+0000", 2024-09-21T21:41:42.804 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.804 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.804 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.804 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:42.804 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923187+0000", 2024-09-21T21:41:42.804 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.804 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.804 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.805 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:42.805 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923188+0000", 2024-09-21T21:41:42.805 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3999999999999998e-07 2024-09-21T21:41:42.805 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.805 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.805 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:42.805 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923189+0000", 2024-09-21T21:41:42.805 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4169999999999999e-06 2024-09-21T21:41:42.805 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.806 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.806 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:42.806 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923194+0000", 2024-09-21T21:41:42.806 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1819999999999997e-06 2024-09-21T21:41:42.806 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.806 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.806 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:42.806 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923198+0000", 2024-09-21T21:41:42.806 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.143e-06 2024-09-21T21:41:42.807 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.807 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.807 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:42.807 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966135+0000", 2024-09-21T21:41:42.807 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042936297999999998 2024-09-21T21:41:42.807 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.807 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.807 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:42.807 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966204+0000", 2024-09-21T21:41:42.807 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8989e-05 2024-09-21T21:41:42.807 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.807 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.807 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:42.808 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966292+0000", 2024-09-21T21:41:42.808 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7997999999999993e-05 2024-09-21T21:41:42.808 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.808 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:42.808 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.808 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.808 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.808 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7001 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5441536~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:42.808 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923211+0000", 2024-09-21T21:41:42.808 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.817677807000001, 2024-09-21T21:41:42.808 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043196518000000003, 2024-09-21T21:41:42.808 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:42.808 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:42.808 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:42.808 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:42.809 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:42.809 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:42.809 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7001 2024-09-21T21:41:42.809 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.809 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:42.809 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.809 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:42.809 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923211+0000", 2024-09-21T21:41:42.809 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.809 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.809 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.809 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:42.809 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923211+0000", 2024-09-21T21:41:42.809 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.809 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.810 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.810 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:42.810 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923212+0000", 2024-09-21T21:41:42.810 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3900000000000005e-07 2024-09-21T21:41:42.810 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.810 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.810 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:42.810 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923214+0000", 2024-09-21T21:41:42.810 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9970000000000001e-06 2024-09-21T21:41:42.810 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.810 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.810 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:42.810 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923219+0000", 2024-09-21T21:41:42.810 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4310000000000002e-06 2024-09-21T21:41:42.811 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.811 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.811 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:42.811 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923224+0000", 2024-09-21T21:41:42.811 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6410000000000001e-06 2024-09-21T21:41:42.811 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.811 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.811 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:42.811 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966318+0000", 2024-09-21T21:41:42.811 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043093842 2024-09-21T21:41:42.811 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.811 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.811 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:42.812 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966359+0000", 2024-09-21T21:41:42.812 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1260999999999997e-05 2024-09-21T21:41:42.812 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.812 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.812 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:42.812 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966408+0000", 2024-09-21T21:41:42.812 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8807000000000002e-05 2024-09-21T21:41:42.812 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.812 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:42.812 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.812 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.813 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.813 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7002 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5455872~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:42.813 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923229+0000", 2024-09-21T21:41:42.813 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.817659397, 2024-09-21T21:41:42.813 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043364568999999999, 2024-09-21T21:41:42.813 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:42.813 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:42.813 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:42.813 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:42.813 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:42.813 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:42.813 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7002 2024-09-21T21:41:42.813 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.813 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:42.813 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.814 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:42.814 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923229+0000", 2024-09-21T21:41:42.814 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.814 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.814 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.814 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:42.814 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923229+0000", 2024-09-21T21:41:42.814 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.814 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.814 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.814 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:42.814 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923230+0000", 2024-09-21T21:41:42.814 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4700000000000002e-07 2024-09-21T21:41:42.814 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.815 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.815 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:42.815 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923232+0000", 2024-09-21T21:41:42.815 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0719999999999998e-06 2024-09-21T21:41:42.815 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.815 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.815 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:42.815 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923237+0000", 2024-09-21T21:41:42.815 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2290000000000002e-06 2024-09-21T21:41:42.815 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.815 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.815 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:42.815 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923245+0000", 2024-09-21T21:41:42.815 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2959999999999999e-06 2024-09-21T21:41:42.816 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.816 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.816 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:42.816 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966438+0000", 2024-09-21T21:41:42.816 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043193027000000002 2024-09-21T21:41:42.816 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.816 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.816 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:42.816 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966506+0000", 2024-09-21T21:41:42.817 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7707999999999996e-05 2024-09-21T21:41:42.817 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.817 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.817 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:42.817 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966594+0000", 2024-09-21T21:41:42.817 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.789e-05 2024-09-21T21:41:42.817 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.817 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:42.817 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.817 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.817 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.817 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7005 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5470208~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:42.817 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923254+0000", 2024-09-21T21:41:42.817 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.817635166000001, 2024-09-21T21:41:42.817 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043457176, 2024-09-21T21:41:42.818 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:42.818 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:42.818 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:42.818 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:42.818 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:42.818 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:42.818 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7005 2024-09-21T21:41:42.818 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.818 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:42.818 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.818 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:42.818 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923254+0000", 2024-09-21T21:41:42.818 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.819 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.819 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.819 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:42.819 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923254+0000", 2024-09-21T21:41:42.819 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.819 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.819 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.819 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:42.819 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923254+0000", 2024-09-21T21:41:42.819 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4700000000000002e-07 2024-09-21T21:41:42.819 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.820 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.820 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:42.820 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923256+0000", 2024-09-21T21:41:42.820 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.007e-06 2024-09-21T21:41:42.820 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.820 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.820 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:42.820 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923266+0000", 2024-09-21T21:41:42.820 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.4700000000000008e-06 2024-09-21T21:41:42.820 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.820 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.820 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:42.820 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923271+0000", 2024-09-21T21:41:42.820 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2809999999999999e-06 2024-09-21T21:41:42.820 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.821 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.821 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:42.821 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966619+0000", 2024-09-21T21:41:42.821 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043347951000000003 2024-09-21T21:41:42.821 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.821 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.821 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:42.821 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966661+0000", 2024-09-21T21:41:42.821 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2614999999999998e-05 2024-09-21T21:41:42.821 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.821 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.821 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:42.821 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966711+0000", 2024-09-21T21:41:42.822 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9505000000000001e-05 2024-09-21T21:41:42.822 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.822 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:42.822 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.822 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.822 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.822 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7008 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5484544~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:42.822 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923278+0000", 2024-09-21T21:41:42.822 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.817610519, 2024-09-21T21:41:42.822 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043631071, 2024-09-21T21:41:42.822 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:42.822 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:42.822 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:42.822 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:42.823 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:42.823 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:42.823 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7008 2024-09-21T21:41:42.823 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.823 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:42.823 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.823 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:42.823 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923278+0000", 2024-09-21T21:41:42.823 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.823 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.823 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.823 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:42.823 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923278+0000", 2024-09-21T21:41:42.823 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.824 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.824 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.824 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:42.824 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923279+0000", 2024-09-21T21:41:42.824 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.77e-07 2024-09-21T21:41:42.824 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.824 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.824 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:42.824 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923281+0000", 2024-09-21T21:41:42.824 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0219999999999999e-06 2024-09-21T21:41:42.824 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.824 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.824 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:42.824 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923287+0000", 2024-09-21T21:41:42.824 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7799999999999997e-06 2024-09-21T21:41:42.825 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.825 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.825 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:42.825 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923291+0000", 2024-09-21T21:41:42.825 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1699999999999999e-06 2024-09-21T21:41:42.825 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.825 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.825 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:42.825 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966742+0000", 2024-09-21T21:41:42.825 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043451344000000003 2024-09-21T21:41:42.825 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.825 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.825 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:42.825 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966809+0000", 2024-09-21T21:41:42.826 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7070999999999995e-05 2024-09-21T21:41:42.826 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.826 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.826 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:42.826 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966909+0000", 2024-09-21T21:41:42.826 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000100307 2024-09-21T21:41:42.826 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.826 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:42.826 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.826 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.826 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.826 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7010 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5498880~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:42.826 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923301+0000", 2024-09-21T21:41:42.827 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.817587624, 2024-09-21T21:41:42.827 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043717830999999999, 2024-09-21T21:41:42.827 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:42.827 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:42.827 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:42.827 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:42.827 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:42.827 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:42.827 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7010 2024-09-21T21:41:42.828 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.828 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:42.828 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.828 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:42.828 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923301+0000", 2024-09-21T21:41:42.829 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.829 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.829 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.830 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:42.830 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923301+0000", 2024-09-21T21:41:42.830 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.830 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.830 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.830 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:42.830 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923302+0000", 2024-09-21T21:41:42.830 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.41e-07 2024-09-21T21:41:42.830 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.831 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.831 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:42.831 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923304+0000", 2024-09-21T21:41:42.831 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0990000000000001e-06 2024-09-21T21:41:42.831 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.831 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.831 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:42.831 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923310+0000", 2024-09-21T21:41:42.831 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3860000000000004e-06 2024-09-21T21:41:42.831 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.831 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.831 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:42.831 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923315+0000", 2024-09-21T21:41:42.831 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8459999999999999e-06 2024-09-21T21:41:42.832 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.832 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.832 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:42.832 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966938+0000", 2024-09-21T21:41:42.832 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043622693999999997 2024-09-21T21:41:42.832 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.832 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.832 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:42.832 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966975+0000", 2024-09-21T21:41:42.832 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7608999999999999e-05 2024-09-21T21:41:42.832 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.832 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.832 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:42.832 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.967019+0000", 2024-09-21T21:41:42.833 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3856000000000002e-05 2024-09-21T21:41:42.833 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.833 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:42.833 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.833 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.833 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.833 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7354 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 2107392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:42.833 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.936932+0000", 2024-09-21T21:41:42.833 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.803956768999999, 2024-09-21T21:41:42.833 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041607708, 2024-09-21T21:41:42.833 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:42.833 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:42.833 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:42.834 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:42.834 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:42.834 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:42.834 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7354 2024-09-21T21:41:42.834 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.834 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:42.834 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.834 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:42.834 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.936932+0000", 2024-09-21T21:41:42.834 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.834 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.834 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.834 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:42.834 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.936932+0000", 2024-09-21T21:41:42.834 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.835 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.835 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.835 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:42.835 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.937276+0000", 2024-09-21T21:41:42.835 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00034344200000000001 2024-09-21T21:41:42.835 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.835 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.835 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:42.835 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.937279+0000", 2024-09-21T21:41:42.835 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.844e-06 2024-09-21T21:41:42.835 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.835 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.835 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:42.835 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.937288+0000", 2024-09-21T21:41:42.836 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2409999999999995e-06 2024-09-21T21:41:42.836 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.836 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.836 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:42.836 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.937293+0000", 2024-09-21T21:41:42.836 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6559999999999997e-06 2024-09-21T21:41:42.836 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.836 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.836 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:42.836 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.978448+0000", 2024-09-21T21:41:42.836 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041154621000000002 2024-09-21T21:41:42.836 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.836 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.836 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:42.837 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.978487+0000", 2024-09-21T21:41:42.837 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9254000000000001e-05 2024-09-21T21:41:42.837 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.837 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.837 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:42.837 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.978540+0000", 2024-09-21T21:41:42.837 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2649999999999999e-05 2024-09-21T21:41:42.837 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.837 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:42.837 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.837 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.837 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.837 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5215.0:1054 3.9 3:946d2401:::rbd_data.1121524d2522.000000000000000b:head [set-alloc-hint object_size 16777216 write_size 16777216,write 6615040~59392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-21T21:41:42.837 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:38.148979+0000", 2024-09-21T21:41:42.837 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.591909802, 2024-09-21T21:41:42.838 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047870388999999999, 2024-09-21T21:41:42.838 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:42.838 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:42.838 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:41:42.838 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:42.838 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5215", 2024-09-21T21:41:42.838 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1580282168", 2024-09-21T21:41:42.838 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1054 2024-09-21T21:41:42.838 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.838 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:42.838 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.838 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:42.838 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.148979+0000", 2024-09-21T21:41:42.838 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.839 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.839 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.839 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:42.839 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.148979+0000", 2024-09-21T21:41:42.839 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.839 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.839 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.839 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:42.839 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.148980+0000", 2024-09-21T21:41:42.839 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5799999999999999e-07 2024-09-21T21:41:42.839 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.839 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.839 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:42.839 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.149039+0000", 2024-09-21T21:41:42.839 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9151000000000003e-05 2024-09-21T21:41:42.840 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.840 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.840 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:42.840 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.149047+0000", 2024-09-21T21:41:42.840 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1000000000000004e-06 2024-09-21T21:41:42.840 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.840 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.840 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:42.840 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.149052+0000", 2024-09-21T21:41:42.840 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5959999999999997e-06 2024-09-21T21:41:42.840 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.840 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.840 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:42.840 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.195288+0000", 2024-09-21T21:41:42.840 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.046236248000000001 2024-09-21T21:41:42.841 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.841 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.841 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:42.841 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.195334+0000", 2024-09-21T21:41:42.841 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6575e-05 2024-09-21T21:41:42.841 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.841 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.841 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 2", 2024-09-21T21:41:42.841 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.195406+0000", 2024-09-21T21:41:42.841 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1715e-05 2024-09-21T21:41:42.841 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.841 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.841 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:41:42.841 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.196018+0000", 2024-09-21T21:41:42.841 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00061218999999999998 2024-09-21T21:41:42.842 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.842 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.842 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:41:42.842 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.196794+0000", 2024-09-21T21:41:42.842 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000775814 2024-09-21T21:41:42.842 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.842 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.842 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:41:42.842 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.196827+0000", 2024-09-21T21:41:42.842 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2419000000000003e-05 2024-09-21T21:41:42.842 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.842 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.842 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:42.842 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.196849+0000", 2024-09-21T21:41:42.843 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2923000000000001e-05 2024-09-21T21:41:42.843 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.843 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:42.843 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.843 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.843 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:42.843 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:42.844 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T21:41:42.845 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:42.843+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:42.945 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:42.944+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:42.987 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:42.987 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:41:42.987 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.987 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5415.0:380 3.16 3.d1291116 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-21T21:41:42.987 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:42.210777+0000", 2024-09-21T21:41:42.987 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.77538764400000004, 2024-09-21T21:41:42.988 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.77540836599999996, 2024-09-21T21:41:42.988 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:42.988 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:42.988 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "waiting for peered", 2024-09-21T21:41:42.988 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:42.988 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5415", 2024-09-21T21:41:42.988 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1212816607", 2024-09-21T21:41:42.988 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 380 2024-09-21T21:41:42.988 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.988 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:42.988 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.988 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:42.988 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210777+0000", 2024-09-21T21:41:42.988 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.989 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.989 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.989 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:42.989 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210777+0000", 2024-09-21T21:41:42.989 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.989 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.989 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.989 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:42.989 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210778+0000", 2024-09-21T21:41:42.989 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8999999999999996e-07 2024-09-21T21:41:42.989 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.989 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.989 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:42.989 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210780+0000", 2024-09-21T21:41:42.989 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.503e-06 2024-09-21T21:41:42.990 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.990 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.990 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:42.990 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210788+0000", 2024-09-21T21:41:42.990 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0999999999999998e-06 2024-09-21T21:41:42.990 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.990 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.990 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:42.990 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210799+0000", 2024-09-21T21:41:42.990 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1703e-05 2024-09-21T21:41:42.990 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.990 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.990 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:42.991 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210833+0000", 2024-09-21T21:41:42.991 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4008999999999999e-05 2024-09-21T21:41:42.991 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.991 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.991 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:41:42.991 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210838+0000", 2024-09-21T21:41:42.991 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7480000000000001e-06 2024-09-21T21:41:42.991 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.991 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:42.991 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.991 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.991 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.991 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4385.0:15202 3.16 3.d1291116 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-21T21:41:42.991 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:41.736601+0000", 2024-09-21T21:41:42.992 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 1.249563741, 2024-09-21T21:41:42.992 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2496273200000001, 2024-09-21T21:41:42.992 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:42.992 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:42.992 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "waiting for peered", 2024-09-21T21:41:42.992 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:42.992 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4385", 2024-09-21T21:41:42.992 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:41:42.992 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 15202 2024-09-21T21:41:42.992 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.992 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:42.992 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.993 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:42.993 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736601+0000", 2024-09-21T21:41:42.993 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.993 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.993 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.993 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:42.993 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736601+0000", 2024-09-21T21:41:42.993 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.993 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.993 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.993 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:42.993 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736602+0000", 2024-09-21T21:41:42.993 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0699999999999999e-06 2024-09-21T21:41:42.993 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.994 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.994 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:42.994 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736607+0000", 2024-09-21T21:41:42.994 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2520000000000001e-06 2024-09-21T21:41:42.994 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.994 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.994 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:42.994 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736617+0000", 2024-09-21T21:41:42.994 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0074e-05 2024-09-21T21:41:42.994 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.994 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.994 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:42.994 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736622+0000", 2024-09-21T21:41:42.994 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.101e-06 2024-09-21T21:41:42.995 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.995 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.995 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:42.995 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736664+0000", 2024-09-21T21:41:42.995 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1869999999999997e-05 2024-09-21T21:41:42.995 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.995 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.995 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op must wait for map", 2024-09-21T21:41:42.995 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736667+0000", 2024-09-21T21:41:42.995 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4750000000000002e-06 2024-09-21T21:41:42.995 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.995 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.995 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:42.996 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742886+0000", 2024-09-21T21:41:42.996 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0062187579999999996 2024-09-21T21:41:42.996 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.996 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.996 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:41:42.996 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742892+0000", 2024-09-21T21:41:42.996 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0440000000000001e-06 2024-09-21T21:41:42.996 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.996 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.996 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:42.996 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210307+0000", 2024-09-21T21:41:42.996 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.467414781 2024-09-21T21:41:42.996 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.996 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.997 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:41:42.997 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210322+0000", 2024-09-21T21:41:42.997 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5311e-05 2024-09-21T21:41:42.997 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.997 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:42.997 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:42.997 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.997 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.997 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5415.0:379 3.16 3.d1291116 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-21T21:41:42.997 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:41.739544+0000", 2024-09-21T21:41:42.997 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 1.246621312, 2024-09-21T21:41:42.997 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2467301900000001, 2024-09-21T21:41:42.997 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:42.997 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:42.998 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "waiting for peered", 2024-09-21T21:41:42.998 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:42.998 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5415", 2024-09-21T21:41:42.998 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1212816607", 2024-09-21T21:41:42.998 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 379 2024-09-21T21:41:42.998 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.998 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:42.998 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.998 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:42.998 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739544+0000", 2024-09-21T21:41:42.998 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.998 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.998 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.999 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:42.999 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739544+0000", 2024-09-21T21:41:42.999 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:42.999 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.999 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.999 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:42.999 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739545+0000", 2024-09-21T21:41:42.999 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.065e-06 2024-09-21T21:41:42.999 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:42.999 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:42.999 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:42.999 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739550+0000", 2024-09-21T21:41:42.999 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9599999999999999e-06 2024-09-21T21:41:42.999 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.000 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.000 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:43.000 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739565+0000", 2024-09-21T21:41:43.000 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5709000000000002e-05 2024-09-21T21:41:43.000 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.000 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.000 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:43.000 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739573+0000", 2024-09-21T21:41:43.000 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7849999999999992e-06 2024-09-21T21:41:43.000 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.000 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.000 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:43.000 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739634+0000", 2024-09-21T21:41:43.000 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0732000000000001e-05 2024-09-21T21:41:43.000 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.001 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.001 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op must wait for map", 2024-09-21T21:41:43.001 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739640+0000", 2024-09-21T21:41:43.001 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0000000000000002e-06 2024-09-21T21:41:43.001 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.001 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.001 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:43.001 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742958+0000", 2024-09-21T21:41:43.001 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0033182390000000002 2024-09-21T21:41:43.001 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.001 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.001 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:41:43.001 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742966+0000", 2024-09-21T21:41:43.001 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1610000000000003e-06 2024-09-21T21:41:43.001 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.002 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.002 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:43.002 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210405+0000", 2024-09-21T21:41:43.002 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.467438725 2024-09-21T21:41:43.002 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.002 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.002 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:41:43.002 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210415+0000", 2024-09-21T21:41:43.002 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0234000000000001e-05 2024-09-21T21:41:43.002 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:43.002 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:43.002 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:43.002 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.002 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.003 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4385.0:15203 3.16 3.d1291116 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-21T21:41:43.003 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:41.739737+0000", 2024-09-21T21:41:43.003 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 1.2464283009999999, 2024-09-21T21:41:43.003 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2465956519999999, 2024-09-21T21:41:43.003 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:43.003 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:43.003 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "waiting for peered", 2024-09-21T21:41:43.003 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:43.003 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4385", 2024-09-21T21:41:43.003 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:41:43.003 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 15203 2024-09-21T21:41:43.003 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.003 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:43.003 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.003 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:43.004 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739737+0000", 2024-09-21T21:41:43.004 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:43.004 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.004 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.004 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:43.004 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739737+0000", 2024-09-21T21:41:43.004 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:43.004 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.004 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.004 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:43.004 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739738+0000", 2024-09-21T21:41:43.004 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.7499999999999998e-07 2024-09-21T21:41:43.004 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.004 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.004 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:43.005 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739743+0000", 2024-09-21T21:41:43.005 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5049999999999998e-06 2024-09-21T21:41:43.005 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.005 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.005 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:43.005 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739757+0000", 2024-09-21T21:41:43.005 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3947e-05 2024-09-21T21:41:43.005 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.005 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.005 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:43.005 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739765+0000", 2024-09-21T21:41:43.005 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9310000000000008e-06 2024-09-21T21:41:43.005 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.005 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.005 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:43.006 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739816+0000", 2024-09-21T21:41:43.006 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0559999999999997e-05 2024-09-21T21:41:43.006 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.006 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.006 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting_for_map not empty", 2024-09-21T21:41:43.006 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739821+0000", 2024-09-21T21:41:43.006 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3539999999999999e-06 2024-09-21T21:41:43.006 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.006 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.006 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:43.006 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742928+0000", 2024-09-21T21:41:43.006 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0031074399999999999 2024-09-21T21:41:43.006 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.006 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.007 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:41:43.007 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742937+0000", 2024-09-21T21:41:43.007 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1289999999999996e-06 2024-09-21T21:41:43.007 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.007 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.007 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:43.007 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210350+0000", 2024-09-21T21:41:43.007 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.46741311299999999 2024-09-21T21:41:43.007 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.007 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.007 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:41:43.007 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210358+0000", 2024-09-21T21:41:43.007 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1869999999999997e-06 2024-09-21T21:41:43.007 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:43.008 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:43.008 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:43.008 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.008 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.008 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4385.0:15204 3.16 3.d1291116 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-21T21:41:43.008 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:42.210726+0000", 2024-09-21T21:41:43.008 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.77543942300000002, 2024-09-21T21:41:43.008 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.77564745999999996, 2024-09-21T21:41:43.008 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:43.008 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:43.008 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "waiting for peered", 2024-09-21T21:41:43.008 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:43.008 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4385", 2024-09-21T21:41:43.008 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:41:43.008 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 15204 2024-09-21T21:41:43.009 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.009 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:43.009 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.009 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:43.009 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210726+0000", 2024-09-21T21:41:43.009 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:43.009 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.009 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.009 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:43.009 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210726+0000", 2024-09-21T21:41:43.009 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:43.009 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.009 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.009 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:43.009 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210727+0000", 2024-09-21T21:41:43.010 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.204e-06 2024-09-21T21:41:43.010 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.010 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.010 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:43.010 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210732+0000", 2024-09-21T21:41:43.010 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5249999999999996e-06 2024-09-21T21:41:43.010 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.010 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.010 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:43.010 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210744+0000", 2024-09-21T21:41:43.010 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1405e-05 2024-09-21T21:41:43.010 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.010 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.011 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:43.011 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210754+0000", 2024-09-21T21:41:43.011 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0122999999999999e-05 2024-09-21T21:41:43.011 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.011 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.011 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:43.011 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210798+0000", 2024-09-21T21:41:43.011 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4005000000000003e-05 2024-09-21T21:41:43.011 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.011 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.011 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:41:43.011 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210803+0000", 2024-09-21T21:41:43.011 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5130000000000004e-06 2024-09-21T21:41:43.011 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:43.011 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:43.012 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:43.012 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:43.012 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-21T21:41:43.012 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 5 2024-09-21T21:41:43.012 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:43.013 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T21:41:43.056 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:43.055+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:43.115 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:43.115 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:41:43.115 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:41:43.116 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:41:43.116 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:43.124 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T21:41:43.166 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:43.164+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:43.248 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:43.246+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:43.291 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:43.291 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-21T21:41:43.291 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-21T21:41:43.291 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:41:43.292 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.292 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1484 3.16 3:6a41a830:::rbd_data.112145fae779.000000000000000a:a [read 303104~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:43.292 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634086+0000", 2024-09-21T21:41:43.292 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 41.654597994, 2024-09-21T21:41:43.292 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.058257055000000002, 2024-09-21T21:41:43.292 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:43.292 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:43.292 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:43.292 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:43.292 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:43.292 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:43.292 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1484 2024-09-21T21:41:43.292 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.292 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:43.293 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.293 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:43.293 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634086+0000", 2024-09-21T21:41:43.293 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:43.293 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.293 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.293 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:43.293 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634086+0000", 2024-09-21T21:41:43.293 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:43.293 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.293 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.293 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:43.293 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634086+0000", 2024-09-21T21:41:43.294 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6100000000000001e-07 2024-09-21T21:41:43.294 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.294 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.294 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:43.294 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634088+0000", 2024-09-21T21:41:43.294 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1100000000000001e-06 2024-09-21T21:41:43.294 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.294 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.294 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:43.294 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634095+0000", 2024-09-21T21:41:43.294 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4080000000000003e-06 2024-09-21T21:41:43.294 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.294 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.294 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:43.295 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634100+0000", 2024-09-21T21:41:43.295 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0730000000000004e-06 2024-09-21T21:41:43.295 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.295 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.295 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:43.295 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.692197+0000", 2024-09-21T21:41:43.295 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.058097331000000002 2024-09-21T21:41:43.295 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.295 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.296 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:43.296 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.692271+0000", 2024-09-21T21:41:43.296 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3423999999999994e-05 2024-09-21T21:41:43.296 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.296 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.296 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:43.296 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.692343+0000", 2024-09-21T21:41:43.296 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2248000000000004e-05 2024-09-21T21:41:43.296 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:43.296 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:43.296 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:43.296 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.296 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.296 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1485 3.16 3:695c582e:::rbd_data.1121e540c096.0000000000000008:b [read 335872~36864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:43.296 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634106+0000", 2024-09-21T21:41:43.297 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 41.654578196999999, 2024-09-21T21:41:43.297 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059223685999999998, 2024-09-21T21:41:43.297 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:43.297 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:43.297 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:43.297 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:43.297 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:43.297 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:43.297 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1485 2024-09-21T21:41:43.297 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.297 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:43.297 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.297 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:43.298 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634106+0000", 2024-09-21T21:41:43.298 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:43.298 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.298 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.298 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:43.298 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634106+0000", 2024-09-21T21:41:43.298 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:43.298 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.298 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.298 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:43.298 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634106+0000", 2024-09-21T21:41:43.299 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3300000000000002e-07 2024-09-21T21:41:43.299 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.299 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.299 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:43.299 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634108+0000", 2024-09-21T21:41:43.299 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.996e-06 2024-09-21T21:41:43.299 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.299 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.299 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:43.299 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634115+0000", 2024-09-21T21:41:43.299 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5139999999999998e-06 2024-09-21T21:41:43.299 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.299 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.299 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:43.300 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634120+0000", 2024-09-21T21:41:43.300 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1050000000000003e-06 2024-09-21T21:41:43.300 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.300 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.300 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:43.300 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.692374+0000", 2024-09-21T21:41:43.300 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.058254729999999998 2024-09-21T21:41:43.300 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.300 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.300 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:43.300 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.692440+0000", 2024-09-21T21:41:43.300 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6076999999999998e-05 2024-09-21T21:41:43.300 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.301 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.301 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:43.301 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693329+0000", 2024-09-21T21:41:43.301 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00088873100000000003 2024-09-21T21:41:43.301 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:43.301 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:43.301 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:43.301 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.301 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.301 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1486 3.16 3:6a41a830:::rbd_data.112145fae779.000000000000000a:a [read 360448~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:43.301 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634126+0000", 2024-09-21T21:41:43.301 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 41.654557904999997, 2024-09-21T21:41:43.301 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059399528, 2024-09-21T21:41:43.301 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:43.301 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:43.302 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:43.302 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:43.302 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:43.302 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:43.302 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1486 2024-09-21T21:41:43.302 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.302 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:43.302 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.302 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:43.302 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634126+0000", 2024-09-21T21:41:43.302 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:43.302 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.302 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.302 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:43.303 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634126+0000", 2024-09-21T21:41:43.303 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:43.303 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.303 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.303 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:43.303 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634126+0000", 2024-09-21T21:41:43.303 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4099999999999999e-07 2024-09-21T21:41:43.303 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.303 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.303 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:43.303 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634128+0000", 2024-09-21T21:41:43.303 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1069999999999999e-06 2024-09-21T21:41:43.303 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.303 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.303 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:43.304 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634135+0000", 2024-09-21T21:41:43.304 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.212e-06 2024-09-21T21:41:43.304 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.304 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.304 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:43.304 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634140+0000", 2024-09-21T21:41:43.304 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1429999999999997e-06 2024-09-21T21:41:43.304 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.304 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.304 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:43.304 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693363+0000", 2024-09-21T21:41:43.304 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059223614000000001 2024-09-21T21:41:43.305 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.305 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.305 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:43.305 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693431+0000", 2024-09-21T21:41:43.305 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7294999999999999e-05 2024-09-21T21:41:43.305 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.305 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.305 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:43.305 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693525+0000", 2024-09-21T21:41:43.305 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.4716000000000006e-05 2024-09-21T21:41:43.305 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:43.305 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:43.306 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:43.306 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.306 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.306 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1499 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 49152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:43.306 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634186+0000", 2024-09-21T21:41:43.306 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 41.654498027999999, 2024-09-21T21:41:43.306 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059339256999999999, 2024-09-21T21:41:43.306 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:43.306 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:43.306 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:43.306 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:43.306 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:43.306 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:43.307 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1499 2024-09-21T21:41:43.307 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.307 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:43.307 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.307 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:43.307 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634186+0000", 2024-09-21T21:41:43.307 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:43.307 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.307 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.307 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:43.307 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634186+0000", 2024-09-21T21:41:43.308 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:43.308 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.308 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.308 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:43.308 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634186+0000", 2024-09-21T21:41:43.308 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0999999999999999e-07 2024-09-21T21:41:43.308 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.308 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.308 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:43.308 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634188+0000", 2024-09-21T21:41:43.308 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.86e-06 2024-09-21T21:41:43.308 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.308 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.309 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:43.309 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634194+0000", 2024-09-21T21:41:43.309 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1109999999999996e-06 2024-09-21T21:41:43.309 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.309 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.309 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:43.309 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634199+0000", 2024-09-21T21:41:43.309 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7419999999999997e-06 2024-09-21T21:41:43.309 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.309 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.309 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:43.309 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693357+0000", 2024-09-21T21:41:43.309 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059157847 2024-09-21T21:41:43.309 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.310 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.310 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:43.310 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693430+0000", 2024-09-21T21:41:43.310 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3203999999999999e-05 2024-09-21T21:41:43.310 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.310 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.310 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:43.310 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693525+0000", 2024-09-21T21:41:43.310 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5082999999999997e-05 2024-09-21T21:41:43.311 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:43.311 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:43.311 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:43.311 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.311 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.311 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1501 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 106496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:43.311 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634205+0000", 2024-09-21T21:41:43.311 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 41.654478937999997, 2024-09-21T21:41:43.311 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059482394000000001, 2024-09-21T21:41:43.311 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:43.311 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:43.311 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:43.311 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:43.312 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:43.312 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:43.312 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1501 2024-09-21T21:41:43.312 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.312 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:43.312 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.312 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:43.312 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634205+0000", 2024-09-21T21:41:43.312 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:43.312 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.312 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.312 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:43.312 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634205+0000", 2024-09-21T21:41:43.313 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:43.313 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.313 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.313 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:43.313 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634205+0000", 2024-09-21T21:41:43.313 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.03e-07 2024-09-21T21:41:43.313 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.313 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.313 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:43.313 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634207+0000", 2024-09-21T21:41:43.313 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0389999999999999e-06 2024-09-21T21:41:43.313 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.313 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.314 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:43.314 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634213+0000", 2024-09-21T21:41:43.314 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2600000000000002e-06 2024-09-21T21:41:43.314 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.314 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.314 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:43.314 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634218+0000", 2024-09-21T21:41:43.314 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8570000000000003e-06 2024-09-21T21:41:43.314 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.314 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.314 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:43.314 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693552+0000", 2024-09-21T21:41:43.314 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059333636000000002 2024-09-21T21:41:43.315 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.315 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.315 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:43.315 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693613+0000", 2024-09-21T21:41:43.315 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0560000000000003e-05 2024-09-21T21:41:43.315 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.315 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.315 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:43.315 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693687+0000", 2024-09-21T21:41:43.315 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4639000000000001e-05 2024-09-21T21:41:43.315 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:43.315 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:43.315 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:43.315 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.316 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.316 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1515 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 434176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:43.316 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634318+0000", 2024-09-21T21:41:43.316 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 41.654365251000002, 2024-09-21T21:41:43.316 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059565812000000003, 2024-09-21T21:41:43.316 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:43.316 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:43.316 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:43.316 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:43.316 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:43.316 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:43.316 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1515 2024-09-21T21:41:43.316 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.316 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:43.317 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.317 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:43.317 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634318+0000", 2024-09-21T21:41:43.317 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:43.317 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.317 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.317 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:43.317 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634318+0000", 2024-09-21T21:41:43.317 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:43.317 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.317 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.317 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:43.317 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634319+0000", 2024-09-21T21:41:43.317 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9400000000000001e-07 2024-09-21T21:41:43.318 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.318 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.318 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:43.318 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634321+0000", 2024-09-21T21:41:43.318 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.154e-06 2024-09-21T21:41:43.318 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.318 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.320 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:43.321 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634327+0000", 2024-09-21T21:41:43.321 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9830000000000001e-06 2024-09-21T21:41:43.321 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.321 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.321 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:43.321 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634332+0000", 2024-09-21T21:41:43.321 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9080000000000003e-06 2024-09-21T21:41:43.321 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.321 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.321 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:43.321 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693717+0000", 2024-09-21T21:41:43.321 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059385447000000001 2024-09-21T21:41:43.321 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.321 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.322 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:43.322 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693795+0000", 2024-09-21T21:41:43.322 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7267000000000004e-05 2024-09-21T21:41:43.322 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.322 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.322 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:43.322 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693884+0000", 2024-09-21T21:41:43.322 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9659000000000003e-05 2024-09-21T21:41:43.322 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:43.322 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:43.322 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:43.322 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.322 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.323 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1517 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 491520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:43.323 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634338+0000", 2024-09-21T21:41:43.323 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 41.654345478000003, 2024-09-21T21:41:43.323 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060100745999999997, 2024-09-21T21:41:43.323 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:43.323 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:43.323 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:43.323 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:43.323 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:43.323 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:43.323 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1517 2024-09-21T21:41:43.323 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.323 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:43.324 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.324 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:43.324 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634338+0000", 2024-09-21T21:41:43.324 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:43.324 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.324 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.324 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:43.324 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634338+0000", 2024-09-21T21:41:43.324 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:43.324 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.324 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.324 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:43.324 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634339+0000", 2024-09-21T21:41:43.324 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2899999999999999e-07 2024-09-21T21:41:43.324 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.325 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.325 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:43.325 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634341+0000", 2024-09-21T21:41:43.325 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.8959999999999999e-06 2024-09-21T21:41:43.325 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.325 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.325 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:43.325 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634347+0000", 2024-09-21T21:41:43.325 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.302e-06 2024-09-21T21:41:43.325 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.325 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.325 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:43.325 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634351+0000", 2024-09-21T21:41:43.325 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6070000000000001e-06 2024-09-21T21:41:43.326 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.326 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.326 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:43.326 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694279+0000", 2024-09-21T21:41:43.326 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059927046999999997 2024-09-21T21:41:43.326 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.326 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.326 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:43.326 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694355+0000", 2024-09-21T21:41:43.326 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6483999999999995e-05 2024-09-21T21:41:43.326 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.326 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.326 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:43.327 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694439+0000", 2024-09-21T21:41:43.327 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3980999999999994e-05 2024-09-21T21:41:43.327 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:43.327 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:43.327 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:43.327 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.327 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.327 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1518 3.16 3:6a41a830:::rbd_data.112145fae779.000000000000000a:a [read 294912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:43.327 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634357+0000", 2024-09-21T21:41:43.327 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 41.654326572000002, 2024-09-21T21:41:43.327 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059712318, 2024-09-21T21:41:43.327 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:43.327 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:43.328 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:43.328 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:43.328 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:43.328 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:43.328 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1518 2024-09-21T21:41:43.328 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.328 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:43.328 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.328 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:43.328 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634357+0000", 2024-09-21T21:41:43.328 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:43.328 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.328 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.328 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:43.328 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634357+0000", 2024-09-21T21:41:43.329 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:43.329 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.329 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.329 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:43.329 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634358+0000", 2024-09-21T21:41:43.329 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2100000000000002e-07 2024-09-21T21:41:43.329 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.329 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.329 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:43.329 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634360+0000", 2024-09-21T21:41:43.329 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0940000000000002e-06 2024-09-21T21:41:43.329 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.329 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.329 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:43.329 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634366+0000", 2024-09-21T21:41:43.330 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3389999999999998e-06 2024-09-21T21:41:43.330 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.330 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.330 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:43.330 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634371+0000", 2024-09-21T21:41:43.330 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6670000000000004e-06 2024-09-21T21:41:43.330 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.330 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.330 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:43.330 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693910+0000", 2024-09-21T21:41:43.330 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059538973000000002 2024-09-21T21:41:43.330 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.330 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.330 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:43.331 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693982+0000", 2024-09-21T21:41:43.331 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2806999999999996e-05 2024-09-21T21:41:43.331 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.331 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.331 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:43.331 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694069+0000", 2024-09-21T21:41:43.331 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7016999999999997e-05 2024-09-21T21:41:43.331 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:43.331 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:43.331 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:43.331 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.332 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.332 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1519 3.16 3:6a41a830:::rbd_data.112145fae779.000000000000000a:a [read 352256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:43.332 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634377+0000", 2024-09-21T21:41:43.332 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 41.654306869999999, 2024-09-21T21:41:43.332 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059872213000000001, 2024-09-21T21:41:43.332 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:43.332 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:43.332 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:43.332 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:43.332 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:43.332 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:43.332 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1519 2024-09-21T21:41:43.332 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.332 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:43.333 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.333 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:43.333 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634377+0000", 2024-09-21T21:41:43.333 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:43.333 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.333 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.333 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:43.333 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634377+0000", 2024-09-21T21:41:43.333 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:43.333 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.333 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.333 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:43.333 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634377+0000", 2024-09-21T21:41:43.333 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2600000000000002e-07 2024-09-21T21:41:43.333 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.334 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.334 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:43.334 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634379+0000", 2024-09-21T21:41:43.334 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0729999999999999e-06 2024-09-21T21:41:43.334 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.334 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.334 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:43.334 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634386+0000", 2024-09-21T21:41:43.334 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4690000000000002e-06 2024-09-21T21:41:43.334 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.334 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.334 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:43.335 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634390+0000", 2024-09-21T21:41:43.335 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4580000000000003e-06 2024-09-21T21:41:43.335 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.335 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.335 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:43.335 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694094+0000", 2024-09-21T21:41:43.335 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059703830999999999 2024-09-21T21:41:43.335 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.335 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.336 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:43.336 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694168+0000", 2024-09-21T21:41:43.336 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4222999999999997e-05 2024-09-21T21:41:43.336 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.336 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.336 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:43.336 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694249+0000", 2024-09-21T21:41:43.336 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0632999999999999e-05 2024-09-21T21:41:43.336 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:43.336 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:43.336 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:43.336 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.336 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.336 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1529 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 53248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:43.336 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634397+0000", 2024-09-21T21:41:43.337 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 41.654286718999998, 2024-09-21T21:41:43.337 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060189919000000001, 2024-09-21T21:41:43.337 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:43.337 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:43.337 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:43.337 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:43.337 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:43.337 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:43.337 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1529 2024-09-21T21:41:43.337 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.337 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:43.337 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.337 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:43.337 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634397+0000", 2024-09-21T21:41:43.337 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:43.338 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.338 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.338 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:43.338 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634397+0000", 2024-09-21T21:41:43.338 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:43.338 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.338 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.338 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:43.338 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634397+0000", 2024-09-21T21:41:43.338 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2e-07 2024-09-21T21:41:43.338 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.338 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.339 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:43.339 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634399+0000", 2024-09-21T21:41:43.339 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0609999999999998e-06 2024-09-21T21:41:43.339 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.339 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.339 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:43.339 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634406+0000", 2024-09-21T21:41:43.339 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7259999999999996e-06 2024-09-21T21:41:43.339 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.339 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.339 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:43.339 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634411+0000", 2024-09-21T21:41:43.339 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9339999999999997e-06 2024-09-21T21:41:43.339 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.339 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.340 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:43.340 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694480+0000", 2024-09-21T21:41:43.340 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060069105999999997 2024-09-21T21:41:43.340 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.340 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.340 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:43.340 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694527+0000", 2024-09-21T21:41:43.340 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7120000000000003e-05 2024-09-21T21:41:43.340 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.340 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.340 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:43.340 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694587+0000", 2024-09-21T21:41:43.340 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9552000000000002e-05 2024-09-21T21:41:43.340 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:43.340 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:43.341 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:43.341 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.341 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.341 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1531 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 110592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:43.341 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634417+0000", 2024-09-21T21:41:43.341 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 41.654266784999997, 2024-09-21T21:41:43.341 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060347287999999999, 2024-09-21T21:41:43.341 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:43.341 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:43.341 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:43.341 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:43.341 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:43.341 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:43.341 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1531 2024-09-21T21:41:43.341 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.342 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:43.342 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.342 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:43.342 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634417+0000", 2024-09-21T21:41:43.342 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:43.342 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.342 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.342 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:43.342 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634417+0000", 2024-09-21T21:41:43.342 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:43.342 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.342 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.342 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:43.342 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634417+0000", 2024-09-21T21:41:43.343 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1699999999999999e-07 2024-09-21T21:41:43.343 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.343 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.343 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:43.343 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634420+0000", 2024-09-21T21:41:43.343 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.243e-06 2024-09-21T21:41:43.343 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.343 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.343 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:43.343 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634426+0000", 2024-09-21T21:41:43.343 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2550000000000003e-06 2024-09-21T21:41:43.343 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.343 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.343 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:43.344 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634431+0000", 2024-09-21T21:41:43.344 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0150000000000003e-06 2024-09-21T21:41:43.344 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.344 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.344 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:43.344 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694618+0000", 2024-09-21T21:41:43.344 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060186836000000001 2024-09-21T21:41:43.344 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.344 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.344 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:43.344 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694685+0000", 2024-09-21T21:41:43.344 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6970000000000004e-05 2024-09-21T21:41:43.344 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.344 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.344 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:43.345 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694764+0000", 2024-09-21T21:41:43.345 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9552e-05 2024-09-21T21:41:43.345 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:43.345 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:43.345 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:43.345 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.345 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.345 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1532 3.e 3:724b31a3:::rbd_data.1121e540c096.0000000000000015:b [read 151552~36864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:43.345 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634437+0000", 2024-09-21T21:41:43.345 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 41.654246989000001, 2024-09-21T21:41:43.345 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060474314000000001, 2024-09-21T21:41:43.346 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:43.346 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:43.346 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:43.346 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:43.346 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:43.346 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:43.346 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1532 2024-09-21T21:41:43.346 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.346 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:43.346 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.346 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:43.346 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634437+0000", 2024-09-21T21:41:43.346 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:43.346 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.347 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.347 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:43.347 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634437+0000", 2024-09-21T21:41:43.347 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:43.347 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.347 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.347 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:43.347 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634437+0000", 2024-09-21T21:41:43.347 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9400000000000001e-07 2024-09-21T21:41:43.347 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.347 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.347 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:43.347 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634439+0000", 2024-09-21T21:41:43.347 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.114e-06 2024-09-21T21:41:43.348 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.348 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.348 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:43.348 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634446+0000", 2024-09-21T21:41:43.348 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1389999999999998e-06 2024-09-21T21:41:43.348 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.348 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.348 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:43.348 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634451+0000", 2024-09-21T21:41:43.348 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0010000000000001e-06 2024-09-21T21:41:43.348 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.348 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.348 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:43.348 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694794+0000", 2024-09-21T21:41:43.349 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060342509000000003 2024-09-21T21:41:43.349 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.349 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.349 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:43.349 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694835+0000", 2024-09-21T21:41:43.349 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0728e-05 2024-09-21T21:41:43.349 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.349 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.349 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:43.349 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694911+0000", 2024-09-21T21:41:43.349 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6428999999999996e-05 2024-09-21T21:41:43.349 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:43.349 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:43.349 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:43.349 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.350 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.350 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1534 3.e 3:724b31a3:::rbd_data.1121e540c096.0000000000000015:b [read 487424~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:43.350 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634477+0000", 2024-09-21T21:41:43.350 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 41.654206586000001, 2024-09-21T21:41:43.350 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060624009, 2024-09-21T21:41:43.350 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:43.350 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:43.350 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:43.350 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:43.350 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:43.350 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:43.350 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1534 2024-09-21T21:41:43.350 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.350 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:43.351 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.351 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:43.351 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634477+0000", 2024-09-21T21:41:43.351 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:43.351 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.351 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.351 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:43.351 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634477+0000", 2024-09-21T21:41:43.351 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:43.351 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.351 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.351 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:43.351 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634478+0000", 2024-09-21T21:41:43.351 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1399999999999997e-07 2024-09-21T21:41:43.352 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.352 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.352 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:43.352 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634480+0000", 2024-09-21T21:41:43.352 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1009999999999999e-06 2024-09-21T21:41:43.352 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.352 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.352 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:43.352 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634486+0000", 2024-09-21T21:41:43.352 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3289999999999999e-06 2024-09-21T21:41:43.352 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.352 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.352 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:43.352 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634491+0000", 2024-09-21T21:41:43.353 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6260000000000003e-06 2024-09-21T21:41:43.353 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.353 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.353 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:43.353 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694943+0000", 2024-09-21T21:41:43.353 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060451960999999999 2024-09-21T21:41:43.353 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.353 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.353 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:43.353 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695014+0000", 2024-09-21T21:41:43.353 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1587000000000004e-05 2024-09-21T21:41:43.353 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.353 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.353 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:43.353 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695101+0000", 2024-09-21T21:41:43.353 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6990999999999994e-05 2024-09-21T21:41:43.354 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:43.354 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:43.354 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:43.354 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.354 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.354 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1545 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 438272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:43.354 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634515+0000", 2024-09-21T21:41:43.354 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 41.654168386000002, 2024-09-21T21:41:43.354 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060707944, 2024-09-21T21:41:43.354 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:43.354 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:43.354 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:43.354 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:43.354 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:43.355 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:43.355 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1545 2024-09-21T21:41:43.355 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.355 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:43.355 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.355 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:43.355 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634515+0000", 2024-09-21T21:41:43.355 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:43.355 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.355 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.355 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:43.355 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634515+0000", 2024-09-21T21:41:43.355 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:43.355 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.355 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.356 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:43.356 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634516+0000", 2024-09-21T21:41:43.356 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0999999999999999e-07 2024-09-21T21:41:43.356 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.356 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.356 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:43.356 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634518+0000", 2024-09-21T21:41:43.356 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.846e-06 2024-09-21T21:41:43.356 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.356 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.357 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:43.357 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634524+0000", 2024-09-21T21:41:43.357 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2550000000000003e-06 2024-09-21T21:41:43.357 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.357 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.357 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:43.357 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634529+0000", 2024-09-21T21:41:43.357 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6940000000000003e-06 2024-09-21T21:41:43.357 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.357 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.357 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:43.357 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695127+0000", 2024-09-21T21:41:43.357 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060598921 2024-09-21T21:41:43.357 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.358 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.358 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:43.358 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695171+0000", 2024-09-21T21:41:43.358 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3588999999999999e-05 2024-09-21T21:41:43.358 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.358 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.358 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:43.358 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695223+0000", 2024-09-21T21:41:43.358 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2228999999999997e-05 2024-09-21T21:41:43.358 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:43.358 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:43.358 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:43.358 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.358 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.359 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1547 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 495616~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:43.359 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634534+0000", 2024-09-21T21:41:43.359 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 41.654149595, 2024-09-21T21:41:43.359 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060881131999999998, 2024-09-21T21:41:43.359 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:43.359 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:43.359 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:43.359 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:43.359 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:43.359 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:43.359 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1547 2024-09-21T21:41:43.359 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.359 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:43.359 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.360 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:43.360 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634534+0000", 2024-09-21T21:41:43.360 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:43.360 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.360 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.360 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:43.360 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634534+0000", 2024-09-21T21:41:43.360 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:43.360 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.360 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.360 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:43.360 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634535+0000", 2024-09-21T21:41:43.360 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.77e-07 2024-09-21T21:41:43.360 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.361 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.361 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:43.361 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634542+0000", 2024-09-21T21:41:43.361 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9000000000000006e-06 2024-09-21T21:41:43.361 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.361 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.361 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:43.361 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634549+0000", 2024-09-21T21:41:43.361 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5910000000000001e-06 2024-09-21T21:41:43.361 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.361 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.361 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:43.361 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634554+0000", 2024-09-21T21:41:43.361 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7940000000000002e-06 2024-09-21T21:41:43.361 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.362 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.362 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:43.362 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695256+0000", 2024-09-21T21:41:43.362 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.06070246 2024-09-21T21:41:43.362 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.362 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.362 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:43.362 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695332+0000", 2024-09-21T21:41:43.362 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6248000000000006e-05 2024-09-21T21:41:43.362 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.362 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.362 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:43.362 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695415+0000", 2024-09-21T21:41:43.362 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2762000000000005e-05 2024-09-21T21:41:43.363 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:43.363 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:43.363 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:43.363 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.363 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.363 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1548 3.16 3:6a41a830:::rbd_data.112145fae779.000000000000000a:a [read 299008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:43.363 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634561+0000", 2024-09-21T21:41:43.363 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 41.654123018, 2024-09-21T21:41:43.363 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060786743999999997, 2024-09-21T21:41:43.363 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:43.363 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:43.363 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:43.363 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:43.363 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:43.363 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:43.364 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1548 2024-09-21T21:41:43.364 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.364 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:43.364 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.364 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:43.364 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634561+0000", 2024-09-21T21:41:43.364 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:43.364 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.364 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.364 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:43.364 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634561+0000", 2024-09-21T21:41:43.364 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:43.364 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.364 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.364 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:43.365 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634561+0000", 2024-09-21T21:41:43.365 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5199999999999997e-07 2024-09-21T21:41:43.365 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.365 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.365 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:43.365 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634564+0000", 2024-09-21T21:41:43.365 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4040000000000002e-06 2024-09-21T21:41:43.365 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.365 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.365 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:43.365 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634571+0000", 2024-09-21T21:41:43.365 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4379999999999998e-06 2024-09-21T21:41:43.365 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.365 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.366 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:43.366 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634576+0000", 2024-09-21T21:41:43.366 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1399999999999999e-06 2024-09-21T21:41:43.366 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.366 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.366 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:43.366 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695247+0000", 2024-09-21T21:41:43.366 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060670397000000001 2024-09-21T21:41:43.366 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.366 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.367 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:43.367 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695299+0000", 2024-09-21T21:41:43.367 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2564e-05 2024-09-21T21:41:43.367 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.367 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.367 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:43.367 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695347+0000", 2024-09-21T21:41:43.367 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8248999999999998e-05 2024-09-21T21:41:43.367 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:43.367 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:43.367 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:43.367 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.367 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.367 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1549 3.16 3:6a41a830:::rbd_data.112145fae779.000000000000000a:a [read 356352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:43.367 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634583+0000", 2024-09-21T21:41:43.368 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 41.654101105000002, 2024-09-21T21:41:43.368 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060889459, 2024-09-21T21:41:43.368 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:43.368 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:43.368 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:43.368 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:43.368 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:43.368 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:43.368 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1549 2024-09-21T21:41:43.368 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.368 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:43.369 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.369 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:43.369 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634583+0000", 2024-09-21T21:41:43.369 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:43.369 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.369 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.369 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:43.369 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634583+0000", 2024-09-21T21:41:43.369 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:43.369 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.369 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.369 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:43.369 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634583+0000", 2024-09-21T21:41:43.369 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1899999999999998e-07 2024-09-21T21:41:43.370 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.370 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.370 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:43.370 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634585+0000", 2024-09-21T21:41:43.370 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2129999999999998e-06 2024-09-21T21:41:43.370 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.370 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.370 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:43.370 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634592+0000", 2024-09-21T21:41:43.370 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2840000000000003e-06 2024-09-21T21:41:43.370 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.370 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.370 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:43.370 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634596+0000", 2024-09-21T21:41:43.370 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9309999999999999e-06 2024-09-21T21:41:43.371 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.371 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.371 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:43.371 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695371+0000", 2024-09-21T21:41:43.371 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060774425999999999 2024-09-21T21:41:43.371 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.371 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.371 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:43.371 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695419+0000", 2024-09-21T21:41:43.371 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8607999999999999e-05 2024-09-21T21:41:43.371 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.371 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.371 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:43.372 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695472+0000", 2024-09-21T21:41:43.372 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2577999999999997e-05 2024-09-21T21:41:43.372 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:43.372 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:43.372 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:43.372 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.372 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.372 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1588 3.16 3:695c582e:::rbd_data.1121e540c096.0000000000000008:b [read 147456~40960] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:43.372 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.636942+0000", 2024-09-21T21:41:43.372 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 41.651741328, 2024-09-21T21:41:43.372 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.058643811999999997, 2024-09-21T21:41:43.372 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:43.372 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:43.372 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:43.372 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:43.373 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:43.373 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:43.373 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1588 2024-09-21T21:41:43.373 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.373 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:43.373 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.373 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:43.373 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636942+0000", 2024-09-21T21:41:43.373 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:43.373 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.373 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.373 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:43.373 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636942+0000", 2024-09-21T21:41:43.373 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:43.373 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.374 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.374 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:43.374 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636943+0000", 2024-09-21T21:41:43.374 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4199999999999996e-07 2024-09-21T21:41:43.374 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.374 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.374 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:43.374 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636945+0000", 2024-09-21T21:41:43.374 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.401e-06 2024-09-21T21:41:43.374 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.374 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.374 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:43.374 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636952+0000", 2024-09-21T21:41:43.374 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5379999999999999e-06 2024-09-21T21:41:43.374 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.375 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.375 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:43.375 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636958+0000", 2024-09-21T21:41:43.375 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.489e-06 2024-09-21T21:41:43.375 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.375 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.375 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:43.375 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695497+0000", 2024-09-21T21:41:43.375 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.058538504999999998 2024-09-21T21:41:43.375 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.375 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.375 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:43.375 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695533+0000", 2024-09-21T21:41:43.375 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6118000000000002e-05 2024-09-21T21:41:43.375 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.376 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.376 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:43.376 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695586+0000", 2024-09-21T21:41:43.376 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3219e-05 2024-09-21T21:41:43.376 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:43.376 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:43.376 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:43.376 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.376 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.376 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1589 3.16 3:695c582e:::rbd_data.1121e540c096.0000000000000008:b [read 487424~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:43.376 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.636965+0000", 2024-09-21T21:41:43.376 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 41.651718236999997, 2024-09-21T21:41:43.376 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.058728152999999998, 2024-09-21T21:41:43.376 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:43.377 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:43.377 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:43.377 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:43.377 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:43.377 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:43.377 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1589 2024-09-21T21:41:43.377 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.377 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:43.377 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.377 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:43.377 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636965+0000", 2024-09-21T21:41:43.377 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:43.377 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.377 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.377 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:43.378 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636965+0000", 2024-09-21T21:41:43.378 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:43.378 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.378 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.378 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:43.378 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636966+0000", 2024-09-21T21:41:43.378 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3799999999999998e-07 2024-09-21T21:41:43.378 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.378 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.378 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:43.378 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636968+0000", 2024-09-21T21:41:43.378 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2400000000000002e-06 2024-09-21T21:41:43.378 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.378 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.378 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:43.379 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636976+0000", 2024-09-21T21:41:43.379 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5050000000000001e-06 2024-09-21T21:41:43.379 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.379 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.379 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:43.379 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.636981+0000", 2024-09-21T21:41:43.379 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2170000000000002e-06 2024-09-21T21:41:43.379 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.379 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.379 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:43.379 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695610+0000", 2024-09-21T21:41:43.379 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.058628998000000002 2024-09-21T21:41:43.379 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.379 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.380 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:43.380 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695647+0000", 2024-09-21T21:41:43.380 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6752000000000003e-05 2024-09-21T21:41:43.380 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.380 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.380 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:43.380 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695694+0000", 2024-09-21T21:41:43.380 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7002999999999997e-05 2024-09-21T21:41:43.380 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:43.380 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:43.380 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:43.380 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.380 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.380 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1607 3.e 3:724b31a3:::rbd_data.1121e540c096.0000000000000015:b [read 299008~36864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:43.380 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.637062+0000", 2024-09-21T21:41:43.381 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 41.651622146000001, 2024-09-21T21:41:43.381 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.058623078000000002, 2024-09-21T21:41:43.381 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:43.381 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:43.381 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:43.381 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:43.381 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:43.381 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:43.381 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1607 2024-09-21T21:41:43.381 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.381 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:43.381 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.381 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:43.381 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.637062+0000", 2024-09-21T21:41:43.381 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:43.382 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.382 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.382 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:43.382 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.637062+0000", 2024-09-21T21:41:43.382 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:43.382 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.382 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.382 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:43.382 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.637062+0000", 2024-09-21T21:41:43.382 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0900000000000002e-07 2024-09-21T21:41:43.382 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.382 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.382 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:43.382 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.637069+0000", 2024-09-21T21:41:43.383 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.267e-06 2024-09-21T21:41:43.383 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.383 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.383 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:43.383 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.637076+0000", 2024-09-21T21:41:43.383 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7320000000000001e-06 2024-09-21T21:41:43.383 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.383 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.383 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:43.383 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.637082+0000", 2024-09-21T21:41:43.383 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4760000000000003e-06 2024-09-21T21:41:43.383 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.383 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.383 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:43.383 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695604+0000", 2024-09-21T21:41:43.384 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.058521261999999998 2024-09-21T21:41:43.384 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.384 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.384 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:43.384 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695638+0000", 2024-09-21T21:41:43.384 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4292999999999998e-05 2024-09-21T21:41:43.384 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:43.384 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:43.384 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:43.384 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695685+0000", 2024-09-21T21:41:43.384 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6638999999999998e-05 2024-09-21T21:41:43.384 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:43.384 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:43.384 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:43.384 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:43.385 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:43.385 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:43.386 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:43.384+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:43.386 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T21:41:43.446 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T21:41:43.486 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:43.485+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:43.526 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:43.526 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:41:43.526 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:43.526 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "MOSDPGRecoveryDeleteReply(3.15 e72,71 3:a9d3873a:::rbd_data.1121e0174ab1.0000000000000146:head,48'119)", 2024-09-21T21:41:43.526 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:43.524940+0000", 2024-09-21T21:41:43.526 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.00014293899999999999, 2024-09-21T21:41:43.526 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00016236199999999999, 2024-09-21T21:41:43.526 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:43.527 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:43.527 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:41:43.527 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:43.527 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:43.527 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:43.527 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:43.524940+0000", 2024-09-21T21:41:43.527 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:43.527 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:43.527 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:43.527 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:43.527 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:43.524940+0000", 2024-09-21T21:41:43.527 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:43.527 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:43.527 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:43.528 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:43.528 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:43.524941+0000", 2024-09-21T21:41:43.528 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9999999999999997e-07 2024-09-21T21:41:43.528 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:43.528 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:43.528 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:43.528 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:43.524946+0000", 2024-09-21T21:41:43.528 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.797e-06 2024-09-21T21:41:43.528 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:43.528 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:43.528 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:43.528 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:43.524956+0000", 2024-09-21T21:41:43.528 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0604999999999999e-05 2024-09-21T21:41:43.528 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:43.528 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:43.529 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:43.529 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:43.524962+0000", 2024-09-21T21:41:43.529 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2939999999999996e-06 2024-09-21T21:41:43.529 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:43.529 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:43.529 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:43.529 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:43.525014+0000", 2024-09-21T21:41:43.529 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2083000000000003e-05 2024-09-21T21:41:43.529 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:43.529 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:43.529 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:43.529 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:43.529 INFO:teuthology.orchestra.run.smithi081.stdout: ], 2024-09-21T21:41:43.529 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 1 2024-09-21T21:41:43.530 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:43.537 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T21:41:43.587 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:43.585+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:43.598 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0: {} 2024-09-21T21:41:43.598 INFO:teuthology.orchestra.run.smithi081.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T21:41:43.620 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1: {} 2024-09-21T21:41:43.620 INFO:teuthology.orchestra.run.smithi081.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T21:41:43.639 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2: {} 2024-09-21T21:41:43.639 INFO:teuthology.orchestra.run.smithi081.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T21:41:43.659 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3: {} 2024-09-21T21:41:43.660 INFO:teuthology.orchestra.run.smithi081.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T21:41:43.660 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:43.660 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:43.670 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T21:41:43.678 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4: {} 2024-09-21T21:41:43.679 INFO:teuthology.orchestra.run.smithi081.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T21:41:43.688 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:43.686+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:43.700 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5: {} 2024-09-21T21:41:43.700 INFO:teuthology.orchestra.run.smithi081.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T21:41:43.782 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:43.782 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:43.789 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:43.787+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:43.793 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T21:41:43.889 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:43.888+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:43.900 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:43.901 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:43.910 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T21:41:43.990 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:43.989+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:44.013 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:44.013 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:44.022 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T21:41:44.092 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:44.090+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:44.165 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:44.165 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:44.173 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T21:41:44.192 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:44.190+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:44.281 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:44.281 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:44.290 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T21:41:44.292 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:44.291+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:44.336 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T21:41:44.394 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:44.392+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:44.394 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:44.395 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:44.404 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T21:41:44.495 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:44.494+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:44.550 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:44.550 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:44.559 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T21:41:44.602 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:44.601+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:44.668 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:44.668 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:44.678 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T21:41:44.697 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:44.696+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:44.798 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:44.797+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:44.820 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:44.820 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:44.829 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T21:41:44.899 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:44.898+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:44.931 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:44.931 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:44.939 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T21:41:45.000 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:44.998+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:45.042 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:45.043 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:45.051 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T21:41:45.101 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:45.100+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:45.194 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:45.194 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:45.201 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:45.200+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:45.202 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T21:41:45.210 INFO:teuthology.orchestra.run.smithi081.stderr:nodeep-scrub is unset 2024-09-21T21:41:45.302 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:45.300+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:45.310 INFO:teuthology.orchestra.run.smithi164.stdout:truncating image image_client.0-clone28 from 0xc0a8200 (overlap 0x93b3c00) to 0x2c4f800 2024-09-21T21:41:45.360 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:45.361 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:45.370 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T21:41:45.413 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:45.412+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:45.474 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:45.475 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:45.483 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T21:41:45.504 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:45.502+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:45.604 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:45.603+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:45.628 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:45.628 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:45.637 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T21:41:45.696 INFO:teuthology.orchestra.run.smithi164.stdout:checking clone #27, image image_client.0-clone27 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent28 2024-09-21T21:41:45.705 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:45.704+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:45.713 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T21:41:45.783 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:45.783 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:45.792 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T21:41:45.806 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:45.805+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:45.829 INFO:teuthology.orchestra.run.smithi164.stdout:3883 clone 30 order 20 su 4096 sc 7 2024-09-21T21:41:45.834 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0: {} 2024-09-21T21:41:45.834 INFO:teuthology.orchestra.run.smithi081.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T21:41:45.852 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1: {} 2024-09-21T21:41:45.852 INFO:teuthology.orchestra.run.smithi081.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T21:41:45.868 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2: {} 2024-09-21T21:41:45.869 INFO:teuthology.orchestra.run.smithi081.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T21:41:45.885 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3: {} 2024-09-21T21:41:45.886 INFO:teuthology.orchestra.run.smithi081.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T21:41:45.900 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:45.900 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:41:45.900 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 0 2024-09-21T21:41:45.900 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:45.901 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4: {} 2024-09-21T21:41:45.902 INFO:teuthology.orchestra.run.smithi081.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T21:41:45.907 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:45.905+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:45.909 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T21:41:45.918 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5: {} 2024-09-21T21:41:45.919 INFO:teuthology.orchestra.run.smithi081.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T21:41:46.008 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:46.006+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:46.021 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:46.021 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:41:46.021 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:41:46.021 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:41:46.021 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:46.030 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T21:41:46.108 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:46.107+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:46.135 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:46.135 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 20, 2024-09-21T21:41:46.135 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 600, 2024-09-21T21:41:46.135 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:41:46.135 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.135 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4952.0:3526 3.10 3:086eb8a4:::rbd_data.1121a3eabc61.0000000000000000:17 [read 2410496~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:46.135 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:29.795290+0000", 2024-09-21T21:41:46.135 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 16.337135639, 2024-09-21T21:41:46.135 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.032825482000000003, 2024-09-21T21:41:46.135 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:46.135 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:46.135 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:46.136 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:46.136 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4952", 2024-09-21T21:41:46.136 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:46.136 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3526 2024-09-21T21:41:46.136 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.136 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:46.136 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.136 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:46.136 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795290+0000", 2024-09-21T21:41:46.136 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:46.136 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.136 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.136 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:46.136 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795290+0000", 2024-09-21T21:41:46.137 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:46.137 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.137 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.137 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:46.137 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795291+0000", 2024-09-21T21:41:46.137 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3499999999999996e-07 2024-09-21T21:41:46.137 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.137 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.137 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:46.138 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795293+0000", 2024-09-21T21:41:46.138 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0779999999999998e-06 2024-09-21T21:41:46.138 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.138 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.138 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:46.138 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795299+0000", 2024-09-21T21:41:46.138 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7270000000000004e-06 2024-09-21T21:41:46.138 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.138 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.138 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:46.138 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795303+0000", 2024-09-21T21:41:46.138 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3579999999999996e-06 2024-09-21T21:41:46.138 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.139 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.139 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:46.139 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.827920+0000", 2024-09-21T21:41:46.139 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.032617118 2024-09-21T21:41:46.139 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.139 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.139 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:46.139 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.828011+0000", 2024-09-21T21:41:46.139 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.0676999999999999e-05 2024-09-21T21:41:46.139 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.139 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.139 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:46.139 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.828116+0000", 2024-09-21T21:41:46.139 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010498899999999999 2024-09-21T21:41:46.140 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:46.140 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:46.140 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:46.140 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.140 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.140 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4952.0:3528 3.10 3:086eb8a4:::rbd_data.1121a3eabc61.0000000000000000:17 [read 2424832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:46.140 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:29.795308+0000", 2024-09-21T21:41:46.140 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 16.337117917, 2024-09-21T21:41:46.140 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033028465999999999, 2024-09-21T21:41:46.140 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:46.140 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:46.140 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:46.140 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:46.140 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4952", 2024-09-21T21:41:46.141 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:46.141 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3528 2024-09-21T21:41:46.141 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.141 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:46.141 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.141 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:46.141 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795308+0000", 2024-09-21T21:41:46.141 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:46.141 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.141 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.141 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:46.141 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795308+0000", 2024-09-21T21:41:46.141 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:46.141 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.142 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.142 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:46.142 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795308+0000", 2024-09-21T21:41:46.142 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.22e-07 2024-09-21T21:41:46.142 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.142 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.142 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:46.142 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795310+0000", 2024-09-21T21:41:46.142 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.7940000000000001e-06 2024-09-21T21:41:46.142 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.142 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.142 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:46.142 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795316+0000", 2024-09-21T21:41:46.142 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9410000000000004e-06 2024-09-21T21:41:46.143 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.143 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.143 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:46.143 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795321+0000", 2024-09-21T21:41:46.143 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4800000000000003e-06 2024-09-21T21:41:46.143 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.143 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.143 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:46.143 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.828149+0000", 2024-09-21T21:41:46.143 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.032828001000000002 2024-09-21T21:41:46.143 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.143 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.143 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:46.143 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.828223+0000", 2024-09-21T21:41:46.143 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.4048000000000007e-05 2024-09-21T21:41:46.144 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.144 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.144 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:46.144 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.828337+0000", 2024-09-21T21:41:46.144 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011388 2024-09-21T21:41:46.144 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:46.144 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:46.144 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:46.144 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.144 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.144 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4952.0:3529 3.10 3:086eb8a4:::rbd_data.1121a3eabc61.0000000000000000:17 [read 2439168~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:46.144 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:29.795328+0000", 2024-09-21T21:41:46.144 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 16.337097821, 2024-09-21T21:41:46.144 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033236716999999999, 2024-09-21T21:41:46.145 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:46.145 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:46.145 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:46.145 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:46.145 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4952", 2024-09-21T21:41:46.145 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:46.145 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3529 2024-09-21T21:41:46.145 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.145 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:46.145 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.145 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:46.145 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795328+0000", 2024-09-21T21:41:46.145 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:46.145 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.145 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.146 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:46.146 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795328+0000", 2024-09-21T21:41:46.146 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:46.146 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.146 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.146 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:46.146 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795329+0000", 2024-09-21T21:41:46.146 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.44e-07 2024-09-21T21:41:46.146 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.146 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.146 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:46.146 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795335+0000", 2024-09-21T21:41:46.146 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7400000000000001e-06 2024-09-21T21:41:46.146 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.147 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.147 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:46.147 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795340+0000", 2024-09-21T21:41:46.147 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9009999999999999e-06 2024-09-21T21:41:46.147 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.147 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.147 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:46.147 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795345+0000", 2024-09-21T21:41:46.147 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4839999999999998e-06 2024-09-21T21:41:46.147 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.147 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.147 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:46.147 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.828382+0000", 2024-09-21T21:41:46.147 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033037113 2024-09-21T21:41:46.147 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.148 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.148 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:46.148 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.828473+0000", 2024-09-21T21:41:46.148 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.1056000000000004e-05 2024-09-21T21:41:46.148 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.148 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.148 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:46.148 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.828565+0000", 2024-09-21T21:41:46.148 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.1879000000000006e-05 2024-09-21T21:41:46.148 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:46.148 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:46.148 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:46.148 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.148 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.149 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4952.0:3531 3.10 3:086eb8a4:::rbd_data.1121a3eabc61.0000000000000000:17 [read 2453504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:46.149 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:29.795353+0000", 2024-09-21T21:41:46.149 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 16.337073092000001, 2024-09-21T21:41:46.149 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033394973000000001, 2024-09-21T21:41:46.149 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:46.149 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:46.149 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:46.149 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:46.149 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4952", 2024-09-21T21:41:46.149 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:46.149 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3531 2024-09-21T21:41:46.149 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.149 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:46.149 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.150 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:46.150 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795353+0000", 2024-09-21T21:41:46.150 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:46.150 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.150 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.150 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:46.150 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795353+0000", 2024-09-21T21:41:46.150 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:46.150 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.150 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.150 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:46.150 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795353+0000", 2024-09-21T21:41:46.150 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7300000000000001e-07 2024-09-21T21:41:46.151 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.151 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.151 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:46.151 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795360+0000", 2024-09-21T21:41:46.151 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2829999999999998e-06 2024-09-21T21:41:46.151 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.151 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.151 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:46.151 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795366+0000", 2024-09-21T21:41:46.151 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1260000000000003e-06 2024-09-21T21:41:46.151 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.151 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.151 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:46.151 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795370+0000", 2024-09-21T21:41:46.152 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1899999999999997e-06 2024-09-21T21:41:46.152 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.152 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.152 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:46.152 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.828587+0000", 2024-09-21T21:41:46.152 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033216516000000001 2024-09-21T21:41:46.152 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.152 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.152 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:46.152 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.828662+0000", 2024-09-21T21:41:46.152 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5316999999999997e-05 2024-09-21T21:41:46.152 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.152 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.152 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:46.153 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.828748+0000", 2024-09-21T21:41:46.153 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.6068000000000003e-05 2024-09-21T21:41:46.153 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:46.153 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:46.153 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:46.153 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.153 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.153 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4952.0:3533 3.10 3:086eb8a4:::rbd_data.1121a3eabc61.0000000000000000:17 [read 2467840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:46.153 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:29.795376+0000", 2024-09-21T21:41:46.153 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 16.337049890999999, 2024-09-21T21:41:46.153 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033574701999999998, 2024-09-21T21:41:46.153 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:46.153 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:46.154 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:46.154 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:46.154 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4952", 2024-09-21T21:41:46.154 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:46.154 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3533 2024-09-21T21:41:46.154 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.154 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:46.154 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.154 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:46.154 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795376+0000", 2024-09-21T21:41:46.154 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:46.154 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.154 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.155 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:46.155 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795376+0000", 2024-09-21T21:41:46.155 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:46.155 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.155 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.155 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:46.155 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795377+0000", 2024-09-21T21:41:46.155 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5400000000000002e-07 2024-09-21T21:41:46.155 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.155 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.155 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:46.155 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795379+0000", 2024-09-21T21:41:46.155 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9659999999999999e-06 2024-09-21T21:41:46.155 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.156 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.156 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:46.156 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795385+0000", 2024-09-21T21:41:46.156 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1519999999999997e-06 2024-09-21T21:41:46.156 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.156 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.156 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:46.156 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795389+0000", 2024-09-21T21:41:46.156 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4399999999999998e-06 2024-09-21T21:41:46.156 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.156 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.156 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:46.156 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.828777+0000", 2024-09-21T21:41:46.157 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033388099999999997 2024-09-21T21:41:46.157 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.157 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.157 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:46.157 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.828856+0000", 2024-09-21T21:41:46.157 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.8770999999999995e-05 2024-09-21T21:41:46.157 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.157 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.157 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:46.157 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.828951+0000", 2024-09-21T21:41:46.157 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.4919000000000004e-05 2024-09-21T21:41:46.157 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:46.157 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:46.157 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:46.158 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.158 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.158 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4952.0:3535 3.10 3:086eb8a4:::rbd_data.1121a3eabc61.0000000000000000:17 [read 2482176~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:46.158 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:29.795394+0000", 2024-09-21T21:41:46.158 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 16.337031761999999, 2024-09-21T21:41:46.158 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033738335000000001, 2024-09-21T21:41:46.158 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:46.158 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:46.158 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:46.158 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:46.158 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4952", 2024-09-21T21:41:46.158 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:46.159 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3535 2024-09-21T21:41:46.159 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.159 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:46.159 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.159 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:46.159 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795394+0000", 2024-09-21T21:41:46.159 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:46.159 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.159 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.159 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:46.159 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.795394+0000", 2024-09-21T21:41:46.159 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:46.159 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.160 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.160 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:46.160 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.796945+0000", 2024-09-21T21:41:46.160 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.001551097 2024-09-21T21:41:46.160 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.160 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.160 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:46.160 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.796949+0000", 2024-09-21T21:41:46.160 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5499999999999999e-06 2024-09-21T21:41:46.160 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.160 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.160 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:46.160 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.796958+0000", 2024-09-21T21:41:46.160 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.1649999999999995e-06 2024-09-21T21:41:46.161 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.161 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.161 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:46.161 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.796964+0000", 2024-09-21T21:41:46.161 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1199999999999999e-06 2024-09-21T21:41:46.161 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.161 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.161 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:46.161 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.828983+0000", 2024-09-21T21:41:46.161 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.032018396999999997 2024-09-21T21:41:46.161 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.161 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.161 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:46.162 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.829050+0000", 2024-09-21T21:41:46.162 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.7020000000000005e-05 2024-09-21T21:41:46.162 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.162 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.162 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:46.162 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.829133+0000", 2024-09-21T21:41:46.162 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.2985999999999995e-05 2024-09-21T21:41:46.162 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:46.162 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:46.162 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:46.162 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.162 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.162 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4952.0:3546 3.10 3:086eb8a4:::rbd_data.1121a3eabc61.0000000000000000:17 [read 2576384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:46.162 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:29.797117+0000", 2024-09-21T21:41:46.163 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 16.335308908000002, 2024-09-21T21:41:46.163 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.03288253, 2024-09-21T21:41:46.163 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:46.163 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:46.163 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:46.163 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:46.163 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4952", 2024-09-21T21:41:46.163 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:46.163 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3546 2024-09-21T21:41:46.163 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.163 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:46.163 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.163 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:46.163 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797117+0000", 2024-09-21T21:41:46.164 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:46.164 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.164 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.164 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:46.164 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797117+0000", 2024-09-21T21:41:46.164 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:46.164 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.164 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.164 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:46.164 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797118+0000", 2024-09-21T21:41:46.164 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.39e-07 2024-09-21T21:41:46.164 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.164 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.164 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:46.165 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797120+0000", 2024-09-21T21:41:46.165 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9980000000000002e-06 2024-09-21T21:41:46.165 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.165 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.165 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:46.165 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797125+0000", 2024-09-21T21:41:46.165 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7409999999999998e-06 2024-09-21T21:41:46.165 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.165 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.165 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:46.165 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797130+0000", 2024-09-21T21:41:46.165 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5820000000000003e-06 2024-09-21T21:41:46.165 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.165 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.165 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:46.166 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.829898+0000", 2024-09-21T21:41:46.166 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.032768600000000002 2024-09-21T21:41:46.166 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.166 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.166 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:46.166 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.829934+0000", 2024-09-21T21:41:46.166 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5342000000000001e-05 2024-09-21T21:41:46.166 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.166 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.166 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:46.166 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830000+0000", 2024-09-21T21:41:46.166 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5927999999999996e-05 2024-09-21T21:41:46.166 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:46.166 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:46.167 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:46.167 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.167 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.167 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4952.0:3548 3.10 3:086eb8a4:::rbd_data.1121a3eabc61.0000000000000000:17 [read 2590720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:46.167 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:29.797136+0000", 2024-09-21T21:41:46.167 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 16.335290468, 2024-09-21T21:41:46.167 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033006742999999998, 2024-09-21T21:41:46.167 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:46.167 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:46.167 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:46.167 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:46.167 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4952", 2024-09-21T21:41:46.167 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:46.167 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3548 2024-09-21T21:41:46.168 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.168 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:46.168 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.168 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:46.168 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797136+0000", 2024-09-21T21:41:46.168 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:46.168 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.168 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.168 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:46.168 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797136+0000", 2024-09-21T21:41:46.168 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:46.168 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.168 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.168 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:46.169 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797136+0000", 2024-09-21T21:41:46.169 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3600000000000004e-07 2024-09-21T21:41:46.169 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.169 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.169 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:46.169 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797138+0000", 2024-09-21T21:41:46.169 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.933e-06 2024-09-21T21:41:46.169 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.169 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.169 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:46.169 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797144+0000", 2024-09-21T21:41:46.169 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6099999999999997e-06 2024-09-21T21:41:46.169 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.169 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.170 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:46.170 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797149+0000", 2024-09-21T21:41:46.170 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0690000000000001e-06 2024-09-21T21:41:46.170 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.170 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.170 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:46.170 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830038+0000", 2024-09-21T21:41:46.170 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.032889535999999997 2024-09-21T21:41:46.170 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.170 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.170 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:46.170 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830087+0000", 2024-09-21T21:41:46.170 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9159999999999997e-05 2024-09-21T21:41:46.170 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.171 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.171 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:46.171 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830142+0000", 2024-09-21T21:41:46.171 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4898999999999997e-05 2024-09-21T21:41:46.171 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:46.171 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:46.171 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:46.171 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.171 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.171 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4952.0:3550 3.10 3:086eb8a4:::rbd_data.1121a3eabc61.0000000000000000:17 [read 2605056~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:46.171 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:29.797154+0000", 2024-09-21T21:41:46.171 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 16.335271909999999, 2024-09-21T21:41:46.172 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033113318000000003, 2024-09-21T21:41:46.172 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:46.172 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:46.172 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:46.172 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:46.172 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4952", 2024-09-21T21:41:46.172 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:46.172 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3550 2024-09-21T21:41:46.172 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.172 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:46.172 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.172 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:46.172 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797154+0000", 2024-09-21T21:41:46.172 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:46.172 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.173 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.173 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:46.173 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797154+0000", 2024-09-21T21:41:46.173 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:46.173 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.173 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.173 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:46.173 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797155+0000", 2024-09-21T21:41:46.173 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1199999999999998e-07 2024-09-21T21:41:46.173 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.173 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.173 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:46.173 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797165+0000", 2024-09-21T21:41:46.173 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0635000000000001e-05 2024-09-21T21:41:46.174 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.174 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.174 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:46.174 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797172+0000", 2024-09-21T21:41:46.174 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5300000000000002e-06 2024-09-21T21:41:46.174 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.174 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.174 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:46.174 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797176+0000", 2024-09-21T21:41:46.174 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7419999999999997e-06 2024-09-21T21:41:46.174 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.174 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.174 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:46.174 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830167+0000", 2024-09-21T21:41:46.175 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.032990468000000002 2024-09-21T21:41:46.175 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.175 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.175 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:46.175 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830202+0000", 2024-09-21T21:41:46.175 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5382000000000001e-05 2024-09-21T21:41:46.175 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.175 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.175 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:46.175 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830267+0000", 2024-09-21T21:41:46.175 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5148999999999996e-05 2024-09-21T21:41:46.175 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:46.175 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:46.176 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:46.176 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.176 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.176 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4952.0:3551 3.10 3:086eb8a4:::rbd_data.1121a3eabc61.0000000000000000:17 [read 2619392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:46.176 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:29.797183+0000", 2024-09-21T21:41:46.176 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 16.335243298000002, 2024-09-21T21:41:46.176 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033198722, 2024-09-21T21:41:46.176 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:46.176 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:46.176 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:46.176 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:46.176 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4952", 2024-09-21T21:41:46.177 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:46.177 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3551 2024-09-21T21:41:46.177 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.177 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:46.177 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.177 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:46.177 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797183+0000", 2024-09-21T21:41:46.177 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:46.177 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.177 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.177 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:46.178 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797183+0000", 2024-09-21T21:41:46.178 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:46.178 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.178 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.178 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:46.178 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797183+0000", 2024-09-21T21:41:46.178 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8000000000000001e-07 2024-09-21T21:41:46.178 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.179 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.179 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:46.179 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797185+0000", 2024-09-21T21:41:46.179 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0940000000000002e-06 2024-09-21T21:41:46.179 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.179 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.179 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:46.179 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797191+0000", 2024-09-21T21:41:46.179 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8300000000000001e-06 2024-09-21T21:41:46.179 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.179 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.180 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:46.180 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797196+0000", 2024-09-21T21:41:46.180 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4530000000000004e-06 2024-09-21T21:41:46.180 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.180 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.180 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:46.180 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830295+0000", 2024-09-21T21:41:46.180 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033099669999999998 2024-09-21T21:41:46.180 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.180 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.180 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:46.180 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830332+0000", 2024-09-21T21:41:46.181 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7156999999999997e-05 2024-09-21T21:41:46.181 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.181 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.181 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:46.181 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830381+0000", 2024-09-21T21:41:46.181 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9138000000000003e-05 2024-09-21T21:41:46.181 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:46.181 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:46.181 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:46.181 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.181 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.181 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4952.0:3553 3.10 3:086eb8a4:::rbd_data.1121a3eabc61.0000000000000000:17 [read 2627584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:46.181 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:29.797201+0000", 2024-09-21T21:41:46.182 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 16.335225078000001, 2024-09-21T21:41:46.182 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033313262000000003, 2024-09-21T21:41:46.182 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:46.182 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:46.182 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:46.182 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:46.182 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4952", 2024-09-21T21:41:46.182 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:46.182 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3553 2024-09-21T21:41:46.182 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.182 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:46.182 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.183 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:46.183 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797201+0000", 2024-09-21T21:41:46.183 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:46.183 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.183 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.183 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:46.183 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797201+0000", 2024-09-21T21:41:46.183 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:46.183 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.183 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.183 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:46.183 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797201+0000", 2024-09-21T21:41:46.184 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6600000000000002e-07 2024-09-21T21:41:46.184 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.184 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.184 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:46.184 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797203+0000", 2024-09-21T21:41:46.184 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.8810000000000001e-06 2024-09-21T21:41:46.184 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.184 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.184 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:46.185 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797209+0000", 2024-09-21T21:41:46.185 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.037e-06 2024-09-21T21:41:46.185 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.185 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.185 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:46.185 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797214+0000", 2024-09-21T21:41:46.185 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4920000000000004e-06 2024-09-21T21:41:46.185 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.185 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.185 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:46.185 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830408+0000", 2024-09-21T21:41:46.185 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033194199000000001 2024-09-21T21:41:46.185 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.186 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.186 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:46.186 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830455+0000", 2024-09-21T21:41:46.186 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6547e-05 2024-09-21T21:41:46.186 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.186 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.186 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:46.186 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830514+0000", 2024-09-21T21:41:46.186 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9740000000000001e-05 2024-09-21T21:41:46.186 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:46.186 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:46.187 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:46.187 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.187 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.187 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4952.0:3554 3.10 3:086eb8a4:::rbd_data.1121a3eabc61.0000000000000000:17 [read 2641920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:46.187 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:29.797220+0000", 2024-09-21T21:41:46.187 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 16.335205783999999, 2024-09-21T21:41:46.187 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033396381000000003, 2024-09-21T21:41:46.187 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:46.187 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:46.187 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:46.187 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:46.187 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4952", 2024-09-21T21:41:46.187 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:46.187 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3554 2024-09-21T21:41:46.188 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.188 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:46.188 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.188 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:46.188 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797220+0000", 2024-09-21T21:41:46.188 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:46.188 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.188 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.188 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:46.188 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797220+0000", 2024-09-21T21:41:46.188 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:46.188 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.188 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.189 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:46.189 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797221+0000", 2024-09-21T21:41:46.189 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4099999999999999e-07 2024-09-21T21:41:46.189 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.189 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.189 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:46.189 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797223+0000", 2024-09-21T21:41:46.189 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.95e-06 2024-09-21T21:41:46.189 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.189 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.189 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:46.189 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797228+0000", 2024-09-21T21:41:46.189 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6919999999999999e-06 2024-09-21T21:41:46.189 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.190 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.190 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:46.190 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797238+0000", 2024-09-21T21:41:46.190 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6509999999999996e-06 2024-09-21T21:41:46.190 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.190 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.190 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:46.190 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830532+0000", 2024-09-21T21:41:46.190 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033294473999999998 2024-09-21T21:41:46.190 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.190 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.190 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:46.190 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830569+0000", 2024-09-21T21:41:46.190 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6723000000000001e-05 2024-09-21T21:41:46.190 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.191 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.191 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:46.191 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830617+0000", 2024-09-21T21:41:46.191 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7450000000000001e-05 2024-09-21T21:41:46.191 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:46.191 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:46.191 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:46.191 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.191 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.191 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4952.0:3555 3.10 3:086eb8a4:::rbd_data.1121a3eabc61.0000000000000000:17 [read 2656256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:46.191 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:29.797244+0000", 2024-09-21T21:41:46.191 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 16.335181911999999, 2024-09-21T21:41:46.191 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033483065999999999, 2024-09-21T21:41:46.191 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:46.191 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:46.192 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:46.192 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:46.192 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4952", 2024-09-21T21:41:46.192 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:46.192 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3555 2024-09-21T21:41:46.192 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.192 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:46.192 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.192 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:46.192 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797244+0000", 2024-09-21T21:41:46.192 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:46.192 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.192 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.192 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:46.193 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797244+0000", 2024-09-21T21:41:46.193 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:46.193 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.193 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.193 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:46.193 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797245+0000", 2024-09-21T21:41:46.193 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9299999999999998e-07 2024-09-21T21:41:46.193 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.193 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.193 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:46.193 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797247+0000", 2024-09-21T21:41:46.193 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0370000000000001e-06 2024-09-21T21:41:46.193 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.193 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.194 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:46.194 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797253+0000", 2024-09-21T21:41:46.194 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8189999999999997e-06 2024-09-21T21:41:46.194 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.194 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.194 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:46.194 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797257+0000", 2024-09-21T21:41:46.194 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.33e-06 2024-09-21T21:41:46.194 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.194 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.195 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:46.195 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830642+0000", 2024-09-21T21:41:46.195 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033385629999999999 2024-09-21T21:41:46.195 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.195 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.195 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:46.195 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830679+0000", 2024-09-21T21:41:46.195 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6100999999999999e-05 2024-09-21T21:41:46.195 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.195 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.195 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:46.195 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830727+0000", 2024-09-21T21:41:46.195 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8656000000000003e-05 2024-09-21T21:41:46.196 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:46.196 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:46.196 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:46.196 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.196 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.196 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4952.0:3557 3.10 3:086eb8a4:::rbd_data.1121a3eabc61.0000000000000000:17 [read 2670592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:46.196 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:29.797263+0000", 2024-09-21T21:41:46.196 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 16.335163049999998, 2024-09-21T21:41:46.196 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033580213999999997, 2024-09-21T21:41:46.196 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:46.196 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:46.197 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:46.197 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:46.197 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4952", 2024-09-21T21:41:46.197 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:46.197 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3557 2024-09-21T21:41:46.197 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.197 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:46.197 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.197 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:46.197 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797263+0000", 2024-09-21T21:41:46.197 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:46.197 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.198 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.198 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:46.198 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797263+0000", 2024-09-21T21:41:46.198 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:46.198 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.198 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.198 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:46.198 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797263+0000", 2024-09-21T21:41:46.198 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4799999999999999e-07 2024-09-21T21:41:46.198 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.198 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.198 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:46.199 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797265+0000", 2024-09-21T21:41:46.199 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.029e-06 2024-09-21T21:41:46.199 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.199 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.199 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:46.199 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797271+0000", 2024-09-21T21:41:46.199 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0000000000000002e-06 2024-09-21T21:41:46.199 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.199 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.199 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:46.199 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797276+0000", 2024-09-21T21:41:46.199 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5310000000000003e-06 2024-09-21T21:41:46.199 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.200 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.200 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:46.200 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830756+0000", 2024-09-21T21:41:46.200 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033479636 2024-09-21T21:41:46.200 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.200 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.200 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:46.200 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830792+0000", 2024-09-21T21:41:46.200 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6584000000000001e-05 2024-09-21T21:41:46.200 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.200 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.200 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:46.201 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830843+0000", 2024-09-21T21:41:46.201 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0986000000000003e-05 2024-09-21T21:41:46.201 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:46.201 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:46.201 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:46.201 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.201 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.201 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4952.0:3558 3.10 3:086eb8a4:::rbd_data.1121a3eabc61.0000000000000000:17 [read 2684928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:46.201 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:29.797281+0000", 2024-09-21T21:41:46.201 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 16.335144754000002, 2024-09-21T21:41:46.201 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033692174999999998, 2024-09-21T21:41:46.201 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:46.201 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:46.202 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:46.202 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:46.202 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4952", 2024-09-21T21:41:46.202 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:46.202 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3558 2024-09-21T21:41:46.202 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.202 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:46.202 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.202 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:46.202 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797281+0000", 2024-09-21T21:41:46.202 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:46.202 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.202 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.202 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:46.203 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797281+0000", 2024-09-21T21:41:46.203 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:46.203 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.203 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.203 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:46.203 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797282+0000", 2024-09-21T21:41:46.203 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.41e-07 2024-09-21T21:41:46.203 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.203 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.203 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:46.203 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797283+0000", 2024-09-21T21:41:46.203 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.8390000000000001e-06 2024-09-21T21:41:46.203 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.203 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.204 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:46.204 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797289+0000", 2024-09-21T21:41:46.204 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3839999999999996e-06 2024-09-21T21:41:46.204 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.204 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.204 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:46.204 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797294+0000", 2024-09-21T21:41:46.204 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1900000000000003e-06 2024-09-21T21:41:46.204 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.204 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.204 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:46.204 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830874+0000", 2024-09-21T21:41:46.204 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033580384999999997 2024-09-21T21:41:46.204 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.205 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.205 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:46.205 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830912+0000", 2024-09-21T21:41:46.205 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7367000000000003e-05 2024-09-21T21:41:46.205 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.205 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.205 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:46.205 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830973+0000", 2024-09-21T21:41:46.205 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1668999999999995e-05 2024-09-21T21:41:46.205 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:46.205 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:46.205 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:46.205 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.206 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.206 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4952.0:3560 3.10 3:086eb8a4:::rbd_data.1121a3eabc61.0000000000000000:17 [read 2699264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:46.206 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:29.797300+0000", 2024-09-21T21:41:46.206 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 16.335126071000001, 2024-09-21T21:41:46.206 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033777109, 2024-09-21T21:41:46.206 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:46.206 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:46.206 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:46.206 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:46.206 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4952", 2024-09-21T21:41:46.206 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:46.206 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3560 2024-09-21T21:41:46.206 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.207 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:46.207 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.207 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:46.207 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797300+0000", 2024-09-21T21:41:46.207 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:46.207 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.207 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.207 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:46.207 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797300+0000", 2024-09-21T21:41:46.207 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:46.207 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.207 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.207 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:46.207 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797300+0000", 2024-09-21T21:41:46.208 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.2399999999999999e-07 2024-09-21T21:41:46.208 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.208 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.208 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:46.208 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797302+0000", 2024-09-21T21:41:46.208 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0260000000000002e-06 2024-09-21T21:41:46.208 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.208 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.208 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:46.208 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797308+0000", 2024-09-21T21:41:46.208 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8839999999999999e-06 2024-09-21T21:41:46.208 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.208 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.208 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:46.209 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797313+0000", 2024-09-21T21:41:46.209 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3959999999999999e-06 2024-09-21T21:41:46.209 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.209 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.209 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:46.209 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.830997+0000", 2024-09-21T21:41:46.209 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033684506000000003 2024-09-21T21:41:46.209 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.209 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.209 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:46.209 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.831033+0000", 2024-09-21T21:41:46.209 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5964000000000003e-05 2024-09-21T21:41:46.209 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.210 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.210 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:46.210 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.831077+0000", 2024-09-21T21:41:46.210 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4008999999999998e-05 2024-09-21T21:41:46.210 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:46.210 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:46.210 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:46.210 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.210 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.210 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4952.0:3562 3.10 3:086eb8a4:::rbd_data.1121a3eabc61.0000000000000000:17 [read 2713600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:46.210 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:29.797318+0000", 2024-09-21T21:41:46.211 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 16.335108214000002, 2024-09-21T21:41:46.211 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033877712999999997, 2024-09-21T21:41:46.211 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:46.211 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:46.211 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:46.211 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:46.211 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4952", 2024-09-21T21:41:46.211 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:46.211 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3562 2024-09-21T21:41:46.211 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.212 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:46.212 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.212 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:46.212 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797318+0000", 2024-09-21T21:41:46.212 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:46.212 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.212 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.212 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:46.212 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797318+0000", 2024-09-21T21:41:46.212 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:46.212 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.212 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.212 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:46.213 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797318+0000", 2024-09-21T21:41:46.213 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.2000000000000001e-07 2024-09-21T21:41:46.213 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.213 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.213 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:46.213 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797320+0000", 2024-09-21T21:41:46.213 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9700000000000002e-06 2024-09-21T21:41:46.213 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.213 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.213 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:46.213 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797326+0000", 2024-09-21T21:41:46.213 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.212e-06 2024-09-21T21:41:46.214 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.214 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.214 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:46.214 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797331+0000", 2024-09-21T21:41:46.214 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9409999999999998e-06 2024-09-21T21:41:46.214 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.214 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.214 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:46.214 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.831104+0000", 2024-09-21T21:41:46.214 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033772976000000003 2024-09-21T21:41:46.214 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.214 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.215 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:46.215 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.831144+0000", 2024-09-21T21:41:46.215 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.0209000000000001e-05 2024-09-21T21:41:46.215 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.215 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.215 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:46.215 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.831196+0000", 2024-09-21T21:41:46.215 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1085000000000003e-05 2024-09-21T21:41:46.215 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:46.215 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:46.215 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:46.215 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.215 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.216 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4952.0:3563 3.10 3:086eb8a4:::rbd_data.1121a3eabc61.0000000000000000:17 [read 2727936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:46.216 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:29.797350+0000", 2024-09-21T21:41:46.216 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 16.335075962000001, 2024-09-21T21:41:46.216 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033952501000000003, 2024-09-21T21:41:46.216 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:46.216 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:46.216 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:46.216 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:46.216 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4952", 2024-09-21T21:41:46.216 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:46.216 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3563 2024-09-21T21:41:46.216 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.217 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:46.217 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.217 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:46.217 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797350+0000", 2024-09-21T21:41:46.217 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:46.217 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.217 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.217 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:46.217 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797350+0000", 2024-09-21T21:41:46.217 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:46.217 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.217 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.217 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:46.217 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797351+0000", 2024-09-21T21:41:46.218 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5499999999999998e-07 2024-09-21T21:41:46.218 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.218 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.218 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:46.218 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797353+0000", 2024-09-21T21:41:46.218 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2299999999999998e-06 2024-09-21T21:41:46.218 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.218 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.218 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:46.218 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797359+0000", 2024-09-21T21:41:46.218 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3330000000000002e-06 2024-09-21T21:41:46.219 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.219 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.219 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:46.219 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797364+0000", 2024-09-21T21:41:46.219 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4410000000000003e-06 2024-09-21T21:41:46.219 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.219 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.219 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:46.219 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.831223+0000", 2024-09-21T21:41:46.219 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033859002999999999 2024-09-21T21:41:46.219 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.219 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.219 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:46.219 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.831258+0000", 2024-09-21T21:41:46.220 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5417000000000003e-05 2024-09-21T21:41:46.220 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.220 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.220 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:46.220 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.831303+0000", 2024-09-21T21:41:46.220 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4521999999999998e-05 2024-09-21T21:41:46.220 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:46.220 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:46.220 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:46.220 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.220 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.220 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4952.0:3564 3.10 3:086eb8a4:::rbd_data.1121a3eabc61.0000000000000000:17 [read 2742272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:46.221 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:29.797370+0000", 2024-09-21T21:41:46.221 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 16.335056111, 2024-09-21T21:41:46.221 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.034051217000000002, 2024-09-21T21:41:46.221 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:46.221 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:46.221 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:46.221 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:46.221 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4952", 2024-09-21T21:41:46.221 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:46.221 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3564 2024-09-21T21:41:46.221 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.221 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:46.221 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.221 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:46.221 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797370+0000", 2024-09-21T21:41:46.222 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:46.222 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.222 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.222 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:46.222 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797370+0000", 2024-09-21T21:41:46.222 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:46.222 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.222 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.222 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:46.222 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797370+0000", 2024-09-21T21:41:46.222 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5499999999999998e-07 2024-09-21T21:41:46.222 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.222 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.222 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:46.223 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797372+0000", 2024-09-21T21:41:46.223 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9290000000000001e-06 2024-09-21T21:41:46.223 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.223 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.223 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:46.223 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797378+0000", 2024-09-21T21:41:46.223 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9599999999999997e-06 2024-09-21T21:41:46.223 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.223 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.223 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:46.223 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.797388+0000", 2024-09-21T21:41:46.223 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.5040000000000008e-06 2024-09-21T21:41:46.223 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.223 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.224 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:46.224 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.831335+0000", 2024-09-21T21:41:46.224 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.033947374000000002 2024-09-21T21:41:46.224 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.224 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.224 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:46.224 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.831371+0000", 2024-09-21T21:41:46.224 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5868999999999998e-05 2024-09-21T21:41:46.224 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.224 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.224 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:46.224 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:29.831421+0000", 2024-09-21T21:41:46.224 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0126000000000001e-05 2024-09-21T21:41:46.224 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:46.224 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:46.224 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:46.225 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.225 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.225 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5215.0:1073 3.1b 3:db41b1e4:::rbd_data.1121524d2522.0000000000000002:head [write 456704~63488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-21T21:41:46.225 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:38.158066+0000", 2024-09-21T21:41:46.225 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 7.9743605449999997, 2024-09-21T21:41:46.225 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.038046675000000002, 2024-09-21T21:41:46.225 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:46.225 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:46.225 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:41:46.225 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:46.225 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5215", 2024-09-21T21:41:46.225 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1580282168", 2024-09-21T21:41:46.225 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1073 2024-09-21T21:41:46.225 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.226 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:46.226 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.226 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:46.226 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:38.158066+0000", 2024-09-21T21:41:46.226 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:46.226 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.226 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.226 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:46.226 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:38.158066+0000", 2024-09-21T21:41:46.226 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:46.226 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.226 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.226 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:46.226 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:38.158066+0000", 2024-09-21T21:41:46.227 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.3600000000000002e-07 2024-09-21T21:41:46.227 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.227 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.227 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:46.227 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:38.158109+0000", 2024-09-21T21:41:46.227 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2731000000000001e-05 2024-09-21T21:41:46.227 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.227 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.227 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:46.227 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:38.158121+0000", 2024-09-21T21:41:46.227 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1562999999999999e-05 2024-09-21T21:41:46.227 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.227 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.227 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:46.227 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:38.158127+0000", 2024-09-21T21:41:46.228 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.348e-06 2024-09-21T21:41:46.228 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.228 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.228 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:46.228 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:38.158192+0000", 2024-09-21T21:41:46.228 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5246000000000005e-05 2024-09-21T21:41:46.228 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.228 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.228 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:46.229 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:38.158269+0000", 2024-09-21T21:41:46.229 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7103999999999999e-05 2024-09-21T21:41:46.229 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.229 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.229 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 4", 2024-09-21T21:41:46.229 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:38.158393+0000", 2024-09-21T21:41:46.229 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00012402600000000001 2024-09-21T21:41:46.229 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.229 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.229 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:41:46.229 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:38.159240+0000", 2024-09-21T21:41:46.229 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00084709399999999995 2024-09-21T21:41:46.229 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.229 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.230 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:41:46.230 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:38.196002+0000", 2024-09-21T21:41:46.230 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.036761794 2024-09-21T21:41:46.230 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.230 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.230 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:41:46.230 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:38.196064+0000", 2024-09-21T21:41:46.230 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1247000000000005e-05 2024-09-21T21:41:46.230 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.230 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.230 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:46.231 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:38.196112+0000", 2024-09-21T21:41:46.231 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8686000000000001e-05 2024-09-21T21:41:46.231 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:46.231 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:46.231 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:46.231 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:46.231 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:46.231 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:46.233 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T21:41:46.233 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:46.231+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:46.323 INFO:teuthology.orchestra.run.smithi164.stdout:leaving image image_client.0-clone29 intact 2024-09-21T21:41:46.333 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:46.331+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:46.377 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:46.377 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:41:46.377 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 0 2024-09-21T21:41:46.377 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:46.385 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T21:41:46.471 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:46.470+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:46.533 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:46.533 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:41:46.533 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:41:46.533 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:41:46.534 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:46.534 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:46.533+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:46.542 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T21:41:46.635 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:46.634+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:46.694 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:46.694 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 20, 2024-09-21T21:41:46.694 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 600, 2024-09-21T21:41:46.694 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:41:46.694 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.694 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4824.0:1564 3.2 3:4647520c:::rbd_data.1121b97f2925.0000000000000006:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:46.694 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:41:46.694 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 45.055437073, 2024-09-21T21:41:46.695 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056097347999999998, 2024-09-21T21:41:46.695 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:46.695 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:46.695 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:46.695 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:46.695 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4824", 2024-09-21T21:41:46.695 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:46.695 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1564 2024-09-21T21:41:46.695 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.695 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:46.695 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.695 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:46.696 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:41:46.696 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:46.696 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.696 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.696 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:46.696 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:41:46.696 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:46.696 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.696 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.696 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:46.696 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:41:46.696 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6499999999999999e-07 2024-09-21T21:41:46.696 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.696 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.697 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:46.697 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636067+0000", 2024-09-21T21:41:46.697 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3879999999999998e-06 2024-09-21T21:41:46.697 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.697 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.697 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:46.697 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636075+0000", 2024-09-21T21:41:46.697 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5480000000000004e-06 2024-09-21T21:41:46.697 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.697 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.697 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:46.697 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636080+0000", 2024-09-21T21:41:46.697 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.339e-06 2024-09-21T21:41:46.697 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.698 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.699 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:46.699 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636133+0000", 2024-09-21T21:41:46.699 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2930000000000003e-05 2024-09-21T21:41:46.699 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.699 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.699 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:46.699 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636265+0000", 2024-09-21T21:41:46.699 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000132058 2024-09-21T21:41:46.699 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.700 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.700 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:46.700 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.692162+0000", 2024-09-21T21:41:46.700 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055896620000000001 2024-09-21T21:41:46.700 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:46.700 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:46.700 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:46.700 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.700 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.700 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4912.0:871 3.19 3:9ddcaed2:::rbd_data.1121b3a54d7e.0000000000000000:12 [sparse-read 16121856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-21T21:41:46.700 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:20.721293+0000", 2024-09-21T21:41:46.700 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 25.970209118, 2024-09-21T21:41:46.700 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.026485022, 2024-09-21T21:41:46.700 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:46.701 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:46.701 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:46.701 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:46.701 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4912", 2024-09-21T21:41:46.701 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4204709190", 2024-09-21T21:41:46.701 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 871 2024-09-21T21:41:46.701 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.701 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:46.701 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.701 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:46.701 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.721293+0000", 2024-09-21T21:41:46.701 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:46.701 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.702 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.702 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:46.702 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.721293+0000", 2024-09-21T21:41:46.702 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:46.702 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.702 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.702 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:46.702 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.721293+0000", 2024-09-21T21:41:46.702 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3000000000000002e-07 2024-09-21T21:41:46.702 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.702 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.702 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:46.702 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.721295+0000", 2024-09-21T21:41:46.702 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.7749999999999999e-06 2024-09-21T21:41:46.703 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.703 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.703 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:46.703 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.721300+0000", 2024-09-21T21:41:46.703 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3319999999999999e-06 2024-09-21T21:41:46.703 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.703 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.703 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:46.703 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.721305+0000", 2024-09-21T21:41:46.703 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.578e-06 2024-09-21T21:41:46.703 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.703 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.703 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:46.703 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.747610+0000", 2024-09-21T21:41:46.704 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.026305469000000001 2024-09-21T21:41:46.704 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.704 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.704 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:46.704 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.747691+0000", 2024-09-21T21:41:46.704 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.1390999999999994e-05 2024-09-21T21:41:46.704 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.704 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.704 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:46.704 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.747778+0000", 2024-09-21T21:41:46.704 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.6147e-05 2024-09-21T21:41:46.704 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:46.704 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:46.704 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:46.705 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.705 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.705 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4912.0:938 3.19 3:9ddcaed2:::rbd_data.1121b3a54d7e.0000000000000000:12 [sparse-read 12845056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-21T21:41:46.705 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:20.723322+0000", 2024-09-21T21:41:46.705 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 25.968179506999999, 2024-09-21T21:41:46.705 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.026387079000000001, 2024-09-21T21:41:46.705 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:46.705 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:46.705 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:46.705 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:46.705 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4912", 2024-09-21T21:41:46.705 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4204709190", 2024-09-21T21:41:46.705 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 938 2024-09-21T21:41:46.705 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.705 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:46.706 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.706 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:46.706 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723322+0000", 2024-09-21T21:41:46.706 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:46.706 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.706 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.706 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:46.706 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723322+0000", 2024-09-21T21:41:46.706 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:46.706 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.706 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.706 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:46.706 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723322+0000", 2024-09-21T21:41:46.706 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.72e-07 2024-09-21T21:41:46.707 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.707 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.707 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:46.707 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723324+0000", 2024-09-21T21:41:46.707 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9250000000000002e-06 2024-09-21T21:41:46.707 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.707 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.708 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:46.708 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723330+0000", 2024-09-21T21:41:46.709 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6799999999999998e-06 2024-09-21T21:41:46.709 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.709 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.709 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:46.709 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723334+0000", 2024-09-21T21:41:46.709 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3640000000000001e-06 2024-09-21T21:41:46.709 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.709 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.709 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:46.709 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.749545+0000", 2024-09-21T21:41:46.709 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.026210715999999998 2024-09-21T21:41:46.709 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.710 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.710 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:46.710 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.749623+0000", 2024-09-21T21:41:46.710 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7602999999999995e-05 2024-09-21T21:41:46.710 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.710 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.710 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:46.710 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.749709+0000", 2024-09-21T21:41:46.710 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.6419e-05 2024-09-21T21:41:46.710 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:46.710 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:46.711 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:46.711 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.711 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.711 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4912.0:939 3.19 3:9ddcaed2:::rbd_data.1121b3a54d7e.0000000000000000:12 [sparse-read 14548992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-21T21:41:46.711 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:20.723340+0000", 2024-09-21T21:41:46.711 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 25.968161308999999, 2024-09-21T21:41:46.711 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.026505974000000002, 2024-09-21T21:41:46.711 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:46.711 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:46.711 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:46.711 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:46.712 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4912", 2024-09-21T21:41:46.712 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4204709190", 2024-09-21T21:41:46.712 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 939 2024-09-21T21:41:46.712 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.712 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:46.712 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.712 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:46.712 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723340+0000", 2024-09-21T21:41:46.712 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:46.712 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.712 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.712 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:46.712 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723340+0000", 2024-09-21T21:41:46.712 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:46.712 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.713 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.713 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:46.713 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723341+0000", 2024-09-21T21:41:46.713 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6100000000000002e-07 2024-09-21T21:41:46.713 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.713 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.713 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:46.713 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723342+0000", 2024-09-21T21:41:46.713 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.782e-06 2024-09-21T21:41:46.713 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.713 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.713 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:46.714 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723348+0000", 2024-09-21T21:41:46.714 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3990000000000003e-06 2024-09-21T21:41:46.714 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.714 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.714 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:46.714 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723353+0000", 2024-09-21T21:41:46.714 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3009999999999997e-06 2024-09-21T21:41:46.714 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.714 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.714 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:46.714 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.749741+0000", 2024-09-21T21:41:46.714 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.026387911999999999 2024-09-21T21:41:46.714 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.715 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.715 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:46.715 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.749788+0000", 2024-09-21T21:41:46.715 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7354000000000001e-05 2024-09-21T21:41:46.715 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.715 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.715 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:46.715 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.749846+0000", 2024-09-21T21:41:46.715 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7865000000000002e-05 2024-09-21T21:41:46.715 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:46.715 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:46.716 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:46.716 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.716 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.716 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4912.0:940 3.19 3:9ddcaed2:::rbd_data.1121b3a54d7e.0000000000000000:12 [sparse-read 16252928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-21T21:41:46.716 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:20.723360+0000", 2024-09-21T21:41:46.716 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 25.968142094000001, 2024-09-21T21:41:46.716 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.026697166000000001, 2024-09-21T21:41:46.716 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:46.716 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:46.716 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:46.716 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:46.716 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4912", 2024-09-21T21:41:46.716 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4204709190", 2024-09-21T21:41:46.717 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 940 2024-09-21T21:41:46.717 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.717 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:46.717 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.717 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:46.717 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723360+0000", 2024-09-21T21:41:46.717 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:46.717 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.717 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.717 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:46.717 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723360+0000", 2024-09-21T21:41:46.717 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:46.717 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.718 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.718 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:46.718 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723360+0000", 2024-09-21T21:41:46.718 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.9900000000000001e-07 2024-09-21T21:41:46.718 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.718 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.718 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:46.718 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723362+0000", 2024-09-21T21:41:46.718 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1289999999999999e-06 2024-09-21T21:41:46.718 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.718 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.718 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:46.718 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723368+0000", 2024-09-21T21:41:46.719 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.75e-06 2024-09-21T21:41:46.719 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.719 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.719 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:46.719 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723373+0000", 2024-09-21T21:41:46.719 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6959999999999996e-06 2024-09-21T21:41:46.719 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.719 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.719 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:46.719 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.749882+0000", 2024-09-21T21:41:46.719 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.026509893999999999 2024-09-21T21:41:46.719 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.720 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.720 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:46.720 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.749966+0000", 2024-09-21T21:41:46.720 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.3449999999999996e-05 2024-09-21T21:41:46.720 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.720 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.721 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:46.721 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.750057+0000", 2024-09-21T21:41:46.722 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.0847999999999995e-05 2024-09-21T21:41:46.722 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:46.722 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:46.722 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:46.722 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.722 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.722 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4912.0:999 3.19 3:9ddcaed2:::rbd_data.1121b3a54d7e.0000000000000000:12 [sparse-read 1441792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-21T21:41:46.722 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:20.723784+0000", 2024-09-21T21:41:46.722 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 25.967717154999999, 2024-09-21T21:41:46.722 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.026500717, 2024-09-21T21:41:46.722 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:46.722 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:46.723 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:46.723 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:46.723 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4912", 2024-09-21T21:41:46.723 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4204709190", 2024-09-21T21:41:46.723 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 999 2024-09-21T21:41:46.723 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.723 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:46.723 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.723 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:46.723 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723784+0000", 2024-09-21T21:41:46.723 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:46.723 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.723 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.723 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:46.724 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723784+0000", 2024-09-21T21:41:46.724 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:46.724 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.724 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.724 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:46.724 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723785+0000", 2024-09-21T21:41:46.724 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4900000000000001e-07 2024-09-21T21:41:46.724 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.724 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.724 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:46.724 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723787+0000", 2024-09-21T21:41:46.724 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9819999999999998e-06 2024-09-21T21:41:46.724 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.724 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.724 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:46.724 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723793+0000", 2024-09-21T21:41:46.725 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.032e-06 2024-09-21T21:41:46.725 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.725 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.725 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:46.725 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723798+0000", 2024-09-21T21:41:46.725 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7099999999999998e-06 2024-09-21T21:41:46.725 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.725 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.725 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:46.725 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.750104+0000", 2024-09-21T21:41:46.726 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.02630658 2024-09-21T21:41:46.726 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.726 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.726 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:46.726 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.750185+0000", 2024-09-21T21:41:46.726 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0345000000000004e-05 2024-09-21T21:41:46.726 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.726 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.726 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:46.726 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.750285+0000", 2024-09-21T21:41:46.726 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010061899999999999 2024-09-21T21:41:46.727 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:46.727 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:46.727 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:46.727 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.727 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.727 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4912.0:1000 3.19 3:9ddcaed2:::rbd_data.1121b3a54d7e.0000000000000000:12 [sparse-read 3145728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-21T21:41:46.727 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:20.723804+0000", 2024-09-21T21:41:46.727 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 25.967697973, 2024-09-21T21:41:46.728 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.026694361, 2024-09-21T21:41:46.728 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:46.728 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:46.728 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:46.728 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:46.728 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4912", 2024-09-21T21:41:46.728 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4204709190", 2024-09-21T21:41:46.728 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1000 2024-09-21T21:41:46.728 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.728 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:46.728 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.728 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:46.728 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723804+0000", 2024-09-21T21:41:46.728 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:46.729 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.729 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.729 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:46.729 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723804+0000", 2024-09-21T21:41:46.729 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:46.729 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.729 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.729 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:46.729 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723804+0000", 2024-09-21T21:41:46.729 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7399999999999999e-07 2024-09-21T21:41:46.729 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.729 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.730 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:46.730 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723811+0000", 2024-09-21T21:41:46.730 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.1149999999999996e-06 2024-09-21T21:41:46.730 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.730 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.730 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:46.730 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723818+0000", 2024-09-21T21:41:46.730 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9779999999999999e-06 2024-09-21T21:41:46.730 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.730 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.730 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:46.730 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723826+0000", 2024-09-21T21:41:46.731 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0600000000000008e-06 2024-09-21T21:41:46.731 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.731 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.731 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:46.731 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.750308+0000", 2024-09-21T21:41:46.731 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.026482316999999998 2024-09-21T21:41:46.732 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.732 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.732 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:46.732 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.750403+0000", 2024-09-21T21:41:46.732 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.4775e-05 2024-09-21T21:41:46.732 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.732 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.732 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:46.732 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.750498+0000", 2024-09-21T21:41:46.732 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.4741999999999995e-05 2024-09-21T21:41:46.732 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:46.732 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:46.733 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:46.733 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.733 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.733 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4912.0:1001 3.19 3:9ddcaed2:::rbd_data.1121b3a54d7e.0000000000000000:12 [sparse-read 4849664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-21T21:41:46.733 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:20.723833+0000", 2024-09-21T21:41:46.733 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 25.967669116, 2024-09-21T21:41:46.733 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.026886792999999999, 2024-09-21T21:41:46.733 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:46.733 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:46.733 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:46.733 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:46.733 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4912", 2024-09-21T21:41:46.733 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4204709190", 2024-09-21T21:41:46.733 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1001 2024-09-21T21:41:46.734 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.734 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:46.734 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.734 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:46.734 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723833+0000", 2024-09-21T21:41:46.734 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:46.734 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.734 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.734 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:46.734 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723833+0000", 2024-09-21T21:41:46.734 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:46.734 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.734 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.734 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:46.735 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723833+0000", 2024-09-21T21:41:46.735 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4099999999999999e-07 2024-09-21T21:41:46.735 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.735 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.735 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:46.735 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723835+0000", 2024-09-21T21:41:46.735 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.968e-06 2024-09-21T21:41:46.735 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.735 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.735 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:46.735 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723841+0000", 2024-09-21T21:41:46.735 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5209999999999999e-06 2024-09-21T21:41:46.735 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.735 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.735 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:46.736 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723846+0000", 2024-09-21T21:41:46.736 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9810000000000003e-06 2024-09-21T21:41:46.736 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.736 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.736 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:46.736 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.750538+0000", 2024-09-21T21:41:46.736 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.026692041 2024-09-21T21:41:46.736 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.736 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.736 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:46.736 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.750625+0000", 2024-09-21T21:41:46.736 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.6706000000000006e-05 2024-09-21T21:41:46.736 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.736 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.736 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:46.737 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.750719+0000", 2024-09-21T21:41:46.737 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.4135000000000006e-05 2024-09-21T21:41:46.737 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:46.737 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:46.737 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:46.737 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.737 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.737 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4912.0:1002 3.19 3:9ddcaed2:::rbd_data.1121b3a54d7e.0000000000000000:12 [sparse-read 6553600~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-21T21:41:46.737 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:20.723853+0000", 2024-09-21T21:41:46.737 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 25.967649012999999, 2024-09-21T21:41:46.737 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.027244385999999999, 2024-09-21T21:41:46.737 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:46.737 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:46.737 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:46.737 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:46.738 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4912", 2024-09-21T21:41:46.738 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4204709190", 2024-09-21T21:41:46.738 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1002 2024-09-21T21:41:46.738 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.738 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:46.738 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.738 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:46.738 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723853+0000", 2024-09-21T21:41:46.738 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:46.738 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.738 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.738 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:46.738 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723853+0000", 2024-09-21T21:41:46.738 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:46.739 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.739 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.739 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:46.739 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723853+0000", 2024-09-21T21:41:46.739 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6899999999999998e-07 2024-09-21T21:41:46.739 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.739 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.739 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:46.739 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723858+0000", 2024-09-21T21:41:46.739 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0710000000000002e-06 2024-09-21T21:41:46.739 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.739 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.739 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:46.739 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723865+0000", 2024-09-21T21:41:46.739 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.613e-06 2024-09-21T21:41:46.740 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.740 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.740 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:46.740 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723870+0000", 2024-09-21T21:41:46.740 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0350000000000001e-06 2024-09-21T21:41:46.740 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.740 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.740 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:46.740 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.750749+0000", 2024-09-21T21:41:46.740 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.026878808000000001 2024-09-21T21:41:46.740 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.740 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.740 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:46.740 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.750830+0000", 2024-09-21T21:41:46.740 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.1817999999999995e-05 2024-09-21T21:41:46.741 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.741 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.741 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:46.741 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.751097+0000", 2024-09-21T21:41:46.741 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00026667200000000002 2024-09-21T21:41:46.741 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:46.741 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:46.741 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:46.741 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.741 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.741 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4912.0:1003 3.19 3:9ddcaed2:::rbd_data.1121b3a54d7e.0000000000000000:12 [sparse-read 8257536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-21T21:41:46.741 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:20.723876+0000", 2024-09-21T21:41:46.741 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 25.967625236, 2024-09-21T21:41:46.741 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.027396406000000002, 2024-09-21T21:41:46.742 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:46.742 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:46.742 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:46.742 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:46.742 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4912", 2024-09-21T21:41:46.742 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4204709190", 2024-09-21T21:41:46.742 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1003 2024-09-21T21:41:46.742 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.742 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:46.742 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.742 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:46.742 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723876+0000", 2024-09-21T21:41:46.742 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:46.742 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.742 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.743 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:46.743 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723876+0000", 2024-09-21T21:41:46.743 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:46.743 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.743 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.743 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:46.743 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723877+0000", 2024-09-21T21:41:46.743 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5999999999999999e-07 2024-09-21T21:41:46.743 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.743 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.743 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:46.743 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723879+0000", 2024-09-21T21:41:46.743 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1299999999999999e-06 2024-09-21T21:41:46.744 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.744 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.744 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:46.744 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723885+0000", 2024-09-21T21:41:46.744 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9580000000000004e-06 2024-09-21T21:41:46.744 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.744 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.744 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:46.744 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723889+0000", 2024-09-21T21:41:46.744 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4850000000000003e-06 2024-09-21T21:41:46.744 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.744 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.744 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:46.744 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.751125+0000", 2024-09-21T21:41:46.745 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.027236136000000001 2024-09-21T21:41:46.745 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.745 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.745 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:46.745 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.751171+0000", 2024-09-21T21:41:46.745 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5708999999999999e-05 2024-09-21T21:41:46.745 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.745 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.745 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:46.745 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.751273+0000", 2024-09-21T21:41:46.745 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000101628 2024-09-21T21:41:46.745 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:46.745 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:46.745 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:46.745 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.746 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.746 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4912.0:1004 3.19 3:9ddcaed2:::rbd_data.1121b3a54d7e.0000000000000000:12 [sparse-read 9568256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-21T21:41:46.746 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:20.723895+0000", 2024-09-21T21:41:46.746 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 25.967606527000001, 2024-09-21T21:41:46.746 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.027611696000000002, 2024-09-21T21:41:46.746 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:46.746 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:46.746 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:46.746 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:46.746 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4912", 2024-09-21T21:41:46.746 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4204709190", 2024-09-21T21:41:46.746 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1004 2024-09-21T21:41:46.746 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.747 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:46.747 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.747 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:46.747 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723895+0000", 2024-09-21T21:41:46.747 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:46.747 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.747 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.747 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:46.747 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723895+0000", 2024-09-21T21:41:46.747 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:46.747 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.747 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.747 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:46.747 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723895+0000", 2024-09-21T21:41:46.747 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.1199999999999999e-07 2024-09-21T21:41:46.748 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.748 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.748 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:46.748 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723897+0000", 2024-09-21T21:41:46.748 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.8059999999999999e-06 2024-09-21T21:41:46.748 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.748 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.748 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:46.748 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723903+0000", 2024-09-21T21:41:46.748 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9939999999999997e-06 2024-09-21T21:41:46.748 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.748 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.748 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:46.748 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723908+0000", 2024-09-21T21:41:46.749 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5260000000000004e-06 2024-09-21T21:41:46.749 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.749 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.749 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:46.749 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.751316+0000", 2024-09-21T21:41:46.749 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.027408377000000001 2024-09-21T21:41:46.749 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.749 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.749 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:46.749 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.751402+0000", 2024-09-21T21:41:46.749 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.5830999999999999e-05 2024-09-21T21:41:46.749 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.749 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.749 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:46.750 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.751507+0000", 2024-09-21T21:41:46.750 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010485 2024-09-21T21:41:46.750 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:46.750 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:46.750 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:46.750 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.750 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.750 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4912.0:1005 3.19 3:9ddcaed2:::rbd_data.1121b3a54d7e.0000000000000000:12 [sparse-read 11272192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-21T21:41:46.750 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:20.723914+0000", 2024-09-21T21:41:46.750 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 25.967587756, 2024-09-21T21:41:46.750 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.027789695999999999, 2024-09-21T21:41:46.750 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:46.750 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:46.750 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:46.751 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:46.751 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4912", 2024-09-21T21:41:46.751 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4204709190", 2024-09-21T21:41:46.751 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1005 2024-09-21T21:41:46.751 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.751 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:46.751 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.751 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:46.751 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723914+0000", 2024-09-21T21:41:46.751 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:46.751 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.751 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.751 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:46.751 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723914+0000", 2024-09-21T21:41:46.751 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:46.752 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.752 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.752 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:46.752 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723914+0000", 2024-09-21T21:41:46.752 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4099999999999999e-07 2024-09-21T21:41:46.752 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.752 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.752 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:46.752 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723916+0000", 2024-09-21T21:41:46.752 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0559999999999999e-06 2024-09-21T21:41:46.752 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.752 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.752 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:46.752 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723923+0000", 2024-09-21T21:41:46.752 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.139e-06 2024-09-21T21:41:46.753 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.753 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.753 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:46.753 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723927+0000", 2024-09-21T21:41:46.753 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.826e-06 2024-09-21T21:41:46.753 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.753 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.753 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:46.753 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.751550+0000", 2024-09-21T21:41:46.753 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.027622357 2024-09-21T21:41:46.753 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.753 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.753 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:46.753 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.751636+0000", 2024-09-21T21:41:46.754 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.6604e-05 2024-09-21T21:41:46.754 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.754 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.754 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:46.754 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.751704+0000", 2024-09-21T21:41:46.754 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.7173000000000002e-05 2024-09-21T21:41:46.754 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:46.754 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:46.754 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:46.754 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.754 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.754 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4912.0:1006 3.19 3:9ddcaed2:::rbd_data.1121b3a54d7e.0000000000000000:12 [sparse-read 12976128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-21T21:41:46.754 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:20.723939+0000", 2024-09-21T21:41:46.754 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 25.967562274999999, 2024-09-21T21:41:46.755 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.028010094999999999, 2024-09-21T21:41:46.755 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:46.755 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:46.755 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:46.755 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:46.755 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4912", 2024-09-21T21:41:46.755 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4204709190", 2024-09-21T21:41:46.755 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1006 2024-09-21T21:41:46.755 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.755 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:46.755 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.755 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:46.755 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723939+0000", 2024-09-21T21:41:46.755 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:46.755 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.756 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.756 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:46.756 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723939+0000", 2024-09-21T21:41:46.756 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:46.756 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.756 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.756 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:46.756 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723940+0000", 2024-09-21T21:41:46.756 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4999999999999998e-07 2024-09-21T21:41:46.756 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.756 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.756 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:46.756 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723942+0000", 2024-09-21T21:41:46.756 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1660000000000001e-06 2024-09-21T21:41:46.756 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.757 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.757 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:46.757 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723949+0000", 2024-09-21T21:41:46.757 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0949999999999998e-06 2024-09-21T21:41:46.757 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.757 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.757 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:46.757 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723954+0000", 2024-09-21T21:41:46.757 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0250000000000002e-06 2024-09-21T21:41:46.757 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.757 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.757 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:46.757 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.751755+0000", 2024-09-21T21:41:46.757 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.027801228000000001 2024-09-21T21:41:46.758 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.758 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.758 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:46.758 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.751841+0000", 2024-09-21T21:41:46.758 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.5829999999999996e-05 2024-09-21T21:41:46.758 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.758 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.758 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:46.758 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.751949+0000", 2024-09-21T21:41:46.758 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010830100000000001 2024-09-21T21:41:46.758 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:46.758 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:46.758 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:46.758 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.759 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.759 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4912.0:1007 3.19 3:9ddcaed2:::rbd_data.1121b3a54d7e.0000000000000000:12 [sparse-read 14680064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-21T21:41:46.759 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:20.723961+0000", 2024-09-21T21:41:46.759 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 25.967541099999998, 2024-09-21T21:41:46.759 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.028245155000000001, 2024-09-21T21:41:46.759 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:46.759 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:46.759 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:46.759 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:46.759 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4912", 2024-09-21T21:41:46.759 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4204709190", 2024-09-21T21:41:46.759 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1007 2024-09-21T21:41:46.759 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.760 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:46.760 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.760 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:46.760 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723961+0000", 2024-09-21T21:41:46.760 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:46.760 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.760 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.760 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:46.760 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723961+0000", 2024-09-21T21:41:46.760 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:46.760 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.760 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.760 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:46.760 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723961+0000", 2024-09-21T21:41:46.760 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4900000000000001e-07 2024-09-21T21:41:46.761 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.761 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.761 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:46.761 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723963+0000", 2024-09-21T21:41:46.761 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.787e-06 2024-09-21T21:41:46.761 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.761 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.761 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:46.761 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723969+0000", 2024-09-21T21:41:46.761 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8289999999999996e-06 2024-09-21T21:41:46.761 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.761 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.761 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:46.761 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723973+0000", 2024-09-21T21:41:46.762 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5499999999999996e-06 2024-09-21T21:41:46.762 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.762 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.762 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:46.762 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.751994+0000", 2024-09-21T21:41:46.762 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.028021146 2024-09-21T21:41:46.762 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.762 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.762 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:46.762 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.752101+0000", 2024-09-21T21:41:46.762 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010621899999999999 2024-09-21T21:41:46.762 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.762 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.762 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:46.763 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.752206+0000", 2024-09-21T21:41:46.763 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000105175 2024-09-21T21:41:46.763 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:46.763 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:46.763 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:46.763 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.763 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.763 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4912.0:1008 3.19 3:9ddcaed2:::rbd_data.1121b3a54d7e.0000000000000000:12 [sparse-read 16384000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-21T21:41:46.763 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:20.723978+0000", 2024-09-21T21:41:46.763 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 25.967523279000002, 2024-09-21T21:41:46.763 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.028444661999999999, 2024-09-21T21:41:46.763 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:46.763 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:46.763 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:46.764 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:46.764 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4912", 2024-09-21T21:41:46.764 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/4204709190", 2024-09-21T21:41:46.764 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1008 2024-09-21T21:41:46.764 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.764 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:46.764 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.764 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:46.764 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723978+0000", 2024-09-21T21:41:46.764 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:46.764 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.764 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.764 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:46.764 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.723978+0000", 2024-09-21T21:41:46.764 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:46.765 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.765 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.765 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:46.765 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.724409+0000", 2024-09-21T21:41:46.765 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00043058300000000001 2024-09-21T21:41:46.765 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.765 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.765 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:46.765 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.724412+0000", 2024-09-21T21:41:46.765 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3730000000000001e-06 2024-09-21T21:41:46.765 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.765 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.765 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:46.765 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.724421+0000", 2024-09-21T21:41:46.766 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.9840000000000007e-06 2024-09-21T21:41:46.766 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.766 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.766 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:46.766 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.724427+0000", 2024-09-21T21:41:46.766 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2029999999999997e-06 2024-09-21T21:41:46.766 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.766 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.766 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:46.766 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.752238+0000", 2024-09-21T21:41:46.766 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.027810577999999999 2024-09-21T21:41:46.766 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.766 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.766 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:46.767 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.752323+0000", 2024-09-21T21:41:46.767 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.4652000000000002e-05 2024-09-21T21:41:46.767 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.767 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.767 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:46.767 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:20.752423+0000", 2024-09-21T21:41:46.767 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000100289 2024-09-21T21:41:46.767 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:46.767 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:46.767 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:46.767 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.767 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.767 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5196.0:984 3.2 3:404ccd4e:::rbd_data.1121a3eabc61.0000000000000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-21T21:41:46.767 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:27.956266+0000", 2024-09-21T21:41:46.767 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 18.735235460999998, 2024-09-21T21:41:46.768 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.032332125000000003, 2024-09-21T21:41:46.768 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:46.768 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:46.768 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:41:46.768 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:46.768 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5196", 2024-09-21T21:41:46.768 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3707540331", 2024-09-21T21:41:46.768 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 984 2024-09-21T21:41:46.768 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.768 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:46.768 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.768 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:46.768 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.956266+0000", 2024-09-21T21:41:46.768 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:46.768 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.769 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.769 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:46.769 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.956266+0000", 2024-09-21T21:41:46.769 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:46.769 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.769 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.769 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:46.769 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.956267+0000", 2024-09-21T21:41:46.769 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0800000000000004e-07 2024-09-21T21:41:46.769 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.769 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.769 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:46.769 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.956271+0000", 2024-09-21T21:41:46.769 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.0199999999999996e-06 2024-09-21T21:41:46.770 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.770 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.770 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:46.770 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.956281+0000", 2024-09-21T21:41:46.770 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0669000000000001e-05 2024-09-21T21:41:46.770 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.770 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.770 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:46.770 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.956288+0000", 2024-09-21T21:41:46.770 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9639999999999997e-06 2024-09-21T21:41:46.770 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.770 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.770 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:46.770 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.956337+0000", 2024-09-21T21:41:46.771 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8538999999999997e-05 2024-09-21T21:41:46.771 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.771 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.771 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:46.771 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.956413+0000", 2024-09-21T21:41:46.771 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5652e-05 2024-09-21T21:41:46.771 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.771 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.771 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:41:46.771 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.956519+0000", 2024-09-21T21:41:46.771 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010645 2024-09-21T21:41:46.771 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.771 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.771 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:41:46.772 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.988330+0000", 2024-09-21T21:41:46.772 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031810488999999997 2024-09-21T21:41:46.772 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.772 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.772 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:41:46.772 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.988532+0000", 2024-09-21T21:41:46.772 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00020196999999999999 2024-09-21T21:41:46.772 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.772 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.772 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:41:46.772 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.988563+0000", 2024-09-21T21:41:46.772 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.1951e-05 2024-09-21T21:41:46.772 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.772 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.772 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:46.773 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.988598+0000", 2024-09-21T21:41:46.773 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4813e-05 2024-09-21T21:41:46.773 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:46.773 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:46.773 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:46.773 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.773 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.773 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5196.0:991 3.7 3:e594ca4e:::rbd_header.1121a3eabc61:head [notify cookie 139700387993488] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-21T21:41:46.773 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:27.956339+0000", 2024-09-21T21:41:46.773 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 18.735162234000001, 2024-09-21T21:41:46.773 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031791316, 2024-09-21T21:41:46.773 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:46.773 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:46.773 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:46.774 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:46.774 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5196", 2024-09-21T21:41:46.774 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3707540331", 2024-09-21T21:41:46.774 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 991 2024-09-21T21:41:46.774 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.774 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:46.774 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.774 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:46.774 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.956339+0000", 2024-09-21T21:41:46.774 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:46.774 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.774 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.774 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:46.774 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.956339+0000", 2024-09-21T21:41:46.775 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:46.775 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.775 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.775 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:46.775 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.956340+0000", 2024-09-21T21:41:46.775 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7999999999999996e-07 2024-09-21T21:41:46.775 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.775 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.775 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:46.775 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.956343+0000", 2024-09-21T21:41:46.775 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.8260000000000001e-06 2024-09-21T21:41:46.775 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.775 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.775 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:46.776 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.956350+0000", 2024-09-21T21:41:46.776 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.1019999999999999e-06 2024-09-21T21:41:46.776 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.776 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.776 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:46.776 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.956355+0000", 2024-09-21T21:41:46.776 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6149999999999996e-06 2024-09-21T21:41:46.776 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.776 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.776 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:46.776 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.956507+0000", 2024-09-21T21:41:46.776 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000151963 2024-09-21T21:41:46.776 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.776 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.776 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:46.777 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.988010+0000", 2024-09-21T21:41:46.777 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031502636000000001 2024-09-21T21:41:46.777 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.777 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.777 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:46.777 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.988131+0000", 2024-09-21T21:41:46.777 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00012069399999999999 2024-09-21T21:41:46.777 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:46.777 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:46.777 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:46.777 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.777 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.777 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5196.0:992 3.7 3:e594ca4e:::rbd_header.1121a3eabc61:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-21T21:41:46.777 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:27.956555+0000", 2024-09-21T21:41:46.778 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 18.734947006999999, 2024-09-21T21:41:46.778 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031712787999999999, 2024-09-21T21:41:46.778 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:46.778 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:46.778 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:46.778 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:46.778 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5196", 2024-09-21T21:41:46.778 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3707540331", 2024-09-21T21:41:46.778 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 992 2024-09-21T21:41:46.778 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.778 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:46.778 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.778 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:46.778 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.956555+0000", 2024-09-21T21:41:46.779 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:46.779 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.779 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.779 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:46.779 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.956555+0000", 2024-09-21T21:41:46.779 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:46.779 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.779 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.779 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:46.779 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.956555+0000", 2024-09-21T21:41:46.779 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.2900000000000002e-07 2024-09-21T21:41:46.779 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.779 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.779 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:46.779 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.956561+0000", 2024-09-21T21:41:46.780 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1699999999999996e-06 2024-09-21T21:41:46.780 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.780 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.780 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:46.780 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.987985+0000", 2024-09-21T21:41:46.780 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031424872 2024-09-21T21:41:46.780 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.780 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.780 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:46.780 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.987996+0000", 2024-09-21T21:41:46.780 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0742e-05 2024-09-21T21:41:46.780 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.780 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.780 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:46.781 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.988161+0000", 2024-09-21T21:41:46.781 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00016449800000000001 2024-09-21T21:41:46.781 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.781 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.781 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:46.781 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.988214+0000", 2024-09-21T21:41:46.781 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3528e-05 2024-09-21T21:41:46.781 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.781 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.781 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:46.781 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.988267+0000", 2024-09-21T21:41:46.781 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3149000000000002e-05 2024-09-21T21:41:46.781 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:46.781 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:46.781 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:46.782 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.782 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.782 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4385.0:11863 3.7 3:e594ca4e:::rbd_header.1121a3eabc61:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-21T21:41:46.782 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:27.956588+0000", 2024-09-21T21:41:46.782 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 18.734913369000001, 2024-09-21T21:41:46.782 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031871075999999998, 2024-09-21T21:41:46.782 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:46.782 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:46.782 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:46.782 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:46.782 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4385", 2024-09-21T21:41:46.782 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:41:46.782 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11863 2024-09-21T21:41:46.782 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.783 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:46.783 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.783 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:46.783 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.956588+0000", 2024-09-21T21:41:46.783 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:46.783 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.783 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.783 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:46.783 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.956588+0000", 2024-09-21T21:41:46.783 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:46.783 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.783 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.783 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:46.783 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.956589+0000", 2024-09-21T21:41:46.784 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.1699999999999997e-07 2024-09-21T21:41:46.784 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.784 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.784 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:46.784 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.956593+0000", 2024-09-21T21:41:46.784 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1219999999999997e-06 2024-09-21T21:41:46.784 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.784 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.784 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:46.784 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.987976+0000", 2024-09-21T21:41:46.784 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031382366000000002 2024-09-21T21:41:46.784 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.784 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.784 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:46.785 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.987996+0000", 2024-09-21T21:41:46.785 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0010999999999999e-05 2024-09-21T21:41:46.785 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.785 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.785 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:46.785 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.988296+0000", 2024-09-21T21:41:46.785 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00030068500000000003 2024-09-21T21:41:46.785 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.785 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.785 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:46.785 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.988365+0000", 2024-09-21T21:41:46.785 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.8986999999999995e-05 2024-09-21T21:41:46.785 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.785 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.785 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:46.786 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.988459+0000", 2024-09-21T21:41:46.786 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.4087999999999998e-05 2024-09-21T21:41:46.786 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:46.786 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:46.786 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:46.786 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.786 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.786 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_repop_reply(client.5196.0:984 3.2 e58/21 ondisk, result = 0)", 2024-09-21T21:41:46.786 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:27.958905+0000", 2024-09-21T21:41:46.786 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 18.732596261000001, 2024-09-21T21:41:46.786 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.029433239, 2024-09-21T21:41:46.786 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:46.786 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:46.786 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:46.787 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:46.787 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.787 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:46.787 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.958905+0000", 2024-09-21T21:41:46.787 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:46.787 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.787 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.787 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:46.787 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.958905+0000", 2024-09-21T21:41:46.787 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:46.787 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.787 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.787 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:46.787 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.958906+0000", 2024-09-21T21:41:46.787 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.8199999999999999e-07 2024-09-21T21:41:46.788 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.788 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.788 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:46.788 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.958914+0000", 2024-09-21T21:41:46.788 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0180000000000001e-06 2024-09-21T21:41:46.788 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.788 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.788 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:46.788 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.987982+0000", 2024-09-21T21:41:46.788 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.029067829999999999 2024-09-21T21:41:46.788 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.788 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.788 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:46.788 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.987995+0000", 2024-09-21T21:41:46.788 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.3196999999999999e-05 2024-09-21T21:41:46.789 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.789 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.789 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:46.789 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.988312+0000", 2024-09-21T21:41:46.789 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00031657400000000002 2024-09-21T21:41:46.789 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.789 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.789 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:46.789 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.988323+0000", 2024-09-21T21:41:46.789 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1344e-05 2024-09-21T21:41:46.789 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:46.789 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:46.789 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:46.789 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:27.988339+0000", 2024-09-21T21:41:46.789 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.5594e-05 2024-09-21T21:41:46.790 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:46.790 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:46.790 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:46.790 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:46.790 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:46.790 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:46.791 INFO:teuthology.orchestra.run.smithi164.stdout:checking clone #28, image image_client.0-clone28 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent29 2024-09-21T21:41:46.792 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T21:41:46.792 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:46.789+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:46.803 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 5] out_osds: [3] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-21T21:41:46.803 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T21:41:46.804 INFO:tasks.thrashosds.thrasher:Adding osd 3 2024-09-21T21:41:46.804 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 3 2024-09-21T21:41:46.892 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:46.890+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:46.944 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:46.944 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:41:46.944 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 0 2024-09-21T21:41:46.944 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:46.953 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T21:41:46.977 INFO:teuthology.orchestra.run.smithi164.stdout:3889 trunc from 0xc0a8200 to 0x5802600 2024-09-21T21:41:46.993 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:46.991+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:47.094 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:47.093+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:47.136 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:47.136 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:41:47.136 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:41:47.136 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:41:47.137 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:47.146 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T21:41:47.164 INFO:teuthology.orchestra.run.smithi164.stdout:3891 write 0x160c000 thru 0x1614f93 (0x8f94 bytes) 2024-09-21T21:41:47.170 INFO:teuthology.orchestra.run.smithi164.stdout:3893 read 0x3572a00 thru 0x357dbe8 (0xb1e9 bytes) 2024-09-21T21:41:47.172 INFO:teuthology.orchestra.run.smithi164.stdout:3894 write 0x24d000 thru 0x256c49 (0x9c4a bytes) 2024-09-21T21:41:47.173 INFO:teuthology.orchestra.run.smithi164.stdout:3895 read 0x329f800 thru 0x32ad122 (0xd923 bytes) 2024-09-21T21:41:47.173 INFO:teuthology.orchestra.run.smithi164.stdout:3896 punch from 0x3c36000 to 0x3c43e00, (0xde00 bytes) 2024-09-21T21:41:47.195 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:47.194+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:47.234 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T21:41:47.262 INFO:tasks.ceph.mon.a.smithi081.stderr:2024-09-21T21:41:47.260+0000 7f5d67cb4640 -1 mon.a@0(leader).osd e75 definitely_dead 0 2024-09-21T21:41:47.297 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:47.295+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:47.299 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:47.300 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 20, 2024-09-21T21:41:47.300 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 600, 2024-09-21T21:41:47.300 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:41:47.300 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.300 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:1678 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 14483456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:47.300 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.299288+0000", 2024-09-21T21:41:47.301 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 22.997567856, 2024-09-21T21:41:47.301 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031376253, 2024-09-21T21:41:47.301 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:47.301 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:47.301 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:47.301 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:47.302 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:47.302 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:47.302 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1678 2024-09-21T21:41:47.302 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.302 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:47.302 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.302 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:47.302 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299288+0000", 2024-09-21T21:41:47.302 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:47.302 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.302 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.302 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:47.302 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299288+0000", 2024-09-21T21:41:47.303 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:47.303 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.303 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.303 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:47.303 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299288+0000", 2024-09-21T21:41:47.303 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.2500000000000001e-07 2024-09-21T21:41:47.303 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.303 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.303 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:47.303 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299290+0000", 2024-09-21T21:41:47.303 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.945e-06 2024-09-21T21:41:47.303 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.303 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.303 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:47.304 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299295+0000", 2024-09-21T21:41:47.304 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0379999999999999e-06 2024-09-21T21:41:47.304 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.304 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.304 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:47.304 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299299+0000", 2024-09-21T21:41:47.304 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7960000000000002e-06 2024-09-21T21:41:47.304 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.305 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.305 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:47.305 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.330586+0000", 2024-09-21T21:41:47.305 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031287540000000003 2024-09-21T21:41:47.305 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.305 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.305 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:47.305 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.330620+0000", 2024-09-21T21:41:47.305 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3492999999999999e-05 2024-09-21T21:41:47.305 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.305 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.305 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:47.305 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.330664+0000", 2024-09-21T21:41:47.305 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4116000000000002e-05 2024-09-21T21:41:47.306 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:47.306 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:47.306 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:47.306 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.306 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.306 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:1679 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 14942208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:47.306 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.299333+0000", 2024-09-21T21:41:47.306 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 22.997522200999999, 2024-09-21T21:41:47.306 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031433567000000003, 2024-09-21T21:41:47.306 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:47.306 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:47.306 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:47.306 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:47.306 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:47.307 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:47.307 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1679 2024-09-21T21:41:47.307 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.307 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:47.307 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.307 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:47.307 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299333+0000", 2024-09-21T21:41:47.307 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:47.307 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.307 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.307 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:47.307 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299333+0000", 2024-09-21T21:41:47.307 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:47.308 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.308 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.308 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:47.308 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299334+0000", 2024-09-21T21:41:47.308 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0100000000000005e-07 2024-09-21T21:41:47.308 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.308 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.308 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:47.308 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299336+0000", 2024-09-21T21:41:47.308 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3439999999999999e-06 2024-09-21T21:41:47.308 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.308 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.308 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:47.308 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299343+0000", 2024-09-21T21:41:47.309 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0299999999999996e-06 2024-09-21T21:41:47.309 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.309 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.309 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:47.309 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299348+0000", 2024-09-21T21:41:47.309 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2340000000000002e-06 2024-09-21T21:41:47.309 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.309 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.309 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:47.309 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.330687+0000", 2024-09-21T21:41:47.309 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031338132999999997 2024-09-21T21:41:47.309 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.309 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.309 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:47.310 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.330721+0000", 2024-09-21T21:41:47.310 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4156999999999998e-05 2024-09-21T21:41:47.310 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.310 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.310 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:47.310 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.330767+0000", 2024-09-21T21:41:47.310 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6168000000000002e-05 2024-09-21T21:41:47.310 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:47.310 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:47.310 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:47.310 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.310 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.310 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:1680 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 15400960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:47.310 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.299356+0000", 2024-09-21T21:41:47.311 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 22.997499851000001, 2024-09-21T21:41:47.311 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031510096000000001, 2024-09-21T21:41:47.311 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:47.311 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:47.311 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:47.311 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:47.311 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:47.311 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:47.311 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1680 2024-09-21T21:41:47.311 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.311 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:47.311 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.311 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:47.311 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299356+0000", 2024-09-21T21:41:47.311 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:47.312 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.312 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.312 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:47.312 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299356+0000", 2024-09-21T21:41:47.312 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:47.312 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.312 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.312 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:47.312 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299356+0000", 2024-09-21T21:41:47.312 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.63e-07 2024-09-21T21:41:47.312 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.312 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.312 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:47.312 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299358+0000", 2024-09-21T21:41:47.313 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1399999999999998e-06 2024-09-21T21:41:47.313 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.313 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.313 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:47.313 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299364+0000", 2024-09-21T21:41:47.313 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1650000000000003e-06 2024-09-21T21:41:47.313 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.314 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.314 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:47.314 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299369+0000", 2024-09-21T21:41:47.314 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8400000000000002e-06 2024-09-21T21:41:47.314 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.314 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.314 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:47.314 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.330788+0000", 2024-09-21T21:41:47.314 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031418399 2024-09-21T21:41:47.314 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.315 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.315 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:47.315 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.330822+0000", 2024-09-21T21:41:47.315 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4218999999999999e-05 2024-09-21T21:41:47.315 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.315 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.315 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:47.315 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.330866+0000", 2024-09-21T21:41:47.315 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3869999999999998e-05 2024-09-21T21:41:47.315 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:47.315 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:47.315 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:47.315 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.315 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.316 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:1681 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 15859712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:47.316 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.299375+0000", 2024-09-21T21:41:47.316 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 22.997480147000001, 2024-09-21T21:41:47.316 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031600251000000003, 2024-09-21T21:41:47.316 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:47.316 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:47.316 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:47.316 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:47.316 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:47.316 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:47.316 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1681 2024-09-21T21:41:47.316 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.316 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:47.316 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.317 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:47.317 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299375+0000", 2024-09-21T21:41:47.317 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:47.317 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.317 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.317 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:47.317 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299375+0000", 2024-09-21T21:41:47.317 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:47.317 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.317 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.317 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:47.318 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299376+0000", 2024-09-21T21:41:47.318 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8700000000000001e-07 2024-09-21T21:41:47.318 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.318 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.318 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:47.318 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299382+0000", 2024-09-21T21:41:47.318 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3060000000000003e-06 2024-09-21T21:41:47.318 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.318 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.318 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:47.318 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299389+0000", 2024-09-21T21:41:47.318 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.3629999999999996e-06 2024-09-21T21:41:47.318 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.318 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.319 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:47.319 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299395+0000", 2024-09-21T21:41:47.319 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1279999999999999e-06 2024-09-21T21:41:47.319 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.319 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.319 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:47.319 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.330891+0000", 2024-09-21T21:41:47.319 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031496108000000002 2024-09-21T21:41:47.319 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.319 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.319 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:47.319 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.330925+0000", 2024-09-21T21:41:47.319 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4150000000000003e-05 2024-09-21T21:41:47.319 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.320 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.320 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:47.320 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.330976+0000", 2024-09-21T21:41:47.320 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0809000000000001e-05 2024-09-21T21:41:47.320 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:47.320 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:47.320 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:47.320 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.320 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.320 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:1682 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 16318464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:47.320 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.299401+0000", 2024-09-21T21:41:47.320 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 22.997454503, 2024-09-21T21:41:47.320 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031676932999999997, 2024-09-21T21:41:47.320 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:47.320 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:47.321 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:47.321 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:47.321 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:47.321 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:47.321 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1682 2024-09-21T21:41:47.321 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.321 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:47.321 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.321 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:47.321 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299401+0000", 2024-09-21T21:41:47.321 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:47.321 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.321 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.321 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:47.322 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299401+0000", 2024-09-21T21:41:47.322 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:47.322 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.322 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.322 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:47.322 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299401+0000", 2024-09-21T21:41:47.322 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.9299999999999999e-07 2024-09-21T21:41:47.322 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.322 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.322 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:47.322 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299404+0000", 2024-09-21T21:41:47.322 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1720000000000001e-06 2024-09-21T21:41:47.322 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.322 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.323 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:47.323 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299410+0000", 2024-09-21T21:41:47.323 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2480000000000001e-06 2024-09-21T21:41:47.323 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.323 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.323 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:47.323 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.299414+0000", 2024-09-21T21:41:47.323 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6480000000000002e-06 2024-09-21T21:41:47.323 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.323 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.323 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:47.323 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.330998+0000", 2024-09-21T21:41:47.323 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031583364000000003 2024-09-21T21:41:47.323 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.323 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.324 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:47.324 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.331033+0000", 2024-09-21T21:41:47.324 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4759999999999999e-05 2024-09-21T21:41:47.324 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.324 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.324 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:47.324 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.331078+0000", 2024-09-21T21:41:47.324 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5348e-05 2024-09-21T21:41:47.324 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:47.324 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:47.324 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:47.324 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.324 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.324 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:2010 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 7864320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:47.325 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.306462+0000", 2024-09-21T21:41:47.325 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 22.990393417, 2024-09-21T21:41:47.325 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031317032000000002, 2024-09-21T21:41:47.325 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:47.325 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:47.325 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:47.325 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:47.325 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:47.325 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:47.325 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2010 2024-09-21T21:41:47.325 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.325 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:47.325 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.325 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:47.326 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306462+0000", 2024-09-21T21:41:47.326 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:47.326 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.326 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.326 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:47.326 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306462+0000", 2024-09-21T21:41:47.326 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:47.326 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.326 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.326 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:47.326 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306463+0000", 2024-09-21T21:41:47.326 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2500000000000001e-07 2024-09-21T21:41:47.326 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.326 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.327 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:47.327 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306465+0000", 2024-09-21T21:41:47.327 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9740000000000001e-06 2024-09-21T21:41:47.327 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.327 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.327 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:47.327 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306470+0000", 2024-09-21T21:41:47.327 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9689999999999999e-06 2024-09-21T21:41:47.327 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.327 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.327 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:47.327 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306475+0000", 2024-09-21T21:41:47.327 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7199999999999997e-06 2024-09-21T21:41:47.327 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.327 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.328 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:47.328 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.337701+0000", 2024-09-21T21:41:47.328 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031225796 2024-09-21T21:41:47.328 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.328 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.328 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:47.328 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.337735+0000", 2024-09-21T21:41:47.328 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4399e-05 2024-09-21T21:41:47.328 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.328 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.328 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:47.328 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.337779+0000", 2024-09-21T21:41:47.328 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3748999999999997e-05 2024-09-21T21:41:47.329 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:47.329 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:47.329 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:47.329 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.329 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.329 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:2011 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 8323072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:47.329 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.306481+0000", 2024-09-21T21:41:47.329 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 22.990374313, 2024-09-21T21:41:47.329 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031404840000000003, 2024-09-21T21:41:47.329 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:47.329 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:47.329 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:47.330 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:47.330 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:47.330 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:47.330 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2011 2024-09-21T21:41:47.330 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.330 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:47.330 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.330 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:47.330 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306481+0000", 2024-09-21T21:41:47.330 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:47.330 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.330 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.330 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:47.330 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306481+0000", 2024-09-21T21:41:47.331 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:47.331 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.331 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.331 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:47.331 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306482+0000", 2024-09-21T21:41:47.331 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9100000000000004e-07 2024-09-21T21:41:47.331 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.331 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.331 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:47.331 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306484+0000", 2024-09-21T21:41:47.331 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.855e-06 2024-09-21T21:41:47.331 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.331 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.331 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:47.332 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306489+0000", 2024-09-21T21:41:47.332 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7799999999999997e-06 2024-09-21T21:41:47.332 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.332 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.332 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:47.332 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306494+0000", 2024-09-21T21:41:47.332 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7539999999999997e-06 2024-09-21T21:41:47.332 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.332 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.332 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:47.332 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.337802+0000", 2024-09-21T21:41:47.332 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031307684000000002 2024-09-21T21:41:47.332 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.332 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.333 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:47.333 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.337836+0000", 2024-09-21T21:41:47.333 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4560000000000001e-05 2024-09-21T21:41:47.333 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.333 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.333 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:47.333 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.337886+0000", 2024-09-21T21:41:47.333 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9716000000000003e-05 2024-09-21T21:41:47.333 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:47.333 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:47.333 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:47.333 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.333 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.333 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:2012 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 8781824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:47.334 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.306500+0000", 2024-09-21T21:41:47.334 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 22.990355498, 2024-09-21T21:41:47.334 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031483165, 2024-09-21T21:41:47.334 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:47.334 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:47.334 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:47.334 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:47.334 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:47.334 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:47.334 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2012 2024-09-21T21:41:47.334 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.334 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:47.334 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.334 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:47.335 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306500+0000", 2024-09-21T21:41:47.335 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:47.335 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.335 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.335 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:47.335 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306500+0000", 2024-09-21T21:41:47.335 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:47.335 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.335 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.335 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:47.337 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306500+0000", 2024-09-21T21:41:47.337 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7099999999999997e-07 2024-09-21T21:41:47.337 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.337 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.337 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:47.337 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306502+0000", 2024-09-21T21:41:47.338 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1019999999999999e-06 2024-09-21T21:41:47.338 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.338 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.338 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:47.338 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306508+0000", 2024-09-21T21:41:47.338 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8390000000000003e-06 2024-09-21T21:41:47.338 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.338 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.338 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:47.338 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306513+0000", 2024-09-21T21:41:47.338 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4909999999999999e-06 2024-09-21T21:41:47.338 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.338 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.339 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:47.339 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.337898+0000", 2024-09-21T21:41:47.339 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031385437000000002 2024-09-21T21:41:47.339 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.339 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.339 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:47.339 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.337931+0000", 2024-09-21T21:41:47.339 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.2656e-05 2024-09-21T21:41:47.339 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.339 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.339 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:47.339 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.337983+0000", 2024-09-21T21:41:47.339 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2268999999999997e-05 2024-09-21T21:41:47.339 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:47.339 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:47.340 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:47.340 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.340 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.340 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:2013 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 9240576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:47.340 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.306520+0000", 2024-09-21T21:41:47.340 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 22.990335770000002, 2024-09-21T21:41:47.340 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031570864999999997, 2024-09-21T21:41:47.340 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:47.340 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:47.340 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:47.340 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:47.340 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:47.340 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:47.340 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2013 2024-09-21T21:41:47.340 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.341 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:47.341 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.341 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:47.341 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306520+0000", 2024-09-21T21:41:47.341 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:47.341 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.341 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.341 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:47.341 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306520+0000", 2024-09-21T21:41:47.341 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:47.341 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.341 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.341 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:47.341 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306520+0000", 2024-09-21T21:41:47.342 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3700000000000001e-07 2024-09-21T21:41:47.342 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.342 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.342 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:47.342 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306530+0000", 2024-09-21T21:41:47.342 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0336000000000001e-05 2024-09-21T21:41:47.342 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.342 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.342 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:47.342 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306538+0000", 2024-09-21T21:41:47.342 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.3690000000000001e-06 2024-09-21T21:41:47.342 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.342 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.342 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:47.342 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306543+0000", 2024-09-21T21:41:47.343 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8930000000000004e-06 2024-09-21T21:41:47.343 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.343 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.343 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:47.343 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338005+0000", 2024-09-21T21:41:47.343 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031461973999999997 2024-09-21T21:41:47.343 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.343 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.343 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:47.343 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338039+0000", 2024-09-21T21:41:47.343 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4545999999999998e-05 2024-09-21T21:41:47.343 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.343 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.343 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:47.343 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338091+0000", 2024-09-21T21:41:47.344 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1409999999999997e-05 2024-09-21T21:41:47.344 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:47.344 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:47.344 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:47.344 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.344 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.344 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:2014 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 9699328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:47.344 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.306553+0000", 2024-09-21T21:41:47.344 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 22.990302817, 2024-09-21T21:41:47.344 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031642225000000003, 2024-09-21T21:41:47.344 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:47.344 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:47.344 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:47.344 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:47.345 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:47.345 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:47.345 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2014 2024-09-21T21:41:47.345 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.345 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:47.345 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.345 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:47.345 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306553+0000", 2024-09-21T21:41:47.345 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:47.345 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.345 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.345 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:47.345 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306553+0000", 2024-09-21T21:41:47.345 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:47.346 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.346 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.346 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:47.346 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306553+0000", 2024-09-21T21:41:47.346 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5600000000000001e-07 2024-09-21T21:41:47.346 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.346 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.346 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:47.346 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306555+0000", 2024-09-21T21:41:47.346 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3070000000000001e-06 2024-09-21T21:41:47.346 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.346 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.346 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:47.346 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306561+0000", 2024-09-21T21:41:47.347 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8860000000000001e-06 2024-09-21T21:41:47.347 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.347 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.347 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:47.347 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306566+0000", 2024-09-21T21:41:47.347 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3420000000000001e-06 2024-09-21T21:41:47.347 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.347 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.347 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:47.347 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338115+0000", 2024-09-21T21:41:47.347 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031549333999999998 2024-09-21T21:41:47.347 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.347 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.347 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:47.348 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338150+0000", 2024-09-21T21:41:47.348 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4649e-05 2024-09-21T21:41:47.348 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.348 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.348 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:47.348 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338195+0000", 2024-09-21T21:41:47.348 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5250999999999998e-05 2024-09-21T21:41:47.348 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:47.348 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:47.348 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:47.348 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.348 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.348 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:2015 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 10158080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:47.348 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.306572+0000", 2024-09-21T21:41:47.349 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 22.990283899000001, 2024-09-21T21:41:47.349 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031729593, 2024-09-21T21:41:47.349 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:47.349 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:47.349 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:47.349 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:47.349 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:47.349 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:47.349 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2015 2024-09-21T21:41:47.349 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.349 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:47.349 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.349 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:47.349 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306572+0000", 2024-09-21T21:41:47.349 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:47.350 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.350 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.350 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:47.350 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306572+0000", 2024-09-21T21:41:47.350 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:47.350 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.350 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.350 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:47.350 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306572+0000", 2024-09-21T21:41:47.350 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7500000000000001e-07 2024-09-21T21:41:47.350 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.350 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.350 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:47.350 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306574+0000", 2024-09-21T21:41:47.351 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0669999999999999e-06 2024-09-21T21:41:47.351 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.351 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.351 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:47.351 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306579+0000", 2024-09-21T21:41:47.351 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4009999999999996e-06 2024-09-21T21:41:47.351 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.351 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.351 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:47.351 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306585+0000", 2024-09-21T21:41:47.351 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3349999999999997e-06 2024-09-21T21:41:47.351 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.351 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.351 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:47.351 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338217+0000", 2024-09-21T21:41:47.352 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031631805999999998 2024-09-21T21:41:47.352 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.352 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.352 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:47.352 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338251+0000", 2024-09-21T21:41:47.352 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4813e-05 2024-09-21T21:41:47.352 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.352 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.352 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:47.352 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338301+0000", 2024-09-21T21:41:47.352 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9796000000000002e-05 2024-09-21T21:41:47.352 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:47.352 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:47.352 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:47.352 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.353 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.353 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:2016 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 10616832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:47.353 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.306601+0000", 2024-09-21T21:41:47.353 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 22.990254726, 2024-09-21T21:41:47.353 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031801840999999997, 2024-09-21T21:41:47.353 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:47.353 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:47.353 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:47.353 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:47.353 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:47.353 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:47.353 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2016 2024-09-21T21:41:47.353 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.353 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:47.353 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.354 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:47.354 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306601+0000", 2024-09-21T21:41:47.354 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:47.354 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.354 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.354 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:47.354 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306601+0000", 2024-09-21T21:41:47.354 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:47.354 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.354 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.355 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:47.355 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306601+0000", 2024-09-21T21:41:47.355 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1900000000000003e-07 2024-09-21T21:41:47.355 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.355 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.355 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:47.355 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306603+0000", 2024-09-21T21:41:47.355 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.17e-06 2024-09-21T21:41:47.355 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.355 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.355 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:47.355 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306609+0000", 2024-09-21T21:41:47.355 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8069999999999996e-06 2024-09-21T21:41:47.355 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.355 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.356 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:47.356 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306614+0000", 2024-09-21T21:41:47.356 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5090000000000004e-06 2024-09-21T21:41:47.356 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.356 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.356 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:47.356 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338325+0000", 2024-09-21T21:41:47.356 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031711237000000003 2024-09-21T21:41:47.356 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.356 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.356 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:47.356 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338359+0000", 2024-09-21T21:41:47.356 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3810000000000003e-05 2024-09-21T21:41:47.356 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.356 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.357 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:47.357 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338403+0000", 2024-09-21T21:41:47.357 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3788999999999997e-05 2024-09-21T21:41:47.357 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:47.357 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:47.357 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:47.357 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.357 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.357 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:2017 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 11075584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:47.357 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.306621+0000", 2024-09-21T21:41:47.357 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 22.990234959999999, 2024-09-21T21:41:47.357 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031891271999999998, 2024-09-21T21:41:47.357 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:47.357 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:47.358 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:47.358 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:47.358 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:47.358 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:47.358 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2017 2024-09-21T21:41:47.358 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.358 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:47.358 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.358 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:47.358 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306621+0000", 2024-09-21T21:41:47.358 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:47.358 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.358 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.358 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:47.358 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306621+0000", 2024-09-21T21:41:47.359 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:47.359 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.359 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.359 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:47.359 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306621+0000", 2024-09-21T21:41:47.359 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5400000000000002e-07 2024-09-21T21:41:47.359 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.359 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.359 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:47.359 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306623+0000", 2024-09-21T21:41:47.359 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1100000000000001e-06 2024-09-21T21:41:47.359 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.359 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.359 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:47.360 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306629+0000", 2024-09-21T21:41:47.360 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7810000000000002e-06 2024-09-21T21:41:47.360 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.360 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.360 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:47.360 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306633+0000", 2024-09-21T21:41:47.360 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5589999999999999e-06 2024-09-21T21:41:47.360 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.360 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.360 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:47.360 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338427+0000", 2024-09-21T21:41:47.360 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031793948000000002 2024-09-21T21:41:47.360 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.360 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.360 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:47.361 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338462+0000", 2024-09-21T21:41:47.361 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4785e-05 2024-09-21T21:41:47.361 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.361 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.361 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:47.361 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338512+0000", 2024-09-21T21:41:47.361 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9734999999999997e-05 2024-09-21T21:41:47.361 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:47.361 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:47.361 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:47.361 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.361 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.361 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:2018 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 11534336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:47.361 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.306639+0000", 2024-09-21T21:41:47.361 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 22.990216330999999, 2024-09-21T21:41:47.362 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031963189000000003, 2024-09-21T21:41:47.362 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:47.362 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:47.362 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:47.362 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:47.362 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:47.362 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:47.362 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2018 2024-09-21T21:41:47.362 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.362 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:47.362 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.362 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:47.362 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306639+0000", 2024-09-21T21:41:47.362 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:47.363 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.363 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.363 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:47.363 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306639+0000", 2024-09-21T21:41:47.363 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:47.363 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.363 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.363 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:47.363 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306640+0000", 2024-09-21T21:41:47.363 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4600000000000005e-07 2024-09-21T21:41:47.363 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.363 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.363 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:47.363 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306642+0000", 2024-09-21T21:41:47.364 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.916e-06 2024-09-21T21:41:47.364 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.364 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.364 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:47.364 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306647+0000", 2024-09-21T21:41:47.364 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7590000000000003e-06 2024-09-21T21:41:47.364 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.364 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.364 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:47.364 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306652+0000", 2024-09-21T21:41:47.364 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4580000000000003e-06 2024-09-21T21:41:47.364 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.364 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.364 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:47.365 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338524+0000", 2024-09-21T21:41:47.365 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031872172999999997 2024-09-21T21:41:47.365 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.365 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.365 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:47.365 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338557+0000", 2024-09-21T21:41:47.365 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3136000000000003e-05 2024-09-21T21:41:47.365 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.365 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.365 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:47.365 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338602+0000", 2024-09-21T21:41:47.365 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5201000000000003e-05 2024-09-21T21:41:47.365 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:47.365 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:47.365 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:47.366 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.366 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.366 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:2019 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 11993088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:47.366 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.306658+0000", 2024-09-21T21:41:47.366 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 22.990197937000001, 2024-09-21T21:41:47.366 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.032059007, 2024-09-21T21:41:47.366 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:47.366 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:47.366 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:47.366 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:47.366 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:47.366 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:47.366 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2019 2024-09-21T21:41:47.367 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.367 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:47.367 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.367 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:47.367 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306658+0000", 2024-09-21T21:41:47.367 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:47.367 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.367 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.367 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:47.367 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.306658+0000", 2024-09-21T21:41:47.367 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:47.367 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.367 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.367 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:47.368 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.307950+0000", 2024-09-21T21:41:47.368 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.001292199 2024-09-21T21:41:47.368 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.368 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.368 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:47.368 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.307954+0000", 2024-09-21T21:41:47.368 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5689999999999998e-06 2024-09-21T21:41:47.368 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.368 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.368 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:47.368 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.307964+0000", 2024-09-21T21:41:47.368 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6749999999999997e-06 2024-09-21T21:41:47.368 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.369 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.369 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:47.369 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.307971+0000", 2024-09-21T21:41:47.369 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5189999999999998e-06 2024-09-21T21:41:47.369 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.369 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.369 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:47.369 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338625+0000", 2024-09-21T21:41:47.369 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.030654569 2024-09-21T21:41:47.369 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.369 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.369 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:47.370 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338660+0000", 2024-09-21T21:41:47.370 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4656999999999997e-05 2024-09-21T21:41:47.370 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.370 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.370 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:47.370 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.338717+0000", 2024-09-21T21:41:47.370 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6818999999999999e-05 2024-09-21T21:41:47.370 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:47.370 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:47.370 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:47.370 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.370 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.370 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:2026 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 15204352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:47.370 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.308109+0000", 2024-09-21T21:41:47.370 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 22.988746336999998, 2024-09-21T21:41:47.371 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031315883000000003, 2024-09-21T21:41:47.371 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:47.371 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:47.371 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:47.371 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:47.371 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:47.371 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:47.371 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2026 2024-09-21T21:41:47.371 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.371 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:47.371 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.371 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:47.371 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308109+0000", 2024-09-21T21:41:47.371 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:47.372 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.372 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.372 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:47.372 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308109+0000", 2024-09-21T21:41:47.372 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:47.372 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.372 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.372 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:47.372 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308110+0000", 2024-09-21T21:41:47.372 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.96e-07 2024-09-21T21:41:47.372 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.372 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.372 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:47.372 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308112+0000", 2024-09-21T21:41:47.373 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0269999999999998e-06 2024-09-21T21:41:47.373 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.373 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.373 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:47.373 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308117+0000", 2024-09-21T21:41:47.373 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.631e-06 2024-09-21T21:41:47.373 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.373 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.373 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:47.373 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308122+0000", 2024-09-21T21:41:47.373 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7310000000000001e-06 2024-09-21T21:41:47.373 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.373 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.373 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:47.373 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.339341+0000", 2024-09-21T21:41:47.374 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031218610000000001 2024-09-21T21:41:47.374 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.374 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.374 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:47.374 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.339376+0000", 2024-09-21T21:41:47.374 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5305000000000002e-05 2024-09-21T21:41:47.374 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.374 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.374 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:47.374 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.339425+0000", 2024-09-21T21:41:47.374 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9183e-05 2024-09-21T21:41:47.374 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:47.374 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:47.374 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:47.375 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.375 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.375 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:2027 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 15663104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:47.375 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.308128+0000", 2024-09-21T21:41:47.375 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 22.988727801, 2024-09-21T21:41:47.375 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031387452000000003, 2024-09-21T21:41:47.375 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:47.375 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:47.375 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:47.375 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:47.375 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:47.375 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:47.375 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2027 2024-09-21T21:41:47.375 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.376 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:47.376 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.376 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:47.376 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308128+0000", 2024-09-21T21:41:47.376 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:47.376 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.376 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.376 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:47.376 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308128+0000", 2024-09-21T21:41:47.376 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:47.376 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.376 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.376 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:47.376 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308128+0000", 2024-09-21T21:41:47.377 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4700000000000002e-07 2024-09-21T21:41:47.377 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.377 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.377 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:47.377 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308130+0000", 2024-09-21T21:41:47.377 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.8670000000000001e-06 2024-09-21T21:41:47.377 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.377 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.377 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:47.377 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308136+0000", 2024-09-21T21:41:47.377 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0299999999999999e-06 2024-09-21T21:41:47.377 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.377 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.377 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:47.377 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308140+0000", 2024-09-21T21:41:47.378 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3830000000000002e-06 2024-09-21T21:41:47.378 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.378 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.378 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:47.378 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.339437+0000", 2024-09-21T21:41:47.378 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031296864000000001 2024-09-21T21:41:47.378 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.378 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.378 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:47.378 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.339471+0000", 2024-09-21T21:41:47.378 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3302000000000001e-05 2024-09-21T21:41:47.378 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.378 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.378 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:47.379 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.339515+0000", 2024-09-21T21:41:47.379 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4659000000000001e-05 2024-09-21T21:41:47.379 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:47.379 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:47.379 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:47.379 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.379 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.379 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:2028 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 16121856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:47.379 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.308147+0000", 2024-09-21T21:41:47.379 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 22.988708874, 2024-09-21T21:41:47.379 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031474477000000001, 2024-09-21T21:41:47.379 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:47.379 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:47.379 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:47.379 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:47.380 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:47.380 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:47.380 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2028 2024-09-21T21:41:47.380 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.380 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:47.380 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.380 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:47.380 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308147+0000", 2024-09-21T21:41:47.380 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:47.380 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.380 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.380 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:47.380 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308147+0000", 2024-09-21T21:41:47.380 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:47.380 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.381 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.381 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:47.381 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308147+0000", 2024-09-21T21:41:47.381 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1900000000000003e-07 2024-09-21T21:41:47.381 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.381 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.381 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:47.381 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308149+0000", 2024-09-21T21:41:47.381 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9479999999999998e-06 2024-09-21T21:41:47.381 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.381 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.381 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:47.381 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308155+0000", 2024-09-21T21:41:47.381 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6200000000000004e-06 2024-09-21T21:41:47.381 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.382 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.382 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:47.382 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308160+0000", 2024-09-21T21:41:47.382 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0799999999999996e-06 2024-09-21T21:41:47.382 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.382 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.382 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:47.382 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.339539+0000", 2024-09-21T21:41:47.382 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031378902 2024-09-21T21:41:47.382 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.382 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.382 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:47.382 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.339574+0000", 2024-09-21T21:41:47.382 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5176999999999999e-05 2024-09-21T21:41:47.383 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.383 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.383 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:47.383 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.339621+0000", 2024-09-21T21:41:47.383 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7231000000000002e-05 2024-09-21T21:41:47.383 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:47.383 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:47.383 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:47.383 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.383 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.383 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:2029 3.5 3:a3fdb1a9:::rbd_data.1121fb38290b.0000000000000005:15 [sparse-read 16580608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:47.383 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.308167+0000", 2024-09-21T21:41:47.383 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 22.988688485000001, 2024-09-21T21:41:47.383 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031555582999999998, 2024-09-21T21:41:47.383 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:47.384 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:47.384 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:47.384 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:47.384 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:47.384 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:47.384 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2029 2024-09-21T21:41:47.384 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.384 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:47.384 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.384 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:47.384 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308167+0000", 2024-09-21T21:41:47.384 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:47.384 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.384 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.384 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:47.385 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308167+0000", 2024-09-21T21:41:47.385 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:47.385 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.385 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.385 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:47.385 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308167+0000", 2024-09-21T21:41:47.385 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6600000000000002e-07 2024-09-21T21:41:47.385 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.385 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.385 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:47.385 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308169+0000", 2024-09-21T21:41:47.385 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9819999999999998e-06 2024-09-21T21:41:47.385 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.385 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.385 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:47.386 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308175+0000", 2024-09-21T21:41:47.386 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8289999999999996e-06 2024-09-21T21:41:47.386 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.386 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.386 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:47.386 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308180+0000", 2024-09-21T21:41:47.386 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6439999999999999e-06 2024-09-21T21:41:47.386 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.386 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.386 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:47.386 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.339645+0000", 2024-09-21T21:41:47.386 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031465083999999997 2024-09-21T21:41:47.386 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.387 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.387 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:47.387 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.339679+0000", 2024-09-21T21:41:47.387 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3572999999999999e-05 2024-09-21T21:41:47.387 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.387 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.387 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:47.387 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.339723+0000", 2024-09-21T21:41:47.387 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4104999999999999e-05 2024-09-21T21:41:47.387 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:47.387 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:47.387 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:47.387 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.387 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.388 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4999.0:2059 3.5 3:a6b41377:::rbd_data.11213654c685.0000000000000008:head [stat,truncate 13631488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-21T21:41:47.388 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:24.308186+0000", 2024-09-21T21:41:47.388 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 22.988669906999998, 2024-09-21T21:41:47.388 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.032997110000000003, 2024-09-21T21:41:47.388 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:47.388 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:47.388 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:41:47.388 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:47.388 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4999", 2024-09-21T21:41:47.388 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/1779836824", 2024-09-21T21:41:47.388 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 2059 2024-09-21T21:41:47.388 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.388 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:47.388 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.388 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:47.389 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308186+0000", 2024-09-21T21:41:47.389 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:47.389 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.389 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.389 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:47.389 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308186+0000", 2024-09-21T21:41:47.389 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:47.389 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.389 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.389 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:47.389 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308186+0000", 2024-09-21T21:41:47.389 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1899999999999998e-07 2024-09-21T21:41:47.389 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.389 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.390 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:47.390 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308188+0000", 2024-09-21T21:41:47.390 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0099999999999998e-06 2024-09-21T21:41:47.390 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.390 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.390 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:47.390 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308195+0000", 2024-09-21T21:41:47.390 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0589999999999997e-06 2024-09-21T21:41:47.390 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.390 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.390 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:47.390 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.308199+0000", 2024-09-21T21:41:47.390 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2930000000000002e-06 2024-09-21T21:41:47.390 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.390 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.391 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:47.391 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.339745+0000", 2024-09-21T21:41:47.391 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.031546091999999998 2024-09-21T21:41:47.391 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.391 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.391 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:47.391 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.339782+0000", 2024-09-21T21:41:47.391 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6847999999999997e-05 2024-09-21T21:41:47.391 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.391 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.391 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:41:47.391 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.339856+0000", 2024-09-21T21:41:47.391 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.3992999999999994e-05 2024-09-21T21:41:47.391 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.392 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.392 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:41:47.392 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.341013+0000", 2024-09-21T21:41:47.392 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00115642 2024-09-21T21:41:47.392 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.392 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.392 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:41:47.392 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.341138+0000", 2024-09-21T21:41:47.392 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00012540699999999999 2024-09-21T21:41:47.392 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.392 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.392 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:41:47.392 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.341164+0000", 2024-09-21T21:41:47.392 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.6168000000000001e-05 2024-09-21T21:41:47.392 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:47.393 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:47.393 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:47.393 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:24.341183+0000", 2024-09-21T21:41:47.393 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.8400999999999999e-05 2024-09-21T21:41:47.393 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:47.393 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:47.393 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:47.393 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:47.393 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:47.393 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:47.394 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T21:41:47.435 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:47.434+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:47.498 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:47.496+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:47.498 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:47.499 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:41:47.499 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-21T21:41:47.499 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:47.508 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T21:41:47.599 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:47.598+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:47.658 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:47.658 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:41:47.658 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:41:47.658 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:41:47.658 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:47.667 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T21:41:47.668 INFO:teuthology.orchestra.run.smithi164.stdout:3899 write 0x6f4a000 thru 0x6f4dc5e (0x3c5f bytes) 2024-09-21T21:41:47.692 INFO:teuthology.orchestra.run.smithi164.stdout:3900 write 0xad4d800 thru 0xad546f3 (0x6ef4 bytes) 2024-09-21T21:41:47.692 INFO:teuthology.orchestra.run.smithi164.stdout:3902 read 0x9719600 thru 0x971c54b (0x2f4c bytes) 2024-09-21T21:41:47.694 INFO:teuthology.orchestra.run.smithi164.stdout:3905 trunc from 0xad54800 to 0xc45e400 2024-09-21T21:41:47.713 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:47.711+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:47.716 INFO:teuthology.orchestra.run.smithi164.stdout:3908 trunc from 0xc45e400 to 0x95d2c00 2024-09-21T21:41:47.733 INFO:teuthology.orchestra.run.smithi164.stdout:3909 read 0x5bf9e00 thru 0x5bfb36a (0x156b bytes) 2024-09-21T21:41:47.733 INFO:teuthology.orchestra.run.smithi164.stdout:3910 clone 31 order 21 su 131072 sc 7 2024-09-21T21:41:47.781 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:47.781 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-21T21:41:47.781 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-21T21:41:47.781 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:41:47.781 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.781 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2617 3.d 3:b26310db:::rbd_data.112145fae779.0000000000000006:a [read 3686400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:47.781 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.411596+0000", 2024-09-21T21:41:47.781 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 49.366818029000001, 2024-09-21T21:41:47.781 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.045094515000000002, 2024-09-21T21:41:47.781 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:47.782 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:47.782 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:47.782 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:47.782 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:47.782 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:47.782 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2617 2024-09-21T21:41:47.782 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.782 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:47.782 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.782 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:47.782 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411596+0000", 2024-09-21T21:41:47.782 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:47.782 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.782 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.783 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:47.783 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411596+0000", 2024-09-21T21:41:47.783 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:47.783 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.783 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.783 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:47.783 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411596+0000", 2024-09-21T21:41:47.783 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3500000000000002e-07 2024-09-21T21:41:47.783 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.783 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.783 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:47.783 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411598+0000", 2024-09-21T21:41:47.783 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.871e-06 2024-09-21T21:41:47.783 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.784 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.784 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:47.784 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411603+0000", 2024-09-21T21:41:47.784 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6420000000000004e-06 2024-09-21T21:41:47.784 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.784 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.784 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:47.784 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411608+0000", 2024-09-21T21:41:47.784 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4780000000000002e-06 2024-09-21T21:41:47.784 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.784 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.784 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:47.784 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.456529+0000", 2024-09-21T21:41:47.784 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.044921232999999998 2024-09-21T21:41:47.785 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.785 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.785 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:47.785 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.456616+0000", 2024-09-21T21:41:47.785 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6712000000000006e-05 2024-09-21T21:41:47.785 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.785 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.785 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:47.785 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.456690+0000", 2024-09-21T21:41:47.785 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4244000000000003e-05 2024-09-21T21:41:47.785 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:47.785 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:47.785 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:47.785 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.786 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.786 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4797.0:2618 3.d 3:b26310db:::rbd_data.112145fae779.0000000000000006:a [read 3743744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-21T21:41:47.786 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:40:58.411613+0000", 2024-09-21T21:41:47.786 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 49.36680045, 2024-09-21T21:41:47.786 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.045180423999999997, 2024-09-21T21:41:47.786 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:47.786 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:47.786 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:47.786 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:47.786 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4797", 2024-09-21T21:41:47.786 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/4109709710", 2024-09-21T21:41:47.786 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2618 2024-09-21T21:41:47.786 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.786 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:47.787 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.787 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:47.787 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411613+0000", 2024-09-21T21:41:47.787 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:47.787 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.787 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.787 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:47.787 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.411613+0000", 2024-09-21T21:41:47.787 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:47.787 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.787 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.787 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:47.787 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.412868+0000", 2024-09-21T21:41:47.787 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012549 2024-09-21T21:41:47.788 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.788 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.788 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:47.788 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.412873+0000", 2024-09-21T21:41:47.788 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9060000000000001e-06 2024-09-21T21:41:47.788 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.788 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.788 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:47.788 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.412884+0000", 2024-09-21T21:41:47.788 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1164e-05 2024-09-21T21:41:47.788 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.788 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.788 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:47.788 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.412892+0000", 2024-09-21T21:41:47.789 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5719999999999997e-06 2024-09-21T21:41:47.789 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.789 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.789 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:47.789 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.456717+0000", 2024-09-21T21:41:47.789 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043825559 2024-09-21T21:41:47.789 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.789 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.789 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:47.790 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.456753+0000", 2024-09-21T21:41:47.790 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6158000000000002e-05 2024-09-21T21:41:47.790 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.790 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.790 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:47.790 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:40:58.456794+0000", 2024-09-21T21:41:47.790 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0164999999999999e-05 2024-09-21T21:41:47.790 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:47.790 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:47.790 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:47.790 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.790 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.790 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5018.0:2937 3.15 3:af2e3e9a:::rbd_data.11213fc02c9d.0000000000000000:19 [read 475136~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T21:41:47.790 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:33.865486+0000", 2024-09-21T21:41:47.791 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.912927634000001, 2024-09-21T21:41:47.791 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047419950000000002, 2024-09-21T21:41:47.791 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:47.791 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:47.791 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:47.791 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:47.791 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5018", 2024-09-21T21:41:47.791 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3046875242", 2024-09-21T21:41:47.791 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2937 2024-09-21T21:41:47.791 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.791 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:47.791 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.791 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:47.791 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865486+0000", 2024-09-21T21:41:47.791 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:47.792 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.792 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.792 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:47.792 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865486+0000", 2024-09-21T21:41:47.792 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:47.792 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.792 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.792 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:47.792 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865486+0000", 2024-09-21T21:41:47.792 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1800000000000001e-07 2024-09-21T21:41:47.792 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.792 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.792 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:47.792 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865488+0000", 2024-09-21T21:41:47.792 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9230000000000001e-06 2024-09-21T21:41:47.793 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.793 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.793 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:47.793 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865496+0000", 2024-09-21T21:41:47.793 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6260000000000003e-06 2024-09-21T21:41:47.793 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.793 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.793 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:47.794 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865501+0000", 2024-09-21T21:41:47.794 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7620000000000003e-06 2024-09-21T21:41:47.794 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.794 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.794 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:47.794 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.912653+0000", 2024-09-21T21:41:47.794 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047152816 2024-09-21T21:41:47.794 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.795 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.795 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:47.795 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.912768+0000", 2024-09-21T21:41:47.795 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000114558 2024-09-21T21:41:47.795 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.795 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.795 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:47.795 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.912906+0000", 2024-09-21T21:41:47.795 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000137847 2024-09-21T21:41:47.795 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:47.795 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:47.795 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:47.795 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.795 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.796 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5018.0:2938 3.15 3:af2e3e9a:::rbd_data.11213fc02c9d.0000000000000000:19 [read 417792~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T21:41:47.796 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:33.865505+0000", 2024-09-21T21:41:47.796 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.912908898, 2024-09-21T21:41:47.796 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047654654999999997, 2024-09-21T21:41:47.796 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:47.796 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:47.796 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:47.796 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:47.796 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5018", 2024-09-21T21:41:47.796 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3046875242", 2024-09-21T21:41:47.796 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2938 2024-09-21T21:41:47.796 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.796 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:47.797 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.797 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:47.797 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865505+0000", 2024-09-21T21:41:47.797 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:47.797 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.797 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.797 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:47.797 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865505+0000", 2024-09-21T21:41:47.797 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:47.797 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.797 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.798 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:47.798 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865505+0000", 2024-09-21T21:41:47.798 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2999999999999999e-07 2024-09-21T21:41:47.798 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.798 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.798 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:47.798 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865506+0000", 2024-09-21T21:41:47.798 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2979999999999999e-06 2024-09-21T21:41:47.798 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.798 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.798 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:47.798 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865509+0000", 2024-09-21T21:41:47.798 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.163e-06 2024-09-21T21:41:47.798 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.799 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.799 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:47.799 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865512+0000", 2024-09-21T21:41:47.799 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7429999999999998e-06 2024-09-21T21:41:47.799 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.799 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.799 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:47.799 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.912962+0000", 2024-09-21T21:41:47.799 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047450349000000003 2024-09-21T21:41:47.800 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.800 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.800 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:47.800 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.913054+0000", 2024-09-21T21:41:47.800 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.1384999999999993e-05 2024-09-21T21:41:47.800 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.800 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.800 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:47.800 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.913159+0000", 2024-09-21T21:41:47.800 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000105487 2024-09-21T21:41:47.800 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:47.800 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:47.800 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:47.800 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.801 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.801 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5018.0:2939 3.15 3:af2e3e9a:::rbd_data.11213fc02c9d.0000000000000000:19 [read 425984~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T21:41:47.801 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:33.865515+0000", 2024-09-21T21:41:47.801 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.912898397999999, 2024-09-21T21:41:47.801 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047853433000000001, 2024-09-21T21:41:47.801 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:47.801 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:47.801 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:47.801 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:47.801 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5018", 2024-09-21T21:41:47.801 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3046875242", 2024-09-21T21:41:47.801 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2939 2024-09-21T21:41:47.801 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.801 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:47.802 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.802 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:47.802 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865515+0000", 2024-09-21T21:41:47.802 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:47.802 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.802 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.802 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:47.802 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865515+0000", 2024-09-21T21:41:47.802 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:47.802 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.802 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.802 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:47.802 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865516+0000", 2024-09-21T21:41:47.802 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6800000000000001e-07 2024-09-21T21:41:47.802 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.803 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.803 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:47.803 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865517+0000", 2024-09-21T21:41:47.803 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.083e-06 2024-09-21T21:41:47.803 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.803 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.803 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:47.803 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865520+0000", 2024-09-21T21:41:47.803 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0699999999999998e-06 2024-09-21T21:41:47.803 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.804 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.804 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:47.804 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865523+0000", 2024-09-21T21:41:47.804 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.959e-06 2024-09-21T21:41:47.804 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.804 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.804 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:47.804 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.913199+0000", 2024-09-21T21:41:47.805 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047676653999999999 2024-09-21T21:41:47.805 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.805 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.805 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:47.805 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.913275+0000", 2024-09-21T21:41:47.805 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6154000000000003e-05 2024-09-21T21:41:47.805 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.805 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.805 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:47.805 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.913369+0000", 2024-09-21T21:41:47.805 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3145000000000004e-05 2024-09-21T21:41:47.805 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:47.805 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:47.806 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:47.806 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.806 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.806 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5018.0:2940 3.15 3:af2e3e9a:::rbd_data.11213fc02c9d.0000000000000000:19 [read 434176~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T21:41:47.806 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:33.865528+0000", 2024-09-21T21:41:47.806 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.912885943999999, 2024-09-21T21:41:47.806 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.048052481000000001, 2024-09-21T21:41:47.806 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:47.806 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:47.806 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:47.806 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:47.807 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5018", 2024-09-21T21:41:47.807 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3046875242", 2024-09-21T21:41:47.807 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2940 2024-09-21T21:41:47.807 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.807 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:47.807 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.807 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:47.807 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865528+0000", 2024-09-21T21:41:47.807 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:47.807 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.807 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.807 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:47.807 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865528+0000", 2024-09-21T21:41:47.807 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:47.808 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.808 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.808 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:47.808 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865528+0000", 2024-09-21T21:41:47.808 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3500000000000002e-07 2024-09-21T21:41:47.808 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.808 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.808 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:47.808 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865530+0000", 2024-09-21T21:41:47.808 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1689999999999999e-06 2024-09-21T21:41:47.808 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.808 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.808 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:47.808 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865536+0000", 2024-09-21T21:41:47.808 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8560000000000003e-06 2024-09-21T21:41:47.809 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.809 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.809 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:47.809 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865541+0000", 2024-09-21T21:41:47.809 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.899e-06 2024-09-21T21:41:47.809 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.809 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.809 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:47.809 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.913406+0000", 2024-09-21T21:41:47.809 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047865216000000002 2024-09-21T21:41:47.809 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.809 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.809 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:47.809 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.913483+0000", 2024-09-21T21:41:47.810 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6525000000000003e-05 2024-09-21T21:41:47.810 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.810 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.810 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:47.810 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.913580+0000", 2024-09-21T21:41:47.810 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.7480999999999997e-05 2024-09-21T21:41:47.810 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:47.810 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:47.810 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:47.810 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.810 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.810 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5018.0:2941 3.15 3:af2e3e9a:::rbd_data.11213fc02c9d.0000000000000000:19 [read 483328~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T21:41:47.810 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:33.865558+0000", 2024-09-21T21:41:47.810 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.912855674999999, 2024-09-21T21:41:47.811 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.048220161999999997, 2024-09-21T21:41:47.811 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:47.811 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:47.811 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:47.811 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:47.811 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5018", 2024-09-21T21:41:47.811 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3046875242", 2024-09-21T21:41:47.811 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2941 2024-09-21T21:41:47.811 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.811 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:47.811 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.811 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:47.811 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865558+0000", 2024-09-21T21:41:47.811 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:47.811 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.812 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.812 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:47.812 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865558+0000", 2024-09-21T21:41:47.812 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:47.812 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.812 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.812 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:47.812 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865558+0000", 2024-09-21T21:41:47.812 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1799999999999995e-07 2024-09-21T21:41:47.812 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.812 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.812 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:47.812 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865561+0000", 2024-09-21T21:41:47.812 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.317e-06 2024-09-21T21:41:47.813 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.813 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.813 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:47.813 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865568+0000", 2024-09-21T21:41:47.813 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8510000000000001e-06 2024-09-21T21:41:47.813 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.813 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.813 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:47.813 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865573+0000", 2024-09-21T21:41:47.813 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1730000000000003e-06 2024-09-21T21:41:47.813 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.813 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.813 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:47.813 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.913617+0000", 2024-09-21T21:41:47.814 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.048044302999999997 2024-09-21T21:41:47.814 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.814 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.814 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:47.814 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.913689+0000", 2024-09-21T21:41:47.814 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1658999999999999e-05 2024-09-21T21:41:47.814 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.814 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.814 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:47.814 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.913778+0000", 2024-09-21T21:41:47.814 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9340999999999997e-05 2024-09-21T21:41:47.814 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:47.814 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:47.814 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:47.815 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.815 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.815 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5018.0:2942 3.15 3:af2e3e9a:::rbd_data.11213fc02c9d.0000000000000000:19 [read 491520~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T21:41:47.815 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:33.865580+0000", 2024-09-21T21:41:47.815 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.912833175999999, 2024-09-21T21:41:47.815 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.048410436000000001, 2024-09-21T21:41:47.815 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:47.815 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:47.815 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:47.815 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:47.815 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5018", 2024-09-21T21:41:47.815 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3046875242", 2024-09-21T21:41:47.815 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2942 2024-09-21T21:41:47.815 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.815 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:47.816 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.816 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:47.816 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865580+0000", 2024-09-21T21:41:47.816 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:47.816 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.816 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.816 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:47.816 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865580+0000", 2024-09-21T21:41:47.816 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:47.816 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.816 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.816 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:47.816 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865581+0000", 2024-09-21T21:41:47.816 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.51e-07 2024-09-21T21:41:47.817 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.817 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.817 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:47.817 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865583+0000", 2024-09-21T21:41:47.817 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.12e-06 2024-09-21T21:41:47.817 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.817 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.817 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:47.817 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865589+0000", 2024-09-21T21:41:47.817 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3080000000000004e-06 2024-09-21T21:41:47.817 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.817 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.817 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:47.817 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865594+0000", 2024-09-21T21:41:47.817 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9300000000000002e-06 2024-09-21T21:41:47.818 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.818 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.818 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:47.818 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.913813+0000", 2024-09-21T21:41:47.818 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.048218924000000003 2024-09-21T21:41:47.818 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.818 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.818 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:47.818 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.913891+0000", 2024-09-21T21:41:47.818 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7523999999999998e-05 2024-09-21T21:41:47.818 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.818 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.818 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:47.818 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.913991+0000", 2024-09-21T21:41:47.819 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000100179 2024-09-21T21:41:47.819 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:47.819 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:47.819 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:47.819 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.819 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.819 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5018.0:2943 3.15 3:af2e3e9a:::rbd_data.11213fc02c9d.0000000000000000:19 [read 393216~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T21:41:47.819 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:33.865612+0000", 2024-09-21T21:41:47.819 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.912801041, 2024-09-21T21:41:47.819 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.048585616999999998, 2024-09-21T21:41:47.819 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:47.819 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:47.819 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:47.819 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:47.819 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5018", 2024-09-21T21:41:47.820 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3046875242", 2024-09-21T21:41:47.820 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2943 2024-09-21T21:41:47.820 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.820 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:47.820 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.820 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:47.820 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865612+0000", 2024-09-21T21:41:47.820 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:47.820 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.820 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.820 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:47.820 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865612+0000", 2024-09-21T21:41:47.820 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:47.820 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.820 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.821 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:47.821 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865613+0000", 2024-09-21T21:41:47.821 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9200000000000002e-07 2024-09-21T21:41:47.821 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.821 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.821 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:47.821 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865618+0000", 2024-09-21T21:41:47.821 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9729999999999997e-06 2024-09-21T21:41:47.821 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.821 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.821 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:47.821 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865623+0000", 2024-09-21T21:41:47.821 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6970000000000001e-06 2024-09-21T21:41:47.821 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.821 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.822 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:47.822 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865626+0000", 2024-09-21T21:41:47.822 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0879999999999999e-06 2024-09-21T21:41:47.822 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.822 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.822 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:47.822 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.914028+0000", 2024-09-21T21:41:47.822 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.048401921000000001 2024-09-21T21:41:47.822 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.822 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.822 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:47.822 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.914104+0000", 2024-09-21T21:41:47.823 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6312999999999999e-05 2024-09-21T21:41:47.823 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.823 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.823 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:47.823 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.914198+0000", 2024-09-21T21:41:47.823 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.4233000000000004e-05 2024-09-21T21:41:47.823 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:47.823 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:47.823 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:47.823 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.823 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.823 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5018.0:2944 3.15 3:af2e3e9a:::rbd_data.11213fc02c9d.0000000000000000:19 [read 442368~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T21:41:47.823 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:33.865654+0000", 2024-09-21T21:41:47.823 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.912759538, 2024-09-21T21:41:47.823 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.048771901999999999, 2024-09-21T21:41:47.824 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:47.824 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:47.824 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:47.824 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:47.824 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5018", 2024-09-21T21:41:47.824 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3046875242", 2024-09-21T21:41:47.824 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2944 2024-09-21T21:41:47.824 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.824 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:47.824 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.824 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:47.824 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865654+0000", 2024-09-21T21:41:47.824 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:47.824 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.824 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.825 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:47.825 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865654+0000", 2024-09-21T21:41:47.825 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:47.825 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.825 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.825 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:47.825 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865655+0000", 2024-09-21T21:41:47.825 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4199999999999995e-07 2024-09-21T21:41:47.825 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.825 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.825 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:47.825 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865656+0000", 2024-09-21T21:41:47.825 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.8139999999999999e-06 2024-09-21T21:41:47.825 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.826 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.826 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:47.826 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865661+0000", 2024-09-21T21:41:47.826 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6489999999999999e-06 2024-09-21T21:41:47.826 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.826 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.826 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:47.826 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865664+0000", 2024-09-21T21:41:47.826 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2119999999999999e-06 2024-09-21T21:41:47.826 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.826 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.826 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:47.826 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.914236+0000", 2024-09-21T21:41:47.826 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.048571707999999998 2024-09-21T21:41:47.827 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.827 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.827 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:47.827 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.914319+0000", 2024-09-21T21:41:47.827 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2689999999999996e-05 2024-09-21T21:41:47.827 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.827 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.827 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:47.827 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.914426+0000", 2024-09-21T21:41:47.827 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000107187 2024-09-21T21:41:47.827 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:47.827 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:47.827 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:47.827 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.828 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.828 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5018.0:2945 3.15 3:af2e3e9a:::rbd_data.11213fc02c9d.0000000000000000:19 [read 507904~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T21:41:47.828 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:33.865670+0000", 2024-09-21T21:41:47.828 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.912744031999999, 2024-09-21T21:41:47.828 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.048961808000000002, 2024-09-21T21:41:47.828 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:47.828 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:47.828 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:47.828 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:47.828 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5018", 2024-09-21T21:41:47.828 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3046875242", 2024-09-21T21:41:47.828 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2945 2024-09-21T21:41:47.828 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.828 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:47.828 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.829 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:47.829 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865670+0000", 2024-09-21T21:41:47.829 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:47.829 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.829 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.829 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:47.829 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865670+0000", 2024-09-21T21:41:47.829 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:47.829 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.829 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.829 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:47.829 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865670+0000", 2024-09-21T21:41:47.829 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8400000000000005e-07 2024-09-21T21:41:47.829 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.829 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.830 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:47.830 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865672+0000", 2024-09-21T21:41:47.830 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.691e-06 2024-09-21T21:41:47.830 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.830 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.830 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:47.830 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865676+0000", 2024-09-21T21:41:47.830 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5390000000000001e-06 2024-09-21T21:41:47.830 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.830 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.830 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:47.830 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865680+0000", 2024-09-21T21:41:47.830 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.67e-06 2024-09-21T21:41:47.830 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.831 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.831 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:47.831 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.914460+0000", 2024-09-21T21:41:47.831 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.048780221999999998 2024-09-21T21:41:47.831 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.831 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.831 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:47.831 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.914539+0000", 2024-09-21T21:41:47.831 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8634000000000006e-05 2024-09-21T21:41:47.831 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.831 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.831 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:47.831 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.914631+0000", 2024-09-21T21:41:47.831 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.2567999999999999e-05 2024-09-21T21:41:47.831 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:47.832 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:47.832 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:47.832 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.832 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.832 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5018.0:2946 3.15 3:af2e3e9a:::rbd_data.11213fc02c9d.0000000000000000:19 [read 450560~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T21:41:47.832 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:33.865694+0000", 2024-09-21T21:41:47.832 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.912719104000001, 2024-09-21T21:41:47.832 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.049163473999999999, 2024-09-21T21:41:47.832 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:47.832 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:47.832 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:47.832 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:47.832 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5018", 2024-09-21T21:41:47.832 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3046875242", 2024-09-21T21:41:47.832 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2946 2024-09-21T21:41:47.833 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.833 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:47.833 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.833 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:47.833 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865694+0000", 2024-09-21T21:41:47.833 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:47.833 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.833 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.833 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:47.833 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865694+0000", 2024-09-21T21:41:47.833 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:47.833 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.833 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.833 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:47.834 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865695+0000", 2024-09-21T21:41:47.834 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3299999999999998e-07 2024-09-21T21:41:47.834 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.834 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.834 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:47.834 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865697+0000", 2024-09-21T21:41:47.834 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.08e-06 2024-09-21T21:41:47.834 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.834 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.834 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:47.834 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865703+0000", 2024-09-21T21:41:47.834 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6840000000000001e-06 2024-09-21T21:41:47.834 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.834 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.834 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:47.835 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865708+0000", 2024-09-21T21:41:47.835 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7690000000000002e-06 2024-09-21T21:41:47.835 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.835 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.835 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:47.835 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.914670+0000", 2024-09-21T21:41:47.835 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.048961849000000002 2024-09-21T21:41:47.835 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.835 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.835 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:47.835 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.914753+0000", 2024-09-21T21:41:47.835 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2596000000000007e-05 2024-09-21T21:41:47.835 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.835 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.835 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:47.836 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.914858+0000", 2024-09-21T21:41:47.836 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000105163 2024-09-21T21:41:47.836 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:47.836 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:47.836 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:47.836 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.836 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.836 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5018.0:2947 3.15 3:af2e3e9a:::rbd_data.11213fc02c9d.0000000000000000:19 [read 499712~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-21T21:41:47.836 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:33.865714+0000", 2024-09-21T21:41:47.836 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.912699427, 2024-09-21T21:41:47.836 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.049362488000000003, 2024-09-21T21:41:47.836 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:47.836 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:47.836 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:47.837 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:47.837 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5018", 2024-09-21T21:41:47.837 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3046875242", 2024-09-21T21:41:47.837 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 2947 2024-09-21T21:41:47.837 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.837 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:47.837 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.837 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:47.837 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865714+0000", 2024-09-21T21:41:47.837 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:47.837 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.837 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.837 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:47.837 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865714+0000", 2024-09-21T21:41:47.837 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:47.838 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.838 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.838 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:47.838 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865714+0000", 2024-09-21T21:41:47.838 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.27e-07 2024-09-21T21:41:47.838 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.838 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.838 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:47.838 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865716+0000", 2024-09-21T21:41:47.838 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.8789999999999999e-06 2024-09-21T21:41:47.838 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.838 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.838 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:47.838 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865722+0000", 2024-09-21T21:41:47.839 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7930000000000003e-06 2024-09-21T21:41:47.839 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.839 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.839 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:47.839 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865728+0000", 2024-09-21T21:41:47.839 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7960000000000001e-06 2024-09-21T21:41:47.839 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.839 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.839 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:47.839 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.914902+0000", 2024-09-21T21:41:47.839 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.049174021999999998 2024-09-21T21:41:47.839 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.839 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.839 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:47.840 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.914983+0000", 2024-09-21T21:41:47.840 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1446999999999995e-05 2024-09-21T21:41:47.840 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.840 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.840 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:47.840 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.915077+0000", 2024-09-21T21:41:47.840 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3224000000000001e-05 2024-09-21T21:41:47.840 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:47.840 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:47.840 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:47.840 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.840 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.840 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5018.0:3004 3.15 3:aa7edf89:::rbd_object_map.1121c78ac9e5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e64)", 2024-09-21T21:41:47.840 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:33.865734+0000", 2024-09-21T21:41:47.840 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.912679884999999, 2024-09-21T21:41:47.841 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.052869125000000003, 2024-09-21T21:41:47.841 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:47.841 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:47.841 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:41:47.841 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:47.841 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5018", 2024-09-21T21:41:47.841 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3046875242", 2024-09-21T21:41:47.841 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 3004 2024-09-21T21:41:47.841 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.841 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:47.841 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.841 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:47.841 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865734+0000", 2024-09-21T21:41:47.841 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:47.841 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.842 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.842 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:47.842 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865734+0000", 2024-09-21T21:41:47.842 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:47.842 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.842 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.842 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:47.842 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865734+0000", 2024-09-21T21:41:47.842 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0999999999999999e-07 2024-09-21T21:41:47.842 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.842 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.842 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:47.842 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865737+0000", 2024-09-21T21:41:47.842 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6709999999999999e-06 2024-09-21T21:41:47.843 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.843 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.843 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:47.843 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865743+0000", 2024-09-21T21:41:47.843 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9109999999999998e-06 2024-09-21T21:41:47.843 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.843 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.843 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:47.843 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865747+0000", 2024-09-21T21:41:47.843 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4819999999999996e-06 2024-09-21T21:41:47.843 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.843 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.843 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:47.843 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.915114+0000", 2024-09-21T21:41:47.843 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.049367220000000003 2024-09-21T21:41:47.844 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.844 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.844 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:47.844 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.915203+0000", 2024-09-21T21:41:47.844 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9068999999999997e-05 2024-09-21T21:41:47.844 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.844 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.844 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:41:47.844 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.915603+0000", 2024-09-21T21:41:47.844 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00039933999999999998 2024-09-21T21:41:47.844 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.844 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.844 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:41:47.844 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.918432+0000", 2024-09-21T21:41:47.845 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0028295590000000002 2024-09-21T21:41:47.845 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.845 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.845 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:41:47.845 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.918489+0000", 2024-09-21T21:41:47.845 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6866000000000001e-05 2024-09-21T21:41:47.845 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.845 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.845 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:41:47.845 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.918550+0000", 2024-09-21T21:41:47.845 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0557000000000003e-05 2024-09-21T21:41:47.845 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.845 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.845 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:47.845 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.918603+0000", 2024-09-21T21:41:47.846 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.304e-05 2024-09-21T21:41:47.846 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:47.846 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:47.846 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:47.846 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.846 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.846 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5018.0:3005 3.15 3:aa7edf89:::rbd_object_map.1121c78ac9e5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e64)", 2024-09-21T21:41:47.846 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:33.865763+0000", 2024-09-21T21:41:47.846 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.912650161, 2024-09-21T21:41:47.846 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.052947052000000001, 2024-09-21T21:41:47.846 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:47.846 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:47.846 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:41:47.846 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:47.847 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5018", 2024-09-21T21:41:47.847 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3046875242", 2024-09-21T21:41:47.847 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 3005 2024-09-21T21:41:47.847 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.847 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:47.847 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.847 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:47.847 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865763+0000", 2024-09-21T21:41:47.847 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:47.847 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.847 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.847 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:47.847 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865763+0000", 2024-09-21T21:41:47.847 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:47.847 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.848 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.848 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:47.848 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865764+0000", 2024-09-21T21:41:47.848 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8099999999999998e-07 2024-09-21T21:41:47.848 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.848 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.848 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:47.848 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865766+0000", 2024-09-21T21:41:47.848 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6579999999999998e-06 2024-09-21T21:41:47.848 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.848 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.848 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:47.848 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865774+0000", 2024-09-21T21:41:47.848 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5660000000000001e-06 2024-09-21T21:41:47.848 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.849 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.849 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:47.849 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865779+0000", 2024-09-21T21:41:47.849 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9860000000000002e-06 2024-09-21T21:41:47.849 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.849 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.849 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:47.849 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.916225+0000", 2024-09-21T21:41:47.849 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.050445613 2024-09-21T21:41:47.849 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.849 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.849 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:47.849 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.916277+0000", 2024-09-21T21:41:47.849 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2380000000000003e-05 2024-09-21T21:41:47.849 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.850 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.850 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:41:47.850 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.916482+0000", 2024-09-21T21:41:47.850 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00020543699999999999 2024-09-21T21:41:47.850 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.850 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.850 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:41:47.850 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.918649+0000", 2024-09-21T21:41:47.850 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021668070000000002 2024-09-21T21:41:47.850 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.850 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.850 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:41:47.850 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.918658+0000", 2024-09-21T21:41:47.850 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3960000000000006e-06 2024-09-21T21:41:47.850 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.851 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.851 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:41:47.851 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.918686+0000", 2024-09-21T21:41:47.851 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8464000000000001e-05 2024-09-21T21:41:47.851 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.851 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.851 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:47.851 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.918710+0000", 2024-09-21T21:41:47.851 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4363999999999999e-05 2024-09-21T21:41:47.851 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:47.851 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:47.851 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:47.851 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.851 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.852 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5018.0:3006 3.15 3:aa7edf89:::rbd_object_map.1121c78ac9e5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e64)", 2024-09-21T21:41:47.852 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:33.865787+0000", 2024-09-21T21:41:47.852 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.912626511999999, 2024-09-21T21:41:47.852 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.055611471000000003, 2024-09-21T21:41:47.852 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:47.852 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:47.852 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:41:47.852 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:47.852 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5018", 2024-09-21T21:41:47.852 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3046875242", 2024-09-21T21:41:47.852 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 3006 2024-09-21T21:41:47.852 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.852 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:47.852 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.852 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:47.853 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865787+0000", 2024-09-21T21:41:47.853 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:47.853 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.853 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.853 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:47.853 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865787+0000", 2024-09-21T21:41:47.853 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:47.853 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.853 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.853 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:47.853 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865787+0000", 2024-09-21T21:41:47.853 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6600000000000002e-07 2024-09-21T21:41:47.853 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.853 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.854 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:47.854 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865790+0000", 2024-09-21T21:41:47.854 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4899999999999999e-06 2024-09-21T21:41:47.854 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.854 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.854 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:47.854 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865795+0000", 2024-09-21T21:41:47.854 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.045e-06 2024-09-21T21:41:47.854 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.854 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.854 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:47.854 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865799+0000", 2024-09-21T21:41:47.854 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7890000000000001e-06 2024-09-21T21:41:47.854 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.854 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.855 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:47.855 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.916851+0000", 2024-09-21T21:41:47.855 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.051052202999999997 2024-09-21T21:41:47.855 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.855 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.855 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:47.855 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.916938+0000", 2024-09-21T21:41:47.855 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7083999999999995e-05 2024-09-21T21:41:47.855 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.855 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.855 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:41:47.855 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.917291+0000", 2024-09-21T21:41:47.855 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00035321400000000002 2024-09-21T21:41:47.855 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.856 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.856 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:41:47.856 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.918717+0000", 2024-09-21T21:41:47.856 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001425548 2024-09-21T21:41:47.856 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.856 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.856 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:41:47.856 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.921291+0000", 2024-09-21T21:41:47.856 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025744460000000002 2024-09-21T21:41:47.856 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.856 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.856 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:41:47.856 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.921359+0000", 2024-09-21T21:41:47.856 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8267000000000002e-05 2024-09-21T21:41:47.857 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.857 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.857 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:47.857 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.921398+0000", 2024-09-21T21:41:47.857 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9019e-05 2024-09-21T21:41:47.857 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:47.857 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:47.857 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:47.857 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.857 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.857 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5018.0:3007 3.15 3:aa7edf89:::rbd_object_map.1121c78ac9e5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e64)", 2024-09-21T21:41:47.857 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:33.865803+0000", 2024-09-21T21:41:47.857 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.912610532, 2024-09-21T21:41:47.857 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.055713580999999998, 2024-09-21T21:41:47.857 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:47.858 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:47.858 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:41:47.858 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:47.858 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5018", 2024-09-21T21:41:47.858 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3046875242", 2024-09-21T21:41:47.858 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 3007 2024-09-21T21:41:47.858 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.858 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:47.858 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.858 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:47.858 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865803+0000", 2024-09-21T21:41:47.858 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:47.858 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.858 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.859 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:47.859 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865803+0000", 2024-09-21T21:41:47.859 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:47.859 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.859 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.859 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:47.859 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865803+0000", 2024-09-21T21:41:47.859 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.22e-07 2024-09-21T21:41:47.859 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.859 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.859 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:47.859 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865806+0000", 2024-09-21T21:41:47.859 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3099999999999999e-06 2024-09-21T21:41:47.859 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.860 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.860 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:47.860 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865811+0000", 2024-09-21T21:41:47.860 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3669999999999996e-06 2024-09-21T21:41:47.860 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.860 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.860 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:47.860 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865815+0000", 2024-09-21T21:41:47.860 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2180000000000001e-06 2024-09-21T21:41:47.860 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.860 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.860 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:47.860 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.917759+0000", 2024-09-21T21:41:47.860 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.051943277000000003 2024-09-21T21:41:47.860 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.861 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.861 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:47.861 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.917843+0000", 2024-09-21T21:41:47.861 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.4202999999999993e-05 2024-09-21T21:41:47.861 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.861 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.861 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:41:47.861 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.918148+0000", 2024-09-21T21:41:47.861 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000305524 2024-09-21T21:41:47.861 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.861 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.861 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:41:47.861 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.920498+0000", 2024-09-21T21:41:47.861 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0023499250000000001 2024-09-21T21:41:47.861 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.861 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.862 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:41:47.862 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.921455+0000", 2024-09-21T21:41:47.862 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00095657900000000004 2024-09-21T21:41:47.862 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.862 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.862 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:41:47.862 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.921484+0000", 2024-09-21T21:41:47.862 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9479999999999999e-05 2024-09-21T21:41:47.862 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.862 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.862 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:47.862 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.921517+0000", 2024-09-21T21:41:47.862 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2376000000000003e-05 2024-09-21T21:41:47.862 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:47.863 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:47.863 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:47.863 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.863 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.863 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5018.0:3009 3.15 3:aa7edf89:::rbd_object_map.1121c78ac9e5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-21T21:41:47.863 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:33.865859+0000", 2024-09-21T21:41:47.863 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.912554251, 2024-09-21T21:41:47.863 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.055778333999999999, 2024-09-21T21:41:47.863 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:47.863 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:47.863 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:41:47.863 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:47.863 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5018", 2024-09-21T21:41:47.863 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3046875242", 2024-09-21T21:41:47.863 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 3009 2024-09-21T21:41:47.864 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.864 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:47.864 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.864 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:47.864 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865859+0000", 2024-09-21T21:41:47.864 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:47.864 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.864 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.864 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:47.864 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865859+0000", 2024-09-21T21:41:47.864 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:47.864 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.864 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.864 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:47.864 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865860+0000", 2024-09-21T21:41:47.865 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5600000000000001e-07 2024-09-21T21:41:47.865 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.865 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.865 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:47.865 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865867+0000", 2024-09-21T21:41:47.865 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6890000000000004e-06 2024-09-21T21:41:47.865 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.865 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.865 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:47.865 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865874+0000", 2024-09-21T21:41:47.865 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1e-06 2024-09-21T21:41:47.865 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.865 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.865 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:47.866 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.865878+0000", 2024-09-21T21:41:47.866 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6929999999999998e-06 2024-09-21T21:41:47.866 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.866 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.866 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:47.866 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.918751+0000", 2024-09-21T21:41:47.866 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.052873113999999999 2024-09-21T21:41:47.866 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.866 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.866 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:47.866 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.918830+0000", 2024-09-21T21:41:47.866 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8862999999999994e-05 2024-09-21T21:41:47.866 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.866 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.867 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:41:47.867 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.919165+0000", 2024-09-21T21:41:47.867 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00033459599999999999 2024-09-21T21:41:47.867 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.867 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.867 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:41:47.867 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.921232+0000", 2024-09-21T21:41:47.867 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002067255 2024-09-21T21:41:47.867 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.867 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.867 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:41:47.867 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.921560+0000", 2024-09-21T21:41:47.867 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00032807300000000001 2024-09-21T21:41:47.867 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.867 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.868 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:41:47.868 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.921600+0000", 2024-09-21T21:41:47.868 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9421000000000001e-05 2024-09-21T21:41:47.868 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.868 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.868 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:47.868 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.921638+0000", 2024-09-21T21:41:47.868 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8074000000000002e-05 2024-09-21T21:41:47.868 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:47.868 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:47.868 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:47.868 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.868 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.868 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5018.0:3036 3.15 3:aa7edf89:::rbd_object_map.1121c78ac9e5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-21T21:41:47.868 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:33.866205+0000", 2024-09-21T21:41:47.869 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.912208545, 2024-09-21T21:41:47.869 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.057529386000000002, 2024-09-21T21:41:47.869 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:47.869 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:47.869 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:41:47.869 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:47.869 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5018", 2024-09-21T21:41:47.869 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3046875242", 2024-09-21T21:41:47.869 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 3036 2024-09-21T21:41:47.869 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.869 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:47.869 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.869 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:47.869 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.866205+0000", 2024-09-21T21:41:47.870 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:47.870 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.870 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.870 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:47.870 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.866205+0000", 2024-09-21T21:41:47.870 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:47.870 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.870 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.870 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:47.870 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.866206+0000", 2024-09-21T21:41:47.870 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5499999999999997e-07 2024-09-21T21:41:47.870 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.870 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.870 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:47.870 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.866211+0000", 2024-09-21T21:41:47.871 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8389999999999997e-06 2024-09-21T21:41:47.871 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.871 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.871 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:47.871 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.866223+0000", 2024-09-21T21:41:47.871 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2061e-05 2024-09-21T21:41:47.871 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.871 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.871 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:47.871 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.866230+0000", 2024-09-21T21:41:47.871 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7560000000000002e-06 2024-09-21T21:41:47.871 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.871 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.871 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:47.872 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.919617+0000", 2024-09-21T21:41:47.872 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.053387807000000002 2024-09-21T21:41:47.872 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.872 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.872 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:47.872 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.919700+0000", 2024-09-21T21:41:47.872 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2682999999999994e-05 2024-09-21T21:41:47.872 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.872 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.872 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:41:47.872 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.920067+0000", 2024-09-21T21:41:47.872 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000367408 2024-09-21T21:41:47.872 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.872 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.872 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:41:47.873 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.921529+0000", 2024-09-21T21:41:47.873 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0014611839999999999 2024-09-21T21:41:47.873 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.873 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.873 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:41:47.873 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.923641+0000", 2024-09-21T21:41:47.873 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021122670000000001 2024-09-21T21:41:47.873 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.873 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.873 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:41:47.873 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.923696+0000", 2024-09-21T21:41:47.873 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4756000000000002e-05 2024-09-21T21:41:47.873 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.873 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.873 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:47.874 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.923734+0000", 2024-09-21T21:41:47.874 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8770000000000003e-05 2024-09-21T21:41:47.874 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:47.874 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:47.874 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:47.874 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.874 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.874 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5018.0:3125 3.15 3:aa7edf89:::rbd_object_map.1121c78ac9e5:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-21T21:41:47.874 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:33.871152+0000", 2024-09-21T21:41:47.874 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.907261769, 2024-09-21T21:41:47.874 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.052772614000000002, 2024-09-21T21:41:47.874 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:47.874 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:47.874 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:41:47.875 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:47.875 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5018", 2024-09-21T21:41:47.875 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3046875242", 2024-09-21T21:41:47.875 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 3125 2024-09-21T21:41:47.875 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.875 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:47.875 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.875 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:47.875 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.871152+0000", 2024-09-21T21:41:47.875 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:47.875 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.875 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.875 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:47.875 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.871152+0000", 2024-09-21T21:41:47.875 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:47.876 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.876 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.876 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:47.876 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.871153+0000", 2024-09-21T21:41:47.876 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.4e-07 2024-09-21T21:41:47.876 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.876 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.876 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:47.876 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.871157+0000", 2024-09-21T21:41:47.876 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1069999999999998e-06 2024-09-21T21:41:47.876 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.876 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.876 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:47.876 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.912625+0000", 2024-09-21T21:41:47.876 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041468659999999997 2024-09-21T21:41:47.877 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.877 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.877 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:47.877 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.912644+0000", 2024-09-21T21:41:47.877 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.8652e-05 2024-09-21T21:41:47.877 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.877 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.877 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:47.877 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.920531+0000", 2024-09-21T21:41:47.877 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0078870910000000006 2024-09-21T21:41:47.877 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.877 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.877 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:47.877 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.920612+0000", 2024-09-21T21:41:47.878 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0933999999999995e-05 2024-09-21T21:41:47.878 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.878 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.878 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:41:47.878 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.920926+0000", 2024-09-21T21:41:47.878 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00031363700000000002 2024-09-21T21:41:47.878 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.878 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.878 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:41:47.878 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.923748+0000", 2024-09-21T21:41:47.878 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0028222479999999999 2024-09-21T21:41:47.878 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.878 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.878 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:41:47.879 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.923802+0000", 2024-09-21T21:41:47.879 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3912999999999997e-05 2024-09-21T21:41:47.879 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.879 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.879 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:41:47.879 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.923857+0000", 2024-09-21T21:41:47.879 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5312000000000002e-05 2024-09-21T21:41:47.879 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:47.879 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:47.879 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:47.879 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:33.923924+0000", 2024-09-21T21:41:47.879 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7219999999999997e-05 2024-09-21T21:41:47.879 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:47.879 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:47.879 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:47.880 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:47.880 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:47.880 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:47.881 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T21:41:47.881 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:47.880+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:47.930 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T21:41:47.981 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:47.980+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:48.025 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:48.025 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:41:48.025 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-21T21:41:48.025 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:48.034 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T21:41:48.049 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0: {} 2024-09-21T21:41:48.066 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1: {} 2024-09-21T21:41:48.066 INFO:teuthology.orchestra.run.smithi081.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T21:41:48.082 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2: {} 2024-09-21T21:41:48.082 INFO:teuthology.orchestra.run.smithi081.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T21:41:48.082 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:48.080+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:48.098 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3: {} 2024-09-21T21:41:48.098 INFO:teuthology.orchestra.run.smithi081.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T21:41:48.114 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4: {} 2024-09-21T21:41:48.115 INFO:teuthology.orchestra.run.smithi081.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T21:41:48.131 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5: {} 2024-09-21T21:41:48.131 INFO:teuthology.orchestra.run.smithi081.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T21:41:48.177 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:48.177 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:48.182 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:48.181+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:48.185 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T21:41:48.247 INFO:teuthology.orchestra.run.smithi081.stderr:marked in osd.3. 2024-09-21T21:41:48.253 INFO:teuthology.orchestra.run.smithi081.stderr:noscrub is set 2024-09-21T21:41:48.265 INFO:tasks.thrashosds.thrasher:Added osd 3 2024-09-21T21:41:48.293 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:48.293 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:48.302 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T21:41:48.310 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:48.309+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:48.384 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:48.383+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:48.405 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:48.405 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:48.414 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T21:41:48.498 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:48.497+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:48.558 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:48.558 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:48.567 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T21:41:48.608 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:48.606+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:48.687 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:48.685+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:48.759 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:48.759 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:48.760 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T21:41:48.810 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:48.809+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:48.868 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:48.868 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:48.876 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T21:41:48.888 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:48.886+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:48.988 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:48.987+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:49.020 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:49.020 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:49.028 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T21:41:49.112 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:49.111+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:49.172 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:49.172 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:49.181 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T21:41:49.190 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:49.189+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:49.291 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:49.290+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:49.329 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:49.329 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:49.337 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T21:41:49.392 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:49.391+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:49.481 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:49.481 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:49.489 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T21:41:49.492 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:49.491+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:49.591 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:49.591 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:49.592 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:49.591+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:49.600 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T21:41:49.693 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:49.691+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:49.702 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:49.702 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:49.711 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T21:41:49.795 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:49.794+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:49.853 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:49.854 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:49.862 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T21:41:49.895 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:49.894+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:49.998 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:49.997+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:50.005 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:50.005 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:50.014 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T21:41:50.100 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:50.099+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:50.134 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:50.134 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:50.142 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T21:41:50.144 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T21:41:50.202 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:50.201+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:50.249 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:50.250 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:50.259 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T21:41:50.267 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0: {} 2024-09-21T21:41:50.268 INFO:teuthology.orchestra.run.smithi081.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T21:41:50.270 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T21:41:50.287 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1: {} 2024-09-21T21:41:50.287 INFO:teuthology.orchestra.run.smithi081.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T21:41:50.306 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:50.305+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:50.367 INFO:teuthology.orchestra.run.smithi164.stdout:flattening image image_client.0-clone30 2024-09-21T21:41:50.383 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:50.383 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:50.393 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T21:41:50.434 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:50.433+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:50.506 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:50.505+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:50.513 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2: {} 2024-09-21T21:41:50.513 INFO:teuthology.orchestra.run.smithi081.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T21:41:50.535 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3: {} 2024-09-21T21:41:50.535 INFO:teuthology.orchestra.run.smithi081.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T21:41:50.547 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:50.548 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:50.558 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T21:41:50.559 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4: {} 2024-09-21T21:41:50.559 INFO:teuthology.orchestra.run.smithi081.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T21:41:50.586 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5: {} 2024-09-21T21:41:50.586 INFO:teuthology.orchestra.run.smithi081.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T21:41:50.595 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-21T21:41:50.594+0000 7fbdf7fff640 0 --2- 172.21.15.81:0/1737889002 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7fbe000703d0 0x7fbe000707f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-21T21:41:50.643 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:50.641+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:50.707 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:50.707 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:41:50.707 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:41:50.707 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:41:50.707 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:50.710 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:50.708+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:50.716 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T21:41:50.810 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:50.809+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:50.831 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:50.831 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-21T21:41:50.831 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-21T21:41:50.832 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:41:50.832 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.832 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6974 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5246976~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:50.832 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.922932+0000", 2024-09-21T21:41:50.832 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 20.906201652, 2024-09-21T21:41:50.832 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041773352999999999, 2024-09-21T21:41:50.832 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:50.832 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:50.832 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:50.832 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:50.832 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:50.832 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:50.833 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6974 2024-09-21T21:41:50.833 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.833 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:50.833 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.833 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:50.833 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922932+0000", 2024-09-21T21:41:50.833 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:50.833 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.833 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.833 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:50.833 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922932+0000", 2024-09-21T21:41:50.833 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:50.833 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.833 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.834 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:50.834 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922933+0000", 2024-09-21T21:41:50.834 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.46e-07 2024-09-21T21:41:50.834 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.834 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.834 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:50.834 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922940+0000", 2024-09-21T21:41:50.834 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6140000000000003e-06 2024-09-21T21:41:50.834 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.834 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.834 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:50.834 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922949+0000", 2024-09-21T21:41:50.834 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6899999999999998e-06 2024-09-21T21:41:50.835 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.835 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.835 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:50.835 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922955+0000", 2024-09-21T21:41:50.835 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5140000000000001e-06 2024-09-21T21:41:50.835 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.835 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.835 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:50.835 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964547+0000", 2024-09-21T21:41:50.835 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041592854999999998 2024-09-21T21:41:50.835 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.835 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.835 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:50.835 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964617+0000", 2024-09-21T21:41:50.836 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9058000000000002e-05 2024-09-21T21:41:50.836 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.836 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.836 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:50.836 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964706+0000", 2024-09-21T21:41:50.836 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9276000000000004e-05 2024-09-21T21:41:50.836 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:50.836 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:50.836 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:50.836 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.836 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.836 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6976 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5261312~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:50.836 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.922961+0000", 2024-09-21T21:41:50.836 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 20.906172606999998, 2024-09-21T21:41:50.836 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041863648000000003, 2024-09-21T21:41:50.837 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:50.837 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:50.837 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:50.837 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:50.837 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:50.837 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:50.837 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6976 2024-09-21T21:41:50.837 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.837 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:50.837 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.837 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:50.837 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922961+0000", 2024-09-21T21:41:50.837 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:50.837 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.838 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.838 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:50.838 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922961+0000", 2024-09-21T21:41:50.838 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:50.838 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.838 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.838 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:50.838 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922962+0000", 2024-09-21T21:41:50.838 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.89e-07 2024-09-21T21:41:50.838 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.838 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.838 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:50.838 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922964+0000", 2024-09-21T21:41:50.838 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1270000000000001e-06 2024-09-21T21:41:50.839 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.839 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.839 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:50.839 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922970+0000", 2024-09-21T21:41:50.839 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8030000000000002e-06 2024-09-21T21:41:50.839 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.839 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.839 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:50.839 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922979+0000", 2024-09-21T21:41:50.839 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5559999999999996e-06 2024-09-21T21:41:50.839 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.839 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.839 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:50.839 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964734+0000", 2024-09-21T21:41:50.840 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041754415000000003 2024-09-21T21:41:50.840 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.840 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.840 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:50.840 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964775+0000", 2024-09-21T21:41:50.840 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0806999999999997e-05 2024-09-21T21:41:50.840 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.840 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.840 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:50.840 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964825+0000", 2024-09-21T21:41:50.840 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0550999999999998e-05 2024-09-21T21:41:50.840 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:50.840 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:50.840 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:50.840 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.841 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.841 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6978 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5275648~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:50.841 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.922987+0000", 2024-09-21T21:41:50.841 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 20.906146822, 2024-09-21T21:41:50.841 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042043876000000001, 2024-09-21T21:41:50.841 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:50.841 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:50.841 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:50.841 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:50.841 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:50.841 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:50.841 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6978 2024-09-21T21:41:50.841 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.841 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:50.841 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.842 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:50.842 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922987+0000", 2024-09-21T21:41:50.842 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:50.842 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.842 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.842 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:50.842 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922987+0000", 2024-09-21T21:41:50.842 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:50.842 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.842 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.842 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:50.842 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922988+0000", 2024-09-21T21:41:50.842 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1199999999999998e-07 2024-09-21T21:41:50.842 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.843 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.843 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:50.843 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922990+0000", 2024-09-21T21:41:50.843 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.142e-06 2024-09-21T21:41:50.843 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.843 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.843 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:50.843 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922997+0000", 2024-09-21T21:41:50.843 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.199e-06 2024-09-21T21:41:50.843 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.843 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.843 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:50.843 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923004+0000", 2024-09-21T21:41:50.843 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3599999999999998e-06 2024-09-21T21:41:50.844 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.844 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.844 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:50.844 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964856+0000", 2024-09-21T21:41:50.844 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041851521000000003 2024-09-21T21:41:50.844 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.844 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.844 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:50.844 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964944+0000", 2024-09-21T21:41:50.844 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7859e-05 2024-09-21T21:41:50.844 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.844 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.844 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:50.844 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965031+0000", 2024-09-21T21:41:50.844 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7383e-05 2024-09-21T21:41:50.845 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:50.845 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:50.845 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:50.845 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.845 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.845 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6980 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5289984~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:50.845 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923011+0000", 2024-09-21T21:41:50.845 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 20.906123484999998, 2024-09-21T21:41:50.845 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042139337999999998, 2024-09-21T21:41:50.845 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:50.845 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:50.845 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:50.845 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:50.845 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:50.846 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:50.846 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6980 2024-09-21T21:41:50.846 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.846 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:50.846 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.846 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:50.846 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923011+0000", 2024-09-21T21:41:50.846 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:50.846 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.846 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.846 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:50.846 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923011+0000", 2024-09-21T21:41:50.846 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:50.847 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.847 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.847 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:50.847 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923011+0000", 2024-09-21T21:41:50.847 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6699999999999999e-07 2024-09-21T21:41:50.847 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.847 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.847 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:50.847 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923013+0000", 2024-09-21T21:41:50.847 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0279999999999999e-06 2024-09-21T21:41:50.847 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.847 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.847 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:50.847 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923018+0000", 2024-09-21T21:41:50.847 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4790000000000004e-06 2024-09-21T21:41:50.848 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.848 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.848 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:50.848 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923029+0000", 2024-09-21T21:41:50.848 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0601999999999999e-05 2024-09-21T21:41:50.848 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.848 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.848 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:50.848 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965059+0000", 2024-09-21T21:41:50.848 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042029947999999998 2024-09-21T21:41:50.848 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.848 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.848 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:50.848 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965100+0000", 2024-09-21T21:41:50.849 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1170000000000001e-05 2024-09-21T21:41:50.849 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.849 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.849 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:50.849 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965150+0000", 2024-09-21T21:41:50.849 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9743999999999997e-05 2024-09-21T21:41:50.849 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:50.849 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:50.849 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:50.849 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.849 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.849 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6982 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5304320~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:50.849 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923036+0000", 2024-09-21T21:41:50.850 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 20.906097664000001, 2024-09-21T21:41:50.850 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042225666000000002, 2024-09-21T21:41:50.850 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:50.850 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:50.850 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:50.850 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:50.850 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:50.850 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:50.850 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6982 2024-09-21T21:41:50.850 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.850 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:50.850 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.850 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:50.850 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923036+0000", 2024-09-21T21:41:50.850 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:50.851 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.851 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.851 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:50.851 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923036+0000", 2024-09-21T21:41:50.851 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:50.851 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.851 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.851 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:50.851 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923037+0000", 2024-09-21T21:41:50.851 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2800000000000002e-07 2024-09-21T21:41:50.851 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.851 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.851 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:50.851 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923039+0000", 2024-09-21T21:41:50.852 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1500000000000002e-06 2024-09-21T21:41:50.852 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.852 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.852 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:50.852 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923049+0000", 2024-09-21T21:41:50.852 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0236e-05 2024-09-21T21:41:50.852 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.852 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.852 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:50.852 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923056+0000", 2024-09-21T21:41:50.852 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2540000000000003e-06 2024-09-21T21:41:50.852 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.852 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.852 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:50.853 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965173+0000", 2024-09-21T21:41:50.853 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042116686 2024-09-21T21:41:50.853 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.853 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.853 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:50.853 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965215+0000", 2024-09-21T21:41:50.853 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1696000000000002e-05 2024-09-21T21:41:50.853 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.853 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.853 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:50.853 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965262+0000", 2024-09-21T21:41:50.853 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7216000000000003e-05 2024-09-21T21:41:50.853 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:50.853 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:50.853 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:50.854 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.854 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.854 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6984 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5318656~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:50.854 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923063+0000", 2024-09-21T21:41:50.854 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 20.906071010000002, 2024-09-21T21:41:50.854 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042318651999999998, 2024-09-21T21:41:50.854 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:50.854 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:50.854 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:50.854 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:50.854 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:50.854 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:50.854 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6984 2024-09-21T21:41:50.854 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.855 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:50.855 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.855 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:50.855 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923063+0000", 2024-09-21T21:41:50.855 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:50.855 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.855 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.855 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:50.855 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923063+0000", 2024-09-21T21:41:50.855 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:50.855 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.855 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.855 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:50.855 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923064+0000", 2024-09-21T21:41:50.855 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2e-07 2024-09-21T21:41:50.856 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.856 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.856 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:50.856 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923066+0000", 2024-09-21T21:41:50.856 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.057e-06 2024-09-21T21:41:50.856 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.856 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.856 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:50.856 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923071+0000", 2024-09-21T21:41:50.856 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8810000000000001e-06 2024-09-21T21:41:50.856 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.856 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.856 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:50.856 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923078+0000", 2024-09-21T21:41:50.856 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5479999999999998e-06 2024-09-21T21:41:50.857 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.857 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.857 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:50.857 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965288+0000", 2024-09-21T21:41:50.857 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042209576999999998 2024-09-21T21:41:50.857 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.857 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.857 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:50.857 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965332+0000", 2024-09-21T21:41:50.857 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4366999999999997e-05 2024-09-21T21:41:50.857 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.857 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.857 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:50.857 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965382+0000", 2024-09-21T21:41:50.857 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9802000000000002e-05 2024-09-21T21:41:50.858 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:50.858 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:50.858 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:50.858 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.858 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.858 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6987 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5332992~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:50.858 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923085+0000", 2024-09-21T21:41:50.858 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 20.906049485, 2024-09-21T21:41:50.858 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042412103999999999, 2024-09-21T21:41:50.858 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:50.858 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:50.858 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:50.858 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:50.858 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:50.859 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:50.859 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6987 2024-09-21T21:41:50.859 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.859 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:50.859 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.859 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:50.859 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923085+0000", 2024-09-21T21:41:50.859 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:50.859 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.859 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.859 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:50.859 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923085+0000", 2024-09-21T21:41:50.859 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:50.859 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.860 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.860 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:50.860 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923085+0000", 2024-09-21T21:41:50.860 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.89e-07 2024-09-21T21:41:50.860 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.860 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.860 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:50.860 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923087+0000", 2024-09-21T21:41:50.860 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0810000000000001e-06 2024-09-21T21:41:50.860 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.860 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.860 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:50.860 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923098+0000", 2024-09-21T21:41:50.860 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1299e-05 2024-09-21T21:41:50.860 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.861 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.861 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:50.861 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923104+0000", 2024-09-21T21:41:50.861 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9100000000000002e-06 2024-09-21T21:41:50.861 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.861 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.861 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:50.861 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965404+0000", 2024-09-21T21:41:50.861 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042299622000000002 2024-09-21T21:41:50.861 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.861 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.861 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:50.861 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965445+0000", 2024-09-21T21:41:50.861 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0701000000000002e-05 2024-09-21T21:41:50.862 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.862 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.862 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:50.862 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965497+0000", 2024-09-21T21:41:50.862 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2101999999999997e-05 2024-09-21T21:41:50.862 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:50.862 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:50.862 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:50.862 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.862 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.862 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6989 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5347328~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:50.862 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923111+0000", 2024-09-21T21:41:50.862 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 20.906023144999999, 2024-09-21T21:41:50.862 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042505188999999999, 2024-09-21T21:41:50.863 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:50.863 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:50.863 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:50.863 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:50.863 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:50.863 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:50.863 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6989 2024-09-21T21:41:50.863 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.863 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:50.863 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.863 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:50.863 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923111+0000", 2024-09-21T21:41:50.863 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:50.863 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.864 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.864 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:50.864 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923111+0000", 2024-09-21T21:41:50.864 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:50.864 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.864 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.864 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:50.864 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923111+0000", 2024-09-21T21:41:50.864 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4499999999999997e-07 2024-09-21T21:41:50.864 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.864 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.864 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:50.864 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923114+0000", 2024-09-21T21:41:50.864 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1569999999999998e-06 2024-09-21T21:41:50.864 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.865 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.865 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:50.865 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923120+0000", 2024-09-21T21:41:50.865 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1759999999999998e-06 2024-09-21T21:41:50.865 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.865 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.865 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:50.865 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923125+0000", 2024-09-21T21:41:50.865 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2560000000000002e-06 2024-09-21T21:41:50.865 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.865 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.865 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:50.865 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965523+0000", 2024-09-21T21:41:50.865 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042398094999999997 2024-09-21T21:41:50.866 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.866 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.866 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:50.866 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965566+0000", 2024-09-21T21:41:50.866 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3393999999999998e-05 2024-09-21T21:41:50.866 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.866 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.866 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:50.866 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965616+0000", 2024-09-21T21:41:50.866 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9666000000000002e-05 2024-09-21T21:41:50.866 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:50.866 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:50.866 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:50.866 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.866 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.867 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6991 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5361664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:50.867 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923132+0000", 2024-09-21T21:41:50.867 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 20.906001856, 2024-09-21T21:41:50.867 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042600779999999998, 2024-09-21T21:41:50.867 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:50.867 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:50.867 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:50.867 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:50.867 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:50.867 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:50.867 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6991 2024-09-21T21:41:50.867 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.867 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:50.867 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.868 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:50.868 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923132+0000", 2024-09-21T21:41:50.868 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:50.868 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.868 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.868 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:50.868 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923132+0000", 2024-09-21T21:41:50.868 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:50.868 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.868 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.868 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:50.868 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923132+0000", 2024-09-21T21:41:50.868 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4200000000000002e-07 2024-09-21T21:41:50.868 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.869 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.869 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:50.869 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923134+0000", 2024-09-21T21:41:50.869 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2270000000000001e-06 2024-09-21T21:41:50.869 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.869 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.869 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:50.869 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923137+0000", 2024-09-21T21:41:50.869 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1930000000000002e-06 2024-09-21T21:41:50.869 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.869 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.869 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:50.869 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923139+0000", 2024-09-21T21:41:50.869 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4710000000000001e-06 2024-09-21T21:41:50.870 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.870 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.870 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:50.870 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965639+0000", 2024-09-21T21:41:50.870 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.04249965 2024-09-21T21:41:50.870 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.870 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.870 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:50.870 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965679+0000", 2024-09-21T21:41:50.870 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9852999999999999e-05 2024-09-21T21:41:50.870 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.870 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.870 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:50.870 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965733+0000", 2024-09-21T21:41:50.870 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4144000000000002e-05 2024-09-21T21:41:50.871 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:50.871 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:50.871 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:50.871 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.871 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.871 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6993 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5384192~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:50.872 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923143+0000", 2024-09-21T21:41:50.872 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 20.905991061999998, 2024-09-21T21:41:50.872 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042708182999999997, 2024-09-21T21:41:50.872 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:50.872 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:50.872 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:50.873 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:50.873 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:50.873 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:50.873 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6993 2024-09-21T21:41:50.873 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.873 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:50.873 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.873 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:50.873 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923143+0000", 2024-09-21T21:41:50.873 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:50.874 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.874 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.874 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:50.874 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923143+0000", 2024-09-21T21:41:50.874 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:50.874 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.874 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.874 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:50.874 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923143+0000", 2024-09-21T21:41:50.874 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0800000000000001e-07 2024-09-21T21:41:50.874 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.874 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.874 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:50.875 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923144+0000", 2024-09-21T21:41:50.875 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.009e-06 2024-09-21T21:41:50.875 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.875 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.875 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:50.875 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923148+0000", 2024-09-21T21:41:50.875 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.737e-06 2024-09-21T21:41:50.875 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.875 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.875 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:50.875 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923150+0000", 2024-09-21T21:41:50.875 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4669999999999998e-06 2024-09-21T21:41:50.875 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.875 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.876 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:50.876 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965760+0000", 2024-09-21T21:41:50.876 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042609399999999999 2024-09-21T21:41:50.876 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.876 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.876 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:50.876 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965801+0000", 2024-09-21T21:41:50.876 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1399999999999997e-05 2024-09-21T21:41:50.876 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.876 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.876 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:50.876 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965851+0000", 2024-09-21T21:41:50.876 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9962e-05 2024-09-21T21:41:50.876 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:50.876 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:50.877 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:50.877 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.877 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.877 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6995 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5398528~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:50.877 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923153+0000", 2024-09-21T21:41:50.877 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 20.905980698, 2024-09-21T21:41:50.877 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042828034000000001, 2024-09-21T21:41:50.877 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:50.877 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:50.877 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:50.877 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:50.877 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:50.877 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:50.877 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6995 2024-09-21T21:41:50.878 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.878 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:50.878 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.878 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:50.878 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923153+0000", 2024-09-21T21:41:50.878 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:50.878 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.878 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.878 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:50.878 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923153+0000", 2024-09-21T21:41:50.878 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:50.878 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.878 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.878 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:50.879 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923154+0000", 2024-09-21T21:41:50.879 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2600000000000001e-07 2024-09-21T21:41:50.879 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.879 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.879 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:50.879 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923158+0000", 2024-09-21T21:41:50.879 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0559999999999998e-06 2024-09-21T21:41:50.879 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.879 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.879 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:50.879 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923161+0000", 2024-09-21T21:41:50.879 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6519999999999999e-06 2024-09-21T21:41:50.879 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.879 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.879 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:50.880 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923170+0000", 2024-09-21T21:41:50.880 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5720000000000003e-06 2024-09-21T21:41:50.880 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.880 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.880 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:50.880 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965887+0000", 2024-09-21T21:41:50.880 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042716863000000001 2024-09-21T21:41:50.880 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.880 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.880 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:50.880 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965927+0000", 2024-09-21T21:41:50.880 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0346999999999998e-05 2024-09-21T21:41:50.880 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.880 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.880 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:50.881 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965981+0000", 2024-09-21T21:41:50.881 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4317999999999997e-05 2024-09-21T21:41:50.881 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:50.881 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:50.882 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:50.882 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.882 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.882 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6997 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5412864~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:50.882 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923174+0000", 2024-09-21T21:41:50.882 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 20.905960542999999, 2024-09-21T21:41:50.882 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042928296999999997, 2024-09-21T21:41:50.882 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:50.882 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:50.882 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:50.882 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:50.883 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:50.883 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:50.883 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6997 2024-09-21T21:41:50.883 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.883 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:50.883 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.883 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:50.883 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923174+0000", 2024-09-21T21:41:50.883 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:50.883 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.883 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.883 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:50.883 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923174+0000", 2024-09-21T21:41:50.883 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:50.883 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.884 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.884 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:50.884 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923174+0000", 2024-09-21T21:41:50.884 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.67e-07 2024-09-21T21:41:50.884 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.884 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.884 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:50.884 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923178+0000", 2024-09-21T21:41:50.884 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1330000000000001e-06 2024-09-21T21:41:50.884 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.884 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.884 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:50.884 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923182+0000", 2024-09-21T21:41:50.884 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.794e-06 2024-09-21T21:41:50.885 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.885 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.885 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:50.885 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923184+0000", 2024-09-21T21:41:50.885 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.435e-06 2024-09-21T21:41:50.885 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.885 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.885 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:50.885 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966009+0000", 2024-09-21T21:41:50.885 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042824327000000002 2024-09-21T21:41:50.885 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.885 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.885 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:50.885 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966052+0000", 2024-09-21T21:41:50.886 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3034000000000002e-05 2024-09-21T21:41:50.886 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.886 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.886 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:50.886 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966102+0000", 2024-09-21T21:41:50.886 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0306999999999998e-05 2024-09-21T21:41:50.886 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:50.886 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:50.886 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:50.886 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.886 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.886 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6999 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5427200~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:50.886 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923187+0000", 2024-09-21T21:41:50.886 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 20.905946815, 2024-09-21T21:41:50.887 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043104267000000002, 2024-09-21T21:41:50.887 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:50.887 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:50.887 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:50.887 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:50.887 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:50.887 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:50.887 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6999 2024-09-21T21:41:50.887 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.887 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:50.887 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.887 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:50.887 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923187+0000", 2024-09-21T21:41:50.887 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:50.887 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.888 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.888 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:50.888 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923187+0000", 2024-09-21T21:41:50.888 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:50.888 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.888 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.888 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:50.888 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923188+0000", 2024-09-21T21:41:50.888 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3999999999999998e-07 2024-09-21T21:41:50.888 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.888 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.888 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:50.888 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923189+0000", 2024-09-21T21:41:50.888 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4169999999999999e-06 2024-09-21T21:41:50.888 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.889 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.889 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:50.889 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923194+0000", 2024-09-21T21:41:50.889 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1819999999999997e-06 2024-09-21T21:41:50.889 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.889 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.889 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:50.889 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923198+0000", 2024-09-21T21:41:50.889 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.143e-06 2024-09-21T21:41:50.889 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.889 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.889 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:50.889 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966135+0000", 2024-09-21T21:41:50.889 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042936297999999998 2024-09-21T21:41:50.890 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.890 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.890 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:50.890 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966204+0000", 2024-09-21T21:41:50.890 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8989e-05 2024-09-21T21:41:50.890 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.890 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.890 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:50.890 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966292+0000", 2024-09-21T21:41:50.890 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7997999999999993e-05 2024-09-21T21:41:50.890 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:50.890 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:50.890 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:50.890 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.891 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.891 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7001 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5441536~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:50.891 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923211+0000", 2024-09-21T21:41:50.891 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 20.905923024, 2024-09-21T21:41:50.891 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043196518000000003, 2024-09-21T21:41:50.891 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:50.891 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:50.891 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:50.891 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:50.891 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:50.891 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:50.891 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7001 2024-09-21T21:41:50.891 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.891 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:50.891 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.892 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:50.892 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923211+0000", 2024-09-21T21:41:50.892 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:50.892 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.892 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.892 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:50.892 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923211+0000", 2024-09-21T21:41:50.892 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:50.892 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.892 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.892 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:50.892 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923212+0000", 2024-09-21T21:41:50.892 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3900000000000005e-07 2024-09-21T21:41:50.892 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.893 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.893 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:50.893 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923214+0000", 2024-09-21T21:41:50.893 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9970000000000001e-06 2024-09-21T21:41:50.893 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.893 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.893 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:50.893 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923219+0000", 2024-09-21T21:41:50.893 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4310000000000002e-06 2024-09-21T21:41:50.893 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.893 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.893 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:50.893 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923224+0000", 2024-09-21T21:41:50.893 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6410000000000001e-06 2024-09-21T21:41:50.893 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.894 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.894 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:50.894 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966318+0000", 2024-09-21T21:41:50.894 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043093842 2024-09-21T21:41:50.894 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.894 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.894 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:50.894 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966359+0000", 2024-09-21T21:41:50.894 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1260999999999997e-05 2024-09-21T21:41:50.894 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.894 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.894 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:50.894 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966408+0000", 2024-09-21T21:41:50.894 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8807000000000002e-05 2024-09-21T21:41:50.895 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:50.895 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:50.895 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:50.895 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.895 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.895 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7002 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5455872~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:50.895 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923229+0000", 2024-09-21T21:41:50.895 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 20.905904614000001, 2024-09-21T21:41:50.895 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043364568999999999, 2024-09-21T21:41:50.895 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:50.895 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:50.895 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:50.895 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:50.895 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:50.895 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:50.896 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7002 2024-09-21T21:41:50.896 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.896 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:50.896 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.896 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:50.896 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923229+0000", 2024-09-21T21:41:50.896 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:50.896 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.896 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.896 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:50.896 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923229+0000", 2024-09-21T21:41:50.896 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:50.896 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.896 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.896 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:50.897 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923230+0000", 2024-09-21T21:41:50.897 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4700000000000002e-07 2024-09-21T21:41:50.897 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.897 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.897 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:50.897 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923232+0000", 2024-09-21T21:41:50.897 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0719999999999998e-06 2024-09-21T21:41:50.897 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.897 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.897 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:50.897 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923237+0000", 2024-09-21T21:41:50.897 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2290000000000002e-06 2024-09-21T21:41:50.897 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.897 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.898 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:50.898 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923245+0000", 2024-09-21T21:41:50.898 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2959999999999999e-06 2024-09-21T21:41:50.898 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.898 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.898 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:50.898 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966438+0000", 2024-09-21T21:41:50.898 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043193027000000002 2024-09-21T21:41:50.898 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.898 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.898 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:50.898 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966506+0000", 2024-09-21T21:41:50.898 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7707999999999996e-05 2024-09-21T21:41:50.898 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.898 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.899 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:50.899 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966594+0000", 2024-09-21T21:41:50.899 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.789e-05 2024-09-21T21:41:50.899 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:50.899 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:50.899 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:50.899 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.899 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.899 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7005 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5470208~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:50.899 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923254+0000", 2024-09-21T21:41:50.899 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 20.905880383, 2024-09-21T21:41:50.899 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043457176, 2024-09-21T21:41:50.899 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:50.899 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:50.900 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:50.900 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:50.900 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:50.900 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:50.900 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7005 2024-09-21T21:41:50.900 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.900 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:50.900 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.900 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:50.900 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923254+0000", 2024-09-21T21:41:50.900 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:50.900 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.900 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.900 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:50.900 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923254+0000", 2024-09-21T21:41:50.901 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:50.901 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.901 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.901 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:50.901 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923254+0000", 2024-09-21T21:41:50.901 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4700000000000002e-07 2024-09-21T21:41:50.901 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.901 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.901 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:50.901 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923256+0000", 2024-09-21T21:41:50.901 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.007e-06 2024-09-21T21:41:50.901 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.901 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.901 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:50.902 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923266+0000", 2024-09-21T21:41:50.902 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.4700000000000008e-06 2024-09-21T21:41:50.902 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.902 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.902 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:50.902 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923271+0000", 2024-09-21T21:41:50.902 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2809999999999999e-06 2024-09-21T21:41:50.902 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.902 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.902 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:50.902 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966619+0000", 2024-09-21T21:41:50.902 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043347951000000003 2024-09-21T21:41:50.902 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.902 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.903 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:50.903 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966661+0000", 2024-09-21T21:41:50.903 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2614999999999998e-05 2024-09-21T21:41:50.903 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.903 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.903 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:50.903 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966711+0000", 2024-09-21T21:41:50.903 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9505000000000001e-05 2024-09-21T21:41:50.903 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:50.903 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:50.903 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:50.903 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.903 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.903 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7008 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5484544~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:50.904 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923278+0000", 2024-09-21T21:41:50.904 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 20.905855735999999, 2024-09-21T21:41:50.904 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043631071, 2024-09-21T21:41:50.904 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:50.904 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:50.904 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:50.904 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:50.904 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:50.904 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:50.904 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7008 2024-09-21T21:41:50.904 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.904 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:50.904 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.904 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:50.905 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923278+0000", 2024-09-21T21:41:50.905 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:50.905 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.905 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.905 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:50.905 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923278+0000", 2024-09-21T21:41:50.905 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:50.905 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.905 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.905 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:50.905 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923279+0000", 2024-09-21T21:41:50.905 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.77e-07 2024-09-21T21:41:50.905 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.905 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.906 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:50.906 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923281+0000", 2024-09-21T21:41:50.906 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0219999999999999e-06 2024-09-21T21:41:50.906 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.906 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.906 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:50.906 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923287+0000", 2024-09-21T21:41:50.906 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7799999999999997e-06 2024-09-21T21:41:50.906 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.906 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.906 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:50.906 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923291+0000", 2024-09-21T21:41:50.906 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1699999999999999e-06 2024-09-21T21:41:50.906 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.907 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.907 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:50.907 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966742+0000", 2024-09-21T21:41:50.907 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043451344000000003 2024-09-21T21:41:50.907 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.907 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.907 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:50.907 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966809+0000", 2024-09-21T21:41:50.907 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7070999999999995e-05 2024-09-21T21:41:50.907 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.907 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.907 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:50.907 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966909+0000", 2024-09-21T21:41:50.907 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000100307 2024-09-21T21:41:50.908 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:50.908 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:50.908 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:50.908 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.908 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.908 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7010 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5498880~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:50.908 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923301+0000", 2024-09-21T21:41:50.908 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 20.905832840999999, 2024-09-21T21:41:50.908 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043717830999999999, 2024-09-21T21:41:50.908 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:50.908 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:50.908 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:50.908 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:50.908 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:50.909 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:50.909 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7010 2024-09-21T21:41:50.909 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.909 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:50.909 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.909 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:50.909 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923301+0000", 2024-09-21T21:41:50.909 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:50.909 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.909 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.909 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:50.909 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923301+0000", 2024-09-21T21:41:50.909 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:50.910 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.910 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.910 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:50.910 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923302+0000", 2024-09-21T21:41:50.910 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.41e-07 2024-09-21T21:41:50.910 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.910 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.910 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:50.910 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923304+0000", 2024-09-21T21:41:50.910 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0990000000000001e-06 2024-09-21T21:41:50.910 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.910 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.910 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:50.910 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923310+0000", 2024-09-21T21:41:50.910 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3860000000000004e-06 2024-09-21T21:41:50.911 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.911 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.911 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:50.911 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923315+0000", 2024-09-21T21:41:50.911 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8459999999999999e-06 2024-09-21T21:41:50.911 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.911 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.911 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:50.911 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966938+0000", 2024-09-21T21:41:50.911 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043622693999999997 2024-09-21T21:41:50.911 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.911 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.911 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:50.911 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966975+0000", 2024-09-21T21:41:50.911 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7608999999999999e-05 2024-09-21T21:41:50.912 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.912 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.912 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:50.912 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.967019+0000", 2024-09-21T21:41:50.912 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3856000000000002e-05 2024-09-21T21:41:50.912 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:50.912 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:50.912 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:50.912 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.912 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.912 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7354 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 2107392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:50.912 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.936932+0000", 2024-09-21T21:41:50.912 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 20.892201986, 2024-09-21T21:41:50.912 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041607708, 2024-09-21T21:41:50.912 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:50.913 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:50.913 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:50.913 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:50.913 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:50.913 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:50.913 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7354 2024-09-21T21:41:50.913 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.913 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:50.913 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.913 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:50.913 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.936932+0000", 2024-09-21T21:41:50.913 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:50.913 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.913 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.913 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:50.914 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.936932+0000", 2024-09-21T21:41:50.914 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:50.914 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.914 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.914 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:50.914 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.937276+0000", 2024-09-21T21:41:50.914 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00034344200000000001 2024-09-21T21:41:50.914 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.914 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.914 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:50.914 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.937279+0000", 2024-09-21T21:41:50.914 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.844e-06 2024-09-21T21:41:50.914 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.914 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.915 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:50.915 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.937288+0000", 2024-09-21T21:41:50.915 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2409999999999995e-06 2024-09-21T21:41:50.915 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.915 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.915 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:50.915 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.937293+0000", 2024-09-21T21:41:50.915 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6559999999999997e-06 2024-09-21T21:41:50.915 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.915 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.915 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:50.915 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.978448+0000", 2024-09-21T21:41:50.915 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041154621000000002 2024-09-21T21:41:50.915 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.916 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.916 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:50.916 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.978487+0000", 2024-09-21T21:41:50.916 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9254000000000001e-05 2024-09-21T21:41:50.916 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.916 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.916 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:50.916 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.978540+0000", 2024-09-21T21:41:50.916 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2649999999999999e-05 2024-09-21T21:41:50.916 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:50.916 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:50.916 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:50.916 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.916 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.916 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5215.0:1054 3.9 3:946d2401:::rbd_data.1121524d2522.000000000000000b:head [set-alloc-hint object_size 16777216 write_size 16777216,write 6615040~59392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-21T21:41:50.917 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:38.148979+0000", 2024-09-21T21:41:50.917 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.680155019000001, 2024-09-21T21:41:50.917 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047870388999999999, 2024-09-21T21:41:50.917 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:50.917 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:50.917 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:41:50.917 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:50.917 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5215", 2024-09-21T21:41:50.917 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1580282168", 2024-09-21T21:41:50.917 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1054 2024-09-21T21:41:50.917 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.917 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:50.917 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.917 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:50.918 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.148979+0000", 2024-09-21T21:41:50.918 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:50.918 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.918 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.918 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:50.918 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.148979+0000", 2024-09-21T21:41:50.918 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:50.918 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.918 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.918 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:50.918 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.148980+0000", 2024-09-21T21:41:50.918 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5799999999999999e-07 2024-09-21T21:41:50.918 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.918 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.919 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:50.919 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.149039+0000", 2024-09-21T21:41:50.919 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9151000000000003e-05 2024-09-21T21:41:50.919 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.919 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.919 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:50.919 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.149047+0000", 2024-09-21T21:41:50.919 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1000000000000004e-06 2024-09-21T21:41:50.919 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.919 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.919 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:50.919 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.149052+0000", 2024-09-21T21:41:50.919 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5959999999999997e-06 2024-09-21T21:41:50.919 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.920 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.920 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:50.920 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.195288+0000", 2024-09-21T21:41:50.920 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.046236248000000001 2024-09-21T21:41:50.920 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.920 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.920 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:50.920 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.195334+0000", 2024-09-21T21:41:50.920 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6575e-05 2024-09-21T21:41:50.920 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.920 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.921 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 2", 2024-09-21T21:41:50.921 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.195406+0000", 2024-09-21T21:41:50.921 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1715e-05 2024-09-21T21:41:50.921 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.921 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.921 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:41:50.921 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.196018+0000", 2024-09-21T21:41:50.921 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00061218999999999998 2024-09-21T21:41:50.921 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.921 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.921 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:41:50.921 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.196794+0000", 2024-09-21T21:41:50.921 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000775814 2024-09-21T21:41:50.921 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.921 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.922 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:41:50.922 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.196827+0000", 2024-09-21T21:41:50.922 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2419000000000003e-05 2024-09-21T21:41:50.922 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:50.922 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:50.922 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:50.922 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.196849+0000", 2024-09-21T21:41:50.922 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2923000000000001e-05 2024-09-21T21:41:50.922 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:50.922 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:50.922 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:50.922 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:50.922 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:50.922 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:50.924 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:50.922+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:50.924 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T21:41:51.025 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:51.023+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:51.075 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:51.075 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:41:51.075 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.075 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:16871 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.075 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.047838+0000", 2024-09-21T21:41:51.075 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.018846753000000001, 2024-09-21T21:41:51.075 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.018867707000000001, 2024-09-21T21:41:51.076 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.076 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.076 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.076 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.076 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.076 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.076 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 16871 2024-09-21T21:41:51.076 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.076 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.076 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.076 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.076 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.047838+0000", 2024-09-21T21:41:51.076 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.076 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.077 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.077 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.077 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.047838+0000", 2024-09-21T21:41:51.077 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.077 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.077 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.077 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.077 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.047839+0000", 2024-09-21T21:41:51.077 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0999999999999998e-07 2024-09-21T21:41:51.077 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.077 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.077 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.077 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.047843+0000", 2024-09-21T21:41:51.077 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9190000000000001e-06 2024-09-21T21:41:51.077 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.078 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.078 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.078 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.047852+0000", 2024-09-21T21:41:51.078 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8140000000000007e-06 2024-09-21T21:41:51.078 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.078 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.078 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.078 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.047857+0000", 2024-09-21T21:41:51.078 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6779999999999999e-06 2024-09-21T21:41:51.078 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.078 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.078 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.078 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.078 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.079 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:16872 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.079 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.047877+0000", 2024-09-21T21:41:51.079 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.018808531999999999, 2024-09-21T21:41:51.079 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.018890400000000002, 2024-09-21T21:41:51.079 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.079 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.079 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.079 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.079 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.079 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.079 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 16872 2024-09-21T21:41:51.079 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.079 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.080 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.080 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.080 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.047877+0000", 2024-09-21T21:41:51.080 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.080 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.080 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.080 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.080 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.047877+0000", 2024-09-21T21:41:51.080 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.080 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.080 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.080 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.080 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.047877+0000", 2024-09-21T21:41:51.080 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4999999999999998e-07 2024-09-21T21:41:51.081 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.081 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.081 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.081 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.047879+0000", 2024-09-21T21:41:51.081 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.052e-06 2024-09-21T21:41:51.081 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.081 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.081 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.081 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.047886+0000", 2024-09-21T21:41:51.081 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6490000000000002e-06 2024-09-21T21:41:51.081 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.081 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.081 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.081 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.047889+0000", 2024-09-21T21:41:51.082 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.506e-06 2024-09-21T21:41:51.082 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.082 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.082 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.082 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.082 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.082 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:17065 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.082 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.056482+0000", 2024-09-21T21:41:51.082 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.010203494, 2024-09-21T21:41:51.082 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.010339568, 2024-09-21T21:41:51.082 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.082 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.082 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.082 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.083 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.083 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.083 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 17065 2024-09-21T21:41:51.083 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.083 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.083 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.083 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.083 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.056482+0000", 2024-09-21T21:41:51.083 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.083 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.083 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.083 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.083 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.056482+0000", 2024-09-21T21:41:51.083 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.084 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.084 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.084 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.084 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.056483+0000", 2024-09-21T21:41:51.084 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.0699999999999996e-07 2024-09-21T21:41:51.084 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.084 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.084 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.084 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.056487+0000", 2024-09-21T21:41:51.084 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5499999999999996e-06 2024-09-21T21:41:51.084 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.084 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.084 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.084 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.056499+0000", 2024-09-21T21:41:51.085 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1394e-05 2024-09-21T21:41:51.085 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.085 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.085 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.085 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.056506+0000", 2024-09-21T21:41:51.085 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0600000000000002e-06 2024-09-21T21:41:51.085 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.085 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.085 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.085 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.085 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.085 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:17185 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.085 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.062005+0000", 2024-09-21T21:41:51.085 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0046801029999999997, 2024-09-21T21:41:51.086 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048752769999999999, 2024-09-21T21:41:51.086 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.086 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.086 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.086 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.086 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.086 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.086 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 17185 2024-09-21T21:41:51.086 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.086 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.086 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.086 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.086 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.062005+0000", 2024-09-21T21:41:51.086 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.087 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.087 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.087 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.087 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.062005+0000", 2024-09-21T21:41:51.087 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.087 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.087 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.087 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.087 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.062006+0000", 2024-09-21T21:41:51.087 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5400000000000001e-07 2024-09-21T21:41:51.087 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.087 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.087 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.087 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.062009+0000", 2024-09-21T21:41:51.087 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2210000000000002e-06 2024-09-21T21:41:51.088 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.088 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.088 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.088 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.062020+0000", 2024-09-21T21:41:51.088 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1026000000000001e-05 2024-09-21T21:41:51.088 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.088 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.088 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.088 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.062026+0000", 2024-09-21T21:41:51.088 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9100000000000002e-06 2024-09-21T21:41:51.088 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.088 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.088 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.088 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.088 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.089 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:17240 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.089 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.064313+0000", 2024-09-21T21:41:51.089 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0023725090000000001, 2024-09-21T21:41:51.089 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002618769, 2024-09-21T21:41:51.089 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.089 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.089 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.089 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.089 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.089 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.089 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 17240 2024-09-21T21:41:51.089 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.090 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.090 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.090 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.090 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.064313+0000", 2024-09-21T21:41:51.090 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.090 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.090 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.090 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.090 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.064313+0000", 2024-09-21T21:41:51.090 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.090 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.090 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.090 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.090 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.064313+0000", 2024-09-21T21:41:51.091 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7500000000000001e-07 2024-09-21T21:41:51.091 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.091 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.091 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.091 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.064315+0000", 2024-09-21T21:41:51.091 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.8610000000000001e-06 2024-09-21T21:41:51.091 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.091 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.091 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.091 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.064318+0000", 2024-09-21T21:41:51.091 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4189999999999998e-06 2024-09-21T21:41:51.091 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.091 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.091 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.091 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.064321+0000", 2024-09-21T21:41:51.092 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1159999999999999e-06 2024-09-21T21:41:51.092 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.092 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.092 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.093 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.093 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.093 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:16813 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.094 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.044787+0000", 2024-09-21T21:41:51.094 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.021898575, 2024-09-21T21:41:51.094 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.022174795000000001, 2024-09-21T21:41:51.094 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.094 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.094 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.094 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.094 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.094 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.094 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 16813 2024-09-21T21:41:51.094 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.094 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.095 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.095 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.095 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.044787+0000", 2024-09-21T21:41:51.095 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.095 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.095 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.095 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.095 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.044787+0000", 2024-09-21T21:41:51.095 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.095 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.095 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.095 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.095 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.044787+0000", 2024-09-21T21:41:51.095 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6199999999999997e-07 2024-09-21T21:41:51.096 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.096 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.096 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.096 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.044791+0000", 2024-09-21T21:41:51.096 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6480000000000001e-06 2024-09-21T21:41:51.096 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.096 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.096 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.096 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.044799+0000", 2024-09-21T21:41:51.096 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0579999999999998e-06 2024-09-21T21:41:51.096 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.096 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.096 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.096 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.044804+0000", 2024-09-21T21:41:51.097 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7040000000000002e-06 2024-09-21T21:41:51.097 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.097 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.097 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.097 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.097 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.097 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:17066 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.097 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.056525+0000", 2024-09-21T21:41:51.097 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.010160084999999999, 2024-09-21T21:41:51.097 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.010465871999999999, 2024-09-21T21:41:51.097 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.097 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.097 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.097 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.098 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.098 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.098 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 17066 2024-09-21T21:41:51.098 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.098 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.098 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.098 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.098 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.056525+0000", 2024-09-21T21:41:51.098 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.098 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.098 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.098 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.099 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.056525+0000", 2024-09-21T21:41:51.099 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.099 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.099 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.099 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.099 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.056526+0000", 2024-09-21T21:41:51.099 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3499999999999996e-07 2024-09-21T21:41:51.099 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.099 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.099 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.099 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.056529+0000", 2024-09-21T21:41:51.099 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8459999999999999e-06 2024-09-21T21:41:51.099 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.099 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.100 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.100 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.056536+0000", 2024-09-21T21:41:51.100 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5789999999999998e-06 2024-09-21T21:41:51.100 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.100 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.100 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.100 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.056544+0000", 2024-09-21T21:41:51.100 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2530000000000004e-06 2024-09-21T21:41:51.100 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.100 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.100 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.100 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.100 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.100 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:17121 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.101 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.058764+0000", 2024-09-21T21:41:51.101 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0079207619999999996, 2024-09-21T21:41:51.101 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0082551610000000004, 2024-09-21T21:41:51.101 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.101 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.101 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.101 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.101 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.101 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.101 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 17121 2024-09-21T21:41:51.101 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.101 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.101 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.102 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.102 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.058764+0000", 2024-09-21T21:41:51.102 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.102 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.102 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.102 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.102 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.058764+0000", 2024-09-21T21:41:51.102 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.102 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.102 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.102 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.102 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.058765+0000", 2024-09-21T21:41:51.102 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2999999999999999e-07 2024-09-21T21:41:51.103 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.103 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.103 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.103 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.058769+0000", 2024-09-21T21:41:51.103 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0040000000000001e-06 2024-09-21T21:41:51.103 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.103 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.103 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.103 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.058777+0000", 2024-09-21T21:41:51.103 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2490000000000001e-06 2024-09-21T21:41:51.103 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.103 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.103 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.103 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.058784+0000", 2024-09-21T21:41:51.103 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3659999999999997e-06 2024-09-21T21:41:51.104 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.104 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.104 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.104 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.104 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.104 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:17186 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.104 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.062121+0000", 2024-09-21T21:41:51.104 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0045642060000000003, 2024-09-21T21:41:51.104 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0049257040000000004, 2024-09-21T21:41:51.104 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.105 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.105 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.105 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.105 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.105 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.105 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 17186 2024-09-21T21:41:51.105 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.105 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.105 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.105 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.105 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.062121+0000", 2024-09-21T21:41:51.105 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.105 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.105 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.105 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.106 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.062121+0000", 2024-09-21T21:41:51.106 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.106 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.106 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.106 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.106 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.062122+0000", 2024-09-21T21:41:51.106 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5199999999999995e-07 2024-09-21T21:41:51.106 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.106 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.107 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.107 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.062125+0000", 2024-09-21T21:41:51.107 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4989999999999999e-06 2024-09-21T21:41:51.107 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.107 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.107 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.107 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.062133+0000", 2024-09-21T21:41:51.107 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5889999999999997e-06 2024-09-21T21:41:51.107 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.107 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.107 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.107 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.062138+0000", 2024-09-21T21:41:51.107 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3600000000000004e-06 2024-09-21T21:41:51.108 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.108 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.108 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.108 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.108 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.108 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:17241 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.108 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.064326+0000", 2024-09-21T21:41:51.108 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0023590130000000001, 2024-09-21T21:41:51.108 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0027544980000000002, 2024-09-21T21:41:51.108 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.108 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.109 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.109 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.109 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.109 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.109 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 17241 2024-09-21T21:41:51.109 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.109 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.109 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.109 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.109 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.064326+0000", 2024-09-21T21:41:51.109 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.109 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.109 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.110 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.110 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.064326+0000", 2024-09-21T21:41:51.110 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.110 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.110 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.110 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.110 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.064326+0000", 2024-09-21T21:41:51.110 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1500000000000001e-07 2024-09-21T21:41:51.110 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.110 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.111 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.111 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.064328+0000", 2024-09-21T21:41:51.111 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2130000000000001e-06 2024-09-21T21:41:51.111 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.111 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.111 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.111 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.064331+0000", 2024-09-21T21:41:51.111 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9869999999999999e-06 2024-09-21T21:41:51.111 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.111 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.111 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.112 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.064333+0000", 2024-09-21T21:41:51.112 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4679999999999999e-06 2024-09-21T21:41:51.112 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.112 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.112 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.112 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.112 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.112 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:16814 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.112 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.044816+0000", 2024-09-21T21:41:51.112 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.021868808, 2024-09-21T21:41:51.112 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.022291373999999999, 2024-09-21T21:41:51.112 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.112 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.113 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.113 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.113 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.113 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.113 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 16814 2024-09-21T21:41:51.113 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.113 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.113 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.113 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.113 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.044816+0000", 2024-09-21T21:41:51.113 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.113 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.113 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.113 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.113 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.044816+0000", 2024-09-21T21:41:51.114 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.114 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.114 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.114 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.114 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.044817+0000", 2024-09-21T21:41:51.114 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.39e-07 2024-09-21T21:41:51.114 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.114 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.114 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.114 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.044819+0000", 2024-09-21T21:41:51.114 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2110000000000001e-06 2024-09-21T21:41:51.114 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.114 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.115 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.115 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.044823+0000", 2024-09-21T21:41:51.115 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9689999999999996e-06 2024-09-21T21:41:51.115 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.115 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.115 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.115 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.044826+0000", 2024-09-21T21:41:51.115 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.033e-06 2024-09-21T21:41:51.115 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.115 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.115 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.115 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.115 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.116 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:17122 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.116 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.058920+0000", 2024-09-21T21:41:51.116 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0077648140000000001, 2024-09-21T21:41:51.116 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0082169830000000006, 2024-09-21T21:41:51.116 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.116 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.116 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.116 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.116 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.116 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.116 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 17122 2024-09-21T21:41:51.116 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.116 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.117 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.117 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.117 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.058920+0000", 2024-09-21T21:41:51.117 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.117 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.117 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.117 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.117 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.058920+0000", 2024-09-21T21:41:51.117 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.117 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.117 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.118 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.118 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.058921+0000", 2024-09-21T21:41:51.118 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.4200000000000005e-07 2024-09-21T21:41:51.118 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.118 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.118 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.118 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.058926+0000", 2024-09-21T21:41:51.118 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0409999999999997e-06 2024-09-21T21:41:51.118 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.118 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.118 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.119 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.058935+0000", 2024-09-21T21:41:51.119 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5979999999999997e-06 2024-09-21T21:41:51.119 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.119 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.119 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.119 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.058941+0000", 2024-09-21T21:41:51.119 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2670000000000003e-06 2024-09-21T21:41:51.119 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.119 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.119 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.119 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.119 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.119 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:17187 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.119 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.062159+0000", 2024-09-21T21:41:51.120 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0045257500000000003, 2024-09-21T21:41:51.120 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0050068939999999996, 2024-09-21T21:41:51.120 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.120 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.120 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.120 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.120 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.120 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.120 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 17187 2024-09-21T21:41:51.120 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.120 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.121 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.121 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.121 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.062159+0000", 2024-09-21T21:41:51.121 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.121 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.121 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.121 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.121 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.062159+0000", 2024-09-21T21:41:51.121 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.121 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.121 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.122 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.122 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.062160+0000", 2024-09-21T21:41:51.122 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9700000000000002e-07 2024-09-21T21:41:51.122 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.122 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.122 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.122 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.062162+0000", 2024-09-21T21:41:51.122 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5509999999999998e-06 2024-09-21T21:41:51.122 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.123 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.123 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.123 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.062167+0000", 2024-09-21T21:41:51.123 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3510000000000004e-06 2024-09-21T21:41:51.123 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.123 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.123 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.123 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.062170+0000", 2024-09-21T21:41:51.123 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2279999999999998e-06 2024-09-21T21:41:51.123 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.124 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.124 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.124 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.124 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.124 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:17242 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.124 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.064336+0000", 2024-09-21T21:41:51.124 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0023490249999999998, 2024-09-21T21:41:51.124 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0028953020000000002, 2024-09-21T21:41:51.124 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.125 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.125 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.125 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.125 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.125 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.125 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 17242 2024-09-21T21:41:51.125 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.125 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.125 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.126 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.126 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.064336+0000", 2024-09-21T21:41:51.126 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.126 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.126 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.126 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.126 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.064336+0000", 2024-09-21T21:41:51.126 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.126 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.127 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.127 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.127 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.064336+0000", 2024-09-21T21:41:51.127 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6600000000000003e-07 2024-09-21T21:41:51.127 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.127 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.127 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.127 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.064338+0000", 2024-09-21T21:41:51.127 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4410000000000001e-06 2024-09-21T21:41:51.127 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.128 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.128 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.128 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.064342+0000", 2024-09-21T21:41:51.128 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9199999999999997e-06 2024-09-21T21:41:51.128 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.128 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.128 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.128 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.064344+0000", 2024-09-21T21:41:51.128 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7049999999999999e-06 2024-09-21T21:41:51.128 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.128 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.128 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.129 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.129 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.129 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:17123 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.129 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.058960+0000", 2024-09-21T21:41:51.129 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0077250820000000003, 2024-09-21T21:41:51.129 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0083157030000000007, 2024-09-21T21:41:51.129 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.129 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.129 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.129 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.129 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.129 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.129 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 17123 2024-09-21T21:41:51.129 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.129 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.130 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.130 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.130 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.058960+0000", 2024-09-21T21:41:51.130 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.130 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.130 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.130 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.130 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.058960+0000", 2024-09-21T21:41:51.130 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.130 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.130 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.130 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.130 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.058961+0000", 2024-09-21T21:41:51.130 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0900000000000001e-07 2024-09-21T21:41:51.131 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.131 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.131 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.131 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.058964+0000", 2024-09-21T21:41:51.131 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9280000000000002e-06 2024-09-21T21:41:51.131 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.131 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.131 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.131 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.058974+0000", 2024-09-21T21:41:51.131 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0040999999999999e-05 2024-09-21T21:41:51.131 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.131 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.131 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.131 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.058979+0000", 2024-09-21T21:41:51.131 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2680000000000002e-06 2024-09-21T21:41:51.132 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.132 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.132 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.132 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.132 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.132 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:16753 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.132 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.041385+0000", 2024-09-21T21:41:51.132 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.025299682, 2024-09-21T21:41:51.132 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.025918165999999999, 2024-09-21T21:41:51.132 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.132 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.132 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.132 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.133 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.133 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.133 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 16753 2024-09-21T21:41:51.133 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.133 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.133 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.133 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.133 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.041385+0000", 2024-09-21T21:41:51.133 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.133 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.133 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.133 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.133 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.041385+0000", 2024-09-21T21:41:51.133 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.134 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.134 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.134 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.134 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.041386+0000", 2024-09-21T21:41:51.134 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3200000000000004e-07 2024-09-21T21:41:51.134 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.134 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.134 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.134 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.041390+0000", 2024-09-21T21:41:51.134 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1620000000000001e-06 2024-09-21T21:41:51.134 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.134 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.134 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.134 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.041401+0000", 2024-09-21T21:41:51.135 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0295000000000001e-05 2024-09-21T21:41:51.135 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.135 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.135 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.135 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.041408+0000", 2024-09-21T21:41:51.135 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9040000000000003e-06 2024-09-21T21:41:51.135 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.135 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.135 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.135 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.135 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.135 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:16879 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.135 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.048539+0000", 2024-09-21T21:41:51.135 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.018146445000000001, 2024-09-21T21:41:51.136 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.01879165, 2024-09-21T21:41:51.136 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.136 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.136 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.136 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.136 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.136 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.136 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 16879 2024-09-21T21:41:51.136 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.136 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.136 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.136 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.136 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.048539+0000", 2024-09-21T21:41:51.136 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.136 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.137 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.137 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.137 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.048539+0000", 2024-09-21T21:41:51.137 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.137 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.137 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.137 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.137 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.048539+0000", 2024-09-21T21:41:51.137 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0500000000000004e-07 2024-09-21T21:41:51.137 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.137 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.137 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.137 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.048542+0000", 2024-09-21T21:41:51.137 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4219999999999999e-06 2024-09-21T21:41:51.138 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.138 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.138 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.138 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.048548+0000", 2024-09-21T21:41:51.138 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1789999999999996e-06 2024-09-21T21:41:51.138 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.138 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.138 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.138 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.048553+0000", 2024-09-21T21:41:51.138 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4489999999999998e-06 2024-09-21T21:41:51.138 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.138 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.139 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.139 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.139 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.139 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:17074 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.139 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.056615+0000", 2024-09-21T21:41:51.139 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.010070066000000001, 2024-09-21T21:41:51.139 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.010746336, 2024-09-21T21:41:51.139 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.139 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.139 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.139 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.139 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.139 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.140 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 17074 2024-09-21T21:41:51.140 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.140 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.140 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.140 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.140 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.056615+0000", 2024-09-21T21:41:51.140 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.140 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.140 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.140 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.140 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.056615+0000", 2024-09-21T21:41:51.140 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.140 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.141 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.141 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.141 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.056616+0000", 2024-09-21T21:41:51.141 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1200000000000003e-07 2024-09-21T21:41:51.141 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.141 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.141 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.141 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.056618+0000", 2024-09-21T21:41:51.141 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7719999999999999e-06 2024-09-21T21:41:51.141 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.141 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.141 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.141 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.056626+0000", 2024-09-21T21:41:51.141 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3200000000000002e-06 2024-09-21T21:41:51.141 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.142 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.142 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.142 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.056631+0000", 2024-09-21T21:41:51.142 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5809999999999996e-06 2024-09-21T21:41:51.142 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.142 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.142 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.142 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.142 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.142 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:16754 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.142 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.041425+0000", 2024-09-21T21:41:51.142 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.025260161999999999, 2024-09-21T21:41:51.142 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.025964814999999999, 2024-09-21T21:41:51.142 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.143 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.143 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.143 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.143 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.143 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.143 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 16754 2024-09-21T21:41:51.143 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.143 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.143 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.143 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.143 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.041425+0000", 2024-09-21T21:41:51.143 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.143 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.143 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.143 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.144 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.041425+0000", 2024-09-21T21:41:51.144 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.144 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.144 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.144 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.144 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.041426+0000", 2024-09-21T21:41:51.144 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6000000000000003e-07 2024-09-21T21:41:51.144 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.144 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.144 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.144 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.041429+0000", 2024-09-21T21:41:51.144 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.903e-06 2024-09-21T21:41:51.144 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.144 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.145 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.145 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.041437+0000", 2024-09-21T21:41:51.145 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8929999999999994e-06 2024-09-21T21:41:51.145 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.145 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.145 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.145 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.041443+0000", 2024-09-21T21:41:51.145 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7440000000000004e-06 2024-09-21T21:41:51.145 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.145 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.145 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.145 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.145 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.145 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:16821 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.146 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.045148+0000", 2024-09-21T21:41:51.146 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.021537325, 2024-09-21T21:41:51.146 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.022290606000000001, 2024-09-21T21:41:51.146 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.146 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.146 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.146 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.146 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.146 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.146 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 16821 2024-09-21T21:41:51.146 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.146 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.146 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.146 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.147 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.045148+0000", 2024-09-21T21:41:51.147 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.147 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.147 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.147 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.147 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.045148+0000", 2024-09-21T21:41:51.147 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.147 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.147 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.147 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.147 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.045149+0000", 2024-09-21T21:41:51.147 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3799999999999996e-07 2024-09-21T21:41:51.147 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.147 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.147 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.148 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.045153+0000", 2024-09-21T21:41:51.148 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8299999999999998e-06 2024-09-21T21:41:51.148 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.148 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.148 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.148 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.045163+0000", 2024-09-21T21:41:51.148 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0475000000000001e-05 2024-09-21T21:41:51.148 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.148 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.148 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.148 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.045170+0000", 2024-09-21T21:41:51.148 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7109999999999998e-06 2024-09-21T21:41:51.148 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.148 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.149 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.149 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.149 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.149 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:16880 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.149 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.048594+0000", 2024-09-21T21:41:51.149 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.018090974999999999, 2024-09-21T21:41:51.149 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.018896691, 2024-09-21T21:41:51.149 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.149 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.149 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.149 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.149 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.149 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.149 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 16880 2024-09-21T21:41:51.149 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.150 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.150 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.150 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.150 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.048594+0000", 2024-09-21T21:41:51.150 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.150 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.150 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.150 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.150 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.048594+0000", 2024-09-21T21:41:51.150 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.150 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.150 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.150 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.150 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.048595+0000", 2024-09-21T21:41:51.151 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5799999999999998e-07 2024-09-21T21:41:51.151 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.151 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.151 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.151 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.048601+0000", 2024-09-21T21:41:51.151 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.592e-06 2024-09-21T21:41:51.151 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.151 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.151 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.151 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.048608+0000", 2024-09-21T21:41:51.151 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6369999999999999e-06 2024-09-21T21:41:51.151 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.151 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.151 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.152 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.048613+0000", 2024-09-21T21:41:51.152 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2889999999999997e-06 2024-09-21T21:41:51.152 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.152 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.152 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.152 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.152 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.152 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:17075 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.152 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.056639+0000", 2024-09-21T21:41:51.152 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.010045854, 2024-09-21T21:41:51.152 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.010904260000000001, 2024-09-21T21:41:51.152 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.152 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.152 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.153 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.153 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.153 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.153 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 17075 2024-09-21T21:41:51.153 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.153 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.153 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.153 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.153 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.056639+0000", 2024-09-21T21:41:51.153 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.153 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.153 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.153 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.154 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.056639+0000", 2024-09-21T21:41:51.154 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.154 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.154 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.154 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.154 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.056640+0000", 2024-09-21T21:41:51.154 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4000000000000002e-07 2024-09-21T21:41:51.154 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.154 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.154 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.154 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.056642+0000", 2024-09-21T21:41:51.154 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0269999999999998e-06 2024-09-21T21:41:51.154 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.154 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.155 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.155 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.056648+0000", 2024-09-21T21:41:51.155 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3770000000000001e-06 2024-09-21T21:41:51.155 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.155 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.155 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.155 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.056653+0000", 2024-09-21T21:41:51.155 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9899999999999997e-06 2024-09-21T21:41:51.155 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.155 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.155 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.155 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.155 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.155 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:17249 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.155 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.064407+0000", 2024-09-21T21:41:51.156 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0022779710000000002, 2024-09-21T21:41:51.156 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0031895420000000001, 2024-09-21T21:41:51.156 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.156 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.156 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.156 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.156 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.156 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.156 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 17249 2024-09-21T21:41:51.156 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.156 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.157 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.157 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.157 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.064407+0000", 2024-09-21T21:41:51.157 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.157 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.157 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.157 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.157 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.064407+0000", 2024-09-21T21:41:51.157 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.157 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.157 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.157 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.157 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.064408+0000", 2024-09-21T21:41:51.158 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1800000000000001e-07 2024-09-21T21:41:51.158 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.158 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.158 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.158 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.064410+0000", 2024-09-21T21:41:51.158 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9300000000000002e-06 2024-09-21T21:41:51.158 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.158 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.158 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.158 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.064414+0000", 2024-09-21T21:41:51.159 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3229999999999999e-06 2024-09-21T21:41:51.159 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.159 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.159 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.159 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.064418+0000", 2024-09-21T21:41:51.159 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7589999999999999e-06 2024-09-21T21:41:51.159 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.159 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.159 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.159 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.159 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.159 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:16755 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.159 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.041454+0000", 2024-09-21T21:41:51.160 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.02523135, 2024-09-21T21:41:51.160 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.026195626, 2024-09-21T21:41:51.160 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.160 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.160 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.160 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.160 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.160 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.160 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 16755 2024-09-21T21:41:51.160 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.160 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.160 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.161 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.161 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.041454+0000", 2024-09-21T21:41:51.161 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.161 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.161 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.161 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.161 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.041454+0000", 2024-09-21T21:41:51.161 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.161 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.161 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.161 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.161 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.041454+0000", 2024-09-21T21:41:51.161 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7700000000000004e-07 2024-09-21T21:41:51.162 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.162 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.162 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.162 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.041457+0000", 2024-09-21T21:41:51.162 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8289999999999999e-06 2024-09-21T21:41:51.162 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.162 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.162 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.162 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.041466+0000", 2024-09-21T21:41:51.162 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3089999999999996e-06 2024-09-21T21:41:51.162 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.162 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.162 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.163 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.041472+0000", 2024-09-21T21:41:51.163 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9719999999999997e-06 2024-09-21T21:41:51.163 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.163 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.163 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.163 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.163 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.163 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:16822 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.163 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.045188+0000", 2024-09-21T21:41:51.163 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.021497077, 2024-09-21T21:41:51.163 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.022512833999999999, 2024-09-21T21:41:51.163 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.163 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.164 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.164 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.164 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.164 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.164 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 16822 2024-09-21T21:41:51.164 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.164 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.164 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.164 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.164 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.045188+0000", 2024-09-21T21:41:51.164 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.164 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.164 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.165 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.165 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.045188+0000", 2024-09-21T21:41:51.165 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.165 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.165 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.165 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.165 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.045189+0000", 2024-09-21T21:41:51.165 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3e-07 2024-09-21T21:41:51.165 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.165 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.165 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.165 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.045192+0000", 2024-09-21T21:41:51.165 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9349999999999999e-06 2024-09-21T21:41:51.166 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.166 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.166 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.166 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.045199+0000", 2024-09-21T21:41:51.166 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7700000000000001e-06 2024-09-21T21:41:51.166 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.166 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.166 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.166 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.045205+0000", 2024-09-21T21:41:51.166 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7780000000000004e-06 2024-09-21T21:41:51.166 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.166 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.167 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.167 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.167 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.167 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:17131 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.167 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.059665+0000", 2024-09-21T21:41:51.167 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0070204029999999997, 2024-09-21T21:41:51.167 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0080733100000000002, 2024-09-21T21:41:51.167 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.167 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.167 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.167 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.167 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.167 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.167 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 17131 2024-09-21T21:41:51.168 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.168 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.168 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.168 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.168 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.059665+0000", 2024-09-21T21:41:51.168 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.168 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.168 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.168 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.168 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.059665+0000", 2024-09-21T21:41:51.168 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.168 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.168 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.169 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.169 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.059666+0000", 2024-09-21T21:41:51.169 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.23e-07 2024-09-21T21:41:51.169 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.169 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.169 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.169 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.059669+0000", 2024-09-21T21:41:51.169 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6990000000000001e-06 2024-09-21T21:41:51.169 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.169 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.170 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.170 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.059678+0000", 2024-09-21T21:41:51.170 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6289999999999999e-06 2024-09-21T21:41:51.170 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.170 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.170 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.170 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.059685+0000", 2024-09-21T21:41:51.170 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7089999999999996e-06 2024-09-21T21:41:51.170 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.170 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.170 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.171 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.171 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.171 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:17196 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.171 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.062487+0000", 2024-09-21T21:41:51.171 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0041980639999999996, 2024-09-21T21:41:51.171 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0052778979999999996, 2024-09-21T21:41:51.171 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.171 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.171 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.171 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.171 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.171 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.172 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 17196 2024-09-21T21:41:51.172 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.172 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.172 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.172 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.172 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.062487+0000", 2024-09-21T21:41:51.172 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.172 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.172 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.172 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.173 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.062487+0000", 2024-09-21T21:41:51.173 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.173 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.173 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.173 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.173 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.062488+0000", 2024-09-21T21:41:51.173 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9999999999999996e-07 2024-09-21T21:41:51.173 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.173 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.173 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.173 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.062492+0000", 2024-09-21T21:41:51.173 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4009999999999999e-06 2024-09-21T21:41:51.174 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.174 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.174 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.174 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.062502+0000", 2024-09-21T21:41:51.174 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.6379999999999999e-06 2024-09-21T21:41:51.174 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.174 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.174 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.175 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.062508+0000", 2024-09-21T21:41:51.175 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5540000000000003e-06 2024-09-21T21:41:51.175 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.175 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.175 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.175 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.175 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.175 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:17250 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.175 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.064436+0000", 2024-09-21T21:41:51.175 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0022492839999999998, 2024-09-21T21:41:51.175 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003355125, 2024-09-21T21:41:51.176 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.176 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.176 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.176 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.176 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.176 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.176 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 17250 2024-09-21T21:41:51.176 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.176 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.177 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.177 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.177 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.064436+0000", 2024-09-21T21:41:51.177 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.177 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.177 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.177 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.177 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.064436+0000", 2024-09-21T21:41:51.177 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.177 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.177 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.178 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.178 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.064436+0000", 2024-09-21T21:41:51.178 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9099999999999999e-07 2024-09-21T21:41:51.178 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.178 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.178 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.178 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.064438+0000", 2024-09-21T21:41:51.178 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1220000000000002e-06 2024-09-21T21:41:51.178 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.178 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.178 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.178 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.064442+0000", 2024-09-21T21:41:51.179 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7450000000000001e-06 2024-09-21T21:41:51.179 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.179 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.179 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.179 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.064445+0000", 2024-09-21T21:41:51.179 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9359999999999999e-06 2024-09-21T21:41:51.179 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.179 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.179 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.179 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.179 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.179 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:16760 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.179 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.041712+0000", 2024-09-21T21:41:51.179 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.024972956000000001, 2024-09-21T21:41:51.180 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.026109706999999999, 2024-09-21T21:41:51.180 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.180 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.180 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.180 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.180 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.180 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.180 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 16760 2024-09-21T21:41:51.180 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.180 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.180 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.180 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.180 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.041712+0000", 2024-09-21T21:41:51.181 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.181 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.181 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.181 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.181 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.041712+0000", 2024-09-21T21:41:51.181 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.181 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.181 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.181 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.181 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.041713+0000", 2024-09-21T21:41:51.181 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5600000000000004e-07 2024-09-21T21:41:51.181 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.182 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.182 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.182 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.041717+0000", 2024-09-21T21:41:51.182 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9709999999999998e-06 2024-09-21T21:41:51.182 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.182 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.182 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.182 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.041728+0000", 2024-09-21T21:41:51.182 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0589e-05 2024-09-21T21:41:51.182 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.182 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.183 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.183 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.041734+0000", 2024-09-21T21:41:51.183 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7909999999999999e-06 2024-09-21T21:41:51.183 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.183 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.183 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.183 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.183 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.183 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:17132 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.183 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.059701+0000", 2024-09-21T21:41:51.183 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0069844709999999999, 2024-09-21T21:41:51.184 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0081479499999999993, 2024-09-21T21:41:51.184 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.184 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.184 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.184 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.184 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.184 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.184 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 17132 2024-09-21T21:41:51.184 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.184 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.184 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.184 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.185 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.059701+0000", 2024-09-21T21:41:51.185 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.185 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.185 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.185 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.185 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.059701+0000", 2024-09-21T21:41:51.185 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.185 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.185 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.185 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.185 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.059701+0000", 2024-09-21T21:41:51.185 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2699999999999999e-07 2024-09-21T21:41:51.185 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.185 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.185 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.186 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.059704+0000", 2024-09-21T21:41:51.186 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.616e-06 2024-09-21T21:41:51.186 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.186 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.186 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.186 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.059711+0000", 2024-09-21T21:41:51.186 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9299999999999997e-06 2024-09-21T21:41:51.186 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.186 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.186 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.186 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.059716+0000", 2024-09-21T21:41:51.186 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2469999999999999e-06 2024-09-21T21:41:51.187 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.187 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.187 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.187 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.187 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.187 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:17197 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.187 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.062545+0000", 2024-09-21T21:41:51.187 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0041398440000000002, 2024-09-21T21:41:51.187 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0053498629999999998, 2024-09-21T21:41:51.187 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.187 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.187 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.187 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.187 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.188 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.188 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 17197 2024-09-21T21:41:51.188 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.188 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.188 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.188 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.188 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.062545+0000", 2024-09-21T21:41:51.188 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.188 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.188 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.188 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.188 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.062545+0000", 2024-09-21T21:41:51.188 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.188 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.189 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.189 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.189 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.062546+0000", 2024-09-21T21:41:51.189 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0100000000000005e-07 2024-09-21T21:41:51.189 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.189 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.189 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.189 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.062549+0000", 2024-09-21T21:41:51.189 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3869999999999999e-06 2024-09-21T21:41:51.189 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.189 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.189 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.189 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.062557+0000", 2024-09-21T21:41:51.189 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2250000000000003e-06 2024-09-21T21:41:51.190 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.190 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.190 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.190 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.062562+0000", 2024-09-21T21:41:51.190 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6869999999999999e-06 2024-09-21T21:41:51.190 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.190 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.190 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.190 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.190 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.190 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:17251 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.190 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.064487+0000", 2024-09-21T21:41:51.190 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0021980060000000002, 2024-09-21T21:41:51.190 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0034599840000000001, 2024-09-21T21:41:51.190 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.191 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.191 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.191 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.191 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.191 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.191 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 17251 2024-09-21T21:41:51.191 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.191 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.191 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.191 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.192 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.064487+0000", 2024-09-21T21:41:51.192 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.192 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.192 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.192 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.192 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.064487+0000", 2024-09-21T21:41:51.192 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.192 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.192 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.192 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.192 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.064488+0000", 2024-09-21T21:41:51.192 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.68e-07 2024-09-21T21:41:51.192 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.192 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.192 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.193 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.064490+0000", 2024-09-21T21:41:51.193 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5069999999999999e-06 2024-09-21T21:41:51.193 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.193 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.193 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.193 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.064495+0000", 2024-09-21T21:41:51.193 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7439999999999998e-06 2024-09-21T21:41:51.193 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.193 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.193 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.193 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.064498+0000", 2024-09-21T21:41:51.193 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3900000000000002e-06 2024-09-21T21:41:51.194 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.194 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.194 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.194 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.194 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.194 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:16761 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.194 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.041751+0000", 2024-09-21T21:41:51.194 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.024933905999999999, 2024-09-21T21:41:51.194 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.026255957, 2024-09-21T21:41:51.194 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.194 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.194 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.194 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.195 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.195 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.195 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 16761 2024-09-21T21:41:51.195 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.195 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.195 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.195 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.195 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.041751+0000", 2024-09-21T21:41:51.195 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.195 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.195 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.195 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.195 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.041751+0000", 2024-09-21T21:41:51.196 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.196 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.196 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.196 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.196 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.041752+0000", 2024-09-21T21:41:51.196 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0399999999999996e-07 2024-09-21T21:41:51.196 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.196 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.196 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.196 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.041755+0000", 2024-09-21T21:41:51.196 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0149999999999999e-06 2024-09-21T21:41:51.196 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.196 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.196 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.197 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.041763+0000", 2024-09-21T21:41:51.197 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9610000000000005e-06 2024-09-21T21:41:51.197 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.197 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.197 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.197 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.041768+0000", 2024-09-21T21:41:51.197 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5550000000000002e-06 2024-09-21T21:41:51.197 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.197 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.197 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.197 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.197 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.197 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:17133 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.198 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.059728+0000", 2024-09-21T21:41:51.198 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0069567680000000003, 2024-09-21T21:41:51.198 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0083344850000000009, 2024-09-21T21:41:51.198 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.198 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.198 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.198 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.198 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.198 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.198 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 17133 2024-09-21T21:41:51.198 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.198 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.198 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.198 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.199 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.059728+0000", 2024-09-21T21:41:51.199 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.199 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.199 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.199 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.199 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.059728+0000", 2024-09-21T21:41:51.199 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.199 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.199 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.199 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.199 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.059729+0000", 2024-09-21T21:41:51.199 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7800000000000002e-07 2024-09-21T21:41:51.199 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.200 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.200 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.200 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.059732+0000", 2024-09-21T21:41:51.200 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.762e-06 2024-09-21T21:41:51.200 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.200 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.200 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.200 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.059742+0000", 2024-09-21T21:41:51.200 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0318e-05 2024-09-21T21:41:51.200 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.200 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.200 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.201 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.059747+0000", 2024-09-21T21:41:51.201 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0560000000000004e-06 2024-09-21T21:41:51.201 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.201 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.201 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.201 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.201 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.201 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:16766 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.201 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.042099+0000", 2024-09-21T21:41:51.201 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.024586554, 2024-09-21T21:41:51.201 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.026028484000000001, 2024-09-21T21:41:51.201 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.202 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.202 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.202 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.202 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.202 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.202 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 16766 2024-09-21T21:41:51.202 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.202 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.202 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.202 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.202 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.042099+0000", 2024-09-21T21:41:51.202 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.203 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.203 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.203 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.203 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.042099+0000", 2024-09-21T21:41:51.203 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.203 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.203 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.203 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.203 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.042099+0000", 2024-09-21T21:41:51.203 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.85e-07 2024-09-21T21:41:51.203 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.203 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.204 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.204 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.042107+0000", 2024-09-21T21:41:51.204 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8909999999999995e-06 2024-09-21T21:41:51.204 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.204 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.204 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.204 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.042119+0000", 2024-09-21T21:41:51.204 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1804000000000001e-05 2024-09-21T21:41:51.204 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.204 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.204 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.205 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.042127+0000", 2024-09-21T21:41:51.205 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3440000000000003e-06 2024-09-21T21:41:51.205 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.205 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.205 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.205 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.205 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.205 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:16830 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.205 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.045812+0000", 2024-09-21T21:41:51.205 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.020873299000000001, 2024-09-21T21:41:51.205 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.022367345, 2024-09-21T21:41:51.205 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.206 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.206 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.206 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.206 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.206 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.206 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 16830 2024-09-21T21:41:51.206 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.206 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.206 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.206 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.206 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.045812+0000", 2024-09-21T21:41:51.206 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.206 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.206 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.207 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.207 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.045812+0000", 2024-09-21T21:41:51.207 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.207 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.207 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.207 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.207 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.045813+0000", 2024-09-21T21:41:51.207 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.2200000000000002e-07 2024-09-21T21:41:51.207 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.207 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.207 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.207 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.045817+0000", 2024-09-21T21:41:51.207 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8999999999999999e-06 2024-09-21T21:41:51.207 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.207 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.208 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.208 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.045829+0000", 2024-09-21T21:41:51.208 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2036e-05 2024-09-21T21:41:51.208 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.208 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.208 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.208 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.045836+0000", 2024-09-21T21:41:51.208 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3860000000000001e-06 2024-09-21T21:41:51.208 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.208 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.208 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.208 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.208 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.208 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:16886 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.209 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.048917+0000", 2024-09-21T21:41:51.209 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.017768244999999998, 2024-09-21T21:41:51.209 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.019293767, 2024-09-21T21:41:51.209 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.209 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.209 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.209 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.209 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.209 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.209 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 16886 2024-09-21T21:41:51.209 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.209 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.209 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.209 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.209 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.048917+0000", 2024-09-21T21:41:51.210 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.210 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.210 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.210 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.210 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.048917+0000", 2024-09-21T21:41:51.210 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.210 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.210 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.210 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.210 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.048918+0000", 2024-09-21T21:41:51.210 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6000000000000003e-07 2024-09-21T21:41:51.210 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.210 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.210 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.210 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.048921+0000", 2024-09-21T21:41:51.211 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6009999999999999e-06 2024-09-21T21:41:51.211 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.211 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.211 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.211 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.048931+0000", 2024-09-21T21:41:51.211 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5060000000000001e-06 2024-09-21T21:41:51.211 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.211 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.211 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.211 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.048937+0000", 2024-09-21T21:41:51.211 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3629999999999999e-06 2024-09-21T21:41:51.211 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.211 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.211 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.212 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.212 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.212 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:17084 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.212 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.056725+0000", 2024-09-21T21:41:51.212 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0099603480000000008, 2024-09-21T21:41:51.212 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.011512395, 2024-09-21T21:41:51.212 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.212 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.212 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.212 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.212 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.212 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.212 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 17084 2024-09-21T21:41:51.212 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.213 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.213 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.213 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.213 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.056725+0000", 2024-09-21T21:41:51.213 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.213 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.213 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.213 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.213 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.056725+0000", 2024-09-21T21:41:51.213 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.213 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.213 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.213 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.213 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.056725+0000", 2024-09-21T21:41:51.213 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5600000000000001e-07 2024-09-21T21:41:51.214 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.214 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.214 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.214 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.056727+0000", 2024-09-21T21:41:51.214 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0549999999999998e-06 2024-09-21T21:41:51.214 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.214 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.214 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.214 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.056732+0000", 2024-09-21T21:41:51.214 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6349999999999997e-06 2024-09-21T21:41:51.214 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.214 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.214 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.214 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.056736+0000", 2024-09-21T21:41:51.215 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0589999999999996e-06 2024-09-21T21:41:51.215 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.215 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.215 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.215 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.215 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.215 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:16767 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.215 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.042146+0000", 2024-09-21T21:41:51.215 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.024539288999999999, 2024-09-21T21:41:51.215 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.026117600000000001, 2024-09-21T21:41:51.215 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.215 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.215 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.215 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.215 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.216 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.216 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 16767 2024-09-21T21:41:51.216 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.216 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.216 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.216 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.216 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.042146+0000", 2024-09-21T21:41:51.216 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.216 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.216 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.216 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.216 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.042146+0000", 2024-09-21T21:41:51.216 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.216 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.217 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.217 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.217 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.042147+0000", 2024-09-21T21:41:51.217 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3900000000000004e-07 2024-09-21T21:41:51.217 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.217 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.217 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.217 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.042149+0000", 2024-09-21T21:41:51.217 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7219999999999999e-06 2024-09-21T21:41:51.217 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.217 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.217 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.217 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.042157+0000", 2024-09-21T21:41:51.217 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5170000000000002e-06 2024-09-21T21:41:51.217 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.218 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.218 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.218 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.042163+0000", 2024-09-21T21:41:51.218 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8359999999999997e-06 2024-09-21T21:41:51.218 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.218 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.218 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.218 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.218 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.218 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:16831 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.218 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.045847+0000", 2024-09-21T21:41:51.218 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.020837954999999998, 2024-09-21T21:41:51.218 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.022442714999999998, 2024-09-21T21:41:51.218 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.218 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.219 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.219 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.219 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.219 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.219 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 16831 2024-09-21T21:41:51.219 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.219 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.219 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.219 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.219 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.045847+0000", 2024-09-21T21:41:51.219 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.219 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.219 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.219 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.220 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.045847+0000", 2024-09-21T21:41:51.220 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.220 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.220 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.220 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.220 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.045848+0000", 2024-09-21T21:41:51.220 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5799999999999999e-07 2024-09-21T21:41:51.220 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.220 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.220 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.220 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.045850+0000", 2024-09-21T21:41:51.220 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.469e-06 2024-09-21T21:41:51.221 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.221 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.221 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.221 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.045858+0000", 2024-09-21T21:41:51.221 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.537e-06 2024-09-21T21:41:51.221 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.222 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.222 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.222 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.045871+0000", 2024-09-21T21:41:51.222 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.33e-05 2024-09-21T21:41:51.222 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.222 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.222 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.222 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.222 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.222 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:16887 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.222 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.048958+0000", 2024-09-21T21:41:51.222 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.017727388, 2024-09-21T21:41:51.223 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.019364407, 2024-09-21T21:41:51.223 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.223 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.223 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.223 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.223 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.223 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.223 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 16887 2024-09-21T21:41:51.223 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.223 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.223 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.223 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.223 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.048958+0000", 2024-09-21T21:41:51.223 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.224 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.224 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.224 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.224 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.048958+0000", 2024-09-21T21:41:51.224 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.224 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.224 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.224 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.224 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.048958+0000", 2024-09-21T21:41:51.224 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4499999999999997e-07 2024-09-21T21:41:51.224 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.224 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.224 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.224 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.048961+0000", 2024-09-21T21:41:51.225 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.92e-06 2024-09-21T21:41:51.225 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.225 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.225 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.225 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.048968+0000", 2024-09-21T21:41:51.225 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9179999999999997e-06 2024-09-21T21:41:51.225 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.225 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.225 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.225 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.048974+0000", 2024-09-21T21:41:51.225 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9089999999999997e-06 2024-09-21T21:41:51.225 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.225 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.225 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.226 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.226 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.226 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:17085 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.226 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.056781+0000", 2024-09-21T21:41:51.226 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.009903729, 2024-09-21T21:41:51.226 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.011567600000000001, 2024-09-21T21:41:51.226 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.226 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.226 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.226 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.226 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.226 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.226 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 17085 2024-09-21T21:41:51.226 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.227 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.227 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.227 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.227 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.056781+0000", 2024-09-21T21:41:51.227 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.227 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.227 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.227 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.227 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.056781+0000", 2024-09-21T21:41:51.227 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.227 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.227 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.227 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.227 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.056782+0000", 2024-09-21T21:41:51.228 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3299999999999998e-07 2024-09-21T21:41:51.228 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.228 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.228 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.228 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.056783+0000", 2024-09-21T21:41:51.228 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.629e-06 2024-09-21T21:41:51.228 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.228 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.228 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.228 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.056788+0000", 2024-09-21T21:41:51.228 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2019999999999997e-06 2024-09-21T21:41:51.228 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.228 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.228 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.229 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.056791+0000", 2024-09-21T21:41:51.229 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1190000000000001e-06 2024-09-21T21:41:51.229 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.229 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.229 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.229 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.229 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.229 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:17202 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.229 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.062795+0000", 2024-09-21T21:41:51.229 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0038904759999999999, 2024-09-21T21:41:51.229 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0055990600000000003, 2024-09-21T21:41:51.229 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.229 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.229 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.230 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.230 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.230 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.230 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 17202 2024-09-21T21:41:51.230 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.230 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.230 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.230 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.230 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.062795+0000", 2024-09-21T21:41:51.230 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.230 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.230 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.231 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.231 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.062795+0000", 2024-09-21T21:41:51.231 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.231 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.231 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.231 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.231 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.062795+0000", 2024-09-21T21:41:51.231 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5700000000000002e-07 2024-09-21T21:41:51.231 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.231 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.231 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.231 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.062799+0000", 2024-09-21T21:41:51.231 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2480000000000001e-06 2024-09-21T21:41:51.232 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.232 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.232 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.232 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.062806+0000", 2024-09-21T21:41:51.232 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.08e-06 2024-09-21T21:41:51.232 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.232 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.232 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.232 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.062812+0000", 2024-09-21T21:41:51.232 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.596e-06 2024-09-21T21:41:51.232 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.232 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.232 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.232 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.233 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.233 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:17258 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.233 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.064678+0000", 2024-09-21T21:41:51.233 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.002007141, 2024-09-21T21:41:51.233 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0037777000000000002, 2024-09-21T21:41:51.233 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.233 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.233 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.233 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.233 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.233 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.233 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 17258 2024-09-21T21:41:51.233 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.233 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.234 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.234 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.234 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.064678+0000", 2024-09-21T21:41:51.234 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.234 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.234 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.234 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.234 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.064678+0000", 2024-09-21T21:41:51.234 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.234 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.234 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.234 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.234 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.064679+0000", 2024-09-21T21:41:51.235 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7599999999999996e-07 2024-09-21T21:41:51.235 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.235 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.235 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.235 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.064682+0000", 2024-09-21T21:41:51.235 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6629999999999999e-06 2024-09-21T21:41:51.235 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.235 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.235 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.235 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.064690+0000", 2024-09-21T21:41:51.235 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5719999999999997e-06 2024-09-21T21:41:51.235 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.235 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.235 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.236 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.064696+0000", 2024-09-21T21:41:51.236 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0229999999999998e-06 2024-09-21T21:41:51.236 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.236 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.236 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.236 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.236 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.236 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:16888 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.236 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.049066+0000", 2024-09-21T21:41:51.236 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.017618690999999999, 2024-09-21T21:41:51.236 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.019442658000000002, 2024-09-21T21:41:51.236 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.236 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.237 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.237 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.237 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.237 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.237 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 16888 2024-09-21T21:41:51.237 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.237 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.237 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.237 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.237 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.049066+0000", 2024-09-21T21:41:51.237 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.237 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.237 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.237 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.238 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.049066+0000", 2024-09-21T21:41:51.238 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.238 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.238 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.238 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.238 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.049067+0000", 2024-09-21T21:41:51.238 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1200000000000002e-07 2024-09-21T21:41:51.238 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.238 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.238 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.238 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.049071+0000", 2024-09-21T21:41:51.238 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6710000000000001e-06 2024-09-21T21:41:51.238 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.238 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.238 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.239 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.049079+0000", 2024-09-21T21:41:51.239 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8290000000000008e-06 2024-09-21T21:41:51.239 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.239 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.239 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.239 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.049084+0000", 2024-09-21T21:41:51.239 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5929999999999996e-06 2024-09-21T21:41:51.239 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.239 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.239 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.239 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.239 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.239 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:17086 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.240 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.056797+0000", 2024-09-21T21:41:51.240 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0098884039999999999, 2024-09-21T21:41:51.240 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.011764350999999999, 2024-09-21T21:41:51.240 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.240 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.240 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.240 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.240 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.240 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.240 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 17086 2024-09-21T21:41:51.240 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.240 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.240 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.241 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.241 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.056797+0000", 2024-09-21T21:41:51.241 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.241 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.241 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.241 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.241 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.056797+0000", 2024-09-21T21:41:51.241 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.241 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.241 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.241 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.241 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.056797+0000", 2024-09-21T21:41:51.241 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2999999999999999e-07 2024-09-21T21:41:51.241 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.242 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.242 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.242 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.056798+0000", 2024-09-21T21:41:51.242 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2640000000000001e-06 2024-09-21T21:41:51.242 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.242 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.242 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.242 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.056801+0000", 2024-09-21T21:41:51.242 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.134e-06 2024-09-21T21:41:51.242 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.242 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.242 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.242 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.056804+0000", 2024-09-21T21:41:51.242 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.711e-06 2024-09-21T21:41:51.243 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.243 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.243 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.243 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.243 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.243 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:17140 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.243 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.059881+0000", 2024-09-21T21:41:51.243 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0068042140000000003, 2024-09-21T21:41:51.243 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0087309399999999995, 2024-09-21T21:41:51.243 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.243 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.243 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.243 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.243 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.243 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.244 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 17140 2024-09-21T21:41:51.244 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.244 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.244 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.244 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.244 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.059881+0000", 2024-09-21T21:41:51.244 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.244 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.244 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.244 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.244 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.059881+0000", 2024-09-21T21:41:51.244 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.244 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.245 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.245 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.245 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.059882+0000", 2024-09-21T21:41:51.245 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7000000000000004e-07 2024-09-21T21:41:51.245 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.245 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.245 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.245 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.059885+0000", 2024-09-21T21:41:51.245 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4659999999999999e-06 2024-09-21T21:41:51.245 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.245 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.245 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.245 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.059895+0000", 2024-09-21T21:41:51.245 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.7510000000000003e-06 2024-09-21T21:41:51.246 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.246 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.246 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.246 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.059901+0000", 2024-09-21T21:41:51.246 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4520000000000002e-06 2024-09-21T21:41:51.246 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.246 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.246 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.246 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.246 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.246 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:17203 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.246 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.062850+0000", 2024-09-21T21:41:51.246 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0038349909999999998, 2024-09-21T21:41:51.247 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0058131729999999996, 2024-09-21T21:41:51.247 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.247 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.247 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.247 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.247 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.247 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.247 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 17203 2024-09-21T21:41:51.247 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.247 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.247 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.247 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.247 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.062850+0000", 2024-09-21T21:41:51.247 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.248 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.248 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.248 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.248 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.062850+0000", 2024-09-21T21:41:51.248 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.248 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.248 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.248 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.248 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.062851+0000", 2024-09-21T21:41:51.248 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4199999999999996e-07 2024-09-21T21:41:51.248 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.248 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.248 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.248 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.062854+0000", 2024-09-21T21:41:51.248 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.847e-06 2024-09-21T21:41:51.249 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.249 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.249 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.249 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.062865+0000", 2024-09-21T21:41:51.249 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1219e-05 2024-09-21T21:41:51.249 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.249 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.249 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.249 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.062870+0000", 2024-09-21T21:41:51.249 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6030000000000004e-06 2024-09-21T21:41:51.249 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.249 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.249 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.249 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.250 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.250 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:17259 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.250 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.064710+0000", 2024-09-21T21:41:51.250 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0019746899999999999, 2024-09-21T21:41:51.250 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0040049209999999998, 2024-09-21T21:41:51.250 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.250 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.250 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.250 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.250 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.250 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.250 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 17259 2024-09-21T21:41:51.250 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.250 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.251 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.251 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.251 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.064710+0000", 2024-09-21T21:41:51.251 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.251 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.251 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.251 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.251 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.064710+0000", 2024-09-21T21:41:51.251 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.251 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.251 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.251 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.251 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.064711+0000", 2024-09-21T21:41:51.251 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1900000000000002e-07 2024-09-21T21:41:51.252 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.252 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.252 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.252 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.064714+0000", 2024-09-21T21:41:51.252 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5110000000000002e-06 2024-09-21T21:41:51.252 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.252 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.252 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.252 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.064719+0000", 2024-09-21T21:41:51.252 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6999999999999996e-06 2024-09-21T21:41:51.252 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.252 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.252 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.252 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.064724+0000", 2024-09-21T21:41:51.252 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8910000000000003e-06 2024-09-21T21:41:51.253 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.253 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.253 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.253 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.253 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.253 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:17141 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.253 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.059988+0000", 2024-09-21T21:41:51.253 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0066968449999999999, 2024-09-21T21:41:51.253 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0087795100000000008, 2024-09-21T21:41:51.253 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.253 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.253 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.253 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.253 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.253 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.254 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 17141 2024-09-21T21:41:51.254 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.254 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.254 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.254 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.254 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.059988+0000", 2024-09-21T21:41:51.254 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.254 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.254 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.254 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.254 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.059988+0000", 2024-09-21T21:41:51.254 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.254 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.254 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.255 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.255 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.059989+0000", 2024-09-21T21:41:51.255 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1699999999999997e-07 2024-09-21T21:41:51.255 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.255 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.255 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.255 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.059993+0000", 2024-09-21T21:41:51.255 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.473e-06 2024-09-21T21:41:51.255 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.255 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.255 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.255 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.060001+0000", 2024-09-21T21:41:51.255 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3909999999999998e-06 2024-09-21T21:41:51.255 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.256 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.256 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.256 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.060007+0000", 2024-09-21T21:41:51.256 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3600000000000001e-06 2024-09-21T21:41:51.256 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.256 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.256 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.256 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.256 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.256 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:16774 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.256 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.042583+0000", 2024-09-21T21:41:51.256 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.024101858, 2024-09-21T21:41:51.256 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.026236846000000001, 2024-09-21T21:41:51.256 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.256 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.257 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.257 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.257 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.257 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.257 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 16774 2024-09-21T21:41:51.257 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.257 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.257 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.257 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.257 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.042583+0000", 2024-09-21T21:41:51.257 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.257 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.257 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.257 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.257 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.042583+0000", 2024-09-21T21:41:51.258 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.258 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.258 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.258 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.258 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.042584+0000", 2024-09-21T21:41:51.258 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4899999999999995e-07 2024-09-21T21:41:51.258 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.258 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.258 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.258 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.042587+0000", 2024-09-21T21:41:51.258 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0419999999999998e-06 2024-09-21T21:41:51.258 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.258 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.258 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.259 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.042593+0000", 2024-09-21T21:41:51.259 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1689999999999997e-06 2024-09-21T21:41:51.259 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.259 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.259 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.259 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.042597+0000", 2024-09-21T21:41:51.259 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1420000000000003e-06 2024-09-21T21:41:51.259 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.259 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.259 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.259 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.259 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.259 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:16775 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.259 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.042609+0000", 2024-09-21T21:41:51.260 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.024076531000000002, 2024-09-21T21:41:51.260 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.026271161000000001, 2024-09-21T21:41:51.260 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.260 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.260 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.260 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.260 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.260 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.260 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 16775 2024-09-21T21:41:51.260 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.260 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.260 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.260 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.260 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.042609+0000", 2024-09-21T21:41:51.261 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.261 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.261 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.261 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.261 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.042609+0000", 2024-09-21T21:41:51.261 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.261 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.261 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.261 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.261 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.042609+0000", 2024-09-21T21:41:51.261 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9299999999999999e-07 2024-09-21T21:41:51.261 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.261 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.261 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.261 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.042611+0000", 2024-09-21T21:41:51.262 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.8670000000000001e-06 2024-09-21T21:41:51.262 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.262 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.262 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.262 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.042615+0000", 2024-09-21T21:41:51.262 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7620000000000001e-06 2024-09-21T21:41:51.262 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.262 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.262 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.262 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.042617+0000", 2024-09-21T21:41:51.262 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.914e-06 2024-09-21T21:41:51.262 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.262 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.262 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.262 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.263 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.263 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:16840 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.263 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.046106+0000", 2024-09-21T21:41:51.263 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.020579666999999999, 2024-09-21T21:41:51.263 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.022826816, 2024-09-21T21:41:51.263 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.263 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.263 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.263 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.263 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.263 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.263 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 16840 2024-09-21T21:41:51.263 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.263 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.263 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.264 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.264 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.046106+0000", 2024-09-21T21:41:51.264 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.264 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.264 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.264 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.264 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.046106+0000", 2024-09-21T21:41:51.264 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.264 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.264 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.264 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.264 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.046106+0000", 2024-09-21T21:41:51.264 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6300000000000004e-07 2024-09-21T21:41:51.264 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.264 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.265 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.265 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.046109+0000", 2024-09-21T21:41:51.265 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1470000000000001e-06 2024-09-21T21:41:51.265 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.265 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.265 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.265 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.046118+0000", 2024-09-21T21:41:51.265 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9660000000000002e-06 2024-09-21T21:41:51.265 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.265 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.265 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.265 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.046125+0000", 2024-09-21T21:41:51.265 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2949999999999999e-06 2024-09-21T21:41:51.265 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.266 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.266 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.266 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.266 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.266 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:16895 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.266 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.049608+0000", 2024-09-21T21:41:51.266 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.01707724, 2024-09-21T21:41:51.266 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.019364069000000001, 2024-09-21T21:41:51.266 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.266 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.266 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.266 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.266 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.266 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.267 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 16895 2024-09-21T21:41:51.267 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.267 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.267 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.267 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.267 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.049608+0000", 2024-09-21T21:41:51.267 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.267 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.267 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.267 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.267 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.049608+0000", 2024-09-21T21:41:51.267 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.268 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.268 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.268 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.268 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.049613+0000", 2024-09-21T21:41:51.268 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6009999999999997e-06 2024-09-21T21:41:51.268 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.268 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.268 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.268 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.049617+0000", 2024-09-21T21:41:51.268 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2169999999999996e-06 2024-09-21T21:41:51.268 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.268 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.268 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.268 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.049628+0000", 2024-09-21T21:41:51.269 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.077e-05 2024-09-21T21:41:51.269 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.269 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.269 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.269 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.049635+0000", 2024-09-21T21:41:51.269 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2150000000000004e-06 2024-09-21T21:41:51.269 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.269 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.269 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.269 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.269 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.269 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:17208 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.269 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.062934+0000", 2024-09-21T21:41:51.269 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0037508289999999998, 2024-09-21T21:41:51.270 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0060645159999999998, 2024-09-21T21:41:51.270 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.270 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.270 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.270 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.270 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.270 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.270 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 17208 2024-09-21T21:41:51.270 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.270 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.270 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.270 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.270 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.062934+0000", 2024-09-21T21:41:51.270 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.270 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.271 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.271 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.271 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.062934+0000", 2024-09-21T21:41:51.271 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.271 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.271 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.271 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.271 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.062935+0000", 2024-09-21T21:41:51.271 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3599999999999999e-07 2024-09-21T21:41:51.271 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.271 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.271 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.271 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.062936+0000", 2024-09-21T21:41:51.271 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5009999999999999e-06 2024-09-21T21:41:51.271 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.272 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.272 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.272 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.062940+0000", 2024-09-21T21:41:51.272 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9480000000000001e-06 2024-09-21T21:41:51.272 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.272 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.272 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.272 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.062943+0000", 2024-09-21T21:41:51.272 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9050000000000001e-06 2024-09-21T21:41:51.272 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.272 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.272 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.272 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.272 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.273 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:16841 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.273 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.046169+0000", 2024-09-21T21:41:51.273 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.020515756, 2024-09-21T21:41:51.273 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.022856853999999999, 2024-09-21T21:41:51.273 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.273 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.273 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.273 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.273 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.273 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.273 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 16841 2024-09-21T21:41:51.273 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.273 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.273 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.273 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.274 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.046169+0000", 2024-09-21T21:41:51.274 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.274 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.274 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.274 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.274 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.046169+0000", 2024-09-21T21:41:51.274 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.274 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.274 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.274 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.274 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.046170+0000", 2024-09-21T21:41:51.274 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7299999999999995e-07 2024-09-21T21:41:51.275 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.275 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.275 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.275 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.046173+0000", 2024-09-21T21:41:51.275 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1879999999999998e-06 2024-09-21T21:41:51.275 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.275 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.275 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.275 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.046182+0000", 2024-09-21T21:41:51.275 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3089999999999996e-06 2024-09-21T21:41:51.275 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.275 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.275 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.275 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.046188+0000", 2024-09-21T21:41:51.276 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7000000000000002e-06 2024-09-21T21:41:51.276 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.276 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.276 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.276 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.276 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.276 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:16896 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.276 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.049653+0000", 2024-09-21T21:41:51.276 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.017032348999999999, 2024-09-21T21:41:51.276 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.019399856, 2024-09-21T21:41:51.276 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.276 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.276 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.276 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.276 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.277 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.277 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 16896 2024-09-21T21:41:51.277 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.277 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.277 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.277 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.277 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.049653+0000", 2024-09-21T21:41:51.277 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.277 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.277 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.277 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.277 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.049653+0000", 2024-09-21T21:41:51.277 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.277 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.278 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.278 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.278 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.049653+0000", 2024-09-21T21:41:51.278 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4700000000000001e-07 2024-09-21T21:41:51.278 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.278 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.278 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.278 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.049657+0000", 2024-09-21T21:41:51.278 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0340000000000001e-06 2024-09-21T21:41:51.278 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.278 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.278 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.278 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.049664+0000", 2024-09-21T21:41:51.278 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9559999999999997e-06 2024-09-21T21:41:51.279 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.279 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.279 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.279 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.049670+0000", 2024-09-21T21:41:51.279 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8939999999999998e-06 2024-09-21T21:41:51.279 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.279 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.279 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.279 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.279 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.279 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:17094 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.279 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.057150+0000", 2024-09-21T21:41:51.279 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0095353810000000008, 2024-09-21T21:41:51.279 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.011948347, 2024-09-21T21:41:51.280 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.280 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.280 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.280 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.280 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.280 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.280 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 17094 2024-09-21T21:41:51.280 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.280 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.280 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.280 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.280 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.057150+0000", 2024-09-21T21:41:51.280 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.280 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.280 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.281 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.281 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.057150+0000", 2024-09-21T21:41:51.281 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.281 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.281 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.281 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.281 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.057150+0000", 2024-09-21T21:41:51.281 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0500000000000003e-07 2024-09-21T21:41:51.281 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.281 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.281 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.281 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.057153+0000", 2024-09-21T21:41:51.282 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9529999999999999e-06 2024-09-21T21:41:51.282 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.282 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.282 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.282 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.057160+0000", 2024-09-21T21:41:51.282 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6880000000000002e-06 2024-09-21T21:41:51.282 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.282 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.282 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.282 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.057164+0000", 2024-09-21T21:41:51.282 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.177e-06 2024-09-21T21:41:51.282 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.282 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.282 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.283 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.283 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.283 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:17148 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.283 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.060375+0000", 2024-09-21T21:41:51.283 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.006310268, 2024-09-21T21:41:51.283 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0087804019999999997, 2024-09-21T21:41:51.283 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.283 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.283 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.283 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.283 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.283 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.283 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 17148 2024-09-21T21:41:51.284 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.284 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.284 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.284 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.284 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.060375+0000", 2024-09-21T21:41:51.284 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.284 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.284 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.284 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.284 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.060375+0000", 2024-09-21T21:41:51.284 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.284 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.284 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.284 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.284 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.060376+0000", 2024-09-21T21:41:51.285 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0100000000000004e-07 2024-09-21T21:41:51.285 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.285 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.285 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.285 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.060379+0000", 2024-09-21T21:41:51.285 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0539999999999999e-06 2024-09-21T21:41:51.285 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.285 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.285 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.285 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.060386+0000", 2024-09-21T21:41:51.285 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4909999999999999e-06 2024-09-21T21:41:51.285 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.285 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.285 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.286 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.060392+0000", 2024-09-21T21:41:51.286 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6049999999999997e-06 2024-09-21T21:41:51.286 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.286 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.286 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.286 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.286 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.286 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:17209 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.286 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.062949+0000", 2024-09-21T21:41:51.286 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0037363980000000001, 2024-09-21T21:41:51.286 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0062424239999999999, 2024-09-21T21:41:51.286 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.286 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.287 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.287 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.287 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.287 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.287 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 17209 2024-09-21T21:41:51.287 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.287 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.287 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.287 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.287 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.062949+0000", 2024-09-21T21:41:51.287 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.287 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.287 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.287 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.288 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.062949+0000", 2024-09-21T21:41:51.288 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.288 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.288 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.288 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.288 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.062949+0000", 2024-09-21T21:41:51.288 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4900000000000002e-07 2024-09-21T21:41:51.288 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.288 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.288 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.288 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.062950+0000", 2024-09-21T21:41:51.288 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.175e-06 2024-09-21T21:41:51.288 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.288 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.289 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.289 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.062953+0000", 2024-09-21T21:41:51.289 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.179e-06 2024-09-21T21:41:51.289 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.289 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.289 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.289 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.062957+0000", 2024-09-21T21:41:51.289 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6210000000000001e-06 2024-09-21T21:41:51.289 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.289 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.289 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.289 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.289 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.289 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:16842 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.290 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.046201+0000", 2024-09-21T21:41:51.290 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.020484539, 2024-09-21T21:41:51.290 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.023017973000000001, 2024-09-21T21:41:51.290 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.290 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.290 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.290 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.290 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.290 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.290 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 16842 2024-09-21T21:41:51.290 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.290 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.290 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.291 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.291 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.046201+0000", 2024-09-21T21:41:51.291 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.291 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.291 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.291 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.291 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.046201+0000", 2024-09-21T21:41:51.291 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.291 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.291 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.291 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.291 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.046201+0000", 2024-09-21T21:41:51.292 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6599999999999996e-07 2024-09-21T21:41:51.292 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.292 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.292 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.292 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.046204+0000", 2024-09-21T21:41:51.292 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5629999999999999e-06 2024-09-21T21:41:51.292 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.292 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.292 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.292 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.046211+0000", 2024-09-21T21:41:51.292 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1539999999999996e-06 2024-09-21T21:41:51.292 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.292 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.292 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.293 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.046216+0000", 2024-09-21T21:41:51.293 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.31e-06 2024-09-21T21:41:51.293 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.293 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.293 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.293 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.293 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.293 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:16897 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.293 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.049758+0000", 2024-09-21T21:41:51.293 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.016927212000000001, 2024-09-21T21:41:51.293 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.019488892000000001, 2024-09-21T21:41:51.293 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.293 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.294 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.294 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.294 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.294 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.294 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 16897 2024-09-21T21:41:51.294 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.294 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.294 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.294 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.294 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.049758+0000", 2024-09-21T21:41:51.294 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.294 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.294 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.294 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.295 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.049758+0000", 2024-09-21T21:41:51.295 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.295 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.295 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.295 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.295 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.049759+0000", 2024-09-21T21:41:51.295 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.09e-07 2024-09-21T21:41:51.295 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.295 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.295 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.295 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.049762+0000", 2024-09-21T21:41:51.295 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.658e-06 2024-09-21T21:41:51.295 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.295 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.296 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.296 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.049774+0000", 2024-09-21T21:41:51.296 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1464000000000001e-05 2024-09-21T21:41:51.296 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.296 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.296 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.296 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.049781+0000", 2024-09-21T21:41:51.296 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.63e-06 2024-09-21T21:41:51.296 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.296 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.296 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.296 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.296 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.296 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:17095 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.296 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.057350+0000", 2024-09-21T21:41:51.297 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0093353710000000003, 2024-09-21T21:41:51.297 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.011924119, 2024-09-21T21:41:51.297 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.297 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.297 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.297 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.297 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.297 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.297 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 17095 2024-09-21T21:41:51.297 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.297 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.297 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.297 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.297 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.057350+0000", 2024-09-21T21:41:51.297 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.297 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.298 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.298 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.298 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.057350+0000", 2024-09-21T21:41:51.298 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.298 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.298 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.298 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.298 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.057351+0000", 2024-09-21T21:41:51.298 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7199999999999997e-07 2024-09-21T21:41:51.298 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.298 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.298 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.298 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.057353+0000", 2024-09-21T21:41:51.299 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6599999999999999e-06 2024-09-21T21:41:51.299 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.299 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.299 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.299 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.057361+0000", 2024-09-21T21:41:51.299 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8909999999999995e-06 2024-09-21T21:41:51.299 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.299 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.299 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.299 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.057367+0000", 2024-09-21T21:41:51.299 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9410000000000004e-06 2024-09-21T21:41:51.299 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.299 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.299 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.300 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.300 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.300 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:17149 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.300 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.060400+0000", 2024-09-21T21:41:51.300 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0062855680000000001, 2024-09-21T21:41:51.300 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0089005999999999998, 2024-09-21T21:41:51.300 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.300 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.300 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.300 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.300 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.300 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.300 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 17149 2024-09-21T21:41:51.301 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.301 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.301 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.301 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.301 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.060400+0000", 2024-09-21T21:41:51.301 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.301 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.301 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.301 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.301 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.060400+0000", 2024-09-21T21:41:51.301 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.301 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.301 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.301 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.301 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.060400+0000", 2024-09-21T21:41:51.302 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.75e-07 2024-09-21T21:41:51.302 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.302 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.302 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.302 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.060402+0000", 2024-09-21T21:41:51.302 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1749999999999999e-06 2024-09-21T21:41:51.302 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.302 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.302 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.302 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.060408+0000", 2024-09-21T21:41:51.302 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9959999999999999e-06 2024-09-21T21:41:51.302 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.302 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.303 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.303 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.060413+0000", 2024-09-21T21:41:51.303 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.707e-06 2024-09-21T21:41:51.303 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.303 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.303 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.303 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.303 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.303 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:16784 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.303 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.043026+0000", 2024-09-21T21:41:51.303 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.023659248000000001, 2024-09-21T21:41:51.303 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.026301115, 2024-09-21T21:41:51.303 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.303 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.303 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.304 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.304 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.304 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.304 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 16784 2024-09-21T21:41:51.304 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.304 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.304 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.304 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.304 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.043026+0000", 2024-09-21T21:41:51.304 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.304 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.304 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.304 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.304 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.043026+0000", 2024-09-21T21:41:51.304 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.305 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.305 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.305 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.305 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.043027+0000", 2024-09-21T21:41:51.305 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0399999999999995e-07 2024-09-21T21:41:51.305 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.305 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.305 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.305 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.043030+0000", 2024-09-21T21:41:51.305 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8249999999999998e-06 2024-09-21T21:41:51.305 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.305 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.305 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.305 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.043038+0000", 2024-09-21T21:41:51.306 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4540000000000001e-06 2024-09-21T21:41:51.306 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.306 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.306 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.306 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.043044+0000", 2024-09-21T21:41:51.306 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6899999999999997e-06 2024-09-21T21:41:51.306 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.306 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.306 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.306 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.306 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.306 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:17214 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.306 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.063181+0000", 2024-09-21T21:41:51.306 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0035044149999999999, 2024-09-21T21:41:51.306 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0061723309999999997, 2024-09-21T21:41:51.307 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.307 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.307 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.307 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.307 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.307 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.307 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 17214 2024-09-21T21:41:51.307 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.307 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.307 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.307 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.307 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.063181+0000", 2024-09-21T21:41:51.307 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.307 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.308 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.308 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.308 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.063181+0000", 2024-09-21T21:41:51.308 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.308 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.308 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.308 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.308 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.063181+0000", 2024-09-21T21:41:51.308 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8699999999999995e-07 2024-09-21T21:41:51.308 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.308 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.308 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.308 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.063184+0000", 2024-09-21T21:41:51.308 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5320000000000001e-06 2024-09-21T21:41:51.308 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.309 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.309 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.309 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.063191+0000", 2024-09-21T21:41:51.309 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7279999999999998e-06 2024-09-21T21:41:51.309 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.309 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.309 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.309 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.063196+0000", 2024-09-21T21:41:51.309 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1120000000000004e-06 2024-09-21T21:41:51.309 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.309 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.309 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.309 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.309 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.310 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:16850 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.310 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.046641+0000", 2024-09-21T21:41:51.310 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.020044347000000001, 2024-09-21T21:41:51.310 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.022773906999999999, 2024-09-21T21:41:51.310 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.310 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.310 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.310 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.310 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.310 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.310 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 16850 2024-09-21T21:41:51.310 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.310 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.310 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.310 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.311 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.046641+0000", 2024-09-21T21:41:51.311 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.311 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.311 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.311 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.311 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.046641+0000", 2024-09-21T21:41:51.311 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.311 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.311 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.311 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.311 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.046641+0000", 2024-09-21T21:41:51.311 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9100000000000004e-07 2024-09-21T21:41:51.311 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.311 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.311 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.312 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.046643+0000", 2024-09-21T21:41:51.312 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9180000000000001e-06 2024-09-21T21:41:51.312 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.312 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.312 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.312 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.046651+0000", 2024-09-21T21:41:51.312 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1089999999999998e-06 2024-09-21T21:41:51.312 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.312 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.312 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.312 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.046655+0000", 2024-09-21T21:41:51.312 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7919999999999999e-06 2024-09-21T21:41:51.312 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.312 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.313 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.313 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.313 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.313 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:16904 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.313 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.050235+0000", 2024-09-21T21:41:51.313 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.016450341, 2024-09-21T21:41:51.313 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.019216806999999999, 2024-09-21T21:41:51.313 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.313 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.313 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.313 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.313 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.313 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.313 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 16904 2024-09-21T21:41:51.313 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.314 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.314 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.314 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.314 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.050235+0000", 2024-09-21T21:41:51.314 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.314 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.314 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.314 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.314 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.050235+0000", 2024-09-21T21:41:51.314 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.314 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.314 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.314 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.314 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.050236+0000", 2024-09-21T21:41:51.314 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.4600000000000003e-07 2024-09-21T21:41:51.315 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.315 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.315 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.315 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.050240+0000", 2024-09-21T21:41:51.315 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1629999999999998e-06 2024-09-21T21:41:51.315 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.315 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.315 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.315 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.050249+0000", 2024-09-21T21:41:51.315 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3559999999999999e-06 2024-09-21T21:41:51.315 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.315 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.315 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.315 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.050256+0000", 2024-09-21T21:41:51.315 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7240000000000003e-06 2024-09-21T21:41:51.316 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.316 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.316 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.316 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.316 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.316 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:17100 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.316 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.057734+0000", 2024-09-21T21:41:51.316 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0089513079999999998, 2024-09-21T21:41:51.316 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.01182383, 2024-09-21T21:41:51.316 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.316 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.316 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.316 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.316 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.317 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.317 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 17100 2024-09-21T21:41:51.317 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.317 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.317 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.317 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.317 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.057734+0000", 2024-09-21T21:41:51.317 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.317 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.317 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.317 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.317 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.057734+0000", 2024-09-21T21:41:51.317 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.317 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.317 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.318 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.318 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.057734+0000", 2024-09-21T21:41:51.318 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.82e-07 2024-09-21T21:41:51.318 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.318 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.318 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.318 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.057737+0000", 2024-09-21T21:41:51.318 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.869e-06 2024-09-21T21:41:51.318 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.318 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.318 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.318 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.057743+0000", 2024-09-21T21:41:51.318 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2390000000000002e-06 2024-09-21T21:41:51.318 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.319 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.319 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.319 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.057746+0000", 2024-09-21T21:41:51.319 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8e-06 2024-09-21T21:41:51.319 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.319 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.319 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.319 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.319 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.319 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:17157 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.319 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.061081+0000", 2024-09-21T21:41:51.319 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0056045299999999999, 2024-09-21T21:41:51.319 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0085173060000000005, 2024-09-21T21:41:51.319 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.319 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.320 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.320 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.320 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.320 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.320 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 17157 2024-09-21T21:41:51.320 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.320 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.320 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.320 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.320 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.061081+0000", 2024-09-21T21:41:51.320 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.320 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.320 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.320 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.321 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.061081+0000", 2024-09-21T21:41:51.321 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.321 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.321 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.321 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.321 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.061081+0000", 2024-09-21T21:41:51.321 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4700000000000001e-07 2024-09-21T21:41:51.321 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.321 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.321 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.321 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.061086+0000", 2024-09-21T21:41:51.321 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1819999999999999e-06 2024-09-21T21:41:51.321 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.321 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.321 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.322 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.061093+0000", 2024-09-21T21:41:51.322 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5329999999999997e-06 2024-09-21T21:41:51.322 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.322 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.322 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.322 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.061099+0000", 2024-09-21T21:41:51.322 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2269999999999998e-06 2024-09-21T21:41:51.322 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.322 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.322 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.322 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.322 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.322 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:17215 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.322 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.063203+0000", 2024-09-21T21:41:51.322 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0034820670000000001, 2024-09-21T21:41:51.323 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0064247920000000003, 2024-09-21T21:41:51.323 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.323 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.323 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.323 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.323 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.323 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.323 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 17215 2024-09-21T21:41:51.323 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.323 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.323 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.323 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.323 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.063203+0000", 2024-09-21T21:41:51.323 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.324 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.324 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.324 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.324 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.063203+0000", 2024-09-21T21:41:51.324 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.324 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.324 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.324 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.324 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.063204+0000", 2024-09-21T21:41:51.324 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0400000000000002e-07 2024-09-21T21:41:51.324 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.324 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.324 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.324 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.063206+0000", 2024-09-21T21:41:51.324 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.035e-06 2024-09-21T21:41:51.325 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.325 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.325 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.325 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.063211+0000", 2024-09-21T21:41:51.325 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7790000000000001e-06 2024-09-21T21:41:51.325 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.325 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.325 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.325 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.063216+0000", 2024-09-21T21:41:51.325 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7539999999999997e-06 2024-09-21T21:41:51.325 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.325 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.325 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.325 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.325 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.326 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:16786 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.326 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.043195+0000", 2024-09-21T21:41:51.326 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.023490098000000001, 2024-09-21T21:41:51.326 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.02647205, 2024-09-21T21:41:51.326 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.326 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.326 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.326 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.326 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.326 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.326 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 16786 2024-09-21T21:41:51.326 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.326 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.326 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.327 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.327 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.043195+0000", 2024-09-21T21:41:51.327 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.327 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.327 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.327 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.327 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.043195+0000", 2024-09-21T21:41:51.327 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.327 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.327 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.327 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.327 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.043196+0000", 2024-09-21T21:41:51.327 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1500000000000004e-07 2024-09-21T21:41:51.327 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.328 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.328 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.328 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.043199+0000", 2024-09-21T21:41:51.328 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0410000000000002e-06 2024-09-21T21:41:51.328 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.328 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.328 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.328 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.043206+0000", 2024-09-21T21:41:51.328 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4449999999999999e-06 2024-09-21T21:41:51.328 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.328 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.328 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.328 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.043212+0000", 2024-09-21T21:41:51.328 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4550000000000003e-06 2024-09-21T21:41:51.328 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.329 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.329 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.329 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.329 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.329 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:16851 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.329 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.046814+0000", 2024-09-21T21:41:51.329 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.019871468999999999, 2024-09-21T21:41:51.329 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.022883753, 2024-09-21T21:41:51.329 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.329 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.329 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.329 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.329 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.329 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.329 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 16851 2024-09-21T21:41:51.330 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.330 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.330 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.330 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.330 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.046814+0000", 2024-09-21T21:41:51.330 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.330 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.330 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.330 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.330 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.046814+0000", 2024-09-21T21:41:51.330 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.330 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.330 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.331 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.331 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.046814+0000", 2024-09-21T21:41:51.331 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4099999999999999e-07 2024-09-21T21:41:51.331 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.331 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.331 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.331 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.046817+0000", 2024-09-21T21:41:51.331 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2869999999999999e-06 2024-09-21T21:41:51.331 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.331 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.331 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.331 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.046821+0000", 2024-09-21T21:41:51.331 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8350000000000003e-06 2024-09-21T21:41:51.331 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.331 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.332 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.332 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.046825+0000", 2024-09-21T21:41:51.332 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2499999999999998e-06 2024-09-21T21:41:51.332 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.332 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.332 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.332 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.332 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.332 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:16905 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.332 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.050292+0000", 2024-09-21T21:41:51.332 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.016393562, 2024-09-21T21:41:51.332 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.019435352999999999, 2024-09-21T21:41:51.332 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.332 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.333 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.333 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.333 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.333 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.333 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 16905 2024-09-21T21:41:51.333 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.333 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.333 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.333 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.333 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.050292+0000", 2024-09-21T21:41:51.333 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.333 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.333 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.333 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.333 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.050292+0000", 2024-09-21T21:41:51.334 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.334 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.334 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.334 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.334 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.050292+0000", 2024-09-21T21:41:51.334 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1900000000000002e-07 2024-09-21T21:41:51.334 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.334 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.334 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.334 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.050295+0000", 2024-09-21T21:41:51.334 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1049999999999999e-06 2024-09-21T21:41:51.334 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.334 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.334 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.335 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.050303+0000", 2024-09-21T21:41:51.335 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8800000000000008e-06 2024-09-21T21:41:51.335 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.335 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.335 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.335 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.050310+0000", 2024-09-21T21:41:51.335 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8390000000000001e-06 2024-09-21T21:41:51.335 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.335 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.335 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.335 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.335 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.335 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:17101 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.335 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.057897+0000", 2024-09-21T21:41:51.336 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0087876840000000005, 2024-09-21T21:41:51.336 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.011857872, 2024-09-21T21:41:51.336 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.336 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.336 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.336 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.336 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.336 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.336 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 17101 2024-09-21T21:41:51.336 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.336 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.336 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.336 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.336 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.057897+0000", 2024-09-21T21:41:51.336 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.337 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.337 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.337 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.337 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.057897+0000", 2024-09-21T21:41:51.337 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.337 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.337 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.337 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.337 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.057898+0000", 2024-09-21T21:41:51.337 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1500000000000004e-07 2024-09-21T21:41:51.337 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.337 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.337 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.337 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.057902+0000", 2024-09-21T21:41:51.337 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5099999999999999e-06 2024-09-21T21:41:51.338 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.338 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.338 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.338 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.057914+0000", 2024-09-21T21:41:51.338 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2548000000000001e-05 2024-09-21T21:41:51.338 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.338 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.338 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.338 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.057920+0000", 2024-09-21T21:41:51.338 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1820000000000003e-06 2024-09-21T21:41:51.338 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.338 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.338 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.338 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.338 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.339 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:17158 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.339 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.061113+0000", 2024-09-21T21:41:51.339 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0055725840000000002, 2024-09-21T21:41:51.339 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0086839050000000004, 2024-09-21T21:41:51.339 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.339 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.339 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.339 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.339 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.339 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.339 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 17158 2024-09-21T21:41:51.339 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.339 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.339 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.340 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.340 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.061113+0000", 2024-09-21T21:41:51.340 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.340 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.340 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.340 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.340 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.061113+0000", 2024-09-21T21:41:51.340 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.340 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.340 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.340 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.340 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.061113+0000", 2024-09-21T21:41:51.340 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0900000000000002e-07 2024-09-21T21:41:51.340 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.340 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.341 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.341 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.061115+0000", 2024-09-21T21:41:51.341 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3530000000000002e-06 2024-09-21T21:41:51.341 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.341 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.341 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.341 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.061122+0000", 2024-09-21T21:41:51.341 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3470000000000004e-06 2024-09-21T21:41:51.341 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.341 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.341 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.341 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.061127+0000", 2024-09-21T21:41:51.341 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8060000000000002e-06 2024-09-21T21:41:51.341 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.342 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.342 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.342 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.342 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.342 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:17221 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.342 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.063581+0000", 2024-09-21T21:41:51.342 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.003104402, 2024-09-21T21:41:51.342 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0062607840000000001, 2024-09-21T21:41:51.342 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.342 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.342 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.342 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.342 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.342 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.342 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 17221 2024-09-21T21:41:51.343 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.343 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.343 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.343 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.343 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.063581+0000", 2024-09-21T21:41:51.343 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.343 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.343 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.343 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.343 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.063581+0000", 2024-09-21T21:41:51.343 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.343 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.343 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.343 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.344 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.063581+0000", 2024-09-21T21:41:51.344 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.37e-07 2024-09-21T21:41:51.344 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.344 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.344 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.344 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.063584+0000", 2024-09-21T21:41:51.344 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.367e-06 2024-09-21T21:41:51.344 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.344 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.344 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.344 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.063588+0000", 2024-09-21T21:41:51.344 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.476e-06 2024-09-21T21:41:51.344 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.344 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.345 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.345 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.063593+0000", 2024-09-21T21:41:51.345 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1460000000000004e-06 2024-09-21T21:41:51.345 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.345 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.345 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.345 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.345 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.345 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:16794 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.345 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.043597+0000", 2024-09-21T21:41:51.345 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.023088516999999999, 2024-09-21T21:41:51.345 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.026309847000000001, 2024-09-21T21:41:51.345 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.345 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.346 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.346 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.346 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.346 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.346 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 16794 2024-09-21T21:41:51.346 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.346 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.346 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.346 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.346 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.043597+0000", 2024-09-21T21:41:51.346 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.346 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.346 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.346 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.347 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.043597+0000", 2024-09-21T21:41:51.347 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.347 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.347 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.347 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.347 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.043598+0000", 2024-09-21T21:41:51.347 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8899999999999998e-07 2024-09-21T21:41:51.347 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.347 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.347 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.347 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.043602+0000", 2024-09-21T21:41:51.347 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3089999999999997e-06 2024-09-21T21:41:51.347 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.347 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.348 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.348 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.043617+0000", 2024-09-21T21:41:51.348 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4858000000000001e-05 2024-09-21T21:41:51.348 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.348 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.348 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.348 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.043625+0000", 2024-09-21T21:41:51.348 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8800000000000008e-06 2024-09-21T21:41:51.348 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.348 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.348 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.348 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.348 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.348 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:16858 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.349 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.047142+0000", 2024-09-21T21:41:51.349 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.019543105000000002, 2024-09-21T21:41:51.349 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.022808963000000002, 2024-09-21T21:41:51.349 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.349 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.349 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.349 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.349 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.349 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.349 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 16858 2024-09-21T21:41:51.349 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.349 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.349 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.349 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.349 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.047142+0000", 2024-09-21T21:41:51.350 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.350 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.350 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.350 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.350 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.047142+0000", 2024-09-21T21:41:51.350 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.350 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.350 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.350 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.350 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.047143+0000", 2024-09-21T21:41:51.350 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2799999999999995e-07 2024-09-21T21:41:51.350 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.350 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.350 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.350 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.047147+0000", 2024-09-21T21:41:51.351 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8510000000000001e-06 2024-09-21T21:41:51.351 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.351 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.351 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.351 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.047157+0000", 2024-09-21T21:41:51.351 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.9110000000000005e-06 2024-09-21T21:41:51.351 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.351 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.351 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.351 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.047163+0000", 2024-09-21T21:41:51.351 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9900000000000002e-06 2024-09-21T21:41:51.351 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.351 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.351 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.352 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.352 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.352 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:16912 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.352 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.050527+0000", 2024-09-21T21:41:51.352 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.016158552, 2024-09-21T21:41:51.352 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.019460159000000001, 2024-09-21T21:41:51.352 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.352 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.352 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.352 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.352 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.352 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.352 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 16912 2024-09-21T21:41:51.352 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.353 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.353 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.353 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.353 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.050527+0000", 2024-09-21T21:41:51.353 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.353 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.353 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.353 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.353 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.050527+0000", 2024-09-21T21:41:51.353 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.353 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.353 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.353 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.353 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.050527+0000", 2024-09-21T21:41:51.353 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3799999999999998e-07 2024-09-21T21:41:51.354 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.354 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.354 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.354 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.050529+0000", 2024-09-21T21:41:51.354 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7260000000000001e-06 2024-09-21T21:41:51.354 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.354 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.354 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.354 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.050533+0000", 2024-09-21T21:41:51.354 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2069999999999997e-06 2024-09-21T21:41:51.354 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.354 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.354 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.354 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.050536+0000", 2024-09-21T21:41:51.355 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9869999999999999e-06 2024-09-21T21:41:51.355 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.355 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.355 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.355 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.355 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.355 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:17106 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.355 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.058127+0000", 2024-09-21T21:41:51.355 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0085583550000000001, 2024-09-21T21:41:51.355 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.011892649, 2024-09-21T21:41:51.355 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.355 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.355 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.355 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.355 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.356 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.356 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 17106 2024-09-21T21:41:51.356 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.356 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.356 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.356 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.356 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.058127+0000", 2024-09-21T21:41:51.356 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.356 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.356 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.356 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.356 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.058127+0000", 2024-09-21T21:41:51.356 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.356 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.356 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.357 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.357 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.058127+0000", 2024-09-21T21:41:51.357 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5699999999999998e-07 2024-09-21T21:41:51.357 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.357 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.357 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.357 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.058130+0000", 2024-09-21T21:41:51.357 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7989999999999998e-06 2024-09-21T21:41:51.357 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.357 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.357 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.357 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.058137+0000", 2024-09-21T21:41:51.357 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1029999999999996e-06 2024-09-21T21:41:51.357 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.358 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.358 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.358 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.058142+0000", 2024-09-21T21:41:51.358 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0409999999999997e-06 2024-09-21T21:41:51.358 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.358 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.358 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.358 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.358 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.358 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:17166 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.358 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.061222+0000", 2024-09-21T21:41:51.358 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0054631339999999997, 2024-09-21T21:41:51.358 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0088310799999999998, 2024-09-21T21:41:51.358 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.359 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.359 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.359 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.359 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.359 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.359 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 17166 2024-09-21T21:41:51.359 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.359 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.359 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.359 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.359 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.061222+0000", 2024-09-21T21:41:51.359 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.359 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.359 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.360 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.360 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.061222+0000", 2024-09-21T21:41:51.360 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.360 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.360 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.360 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.360 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.061223+0000", 2024-09-21T21:41:51.360 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7000000000000004e-07 2024-09-21T21:41:51.360 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.360 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.360 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.360 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.061226+0000", 2024-09-21T21:41:51.360 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.852e-06 2024-09-21T21:41:51.360 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.361 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.361 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.361 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.061233+0000", 2024-09-21T21:41:51.361 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3769999999999998e-06 2024-09-21T21:41:51.361 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.361 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.361 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.361 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.061238+0000", 2024-09-21T21:41:51.361 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2689999999999999e-06 2024-09-21T21:41:51.361 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.361 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.361 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.361 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.361 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.361 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:17222 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.362 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.063698+0000", 2024-09-21T21:41:51.362 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0029876260000000002, 2024-09-21T21:41:51.362 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0064198049999999998, 2024-09-21T21:41:51.362 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.362 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.362 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.362 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.362 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.362 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.362 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 17222 2024-09-21T21:41:51.362 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.362 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.362 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.362 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.363 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.063698+0000", 2024-09-21T21:41:51.363 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.363 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.363 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.363 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.363 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.063698+0000", 2024-09-21T21:41:51.363 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.363 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.363 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.363 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.363 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.063698+0000", 2024-09-21T21:41:51.363 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4600000000000004e-07 2024-09-21T21:41:51.363 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.363 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.363 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.364 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.063702+0000", 2024-09-21T21:41:51.364 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5590000000000001e-06 2024-09-21T21:41:51.364 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.364 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.364 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.364 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.063710+0000", 2024-09-21T21:41:51.364 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7840000000000004e-06 2024-09-21T21:41:51.364 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.364 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.364 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.364 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.063716+0000", 2024-09-21T21:41:51.364 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7499999999999997e-06 2024-09-21T21:41:51.364 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.364 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.364 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.365 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.365 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.365 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:16795 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.365 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.043644+0000", 2024-09-21T21:41:51.365 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.023041051999999999, 2024-09-21T21:41:51.365 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.026557001, 2024-09-21T21:41:51.365 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.365 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.365 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.365 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.365 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.365 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.365 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 16795 2024-09-21T21:41:51.366 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.366 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.366 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.366 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.366 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.043644+0000", 2024-09-21T21:41:51.366 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.366 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.366 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.366 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.366 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.043644+0000", 2024-09-21T21:41:51.366 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.366 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.366 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.366 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.366 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.043645+0000", 2024-09-21T21:41:51.367 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6499999999999999e-07 2024-09-21T21:41:51.367 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.367 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.367 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.367 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.043648+0000", 2024-09-21T21:41:51.367 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8339999999999999e-06 2024-09-21T21:41:51.367 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.367 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.367 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.367 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.043655+0000", 2024-09-21T21:41:51.367 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6429999999999995e-06 2024-09-21T21:41:51.367 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.367 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.367 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.368 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.043661+0000", 2024-09-21T21:41:51.368 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6960000000000002e-06 2024-09-21T21:41:51.368 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.368 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.368 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.368 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.368 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.368 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:16859 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.368 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.047179+0000", 2024-09-21T21:41:51.368 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.019506359000000001, 2024-09-21T21:41:51.368 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.023090907000000001, 2024-09-21T21:41:51.368 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.368 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.368 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.368 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.369 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.369 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.369 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 16859 2024-09-21T21:41:51.369 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.369 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.369 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.369 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.369 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.047179+0000", 2024-09-21T21:41:51.369 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.369 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.369 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.369 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.369 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.047179+0000", 2024-09-21T21:41:51.370 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.370 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.370 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.370 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.370 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.047179+0000", 2024-09-21T21:41:51.370 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1900000000000002e-07 2024-09-21T21:41:51.370 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.370 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.370 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.370 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.047182+0000", 2024-09-21T21:41:51.370 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4250000000000001e-06 2024-09-21T21:41:51.370 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.370 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.370 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.370 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.047189+0000", 2024-09-21T21:41:51.371 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.216e-06 2024-09-21T21:41:51.371 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.371 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.371 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.371 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.047194+0000", 2024-09-21T21:41:51.371 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9629999999999997e-06 2024-09-21T21:41:51.371 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.371 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.371 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.371 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.371 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.371 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:16913 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.371 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.050568+0000", 2024-09-21T21:41:51.371 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.01611711, 2024-09-21T21:41:51.371 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.019777568999999998, 2024-09-21T21:41:51.372 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.372 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.372 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.372 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.372 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.372 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.372 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 16913 2024-09-21T21:41:51.372 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.372 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.372 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.372 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.372 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.050568+0000", 2024-09-21T21:41:51.372 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.372 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.373 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.373 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.373 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.050568+0000", 2024-09-21T21:41:51.373 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.373 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.373 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.373 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.373 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.050568+0000", 2024-09-21T21:41:51.373 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6600000000000002e-07 2024-09-21T21:41:51.373 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.373 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.373 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.373 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.050570+0000", 2024-09-21T21:41:51.373 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7740000000000001e-06 2024-09-21T21:41:51.373 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.373 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.374 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.374 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.050576+0000", 2024-09-21T21:41:51.374 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9959999999999999e-06 2024-09-21T21:41:51.374 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.374 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.374 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.374 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.050579+0000", 2024-09-21T21:41:51.374 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.044e-06 2024-09-21T21:41:51.374 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.374 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.374 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.374 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.374 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.374 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:17107 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.375 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.058153+0000", 2024-09-21T21:41:51.375 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0085326459999999996, 2024-09-21T21:41:51.375 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.012267070999999999, 2024-09-21T21:41:51.375 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.375 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.375 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.375 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.375 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.375 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.375 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 17107 2024-09-21T21:41:51.375 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.375 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.375 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.375 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.376 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.058153+0000", 2024-09-21T21:41:51.376 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.376 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.376 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.376 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.376 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.058153+0000", 2024-09-21T21:41:51.376 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.376 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.376 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.376 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.376 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.058153+0000", 2024-09-21T21:41:51.376 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7999999999999995e-07 2024-09-21T21:41:51.376 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.376 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.377 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.377 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.058156+0000", 2024-09-21T21:41:51.377 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.486e-06 2024-09-21T21:41:51.377 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.377 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.377 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.377 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.058162+0000", 2024-09-21T21:41:51.377 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4370000000000004e-06 2024-09-21T21:41:51.377 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.377 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.377 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.377 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.058167+0000", 2024-09-21T21:41:51.377 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0690000000000001e-06 2024-09-21T21:41:51.378 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.378 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.378 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.378 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.378 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.378 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:17167 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.378 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.061253+0000", 2024-09-21T21:41:51.378 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0054318939999999996, 2024-09-21T21:41:51.378 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0092287099999999993, 2024-09-21T21:41:51.378 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.378 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.378 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.378 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.378 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.379 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.379 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 17167 2024-09-21T21:41:51.379 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.379 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.379 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.379 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.379 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.061253+0000", 2024-09-21T21:41:51.379 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.379 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.379 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.379 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.379 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.061253+0000", 2024-09-21T21:41:51.379 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.379 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.380 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.380 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.380 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.061254+0000", 2024-09-21T21:41:51.380 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0800000000000005e-07 2024-09-21T21:41:51.380 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.380 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.380 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.380 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.061256+0000", 2024-09-21T21:41:51.380 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.508e-06 2024-09-21T21:41:51.380 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.380 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.380 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.380 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.061263+0000", 2024-09-21T21:41:51.380 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5470000000000002e-06 2024-09-21T21:41:51.380 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.381 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.381 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.381 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.061266+0000", 2024-09-21T21:41:51.381 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.613e-06 2024-09-21T21:41:51.381 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.381 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.381 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.381 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.381 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.381 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:16796 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.381 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.043792+0000", 2024-09-21T21:41:51.381 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.022893302000000001, 2024-09-21T21:41:51.381 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.026727414000000001, 2024-09-21T21:41:51.381 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.382 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.382 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.382 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.382 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.382 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.382 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 16796 2024-09-21T21:41:51.382 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.382 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.382 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.382 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.382 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.043792+0000", 2024-09-21T21:41:51.382 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.382 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.382 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.383 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.383 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.043792+0000", 2024-09-21T21:41:51.383 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.383 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.383 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.383 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.383 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.043793+0000", 2024-09-21T21:41:51.383 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5099999999999998e-07 2024-09-21T21:41:51.383 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.383 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.383 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.383 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.043797+0000", 2024-09-21T21:41:51.383 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1389999999999997e-06 2024-09-21T21:41:51.383 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.383 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.384 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.384 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.043807+0000", 2024-09-21T21:41:51.384 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0507e-05 2024-09-21T21:41:51.384 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.384 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.384 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.384 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.043814+0000", 2024-09-21T21:41:51.384 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0979999999999996e-06 2024-09-21T21:41:51.384 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.384 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.384 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.384 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.384 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.384 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:16864 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.384 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.047473+0000", 2024-09-21T21:41:51.385 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.019211762, 2024-09-21T21:41:51.385 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.023072886000000001, 2024-09-21T21:41:51.385 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.385 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.385 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.385 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.385 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.385 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.385 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 16864 2024-09-21T21:41:51.385 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.385 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.385 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.385 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.385 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.047473+0000", 2024-09-21T21:41:51.386 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.386 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.386 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.386 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.386 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.047473+0000", 2024-09-21T21:41:51.386 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.386 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.386 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.386 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.386 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.047474+0000", 2024-09-21T21:41:51.386 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5099999999999999e-07 2024-09-21T21:41:51.386 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.386 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.386 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.387 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.047478+0000", 2024-09-21T21:41:51.387 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0099999999999997e-06 2024-09-21T21:41:51.387 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.387 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.387 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.387 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.047488+0000", 2024-09-21T21:41:51.387 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0294e-05 2024-09-21T21:41:51.387 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.387 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.387 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.387 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.047495+0000", 2024-09-21T21:41:51.387 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8900000000000001e-06 2024-09-21T21:41:51.387 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.387 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.388 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.388 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.388 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.388 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:16921 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.388 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.050708+0000", 2024-09-21T21:41:51.388 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.015977524999999999, 2024-09-21T21:41:51.388 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.019889245, 2024-09-21T21:41:51.388 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.388 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.388 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.388 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.388 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.388 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.388 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 16921 2024-09-21T21:41:51.389 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.389 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.389 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.389 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.389 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.050708+0000", 2024-09-21T21:41:51.389 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.389 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.389 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.389 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.389 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.050708+0000", 2024-09-21T21:41:51.389 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.389 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.389 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.389 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.390 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.050708+0000", 2024-09-21T21:41:51.390 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3300000000000002e-07 2024-09-21T21:41:51.390 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.390 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.390 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.390 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.050712+0000", 2024-09-21T21:41:51.390 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3689999999999998e-06 2024-09-21T21:41:51.390 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.390 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.390 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.390 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.050719+0000", 2024-09-21T21:41:51.390 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4279999999999999e-06 2024-09-21T21:41:51.391 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.391 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.391 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.391 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.050725+0000", 2024-09-21T21:41:51.391 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7169999999999996e-06 2024-09-21T21:41:51.391 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.391 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.391 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.391 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.391 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.391 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:17175 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.391 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.061361+0000", 2024-09-21T21:41:51.391 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0053237579999999996, 2024-09-21T21:41:51.391 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0092787560000000009, 2024-09-21T21:41:51.392 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.392 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.392 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.392 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.392 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.392 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.392 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 17175 2024-09-21T21:41:51.392 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.392 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.392 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.392 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.392 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.061361+0000", 2024-09-21T21:41:51.392 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.392 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.393 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.393 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.393 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.061361+0000", 2024-09-21T21:41:51.393 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.393 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.393 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.393 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.393 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.061362+0000", 2024-09-21T21:41:51.393 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8800000000000002e-07 2024-09-21T21:41:51.393 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.393 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.393 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.393 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.061365+0000", 2024-09-21T21:41:51.393 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2160000000000002e-06 2024-09-21T21:41:51.393 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.394 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.394 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.394 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.061378+0000", 2024-09-21T21:41:51.394 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2781e-05 2024-09-21T21:41:51.394 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.394 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.394 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.394 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.061384+0000", 2024-09-21T21:41:51.394 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3940000000000001e-06 2024-09-21T21:41:51.394 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.394 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.394 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.394 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.394 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.394 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:17231 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.395 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.064184+0000", 2024-09-21T21:41:51.395 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0025008399999999998, 2024-09-21T21:41:51.395 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0065241179999999998, 2024-09-21T21:41:51.395 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.395 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.395 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.395 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.395 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.395 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.395 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 17231 2024-09-21T21:41:51.395 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.395 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.395 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.395 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.395 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.064184+0000", 2024-09-21T21:41:51.396 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.396 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.396 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.396 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.396 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.064184+0000", 2024-09-21T21:41:51.396 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.396 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.396 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.396 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.396 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.064185+0000", 2024-09-21T21:41:51.396 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4300000000000002e-07 2024-09-21T21:41:51.396 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.396 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.396 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.396 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.064188+0000", 2024-09-21T21:41:51.397 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3809999999999999e-06 2024-09-21T21:41:51.397 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.397 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.397 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.397 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.064196+0000", 2024-09-21T21:41:51.397 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8890000000000002e-06 2024-09-21T21:41:51.397 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.397 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.397 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.397 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.064204+0000", 2024-09-21T21:41:51.397 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1779999999999997e-06 2024-09-21T21:41:51.397 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.397 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.397 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.398 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.398 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.398 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:16804 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.398 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.044209+0000", 2024-09-21T21:41:51.398 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.022475868999999999, 2024-09-21T21:41:51.398 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.026570343999999999, 2024-09-21T21:41:51.398 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.398 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.398 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.398 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.398 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.398 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.398 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 16804 2024-09-21T21:41:51.398 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.399 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.399 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.399 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.399 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.044209+0000", 2024-09-21T21:41:51.399 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.399 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.399 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.399 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.399 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.044209+0000", 2024-09-21T21:41:51.399 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.399 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.399 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.399 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.399 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.044210+0000", 2024-09-21T21:41:51.399 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9999999999999996e-07 2024-09-21T21:41:51.400 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.400 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.400 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.400 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.044214+0000", 2024-09-21T21:41:51.400 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7639999999999999e-06 2024-09-21T21:41:51.400 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.400 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.400 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.400 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.044223+0000", 2024-09-21T21:41:51.400 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.0610000000000002e-06 2024-09-21T21:41:51.400 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.400 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.400 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.400 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.044231+0000", 2024-09-21T21:41:51.401 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5979999999999999e-06 2024-09-21T21:41:51.401 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.401 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.401 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.401 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.401 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.401 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:16865 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.401 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.047511+0000", 2024-09-21T21:41:51.401 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.019173823, 2024-09-21T21:41:51.401 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.023329288, 2024-09-21T21:41:51.401 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.401 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.401 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.401 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.401 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.402 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.402 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 16865 2024-09-21T21:41:51.402 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.402 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.402 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.402 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.402 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.047511+0000", 2024-09-21T21:41:51.402 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.402 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.402 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.402 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.402 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.047511+0000", 2024-09-21T21:41:51.402 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.402 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.402 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.403 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.403 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.047512+0000", 2024-09-21T21:41:51.403 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1600000000000001e-07 2024-09-21T21:41:51.403 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.403 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.403 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.403 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.047515+0000", 2024-09-21T21:41:51.403 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8210000000000002e-06 2024-09-21T21:41:51.403 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.403 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.403 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.403 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.047523+0000", 2024-09-21T21:41:51.403 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9999999999999996e-06 2024-09-21T21:41:51.403 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.404 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.404 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.404 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.047529+0000", 2024-09-21T21:41:51.404 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5529999999999998e-06 2024-09-21T21:41:51.404 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.404 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.404 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.404 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.404 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.404 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:16922 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.404 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.050751+0000", 2024-09-21T21:41:51.404 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.015934663000000002, 2024-09-21T21:41:51.404 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.020151202, 2024-09-21T21:41:51.404 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.404 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.405 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.405 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.405 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.405 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.405 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 16922 2024-09-21T21:41:51.405 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.405 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.405 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.405 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.405 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.050751+0000", 2024-09-21T21:41:51.405 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.405 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.405 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.405 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.406 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.050751+0000", 2024-09-21T21:41:51.406 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.406 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.406 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.406 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.406 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.050751+0000", 2024-09-21T21:41:51.406 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1600000000000001e-07 2024-09-21T21:41:51.406 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.406 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.406 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.406 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.050754+0000", 2024-09-21T21:41:51.406 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6400000000000001e-06 2024-09-21T21:41:51.406 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.407 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.407 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.407 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.050761+0000", 2024-09-21T21:41:51.407 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7479999999999996e-06 2024-09-21T21:41:51.407 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.407 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.407 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.407 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.050772+0000", 2024-09-21T21:41:51.407 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1296e-05 2024-09-21T21:41:51.407 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.407 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.407 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.407 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.407 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.407 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:17113 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.408 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.058352+0000", 2024-09-21T21:41:51.408 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0083329219999999996, 2024-09-21T21:41:51.408 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.012601057000000001, 2024-09-21T21:41:51.408 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.408 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.408 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.408 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.408 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.408 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.408 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 17113 2024-09-21T21:41:51.409 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.409 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.409 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.409 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.409 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.058352+0000", 2024-09-21T21:41:51.409 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.409 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.409 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.409 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.409 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.058352+0000", 2024-09-21T21:41:51.409 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.409 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.409 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.410 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.410 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.058353+0000", 2024-09-21T21:41:51.410 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5799999999999998e-07 2024-09-21T21:41:51.410 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.410 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.410 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.410 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.058356+0000", 2024-09-21T21:41:51.410 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0939999999999999e-06 2024-09-21T21:41:51.410 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.410 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.410 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.410 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.058363+0000", 2024-09-21T21:41:51.410 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8369999999999999e-06 2024-09-21T21:41:51.411 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.411 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.411 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.411 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.058369+0000", 2024-09-21T21:41:51.411 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8749999999999997e-06 2024-09-21T21:41:51.411 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.411 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.411 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.411 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.411 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.411 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:17176 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.411 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.061417+0000", 2024-09-21T21:41:51.412 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0052677490000000004, 2024-09-21T21:41:51.412 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0095698799999999994, 2024-09-21T21:41:51.412 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.412 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.412 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.412 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.412 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.412 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.412 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 17176 2024-09-21T21:41:51.412 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.412 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.412 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.412 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.413 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.061417+0000", 2024-09-21T21:41:51.413 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.413 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.413 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.413 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.413 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.061417+0000", 2024-09-21T21:41:51.413 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.413 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.413 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.413 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.413 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.061418+0000", 2024-09-21T21:41:51.413 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1500000000000004e-07 2024-09-21T21:41:51.413 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.414 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.414 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.414 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.061421+0000", 2024-09-21T21:41:51.414 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7879999999999998e-06 2024-09-21T21:41:51.414 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.414 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.414 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.414 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.061428+0000", 2024-09-21T21:41:51.414 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9280000000000004e-06 2024-09-21T21:41:51.414 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.414 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.414 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.414 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.061433+0000", 2024-09-21T21:41:51.415 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5180000000000004e-06 2024-09-21T21:41:51.415 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.415 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.415 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.415 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.415 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.415 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:17232 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.415 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.064218+0000", 2024-09-21T21:41:51.415 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0024672380000000001, 2024-09-21T21:41:51.415 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0068026199999999997, 2024-09-21T21:41:51.415 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.415 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.415 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.416 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.416 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.416 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.416 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 17232 2024-09-21T21:41:51.416 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.416 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.416 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.416 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.416 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.064218+0000", 2024-09-21T21:41:51.416 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.416 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.416 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.416 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.416 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.064218+0000", 2024-09-21T21:41:51.416 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.417 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.417 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.417 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.417 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.064218+0000", 2024-09-21T21:41:51.417 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.58e-07 2024-09-21T21:41:51.417 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.417 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.417 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.417 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.064221+0000", 2024-09-21T21:41:51.417 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.373e-06 2024-09-21T21:41:51.417 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.417 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.417 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.417 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.064227+0000", 2024-09-21T21:41:51.417 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7209999999999997e-06 2024-09-21T21:41:51.418 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.418 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.418 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.418 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.064233+0000", 2024-09-21T21:41:51.418 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0660000000000003e-06 2024-09-21T21:41:51.418 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.418 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.418 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.418 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.418 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.418 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:16805 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.418 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.044353+0000", 2024-09-21T21:41:51.418 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.022332505999999998, 2024-09-21T21:41:51.419 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.026695039, 2024-09-21T21:41:51.419 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.419 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.419 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.419 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.419 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.419 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.419 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 16805 2024-09-21T21:41:51.419 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.419 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.419 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.419 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.419 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.044353+0000", 2024-09-21T21:41:51.420 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.420 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.420 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.420 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.420 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.044353+0000", 2024-09-21T21:41:51.420 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.420 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.420 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.420 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.420 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.044354+0000", 2024-09-21T21:41:51.420 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9999999999999996e-07 2024-09-21T21:41:51.420 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.420 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.420 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.420 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.044357+0000", 2024-09-21T21:41:51.421 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3969999999999998e-06 2024-09-21T21:41:51.421 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.421 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.421 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.421 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.044366+0000", 2024-09-21T21:41:51.421 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5259999999999999e-06 2024-09-21T21:41:51.421 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.421 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.421 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.421 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.044373+0000", 2024-09-21T21:41:51.421 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3439999999999997e-06 2024-09-21T21:41:51.421 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.421 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.421 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.422 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.422 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.422 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:17114 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.422 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.058430+0000", 2024-09-21T21:41:51.422 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0082555619999999993, 2024-09-21T21:41:51.422 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.012644842999999999, 2024-09-21T21:41:51.422 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.422 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.422 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.422 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.422 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.422 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.422 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 17114 2024-09-21T21:41:51.422 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.423 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.423 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.423 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.423 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.058430+0000", 2024-09-21T21:41:51.423 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.423 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.423 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.423 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.423 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.058430+0000", 2024-09-21T21:41:51.423 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.423 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.423 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.423 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.424 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.058430+0000", 2024-09-21T21:41:51.424 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6700000000000003e-07 2024-09-21T21:41:51.424 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.424 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.424 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.424 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.058434+0000", 2024-09-21T21:41:51.424 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1410000000000001e-06 2024-09-21T21:41:51.424 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.424 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.424 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.424 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.058446+0000", 2024-09-21T21:41:51.424 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2228999999999999e-05 2024-09-21T21:41:51.425 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.425 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.425 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.425 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.058452+0000", 2024-09-21T21:41:51.425 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5400000000000001e-06 2024-09-21T21:41:51.425 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.425 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.425 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.425 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.425 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.425 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:16806 3.f 3.672f67af (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:51.425 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.044388+0000", 2024-09-21T21:41:51.425 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.022296849000000001, 2024-09-21T21:41:51.425 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.026717098000000002, 2024-09-21T21:41:51.425 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.426 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.426 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-21T21:41:51.426 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.426 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:51.426 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:51.426 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 16806 2024-09-21T21:41:51.426 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.426 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.426 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.426 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.426 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.044388+0000", 2024-09-21T21:41:51.426 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.426 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.427 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.427 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.427 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.044388+0000", 2024-09-21T21:41:51.427 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.427 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.427 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.427 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.427 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.044389+0000", 2024-09-21T21:41:51.427 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0100000000000005e-07 2024-09-21T21:41:51.427 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.427 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.427 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.427 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.044392+0000", 2024-09-21T21:41:51.427 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7650000000000002e-06 2024-09-21T21:41:51.428 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.428 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.428 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.428 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.044399+0000", 2024-09-21T21:41:51.428 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.153e-06 2024-09-21T21:41:51.428 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.428 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.428 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.428 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.044405+0000", 2024-09-21T21:41:51.428 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5710000000000003e-06 2024-09-21T21:41:51.428 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.428 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.428 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.428 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.428 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-21T21:41:51.429 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 102 2024-09-21T21:41:51.429 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:51.430 INFO:teuthology.orchestra.run.smithi081.stderr:nodeep-scrub is set 2024-09-21T21:41:51.431 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T21:41:51.431 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:51.428+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:51.530 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:51.529+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:51.544 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:51.544 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:41:51.544 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:41:51.544 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:41:51.544 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:51.553 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T21:41:51.631 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:51.629+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:51.703 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:51.703 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-21T21:41:51.703 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-21T21:41:51.703 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:41:51.704 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.704 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1486 3.16 3:6a41a830:::rbd_data.112145fae779.000000000000000a:a [read 360448~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:51.704 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634126+0000", 2024-09-21T21:41:51.704 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 50.066994499000003, 2024-09-21T21:41:51.704 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059399528, 2024-09-21T21:41:51.704 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.704 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.704 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:51.704 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.704 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:51.704 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:51.704 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1486 2024-09-21T21:41:51.704 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.704 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.705 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.705 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.705 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634126+0000", 2024-09-21T21:41:51.705 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.705 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.705 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.705 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.705 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634126+0000", 2024-09-21T21:41:51.705 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.705 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.705 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.705 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.705 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634126+0000", 2024-09-21T21:41:51.705 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4099999999999999e-07 2024-09-21T21:41:51.705 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.706 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.706 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.706 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634128+0000", 2024-09-21T21:41:51.706 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1069999999999999e-06 2024-09-21T21:41:51.706 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.706 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.706 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.706 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634135+0000", 2024-09-21T21:41:51.706 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.212e-06 2024-09-21T21:41:51.706 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.706 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.706 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.706 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634140+0000", 2024-09-21T21:41:51.707 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1429999999999997e-06 2024-09-21T21:41:51.707 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.707 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.707 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:51.707 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693363+0000", 2024-09-21T21:41:51.707 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059223614000000001 2024-09-21T21:41:51.707 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.707 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.707 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:51.707 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693431+0000", 2024-09-21T21:41:51.707 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7294999999999999e-05 2024-09-21T21:41:51.707 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.707 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.707 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:51.708 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693525+0000", 2024-09-21T21:41:51.708 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.4716000000000006e-05 2024-09-21T21:41:51.708 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.708 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.708 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.708 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.708 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.708 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1499 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 49152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:51.708 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634186+0000", 2024-09-21T21:41:51.708 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 50.066934621999998, 2024-09-21T21:41:51.708 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059339256999999999, 2024-09-21T21:41:51.708 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.708 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.708 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:51.709 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.709 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:51.709 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:51.709 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1499 2024-09-21T21:41:51.709 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.709 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.709 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.709 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.709 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634186+0000", 2024-09-21T21:41:51.709 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.709 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.709 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.709 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.709 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634186+0000", 2024-09-21T21:41:51.710 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.710 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.710 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.710 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.710 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634186+0000", 2024-09-21T21:41:51.710 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0999999999999999e-07 2024-09-21T21:41:51.710 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.710 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.710 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.710 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634188+0000", 2024-09-21T21:41:51.710 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.86e-06 2024-09-21T21:41:51.710 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.710 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.710 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.710 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634194+0000", 2024-09-21T21:41:51.711 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1109999999999996e-06 2024-09-21T21:41:51.711 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.711 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.711 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.711 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634199+0000", 2024-09-21T21:41:51.711 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7419999999999997e-06 2024-09-21T21:41:51.711 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.711 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.711 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:51.711 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693357+0000", 2024-09-21T21:41:51.711 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059157847 2024-09-21T21:41:51.711 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.711 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.711 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:51.711 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693430+0000", 2024-09-21T21:41:51.712 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3203999999999999e-05 2024-09-21T21:41:51.712 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.712 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.712 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:51.712 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693525+0000", 2024-09-21T21:41:51.712 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5082999999999997e-05 2024-09-21T21:41:51.712 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.712 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.712 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.712 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.712 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.712 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1501 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 106496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:51.712 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634205+0000", 2024-09-21T21:41:51.712 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 50.066915532000003, 2024-09-21T21:41:51.713 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059482394000000001, 2024-09-21T21:41:51.713 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.713 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.713 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:51.713 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.713 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:51.713 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:51.713 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1501 2024-09-21T21:41:51.713 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.713 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.713 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.713 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.713 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634205+0000", 2024-09-21T21:41:51.713 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.713 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.714 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.714 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.714 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634205+0000", 2024-09-21T21:41:51.714 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.714 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.714 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.714 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.714 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634205+0000", 2024-09-21T21:41:51.714 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.03e-07 2024-09-21T21:41:51.714 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.714 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.714 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.714 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634207+0000", 2024-09-21T21:41:51.714 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0389999999999999e-06 2024-09-21T21:41:51.715 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.715 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.715 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.715 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634213+0000", 2024-09-21T21:41:51.715 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2600000000000002e-06 2024-09-21T21:41:51.715 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.715 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.715 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.715 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634218+0000", 2024-09-21T21:41:51.715 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8570000000000003e-06 2024-09-21T21:41:51.715 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.715 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.715 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:51.715 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693552+0000", 2024-09-21T21:41:51.716 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059333636000000002 2024-09-21T21:41:51.716 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.716 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.716 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:51.716 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693613+0000", 2024-09-21T21:41:51.716 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0560000000000003e-05 2024-09-21T21:41:51.716 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.716 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.716 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:51.716 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693687+0000", 2024-09-21T21:41:51.716 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4639000000000001e-05 2024-09-21T21:41:51.716 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.716 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.716 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.716 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.717 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.717 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1515 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 434176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:51.717 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634318+0000", 2024-09-21T21:41:51.717 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 50.066801845000001, 2024-09-21T21:41:51.717 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059565812000000003, 2024-09-21T21:41:51.717 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.717 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.717 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:51.717 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.717 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:51.717 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:51.717 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1515 2024-09-21T21:41:51.717 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.718 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.718 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.718 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.718 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634318+0000", 2024-09-21T21:41:51.718 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.718 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.718 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.718 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.718 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634318+0000", 2024-09-21T21:41:51.718 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.718 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.718 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.718 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.719 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634319+0000", 2024-09-21T21:41:51.719 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9400000000000001e-07 2024-09-21T21:41:51.719 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.719 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.719 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.719 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634321+0000", 2024-09-21T21:41:51.719 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.154e-06 2024-09-21T21:41:51.719 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.719 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.719 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.719 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634327+0000", 2024-09-21T21:41:51.719 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9830000000000001e-06 2024-09-21T21:41:51.719 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.719 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.720 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.720 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634332+0000", 2024-09-21T21:41:51.720 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9080000000000003e-06 2024-09-21T21:41:51.720 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.720 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.720 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:51.720 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693717+0000", 2024-09-21T21:41:51.720 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059385447000000001 2024-09-21T21:41:51.720 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.720 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.720 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:51.720 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693795+0000", 2024-09-21T21:41:51.720 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7267000000000004e-05 2024-09-21T21:41:51.721 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.721 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.721 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:51.721 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693884+0000", 2024-09-21T21:41:51.721 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9659000000000003e-05 2024-09-21T21:41:51.721 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.721 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.721 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.721 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.721 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.721 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1517 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 491520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:51.721 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634338+0000", 2024-09-21T21:41:51.721 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 50.066782072000002, 2024-09-21T21:41:51.721 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060100745999999997, 2024-09-21T21:41:51.722 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.722 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.722 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:51.722 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.722 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:51.722 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:51.722 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1517 2024-09-21T21:41:51.722 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.722 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.722 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.722 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.722 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634338+0000", 2024-09-21T21:41:51.722 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.723 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.723 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.723 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.723 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634338+0000", 2024-09-21T21:41:51.723 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.723 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.723 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.723 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.723 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634339+0000", 2024-09-21T21:41:51.723 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2899999999999999e-07 2024-09-21T21:41:51.723 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.723 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.723 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.723 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634341+0000", 2024-09-21T21:41:51.724 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.8959999999999999e-06 2024-09-21T21:41:51.724 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.724 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.724 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.724 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634347+0000", 2024-09-21T21:41:51.724 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.302e-06 2024-09-21T21:41:51.724 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.724 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.724 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.724 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634351+0000", 2024-09-21T21:41:51.724 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6070000000000001e-06 2024-09-21T21:41:51.724 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.724 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.724 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:51.725 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694279+0000", 2024-09-21T21:41:51.725 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059927046999999997 2024-09-21T21:41:51.725 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.725 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.725 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:51.725 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694355+0000", 2024-09-21T21:41:51.725 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6483999999999995e-05 2024-09-21T21:41:51.725 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.725 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.725 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:51.725 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694439+0000", 2024-09-21T21:41:51.725 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3980999999999994e-05 2024-09-21T21:41:51.725 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.725 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.726 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.726 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.726 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.726 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1518 3.16 3:6a41a830:::rbd_data.112145fae779.000000000000000a:a [read 294912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:51.726 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634357+0000", 2024-09-21T21:41:51.726 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 50.066763166000001, 2024-09-21T21:41:51.726 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059712318, 2024-09-21T21:41:51.726 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.726 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.726 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:51.726 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.726 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:51.726 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:51.726 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1518 2024-09-21T21:41:51.726 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.727 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.727 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.727 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.727 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634357+0000", 2024-09-21T21:41:51.727 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.727 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.727 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.727 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.727 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634357+0000", 2024-09-21T21:41:51.727 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.727 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.727 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.727 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.727 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634358+0000", 2024-09-21T21:41:51.727 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2100000000000002e-07 2024-09-21T21:41:51.728 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.728 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.728 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.728 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634360+0000", 2024-09-21T21:41:51.728 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0940000000000002e-06 2024-09-21T21:41:51.728 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.728 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.728 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.728 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634366+0000", 2024-09-21T21:41:51.728 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3389999999999998e-06 2024-09-21T21:41:51.728 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.728 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.728 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.728 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634371+0000", 2024-09-21T21:41:51.729 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6670000000000004e-06 2024-09-21T21:41:51.729 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.729 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.729 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:51.729 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693910+0000", 2024-09-21T21:41:51.729 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059538973000000002 2024-09-21T21:41:51.729 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.729 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.729 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:51.729 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693982+0000", 2024-09-21T21:41:51.729 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2806999999999996e-05 2024-09-21T21:41:51.729 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.729 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.729 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:51.729 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694069+0000", 2024-09-21T21:41:51.730 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7016999999999997e-05 2024-09-21T21:41:51.730 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.730 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.730 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.730 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.730 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.730 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1519 3.16 3:6a41a830:::rbd_data.112145fae779.000000000000000a:a [read 352256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:51.730 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634377+0000", 2024-09-21T21:41:51.730 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 50.066743463999998, 2024-09-21T21:41:51.730 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059872213000000001, 2024-09-21T21:41:51.730 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.730 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.730 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:51.731 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.731 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:51.731 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:51.731 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1519 2024-09-21T21:41:51.731 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.731 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.731 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.731 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.731 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634377+0000", 2024-09-21T21:41:51.731 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.731 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.731 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.731 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.731 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634377+0000", 2024-09-21T21:41:51.731 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.732 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.732 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.732 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.732 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634377+0000", 2024-09-21T21:41:51.732 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2600000000000002e-07 2024-09-21T21:41:51.732 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.732 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.732 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.732 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634379+0000", 2024-09-21T21:41:51.732 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0729999999999999e-06 2024-09-21T21:41:51.732 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.732 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.732 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.732 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634386+0000", 2024-09-21T21:41:51.732 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4690000000000002e-06 2024-09-21T21:41:51.733 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.733 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.733 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.733 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634390+0000", 2024-09-21T21:41:51.733 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4580000000000003e-06 2024-09-21T21:41:51.733 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.733 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.733 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:51.733 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694094+0000", 2024-09-21T21:41:51.733 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059703830999999999 2024-09-21T21:41:51.733 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.733 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.733 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:51.733 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694168+0000", 2024-09-21T21:41:51.733 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4222999999999997e-05 2024-09-21T21:41:51.734 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.734 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.734 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:51.734 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694249+0000", 2024-09-21T21:41:51.734 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0632999999999999e-05 2024-09-21T21:41:51.734 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.734 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.734 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.734 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.734 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.734 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1529 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 53248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:51.734 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634397+0000", 2024-09-21T21:41:51.734 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 50.066723312999997, 2024-09-21T21:41:51.735 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060189919000000001, 2024-09-21T21:41:51.735 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.735 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.735 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:51.735 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.735 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:51.735 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:51.735 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1529 2024-09-21T21:41:51.735 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.735 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.735 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.735 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.735 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634397+0000", 2024-09-21T21:41:51.735 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.735 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.736 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.736 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.736 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634397+0000", 2024-09-21T21:41:51.736 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.736 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.736 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.736 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.736 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634397+0000", 2024-09-21T21:41:51.736 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2e-07 2024-09-21T21:41:51.736 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.736 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.736 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.736 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634399+0000", 2024-09-21T21:41:51.736 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0609999999999998e-06 2024-09-21T21:41:51.737 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.737 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.737 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.737 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634406+0000", 2024-09-21T21:41:51.737 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7259999999999996e-06 2024-09-21T21:41:51.737 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.737 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.737 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.737 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634411+0000", 2024-09-21T21:41:51.737 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9339999999999997e-06 2024-09-21T21:41:51.737 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.737 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.737 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:51.737 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694480+0000", 2024-09-21T21:41:51.737 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060069105999999997 2024-09-21T21:41:51.737 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.738 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.738 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:51.738 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694527+0000", 2024-09-21T21:41:51.738 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7120000000000003e-05 2024-09-21T21:41:51.738 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.738 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.738 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:51.738 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694587+0000", 2024-09-21T21:41:51.738 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9552000000000002e-05 2024-09-21T21:41:51.738 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.738 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.738 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.738 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.738 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.738 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1531 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 110592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:51.739 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634417+0000", 2024-09-21T21:41:51.739 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 50.066703379000003, 2024-09-21T21:41:51.739 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060347287999999999, 2024-09-21T21:41:51.739 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.739 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.739 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:51.739 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.739 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:51.739 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:51.739 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1531 2024-09-21T21:41:51.739 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.739 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.739 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.740 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.740 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634417+0000", 2024-09-21T21:41:51.740 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.740 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.740 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.740 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.740 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634417+0000", 2024-09-21T21:41:51.740 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.740 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.740 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.740 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.740 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634417+0000", 2024-09-21T21:41:51.740 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1699999999999999e-07 2024-09-21T21:41:51.740 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.740 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.741 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.741 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634420+0000", 2024-09-21T21:41:51.741 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.243e-06 2024-09-21T21:41:51.741 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.741 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.741 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.741 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634426+0000", 2024-09-21T21:41:51.741 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2550000000000003e-06 2024-09-21T21:41:51.741 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.741 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.741 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.741 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634431+0000", 2024-09-21T21:41:51.741 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0150000000000003e-06 2024-09-21T21:41:51.741 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.741 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.742 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:51.742 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694618+0000", 2024-09-21T21:41:51.742 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060186836000000001 2024-09-21T21:41:51.742 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.742 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.742 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:51.742 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694685+0000", 2024-09-21T21:41:51.742 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6970000000000004e-05 2024-09-21T21:41:51.742 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.742 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.742 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:51.742 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694764+0000", 2024-09-21T21:41:51.742 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9552e-05 2024-09-21T21:41:51.742 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.743 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.743 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.743 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.743 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.743 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1532 3.e 3:724b31a3:::rbd_data.1121e540c096.0000000000000015:b [read 151552~36864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:51.743 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634437+0000", 2024-09-21T21:41:51.743 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 50.066683583, 2024-09-21T21:41:51.743 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060474314000000001, 2024-09-21T21:41:51.743 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.743 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.743 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:51.743 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.743 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:51.743 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:51.744 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1532 2024-09-21T21:41:51.744 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.744 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.744 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.744 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.744 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634437+0000", 2024-09-21T21:41:51.744 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.744 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.744 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.744 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.744 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634437+0000", 2024-09-21T21:41:51.744 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.744 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.744 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.745 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.745 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634437+0000", 2024-09-21T21:41:51.745 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9400000000000001e-07 2024-09-21T21:41:51.745 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.745 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.745 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.745 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634439+0000", 2024-09-21T21:41:51.745 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.114e-06 2024-09-21T21:41:51.745 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.745 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.745 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.745 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634446+0000", 2024-09-21T21:41:51.745 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1389999999999998e-06 2024-09-21T21:41:51.746 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.746 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.746 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.746 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634451+0000", 2024-09-21T21:41:51.746 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0010000000000001e-06 2024-09-21T21:41:51.746 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.746 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.746 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:51.746 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694794+0000", 2024-09-21T21:41:51.746 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060342509000000003 2024-09-21T21:41:51.746 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.746 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.747 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:51.747 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694835+0000", 2024-09-21T21:41:51.747 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0728e-05 2024-09-21T21:41:51.747 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.747 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.747 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:51.747 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694911+0000", 2024-09-21T21:41:51.747 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6428999999999996e-05 2024-09-21T21:41:51.747 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.747 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.747 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.747 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.747 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.748 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1534 3.e 3:724b31a3:::rbd_data.1121e540c096.0000000000000015:b [read 487424~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:51.748 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634477+0000", 2024-09-21T21:41:51.748 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 50.06664318, 2024-09-21T21:41:51.748 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060624009, 2024-09-21T21:41:51.748 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.748 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.748 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:51.748 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.748 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:51.748 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:51.748 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1534 2024-09-21T21:41:51.748 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.748 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.748 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.748 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.749 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634477+0000", 2024-09-21T21:41:51.749 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.749 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.749 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.749 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.749 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634477+0000", 2024-09-21T21:41:51.749 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.749 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.749 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.749 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.749 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634478+0000", 2024-09-21T21:41:51.749 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1399999999999997e-07 2024-09-21T21:41:51.749 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.749 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.750 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.750 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634480+0000", 2024-09-21T21:41:51.750 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1009999999999999e-06 2024-09-21T21:41:51.750 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.750 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.750 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.750 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634486+0000", 2024-09-21T21:41:51.750 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3289999999999999e-06 2024-09-21T21:41:51.750 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.750 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.750 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.750 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634491+0000", 2024-09-21T21:41:51.750 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6260000000000003e-06 2024-09-21T21:41:51.750 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.751 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.751 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:51.751 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694943+0000", 2024-09-21T21:41:51.751 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060451960999999999 2024-09-21T21:41:51.751 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.751 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.751 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:51.751 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695014+0000", 2024-09-21T21:41:51.751 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1587000000000004e-05 2024-09-21T21:41:51.751 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.751 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.751 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:51.751 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695101+0000", 2024-09-21T21:41:51.751 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6990999999999994e-05 2024-09-21T21:41:51.752 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.752 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.752 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.752 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.752 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.752 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1545 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 438272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:51.752 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634515+0000", 2024-09-21T21:41:51.752 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 50.066604980000001, 2024-09-21T21:41:51.752 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060707944, 2024-09-21T21:41:51.752 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.752 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.752 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:51.752 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.752 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:51.753 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:51.753 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1545 2024-09-21T21:41:51.753 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.753 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.753 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.753 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.753 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634515+0000", 2024-09-21T21:41:51.753 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.753 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.753 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.753 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.753 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634515+0000", 2024-09-21T21:41:51.753 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.753 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.753 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.754 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.754 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634516+0000", 2024-09-21T21:41:51.754 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0999999999999999e-07 2024-09-21T21:41:51.754 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.754 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.754 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.754 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634518+0000", 2024-09-21T21:41:51.754 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.846e-06 2024-09-21T21:41:51.754 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.754 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.754 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.754 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634524+0000", 2024-09-21T21:41:51.754 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2550000000000003e-06 2024-09-21T21:41:51.754 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.755 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.755 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.755 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634529+0000", 2024-09-21T21:41:51.755 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6940000000000003e-06 2024-09-21T21:41:51.755 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.755 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.755 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:51.755 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695127+0000", 2024-09-21T21:41:51.755 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060598921 2024-09-21T21:41:51.755 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.755 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.755 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:51.755 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695171+0000", 2024-09-21T21:41:51.755 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3588999999999999e-05 2024-09-21T21:41:51.755 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.756 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.756 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:51.756 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695223+0000", 2024-09-21T21:41:51.756 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2228999999999997e-05 2024-09-21T21:41:51.756 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.756 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.756 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.756 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.756 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.756 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1547 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 495616~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:51.756 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634534+0000", 2024-09-21T21:41:51.756 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 50.066586188999999, 2024-09-21T21:41:51.756 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060881131999999998, 2024-09-21T21:41:51.756 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.757 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.757 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:51.757 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.757 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:51.757 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:51.757 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1547 2024-09-21T21:41:51.757 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.757 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.757 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.757 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.757 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634534+0000", 2024-09-21T21:41:51.757 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.757 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.757 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.758 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.758 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634534+0000", 2024-09-21T21:41:51.758 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.758 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.758 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.758 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.758 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634535+0000", 2024-09-21T21:41:51.758 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.77e-07 2024-09-21T21:41:51.758 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.758 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.758 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.758 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634542+0000", 2024-09-21T21:41:51.758 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9000000000000006e-06 2024-09-21T21:41:51.758 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.758 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.759 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.759 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634549+0000", 2024-09-21T21:41:51.759 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5910000000000001e-06 2024-09-21T21:41:51.759 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.759 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.759 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.759 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634554+0000", 2024-09-21T21:41:51.759 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7940000000000002e-06 2024-09-21T21:41:51.759 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.759 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.759 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:51.759 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695256+0000", 2024-09-21T21:41:51.759 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.06070246 2024-09-21T21:41:51.759 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.760 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.760 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:51.760 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695332+0000", 2024-09-21T21:41:51.760 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6248000000000006e-05 2024-09-21T21:41:51.760 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.760 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.760 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:51.760 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695415+0000", 2024-09-21T21:41:51.760 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2762000000000005e-05 2024-09-21T21:41:51.760 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.760 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.760 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.760 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.760 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.760 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1548 3.16 3:6a41a830:::rbd_data.112145fae779.000000000000000a:a [read 299008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:51.761 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634561+0000", 2024-09-21T21:41:51.761 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 50.066559611999999, 2024-09-21T21:41:51.761 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060786743999999997, 2024-09-21T21:41:51.761 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.761 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.761 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:51.761 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.761 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:51.761 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:51.761 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1548 2024-09-21T21:41:51.761 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.761 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.761 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.761 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.761 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634561+0000", 2024-09-21T21:41:51.762 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.762 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.762 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.762 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.762 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634561+0000", 2024-09-21T21:41:51.762 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.762 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.762 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.762 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.762 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634561+0000", 2024-09-21T21:41:51.762 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5199999999999997e-07 2024-09-21T21:41:51.762 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.762 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.762 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.763 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634564+0000", 2024-09-21T21:41:51.763 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4040000000000002e-06 2024-09-21T21:41:51.763 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.763 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.763 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.763 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634571+0000", 2024-09-21T21:41:51.763 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4379999999999998e-06 2024-09-21T21:41:51.763 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.763 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.763 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.763 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634576+0000", 2024-09-21T21:41:51.763 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1399999999999999e-06 2024-09-21T21:41:51.763 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.763 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.764 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:51.764 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695247+0000", 2024-09-21T21:41:51.764 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060670397000000001 2024-09-21T21:41:51.764 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.764 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.764 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:51.764 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695299+0000", 2024-09-21T21:41:51.764 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2564e-05 2024-09-21T21:41:51.764 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.764 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.764 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:51.764 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695347+0000", 2024-09-21T21:41:51.764 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8248999999999998e-05 2024-09-21T21:41:51.765 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.765 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.765 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.765 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.765 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.765 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1549 3.16 3:6a41a830:::rbd_data.112145fae779.000000000000000a:a [read 356352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:51.765 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634583+0000", 2024-09-21T21:41:51.765 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 50.066537699000001, 2024-09-21T21:41:51.765 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060889459, 2024-09-21T21:41:51.765 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.765 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.765 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:51.765 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.765 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:51.765 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:51.766 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1549 2024-09-21T21:41:51.766 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.766 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.766 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.766 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.766 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634583+0000", 2024-09-21T21:41:51.766 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.766 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.766 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.766 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.766 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634583+0000", 2024-09-21T21:41:51.766 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.766 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.766 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.767 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.767 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634583+0000", 2024-09-21T21:41:51.767 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1899999999999998e-07 2024-09-21T21:41:51.767 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.767 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.767 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.767 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634585+0000", 2024-09-21T21:41:51.767 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2129999999999998e-06 2024-09-21T21:41:51.767 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.767 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.767 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.767 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634592+0000", 2024-09-21T21:41:51.767 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2840000000000003e-06 2024-09-21T21:41:51.767 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.768 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.768 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.768 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634596+0000", 2024-09-21T21:41:51.768 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9309999999999999e-06 2024-09-21T21:41:51.768 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.768 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.768 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:51.768 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695371+0000", 2024-09-21T21:41:51.768 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060774425999999999 2024-09-21T21:41:51.768 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.768 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.768 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:51.768 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695419+0000", 2024-09-21T21:41:51.768 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8607999999999999e-05 2024-09-21T21:41:51.769 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.769 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.769 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:51.769 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695472+0000", 2024-09-21T21:41:51.769 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2577999999999997e-05 2024-09-21T21:41:51.769 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.769 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.769 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.769 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.769 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.769 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4385.0:15202 3.16 3:6888948b:::rbd_header.1121493f6459:head [watch reconnect cookie 140325440202496 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-21T21:41:51.769 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:41.736601+0000", 2024-09-21T21:41:51.769 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 9.9645190659999994, 2024-09-21T21:41:51.769 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5040056769999999, 2024-09-21T21:41:51.770 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.770 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.770 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:51.770 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.770 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4385", 2024-09-21T21:41:51.770 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:41:51.770 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 15202 2024-09-21T21:41:51.770 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.770 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.770 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.770 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.770 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736601+0000", 2024-09-21T21:41:51.770 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.770 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.770 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.771 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.771 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736601+0000", 2024-09-21T21:41:51.771 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.771 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.771 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.771 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.771 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736602+0000", 2024-09-21T21:41:51.771 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0699999999999999e-06 2024-09-21T21:41:51.771 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.771 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.771 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.771 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736607+0000", 2024-09-21T21:41:51.771 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2520000000000001e-06 2024-09-21T21:41:51.771 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.772 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.772 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.772 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736617+0000", 2024-09-21T21:41:51.772 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0074e-05 2024-09-21T21:41:51.772 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.772 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.772 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.772 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736622+0000", 2024-09-21T21:41:51.772 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.101e-06 2024-09-21T21:41:51.772 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.772 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.772 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:51.772 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736664+0000", 2024-09-21T21:41:51.772 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1869999999999997e-05 2024-09-21T21:41:51.773 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.773 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.773 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op must wait for map", 2024-09-21T21:41:51.773 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736667+0000", 2024-09-21T21:41:51.773 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4750000000000002e-06 2024-09-21T21:41:51.773 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.773 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.773 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:51.773 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742886+0000", 2024-09-21T21:41:51.773 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0062187579999999996 2024-09-21T21:41:51.773 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.773 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.773 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:41:51.773 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742892+0000", 2024-09-21T21:41:51.773 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0440000000000001e-06 2024-09-21T21:41:51.774 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.774 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.774 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:51.774 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210307+0000", 2024-09-21T21:41:51.774 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.467414781 2024-09-21T21:41:51.774 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.774 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.774 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:41:51.774 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210322+0000", 2024-09-21T21:41:51.774 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5311e-05 2024-09-21T21:41:51.774 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.774 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.774 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:51.774 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.237624+0000", 2024-09-21T21:41:51.775 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0273018920000001 2024-09-21T21:41:51.775 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.775 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.775 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:51.775 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.237870+0000", 2024-09-21T21:41:51.775 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00024622899999999999 2024-09-21T21:41:51.775 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.775 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.775 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:51.775 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.240607+0000", 2024-09-21T21:41:51.776 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0027368200000000001 2024-09-21T21:41:51.776 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.776 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.776 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.776 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.776 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.776 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5415.0:379 3.16 3:6888948b:::rbd_header.1121493f6459:head [watch reconnect cookie 139702602617920 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-21T21:41:51.776 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:41.739544+0000", 2024-09-21T21:41:51.776 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 9.9615766370000003, 2024-09-21T21:41:51.776 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.501180645, 2024-09-21T21:41:51.776 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.776 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.776 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:51.776 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.777 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5415", 2024-09-21T21:41:51.777 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1212816607", 2024-09-21T21:41:51.777 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 379 2024-09-21T21:41:51.777 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.777 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.777 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.777 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.777 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739544+0000", 2024-09-21T21:41:51.777 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.777 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.777 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.777 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.777 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739544+0000", 2024-09-21T21:41:51.777 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.778 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.778 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.778 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.778 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739545+0000", 2024-09-21T21:41:51.778 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.065e-06 2024-09-21T21:41:51.778 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.778 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.778 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.778 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739550+0000", 2024-09-21T21:41:51.778 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9599999999999999e-06 2024-09-21T21:41:51.778 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.778 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.778 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.779 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739565+0000", 2024-09-21T21:41:51.779 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5709000000000002e-05 2024-09-21T21:41:51.779 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.779 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.779 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.779 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739573+0000", 2024-09-21T21:41:51.779 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7849999999999992e-06 2024-09-21T21:41:51.779 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.779 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.779 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:51.779 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739634+0000", 2024-09-21T21:41:51.779 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0732000000000001e-05 2024-09-21T21:41:51.779 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.780 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.780 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op must wait for map", 2024-09-21T21:41:51.780 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739640+0000", 2024-09-21T21:41:51.780 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0000000000000002e-06 2024-09-21T21:41:51.780 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.780 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.780 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:51.780 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742958+0000", 2024-09-21T21:41:51.780 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0033182390000000002 2024-09-21T21:41:51.780 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.780 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.780 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:41:51.780 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742966+0000", 2024-09-21T21:41:51.781 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1610000000000003e-06 2024-09-21T21:41:51.781 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.781 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.781 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:51.781 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210405+0000", 2024-09-21T21:41:51.781 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.467438725 2024-09-21T21:41:51.781 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.781 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.781 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:41:51.781 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210415+0000", 2024-09-21T21:41:51.781 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0234000000000001e-05 2024-09-21T21:41:51.781 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.781 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.781 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:51.781 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.238743+0000", 2024-09-21T21:41:51.782 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0283273070000001 2024-09-21T21:41:51.782 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.782 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.782 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:51.782 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.238842+0000", 2024-09-21T21:41:51.782 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.9896999999999996e-05 2024-09-21T21:41:51.782 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.782 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.782 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:51.782 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.240724+0000", 2024-09-21T21:41:51.782 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001881831 2024-09-21T21:41:51.782 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.782 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.782 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.783 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.783 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.783 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4385.0:15203 3.16 3:6888948b:::rbd_header.1121493f6459:head [call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-21T21:41:51.783 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:41.739737+0000", 2024-09-21T21:41:51.783 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 9.9613836259999999, 2024-09-21T21:41:51.783 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4989621799999999, 2024-09-21T21:41:51.783 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.783 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.783 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:51.783 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.783 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4385", 2024-09-21T21:41:51.783 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:41:51.783 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 15203 2024-09-21T21:41:51.783 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.783 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.784 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.784 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.784 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739737+0000", 2024-09-21T21:41:51.784 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.784 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.784 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.784 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.784 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739737+0000", 2024-09-21T21:41:51.784 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.784 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.784 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.784 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.784 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739738+0000", 2024-09-21T21:41:51.784 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.7499999999999998e-07 2024-09-21T21:41:51.784 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.785 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.785 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.785 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739743+0000", 2024-09-21T21:41:51.785 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5049999999999998e-06 2024-09-21T21:41:51.785 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.785 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.785 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.785 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739757+0000", 2024-09-21T21:41:51.785 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3947e-05 2024-09-21T21:41:51.785 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.785 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.785 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.785 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739765+0000", 2024-09-21T21:41:51.785 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9310000000000008e-06 2024-09-21T21:41:51.785 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.786 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.786 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:51.786 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739816+0000", 2024-09-21T21:41:51.786 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0559999999999997e-05 2024-09-21T21:41:51.786 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.786 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.786 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting_for_map not empty", 2024-09-21T21:41:51.786 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739821+0000", 2024-09-21T21:41:51.786 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3539999999999999e-06 2024-09-21T21:41:51.786 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.786 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.786 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:51.786 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742928+0000", 2024-09-21T21:41:51.786 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0031074399999999999 2024-09-21T21:41:51.787 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.787 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.787 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:41:51.787 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742937+0000", 2024-09-21T21:41:51.787 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1289999999999996e-06 2024-09-21T21:41:51.787 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.787 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.787 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:51.787 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210350+0000", 2024-09-21T21:41:51.787 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.46741311299999999 2024-09-21T21:41:51.787 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.787 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.787 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:41:51.787 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210358+0000", 2024-09-21T21:41:51.788 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1869999999999997e-06 2024-09-21T21:41:51.788 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.788 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.788 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:51.788 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.238398+0000", 2024-09-21T21:41:51.788 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0280397830000001 2024-09-21T21:41:51.788 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.788 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.788 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:51.788 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.238489+0000", 2024-09-21T21:41:51.788 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.0958000000000006e-05 2024-09-21T21:41:51.788 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.788 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.789 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:51.789 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.238699+0000", 2024-09-21T21:41:51.789 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00021029800000000001 2024-09-21T21:41:51.789 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.789 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.789 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.789 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.789 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.789 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4385.0:15204 3.16 3:6888948b:::rbd_header.1121493f6459:head [watch reconnect cookie 140325440202496 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-21T21:41:51.789 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:42.210726+0000", 2024-09-21T21:41:51.789 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 9.4903947479999999, 2024-09-21T21:41:51.789 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.030304557, 2024-09-21T21:41:51.789 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.789 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.790 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:51.790 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.790 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4385", 2024-09-21T21:41:51.790 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:41:51.790 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 15204 2024-09-21T21:41:51.790 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.790 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.790 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.790 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.790 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210726+0000", 2024-09-21T21:41:51.790 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.790 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.790 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.790 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.790 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210726+0000", 2024-09-21T21:41:51.791 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.791 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.791 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.791 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.791 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210727+0000", 2024-09-21T21:41:51.791 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.204e-06 2024-09-21T21:41:51.791 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.791 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.791 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.791 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210732+0000", 2024-09-21T21:41:51.791 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5249999999999996e-06 2024-09-21T21:41:51.791 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.791 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.791 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.791 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210744+0000", 2024-09-21T21:41:51.792 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1405e-05 2024-09-21T21:41:51.792 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.792 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.792 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.792 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210754+0000", 2024-09-21T21:41:51.792 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0122999999999999e-05 2024-09-21T21:41:51.792 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.792 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.792 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:51.792 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210798+0000", 2024-09-21T21:41:51.792 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4005000000000003e-05 2024-09-21T21:41:51.792 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.792 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.792 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:41:51.792 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210803+0000", 2024-09-21T21:41:51.792 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5130000000000004e-06 2024-09-21T21:41:51.793 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.793 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.793 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:51.793 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.239260+0000", 2024-09-21T21:41:51.793 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0284563040000001 2024-09-21T21:41:51.793 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.793 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.793 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:51.793 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.239318+0000", 2024-09-21T21:41:51.793 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8486000000000002e-05 2024-09-21T21:41:51.793 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.793 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.793 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:51.793 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.241030+0000", 2024-09-21T21:41:51.794 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0017119920000000001 2024-09-21T21:41:51.794 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.794 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.794 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.794 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.794 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.794 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5415.0:380 3.16 3:6888948b:::rbd_header.1121493f6459:head [watch reconnect cookie 139702602617920 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-21T21:41:51.794 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:42.210777+0000", 2024-09-21T21:41:51.794 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 9.4903429690000003, 2024-09-21T21:41:51.794 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.030410566, 2024-09-21T21:41:51.794 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:51.794 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:51.794 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:51.794 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:51.795 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5415", 2024-09-21T21:41:51.795 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1212816607", 2024-09-21T21:41:51.795 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 380 2024-09-21T21:41:51.795 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.795 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:51.795 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.795 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:51.795 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210777+0000", 2024-09-21T21:41:51.795 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.795 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.795 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.795 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:51.795 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210777+0000", 2024-09-21T21:41:51.795 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:51.796 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.796 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.796 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:51.796 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210778+0000", 2024-09-21T21:41:51.796 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8999999999999996e-07 2024-09-21T21:41:51.796 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.796 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.796 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:51.796 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210780+0000", 2024-09-21T21:41:51.796 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.503e-06 2024-09-21T21:41:51.796 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.796 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.796 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:51.796 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210788+0000", 2024-09-21T21:41:51.797 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0999999999999998e-06 2024-09-21T21:41:51.797 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.797 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.797 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:51.797 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210799+0000", 2024-09-21T21:41:51.797 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1703e-05 2024-09-21T21:41:51.797 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.797 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.797 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:51.797 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210833+0000", 2024-09-21T21:41:51.797 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4008999999999999e-05 2024-09-21T21:41:51.797 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.797 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.798 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:41:51.798 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210838+0000", 2024-09-21T21:41:51.798 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7480000000000001e-06 2024-09-21T21:41:51.798 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.798 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.798 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:51.798 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.239612+0000", 2024-09-21T21:41:51.798 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.028773554 2024-09-21T21:41:51.798 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.798 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.798 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:51.798 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.239677+0000", 2024-09-21T21:41:51.798 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5053000000000002e-05 2024-09-21T21:41:51.799 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:51.799 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:51.799 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:51.799 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.241188+0000", 2024-09-21T21:41:51.799 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001511306 2024-09-21T21:41:51.799 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.799 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.799 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.799 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:51.799 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:51.799 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:51.801 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:51.799+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:51.801 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T21:41:51.901 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:51.900+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:51.906 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:51.906 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:41:51.906 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 0 2024-09-21T21:41:51.906 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:51.915 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T21:41:52.002 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:52.001+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:52.023 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:52.023 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:41:52.023 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:41:52.024 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:41:52.024 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:52.033 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T21:41:52.103 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:52.101+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:52.138 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:52.138 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 20, 2024-09-21T21:41:52.138 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 600, 2024-09-21T21:41:52.138 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:41:52.139 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.139 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11319 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 5492736~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:52.139 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.712385+0000", 2024-09-21T21:41:52.139 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 1.423762497, 2024-09-21T21:41:52.139 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078627889000000006, 2024-09-21T21:41:52.139 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:52.139 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:52.140 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:52.140 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:52.140 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:41:52.140 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:52.140 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11319 2024-09-21T21:41:52.140 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.140 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:52.140 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.140 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:52.140 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712385+0000", 2024-09-21T21:41:52.141 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:52.141 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.141 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.141 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:52.141 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712385+0000", 2024-09-21T21:41:52.141 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:52.141 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.141 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.141 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:52.141 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712385+0000", 2024-09-21T21:41:52.141 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2800000000000002e-07 2024-09-21T21:41:52.141 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.141 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.141 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:52.141 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712387+0000", 2024-09-21T21:41:52.142 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1569999999999998e-06 2024-09-21T21:41:52.142 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.142 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.142 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:52.142 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712393+0000", 2024-09-21T21:41:52.142 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9370000000000001e-06 2024-09-21T21:41:52.142 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.142 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.142 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:52.142 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712398+0000", 2024-09-21T21:41:52.142 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5020000000000003e-06 2024-09-21T21:41:52.142 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.142 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.143 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:52.143 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.790746+0000", 2024-09-21T21:41:52.143 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078348589999999996 2024-09-21T21:41:52.145 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.145 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.146 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:52.146 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.790844+0000", 2024-09-21T21:41:52.146 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.7526000000000001e-05 2024-09-21T21:41:52.146 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.146 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.146 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:52.147 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.791013+0000", 2024-09-21T21:41:52.147 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00016874899999999999 2024-09-21T21:41:52.147 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:52.147 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:52.147 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:52.147 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.147 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.148 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11320 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 6180864~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:52.148 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.712405+0000", 2024-09-21T21:41:52.148 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 1.4237425379999999, 2024-09-21T21:41:52.148 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078882870999999993, 2024-09-21T21:41:52.148 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:52.148 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:52.148 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:52.149 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:52.149 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:41:52.149 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:52.149 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11320 2024-09-21T21:41:52.149 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.149 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:52.149 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.149 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:52.149 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712405+0000", 2024-09-21T21:41:52.150 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:52.150 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.150 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.150 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:52.150 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712405+0000", 2024-09-21T21:41:52.150 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:52.150 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.151 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.151 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:52.151 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.713942+0000", 2024-09-21T21:41:52.151 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.001536882 2024-09-21T21:41:52.151 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.151 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.151 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:52.152 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.713946+0000", 2024-09-21T21:41:52.152 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4549999999999997e-06 2024-09-21T21:41:52.152 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.152 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.152 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:52.152 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.713958+0000", 2024-09-21T21:41:52.152 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1775e-05 2024-09-21T21:41:52.153 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.153 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.153 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:52.153 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.713965+0000", 2024-09-21T21:41:52.153 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5100000000000001e-06 2024-09-21T21:41:52.153 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.154 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.154 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:52.154 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.791059+0000", 2024-09-21T21:41:52.154 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.077094036000000005 2024-09-21T21:41:52.154 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.154 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.154 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:52.154 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.791175+0000", 2024-09-21T21:41:52.154 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000115895 2024-09-21T21:41:52.154 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.155 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.155 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:52.155 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.791288+0000", 2024-09-21T21:41:52.155 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000112318 2024-09-21T21:41:52.155 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:52.155 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:52.155 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:52.155 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.155 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.155 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11325 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 5414912~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:52.155 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.714095+0000", 2024-09-21T21:41:52.155 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 1.4220525429999999, 2024-09-21T21:41:52.155 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078631894999999993, 2024-09-21T21:41:52.155 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:52.156 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:52.156 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:52.156 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:52.156 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:41:52.156 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:52.156 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11325 2024-09-21T21:41:52.156 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.156 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:52.156 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.156 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:52.156 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714095+0000", 2024-09-21T21:41:52.156 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:52.157 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.157 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.157 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:52.157 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714095+0000", 2024-09-21T21:41:52.157 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:52.157 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.157 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.157 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:52.157 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714095+0000", 2024-09-21T21:41:52.157 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7500000000000001e-07 2024-09-21T21:41:52.158 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.158 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.158 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:52.158 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714097+0000", 2024-09-21T21:41:52.158 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.12e-06 2024-09-21T21:41:52.158 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.158 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.158 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:52.158 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714104+0000", 2024-09-21T21:41:52.158 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3520000000000003e-06 2024-09-21T21:41:52.159 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.159 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.159 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:52.159 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714108+0000", 2024-09-21T21:41:52.159 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8810000000000004e-06 2024-09-21T21:41:52.159 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.159 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.159 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:52.159 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792509+0000", 2024-09-21T21:41:52.159 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078400739999999997 2024-09-21T21:41:52.159 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.159 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.159 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:52.159 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792613+0000", 2024-09-21T21:41:52.160 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000103939 2024-09-21T21:41:52.160 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.160 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.160 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:52.160 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792727+0000", 2024-09-21T21:41:52.160 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011348799999999999 2024-09-21T21:41:52.160 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:52.160 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:52.160 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:52.160 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.160 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.160 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11326 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 6103040~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:52.160 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.714115+0000", 2024-09-21T21:41:52.160 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 1.4220325149999999, 2024-09-21T21:41:52.161 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078963056000000004, 2024-09-21T21:41:52.161 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:52.161 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:52.161 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:52.161 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:52.161 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:41:52.161 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:52.161 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11326 2024-09-21T21:41:52.161 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.161 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:52.161 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.162 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:52.162 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714115+0000", 2024-09-21T21:41:52.162 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:52.162 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.162 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.162 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:52.162 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714115+0000", 2024-09-21T21:41:52.162 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:52.162 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.162 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.162 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:52.162 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.715940+0000", 2024-09-21T21:41:52.163 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0018254969999999999 2024-09-21T21:41:52.163 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.163 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.163 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:52.163 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.715945+0000", 2024-09-21T21:41:52.163 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6120000000000001e-06 2024-09-21T21:41:52.163 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.163 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.163 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:52.163 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.715954+0000", 2024-09-21T21:41:52.163 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.5899999999999997e-06 2024-09-21T21:41:52.163 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.163 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.164 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:52.164 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.715959+0000", 2024-09-21T21:41:52.164 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0849999999999996e-06 2024-09-21T21:41:52.164 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.164 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.164 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:52.164 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792771+0000", 2024-09-21T21:41:52.164 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.076811829999999998 2024-09-21T21:41:52.164 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.164 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.164 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:52.164 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792882+0000", 2024-09-21T21:41:52.164 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000110857 2024-09-21T21:41:52.164 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.165 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.165 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:52.165 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.793078+0000", 2024-09-21T21:41:52.165 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00019558499999999999 2024-09-21T21:41:52.165 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:52.165 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:52.165 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:52.165 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.165 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.165 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11332 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 7565312~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:52.165 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.716063+0000", 2024-09-21T21:41:52.165 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 1.4200839809999999, 2024-09-21T21:41:52.166 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078787482000000006, 2024-09-21T21:41:52.166 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:52.166 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:52.166 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:52.166 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:52.166 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:41:52.166 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:52.166 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11332 2024-09-21T21:41:52.166 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.167 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:52.167 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.167 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:52.167 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.716063+0000", 2024-09-21T21:41:52.167 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:52.167 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.167 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.167 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:52.167 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.716063+0000", 2024-09-21T21:41:52.167 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:52.167 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.168 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.168 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:52.168 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.717944+0000", 2024-09-21T21:41:52.168 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0018812309999999999 2024-09-21T21:41:52.168 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.168 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.168 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:52.168 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.717949+0000", 2024-09-21T21:41:52.168 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9389999999999996e-06 2024-09-21T21:41:52.169 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.169 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.169 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:52.169 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.717963+0000", 2024-09-21T21:41:52.169 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.3899e-05 2024-09-21T21:41:52.169 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.169 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.169 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:52.169 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.717970+0000", 2024-09-21T21:41:52.169 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.426e-06 2024-09-21T21:41:52.169 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.169 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.169 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:52.170 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.794573+0000", 2024-09-21T21:41:52.170 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.076602990999999995 2024-09-21T21:41:52.170 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.170 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.170 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:52.170 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.794682+0000", 2024-09-21T21:41:52.170 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000109541 2024-09-21T21:41:52.171 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.171 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.171 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:52.171 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.794851+0000", 2024-09-21T21:41:52.171 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000168455 2024-09-21T21:41:52.171 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:52.171 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:52.171 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:52.171 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.172 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.172 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11339 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 6266880~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:52.172 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.718117+0000", 2024-09-21T21:41:52.172 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 1.418029955, 2024-09-21T21:41:52.172 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078703717000000006, 2024-09-21T21:41:52.172 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:52.172 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:52.173 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:52.173 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:52.173 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:41:52.173 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:52.173 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11339 2024-09-21T21:41:52.173 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.173 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:52.173 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.173 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:52.173 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.718117+0000", 2024-09-21T21:41:52.174 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:52.174 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.174 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.174 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:52.174 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.718117+0000", 2024-09-21T21:41:52.174 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:52.174 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.174 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.175 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:52.175 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720019+0000", 2024-09-21T21:41:52.175 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0019012549999999999 2024-09-21T21:41:52.175 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.175 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.175 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:52.175 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720024+0000", 2024-09-21T21:41:52.175 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1100000000000002e-06 2024-09-21T21:41:52.175 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.176 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.176 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:52.176 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720035+0000", 2024-09-21T21:41:52.176 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.094e-05 2024-09-21T21:41:52.176 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.176 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.176 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:52.176 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720040+0000", 2024-09-21T21:41:52.176 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1830000000000002e-06 2024-09-21T21:41:52.177 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.177 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.177 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:52.177 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796618+0000", 2024-09-21T21:41:52.177 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.076578033000000004 2024-09-21T21:41:52.177 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.177 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.177 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:52.177 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796696+0000", 2024-09-21T21:41:52.178 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7836000000000005e-05 2024-09-21T21:41:52.178 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.178 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.178 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:52.178 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796821+0000", 2024-09-21T21:41:52.178 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00012536000000000001 2024-09-21T21:41:52.178 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:52.178 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:52.179 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:52.179 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.179 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.179 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11345 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 5472256~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:52.179 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.720116+0000", 2024-09-21T21:41:52.179 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 1.4160310549999999, 2024-09-21T21:41:52.179 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078667730000000005, 2024-09-21T21:41:52.179 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:52.179 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:52.180 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:52.180 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:52.180 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:41:52.180 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:52.180 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11345 2024-09-21T21:41:52.180 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.180 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:52.180 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.181 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:52.181 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720116+0000", 2024-09-21T21:41:52.181 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:52.181 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.181 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.181 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:52.181 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720116+0000", 2024-09-21T21:41:52.181 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:52.181 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.182 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.182 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:52.182 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720116+0000", 2024-09-21T21:41:52.182 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.04e-07 2024-09-21T21:41:52.182 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.182 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.182 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:52.182 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720118+0000", 2024-09-21T21:41:52.183 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.181e-06 2024-09-21T21:41:52.183 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.183 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.183 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:52.183 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720121+0000", 2024-09-21T21:41:52.183 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0199999999999999e-06 2024-09-21T21:41:52.183 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.184 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.184 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:52.184 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720123+0000", 2024-09-21T21:41:52.184 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.5469999999999999e-06 2024-09-21T21:41:52.184 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.184 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.184 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:52.184 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798626+0000", 2024-09-21T21:41:52.184 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078503137000000001 2024-09-21T21:41:52.184 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.184 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.185 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:52.185 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798677+0000", 2024-09-21T21:41:52.185 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0930999999999997e-05 2024-09-21T21:41:52.185 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.185 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.185 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:52.185 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798784+0000", 2024-09-21T21:41:52.185 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010671 2024-09-21T21:41:52.185 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:52.185 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:52.185 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:52.185 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.185 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.186 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11346 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 5500928~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:52.186 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.720126+0000", 2024-09-21T21:41:52.186 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 1.416020982, 2024-09-21T21:41:52.186 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078863421000000003, 2024-09-21T21:41:52.186 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:52.186 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:52.186 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:52.186 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:52.186 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:41:52.186 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:52.186 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11346 2024-09-21T21:41:52.186 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.186 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:52.187 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.187 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:52.187 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720126+0000", 2024-09-21T21:41:52.187 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:52.187 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.187 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.187 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:52.187 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720126+0000", 2024-09-21T21:41:52.187 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:52.187 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.187 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.187 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:52.188 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.722145+0000", 2024-09-21T21:41:52.188 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0020183219999999999 2024-09-21T21:41:52.188 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.188 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.188 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:52.188 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.722150+0000", 2024-09-21T21:41:52.188 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1760000000000001e-06 2024-09-21T21:41:52.188 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.188 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.188 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:52.188 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.722161+0000", 2024-09-21T21:41:52.188 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1566000000000001e-05 2024-09-21T21:41:52.188 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.189 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.189 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:52.189 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.722169+0000", 2024-09-21T21:41:52.189 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.2840000000000001e-06 2024-09-21T21:41:52.189 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.189 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.189 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:52.189 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798815+0000", 2024-09-21T21:41:52.189 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.076646607000000005 2024-09-21T21:41:52.190 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.190 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.190 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:52.190 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798870+0000", 2024-09-21T21:41:52.190 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4486e-05 2024-09-21T21:41:52.190 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.190 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.190 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:52.190 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798990+0000", 2024-09-21T21:41:52.190 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011998 2024-09-21T21:41:52.190 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:52.190 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:52.190 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:52.190 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.191 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.191 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13629 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7852032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:52.191 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943319+0000", 2024-09-21T21:41:52.191 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 1.1928283399999999, 2024-09-21T21:41:52.191 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078610975999999999, 2024-09-21T21:41:52.191 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:52.191 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:52.191 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:52.191 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:52.191 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:41:52.191 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:52.191 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13629 2024-09-21T21:41:52.191 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.192 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:52.192 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.192 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:52.192 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943319+0000", 2024-09-21T21:41:52.192 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:52.192 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.192 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.192 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:52.192 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943319+0000", 2024-09-21T21:41:52.192 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:52.192 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.193 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.193 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:52.193 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943319+0000", 2024-09-21T21:41:52.193 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6199999999999999e-07 2024-09-21T21:41:52.193 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.193 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.193 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:52.193 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943321+0000", 2024-09-21T21:41:52.193 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1009999999999999e-06 2024-09-21T21:41:52.193 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.193 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.194 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:52.194 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943328+0000", 2024-09-21T21:41:52.194 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.1029999999999996e-06 2024-09-21T21:41:52.194 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.194 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.194 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:52.194 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943333+0000", 2024-09-21T21:41:52.194 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7709999999999997e-06 2024-09-21T21:41:52.194 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.194 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.194 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:52.194 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.021679+0000", 2024-09-21T21:41:52.194 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078345673000000005 2024-09-21T21:41:52.195 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.195 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.195 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:52.195 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.021771+0000", 2024-09-21T21:41:52.195 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2201999999999996e-05 2024-09-21T21:41:52.195 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.195 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.195 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:52.195 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.021930+0000", 2024-09-21T21:41:52.195 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00015876400000000001 2024-09-21T21:41:52.195 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:52.195 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:52.195 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:52.195 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.195 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.196 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13630 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7880704~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:52.196 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943339+0000", 2024-09-21T21:41:52.196 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 1.192808152, 2024-09-21T21:41:52.196 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078939410000000002, 2024-09-21T21:41:52.196 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:52.196 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:52.196 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:52.196 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:52.196 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:41:52.196 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:52.196 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13630 2024-09-21T21:41:52.196 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.196 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:52.196 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.197 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:52.197 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943339+0000", 2024-09-21T21:41:52.197 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:52.197 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.197 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.197 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:52.197 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943339+0000", 2024-09-21T21:41:52.197 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:52.197 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.197 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.197 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:52.197 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943339+0000", 2024-09-21T21:41:52.197 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8000000000000001e-07 2024-09-21T21:41:52.198 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.198 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.198 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:52.198 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943341+0000", 2024-09-21T21:41:52.198 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9850000000000001e-06 2024-09-21T21:41:52.198 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.198 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.198 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:52.198 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943347+0000", 2024-09-21T21:41:52.198 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2979999999999999e-06 2024-09-21T21:41:52.198 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.198 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.198 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:52.198 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943354+0000", 2024-09-21T21:41:52.199 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.2969999999999998e-06 2024-09-21T21:41:52.199 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.199 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.199 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:52.199 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.021996+0000", 2024-09-21T21:41:52.199 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078641680000000005 2024-09-21T21:41:52.199 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.199 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.199 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:52.199 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022104+0000", 2024-09-21T21:41:52.199 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000107902 2024-09-21T21:41:52.199 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.200 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.200 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:52.200 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022278+0000", 2024-09-21T21:41:52.200 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00017486800000000001 2024-09-21T21:41:52.200 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:52.200 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:52.200 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:52.200 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.200 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.200 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13631 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7909376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:52.200 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943362+0000", 2024-09-21T21:41:52.200 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 1.1927853319999999, 2024-09-21T21:41:52.201 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079195644999999995, 2024-09-21T21:41:52.201 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:52.201 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:52.201 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:52.201 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:52.201 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:41:52.201 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:52.201 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13631 2024-09-21T21:41:52.201 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.201 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:52.201 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.201 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:52.201 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943362+0000", 2024-09-21T21:41:52.201 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:52.202 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.202 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.202 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:52.202 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943362+0000", 2024-09-21T21:41:52.202 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:52.202 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.202 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.202 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:52.202 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943362+0000", 2024-09-21T21:41:52.202 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8799999999999998e-07 2024-09-21T21:41:52.202 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.202 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.202 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:52.202 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943364+0000", 2024-09-21T21:41:52.203 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9850000000000001e-06 2024-09-21T21:41:52.203 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.203 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.203 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:52.203 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943371+0000", 2024-09-21T21:41:52.203 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0650000000000001e-06 2024-09-21T21:41:52.203 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.203 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.203 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:52.203 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943376+0000", 2024-09-21T21:41:52.203 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8250000000000004e-06 2024-09-21T21:41:52.203 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.203 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.204 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:52.204 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022329+0000", 2024-09-21T21:41:52.204 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078952888999999998 2024-09-21T21:41:52.204 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.204 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.204 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:52.204 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022419+0000", 2024-09-21T21:41:52.204 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.9738e-05 2024-09-21T21:41:52.204 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.204 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.204 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:52.204 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022558+0000", 2024-09-21T21:41:52.204 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00013875500000000001 2024-09-21T21:41:52.204 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:52.204 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:52.205 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:52.205 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.205 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.205 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13632 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7966720~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:52.205 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943382+0000", 2024-09-21T21:41:52.205 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 1.192765098, 2024-09-21T21:41:52.205 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079495727000000002, 2024-09-21T21:41:52.205 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:52.205 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:52.205 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:52.205 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:52.205 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:41:52.205 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:52.206 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13632 2024-09-21T21:41:52.206 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.206 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:52.206 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.206 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:52.206 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943382+0000", 2024-09-21T21:41:52.206 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:52.206 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.206 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.206 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:52.206 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943382+0000", 2024-09-21T21:41:52.206 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:52.206 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.206 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.207 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:52.207 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943382+0000", 2024-09-21T21:41:52.207 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7e-07 2024-09-21T21:41:52.207 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.207 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.207 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:52.207 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943385+0000", 2024-09-21T21:41:52.207 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2230000000000001e-06 2024-09-21T21:41:52.207 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.207 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.207 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:52.207 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943391+0000", 2024-09-21T21:41:52.207 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6810000000000001e-06 2024-09-21T21:41:52.207 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.207 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.208 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:52.208 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943397+0000", 2024-09-21T21:41:52.208 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6559999999999997e-06 2024-09-21T21:41:52.208 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.208 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.208 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:52.208 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022601+0000", 2024-09-21T21:41:52.208 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079204051999999997 2024-09-21T21:41:52.208 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.208 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.208 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:52.208 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022704+0000", 2024-09-21T21:41:52.208 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010329599999999999 2024-09-21T21:41:52.208 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.209 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.209 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:52.209 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022878+0000", 2024-09-21T21:41:52.209 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000173449 2024-09-21T21:41:52.209 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:52.209 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:52.209 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:52.209 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.209 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.209 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13633 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7938048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:52.209 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943403+0000", 2024-09-21T21:41:52.209 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 1.1927441679999999, 2024-09-21T21:41:52.210 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079794697999999997, 2024-09-21T21:41:52.210 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:52.210 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:52.210 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:52.210 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:52.211 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:41:52.211 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:52.211 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13633 2024-09-21T21:41:52.211 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.211 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:52.211 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.211 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:52.211 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943403+0000", 2024-09-21T21:41:52.211 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:52.211 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.212 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.212 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:52.212 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943403+0000", 2024-09-21T21:41:52.212 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:52.212 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.212 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.212 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:52.212 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943403+0000", 2024-09-21T21:41:52.212 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.65e-07 2024-09-21T21:41:52.212 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.212 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.212 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:52.212 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943405+0000", 2024-09-21T21:41:52.213 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0020000000000001e-06 2024-09-21T21:41:52.213 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.213 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.213 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:52.213 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943412+0000", 2024-09-21T21:41:52.213 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.8379999999999996e-06 2024-09-21T21:41:52.213 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.213 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.213 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:52.213 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943417+0000", 2024-09-21T21:41:52.213 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7650000000000001e-06 2024-09-21T21:41:52.213 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.213 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.213 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:52.214 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022922+0000", 2024-09-21T21:41:52.214 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079504512999999999 2024-09-21T21:41:52.214 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.214 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.214 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:52.214 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023039+0000", 2024-09-21T21:41:52.214 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000117044 2024-09-21T21:41:52.214 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.214 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.214 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:52.214 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023198+0000", 2024-09-21T21:41:52.214 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000159171 2024-09-21T21:41:52.214 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:52.215 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:52.215 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:52.215 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.215 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.215 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13634 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7995392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:52.215 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943423+0000", 2024-09-21T21:41:52.215 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 1.19272446, 2024-09-21T21:41:52.215 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.080142358999999996, 2024-09-21T21:41:52.215 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:52.215 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:52.216 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:52.216 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:52.216 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:41:52.216 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:52.216 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13634 2024-09-21T21:41:52.216 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.216 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:52.216 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.216 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:52.216 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943423+0000", 2024-09-21T21:41:52.216 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:52.216 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.216 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.217 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:52.217 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943423+0000", 2024-09-21T21:41:52.217 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:52.217 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.217 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.217 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:52.217 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943423+0000", 2024-09-21T21:41:52.217 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.58e-07 2024-09-21T21:41:52.217 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.217 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.217 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:52.217 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943425+0000", 2024-09-21T21:41:52.218 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9180000000000001e-06 2024-09-21T21:41:52.218 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.218 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.218 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:52.218 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943431+0000", 2024-09-21T21:41:52.218 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4439999999999996e-06 2024-09-21T21:41:52.218 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.218 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.219 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:52.219 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943436+0000", 2024-09-21T21:41:52.219 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.887e-06 2024-09-21T21:41:52.219 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.219 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.219 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:52.219 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023266+0000", 2024-09-21T21:41:52.219 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079829622000000003 2024-09-21T21:41:52.219 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.219 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.219 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:52.219 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023384+0000", 2024-09-21T21:41:52.219 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000117658 2024-09-21T21:41:52.219 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.220 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.220 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:52.220 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023565+0000", 2024-09-21T21:41:52.220 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00018147200000000001 2024-09-21T21:41:52.220 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:52.220 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:52.220 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:52.220 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.220 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.220 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13635 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 8024064~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:52.220 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943447+0000", 2024-09-21T21:41:52.220 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 1.1926999199999999, 2024-09-21T21:41:52.220 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.080417653000000006, 2024-09-21T21:41:52.221 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:52.221 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:52.221 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:52.221 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:52.221 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:41:52.222 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:52.222 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13635 2024-09-21T21:41:52.222 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.222 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:52.222 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.222 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:52.222 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943447+0000", 2024-09-21T21:41:52.222 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:52.222 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.222 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.222 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:52.223 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943447+0000", 2024-09-21T21:41:52.223 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:52.223 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.223 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.223 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:52.223 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943448+0000", 2024-09-21T21:41:52.223 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1699999999999999e-07 2024-09-21T21:41:52.223 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.223 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.223 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:52.223 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943450+0000", 2024-09-21T21:41:52.223 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1639999999999999e-06 2024-09-21T21:41:52.223 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.223 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.224 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:52.224 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943456+0000", 2024-09-21T21:41:52.224 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8379999999999998e-06 2024-09-21T21:41:52.224 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.224 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.224 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:52.224 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943460+0000", 2024-09-21T21:41:52.224 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4700000000000004e-06 2024-09-21T21:41:52.224 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.224 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.224 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:52.224 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023595+0000", 2024-09-21T21:41:52.224 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.080134869999999997 2024-09-21T21:41:52.224 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.224 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.225 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:52.225 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023688+0000", 2024-09-21T21:41:52.225 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2794000000000006e-05 2024-09-21T21:41:52.225 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.225 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.225 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:52.225 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023865+0000", 2024-09-21T21:41:52.225 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0001771 2024-09-21T21:41:52.225 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:52.225 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:52.225 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:52.225 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.225 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.225 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13636 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 8052736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:52.226 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943467+0000", 2024-09-21T21:41:52.226 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 1.192680183, 2024-09-21T21:41:52.226 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.080694399999999999, 2024-09-21T21:41:52.226 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:52.226 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:52.226 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:52.226 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:52.226 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:41:52.226 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:52.226 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13636 2024-09-21T21:41:52.226 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.226 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:52.226 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.226 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:52.227 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943467+0000", 2024-09-21T21:41:52.227 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:52.227 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.227 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.227 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:52.227 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943467+0000", 2024-09-21T21:41:52.227 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:52.227 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.227 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.227 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:52.227 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944940+0000", 2024-09-21T21:41:52.227 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0014725109999999999 2024-09-21T21:41:52.227 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.227 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.227 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:52.228 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944949+0000", 2024-09-21T21:41:52.228 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2930000000000006e-06 2024-09-21T21:41:52.228 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.228 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.228 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:52.228 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944958+0000", 2024-09-21T21:41:52.228 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.6899999999999998e-06 2024-09-21T21:41:52.228 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.228 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.228 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:52.228 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944966+0000", 2024-09-21T21:41:52.228 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0690000000000002e-06 2024-09-21T21:41:52.228 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.229 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.229 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:52.229 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023900+0000", 2024-09-21T21:41:52.229 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078934371000000003 2024-09-21T21:41:52.229 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.229 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.229 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:52.229 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024009+0000", 2024-09-21T21:41:52.229 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000108857 2024-09-21T21:41:52.229 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.229 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.229 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:52.230 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024161+0000", 2024-09-21T21:41:52.230 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000152609 2024-09-21T21:41:52.230 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:52.230 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:52.230 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:52.230 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.230 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.230 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13637 3.12 3.1dc31b72 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:52.230 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.944978+0000", 2024-09-21T21:41:52.231 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 1.191169248, 2024-09-21T21:41:52.231 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079232604999999998, 2024-09-21T21:41:52.231 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:52.231 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:52.231 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:41:52.231 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:52.231 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:41:52.231 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:52.231 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13637 2024-09-21T21:41:52.232 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.232 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:52.232 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.232 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:52.232 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944978+0000", 2024-09-21T21:41:52.232 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:52.232 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.232 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.232 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:52.233 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944978+0000", 2024-09-21T21:41:52.233 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:52.233 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.233 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.233 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:52.233 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944978+0000", 2024-09-21T21:41:52.233 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9100000000000004e-07 2024-09-21T21:41:52.233 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.233 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.233 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:52.234 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944981+0000", 2024-09-21T21:41:52.234 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3290000000000001e-06 2024-09-21T21:41:52.234 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.234 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.234 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:52.234 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944987+0000", 2024-09-21T21:41:52.234 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.28e-06 2024-09-21T21:41:52.234 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.235 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.235 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:52.235 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944992+0000", 2024-09-21T21:41:52.235 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8520000000000003e-06 2024-09-21T21:41:52.235 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.235 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.235 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:52.235 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024196+0000", 2024-09-21T21:41:52.235 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079203673000000002 2024-09-21T21:41:52.235 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.236 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.236 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:52.236 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024211+0000", 2024-09-21T21:41:52.236 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.4980000000000001e-05 2024-09-21T21:41:52.236 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:52.236 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:52.236 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:52.236 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.236 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.236 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13638 3.12 3.1dc31b72 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:52.236 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.945000+0000", 2024-09-21T21:41:52.236 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 1.1911471200000001, 2024-09-21T21:41:52.236 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079257465999999999, 2024-09-21T21:41:52.237 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:52.237 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:52.237 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:41:52.237 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:52.237 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:41:52.237 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:52.237 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13638 2024-09-21T21:41:52.237 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.237 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:52.237 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.237 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:52.238 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945000+0000", 2024-09-21T21:41:52.238 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:52.238 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.238 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.238 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:52.238 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945000+0000", 2024-09-21T21:41:52.238 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:52.238 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.238 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.238 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:52.238 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945001+0000", 2024-09-21T21:41:52.239 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.32e-07 2024-09-21T21:41:52.239 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.239 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.239 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:52.239 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945003+0000", 2024-09-21T21:41:52.239 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2309999999999999e-06 2024-09-21T21:41:52.239 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.239 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.239 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:52.239 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945009+0000", 2024-09-21T21:41:52.239 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7450000000000001e-06 2024-09-21T21:41:52.239 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.239 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.239 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:52.240 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945013+0000", 2024-09-21T21:41:52.240 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.673e-06 2024-09-21T21:41:52.240 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.240 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.240 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:52.240 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024245+0000", 2024-09-21T21:41:52.240 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079232160999999995 2024-09-21T21:41:52.240 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.240 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.240 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:52.240 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024258+0000", 2024-09-21T21:41:52.241 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.2224e-05 2024-09-21T21:41:52.241 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:52.241 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:52.241 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:52.241 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.241 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.241 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13639 3.12 3.1dc31b72 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:52.241 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.945034+0000", 2024-09-21T21:41:52.242 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 1.191113321, 2024-09-21T21:41:52.242 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079260627, 2024-09-21T21:41:52.242 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:52.242 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:52.242 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:41:52.242 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:52.242 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:41:52.242 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:52.242 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13639 2024-09-21T21:41:52.242 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.243 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:52.243 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.243 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:52.243 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945034+0000", 2024-09-21T21:41:52.243 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:52.243 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.243 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.243 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:52.243 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945034+0000", 2024-09-21T21:41:52.243 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:52.243 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.243 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.244 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:52.244 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945034+0000", 2024-09-21T21:41:52.244 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3799999999999998e-07 2024-09-21T21:41:52.244 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.244 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.244 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:52.244 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945036+0000", 2024-09-21T21:41:52.244 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.052e-06 2024-09-21T21:41:52.244 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.244 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.244 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:52.244 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945040+0000", 2024-09-21T21:41:52.244 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1169999999999997e-06 2024-09-21T21:41:52.245 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.245 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.245 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:52.245 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945043+0000", 2024-09-21T21:41:52.245 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.8439999999999998e-06 2024-09-21T21:41:52.245 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.245 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.245 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:52.245 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024286+0000", 2024-09-21T21:41:52.245 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079242483000000002 2024-09-21T21:41:52.245 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.245 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.245 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:52.245 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024295+0000", 2024-09-21T21:41:52.246 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.7930000000000003e-06 2024-09-21T21:41:52.246 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:52.246 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:52.246 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:52.246 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.246 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.246 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13640 3.12 3.1dc31b72 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:52.246 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.945047+0000", 2024-09-21T21:41:52.246 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 1.1910997990000001, 2024-09-21T21:41:52.246 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079412321999999994, 2024-09-21T21:41:52.246 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:52.247 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:52.247 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:41:52.247 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:52.247 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:41:52.247 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:52.247 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13640 2024-09-21T21:41:52.247 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.247 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:52.247 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.247 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:52.247 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945047+0000", 2024-09-21T21:41:52.247 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:52.247 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.247 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.247 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:52.248 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945047+0000", 2024-09-21T21:41:52.248 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:52.248 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.248 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.248 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:52.248 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945048+0000", 2024-09-21T21:41:52.248 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0499999999999999e-07 2024-09-21T21:41:52.248 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.248 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.248 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:52.248 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945049+0000", 2024-09-21T21:41:52.248 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.3939999999999999e-06 2024-09-21T21:41:52.248 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.249 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.249 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:52.249 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945053+0000", 2024-09-21T21:41:52.249 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1760000000000003e-06 2024-09-21T21:41:52.249 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.249 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.249 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:52.249 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945056+0000", 2024-09-21T21:41:52.249 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.6129999999999998e-06 2024-09-21T21:41:52.249 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.249 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.249 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:52.249 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024360+0000", 2024-09-21T21:41:52.250 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079303645000000006 2024-09-21T21:41:52.250 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.250 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.250 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:52.250 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024460+0000", 2024-09-21T21:41:52.250 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000100189 2024-09-21T21:41:52.250 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:52.250 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:52.250 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:52.250 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:52.250 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:52.250 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:52.252 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T21:41:52.253 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:52.250+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:52.352 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:52.351+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:52.395 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:52.395 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:41:52.395 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 0 2024-09-21T21:41:52.395 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:52.403 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T21:41:52.487 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:52.486+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:52.546 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:52.547 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:41:52.547 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:41:52.547 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:41:52.547 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:52.555 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:52.554+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:52.556 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T21:41:52.602 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T21:41:52.683 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:52.682+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:52.703 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:52.703 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 20, 2024-09-21T21:41:52.703 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 600, 2024-09-21T21:41:52.703 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:41:52.703 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.703 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4824.0:1564 3.2 3:4647520c:::rbd_data.1121b97f2925.0000000000000006:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:52.703 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:41:52.704 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 51.064882507999997, 2024-09-21T21:41:52.704 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056097347999999998, 2024-09-21T21:41:52.704 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:52.704 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:52.704 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:52.704 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:52.704 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4824", 2024-09-21T21:41:52.704 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:52.704 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1564 2024-09-21T21:41:52.704 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.704 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:52.704 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.705 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:52.705 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:41:52.705 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:52.705 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.705 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.705 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:52.705 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:41:52.705 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:52.705 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.705 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.705 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:52.705 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:41:52.705 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6499999999999999e-07 2024-09-21T21:41:52.706 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.706 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.706 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:52.706 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636067+0000", 2024-09-21T21:41:52.706 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3879999999999998e-06 2024-09-21T21:41:52.706 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.706 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.706 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:52.706 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636075+0000", 2024-09-21T21:41:52.706 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5480000000000004e-06 2024-09-21T21:41:52.706 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.706 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.707 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:52.707 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636080+0000", 2024-09-21T21:41:52.707 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.339e-06 2024-09-21T21:41:52.707 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.707 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.707 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:52.707 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636133+0000", 2024-09-21T21:41:52.707 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2930000000000003e-05 2024-09-21T21:41:52.707 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.707 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.707 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:52.708 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636265+0000", 2024-09-21T21:41:52.708 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000132058 2024-09-21T21:41:52.708 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.708 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.708 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:52.708 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.692162+0000", 2024-09-21T21:41:52.708 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055896620000000001 2024-09-21T21:41:52.708 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:52.708 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:52.708 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:52.708 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.708 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.709 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3258 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 1409024~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:41:52.709 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.196243+0000", 2024-09-21T21:41:52.709 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 5.5047036680000003, 2024-09-21T21:41:52.709 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055725496999999999, 2024-09-21T21:41:52.709 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:52.709 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:52.709 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:52.709 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:52.709 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:41:52.709 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:52.709 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3258 2024-09-21T21:41:52.709 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.710 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:52.710 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.710 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:52.710 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196243+0000", 2024-09-21T21:41:52.710 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:52.710 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.710 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.710 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:52.710 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196243+0000", 2024-09-21T21:41:52.710 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:52.710 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.711 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.711 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:52.711 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196244+0000", 2024-09-21T21:41:52.711 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4799999999999999e-07 2024-09-21T21:41:52.711 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.711 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.711 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:52.711 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196246+0000", 2024-09-21T21:41:52.711 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1390000000000002e-06 2024-09-21T21:41:52.711 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.712 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.712 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:52.712 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196255+0000", 2024-09-21T21:41:52.712 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.5779999999999999e-06 2024-09-21T21:41:52.712 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.712 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.712 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:52.712 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196267+0000", 2024-09-21T21:41:52.712 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.2726e-05 2024-09-21T21:41:52.712 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.712 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.712 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:52.712 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.251811+0000", 2024-09-21T21:41:52.713 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055544068000000002 2024-09-21T21:41:52.713 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.713 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.713 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:52.713 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.251878+0000", 2024-09-21T21:41:52.713 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6898999999999998e-05 2024-09-21T21:41:52.713 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.713 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.713 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:52.713 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.251969+0000", 2024-09-21T21:41:52.713 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.0638999999999997e-05 2024-09-21T21:41:52.713 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:52.714 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:52.714 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:52.714 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.714 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.714 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3259 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 663552~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:41:52.714 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.196275+0000", 2024-09-21T21:41:52.714 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 5.5046723599999998, 2024-09-21T21:41:52.715 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055889580000000001, 2024-09-21T21:41:52.715 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:52.715 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:52.715 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:52.715 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:52.715 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:41:52.715 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:52.715 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3259 2024-09-21T21:41:52.715 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.716 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:52.716 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.716 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:52.716 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196275+0000", 2024-09-21T21:41:52.716 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:52.716 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.716 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.716 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:52.716 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196275+0000", 2024-09-21T21:41:52.716 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:52.716 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.716 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.717 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:52.717 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196275+0000", 2024-09-21T21:41:52.717 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2199999999999999e-07 2024-09-21T21:41:52.717 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.717 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.717 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:52.717 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196277+0000", 2024-09-21T21:41:52.717 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1270000000000001e-06 2024-09-21T21:41:52.717 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.717 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.717 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:52.717 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196285+0000", 2024-09-21T21:41:52.718 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.2290000000000003e-06 2024-09-21T21:41:52.718 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.718 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.718 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:52.718 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196291+0000", 2024-09-21T21:41:52.718 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1379999999999998e-06 2024-09-21T21:41:52.718 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.718 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.718 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:52.718 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.251997+0000", 2024-09-21T21:41:52.718 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055705971999999999 2024-09-21T21:41:52.719 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.719 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.719 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:52.719 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252077+0000", 2024-09-21T21:41:52.719 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0179999999999995e-05 2024-09-21T21:41:52.719 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.719 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.719 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:52.719 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252164+0000", 2024-09-21T21:41:52.719 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.7511999999999998e-05 2024-09-21T21:41:52.719 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:52.719 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:52.720 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:52.720 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.720 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.720 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3260 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 1437696~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:41:52.720 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.196297+0000", 2024-09-21T21:41:52.720 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 5.5046502979999996, 2024-09-21T21:41:52.720 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056071022999999998, 2024-09-21T21:41:52.720 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:52.720 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:52.720 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:52.720 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:52.720 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:41:52.721 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:52.721 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3260 2024-09-21T21:41:52.721 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.721 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:52.721 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.721 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:52.721 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196297+0000", 2024-09-21T21:41:52.721 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:52.721 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.721 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.721 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:52.722 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196297+0000", 2024-09-21T21:41:52.722 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:52.722 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.722 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.722 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:52.722 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196918+0000", 2024-09-21T21:41:52.722 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00062148699999999999 2024-09-21T21:41:52.722 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.722 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.722 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:52.722 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196930+0000", 2024-09-21T21:41:52.722 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1657e-05 2024-09-21T21:41:52.723 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.723 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.723 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:52.723 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196944+0000", 2024-09-21T21:41:52.723 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.4549000000000001e-05 2024-09-21T21:41:52.723 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.723 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.723 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:52.723 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196951+0000", 2024-09-21T21:41:52.723 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6749999999999996e-06 2024-09-21T21:41:52.723 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.724 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.724 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:52.724 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252208+0000", 2024-09-21T21:41:52.724 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055256656000000001 2024-09-21T21:41:52.724 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.724 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.724 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:52.724 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252278+0000", 2024-09-21T21:41:52.724 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9882000000000006e-05 2024-09-21T21:41:52.724 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.724 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.724 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:52.725 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252368+0000", 2024-09-21T21:41:52.725 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.0117000000000005e-05 2024-09-21T21:41:52.725 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:52.725 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:52.725 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:52.725 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.725 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.725 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3261 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 2842624~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:41:52.725 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.196963+0000", 2024-09-21T21:41:52.725 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 5.5039839109999997, 2024-09-21T21:41:52.725 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055594613000000001, 2024-09-21T21:41:52.725 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:52.726 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:52.726 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:52.726 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:52.726 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:41:52.726 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:52.726 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3261 2024-09-21T21:41:52.726 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.726 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:52.726 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.726 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:52.726 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196963+0000", 2024-09-21T21:41:52.726 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:52.726 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.727 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.727 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:52.727 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196963+0000", 2024-09-21T21:41:52.727 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:52.727 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.727 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.727 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:52.727 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196964+0000", 2024-09-21T21:41:52.727 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3000000000000001e-07 2024-09-21T21:41:52.727 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.727 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.727 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:52.728 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196966+0000", 2024-09-21T21:41:52.728 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.475e-06 2024-09-21T21:41:52.728 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.728 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.728 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:52.728 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196973+0000", 2024-09-21T21:41:52.728 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5540000000000003e-06 2024-09-21T21:41:52.728 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.728 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.728 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:52.728 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196978+0000", 2024-09-21T21:41:52.728 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0719999999999999e-06 2024-09-21T21:41:52.729 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.729 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.729 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:52.729 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252395+0000", 2024-09-21T21:41:52.729 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055417301000000002 2024-09-21T21:41:52.729 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.729 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.729 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:52.729 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252468+0000", 2024-09-21T21:41:52.729 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.2692000000000001e-05 2024-09-21T21:41:52.729 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.729 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.730 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:52.730 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252558+0000", 2024-09-21T21:41:52.730 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.9988999999999995e-05 2024-09-21T21:41:52.730 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:52.730 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:52.730 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:52.730 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.730 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.730 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3262 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 2871296~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:41:52.730 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.196985+0000", 2024-09-21T21:41:52.730 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 5.5039624190000005, 2024-09-21T21:41:52.731 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055777262000000001, 2024-09-21T21:41:52.731 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:52.731 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:52.731 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:52.731 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:52.731 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:41:52.731 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:52.731 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3262 2024-09-21T21:41:52.731 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.731 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:52.731 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.731 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:52.732 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196985+0000", 2024-09-21T21:41:52.732 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:52.732 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.732 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.732 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:52.732 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196985+0000", 2024-09-21T21:41:52.732 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:52.732 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.732 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.732 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:52.732 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197024+0000", 2024-09-21T21:41:52.732 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.9558000000000003e-05 2024-09-21T21:41:52.732 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.733 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.733 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:52.733 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197026+0000", 2024-09-21T21:41:52.733 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0899999999999999e-06 2024-09-21T21:41:52.733 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.733 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.733 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:52.733 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197031+0000", 2024-09-21T21:41:52.733 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8849999999999998e-06 2024-09-21T21:41:52.733 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.733 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.733 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:52.733 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197035+0000", 2024-09-21T21:41:52.734 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5719999999999999e-06 2024-09-21T21:41:52.734 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.734 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.734 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:52.734 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252587+0000", 2024-09-21T21:41:52.734 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055552193 2024-09-21T21:41:52.734 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.734 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.734 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:52.734 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252670+0000", 2024-09-21T21:41:52.734 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.2687000000000003e-05 2024-09-21T21:41:52.734 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.735 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.735 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:52.735 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252762+0000", 2024-09-21T21:41:52.735 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2276999999999998e-05 2024-09-21T21:41:52.735 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:52.735 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:52.735 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:52.735 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.735 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.735 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3263 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 3559424~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:41:52.735 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.197041+0000", 2024-09-21T21:41:52.735 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 5.5039064839999998, 2024-09-21T21:41:52.736 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055935804999999998, 2024-09-21T21:41:52.736 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:52.736 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:52.736 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:52.736 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:52.736 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:41:52.736 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:52.736 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3263 2024-09-21T21:41:52.736 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.736 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:52.736 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.736 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:52.737 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197041+0000", 2024-09-21T21:41:52.737 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:52.737 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.737 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.737 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:52.737 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197041+0000", 2024-09-21T21:41:52.737 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:52.737 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.737 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.737 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:52.737 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197624+0000", 2024-09-21T21:41:52.737 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00058358100000000001 2024-09-21T21:41:52.737 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.738 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.738 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:52.738 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197629+0000", 2024-09-21T21:41:52.738 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5909999999999998e-06 2024-09-21T21:41:52.738 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.738 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.738 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:52.738 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197639+0000", 2024-09-21T21:41:52.738 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0257e-05 2024-09-21T21:41:52.738 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.738 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.738 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:52.739 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197646+0000", 2024-09-21T21:41:52.739 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.7229999999999998e-06 2024-09-21T21:41:52.739 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.739 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.739 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:52.739 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252804+0000", 2024-09-21T21:41:52.739 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055158134999999997 2024-09-21T21:41:52.739 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.739 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.739 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:52.739 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252882+0000", 2024-09-21T21:41:52.739 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7791000000000001e-05 2024-09-21T21:41:52.740 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.740 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.740 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:52.740 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252976+0000", 2024-09-21T21:41:52.740 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.4727000000000003e-05 2024-09-21T21:41:52.740 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:52.740 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:52.740 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:52.740 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.740 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.740 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3325 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:41:52.740 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.197722+0000", 2024-09-21T21:41:52.741 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 5.5032246669999996, 2024-09-21T21:41:52.741 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055712048, 2024-09-21T21:41:52.741 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:52.741 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:52.741 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:52.741 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:52.741 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:41:52.741 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:52.741 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3325 2024-09-21T21:41:52.741 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.741 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:52.742 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.742 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:52.742 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197722+0000", 2024-09-21T21:41:52.742 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:52.742 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.742 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.742 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:52.742 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197722+0000", 2024-09-21T21:41:52.742 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:52.742 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.742 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.743 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:52.743 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.198052+0000", 2024-09-21T21:41:52.743 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00033006600000000001 2024-09-21T21:41:52.743 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.743 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.743 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:52.743 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.198056+0000", 2024-09-21T21:41:52.743 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7330000000000001e-06 2024-09-21T21:41:52.743 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.743 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.743 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:52.743 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.198065+0000", 2024-09-21T21:41:52.744 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.8990000000000007e-06 2024-09-21T21:41:52.744 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.744 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.744 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:52.744 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.198072+0000", 2024-09-21T21:41:52.744 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5339999999999996e-06 2024-09-21T21:41:52.744 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.744 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.744 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:52.744 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.253244+0000", 2024-09-21T21:41:52.744 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055172064999999999 2024-09-21T21:41:52.744 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.745 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.745 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:52.745 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.253342+0000", 2024-09-21T21:41:52.745 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.8175000000000001e-05 2024-09-21T21:41:52.745 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.745 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.745 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:52.745 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.253434+0000", 2024-09-21T21:41:52.745 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2576000000000003e-05 2024-09-21T21:41:52.745 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:52.745 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:52.745 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:52.746 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.746 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.746 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3437 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 925696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:41:52.746 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.205888+0000", 2024-09-21T21:41:52.746 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 5.4950592030000003, 2024-09-21T21:41:52.746 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055953808000000001, 2024-09-21T21:41:52.746 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:52.746 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:52.746 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:52.746 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:52.746 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:41:52.746 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:52.747 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3437 2024-09-21T21:41:52.747 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.747 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:52.747 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.747 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:52.747 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205888+0000", 2024-09-21T21:41:52.747 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:52.747 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.747 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.747 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:52.747 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205888+0000", 2024-09-21T21:41:52.747 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:52.748 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.748 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.748 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:52.748 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205888+0000", 2024-09-21T21:41:52.748 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.75e-07 2024-09-21T21:41:52.748 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.748 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.748 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:52.748 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205891+0000", 2024-09-21T21:41:52.748 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3750000000000001e-06 2024-09-21T21:41:52.748 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.748 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.749 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:52.749 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205897+0000", 2024-09-21T21:41:52.749 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.285e-06 2024-09-21T21:41:52.749 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.749 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.749 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:52.749 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205902+0000", 2024-09-21T21:41:52.749 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6290000000000001e-06 2024-09-21T21:41:52.749 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.749 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.749 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:52.749 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261686+0000", 2024-09-21T21:41:52.750 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055784586999999997 2024-09-21T21:41:52.750 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.750 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.750 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:52.750 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261764+0000", 2024-09-21T21:41:52.750 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7591999999999998e-05 2024-09-21T21:41:52.750 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.750 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.750 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:52.750 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261842+0000", 2024-09-21T21:41:52.750 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7865000000000001e-05 2024-09-21T21:41:52.750 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:52.751 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:52.751 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:52.751 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.751 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.751 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3438 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 954368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:41:52.751 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.205908+0000", 2024-09-21T21:41:52.751 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 5.4950393379999998, 2024-09-21T21:41:52.751 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056083169000000002, 2024-09-21T21:41:52.751 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:52.751 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:52.751 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:52.751 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:52.752 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:41:52.752 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:52.752 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3438 2024-09-21T21:41:52.752 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.752 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:52.752 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.752 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:52.752 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205908+0000", 2024-09-21T21:41:52.752 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:52.752 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.752 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.753 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:52.753 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205908+0000", 2024-09-21T21:41:52.753 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:52.753 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.753 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.753 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:52.753 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206431+0000", 2024-09-21T21:41:52.753 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00052282100000000003 2024-09-21T21:41:52.753 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.753 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.753 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:52.754 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206442+0000", 2024-09-21T21:41:52.754 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1695000000000001e-05 2024-09-21T21:41:52.754 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.754 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.754 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:52.754 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206452+0000", 2024-09-21T21:41:52.754 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.4560000000000006e-06 2024-09-21T21:41:52.754 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.754 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.754 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:52.754 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206458+0000", 2024-09-21T21:41:52.755 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4670000000000001e-06 2024-09-21T21:41:52.755 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.755 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.755 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:52.755 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261871+0000", 2024-09-21T21:41:52.755 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055412712000000003 2024-09-21T21:41:52.755 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.755 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.755 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:52.755 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261928+0000", 2024-09-21T21:41:52.755 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6631e-05 2024-09-21T21:41:52.756 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.756 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.756 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:52.756 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261991+0000", 2024-09-21T21:41:52.756 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3386999999999995e-05 2024-09-21T21:41:52.756 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:52.756 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:52.756 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:52.756 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.756 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.756 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3440 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 1642496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:41:52.756 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.206491+0000", 2024-09-21T21:41:52.757 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 5.4944561590000003, 2024-09-21T21:41:52.757 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055633136, 2024-09-21T21:41:52.757 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:52.757 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:52.757 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:52.757 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:52.757 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:41:52.757 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:52.757 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3440 2024-09-21T21:41:52.757 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.757 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:52.757 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.758 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:52.758 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206491+0000", 2024-09-21T21:41:52.758 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:52.758 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.758 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.758 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:52.758 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206491+0000", 2024-09-21T21:41:52.758 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:52.758 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.758 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.759 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:52.759 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206491+0000", 2024-09-21T21:41:52.759 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6100000000000001e-07 2024-09-21T21:41:52.759 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.759 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.759 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:52.759 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206494+0000", 2024-09-21T21:41:52.760 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.9349999999999999e-06 2024-09-21T21:41:52.760 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.760 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.760 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:52.760 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206501+0000", 2024-09-21T21:41:52.760 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6939999999999998e-06 2024-09-21T21:41:52.760 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.761 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.761 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:52.761 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206506+0000", 2024-09-21T21:41:52.761 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2979999999999999e-06 2024-09-21T21:41:52.761 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.761 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.761 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:52.761 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262019+0000", 2024-09-21T21:41:52.761 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055513094999999998 2024-09-21T21:41:52.761 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.761 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.761 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:52.761 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262073+0000", 2024-09-21T21:41:52.762 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3486999999999998e-05 2024-09-21T21:41:52.762 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.762 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.762 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:52.762 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262124+0000", 2024-09-21T21:41:52.762 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1066000000000002e-05 2024-09-21T21:41:52.762 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:52.762 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:52.762 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:52.762 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.762 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.762 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3442 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 1671168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:41:52.762 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.206515+0000", 2024-09-21T21:41:52.763 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 5.4944324570000003, 2024-09-21T21:41:52.763 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055792680999999997, 2024-09-21T21:41:52.763 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:52.763 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:52.763 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:52.763 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:52.763 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:41:52.763 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:52.763 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3442 2024-09-21T21:41:52.763 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.763 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:52.763 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.764 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:52.764 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206515+0000", 2024-09-21T21:41:52.764 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:52.764 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.764 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.764 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:52.764 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206515+0000", 2024-09-21T21:41:52.764 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:52.764 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.764 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.764 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:52.764 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206515+0000", 2024-09-21T21:41:52.764 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2500000000000001e-07 2024-09-21T21:41:52.765 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.765 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.765 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:52.765 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206517+0000", 2024-09-21T21:41:52.765 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0420000000000001e-06 2024-09-21T21:41:52.765 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.765 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.765 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:52.765 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206522+0000", 2024-09-21T21:41:52.765 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3600000000000004e-06 2024-09-21T21:41:52.765 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.765 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.766 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:52.766 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206527+0000", 2024-09-21T21:41:52.766 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.809e-06 2024-09-21T21:41:52.766 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.766 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.766 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:52.766 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262151+0000", 2024-09-21T21:41:52.766 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055623458000000001 2024-09-21T21:41:52.766 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.766 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.766 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:52.766 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262210+0000", 2024-09-21T21:41:52.767 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9120999999999999e-05 2024-09-21T21:41:52.767 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.767 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.767 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:52.767 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262307+0000", 2024-09-21T21:41:52.767 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.7466000000000005e-05 2024-09-21T21:41:52.767 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:52.767 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:52.767 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:52.767 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.767 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.767 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3444 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 2359296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:41:52.768 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.206533+0000", 2024-09-21T21:41:52.768 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 5.4944138179999999, 2024-09-21T21:41:52.768 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055974239000000002, 2024-09-21T21:41:52.768 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:52.768 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:52.768 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:52.768 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:52.768 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:41:52.768 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:52.768 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3444 2024-09-21T21:41:52.768 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.769 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:52.769 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.769 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:52.769 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206533+0000", 2024-09-21T21:41:52.769 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:52.769 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.769 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.769 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:52.769 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206533+0000", 2024-09-21T21:41:52.769 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:52.769 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.769 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.769 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:52.770 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206710+0000", 2024-09-21T21:41:52.770 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000176293 2024-09-21T21:41:52.770 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.770 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.770 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:52.770 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206714+0000", 2024-09-21T21:41:52.770 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1989999999999999e-06 2024-09-21T21:41:52.770 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.770 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.770 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:52.770 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206723+0000", 2024-09-21T21:41:52.771 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.9199999999999993e-06 2024-09-21T21:41:52.771 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.771 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.771 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:52.771 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206729+0000", 2024-09-21T21:41:52.771 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0739999999999998e-06 2024-09-21T21:41:52.771 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.771 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.771 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:52.771 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262339+0000", 2024-09-21T21:41:52.771 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055610166000000003 2024-09-21T21:41:52.771 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.772 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.772 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:52.772 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262419+0000", 2024-09-21T21:41:52.772 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0432999999999994e-05 2024-09-21T21:41:52.772 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.772 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.772 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:52.772 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262507+0000", 2024-09-21T21:41:52.772 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.8153999999999997e-05 2024-09-21T21:41:52.772 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:52.772 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:52.773 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:52.773 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.773 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.773 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3445 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 1699840~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:41:52.773 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.206746+0000", 2024-09-21T21:41:52.773 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 5.4942008940000004, 2024-09-21T21:41:52.773 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055880738999999999, 2024-09-21T21:41:52.773 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:52.773 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:52.773 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:52.773 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:52.773 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:41:52.774 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:52.774 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3445 2024-09-21T21:41:52.774 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.774 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:52.774 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.774 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:52.774 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206746+0000", 2024-09-21T21:41:52.774 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:52.774 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.774 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.774 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:52.774 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206746+0000", 2024-09-21T21:41:52.775 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:52.775 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.775 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.775 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:52.775 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207412+0000", 2024-09-21T21:41:52.775 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00066611999999999997 2024-09-21T21:41:52.775 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.775 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.775 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:52.775 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207417+0000", 2024-09-21T21:41:52.775 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8539999999999996e-06 2024-09-21T21:41:52.775 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.776 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.776 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:52.776 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207437+0000", 2024-09-21T21:41:52.776 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9463e-05 2024-09-21T21:41:52.776 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.776 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.776 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:52.776 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207445+0000", 2024-09-21T21:41:52.776 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.5790000000000004e-06 2024-09-21T21:41:52.776 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.776 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.777 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:52.777 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262537+0000", 2024-09-21T21:41:52.777 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055091531999999999 2024-09-21T21:41:52.777 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.777 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.777 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:52.777 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262578+0000", 2024-09-21T21:41:52.777 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1220000000000002e-05 2024-09-21T21:41:52.777 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.777 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.777 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:52.777 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262627+0000", 2024-09-21T21:41:52.778 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8971000000000003e-05 2024-09-21T21:41:52.778 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:52.778 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:52.778 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:52.778 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.778 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.778 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3451 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 3104768~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:41:52.778 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.207509+0000", 2024-09-21T21:41:52.778 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 5.4934378290000003, 2024-09-21T21:41:52.778 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055698944, 2024-09-21T21:41:52.778 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:52.778 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:52.779 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:52.779 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:52.779 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:41:52.779 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:52.779 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3451 2024-09-21T21:41:52.779 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.779 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:52.779 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.779 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:52.779 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207509+0000", 2024-09-21T21:41:52.779 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:52.779 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.780 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.780 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:52.780 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207509+0000", 2024-09-21T21:41:52.780 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:52.780 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.780 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.780 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:52.780 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207661+0000", 2024-09-21T21:41:52.780 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00015181799999999999 2024-09-21T21:41:52.780 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.780 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.780 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:52.781 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207665+0000", 2024-09-21T21:41:52.781 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6380000000000002e-06 2024-09-21T21:41:52.781 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.781 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.781 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:52.781 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207674+0000", 2024-09-21T21:41:52.781 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.0950000000000002e-06 2024-09-21T21:41:52.781 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.781 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.781 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:52.781 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207689+0000", 2024-09-21T21:41:52.781 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.4904e-05 2024-09-21T21:41:52.781 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.782 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.782 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:52.782 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.263055+0000", 2024-09-21T21:41:52.782 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055366268000000003 2024-09-21T21:41:52.782 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.782 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.782 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:52.782 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.263101+0000", 2024-09-21T21:41:52.782 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6498000000000001e-05 2024-09-21T21:41:52.782 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.782 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.782 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:52.783 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.263208+0000", 2024-09-21T21:41:52.783 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000106723 2024-09-21T21:41:52.783 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:52.783 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:52.783 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:52.783 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.783 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.783 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3452 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 3133440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:41:52.783 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.207699+0000", 2024-09-21T21:41:52.783 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 5.4932483999999997, 2024-09-21T21:41:52.783 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055702591000000003, 2024-09-21T21:41:52.784 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:52.784 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:52.784 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:52.784 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:52.784 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:41:52.784 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:52.784 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3452 2024-09-21T21:41:52.784 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.784 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:52.784 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.784 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:52.784 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207699+0000", 2024-09-21T21:41:52.784 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:52.785 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.785 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.785 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:52.785 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207699+0000", 2024-09-21T21:41:52.785 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:52.785 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.785 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.785 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:52.785 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207813+0000", 2024-09-21T21:41:52.785 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000114355 2024-09-21T21:41:52.785 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.785 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.786 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:52.786 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207826+0000", 2024-09-21T21:41:52.786 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.3293e-05 2024-09-21T21:41:52.786 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.786 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.786 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:52.786 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207835+0000", 2024-09-21T21:41:52.786 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.8310000000000007e-06 2024-09-21T21:41:52.786 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.786 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.786 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:52.786 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207842+0000", 2024-09-21T21:41:52.787 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5150000000000003e-06 2024-09-21T21:41:52.787 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.787 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.787 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:52.787 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.263240+0000", 2024-09-21T21:41:52.787 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055398007999999999 2024-09-21T21:41:52.787 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.787 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.787 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:52.787 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.263315+0000", 2024-09-21T21:41:52.787 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5729999999999995e-05 2024-09-21T21:41:52.788 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.788 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.788 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:52.788 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.263401+0000", 2024-09-21T21:41:52.788 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.5859000000000006e-05 2024-09-21T21:41:52.788 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:52.788 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:52.788 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:52.788 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.788 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.788 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3455 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 3821568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:41:52.788 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.207852+0000", 2024-09-21T21:41:52.789 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 5.4930955260000003, 2024-09-21T21:41:52.789 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055704138, 2024-09-21T21:41:52.789 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:52.789 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:52.789 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:52.789 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:52.789 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:41:52.789 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:52.789 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3455 2024-09-21T21:41:52.789 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.789 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:52.789 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.790 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:52.790 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207852+0000", 2024-09-21T21:41:52.790 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:52.790 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.790 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.790 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:52.790 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207852+0000", 2024-09-21T21:41:52.790 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:52.790 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.790 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.790 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:52.790 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207983+0000", 2024-09-21T21:41:52.791 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00013183199999999999 2024-09-21T21:41:52.791 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.791 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.791 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:52.791 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207988+0000", 2024-09-21T21:41:52.791 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4290000000000003e-06 2024-09-21T21:41:52.791 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.791 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.791 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:52.791 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.208002+0000", 2024-09-21T21:41:52.791 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.4156e-05 2024-09-21T21:41:52.791 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.792 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.792 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:52.792 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.208009+0000", 2024-09-21T21:41:52.792 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.8220000000000001e-06 2024-09-21T21:41:52.792 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.792 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.792 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:52.792 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.263444+0000", 2024-09-21T21:41:52.792 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055435607999999997 2024-09-21T21:41:52.792 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.792 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.792 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:52.793 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.263493+0000", 2024-09-21T21:41:52.793 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8986999999999997e-05 2024-09-21T21:41:52.793 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.793 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.793 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:52.793 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.263556+0000", 2024-09-21T21:41:52.793 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2304000000000005e-05 2024-09-21T21:41:52.793 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:52.793 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:52.793 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:52.793 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.793 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.794 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3456 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 3850240~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:41:52.794 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.208025+0000", 2024-09-21T21:41:52.794 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 5.4929223990000002, 2024-09-21T21:41:52.794 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055714048000000002, 2024-09-21T21:41:52.794 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:52.794 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:52.794 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:52.794 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:52.794 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:41:52.794 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:52.794 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3456 2024-09-21T21:41:52.794 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.795 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:52.795 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.795 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:52.795 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.208025+0000", 2024-09-21T21:41:52.795 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:52.795 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.795 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.795 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:52.795 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.208025+0000", 2024-09-21T21:41:52.795 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:52.795 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.796 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.796 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:52.796 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.208264+0000", 2024-09-21T21:41:52.796 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00023975999999999999 2024-09-21T21:41:52.796 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.796 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.796 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:52.796 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.208269+0000", 2024-09-21T21:41:52.796 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7790000000000003e-06 2024-09-21T21:41:52.796 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.796 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.797 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:52.797 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.208278+0000", 2024-09-21T21:41:52.797 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2110000000000004e-06 2024-09-21T21:41:52.797 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.797 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.797 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:52.797 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.208286+0000", 2024-09-21T21:41:52.797 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0899999999999999e-06 2024-09-21T21:41:52.797 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.797 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.797 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:52.798 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.263583+0000", 2024-09-21T21:41:52.798 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055297330999999998 2024-09-21T21:41:52.798 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.798 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.798 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:52.798 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.263647+0000", 2024-09-21T21:41:52.798 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4654999999999997e-05 2024-09-21T21:41:52.798 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.798 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.798 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:52.798 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.263739+0000", 2024-09-21T21:41:52.798 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.1222000000000002e-05 2024-09-21T21:41:52.799 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:52.799 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:52.799 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:52.799 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.799 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.799 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3468 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 266240~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:41:52.799 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.208298+0000", 2024-09-21T21:41:52.799 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 5.4926492380000003, 2024-09-21T21:41:52.799 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055668849999999999, 2024-09-21T21:41:52.799 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:52.799 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:52.799 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:52.800 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:52.800 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:41:52.800 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:52.800 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3468 2024-09-21T21:41:52.800 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.800 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:52.800 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.800 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:52.800 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.208298+0000", 2024-09-21T21:41:52.800 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:52.800 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.800 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.801 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:52.801 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.208298+0000", 2024-09-21T21:41:52.801 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:52.801 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.801 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.801 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:52.801 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.208465+0000", 2024-09-21T21:41:52.801 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00016740700000000001 2024-09-21T21:41:52.801 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.801 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.801 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:52.801 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.208484+0000", 2024-09-21T21:41:52.801 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.8963000000000001e-05 2024-09-21T21:41:52.802 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.802 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.802 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:52.802 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.208500+0000", 2024-09-21T21:41:52.802 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.6013999999999999e-05 2024-09-21T21:41:52.802 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.802 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.802 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:52.802 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.208508+0000", 2024-09-21T21:41:52.802 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.4279999999999999e-06 2024-09-21T21:41:52.802 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.802 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.803 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:52.803 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.263780+0000", 2024-09-21T21:41:52.803 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.05527232 2024-09-21T21:41:52.803 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.803 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.803 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:52.803 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.263863+0000", 2024-09-21T21:41:52.803 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.2723000000000001e-05 2024-09-21T21:41:52.803 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.803 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.803 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:52.804 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.263967+0000", 2024-09-21T21:41:52.804 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000103995 2024-09-21T21:41:52.804 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:52.804 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:52.804 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:52.804 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.804 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.804 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3470 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 294912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:41:52.804 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.208525+0000", 2024-09-21T21:41:52.804 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 5.4924216189999999, 2024-09-21T21:41:52.804 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055651946000000001, 2024-09-21T21:41:52.804 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:52.804 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:52.805 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:52.805 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:52.805 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:41:52.805 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:52.805 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3470 2024-09-21T21:41:52.805 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.805 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:52.805 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.805 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:52.805 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.208525+0000", 2024-09-21T21:41:52.805 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:52.805 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.806 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.806 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:52.806 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.208525+0000", 2024-09-21T21:41:52.806 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:52.806 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.806 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.806 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:52.806 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.209091+0000", 2024-09-21T21:41:52.806 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00056559299999999998 2024-09-21T21:41:52.806 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.806 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.806 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:52.807 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.209096+0000", 2024-09-21T21:41:52.807 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.882e-06 2024-09-21T21:41:52.807 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.807 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.807 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:52.807 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.209106+0000", 2024-09-21T21:41:52.807 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.8900000000000002e-06 2024-09-21T21:41:52.807 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.807 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.807 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:52.807 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.209114+0000", 2024-09-21T21:41:52.807 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.6580000000000008e-06 2024-09-21T21:41:52.807 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.808 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.808 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:52.808 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.264030+0000", 2024-09-21T21:41:52.808 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.054915692000000002 2024-09-21T21:41:52.808 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.808 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.808 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:52.808 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.264107+0000", 2024-09-21T21:41:52.808 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.6860000000000006e-05 2024-09-21T21:41:52.808 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:52.808 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:52.808 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:52.808 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.264177+0000", 2024-09-21T21:41:52.809 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0370999999999994e-05 2024-09-21T21:41:52.809 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:52.809 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:52.809 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:52.809 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:52.809 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:52.809 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:52.810 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:52.809+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:52.811 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T21:41:52.812 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0: {} 2024-09-21T21:41:52.812 INFO:teuthology.orchestra.run.smithi081.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T21:41:52.812 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1: {} 2024-09-21T21:41:52.812 INFO:teuthology.orchestra.run.smithi081.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T21:41:52.813 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2: {} 2024-09-21T21:41:52.813 INFO:teuthology.orchestra.run.smithi081.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T21:41:52.813 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3: {} 2024-09-21T21:41:52.814 INFO:teuthology.orchestra.run.smithi081.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T21:41:52.825 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4: {} 2024-09-21T21:41:52.826 INFO:teuthology.orchestra.run.smithi081.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T21:41:52.843 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5: {} 2024-09-21T21:41:52.843 INFO:teuthology.orchestra.run.smithi081.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T21:41:52.911 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:52.910+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:52.929 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:52.929 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:52.938 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T21:41:53.012 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:53.010+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:53.041 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:53.041 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:53.050 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T21:41:53.135 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:53.134+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:53.193 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:53.194 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:53.202 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T21:41:53.246 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:53.245+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:53.266 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 5, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-21T21:41:53.266 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T21:41:53.266 INFO:tasks.thrashosds.thrasher:inject_pause on osd.3 2024-09-21T21:41:53.266 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-21T21:41:53.266 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-21T21:41:53.266 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok config set filestore_inject_stall 3 2024-09-21T21:41:53.313 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:53.311+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:53.320 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:53.320 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:53.330 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T21:41:53.375 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:53.375 INFO:teuthology.orchestra.run.smithi122.stdout: "success": "" 2024-09-21T21:41:53.375 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:53.413 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:53.412+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:53.432 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T21:41:53.472 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:53.473 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:53.481 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T21:41:53.514 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:53.513+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:53.584 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:53.585 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:53.593 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T21:41:53.615 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:53.614+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:53.696 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:53.696 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:53.705 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T21:41:53.748 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:53.747+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:53.809 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:53.814 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:53.817 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T21:41:53.818 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:53.816+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:53.918 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:53.916+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:53.962 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:53.962 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:53.970 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T21:41:54.055 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:54.053+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:54.114 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:54.114 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:54.119 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:54.118+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:54.122 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T21:41:54.207 INFO:teuthology.orchestra.run.smithi081.stderr:noscrub is unset 2024-09-21T21:41:54.220 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:54.218+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:54.270 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:54.270 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:54.279 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T21:41:54.321 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:54.319+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:54.421 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:54.420+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:54.426 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:54.427 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:54.436 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T21:41:54.449 INFO:teuthology.orchestra.run.smithi164.stdout:checking clone #29, image image_client.0-clone29 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent30 2024-09-21T21:41:54.522 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:54.520+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:54.554 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:54.554 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:54.564 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T21:41:54.648 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:54.647+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:54.724 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:54.724 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:54.724 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:54.722+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:54.733 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T21:41:54.825 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:54.823+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:54.855 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T21:41:54.899 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:54.899 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:54.911 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T21:41:54.955 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:54.954+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:55.002 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0: {} 2024-09-21T21:41:55.003 INFO:teuthology.orchestra.run.smithi081.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T21:41:55.022 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1: {} 2024-09-21T21:41:55.022 INFO:teuthology.orchestra.run.smithi081.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T21:41:55.026 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:55.025+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:55.037 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:55.037 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:41:55.037 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 0 2024-09-21T21:41:55.037 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:55.043 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2: {} 2024-09-21T21:41:55.043 INFO:teuthology.orchestra.run.smithi081.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T21:41:55.048 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T21:41:55.064 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3: {} 2024-09-21T21:41:55.064 INFO:teuthology.orchestra.run.smithi081.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T21:41:55.082 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4: {} 2024-09-21T21:41:55.082 INFO:teuthology.orchestra.run.smithi081.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T21:41:55.101 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5: {} 2024-09-21T21:41:55.102 INFO:teuthology.orchestra.run.smithi081.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T21:41:55.155 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:55.154+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:55.170 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:55.170 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:41:55.170 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:41:55.170 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:41:55.170 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:55.179 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T21:41:55.227 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:55.226+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:55.278 INFO:teuthology.orchestra.run.smithi164.stdout:3913 trunc from 0x95d2c00 to 0x14aa400 2024-09-21T21:41:55.329 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:55.327+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:55.336 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:55.337 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 20, 2024-09-21T21:41:55.337 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 600, 2024-09-21T21:41:55.337 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:41:55.337 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.337 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4824.0:1564 3.2 3:4647520c:::rbd_data.1121b97f2925.0000000000000006:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:55.337 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:41:55.337 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 53.698000458999999, 2024-09-21T21:41:55.337 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056097347999999998, 2024-09-21T21:41:55.337 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:55.337 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:55.337 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:55.337 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:55.338 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4824", 2024-09-21T21:41:55.338 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:55.338 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1564 2024-09-21T21:41:55.338 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.338 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:55.338 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.338 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:55.338 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:41:55.338 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:55.338 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.338 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.339 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:55.339 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:41:55.339 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:55.339 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.339 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.339 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:55.339 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:41:55.339 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6499999999999999e-07 2024-09-21T21:41:55.339 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.339 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.339 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:55.339 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636067+0000", 2024-09-21T21:41:55.340 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3879999999999998e-06 2024-09-21T21:41:55.340 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.340 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.340 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:55.340 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636075+0000", 2024-09-21T21:41:55.340 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5480000000000004e-06 2024-09-21T21:41:55.340 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.340 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.340 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:55.340 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636080+0000", 2024-09-21T21:41:55.340 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.339e-06 2024-09-21T21:41:55.340 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.341 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.341 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:55.341 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636133+0000", 2024-09-21T21:41:55.341 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2930000000000003e-05 2024-09-21T21:41:55.341 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.341 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.341 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:55.341 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636265+0000", 2024-09-21T21:41:55.341 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000132058 2024-09-21T21:41:55.341 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.341 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.341 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:55.342 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.692162+0000", 2024-09-21T21:41:55.342 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055896620000000001 2024-09-21T21:41:55.342 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:55.342 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:55.342 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:55.342 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.342 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.342 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3258 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 1409024~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:41:55.342 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.196243+0000", 2024-09-21T21:41:55.342 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 8.1378216190000003, 2024-09-21T21:41:55.342 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055725496999999999, 2024-09-21T21:41:55.342 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:55.343 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:55.343 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:55.343 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:55.343 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:41:55.343 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:55.343 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3258 2024-09-21T21:41:55.343 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.343 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:55.343 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.343 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:55.343 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196243+0000", 2024-09-21T21:41:55.343 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:55.343 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.344 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.344 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:55.344 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196243+0000", 2024-09-21T21:41:55.344 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:55.344 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.344 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.344 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:55.344 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196244+0000", 2024-09-21T21:41:55.344 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4799999999999999e-07 2024-09-21T21:41:55.344 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.344 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.344 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:55.345 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196246+0000", 2024-09-21T21:41:55.345 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1390000000000002e-06 2024-09-21T21:41:55.345 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.345 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.345 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:55.345 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196255+0000", 2024-09-21T21:41:55.345 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.5779999999999999e-06 2024-09-21T21:41:55.345 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.345 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.345 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:55.345 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196267+0000", 2024-09-21T21:41:55.345 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.2726e-05 2024-09-21T21:41:55.346 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.346 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.346 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:55.346 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.251811+0000", 2024-09-21T21:41:55.346 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055544068000000002 2024-09-21T21:41:55.346 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.346 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.346 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:55.346 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.251878+0000", 2024-09-21T21:41:55.346 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6898999999999998e-05 2024-09-21T21:41:55.346 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.346 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.347 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:55.347 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.251969+0000", 2024-09-21T21:41:55.347 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.0638999999999997e-05 2024-09-21T21:41:55.347 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:55.347 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:55.347 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:55.347 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.347 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.347 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3259 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 663552~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:41:55.347 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.196275+0000", 2024-09-21T21:41:55.347 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 8.1377903109999998, 2024-09-21T21:41:55.347 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055889580000000001, 2024-09-21T21:41:55.348 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:55.348 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:55.348 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:55.348 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:55.348 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:41:55.348 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:55.348 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3259 2024-09-21T21:41:55.348 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.348 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:55.348 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.348 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:55.348 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196275+0000", 2024-09-21T21:41:55.349 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:55.349 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.349 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.349 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:55.349 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196275+0000", 2024-09-21T21:41:55.349 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:55.349 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.349 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.349 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:55.349 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196275+0000", 2024-09-21T21:41:55.349 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2199999999999999e-07 2024-09-21T21:41:55.349 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.350 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.350 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:55.350 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196277+0000", 2024-09-21T21:41:55.350 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1270000000000001e-06 2024-09-21T21:41:55.350 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.350 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.350 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:55.350 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196285+0000", 2024-09-21T21:41:55.350 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.2290000000000003e-06 2024-09-21T21:41:55.350 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.350 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.350 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:55.351 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196291+0000", 2024-09-21T21:41:55.351 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1379999999999998e-06 2024-09-21T21:41:55.351 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.351 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.351 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:55.351 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.251997+0000", 2024-09-21T21:41:55.351 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055705971999999999 2024-09-21T21:41:55.351 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.351 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.351 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:55.351 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252077+0000", 2024-09-21T21:41:55.351 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0179999999999995e-05 2024-09-21T21:41:55.352 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.352 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.352 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:55.352 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252164+0000", 2024-09-21T21:41:55.352 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.7511999999999998e-05 2024-09-21T21:41:55.352 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:55.352 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:55.352 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:55.352 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.352 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.352 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3260 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 1437696~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:41:55.352 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.196297+0000", 2024-09-21T21:41:55.352 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 8.1377682490000005, 2024-09-21T21:41:55.353 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056071022999999998, 2024-09-21T21:41:55.353 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:55.353 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:55.353 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:55.353 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:55.353 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:41:55.353 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:55.353 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3260 2024-09-21T21:41:55.353 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.353 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:55.353 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.354 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:55.354 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196297+0000", 2024-09-21T21:41:55.354 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:55.354 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.354 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.354 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:55.354 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196297+0000", 2024-09-21T21:41:55.354 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:55.354 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.354 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.354 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:55.354 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196918+0000", 2024-09-21T21:41:55.354 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00062148699999999999 2024-09-21T21:41:55.355 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.355 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.355 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:55.355 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196930+0000", 2024-09-21T21:41:55.355 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1657e-05 2024-09-21T21:41:55.355 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.355 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.355 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:55.355 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196944+0000", 2024-09-21T21:41:55.355 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.4549000000000001e-05 2024-09-21T21:41:55.355 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.355 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.356 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:55.356 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196951+0000", 2024-09-21T21:41:55.356 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6749999999999996e-06 2024-09-21T21:41:55.356 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.356 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.356 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:55.356 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252208+0000", 2024-09-21T21:41:55.356 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055256656000000001 2024-09-21T21:41:55.356 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.356 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.356 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:55.356 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252278+0000", 2024-09-21T21:41:55.356 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9882000000000006e-05 2024-09-21T21:41:55.357 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.357 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.357 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:55.357 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252368+0000", 2024-09-21T21:41:55.357 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.0117000000000005e-05 2024-09-21T21:41:55.357 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:55.357 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:55.357 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:55.357 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.357 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.357 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3261 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 2842624~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:41:55.357 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.196963+0000", 2024-09-21T21:41:55.358 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 8.1371018619999997, 2024-09-21T21:41:55.358 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055594613000000001, 2024-09-21T21:41:55.358 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:55.358 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:55.358 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:55.358 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:55.358 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:41:55.358 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:55.358 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3261 2024-09-21T21:41:55.358 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.358 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:55.358 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.358 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:55.359 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196963+0000", 2024-09-21T21:41:55.359 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:55.359 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.359 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.359 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:55.359 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196963+0000", 2024-09-21T21:41:55.359 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:55.359 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.359 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.359 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:55.359 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196964+0000", 2024-09-21T21:41:55.359 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3000000000000001e-07 2024-09-21T21:41:55.360 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.360 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.360 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:55.360 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196966+0000", 2024-09-21T21:41:55.360 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.475e-06 2024-09-21T21:41:55.360 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.360 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.360 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:55.360 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196973+0000", 2024-09-21T21:41:55.360 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5540000000000003e-06 2024-09-21T21:41:55.360 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.360 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.360 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:55.361 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196978+0000", 2024-09-21T21:41:55.361 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0719999999999999e-06 2024-09-21T21:41:55.361 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.361 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.361 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:55.361 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252395+0000", 2024-09-21T21:41:55.361 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055417301000000002 2024-09-21T21:41:55.361 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.361 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.361 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:55.361 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252468+0000", 2024-09-21T21:41:55.361 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.2692000000000001e-05 2024-09-21T21:41:55.361 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.362 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.362 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:55.362 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252558+0000", 2024-09-21T21:41:55.362 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.9988999999999995e-05 2024-09-21T21:41:55.362 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:55.362 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:55.362 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:55.362 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.362 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.362 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3262 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 2871296~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:41:55.362 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.196985+0000", 2024-09-21T21:41:55.363 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 8.1370803699999996, 2024-09-21T21:41:55.363 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055777262000000001, 2024-09-21T21:41:55.363 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:55.363 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:55.363 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:55.363 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:55.363 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:41:55.363 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:55.363 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3262 2024-09-21T21:41:55.363 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.363 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:55.363 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.364 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:55.364 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196985+0000", 2024-09-21T21:41:55.364 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:55.364 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.364 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.364 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:55.364 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196985+0000", 2024-09-21T21:41:55.364 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:55.364 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.364 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.364 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:55.364 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197024+0000", 2024-09-21T21:41:55.364 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.9558000000000003e-05 2024-09-21T21:41:55.365 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.365 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.365 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:55.365 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197026+0000", 2024-09-21T21:41:55.365 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0899999999999999e-06 2024-09-21T21:41:55.365 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.365 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.365 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:55.365 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197031+0000", 2024-09-21T21:41:55.365 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8849999999999998e-06 2024-09-21T21:41:55.365 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.365 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.365 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:55.366 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197035+0000", 2024-09-21T21:41:55.366 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5719999999999999e-06 2024-09-21T21:41:55.366 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.366 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.366 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:55.366 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252587+0000", 2024-09-21T21:41:55.366 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055552193 2024-09-21T21:41:55.366 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.366 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.366 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:55.366 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252670+0000", 2024-09-21T21:41:55.367 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.2687000000000003e-05 2024-09-21T21:41:55.367 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.367 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.367 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:55.367 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252762+0000", 2024-09-21T21:41:55.367 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2276999999999998e-05 2024-09-21T21:41:55.367 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:55.367 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:55.367 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:55.367 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.367 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.367 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3263 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 3559424~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:41:55.368 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.197041+0000", 2024-09-21T21:41:55.368 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 8.1370244350000007, 2024-09-21T21:41:55.368 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055935804999999998, 2024-09-21T21:41:55.368 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:55.368 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:55.368 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:55.368 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:55.368 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:41:55.368 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:55.368 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3263 2024-09-21T21:41:55.368 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.368 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:55.369 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.369 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:55.369 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197041+0000", 2024-09-21T21:41:55.369 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:55.369 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.369 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.369 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:55.369 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197041+0000", 2024-09-21T21:41:55.369 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:55.369 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.369 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.369 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:55.370 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197624+0000", 2024-09-21T21:41:55.370 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00058358100000000001 2024-09-21T21:41:55.370 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.370 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.370 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:55.370 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197629+0000", 2024-09-21T21:41:55.370 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5909999999999998e-06 2024-09-21T21:41:55.370 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.370 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.370 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:55.370 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197639+0000", 2024-09-21T21:41:55.370 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0257e-05 2024-09-21T21:41:55.371 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.371 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.371 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:55.371 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197646+0000", 2024-09-21T21:41:55.371 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.7229999999999998e-06 2024-09-21T21:41:55.371 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.371 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.371 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:55.371 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252804+0000", 2024-09-21T21:41:55.371 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055158134999999997 2024-09-21T21:41:55.371 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.371 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.371 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:55.372 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252882+0000", 2024-09-21T21:41:55.372 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7791000000000001e-05 2024-09-21T21:41:55.372 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.372 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.372 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:55.372 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252976+0000", 2024-09-21T21:41:55.372 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.4727000000000003e-05 2024-09-21T21:41:55.372 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:55.372 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:55.372 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:55.372 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.372 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.372 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3325 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:41:55.373 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.197722+0000", 2024-09-21T21:41:55.373 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 8.1363426180000005, 2024-09-21T21:41:55.373 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055712048, 2024-09-21T21:41:55.373 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:55.373 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:55.373 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:55.373 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:55.373 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:41:55.373 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:55.373 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3325 2024-09-21T21:41:55.373 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.373 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:55.374 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.374 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:55.374 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197722+0000", 2024-09-21T21:41:55.374 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:55.374 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.374 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.374 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:55.374 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197722+0000", 2024-09-21T21:41:55.374 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:55.374 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.374 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.374 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:55.374 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.198052+0000", 2024-09-21T21:41:55.375 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00033006600000000001 2024-09-21T21:41:55.375 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.375 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.375 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:55.375 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.198056+0000", 2024-09-21T21:41:55.375 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7330000000000001e-06 2024-09-21T21:41:55.375 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.375 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.375 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:55.375 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.198065+0000", 2024-09-21T21:41:55.375 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.8990000000000007e-06 2024-09-21T21:41:55.375 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.376 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.376 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:55.376 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.198072+0000", 2024-09-21T21:41:55.376 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5339999999999996e-06 2024-09-21T21:41:55.376 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.376 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.376 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:55.376 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.253244+0000", 2024-09-21T21:41:55.376 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055172064999999999 2024-09-21T21:41:55.376 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.376 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.376 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:55.377 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.253342+0000", 2024-09-21T21:41:55.377 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.8175000000000001e-05 2024-09-21T21:41:55.377 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.377 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.377 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:55.377 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.253434+0000", 2024-09-21T21:41:55.377 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2576000000000003e-05 2024-09-21T21:41:55.377 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:55.377 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:55.377 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:55.377 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.377 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.378 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3437 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 925696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:41:55.378 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.205888+0000", 2024-09-21T21:41:55.378 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 8.1281771539999994, 2024-09-21T21:41:55.378 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055953808000000001, 2024-09-21T21:41:55.378 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:55.378 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:55.378 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:55.378 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:55.378 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:41:55.378 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:55.378 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3437 2024-09-21T21:41:55.378 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.378 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:55.379 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.379 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:55.379 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205888+0000", 2024-09-21T21:41:55.379 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:55.379 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.379 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.379 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:55.379 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205888+0000", 2024-09-21T21:41:55.379 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:55.379 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.379 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.379 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:55.380 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205888+0000", 2024-09-21T21:41:55.380 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.75e-07 2024-09-21T21:41:55.380 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.380 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.380 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:55.380 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205891+0000", 2024-09-21T21:41:55.380 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3750000000000001e-06 2024-09-21T21:41:55.380 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.380 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.380 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:55.380 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205897+0000", 2024-09-21T21:41:55.380 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.285e-06 2024-09-21T21:41:55.380 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.381 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.381 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:55.381 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205902+0000", 2024-09-21T21:41:55.381 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6290000000000001e-06 2024-09-21T21:41:55.381 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.381 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.381 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:55.381 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261686+0000", 2024-09-21T21:41:55.381 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055784586999999997 2024-09-21T21:41:55.381 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.381 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.381 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:55.381 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261764+0000", 2024-09-21T21:41:55.382 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7591999999999998e-05 2024-09-21T21:41:55.382 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.382 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.382 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:55.382 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261842+0000", 2024-09-21T21:41:55.382 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7865000000000001e-05 2024-09-21T21:41:55.382 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:55.382 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:55.382 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:55.382 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.382 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.382 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3438 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 954368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:41:55.383 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.205908+0000", 2024-09-21T21:41:55.383 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 8.1281572890000007, 2024-09-21T21:41:55.383 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056083169000000002, 2024-09-21T21:41:55.383 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:55.383 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:55.383 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:55.383 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:55.383 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:41:55.383 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:55.383 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3438 2024-09-21T21:41:55.383 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.383 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:55.384 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.384 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:55.384 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205908+0000", 2024-09-21T21:41:55.384 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:55.384 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.384 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.384 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:55.384 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205908+0000", 2024-09-21T21:41:55.384 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:55.384 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.384 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.384 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:55.385 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206431+0000", 2024-09-21T21:41:55.385 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00052282100000000003 2024-09-21T21:41:55.385 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.385 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.385 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:55.385 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206442+0000", 2024-09-21T21:41:55.385 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1695000000000001e-05 2024-09-21T21:41:55.385 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.385 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.385 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:55.385 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206452+0000", 2024-09-21T21:41:55.385 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.4560000000000006e-06 2024-09-21T21:41:55.385 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.386 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.386 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:55.386 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206458+0000", 2024-09-21T21:41:55.386 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4670000000000001e-06 2024-09-21T21:41:55.386 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.386 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.386 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:55.386 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261871+0000", 2024-09-21T21:41:55.386 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055412712000000003 2024-09-21T21:41:55.386 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.386 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.387 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:55.387 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261928+0000", 2024-09-21T21:41:55.387 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6631e-05 2024-09-21T21:41:55.387 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.387 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.387 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:55.387 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261991+0000", 2024-09-21T21:41:55.387 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3386999999999995e-05 2024-09-21T21:41:55.387 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:55.387 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:55.387 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:55.388 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.388 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.388 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3440 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 1642496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:41:55.388 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.206491+0000", 2024-09-21T21:41:55.388 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 8.1275741099999994, 2024-09-21T21:41:55.388 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055633136, 2024-09-21T21:41:55.388 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:55.388 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:55.388 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:55.388 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:55.388 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:41:55.388 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:55.388 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3440 2024-09-21T21:41:55.389 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.389 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:55.389 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.389 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:55.389 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206491+0000", 2024-09-21T21:41:55.389 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:55.389 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.389 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.389 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:55.389 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206491+0000", 2024-09-21T21:41:55.389 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:55.389 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.390 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.390 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:55.390 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206491+0000", 2024-09-21T21:41:55.390 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6100000000000001e-07 2024-09-21T21:41:55.390 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.390 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.390 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:55.390 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206494+0000", 2024-09-21T21:41:55.390 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.9349999999999999e-06 2024-09-21T21:41:55.390 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.390 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.391 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:55.391 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206501+0000", 2024-09-21T21:41:55.391 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6939999999999998e-06 2024-09-21T21:41:55.391 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.391 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.391 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:55.391 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206506+0000", 2024-09-21T21:41:55.391 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2979999999999999e-06 2024-09-21T21:41:55.391 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.391 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.391 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:55.391 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262019+0000", 2024-09-21T21:41:55.392 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055513094999999998 2024-09-21T21:41:55.392 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.392 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.392 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:55.392 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262073+0000", 2024-09-21T21:41:55.392 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3486999999999998e-05 2024-09-21T21:41:55.392 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.392 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.392 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:55.392 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262124+0000", 2024-09-21T21:41:55.392 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1066000000000002e-05 2024-09-21T21:41:55.393 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:55.393 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:55.393 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:55.393 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.393 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.393 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3442 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 1671168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:41:55.393 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.206515+0000", 2024-09-21T21:41:55.393 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 8.1275504079999994, 2024-09-21T21:41:55.393 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055792680999999997, 2024-09-21T21:41:55.393 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:55.393 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:55.393 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:55.394 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:55.394 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:41:55.394 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:55.394 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3442 2024-09-21T21:41:55.394 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.394 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:55.394 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.394 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:55.394 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206515+0000", 2024-09-21T21:41:55.394 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:55.394 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.394 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.395 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:55.395 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206515+0000", 2024-09-21T21:41:55.395 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:55.395 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.395 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.395 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:55.395 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206515+0000", 2024-09-21T21:41:55.395 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2500000000000001e-07 2024-09-21T21:41:55.395 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.395 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.395 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:55.395 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206517+0000", 2024-09-21T21:41:55.396 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0420000000000001e-06 2024-09-21T21:41:55.396 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.396 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.396 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:55.396 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206522+0000", 2024-09-21T21:41:55.396 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3600000000000004e-06 2024-09-21T21:41:55.396 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.396 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.396 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:55.396 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206527+0000", 2024-09-21T21:41:55.396 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.809e-06 2024-09-21T21:41:55.396 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.397 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.397 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:55.397 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262151+0000", 2024-09-21T21:41:55.397 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055623458000000001 2024-09-21T21:41:55.397 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.397 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.397 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:55.397 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262210+0000", 2024-09-21T21:41:55.397 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9120999999999999e-05 2024-09-21T21:41:55.397 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.397 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.397 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:55.398 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262307+0000", 2024-09-21T21:41:55.398 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.7466000000000005e-05 2024-09-21T21:41:55.398 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:55.398 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:55.398 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:55.398 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.398 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.398 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3444 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 2359296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:41:55.398 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.206533+0000", 2024-09-21T21:41:55.398 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 8.1275317690000008, 2024-09-21T21:41:55.398 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055974239000000002, 2024-09-21T21:41:55.399 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:55.399 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:55.399 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:55.399 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:55.399 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:41:55.399 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:55.399 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3444 2024-09-21T21:41:55.399 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.399 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:55.399 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.399 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:55.400 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206533+0000", 2024-09-21T21:41:55.400 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:55.400 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.400 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.400 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:55.400 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206533+0000", 2024-09-21T21:41:55.400 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:55.400 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.400 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.400 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:55.400 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206710+0000", 2024-09-21T21:41:55.400 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000176293 2024-09-21T21:41:55.400 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.401 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.401 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:55.401 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206714+0000", 2024-09-21T21:41:55.401 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1989999999999999e-06 2024-09-21T21:41:55.401 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.401 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.401 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:55.401 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206723+0000", 2024-09-21T21:41:55.401 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.9199999999999993e-06 2024-09-21T21:41:55.401 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.401 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.401 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:55.401 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206729+0000", 2024-09-21T21:41:55.401 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0739999999999998e-06 2024-09-21T21:41:55.402 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.402 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.402 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:55.402 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262339+0000", 2024-09-21T21:41:55.402 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055610166000000003 2024-09-21T21:41:55.402 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.402 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.402 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:55.402 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262419+0000", 2024-09-21T21:41:55.402 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0432999999999994e-05 2024-09-21T21:41:55.402 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.402 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.403 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:55.403 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262507+0000", 2024-09-21T21:41:55.403 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.8153999999999997e-05 2024-09-21T21:41:55.403 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:55.403 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:55.403 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:55.403 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.403 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.403 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3445 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 1699840~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:41:55.403 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.206746+0000", 2024-09-21T21:41:55.403 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 8.1273188449999996, 2024-09-21T21:41:55.403 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055880738999999999, 2024-09-21T21:41:55.404 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:55.404 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:55.404 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:55.404 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:55.404 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:41:55.404 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:55.404 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3445 2024-09-21T21:41:55.404 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.404 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:55.404 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.404 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:55.404 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206746+0000", 2024-09-21T21:41:55.405 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:55.405 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.405 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.405 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:55.405 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206746+0000", 2024-09-21T21:41:55.405 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:55.405 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.405 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.405 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:55.405 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207412+0000", 2024-09-21T21:41:55.405 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00066611999999999997 2024-09-21T21:41:55.405 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.406 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.406 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:55.406 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207417+0000", 2024-09-21T21:41:55.406 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8539999999999996e-06 2024-09-21T21:41:55.406 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.406 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.406 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:55.406 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207437+0000", 2024-09-21T21:41:55.406 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9463e-05 2024-09-21T21:41:55.406 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.406 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.406 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:55.407 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207445+0000", 2024-09-21T21:41:55.407 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.5790000000000004e-06 2024-09-21T21:41:55.407 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.407 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.407 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:55.407 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262537+0000", 2024-09-21T21:41:55.407 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055091531999999999 2024-09-21T21:41:55.407 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.407 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.407 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:55.407 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262578+0000", 2024-09-21T21:41:55.407 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1220000000000002e-05 2024-09-21T21:41:55.408 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.408 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.408 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:55.408 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262627+0000", 2024-09-21T21:41:55.408 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8971000000000003e-05 2024-09-21T21:41:55.408 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:55.408 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:55.408 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:55.408 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.408 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.408 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3451 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 3104768~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:41:55.409 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.207509+0000", 2024-09-21T21:41:55.409 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 8.1265557800000003, 2024-09-21T21:41:55.409 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055698944, 2024-09-21T21:41:55.409 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:55.409 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:55.409 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:55.409 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:55.409 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:41:55.409 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:55.409 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3451 2024-09-21T21:41:55.409 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.409 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:55.410 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.410 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:55.410 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207509+0000", 2024-09-21T21:41:55.410 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:55.410 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.410 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.410 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:55.410 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207509+0000", 2024-09-21T21:41:55.410 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:55.411 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.411 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.411 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:55.411 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207661+0000", 2024-09-21T21:41:55.411 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00015181799999999999 2024-09-21T21:41:55.411 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.411 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.411 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:55.411 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207665+0000", 2024-09-21T21:41:55.411 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6380000000000002e-06 2024-09-21T21:41:55.411 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.411 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.412 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:55.412 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207674+0000", 2024-09-21T21:41:55.412 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.0950000000000002e-06 2024-09-21T21:41:55.412 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.412 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.412 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:55.412 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207689+0000", 2024-09-21T21:41:55.412 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.4904e-05 2024-09-21T21:41:55.412 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.412 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.412 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:55.413 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.263055+0000", 2024-09-21T21:41:55.413 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055366268000000003 2024-09-21T21:41:55.413 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.413 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.413 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:55.413 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.263101+0000", 2024-09-21T21:41:55.413 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6498000000000001e-05 2024-09-21T21:41:55.413 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.413 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.413 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:55.413 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.263208+0000", 2024-09-21T21:41:55.413 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000106723 2024-09-21T21:41:55.414 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:55.414 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:55.414 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:55.414 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.414 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.414 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3452 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 3133440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:41:55.414 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.207699+0000", 2024-09-21T21:41:55.414 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 8.1263663509999997, 2024-09-21T21:41:55.414 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055702591000000003, 2024-09-21T21:41:55.414 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:55.414 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:55.414 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:55.415 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:55.415 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:41:55.415 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:55.415 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3452 2024-09-21T21:41:55.415 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.415 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:55.415 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.415 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:55.415 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207699+0000", 2024-09-21T21:41:55.415 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:55.415 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.415 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.416 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:55.416 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207699+0000", 2024-09-21T21:41:55.416 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:55.416 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.416 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.416 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:55.416 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207813+0000", 2024-09-21T21:41:55.416 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000114355 2024-09-21T21:41:55.416 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.416 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.416 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:55.416 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207826+0000", 2024-09-21T21:41:55.417 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.3293e-05 2024-09-21T21:41:55.417 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.417 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.417 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:55.417 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207835+0000", 2024-09-21T21:41:55.417 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.8310000000000007e-06 2024-09-21T21:41:55.417 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.417 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.417 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:55.417 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207842+0000", 2024-09-21T21:41:55.417 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5150000000000003e-06 2024-09-21T21:41:55.417 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.418 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.418 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:55.418 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.263240+0000", 2024-09-21T21:41:55.418 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055398007999999999 2024-09-21T21:41:55.418 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.418 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.418 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:55.418 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.263315+0000", 2024-09-21T21:41:55.418 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5729999999999995e-05 2024-09-21T21:41:55.418 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.418 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.419 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:55.419 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.263401+0000", 2024-09-21T21:41:55.419 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.5859000000000006e-05 2024-09-21T21:41:55.419 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:55.419 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:55.419 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:55.419 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.419 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.419 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3455 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 3821568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:41:55.419 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.207852+0000", 2024-09-21T21:41:55.419 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 8.1262134770000003, 2024-09-21T21:41:55.420 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055704138, 2024-09-21T21:41:55.420 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:55.420 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:55.420 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:55.420 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:55.420 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:41:55.420 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:55.420 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3455 2024-09-21T21:41:55.420 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.420 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:55.420 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.420 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:55.421 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207852+0000", 2024-09-21T21:41:55.421 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:55.421 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.421 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.421 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:55.421 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207852+0000", 2024-09-21T21:41:55.421 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:55.421 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.421 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.421 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:55.421 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207983+0000", 2024-09-21T21:41:55.421 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00013183199999999999 2024-09-21T21:41:55.422 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.422 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.422 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:55.422 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207988+0000", 2024-09-21T21:41:55.422 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4290000000000003e-06 2024-09-21T21:41:55.422 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.422 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.422 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:55.422 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.208002+0000", 2024-09-21T21:41:55.422 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.4156e-05 2024-09-21T21:41:55.422 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.422 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.423 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:55.423 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.208009+0000", 2024-09-21T21:41:55.423 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.8220000000000001e-06 2024-09-21T21:41:55.423 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.423 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.423 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:55.423 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.263444+0000", 2024-09-21T21:41:55.423 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055435607999999997 2024-09-21T21:41:55.423 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.423 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.423 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:55.423 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.263493+0000", 2024-09-21T21:41:55.424 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8986999999999997e-05 2024-09-21T21:41:55.424 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.424 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.424 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:55.424 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.263556+0000", 2024-09-21T21:41:55.424 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2304000000000005e-05 2024-09-21T21:41:55.424 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:55.424 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:55.424 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:55.424 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.424 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.424 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3456 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 3850240~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:41:55.425 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.208025+0000", 2024-09-21T21:41:55.425 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 8.1260403500000002, 2024-09-21T21:41:55.425 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055714048000000002, 2024-09-21T21:41:55.425 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:55.425 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:55.425 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:55.425 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:55.425 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:41:55.425 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:55.425 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3456 2024-09-21T21:41:55.425 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.425 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:55.425 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.426 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:55.426 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.208025+0000", 2024-09-21T21:41:55.426 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:55.426 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.426 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.426 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:55.426 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.208025+0000", 2024-09-21T21:41:55.426 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:55.426 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.426 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.426 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:55.426 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.208264+0000", 2024-09-21T21:41:55.427 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00023975999999999999 2024-09-21T21:41:55.427 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.427 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.427 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:55.427 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.208269+0000", 2024-09-21T21:41:55.427 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7790000000000003e-06 2024-09-21T21:41:55.427 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.427 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.427 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:55.427 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.208278+0000", 2024-09-21T21:41:55.427 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2110000000000004e-06 2024-09-21T21:41:55.427 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.428 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.428 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:55.428 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.208286+0000", 2024-09-21T21:41:55.428 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0899999999999999e-06 2024-09-21T21:41:55.428 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.428 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.428 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:55.428 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.263583+0000", 2024-09-21T21:41:55.428 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055297330999999998 2024-09-21T21:41:55.428 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.428 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.428 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:55.429 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.263647+0000", 2024-09-21T21:41:55.429 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4654999999999997e-05 2024-09-21T21:41:55.429 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.429 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.429 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:55.429 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.263739+0000", 2024-09-21T21:41:55.429 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.1222000000000002e-05 2024-09-21T21:41:55.429 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:55.429 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:55.429 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:55.429 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.429 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.430 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3468 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 266240~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:41:55.430 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.208298+0000", 2024-09-21T21:41:55.430 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 8.1257671889999994, 2024-09-21T21:41:55.430 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055668849999999999, 2024-09-21T21:41:55.430 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:55.430 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:55.430 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:55.430 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:55.430 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:41:55.430 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:55.430 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3468 2024-09-21T21:41:55.430 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.430 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:55.431 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.431 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:55.431 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.208298+0000", 2024-09-21T21:41:55.431 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:55.431 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.431 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.431 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:55.431 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.208298+0000", 2024-09-21T21:41:55.431 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:55.431 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.431 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.431 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:55.432 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.208465+0000", 2024-09-21T21:41:55.432 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00016740700000000001 2024-09-21T21:41:55.432 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.432 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.432 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:55.432 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.208484+0000", 2024-09-21T21:41:55.432 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.8963000000000001e-05 2024-09-21T21:41:55.432 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.432 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.432 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:55.432 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.208500+0000", 2024-09-21T21:41:55.432 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.6013999999999999e-05 2024-09-21T21:41:55.433 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.433 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.433 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:55.433 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.208508+0000", 2024-09-21T21:41:55.433 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.4279999999999999e-06 2024-09-21T21:41:55.433 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.433 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.433 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:55.433 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.263780+0000", 2024-09-21T21:41:55.433 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.05527232 2024-09-21T21:41:55.433 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.433 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.434 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:55.434 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.263863+0000", 2024-09-21T21:41:55.434 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.2723000000000001e-05 2024-09-21T21:41:55.434 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.434 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.434 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:55.434 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.263967+0000", 2024-09-21T21:41:55.434 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000103995 2024-09-21T21:41:55.434 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:55.434 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:55.434 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:55.434 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.435 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.435 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3470 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 294912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:41:55.435 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.208525+0000", 2024-09-21T21:41:55.435 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 8.1255395700000008, 2024-09-21T21:41:55.435 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055651946000000001, 2024-09-21T21:41:55.435 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:55.435 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:55.435 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:55.435 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:55.435 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:41:55.435 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:55.435 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3470 2024-09-21T21:41:55.436 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.436 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:55.436 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.436 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:55.436 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.208525+0000", 2024-09-21T21:41:55.436 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:55.436 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.436 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.436 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:55.436 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.208525+0000", 2024-09-21T21:41:55.436 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:55.436 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.436 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.437 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:55.437 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.209091+0000", 2024-09-21T21:41:55.437 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00056559299999999998 2024-09-21T21:41:55.437 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.437 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.437 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:55.437 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.209096+0000", 2024-09-21T21:41:55.437 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.882e-06 2024-09-21T21:41:55.437 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.437 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.437 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:55.437 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.209106+0000", 2024-09-21T21:41:55.437 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.8900000000000002e-06 2024-09-21T21:41:55.437 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.438 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.438 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:55.438 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.209114+0000", 2024-09-21T21:41:55.438 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.6580000000000008e-06 2024-09-21T21:41:55.438 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.438 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.438 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:55.438 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.264030+0000", 2024-09-21T21:41:55.438 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.054915692000000002 2024-09-21T21:41:55.438 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.438 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.438 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:55.439 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.264107+0000", 2024-09-21T21:41:55.439 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.6860000000000006e-05 2024-09-21T21:41:55.439 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.439 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.439 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:55.439 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.264177+0000", 2024-09-21T21:41:55.439 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0370999999999994e-05 2024-09-21T21:41:55.439 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:55.439 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:55.439 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:55.439 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:55.439 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:55.440 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:55.442 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T21:41:55.442 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:55.440+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:55.443 INFO:teuthology.orchestra.run.smithi164.stdout:3914 write 0xd2d2a00 thru 0xd2e1180 (0xe781 bytes) 2024-09-21T21:41:55.443 INFO:teuthology.orchestra.run.smithi164.stdout:3920 punch from 0x8060400 to 0x8067e00, (0x7a00 bytes) 2024-09-21T21:41:55.443 INFO:teuthology.orchestra.run.smithi164.stdout:3922 read 0xd2bf600 thru 0xd2c73e2 (0x7de3 bytes) 2024-09-21T21:41:55.443 INFO:teuthology.orchestra.run.smithi164.stdout:3924 trunc from 0xd2e1200 to 0x7da1000 2024-09-21T21:41:55.444 INFO:teuthology.orchestra.run.smithi164.stdout:3926 punch from 0x71c1800 to 0x71d0c00, (0xf400 bytes) 2024-09-21T21:41:55.444 INFO:teuthology.orchestra.run.smithi164.stdout:3930 read 0x5daec00 thru 0x5db1574 (0x2975 bytes) 2024-09-21T21:41:55.444 INFO:teuthology.orchestra.run.smithi164.stdout:3937 read 0x78ce800 thru 0x78ceae5 (0x2e6 bytes) 2024-09-21T21:41:55.444 INFO:teuthology.orchestra.run.smithi164.stdout:3944 write 0x12b6200 thru 0x12c5f9e (0xfd9f bytes) 2024-09-21T21:41:55.444 INFO:teuthology.orchestra.run.smithi164.stdout:3945 read 0x3855800 thru 0x385c01a (0x681b bytes) 2024-09-21T21:41:55.444 INFO:teuthology.orchestra.run.smithi164.stdout:3949 write 0x16c400 thru 0x172808 (0x6409 bytes) 2024-09-21T21:41:55.444 INFO:teuthology.orchestra.run.smithi164.stdout:3951 read 0x23d8200 thru 0x23d9b43 (0x1944 bytes) 2024-09-21T21:41:55.444 INFO:teuthology.orchestra.run.smithi164.stdout:3953 clone 32 order 24 su 4194304 sc 8 2024-09-21T21:41:55.542 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:55.541+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:55.585 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:55.585 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:41:55.585 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 0 2024-09-21T21:41:55.585 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:55.594 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T21:41:55.643 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:55.642+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:55.697 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:55.697 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:41:55.697 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:41:55.697 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:41:55.697 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:55.705 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T21:41:55.744 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:55.743+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:55.808 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:55.808 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 20, 2024-09-21T21:41:55.809 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 600, 2024-09-21T21:41:55.809 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:41:55.809 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.809 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13295 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3330048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:55.809 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.796526+0000", 2024-09-21T21:41:55.809 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 5.0098690369999996, 2024-09-21T21:41:55.809 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071630928999999996, 2024-09-21T21:41:55.809 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:55.809 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:55.809 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:55.809 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:55.809 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:41:55.809 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:55.810 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13295 2024-09-21T21:41:55.810 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.810 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:55.810 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.810 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:55.810 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796526+0000", 2024-09-21T21:41:55.810 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:55.810 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.810 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.810 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:55.810 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796526+0000", 2024-09-21T21:41:55.810 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:55.810 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.810 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.811 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:55.811 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796526+0000", 2024-09-21T21:41:55.811 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5900000000000003e-07 2024-09-21T21:41:55.811 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.811 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.811 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:55.811 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796529+0000", 2024-09-21T21:41:55.811 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3910000000000001e-06 2024-09-21T21:41:55.811 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.811 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.811 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:55.811 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796537+0000", 2024-09-21T21:41:55.811 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.2819999999999996e-06 2024-09-21T21:41:55.812 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.812 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.812 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:55.812 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796541+0000", 2024-09-21T21:41:55.812 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.9400000000000004e-06 2024-09-21T21:41:55.812 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.812 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.812 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:55.812 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.868008+0000", 2024-09-21T21:41:55.812 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071467586999999999 2024-09-21T21:41:55.812 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.812 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.812 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:55.813 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.868061+0000", 2024-09-21T21:41:55.813 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2349000000000003e-05 2024-09-21T21:41:55.813 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.813 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.813 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:55.813 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.868157+0000", 2024-09-21T21:41:55.813 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6020999999999994e-05 2024-09-21T21:41:55.813 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:55.813 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:55.813 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:55.813 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.813 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.813 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13296 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 4046848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:55.814 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.796643+0000", 2024-09-21T21:41:55.814 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 5.009752218, 2024-09-21T21:41:55.814 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071682019999999999, 2024-09-21T21:41:55.814 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:55.814 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:55.814 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:55.814 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:55.814 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:41:55.814 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:55.814 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13296 2024-09-21T21:41:55.814 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.814 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:55.814 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.815 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:55.815 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796643+0000", 2024-09-21T21:41:55.815 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:55.815 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.815 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.815 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:55.815 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796643+0000", 2024-09-21T21:41:55.815 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:55.815 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.815 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.815 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:55.815 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796643+0000", 2024-09-21T21:41:55.815 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.7199999999999998e-07 2024-09-21T21:41:55.816 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.816 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.816 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:55.816 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796647+0000", 2024-09-21T21:41:55.816 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6770000000000001e-06 2024-09-21T21:41:55.816 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.816 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.816 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:55.816 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796656+0000", 2024-09-21T21:41:55.816 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2059999999999996e-06 2024-09-21T21:41:55.816 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.816 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.816 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:55.816 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796664+0000", 2024-09-21T21:41:55.817 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7919999999999993e-06 2024-09-21T21:41:55.817 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.817 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.817 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:55.817 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.868186+0000", 2024-09-21T21:41:55.817 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071522273999999997 2024-09-21T21:41:55.817 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.817 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.817 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:55.817 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.868239+0000", 2024-09-21T21:41:55.817 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2787999999999997e-05 2024-09-21T21:41:55.817 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.817 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.818 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:55.818 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.868325+0000", 2024-09-21T21:41:55.818 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.5611000000000004e-05 2024-09-21T21:41:55.818 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:55.818 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:55.818 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:55.818 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.818 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.818 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13561 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 1728512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:55.818 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.823991+0000", 2024-09-21T21:41:55.818 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 4.9824039170000001, 2024-09-21T21:41:55.818 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071654762999999996, 2024-09-21T21:41:55.818 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:55.819 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:55.819 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:55.819 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:55.819 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:41:55.819 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:55.819 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13561 2024-09-21T21:41:55.819 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.819 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:55.819 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.819 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:55.819 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.823991+0000", 2024-09-21T21:41:55.819 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:55.819 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.819 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.819 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:55.820 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.823991+0000", 2024-09-21T21:41:55.820 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:55.820 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.820 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.820 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:55.820 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.823991+0000", 2024-09-21T21:41:55.820 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9299999999999998e-07 2024-09-21T21:41:55.820 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.820 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.820 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:55.820 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.823994+0000", 2024-09-21T21:41:55.820 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2369999999999999e-06 2024-09-21T21:41:55.820 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.820 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.821 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:55.821 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824000+0000", 2024-09-21T21:41:55.821 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.55e-06 2024-09-21T21:41:55.821 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.821 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.821 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:55.821 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824005+0000", 2024-09-21T21:41:55.821 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4329999999999997e-06 2024-09-21T21:41:55.821 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.821 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.821 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:55.821 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.895360+0000", 2024-09-21T21:41:55.821 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071355426 2024-09-21T21:41:55.822 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.822 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.822 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:55.822 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.895477+0000", 2024-09-21T21:41:55.822 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0001171 2024-09-21T21:41:55.822 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.822 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.822 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:55.822 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.895646+0000", 2024-09-21T21:41:55.822 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000168524 2024-09-21T21:41:55.822 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:55.822 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:55.822 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:55.822 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.822 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.823 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13562 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 1757184~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:55.823 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.824011+0000", 2024-09-21T21:41:55.823 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 4.9823840400000003, 2024-09-21T21:41:55.823 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072020453999999998, 2024-09-21T21:41:55.823 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:55.823 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:55.823 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:55.823 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:55.823 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:41:55.823 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:55.823 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13562 2024-09-21T21:41:55.823 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.823 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:55.823 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.823 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:55.824 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824011+0000", 2024-09-21T21:41:55.824 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:55.824 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.824 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.824 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:55.824 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824011+0000", 2024-09-21T21:41:55.824 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:55.824 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.824 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.824 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:55.824 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824011+0000", 2024-09-21T21:41:55.824 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5600000000000001e-07 2024-09-21T21:41:55.824 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.824 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.825 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:55.825 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824013+0000", 2024-09-21T21:41:55.825 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9800000000000001e-06 2024-09-21T21:41:55.825 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.825 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.825 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:55.825 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824019+0000", 2024-09-21T21:41:55.825 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7339999999999996e-06 2024-09-21T21:41:55.825 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.825 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.825 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:55.825 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824023+0000", 2024-09-21T21:41:55.825 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3680000000000004e-06 2024-09-21T21:41:55.825 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.825 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.826 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:55.826 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.895699+0000", 2024-09-21T21:41:55.826 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071675597999999993 2024-09-21T21:41:55.826 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.826 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.826 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:55.826 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.895830+0000", 2024-09-21T21:41:55.826 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000130858 2024-09-21T21:41:55.826 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.826 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.826 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:55.826 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896031+0000", 2024-09-21T21:41:55.826 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00020155999999999999 2024-09-21T21:41:55.827 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:55.827 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:55.827 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:55.827 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.827 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.827 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13563 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 323584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:55.827 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.824029+0000", 2024-09-21T21:41:55.827 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 4.9823655999999996, 2024-09-21T21:41:55.827 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072307563000000005, 2024-09-21T21:41:55.827 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:55.827 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:55.827 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:55.827 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:55.827 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:41:55.828 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:55.828 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13563 2024-09-21T21:41:55.828 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.828 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:55.828 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.828 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:55.828 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824029+0000", 2024-09-21T21:41:55.828 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:55.828 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.828 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.828 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:55.828 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824029+0000", 2024-09-21T21:41:55.828 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:55.828 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.828 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.829 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:55.829 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824030+0000", 2024-09-21T21:41:55.829 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1699999999999999e-07 2024-09-21T21:41:55.829 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.829 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.829 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:55.829 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824032+0000", 2024-09-21T21:41:55.829 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0320000000000002e-06 2024-09-21T21:41:55.829 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.829 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.829 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:55.829 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824038+0000", 2024-09-21T21:41:55.829 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8769999999999998e-06 2024-09-21T21:41:55.829 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.830 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.830 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:55.830 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824042+0000", 2024-09-21T21:41:55.830 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5650000000000003e-06 2024-09-21T21:41:55.830 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.830 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.830 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:55.830 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896058+0000", 2024-09-21T21:41:55.830 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072015941 2024-09-21T21:41:55.830 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.830 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.830 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:55.830 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896164+0000", 2024-09-21T21:41:55.830 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010593499999999999 2024-09-21T21:41:55.830 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.831 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.831 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:55.831 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896337+0000", 2024-09-21T21:41:55.831 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00017279599999999999 2024-09-21T21:41:55.831 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:55.831 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:55.831 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:55.831 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.831 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.831 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13564 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 983040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:55.831 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.824048+0000", 2024-09-21T21:41:55.831 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 4.9823465249999996, 2024-09-21T21:41:55.831 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072491193999999995, 2024-09-21T21:41:55.831 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:55.832 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:55.832 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:55.832 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:55.832 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:41:55.832 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:55.832 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13564 2024-09-21T21:41:55.832 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.832 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:55.832 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.832 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:55.832 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824048+0000", 2024-09-21T21:41:55.832 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:55.832 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.832 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.832 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:55.833 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824048+0000", 2024-09-21T21:41:55.833 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:55.833 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.833 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.833 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:55.833 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824663+0000", 2024-09-21T21:41:55.833 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00061494299999999996 2024-09-21T21:41:55.833 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.833 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.833 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:55.833 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824669+0000", 2024-09-21T21:41:55.833 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2900000000000002e-06 2024-09-21T21:41:55.834 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.834 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.834 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:55.834 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824679+0000", 2024-09-21T21:41:55.834 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0338e-05 2024-09-21T21:41:55.834 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.834 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.834 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:55.834 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824687+0000", 2024-09-21T21:41:55.834 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.9049999999999997e-06 2024-09-21T21:41:55.834 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.834 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.834 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:55.834 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896381+0000", 2024-09-21T21:41:55.835 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071694305999999999 2024-09-21T21:41:55.835 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.835 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.835 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:55.835 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896445+0000", 2024-09-21T21:41:55.835 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3869999999999997e-05 2024-09-21T21:41:55.835 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.835 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.835 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:55.835 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896540+0000", 2024-09-21T21:41:55.835 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.4542000000000004e-05 2024-09-21T21:41:55.835 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:55.835 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:55.835 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:55.836 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.836 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.836 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13565 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3190784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:55.836 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.824704+0000", 2024-09-21T21:41:55.836 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 4.9816906540000003, 2024-09-21T21:41:55.836 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072014745000000005, 2024-09-21T21:41:55.836 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:55.836 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:55.836 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:55.836 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:55.836 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:41:55.836 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:55.836 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13565 2024-09-21T21:41:55.837 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.837 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:55.837 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.837 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:55.837 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824704+0000", 2024-09-21T21:41:55.837 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:55.837 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.837 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.837 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:55.837 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824704+0000", 2024-09-21T21:41:55.837 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:55.837 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.837 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.838 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:55.838 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824705+0000", 2024-09-21T21:41:55.838 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8999999999999997e-07 2024-09-21T21:41:55.838 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.838 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.838 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:55.838 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824707+0000", 2024-09-21T21:41:55.838 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.7669999999999999e-06 2024-09-21T21:41:55.838 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.838 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.838 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:55.838 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824715+0000", 2024-09-21T21:41:55.839 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7700000000000001e-06 2024-09-21T21:41:55.839 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.839 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.839 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:55.839 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824721+0000", 2024-09-21T21:41:55.839 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9560000000000002e-06 2024-09-21T21:41:55.839 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.839 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.839 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:55.839 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896570+0000", 2024-09-21T21:41:55.839 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071848870999999995 2024-09-21T21:41:55.839 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.839 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.839 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:55.840 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896626+0000", 2024-09-21T21:41:55.840 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5578000000000002e-05 2024-09-21T21:41:55.840 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.840 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.840 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:55.840 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896719+0000", 2024-09-21T21:41:55.840 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.3313000000000006e-05 2024-09-21T21:41:55.840 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:55.840 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:55.840 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:55.840 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.840 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.840 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13566 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3162112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:55.841 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.824729+0000", 2024-09-21T21:41:55.841 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 4.981665767, 2024-09-21T21:41:55.841 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072356907999999998, 2024-09-21T21:41:55.841 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:55.841 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:55.841 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:55.841 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:55.841 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:41:55.841 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:55.841 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13566 2024-09-21T21:41:55.841 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.841 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:55.841 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.841 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:55.842 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824729+0000", 2024-09-21T21:41:55.842 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:55.842 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.842 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.842 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:55.842 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824729+0000", 2024-09-21T21:41:55.842 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:55.842 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.842 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.842 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:55.842 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825630+0000", 2024-09-21T21:41:55.842 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00090139400000000002 2024-09-21T21:41:55.842 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.843 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.843 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:55.843 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825634+0000", 2024-09-21T21:41:55.843 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4960000000000001e-06 2024-09-21T21:41:55.843 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.843 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.843 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:55.843 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825644+0000", 2024-09-21T21:41:55.843 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.9650000000000003e-06 2024-09-21T21:41:55.843 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.843 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.843 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:55.843 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825649+0000", 2024-09-21T21:41:55.843 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0359999999999997e-06 2024-09-21T21:41:55.844 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.844 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.844 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:55.844 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896757+0000", 2024-09-21T21:41:55.844 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071107717000000001 2024-09-21T21:41:55.844 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.844 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.844 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:55.844 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896878+0000", 2024-09-21T21:41:55.844 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000120834 2024-09-21T21:41:55.844 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.844 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.844 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:55.845 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897086+0000", 2024-09-21T21:41:55.845 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00020846600000000001 2024-09-21T21:41:55.845 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:55.845 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:55.845 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:55.845 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.845 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.845 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13567 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3878912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:55.845 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.825658+0000", 2024-09-21T21:41:55.845 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 4.9807367679999999, 2024-09-21T21:41:55.845 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071597694000000003, 2024-09-21T21:41:55.845 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:55.845 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:55.846 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:55.846 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:55.846 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:41:55.846 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:55.846 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13567 2024-09-21T21:41:55.846 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.846 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:55.846 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.846 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:55.846 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825658+0000", 2024-09-21T21:41:55.846 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:55.846 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.846 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.847 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:55.847 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825658+0000", 2024-09-21T21:41:55.847 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:55.847 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.847 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.847 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:55.847 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825658+0000", 2024-09-21T21:41:55.847 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.5699999999999999e-07 2024-09-21T21:41:55.847 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.847 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.847 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:55.847 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825660+0000", 2024-09-21T21:41:55.847 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.638e-06 2024-09-21T21:41:55.847 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.847 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.848 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:55.848 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825664+0000", 2024-09-21T21:41:55.848 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4340000000000002e-06 2024-09-21T21:41:55.848 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.848 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.848 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:55.848 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825668+0000", 2024-09-21T21:41:55.848 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.1669999999999999e-06 2024-09-21T21:41:55.848 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.848 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.848 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:55.848 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897113+0000", 2024-09-21T21:41:55.848 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071445358 2024-09-21T21:41:55.848 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.849 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.849 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:55.849 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897169+0000", 2024-09-21T21:41:55.849 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5813000000000002e-05 2024-09-21T21:41:55.849 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.849 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.849 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:55.849 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897256+0000", 2024-09-21T21:41:55.849 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.7027000000000006e-05 2024-09-21T21:41:55.849 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:55.849 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:55.849 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:55.849 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.849 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.850 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13568 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3907584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:55.850 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.825671+0000", 2024-09-21T21:41:55.850 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 4.9807233579999997, 2024-09-21T21:41:55.850 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071794595000000003, 2024-09-21T21:41:55.850 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:55.850 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:55.850 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:55.850 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:55.850 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:41:55.850 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:55.850 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13568 2024-09-21T21:41:55.850 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.850 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:55.851 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.851 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:55.851 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825671+0000", 2024-09-21T21:41:55.851 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:55.851 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.851 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.851 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:55.851 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825671+0000", 2024-09-21T21:41:55.851 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:55.851 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.851 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.851 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:55.851 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825672+0000", 2024-09-21T21:41:55.851 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2999999999999999e-07 2024-09-21T21:41:55.852 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.852 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.852 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:55.852 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825673+0000", 2024-09-21T21:41:55.852 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.246e-06 2024-09-21T21:41:55.852 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.852 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.852 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:55.852 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825693+0000", 2024-09-21T21:41:55.852 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9789999999999999e-05 2024-09-21T21:41:55.852 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.852 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.852 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:55.852 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825697+0000", 2024-09-21T21:41:55.853 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2379999999999999e-06 2024-09-21T21:41:55.853 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.853 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.853 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:55.853 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897284+0000", 2024-09-21T21:41:55.853 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071586710999999997 2024-09-21T21:41:55.853 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.853 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.853 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:55.853 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897337+0000", 2024-09-21T21:41:55.853 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2800000000000003e-05 2024-09-21T21:41:55.853 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.853 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.853 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:55.853 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897466+0000", 2024-09-21T21:41:55.854 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00012957999999999999 2024-09-21T21:41:55.854 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:55.854 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:55.854 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:55.854 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.854 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.854 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13574 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 352256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:55.854 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.825702+0000", 2024-09-21T21:41:55.854 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 4.9806932100000001, 2024-09-21T21:41:55.854 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072080377000000001, 2024-09-21T21:41:55.854 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:55.854 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:55.854 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:55.854 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:55.855 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:41:55.855 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:55.855 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13574 2024-09-21T21:41:55.855 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.855 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:55.855 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.855 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:55.855 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825702+0000", 2024-09-21T21:41:55.855 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:55.855 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.855 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.855 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:55.855 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825702+0000", 2024-09-21T21:41:55.855 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:55.856 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.856 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.856 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:55.856 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825943+0000", 2024-09-21T21:41:55.856 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00024105200000000001 2024-09-21T21:41:55.856 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.856 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.856 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:55.856 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825946+0000", 2024-09-21T21:41:55.856 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.743e-06 2024-09-21T21:41:55.856 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.856 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.856 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:55.857 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825956+0000", 2024-09-21T21:41:55.857 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6609999999999995e-06 2024-09-21T21:41:55.857 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.857 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.857 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:55.857 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825963+0000", 2024-09-21T21:41:55.857 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4609999999999996e-06 2024-09-21T21:41:55.857 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.857 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.857 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:55.857 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897502+0000", 2024-09-21T21:41:55.857 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071539354999999999 2024-09-21T21:41:55.857 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.857 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.858 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:55.858 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897619+0000", 2024-09-21T21:41:55.858 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011665 2024-09-21T21:41:55.858 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.858 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.858 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:55.858 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897782+0000", 2024-09-21T21:41:55.858 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00016345500000000001 2024-09-21T21:41:55.858 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:55.858 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:55.858 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:55.858 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.858 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.859 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13575 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 380928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:55.859 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.825973+0000", 2024-09-21T21:41:55.859 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 4.9804217739999999, 2024-09-21T21:41:55.859 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072012723000000001, 2024-09-21T21:41:55.859 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:55.859 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:55.859 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:55.859 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:55.859 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:41:55.859 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:55.859 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13575 2024-09-21T21:41:55.859 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.859 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:55.859 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.859 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:55.860 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825973+0000", 2024-09-21T21:41:55.860 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:55.860 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.860 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.860 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:55.860 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825973+0000", 2024-09-21T21:41:55.860 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:55.860 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.860 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.860 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:55.860 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826746+0000", 2024-09-21T21:41:55.860 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00077326800000000002 2024-09-21T21:41:55.860 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.860 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.861 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:55.861 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826766+0000", 2024-09-21T21:41:55.861 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9465000000000001e-05 2024-09-21T21:41:55.861 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.861 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.861 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:55.861 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826812+0000", 2024-09-21T21:41:55.861 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5927999999999998e-05 2024-09-21T21:41:55.861 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.861 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.861 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:55.861 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826833+0000", 2024-09-21T21:41:55.861 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1710999999999999e-05 2024-09-21T21:41:55.862 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.862 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.862 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:55.862 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897809+0000", 2024-09-21T21:41:55.862 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.070975229000000001 2024-09-21T21:41:55.862 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.862 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.862 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:55.862 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897864+0000", 2024-09-21T21:41:55.862 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5334999999999998e-05 2024-09-21T21:41:55.862 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.862 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.862 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:55.863 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897986+0000", 2024-09-21T21:41:55.863 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000121787 2024-09-21T21:41:55.863 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:55.863 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:55.863 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:55.863 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.863 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.863 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13577 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 1814528~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:55.863 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.826874+0000", 2024-09-21T21:41:55.863 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 4.9795207399999999, 2024-09-21T21:41:55.863 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071639123999999998, 2024-09-21T21:41:55.863 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:55.863 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:55.863 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:55.864 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:55.864 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:41:55.864 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:55.864 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13577 2024-09-21T21:41:55.864 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.864 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:55.864 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.864 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:55.864 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826874+0000", 2024-09-21T21:41:55.864 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:55.864 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.864 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.864 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:55.865 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826874+0000", 2024-09-21T21:41:55.865 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:55.865 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.865 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.865 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:55.865 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826875+0000", 2024-09-21T21:41:55.865 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.82e-07 2024-09-21T21:41:55.865 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.865 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.865 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:55.865 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826877+0000", 2024-09-21T21:41:55.865 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.295e-06 2024-09-21T21:41:55.865 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.865 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.866 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:55.866 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826883+0000", 2024-09-21T21:41:55.866 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2609999999999999e-06 2024-09-21T21:41:55.866 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.866 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.866 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:55.866 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826889+0000", 2024-09-21T21:41:55.866 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5589999999999996e-06 2024-09-21T21:41:55.866 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.866 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.866 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:55.866 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898301+0000", 2024-09-21T21:41:55.866 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071412188000000001 2024-09-21T21:41:55.867 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.867 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.867 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:55.867 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898416+0000", 2024-09-21T21:41:55.867 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000115538 2024-09-21T21:41:55.867 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.867 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.867 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:55.867 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898513+0000", 2024-09-21T21:41:55.867 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6800999999999997e-05 2024-09-21T21:41:55.867 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:55.867 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:55.867 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:55.868 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.868 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.868 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13578 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 2502656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:55.868 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.826896+0000", 2024-09-21T21:41:55.868 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 4.9794992840000001, 2024-09-21T21:41:55.868 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071789411999999997, 2024-09-21T21:41:55.868 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:55.868 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:55.868 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:55.868 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:55.868 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:41:55.868 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:55.869 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13578 2024-09-21T21:41:55.869 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.869 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:55.869 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.869 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:55.869 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826896+0000", 2024-09-21T21:41:55.869 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:55.869 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.869 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.869 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:55.869 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826896+0000", 2024-09-21T21:41:55.869 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:55.869 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.869 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.870 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:55.870 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827327+0000", 2024-09-21T21:41:55.870 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00043094500000000002 2024-09-21T21:41:55.870 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.870 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.870 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:55.870 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827331+0000", 2024-09-21T21:41:55.870 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.0969999999999999e-06 2024-09-21T21:41:55.870 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.870 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.870 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:55.870 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827341+0000", 2024-09-21T21:41:55.871 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0199e-05 2024-09-21T21:41:55.871 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.871 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.871 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:55.871 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827346+0000", 2024-09-21T21:41:55.871 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3210000000000004e-06 2024-09-21T21:41:55.871 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.871 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.871 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:55.871 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898540+0000", 2024-09-21T21:41:55.871 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071194246000000003 2024-09-21T21:41:55.871 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.871 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.871 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:55.872 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898598+0000", 2024-09-21T21:41:55.872 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7658999999999998e-05 2024-09-21T21:41:55.872 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.872 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.872 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:55.872 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898685+0000", 2024-09-21T21:41:55.872 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.6945000000000002e-05 2024-09-21T21:41:55.872 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:55.872 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:55.872 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:55.872 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.872 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.872 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13579 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 2531328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:55.873 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.827355+0000", 2024-09-21T21:41:55.873 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 4.9790397530000003, 2024-09-21T21:41:55.873 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071650933, 2024-09-21T21:41:55.873 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:55.873 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:55.873 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:55.873 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:55.873 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:41:55.873 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:55.873 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13579 2024-09-21T21:41:55.873 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.873 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:55.873 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.873 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:55.873 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827355+0000", 2024-09-21T21:41:55.874 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:55.874 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.874 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.874 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:55.874 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827355+0000", 2024-09-21T21:41:55.874 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:55.874 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.874 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.874 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:55.874 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827486+0000", 2024-09-21T21:41:55.874 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00013066799999999999 2024-09-21T21:41:55.874 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.874 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.875 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:55.875 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827490+0000", 2024-09-21T21:41:55.875 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.9009999999999996e-06 2024-09-21T21:41:55.875 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.875 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.875 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:55.875 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827499+0000", 2024-09-21T21:41:55.875 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6209999999999999e-06 2024-09-21T21:41:55.875 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.875 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.875 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:55.875 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827506+0000", 2024-09-21T21:41:55.875 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6490000000000002e-06 2024-09-21T21:41:55.875 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.876 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.876 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:55.876 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898713+0000", 2024-09-21T21:41:55.876 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071206937999999997 2024-09-21T21:41:55.876 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.876 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.876 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:55.876 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898811+0000", 2024-09-21T21:41:55.876 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.8159999999999995e-05 2024-09-21T21:41:55.876 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.876 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.876 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:55.877 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899006+0000", 2024-09-21T21:41:55.877 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00019499599999999999 2024-09-21T21:41:55.877 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:55.877 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:55.877 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:55.877 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.877 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.877 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13580 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3219456~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:55.877 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.827522+0000", 2024-09-21T21:41:55.877 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 4.9788733220000001, 2024-09-21T21:41:55.877 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071813298999999997, 2024-09-21T21:41:55.877 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:55.877 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:55.878 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:55.878 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:55.878 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:41:55.878 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:55.878 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13580 2024-09-21T21:41:55.878 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.878 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:55.878 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.878 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:55.878 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827522+0000", 2024-09-21T21:41:55.878 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:55.878 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.878 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.879 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:55.879 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827522+0000", 2024-09-21T21:41:55.879 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:55.879 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.879 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.879 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:55.879 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827898+0000", 2024-09-21T21:41:55.879 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000376876 2024-09-21T21:41:55.879 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.879 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.879 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:55.879 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827903+0000", 2024-09-21T21:41:55.880 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4129999999999999e-06 2024-09-21T21:41:55.880 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.880 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.880 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:55.880 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827911+0000", 2024-09-21T21:41:55.880 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.4540000000000007e-06 2024-09-21T21:41:55.880 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.880 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.880 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:55.880 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827917+0000", 2024-09-21T21:41:55.880 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2110000000000003e-06 2024-09-21T21:41:55.880 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.880 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.881 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:55.881 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899041+0000", 2024-09-21T21:41:55.881 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071123965999999997 2024-09-21T21:41:55.881 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.881 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.881 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:55.881 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899157+0000", 2024-09-21T21:41:55.881 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011573700000000001 2024-09-21T21:41:55.881 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.881 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.881 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:55.881 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899335+0000", 2024-09-21T21:41:55.881 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00017764199999999999 2024-09-21T21:41:55.881 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:55.882 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:55.882 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:55.882 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.882 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.882 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13581 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3248128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:55.882 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.827929+0000", 2024-09-21T21:41:55.882 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 4.9784659229999999, 2024-09-21T21:41:55.882 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071730151000000006, 2024-09-21T21:41:55.882 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:55.882 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:55.882 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:55.882 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:55.882 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:41:55.883 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:55.883 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13581 2024-09-21T21:41:55.883 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.883 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:55.883 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.883 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:55.883 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827929+0000", 2024-09-21T21:41:55.883 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:55.883 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.883 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.883 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:55.883 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827929+0000", 2024-09-21T21:41:55.883 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:55.883 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.883 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.884 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:55.884 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.829943+0000", 2024-09-21T21:41:55.884 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0020143750000000001 2024-09-21T21:41:55.884 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.884 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.884 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:55.884 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.829948+0000", 2024-09-21T21:41:55.884 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9400000000000001e-06 2024-09-21T21:41:55.884 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.884 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.884 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:55.884 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.829960+0000", 2024-09-21T21:41:55.884 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.2123999999999999e-05 2024-09-21T21:41:55.884 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.884 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.885 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:55.885 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.829968+0000", 2024-09-21T21:41:55.885 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7000000000000008e-06 2024-09-21T21:41:55.885 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.885 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.885 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:55.885 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899370+0000", 2024-09-21T21:41:55.885 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.069401672999999997 2024-09-21T21:41:55.885 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.885 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.885 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:55.885 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899481+0000", 2024-09-21T21:41:55.885 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011145300000000001 2024-09-21T21:41:55.885 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.886 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.886 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:55.886 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899659+0000", 2024-09-21T21:41:55.886 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00017788600000000001 2024-09-21T21:41:55.886 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:55.886 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:55.886 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:55.886 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.886 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.886 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13595 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 1871872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:55.886 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.830180+0000", 2024-09-21T21:41:55.886 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 4.976214809, 2024-09-21T21:41:55.886 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071809244999999994, 2024-09-21T21:41:55.887 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:55.887 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:55.887 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:55.887 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:55.887 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:41:55.887 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:55.887 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13595 2024-09-21T21:41:55.887 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.887 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:55.887 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.887 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:55.887 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.830180+0000", 2024-09-21T21:41:55.888 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:55.888 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.888 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.888 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:55.888 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.830180+0000", 2024-09-21T21:41:55.888 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:55.888 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.888 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.888 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:55.888 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.831967+0000", 2024-09-21T21:41:55.888 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0017868350000000001 2024-09-21T21:41:55.888 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.888 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.888 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:55.889 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.831971+0000", 2024-09-21T21:41:55.889 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3950000000000003e-06 2024-09-21T21:41:55.889 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.889 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.889 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:55.889 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.831983+0000", 2024-09-21T21:41:55.889 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1815e-05 2024-09-21T21:41:55.889 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.889 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.889 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:55.889 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.831993+0000", 2024-09-21T21:41:55.889 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6339999999999996e-06 2024-09-21T21:41:55.890 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.890 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.890 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:55.890 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.901693+0000", 2024-09-21T21:41:55.890 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.069700082999999996 2024-09-21T21:41:55.890 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.890 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.890 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:55.890 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.901813+0000", 2024-09-21T21:41:55.890 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011971 2024-09-21T21:41:55.890 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.890 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.890 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:55.890 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.901989+0000", 2024-09-21T21:41:55.891 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000176773 2024-09-21T21:41:55.891 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:55.891 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:55.891 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:55.891 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.891 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.891 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13600 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 2617344~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:55.891 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.832109+0000", 2024-09-21T21:41:55.891 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 4.9742860560000004, 2024-09-21T21:41:55.891 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071551828999999997, 2024-09-21T21:41:55.891 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:55.891 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:55.891 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:55.892 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:55.892 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:41:55.892 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:55.892 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13600 2024-09-21T21:41:55.892 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.892 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:55.892 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.892 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:55.892 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.832109+0000", 2024-09-21T21:41:55.892 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:55.892 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.892 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.892 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:55.892 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.832109+0000", 2024-09-21T21:41:55.893 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:55.893 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.893 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.893 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:55.893 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.832109+0000", 2024-09-21T21:41:55.893 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4999999999999998e-07 2024-09-21T21:41:55.893 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.893 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.893 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:55.893 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.832112+0000", 2024-09-21T21:41:55.893 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.5019999999999999e-06 2024-09-21T21:41:55.893 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.893 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.894 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:55.894 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.832118+0000", 2024-09-21T21:41:55.894 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6270000000000002e-06 2024-09-21T21:41:55.894 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.894 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.894 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:55.894 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.832124+0000", 2024-09-21T21:41:55.894 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1850000000000003e-06 2024-09-21T21:41:55.894 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.894 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.894 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:55.894 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.903366+0000", 2024-09-21T21:41:55.895 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071241961000000006 2024-09-21T21:41:55.895 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.895 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.895 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:55.895 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.903485+0000", 2024-09-21T21:41:55.895 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000119202 2024-09-21T21:41:55.895 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.895 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.895 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:55.895 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.903661+0000", 2024-09-21T21:41:55.895 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000175902 2024-09-21T21:41:55.895 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:55.895 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:55.896 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:55.896 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.896 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.896 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13601 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3334144~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:41:55.896 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.832130+0000", 2024-09-21T21:41:55.896 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 4.9742649370000001, 2024-09-21T21:41:55.896 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071861727, 2024-09-21T21:41:55.896 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:41:55.896 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:41:55.896 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:41:55.896 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:41:55.896 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:41:55.897 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:55.897 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13601 2024-09-21T21:41:55.897 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.897 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:41:55.897 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.897 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:41:55.897 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.832130+0000", 2024-09-21T21:41:55.897 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:55.897 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.897 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.897 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:41:55.897 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.832130+0000", 2024-09-21T21:41:55.897 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:41:55.897 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.898 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.898 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:41:55.898 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.833939+0000", 2024-09-21T21:41:55.898 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0018094400000000001 2024-09-21T21:41:55.898 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.898 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.898 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:41:55.898 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.833944+0000", 2024-09-21T21:41:55.898 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5979999999999999e-06 2024-09-21T21:41:55.898 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.898 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.898 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:41:55.898 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.833953+0000", 2024-09-21T21:41:55.899 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.1460000000000002e-06 2024-09-21T21:41:55.899 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.899 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.899 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:41:55.899 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.833960+0000", 2024-09-21T21:41:55.899 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.7989999999999996e-06 2024-09-21T21:41:55.899 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.899 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.899 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:41:55.899 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.903693+0000", 2024-09-21T21:41:55.899 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.069732901 2024-09-21T21:41:55.899 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.899 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.899 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:41:55.900 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.903814+0000", 2024-09-21T21:41:55.900 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000121585 2024-09-21T21:41:55.900 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:41:55.900 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:41:55.900 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:41:55.900 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.903992+0000", 2024-09-21T21:41:55.900 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00017725799999999999 2024-09-21T21:41:55.900 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:55.900 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:55.900 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:55.900 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:41:55.900 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:41:55.900 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:55.902 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T21:41:55.902 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:55.900+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:56.002 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:56.001+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:56.006 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:56.006 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:41:56.006 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-21T21:41:56.006 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:56.015 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T21:41:56.104 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:56.102+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:56.117 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:56.118 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:41:56.118 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:41:56.118 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:41:56.118 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:56.127 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T21:41:56.205 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:56.204+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:56.223 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T21:41:56.274 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:56.274 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-21T21:41:56.274 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-21T21:41:56.274 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:41:56.274 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.274 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19744 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6471680~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:41:56.274 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.272996+0000", 2024-09-21T21:41:56.274 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.9986062059999998, 2024-09-21T21:41:56.275 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97298795400000004, 2024-09-21T21:41:56.275 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:56.275 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:56.275 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:56.275 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:56.275 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:56.275 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:56.275 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19744 2024-09-21T21:41:56.275 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.275 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:56.275 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.275 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:56.275 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.272996+0000", 2024-09-21T21:41:56.275 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:56.276 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.276 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.276 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:56.276 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.272996+0000", 2024-09-21T21:41:56.276 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:56.276 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.276 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.276 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:56.276 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.272996+0000", 2024-09-21T21:41:56.276 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.53e-07 2024-09-21T21:41:56.276 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.276 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.276 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:56.276 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.272998+0000", 2024-09-21T21:41:56.276 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.888e-06 2024-09-21T21:41:56.277 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.277 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.277 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:56.277 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273005+0000", 2024-09-21T21:41:56.277 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4350000000000002e-06 2024-09-21T21:41:56.277 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.277 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.277 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:56.277 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273009+0000", 2024-09-21T21:41:56.277 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3239999999999996e-06 2024-09-21T21:41:56.277 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.277 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.277 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:56.277 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277855+0000", 2024-09-21T21:41:56.278 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048463849999999999 2024-09-21T21:41:56.278 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.278 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.278 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:41:56.278 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277858+0000", 2024-09-21T21:41:56.278 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9490000000000001e-06 2024-09-21T21:41:56.278 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.278 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.278 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:56.278 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.245879+0000", 2024-09-21T21:41:56.278 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96802068399999996 2024-09-21T21:41:56.278 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.278 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.278 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:56.279 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.245943+0000", 2024-09-21T21:41:56.279 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3690999999999997e-05 2024-09-21T21:41:56.279 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.279 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.279 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:56.279 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.245984+0000", 2024-09-21T21:41:56.279 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1245000000000002e-05 2024-09-21T21:41:56.279 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:56.279 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:56.279 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:56.279 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.279 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.279 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19745 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 7876608~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:41:56.279 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273015+0000", 2024-09-21T21:41:56.280 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.9985875570000005, 2024-09-21T21:41:56.280 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97312649200000001, 2024-09-21T21:41:56.280 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:56.280 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:56.280 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:56.280 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:56.280 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:56.280 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:56.280 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19745 2024-09-21T21:41:56.280 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.280 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:56.280 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.280 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:56.280 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273015+0000", 2024-09-21T21:41:56.280 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:56.281 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.281 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.281 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:56.281 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273015+0000", 2024-09-21T21:41:56.281 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:56.281 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.281 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.281 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:56.281 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273015+0000", 2024-09-21T21:41:56.281 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2300000000000002e-07 2024-09-21T21:41:56.281 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.281 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.281 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:56.281 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273017+0000", 2024-09-21T21:41:56.282 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0480000000000001e-06 2024-09-21T21:41:56.282 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.282 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.282 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:56.282 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273023+0000", 2024-09-21T21:41:56.282 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.699e-06 2024-09-21T21:41:56.282 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.282 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.282 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:56.282 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273027+0000", 2024-09-21T21:41:56.282 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6689999999999997e-06 2024-09-21T21:41:56.282 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.282 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.282 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:56.283 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277895+0000", 2024-09-21T21:41:56.283 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048675039999999999 2024-09-21T21:41:56.283 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.283 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.283 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:41:56.283 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277899+0000", 2024-09-21T21:41:56.283 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5519999999999998e-06 2024-09-21T21:41:56.283 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.283 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.283 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:56.283 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246018+0000", 2024-09-21T21:41:56.283 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96811853599999997 2024-09-21T21:41:56.283 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.283 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.284 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:56.284 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246090+0000", 2024-09-21T21:41:56.284 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2502999999999993e-05 2024-09-21T21:41:56.284 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.284 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.284 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:56.284 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246141+0000", 2024-09-21T21:41:56.284 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0658000000000002e-05 2024-09-21T21:41:56.284 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:56.284 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:56.284 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:56.284 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.284 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.284 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19746 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 7188480~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:41:56.284 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273033+0000", 2024-09-21T21:41:56.285 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.9985689740000003, 2024-09-21T21:41:56.285 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97319625399999998, 2024-09-21T21:41:56.285 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:56.285 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:56.285 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:56.285 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:56.285 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:56.285 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:56.285 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19746 2024-09-21T21:41:56.285 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.285 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:56.285 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.285 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:56.285 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273033+0000", 2024-09-21T21:41:56.286 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:56.286 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.286 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.286 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:56.286 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273033+0000", 2024-09-21T21:41:56.286 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:56.286 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.286 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.286 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:56.286 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273034+0000", 2024-09-21T21:41:56.286 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2100000000000002e-07 2024-09-21T21:41:56.286 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.286 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.286 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:56.286 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273036+0000", 2024-09-21T21:41:56.287 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1919999999999999e-06 2024-09-21T21:41:56.287 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.287 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.287 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:56.287 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273044+0000", 2024-09-21T21:41:56.287 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2369999999999992e-06 2024-09-21T21:41:56.287 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.287 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.287 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:56.287 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273048+0000", 2024-09-21T21:41:56.287 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5090000000000004e-06 2024-09-21T21:41:56.287 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.287 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.287 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:56.288 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277921+0000", 2024-09-21T21:41:56.288 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048723710000000003 2024-09-21T21:41:56.288 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.288 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.288 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:41:56.288 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277924+0000", 2024-09-21T21:41:56.288 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6509999999999999e-06 2024-09-21T21:41:56.288 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.288 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.288 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:56.288 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246153+0000", 2024-09-21T21:41:56.288 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96822869899999997 2024-09-21T21:41:56.288 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.288 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.289 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:56.289 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246192+0000", 2024-09-21T21:41:56.289 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8747e-05 2024-09-21T21:41:56.289 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.289 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.289 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:56.289 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246229+0000", 2024-09-21T21:41:56.289 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7426999999999999e-05 2024-09-21T21:41:56.289 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:56.289 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:56.289 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:56.289 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.289 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.289 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19747 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 7905280~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:41:56.289 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273058+0000", 2024-09-21T21:41:56.290 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.9985442520000003, 2024-09-21T21:41:56.290 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97338591399999996, 2024-09-21T21:41:56.290 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:56.290 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:56.290 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:56.290 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:56.290 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:56.290 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:56.290 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19747 2024-09-21T21:41:56.290 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.290 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:56.290 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.290 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:56.290 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273058+0000", 2024-09-21T21:41:56.291 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:56.291 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.291 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.291 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:56.291 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273058+0000", 2024-09-21T21:41:56.291 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:56.291 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.291 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.291 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:56.291 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273058+0000", 2024-09-21T21:41:56.291 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5900000000000003e-07 2024-09-21T21:41:56.291 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.291 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.291 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:56.292 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273061+0000", 2024-09-21T21:41:56.292 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3479999999999998e-06 2024-09-21T21:41:56.292 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.292 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.292 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:56.292 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273067+0000", 2024-09-21T21:41:56.292 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1469999999999998e-06 2024-09-21T21:41:56.292 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.292 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.292 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:56.292 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273071+0000", 2024-09-21T21:41:56.292 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7339999999999999e-06 2024-09-21T21:41:56.292 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.292 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.293 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:56.293 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277945+0000", 2024-09-21T21:41:56.293 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.004873474 2024-09-21T21:41:56.293 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.293 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.293 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:41:56.293 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277948+0000", 2024-09-21T21:41:56.293 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0299999999999998e-06 2024-09-21T21:41:56.293 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.293 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.293 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:56.293 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246269+0000", 2024-09-21T21:41:56.293 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96832060200000003 2024-09-21T21:41:56.293 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.294 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.294 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:56.294 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246362+0000", 2024-09-21T21:41:56.294 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3867000000000001e-05 2024-09-21T21:41:56.294 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.294 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.294 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:56.294 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246444+0000", 2024-09-21T21:41:56.294 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1353000000000005e-05 2024-09-21T21:41:56.294 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:56.294 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:56.294 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:56.294 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.295 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.295 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19755 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 5783552~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:41:56.295 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273077+0000", 2024-09-21T21:41:56.295 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.998524636, 2024-09-21T21:41:56.295 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97350203400000002, 2024-09-21T21:41:56.295 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:56.295 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:56.295 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:56.295 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:56.295 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:56.295 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:56.295 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19755 2024-09-21T21:41:56.295 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.295 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:56.296 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.296 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:56.296 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273077+0000", 2024-09-21T21:41:56.296 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:56.296 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.296 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.296 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:56.296 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273077+0000", 2024-09-21T21:41:56.296 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:56.296 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.296 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.296 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:56.296 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273078+0000", 2024-09-21T21:41:56.296 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5699999999999998e-07 2024-09-21T21:41:56.297 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.297 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.297 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:56.297 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273083+0000", 2024-09-21T21:41:56.297 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8670000000000002e-06 2024-09-21T21:41:56.297 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.297 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.297 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:56.297 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273088+0000", 2024-09-21T21:41:56.297 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8370000000000002e-06 2024-09-21T21:41:56.297 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.297 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.297 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:56.297 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273093+0000", 2024-09-21T21:41:56.297 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5709999999999999e-06 2024-09-21T21:41:56.298 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.298 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.298 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:56.298 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277968+0000", 2024-09-21T21:41:56.298 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048753909999999998 2024-09-21T21:41:56.298 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.298 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.298 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:41:56.298 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277971+0000", 2024-09-21T21:41:56.298 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0189999999999998e-06 2024-09-21T21:41:56.298 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.298 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.298 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:56.298 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246485+0000", 2024-09-21T21:41:56.299 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96851399299999996 2024-09-21T21:41:56.299 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.299 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.299 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:56.299 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246536+0000", 2024-09-21T21:41:56.299 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0430000000000003e-05 2024-09-21T21:41:56.299 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.299 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.299 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:56.299 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246579+0000", 2024-09-21T21:41:56.299 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3569000000000002e-05 2024-09-21T21:41:56.299 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:56.299 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:56.299 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:56.299 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.300 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.300 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19756 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6500352~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:41:56.300 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273099+0000", 2024-09-21T21:41:56.300 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.9985032629999999, 2024-09-21T21:41:56.300 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97359272799999996, 2024-09-21T21:41:56.300 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:56.300 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:56.300 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:56.300 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:56.300 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:56.300 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:56.300 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19756 2024-09-21T21:41:56.300 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.300 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:56.300 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.301 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:56.301 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273099+0000", 2024-09-21T21:41:56.301 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:56.301 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.301 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.301 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:56.301 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273099+0000", 2024-09-21T21:41:56.301 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:56.301 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.301 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.301 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:56.301 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273099+0000", 2024-09-21T21:41:56.301 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0900000000000002e-07 2024-09-21T21:41:56.301 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.302 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.302 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:56.302 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273101+0000", 2024-09-21T21:41:56.302 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0320000000000002e-06 2024-09-21T21:41:56.302 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.302 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.302 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:56.302 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273107+0000", 2024-09-21T21:41:56.302 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2830000000000001e-06 2024-09-21T21:41:56.302 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.302 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.302 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:56.302 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273111+0000", 2024-09-21T21:41:56.302 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2230000000000001e-06 2024-09-21T21:41:56.303 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.303 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.303 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:56.303 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277994+0000", 2024-09-21T21:41:56.303 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048837070000000002 2024-09-21T21:41:56.303 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.303 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.303 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:41:56.303 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278000+0000", 2024-09-21T21:41:56.303 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3149999999999999e-06 2024-09-21T21:41:56.303 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.303 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.303 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:56.303 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246607+0000", 2024-09-21T21:41:56.303 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.968606839 2024-09-21T21:41:56.304 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.304 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.304 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:56.304 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246651+0000", 2024-09-21T21:41:56.304 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4734999999999998e-05 2024-09-21T21:41:56.304 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.304 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.304 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:56.304 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246692+0000", 2024-09-21T21:41:56.304 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0185000000000002e-05 2024-09-21T21:41:56.304 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:56.304 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:56.304 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:56.304 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.305 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.305 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19757 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 5812224~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:41:56.305 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273116+0000", 2024-09-21T21:41:56.305 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.998485606, 2024-09-21T21:41:56.305 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97367813999999997, 2024-09-21T21:41:56.305 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:56.305 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:56.305 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:56.305 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:56.305 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:56.305 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:56.305 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19757 2024-09-21T21:41:56.305 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.305 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:56.306 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.306 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:56.306 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273116+0000", 2024-09-21T21:41:56.306 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:56.306 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.306 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.306 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:56.306 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273116+0000", 2024-09-21T21:41:56.306 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:56.306 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.306 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.306 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:56.306 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273117+0000", 2024-09-21T21:41:56.307 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0999999999999999e-07 2024-09-21T21:41:56.307 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.307 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.307 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:56.307 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273119+0000", 2024-09-21T21:41:56.307 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9970000000000001e-06 2024-09-21T21:41:56.307 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.307 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.307 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:56.307 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273125+0000", 2024-09-21T21:41:56.307 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9540000000000001e-06 2024-09-21T21:41:56.307 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.307 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.308 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:56.308 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273135+0000", 2024-09-21T21:41:56.308 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0597999999999999e-05 2024-09-21T21:41:56.308 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.308 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.308 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:56.308 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278019+0000", 2024-09-21T21:41:56.308 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048834849999999999 2024-09-21T21:41:56.308 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.308 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.308 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:41:56.308 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278022+0000", 2024-09-21T21:41:56.308 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9560000000000002e-06 2024-09-21T21:41:56.308 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.309 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.309 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:56.309 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246716+0000", 2024-09-21T21:41:56.309 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.968694202 2024-09-21T21:41:56.309 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.309 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.309 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:56.309 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246758+0000", 2024-09-21T21:41:56.309 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1607000000000003e-05 2024-09-21T21:41:56.309 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.309 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.309 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:56.309 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246795+0000", 2024-09-21T21:41:56.309 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6931000000000003e-05 2024-09-21T21:41:56.309 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:56.310 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:56.310 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:56.310 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.310 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.310 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19758 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6529024~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:41:56.310 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273142+0000", 2024-09-21T21:41:56.310 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.9984603849999996, 2024-09-21T21:41:56.310 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97375924599999997, 2024-09-21T21:41:56.310 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:56.310 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:56.310 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:56.310 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:56.310 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:56.311 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:56.311 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19758 2024-09-21T21:41:56.311 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.311 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:56.311 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.311 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:56.311 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273142+0000", 2024-09-21T21:41:56.311 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:56.311 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.311 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.311 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:56.311 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273142+0000", 2024-09-21T21:41:56.311 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:56.312 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.312 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.312 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:56.312 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273142+0000", 2024-09-21T21:41:56.312 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0200000000000003e-07 2024-09-21T21:41:56.312 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.312 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.312 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:56.312 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273144+0000", 2024-09-21T21:41:56.312 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1390000000000002e-06 2024-09-21T21:41:56.312 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.312 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.312 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:56.313 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273150+0000", 2024-09-21T21:41:56.313 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6409999999999999e-06 2024-09-21T21:41:56.313 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.313 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.313 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:56.313 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273157+0000", 2024-09-21T21:41:56.313 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1359999999999999e-06 2024-09-21T21:41:56.313 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.313 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.313 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:56.313 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278050+0000", 2024-09-21T21:41:56.313 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048926170000000001 2024-09-21T21:41:56.313 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.313 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.314 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:41:56.314 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278053+0000", 2024-09-21T21:41:56.314 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4879999999999999e-06 2024-09-21T21:41:56.314 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.314 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.314 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:56.314 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246819+0000", 2024-09-21T21:41:56.314 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96876590399999996 2024-09-21T21:41:56.314 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.314 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.314 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:56.314 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246856+0000", 2024-09-21T21:41:56.314 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7287999999999999e-05 2024-09-21T21:41:56.314 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.315 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.315 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:56.315 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246901+0000", 2024-09-21T21:41:56.315 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4631e-05 2024-09-21T21:41:56.315 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:56.315 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:56.315 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:56.315 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.315 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.315 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19759 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 7933952~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:41:56.315 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273172+0000", 2024-09-21T21:41:56.315 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.9984301210000002, 2024-09-21T21:41:56.315 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97384789500000002, 2024-09-21T21:41:56.315 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:56.316 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:56.316 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:56.316 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:56.316 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:56.316 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:56.316 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19759 2024-09-21T21:41:56.316 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.316 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:56.316 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.316 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:56.316 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273172+0000", 2024-09-21T21:41:56.316 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:56.316 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.316 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.317 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:56.317 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273172+0000", 2024-09-21T21:41:56.317 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:56.317 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.317 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.317 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:56.317 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273172+0000", 2024-09-21T21:41:56.317 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6100000000000002e-07 2024-09-21T21:41:56.317 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.317 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.317 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:56.317 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273174+0000", 2024-09-21T21:41:56.317 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.691e-06 2024-09-21T21:41:56.317 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.318 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.318 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:56.318 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273178+0000", 2024-09-21T21:41:56.318 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7579999999999998e-06 2024-09-21T21:41:56.318 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.318 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.318 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:56.318 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273180+0000", 2024-09-21T21:41:56.318 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7199999999999998e-06 2024-09-21T21:41:56.318 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.318 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.318 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:56.318 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278077+0000", 2024-09-21T21:41:56.318 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048969529999999999 2024-09-21T21:41:56.318 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.319 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.319 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:41:56.319 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278080+0000", 2024-09-21T21:41:56.319 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.066e-06 2024-09-21T21:41:56.319 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.319 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.319 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:56.319 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246925+0000", 2024-09-21T21:41:56.319 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96884490199999995 2024-09-21T21:41:56.319 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.319 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.319 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:56.319 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246974+0000", 2024-09-21T21:41:56.319 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8387999999999998e-05 2024-09-21T21:41:56.320 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.320 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.320 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:56.320 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247020+0000", 2024-09-21T21:41:56.320 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6156000000000003e-05 2024-09-21T21:41:56.320 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:56.320 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:56.320 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:56.320 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.320 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.320 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19760 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 7991296~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:41:56.320 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273220+0000", 2024-09-21T21:41:56.320 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.9983816069999998, 2024-09-21T21:41:56.320 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97403125499999998, 2024-09-21T21:41:56.320 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:56.321 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:56.321 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:56.321 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:56.321 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:56.321 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:56.321 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19760 2024-09-21T21:41:56.321 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.321 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:56.321 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.321 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:56.321 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273220+0000", 2024-09-21T21:41:56.321 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:56.321 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.321 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.321 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:56.322 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273220+0000", 2024-09-21T21:41:56.322 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:56.322 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.322 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.322 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:56.322 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273221+0000", 2024-09-21T21:41:56.322 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.65e-07 2024-09-21T21:41:56.322 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.322 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.322 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:56.322 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273228+0000", 2024-09-21T21:41:56.322 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6040000000000004e-06 2024-09-21T21:41:56.322 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.322 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.323 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:56.323 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273233+0000", 2024-09-21T21:41:56.323 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1569999999999999e-06 2024-09-21T21:41:56.323 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.323 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.323 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:56.323 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273238+0000", 2024-09-21T21:41:56.323 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0050000000000004e-06 2024-09-21T21:41:56.323 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.323 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.323 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:56.323 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278107+0000", 2024-09-21T21:41:56.323 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00486857 2024-09-21T21:41:56.323 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.323 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.324 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:41:56.324 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278111+0000", 2024-09-21T21:41:56.324 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6389999999999998e-06 2024-09-21T21:41:56.324 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.324 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.324 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:56.324 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247069+0000", 2024-09-21T21:41:56.324 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96895796899999997 2024-09-21T21:41:56.324 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.324 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.324 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:56.324 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247166+0000", 2024-09-21T21:41:56.324 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.7158999999999996e-05 2024-09-21T21:41:56.324 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.324 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.325 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:56.325 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247252+0000", 2024-09-21T21:41:56.325 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5886999999999999e-05 2024-09-21T21:41:56.325 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:56.325 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:56.325 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:56.325 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.325 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.325 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19761 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 7962624~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:41:56.325 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273246+0000", 2024-09-21T21:41:56.325 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.9983565470000002, 2024-09-21T21:41:56.325 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97415972200000001, 2024-09-21T21:41:56.325 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:56.325 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:56.325 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:56.326 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:56.326 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:56.326 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:56.326 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19761 2024-09-21T21:41:56.326 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.326 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:56.326 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.326 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:56.326 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273246+0000", 2024-09-21T21:41:56.326 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:56.326 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.326 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.326 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:56.326 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273246+0000", 2024-09-21T21:41:56.326 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:56.327 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.327 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.327 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:56.327 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273246+0000", 2024-09-21T21:41:56.327 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5100000000000001e-07 2024-09-21T21:41:56.327 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.327 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.327 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:56.327 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273248+0000", 2024-09-21T21:41:56.327 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1550000000000001e-06 2024-09-21T21:41:56.327 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.327 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.327 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:56.327 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273254+0000", 2024-09-21T21:41:56.327 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.964e-06 2024-09-21T21:41:56.328 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.328 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.328 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:56.328 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273259+0000", 2024-09-21T21:41:56.328 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5499999999999996e-06 2024-09-21T21:41:56.328 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.328 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.328 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:56.328 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278130+0000", 2024-09-21T21:41:56.328 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048715779999999997 2024-09-21T21:41:56.328 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.328 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.328 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:41:56.328 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278133+0000", 2024-09-21T21:41:56.328 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8739999999999999e-06 2024-09-21T21:41:56.329 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.329 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.329 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:56.329 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247270+0000", 2024-09-21T21:41:56.329 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.969137099 2024-09-21T21:41:56.329 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.329 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.329 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:56.329 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247338+0000", 2024-09-21T21:41:56.329 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7717000000000002e-05 2024-09-21T21:41:56.329 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.329 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.329 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:56.329 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247405+0000", 2024-09-21T21:41:56.329 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7434000000000005e-05 2024-09-21T21:41:56.330 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:56.330 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:56.330 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:56.330 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.330 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.330 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19769 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 5840896~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:41:56.330 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273287+0000", 2024-09-21T21:41:56.330 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.9983152630000003, 2024-09-21T21:41:56.330 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97435390200000005, 2024-09-21T21:41:56.330 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:56.330 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:56.330 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:56.330 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:56.331 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:56.331 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:56.331 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19769 2024-09-21T21:41:56.331 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.331 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:56.331 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.331 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:56.331 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273287+0000", 2024-09-21T21:41:56.331 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:56.331 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.331 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.331 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:56.331 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273287+0000", 2024-09-21T21:41:56.331 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:56.331 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.332 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.332 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:56.332 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273287+0000", 2024-09-21T21:41:56.332 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6699999999999999e-07 2024-09-21T21:41:56.332 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.332 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.332 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:56.332 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273289+0000", 2024-09-21T21:41:56.332 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.708e-06 2024-09-21T21:41:56.332 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.332 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.332 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:56.332 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273296+0000", 2024-09-21T21:41:56.333 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8109999999999997e-06 2024-09-21T21:41:56.333 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.333 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.333 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:56.333 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273299+0000", 2024-09-21T21:41:56.333 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0740000000000001e-06 2024-09-21T21:41:56.333 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.333 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.333 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:56.333 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278159+0000", 2024-09-21T21:41:56.333 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048603240000000001 2024-09-21T21:41:56.333 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.333 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.334 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:41:56.334 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278162+0000", 2024-09-21T21:41:56.334 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1930000000000002e-06 2024-09-21T21:41:56.334 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.334 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.334 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:56.334 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247444+0000", 2024-09-21T21:41:56.334 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96928199100000001 2024-09-21T21:41:56.334 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.334 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.334 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:56.334 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247533+0000", 2024-09-21T21:41:56.334 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8490000000000001e-05 2024-09-21T21:41:56.335 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.335 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.335 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:56.335 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247641+0000", 2024-09-21T21:41:56.335 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000107944 2024-09-21T21:41:56.335 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:56.335 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:56.335 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:56.335 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.335 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.335 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19770 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 5869568~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:41:56.335 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273313+0000", 2024-09-21T21:41:56.336 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.9982891609999998, 2024-09-21T21:41:56.336 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97449038899999996, 2024-09-21T21:41:56.336 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:56.336 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:56.336 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:56.337 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:56.337 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:56.338 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:56.338 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19770 2024-09-21T21:41:56.338 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.338 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:56.338 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.338 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:56.338 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273313+0000", 2024-09-21T21:41:56.338 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:56.338 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.338 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.338 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:56.338 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273313+0000", 2024-09-21T21:41:56.339 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:56.339 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.339 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.339 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:56.339 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273313+0000", 2024-09-21T21:41:56.339 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5699999999999998e-07 2024-09-21T21:41:56.339 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.339 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.339 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:56.339 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273315+0000", 2024-09-21T21:41:56.339 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0099999999999998e-06 2024-09-21T21:41:56.339 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.339 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.340 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:56.340 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273321+0000", 2024-09-21T21:41:56.340 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7529999999999998e-06 2024-09-21T21:41:56.340 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.340 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.340 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:56.340 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273328+0000", 2024-09-21T21:41:56.340 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9009999999999997e-06 2024-09-21T21:41:56.340 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.340 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.340 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:56.340 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278183+0000", 2024-09-21T21:41:56.340 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048547770000000002 2024-09-21T21:41:56.340 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.341 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.341 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:41:56.341 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278186+0000", 2024-09-21T21:41:56.341 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3100000000000001e-06 2024-09-21T21:41:56.341 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.341 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.341 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:56.341 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247676+0000", 2024-09-21T21:41:56.341 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96949004299999997 2024-09-21T21:41:56.341 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.341 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.341 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:56.341 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247757+0000", 2024-09-21T21:41:56.341 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0479e-05 2024-09-21T21:41:56.342 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.342 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.342 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:56.342 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247803+0000", 2024-09-21T21:41:56.342 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6758999999999997e-05 2024-09-21T21:41:56.342 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:56.342 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:56.342 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:56.342 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.342 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.342 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19771 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6586368~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:41:56.342 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273335+0000", 2024-09-21T21:41:56.342 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.9982673740000001, 2024-09-21T21:41:56.343 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97458947699999998, 2024-09-21T21:41:56.343 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:56.343 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:56.343 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:56.343 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:56.343 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:56.343 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:56.343 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19771 2024-09-21T21:41:56.343 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.343 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:56.343 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.343 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:56.343 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273335+0000", 2024-09-21T21:41:56.343 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:56.343 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.344 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.344 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:56.344 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273335+0000", 2024-09-21T21:41:56.344 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:56.344 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.344 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.344 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:56.344 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273335+0000", 2024-09-21T21:41:56.344 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3599999999999999e-07 2024-09-21T21:41:56.344 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.344 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.344 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:56.344 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273337+0000", 2024-09-21T21:41:56.344 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0789999999999999e-06 2024-09-21T21:41:56.345 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.345 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.345 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:56.345 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273343+0000", 2024-09-21T21:41:56.345 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6280000000000002e-06 2024-09-21T21:41:56.345 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.345 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.345 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:56.345 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273346+0000", 2024-09-21T21:41:56.345 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.507e-06 2024-09-21T21:41:56.345 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.345 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.345 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:56.345 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278206+0000", 2024-09-21T21:41:56.345 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048600969999999999 2024-09-21T21:41:56.346 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.346 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.346 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:41:56.346 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278212+0000", 2024-09-21T21:41:56.346 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2079999999999999e-06 2024-09-21T21:41:56.346 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.346 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.346 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:56.346 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247828+0000", 2024-09-21T21:41:56.347 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.969616547 2024-09-21T21:41:56.347 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.347 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.347 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:56.347 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247881+0000", 2024-09-21T21:41:56.347 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3180999999999998e-05 2024-09-21T21:41:56.347 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.347 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.347 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:56.347 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247924+0000", 2024-09-21T21:41:56.347 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2894e-05 2024-09-21T21:41:56.347 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:56.347 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:56.347 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:56.348 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.348 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.348 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19772 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6557696~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:41:56.348 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273358+0000", 2024-09-21T21:41:56.348 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.9982439940000001, 2024-09-21T21:41:56.348 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97467031800000004, 2024-09-21T21:41:56.348 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:56.348 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:56.348 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:56.348 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:56.348 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:56.348 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:56.348 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19772 2024-09-21T21:41:56.348 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.349 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:56.349 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.349 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:56.349 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273358+0000", 2024-09-21T21:41:56.349 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:56.349 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.349 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.349 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:56.349 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273358+0000", 2024-09-21T21:41:56.349 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:56.349 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.349 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.349 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:56.349 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273358+0000", 2024-09-21T21:41:56.350 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4499999999999998e-07 2024-09-21T21:41:56.350 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.350 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.350 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:56.350 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273360+0000", 2024-09-21T21:41:56.350 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9190000000000002e-06 2024-09-21T21:41:56.350 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.350 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.350 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:56.350 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273366+0000", 2024-09-21T21:41:56.350 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.891e-06 2024-09-21T21:41:56.350 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.350 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.351 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:56.351 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273371+0000", 2024-09-21T21:41:56.351 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1880000000000001e-06 2024-09-21T21:41:56.351 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.351 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.351 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:56.351 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278237+0000", 2024-09-21T21:41:56.351 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.004865472 2024-09-21T21:41:56.351 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.351 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.351 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:41:56.351 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278240+0000", 2024-09-21T21:41:56.351 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9550000000000001e-06 2024-09-21T21:41:56.351 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.351 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.352 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:56.352 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247949+0000", 2024-09-21T21:41:56.352 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.969709127 2024-09-21T21:41:56.352 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.352 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.352 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:56.352 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247990+0000", 2024-09-21T21:41:56.352 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1202000000000003e-05 2024-09-21T21:41:56.352 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.352 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.352 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:56.352 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248028+0000", 2024-09-21T21:41:56.352 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8219000000000001e-05 2024-09-21T21:41:56.353 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:56.353 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:56.353 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:56.353 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.353 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.353 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19773 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6615040~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:41:56.353 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273390+0000", 2024-09-21T21:41:56.353 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.998212133, 2024-09-21T21:41:56.353 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97476086200000001, 2024-09-21T21:41:56.353 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:56.353 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:56.353 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:56.353 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:56.353 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:56.353 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:56.354 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19773 2024-09-21T21:41:56.354 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.354 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:56.354 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.354 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:56.354 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273390+0000", 2024-09-21T21:41:56.354 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:56.354 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.354 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.354 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:56.354 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273390+0000", 2024-09-21T21:41:56.354 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:56.354 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.354 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.355 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:56.355 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273390+0000", 2024-09-21T21:41:56.355 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9999999999999999e-07 2024-09-21T21:41:56.355 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.355 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.355 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:56.355 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273392+0000", 2024-09-21T21:41:56.355 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0640000000000001e-06 2024-09-21T21:41:56.355 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.355 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.355 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:56.356 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273397+0000", 2024-09-21T21:41:56.356 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.113e-06 2024-09-21T21:41:56.356 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.356 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.356 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:56.356 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273401+0000", 2024-09-21T21:41:56.356 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3119999999999998e-06 2024-09-21T21:41:56.356 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.356 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.356 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:56.356 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278271+0000", 2024-09-21T21:41:56.356 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048698880000000002 2024-09-21T21:41:56.356 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.356 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.357 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:41:56.357 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278278+0000", 2024-09-21T21:41:56.357 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1210000000000001e-06 2024-09-21T21:41:56.357 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.357 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.357 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:56.357 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248060+0000", 2024-09-21T21:41:56.357 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96978224999999996 2024-09-21T21:41:56.357 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.357 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.357 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:56.357 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248111+0000", 2024-09-21T21:41:56.357 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.096e-05 2024-09-21T21:41:56.357 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.358 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.358 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:56.358 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248151+0000", 2024-09-21T21:41:56.358 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9854000000000002e-05 2024-09-21T21:41:56.358 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:56.358 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:56.358 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:56.358 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.358 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.358 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19784 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6643712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:41:56.358 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273405+0000", 2024-09-21T21:41:56.358 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.9981974119999997, 2024-09-21T21:41:56.358 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97491798900000004, 2024-09-21T21:41:56.358 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:56.359 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:56.359 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:56.359 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:56.359 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:56.359 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:56.359 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19784 2024-09-21T21:41:56.359 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.359 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:56.359 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.359 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:56.359 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273405+0000", 2024-09-21T21:41:56.359 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:56.359 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.359 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.360 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:56.360 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273405+0000", 2024-09-21T21:41:56.360 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:56.360 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.360 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.360 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:56.360 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273405+0000", 2024-09-21T21:41:56.360 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2100000000000001e-07 2024-09-21T21:41:56.360 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.360 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.360 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:56.360 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273406+0000", 2024-09-21T21:41:56.360 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2049999999999999e-06 2024-09-21T21:41:56.360 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.361 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.361 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:56.361 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273409+0000", 2024-09-21T21:41:56.361 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0599999999999999e-06 2024-09-21T21:41:56.361 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.361 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.361 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:56.361 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273412+0000", 2024-09-21T21:41:56.361 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0730000000000001e-06 2024-09-21T21:41:56.361 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.361 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.361 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:56.361 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278317+0000", 2024-09-21T21:41:56.361 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0049045520000000004 2024-09-21T21:41:56.361 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.362 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.362 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:41:56.362 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278321+0000", 2024-09-21T21:41:56.362 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6249999999999998e-06 2024-09-21T21:41:56.362 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.362 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.362 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:56.362 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248179+0000", 2024-09-21T21:41:56.362 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96985744500000004 2024-09-21T21:41:56.362 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.362 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.362 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:56.362 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248253+0000", 2024-09-21T21:41:56.362 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3829000000000001e-05 2024-09-21T21:41:56.363 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.363 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.363 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:56.363 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248323+0000", 2024-09-21T21:41:56.363 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9979000000000002e-05 2024-09-21T21:41:56.363 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:56.363 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:56.363 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:56.363 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.363 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.364 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19785 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6672384~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:41:56.364 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273415+0000", 2024-09-21T21:41:56.364 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.9981868259999995, 2024-09-21T21:41:56.364 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97500777800000005, 2024-09-21T21:41:56.364 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:56.364 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:56.364 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:56.364 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:56.364 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:56.364 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:56.364 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19785 2024-09-21T21:41:56.364 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.364 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:56.365 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.365 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:56.365 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273415+0000", 2024-09-21T21:41:56.365 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:56.365 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.365 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.365 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:56.365 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273415+0000", 2024-09-21T21:41:56.365 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:56.365 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.365 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.365 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:56.365 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273415+0000", 2024-09-21T21:41:56.366 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0100000000000001e-07 2024-09-21T21:41:56.366 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.366 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.366 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:56.366 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273417+0000", 2024-09-21T21:41:56.366 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.114e-06 2024-09-21T21:41:56.366 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.366 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.366 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:56.366 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273420+0000", 2024-09-21T21:41:56.366 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6770000000000001e-06 2024-09-21T21:41:56.366 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.367 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.367 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:56.367 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273423+0000", 2024-09-21T21:41:56.367 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5450000000000002e-06 2024-09-21T21:41:56.367 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.367 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.367 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:56.367 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278355+0000", 2024-09-21T21:41:56.367 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0049318590000000002 2024-09-21T21:41:56.368 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.368 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.368 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:41:56.368 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278360+0000", 2024-09-21T21:41:56.368 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3079999999999998e-06 2024-09-21T21:41:56.368 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.368 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.368 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:56.368 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248349+0000", 2024-09-21T21:41:56.368 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96998909899999997 2024-09-21T21:41:56.368 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.368 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.368 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:56.369 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248387+0000", 2024-09-21T21:41:56.369 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7657000000000002e-05 2024-09-21T21:41:56.369 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.369 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.369 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:56.369 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248423+0000", 2024-09-21T21:41:56.369 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6318e-05 2024-09-21T21:41:56.369 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:56.369 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:56.369 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:56.369 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.369 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.369 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19842 3.d 3:b7fa2ca0:::rbd_data.112157a479c.000000000000004d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:41:56.370 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273426+0000", 2024-09-21T21:41:56.370 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.9981757739999999, 2024-09-21T21:41:56.370 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.98136526800000001, 2024-09-21T21:41:56.370 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:56.370 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:56.370 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:41:56.370 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:56.370 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:56.370 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:56.370 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19842 2024-09-21T21:41:56.370 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.370 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:56.370 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.371 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:56.371 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273426+0000", 2024-09-21T21:41:56.371 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:56.371 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.371 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.371 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:56.371 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273426+0000", 2024-09-21T21:41:56.371 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:56.371 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.371 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.371 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:56.371 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273427+0000", 2024-09-21T21:41:56.371 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.72e-07 2024-09-21T21:41:56.372 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.372 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.372 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:56.372 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275836+0000", 2024-09-21T21:41:56.372 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00240909 2024-09-21T21:41:56.372 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.372 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.372 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:56.372 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275844+0000", 2024-09-21T21:41:56.372 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3750000000000003e-06 2024-09-21T21:41:56.372 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.372 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.372 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:56.372 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275849+0000", 2024-09-21T21:41:56.373 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0100000000000003e-06 2024-09-21T21:41:56.373 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.373 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.373 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:56.373 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278394+0000", 2024-09-21T21:41:56.373 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025449890000000001 2024-09-21T21:41:56.373 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.373 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.373 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:41:56.373 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278402+0000", 2024-09-21T21:41:56.373 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6359999999999994e-06 2024-09-21T21:41:56.373 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.373 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.374 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:56.374 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248454+0000", 2024-09-21T21:41:56.374 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97005259499999996 2024-09-21T21:41:56.374 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.374 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.374 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:56.374 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248576+0000", 2024-09-21T21:41:56.374 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000122127 2024-09-21T21:41:56.374 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.374 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.374 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 2", 2024-09-21T21:41:56.375 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248844+0000", 2024-09-21T21:41:56.375 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00026784699999999999 2024-09-21T21:41:56.375 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.375 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.375 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:41:56.375 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.252487+0000", 2024-09-21T21:41:56.375 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0036427920000000002 2024-09-21T21:41:56.375 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.375 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.375 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:41:56.375 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.254691+0000", 2024-09-21T21:41:56.375 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002203899 2024-09-21T21:41:56.375 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.376 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.376 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:41:56.376 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.254748+0000", 2024-09-21T21:41:56.376 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7163000000000001e-05 2024-09-21T21:41:56.376 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.376 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.376 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:56.376 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.254792+0000", 2024-09-21T21:41:56.376 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3473000000000002e-05 2024-09-21T21:41:56.376 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:56.376 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:56.376 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:56.376 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.376 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.377 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19843 3.d 3:b7fa2ca0:::rbd_data.112157a479c.000000000000004d:head [write 0~0] snapc 21=[21] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:41:56.377 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.275860+0000", 2024-09-21T21:41:56.377 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 4.9957418740000001, 2024-09-21T21:41:56.377 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.98183651999999999, 2024-09-21T21:41:56.377 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:56.377 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:56.377 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:41:56.377 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:56.377 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:41:56.377 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:41:56.377 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19843 2024-09-21T21:41:56.377 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.377 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:56.377 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.378 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:56.378 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275860+0000", 2024-09-21T21:41:56.378 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:56.378 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.378 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.378 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:56.378 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275860+0000", 2024-09-21T21:41:56.378 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:56.378 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.378 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.378 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:56.378 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275869+0000", 2024-09-21T21:41:56.378 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6300000000000004e-06 2024-09-21T21:41:56.378 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.379 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.379 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:56.379 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275871+0000", 2024-09-21T21:41:56.379 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2060000000000001e-06 2024-09-21T21:41:56.379 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.379 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.379 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:56.379 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275876+0000", 2024-09-21T21:41:56.379 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8180000000000003e-06 2024-09-21T21:41:56.379 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.379 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.379 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:56.379 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275880+0000", 2024-09-21T21:41:56.379 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1829999999999996e-06 2024-09-21T21:41:56.380 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.380 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.380 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:56.380 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278416+0000", 2024-09-21T21:41:56.380 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025361149999999998 2024-09-21T21:41:56.380 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.380 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.380 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:41:56.380 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278419+0000", 2024-09-21T21:41:56.380 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.163e-06 2024-09-21T21:41:56.380 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.380 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.380 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:56.380 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.250249+0000", 2024-09-21T21:41:56.381 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97182984299999997 2024-09-21T21:41:56.381 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.381 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.381 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:56.381 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.250321+0000", 2024-09-21T21:41:56.381 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1656000000000006e-05 2024-09-21T21:41:56.381 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.381 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.381 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 2", 2024-09-21T21:41:56.381 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.250419+0000", 2024-09-21T21:41:56.381 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.8277999999999997e-05 2024-09-21T21:41:56.381 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.381 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.381 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:41:56.382 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.252516+0000", 2024-09-21T21:41:56.382 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002097224 2024-09-21T21:41:56.382 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.382 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.382 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:41:56.382 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.257643+0000", 2024-09-21T21:41:56.382 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0051262319999999997 2024-09-21T21:41:56.382 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.382 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.382 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:41:56.382 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.257673+0000", 2024-09-21T21:41:56.382 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0023e-05 2024-09-21T21:41:56.382 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.382 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.383 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:56.383 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.257697+0000", 2024-09-21T21:41:56.383 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4148999999999999e-05 2024-09-21T21:41:56.383 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:56.383 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:56.383 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:56.383 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:56.383 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:56.383 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:56.384 INFO:teuthology.orchestra.run.smithi164.stdout:truncating image image_client.0-clone31 from 0x7da1000 (overlap 0x14aa400) to 0x211000 2024-09-21T21:41:56.385 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:56.383+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:56.385 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T21:41:56.485 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:56.483+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:56.489 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:56.489 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:41:56.490 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-21T21:41:56.490 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:56.498 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T21:41:56.586 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:56.584+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:56.603 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:56.603 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:41:56.603 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:41:56.603 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:41:56.603 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:56.611 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T21:41:56.652 INFO:teuthology.orchestra.run.smithi164.stdout:checking clone #30, image image_client.0-clone30 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent31 2024-09-21T21:41:56.686 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:56.684+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:56.732 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:56.732 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-21T21:41:56.732 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-21T21:41:56.732 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:41:56.732 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.732 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6974 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5246976~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:56.732 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.922932+0000", 2024-09-21T21:41:56.732 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 26.806719697999998, 2024-09-21T21:41:56.732 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041773352999999999, 2024-09-21T21:41:56.732 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:56.733 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:56.733 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:56.733 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:56.733 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:56.733 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:56.733 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6974 2024-09-21T21:41:56.733 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.733 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:56.733 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.733 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:56.733 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922932+0000", 2024-09-21T21:41:56.733 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:56.734 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.734 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.734 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:56.734 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922932+0000", 2024-09-21T21:41:56.734 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:56.734 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.734 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.734 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:56.734 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922933+0000", 2024-09-21T21:41:56.734 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.46e-07 2024-09-21T21:41:56.734 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.734 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.734 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:56.734 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922940+0000", 2024-09-21T21:41:56.735 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6140000000000003e-06 2024-09-21T21:41:56.735 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.735 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.735 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:56.735 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922949+0000", 2024-09-21T21:41:56.735 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6899999999999998e-06 2024-09-21T21:41:56.735 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.735 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.735 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:56.735 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922955+0000", 2024-09-21T21:41:56.735 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5140000000000001e-06 2024-09-21T21:41:56.735 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.735 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.735 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:56.736 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964547+0000", 2024-09-21T21:41:56.736 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041592854999999998 2024-09-21T21:41:56.736 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.736 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.736 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:56.736 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964617+0000", 2024-09-21T21:41:56.736 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9058000000000002e-05 2024-09-21T21:41:56.736 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.736 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.736 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:56.736 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964706+0000", 2024-09-21T21:41:56.736 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9276000000000004e-05 2024-09-21T21:41:56.736 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:56.736 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:56.737 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:56.737 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.737 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.737 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6976 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5261312~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:56.737 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.922961+0000", 2024-09-21T21:41:56.737 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 26.806690653, 2024-09-21T21:41:56.737 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041863648000000003, 2024-09-21T21:41:56.737 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:56.737 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:56.737 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:56.737 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:56.737 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:56.737 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:56.737 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6976 2024-09-21T21:41:56.737 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.737 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:56.738 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.738 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:56.738 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922961+0000", 2024-09-21T21:41:56.738 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:56.738 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.738 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.738 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:56.738 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922961+0000", 2024-09-21T21:41:56.738 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:56.738 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.738 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.738 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:56.738 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922962+0000", 2024-09-21T21:41:56.739 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.89e-07 2024-09-21T21:41:56.739 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.739 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.739 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:56.739 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922964+0000", 2024-09-21T21:41:56.739 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1270000000000001e-06 2024-09-21T21:41:56.739 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.739 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.739 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:56.739 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922970+0000", 2024-09-21T21:41:56.739 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8030000000000002e-06 2024-09-21T21:41:56.739 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.739 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.739 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:56.740 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922979+0000", 2024-09-21T21:41:56.740 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5559999999999996e-06 2024-09-21T21:41:56.740 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.740 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.740 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:56.740 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964734+0000", 2024-09-21T21:41:56.740 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041754415000000003 2024-09-21T21:41:56.740 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.740 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.740 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:56.740 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964775+0000", 2024-09-21T21:41:56.740 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0806999999999997e-05 2024-09-21T21:41:56.740 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.741 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.741 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:56.741 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964825+0000", 2024-09-21T21:41:56.741 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0550999999999998e-05 2024-09-21T21:41:56.741 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:56.741 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:56.741 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:56.741 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.741 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.741 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6978 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5275648~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:56.741 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.922987+0000", 2024-09-21T21:41:56.741 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 26.806664867999999, 2024-09-21T21:41:56.741 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042043876000000001, 2024-09-21T21:41:56.742 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:56.742 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:56.742 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:56.742 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:56.742 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:56.742 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:56.742 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6978 2024-09-21T21:41:56.742 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.742 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:56.742 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.742 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:56.742 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922987+0000", 2024-09-21T21:41:56.742 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:56.742 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.743 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.743 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:56.743 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922987+0000", 2024-09-21T21:41:56.743 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:56.743 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.743 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.743 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:56.743 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922988+0000", 2024-09-21T21:41:56.743 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1199999999999998e-07 2024-09-21T21:41:56.743 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.743 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.743 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:56.743 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922990+0000", 2024-09-21T21:41:56.743 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.142e-06 2024-09-21T21:41:56.744 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.744 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.744 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:56.744 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922997+0000", 2024-09-21T21:41:56.744 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.199e-06 2024-09-21T21:41:56.744 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.744 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.744 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:56.744 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923004+0000", 2024-09-21T21:41:56.744 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3599999999999998e-06 2024-09-21T21:41:56.744 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.744 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.744 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:56.744 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964856+0000", 2024-09-21T21:41:56.745 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041851521000000003 2024-09-21T21:41:56.745 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.745 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.745 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:56.745 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964944+0000", 2024-09-21T21:41:56.745 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7859e-05 2024-09-21T21:41:56.745 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.745 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.745 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:56.745 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965031+0000", 2024-09-21T21:41:56.745 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7383e-05 2024-09-21T21:41:56.745 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:56.745 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:56.745 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:56.745 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.746 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.746 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6980 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5289984~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:56.746 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923011+0000", 2024-09-21T21:41:56.746 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 26.806641531, 2024-09-21T21:41:56.746 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042139337999999998, 2024-09-21T21:41:56.746 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:56.746 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:56.746 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:56.746 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:56.746 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:56.746 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:56.746 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6980 2024-09-21T21:41:56.746 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.746 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:56.747 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.747 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:56.747 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923011+0000", 2024-09-21T21:41:56.747 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:56.747 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.747 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.747 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:56.747 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923011+0000", 2024-09-21T21:41:56.747 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:56.747 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.747 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.747 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:56.747 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923011+0000", 2024-09-21T21:41:56.747 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6699999999999999e-07 2024-09-21T21:41:56.748 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.748 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.748 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:56.748 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923013+0000", 2024-09-21T21:41:56.748 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0279999999999999e-06 2024-09-21T21:41:56.748 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.748 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.748 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:56.748 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923018+0000", 2024-09-21T21:41:56.748 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4790000000000004e-06 2024-09-21T21:41:56.748 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.748 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.748 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:56.748 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923029+0000", 2024-09-21T21:41:56.748 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0601999999999999e-05 2024-09-21T21:41:56.749 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.749 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.749 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:56.749 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965059+0000", 2024-09-21T21:41:56.749 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042029947999999998 2024-09-21T21:41:56.749 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.749 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.749 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:56.749 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965100+0000", 2024-09-21T21:41:56.749 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1170000000000001e-05 2024-09-21T21:41:56.749 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.749 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.749 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:56.749 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965150+0000", 2024-09-21T21:41:56.750 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9743999999999997e-05 2024-09-21T21:41:56.750 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:56.750 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:56.750 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:56.750 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.750 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.750 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6982 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5304320~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:56.750 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923036+0000", 2024-09-21T21:41:56.750 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 26.806615709999999, 2024-09-21T21:41:56.750 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042225666000000002, 2024-09-21T21:41:56.750 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:56.750 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:56.750 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:56.751 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:56.751 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:56.751 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:56.751 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6982 2024-09-21T21:41:56.751 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.751 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:56.751 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.751 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:56.751 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923036+0000", 2024-09-21T21:41:56.751 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:56.751 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.751 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.751 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:56.751 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923036+0000", 2024-09-21T21:41:56.752 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:56.752 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.752 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.752 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:56.752 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923037+0000", 2024-09-21T21:41:56.752 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2800000000000002e-07 2024-09-21T21:41:56.752 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.752 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.752 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:56.752 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923039+0000", 2024-09-21T21:41:56.752 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1500000000000002e-06 2024-09-21T21:41:56.752 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.752 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.752 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:56.752 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923049+0000", 2024-09-21T21:41:56.753 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0236e-05 2024-09-21T21:41:56.753 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.753 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.753 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:56.753 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923056+0000", 2024-09-21T21:41:56.753 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2540000000000003e-06 2024-09-21T21:41:56.753 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.753 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.753 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:56.753 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965173+0000", 2024-09-21T21:41:56.753 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042116686 2024-09-21T21:41:56.753 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.753 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.753 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:56.753 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965215+0000", 2024-09-21T21:41:56.754 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1696000000000002e-05 2024-09-21T21:41:56.754 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.754 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.754 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:56.754 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965262+0000", 2024-09-21T21:41:56.754 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7216000000000003e-05 2024-09-21T21:41:56.754 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:56.754 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:56.754 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:56.754 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.754 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.754 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6984 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5318656~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:56.754 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923063+0000", 2024-09-21T21:41:56.754 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 26.806589056, 2024-09-21T21:41:56.755 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042318651999999998, 2024-09-21T21:41:56.755 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:56.755 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:56.755 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:56.755 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:56.755 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:56.755 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:56.755 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6984 2024-09-21T21:41:56.755 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.755 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:56.755 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.755 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:56.755 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923063+0000", 2024-09-21T21:41:56.755 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:56.755 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.756 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.756 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:56.756 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923063+0000", 2024-09-21T21:41:56.756 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:56.756 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.756 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.756 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:56.756 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923064+0000", 2024-09-21T21:41:56.756 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2e-07 2024-09-21T21:41:56.756 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.756 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.756 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:56.756 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923066+0000", 2024-09-21T21:41:56.756 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.057e-06 2024-09-21T21:41:56.757 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.757 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.757 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:56.757 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923071+0000", 2024-09-21T21:41:56.757 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8810000000000001e-06 2024-09-21T21:41:56.757 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.757 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.757 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:56.757 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923078+0000", 2024-09-21T21:41:56.757 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5479999999999998e-06 2024-09-21T21:41:56.757 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.757 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.757 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:56.757 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965288+0000", 2024-09-21T21:41:56.757 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042209576999999998 2024-09-21T21:41:56.758 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.758 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.758 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:56.758 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965332+0000", 2024-09-21T21:41:56.758 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4366999999999997e-05 2024-09-21T21:41:56.758 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.758 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.758 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:56.758 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965382+0000", 2024-09-21T21:41:56.758 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9802000000000002e-05 2024-09-21T21:41:56.758 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:56.758 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:56.758 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:56.758 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.759 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.759 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6987 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5332992~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:56.759 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923085+0000", 2024-09-21T21:41:56.759 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 26.806567530999999, 2024-09-21T21:41:56.759 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042412103999999999, 2024-09-21T21:41:56.759 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:56.759 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:56.759 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:56.759 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:56.759 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:56.759 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:56.759 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6987 2024-09-21T21:41:56.759 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.760 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:56.760 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.760 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:56.760 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923085+0000", 2024-09-21T21:41:56.760 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:56.760 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.760 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.760 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:56.760 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923085+0000", 2024-09-21T21:41:56.760 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:56.760 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.760 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.760 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:56.760 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923085+0000", 2024-09-21T21:41:56.760 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.89e-07 2024-09-21T21:41:56.761 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.761 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.761 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:56.761 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923087+0000", 2024-09-21T21:41:56.761 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0810000000000001e-06 2024-09-21T21:41:56.761 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.761 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.761 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:56.761 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923098+0000", 2024-09-21T21:41:56.761 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1299e-05 2024-09-21T21:41:56.761 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.761 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.761 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:56.761 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923104+0000", 2024-09-21T21:41:56.761 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9100000000000002e-06 2024-09-21T21:41:56.762 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.762 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.762 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:56.762 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965404+0000", 2024-09-21T21:41:56.762 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042299622000000002 2024-09-21T21:41:56.762 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.762 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.762 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:56.762 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965445+0000", 2024-09-21T21:41:56.762 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0701000000000002e-05 2024-09-21T21:41:56.762 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.762 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.762 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:56.762 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965497+0000", 2024-09-21T21:41:56.763 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2101999999999997e-05 2024-09-21T21:41:56.763 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:56.763 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:56.763 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:56.763 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.763 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.763 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6989 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5347328~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:56.763 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923111+0000", 2024-09-21T21:41:56.763 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 26.806541191000001, 2024-09-21T21:41:56.763 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042505188999999999, 2024-09-21T21:41:56.763 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:56.763 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:56.763 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:56.763 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:56.764 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:56.764 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:56.764 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6989 2024-09-21T21:41:56.764 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.764 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:56.764 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.764 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:56.764 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923111+0000", 2024-09-21T21:41:56.764 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:56.764 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.764 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.764 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:56.764 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923111+0000", 2024-09-21T21:41:56.764 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:56.764 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.765 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.765 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:56.765 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923111+0000", 2024-09-21T21:41:56.765 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4499999999999997e-07 2024-09-21T21:41:56.765 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.765 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.765 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:56.765 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923114+0000", 2024-09-21T21:41:56.765 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1569999999999998e-06 2024-09-21T21:41:56.765 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.765 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.765 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:56.765 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923120+0000", 2024-09-21T21:41:56.765 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1759999999999998e-06 2024-09-21T21:41:56.766 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.766 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.766 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:56.766 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923125+0000", 2024-09-21T21:41:56.766 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2560000000000002e-06 2024-09-21T21:41:56.766 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.766 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.766 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:56.766 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965523+0000", 2024-09-21T21:41:56.766 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042398094999999997 2024-09-21T21:41:56.766 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.766 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.766 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:56.766 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965566+0000", 2024-09-21T21:41:56.767 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3393999999999998e-05 2024-09-21T21:41:56.767 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.767 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.767 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:56.767 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965616+0000", 2024-09-21T21:41:56.767 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9666000000000002e-05 2024-09-21T21:41:56.767 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:56.767 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:56.767 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:56.767 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.767 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.767 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6991 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5361664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:56.767 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923132+0000", 2024-09-21T21:41:56.767 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 26.806519902000002, 2024-09-21T21:41:56.767 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042600779999999998, 2024-09-21T21:41:56.768 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:56.768 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:56.768 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:56.768 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:56.768 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:56.768 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:56.768 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6991 2024-09-21T21:41:56.768 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.768 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:56.768 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.768 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:56.768 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923132+0000", 2024-09-21T21:41:56.768 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:56.768 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.769 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.769 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:56.769 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923132+0000", 2024-09-21T21:41:56.769 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:56.769 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.769 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.769 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:56.769 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923132+0000", 2024-09-21T21:41:56.769 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4200000000000002e-07 2024-09-21T21:41:56.769 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.769 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.769 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:56.769 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923134+0000", 2024-09-21T21:41:56.769 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2270000000000001e-06 2024-09-21T21:41:56.769 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.770 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.770 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:56.770 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923137+0000", 2024-09-21T21:41:56.770 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1930000000000002e-06 2024-09-21T21:41:56.770 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.770 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.770 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:56.770 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923139+0000", 2024-09-21T21:41:56.770 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4710000000000001e-06 2024-09-21T21:41:56.770 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.770 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.770 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:56.770 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965639+0000", 2024-09-21T21:41:56.770 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.04249965 2024-09-21T21:41:56.771 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.771 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.771 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:56.771 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965679+0000", 2024-09-21T21:41:56.771 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9852999999999999e-05 2024-09-21T21:41:56.771 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.771 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.771 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:56.771 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965733+0000", 2024-09-21T21:41:56.771 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4144000000000002e-05 2024-09-21T21:41:56.771 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:56.771 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:56.771 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:56.771 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.771 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.772 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6993 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5384192~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:56.772 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923143+0000", 2024-09-21T21:41:56.772 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 26.806509108, 2024-09-21T21:41:56.772 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042708182999999997, 2024-09-21T21:41:56.772 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:56.772 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:56.772 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:56.772 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:56.772 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:56.772 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:56.772 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6993 2024-09-21T21:41:56.772 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.772 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:56.772 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.772 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:56.773 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923143+0000", 2024-09-21T21:41:56.773 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:56.773 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.773 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.773 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:56.773 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923143+0000", 2024-09-21T21:41:56.773 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:56.773 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.773 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.773 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:56.773 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923143+0000", 2024-09-21T21:41:56.773 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0800000000000001e-07 2024-09-21T21:41:56.773 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.773 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.774 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:56.774 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923144+0000", 2024-09-21T21:41:56.774 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.009e-06 2024-09-21T21:41:56.774 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.774 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.774 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:56.774 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923148+0000", 2024-09-21T21:41:56.774 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.737e-06 2024-09-21T21:41:56.774 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.774 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.774 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:56.774 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923150+0000", 2024-09-21T21:41:56.774 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4669999999999998e-06 2024-09-21T21:41:56.774 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.775 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.775 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:56.775 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965760+0000", 2024-09-21T21:41:56.775 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042609399999999999 2024-09-21T21:41:56.775 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.775 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.775 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:56.775 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965801+0000", 2024-09-21T21:41:56.775 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1399999999999997e-05 2024-09-21T21:41:56.775 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.775 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.775 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:56.775 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965851+0000", 2024-09-21T21:41:56.775 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9962e-05 2024-09-21T21:41:56.776 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:56.776 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:56.776 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:56.776 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.776 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.776 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6995 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5398528~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:56.776 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923153+0000", 2024-09-21T21:41:56.776 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 26.806498743999999, 2024-09-21T21:41:56.776 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042828034000000001, 2024-09-21T21:41:56.776 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:56.776 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:56.776 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:56.776 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:56.776 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:56.777 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:56.777 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6995 2024-09-21T21:41:56.777 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.777 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:56.777 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.777 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:56.777 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923153+0000", 2024-09-21T21:41:56.777 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:56.777 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.777 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.777 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:56.777 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923153+0000", 2024-09-21T21:41:56.777 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:56.777 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.778 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.778 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:56.778 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923154+0000", 2024-09-21T21:41:56.778 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2600000000000001e-07 2024-09-21T21:41:56.778 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.778 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.778 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:56.778 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923158+0000", 2024-09-21T21:41:56.778 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0559999999999998e-06 2024-09-21T21:41:56.778 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.778 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.778 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:56.778 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923161+0000", 2024-09-21T21:41:56.778 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6519999999999999e-06 2024-09-21T21:41:56.779 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.779 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.779 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:56.779 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923170+0000", 2024-09-21T21:41:56.779 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5720000000000003e-06 2024-09-21T21:41:56.779 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.779 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.779 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:56.779 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965887+0000", 2024-09-21T21:41:56.779 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042716863000000001 2024-09-21T21:41:56.779 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.779 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.779 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:56.779 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965927+0000", 2024-09-21T21:41:56.780 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0346999999999998e-05 2024-09-21T21:41:56.780 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.780 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.780 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:56.780 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965981+0000", 2024-09-21T21:41:56.780 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4317999999999997e-05 2024-09-21T21:41:56.780 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:56.780 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:56.780 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:56.780 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.780 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.780 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6997 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5412864~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:56.780 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923174+0000", 2024-09-21T21:41:56.780 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 26.806478589000001, 2024-09-21T21:41:56.780 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042928296999999997, 2024-09-21T21:41:56.781 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:56.781 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:56.781 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:56.781 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:56.781 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:56.781 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:56.781 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6997 2024-09-21T21:41:56.781 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.781 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:56.781 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.781 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:56.781 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923174+0000", 2024-09-21T21:41:56.781 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:56.781 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.781 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.782 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:56.782 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923174+0000", 2024-09-21T21:41:56.782 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:56.782 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.782 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.782 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:56.782 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923174+0000", 2024-09-21T21:41:56.782 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.67e-07 2024-09-21T21:41:56.782 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.782 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.782 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:56.782 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923178+0000", 2024-09-21T21:41:56.782 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1330000000000001e-06 2024-09-21T21:41:56.782 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.783 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.783 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:56.783 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923182+0000", 2024-09-21T21:41:56.783 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.794e-06 2024-09-21T21:41:56.783 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.783 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.783 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:56.783 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923184+0000", 2024-09-21T21:41:56.783 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.435e-06 2024-09-21T21:41:56.783 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.783 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.783 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:56.783 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966009+0000", 2024-09-21T21:41:56.783 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042824327000000002 2024-09-21T21:41:56.784 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.784 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.784 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:56.784 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966052+0000", 2024-09-21T21:41:56.784 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3034000000000002e-05 2024-09-21T21:41:56.784 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.784 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.784 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:56.784 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966102+0000", 2024-09-21T21:41:56.784 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0306999999999998e-05 2024-09-21T21:41:56.784 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:56.784 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:56.784 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:56.784 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.784 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.785 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6999 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5427200~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:56.785 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923187+0000", 2024-09-21T21:41:56.785 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 26.806464860999998, 2024-09-21T21:41:56.785 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043104267000000002, 2024-09-21T21:41:56.785 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:56.785 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:56.785 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:56.785 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:56.785 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:56.785 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:56.785 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6999 2024-09-21T21:41:56.785 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.785 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:56.785 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.786 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:56.786 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923187+0000", 2024-09-21T21:41:56.786 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:56.786 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.786 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.786 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:56.786 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923187+0000", 2024-09-21T21:41:56.786 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:56.786 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.786 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.786 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:56.786 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923188+0000", 2024-09-21T21:41:56.786 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3999999999999998e-07 2024-09-21T21:41:56.786 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.787 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.787 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:56.787 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923189+0000", 2024-09-21T21:41:56.787 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4169999999999999e-06 2024-09-21T21:41:56.787 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.787 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.787 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:56.787 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923194+0000", 2024-09-21T21:41:56.787 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1819999999999997e-06 2024-09-21T21:41:56.787 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.787 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.787 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:56.787 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923198+0000", 2024-09-21T21:41:56.787 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.143e-06 2024-09-21T21:41:56.787 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.788 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.788 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:56.788 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966135+0000", 2024-09-21T21:41:56.788 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042936297999999998 2024-09-21T21:41:56.788 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.788 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.788 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:56.788 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966204+0000", 2024-09-21T21:41:56.788 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8989e-05 2024-09-21T21:41:56.788 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.788 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.788 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:56.788 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966292+0000", 2024-09-21T21:41:56.789 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7997999999999993e-05 2024-09-21T21:41:56.789 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:56.789 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:56.789 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:56.789 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.789 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.789 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7001 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5441536~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:56.789 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923211+0000", 2024-09-21T21:41:56.789 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 26.806441069999998, 2024-09-21T21:41:56.789 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043196518000000003, 2024-09-21T21:41:56.789 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:56.789 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:56.789 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:56.789 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:56.790 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:56.790 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:56.790 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7001 2024-09-21T21:41:56.790 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.790 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:56.790 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.790 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:56.790 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923211+0000", 2024-09-21T21:41:56.790 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:56.790 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.790 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.790 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:56.790 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923211+0000", 2024-09-21T21:41:56.790 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:56.791 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.791 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.791 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:56.791 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923212+0000", 2024-09-21T21:41:56.791 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3900000000000005e-07 2024-09-21T21:41:56.791 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.791 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.791 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:56.791 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923214+0000", 2024-09-21T21:41:56.791 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9970000000000001e-06 2024-09-21T21:41:56.791 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.791 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.791 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:56.791 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923219+0000", 2024-09-21T21:41:56.791 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4310000000000002e-06 2024-09-21T21:41:56.792 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.792 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.792 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:56.792 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923224+0000", 2024-09-21T21:41:56.792 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6410000000000001e-06 2024-09-21T21:41:56.792 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.792 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.792 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:56.792 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966318+0000", 2024-09-21T21:41:56.792 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043093842 2024-09-21T21:41:56.792 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.792 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.792 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:56.792 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966359+0000", 2024-09-21T21:41:56.792 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1260999999999997e-05 2024-09-21T21:41:56.793 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.793 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.793 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:56.793 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966408+0000", 2024-09-21T21:41:56.793 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8807000000000002e-05 2024-09-21T21:41:56.793 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:56.793 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:56.793 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:56.793 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.793 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.793 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7002 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5455872~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:56.793 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923229+0000", 2024-09-21T21:41:56.793 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 26.806422659999999, 2024-09-21T21:41:56.794 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043364568999999999, 2024-09-21T21:41:56.794 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:56.794 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:56.794 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:56.794 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:56.794 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:56.794 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:56.794 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7002 2024-09-21T21:41:56.794 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.794 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:56.794 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.794 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:56.794 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923229+0000", 2024-09-21T21:41:56.794 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:56.795 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.795 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.795 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:56.795 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923229+0000", 2024-09-21T21:41:56.795 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:56.795 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.795 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.795 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:56.795 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923230+0000", 2024-09-21T21:41:56.795 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4700000000000002e-07 2024-09-21T21:41:56.795 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.795 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.795 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:56.796 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923232+0000", 2024-09-21T21:41:56.796 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0719999999999998e-06 2024-09-21T21:41:56.796 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.796 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.796 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:56.796 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923237+0000", 2024-09-21T21:41:56.796 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2290000000000002e-06 2024-09-21T21:41:56.796 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.796 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.796 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:56.796 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923245+0000", 2024-09-21T21:41:56.796 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2959999999999999e-06 2024-09-21T21:41:56.796 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.796 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.796 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:56.797 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966438+0000", 2024-09-21T21:41:56.797 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043193027000000002 2024-09-21T21:41:56.797 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.797 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.797 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:56.797 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966506+0000", 2024-09-21T21:41:56.797 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7707999999999996e-05 2024-09-21T21:41:56.797 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.797 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.797 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:56.797 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966594+0000", 2024-09-21T21:41:56.797 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.789e-05 2024-09-21T21:41:56.797 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:56.798 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:56.798 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:56.798 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.798 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.798 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7005 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5470208~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:56.798 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923254+0000", 2024-09-21T21:41:56.798 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 26.806398429000001, 2024-09-21T21:41:56.798 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043457176, 2024-09-21T21:41:56.798 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:56.798 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:56.798 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:56.798 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:56.798 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:56.798 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:56.798 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7005 2024-09-21T21:41:56.799 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.799 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:56.799 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.799 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:56.799 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923254+0000", 2024-09-21T21:41:56.799 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:56.799 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.799 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.799 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:56.799 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923254+0000", 2024-09-21T21:41:56.799 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:56.799 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.799 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.799 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:56.800 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923254+0000", 2024-09-21T21:41:56.800 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4700000000000002e-07 2024-09-21T21:41:56.800 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.800 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.800 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:56.800 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923256+0000", 2024-09-21T21:41:56.800 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.007e-06 2024-09-21T21:41:56.800 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.800 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.800 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:56.800 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923266+0000", 2024-09-21T21:41:56.800 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.4700000000000008e-06 2024-09-21T21:41:56.800 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.800 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.801 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:56.801 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923271+0000", 2024-09-21T21:41:56.801 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2809999999999999e-06 2024-09-21T21:41:56.801 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.801 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.801 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:56.801 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966619+0000", 2024-09-21T21:41:56.801 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043347951000000003 2024-09-21T21:41:56.801 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.801 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.801 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:56.801 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966661+0000", 2024-09-21T21:41:56.801 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2614999999999998e-05 2024-09-21T21:41:56.801 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.802 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.802 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:56.802 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966711+0000", 2024-09-21T21:41:56.802 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9505000000000001e-05 2024-09-21T21:41:56.802 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:56.802 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:56.802 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:56.802 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.802 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.802 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7008 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5484544~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:56.802 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923278+0000", 2024-09-21T21:41:56.802 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 26.806373782000001, 2024-09-21T21:41:56.802 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043631071, 2024-09-21T21:41:56.803 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:56.803 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:56.803 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:56.803 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:56.803 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:56.803 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:56.803 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7008 2024-09-21T21:41:56.803 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.803 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:56.803 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.803 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:56.803 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923278+0000", 2024-09-21T21:41:56.803 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:56.803 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.804 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.804 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:56.804 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923278+0000", 2024-09-21T21:41:56.804 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:56.804 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.804 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.804 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:56.804 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923279+0000", 2024-09-21T21:41:56.804 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.77e-07 2024-09-21T21:41:56.804 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.804 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.804 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:56.804 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923281+0000", 2024-09-21T21:41:56.805 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0219999999999999e-06 2024-09-21T21:41:56.805 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.805 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.805 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:56.805 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923287+0000", 2024-09-21T21:41:56.805 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7799999999999997e-06 2024-09-21T21:41:56.805 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.805 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.805 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:56.805 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923291+0000", 2024-09-21T21:41:56.805 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1699999999999999e-06 2024-09-21T21:41:56.805 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.805 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.806 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:56.806 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966742+0000", 2024-09-21T21:41:56.806 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043451344000000003 2024-09-21T21:41:56.806 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.806 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.806 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:56.806 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966809+0000", 2024-09-21T21:41:56.806 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7070999999999995e-05 2024-09-21T21:41:56.806 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.806 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.806 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:56.806 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966909+0000", 2024-09-21T21:41:56.806 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000100307 2024-09-21T21:41:56.806 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:56.807 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:56.807 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:56.807 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.807 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.807 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7010 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5498880~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:56.807 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923301+0000", 2024-09-21T21:41:56.807 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 26.806350887000001, 2024-09-21T21:41:56.807 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043717830999999999, 2024-09-21T21:41:56.807 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:56.807 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:56.807 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:56.807 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:56.807 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:56.807 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:56.807 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7010 2024-09-21T21:41:56.808 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.808 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:56.808 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.808 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:56.808 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923301+0000", 2024-09-21T21:41:56.808 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:56.808 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.808 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.808 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:56.808 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923301+0000", 2024-09-21T21:41:56.808 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:56.808 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.808 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.808 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:56.809 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923302+0000", 2024-09-21T21:41:56.809 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.41e-07 2024-09-21T21:41:56.809 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.809 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.809 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:56.809 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923304+0000", 2024-09-21T21:41:56.809 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0990000000000001e-06 2024-09-21T21:41:56.809 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.809 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.809 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:56.809 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923310+0000", 2024-09-21T21:41:56.809 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3860000000000004e-06 2024-09-21T21:41:56.809 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.809 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.810 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:56.810 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923315+0000", 2024-09-21T21:41:56.810 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8459999999999999e-06 2024-09-21T21:41:56.810 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.810 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.810 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:56.810 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966938+0000", 2024-09-21T21:41:56.810 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043622693999999997 2024-09-21T21:41:56.810 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.810 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.810 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:56.810 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966975+0000", 2024-09-21T21:41:56.810 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7608999999999999e-05 2024-09-21T21:41:56.810 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.811 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.811 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:56.811 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.967019+0000", 2024-09-21T21:41:56.811 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3856000000000002e-05 2024-09-21T21:41:56.811 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:56.811 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:56.811 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:56.811 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.811 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.811 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7354 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 2107392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:41:56.811 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.936932+0000", 2024-09-21T21:41:56.811 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 26.792720031999998, 2024-09-21T21:41:56.811 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041607708, 2024-09-21T21:41:56.811 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:56.811 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:56.811 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:56.812 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:56.812 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:41:56.812 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:41:56.812 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7354 2024-09-21T21:41:56.812 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.812 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:56.812 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.812 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:56.812 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.936932+0000", 2024-09-21T21:41:56.812 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:56.812 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.812 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.812 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:56.812 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.936932+0000", 2024-09-21T21:41:56.813 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:56.813 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.813 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.813 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:56.813 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.937276+0000", 2024-09-21T21:41:56.813 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00034344200000000001 2024-09-21T21:41:56.813 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.813 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.813 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:56.813 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.937279+0000", 2024-09-21T21:41:56.813 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.844e-06 2024-09-21T21:41:56.813 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.813 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.813 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:56.813 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.937288+0000", 2024-09-21T21:41:56.814 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2409999999999995e-06 2024-09-21T21:41:56.814 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.814 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.814 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:56.814 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.937293+0000", 2024-09-21T21:41:56.814 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6559999999999997e-06 2024-09-21T21:41:56.814 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.814 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.814 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:56.814 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.978448+0000", 2024-09-21T21:41:56.814 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041154621000000002 2024-09-21T21:41:56.814 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.814 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.814 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:56.814 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.978487+0000", 2024-09-21T21:41:56.815 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9254000000000001e-05 2024-09-21T21:41:56.815 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.815 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.815 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:56.815 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.978540+0000", 2024-09-21T21:41:56.815 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2649999999999999e-05 2024-09-21T21:41:56.815 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:56.815 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:56.815 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:56.815 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.815 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.815 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5215.0:1054 3.9 3:946d2401:::rbd_data.1121524d2522.000000000000000b:head [set-alloc-hint object_size 16777216 write_size 16777216,write 6615040~59392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-21T21:41:56.815 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:38.148979+0000", 2024-09-21T21:41:56.815 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 18.580673064999999, 2024-09-21T21:41:56.816 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047870388999999999, 2024-09-21T21:41:56.816 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:56.816 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:56.816 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:41:56.816 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:56.816 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5215", 2024-09-21T21:41:56.816 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1580282168", 2024-09-21T21:41:56.816 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1054 2024-09-21T21:41:56.816 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.816 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:56.816 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.816 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:56.816 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.148979+0000", 2024-09-21T21:41:56.816 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:56.816 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.817 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.817 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:56.817 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.148979+0000", 2024-09-21T21:41:56.817 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:56.817 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.817 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.817 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:56.817 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.148980+0000", 2024-09-21T21:41:56.817 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5799999999999999e-07 2024-09-21T21:41:56.817 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.817 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.817 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:56.817 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.149039+0000", 2024-09-21T21:41:56.817 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9151000000000003e-05 2024-09-21T21:41:56.817 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.817 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.818 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:56.818 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.149047+0000", 2024-09-21T21:41:56.818 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1000000000000004e-06 2024-09-21T21:41:56.818 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.818 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.818 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:56.818 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.149052+0000", 2024-09-21T21:41:56.818 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5959999999999997e-06 2024-09-21T21:41:56.818 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.818 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.818 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:56.818 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.195288+0000", 2024-09-21T21:41:56.818 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.046236248000000001 2024-09-21T21:41:56.819 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.819 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.819 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:56.819 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.195334+0000", 2024-09-21T21:41:56.819 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6575e-05 2024-09-21T21:41:56.819 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.819 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.819 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 2", 2024-09-21T21:41:56.819 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.195406+0000", 2024-09-21T21:41:56.819 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1715e-05 2024-09-21T21:41:56.819 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.819 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.819 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:41:56.819 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.196018+0000", 2024-09-21T21:41:56.819 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00061218999999999998 2024-09-21T21:41:56.820 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.820 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.820 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:41:56.820 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.196794+0000", 2024-09-21T21:41:56.820 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000775814 2024-09-21T21:41:56.820 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.820 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.820 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:41:56.820 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.196827+0000", 2024-09-21T21:41:56.820 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2419000000000003e-05 2024-09-21T21:41:56.820 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:56.820 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:56.820 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:56.820 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.196849+0000", 2024-09-21T21:41:56.821 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2923000000000001e-05 2024-09-21T21:41:56.821 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:56.821 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:56.821 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:56.821 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:56.821 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:56.821 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:56.822 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T21:41:56.823 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:56.820+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:56.923 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:56.921+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:57.010 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:57.010 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:41:57.010 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-21T21:41:57.010 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:57.022 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T21:41:57.062 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:57.061+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:57.114 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T21:41:57.158 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:57.157+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:57.168 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:57.168 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:41:57.168 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:41:57.168 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:41:57.168 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:57.177 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T21:41:57.225 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:57.224+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:57.244 INFO:teuthology.orchestra.run.smithi164.stdout:3954 trunc from 0x7da1000 to 0xc584200 2024-09-21T21:41:57.260 INFO:teuthology.orchestra.run.smithi081.stderr:nodeep-scrub is unset 2024-09-21T21:41:57.280 INFO:teuthology.orchestra.run.smithi164.stdout:3955 trunc from 0xc584200 to 0x881600 2024-09-21T21:41:57.288 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:41:57.288 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-21T21:41:57.289 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-21T21:41:57.289 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:41:57.289 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.289 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1486 3.16 3:6a41a830:::rbd_data.112145fae779.000000000000000a:a [read 360448~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:57.289 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634126+0000", 2024-09-21T21:41:57.289 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 55.651105031999997, 2024-09-21T21:41:57.289 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059399528, 2024-09-21T21:41:57.289 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:57.289 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:57.289 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:57.289 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:57.290 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:57.290 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:57.290 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1486 2024-09-21T21:41:57.290 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.290 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:57.290 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.290 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:57.290 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634126+0000", 2024-09-21T21:41:57.290 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:57.290 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.290 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.290 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:57.291 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634126+0000", 2024-09-21T21:41:57.291 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:57.291 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.291 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.291 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:57.291 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634126+0000", 2024-09-21T21:41:57.291 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4099999999999999e-07 2024-09-21T21:41:57.291 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.291 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.291 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:57.291 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634128+0000", 2024-09-21T21:41:57.291 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1069999999999999e-06 2024-09-21T21:41:57.291 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.292 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.292 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:57.292 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634135+0000", 2024-09-21T21:41:57.292 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.212e-06 2024-09-21T21:41:57.292 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.292 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.292 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:57.292 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634140+0000", 2024-09-21T21:41:57.292 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1429999999999997e-06 2024-09-21T21:41:57.292 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.292 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.292 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:57.292 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693363+0000", 2024-09-21T21:41:57.293 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059223614000000001 2024-09-21T21:41:57.293 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.293 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.293 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:57.293 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693431+0000", 2024-09-21T21:41:57.293 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7294999999999999e-05 2024-09-21T21:41:57.293 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.293 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.293 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:57.293 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693525+0000", 2024-09-21T21:41:57.293 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.4716000000000006e-05 2024-09-21T21:41:57.293 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:57.293 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:57.293 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:57.293 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.293 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.294 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1499 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 49152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:57.294 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634186+0000", 2024-09-21T21:41:57.294 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 55.651045154999998, 2024-09-21T21:41:57.294 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059339256999999999, 2024-09-21T21:41:57.294 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:57.294 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:57.294 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:57.294 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:57.294 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:57.294 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:57.294 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1499 2024-09-21T21:41:57.294 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.294 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:57.295 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.295 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:57.295 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634186+0000", 2024-09-21T21:41:57.295 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:57.295 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.295 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.295 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:57.295 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634186+0000", 2024-09-21T21:41:57.295 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:57.295 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.295 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.295 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:57.295 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634186+0000", 2024-09-21T21:41:57.295 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0999999999999999e-07 2024-09-21T21:41:57.296 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.296 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.296 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:57.296 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634188+0000", 2024-09-21T21:41:57.296 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.86e-06 2024-09-21T21:41:57.296 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.296 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.296 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:57.296 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634194+0000", 2024-09-21T21:41:57.296 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1109999999999996e-06 2024-09-21T21:41:57.296 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.296 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.296 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:57.296 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634199+0000", 2024-09-21T21:41:57.296 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7419999999999997e-06 2024-09-21T21:41:57.297 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.297 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.297 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:57.297 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693357+0000", 2024-09-21T21:41:57.297 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059157847 2024-09-21T21:41:57.297 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.297 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.297 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:57.297 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693430+0000", 2024-09-21T21:41:57.297 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3203999999999999e-05 2024-09-21T21:41:57.297 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.297 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.297 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:57.298 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693525+0000", 2024-09-21T21:41:57.298 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5082999999999997e-05 2024-09-21T21:41:57.298 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:57.298 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:57.298 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:57.298 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.298 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.298 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1501 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 106496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:57.298 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634205+0000", 2024-09-21T21:41:57.298 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 55.651026065000003, 2024-09-21T21:41:57.298 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059482394000000001, 2024-09-21T21:41:57.298 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:57.298 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:57.298 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:57.299 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:57.299 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:57.299 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:57.299 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1501 2024-09-21T21:41:57.299 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.299 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:57.299 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.299 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:57.299 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634205+0000", 2024-09-21T21:41:57.299 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:57.299 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.299 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.299 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:57.299 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634205+0000", 2024-09-21T21:41:57.300 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:57.300 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.300 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.300 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:57.300 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634205+0000", 2024-09-21T21:41:57.300 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.03e-07 2024-09-21T21:41:57.300 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.300 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.300 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:57.300 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634207+0000", 2024-09-21T21:41:57.300 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0389999999999999e-06 2024-09-21T21:41:57.300 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.300 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.300 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:57.301 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634213+0000", 2024-09-21T21:41:57.301 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2600000000000002e-06 2024-09-21T21:41:57.301 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.301 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.301 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:57.301 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634218+0000", 2024-09-21T21:41:57.301 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8570000000000003e-06 2024-09-21T21:41:57.301 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.301 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.301 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:57.301 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693552+0000", 2024-09-21T21:41:57.301 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059333636000000002 2024-09-21T21:41:57.301 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.301 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.302 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:57.302 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693613+0000", 2024-09-21T21:41:57.302 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0560000000000003e-05 2024-09-21T21:41:57.302 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.302 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.302 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:57.302 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693687+0000", 2024-09-21T21:41:57.302 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4639000000000001e-05 2024-09-21T21:41:57.302 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:57.302 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:57.302 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:57.302 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.302 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.302 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1515 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 434176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:57.303 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634318+0000", 2024-09-21T21:41:57.303 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 55.650912378000001, 2024-09-21T21:41:57.303 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059565812000000003, 2024-09-21T21:41:57.303 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:57.303 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:57.303 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:57.303 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:57.303 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:57.303 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:57.303 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1515 2024-09-21T21:41:57.303 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.303 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:57.303 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.304 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:57.304 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634318+0000", 2024-09-21T21:41:57.304 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:57.304 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.304 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.304 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:57.304 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634318+0000", 2024-09-21T21:41:57.304 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:57.304 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.304 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.304 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:57.304 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634319+0000", 2024-09-21T21:41:57.304 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9400000000000001e-07 2024-09-21T21:41:57.305 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.305 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.305 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:57.305 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634321+0000", 2024-09-21T21:41:57.305 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.154e-06 2024-09-21T21:41:57.305 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.305 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.305 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:57.305 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634327+0000", 2024-09-21T21:41:57.305 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9830000000000001e-06 2024-09-21T21:41:57.305 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.306 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.306 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:57.306 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634332+0000", 2024-09-21T21:41:57.306 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9080000000000003e-06 2024-09-21T21:41:57.306 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.306 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.306 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:57.306 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693717+0000", 2024-09-21T21:41:57.306 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059385447000000001 2024-09-21T21:41:57.306 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.306 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.306 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:57.306 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693795+0000", 2024-09-21T21:41:57.306 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7267000000000004e-05 2024-09-21T21:41:57.307 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.307 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.307 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:57.307 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693884+0000", 2024-09-21T21:41:57.307 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9659000000000003e-05 2024-09-21T21:41:57.307 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:57.307 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:57.307 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:57.307 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.307 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.307 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1517 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 491520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:57.307 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634338+0000", 2024-09-21T21:41:57.307 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 55.650892605000003, 2024-09-21T21:41:57.308 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060100745999999997, 2024-09-21T21:41:57.308 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:57.308 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:57.308 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:57.308 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:57.308 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:57.308 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:57.308 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1517 2024-09-21T21:41:57.308 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.308 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:57.308 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.308 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:57.308 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634338+0000", 2024-09-21T21:41:57.308 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:57.308 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.309 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.309 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:57.309 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634338+0000", 2024-09-21T21:41:57.309 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:57.309 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.309 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.309 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:57.309 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634339+0000", 2024-09-21T21:41:57.309 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2899999999999999e-07 2024-09-21T21:41:57.309 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.309 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.309 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:57.309 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634341+0000", 2024-09-21T21:41:57.309 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.8959999999999999e-06 2024-09-21T21:41:57.310 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.310 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.310 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:57.310 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634347+0000", 2024-09-21T21:41:57.310 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.302e-06 2024-09-21T21:41:57.310 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.310 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.310 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:57.310 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634351+0000", 2024-09-21T21:41:57.310 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6070000000000001e-06 2024-09-21T21:41:57.310 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.310 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.310 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:57.310 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694279+0000", 2024-09-21T21:41:57.310 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059927046999999997 2024-09-21T21:41:57.311 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.311 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.311 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:57.311 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694355+0000", 2024-09-21T21:41:57.311 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6483999999999995e-05 2024-09-21T21:41:57.311 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.311 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.311 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:57.311 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694439+0000", 2024-09-21T21:41:57.311 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3980999999999994e-05 2024-09-21T21:41:57.311 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:57.311 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:57.311 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:57.311 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.311 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.312 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1518 3.16 3:6a41a830:::rbd_data.112145fae779.000000000000000a:a [read 294912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:57.312 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634357+0000", 2024-09-21T21:41:57.312 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 55.650873699000002, 2024-09-21T21:41:57.312 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059712318, 2024-09-21T21:41:57.312 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:57.312 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:57.312 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:57.312 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:57.312 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:57.312 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:57.312 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1518 2024-09-21T21:41:57.312 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.312 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:57.312 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.312 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:57.313 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634357+0000", 2024-09-21T21:41:57.313 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:57.313 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.313 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.313 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:57.313 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634357+0000", 2024-09-21T21:41:57.313 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:57.313 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.313 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.313 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:57.313 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634358+0000", 2024-09-21T21:41:57.313 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2100000000000002e-07 2024-09-21T21:41:57.313 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.313 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.313 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:57.314 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634360+0000", 2024-09-21T21:41:57.314 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0940000000000002e-06 2024-09-21T21:41:57.314 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.314 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.314 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:57.314 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634366+0000", 2024-09-21T21:41:57.314 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3389999999999998e-06 2024-09-21T21:41:57.314 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.314 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.314 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:57.314 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634371+0000", 2024-09-21T21:41:57.314 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6670000000000004e-06 2024-09-21T21:41:57.314 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.315 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.315 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:57.315 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693910+0000", 2024-09-21T21:41:57.315 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059538973000000002 2024-09-21T21:41:57.315 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.315 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.315 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:57.315 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693982+0000", 2024-09-21T21:41:57.315 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2806999999999996e-05 2024-09-21T21:41:57.315 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.315 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.315 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:57.316 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694069+0000", 2024-09-21T21:41:57.316 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7016999999999997e-05 2024-09-21T21:41:57.316 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:57.316 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:57.316 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:57.316 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.316 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.316 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1519 3.16 3:6a41a830:::rbd_data.112145fae779.000000000000000a:a [read 352256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:57.316 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634377+0000", 2024-09-21T21:41:57.316 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 55.650853996999999, 2024-09-21T21:41:57.316 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059872213000000001, 2024-09-21T21:41:57.316 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:57.316 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:57.316 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:57.317 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:57.317 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:57.317 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:57.317 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1519 2024-09-21T21:41:57.317 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.317 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:57.317 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.317 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:57.317 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634377+0000", 2024-09-21T21:41:57.317 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:57.317 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.317 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.318 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:57.318 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634377+0000", 2024-09-21T21:41:57.318 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:57.318 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.318 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.318 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:57.318 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634377+0000", 2024-09-21T21:41:57.318 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2600000000000002e-07 2024-09-21T21:41:57.318 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.318 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.318 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:57.318 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634379+0000", 2024-09-21T21:41:57.319 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0729999999999999e-06 2024-09-21T21:41:57.319 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.319 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.319 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:57.319 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634386+0000", 2024-09-21T21:41:57.319 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4690000000000002e-06 2024-09-21T21:41:57.319 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.319 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.319 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:57.319 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634390+0000", 2024-09-21T21:41:57.319 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4580000000000003e-06 2024-09-21T21:41:57.320 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.320 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.320 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:57.320 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694094+0000", 2024-09-21T21:41:57.320 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059703830999999999 2024-09-21T21:41:57.320 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.320 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.320 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:57.320 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694168+0000", 2024-09-21T21:41:57.320 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4222999999999997e-05 2024-09-21T21:41:57.320 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.320 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.321 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:57.321 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694249+0000", 2024-09-21T21:41:57.321 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0632999999999999e-05 2024-09-21T21:41:57.321 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:57.321 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:57.321 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:57.321 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.321 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.321 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1529 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 53248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:57.321 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634397+0000", 2024-09-21T21:41:57.321 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 55.650833845999998, 2024-09-21T21:41:57.321 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060189919000000001, 2024-09-21T21:41:57.322 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:57.322 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:57.322 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:57.322 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:57.322 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:57.322 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:57.322 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1529 2024-09-21T21:41:57.322 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.322 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:57.322 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.322 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:57.322 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634397+0000", 2024-09-21T21:41:57.322 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:57.323 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.323 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.323 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:57.323 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634397+0000", 2024-09-21T21:41:57.323 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:57.323 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.323 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.323 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:57.323 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634397+0000", 2024-09-21T21:41:57.323 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2e-07 2024-09-21T21:41:57.323 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.323 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.323 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:57.324 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634399+0000", 2024-09-21T21:41:57.324 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0609999999999998e-06 2024-09-21T21:41:57.324 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.324 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.324 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:57.324 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634406+0000", 2024-09-21T21:41:57.324 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7259999999999996e-06 2024-09-21T21:41:57.324 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.324 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.324 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:57.324 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634411+0000", 2024-09-21T21:41:57.324 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9339999999999997e-06 2024-09-21T21:41:57.324 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.325 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.325 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:57.325 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694480+0000", 2024-09-21T21:41:57.325 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060069105999999997 2024-09-21T21:41:57.325 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.325 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.325 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:57.325 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694527+0000", 2024-09-21T21:41:57.325 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7120000000000003e-05 2024-09-21T21:41:57.325 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.325 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.325 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:57.325 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694587+0000", 2024-09-21T21:41:57.326 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9552000000000002e-05 2024-09-21T21:41:57.326 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:57.326 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:57.326 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:57.326 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.326 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.326 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1531 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 110592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:57.326 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634417+0000", 2024-09-21T21:41:57.326 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 55.650813911999997, 2024-09-21T21:41:57.326 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060347287999999999, 2024-09-21T21:41:57.326 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:57.326 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:57.326 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:57.327 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:57.327 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:57.327 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:57.327 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1531 2024-09-21T21:41:57.327 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.327 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:57.327 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.327 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:57.327 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634417+0000", 2024-09-21T21:41:57.327 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:57.327 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.327 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.327 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:57.327 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634417+0000", 2024-09-21T21:41:57.328 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:57.328 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.328 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.328 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:57.328 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634417+0000", 2024-09-21T21:41:57.328 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1699999999999999e-07 2024-09-21T21:41:57.328 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.328 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.328 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:57.328 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634420+0000", 2024-09-21T21:41:57.328 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.243e-06 2024-09-21T21:41:57.328 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.328 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.329 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:57.329 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634426+0000", 2024-09-21T21:41:57.329 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2550000000000003e-06 2024-09-21T21:41:57.329 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.329 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.329 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:57.329 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634431+0000", 2024-09-21T21:41:57.329 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0150000000000003e-06 2024-09-21T21:41:57.329 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.329 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.329 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:57.329 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694618+0000", 2024-09-21T21:41:57.329 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060186836000000001 2024-09-21T21:41:57.329 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.329 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.330 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:57.330 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694685+0000", 2024-09-21T21:41:57.330 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6970000000000004e-05 2024-09-21T21:41:57.330 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.330 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.330 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:57.330 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694764+0000", 2024-09-21T21:41:57.330 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9552e-05 2024-09-21T21:41:57.330 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:57.330 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:57.330 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:57.330 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.331 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.331 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1532 3.e 3:724b31a3:::rbd_data.1121e540c096.0000000000000015:b [read 151552~36864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:57.331 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634437+0000", 2024-09-21T21:41:57.331 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 55.650794116, 2024-09-21T21:41:57.331 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060474314000000001, 2024-09-21T21:41:57.331 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:57.331 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:57.331 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:57.331 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:57.331 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:57.331 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:57.331 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1532 2024-09-21T21:41:57.331 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.331 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:57.332 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.332 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:57.332 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634437+0000", 2024-09-21T21:41:57.332 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:57.332 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.332 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.332 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:57.332 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634437+0000", 2024-09-21T21:41:57.332 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:57.332 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.332 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.332 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:57.332 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634437+0000", 2024-09-21T21:41:57.333 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9400000000000001e-07 2024-09-21T21:41:57.333 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.333 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.333 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:57.333 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634439+0000", 2024-09-21T21:41:57.333 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.114e-06 2024-09-21T21:41:57.333 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.333 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.333 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:57.333 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634446+0000", 2024-09-21T21:41:57.333 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1389999999999998e-06 2024-09-21T21:41:57.333 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.333 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.334 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:57.334 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634451+0000", 2024-09-21T21:41:57.334 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0010000000000001e-06 2024-09-21T21:41:57.334 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.334 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.334 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:57.334 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694794+0000", 2024-09-21T21:41:57.334 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060342509000000003 2024-09-21T21:41:57.334 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.334 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.334 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:57.334 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694835+0000", 2024-09-21T21:41:57.334 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0728e-05 2024-09-21T21:41:57.335 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.335 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.335 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:57.335 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694911+0000", 2024-09-21T21:41:57.335 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6428999999999996e-05 2024-09-21T21:41:57.335 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:57.335 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:57.335 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:57.335 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.335 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.335 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1534 3.e 3:724b31a3:::rbd_data.1121e540c096.0000000000000015:b [read 487424~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:57.335 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634477+0000", 2024-09-21T21:41:57.335 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 55.650753713, 2024-09-21T21:41:57.336 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060624009, 2024-09-21T21:41:57.336 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:57.336 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:57.336 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:57.336 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:57.336 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:57.336 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:57.336 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1534 2024-09-21T21:41:57.336 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.336 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:57.336 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.336 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:57.337 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634477+0000", 2024-09-21T21:41:57.337 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:57.337 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.337 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.337 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:57.337 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634477+0000", 2024-09-21T21:41:57.337 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:57.337 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.337 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.337 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:57.337 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634478+0000", 2024-09-21T21:41:57.337 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1399999999999997e-07 2024-09-21T21:41:57.337 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.337 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.338 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:57.338 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634480+0000", 2024-09-21T21:41:57.338 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1009999999999999e-06 2024-09-21T21:41:57.338 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.338 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.338 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:57.338 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634486+0000", 2024-09-21T21:41:57.338 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3289999999999999e-06 2024-09-21T21:41:57.338 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.338 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.338 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:57.338 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634491+0000", 2024-09-21T21:41:57.338 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6260000000000003e-06 2024-09-21T21:41:57.338 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.339 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.339 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:57.339 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694943+0000", 2024-09-21T21:41:57.339 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060451960999999999 2024-09-21T21:41:57.339 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.339 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.339 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:57.339 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695014+0000", 2024-09-21T21:41:57.339 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1587000000000004e-05 2024-09-21T21:41:57.339 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.339 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.339 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:57.339 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695101+0000", 2024-09-21T21:41:57.339 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6990999999999994e-05 2024-09-21T21:41:57.340 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:57.340 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:57.340 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:57.340 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.340 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.340 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1545 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 438272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:57.340 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634515+0000", 2024-09-21T21:41:57.340 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 55.650715513000002, 2024-09-21T21:41:57.340 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060707944, 2024-09-21T21:41:57.340 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:57.340 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:57.340 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:57.340 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:57.340 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:57.341 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:57.341 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1545 2024-09-21T21:41:57.341 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.341 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:57.341 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.341 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:57.341 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634515+0000", 2024-09-21T21:41:57.341 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:57.341 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.341 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.341 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:57.341 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634515+0000", 2024-09-21T21:41:57.341 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:57.341 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.342 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.342 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:57.342 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634516+0000", 2024-09-21T21:41:57.342 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0999999999999999e-07 2024-09-21T21:41:57.342 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.342 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.342 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:57.342 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634518+0000", 2024-09-21T21:41:57.342 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.846e-06 2024-09-21T21:41:57.342 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.342 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.342 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:57.342 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634524+0000", 2024-09-21T21:41:57.343 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2550000000000003e-06 2024-09-21T21:41:57.343 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.343 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.343 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:57.343 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634529+0000", 2024-09-21T21:41:57.343 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6940000000000003e-06 2024-09-21T21:41:57.343 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.343 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.343 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:57.343 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695127+0000", 2024-09-21T21:41:57.343 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060598921 2024-09-21T21:41:57.343 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.343 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.343 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:57.344 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695171+0000", 2024-09-21T21:41:57.344 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3588999999999999e-05 2024-09-21T21:41:57.344 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.344 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.344 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:57.344 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695223+0000", 2024-09-21T21:41:57.344 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2228999999999997e-05 2024-09-21T21:41:57.344 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:57.344 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:57.344 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:57.344 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.344 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.344 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1547 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 495616~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:57.344 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634534+0000", 2024-09-21T21:41:57.345 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 55.650696721999999, 2024-09-21T21:41:57.345 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060881131999999998, 2024-09-21T21:41:57.345 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:57.345 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:57.345 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:57.345 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:57.345 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:57.345 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:57.345 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1547 2024-09-21T21:41:57.345 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.345 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:57.345 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.345 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:57.345 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634534+0000", 2024-09-21T21:41:57.345 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:57.345 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.346 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.346 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:57.346 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634534+0000", 2024-09-21T21:41:57.346 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:57.346 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.346 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.346 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:57.346 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634535+0000", 2024-09-21T21:41:57.346 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.77e-07 2024-09-21T21:41:57.346 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.346 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.346 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:57.346 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634542+0000", 2024-09-21T21:41:57.347 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9000000000000006e-06 2024-09-21T21:41:57.347 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.347 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.347 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:57.347 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634549+0000", 2024-09-21T21:41:57.347 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5910000000000001e-06 2024-09-21T21:41:57.347 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.347 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.347 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:57.347 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634554+0000", 2024-09-21T21:41:57.347 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7940000000000002e-06 2024-09-21T21:41:57.347 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.347 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.347 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:57.347 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695256+0000", 2024-09-21T21:41:57.348 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.06070246 2024-09-21T21:41:57.348 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.348 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.348 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:57.348 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695332+0000", 2024-09-21T21:41:57.348 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6248000000000006e-05 2024-09-21T21:41:57.348 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.348 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.348 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:57.348 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695415+0000", 2024-09-21T21:41:57.348 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2762000000000005e-05 2024-09-21T21:41:57.348 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:57.348 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:57.348 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:57.349 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.349 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.349 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1548 3.16 3:6a41a830:::rbd_data.112145fae779.000000000000000a:a [read 299008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:57.349 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634561+0000", 2024-09-21T21:41:57.349 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 55.650670144999999, 2024-09-21T21:41:57.349 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060786743999999997, 2024-09-21T21:41:57.349 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:57.349 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:57.349 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:57.349 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:57.349 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:57.350 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:57.350 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1548 2024-09-21T21:41:57.350 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.350 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:57.350 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.350 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:57.350 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634561+0000", 2024-09-21T21:41:57.350 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:57.350 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.350 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.350 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:57.350 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634561+0000", 2024-09-21T21:41:57.350 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:57.351 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.351 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.351 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:57.351 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634561+0000", 2024-09-21T21:41:57.351 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5199999999999997e-07 2024-09-21T21:41:57.351 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.351 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.351 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:57.351 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634564+0000", 2024-09-21T21:41:57.351 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4040000000000002e-06 2024-09-21T21:41:57.351 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.351 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.351 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:57.351 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634571+0000", 2024-09-21T21:41:57.352 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4379999999999998e-06 2024-09-21T21:41:57.352 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.352 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.352 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:57.352 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634576+0000", 2024-09-21T21:41:57.352 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1399999999999999e-06 2024-09-21T21:41:57.352 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.352 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.352 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:57.352 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695247+0000", 2024-09-21T21:41:57.352 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060670397000000001 2024-09-21T21:41:57.352 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.352 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.352 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:57.352 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695299+0000", 2024-09-21T21:41:57.353 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2564e-05 2024-09-21T21:41:57.353 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.353 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.353 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:57.353 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695347+0000", 2024-09-21T21:41:57.353 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8248999999999998e-05 2024-09-21T21:41:57.353 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:57.353 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:57.353 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:57.353 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.353 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.353 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1549 3.16 3:6a41a830:::rbd_data.112145fae779.000000000000000a:a [read 356352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:41:57.353 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634583+0000", 2024-09-21T21:41:57.353 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 55.650648232000002, 2024-09-21T21:41:57.354 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060889459, 2024-09-21T21:41:57.354 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:57.354 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:57.354 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:57.354 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:57.354 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:41:57.354 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:41:57.354 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1549 2024-09-21T21:41:57.354 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.354 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:57.354 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.354 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:57.354 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634583+0000", 2024-09-21T21:41:57.355 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:57.355 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.355 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.355 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:57.355 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634583+0000", 2024-09-21T21:41:57.355 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:57.355 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.355 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.355 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:57.355 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634583+0000", 2024-09-21T21:41:57.355 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1899999999999998e-07 2024-09-21T21:41:57.355 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.355 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.355 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:57.355 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634585+0000", 2024-09-21T21:41:57.356 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2129999999999998e-06 2024-09-21T21:41:57.356 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.356 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.356 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:57.356 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634592+0000", 2024-09-21T21:41:57.356 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2840000000000003e-06 2024-09-21T21:41:57.356 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.356 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.356 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:57.356 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634596+0000", 2024-09-21T21:41:57.356 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9309999999999999e-06 2024-09-21T21:41:57.356 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.356 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.356 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:57.356 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695371+0000", 2024-09-21T21:41:57.357 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060774425999999999 2024-09-21T21:41:57.357 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.357 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.357 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:57.357 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695419+0000", 2024-09-21T21:41:57.357 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8607999999999999e-05 2024-09-21T21:41:57.357 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.357 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.357 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:57.357 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695472+0000", 2024-09-21T21:41:57.357 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2577999999999997e-05 2024-09-21T21:41:57.357 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:57.357 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:57.357 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:57.358 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.358 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.358 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4385.0:15202 3.16 3:6888948b:::rbd_header.1121493f6459:head [watch reconnect cookie 140325440202496 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-21T21:41:57.358 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:41.736601+0000", 2024-09-21T21:41:57.358 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 15.548629599, 2024-09-21T21:41:57.358 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5040056769999999, 2024-09-21T21:41:57.358 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:57.358 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:57.358 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:57.358 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:57.358 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4385", 2024-09-21T21:41:57.358 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:41:57.359 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 15202 2024-09-21T21:41:57.359 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.359 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:57.359 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.359 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:57.359 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736601+0000", 2024-09-21T21:41:57.359 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:57.359 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.359 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.359 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:57.359 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736601+0000", 2024-09-21T21:41:57.359 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:57.359 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.360 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.360 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:57.360 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736602+0000", 2024-09-21T21:41:57.360 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0699999999999999e-06 2024-09-21T21:41:57.360 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.360 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.360 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:57.360 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736607+0000", 2024-09-21T21:41:57.360 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2520000000000001e-06 2024-09-21T21:41:57.360 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.360 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.360 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:57.360 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736617+0000", 2024-09-21T21:41:57.360 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0074e-05 2024-09-21T21:41:57.361 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.361 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.361 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:57.361 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736622+0000", 2024-09-21T21:41:57.361 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.101e-06 2024-09-21T21:41:57.361 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.361 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.361 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:57.361 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736664+0000", 2024-09-21T21:41:57.361 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1869999999999997e-05 2024-09-21T21:41:57.361 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.361 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.361 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op must wait for map", 2024-09-21T21:41:57.361 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736667+0000", 2024-09-21T21:41:57.362 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4750000000000002e-06 2024-09-21T21:41:57.362 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.362 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.362 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:57.362 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742886+0000", 2024-09-21T21:41:57.362 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0062187579999999996 2024-09-21T21:41:57.362 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.362 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.362 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:41:57.362 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742892+0000", 2024-09-21T21:41:57.362 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0440000000000001e-06 2024-09-21T21:41:57.362 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.362 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.362 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:57.362 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210307+0000", 2024-09-21T21:41:57.363 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.467414781 2024-09-21T21:41:57.363 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.363 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.363 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:41:57.363 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210322+0000", 2024-09-21T21:41:57.363 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5311e-05 2024-09-21T21:41:57.363 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.363 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.363 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:57.363 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.237624+0000", 2024-09-21T21:41:57.363 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0273018920000001 2024-09-21T21:41:57.363 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.363 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.363 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:57.364 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.237870+0000", 2024-09-21T21:41:57.364 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00024622899999999999 2024-09-21T21:41:57.364 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.365 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.365 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:57.365 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.240607+0000", 2024-09-21T21:41:57.365 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0027368200000000001 2024-09-21T21:41:57.365 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:57.365 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:57.365 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:57.366 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.366 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.366 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5415.0:379 3.16 3:6888948b:::rbd_header.1121493f6459:head [watch reconnect cookie 139702602617920 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-21T21:41:57.366 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:41.739544+0000", 2024-09-21T21:41:57.366 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 15.545687170000001, 2024-09-21T21:41:57.366 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.501180645, 2024-09-21T21:41:57.366 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:57.366 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:57.366 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:57.366 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:57.366 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5415", 2024-09-21T21:41:57.366 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1212816607", 2024-09-21T21:41:57.366 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 379 2024-09-21T21:41:57.366 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.366 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:57.367 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.367 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:57.367 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739544+0000", 2024-09-21T21:41:57.367 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:57.367 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.367 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.367 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:57.367 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739544+0000", 2024-09-21T21:41:57.367 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:57.367 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.367 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.367 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:57.367 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739545+0000", 2024-09-21T21:41:57.367 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.065e-06 2024-09-21T21:41:57.367 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.368 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.368 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:57.368 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739550+0000", 2024-09-21T21:41:57.368 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9599999999999999e-06 2024-09-21T21:41:57.368 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.368 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.368 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:57.368 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739565+0000", 2024-09-21T21:41:57.368 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5709000000000002e-05 2024-09-21T21:41:57.368 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.368 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.368 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:57.368 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739573+0000", 2024-09-21T21:41:57.368 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7849999999999992e-06 2024-09-21T21:41:57.369 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.369 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.369 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:57.369 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739634+0000", 2024-09-21T21:41:57.369 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0732000000000001e-05 2024-09-21T21:41:57.369 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.369 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.369 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op must wait for map", 2024-09-21T21:41:57.369 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739640+0000", 2024-09-21T21:41:57.369 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0000000000000002e-06 2024-09-21T21:41:57.369 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.369 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.369 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:57.369 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742958+0000", 2024-09-21T21:41:57.369 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0033182390000000002 2024-09-21T21:41:57.370 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.370 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.370 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:41:57.370 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742966+0000", 2024-09-21T21:41:57.370 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1610000000000003e-06 2024-09-21T21:41:57.370 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.370 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.370 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:57.370 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210405+0000", 2024-09-21T21:41:57.370 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.467438725 2024-09-21T21:41:57.370 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.370 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.370 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:41:57.370 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210415+0000", 2024-09-21T21:41:57.371 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0234000000000001e-05 2024-09-21T21:41:57.371 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.371 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.371 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:57.371 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.238743+0000", 2024-09-21T21:41:57.371 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0283273070000001 2024-09-21T21:41:57.371 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.371 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.371 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:57.371 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.238842+0000", 2024-09-21T21:41:57.371 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.9896999999999996e-05 2024-09-21T21:41:57.371 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.371 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.371 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:57.372 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.240724+0000", 2024-09-21T21:41:57.372 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001881831 2024-09-21T21:41:57.372 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:57.372 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:57.372 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:57.372 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.372 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.372 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4385.0:15203 3.16 3:6888948b:::rbd_header.1121493f6459:head [call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-21T21:41:57.372 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:41.739737+0000", 2024-09-21T21:41:57.372 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 15.545494159, 2024-09-21T21:41:57.372 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4989621799999999, 2024-09-21T21:41:57.372 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:57.372 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:57.372 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:57.373 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:57.373 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4385", 2024-09-21T21:41:57.373 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:41:57.373 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 15203 2024-09-21T21:41:57.373 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.373 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:57.373 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.373 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:57.373 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739737+0000", 2024-09-21T21:41:57.373 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:57.373 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.373 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.373 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:57.373 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739737+0000", 2024-09-21T21:41:57.374 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:57.374 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.374 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.374 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:57.374 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739738+0000", 2024-09-21T21:41:57.374 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.7499999999999998e-07 2024-09-21T21:41:57.374 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.374 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.374 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:57.374 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739743+0000", 2024-09-21T21:41:57.374 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5049999999999998e-06 2024-09-21T21:41:57.374 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.374 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.374 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:57.375 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739757+0000", 2024-09-21T21:41:57.375 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3947e-05 2024-09-21T21:41:57.375 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.375 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.375 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:57.375 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739765+0000", 2024-09-21T21:41:57.375 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9310000000000008e-06 2024-09-21T21:41:57.375 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.375 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.375 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:57.375 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739816+0000", 2024-09-21T21:41:57.375 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0559999999999997e-05 2024-09-21T21:41:57.375 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.375 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.376 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting_for_map not empty", 2024-09-21T21:41:57.376 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739821+0000", 2024-09-21T21:41:57.376 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3539999999999999e-06 2024-09-21T21:41:57.376 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.376 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.376 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:57.376 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742928+0000", 2024-09-21T21:41:57.376 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0031074399999999999 2024-09-21T21:41:57.376 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.376 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.376 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:41:57.376 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742937+0000", 2024-09-21T21:41:57.376 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1289999999999996e-06 2024-09-21T21:41:57.376 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.377 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.377 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:57.377 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210350+0000", 2024-09-21T21:41:57.377 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.46741311299999999 2024-09-21T21:41:57.377 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.377 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.377 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:41:57.377 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210358+0000", 2024-09-21T21:41:57.377 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1869999999999997e-06 2024-09-21T21:41:57.377 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.377 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.377 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:57.377 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.238398+0000", 2024-09-21T21:41:57.377 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0280397830000001 2024-09-21T21:41:57.378 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.378 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.378 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:57.378 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.238489+0000", 2024-09-21T21:41:57.378 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.0958000000000006e-05 2024-09-21T21:41:57.378 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.378 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.378 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:57.378 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.238699+0000", 2024-09-21T21:41:57.378 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00021029800000000001 2024-09-21T21:41:57.378 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:57.378 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:57.378 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:57.378 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.379 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.379 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4385.0:15204 3.16 3:6888948b:::rbd_header.1121493f6459:head [watch reconnect cookie 140325440202496 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-21T21:41:57.379 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:42.210726+0000", 2024-09-21T21:41:57.379 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 15.074505281, 2024-09-21T21:41:57.379 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.030304557, 2024-09-21T21:41:57.379 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:57.379 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:57.379 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:57.379 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:57.379 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4385", 2024-09-21T21:41:57.379 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:41:57.379 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 15204 2024-09-21T21:41:57.379 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.379 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:57.380 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.380 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:57.380 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210726+0000", 2024-09-21T21:41:57.380 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:57.380 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.380 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.380 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:57.380 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210726+0000", 2024-09-21T21:41:57.380 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:57.380 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.380 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.380 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:57.380 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210727+0000", 2024-09-21T21:41:57.380 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.204e-06 2024-09-21T21:41:57.381 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.381 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.381 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:57.381 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210732+0000", 2024-09-21T21:41:57.381 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5249999999999996e-06 2024-09-21T21:41:57.381 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.381 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.381 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:57.381 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210744+0000", 2024-09-21T21:41:57.381 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1405e-05 2024-09-21T21:41:57.381 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.381 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.381 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:57.381 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210754+0000", 2024-09-21T21:41:57.382 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0122999999999999e-05 2024-09-21T21:41:57.382 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.382 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.382 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:57.382 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210798+0000", 2024-09-21T21:41:57.382 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4005000000000003e-05 2024-09-21T21:41:57.382 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.382 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.382 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:41:57.382 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210803+0000", 2024-09-21T21:41:57.382 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5130000000000004e-06 2024-09-21T21:41:57.382 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.382 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.383 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:57.383 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.239260+0000", 2024-09-21T21:41:57.383 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0284563040000001 2024-09-21T21:41:57.383 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.383 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.383 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:57.383 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.239318+0000", 2024-09-21T21:41:57.383 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8486000000000002e-05 2024-09-21T21:41:57.383 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.383 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.383 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:57.383 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.241030+0000", 2024-09-21T21:41:57.383 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0017119920000000001 2024-09-21T21:41:57.383 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:57.383 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:57.384 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:57.384 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.384 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.384 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5415.0:380 3.16 3:6888948b:::rbd_header.1121493f6459:head [watch reconnect cookie 139702602617920 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-21T21:41:57.384 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:42.210777+0000", 2024-09-21T21:41:57.384 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 15.074453502000001, 2024-09-21T21:41:57.384 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.030410566, 2024-09-21T21:41:57.384 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:41:57.384 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:41:57.384 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:41:57.384 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:41:57.384 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5415", 2024-09-21T21:41:57.384 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1212816607", 2024-09-21T21:41:57.384 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 380 2024-09-21T21:41:57.385 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.385 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:41:57.385 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.385 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:41:57.385 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210777+0000", 2024-09-21T21:41:57.385 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:57.385 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.385 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.385 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:41:57.385 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210777+0000", 2024-09-21T21:41:57.385 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:41:57.385 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.385 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.386 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:41:57.386 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210778+0000", 2024-09-21T21:41:57.386 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8999999999999996e-07 2024-09-21T21:41:57.386 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.386 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.386 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:41:57.386 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210780+0000", 2024-09-21T21:41:57.386 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.503e-06 2024-09-21T21:41:57.386 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.386 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.386 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:41:57.386 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210788+0000", 2024-09-21T21:41:57.386 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0999999999999998e-06 2024-09-21T21:41:57.386 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.387 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.387 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:41:57.387 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210799+0000", 2024-09-21T21:41:57.387 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1703e-05 2024-09-21T21:41:57.387 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.387 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.387 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:57.387 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210833+0000", 2024-09-21T21:41:57.387 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4008999999999999e-05 2024-09-21T21:41:57.387 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.387 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.387 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:41:57.387 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210838+0000", 2024-09-21T21:41:57.388 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7480000000000001e-06 2024-09-21T21:41:57.388 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.388 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.388 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:41:57.388 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.239612+0000", 2024-09-21T21:41:57.388 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.028773554 2024-09-21T21:41:57.388 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.388 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.388 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:41:57.388 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.239677+0000", 2024-09-21T21:41:57.388 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5053000000000002e-05 2024-09-21T21:41:57.388 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:41:57.388 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:41:57.388 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:41:57.388 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.241188+0000", 2024-09-21T21:41:57.389 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001511306 2024-09-21T21:41:57.389 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:57.389 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:57.389 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:57.389 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:41:57.389 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:41:57.389 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:41:57.391 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:57.389+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:57.392 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T21:41:57.393 INFO:teuthology.orchestra.run.smithi164.stdout:3956 punch from 0x1e400 to 0x2aa00, (0xc600 bytes) 2024-09-21T21:41:57.393 INFO:teuthology.orchestra.run.smithi164.stdout:3957 trunc from 0x881600 to 0x4a5be00 2024-09-21T21:41:57.393 INFO:teuthology.orchestra.run.smithi164.stdout:3958 read 0x1c0ea00 thru 0x1c0f8a9 (0xeaa bytes) 2024-09-21T21:41:57.393 INFO:teuthology.orchestra.run.smithi164.stdout:3959 read 0x1b39000 thru 0x1b46818 (0xd819 bytes) 2024-09-21T21:41:57.393 INFO:teuthology.orchestra.run.smithi164.stdout:3960 read 0x1b15a00 thru 0x1b1ab3e (0x513f bytes) 2024-09-21T21:41:57.393 INFO:teuthology.orchestra.run.smithi164.stdout:3961 read 0x15ac800 thru 0x15b6ed8 (0xa6d9 bytes) 2024-09-21T21:41:57.393 INFO:teuthology.orchestra.run.smithi164.stdout:3962 read 0x1b44600 thru 0x1b51f29 (0xd92a bytes) 2024-09-21T21:41:57.393 INFO:teuthology.orchestra.run.smithi164.stdout:3964 punch from 0x2548c00 to 0x254ca00, (0x3e00 bytes) 2024-09-21T21:41:57.393 INFO:teuthology.orchestra.run.smithi164.stdout:3966 read 0x1b13600 thru 0x1b198fa (0x62fb bytes) 2024-09-21T21:41:57.393 INFO:teuthology.orchestra.run.smithi164.stdout:3968 punch from 0x2126400 to 0x2130800, (0xa400 bytes) 2024-09-21T21:41:57.393 INFO:teuthology.orchestra.run.smithi164.stdout:3969 write 0x907200 thru 0x90b8b2 (0x46b3 bytes) 2024-09-21T21:41:57.393 INFO:teuthology.orchestra.run.smithi164.stdout:3971 read 0x6ff600 thru 0x705f80 (0x6981 bytes) 2024-09-21T21:41:57.393 INFO:teuthology.orchestra.run.smithi164.stdout:3972 trunc from 0x4a5be00 to 0x4fee600 2024-09-21T21:41:57.394 INFO:teuthology.orchestra.run.smithi164.stdout:3975 read 0x276200 thru 0x27dd1a (0x7b1b bytes) 2024-09-21T21:41:57.396 INFO:teuthology.orchestra.run.smithi164.stdout:3978 write 0xcbb8a00 thru 0xcbbfe6d (0x746e bytes) 2024-09-21T21:41:57.397 INFO:teuthology.orchestra.run.smithi164.stdout:3980 punch from 0xf48200 to 0xf4ec00, (0x6a00 bytes) 2024-09-21T21:41:57.398 INFO:teuthology.orchestra.run.smithi164.stdout:3982 write 0xf45800 thru 0xf4803d (0x283e bytes) 2024-09-21T21:41:57.405 INFO:teuthology.orchestra.run.smithi164.stdout:3983 write 0x9d95000 thru 0x9da405d (0xf05e bytes) 2024-09-21T21:41:57.409 INFO:teuthology.orchestra.run.smithi164.stdout:3984 trunc from 0xcbc0000 to 0x9034e00 2024-09-21T21:41:57.434 INFO:teuthology.orchestra.run.smithi164.stdout:3985 punch from 0x5ac4e00 to 0x5acec00, (0x9e00 bytes) 2024-09-21T21:41:57.435 INFO:teuthology.orchestra.run.smithi164.stdout:3987 write 0x1680000 thru 0x168a660 (0xa661 bytes) 2024-09-21T21:41:57.435 INFO:teuthology.orchestra.run.smithi164.stdout:3990 write 0x6c4f600 thru 0x6c52d4b (0x374c bytes) 2024-09-21T21:41:57.435 INFO:teuthology.orchestra.run.smithi164.stdout:3991 read 0x69df000 thru 0x69e06f0 (0x16f1 bytes) 2024-09-21T21:41:57.436 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0: {} 2024-09-21T21:41:57.437 INFO:teuthology.orchestra.run.smithi081.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T21:41:57.437 INFO:teuthology.orchestra.run.smithi164.stdout:3992 trunc from 0x9034e00 to 0x721ca00 2024-09-21T21:41:57.455 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1: {} 2024-09-21T21:41:57.455 INFO:teuthology.orchestra.run.smithi081.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T21:41:57.465 INFO:teuthology.orchestra.run.smithi164.stdout:3995 punch from 0x5926e00 to 0x5927400, (0x600 bytes) 2024-09-21T21:41:57.472 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2: {} 2024-09-21T21:41:57.473 INFO:teuthology.orchestra.run.smithi081.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T21:41:57.488 INFO:teuthology.orchestra.run.smithi164.stdout:3996 write 0xc512c00 thru 0xc51572b (0x2b2c bytes) 2024-09-21T21:41:57.488 INFO:teuthology.orchestra.run.smithi164.stdout:3998 read 0x5255e00 thru 0x525f0fd (0x92fe bytes) 2024-09-21T21:41:57.491 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:57.490+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:57.492 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3: {} 2024-09-21T21:41:57.492 INFO:teuthology.orchestra.run.smithi081.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T21:41:57.508 INFO:teuthology.orchestra.run.smithi164.stdout:3999 write 0xe75f200 thru 0xe760f58 (0x1d59 bytes) 2024-09-21T21:41:57.510 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:57.511 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:57.512 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4: {} 2024-09-21T21:41:57.512 INFO:teuthology.orchestra.run.smithi081.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T21:41:57.518 INFO:teuthology.orchestra.run.smithi164.stdout:4001 trunc from 0xe761000 to 0x586fc00 2024-09-21T21:41:57.520 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T21:41:57.532 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5: {} 2024-09-21T21:41:57.532 INFO:teuthology.orchestra.run.smithi081.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T21:41:57.542 INFO:teuthology.orchestra.run.smithi164.stdout:4002 write 0x854000 thru 0x8635e9 (0xf5ea bytes) 2024-09-21T21:41:57.542 INFO:teuthology.orchestra.run.smithi164.stdout:4004 read 0x2326e00 thru 0x232d432 (0x6633 bytes) 2024-09-21T21:41:57.554 INFO:teuthology.orchestra.run.smithi164.stdout:4005 write 0x6407800 thru 0x640bddb (0x45dc bytes) 2024-09-21T21:41:57.555 INFO:teuthology.orchestra.run.smithi164.stdout:4006 read 0x4ec4400 thru 0x4ec4f21 (0xb22 bytes) 2024-09-21T21:41:57.556 INFO:teuthology.orchestra.run.smithi164.stdout:4007 read 0x77c800 thru 0x7819d9 (0x51da bytes) 2024-09-21T21:41:57.556 INFO:teuthology.orchestra.run.smithi164.stdout:4008 read 0x82a00 thru 0x8f786 (0xcd87 bytes) 2024-09-21T21:41:57.572 INFO:teuthology.orchestra.run.smithi164.stdout:4010 write 0xb123400 thru 0xb12f21f (0xbe20 bytes) 2024-09-21T21:41:57.574 INFO:teuthology.orchestra.run.smithi164.stdout:4011 write 0x64d8a00 thru 0x64e3861 (0xae62 bytes) 2024-09-21T21:41:57.574 INFO:teuthology.orchestra.run.smithi164.stdout:4012 write 0x4839000 thru 0x4846b2d (0xdb2e bytes) 2024-09-21T21:41:57.577 INFO:teuthology.orchestra.run.smithi164.stdout:4014 read 0x7d1400 thru 0x7d8270 (0x6e71 bytes) 2024-09-21T21:41:57.578 INFO:teuthology.orchestra.run.smithi164.stdout:4015 trunc from 0xb12f400 to 0xb1f1c00 2024-09-21T21:41:57.591 INFO:teuthology.orchestra.run.smithi164.stdout:4016 write 0x851d400 thru 0x85265a3 (0x91a4 bytes) 2024-09-21T21:41:57.592 INFO:teuthology.orchestra.run.smithi164.stdout:4017 write 0x8ebb000 thru 0x8ebb3cd (0x3ce bytes) 2024-09-21T21:41:57.592 INFO:teuthology.orchestra.run.smithi164.stdout:4018 read 0x9fc3000 thru 0x9fcf60c (0xc60d bytes) 2024-09-21T21:41:57.593 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:57.590+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:57.603 INFO:teuthology.orchestra.run.smithi164.stdout:4021 trunc from 0xb1f1c00 to 0x14e4000 2024-09-21T21:41:57.631 INFO:teuthology.orchestra.run.smithi164.stdout:4025 read 0x93ae00 thru 0x948ee5 (0xe0e6 bytes) 2024-09-21T21:41:57.643 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:57.643 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:57.650 INFO:teuthology.orchestra.run.smithi164.stdout:4027 write 0x8727a00 thru 0x872a653 (0x2c54 bytes) 2024-09-21T21:41:57.653 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T21:41:57.657 INFO:teuthology.orchestra.run.smithi164.stdout:4028 trunc from 0x872a800 to 0x22dde00 2024-09-21T21:41:57.696 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:57.695+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:57.697 INFO:teuthology.orchestra.run.smithi164.stdout:4030 write 0xb89d200 thru 0xb8a930b (0xc10c bytes) 2024-09-21T21:41:57.708 INFO:teuthology.orchestra.run.smithi164.stdout:4031 trunc from 0xb8a9400 to 0x22efc00 2024-09-21T21:41:57.741 INFO:teuthology.orchestra.run.smithi164.stdout:4032 write 0x4ab3600 thru 0x4abf382 (0xbd83 bytes) 2024-09-21T21:41:57.742 INFO:teuthology.orchestra.run.smithi164.stdout:4033 read 0x1869400 thru 0x186abaf (0x17b0 bytes) 2024-09-21T21:41:57.742 INFO:teuthology.orchestra.run.smithi164.stdout:4034 read 0x370ce00 thru 0x371cd0c (0xff0d bytes) 2024-09-21T21:41:57.744 INFO:teuthology.orchestra.run.smithi164.stdout:4035 trunc from 0x4abf400 to 0x3c08200 2024-09-21T21:41:57.762 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:57.762 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:57.762 INFO:teuthology.orchestra.run.smithi164.stdout:4036 write 0x2227c00 thru 0x222ac46 (0x3047 bytes) 2024-09-21T21:41:57.764 INFO:teuthology.orchestra.run.smithi164.stdout:4037 trunc from 0x3c08200 to 0x50a9400 2024-09-21T21:41:57.771 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T21:41:57.774 INFO:teuthology.orchestra.run.smithi164.stdout:4039 punch from 0x4ef8800 to 0x4efc200, (0x3a00 bytes) 2024-09-21T21:41:57.781 INFO:teuthology.orchestra.run.smithi164.stdout:4040 trunc from 0x50a9400 to 0x3428800 2024-09-21T21:41:57.792 INFO:teuthology.orchestra.run.smithi164.stdout:4041 punch from 0x2fc7000 to 0x2fc9000, (0x2000 bytes) 2024-09-21T21:41:57.795 INFO:teuthology.orchestra.run.smithi164.stdout:4042 trunc from 0x3428800 to 0x3bcc000 2024-09-21T21:41:57.795 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:57.793+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:57.826 INFO:teuthology.orchestra.run.smithi164.stdout:4044 write 0xec2c200 thru 0xec34d66 (0x8b67 bytes) 2024-09-21T21:41:57.827 INFO:teuthology.orchestra.run.smithi164.stdout:4046 write 0x3af1400 thru 0x3af32cb (0x1ecc bytes) 2024-09-21T21:41:57.827 INFO:teuthology.orchestra.run.smithi164.stdout:4047 punch from 0x556a000 to 0x5573a00, (0x9a00 bytes) 2024-09-21T21:41:57.831 INFO:teuthology.orchestra.run.smithi164.stdout:4049 write 0x3f53c00 thru 0x3f56d5b (0x315c bytes) 2024-09-21T21:41:57.831 INFO:teuthology.orchestra.run.smithi164.stdout:4052 punch from 0xaffee00 to 0xb00a000, (0xb200 bytes) 2024-09-21T21:41:57.832 INFO:teuthology.orchestra.run.smithi164.stdout:4053 punch from 0x5945600 to 0x5949c00, (0x4600 bytes) 2024-09-21T21:41:57.832 INFO:teuthology.orchestra.run.smithi164.stdout:4056 punch from 0x61f4c00 to 0x6202e00, (0xe200 bytes) 2024-09-21T21:41:57.832 INFO:teuthology.orchestra.run.smithi164.stdout:4057 write 0xc01200 thru 0xc0f977 (0xe778 bytes) 2024-09-21T21:41:57.845 INFO:teuthology.orchestra.run.smithi164.stdout:4058 trunc from 0xec34e00 to 0x4b48600 2024-09-21T21:41:57.884 INFO:teuthology.orchestra.run.smithi164.stdout:4059 write 0xc1c5c00 thru 0xc1d1a4e (0xbe4f bytes) 2024-09-21T21:41:57.885 INFO:teuthology.orchestra.run.smithi164.stdout:4064 punch from 0xb5da00 to 0xb61a00, (0x4000 bytes) 2024-09-21T21:41:57.885 INFO:teuthology.orchestra.run.smithi164.stdout:4065 write 0xa3600 thru 0xae9d5 (0xb3d6 bytes) 2024-09-21T21:41:57.895 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:57.894+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:57.901 INFO:teuthology.orchestra.run.smithi164.stdout:4066 trunc from 0xc1d1c00 to 0x45f000 2024-09-21T21:41:57.920 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:57.921 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:57.930 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T21:41:57.955 INFO:teuthology.orchestra.run.smithi164.stdout:4067 trunc from 0x45f000 to 0x5c73000 2024-09-21T21:41:57.979 INFO:teuthology.orchestra.run.smithi164.stdout:4068 write 0x8812a00 thru 0x882111d (0xe71e bytes) 2024-09-21T21:41:57.979 INFO:teuthology.orchestra.run.smithi164.stdout:4069 write 0x6297800 thru 0x629925d (0x1a5e bytes) 2024-09-21T21:41:57.980 INFO:teuthology.orchestra.run.smithi164.stdout:4071 read 0x5903e00 thru 0x5913d73 (0xff74 bytes) 2024-09-21T21:41:57.981 INFO:teuthology.orchestra.run.smithi164.stdout:4072 read 0x4313400 thru 0x4322106 (0xed07 bytes) 2024-09-21T21:41:57.986 INFO:teuthology.orchestra.run.smithi164.stdout:4073 trunc from 0x8821200 to 0x43fe000 2024-09-21T21:41:57.996 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:57.994+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:58.015 INFO:teuthology.orchestra.run.smithi164.stdout:4077 trunc from 0x43fe000 to 0xc72fe00 2024-09-21T21:41:58.040 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:58.041 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:58.042 INFO:teuthology.orchestra.run.smithi164.stdout:4079 trunc from 0xc72fe00 to 0xa5d000 2024-09-21T21:41:58.050 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T21:41:58.072 INFO:teuthology.orchestra.run.smithi164.stdout:4081 write 0x342ec00 thru 0x3439e73 (0xb274 bytes) 2024-09-21T21:41:58.073 INFO:teuthology.orchestra.run.smithi164.stdout:4082 read 0x2edfa00 thru 0x2ee6d91 (0x7392 bytes) 2024-09-21T21:41:58.073 INFO:teuthology.orchestra.run.smithi164.stdout:4085 read 0x486c00 thru 0x48d112 (0x6513 bytes) 2024-09-21T21:41:58.077 INFO:teuthology.orchestra.run.smithi164.stdout:4089 trunc from 0x343a000 to 0x4d17800 2024-09-21T21:41:58.090 INFO:teuthology.orchestra.run.smithi164.stdout:4090 read 0x4cb2a00 thru 0x4cc025f (0xd860 bytes) 2024-09-21T21:41:58.090 INFO:teuthology.orchestra.run.smithi164.stdout:4091 punch from 0x3f53e00 to 0x3f59400, (0x5600 bytes) 2024-09-21T21:41:58.090 INFO:teuthology.orchestra.run.smithi164.stdout:4092 read 0x304b600 thru 0x3050850 (0x5251 bytes) 2024-09-21T21:41:58.102 INFO:teuthology.orchestra.run.smithi164.stdout:4093 write 0x88a0c00 thru 0x88ac391 (0xb792 bytes) 2024-09-21T21:41:58.118 INFO:teuthology.orchestra.run.smithi164.stdout:4094 write 0xb8e5a00 thru 0xb8e94cb (0x3acc bytes) 2024-09-21T21:41:58.119 INFO:teuthology.orchestra.run.smithi164.stdout:4095 write 0x2540600 thru 0x2540cc1 (0x6c2 bytes) 2024-09-21T21:41:58.120 INFO:teuthology.orchestra.run.smithi164.stdout:4096 read 0x50a1000 thru 0x50a6170 (0x5171 bytes) 2024-09-21T21:41:58.120 INFO:teuthology.orchestra.run.smithi164.stdout:4100 punch from 0x7852000 to 0x7856e00, (0x4e00 bytes) 2024-09-21T21:41:58.120 INFO:teuthology.orchestra.run.smithi164.stdout:4102 read 0x1b31400 thru 0x1b391d5 (0x7dd6 bytes) 2024-09-21T21:41:58.120 INFO:teuthology.orchestra.run.smithi164.stdout:4104 read 0x6f7e800 thru 0x6f8bf8b (0xd78c bytes) 2024-09-21T21:41:58.120 INFO:teuthology.orchestra.run.smithi164.stdout:4106 write 0x45d4000 thru 0x45e25c0 (0xe5c1 bytes) 2024-09-21T21:41:58.121 INFO:teuthology.orchestra.run.smithi164.stdout:4109 read 0x2753c00 thru 0x2761365 (0xd766 bytes) 2024-09-21T21:41:58.122 INFO:teuthology.orchestra.run.smithi164.stdout:4113 write 0x24b0200 thru 0x24b678f (0x6590 bytes) 2024-09-21T21:41:58.122 INFO:teuthology.orchestra.run.smithi164.stdout:4114 read 0x1372800 thru 0x13751f9 (0x29fa bytes) 2024-09-21T21:41:58.123 INFO:teuthology.orchestra.run.smithi164.stdout:4115 write 0x36d9a00 thru 0x36e238c (0x898d bytes) 2024-09-21T21:41:58.123 INFO:teuthology.orchestra.run.smithi164.stdout:4118 read 0xa512000 thru 0xa51ee5d (0xce5e bytes) 2024-09-21T21:41:58.123 INFO:teuthology.orchestra.run.smithi164.stdout:4119 punch from 0x6d3f800 to 0x6d4a800, (0xb000 bytes) 2024-09-21T21:41:58.133 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:58.132+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:58.134 INFO:teuthology.orchestra.run.smithi164.stdout:4120 trunc from 0xb8e9600 to 0x281f400 2024-09-21T21:41:58.152 INFO:teuthology.orchestra.run.smithi164.stdout:4121 read 0x12ff200 thru 0x13011a8 (0x1fa9 bytes) 2024-09-21T21:41:58.152 INFO:teuthology.orchestra.run.smithi164.stdout:4123 punch from 0x1ce0800 to 0x1ceea00, (0xe200 bytes) 2024-09-21T21:41:58.153 INFO:teuthology.orchestra.run.smithi164.stdout:4124 read 0x22f400 thru 0x23c1b7 (0xcdb8 bytes) 2024-09-21T21:41:58.154 INFO:teuthology.orchestra.run.smithi164.stdout:4125 read 0x246ca00 thru 0x2472b4f (0x6150 bytes) 2024-09-21T21:41:58.176 INFO:teuthology.orchestra.run.smithi164.stdout:4126 write 0xd0fb800 thru 0xd0fb951 (0x152 bytes) 2024-09-21T21:41:58.177 INFO:teuthology.orchestra.run.smithi164.stdout:4127 write 0xa5f6600 thru 0xa603e6b (0xd86c bytes) 2024-09-21T21:41:58.177 INFO:teuthology.orchestra.run.smithi164.stdout:4128 read 0x3a9d000 thru 0x3aa89a1 (0xb9a2 bytes) 2024-09-21T21:41:58.178 INFO:teuthology.orchestra.run.smithi164.stdout:skipping zero length punch hole 2024-09-21T21:41:58.178 INFO:teuthology.orchestra.run.smithi164.stdout:4136 write 0x666c000 thru 0x66719de (0x59df bytes) 2024-09-21T21:41:58.178 INFO:teuthology.orchestra.run.smithi164.stdout:4137 read 0x8a88400 thru 0x8a899f2 (0x15f3 bytes) 2024-09-21T21:41:58.178 INFO:teuthology.orchestra.run.smithi164.stdout:4138 read 0xac04200 thru 0xac0c84d (0x864e bytes) 2024-09-21T21:41:58.194 INFO:teuthology.orchestra.run.smithi164.stdout:4139 trunc from 0xd0fba00 to 0x14a1200 2024-09-21T21:41:58.197 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:58.196+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:58.200 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:58.200 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:58.210 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T21:41:58.224 INFO:teuthology.orchestra.run.smithi164.stdout:4143 punch from 0xc44400 to 0xc53800, (0xf400 bytes) 2024-09-21T21:41:58.246 INFO:teuthology.orchestra.run.smithi164.stdout:4145 write 0xc547a00 thru 0xc54ca58 (0x5059 bytes) 2024-09-21T21:41:58.247 INFO:teuthology.orchestra.run.smithi164.stdout:4146 read 0x11ec600 thru 0x11fc2ca (0xfccb bytes) 2024-09-21T21:41:58.247 INFO:teuthology.orchestra.run.smithi164.stdout:4147 punch from 0x130c600 to 0x131ae00, (0xe800 bytes) 2024-09-21T21:41:58.247 INFO:teuthology.orchestra.run.smithi164.stdout:4148 clone 33 order 20 su 524288 sc 6 2024-09-21T21:41:58.298 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:58.297+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:58.339 INFO:teuthology.orchestra.run.smithi164.stdout:truncating image image_client.0-clone32 from 0xc54cc00 (overlap 0x45f000) to 0x37f200 2024-09-21T21:41:58.367 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:58.367 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:58.377 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T21:41:58.385 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 5, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-21T21:41:58.385 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T21:41:58.385 INFO:tasks.thrashosds.thrasher:Removing osd 0, in_osds are: [0, 1, 2, 4, 5, 3] 2024-09-21T21:41:58.385 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 0 2024-09-21T21:41:58.399 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:58.397+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:58.484 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:58.484 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:58.493 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T21:41:58.537 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:58.536+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:58.600 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:58.598+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:58.602 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:58.602 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:58.611 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T21:41:58.682 INFO:teuthology.orchestra.run.smithi164.stdout:checking clone #31, image image_client.0-clone31 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent32 2024-09-21T21:41:58.700 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:58.699+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:58.718 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:58.718 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:58.726 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T21:41:58.729 INFO:tasks.ceph.mon.a.smithi081.stderr:2024-09-21T21:41:58.728+0000 7f5d67cb4640 -1 mon.a@0(leader).osd e85 definitely_dead 0 2024-09-21T21:41:58.751 INFO:teuthology.orchestra.run.smithi164.stdout:4153 punch from 0x6e60600 to 0x6e62e00, (0x2800 bytes) 2024-09-21T21:41:58.759 INFO:teuthology.orchestra.run.smithi164.stdout:4154 read 0xd32000 thru 0xd3e926 (0xc927 bytes) 2024-09-21T21:41:58.759 INFO:teuthology.orchestra.run.smithi164.stdout:4155 read 0x1d75200 thru 0x1d7deae (0x8caf bytes) 2024-09-21T21:41:58.760 INFO:teuthology.orchestra.run.smithi164.stdout:4157 read 0x6a53200 thru 0x6a59bc2 (0x69c3 bytes) 2024-09-21T21:41:58.760 INFO:teuthology.orchestra.run.smithi164.stdout:4161 write 0x611e600 thru 0x61257ba (0x71bb bytes) 2024-09-21T21:41:58.761 INFO:teuthology.orchestra.run.smithi164.stdout:4164 trunc from 0xc54cc00 to 0xbfd6c00 2024-09-21T21:41:58.777 INFO:teuthology.orchestra.run.smithi164.stdout:4165 punch from 0xaa42c00 to 0xaa4b400, (0x8800 bytes) 2024-09-21T21:41:58.785 INFO:teuthology.orchestra.run.smithi164.stdout:4169 punch from 0x88a8e00 to 0x88b1400, (0x8600 bytes) 2024-09-21T21:41:58.792 INFO:teuthology.orchestra.run.smithi164.stdout:4170 trunc from 0xbfd6c00 to 0x533ac00 2024-09-21T21:41:58.801 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:58.799+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:58.830 INFO:teuthology.orchestra.run.smithi164.stdout:4171 write 0x8a00400 thru 0x8a0c8aa (0xc4ab bytes) 2024-09-21T21:41:58.833 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:58.833 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:58.835 INFO:teuthology.orchestra.run.smithi164.stdout:4175 read 0x27cc800 thru 0x27d0c80 (0x4481 bytes) 2024-09-21T21:41:58.843 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T21:41:58.845 INFO:teuthology.orchestra.run.smithi164.stdout:4176 trunc from 0x8a0ca00 to 0xae5e00 2024-09-21T21:41:58.876 INFO:teuthology.orchestra.run.smithi164.stdout:4177 read 0x4bbe00 thru 0x4c8823 (0xca24 bytes) 2024-09-21T21:41:58.877 INFO:teuthology.orchestra.run.smithi164.stdout:4180 read 0x834000 thru 0x83b801 (0x7802 bytes) 2024-09-21T21:41:58.877 INFO:teuthology.orchestra.run.smithi164.stdout:4181 read 0x7a800 thru 0x81a1c (0x721d bytes) 2024-09-21T21:41:58.886 INFO:teuthology.orchestra.run.smithi164.stdout:4183 trunc from 0xae5e00 to 0x7947000 2024-09-21T21:41:58.902 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:41:58.901+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:41:58.906 INFO:teuthology.orchestra.run.smithi164.stdout:4187 trunc from 0x7947000 to 0xcd6d200 2024-09-21T21:41:58.915 INFO:teuthology.orchestra.run.smithi164.stdout:4188 write 0x89a8800 thru 0x89aef72 (0x6773 bytes) 2024-09-21T21:41:58.921 INFO:teuthology.orchestra.run.smithi164.stdout:4190 read 0x528d200 thru 0x529a68e (0xd48f bytes) 2024-09-21T21:41:58.921 INFO:teuthology.orchestra.run.smithi164.stdout:4195 read 0x7210800 thru 0x7213c24 (0x3425 bytes) 2024-09-21T21:41:58.921 INFO:teuthology.orchestra.run.smithi164.stdout:4196 write 0x6078600 thru 0x6086689 (0xe08a bytes) 2024-09-21T21:41:58.921 INFO:teuthology.orchestra.run.smithi164.stdout:4199 write 0xa58b400 thru 0xa595ab4 (0xa6b5 bytes) 2024-09-21T21:41:58.922 INFO:teuthology.orchestra.run.smithi164.stdout:4200 trunc from 0xcd6d200 to 0xd8a4800 2024-09-21T21:41:58.941 INFO:teuthology.orchestra.run.smithi164.stdout:4201 write 0x5e6bc00 thru 0x5e72294 (0x6695 bytes) 2024-09-21T21:41:58.952 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:58.953 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:58.962 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T21:41:58.962 INFO:teuthology.orchestra.run.smithi164.stdout:4202 trunc from 0xd8a4800 to 0xb25e00 2024-09-21T21:41:58.986 INFO:teuthology.orchestra.run.smithi164.stdout:4204 read 0x99f600 thru 0x9a7526 (0x7f27 bytes) 2024-09-21T21:41:59.003 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:59.001+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:59.015 INFO:teuthology.orchestra.run.smithi164.stdout:4205 write 0xde85200 thru 0xde90f82 (0xbd83 bytes) 2024-09-21T21:41:59.015 INFO:teuthology.orchestra.run.smithi164.stdout:4209 write 0xbdfbc00 thru 0xbe05384 (0x9785 bytes) 2024-09-21T21:41:59.015 INFO:teuthology.orchestra.run.smithi164.stdout:4210 read 0x1a1a600 thru 0x1a2465a (0xa05b bytes) 2024-09-21T21:41:59.015 INFO:teuthology.orchestra.run.smithi164.stdout:4211 read 0x8b5000 thru 0x8c3357 (0xe358 bytes) 2024-09-21T21:41:59.020 INFO:teuthology.orchestra.run.smithi164.stdout:4213 trunc from 0xde91000 to 0xa462000 2024-09-21T21:41:59.038 INFO:teuthology.orchestra.run.smithi164.stdout:4214 write 0x6b5f400 thru 0x6b642a8 (0x4ea9 bytes) 2024-09-21T21:41:59.051 INFO:teuthology.orchestra.run.smithi164.stdout:4215 write 0xad6f200 thru 0xad71965 (0x2766 bytes) 2024-09-21T21:41:59.059 INFO:teuthology.orchestra.run.smithi164.stdout:4219 trunc from 0xad71a00 to 0x518de00 2024-09-21T21:41:59.075 INFO:teuthology.orchestra.run.smithi164.stdout:4220 read 0x2018400 thru 0x2022768 (0xa369 bytes) 2024-09-21T21:41:59.075 INFO:teuthology.orchestra.run.smithi164.stdout:4228 write 0x4b90200 thru 0x4b90f74 (0xd75 bytes) 2024-09-21T21:41:59.076 INFO:teuthology.orchestra.run.smithi164.stdout:4229 clone 34 order 18 su 1024 sc 13 2024-09-21T21:41:59.104 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:59.101+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:59.111 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:59.111 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:59.120 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T21:41:59.205 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:59.202+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:59.223 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:59.223 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:59.231 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T21:41:59.271 INFO:teuthology.orchestra.run.smithi081.stderr:marked out osd.0. 2024-09-21T21:41:59.275 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T21:41:59.305 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:41:59.304+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:41:59.338 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:59.338 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:59.346 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T21:41:59.406 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:59.404+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:59.489 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:59.489 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:59.498 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T21:41:59.507 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:59.504+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:59.544 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T21:41:59.601 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:59.601 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:59.608 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:41:59.605+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:41:59.609 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T21:41:59.664 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0: {} 2024-09-21T21:41:59.664 INFO:teuthology.orchestra.run.smithi081.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T21:41:59.680 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1: {} 2024-09-21T21:41:59.681 INFO:teuthology.orchestra.run.smithi081.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T21:41:59.697 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2: {} 2024-09-21T21:41:59.697 INFO:teuthology.orchestra.run.smithi081.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T21:41:59.709 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:41:59.707+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:41:59.712 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:41:59.712 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:41:59.713 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3: {} 2024-09-21T21:41:59.714 INFO:teuthology.orchestra.run.smithi081.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T21:41:59.721 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T21:41:59.731 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4: {} 2024-09-21T21:41:59.731 INFO:teuthology.orchestra.run.smithi081.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T21:41:59.748 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5: {} 2024-09-21T21:41:59.748 INFO:teuthology.orchestra.run.smithi081.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T21:41:59.810 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:41:59.807+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:41:59.825 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:59.825 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:41:59.825 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 0 2024-09-21T21:41:59.825 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:59.834 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T21:41:59.910 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:41:59.909+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:41:59.978 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:41:59.978 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:41:59.978 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:41:59.978 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:41:59.978 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:41:59.986 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T21:42:00.011 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:00.009+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:00.090 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:00.090 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 20, 2024-09-21T21:42:00.091 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 600, 2024-09-21T21:42:00.091 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:42:00.091 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.091 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11319 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 5492736~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:00.091 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.712385+0000", 2024-09-21T21:42:00.091 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.3760642369999996, 2024-09-21T21:42:00.091 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078627889000000006, 2024-09-21T21:42:00.091 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:00.091 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:00.091 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:00.091 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:00.091 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:00.091 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:00.091 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11319 2024-09-21T21:42:00.092 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.092 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:00.092 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.092 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:00.092 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712385+0000", 2024-09-21T21:42:00.092 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:00.092 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.092 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.092 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:00.092 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712385+0000", 2024-09-21T21:42:00.092 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:00.092 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.092 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.092 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:00.093 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712385+0000", 2024-09-21T21:42:00.093 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2800000000000002e-07 2024-09-21T21:42:00.093 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.093 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.093 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:00.093 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712387+0000", 2024-09-21T21:42:00.093 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1569999999999998e-06 2024-09-21T21:42:00.093 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.093 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.093 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:00.093 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712393+0000", 2024-09-21T21:42:00.093 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9370000000000001e-06 2024-09-21T21:42:00.093 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.093 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.094 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:00.094 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712398+0000", 2024-09-21T21:42:00.094 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5020000000000003e-06 2024-09-21T21:42:00.094 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.094 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.094 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:00.094 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.790746+0000", 2024-09-21T21:42:00.094 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078348589999999996 2024-09-21T21:42:00.094 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.094 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.094 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:00.094 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.790844+0000", 2024-09-21T21:42:00.094 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.7526000000000001e-05 2024-09-21T21:42:00.094 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.095 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.095 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:00.095 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.791013+0000", 2024-09-21T21:42:00.095 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00016874899999999999 2024-09-21T21:42:00.095 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:00.095 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:00.095 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:00.095 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.095 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.095 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11320 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 6180864~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:00.095 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.712405+0000", 2024-09-21T21:42:00.095 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.3760442780000002, 2024-09-21T21:42:00.095 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078882870999999993, 2024-09-21T21:42:00.095 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:00.095 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:00.096 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:00.096 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:00.096 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:00.096 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:00.096 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11320 2024-09-21T21:42:00.096 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.096 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:00.096 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.096 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:00.096 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712405+0000", 2024-09-21T21:42:00.096 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:00.096 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.096 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.096 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:00.097 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712405+0000", 2024-09-21T21:42:00.097 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:00.097 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.097 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.097 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:00.097 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.713942+0000", 2024-09-21T21:42:00.097 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.001536882 2024-09-21T21:42:00.097 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.097 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.097 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:00.097 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.713946+0000", 2024-09-21T21:42:00.097 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4549999999999997e-06 2024-09-21T21:42:00.097 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.097 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.097 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:00.098 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.713958+0000", 2024-09-21T21:42:00.098 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1775e-05 2024-09-21T21:42:00.098 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.098 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.098 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:00.098 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.713965+0000", 2024-09-21T21:42:00.098 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5100000000000001e-06 2024-09-21T21:42:00.098 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.098 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.098 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:00.098 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.791059+0000", 2024-09-21T21:42:00.098 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.077094036000000005 2024-09-21T21:42:00.098 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.098 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.099 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:00.099 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.791175+0000", 2024-09-21T21:42:00.099 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000115895 2024-09-21T21:42:00.099 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.099 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.099 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:00.099 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.791288+0000", 2024-09-21T21:42:00.099 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000112318 2024-09-21T21:42:00.099 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:00.099 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:00.099 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:00.099 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.099 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.099 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11325 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 5414912~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:00.099 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.714095+0000", 2024-09-21T21:42:00.100 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.3743542830000006, 2024-09-21T21:42:00.100 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078631894999999993, 2024-09-21T21:42:00.100 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:00.100 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:00.100 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:00.100 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:00.100 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:00.100 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:00.100 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11325 2024-09-21T21:42:00.100 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.100 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:00.100 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.100 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:00.100 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714095+0000", 2024-09-21T21:42:00.100 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:00.101 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.101 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.101 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:00.101 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714095+0000", 2024-09-21T21:42:00.101 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:00.101 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.101 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.101 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:00.101 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714095+0000", 2024-09-21T21:42:00.101 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7500000000000001e-07 2024-09-21T21:42:00.101 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.101 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.101 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:00.101 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714097+0000", 2024-09-21T21:42:00.101 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.12e-06 2024-09-21T21:42:00.102 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.102 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.102 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:00.102 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714104+0000", 2024-09-21T21:42:00.102 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3520000000000003e-06 2024-09-21T21:42:00.102 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.102 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.102 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:00.102 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714108+0000", 2024-09-21T21:42:00.102 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8810000000000004e-06 2024-09-21T21:42:00.102 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.102 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.102 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:00.102 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792509+0000", 2024-09-21T21:42:00.102 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078400739999999997 2024-09-21T21:42:00.103 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.103 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.103 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:00.103 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792613+0000", 2024-09-21T21:42:00.103 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000103939 2024-09-21T21:42:00.103 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.103 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.103 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:00.103 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792727+0000", 2024-09-21T21:42:00.103 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011348799999999999 2024-09-21T21:42:00.103 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:00.103 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:00.103 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:00.103 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.104 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.104 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11326 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 6103040~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:00.104 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.714115+0000", 2024-09-21T21:42:00.104 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.3743342550000008, 2024-09-21T21:42:00.104 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078963056000000004, 2024-09-21T21:42:00.104 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:00.104 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:00.104 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:00.104 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:00.104 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:00.104 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:00.104 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11326 2024-09-21T21:42:00.104 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.104 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:00.104 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.105 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:00.105 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714115+0000", 2024-09-21T21:42:00.105 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:00.105 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.105 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.105 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:00.105 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714115+0000", 2024-09-21T21:42:00.105 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:00.105 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.105 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.105 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:00.105 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.715940+0000", 2024-09-21T21:42:00.105 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0018254969999999999 2024-09-21T21:42:00.105 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.106 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.106 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:00.106 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.715945+0000", 2024-09-21T21:42:00.106 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6120000000000001e-06 2024-09-21T21:42:00.106 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.106 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.106 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:00.106 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.715954+0000", 2024-09-21T21:42:00.106 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.5899999999999997e-06 2024-09-21T21:42:00.106 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.106 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.106 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:00.106 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.715959+0000", 2024-09-21T21:42:00.106 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0849999999999996e-06 2024-09-21T21:42:00.106 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.107 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.107 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:00.107 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792771+0000", 2024-09-21T21:42:00.107 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.076811829999999998 2024-09-21T21:42:00.107 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.107 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.107 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:00.107 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792882+0000", 2024-09-21T21:42:00.107 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000110857 2024-09-21T21:42:00.107 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.107 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.107 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:00.107 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.793078+0000", 2024-09-21T21:42:00.107 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00019558499999999999 2024-09-21T21:42:00.108 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:00.108 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:00.108 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:00.108 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.108 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.108 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11332 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 7565312~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:00.108 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.716063+0000", 2024-09-21T21:42:00.108 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.3723857210000006, 2024-09-21T21:42:00.108 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078787482000000006, 2024-09-21T21:42:00.108 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:00.108 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:00.108 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:00.108 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:00.108 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:00.108 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:00.109 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11332 2024-09-21T21:42:00.109 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.109 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:00.109 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.109 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:00.109 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.716063+0000", 2024-09-21T21:42:00.109 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:00.109 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.109 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.109 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:00.109 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.716063+0000", 2024-09-21T21:42:00.109 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:00.109 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.109 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.109 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:00.110 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.717944+0000", 2024-09-21T21:42:00.110 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0018812309999999999 2024-09-21T21:42:00.110 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.110 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.110 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:00.110 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.717949+0000", 2024-09-21T21:42:00.110 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9389999999999996e-06 2024-09-21T21:42:00.110 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.110 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.110 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:00.110 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.717963+0000", 2024-09-21T21:42:00.110 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.3899e-05 2024-09-21T21:42:00.110 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.110 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.110 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:00.111 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.717970+0000", 2024-09-21T21:42:00.111 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.426e-06 2024-09-21T21:42:00.111 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.111 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.111 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:00.111 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.794573+0000", 2024-09-21T21:42:00.111 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.076602990999999995 2024-09-21T21:42:00.111 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.111 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.111 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:00.111 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.794682+0000", 2024-09-21T21:42:00.111 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000109541 2024-09-21T21:42:00.111 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.111 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.112 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:00.112 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.794851+0000", 2024-09-21T21:42:00.112 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000168455 2024-09-21T21:42:00.112 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:00.112 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:00.112 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:00.112 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.112 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.112 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11339 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 6266880~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:00.112 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.718117+0000", 2024-09-21T21:42:00.112 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.3703316950000008, 2024-09-21T21:42:00.112 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078703717000000006, 2024-09-21T21:42:00.112 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:00.112 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:00.112 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:00.113 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:00.113 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:00.113 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:00.113 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11339 2024-09-21T21:42:00.113 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.113 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:00.113 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.113 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:00.113 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.718117+0000", 2024-09-21T21:42:00.113 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:00.113 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.113 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.113 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:00.113 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.718117+0000", 2024-09-21T21:42:00.113 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:00.114 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.114 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.114 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:00.114 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720019+0000", 2024-09-21T21:42:00.114 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0019012549999999999 2024-09-21T21:42:00.114 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.114 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.114 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:00.114 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720024+0000", 2024-09-21T21:42:00.114 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1100000000000002e-06 2024-09-21T21:42:00.114 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.114 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.114 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:00.114 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720035+0000", 2024-09-21T21:42:00.115 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.094e-05 2024-09-21T21:42:00.115 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.115 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.115 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:00.115 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720040+0000", 2024-09-21T21:42:00.115 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1830000000000002e-06 2024-09-21T21:42:00.115 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.115 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.115 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:00.115 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796618+0000", 2024-09-21T21:42:00.115 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.076578033000000004 2024-09-21T21:42:00.115 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.115 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.115 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:00.115 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796696+0000", 2024-09-21T21:42:00.116 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7836000000000005e-05 2024-09-21T21:42:00.116 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.116 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.116 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:00.116 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796821+0000", 2024-09-21T21:42:00.116 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00012536000000000001 2024-09-21T21:42:00.116 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:00.116 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:00.116 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:00.116 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.116 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.116 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11345 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 5472256~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:00.116 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.720116+0000", 2024-09-21T21:42:00.116 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.3683327950000006, 2024-09-21T21:42:00.116 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078667730000000005, 2024-09-21T21:42:00.117 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:00.117 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:00.117 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:00.117 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:00.117 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:00.117 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:00.117 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11345 2024-09-21T21:42:00.117 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.117 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:00.117 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.117 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:00.117 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720116+0000", 2024-09-21T21:42:00.117 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:00.117 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.118 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.118 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:00.118 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720116+0000", 2024-09-21T21:42:00.118 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:00.118 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.118 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.118 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:00.118 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720116+0000", 2024-09-21T21:42:00.118 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.04e-07 2024-09-21T21:42:00.118 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.118 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.118 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:00.118 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720118+0000", 2024-09-21T21:42:00.118 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.181e-06 2024-09-21T21:42:00.119 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.119 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.119 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:00.119 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720121+0000", 2024-09-21T21:42:00.119 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0199999999999999e-06 2024-09-21T21:42:00.119 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.119 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.119 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:00.119 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720123+0000", 2024-09-21T21:42:00.119 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.5469999999999999e-06 2024-09-21T21:42:00.119 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.119 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.119 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:00.119 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798626+0000", 2024-09-21T21:42:00.119 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078503137000000001 2024-09-21T21:42:00.120 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.120 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.120 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:00.120 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798677+0000", 2024-09-21T21:42:00.120 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0930999999999997e-05 2024-09-21T21:42:00.120 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.120 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.120 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:00.120 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798784+0000", 2024-09-21T21:42:00.120 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010671 2024-09-21T21:42:00.120 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:00.120 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:00.120 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:00.120 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.120 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.121 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11346 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 5500928~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:00.121 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.720126+0000", 2024-09-21T21:42:00.121 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.3683227220000003, 2024-09-21T21:42:00.121 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078863421000000003, 2024-09-21T21:42:00.121 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:00.121 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:00.121 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:00.121 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:00.121 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:00.121 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:00.121 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11346 2024-09-21T21:42:00.121 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.121 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:00.121 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.122 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:00.122 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720126+0000", 2024-09-21T21:42:00.122 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:00.122 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.122 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.122 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:00.122 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720126+0000", 2024-09-21T21:42:00.122 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:00.122 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.122 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.122 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:00.122 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.722145+0000", 2024-09-21T21:42:00.122 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0020183219999999999 2024-09-21T21:42:00.122 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.123 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.123 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:00.123 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.722150+0000", 2024-09-21T21:42:00.123 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1760000000000001e-06 2024-09-21T21:42:00.123 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.123 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.123 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:00.123 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.722161+0000", 2024-09-21T21:42:00.123 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1566000000000001e-05 2024-09-21T21:42:00.123 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.123 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.123 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:00.123 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.722169+0000", 2024-09-21T21:42:00.123 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.2840000000000001e-06 2024-09-21T21:42:00.123 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.124 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.124 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:00.124 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798815+0000", 2024-09-21T21:42:00.124 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.076646607000000005 2024-09-21T21:42:00.124 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.124 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.124 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:00.124 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798870+0000", 2024-09-21T21:42:00.124 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4486e-05 2024-09-21T21:42:00.124 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.124 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.124 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:00.124 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798990+0000", 2024-09-21T21:42:00.124 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011998 2024-09-21T21:42:00.125 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:00.125 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:00.125 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:00.125 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.125 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.125 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13629 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7852032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:00.125 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943319+0000", 2024-09-21T21:42:00.125 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.1451300799999995, 2024-09-21T21:42:00.125 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078610975999999999, 2024-09-21T21:42:00.125 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:00.125 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:00.125 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:00.125 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:00.125 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:00.125 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:00.126 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13629 2024-09-21T21:42:00.126 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.126 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:00.126 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.126 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:00.126 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943319+0000", 2024-09-21T21:42:00.126 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:00.126 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.126 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.126 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:00.126 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943319+0000", 2024-09-21T21:42:00.126 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:00.126 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.126 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.127 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:00.127 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943319+0000", 2024-09-21T21:42:00.127 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6199999999999999e-07 2024-09-21T21:42:00.127 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.127 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.127 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:00.127 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943321+0000", 2024-09-21T21:42:00.127 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1009999999999999e-06 2024-09-21T21:42:00.127 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.127 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.127 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:00.127 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943328+0000", 2024-09-21T21:42:00.127 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.1029999999999996e-06 2024-09-21T21:42:00.127 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.127 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.128 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:00.128 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943333+0000", 2024-09-21T21:42:00.128 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7709999999999997e-06 2024-09-21T21:42:00.128 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.128 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.128 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:00.128 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.021679+0000", 2024-09-21T21:42:00.128 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078345673000000005 2024-09-21T21:42:00.128 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.128 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.128 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:00.128 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.021771+0000", 2024-09-21T21:42:00.128 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2201999999999996e-05 2024-09-21T21:42:00.128 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.128 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.129 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:00.129 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.021930+0000", 2024-09-21T21:42:00.129 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00015876400000000001 2024-09-21T21:42:00.129 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:00.129 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:00.129 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:00.129 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.129 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.129 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13630 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7880704~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:00.129 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943339+0000", 2024-09-21T21:42:00.129 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.1451098920000007, 2024-09-21T21:42:00.129 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078939410000000002, 2024-09-21T21:42:00.129 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:00.129 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:00.129 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:00.130 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:00.130 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:00.130 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:00.130 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13630 2024-09-21T21:42:00.130 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.130 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:00.130 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.130 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:00.130 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943339+0000", 2024-09-21T21:42:00.130 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:00.130 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.130 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.130 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:00.130 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943339+0000", 2024-09-21T21:42:00.130 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:00.131 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.131 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.131 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:00.131 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943339+0000", 2024-09-21T21:42:00.131 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8000000000000001e-07 2024-09-21T21:42:00.131 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.131 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.131 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:00.131 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943341+0000", 2024-09-21T21:42:00.131 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9850000000000001e-06 2024-09-21T21:42:00.131 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.131 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.131 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:00.131 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943347+0000", 2024-09-21T21:42:00.132 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2979999999999999e-06 2024-09-21T21:42:00.132 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.132 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.132 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:00.132 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943354+0000", 2024-09-21T21:42:00.132 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.2969999999999998e-06 2024-09-21T21:42:00.132 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.132 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.132 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:00.132 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.021996+0000", 2024-09-21T21:42:00.132 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078641680000000005 2024-09-21T21:42:00.132 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.132 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.132 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:00.132 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022104+0000", 2024-09-21T21:42:00.133 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000107902 2024-09-21T21:42:00.133 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.133 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.133 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:00.133 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022278+0000", 2024-09-21T21:42:00.133 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00017486800000000001 2024-09-21T21:42:00.133 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:00.133 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:00.133 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:00.133 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.133 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.133 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13631 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7909376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:00.133 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943362+0000", 2024-09-21T21:42:00.133 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.1450870720000008, 2024-09-21T21:42:00.133 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079195644999999995, 2024-09-21T21:42:00.134 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:00.134 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:00.134 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:00.134 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:00.134 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:00.134 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:00.134 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13631 2024-09-21T21:42:00.134 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.134 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:00.134 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.134 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:00.134 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943362+0000", 2024-09-21T21:42:00.134 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:00.134 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.135 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.135 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:00.135 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943362+0000", 2024-09-21T21:42:00.135 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:00.135 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.135 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.135 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:00.135 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943362+0000", 2024-09-21T21:42:00.135 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8799999999999998e-07 2024-09-21T21:42:00.135 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.135 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.135 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:00.135 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943364+0000", 2024-09-21T21:42:00.135 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9850000000000001e-06 2024-09-21T21:42:00.135 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.136 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.136 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:00.136 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943371+0000", 2024-09-21T21:42:00.136 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0650000000000001e-06 2024-09-21T21:42:00.136 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.136 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.136 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:00.136 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943376+0000", 2024-09-21T21:42:00.136 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8250000000000004e-06 2024-09-21T21:42:00.136 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.136 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.136 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:00.136 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022329+0000", 2024-09-21T21:42:00.136 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078952888999999998 2024-09-21T21:42:00.136 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.137 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.137 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:00.137 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022419+0000", 2024-09-21T21:42:00.137 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.9738e-05 2024-09-21T21:42:00.137 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.137 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.137 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:00.137 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022558+0000", 2024-09-21T21:42:00.137 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00013875500000000001 2024-09-21T21:42:00.137 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:00.137 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:00.137 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:00.137 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.137 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.137 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13632 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7966720~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:00.138 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943382+0000", 2024-09-21T21:42:00.138 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.145066838, 2024-09-21T21:42:00.138 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079495727000000002, 2024-09-21T21:42:00.138 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:00.138 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:00.138 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:00.138 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:00.138 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:00.138 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:00.138 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13632 2024-09-21T21:42:00.138 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.138 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:00.138 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.138 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:00.138 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943382+0000", 2024-09-21T21:42:00.139 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:00.139 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.139 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.139 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:00.139 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943382+0000", 2024-09-21T21:42:00.139 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:00.139 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.139 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.139 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:00.139 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943382+0000", 2024-09-21T21:42:00.139 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7e-07 2024-09-21T21:42:00.139 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.139 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.139 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:00.139 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943385+0000", 2024-09-21T21:42:00.140 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2230000000000001e-06 2024-09-21T21:42:00.140 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.140 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.140 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:00.140 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943391+0000", 2024-09-21T21:42:00.140 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6810000000000001e-06 2024-09-21T21:42:00.140 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.140 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.140 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:00.140 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943397+0000", 2024-09-21T21:42:00.140 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6559999999999997e-06 2024-09-21T21:42:00.140 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.140 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.140 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:00.140 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022601+0000", 2024-09-21T21:42:00.141 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079204051999999997 2024-09-21T21:42:00.141 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.141 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.141 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:00.141 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022704+0000", 2024-09-21T21:42:00.141 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010329599999999999 2024-09-21T21:42:00.141 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.141 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.141 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:00.141 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022878+0000", 2024-09-21T21:42:00.141 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000173449 2024-09-21T21:42:00.141 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:00.141 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:00.141 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:00.142 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.142 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.142 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13633 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7938048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:00.142 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943403+0000", 2024-09-21T21:42:00.142 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.1450459080000002, 2024-09-21T21:42:00.142 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079794697999999997, 2024-09-21T21:42:00.142 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:00.142 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:00.142 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:00.142 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:00.142 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:00.142 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:00.142 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13633 2024-09-21T21:42:00.142 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.142 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:00.143 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.143 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:00.143 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943403+0000", 2024-09-21T21:42:00.143 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:00.143 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.143 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.143 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:00.143 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943403+0000", 2024-09-21T21:42:00.143 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:00.143 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.143 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.143 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:00.143 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943403+0000", 2024-09-21T21:42:00.143 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.65e-07 2024-09-21T21:42:00.143 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.144 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.144 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:00.144 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943405+0000", 2024-09-21T21:42:00.144 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0020000000000001e-06 2024-09-21T21:42:00.144 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.144 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.144 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:00.144 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943412+0000", 2024-09-21T21:42:00.144 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.8379999999999996e-06 2024-09-21T21:42:00.144 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.144 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.144 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:00.144 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943417+0000", 2024-09-21T21:42:00.144 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7650000000000001e-06 2024-09-21T21:42:00.145 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.145 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.145 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:00.145 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022922+0000", 2024-09-21T21:42:00.145 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079504512999999999 2024-09-21T21:42:00.145 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.145 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.145 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:00.145 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023039+0000", 2024-09-21T21:42:00.145 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000117044 2024-09-21T21:42:00.145 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.145 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.145 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:00.145 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023198+0000", 2024-09-21T21:42:00.145 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000159171 2024-09-21T21:42:00.146 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:00.146 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:00.146 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:00.146 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.146 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.146 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13634 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7995392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:00.146 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943423+0000", 2024-09-21T21:42:00.146 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.1450262000000002, 2024-09-21T21:42:00.146 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.080142358999999996, 2024-09-21T21:42:00.146 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:00.146 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:00.146 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:00.146 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:00.146 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:00.146 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:00.147 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13634 2024-09-21T21:42:00.147 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.147 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:00.147 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.147 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:00.147 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943423+0000", 2024-09-21T21:42:00.147 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:00.147 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.147 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.147 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:00.147 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943423+0000", 2024-09-21T21:42:00.147 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:00.147 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.147 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.147 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:00.148 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943423+0000", 2024-09-21T21:42:00.148 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.58e-07 2024-09-21T21:42:00.148 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.148 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.148 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:00.148 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943425+0000", 2024-09-21T21:42:00.148 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9180000000000001e-06 2024-09-21T21:42:00.148 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.148 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.148 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:00.148 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943431+0000", 2024-09-21T21:42:00.148 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4439999999999996e-06 2024-09-21T21:42:00.148 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.148 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.149 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:00.149 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943436+0000", 2024-09-21T21:42:00.149 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.887e-06 2024-09-21T21:42:00.149 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.149 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.149 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:00.149 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023266+0000", 2024-09-21T21:42:00.149 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079829622000000003 2024-09-21T21:42:00.149 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.149 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.149 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:00.149 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023384+0000", 2024-09-21T21:42:00.149 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000117658 2024-09-21T21:42:00.149 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.149 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.150 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:00.150 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023565+0000", 2024-09-21T21:42:00.150 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00018147200000000001 2024-09-21T21:42:00.150 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:00.150 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:00.150 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:00.150 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.150 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.150 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13635 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 8024064~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:00.150 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943447+0000", 2024-09-21T21:42:00.150 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.1450016600000001, 2024-09-21T21:42:00.150 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.080417653000000006, 2024-09-21T21:42:00.150 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:00.150 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:00.151 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:00.151 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:00.151 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:00.151 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:00.151 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13635 2024-09-21T21:42:00.151 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.151 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:00.151 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.151 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:00.151 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943447+0000", 2024-09-21T21:42:00.151 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:00.151 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.151 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.151 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:00.151 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943447+0000", 2024-09-21T21:42:00.152 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:00.152 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.152 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.152 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:00.152 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943448+0000", 2024-09-21T21:42:00.152 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1699999999999999e-07 2024-09-21T21:42:00.152 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.152 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.152 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:00.152 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943450+0000", 2024-09-21T21:42:00.152 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1639999999999999e-06 2024-09-21T21:42:00.152 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.152 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.152 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:00.152 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943456+0000", 2024-09-21T21:42:00.153 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8379999999999998e-06 2024-09-21T21:42:00.153 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.153 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.153 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:00.153 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943460+0000", 2024-09-21T21:42:00.153 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4700000000000004e-06 2024-09-21T21:42:00.153 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.153 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.153 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:00.153 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023595+0000", 2024-09-21T21:42:00.153 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.080134869999999997 2024-09-21T21:42:00.153 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.153 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.153 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:00.153 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023688+0000", 2024-09-21T21:42:00.154 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2794000000000006e-05 2024-09-21T21:42:00.154 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.154 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.154 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:00.154 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023865+0000", 2024-09-21T21:42:00.154 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0001771 2024-09-21T21:42:00.154 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:00.154 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:00.154 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:00.154 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.154 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.154 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13636 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 8052736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:00.154 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943467+0000", 2024-09-21T21:42:00.154 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.1449819229999996, 2024-09-21T21:42:00.154 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.080694399999999999, 2024-09-21T21:42:00.155 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:00.155 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:00.155 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:00.155 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:00.155 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:00.155 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:00.155 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13636 2024-09-21T21:42:00.155 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.155 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:00.155 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.155 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:00.155 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943467+0000", 2024-09-21T21:42:00.155 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:00.155 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.155 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.156 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:00.156 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943467+0000", 2024-09-21T21:42:00.156 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:00.156 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.156 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.156 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:00.156 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944940+0000", 2024-09-21T21:42:00.156 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0014725109999999999 2024-09-21T21:42:00.156 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.156 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.156 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:00.156 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944949+0000", 2024-09-21T21:42:00.156 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2930000000000006e-06 2024-09-21T21:42:00.156 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.156 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.157 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:00.157 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944958+0000", 2024-09-21T21:42:00.157 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.6899999999999998e-06 2024-09-21T21:42:00.157 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.157 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.157 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:00.157 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944966+0000", 2024-09-21T21:42:00.157 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0690000000000002e-06 2024-09-21T21:42:00.157 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.157 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.157 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:00.157 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023900+0000", 2024-09-21T21:42:00.157 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078934371000000003 2024-09-21T21:42:00.157 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.158 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.158 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:00.158 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024009+0000", 2024-09-21T21:42:00.158 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000108857 2024-09-21T21:42:00.158 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.158 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.158 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:00.158 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024161+0000", 2024-09-21T21:42:00.158 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000152609 2024-09-21T21:42:00.158 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:00.158 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:00.158 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:00.159 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.159 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.159 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13637 3.12 3.1dc31b72 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:00.159 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.944978+0000", 2024-09-21T21:42:00.159 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.1434709880000007, 2024-09-21T21:42:00.159 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079232604999999998, 2024-09-21T21:42:00.159 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:00.159 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:00.159 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:42:00.159 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:00.159 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:00.159 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:00.159 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13637 2024-09-21T21:42:00.159 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.159 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:00.160 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.160 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:00.160 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944978+0000", 2024-09-21T21:42:00.160 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:00.160 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.160 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.160 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:00.160 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944978+0000", 2024-09-21T21:42:00.160 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:00.160 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.160 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.160 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:00.160 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944978+0000", 2024-09-21T21:42:00.160 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9100000000000004e-07 2024-09-21T21:42:00.160 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.161 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.161 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:00.161 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944981+0000", 2024-09-21T21:42:00.161 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3290000000000001e-06 2024-09-21T21:42:00.161 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.161 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.161 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:00.161 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944987+0000", 2024-09-21T21:42:00.161 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.28e-06 2024-09-21T21:42:00.161 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.161 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.161 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:00.161 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944992+0000", 2024-09-21T21:42:00.161 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8520000000000003e-06 2024-09-21T21:42:00.162 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.162 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.162 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:00.162 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024196+0000", 2024-09-21T21:42:00.162 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079203673000000002 2024-09-21T21:42:00.162 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.162 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.162 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:00.162 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024211+0000", 2024-09-21T21:42:00.162 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.4980000000000001e-05 2024-09-21T21:42:00.162 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:00.162 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:00.162 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:00.163 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.163 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.163 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13638 3.12 3.1dc31b72 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:00.163 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.945000+0000", 2024-09-21T21:42:00.163 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.1434488599999995, 2024-09-21T21:42:00.163 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079257465999999999, 2024-09-21T21:42:00.163 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:00.163 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:00.163 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:42:00.163 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:00.163 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:00.163 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:00.163 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13638 2024-09-21T21:42:00.163 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.164 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:00.164 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.164 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:00.164 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945000+0000", 2024-09-21T21:42:00.164 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:00.164 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.164 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.164 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:00.164 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945000+0000", 2024-09-21T21:42:00.164 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:00.164 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.164 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.164 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:00.164 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945001+0000", 2024-09-21T21:42:00.164 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.32e-07 2024-09-21T21:42:00.165 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.165 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.165 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:00.165 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945003+0000", 2024-09-21T21:42:00.165 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2309999999999999e-06 2024-09-21T21:42:00.165 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.165 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.165 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:00.165 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945009+0000", 2024-09-21T21:42:00.165 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7450000000000001e-06 2024-09-21T21:42:00.165 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.165 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.165 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:00.165 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945013+0000", 2024-09-21T21:42:00.165 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.673e-06 2024-09-21T21:42:00.166 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.166 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.166 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:00.166 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024245+0000", 2024-09-21T21:42:00.166 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079232160999999995 2024-09-21T21:42:00.166 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.166 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.166 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:00.166 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024258+0000", 2024-09-21T21:42:00.166 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.2224e-05 2024-09-21T21:42:00.166 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:00.166 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:00.166 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:00.166 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.167 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.167 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13639 3.12 3.1dc31b72 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:00.167 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.945034+0000", 2024-09-21T21:42:00.167 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.1434150610000007, 2024-09-21T21:42:00.167 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079260627, 2024-09-21T21:42:00.167 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:00.167 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:00.167 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:42:00.167 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:00.167 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:00.167 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:00.167 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13639 2024-09-21T21:42:00.167 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.167 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:00.167 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.168 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:00.168 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945034+0000", 2024-09-21T21:42:00.168 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:00.168 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.168 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.168 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:00.168 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945034+0000", 2024-09-21T21:42:00.168 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:00.168 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.168 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.168 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:00.168 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945034+0000", 2024-09-21T21:42:00.168 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3799999999999998e-07 2024-09-21T21:42:00.168 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.169 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.169 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:00.169 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945036+0000", 2024-09-21T21:42:00.169 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.052e-06 2024-09-21T21:42:00.169 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.169 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.169 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:00.169 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945040+0000", 2024-09-21T21:42:00.169 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1169999999999997e-06 2024-09-21T21:42:00.169 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.169 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.169 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:00.169 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945043+0000", 2024-09-21T21:42:00.169 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.8439999999999998e-06 2024-09-21T21:42:00.169 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.169 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.170 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:00.170 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024286+0000", 2024-09-21T21:42:00.170 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079242483000000002 2024-09-21T21:42:00.170 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.170 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.170 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:00.170 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024295+0000", 2024-09-21T21:42:00.170 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.7930000000000003e-06 2024-09-21T21:42:00.170 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:00.170 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:00.170 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:00.170 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.170 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.170 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13640 3.12 3.1dc31b72 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:00.171 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.945047+0000", 2024-09-21T21:42:00.171 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.1434015389999992, 2024-09-21T21:42:00.171 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079412321999999994, 2024-09-21T21:42:00.171 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:00.171 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:00.171 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:42:00.171 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:00.171 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:00.171 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:00.171 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13640 2024-09-21T21:42:00.171 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.171 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:00.171 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.171 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:00.172 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945047+0000", 2024-09-21T21:42:00.172 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:00.172 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.172 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.172 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:00.172 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945047+0000", 2024-09-21T21:42:00.172 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:00.172 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.172 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.172 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:00.172 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945048+0000", 2024-09-21T21:42:00.172 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0499999999999999e-07 2024-09-21T21:42:00.172 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.172 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.172 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:00.172 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945049+0000", 2024-09-21T21:42:00.173 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.3939999999999999e-06 2024-09-21T21:42:00.173 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.173 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.173 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:00.173 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945053+0000", 2024-09-21T21:42:00.173 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1760000000000003e-06 2024-09-21T21:42:00.173 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.173 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.173 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:00.173 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945056+0000", 2024-09-21T21:42:00.173 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.6129999999999998e-06 2024-09-21T21:42:00.173 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.173 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.173 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:00.173 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024360+0000", 2024-09-21T21:42:00.174 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079303645000000006 2024-09-21T21:42:00.174 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.174 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.174 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:00.174 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024460+0000", 2024-09-21T21:42:00.174 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000100189 2024-09-21T21:42:00.174 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:00.174 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:00.174 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:00.174 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:00.174 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:00.174 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:00.176 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T21:42:00.176 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:00.174+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:00.269 INFO:teuthology.orchestra.run.smithi081.stderr:noscrub is set 2024-09-21T21:42:00.276 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:00.274+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:00.327 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:00.327 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:42:00.327 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 0 2024-09-21T21:42:00.327 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:00.336 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T21:42:00.377 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:00.376+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:00.478 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:00.477+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:00.479 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:00.480 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:42:00.480 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:42:00.480 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:42:00.480 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:00.488 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T21:42:00.579 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:00.578+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:00.592 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:00.592 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 20, 2024-09-21T21:42:00.592 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 600, 2024-09-21T21:42:00.593 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:42:00.593 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.593 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4824.0:1564 3.2 3:4647520c:::rbd_data.1121b97f2925.0000000000000006:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:42:00.593 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:42:00.593 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 58.954290637, 2024-09-21T21:42:00.593 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056097347999999998, 2024-09-21T21:42:00.593 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:00.593 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:00.593 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:00.593 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:00.593 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4824", 2024-09-21T21:42:00.593 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:42:00.593 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1564 2024-09-21T21:42:00.594 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.594 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:00.594 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.594 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:00.594 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:42:00.594 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:00.594 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.594 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.594 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:00.594 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:42:00.594 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:00.595 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.595 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.595 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:00.595 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:42:00.595 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6499999999999999e-07 2024-09-21T21:42:00.595 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.595 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.595 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:00.595 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636067+0000", 2024-09-21T21:42:00.595 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3879999999999998e-06 2024-09-21T21:42:00.595 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.596 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.596 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:00.596 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636075+0000", 2024-09-21T21:42:00.596 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5480000000000004e-06 2024-09-21T21:42:00.596 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.596 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.596 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:00.596 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636080+0000", 2024-09-21T21:42:00.596 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.339e-06 2024-09-21T21:42:00.596 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.596 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.596 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:00.596 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636133+0000", 2024-09-21T21:42:00.597 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2930000000000003e-05 2024-09-21T21:42:00.597 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.597 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.597 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:00.597 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636265+0000", 2024-09-21T21:42:00.597 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000132058 2024-09-21T21:42:00.597 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.597 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.597 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:00.597 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.692162+0000", 2024-09-21T21:42:00.597 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055896620000000001 2024-09-21T21:42:00.597 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:00.597 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:00.597 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:00.598 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.598 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.598 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3258 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 1409024~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:42:00.598 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.196243+0000", 2024-09-21T21:42:00.598 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 13.394111797000001, 2024-09-21T21:42:00.598 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055725496999999999, 2024-09-21T21:42:00.598 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:00.598 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:00.598 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:00.598 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:00.598 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:00.598 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:00.598 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3258 2024-09-21T21:42:00.598 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.599 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:00.599 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.599 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:00.599 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196243+0000", 2024-09-21T21:42:00.599 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:00.599 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.599 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.599 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:00.599 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196243+0000", 2024-09-21T21:42:00.599 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:00.599 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.599 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.599 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:00.599 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196244+0000", 2024-09-21T21:42:00.600 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4799999999999999e-07 2024-09-21T21:42:00.600 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.600 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.600 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:00.600 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196246+0000", 2024-09-21T21:42:00.600 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1390000000000002e-06 2024-09-21T21:42:00.600 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.600 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.600 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:00.601 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196255+0000", 2024-09-21T21:42:00.601 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.5779999999999999e-06 2024-09-21T21:42:00.601 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.601 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.601 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:00.601 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196267+0000", 2024-09-21T21:42:00.601 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.2726e-05 2024-09-21T21:42:00.601 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.601 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.602 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:00.602 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.251811+0000", 2024-09-21T21:42:00.602 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055544068000000002 2024-09-21T21:42:00.602 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.602 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.602 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:00.602 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.251878+0000", 2024-09-21T21:42:00.602 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6898999999999998e-05 2024-09-21T21:42:00.603 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.603 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.603 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:00.603 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.251969+0000", 2024-09-21T21:42:00.603 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.0638999999999997e-05 2024-09-21T21:42:00.603 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:00.603 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:00.603 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:00.603 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.604 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.604 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3259 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 663552~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:42:00.604 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.196275+0000", 2024-09-21T21:42:00.604 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 13.394080489, 2024-09-21T21:42:00.604 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055889580000000001, 2024-09-21T21:42:00.604 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:00.604 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:00.604 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:00.604 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:00.605 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:00.605 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:00.605 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3259 2024-09-21T21:42:00.605 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.605 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:00.605 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.605 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:00.605 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196275+0000", 2024-09-21T21:42:00.606 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:00.606 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.606 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.606 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:00.606 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196275+0000", 2024-09-21T21:42:00.606 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:00.606 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.606 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.607 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:00.607 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196275+0000", 2024-09-21T21:42:00.607 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2199999999999999e-07 2024-09-21T21:42:00.607 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.607 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.607 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:00.607 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196277+0000", 2024-09-21T21:42:00.607 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1270000000000001e-06 2024-09-21T21:42:00.607 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.608 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.608 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:00.608 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196285+0000", 2024-09-21T21:42:00.608 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.2290000000000003e-06 2024-09-21T21:42:00.608 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.608 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.608 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:00.608 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196291+0000", 2024-09-21T21:42:00.608 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1379999999999998e-06 2024-09-21T21:42:00.609 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.609 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.609 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:00.609 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.251997+0000", 2024-09-21T21:42:00.609 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055705971999999999 2024-09-21T21:42:00.609 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.609 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.609 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:00.609 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252077+0000", 2024-09-21T21:42:00.610 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0179999999999995e-05 2024-09-21T21:42:00.610 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.610 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.610 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:00.610 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252164+0000", 2024-09-21T21:42:00.610 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.7511999999999998e-05 2024-09-21T21:42:00.610 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:00.610 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:00.610 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:00.611 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.611 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.611 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3260 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 1437696~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:42:00.611 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.196297+0000", 2024-09-21T21:42:00.611 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 13.394058426999999, 2024-09-21T21:42:00.611 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056071022999999998, 2024-09-21T21:42:00.611 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:00.611 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:00.612 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:00.612 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:00.612 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:00.612 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:00.612 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3260 2024-09-21T21:42:00.612 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.612 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:00.612 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.612 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:00.613 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196297+0000", 2024-09-21T21:42:00.613 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:00.613 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.613 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.613 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:00.613 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196297+0000", 2024-09-21T21:42:00.613 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:00.613 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.613 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.614 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:00.614 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196918+0000", 2024-09-21T21:42:00.614 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00062148699999999999 2024-09-21T21:42:00.614 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.614 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.614 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:00.614 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196930+0000", 2024-09-21T21:42:00.614 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1657e-05 2024-09-21T21:42:00.614 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.615 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.615 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:00.615 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196944+0000", 2024-09-21T21:42:00.615 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.4549000000000001e-05 2024-09-21T21:42:00.615 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.615 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.615 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:00.615 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196951+0000", 2024-09-21T21:42:00.616 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6749999999999996e-06 2024-09-21T21:42:00.616 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.616 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.616 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:00.616 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252208+0000", 2024-09-21T21:42:00.616 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055256656000000001 2024-09-21T21:42:00.616 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.616 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.616 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:00.616 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252278+0000", 2024-09-21T21:42:00.616 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9882000000000006e-05 2024-09-21T21:42:00.617 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.617 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.617 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:00.617 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252368+0000", 2024-09-21T21:42:00.617 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.0117000000000005e-05 2024-09-21T21:42:00.617 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:00.617 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:00.617 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:00.617 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.617 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.617 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3261 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 2842624~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:42:00.617 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.196963+0000", 2024-09-21T21:42:00.617 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 13.39339204, 2024-09-21T21:42:00.617 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055594613000000001, 2024-09-21T21:42:00.617 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:00.618 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:00.618 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:00.618 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:00.618 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:00.618 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:00.618 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3261 2024-09-21T21:42:00.618 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.618 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:00.618 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.618 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:00.618 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196963+0000", 2024-09-21T21:42:00.618 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:00.618 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.618 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.619 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:00.619 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196963+0000", 2024-09-21T21:42:00.619 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:00.619 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.619 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.619 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:00.619 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196964+0000", 2024-09-21T21:42:00.619 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3000000000000001e-07 2024-09-21T21:42:00.619 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.619 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.620 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:00.620 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196966+0000", 2024-09-21T21:42:00.620 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.475e-06 2024-09-21T21:42:00.620 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.620 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.620 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:00.620 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196973+0000", 2024-09-21T21:42:00.620 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5540000000000003e-06 2024-09-21T21:42:00.620 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.620 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.620 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:00.620 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196978+0000", 2024-09-21T21:42:00.620 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0719999999999999e-06 2024-09-21T21:42:00.620 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.621 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.621 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:00.621 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252395+0000", 2024-09-21T21:42:00.621 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055417301000000002 2024-09-21T21:42:00.621 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.621 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.621 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:00.621 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252468+0000", 2024-09-21T21:42:00.621 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.2692000000000001e-05 2024-09-21T21:42:00.622 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.622 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.622 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:00.622 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252558+0000", 2024-09-21T21:42:00.622 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.9988999999999995e-05 2024-09-21T21:42:00.622 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:00.622 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:00.622 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:00.622 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.622 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.622 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3262 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 2871296~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:42:00.622 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.196985+0000", 2024-09-21T21:42:00.622 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 13.393370548, 2024-09-21T21:42:00.623 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055777262000000001, 2024-09-21T21:42:00.623 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:00.623 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:00.623 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:00.623 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:00.623 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:00.623 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:00.623 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3262 2024-09-21T21:42:00.623 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.623 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:00.623 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.623 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:00.623 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196985+0000", 2024-09-21T21:42:00.623 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:00.624 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.624 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.624 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:00.624 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196985+0000", 2024-09-21T21:42:00.624 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:00.624 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.624 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.624 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:00.624 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197024+0000", 2024-09-21T21:42:00.624 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.9558000000000003e-05 2024-09-21T21:42:00.624 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.624 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.624 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:00.624 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197026+0000", 2024-09-21T21:42:00.625 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0899999999999999e-06 2024-09-21T21:42:00.625 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.625 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.625 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:00.625 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197031+0000", 2024-09-21T21:42:00.625 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8849999999999998e-06 2024-09-21T21:42:00.625 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.625 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.625 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:00.625 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197035+0000", 2024-09-21T21:42:00.625 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5719999999999999e-06 2024-09-21T21:42:00.625 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.625 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.625 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:00.625 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252587+0000", 2024-09-21T21:42:00.626 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055552193 2024-09-21T21:42:00.626 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.626 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.626 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:00.626 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252670+0000", 2024-09-21T21:42:00.626 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.2687000000000003e-05 2024-09-21T21:42:00.626 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.626 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.626 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:00.626 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252762+0000", 2024-09-21T21:42:00.626 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2276999999999998e-05 2024-09-21T21:42:00.626 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:00.626 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:00.626 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:00.627 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.627 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.627 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3263 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 3559424~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:42:00.627 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.197041+0000", 2024-09-21T21:42:00.627 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 13.393314612999999, 2024-09-21T21:42:00.627 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055935804999999998, 2024-09-21T21:42:00.627 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:00.627 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:00.627 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:00.627 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:00.627 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:00.627 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:00.627 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3263 2024-09-21T21:42:00.627 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.627 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:00.628 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.628 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:00.628 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197041+0000", 2024-09-21T21:42:00.628 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:00.628 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.628 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.628 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:00.628 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197041+0000", 2024-09-21T21:42:00.628 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:00.628 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.628 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.628 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:00.628 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197624+0000", 2024-09-21T21:42:00.628 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00058358100000000001 2024-09-21T21:42:00.629 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.629 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.629 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:00.629 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197629+0000", 2024-09-21T21:42:00.629 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5909999999999998e-06 2024-09-21T21:42:00.629 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.629 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.629 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:00.629 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197639+0000", 2024-09-21T21:42:00.629 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0257e-05 2024-09-21T21:42:00.629 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.629 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.629 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:00.629 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197646+0000", 2024-09-21T21:42:00.629 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.7229999999999998e-06 2024-09-21T21:42:00.630 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.630 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.630 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:00.630 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252804+0000", 2024-09-21T21:42:00.630 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055158134999999997 2024-09-21T21:42:00.630 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.630 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.630 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:00.630 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252882+0000", 2024-09-21T21:42:00.630 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7791000000000001e-05 2024-09-21T21:42:00.630 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.630 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.630 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:00.630 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252976+0000", 2024-09-21T21:42:00.631 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.4727000000000003e-05 2024-09-21T21:42:00.631 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:00.631 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:00.631 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:00.631 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.631 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.631 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3325 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:42:00.631 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.197722+0000", 2024-09-21T21:42:00.631 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 13.392632796000001, 2024-09-21T21:42:00.631 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055712048, 2024-09-21T21:42:00.631 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:00.631 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:00.632 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:00.632 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:00.632 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:00.632 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:00.632 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3325 2024-09-21T21:42:00.632 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.632 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:00.632 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.632 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:00.632 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197722+0000", 2024-09-21T21:42:00.632 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:00.632 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.633 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.633 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:00.633 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197722+0000", 2024-09-21T21:42:00.633 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:00.633 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.633 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.633 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:00.633 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.198052+0000", 2024-09-21T21:42:00.633 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00033006600000000001 2024-09-21T21:42:00.633 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.633 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.633 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:00.633 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.198056+0000", 2024-09-21T21:42:00.633 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7330000000000001e-06 2024-09-21T21:42:00.634 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.634 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.634 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:00.634 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.198065+0000", 2024-09-21T21:42:00.634 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.8990000000000007e-06 2024-09-21T21:42:00.634 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.634 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.634 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:00.634 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.198072+0000", 2024-09-21T21:42:00.634 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5339999999999996e-06 2024-09-21T21:42:00.635 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.635 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.635 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:00.635 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.253244+0000", 2024-09-21T21:42:00.635 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055172064999999999 2024-09-21T21:42:00.635 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.635 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.635 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:00.635 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.253342+0000", 2024-09-21T21:42:00.635 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.8175000000000001e-05 2024-09-21T21:42:00.635 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.636 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.636 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:00.636 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.253434+0000", 2024-09-21T21:42:00.636 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2576000000000003e-05 2024-09-21T21:42:00.636 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:00.636 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:00.636 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:00.636 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.636 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.636 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3437 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 925696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:42:00.636 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.205888+0000", 2024-09-21T21:42:00.636 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 13.384467332, 2024-09-21T21:42:00.636 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055953808000000001, 2024-09-21T21:42:00.636 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:00.637 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:00.637 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:00.637 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:00.637 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:00.637 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:00.637 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3437 2024-09-21T21:42:00.637 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.637 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:00.637 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.637 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:00.637 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205888+0000", 2024-09-21T21:42:00.637 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:00.637 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.638 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.638 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:00.638 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205888+0000", 2024-09-21T21:42:00.638 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:00.638 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.638 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.638 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:00.638 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205888+0000", 2024-09-21T21:42:00.638 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.75e-07 2024-09-21T21:42:00.638 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.638 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.638 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:00.638 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205891+0000", 2024-09-21T21:42:00.638 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3750000000000001e-06 2024-09-21T21:42:00.639 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.639 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.639 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:00.639 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205897+0000", 2024-09-21T21:42:00.639 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.285e-06 2024-09-21T21:42:00.639 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.639 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.639 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:00.639 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205902+0000", 2024-09-21T21:42:00.639 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6290000000000001e-06 2024-09-21T21:42:00.639 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.641 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.641 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:00.641 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261686+0000", 2024-09-21T21:42:00.641 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055784586999999997 2024-09-21T21:42:00.642 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.642 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.642 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:00.642 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261764+0000", 2024-09-21T21:42:00.642 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7591999999999998e-05 2024-09-21T21:42:00.642 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.642 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.642 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:00.642 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261842+0000", 2024-09-21T21:42:00.643 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7865000000000001e-05 2024-09-21T21:42:00.643 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:00.643 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:00.643 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:00.643 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.643 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.643 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3438 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 954368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:42:00.643 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.205908+0000", 2024-09-21T21:42:00.643 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 13.384447466999999, 2024-09-21T21:42:00.643 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056083169000000002, 2024-09-21T21:42:00.643 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:00.643 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:00.643 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:00.643 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:00.644 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:00.644 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:00.644 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3438 2024-09-21T21:42:00.644 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.644 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:00.644 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.644 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:00.644 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205908+0000", 2024-09-21T21:42:00.644 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:00.644 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.644 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.644 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:00.644 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205908+0000", 2024-09-21T21:42:00.644 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:00.645 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.645 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.645 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:00.645 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206431+0000", 2024-09-21T21:42:00.645 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00052282100000000003 2024-09-21T21:42:00.645 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.645 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.645 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:00.645 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206442+0000", 2024-09-21T21:42:00.645 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1695000000000001e-05 2024-09-21T21:42:00.645 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.645 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.645 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:00.645 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206452+0000", 2024-09-21T21:42:00.646 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.4560000000000006e-06 2024-09-21T21:42:00.646 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.646 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.646 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:00.646 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206458+0000", 2024-09-21T21:42:00.646 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4670000000000001e-06 2024-09-21T21:42:00.646 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.646 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.646 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:00.646 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261871+0000", 2024-09-21T21:42:00.646 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055412712000000003 2024-09-21T21:42:00.646 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.646 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.646 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:00.647 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261928+0000", 2024-09-21T21:42:00.647 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6631e-05 2024-09-21T21:42:00.647 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.647 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.647 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:00.647 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261991+0000", 2024-09-21T21:42:00.647 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3386999999999995e-05 2024-09-21T21:42:00.647 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:00.647 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:00.647 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:00.647 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.647 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.647 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3440 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 1642496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:42:00.647 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.206491+0000", 2024-09-21T21:42:00.648 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 13.383864288, 2024-09-21T21:42:00.648 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055633136, 2024-09-21T21:42:00.648 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:00.648 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:00.648 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:00.648 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:00.648 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:00.648 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:00.648 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3440 2024-09-21T21:42:00.648 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.648 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:00.648 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.648 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:00.648 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206491+0000", 2024-09-21T21:42:00.649 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:00.649 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.649 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.649 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:00.649 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206491+0000", 2024-09-21T21:42:00.649 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:00.649 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.649 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.649 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:00.649 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206491+0000", 2024-09-21T21:42:00.649 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6100000000000001e-07 2024-09-21T21:42:00.649 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.649 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.649 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:00.649 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206494+0000", 2024-09-21T21:42:00.650 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.9349999999999999e-06 2024-09-21T21:42:00.650 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.650 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.650 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:00.650 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206501+0000", 2024-09-21T21:42:00.650 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6939999999999998e-06 2024-09-21T21:42:00.650 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.650 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.650 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:00.650 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206506+0000", 2024-09-21T21:42:00.650 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2979999999999999e-06 2024-09-21T21:42:00.650 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.650 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.650 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:00.651 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262019+0000", 2024-09-21T21:42:00.651 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055513094999999998 2024-09-21T21:42:00.651 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.651 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.651 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:00.651 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262073+0000", 2024-09-21T21:42:00.651 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3486999999999998e-05 2024-09-21T21:42:00.651 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.651 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.651 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:00.651 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262124+0000", 2024-09-21T21:42:00.651 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1066000000000002e-05 2024-09-21T21:42:00.651 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:00.651 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:00.651 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:00.652 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.652 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.652 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3442 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 1671168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:42:00.652 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.206515+0000", 2024-09-21T21:42:00.652 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 13.383840586, 2024-09-21T21:42:00.652 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055792680999999997, 2024-09-21T21:42:00.652 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:00.652 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:00.652 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:00.652 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:00.652 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:00.652 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:00.653 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3442 2024-09-21T21:42:00.653 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.653 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:00.653 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.653 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:00.653 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206515+0000", 2024-09-21T21:42:00.653 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:00.653 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.653 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.653 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:00.653 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206515+0000", 2024-09-21T21:42:00.654 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:00.654 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.654 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.654 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:00.654 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206515+0000", 2024-09-21T21:42:00.654 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2500000000000001e-07 2024-09-21T21:42:00.654 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.654 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.654 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:00.654 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206517+0000", 2024-09-21T21:42:00.654 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0420000000000001e-06 2024-09-21T21:42:00.654 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.654 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.654 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:00.655 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206522+0000", 2024-09-21T21:42:00.655 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3600000000000004e-06 2024-09-21T21:42:00.655 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.655 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.655 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:00.655 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206527+0000", 2024-09-21T21:42:00.655 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.809e-06 2024-09-21T21:42:00.655 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.655 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.655 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:00.655 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262151+0000", 2024-09-21T21:42:00.655 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055623458000000001 2024-09-21T21:42:00.655 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.655 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.656 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:00.656 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262210+0000", 2024-09-21T21:42:00.656 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9120999999999999e-05 2024-09-21T21:42:00.656 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.656 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.656 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:00.656 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262307+0000", 2024-09-21T21:42:00.656 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.7466000000000005e-05 2024-09-21T21:42:00.656 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:00.656 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:00.656 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:00.656 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.656 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.656 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3444 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 2359296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:42:00.657 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.206533+0000", 2024-09-21T21:42:00.657 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 13.383821946999999, 2024-09-21T21:42:00.657 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055974239000000002, 2024-09-21T21:42:00.657 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:00.657 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:00.657 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:00.657 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:00.657 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:00.657 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:00.657 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3444 2024-09-21T21:42:00.657 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.657 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:00.657 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.657 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:00.658 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206533+0000", 2024-09-21T21:42:00.658 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:00.658 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.658 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.658 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:00.658 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206533+0000", 2024-09-21T21:42:00.658 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:00.658 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.658 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.658 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:00.658 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206710+0000", 2024-09-21T21:42:00.658 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000176293 2024-09-21T21:42:00.658 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.659 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.659 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:00.659 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206714+0000", 2024-09-21T21:42:00.659 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1989999999999999e-06 2024-09-21T21:42:00.659 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.659 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.659 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:00.659 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206723+0000", 2024-09-21T21:42:00.659 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.9199999999999993e-06 2024-09-21T21:42:00.659 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.659 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.659 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:00.660 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206729+0000", 2024-09-21T21:42:00.660 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0739999999999998e-06 2024-09-21T21:42:00.660 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.660 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.660 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:00.660 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262339+0000", 2024-09-21T21:42:00.660 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055610166000000003 2024-09-21T21:42:00.660 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.660 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.660 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:00.660 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262419+0000", 2024-09-21T21:42:00.660 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0432999999999994e-05 2024-09-21T21:42:00.660 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.661 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.661 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:00.661 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262507+0000", 2024-09-21T21:42:00.661 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.8153999999999997e-05 2024-09-21T21:42:00.661 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:00.661 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:00.661 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:00.661 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.661 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.661 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3445 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 1699840~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:42:00.661 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.206746+0000", 2024-09-21T21:42:00.661 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 13.383609023, 2024-09-21T21:42:00.661 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055880738999999999, 2024-09-21T21:42:00.662 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:00.662 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:00.662 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:00.662 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:00.662 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:00.662 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:00.662 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3445 2024-09-21T21:42:00.662 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.662 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:00.662 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.662 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:00.662 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206746+0000", 2024-09-21T21:42:00.662 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:00.662 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.663 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.663 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:00.663 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206746+0000", 2024-09-21T21:42:00.663 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:00.663 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.663 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.663 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:00.663 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207412+0000", 2024-09-21T21:42:00.664 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00066611999999999997 2024-09-21T21:42:00.664 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.664 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.664 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:00.664 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207417+0000", 2024-09-21T21:42:00.664 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8539999999999996e-06 2024-09-21T21:42:00.664 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.665 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.665 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:00.665 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207437+0000", 2024-09-21T21:42:00.665 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9463e-05 2024-09-21T21:42:00.665 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.665 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.665 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:00.665 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207445+0000", 2024-09-21T21:42:00.665 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.5790000000000004e-06 2024-09-21T21:42:00.665 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.665 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.666 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:00.666 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262537+0000", 2024-09-21T21:42:00.666 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055091531999999999 2024-09-21T21:42:00.666 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.666 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.666 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:00.666 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262578+0000", 2024-09-21T21:42:00.666 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1220000000000002e-05 2024-09-21T21:42:00.666 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.666 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.666 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:00.666 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262627+0000", 2024-09-21T21:42:00.666 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8971000000000003e-05 2024-09-21T21:42:00.666 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:00.667 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:00.667 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:00.667 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.667 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.667 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3451 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 3104768~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:42:00.667 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.207509+0000", 2024-09-21T21:42:00.667 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 13.382845958000001, 2024-09-21T21:42:00.667 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055698944, 2024-09-21T21:42:00.667 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:00.667 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:00.667 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:00.667 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:00.667 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:00.668 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:00.668 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3451 2024-09-21T21:42:00.668 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.668 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:00.668 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.668 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:00.668 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207509+0000", 2024-09-21T21:42:00.668 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:00.668 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.668 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.668 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:00.668 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207509+0000", 2024-09-21T21:42:00.668 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:00.668 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.669 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.669 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:00.669 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207661+0000", 2024-09-21T21:42:00.669 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00015181799999999999 2024-09-21T21:42:00.669 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.669 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.669 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:00.669 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207665+0000", 2024-09-21T21:42:00.669 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6380000000000002e-06 2024-09-21T21:42:00.669 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.669 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.669 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:00.669 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207674+0000", 2024-09-21T21:42:00.670 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.0950000000000002e-06 2024-09-21T21:42:00.670 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.670 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.670 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:00.670 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207689+0000", 2024-09-21T21:42:00.670 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.4904e-05 2024-09-21T21:42:00.670 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.670 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.670 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:00.670 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.263055+0000", 2024-09-21T21:42:00.670 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055366268000000003 2024-09-21T21:42:00.670 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.670 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.671 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:00.671 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.263101+0000", 2024-09-21T21:42:00.671 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6498000000000001e-05 2024-09-21T21:42:00.671 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.671 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.671 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:00.671 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.263208+0000", 2024-09-21T21:42:00.671 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000106723 2024-09-21T21:42:00.671 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:00.671 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:00.671 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:00.671 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.671 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.671 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3452 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 3133440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:42:00.671 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.207699+0000", 2024-09-21T21:42:00.672 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 13.382656529, 2024-09-21T21:42:00.672 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055702591000000003, 2024-09-21T21:42:00.672 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:00.672 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:00.672 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:00.672 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:00.672 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:00.672 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:00.672 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3452 2024-09-21T21:42:00.672 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.672 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:00.672 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.672 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:00.672 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207699+0000", 2024-09-21T21:42:00.673 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:00.673 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.673 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.673 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:00.673 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207699+0000", 2024-09-21T21:42:00.673 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:00.673 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.673 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.673 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:00.673 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207813+0000", 2024-09-21T21:42:00.673 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000114355 2024-09-21T21:42:00.673 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.673 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.674 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:00.674 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207826+0000", 2024-09-21T21:42:00.674 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.3293e-05 2024-09-21T21:42:00.674 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.674 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.674 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:00.674 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207835+0000", 2024-09-21T21:42:00.674 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.8310000000000007e-06 2024-09-21T21:42:00.674 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.674 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.674 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:00.675 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207842+0000", 2024-09-21T21:42:00.675 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5150000000000003e-06 2024-09-21T21:42:00.675 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.675 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.675 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:00.675 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.263240+0000", 2024-09-21T21:42:00.675 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055398007999999999 2024-09-21T21:42:00.675 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.675 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.675 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:00.675 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.263315+0000", 2024-09-21T21:42:00.675 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5729999999999995e-05 2024-09-21T21:42:00.675 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.676 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.676 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:00.676 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.263401+0000", 2024-09-21T21:42:00.676 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.5859000000000006e-05 2024-09-21T21:42:00.676 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:00.676 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:00.676 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:00.676 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.676 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.676 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3455 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 3821568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:42:00.676 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.207852+0000", 2024-09-21T21:42:00.676 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 13.382503655000001, 2024-09-21T21:42:00.676 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055704138, 2024-09-21T21:42:00.676 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:00.677 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:00.677 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:00.677 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:00.677 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:00.677 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:00.677 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3455 2024-09-21T21:42:00.677 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.677 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:00.677 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.677 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:00.677 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207852+0000", 2024-09-21T21:42:00.677 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:00.677 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.677 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.678 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:00.678 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207852+0000", 2024-09-21T21:42:00.678 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:00.678 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.678 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.678 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:00.678 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207983+0000", 2024-09-21T21:42:00.678 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00013183199999999999 2024-09-21T21:42:00.678 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.678 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.678 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:00.678 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207988+0000", 2024-09-21T21:42:00.678 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4290000000000003e-06 2024-09-21T21:42:00.678 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.679 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.679 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:00.679 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.208002+0000", 2024-09-21T21:42:00.679 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.4156e-05 2024-09-21T21:42:00.679 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.679 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.679 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:00.679 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.208009+0000", 2024-09-21T21:42:00.679 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.8220000000000001e-06 2024-09-21T21:42:00.679 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.679 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.679 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:00.679 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.263444+0000", 2024-09-21T21:42:00.679 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055435607999999997 2024-09-21T21:42:00.679 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.680 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.680 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:00.680 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.263493+0000", 2024-09-21T21:42:00.680 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8986999999999997e-05 2024-09-21T21:42:00.680 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.680 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.680 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:00.680 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.263556+0000", 2024-09-21T21:42:00.680 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2304000000000005e-05 2024-09-21T21:42:00.680 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:00.680 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:00.680 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:00.680 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.681 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.681 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3456 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 3850240~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:42:00.681 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.208025+0000", 2024-09-21T21:42:00.681 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 13.382330528000001, 2024-09-21T21:42:00.681 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055714048000000002, 2024-09-21T21:42:00.681 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:00.681 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:00.681 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:00.681 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:00.681 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:00.681 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:00.681 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3456 2024-09-21T21:42:00.681 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.681 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:00.682 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.682 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:00.682 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.208025+0000", 2024-09-21T21:42:00.682 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:00.682 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.682 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.682 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:00.682 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.208025+0000", 2024-09-21T21:42:00.682 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:00.682 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.683 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.683 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:00.683 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.208264+0000", 2024-09-21T21:42:00.683 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00023975999999999999 2024-09-21T21:42:00.683 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.683 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.683 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:00.683 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.208269+0000", 2024-09-21T21:42:00.683 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7790000000000003e-06 2024-09-21T21:42:00.683 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.683 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.683 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:00.684 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.208278+0000", 2024-09-21T21:42:00.684 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2110000000000004e-06 2024-09-21T21:42:00.684 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.684 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.684 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:00.684 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.208286+0000", 2024-09-21T21:42:00.684 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0899999999999999e-06 2024-09-21T21:42:00.684 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.684 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.684 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:00.684 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.263583+0000", 2024-09-21T21:42:00.684 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055297330999999998 2024-09-21T21:42:00.684 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.685 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.685 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:00.685 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.263647+0000", 2024-09-21T21:42:00.685 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4654999999999997e-05 2024-09-21T21:42:00.685 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.685 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.685 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:00.685 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.263739+0000", 2024-09-21T21:42:00.685 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.1222000000000002e-05 2024-09-21T21:42:00.685 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:00.685 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:00.685 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:00.685 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.686 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.686 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3468 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 266240~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:42:00.686 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.208298+0000", 2024-09-21T21:42:00.686 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 13.382057367, 2024-09-21T21:42:00.686 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055668849999999999, 2024-09-21T21:42:00.686 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:00.686 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:00.686 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:00.686 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:00.686 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:00.686 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:00.687 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3468 2024-09-21T21:42:00.687 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.687 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:00.687 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.687 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:00.687 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.208298+0000", 2024-09-21T21:42:00.687 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:00.687 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.687 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.687 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:00.687 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.208298+0000", 2024-09-21T21:42:00.688 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:00.688 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.688 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.688 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:00.688 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.208465+0000", 2024-09-21T21:42:00.688 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00016740700000000001 2024-09-21T21:42:00.688 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.688 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.688 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:00.688 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.208484+0000", 2024-09-21T21:42:00.688 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.8963000000000001e-05 2024-09-21T21:42:00.688 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.688 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.689 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:00.689 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.208500+0000", 2024-09-21T21:42:00.689 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.6013999999999999e-05 2024-09-21T21:42:00.689 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.689 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.689 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:00.689 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.208508+0000", 2024-09-21T21:42:00.689 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.4279999999999999e-06 2024-09-21T21:42:00.689 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.689 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.689 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:00.690 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.263780+0000", 2024-09-21T21:42:00.690 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.05527232 2024-09-21T21:42:00.690 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.690 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.690 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:00.690 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.263863+0000", 2024-09-21T21:42:00.690 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.2723000000000001e-05 2024-09-21T21:42:00.690 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.690 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.690 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:00.690 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.263967+0000", 2024-09-21T21:42:00.691 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000103995 2024-09-21T21:42:00.691 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:00.691 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:00.691 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:00.691 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.691 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.691 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3470 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 294912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:42:00.691 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.208525+0000", 2024-09-21T21:42:00.691 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 13.381829747999999, 2024-09-21T21:42:00.691 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055651946000000001, 2024-09-21T21:42:00.691 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:00.691 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:00.692 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:00.692 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:00.692 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:00.692 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:00.692 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3470 2024-09-21T21:42:00.692 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.692 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:00.692 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.692 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:00.692 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.208525+0000", 2024-09-21T21:42:00.692 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:00.692 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.692 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.692 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:00.693 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.208525+0000", 2024-09-21T21:42:00.693 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:00.693 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.693 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.693 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:00.693 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.209091+0000", 2024-09-21T21:42:00.693 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00056559299999999998 2024-09-21T21:42:00.693 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.693 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.693 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:00.693 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.209096+0000", 2024-09-21T21:42:00.693 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.882e-06 2024-09-21T21:42:00.693 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.694 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.694 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:00.694 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.209106+0000", 2024-09-21T21:42:00.694 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.8900000000000002e-06 2024-09-21T21:42:00.694 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.694 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.694 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:00.694 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.209114+0000", 2024-09-21T21:42:00.694 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.6580000000000008e-06 2024-09-21T21:42:00.694 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.694 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.694 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:00.695 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.264030+0000", 2024-09-21T21:42:00.695 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.054915692000000002 2024-09-21T21:42:00.695 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.695 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.695 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:00.695 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.264107+0000", 2024-09-21T21:42:00.695 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.6860000000000006e-05 2024-09-21T21:42:00.695 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:00.695 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:00.695 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:00.695 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.264177+0000", 2024-09-21T21:42:00.695 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0370999999999994e-05 2024-09-21T21:42:00.696 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:00.696 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:00.696 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:00.696 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:00.696 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:00.696 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:00.697 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T21:42:00.698 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:00.696+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:00.798 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:00.795+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:00.802 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:00.802 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:42:00.802 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 0 2024-09-21T21:42:00.802 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:00.811 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T21:42:00.899 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:00.898+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:00.952 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:00.953 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:42:00.953 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:42:00.953 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:42:00.953 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:00.961 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T21:42:01.000 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:00.999+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:01.064 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:01.065 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 20, 2024-09-21T21:42:01.065 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 600, 2024-09-21T21:42:01.065 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:42:01.065 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.065 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13295 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3330048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:01.065 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.796526+0000", 2024-09-21T21:42:01.065 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 10.265568785999999, 2024-09-21T21:42:01.065 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071630928999999996, 2024-09-21T21:42:01.065 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:01.065 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:01.065 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:01.066 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:01.066 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:01.066 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:01.066 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13295 2024-09-21T21:42:01.066 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.066 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:01.066 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.066 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:01.066 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796526+0000", 2024-09-21T21:42:01.066 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:01.066 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.066 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.066 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:01.067 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796526+0000", 2024-09-21T21:42:01.067 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:01.067 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.067 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.067 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:01.067 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796526+0000", 2024-09-21T21:42:01.067 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5900000000000003e-07 2024-09-21T21:42:01.067 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.067 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.067 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:01.067 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796529+0000", 2024-09-21T21:42:01.067 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3910000000000001e-06 2024-09-21T21:42:01.067 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.068 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.068 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:01.068 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796537+0000", 2024-09-21T21:42:01.068 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.2819999999999996e-06 2024-09-21T21:42:01.068 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.068 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.068 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:01.068 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796541+0000", 2024-09-21T21:42:01.068 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.9400000000000004e-06 2024-09-21T21:42:01.068 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.068 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.068 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:01.068 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.868008+0000", 2024-09-21T21:42:01.069 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071467586999999999 2024-09-21T21:42:01.069 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.069 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.069 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:01.069 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.868061+0000", 2024-09-21T21:42:01.069 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2349000000000003e-05 2024-09-21T21:42:01.069 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.069 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.069 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:01.069 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.868157+0000", 2024-09-21T21:42:01.069 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6020999999999994e-05 2024-09-21T21:42:01.070 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:01.070 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:01.070 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:01.070 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.070 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.070 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13296 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 4046848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:01.070 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.796643+0000", 2024-09-21T21:42:01.070 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 10.265451967000001, 2024-09-21T21:42:01.070 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071682019999999999, 2024-09-21T21:42:01.070 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:01.070 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:01.070 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:01.070 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:01.071 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:01.071 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:01.071 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13296 2024-09-21T21:42:01.071 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.071 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:01.071 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.071 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:01.071 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796643+0000", 2024-09-21T21:42:01.071 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:01.071 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.071 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.071 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:01.071 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796643+0000", 2024-09-21T21:42:01.072 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:01.072 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.072 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.072 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:01.072 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796643+0000", 2024-09-21T21:42:01.072 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.7199999999999998e-07 2024-09-21T21:42:01.072 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.072 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.072 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:01.072 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796647+0000", 2024-09-21T21:42:01.072 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6770000000000001e-06 2024-09-21T21:42:01.072 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.073 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.073 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:01.073 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796656+0000", 2024-09-21T21:42:01.073 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2059999999999996e-06 2024-09-21T21:42:01.073 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.073 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.073 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:01.073 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796664+0000", 2024-09-21T21:42:01.073 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7919999999999993e-06 2024-09-21T21:42:01.073 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.073 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.073 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:01.073 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.868186+0000", 2024-09-21T21:42:01.074 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071522273999999997 2024-09-21T21:42:01.074 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.074 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.074 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:01.074 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.868239+0000", 2024-09-21T21:42:01.074 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2787999999999997e-05 2024-09-21T21:42:01.074 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.074 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.074 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:01.074 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.868325+0000", 2024-09-21T21:42:01.074 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.5611000000000004e-05 2024-09-21T21:42:01.074 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:01.075 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:01.075 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:01.075 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.075 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.075 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13561 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 1728512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:01.075 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.823991+0000", 2024-09-21T21:42:01.075 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 10.238103666000001, 2024-09-21T21:42:01.075 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071654762999999996, 2024-09-21T21:42:01.075 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:01.075 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:01.075 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:01.076 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:01.076 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:01.076 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:01.076 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13561 2024-09-21T21:42:01.076 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.076 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:01.076 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.076 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:01.076 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.823991+0000", 2024-09-21T21:42:01.076 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:01.077 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.077 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.077 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:01.077 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.823991+0000", 2024-09-21T21:42:01.077 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:01.077 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.077 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.077 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:01.077 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.823991+0000", 2024-09-21T21:42:01.077 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9299999999999998e-07 2024-09-21T21:42:01.077 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.077 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.078 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:01.078 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.823994+0000", 2024-09-21T21:42:01.078 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2369999999999999e-06 2024-09-21T21:42:01.078 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.078 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.078 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:01.078 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824000+0000", 2024-09-21T21:42:01.078 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.55e-06 2024-09-21T21:42:01.078 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.078 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.078 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:01.078 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824005+0000", 2024-09-21T21:42:01.079 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4329999999999997e-06 2024-09-21T21:42:01.079 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.079 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.079 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:01.079 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.895360+0000", 2024-09-21T21:42:01.079 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071355426 2024-09-21T21:42:01.079 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.079 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.079 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:01.079 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.895477+0000", 2024-09-21T21:42:01.079 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0001171 2024-09-21T21:42:01.079 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.080 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.080 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:01.080 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.895646+0000", 2024-09-21T21:42:01.080 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000168524 2024-09-21T21:42:01.080 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:01.080 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:01.080 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:01.080 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.080 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.080 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13562 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 1757184~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:01.080 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.824011+0000", 2024-09-21T21:42:01.080 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 10.238083788999999, 2024-09-21T21:42:01.081 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072020453999999998, 2024-09-21T21:42:01.081 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:01.081 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:01.081 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:01.081 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:01.081 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:01.081 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:01.081 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13562 2024-09-21T21:42:01.081 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.081 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:01.081 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.082 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:01.082 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824011+0000", 2024-09-21T21:42:01.082 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:01.082 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.082 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.082 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:01.082 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824011+0000", 2024-09-21T21:42:01.082 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:01.082 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.082 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.082 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:01.083 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824011+0000", 2024-09-21T21:42:01.083 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5600000000000001e-07 2024-09-21T21:42:01.083 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.083 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.083 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:01.083 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824013+0000", 2024-09-21T21:42:01.083 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9800000000000001e-06 2024-09-21T21:42:01.083 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.083 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.083 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:01.083 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824019+0000", 2024-09-21T21:42:01.083 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7339999999999996e-06 2024-09-21T21:42:01.084 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.084 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.084 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:01.084 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824023+0000", 2024-09-21T21:42:01.084 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3680000000000004e-06 2024-09-21T21:42:01.084 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.084 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.084 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:01.084 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.895699+0000", 2024-09-21T21:42:01.084 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071675597999999993 2024-09-21T21:42:01.084 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.085 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.085 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:01.085 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.895830+0000", 2024-09-21T21:42:01.085 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000130858 2024-09-21T21:42:01.085 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.085 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.085 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:01.085 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896031+0000", 2024-09-21T21:42:01.085 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00020155999999999999 2024-09-21T21:42:01.085 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:01.085 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:01.085 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:01.086 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.086 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.086 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13563 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 323584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:01.086 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.824029+0000", 2024-09-21T21:42:01.086 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 10.238065348999999, 2024-09-21T21:42:01.086 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072307563000000005, 2024-09-21T21:42:01.086 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:01.086 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:01.086 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:01.086 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:01.086 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:01.086 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:01.087 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13563 2024-09-21T21:42:01.087 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.087 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:01.087 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.087 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:01.087 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824029+0000", 2024-09-21T21:42:01.087 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:01.087 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.087 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.087 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:01.087 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824029+0000", 2024-09-21T21:42:01.088 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:01.088 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.088 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.088 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:01.088 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824030+0000", 2024-09-21T21:42:01.088 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1699999999999999e-07 2024-09-21T21:42:01.088 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.088 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.088 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:01.088 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824032+0000", 2024-09-21T21:42:01.088 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0320000000000002e-06 2024-09-21T21:42:01.089 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.089 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.089 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:01.089 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824038+0000", 2024-09-21T21:42:01.089 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8769999999999998e-06 2024-09-21T21:42:01.089 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.089 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.089 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:01.089 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824042+0000", 2024-09-21T21:42:01.089 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5650000000000003e-06 2024-09-21T21:42:01.089 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.090 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.090 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:01.090 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896058+0000", 2024-09-21T21:42:01.090 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072015941 2024-09-21T21:42:01.090 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.090 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.090 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:01.090 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896164+0000", 2024-09-21T21:42:01.090 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010593499999999999 2024-09-21T21:42:01.090 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.090 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.091 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:01.091 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896337+0000", 2024-09-21T21:42:01.091 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00017279599999999999 2024-09-21T21:42:01.091 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:01.091 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:01.091 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:01.091 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.091 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.091 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13564 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 983040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:01.091 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.824048+0000", 2024-09-21T21:42:01.091 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 10.238046274, 2024-09-21T21:42:01.092 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072491193999999995, 2024-09-21T21:42:01.092 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:01.092 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:01.092 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:01.092 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:01.092 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:01.092 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:01.092 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13564 2024-09-21T21:42:01.092 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.092 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:01.092 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.092 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:01.093 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824048+0000", 2024-09-21T21:42:01.093 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:01.093 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.093 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.093 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:01.093 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824048+0000", 2024-09-21T21:42:01.093 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:01.093 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.093 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.093 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:01.093 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824663+0000", 2024-09-21T21:42:01.093 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00061494299999999996 2024-09-21T21:42:01.094 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.094 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.094 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:01.094 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824669+0000", 2024-09-21T21:42:01.094 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2900000000000002e-06 2024-09-21T21:42:01.094 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.094 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.094 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:01.094 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824679+0000", 2024-09-21T21:42:01.094 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0338e-05 2024-09-21T21:42:01.094 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.094 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.095 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:01.095 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824687+0000", 2024-09-21T21:42:01.095 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.9049999999999997e-06 2024-09-21T21:42:01.095 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.095 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.095 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:01.095 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896381+0000", 2024-09-21T21:42:01.095 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071694305999999999 2024-09-21T21:42:01.095 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.095 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.095 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:01.095 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896445+0000", 2024-09-21T21:42:01.095 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3869999999999997e-05 2024-09-21T21:42:01.096 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.096 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.096 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:01.096 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896540+0000", 2024-09-21T21:42:01.096 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.4542000000000004e-05 2024-09-21T21:42:01.096 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:01.096 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:01.096 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:01.096 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.096 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.096 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13565 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3190784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:01.096 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.824704+0000", 2024-09-21T21:42:01.097 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 10.237390402999999, 2024-09-21T21:42:01.097 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072014745000000005, 2024-09-21T21:42:01.097 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:01.097 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:01.097 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:01.097 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:01.097 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:01.097 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:01.097 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13565 2024-09-21T21:42:01.097 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.097 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:01.097 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.098 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:01.098 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824704+0000", 2024-09-21T21:42:01.098 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:01.098 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.098 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.098 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:01.098 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824704+0000", 2024-09-21T21:42:01.098 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:01.098 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.098 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.098 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:01.099 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824705+0000", 2024-09-21T21:42:01.099 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8999999999999997e-07 2024-09-21T21:42:01.099 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.099 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.099 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:01.099 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824707+0000", 2024-09-21T21:42:01.099 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.7669999999999999e-06 2024-09-21T21:42:01.099 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.099 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.099 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:01.099 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824715+0000", 2024-09-21T21:42:01.099 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7700000000000001e-06 2024-09-21T21:42:01.100 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.100 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.100 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:01.100 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824721+0000", 2024-09-21T21:42:01.100 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9560000000000002e-06 2024-09-21T21:42:01.100 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.100 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.100 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:01.100 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896570+0000", 2024-09-21T21:42:01.100 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071848870999999995 2024-09-21T21:42:01.100 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.100 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.100 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:01.101 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896626+0000", 2024-09-21T21:42:01.101 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5578000000000002e-05 2024-09-21T21:42:01.101 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.101 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.101 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:01.101 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896719+0000", 2024-09-21T21:42:01.101 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.3313000000000006e-05 2024-09-21T21:42:01.101 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:01.101 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:01.101 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:01.101 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.101 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.101 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13566 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3162112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:01.101 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.824729+0000", 2024-09-21T21:42:01.102 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 10.237365516000001, 2024-09-21T21:42:01.102 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072356907999999998, 2024-09-21T21:42:01.102 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:01.102 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:01.102 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:01.102 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:01.102 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:01.102 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:01.102 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13566 2024-09-21T21:42:01.102 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.102 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:01.102 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.102 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:01.103 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824729+0000", 2024-09-21T21:42:01.103 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:01.103 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.103 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.103 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:01.103 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824729+0000", 2024-09-21T21:42:01.103 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:01.103 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.103 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.103 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:01.103 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825630+0000", 2024-09-21T21:42:01.103 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00090139400000000002 2024-09-21T21:42:01.104 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.104 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.104 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:01.104 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825634+0000", 2024-09-21T21:42:01.104 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4960000000000001e-06 2024-09-21T21:42:01.104 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.104 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.104 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:01.104 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825644+0000", 2024-09-21T21:42:01.104 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.9650000000000003e-06 2024-09-21T21:42:01.104 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.104 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.105 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:01.105 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825649+0000", 2024-09-21T21:42:01.105 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0359999999999997e-06 2024-09-21T21:42:01.105 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.105 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.105 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:01.105 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896757+0000", 2024-09-21T21:42:01.105 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071107717000000001 2024-09-21T21:42:01.105 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.105 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.105 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:01.106 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896878+0000", 2024-09-21T21:42:01.106 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000120834 2024-09-21T21:42:01.106 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.106 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.106 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:01.106 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897086+0000", 2024-09-21T21:42:01.106 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00020846600000000001 2024-09-21T21:42:01.106 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:01.106 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:01.106 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:01.106 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.106 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.107 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13567 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3878912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:01.107 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.825658+0000", 2024-09-21T21:42:01.107 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 10.236436517, 2024-09-21T21:42:01.107 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071597694000000003, 2024-09-21T21:42:01.107 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:01.107 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:01.107 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:01.107 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:01.107 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:01.107 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:01.107 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13567 2024-09-21T21:42:01.108 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.108 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:01.108 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.108 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:01.108 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825658+0000", 2024-09-21T21:42:01.108 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:01.108 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.108 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.108 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:01.108 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825658+0000", 2024-09-21T21:42:01.109 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:01.109 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.109 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.109 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:01.109 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825658+0000", 2024-09-21T21:42:01.109 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.5699999999999999e-07 2024-09-21T21:42:01.109 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.109 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.109 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:01.109 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825660+0000", 2024-09-21T21:42:01.109 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.638e-06 2024-09-21T21:42:01.109 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.110 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.110 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:01.110 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825664+0000", 2024-09-21T21:42:01.110 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4340000000000002e-06 2024-09-21T21:42:01.110 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.110 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.110 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:01.110 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825668+0000", 2024-09-21T21:42:01.110 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.1669999999999999e-06 2024-09-21T21:42:01.110 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.111 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.111 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:01.111 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897113+0000", 2024-09-21T21:42:01.111 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071445358 2024-09-21T21:42:01.111 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.111 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.111 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:01.111 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897169+0000", 2024-09-21T21:42:01.111 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5813000000000002e-05 2024-09-21T21:42:01.111 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.111 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.111 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:01.112 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897256+0000", 2024-09-21T21:42:01.112 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.7027000000000006e-05 2024-09-21T21:42:01.112 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:01.112 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:01.112 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:01.112 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.112 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.112 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13568 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3907584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:01.112 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.825671+0000", 2024-09-21T21:42:01.112 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 10.236423107, 2024-09-21T21:42:01.112 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071794595000000003, 2024-09-21T21:42:01.112 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:01.113 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:01.113 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:01.113 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:01.113 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:01.113 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:01.113 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13568 2024-09-21T21:42:01.113 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.113 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:01.113 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.113 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:01.113 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825671+0000", 2024-09-21T21:42:01.114 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:01.114 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.114 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.114 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:01.114 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825671+0000", 2024-09-21T21:42:01.114 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:01.114 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.114 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.114 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:01.114 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825672+0000", 2024-09-21T21:42:01.115 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2999999999999999e-07 2024-09-21T21:42:01.115 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.115 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.115 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:01.115 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825673+0000", 2024-09-21T21:42:01.115 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.246e-06 2024-09-21T21:42:01.115 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.115 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.115 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:01.115 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825693+0000", 2024-09-21T21:42:01.115 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9789999999999999e-05 2024-09-21T21:42:01.115 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.116 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.116 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:01.116 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825697+0000", 2024-09-21T21:42:01.116 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2379999999999999e-06 2024-09-21T21:42:01.116 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.116 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.116 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:01.116 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897284+0000", 2024-09-21T21:42:01.116 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071586710999999997 2024-09-21T21:42:01.116 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.116 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.116 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:01.117 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897337+0000", 2024-09-21T21:42:01.117 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2800000000000003e-05 2024-09-21T21:42:01.117 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.117 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.117 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:01.117 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897466+0000", 2024-09-21T21:42:01.117 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00012957999999999999 2024-09-21T21:42:01.117 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:01.117 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:01.117 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:01.117 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.118 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.118 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13574 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 352256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:01.118 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.825702+0000", 2024-09-21T21:42:01.118 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 10.236392959, 2024-09-21T21:42:01.118 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072080377000000001, 2024-09-21T21:42:01.118 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:01.118 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:01.118 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:01.118 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:01.118 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:01.118 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:01.119 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13574 2024-09-21T21:42:01.119 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.119 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:01.119 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.119 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:01.119 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825702+0000", 2024-09-21T21:42:01.119 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:01.119 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.119 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.119 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:01.119 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825702+0000", 2024-09-21T21:42:01.119 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:01.120 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.120 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.120 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:01.120 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825943+0000", 2024-09-21T21:42:01.120 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00024105200000000001 2024-09-21T21:42:01.120 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.120 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.120 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:01.120 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825946+0000", 2024-09-21T21:42:01.120 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.743e-06 2024-09-21T21:42:01.120 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.121 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.121 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:01.121 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825956+0000", 2024-09-21T21:42:01.121 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6609999999999995e-06 2024-09-21T21:42:01.121 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.121 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.121 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:01.121 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825963+0000", 2024-09-21T21:42:01.121 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4609999999999996e-06 2024-09-21T21:42:01.121 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.121 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.121 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:01.122 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897502+0000", 2024-09-21T21:42:01.122 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071539354999999999 2024-09-21T21:42:01.122 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.122 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.122 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:01.122 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897619+0000", 2024-09-21T21:42:01.122 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011665 2024-09-21T21:42:01.122 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.122 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.122 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:01.122 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897782+0000", 2024-09-21T21:42:01.123 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00016345500000000001 2024-09-21T21:42:01.123 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:01.123 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:01.123 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:01.123 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.123 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.123 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13575 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 380928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:01.123 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.825973+0000", 2024-09-21T21:42:01.123 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 10.236121523, 2024-09-21T21:42:01.123 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072012723000000001, 2024-09-21T21:42:01.123 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:01.124 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:01.124 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:01.124 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:01.124 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:01.124 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:01.124 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13575 2024-09-21T21:42:01.124 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.124 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:01.124 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.124 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:01.124 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825973+0000", 2024-09-21T21:42:01.124 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:01.125 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.125 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.125 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:01.125 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825973+0000", 2024-09-21T21:42:01.125 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:01.125 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.125 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.125 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:01.125 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826746+0000", 2024-09-21T21:42:01.125 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00077326800000000002 2024-09-21T21:42:01.125 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.126 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.126 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:01.126 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826766+0000", 2024-09-21T21:42:01.126 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9465000000000001e-05 2024-09-21T21:42:01.126 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.126 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.126 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:01.126 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826812+0000", 2024-09-21T21:42:01.126 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5927999999999998e-05 2024-09-21T21:42:01.126 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.126 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.127 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:01.127 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826833+0000", 2024-09-21T21:42:01.127 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1710999999999999e-05 2024-09-21T21:42:01.127 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.127 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.127 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:01.127 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897809+0000", 2024-09-21T21:42:01.127 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.070975229000000001 2024-09-21T21:42:01.127 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.127 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.127 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:01.127 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897864+0000", 2024-09-21T21:42:01.127 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5334999999999998e-05 2024-09-21T21:42:01.128 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.128 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.128 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:01.128 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897986+0000", 2024-09-21T21:42:01.128 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000121787 2024-09-21T21:42:01.128 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:01.128 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:01.128 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:01.128 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.128 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.128 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13577 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 1814528~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:01.129 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.826874+0000", 2024-09-21T21:42:01.129 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 10.235220489, 2024-09-21T21:42:01.129 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071639123999999998, 2024-09-21T21:42:01.129 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:01.129 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:01.129 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:01.129 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:01.129 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:01.129 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:01.129 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13577 2024-09-21T21:42:01.129 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.129 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:01.130 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.130 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:01.130 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826874+0000", 2024-09-21T21:42:01.130 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:01.130 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.130 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.130 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:01.130 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826874+0000", 2024-09-21T21:42:01.130 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:01.130 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.130 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.131 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:01.131 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826875+0000", 2024-09-21T21:42:01.131 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.82e-07 2024-09-21T21:42:01.131 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.131 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.131 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:01.131 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826877+0000", 2024-09-21T21:42:01.131 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.295e-06 2024-09-21T21:42:01.131 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.131 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.131 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:01.131 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826883+0000", 2024-09-21T21:42:01.132 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2609999999999999e-06 2024-09-21T21:42:01.132 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.132 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.132 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:01.132 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826889+0000", 2024-09-21T21:42:01.132 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5589999999999996e-06 2024-09-21T21:42:01.132 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.132 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.132 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:01.132 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898301+0000", 2024-09-21T21:42:01.132 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071412188000000001 2024-09-21T21:42:01.133 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.133 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.133 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:01.133 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898416+0000", 2024-09-21T21:42:01.133 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000115538 2024-09-21T21:42:01.133 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.133 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.133 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:01.133 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898513+0000", 2024-09-21T21:42:01.133 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6800999999999997e-05 2024-09-21T21:42:01.133 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:01.133 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:01.133 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:01.134 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.134 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.134 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13578 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 2502656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:01.134 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.826896+0000", 2024-09-21T21:42:01.134 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 10.235199033000001, 2024-09-21T21:42:01.134 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071789411999999997, 2024-09-21T21:42:01.134 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:01.134 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:01.134 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:01.134 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:01.134 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:01.134 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:01.135 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13578 2024-09-21T21:42:01.135 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.135 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:01.135 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.135 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:01.135 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826896+0000", 2024-09-21T21:42:01.135 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:01.135 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.135 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.135 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:01.135 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826896+0000", 2024-09-21T21:42:01.135 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:01.136 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.136 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.136 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:01.136 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827327+0000", 2024-09-21T21:42:01.136 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00043094500000000002 2024-09-21T21:42:01.136 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.136 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.136 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:01.136 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827331+0000", 2024-09-21T21:42:01.136 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.0969999999999999e-06 2024-09-21T21:42:01.136 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.137 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.137 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:01.137 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827341+0000", 2024-09-21T21:42:01.137 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0199e-05 2024-09-21T21:42:01.137 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.137 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.137 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:01.137 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827346+0000", 2024-09-21T21:42:01.137 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3210000000000004e-06 2024-09-21T21:42:01.137 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.137 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.137 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:01.137 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898540+0000", 2024-09-21T21:42:01.138 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071194246000000003 2024-09-21T21:42:01.138 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.138 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.138 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:01.138 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898598+0000", 2024-09-21T21:42:01.138 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7658999999999998e-05 2024-09-21T21:42:01.138 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.138 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.138 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:01.138 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898685+0000", 2024-09-21T21:42:01.138 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.6945000000000002e-05 2024-09-21T21:42:01.139 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:01.139 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:01.139 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:01.139 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.139 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.139 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13579 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 2531328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:01.139 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.827355+0000", 2024-09-21T21:42:01.139 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 10.234739502, 2024-09-21T21:42:01.139 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071650933, 2024-09-21T21:42:01.139 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:01.139 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:01.139 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:01.140 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:01.140 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:01.140 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:01.140 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13579 2024-09-21T21:42:01.140 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.140 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:01.140 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.140 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:01.140 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827355+0000", 2024-09-21T21:42:01.140 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:01.140 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.141 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.141 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:01.141 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827355+0000", 2024-09-21T21:42:01.141 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:01.141 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.141 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.141 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:01.141 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827486+0000", 2024-09-21T21:42:01.141 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00013066799999999999 2024-09-21T21:42:01.141 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.141 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.141 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:01.142 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827490+0000", 2024-09-21T21:42:01.142 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.9009999999999996e-06 2024-09-21T21:42:01.142 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.142 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.142 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:01.142 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827499+0000", 2024-09-21T21:42:01.142 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6209999999999999e-06 2024-09-21T21:42:01.142 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.142 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.142 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:01.142 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827506+0000", 2024-09-21T21:42:01.142 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6490000000000002e-06 2024-09-21T21:42:01.143 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.143 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.143 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:01.143 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898713+0000", 2024-09-21T21:42:01.143 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071206937999999997 2024-09-21T21:42:01.143 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.143 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.143 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:01.143 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898811+0000", 2024-09-21T21:42:01.143 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.8159999999999995e-05 2024-09-21T21:42:01.143 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.143 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.143 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:01.143 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899006+0000", 2024-09-21T21:42:01.144 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00019499599999999999 2024-09-21T21:42:01.144 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:01.144 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:01.144 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:01.144 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.144 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.144 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13580 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3219456~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:01.144 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.827522+0000", 2024-09-21T21:42:01.144 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 10.234573071, 2024-09-21T21:42:01.144 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071813298999999997, 2024-09-21T21:42:01.144 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:01.144 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:01.144 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:01.144 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:01.145 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:01.145 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:01.145 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13580 2024-09-21T21:42:01.145 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.145 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:01.145 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.145 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:01.145 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827522+0000", 2024-09-21T21:42:01.145 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:01.145 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.145 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.145 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:01.145 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827522+0000", 2024-09-21T21:42:01.145 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:01.146 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.146 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.146 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:01.146 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827898+0000", 2024-09-21T21:42:01.146 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000376876 2024-09-21T21:42:01.146 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.146 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.146 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:01.146 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827903+0000", 2024-09-21T21:42:01.146 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4129999999999999e-06 2024-09-21T21:42:01.146 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.146 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.146 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:01.147 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827911+0000", 2024-09-21T21:42:01.147 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.4540000000000007e-06 2024-09-21T21:42:01.147 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.147 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.147 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:01.147 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827917+0000", 2024-09-21T21:42:01.147 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2110000000000003e-06 2024-09-21T21:42:01.147 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.147 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.147 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:01.147 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899041+0000", 2024-09-21T21:42:01.147 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071123965999999997 2024-09-21T21:42:01.148 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.148 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.148 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:01.148 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899157+0000", 2024-09-21T21:42:01.148 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011573700000000001 2024-09-21T21:42:01.148 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.148 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.148 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:01.148 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899335+0000", 2024-09-21T21:42:01.148 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00017764199999999999 2024-09-21T21:42:01.148 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:01.148 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:01.149 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:01.149 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.149 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.149 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13581 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3248128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:01.149 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.827929+0000", 2024-09-21T21:42:01.149 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 10.234165672, 2024-09-21T21:42:01.149 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071730151000000006, 2024-09-21T21:42:01.149 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:01.149 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:01.149 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:01.149 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:01.150 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:01.150 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:01.150 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13581 2024-09-21T21:42:01.150 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.150 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:01.150 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.150 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:01.150 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827929+0000", 2024-09-21T21:42:01.150 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:01.150 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.150 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.150 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:01.150 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827929+0000", 2024-09-21T21:42:01.151 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:01.151 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.151 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.151 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:01.151 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.829943+0000", 2024-09-21T21:42:01.151 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0020143750000000001 2024-09-21T21:42:01.151 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.151 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.151 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:01.151 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.829948+0000", 2024-09-21T21:42:01.151 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9400000000000001e-06 2024-09-21T21:42:01.151 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.152 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.152 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:01.152 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.829960+0000", 2024-09-21T21:42:01.152 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.2123999999999999e-05 2024-09-21T21:42:01.152 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.152 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.152 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:01.152 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.829968+0000", 2024-09-21T21:42:01.152 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7000000000000008e-06 2024-09-21T21:42:01.152 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.152 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.152 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:01.152 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899370+0000", 2024-09-21T21:42:01.152 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.069401672999999997 2024-09-21T21:42:01.153 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.153 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.153 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:01.153 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899481+0000", 2024-09-21T21:42:01.153 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011145300000000001 2024-09-21T21:42:01.153 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.153 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.153 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:01.153 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899659+0000", 2024-09-21T21:42:01.153 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00017788600000000001 2024-09-21T21:42:01.153 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:01.153 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:01.154 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:01.154 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.154 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.154 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13595 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 1871872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:01.154 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.830180+0000", 2024-09-21T21:42:01.154 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 10.231914558, 2024-09-21T21:42:01.154 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071809244999999994, 2024-09-21T21:42:01.154 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:01.154 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:01.154 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:01.154 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:01.154 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:01.155 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:01.155 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13595 2024-09-21T21:42:01.155 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.155 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:01.155 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.155 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:01.155 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.830180+0000", 2024-09-21T21:42:01.155 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:01.155 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.155 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.155 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:01.155 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.830180+0000", 2024-09-21T21:42:01.156 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:01.156 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.156 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.156 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:01.156 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.831967+0000", 2024-09-21T21:42:01.156 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0017868350000000001 2024-09-21T21:42:01.156 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.156 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.156 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:01.156 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.831971+0000", 2024-09-21T21:42:01.156 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3950000000000003e-06 2024-09-21T21:42:01.156 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.157 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.157 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:01.157 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.831983+0000", 2024-09-21T21:42:01.157 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1815e-05 2024-09-21T21:42:01.157 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.157 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.157 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:01.157 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.831993+0000", 2024-09-21T21:42:01.157 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6339999999999996e-06 2024-09-21T21:42:01.157 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.157 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.157 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:01.158 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.901693+0000", 2024-09-21T21:42:01.158 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.069700082999999996 2024-09-21T21:42:01.158 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.158 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.158 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:01.158 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.901813+0000", 2024-09-21T21:42:01.158 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011971 2024-09-21T21:42:01.158 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.158 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.158 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:01.158 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.901989+0000", 2024-09-21T21:42:01.159 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000176773 2024-09-21T21:42:01.159 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:01.159 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:01.159 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:01.159 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.159 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.159 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13600 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 2617344~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:01.159 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.832109+0000", 2024-09-21T21:42:01.159 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 10.229985805, 2024-09-21T21:42:01.159 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071551828999999997, 2024-09-21T21:42:01.159 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:01.159 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:01.159 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:01.160 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:01.160 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:01.160 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:01.160 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13600 2024-09-21T21:42:01.160 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.160 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:01.160 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.160 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:01.160 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.832109+0000", 2024-09-21T21:42:01.160 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:01.160 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.161 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.161 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:01.161 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.832109+0000", 2024-09-21T21:42:01.161 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:01.161 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.161 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.161 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:01.161 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.832109+0000", 2024-09-21T21:42:01.161 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4999999999999998e-07 2024-09-21T21:42:01.161 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.161 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.161 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:01.162 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.832112+0000", 2024-09-21T21:42:01.162 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.5019999999999999e-06 2024-09-21T21:42:01.162 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.162 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.162 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:01.162 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.832118+0000", 2024-09-21T21:42:01.162 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6270000000000002e-06 2024-09-21T21:42:01.162 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.162 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.162 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:01.162 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.832124+0000", 2024-09-21T21:42:01.163 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1850000000000003e-06 2024-09-21T21:42:01.163 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.163 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.163 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:01.163 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.903366+0000", 2024-09-21T21:42:01.163 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071241961000000006 2024-09-21T21:42:01.163 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.163 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.163 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:01.163 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.903485+0000", 2024-09-21T21:42:01.163 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000119202 2024-09-21T21:42:01.163 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.164 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.164 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:01.164 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.903661+0000", 2024-09-21T21:42:01.164 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000175902 2024-09-21T21:42:01.164 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:01.164 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:01.164 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:01.164 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.164 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.164 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13601 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3334144~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:01.164 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.832130+0000", 2024-09-21T21:42:01.164 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 10.229964686000001, 2024-09-21T21:42:01.165 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071861727, 2024-09-21T21:42:01.165 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:01.165 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:01.165 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:01.165 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:01.165 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:01.165 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:01.165 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13601 2024-09-21T21:42:01.165 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.165 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:01.165 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.165 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:01.166 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.832130+0000", 2024-09-21T21:42:01.166 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:01.166 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.166 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.166 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:01.166 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.832130+0000", 2024-09-21T21:42:01.166 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:01.166 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.166 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.166 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:01.166 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.833939+0000", 2024-09-21T21:42:01.166 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0018094400000000001 2024-09-21T21:42:01.167 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.167 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.167 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:01.167 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.833944+0000", 2024-09-21T21:42:01.167 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5979999999999999e-06 2024-09-21T21:42:01.167 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.167 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.167 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:01.167 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.833953+0000", 2024-09-21T21:42:01.167 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.1460000000000002e-06 2024-09-21T21:42:01.167 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.167 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.168 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:01.168 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.833960+0000", 2024-09-21T21:42:01.168 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.7989999999999996e-06 2024-09-21T21:42:01.168 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.168 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.168 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:01.168 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.903693+0000", 2024-09-21T21:42:01.168 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.069732901 2024-09-21T21:42:01.168 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.168 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.168 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:01.169 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.903814+0000", 2024-09-21T21:42:01.169 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000121585 2024-09-21T21:42:01.169 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:01.169 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:01.169 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:01.169 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.903992+0000", 2024-09-21T21:42:01.169 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00017725799999999999 2024-09-21T21:42:01.169 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:01.169 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:01.169 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:01.169 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:01.169 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:01.170 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:01.171 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T21:42:01.172 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:01.169+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:01.271 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:01.269+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:01.310 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:01.310 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:42:01.310 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-21T21:42:01.311 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:01.322 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T21:42:01.372 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:01.370+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:01.389 INFO:teuthology.orchestra.run.smithi164.stdout:truncating image image_client.0-clone33 from 0x518de00 (overlap 0xae5e00) to 0x7a0e00 2024-09-21T21:42:01.449 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:01.449 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:42:01.449 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:42:01.449 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:42:01.449 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:01.459 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T21:42:01.500 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:01.499+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:01.573 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:01.571+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:01.631 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:01.631 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-21T21:42:01.631 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-21T21:42:01.631 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:42:01.631 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.631 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19744 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6471680~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:01.631 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.272996+0000", 2024-09-21T21:42:01.632 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 10.355592262, 2024-09-21T21:42:01.632 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97298795400000004, 2024-09-21T21:42:01.632 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:01.632 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:01.632 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:01.632 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:01.632 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:01.632 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:01.632 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19744 2024-09-21T21:42:01.632 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.632 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:01.632 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.632 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:01.632 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.272996+0000", 2024-09-21T21:42:01.633 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:01.633 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.633 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.633 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:01.633 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.272996+0000", 2024-09-21T21:42:01.633 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:01.633 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.633 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.633 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:01.633 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.272996+0000", 2024-09-21T21:42:01.633 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.53e-07 2024-09-21T21:42:01.633 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.633 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.633 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:01.633 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.272998+0000", 2024-09-21T21:42:01.634 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.888e-06 2024-09-21T21:42:01.634 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.634 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.634 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:01.634 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273005+0000", 2024-09-21T21:42:01.634 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4350000000000002e-06 2024-09-21T21:42:01.634 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.634 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.634 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:01.634 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273009+0000", 2024-09-21T21:42:01.634 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3239999999999996e-06 2024-09-21T21:42:01.634 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.634 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.634 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:01.635 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277855+0000", 2024-09-21T21:42:01.635 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048463849999999999 2024-09-21T21:42:01.635 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.635 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.635 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:01.635 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277858+0000", 2024-09-21T21:42:01.635 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9490000000000001e-06 2024-09-21T21:42:01.635 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.635 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.635 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:01.635 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.245879+0000", 2024-09-21T21:42:01.635 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96802068399999996 2024-09-21T21:42:01.635 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.635 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.636 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:01.636 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.245943+0000", 2024-09-21T21:42:01.636 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3690999999999997e-05 2024-09-21T21:42:01.636 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.636 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.636 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:01.636 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.245984+0000", 2024-09-21T21:42:01.636 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1245000000000002e-05 2024-09-21T21:42:01.636 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:01.637 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:01.637 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:01.637 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.637 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.637 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19745 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 7876608~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:01.637 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273015+0000", 2024-09-21T21:42:01.637 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 10.355573613000001, 2024-09-21T21:42:01.637 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97312649200000001, 2024-09-21T21:42:01.637 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:01.637 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:01.637 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:01.637 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:01.637 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:01.637 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:01.638 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19745 2024-09-21T21:42:01.638 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.638 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:01.638 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.638 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:01.638 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273015+0000", 2024-09-21T21:42:01.638 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:01.638 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.638 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.638 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:01.638 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273015+0000", 2024-09-21T21:42:01.638 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:01.638 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.638 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.639 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:01.639 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273015+0000", 2024-09-21T21:42:01.639 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2300000000000002e-07 2024-09-21T21:42:01.639 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.639 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.639 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:01.639 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273017+0000", 2024-09-21T21:42:01.639 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0480000000000001e-06 2024-09-21T21:42:01.639 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.639 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.639 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:01.639 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273023+0000", 2024-09-21T21:42:01.639 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.699e-06 2024-09-21T21:42:01.639 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.640 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.640 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:01.640 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273027+0000", 2024-09-21T21:42:01.640 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6689999999999997e-06 2024-09-21T21:42:01.640 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.640 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.640 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:01.640 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277895+0000", 2024-09-21T21:42:01.640 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048675039999999999 2024-09-21T21:42:01.640 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.640 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.640 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:01.640 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277899+0000", 2024-09-21T21:42:01.640 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5519999999999998e-06 2024-09-21T21:42:01.641 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.641 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.641 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:01.641 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246018+0000", 2024-09-21T21:42:01.641 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96811853599999997 2024-09-21T21:42:01.641 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.641 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.641 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:01.641 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246090+0000", 2024-09-21T21:42:01.641 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2502999999999993e-05 2024-09-21T21:42:01.641 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.641 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.641 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:01.641 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246141+0000", 2024-09-21T21:42:01.641 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0658000000000002e-05 2024-09-21T21:42:01.641 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:01.642 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:01.642 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:01.642 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.642 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.642 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19746 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 7188480~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:01.642 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273033+0000", 2024-09-21T21:42:01.642 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 10.35555503, 2024-09-21T21:42:01.642 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97319625399999998, 2024-09-21T21:42:01.642 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:01.642 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:01.642 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:01.642 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:01.642 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:01.642 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:01.643 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19746 2024-09-21T21:42:01.643 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.643 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:01.643 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.643 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:01.643 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273033+0000", 2024-09-21T21:42:01.643 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:01.643 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.643 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.643 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:01.643 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273033+0000", 2024-09-21T21:42:01.644 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:01.644 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.644 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.644 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:01.644 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273034+0000", 2024-09-21T21:42:01.644 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2100000000000002e-07 2024-09-21T21:42:01.644 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.644 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.644 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:01.644 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273036+0000", 2024-09-21T21:42:01.644 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1919999999999999e-06 2024-09-21T21:42:01.644 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.644 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.644 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:01.644 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273044+0000", 2024-09-21T21:42:01.645 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2369999999999992e-06 2024-09-21T21:42:01.645 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.645 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.645 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:01.645 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273048+0000", 2024-09-21T21:42:01.645 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5090000000000004e-06 2024-09-21T21:42:01.645 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.645 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.645 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:01.645 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277921+0000", 2024-09-21T21:42:01.645 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048723710000000003 2024-09-21T21:42:01.645 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.645 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.646 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:01.646 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277924+0000", 2024-09-21T21:42:01.646 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6509999999999999e-06 2024-09-21T21:42:01.646 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.646 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.646 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:01.646 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246153+0000", 2024-09-21T21:42:01.646 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96822869899999997 2024-09-21T21:42:01.646 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.646 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.646 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:01.646 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246192+0000", 2024-09-21T21:42:01.646 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8747e-05 2024-09-21T21:42:01.646 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.646 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.647 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:01.647 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246229+0000", 2024-09-21T21:42:01.647 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7426999999999999e-05 2024-09-21T21:42:01.647 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:01.647 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:01.647 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:01.647 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.647 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.647 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19747 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 7905280~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:01.647 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273058+0000", 2024-09-21T21:42:01.647 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 10.355530308000001, 2024-09-21T21:42:01.647 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97338591399999996, 2024-09-21T21:42:01.647 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:01.647 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:01.647 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:01.648 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:01.648 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:01.648 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:01.648 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19747 2024-09-21T21:42:01.648 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.648 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:01.648 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.648 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:01.648 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273058+0000", 2024-09-21T21:42:01.648 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:01.648 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.648 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.648 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:01.648 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273058+0000", 2024-09-21T21:42:01.648 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:01.649 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.649 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.649 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:01.649 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273058+0000", 2024-09-21T21:42:01.649 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5900000000000003e-07 2024-09-21T21:42:01.649 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.649 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.649 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:01.649 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273061+0000", 2024-09-21T21:42:01.649 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3479999999999998e-06 2024-09-21T21:42:01.649 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.649 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.649 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:01.649 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273067+0000", 2024-09-21T21:42:01.649 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1469999999999998e-06 2024-09-21T21:42:01.650 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.650 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.650 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:01.650 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273071+0000", 2024-09-21T21:42:01.650 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7339999999999999e-06 2024-09-21T21:42:01.650 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.650 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.650 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:01.650 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277945+0000", 2024-09-21T21:42:01.650 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.004873474 2024-09-21T21:42:01.650 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.650 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.650 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:01.650 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277948+0000", 2024-09-21T21:42:01.650 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0299999999999998e-06 2024-09-21T21:42:01.651 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.651 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.651 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:01.651 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246269+0000", 2024-09-21T21:42:01.651 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96832060200000003 2024-09-21T21:42:01.651 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.651 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.651 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:01.651 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246362+0000", 2024-09-21T21:42:01.651 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3867000000000001e-05 2024-09-21T21:42:01.651 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.651 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.651 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:01.651 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246444+0000", 2024-09-21T21:42:01.651 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1353000000000005e-05 2024-09-21T21:42:01.652 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:01.652 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:01.652 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:01.652 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.652 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.652 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19755 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 5783552~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:01.652 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273077+0000", 2024-09-21T21:42:01.652 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 10.355510691999999, 2024-09-21T21:42:01.652 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97350203400000002, 2024-09-21T21:42:01.652 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:01.652 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:01.652 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:01.652 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:01.652 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:01.652 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:01.653 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19755 2024-09-21T21:42:01.653 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.653 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:01.653 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.653 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:01.653 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273077+0000", 2024-09-21T21:42:01.653 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:01.653 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.653 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.653 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:01.653 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273077+0000", 2024-09-21T21:42:01.653 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:01.653 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.653 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.653 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:01.654 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273078+0000", 2024-09-21T21:42:01.654 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5699999999999998e-07 2024-09-21T21:42:01.654 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.654 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.654 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:01.654 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273083+0000", 2024-09-21T21:42:01.654 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8670000000000002e-06 2024-09-21T21:42:01.654 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.654 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.654 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:01.654 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273088+0000", 2024-09-21T21:42:01.654 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8370000000000002e-06 2024-09-21T21:42:01.654 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.654 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.655 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:01.655 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273093+0000", 2024-09-21T21:42:01.655 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5709999999999999e-06 2024-09-21T21:42:01.655 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.655 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.655 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:01.655 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277968+0000", 2024-09-21T21:42:01.655 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048753909999999998 2024-09-21T21:42:01.655 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.655 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.655 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:01.655 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277971+0000", 2024-09-21T21:42:01.655 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0189999999999998e-06 2024-09-21T21:42:01.655 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.655 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.655 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:01.656 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246485+0000", 2024-09-21T21:42:01.656 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96851399299999996 2024-09-21T21:42:01.656 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.656 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.656 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:01.656 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246536+0000", 2024-09-21T21:42:01.656 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0430000000000003e-05 2024-09-21T21:42:01.656 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.656 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.656 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:01.656 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246579+0000", 2024-09-21T21:42:01.656 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3569000000000002e-05 2024-09-21T21:42:01.656 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:01.657 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:01.657 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:01.657 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.657 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.657 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19756 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6500352~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:01.657 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273099+0000", 2024-09-21T21:42:01.657 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 10.355489319, 2024-09-21T21:42:01.657 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97359272799999996, 2024-09-21T21:42:01.657 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:01.657 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:01.657 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:01.657 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:01.657 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:01.657 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:01.657 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19756 2024-09-21T21:42:01.658 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.658 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:01.658 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.658 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:01.658 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273099+0000", 2024-09-21T21:42:01.658 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:01.658 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.658 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.658 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:01.658 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273099+0000", 2024-09-21T21:42:01.658 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:01.658 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.658 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.658 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:01.659 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273099+0000", 2024-09-21T21:42:01.659 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0900000000000002e-07 2024-09-21T21:42:01.659 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.659 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.659 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:01.659 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273101+0000", 2024-09-21T21:42:01.659 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0320000000000002e-06 2024-09-21T21:42:01.659 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.659 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.659 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:01.659 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273107+0000", 2024-09-21T21:42:01.659 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2830000000000001e-06 2024-09-21T21:42:01.659 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.659 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.660 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:01.660 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273111+0000", 2024-09-21T21:42:01.660 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2230000000000001e-06 2024-09-21T21:42:01.660 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.660 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.660 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:01.660 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277994+0000", 2024-09-21T21:42:01.662 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048837070000000002 2024-09-21T21:42:01.662 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.662 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.662 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:01.662 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278000+0000", 2024-09-21T21:42:01.662 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3149999999999999e-06 2024-09-21T21:42:01.662 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.662 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.662 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:01.662 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246607+0000", 2024-09-21T21:42:01.662 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.968606839 2024-09-21T21:42:01.662 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.662 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.663 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:01.663 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246651+0000", 2024-09-21T21:42:01.663 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4734999999999998e-05 2024-09-21T21:42:01.663 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.663 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.663 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:01.663 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246692+0000", 2024-09-21T21:42:01.663 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0185000000000002e-05 2024-09-21T21:42:01.663 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:01.663 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:01.663 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:01.663 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.663 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.664 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19757 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 5812224~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:01.664 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273116+0000", 2024-09-21T21:42:01.664 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 10.355471661999999, 2024-09-21T21:42:01.664 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97367813999999997, 2024-09-21T21:42:01.664 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:01.664 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:01.664 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:01.664 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:01.664 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:01.664 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:01.664 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19757 2024-09-21T21:42:01.664 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.664 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:01.664 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.664 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:01.665 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273116+0000", 2024-09-21T21:42:01.665 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:01.665 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.665 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.665 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:01.665 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273116+0000", 2024-09-21T21:42:01.665 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:01.665 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.665 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.665 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:01.665 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273117+0000", 2024-09-21T21:42:01.665 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0999999999999999e-07 2024-09-21T21:42:01.665 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.665 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.665 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:01.666 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273119+0000", 2024-09-21T21:42:01.666 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9970000000000001e-06 2024-09-21T21:42:01.666 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.666 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.666 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:01.666 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273125+0000", 2024-09-21T21:42:01.666 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9540000000000001e-06 2024-09-21T21:42:01.666 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.666 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.666 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:01.666 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273135+0000", 2024-09-21T21:42:01.666 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0597999999999999e-05 2024-09-21T21:42:01.666 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.666 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.667 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:01.667 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278019+0000", 2024-09-21T21:42:01.667 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048834849999999999 2024-09-21T21:42:01.667 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.667 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.667 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:01.667 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278022+0000", 2024-09-21T21:42:01.667 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9560000000000002e-06 2024-09-21T21:42:01.667 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.667 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.667 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:01.667 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246716+0000", 2024-09-21T21:42:01.667 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.968694202 2024-09-21T21:42:01.667 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.668 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.668 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:01.668 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246758+0000", 2024-09-21T21:42:01.668 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1607000000000003e-05 2024-09-21T21:42:01.668 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.668 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.668 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:01.668 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246795+0000", 2024-09-21T21:42:01.668 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6931000000000003e-05 2024-09-21T21:42:01.668 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:01.668 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:01.668 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:01.668 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.668 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.669 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19758 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6529024~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:01.669 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273142+0000", 2024-09-21T21:42:01.669 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 10.355446441, 2024-09-21T21:42:01.669 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97375924599999997, 2024-09-21T21:42:01.669 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:01.669 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:01.669 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:01.669 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:01.669 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:01.669 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:01.669 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19758 2024-09-21T21:42:01.669 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.669 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:01.669 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.670 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:01.670 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273142+0000", 2024-09-21T21:42:01.670 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:01.670 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.670 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.670 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:01.670 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273142+0000", 2024-09-21T21:42:01.670 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:01.670 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.670 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.670 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:01.670 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273142+0000", 2024-09-21T21:42:01.670 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0200000000000003e-07 2024-09-21T21:42:01.670 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.671 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.671 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:01.671 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273144+0000", 2024-09-21T21:42:01.671 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1390000000000002e-06 2024-09-21T21:42:01.671 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.671 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.671 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:01.671 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273150+0000", 2024-09-21T21:42:01.671 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6409999999999999e-06 2024-09-21T21:42:01.671 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.671 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.671 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:01.671 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273157+0000", 2024-09-21T21:42:01.671 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1359999999999999e-06 2024-09-21T21:42:01.672 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.672 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.672 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:01.672 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278050+0000", 2024-09-21T21:42:01.672 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048926170000000001 2024-09-21T21:42:01.672 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.672 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.672 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:01.672 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278053+0000", 2024-09-21T21:42:01.672 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4879999999999999e-06 2024-09-21T21:42:01.672 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.672 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.672 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:01.672 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246819+0000", 2024-09-21T21:42:01.672 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96876590399999996 2024-09-21T21:42:01.672 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.673 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.673 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:01.673 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246856+0000", 2024-09-21T21:42:01.673 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7287999999999999e-05 2024-09-21T21:42:01.673 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.673 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.673 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:01.673 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246901+0000", 2024-09-21T21:42:01.673 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4631e-05 2024-09-21T21:42:01.673 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:01.673 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:01.673 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:01.673 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.673 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.674 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19759 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 7933952~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:01.674 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273172+0000", 2024-09-21T21:42:01.674 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 10.355416177, 2024-09-21T21:42:01.674 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97384789500000002, 2024-09-21T21:42:01.674 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:01.674 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:01.674 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:01.674 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:01.674 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:01.674 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:01.674 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19759 2024-09-21T21:42:01.674 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.674 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:01.674 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.674 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:01.675 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273172+0000", 2024-09-21T21:42:01.675 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:01.675 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.675 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.675 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:01.675 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273172+0000", 2024-09-21T21:42:01.675 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:01.675 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.675 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.675 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:01.675 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273172+0000", 2024-09-21T21:42:01.675 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6100000000000002e-07 2024-09-21T21:42:01.675 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.675 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.676 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:01.676 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273174+0000", 2024-09-21T21:42:01.676 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.691e-06 2024-09-21T21:42:01.676 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.676 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.676 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:01.676 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273178+0000", 2024-09-21T21:42:01.676 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7579999999999998e-06 2024-09-21T21:42:01.676 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.676 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.676 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:01.676 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273180+0000", 2024-09-21T21:42:01.676 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7199999999999998e-06 2024-09-21T21:42:01.676 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.677 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.677 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:01.677 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278077+0000", 2024-09-21T21:42:01.677 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048969529999999999 2024-09-21T21:42:01.677 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.677 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.677 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:01.677 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278080+0000", 2024-09-21T21:42:01.677 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.066e-06 2024-09-21T21:42:01.677 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.677 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.677 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:01.677 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246925+0000", 2024-09-21T21:42:01.677 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96884490199999995 2024-09-21T21:42:01.677 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.678 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.678 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:01.678 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246974+0000", 2024-09-21T21:42:01.678 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8387999999999998e-05 2024-09-21T21:42:01.678 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.678 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.678 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:01.678 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247020+0000", 2024-09-21T21:42:01.678 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6156000000000003e-05 2024-09-21T21:42:01.678 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:01.678 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:01.678 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:01.678 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.678 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.678 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19760 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 7991296~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:01.679 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273220+0000", 2024-09-21T21:42:01.679 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 10.355367662999999, 2024-09-21T21:42:01.679 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97403125499999998, 2024-09-21T21:42:01.679 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:01.679 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:01.679 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:01.679 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:01.679 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:01.679 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:01.679 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19760 2024-09-21T21:42:01.679 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.679 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:01.679 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.679 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:01.679 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273220+0000", 2024-09-21T21:42:01.680 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:01.680 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.680 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.680 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:01.680 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273220+0000", 2024-09-21T21:42:01.680 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:01.680 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.680 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.680 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:01.680 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273221+0000", 2024-09-21T21:42:01.680 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.65e-07 2024-09-21T21:42:01.680 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.680 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.680 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:01.680 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273228+0000", 2024-09-21T21:42:01.681 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6040000000000004e-06 2024-09-21T21:42:01.681 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.681 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.681 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:01.681 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273233+0000", 2024-09-21T21:42:01.681 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1569999999999999e-06 2024-09-21T21:42:01.681 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.681 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.681 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:01.681 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273238+0000", 2024-09-21T21:42:01.681 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0050000000000004e-06 2024-09-21T21:42:01.681 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.681 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.681 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:01.681 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278107+0000", 2024-09-21T21:42:01.682 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00486857 2024-09-21T21:42:01.682 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.682 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.682 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:01.682 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278111+0000", 2024-09-21T21:42:01.682 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6389999999999998e-06 2024-09-21T21:42:01.682 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.682 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.682 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:01.682 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247069+0000", 2024-09-21T21:42:01.682 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96895796899999997 2024-09-21T21:42:01.682 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.682 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.682 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:01.683 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247166+0000", 2024-09-21T21:42:01.683 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.7158999999999996e-05 2024-09-21T21:42:01.683 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.683 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.683 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:01.683 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247252+0000", 2024-09-21T21:42:01.683 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5886999999999999e-05 2024-09-21T21:42:01.683 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:01.683 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:01.683 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:01.683 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.683 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.683 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19761 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 7962624~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:01.683 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273246+0000", 2024-09-21T21:42:01.683 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 10.355342603, 2024-09-21T21:42:01.684 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97415972200000001, 2024-09-21T21:42:01.684 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:01.684 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:01.684 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:01.684 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:01.684 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:01.684 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:01.684 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19761 2024-09-21T21:42:01.684 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.684 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:01.684 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.684 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:01.684 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273246+0000", 2024-09-21T21:42:01.684 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:01.684 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.685 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.685 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:01.685 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273246+0000", 2024-09-21T21:42:01.685 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:01.685 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.685 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.685 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:01.685 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273246+0000", 2024-09-21T21:42:01.685 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5100000000000001e-07 2024-09-21T21:42:01.685 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.685 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.685 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:01.685 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273248+0000", 2024-09-21T21:42:01.685 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1550000000000001e-06 2024-09-21T21:42:01.685 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.686 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.686 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:01.686 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273254+0000", 2024-09-21T21:42:01.686 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.964e-06 2024-09-21T21:42:01.686 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.686 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.686 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:01.686 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273259+0000", 2024-09-21T21:42:01.686 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5499999999999996e-06 2024-09-21T21:42:01.686 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.686 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.686 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:01.686 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278130+0000", 2024-09-21T21:42:01.686 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048715779999999997 2024-09-21T21:42:01.687 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.687 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.687 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:01.687 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278133+0000", 2024-09-21T21:42:01.687 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8739999999999999e-06 2024-09-21T21:42:01.687 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.687 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.687 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:01.687 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247270+0000", 2024-09-21T21:42:01.687 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.969137099 2024-09-21T21:42:01.687 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.687 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.687 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:01.687 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247338+0000", 2024-09-21T21:42:01.688 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7717000000000002e-05 2024-09-21T21:42:01.688 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.688 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.688 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:01.688 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247405+0000", 2024-09-21T21:42:01.688 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7434000000000005e-05 2024-09-21T21:42:01.688 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:01.688 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:01.688 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:01.688 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.688 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.688 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19769 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 5840896~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:01.688 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273287+0000", 2024-09-21T21:42:01.689 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 10.355301319000001, 2024-09-21T21:42:01.689 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97435390200000005, 2024-09-21T21:42:01.689 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:01.689 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:01.689 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:01.689 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:01.689 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:01.689 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:01.689 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19769 2024-09-21T21:42:01.689 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.689 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:01.689 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.689 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:01.689 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273287+0000", 2024-09-21T21:42:01.690 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:01.690 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.690 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.690 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:01.690 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273287+0000", 2024-09-21T21:42:01.690 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:01.690 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.690 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.690 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:01.690 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273287+0000", 2024-09-21T21:42:01.690 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6699999999999999e-07 2024-09-21T21:42:01.690 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.690 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.690 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:01.691 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273289+0000", 2024-09-21T21:42:01.691 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.708e-06 2024-09-21T21:42:01.691 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.691 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.691 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:01.691 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273296+0000", 2024-09-21T21:42:01.691 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8109999999999997e-06 2024-09-21T21:42:01.691 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.691 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.691 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:01.691 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273299+0000", 2024-09-21T21:42:01.691 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0740000000000001e-06 2024-09-21T21:42:01.691 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.692 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.692 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:01.692 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278159+0000", 2024-09-21T21:42:01.692 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048603240000000001 2024-09-21T21:42:01.692 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.692 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.692 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:01.692 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278162+0000", 2024-09-21T21:42:01.692 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1930000000000002e-06 2024-09-21T21:42:01.692 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.692 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.692 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:01.692 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247444+0000", 2024-09-21T21:42:01.692 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96928199100000001 2024-09-21T21:42:01.693 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.693 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.693 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:01.693 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247533+0000", 2024-09-21T21:42:01.693 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8490000000000001e-05 2024-09-21T21:42:01.693 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.693 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.693 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:01.693 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247641+0000", 2024-09-21T21:42:01.693 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000107944 2024-09-21T21:42:01.693 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:01.694 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:01.694 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:01.694 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.694 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.694 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19770 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 5869568~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:01.694 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273313+0000", 2024-09-21T21:42:01.694 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 10.355275216999999, 2024-09-21T21:42:01.694 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97449038899999996, 2024-09-21T21:42:01.694 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:01.694 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:01.694 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:01.694 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:01.694 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:01.695 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:01.695 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19770 2024-09-21T21:42:01.695 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.695 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:01.695 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.695 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:01.695 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273313+0000", 2024-09-21T21:42:01.695 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:01.695 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.695 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.695 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:01.695 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273313+0000", 2024-09-21T21:42:01.695 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:01.695 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.695 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.696 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:01.696 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273313+0000", 2024-09-21T21:42:01.696 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5699999999999998e-07 2024-09-21T21:42:01.696 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.696 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.696 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:01.696 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273315+0000", 2024-09-21T21:42:01.696 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0099999999999998e-06 2024-09-21T21:42:01.696 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.696 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.696 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:01.696 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273321+0000", 2024-09-21T21:42:01.696 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7529999999999998e-06 2024-09-21T21:42:01.696 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.696 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.697 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:01.697 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273328+0000", 2024-09-21T21:42:01.697 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9009999999999997e-06 2024-09-21T21:42:01.697 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.697 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.697 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:01.697 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278183+0000", 2024-09-21T21:42:01.697 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048547770000000002 2024-09-21T21:42:01.697 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.697 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.697 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:01.697 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278186+0000", 2024-09-21T21:42:01.697 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3100000000000001e-06 2024-09-21T21:42:01.697 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.697 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.697 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:01.698 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247676+0000", 2024-09-21T21:42:01.698 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96949004299999997 2024-09-21T21:42:01.698 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.698 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.698 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:01.698 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247757+0000", 2024-09-21T21:42:01.698 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0479e-05 2024-09-21T21:42:01.698 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.698 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.698 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:01.698 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247803+0000", 2024-09-21T21:42:01.698 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6758999999999997e-05 2024-09-21T21:42:01.698 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:01.698 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:01.698 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:01.699 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.699 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.699 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19771 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6586368~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:01.699 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273335+0000", 2024-09-21T21:42:01.699 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 10.355253429999999, 2024-09-21T21:42:01.699 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97458947699999998, 2024-09-21T21:42:01.699 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:01.699 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:01.699 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:01.699 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:01.699 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:01.699 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:01.699 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19771 2024-09-21T21:42:01.699 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.699 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:01.700 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.700 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:01.700 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273335+0000", 2024-09-21T21:42:01.700 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:01.700 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.700 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.700 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:01.700 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273335+0000", 2024-09-21T21:42:01.700 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:01.700 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.700 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.700 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:01.700 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273335+0000", 2024-09-21T21:42:01.700 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3599999999999999e-07 2024-09-21T21:42:01.700 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.701 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.701 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:01.701 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273337+0000", 2024-09-21T21:42:01.701 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0789999999999999e-06 2024-09-21T21:42:01.701 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.701 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.701 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:01.701 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273343+0000", 2024-09-21T21:42:01.701 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6280000000000002e-06 2024-09-21T21:42:01.701 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.701 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.701 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:01.701 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273346+0000", 2024-09-21T21:42:01.701 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.507e-06 2024-09-21T21:42:01.701 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.701 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.702 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:01.702 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278206+0000", 2024-09-21T21:42:01.702 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048600969999999999 2024-09-21T21:42:01.702 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.702 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.702 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:01.702 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278212+0000", 2024-09-21T21:42:01.702 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2079999999999999e-06 2024-09-21T21:42:01.702 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.702 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.702 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:01.702 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247828+0000", 2024-09-21T21:42:01.702 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.969616547 2024-09-21T21:42:01.703 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.703 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.703 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:01.703 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247881+0000", 2024-09-21T21:42:01.703 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3180999999999998e-05 2024-09-21T21:42:01.703 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.703 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.703 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:01.703 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247924+0000", 2024-09-21T21:42:01.703 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2894e-05 2024-09-21T21:42:01.703 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:01.703 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:01.703 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:01.704 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.704 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.704 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19772 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6557696~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:01.704 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273358+0000", 2024-09-21T21:42:01.704 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 10.355230049999999, 2024-09-21T21:42:01.704 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97467031800000004, 2024-09-21T21:42:01.704 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:01.704 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:01.704 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:01.704 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:01.704 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:01.704 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:01.704 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19772 2024-09-21T21:42:01.704 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.705 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:01.705 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.705 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:01.705 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273358+0000", 2024-09-21T21:42:01.705 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:01.705 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.705 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.705 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:01.705 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273358+0000", 2024-09-21T21:42:01.705 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:01.705 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.705 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.705 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:01.705 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273358+0000", 2024-09-21T21:42:01.705 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4499999999999998e-07 2024-09-21T21:42:01.705 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.706 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.706 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:01.706 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273360+0000", 2024-09-21T21:42:01.706 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9190000000000002e-06 2024-09-21T21:42:01.706 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.706 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.706 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:01.706 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273366+0000", 2024-09-21T21:42:01.706 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.891e-06 2024-09-21T21:42:01.706 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.706 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.706 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:01.706 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273371+0000", 2024-09-21T21:42:01.706 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1880000000000001e-06 2024-09-21T21:42:01.706 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.706 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.707 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:01.707 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278237+0000", 2024-09-21T21:42:01.707 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.004865472 2024-09-21T21:42:01.707 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.707 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.707 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:01.707 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278240+0000", 2024-09-21T21:42:01.707 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9550000000000001e-06 2024-09-21T21:42:01.707 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.707 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.707 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:01.707 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247949+0000", 2024-09-21T21:42:01.707 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.969709127 2024-09-21T21:42:01.707 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.708 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.708 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:01.708 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247990+0000", 2024-09-21T21:42:01.708 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1202000000000003e-05 2024-09-21T21:42:01.708 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.708 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.708 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:01.708 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248028+0000", 2024-09-21T21:42:01.708 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8219000000000001e-05 2024-09-21T21:42:01.708 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:01.708 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:01.708 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:01.708 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.708 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.708 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19773 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6615040~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:01.708 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273390+0000", 2024-09-21T21:42:01.708 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 10.355198188999999, 2024-09-21T21:42:01.709 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97476086200000001, 2024-09-21T21:42:01.709 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:01.709 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:01.709 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:01.709 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:01.709 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:01.709 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:01.709 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19773 2024-09-21T21:42:01.709 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.709 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:01.709 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.709 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:01.709 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273390+0000", 2024-09-21T21:42:01.709 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:01.709 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.709 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.710 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:01.710 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273390+0000", 2024-09-21T21:42:01.710 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:01.710 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.710 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.710 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:01.710 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273390+0000", 2024-09-21T21:42:01.710 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9999999999999999e-07 2024-09-21T21:42:01.710 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.710 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.710 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:01.710 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273392+0000", 2024-09-21T21:42:01.710 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0640000000000001e-06 2024-09-21T21:42:01.710 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.710 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.711 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:01.711 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273397+0000", 2024-09-21T21:42:01.711 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.113e-06 2024-09-21T21:42:01.711 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.711 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.711 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:01.711 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273401+0000", 2024-09-21T21:42:01.711 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3119999999999998e-06 2024-09-21T21:42:01.711 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.711 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.711 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:01.711 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278271+0000", 2024-09-21T21:42:01.711 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048698880000000002 2024-09-21T21:42:01.711 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.711 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.712 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:01.712 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278278+0000", 2024-09-21T21:42:01.712 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1210000000000001e-06 2024-09-21T21:42:01.712 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.712 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.712 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:01.712 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248060+0000", 2024-09-21T21:42:01.712 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96978224999999996 2024-09-21T21:42:01.712 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.712 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.712 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:01.712 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248111+0000", 2024-09-21T21:42:01.712 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.096e-05 2024-09-21T21:42:01.712 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.712 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.712 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:01.713 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248151+0000", 2024-09-21T21:42:01.713 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9854000000000002e-05 2024-09-21T21:42:01.713 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:01.713 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:01.713 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:01.713 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.713 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.713 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19784 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6643712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:01.713 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273405+0000", 2024-09-21T21:42:01.713 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 10.355183468, 2024-09-21T21:42:01.713 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97491798900000004, 2024-09-21T21:42:01.713 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:01.713 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:01.714 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:01.714 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:01.714 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:01.714 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:01.714 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19784 2024-09-21T21:42:01.714 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.714 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:01.714 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.714 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:01.714 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273405+0000", 2024-09-21T21:42:01.714 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:01.714 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.714 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.714 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:01.715 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273405+0000", 2024-09-21T21:42:01.715 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:01.715 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.715 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.715 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:01.715 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273405+0000", 2024-09-21T21:42:01.715 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2100000000000001e-07 2024-09-21T21:42:01.715 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.715 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.715 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:01.715 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273406+0000", 2024-09-21T21:42:01.715 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2049999999999999e-06 2024-09-21T21:42:01.715 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.715 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.716 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:01.716 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273409+0000", 2024-09-21T21:42:01.716 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0599999999999999e-06 2024-09-21T21:42:01.716 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.716 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.716 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:01.716 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273412+0000", 2024-09-21T21:42:01.716 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0730000000000001e-06 2024-09-21T21:42:01.716 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.716 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.716 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:01.716 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278317+0000", 2024-09-21T21:42:01.716 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0049045520000000004 2024-09-21T21:42:01.716 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.717 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.717 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:01.717 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278321+0000", 2024-09-21T21:42:01.717 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6249999999999998e-06 2024-09-21T21:42:01.717 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.717 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.717 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:01.717 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248179+0000", 2024-09-21T21:42:01.717 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96985744500000004 2024-09-21T21:42:01.717 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.717 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.717 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:01.717 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248253+0000", 2024-09-21T21:42:01.717 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3829000000000001e-05 2024-09-21T21:42:01.718 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.718 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.718 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:01.718 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248323+0000", 2024-09-21T21:42:01.718 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9979000000000002e-05 2024-09-21T21:42:01.718 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:01.718 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:01.718 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:01.718 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.718 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.718 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19785 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6672384~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:01.718 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273415+0000", 2024-09-21T21:42:01.718 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 10.355172882, 2024-09-21T21:42:01.718 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97500777800000005, 2024-09-21T21:42:01.718 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:01.719 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:01.719 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:01.719 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:01.719 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:01.719 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:01.719 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19785 2024-09-21T21:42:01.719 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.719 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:01.719 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.719 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:01.719 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273415+0000", 2024-09-21T21:42:01.719 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:01.719 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.719 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.719 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:01.720 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273415+0000", 2024-09-21T21:42:01.720 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:01.720 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.720 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.720 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:01.720 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273415+0000", 2024-09-21T21:42:01.720 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0100000000000001e-07 2024-09-21T21:42:01.720 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.720 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.720 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:01.720 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273417+0000", 2024-09-21T21:42:01.720 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.114e-06 2024-09-21T21:42:01.720 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.720 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.720 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:01.721 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273420+0000", 2024-09-21T21:42:01.721 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6770000000000001e-06 2024-09-21T21:42:01.721 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.721 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.721 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:01.721 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273423+0000", 2024-09-21T21:42:01.721 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5450000000000002e-06 2024-09-21T21:42:01.721 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.721 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.721 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:01.721 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278355+0000", 2024-09-21T21:42:01.721 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0049318590000000002 2024-09-21T21:42:01.721 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.721 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.722 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:01.722 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278360+0000", 2024-09-21T21:42:01.722 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3079999999999998e-06 2024-09-21T21:42:01.722 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.722 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.722 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:01.722 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248349+0000", 2024-09-21T21:42:01.722 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96998909899999997 2024-09-21T21:42:01.722 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.722 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.722 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:01.722 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248387+0000", 2024-09-21T21:42:01.722 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7657000000000002e-05 2024-09-21T21:42:01.722 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.722 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.723 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:01.723 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248423+0000", 2024-09-21T21:42:01.723 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6318e-05 2024-09-21T21:42:01.723 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:01.723 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:01.723 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:01.723 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.723 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.723 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19842 3.d 3:b7fa2ca0:::rbd_data.112157a479c.000000000000004d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:01.723 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273426+0000", 2024-09-21T21:42:01.723 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 10.35516183, 2024-09-21T21:42:01.723 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.98136526800000001, 2024-09-21T21:42:01.723 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:01.723 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:01.723 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:01.724 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:01.724 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:01.724 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:01.724 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19842 2024-09-21T21:42:01.724 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.724 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:01.724 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.724 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:01.724 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273426+0000", 2024-09-21T21:42:01.724 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:01.724 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.724 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.725 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:01.725 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273426+0000", 2024-09-21T21:42:01.725 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:01.725 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.725 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.725 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:01.725 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273427+0000", 2024-09-21T21:42:01.725 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.72e-07 2024-09-21T21:42:01.725 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.725 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.725 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:01.725 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275836+0000", 2024-09-21T21:42:01.725 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00240909 2024-09-21T21:42:01.725 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.725 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.726 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:01.726 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275844+0000", 2024-09-21T21:42:01.726 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3750000000000003e-06 2024-09-21T21:42:01.726 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.726 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.726 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:01.726 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275849+0000", 2024-09-21T21:42:01.726 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0100000000000003e-06 2024-09-21T21:42:01.726 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.726 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.726 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:01.726 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278394+0000", 2024-09-21T21:42:01.726 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025449890000000001 2024-09-21T21:42:01.726 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.727 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.727 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:01.727 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278402+0000", 2024-09-21T21:42:01.727 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6359999999999994e-06 2024-09-21T21:42:01.727 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.727 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.727 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:01.727 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248454+0000", 2024-09-21T21:42:01.727 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97005259499999996 2024-09-21T21:42:01.727 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.728 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.728 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:01.728 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248576+0000", 2024-09-21T21:42:01.728 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000122127 2024-09-21T21:42:01.728 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.728 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.728 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 2", 2024-09-21T21:42:01.728 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248844+0000", 2024-09-21T21:42:01.728 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00026784699999999999 2024-09-21T21:42:01.728 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.728 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.728 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:01.728 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.252487+0000", 2024-09-21T21:42:01.729 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0036427920000000002 2024-09-21T21:42:01.729 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.729 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.729 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:01.729 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.254691+0000", 2024-09-21T21:42:01.729 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002203899 2024-09-21T21:42:01.729 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.729 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.729 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:01.729 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.254748+0000", 2024-09-21T21:42:01.729 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7163000000000001e-05 2024-09-21T21:42:01.729 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.729 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.729 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:01.730 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.254792+0000", 2024-09-21T21:42:01.730 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3473000000000002e-05 2024-09-21T21:42:01.730 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:01.730 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:01.730 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:01.730 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.730 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.730 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19843 3.d 3:b7fa2ca0:::rbd_data.112157a479c.000000000000004d:head [write 0~0] snapc 21=[21] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:01.730 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.275860+0000", 2024-09-21T21:42:01.730 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 10.35272793, 2024-09-21T21:42:01.730 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.98183651999999999, 2024-09-21T21:42:01.730 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:01.730 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:01.730 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:01.731 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:01.731 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:01.731 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:01.731 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19843 2024-09-21T21:42:01.731 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.731 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:01.731 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.731 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:01.731 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275860+0000", 2024-09-21T21:42:01.731 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:01.731 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.731 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.731 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:01.731 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275860+0000", 2024-09-21T21:42:01.732 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:01.732 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.732 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.732 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:01.732 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275869+0000", 2024-09-21T21:42:01.732 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6300000000000004e-06 2024-09-21T21:42:01.732 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.732 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.732 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:01.732 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275871+0000", 2024-09-21T21:42:01.732 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2060000000000001e-06 2024-09-21T21:42:01.732 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.732 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.732 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:01.732 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275876+0000", 2024-09-21T21:42:01.733 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8180000000000003e-06 2024-09-21T21:42:01.733 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.733 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.733 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:01.733 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275880+0000", 2024-09-21T21:42:01.733 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1829999999999996e-06 2024-09-21T21:42:01.733 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.733 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.733 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:01.733 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278416+0000", 2024-09-21T21:42:01.733 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025361149999999998 2024-09-21T21:42:01.733 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.733 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.733 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:01.734 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278419+0000", 2024-09-21T21:42:01.734 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.163e-06 2024-09-21T21:42:01.734 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.734 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.734 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:01.734 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.250249+0000", 2024-09-21T21:42:01.734 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97182984299999997 2024-09-21T21:42:01.734 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.734 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.734 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:01.734 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.250321+0000", 2024-09-21T21:42:01.734 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1656000000000006e-05 2024-09-21T21:42:01.734 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.734 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.735 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 2", 2024-09-21T21:42:01.735 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.250419+0000", 2024-09-21T21:42:01.735 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.8277999999999997e-05 2024-09-21T21:42:01.735 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.735 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.735 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:01.735 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.252516+0000", 2024-09-21T21:42:01.735 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002097224 2024-09-21T21:42:01.735 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.735 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.735 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:01.735 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.257643+0000", 2024-09-21T21:42:01.735 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0051262319999999997 2024-09-21T21:42:01.735 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.736 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.736 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:01.736 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.257673+0000", 2024-09-21T21:42:01.736 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0023e-05 2024-09-21T21:42:01.736 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.736 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.736 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:01.736 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.257697+0000", 2024-09-21T21:42:01.736 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4148999999999999e-05 2024-09-21T21:42:01.736 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:01.736 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:01.736 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:01.736 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:01.736 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:01.736 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:01.738 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T21:42:01.738 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:01.735+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:01.759 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T21:42:01.789 INFO:teuthology.orchestra.run.smithi164.stdout:checking clone #32, image image_client.0-clone32 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent33 2024-09-21T21:42:01.839 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:01.836+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:01.903 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:01.903 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:42:01.903 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.903 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "MOSDPGRecoveryDelete(3.15 e88,87 3:af3efed0:::rbd_data.1121900de931.0000000000000174:head,43'71)", 2024-09-21T21:42:01.903 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:01.901970+0000", 2024-09-21T21:42:01.904 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.000119212, 2024-09-21T21:42:01.904 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013461699999999999, 2024-09-21T21:42:01.904 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:01.904 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:01.904 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:01.904 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:01.904 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.904 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:01.904 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:01.901970+0000", 2024-09-21T21:42:01.904 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:01.904 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.904 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.905 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:01.905 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:01.901970+0000", 2024-09-21T21:42:01.905 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:01.905 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.905 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.905 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:01.905 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:01.901971+0000", 2024-09-21T21:42:01.905 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8599999999999998e-07 2024-09-21T21:42:01.905 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.905 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.905 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:01.905 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:01.901974+0000", 2024-09-21T21:42:01.906 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.501e-06 2024-09-21T21:42:01.906 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.906 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.906 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:01.906 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:01.901985+0000", 2024-09-21T21:42:01.906 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0441e-05 2024-09-21T21:42:01.906 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.906 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.906 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:01.906 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:01.901990+0000", 2024-09-21T21:42:01.906 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2970000000000003e-06 2024-09-21T21:42:01.907 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.907 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.907 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:01.907 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:01.902050+0000", 2024-09-21T21:42:01.907 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9562999999999999e-05 2024-09-21T21:42:01.907 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.907 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.907 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:01.907 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:01.902059+0000", 2024-09-21T21:42:01.907 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.6379999999999999e-06 2024-09-21T21:42:01.908 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:01.908 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:01.908 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:01.908 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:01.902109+0000", 2024-09-21T21:42:01.908 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9515000000000003e-05 2024-09-21T21:42:01.908 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:01.908 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:01.908 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:01.908 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:01.908 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-21T21:42:01.908 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 1 2024-09-21T21:42:01.908 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:01.909 INFO:teuthology.orchestra.run.smithi164.stdout:4230 write 0xe723e00 thru 0xe7264f7 (0x26f8 bytes) 2024-09-21T21:42:01.910 INFO:teuthology.orchestra.run.smithi164.stdout:4232 read 0x60e2200 thru 0x60e9897 (0x7698 bytes) 2024-09-21T21:42:01.910 INFO:teuthology.orchestra.run.smithi164.stdout:4234 write 0x986ae00 thru 0x986b0ee (0x2ef bytes) 2024-09-21T21:42:01.910 INFO:teuthology.orchestra.run.smithi164.stdout:4235 write 0xdbec600 thru 0xdbf21a6 (0x5ba7 bytes) 2024-09-21T21:42:01.910 INFO:teuthology.orchestra.run.smithi164.stdout:4238 read 0x300ec00 thru 0x3011a24 (0x2e25 bytes) 2024-09-21T21:42:01.910 INFO:teuthology.orchestra.run.smithi164.stdout:4242 write 0x5e2be00 thru 0x5e358a0 (0x9aa1 bytes) 2024-09-21T21:42:01.910 INFO:teuthology.orchestra.run.smithi164.stdout:4243 write 0x56e3000 thru 0x56edd82 (0xad83 bytes) 2024-09-21T21:42:01.912 INFO:teuthology.orchestra.run.smithi164.stdout:4244 trunc from 0xe726600 to 0xa355000 2024-09-21T21:42:01.915 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T21:42:01.959 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:01.957+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:02.041 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:02.040+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:02.044 INFO:teuthology.orchestra.run.smithi164.stdout:4245 write 0xd555200 thru 0xd55d80e (0x860f bytes) 2024-09-21T21:42:02.045 INFO:teuthology.orchestra.run.smithi164.stdout:4246 punch from 0x9d46200 to 0x9d49400, (0x3200 bytes) 2024-09-21T21:42:02.047 INFO:teuthology.orchestra.run.smithi164.stdout:4248 write 0x527000 thru 0x532653 (0xb654 bytes) 2024-09-21T21:42:02.049 INFO:teuthology.orchestra.run.smithi164.stdout:4249 punch from 0x8bd6c00 to 0x8bdbe00, (0x5200 bytes) 2024-09-21T21:42:02.065 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:02.065 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:42:02.065 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:42:02.065 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:42:02.065 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:02.066 INFO:teuthology.orchestra.run.smithi081.stderr:2024-09-21T21:42:02.064+0000 7f8dea7fc640 0 --2- 172.21.15.81:0/2490243561 >> [v2:172.21.15.81:3301/0,v1:172.21.15.81:6790/0] conn(0x7f8dec073a50 0x7f8dec139070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-21T21:42:02.079 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T21:42:02.089 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0: {} 2024-09-21T21:42:02.089 INFO:teuthology.orchestra.run.smithi081.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T21:42:02.105 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1: {} 2024-09-21T21:42:02.105 INFO:teuthology.orchestra.run.smithi081.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T21:42:02.121 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2: {} 2024-09-21T21:42:02.122 INFO:teuthology.orchestra.run.smithi081.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T21:42:02.139 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3: {} 2024-09-21T21:42:02.139 INFO:teuthology.orchestra.run.smithi081.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T21:42:02.142 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:02.140+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:02.156 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4: {} 2024-09-21T21:42:02.157 INFO:teuthology.orchestra.run.smithi081.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T21:42:02.176 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5: {} 2024-09-21T21:42:02.176 INFO:teuthology.orchestra.run.smithi081.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T21:42:02.243 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:02.241+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:02.252 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:02.252 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:02.263 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T21:42:02.287 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T21:42:02.347 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:02.346+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:02.353 INFO:teuthology.orchestra.run.smithi164.stdout:4253 read 0x2651800 thru 0x265e77f (0xcf80 bytes) 2024-09-21T21:42:02.435 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:02.435 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:02.444 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T21:42:02.445 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:02.442+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:02.545 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:02.544+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:02.600 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:02.600 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:02.609 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T21:42:02.650 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:02.648+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:02.747 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:02.745+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:02.752 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:02.752 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:02.760 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T21:42:02.848 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:02.846+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:02.871 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:02.871 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:02.880 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T21:42:02.963 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:02.962+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:03.023 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:03.023 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:03.031 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T21:42:03.073 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:03.071+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:03.150 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:03.149+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:03.175 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:03.175 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:03.184 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T21:42:03.211 INFO:teuthology.orchestra.run.smithi081.stderr:nodeep-scrub is set 2024-09-21T21:42:03.259 INFO:teuthology.orchestra.run.smithi164.stdout:4260 write 0xedadc00 thru 0xedbcf92 (0xf393 bytes) 2024-09-21T21:42:03.261 INFO:teuthology.orchestra.run.smithi164.stdout:4261 read 0x6f97200 thru 0x6f9a00a (0x2e0b bytes) 2024-09-21T21:42:03.261 INFO:teuthology.orchestra.run.smithi164.stdout:4263 punch from 0xe237200 to 0xe246400, (0xf200 bytes) 2024-09-21T21:42:03.262 INFO:teuthology.orchestra.run.smithi164.stdout:4264 read 0x342ba00 thru 0x34375bf (0xbbc0 bytes) 2024-09-21T21:42:03.262 INFO:teuthology.orchestra.run.smithi164.stdout:4266 read 0x5bc3c00 thru 0x5bcb2b5 (0x76b6 bytes) 2024-09-21T21:42:03.275 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:03.274+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:03.287 INFO:teuthology.orchestra.run.smithi164.stdout:4267 trunc from 0xedbd000 to 0x9f4600 2024-09-21T21:42:03.334 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:03.334 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:03.343 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T21:42:03.385 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:03.383+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:03.452 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:03.451+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:03.481 INFO:teuthology.orchestra.run.smithi164.stdout:4268 read 0x6cac00 thru 0x6ccbab (0x1fac bytes) 2024-09-21T21:42:03.497 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:03.497 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:03.498 INFO:teuthology.orchestra.run.smithi164.stdout:4270 trunc from 0x9f4600 to 0x6922000 2024-09-21T21:42:03.506 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T21:42:03.508 INFO:teuthology.orchestra.run.smithi164.stdout:4271 read 0x6816200 thru 0x6816ea4 (0xca5 bytes) 2024-09-21T21:42:03.519 INFO:teuthology.orchestra.run.smithi164.stdout:4272 trunc from 0x6922000 to 0x2d36000 2024-09-21T21:42:03.528 INFO:teuthology.orchestra.run.smithi164.stdout:4273 read 0x1579000 thru 0x158459c (0xb59d bytes) 2024-09-21T21:42:03.534 INFO:teuthology.orchestra.run.smithi164.stdout:4274 punch from 0x1a5ee00 to 0x1a63a00, (0x4c00 bytes) 2024-09-21T21:42:03.536 INFO:teuthology.orchestra.run.smithi164.stdout:4276 read 0x21ecc00 thru 0x21f700b (0xa40c bytes) 2024-09-21T21:42:03.536 INFO:teuthology.orchestra.run.smithi164.stdout:4279 read 0x109fe00 thru 0x10a6341 (0x6542 bytes) 2024-09-21T21:42:03.536 INFO:teuthology.orchestra.run.smithi164.stdout:4280 read 0x27aac00 thru 0x27b43ea (0x97eb bytes) 2024-09-21T21:42:03.553 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:03.552+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:03.579 INFO:teuthology.orchestra.run.smithi164.stdout:4281 write 0x61d4a00 thru 0x61e3a0b (0xf00c bytes) 2024-09-21T21:42:03.585 INFO:teuthology.orchestra.run.smithi164.stdout:4283 read 0x3352e00 thru 0x33578a8 (0x4aa9 bytes) 2024-09-21T21:42:03.585 INFO:teuthology.orchestra.run.smithi164.stdout:4284 read 0x57cb600 thru 0x57d2494 (0x6e95 bytes) 2024-09-21T21:42:03.589 INFO:teuthology.orchestra.run.smithi164.stdout:4285 trunc from 0x61e3c00 to 0x9ab0c00 2024-09-21T21:42:03.611 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:03.611 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:03.620 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T21:42:03.644 INFO:teuthology.orchestra.run.smithi164.stdout:4286 read 0x49bf800 thru 0x49c1923 (0x2124 bytes) 2024-09-21T21:42:03.651 INFO:teuthology.orchestra.run.smithi164.stdout:4287 read 0x171b000 thru 0x17236c4 (0x86c5 bytes) 2024-09-21T21:42:03.663 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:03.662+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:03.668 INFO:teuthology.orchestra.run.smithi164.stdout:4289 write 0xe5ff600 thru 0xe60d897 (0xe298 bytes) 2024-09-21T21:42:03.675 INFO:teuthology.orchestra.run.smithi164.stdout:4293 read 0x87c9c00 thru 0x87d312e (0x952f bytes) 2024-09-21T21:42:03.675 INFO:teuthology.orchestra.run.smithi164.stdout:4294 read 0xd1eda00 thru 0xd1fd0e2 (0xf6e3 bytes) 2024-09-21T21:42:03.676 INFO:teuthology.orchestra.run.smithi164.stdout:4295 write 0xa5ca400 thru 0xa5d18da (0x74db bytes) 2024-09-21T21:42:03.676 INFO:teuthology.orchestra.run.smithi164.stdout:4296 read 0x95d3600 thru 0x95e1fd1 (0xe9d2 bytes) 2024-09-21T21:42:03.676 INFO:teuthology.orchestra.run.smithi164.stdout:4297 write 0xc000000 thru 0xc00c729 (0xc72a bytes) 2024-09-21T21:42:03.681 INFO:teuthology.orchestra.run.smithi164.stdout:4298 trunc from 0xe60da00 to 0x8d27200 2024-09-21T21:42:03.727 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:03.727 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:03.736 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T21:42:03.755 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:03.754+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:03.803 INFO:teuthology.orchestra.run.smithi164.stdout:4304 read 0x5720e00 thru 0x572a7ed (0x99ee bytes) 2024-09-21T21:42:03.803 INFO:teuthology.orchestra.run.smithi164.stdout:4308 punch from 0x29bc800 to 0x29c0400, (0x3c00 bytes) 2024-09-21T21:42:03.836 INFO:teuthology.orchestra.run.smithi164.stdout:4310 write 0xd39aa00 thru 0xd39ebd1 (0x41d2 bytes) 2024-09-21T21:42:03.842 INFO:teuthology.orchestra.run.smithi164.stdout:4311 punch from 0x650fa00 to 0x6511c00, (0x2200 bytes) 2024-09-21T21:42:03.848 INFO:teuthology.orchestra.run.smithi164.stdout:4312 read 0x30a9600 thru 0x30b8a91 (0xf492 bytes) 2024-09-21T21:42:03.848 INFO:teuthology.orchestra.run.smithi164.stdout:4313 write 0xc628c00 thru 0xc62c084 (0x3485 bytes) 2024-09-21T21:42:03.848 INFO:teuthology.orchestra.run.smithi164.stdout:4314 read 0xa60d600 thru 0xa61a288 (0xcc89 bytes) 2024-09-21T21:42:03.848 INFO:teuthology.orchestra.run.smithi164.stdout:4315 read 0x11ab600 thru 0x11b18f6 (0x62f7 bytes) 2024-09-21T21:42:03.856 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:03.855+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:03.882 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:03.882 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:03.892 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T21:42:03.976 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:03.975+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:04.035 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:04.035 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:04.044 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T21:42:04.088 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:04.087+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:04.148 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:04.148 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:04.156 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T21:42:04.172 INFO:teuthology.orchestra.run.smithi164.stdout:4318 write 0xd80f200 thru 0xd81b358 (0xc159 bytes) 2024-09-21T21:42:04.172 INFO:teuthology.orchestra.run.smithi164.stdout:4319 read 0x719c800 thru 0x719e67c (0x1e7d bytes) 2024-09-21T21:42:04.173 INFO:teuthology.orchestra.run.smithi164.stdout:4321 read 0x364f200 thru 0x365b69b (0xc49c bytes) 2024-09-21T21:42:04.173 INFO:teuthology.orchestra.run.smithi164.stdout:4322 write 0xcf55600 thru 0xcf61e3d (0xc83e bytes) 2024-09-21T21:42:04.182 INFO:teuthology.orchestra.run.smithi164.stdout:4323 trunc from 0xd81b400 to 0x58fc200 2024-09-21T21:42:04.188 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T21:42:04.197 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:04.196+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:04.260 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:04.258+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:04.261 INFO:teuthology.orchestra.run.smithi164.stdout:4324 punch from 0x5739600 to 0x573b000, (0x1a00 bytes) 2024-09-21T21:42:04.269 INFO:teuthology.orchestra.run.smithi164.stdout:4326 trunc from 0x58fc200 to 0x6460e00 2024-09-21T21:42:04.287 INFO:teuthology.orchestra.run.smithi164.stdout:4329 read 0x3d0fc00 thru 0x3d13c28 (0x4029 bytes) 2024-09-21T21:42:04.288 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 4, 5, 3] out_osds: [0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-21T21:42:04.288 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T21:42:04.288 INFO:tasks.thrashosds.thrasher:reset_purged_snaps_last 2024-09-21T21:42:04.288 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 reset_purged_snaps_last 2024-09-21T21:42:04.293 INFO:teuthology.orchestra.run.smithi164.stdout:4330 read 0xd3f600 thru 0xd40bfd (0x15fe bytes) 2024-09-21T21:42:04.307 INFO:teuthology.orchestra.run.smithi164.stdout:4332 write 0x6abaa00 thru 0x6ac0f36 (0x6537 bytes) 2024-09-21T21:42:04.309 INFO:teuthology.orchestra.run.smithi164.stdout:4336 trunc from 0x6ac1000 to 0x634f000 2024-09-21T21:42:04.313 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0: {} 2024-09-21T21:42:04.313 INFO:teuthology.orchestra.run.smithi081.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T21:42:04.331 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1: {} 2024-09-21T21:42:04.331 INFO:teuthology.orchestra.run.smithi081.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T21:42:04.333 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:04.333 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:04.344 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T21:42:04.347 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2: {} 2024-09-21T21:42:04.348 INFO:teuthology.orchestra.run.smithi081.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T21:42:04.361 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:04.359+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:04.366 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3: {} 2024-09-21T21:42:04.366 INFO:teuthology.orchestra.run.smithi081.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T21:42:04.383 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4: {} 2024-09-21T21:42:04.383 INFO:teuthology.orchestra.run.smithi081.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T21:42:04.396 INFO:teuthology.orchestra.run.smithi164.stdout:4337 read 0x21c9200 thru 0x21d41c4 (0xafc5 bytes) 2024-09-21T21:42:04.397 INFO:teuthology.orchestra.run.smithi164.stdout:4339 punch from 0x1760e00 to 0x1764800, (0x3a00 bytes) 2024-09-21T21:42:04.399 INFO:teuthology.orchestra.run.smithi164.stdout:4343 read 0xc4d600 thru 0xc4dc0e (0x60f bytes) 2024-09-21T21:42:04.402 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5: {} 2024-09-21T21:42:04.403 INFO:teuthology.orchestra.run.smithi081.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T21:42:04.419 INFO:teuthology.orchestra.run.smithi164.stdout:4344 trunc from 0x634f000 to 0xa6e200 2024-09-21T21:42:04.427 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 reset_purged_snaps_last 2024-09-21T21:42:04.462 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:04.460+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:04.519 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:04.519 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-21T21:42:04.519 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-21T21:42:04.519 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:42:04.519 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.520 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19744 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6471680~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:04.520 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.272996+0000", 2024-09-21T21:42:04.520 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.243832408999999, 2024-09-21T21:42:04.520 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97298795400000004, 2024-09-21T21:42:04.520 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:04.520 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:04.520 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:04.520 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:04.520 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:04.520 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:04.520 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19744 2024-09-21T21:42:04.520 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.520 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:04.521 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.521 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:04.521 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.272996+0000", 2024-09-21T21:42:04.521 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:04.521 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.521 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.521 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:04.521 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.272996+0000", 2024-09-21T21:42:04.521 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:04.521 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.521 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.521 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:04.521 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.272996+0000", 2024-09-21T21:42:04.521 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.53e-07 2024-09-21T21:42:04.522 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.522 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.522 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:04.522 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.272998+0000", 2024-09-21T21:42:04.522 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.888e-06 2024-09-21T21:42:04.522 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.522 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.522 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:04.522 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273005+0000", 2024-09-21T21:42:04.522 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4350000000000002e-06 2024-09-21T21:42:04.522 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.522 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.522 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:04.522 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273009+0000", 2024-09-21T21:42:04.522 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3239999999999996e-06 2024-09-21T21:42:04.523 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.523 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.523 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:04.523 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277855+0000", 2024-09-21T21:42:04.523 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048463849999999999 2024-09-21T21:42:04.523 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.523 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.523 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:04.523 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277858+0000", 2024-09-21T21:42:04.523 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9490000000000001e-06 2024-09-21T21:42:04.523 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.523 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.524 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:04.524 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.245879+0000", 2024-09-21T21:42:04.524 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96802068399999996 2024-09-21T21:42:04.524 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.524 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.524 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:04.524 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.245943+0000", 2024-09-21T21:42:04.524 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3690999999999997e-05 2024-09-21T21:42:04.524 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.524 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.524 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:04.524 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.245984+0000", 2024-09-21T21:42:04.524 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1245000000000002e-05 2024-09-21T21:42:04.524 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:04.525 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:04.525 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:04.525 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.525 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.525 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19745 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 7876608~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:04.525 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273015+0000", 2024-09-21T21:42:04.525 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.24381376, 2024-09-21T21:42:04.525 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97312649200000001, 2024-09-21T21:42:04.525 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:04.525 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:04.525 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:04.525 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:04.525 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:04.526 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:04.526 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19745 2024-09-21T21:42:04.526 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.526 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:04.526 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.526 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:04.526 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273015+0000", 2024-09-21T21:42:04.526 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:04.526 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.526 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.526 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:04.526 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273015+0000", 2024-09-21T21:42:04.526 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:04.527 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.527 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.527 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:04.527 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273015+0000", 2024-09-21T21:42:04.527 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2300000000000002e-07 2024-09-21T21:42:04.527 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.527 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.527 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:04.527 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273017+0000", 2024-09-21T21:42:04.527 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0480000000000001e-06 2024-09-21T21:42:04.527 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.527 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.527 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:04.528 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273023+0000", 2024-09-21T21:42:04.528 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.699e-06 2024-09-21T21:42:04.528 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.528 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.528 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:04.528 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273027+0000", 2024-09-21T21:42:04.528 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6689999999999997e-06 2024-09-21T21:42:04.528 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.528 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.528 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:04.528 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277895+0000", 2024-09-21T21:42:04.528 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048675039999999999 2024-09-21T21:42:04.528 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.529 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.529 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:04.529 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277899+0000", 2024-09-21T21:42:04.529 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5519999999999998e-06 2024-09-21T21:42:04.529 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.529 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.529 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:04.529 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246018+0000", 2024-09-21T21:42:04.529 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96811853599999997 2024-09-21T21:42:04.529 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.529 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.529 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:04.529 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246090+0000", 2024-09-21T21:42:04.529 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2502999999999993e-05 2024-09-21T21:42:04.530 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.530 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.530 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:04.530 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246141+0000", 2024-09-21T21:42:04.530 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0658000000000002e-05 2024-09-21T21:42:04.530 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:04.530 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:04.530 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:04.530 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.530 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.530 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19746 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 7188480~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:04.530 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273033+0000", 2024-09-21T21:42:04.530 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.243795177000001, 2024-09-21T21:42:04.530 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97319625399999998, 2024-09-21T21:42:04.530 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:04.531 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:04.531 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:04.531 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:04.531 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:04.531 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:04.531 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19746 2024-09-21T21:42:04.531 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.531 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:04.531 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.531 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:04.531 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273033+0000", 2024-09-21T21:42:04.531 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:04.531 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.532 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.532 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:04.532 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273033+0000", 2024-09-21T21:42:04.532 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:04.532 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.532 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.532 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:04.532 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273034+0000", 2024-09-21T21:42:04.532 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2100000000000002e-07 2024-09-21T21:42:04.532 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.532 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.532 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:04.532 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273036+0000", 2024-09-21T21:42:04.532 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1919999999999999e-06 2024-09-21T21:42:04.532 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.533 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.533 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:04.533 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273044+0000", 2024-09-21T21:42:04.533 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2369999999999992e-06 2024-09-21T21:42:04.533 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.533 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.533 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:04.533 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273048+0000", 2024-09-21T21:42:04.533 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5090000000000004e-06 2024-09-21T21:42:04.533 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.533 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.533 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:04.533 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277921+0000", 2024-09-21T21:42:04.533 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048723710000000003 2024-09-21T21:42:04.533 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.534 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.534 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:04.534 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277924+0000", 2024-09-21T21:42:04.534 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6509999999999999e-06 2024-09-21T21:42:04.534 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.534 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.534 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:04.534 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246153+0000", 2024-09-21T21:42:04.534 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96822869899999997 2024-09-21T21:42:04.534 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.534 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.534 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:04.534 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246192+0000", 2024-09-21T21:42:04.534 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8747e-05 2024-09-21T21:42:04.535 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.535 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.535 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:04.535 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246229+0000", 2024-09-21T21:42:04.535 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7426999999999999e-05 2024-09-21T21:42:04.535 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:04.535 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:04.535 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:04.535 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.535 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.535 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19747 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 7905280~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:04.535 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273058+0000", 2024-09-21T21:42:04.535 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.243770455, 2024-09-21T21:42:04.535 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97338591399999996, 2024-09-21T21:42:04.536 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:04.536 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:04.536 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:04.536 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:04.536 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:04.536 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:04.536 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19747 2024-09-21T21:42:04.536 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.536 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:04.536 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.536 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:04.536 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273058+0000", 2024-09-21T21:42:04.536 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:04.537 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.537 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.537 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:04.537 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273058+0000", 2024-09-21T21:42:04.537 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:04.537 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.537 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.537 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:04.537 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273058+0000", 2024-09-21T21:42:04.537 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5900000000000003e-07 2024-09-21T21:42:04.537 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.537 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.537 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:04.538 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273061+0000", 2024-09-21T21:42:04.538 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3479999999999998e-06 2024-09-21T21:42:04.538 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.538 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.538 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:04.538 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273067+0000", 2024-09-21T21:42:04.538 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1469999999999998e-06 2024-09-21T21:42:04.538 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.538 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.538 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:04.538 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273071+0000", 2024-09-21T21:42:04.538 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7339999999999999e-06 2024-09-21T21:42:04.539 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.539 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.539 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:04.539 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277945+0000", 2024-09-21T21:42:04.539 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.004873474 2024-09-21T21:42:04.539 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.539 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.539 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:04.539 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277948+0000", 2024-09-21T21:42:04.539 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0299999999999998e-06 2024-09-21T21:42:04.539 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.539 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.539 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:04.539 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246269+0000", 2024-09-21T21:42:04.540 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96832060200000003 2024-09-21T21:42:04.540 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.540 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.540 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:04.540 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246362+0000", 2024-09-21T21:42:04.540 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3867000000000001e-05 2024-09-21T21:42:04.540 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.540 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.540 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:04.540 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246444+0000", 2024-09-21T21:42:04.540 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1353000000000005e-05 2024-09-21T21:42:04.540 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:04.540 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:04.541 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:04.541 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.541 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.541 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19755 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 5783552~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:04.541 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273077+0000", 2024-09-21T21:42:04.541 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.243750839, 2024-09-21T21:42:04.541 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97350203400000002, 2024-09-21T21:42:04.541 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:04.541 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:04.541 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:04.541 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:04.541 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:04.542 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:04.542 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19755 2024-09-21T21:42:04.542 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.542 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:04.542 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.542 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:04.542 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273077+0000", 2024-09-21T21:42:04.542 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:04.542 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.542 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.543 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:04.543 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273077+0000", 2024-09-21T21:42:04.543 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:04.543 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.543 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.543 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:04.543 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273078+0000", 2024-09-21T21:42:04.543 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5699999999999998e-07 2024-09-21T21:42:04.543 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.543 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.543 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:04.544 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273083+0000", 2024-09-21T21:42:04.544 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8670000000000002e-06 2024-09-21T21:42:04.544 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.544 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.544 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:04.544 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273088+0000", 2024-09-21T21:42:04.544 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8370000000000002e-06 2024-09-21T21:42:04.544 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.544 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.544 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:04.544 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273093+0000", 2024-09-21T21:42:04.544 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5709999999999999e-06 2024-09-21T21:42:04.544 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.545 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.545 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:04.545 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277968+0000", 2024-09-21T21:42:04.545 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048753909999999998 2024-09-21T21:42:04.545 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.545 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.545 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:04.545 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277971+0000", 2024-09-21T21:42:04.545 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0189999999999998e-06 2024-09-21T21:42:04.545 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.545 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.545 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:04.545 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246485+0000", 2024-09-21T21:42:04.546 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96851399299999996 2024-09-21T21:42:04.546 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.546 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.546 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:04.546 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246536+0000", 2024-09-21T21:42:04.546 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0430000000000003e-05 2024-09-21T21:42:04.546 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.546 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.546 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:04.546 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246579+0000", 2024-09-21T21:42:04.546 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3569000000000002e-05 2024-09-21T21:42:04.546 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:04.546 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:04.546 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:04.546 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.547 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.547 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19756 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6500352~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:04.547 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273099+0000", 2024-09-21T21:42:04.547 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.243729466, 2024-09-21T21:42:04.547 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97359272799999996, 2024-09-21T21:42:04.547 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:04.547 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:04.547 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:04.547 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:04.547 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:04.547 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:04.547 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19756 2024-09-21T21:42:04.547 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.547 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:04.547 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.548 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:04.548 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273099+0000", 2024-09-21T21:42:04.548 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:04.548 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.548 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.548 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:04.548 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273099+0000", 2024-09-21T21:42:04.548 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:04.548 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.548 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.548 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:04.548 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273099+0000", 2024-09-21T21:42:04.548 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0900000000000002e-07 2024-09-21T21:42:04.548 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.548 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.549 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:04.549 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273101+0000", 2024-09-21T21:42:04.549 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0320000000000002e-06 2024-09-21T21:42:04.549 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.549 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.549 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:04.549 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273107+0000", 2024-09-21T21:42:04.549 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2830000000000001e-06 2024-09-21T21:42:04.549 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.549 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.549 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:04.549 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273111+0000", 2024-09-21T21:42:04.549 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2230000000000001e-06 2024-09-21T21:42:04.549 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.549 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.550 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:04.550 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277994+0000", 2024-09-21T21:42:04.550 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048837070000000002 2024-09-21T21:42:04.550 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.550 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.550 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:04.550 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278000+0000", 2024-09-21T21:42:04.550 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3149999999999999e-06 2024-09-21T21:42:04.550 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.550 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.550 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:04.550 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246607+0000", 2024-09-21T21:42:04.550 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.968606839 2024-09-21T21:42:04.550 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.551 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.551 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:04.551 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246651+0000", 2024-09-21T21:42:04.551 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4734999999999998e-05 2024-09-21T21:42:04.551 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.551 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.551 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:04.551 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246692+0000", 2024-09-21T21:42:04.551 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0185000000000002e-05 2024-09-21T21:42:04.551 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:04.551 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:04.551 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:04.551 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.552 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.552 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19757 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 5812224~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:04.552 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273116+0000", 2024-09-21T21:42:04.552 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.243711809000001, 2024-09-21T21:42:04.552 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97367813999999997, 2024-09-21T21:42:04.552 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:04.552 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:04.552 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:04.552 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:04.552 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:04.552 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:04.552 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19757 2024-09-21T21:42:04.552 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.552 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:04.553 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.553 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:04.553 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273116+0000", 2024-09-21T21:42:04.553 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:04.553 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.553 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.553 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:04.553 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273116+0000", 2024-09-21T21:42:04.553 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:04.553 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.553 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.553 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:04.554 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273117+0000", 2024-09-21T21:42:04.554 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0999999999999999e-07 2024-09-21T21:42:04.554 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.554 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.554 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:04.554 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273119+0000", 2024-09-21T21:42:04.554 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9970000000000001e-06 2024-09-21T21:42:04.554 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.554 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.554 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:04.554 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273125+0000", 2024-09-21T21:42:04.554 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9540000000000001e-06 2024-09-21T21:42:04.554 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.554 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.555 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:04.555 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273135+0000", 2024-09-21T21:42:04.555 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0597999999999999e-05 2024-09-21T21:42:04.555 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.555 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.555 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:04.555 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278019+0000", 2024-09-21T21:42:04.555 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048834849999999999 2024-09-21T21:42:04.555 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.555 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.555 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:04.555 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278022+0000", 2024-09-21T21:42:04.555 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9560000000000002e-06 2024-09-21T21:42:04.555 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.555 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.556 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:04.556 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246716+0000", 2024-09-21T21:42:04.556 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.968694202 2024-09-21T21:42:04.556 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.556 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.556 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:04.556 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246758+0000", 2024-09-21T21:42:04.556 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1607000000000003e-05 2024-09-21T21:42:04.556 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.556 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.556 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:04.556 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246795+0000", 2024-09-21T21:42:04.556 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6931000000000003e-05 2024-09-21T21:42:04.556 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:04.556 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:04.557 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:04.557 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.557 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.557 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19758 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6529024~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:04.557 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273142+0000", 2024-09-21T21:42:04.557 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.243686587999999, 2024-09-21T21:42:04.557 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97375924599999997, 2024-09-21T21:42:04.557 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:04.557 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:04.557 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:04.557 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:04.557 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:04.557 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:04.557 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19758 2024-09-21T21:42:04.558 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.558 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:04.558 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.558 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:04.558 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273142+0000", 2024-09-21T21:42:04.558 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:04.558 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.558 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.558 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:04.558 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273142+0000", 2024-09-21T21:42:04.558 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:04.558 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.558 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.558 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:04.559 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273142+0000", 2024-09-21T21:42:04.559 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0200000000000003e-07 2024-09-21T21:42:04.559 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.559 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.559 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:04.559 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273144+0000", 2024-09-21T21:42:04.559 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1390000000000002e-06 2024-09-21T21:42:04.559 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.559 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.559 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:04.559 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273150+0000", 2024-09-21T21:42:04.559 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6409999999999999e-06 2024-09-21T21:42:04.559 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.559 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.560 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:04.560 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273157+0000", 2024-09-21T21:42:04.560 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1359999999999999e-06 2024-09-21T21:42:04.560 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.560 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.560 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:04.560 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278050+0000", 2024-09-21T21:42:04.560 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048926170000000001 2024-09-21T21:42:04.560 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.560 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.560 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:04.560 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278053+0000", 2024-09-21T21:42:04.560 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4879999999999999e-06 2024-09-21T21:42:04.560 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.561 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.561 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:04.561 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246819+0000", 2024-09-21T21:42:04.561 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96876590399999996 2024-09-21T21:42:04.561 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.561 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.561 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:04.561 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246856+0000", 2024-09-21T21:42:04.561 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7287999999999999e-05 2024-09-21T21:42:04.561 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.561 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.561 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:04.561 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246901+0000", 2024-09-21T21:42:04.562 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4631e-05 2024-09-21T21:42:04.562 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:04.562 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:04.562 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:04.562 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.562 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.562 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19759 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 7933952~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:04.562 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273172+0000", 2024-09-21T21:42:04.562 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.243656324, 2024-09-21T21:42:04.562 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97384789500000002, 2024-09-21T21:42:04.562 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:04.562 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:04.562 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:04.562 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:04.563 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:04.563 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:04.563 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19759 2024-09-21T21:42:04.563 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.563 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:04.563 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.563 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:04.563 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273172+0000", 2024-09-21T21:42:04.563 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:04.563 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.563 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.563 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:04.563 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273172+0000", 2024-09-21T21:42:04.563 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:04.563 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.564 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.564 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:04.564 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273172+0000", 2024-09-21T21:42:04.564 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6100000000000002e-07 2024-09-21T21:42:04.564 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.564 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.564 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:04.564 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273174+0000", 2024-09-21T21:42:04.564 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.691e-06 2024-09-21T21:42:04.564 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.564 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.564 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:04.564 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273178+0000", 2024-09-21T21:42:04.564 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7579999999999998e-06 2024-09-21T21:42:04.564 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.565 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.565 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:04.565 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273180+0000", 2024-09-21T21:42:04.565 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7199999999999998e-06 2024-09-21T21:42:04.565 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.565 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.565 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:04.565 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278077+0000", 2024-09-21T21:42:04.565 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048969529999999999 2024-09-21T21:42:04.565 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.565 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.565 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:04.565 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278080+0000", 2024-09-21T21:42:04.565 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.066e-06 2024-09-21T21:42:04.565 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.566 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.566 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:04.566 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246925+0000", 2024-09-21T21:42:04.566 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96884490199999995 2024-09-21T21:42:04.566 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.566 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.566 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:04.566 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246974+0000", 2024-09-21T21:42:04.566 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8387999999999998e-05 2024-09-21T21:42:04.566 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.566 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.566 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:04.566 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247020+0000", 2024-09-21T21:42:04.566 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6156000000000003e-05 2024-09-21T21:42:04.567 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:04.567 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:04.567 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:04.567 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.567 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.567 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19760 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 7991296~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:04.567 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273220+0000", 2024-09-21T21:42:04.567 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.24360781, 2024-09-21T21:42:04.567 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97403125499999998, 2024-09-21T21:42:04.567 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:04.567 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:04.567 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:04.567 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:04.567 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:04.567 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:04.568 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19760 2024-09-21T21:42:04.568 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.568 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:04.568 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.568 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:04.568 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273220+0000", 2024-09-21T21:42:04.568 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:04.568 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.568 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.568 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:04.568 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273220+0000", 2024-09-21T21:42:04.568 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:04.568 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.568 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.568 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:04.569 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273221+0000", 2024-09-21T21:42:04.569 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.65e-07 2024-09-21T21:42:04.569 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.569 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.569 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:04.569 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273228+0000", 2024-09-21T21:42:04.569 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6040000000000004e-06 2024-09-21T21:42:04.569 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.569 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.569 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:04.569 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273233+0000", 2024-09-21T21:42:04.569 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1569999999999999e-06 2024-09-21T21:42:04.569 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.569 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.570 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:04.570 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273238+0000", 2024-09-21T21:42:04.570 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0050000000000004e-06 2024-09-21T21:42:04.570 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.570 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.570 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:04.570 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278107+0000", 2024-09-21T21:42:04.570 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00486857 2024-09-21T21:42:04.570 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.570 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.570 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:04.570 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278111+0000", 2024-09-21T21:42:04.570 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6389999999999998e-06 2024-09-21T21:42:04.570 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.571 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.571 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:04.571 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247069+0000", 2024-09-21T21:42:04.571 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96895796899999997 2024-09-21T21:42:04.571 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.571 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.571 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:04.571 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247166+0000", 2024-09-21T21:42:04.571 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.7158999999999996e-05 2024-09-21T21:42:04.571 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.571 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.571 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:04.571 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247252+0000", 2024-09-21T21:42:04.571 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5886999999999999e-05 2024-09-21T21:42:04.571 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:04.572 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:04.572 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:04.572 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.572 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.572 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19761 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 7962624~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:04.572 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273246+0000", 2024-09-21T21:42:04.572 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.24358275, 2024-09-21T21:42:04.572 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97415972200000001, 2024-09-21T21:42:04.572 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:04.572 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:04.572 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:04.572 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:04.572 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:04.572 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:04.573 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19761 2024-09-21T21:42:04.573 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.573 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:04.573 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.573 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:04.573 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273246+0000", 2024-09-21T21:42:04.573 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:04.573 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.573 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.573 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:04.573 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273246+0000", 2024-09-21T21:42:04.573 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:04.573 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.573 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.573 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:04.573 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273246+0000", 2024-09-21T21:42:04.574 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5100000000000001e-07 2024-09-21T21:42:04.574 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.574 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.574 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:04.574 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273248+0000", 2024-09-21T21:42:04.574 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1550000000000001e-06 2024-09-21T21:42:04.574 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.574 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.574 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:04.574 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273254+0000", 2024-09-21T21:42:04.574 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.964e-06 2024-09-21T21:42:04.574 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.574 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.574 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:04.575 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273259+0000", 2024-09-21T21:42:04.575 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5499999999999996e-06 2024-09-21T21:42:04.575 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.575 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.575 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:04.575 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278130+0000", 2024-09-21T21:42:04.575 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048715779999999997 2024-09-21T21:42:04.575 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.575 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.575 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:04.575 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278133+0000", 2024-09-21T21:42:04.575 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8739999999999999e-06 2024-09-21T21:42:04.575 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.575 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.576 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:04.576 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247270+0000", 2024-09-21T21:42:04.576 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.969137099 2024-09-21T21:42:04.576 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.576 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.576 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:04.576 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247338+0000", 2024-09-21T21:42:04.576 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7717000000000002e-05 2024-09-21T21:42:04.576 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.576 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.576 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:04.576 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247405+0000", 2024-09-21T21:42:04.576 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7434000000000005e-05 2024-09-21T21:42:04.576 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:04.576 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:04.577 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:04.577 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.577 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.577 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19769 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 5840896~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:04.577 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273287+0000", 2024-09-21T21:42:04.577 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.243541466, 2024-09-21T21:42:04.577 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97435390200000005, 2024-09-21T21:42:04.577 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:04.577 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:04.577 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:04.577 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:04.577 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:04.577 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:04.577 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19769 2024-09-21T21:42:04.578 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.578 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:04.578 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.578 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:04.578 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273287+0000", 2024-09-21T21:42:04.578 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:04.578 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.578 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.578 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:04.578 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273287+0000", 2024-09-21T21:42:04.578 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:04.578 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.578 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.578 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:04.578 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273287+0000", 2024-09-21T21:42:04.579 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6699999999999999e-07 2024-09-21T21:42:04.579 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.579 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.579 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:04.579 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273289+0000", 2024-09-21T21:42:04.579 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.708e-06 2024-09-21T21:42:04.579 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.579 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.579 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:04.579 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273296+0000", 2024-09-21T21:42:04.579 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8109999999999997e-06 2024-09-21T21:42:04.579 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.579 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.579 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:04.579 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273299+0000", 2024-09-21T21:42:04.580 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0740000000000001e-06 2024-09-21T21:42:04.580 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.580 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.580 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:04.580 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278159+0000", 2024-09-21T21:42:04.580 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048603240000000001 2024-09-21T21:42:04.580 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.580 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.580 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:04.580 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278162+0000", 2024-09-21T21:42:04.580 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1930000000000002e-06 2024-09-21T21:42:04.580 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.580 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.580 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:04.580 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247444+0000", 2024-09-21T21:42:04.581 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96928199100000001 2024-09-21T21:42:04.581 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.581 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.581 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:04.581 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247533+0000", 2024-09-21T21:42:04.581 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8490000000000001e-05 2024-09-21T21:42:04.581 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.581 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.581 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:04.581 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247641+0000", 2024-09-21T21:42:04.581 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000107944 2024-09-21T21:42:04.581 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:04.581 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:04.581 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:04.582 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.582 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.582 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19770 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 5869568~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:04.582 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273313+0000", 2024-09-21T21:42:04.582 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.243515364, 2024-09-21T21:42:04.582 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97449038899999996, 2024-09-21T21:42:04.582 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:04.582 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:04.582 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:04.582 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:04.582 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:04.582 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:04.582 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19770 2024-09-21T21:42:04.582 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.583 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:04.583 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.583 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:04.583 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273313+0000", 2024-09-21T21:42:04.583 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:04.583 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.583 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.583 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:04.583 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273313+0000", 2024-09-21T21:42:04.583 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:04.583 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.583 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.583 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:04.583 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273313+0000", 2024-09-21T21:42:04.583 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5699999999999998e-07 2024-09-21T21:42:04.584 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.584 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.584 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:04.584 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273315+0000", 2024-09-21T21:42:04.584 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0099999999999998e-06 2024-09-21T21:42:04.584 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.584 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.584 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:04.584 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273321+0000", 2024-09-21T21:42:04.584 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7529999999999998e-06 2024-09-21T21:42:04.584 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.584 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.584 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:04.584 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273328+0000", 2024-09-21T21:42:04.585 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9009999999999997e-06 2024-09-21T21:42:04.585 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.585 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.585 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:04.585 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278183+0000", 2024-09-21T21:42:04.585 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048547770000000002 2024-09-21T21:42:04.585 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.585 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.585 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:04.585 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278186+0000", 2024-09-21T21:42:04.585 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3100000000000001e-06 2024-09-21T21:42:04.585 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.585 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.585 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:04.585 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247676+0000", 2024-09-21T21:42:04.585 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96949004299999997 2024-09-21T21:42:04.586 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.586 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.586 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:04.586 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247757+0000", 2024-09-21T21:42:04.586 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0479e-05 2024-09-21T21:42:04.586 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.586 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.586 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:04.586 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247803+0000", 2024-09-21T21:42:04.586 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6758999999999997e-05 2024-09-21T21:42:04.586 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:04.586 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:04.586 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:04.586 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.587 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.587 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19771 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6586368~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:04.587 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273335+0000", 2024-09-21T21:42:04.587 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.243493577000001, 2024-09-21T21:42:04.587 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97458947699999998, 2024-09-21T21:42:04.587 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:04.587 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:04.587 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:04.587 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:04.587 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:04.587 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:04.587 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19771 2024-09-21T21:42:04.587 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.587 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:04.587 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.588 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:04.588 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273335+0000", 2024-09-21T21:42:04.588 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:04.588 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.588 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.588 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:04.588 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273335+0000", 2024-09-21T21:42:04.588 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:04.588 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.588 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.588 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:04.588 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273335+0000", 2024-09-21T21:42:04.588 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3599999999999999e-07 2024-09-21T21:42:04.588 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.588 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.589 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:04.589 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273337+0000", 2024-09-21T21:42:04.589 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0789999999999999e-06 2024-09-21T21:42:04.589 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.589 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.589 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:04.589 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273343+0000", 2024-09-21T21:42:04.589 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6280000000000002e-06 2024-09-21T21:42:04.589 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.589 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.589 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:04.589 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273346+0000", 2024-09-21T21:42:04.589 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.507e-06 2024-09-21T21:42:04.589 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.589 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.590 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:04.590 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278206+0000", 2024-09-21T21:42:04.590 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048600969999999999 2024-09-21T21:42:04.590 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.590 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.590 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:04.590 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278212+0000", 2024-09-21T21:42:04.590 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2079999999999999e-06 2024-09-21T21:42:04.590 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.590 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.590 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:04.590 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247828+0000", 2024-09-21T21:42:04.590 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.969616547 2024-09-21T21:42:04.590 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.591 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.591 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:04.591 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247881+0000", 2024-09-21T21:42:04.591 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3180999999999998e-05 2024-09-21T21:42:04.591 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.591 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.591 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:04.591 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247924+0000", 2024-09-21T21:42:04.591 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2894e-05 2024-09-21T21:42:04.591 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:04.591 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:04.591 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:04.591 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.591 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.591 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19772 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6557696~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:04.592 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273358+0000", 2024-09-21T21:42:04.592 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.243470197000001, 2024-09-21T21:42:04.592 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97467031800000004, 2024-09-21T21:42:04.592 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:04.592 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:04.592 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:04.592 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:04.592 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:04.592 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:04.592 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19772 2024-09-21T21:42:04.592 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.592 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:04.592 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.592 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:04.593 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273358+0000", 2024-09-21T21:42:04.593 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:04.593 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.593 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.593 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:04.593 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273358+0000", 2024-09-21T21:42:04.593 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:04.593 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.593 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.593 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:04.593 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273358+0000", 2024-09-21T21:42:04.593 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4499999999999998e-07 2024-09-21T21:42:04.593 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.593 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.593 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:04.594 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273360+0000", 2024-09-21T21:42:04.594 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9190000000000002e-06 2024-09-21T21:42:04.594 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.594 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.594 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:04.594 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273366+0000", 2024-09-21T21:42:04.594 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.891e-06 2024-09-21T21:42:04.594 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.594 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.594 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:04.594 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273371+0000", 2024-09-21T21:42:04.594 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1880000000000001e-06 2024-09-21T21:42:04.594 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.594 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.595 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:04.595 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278237+0000", 2024-09-21T21:42:04.595 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.004865472 2024-09-21T21:42:04.595 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.595 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.595 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:04.595 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278240+0000", 2024-09-21T21:42:04.595 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9550000000000001e-06 2024-09-21T21:42:04.595 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.595 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.595 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:04.595 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247949+0000", 2024-09-21T21:42:04.595 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.969709127 2024-09-21T21:42:04.595 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.595 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.596 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:04.596 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247990+0000", 2024-09-21T21:42:04.596 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1202000000000003e-05 2024-09-21T21:42:04.596 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.596 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.596 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:04.596 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248028+0000", 2024-09-21T21:42:04.596 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8219000000000001e-05 2024-09-21T21:42:04.596 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:04.596 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:04.596 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:04.596 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.596 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.596 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19773 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6615040~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:04.597 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273390+0000", 2024-09-21T21:42:04.597 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.243438336000001, 2024-09-21T21:42:04.597 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97476086200000001, 2024-09-21T21:42:04.597 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:04.597 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:04.597 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:04.597 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:04.597 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:04.597 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:04.597 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19773 2024-09-21T21:42:04.597 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.597 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:04.597 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.597 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:04.597 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273390+0000", 2024-09-21T21:42:04.598 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:04.598 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.598 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.598 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:04.598 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273390+0000", 2024-09-21T21:42:04.598 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:04.598 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.598 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.598 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:04.598 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273390+0000", 2024-09-21T21:42:04.598 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9999999999999999e-07 2024-09-21T21:42:04.598 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.598 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.598 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:04.599 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273392+0000", 2024-09-21T21:42:04.599 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0640000000000001e-06 2024-09-21T21:42:04.599 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.599 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.599 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:04.599 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273397+0000", 2024-09-21T21:42:04.599 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.113e-06 2024-09-21T21:42:04.599 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.599 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.599 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:04.599 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273401+0000", 2024-09-21T21:42:04.599 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3119999999999998e-06 2024-09-21T21:42:04.599 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.599 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.599 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:04.600 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278271+0000", 2024-09-21T21:42:04.600 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048698880000000002 2024-09-21T21:42:04.600 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.600 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.600 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:04.600 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278278+0000", 2024-09-21T21:42:04.600 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1210000000000001e-06 2024-09-21T21:42:04.600 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.600 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.600 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:04.600 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248060+0000", 2024-09-21T21:42:04.600 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96978224999999996 2024-09-21T21:42:04.600 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.600 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.600 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:04.601 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248111+0000", 2024-09-21T21:42:04.601 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.096e-05 2024-09-21T21:42:04.601 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.601 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.601 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:04.601 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248151+0000", 2024-09-21T21:42:04.601 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9854000000000002e-05 2024-09-21T21:42:04.601 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:04.601 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:04.601 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:04.601 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.601 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.601 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19784 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6643712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:04.601 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273405+0000", 2024-09-21T21:42:04.601 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.243423614999999, 2024-09-21T21:42:04.602 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97491798900000004, 2024-09-21T21:42:04.602 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:04.602 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:04.602 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:04.602 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:04.602 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:04.602 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:04.602 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19784 2024-09-21T21:42:04.602 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.602 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:04.602 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.602 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:04.602 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273405+0000", 2024-09-21T21:42:04.603 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:04.603 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.603 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.603 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:04.603 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273405+0000", 2024-09-21T21:42:04.603 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:04.603 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.603 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.603 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:04.603 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273405+0000", 2024-09-21T21:42:04.603 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2100000000000001e-07 2024-09-21T21:42:04.603 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.603 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.603 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:04.604 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273406+0000", 2024-09-21T21:42:04.604 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2049999999999999e-06 2024-09-21T21:42:04.604 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.604 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.604 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:04.604 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273409+0000", 2024-09-21T21:42:04.604 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0599999999999999e-06 2024-09-21T21:42:04.604 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.604 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.604 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:04.604 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273412+0000", 2024-09-21T21:42:04.604 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0730000000000001e-06 2024-09-21T21:42:04.604 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.605 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.605 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:04.605 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278317+0000", 2024-09-21T21:42:04.605 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0049045520000000004 2024-09-21T21:42:04.605 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.605 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.605 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:04.605 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278321+0000", 2024-09-21T21:42:04.605 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6249999999999998e-06 2024-09-21T21:42:04.605 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.605 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.605 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:04.605 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248179+0000", 2024-09-21T21:42:04.605 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96985744500000004 2024-09-21T21:42:04.606 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.606 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.606 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:04.606 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248253+0000", 2024-09-21T21:42:04.606 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3829000000000001e-05 2024-09-21T21:42:04.606 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.606 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.606 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:04.606 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248323+0000", 2024-09-21T21:42:04.606 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9979000000000002e-05 2024-09-21T21:42:04.606 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:04.607 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:04.607 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:04.607 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.607 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.607 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19785 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6672384~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:04.607 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273415+0000", 2024-09-21T21:42:04.607 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.243413028999999, 2024-09-21T21:42:04.607 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97500777800000005, 2024-09-21T21:42:04.607 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:04.607 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:04.607 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:04.607 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:04.607 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:04.608 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:04.608 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19785 2024-09-21T21:42:04.608 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.608 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:04.608 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.608 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:04.608 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273415+0000", 2024-09-21T21:42:04.608 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:04.608 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.608 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.608 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:04.608 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273415+0000", 2024-09-21T21:42:04.608 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:04.609 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.609 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.609 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:04.609 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273415+0000", 2024-09-21T21:42:04.609 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0100000000000001e-07 2024-09-21T21:42:04.609 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.609 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.609 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:04.609 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273417+0000", 2024-09-21T21:42:04.609 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.114e-06 2024-09-21T21:42:04.609 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.609 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.609 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:04.610 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273420+0000", 2024-09-21T21:42:04.610 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6770000000000001e-06 2024-09-21T21:42:04.610 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.610 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.610 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:04.610 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273423+0000", 2024-09-21T21:42:04.610 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5450000000000002e-06 2024-09-21T21:42:04.610 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.610 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.610 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:04.610 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278355+0000", 2024-09-21T21:42:04.610 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0049318590000000002 2024-09-21T21:42:04.610 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.611 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.611 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:04.611 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278360+0000", 2024-09-21T21:42:04.611 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3079999999999998e-06 2024-09-21T21:42:04.611 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.611 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.611 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:04.611 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248349+0000", 2024-09-21T21:42:04.611 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96998909899999997 2024-09-21T21:42:04.611 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.611 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.611 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:04.611 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248387+0000", 2024-09-21T21:42:04.611 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7657000000000002e-05 2024-09-21T21:42:04.611 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.612 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.612 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:04.612 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248423+0000", 2024-09-21T21:42:04.612 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6318e-05 2024-09-21T21:42:04.612 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:04.612 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:04.612 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:04.612 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.612 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.612 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19842 3.d 3:b7fa2ca0:::rbd_data.112157a479c.000000000000004d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:04.612 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273426+0000", 2024-09-21T21:42:04.612 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.243401977, 2024-09-21T21:42:04.612 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.98136526800000001, 2024-09-21T21:42:04.612 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:04.612 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:04.613 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:04.613 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:04.613 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:04.613 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:04.613 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19842 2024-09-21T21:42:04.613 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.613 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:04.613 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.613 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:04.613 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273426+0000", 2024-09-21T21:42:04.613 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:04.613 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.613 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.613 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:04.614 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273426+0000", 2024-09-21T21:42:04.614 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:04.614 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.614 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.614 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:04.614 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273427+0000", 2024-09-21T21:42:04.614 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.72e-07 2024-09-21T21:42:04.614 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.614 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.614 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:04.614 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275836+0000", 2024-09-21T21:42:04.614 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00240909 2024-09-21T21:42:04.614 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.614 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.615 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:04.615 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275844+0000", 2024-09-21T21:42:04.615 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3750000000000003e-06 2024-09-21T21:42:04.615 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.615 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.615 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:04.615 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275849+0000", 2024-09-21T21:42:04.615 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0100000000000003e-06 2024-09-21T21:42:04.615 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.615 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.615 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:04.615 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278394+0000", 2024-09-21T21:42:04.615 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025449890000000001 2024-09-21T21:42:04.615 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.615 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.616 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:04.616 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278402+0000", 2024-09-21T21:42:04.616 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6359999999999994e-06 2024-09-21T21:42:04.616 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.616 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.616 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:04.616 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248454+0000", 2024-09-21T21:42:04.616 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97005259499999996 2024-09-21T21:42:04.616 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.616 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.616 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:04.616 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248576+0000", 2024-09-21T21:42:04.616 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000122127 2024-09-21T21:42:04.616 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.616 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.616 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 2", 2024-09-21T21:42:04.617 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248844+0000", 2024-09-21T21:42:04.617 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00026784699999999999 2024-09-21T21:42:04.617 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.617 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.617 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:04.617 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.252487+0000", 2024-09-21T21:42:04.617 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0036427920000000002 2024-09-21T21:42:04.617 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.617 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.617 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:04.617 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.254691+0000", 2024-09-21T21:42:04.617 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002203899 2024-09-21T21:42:04.617 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.617 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.617 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:04.618 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.254748+0000", 2024-09-21T21:42:04.618 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7163000000000001e-05 2024-09-21T21:42:04.618 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.618 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.618 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:04.618 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.254792+0000", 2024-09-21T21:42:04.618 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3473000000000002e-05 2024-09-21T21:42:04.618 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:04.618 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:04.618 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:04.619 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.619 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.619 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19843 3.d 3:b7fa2ca0:::rbd_data.112157a479c.000000000000004d:head [write 0~0] snapc 21=[21] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:04.619 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.275860+0000", 2024-09-21T21:42:04.619 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.240968077, 2024-09-21T21:42:04.619 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.98183651999999999, 2024-09-21T21:42:04.619 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:04.619 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:04.619 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:04.619 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:04.619 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:04.619 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:04.619 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19843 2024-09-21T21:42:04.619 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.620 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:04.620 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.620 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:04.620 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275860+0000", 2024-09-21T21:42:04.620 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:04.620 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.620 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.620 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:04.620 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275860+0000", 2024-09-21T21:42:04.620 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:04.620 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.620 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.620 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:04.620 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275869+0000", 2024-09-21T21:42:04.620 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6300000000000004e-06 2024-09-21T21:42:04.621 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.621 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.621 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:04.621 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275871+0000", 2024-09-21T21:42:04.621 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2060000000000001e-06 2024-09-21T21:42:04.621 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.621 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.621 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:04.621 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275876+0000", 2024-09-21T21:42:04.621 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8180000000000003e-06 2024-09-21T21:42:04.621 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.621 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.621 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:04.621 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275880+0000", 2024-09-21T21:42:04.622 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1829999999999996e-06 2024-09-21T21:42:04.622 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.622 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.622 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:04.622 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278416+0000", 2024-09-21T21:42:04.622 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025361149999999998 2024-09-21T21:42:04.622 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.622 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.622 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:04.622 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278419+0000", 2024-09-21T21:42:04.622 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.163e-06 2024-09-21T21:42:04.622 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.622 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.622 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:04.623 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.250249+0000", 2024-09-21T21:42:04.623 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97182984299999997 2024-09-21T21:42:04.623 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.623 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.623 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:04.623 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.250321+0000", 2024-09-21T21:42:04.623 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1656000000000006e-05 2024-09-21T21:42:04.623 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.623 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.623 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 2", 2024-09-21T21:42:04.623 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.250419+0000", 2024-09-21T21:42:04.623 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.8277999999999997e-05 2024-09-21T21:42:04.623 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.623 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.623 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:04.624 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.252516+0000", 2024-09-21T21:42:04.624 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002097224 2024-09-21T21:42:04.624 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.624 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.624 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:04.624 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.257643+0000", 2024-09-21T21:42:04.624 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0051262319999999997 2024-09-21T21:42:04.624 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.624 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.624 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:04.624 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.257673+0000", 2024-09-21T21:42:04.624 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0023e-05 2024-09-21T21:42:04.624 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:04.625 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:04.625 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:04.625 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.257697+0000", 2024-09-21T21:42:04.625 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4148999999999999e-05 2024-09-21T21:42:04.625 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:04.625 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:04.625 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:04.625 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:04.625 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:04.625 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:04.627 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 reset_purged_snaps_last 2024-09-21T21:42:04.628 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:04.625+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:04.628 INFO:teuthology.orchestra.run.smithi164.stdout:4345 trunc from 0xa6e200 to 0x49dfc00 2024-09-21T21:42:04.628 INFO:teuthology.orchestra.run.smithi164.stdout:4349 write 0xd702200 thru 0xd704bde (0x29df bytes) 2024-09-21T21:42:04.628 INFO:teuthology.orchestra.run.smithi164.stdout:4350 read 0x3f8ea00 thru 0x3f9c038 (0xd639 bytes) 2024-09-21T21:42:04.628 INFO:teuthology.orchestra.run.smithi164.stdout:4353 punch from 0x5fc4e00 to 0x5fc5e00, (0x1000 bytes) 2024-09-21T21:42:04.628 INFO:teuthology.orchestra.run.smithi164.stdout:4355 write 0xb8cfc00 thru 0xb8de214 (0xe615 bytes) 2024-09-21T21:42:04.628 INFO:teuthology.orchestra.run.smithi164.stdout:4356 read 0x5684600 thru 0x568bcab (0x76ac bytes) 2024-09-21T21:42:04.629 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T21:42:04.635 INFO:teuthology.orchestra.run.smithi164.stdout:4359 write 0xd924400 thru 0xd92cee9 (0x8aea bytes) 2024-09-21T21:42:04.638 INFO:teuthology.orchestra.run.smithi164.stdout:4360 write 0x72f5800 thru 0x72fc05d (0x685e bytes) 2024-09-21T21:42:04.638 INFO:teuthology.orchestra.run.smithi164.stdout:4362 write 0x8254600 thru 0x8256cd3 (0x26d4 bytes) 2024-09-21T21:42:04.639 INFO:teuthology.orchestra.run.smithi164.stdout:4364 write 0x60cda00 thru 0x60d0caf (0x32b0 bytes) 2024-09-21T21:42:04.639 INFO:teuthology.orchestra.run.smithi164.stdout:4366 read 0x2a54400 thru 0x2a61ca9 (0xd8aa bytes) 2024-09-21T21:42:04.639 INFO:teuthology.orchestra.run.smithi164.stdout:4367 punch from 0x3070600 to 0x307cc00, (0xc600 bytes) 2024-09-21T21:42:04.648 INFO:teuthology.orchestra.run.smithi164.stdout:4368 write 0x2551800 thru 0x255a29f (0x8aa0 bytes) 2024-09-21T21:42:04.649 INFO:teuthology.orchestra.run.smithi164.stdout:4370 clone 35 order 22 su 524288 sc 13 2024-09-21T21:42:04.727 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:04.725+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:04.799 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:04.800 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:42:04.800 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-21T21:42:04.800 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:04.804 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 reset_purged_snaps_last 2024-09-21T21:42:04.810 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T21:42:04.853 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:04.852+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:04.912 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:04.912 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:42:04.912 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:42:04.912 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:42:04.913 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:04.921 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T21:42:04.929 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:04.928+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:04.977 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 reset_purged_snaps_last 2024-09-21T21:42:05.030 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:05.028+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:05.065 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:05.065 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-21T21:42:05.065 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-21T21:42:05.065 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:42:05.065 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.065 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6974 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5246976~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:05.065 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.922932+0000", 2024-09-21T21:42:05.066 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 35.140085870999997, 2024-09-21T21:42:05.066 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041773352999999999, 2024-09-21T21:42:05.066 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:05.066 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:05.066 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:05.066 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:05.066 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:05.066 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:05.066 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6974 2024-09-21T21:42:05.066 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.066 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:05.066 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.067 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:05.067 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922932+0000", 2024-09-21T21:42:05.067 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:05.067 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.067 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.067 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:05.067 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922932+0000", 2024-09-21T21:42:05.067 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:05.067 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.067 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.067 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:05.067 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922933+0000", 2024-09-21T21:42:05.068 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.46e-07 2024-09-21T21:42:05.068 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.068 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.068 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:05.068 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922940+0000", 2024-09-21T21:42:05.068 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6140000000000003e-06 2024-09-21T21:42:05.068 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.068 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.068 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:05.068 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922949+0000", 2024-09-21T21:42:05.068 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6899999999999998e-06 2024-09-21T21:42:05.068 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.069 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.069 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:05.069 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922955+0000", 2024-09-21T21:42:05.069 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5140000000000001e-06 2024-09-21T21:42:05.069 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.069 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.069 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:05.069 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964547+0000", 2024-09-21T21:42:05.069 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041592854999999998 2024-09-21T21:42:05.069 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.069 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.069 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:05.069 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964617+0000", 2024-09-21T21:42:05.070 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9058000000000002e-05 2024-09-21T21:42:05.070 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.070 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.070 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:05.070 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964706+0000", 2024-09-21T21:42:05.070 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9276000000000004e-05 2024-09-21T21:42:05.070 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:05.070 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:05.070 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:05.070 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.070 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.070 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6976 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5261312~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:05.071 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.922961+0000", 2024-09-21T21:42:05.071 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 35.140056825999999, 2024-09-21T21:42:05.071 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041863648000000003, 2024-09-21T21:42:05.071 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:05.071 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:05.071 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:05.071 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:05.071 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:05.071 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:05.071 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6976 2024-09-21T21:42:05.071 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.072 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:05.072 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.072 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:05.072 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922961+0000", 2024-09-21T21:42:05.072 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:05.072 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.072 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.072 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:05.072 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922961+0000", 2024-09-21T21:42:05.072 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:05.072 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.072 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.072 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:05.073 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922962+0000", 2024-09-21T21:42:05.073 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.89e-07 2024-09-21T21:42:05.073 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.073 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.073 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:05.073 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922964+0000", 2024-09-21T21:42:05.073 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1270000000000001e-06 2024-09-21T21:42:05.073 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.073 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.073 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:05.073 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922970+0000", 2024-09-21T21:42:05.073 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8030000000000002e-06 2024-09-21T21:42:05.073 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.073 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.074 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:05.074 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922979+0000", 2024-09-21T21:42:05.074 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5559999999999996e-06 2024-09-21T21:42:05.074 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.074 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.074 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:05.074 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964734+0000", 2024-09-21T21:42:05.074 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041754415000000003 2024-09-21T21:42:05.074 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.074 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.074 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:05.074 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964775+0000", 2024-09-21T21:42:05.075 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0806999999999997e-05 2024-09-21T21:42:05.075 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.075 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.075 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:05.075 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964825+0000", 2024-09-21T21:42:05.075 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0550999999999998e-05 2024-09-21T21:42:05.075 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:05.075 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:05.075 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:05.075 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.075 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.076 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6978 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5275648~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:05.076 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.922987+0000", 2024-09-21T21:42:05.076 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 35.140031041, 2024-09-21T21:42:05.076 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042043876000000001, 2024-09-21T21:42:05.076 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:05.076 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:05.076 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:05.076 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:05.076 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:05.076 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:05.076 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6978 2024-09-21T21:42:05.076 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.076 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:05.076 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.077 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:05.077 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922987+0000", 2024-09-21T21:42:05.077 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:05.077 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.077 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.077 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:05.077 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922987+0000", 2024-09-21T21:42:05.077 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:05.077 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.077 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.077 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:05.077 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922988+0000", 2024-09-21T21:42:05.077 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1199999999999998e-07 2024-09-21T21:42:05.077 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.077 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.078 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:05.078 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922990+0000", 2024-09-21T21:42:05.078 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.142e-06 2024-09-21T21:42:05.078 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.078 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.078 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:05.078 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922997+0000", 2024-09-21T21:42:05.078 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.199e-06 2024-09-21T21:42:05.078 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.078 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.078 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:05.078 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923004+0000", 2024-09-21T21:42:05.078 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3599999999999998e-06 2024-09-21T21:42:05.078 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.078 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.079 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:05.079 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964856+0000", 2024-09-21T21:42:05.079 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041851521000000003 2024-09-21T21:42:05.079 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.079 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.079 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:05.079 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964944+0000", 2024-09-21T21:42:05.079 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7859e-05 2024-09-21T21:42:05.079 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.079 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.079 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:05.079 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965031+0000", 2024-09-21T21:42:05.080 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7383e-05 2024-09-21T21:42:05.080 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:05.080 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:05.080 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:05.080 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.080 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.080 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6980 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5289984~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:05.080 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923011+0000", 2024-09-21T21:42:05.080 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 35.140007703999999, 2024-09-21T21:42:05.080 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042139337999999998, 2024-09-21T21:42:05.080 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:05.080 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:05.080 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:05.080 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:05.081 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:05.081 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:05.081 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6980 2024-09-21T21:42:05.081 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.081 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:05.081 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.081 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:05.081 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923011+0000", 2024-09-21T21:42:05.081 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:05.081 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.081 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.081 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:05.081 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923011+0000", 2024-09-21T21:42:05.081 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:05.082 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.082 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.082 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:05.082 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923011+0000", 2024-09-21T21:42:05.082 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6699999999999999e-07 2024-09-21T21:42:05.082 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.082 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.082 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:05.082 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923013+0000", 2024-09-21T21:42:05.082 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0279999999999999e-06 2024-09-21T21:42:05.082 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.082 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.082 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:05.082 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923018+0000", 2024-09-21T21:42:05.082 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4790000000000004e-06 2024-09-21T21:42:05.083 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.083 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.083 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:05.083 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923029+0000", 2024-09-21T21:42:05.083 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0601999999999999e-05 2024-09-21T21:42:05.083 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.083 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.083 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:05.083 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965059+0000", 2024-09-21T21:42:05.083 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042029947999999998 2024-09-21T21:42:05.083 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.083 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.083 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:05.083 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965100+0000", 2024-09-21T21:42:05.083 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1170000000000001e-05 2024-09-21T21:42:05.084 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.084 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.084 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:05.084 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965150+0000", 2024-09-21T21:42:05.084 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9743999999999997e-05 2024-09-21T21:42:05.084 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:05.084 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:05.084 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:05.084 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.084 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.084 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6982 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5304320~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:05.084 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923036+0000", 2024-09-21T21:42:05.084 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 35.139981882999997, 2024-09-21T21:42:05.084 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042225666000000002, 2024-09-21T21:42:05.084 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:05.085 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:05.085 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:05.085 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:05.085 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:05.085 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:05.085 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6982 2024-09-21T21:42:05.085 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.085 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:05.085 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.085 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:05.085 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923036+0000", 2024-09-21T21:42:05.085 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:05.085 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.085 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.086 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:05.086 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923036+0000", 2024-09-21T21:42:05.086 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:05.086 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.086 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.086 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:05.086 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923037+0000", 2024-09-21T21:42:05.086 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2800000000000002e-07 2024-09-21T21:42:05.086 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.086 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.086 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:05.086 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923039+0000", 2024-09-21T21:42:05.086 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1500000000000002e-06 2024-09-21T21:42:05.086 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.087 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.087 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:05.087 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923049+0000", 2024-09-21T21:42:05.087 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0236e-05 2024-09-21T21:42:05.087 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.087 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.087 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:05.087 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923056+0000", 2024-09-21T21:42:05.087 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2540000000000003e-06 2024-09-21T21:42:05.087 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.087 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.087 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:05.087 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965173+0000", 2024-09-21T21:42:05.088 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042116686 2024-09-21T21:42:05.088 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.088 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.088 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:05.088 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965215+0000", 2024-09-21T21:42:05.088 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1696000000000002e-05 2024-09-21T21:42:05.088 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.088 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.088 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:05.088 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965262+0000", 2024-09-21T21:42:05.088 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7216000000000003e-05 2024-09-21T21:42:05.088 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:05.088 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:05.088 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:05.089 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.089 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.089 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6984 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5318656~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:05.089 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923063+0000", 2024-09-21T21:42:05.089 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 35.139955229000002, 2024-09-21T21:42:05.089 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042318651999999998, 2024-09-21T21:42:05.089 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:05.089 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:05.089 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:05.089 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:05.089 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:05.089 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:05.089 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6984 2024-09-21T21:42:05.090 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.090 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:05.090 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.090 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:05.090 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923063+0000", 2024-09-21T21:42:05.090 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:05.090 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.090 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.090 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:05.090 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923063+0000", 2024-09-21T21:42:05.090 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:05.090 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.090 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.091 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:05.091 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923064+0000", 2024-09-21T21:42:05.091 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2e-07 2024-09-21T21:42:05.091 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.091 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.091 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:05.091 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923066+0000", 2024-09-21T21:42:05.091 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.057e-06 2024-09-21T21:42:05.091 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.091 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.091 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:05.091 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923071+0000", 2024-09-21T21:42:05.091 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8810000000000001e-06 2024-09-21T21:42:05.092 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.092 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.092 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:05.092 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923078+0000", 2024-09-21T21:42:05.092 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5479999999999998e-06 2024-09-21T21:42:05.092 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.092 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.092 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:05.092 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965288+0000", 2024-09-21T21:42:05.092 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042209576999999998 2024-09-21T21:42:05.092 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.092 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.092 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:05.093 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965332+0000", 2024-09-21T21:42:05.093 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4366999999999997e-05 2024-09-21T21:42:05.093 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.093 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.093 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:05.093 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965382+0000", 2024-09-21T21:42:05.093 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9802000000000002e-05 2024-09-21T21:42:05.093 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:05.093 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:05.093 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:05.093 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.093 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.094 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6987 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5332992~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:05.094 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923085+0000", 2024-09-21T21:42:05.094 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 35.139933704000001, 2024-09-21T21:42:05.094 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042412103999999999, 2024-09-21T21:42:05.094 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:05.094 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:05.094 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:05.094 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:05.094 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:05.094 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:05.094 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6987 2024-09-21T21:42:05.094 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.094 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:05.095 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.095 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:05.095 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923085+0000", 2024-09-21T21:42:05.095 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:05.095 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.095 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.095 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:05.095 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923085+0000", 2024-09-21T21:42:05.095 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:05.095 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.095 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.095 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:05.095 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923085+0000", 2024-09-21T21:42:05.096 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.89e-07 2024-09-21T21:42:05.096 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.096 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.096 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:05.096 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923087+0000", 2024-09-21T21:42:05.096 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0810000000000001e-06 2024-09-21T21:42:05.096 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.096 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.096 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:05.096 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923098+0000", 2024-09-21T21:42:05.096 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1299e-05 2024-09-21T21:42:05.096 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.096 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.096 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:05.097 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923104+0000", 2024-09-21T21:42:05.097 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9100000000000002e-06 2024-09-21T21:42:05.097 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.097 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.097 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:05.097 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965404+0000", 2024-09-21T21:42:05.097 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042299622000000002 2024-09-21T21:42:05.097 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.097 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.097 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:05.097 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965445+0000", 2024-09-21T21:42:05.097 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0701000000000002e-05 2024-09-21T21:42:05.097 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.097 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.098 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:05.098 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965497+0000", 2024-09-21T21:42:05.098 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2101999999999997e-05 2024-09-21T21:42:05.098 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:05.098 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:05.098 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:05.098 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.098 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.098 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6989 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5347328~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:05.098 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923111+0000", 2024-09-21T21:42:05.098 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 35.139907364000003, 2024-09-21T21:42:05.098 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042505188999999999, 2024-09-21T21:42:05.098 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:05.098 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:05.099 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:05.099 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:05.099 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:05.099 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:05.099 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6989 2024-09-21T21:42:05.099 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.099 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:05.099 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.099 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:05.099 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923111+0000", 2024-09-21T21:42:05.099 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:05.099 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.099 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.099 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:05.099 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923111+0000", 2024-09-21T21:42:05.100 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:05.100 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.100 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.100 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:05.100 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923111+0000", 2024-09-21T21:42:05.100 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4499999999999997e-07 2024-09-21T21:42:05.100 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.100 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.100 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:05.100 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923114+0000", 2024-09-21T21:42:05.100 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1569999999999998e-06 2024-09-21T21:42:05.100 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.101 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.101 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:05.101 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923120+0000", 2024-09-21T21:42:05.101 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1759999999999998e-06 2024-09-21T21:42:05.101 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.101 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.101 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:05.101 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923125+0000", 2024-09-21T21:42:05.101 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2560000000000002e-06 2024-09-21T21:42:05.101 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.101 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.101 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:05.101 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965523+0000", 2024-09-21T21:42:05.101 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042398094999999997 2024-09-21T21:42:05.102 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.102 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.102 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:05.102 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965566+0000", 2024-09-21T21:42:05.102 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3393999999999998e-05 2024-09-21T21:42:05.102 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.102 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.102 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:05.102 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965616+0000", 2024-09-21T21:42:05.102 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9666000000000002e-05 2024-09-21T21:42:05.102 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:05.102 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:05.103 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:05.103 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.103 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.103 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6991 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5361664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:05.103 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923132+0000", 2024-09-21T21:42:05.103 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 35.139886075, 2024-09-21T21:42:05.103 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042600779999999998, 2024-09-21T21:42:05.103 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:05.103 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:05.103 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:05.103 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:05.103 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:05.103 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:05.103 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6991 2024-09-21T21:42:05.104 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.104 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:05.104 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.104 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:05.104 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923132+0000", 2024-09-21T21:42:05.104 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:05.104 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.104 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.104 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:05.104 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923132+0000", 2024-09-21T21:42:05.104 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:05.104 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.104 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.104 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:05.105 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923132+0000", 2024-09-21T21:42:05.105 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4200000000000002e-07 2024-09-21T21:42:05.105 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.105 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.105 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:05.105 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923134+0000", 2024-09-21T21:42:05.105 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2270000000000001e-06 2024-09-21T21:42:05.105 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.105 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.105 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:05.105 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923137+0000", 2024-09-21T21:42:05.105 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1930000000000002e-06 2024-09-21T21:42:05.105 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.105 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.105 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:05.106 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923139+0000", 2024-09-21T21:42:05.106 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4710000000000001e-06 2024-09-21T21:42:05.106 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.106 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.106 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:05.106 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965639+0000", 2024-09-21T21:42:05.106 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.04249965 2024-09-21T21:42:05.106 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.106 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.106 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:05.106 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965679+0000", 2024-09-21T21:42:05.106 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9852999999999999e-05 2024-09-21T21:42:05.106 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.106 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.106 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:05.107 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965733+0000", 2024-09-21T21:42:05.107 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4144000000000002e-05 2024-09-21T21:42:05.107 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:05.107 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:05.107 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:05.107 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.107 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.107 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6993 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5384192~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:05.107 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923143+0000", 2024-09-21T21:42:05.107 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 35.139875281000002, 2024-09-21T21:42:05.107 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042708182999999997, 2024-09-21T21:42:05.107 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:05.107 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:05.108 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:05.108 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:05.108 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:05.108 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:05.108 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6993 2024-09-21T21:42:05.108 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.108 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:05.108 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.108 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:05.108 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923143+0000", 2024-09-21T21:42:05.108 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:05.108 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.108 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.108 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:05.109 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923143+0000", 2024-09-21T21:42:05.109 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:05.109 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.109 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.109 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:05.109 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923143+0000", 2024-09-21T21:42:05.109 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0800000000000001e-07 2024-09-21T21:42:05.109 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.109 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.109 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:05.109 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923144+0000", 2024-09-21T21:42:05.109 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.009e-06 2024-09-21T21:42:05.109 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.109 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.110 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:05.110 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923148+0000", 2024-09-21T21:42:05.110 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.737e-06 2024-09-21T21:42:05.110 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.110 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.110 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:05.110 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923150+0000", 2024-09-21T21:42:05.110 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4669999999999998e-06 2024-09-21T21:42:05.110 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.110 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.110 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:05.110 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965760+0000", 2024-09-21T21:42:05.110 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042609399999999999 2024-09-21T21:42:05.111 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.111 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.111 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:05.111 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965801+0000", 2024-09-21T21:42:05.111 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1399999999999997e-05 2024-09-21T21:42:05.111 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.111 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.111 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:05.111 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965851+0000", 2024-09-21T21:42:05.111 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9962e-05 2024-09-21T21:42:05.111 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:05.111 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:05.111 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:05.111 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.112 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.112 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6995 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5398528~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:05.112 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923153+0000", 2024-09-21T21:42:05.112 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 35.139864916999997, 2024-09-21T21:42:05.112 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042828034000000001, 2024-09-21T21:42:05.112 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:05.112 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:05.112 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:05.112 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:05.112 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:05.112 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:05.112 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6995 2024-09-21T21:42:05.113 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.113 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:05.113 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.113 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:05.113 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923153+0000", 2024-09-21T21:42:05.113 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:05.113 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.113 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.113 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:05.113 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923153+0000", 2024-09-21T21:42:05.113 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:05.113 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.113 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.114 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:05.114 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923154+0000", 2024-09-21T21:42:05.114 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2600000000000001e-07 2024-09-21T21:42:05.114 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.114 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.114 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:05.114 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923158+0000", 2024-09-21T21:42:05.114 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0559999999999998e-06 2024-09-21T21:42:05.114 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.114 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.114 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:05.114 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923161+0000", 2024-09-21T21:42:05.114 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6519999999999999e-06 2024-09-21T21:42:05.115 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.115 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.115 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:05.115 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923170+0000", 2024-09-21T21:42:05.115 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5720000000000003e-06 2024-09-21T21:42:05.115 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.115 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.115 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:05.115 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965887+0000", 2024-09-21T21:42:05.115 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042716863000000001 2024-09-21T21:42:05.115 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.115 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.115 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:05.115 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965927+0000", 2024-09-21T21:42:05.116 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0346999999999998e-05 2024-09-21T21:42:05.116 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.116 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.116 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:05.116 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965981+0000", 2024-09-21T21:42:05.116 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4317999999999997e-05 2024-09-21T21:42:05.116 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:05.116 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:05.116 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:05.116 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.116 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.116 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6997 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5412864~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:05.116 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923174+0000", 2024-09-21T21:42:05.116 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 35.139844762000003, 2024-09-21T21:42:05.116 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042928296999999997, 2024-09-21T21:42:05.117 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:05.117 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:05.117 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:05.117 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:05.117 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:05.117 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:05.117 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6997 2024-09-21T21:42:05.117 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.117 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:05.117 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.117 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:05.117 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923174+0000", 2024-09-21T21:42:05.117 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:05.117 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.117 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.118 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:05.118 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923174+0000", 2024-09-21T21:42:05.118 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:05.118 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.118 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.118 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:05.118 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923174+0000", 2024-09-21T21:42:05.118 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.67e-07 2024-09-21T21:42:05.118 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.118 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.118 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:05.118 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923178+0000", 2024-09-21T21:42:05.118 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1330000000000001e-06 2024-09-21T21:42:05.119 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.119 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.119 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:05.119 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923182+0000", 2024-09-21T21:42:05.119 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.794e-06 2024-09-21T21:42:05.119 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.119 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.119 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:05.119 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923184+0000", 2024-09-21T21:42:05.119 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.435e-06 2024-09-21T21:42:05.119 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.119 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.119 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:05.119 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966009+0000", 2024-09-21T21:42:05.119 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042824327000000002 2024-09-21T21:42:05.120 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.120 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.120 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:05.120 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966052+0000", 2024-09-21T21:42:05.120 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3034000000000002e-05 2024-09-21T21:42:05.120 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.120 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.120 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:05.120 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966102+0000", 2024-09-21T21:42:05.120 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0306999999999998e-05 2024-09-21T21:42:05.120 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:05.120 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:05.120 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:05.120 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.120 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.121 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6999 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5427200~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:05.121 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923187+0000", 2024-09-21T21:42:05.121 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 35.139831033999997, 2024-09-21T21:42:05.121 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043104267000000002, 2024-09-21T21:42:05.121 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:05.121 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:05.121 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:05.121 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:05.121 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:05.121 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:05.121 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6999 2024-09-21T21:42:05.121 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.121 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:05.121 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.122 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:05.122 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923187+0000", 2024-09-21T21:42:05.122 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:05.122 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.122 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.122 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:05.122 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923187+0000", 2024-09-21T21:42:05.122 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:05.122 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.122 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.122 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:05.122 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923188+0000", 2024-09-21T21:42:05.122 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3999999999999998e-07 2024-09-21T21:42:05.122 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.123 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.123 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:05.123 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923189+0000", 2024-09-21T21:42:05.123 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4169999999999999e-06 2024-09-21T21:42:05.123 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.123 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.123 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:05.123 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923194+0000", 2024-09-21T21:42:05.123 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1819999999999997e-06 2024-09-21T21:42:05.123 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.123 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.123 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:05.123 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923198+0000", 2024-09-21T21:42:05.123 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.143e-06 2024-09-21T21:42:05.124 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.124 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.124 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:05.124 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966135+0000", 2024-09-21T21:42:05.124 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042936297999999998 2024-09-21T21:42:05.124 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.124 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.124 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:05.124 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966204+0000", 2024-09-21T21:42:05.124 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8989e-05 2024-09-21T21:42:05.124 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.124 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.124 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:05.124 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966292+0000", 2024-09-21T21:42:05.125 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7997999999999993e-05 2024-09-21T21:42:05.125 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:05.125 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:05.125 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:05.125 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.125 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.125 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7001 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5441536~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:05.125 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923211+0000", 2024-09-21T21:42:05.125 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 35.139807243, 2024-09-21T21:42:05.125 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043196518000000003, 2024-09-21T21:42:05.125 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:05.125 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:05.125 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:05.125 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:05.125 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:05.125 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:05.125 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7001 2024-09-21T21:42:05.126 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.126 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:05.126 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.126 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:05.126 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923211+0000", 2024-09-21T21:42:05.126 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:05.126 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.126 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.126 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:05.126 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923211+0000", 2024-09-21T21:42:05.126 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:05.127 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.127 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.127 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:05.127 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923212+0000", 2024-09-21T21:42:05.127 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3900000000000005e-07 2024-09-21T21:42:05.127 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.127 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.127 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:05.127 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923214+0000", 2024-09-21T21:42:05.127 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9970000000000001e-06 2024-09-21T21:42:05.127 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.127 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.127 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:05.127 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923219+0000", 2024-09-21T21:42:05.128 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4310000000000002e-06 2024-09-21T21:42:05.128 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.128 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.128 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:05.128 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923224+0000", 2024-09-21T21:42:05.128 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6410000000000001e-06 2024-09-21T21:42:05.128 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.128 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.128 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:05.128 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966318+0000", 2024-09-21T21:42:05.128 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043093842 2024-09-21T21:42:05.129 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.129 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.130 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:05.130 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966359+0000", 2024-09-21T21:42:05.130 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1260999999999997e-05 2024-09-21T21:42:05.130 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.130 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.130 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:05.130 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966408+0000", 2024-09-21T21:42:05.130 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8807000000000002e-05 2024-09-21T21:42:05.130 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:05.130 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:05.130 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:05.130 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.130 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.131 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7002 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5455872~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:05.131 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923229+0000", 2024-09-21T21:42:05.131 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 35.139788832999997, 2024-09-21T21:42:05.131 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043364568999999999, 2024-09-21T21:42:05.131 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:05.131 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:05.131 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:05.131 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:05.131 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:05.131 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:05.131 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7002 2024-09-21T21:42:05.131 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.131 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:05.132 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.132 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:05.132 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923229+0000", 2024-09-21T21:42:05.132 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:05.132 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.132 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.132 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:05.132 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923229+0000", 2024-09-21T21:42:05.132 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:05.132 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.132 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.132 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:05.132 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923230+0000", 2024-09-21T21:42:05.132 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4700000000000002e-07 2024-09-21T21:42:05.133 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.133 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.133 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:05.133 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923232+0000", 2024-09-21T21:42:05.133 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0719999999999998e-06 2024-09-21T21:42:05.133 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.133 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.133 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:05.133 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923237+0000", 2024-09-21T21:42:05.133 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2290000000000002e-06 2024-09-21T21:42:05.133 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.133 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.133 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:05.133 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923245+0000", 2024-09-21T21:42:05.134 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2959999999999999e-06 2024-09-21T21:42:05.134 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.134 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.134 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:05.134 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966438+0000", 2024-09-21T21:42:05.134 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043193027000000002 2024-09-21T21:42:05.134 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.134 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.134 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:05.134 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966506+0000", 2024-09-21T21:42:05.134 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7707999999999996e-05 2024-09-21T21:42:05.134 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.134 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.134 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:05.135 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966594+0000", 2024-09-21T21:42:05.135 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.789e-05 2024-09-21T21:42:05.135 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:05.135 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:05.135 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:05.135 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.135 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.135 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7005 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5470208~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:05.135 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923254+0000", 2024-09-21T21:42:05.135 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 35.139764602, 2024-09-21T21:42:05.135 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043457176, 2024-09-21T21:42:05.135 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:05.135 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:05.135 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:05.136 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:05.136 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:05.136 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:05.136 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7005 2024-09-21T21:42:05.136 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.136 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:05.136 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.136 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:05.136 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923254+0000", 2024-09-21T21:42:05.136 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:05.136 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.137 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.137 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:05.137 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923254+0000", 2024-09-21T21:42:05.137 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:05.137 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.137 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.137 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:05.137 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923254+0000", 2024-09-21T21:42:05.137 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4700000000000002e-07 2024-09-21T21:42:05.137 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.137 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.137 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:05.137 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923256+0000", 2024-09-21T21:42:05.137 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.007e-06 2024-09-21T21:42:05.138 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.138 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.138 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:05.138 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923266+0000", 2024-09-21T21:42:05.138 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.4700000000000008e-06 2024-09-21T21:42:05.138 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.138 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.138 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:05.138 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923271+0000", 2024-09-21T21:42:05.138 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2809999999999999e-06 2024-09-21T21:42:05.138 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.138 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.138 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:05.139 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966619+0000", 2024-09-21T21:42:05.139 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043347951000000003 2024-09-21T21:42:05.139 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.139 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.139 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:05.139 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966661+0000", 2024-09-21T21:42:05.139 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2614999999999998e-05 2024-09-21T21:42:05.139 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.139 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.139 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:05.139 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966711+0000", 2024-09-21T21:42:05.139 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9505000000000001e-05 2024-09-21T21:42:05.139 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:05.139 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:05.140 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:05.140 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.140 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.140 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7008 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5484544~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:05.140 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923278+0000", 2024-09-21T21:42:05.140 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 35.139739955000003, 2024-09-21T21:42:05.140 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043631071, 2024-09-21T21:42:05.140 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:05.140 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:05.140 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:05.140 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:05.140 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:05.140 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:05.141 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7008 2024-09-21T21:42:05.141 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.141 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:05.141 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.141 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:05.141 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923278+0000", 2024-09-21T21:42:05.141 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:05.141 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.141 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.141 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:05.141 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923278+0000", 2024-09-21T21:42:05.141 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:05.142 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.142 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.142 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:05.142 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923279+0000", 2024-09-21T21:42:05.142 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.77e-07 2024-09-21T21:42:05.142 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.142 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.142 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:05.142 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923281+0000", 2024-09-21T21:42:05.142 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0219999999999999e-06 2024-09-21T21:42:05.143 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.143 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.143 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:05.143 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923287+0000", 2024-09-21T21:42:05.143 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7799999999999997e-06 2024-09-21T21:42:05.143 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.143 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.143 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:05.143 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923291+0000", 2024-09-21T21:42:05.143 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1699999999999999e-06 2024-09-21T21:42:05.143 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.143 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.143 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:05.144 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966742+0000", 2024-09-21T21:42:05.144 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043451344000000003 2024-09-21T21:42:05.144 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.144 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.144 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:05.144 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966809+0000", 2024-09-21T21:42:05.144 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7070999999999995e-05 2024-09-21T21:42:05.144 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.144 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.144 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:05.144 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966909+0000", 2024-09-21T21:42:05.144 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000100307 2024-09-21T21:42:05.144 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:05.145 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:05.145 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:05.145 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.145 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.145 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7010 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5498880~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:05.145 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923301+0000", 2024-09-21T21:42:05.145 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 35.139717060000002, 2024-09-21T21:42:05.145 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043717830999999999, 2024-09-21T21:42:05.145 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:05.145 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:05.145 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:05.145 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:05.145 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:05.146 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:05.146 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7010 2024-09-21T21:42:05.146 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.146 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:05.146 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.146 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:05.146 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923301+0000", 2024-09-21T21:42:05.146 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:05.146 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.146 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.146 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:05.146 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923301+0000", 2024-09-21T21:42:05.146 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:05.147 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.147 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.147 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:05.147 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923302+0000", 2024-09-21T21:42:05.147 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.41e-07 2024-09-21T21:42:05.147 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.147 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.147 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:05.147 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923304+0000", 2024-09-21T21:42:05.147 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0990000000000001e-06 2024-09-21T21:42:05.147 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.147 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.147 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:05.147 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923310+0000", 2024-09-21T21:42:05.148 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3860000000000004e-06 2024-09-21T21:42:05.148 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.148 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.148 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:05.148 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923315+0000", 2024-09-21T21:42:05.148 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8459999999999999e-06 2024-09-21T21:42:05.148 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.148 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.148 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:05.148 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966938+0000", 2024-09-21T21:42:05.148 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043622693999999997 2024-09-21T21:42:05.148 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.148 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.149 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:05.149 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966975+0000", 2024-09-21T21:42:05.149 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7608999999999999e-05 2024-09-21T21:42:05.149 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.149 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.149 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:05.149 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.967019+0000", 2024-09-21T21:42:05.149 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3856000000000002e-05 2024-09-21T21:42:05.149 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:05.149 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:05.149 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:05.149 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.149 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.149 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7354 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 2107392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:05.150 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.936932+0000", 2024-09-21T21:42:05.150 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 35.126086205, 2024-09-21T21:42:05.150 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041607708, 2024-09-21T21:42:05.150 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:05.150 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:05.150 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:05.150 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:05.150 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:05.150 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:05.150 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7354 2024-09-21T21:42:05.150 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.150 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:05.150 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.151 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:05.151 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.936932+0000", 2024-09-21T21:42:05.151 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:05.151 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.151 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.151 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:05.151 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.936932+0000", 2024-09-21T21:42:05.151 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:05.151 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.151 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.151 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:05.151 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.937276+0000", 2024-09-21T21:42:05.151 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00034344200000000001 2024-09-21T21:42:05.151 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.152 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.152 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:05.152 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.937279+0000", 2024-09-21T21:42:05.152 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.844e-06 2024-09-21T21:42:05.152 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.152 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.152 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:05.152 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.937288+0000", 2024-09-21T21:42:05.152 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2409999999999995e-06 2024-09-21T21:42:05.152 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.152 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.152 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:05.152 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.937293+0000", 2024-09-21T21:42:05.152 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6559999999999997e-06 2024-09-21T21:42:05.153 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.153 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.153 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:05.153 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.978448+0000", 2024-09-21T21:42:05.153 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041154621000000002 2024-09-21T21:42:05.153 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.153 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.153 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:05.153 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.978487+0000", 2024-09-21T21:42:05.153 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9254000000000001e-05 2024-09-21T21:42:05.153 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.153 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.153 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:05.154 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.978540+0000", 2024-09-21T21:42:05.154 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2649999999999999e-05 2024-09-21T21:42:05.154 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:05.154 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:05.154 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:05.154 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.154 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.154 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5215.0:1054 3.9 3:946d2401:::rbd_data.1121524d2522.000000000000000b:head [set-alloc-hint object_size 16777216 write_size 16777216,write 6615040~59392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-21T21:42:05.154 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:38.148979+0000", 2024-09-21T21:42:05.154 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 26.914039238000001, 2024-09-21T21:42:05.154 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047870388999999999, 2024-09-21T21:42:05.154 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:05.154 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:05.155 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:05.155 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:05.155 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5215", 2024-09-21T21:42:05.155 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1580282168", 2024-09-21T21:42:05.155 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1054 2024-09-21T21:42:05.155 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.155 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:05.155 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.155 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:05.155 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.148979+0000", 2024-09-21T21:42:05.155 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:05.155 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.155 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.155 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:05.155 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.148979+0000", 2024-09-21T21:42:05.156 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:05.156 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.156 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.156 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:05.156 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.148980+0000", 2024-09-21T21:42:05.156 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5799999999999999e-07 2024-09-21T21:42:05.156 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.156 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.156 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:05.156 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.149039+0000", 2024-09-21T21:42:05.156 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9151000000000003e-05 2024-09-21T21:42:05.156 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.156 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.156 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:05.157 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.149047+0000", 2024-09-21T21:42:05.157 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1000000000000004e-06 2024-09-21T21:42:05.157 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.157 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.157 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:05.157 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.149052+0000", 2024-09-21T21:42:05.157 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5959999999999997e-06 2024-09-21T21:42:05.157 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.157 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.157 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:05.157 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.195288+0000", 2024-09-21T21:42:05.157 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.046236248000000001 2024-09-21T21:42:05.157 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.158 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.158 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:05.158 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.195334+0000", 2024-09-21T21:42:05.158 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6575e-05 2024-09-21T21:42:05.158 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.158 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.158 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 2", 2024-09-21T21:42:05.158 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.195406+0000", 2024-09-21T21:42:05.158 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1715e-05 2024-09-21T21:42:05.158 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.158 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.158 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:05.158 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.196018+0000", 2024-09-21T21:42:05.159 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00061218999999999998 2024-09-21T21:42:05.159 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.159 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.159 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:05.159 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.196794+0000", 2024-09-21T21:42:05.159 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000775814 2024-09-21T21:42:05.159 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.159 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.159 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:05.159 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.196827+0000", 2024-09-21T21:42:05.159 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2419000000000003e-05 2024-09-21T21:42:05.159 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.159 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.159 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:05.160 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.196849+0000", 2024-09-21T21:42:05.160 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2923000000000001e-05 2024-09-21T21:42:05.160 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:05.160 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:05.160 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:05.160 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:05.160 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:05.160 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:05.162 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T21:42:05.163 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:05.160+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:05.234 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T21:42:05.278 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:05.277+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:05.305 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:05.306 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:42:05.306 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-21T21:42:05.306 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:05.314 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T21:42:05.362 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:05.361+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:05.427 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:05.427 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:42:05.427 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:42:05.428 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:42:05.428 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:05.437 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T21:42:05.455 INFO:teuthology.orchestra.run.smithi164.stdout:truncating image image_client.0-clone34 from 0xd92d000 (overlap 0x9f4600) to 0x0 2024-09-21T21:42:05.478 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:05.477+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:05.565 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:05.563+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:05.641 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:05.641 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-21T21:42:05.641 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-21T21:42:05.641 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:42:05.642 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.642 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1486 3.16 3:6a41a830:::rbd_data.112145fae779.000000000000000a:a [read 360448~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:42:05.642 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634126+0000", 2024-09-21T21:42:05.642 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 64.004525237999999, 2024-09-21T21:42:05.642 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059399528, 2024-09-21T21:42:05.642 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:05.642 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:05.642 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:05.642 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:05.642 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:42:05.642 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:42:05.642 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1486 2024-09-21T21:42:05.643 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.643 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:05.643 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.643 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:05.643 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634126+0000", 2024-09-21T21:42:05.643 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:05.643 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.643 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.643 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:05.644 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634126+0000", 2024-09-21T21:42:05.644 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:05.644 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.644 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.644 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:05.644 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634126+0000", 2024-09-21T21:42:05.644 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4099999999999999e-07 2024-09-21T21:42:05.644 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.644 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.644 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:05.644 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634128+0000", 2024-09-21T21:42:05.644 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1069999999999999e-06 2024-09-21T21:42:05.644 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.644 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.645 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:05.645 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634135+0000", 2024-09-21T21:42:05.645 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.212e-06 2024-09-21T21:42:05.645 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.645 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.645 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:05.645 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634140+0000", 2024-09-21T21:42:05.645 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1429999999999997e-06 2024-09-21T21:42:05.645 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.645 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.645 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:05.645 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693363+0000", 2024-09-21T21:42:05.645 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059223614000000001 2024-09-21T21:42:05.645 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.646 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.646 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:05.646 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693431+0000", 2024-09-21T21:42:05.646 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7294999999999999e-05 2024-09-21T21:42:05.646 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.646 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.646 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:05.646 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693525+0000", 2024-09-21T21:42:05.646 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.4716000000000006e-05 2024-09-21T21:42:05.646 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:05.646 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:05.646 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:05.646 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.647 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.647 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1499 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 49152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:42:05.647 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634186+0000", 2024-09-21T21:42:05.647 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 64.004465361000001, 2024-09-21T21:42:05.647 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059339256999999999, 2024-09-21T21:42:05.647 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:05.647 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:05.647 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:05.647 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:05.647 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:42:05.647 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:42:05.647 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1499 2024-09-21T21:42:05.648 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.648 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:05.648 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.648 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:05.648 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634186+0000", 2024-09-21T21:42:05.648 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:05.648 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.648 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.648 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:05.648 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634186+0000", 2024-09-21T21:42:05.648 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:05.648 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.648 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.648 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:05.649 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634186+0000", 2024-09-21T21:42:05.649 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0999999999999999e-07 2024-09-21T21:42:05.649 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.649 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.649 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:05.649 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634188+0000", 2024-09-21T21:42:05.649 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.86e-06 2024-09-21T21:42:05.649 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.649 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.649 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:05.649 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634194+0000", 2024-09-21T21:42:05.649 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1109999999999996e-06 2024-09-21T21:42:05.649 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.649 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.650 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:05.650 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634199+0000", 2024-09-21T21:42:05.650 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7419999999999997e-06 2024-09-21T21:42:05.650 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.650 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.650 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:05.650 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693357+0000", 2024-09-21T21:42:05.650 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059157847 2024-09-21T21:42:05.650 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.650 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.650 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:05.650 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693430+0000", 2024-09-21T21:42:05.650 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3203999999999999e-05 2024-09-21T21:42:05.650 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.651 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.651 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:05.651 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693525+0000", 2024-09-21T21:42:05.651 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5082999999999997e-05 2024-09-21T21:42:05.651 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:05.651 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:05.651 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:05.651 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.651 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.651 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1501 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 106496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:42:05.651 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634205+0000", 2024-09-21T21:42:05.651 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 64.004446271000006, 2024-09-21T21:42:05.651 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059482394000000001, 2024-09-21T21:42:05.651 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:05.652 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:05.652 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:05.652 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:05.652 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:42:05.652 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:42:05.652 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1501 2024-09-21T21:42:05.652 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.652 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:05.652 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.652 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:05.652 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634205+0000", 2024-09-21T21:42:05.652 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:05.652 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.652 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.652 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:05.653 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634205+0000", 2024-09-21T21:42:05.653 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:05.653 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.653 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.653 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:05.653 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634205+0000", 2024-09-21T21:42:05.653 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.03e-07 2024-09-21T21:42:05.653 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.653 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.653 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:05.653 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634207+0000", 2024-09-21T21:42:05.653 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0389999999999999e-06 2024-09-21T21:42:05.653 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.653 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.653 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:05.654 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634213+0000", 2024-09-21T21:42:05.654 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2600000000000002e-06 2024-09-21T21:42:05.654 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.654 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.654 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:05.654 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634218+0000", 2024-09-21T21:42:05.654 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8570000000000003e-06 2024-09-21T21:42:05.654 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.654 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.654 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:05.654 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693552+0000", 2024-09-21T21:42:05.654 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059333636000000002 2024-09-21T21:42:05.654 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.655 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.655 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:05.655 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693613+0000", 2024-09-21T21:42:05.655 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0560000000000003e-05 2024-09-21T21:42:05.655 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.655 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.655 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:05.655 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693687+0000", 2024-09-21T21:42:05.655 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4639000000000001e-05 2024-09-21T21:42:05.655 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:05.655 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:05.655 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:05.655 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.655 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.656 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1515 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 434176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:42:05.656 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634318+0000", 2024-09-21T21:42:05.656 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 64.004332583999997, 2024-09-21T21:42:05.656 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059565812000000003, 2024-09-21T21:42:05.656 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:05.656 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:05.656 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:05.656 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:05.656 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:42:05.656 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:42:05.656 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1515 2024-09-21T21:42:05.656 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.656 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:05.656 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.657 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:05.657 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634318+0000", 2024-09-21T21:42:05.657 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:05.657 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.657 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.657 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:05.657 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634318+0000", 2024-09-21T21:42:05.657 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:05.657 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.657 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.657 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:05.657 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634319+0000", 2024-09-21T21:42:05.657 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9400000000000001e-07 2024-09-21T21:42:05.657 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.657 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.658 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:05.658 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634321+0000", 2024-09-21T21:42:05.658 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.154e-06 2024-09-21T21:42:05.658 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.658 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.658 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:05.658 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634327+0000", 2024-09-21T21:42:05.658 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9830000000000001e-06 2024-09-21T21:42:05.658 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.658 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.658 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:05.658 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634332+0000", 2024-09-21T21:42:05.658 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9080000000000003e-06 2024-09-21T21:42:05.658 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.659 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.659 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:05.659 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693717+0000", 2024-09-21T21:42:05.659 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059385447000000001 2024-09-21T21:42:05.659 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.659 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.659 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:05.659 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693795+0000", 2024-09-21T21:42:05.659 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7267000000000004e-05 2024-09-21T21:42:05.659 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.659 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.659 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:05.659 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693884+0000", 2024-09-21T21:42:05.659 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9659000000000003e-05 2024-09-21T21:42:05.660 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:05.660 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:05.660 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:05.660 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.660 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.660 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1517 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 491520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:42:05.660 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634338+0000", 2024-09-21T21:42:05.660 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 64.004312811000005, 2024-09-21T21:42:05.660 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060100745999999997, 2024-09-21T21:42:05.660 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:05.660 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:05.660 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:05.660 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:05.660 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:42:05.660 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:42:05.661 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1517 2024-09-21T21:42:05.661 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.661 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:05.661 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.661 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:05.661 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634338+0000", 2024-09-21T21:42:05.661 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:05.661 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.661 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.661 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:05.662 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634338+0000", 2024-09-21T21:42:05.662 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:05.662 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.662 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.662 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:05.662 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634339+0000", 2024-09-21T21:42:05.662 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2899999999999999e-07 2024-09-21T21:42:05.662 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.662 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.662 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:05.662 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634341+0000", 2024-09-21T21:42:05.662 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.8959999999999999e-06 2024-09-21T21:42:05.663 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.663 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.663 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:05.663 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634347+0000", 2024-09-21T21:42:05.663 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.302e-06 2024-09-21T21:42:05.663 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.663 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.663 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:05.663 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634351+0000", 2024-09-21T21:42:05.663 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6070000000000001e-06 2024-09-21T21:42:05.663 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.663 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.663 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:05.664 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694279+0000", 2024-09-21T21:42:05.664 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059927046999999997 2024-09-21T21:42:05.664 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.664 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.664 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:05.664 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694355+0000", 2024-09-21T21:42:05.664 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6483999999999995e-05 2024-09-21T21:42:05.664 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.664 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.664 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:05.664 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694439+0000", 2024-09-21T21:42:05.664 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3980999999999994e-05 2024-09-21T21:42:05.664 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:05.664 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:05.664 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:05.665 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.665 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.665 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1518 3.16 3:6a41a830:::rbd_data.112145fae779.000000000000000a:a [read 294912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:42:05.665 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634357+0000", 2024-09-21T21:42:05.665 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 64.004293904999997, 2024-09-21T21:42:05.665 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059712318, 2024-09-21T21:42:05.665 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:05.665 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:05.665 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:05.665 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:05.665 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:42:05.665 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:42:05.665 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1518 2024-09-21T21:42:05.665 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.666 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:05.666 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.666 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:05.666 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634357+0000", 2024-09-21T21:42:05.666 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:05.666 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.666 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.666 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:05.666 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634357+0000", 2024-09-21T21:42:05.666 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:05.666 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.666 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.666 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:05.666 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634358+0000", 2024-09-21T21:42:05.667 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2100000000000002e-07 2024-09-21T21:42:05.667 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.667 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.667 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:05.667 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634360+0000", 2024-09-21T21:42:05.667 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0940000000000002e-06 2024-09-21T21:42:05.667 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.667 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.667 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:05.667 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634366+0000", 2024-09-21T21:42:05.667 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3389999999999998e-06 2024-09-21T21:42:05.667 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.667 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.668 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:05.668 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634371+0000", 2024-09-21T21:42:05.668 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6670000000000004e-06 2024-09-21T21:42:05.668 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.668 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.668 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:05.668 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693910+0000", 2024-09-21T21:42:05.668 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059538973000000002 2024-09-21T21:42:05.668 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.668 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.668 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:05.668 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693982+0000", 2024-09-21T21:42:05.668 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2806999999999996e-05 2024-09-21T21:42:05.668 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.669 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.669 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:05.669 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694069+0000", 2024-09-21T21:42:05.669 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7016999999999997e-05 2024-09-21T21:42:05.669 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:05.669 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:05.669 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:05.669 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.669 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.669 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1519 3.16 3:6a41a830:::rbd_data.112145fae779.000000000000000a:a [read 352256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:42:05.669 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634377+0000", 2024-09-21T21:42:05.669 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 64.004274202999994, 2024-09-21T21:42:05.669 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059872213000000001, 2024-09-21T21:42:05.670 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:05.670 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:05.670 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:05.670 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:05.670 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:42:05.670 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:42:05.670 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1519 2024-09-21T21:42:05.670 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.670 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:05.670 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.670 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:05.670 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634377+0000", 2024-09-21T21:42:05.670 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:05.671 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.671 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.671 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:05.671 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634377+0000", 2024-09-21T21:42:05.671 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:05.671 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.671 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.671 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:05.671 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634377+0000", 2024-09-21T21:42:05.671 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2600000000000002e-07 2024-09-21T21:42:05.671 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.671 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.671 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:05.672 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634379+0000", 2024-09-21T21:42:05.672 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0729999999999999e-06 2024-09-21T21:42:05.672 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.672 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.672 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:05.672 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634386+0000", 2024-09-21T21:42:05.672 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4690000000000002e-06 2024-09-21T21:42:05.672 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.672 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.672 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:05.672 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634390+0000", 2024-09-21T21:42:05.672 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4580000000000003e-06 2024-09-21T21:42:05.672 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.672 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.673 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:05.673 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694094+0000", 2024-09-21T21:42:05.673 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059703830999999999 2024-09-21T21:42:05.673 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.673 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.673 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:05.673 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694168+0000", 2024-09-21T21:42:05.673 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4222999999999997e-05 2024-09-21T21:42:05.673 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.673 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.673 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:05.673 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694249+0000", 2024-09-21T21:42:05.673 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0632999999999999e-05 2024-09-21T21:42:05.673 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:05.674 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:05.674 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:05.674 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.674 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.674 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1529 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 53248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:42:05.674 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634397+0000", 2024-09-21T21:42:05.674 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 64.004254051999993, 2024-09-21T21:42:05.674 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060189919000000001, 2024-09-21T21:42:05.674 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:05.674 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:05.674 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:05.674 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:05.674 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:42:05.675 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:42:05.675 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1529 2024-09-21T21:42:05.675 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.675 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:05.675 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.675 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:05.675 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634397+0000", 2024-09-21T21:42:05.675 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:05.675 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.675 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.675 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:05.675 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634397+0000", 2024-09-21T21:42:05.675 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:05.676 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.676 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.676 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:05.676 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634397+0000", 2024-09-21T21:42:05.676 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2e-07 2024-09-21T21:42:05.676 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.676 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.676 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:05.676 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634399+0000", 2024-09-21T21:42:05.676 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0609999999999998e-06 2024-09-21T21:42:05.676 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.676 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.676 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:05.676 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634406+0000", 2024-09-21T21:42:05.677 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7259999999999996e-06 2024-09-21T21:42:05.677 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.677 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.677 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:05.677 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634411+0000", 2024-09-21T21:42:05.677 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9339999999999997e-06 2024-09-21T21:42:05.677 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.677 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.677 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:05.677 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694480+0000", 2024-09-21T21:42:05.677 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060069105999999997 2024-09-21T21:42:05.677 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.677 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.677 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:05.677 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694527+0000", 2024-09-21T21:42:05.677 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7120000000000003e-05 2024-09-21T21:42:05.678 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.678 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.678 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:05.678 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694587+0000", 2024-09-21T21:42:05.678 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9552000000000002e-05 2024-09-21T21:42:05.678 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:05.678 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:05.678 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:05.678 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.678 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.678 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1531 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 110592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:42:05.678 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634417+0000", 2024-09-21T21:42:05.678 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 64.004234117999999, 2024-09-21T21:42:05.678 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060347287999999999, 2024-09-21T21:42:05.678 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:05.679 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:05.679 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:05.679 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:05.679 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:42:05.679 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:42:05.679 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1531 2024-09-21T21:42:05.679 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.679 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:05.679 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.679 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:05.679 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634417+0000", 2024-09-21T21:42:05.679 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:05.679 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.679 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.679 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:05.680 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634417+0000", 2024-09-21T21:42:05.680 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:05.680 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.680 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.680 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:05.680 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634417+0000", 2024-09-21T21:42:05.680 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1699999999999999e-07 2024-09-21T21:42:05.680 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.680 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.680 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:05.680 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634420+0000", 2024-09-21T21:42:05.680 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.243e-06 2024-09-21T21:42:05.680 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.680 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.680 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:05.680 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634426+0000", 2024-09-21T21:42:05.681 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2550000000000003e-06 2024-09-21T21:42:05.681 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.681 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.681 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:05.681 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634431+0000", 2024-09-21T21:42:05.681 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0150000000000003e-06 2024-09-21T21:42:05.681 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.681 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.681 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:05.681 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694618+0000", 2024-09-21T21:42:05.681 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060186836000000001 2024-09-21T21:42:05.681 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.681 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.681 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:05.681 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694685+0000", 2024-09-21T21:42:05.681 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6970000000000004e-05 2024-09-21T21:42:05.682 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.682 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.682 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:05.682 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694764+0000", 2024-09-21T21:42:05.682 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9552e-05 2024-09-21T21:42:05.682 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:05.682 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:05.682 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:05.682 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.682 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.682 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1532 3.e 3:724b31a3:::rbd_data.1121e540c096.0000000000000015:b [read 151552~36864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:42:05.682 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634437+0000", 2024-09-21T21:42:05.682 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 64.004214321999996, 2024-09-21T21:42:05.682 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060474314000000001, 2024-09-21T21:42:05.682 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:05.683 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:05.683 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:05.683 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:05.683 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:42:05.683 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:42:05.683 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1532 2024-09-21T21:42:05.683 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.683 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:05.683 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.683 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:05.683 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634437+0000", 2024-09-21T21:42:05.683 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:05.683 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.683 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.683 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:05.684 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634437+0000", 2024-09-21T21:42:05.684 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:05.684 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.684 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.684 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:05.684 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634437+0000", 2024-09-21T21:42:05.684 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9400000000000001e-07 2024-09-21T21:42:05.684 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.684 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.684 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:05.684 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634439+0000", 2024-09-21T21:42:05.684 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.114e-06 2024-09-21T21:42:05.684 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.684 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.684 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:05.684 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634446+0000", 2024-09-21T21:42:05.684 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1389999999999998e-06 2024-09-21T21:42:05.685 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.685 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.685 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:05.685 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634451+0000", 2024-09-21T21:42:05.685 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0010000000000001e-06 2024-09-21T21:42:05.685 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.685 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.685 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:05.685 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694794+0000", 2024-09-21T21:42:05.685 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060342509000000003 2024-09-21T21:42:05.685 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.685 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.685 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:05.685 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694835+0000", 2024-09-21T21:42:05.685 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0728e-05 2024-09-21T21:42:05.685 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.686 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.686 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:05.686 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694911+0000", 2024-09-21T21:42:05.686 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6428999999999996e-05 2024-09-21T21:42:05.686 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:05.686 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:05.686 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:05.686 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.686 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.686 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1534 3.e 3:724b31a3:::rbd_data.1121e540c096.0000000000000015:b [read 487424~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:42:05.686 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634477+0000", 2024-09-21T21:42:05.686 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 64.004173918999996, 2024-09-21T21:42:05.686 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060624009, 2024-09-21T21:42:05.686 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:05.686 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:05.687 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:05.687 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:05.687 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:42:05.687 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:42:05.687 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1534 2024-09-21T21:42:05.687 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.687 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:05.687 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.687 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:05.687 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634477+0000", 2024-09-21T21:42:05.687 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:05.687 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.687 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.687 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:05.688 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634477+0000", 2024-09-21T21:42:05.688 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:05.688 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.688 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.688 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:05.688 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634478+0000", 2024-09-21T21:42:05.688 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1399999999999997e-07 2024-09-21T21:42:05.688 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.688 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.688 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:05.688 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634480+0000", 2024-09-21T21:42:05.688 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1009999999999999e-06 2024-09-21T21:42:05.688 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.688 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.689 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:05.689 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634486+0000", 2024-09-21T21:42:05.689 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3289999999999999e-06 2024-09-21T21:42:05.689 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.689 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.689 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:05.689 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634491+0000", 2024-09-21T21:42:05.689 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6260000000000003e-06 2024-09-21T21:42:05.689 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.689 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.689 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:05.689 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694943+0000", 2024-09-21T21:42:05.689 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060451960999999999 2024-09-21T21:42:05.690 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.690 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.690 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:05.690 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695014+0000", 2024-09-21T21:42:05.690 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1587000000000004e-05 2024-09-21T21:42:05.690 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.690 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.690 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:05.690 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695101+0000", 2024-09-21T21:42:05.690 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6990999999999994e-05 2024-09-21T21:42:05.690 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:05.690 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:05.690 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:05.691 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.691 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.691 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1545 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 438272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:42:05.691 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634515+0000", 2024-09-21T21:42:05.691 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 64.004135719000004, 2024-09-21T21:42:05.691 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060707944, 2024-09-21T21:42:05.691 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:05.691 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:05.691 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:05.691 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:05.691 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:42:05.691 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:42:05.691 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1545 2024-09-21T21:42:05.692 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.692 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:05.692 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.692 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:05.692 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634515+0000", 2024-09-21T21:42:05.692 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:05.692 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.692 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.692 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:05.692 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634515+0000", 2024-09-21T21:42:05.692 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:05.692 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.692 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.693 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:05.693 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634516+0000", 2024-09-21T21:42:05.693 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0999999999999999e-07 2024-09-21T21:42:05.693 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.693 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.693 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:05.693 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634518+0000", 2024-09-21T21:42:05.693 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.846e-06 2024-09-21T21:42:05.693 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.693 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.693 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:05.693 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634524+0000", 2024-09-21T21:42:05.693 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2550000000000003e-06 2024-09-21T21:42:05.693 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.694 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.694 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:05.694 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634529+0000", 2024-09-21T21:42:05.694 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6940000000000003e-06 2024-09-21T21:42:05.694 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.694 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.694 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:05.694 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695127+0000", 2024-09-21T21:42:05.694 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060598921 2024-09-21T21:42:05.694 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.694 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.694 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:05.695 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695171+0000", 2024-09-21T21:42:05.695 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3588999999999999e-05 2024-09-21T21:42:05.695 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.695 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.695 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:05.695 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695223+0000", 2024-09-21T21:42:05.695 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2228999999999997e-05 2024-09-21T21:42:05.695 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:05.695 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:05.695 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:05.695 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.695 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.695 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1547 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 495616~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:42:05.696 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634534+0000", 2024-09-21T21:42:05.696 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 64.004116928000002, 2024-09-21T21:42:05.696 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060881131999999998, 2024-09-21T21:42:05.696 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:05.696 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:05.696 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:05.696 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:05.696 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:42:05.696 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:42:05.696 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1547 2024-09-21T21:42:05.696 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.696 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:05.697 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.697 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:05.697 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634534+0000", 2024-09-21T21:42:05.697 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:05.697 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.697 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.697 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:05.697 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634534+0000", 2024-09-21T21:42:05.697 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:05.697 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.697 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.697 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:05.697 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634535+0000", 2024-09-21T21:42:05.697 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.77e-07 2024-09-21T21:42:05.698 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.698 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.698 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:05.698 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634542+0000", 2024-09-21T21:42:05.698 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9000000000000006e-06 2024-09-21T21:42:05.698 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.698 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.698 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:05.698 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634549+0000", 2024-09-21T21:42:05.698 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5910000000000001e-06 2024-09-21T21:42:05.698 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.698 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.698 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:05.698 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634554+0000", 2024-09-21T21:42:05.699 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7940000000000002e-06 2024-09-21T21:42:05.699 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.699 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.699 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:05.699 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695256+0000", 2024-09-21T21:42:05.699 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.06070246 2024-09-21T21:42:05.699 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.699 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.699 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:05.699 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695332+0000", 2024-09-21T21:42:05.699 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6248000000000006e-05 2024-09-21T21:42:05.699 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.699 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.700 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:05.700 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695415+0000", 2024-09-21T21:42:05.700 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2762000000000005e-05 2024-09-21T21:42:05.700 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:05.700 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:05.700 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:05.700 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.700 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.700 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1548 3.16 3:6a41a830:::rbd_data.112145fae779.000000000000000a:a [read 299008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:42:05.700 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634561+0000", 2024-09-21T21:42:05.700 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 64.004090351000002, 2024-09-21T21:42:05.700 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060786743999999997, 2024-09-21T21:42:05.700 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:05.700 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:05.701 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:05.701 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:05.701 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:42:05.701 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:42:05.701 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1548 2024-09-21T21:42:05.701 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.701 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:05.701 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.701 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:05.701 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634561+0000", 2024-09-21T21:42:05.701 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:05.701 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.701 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.701 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:05.701 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634561+0000", 2024-09-21T21:42:05.702 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:05.702 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.702 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.702 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:05.702 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634561+0000", 2024-09-21T21:42:05.702 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5199999999999997e-07 2024-09-21T21:42:05.702 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.702 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.702 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:05.702 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634564+0000", 2024-09-21T21:42:05.702 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4040000000000002e-06 2024-09-21T21:42:05.702 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.702 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.703 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:05.703 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634571+0000", 2024-09-21T21:42:05.703 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4379999999999998e-06 2024-09-21T21:42:05.703 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.703 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.703 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:05.703 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634576+0000", 2024-09-21T21:42:05.703 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1399999999999999e-06 2024-09-21T21:42:05.703 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.703 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.703 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:05.703 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695247+0000", 2024-09-21T21:42:05.703 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060670397000000001 2024-09-21T21:42:05.703 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.703 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.704 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:05.704 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695299+0000", 2024-09-21T21:42:05.704 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2564e-05 2024-09-21T21:42:05.704 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.704 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.704 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:05.704 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695347+0000", 2024-09-21T21:42:05.704 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8248999999999998e-05 2024-09-21T21:42:05.704 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:05.704 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:05.704 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:05.704 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.704 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.704 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1549 3.16 3:6a41a830:::rbd_data.112145fae779.000000000000000a:a [read 356352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:42:05.705 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634583+0000", 2024-09-21T21:42:05.705 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 64.004068438000004, 2024-09-21T21:42:05.705 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060889459, 2024-09-21T21:42:05.705 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:05.705 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:05.705 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:05.705 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:05.705 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:42:05.705 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:42:05.705 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1549 2024-09-21T21:42:05.705 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.705 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:05.705 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.705 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:05.705 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634583+0000", 2024-09-21T21:42:05.706 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:05.706 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.706 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.706 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:05.706 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634583+0000", 2024-09-21T21:42:05.706 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:05.706 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.706 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.706 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:05.706 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634583+0000", 2024-09-21T21:42:05.706 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1899999999999998e-07 2024-09-21T21:42:05.706 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.706 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.706 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:05.707 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634585+0000", 2024-09-21T21:42:05.707 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2129999999999998e-06 2024-09-21T21:42:05.707 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.707 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.707 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:05.707 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634592+0000", 2024-09-21T21:42:05.707 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2840000000000003e-06 2024-09-21T21:42:05.707 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.707 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.707 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:05.707 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634596+0000", 2024-09-21T21:42:05.707 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9309999999999999e-06 2024-09-21T21:42:05.707 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.707 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.708 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:05.708 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695371+0000", 2024-09-21T21:42:05.708 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060774425999999999 2024-09-21T21:42:05.708 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.708 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.708 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:05.708 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695419+0000", 2024-09-21T21:42:05.708 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8607999999999999e-05 2024-09-21T21:42:05.708 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.708 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.708 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:05.708 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695472+0000", 2024-09-21T21:42:05.708 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2577999999999997e-05 2024-09-21T21:42:05.709 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:05.709 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:05.709 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:05.709 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.709 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.709 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4385.0:15202 3.16 3:6888948b:::rbd_header.1121493f6459:head [watch reconnect cookie 140325440202496 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-21T21:42:05.709 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:41.736601+0000", 2024-09-21T21:42:05.709 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 23.902049805000001, 2024-09-21T21:42:05.709 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5040056769999999, 2024-09-21T21:42:05.709 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:05.709 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:05.709 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:05.709 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:05.710 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4385", 2024-09-21T21:42:05.710 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:42:05.710 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 15202 2024-09-21T21:42:05.710 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.710 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:05.710 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.710 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:05.710 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736601+0000", 2024-09-21T21:42:05.710 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:05.710 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.710 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.710 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:05.710 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736601+0000", 2024-09-21T21:42:05.710 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:05.711 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.711 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.711 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:05.711 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736602+0000", 2024-09-21T21:42:05.711 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0699999999999999e-06 2024-09-21T21:42:05.711 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.711 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.711 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:05.711 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736607+0000", 2024-09-21T21:42:05.711 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2520000000000001e-06 2024-09-21T21:42:05.711 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.711 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.711 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:05.711 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736617+0000", 2024-09-21T21:42:05.712 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0074e-05 2024-09-21T21:42:05.712 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.712 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.712 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:05.712 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736622+0000", 2024-09-21T21:42:05.712 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.101e-06 2024-09-21T21:42:05.712 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.712 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.712 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:05.712 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736664+0000", 2024-09-21T21:42:05.712 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1869999999999997e-05 2024-09-21T21:42:05.712 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.712 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.713 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op must wait for map", 2024-09-21T21:42:05.713 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736667+0000", 2024-09-21T21:42:05.713 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4750000000000002e-06 2024-09-21T21:42:05.713 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.713 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.713 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:05.713 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742886+0000", 2024-09-21T21:42:05.713 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0062187579999999996 2024-09-21T21:42:05.713 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.713 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.713 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:05.713 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742892+0000", 2024-09-21T21:42:05.713 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0440000000000001e-06 2024-09-21T21:42:05.713 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.713 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.713 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:05.714 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210307+0000", 2024-09-21T21:42:05.714 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.467414781 2024-09-21T21:42:05.714 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.714 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.714 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:05.714 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210322+0000", 2024-09-21T21:42:05.714 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5311e-05 2024-09-21T21:42:05.714 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.714 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.714 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:05.714 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.237624+0000", 2024-09-21T21:42:05.714 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0273018920000001 2024-09-21T21:42:05.714 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.715 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.715 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:05.715 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.237870+0000", 2024-09-21T21:42:05.715 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00024622899999999999 2024-09-21T21:42:05.715 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.715 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.715 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:05.715 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.240607+0000", 2024-09-21T21:42:05.715 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0027368200000000001 2024-09-21T21:42:05.715 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:05.715 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:05.715 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:05.715 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.715 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.716 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5415.0:379 3.16 3:6888948b:::rbd_header.1121493f6459:head [watch reconnect cookie 139702602617920 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-21T21:42:05.716 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:41.739544+0000", 2024-09-21T21:42:05.716 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 23.899107376, 2024-09-21T21:42:05.716 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.501180645, 2024-09-21T21:42:05.716 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:05.716 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:05.716 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:05.716 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:05.716 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5415", 2024-09-21T21:42:05.716 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1212816607", 2024-09-21T21:42:05.716 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 379 2024-09-21T21:42:05.716 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.716 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:05.717 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.717 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:05.717 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739544+0000", 2024-09-21T21:42:05.717 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:05.717 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.717 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.717 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:05.717 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739544+0000", 2024-09-21T21:42:05.717 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:05.717 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.717 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.717 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:05.717 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739545+0000", 2024-09-21T21:42:05.717 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.065e-06 2024-09-21T21:42:05.717 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.718 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.718 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:05.718 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739550+0000", 2024-09-21T21:42:05.718 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9599999999999999e-06 2024-09-21T21:42:05.718 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.718 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.718 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:05.718 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739565+0000", 2024-09-21T21:42:05.718 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5709000000000002e-05 2024-09-21T21:42:05.718 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.718 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.718 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:05.718 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739573+0000", 2024-09-21T21:42:05.719 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7849999999999992e-06 2024-09-21T21:42:05.719 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.719 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.719 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:05.719 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739634+0000", 2024-09-21T21:42:05.719 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0732000000000001e-05 2024-09-21T21:42:05.719 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.719 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.719 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op must wait for map", 2024-09-21T21:42:05.719 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739640+0000", 2024-09-21T21:42:05.719 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0000000000000002e-06 2024-09-21T21:42:05.719 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.719 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.719 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:05.719 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742958+0000", 2024-09-21T21:42:05.720 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0033182390000000002 2024-09-21T21:42:05.720 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.720 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.720 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:05.720 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742966+0000", 2024-09-21T21:42:05.720 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1610000000000003e-06 2024-09-21T21:42:05.720 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.720 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.720 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:05.720 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210405+0000", 2024-09-21T21:42:05.720 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.467438725 2024-09-21T21:42:05.720 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.720 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.720 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:05.720 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210415+0000", 2024-09-21T21:42:05.721 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0234000000000001e-05 2024-09-21T21:42:05.721 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.721 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.721 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:05.721 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.238743+0000", 2024-09-21T21:42:05.721 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0283273070000001 2024-09-21T21:42:05.721 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.721 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.721 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:05.722 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.238842+0000", 2024-09-21T21:42:05.722 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.9896999999999996e-05 2024-09-21T21:42:05.722 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.722 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.722 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:05.722 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.240724+0000", 2024-09-21T21:42:05.722 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001881831 2024-09-21T21:42:05.722 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:05.722 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:05.722 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:05.723 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.723 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.723 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4385.0:15203 3.16 3:6888948b:::rbd_header.1121493f6459:head [call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-21T21:42:05.723 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:41.739737+0000", 2024-09-21T21:42:05.723 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 23.898914365, 2024-09-21T21:42:05.723 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4989621799999999, 2024-09-21T21:42:05.723 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:05.723 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:05.723 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:05.723 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:05.723 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4385", 2024-09-21T21:42:05.723 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:42:05.723 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 15203 2024-09-21T21:42:05.723 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.723 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:05.724 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.724 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:05.724 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739737+0000", 2024-09-21T21:42:05.724 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:05.724 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.724 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.724 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:05.724 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739737+0000", 2024-09-21T21:42:05.724 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:05.724 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.724 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.724 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:05.724 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739738+0000", 2024-09-21T21:42:05.724 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.7499999999999998e-07 2024-09-21T21:42:05.724 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.725 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.725 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:05.725 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739743+0000", 2024-09-21T21:42:05.725 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5049999999999998e-06 2024-09-21T21:42:05.725 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.725 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.725 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:05.725 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739757+0000", 2024-09-21T21:42:05.725 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3947e-05 2024-09-21T21:42:05.725 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.725 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.725 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:05.725 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739765+0000", 2024-09-21T21:42:05.725 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9310000000000008e-06 2024-09-21T21:42:05.726 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.726 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.726 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:05.726 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739816+0000", 2024-09-21T21:42:05.726 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0559999999999997e-05 2024-09-21T21:42:05.726 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.726 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.726 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting_for_map not empty", 2024-09-21T21:42:05.726 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739821+0000", 2024-09-21T21:42:05.726 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3539999999999999e-06 2024-09-21T21:42:05.726 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.726 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.726 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:05.726 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742928+0000", 2024-09-21T21:42:05.727 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0031074399999999999 2024-09-21T21:42:05.727 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.727 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.727 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:05.727 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742937+0000", 2024-09-21T21:42:05.727 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1289999999999996e-06 2024-09-21T21:42:05.727 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.727 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.727 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:05.727 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210350+0000", 2024-09-21T21:42:05.727 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.46741311299999999 2024-09-21T21:42:05.727 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.727 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.727 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:05.728 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210358+0000", 2024-09-21T21:42:05.728 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1869999999999997e-06 2024-09-21T21:42:05.728 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.728 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.728 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:05.728 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.238398+0000", 2024-09-21T21:42:05.728 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0280397830000001 2024-09-21T21:42:05.728 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.728 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.728 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:05.728 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.238489+0000", 2024-09-21T21:42:05.728 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.0958000000000006e-05 2024-09-21T21:42:05.728 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.728 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.728 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:05.729 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.238699+0000", 2024-09-21T21:42:05.729 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00021029800000000001 2024-09-21T21:42:05.729 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:05.729 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:05.729 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:05.729 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.729 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.729 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4385.0:15204 3.16 3:6888948b:::rbd_header.1121493f6459:head [watch reconnect cookie 140325440202496 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-21T21:42:05.729 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:42.210726+0000", 2024-09-21T21:42:05.729 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 23.427925487, 2024-09-21T21:42:05.729 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.030304557, 2024-09-21T21:42:05.730 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:05.730 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:05.730 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:05.730 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:05.730 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4385", 2024-09-21T21:42:05.730 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:42:05.730 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 15204 2024-09-21T21:42:05.730 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.730 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:05.730 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.730 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:05.730 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210726+0000", 2024-09-21T21:42:05.730 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:05.730 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.731 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.731 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:05.731 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210726+0000", 2024-09-21T21:42:05.731 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:05.731 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.731 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.731 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:05.731 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210727+0000", 2024-09-21T21:42:05.731 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.204e-06 2024-09-21T21:42:05.731 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.731 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.731 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:05.731 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210732+0000", 2024-09-21T21:42:05.731 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5249999999999996e-06 2024-09-21T21:42:05.732 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.732 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.732 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:05.732 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210744+0000", 2024-09-21T21:42:05.732 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1405e-05 2024-09-21T21:42:05.732 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.732 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.732 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:05.732 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210754+0000", 2024-09-21T21:42:05.732 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0122999999999999e-05 2024-09-21T21:42:05.732 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.732 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.732 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:05.732 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210798+0000", 2024-09-21T21:42:05.733 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4005000000000003e-05 2024-09-21T21:42:05.733 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.733 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.733 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:05.733 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210803+0000", 2024-09-21T21:42:05.733 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5130000000000004e-06 2024-09-21T21:42:05.733 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.733 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.733 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:05.733 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.239260+0000", 2024-09-21T21:42:05.733 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0284563040000001 2024-09-21T21:42:05.733 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.733 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.733 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:05.734 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.239318+0000", 2024-09-21T21:42:05.734 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8486000000000002e-05 2024-09-21T21:42:05.734 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.734 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.734 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:05.734 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.241030+0000", 2024-09-21T21:42:05.734 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0017119920000000001 2024-09-21T21:42:05.734 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:05.734 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:05.734 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:05.734 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.734 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.734 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5415.0:380 3.16 3:6888948b:::rbd_header.1121493f6459:head [watch reconnect cookie 139702602617920 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-21T21:42:05.734 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:42.210777+0000", 2024-09-21T21:42:05.735 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 23.427873708, 2024-09-21T21:42:05.735 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.030410566, 2024-09-21T21:42:05.735 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:05.735 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:05.735 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:05.735 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:05.735 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5415", 2024-09-21T21:42:05.735 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1212816607", 2024-09-21T21:42:05.735 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 380 2024-09-21T21:42:05.735 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.735 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:05.735 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.735 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:05.735 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210777+0000", 2024-09-21T21:42:05.736 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:05.736 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.736 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.736 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:05.736 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210777+0000", 2024-09-21T21:42:05.736 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:05.736 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.736 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.736 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:05.736 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210778+0000", 2024-09-21T21:42:05.736 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8999999999999996e-07 2024-09-21T21:42:05.736 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.736 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.736 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:05.736 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210780+0000", 2024-09-21T21:42:05.737 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.503e-06 2024-09-21T21:42:05.737 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.737 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.737 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:05.737 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210788+0000", 2024-09-21T21:42:05.737 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0999999999999998e-06 2024-09-21T21:42:05.737 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.737 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.737 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:05.737 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210799+0000", 2024-09-21T21:42:05.737 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1703e-05 2024-09-21T21:42:05.737 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.737 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.737 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:05.737 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210833+0000", 2024-09-21T21:42:05.738 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4008999999999999e-05 2024-09-21T21:42:05.738 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.738 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.738 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:05.738 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210838+0000", 2024-09-21T21:42:05.738 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7480000000000001e-06 2024-09-21T21:42:05.738 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.738 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.738 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:05.738 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.239612+0000", 2024-09-21T21:42:05.738 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.028773554 2024-09-21T21:42:05.738 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.738 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.738 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:05.739 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.239677+0000", 2024-09-21T21:42:05.739 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5053000000000002e-05 2024-09-21T21:42:05.739 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:05.739 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:05.739 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:05.739 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.241188+0000", 2024-09-21T21:42:05.739 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001511306 2024-09-21T21:42:05.739 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:05.739 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:05.739 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:05.739 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:05.739 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:05.739 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:05.741 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:05.739+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:05.741 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T21:42:05.842 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:05.840+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:05.844 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:05.844 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:42:05.844 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 0 2024-09-21T21:42:05.844 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:05.853 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T21:42:05.943 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:05.942+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:05.994 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:05.994 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:42:05.995 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:42:05.995 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:42:05.995 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:06.003 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T21:42:06.044 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:06.043+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:06.107 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:06.107 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 20, 2024-09-21T21:42:06.107 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 600, 2024-09-21T21:42:06.107 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:42:06.107 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.107 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11319 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 5492736~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:06.107 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.712385+0000", 2024-09-21T21:42:06.108 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 15.392581706, 2024-09-21T21:42:06.108 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078627889000000006, 2024-09-21T21:42:06.108 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:06.108 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:06.108 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:06.108 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:06.108 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:06.108 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:06.108 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11319 2024-09-21T21:42:06.108 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.108 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:06.108 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.108 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:06.108 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712385+0000", 2024-09-21T21:42:06.108 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:06.109 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.109 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.109 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:06.109 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712385+0000", 2024-09-21T21:42:06.109 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:06.109 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.109 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.109 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:06.109 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712385+0000", 2024-09-21T21:42:06.109 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2800000000000002e-07 2024-09-21T21:42:06.109 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.109 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.109 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:06.109 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712387+0000", 2024-09-21T21:42:06.110 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1569999999999998e-06 2024-09-21T21:42:06.110 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.110 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.110 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:06.110 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712393+0000", 2024-09-21T21:42:06.110 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9370000000000001e-06 2024-09-21T21:42:06.110 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.110 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.110 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:06.110 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712398+0000", 2024-09-21T21:42:06.110 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5020000000000003e-06 2024-09-21T21:42:06.110 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.110 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.110 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:06.110 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.790746+0000", 2024-09-21T21:42:06.111 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078348589999999996 2024-09-21T21:42:06.111 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.111 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.111 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:06.111 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.790844+0000", 2024-09-21T21:42:06.111 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.7526000000000001e-05 2024-09-21T21:42:06.111 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.111 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.111 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:06.111 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.791013+0000", 2024-09-21T21:42:06.111 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00016874899999999999 2024-09-21T21:42:06.111 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:06.111 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:06.111 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:06.111 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.111 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.112 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11320 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 6180864~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:06.112 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.712405+0000", 2024-09-21T21:42:06.112 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 15.392561747, 2024-09-21T21:42:06.112 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078882870999999993, 2024-09-21T21:42:06.112 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:06.112 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:06.112 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:06.112 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:06.112 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:06.112 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:06.112 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11320 2024-09-21T21:42:06.112 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.112 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:06.112 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.112 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:06.112 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712405+0000", 2024-09-21T21:42:06.113 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:06.113 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.113 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.113 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:06.113 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712405+0000", 2024-09-21T21:42:06.113 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:06.113 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.113 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.113 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:06.113 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.713942+0000", 2024-09-21T21:42:06.113 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.001536882 2024-09-21T21:42:06.113 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.113 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.113 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:06.114 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.713946+0000", 2024-09-21T21:42:06.114 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4549999999999997e-06 2024-09-21T21:42:06.114 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.114 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.114 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:06.114 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.713958+0000", 2024-09-21T21:42:06.114 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1775e-05 2024-09-21T21:42:06.114 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.114 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.114 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:06.114 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.713965+0000", 2024-09-21T21:42:06.114 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5100000000000001e-06 2024-09-21T21:42:06.114 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.114 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.114 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:06.114 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.791059+0000", 2024-09-21T21:42:06.115 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.077094036000000005 2024-09-21T21:42:06.115 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.115 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.115 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:06.115 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.791175+0000", 2024-09-21T21:42:06.115 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000115895 2024-09-21T21:42:06.115 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.115 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.115 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:06.115 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.791288+0000", 2024-09-21T21:42:06.115 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000112318 2024-09-21T21:42:06.115 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:06.115 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:06.115 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:06.115 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.115 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.116 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11325 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 5414912~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:06.116 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.714095+0000", 2024-09-21T21:42:06.116 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 15.390871752000001, 2024-09-21T21:42:06.116 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078631894999999993, 2024-09-21T21:42:06.116 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:06.116 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:06.116 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:06.116 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:06.116 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:06.116 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:06.116 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11325 2024-09-21T21:42:06.116 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.116 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:06.116 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.116 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:06.117 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714095+0000", 2024-09-21T21:42:06.117 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:06.117 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.117 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.117 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:06.117 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714095+0000", 2024-09-21T21:42:06.117 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:06.117 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.117 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.117 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:06.117 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714095+0000", 2024-09-21T21:42:06.117 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7500000000000001e-07 2024-09-21T21:42:06.117 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.117 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.117 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:06.117 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714097+0000", 2024-09-21T21:42:06.118 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.12e-06 2024-09-21T21:42:06.118 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.118 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.118 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:06.118 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714104+0000", 2024-09-21T21:42:06.118 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3520000000000003e-06 2024-09-21T21:42:06.118 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.118 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.118 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:06.118 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714108+0000", 2024-09-21T21:42:06.118 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8810000000000004e-06 2024-09-21T21:42:06.118 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.118 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.118 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:06.118 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792509+0000", 2024-09-21T21:42:06.119 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078400739999999997 2024-09-21T21:42:06.119 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.119 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.119 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:06.119 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792613+0000", 2024-09-21T21:42:06.119 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000103939 2024-09-21T21:42:06.119 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.119 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.119 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:06.119 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792727+0000", 2024-09-21T21:42:06.119 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011348799999999999 2024-09-21T21:42:06.119 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:06.119 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:06.119 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:06.120 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.120 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.120 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11326 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 6103040~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:06.120 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.714115+0000", 2024-09-21T21:42:06.120 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 15.390851724000001, 2024-09-21T21:42:06.120 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078963056000000004, 2024-09-21T21:42:06.120 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:06.120 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:06.120 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:06.120 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:06.120 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:06.120 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:06.120 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11326 2024-09-21T21:42:06.120 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.120 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:06.121 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.121 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:06.121 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714115+0000", 2024-09-21T21:42:06.121 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:06.121 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.121 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.121 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:06.121 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714115+0000", 2024-09-21T21:42:06.121 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:06.121 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.121 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.121 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:06.121 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.715940+0000", 2024-09-21T21:42:06.121 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0018254969999999999 2024-09-21T21:42:06.121 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.122 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.122 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:06.122 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.715945+0000", 2024-09-21T21:42:06.122 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6120000000000001e-06 2024-09-21T21:42:06.122 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.122 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.122 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:06.122 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.715954+0000", 2024-09-21T21:42:06.122 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.5899999999999997e-06 2024-09-21T21:42:06.122 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.122 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.122 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:06.122 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.715959+0000", 2024-09-21T21:42:06.122 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0849999999999996e-06 2024-09-21T21:42:06.122 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.123 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.123 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:06.123 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792771+0000", 2024-09-21T21:42:06.123 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.076811829999999998 2024-09-21T21:42:06.123 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.123 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.123 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:06.123 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792882+0000", 2024-09-21T21:42:06.123 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000110857 2024-09-21T21:42:06.123 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.123 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.123 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:06.123 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.793078+0000", 2024-09-21T21:42:06.123 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00019558499999999999 2024-09-21T21:42:06.123 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:06.124 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:06.124 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:06.124 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.124 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.124 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11332 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 7565312~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:06.124 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.716063+0000", 2024-09-21T21:42:06.124 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 15.388903190000001, 2024-09-21T21:42:06.124 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078787482000000006, 2024-09-21T21:42:06.124 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:06.124 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:06.124 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:06.124 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:06.124 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:06.124 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:06.124 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11332 2024-09-21T21:42:06.125 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.125 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:06.125 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.125 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:06.125 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.716063+0000", 2024-09-21T21:42:06.125 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:06.125 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.125 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.125 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:06.125 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.716063+0000", 2024-09-21T21:42:06.125 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:06.125 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.125 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.125 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:06.125 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.717944+0000", 2024-09-21T21:42:06.125 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0018812309999999999 2024-09-21T21:42:06.126 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.126 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.126 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:06.126 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.717949+0000", 2024-09-21T21:42:06.126 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9389999999999996e-06 2024-09-21T21:42:06.126 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.126 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.126 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:06.126 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.717963+0000", 2024-09-21T21:42:06.126 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.3899e-05 2024-09-21T21:42:06.126 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.126 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.126 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:06.126 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.717970+0000", 2024-09-21T21:42:06.126 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.426e-06 2024-09-21T21:42:06.127 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.127 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.127 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:06.127 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.794573+0000", 2024-09-21T21:42:06.127 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.076602990999999995 2024-09-21T21:42:06.127 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.127 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.127 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:06.127 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.794682+0000", 2024-09-21T21:42:06.127 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000109541 2024-09-21T21:42:06.127 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.127 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.127 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:06.127 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.794851+0000", 2024-09-21T21:42:06.127 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000168455 2024-09-21T21:42:06.128 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:06.128 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:06.128 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:06.128 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.128 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.128 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11339 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 6266880~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:06.128 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.718117+0000", 2024-09-21T21:42:06.128 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 15.386849164000001, 2024-09-21T21:42:06.128 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078703717000000006, 2024-09-21T21:42:06.128 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:06.128 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:06.128 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:06.128 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:06.128 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:06.129 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:06.129 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11339 2024-09-21T21:42:06.129 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.129 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:06.129 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.129 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:06.129 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.718117+0000", 2024-09-21T21:42:06.129 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:06.129 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.129 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.129 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:06.129 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.718117+0000", 2024-09-21T21:42:06.129 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:06.129 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.130 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.130 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:06.130 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720019+0000", 2024-09-21T21:42:06.130 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0019012549999999999 2024-09-21T21:42:06.130 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.130 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.130 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:06.130 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720024+0000", 2024-09-21T21:42:06.130 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1100000000000002e-06 2024-09-21T21:42:06.130 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.130 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.130 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:06.130 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720035+0000", 2024-09-21T21:42:06.131 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.094e-05 2024-09-21T21:42:06.131 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.131 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.131 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:06.131 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720040+0000", 2024-09-21T21:42:06.131 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1830000000000002e-06 2024-09-21T21:42:06.131 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.131 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.131 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:06.131 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796618+0000", 2024-09-21T21:42:06.131 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.076578033000000004 2024-09-21T21:42:06.131 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.131 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.131 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:06.132 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796696+0000", 2024-09-21T21:42:06.132 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7836000000000005e-05 2024-09-21T21:42:06.132 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.132 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.132 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:06.132 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796821+0000", 2024-09-21T21:42:06.132 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00012536000000000001 2024-09-21T21:42:06.132 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:06.132 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:06.132 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:06.132 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.132 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.132 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11345 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 5472256~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:06.132 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.720116+0000", 2024-09-21T21:42:06.132 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 15.384850264000001, 2024-09-21T21:42:06.133 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078667730000000005, 2024-09-21T21:42:06.133 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:06.133 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:06.133 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:06.133 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:06.133 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:06.133 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:06.133 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11345 2024-09-21T21:42:06.133 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.133 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:06.133 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.133 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:06.133 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720116+0000", 2024-09-21T21:42:06.133 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:06.134 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.134 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.134 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:06.134 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720116+0000", 2024-09-21T21:42:06.134 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:06.134 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.134 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.134 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:06.134 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720116+0000", 2024-09-21T21:42:06.134 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.04e-07 2024-09-21T21:42:06.134 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.134 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.134 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:06.134 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720118+0000", 2024-09-21T21:42:06.134 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.181e-06 2024-09-21T21:42:06.135 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.135 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.135 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:06.135 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720121+0000", 2024-09-21T21:42:06.135 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0199999999999999e-06 2024-09-21T21:42:06.135 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.135 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.135 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:06.135 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720123+0000", 2024-09-21T21:42:06.135 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.5469999999999999e-06 2024-09-21T21:42:06.135 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.135 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.135 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:06.136 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798626+0000", 2024-09-21T21:42:06.136 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078503137000000001 2024-09-21T21:42:06.136 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.136 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.136 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:06.136 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798677+0000", 2024-09-21T21:42:06.136 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0930999999999997e-05 2024-09-21T21:42:06.136 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.136 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.136 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:06.136 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798784+0000", 2024-09-21T21:42:06.136 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010671 2024-09-21T21:42:06.136 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:06.136 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:06.136 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:06.137 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.137 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.137 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11346 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 5500928~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:06.137 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.720126+0000", 2024-09-21T21:42:06.137 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 15.384840191, 2024-09-21T21:42:06.137 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078863421000000003, 2024-09-21T21:42:06.137 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:06.137 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:06.137 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:06.137 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:06.137 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:06.137 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:06.137 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11346 2024-09-21T21:42:06.137 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.137 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:06.137 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.138 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:06.138 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720126+0000", 2024-09-21T21:42:06.138 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:06.138 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.138 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.138 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:06.138 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720126+0000", 2024-09-21T21:42:06.138 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:06.138 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.138 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.138 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:06.138 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.722145+0000", 2024-09-21T21:42:06.138 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0020183219999999999 2024-09-21T21:42:06.138 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.138 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.138 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:06.139 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.722150+0000", 2024-09-21T21:42:06.139 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1760000000000001e-06 2024-09-21T21:42:06.139 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.139 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.139 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:06.139 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.722161+0000", 2024-09-21T21:42:06.139 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1566000000000001e-05 2024-09-21T21:42:06.139 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.139 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.139 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:06.139 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.722169+0000", 2024-09-21T21:42:06.139 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.2840000000000001e-06 2024-09-21T21:42:06.139 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.140 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.140 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:06.140 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798815+0000", 2024-09-21T21:42:06.140 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.076646607000000005 2024-09-21T21:42:06.140 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.140 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.140 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:06.140 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798870+0000", 2024-09-21T21:42:06.140 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4486e-05 2024-09-21T21:42:06.140 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.140 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.140 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:06.140 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798990+0000", 2024-09-21T21:42:06.140 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011998 2024-09-21T21:42:06.140 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:06.140 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:06.141 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:06.141 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.141 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.141 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13629 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7852032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:06.141 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943319+0000", 2024-09-21T21:42:06.141 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 15.161647549, 2024-09-21T21:42:06.141 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078610975999999999, 2024-09-21T21:42:06.141 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:06.141 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:06.141 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:06.141 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:06.141 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:06.141 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:06.141 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13629 2024-09-21T21:42:06.141 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.141 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:06.141 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.142 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:06.142 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943319+0000", 2024-09-21T21:42:06.142 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:06.142 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.142 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.142 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:06.142 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943319+0000", 2024-09-21T21:42:06.142 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:06.142 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.142 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.142 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:06.142 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943319+0000", 2024-09-21T21:42:06.142 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6199999999999999e-07 2024-09-21T21:42:06.142 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.142 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.143 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:06.143 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943321+0000", 2024-09-21T21:42:06.143 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1009999999999999e-06 2024-09-21T21:42:06.143 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.143 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.143 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:06.143 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943328+0000", 2024-09-21T21:42:06.143 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.1029999999999996e-06 2024-09-21T21:42:06.143 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.143 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.143 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:06.143 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943333+0000", 2024-09-21T21:42:06.143 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7709999999999997e-06 2024-09-21T21:42:06.143 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.144 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.144 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:06.144 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.021679+0000", 2024-09-21T21:42:06.144 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078345673000000005 2024-09-21T21:42:06.144 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.144 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.144 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:06.144 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.021771+0000", 2024-09-21T21:42:06.144 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2201999999999996e-05 2024-09-21T21:42:06.144 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.144 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.144 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:06.144 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.021930+0000", 2024-09-21T21:42:06.144 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00015876400000000001 2024-09-21T21:42:06.144 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:06.144 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:06.144 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:06.145 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.145 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.145 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13630 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7880704~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:06.145 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943339+0000", 2024-09-21T21:42:06.145 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 15.161627361000001, 2024-09-21T21:42:06.145 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078939410000000002, 2024-09-21T21:42:06.145 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:06.145 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:06.145 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:06.145 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:06.145 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:06.145 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:06.145 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13630 2024-09-21T21:42:06.145 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.145 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:06.145 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.145 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:06.146 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943339+0000", 2024-09-21T21:42:06.146 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:06.146 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.146 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.146 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:06.146 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943339+0000", 2024-09-21T21:42:06.146 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:06.146 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.146 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.146 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:06.146 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943339+0000", 2024-09-21T21:42:06.146 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8000000000000001e-07 2024-09-21T21:42:06.146 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.146 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.146 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:06.147 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943341+0000", 2024-09-21T21:42:06.147 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9850000000000001e-06 2024-09-21T21:42:06.147 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.147 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.147 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:06.147 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943347+0000", 2024-09-21T21:42:06.147 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2979999999999999e-06 2024-09-21T21:42:06.147 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.147 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.147 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:06.147 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943354+0000", 2024-09-21T21:42:06.147 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.2969999999999998e-06 2024-09-21T21:42:06.147 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.147 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.147 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:06.147 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.021996+0000", 2024-09-21T21:42:06.148 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078641680000000005 2024-09-21T21:42:06.148 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.148 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.148 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:06.148 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022104+0000", 2024-09-21T21:42:06.148 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000107902 2024-09-21T21:42:06.148 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.148 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.148 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:06.148 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022278+0000", 2024-09-21T21:42:06.148 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00017486800000000001 2024-09-21T21:42:06.148 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:06.148 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:06.148 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:06.148 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.149 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.149 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13631 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7909376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:06.149 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943362+0000", 2024-09-21T21:42:06.149 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 15.161604540999999, 2024-09-21T21:42:06.149 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079195644999999995, 2024-09-21T21:42:06.149 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:06.149 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:06.149 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:06.149 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:06.149 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:06.149 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:06.149 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13631 2024-09-21T21:42:06.149 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.149 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:06.149 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.150 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:06.150 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943362+0000", 2024-09-21T21:42:06.150 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:06.150 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.150 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.150 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:06.150 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943362+0000", 2024-09-21T21:42:06.150 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:06.150 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.150 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.150 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:06.150 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943362+0000", 2024-09-21T21:42:06.150 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8799999999999998e-07 2024-09-21T21:42:06.150 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.150 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.151 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:06.151 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943364+0000", 2024-09-21T21:42:06.151 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9850000000000001e-06 2024-09-21T21:42:06.151 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.151 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.151 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:06.151 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943371+0000", 2024-09-21T21:42:06.151 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0650000000000001e-06 2024-09-21T21:42:06.151 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.151 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.151 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:06.151 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943376+0000", 2024-09-21T21:42:06.151 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8250000000000004e-06 2024-09-21T21:42:06.151 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.151 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.152 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:06.152 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022329+0000", 2024-09-21T21:42:06.152 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078952888999999998 2024-09-21T21:42:06.152 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.152 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.152 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:06.152 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022419+0000", 2024-09-21T21:42:06.152 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.9738e-05 2024-09-21T21:42:06.152 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.152 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.152 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:06.152 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022558+0000", 2024-09-21T21:42:06.152 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00013875500000000001 2024-09-21T21:42:06.152 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:06.153 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:06.153 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:06.153 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.153 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.153 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13632 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7966720~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:06.153 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943382+0000", 2024-09-21T21:42:06.153 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 15.161584307, 2024-09-21T21:42:06.153 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079495727000000002, 2024-09-21T21:42:06.153 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:06.153 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:06.153 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:06.153 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:06.153 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:06.154 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:06.154 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13632 2024-09-21T21:42:06.154 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.154 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:06.154 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.154 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:06.154 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943382+0000", 2024-09-21T21:42:06.154 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:06.154 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.154 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.154 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:06.154 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943382+0000", 2024-09-21T21:42:06.154 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:06.154 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.154 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.155 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:06.155 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943382+0000", 2024-09-21T21:42:06.155 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7e-07 2024-09-21T21:42:06.155 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.155 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.155 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:06.155 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943385+0000", 2024-09-21T21:42:06.155 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2230000000000001e-06 2024-09-21T21:42:06.155 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.155 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.155 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:06.155 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943391+0000", 2024-09-21T21:42:06.155 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6810000000000001e-06 2024-09-21T21:42:06.156 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.156 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.156 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:06.156 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943397+0000", 2024-09-21T21:42:06.156 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6559999999999997e-06 2024-09-21T21:42:06.156 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.156 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.156 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:06.156 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022601+0000", 2024-09-21T21:42:06.156 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079204051999999997 2024-09-21T21:42:06.156 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.156 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.156 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:06.156 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022704+0000", 2024-09-21T21:42:06.156 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010329599999999999 2024-09-21T21:42:06.156 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.157 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.157 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:06.157 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022878+0000", 2024-09-21T21:42:06.157 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000173449 2024-09-21T21:42:06.157 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:06.157 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:06.157 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:06.157 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.157 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.157 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13633 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7938048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:06.157 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943403+0000", 2024-09-21T21:42:06.157 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 15.161563377, 2024-09-21T21:42:06.157 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079794697999999997, 2024-09-21T21:42:06.157 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:06.158 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:06.158 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:06.158 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:06.158 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:06.158 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:06.158 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13633 2024-09-21T21:42:06.158 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.158 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:06.158 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.158 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:06.158 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943403+0000", 2024-09-21T21:42:06.158 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:06.158 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.159 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.159 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:06.159 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943403+0000", 2024-09-21T21:42:06.159 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:06.159 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.159 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.159 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:06.159 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943403+0000", 2024-09-21T21:42:06.159 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.65e-07 2024-09-21T21:42:06.159 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.159 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.159 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:06.159 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943405+0000", 2024-09-21T21:42:06.159 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0020000000000001e-06 2024-09-21T21:42:06.159 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.160 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.160 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:06.160 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943412+0000", 2024-09-21T21:42:06.160 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.8379999999999996e-06 2024-09-21T21:42:06.160 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.160 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.160 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:06.160 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943417+0000", 2024-09-21T21:42:06.160 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7650000000000001e-06 2024-09-21T21:42:06.160 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.160 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.160 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:06.160 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022922+0000", 2024-09-21T21:42:06.160 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079504512999999999 2024-09-21T21:42:06.160 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.160 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.160 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:06.161 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023039+0000", 2024-09-21T21:42:06.161 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000117044 2024-09-21T21:42:06.161 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.161 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.161 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:06.161 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023198+0000", 2024-09-21T21:42:06.161 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000159171 2024-09-21T21:42:06.161 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:06.161 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:06.161 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:06.161 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.161 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.161 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13634 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7995392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:06.161 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943423+0000", 2024-09-21T21:42:06.161 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 15.161543669, 2024-09-21T21:42:06.162 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.080142358999999996, 2024-09-21T21:42:06.162 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:06.162 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:06.162 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:06.162 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:06.162 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:06.162 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:06.162 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13634 2024-09-21T21:42:06.162 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.162 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:06.162 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.162 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:06.162 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943423+0000", 2024-09-21T21:42:06.162 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:06.162 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.163 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.163 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:06.163 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943423+0000", 2024-09-21T21:42:06.163 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:06.163 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.163 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.163 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:06.163 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943423+0000", 2024-09-21T21:42:06.163 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.58e-07 2024-09-21T21:42:06.163 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.163 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.163 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:06.163 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943425+0000", 2024-09-21T21:42:06.163 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9180000000000001e-06 2024-09-21T21:42:06.164 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.164 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.164 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:06.164 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943431+0000", 2024-09-21T21:42:06.164 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4439999999999996e-06 2024-09-21T21:42:06.164 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.164 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.164 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:06.164 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943436+0000", 2024-09-21T21:42:06.164 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.887e-06 2024-09-21T21:42:06.164 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.164 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.164 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:06.164 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023266+0000", 2024-09-21T21:42:06.164 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079829622000000003 2024-09-21T21:42:06.165 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.165 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.165 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:06.165 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023384+0000", 2024-09-21T21:42:06.165 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000117658 2024-09-21T21:42:06.165 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.165 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.165 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:06.165 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023565+0000", 2024-09-21T21:42:06.165 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00018147200000000001 2024-09-21T21:42:06.165 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:06.165 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:06.165 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:06.165 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.165 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.166 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13635 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 8024064~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:06.166 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943447+0000", 2024-09-21T21:42:06.166 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 15.161519129, 2024-09-21T21:42:06.166 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.080417653000000006, 2024-09-21T21:42:06.166 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:06.166 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:06.166 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:06.166 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:06.166 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:06.166 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:06.166 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13635 2024-09-21T21:42:06.166 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.166 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:06.166 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.167 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:06.167 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943447+0000", 2024-09-21T21:42:06.167 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:06.167 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.167 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.167 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:06.167 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943447+0000", 2024-09-21T21:42:06.167 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:06.167 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.167 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.167 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:06.167 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943448+0000", 2024-09-21T21:42:06.167 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1699999999999999e-07 2024-09-21T21:42:06.167 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.168 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.168 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:06.168 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943450+0000", 2024-09-21T21:42:06.168 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1639999999999999e-06 2024-09-21T21:42:06.168 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.168 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.168 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:06.168 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943456+0000", 2024-09-21T21:42:06.168 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8379999999999998e-06 2024-09-21T21:42:06.168 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.168 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.168 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:06.169 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943460+0000", 2024-09-21T21:42:06.169 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4700000000000004e-06 2024-09-21T21:42:06.169 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.169 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.169 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:06.169 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023595+0000", 2024-09-21T21:42:06.169 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.080134869999999997 2024-09-21T21:42:06.169 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.169 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.169 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:06.169 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023688+0000", 2024-09-21T21:42:06.169 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2794000000000006e-05 2024-09-21T21:42:06.169 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.169 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.170 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:06.170 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023865+0000", 2024-09-21T21:42:06.170 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0001771 2024-09-21T21:42:06.170 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:06.170 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:06.170 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:06.170 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.170 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.170 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13636 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 8052736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:06.170 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943467+0000", 2024-09-21T21:42:06.170 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 15.161499392, 2024-09-21T21:42:06.170 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.080694399999999999, 2024-09-21T21:42:06.170 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:06.171 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:06.173 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:06.173 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:06.173 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:06.173 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:06.173 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13636 2024-09-21T21:42:06.174 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.174 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:06.174 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.174 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:06.174 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943467+0000", 2024-09-21T21:42:06.174 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:06.174 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.174 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.174 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:06.174 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943467+0000", 2024-09-21T21:42:06.174 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:06.174 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.174 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.174 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:06.175 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944940+0000", 2024-09-21T21:42:06.175 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0014725109999999999 2024-09-21T21:42:06.175 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.175 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.175 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:06.175 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944949+0000", 2024-09-21T21:42:06.175 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2930000000000006e-06 2024-09-21T21:42:06.175 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.175 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.175 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:06.175 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944958+0000", 2024-09-21T21:42:06.175 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.6899999999999998e-06 2024-09-21T21:42:06.175 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.175 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.176 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:06.176 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944966+0000", 2024-09-21T21:42:06.176 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0690000000000002e-06 2024-09-21T21:42:06.176 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.176 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.176 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:06.176 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023900+0000", 2024-09-21T21:42:06.176 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078934371000000003 2024-09-21T21:42:06.176 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.176 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.176 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:06.176 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024009+0000", 2024-09-21T21:42:06.176 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000108857 2024-09-21T21:42:06.176 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.177 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.177 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:06.177 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024161+0000", 2024-09-21T21:42:06.177 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000152609 2024-09-21T21:42:06.177 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:06.177 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:06.177 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:06.177 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.177 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.177 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13637 3.12 3.1dc31b72 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:06.177 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.944978+0000", 2024-09-21T21:42:06.177 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 15.159988457000001, 2024-09-21T21:42:06.177 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079232604999999998, 2024-09-21T21:42:06.177 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:06.178 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:06.178 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:42:06.178 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:06.178 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:06.178 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:06.178 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13637 2024-09-21T21:42:06.178 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.178 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:06.178 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.178 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:06.178 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944978+0000", 2024-09-21T21:42:06.178 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:06.178 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.179 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.179 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:06.179 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944978+0000", 2024-09-21T21:42:06.179 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:06.179 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.179 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.179 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:06.179 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944978+0000", 2024-09-21T21:42:06.179 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9100000000000004e-07 2024-09-21T21:42:06.179 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.179 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.179 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:06.179 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944981+0000", 2024-09-21T21:42:06.179 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3290000000000001e-06 2024-09-21T21:42:06.180 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.180 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.180 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:06.180 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944987+0000", 2024-09-21T21:42:06.180 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.28e-06 2024-09-21T21:42:06.180 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.180 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.180 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:06.180 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944992+0000", 2024-09-21T21:42:06.180 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8520000000000003e-06 2024-09-21T21:42:06.180 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.180 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.180 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:06.180 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024196+0000", 2024-09-21T21:42:06.181 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079203673000000002 2024-09-21T21:42:06.181 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.181 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.181 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:06.181 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024211+0000", 2024-09-21T21:42:06.181 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.4980000000000001e-05 2024-09-21T21:42:06.181 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:06.181 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:06.181 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:06.181 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.181 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.181 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13638 3.12 3.1dc31b72 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:06.182 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.945000+0000", 2024-09-21T21:42:06.182 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 15.159966329, 2024-09-21T21:42:06.182 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079257465999999999, 2024-09-21T21:42:06.182 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:06.182 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:06.182 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:42:06.182 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:06.182 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:06.182 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:06.182 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13638 2024-09-21T21:42:06.182 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.182 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:06.182 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.183 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:06.183 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945000+0000", 2024-09-21T21:42:06.183 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:06.183 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.183 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.183 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:06.183 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945000+0000", 2024-09-21T21:42:06.183 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:06.183 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.183 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.183 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:06.183 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945001+0000", 2024-09-21T21:42:06.183 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.32e-07 2024-09-21T21:42:06.184 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.184 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.184 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:06.184 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945003+0000", 2024-09-21T21:42:06.184 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2309999999999999e-06 2024-09-21T21:42:06.184 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.184 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.184 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:06.184 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945009+0000", 2024-09-21T21:42:06.184 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7450000000000001e-06 2024-09-21T21:42:06.184 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.184 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.184 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:06.185 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945013+0000", 2024-09-21T21:42:06.185 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.673e-06 2024-09-21T21:42:06.185 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.185 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.185 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:06.185 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024245+0000", 2024-09-21T21:42:06.185 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079232160999999995 2024-09-21T21:42:06.185 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.185 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.185 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:06.185 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024258+0000", 2024-09-21T21:42:06.185 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.2224e-05 2024-09-21T21:42:06.185 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:06.185 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:06.186 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:06.186 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.186 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.186 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13639 3.12 3.1dc31b72 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:06.186 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.945034+0000", 2024-09-21T21:42:06.186 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 15.159932530000001, 2024-09-21T21:42:06.186 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079260627, 2024-09-21T21:42:06.186 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:06.186 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:06.186 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:42:06.186 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:06.186 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:06.186 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:06.186 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13639 2024-09-21T21:42:06.187 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.187 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:06.187 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.187 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:06.187 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945034+0000", 2024-09-21T21:42:06.187 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:06.187 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.187 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.187 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:06.187 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945034+0000", 2024-09-21T21:42:06.187 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:06.187 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.188 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.188 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:06.188 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945034+0000", 2024-09-21T21:42:06.188 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3799999999999998e-07 2024-09-21T21:42:06.188 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.188 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.188 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:06.188 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945036+0000", 2024-09-21T21:42:06.188 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.052e-06 2024-09-21T21:42:06.188 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.188 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.188 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:06.188 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945040+0000", 2024-09-21T21:42:06.188 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1169999999999997e-06 2024-09-21T21:42:06.189 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.189 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.189 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:06.189 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945043+0000", 2024-09-21T21:42:06.189 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.8439999999999998e-06 2024-09-21T21:42:06.189 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.189 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.189 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:06.189 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024286+0000", 2024-09-21T21:42:06.189 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079242483000000002 2024-09-21T21:42:06.189 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.189 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.189 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:06.189 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024295+0000", 2024-09-21T21:42:06.190 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.7930000000000003e-06 2024-09-21T21:42:06.190 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:06.190 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:06.190 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:06.190 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.190 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.190 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13640 3.12 3.1dc31b72 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:06.190 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.945047+0000", 2024-09-21T21:42:06.190 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 15.159919007999999, 2024-09-21T21:42:06.190 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079412321999999994, 2024-09-21T21:42:06.190 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:06.190 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:06.190 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:42:06.190 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:06.191 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:06.191 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:06.191 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13640 2024-09-21T21:42:06.191 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.191 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:06.191 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.191 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:06.191 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945047+0000", 2024-09-21T21:42:06.191 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:06.191 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.191 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.191 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:06.191 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945047+0000", 2024-09-21T21:42:06.191 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:06.192 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.192 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.192 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:06.192 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945048+0000", 2024-09-21T21:42:06.192 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0499999999999999e-07 2024-09-21T21:42:06.192 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.192 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.192 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:06.192 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945049+0000", 2024-09-21T21:42:06.192 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.3939999999999999e-06 2024-09-21T21:42:06.192 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.192 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.192 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:06.192 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945053+0000", 2024-09-21T21:42:06.193 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1760000000000003e-06 2024-09-21T21:42:06.193 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.193 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.193 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:06.193 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945056+0000", 2024-09-21T21:42:06.193 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.6129999999999998e-06 2024-09-21T21:42:06.193 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.193 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.193 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:06.193 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024360+0000", 2024-09-21T21:42:06.193 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079303645000000006 2024-09-21T21:42:06.193 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:06.193 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:06.193 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:06.194 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024460+0000", 2024-09-21T21:42:06.194 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000100189 2024-09-21T21:42:06.194 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:06.194 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:06.194 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:06.194 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:06.194 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:06.194 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:06.196 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T21:42:06.196 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:06.194+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:06.296 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:06.294+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:06.345 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:06.346 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:42:06.346 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 0 2024-09-21T21:42:06.346 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:06.354 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T21:42:06.371 INFO:teuthology.orchestra.run.smithi081.stderr:noscrub is unset 2024-09-21T21:42:06.396 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:06.395+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:06.415 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T21:42:06.460 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:06.460 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:42:06.460 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:42:06.460 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:42:06.460 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:06.469 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T21:42:06.510 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:06.509+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:06.546 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0: {} 2024-09-21T21:42:06.546 INFO:teuthology.orchestra.run.smithi081.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T21:42:06.563 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1: {} 2024-09-21T21:42:06.563 INFO:teuthology.orchestra.run.smithi081.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T21:42:06.579 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2: {} 2024-09-21T21:42:06.579 INFO:teuthology.orchestra.run.smithi081.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T21:42:06.595 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3: {} 2024-09-21T21:42:06.596 INFO:teuthology.orchestra.run.smithi081.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T21:42:06.599 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:06.597+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:06.612 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4: {} 2024-09-21T21:42:06.613 INFO:teuthology.orchestra.run.smithi081.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T21:42:06.615 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:06.615 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:06.625 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T21:42:06.629 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5: {} 2024-09-21T21:42:06.629 INFO:teuthology.orchestra.run.smithi081.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T21:42:06.699 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:06.698+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:06.733 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:06.733 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:06.742 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T21:42:06.800 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:06.799+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:06.806 INFO:teuthology.orchestra.run.smithi164.stdout:checking clone #33, image image_client.0-clone33 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent34 2024-09-21T21:42:06.845 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:06.845 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:06.854 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T21:42:06.900 INFO:teuthology.orchestra.run.smithi164.stdout:4371 write 0x3250600 thru 0x325125c (0xc5d bytes) 2024-09-21T21:42:06.901 INFO:teuthology.orchestra.run.smithi164.stdout:4375 write 0x9cac000 thru 0x9cbb459 (0xf45a bytes) 2024-09-21T21:42:06.904 INFO:teuthology.orchestra.run.smithi164.stdout:4377 trunc from 0xd92d000 to 0xbff2a00 2024-09-21T21:42:06.938 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:06.937+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:06.957 INFO:teuthology.orchestra.run.smithi164.stdout:4381 punch from 0xbb11e00 to 0xbb12a00, (0xc00 bytes) 2024-09-21T21:42:06.959 INFO:teuthology.orchestra.run.smithi164.stdout:4382 read 0xad1b200 thru 0xad1e010 (0x2e11 bytes) 2024-09-21T21:42:06.959 INFO:teuthology.orchestra.run.smithi164.stdout:4385 read 0x4ff8400 thru 0x4ff999f (0x15a0 bytes) 2024-09-21T21:42:06.959 INFO:teuthology.orchestra.run.smithi164.stdout:4387 write 0xb0a1a00 thru 0xb0a3237 (0x1838 bytes) 2024-09-21T21:42:06.960 INFO:teuthology.orchestra.run.smithi164.stdout:4390 trunc from 0xbff2a00 to 0xc7efe00 2024-09-21T21:42:06.974 INFO:teuthology.orchestra.run.smithi164.stdout:4391 trunc from 0xc7efe00 to 0xb001800 2024-09-21T21:42:06.991 INFO:teuthology.orchestra.run.smithi164.stdout:4392 write 0x364be00 thru 0x364f51f (0x3720 bytes) 2024-09-21T21:42:06.991 INFO:teuthology.orchestra.run.smithi164.stdout:4393 read 0x5841600 thru 0x584d572 (0xbf73 bytes) 2024-09-21T21:42:06.993 INFO:teuthology.orchestra.run.smithi164.stdout:4395 trunc from 0xb001800 to 0xb564000 2024-09-21T21:42:07.001 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:07.000+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:07.005 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:07.005 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:07.014 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T21:42:07.015 INFO:teuthology.orchestra.run.smithi164.stdout:4396 write 0xc3ee800 thru 0xc3f9517 (0xad18 bytes) 2024-09-21T21:42:07.016 INFO:teuthology.orchestra.run.smithi164.stdout:4398 read 0x1ee000 thru 0x1f6827 (0x8828 bytes) 2024-09-21T21:42:07.017 INFO:teuthology.orchestra.run.smithi164.stdout:4400 punch from 0xbcd0200 to 0xbcd5200, (0x5000 bytes) 2024-09-21T21:42:07.017 INFO:teuthology.orchestra.run.smithi164.stdout:4401 write 0xc00800 thru 0xc09857 (0x9058 bytes) 2024-09-21T21:42:07.019 INFO:teuthology.orchestra.run.smithi164.stdout:4402 punch from 0x6326400 to 0x6335000, (0xec00 bytes) 2024-09-21T21:42:07.021 INFO:teuthology.orchestra.run.smithi164.stdout:4407 write 0xa563800 thru 0xa57188f (0xe090 bytes) 2024-09-21T21:42:07.022 INFO:teuthology.orchestra.run.smithi164.stdout:4408 read 0x1f08c00 thru 0x1f0e808 (0x5c09 bytes) 2024-09-21T21:42:07.022 INFO:teuthology.orchestra.run.smithi164.stdout:4409 write 0x3492c00 thru 0x349dcb7 (0xb0b8 bytes) 2024-09-21T21:42:07.022 INFO:teuthology.orchestra.run.smithi164.stdout:4413 read 0x230ac00 thru 0x23170ce (0xc4cf bytes) 2024-09-21T21:42:07.022 INFO:teuthology.orchestra.run.smithi164.stdout:4414 punch from 0x836cc00 to 0x836ec00, (0x2000 bytes) 2024-09-21T21:42:07.025 INFO:teuthology.orchestra.run.smithi164.stdout:4417 write 0xa939200 thru 0xa942229 (0x902a bytes) 2024-09-21T21:42:07.028 INFO:teuthology.orchestra.run.smithi164.stdout:4418 read 0xaa85400 thru 0xaa8a1b4 (0x4db5 bytes) 2024-09-21T21:42:07.029 INFO:teuthology.orchestra.run.smithi164.stdout:4419 punch from 0x8b66000 to 0x8b66200, (0x200 bytes) 2024-09-21T21:42:07.030 INFO:teuthology.orchestra.run.smithi164.stdout:4420 read 0x523e000 thru 0x5245a5e (0x7a5f bytes) 2024-09-21T21:42:07.030 INFO:teuthology.orchestra.run.smithi164.stdout:4421 write 0x86f2200 thru 0x86f9569 (0x736a bytes) 2024-09-21T21:42:07.030 INFO:teuthology.orchestra.run.smithi164.stdout:4424 punch from 0x8147800 to 0x8150a00, (0x9200 bytes) 2024-09-21T21:42:07.052 INFO:teuthology.orchestra.run.smithi164.stdout:4425 write 0xe662e00 thru 0xe66cef3 (0xa0f4 bytes) 2024-09-21T21:42:07.052 INFO:teuthology.orchestra.run.smithi164.stdout:4426 punch from 0x9343600 to 0x9345800, (0x2200 bytes) 2024-09-21T21:42:07.069 INFO:teuthology.orchestra.run.smithi164.stdout:4427 trunc from 0xe66d000 to 0x21fda00 2024-09-21T21:42:07.102 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:07.101+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:07.175 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:07.175 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:07.188 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T21:42:07.233 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:07.231+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:07.304 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:07.303+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:07.320 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:07.320 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:07.330 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T21:42:07.347 INFO:teuthology.orchestra.run.smithi164.stdout:4432 read 0x181a000 thru 0x181a1e6 (0x1e7 bytes) 2024-09-21T21:42:07.368 INFO:teuthology.orchestra.run.smithi164.stdout:4436 write 0xc128400 thru 0xc128d9f (0x9a0 bytes) 2024-09-21T21:42:07.369 INFO:teuthology.orchestra.run.smithi164.stdout:4437 punch from 0x38f6800 to 0x3900800, (0xa000 bytes) 2024-09-21T21:42:07.369 INFO:teuthology.orchestra.run.smithi164.stdout:4438 punch from 0x7835c00 to 0x7845a00, (0xfe00 bytes) 2024-09-21T21:42:07.369 INFO:teuthology.orchestra.run.smithi164.stdout:4439 read 0x25d4800 thru 0x25dc0d0 (0x78d1 bytes) 2024-09-21T21:42:07.369 INFO:teuthology.orchestra.run.smithi164.stdout:4440 write 0xa966800 thru 0xa970bb4 (0xa3b5 bytes) 2024-09-21T21:42:07.393 INFO:teuthology.orchestra.run.smithi164.stdout:4441 write 0xee02000 thru 0xee0bcb9 (0x9cba bytes) 2024-09-21T21:42:07.394 INFO:teuthology.orchestra.run.smithi164.stdout:4442 clone 36 order 22 su 131072 sc 7 2024-09-21T21:42:07.415 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:07.414+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:07.477 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:07.477 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:07.485 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T21:42:07.506 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:07.504+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:07.608 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:07.606+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:07.628 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:07.628 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:07.637 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T21:42:07.708 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:07.706+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:07.739 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:07.740 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:07.748 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T21:42:07.809 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:07.807+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:07.891 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:07.891 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:07.900 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T21:42:07.910 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:07.908+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:08.002 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:08.002 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:08.011 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T21:42:08.011 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:08.008+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:08.112 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:08.109+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:08.153 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:08.153 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:08.162 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T21:42:08.213 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:08.211+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:08.306 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:08.306 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:08.314 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:08.312+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:08.315 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T21:42:08.385 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T21:42:08.415 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:08.412+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:08.434 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:08.435 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:08.444 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T21:42:08.488 INFO:teuthology.orchestra.run.smithi164.stdout:truncating image image_client.0-clone35 from 0xee0be00 (overlap 0x21fda00) to 0x0 2024-09-21T21:42:08.516 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:08.514+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:08.554 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:08.554 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:08.563 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T21:42:08.617 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:08.614+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:08.641 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T21:42:08.710 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:08.710 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:08.718 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:08.717+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:08.719 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T21:42:08.768 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0: {} 2024-09-21T21:42:08.769 INFO:teuthology.orchestra.run.smithi081.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T21:42:08.786 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1: {} 2024-09-21T21:42:08.786 INFO:teuthology.orchestra.run.smithi081.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T21:42:08.802 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2: {} 2024-09-21T21:42:08.802 INFO:teuthology.orchestra.run.smithi081.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T21:42:08.819 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:08.816+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:08.819 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3: {} 2024-09-21T21:42:08.819 INFO:teuthology.orchestra.run.smithi081.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T21:42:08.836 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4: {} 2024-09-21T21:42:08.836 INFO:teuthology.orchestra.run.smithi081.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T21:42:08.852 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5: {} 2024-09-21T21:42:08.853 INFO:teuthology.orchestra.run.smithi081.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T21:42:08.864 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:08.864 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:42:08.864 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 0 2024-09-21T21:42:08.864 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:08.873 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T21:42:08.919 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:08.917+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:09.017 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:09.018 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:42:09.018 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:42:09.018 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:42:09.018 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:09.020 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:09.017+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:09.027 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T21:42:09.043 INFO:teuthology.orchestra.run.smithi164.stdout:checking clone #34, image image_client.0-clone34 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent35 2024-09-21T21:42:09.102 INFO:teuthology.orchestra.run.smithi164.stdout:4443 trunc from 0xee0be00 to 0xd10c200 2024-09-21T21:42:09.121 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:09.120+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:09.127 INFO:teuthology.orchestra.run.smithi164.stdout:4444 write 0x7caa400 thru 0x7cb8102 (0xdd03 bytes) 2024-09-21T21:42:09.128 INFO:teuthology.orchestra.run.smithi164.stdout:4447 write 0x7bcce00 thru 0x7bdcd66 (0xff67 bytes) 2024-09-21T21:42:09.128 INFO:teuthology.orchestra.run.smithi164.stdout:4448 write 0xc831000 thru 0xc831eb1 (0xeb2 bytes) 2024-09-21T21:42:09.128 INFO:teuthology.orchestra.run.smithi164.stdout:4451 read 0x4279a00 thru 0x427ee0d (0x540e bytes) 2024-09-21T21:42:09.129 INFO:teuthology.orchestra.run.smithi164.stdout:4452 trunc from 0xd10c200 to 0xc5a3600 2024-09-21T21:42:09.134 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:09.134 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 20, 2024-09-21T21:42:09.134 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 600, 2024-09-21T21:42:09.134 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:42:09.134 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.134 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4824.0:1564 3.2 3:4647520c:::rbd_data.1121b97f2925.0000000000000006:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:42:09.134 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:42:09.135 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 67.495723226999999, 2024-09-21T21:42:09.135 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056097347999999998, 2024-09-21T21:42:09.135 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:09.135 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:09.135 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:09.135 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:09.135 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4824", 2024-09-21T21:42:09.135 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:42:09.135 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1564 2024-09-21T21:42:09.135 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.135 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:09.135 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.135 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:09.135 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:42:09.136 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:09.136 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.136 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.136 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:09.136 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:42:09.136 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:09.136 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.136 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.136 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:09.136 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:42:09.136 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6499999999999999e-07 2024-09-21T21:42:09.136 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.136 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.136 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:09.136 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636067+0000", 2024-09-21T21:42:09.136 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3879999999999998e-06 2024-09-21T21:42:09.136 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.137 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.137 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:09.137 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636075+0000", 2024-09-21T21:42:09.137 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5480000000000004e-06 2024-09-21T21:42:09.137 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.137 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.137 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:09.137 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636080+0000", 2024-09-21T21:42:09.137 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.339e-06 2024-09-21T21:42:09.137 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.137 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.137 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:09.137 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636133+0000", 2024-09-21T21:42:09.137 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2930000000000003e-05 2024-09-21T21:42:09.137 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.137 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.138 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:09.138 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636265+0000", 2024-09-21T21:42:09.138 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000132058 2024-09-21T21:42:09.138 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.138 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.138 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:09.138 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.692162+0000", 2024-09-21T21:42:09.138 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055896620000000001 2024-09-21T21:42:09.138 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:09.138 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:09.138 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:09.138 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.138 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.139 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3258 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 1409024~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:42:09.139 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.196243+0000", 2024-09-21T21:42:09.139 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 21.935544387, 2024-09-21T21:42:09.139 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055725496999999999, 2024-09-21T21:42:09.139 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:09.139 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:09.139 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:09.139 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:09.139 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:09.139 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:09.139 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3258 2024-09-21T21:42:09.139 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.139 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:09.139 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.139 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:09.140 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196243+0000", 2024-09-21T21:42:09.140 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:09.140 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.140 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.140 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:09.140 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196243+0000", 2024-09-21T21:42:09.140 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:09.140 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.140 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.140 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:09.140 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196244+0000", 2024-09-21T21:42:09.140 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4799999999999999e-07 2024-09-21T21:42:09.140 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.140 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.141 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:09.141 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196246+0000", 2024-09-21T21:42:09.141 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1390000000000002e-06 2024-09-21T21:42:09.141 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.141 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.141 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:09.141 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196255+0000", 2024-09-21T21:42:09.141 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.5779999999999999e-06 2024-09-21T21:42:09.141 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.141 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.141 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:09.141 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196267+0000", 2024-09-21T21:42:09.141 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.2726e-05 2024-09-21T21:42:09.141 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.141 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.141 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:09.142 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.251811+0000", 2024-09-21T21:42:09.142 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055544068000000002 2024-09-21T21:42:09.142 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.142 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.142 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:09.142 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.251878+0000", 2024-09-21T21:42:09.142 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6898999999999998e-05 2024-09-21T21:42:09.142 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.142 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.142 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:09.142 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.251969+0000", 2024-09-21T21:42:09.142 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.0638999999999997e-05 2024-09-21T21:42:09.142 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:09.142 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:09.142 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:09.143 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.143 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.143 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3259 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 663552~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:42:09.143 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.196275+0000", 2024-09-21T21:42:09.143 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 21.935513079, 2024-09-21T21:42:09.143 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055889580000000001, 2024-09-21T21:42:09.143 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:09.143 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:09.143 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:09.143 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:09.143 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:09.143 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:09.143 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3259 2024-09-21T21:42:09.143 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.143 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:09.143 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.144 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:09.144 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196275+0000", 2024-09-21T21:42:09.144 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:09.144 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.144 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.144 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:09.144 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196275+0000", 2024-09-21T21:42:09.144 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:09.144 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.144 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.144 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:09.144 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196275+0000", 2024-09-21T21:42:09.144 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2199999999999999e-07 2024-09-21T21:42:09.144 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.144 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.144 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:09.144 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196277+0000", 2024-09-21T21:42:09.145 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1270000000000001e-06 2024-09-21T21:42:09.145 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.145 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.145 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:09.145 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196285+0000", 2024-09-21T21:42:09.145 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.2290000000000003e-06 2024-09-21T21:42:09.145 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.145 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.145 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:09.145 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196291+0000", 2024-09-21T21:42:09.145 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1379999999999998e-06 2024-09-21T21:42:09.145 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.145 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.145 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:09.145 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.251997+0000", 2024-09-21T21:42:09.145 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055705971999999999 2024-09-21T21:42:09.145 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.146 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.146 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:09.146 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252077+0000", 2024-09-21T21:42:09.146 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0179999999999995e-05 2024-09-21T21:42:09.146 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.146 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.146 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:09.146 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252164+0000", 2024-09-21T21:42:09.146 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.7511999999999998e-05 2024-09-21T21:42:09.146 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:09.146 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:09.146 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:09.146 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.146 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.146 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3260 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 1437696~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:42:09.146 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.196297+0000", 2024-09-21T21:42:09.147 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 21.935491017, 2024-09-21T21:42:09.147 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056071022999999998, 2024-09-21T21:42:09.147 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:09.147 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:09.147 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:09.147 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:09.147 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:09.147 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:09.147 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3260 2024-09-21T21:42:09.147 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.147 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:09.147 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.147 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:09.148 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196297+0000", 2024-09-21T21:42:09.148 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:09.148 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.148 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.148 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:09.148 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196297+0000", 2024-09-21T21:42:09.148 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:09.148 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.148 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.148 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:09.148 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196918+0000", 2024-09-21T21:42:09.148 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00062148699999999999 2024-09-21T21:42:09.148 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.148 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.148 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:09.149 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196930+0000", 2024-09-21T21:42:09.149 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1657e-05 2024-09-21T21:42:09.149 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.149 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.149 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:09.149 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196944+0000", 2024-09-21T21:42:09.149 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.4549000000000001e-05 2024-09-21T21:42:09.149 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.149 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.149 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:09.149 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196951+0000", 2024-09-21T21:42:09.149 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6749999999999996e-06 2024-09-21T21:42:09.149 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.150 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.150 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:09.150 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252208+0000", 2024-09-21T21:42:09.150 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055256656000000001 2024-09-21T21:42:09.150 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.150 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.150 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:09.150 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252278+0000", 2024-09-21T21:42:09.150 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9882000000000006e-05 2024-09-21T21:42:09.150 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.150 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.150 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:09.150 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252368+0000", 2024-09-21T21:42:09.150 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.0117000000000005e-05 2024-09-21T21:42:09.150 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:09.151 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:09.151 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:09.151 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.151 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.151 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3261 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 2842624~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:42:09.151 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.196963+0000", 2024-09-21T21:42:09.151 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 21.934824630000001, 2024-09-21T21:42:09.151 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055594613000000001, 2024-09-21T21:42:09.151 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:09.151 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:09.151 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:09.151 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:09.151 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:09.151 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:09.151 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3261 2024-09-21T21:42:09.152 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.152 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:09.152 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.152 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:09.152 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196963+0000", 2024-09-21T21:42:09.152 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:09.152 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.152 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.152 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:09.152 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196963+0000", 2024-09-21T21:42:09.152 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:09.152 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.152 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.152 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:09.153 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196964+0000", 2024-09-21T21:42:09.153 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3000000000000001e-07 2024-09-21T21:42:09.153 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.153 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.153 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:09.153 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196966+0000", 2024-09-21T21:42:09.153 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.475e-06 2024-09-21T21:42:09.153 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.153 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.153 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:09.153 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196973+0000", 2024-09-21T21:42:09.153 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5540000000000003e-06 2024-09-21T21:42:09.153 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.153 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.153 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:09.154 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196978+0000", 2024-09-21T21:42:09.154 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0719999999999999e-06 2024-09-21T21:42:09.154 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.154 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.154 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:09.154 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252395+0000", 2024-09-21T21:42:09.154 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055417301000000002 2024-09-21T21:42:09.154 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.154 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.154 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:09.154 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252468+0000", 2024-09-21T21:42:09.154 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.2692000000000001e-05 2024-09-21T21:42:09.154 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.154 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.154 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:09.155 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252558+0000", 2024-09-21T21:42:09.155 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.9988999999999995e-05 2024-09-21T21:42:09.155 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:09.155 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:09.155 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:09.155 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.155 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.155 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3262 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 2871296~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:42:09.155 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.196985+0000", 2024-09-21T21:42:09.155 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 21.934803137999999, 2024-09-21T21:42:09.155 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055777262000000001, 2024-09-21T21:42:09.155 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:09.155 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:09.155 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:09.156 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:09.156 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:09.156 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:09.156 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3262 2024-09-21T21:42:09.156 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.156 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:09.156 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.156 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:09.156 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196985+0000", 2024-09-21T21:42:09.156 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:09.156 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.156 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.156 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:09.156 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196985+0000", 2024-09-21T21:42:09.157 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:09.157 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.157 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.157 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:09.157 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197024+0000", 2024-09-21T21:42:09.157 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.9558000000000003e-05 2024-09-21T21:42:09.157 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.157 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.157 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:09.157 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197026+0000", 2024-09-21T21:42:09.157 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0899999999999999e-06 2024-09-21T21:42:09.157 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.157 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.158 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:09.158 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197031+0000", 2024-09-21T21:42:09.158 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8849999999999998e-06 2024-09-21T21:42:09.158 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.158 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.158 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:09.158 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197035+0000", 2024-09-21T21:42:09.158 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5719999999999999e-06 2024-09-21T21:42:09.158 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.158 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.158 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:09.158 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252587+0000", 2024-09-21T21:42:09.158 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055552193 2024-09-21T21:42:09.158 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.159 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.159 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:09.159 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252670+0000", 2024-09-21T21:42:09.159 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.2687000000000003e-05 2024-09-21T21:42:09.159 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.159 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.159 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:09.159 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252762+0000", 2024-09-21T21:42:09.159 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2276999999999998e-05 2024-09-21T21:42:09.159 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:09.159 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:09.159 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:09.159 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.159 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.160 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3263 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 3559424~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:42:09.160 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.197041+0000", 2024-09-21T21:42:09.160 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 21.934747203000001, 2024-09-21T21:42:09.160 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055935804999999998, 2024-09-21T21:42:09.160 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:09.160 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:09.160 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:09.160 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:09.160 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:09.160 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:09.160 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3263 2024-09-21T21:42:09.160 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.160 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:09.160 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.161 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:09.161 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197041+0000", 2024-09-21T21:42:09.161 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:09.161 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.161 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.161 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:09.161 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197041+0000", 2024-09-21T21:42:09.161 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:09.161 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.161 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.161 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:09.161 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197624+0000", 2024-09-21T21:42:09.161 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00058358100000000001 2024-09-21T21:42:09.161 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.161 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.162 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:09.162 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197629+0000", 2024-09-21T21:42:09.162 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5909999999999998e-06 2024-09-21T21:42:09.162 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.162 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.162 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:09.162 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197639+0000", 2024-09-21T21:42:09.162 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0257e-05 2024-09-21T21:42:09.162 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.162 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.162 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:09.162 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197646+0000", 2024-09-21T21:42:09.162 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.7229999999999998e-06 2024-09-21T21:42:09.162 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.162 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.163 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:09.163 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252804+0000", 2024-09-21T21:42:09.163 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055158134999999997 2024-09-21T21:42:09.163 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.163 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.163 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:09.163 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252882+0000", 2024-09-21T21:42:09.163 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7791000000000001e-05 2024-09-21T21:42:09.163 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.163 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.163 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:09.163 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252976+0000", 2024-09-21T21:42:09.163 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.4727000000000003e-05 2024-09-21T21:42:09.163 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:09.163 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:09.163 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:09.164 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.164 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.164 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3325 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:42:09.164 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.197722+0000", 2024-09-21T21:42:09.164 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 21.934065386, 2024-09-21T21:42:09.164 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055712048, 2024-09-21T21:42:09.164 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:09.164 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:09.164 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:09.164 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:09.164 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:09.164 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:09.164 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3325 2024-09-21T21:42:09.164 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.164 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:09.165 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.165 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:09.165 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197722+0000", 2024-09-21T21:42:09.165 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:09.165 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.165 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.165 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:09.165 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197722+0000", 2024-09-21T21:42:09.165 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:09.165 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.165 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.165 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:09.165 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.198052+0000", 2024-09-21T21:42:09.165 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00033006600000000001 2024-09-21T21:42:09.165 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.166 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.166 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:09.166 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.198056+0000", 2024-09-21T21:42:09.166 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7330000000000001e-06 2024-09-21T21:42:09.166 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.166 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.166 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:09.166 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.198065+0000", 2024-09-21T21:42:09.166 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.8990000000000007e-06 2024-09-21T21:42:09.166 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.166 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.166 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:09.166 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.198072+0000", 2024-09-21T21:42:09.167 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5339999999999996e-06 2024-09-21T21:42:09.167 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.167 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.167 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:09.167 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.253244+0000", 2024-09-21T21:42:09.167 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055172064999999999 2024-09-21T21:42:09.167 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.167 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.167 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:09.167 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.253342+0000", 2024-09-21T21:42:09.167 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.8175000000000001e-05 2024-09-21T21:42:09.167 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.167 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.167 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:09.167 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.253434+0000", 2024-09-21T21:42:09.168 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2576000000000003e-05 2024-09-21T21:42:09.168 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:09.168 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:09.168 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:09.168 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.168 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.168 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3437 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 925696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:42:09.168 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.205888+0000", 2024-09-21T21:42:09.168 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 21.925899921999999, 2024-09-21T21:42:09.168 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055953808000000001, 2024-09-21T21:42:09.168 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:09.168 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:09.168 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:09.168 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:09.168 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:09.169 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:09.169 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3437 2024-09-21T21:42:09.169 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.169 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:09.169 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.169 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:09.169 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205888+0000", 2024-09-21T21:42:09.169 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:09.169 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.169 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.169 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:09.169 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205888+0000", 2024-09-21T21:42:09.169 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:09.169 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.170 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.170 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:09.170 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205888+0000", 2024-09-21T21:42:09.170 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.75e-07 2024-09-21T21:42:09.170 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.170 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.170 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:09.170 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205891+0000", 2024-09-21T21:42:09.170 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3750000000000001e-06 2024-09-21T21:42:09.170 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.170 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.170 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:09.170 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205897+0000", 2024-09-21T21:42:09.170 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.285e-06 2024-09-21T21:42:09.170 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.171 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.171 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:09.171 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205902+0000", 2024-09-21T21:42:09.171 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6290000000000001e-06 2024-09-21T21:42:09.171 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.171 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.171 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:09.171 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261686+0000", 2024-09-21T21:42:09.171 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055784586999999997 2024-09-21T21:42:09.171 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.171 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.171 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:09.171 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261764+0000", 2024-09-21T21:42:09.171 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7591999999999998e-05 2024-09-21T21:42:09.171 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.172 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.172 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:09.172 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261842+0000", 2024-09-21T21:42:09.172 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7865000000000001e-05 2024-09-21T21:42:09.172 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:09.172 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:09.172 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:09.172 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.172 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.172 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3438 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 954368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:42:09.172 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.205908+0000", 2024-09-21T21:42:09.172 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 21.925880057000001, 2024-09-21T21:42:09.172 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056083169000000002, 2024-09-21T21:42:09.173 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:09.173 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:09.173 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:09.173 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:09.173 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:09.173 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:09.173 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3438 2024-09-21T21:42:09.173 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.173 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:09.173 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.173 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:09.173 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205908+0000", 2024-09-21T21:42:09.173 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:09.173 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.173 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.173 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:09.174 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205908+0000", 2024-09-21T21:42:09.174 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:09.174 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.174 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.174 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:09.174 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206431+0000", 2024-09-21T21:42:09.174 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00052282100000000003 2024-09-21T21:42:09.174 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.174 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.174 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:09.174 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206442+0000", 2024-09-21T21:42:09.174 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1695000000000001e-05 2024-09-21T21:42:09.174 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.174 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.175 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:09.175 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206452+0000", 2024-09-21T21:42:09.175 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.4560000000000006e-06 2024-09-21T21:42:09.175 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.175 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.175 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:09.175 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206458+0000", 2024-09-21T21:42:09.175 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4670000000000001e-06 2024-09-21T21:42:09.175 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.175 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.175 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:09.175 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261871+0000", 2024-09-21T21:42:09.175 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055412712000000003 2024-09-21T21:42:09.175 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.175 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.176 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:09.176 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261928+0000", 2024-09-21T21:42:09.176 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6631e-05 2024-09-21T21:42:09.176 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.176 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.176 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:09.176 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261991+0000", 2024-09-21T21:42:09.176 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3386999999999995e-05 2024-09-21T21:42:09.176 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:09.176 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:09.176 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:09.176 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.176 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.177 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3440 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 1642496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:42:09.177 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.206491+0000", 2024-09-21T21:42:09.177 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 21.925296878000001, 2024-09-21T21:42:09.177 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055633136, 2024-09-21T21:42:09.177 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:09.177 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:09.177 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:09.177 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:09.177 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:09.177 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:09.177 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3440 2024-09-21T21:42:09.177 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.177 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:09.177 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.178 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:09.178 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206491+0000", 2024-09-21T21:42:09.178 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:09.178 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.178 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.178 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:09.178 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206491+0000", 2024-09-21T21:42:09.178 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:09.178 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.178 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.178 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:09.178 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206491+0000", 2024-09-21T21:42:09.178 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6100000000000001e-07 2024-09-21T21:42:09.178 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.179 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.179 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:09.179 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206494+0000", 2024-09-21T21:42:09.179 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.9349999999999999e-06 2024-09-21T21:42:09.179 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.179 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.179 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:09.179 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206501+0000", 2024-09-21T21:42:09.179 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6939999999999998e-06 2024-09-21T21:42:09.179 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.179 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.179 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:09.179 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206506+0000", 2024-09-21T21:42:09.179 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2979999999999999e-06 2024-09-21T21:42:09.179 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.180 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.180 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:09.180 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262019+0000", 2024-09-21T21:42:09.180 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055513094999999998 2024-09-21T21:42:09.180 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.180 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.180 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:09.180 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262073+0000", 2024-09-21T21:42:09.180 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3486999999999998e-05 2024-09-21T21:42:09.180 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.180 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.180 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:09.180 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262124+0000", 2024-09-21T21:42:09.180 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1066000000000002e-05 2024-09-21T21:42:09.181 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:09.181 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:09.181 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:09.181 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.181 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.181 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3442 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 1671168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:42:09.181 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.206515+0000", 2024-09-21T21:42:09.181 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 21.925273176000001, 2024-09-21T21:42:09.181 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055792680999999997, 2024-09-21T21:42:09.181 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:09.181 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:09.181 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:09.181 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:09.181 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:09.182 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:09.182 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3442 2024-09-21T21:42:09.182 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.182 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:09.182 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.182 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:09.182 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206515+0000", 2024-09-21T21:42:09.182 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:09.182 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.182 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.182 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:09.182 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206515+0000", 2024-09-21T21:42:09.182 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:09.182 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.183 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.183 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:09.183 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206515+0000", 2024-09-21T21:42:09.183 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2500000000000001e-07 2024-09-21T21:42:09.183 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.183 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.183 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:09.183 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206517+0000", 2024-09-21T21:42:09.183 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0420000000000001e-06 2024-09-21T21:42:09.183 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.183 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.183 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:09.183 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206522+0000", 2024-09-21T21:42:09.183 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3600000000000004e-06 2024-09-21T21:42:09.184 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.184 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.184 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:09.184 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206527+0000", 2024-09-21T21:42:09.184 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.809e-06 2024-09-21T21:42:09.184 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.184 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.184 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:09.184 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262151+0000", 2024-09-21T21:42:09.184 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055623458000000001 2024-09-21T21:42:09.184 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.184 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.184 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:09.184 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262210+0000", 2024-09-21T21:42:09.184 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9120999999999999e-05 2024-09-21T21:42:09.185 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.185 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.185 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:09.185 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262307+0000", 2024-09-21T21:42:09.185 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.7466000000000005e-05 2024-09-21T21:42:09.185 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:09.185 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:09.185 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:09.185 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.185 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.185 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3444 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 2359296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:42:09.185 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.206533+0000", 2024-09-21T21:42:09.185 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 21.925254537000001, 2024-09-21T21:42:09.185 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055974239000000002, 2024-09-21T21:42:09.185 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:09.185 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:09.186 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:09.186 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:09.186 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:09.186 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:09.186 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3444 2024-09-21T21:42:09.186 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.186 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:09.186 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.186 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:09.186 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206533+0000", 2024-09-21T21:42:09.186 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:09.186 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.186 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.186 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:09.186 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206533+0000", 2024-09-21T21:42:09.187 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:09.187 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.187 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.187 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:09.187 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206710+0000", 2024-09-21T21:42:09.187 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000176293 2024-09-21T21:42:09.187 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.187 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.187 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:09.187 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206714+0000", 2024-09-21T21:42:09.187 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1989999999999999e-06 2024-09-21T21:42:09.187 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.187 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.187 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:09.188 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206723+0000", 2024-09-21T21:42:09.188 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.9199999999999993e-06 2024-09-21T21:42:09.188 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.188 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.188 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:09.188 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206729+0000", 2024-09-21T21:42:09.188 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0739999999999998e-06 2024-09-21T21:42:09.188 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.188 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.188 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:09.188 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262339+0000", 2024-09-21T21:42:09.188 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055610166000000003 2024-09-21T21:42:09.188 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.188 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.188 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:09.189 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262419+0000", 2024-09-21T21:42:09.189 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0432999999999994e-05 2024-09-21T21:42:09.189 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.189 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.189 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:09.189 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262507+0000", 2024-09-21T21:42:09.189 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.8153999999999997e-05 2024-09-21T21:42:09.189 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:09.189 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:09.189 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:09.189 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.189 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.189 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3445 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 1699840~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:42:09.189 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.206746+0000", 2024-09-21T21:42:09.190 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 21.925041613000001, 2024-09-21T21:42:09.190 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055880738999999999, 2024-09-21T21:42:09.190 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:09.190 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:09.190 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:09.190 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:09.190 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:09.190 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:09.190 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3445 2024-09-21T21:42:09.190 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.190 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:09.190 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.190 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:09.190 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206746+0000", 2024-09-21T21:42:09.190 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:09.191 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.191 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.191 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:09.191 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206746+0000", 2024-09-21T21:42:09.191 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:09.191 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.191 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.191 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:09.191 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207412+0000", 2024-09-21T21:42:09.191 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00066611999999999997 2024-09-21T21:42:09.191 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.191 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.191 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:09.191 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207417+0000", 2024-09-21T21:42:09.191 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8539999999999996e-06 2024-09-21T21:42:09.191 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.192 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.192 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:09.192 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207437+0000", 2024-09-21T21:42:09.192 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9463e-05 2024-09-21T21:42:09.192 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.192 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.192 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:09.192 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207445+0000", 2024-09-21T21:42:09.192 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.5790000000000004e-06 2024-09-21T21:42:09.192 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.192 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.192 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:09.192 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262537+0000", 2024-09-21T21:42:09.192 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055091531999999999 2024-09-21T21:42:09.193 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.193 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.193 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:09.193 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262578+0000", 2024-09-21T21:42:09.193 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1220000000000002e-05 2024-09-21T21:42:09.193 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.193 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.193 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:09.193 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262627+0000", 2024-09-21T21:42:09.193 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8971000000000003e-05 2024-09-21T21:42:09.193 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:09.193 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:09.193 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:09.193 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.194 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.194 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3451 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 3104768~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:42:09.194 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.207509+0000", 2024-09-21T21:42:09.194 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 21.924278548, 2024-09-21T21:42:09.194 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055698944, 2024-09-21T21:42:09.194 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:09.194 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:09.194 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:09.194 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:09.194 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:09.194 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:09.194 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3451 2024-09-21T21:42:09.194 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.194 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:09.194 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.195 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:09.195 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207509+0000", 2024-09-21T21:42:09.195 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:09.195 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.195 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.195 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:09.195 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207509+0000", 2024-09-21T21:42:09.195 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:09.195 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.195 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.195 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:09.195 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207661+0000", 2024-09-21T21:42:09.195 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00015181799999999999 2024-09-21T21:42:09.195 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.196 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.196 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:09.196 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207665+0000", 2024-09-21T21:42:09.196 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6380000000000002e-06 2024-09-21T21:42:09.196 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.196 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.196 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:09.196 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207674+0000", 2024-09-21T21:42:09.196 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.0950000000000002e-06 2024-09-21T21:42:09.196 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.196 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.196 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:09.196 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207689+0000", 2024-09-21T21:42:09.196 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.4904e-05 2024-09-21T21:42:09.196 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.197 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.197 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:09.197 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.263055+0000", 2024-09-21T21:42:09.197 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055366268000000003 2024-09-21T21:42:09.197 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.197 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.197 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:09.197 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.263101+0000", 2024-09-21T21:42:09.197 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6498000000000001e-05 2024-09-21T21:42:09.197 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.197 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.197 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:09.197 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.263208+0000", 2024-09-21T21:42:09.197 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000106723 2024-09-21T21:42:09.197 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:09.197 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:09.198 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:09.198 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.198 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.198 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3452 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 3133440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:42:09.198 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.207699+0000", 2024-09-21T21:42:09.198 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 21.924089119000001, 2024-09-21T21:42:09.198 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055702591000000003, 2024-09-21T21:42:09.198 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:09.198 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:09.198 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:09.198 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:09.198 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:09.198 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:09.198 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3452 2024-09-21T21:42:09.199 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.199 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:09.199 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.199 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:09.199 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207699+0000", 2024-09-21T21:42:09.199 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:09.199 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.199 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.199 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:09.199 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207699+0000", 2024-09-21T21:42:09.199 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:09.199 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.199 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.199 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:09.200 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207813+0000", 2024-09-21T21:42:09.200 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000114355 2024-09-21T21:42:09.200 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.200 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.200 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:09.200 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207826+0000", 2024-09-21T21:42:09.200 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.3293e-05 2024-09-21T21:42:09.200 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.200 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.200 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:09.200 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207835+0000", 2024-09-21T21:42:09.200 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.8310000000000007e-06 2024-09-21T21:42:09.200 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.200 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.200 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:09.200 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207842+0000", 2024-09-21T21:42:09.201 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5150000000000003e-06 2024-09-21T21:42:09.201 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.201 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.201 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:09.201 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.263240+0000", 2024-09-21T21:42:09.201 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055398007999999999 2024-09-21T21:42:09.201 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.201 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.201 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:09.201 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.263315+0000", 2024-09-21T21:42:09.201 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5729999999999995e-05 2024-09-21T21:42:09.201 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.201 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.201 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:09.201 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.263401+0000", 2024-09-21T21:42:09.202 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.5859000000000006e-05 2024-09-21T21:42:09.202 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:09.202 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:09.202 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:09.202 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.202 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.202 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3455 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 3821568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:42:09.202 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.207852+0000", 2024-09-21T21:42:09.202 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 21.923936245, 2024-09-21T21:42:09.202 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055704138, 2024-09-21T21:42:09.202 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:09.202 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:09.202 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:09.203 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:09.203 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:09.203 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:09.203 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3455 2024-09-21T21:42:09.203 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.203 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:09.203 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.203 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:09.203 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207852+0000", 2024-09-21T21:42:09.203 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:09.203 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.203 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.203 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:09.203 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207852+0000", 2024-09-21T21:42:09.203 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:09.204 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.204 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.204 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:09.204 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207983+0000", 2024-09-21T21:42:09.204 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00013183199999999999 2024-09-21T21:42:09.204 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.204 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.204 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:09.204 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207988+0000", 2024-09-21T21:42:09.204 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4290000000000003e-06 2024-09-21T21:42:09.204 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.204 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.204 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:09.204 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.208002+0000", 2024-09-21T21:42:09.204 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.4156e-05 2024-09-21T21:42:09.205 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.205 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.205 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:09.205 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.208009+0000", 2024-09-21T21:42:09.205 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.8220000000000001e-06 2024-09-21T21:42:09.205 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.205 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.205 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:09.205 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.263444+0000", 2024-09-21T21:42:09.205 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055435607999999997 2024-09-21T21:42:09.205 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.205 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.205 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:09.205 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.263493+0000", 2024-09-21T21:42:09.206 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8986999999999997e-05 2024-09-21T21:42:09.206 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.206 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.206 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:09.206 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.263556+0000", 2024-09-21T21:42:09.206 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2304000000000005e-05 2024-09-21T21:42:09.206 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:09.206 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:09.206 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:09.206 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.206 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.206 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3456 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 3850240~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:42:09.206 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.208025+0000", 2024-09-21T21:42:09.207 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 21.923763118, 2024-09-21T21:42:09.207 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055714048000000002, 2024-09-21T21:42:09.207 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:09.207 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:09.207 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:09.207 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:09.207 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:09.207 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:09.207 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3456 2024-09-21T21:42:09.207 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.207 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:09.207 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.208 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:09.208 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.208025+0000", 2024-09-21T21:42:09.208 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:09.208 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.208 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.208 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:09.208 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.208025+0000", 2024-09-21T21:42:09.208 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:09.208 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.208 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.208 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:09.208 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.208264+0000", 2024-09-21T21:42:09.208 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00023975999999999999 2024-09-21T21:42:09.209 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.209 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.209 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:09.209 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.208269+0000", 2024-09-21T21:42:09.209 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7790000000000003e-06 2024-09-21T21:42:09.209 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.209 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.209 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:09.209 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.208278+0000", 2024-09-21T21:42:09.209 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2110000000000004e-06 2024-09-21T21:42:09.209 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.209 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.209 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:09.210 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.208286+0000", 2024-09-21T21:42:09.210 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0899999999999999e-06 2024-09-21T21:42:09.210 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.210 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.210 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:09.210 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.263583+0000", 2024-09-21T21:42:09.210 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055297330999999998 2024-09-21T21:42:09.210 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.210 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.210 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:09.210 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.263647+0000", 2024-09-21T21:42:09.210 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4654999999999997e-05 2024-09-21T21:42:09.210 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.210 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.211 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:09.211 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.263739+0000", 2024-09-21T21:42:09.211 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.1222000000000002e-05 2024-09-21T21:42:09.211 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:09.211 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:09.211 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:09.211 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.211 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.211 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3468 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 266240~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:42:09.211 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.208298+0000", 2024-09-21T21:42:09.211 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 21.923489957000001, 2024-09-21T21:42:09.211 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055668849999999999, 2024-09-21T21:42:09.211 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:09.211 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:09.212 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:09.212 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:09.212 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:09.212 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:09.212 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3468 2024-09-21T21:42:09.212 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.212 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:09.212 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.212 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:09.212 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.208298+0000", 2024-09-21T21:42:09.212 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:09.212 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.212 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.212 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:09.212 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.208298+0000", 2024-09-21T21:42:09.213 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:09.213 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.213 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.213 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:09.213 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.208465+0000", 2024-09-21T21:42:09.213 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00016740700000000001 2024-09-21T21:42:09.213 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.213 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.213 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:09.213 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.208484+0000", 2024-09-21T21:42:09.213 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.8963000000000001e-05 2024-09-21T21:42:09.213 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.213 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.213 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:09.214 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.208500+0000", 2024-09-21T21:42:09.214 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.6013999999999999e-05 2024-09-21T21:42:09.214 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.214 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.214 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:09.214 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.208508+0000", 2024-09-21T21:42:09.214 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.4279999999999999e-06 2024-09-21T21:42:09.214 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.214 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.214 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:09.214 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.263780+0000", 2024-09-21T21:42:09.214 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.05527232 2024-09-21T21:42:09.214 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.214 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.214 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:09.215 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.263863+0000", 2024-09-21T21:42:09.215 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.2723000000000001e-05 2024-09-21T21:42:09.215 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.215 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.215 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:09.215 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.263967+0000", 2024-09-21T21:42:09.215 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000103995 2024-09-21T21:42:09.215 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:09.215 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:09.215 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:09.215 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.215 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.215 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3470 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 294912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:42:09.215 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.208525+0000", 2024-09-21T21:42:09.215 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 21.923262338000001, 2024-09-21T21:42:09.215 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055651946000000001, 2024-09-21T21:42:09.216 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:09.216 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:09.216 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:09.216 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:09.216 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:09.216 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:09.216 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3470 2024-09-21T21:42:09.216 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.216 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:09.216 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.216 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:09.216 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.208525+0000", 2024-09-21T21:42:09.216 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:09.216 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.216 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.216 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:09.217 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.208525+0000", 2024-09-21T21:42:09.217 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:09.217 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.217 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.217 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:09.217 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.209091+0000", 2024-09-21T21:42:09.217 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00056559299999999998 2024-09-21T21:42:09.217 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.217 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.217 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:09.217 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.209096+0000", 2024-09-21T21:42:09.217 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.882e-06 2024-09-21T21:42:09.217 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.217 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.217 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:09.217 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.209106+0000", 2024-09-21T21:42:09.217 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.8900000000000002e-06 2024-09-21T21:42:09.218 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.218 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.218 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:09.218 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.209114+0000", 2024-09-21T21:42:09.218 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.6580000000000008e-06 2024-09-21T21:42:09.218 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.218 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.218 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:09.218 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.264030+0000", 2024-09-21T21:42:09.218 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.054915692000000002 2024-09-21T21:42:09.218 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.218 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.218 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:09.218 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.264107+0000", 2024-09-21T21:42:09.218 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.6860000000000006e-05 2024-09-21T21:42:09.218 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.219 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.219 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:09.219 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.264177+0000", 2024-09-21T21:42:09.219 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0370999999999994e-05 2024-09-21T21:42:09.219 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:09.219 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:09.219 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:09.219 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:09.219 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:09.219 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:09.221 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T21:42:09.221 INFO:teuthology.orchestra.run.smithi164.stdout:4453 write 0x72f9200 thru 0x7302d98 (0x9b99 bytes) 2024-09-21T21:42:09.221 INFO:teuthology.orchestra.run.smithi164.stdout:4456 read 0x5af7000 thru 0x5afd5e9 (0x65ea bytes) 2024-09-21T21:42:09.221 INFO:teuthology.orchestra.run.smithi164.stdout:4458 punch from 0x60b4400 to 0x60b9c00, (0x5800 bytes) 2024-09-21T21:42:09.221 INFO:teuthology.orchestra.run.smithi164.stdout:4459 read 0x6226400 thru 0x622ef8c (0x8b8d bytes) 2024-09-21T21:42:09.221 INFO:teuthology.orchestra.run.smithi164.stdout:4460 read 0x7a28e00 thru 0x7a37471 (0xe672 bytes) 2024-09-21T21:42:09.221 INFO:teuthology.orchestra.run.smithi164.stdout:4461 trunc from 0xc5a3600 to 0x4637200 2024-09-21T21:42:09.221 INFO:teuthology.orchestra.run.smithi164.stdout:4462 trunc from 0x4637200 to 0x88ec00 2024-09-21T21:42:09.222 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:09.220+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:09.233 INFO:teuthology.orchestra.run.smithi164.stdout:4463 read 0x475600 thru 0x483db7 (0xe7b8 bytes) 2024-09-21T21:42:09.256 INFO:teuthology.orchestra.run.smithi164.stdout:4466 write 0x830ba00 thru 0x830f4fc (0x3afd bytes) 2024-09-21T21:42:09.257 INFO:teuthology.orchestra.run.smithi164.stdout:4467 read 0x6a68c00 thru 0x6a6d5aa (0x49ab bytes) 2024-09-21T21:42:09.276 INFO:teuthology.orchestra.run.smithi164.stdout:4469 write 0xe306000 thru 0xe30e13d (0x813e bytes) 2024-09-21T21:42:09.294 INFO:teuthology.orchestra.run.smithi164.stdout:4471 trunc from 0xe30e200 to 0x293400 2024-09-21T21:42:09.323 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:09.320+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:09.323 INFO:teuthology.orchestra.run.smithi164.stdout:4473 trunc from 0x293400 to 0x723a600 2024-09-21T21:42:09.333 INFO:teuthology.orchestra.run.smithi164.stdout:4479 punch from 0x370ce00 to 0x3713400, (0x6600 bytes) 2024-09-21T21:42:09.335 INFO:teuthology.orchestra.run.smithi164.stdout:4485 punch from 0x7095c00 to 0x70a5600, (0xfa00 bytes) 2024-09-21T21:42:09.335 INFO:teuthology.orchestra.run.smithi164.stdout:4486 write 0x284a800 thru 0x2852e6e (0x866f bytes) 2024-09-21T21:42:09.336 INFO:teuthology.orchestra.run.smithi164.stdout:4487 read 0x4200200 thru 0x4202b27 (0x2928 bytes) 2024-09-21T21:42:09.336 INFO:teuthology.orchestra.run.smithi164.stdout:4488 punch from 0x143c00 to 0x151600, (0xda00 bytes) 2024-09-21T21:42:09.337 INFO:teuthology.orchestra.run.smithi164.stdout:4491 read 0xdd2e00 thru 0xdda1ba (0x73bb bytes) 2024-09-21T21:42:09.337 INFO:teuthology.orchestra.run.smithi164.stdout:4493 read 0x3a34a00 thru 0x3a41ec5 (0xd4c6 bytes) 2024-09-21T21:42:09.337 INFO:teuthology.orchestra.run.smithi164.stdout:4494 read 0x2c14200 thru 0x2c150f9 (0xefa bytes) 2024-09-21T21:42:09.337 INFO:teuthology.orchestra.run.smithi164.stdout:4495 read 0x233d000 thru 0x23418a7 (0x48a8 bytes) 2024-09-21T21:42:09.337 INFO:teuthology.orchestra.run.smithi164.stdout:4501 punch from 0x695aa00 to 0x6968400, (0xda00 bytes) 2024-09-21T21:42:09.337 INFO:teuthology.orchestra.run.smithi164.stdout:4502 read 0x5709a00 thru 0x570fb7d (0x617e bytes) 2024-09-21T21:42:09.337 INFO:teuthology.orchestra.run.smithi164.stdout:4504 read 0x25b0c00 thru 0x25b6e3b (0x623c bytes) 2024-09-21T21:42:09.337 INFO:teuthology.orchestra.run.smithi164.stdout:4506 write 0x6ccdc00 thru 0x6cd9418 (0xb819 bytes) 2024-09-21T21:42:09.352 INFO:teuthology.orchestra.run.smithi164.stdout:4507 write 0x8582200 thru 0x8583d8c (0x1b8d bytes) 2024-09-21T21:42:09.354 INFO:teuthology.orchestra.run.smithi164.stdout:4508 read 0x7ef6200 thru 0x7efe15a (0x7f5b bytes) 2024-09-21T21:42:09.354 INFO:teuthology.orchestra.run.smithi164.stdout:4509 write 0x4ad5400 thru 0x4ae38b3 (0xe4b4 bytes) 2024-09-21T21:42:09.354 INFO:teuthology.orchestra.run.smithi164.stdout:4510 read 0x511f400 thru 0x512e44b (0xf04c bytes) 2024-09-21T21:42:09.359 INFO:teuthology.orchestra.run.smithi164.stdout:4511 trunc from 0x8583e00 to 0x40e8a00 2024-09-21T21:42:09.370 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:09.370 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:42:09.370 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 0 2024-09-21T21:42:09.370 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:09.378 INFO:teuthology.orchestra.run.smithi081.stderr:nodeep-scrub is unset 2024-09-21T21:42:09.381 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T21:42:09.394 INFO:teuthology.orchestra.run.smithi164.stdout:4515 trunc from 0x40e8a00 to 0xc5d1e00 2024-09-21T21:42:09.404 INFO:teuthology.orchestra.run.smithi164.stdout:4517 read 0x71f5c00 thru 0x71fb859 (0x5c5a bytes) 2024-09-21T21:42:09.405 INFO:teuthology.orchestra.run.smithi164.stdout:4518 read 0x98bce00 thru 0x98bf918 (0x2b19 bytes) 2024-09-21T21:42:09.405 INFO:teuthology.orchestra.run.smithi164.stdout:4522 write 0x1a69a00 thru 0x1a773cf (0xd9d0 bytes) 2024-09-21T21:42:09.405 INFO:teuthology.orchestra.run.smithi164.stdout:4527 write 0x7350400 thru 0x7353dde (0x39df bytes) 2024-09-21T21:42:09.421 INFO:teuthology.orchestra.run.smithi164.stdout:4528 write 0xd74e000 thru 0xd75a66e (0xc66f bytes) 2024-09-21T21:42:09.422 INFO:teuthology.orchestra.run.smithi164.stdout:4529 read 0x7056c00 thru 0x705d395 (0x6796 bytes) 2024-09-21T21:42:09.422 INFO:teuthology.orchestra.run.smithi164.stdout:4530 read 0x5ec7400 thru 0x5ed54e9 (0xe0ea bytes) 2024-09-21T21:42:09.422 INFO:teuthology.orchestra.run.smithi164.stdout:4531 write 0x3e3a800 thru 0x3e3afef (0x7f0 bytes) 2024-09-21T21:42:09.423 INFO:teuthology.orchestra.run.smithi164.stdout:4532 write 0x69fdc00 thru 0x6a04c96 (0x7097 bytes) 2024-09-21T21:42:09.423 INFO:teuthology.orchestra.run.smithi164.stdout:4533 write 0x8a99000 thru 0x8aa7664 (0xe665 bytes) 2024-09-21T21:42:09.423 INFO:teuthology.orchestra.run.smithi164.stdout:4534 read 0x96b7200 thru 0x96c6533 (0xf334 bytes) 2024-09-21T21:42:09.423 INFO:teuthology.orchestra.run.smithi164.stdout:4535 punch from 0x1b6e00 to 0x1c0800, (0x9a00 bytes) 2024-09-21T21:42:09.424 INFO:teuthology.orchestra.run.smithi164.stdout:4538 read 0xb121800 thru 0xb12838f (0x6b90 bytes) 2024-09-21T21:42:09.424 INFO:teuthology.orchestra.run.smithi164.stdout:4539 read 0x5e2d200 thru 0x5e392fa (0xc0fb bytes) 2024-09-21T21:42:09.424 INFO:teuthology.orchestra.run.smithi164.stdout:4540 read 0x1478000 thru 0x14854d5 (0xd4d6 bytes) 2024-09-21T21:42:09.424 INFO:teuthology.orchestra.run.smithi164.stdout:4542 read 0xcfc2000 thru 0xcfcb7fb (0x97fc bytes) 2024-09-21T21:42:09.424 INFO:teuthology.orchestra.run.smithi164.stdout:4543 write 0x35a9e00 thru 0x35af625 (0x5826 bytes) 2024-09-21T21:42:09.425 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:09.421+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:09.427 INFO:teuthology.orchestra.run.smithi164.stdout:4545 trunc from 0xd75a800 to 0xb08d400 2024-09-21T21:42:09.453 INFO:teuthology.orchestra.run.smithi164.stdout:4547 punch from 0xad4b600 to 0xad4e000, (0x2a00 bytes) 2024-09-21T21:42:09.462 INFO:teuthology.orchestra.run.smithi164.stdout:4548 read 0x785200 thru 0x78947a (0x427b bytes) 2024-09-21T21:42:09.462 INFO:teuthology.orchestra.run.smithi164.stdout:4549 write 0xa241400 thru 0xa241bf0 (0x7f1 bytes) 2024-09-21T21:42:09.462 INFO:teuthology.orchestra.run.smithi164.stdout:4550 write 0xd4e800 thru 0xd5c57c (0xdd7d bytes) 2024-09-21T21:42:09.463 INFO:teuthology.orchestra.run.smithi164.stdout:4551 read 0x26cf400 thru 0x26d9f34 (0xab35 bytes) 2024-09-21T21:42:09.463 INFO:teuthology.orchestra.run.smithi164.stdout:4552 read 0x6777a00 thru 0x6786df1 (0xf3f2 bytes) 2024-09-21T21:42:09.463 INFO:teuthology.orchestra.run.smithi164.stdout:4553 write 0x4babe00 thru 0x4bb8fe9 (0xd1ea bytes) 2024-09-21T21:42:09.464 INFO:teuthology.orchestra.run.smithi164.stdout:4554 read 0x144a800 thru 0x144ffa6 (0x57a7 bytes) 2024-09-21T21:42:09.464 INFO:teuthology.orchestra.run.smithi164.stdout:4557 punch from 0x1af6e00 to 0x1afbc00, (0x4e00 bytes) 2024-09-21T21:42:09.465 INFO:teuthology.orchestra.run.smithi164.stdout:4560 punch from 0x9312800 to 0x9322400, (0xfc00 bytes) 2024-09-21T21:42:09.465 INFO:teuthology.orchestra.run.smithi164.stdout:4562 read 0x3990c00 thru 0x3995671 (0x4a72 bytes) 2024-09-21T21:42:09.465 INFO:teuthology.orchestra.run.smithi164.stdout:4564 write 0x251be00 thru 0x2528c21 (0xce22 bytes) 2024-09-21T21:42:09.465 INFO:teuthology.orchestra.run.smithi164.stdout:4566 read 0x84eae00 thru 0x84f7f88 (0xd189 bytes) 2024-09-21T21:42:09.465 INFO:teuthology.orchestra.run.smithi164.stdout:4567 punch from 0xfb7200 to 0xfb9800, (0x2600 bytes) 2024-09-21T21:42:09.466 INFO:teuthology.orchestra.run.smithi164.stdout:4569 punch from 0x254bc00 to 0x2556a00, (0xae00 bytes) 2024-09-21T21:42:09.466 INFO:teuthology.orchestra.run.smithi164.stdout:4570 write 0x2108a00 thru 0x210ce85 (0x4486 bytes) 2024-09-21T21:42:09.484 INFO:teuthology.orchestra.run.smithi164.stdout:4571 write 0xc055800 thru 0xc0624ee (0xccef bytes) 2024-09-21T21:42:09.484 INFO:teuthology.orchestra.run.smithi164.stdout:4573 punch from 0xa668400 to 0xa670400, (0x8000 bytes) 2024-09-21T21:42:09.489 INFO:teuthology.orchestra.run.smithi164.stdout:4574 read 0x2bae200 thru 0x2bae6a0 (0x4a1 bytes) 2024-09-21T21:42:09.492 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:09.492 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:42:09.492 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:42:09.492 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:42:09.492 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:09.501 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T21:42:09.504 INFO:teuthology.orchestra.run.smithi164.stdout:4575 trunc from 0xc062600 to 0x18fe00 2024-09-21T21:42:09.524 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:09.522+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:09.555 INFO:teuthology.orchestra.run.smithi164.stdout:4579 write 0x1802800 thru 0x1802fa8 (0x7a9 bytes) 2024-09-21T21:42:09.578 INFO:teuthology.orchestra.run.smithi164.stdout:4580 write 0xa3efc00 thru 0xa3fa07f (0xa480 bytes) 2024-09-21T21:42:09.585 INFO:teuthology.orchestra.run.smithi164.stdout:4585 read 0x649c800 thru 0x64ab869 (0xf06a bytes) 2024-09-21T21:42:09.585 INFO:teuthology.orchestra.run.smithi164.stdout:4587 read 0xb53a00 thru 0xb61d14 (0xe315 bytes) 2024-09-21T21:42:09.595 INFO:teuthology.orchestra.run.smithi164.stdout:4588 trunc from 0xa3fa200 to 0x2c91000 2024-09-21T21:42:09.615 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:09.615 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 20, 2024-09-21T21:42:09.615 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 600, 2024-09-21T21:42:09.615 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:42:09.615 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.615 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13295 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3330048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:09.615 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.796526+0000", 2024-09-21T21:42:09.615 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 18.816178466, 2024-09-21T21:42:09.615 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071630928999999996, 2024-09-21T21:42:09.615 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:09.616 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:09.616 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:09.616 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:09.616 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:09.616 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:09.616 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13295 2024-09-21T21:42:09.616 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.616 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:09.616 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.616 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:09.616 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796526+0000", 2024-09-21T21:42:09.616 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:09.616 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.616 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.616 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:09.617 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796526+0000", 2024-09-21T21:42:09.617 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:09.617 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.617 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.617 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:09.617 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796526+0000", 2024-09-21T21:42:09.617 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5900000000000003e-07 2024-09-21T21:42:09.617 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.617 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.617 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:09.617 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796529+0000", 2024-09-21T21:42:09.617 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3910000000000001e-06 2024-09-21T21:42:09.617 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.617 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.617 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:09.617 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796537+0000", 2024-09-21T21:42:09.618 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.2819999999999996e-06 2024-09-21T21:42:09.618 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.618 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.618 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:09.618 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796541+0000", 2024-09-21T21:42:09.618 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.9400000000000004e-06 2024-09-21T21:42:09.618 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.618 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.618 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:09.618 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.868008+0000", 2024-09-21T21:42:09.618 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071467586999999999 2024-09-21T21:42:09.618 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.618 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.618 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:09.619 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.868061+0000", 2024-09-21T21:42:09.619 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2349000000000003e-05 2024-09-21T21:42:09.619 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.619 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.619 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:09.619 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.868157+0000", 2024-09-21T21:42:09.619 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6020999999999994e-05 2024-09-21T21:42:09.619 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:09.619 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:09.619 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:09.619 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.619 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.619 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13296 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 4046848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:09.619 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.796643+0000", 2024-09-21T21:42:09.619 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 18.816061647000001, 2024-09-21T21:42:09.620 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071682019999999999, 2024-09-21T21:42:09.620 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:09.620 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:09.620 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:09.620 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:09.620 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:09.620 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:09.620 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13296 2024-09-21T21:42:09.620 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.620 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:09.620 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.620 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:09.620 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796643+0000", 2024-09-21T21:42:09.620 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:09.620 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.621 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.621 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:09.621 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796643+0000", 2024-09-21T21:42:09.621 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:09.621 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.621 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.621 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:09.621 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796643+0000", 2024-09-21T21:42:09.621 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.7199999999999998e-07 2024-09-21T21:42:09.621 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.621 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.621 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:09.621 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796647+0000", 2024-09-21T21:42:09.621 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6770000000000001e-06 2024-09-21T21:42:09.621 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.622 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.622 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:09.622 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796656+0000", 2024-09-21T21:42:09.622 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2059999999999996e-06 2024-09-21T21:42:09.622 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.622 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.622 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:09.622 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796664+0000", 2024-09-21T21:42:09.622 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7919999999999993e-06 2024-09-21T21:42:09.622 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.622 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.622 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:09.622 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.868186+0000", 2024-09-21T21:42:09.623 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071522273999999997 2024-09-21T21:42:09.623 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.623 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.623 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:09.623 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.868239+0000", 2024-09-21T21:42:09.623 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2787999999999997e-05 2024-09-21T21:42:09.623 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.623 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.623 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:09.623 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.868325+0000", 2024-09-21T21:42:09.623 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.5611000000000004e-05 2024-09-21T21:42:09.623 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:09.623 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:09.624 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:09.624 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.624 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.624 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13561 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 1728512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:09.624 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.823991+0000", 2024-09-21T21:42:09.624 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 18.788713346000002, 2024-09-21T21:42:09.624 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071654762999999996, 2024-09-21T21:42:09.624 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:09.624 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:09.624 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:09.624 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:09.624 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:09.625 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:09.625 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13561 2024-09-21T21:42:09.625 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.625 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:09.625 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.625 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:09.625 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.823991+0000", 2024-09-21T21:42:09.625 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:09.625 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.625 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.625 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:09.625 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.823991+0000", 2024-09-21T21:42:09.625 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:09.625 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.626 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.626 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:09.626 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.823991+0000", 2024-09-21T21:42:09.626 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9299999999999998e-07 2024-09-21T21:42:09.626 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.626 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.626 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:09.626 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.823994+0000", 2024-09-21T21:42:09.626 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2369999999999999e-06 2024-09-21T21:42:09.626 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.626 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.626 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:09.626 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824000+0000", 2024-09-21T21:42:09.626 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.55e-06 2024-09-21T21:42:09.627 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.627 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.627 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:09.627 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824005+0000", 2024-09-21T21:42:09.627 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4329999999999997e-06 2024-09-21T21:42:09.627 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.627 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.627 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:09.627 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.895360+0000", 2024-09-21T21:42:09.627 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071355426 2024-09-21T21:42:09.627 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.627 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.627 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:09.627 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.895477+0000", 2024-09-21T21:42:09.627 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0001171 2024-09-21T21:42:09.628 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.628 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.628 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:09.628 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.895646+0000", 2024-09-21T21:42:09.628 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000168524 2024-09-21T21:42:09.628 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:09.628 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:09.628 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:09.628 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.628 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.628 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13562 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 1757184~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:09.628 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.824011+0000", 2024-09-21T21:42:09.628 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 18.788693469000002, 2024-09-21T21:42:09.628 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072020453999999998, 2024-09-21T21:42:09.629 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:09.629 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:09.629 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:09.629 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:09.629 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:09.629 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:09.629 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13562 2024-09-21T21:42:09.629 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.629 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:09.629 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.629 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:09.629 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824011+0000", 2024-09-21T21:42:09.629 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:09.629 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.630 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.630 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:09.630 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824011+0000", 2024-09-21T21:42:09.630 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:09.630 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.630 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.630 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:09.630 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824011+0000", 2024-09-21T21:42:09.630 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5600000000000001e-07 2024-09-21T21:42:09.630 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.630 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.630 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:09.630 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824013+0000", 2024-09-21T21:42:09.630 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9800000000000001e-06 2024-09-21T21:42:09.631 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.631 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.631 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:09.631 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824019+0000", 2024-09-21T21:42:09.631 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7339999999999996e-06 2024-09-21T21:42:09.631 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.631 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.631 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:09.631 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824023+0000", 2024-09-21T21:42:09.631 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3680000000000004e-06 2024-09-21T21:42:09.631 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.631 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.631 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:09.631 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.895699+0000", 2024-09-21T21:42:09.632 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071675597999999993 2024-09-21T21:42:09.632 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.632 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.632 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:09.632 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.895830+0000", 2024-09-21T21:42:09.632 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000130858 2024-09-21T21:42:09.632 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.632 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.632 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:09.632 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896031+0000", 2024-09-21T21:42:09.632 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00020155999999999999 2024-09-21T21:42:09.632 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:09.632 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:09.632 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:09.632 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.632 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.633 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13563 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 323584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:09.633 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.824029+0000", 2024-09-21T21:42:09.633 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 18.788675029, 2024-09-21T21:42:09.633 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072307563000000005, 2024-09-21T21:42:09.633 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:09.633 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:09.633 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:09.633 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:09.633 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:09.633 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:09.633 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13563 2024-09-21T21:42:09.633 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.633 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:09.633 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.633 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:09.634 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824029+0000", 2024-09-21T21:42:09.634 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:09.634 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.634 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.634 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:09.634 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824029+0000", 2024-09-21T21:42:09.634 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:09.634 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.634 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.634 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:09.634 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824030+0000", 2024-09-21T21:42:09.634 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1699999999999999e-07 2024-09-21T21:42:09.634 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.634 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.635 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:09.635 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824032+0000", 2024-09-21T21:42:09.635 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0320000000000002e-06 2024-09-21T21:42:09.635 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.635 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.635 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:09.635 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824038+0000", 2024-09-21T21:42:09.635 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8769999999999998e-06 2024-09-21T21:42:09.635 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.635 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.635 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:09.635 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824042+0000", 2024-09-21T21:42:09.635 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5650000000000003e-06 2024-09-21T21:42:09.635 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.635 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.636 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:09.636 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896058+0000", 2024-09-21T21:42:09.636 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072015941 2024-09-21T21:42:09.636 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.636 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.636 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:09.636 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896164+0000", 2024-09-21T21:42:09.636 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010593499999999999 2024-09-21T21:42:09.636 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.636 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.636 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:09.636 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896337+0000", 2024-09-21T21:42:09.636 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00017279599999999999 2024-09-21T21:42:09.636 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:09.636 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:09.637 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:09.637 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.637 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.637 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13564 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 983040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:09.637 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.824048+0000", 2024-09-21T21:42:09.637 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 18.788655953999999, 2024-09-21T21:42:09.637 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072491193999999995, 2024-09-21T21:42:09.637 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:09.637 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:09.637 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:09.637 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:09.637 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:09.637 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:09.637 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13564 2024-09-21T21:42:09.637 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.637 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:09.638 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.638 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:09.638 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824048+0000", 2024-09-21T21:42:09.638 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:09.638 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.638 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.638 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:09.638 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824048+0000", 2024-09-21T21:42:09.638 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:09.638 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.638 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.638 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:09.638 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824663+0000", 2024-09-21T21:42:09.638 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00061494299999999996 2024-09-21T21:42:09.639 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.639 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.639 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:09.639 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824669+0000", 2024-09-21T21:42:09.639 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2900000000000002e-06 2024-09-21T21:42:09.639 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.639 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.639 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:09.639 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824679+0000", 2024-09-21T21:42:09.639 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0338e-05 2024-09-21T21:42:09.639 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.639 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.639 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:09.639 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824687+0000", 2024-09-21T21:42:09.639 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.9049999999999997e-06 2024-09-21T21:42:09.640 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.640 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.640 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:09.640 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896381+0000", 2024-09-21T21:42:09.640 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071694305999999999 2024-09-21T21:42:09.640 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.640 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.640 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:09.640 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896445+0000", 2024-09-21T21:42:09.640 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3869999999999997e-05 2024-09-21T21:42:09.640 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.640 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.640 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:09.641 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896540+0000", 2024-09-21T21:42:09.641 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.4542000000000004e-05 2024-09-21T21:42:09.641 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:09.641 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:09.641 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:09.641 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.641 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.641 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13565 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3190784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:09.641 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.824704+0000", 2024-09-21T21:42:09.641 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 18.788000083, 2024-09-21T21:42:09.641 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072014745000000005, 2024-09-21T21:42:09.641 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:09.641 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:09.641 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:09.641 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:09.642 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:09.642 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:09.642 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13565 2024-09-21T21:42:09.642 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.642 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:09.642 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.642 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:09.642 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824704+0000", 2024-09-21T21:42:09.642 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:09.642 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.642 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.642 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:09.643 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824704+0000", 2024-09-21T21:42:09.643 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:09.643 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.643 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.643 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:09.643 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824705+0000", 2024-09-21T21:42:09.643 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8999999999999997e-07 2024-09-21T21:42:09.643 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.643 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.643 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:09.643 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824707+0000", 2024-09-21T21:42:09.643 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.7669999999999999e-06 2024-09-21T21:42:09.643 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.643 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.643 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:09.644 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824715+0000", 2024-09-21T21:42:09.644 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7700000000000001e-06 2024-09-21T21:42:09.644 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.644 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.644 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:09.644 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824721+0000", 2024-09-21T21:42:09.644 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9560000000000002e-06 2024-09-21T21:42:09.644 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.644 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.644 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:09.644 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896570+0000", 2024-09-21T21:42:09.644 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071848870999999995 2024-09-21T21:42:09.644 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.644 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.644 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:09.645 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896626+0000", 2024-09-21T21:42:09.645 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5578000000000002e-05 2024-09-21T21:42:09.645 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.645 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.645 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:09.645 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896719+0000", 2024-09-21T21:42:09.645 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.3313000000000006e-05 2024-09-21T21:42:09.645 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:09.645 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:09.645 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:09.645 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.645 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.645 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13566 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3162112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:09.645 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.824729+0000", 2024-09-21T21:42:09.646 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 18.787975196000001, 2024-09-21T21:42:09.646 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072356907999999998, 2024-09-21T21:42:09.646 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:09.646 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:09.646 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:09.646 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:09.646 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:09.646 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:09.646 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13566 2024-09-21T21:42:09.646 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.646 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:09.646 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.646 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:09.646 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824729+0000", 2024-09-21T21:42:09.646 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:09.646 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.647 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.647 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:09.647 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824729+0000", 2024-09-21T21:42:09.647 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:09.647 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.647 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.647 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:09.647 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825630+0000", 2024-09-21T21:42:09.647 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00090139400000000002 2024-09-21T21:42:09.647 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.647 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.647 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:09.647 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825634+0000", 2024-09-21T21:42:09.647 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4960000000000001e-06 2024-09-21T21:42:09.647 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.647 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.648 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:09.648 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825644+0000", 2024-09-21T21:42:09.648 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.9650000000000003e-06 2024-09-21T21:42:09.648 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.648 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.648 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:09.649 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825649+0000", 2024-09-21T21:42:09.649 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0359999999999997e-06 2024-09-21T21:42:09.649 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.650 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.650 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:09.650 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896757+0000", 2024-09-21T21:42:09.650 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071107717000000001 2024-09-21T21:42:09.650 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.650 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.650 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:09.650 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896878+0000", 2024-09-21T21:42:09.650 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000120834 2024-09-21T21:42:09.650 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.650 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.650 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:09.650 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897086+0000", 2024-09-21T21:42:09.650 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00020846600000000001 2024-09-21T21:42:09.650 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:09.651 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:09.651 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:09.651 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.651 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.651 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13567 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3878912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:09.651 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.825658+0000", 2024-09-21T21:42:09.651 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 18.787046196999999, 2024-09-21T21:42:09.651 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071597694000000003, 2024-09-21T21:42:09.651 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:09.651 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:09.651 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:09.651 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:09.651 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:09.652 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:09.652 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13567 2024-09-21T21:42:09.652 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.652 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:09.652 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.652 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:09.652 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825658+0000", 2024-09-21T21:42:09.652 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:09.652 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.652 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.652 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:09.652 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825658+0000", 2024-09-21T21:42:09.652 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:09.652 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.652 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.653 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:09.653 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825658+0000", 2024-09-21T21:42:09.653 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.5699999999999999e-07 2024-09-21T21:42:09.653 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.653 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.653 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:09.653 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825660+0000", 2024-09-21T21:42:09.653 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.638e-06 2024-09-21T21:42:09.653 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.653 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.653 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:09.653 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825664+0000", 2024-09-21T21:42:09.653 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4340000000000002e-06 2024-09-21T21:42:09.653 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.654 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.654 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:09.654 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825668+0000", 2024-09-21T21:42:09.654 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.1669999999999999e-06 2024-09-21T21:42:09.654 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.654 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.654 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:09.654 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897113+0000", 2024-09-21T21:42:09.654 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071445358 2024-09-21T21:42:09.654 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.654 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.654 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:09.654 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897169+0000", 2024-09-21T21:42:09.655 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5813000000000002e-05 2024-09-21T21:42:09.655 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.655 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.655 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:09.655 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897256+0000", 2024-09-21T21:42:09.655 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.7027000000000006e-05 2024-09-21T21:42:09.655 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:09.655 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:09.655 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:09.655 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.655 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.655 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13568 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3907584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:09.655 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.825671+0000", 2024-09-21T21:42:09.655 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 18.787032787000001, 2024-09-21T21:42:09.655 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071794595000000003, 2024-09-21T21:42:09.655 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:09.656 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:09.656 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:09.656 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:09.656 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:09.656 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:09.656 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13568 2024-09-21T21:42:09.656 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.656 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:09.656 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.656 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:09.656 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825671+0000", 2024-09-21T21:42:09.657 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:09.657 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.657 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.657 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:09.657 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825671+0000", 2024-09-21T21:42:09.657 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:09.657 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.657 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.657 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:09.657 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825672+0000", 2024-09-21T21:42:09.657 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2999999999999999e-07 2024-09-21T21:42:09.657 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.657 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.657 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:09.658 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825673+0000", 2024-09-21T21:42:09.658 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.246e-06 2024-09-21T21:42:09.658 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.658 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.658 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:09.658 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825693+0000", 2024-09-21T21:42:09.658 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9789999999999999e-05 2024-09-21T21:42:09.658 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.658 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.658 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:09.658 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825697+0000", 2024-09-21T21:42:09.658 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2379999999999999e-06 2024-09-21T21:42:09.658 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.659 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.659 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:09.659 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897284+0000", 2024-09-21T21:42:09.659 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071586710999999997 2024-09-21T21:42:09.659 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.659 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.659 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:09.659 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897337+0000", 2024-09-21T21:42:09.659 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2800000000000003e-05 2024-09-21T21:42:09.659 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.659 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.659 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:09.659 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897466+0000", 2024-09-21T21:42:09.659 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00012957999999999999 2024-09-21T21:42:09.659 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:09.660 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:09.660 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:09.660 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.660 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.660 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13574 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 352256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:09.660 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.825702+0000", 2024-09-21T21:42:09.660 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 18.787002639000001, 2024-09-21T21:42:09.660 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072080377000000001, 2024-09-21T21:42:09.660 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:09.660 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:09.660 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:09.660 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:09.660 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:09.660 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:09.661 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13574 2024-09-21T21:42:09.661 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.661 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:09.661 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.661 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:09.661 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825702+0000", 2024-09-21T21:42:09.661 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:09.661 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.661 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.661 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:09.661 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825702+0000", 2024-09-21T21:42:09.661 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:09.661 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.661 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.661 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:09.661 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825943+0000", 2024-09-21T21:42:09.662 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00024105200000000001 2024-09-21T21:42:09.662 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.662 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.662 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:09.662 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825946+0000", 2024-09-21T21:42:09.662 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.743e-06 2024-09-21T21:42:09.662 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.662 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.662 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:09.662 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825956+0000", 2024-09-21T21:42:09.662 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6609999999999995e-06 2024-09-21T21:42:09.662 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.662 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.662 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:09.662 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825963+0000", 2024-09-21T21:42:09.662 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4609999999999996e-06 2024-09-21T21:42:09.663 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.663 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.663 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:09.663 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897502+0000", 2024-09-21T21:42:09.663 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071539354999999999 2024-09-21T21:42:09.663 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.663 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.663 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:09.663 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897619+0000", 2024-09-21T21:42:09.663 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011665 2024-09-21T21:42:09.663 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.663 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.663 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:09.663 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897782+0000", 2024-09-21T21:42:09.663 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00016345500000000001 2024-09-21T21:42:09.664 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:09.664 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:09.664 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:09.664 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.664 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.664 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13575 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 380928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:09.664 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.825973+0000", 2024-09-21T21:42:09.664 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 18.786731202999999, 2024-09-21T21:42:09.664 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072012723000000001, 2024-09-21T21:42:09.664 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:09.664 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:09.664 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:09.664 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:09.664 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:09.664 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:09.665 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13575 2024-09-21T21:42:09.665 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.665 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:09.665 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.665 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:09.665 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825973+0000", 2024-09-21T21:42:09.665 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:09.665 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.665 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.665 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:09.665 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825973+0000", 2024-09-21T21:42:09.665 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:09.665 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.665 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.665 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:09.665 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826746+0000", 2024-09-21T21:42:09.666 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00077326800000000002 2024-09-21T21:42:09.666 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.666 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.666 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:09.666 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826766+0000", 2024-09-21T21:42:09.666 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9465000000000001e-05 2024-09-21T21:42:09.666 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.666 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.666 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:09.666 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826812+0000", 2024-09-21T21:42:09.666 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5927999999999998e-05 2024-09-21T21:42:09.666 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.666 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.666 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:09.667 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826833+0000", 2024-09-21T21:42:09.667 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1710999999999999e-05 2024-09-21T21:42:09.667 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.667 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.667 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:09.667 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897809+0000", 2024-09-21T21:42:09.667 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.070975229000000001 2024-09-21T21:42:09.667 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.667 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.667 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:09.667 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897864+0000", 2024-09-21T21:42:09.667 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5334999999999998e-05 2024-09-21T21:42:09.667 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.667 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.667 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:09.668 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897986+0000", 2024-09-21T21:42:09.668 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000121787 2024-09-21T21:42:09.668 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:09.668 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:09.668 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:09.668 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.668 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.668 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13577 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 1814528~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:09.668 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.826874+0000", 2024-09-21T21:42:09.668 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 18.785830169, 2024-09-21T21:42:09.668 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071639123999999998, 2024-09-21T21:42:09.668 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:09.668 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:09.668 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:09.669 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:09.669 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:09.669 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:09.669 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13577 2024-09-21T21:42:09.669 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.669 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:09.669 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.669 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:09.669 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826874+0000", 2024-09-21T21:42:09.669 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:09.669 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.669 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.669 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:09.669 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826874+0000", 2024-09-21T21:42:09.670 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:09.670 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.670 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.670 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:09.670 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826875+0000", 2024-09-21T21:42:09.670 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.82e-07 2024-09-21T21:42:09.670 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.670 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.670 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:09.670 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826877+0000", 2024-09-21T21:42:09.670 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.295e-06 2024-09-21T21:42:09.670 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.670 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.671 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:09.671 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826883+0000", 2024-09-21T21:42:09.671 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2609999999999999e-06 2024-09-21T21:42:09.671 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.671 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.671 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:09.671 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826889+0000", 2024-09-21T21:42:09.671 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5589999999999996e-06 2024-09-21T21:42:09.671 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.671 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.671 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:09.671 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898301+0000", 2024-09-21T21:42:09.671 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071412188000000001 2024-09-21T21:42:09.671 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.671 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.672 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:09.672 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898416+0000", 2024-09-21T21:42:09.672 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000115538 2024-09-21T21:42:09.672 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.672 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.672 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:09.672 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898513+0000", 2024-09-21T21:42:09.672 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6800999999999997e-05 2024-09-21T21:42:09.672 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:09.672 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:09.672 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:09.672 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.672 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.672 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13578 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 2502656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:09.673 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.826896+0000", 2024-09-21T21:42:09.673 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 18.785808713000002, 2024-09-21T21:42:09.673 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071789411999999997, 2024-09-21T21:42:09.673 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:09.673 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:09.673 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:09.673 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:09.673 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:09.673 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:09.673 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13578 2024-09-21T21:42:09.673 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.673 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:09.673 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.673 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:09.674 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826896+0000", 2024-09-21T21:42:09.674 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:09.674 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.674 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.674 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:09.674 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826896+0000", 2024-09-21T21:42:09.674 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:09.674 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.674 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.674 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:09.674 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827327+0000", 2024-09-21T21:42:09.674 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00043094500000000002 2024-09-21T21:42:09.674 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.674 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.674 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:09.675 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827331+0000", 2024-09-21T21:42:09.675 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.0969999999999999e-06 2024-09-21T21:42:09.675 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.675 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.675 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:09.675 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827341+0000", 2024-09-21T21:42:09.675 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0199e-05 2024-09-21T21:42:09.675 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.675 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.675 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:09.675 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827346+0000", 2024-09-21T21:42:09.675 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3210000000000004e-06 2024-09-21T21:42:09.675 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.675 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.675 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:09.675 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898540+0000", 2024-09-21T21:42:09.676 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071194246000000003 2024-09-21T21:42:09.676 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.676 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.676 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:09.676 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898598+0000", 2024-09-21T21:42:09.676 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7658999999999998e-05 2024-09-21T21:42:09.676 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.676 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.676 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:09.676 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898685+0000", 2024-09-21T21:42:09.676 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.6945000000000002e-05 2024-09-21T21:42:09.677 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:09.677 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:09.677 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:09.677 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.677 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.677 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13579 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 2531328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:09.677 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.827355+0000", 2024-09-21T21:42:09.677 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 18.785349182000001, 2024-09-21T21:42:09.677 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071650933, 2024-09-21T21:42:09.677 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:09.677 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:09.677 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:09.677 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:09.677 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:09.678 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:09.678 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13579 2024-09-21T21:42:09.678 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.678 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:09.678 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.678 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:09.678 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827355+0000", 2024-09-21T21:42:09.678 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:09.678 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.678 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.678 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:09.678 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827355+0000", 2024-09-21T21:42:09.678 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:09.678 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.679 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.679 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:09.679 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827486+0000", 2024-09-21T21:42:09.679 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00013066799999999999 2024-09-21T21:42:09.679 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.679 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.679 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:09.679 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827490+0000", 2024-09-21T21:42:09.679 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.9009999999999996e-06 2024-09-21T21:42:09.679 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.679 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.679 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:09.679 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827499+0000", 2024-09-21T21:42:09.679 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6209999999999999e-06 2024-09-21T21:42:09.679 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.679 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.680 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:09.680 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827506+0000", 2024-09-21T21:42:09.680 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6490000000000002e-06 2024-09-21T21:42:09.680 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.680 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.680 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:09.680 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898713+0000", 2024-09-21T21:42:09.680 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071206937999999997 2024-09-21T21:42:09.680 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.680 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.680 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:09.680 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898811+0000", 2024-09-21T21:42:09.680 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.8159999999999995e-05 2024-09-21T21:42:09.680 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.680 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.680 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:09.681 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899006+0000", 2024-09-21T21:42:09.681 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00019499599999999999 2024-09-21T21:42:09.681 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:09.681 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:09.681 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:09.681 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.681 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.681 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13580 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3219456~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:09.681 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.827522+0000", 2024-09-21T21:42:09.681 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 18.785182751000001, 2024-09-21T21:42:09.681 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071813298999999997, 2024-09-21T21:42:09.681 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:09.681 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:09.681 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:09.681 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:09.681 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:09.682 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:09.682 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13580 2024-09-21T21:42:09.682 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.682 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:09.682 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.682 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:09.682 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827522+0000", 2024-09-21T21:42:09.682 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:09.682 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.682 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.682 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:09.682 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827522+0000", 2024-09-21T21:42:09.682 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:09.682 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.682 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.682 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:09.683 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827898+0000", 2024-09-21T21:42:09.683 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000376876 2024-09-21T21:42:09.683 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.683 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.683 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:09.683 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827903+0000", 2024-09-21T21:42:09.683 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4129999999999999e-06 2024-09-21T21:42:09.683 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.683 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.683 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:09.683 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827911+0000", 2024-09-21T21:42:09.683 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.4540000000000007e-06 2024-09-21T21:42:09.683 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.683 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.683 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:09.683 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827917+0000", 2024-09-21T21:42:09.684 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2110000000000003e-06 2024-09-21T21:42:09.684 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.684 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.684 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:09.684 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899041+0000", 2024-09-21T21:42:09.684 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071123965999999997 2024-09-21T21:42:09.684 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.684 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.684 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:09.684 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899157+0000", 2024-09-21T21:42:09.684 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011573700000000001 2024-09-21T21:42:09.684 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.684 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.684 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:09.684 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899335+0000", 2024-09-21T21:42:09.684 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00017764199999999999 2024-09-21T21:42:09.685 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:09.685 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:09.685 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:09.685 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.685 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.685 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13581 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3248128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:09.685 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.827929+0000", 2024-09-21T21:42:09.685 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 18.784775352, 2024-09-21T21:42:09.685 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071730151000000006, 2024-09-21T21:42:09.685 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:09.685 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:09.685 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:09.685 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:09.685 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:09.685 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:09.685 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13581 2024-09-21T21:42:09.686 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.686 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:09.686 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.686 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:09.686 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827929+0000", 2024-09-21T21:42:09.686 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:09.686 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.686 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.686 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:09.686 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827929+0000", 2024-09-21T21:42:09.686 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:09.686 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.686 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.686 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:09.686 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.829943+0000", 2024-09-21T21:42:09.686 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0020143750000000001 2024-09-21T21:42:09.686 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.687 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.687 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:09.687 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.829948+0000", 2024-09-21T21:42:09.687 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9400000000000001e-06 2024-09-21T21:42:09.687 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.687 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.687 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:09.687 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.829960+0000", 2024-09-21T21:42:09.687 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.2123999999999999e-05 2024-09-21T21:42:09.687 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.687 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.687 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:09.687 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.829968+0000", 2024-09-21T21:42:09.687 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7000000000000008e-06 2024-09-21T21:42:09.687 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.688 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.688 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:09.688 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899370+0000", 2024-09-21T21:42:09.688 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.069401672999999997 2024-09-21T21:42:09.688 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.688 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.688 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:09.688 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899481+0000", 2024-09-21T21:42:09.688 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011145300000000001 2024-09-21T21:42:09.688 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.688 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.688 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:09.688 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899659+0000", 2024-09-21T21:42:09.688 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00017788600000000001 2024-09-21T21:42:09.688 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:09.689 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:09.689 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:09.689 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.689 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.689 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13595 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 1871872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:09.689 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.830180+0000", 2024-09-21T21:42:09.689 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 18.782524238000001, 2024-09-21T21:42:09.689 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071809244999999994, 2024-09-21T21:42:09.689 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:09.689 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:09.689 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:09.689 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:09.689 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:09.689 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:09.690 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13595 2024-09-21T21:42:09.690 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.690 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:09.690 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.690 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:09.690 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.830180+0000", 2024-09-21T21:42:09.690 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:09.690 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.690 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.690 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:09.690 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.830180+0000", 2024-09-21T21:42:09.690 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:09.690 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.690 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.690 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:09.690 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.831967+0000", 2024-09-21T21:42:09.691 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0017868350000000001 2024-09-21T21:42:09.691 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.691 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.691 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:09.691 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.831971+0000", 2024-09-21T21:42:09.691 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3950000000000003e-06 2024-09-21T21:42:09.691 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.691 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.691 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:09.691 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.831983+0000", 2024-09-21T21:42:09.691 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1815e-05 2024-09-21T21:42:09.691 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.691 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.691 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:09.691 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.831993+0000", 2024-09-21T21:42:09.691 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6339999999999996e-06 2024-09-21T21:42:09.691 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.692 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.692 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:09.692 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.901693+0000", 2024-09-21T21:42:09.692 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.069700082999999996 2024-09-21T21:42:09.692 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.692 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.692 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:09.692 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.901813+0000", 2024-09-21T21:42:09.692 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011971 2024-09-21T21:42:09.692 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.692 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.692 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:09.692 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.901989+0000", 2024-09-21T21:42:09.692 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000176773 2024-09-21T21:42:09.692 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:09.692 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:09.692 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:09.693 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.693 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.693 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13600 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 2617344~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:09.693 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.832109+0000", 2024-09-21T21:42:09.693 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 18.780595484999999, 2024-09-21T21:42:09.693 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071551828999999997, 2024-09-21T21:42:09.693 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:09.693 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:09.693 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:09.693 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:09.693 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:09.693 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:09.693 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13600 2024-09-21T21:42:09.693 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.693 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:09.693 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.693 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:09.694 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.832109+0000", 2024-09-21T21:42:09.694 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:09.694 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.694 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.694 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:09.694 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.832109+0000", 2024-09-21T21:42:09.694 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:09.694 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.694 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.694 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:09.694 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.832109+0000", 2024-09-21T21:42:09.694 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4999999999999998e-07 2024-09-21T21:42:09.694 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.694 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.694 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:09.694 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.832112+0000", 2024-09-21T21:42:09.695 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.5019999999999999e-06 2024-09-21T21:42:09.695 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.695 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.695 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:09.695 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.832118+0000", 2024-09-21T21:42:09.695 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6270000000000002e-06 2024-09-21T21:42:09.695 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.695 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.695 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:09.695 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.832124+0000", 2024-09-21T21:42:09.695 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1850000000000003e-06 2024-09-21T21:42:09.695 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.695 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.695 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:09.695 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.903366+0000", 2024-09-21T21:42:09.695 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071241961000000006 2024-09-21T21:42:09.696 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.696 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.696 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:09.696 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.903485+0000", 2024-09-21T21:42:09.696 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000119202 2024-09-21T21:42:09.696 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.696 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.696 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:09.696 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.903661+0000", 2024-09-21T21:42:09.696 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000175902 2024-09-21T21:42:09.696 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:09.696 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:09.696 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:09.696 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.696 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.696 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13601 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3334144~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:09.697 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.832130+0000", 2024-09-21T21:42:09.697 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 18.780574366, 2024-09-21T21:42:09.697 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071861727, 2024-09-21T21:42:09.697 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:09.697 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:09.697 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:09.697 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:09.697 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:09.697 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:09.697 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13601 2024-09-21T21:42:09.697 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.697 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:09.697 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.697 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:09.697 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.832130+0000", 2024-09-21T21:42:09.697 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:09.698 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.698 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.698 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:09.698 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.832130+0000", 2024-09-21T21:42:09.698 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:09.698 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.698 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.698 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:09.698 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.833939+0000", 2024-09-21T21:42:09.698 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0018094400000000001 2024-09-21T21:42:09.698 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.698 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.698 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:09.698 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.833944+0000", 2024-09-21T21:42:09.698 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5979999999999999e-06 2024-09-21T21:42:09.698 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.699 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.699 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:09.699 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.833953+0000", 2024-09-21T21:42:09.699 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.1460000000000002e-06 2024-09-21T21:42:09.699 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.699 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.699 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:09.699 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.833960+0000", 2024-09-21T21:42:09.699 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.7989999999999996e-06 2024-09-21T21:42:09.699 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.699 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.699 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:09.699 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.903693+0000", 2024-09-21T21:42:09.699 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.069732901 2024-09-21T21:42:09.699 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.699 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.700 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:09.700 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.903814+0000", 2024-09-21T21:42:09.700 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000121585 2024-09-21T21:42:09.700 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:09.700 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:09.700 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:09.700 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.903992+0000", 2024-09-21T21:42:09.700 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00017725799999999999 2024-09-21T21:42:09.700 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:09.700 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:09.700 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:09.700 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:09.700 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:09.701 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:09.702 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T21:42:09.703 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:09.700+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:09.703 INFO:teuthology.orchestra.run.smithi164.stdout:4589 write 0xa7f4c00 thru 0xa803258 (0xe659 bytes) 2024-09-21T21:42:09.703 INFO:teuthology.orchestra.run.smithi164.stdout:4592 trunc from 0xa803400 to 0xc136e00 2024-09-21T21:42:09.703 INFO:teuthology.orchestra.run.smithi164.stdout:4594 punch from 0x6860c00 to 0x6865400, (0x4800 bytes) 2024-09-21T21:42:09.703 INFO:teuthology.orchestra.run.smithi164.stdout:4596 trunc from 0xc136e00 to 0x869f200 2024-09-21T21:42:09.704 INFO:teuthology.orchestra.run.smithi164.stdout:4597 trunc from 0x869f200 to 0x98ee800 2024-09-21T21:42:09.704 INFO:teuthology.orchestra.run.smithi164.stdout:4599 write 0xd014a00 thru 0xd019cd9 (0x52da bytes) 2024-09-21T21:42:09.704 INFO:teuthology.orchestra.run.smithi164.stdout:4603 write 0x24a7000 thru 0x24a7699 (0x69a bytes) 2024-09-21T21:42:09.704 INFO:teuthology.orchestra.run.smithi164.stdout:truncating to largest ever: 0xed94800 2024-09-21T21:42:09.704 INFO:teuthology.orchestra.run.smithi164.stdout:4607 trunc from 0xd019e00 to 0xed94800 2024-09-21T21:42:09.724 INFO:teuthology.orchestra.run.smithi164.stdout:4609 write 0x4e0a200 thru 0x4e0a777 (0x578 bytes) 2024-09-21T21:42:09.724 INFO:teuthology.orchestra.run.smithi164.stdout:4611 punch from 0x9b5f000 to 0x9b6be00, (0xce00 bytes) 2024-09-21T21:42:09.724 INFO:teuthology.orchestra.run.smithi164.stdout:4613 read 0x7cda800 thru 0x7ce3511 (0x8d12 bytes) 2024-09-21T21:42:09.724 INFO:teuthology.orchestra.run.smithi164.stdout:4614 write 0x59cae00 thru 0x59ceda4 (0x3fa5 bytes) 2024-09-21T21:42:09.724 INFO:teuthology.orchestra.run.smithi164.stdout:4617 read 0x5720000 thru 0x572f1d8 (0xf1d9 bytes) 2024-09-21T21:42:09.724 INFO:teuthology.orchestra.run.smithi164.stdout:4619 write 0x25bce00 thru 0x25c36ad (0x68ae bytes) 2024-09-21T21:42:09.724 INFO:teuthology.orchestra.run.smithi164.stdout:4620 read 0x590fc00 thru 0x5911bc3 (0x1fc4 bytes) 2024-09-21T21:42:09.724 INFO:teuthology.orchestra.run.smithi164.stdout:4621 punch from 0x2ea8a00 to 0x2eb4400, (0xba00 bytes) 2024-09-21T21:42:09.725 INFO:teuthology.orchestra.run.smithi164.stdout:4622 read 0x479a400 thru 0x47a6f3a (0xcb3b bytes) 2024-09-21T21:42:09.725 INFO:teuthology.orchestra.run.smithi164.stdout:4624 punch from 0x784ae00 to 0x7856800, (0xba00 bytes) 2024-09-21T21:42:09.725 INFO:teuthology.orchestra.run.smithi164.stdout:4625 punch from 0x7d15e00 to 0x7d1dc00, (0x7e00 bytes) 2024-09-21T21:42:09.725 INFO:teuthology.orchestra.run.smithi164.stdout:4628 read 0x8480400 thru 0x8484f89 (0x4b8a bytes) 2024-09-21T21:42:09.725 INFO:teuthology.orchestra.run.smithi164.stdout:4630 read 0x1752800 thru 0x1758045 (0x5846 bytes) 2024-09-21T21:42:09.725 INFO:teuthology.orchestra.run.smithi164.stdout:4631 write 0xc66a600 thru 0xc677d4a (0xd74b bytes) 2024-09-21T21:42:09.726 INFO:teuthology.orchestra.run.smithi164.stdout:4633 read 0x4673e00 thru 0x46817ae (0xd9af bytes) 2024-09-21T21:42:09.726 INFO:teuthology.orchestra.run.smithi164.stdout:4635 read 0x9c7a400 thru 0x9c7eeaf (0x4ab0 bytes) 2024-09-21T21:42:09.726 INFO:teuthology.orchestra.run.smithi164.stdout:4637 write 0x96c1200 thru 0x96d0018 (0xee19 bytes) 2024-09-21T21:42:09.726 INFO:teuthology.orchestra.run.smithi164.stdout:4638 read 0x4d81a00 thru 0x4d90258 (0xe859 bytes) 2024-09-21T21:42:09.726 INFO:teuthology.orchestra.run.smithi164.stdout:4642 read 0x79e4800 thru 0x79f0bb7 (0xc3b8 bytes) 2024-09-21T21:42:09.726 INFO:teuthology.orchestra.run.smithi164.stdout:4647 write 0xb37d400 thru 0xb38d3aa (0xffab bytes) 2024-09-21T21:42:09.727 INFO:teuthology.orchestra.run.smithi164.stdout:4648 write 0xe0d6a00 thru 0xe0e2dae (0xc3af bytes) 2024-09-21T21:42:09.732 INFO:teuthology.orchestra.run.smithi164.stdout:4651 trunc from 0xed94800 to 0x98d4e00 2024-09-21T21:42:09.765 INFO:teuthology.orchestra.run.smithi164.stdout:4653 read 0x873d400 thru 0x87455a6 (0x81a7 bytes) 2024-09-21T21:42:09.767 INFO:teuthology.orchestra.run.smithi164.stdout:4654 read 0x6b9cc00 thru 0x6b9e36f (0x1770 bytes) 2024-09-21T21:42:09.767 INFO:teuthology.orchestra.run.smithi164.stdout:4655 read 0x32de00 thru 0x336a2b (0x8c2c bytes) 2024-09-21T21:42:09.767 INFO:teuthology.orchestra.run.smithi164.stdout:4656 read 0x76a8000 thru 0x76b77c3 (0xf7c4 bytes) 2024-09-21T21:42:09.767 INFO:teuthology.orchestra.run.smithi164.stdout:4657 read 0x7456c00 thru 0x746335a (0xc75b bytes) 2024-09-21T21:42:09.768 INFO:teuthology.orchestra.run.smithi164.stdout:4658 punch from 0x32b7e00 to 0x32bda00, (0x5c00 bytes) 2024-09-21T21:42:09.768 INFO:teuthology.orchestra.run.smithi164.stdout:4659 read 0x25a0400 thru 0x25a36c4 (0x32c5 bytes) 2024-09-21T21:42:09.771 INFO:teuthology.orchestra.run.smithi164.stdout:4663 trunc from 0x98d4e00 to 0x78d0000 2024-09-21T21:42:09.790 INFO:teuthology.orchestra.run.smithi164.stdout:4664 read 0x230a200 thru 0x2312876 (0x8677 bytes) 2024-09-21T21:42:09.792 INFO:teuthology.orchestra.run.smithi164.stdout:4665 read 0x73c4200 thru 0x73d252d (0xe32e bytes) 2024-09-21T21:42:09.792 INFO:teuthology.orchestra.run.smithi164.stdout:4666 read 0x4657e00 thru 0x4660c73 (0x8e74 bytes) 2024-09-21T21:42:09.792 INFO:teuthology.orchestra.run.smithi164.stdout:4667 read 0x23ace00 thru 0x23ae158 (0x1359 bytes) 2024-09-21T21:42:09.802 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:09.800+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:09.811 INFO:teuthology.orchestra.run.smithi164.stdout:4668 write 0xdcb5600 thru 0xdcbfc47 (0xa648 bytes) 2024-09-21T21:42:09.811 INFO:teuthology.orchestra.run.smithi164.stdout:4669 punch from 0xb26c600 to 0xb276000, (0x9a00 bytes) 2024-09-21T21:42:09.813 INFO:teuthology.orchestra.run.smithi164.stdout:4671 write 0x7ccea00 thru 0x7cddff9 (0xf5fa bytes) 2024-09-21T21:42:09.823 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:09.823 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:42:09.823 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-21T21:42:09.823 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:09.828 INFO:teuthology.orchestra.run.smithi164.stdout:4673 trunc from 0xdcbfe00 to 0xbe7600 2024-09-21T21:42:09.832 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T21:42:09.850 INFO:teuthology.orchestra.run.smithi164.stdout:4674 read 0x74c00 thru 0x8195d (0xcd5e bytes) 2024-09-21T21:42:09.853 INFO:teuthology.orchestra.run.smithi164.stdout:4675 read 0x1c7400 thru 0x1d4d68 (0xd969 bytes) 2024-09-21T21:42:09.854 INFO:teuthology.orchestra.run.smithi164.stdout:4677 read 0xba3400 thru 0xba61a6 (0x2da7 bytes) 2024-09-21T21:42:09.869 INFO:teuthology.orchestra.run.smithi164.stdout:4682 write 0x72bea00 thru 0x72bffdc (0x15dd bytes) 2024-09-21T21:42:09.885 INFO:teuthology.orchestra.run.smithi164.stdout:4684 write 0x89ef000 thru 0x89f05e1 (0x15e2 bytes) 2024-09-21T21:42:09.887 INFO:teuthology.orchestra.run.smithi164.stdout:4686 read 0x6aabc00 thru 0x6ab2c29 (0x702a bytes) 2024-09-21T21:42:09.894 INFO:teuthology.orchestra.run.smithi164.stdout:4687 trunc from 0x89f0600 to 0x206b200 2024-09-21T21:42:09.916 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:09.914+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:09.934 INFO:teuthology.orchestra.run.smithi164.stdout:4688 write 0xba40000 thru 0xba49b7f (0x9b80 bytes) 2024-09-21T21:42:09.934 INFO:teuthology.orchestra.run.smithi164.stdout:4692 read 0x4efde00 thru 0x4f004ef (0x26f0 bytes) 2024-09-21T21:42:09.934 INFO:teuthology.orchestra.run.smithi164.stdout:4693 punch from 0x4808a00 to 0x480f600, (0x6c00 bytes) 2024-09-21T21:42:09.936 INFO:teuthology.orchestra.run.smithi164.stdout:4695 read 0x6e04600 thru 0x6e0baff (0x7500 bytes) 2024-09-21T21:42:09.936 INFO:teuthology.orchestra.run.smithi164.stdout:4698 punch from 0xf6cc00 to 0xf75200, (0x8600 bytes) 2024-09-21T21:42:09.936 INFO:teuthology.orchestra.run.smithi164.stdout:4699 read 0x5d75000 thru 0x5d7f224 (0xa225 bytes) 2024-09-21T21:42:09.936 INFO:teuthology.orchestra.run.smithi164.stdout:4700 write 0xb860800 thru 0xb86fd69 (0xf56a bytes) 2024-09-21T21:42:09.936 INFO:teuthology.orchestra.run.smithi164.stdout:4701 write 0xa718a00 thru 0xa72543b (0xca3c bytes) 2024-09-21T21:42:09.936 INFO:teuthology.orchestra.run.smithi164.stdout:4704 read 0x69e1c00 thru 0x69ea5f7 (0x89f8 bytes) 2024-09-21T21:42:09.936 INFO:teuthology.orchestra.run.smithi164.stdout:4705 read 0x87a7400 thru 0x87b29df (0xb5e0 bytes) 2024-09-21T21:42:09.936 INFO:teuthology.orchestra.run.smithi164.stdout:4706 read 0x6404000 thru 0x6411bf1 (0xdbf2 bytes) 2024-09-21T21:42:09.936 INFO:teuthology.orchestra.run.smithi164.stdout:4707 read 0x1b77a00 thru 0x1b83ba9 (0xc1aa bytes) 2024-09-21T21:42:09.937 INFO:teuthology.orchestra.run.smithi164.stdout:4712 read 0x314ac00 thru 0x314fea1 (0x52a2 bytes) 2024-09-21T21:42:09.957 INFO:teuthology.orchestra.run.smithi164.stdout:4715 write 0xec5ea00 thru 0xec62c5b (0x425c bytes) 2024-09-21T21:42:09.958 INFO:teuthology.orchestra.run.smithi164.stdout:4717 write 0x695e400 thru 0x696795b (0x955c bytes) 2024-09-21T21:42:09.958 INFO:teuthology.orchestra.run.smithi164.stdout:4719 write 0x50a9000 thru 0x50b5f95 (0xcf96 bytes) 2024-09-21T21:42:09.967 INFO:teuthology.orchestra.run.smithi164.stdout:4720 trunc from 0xec62e00 to 0x71fbe00 2024-09-21T21:42:09.987 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:09.987 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:42:09.988 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:42:09.988 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:42:09.988 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:09.997 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T21:42:10.017 INFO:teuthology.orchestra.run.smithi164.stdout:4723 write 0xbaf4600 thru 0xbb041bd (0xfbbe bytes) 2024-09-21T21:42:10.017 INFO:teuthology.orchestra.run.smithi164.stdout:4724 punch from 0x99f8000 to 0x99fc000, (0x4000 bytes) 2024-09-21T21:42:10.017 INFO:teuthology.orchestra.run.smithi164.stdout:4725 write 0xa0d4e00 thru 0xa0d8d22 (0x3f23 bytes) 2024-09-21T21:42:10.017 INFO:teuthology.orchestra.run.smithi164.stdout:4726 read 0x5e03000 thru 0x5e08ec1 (0x5ec2 bytes) 2024-09-21T21:42:10.017 INFO:teuthology.orchestra.run.smithi164.stdout:4727 write 0x8225800 thru 0x8234cf6 (0xf4f7 bytes) 2024-09-21T21:42:10.038 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:10.036+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:10.045 INFO:teuthology.orchestra.run.smithi164.stdout:4728 write 0xcff1400 thru 0xcff4dc4 (0x39c5 bytes) 2024-09-21T21:42:10.045 INFO:teuthology.orchestra.run.smithi164.stdout:4730 read 0xcd32200 thru 0xcd3c205 (0xa006 bytes) 2024-09-21T21:42:10.055 INFO:teuthology.orchestra.run.smithi164.stdout:4731 trunc from 0xcff4e00 to 0x49e0200 2024-09-21T21:42:10.084 INFO:teuthology.orchestra.run.smithi164.stdout:4732 punch from 0x1366e00 to 0x1372800, (0xba00 bytes) 2024-09-21T21:42:10.102 INFO:teuthology.orchestra.run.smithi164.stdout:4733 write 0x982ce00 thru 0x983055f (0x3760 bytes) 2024-09-21T21:42:10.103 INFO:teuthology.orchestra.run.smithi164.stdout:4736 punch from 0x22a200 to 0x22d200, (0x3000 bytes) 2024-09-21T21:42:10.105 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:10.104+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:10.109 INFO:teuthology.orchestra.run.smithi164.stdout:4737 trunc from 0x9830600 to 0xd224e00 2024-09-21T21:42:10.119 INFO:teuthology.orchestra.run.smithi164.stdout:4738 read 0x4b29200 thru 0x4b2ac7b (0x1a7c bytes) 2024-09-21T21:42:10.130 INFO:teuthology.orchestra.run.smithi164.stdout:4739 trunc from 0xd224e00 to 0x555c800 2024-09-21T21:42:10.146 INFO:teuthology.orchestra.run.smithi164.stdout:4740 read 0x479b800 thru 0x479e7e0 (0x2fe1 bytes) 2024-09-21T21:42:10.147 INFO:teuthology.orchestra.run.smithi164.stdout:4742 read 0x19b1e00 thru 0x19b4e71 (0x3072 bytes) 2024-09-21T21:42:10.148 INFO:teuthology.orchestra.run.smithi164.stdout:4746 read 0x406e00 thru 0x40e2cc (0x74cd bytes) 2024-09-21T21:42:10.148 INFO:teuthology.orchestra.run.smithi164.stdout:4748 punch from 0x235e800 to 0x235fa00, (0x1200 bytes) 2024-09-21T21:42:10.149 INFO:teuthology.orchestra.run.smithi164.stdout:4750 write 0x14f8200 thru 0x150336a (0xb16b bytes) 2024-09-21T21:42:10.149 INFO:teuthology.orchestra.run.smithi164.stdout:4752 clone 37 order 20 su 4096 sc 10 2024-09-21T21:42:10.154 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:10.154 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-21T21:42:10.154 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-21T21:42:10.154 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:42:10.154 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.154 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19744 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6471680~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:10.154 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.272996+0000", 2024-09-21T21:42:10.154 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 18.877787960999999, 2024-09-21T21:42:10.154 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97298795400000004, 2024-09-21T21:42:10.154 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:10.154 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:10.154 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:10.154 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:10.155 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:10.155 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:10.155 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19744 2024-09-21T21:42:10.155 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.155 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:10.155 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.155 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:10.155 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.272996+0000", 2024-09-21T21:42:10.155 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:10.155 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.155 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.155 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:10.155 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.272996+0000", 2024-09-21T21:42:10.155 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:10.156 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.156 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.156 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:10.156 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.272996+0000", 2024-09-21T21:42:10.156 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.53e-07 2024-09-21T21:42:10.156 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.156 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.156 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:10.156 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.272998+0000", 2024-09-21T21:42:10.156 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.888e-06 2024-09-21T21:42:10.156 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.156 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.156 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:10.156 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273005+0000", 2024-09-21T21:42:10.156 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4350000000000002e-06 2024-09-21T21:42:10.157 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.157 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.157 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:10.157 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273009+0000", 2024-09-21T21:42:10.157 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3239999999999996e-06 2024-09-21T21:42:10.157 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.157 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.157 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:10.157 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277855+0000", 2024-09-21T21:42:10.157 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048463849999999999 2024-09-21T21:42:10.157 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.157 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.157 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:10.158 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277858+0000", 2024-09-21T21:42:10.158 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9490000000000001e-06 2024-09-21T21:42:10.158 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.158 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.158 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:10.158 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.245879+0000", 2024-09-21T21:42:10.158 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96802068399999996 2024-09-21T21:42:10.158 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.158 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.158 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:10.158 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.245943+0000", 2024-09-21T21:42:10.158 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3690999999999997e-05 2024-09-21T21:42:10.158 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.158 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.159 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:10.159 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.245984+0000", 2024-09-21T21:42:10.159 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1245000000000002e-05 2024-09-21T21:42:10.159 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:10.159 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:10.159 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:10.159 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.159 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.159 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19745 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 7876608~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:10.159 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273015+0000", 2024-09-21T21:42:10.159 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 18.877769312000002, 2024-09-21T21:42:10.159 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97312649200000001, 2024-09-21T21:42:10.159 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:10.159 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:10.159 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:10.160 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:10.160 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:10.160 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:10.160 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19745 2024-09-21T21:42:10.160 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.160 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:10.160 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.160 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:10.160 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273015+0000", 2024-09-21T21:42:10.160 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:10.160 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.160 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.160 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:10.160 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273015+0000", 2024-09-21T21:42:10.160 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:10.161 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.161 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.161 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:10.161 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273015+0000", 2024-09-21T21:42:10.161 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2300000000000002e-07 2024-09-21T21:42:10.161 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.161 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.161 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:10.161 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273017+0000", 2024-09-21T21:42:10.161 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0480000000000001e-06 2024-09-21T21:42:10.161 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.161 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.161 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:10.161 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273023+0000", 2024-09-21T21:42:10.161 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.699e-06 2024-09-21T21:42:10.161 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.162 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.162 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:10.162 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273027+0000", 2024-09-21T21:42:10.162 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6689999999999997e-06 2024-09-21T21:42:10.162 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.162 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.162 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:10.162 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277895+0000", 2024-09-21T21:42:10.162 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048675039999999999 2024-09-21T21:42:10.162 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.162 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.162 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:10.162 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277899+0000", 2024-09-21T21:42:10.163 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5519999999999998e-06 2024-09-21T21:42:10.163 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.163 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.163 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:10.163 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246018+0000", 2024-09-21T21:42:10.163 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96811853599999997 2024-09-21T21:42:10.163 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.163 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.163 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:10.163 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246090+0000", 2024-09-21T21:42:10.163 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2502999999999993e-05 2024-09-21T21:42:10.163 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.163 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.164 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:10.164 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246141+0000", 2024-09-21T21:42:10.164 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0658000000000002e-05 2024-09-21T21:42:10.164 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:10.164 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:10.164 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:10.164 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.164 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.164 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19746 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 7188480~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:10.164 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273033+0000", 2024-09-21T21:42:10.164 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 18.877750728999999, 2024-09-21T21:42:10.164 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97319625399999998, 2024-09-21T21:42:10.164 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:10.164 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:10.165 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:10.165 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:10.165 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:10.165 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:10.165 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19746 2024-09-21T21:42:10.165 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.165 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:10.165 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.165 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:10.165 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273033+0000", 2024-09-21T21:42:10.165 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:10.165 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.165 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.165 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:10.165 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273033+0000", 2024-09-21T21:42:10.166 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:10.166 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.166 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.166 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:10.166 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273034+0000", 2024-09-21T21:42:10.166 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2100000000000002e-07 2024-09-21T21:42:10.166 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.166 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.166 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:10.166 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273036+0000", 2024-09-21T21:42:10.166 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1919999999999999e-06 2024-09-21T21:42:10.166 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.166 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.166 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:10.167 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273044+0000", 2024-09-21T21:42:10.167 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2369999999999992e-06 2024-09-21T21:42:10.167 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.167 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.167 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:10.167 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273048+0000", 2024-09-21T21:42:10.167 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5090000000000004e-06 2024-09-21T21:42:10.167 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.167 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.167 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:10.167 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277921+0000", 2024-09-21T21:42:10.167 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048723710000000003 2024-09-21T21:42:10.167 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.167 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.168 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:10.168 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277924+0000", 2024-09-21T21:42:10.168 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6509999999999999e-06 2024-09-21T21:42:10.168 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.168 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.168 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:10.168 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246153+0000", 2024-09-21T21:42:10.168 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96822869899999997 2024-09-21T21:42:10.168 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.168 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.168 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:10.168 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246192+0000", 2024-09-21T21:42:10.168 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8747e-05 2024-09-21T21:42:10.169 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.169 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.169 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:10.169 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246229+0000", 2024-09-21T21:42:10.169 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7426999999999999e-05 2024-09-21T21:42:10.169 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:10.169 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:10.169 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:10.169 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.169 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.169 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19747 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 7905280~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:10.169 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273058+0000", 2024-09-21T21:42:10.169 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 18.877726007, 2024-09-21T21:42:10.170 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97338591399999996, 2024-09-21T21:42:10.170 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:10.170 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:10.170 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:10.170 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:10.170 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:10.170 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:10.170 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19747 2024-09-21T21:42:10.170 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.170 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:10.170 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.170 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:10.170 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273058+0000", 2024-09-21T21:42:10.170 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:10.170 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.171 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.171 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:10.171 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273058+0000", 2024-09-21T21:42:10.171 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:10.171 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.171 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.171 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:10.171 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273058+0000", 2024-09-21T21:42:10.171 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5900000000000003e-07 2024-09-21T21:42:10.171 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.171 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.171 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:10.171 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273061+0000", 2024-09-21T21:42:10.171 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3479999999999998e-06 2024-09-21T21:42:10.172 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.172 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.172 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:10.172 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273067+0000", 2024-09-21T21:42:10.172 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1469999999999998e-06 2024-09-21T21:42:10.172 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.172 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.172 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:10.172 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273071+0000", 2024-09-21T21:42:10.172 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7339999999999999e-06 2024-09-21T21:42:10.172 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.172 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.172 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:10.172 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277945+0000", 2024-09-21T21:42:10.173 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.004873474 2024-09-21T21:42:10.173 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.173 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.173 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:10.173 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277948+0000", 2024-09-21T21:42:10.173 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0299999999999998e-06 2024-09-21T21:42:10.173 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.173 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.173 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:10.173 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246269+0000", 2024-09-21T21:42:10.173 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96832060200000003 2024-09-21T21:42:10.173 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.173 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.174 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:10.174 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246362+0000", 2024-09-21T21:42:10.174 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3867000000000001e-05 2024-09-21T21:42:10.174 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.174 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.174 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:10.174 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246444+0000", 2024-09-21T21:42:10.174 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1353000000000005e-05 2024-09-21T21:42:10.174 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:10.174 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:10.174 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:10.174 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.174 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.175 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19755 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 5783552~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:10.175 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273077+0000", 2024-09-21T21:42:10.175 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 18.877706391, 2024-09-21T21:42:10.175 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97350203400000002, 2024-09-21T21:42:10.175 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:10.175 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:10.175 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:10.175 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:10.175 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:10.175 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:10.175 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19755 2024-09-21T21:42:10.175 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.175 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:10.175 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.176 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:10.176 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273077+0000", 2024-09-21T21:42:10.176 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:10.176 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.176 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.176 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:10.176 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273077+0000", 2024-09-21T21:42:10.176 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:10.176 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.176 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.176 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:10.176 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273078+0000", 2024-09-21T21:42:10.176 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5699999999999998e-07 2024-09-21T21:42:10.176 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.177 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.177 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:10.177 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273083+0000", 2024-09-21T21:42:10.177 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8670000000000002e-06 2024-09-21T21:42:10.177 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.177 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.177 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:10.177 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273088+0000", 2024-09-21T21:42:10.177 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8370000000000002e-06 2024-09-21T21:42:10.177 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.177 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.177 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:10.177 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273093+0000", 2024-09-21T21:42:10.177 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5709999999999999e-06 2024-09-21T21:42:10.178 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.178 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.178 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:10.178 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277968+0000", 2024-09-21T21:42:10.178 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048753909999999998 2024-09-21T21:42:10.178 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.178 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.178 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:10.178 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277971+0000", 2024-09-21T21:42:10.178 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0189999999999998e-06 2024-09-21T21:42:10.178 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.178 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.178 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:10.178 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246485+0000", 2024-09-21T21:42:10.179 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96851399299999996 2024-09-21T21:42:10.179 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.179 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.179 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:10.179 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246536+0000", 2024-09-21T21:42:10.179 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0430000000000003e-05 2024-09-21T21:42:10.179 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.179 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.179 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:10.179 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246579+0000", 2024-09-21T21:42:10.179 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3569000000000002e-05 2024-09-21T21:42:10.179 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:10.179 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:10.180 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:10.180 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.180 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.180 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19756 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6500352~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:10.180 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273099+0000", 2024-09-21T21:42:10.180 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 18.877685018000001, 2024-09-21T21:42:10.180 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97359272799999996, 2024-09-21T21:42:10.180 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:10.180 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:10.180 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:10.180 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:10.180 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:10.180 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:10.180 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19756 2024-09-21T21:42:10.181 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.181 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:10.181 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.181 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:10.181 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273099+0000", 2024-09-21T21:42:10.181 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:10.181 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.181 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.181 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:10.181 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273099+0000", 2024-09-21T21:42:10.181 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:10.181 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.181 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.181 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:10.181 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273099+0000", 2024-09-21T21:42:10.182 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0900000000000002e-07 2024-09-21T21:42:10.182 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.182 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.182 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:10.182 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273101+0000", 2024-09-21T21:42:10.182 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0320000000000002e-06 2024-09-21T21:42:10.182 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.182 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.182 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:10.182 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273107+0000", 2024-09-21T21:42:10.182 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2830000000000001e-06 2024-09-21T21:42:10.182 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.182 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.182 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:10.183 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273111+0000", 2024-09-21T21:42:10.183 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2230000000000001e-06 2024-09-21T21:42:10.183 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.183 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.183 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:10.183 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277994+0000", 2024-09-21T21:42:10.183 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048837070000000002 2024-09-21T21:42:10.183 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.183 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.183 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:10.183 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278000+0000", 2024-09-21T21:42:10.183 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3149999999999999e-06 2024-09-21T21:42:10.183 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.183 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.184 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:10.184 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246607+0000", 2024-09-21T21:42:10.184 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.968606839 2024-09-21T21:42:10.184 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.184 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.184 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:10.184 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246651+0000", 2024-09-21T21:42:10.184 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4734999999999998e-05 2024-09-21T21:42:10.184 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.184 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.184 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:10.184 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246692+0000", 2024-09-21T21:42:10.184 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0185000000000002e-05 2024-09-21T21:42:10.184 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:10.184 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:10.185 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:10.185 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.185 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.185 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19757 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 5812224~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:10.185 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273116+0000", 2024-09-21T21:42:10.185 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 18.877667361, 2024-09-21T21:42:10.185 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97367813999999997, 2024-09-21T21:42:10.185 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:10.185 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:10.185 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:10.185 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:10.185 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:10.185 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:10.185 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19757 2024-09-21T21:42:10.185 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.186 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:10.186 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.186 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:10.186 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273116+0000", 2024-09-21T21:42:10.186 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:10.186 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.186 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.186 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:10.186 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273116+0000", 2024-09-21T21:42:10.186 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:10.186 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.186 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.186 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:10.187 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273117+0000", 2024-09-21T21:42:10.187 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0999999999999999e-07 2024-09-21T21:42:10.187 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.187 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.187 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:10.187 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273119+0000", 2024-09-21T21:42:10.187 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9970000000000001e-06 2024-09-21T21:42:10.187 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.187 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.187 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:10.187 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273125+0000", 2024-09-21T21:42:10.187 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9540000000000001e-06 2024-09-21T21:42:10.188 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.188 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.188 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:10.188 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273135+0000", 2024-09-21T21:42:10.188 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0597999999999999e-05 2024-09-21T21:42:10.188 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.188 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.188 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:10.188 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278019+0000", 2024-09-21T21:42:10.188 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048834849999999999 2024-09-21T21:42:10.188 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.188 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.188 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:10.188 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278022+0000", 2024-09-21T21:42:10.189 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9560000000000002e-06 2024-09-21T21:42:10.189 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.189 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.189 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:10.189 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246716+0000", 2024-09-21T21:42:10.189 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.968694202 2024-09-21T21:42:10.189 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.189 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.189 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:10.189 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246758+0000", 2024-09-21T21:42:10.189 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1607000000000003e-05 2024-09-21T21:42:10.189 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.189 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.189 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:10.189 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246795+0000", 2024-09-21T21:42:10.190 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6931000000000003e-05 2024-09-21T21:42:10.190 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:10.190 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:10.190 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:10.190 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.190 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.190 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19758 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6529024~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:10.190 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273142+0000", 2024-09-21T21:42:10.190 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 18.877642139999999, 2024-09-21T21:42:10.190 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97375924599999997, 2024-09-21T21:42:10.190 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:10.190 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:10.190 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:10.190 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:10.191 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:10.191 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:10.191 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19758 2024-09-21T21:42:10.191 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.191 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:10.191 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.191 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:10.191 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273142+0000", 2024-09-21T21:42:10.191 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:10.191 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.191 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.191 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:10.191 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273142+0000", 2024-09-21T21:42:10.191 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:10.191 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.192 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.192 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:10.192 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273142+0000", 2024-09-21T21:42:10.192 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0200000000000003e-07 2024-09-21T21:42:10.192 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.192 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.192 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:10.192 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273144+0000", 2024-09-21T21:42:10.192 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1390000000000002e-06 2024-09-21T21:42:10.192 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.192 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.192 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:10.192 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273150+0000", 2024-09-21T21:42:10.192 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6409999999999999e-06 2024-09-21T21:42:10.193 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.193 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.193 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:10.193 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273157+0000", 2024-09-21T21:42:10.193 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1359999999999999e-06 2024-09-21T21:42:10.193 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.193 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.193 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:10.193 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278050+0000", 2024-09-21T21:42:10.193 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048926170000000001 2024-09-21T21:42:10.193 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.193 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.193 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:10.193 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278053+0000", 2024-09-21T21:42:10.194 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4879999999999999e-06 2024-09-21T21:42:10.194 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.194 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.194 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:10.194 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246819+0000", 2024-09-21T21:42:10.194 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96876590399999996 2024-09-21T21:42:10.194 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.194 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.194 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:10.194 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246856+0000", 2024-09-21T21:42:10.194 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7287999999999999e-05 2024-09-21T21:42:10.194 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.194 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.194 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:10.195 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246901+0000", 2024-09-21T21:42:10.195 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4631e-05 2024-09-21T21:42:10.195 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:10.195 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:10.195 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:10.195 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.195 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.195 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19759 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 7933952~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:10.195 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273172+0000", 2024-09-21T21:42:10.195 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 18.877611876, 2024-09-21T21:42:10.195 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97384789500000002, 2024-09-21T21:42:10.195 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:10.195 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:10.196 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:10.196 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:10.196 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:10.196 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:10.196 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19759 2024-09-21T21:42:10.196 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.196 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:10.196 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.196 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:10.196 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273172+0000", 2024-09-21T21:42:10.196 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:10.196 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.196 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.196 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:10.196 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273172+0000", 2024-09-21T21:42:10.197 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:10.197 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.197 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.197 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:10.197 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273172+0000", 2024-09-21T21:42:10.197 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6100000000000002e-07 2024-09-21T21:42:10.197 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.197 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.197 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:10.197 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273174+0000", 2024-09-21T21:42:10.197 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.691e-06 2024-09-21T21:42:10.197 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.197 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.197 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:10.198 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273178+0000", 2024-09-21T21:42:10.198 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7579999999999998e-06 2024-09-21T21:42:10.198 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.198 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.198 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:10.198 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273180+0000", 2024-09-21T21:42:10.198 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7199999999999998e-06 2024-09-21T21:42:10.198 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.198 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.198 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:10.198 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278077+0000", 2024-09-21T21:42:10.198 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048969529999999999 2024-09-21T21:42:10.198 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.198 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.199 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:10.199 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278080+0000", 2024-09-21T21:42:10.199 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.066e-06 2024-09-21T21:42:10.199 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.199 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.199 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:10.199 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246925+0000", 2024-09-21T21:42:10.199 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96884490199999995 2024-09-21T21:42:10.199 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.199 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.199 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:10.199 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246974+0000", 2024-09-21T21:42:10.199 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8387999999999998e-05 2024-09-21T21:42:10.200 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.200 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.200 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:10.200 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247020+0000", 2024-09-21T21:42:10.200 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6156000000000003e-05 2024-09-21T21:42:10.200 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:10.200 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:10.200 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:10.200 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.200 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.200 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19760 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 7991296~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:10.200 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273220+0000", 2024-09-21T21:42:10.200 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 18.877563362, 2024-09-21T21:42:10.201 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97403125499999998, 2024-09-21T21:42:10.201 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:10.201 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:10.201 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:10.201 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:10.201 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:10.201 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:10.201 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19760 2024-09-21T21:42:10.201 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.201 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:10.201 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.201 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:10.201 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273220+0000", 2024-09-21T21:42:10.201 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:10.201 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.202 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.202 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:10.202 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273220+0000", 2024-09-21T21:42:10.202 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:10.202 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.202 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.202 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:10.202 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273221+0000", 2024-09-21T21:42:10.202 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.65e-07 2024-09-21T21:42:10.202 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.202 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.202 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:10.202 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273228+0000", 2024-09-21T21:42:10.202 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6040000000000004e-06 2024-09-21T21:42:10.203 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.203 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.203 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:10.203 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273233+0000", 2024-09-21T21:42:10.203 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1569999999999999e-06 2024-09-21T21:42:10.203 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.203 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.203 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:10.203 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273238+0000", 2024-09-21T21:42:10.203 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0050000000000004e-06 2024-09-21T21:42:10.203 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.203 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.203 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:10.203 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278107+0000", 2024-09-21T21:42:10.203 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00486857 2024-09-21T21:42:10.204 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.204 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.204 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:10.204 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278111+0000", 2024-09-21T21:42:10.204 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6389999999999998e-06 2024-09-21T21:42:10.204 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.204 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.204 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:10.204 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247069+0000", 2024-09-21T21:42:10.204 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96895796899999997 2024-09-21T21:42:10.204 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.204 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.204 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:10.204 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247166+0000", 2024-09-21T21:42:10.204 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.7158999999999996e-05 2024-09-21T21:42:10.205 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.205 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.205 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:10.205 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247252+0000", 2024-09-21T21:42:10.205 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5886999999999999e-05 2024-09-21T21:42:10.205 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:10.205 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:10.205 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:10.205 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.205 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.205 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19761 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 7962624~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:10.205 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273246+0000", 2024-09-21T21:42:10.205 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 18.877538302000001, 2024-09-21T21:42:10.205 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97415972200000001, 2024-09-21T21:42:10.205 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:10.205 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:10.205 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:10.206 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:10.206 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:10.206 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:10.206 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19761 2024-09-21T21:42:10.206 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.206 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:10.206 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.206 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:10.206 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273246+0000", 2024-09-21T21:42:10.206 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:10.206 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.206 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.206 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:10.206 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273246+0000", 2024-09-21T21:42:10.206 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:10.207 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.207 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.207 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:10.207 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273246+0000", 2024-09-21T21:42:10.207 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5100000000000001e-07 2024-09-21T21:42:10.207 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.207 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.207 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:10.207 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273248+0000", 2024-09-21T21:42:10.207 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1550000000000001e-06 2024-09-21T21:42:10.207 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.207 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.207 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:10.208 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273254+0000", 2024-09-21T21:42:10.208 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.964e-06 2024-09-21T21:42:10.208 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.208 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.208 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:10.208 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273259+0000", 2024-09-21T21:42:10.208 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5499999999999996e-06 2024-09-21T21:42:10.208 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.208 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.208 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:10.208 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278130+0000", 2024-09-21T21:42:10.208 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048715779999999997 2024-09-21T21:42:10.208 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.208 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.209 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:10.209 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278133+0000", 2024-09-21T21:42:10.209 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8739999999999999e-06 2024-09-21T21:42:10.209 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.209 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.209 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:10.209 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247270+0000", 2024-09-21T21:42:10.209 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.969137099 2024-09-21T21:42:10.209 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.209 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.209 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:10.209 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247338+0000", 2024-09-21T21:42:10.209 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7717000000000002e-05 2024-09-21T21:42:10.210 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.210 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.210 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:10.210 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247405+0000", 2024-09-21T21:42:10.210 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7434000000000005e-05 2024-09-21T21:42:10.210 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:10.210 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:10.210 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:10.210 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.210 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.210 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19769 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 5840896~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:10.210 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273287+0000", 2024-09-21T21:42:10.210 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 18.877497018, 2024-09-21T21:42:10.210 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97435390200000005, 2024-09-21T21:42:10.210 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:10.211 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:10.211 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:10.211 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:10.211 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:10.211 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:10.211 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19769 2024-09-21T21:42:10.211 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.211 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:10.211 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.211 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:10.211 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273287+0000", 2024-09-21T21:42:10.211 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:10.211 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.212 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.212 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:10.212 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273287+0000", 2024-09-21T21:42:10.212 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:10.212 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.212 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.212 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:10.212 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273287+0000", 2024-09-21T21:42:10.212 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6699999999999999e-07 2024-09-21T21:42:10.212 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.212 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.212 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:10.212 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273289+0000", 2024-09-21T21:42:10.212 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.708e-06 2024-09-21T21:42:10.213 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.213 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.213 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:10.213 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273296+0000", 2024-09-21T21:42:10.213 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8109999999999997e-06 2024-09-21T21:42:10.213 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.213 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.213 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:10.213 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273299+0000", 2024-09-21T21:42:10.213 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0740000000000001e-06 2024-09-21T21:42:10.213 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.213 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.213 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:10.213 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278159+0000", 2024-09-21T21:42:10.214 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048603240000000001 2024-09-21T21:42:10.214 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.214 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.214 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:10.214 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278162+0000", 2024-09-21T21:42:10.214 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1930000000000002e-06 2024-09-21T21:42:10.214 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.214 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.214 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:10.214 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247444+0000", 2024-09-21T21:42:10.214 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96928199100000001 2024-09-21T21:42:10.214 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.214 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.215 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:10.215 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247533+0000", 2024-09-21T21:42:10.215 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8490000000000001e-05 2024-09-21T21:42:10.215 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.215 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.215 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:10.215 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247641+0000", 2024-09-21T21:42:10.215 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000107944 2024-09-21T21:42:10.215 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:10.215 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:10.215 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:10.215 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.215 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.215 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19770 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 5869568~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:10.216 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273313+0000", 2024-09-21T21:42:10.216 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 18.877470916, 2024-09-21T21:42:10.216 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97449038899999996, 2024-09-21T21:42:10.216 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:10.216 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:10.216 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:10.216 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:10.216 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:10.216 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:10.216 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19770 2024-09-21T21:42:10.216 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.216 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:10.216 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.216 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:10.217 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273313+0000", 2024-09-21T21:42:10.217 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:10.217 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.217 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.217 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:10.217 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273313+0000", 2024-09-21T21:42:10.217 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:10.217 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.217 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.217 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:10.217 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273313+0000", 2024-09-21T21:42:10.217 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5699999999999998e-07 2024-09-21T21:42:10.217 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.217 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.218 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:10.218 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273315+0000", 2024-09-21T21:42:10.218 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0099999999999998e-06 2024-09-21T21:42:10.218 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.218 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.218 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:10.218 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273321+0000", 2024-09-21T21:42:10.218 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7529999999999998e-06 2024-09-21T21:42:10.218 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.218 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.218 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:10.218 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273328+0000", 2024-09-21T21:42:10.218 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9009999999999997e-06 2024-09-21T21:42:10.218 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.219 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.219 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:10.219 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278183+0000", 2024-09-21T21:42:10.219 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048547770000000002 2024-09-21T21:42:10.219 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.219 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.219 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:10.219 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278186+0000", 2024-09-21T21:42:10.219 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3100000000000001e-06 2024-09-21T21:42:10.219 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.219 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.219 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:10.219 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247676+0000", 2024-09-21T21:42:10.220 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96949004299999997 2024-09-21T21:42:10.220 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.220 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.220 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:10.220 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247757+0000", 2024-09-21T21:42:10.220 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0479e-05 2024-09-21T21:42:10.220 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.220 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.220 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:10.220 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247803+0000", 2024-09-21T21:42:10.220 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6758999999999997e-05 2024-09-21T21:42:10.220 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:10.220 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:10.220 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:10.221 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.221 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.221 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19771 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6586368~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:10.221 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273335+0000", 2024-09-21T21:42:10.221 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 18.877449128999999, 2024-09-21T21:42:10.221 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97458947699999998, 2024-09-21T21:42:10.221 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:10.221 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:10.221 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:10.221 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:10.221 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:10.221 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:10.221 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19771 2024-09-21T21:42:10.221 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.222 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:10.222 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.222 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:10.222 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273335+0000", 2024-09-21T21:42:10.222 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:10.222 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.222 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.222 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:10.222 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273335+0000", 2024-09-21T21:42:10.222 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:10.222 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.222 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.222 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:10.222 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273335+0000", 2024-09-21T21:42:10.223 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3599999999999999e-07 2024-09-21T21:42:10.223 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.223 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.223 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:10.223 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273337+0000", 2024-09-21T21:42:10.223 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0789999999999999e-06 2024-09-21T21:42:10.223 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.223 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.223 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:10.223 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273343+0000", 2024-09-21T21:42:10.223 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6280000000000002e-06 2024-09-21T21:42:10.223 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.223 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.223 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:10.223 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273346+0000", 2024-09-21T21:42:10.224 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.507e-06 2024-09-21T21:42:10.224 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.224 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.224 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:10.224 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278206+0000", 2024-09-21T21:42:10.224 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048600969999999999 2024-09-21T21:42:10.224 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.224 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.224 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:10.224 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278212+0000", 2024-09-21T21:42:10.224 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2079999999999999e-06 2024-09-21T21:42:10.224 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.224 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.224 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:10.225 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247828+0000", 2024-09-21T21:42:10.225 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.969616547 2024-09-21T21:42:10.225 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.225 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.225 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:10.225 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247881+0000", 2024-09-21T21:42:10.225 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3180999999999998e-05 2024-09-21T21:42:10.225 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.225 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.225 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:10.225 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247924+0000", 2024-09-21T21:42:10.225 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2894e-05 2024-09-21T21:42:10.225 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:10.226 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:10.226 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:10.226 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.226 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.226 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19772 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6557696~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:10.226 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273358+0000", 2024-09-21T21:42:10.226 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 18.877425749, 2024-09-21T21:42:10.226 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97467031800000004, 2024-09-21T21:42:10.226 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:10.226 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:10.226 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:10.226 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:10.226 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:10.226 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:10.227 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19772 2024-09-21T21:42:10.227 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.227 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:10.227 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.227 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:10.227 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273358+0000", 2024-09-21T21:42:10.227 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:10.227 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.227 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.227 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:10.227 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273358+0000", 2024-09-21T21:42:10.227 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:10.227 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.227 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.227 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:10.228 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273358+0000", 2024-09-21T21:42:10.228 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4499999999999998e-07 2024-09-21T21:42:10.228 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.228 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.228 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:10.228 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273360+0000", 2024-09-21T21:42:10.228 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9190000000000002e-06 2024-09-21T21:42:10.228 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.228 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.228 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:10.228 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273366+0000", 2024-09-21T21:42:10.228 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.891e-06 2024-09-21T21:42:10.228 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.228 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.229 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:10.229 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273371+0000", 2024-09-21T21:42:10.229 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1880000000000001e-06 2024-09-21T21:42:10.229 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.229 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.229 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:10.229 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278237+0000", 2024-09-21T21:42:10.229 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.004865472 2024-09-21T21:42:10.229 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.229 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.229 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:10.229 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278240+0000", 2024-09-21T21:42:10.229 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9550000000000001e-06 2024-09-21T21:42:10.229 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.230 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.230 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:10.230 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247949+0000", 2024-09-21T21:42:10.230 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.969709127 2024-09-21T21:42:10.230 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.230 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.230 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:10.230 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247990+0000", 2024-09-21T21:42:10.230 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1202000000000003e-05 2024-09-21T21:42:10.230 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.230 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.230 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:10.230 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248028+0000", 2024-09-21T21:42:10.230 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8219000000000001e-05 2024-09-21T21:42:10.231 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:10.231 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:10.231 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:10.231 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.231 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.231 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19773 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6615040~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:10.231 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273390+0000", 2024-09-21T21:42:10.231 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 18.877393888, 2024-09-21T21:42:10.231 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97476086200000001, 2024-09-21T21:42:10.231 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:10.231 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:10.231 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:10.231 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:10.231 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:10.232 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:10.232 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19773 2024-09-21T21:42:10.232 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.232 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:10.232 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.232 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:10.232 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273390+0000", 2024-09-21T21:42:10.232 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:10.232 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.232 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.232 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:10.232 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273390+0000", 2024-09-21T21:42:10.232 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:10.232 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.233 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.233 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:10.233 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273390+0000", 2024-09-21T21:42:10.233 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9999999999999999e-07 2024-09-21T21:42:10.233 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.233 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.233 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:10.233 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273392+0000", 2024-09-21T21:42:10.233 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0640000000000001e-06 2024-09-21T21:42:10.233 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.233 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.233 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:10.233 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273397+0000", 2024-09-21T21:42:10.233 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.113e-06 2024-09-21T21:42:10.233 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.234 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.234 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:10.234 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273401+0000", 2024-09-21T21:42:10.234 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3119999999999998e-06 2024-09-21T21:42:10.234 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.234 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.234 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:10.234 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278271+0000", 2024-09-21T21:42:10.234 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048698880000000002 2024-09-21T21:42:10.234 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.234 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.234 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:10.234 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278278+0000", 2024-09-21T21:42:10.234 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1210000000000001e-06 2024-09-21T21:42:10.235 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.235 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.235 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:10.235 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248060+0000", 2024-09-21T21:42:10.235 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96978224999999996 2024-09-21T21:42:10.235 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.235 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.235 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:10.235 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248111+0000", 2024-09-21T21:42:10.235 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.096e-05 2024-09-21T21:42:10.235 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.235 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.235 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:10.235 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248151+0000", 2024-09-21T21:42:10.235 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9854000000000002e-05 2024-09-21T21:42:10.236 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:10.236 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:10.236 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:10.236 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.236 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.236 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19784 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6643712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:10.236 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273405+0000", 2024-09-21T21:42:10.236 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 18.877379167000001, 2024-09-21T21:42:10.236 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97491798900000004, 2024-09-21T21:42:10.236 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:10.236 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:10.236 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:10.236 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:10.237 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:10.237 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:10.237 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19784 2024-09-21T21:42:10.237 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.237 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:10.237 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.237 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:10.237 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273405+0000", 2024-09-21T21:42:10.237 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:10.237 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.237 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.237 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:10.237 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273405+0000", 2024-09-21T21:42:10.237 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:10.238 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.238 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.238 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:10.238 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273405+0000", 2024-09-21T21:42:10.238 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2100000000000001e-07 2024-09-21T21:42:10.238 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.238 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.238 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:10.238 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273406+0000", 2024-09-21T21:42:10.238 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2049999999999999e-06 2024-09-21T21:42:10.238 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.238 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.238 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:10.239 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273409+0000", 2024-09-21T21:42:10.239 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0599999999999999e-06 2024-09-21T21:42:10.239 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.239 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.239 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:10.239 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273412+0000", 2024-09-21T21:42:10.239 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0730000000000001e-06 2024-09-21T21:42:10.239 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.239 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.239 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:10.239 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278317+0000", 2024-09-21T21:42:10.239 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0049045520000000004 2024-09-21T21:42:10.239 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.239 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.240 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:10.240 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278321+0000", 2024-09-21T21:42:10.240 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6249999999999998e-06 2024-09-21T21:42:10.240 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.240 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.240 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:10.240 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248179+0000", 2024-09-21T21:42:10.240 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96985744500000004 2024-09-21T21:42:10.240 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.240 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.240 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:10.240 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248253+0000", 2024-09-21T21:42:10.240 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3829000000000001e-05 2024-09-21T21:42:10.240 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.241 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.241 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:10.241 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248323+0000", 2024-09-21T21:42:10.241 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9979000000000002e-05 2024-09-21T21:42:10.241 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:10.241 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:10.241 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:10.241 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.241 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.241 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19785 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6672384~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:10.241 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273415+0000", 2024-09-21T21:42:10.241 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 18.877368580999999, 2024-09-21T21:42:10.241 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97500777800000005, 2024-09-21T21:42:10.241 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:10.242 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:10.242 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:10.242 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:10.242 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:10.242 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:10.242 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19785 2024-09-21T21:42:10.242 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.242 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:10.242 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.242 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:10.242 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273415+0000", 2024-09-21T21:42:10.242 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:10.242 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.242 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.242 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:10.243 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273415+0000", 2024-09-21T21:42:10.243 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:10.243 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.243 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.243 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:10.243 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273415+0000", 2024-09-21T21:42:10.243 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0100000000000001e-07 2024-09-21T21:42:10.243 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.243 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.243 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:10.243 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273417+0000", 2024-09-21T21:42:10.243 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.114e-06 2024-09-21T21:42:10.243 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.243 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.243 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:10.244 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273420+0000", 2024-09-21T21:42:10.244 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6770000000000001e-06 2024-09-21T21:42:10.244 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.244 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.244 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:10.244 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273423+0000", 2024-09-21T21:42:10.244 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5450000000000002e-06 2024-09-21T21:42:10.244 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.244 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.244 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:10.244 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278355+0000", 2024-09-21T21:42:10.244 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0049318590000000002 2024-09-21T21:42:10.244 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.244 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.245 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:10.245 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278360+0000", 2024-09-21T21:42:10.245 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3079999999999998e-06 2024-09-21T21:42:10.245 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.245 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.245 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:10.245 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248349+0000", 2024-09-21T21:42:10.245 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96998909899999997 2024-09-21T21:42:10.245 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.245 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.245 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:10.245 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248387+0000", 2024-09-21T21:42:10.245 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7657000000000002e-05 2024-09-21T21:42:10.245 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.246 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.246 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:10.246 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248423+0000", 2024-09-21T21:42:10.246 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6318e-05 2024-09-21T21:42:10.246 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:10.246 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:10.246 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:10.246 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.246 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.246 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19842 3.d 3:b7fa2ca0:::rbd_data.112157a479c.000000000000004d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:10.246 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273426+0000", 2024-09-21T21:42:10.246 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 18.877357529000001, 2024-09-21T21:42:10.246 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.98136526800000001, 2024-09-21T21:42:10.246 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:10.246 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:10.247 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:10.247 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:10.247 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:10.247 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:10.247 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19842 2024-09-21T21:42:10.247 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.247 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:10.247 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.247 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:10.247 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273426+0000", 2024-09-21T21:42:10.247 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:10.247 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.247 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.247 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:10.247 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273426+0000", 2024-09-21T21:42:10.248 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:10.248 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.248 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.248 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:10.248 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273427+0000", 2024-09-21T21:42:10.248 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.72e-07 2024-09-21T21:42:10.248 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.248 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.248 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:10.248 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275836+0000", 2024-09-21T21:42:10.248 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00240909 2024-09-21T21:42:10.248 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.248 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.248 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:10.249 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275844+0000", 2024-09-21T21:42:10.249 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3750000000000003e-06 2024-09-21T21:42:10.249 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.249 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.249 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:10.249 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275849+0000", 2024-09-21T21:42:10.249 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0100000000000003e-06 2024-09-21T21:42:10.249 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.249 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.249 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:10.249 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278394+0000", 2024-09-21T21:42:10.249 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025449890000000001 2024-09-21T21:42:10.249 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.249 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.249 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:10.250 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278402+0000", 2024-09-21T21:42:10.250 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6359999999999994e-06 2024-09-21T21:42:10.250 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.250 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.250 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:10.250 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248454+0000", 2024-09-21T21:42:10.250 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97005259499999996 2024-09-21T21:42:10.250 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.250 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.250 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:10.250 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248576+0000", 2024-09-21T21:42:10.250 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000122127 2024-09-21T21:42:10.250 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.250 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.251 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 2", 2024-09-21T21:42:10.251 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248844+0000", 2024-09-21T21:42:10.251 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00026784699999999999 2024-09-21T21:42:10.251 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.251 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.251 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:10.251 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.252487+0000", 2024-09-21T21:42:10.251 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0036427920000000002 2024-09-21T21:42:10.251 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.251 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.251 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:10.251 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.254691+0000", 2024-09-21T21:42:10.251 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002203899 2024-09-21T21:42:10.251 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.252 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.252 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:10.252 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.254748+0000", 2024-09-21T21:42:10.252 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7163000000000001e-05 2024-09-21T21:42:10.252 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.252 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.252 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:10.252 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.254792+0000", 2024-09-21T21:42:10.252 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3473000000000002e-05 2024-09-21T21:42:10.252 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:10.252 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:10.252 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:10.252 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.252 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.253 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19843 3.d 3:b7fa2ca0:::rbd_data.112157a479c.000000000000004d:head [write 0~0] snapc 21=[21] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:10.253 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.275860+0000", 2024-09-21T21:42:10.253 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 18.874923629000001, 2024-09-21T21:42:10.253 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.98183651999999999, 2024-09-21T21:42:10.253 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:10.253 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:10.253 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:10.253 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:10.253 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:10.253 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:10.253 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19843 2024-09-21T21:42:10.253 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.253 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:10.253 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.253 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:10.254 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275860+0000", 2024-09-21T21:42:10.254 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:10.254 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.254 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.254 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:10.254 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275860+0000", 2024-09-21T21:42:10.254 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:10.254 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.254 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.254 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:10.254 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275869+0000", 2024-09-21T21:42:10.254 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6300000000000004e-06 2024-09-21T21:42:10.254 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.254 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.255 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:10.255 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275871+0000", 2024-09-21T21:42:10.255 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2060000000000001e-06 2024-09-21T21:42:10.255 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.255 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.255 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:10.255 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275876+0000", 2024-09-21T21:42:10.255 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8180000000000003e-06 2024-09-21T21:42:10.255 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.255 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.255 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:10.255 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275880+0000", 2024-09-21T21:42:10.256 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1829999999999996e-06 2024-09-21T21:42:10.256 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.256 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.256 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:10.256 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278416+0000", 2024-09-21T21:42:10.256 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025361149999999998 2024-09-21T21:42:10.256 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.256 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.256 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:10.256 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278419+0000", 2024-09-21T21:42:10.256 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.163e-06 2024-09-21T21:42:10.256 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.256 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.256 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:10.257 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.250249+0000", 2024-09-21T21:42:10.257 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97182984299999997 2024-09-21T21:42:10.257 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.257 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.257 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:10.257 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.250321+0000", 2024-09-21T21:42:10.257 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1656000000000006e-05 2024-09-21T21:42:10.257 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.257 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.257 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 2", 2024-09-21T21:42:10.257 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.250419+0000", 2024-09-21T21:42:10.257 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.8277999999999997e-05 2024-09-21T21:42:10.257 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.258 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.258 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:10.258 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.252516+0000", 2024-09-21T21:42:10.258 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002097224 2024-09-21T21:42:10.258 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.258 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.258 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:10.258 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.257643+0000", 2024-09-21T21:42:10.258 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0051262319999999997 2024-09-21T21:42:10.258 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.258 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.258 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:10.258 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.257673+0000", 2024-09-21T21:42:10.259 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0023e-05 2024-09-21T21:42:10.259 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.259 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.259 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:10.259 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.257697+0000", 2024-09-21T21:42:10.259 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4148999999999999e-05 2024-09-21T21:42:10.259 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:10.259 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:10.259 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:10.259 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:10.259 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:10.259 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:10.259 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 4, 5, 3] out_osds: [0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-21T21:42:10.260 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T21:42:10.260 INFO:tasks.thrashosds.thrasher:Removing osd 4, in_osds are: [1, 2, 4, 5, 3] 2024-09-21T21:42:10.260 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 4 2024-09-21T21:42:10.262 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T21:42:10.262 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:10.260+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:10.362 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:10.360+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:10.365 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:10.365 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:42:10.365 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-21T21:42:10.365 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:10.374 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T21:42:10.463 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:10.461+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:10.485 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:10.485 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:42:10.485 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:42:10.485 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:42:10.485 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:10.495 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T21:42:10.502 INFO:teuthology.orchestra.run.smithi164.stdout:truncating image image_client.0-clone36 from 0x555c800 (overlap 0x18fe00) to 0x13fe00 2024-09-21T21:42:10.557 INFO:tasks.ceph.mon.a.smithi081.stderr:2024-09-21T21:42:10.556+0000 7f5d67cb4640 -1 mon.a@0(leader).osd e95 definitely_dead 0 2024-09-21T21:42:10.564 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:10.563+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:10.607 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:10.607 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-21T21:42:10.607 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-21T21:42:10.607 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:42:10.607 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.607 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6974 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5246976~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:10.607 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.922932+0000", 2024-09-21T21:42:10.607 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 40.681846442999998, 2024-09-21T21:42:10.607 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041773352999999999, 2024-09-21T21:42:10.608 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:10.608 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:10.608 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:10.608 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:10.608 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:10.608 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:10.608 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6974 2024-09-21T21:42:10.608 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.608 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:10.608 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.608 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:10.608 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922932+0000", 2024-09-21T21:42:10.608 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:10.608 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.609 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.609 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:10.609 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922932+0000", 2024-09-21T21:42:10.609 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:10.609 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.609 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.609 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:10.609 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922933+0000", 2024-09-21T21:42:10.609 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.46e-07 2024-09-21T21:42:10.609 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.609 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.609 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:10.609 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922940+0000", 2024-09-21T21:42:10.609 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6140000000000003e-06 2024-09-21T21:42:10.609 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.610 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.610 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:10.610 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922949+0000", 2024-09-21T21:42:10.610 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6899999999999998e-06 2024-09-21T21:42:10.610 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.610 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.610 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:10.610 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922955+0000", 2024-09-21T21:42:10.610 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5140000000000001e-06 2024-09-21T21:42:10.610 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.610 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.610 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:10.610 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964547+0000", 2024-09-21T21:42:10.610 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041592854999999998 2024-09-21T21:42:10.611 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.611 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.611 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:10.611 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964617+0000", 2024-09-21T21:42:10.611 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9058000000000002e-05 2024-09-21T21:42:10.611 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.611 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.611 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:10.611 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964706+0000", 2024-09-21T21:42:10.611 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9276000000000004e-05 2024-09-21T21:42:10.611 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:10.611 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:10.612 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:10.612 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.612 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.612 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6976 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5261312~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:10.612 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.922961+0000", 2024-09-21T21:42:10.612 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 40.681817398, 2024-09-21T21:42:10.612 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041863648000000003, 2024-09-21T21:42:10.612 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:10.612 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:10.612 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:10.612 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:10.612 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:10.612 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:10.613 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6976 2024-09-21T21:42:10.613 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.613 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:10.613 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.613 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:10.613 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922961+0000", 2024-09-21T21:42:10.613 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:10.613 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.613 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.613 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:10.613 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922961+0000", 2024-09-21T21:42:10.613 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:10.613 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.613 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.614 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:10.614 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922962+0000", 2024-09-21T21:42:10.614 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.89e-07 2024-09-21T21:42:10.614 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.614 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.614 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:10.614 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922964+0000", 2024-09-21T21:42:10.614 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1270000000000001e-06 2024-09-21T21:42:10.614 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.614 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.614 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:10.614 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922970+0000", 2024-09-21T21:42:10.614 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8030000000000002e-06 2024-09-21T21:42:10.614 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.614 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.615 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:10.615 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922979+0000", 2024-09-21T21:42:10.615 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5559999999999996e-06 2024-09-21T21:42:10.615 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.615 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.615 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:10.615 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964734+0000", 2024-09-21T21:42:10.615 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041754415000000003 2024-09-21T21:42:10.615 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.615 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.615 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:10.615 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964775+0000", 2024-09-21T21:42:10.615 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0806999999999997e-05 2024-09-21T21:42:10.615 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.616 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.616 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:10.616 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964825+0000", 2024-09-21T21:42:10.616 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0550999999999998e-05 2024-09-21T21:42:10.616 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:10.616 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:10.616 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:10.616 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.616 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.616 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6978 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5275648~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:10.616 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.922987+0000", 2024-09-21T21:42:10.616 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 40.681791613000001, 2024-09-21T21:42:10.616 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042043876000000001, 2024-09-21T21:42:10.616 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:10.617 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:10.617 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:10.617 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:10.617 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:10.617 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:10.617 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6978 2024-09-21T21:42:10.617 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.617 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:10.617 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.617 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:10.617 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922987+0000", 2024-09-21T21:42:10.617 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:10.617 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.617 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.617 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:10.618 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922987+0000", 2024-09-21T21:42:10.618 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:10.618 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.618 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.618 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:10.618 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922988+0000", 2024-09-21T21:42:10.618 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1199999999999998e-07 2024-09-21T21:42:10.618 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.618 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.618 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:10.618 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922990+0000", 2024-09-21T21:42:10.618 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.142e-06 2024-09-21T21:42:10.618 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.618 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.619 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:10.619 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922997+0000", 2024-09-21T21:42:10.619 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.199e-06 2024-09-21T21:42:10.619 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.619 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.619 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:10.619 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923004+0000", 2024-09-21T21:42:10.619 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3599999999999998e-06 2024-09-21T21:42:10.619 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.619 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.619 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:10.619 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964856+0000", 2024-09-21T21:42:10.619 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041851521000000003 2024-09-21T21:42:10.619 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.620 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.620 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:10.620 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964944+0000", 2024-09-21T21:42:10.620 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7859e-05 2024-09-21T21:42:10.620 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.620 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.620 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:10.620 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965031+0000", 2024-09-21T21:42:10.620 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7383e-05 2024-09-21T21:42:10.620 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:10.620 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:10.620 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:10.620 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.620 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.621 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6980 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5289984~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:10.621 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923011+0000", 2024-09-21T21:42:10.621 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 40.681768276, 2024-09-21T21:42:10.621 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042139337999999998, 2024-09-21T21:42:10.621 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:10.621 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:10.621 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:10.621 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:10.621 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:10.621 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:10.621 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6980 2024-09-21T21:42:10.621 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.621 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:10.621 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.621 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:10.622 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923011+0000", 2024-09-21T21:42:10.622 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:10.622 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.622 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.622 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:10.622 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923011+0000", 2024-09-21T21:42:10.622 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:10.622 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.622 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.622 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:10.622 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923011+0000", 2024-09-21T21:42:10.622 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6699999999999999e-07 2024-09-21T21:42:10.622 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.623 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.623 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:10.623 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923013+0000", 2024-09-21T21:42:10.623 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0279999999999999e-06 2024-09-21T21:42:10.623 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.623 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.623 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:10.623 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923018+0000", 2024-09-21T21:42:10.623 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4790000000000004e-06 2024-09-21T21:42:10.623 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.623 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.623 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:10.623 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923029+0000", 2024-09-21T21:42:10.623 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0601999999999999e-05 2024-09-21T21:42:10.623 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.624 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.624 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:10.624 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965059+0000", 2024-09-21T21:42:10.624 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042029947999999998 2024-09-21T21:42:10.624 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.624 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.624 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:10.624 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965100+0000", 2024-09-21T21:42:10.624 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1170000000000001e-05 2024-09-21T21:42:10.624 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.624 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.624 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:10.624 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965150+0000", 2024-09-21T21:42:10.624 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9743999999999997e-05 2024-09-21T21:42:10.625 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:10.625 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:10.625 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:10.625 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.625 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.625 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6982 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5304320~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:10.625 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923036+0000", 2024-09-21T21:42:10.625 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 40.681742454999998, 2024-09-21T21:42:10.625 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042225666000000002, 2024-09-21T21:42:10.625 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:10.625 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:10.625 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:10.625 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:10.625 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:10.625 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:10.626 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6982 2024-09-21T21:42:10.626 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.626 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:10.626 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.626 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:10.626 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923036+0000", 2024-09-21T21:42:10.626 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:10.626 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.626 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.626 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:10.626 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923036+0000", 2024-09-21T21:42:10.626 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:10.626 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.626 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.627 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:10.627 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923037+0000", 2024-09-21T21:42:10.627 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2800000000000002e-07 2024-09-21T21:42:10.627 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.627 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.627 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:10.627 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923039+0000", 2024-09-21T21:42:10.627 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1500000000000002e-06 2024-09-21T21:42:10.627 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.627 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.627 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:10.627 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923049+0000", 2024-09-21T21:42:10.627 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0236e-05 2024-09-21T21:42:10.627 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.627 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.628 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:10.628 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923056+0000", 2024-09-21T21:42:10.628 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2540000000000003e-06 2024-09-21T21:42:10.628 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.628 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.628 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:10.628 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965173+0000", 2024-09-21T21:42:10.628 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042116686 2024-09-21T21:42:10.628 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.628 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.628 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:10.628 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965215+0000", 2024-09-21T21:42:10.628 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1696000000000002e-05 2024-09-21T21:42:10.628 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.628 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.629 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:10.629 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965262+0000", 2024-09-21T21:42:10.629 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7216000000000003e-05 2024-09-21T21:42:10.629 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:10.629 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:10.629 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:10.629 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.629 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.629 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6984 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5318656~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:10.629 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923063+0000", 2024-09-21T21:42:10.629 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 40.681715801000003, 2024-09-21T21:42:10.629 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042318651999999998, 2024-09-21T21:42:10.629 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:10.629 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:10.630 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:10.630 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:10.630 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:10.630 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:10.630 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6984 2024-09-21T21:42:10.630 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.630 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:10.630 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.630 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:10.630 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923063+0000", 2024-09-21T21:42:10.630 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:10.630 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.630 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.630 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:10.631 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923063+0000", 2024-09-21T21:42:10.631 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:10.631 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.631 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.631 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:10.631 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923064+0000", 2024-09-21T21:42:10.631 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2e-07 2024-09-21T21:42:10.631 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.631 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.631 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:10.631 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923066+0000", 2024-09-21T21:42:10.631 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.057e-06 2024-09-21T21:42:10.631 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.631 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.632 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:10.632 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923071+0000", 2024-09-21T21:42:10.632 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8810000000000001e-06 2024-09-21T21:42:10.632 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.632 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.632 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:10.632 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923078+0000", 2024-09-21T21:42:10.632 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5479999999999998e-06 2024-09-21T21:42:10.632 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.632 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.632 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:10.632 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965288+0000", 2024-09-21T21:42:10.632 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042209576999999998 2024-09-21T21:42:10.632 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.632 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.633 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:10.633 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965332+0000", 2024-09-21T21:42:10.633 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4366999999999997e-05 2024-09-21T21:42:10.633 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.633 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.633 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:10.633 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965382+0000", 2024-09-21T21:42:10.633 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9802000000000002e-05 2024-09-21T21:42:10.633 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:10.633 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:10.633 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:10.633 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.633 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.633 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6987 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5332992~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:10.634 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923085+0000", 2024-09-21T21:42:10.634 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 40.681694276000002, 2024-09-21T21:42:10.634 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042412103999999999, 2024-09-21T21:42:10.634 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:10.634 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:10.634 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:10.634 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:10.634 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:10.634 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:10.634 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6987 2024-09-21T21:42:10.634 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.634 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:10.634 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.635 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:10.635 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923085+0000", 2024-09-21T21:42:10.635 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:10.635 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.635 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.635 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:10.635 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923085+0000", 2024-09-21T21:42:10.635 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:10.635 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.635 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.635 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:10.635 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923085+0000", 2024-09-21T21:42:10.635 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.89e-07 2024-09-21T21:42:10.635 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.635 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.636 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:10.636 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923087+0000", 2024-09-21T21:42:10.636 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0810000000000001e-06 2024-09-21T21:42:10.636 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.636 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.636 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:10.636 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923098+0000", 2024-09-21T21:42:10.636 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1299e-05 2024-09-21T21:42:10.636 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.636 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.636 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:10.636 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923104+0000", 2024-09-21T21:42:10.636 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9100000000000002e-06 2024-09-21T21:42:10.636 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.637 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.637 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:10.637 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965404+0000", 2024-09-21T21:42:10.637 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042299622000000002 2024-09-21T21:42:10.637 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.637 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.637 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:10.637 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965445+0000", 2024-09-21T21:42:10.637 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0701000000000002e-05 2024-09-21T21:42:10.637 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.637 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.637 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:10.637 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965497+0000", 2024-09-21T21:42:10.637 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2101999999999997e-05 2024-09-21T21:42:10.638 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:10.638 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:10.638 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:10.638 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.638 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.638 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6989 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5347328~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:10.638 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923111+0000", 2024-09-21T21:42:10.638 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 40.681667935999997, 2024-09-21T21:42:10.638 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042505188999999999, 2024-09-21T21:42:10.638 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:10.638 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:10.638 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:10.638 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:10.638 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:10.639 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:10.639 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6989 2024-09-21T21:42:10.639 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.639 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:10.639 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.639 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:10.639 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923111+0000", 2024-09-21T21:42:10.639 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:10.639 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.639 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.639 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:10.639 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923111+0000", 2024-09-21T21:42:10.639 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:10.639 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.639 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.640 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:10.640 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923111+0000", 2024-09-21T21:42:10.640 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4499999999999997e-07 2024-09-21T21:42:10.640 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.640 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.640 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:10.640 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923114+0000", 2024-09-21T21:42:10.640 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1569999999999998e-06 2024-09-21T21:42:10.640 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.640 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.640 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:10.640 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923120+0000", 2024-09-21T21:42:10.640 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1759999999999998e-06 2024-09-21T21:42:10.640 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.641 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.641 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:10.641 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923125+0000", 2024-09-21T21:42:10.641 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2560000000000002e-06 2024-09-21T21:42:10.641 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.641 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.641 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:10.641 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965523+0000", 2024-09-21T21:42:10.641 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042398094999999997 2024-09-21T21:42:10.641 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.641 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.641 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:10.641 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965566+0000", 2024-09-21T21:42:10.641 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3393999999999998e-05 2024-09-21T21:42:10.642 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.642 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.642 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:10.642 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965616+0000", 2024-09-21T21:42:10.642 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9666000000000002e-05 2024-09-21T21:42:10.642 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:10.642 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:10.642 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:10.642 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.642 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.642 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6991 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5361664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:10.642 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923132+0000", 2024-09-21T21:42:10.642 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 40.681646647000001, 2024-09-21T21:42:10.642 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042600779999999998, 2024-09-21T21:42:10.643 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:10.643 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:10.643 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:10.643 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:10.643 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:10.643 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:10.643 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6991 2024-09-21T21:42:10.643 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.643 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:10.643 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.643 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:10.643 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923132+0000", 2024-09-21T21:42:10.643 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:10.644 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.644 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.644 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:10.644 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923132+0000", 2024-09-21T21:42:10.644 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:10.644 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.644 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.644 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:10.644 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923132+0000", 2024-09-21T21:42:10.644 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4200000000000002e-07 2024-09-21T21:42:10.644 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.644 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.644 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:10.644 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923134+0000", 2024-09-21T21:42:10.644 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2270000000000001e-06 2024-09-21T21:42:10.645 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.645 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.645 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:10.645 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923137+0000", 2024-09-21T21:42:10.645 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1930000000000002e-06 2024-09-21T21:42:10.645 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.645 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.645 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:10.645 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923139+0000", 2024-09-21T21:42:10.645 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4710000000000001e-06 2024-09-21T21:42:10.645 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.645 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.645 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:10.645 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965639+0000", 2024-09-21T21:42:10.646 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.04249965 2024-09-21T21:42:10.646 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.646 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.646 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:10.646 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965679+0000", 2024-09-21T21:42:10.646 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9852999999999999e-05 2024-09-21T21:42:10.646 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.646 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.646 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:10.646 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965733+0000", 2024-09-21T21:42:10.646 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4144000000000002e-05 2024-09-21T21:42:10.646 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:10.646 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:10.647 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:10.647 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.647 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.647 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6993 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5384192~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:10.647 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923143+0000", 2024-09-21T21:42:10.647 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 40.681635853000003, 2024-09-21T21:42:10.647 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042708182999999997, 2024-09-21T21:42:10.647 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:10.647 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:10.647 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:10.647 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:10.647 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:10.647 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:10.647 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6993 2024-09-21T21:42:10.647 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.647 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:10.647 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.648 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:10.648 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923143+0000", 2024-09-21T21:42:10.648 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:10.648 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.648 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.648 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:10.648 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923143+0000", 2024-09-21T21:42:10.648 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:10.648 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.648 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.648 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:10.648 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923143+0000", 2024-09-21T21:42:10.648 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0800000000000001e-07 2024-09-21T21:42:10.648 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.648 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.649 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:10.649 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923144+0000", 2024-09-21T21:42:10.649 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.009e-06 2024-09-21T21:42:10.649 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.649 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.649 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:10.649 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923148+0000", 2024-09-21T21:42:10.649 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.737e-06 2024-09-21T21:42:10.649 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.649 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.649 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:10.649 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923150+0000", 2024-09-21T21:42:10.649 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4669999999999998e-06 2024-09-21T21:42:10.650 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.650 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.650 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:10.650 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965760+0000", 2024-09-21T21:42:10.650 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042609399999999999 2024-09-21T21:42:10.650 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.650 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.650 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:10.650 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965801+0000", 2024-09-21T21:42:10.650 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1399999999999997e-05 2024-09-21T21:42:10.650 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.650 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.650 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:10.650 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965851+0000", 2024-09-21T21:42:10.651 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9962e-05 2024-09-21T21:42:10.651 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:10.651 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:10.651 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:10.651 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.651 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.651 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6995 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5398528~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:10.651 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923153+0000", 2024-09-21T21:42:10.651 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 40.681625488999998, 2024-09-21T21:42:10.651 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042828034000000001, 2024-09-21T21:42:10.651 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:10.651 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:10.651 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:10.651 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:10.651 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:10.652 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:10.652 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6995 2024-09-21T21:42:10.652 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.652 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:10.652 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.652 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:10.652 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923153+0000", 2024-09-21T21:42:10.652 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:10.652 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.652 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.652 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:10.652 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923153+0000", 2024-09-21T21:42:10.652 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:10.652 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.652 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.653 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:10.653 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923154+0000", 2024-09-21T21:42:10.653 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2600000000000001e-07 2024-09-21T21:42:10.653 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.653 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.653 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:10.653 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923158+0000", 2024-09-21T21:42:10.653 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0559999999999998e-06 2024-09-21T21:42:10.653 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.653 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.653 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:10.653 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923161+0000", 2024-09-21T21:42:10.653 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6519999999999999e-06 2024-09-21T21:42:10.653 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.653 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.654 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:10.654 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923170+0000", 2024-09-21T21:42:10.654 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5720000000000003e-06 2024-09-21T21:42:10.654 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.654 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.654 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:10.654 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965887+0000", 2024-09-21T21:42:10.654 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042716863000000001 2024-09-21T21:42:10.654 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.654 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.654 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:10.654 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965927+0000", 2024-09-21T21:42:10.654 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0346999999999998e-05 2024-09-21T21:42:10.654 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.654 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.654 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:10.655 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965981+0000", 2024-09-21T21:42:10.655 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4317999999999997e-05 2024-09-21T21:42:10.655 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:10.655 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:10.655 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:10.655 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.655 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.655 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6997 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5412864~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:10.655 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923174+0000", 2024-09-21T21:42:10.655 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 40.681605333999997, 2024-09-21T21:42:10.655 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042928296999999997, 2024-09-21T21:42:10.655 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:10.655 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:10.655 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:10.656 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:10.656 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:10.656 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:10.656 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6997 2024-09-21T21:42:10.656 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.656 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:10.656 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.656 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:10.656 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923174+0000", 2024-09-21T21:42:10.656 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:10.656 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.656 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.656 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:10.656 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923174+0000", 2024-09-21T21:42:10.656 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:10.656 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.657 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.657 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:10.657 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923174+0000", 2024-09-21T21:42:10.657 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.67e-07 2024-09-21T21:42:10.657 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.657 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.657 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:10.657 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923178+0000", 2024-09-21T21:42:10.657 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1330000000000001e-06 2024-09-21T21:42:10.657 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.657 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.657 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:10.657 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923182+0000", 2024-09-21T21:42:10.657 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.794e-06 2024-09-21T21:42:10.657 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.658 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.658 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:10.658 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923184+0000", 2024-09-21T21:42:10.658 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.435e-06 2024-09-21T21:42:10.658 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.658 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.658 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:10.658 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966009+0000", 2024-09-21T21:42:10.658 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042824327000000002 2024-09-21T21:42:10.658 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.658 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.658 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:10.658 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966052+0000", 2024-09-21T21:42:10.658 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3034000000000002e-05 2024-09-21T21:42:10.659 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.659 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.659 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:10.659 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966102+0000", 2024-09-21T21:42:10.659 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0306999999999998e-05 2024-09-21T21:42:10.659 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:10.659 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:10.659 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:10.659 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.659 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.659 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6999 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5427200~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:10.659 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923187+0000", 2024-09-21T21:42:10.659 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 40.681591605999998, 2024-09-21T21:42:10.659 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043104267000000002, 2024-09-21T21:42:10.659 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:10.660 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:10.660 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:10.660 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:10.660 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:10.660 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:10.660 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6999 2024-09-21T21:42:10.660 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.660 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:10.660 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.660 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:10.660 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923187+0000", 2024-09-21T21:42:10.660 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:10.660 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.660 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.661 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:10.661 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923187+0000", 2024-09-21T21:42:10.661 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:10.661 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.661 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.661 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:10.661 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923188+0000", 2024-09-21T21:42:10.661 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3999999999999998e-07 2024-09-21T21:42:10.661 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.661 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.661 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:10.661 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923189+0000", 2024-09-21T21:42:10.661 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4169999999999999e-06 2024-09-21T21:42:10.661 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.661 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.662 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:10.662 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923194+0000", 2024-09-21T21:42:10.662 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1819999999999997e-06 2024-09-21T21:42:10.662 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.662 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.662 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:10.662 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923198+0000", 2024-09-21T21:42:10.662 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.143e-06 2024-09-21T21:42:10.662 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.662 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.662 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:10.662 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966135+0000", 2024-09-21T21:42:10.662 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042936297999999998 2024-09-21T21:42:10.662 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.663 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.663 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:10.663 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966204+0000", 2024-09-21T21:42:10.663 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8989e-05 2024-09-21T21:42:10.663 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.663 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.663 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:10.663 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966292+0000", 2024-09-21T21:42:10.663 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7997999999999993e-05 2024-09-21T21:42:10.663 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:10.663 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:10.663 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:10.663 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.663 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.664 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7001 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5441536~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:10.664 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923211+0000", 2024-09-21T21:42:10.664 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 40.681567815000001, 2024-09-21T21:42:10.664 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043196518000000003, 2024-09-21T21:42:10.664 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:10.664 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:10.664 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:10.664 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:10.664 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:10.664 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:10.664 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7001 2024-09-21T21:42:10.664 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.664 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:10.664 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.665 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:10.665 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923211+0000", 2024-09-21T21:42:10.665 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:10.665 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.665 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.665 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:10.665 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923211+0000", 2024-09-21T21:42:10.665 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:10.665 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.665 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.665 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:10.665 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923212+0000", 2024-09-21T21:42:10.665 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3900000000000005e-07 2024-09-21T21:42:10.665 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.666 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.666 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:10.666 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923214+0000", 2024-09-21T21:42:10.666 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9970000000000001e-06 2024-09-21T21:42:10.666 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.666 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.666 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:10.666 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923219+0000", 2024-09-21T21:42:10.666 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4310000000000002e-06 2024-09-21T21:42:10.666 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.666 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.666 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:10.666 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923224+0000", 2024-09-21T21:42:10.666 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6410000000000001e-06 2024-09-21T21:42:10.667 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.667 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.667 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:10.667 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966318+0000", 2024-09-21T21:42:10.667 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043093842 2024-09-21T21:42:10.667 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.667 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.667 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:10.667 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966359+0000", 2024-09-21T21:42:10.667 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1260999999999997e-05 2024-09-21T21:42:10.667 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.667 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.667 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:10.667 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966408+0000", 2024-09-21T21:42:10.667 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8807000000000002e-05 2024-09-21T21:42:10.668 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:10.668 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:10.668 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:10.668 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.668 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.668 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7002 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5455872~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:10.668 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923229+0000", 2024-09-21T21:42:10.668 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 40.681549404999998, 2024-09-21T21:42:10.668 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043364568999999999, 2024-09-21T21:42:10.668 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:10.668 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:10.668 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:10.668 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:10.668 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:10.668 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:10.669 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7002 2024-09-21T21:42:10.669 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.669 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:10.669 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.669 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:10.669 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923229+0000", 2024-09-21T21:42:10.669 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:10.669 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.669 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.669 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:10.669 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923229+0000", 2024-09-21T21:42:10.669 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:10.669 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.670 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.670 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:10.670 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923230+0000", 2024-09-21T21:42:10.670 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4700000000000002e-07 2024-09-21T21:42:10.670 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.670 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.670 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:10.670 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923232+0000", 2024-09-21T21:42:10.670 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0719999999999998e-06 2024-09-21T21:42:10.670 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.670 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.670 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:10.670 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923237+0000", 2024-09-21T21:42:10.670 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2290000000000002e-06 2024-09-21T21:42:10.671 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.671 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.671 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:10.671 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923245+0000", 2024-09-21T21:42:10.671 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2959999999999999e-06 2024-09-21T21:42:10.671 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.671 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.671 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:10.671 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966438+0000", 2024-09-21T21:42:10.671 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043193027000000002 2024-09-21T21:42:10.671 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.671 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.671 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:10.671 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966506+0000", 2024-09-21T21:42:10.672 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7707999999999996e-05 2024-09-21T21:42:10.672 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.672 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.672 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:10.672 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966594+0000", 2024-09-21T21:42:10.672 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.789e-05 2024-09-21T21:42:10.672 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:10.672 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:10.672 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:10.672 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.672 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.672 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7005 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5470208~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:10.672 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923254+0000", 2024-09-21T21:42:10.672 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 40.681525174000001, 2024-09-21T21:42:10.672 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043457176, 2024-09-21T21:42:10.675 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:10.675 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:10.675 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:10.675 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:10.675 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:10.675 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:10.675 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7005 2024-09-21T21:42:10.675 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.675 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:10.675 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.675 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:10.675 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923254+0000", 2024-09-21T21:42:10.675 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:10.675 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.676 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.676 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:10.676 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923254+0000", 2024-09-21T21:42:10.676 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:10.676 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.676 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.676 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:10.676 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923254+0000", 2024-09-21T21:42:10.676 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4700000000000002e-07 2024-09-21T21:42:10.676 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.676 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.676 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:10.676 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923256+0000", 2024-09-21T21:42:10.676 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.007e-06 2024-09-21T21:42:10.676 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.677 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.677 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:10.677 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923266+0000", 2024-09-21T21:42:10.677 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.4700000000000008e-06 2024-09-21T21:42:10.677 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.677 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.677 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:10.677 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923271+0000", 2024-09-21T21:42:10.677 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2809999999999999e-06 2024-09-21T21:42:10.677 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.677 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.677 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:10.677 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966619+0000", 2024-09-21T21:42:10.678 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043347951000000003 2024-09-21T21:42:10.678 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.678 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.678 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:10.678 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966661+0000", 2024-09-21T21:42:10.678 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2614999999999998e-05 2024-09-21T21:42:10.678 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.678 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.678 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:10.678 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966711+0000", 2024-09-21T21:42:10.678 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9505000000000001e-05 2024-09-21T21:42:10.678 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:10.678 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:10.678 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:10.679 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.679 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.679 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7008 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5484544~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:10.679 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923278+0000", 2024-09-21T21:42:10.679 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 40.681500526999997, 2024-09-21T21:42:10.679 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043631071, 2024-09-21T21:42:10.679 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:10.679 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:10.679 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:10.679 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:10.679 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:10.679 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:10.679 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7008 2024-09-21T21:42:10.679 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.680 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:10.680 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.680 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:10.680 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923278+0000", 2024-09-21T21:42:10.680 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:10.680 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.680 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.680 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:10.680 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923278+0000", 2024-09-21T21:42:10.680 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:10.680 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.680 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.680 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:10.680 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923279+0000", 2024-09-21T21:42:10.680 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.77e-07 2024-09-21T21:42:10.681 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.681 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.681 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:10.681 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923281+0000", 2024-09-21T21:42:10.681 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0219999999999999e-06 2024-09-21T21:42:10.681 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.681 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.681 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:10.681 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923287+0000", 2024-09-21T21:42:10.681 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7799999999999997e-06 2024-09-21T21:42:10.681 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.681 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.681 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:10.681 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923291+0000", 2024-09-21T21:42:10.682 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1699999999999999e-06 2024-09-21T21:42:10.682 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.682 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.682 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:10.682 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966742+0000", 2024-09-21T21:42:10.682 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043451344000000003 2024-09-21T21:42:10.682 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.682 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.682 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:10.682 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966809+0000", 2024-09-21T21:42:10.682 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7070999999999995e-05 2024-09-21T21:42:10.682 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.682 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.682 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:10.683 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966909+0000", 2024-09-21T21:42:10.683 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000100307 2024-09-21T21:42:10.683 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:10.683 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:10.683 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:10.683 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.683 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.683 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7010 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5498880~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:10.683 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923301+0000", 2024-09-21T21:42:10.683 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 40.681477631999996, 2024-09-21T21:42:10.683 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043717830999999999, 2024-09-21T21:42:10.683 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:10.683 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:10.683 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:10.684 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:10.684 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:10.684 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:10.684 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7010 2024-09-21T21:42:10.684 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.684 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:10.684 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.684 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:10.684 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923301+0000", 2024-09-21T21:42:10.684 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:10.684 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.684 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.684 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:10.684 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923301+0000", 2024-09-21T21:42:10.684 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:10.685 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.685 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.685 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:10.685 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923302+0000", 2024-09-21T21:42:10.685 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.41e-07 2024-09-21T21:42:10.685 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.685 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.685 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:10.685 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923304+0000", 2024-09-21T21:42:10.685 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0990000000000001e-06 2024-09-21T21:42:10.685 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.685 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.685 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:10.685 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923310+0000", 2024-09-21T21:42:10.686 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3860000000000004e-06 2024-09-21T21:42:10.686 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.686 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.686 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:10.686 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923315+0000", 2024-09-21T21:42:10.686 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8459999999999999e-06 2024-09-21T21:42:10.686 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.686 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.686 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:10.686 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966938+0000", 2024-09-21T21:42:10.686 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043622693999999997 2024-09-21T21:42:10.686 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.686 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.686 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:10.686 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966975+0000", 2024-09-21T21:42:10.687 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7608999999999999e-05 2024-09-21T21:42:10.687 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.687 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.687 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:10.687 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.967019+0000", 2024-09-21T21:42:10.687 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3856000000000002e-05 2024-09-21T21:42:10.687 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:10.687 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:10.687 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:10.687 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.687 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.687 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7354 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 2107392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:10.687 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.936932+0000", 2024-09-21T21:42:10.688 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 40.667846777000001, 2024-09-21T21:42:10.688 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041607708, 2024-09-21T21:42:10.688 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:10.688 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:10.688 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:10.688 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:10.688 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:10.688 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:10.688 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7354 2024-09-21T21:42:10.688 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.688 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:10.688 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.688 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:10.688 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.936932+0000", 2024-09-21T21:42:10.688 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:10.689 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.689 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.689 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:10.689 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.936932+0000", 2024-09-21T21:42:10.689 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:10.689 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.689 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.689 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:10.689 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.937276+0000", 2024-09-21T21:42:10.689 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00034344200000000001 2024-09-21T21:42:10.689 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.689 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.689 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:10.689 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.937279+0000", 2024-09-21T21:42:10.690 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.844e-06 2024-09-21T21:42:10.690 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.690 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.690 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:10.690 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.937288+0000", 2024-09-21T21:42:10.690 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2409999999999995e-06 2024-09-21T21:42:10.690 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.690 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.690 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:10.690 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.937293+0000", 2024-09-21T21:42:10.690 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6559999999999997e-06 2024-09-21T21:42:10.690 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.690 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.691 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:10.691 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.978448+0000", 2024-09-21T21:42:10.691 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041154621000000002 2024-09-21T21:42:10.691 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.691 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.691 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:10.691 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.978487+0000", 2024-09-21T21:42:10.691 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9254000000000001e-05 2024-09-21T21:42:10.691 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.691 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.691 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:10.691 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.978540+0000", 2024-09-21T21:42:10.691 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2649999999999999e-05 2024-09-21T21:42:10.691 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:10.692 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:10.692 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:10.692 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.692 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.692 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5215.0:1054 3.9 3:946d2401:::rbd_data.1121524d2522.000000000000000b:head [set-alloc-hint object_size 16777216 write_size 16777216,write 6615040~59392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-21T21:42:10.692 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:38.148979+0000", 2024-09-21T21:42:10.692 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 32.455799810000002, 2024-09-21T21:42:10.692 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047870388999999999, 2024-09-21T21:42:10.692 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:10.692 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:10.692 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:10.692 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:10.692 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5215", 2024-09-21T21:42:10.692 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1580282168", 2024-09-21T21:42:10.693 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1054 2024-09-21T21:42:10.693 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.693 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:10.693 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.693 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:10.693 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.148979+0000", 2024-09-21T21:42:10.693 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:10.693 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.693 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.693 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:10.693 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.148979+0000", 2024-09-21T21:42:10.693 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:10.693 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.694 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.694 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:10.694 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.148980+0000", 2024-09-21T21:42:10.694 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5799999999999999e-07 2024-09-21T21:42:10.694 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.694 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.694 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:10.694 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.149039+0000", 2024-09-21T21:42:10.694 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9151000000000003e-05 2024-09-21T21:42:10.694 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.694 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.694 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:10.694 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.149047+0000", 2024-09-21T21:42:10.694 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1000000000000004e-06 2024-09-21T21:42:10.695 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.695 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.695 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:10.695 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.149052+0000", 2024-09-21T21:42:10.695 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5959999999999997e-06 2024-09-21T21:42:10.695 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.695 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.695 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:10.695 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.195288+0000", 2024-09-21T21:42:10.695 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.046236248000000001 2024-09-21T21:42:10.695 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.695 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.695 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:10.695 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.195334+0000", 2024-09-21T21:42:10.696 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6575e-05 2024-09-21T21:42:10.696 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.696 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.696 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 2", 2024-09-21T21:42:10.696 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.195406+0000", 2024-09-21T21:42:10.696 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1715e-05 2024-09-21T21:42:10.696 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.696 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.696 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:10.696 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.196018+0000", 2024-09-21T21:42:10.696 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00061218999999999998 2024-09-21T21:42:10.696 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.696 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.696 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:10.697 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.196794+0000", 2024-09-21T21:42:10.697 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000775814 2024-09-21T21:42:10.697 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.697 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.697 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:10.697 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.196827+0000", 2024-09-21T21:42:10.697 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2419000000000003e-05 2024-09-21T21:42:10.697 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:10.697 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:10.697 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:10.697 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.196849+0000", 2024-09-21T21:42:10.697 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2923000000000001e-05 2024-09-21T21:42:10.697 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:10.697 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:10.698 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:10.698 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:10.698 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:10.698 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:10.699 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T21:42:10.700 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:10.697+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:10.800 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:10.798+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:10.849 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:10.849 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:42:10.849 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-21T21:42:10.849 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:10.858 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T21:42:10.864 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T21:42:10.880 INFO:teuthology.orchestra.run.smithi164.stdout:checking clone #35, image image_client.0-clone35 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent36 2024-09-21T21:42:10.901 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:10.899+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:10.936 INFO:teuthology.orchestra.run.smithi164.stdout:4753 punch from 0x1683800 to 0x168e600, (0xae00 bytes) 2024-09-21T21:42:10.977 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:10.977 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:42:10.977 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:42:10.978 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:42:10.978 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:10.991 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T21:42:10.992 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0: {} 2024-09-21T21:42:10.992 INFO:teuthology.orchestra.run.smithi081.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T21:42:11.001 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:11.000+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:11.009 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1: {} 2024-09-21T21:42:11.010 INFO:teuthology.orchestra.run.smithi081.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T21:42:11.028 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2: {} 2024-09-21T21:42:11.028 INFO:teuthology.orchestra.run.smithi081.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T21:42:11.036 INFO:teuthology.orchestra.run.smithi164.stdout:4755 write 0x508de00 thru 0x509cca6 (0xeea7 bytes) 2024-09-21T21:42:11.046 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3: {} 2024-09-21T21:42:11.046 INFO:teuthology.orchestra.run.smithi081.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T21:42:11.046 INFO:teuthology.orchestra.run.smithi164.stdout:4757 trunc from 0x555c800 to 0xdfc5e00 2024-09-21T21:42:11.065 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4: {} 2024-09-21T21:42:11.065 INFO:teuthology.orchestra.run.smithi081.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T21:42:11.083 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5: {} 2024-09-21T21:42:11.083 INFO:teuthology.orchestra.run.smithi081.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T21:42:11.091 INFO:teuthology.orchestra.run.smithi164.stdout:4759 punch from 0x2d5de00 to 0x2d63c00, (0x5e00 bytes) 2024-09-21T21:42:11.097 INFO:teuthology.orchestra.run.smithi164.stdout:4763 read 0xc7a6600 thru 0xc7ae4c3 (0x7ec4 bytes) 2024-09-21T21:42:11.097 INFO:teuthology.orchestra.run.smithi164.stdout:4765 write 0xbd0c200 thru 0xbd0ddd2 (0x1bd3 bytes) 2024-09-21T21:42:11.097 INFO:teuthology.orchestra.run.smithi164.stdout:4766 read 0x6e16c00 thru 0x6e1bf06 (0x5307 bytes) 2024-09-21T21:42:11.097 INFO:teuthology.orchestra.run.smithi164.stdout:4768 write 0x2c10000 thru 0x2c1d622 (0xd623 bytes) 2024-09-21T21:42:11.102 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:11.100+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:11.128 INFO:teuthology.orchestra.run.smithi164.stdout:4769 write 0x78cb200 thru 0x78cf424 (0x4225 bytes) 2024-09-21T21:42:11.128 INFO:teuthology.orchestra.run.smithi164.stdout:4771 read 0x8ea3800 thru 0x8ea3c3c (0x43d bytes) 2024-09-21T21:42:11.128 INFO:teuthology.orchestra.run.smithi164.stdout:4772 write 0xa188000 thru 0xa192c9e (0xac9f bytes) 2024-09-21T21:42:11.128 INFO:teuthology.orchestra.run.smithi164.stdout:4773 read 0x3dd1200 thru 0x3dd4381 (0x3182 bytes) 2024-09-21T21:42:11.128 INFO:teuthology.orchestra.run.smithi164.stdout:4774 punch from 0xa5fca00 to 0xa60b400, (0xea00 bytes) 2024-09-21T21:42:11.130 INFO:teuthology.orchestra.run.smithi164.stdout:4775 write 0x10a9000 thru 0x10ace89 (0x3e8a bytes) 2024-09-21T21:42:11.132 INFO:teuthology.orchestra.run.smithi164.stdout:4776 trunc from 0xdfc5e00 to 0xecf2a00 2024-09-21T21:42:11.170 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:11.170 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:11.184 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T21:42:11.200 INFO:teuthology.orchestra.run.smithi164.stdout:4777 write 0x5dd400 thru 0x5e2cb6 (0x58b7 bytes) 2024-09-21T21:42:11.201 INFO:teuthology.orchestra.run.smithi164.stdout:4778 write 0x4abd400 thru 0x4aca7a7 (0xd3a8 bytes) 2024-09-21T21:42:11.201 INFO:teuthology.orchestra.run.smithi164.stdout:4780 read 0x2e47400 thru 0x2e48937 (0x1538 bytes) 2024-09-21T21:42:11.205 INFO:teuthology.orchestra.run.smithi164.stdout:4783 write 0xe441400 thru 0xe44e324 (0xcf25 bytes) 2024-09-21T21:42:11.206 INFO:teuthology.orchestra.run.smithi164.stdout:4785 trunc from 0xecf2a00 to 0xdc3ec00 2024-09-21T21:42:11.226 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:11.224+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:11.296 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:11.296 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:11.304 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:11.302+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:11.306 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T21:42:11.347 INFO:teuthology.orchestra.run.smithi164.stdout:4790 read 0xadc1c00 thru 0xadd0b2f (0xef30 bytes) 2024-09-21T21:42:11.347 INFO:teuthology.orchestra.run.smithi164.stdout:4795 read 0xab5400 thru 0xac4827 (0xf428 bytes) 2024-09-21T21:42:11.361 INFO:teuthology.orchestra.run.smithi164.stdout:4798 trunc from 0xdc3ec00 to 0x5ba2e00 2024-09-21T21:42:11.390 INFO:teuthology.orchestra.run.smithi164.stdout:4800 punch from 0x1a76a00 to 0x1a84c00, (0xe200 bytes) 2024-09-21T21:42:11.393 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T21:42:11.396 INFO:teuthology.orchestra.run.smithi081.stderr:marked out osd.4. 2024-09-21T21:42:11.412 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:11.411+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:11.458 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:11.459 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:11.468 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T21:42:11.506 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:11.503+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:11.574 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:11.574 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:11.583 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T21:42:11.626 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:11.625+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:11.687 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:11.687 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:11.697 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T21:42:11.707 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:11.705+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:11.808 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:11.806+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:11.839 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:11.839 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:11.848 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T21:42:11.910 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:11.908+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:11.951 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:11.951 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:11.960 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T21:42:12.011 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:12.008+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:12.063 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:12.063 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:12.072 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T21:42:12.111 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:12.109+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:12.175 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:12.175 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:12.184 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T21:42:12.208 INFO:teuthology.orchestra.run.smithi081.stderr:noscrub is set 2024-09-21T21:42:12.212 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:12.209+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:12.291 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:12.292 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:12.300 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T21:42:12.312 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:12.311+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:12.413 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:12.411+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:12.414 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:12.414 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:12.423 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T21:42:12.514 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:12.513+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:12.567 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:12.567 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:12.575 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T21:42:12.615 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:12.614+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:12.678 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:12.678 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:12.686 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T21:42:12.716 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:12.714+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:12.789 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:12.790 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:12.798 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T21:42:12.817 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:12.815+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:12.900 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:12.900 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:12.909 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T21:42:12.952 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:12.950+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:13.011 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:13.012 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:13.019 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:13.017+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:13.020 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T21:42:13.095 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T21:42:13.139 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:13.138+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:13.165 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:13.166 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:13.174 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T21:42:13.224 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:13.220+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:13.230 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0: {} 2024-09-21T21:42:13.230 INFO:teuthology.orchestra.run.smithi081.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T21:42:13.239 INFO:teuthology.orchestra.run.smithi164.stdout:4801 read 0x870e00 thru 0x878e5e (0x805f bytes) 2024-09-21T21:42:13.247 INFO:teuthology.orchestra.run.smithi164.stdout:4803 punch from 0x1e83800 to 0x1e8ce00, (0x9600 bytes) 2024-09-21T21:42:13.257 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1: {} 2024-09-21T21:42:13.258 INFO:teuthology.orchestra.run.smithi081.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T21:42:13.280 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2: {} 2024-09-21T21:42:13.280 INFO:teuthology.orchestra.run.smithi081.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T21:42:13.297 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:13.297 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:13.305 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3: {} 2024-09-21T21:42:13.305 INFO:teuthology.orchestra.run.smithi081.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T21:42:13.308 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T21:42:13.330 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4: {} 2024-09-21T21:42:13.330 INFO:teuthology.orchestra.run.smithi081.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T21:42:13.339 INFO:teuthology.orchestra.run.smithi164.stdout:4804 write 0xc5a6600 thru 0xc5b44da (0xdedb bytes) 2024-09-21T21:42:13.351 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5: {} 2024-09-21T21:42:13.351 INFO:teuthology.orchestra.run.smithi081.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T21:42:13.352 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:13.350+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:13.358 INFO:teuthology.orchestra.run.smithi164.stdout:4805 trunc from 0xc5b4600 to 0x1412c00 2024-09-21T21:42:13.422 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:13.421+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:13.426 INFO:teuthology.orchestra.run.smithi164.stdout:4807 read 0xcb8400 thru 0xcc5e4f (0xda50 bytes) 2024-09-21T21:42:13.442 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:13.442 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-21T21:42:13.442 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-21T21:42:13.442 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:42:13.442 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.442 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1486 3.16 3:6a41a830:::rbd_data.112145fae779.000000000000000a:a [read 360448~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:42:13.442 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634126+0000", 2024-09-21T21:42:13.442 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 71.805462695000003, 2024-09-21T21:42:13.442 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059399528, 2024-09-21T21:42:13.442 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:13.443 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:13.443 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:13.443 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:13.443 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:42:13.443 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:42:13.443 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1486 2024-09-21T21:42:13.443 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.443 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:13.443 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.443 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:13.443 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634126+0000", 2024-09-21T21:42:13.443 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:13.444 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.444 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.444 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:13.444 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634126+0000", 2024-09-21T21:42:13.444 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:13.444 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.444 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.444 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:13.444 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634126+0000", 2024-09-21T21:42:13.444 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4099999999999999e-07 2024-09-21T21:42:13.444 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.444 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.444 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:13.445 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634128+0000", 2024-09-21T21:42:13.445 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1069999999999999e-06 2024-09-21T21:42:13.445 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.445 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.445 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:13.445 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634135+0000", 2024-09-21T21:42:13.445 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.212e-06 2024-09-21T21:42:13.445 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.445 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.445 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:13.445 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634140+0000", 2024-09-21T21:42:13.445 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1429999999999997e-06 2024-09-21T21:42:13.445 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.446 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.446 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:13.446 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693363+0000", 2024-09-21T21:42:13.446 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059223614000000001 2024-09-21T21:42:13.446 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.446 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.446 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:13.446 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693431+0000", 2024-09-21T21:42:13.446 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7294999999999999e-05 2024-09-21T21:42:13.446 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.446 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.446 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:13.446 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693525+0000", 2024-09-21T21:42:13.447 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.4716000000000006e-05 2024-09-21T21:42:13.447 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:13.447 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:13.447 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:13.447 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.447 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.447 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1499 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 49152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:42:13.447 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634186+0000", 2024-09-21T21:42:13.447 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 71.805402818000005, 2024-09-21T21:42:13.447 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059339256999999999, 2024-09-21T21:42:13.447 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:13.447 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:13.447 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:13.448 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:13.448 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:42:13.448 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:42:13.448 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1499 2024-09-21T21:42:13.448 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.448 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:13.448 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.448 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:13.448 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634186+0000", 2024-09-21T21:42:13.448 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:13.448 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.448 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.448 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:13.449 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634186+0000", 2024-09-21T21:42:13.449 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:13.449 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.449 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.449 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:13.449 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634186+0000", 2024-09-21T21:42:13.449 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0999999999999999e-07 2024-09-21T21:42:13.449 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.449 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.449 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:13.449 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634188+0000", 2024-09-21T21:42:13.449 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.86e-06 2024-09-21T21:42:13.449 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.449 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.450 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:13.450 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634194+0000", 2024-09-21T21:42:13.450 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1109999999999996e-06 2024-09-21T21:42:13.450 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.450 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.450 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:13.450 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634199+0000", 2024-09-21T21:42:13.450 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7419999999999997e-06 2024-09-21T21:42:13.450 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.450 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.450 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:13.450 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693357+0000", 2024-09-21T21:42:13.450 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059157847 2024-09-21T21:42:13.450 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.451 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.451 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:13.451 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693430+0000", 2024-09-21T21:42:13.451 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3203999999999999e-05 2024-09-21T21:42:13.451 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.451 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.451 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:13.451 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693525+0000", 2024-09-21T21:42:13.451 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5082999999999997e-05 2024-09-21T21:42:13.451 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:13.451 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:13.451 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:13.452 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.452 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.452 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1501 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 106496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:42:13.452 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634205+0000", 2024-09-21T21:42:13.452 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 71.805383727999995, 2024-09-21T21:42:13.452 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059482394000000001, 2024-09-21T21:42:13.452 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:13.452 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:13.452 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:13.452 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:13.452 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:42:13.452 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:42:13.452 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1501 2024-09-21T21:42:13.452 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.453 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:13.453 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.453 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:13.453 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634205+0000", 2024-09-21T21:42:13.453 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:13.453 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.453 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.453 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:13.453 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634205+0000", 2024-09-21T21:42:13.453 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:13.453 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.453 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.453 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:13.453 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634205+0000", 2024-09-21T21:42:13.454 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.03e-07 2024-09-21T21:42:13.454 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.454 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.454 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:13.454 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634207+0000", 2024-09-21T21:42:13.454 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0389999999999999e-06 2024-09-21T21:42:13.454 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.454 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.454 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:13.454 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634213+0000", 2024-09-21T21:42:13.454 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2600000000000002e-06 2024-09-21T21:42:13.454 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.454 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.454 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:13.455 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634218+0000", 2024-09-21T21:42:13.455 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8570000000000003e-06 2024-09-21T21:42:13.455 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.455 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.455 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:13.455 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693552+0000", 2024-09-21T21:42:13.455 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059333636000000002 2024-09-21T21:42:13.455 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.455 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.455 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:13.455 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693613+0000", 2024-09-21T21:42:13.455 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0560000000000003e-05 2024-09-21T21:42:13.455 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.455 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.456 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:13.456 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693687+0000", 2024-09-21T21:42:13.456 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4639000000000001e-05 2024-09-21T21:42:13.456 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:13.456 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:13.456 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:13.456 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.456 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.456 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1515 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 434176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:42:13.456 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634318+0000", 2024-09-21T21:42:13.456 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 71.805270041, 2024-09-21T21:42:13.456 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059565812000000003, 2024-09-21T21:42:13.456 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:13.456 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:13.457 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:13.457 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:13.457 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:42:13.457 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:42:13.457 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1515 2024-09-21T21:42:13.457 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.457 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:13.457 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.457 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:13.457 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634318+0000", 2024-09-21T21:42:13.457 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:13.457 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.457 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.457 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:13.458 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634318+0000", 2024-09-21T21:42:13.458 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:13.458 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.458 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.458 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:13.458 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634319+0000", 2024-09-21T21:42:13.458 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9400000000000001e-07 2024-09-21T21:42:13.458 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.458 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.458 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:13.458 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634321+0000", 2024-09-21T21:42:13.458 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.154e-06 2024-09-21T21:42:13.458 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.458 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.458 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:13.459 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634327+0000", 2024-09-21T21:42:13.459 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9830000000000001e-06 2024-09-21T21:42:13.459 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.459 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.459 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:13.459 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634332+0000", 2024-09-21T21:42:13.459 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9080000000000003e-06 2024-09-21T21:42:13.459 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.459 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.459 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:13.459 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693717+0000", 2024-09-21T21:42:13.459 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059385447000000001 2024-09-21T21:42:13.459 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.460 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.460 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:13.460 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693795+0000", 2024-09-21T21:42:13.460 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7267000000000004e-05 2024-09-21T21:42:13.460 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.460 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.460 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:13.460 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693884+0000", 2024-09-21T21:42:13.460 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9659000000000003e-05 2024-09-21T21:42:13.460 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:13.460 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:13.460 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:13.460 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.460 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.461 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1517 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 491520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:42:13.461 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634338+0000", 2024-09-21T21:42:13.461 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 71.805250267999995, 2024-09-21T21:42:13.461 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060100745999999997, 2024-09-21T21:42:13.461 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:13.461 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:13.461 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:13.461 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:13.461 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:42:13.461 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:42:13.461 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1517 2024-09-21T21:42:13.461 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.461 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:13.461 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.462 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:13.462 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634338+0000", 2024-09-21T21:42:13.462 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:13.462 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.462 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.462 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:13.462 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634338+0000", 2024-09-21T21:42:13.462 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:13.462 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.462 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.462 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:13.462 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634339+0000", 2024-09-21T21:42:13.462 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2899999999999999e-07 2024-09-21T21:42:13.462 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.463 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.463 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:13.463 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634341+0000", 2024-09-21T21:42:13.463 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.8959999999999999e-06 2024-09-21T21:42:13.463 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.463 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.463 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:13.463 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634347+0000", 2024-09-21T21:42:13.463 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.302e-06 2024-09-21T21:42:13.463 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.463 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.463 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:13.463 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634351+0000", 2024-09-21T21:42:13.464 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6070000000000001e-06 2024-09-21T21:42:13.464 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.464 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.464 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:13.464 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694279+0000", 2024-09-21T21:42:13.464 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059927046999999997 2024-09-21T21:42:13.464 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.464 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.464 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:13.464 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694355+0000", 2024-09-21T21:42:13.464 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6483999999999995e-05 2024-09-21T21:42:13.464 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.464 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.464 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:13.465 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694439+0000", 2024-09-21T21:42:13.465 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3980999999999994e-05 2024-09-21T21:42:13.465 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:13.465 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:13.465 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:13.465 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.465 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.465 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1518 3.16 3:6a41a830:::rbd_data.112145fae779.000000000000000a:a [read 294912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:42:13.465 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634357+0000", 2024-09-21T21:42:13.465 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 71.805231362000001, 2024-09-21T21:42:13.465 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059712318, 2024-09-21T21:42:13.465 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:13.465 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:13.466 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:13.466 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:13.466 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:42:13.466 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:42:13.466 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1518 2024-09-21T21:42:13.466 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.466 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:13.466 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.466 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:13.466 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634357+0000", 2024-09-21T21:42:13.466 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:13.466 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.466 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.466 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:13.467 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634357+0000", 2024-09-21T21:42:13.467 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:13.467 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.467 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.467 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:13.467 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634358+0000", 2024-09-21T21:42:13.467 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2100000000000002e-07 2024-09-21T21:42:13.467 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.467 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.467 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:13.467 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634360+0000", 2024-09-21T21:42:13.467 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0940000000000002e-06 2024-09-21T21:42:13.467 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.467 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.468 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:13.468 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634366+0000", 2024-09-21T21:42:13.468 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3389999999999998e-06 2024-09-21T21:42:13.468 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.468 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.468 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:13.468 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634371+0000", 2024-09-21T21:42:13.468 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6670000000000004e-06 2024-09-21T21:42:13.468 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.468 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.468 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:13.468 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693910+0000", 2024-09-21T21:42:13.468 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059538973000000002 2024-09-21T21:42:13.468 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.468 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.469 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:13.469 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693982+0000", 2024-09-21T21:42:13.469 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2806999999999996e-05 2024-09-21T21:42:13.469 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.469 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.469 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:13.469 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694069+0000", 2024-09-21T21:42:13.469 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7016999999999997e-05 2024-09-21T21:42:13.469 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:13.469 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:13.469 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:13.469 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.469 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.469 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1519 3.16 3:6a41a830:::rbd_data.112145fae779.000000000000000a:a [read 352256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:42:13.470 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634377+0000", 2024-09-21T21:42:13.470 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 71.805211659999998, 2024-09-21T21:42:13.470 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059872213000000001, 2024-09-21T21:42:13.470 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:13.470 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:13.470 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:13.470 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:13.470 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:42:13.470 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:42:13.470 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1519 2024-09-21T21:42:13.470 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.470 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:13.470 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.471 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:13.471 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634377+0000", 2024-09-21T21:42:13.471 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:13.471 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.471 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.471 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:13.471 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634377+0000", 2024-09-21T21:42:13.471 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:13.471 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.471 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.471 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:13.471 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634377+0000", 2024-09-21T21:42:13.471 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2600000000000002e-07 2024-09-21T21:42:13.471 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.472 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.472 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:13.472 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634379+0000", 2024-09-21T21:42:13.472 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0729999999999999e-06 2024-09-21T21:42:13.472 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.472 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.472 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:13.472 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634386+0000", 2024-09-21T21:42:13.472 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4690000000000002e-06 2024-09-21T21:42:13.472 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.472 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.472 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:13.472 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634390+0000", 2024-09-21T21:42:13.472 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4580000000000003e-06 2024-09-21T21:42:13.472 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.472 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.473 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:13.473 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694094+0000", 2024-09-21T21:42:13.473 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059703830999999999 2024-09-21T21:42:13.473 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.473 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.473 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:13.473 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694168+0000", 2024-09-21T21:42:13.473 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4222999999999997e-05 2024-09-21T21:42:13.473 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.473 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.473 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:13.473 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694249+0000", 2024-09-21T21:42:13.473 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0632999999999999e-05 2024-09-21T21:42:13.473 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:13.474 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:13.474 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:13.474 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.474 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.474 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1529 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 53248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:42:13.474 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634397+0000", 2024-09-21T21:42:13.474 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 71.805191508999997, 2024-09-21T21:42:13.474 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060189919000000001, 2024-09-21T21:42:13.474 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:13.474 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:13.474 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:13.474 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:13.474 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:42:13.474 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:42:13.475 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1529 2024-09-21T21:42:13.475 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.475 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:13.475 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.475 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:13.475 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634397+0000", 2024-09-21T21:42:13.475 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:13.475 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.475 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.475 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:13.475 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634397+0000", 2024-09-21T21:42:13.475 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:13.475 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.476 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.476 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:13.476 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634397+0000", 2024-09-21T21:42:13.476 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2e-07 2024-09-21T21:42:13.476 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.476 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.476 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:13.476 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634399+0000", 2024-09-21T21:42:13.476 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0609999999999998e-06 2024-09-21T21:42:13.476 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.476 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.476 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:13.476 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634406+0000", 2024-09-21T21:42:13.476 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7259999999999996e-06 2024-09-21T21:42:13.477 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.477 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.477 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:13.477 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634411+0000", 2024-09-21T21:42:13.477 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9339999999999997e-06 2024-09-21T21:42:13.477 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.477 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.477 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:13.477 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694480+0000", 2024-09-21T21:42:13.477 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060069105999999997 2024-09-21T21:42:13.477 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.477 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.477 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:13.477 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694527+0000", 2024-09-21T21:42:13.477 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7120000000000003e-05 2024-09-21T21:42:13.477 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.478 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.478 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:13.478 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694587+0000", 2024-09-21T21:42:13.478 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9552000000000002e-05 2024-09-21T21:42:13.478 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:13.478 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:13.478 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:13.478 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.478 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.478 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1531 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 110592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:42:13.478 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634417+0000", 2024-09-21T21:42:13.478 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 71.805171575000003, 2024-09-21T21:42:13.478 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060347287999999999, 2024-09-21T21:42:13.478 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:13.478 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:13.479 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:13.479 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:13.479 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:42:13.479 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:42:13.479 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1531 2024-09-21T21:42:13.479 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.479 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:13.479 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.479 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:13.479 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634417+0000", 2024-09-21T21:42:13.479 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:13.479 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.479 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.479 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:13.479 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634417+0000", 2024-09-21T21:42:13.480 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:13.480 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.480 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.480 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:13.480 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634417+0000", 2024-09-21T21:42:13.480 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1699999999999999e-07 2024-09-21T21:42:13.480 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.480 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.480 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:13.480 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634420+0000", 2024-09-21T21:42:13.480 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.243e-06 2024-09-21T21:42:13.480 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.480 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.480 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:13.480 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634426+0000", 2024-09-21T21:42:13.480 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2550000000000003e-06 2024-09-21T21:42:13.481 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.481 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.481 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:13.481 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634431+0000", 2024-09-21T21:42:13.481 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0150000000000003e-06 2024-09-21T21:42:13.481 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.481 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.481 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:13.481 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694618+0000", 2024-09-21T21:42:13.481 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060186836000000001 2024-09-21T21:42:13.481 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.481 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.481 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:13.481 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694685+0000", 2024-09-21T21:42:13.481 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6970000000000004e-05 2024-09-21T21:42:13.482 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.482 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.482 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:13.482 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694764+0000", 2024-09-21T21:42:13.482 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9552e-05 2024-09-21T21:42:13.482 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:13.482 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:13.482 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:13.482 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.482 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.482 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1532 3.e 3:724b31a3:::rbd_data.1121e540c096.0000000000000015:b [read 151552~36864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:42:13.482 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634437+0000", 2024-09-21T21:42:13.482 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 71.805151778999999, 2024-09-21T21:42:13.482 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060474314000000001, 2024-09-21T21:42:13.482 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:13.482 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:13.483 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:13.483 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:13.483 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:42:13.483 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:42:13.483 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1532 2024-09-21T21:42:13.483 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.483 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:13.483 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.483 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:13.483 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634437+0000", 2024-09-21T21:42:13.483 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:13.483 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.483 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.483 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:13.484 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634437+0000", 2024-09-21T21:42:13.484 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:13.484 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.484 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.484 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:13.484 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634437+0000", 2024-09-21T21:42:13.484 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9400000000000001e-07 2024-09-21T21:42:13.484 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.484 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.484 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:13.484 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634439+0000", 2024-09-21T21:42:13.484 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.114e-06 2024-09-21T21:42:13.484 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.484 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.485 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:13.485 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634446+0000", 2024-09-21T21:42:13.485 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1389999999999998e-06 2024-09-21T21:42:13.485 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.485 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.485 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:13.485 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634451+0000", 2024-09-21T21:42:13.485 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0010000000000001e-06 2024-09-21T21:42:13.485 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.485 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.485 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:13.485 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694794+0000", 2024-09-21T21:42:13.485 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060342509000000003 2024-09-21T21:42:13.485 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.485 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.485 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:13.486 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694835+0000", 2024-09-21T21:42:13.486 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0728e-05 2024-09-21T21:42:13.486 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.486 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.486 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:13.486 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694911+0000", 2024-09-21T21:42:13.486 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6428999999999996e-05 2024-09-21T21:42:13.486 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:13.486 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:13.486 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:13.486 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.486 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.486 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1534 3.e 3:724b31a3:::rbd_data.1121e540c096.0000000000000015:b [read 487424~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:42:13.486 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634477+0000", 2024-09-21T21:42:13.486 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 71.805111375999999, 2024-09-21T21:42:13.487 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060624009, 2024-09-21T21:42:13.487 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:13.487 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:13.487 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:13.487 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:13.487 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:42:13.487 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:42:13.487 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1534 2024-09-21T21:42:13.487 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.487 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:13.487 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.487 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:13.487 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634477+0000", 2024-09-21T21:42:13.487 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:13.487 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.488 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.488 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:13.488 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634477+0000", 2024-09-21T21:42:13.488 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:13.488 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.488 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.488 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:13.488 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634478+0000", 2024-09-21T21:42:13.488 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1399999999999997e-07 2024-09-21T21:42:13.488 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.488 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.488 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:13.488 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634480+0000", 2024-09-21T21:42:13.488 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1009999999999999e-06 2024-09-21T21:42:13.489 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.489 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.489 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:13.489 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634486+0000", 2024-09-21T21:42:13.489 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3289999999999999e-06 2024-09-21T21:42:13.489 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.489 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.489 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:13.489 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634491+0000", 2024-09-21T21:42:13.489 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6260000000000003e-06 2024-09-21T21:42:13.489 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.489 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.489 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:13.490 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694943+0000", 2024-09-21T21:42:13.490 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060451960999999999 2024-09-21T21:42:13.490 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.490 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.490 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:13.490 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695014+0000", 2024-09-21T21:42:13.490 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1587000000000004e-05 2024-09-21T21:42:13.490 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.490 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.490 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:13.490 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695101+0000", 2024-09-21T21:42:13.490 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6990999999999994e-05 2024-09-21T21:42:13.491 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:13.491 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:13.491 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:13.491 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.491 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.491 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1545 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 438272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:42:13.491 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634515+0000", 2024-09-21T21:42:13.491 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 71.805073175999993, 2024-09-21T21:42:13.491 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060707944, 2024-09-21T21:42:13.491 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:13.491 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:13.491 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:13.491 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:13.492 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:42:13.492 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:42:13.492 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1545 2024-09-21T21:42:13.492 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.492 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:13.492 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.492 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:13.492 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634515+0000", 2024-09-21T21:42:13.492 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:13.492 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.492 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.492 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:13.492 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634515+0000", 2024-09-21T21:42:13.493 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:13.493 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.493 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.493 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:13.493 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634516+0000", 2024-09-21T21:42:13.493 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0999999999999999e-07 2024-09-21T21:42:13.493 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.493 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.493 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:13.493 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634518+0000", 2024-09-21T21:42:13.493 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.846e-06 2024-09-21T21:42:13.493 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.493 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.493 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:13.494 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634524+0000", 2024-09-21T21:42:13.494 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2550000000000003e-06 2024-09-21T21:42:13.494 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.494 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.494 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:13.494 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634529+0000", 2024-09-21T21:42:13.494 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6940000000000003e-06 2024-09-21T21:42:13.494 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.494 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.494 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:13.494 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695127+0000", 2024-09-21T21:42:13.494 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060598921 2024-09-21T21:42:13.494 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.494 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.495 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:13.495 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695171+0000", 2024-09-21T21:42:13.495 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3588999999999999e-05 2024-09-21T21:42:13.495 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.495 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.495 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:13.495 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695223+0000", 2024-09-21T21:42:13.495 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2228999999999997e-05 2024-09-21T21:42:13.495 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:13.495 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:13.495 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:13.495 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.496 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.496 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1547 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 495616~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:42:13.496 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634534+0000", 2024-09-21T21:42:13.496 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 71.805054385000005, 2024-09-21T21:42:13.496 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060881131999999998, 2024-09-21T21:42:13.496 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:13.496 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:13.496 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:13.496 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:13.496 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:42:13.496 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:42:13.496 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1547 2024-09-21T21:42:13.496 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.497 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:13.497 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.497 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:13.497 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634534+0000", 2024-09-21T21:42:13.497 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:13.497 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.497 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.497 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:13.497 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634534+0000", 2024-09-21T21:42:13.497 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:13.497 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.497 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.497 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:13.497 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634535+0000", 2024-09-21T21:42:13.498 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.77e-07 2024-09-21T21:42:13.498 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.498 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.498 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:13.498 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634542+0000", 2024-09-21T21:42:13.498 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9000000000000006e-06 2024-09-21T21:42:13.498 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.498 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.498 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:13.498 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634549+0000", 2024-09-21T21:42:13.498 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5910000000000001e-06 2024-09-21T21:42:13.498 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.498 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.499 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:13.499 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634554+0000", 2024-09-21T21:42:13.499 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7940000000000002e-06 2024-09-21T21:42:13.499 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.499 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.499 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:13.499 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695256+0000", 2024-09-21T21:42:13.499 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.06070246 2024-09-21T21:42:13.499 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.499 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.499 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:13.499 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695332+0000", 2024-09-21T21:42:13.499 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6248000000000006e-05 2024-09-21T21:42:13.499 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.500 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.500 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:13.500 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695415+0000", 2024-09-21T21:42:13.500 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2762000000000005e-05 2024-09-21T21:42:13.500 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:13.500 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:13.500 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:13.500 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.500 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.500 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1548 3.16 3:6a41a830:::rbd_data.112145fae779.000000000000000a:a [read 299008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:42:13.500 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634561+0000", 2024-09-21T21:42:13.500 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 71.805027808000006, 2024-09-21T21:42:13.500 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060786743999999997, 2024-09-21T21:42:13.501 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:13.501 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:13.501 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:13.501 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:13.501 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:42:13.501 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:42:13.501 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1548 2024-09-21T21:42:13.501 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.501 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:13.501 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.501 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:13.501 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634561+0000", 2024-09-21T21:42:13.501 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:13.501 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.502 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.502 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:13.502 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634561+0000", 2024-09-21T21:42:13.502 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:13.502 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.502 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.502 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:13.502 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634561+0000", 2024-09-21T21:42:13.502 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5199999999999997e-07 2024-09-21T21:42:13.502 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.502 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.502 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:13.502 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634564+0000", 2024-09-21T21:42:13.503 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4040000000000002e-06 2024-09-21T21:42:13.503 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.503 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.503 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:13.503 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634571+0000", 2024-09-21T21:42:13.503 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4379999999999998e-06 2024-09-21T21:42:13.503 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.503 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.503 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:13.503 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634576+0000", 2024-09-21T21:42:13.503 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1399999999999999e-06 2024-09-21T21:42:13.503 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.503 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.504 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:13.504 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695247+0000", 2024-09-21T21:42:13.504 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060670397000000001 2024-09-21T21:42:13.504 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.504 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.504 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:13.504 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695299+0000", 2024-09-21T21:42:13.504 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2564e-05 2024-09-21T21:42:13.504 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.504 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.504 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:13.504 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695347+0000", 2024-09-21T21:42:13.504 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8248999999999998e-05 2024-09-21T21:42:13.504 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:13.505 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:13.505 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:13.505 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.505 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.505 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1549 3.16 3:6a41a830:::rbd_data.112145fae779.000000000000000a:a [read 356352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:42:13.505 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634583+0000", 2024-09-21T21:42:13.505 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 71.805005894999994, 2024-09-21T21:42:13.505 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060889459, 2024-09-21T21:42:13.505 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:13.505 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:13.505 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:13.505 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:13.505 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:42:13.505 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:42:13.506 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1549 2024-09-21T21:42:13.506 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.506 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:13.506 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.506 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:13.506 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634583+0000", 2024-09-21T21:42:13.506 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:13.506 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.506 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.506 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:13.506 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634583+0000", 2024-09-21T21:42:13.506 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:13.506 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.507 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.507 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:13.507 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634583+0000", 2024-09-21T21:42:13.507 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1899999999999998e-07 2024-09-21T21:42:13.507 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.507 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.507 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:13.507 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634585+0000", 2024-09-21T21:42:13.507 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2129999999999998e-06 2024-09-21T21:42:13.507 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.507 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.507 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:13.507 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634592+0000", 2024-09-21T21:42:13.507 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2840000000000003e-06 2024-09-21T21:42:13.507 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.508 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.508 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:13.508 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634596+0000", 2024-09-21T21:42:13.508 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9309999999999999e-06 2024-09-21T21:42:13.508 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.508 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.508 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:13.508 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695371+0000", 2024-09-21T21:42:13.508 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060774425999999999 2024-09-21T21:42:13.508 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.508 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.508 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:13.508 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695419+0000", 2024-09-21T21:42:13.508 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8607999999999999e-05 2024-09-21T21:42:13.509 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.509 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.509 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:13.509 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695472+0000", 2024-09-21T21:42:13.509 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2577999999999997e-05 2024-09-21T21:42:13.509 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:13.509 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:13.509 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:13.509 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.509 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.509 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4385.0:15202 3.16 3:6888948b:::rbd_header.1121493f6459:head [watch reconnect cookie 140325440202496 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-21T21:42:13.509 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:41.736601+0000", 2024-09-21T21:42:13.509 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 31.702987262000001, 2024-09-21T21:42:13.510 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5040056769999999, 2024-09-21T21:42:13.510 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:13.510 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:13.510 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:13.510 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:13.510 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4385", 2024-09-21T21:42:13.510 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:42:13.510 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 15202 2024-09-21T21:42:13.510 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.510 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:13.510 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.510 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:13.510 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736601+0000", 2024-09-21T21:42:13.510 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:13.511 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.511 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.511 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:13.511 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736601+0000", 2024-09-21T21:42:13.511 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:13.511 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.511 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.511 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:13.511 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736602+0000", 2024-09-21T21:42:13.511 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0699999999999999e-06 2024-09-21T21:42:13.511 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.511 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.511 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:13.511 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736607+0000", 2024-09-21T21:42:13.511 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2520000000000001e-06 2024-09-21T21:42:13.512 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.512 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.512 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:13.512 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736617+0000", 2024-09-21T21:42:13.512 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0074e-05 2024-09-21T21:42:13.512 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.512 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.512 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:13.512 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736622+0000", 2024-09-21T21:42:13.512 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.101e-06 2024-09-21T21:42:13.512 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.512 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.512 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:13.512 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736664+0000", 2024-09-21T21:42:13.513 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1869999999999997e-05 2024-09-21T21:42:13.513 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.513 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.513 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op must wait for map", 2024-09-21T21:42:13.513 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736667+0000", 2024-09-21T21:42:13.513 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4750000000000002e-06 2024-09-21T21:42:13.513 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.513 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.513 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:13.513 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742886+0000", 2024-09-21T21:42:13.513 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0062187579999999996 2024-09-21T21:42:13.513 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.513 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.513 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:13.514 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742892+0000", 2024-09-21T21:42:13.514 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0440000000000001e-06 2024-09-21T21:42:13.514 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.514 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.514 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:13.514 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210307+0000", 2024-09-21T21:42:13.514 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.467414781 2024-09-21T21:42:13.514 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.514 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.514 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:13.514 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210322+0000", 2024-09-21T21:42:13.514 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5311e-05 2024-09-21T21:42:13.514 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.514 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.515 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:13.515 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.237624+0000", 2024-09-21T21:42:13.515 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0273018920000001 2024-09-21T21:42:13.515 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.515 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.515 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:13.515 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.237870+0000", 2024-09-21T21:42:13.515 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00024622899999999999 2024-09-21T21:42:13.515 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.515 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.515 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:13.515 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.240607+0000", 2024-09-21T21:42:13.515 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0027368200000000001 2024-09-21T21:42:13.515 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:13.516 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:13.516 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:13.516 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.516 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.516 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5415.0:379 3.16 3:6888948b:::rbd_header.1121493f6459:head [watch reconnect cookie 139702602617920 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-21T21:42:13.516 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:41.739544+0000", 2024-09-21T21:42:13.516 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 31.700044833, 2024-09-21T21:42:13.516 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.501180645, 2024-09-21T21:42:13.516 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:13.516 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:13.516 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:13.516 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:13.516 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5415", 2024-09-21T21:42:13.516 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1212816607", 2024-09-21T21:42:13.517 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 379 2024-09-21T21:42:13.517 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.517 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:13.517 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.517 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:13.517 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739544+0000", 2024-09-21T21:42:13.517 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:13.517 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.517 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.517 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:13.517 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739544+0000", 2024-09-21T21:42:13.517 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:13.517 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.517 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.517 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:13.518 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739545+0000", 2024-09-21T21:42:13.518 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.065e-06 2024-09-21T21:42:13.518 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.518 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.518 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:13.518 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739550+0000", 2024-09-21T21:42:13.518 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9599999999999999e-06 2024-09-21T21:42:13.518 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.518 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.518 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:13.518 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739565+0000", 2024-09-21T21:42:13.518 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5709000000000002e-05 2024-09-21T21:42:13.518 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.518 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.519 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:13.519 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739573+0000", 2024-09-21T21:42:13.519 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7849999999999992e-06 2024-09-21T21:42:13.519 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.519 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.519 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:13.519 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739634+0000", 2024-09-21T21:42:13.519 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0732000000000001e-05 2024-09-21T21:42:13.519 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.519 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.519 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op must wait for map", 2024-09-21T21:42:13.519 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739640+0000", 2024-09-21T21:42:13.519 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0000000000000002e-06 2024-09-21T21:42:13.519 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.520 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.520 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:13.520 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742958+0000", 2024-09-21T21:42:13.520 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0033182390000000002 2024-09-21T21:42:13.520 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.520 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.520 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:13.520 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742966+0000", 2024-09-21T21:42:13.520 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1610000000000003e-06 2024-09-21T21:42:13.520 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.520 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.520 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:13.520 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210405+0000", 2024-09-21T21:42:13.521 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.467438725 2024-09-21T21:42:13.521 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.521 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.521 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:13.521 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210415+0000", 2024-09-21T21:42:13.521 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0234000000000001e-05 2024-09-21T21:42:13.521 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.521 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.521 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:13.521 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.238743+0000", 2024-09-21T21:42:13.521 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0283273070000001 2024-09-21T21:42:13.521 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.521 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.521 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:13.521 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.238842+0000", 2024-09-21T21:42:13.522 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.9896999999999996e-05 2024-09-21T21:42:13.522 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.522 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.522 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:13.522 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.240724+0000", 2024-09-21T21:42:13.522 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001881831 2024-09-21T21:42:13.522 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:13.522 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:13.522 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:13.522 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.522 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.522 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4385.0:15203 3.16 3:6888948b:::rbd_header.1121493f6459:head [call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-21T21:42:13.522 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:41.739737+0000", 2024-09-21T21:42:13.522 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 31.699851821999999, 2024-09-21T21:42:13.523 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4989621799999999, 2024-09-21T21:42:13.523 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:13.523 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:13.523 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:13.523 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:13.523 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4385", 2024-09-21T21:42:13.523 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:42:13.523 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 15203 2024-09-21T21:42:13.523 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.523 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:13.523 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.523 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:13.523 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739737+0000", 2024-09-21T21:42:13.523 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:13.524 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.524 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.524 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:13.524 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739737+0000", 2024-09-21T21:42:13.524 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:13.524 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.524 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.524 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:13.524 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739738+0000", 2024-09-21T21:42:13.524 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.7499999999999998e-07 2024-09-21T21:42:13.524 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.524 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.524 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:13.524 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739743+0000", 2024-09-21T21:42:13.525 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5049999999999998e-06 2024-09-21T21:42:13.525 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.525 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.525 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:13.525 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739757+0000", 2024-09-21T21:42:13.525 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3947e-05 2024-09-21T21:42:13.525 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.525 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.525 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:13.525 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739765+0000", 2024-09-21T21:42:13.525 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9310000000000008e-06 2024-09-21T21:42:13.525 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.525 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.525 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:13.526 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739816+0000", 2024-09-21T21:42:13.526 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0559999999999997e-05 2024-09-21T21:42:13.526 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.526 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.526 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting_for_map not empty", 2024-09-21T21:42:13.526 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739821+0000", 2024-09-21T21:42:13.526 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3539999999999999e-06 2024-09-21T21:42:13.526 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.526 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.526 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:13.526 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742928+0000", 2024-09-21T21:42:13.526 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0031074399999999999 2024-09-21T21:42:13.526 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.526 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.527 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:13.527 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742937+0000", 2024-09-21T21:42:13.527 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1289999999999996e-06 2024-09-21T21:42:13.527 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.527 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.527 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:13.527 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210350+0000", 2024-09-21T21:42:13.527 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.46741311299999999 2024-09-21T21:42:13.527 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.527 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.527 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:13.527 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210358+0000", 2024-09-21T21:42:13.527 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1869999999999997e-06 2024-09-21T21:42:13.528 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.528 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.528 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:13.528 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.238398+0000", 2024-09-21T21:42:13.528 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0280397830000001 2024-09-21T21:42:13.528 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.528 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.528 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:13.528 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.238489+0000", 2024-09-21T21:42:13.528 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.0958000000000006e-05 2024-09-21T21:42:13.528 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.528 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.528 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:13.529 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.238699+0000", 2024-09-21T21:42:13.529 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00021029800000000001 2024-09-21T21:42:13.529 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:13.529 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:13.529 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:13.529 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.529 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.529 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4385.0:15204 3.16 3:6888948b:::rbd_header.1121493f6459:head [watch reconnect cookie 140325440202496 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-21T21:42:13.529 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:42.210726+0000", 2024-09-21T21:42:13.529 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 31.228862943999999, 2024-09-21T21:42:13.529 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.030304557, 2024-09-21T21:42:13.529 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:13.529 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:13.529 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:13.530 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:13.530 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4385", 2024-09-21T21:42:13.530 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:42:13.530 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 15204 2024-09-21T21:42:13.530 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.530 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:13.530 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.530 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:13.530 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210726+0000", 2024-09-21T21:42:13.530 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:13.530 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.530 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.530 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:13.530 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210726+0000", 2024-09-21T21:42:13.531 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:13.531 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.531 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.531 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:13.531 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210727+0000", 2024-09-21T21:42:13.531 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.204e-06 2024-09-21T21:42:13.531 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.531 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.531 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:13.531 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210732+0000", 2024-09-21T21:42:13.531 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5249999999999996e-06 2024-09-21T21:42:13.531 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.531 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.531 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:13.532 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210744+0000", 2024-09-21T21:42:13.532 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1405e-05 2024-09-21T21:42:13.532 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.532 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.532 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:13.532 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210754+0000", 2024-09-21T21:42:13.532 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0122999999999999e-05 2024-09-21T21:42:13.532 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.532 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.532 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:13.532 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210798+0000", 2024-09-21T21:42:13.532 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4005000000000003e-05 2024-09-21T21:42:13.532 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.532 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.533 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:13.533 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210803+0000", 2024-09-21T21:42:13.533 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5130000000000004e-06 2024-09-21T21:42:13.533 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.533 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.533 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:13.533 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.239260+0000", 2024-09-21T21:42:13.533 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0284563040000001 2024-09-21T21:42:13.533 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.533 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.533 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:13.533 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.239318+0000", 2024-09-21T21:42:13.533 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8486000000000002e-05 2024-09-21T21:42:13.533 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.534 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.534 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:13.534 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.241030+0000", 2024-09-21T21:42:13.534 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0017119920000000001 2024-09-21T21:42:13.534 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:13.534 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:13.534 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:13.534 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.534 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.534 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5415.0:380 3.16 3:6888948b:::rbd_header.1121493f6459:head [watch reconnect cookie 139702602617920 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-21T21:42:13.534 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:42.210777+0000", 2024-09-21T21:42:13.534 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 31.228811165, 2024-09-21T21:42:13.534 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.030410566, 2024-09-21T21:42:13.535 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:13.535 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:13.535 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:13.535 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:13.535 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5415", 2024-09-21T21:42:13.535 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1212816607", 2024-09-21T21:42:13.535 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 380 2024-09-21T21:42:13.535 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.535 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:13.535 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.535 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:13.535 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210777+0000", 2024-09-21T21:42:13.535 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:13.535 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.536 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.536 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:13.536 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210777+0000", 2024-09-21T21:42:13.536 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:13.536 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.536 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.536 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:13.536 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210778+0000", 2024-09-21T21:42:13.536 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8999999999999996e-07 2024-09-21T21:42:13.536 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.536 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.536 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:13.536 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210780+0000", 2024-09-21T21:42:13.537 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.503e-06 2024-09-21T21:42:13.537 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.537 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.537 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:13.537 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210788+0000", 2024-09-21T21:42:13.537 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0999999999999998e-06 2024-09-21T21:42:13.537 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.537 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.537 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:13.537 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210799+0000", 2024-09-21T21:42:13.537 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1703e-05 2024-09-21T21:42:13.537 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.537 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.537 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:13.538 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210833+0000", 2024-09-21T21:42:13.538 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4008999999999999e-05 2024-09-21T21:42:13.538 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.538 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.538 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:13.538 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210838+0000", 2024-09-21T21:42:13.538 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7480000000000001e-06 2024-09-21T21:42:13.538 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.538 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.538 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:13.538 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.239612+0000", 2024-09-21T21:42:13.538 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.028773554 2024-09-21T21:42:13.538 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.539 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.539 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:13.539 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.239677+0000", 2024-09-21T21:42:13.539 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5053000000000002e-05 2024-09-21T21:42:13.539 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:13.539 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:13.539 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:13.539 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.241188+0000", 2024-09-21T21:42:13.539 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001511306 2024-09-21T21:42:13.539 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:13.539 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:13.539 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:13.539 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:13.539 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:13.539 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:13.541 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:13.538+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:13.542 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T21:42:13.542 INFO:teuthology.orchestra.run.smithi164.stdout:4810 write 0xaf22200 thru 0xaf28e89 (0x6c8a bytes) 2024-09-21T21:42:13.542 INFO:teuthology.orchestra.run.smithi164.stdout:4813 trunc from 0xaf29000 to 0x5b7e400 2024-09-21T21:42:13.542 INFO:teuthology.orchestra.run.smithi164.stdout:4814 punch from 0x3ada200 to 0x3adde00, (0x3c00 bytes) 2024-09-21T21:42:13.542 INFO:teuthology.orchestra.run.smithi164.stdout:4819 read 0x2f7ba00 thru 0x2f8b0e0 (0xf6e1 bytes) 2024-09-21T21:42:13.542 INFO:teuthology.orchestra.run.smithi164.stdout:4825 write 0x7597c00 thru 0x759f7f9 (0x7bfa bytes) 2024-09-21T21:42:13.543 INFO:teuthology.orchestra.run.smithi164.stdout:4829 read 0x42eaa00 thru 0x42f37a6 (0x8da7 bytes) 2024-09-21T21:42:13.543 INFO:teuthology.orchestra.run.smithi164.stdout:4831 read 0x1383800 thru 0x138f1ec (0xb9ed bytes) 2024-09-21T21:42:13.543 INFO:teuthology.orchestra.run.smithi164.stdout:4834 read 0x2652600 thru 0x265e896 (0xc297 bytes) 2024-09-21T21:42:13.543 INFO:teuthology.orchestra.run.smithi164.stdout:4839 write 0x1787000 thru 0x178832b (0x132c bytes) 2024-09-21T21:42:13.545 INFO:teuthology.orchestra.run.smithi164.stdout:4840 trunc from 0x759f800 to 0xb78ec00 2024-09-21T21:42:13.569 INFO:teuthology.orchestra.run.smithi164.stdout:4841 read 0x7908800 thru 0x7910977 (0x8178 bytes) 2024-09-21T21:42:13.580 INFO:teuthology.orchestra.run.smithi164.stdout:4842 trunc from 0xb78ec00 to 0x2d01800 2024-09-21T21:42:13.605 INFO:teuthology.orchestra.run.smithi164.stdout:4843 read 0x230c000 thru 0x231a4fa (0xe4fb bytes) 2024-09-21T21:42:13.606 INFO:teuthology.orchestra.run.smithi164.stdout:4844 punch from 0x2126200 to 0x212e800, (0x8600 bytes) 2024-09-21T21:42:13.614 INFO:teuthology.orchestra.run.smithi164.stdout:4846 trunc from 0x2d01800 to 0x8e27400 2024-09-21T21:42:13.642 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:13.640+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:13.650 INFO:teuthology.orchestra.run.smithi164.stdout:4847 write 0xcd6a400 thru 0xcd7634d (0xbf4e bytes) 2024-09-21T21:42:13.651 INFO:teuthology.orchestra.run.smithi164.stdout:4848 punch from 0x9f9f800 to 0x9fa9000, (0x9800 bytes) 2024-09-21T21:42:13.652 INFO:teuthology.orchestra.run.smithi164.stdout:4849 read 0xb5e5200 thru 0xb5f2597 (0xd398 bytes) 2024-09-21T21:42:13.653 INFO:teuthology.orchestra.run.smithi164.stdout:4850 read 0xc570c00 thru 0xc57341d (0x281e bytes) 2024-09-21T21:42:13.653 INFO:teuthology.orchestra.run.smithi164.stdout:4851 read 0x4b6400 thru 0x4bab57 (0x4758 bytes) 2024-09-21T21:42:13.669 INFO:teuthology.orchestra.run.smithi164.stdout:4852 trunc from 0xcd76400 to 0x16dee00 2024-09-21T21:42:13.681 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:13.681 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:42:13.681 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 0 2024-09-21T21:42:13.681 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:13.692 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T21:42:13.723 INFO:teuthology.orchestra.run.smithi164.stdout:4853 trunc from 0x16dee00 to 0xa1a00 2024-09-21T21:42:13.778 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:13.777+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:13.837 INFO:teuthology.orchestra.run.smithi164.stdout:4854 write 0xe3d0e00 thru 0xe3dbe29 (0xb02a bytes) 2024-09-21T21:42:13.838 INFO:teuthology.orchestra.run.smithi164.stdout:4856 read 0xa726400 thru 0xa72f45b (0x905c bytes) 2024-09-21T21:42:13.838 INFO:teuthology.orchestra.run.smithi164.stdout:4857 punch from 0x856f200 to 0x8570800, (0x1600 bytes) 2024-09-21T21:42:13.839 INFO:teuthology.orchestra.run.smithi164.stdout:4859 read 0xb365800 thru 0xb371bff (0xc400 bytes) 2024-09-21T21:42:13.839 INFO:teuthology.orchestra.run.smithi164.stdout:4862 read 0xae7f600 thru 0xae875b6 (0x7fb7 bytes) 2024-09-21T21:42:13.839 INFO:teuthology.orchestra.run.smithi164.stdout:4863 write 0x9297200 thru 0x929acb9 (0x3aba bytes) 2024-09-21T21:42:13.839 INFO:teuthology.orchestra.run.smithi164.stdout:4865 read 0xb567200 thru 0xb56a06b (0x2e6c bytes) 2024-09-21T21:42:13.839 INFO:teuthology.orchestra.run.smithi164.stdout:4868 read 0xaf38c00 thru 0xaf39133 (0x534 bytes) 2024-09-21T21:42:13.840 INFO:teuthology.orchestra.run.smithi164.stdout:4871 write 0xa6f3a00 thru 0xa6f9309 (0x590a bytes) 2024-09-21T21:42:13.840 INFO:teuthology.orchestra.run.smithi164.stdout:4875 write 0xb8ea800 thru 0xb8ede9d (0x369e bytes) 2024-09-21T21:42:13.840 INFO:teuthology.orchestra.run.smithi164.stdout:4876 write 0x5686c00 thru 0x568800b (0x140c bytes) 2024-09-21T21:42:13.840 INFO:teuthology.orchestra.run.smithi164.stdout:4877 write 0xbe4600 thru 0xbf30f3 (0xeaf4 bytes) 2024-09-21T21:42:13.840 INFO:teuthology.orchestra.run.smithi164.stdout:4878 write 0xd019200 thru 0xd026a84 (0xd885 bytes) 2024-09-21T21:42:13.840 INFO:teuthology.orchestra.run.smithi164.stdout:4879 punch from 0xc8ca00 to 0xc98c00, (0xc200 bytes) 2024-09-21T21:42:13.841 INFO:teuthology.orchestra.run.smithi164.stdout:4880 write 0xe42800 thru 0xe51462 (0xec63 bytes) 2024-09-21T21:42:13.841 INFO:teuthology.orchestra.run.smithi164.stdout:4882 read 0xcbe7800 thru 0xcbed598 (0x5d99 bytes) 2024-09-21T21:42:13.841 INFO:teuthology.orchestra.run.smithi164.stdout:4883 punch from 0xd4c6600 to 0xd4c9c00, (0x3600 bytes) 2024-09-21T21:42:13.841 INFO:teuthology.orchestra.run.smithi164.stdout:4884 write 0xd01e00 thru 0xd01ff4 (0x1f5 bytes) 2024-09-21T21:42:13.841 INFO:teuthology.orchestra.run.smithi164.stdout:4886 punch from 0x1de0800 to 0x1de6800, (0x6000 bytes) 2024-09-21T21:42:13.841 INFO:teuthology.orchestra.run.smithi164.stdout:4889 read 0x434de00 thru 0x435b704 (0xd905 bytes) 2024-09-21T21:42:13.842 INFO:teuthology.orchestra.run.smithi164.stdout:4891 read 0x856800 thru 0x85ecca (0x84cb bytes) 2024-09-21T21:42:13.842 INFO:teuthology.orchestra.run.smithi164.stdout:4892 read 0x8b12000 thru 0x8b1a654 (0x8655 bytes) 2024-09-21T21:42:13.842 INFO:teuthology.orchestra.run.smithi164.stdout:4893 read 0x17fca00 thru 0x1809b11 (0xd112 bytes) 2024-09-21T21:42:13.843 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:13.842+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:13.845 INFO:teuthology.orchestra.run.smithi164.stdout:4894 trunc from 0xe3dc000 to 0xbd40a00 2024-09-21T21:42:13.873 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:13.874 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:42:13.874 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:42:13.874 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:42:13.874 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:13.887 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T21:42:13.940 INFO:teuthology.orchestra.run.smithi164.stdout:4895 write 0xa074800 thru 0xa0783d5 (0x3bd6 bytes) 2024-09-21T21:42:13.942 INFO:teuthology.orchestra.run.smithi164.stdout:4899 punch from 0x5d48a00 to 0x5d4c400, (0x3a00 bytes) 2024-09-21T21:42:13.942 INFO:teuthology.orchestra.run.smithi164.stdout:4901 punch from 0x71d3400 to 0x71da800, (0x7400 bytes) 2024-09-21T21:42:13.943 INFO:teuthology.orchestra.run.smithi164.stdout:4903 flatten 2024-09-21T21:42:13.943 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:13.942+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:13.982 INFO:teuthology.orchestra.run.smithi164.stdout:4909 write 0x216c00 thru 0x21f54a (0x894b bytes) 2024-09-21T21:42:13.983 INFO:teuthology.orchestra.run.smithi164.stdout:4910 punch from 0x5a0e800 to 0x5a19400, (0xac00 bytes) 2024-09-21T21:42:13.984 INFO:teuthology.orchestra.run.smithi164.stdout:4911 read 0x87afc00 thru 0x87bfbdc (0xffdd bytes) 2024-09-21T21:42:13.987 INFO:teuthology.orchestra.run.smithi164.stdout:4914 read 0x7921400 thru 0x7922494 (0x1095 bytes) 2024-09-21T21:42:13.992 INFO:teuthology.orchestra.run.smithi164.stdout:4915 trunc from 0xbd40a00 to 0x7a76c00 2024-09-21T21:42:14.027 INFO:teuthology.orchestra.run.smithi164.stdout:4916 write 0xf53200 thru 0xf582cc (0x50cd bytes) 2024-09-21T21:42:14.044 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:14.043+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:14.049 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:14.049 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 20, 2024-09-21T21:42:14.049 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 600, 2024-09-21T21:42:14.049 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:42:14.049 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.049 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11319 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 5492736~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:14.049 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.712385+0000", 2024-09-21T21:42:14.049 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 23.334398785000001, 2024-09-21T21:42:14.050 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078627889000000006, 2024-09-21T21:42:14.050 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:14.050 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:14.050 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:14.050 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:14.050 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:14.050 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:14.050 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11319 2024-09-21T21:42:14.050 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.050 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:14.050 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.050 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:14.050 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712385+0000", 2024-09-21T21:42:14.051 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.051 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.051 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.051 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:14.051 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712385+0000", 2024-09-21T21:42:14.051 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.051 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.051 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.051 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:14.051 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712385+0000", 2024-09-21T21:42:14.051 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2800000000000002e-07 2024-09-21T21:42:14.051 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.052 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.052 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:14.052 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712387+0000", 2024-09-21T21:42:14.052 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1569999999999998e-06 2024-09-21T21:42:14.052 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.052 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.052 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:14.052 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712393+0000", 2024-09-21T21:42:14.052 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9370000000000001e-06 2024-09-21T21:42:14.052 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.052 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.052 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:14.052 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712398+0000", 2024-09-21T21:42:14.053 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5020000000000003e-06 2024-09-21T21:42:14.053 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.053 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.053 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:14.053 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.790746+0000", 2024-09-21T21:42:14.053 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078348589999999996 2024-09-21T21:42:14.053 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.053 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.053 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:14.053 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.790844+0000", 2024-09-21T21:42:14.053 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.7526000000000001e-05 2024-09-21T21:42:14.053 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.053 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.054 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:14.054 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.791013+0000", 2024-09-21T21:42:14.054 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00016874899999999999 2024-09-21T21:42:14.054 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.054 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:14.054 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.054 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.054 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.054 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11320 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 6180864~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:14.054 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.712405+0000", 2024-09-21T21:42:14.054 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 23.334378825999998, 2024-09-21T21:42:14.054 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078882870999999993, 2024-09-21T21:42:14.054 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:14.054 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:14.055 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:14.055 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:14.055 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:14.055 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:14.055 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11320 2024-09-21T21:42:14.055 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.055 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:14.055 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.055 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:14.055 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712405+0000", 2024-09-21T21:42:14.055 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.055 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.055 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.055 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:14.055 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712405+0000", 2024-09-21T21:42:14.056 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.056 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.056 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.056 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:14.056 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.713942+0000", 2024-09-21T21:42:14.056 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.001536882 2024-09-21T21:42:14.056 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.056 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.056 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:14.056 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.713946+0000", 2024-09-21T21:42:14.056 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4549999999999997e-06 2024-09-21T21:42:14.056 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.057 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.057 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:14.057 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.713958+0000", 2024-09-21T21:42:14.057 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1775e-05 2024-09-21T21:42:14.057 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.057 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.057 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:14.057 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.713965+0000", 2024-09-21T21:42:14.057 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5100000000000001e-06 2024-09-21T21:42:14.057 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.057 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.057 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:14.057 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.791059+0000", 2024-09-21T21:42:14.058 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.077094036000000005 2024-09-21T21:42:14.058 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.058 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.058 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:14.058 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.791175+0000", 2024-09-21T21:42:14.058 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000115895 2024-09-21T21:42:14.058 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.058 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.058 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:14.058 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.791288+0000", 2024-09-21T21:42:14.058 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000112318 2024-09-21T21:42:14.058 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.058 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:14.058 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.059 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.059 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.059 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11325 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 5414912~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:14.059 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.714095+0000", 2024-09-21T21:42:14.059 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 23.332688830999999, 2024-09-21T21:42:14.059 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078631894999999993, 2024-09-21T21:42:14.059 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:14.059 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:14.059 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:14.059 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:14.059 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:14.059 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:14.059 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11325 2024-09-21T21:42:14.059 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.059 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:14.060 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.060 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:14.060 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714095+0000", 2024-09-21T21:42:14.060 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.060 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.060 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.060 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:14.060 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714095+0000", 2024-09-21T21:42:14.060 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.060 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.060 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.060 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:14.060 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714095+0000", 2024-09-21T21:42:14.060 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7500000000000001e-07 2024-09-21T21:42:14.060 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.061 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.061 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:14.061 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714097+0000", 2024-09-21T21:42:14.061 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.12e-06 2024-09-21T21:42:14.061 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.061 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.061 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:14.061 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714104+0000", 2024-09-21T21:42:14.061 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3520000000000003e-06 2024-09-21T21:42:14.061 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.061 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.061 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:14.061 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714108+0000", 2024-09-21T21:42:14.061 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8810000000000004e-06 2024-09-21T21:42:14.062 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.062 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.062 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:14.062 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792509+0000", 2024-09-21T21:42:14.062 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078400739999999997 2024-09-21T21:42:14.062 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.062 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.062 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:14.062 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792613+0000", 2024-09-21T21:42:14.062 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000103939 2024-09-21T21:42:14.062 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.062 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.062 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:14.062 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792727+0000", 2024-09-21T21:42:14.062 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011348799999999999 2024-09-21T21:42:14.062 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.063 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:14.063 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.063 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.063 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.063 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11326 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 6103040~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:14.063 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.714115+0000", 2024-09-21T21:42:14.063 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 23.332668803000001, 2024-09-21T21:42:14.063 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078963056000000004, 2024-09-21T21:42:14.063 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:14.063 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:14.063 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:14.063 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:14.063 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:14.063 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:14.063 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11326 2024-09-21T21:42:14.064 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.064 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:14.064 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.064 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:14.064 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714115+0000", 2024-09-21T21:42:14.064 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.064 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.064 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.064 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:14.064 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714115+0000", 2024-09-21T21:42:14.064 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.064 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.064 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.064 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:14.064 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.715940+0000", 2024-09-21T21:42:14.064 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0018254969999999999 2024-09-21T21:42:14.064 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.065 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.065 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:14.065 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.715945+0000", 2024-09-21T21:42:14.065 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6120000000000001e-06 2024-09-21T21:42:14.065 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.065 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.065 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:14.065 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.715954+0000", 2024-09-21T21:42:14.065 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.5899999999999997e-06 2024-09-21T21:42:14.065 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.065 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.065 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:14.065 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.715959+0000", 2024-09-21T21:42:14.065 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0849999999999996e-06 2024-09-21T21:42:14.065 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.065 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.066 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:14.066 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792771+0000", 2024-09-21T21:42:14.066 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.076811829999999998 2024-09-21T21:42:14.066 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.066 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.066 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:14.066 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792882+0000", 2024-09-21T21:42:14.066 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000110857 2024-09-21T21:42:14.066 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.066 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.066 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:14.066 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.793078+0000", 2024-09-21T21:42:14.066 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00019558499999999999 2024-09-21T21:42:14.066 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.066 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:14.067 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.067 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.067 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.067 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11332 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 7565312~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:14.067 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.716063+0000", 2024-09-21T21:42:14.067 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 23.330720269, 2024-09-21T21:42:14.067 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078787482000000006, 2024-09-21T21:42:14.067 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:14.067 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:14.067 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:14.067 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:14.067 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:14.067 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:14.067 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11332 2024-09-21T21:42:14.067 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.068 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:14.068 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.068 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:14.068 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.716063+0000", 2024-09-21T21:42:14.068 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.068 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.068 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.068 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:14.068 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.716063+0000", 2024-09-21T21:42:14.068 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.068 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.068 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.068 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:14.068 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.717944+0000", 2024-09-21T21:42:14.068 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0018812309999999999 2024-09-21T21:42:14.068 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.069 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.069 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:14.069 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.717949+0000", 2024-09-21T21:42:14.069 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9389999999999996e-06 2024-09-21T21:42:14.069 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.069 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.069 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:14.069 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.717963+0000", 2024-09-21T21:42:14.069 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.3899e-05 2024-09-21T21:42:14.069 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.069 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.069 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:14.069 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.717970+0000", 2024-09-21T21:42:14.069 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.426e-06 2024-09-21T21:42:14.070 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.070 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.070 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:14.070 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.794573+0000", 2024-09-21T21:42:14.070 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.076602990999999995 2024-09-21T21:42:14.070 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.070 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.070 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:14.070 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.794682+0000", 2024-09-21T21:42:14.070 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000109541 2024-09-21T21:42:14.070 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.070 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.070 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:14.070 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.794851+0000", 2024-09-21T21:42:14.070 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000168455 2024-09-21T21:42:14.071 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.071 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:14.071 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.071 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.071 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.071 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11339 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 6266880~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:14.071 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.718117+0000", 2024-09-21T21:42:14.071 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 23.328666243000001, 2024-09-21T21:42:14.071 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078703717000000006, 2024-09-21T21:42:14.071 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:14.071 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:14.071 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:14.071 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:14.072 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:14.072 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:14.072 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11339 2024-09-21T21:42:14.072 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.072 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:14.072 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.072 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:14.072 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.718117+0000", 2024-09-21T21:42:14.072 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.072 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.072 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.072 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:14.072 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.718117+0000", 2024-09-21T21:42:14.072 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.073 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.073 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.073 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:14.073 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720019+0000", 2024-09-21T21:42:14.073 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0019012549999999999 2024-09-21T21:42:14.073 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.073 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.073 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:14.073 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720024+0000", 2024-09-21T21:42:14.073 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1100000000000002e-06 2024-09-21T21:42:14.073 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.073 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.073 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:14.073 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720035+0000", 2024-09-21T21:42:14.073 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.094e-05 2024-09-21T21:42:14.073 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.074 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.074 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:14.074 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720040+0000", 2024-09-21T21:42:14.074 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1830000000000002e-06 2024-09-21T21:42:14.074 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.074 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.074 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:14.074 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796618+0000", 2024-09-21T21:42:14.074 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.076578033000000004 2024-09-21T21:42:14.074 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.074 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.074 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:14.074 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796696+0000", 2024-09-21T21:42:14.074 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7836000000000005e-05 2024-09-21T21:42:14.074 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.074 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.075 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:14.075 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796821+0000", 2024-09-21T21:42:14.075 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00012536000000000001 2024-09-21T21:42:14.075 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.075 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:14.075 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.075 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.075 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.075 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11345 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 5472256~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:14.075 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.720116+0000", 2024-09-21T21:42:14.075 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 23.326667343, 2024-09-21T21:42:14.075 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078667730000000005, 2024-09-21T21:42:14.075 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:14.075 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:14.075 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:14.075 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:14.076 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:14.076 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:14.076 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11345 2024-09-21T21:42:14.076 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.076 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:14.076 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.076 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:14.076 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720116+0000", 2024-09-21T21:42:14.076 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.076 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.076 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.076 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:14.076 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720116+0000", 2024-09-21T21:42:14.076 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.077 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.077 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.077 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:14.077 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720116+0000", 2024-09-21T21:42:14.077 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.04e-07 2024-09-21T21:42:14.077 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.077 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.077 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:14.077 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720118+0000", 2024-09-21T21:42:14.077 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.181e-06 2024-09-21T21:42:14.077 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.077 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.078 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:14.078 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720121+0000", 2024-09-21T21:42:14.078 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0199999999999999e-06 2024-09-21T21:42:14.078 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.078 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.078 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:14.078 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720123+0000", 2024-09-21T21:42:14.078 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.5469999999999999e-06 2024-09-21T21:42:14.078 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.078 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.078 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:14.079 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798626+0000", 2024-09-21T21:42:14.079 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078503137000000001 2024-09-21T21:42:14.079 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.079 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.079 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:14.079 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798677+0000", 2024-09-21T21:42:14.079 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0930999999999997e-05 2024-09-21T21:42:14.079 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.079 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.079 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:14.079 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798784+0000", 2024-09-21T21:42:14.079 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010671 2024-09-21T21:42:14.079 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.079 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:14.080 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.080 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.080 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.080 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11346 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 5500928~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:14.080 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.720126+0000", 2024-09-21T21:42:14.080 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 23.326657269999998, 2024-09-21T21:42:14.080 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078863421000000003, 2024-09-21T21:42:14.080 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:14.080 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:14.080 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:14.080 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:14.080 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:14.080 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:14.080 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11346 2024-09-21T21:42:14.080 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.081 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:14.081 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.081 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:14.081 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720126+0000", 2024-09-21T21:42:14.081 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.081 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.081 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.081 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:14.081 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720126+0000", 2024-09-21T21:42:14.081 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.081 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.081 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.081 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:14.081 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.722145+0000", 2024-09-21T21:42:14.082 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0020183219999999999 2024-09-21T21:42:14.082 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.082 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.082 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:14.082 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.722150+0000", 2024-09-21T21:42:14.082 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1760000000000001e-06 2024-09-21T21:42:14.082 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.082 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.082 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:14.082 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.722161+0000", 2024-09-21T21:42:14.082 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1566000000000001e-05 2024-09-21T21:42:14.082 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.082 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.082 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:14.082 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.722169+0000", 2024-09-21T21:42:14.083 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.2840000000000001e-06 2024-09-21T21:42:14.083 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.083 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.083 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:14.083 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798815+0000", 2024-09-21T21:42:14.083 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.076646607000000005 2024-09-21T21:42:14.083 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.083 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.083 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:14.083 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798870+0000", 2024-09-21T21:42:14.083 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4486e-05 2024-09-21T21:42:14.083 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.083 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.083 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:14.083 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798990+0000", 2024-09-21T21:42:14.084 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011998 2024-09-21T21:42:14.084 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.084 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:14.084 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.084 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.084 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.084 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13629 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7852032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:14.084 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943319+0000", 2024-09-21T21:42:14.084 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 23.103464628000001, 2024-09-21T21:42:14.084 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078610975999999999, 2024-09-21T21:42:14.084 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:14.084 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:14.084 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:14.084 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:14.084 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:14.085 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:14.085 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13629 2024-09-21T21:42:14.085 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.085 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:14.085 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.085 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:14.085 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943319+0000", 2024-09-21T21:42:14.085 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.085 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.085 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.085 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:14.085 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943319+0000", 2024-09-21T21:42:14.085 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.085 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.086 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.086 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:14.086 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943319+0000", 2024-09-21T21:42:14.086 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6199999999999999e-07 2024-09-21T21:42:14.086 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.086 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.086 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:14.086 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943321+0000", 2024-09-21T21:42:14.086 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1009999999999999e-06 2024-09-21T21:42:14.086 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.086 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.086 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:14.086 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943328+0000", 2024-09-21T21:42:14.087 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.1029999999999996e-06 2024-09-21T21:42:14.087 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.087 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.087 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:14.087 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943333+0000", 2024-09-21T21:42:14.087 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7709999999999997e-06 2024-09-21T21:42:14.087 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.087 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.087 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:14.087 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.021679+0000", 2024-09-21T21:42:14.087 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078345673000000005 2024-09-21T21:42:14.087 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.087 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.088 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:14.088 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.021771+0000", 2024-09-21T21:42:14.088 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2201999999999996e-05 2024-09-21T21:42:14.088 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.088 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.088 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:14.088 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.021930+0000", 2024-09-21T21:42:14.088 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00015876400000000001 2024-09-21T21:42:14.088 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.088 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:14.088 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.088 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.088 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.088 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13630 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7880704~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:14.089 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943339+0000", 2024-09-21T21:42:14.089 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 23.103444440000001, 2024-09-21T21:42:14.089 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078939410000000002, 2024-09-21T21:42:14.089 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:14.089 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:14.089 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:14.089 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:14.089 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:14.089 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:14.089 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13630 2024-09-21T21:42:14.089 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.089 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:14.089 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.089 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:14.090 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943339+0000", 2024-09-21T21:42:14.090 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.090 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.090 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.090 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:14.090 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943339+0000", 2024-09-21T21:42:14.090 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.090 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.090 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.090 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:14.090 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943339+0000", 2024-09-21T21:42:14.090 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8000000000000001e-07 2024-09-21T21:42:14.090 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.090 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.091 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:14.091 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943341+0000", 2024-09-21T21:42:14.091 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9850000000000001e-06 2024-09-21T21:42:14.091 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.091 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.091 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:14.091 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943347+0000", 2024-09-21T21:42:14.091 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2979999999999999e-06 2024-09-21T21:42:14.091 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.091 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.091 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:14.091 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943354+0000", 2024-09-21T21:42:14.091 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.2969999999999998e-06 2024-09-21T21:42:14.091 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.091 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.091 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:14.092 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.021996+0000", 2024-09-21T21:42:14.092 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078641680000000005 2024-09-21T21:42:14.092 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.092 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.092 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:14.092 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022104+0000", 2024-09-21T21:42:14.092 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000107902 2024-09-21T21:42:14.092 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.092 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.092 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:14.092 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022278+0000", 2024-09-21T21:42:14.092 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00017486800000000001 2024-09-21T21:42:14.092 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.092 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:14.093 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.093 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.093 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.093 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13631 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7909376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:14.093 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943362+0000", 2024-09-21T21:42:14.093 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 23.103421619999999, 2024-09-21T21:42:14.093 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079195644999999995, 2024-09-21T21:42:14.093 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:14.093 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:14.093 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:14.093 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:14.093 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:14.093 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:14.093 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13631 2024-09-21T21:42:14.094 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.094 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:14.094 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.094 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:14.094 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943362+0000", 2024-09-21T21:42:14.094 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.094 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.094 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.094 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:14.094 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943362+0000", 2024-09-21T21:42:14.094 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.094 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.094 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.094 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:14.094 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943362+0000", 2024-09-21T21:42:14.095 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8799999999999998e-07 2024-09-21T21:42:14.095 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.095 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.095 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:14.095 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943364+0000", 2024-09-21T21:42:14.095 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9850000000000001e-06 2024-09-21T21:42:14.095 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.095 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.095 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:14.095 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943371+0000", 2024-09-21T21:42:14.095 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0650000000000001e-06 2024-09-21T21:42:14.095 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.095 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.095 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:14.096 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943376+0000", 2024-09-21T21:42:14.096 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8250000000000004e-06 2024-09-21T21:42:14.096 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.096 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.096 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:14.096 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022329+0000", 2024-09-21T21:42:14.096 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078952888999999998 2024-09-21T21:42:14.096 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.096 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.096 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:14.096 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022419+0000", 2024-09-21T21:42:14.096 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.9738e-05 2024-09-21T21:42:14.096 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.097 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.097 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:14.097 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022558+0000", 2024-09-21T21:42:14.097 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00013875500000000001 2024-09-21T21:42:14.097 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.097 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:14.097 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.097 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.097 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.097 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13632 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7966720~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:14.097 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943382+0000", 2024-09-21T21:42:14.097 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 23.103401386000002, 2024-09-21T21:42:14.097 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079495727000000002, 2024-09-21T21:42:14.098 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:14.098 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:14.098 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:14.098 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:14.098 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:14.098 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:14.098 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13632 2024-09-21T21:42:14.098 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.098 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:14.098 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.098 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:14.098 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943382+0000", 2024-09-21T21:42:14.098 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.099 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.099 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.099 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:14.099 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943382+0000", 2024-09-21T21:42:14.099 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.099 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.099 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.099 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:14.099 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943382+0000", 2024-09-21T21:42:14.099 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7e-07 2024-09-21T21:42:14.099 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.099 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.099 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:14.100 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943385+0000", 2024-09-21T21:42:14.100 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2230000000000001e-06 2024-09-21T21:42:14.100 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.100 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.100 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:14.100 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943391+0000", 2024-09-21T21:42:14.100 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6810000000000001e-06 2024-09-21T21:42:14.100 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.100 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.100 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:14.100 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943397+0000", 2024-09-21T21:42:14.100 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6559999999999997e-06 2024-09-21T21:42:14.100 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.100 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.101 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:14.101 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022601+0000", 2024-09-21T21:42:14.101 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079204051999999997 2024-09-21T21:42:14.101 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.101 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.101 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:14.101 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022704+0000", 2024-09-21T21:42:14.101 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010329599999999999 2024-09-21T21:42:14.101 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.101 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.101 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:14.101 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022878+0000", 2024-09-21T21:42:14.101 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000173449 2024-09-21T21:42:14.102 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.102 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:14.102 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.102 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.102 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.102 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13633 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7938048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:14.102 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943403+0000", 2024-09-21T21:42:14.102 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 23.103380456, 2024-09-21T21:42:14.102 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079794697999999997, 2024-09-21T21:42:14.102 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:14.102 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:14.102 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:14.102 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:14.102 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:14.103 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:14.103 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13633 2024-09-21T21:42:14.103 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.103 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:14.103 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.103 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:14.103 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943403+0000", 2024-09-21T21:42:14.103 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.103 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.103 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.103 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:14.103 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943403+0000", 2024-09-21T21:42:14.103 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.104 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.104 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.104 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:14.104 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943403+0000", 2024-09-21T21:42:14.104 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.65e-07 2024-09-21T21:42:14.104 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.104 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.104 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:14.104 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943405+0000", 2024-09-21T21:42:14.104 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0020000000000001e-06 2024-09-21T21:42:14.104 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.104 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.104 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:14.104 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943412+0000", 2024-09-21T21:42:14.105 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.8379999999999996e-06 2024-09-21T21:42:14.105 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.105 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.105 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:14.105 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943417+0000", 2024-09-21T21:42:14.105 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7650000000000001e-06 2024-09-21T21:42:14.105 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.105 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.105 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:14.105 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022922+0000", 2024-09-21T21:42:14.105 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079504512999999999 2024-09-21T21:42:14.105 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.105 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.105 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:14.106 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023039+0000", 2024-09-21T21:42:14.106 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000117044 2024-09-21T21:42:14.106 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.106 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.106 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:14.106 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023198+0000", 2024-09-21T21:42:14.106 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000159171 2024-09-21T21:42:14.106 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.106 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:14.106 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.106 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.106 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.106 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13634 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7995392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:14.107 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943423+0000", 2024-09-21T21:42:14.107 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 23.103360748, 2024-09-21T21:42:14.107 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.080142358999999996, 2024-09-21T21:42:14.107 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:14.107 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:14.107 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:14.107 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:14.107 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:14.107 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:14.107 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13634 2024-09-21T21:42:14.107 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.107 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:14.107 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.107 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:14.107 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943423+0000", 2024-09-21T21:42:14.108 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.108 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.108 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.108 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:14.108 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943423+0000", 2024-09-21T21:42:14.108 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.108 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.108 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.108 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:14.108 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943423+0000", 2024-09-21T21:42:14.108 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.58e-07 2024-09-21T21:42:14.108 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.108 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.108 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:14.109 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943425+0000", 2024-09-21T21:42:14.109 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9180000000000001e-06 2024-09-21T21:42:14.109 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.109 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.109 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:14.109 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943431+0000", 2024-09-21T21:42:14.109 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4439999999999996e-06 2024-09-21T21:42:14.109 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.109 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.109 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:14.109 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943436+0000", 2024-09-21T21:42:14.109 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.887e-06 2024-09-21T21:42:14.109 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.109 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.110 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:14.110 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023266+0000", 2024-09-21T21:42:14.110 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079829622000000003 2024-09-21T21:42:14.110 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.110 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.110 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:14.110 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023384+0000", 2024-09-21T21:42:14.110 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000117658 2024-09-21T21:42:14.110 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.110 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.110 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:14.110 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023565+0000", 2024-09-21T21:42:14.110 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00018147200000000001 2024-09-21T21:42:14.110 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.110 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:14.111 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.111 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.111 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.111 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13635 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 8024064~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:14.111 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943447+0000", 2024-09-21T21:42:14.111 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 23.103336208000002, 2024-09-21T21:42:14.111 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.080417653000000006, 2024-09-21T21:42:14.111 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:14.111 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:14.111 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:14.111 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:14.111 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:14.111 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:14.111 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13635 2024-09-21T21:42:14.111 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.112 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:14.112 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.112 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:14.112 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943447+0000", 2024-09-21T21:42:14.112 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.112 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.112 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.112 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:14.112 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943447+0000", 2024-09-21T21:42:14.112 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.112 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.112 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.112 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:14.112 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943448+0000", 2024-09-21T21:42:14.113 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1699999999999999e-07 2024-09-21T21:42:14.113 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.113 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.113 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:14.113 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943450+0000", 2024-09-21T21:42:14.113 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1639999999999999e-06 2024-09-21T21:42:14.113 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.113 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.113 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:14.113 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943456+0000", 2024-09-21T21:42:14.113 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8379999999999998e-06 2024-09-21T21:42:14.113 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.113 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.113 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:14.113 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943460+0000", 2024-09-21T21:42:14.113 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4700000000000004e-06 2024-09-21T21:42:14.114 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.114 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.114 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:14.114 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023595+0000", 2024-09-21T21:42:14.114 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.080134869999999997 2024-09-21T21:42:14.114 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.114 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.114 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:14.114 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023688+0000", 2024-09-21T21:42:14.114 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2794000000000006e-05 2024-09-21T21:42:14.114 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.114 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.114 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:14.115 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023865+0000", 2024-09-21T21:42:14.115 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0001771 2024-09-21T21:42:14.115 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.115 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:14.115 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.115 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.115 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.115 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13636 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 8052736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:14.115 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943467+0000", 2024-09-21T21:42:14.115 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 23.103316470999999, 2024-09-21T21:42:14.115 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.080694399999999999, 2024-09-21T21:42:14.115 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:14.115 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:14.115 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:14.115 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:14.116 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:14.116 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:14.116 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13636 2024-09-21T21:42:14.116 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.116 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:14.116 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.116 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:14.116 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943467+0000", 2024-09-21T21:42:14.116 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.116 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.116 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.116 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:14.116 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943467+0000", 2024-09-21T21:42:14.116 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.116 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.116 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.117 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:14.117 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944940+0000", 2024-09-21T21:42:14.117 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0014725109999999999 2024-09-21T21:42:14.117 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.117 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.117 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:14.117 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944949+0000", 2024-09-21T21:42:14.117 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2930000000000006e-06 2024-09-21T21:42:14.117 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.117 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.117 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:14.117 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944958+0000", 2024-09-21T21:42:14.117 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.6899999999999998e-06 2024-09-21T21:42:14.117 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.118 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.118 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:14.118 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944966+0000", 2024-09-21T21:42:14.118 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0690000000000002e-06 2024-09-21T21:42:14.118 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.118 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.118 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:14.118 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023900+0000", 2024-09-21T21:42:14.118 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078934371000000003 2024-09-21T21:42:14.118 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.118 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.118 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:14.118 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024009+0000", 2024-09-21T21:42:14.118 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000108857 2024-09-21T21:42:14.119 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.119 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.119 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:14.119 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024161+0000", 2024-09-21T21:42:14.119 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000152609 2024-09-21T21:42:14.119 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.119 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:14.119 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.119 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.119 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.119 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13637 3.12 3.1dc31b72 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:14.119 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.944978+0000", 2024-09-21T21:42:14.119 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 23.101805536000001, 2024-09-21T21:42:14.119 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079232604999999998, 2024-09-21T21:42:14.119 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:14.120 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:14.120 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:42:14.120 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:14.120 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:14.120 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:14.120 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13637 2024-09-21T21:42:14.120 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.120 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:14.120 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.120 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:14.120 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944978+0000", 2024-09-21T21:42:14.120 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.120 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.120 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.120 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:14.121 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944978+0000", 2024-09-21T21:42:14.121 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.121 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.121 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.121 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:14.121 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944978+0000", 2024-09-21T21:42:14.121 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9100000000000004e-07 2024-09-21T21:42:14.121 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.121 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.121 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:14.121 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944981+0000", 2024-09-21T21:42:14.121 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3290000000000001e-06 2024-09-21T21:42:14.121 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.121 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.121 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:14.121 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944987+0000", 2024-09-21T21:42:14.122 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.28e-06 2024-09-21T21:42:14.122 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.122 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.122 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:14.122 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944992+0000", 2024-09-21T21:42:14.122 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8520000000000003e-06 2024-09-21T21:42:14.122 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.122 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.122 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:14.122 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024196+0000", 2024-09-21T21:42:14.122 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079203673000000002 2024-09-21T21:42:14.122 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.122 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.122 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:14.123 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024211+0000", 2024-09-21T21:42:14.123 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.4980000000000001e-05 2024-09-21T21:42:14.123 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.123 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:14.123 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.123 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.123 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.123 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13638 3.12 3.1dc31b72 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:14.123 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.945000+0000", 2024-09-21T21:42:14.123 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 23.101783407999999, 2024-09-21T21:42:14.123 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079257465999999999, 2024-09-21T21:42:14.123 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:14.123 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:14.123 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:42:14.123 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:14.123 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:14.124 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:14.124 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13638 2024-09-21T21:42:14.124 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.124 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:14.124 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.124 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:14.124 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945000+0000", 2024-09-21T21:42:14.124 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.124 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.124 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.124 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:14.124 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945000+0000", 2024-09-21T21:42:14.124 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.124 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.124 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.124 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:14.125 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945001+0000", 2024-09-21T21:42:14.125 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.32e-07 2024-09-21T21:42:14.125 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.125 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.125 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:14.125 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945003+0000", 2024-09-21T21:42:14.125 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2309999999999999e-06 2024-09-21T21:42:14.125 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.125 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.125 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:14.125 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945009+0000", 2024-09-21T21:42:14.125 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7450000000000001e-06 2024-09-21T21:42:14.125 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.125 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.125 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:14.125 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945013+0000", 2024-09-21T21:42:14.125 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.673e-06 2024-09-21T21:42:14.126 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.126 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.126 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:14.126 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024245+0000", 2024-09-21T21:42:14.126 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079232160999999995 2024-09-21T21:42:14.126 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.126 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.126 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:14.126 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024258+0000", 2024-09-21T21:42:14.126 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.2224e-05 2024-09-21T21:42:14.126 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.126 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:14.126 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.126 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.126 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.127 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13639 3.12 3.1dc31b72 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:14.127 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.945034+0000", 2024-09-21T21:42:14.127 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 23.101749608999999, 2024-09-21T21:42:14.127 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079260627, 2024-09-21T21:42:14.127 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:14.127 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:14.127 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:42:14.127 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:14.127 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:14.127 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:14.127 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13639 2024-09-21T21:42:14.127 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.127 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:14.127 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.128 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:14.128 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945034+0000", 2024-09-21T21:42:14.128 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.128 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.128 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.128 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:14.128 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945034+0000", 2024-09-21T21:42:14.128 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.128 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.128 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.128 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:14.128 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945034+0000", 2024-09-21T21:42:14.129 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3799999999999998e-07 2024-09-21T21:42:14.129 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.129 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.129 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:14.129 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945036+0000", 2024-09-21T21:42:14.129 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.052e-06 2024-09-21T21:42:14.129 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.129 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.129 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:14.129 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945040+0000", 2024-09-21T21:42:14.129 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1169999999999997e-06 2024-09-21T21:42:14.129 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.129 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.129 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:14.129 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945043+0000", 2024-09-21T21:42:14.130 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.8439999999999998e-06 2024-09-21T21:42:14.130 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.130 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.130 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:14.130 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024286+0000", 2024-09-21T21:42:14.130 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079242483000000002 2024-09-21T21:42:14.130 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.130 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.130 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:14.130 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024295+0000", 2024-09-21T21:42:14.130 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.7930000000000003e-06 2024-09-21T21:42:14.130 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.130 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:14.130 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.130 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.131 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.131 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13640 3.12 3.1dc31b72 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:14.131 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.945047+0000", 2024-09-21T21:42:14.131 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 23.101736086999999, 2024-09-21T21:42:14.131 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079412321999999994, 2024-09-21T21:42:14.131 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:14.131 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:14.131 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:42:14.131 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:14.131 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:14.131 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:14.131 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13640 2024-09-21T21:42:14.131 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.131 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:14.131 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.131 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:14.132 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945047+0000", 2024-09-21T21:42:14.132 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.132 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.132 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.132 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:14.132 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945047+0000", 2024-09-21T21:42:14.132 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.132 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.132 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.132 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:14.132 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945048+0000", 2024-09-21T21:42:14.132 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0499999999999999e-07 2024-09-21T21:42:14.132 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.132 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.132 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:14.133 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945049+0000", 2024-09-21T21:42:14.133 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.3939999999999999e-06 2024-09-21T21:42:14.133 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.133 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.133 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:14.133 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945053+0000", 2024-09-21T21:42:14.133 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1760000000000003e-06 2024-09-21T21:42:14.133 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.133 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.133 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:14.133 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945056+0000", 2024-09-21T21:42:14.133 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.6129999999999998e-06 2024-09-21T21:42:14.133 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.133 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.133 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:14.134 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024360+0000", 2024-09-21T21:42:14.134 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079303645000000006 2024-09-21T21:42:14.134 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.134 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.134 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:14.134 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024460+0000", 2024-09-21T21:42:14.134 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000100189 2024-09-21T21:42:14.134 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.134 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:14.134 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.134 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.134 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:14.134 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:14.136 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T21:42:14.136 INFO:teuthology.orchestra.run.smithi164.stdout:4919 write 0x8309c00 thru 0x83149c8 (0xadc9 bytes) 2024-09-21T21:42:14.136 INFO:teuthology.orchestra.run.smithi164.stdout:4922 read 0x4d0ec00 thru 0x4d1a3c9 (0xb7ca bytes) 2024-09-21T21:42:14.136 INFO:teuthology.orchestra.run.smithi164.stdout:4923 read 0x5cca400 thru 0x5cce57b (0x417c bytes) 2024-09-21T21:42:14.136 INFO:teuthology.orchestra.run.smithi164.stdout:4926 read 0x230a00 thru 0x23d8a2 (0xcea3 bytes) 2024-09-21T21:42:14.137 INFO:teuthology.orchestra.run.smithi164.stdout:4927 punch from 0x3e82400 to 0x3e8a000, (0x7c00 bytes) 2024-09-21T21:42:14.137 INFO:teuthology.orchestra.run.smithi164.stdout:4930 write 0xd38a800 thru 0xd38dbc4 (0x33c5 bytes) 2024-09-21T21:42:14.137 INFO:teuthology.orchestra.run.smithi164.stdout:4931 trunc from 0xd38dc00 to 0x2186200 2024-09-21T21:42:14.145 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:14.143+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:14.158 INFO:teuthology.orchestra.run.smithi164.stdout:4933 read 0x510600 thru 0x512672 (0x2073 bytes) 2024-09-21T21:42:14.159 INFO:teuthology.orchestra.run.smithi164.stdout:4934 read 0x2ef800 thru 0x2f9153 (0x9954 bytes) 2024-09-21T21:42:14.160 INFO:teuthology.orchestra.run.smithi164.stdout:4935 read 0x1c24a00 thru 0x1c2e438 (0x9a39 bytes) 2024-09-21T21:42:14.161 INFO:teuthology.orchestra.run.smithi164.stdout:4936 read 0x77b000 thru 0x78a9ad (0xf9ae bytes) 2024-09-21T21:42:14.161 INFO:teuthology.orchestra.run.smithi164.stdout:4937 read 0x1b63000 thru 0x1b6e187 (0xb188 bytes) 2024-09-21T21:42:14.173 INFO:teuthology.orchestra.run.smithi164.stdout:4941 write 0x23e0800 thru 0x23ea4eb (0x9cec bytes) 2024-09-21T21:42:14.209 INFO:teuthology.orchestra.run.smithi164.stdout:4944 write 0x4653600 thru 0x4662882 (0xf283 bytes) 2024-09-21T21:42:14.209 INFO:teuthology.orchestra.run.smithi164.stdout:4946 write 0x27edc00 thru 0x27f8cc5 (0xb0c6 bytes) 2024-09-21T21:42:14.210 INFO:teuthology.orchestra.run.smithi164.stdout:4947 punch from 0xc80000 to 0xc8ae00, (0xae00 bytes) 2024-09-21T21:42:14.210 INFO:teuthology.orchestra.run.smithi164.stdout:4948 read 0x2c82200 thru 0x2c83982 (0x1783 bytes) 2024-09-21T21:42:14.210 INFO:teuthology.orchestra.run.smithi164.stdout:4950 punch from 0x404600 to 0x411000, (0xca00 bytes) 2024-09-21T21:42:14.227 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T21:42:14.244 INFO:teuthology.orchestra.run.smithi164.stdout:4953 write 0xbf6d800 thru 0xbf7754a (0x9d4b bytes) 2024-09-21T21:42:14.244 INFO:teuthology.orchestra.run.smithi164.stdout:4955 punch from 0xa0aa600 to 0xa0ad600, (0x3000 bytes) 2024-09-21T21:42:14.244 INFO:teuthology.orchestra.run.smithi164.stdout:4964 punch from 0x1db7400 to 0x1dbf000, (0x7c00 bytes) 2024-09-21T21:42:14.244 INFO:teuthology.orchestra.run.smithi164.stdout:4965 write 0xe5da00 thru 0xe6261f (0x4c20 bytes) 2024-09-21T21:42:14.245 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:14.244+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:14.246 INFO:teuthology.orchestra.run.smithi164.stdout:4966 read 0x24df600 thru 0x24e8e1e (0x981f bytes) 2024-09-21T21:42:14.262 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:14.262 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:42:14.262 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.262 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:11788 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:14.262 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:14.244461+0000", 2024-09-21T21:42:14.263 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.016131804999999999, 2024-09-21T21:42:14.263 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.016155775000000001, 2024-09-21T21:42:14.263 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:14.263 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:14.263 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "waiting for sub ops", 2024-09-21T21:42:14.263 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:14.263 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:14.263 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:14.263 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11788 2024-09-21T21:42:14.263 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.263 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:14.263 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.263 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:14.263 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.244461+0000", 2024-09-21T21:42:14.263 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.263 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.264 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.264 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:14.264 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.244461+0000", 2024-09-21T21:42:14.264 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.264 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.264 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.264 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:14.264 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.244461+0000", 2024-09-21T21:42:14.264 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9900000000000001e-07 2024-09-21T21:42:14.264 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.264 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.265 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:14.265 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.244464+0000", 2024-09-21T21:42:14.265 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.157e-06 2024-09-21T21:42:14.265 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.265 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.265 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:14.265 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.244473+0000", 2024-09-21T21:42:14.265 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.054e-06 2024-09-21T21:42:14.265 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.265 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.265 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:14.265 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.244486+0000", 2024-09-21T21:42:14.265 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.2656e-05 2024-09-21T21:42:14.265 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.266 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.266 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:14.266 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.258720+0000", 2024-09-21T21:42:14.266 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.014234083 2024-09-21T21:42:14.266 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.266 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.266 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:14.266 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.258788+0000", 2024-09-21T21:42:14.266 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.8065000000000006e-05 2024-09-21T21:42:14.266 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.266 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.266 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:14.266 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.259053+0000", 2024-09-21T21:42:14.266 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00026528800000000003 2024-09-21T21:42:14.267 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.267 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:14.267 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.267 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.267 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.267 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:11789 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:14.267 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:14.244496+0000", 2024-09-21T21:42:14.267 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.016096927, 2024-09-21T21:42:14.267 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.016185675, 2024-09-21T21:42:14.267 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:14.267 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:14.267 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "waiting for sub ops", 2024-09-21T21:42:14.267 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:14.267 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:14.267 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:14.267 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11789 2024-09-21T21:42:14.268 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.268 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:14.268 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.268 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:14.268 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.244496+0000", 2024-09-21T21:42:14.268 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.268 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.268 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.268 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:14.268 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.244496+0000", 2024-09-21T21:42:14.268 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.268 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.268 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.268 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:14.268 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.244496+0000", 2024-09-21T21:42:14.269 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4900000000000001e-07 2024-09-21T21:42:14.269 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.269 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.269 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:14.269 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.244499+0000", 2024-09-21T21:42:14.269 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0970000000000002e-06 2024-09-21T21:42:14.269 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.269 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.269 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:14.269 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.244506+0000", 2024-09-21T21:42:14.269 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6460000000000004e-06 2024-09-21T21:42:14.269 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.269 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.269 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:14.269 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.244510+0000", 2024-09-21T21:42:14.270 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5900000000000001e-06 2024-09-21T21:42:14.270 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.270 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.270 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:14.270 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.259722+0000", 2024-09-21T21:42:14.270 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.015211489 2024-09-21T21:42:14.270 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.270 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.270 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:14.270 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.259784+0000", 2024-09-21T21:42:14.270 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1902000000000005e-05 2024-09-21T21:42:14.270 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.270 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.270 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:14.270 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.260035+0000", 2024-09-21T21:42:14.271 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00025145199999999999 2024-09-21T21:42:14.271 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.271 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:14.271 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.271 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.271 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.271 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:11790 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:14.271 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:14.244517+0000", 2024-09-21T21:42:14.271 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.016075376999999998, 2024-09-21T21:42:14.271 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.016242013999999999, 2024-09-21T21:42:14.271 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:14.271 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:14.271 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:14.271 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:14.271 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:14.271 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:14.272 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11790 2024-09-21T21:42:14.272 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.272 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:14.272 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.272 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:14.272 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.244517+0000", 2024-09-21T21:42:14.272 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.272 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.272 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.272 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:14.272 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.244517+0000", 2024-09-21T21:42:14.272 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.272 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.272 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.273 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:14.273 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.244517+0000", 2024-09-21T21:42:14.273 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5600000000000001e-07 2024-09-21T21:42:14.273 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.273 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.273 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:14.273 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.244520+0000", 2024-09-21T21:42:14.273 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.785e-06 2024-09-21T21:42:14.273 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.273 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.273 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:14.273 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.244528+0000", 2024-09-21T21:42:14.273 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.3259999999999998e-06 2024-09-21T21:42:14.273 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.274 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.274 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:14.274 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.244535+0000", 2024-09-21T21:42:14.274 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0090000000000001e-06 2024-09-21T21:42:14.274 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.274 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.274 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:14.274 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.260436+0000", 2024-09-21T21:42:14.274 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.015901530000000001 2024-09-21T21:42:14.274 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.274 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.274 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:14.274 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.260552+0000", 2024-09-21T21:42:14.274 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000116011 2024-09-21T21:42:14.274 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.275 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:14.275 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.275 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.275 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.275 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:11796 3.6 3.6009b686 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:14.275 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:14.244543+0000", 2024-09-21T21:42:14.275 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.01604937, 2024-09-21T21:42:14.275 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.016268004999999999, 2024-09-21T21:42:14.275 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:14.275 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:14.275 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:14.275 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:14.275 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:14.275 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:14.275 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11796 2024-09-21T21:42:14.275 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.276 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:14.276 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.276 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:14.276 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.244543+0000", 2024-09-21T21:42:14.276 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.276 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.276 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.276 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:14.276 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.244543+0000", 2024-09-21T21:42:14.276 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.276 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.276 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.276 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:14.276 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.244544+0000", 2024-09-21T21:42:14.276 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3300000000000002e-07 2024-09-21T21:42:14.276 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.277 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.277 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:14.277 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.244547+0000", 2024-09-21T21:42:14.277 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.977e-06 2024-09-21T21:42:14.277 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.277 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.277 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:14.277 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.244554+0000", 2024-09-21T21:42:14.277 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0509999999999999e-06 2024-09-21T21:42:14.277 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.277 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.277 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:14.277 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.244559+0000", 2024-09-21T21:42:14.277 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9099999999999996e-06 2024-09-21T21:42:14.277 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.277 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:14.278 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.278 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.278 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.278 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:11878 3.6 3.ca259566 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:14.278 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:14.254213+0000", 2024-09-21T21:42:14.278 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.0063790019999999999, 2024-09-21T21:42:14.278 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0066272170000000004, 2024-09-21T21:42:14.278 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:14.278 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:14.278 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:14.278 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:14.278 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:14.278 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:14.278 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11878 2024-09-21T21:42:14.278 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.279 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:14.279 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.279 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:14.279 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.254213+0000", 2024-09-21T21:42:14.279 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.279 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.279 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.279 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:14.279 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.254213+0000", 2024-09-21T21:42:14.279 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.279 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.279 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.279 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:14.279 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.254214+0000", 2024-09-21T21:42:14.279 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8999999999999996e-07 2024-09-21T21:42:14.280 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.280 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.280 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:14.280 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.254217+0000", 2024-09-21T21:42:14.280 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.1530000000000001e-06 2024-09-21T21:42:14.280 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.280 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.280 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:14.280 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.254225+0000", 2024-09-21T21:42:14.280 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.4970000000000004e-06 2024-09-21T21:42:14.280 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.280 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.280 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:14.280 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.254231+0000", 2024-09-21T21:42:14.280 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0759999999999999e-06 2024-09-21T21:42:14.280 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.280 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:14.281 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.281 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.281 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.281 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:11806 3.6 3.1ffb7126 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:14.281 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:14.246129+0000", 2024-09-21T21:42:14.281 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.014463699999999999, 2024-09-21T21:42:14.281 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.014738975, 2024-09-21T21:42:14.281 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:14.281 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:14.281 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:14.281 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:14.281 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:14.281 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:14.281 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11806 2024-09-21T21:42:14.281 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.281 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:14.282 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.282 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:14.282 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.246129+0000", 2024-09-21T21:42:14.282 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.282 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.282 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.282 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:14.282 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.246129+0000", 2024-09-21T21:42:14.282 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.282 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.282 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.282 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:14.282 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.246130+0000", 2024-09-21T21:42:14.282 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.47e-07 2024-09-21T21:42:14.282 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.283 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.283 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:14.283 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.246135+0000", 2024-09-21T21:42:14.283 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2259999999999996e-06 2024-09-21T21:42:14.283 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.283 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.283 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:14.283 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.246155+0000", 2024-09-21T21:42:14.283 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9857999999999999e-05 2024-09-21T21:42:14.283 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.283 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.283 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:14.283 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.246164+0000", 2024-09-21T21:42:14.283 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.0119999999999994e-06 2024-09-21T21:42:14.283 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.283 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:14.283 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.284 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.284 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.284 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:11880 3.b 3:d3441193:::rbd_data.1121faebdfe3.00000000000000c3:head [set-alloc-hint object_size 1048576 write_size 1048576,write 149504~3584 in=3584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:14.284 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:14.256363+0000", 2024-09-21T21:42:14.284 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.0042290799999999996, 2024-09-21T21:42:14.284 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0045360899999999996, 2024-09-21T21:42:14.284 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:14.284 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:14.284 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:14.284 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:14.284 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:14.284 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:14.284 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11880 2024-09-21T21:42:14.284 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.284 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:14.285 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.285 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:14.285 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.256363+0000", 2024-09-21T21:42:14.285 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.285 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.285 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.285 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:14.285 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.256363+0000", 2024-09-21T21:42:14.285 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.285 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.285 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.285 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:14.285 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.256364+0000", 2024-09-21T21:42:14.285 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2500000000000004e-07 2024-09-21T21:42:14.285 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.285 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.285 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:14.286 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.256371+0000", 2024-09-21T21:42:14.286 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.7309999999999996e-06 2024-09-21T21:42:14.286 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.286 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.286 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:14.286 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.256381+0000", 2024-09-21T21:42:14.286 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0319000000000001e-05 2024-09-21T21:42:14.286 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.286 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.286 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:14.286 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.256386+0000", 2024-09-21T21:42:14.286 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8289999999999998e-06 2024-09-21T21:42:14.286 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.286 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.286 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:14.286 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.256419+0000", 2024-09-21T21:42:14.287 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3141e-05 2024-09-21T21:42:14.287 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.287 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.287 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:14.287 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.256486+0000", 2024-09-21T21:42:14.287 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6808000000000001e-05 2024-09-21T21:42:14.287 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.287 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.287 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:14.287 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.256636+0000", 2024-09-21T21:42:14.287 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00014994699999999999 2024-09-21T21:42:14.287 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.287 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.287 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:14.287 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.260437+0000", 2024-09-21T21:42:14.287 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.003800838 2024-09-21T21:42:14.288 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.288 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.288 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:14.288 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.260802+0000", 2024-09-21T21:42:14.288 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00036486599999999999 2024-09-21T21:42:14.288 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.288 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.288 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:14.288 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.260858+0000", 2024-09-21T21:42:14.288 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6289999999999998e-05 2024-09-21T21:42:14.288 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.288 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:14.288 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.288 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.288 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.289 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_repop_reply(client.5626.0:11785 3.6 e98/97)", 2024-09-21T21:42:14.289 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:14.260658+0000", 2024-09-21T21:42:14.289 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 4294967295.9999352, 2024-09-21T21:42:14.289 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000298776, 2024-09-21T21:42:14.289 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:14.289 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:14.289 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:14.289 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:14.289 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.289 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:14.289 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.260658+0000", 2024-09-21T21:42:14.289 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.289 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.289 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.289 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:14.290 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.260658+0000", 2024-09-21T21:42:14.290 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.290 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.290 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.290 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:14.290 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.260658+0000", 2024-09-21T21:42:14.290 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4700000000000001e-07 2024-09-21T21:42:14.290 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.290 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.290 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:14.290 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.260662+0000", 2024-09-21T21:42:14.290 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8970000000000001e-06 2024-09-21T21:42:14.290 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.290 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.290 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:14.290 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.260676+0000", 2024-09-21T21:42:14.291 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.3594000000000001e-05 2024-09-21T21:42:14.291 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.291 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.291 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:14.291 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.260682+0000", 2024-09-21T21:42:14.291 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.251e-06 2024-09-21T21:42:14.291 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.291 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:14.291 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.291 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.291 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.291 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "MOSDPGRecoveryDeleteReply(3.6 e98,97 3:626ea3bb:::rbd_data.1121ae2b432e.0000000000000370:head,88'234)", 2024-09-21T21:42:14.291 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:14.260692+0000", 2024-09-21T21:42:14.291 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 4294967295.9999003, 2024-09-21T21:42:14.291 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00028939900000000003, 2024-09-21T21:42:14.291 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:14.292 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:14.292 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:14.292 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:14.292 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.292 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:14.292 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.260692+0000", 2024-09-21T21:42:14.292 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.292 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.292 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.292 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:14.292 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.260692+0000", 2024-09-21T21:42:14.292 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.292 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.292 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.292 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:14.293 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.260692+0000", 2024-09-21T21:42:14.293 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.7000000000000001e-07 2024-09-21T21:42:14.293 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.293 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.293 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:14.293 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.260696+0000", 2024-09-21T21:42:14.293 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4120000000000001e-06 2024-09-21T21:42:14.293 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.293 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.293 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:14.293 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.260703+0000", 2024-09-21T21:42:14.293 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5499999999999997e-06 2024-09-21T21:42:14.293 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.293 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.293 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:14.293 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.260709+0000", 2024-09-21T21:42:14.293 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9159999999999998e-06 2024-09-21T21:42:14.294 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.294 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:14.294 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.294 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.294 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.294 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_repop_reply(client.5626.0:11786 3.6 e98/97)", 2024-09-21T21:42:14.294 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:14.260747+0000", 2024-09-21T21:42:14.294 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 4294967295.999846, 2024-09-21T21:42:14.294 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00025753499999999998, 2024-09-21T21:42:14.294 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:14.294 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:14.294 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:14.294 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:14.294 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.294 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:14.295 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.260747+0000", 2024-09-21T21:42:14.295 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.295 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.295 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.295 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:14.295 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.260747+0000", 2024-09-21T21:42:14.295 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.295 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.295 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.295 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:14.295 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.260747+0000", 2024-09-21T21:42:14.295 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.99e-07 2024-09-21T21:42:14.295 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.295 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.296 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:14.296 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.260749+0000", 2024-09-21T21:42:14.296 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.95e-06 2024-09-21T21:42:14.296 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.296 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.296 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:14.296 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.260755+0000", 2024-09-21T21:42:14.296 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.93e-06 2024-09-21T21:42:14.296 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.296 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.296 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:14.296 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.260760+0000", 2024-09-21T21:42:14.296 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7990000000000001e-06 2024-09-21T21:42:14.296 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.297 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:14.297 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.297 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.297 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.297 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:11854 3.6 3.ca259566 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:14.297 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:14.251314+0000", 2024-09-21T21:42:14.297 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.009278138, 2024-09-21T21:42:14.297 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0097140660000000004, 2024-09-21T21:42:14.297 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:14.297 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:14.297 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:14.297 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:14.297 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:14.297 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:14.297 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11854 2024-09-21T21:42:14.298 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.298 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:14.298 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.298 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:14.298 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.251314+0000", 2024-09-21T21:42:14.298 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.298 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.298 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.298 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:14.298 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.251314+0000", 2024-09-21T21:42:14.298 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.298 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.298 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.298 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:14.298 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.251315+0000", 2024-09-21T21:42:14.299 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2299999999999999e-07 2024-09-21T21:42:14.299 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.299 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.299 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:14.299 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.251324+0000", 2024-09-21T21:42:14.299 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.4160000000000004e-06 2024-09-21T21:42:14.299 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.299 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.299 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:14.299 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.251334+0000", 2024-09-21T21:42:14.299 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0288e-05 2024-09-21T21:42:14.299 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.299 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.299 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:14.300 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.251341+0000", 2024-09-21T21:42:14.300 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.165e-06 2024-09-21T21:42:14.300 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.300 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:14.300 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.300 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.300 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.300 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:11785 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:14.300 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:14.244220+0000", 2024-09-21T21:42:14.300 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.016372991999999999, 2024-09-21T21:42:14.300 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.016839165999999999, 2024-09-21T21:42:14.300 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:14.300 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:14.300 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "waiting for sub ops", 2024-09-21T21:42:14.301 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:14.301 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:14.301 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:14.301 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11785 2024-09-21T21:42:14.301 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.301 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:14.301 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.301 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:14.301 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.244220+0000", 2024-09-21T21:42:14.301 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.301 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.301 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.301 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:14.301 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.244220+0000", 2024-09-21T21:42:14.301 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.302 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.302 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.302 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:14.302 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.244220+0000", 2024-09-21T21:42:14.302 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3599999999999999e-07 2024-09-21T21:42:14.302 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.302 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.302 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:14.302 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.244222+0000", 2024-09-21T21:42:14.302 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.6510000000000001e-06 2024-09-21T21:42:14.302 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.302 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.303 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:14.303 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.244368+0000", 2024-09-21T21:42:14.303 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000145761 2024-09-21T21:42:14.303 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.303 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.303 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:14.303 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.244380+0000", 2024-09-21T21:42:14.303 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1868999999999999e-05 2024-09-21T21:42:14.303 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.303 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.303 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:14.303 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.255190+0000", 2024-09-21T21:42:14.303 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.010809569999999999 2024-09-21T21:42:14.303 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.303 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.304 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:14.304 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.255253+0000", 2024-09-21T21:42:14.304 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3409000000000002e-05 2024-09-21T21:42:14.304 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.304 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.304 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:14.304 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.255522+0000", 2024-09-21T21:42:14.304 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00026895700000000001 2024-09-21T21:42:14.304 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.304 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.304 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:14.304 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.257543+0000", 2024-09-21T21:42:14.305 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0020209719999999998 2024-09-21T21:42:14.305 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.305 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:14.305 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.305 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.305 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.305 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:11786 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:14.305 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:14.244399+0000", 2024-09-21T21:42:14.305 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.016193362999999999, 2024-09-21T21:42:14.305 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.016712478999999999, 2024-09-21T21:42:14.305 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:14.305 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:14.305 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "waiting for sub ops", 2024-09-21T21:42:14.305 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:14.306 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:14.306 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:14.306 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11786 2024-09-21T21:42:14.306 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.306 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:14.306 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.306 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:14.306 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.244399+0000", 2024-09-21T21:42:14.306 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.306 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.306 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.306 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:14.306 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.244399+0000", 2024-09-21T21:42:14.306 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.307 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.307 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.307 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:14.307 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.244400+0000", 2024-09-21T21:42:14.307 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5899999999999995e-07 2024-09-21T21:42:14.307 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.307 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.307 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:14.307 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.244404+0000", 2024-09-21T21:42:14.307 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2950000000000004e-06 2024-09-21T21:42:14.307 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.307 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.307 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:14.308 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.244413+0000", 2024-09-21T21:42:14.308 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2629999999999992e-06 2024-09-21T21:42:14.308 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.308 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.308 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:14.308 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.244422+0000", 2024-09-21T21:42:14.308 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.4500000000000004e-06 2024-09-21T21:42:14.308 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.308 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.308 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:14.308 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.256416+0000", 2024-09-21T21:42:14.308 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.011994237 2024-09-21T21:42:14.308 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.308 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.309 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:14.309 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.256525+0000", 2024-09-21T21:42:14.309 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010849900000000001 2024-09-21T21:42:14.309 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.309 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.309 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:14.309 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.256974+0000", 2024-09-21T21:42:14.309 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00044899499999999999 2024-09-21T21:42:14.309 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.309 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.309 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:14.309 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.259700+0000", 2024-09-21T21:42:14.309 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0027265980000000002 2024-09-21T21:42:14.309 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.309 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:14.309 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.310 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.310 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.310 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:11787 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:14.310 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:14.244430+0000", 2024-09-21T21:42:14.310 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.016162309, 2024-09-21T21:42:14.310 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.016731729000000001, 2024-09-21T21:42:14.310 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:14.310 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:14.310 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "waiting for sub ops", 2024-09-21T21:42:14.310 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:14.310 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:14.310 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:14.310 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11787 2024-09-21T21:42:14.310 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.310 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:14.311 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.311 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:14.311 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.244430+0000", 2024-09-21T21:42:14.311 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.311 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.311 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.311 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:14.311 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.244430+0000", 2024-09-21T21:42:14.311 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.311 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.311 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.311 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:14.311 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.244431+0000", 2024-09-21T21:42:14.311 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6199999999999998e-07 2024-09-21T21:42:14.311 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.311 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.312 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:14.312 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.244434+0000", 2024-09-21T21:42:14.312 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.2430000000000001e-06 2024-09-21T21:42:14.312 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.312 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.312 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:14.312 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.244443+0000", 2024-09-21T21:42:14.312 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.7539999999999995e-06 2024-09-21T21:42:14.312 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.312 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.312 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:14.312 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.244448+0000", 2024-09-21T21:42:14.312 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0629999999999996e-06 2024-09-21T21:42:14.312 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.312 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.313 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:14.313 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.257577+0000", 2024-09-21T21:42:14.313 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.013128809999999999 2024-09-21T21:42:14.313 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.313 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.313 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:14.313 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.257702+0000", 2024-09-21T21:42:14.313 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00012507299999999999 2024-09-21T21:42:14.313 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.313 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.313 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:14.313 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.258161+0000", 2024-09-21T21:42:14.313 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000458815 2024-09-21T21:42:14.313 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.313 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:14.313 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.314 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.314 INFO:teuthology.orchestra.run.smithi081.stdout: ], 2024-09-21T21:42:14.314 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 14 2024-09-21T21:42:14.314 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:14.315 INFO:teuthology.orchestra.run.smithi164.stdout:4967 punch from 0xb4e5800 to 0xb4f4000, (0xe800 bytes) 2024-09-21T21:42:14.315 INFO:teuthology.orchestra.run.smithi164.stdout:4973 read 0x7ae6c00 thru 0x7ae89b6 (0x1db7 bytes) 2024-09-21T21:42:14.315 INFO:teuthology.orchestra.run.smithi164.stdout:4974 trunc from 0xbf77600 to 0xc71bc00 2024-09-21T21:42:14.315 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T21:42:14.316 INFO:teuthology.orchestra.run.smithi164.stdout:4975 write 0x315e200 thru 0x3161c15 (0x3a16 bytes) 2024-09-21T21:42:14.316 INFO:teuthology.orchestra.run.smithi164.stdout:4977 punch from 0x4405e00 to 0x440be00, (0x6000 bytes) 2024-09-21T21:42:14.316 INFO:teuthology.orchestra.run.smithi164.stdout:4978 punch from 0xd79000 to 0xd7f600, (0x6600 bytes) 2024-09-21T21:42:14.317 INFO:teuthology.orchestra.run.smithi164.stdout:4979 punch from 0x5713200 to 0x5720c00, (0xda00 bytes) 2024-09-21T21:42:14.317 INFO:teuthology.orchestra.run.smithi164.stdout:4981 write 0x42eee00 thru 0x42f1ea2 (0x30a3 bytes) 2024-09-21T21:42:14.317 INFO:teuthology.orchestra.run.smithi164.stdout:4982 punch from 0x6ed5c00 to 0x6ed8c00, (0x3000 bytes) 2024-09-21T21:42:14.329 INFO:teuthology.orchestra.run.smithi164.stdout:4984 trunc from 0xc71bc00 to 0x2cb9800 2024-09-21T21:42:14.356 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:14.355+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:14.421 INFO:teuthology.orchestra.run.smithi164.stdout:4985 trunc from 0x2cb9800 to 0xed8e600 2024-09-21T21:42:14.447 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:14.446+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:14.473 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:14.473 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:42:14.473 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:42:14.473 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:42:14.473 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:14.482 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T21:42:14.566 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:14.565+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:14.628 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:14.629 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 20, 2024-09-21T21:42:14.629 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 600, 2024-09-21T21:42:14.629 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:42:14.629 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.629 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4824.0:1564 3.2 3:4647520c:::rbd_data.1121b97f2925.0000000000000006:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:42:14.629 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:42:14.629 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 72.990157171999996, 2024-09-21T21:42:14.629 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056097347999999998, 2024-09-21T21:42:14.629 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:14.629 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:14.629 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:14.629 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:14.629 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4824", 2024-09-21T21:42:14.630 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:42:14.630 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1564 2024-09-21T21:42:14.630 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.630 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:14.630 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.630 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:14.630 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:42:14.630 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.630 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.630 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.630 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:14.630 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:42:14.630 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.631 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.631 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.631 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:14.631 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:42:14.631 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6499999999999999e-07 2024-09-21T21:42:14.631 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.631 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.631 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:14.631 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636067+0000", 2024-09-21T21:42:14.631 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3879999999999998e-06 2024-09-21T21:42:14.631 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.631 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.631 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:14.632 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636075+0000", 2024-09-21T21:42:14.632 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5480000000000004e-06 2024-09-21T21:42:14.632 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.632 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.632 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:14.632 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636080+0000", 2024-09-21T21:42:14.632 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.339e-06 2024-09-21T21:42:14.632 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.632 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.632 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:14.632 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636133+0000", 2024-09-21T21:42:14.632 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2930000000000003e-05 2024-09-21T21:42:14.632 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.632 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.633 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:14.633 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636265+0000", 2024-09-21T21:42:14.633 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000132058 2024-09-21T21:42:14.633 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.633 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.633 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:14.633 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.692162+0000", 2024-09-21T21:42:14.633 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055896620000000001 2024-09-21T21:42:14.633 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.633 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:14.633 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.633 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.633 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.633 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3259 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 663552~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:42:14.633 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.196275+0000", 2024-09-21T21:42:14.634 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 27.429947024000001, 2024-09-21T21:42:14.634 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055889580000000001, 2024-09-21T21:42:14.634 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:14.634 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:14.634 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:14.634 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:14.634 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:14.634 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:14.634 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3259 2024-09-21T21:42:14.634 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.634 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:14.634 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.634 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:14.634 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196275+0000", 2024-09-21T21:42:14.635 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.635 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.635 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.635 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:14.635 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196275+0000", 2024-09-21T21:42:14.635 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.635 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.635 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.635 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:14.635 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196275+0000", 2024-09-21T21:42:14.635 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2199999999999999e-07 2024-09-21T21:42:14.635 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.635 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.635 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:14.636 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196277+0000", 2024-09-21T21:42:14.636 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1270000000000001e-06 2024-09-21T21:42:14.636 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.636 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.636 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:14.636 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196285+0000", 2024-09-21T21:42:14.636 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.2290000000000003e-06 2024-09-21T21:42:14.636 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.636 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.636 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:14.636 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196291+0000", 2024-09-21T21:42:14.636 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1379999999999998e-06 2024-09-21T21:42:14.636 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.636 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.637 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:14.637 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.251997+0000", 2024-09-21T21:42:14.637 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055705971999999999 2024-09-21T21:42:14.637 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.637 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.637 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:14.637 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252077+0000", 2024-09-21T21:42:14.637 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0179999999999995e-05 2024-09-21T21:42:14.637 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.637 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.637 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:14.637 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252164+0000", 2024-09-21T21:42:14.637 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.7511999999999998e-05 2024-09-21T21:42:14.637 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.638 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:14.638 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.638 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.638 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.638 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3260 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 1437696~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:42:14.638 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.196297+0000", 2024-09-21T21:42:14.638 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 27.429924962000001, 2024-09-21T21:42:14.638 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056071022999999998, 2024-09-21T21:42:14.638 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:14.638 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:14.638 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:14.638 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:14.638 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:14.638 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:14.638 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3260 2024-09-21T21:42:14.639 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.639 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:14.639 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.639 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:14.639 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196297+0000", 2024-09-21T21:42:14.639 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.639 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.639 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.639 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:14.639 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196297+0000", 2024-09-21T21:42:14.639 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.639 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.639 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.640 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:14.640 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196918+0000", 2024-09-21T21:42:14.640 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00062148699999999999 2024-09-21T21:42:14.640 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.640 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.640 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:14.640 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196930+0000", 2024-09-21T21:42:14.640 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1657e-05 2024-09-21T21:42:14.640 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.640 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.640 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:14.640 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196944+0000", 2024-09-21T21:42:14.640 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.4549000000000001e-05 2024-09-21T21:42:14.640 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.641 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.641 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:14.641 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196951+0000", 2024-09-21T21:42:14.641 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6749999999999996e-06 2024-09-21T21:42:14.641 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.641 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.641 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:14.641 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252208+0000", 2024-09-21T21:42:14.641 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055256656000000001 2024-09-21T21:42:14.641 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.641 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.641 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:14.641 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252278+0000", 2024-09-21T21:42:14.641 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9882000000000006e-05 2024-09-21T21:42:14.641 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.642 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.642 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:14.642 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252368+0000", 2024-09-21T21:42:14.642 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.0117000000000005e-05 2024-09-21T21:42:14.642 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.642 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:14.642 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.642 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.642 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.642 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3263 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 3559424~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:42:14.642 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.197041+0000", 2024-09-21T21:42:14.642 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 27.429181148000001, 2024-09-21T21:42:14.642 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055935804999999998, 2024-09-21T21:42:14.642 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:14.642 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:14.643 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:14.643 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:14.643 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:14.643 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:14.643 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3263 2024-09-21T21:42:14.643 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.643 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:14.643 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.643 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:14.643 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197041+0000", 2024-09-21T21:42:14.643 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.643 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.643 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.644 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:14.644 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197041+0000", 2024-09-21T21:42:14.644 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.644 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.644 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.644 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:14.644 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197624+0000", 2024-09-21T21:42:14.644 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00058358100000000001 2024-09-21T21:42:14.644 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.644 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.644 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:14.644 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197629+0000", 2024-09-21T21:42:14.644 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5909999999999998e-06 2024-09-21T21:42:14.644 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.645 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.645 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:14.645 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197639+0000", 2024-09-21T21:42:14.645 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0257e-05 2024-09-21T21:42:14.645 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.645 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.645 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:14.645 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197646+0000", 2024-09-21T21:42:14.645 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.7229999999999998e-06 2024-09-21T21:42:14.645 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.645 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.645 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:14.645 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252804+0000", 2024-09-21T21:42:14.645 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055158134999999997 2024-09-21T21:42:14.645 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.646 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.646 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:14.646 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252882+0000", 2024-09-21T21:42:14.646 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7791000000000001e-05 2024-09-21T21:42:14.646 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.646 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.646 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:14.646 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252976+0000", 2024-09-21T21:42:14.646 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.4727000000000003e-05 2024-09-21T21:42:14.646 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.646 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:14.646 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.646 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.647 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.647 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3437 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 925696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:42:14.647 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.205888+0000", 2024-09-21T21:42:14.647 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 27.420333867, 2024-09-21T21:42:14.647 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055953808000000001, 2024-09-21T21:42:14.647 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:14.647 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:14.647 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:14.647 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:14.647 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:14.647 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:14.647 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3437 2024-09-21T21:42:14.647 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.648 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:14.648 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.648 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:14.648 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205888+0000", 2024-09-21T21:42:14.648 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.648 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.648 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.648 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:14.648 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205888+0000", 2024-09-21T21:42:14.648 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.648 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.648 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.648 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:14.648 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205888+0000", 2024-09-21T21:42:14.649 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.75e-07 2024-09-21T21:42:14.649 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.649 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.649 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:14.649 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205891+0000", 2024-09-21T21:42:14.649 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3750000000000001e-06 2024-09-21T21:42:14.649 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.649 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.649 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:14.649 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205897+0000", 2024-09-21T21:42:14.649 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.285e-06 2024-09-21T21:42:14.649 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.649 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.649 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:14.650 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205902+0000", 2024-09-21T21:42:14.650 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6290000000000001e-06 2024-09-21T21:42:14.650 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.650 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.650 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:14.650 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261686+0000", 2024-09-21T21:42:14.650 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055784586999999997 2024-09-21T21:42:14.650 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.650 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.650 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:14.650 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261764+0000", 2024-09-21T21:42:14.650 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7591999999999998e-05 2024-09-21T21:42:14.650 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.650 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.650 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:14.651 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261842+0000", 2024-09-21T21:42:14.651 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7865000000000001e-05 2024-09-21T21:42:14.651 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.651 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:14.651 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.651 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.651 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.651 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3438 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 954368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:42:14.651 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.205908+0000", 2024-09-21T21:42:14.651 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 27.420314002000001, 2024-09-21T21:42:14.651 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056083169000000002, 2024-09-21T21:42:14.651 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:14.651 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:14.651 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:14.651 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:14.652 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:14.652 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:14.652 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3438 2024-09-21T21:42:14.652 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.652 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:14.652 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.652 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:14.652 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205908+0000", 2024-09-21T21:42:14.652 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.652 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.652 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.652 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:14.652 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205908+0000", 2024-09-21T21:42:14.652 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.652 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.652 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.653 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:14.653 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206431+0000", 2024-09-21T21:42:14.653 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00052282100000000003 2024-09-21T21:42:14.653 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.653 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.653 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:14.653 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206442+0000", 2024-09-21T21:42:14.653 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1695000000000001e-05 2024-09-21T21:42:14.653 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.653 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.653 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:14.653 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206452+0000", 2024-09-21T21:42:14.653 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.4560000000000006e-06 2024-09-21T21:42:14.653 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.654 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.654 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:14.654 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206458+0000", 2024-09-21T21:42:14.654 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4670000000000001e-06 2024-09-21T21:42:14.654 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.654 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.654 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:14.654 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261871+0000", 2024-09-21T21:42:14.654 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055412712000000003 2024-09-21T21:42:14.654 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.654 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.654 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:14.654 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261928+0000", 2024-09-21T21:42:14.654 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6631e-05 2024-09-21T21:42:14.654 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.655 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.655 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:14.655 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261991+0000", 2024-09-21T21:42:14.655 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3386999999999995e-05 2024-09-21T21:42:14.655 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.655 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:14.655 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.655 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.655 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.655 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3444 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 2359296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:42:14.655 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.206533+0000", 2024-09-21T21:42:14.655 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 27.419688482000002, 2024-09-21T21:42:14.655 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055974239000000002, 2024-09-21T21:42:14.655 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:14.656 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:14.656 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:14.656 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:14.656 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:14.656 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:14.656 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3444 2024-09-21T21:42:14.656 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.656 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:14.656 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.656 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:14.656 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206533+0000", 2024-09-21T21:42:14.656 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.656 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.657 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.657 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:14.657 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206533+0000", 2024-09-21T21:42:14.657 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.657 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.657 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.657 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:14.657 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206710+0000", 2024-09-21T21:42:14.657 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000176293 2024-09-21T21:42:14.657 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.657 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.657 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:14.657 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206714+0000", 2024-09-21T21:42:14.657 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1989999999999999e-06 2024-09-21T21:42:14.657 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.658 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.658 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:14.658 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206723+0000", 2024-09-21T21:42:14.658 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.9199999999999993e-06 2024-09-21T21:42:14.658 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.658 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.658 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:14.658 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206729+0000", 2024-09-21T21:42:14.658 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0739999999999998e-06 2024-09-21T21:42:14.658 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.658 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.658 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:14.658 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262339+0000", 2024-09-21T21:42:14.658 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055610166000000003 2024-09-21T21:42:14.659 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.659 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.659 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:14.659 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262419+0000", 2024-09-21T21:42:14.659 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0432999999999994e-05 2024-09-21T21:42:14.659 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.659 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.659 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:14.659 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262507+0000", 2024-09-21T21:42:14.659 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.8153999999999997e-05 2024-09-21T21:42:14.659 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.659 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:14.659 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.660 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.660 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.660 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3445 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 1699840~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:42:14.660 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.206746+0000", 2024-09-21T21:42:14.660 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 27.419475557999998, 2024-09-21T21:42:14.660 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055880738999999999, 2024-09-21T21:42:14.660 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:14.660 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:14.660 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:14.660 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:14.660 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:14.660 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:14.660 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3445 2024-09-21T21:42:14.661 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.661 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:14.661 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.661 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:14.661 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206746+0000", 2024-09-21T21:42:14.661 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.661 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.661 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.661 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:14.661 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206746+0000", 2024-09-21T21:42:14.661 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.661 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.661 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.662 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:14.662 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207412+0000", 2024-09-21T21:42:14.662 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00066611999999999997 2024-09-21T21:42:14.662 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.662 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.662 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:14.662 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207417+0000", 2024-09-21T21:42:14.662 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8539999999999996e-06 2024-09-21T21:42:14.662 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.662 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.662 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:14.662 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207437+0000", 2024-09-21T21:42:14.662 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9463e-05 2024-09-21T21:42:14.662 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.663 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.663 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:14.663 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.207445+0000", 2024-09-21T21:42:14.663 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.5790000000000004e-06 2024-09-21T21:42:14.663 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.663 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.663 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:14.663 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262537+0000", 2024-09-21T21:42:14.663 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055091531999999999 2024-09-21T21:42:14.663 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.663 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.663 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:14.663 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262578+0000", 2024-09-21T21:42:14.663 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1220000000000002e-05 2024-09-21T21:42:14.663 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.664 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.664 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:14.664 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262627+0000", 2024-09-21T21:42:14.664 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8971000000000003e-05 2024-09-21T21:42:14.664 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.664 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:14.664 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.664 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.664 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.664 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10362 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:14.664 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840386+0000", 2024-09-21T21:42:14.664 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.785835638, 2024-09-21T21:42:14.664 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056546515999999998, 2024-09-21T21:42:14.664 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:14.665 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:14.665 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:14.665 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:14.665 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:14.665 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:14.665 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10362 2024-09-21T21:42:14.665 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.665 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:14.665 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.665 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:14.665 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840386+0000", 2024-09-21T21:42:14.665 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.665 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.665 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.666 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:14.666 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840386+0000", 2024-09-21T21:42:14.666 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.666 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.666 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.666 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:14.666 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840386+0000", 2024-09-21T21:42:14.666 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.01e-07 2024-09-21T21:42:14.666 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.666 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.666 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:14.666 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840389+0000", 2024-09-21T21:42:14.666 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.9739999999999998e-06 2024-09-21T21:42:14.666 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.667 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.667 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:14.667 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840396+0000", 2024-09-21T21:42:14.667 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2280000000000003e-06 2024-09-21T21:42:14.667 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.667 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.667 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:14.667 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840400+0000", 2024-09-21T21:42:14.667 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4429999999999996e-06 2024-09-21T21:42:14.667 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.667 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.667 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:14.667 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.890896+0000", 2024-09-21T21:42:14.667 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.050495717000000002 2024-09-21T21:42:14.668 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.668 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.668 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:14.668 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.891012+0000", 2024-09-21T21:42:14.668 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011602 2024-09-21T21:42:14.668 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.668 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.668 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:14.668 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.891274+0000", 2024-09-21T21:42:14.668 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00026240200000000001 2024-09-21T21:42:14.668 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.668 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.668 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:14.668 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.894810+0000", 2024-09-21T21:42:14.669 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0035353630000000001 2024-09-21T21:42:14.669 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.669 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.669 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:14.669 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.896818+0000", 2024-09-21T21:42:14.669 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.002008681 2024-09-21T21:42:14.669 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.669 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.669 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:14.669 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.896850+0000", 2024-09-21T21:42:14.669 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.1826999999999999e-05 2024-09-21T21:42:14.669 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.669 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.669 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:14.669 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.896933+0000", 2024-09-21T21:42:14.670 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.2460000000000007e-05 2024-09-21T21:42:14.670 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.670 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:14.670 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.670 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.670 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.670 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10363 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:14.670 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840606+0000", 2024-09-21T21:42:14.670 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.78561612800000002, 2024-09-21T21:42:14.670 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056546011, 2024-09-21T21:42:14.670 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:14.670 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:14.670 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:14.670 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:14.671 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:14.671 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:14.671 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10363 2024-09-21T21:42:14.671 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.671 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:14.671 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.671 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:14.671 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840606+0000", 2024-09-21T21:42:14.671 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.671 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.671 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.671 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:14.671 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840606+0000", 2024-09-21T21:42:14.671 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.672 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.672 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.672 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:14.672 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840606+0000", 2024-09-21T21:42:14.672 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.4700000000000001e-07 2024-09-21T21:42:14.672 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.672 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.672 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:14.672 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840611+0000", 2024-09-21T21:42:14.672 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4649999999999996e-06 2024-09-21T21:42:14.672 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.672 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.672 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:14.673 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840620+0000", 2024-09-21T21:42:14.673 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.0350000000000007e-06 2024-09-21T21:42:14.673 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.673 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.673 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:14.673 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840626+0000", 2024-09-21T21:42:14.673 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7180000000000001e-06 2024-09-21T21:42:14.673 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.673 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.673 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:14.673 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.891665+0000", 2024-09-21T21:42:14.673 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.051039924 2024-09-21T21:42:14.673 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.674 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.674 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:14.674 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.891770+0000", 2024-09-21T21:42:14.674 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010445700000000001 2024-09-21T21:42:14.674 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.674 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.674 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:14.674 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.892147+0000", 2024-09-21T21:42:14.674 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00037731800000000002 2024-09-21T21:42:14.674 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.674 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.674 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:14.674 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.896776+0000", 2024-09-21T21:42:14.674 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0046289759999999999 2024-09-21T21:42:14.674 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.675 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.675 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:14.675 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897003+0000", 2024-09-21T21:42:14.675 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00022641299999999999 2024-09-21T21:42:14.675 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.675 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.675 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:14.675 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897064+0000", 2024-09-21T21:42:14.675 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1571999999999999e-05 2024-09-21T21:42:14.675 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.675 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.675 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:14.675 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897152+0000", 2024-09-21T21:42:14.675 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.7386000000000007e-05 2024-09-21T21:42:14.676 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.676 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:14.676 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.676 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.676 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.676 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10364 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:14.676 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840641+0000", 2024-09-21T21:42:14.676 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.785580308, 2024-09-21T21:42:14.676 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056671527999999999, 2024-09-21T21:42:14.676 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:14.676 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:14.676 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:14.676 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:14.676 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:14.676 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:14.677 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10364 2024-09-21T21:42:14.677 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.677 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:14.677 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.677 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:14.677 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840641+0000", 2024-09-21T21:42:14.677 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.677 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.677 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.677 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:14.677 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840641+0000", 2024-09-21T21:42:14.677 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.677 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.677 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.677 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:14.678 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840642+0000", 2024-09-21T21:42:14.678 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7899999999999999e-07 2024-09-21T21:42:14.678 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.678 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.678 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:14.678 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840645+0000", 2024-09-21T21:42:14.678 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0010000000000002e-06 2024-09-21T21:42:14.678 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.678 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.678 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:14.678 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840654+0000", 2024-09-21T21:42:14.678 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.8570000000000001e-06 2024-09-21T21:42:14.678 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.679 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.679 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:14.679 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840659+0000", 2024-09-21T21:42:14.679 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.558e-06 2024-09-21T21:42:14.679 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.679 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.679 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:14.679 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.892656+0000", 2024-09-21T21:42:14.679 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.051996586999999997 2024-09-21T21:42:14.679 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.679 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.679 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:14.679 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.892721+0000", 2024-09-21T21:42:14.679 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5010000000000003e-05 2024-09-21T21:42:14.680 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.680 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.680 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:14.680 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.893112+0000", 2024-09-21T21:42:14.680 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00039133899999999999 2024-09-21T21:42:14.680 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.680 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.680 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:14.680 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.896784+0000", 2024-09-21T21:42:14.680 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0036713679999999999 2024-09-21T21:42:14.680 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.680 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.680 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:14.680 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897204+0000", 2024-09-21T21:42:14.681 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00041991400000000002 2024-09-21T21:42:14.681 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.681 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.681 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:14.681 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897234+0000", 2024-09-21T21:42:14.681 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0307999999999998e-05 2024-09-21T21:42:14.681 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.681 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.681 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:14.681 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897313+0000", 2024-09-21T21:42:14.681 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.9107e-05 2024-09-21T21:42:14.681 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.681 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:14.681 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.681 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.681 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.682 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10365 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:14.682 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840668+0000", 2024-09-21T21:42:14.682 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.78555344299999996, 2024-09-21T21:42:14.682 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.058761225, 2024-09-21T21:42:14.682 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:14.682 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:14.682 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:14.682 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:14.682 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:14.682 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:14.682 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10365 2024-09-21T21:42:14.682 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.682 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:14.682 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.683 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:14.683 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840668+0000", 2024-09-21T21:42:14.683 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.683 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.683 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.683 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:14.683 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840668+0000", 2024-09-21T21:42:14.683 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.683 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.683 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.683 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:14.683 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840669+0000", 2024-09-21T21:42:14.683 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6300000000000001e-07 2024-09-21T21:42:14.683 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.684 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.684 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:14.684 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840672+0000", 2024-09-21T21:42:14.684 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.072e-06 2024-09-21T21:42:14.684 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.684 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.684 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:14.684 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840678+0000", 2024-09-21T21:42:14.684 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.7039999999999997e-06 2024-09-21T21:42:14.684 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.684 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.684 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:14.684 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840683+0000", 2024-09-21T21:42:14.684 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6759999999999998e-06 2024-09-21T21:42:14.685 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.685 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.685 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:14.685 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.893929+0000", 2024-09-21T21:42:14.685 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.053246136999999999 2024-09-21T21:42:14.685 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.685 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.685 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:14.685 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.894082+0000", 2024-09-21T21:42:14.685 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00015301799999999999 2024-09-21T21:42:14.685 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.685 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.685 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:14.685 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.894357+0000", 2024-09-21T21:42:14.686 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00027507999999999998 2024-09-21T21:42:14.686 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.686 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.686 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:14.686 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897161+0000", 2024-09-21T21:42:14.686 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0028036810000000001 2024-09-21T21:42:14.686 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.686 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.686 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:14.686 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.899243+0000", 2024-09-21T21:42:14.686 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0020816379999999998 2024-09-21T21:42:14.686 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.686 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.687 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:14.687 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.899306+0000", 2024-09-21T21:42:14.687 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2964000000000002e-05 2024-09-21T21:42:14.687 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.687 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.687 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:14.687 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.899430+0000", 2024-09-21T21:42:14.687 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00012389200000000001 2024-09-21T21:42:14.687 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.687 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:14.687 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.687 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.687 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.687 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10366 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:14.688 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840691+0000", 2024-09-21T21:42:14.688 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.78553074300000003, 2024-09-21T21:42:14.688 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.061540984999999999, 2024-09-21T21:42:14.688 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:14.688 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:14.688 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:14.688 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:14.688 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:14.688 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:14.688 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10366 2024-09-21T21:42:14.688 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.688 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:14.688 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.688 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:14.688 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840691+0000", 2024-09-21T21:42:14.689 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.689 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.689 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.689 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:14.689 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840691+0000", 2024-09-21T21:42:14.689 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.689 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.689 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.689 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:14.689 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840691+0000", 2024-09-21T21:42:14.689 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3099999999999999e-07 2024-09-21T21:42:14.689 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.689 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.689 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:14.689 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840694+0000", 2024-09-21T21:42:14.690 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.5780000000000001e-06 2024-09-21T21:42:14.690 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.690 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.690 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:14.690 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840700+0000", 2024-09-21T21:42:14.690 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9429999999999997e-06 2024-09-21T21:42:14.690 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.690 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.690 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:14.690 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840704+0000", 2024-09-21T21:42:14.690 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6380000000000003e-06 2024-09-21T21:42:14.690 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.690 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.690 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:14.690 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.894846+0000", 2024-09-21T21:42:14.691 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.054141392000000003 2024-09-21T21:42:14.691 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.691 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.691 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:14.691 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.894932+0000", 2024-09-21T21:42:14.691 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.5834000000000005e-05 2024-09-21T21:42:14.691 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.691 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.691 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:14.691 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.895302+0000", 2024-09-21T21:42:14.691 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000370737 2024-09-21T21:42:14.691 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.691 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.691 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:14.691 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897166+0000", 2024-09-21T21:42:14.691 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0018632620000000001 2024-09-21T21:42:14.692 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.692 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.692 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:14.692 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902053+0000", 2024-09-21T21:42:14.692 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0048868289999999997 2024-09-21T21:42:14.692 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.692 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.692 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:14.692 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902111+0000", 2024-09-21T21:42:14.692 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7983999999999999e-05 2024-09-21T21:42:14.692 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.692 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.692 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:14.692 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902232+0000", 2024-09-21T21:42:14.692 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000121457 2024-09-21T21:42:14.692 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.693 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:14.693 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.693 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.693 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.693 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10367 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:14.693 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840842+0000", 2024-09-21T21:42:14.693 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.78537993500000003, 2024-09-21T21:42:14.693 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.061549088000000002, 2024-09-21T21:42:14.693 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:14.693 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:14.693 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:14.693 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:14.693 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:14.693 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:14.693 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10367 2024-09-21T21:42:14.694 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.694 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:14.694 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.694 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:14.694 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840842+0000", 2024-09-21T21:42:14.694 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.694 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.694 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.694 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:14.694 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840842+0000", 2024-09-21T21:42:14.694 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.694 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.694 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.695 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:14.695 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840842+0000", 2024-09-21T21:42:14.695 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3099999999999997e-07 2024-09-21T21:42:14.695 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.695 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.695 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:14.695 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840846+0000", 2024-09-21T21:42:14.695 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.2890000000000002e-06 2024-09-21T21:42:14.695 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.695 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.695 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:14.695 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840855+0000", 2024-09-21T21:42:14.695 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.9619999999999999e-06 2024-09-21T21:42:14.695 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.695 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.695 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:14.696 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840860+0000", 2024-09-21T21:42:14.696 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3759999999999999e-06 2024-09-21T21:42:14.696 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.696 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.696 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:14.696 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.895799+0000", 2024-09-21T21:42:14.696 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.054938569999999999 2024-09-21T21:42:14.696 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.696 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.696 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:14.696 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.895906+0000", 2024-09-21T21:42:14.696 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000107593 2024-09-21T21:42:14.696 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.696 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.696 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:14.697 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.896356+0000", 2024-09-21T21:42:14.697 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00044941700000000002 2024-09-21T21:42:14.697 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.697 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.697 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:14.697 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.898131+0000", 2024-09-21T21:42:14.697 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.001775064 2024-09-21T21:42:14.697 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.697 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.697 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:14.697 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902283+0000", 2024-09-21T21:42:14.697 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0041524839999999997 2024-09-21T21:42:14.697 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.697 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.698 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:14.698 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902316+0000", 2024-09-21T21:42:14.698 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3002e-05 2024-09-21T21:42:14.698 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.698 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.698 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:14.698 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902391+0000", 2024-09-21T21:42:14.698 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.47e-05 2024-09-21T21:42:14.698 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.698 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:14.698 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.698 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.698 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.698 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10368 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:14.699 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840873+0000", 2024-09-21T21:42:14.699 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.78534898099999995, 2024-09-21T21:42:14.699 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.063395291000000006, 2024-09-21T21:42:14.699 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:14.699 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:14.699 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:14.699 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:14.699 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:14.699 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:14.699 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10368 2024-09-21T21:42:14.699 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.699 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:14.699 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.699 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:14.699 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840873+0000", 2024-09-21T21:42:14.700 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.700 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.700 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.700 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:14.700 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840873+0000", 2024-09-21T21:42:14.700 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.700 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.700 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.700 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:14.700 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840873+0000", 2024-09-21T21:42:14.700 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6699999999999999e-07 2024-09-21T21:42:14.700 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.700 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.700 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:14.701 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840876+0000", 2024-09-21T21:42:14.701 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.6410000000000002e-06 2024-09-21T21:42:14.701 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.701 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.701 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:14.701 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840883+0000", 2024-09-21T21:42:14.701 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0389999999999999e-06 2024-09-21T21:42:14.701 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.701 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.701 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:14.701 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840888+0000", 2024-09-21T21:42:14.701 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8679999999999998e-06 2024-09-21T21:42:14.701 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.701 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.701 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:14.701 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897345+0000", 2024-09-21T21:42:14.702 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056457734000000002 2024-09-21T21:42:14.702 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.702 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.702 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:14.702 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897407+0000", 2024-09-21T21:42:14.702 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1822000000000005e-05 2024-09-21T21:42:14.702 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.702 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.702 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:14.702 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897664+0000", 2024-09-21T21:42:14.702 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00025638499999999998 2024-09-21T21:42:14.702 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.702 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.702 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:14.702 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.901136+0000", 2024-09-21T21:42:14.703 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0034728070000000001 2024-09-21T21:42:14.703 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.703 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.703 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:14.703 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904090+0000", 2024-09-21T21:42:14.703 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0029535799999999999 2024-09-21T21:42:14.703 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.703 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.703 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:14.703 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904144+0000", 2024-09-21T21:42:14.703 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4369000000000001e-05 2024-09-21T21:42:14.703 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.703 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.704 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:14.704 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904268+0000", 2024-09-21T21:42:14.704 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00012367899999999999 2024-09-21T21:42:14.704 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.704 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:14.704 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.704 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.704 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.704 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10369 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:14.704 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840902+0000", 2024-09-21T21:42:14.704 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.78531932800000004, 2024-09-21T21:42:14.704 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.063538938000000003, 2024-09-21T21:42:14.704 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:14.704 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:14.705 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:14.705 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:14.705 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:14.705 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:14.705 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10369 2024-09-21T21:42:14.705 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.705 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:14.705 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.705 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:14.705 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840902+0000", 2024-09-21T21:42:14.705 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.705 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.705 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.705 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:14.706 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840902+0000", 2024-09-21T21:42:14.706 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.706 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.706 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.706 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:14.706 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840903+0000", 2024-09-21T21:42:14.706 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2100000000000002e-07 2024-09-21T21:42:14.706 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.706 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.706 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:14.706 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840913+0000", 2024-09-21T21:42:14.706 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.048e-05 2024-09-21T21:42:14.706 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.706 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.707 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:14.707 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840920+0000", 2024-09-21T21:42:14.707 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0330000000000003e-06 2024-09-21T21:42:14.707 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.707 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.707 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:14.707 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840925+0000", 2024-09-21T21:42:14.707 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0170000000000004e-06 2024-09-21T21:42:14.707 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.707 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.707 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:14.707 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.898179+0000", 2024-09-21T21:42:14.707 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.057253671999999999 2024-09-21T21:42:14.707 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.707 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.708 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:14.708 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.898284+0000", 2024-09-21T21:42:14.708 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000105202 2024-09-21T21:42:14.708 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.708 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.708 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:14.708 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.898729+0000", 2024-09-21T21:42:14.708 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00044459300000000002 2024-09-21T21:42:14.708 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.708 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.708 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:14.708 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902246+0000", 2024-09-21T21:42:14.708 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0035171629999999998 2024-09-21T21:42:14.708 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.709 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.709 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:14.709 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904310+0000", 2024-09-21T21:42:14.709 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.002063765 2024-09-21T21:42:14.709 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.709 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.709 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:14.709 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904342+0000", 2024-09-21T21:42:14.709 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.2091999999999997e-05 2024-09-21T21:42:14.709 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.709 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.709 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:14.709 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904441+0000", 2024-09-21T21:42:14.709 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.9500000000000006e-05 2024-09-21T21:42:14.710 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.710 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:14.710 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.710 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.710 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.710 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10370 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:14.710 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840944+0000", 2024-09-21T21:42:14.710 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.78527761500000004, 2024-09-21T21:42:14.710 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.064436853000000002, 2024-09-21T21:42:14.710 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:14.710 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:14.710 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:14.710 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:14.711 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:14.711 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:14.711 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10370 2024-09-21T21:42:14.711 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.711 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:14.711 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.711 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:14.711 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840944+0000", 2024-09-21T21:42:14.711 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.711 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.711 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.711 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:14.711 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840944+0000", 2024-09-21T21:42:14.712 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.712 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.712 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.712 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:14.712 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840945+0000", 2024-09-21T21:42:14.712 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2300000000000002e-07 2024-09-21T21:42:14.712 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.712 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.712 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:14.712 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840947+0000", 2024-09-21T21:42:14.712 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.8200000000000001e-06 2024-09-21T21:42:14.712 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.712 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.712 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:14.713 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840955+0000", 2024-09-21T21:42:14.713 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.8660000000000006e-06 2024-09-21T21:42:14.713 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.713 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.713 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:14.713 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840960+0000", 2024-09-21T21:42:14.713 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8670000000000002e-06 2024-09-21T21:42:14.713 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.713 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.713 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:14.713 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.899460+0000", 2024-09-21T21:42:14.713 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.058499922000000003 2024-09-21T21:42:14.713 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.713 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.713 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:14.713 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.899530+0000", 2024-09-21T21:42:14.714 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9558e-05 2024-09-21T21:42:14.714 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.714 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.714 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:14.714 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.899775+0000", 2024-09-21T21:42:14.714 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000245231 2024-09-21T21:42:14.714 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.714 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.714 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:14.714 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902254+0000", 2024-09-21T21:42:14.714 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0024790519999999998 2024-09-21T21:42:14.714 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.714 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.714 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:14.714 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905183+0000", 2024-09-21T21:42:14.715 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.002929156 2024-09-21T21:42:14.715 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.715 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.715 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:14.715 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905243+0000", 2024-09-21T21:42:14.715 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0055e-05 2024-09-21T21:42:14.715 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.715 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.715 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:14.715 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905381+0000", 2024-09-21T21:42:14.715 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00013790300000000001 2024-09-21T21:42:14.715 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.715 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:14.715 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.716 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.716 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.716 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10371 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:14.716 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840980+0000", 2024-09-21T21:42:14.716 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.78524194899999999, 2024-09-21T21:42:14.716 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.064554376999999996, 2024-09-21T21:42:14.716 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:14.716 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:14.716 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:14.716 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:14.716 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:14.716 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:14.716 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10371 2024-09-21T21:42:14.716 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.716 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:14.717 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.717 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:14.717 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840980+0000", 2024-09-21T21:42:14.717 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.717 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.717 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.717 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:14.717 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840980+0000", 2024-09-21T21:42:14.717 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.717 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.717 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.717 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:14.717 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840980+0000", 2024-09-21T21:42:14.717 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8999999999999997e-07 2024-09-21T21:42:14.717 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.717 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.718 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:14.718 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840983+0000", 2024-09-21T21:42:14.718 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.123e-06 2024-09-21T21:42:14.718 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.718 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.718 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:14.718 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840990+0000", 2024-09-21T21:42:14.718 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9129999999999997e-06 2024-09-21T21:42:14.718 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.718 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.718 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:14.718 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840995+0000", 2024-09-21T21:42:14.718 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0989999999999998e-06 2024-09-21T21:42:14.718 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.718 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.718 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:14.719 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.900168+0000", 2024-09-21T21:42:14.719 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.059172739000000002 2024-09-21T21:42:14.719 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.719 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.719 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:14.719 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.900274+0000", 2024-09-21T21:42:14.719 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010546499999999999 2024-09-21T21:42:14.719 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.719 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.719 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:14.719 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.900703+0000", 2024-09-21T21:42:14.719 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00042978200000000003 2024-09-21T21:42:14.719 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.719 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.720 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:14.720 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.903443+0000", 2024-09-21T21:42:14.720 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0027400749999999998 2024-09-21T21:42:14.720 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.720 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.720 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:14.720 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905430+0000", 2024-09-21T21:42:14.720 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0019868109999999998 2024-09-21T21:42:14.720 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.720 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.720 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:14.720 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905464+0000", 2024-09-21T21:42:14.720 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3599000000000001e-05 2024-09-21T21:42:14.720 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.720 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.721 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:14.721 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905534+0000", 2024-09-21T21:42:14.721 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0281e-05 2024-09-21T21:42:14.721 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.721 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:14.721 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.721 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.721 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.721 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10372 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:14.721 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.841004+0000", 2024-09-21T21:42:14.721 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.78521806000000005, 2024-09-21T21:42:14.721 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.064757812999999997, 2024-09-21T21:42:14.721 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:14.721 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:14.721 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:14.722 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:14.722 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:14.722 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:14.722 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10372 2024-09-21T21:42:14.722 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.722 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:14.722 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.722 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:14.722 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841004+0000", 2024-09-21T21:42:14.722 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.722 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.722 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.722 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:14.722 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841004+0000", 2024-09-21T21:42:14.723 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.723 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.723 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.723 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:14.723 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841004+0000", 2024-09-21T21:42:14.723 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8500000000000002e-07 2024-09-21T21:42:14.723 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.723 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.723 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:14.723 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841007+0000", 2024-09-21T21:42:14.723 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.6390000000000001e-06 2024-09-21T21:42:14.723 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.723 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.723 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:14.724 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841013+0000", 2024-09-21T21:42:14.724 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0070000000000003e-06 2024-09-21T21:42:14.724 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.724 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.724 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:14.724 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841018+0000", 2024-09-21T21:42:14.724 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9119999999999997e-06 2024-09-21T21:42:14.724 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.724 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.724 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:14.724 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.901182+0000", 2024-09-21T21:42:14.724 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.060164444999999997 2024-09-21T21:42:14.724 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.724 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.724 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:14.724 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.901297+0000", 2024-09-21T21:42:14.725 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011468699999999999 2024-09-21T21:42:14.725 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.725 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.725 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:14.725 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.901559+0000", 2024-09-21T21:42:14.725 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000261764 2024-09-21T21:42:14.725 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.725 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.725 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:14.725 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904281+0000", 2024-09-21T21:42:14.725 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0027229329999999999 2024-09-21T21:42:14.725 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.725 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.725 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:14.725 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905588+0000", 2024-09-21T21:42:14.726 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.001306438 2024-09-21T21:42:14.726 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.726 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.726 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:14.726 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905644+0000", 2024-09-21T21:42:14.726 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5747e-05 2024-09-21T21:42:14.726 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.726 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.726 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:14.726 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905761+0000", 2024-09-21T21:42:14.726 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000117856 2024-09-21T21:42:14.726 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.726 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:14.726 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.727 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.727 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.727 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10373 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:14.727 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.841027+0000", 2024-09-21T21:42:14.727 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.78519497599999999, 2024-09-21T21:42:14.727 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.066941839000000003, 2024-09-21T21:42:14.727 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:14.727 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:14.727 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:14.727 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:14.727 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:14.727 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:14.727 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10373 2024-09-21T21:42:14.728 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.728 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:14.728 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.728 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:14.728 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841027+0000", 2024-09-21T21:42:14.728 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.728 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.728 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.728 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:14.728 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841027+0000", 2024-09-21T21:42:14.728 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.728 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.728 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.728 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:14.728 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841027+0000", 2024-09-21T21:42:14.729 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6199999999999999e-07 2024-09-21T21:42:14.729 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.729 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.729 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:14.729 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841030+0000", 2024-09-21T21:42:14.729 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.729e-06 2024-09-21T21:42:14.729 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.729 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.729 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:14.729 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841036+0000", 2024-09-21T21:42:14.729 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1979999999999998e-06 2024-09-21T21:42:14.729 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.729 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.729 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:14.729 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841042+0000", 2024-09-21T21:42:14.730 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4289999999999998e-06 2024-09-21T21:42:14.730 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.730 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.730 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:14.730 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902433+0000", 2024-09-21T21:42:14.730 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.061390329 2024-09-21T21:42:14.730 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.730 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.730 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:14.730 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902541+0000", 2024-09-21T21:42:14.730 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000108349 2024-09-21T21:42:14.730 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.730 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.730 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:14.730 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902983+0000", 2024-09-21T21:42:14.731 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00044158000000000001 2024-09-21T21:42:14.731 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.731 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.731 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:14.731 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904490+0000", 2024-09-21T21:42:14.731 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0015074489999999999 2024-09-21T21:42:14.731 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.731 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.731 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:14.731 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.907671+0000", 2024-09-21T21:42:14.731 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.003180511 2024-09-21T21:42:14.731 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.731 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.731 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:14.732 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.907767+0000", 2024-09-21T21:42:14.732 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6379000000000007e-05 2024-09-21T21:42:14.732 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.732 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.732 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:14.732 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.907969+0000", 2024-09-21T21:42:14.732 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00020152399999999999 2024-09-21T21:42:14.732 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.732 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:14.732 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.732 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.732 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:14.732 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:14.734 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T21:42:14.734 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:14.732+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:14.835 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:14.833+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:14.886 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:14.887 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:42:14.887 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.887 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:12342 3.1 3.c94e82e1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:42:14.887 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:14.411725+0000", 2024-09-21T21:42:14.887 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.47412458299999999, 2024-09-21T21:42:14.887 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.474141802, 2024-09-21T21:42:14.887 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:14.887 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:14.887 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "waiting for peered", 2024-09-21T21:42:14.887 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:14.887 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:14.887 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:14.888 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 12342 2024-09-21T21:42:14.888 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.888 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:14.888 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.888 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:14.888 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411725+0000", 2024-09-21T21:42:14.888 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.888 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.888 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.888 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:14.888 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411725+0000", 2024-09-21T21:42:14.888 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.888 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.888 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.888 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:14.889 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411726+0000", 2024-09-21T21:42:14.889 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.4169999999999999e-06 2024-09-21T21:42:14.889 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.889 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.889 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:14.889 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411734+0000", 2024-09-21T21:42:14.889 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.3799999999999996e-06 2024-09-21T21:42:14.889 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.889 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.889 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:14.889 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411750+0000", 2024-09-21T21:42:14.889 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.6429000000000001e-05 2024-09-21T21:42:14.889 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.889 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.889 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:14.890 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411760+0000", 2024-09-21T21:42:14.890 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.4239999999999999e-06 2024-09-21T21:42:14.890 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.890 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.890 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:14.890 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411828+0000", 2024-09-21T21:42:14.890 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.7935999999999995e-05 2024-09-21T21:42:14.890 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.890 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.890 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op must wait for map", 2024-09-21T21:42:14.890 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411837+0000", 2024-09-21T21:42:14.890 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.0100000000000001e-06 2024-09-21T21:42:14.890 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.891 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.891 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:14.891 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.414984+0000", 2024-09-21T21:42:14.891 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0031472050000000001 2024-09-21T21:42:14.891 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.891 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.891 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for peered", 2024-09-21T21:42:14.891 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.414994+0000", 2024-09-21T21:42:14.891 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0291e-05 2024-09-21T21:42:14.891 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.891 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:14.891 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.891 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.891 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.891 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:12348 3.1 3.c94e82e1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:42:14.892 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:14.411793+0000", 2024-09-21T21:42:14.892 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.47405703700000001, 2024-09-21T21:42:14.892 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.47412223100000001, 2024-09-21T21:42:14.892 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:14.892 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:14.892 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "waiting for peered", 2024-09-21T21:42:14.892 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:14.892 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:14.892 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:14.892 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 12348 2024-09-21T21:42:14.892 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.892 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:14.892 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.892 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:14.892 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411793+0000", 2024-09-21T21:42:14.892 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.893 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.893 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.893 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:14.893 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411793+0000", 2024-09-21T21:42:14.893 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:14.893 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.893 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.893 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:14.893 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411793+0000", 2024-09-21T21:42:14.893 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0999999999999998e-07 2024-09-21T21:42:14.893 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.893 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.893 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:14.894 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411797+0000", 2024-09-21T21:42:14.894 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.49e-06 2024-09-21T21:42:14.894 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.894 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.894 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:14.894 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411809+0000", 2024-09-21T21:42:14.894 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.2510999999999999e-05 2024-09-21T21:42:14.894 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.894 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.894 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:14.894 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411816+0000", 2024-09-21T21:42:14.894 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6420000000000001e-06 2024-09-21T21:42:14.894 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.895 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.895 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:14.895 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411963+0000", 2024-09-21T21:42:14.895 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00014700999999999999 2024-09-21T21:42:14.895 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.895 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.895 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting_for_map not empty", 2024-09-21T21:42:14.895 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411970+0000", 2024-09-21T21:42:14.895 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9800000000000001e-06 2024-09-21T21:42:14.895 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.895 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.895 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:14.895 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.415018+0000", 2024-09-21T21:42:14.895 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0030482299999999999 2024-09-21T21:42:14.895 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:14.896 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:14.896 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for peered", 2024-09-21T21:42:14.896 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.415025+0000", 2024-09-21T21:42:14.896 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.3749999999999997e-06 2024-09-21T21:42:14.896 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.896 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:14.896 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.896 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:14.896 INFO:teuthology.orchestra.run.smithi081.stdout: ], 2024-09-21T21:42:14.896 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 2 2024-09-21T21:42:14.896 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:14.897 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T21:42:14.935 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:14.934+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:14.999 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:15.000 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:42:15.000 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:42:15.000 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:42:15.000 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:15.008 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T21:42:15.036 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:15.035+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:15.114 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:15.114 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 20, 2024-09-21T21:42:15.114 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 600, 2024-09-21T21:42:15.115 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:42:15.115 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.115 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13295 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3330048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:15.115 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.796526+0000", 2024-09-21T21:42:15.115 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 24.315561467999999, 2024-09-21T21:42:15.115 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071630928999999996, 2024-09-21T21:42:15.115 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:15.115 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:15.115 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:15.115 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:15.115 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:15.115 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:15.115 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13295 2024-09-21T21:42:15.116 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.116 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:15.116 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.116 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:15.116 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796526+0000", 2024-09-21T21:42:15.116 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:15.116 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.116 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.116 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:15.116 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796526+0000", 2024-09-21T21:42:15.116 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:15.116 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.116 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.116 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:15.116 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796526+0000", 2024-09-21T21:42:15.117 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5900000000000003e-07 2024-09-21T21:42:15.117 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.117 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.117 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:15.117 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796529+0000", 2024-09-21T21:42:15.117 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3910000000000001e-06 2024-09-21T21:42:15.117 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.117 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.117 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:15.117 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796537+0000", 2024-09-21T21:42:15.117 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.2819999999999996e-06 2024-09-21T21:42:15.117 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.117 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.117 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:15.118 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796541+0000", 2024-09-21T21:42:15.118 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.9400000000000004e-06 2024-09-21T21:42:15.118 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.118 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.118 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:15.118 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.868008+0000", 2024-09-21T21:42:15.118 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071467586999999999 2024-09-21T21:42:15.118 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.118 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.118 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:15.118 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.868061+0000", 2024-09-21T21:42:15.118 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2349000000000003e-05 2024-09-21T21:42:15.118 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.118 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.119 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:15.119 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.868157+0000", 2024-09-21T21:42:15.119 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6020999999999994e-05 2024-09-21T21:42:15.119 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:15.119 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:15.119 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:15.119 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.119 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.119 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13296 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 4046848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:15.119 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.796643+0000", 2024-09-21T21:42:15.119 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 24.315444649, 2024-09-21T21:42:15.119 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071682019999999999, 2024-09-21T21:42:15.119 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:15.119 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:15.119 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:15.120 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:15.120 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:15.120 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:15.120 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13296 2024-09-21T21:42:15.120 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.120 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:15.120 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.120 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:15.120 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796643+0000", 2024-09-21T21:42:15.120 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:15.120 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.120 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.120 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:15.120 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796643+0000", 2024-09-21T21:42:15.120 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:15.121 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.121 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.121 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:15.121 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796643+0000", 2024-09-21T21:42:15.121 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.7199999999999998e-07 2024-09-21T21:42:15.121 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.121 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.121 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:15.121 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796647+0000", 2024-09-21T21:42:15.121 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6770000000000001e-06 2024-09-21T21:42:15.121 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.122 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.122 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:15.122 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796656+0000", 2024-09-21T21:42:15.122 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2059999999999996e-06 2024-09-21T21:42:15.122 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.122 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.122 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:15.122 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796664+0000", 2024-09-21T21:42:15.122 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7919999999999993e-06 2024-09-21T21:42:15.122 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.122 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.122 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:15.122 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.868186+0000", 2024-09-21T21:42:15.122 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071522273999999997 2024-09-21T21:42:15.123 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.123 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.123 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:15.123 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.868239+0000", 2024-09-21T21:42:15.123 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2787999999999997e-05 2024-09-21T21:42:15.123 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.123 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.123 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:15.123 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.868325+0000", 2024-09-21T21:42:15.123 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.5611000000000004e-05 2024-09-21T21:42:15.123 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:15.123 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:15.123 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:15.123 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.124 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.124 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13561 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 1728512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:15.124 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.823991+0000", 2024-09-21T21:42:15.124 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 24.288096348, 2024-09-21T21:42:15.124 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071654762999999996, 2024-09-21T21:42:15.124 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:15.124 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:15.124 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:15.124 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:15.124 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:15.124 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:15.124 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13561 2024-09-21T21:42:15.124 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.124 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:15.125 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.125 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:15.125 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.823991+0000", 2024-09-21T21:42:15.125 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:15.125 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.125 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.125 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:15.125 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.823991+0000", 2024-09-21T21:42:15.125 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:15.125 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.125 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.125 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:15.125 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.823991+0000", 2024-09-21T21:42:15.125 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9299999999999998e-07 2024-09-21T21:42:15.125 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.126 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.126 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:15.126 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.823994+0000", 2024-09-21T21:42:15.126 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2369999999999999e-06 2024-09-21T21:42:15.126 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.126 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.126 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:15.126 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824000+0000", 2024-09-21T21:42:15.126 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.55e-06 2024-09-21T21:42:15.126 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.126 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.126 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:15.126 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824005+0000", 2024-09-21T21:42:15.126 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4329999999999997e-06 2024-09-21T21:42:15.126 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.127 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.127 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:15.127 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.895360+0000", 2024-09-21T21:42:15.127 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071355426 2024-09-21T21:42:15.127 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.127 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.127 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:15.127 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.895477+0000", 2024-09-21T21:42:15.127 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0001171 2024-09-21T21:42:15.127 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.127 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.127 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:15.128 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.895646+0000", 2024-09-21T21:42:15.128 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000168524 2024-09-21T21:42:15.128 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:15.128 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:15.128 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:15.128 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.128 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.128 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13562 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 1757184~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:15.128 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.824011+0000", 2024-09-21T21:42:15.128 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 24.288076471, 2024-09-21T21:42:15.128 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072020453999999998, 2024-09-21T21:42:15.128 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:15.128 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:15.128 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:15.129 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:15.129 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:15.129 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:15.129 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13562 2024-09-21T21:42:15.129 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.129 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:15.129 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.129 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:15.129 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824011+0000", 2024-09-21T21:42:15.129 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:15.129 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.129 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.129 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:15.129 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824011+0000", 2024-09-21T21:42:15.130 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:15.130 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.130 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.130 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:15.130 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824011+0000", 2024-09-21T21:42:15.130 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5600000000000001e-07 2024-09-21T21:42:15.130 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.130 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.130 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:15.130 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824013+0000", 2024-09-21T21:42:15.130 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9800000000000001e-06 2024-09-21T21:42:15.130 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.131 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.131 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:15.131 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824019+0000", 2024-09-21T21:42:15.131 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7339999999999996e-06 2024-09-21T21:42:15.131 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.131 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.131 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:15.131 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824023+0000", 2024-09-21T21:42:15.131 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3680000000000004e-06 2024-09-21T21:42:15.131 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.131 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.131 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:15.131 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.895699+0000", 2024-09-21T21:42:15.131 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071675597999999993 2024-09-21T21:42:15.132 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.132 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.132 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:15.132 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.895830+0000", 2024-09-21T21:42:15.132 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000130858 2024-09-21T21:42:15.132 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.132 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.132 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:15.132 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896031+0000", 2024-09-21T21:42:15.132 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00020155999999999999 2024-09-21T21:42:15.132 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:15.132 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:15.132 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:15.132 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.132 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.133 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13563 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 323584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:15.133 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.824029+0000", 2024-09-21T21:42:15.133 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 24.288058030999998, 2024-09-21T21:42:15.133 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072307563000000005, 2024-09-21T21:42:15.133 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:15.133 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:15.133 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:15.133 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:15.133 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:15.133 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:15.133 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13563 2024-09-21T21:42:15.133 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.133 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:15.133 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.133 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:15.134 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824029+0000", 2024-09-21T21:42:15.134 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:15.134 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.134 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.134 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:15.134 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824029+0000", 2024-09-21T21:42:15.134 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:15.134 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.134 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.134 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:15.134 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824030+0000", 2024-09-21T21:42:15.134 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1699999999999999e-07 2024-09-21T21:42:15.134 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.134 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.135 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:15.135 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824032+0000", 2024-09-21T21:42:15.135 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0320000000000002e-06 2024-09-21T21:42:15.135 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.135 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.135 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:15.135 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824038+0000", 2024-09-21T21:42:15.135 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8769999999999998e-06 2024-09-21T21:42:15.135 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.135 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.135 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:15.135 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824042+0000", 2024-09-21T21:42:15.135 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5650000000000003e-06 2024-09-21T21:42:15.135 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.135 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.136 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:15.136 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896058+0000", 2024-09-21T21:42:15.136 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072015941 2024-09-21T21:42:15.136 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.136 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.136 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:15.136 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896164+0000", 2024-09-21T21:42:15.136 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010593499999999999 2024-09-21T21:42:15.136 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.136 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.136 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:15.136 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896337+0000", 2024-09-21T21:42:15.136 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00017279599999999999 2024-09-21T21:42:15.136 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:15.137 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:15.137 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:15.137 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.137 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.137 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13564 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 983040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:15.137 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.824048+0000", 2024-09-21T21:42:15.137 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 24.288038956000001, 2024-09-21T21:42:15.137 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072491193999999995, 2024-09-21T21:42:15.137 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:15.137 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:15.137 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:15.137 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:15.137 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:15.137 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:15.137 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13564 2024-09-21T21:42:15.138 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.138 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:15.138 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.138 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:15.138 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824048+0000", 2024-09-21T21:42:15.138 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:15.138 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.138 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.138 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:15.138 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824048+0000", 2024-09-21T21:42:15.138 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:15.138 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.138 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.138 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:15.139 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824663+0000", 2024-09-21T21:42:15.139 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00061494299999999996 2024-09-21T21:42:15.139 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.139 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.139 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:15.139 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824669+0000", 2024-09-21T21:42:15.139 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2900000000000002e-06 2024-09-21T21:42:15.139 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.139 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.139 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:15.139 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824679+0000", 2024-09-21T21:42:15.139 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0338e-05 2024-09-21T21:42:15.139 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.139 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.139 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:15.140 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824687+0000", 2024-09-21T21:42:15.140 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.9049999999999997e-06 2024-09-21T21:42:15.140 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.140 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.140 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:15.140 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896381+0000", 2024-09-21T21:42:15.140 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071694305999999999 2024-09-21T21:42:15.140 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.140 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.140 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:15.140 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896445+0000", 2024-09-21T21:42:15.140 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3869999999999997e-05 2024-09-21T21:42:15.140 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.141 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.141 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:15.141 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896540+0000", 2024-09-21T21:42:15.141 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.4542000000000004e-05 2024-09-21T21:42:15.141 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:15.141 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:15.141 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:15.141 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.141 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.141 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13565 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3190784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:15.141 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.824704+0000", 2024-09-21T21:42:15.141 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 24.287383084999998, 2024-09-21T21:42:15.141 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072014745000000005, 2024-09-21T21:42:15.141 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:15.141 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:15.142 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:15.142 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:15.142 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:15.142 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:15.142 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13565 2024-09-21T21:42:15.142 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.142 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:15.142 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.142 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:15.142 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824704+0000", 2024-09-21T21:42:15.142 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:15.142 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.142 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.142 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:15.142 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824704+0000", 2024-09-21T21:42:15.143 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:15.143 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.143 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.143 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:15.143 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824705+0000", 2024-09-21T21:42:15.143 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8999999999999997e-07 2024-09-21T21:42:15.143 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.143 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.143 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:15.143 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824707+0000", 2024-09-21T21:42:15.143 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.7669999999999999e-06 2024-09-21T21:42:15.143 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.143 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.143 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:15.144 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824715+0000", 2024-09-21T21:42:15.144 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7700000000000001e-06 2024-09-21T21:42:15.144 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.144 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.144 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:15.144 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824721+0000", 2024-09-21T21:42:15.144 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9560000000000002e-06 2024-09-21T21:42:15.144 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.144 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.144 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:15.144 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896570+0000", 2024-09-21T21:42:15.144 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071848870999999995 2024-09-21T21:42:15.144 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.144 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.144 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:15.144 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896626+0000", 2024-09-21T21:42:15.145 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5578000000000002e-05 2024-09-21T21:42:15.145 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.145 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.145 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:15.145 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896719+0000", 2024-09-21T21:42:15.145 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.3313000000000006e-05 2024-09-21T21:42:15.145 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:15.145 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:15.145 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:15.145 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.145 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.146 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13566 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3162112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:15.146 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.824729+0000", 2024-09-21T21:42:15.146 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 24.287358198, 2024-09-21T21:42:15.146 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072356907999999998, 2024-09-21T21:42:15.146 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:15.146 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:15.146 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:15.146 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:15.146 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:15.146 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:15.146 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13566 2024-09-21T21:42:15.146 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.146 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:15.147 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.147 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:15.147 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824729+0000", 2024-09-21T21:42:15.147 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:15.147 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.147 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.147 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:15.147 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824729+0000", 2024-09-21T21:42:15.147 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:15.147 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.147 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.147 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:15.147 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825630+0000", 2024-09-21T21:42:15.147 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00090139400000000002 2024-09-21T21:42:15.148 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.148 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.148 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:15.148 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825634+0000", 2024-09-21T21:42:15.148 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4960000000000001e-06 2024-09-21T21:42:15.148 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.148 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.148 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:15.148 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825644+0000", 2024-09-21T21:42:15.148 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.9650000000000003e-06 2024-09-21T21:42:15.148 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.148 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.148 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:15.148 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825649+0000", 2024-09-21T21:42:15.148 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0359999999999997e-06 2024-09-21T21:42:15.149 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.149 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.149 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:15.149 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896757+0000", 2024-09-21T21:42:15.149 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071107717000000001 2024-09-21T21:42:15.149 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.149 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.149 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:15.149 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896878+0000", 2024-09-21T21:42:15.149 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000120834 2024-09-21T21:42:15.149 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.149 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.149 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:15.149 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897086+0000", 2024-09-21T21:42:15.149 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00020846600000000001 2024-09-21T21:42:15.150 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:15.150 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:15.150 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:15.150 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.150 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.150 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13567 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3878912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:15.150 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.825658+0000", 2024-09-21T21:42:15.150 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 24.286429199000001, 2024-09-21T21:42:15.150 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071597694000000003, 2024-09-21T21:42:15.150 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:15.150 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:15.150 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:15.150 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:15.150 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:15.151 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:15.151 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13567 2024-09-21T21:42:15.151 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.151 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:15.151 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.151 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:15.151 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825658+0000", 2024-09-21T21:42:15.151 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:15.151 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.151 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.151 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:15.151 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825658+0000", 2024-09-21T21:42:15.151 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:15.151 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.152 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.152 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:15.152 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825658+0000", 2024-09-21T21:42:15.152 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.5699999999999999e-07 2024-09-21T21:42:15.152 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.152 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.152 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:15.152 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825660+0000", 2024-09-21T21:42:15.152 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.638e-06 2024-09-21T21:42:15.152 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.152 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.152 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:15.152 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825664+0000", 2024-09-21T21:42:15.152 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4340000000000002e-06 2024-09-21T21:42:15.152 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.153 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.153 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:15.153 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825668+0000", 2024-09-21T21:42:15.153 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.1669999999999999e-06 2024-09-21T21:42:15.153 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.153 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.153 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:15.153 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897113+0000", 2024-09-21T21:42:15.153 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071445358 2024-09-21T21:42:15.153 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.153 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.153 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:15.153 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897169+0000", 2024-09-21T21:42:15.153 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5813000000000002e-05 2024-09-21T21:42:15.153 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.154 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.154 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:15.154 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897256+0000", 2024-09-21T21:42:15.154 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.7027000000000006e-05 2024-09-21T21:42:15.154 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:15.154 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:15.154 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:15.154 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.154 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.154 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13568 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3907584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:15.154 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.825671+0000", 2024-09-21T21:42:15.154 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 24.286415788999999, 2024-09-21T21:42:15.154 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071794595000000003, 2024-09-21T21:42:15.154 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:15.155 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:15.155 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:15.155 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:15.155 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:15.155 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:15.155 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13568 2024-09-21T21:42:15.155 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.155 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:15.155 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.155 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:15.155 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825671+0000", 2024-09-21T21:42:15.155 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:15.155 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.155 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.156 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:15.156 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825671+0000", 2024-09-21T21:42:15.156 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:15.156 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.156 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.156 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:15.156 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825672+0000", 2024-09-21T21:42:15.156 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2999999999999999e-07 2024-09-21T21:42:15.156 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.156 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.156 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:15.156 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825673+0000", 2024-09-21T21:42:15.156 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.246e-06 2024-09-21T21:42:15.156 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.157 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.157 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:15.157 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825693+0000", 2024-09-21T21:42:15.157 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9789999999999999e-05 2024-09-21T21:42:15.157 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.157 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.157 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:15.157 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825697+0000", 2024-09-21T21:42:15.157 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2379999999999999e-06 2024-09-21T21:42:15.157 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.157 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.157 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:15.157 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897284+0000", 2024-09-21T21:42:15.157 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071586710999999997 2024-09-21T21:42:15.158 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.158 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.158 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:15.158 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897337+0000", 2024-09-21T21:42:15.158 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2800000000000003e-05 2024-09-21T21:42:15.158 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.158 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.158 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:15.158 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897466+0000", 2024-09-21T21:42:15.158 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00012957999999999999 2024-09-21T21:42:15.158 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:15.158 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:15.158 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:15.158 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.158 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.159 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13574 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 352256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:15.159 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.825702+0000", 2024-09-21T21:42:15.159 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 24.286385640999999, 2024-09-21T21:42:15.159 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072080377000000001, 2024-09-21T21:42:15.159 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:15.159 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:15.159 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:15.159 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:15.159 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:15.159 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:15.159 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13574 2024-09-21T21:42:15.159 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.159 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:15.159 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.159 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:15.160 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825702+0000", 2024-09-21T21:42:15.160 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:15.160 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.160 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.160 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:15.160 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825702+0000", 2024-09-21T21:42:15.160 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:15.160 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.160 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.160 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:15.160 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825943+0000", 2024-09-21T21:42:15.160 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00024105200000000001 2024-09-21T21:42:15.160 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.160 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.160 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:15.161 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825946+0000", 2024-09-21T21:42:15.161 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.743e-06 2024-09-21T21:42:15.161 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.161 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.161 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:15.161 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825956+0000", 2024-09-21T21:42:15.161 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6609999999999995e-06 2024-09-21T21:42:15.161 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.161 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.161 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:15.161 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825963+0000", 2024-09-21T21:42:15.161 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4609999999999996e-06 2024-09-21T21:42:15.161 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.161 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.161 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:15.162 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897502+0000", 2024-09-21T21:42:15.162 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071539354999999999 2024-09-21T21:42:15.162 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.162 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.162 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:15.162 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897619+0000", 2024-09-21T21:42:15.162 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011665 2024-09-21T21:42:15.162 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.162 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.162 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:15.162 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897782+0000", 2024-09-21T21:42:15.162 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00016345500000000001 2024-09-21T21:42:15.162 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:15.162 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:15.163 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:15.163 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.163 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.163 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13575 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 380928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:15.163 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.825973+0000", 2024-09-21T21:42:15.163 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 24.286114205000001, 2024-09-21T21:42:15.163 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072012723000000001, 2024-09-21T21:42:15.163 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:15.163 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:15.163 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:15.163 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:15.163 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:15.163 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:15.163 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13575 2024-09-21T21:42:15.164 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.164 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:15.164 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.164 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:15.164 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825973+0000", 2024-09-21T21:42:15.164 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:15.164 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.164 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.164 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:15.164 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825973+0000", 2024-09-21T21:42:15.164 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:15.164 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.165 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.165 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:15.165 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826746+0000", 2024-09-21T21:42:15.165 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00077326800000000002 2024-09-21T21:42:15.165 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.165 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.165 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:15.165 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826766+0000", 2024-09-21T21:42:15.165 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9465000000000001e-05 2024-09-21T21:42:15.165 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.165 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.166 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:15.166 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826812+0000", 2024-09-21T21:42:15.166 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5927999999999998e-05 2024-09-21T21:42:15.166 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.166 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.166 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:15.166 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826833+0000", 2024-09-21T21:42:15.166 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1710999999999999e-05 2024-09-21T21:42:15.166 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.166 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.166 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:15.166 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897809+0000", 2024-09-21T21:42:15.166 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.070975229000000001 2024-09-21T21:42:15.167 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.167 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.167 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:15.167 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897864+0000", 2024-09-21T21:42:15.167 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5334999999999998e-05 2024-09-21T21:42:15.167 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.167 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.167 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:15.167 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897986+0000", 2024-09-21T21:42:15.167 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000121787 2024-09-21T21:42:15.167 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:15.167 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:15.167 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:15.167 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.167 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.167 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13577 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 1814528~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:15.168 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.826874+0000", 2024-09-21T21:42:15.168 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 24.285213170999999, 2024-09-21T21:42:15.168 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071639123999999998, 2024-09-21T21:42:15.168 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:15.168 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:15.168 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:15.168 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:15.168 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:15.168 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:15.168 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13577 2024-09-21T21:42:15.168 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.168 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:15.168 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.168 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:15.168 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826874+0000", 2024-09-21T21:42:15.168 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:15.169 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.169 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.169 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:15.169 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826874+0000", 2024-09-21T21:42:15.169 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:15.169 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.169 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.169 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:15.169 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826875+0000", 2024-09-21T21:42:15.169 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.82e-07 2024-09-21T21:42:15.169 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.169 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.169 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:15.169 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826877+0000", 2024-09-21T21:42:15.169 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.295e-06 2024-09-21T21:42:15.170 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.170 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.170 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:15.170 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826883+0000", 2024-09-21T21:42:15.170 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2609999999999999e-06 2024-09-21T21:42:15.170 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.170 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.170 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:15.170 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826889+0000", 2024-09-21T21:42:15.170 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5589999999999996e-06 2024-09-21T21:42:15.170 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.170 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.170 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:15.170 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898301+0000", 2024-09-21T21:42:15.171 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071412188000000001 2024-09-21T21:42:15.171 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.171 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.171 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:15.171 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898416+0000", 2024-09-21T21:42:15.171 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000115538 2024-09-21T21:42:15.171 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.171 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.171 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:15.171 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898513+0000", 2024-09-21T21:42:15.171 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6800999999999997e-05 2024-09-21T21:42:15.172 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:15.172 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:15.172 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:15.172 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.172 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.172 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13578 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 2502656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:15.172 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.826896+0000", 2024-09-21T21:42:15.172 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 24.285191715, 2024-09-21T21:42:15.172 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071789411999999997, 2024-09-21T21:42:15.172 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:15.172 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:15.172 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:15.172 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:15.173 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:15.173 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:15.173 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13578 2024-09-21T21:42:15.173 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.173 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:15.173 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.173 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:15.173 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826896+0000", 2024-09-21T21:42:15.173 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:15.173 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.173 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.173 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:15.173 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826896+0000", 2024-09-21T21:42:15.173 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:15.174 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.174 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.174 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:15.174 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827327+0000", 2024-09-21T21:42:15.174 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00043094500000000002 2024-09-21T21:42:15.174 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.174 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.174 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:15.174 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827331+0000", 2024-09-21T21:42:15.174 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.0969999999999999e-06 2024-09-21T21:42:15.174 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.174 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.174 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:15.175 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827341+0000", 2024-09-21T21:42:15.175 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0199e-05 2024-09-21T21:42:15.175 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.175 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.175 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:15.175 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827346+0000", 2024-09-21T21:42:15.175 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3210000000000004e-06 2024-09-21T21:42:15.175 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.175 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.175 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:15.175 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898540+0000", 2024-09-21T21:42:15.175 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071194246000000003 2024-09-21T21:42:15.175 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.175 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.176 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:15.176 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898598+0000", 2024-09-21T21:42:15.176 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7658999999999998e-05 2024-09-21T21:42:15.176 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.176 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.176 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:15.176 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898685+0000", 2024-09-21T21:42:15.176 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.6945000000000002e-05 2024-09-21T21:42:15.176 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:15.176 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:15.176 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:15.176 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.177 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.177 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13579 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 2531328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:15.177 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.827355+0000", 2024-09-21T21:42:15.177 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 24.284732183999999, 2024-09-21T21:42:15.177 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071650933, 2024-09-21T21:42:15.177 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:15.177 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:15.177 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:15.177 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:15.177 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:15.178 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:15.178 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13579 2024-09-21T21:42:15.178 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.178 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:15.178 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.178 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:15.179 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827355+0000", 2024-09-21T21:42:15.179 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:15.179 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.179 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.179 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:15.179 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827355+0000", 2024-09-21T21:42:15.179 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:15.180 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.180 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.181 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:15.181 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827486+0000", 2024-09-21T21:42:15.182 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00013066799999999999 2024-09-21T21:42:15.182 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.182 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.182 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:15.182 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827490+0000", 2024-09-21T21:42:15.182 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.9009999999999996e-06 2024-09-21T21:42:15.183 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.183 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.183 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:15.183 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827499+0000", 2024-09-21T21:42:15.183 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6209999999999999e-06 2024-09-21T21:42:15.183 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.184 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.184 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:15.184 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827506+0000", 2024-09-21T21:42:15.184 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6490000000000002e-06 2024-09-21T21:42:15.184 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.184 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.184 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:15.185 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898713+0000", 2024-09-21T21:42:15.185 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071206937999999997 2024-09-21T21:42:15.185 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.185 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.185 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:15.185 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898811+0000", 2024-09-21T21:42:15.186 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.8159999999999995e-05 2024-09-21T21:42:15.186 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.186 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.186 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:15.186 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899006+0000", 2024-09-21T21:42:15.186 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00019499599999999999 2024-09-21T21:42:15.186 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:15.187 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:15.187 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:15.187 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.187 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.187 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13580 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3219456~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:15.187 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.827522+0000", 2024-09-21T21:42:15.188 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 24.284565752999999, 2024-09-21T21:42:15.188 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071813298999999997, 2024-09-21T21:42:15.188 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:15.188 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:15.188 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:15.188 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:15.188 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:15.189 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:15.189 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13580 2024-09-21T21:42:15.189 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.189 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:15.189 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.189 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:15.190 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827522+0000", 2024-09-21T21:42:15.190 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:15.190 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.190 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.190 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:15.190 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827522+0000", 2024-09-21T21:42:15.190 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:15.191 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.191 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.191 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:15.191 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827898+0000", 2024-09-21T21:42:15.191 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000376876 2024-09-21T21:42:15.191 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.192 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.192 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:15.192 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827903+0000", 2024-09-21T21:42:15.192 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4129999999999999e-06 2024-09-21T21:42:15.192 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.192 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.193 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:15.193 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827911+0000", 2024-09-21T21:42:15.193 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.4540000000000007e-06 2024-09-21T21:42:15.193 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.193 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.193 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:15.194 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827917+0000", 2024-09-21T21:42:15.194 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2110000000000003e-06 2024-09-21T21:42:15.194 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.194 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.194 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:15.194 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899041+0000", 2024-09-21T21:42:15.194 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071123965999999997 2024-09-21T21:42:15.195 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.195 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.195 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:15.195 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899157+0000", 2024-09-21T21:42:15.195 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011573700000000001 2024-09-21T21:42:15.195 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.196 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.196 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:15.196 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899335+0000", 2024-09-21T21:42:15.196 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00017764199999999999 2024-09-21T21:42:15.196 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:15.196 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:15.197 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:15.197 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.197 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.197 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13581 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3248128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:15.197 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.827929+0000", 2024-09-21T21:42:15.197 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 24.284158353999999, 2024-09-21T21:42:15.198 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071730151000000006, 2024-09-21T21:42:15.198 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:15.198 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:15.198 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:15.198 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:15.198 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:15.199 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:15.199 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13581 2024-09-21T21:42:15.199 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.199 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:15.199 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.199 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:15.200 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827929+0000", 2024-09-21T21:42:15.200 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:15.200 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.200 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.200 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:15.200 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827929+0000", 2024-09-21T21:42:15.200 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:15.201 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.201 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.201 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:15.201 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.829943+0000", 2024-09-21T21:42:15.201 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0020143750000000001 2024-09-21T21:42:15.201 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.202 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.202 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:15.202 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.829948+0000", 2024-09-21T21:42:15.202 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9400000000000001e-06 2024-09-21T21:42:15.202 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.202 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.203 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:15.203 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.829960+0000", 2024-09-21T21:42:15.203 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.2123999999999999e-05 2024-09-21T21:42:15.203 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.203 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.203 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:15.203 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.829968+0000", 2024-09-21T21:42:15.203 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7000000000000008e-06 2024-09-21T21:42:15.204 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.204 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.204 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:15.204 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899370+0000", 2024-09-21T21:42:15.204 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.069401672999999997 2024-09-21T21:42:15.204 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.204 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.205 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:15.205 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899481+0000", 2024-09-21T21:42:15.205 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011145300000000001 2024-09-21T21:42:15.205 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.205 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.205 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:15.205 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899659+0000", 2024-09-21T21:42:15.205 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00017788600000000001 2024-09-21T21:42:15.205 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:15.205 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:15.205 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:15.205 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.206 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.206 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13595 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 1871872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:15.206 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.830180+0000", 2024-09-21T21:42:15.206 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 24.281907239999999, 2024-09-21T21:42:15.206 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071809244999999994, 2024-09-21T21:42:15.206 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:15.206 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:15.206 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:15.206 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:15.206 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:15.206 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:15.206 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13595 2024-09-21T21:42:15.206 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.206 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:15.207 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.207 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:15.207 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.830180+0000", 2024-09-21T21:42:15.207 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:15.207 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.207 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.207 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:15.207 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.830180+0000", 2024-09-21T21:42:15.207 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:15.207 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.207 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.207 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:15.208 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.831967+0000", 2024-09-21T21:42:15.208 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0017868350000000001 2024-09-21T21:42:15.208 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.208 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.208 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:15.208 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.831971+0000", 2024-09-21T21:42:15.208 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3950000000000003e-06 2024-09-21T21:42:15.208 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.208 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.208 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:15.208 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.831983+0000", 2024-09-21T21:42:15.208 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1815e-05 2024-09-21T21:42:15.208 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.208 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.209 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:15.209 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.831993+0000", 2024-09-21T21:42:15.209 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6339999999999996e-06 2024-09-21T21:42:15.209 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.209 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.209 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:15.209 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.901693+0000", 2024-09-21T21:42:15.209 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.069700082999999996 2024-09-21T21:42:15.209 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.209 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.209 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:15.209 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.901813+0000", 2024-09-21T21:42:15.210 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011971 2024-09-21T21:42:15.210 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.210 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.210 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:15.210 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.901989+0000", 2024-09-21T21:42:15.210 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000176773 2024-09-21T21:42:15.210 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:15.210 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:15.210 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:15.210 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.210 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.210 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13600 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 2617344~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:15.211 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.832109+0000", 2024-09-21T21:42:15.211 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 24.279978487000001, 2024-09-21T21:42:15.211 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071551828999999997, 2024-09-21T21:42:15.211 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:15.211 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:15.211 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:15.211 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:15.211 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:15.211 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:15.211 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13600 2024-09-21T21:42:15.211 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.211 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:15.211 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.212 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:15.212 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.832109+0000", 2024-09-21T21:42:15.212 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:15.212 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.212 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.212 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:15.212 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.832109+0000", 2024-09-21T21:42:15.212 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:15.212 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.212 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.212 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:15.212 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.832109+0000", 2024-09-21T21:42:15.212 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4999999999999998e-07 2024-09-21T21:42:15.212 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.213 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.213 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:15.213 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.832112+0000", 2024-09-21T21:42:15.213 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.5019999999999999e-06 2024-09-21T21:42:15.213 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.213 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.213 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:15.213 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.832118+0000", 2024-09-21T21:42:15.213 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6270000000000002e-06 2024-09-21T21:42:15.213 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.213 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.213 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:15.213 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.832124+0000", 2024-09-21T21:42:15.213 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1850000000000003e-06 2024-09-21T21:42:15.213 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.214 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.214 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:15.214 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.903366+0000", 2024-09-21T21:42:15.214 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071241961000000006 2024-09-21T21:42:15.214 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.214 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.214 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:15.214 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.903485+0000", 2024-09-21T21:42:15.214 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000119202 2024-09-21T21:42:15.214 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.214 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.214 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:15.214 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.903661+0000", 2024-09-21T21:42:15.214 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000175902 2024-09-21T21:42:15.215 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:15.215 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:15.215 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:15.215 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.215 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.215 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13601 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3334144~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:15.215 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.832130+0000", 2024-09-21T21:42:15.215 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 24.279957368000002, 2024-09-21T21:42:15.215 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071861727, 2024-09-21T21:42:15.215 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:15.215 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:15.215 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:15.215 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:15.215 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:15.216 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:15.216 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13601 2024-09-21T21:42:15.216 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.216 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:15.216 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.216 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:15.216 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.832130+0000", 2024-09-21T21:42:15.216 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:15.216 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.216 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.216 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:15.216 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.832130+0000", 2024-09-21T21:42:15.216 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:15.216 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.216 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.217 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:15.217 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.833939+0000", 2024-09-21T21:42:15.217 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0018094400000000001 2024-09-21T21:42:15.217 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.217 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.217 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:15.217 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.833944+0000", 2024-09-21T21:42:15.217 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5979999999999999e-06 2024-09-21T21:42:15.217 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.217 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.217 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:15.217 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.833953+0000", 2024-09-21T21:42:15.217 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.1460000000000002e-06 2024-09-21T21:42:15.217 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.218 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.218 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:15.218 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.833960+0000", 2024-09-21T21:42:15.218 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.7989999999999996e-06 2024-09-21T21:42:15.218 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.218 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.218 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:15.218 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.903693+0000", 2024-09-21T21:42:15.218 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.069732901 2024-09-21T21:42:15.218 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.218 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.218 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:15.219 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.903814+0000", 2024-09-21T21:42:15.219 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000121585 2024-09-21T21:42:15.219 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:15.219 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:15.219 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:15.219 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.903992+0000", 2024-09-21T21:42:15.219 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00017725799999999999 2024-09-21T21:42:15.219 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:15.219 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:15.219 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:15.219 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:15.219 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:15.219 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:15.221 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T21:42:15.221 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:15.219+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:15.322 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:15.320+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:15.326 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:15.326 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:42:15.326 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:15.327 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5626.0:12345 3.3 3.3615c263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:42:15.327 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:14.411831+0000", 2024-09-21T21:42:15.327 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.91401112600000001, 2024-09-21T21:42:15.327 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.914028595, 2024-09-21T21:42:15.327 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:15.327 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:15.327 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "waiting for peered", 2024-09-21T21:42:15.327 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:15.327 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5626", 2024-09-21T21:42:15.327 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:15.327 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 12345 2024-09-21T21:42:15.327 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:15.327 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:15.327 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:15.327 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:15.328 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411831+0000", 2024-09-21T21:42:15.328 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:15.328 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:15.328 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:15.328 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:15.328 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411831+0000", 2024-09-21T21:42:15.328 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:15.328 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:15.328 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:15.328 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:15.328 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411831+0000", 2024-09-21T21:42:15.328 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.08e-07 2024-09-21T21:42:15.328 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:15.328 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:15.329 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:15.329 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411833+0000", 2024-09-21T21:42:15.329 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3089999999999998e-06 2024-09-21T21:42:15.329 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:15.329 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:15.329 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:15.329 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411840+0000", 2024-09-21T21:42:15.329 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.472e-06 2024-09-21T21:42:15.329 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:15.329 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:15.329 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:15.330 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411845+0000", 2024-09-21T21:42:15.330 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7910000000000004e-06 2024-09-21T21:42:15.330 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:15.330 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:15.330 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:15.330 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411890+0000", 2024-09-21T21:42:15.330 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5763000000000002e-05 2024-09-21T21:42:15.330 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:15.330 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:15.330 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting_for_map not empty", 2024-09-21T21:42:15.330 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411896+0000", 2024-09-21T21:42:15.330 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0190000000000003e-06 2024-09-21T21:42:15.330 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:15.330 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:15.330 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:15.331 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415967+0000", 2024-09-21T21:42:15.331 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0040705749999999999 2024-09-21T21:42:15.331 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:15.331 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:15.331 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:15.331 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415974+0000", 2024-09-21T21:42:15.331 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3200000000000002e-06 2024-09-21T21:42:15.331 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:15.331 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:15.331 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:15.331 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:15.331 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:15.331 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5626.0:12334 3.3 3.3615c263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:42:15.331 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:14.411628+0000", 2024-09-21T21:42:15.331 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.91421336499999994, 2024-09-21T21:42:15.332 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.91428796999999995, 2024-09-21T21:42:15.332 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:15.332 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:15.332 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "waiting for peered", 2024-09-21T21:42:15.332 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:15.332 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5626", 2024-09-21T21:42:15.332 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:15.332 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 12334 2024-09-21T21:42:15.332 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:15.332 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:15.332 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:15.332 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:15.332 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411628+0000", 2024-09-21T21:42:15.332 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:15.333 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:15.333 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:15.333 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:15.333 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411628+0000", 2024-09-21T21:42:15.333 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:15.333 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:15.333 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:15.333 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:15.333 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411629+0000", 2024-09-21T21:42:15.333 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1200000000000002e-07 2024-09-21T21:42:15.333 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:15.333 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:15.333 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:15.333 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411633+0000", 2024-09-21T21:42:15.333 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1699999999999999e-06 2024-09-21T21:42:15.334 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:15.334 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:15.334 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:15.334 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411645+0000", 2024-09-21T21:42:15.334 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1328999999999999e-05 2024-09-21T21:42:15.334 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:15.334 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:15.334 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:15.334 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411652+0000", 2024-09-21T21:42:15.334 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9990000000000002e-06 2024-09-21T21:42:15.334 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:15.334 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:15.335 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:15.335 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411717+0000", 2024-09-21T21:42:15.335 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5519999999999996e-05 2024-09-21T21:42:15.335 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:15.335 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:15.335 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op must wait for map", 2024-09-21T21:42:15.335 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411725+0000", 2024-09-21T21:42:15.336 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2609999999999996e-06 2024-09-21T21:42:15.336 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:15.336 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:15.336 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:15.336 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415904+0000", 2024-09-21T21:42:15.336 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0041792749999999997 2024-09-21T21:42:15.336 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:15.336 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:15.337 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:15.337 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415913+0000", 2024-09-21T21:42:15.337 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.054e-06 2024-09-21T21:42:15.337 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:15.337 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:15.337 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:15.337 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:15.337 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:15.337 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5626.0:12339 3.3 3.3615c263 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:42:15.337 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:14.411749+0000", 2024-09-21T21:42:15.337 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.91409317899999998, 2024-09-21T21:42:15.337 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.91420821100000005, 2024-09-21T21:42:15.337 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:15.337 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:15.338 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "waiting for peered", 2024-09-21T21:42:15.338 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:15.338 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5626", 2024-09-21T21:42:15.338 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:15.338 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 12339 2024-09-21T21:42:15.338 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:15.338 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:15.338 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:15.338 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:15.338 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411749+0000", 2024-09-21T21:42:15.338 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:15.338 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:15.338 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:15.338 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:15.338 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411749+0000", 2024-09-21T21:42:15.339 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:15.339 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:15.339 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:15.339 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:15.339 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411749+0000", 2024-09-21T21:42:15.339 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0900000000000002e-07 2024-09-21T21:42:15.339 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:15.339 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:15.339 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:15.339 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411752+0000", 2024-09-21T21:42:15.339 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5610000000000001e-06 2024-09-21T21:42:15.340 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:15.340 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:15.340 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:15.340 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411758+0000", 2024-09-21T21:42:15.340 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1890000000000004e-06 2024-09-21T21:42:15.340 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:15.340 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:15.340 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:15.340 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411763+0000", 2024-09-21T21:42:15.340 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7589999999999997e-06 2024-09-21T21:42:15.340 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:15.340 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:15.340 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:15.340 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411803+0000", 2024-09-21T21:42:15.341 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9922999999999997e-05 2024-09-21T21:42:15.341 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:15.341 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:15.341 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting_for_map not empty", 2024-09-21T21:42:15.341 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411809+0000", 2024-09-21T21:42:15.341 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3779999999999998e-06 2024-09-21T21:42:15.341 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:15.341 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:15.341 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:15.341 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415937+0000", 2024-09-21T21:42:15.341 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0041276100000000003 2024-09-21T21:42:15.341 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:15.342 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:15.342 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:15.342 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415944+0000", 2024-09-21T21:42:15.342 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8329999999999994e-06 2024-09-21T21:42:15.342 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:15.342 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:15.342 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:15.342 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:15.342 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-21T21:42:15.342 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 3 2024-09-21T21:42:15.342 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:15.343 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T21:42:15.365 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T21:42:15.415 INFO:teuthology.orchestra.run.smithi081.stderr:nodeep-scrub is set 2024-09-21T21:42:15.427 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:15.426+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:15.491 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0: {} 2024-09-21T21:42:15.492 INFO:teuthology.orchestra.run.smithi081.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T21:42:15.497 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:15.497 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:42:15.498 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:42:15.498 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:42:15.498 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:15.506 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T21:42:15.508 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1: {} 2024-09-21T21:42:15.508 INFO:teuthology.orchestra.run.smithi081.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T21:42:15.523 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:15.522+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:15.524 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2: {} 2024-09-21T21:42:15.524 INFO:teuthology.orchestra.run.smithi081.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T21:42:15.541 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3: {} 2024-09-21T21:42:15.541 INFO:teuthology.orchestra.run.smithi081.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T21:42:15.557 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4: {} 2024-09-21T21:42:15.557 INFO:teuthology.orchestra.run.smithi081.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T21:42:15.574 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5: {} 2024-09-21T21:42:15.574 INFO:teuthology.orchestra.run.smithi081.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T21:42:15.624 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:15.623+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:15.650 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:15.651 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:15.659 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T21:42:15.725 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:15.724+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:15.803 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:15.803 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:15.812 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T21:42:15.826 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:15.824+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:15.913 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:15.914 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:15.922 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T21:42:15.926 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:15.925+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:16.025 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:16.025 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:16.027 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:16.026+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:16.034 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T21:42:16.128 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:16.127+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:16.177 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:16.177 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:16.185 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T21:42:16.229 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:16.228+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:16.287 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:16.287 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:16.296 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T21:42:16.339 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:16.338+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:16.398 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:16.399 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:16.407 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T21:42:16.413 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 5, 3] out_osds: [0, 4] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-21T21:42:16.414 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T21:42:16.414 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-21T21:42:16.414 INFO:tasks.thrashosds.thrasher:Setting osd 2 primary_affinity to 0.323014 2024-09-21T21:42:16.414 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 0.323014044388494 2024-09-21T21:42:16.435 INFO:teuthology.orchestra.run.smithi164.stdout:4987 punch from 0xd8d4200 to 0xd8d7200, (0x3000 bytes) 2024-09-21T21:42:16.435 INFO:teuthology.orchestra.run.smithi164.stdout:4989 write 0xa405600 thru 0xa411dc0 (0xc7c1 bytes) 2024-09-21T21:42:16.435 INFO:teuthology.orchestra.run.smithi164.stdout:4990 write 0x24d3200 thru 0x24e1537 (0xe338 bytes) 2024-09-21T21:42:16.439 INFO:teuthology.orchestra.run.smithi164.stdout:4991 read 0x4ccb600 thru 0x4cda59a (0xef9b bytes) 2024-09-21T21:42:16.439 INFO:teuthology.orchestra.run.smithi164.stdout:4992 write 0x72f9400 thru 0x73072d7 (0xded8 bytes) 2024-09-21T21:42:16.450 INFO:teuthology.orchestra.run.smithi164.stdout:4996 trunc from 0xed8e600 to 0x8fd0c00 2024-09-21T21:42:16.451 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:16.450+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:16.510 INFO:teuthology.orchestra.run.smithi164.stdout:4999 read 0x858be00 thru 0x85978de (0xbadf bytes) 2024-09-21T21:42:16.510 INFO:teuthology.orchestra.run.smithi164.stdout:5002 punch from 0x2e27800 to 0x2e2b000, (0x3800 bytes) 2024-09-21T21:42:16.510 INFO:teuthology.orchestra.run.smithi164.stdout:5003 write 0x877e800 thru 0x878583b (0x703c bytes) 2024-09-21T21:42:16.510 INFO:teuthology.orchestra.run.smithi164.stdout:5004 read 0x1414e00 thru 0x141b8f8 (0x6af9 bytes) 2024-09-21T21:42:16.516 INFO:teuthology.orchestra.run.smithi164.stdout:5005 trunc from 0x8fd0c00 to 0xc9eea00 2024-09-21T21:42:16.532 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:16.530+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:16.538 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:16.538 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:16.545 INFO:teuthology.orchestra.run.smithi164.stdout:5006 read 0x620a200 thru 0x621183b (0x763c bytes) 2024-09-21T21:42:16.548 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T21:42:16.549 INFO:teuthology.orchestra.run.smithi164.stdout:5007 trunc from 0xc9eea00 to 0xe7b8400 2024-09-21T21:42:16.558 INFO:teuthology.orchestra.run.smithi164.stdout:5008 write 0x1c96e00 thru 0x1c9b8f4 (0x4af5 bytes) 2024-09-21T21:42:16.564 INFO:teuthology.orchestra.run.smithi164.stdout:5010 write 0xb8b7c00 thru 0xb8c2125 (0xa526 bytes) 2024-09-21T21:42:16.564 INFO:teuthology.orchestra.run.smithi164.stdout:5012 write 0x5163000 thru 0x516b6bb (0x86bc bytes) 2024-09-21T21:42:16.564 INFO:teuthology.orchestra.run.smithi164.stdout:5013 punch from 0x195b800 to 0x195d400, (0x1c00 bytes) 2024-09-21T21:42:16.565 INFO:teuthology.orchestra.run.smithi164.stdout:5014 write 0xcd10800 thru 0xcd1f6a5 (0xeea6 bytes) 2024-09-21T21:42:16.565 INFO:teuthology.orchestra.run.smithi164.stdout:5015 punch from 0x3496a00 to 0x349a800, (0x3e00 bytes) 2024-09-21T21:42:16.575 INFO:teuthology.orchestra.run.smithi164.stdout:5016 trunc from 0xe7b8400 to 0x7477600 2024-09-21T21:42:16.633 INFO:teuthology.orchestra.run.smithi164.stdout:5021 write 0xfeee00 thru 0xffd6c0 (0xe8c1 bytes) 2024-09-21T21:42:16.633 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:16.632+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:16.638 INFO:teuthology.orchestra.run.smithi164.stdout:5022 read 0x1383e00 thru 0x138562a (0x182b bytes) 2024-09-21T21:42:16.639 INFO:teuthology.orchestra.run.smithi164.stdout:5023 read 0x598f200 thru 0x5994cf1 (0x5af2 bytes) 2024-09-21T21:42:16.639 INFO:teuthology.orchestra.run.smithi164.stdout:5025 read 0x521ac00 thru 0x5227fdd (0xd3de bytes) 2024-09-21T21:42:16.640 INFO:teuthology.orchestra.run.smithi164.stdout:5026 read 0x6a11e00 thru 0x6a1dd09 (0xbf0a bytes) 2024-09-21T21:42:16.640 INFO:teuthology.orchestra.run.smithi164.stdout:5027 write 0x7208200 thru 0x7213cb7 (0xbab8 bytes) 2024-09-21T21:42:16.640 INFO:teuthology.orchestra.run.smithi164.stdout:5033 read 0x6355e00 thru 0x635624e (0x44f bytes) 2024-09-21T21:42:16.646 INFO:teuthology.orchestra.run.smithi164.stdout:5034 trunc from 0x7477600 to 0xba49600 2024-09-21T21:42:16.665 INFO:teuthology.orchestra.run.smithi164.stdout:5035 read 0x20b8800 thru 0x20bd2ac (0x4aad bytes) 2024-09-21T21:42:16.667 INFO:teuthology.orchestra.run.smithi164.stdout:5037 write 0x148e00 thru 0x153c64 (0xae65 bytes) 2024-09-21T21:42:16.667 INFO:teuthology.orchestra.run.smithi164.stdout:5038 read 0x37a9800 thru 0x37b4ad7 (0xb2d8 bytes) 2024-09-21T21:42:16.675 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:16.675 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:16.686 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T21:42:16.687 INFO:teuthology.orchestra.run.smithi164.stdout:5039 write 0xee53600 thru 0xee5f842 (0xc243 bytes) 2024-09-21T21:42:16.701 INFO:teuthology.orchestra.run.smithi164.stdout:5040 trunc from 0xee5fa00 to 0x7361800 2024-09-21T21:42:16.733 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:16.732+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:16.756 INFO:teuthology.orchestra.run.smithi164.stdout:5042 write 0x8680800 thru 0x868de2c (0xd62d bytes) 2024-09-21T21:42:16.763 INFO:teuthology.orchestra.run.smithi164.stdout:5044 trunc from 0x868e000 to 0x99ce400 2024-09-21T21:42:16.797 INFO:teuthology.orchestra.run.smithi164.stdout:5046 trunc from 0x99ce400 to 0x391d800 2024-09-21T21:42:16.806 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:16.806 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:16.816 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T21:42:16.844 INFO:teuthology.orchestra.run.smithi164.stdout:5048 write 0xb704400 thru 0xb713fce (0xfbcf bytes) 2024-09-21T21:42:16.847 INFO:teuthology.orchestra.run.smithi164.stdout:5051 trunc from 0xb714000 to 0x9dea200 2024-09-21T21:42:16.858 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:16.857+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:16.883 INFO:teuthology.orchestra.run.smithi164.stdout:5052 punch from 0x731d400 to 0x7327e00, (0xaa00 bytes) 2024-09-21T21:42:16.885 INFO:teuthology.orchestra.run.smithi164.stdout:5053 punch from 0x9cac400 to 0x9cb9600, (0xd200 bytes) 2024-09-21T21:42:16.885 INFO:teuthology.orchestra.run.smithi164.stdout:5055 read 0x7090a00 thru 0x7099773 (0x8d74 bytes) 2024-09-21T21:42:16.921 INFO:teuthology.orchestra.run.smithi164.stdout:5059 write 0xa065200 thru 0xa07468d (0xf48e bytes) 2024-09-21T21:42:16.922 INFO:teuthology.orchestra.run.smithi164.stdout:5063 read 0x388a00 thru 0x3957c4 (0xcdc5 bytes) 2024-09-21T21:42:16.926 INFO:teuthology.orchestra.run.smithi164.stdout:5064 write 0x336f200 thru 0x3379851 (0xa652 bytes) 2024-09-21T21:42:16.926 INFO:teuthology.orchestra.run.smithi164.stdout:5066 punch from 0x88de400 to 0x88eb200, (0xce00 bytes) 2024-09-21T21:42:16.927 INFO:teuthology.orchestra.run.smithi164.stdout:5068 read 0x938e200 thru 0x939be69 (0xdc6a bytes) 2024-09-21T21:42:16.935 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:16.933+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:16.968 INFO:teuthology.orchestra.run.smithi164.stdout:5070 write 0xe5f7e00 thru 0xe5fcd17 (0x4f18 bytes) 2024-09-21T21:42:16.968 INFO:teuthology.orchestra.run.smithi164.stdout:5071 read 0x349da00 thru 0x34ad164 (0xf765 bytes) 2024-09-21T21:42:16.969 INFO:teuthology.orchestra.run.smithi164.stdout:5072 read 0x2213000 thru 0x2220fd3 (0xdfd4 bytes) 2024-09-21T21:42:16.970 INFO:teuthology.orchestra.run.smithi164.stdout:5073 write 0x24e9a00 thru 0x24eb60a (0x1c0b bytes) 2024-09-21T21:42:16.971 INFO:teuthology.orchestra.run.smithi164.stdout:5074 write 0x39ca00 thru 0x3a6f59 (0xa55a bytes) 2024-09-21T21:42:16.971 INFO:teuthology.orchestra.run.smithi164.stdout:5075 clone 38 order 21 su 32768 sc 10 2024-09-21T21:42:16.980 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:16.980 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:16.990 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T21:42:17.036 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:17.035+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:17.092 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:17.093 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:17.101 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T21:42:17.137 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:17.135+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:17.203 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:17.204 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:17.212 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T21:42:17.238 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:17.237+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:17.315 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:17.315 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:17.323 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T21:42:17.366 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:17.365+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:17.425 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:17.426 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:17.431 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T21:42:17.434 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T21:42:17.471 INFO:teuthology.orchestra.run.smithi081.stderr:set osd.2 primary-affinity to 0.323014 (8211692) 2024-09-21T21:42:17.472 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:17.470+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:17.541 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:17.539+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:17.581 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:17.581 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:17.585 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T21:42:17.590 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T21:42:17.641 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:17.640+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:17.698 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:17.698 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:17.705 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0: {} 2024-09-21T21:42:17.705 INFO:teuthology.orchestra.run.smithi081.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T21:42:17.708 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T21:42:17.722 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1: {} 2024-09-21T21:42:17.722 INFO:teuthology.orchestra.run.smithi081.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T21:42:17.738 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2: {} 2024-09-21T21:42:17.738 INFO:teuthology.orchestra.run.smithi081.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T21:42:17.753 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:17.752+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:17.755 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3: {} 2024-09-21T21:42:17.755 INFO:teuthology.orchestra.run.smithi081.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T21:42:17.771 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4: {} 2024-09-21T21:42:17.771 INFO:teuthology.orchestra.run.smithi081.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T21:42:17.790 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5: {} 2024-09-21T21:42:17.790 INFO:teuthology.orchestra.run.smithi081.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T21:42:17.816 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:17.816 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:42:17.816 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:42:17.817 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:42:17.817 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:17.825 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T21:42:17.843 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:17.841+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:17.943 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:17.942+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:17.970 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:17.971 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-21T21:42:17.971 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-21T21:42:17.971 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:42:17.971 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:17.971 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19747 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 7905280~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:17.971 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273058+0000", 2024-09-21T21:42:17.971 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 26.695171396999999, 2024-09-21T21:42:17.971 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97338591399999996, 2024-09-21T21:42:17.971 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:17.971 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:17.971 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:17.971 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:17.971 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:17.972 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:17.972 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19747 2024-09-21T21:42:17.972 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:17.972 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:17.972 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:17.972 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:17.972 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273058+0000", 2024-09-21T21:42:17.972 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:17.972 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:17.972 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:17.972 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:17.972 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273058+0000", 2024-09-21T21:42:17.972 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:17.972 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:17.972 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:17.973 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:17.973 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273058+0000", 2024-09-21T21:42:17.973 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5900000000000003e-07 2024-09-21T21:42:17.973 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:17.973 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:17.973 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:17.973 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273061+0000", 2024-09-21T21:42:17.973 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3479999999999998e-06 2024-09-21T21:42:17.973 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:17.973 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:17.974 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:17.974 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273067+0000", 2024-09-21T21:42:17.974 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1469999999999998e-06 2024-09-21T21:42:17.974 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:17.974 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:17.974 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:17.974 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273071+0000", 2024-09-21T21:42:17.974 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7339999999999999e-06 2024-09-21T21:42:17.974 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:17.974 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:17.974 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:17.974 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277945+0000", 2024-09-21T21:42:17.975 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.004873474 2024-09-21T21:42:17.975 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:17.975 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:17.975 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:17.975 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277948+0000", 2024-09-21T21:42:17.975 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0299999999999998e-06 2024-09-21T21:42:17.975 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:17.975 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:17.975 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:17.975 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246269+0000", 2024-09-21T21:42:17.975 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96832060200000003 2024-09-21T21:42:17.975 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:17.975 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:17.975 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:17.976 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246362+0000", 2024-09-21T21:42:17.976 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3867000000000001e-05 2024-09-21T21:42:17.976 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:17.976 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:17.976 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:17.976 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246444+0000", 2024-09-21T21:42:17.976 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1353000000000005e-05 2024-09-21T21:42:17.976 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:17.976 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:17.976 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:17.976 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:17.977 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:17.977 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19755 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 5783552~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:17.977 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273077+0000", 2024-09-21T21:42:17.977 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 26.695151781, 2024-09-21T21:42:17.977 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97350203400000002, 2024-09-21T21:42:17.977 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:17.977 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:17.977 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:17.977 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:17.977 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:17.977 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:17.977 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19755 2024-09-21T21:42:17.977 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:17.977 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:17.977 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:17.978 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:17.978 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273077+0000", 2024-09-21T21:42:17.978 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:17.978 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:17.978 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:17.978 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:17.978 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273077+0000", 2024-09-21T21:42:17.978 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:17.978 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:17.978 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:17.978 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:17.978 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273078+0000", 2024-09-21T21:42:17.978 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5699999999999998e-07 2024-09-21T21:42:17.978 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:17.978 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:17.979 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:17.979 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273083+0000", 2024-09-21T21:42:17.979 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8670000000000002e-06 2024-09-21T21:42:17.979 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:17.979 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:17.979 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:17.979 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273088+0000", 2024-09-21T21:42:17.979 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8370000000000002e-06 2024-09-21T21:42:17.979 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:17.979 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:17.979 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:17.979 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273093+0000", 2024-09-21T21:42:17.979 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5709999999999999e-06 2024-09-21T21:42:17.979 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:17.979 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:17.979 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:17.980 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277968+0000", 2024-09-21T21:42:17.980 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048753909999999998 2024-09-21T21:42:17.980 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:17.980 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:17.980 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:17.980 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277971+0000", 2024-09-21T21:42:17.980 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0189999999999998e-06 2024-09-21T21:42:17.980 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:17.980 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:17.980 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:17.980 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246485+0000", 2024-09-21T21:42:17.980 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96851399299999996 2024-09-21T21:42:17.980 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:17.980 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:17.980 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:17.981 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246536+0000", 2024-09-21T21:42:17.981 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0430000000000003e-05 2024-09-21T21:42:17.981 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:17.981 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:17.981 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:17.981 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246579+0000", 2024-09-21T21:42:17.981 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3569000000000002e-05 2024-09-21T21:42:17.981 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:17.981 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:17.981 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:17.981 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:17.981 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:17.981 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19756 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6500352~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:17.981 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273099+0000", 2024-09-21T21:42:17.981 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 26.695130408000001, 2024-09-21T21:42:17.982 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97359272799999996, 2024-09-21T21:42:17.982 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:17.982 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:17.982 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:17.982 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:17.982 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:17.982 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:17.982 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19756 2024-09-21T21:42:17.982 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:17.982 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:17.982 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:17.982 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:17.982 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273099+0000", 2024-09-21T21:42:17.982 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:17.982 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:17.982 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:17.983 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:17.983 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273099+0000", 2024-09-21T21:42:17.983 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:17.983 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:17.983 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:17.983 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:17.983 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273099+0000", 2024-09-21T21:42:17.983 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0900000000000002e-07 2024-09-21T21:42:17.983 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:17.983 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:17.983 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:17.984 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273101+0000", 2024-09-21T21:42:17.984 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0320000000000002e-06 2024-09-21T21:42:17.984 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:17.984 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:17.984 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:17.984 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273107+0000", 2024-09-21T21:42:17.984 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2830000000000001e-06 2024-09-21T21:42:17.984 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:17.984 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:17.984 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:17.984 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273111+0000", 2024-09-21T21:42:17.984 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2230000000000001e-06 2024-09-21T21:42:17.984 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:17.984 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:17.984 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:17.984 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277994+0000", 2024-09-21T21:42:17.985 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048837070000000002 2024-09-21T21:42:17.985 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:17.985 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:17.985 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:17.985 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278000+0000", 2024-09-21T21:42:17.985 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3149999999999999e-06 2024-09-21T21:42:17.985 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:17.985 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:17.985 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:17.985 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246607+0000", 2024-09-21T21:42:17.985 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.968606839 2024-09-21T21:42:17.985 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:17.985 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:17.985 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:17.985 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246651+0000", 2024-09-21T21:42:17.985 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4734999999999998e-05 2024-09-21T21:42:17.986 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:17.986 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:17.986 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:17.986 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246692+0000", 2024-09-21T21:42:17.986 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0185000000000002e-05 2024-09-21T21:42:17.986 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:17.986 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:17.986 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:17.986 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:17.986 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:17.987 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19757 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 5812224~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:17.987 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273116+0000", 2024-09-21T21:42:17.987 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 26.695112751, 2024-09-21T21:42:17.987 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97367813999999997, 2024-09-21T21:42:17.987 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:17.987 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:17.987 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:17.987 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:17.987 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:17.987 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:17.987 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19757 2024-09-21T21:42:17.987 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:17.987 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:17.987 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:17.987 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:17.988 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273116+0000", 2024-09-21T21:42:17.988 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:17.988 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:17.988 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:17.988 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:17.988 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273116+0000", 2024-09-21T21:42:17.988 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:17.988 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:17.988 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:17.988 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:17.988 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273117+0000", 2024-09-21T21:42:17.988 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0999999999999999e-07 2024-09-21T21:42:17.988 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:17.988 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:17.988 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:17.989 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273119+0000", 2024-09-21T21:42:17.989 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9970000000000001e-06 2024-09-21T21:42:17.989 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:17.989 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:17.989 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:17.989 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273125+0000", 2024-09-21T21:42:17.989 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9540000000000001e-06 2024-09-21T21:42:17.989 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:17.989 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:17.989 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:17.989 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273135+0000", 2024-09-21T21:42:17.989 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0597999999999999e-05 2024-09-21T21:42:17.989 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:17.989 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:17.990 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:17.990 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278019+0000", 2024-09-21T21:42:17.990 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048834849999999999 2024-09-21T21:42:17.990 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:17.990 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:17.990 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:17.990 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278022+0000", 2024-09-21T21:42:17.990 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9560000000000002e-06 2024-09-21T21:42:17.990 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:17.990 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:17.990 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:17.990 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246716+0000", 2024-09-21T21:42:17.990 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.968694202 2024-09-21T21:42:17.990 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:17.991 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:17.991 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:17.991 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246758+0000", 2024-09-21T21:42:17.991 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1607000000000003e-05 2024-09-21T21:42:17.991 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:17.991 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:17.991 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:17.991 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246795+0000", 2024-09-21T21:42:17.991 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6931000000000003e-05 2024-09-21T21:42:17.991 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:17.991 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:17.991 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:17.991 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:17.991 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:17.991 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19758 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6529024~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:17.992 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273142+0000", 2024-09-21T21:42:17.992 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 26.695087529999999, 2024-09-21T21:42:17.992 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97375924599999997, 2024-09-21T21:42:17.992 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:17.992 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:17.992 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:17.992 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:17.992 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:17.992 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:17.992 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19758 2024-09-21T21:42:17.992 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:17.992 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:17.992 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:17.992 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:17.992 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273142+0000", 2024-09-21T21:42:17.993 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:17.993 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:17.993 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:17.993 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:17.993 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273142+0000", 2024-09-21T21:42:17.993 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:17.993 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:17.993 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:17.993 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:17.993 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273142+0000", 2024-09-21T21:42:17.993 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0200000000000003e-07 2024-09-21T21:42:17.993 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:17.993 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:17.993 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:17.994 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273144+0000", 2024-09-21T21:42:17.994 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1390000000000002e-06 2024-09-21T21:42:17.994 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:17.994 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:17.994 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:17.994 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273150+0000", 2024-09-21T21:42:17.994 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6409999999999999e-06 2024-09-21T21:42:17.994 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:17.994 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:17.995 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:17.995 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273157+0000", 2024-09-21T21:42:17.995 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1359999999999999e-06 2024-09-21T21:42:17.995 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:17.995 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:17.995 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:17.995 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278050+0000", 2024-09-21T21:42:17.995 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048926170000000001 2024-09-21T21:42:17.995 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:17.995 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:17.995 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:17.995 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278053+0000", 2024-09-21T21:42:17.995 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4879999999999999e-06 2024-09-21T21:42:17.995 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:17.996 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:17.996 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:17.996 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246819+0000", 2024-09-21T21:42:17.996 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96876590399999996 2024-09-21T21:42:17.996 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:17.996 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:17.996 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:17.996 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246856+0000", 2024-09-21T21:42:17.996 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7287999999999999e-05 2024-09-21T21:42:17.996 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:17.996 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:17.996 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:17.996 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246901+0000", 2024-09-21T21:42:17.997 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4631e-05 2024-09-21T21:42:17.997 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:17.997 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:17.997 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:17.997 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:17.997 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:17.997 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19759 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 7933952~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:17.997 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273172+0000", 2024-09-21T21:42:17.997 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 26.695057265999999, 2024-09-21T21:42:17.997 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97384789500000002, 2024-09-21T21:42:17.997 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:17.998 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:17.998 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:17.998 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:17.998 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:17.998 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:17.998 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19759 2024-09-21T21:42:17.998 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:17.998 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:17.998 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:17.998 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:17.998 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273172+0000", 2024-09-21T21:42:17.998 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:17.998 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:17.998 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:17.999 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:17.999 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273172+0000", 2024-09-21T21:42:17.999 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:17.999 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:17.999 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:17.999 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:17.999 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273172+0000", 2024-09-21T21:42:17.999 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6100000000000002e-07 2024-09-21T21:42:17.999 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:17.999 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:17.999 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:17.999 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273174+0000", 2024-09-21T21:42:17.999 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.691e-06 2024-09-21T21:42:18.000 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.000 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.000 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:18.000 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273178+0000", 2024-09-21T21:42:18.000 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7579999999999998e-06 2024-09-21T21:42:18.000 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.000 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.000 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:18.000 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273180+0000", 2024-09-21T21:42:18.000 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7199999999999998e-06 2024-09-21T21:42:18.000 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.000 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.000 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:18.000 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278077+0000", 2024-09-21T21:42:18.001 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048969529999999999 2024-09-21T21:42:18.001 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.001 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.001 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:18.001 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278080+0000", 2024-09-21T21:42:18.001 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.066e-06 2024-09-21T21:42:18.001 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.001 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.001 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:18.001 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246925+0000", 2024-09-21T21:42:18.001 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96884490199999995 2024-09-21T21:42:18.001 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.001 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.001 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:18.001 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246974+0000", 2024-09-21T21:42:18.002 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8387999999999998e-05 2024-09-21T21:42:18.002 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.002 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.002 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:18.002 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247020+0000", 2024-09-21T21:42:18.002 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6156000000000003e-05 2024-09-21T21:42:18.002 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:18.002 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:18.002 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:18.002 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.002 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.002 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19760 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 7991296~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:18.002 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273220+0000", 2024-09-21T21:42:18.003 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 26.695008752, 2024-09-21T21:42:18.003 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97403125499999998, 2024-09-21T21:42:18.003 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:18.003 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:18.003 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:18.003 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:18.003 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:18.003 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:18.003 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19760 2024-09-21T21:42:18.003 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.003 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:18.003 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.003 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:18.003 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273220+0000", 2024-09-21T21:42:18.004 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:18.004 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.004 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.004 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:18.004 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273220+0000", 2024-09-21T21:42:18.004 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:18.004 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.004 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.004 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:18.004 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273221+0000", 2024-09-21T21:42:18.004 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.65e-07 2024-09-21T21:42:18.004 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.005 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.005 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:18.005 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273228+0000", 2024-09-21T21:42:18.005 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6040000000000004e-06 2024-09-21T21:42:18.005 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.005 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.005 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:18.005 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273233+0000", 2024-09-21T21:42:18.005 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1569999999999999e-06 2024-09-21T21:42:18.005 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.005 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.005 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:18.005 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273238+0000", 2024-09-21T21:42:18.005 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0050000000000004e-06 2024-09-21T21:42:18.005 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.005 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.006 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:18.006 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278107+0000", 2024-09-21T21:42:18.006 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00486857 2024-09-21T21:42:18.006 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.006 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.006 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:18.006 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278111+0000", 2024-09-21T21:42:18.006 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6389999999999998e-06 2024-09-21T21:42:18.006 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.006 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.006 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:18.006 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247069+0000", 2024-09-21T21:42:18.006 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96895796899999997 2024-09-21T21:42:18.006 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.006 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.007 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:18.007 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247166+0000", 2024-09-21T21:42:18.007 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.7158999999999996e-05 2024-09-21T21:42:18.007 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.007 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.007 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:18.007 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247252+0000", 2024-09-21T21:42:18.007 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5886999999999999e-05 2024-09-21T21:42:18.007 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:18.008 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:18.008 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:18.008 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.008 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.008 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19761 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 7962624~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:18.008 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273246+0000", 2024-09-21T21:42:18.008 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 26.694983692000001, 2024-09-21T21:42:18.008 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97415972200000001, 2024-09-21T21:42:18.008 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:18.008 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:18.008 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:18.008 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:18.008 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:18.008 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:18.008 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19761 2024-09-21T21:42:18.008 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.009 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:18.009 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.009 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:18.009 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273246+0000", 2024-09-21T21:42:18.009 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:18.009 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.009 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.009 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:18.009 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273246+0000", 2024-09-21T21:42:18.009 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:18.009 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.009 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.009 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:18.009 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273246+0000", 2024-09-21T21:42:18.009 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5100000000000001e-07 2024-09-21T21:42:18.009 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.010 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.010 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:18.010 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273248+0000", 2024-09-21T21:42:18.010 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1550000000000001e-06 2024-09-21T21:42:18.010 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.010 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.010 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:18.010 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273254+0000", 2024-09-21T21:42:18.010 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.964e-06 2024-09-21T21:42:18.010 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.010 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.010 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:18.010 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273259+0000", 2024-09-21T21:42:18.010 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5499999999999996e-06 2024-09-21T21:42:18.010 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.011 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.011 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:18.011 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278130+0000", 2024-09-21T21:42:18.011 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048715779999999997 2024-09-21T21:42:18.011 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.011 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.011 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:18.011 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278133+0000", 2024-09-21T21:42:18.011 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8739999999999999e-06 2024-09-21T21:42:18.011 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.011 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.011 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:18.011 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247270+0000", 2024-09-21T21:42:18.011 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.969137099 2024-09-21T21:42:18.011 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.011 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.012 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:18.012 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247338+0000", 2024-09-21T21:42:18.012 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7717000000000002e-05 2024-09-21T21:42:18.012 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.012 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.012 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:18.012 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247405+0000", 2024-09-21T21:42:18.012 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7434000000000005e-05 2024-09-21T21:42:18.012 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:18.012 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:18.012 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:18.012 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.012 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.012 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19769 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 5840896~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:18.012 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273287+0000", 2024-09-21T21:42:18.013 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 26.694942407999999, 2024-09-21T21:42:18.013 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97435390200000005, 2024-09-21T21:42:18.013 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:18.013 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:18.013 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:18.013 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:18.013 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:18.013 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:18.013 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19769 2024-09-21T21:42:18.013 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.013 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:18.013 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.013 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:18.013 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273287+0000", 2024-09-21T21:42:18.013 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:18.013 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.014 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.014 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:18.014 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273287+0000", 2024-09-21T21:42:18.014 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:18.014 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.014 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.014 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:18.014 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273287+0000", 2024-09-21T21:42:18.014 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6699999999999999e-07 2024-09-21T21:42:18.014 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.014 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.014 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:18.014 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273289+0000", 2024-09-21T21:42:18.014 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.708e-06 2024-09-21T21:42:18.014 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.015 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.015 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:18.015 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273296+0000", 2024-09-21T21:42:18.015 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8109999999999997e-06 2024-09-21T21:42:18.015 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.015 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.015 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:18.015 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273299+0000", 2024-09-21T21:42:18.015 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0740000000000001e-06 2024-09-21T21:42:18.015 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.016 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.016 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:18.016 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278159+0000", 2024-09-21T21:42:18.016 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048603240000000001 2024-09-21T21:42:18.016 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.016 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.016 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:18.016 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278162+0000", 2024-09-21T21:42:18.016 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1930000000000002e-06 2024-09-21T21:42:18.016 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.016 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.016 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:18.016 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247444+0000", 2024-09-21T21:42:18.016 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96928199100000001 2024-09-21T21:42:18.016 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.017 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.017 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:18.017 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247533+0000", 2024-09-21T21:42:18.017 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8490000000000001e-05 2024-09-21T21:42:18.017 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.017 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.017 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:18.017 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247641+0000", 2024-09-21T21:42:18.017 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000107944 2024-09-21T21:42:18.017 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:18.017 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:18.017 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:18.017 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.017 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.018 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19770 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 5869568~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:18.018 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273313+0000", 2024-09-21T21:42:18.018 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 26.694916306, 2024-09-21T21:42:18.018 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97449038899999996, 2024-09-21T21:42:18.018 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:18.018 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:18.018 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:18.018 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:18.018 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:18.018 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:18.018 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19770 2024-09-21T21:42:18.019 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.019 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:18.019 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.019 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:18.019 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273313+0000", 2024-09-21T21:42:18.019 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:18.019 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.019 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.019 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:18.019 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273313+0000", 2024-09-21T21:42:18.019 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:18.019 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.019 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.019 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:18.019 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273313+0000", 2024-09-21T21:42:18.020 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5699999999999998e-07 2024-09-21T21:42:18.020 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.020 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.020 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:18.020 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273315+0000", 2024-09-21T21:42:18.020 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0099999999999998e-06 2024-09-21T21:42:18.020 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.020 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.020 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:18.020 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273321+0000", 2024-09-21T21:42:18.020 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7529999999999998e-06 2024-09-21T21:42:18.020 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.020 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.020 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:18.020 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273328+0000", 2024-09-21T21:42:18.021 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9009999999999997e-06 2024-09-21T21:42:18.021 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.021 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.021 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:18.021 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278183+0000", 2024-09-21T21:42:18.021 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048547770000000002 2024-09-21T21:42:18.021 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.021 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.021 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:18.021 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278186+0000", 2024-09-21T21:42:18.021 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3100000000000001e-06 2024-09-21T21:42:18.021 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.021 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.021 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:18.021 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247676+0000", 2024-09-21T21:42:18.021 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96949004299999997 2024-09-21T21:42:18.022 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.022 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.022 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:18.022 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247757+0000", 2024-09-21T21:42:18.022 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0479e-05 2024-09-21T21:42:18.022 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.022 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.022 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:18.022 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247803+0000", 2024-09-21T21:42:18.022 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6758999999999997e-05 2024-09-21T21:42:18.022 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:18.022 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:18.022 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:18.022 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.022 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.023 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19771 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6586368~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:18.023 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273335+0000", 2024-09-21T21:42:18.023 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 26.694894519000002, 2024-09-21T21:42:18.023 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97458947699999998, 2024-09-21T21:42:18.023 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:18.023 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:18.023 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:18.023 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:18.023 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:18.023 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:18.023 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19771 2024-09-21T21:42:18.023 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.023 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:18.023 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.023 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:18.023 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273335+0000", 2024-09-21T21:42:18.023 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:18.024 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.024 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.024 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:18.024 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273335+0000", 2024-09-21T21:42:18.024 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:18.024 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.024 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.024 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:18.024 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273335+0000", 2024-09-21T21:42:18.024 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3599999999999999e-07 2024-09-21T21:42:18.024 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.024 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.024 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:18.024 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273337+0000", 2024-09-21T21:42:18.024 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0789999999999999e-06 2024-09-21T21:42:18.024 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.025 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.025 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:18.025 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273343+0000", 2024-09-21T21:42:18.025 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6280000000000002e-06 2024-09-21T21:42:18.025 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.025 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.025 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:18.025 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273346+0000", 2024-09-21T21:42:18.025 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.507e-06 2024-09-21T21:42:18.025 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.025 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.025 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:18.026 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278206+0000", 2024-09-21T21:42:18.026 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048600969999999999 2024-09-21T21:42:18.026 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.026 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.026 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:18.026 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278212+0000", 2024-09-21T21:42:18.026 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2079999999999999e-06 2024-09-21T21:42:18.026 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.026 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.026 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:18.026 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247828+0000", 2024-09-21T21:42:18.026 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.969616547 2024-09-21T21:42:18.027 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.027 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.027 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:18.027 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247881+0000", 2024-09-21T21:42:18.027 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3180999999999998e-05 2024-09-21T21:42:18.027 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.027 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.027 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:18.027 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247924+0000", 2024-09-21T21:42:18.027 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2894e-05 2024-09-21T21:42:18.027 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:18.027 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:18.027 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:18.027 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.028 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.028 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19772 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6557696~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:18.028 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273358+0000", 2024-09-21T21:42:18.028 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 26.694871139, 2024-09-21T21:42:18.028 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97467031800000004, 2024-09-21T21:42:18.028 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:18.028 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:18.028 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:18.028 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:18.028 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:18.029 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:18.029 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19772 2024-09-21T21:42:18.029 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.029 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:18.029 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.029 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:18.029 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273358+0000", 2024-09-21T21:42:18.029 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:18.029 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.029 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.029 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:18.029 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273358+0000", 2024-09-21T21:42:18.029 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:18.030 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.030 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.030 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:18.030 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273358+0000", 2024-09-21T21:42:18.030 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4499999999999998e-07 2024-09-21T21:42:18.030 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.030 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.030 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:18.031 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273360+0000", 2024-09-21T21:42:18.031 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9190000000000002e-06 2024-09-21T21:42:18.031 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.031 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.031 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:18.031 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273366+0000", 2024-09-21T21:42:18.031 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.891e-06 2024-09-21T21:42:18.031 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.031 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.031 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:18.031 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273371+0000", 2024-09-21T21:42:18.031 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1880000000000001e-06 2024-09-21T21:42:18.031 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.031 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.032 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:18.032 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278237+0000", 2024-09-21T21:42:18.032 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.004865472 2024-09-21T21:42:18.032 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.032 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.032 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:18.032 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278240+0000", 2024-09-21T21:42:18.032 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9550000000000001e-06 2024-09-21T21:42:18.032 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.032 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.032 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:18.032 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247949+0000", 2024-09-21T21:42:18.032 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.969709127 2024-09-21T21:42:18.032 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.032 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.033 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:18.033 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247990+0000", 2024-09-21T21:42:18.033 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1202000000000003e-05 2024-09-21T21:42:18.033 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.033 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.033 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:18.033 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248028+0000", 2024-09-21T21:42:18.033 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8219000000000001e-05 2024-09-21T21:42:18.033 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:18.033 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:18.033 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:18.033 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.033 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.033 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19773 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6615040~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:18.033 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273390+0000", 2024-09-21T21:42:18.034 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 26.694839278, 2024-09-21T21:42:18.034 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97476086200000001, 2024-09-21T21:42:18.034 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:18.034 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:18.034 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:18.034 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:18.034 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:18.034 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:18.034 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19773 2024-09-21T21:42:18.034 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.034 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:18.034 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.035 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:18.035 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273390+0000", 2024-09-21T21:42:18.035 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:18.035 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.035 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.035 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:18.035 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273390+0000", 2024-09-21T21:42:18.035 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:18.035 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.035 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.035 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:18.035 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273390+0000", 2024-09-21T21:42:18.035 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9999999999999999e-07 2024-09-21T21:42:18.036 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.036 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.036 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:18.036 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273392+0000", 2024-09-21T21:42:18.036 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0640000000000001e-06 2024-09-21T21:42:18.036 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.036 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.036 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:18.036 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273397+0000", 2024-09-21T21:42:18.036 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.113e-06 2024-09-21T21:42:18.036 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.036 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.037 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:18.037 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273401+0000", 2024-09-21T21:42:18.037 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3119999999999998e-06 2024-09-21T21:42:18.037 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.037 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.037 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:18.037 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278271+0000", 2024-09-21T21:42:18.037 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048698880000000002 2024-09-21T21:42:18.037 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.037 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.037 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:18.037 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278278+0000", 2024-09-21T21:42:18.037 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1210000000000001e-06 2024-09-21T21:42:18.037 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.037 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.038 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:18.038 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248060+0000", 2024-09-21T21:42:18.038 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96978224999999996 2024-09-21T21:42:18.038 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.038 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.038 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:18.038 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248111+0000", 2024-09-21T21:42:18.038 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.096e-05 2024-09-21T21:42:18.038 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.038 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.038 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:18.038 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248151+0000", 2024-09-21T21:42:18.039 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9854000000000002e-05 2024-09-21T21:42:18.039 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:18.039 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:18.039 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:18.039 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.039 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.039 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19784 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6643712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:18.039 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273405+0000", 2024-09-21T21:42:18.039 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 26.694824557, 2024-09-21T21:42:18.039 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97491798900000004, 2024-09-21T21:42:18.039 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:18.039 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:18.039 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:18.040 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:18.040 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:18.040 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:18.040 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19784 2024-09-21T21:42:18.040 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.040 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:18.040 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.040 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:18.040 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273405+0000", 2024-09-21T21:42:18.040 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:18.040 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.040 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.040 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:18.040 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273405+0000", 2024-09-21T21:42:18.041 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:18.041 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.041 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.041 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:18.041 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273405+0000", 2024-09-21T21:42:18.041 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2100000000000001e-07 2024-09-21T21:42:18.041 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.041 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.041 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:18.041 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273406+0000", 2024-09-21T21:42:18.041 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2049999999999999e-06 2024-09-21T21:42:18.041 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.041 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.041 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:18.042 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273409+0000", 2024-09-21T21:42:18.042 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0599999999999999e-06 2024-09-21T21:42:18.042 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.042 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.042 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:18.042 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273412+0000", 2024-09-21T21:42:18.042 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0730000000000001e-06 2024-09-21T21:42:18.042 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.042 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.042 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:18.042 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278317+0000", 2024-09-21T21:42:18.042 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0049045520000000004 2024-09-21T21:42:18.042 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.043 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.043 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:18.043 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278321+0000", 2024-09-21T21:42:18.043 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6249999999999998e-06 2024-09-21T21:42:18.043 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.043 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.043 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:18.043 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248179+0000", 2024-09-21T21:42:18.043 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96985744500000004 2024-09-21T21:42:18.043 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.043 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.043 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:18.043 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248253+0000", 2024-09-21T21:42:18.043 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3829000000000001e-05 2024-09-21T21:42:18.044 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.044 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.044 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:18.044 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248323+0000", 2024-09-21T21:42:18.044 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9979000000000002e-05 2024-09-21T21:42:18.044 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:18.044 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:18.044 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:18.044 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.044 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.044 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19785 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6672384~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:18.044 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273415+0000", 2024-09-21T21:42:18.044 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 26.694813970999999, 2024-09-21T21:42:18.045 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97500777800000005, 2024-09-21T21:42:18.045 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:18.045 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:18.045 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:18.045 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:18.045 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:18.045 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:18.045 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19785 2024-09-21T21:42:18.045 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.045 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:18.045 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.045 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:18.045 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273415+0000", 2024-09-21T21:42:18.045 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:18.045 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.046 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.046 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:18.046 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273415+0000", 2024-09-21T21:42:18.046 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:18.046 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.046 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.046 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:18.046 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273415+0000", 2024-09-21T21:42:18.046 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0100000000000001e-07 2024-09-21T21:42:18.047 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.047 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.047 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:18.047 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273417+0000", 2024-09-21T21:42:18.047 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.114e-06 2024-09-21T21:42:18.047 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.047 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.047 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:18.047 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273420+0000", 2024-09-21T21:42:18.047 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6770000000000001e-06 2024-09-21T21:42:18.047 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.047 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.047 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:18.047 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273423+0000", 2024-09-21T21:42:18.048 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5450000000000002e-06 2024-09-21T21:42:18.048 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.048 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.048 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:18.048 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278355+0000", 2024-09-21T21:42:18.048 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0049318590000000002 2024-09-21T21:42:18.048 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.048 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.048 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:18.048 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278360+0000", 2024-09-21T21:42:18.048 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3079999999999998e-06 2024-09-21T21:42:18.048 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.048 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.048 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:18.048 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248349+0000", 2024-09-21T21:42:18.049 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96998909899999997 2024-09-21T21:42:18.049 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.049 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.049 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:18.049 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248387+0000", 2024-09-21T21:42:18.049 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7657000000000002e-05 2024-09-21T21:42:18.049 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.049 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.049 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:18.049 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248423+0000", 2024-09-21T21:42:18.050 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6318e-05 2024-09-21T21:42:18.050 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:18.050 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:18.050 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:18.050 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.050 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.050 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19842 3.d 3:b7fa2ca0:::rbd_data.112157a479c.000000000000004d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:18.050 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273426+0000", 2024-09-21T21:42:18.050 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 26.694802919000001, 2024-09-21T21:42:18.050 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.98136526800000001, 2024-09-21T21:42:18.050 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:18.050 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:18.050 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:18.051 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:18.051 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:18.051 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:18.051 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19842 2024-09-21T21:42:18.051 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.051 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:18.051 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.051 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:18.051 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273426+0000", 2024-09-21T21:42:18.051 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:18.051 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.051 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.051 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:18.051 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273426+0000", 2024-09-21T21:42:18.051 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:18.052 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.052 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.052 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:18.052 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273427+0000", 2024-09-21T21:42:18.052 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.72e-07 2024-09-21T21:42:18.052 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.052 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.052 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:18.052 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275836+0000", 2024-09-21T21:42:18.052 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00240909 2024-09-21T21:42:18.052 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.052 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.052 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:18.052 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275844+0000", 2024-09-21T21:42:18.053 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3750000000000003e-06 2024-09-21T21:42:18.053 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.053 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.053 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:18.053 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275849+0000", 2024-09-21T21:42:18.053 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0100000000000003e-06 2024-09-21T21:42:18.053 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.053 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.053 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:18.053 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278394+0000", 2024-09-21T21:42:18.053 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025449890000000001 2024-09-21T21:42:18.053 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.053 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.054 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:18.054 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278402+0000", 2024-09-21T21:42:18.054 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6359999999999994e-06 2024-09-21T21:42:18.054 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.054 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.054 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:18.054 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248454+0000", 2024-09-21T21:42:18.054 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97005259499999996 2024-09-21T21:42:18.054 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.054 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.054 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:18.054 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248576+0000", 2024-09-21T21:42:18.054 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000122127 2024-09-21T21:42:18.054 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.054 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.055 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 2", 2024-09-21T21:42:18.055 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248844+0000", 2024-09-21T21:42:18.055 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00026784699999999999 2024-09-21T21:42:18.055 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.055 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.055 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:18.055 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.252487+0000", 2024-09-21T21:42:18.055 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0036427920000000002 2024-09-21T21:42:18.055 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.055 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.055 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:18.055 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.254691+0000", 2024-09-21T21:42:18.055 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002203899 2024-09-21T21:42:18.056 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.056 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.056 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:18.056 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.254748+0000", 2024-09-21T21:42:18.056 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7163000000000001e-05 2024-09-21T21:42:18.056 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.056 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.056 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:18.056 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.254792+0000", 2024-09-21T21:42:18.056 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3473000000000002e-05 2024-09-21T21:42:18.056 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:18.056 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:18.057 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:18.057 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.057 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.057 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19843 3.d 3:b7fa2ca0:::rbd_data.112157a479c.000000000000004d:head [write 0~0] snapc 21=[21] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:18.057 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.275860+0000", 2024-09-21T21:42:18.057 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 26.692369019000001, 2024-09-21T21:42:18.057 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.98183651999999999, 2024-09-21T21:42:18.057 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:18.057 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:18.057 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:18.057 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:18.057 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:18.057 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:18.058 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19843 2024-09-21T21:42:18.058 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.058 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:18.058 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.058 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:18.058 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275860+0000", 2024-09-21T21:42:18.058 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:18.058 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.058 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.058 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:18.058 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275860+0000", 2024-09-21T21:42:18.058 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:18.058 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.059 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.059 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:18.059 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275869+0000", 2024-09-21T21:42:18.059 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6300000000000004e-06 2024-09-21T21:42:18.059 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.059 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.059 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:18.059 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275871+0000", 2024-09-21T21:42:18.059 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2060000000000001e-06 2024-09-21T21:42:18.059 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.059 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.060 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:18.060 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275876+0000", 2024-09-21T21:42:18.060 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8180000000000003e-06 2024-09-21T21:42:18.060 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.060 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.060 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:18.060 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275880+0000", 2024-09-21T21:42:18.060 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1829999999999996e-06 2024-09-21T21:42:18.060 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.060 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.060 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:18.060 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278416+0000", 2024-09-21T21:42:18.060 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025361149999999998 2024-09-21T21:42:18.060 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.060 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.061 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:18.061 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278419+0000", 2024-09-21T21:42:18.061 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.163e-06 2024-09-21T21:42:18.061 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.061 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.061 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:18.061 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.250249+0000", 2024-09-21T21:42:18.061 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97182984299999997 2024-09-21T21:42:18.061 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.061 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.061 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:18.061 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.250321+0000", 2024-09-21T21:42:18.061 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1656000000000006e-05 2024-09-21T21:42:18.061 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.061 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.061 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 2", 2024-09-21T21:42:18.062 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.250419+0000", 2024-09-21T21:42:18.062 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.8277999999999997e-05 2024-09-21T21:42:18.062 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.062 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.062 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:18.062 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.252516+0000", 2024-09-21T21:42:18.062 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002097224 2024-09-21T21:42:18.062 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.062 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.062 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:18.062 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.257643+0000", 2024-09-21T21:42:18.062 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0051262319999999997 2024-09-21T21:42:18.062 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.062 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.063 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:18.063 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.257673+0000", 2024-09-21T21:42:18.063 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0023e-05 2024-09-21T21:42:18.063 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.063 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.063 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:18.063 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.257697+0000", 2024-09-21T21:42:18.063 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4148999999999999e-05 2024-09-21T21:42:18.063 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:18.063 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:18.063 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:18.063 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.063 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.063 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5626.0:12334 3.3 3:c643a86c:::rbd_data.1121faebdfe3.0000000000000002:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:42:18.064 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:14.411628+0000", 2024-09-21T21:42:18.064 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 3.55660072, 2024-09-21T21:42:18.064 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0086947559999999, 2024-09-21T21:42:18.064 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:18.064 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:18.064 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:18.064 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:18.064 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5626", 2024-09-21T21:42:18.064 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:18.064 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 12334 2024-09-21T21:42:18.064 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.064 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:18.064 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.064 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:18.064 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411628+0000", 2024-09-21T21:42:18.064 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:18.065 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.065 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.065 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:18.065 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411628+0000", 2024-09-21T21:42:18.065 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:18.065 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.065 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.065 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:18.066 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411629+0000", 2024-09-21T21:42:18.066 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1200000000000002e-07 2024-09-21T21:42:18.066 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.066 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.066 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:18.066 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411633+0000", 2024-09-21T21:42:18.066 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1699999999999999e-06 2024-09-21T21:42:18.066 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.066 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.067 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:18.067 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411645+0000", 2024-09-21T21:42:18.067 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1328999999999999e-05 2024-09-21T21:42:18.067 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.067 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.067 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:18.067 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411652+0000", 2024-09-21T21:42:18.067 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9990000000000002e-06 2024-09-21T21:42:18.067 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.067 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.068 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:18.068 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411717+0000", 2024-09-21T21:42:18.068 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5519999999999996e-05 2024-09-21T21:42:18.068 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.068 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.068 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op must wait for map", 2024-09-21T21:42:18.069 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411725+0000", 2024-09-21T21:42:18.069 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2609999999999996e-06 2024-09-21T21:42:18.069 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.069 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.069 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:18.069 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415904+0000", 2024-09-21T21:42:18.069 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0041792749999999997 2024-09-21T21:42:18.069 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.069 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.069 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:18.069 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415913+0000", 2024-09-21T21:42:18.069 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.054e-06 2024-09-21T21:42:18.069 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.069 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.069 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:18.070 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420096+0000", 2024-09-21T21:42:18.070 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0041835670000001 2024-09-21T21:42:18.070 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.070 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.070 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:18.070 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420216+0000", 2024-09-21T21:42:18.070 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000120025 2024-09-21T21:42:18.070 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.071 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.071 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:18.071 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420323+0000", 2024-09-21T21:42:18.071 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010674400000000001 2024-09-21T21:42:18.071 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:18.071 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:18.071 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:18.071 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.071 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.071 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5626.0:12339 3.3 3:c643a86c:::rbd_data.1121faebdfe3.0000000000000002:head [read 4096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:42:18.071 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:14.411749+0000", 2024-09-21T21:42:18.071 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 3.5564805339999999, 2024-09-21T21:42:18.071 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.008818056, 2024-09-21T21:42:18.071 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:18.072 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:18.072 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:18.072 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:18.072 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5626", 2024-09-21T21:42:18.072 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:18.072 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 12339 2024-09-21T21:42:18.072 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.072 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:18.072 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.072 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:18.072 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411749+0000", 2024-09-21T21:42:18.072 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:18.072 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.072 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.073 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:18.073 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411749+0000", 2024-09-21T21:42:18.073 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:18.073 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.073 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.073 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:18.073 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411749+0000", 2024-09-21T21:42:18.073 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0900000000000002e-07 2024-09-21T21:42:18.073 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.073 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.073 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:18.073 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411752+0000", 2024-09-21T21:42:18.073 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5610000000000001e-06 2024-09-21T21:42:18.073 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.074 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.074 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:18.074 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411758+0000", 2024-09-21T21:42:18.074 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1890000000000004e-06 2024-09-21T21:42:18.074 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.074 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.074 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:18.074 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411763+0000", 2024-09-21T21:42:18.074 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7589999999999997e-06 2024-09-21T21:42:18.074 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.074 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.074 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:18.074 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411803+0000", 2024-09-21T21:42:18.074 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9922999999999997e-05 2024-09-21T21:42:18.075 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.075 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.075 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting_for_map not empty", 2024-09-21T21:42:18.075 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411809+0000", 2024-09-21T21:42:18.075 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3779999999999998e-06 2024-09-21T21:42:18.075 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.075 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.075 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:18.075 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415937+0000", 2024-09-21T21:42:18.075 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0041276100000000003 2024-09-21T21:42:18.075 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.075 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.076 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:18.076 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415944+0000", 2024-09-21T21:42:18.076 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8329999999999994e-06 2024-09-21T21:42:18.076 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.076 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.076 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:18.076 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420411+0000", 2024-09-21T21:42:18.076 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0044667300000001 2024-09-21T21:42:18.076 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.076 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.076 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:18.076 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420475+0000", 2024-09-21T21:42:18.076 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4356000000000005e-05 2024-09-21T21:42:18.076 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.077 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.077 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:18.077 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420567+0000", 2024-09-21T21:42:18.077 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.1308000000000001e-05 2024-09-21T21:42:18.077 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:18.077 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:18.077 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:18.077 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.077 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.077 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5626.0:12345 3.3 3:c643a86c:::rbd_data.1121faebdfe3.0000000000000002:head [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:42:18.077 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:14.411831+0000", 2024-09-21T21:42:18.078 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 3.556398481, 2024-09-21T21:42:18.078 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0089163649999999, 2024-09-21T21:42:18.078 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:18.078 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:18.078 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:18.078 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:18.078 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5626", 2024-09-21T21:42:18.078 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:18.078 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 12345 2024-09-21T21:42:18.078 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.078 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:18.078 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.078 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:18.079 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411831+0000", 2024-09-21T21:42:18.079 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:18.079 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.079 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.079 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:18.079 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411831+0000", 2024-09-21T21:42:18.079 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:18.079 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.079 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.079 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:18.079 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411831+0000", 2024-09-21T21:42:18.079 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.08e-07 2024-09-21T21:42:18.079 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.080 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.080 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:18.080 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411833+0000", 2024-09-21T21:42:18.080 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3089999999999998e-06 2024-09-21T21:42:18.080 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.080 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.080 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:18.080 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411840+0000", 2024-09-21T21:42:18.080 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.472e-06 2024-09-21T21:42:18.080 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.081 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.081 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:18.081 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411845+0000", 2024-09-21T21:42:18.081 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7910000000000004e-06 2024-09-21T21:42:18.081 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.081 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.081 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:18.081 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411890+0000", 2024-09-21T21:42:18.081 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5763000000000002e-05 2024-09-21T21:42:18.082 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.082 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.082 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting_for_map not empty", 2024-09-21T21:42:18.082 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411896+0000", 2024-09-21T21:42:18.082 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0190000000000003e-06 2024-09-21T21:42:18.082 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.082 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.082 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:18.082 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415967+0000", 2024-09-21T21:42:18.082 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0040705749999999999 2024-09-21T21:42:18.082 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.083 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.083 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:18.083 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415974+0000", 2024-09-21T21:42:18.083 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3200000000000002e-06 2024-09-21T21:42:18.083 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.083 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.083 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:18.083 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420594+0000", 2024-09-21T21:42:18.083 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.004620093 2024-09-21T21:42:18.083 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.083 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.083 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:18.083 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420658+0000", 2024-09-21T21:42:18.083 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3678999999999998e-05 2024-09-21T21:42:18.084 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.084 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.084 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:18.084 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420747+0000", 2024-09-21T21:42:18.084 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8936000000000004e-05 2024-09-21T21:42:18.084 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:18.084 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:18.084 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:18.084 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:18.084 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:18.084 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:18.086 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T21:42:18.086 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:18.084+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:18.186 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:18.185+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:18.230 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:18.230 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:42:18.230 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-21T21:42:18.230 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:18.238 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T21:42:18.287 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:18.286+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:18.382 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:18.383 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:42:18.383 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:42:18.383 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:42:18.383 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:18.388 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:18.387+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:18.391 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T21:42:18.475 INFO:teuthology.orchestra.run.smithi081.stderr:noscrub is unset 2024-09-21T21:42:18.490 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:18.488+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:18.539 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:18.540 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-21T21:42:18.540 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-21T21:42:18.540 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:42:18.540 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.540 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6974 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5246976~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:18.540 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.922932+0000", 2024-09-21T21:42:18.540 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 48.614489466000002, 2024-09-21T21:42:18.540 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041773352999999999, 2024-09-21T21:42:18.540 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:18.540 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:18.540 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:18.540 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:18.540 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:18.541 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:18.541 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6974 2024-09-21T21:42:18.541 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.541 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:18.541 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.541 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:18.541 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922932+0000", 2024-09-21T21:42:18.541 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:18.541 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.541 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.541 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:18.541 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922932+0000", 2024-09-21T21:42:18.541 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:18.541 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.541 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.542 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:18.542 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922933+0000", 2024-09-21T21:42:18.542 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.46e-07 2024-09-21T21:42:18.542 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.542 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.542 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:18.542 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922940+0000", 2024-09-21T21:42:18.542 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6140000000000003e-06 2024-09-21T21:42:18.542 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.542 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.542 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:18.542 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922949+0000", 2024-09-21T21:42:18.542 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6899999999999998e-06 2024-09-21T21:42:18.542 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.543 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.543 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:18.543 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922955+0000", 2024-09-21T21:42:18.543 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5140000000000001e-06 2024-09-21T21:42:18.543 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.543 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.543 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:18.543 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964547+0000", 2024-09-21T21:42:18.543 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041592854999999998 2024-09-21T21:42:18.543 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.543 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.543 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:18.543 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964617+0000", 2024-09-21T21:42:18.543 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9058000000000002e-05 2024-09-21T21:42:18.544 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.544 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.544 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:18.544 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964706+0000", 2024-09-21T21:42:18.544 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9276000000000004e-05 2024-09-21T21:42:18.544 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:18.544 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:18.544 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:18.544 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.544 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.544 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6976 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5261312~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:18.544 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.922961+0000", 2024-09-21T21:42:18.544 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 48.614460420999997, 2024-09-21T21:42:18.544 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041863648000000003, 2024-09-21T21:42:18.544 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:18.545 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:18.545 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:18.545 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:18.545 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:18.545 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:18.545 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6976 2024-09-21T21:42:18.545 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.545 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:18.545 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.545 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:18.545 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922961+0000", 2024-09-21T21:42:18.545 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:18.545 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.545 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.546 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:18.546 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922961+0000", 2024-09-21T21:42:18.546 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:18.546 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.546 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.546 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:18.546 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922962+0000", 2024-09-21T21:42:18.546 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.89e-07 2024-09-21T21:42:18.546 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.546 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.546 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:18.546 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922964+0000", 2024-09-21T21:42:18.546 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1270000000000001e-06 2024-09-21T21:42:18.546 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.547 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.547 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:18.547 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922970+0000", 2024-09-21T21:42:18.547 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8030000000000002e-06 2024-09-21T21:42:18.547 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.547 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.547 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:18.547 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922979+0000", 2024-09-21T21:42:18.547 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5559999999999996e-06 2024-09-21T21:42:18.547 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.547 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.547 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:18.548 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964734+0000", 2024-09-21T21:42:18.548 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041754415000000003 2024-09-21T21:42:18.548 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.548 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.548 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:18.548 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964775+0000", 2024-09-21T21:42:18.548 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0806999999999997e-05 2024-09-21T21:42:18.548 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.548 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.548 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:18.548 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964825+0000", 2024-09-21T21:42:18.548 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0550999999999998e-05 2024-09-21T21:42:18.548 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:18.548 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:18.549 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:18.549 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.549 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.549 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6978 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5275648~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:18.549 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.922987+0000", 2024-09-21T21:42:18.549 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 48.614434635999999, 2024-09-21T21:42:18.549 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042043876000000001, 2024-09-21T21:42:18.549 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:18.549 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:18.549 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:18.549 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:18.549 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:18.549 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:18.549 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6978 2024-09-21T21:42:18.549 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.550 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:18.550 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.550 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:18.550 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922987+0000", 2024-09-21T21:42:18.550 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:18.550 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.550 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.550 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:18.550 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922987+0000", 2024-09-21T21:42:18.550 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:18.550 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.550 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.550 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:18.550 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922988+0000", 2024-09-21T21:42:18.551 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1199999999999998e-07 2024-09-21T21:42:18.551 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.551 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.551 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:18.551 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922990+0000", 2024-09-21T21:42:18.551 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.142e-06 2024-09-21T21:42:18.551 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.551 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.551 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:18.551 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922997+0000", 2024-09-21T21:42:18.551 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.199e-06 2024-09-21T21:42:18.551 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.551 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.552 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:18.552 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923004+0000", 2024-09-21T21:42:18.552 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3599999999999998e-06 2024-09-21T21:42:18.552 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.552 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.552 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:18.552 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964856+0000", 2024-09-21T21:42:18.552 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041851521000000003 2024-09-21T21:42:18.552 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.552 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.552 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:18.552 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964944+0000", 2024-09-21T21:42:18.552 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7859e-05 2024-09-21T21:42:18.552 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.552 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.552 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:18.553 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965031+0000", 2024-09-21T21:42:18.553 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7383e-05 2024-09-21T21:42:18.553 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:18.553 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:18.553 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:18.553 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.553 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.553 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6980 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5289984~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:18.553 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923011+0000", 2024-09-21T21:42:18.553 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 48.614411298999997, 2024-09-21T21:42:18.553 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042139337999999998, 2024-09-21T21:42:18.553 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:18.553 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:18.553 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:18.553 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:18.554 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:18.554 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:18.554 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6980 2024-09-21T21:42:18.554 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.554 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:18.554 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.554 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:18.554 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923011+0000", 2024-09-21T21:42:18.554 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:18.554 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.554 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.554 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:18.554 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923011+0000", 2024-09-21T21:42:18.554 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:18.555 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.555 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.555 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:18.555 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923011+0000", 2024-09-21T21:42:18.555 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6699999999999999e-07 2024-09-21T21:42:18.555 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.555 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.555 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:18.555 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923013+0000", 2024-09-21T21:42:18.555 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0279999999999999e-06 2024-09-21T21:42:18.555 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.555 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.555 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:18.555 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923018+0000", 2024-09-21T21:42:18.555 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4790000000000004e-06 2024-09-21T21:42:18.556 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.556 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.556 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:18.556 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923029+0000", 2024-09-21T21:42:18.556 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0601999999999999e-05 2024-09-21T21:42:18.556 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.556 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.556 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:18.556 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965059+0000", 2024-09-21T21:42:18.556 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042029947999999998 2024-09-21T21:42:18.556 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.556 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.556 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:18.556 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965100+0000", 2024-09-21T21:42:18.557 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1170000000000001e-05 2024-09-21T21:42:18.557 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.557 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.557 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:18.557 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965150+0000", 2024-09-21T21:42:18.557 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9743999999999997e-05 2024-09-21T21:42:18.557 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:18.557 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:18.557 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:18.557 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.557 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.557 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6982 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5304320~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:18.557 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923036+0000", 2024-09-21T21:42:18.557 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 48.614385478000003, 2024-09-21T21:42:18.557 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042225666000000002, 2024-09-21T21:42:18.558 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:18.558 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:18.558 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:18.558 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:18.558 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:18.558 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:18.558 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6982 2024-09-21T21:42:18.558 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.558 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:18.558 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.558 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:18.558 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923036+0000", 2024-09-21T21:42:18.558 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:18.559 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.559 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.559 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:18.559 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923036+0000", 2024-09-21T21:42:18.559 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:18.559 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.559 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.559 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:18.559 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923037+0000", 2024-09-21T21:42:18.559 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2800000000000002e-07 2024-09-21T21:42:18.559 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.559 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.559 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:18.559 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923039+0000", 2024-09-21T21:42:18.560 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1500000000000002e-06 2024-09-21T21:42:18.560 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.560 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.560 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:18.560 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923049+0000", 2024-09-21T21:42:18.560 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0236e-05 2024-09-21T21:42:18.560 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.560 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.560 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:18.560 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923056+0000", 2024-09-21T21:42:18.560 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2540000000000003e-06 2024-09-21T21:42:18.560 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.560 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.560 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:18.560 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965173+0000", 2024-09-21T21:42:18.561 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042116686 2024-09-21T21:42:18.561 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.561 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.561 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:18.561 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965215+0000", 2024-09-21T21:42:18.561 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1696000000000002e-05 2024-09-21T21:42:18.561 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.561 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.561 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:18.561 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965262+0000", 2024-09-21T21:42:18.561 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7216000000000003e-05 2024-09-21T21:42:18.561 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:18.561 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:18.562 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:18.562 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.562 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.562 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6984 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5318656~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:18.562 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923063+0000", 2024-09-21T21:42:18.562 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 48.614358824, 2024-09-21T21:42:18.562 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042318651999999998, 2024-09-21T21:42:18.562 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:18.562 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:18.562 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:18.562 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:18.562 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:18.562 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:18.562 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6984 2024-09-21T21:42:18.563 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.563 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:18.563 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.563 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:18.563 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923063+0000", 2024-09-21T21:42:18.563 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:18.563 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.563 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.563 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:18.563 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923063+0000", 2024-09-21T21:42:18.563 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:18.563 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.563 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.563 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:18.564 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923064+0000", 2024-09-21T21:42:18.564 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2e-07 2024-09-21T21:42:18.564 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.564 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.564 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:18.564 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923066+0000", 2024-09-21T21:42:18.564 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.057e-06 2024-09-21T21:42:18.564 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.564 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.564 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:18.564 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923071+0000", 2024-09-21T21:42:18.564 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8810000000000001e-06 2024-09-21T21:42:18.564 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.564 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.564 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:18.565 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923078+0000", 2024-09-21T21:42:18.565 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5479999999999998e-06 2024-09-21T21:42:18.565 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.565 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.565 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:18.565 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965288+0000", 2024-09-21T21:42:18.565 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042209576999999998 2024-09-21T21:42:18.565 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.565 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.565 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:18.565 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965332+0000", 2024-09-21T21:42:18.565 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4366999999999997e-05 2024-09-21T21:42:18.565 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.565 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.565 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:18.566 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965382+0000", 2024-09-21T21:42:18.566 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9802000000000002e-05 2024-09-21T21:42:18.566 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:18.566 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:18.566 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:18.566 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.566 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.566 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6987 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5332992~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:18.566 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923085+0000", 2024-09-21T21:42:18.566 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 48.614337298999999, 2024-09-21T21:42:18.566 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042412103999999999, 2024-09-21T21:42:18.566 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:18.566 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:18.566 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:18.567 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:18.567 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:18.567 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:18.567 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6987 2024-09-21T21:42:18.567 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.567 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:18.567 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.567 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:18.567 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923085+0000", 2024-09-21T21:42:18.567 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:18.567 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.567 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.567 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:18.568 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923085+0000", 2024-09-21T21:42:18.568 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:18.568 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.568 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.568 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:18.568 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923085+0000", 2024-09-21T21:42:18.568 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.89e-07 2024-09-21T21:42:18.568 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.568 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.568 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:18.568 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923087+0000", 2024-09-21T21:42:18.568 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0810000000000001e-06 2024-09-21T21:42:18.568 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.568 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.568 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:18.569 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923098+0000", 2024-09-21T21:42:18.569 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1299e-05 2024-09-21T21:42:18.569 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.569 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.569 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:18.569 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923104+0000", 2024-09-21T21:42:18.569 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9100000000000002e-06 2024-09-21T21:42:18.569 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.569 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.569 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:18.569 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965404+0000", 2024-09-21T21:42:18.569 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042299622000000002 2024-09-21T21:42:18.569 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.569 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.570 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:18.570 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965445+0000", 2024-09-21T21:42:18.570 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0701000000000002e-05 2024-09-21T21:42:18.570 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.570 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.570 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:18.570 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965497+0000", 2024-09-21T21:42:18.570 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2101999999999997e-05 2024-09-21T21:42:18.570 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:18.570 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:18.570 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:18.570 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.570 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.570 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6989 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5347328~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:18.571 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923111+0000", 2024-09-21T21:42:18.571 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 48.614310959000001, 2024-09-21T21:42:18.571 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042505188999999999, 2024-09-21T21:42:18.571 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:18.571 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:18.571 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:18.571 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:18.571 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:18.571 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:18.571 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6989 2024-09-21T21:42:18.571 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.571 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:18.571 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.571 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:18.572 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923111+0000", 2024-09-21T21:42:18.572 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:18.572 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.572 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.572 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:18.572 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923111+0000", 2024-09-21T21:42:18.572 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:18.572 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.572 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.572 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:18.572 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923111+0000", 2024-09-21T21:42:18.572 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4499999999999997e-07 2024-09-21T21:42:18.572 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.572 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.572 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:18.573 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923114+0000", 2024-09-21T21:42:18.573 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1569999999999998e-06 2024-09-21T21:42:18.573 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.573 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.573 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:18.573 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923120+0000", 2024-09-21T21:42:18.573 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1759999999999998e-06 2024-09-21T21:42:18.573 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.573 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.573 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:18.573 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923125+0000", 2024-09-21T21:42:18.573 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2560000000000002e-06 2024-09-21T21:42:18.573 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.573 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.573 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:18.574 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965523+0000", 2024-09-21T21:42:18.574 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042398094999999997 2024-09-21T21:42:18.574 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.574 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.574 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:18.574 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965566+0000", 2024-09-21T21:42:18.574 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3393999999999998e-05 2024-09-21T21:42:18.574 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.574 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.574 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:18.574 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965616+0000", 2024-09-21T21:42:18.574 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9666000000000002e-05 2024-09-21T21:42:18.574 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:18.574 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:18.575 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:18.575 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.575 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.575 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6991 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5361664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:18.575 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923132+0000", 2024-09-21T21:42:18.575 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 48.614289669999998, 2024-09-21T21:42:18.575 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042600779999999998, 2024-09-21T21:42:18.575 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:18.575 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:18.575 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:18.575 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:18.575 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:18.575 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:18.576 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6991 2024-09-21T21:42:18.576 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.576 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:18.576 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.576 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:18.576 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923132+0000", 2024-09-21T21:42:18.576 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:18.576 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.576 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.576 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:18.576 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923132+0000", 2024-09-21T21:42:18.576 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:18.576 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.577 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.577 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:18.577 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923132+0000", 2024-09-21T21:42:18.577 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4200000000000002e-07 2024-09-21T21:42:18.577 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.577 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.577 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:18.577 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923134+0000", 2024-09-21T21:42:18.577 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2270000000000001e-06 2024-09-21T21:42:18.577 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.577 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.577 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:18.577 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923137+0000", 2024-09-21T21:42:18.577 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1930000000000002e-06 2024-09-21T21:42:18.578 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.578 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.578 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:18.578 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923139+0000", 2024-09-21T21:42:18.578 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4710000000000001e-06 2024-09-21T21:42:18.578 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.578 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.578 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:18.578 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965639+0000", 2024-09-21T21:42:18.578 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.04249965 2024-09-21T21:42:18.578 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.578 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.578 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:18.578 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965679+0000", 2024-09-21T21:42:18.579 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9852999999999999e-05 2024-09-21T21:42:18.579 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.579 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.579 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:18.579 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965733+0000", 2024-09-21T21:42:18.579 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4144000000000002e-05 2024-09-21T21:42:18.579 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:18.579 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:18.579 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:18.579 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.579 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.579 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6993 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5384192~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:18.579 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923143+0000", 2024-09-21T21:42:18.580 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 48.614278876, 2024-09-21T21:42:18.580 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042708182999999997, 2024-09-21T21:42:18.580 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:18.580 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:18.580 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:18.580 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:18.580 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:18.580 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:18.580 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6993 2024-09-21T21:42:18.580 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.580 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:18.580 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.580 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:18.580 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923143+0000", 2024-09-21T21:42:18.580 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:18.581 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.581 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.581 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:18.581 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923143+0000", 2024-09-21T21:42:18.581 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:18.581 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.581 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.581 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:18.581 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923143+0000", 2024-09-21T21:42:18.581 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0800000000000001e-07 2024-09-21T21:42:18.581 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.581 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.581 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:18.581 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923144+0000", 2024-09-21T21:42:18.582 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.009e-06 2024-09-21T21:42:18.582 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.582 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.582 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:18.582 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923148+0000", 2024-09-21T21:42:18.582 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.737e-06 2024-09-21T21:42:18.582 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.582 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.582 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:18.582 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923150+0000", 2024-09-21T21:42:18.582 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4669999999999998e-06 2024-09-21T21:42:18.582 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.582 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.582 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:18.583 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965760+0000", 2024-09-21T21:42:18.583 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042609399999999999 2024-09-21T21:42:18.583 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.583 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.583 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:18.583 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965801+0000", 2024-09-21T21:42:18.583 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1399999999999997e-05 2024-09-21T21:42:18.583 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.583 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.583 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:18.583 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965851+0000", 2024-09-21T21:42:18.583 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9962e-05 2024-09-21T21:42:18.583 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:18.583 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:18.584 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:18.584 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.584 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.584 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6995 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5398528~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:18.584 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923153+0000", 2024-09-21T21:42:18.584 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 48.614268512000002, 2024-09-21T21:42:18.584 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042828034000000001, 2024-09-21T21:42:18.584 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:18.584 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:18.584 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:18.584 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:18.584 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:18.584 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:18.584 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6995 2024-09-21T21:42:18.585 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.585 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:18.585 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.585 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:18.585 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923153+0000", 2024-09-21T21:42:18.585 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:18.585 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.585 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.585 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:18.585 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923153+0000", 2024-09-21T21:42:18.585 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:18.585 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.585 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.585 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:18.586 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923154+0000", 2024-09-21T21:42:18.586 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2600000000000001e-07 2024-09-21T21:42:18.586 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.586 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.586 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:18.586 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923158+0000", 2024-09-21T21:42:18.586 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0559999999999998e-06 2024-09-21T21:42:18.586 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.586 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.586 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:18.586 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923161+0000", 2024-09-21T21:42:18.586 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6519999999999999e-06 2024-09-21T21:42:18.586 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.586 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.587 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:18.587 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923170+0000", 2024-09-21T21:42:18.587 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5720000000000003e-06 2024-09-21T21:42:18.587 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.587 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.587 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:18.587 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965887+0000", 2024-09-21T21:42:18.587 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042716863000000001 2024-09-21T21:42:18.587 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.587 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.587 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:18.587 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965927+0000", 2024-09-21T21:42:18.587 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0346999999999998e-05 2024-09-21T21:42:18.587 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.588 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.588 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:18.588 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965981+0000", 2024-09-21T21:42:18.588 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4317999999999997e-05 2024-09-21T21:42:18.588 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:18.588 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:18.588 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:18.588 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.588 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.588 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6997 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5412864~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:18.588 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923174+0000", 2024-09-21T21:42:18.588 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 48.614248357000001, 2024-09-21T21:42:18.588 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042928296999999997, 2024-09-21T21:42:18.588 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:18.588 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:18.589 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:18.589 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:18.589 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:18.589 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:18.589 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6997 2024-09-21T21:42:18.589 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.589 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:18.589 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.589 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:18.589 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923174+0000", 2024-09-21T21:42:18.589 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:18.589 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.589 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.589 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:18.589 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923174+0000", 2024-09-21T21:42:18.590 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:18.590 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.590 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.590 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:18.590 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923174+0000", 2024-09-21T21:42:18.590 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.67e-07 2024-09-21T21:42:18.590 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.590 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.590 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:18.590 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923178+0000", 2024-09-21T21:42:18.590 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1330000000000001e-06 2024-09-21T21:42:18.590 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.590 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.590 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:18.590 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923182+0000", 2024-09-21T21:42:18.591 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.794e-06 2024-09-21T21:42:18.591 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.591 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.591 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:18.591 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923184+0000", 2024-09-21T21:42:18.591 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.435e-06 2024-09-21T21:42:18.591 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.591 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.591 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:18.591 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966009+0000", 2024-09-21T21:42:18.591 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042824327000000002 2024-09-21T21:42:18.591 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.591 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.591 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:18.591 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966052+0000", 2024-09-21T21:42:18.592 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3034000000000002e-05 2024-09-21T21:42:18.592 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.592 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.592 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:18.592 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966102+0000", 2024-09-21T21:42:18.592 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0306999999999998e-05 2024-09-21T21:42:18.592 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:18.592 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:18.592 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:18.592 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.592 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.592 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6999 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5427200~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:18.592 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923187+0000", 2024-09-21T21:42:18.592 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 48.614234629000002, 2024-09-21T21:42:18.592 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043104267000000002, 2024-09-21T21:42:18.593 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:18.593 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:18.593 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:18.593 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:18.593 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:18.593 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:18.593 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6999 2024-09-21T21:42:18.593 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.593 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:18.593 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.593 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:18.593 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923187+0000", 2024-09-21T21:42:18.593 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:18.593 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.594 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.594 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:18.594 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923187+0000", 2024-09-21T21:42:18.594 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:18.594 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.594 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.594 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:18.594 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923188+0000", 2024-09-21T21:42:18.594 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3999999999999998e-07 2024-09-21T21:42:18.594 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.594 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.594 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:18.594 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923189+0000", 2024-09-21T21:42:18.594 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4169999999999999e-06 2024-09-21T21:42:18.595 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.595 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.595 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:18.595 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923194+0000", 2024-09-21T21:42:18.595 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1819999999999997e-06 2024-09-21T21:42:18.595 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.595 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.595 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:18.595 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923198+0000", 2024-09-21T21:42:18.595 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.143e-06 2024-09-21T21:42:18.595 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.595 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.595 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:18.595 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966135+0000", 2024-09-21T21:42:18.595 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042936297999999998 2024-09-21T21:42:18.596 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.596 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.596 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:18.596 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966204+0000", 2024-09-21T21:42:18.596 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8989e-05 2024-09-21T21:42:18.596 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.596 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.596 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:18.596 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966292+0000", 2024-09-21T21:42:18.596 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7997999999999993e-05 2024-09-21T21:42:18.596 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:18.596 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:18.596 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:18.596 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.597 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.597 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7001 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5441536~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:18.597 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923211+0000", 2024-09-21T21:42:18.597 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 48.614210837999998, 2024-09-21T21:42:18.597 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043196518000000003, 2024-09-21T21:42:18.597 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:18.597 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:18.597 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:18.597 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:18.597 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:18.597 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:18.597 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7001 2024-09-21T21:42:18.597 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.597 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:18.598 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.598 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:18.598 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923211+0000", 2024-09-21T21:42:18.598 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:18.598 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.598 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.598 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:18.598 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923211+0000", 2024-09-21T21:42:18.598 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:18.598 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.598 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.598 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:18.598 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923212+0000", 2024-09-21T21:42:18.599 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3900000000000005e-07 2024-09-21T21:42:18.599 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.599 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.599 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:18.599 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923214+0000", 2024-09-21T21:42:18.599 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9970000000000001e-06 2024-09-21T21:42:18.599 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.599 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.599 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:18.599 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923219+0000", 2024-09-21T21:42:18.599 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4310000000000002e-06 2024-09-21T21:42:18.599 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.600 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.600 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:18.600 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923224+0000", 2024-09-21T21:42:18.600 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6410000000000001e-06 2024-09-21T21:42:18.600 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.600 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.600 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:18.600 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966318+0000", 2024-09-21T21:42:18.600 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043093842 2024-09-21T21:42:18.600 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.600 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.600 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:18.600 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966359+0000", 2024-09-21T21:42:18.600 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1260999999999997e-05 2024-09-21T21:42:18.601 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.601 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.601 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:18.601 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966408+0000", 2024-09-21T21:42:18.601 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8807000000000002e-05 2024-09-21T21:42:18.601 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:18.601 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:18.601 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:18.601 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.601 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.601 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7002 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5455872~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:18.601 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923229+0000", 2024-09-21T21:42:18.601 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 48.614192428000003, 2024-09-21T21:42:18.602 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043364568999999999, 2024-09-21T21:42:18.602 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:18.602 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:18.602 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:18.602 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:18.602 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:18.602 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:18.602 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7002 2024-09-21T21:42:18.602 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.602 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:18.602 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.602 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:18.602 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923229+0000", 2024-09-21T21:42:18.602 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:18.603 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.603 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.603 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:18.603 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923229+0000", 2024-09-21T21:42:18.603 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:18.603 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.603 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.603 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:18.603 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923230+0000", 2024-09-21T21:42:18.603 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4700000000000002e-07 2024-09-21T21:42:18.603 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.603 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.603 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:18.603 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923232+0000", 2024-09-21T21:42:18.604 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0719999999999998e-06 2024-09-21T21:42:18.604 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.604 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.604 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:18.604 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923237+0000", 2024-09-21T21:42:18.604 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2290000000000002e-06 2024-09-21T21:42:18.604 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.604 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.604 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:18.604 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923245+0000", 2024-09-21T21:42:18.604 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2959999999999999e-06 2024-09-21T21:42:18.604 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.604 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.604 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:18.605 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966438+0000", 2024-09-21T21:42:18.605 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043193027000000002 2024-09-21T21:42:18.605 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.605 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.605 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:18.605 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966506+0000", 2024-09-21T21:42:18.605 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7707999999999996e-05 2024-09-21T21:42:18.605 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.605 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.605 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:18.605 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966594+0000", 2024-09-21T21:42:18.605 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.789e-05 2024-09-21T21:42:18.605 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:18.605 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:18.606 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:18.606 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.606 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.606 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7005 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5470208~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:18.606 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923254+0000", 2024-09-21T21:42:18.606 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 48.614168196999998, 2024-09-21T21:42:18.606 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043457176, 2024-09-21T21:42:18.606 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:18.606 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:18.606 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:18.606 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:18.606 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:18.606 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:18.606 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7005 2024-09-21T21:42:18.607 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.607 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:18.607 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.607 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:18.607 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923254+0000", 2024-09-21T21:42:18.607 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:18.607 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.607 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.607 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:18.607 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923254+0000", 2024-09-21T21:42:18.607 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:18.607 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.607 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.607 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:18.607 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923254+0000", 2024-09-21T21:42:18.608 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4700000000000002e-07 2024-09-21T21:42:18.608 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.608 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.608 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:18.608 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923256+0000", 2024-09-21T21:42:18.608 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.007e-06 2024-09-21T21:42:18.608 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.608 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.608 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:18.608 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923266+0000", 2024-09-21T21:42:18.608 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.4700000000000008e-06 2024-09-21T21:42:18.608 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.608 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.609 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:18.609 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923271+0000", 2024-09-21T21:42:18.609 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2809999999999999e-06 2024-09-21T21:42:18.609 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.609 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.609 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:18.609 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966619+0000", 2024-09-21T21:42:18.609 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043347951000000003 2024-09-21T21:42:18.609 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.609 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.609 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:18.609 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966661+0000", 2024-09-21T21:42:18.609 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2614999999999998e-05 2024-09-21T21:42:18.609 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.610 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.610 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:18.610 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966711+0000", 2024-09-21T21:42:18.610 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9505000000000001e-05 2024-09-21T21:42:18.610 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:18.610 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:18.610 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:18.610 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.610 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.610 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7008 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5484544~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:18.610 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923278+0000", 2024-09-21T21:42:18.610 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 48.614143550000001, 2024-09-21T21:42:18.610 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043631071, 2024-09-21T21:42:18.610 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:18.611 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:18.611 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:18.611 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:18.611 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:18.611 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:18.611 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7008 2024-09-21T21:42:18.611 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.611 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:18.611 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.611 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:18.611 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923278+0000", 2024-09-21T21:42:18.611 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:18.611 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.611 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.612 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:18.612 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923278+0000", 2024-09-21T21:42:18.612 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:18.612 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.612 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.612 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:18.612 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923279+0000", 2024-09-21T21:42:18.612 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.77e-07 2024-09-21T21:42:18.612 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.612 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.612 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:18.612 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923281+0000", 2024-09-21T21:42:18.612 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0219999999999999e-06 2024-09-21T21:42:18.612 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.613 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.613 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:18.613 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923287+0000", 2024-09-21T21:42:18.613 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7799999999999997e-06 2024-09-21T21:42:18.613 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.613 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.613 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:18.613 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923291+0000", 2024-09-21T21:42:18.613 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1699999999999999e-06 2024-09-21T21:42:18.613 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.613 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.613 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:18.613 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966742+0000", 2024-09-21T21:42:18.613 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043451344000000003 2024-09-21T21:42:18.613 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.614 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.614 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:18.614 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966809+0000", 2024-09-21T21:42:18.614 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7070999999999995e-05 2024-09-21T21:42:18.614 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.614 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.614 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:18.614 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966909+0000", 2024-09-21T21:42:18.614 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000100307 2024-09-21T21:42:18.614 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:18.614 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:18.614 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:18.614 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.614 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.614 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7010 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5498880~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:18.615 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923301+0000", 2024-09-21T21:42:18.615 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 48.614120655000001, 2024-09-21T21:42:18.615 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043717830999999999, 2024-09-21T21:42:18.615 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:18.615 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:18.615 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:18.615 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:18.615 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:18.615 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:18.615 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7010 2024-09-21T21:42:18.615 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.615 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:18.615 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.615 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:18.615 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923301+0000", 2024-09-21T21:42:18.616 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:18.616 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.616 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.616 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:18.616 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923301+0000", 2024-09-21T21:42:18.616 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:18.616 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.616 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.616 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:18.616 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923302+0000", 2024-09-21T21:42:18.616 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.41e-07 2024-09-21T21:42:18.616 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.616 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.616 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:18.616 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923304+0000", 2024-09-21T21:42:18.617 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0990000000000001e-06 2024-09-21T21:42:18.617 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.617 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.617 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:18.617 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923310+0000", 2024-09-21T21:42:18.617 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3860000000000004e-06 2024-09-21T21:42:18.617 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.617 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.617 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:18.617 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923315+0000", 2024-09-21T21:42:18.617 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8459999999999999e-06 2024-09-21T21:42:18.617 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.617 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.617 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:18.617 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966938+0000", 2024-09-21T21:42:18.618 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043622693999999997 2024-09-21T21:42:18.618 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.618 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.618 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:18.618 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966975+0000", 2024-09-21T21:42:18.618 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7608999999999999e-05 2024-09-21T21:42:18.618 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.618 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.618 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:18.618 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.967019+0000", 2024-09-21T21:42:18.618 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3856000000000002e-05 2024-09-21T21:42:18.618 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:18.618 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:18.618 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:18.619 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.619 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.619 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7354 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 2107392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:18.619 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.936932+0000", 2024-09-21T21:42:18.619 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 48.600489799999998, 2024-09-21T21:42:18.619 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041607708, 2024-09-21T21:42:18.619 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:18.619 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:18.619 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:18.619 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:18.619 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:18.619 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:18.619 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7354 2024-09-21T21:42:18.619 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.620 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:18.620 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.620 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:18.620 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.936932+0000", 2024-09-21T21:42:18.620 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:18.620 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.620 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.620 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:18.620 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.936932+0000", 2024-09-21T21:42:18.620 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:18.620 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.620 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.620 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:18.620 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.937276+0000", 2024-09-21T21:42:18.621 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00034344200000000001 2024-09-21T21:42:18.621 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.621 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.621 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:18.621 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.937279+0000", 2024-09-21T21:42:18.621 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.844e-06 2024-09-21T21:42:18.621 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.621 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.621 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:18.621 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.937288+0000", 2024-09-21T21:42:18.621 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2409999999999995e-06 2024-09-21T21:42:18.621 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.621 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.621 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:18.621 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.937293+0000", 2024-09-21T21:42:18.622 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6559999999999997e-06 2024-09-21T21:42:18.622 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.622 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.622 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:18.622 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.978448+0000", 2024-09-21T21:42:18.622 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041154621000000002 2024-09-21T21:42:18.622 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.622 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.622 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:18.622 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.978487+0000", 2024-09-21T21:42:18.622 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9254000000000001e-05 2024-09-21T21:42:18.622 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.622 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.622 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:18.623 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.978540+0000", 2024-09-21T21:42:18.623 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2649999999999999e-05 2024-09-21T21:42:18.623 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:18.623 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:18.623 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:18.623 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.623 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.623 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5215.0:1054 3.9 3:946d2401:::rbd_data.1121524d2522.000000000000000b:head [set-alloc-hint object_size 16777216 write_size 16777216,write 6615040~59392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-21T21:42:18.623 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:38.148979+0000", 2024-09-21T21:42:18.623 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 40.388442832999999, 2024-09-21T21:42:18.623 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047870388999999999, 2024-09-21T21:42:18.623 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:18.623 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:18.623 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:18.624 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:18.624 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5215", 2024-09-21T21:42:18.624 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1580282168", 2024-09-21T21:42:18.624 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1054 2024-09-21T21:42:18.624 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.624 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:18.624 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.624 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:18.624 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.148979+0000", 2024-09-21T21:42:18.624 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:18.624 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.624 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.624 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:18.624 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.148979+0000", 2024-09-21T21:42:18.625 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:18.625 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.625 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.625 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:18.625 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.148980+0000", 2024-09-21T21:42:18.625 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5799999999999999e-07 2024-09-21T21:42:18.625 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.625 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.625 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:18.625 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.149039+0000", 2024-09-21T21:42:18.625 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9151000000000003e-05 2024-09-21T21:42:18.625 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.625 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.625 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:18.625 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.149047+0000", 2024-09-21T21:42:18.626 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1000000000000004e-06 2024-09-21T21:42:18.626 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.626 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.626 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:18.626 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.149052+0000", 2024-09-21T21:42:18.626 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5959999999999997e-06 2024-09-21T21:42:18.626 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.626 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.626 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:18.626 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.195288+0000", 2024-09-21T21:42:18.626 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.046236248000000001 2024-09-21T21:42:18.626 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.626 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.626 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:18.627 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.195334+0000", 2024-09-21T21:42:18.627 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6575e-05 2024-09-21T21:42:18.627 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.627 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.627 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 2", 2024-09-21T21:42:18.627 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.195406+0000", 2024-09-21T21:42:18.627 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1715e-05 2024-09-21T21:42:18.627 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.627 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.627 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:18.627 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.196018+0000", 2024-09-21T21:42:18.627 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00061218999999999998 2024-09-21T21:42:18.627 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.627 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.627 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:18.628 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.196794+0000", 2024-09-21T21:42:18.628 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000775814 2024-09-21T21:42:18.628 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.628 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.628 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:18.628 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.196827+0000", 2024-09-21T21:42:18.628 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2419000000000003e-05 2024-09-21T21:42:18.628 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:18.628 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:18.628 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:18.628 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.196849+0000", 2024-09-21T21:42:18.628 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2923000000000001e-05 2024-09-21T21:42:18.628 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:18.628 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:18.629 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:18.629 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:18.629 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:18.629 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:18.630 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T21:42:18.631 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:18.628+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:18.731 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:18.730+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:18.773 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:18.773 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:42:18.773 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-21T21:42:18.773 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:18.781 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T21:42:18.832 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:18.830+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:18.882 INFO:teuthology.orchestra.run.smithi164.stdout:checking clone #36, image image_client.0-clone36 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent37 2024-09-21T21:42:18.885 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:18.885 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:42:18.885 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:42:18.885 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:42:18.885 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:18.894 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T21:42:18.933 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:18.932+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:18.950 INFO:teuthology.orchestra.run.smithi164.stdout:5077 read 0x1d96800 thru 0x1da66b9 (0xfeba bytes) 2024-09-21T21:42:18.958 INFO:teuthology.orchestra.run.smithi164.stdout:5078 trunc from 0xe5fce00 to 0x9470e00 2024-09-21T21:42:19.034 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:19.033+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:19.069 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:19.069 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-21T21:42:19.069 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-21T21:42:19.069 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:42:19.069 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.069 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1486 3.16 3:6a41a830:::rbd_data.112145fae779.000000000000000a:a [read 360448~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:42:19.069 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634126+0000", 2024-09-21T21:42:19.070 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 77.432719125999995, 2024-09-21T21:42:19.070 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059399528, 2024-09-21T21:42:19.070 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:19.070 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:19.070 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:19.070 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:19.070 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:42:19.070 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:42:19.070 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1486 2024-09-21T21:42:19.070 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.070 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:19.070 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.070 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:19.070 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634126+0000", 2024-09-21T21:42:19.071 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:19.071 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.071 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.071 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:19.071 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634126+0000", 2024-09-21T21:42:19.071 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:19.071 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.071 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.071 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:19.071 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634126+0000", 2024-09-21T21:42:19.071 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4099999999999999e-07 2024-09-21T21:42:19.071 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.071 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.072 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:19.072 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634128+0000", 2024-09-21T21:42:19.072 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1069999999999999e-06 2024-09-21T21:42:19.072 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.072 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.072 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:19.072 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634135+0000", 2024-09-21T21:42:19.072 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.212e-06 2024-09-21T21:42:19.072 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.072 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.072 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:19.072 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634140+0000", 2024-09-21T21:42:19.073 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1429999999999997e-06 2024-09-21T21:42:19.073 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.073 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.073 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:19.073 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693363+0000", 2024-09-21T21:42:19.073 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059223614000000001 2024-09-21T21:42:19.073 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.073 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.073 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:19.073 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693431+0000", 2024-09-21T21:42:19.073 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7294999999999999e-05 2024-09-21T21:42:19.073 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.073 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.073 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:19.074 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693525+0000", 2024-09-21T21:42:19.074 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.4716000000000006e-05 2024-09-21T21:42:19.074 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:19.074 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:19.074 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:19.074 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.074 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.074 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1499 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 49152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:42:19.074 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634186+0000", 2024-09-21T21:42:19.074 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 77.432659248999997, 2024-09-21T21:42:19.074 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059339256999999999, 2024-09-21T21:42:19.074 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:19.074 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:19.074 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:19.075 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:19.075 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:42:19.075 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:42:19.075 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1499 2024-09-21T21:42:19.075 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.075 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:19.075 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.075 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:19.075 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634186+0000", 2024-09-21T21:42:19.075 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:19.075 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.075 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.075 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:19.075 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634186+0000", 2024-09-21T21:42:19.076 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:19.076 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.076 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.076 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:19.076 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634186+0000", 2024-09-21T21:42:19.076 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0999999999999999e-07 2024-09-21T21:42:19.076 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.076 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.076 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:19.076 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634188+0000", 2024-09-21T21:42:19.076 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.86e-06 2024-09-21T21:42:19.076 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.076 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.076 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:19.076 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634194+0000", 2024-09-21T21:42:19.077 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1109999999999996e-06 2024-09-21T21:42:19.077 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.077 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.077 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:19.077 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634199+0000", 2024-09-21T21:42:19.077 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7419999999999997e-06 2024-09-21T21:42:19.077 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.077 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.077 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:19.077 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693357+0000", 2024-09-21T21:42:19.077 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059157847 2024-09-21T21:42:19.077 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.078 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.078 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:19.078 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693430+0000", 2024-09-21T21:42:19.078 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3203999999999999e-05 2024-09-21T21:42:19.078 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.078 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.078 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:19.078 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693525+0000", 2024-09-21T21:42:19.078 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5082999999999997e-05 2024-09-21T21:42:19.078 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:19.078 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:19.078 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:19.078 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.079 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.079 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1501 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 106496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:42:19.079 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634205+0000", 2024-09-21T21:42:19.079 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 77.432640159000002, 2024-09-21T21:42:19.079 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059482394000000001, 2024-09-21T21:42:19.079 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:19.079 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:19.079 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:19.079 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:19.079 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:42:19.079 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:42:19.079 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1501 2024-09-21T21:42:19.079 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.080 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:19.080 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.080 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:19.080 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634205+0000", 2024-09-21T21:42:19.080 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:19.080 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.080 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.080 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:19.080 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634205+0000", 2024-09-21T21:42:19.080 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:19.080 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.080 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.080 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:19.080 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634205+0000", 2024-09-21T21:42:19.081 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.03e-07 2024-09-21T21:42:19.081 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.081 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.081 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:19.081 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634207+0000", 2024-09-21T21:42:19.081 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0389999999999999e-06 2024-09-21T21:42:19.081 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.081 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.081 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:19.081 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634213+0000", 2024-09-21T21:42:19.081 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2600000000000002e-06 2024-09-21T21:42:19.081 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.081 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.081 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:19.082 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634218+0000", 2024-09-21T21:42:19.082 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8570000000000003e-06 2024-09-21T21:42:19.082 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.082 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.082 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:19.082 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693552+0000", 2024-09-21T21:42:19.082 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059333636000000002 2024-09-21T21:42:19.082 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.082 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.082 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:19.082 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693613+0000", 2024-09-21T21:42:19.082 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0560000000000003e-05 2024-09-21T21:42:19.082 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.082 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.083 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:19.083 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693687+0000", 2024-09-21T21:42:19.083 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4639000000000001e-05 2024-09-21T21:42:19.083 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:19.083 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:19.083 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:19.083 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.083 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.083 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1515 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 434176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:42:19.083 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634318+0000", 2024-09-21T21:42:19.083 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 77.432526472000006, 2024-09-21T21:42:19.083 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059565812000000003, 2024-09-21T21:42:19.083 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:19.083 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:19.084 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:19.084 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:19.084 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:42:19.084 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:42:19.084 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1515 2024-09-21T21:42:19.084 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.084 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:19.084 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.084 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:19.084 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634318+0000", 2024-09-21T21:42:19.084 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:19.084 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.084 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.084 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:19.085 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634318+0000", 2024-09-21T21:42:19.085 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:19.085 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.085 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.085 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:19.085 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634319+0000", 2024-09-21T21:42:19.085 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9400000000000001e-07 2024-09-21T21:42:19.085 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.085 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.085 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:19.085 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634321+0000", 2024-09-21T21:42:19.085 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.154e-06 2024-09-21T21:42:19.085 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.085 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.086 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:19.086 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634327+0000", 2024-09-21T21:42:19.086 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9830000000000001e-06 2024-09-21T21:42:19.086 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.086 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.086 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:19.086 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634332+0000", 2024-09-21T21:42:19.086 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9080000000000003e-06 2024-09-21T21:42:19.086 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.086 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.087 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:19.087 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693717+0000", 2024-09-21T21:42:19.087 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059385447000000001 2024-09-21T21:42:19.087 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.087 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.087 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:19.087 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693795+0000", 2024-09-21T21:42:19.087 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7267000000000004e-05 2024-09-21T21:42:19.087 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.087 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.087 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:19.087 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693884+0000", 2024-09-21T21:42:19.087 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9659000000000003e-05 2024-09-21T21:42:19.087 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:19.087 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:19.088 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:19.088 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.088 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.088 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1517 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 491520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:42:19.088 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634338+0000", 2024-09-21T21:42:19.088 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 77.432506699000001, 2024-09-21T21:42:19.088 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060100745999999997, 2024-09-21T21:42:19.088 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:19.088 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:19.088 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:19.089 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:19.089 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:42:19.089 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:42:19.089 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1517 2024-09-21T21:42:19.089 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.089 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:19.089 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.089 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:19.089 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634338+0000", 2024-09-21T21:42:19.089 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:19.089 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.089 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.089 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:19.090 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634338+0000", 2024-09-21T21:42:19.090 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:19.090 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.090 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.090 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:19.090 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634339+0000", 2024-09-21T21:42:19.090 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2899999999999999e-07 2024-09-21T21:42:19.090 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.090 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.090 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:19.090 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634341+0000", 2024-09-21T21:42:19.090 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.8959999999999999e-06 2024-09-21T21:42:19.090 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.090 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.091 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:19.091 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634347+0000", 2024-09-21T21:42:19.091 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.302e-06 2024-09-21T21:42:19.091 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.091 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.091 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:19.091 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634351+0000", 2024-09-21T21:42:19.091 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6070000000000001e-06 2024-09-21T21:42:19.091 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.091 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.091 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:19.091 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694279+0000", 2024-09-21T21:42:19.091 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059927046999999997 2024-09-21T21:42:19.091 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.091 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.092 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:19.092 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694355+0000", 2024-09-21T21:42:19.092 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6483999999999995e-05 2024-09-21T21:42:19.092 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.092 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.092 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:19.092 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694439+0000", 2024-09-21T21:42:19.092 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3980999999999994e-05 2024-09-21T21:42:19.092 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:19.092 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:19.092 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:19.092 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.092 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.092 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1518 3.16 3:6a41a830:::rbd_data.112145fae779.000000000000000a:a [read 294912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:42:19.093 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634357+0000", 2024-09-21T21:42:19.093 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 77.432487793000007, 2024-09-21T21:42:19.093 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059712318, 2024-09-21T21:42:19.093 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:19.093 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:19.093 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:19.093 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:19.093 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:42:19.093 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:42:19.093 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1518 2024-09-21T21:42:19.093 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.093 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:19.093 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.093 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:19.094 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634357+0000", 2024-09-21T21:42:19.094 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:19.094 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.094 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.094 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:19.094 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634357+0000", 2024-09-21T21:42:19.094 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:19.094 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.094 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.094 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:19.094 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634358+0000", 2024-09-21T21:42:19.094 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2100000000000002e-07 2024-09-21T21:42:19.094 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.094 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.095 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:19.095 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634360+0000", 2024-09-21T21:42:19.095 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0940000000000002e-06 2024-09-21T21:42:19.095 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.095 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.095 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:19.095 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634366+0000", 2024-09-21T21:42:19.095 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3389999999999998e-06 2024-09-21T21:42:19.095 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.095 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.095 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:19.095 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634371+0000", 2024-09-21T21:42:19.095 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6670000000000004e-06 2024-09-21T21:42:19.095 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.096 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.096 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:19.096 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693910+0000", 2024-09-21T21:42:19.096 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059538973000000002 2024-09-21T21:42:19.096 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.096 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.096 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:19.096 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.693982+0000", 2024-09-21T21:42:19.096 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2806999999999996e-05 2024-09-21T21:42:19.096 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.096 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.096 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:19.096 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694069+0000", 2024-09-21T21:42:19.096 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7016999999999997e-05 2024-09-21T21:42:19.096 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:19.097 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:19.097 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:19.097 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.097 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.097 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1519 3.16 3:6a41a830:::rbd_data.112145fae779.000000000000000a:a [read 352256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:42:19.097 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634377+0000", 2024-09-21T21:42:19.097 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 77.432468091000004, 2024-09-21T21:42:19.097 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059872213000000001, 2024-09-21T21:42:19.097 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:19.097 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:19.097 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:19.097 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:19.097 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:42:19.098 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:42:19.098 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1519 2024-09-21T21:42:19.098 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.098 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:19.098 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.098 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:19.098 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634377+0000", 2024-09-21T21:42:19.098 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:19.098 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.098 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.098 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:19.098 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634377+0000", 2024-09-21T21:42:19.098 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:19.099 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.099 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.099 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:19.099 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634377+0000", 2024-09-21T21:42:19.099 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2600000000000002e-07 2024-09-21T21:42:19.099 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.099 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.099 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:19.099 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634379+0000", 2024-09-21T21:42:19.099 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0729999999999999e-06 2024-09-21T21:42:19.099 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.099 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.099 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:19.100 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634386+0000", 2024-09-21T21:42:19.100 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4690000000000002e-06 2024-09-21T21:42:19.100 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.100 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.100 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:19.100 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634390+0000", 2024-09-21T21:42:19.100 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4580000000000003e-06 2024-09-21T21:42:19.100 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.100 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.100 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:19.100 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694094+0000", 2024-09-21T21:42:19.100 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.059703830999999999 2024-09-21T21:42:19.100 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.101 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.101 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:19.101 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694168+0000", 2024-09-21T21:42:19.101 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4222999999999997e-05 2024-09-21T21:42:19.101 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.101 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.101 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:19.101 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694249+0000", 2024-09-21T21:42:19.101 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0632999999999999e-05 2024-09-21T21:42:19.101 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:19.101 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:19.101 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:19.101 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.101 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.101 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1529 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 53248~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:42:19.102 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634397+0000", 2024-09-21T21:42:19.102 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 77.432447940000003, 2024-09-21T21:42:19.102 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060189919000000001, 2024-09-21T21:42:19.102 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:19.102 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:19.102 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:19.102 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:19.102 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:42:19.102 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:42:19.102 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1529 2024-09-21T21:42:19.102 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.102 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:19.102 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.102 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:19.103 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634397+0000", 2024-09-21T21:42:19.103 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:19.103 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.103 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.103 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:19.103 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634397+0000", 2024-09-21T21:42:19.103 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:19.103 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.103 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.103 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:19.103 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634397+0000", 2024-09-21T21:42:19.103 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2e-07 2024-09-21T21:42:19.103 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.103 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.103 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:19.104 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634399+0000", 2024-09-21T21:42:19.104 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0609999999999998e-06 2024-09-21T21:42:19.104 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.104 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.104 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:19.104 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634406+0000", 2024-09-21T21:42:19.104 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7259999999999996e-06 2024-09-21T21:42:19.104 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.104 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.104 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:19.104 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634411+0000", 2024-09-21T21:42:19.104 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9339999999999997e-06 2024-09-21T21:42:19.104 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.105 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.105 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:19.105 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694480+0000", 2024-09-21T21:42:19.105 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060069105999999997 2024-09-21T21:42:19.105 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.105 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.105 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:19.105 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694527+0000", 2024-09-21T21:42:19.105 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7120000000000003e-05 2024-09-21T21:42:19.105 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.105 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.105 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:19.105 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694587+0000", 2024-09-21T21:42:19.105 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9552000000000002e-05 2024-09-21T21:42:19.106 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:19.106 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:19.106 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:19.106 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.106 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.106 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1531 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 110592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:42:19.106 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634417+0000", 2024-09-21T21:42:19.106 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 77.432428005999995, 2024-09-21T21:42:19.106 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060347287999999999, 2024-09-21T21:42:19.106 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:19.106 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:19.106 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:19.106 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:19.106 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:42:19.106 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:42:19.107 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1531 2024-09-21T21:42:19.107 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.107 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:19.107 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.107 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:19.107 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634417+0000", 2024-09-21T21:42:19.107 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:19.107 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.107 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.107 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:19.107 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634417+0000", 2024-09-21T21:42:19.107 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:19.107 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.107 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.108 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:19.108 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634417+0000", 2024-09-21T21:42:19.108 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1699999999999999e-07 2024-09-21T21:42:19.108 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.108 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.108 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:19.108 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634420+0000", 2024-09-21T21:42:19.108 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.243e-06 2024-09-21T21:42:19.108 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.108 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.108 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:19.108 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634426+0000", 2024-09-21T21:42:19.108 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2550000000000003e-06 2024-09-21T21:42:19.108 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.109 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.109 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:19.109 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634431+0000", 2024-09-21T21:42:19.109 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0150000000000003e-06 2024-09-21T21:42:19.109 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.109 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.109 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:19.109 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694618+0000", 2024-09-21T21:42:19.109 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060186836000000001 2024-09-21T21:42:19.109 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.109 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.109 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:19.109 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694685+0000", 2024-09-21T21:42:19.109 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6970000000000004e-05 2024-09-21T21:42:19.109 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.110 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.110 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:19.110 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694764+0000", 2024-09-21T21:42:19.110 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9552e-05 2024-09-21T21:42:19.110 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:19.110 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:19.110 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:19.110 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.110 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.110 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1532 3.e 3:724b31a3:::rbd_data.1121e540c096.0000000000000015:b [read 151552~36864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:42:19.110 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634437+0000", 2024-09-21T21:42:19.110 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 77.432408210000006, 2024-09-21T21:42:19.110 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060474314000000001, 2024-09-21T21:42:19.110 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:19.111 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:19.111 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:19.111 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:19.111 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:42:19.111 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:42:19.111 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1532 2024-09-21T21:42:19.111 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.111 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:19.111 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.111 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:19.111 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634437+0000", 2024-09-21T21:42:19.111 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:19.111 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.111 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.112 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:19.112 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634437+0000", 2024-09-21T21:42:19.112 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:19.112 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.112 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.112 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:19.112 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634437+0000", 2024-09-21T21:42:19.112 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9400000000000001e-07 2024-09-21T21:42:19.112 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.112 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.112 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:19.112 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634439+0000", 2024-09-21T21:42:19.112 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.114e-06 2024-09-21T21:42:19.112 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.113 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.113 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:19.113 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634446+0000", 2024-09-21T21:42:19.113 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1389999999999998e-06 2024-09-21T21:42:19.113 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.113 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.113 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:19.113 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634451+0000", 2024-09-21T21:42:19.113 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0010000000000001e-06 2024-09-21T21:42:19.113 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.113 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.113 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:19.113 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694794+0000", 2024-09-21T21:42:19.113 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060342509000000003 2024-09-21T21:42:19.113 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.114 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.114 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:19.114 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694835+0000", 2024-09-21T21:42:19.114 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0728e-05 2024-09-21T21:42:19.114 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.114 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.114 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:19.114 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694911+0000", 2024-09-21T21:42:19.114 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6428999999999996e-05 2024-09-21T21:42:19.114 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:19.114 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:19.114 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:19.114 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.114 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.115 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1534 3.e 3:724b31a3:::rbd_data.1121e540c096.0000000000000015:b [read 487424~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:42:19.115 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634477+0000", 2024-09-21T21:42:19.115 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 77.432367807000006, 2024-09-21T21:42:19.115 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060624009, 2024-09-21T21:42:19.115 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:19.115 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:19.115 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:19.115 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:19.115 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:42:19.115 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:42:19.115 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1534 2024-09-21T21:42:19.115 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.115 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:19.115 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.116 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:19.116 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634477+0000", 2024-09-21T21:42:19.116 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:19.116 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.116 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.116 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:19.116 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634477+0000", 2024-09-21T21:42:19.116 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:19.116 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.116 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.116 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:19.116 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634478+0000", 2024-09-21T21:42:19.116 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1399999999999997e-07 2024-09-21T21:42:19.116 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.117 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.117 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:19.117 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634480+0000", 2024-09-21T21:42:19.117 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1009999999999999e-06 2024-09-21T21:42:19.117 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.117 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.117 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:19.117 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634486+0000", 2024-09-21T21:42:19.117 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3289999999999999e-06 2024-09-21T21:42:19.117 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.117 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.117 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:19.117 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634491+0000", 2024-09-21T21:42:19.117 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6260000000000003e-06 2024-09-21T21:42:19.118 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.118 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.118 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:19.118 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.694943+0000", 2024-09-21T21:42:19.118 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060451960999999999 2024-09-21T21:42:19.118 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.118 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.118 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:19.118 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695014+0000", 2024-09-21T21:42:19.118 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1587000000000004e-05 2024-09-21T21:42:19.118 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.118 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.118 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:19.118 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695101+0000", 2024-09-21T21:42:19.119 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6990999999999994e-05 2024-09-21T21:42:19.119 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:19.119 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:19.119 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:19.119 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.119 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.119 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1545 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 438272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:42:19.119 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634515+0000", 2024-09-21T21:42:19.119 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 77.432329607, 2024-09-21T21:42:19.119 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060707944, 2024-09-21T21:42:19.119 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:19.119 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:19.119 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:19.119 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:19.120 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:42:19.120 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:42:19.120 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1545 2024-09-21T21:42:19.120 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.120 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:19.120 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.120 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:19.120 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634515+0000", 2024-09-21T21:42:19.120 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:19.120 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.120 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.120 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:19.120 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634515+0000", 2024-09-21T21:42:19.120 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:19.120 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.121 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.121 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:19.121 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634516+0000", 2024-09-21T21:42:19.121 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0999999999999999e-07 2024-09-21T21:42:19.121 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.121 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.121 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:19.121 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634518+0000", 2024-09-21T21:42:19.121 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.846e-06 2024-09-21T21:42:19.121 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.121 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.121 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:19.121 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634524+0000", 2024-09-21T21:42:19.121 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2550000000000003e-06 2024-09-21T21:42:19.122 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.122 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.122 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:19.122 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634529+0000", 2024-09-21T21:42:19.122 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6940000000000003e-06 2024-09-21T21:42:19.122 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.122 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.122 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:19.122 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695127+0000", 2024-09-21T21:42:19.122 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060598921 2024-09-21T21:42:19.122 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.122 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.122 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:19.122 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695171+0000", 2024-09-21T21:42:19.123 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3588999999999999e-05 2024-09-21T21:42:19.123 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.123 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.123 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:19.123 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695223+0000", 2024-09-21T21:42:19.123 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2228999999999997e-05 2024-09-21T21:42:19.123 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:19.123 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:19.123 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:19.123 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.123 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.123 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1547 3.e 3:700836d6:::rbd_data.112145fae779.0000000000000000:a [read 495616~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:42:19.123 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634534+0000", 2024-09-21T21:42:19.123 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 77.432310815999998, 2024-09-21T21:42:19.123 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060881131999999998, 2024-09-21T21:42:19.124 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:19.124 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:19.124 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:19.124 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:19.124 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:42:19.124 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:42:19.124 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1547 2024-09-21T21:42:19.124 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.124 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:19.124 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.124 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:19.124 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634534+0000", 2024-09-21T21:42:19.124 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:19.124 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.124 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.125 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:19.125 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634534+0000", 2024-09-21T21:42:19.125 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:19.125 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.125 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.125 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:19.125 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634535+0000", 2024-09-21T21:42:19.125 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.77e-07 2024-09-21T21:42:19.125 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.125 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.125 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:19.125 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634542+0000", 2024-09-21T21:42:19.125 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9000000000000006e-06 2024-09-21T21:42:19.125 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.126 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.126 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:19.126 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634549+0000", 2024-09-21T21:42:19.126 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5910000000000001e-06 2024-09-21T21:42:19.126 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.126 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.126 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:19.126 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634554+0000", 2024-09-21T21:42:19.126 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7940000000000002e-06 2024-09-21T21:42:19.126 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.126 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.126 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:19.126 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695256+0000", 2024-09-21T21:42:19.126 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.06070246 2024-09-21T21:42:19.127 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.127 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.127 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:19.127 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695332+0000", 2024-09-21T21:42:19.127 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6248000000000006e-05 2024-09-21T21:42:19.127 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.127 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.127 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:19.127 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695415+0000", 2024-09-21T21:42:19.127 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2762000000000005e-05 2024-09-21T21:42:19.127 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:19.127 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:19.127 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:19.127 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.127 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.128 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1548 3.16 3:6a41a830:::rbd_data.112145fae779.000000000000000a:a [read 299008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:42:19.128 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634561+0000", 2024-09-21T21:42:19.128 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 77.432284238999998, 2024-09-21T21:42:19.128 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060786743999999997, 2024-09-21T21:42:19.128 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:19.128 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:19.128 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:19.128 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:19.128 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:42:19.128 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:42:19.128 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1548 2024-09-21T21:42:19.128 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.128 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:19.129 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.129 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:19.129 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634561+0000", 2024-09-21T21:42:19.129 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:19.129 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.129 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.129 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:19.129 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634561+0000", 2024-09-21T21:42:19.129 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:19.129 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.129 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.129 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:19.129 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634561+0000", 2024-09-21T21:42:19.129 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5199999999999997e-07 2024-09-21T21:42:19.130 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.130 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.130 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:19.130 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634564+0000", 2024-09-21T21:42:19.130 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4040000000000002e-06 2024-09-21T21:42:19.130 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.130 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.130 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:19.130 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634571+0000", 2024-09-21T21:42:19.130 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4379999999999998e-06 2024-09-21T21:42:19.130 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.130 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.130 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:19.130 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634576+0000", 2024-09-21T21:42:19.130 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1399999999999999e-06 2024-09-21T21:42:19.131 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.131 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.131 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:19.131 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695247+0000", 2024-09-21T21:42:19.131 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060670397000000001 2024-09-21T21:42:19.131 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.131 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.131 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:19.131 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695299+0000", 2024-09-21T21:42:19.131 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2564e-05 2024-09-21T21:42:19.131 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.131 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.131 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:19.131 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695347+0000", 2024-09-21T21:42:19.132 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8248999999999998e-05 2024-09-21T21:42:19.132 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:19.132 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:19.132 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:19.132 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.132 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.132 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4824.0:1549 3.16 3:6a41a830:::rbd_data.112145fae779.000000000000000a:a [read 356352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:42:19.132 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:01.634583+0000", 2024-09-21T21:42:19.132 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 77.432262326, 2024-09-21T21:42:19.132 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060889459, 2024-09-21T21:42:19.132 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:19.132 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:19.132 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:19.132 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:19.133 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4824", 2024-09-21T21:42:19.133 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:42:19.133 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1549 2024-09-21T21:42:19.133 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.133 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:19.133 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.133 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:19.133 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634583+0000", 2024-09-21T21:42:19.133 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:19.133 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.133 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.133 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:19.133 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634583+0000", 2024-09-21T21:42:19.133 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:19.134 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.134 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.134 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:19.134 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634583+0000", 2024-09-21T21:42:19.134 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1899999999999998e-07 2024-09-21T21:42:19.134 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.134 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.134 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:19.134 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634585+0000", 2024-09-21T21:42:19.134 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2129999999999998e-06 2024-09-21T21:42:19.134 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.134 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.134 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:19.134 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634592+0000", 2024-09-21T21:42:19.134 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2840000000000003e-06 2024-09-21T21:42:19.135 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.135 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.135 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:19.135 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.634596+0000", 2024-09-21T21:42:19.135 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9309999999999999e-06 2024-09-21T21:42:19.135 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.135 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.135 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:19.135 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695371+0000", 2024-09-21T21:42:19.135 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.060774425999999999 2024-09-21T21:42:19.135 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.135 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.135 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:19.135 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695419+0000", 2024-09-21T21:42:19.136 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8607999999999999e-05 2024-09-21T21:42:19.136 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.136 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.136 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:19.136 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:01.695472+0000", 2024-09-21T21:42:19.136 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2577999999999997e-05 2024-09-21T21:42:19.136 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:19.136 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:19.136 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:19.136 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.136 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.136 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4385.0:15202 3.16 3:6888948b:::rbd_header.1121493f6459:head [watch reconnect cookie 140325440202496 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-21T21:42:19.136 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:41.736601+0000", 2024-09-21T21:42:19.136 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 37.330243693, 2024-09-21T21:42:19.137 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5040056769999999, 2024-09-21T21:42:19.137 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:19.137 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:19.137 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:19.137 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:19.137 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4385", 2024-09-21T21:42:19.137 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:42:19.137 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 15202 2024-09-21T21:42:19.137 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.137 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:19.137 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.137 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:19.137 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736601+0000", 2024-09-21T21:42:19.137 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:19.138 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.138 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.138 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:19.138 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736601+0000", 2024-09-21T21:42:19.138 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:19.138 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.138 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.138 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:19.138 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736602+0000", 2024-09-21T21:42:19.138 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0699999999999999e-06 2024-09-21T21:42:19.138 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.138 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.138 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:19.138 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736607+0000", 2024-09-21T21:42:19.139 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2520000000000001e-06 2024-09-21T21:42:19.139 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.139 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.139 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:19.139 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736617+0000", 2024-09-21T21:42:19.139 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0074e-05 2024-09-21T21:42:19.139 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.139 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.139 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:19.139 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736622+0000", 2024-09-21T21:42:19.139 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.101e-06 2024-09-21T21:42:19.139 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.139 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.139 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:19.140 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736664+0000", 2024-09-21T21:42:19.140 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1869999999999997e-05 2024-09-21T21:42:19.140 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.140 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.140 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op must wait for map", 2024-09-21T21:42:19.140 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736667+0000", 2024-09-21T21:42:19.140 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4750000000000002e-06 2024-09-21T21:42:19.140 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.140 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.140 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:19.140 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742886+0000", 2024-09-21T21:42:19.140 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0062187579999999996 2024-09-21T21:42:19.140 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.140 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.141 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:19.141 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742892+0000", 2024-09-21T21:42:19.141 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0440000000000001e-06 2024-09-21T21:42:19.141 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.141 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.141 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:19.141 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210307+0000", 2024-09-21T21:42:19.141 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.467414781 2024-09-21T21:42:19.141 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.141 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.141 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:19.141 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210322+0000", 2024-09-21T21:42:19.141 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5311e-05 2024-09-21T21:42:19.141 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.142 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.142 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:19.142 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.237624+0000", 2024-09-21T21:42:19.142 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0273018920000001 2024-09-21T21:42:19.142 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.142 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.142 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:19.142 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.237870+0000", 2024-09-21T21:42:19.142 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00024622899999999999 2024-09-21T21:42:19.142 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.142 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.142 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:19.142 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.240607+0000", 2024-09-21T21:42:19.142 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0027368200000000001 2024-09-21T21:42:19.142 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:19.143 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:19.143 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:19.143 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.143 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.143 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5415.0:379 3.16 3:6888948b:::rbd_header.1121493f6459:head [watch reconnect cookie 139702602617920 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-21T21:42:19.143 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:41.739544+0000", 2024-09-21T21:42:19.143 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 37.327301263999999, 2024-09-21T21:42:19.143 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.501180645, 2024-09-21T21:42:19.143 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:19.143 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:19.143 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:19.143 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:19.143 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5415", 2024-09-21T21:42:19.143 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1212816607", 2024-09-21T21:42:19.144 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 379 2024-09-21T21:42:19.144 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.144 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:19.144 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.144 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:19.144 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739544+0000", 2024-09-21T21:42:19.144 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:19.144 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.144 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.144 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:19.144 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739544+0000", 2024-09-21T21:42:19.144 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:19.144 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.144 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.145 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:19.145 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739545+0000", 2024-09-21T21:42:19.145 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.065e-06 2024-09-21T21:42:19.145 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.145 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.145 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:19.145 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739550+0000", 2024-09-21T21:42:19.145 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9599999999999999e-06 2024-09-21T21:42:19.145 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.145 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.145 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:19.145 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739565+0000", 2024-09-21T21:42:19.145 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5709000000000002e-05 2024-09-21T21:42:19.145 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.146 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.146 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:19.146 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739573+0000", 2024-09-21T21:42:19.146 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7849999999999992e-06 2024-09-21T21:42:19.146 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.146 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.146 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:19.146 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739634+0000", 2024-09-21T21:42:19.146 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0732000000000001e-05 2024-09-21T21:42:19.146 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.146 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.146 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op must wait for map", 2024-09-21T21:42:19.146 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739640+0000", 2024-09-21T21:42:19.147 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0000000000000002e-06 2024-09-21T21:42:19.147 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.147 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.147 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:19.147 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742958+0000", 2024-09-21T21:42:19.147 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0033182390000000002 2024-09-21T21:42:19.147 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.147 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.147 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:19.147 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742966+0000", 2024-09-21T21:42:19.147 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1610000000000003e-06 2024-09-21T21:42:19.147 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.147 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.147 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:19.148 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210405+0000", 2024-09-21T21:42:19.148 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.467438725 2024-09-21T21:42:19.148 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.148 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.148 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:19.148 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210415+0000", 2024-09-21T21:42:19.148 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0234000000000001e-05 2024-09-21T21:42:19.148 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.148 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.148 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:19.148 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.238743+0000", 2024-09-21T21:42:19.148 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0283273070000001 2024-09-21T21:42:19.148 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.148 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.148 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:19.149 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.238842+0000", 2024-09-21T21:42:19.149 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.9896999999999996e-05 2024-09-21T21:42:19.149 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.149 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.149 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:19.149 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.240724+0000", 2024-09-21T21:42:19.149 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001881831 2024-09-21T21:42:19.149 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:19.149 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:19.149 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:19.149 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.149 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.149 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4385.0:15203 3.16 3:6888948b:::rbd_header.1121493f6459:head [call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-21T21:42:19.150 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:41.739737+0000", 2024-09-21T21:42:19.150 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 37.327108252999999, 2024-09-21T21:42:19.150 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4989621799999999, 2024-09-21T21:42:19.150 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:19.150 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:19.150 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:19.150 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:19.150 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4385", 2024-09-21T21:42:19.150 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:42:19.150 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 15203 2024-09-21T21:42:19.150 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.150 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:19.150 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.150 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:19.151 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739737+0000", 2024-09-21T21:42:19.151 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:19.151 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.151 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.151 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:19.151 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739737+0000", 2024-09-21T21:42:19.151 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:19.151 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.151 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.151 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:19.151 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739738+0000", 2024-09-21T21:42:19.151 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.7499999999999998e-07 2024-09-21T21:42:19.151 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.151 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.151 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:19.152 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739743+0000", 2024-09-21T21:42:19.152 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5049999999999998e-06 2024-09-21T21:42:19.152 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.152 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.152 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:19.152 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739757+0000", 2024-09-21T21:42:19.152 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3947e-05 2024-09-21T21:42:19.152 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.152 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.152 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:19.152 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739765+0000", 2024-09-21T21:42:19.152 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9310000000000008e-06 2024-09-21T21:42:19.152 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.152 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.153 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:19.153 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739816+0000", 2024-09-21T21:42:19.153 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0559999999999997e-05 2024-09-21T21:42:19.153 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.153 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.153 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting_for_map not empty", 2024-09-21T21:42:19.153 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739821+0000", 2024-09-21T21:42:19.153 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3539999999999999e-06 2024-09-21T21:42:19.153 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.153 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.153 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:19.153 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742928+0000", 2024-09-21T21:42:19.153 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0031074399999999999 2024-09-21T21:42:19.153 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.154 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.154 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:19.154 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742937+0000", 2024-09-21T21:42:19.154 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1289999999999996e-06 2024-09-21T21:42:19.154 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.154 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.154 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:19.154 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210350+0000", 2024-09-21T21:42:19.154 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.46741311299999999 2024-09-21T21:42:19.154 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.154 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.154 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:19.154 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210358+0000", 2024-09-21T21:42:19.154 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1869999999999997e-06 2024-09-21T21:42:19.155 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.155 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.155 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:19.155 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.238398+0000", 2024-09-21T21:42:19.155 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0280397830000001 2024-09-21T21:42:19.155 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.155 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.155 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:19.155 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.238489+0000", 2024-09-21T21:42:19.155 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.0958000000000006e-05 2024-09-21T21:42:19.155 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.155 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.155 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:19.155 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.238699+0000", 2024-09-21T21:42:19.155 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00021029800000000001 2024-09-21T21:42:19.156 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:19.156 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:19.156 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:19.156 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.156 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.156 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4385.0:15204 3.16 3:6888948b:::rbd_header.1121493f6459:head [watch reconnect cookie 140325440202496 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-21T21:42:19.156 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:42.210726+0000", 2024-09-21T21:42:19.156 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 36.856119374999999, 2024-09-21T21:42:19.156 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.030304557, 2024-09-21T21:42:19.156 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:19.156 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:19.156 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:19.156 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:19.156 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4385", 2024-09-21T21:42:19.157 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:42:19.157 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 15204 2024-09-21T21:42:19.157 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.157 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:19.157 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.157 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:19.157 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210726+0000", 2024-09-21T21:42:19.157 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:19.157 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.157 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.157 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:19.157 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210726+0000", 2024-09-21T21:42:19.157 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:19.157 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.158 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.158 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:19.158 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210727+0000", 2024-09-21T21:42:19.158 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.204e-06 2024-09-21T21:42:19.158 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.158 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.158 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:19.158 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210732+0000", 2024-09-21T21:42:19.158 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5249999999999996e-06 2024-09-21T21:42:19.158 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.158 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.158 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:19.158 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210744+0000", 2024-09-21T21:42:19.158 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1405e-05 2024-09-21T21:42:19.158 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.159 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.159 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:19.159 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210754+0000", 2024-09-21T21:42:19.159 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0122999999999999e-05 2024-09-21T21:42:19.159 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.159 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.159 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:19.159 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210798+0000", 2024-09-21T21:42:19.159 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4005000000000003e-05 2024-09-21T21:42:19.159 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.159 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.159 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:19.159 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210803+0000", 2024-09-21T21:42:19.159 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5130000000000004e-06 2024-09-21T21:42:19.160 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.160 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.160 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:19.160 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.239260+0000", 2024-09-21T21:42:19.160 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0284563040000001 2024-09-21T21:42:19.160 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.160 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.160 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:19.160 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.239318+0000", 2024-09-21T21:42:19.160 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8486000000000002e-05 2024-09-21T21:42:19.160 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.160 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.160 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:19.160 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.241030+0000", 2024-09-21T21:42:19.160 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0017119920000000001 2024-09-21T21:42:19.161 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:19.161 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:19.161 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:19.161 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.161 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.161 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5415.0:380 3.16 3:6888948b:::rbd_header.1121493f6459:head [watch reconnect cookie 139702602617920 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-21T21:42:19.161 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:42.210777+0000", 2024-09-21T21:42:19.161 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 36.856067596000003, 2024-09-21T21:42:19.161 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.030410566, 2024-09-21T21:42:19.161 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:19.161 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:19.161 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:19.161 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:19.161 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5415", 2024-09-21T21:42:19.161 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1212816607", 2024-09-21T21:42:19.162 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 380 2024-09-21T21:42:19.162 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.162 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:19.162 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.162 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:19.162 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210777+0000", 2024-09-21T21:42:19.162 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:19.162 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.162 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.162 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:19.162 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210777+0000", 2024-09-21T21:42:19.162 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:19.162 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.162 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.163 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:19.163 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210778+0000", 2024-09-21T21:42:19.163 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8999999999999996e-07 2024-09-21T21:42:19.163 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.163 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.163 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:19.163 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210780+0000", 2024-09-21T21:42:19.163 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.503e-06 2024-09-21T21:42:19.163 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.163 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.163 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:19.163 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210788+0000", 2024-09-21T21:42:19.163 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0999999999999998e-06 2024-09-21T21:42:19.163 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.164 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.164 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:19.164 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210799+0000", 2024-09-21T21:42:19.164 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1703e-05 2024-09-21T21:42:19.164 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.164 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.164 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:19.164 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210833+0000", 2024-09-21T21:42:19.164 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4008999999999999e-05 2024-09-21T21:42:19.164 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.164 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.164 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:19.164 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210838+0000", 2024-09-21T21:42:19.164 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7480000000000001e-06 2024-09-21T21:42:19.164 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.165 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.165 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:19.165 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.239612+0000", 2024-09-21T21:42:19.165 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.028773554 2024-09-21T21:42:19.165 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.165 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.165 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:19.165 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.239677+0000", 2024-09-21T21:42:19.165 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5053000000000002e-05 2024-09-21T21:42:19.165 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:19.165 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:19.165 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:19.165 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.241188+0000", 2024-09-21T21:42:19.165 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001511306 2024-09-21T21:42:19.165 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:19.166 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:19.166 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:19.166 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:19.166 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:19.166 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:19.167 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:19.166+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:19.167 INFO:teuthology.orchestra.run.smithi164.stdout:5085 trunc from 0x9470e00 to 0x59f2400 2024-09-21T21:42:19.167 INFO:teuthology.orchestra.run.smithi164.stdout:5087 write 0x2bb9e00 thru 0x2bc40a3 (0xa2a4 bytes) 2024-09-21T21:42:19.168 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T21:42:19.270 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:19.266+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:19.272 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:19.272 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:42:19.272 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 0 2024-09-21T21:42:19.272 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:19.281 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T21:42:19.356 INFO:teuthology.orchestra.run.smithi164.stdout:5091 write 0x6973a00 thru 0x697594e (0x1f4f bytes) 2024-09-21T21:42:19.357 INFO:teuthology.orchestra.run.smithi164.stdout:5094 write 0x3c8000 thru 0x3ccba0 (0x4ba1 bytes) 2024-09-21T21:42:19.358 INFO:teuthology.orchestra.run.smithi164.stdout:5096 read 0x6678c00 thru 0x667c972 (0x3d73 bytes) 2024-09-21T21:42:19.358 INFO:teuthology.orchestra.run.smithi164.stdout:5097 read 0x426e400 thru 0x427afc9 (0xcbca bytes) 2024-09-21T21:42:19.358 INFO:teuthology.orchestra.run.smithi164.stdout:5098 write 0x312ba00 thru 0x313017c (0x477d bytes) 2024-09-21T21:42:19.358 INFO:teuthology.orchestra.run.smithi164.stdout:5099 write 0x68a6c00 thru 0x68ae81f (0x7c20 bytes) 2024-09-21T21:42:19.366 INFO:teuthology.orchestra.run.smithi164.stdout:5102 trunc from 0x6975a00 to 0xcca0600 2024-09-21T21:42:19.368 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:19.367+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:19.404 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:19.404 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:42:19.404 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:42:19.404 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:42:19.404 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:19.413 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T21:42:19.443 INFO:teuthology.orchestra.run.smithi164.stdout:5104 read 0x3dc8e00 thru 0x3dca28e (0x148f bytes) 2024-09-21T21:42:19.445 INFO:teuthology.orchestra.run.smithi164.stdout:5105 trunc from 0xcca0600 to 0xbc00400 2024-09-21T21:42:19.453 INFO:teuthology.orchestra.run.smithi164.stdout:5107 punch from 0xab98000 to 0xab9a400, (0x2400 bytes) 2024-09-21T21:42:19.457 INFO:teuthology.orchestra.run.smithi164.stdout:5109 write 0x6d8d800 thru 0x6d8da20 (0x221 bytes) 2024-09-21T21:42:19.457 INFO:teuthology.orchestra.run.smithi164.stdout:5114 clone 39 order 19 su 2048 sc 10 2024-09-21T21:42:19.497 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:19.496+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:19.552 INFO:teuthology.orchestra.run.smithi164.stdout:truncating image image_client.0-clone38 from 0xbc00400 (overlap 0x59f2400) to 0x35f7c00 2024-09-21T21:42:19.565 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:19.565 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 20, 2024-09-21T21:42:19.565 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 600, 2024-09-21T21:42:19.565 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:42:19.565 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.565 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11319 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 5492736~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:19.566 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.712385+0000", 2024-09-21T21:42:19.566 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 28.850486842999999, 2024-09-21T21:42:19.566 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078627889000000006, 2024-09-21T21:42:19.566 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:19.566 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:19.566 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:19.566 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:19.566 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:19.566 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:19.566 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11319 2024-09-21T21:42:19.566 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.566 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:19.566 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.567 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:19.567 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712385+0000", 2024-09-21T21:42:19.567 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:19.567 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.567 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.567 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:19.567 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712385+0000", 2024-09-21T21:42:19.567 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:19.567 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.567 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.567 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:19.567 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712385+0000", 2024-09-21T21:42:19.567 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2800000000000002e-07 2024-09-21T21:42:19.567 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.567 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.568 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:19.568 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712387+0000", 2024-09-21T21:42:19.568 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1569999999999998e-06 2024-09-21T21:42:19.568 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.568 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.568 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:19.568 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712393+0000", 2024-09-21T21:42:19.568 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9370000000000001e-06 2024-09-21T21:42:19.568 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.568 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.568 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:19.568 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712398+0000", 2024-09-21T21:42:19.568 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5020000000000003e-06 2024-09-21T21:42:19.569 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.569 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.569 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:19.569 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.790746+0000", 2024-09-21T21:42:19.569 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078348589999999996 2024-09-21T21:42:19.569 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.569 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.569 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:19.569 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.790844+0000", 2024-09-21T21:42:19.569 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.7526000000000001e-05 2024-09-21T21:42:19.569 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.569 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.569 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:19.569 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.791013+0000", 2024-09-21T21:42:19.570 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00016874899999999999 2024-09-21T21:42:19.570 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:19.570 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:19.570 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:19.570 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.570 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.570 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11320 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 6180864~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:19.570 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.712405+0000", 2024-09-21T21:42:19.570 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 28.850466883999999, 2024-09-21T21:42:19.570 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078882870999999993, 2024-09-21T21:42:19.570 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:19.570 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:19.570 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:19.571 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:19.571 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:19.571 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:19.571 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11320 2024-09-21T21:42:19.571 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.571 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:19.571 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.571 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:19.571 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712405+0000", 2024-09-21T21:42:19.571 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:19.571 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.571 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.571 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:19.572 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712405+0000", 2024-09-21T21:42:19.572 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:19.572 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.572 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.572 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:19.572 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.713942+0000", 2024-09-21T21:42:19.572 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.001536882 2024-09-21T21:42:19.572 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.572 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.572 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:19.572 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.713946+0000", 2024-09-21T21:42:19.572 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4549999999999997e-06 2024-09-21T21:42:19.572 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.572 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.572 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:19.573 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.713958+0000", 2024-09-21T21:42:19.573 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1775e-05 2024-09-21T21:42:19.573 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.573 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.573 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:19.573 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.713965+0000", 2024-09-21T21:42:19.573 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5100000000000001e-06 2024-09-21T21:42:19.573 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.573 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.573 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:19.573 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.791059+0000", 2024-09-21T21:42:19.573 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.077094036000000005 2024-09-21T21:42:19.573 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.573 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.574 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:19.574 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.791175+0000", 2024-09-21T21:42:19.574 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000115895 2024-09-21T21:42:19.574 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.574 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.574 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:19.574 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.791288+0000", 2024-09-21T21:42:19.574 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000112318 2024-09-21T21:42:19.574 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:19.574 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:19.574 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:19.574 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.574 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.574 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11325 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 5414912~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:19.575 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.714095+0000", 2024-09-21T21:42:19.575 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 28.848776889, 2024-09-21T21:42:19.575 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078631894999999993, 2024-09-21T21:42:19.575 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:19.575 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:19.575 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:19.575 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:19.575 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:19.575 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:19.575 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11325 2024-09-21T21:42:19.575 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.575 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:19.575 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.575 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:19.576 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714095+0000", 2024-09-21T21:42:19.576 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:19.576 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.576 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.576 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:19.576 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714095+0000", 2024-09-21T21:42:19.576 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:19.576 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.576 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.576 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:19.576 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714095+0000", 2024-09-21T21:42:19.576 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7500000000000001e-07 2024-09-21T21:42:19.576 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.576 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.576 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:19.577 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714097+0000", 2024-09-21T21:42:19.577 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.12e-06 2024-09-21T21:42:19.577 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.577 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.577 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:19.577 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714104+0000", 2024-09-21T21:42:19.577 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3520000000000003e-06 2024-09-21T21:42:19.577 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.577 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.577 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:19.577 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714108+0000", 2024-09-21T21:42:19.577 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8810000000000004e-06 2024-09-21T21:42:19.577 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.577 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.577 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:19.578 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792509+0000", 2024-09-21T21:42:19.578 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078400739999999997 2024-09-21T21:42:19.578 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.578 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.578 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:19.578 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792613+0000", 2024-09-21T21:42:19.578 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000103939 2024-09-21T21:42:19.578 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.578 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.578 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:19.578 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792727+0000", 2024-09-21T21:42:19.578 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011348799999999999 2024-09-21T21:42:19.579 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:19.579 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:19.579 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:19.579 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.579 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.579 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11326 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 6103040~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:19.579 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.714115+0000", 2024-09-21T21:42:19.579 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 28.848756860999998, 2024-09-21T21:42:19.579 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078963056000000004, 2024-09-21T21:42:19.579 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:19.579 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:19.579 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:19.579 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:19.579 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:19.580 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:19.580 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11326 2024-09-21T21:42:19.580 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.580 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:19.580 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.580 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:19.580 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714115+0000", 2024-09-21T21:42:19.580 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:19.580 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.580 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.580 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:19.580 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714115+0000", 2024-09-21T21:42:19.580 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:19.580 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.581 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.581 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:19.581 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.715940+0000", 2024-09-21T21:42:19.581 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0018254969999999999 2024-09-21T21:42:19.581 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.581 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.581 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:19.581 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.715945+0000", 2024-09-21T21:42:19.581 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6120000000000001e-06 2024-09-21T21:42:19.581 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.581 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.581 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:19.581 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.715954+0000", 2024-09-21T21:42:19.581 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.5899999999999997e-06 2024-09-21T21:42:19.582 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.582 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.582 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:19.582 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.715959+0000", 2024-09-21T21:42:19.582 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0849999999999996e-06 2024-09-21T21:42:19.582 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.582 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.582 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:19.582 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792771+0000", 2024-09-21T21:42:19.582 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.076811829999999998 2024-09-21T21:42:19.582 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.582 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.582 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:19.583 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792882+0000", 2024-09-21T21:42:19.583 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000110857 2024-09-21T21:42:19.583 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.583 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.583 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:19.583 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.793078+0000", 2024-09-21T21:42:19.583 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00019558499999999999 2024-09-21T21:42:19.583 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:19.583 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:19.583 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:19.583 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.583 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.583 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11332 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 7565312~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:19.583 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.716063+0000", 2024-09-21T21:42:19.584 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 28.846808327000002, 2024-09-21T21:42:19.584 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078787482000000006, 2024-09-21T21:42:19.584 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:19.584 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:19.584 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:19.584 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:19.584 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:19.584 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:19.584 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11332 2024-09-21T21:42:19.584 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.584 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:19.584 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.584 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:19.584 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.716063+0000", 2024-09-21T21:42:19.584 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:19.585 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.585 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.585 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:19.585 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.716063+0000", 2024-09-21T21:42:19.585 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:19.585 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.585 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.585 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:19.585 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.717944+0000", 2024-09-21T21:42:19.585 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0018812309999999999 2024-09-21T21:42:19.585 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.585 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.585 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:19.585 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.717949+0000", 2024-09-21T21:42:19.586 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9389999999999996e-06 2024-09-21T21:42:19.586 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.586 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.586 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:19.586 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.717963+0000", 2024-09-21T21:42:19.586 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.3899e-05 2024-09-21T21:42:19.586 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.586 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.586 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:19.586 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.717970+0000", 2024-09-21T21:42:19.586 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.426e-06 2024-09-21T21:42:19.586 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.586 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.587 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:19.587 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.794573+0000", 2024-09-21T21:42:19.587 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.076602990999999995 2024-09-21T21:42:19.587 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.587 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.587 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:19.587 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.794682+0000", 2024-09-21T21:42:19.587 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000109541 2024-09-21T21:42:19.587 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.587 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.587 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:19.587 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.794851+0000", 2024-09-21T21:42:19.587 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000168455 2024-09-21T21:42:19.587 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:19.588 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:19.588 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:19.588 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.588 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.588 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11339 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 6266880~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:19.588 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.718117+0000", 2024-09-21T21:42:19.588 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 28.844754301000002, 2024-09-21T21:42:19.588 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078703717000000006, 2024-09-21T21:42:19.588 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:19.588 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:19.588 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:19.588 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:19.588 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:19.588 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:19.588 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11339 2024-09-21T21:42:19.589 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.589 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:19.589 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.589 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:19.589 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.718117+0000", 2024-09-21T21:42:19.589 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:19.589 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.589 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.589 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:19.589 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.718117+0000", 2024-09-21T21:42:19.589 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:19.589 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.589 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.589 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:19.590 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720019+0000", 2024-09-21T21:42:19.590 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0019012549999999999 2024-09-21T21:42:19.590 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.590 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.590 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:19.590 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720024+0000", 2024-09-21T21:42:19.590 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1100000000000002e-06 2024-09-21T21:42:19.590 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.590 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.590 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:19.590 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720035+0000", 2024-09-21T21:42:19.590 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.094e-05 2024-09-21T21:42:19.590 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.590 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.591 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:19.591 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720040+0000", 2024-09-21T21:42:19.591 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1830000000000002e-06 2024-09-21T21:42:19.591 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.591 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.591 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:19.591 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796618+0000", 2024-09-21T21:42:19.591 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.076578033000000004 2024-09-21T21:42:19.591 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.591 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.591 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:19.591 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796696+0000", 2024-09-21T21:42:19.591 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7836000000000005e-05 2024-09-21T21:42:19.591 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.591 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.592 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:19.592 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796821+0000", 2024-09-21T21:42:19.592 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00012536000000000001 2024-09-21T21:42:19.592 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:19.592 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:19.592 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:19.592 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.592 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.592 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11345 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 5472256~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:19.592 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.720116+0000", 2024-09-21T21:42:19.592 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 28.842755401000002, 2024-09-21T21:42:19.592 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078667730000000005, 2024-09-21T21:42:19.592 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:19.592 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:19.593 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:19.593 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:19.593 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:19.593 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:19.593 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11345 2024-09-21T21:42:19.593 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.593 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:19.593 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.593 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:19.593 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720116+0000", 2024-09-21T21:42:19.593 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:19.593 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.593 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.593 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:19.594 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720116+0000", 2024-09-21T21:42:19.594 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:19.594 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.594 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.594 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:19.594 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720116+0000", 2024-09-21T21:42:19.594 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.04e-07 2024-09-21T21:42:19.594 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.594 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.595 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:19.595 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720118+0000", 2024-09-21T21:42:19.595 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.181e-06 2024-09-21T21:42:19.595 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.595 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.595 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:19.595 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720121+0000", 2024-09-21T21:42:19.595 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0199999999999999e-06 2024-09-21T21:42:19.595 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.595 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.595 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:19.595 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720123+0000", 2024-09-21T21:42:19.595 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.5469999999999999e-06 2024-09-21T21:42:19.595 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.595 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.596 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:19.596 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798626+0000", 2024-09-21T21:42:19.596 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078503137000000001 2024-09-21T21:42:19.596 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.596 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.596 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:19.596 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798677+0000", 2024-09-21T21:42:19.596 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0930999999999997e-05 2024-09-21T21:42:19.596 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.596 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.596 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:19.596 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798784+0000", 2024-09-21T21:42:19.596 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010671 2024-09-21T21:42:19.596 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:19.597 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:19.597 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:19.597 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.597 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.597 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11346 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 5500928~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:19.597 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.720126+0000", 2024-09-21T21:42:19.597 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 28.842745327999999, 2024-09-21T21:42:19.597 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078863421000000003, 2024-09-21T21:42:19.597 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:19.597 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:19.597 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:19.597 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:19.597 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:19.597 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:19.598 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11346 2024-09-21T21:42:19.598 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.598 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:19.598 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.598 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:19.598 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720126+0000", 2024-09-21T21:42:19.598 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:19.598 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.598 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.598 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:19.598 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720126+0000", 2024-09-21T21:42:19.598 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:19.598 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.598 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.599 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:19.599 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.722145+0000", 2024-09-21T21:42:19.599 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0020183219999999999 2024-09-21T21:42:19.599 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.599 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.599 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:19.599 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.722150+0000", 2024-09-21T21:42:19.599 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1760000000000001e-06 2024-09-21T21:42:19.599 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.599 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.600 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:19.600 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.722161+0000", 2024-09-21T21:42:19.600 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1566000000000001e-05 2024-09-21T21:42:19.600 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.600 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.600 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:19.600 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.722169+0000", 2024-09-21T21:42:19.600 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.2840000000000001e-06 2024-09-21T21:42:19.600 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.600 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.600 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:19.600 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798815+0000", 2024-09-21T21:42:19.601 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.076646607000000005 2024-09-21T21:42:19.601 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.601 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.601 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:19.601 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798870+0000", 2024-09-21T21:42:19.601 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4486e-05 2024-09-21T21:42:19.601 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.601 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.601 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:19.601 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798990+0000", 2024-09-21T21:42:19.601 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011998 2024-09-21T21:42:19.601 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:19.601 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:19.602 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:19.602 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.602 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.602 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13629 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7852032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:19.602 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943319+0000", 2024-09-21T21:42:19.602 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 28.619552685999999, 2024-09-21T21:42:19.602 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078610975999999999, 2024-09-21T21:42:19.602 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:19.602 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:19.602 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:19.602 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:19.602 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:19.602 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:19.602 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13629 2024-09-21T21:42:19.603 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.603 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:19.603 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.603 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:19.603 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943319+0000", 2024-09-21T21:42:19.603 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:19.603 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.603 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.603 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:19.603 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943319+0000", 2024-09-21T21:42:19.603 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:19.603 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.603 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.603 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:19.604 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943319+0000", 2024-09-21T21:42:19.604 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6199999999999999e-07 2024-09-21T21:42:19.604 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.604 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.604 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:19.604 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943321+0000", 2024-09-21T21:42:19.604 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1009999999999999e-06 2024-09-21T21:42:19.604 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.604 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.604 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:19.604 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943328+0000", 2024-09-21T21:42:19.605 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.1029999999999996e-06 2024-09-21T21:42:19.605 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.605 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.605 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:19.605 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943333+0000", 2024-09-21T21:42:19.605 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7709999999999997e-06 2024-09-21T21:42:19.605 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.605 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.605 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:19.605 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.021679+0000", 2024-09-21T21:42:19.605 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078345673000000005 2024-09-21T21:42:19.605 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.605 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.606 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:19.606 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.021771+0000", 2024-09-21T21:42:19.606 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2201999999999996e-05 2024-09-21T21:42:19.606 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.606 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.606 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:19.606 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.021930+0000", 2024-09-21T21:42:19.606 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00015876400000000001 2024-09-21T21:42:19.606 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:19.606 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:19.606 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:19.606 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.606 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.607 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13630 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7880704~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:19.607 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943339+0000", 2024-09-21T21:42:19.607 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 28.619532498000002, 2024-09-21T21:42:19.607 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078939410000000002, 2024-09-21T21:42:19.607 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:19.607 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:19.607 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:19.607 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:19.607 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:19.607 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:19.607 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13630 2024-09-21T21:42:19.607 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.607 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:19.607 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.608 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:19.608 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943339+0000", 2024-09-21T21:42:19.608 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:19.608 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.608 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.608 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:19.608 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943339+0000", 2024-09-21T21:42:19.608 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:19.608 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.608 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.608 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:19.608 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943339+0000", 2024-09-21T21:42:19.608 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8000000000000001e-07 2024-09-21T21:42:19.609 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.609 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.609 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:19.609 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943341+0000", 2024-09-21T21:42:19.609 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9850000000000001e-06 2024-09-21T21:42:19.609 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.609 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.609 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:19.609 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943347+0000", 2024-09-21T21:42:19.609 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2979999999999999e-06 2024-09-21T21:42:19.609 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.609 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.609 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:19.610 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943354+0000", 2024-09-21T21:42:19.610 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.2969999999999998e-06 2024-09-21T21:42:19.610 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.610 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.610 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:19.610 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.021996+0000", 2024-09-21T21:42:19.610 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078641680000000005 2024-09-21T21:42:19.610 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.610 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.610 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:19.610 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022104+0000", 2024-09-21T21:42:19.610 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000107902 2024-09-21T21:42:19.610 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.611 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.611 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:19.611 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022278+0000", 2024-09-21T21:42:19.611 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00017486800000000001 2024-09-21T21:42:19.611 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:19.611 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:19.611 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:19.611 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.611 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.611 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13631 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7909376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:19.611 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943362+0000", 2024-09-21T21:42:19.611 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 28.619509678, 2024-09-21T21:42:19.611 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079195644999999995, 2024-09-21T21:42:19.611 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:19.612 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:19.612 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:19.612 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:19.612 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:19.612 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:19.612 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13631 2024-09-21T21:42:19.612 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.612 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:19.612 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.612 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:19.612 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943362+0000", 2024-09-21T21:42:19.612 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:19.612 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.613 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.613 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:19.613 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943362+0000", 2024-09-21T21:42:19.613 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:19.613 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.613 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.613 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:19.613 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943362+0000", 2024-09-21T21:42:19.613 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8799999999999998e-07 2024-09-21T21:42:19.613 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.613 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.613 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:19.613 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943364+0000", 2024-09-21T21:42:19.614 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9850000000000001e-06 2024-09-21T21:42:19.614 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.614 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.614 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:19.614 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943371+0000", 2024-09-21T21:42:19.614 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0650000000000001e-06 2024-09-21T21:42:19.614 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.614 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.614 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:19.614 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943376+0000", 2024-09-21T21:42:19.614 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8250000000000004e-06 2024-09-21T21:42:19.614 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.615 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.615 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:19.615 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022329+0000", 2024-09-21T21:42:19.615 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078952888999999998 2024-09-21T21:42:19.615 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.615 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.615 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:19.615 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022419+0000", 2024-09-21T21:42:19.616 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.9738e-05 2024-09-21T21:42:19.616 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.616 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.616 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:19.616 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022558+0000", 2024-09-21T21:42:19.616 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00013875500000000001 2024-09-21T21:42:19.616 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:19.616 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:19.616 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:19.616 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.616 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.616 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13632 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7966720~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:19.616 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943382+0000", 2024-09-21T21:42:19.616 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 28.619489443999999, 2024-09-21T21:42:19.617 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079495727000000002, 2024-09-21T21:42:19.617 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:19.617 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:19.617 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:19.617 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:19.617 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:19.617 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:19.617 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13632 2024-09-21T21:42:19.617 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.617 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:19.617 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.617 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:19.617 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943382+0000", 2024-09-21T21:42:19.617 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:19.617 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.618 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.618 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:19.618 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943382+0000", 2024-09-21T21:42:19.618 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:19.618 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.618 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.618 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:19.618 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943382+0000", 2024-09-21T21:42:19.618 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7e-07 2024-09-21T21:42:19.618 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.618 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.618 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:19.618 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943385+0000", 2024-09-21T21:42:19.619 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2230000000000001e-06 2024-09-21T21:42:19.619 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.619 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.619 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:19.619 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943391+0000", 2024-09-21T21:42:19.619 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6810000000000001e-06 2024-09-21T21:42:19.619 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.619 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.619 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:19.619 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943397+0000", 2024-09-21T21:42:19.619 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6559999999999997e-06 2024-09-21T21:42:19.619 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.619 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.620 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:19.620 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022601+0000", 2024-09-21T21:42:19.620 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079204051999999997 2024-09-21T21:42:19.620 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.620 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.620 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:19.620 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022704+0000", 2024-09-21T21:42:19.620 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010329599999999999 2024-09-21T21:42:19.620 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.620 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.620 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:19.620 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022878+0000", 2024-09-21T21:42:19.620 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000173449 2024-09-21T21:42:19.620 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:19.621 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:19.621 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:19.621 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.621 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.621 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13633 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7938048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:19.621 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943403+0000", 2024-09-21T21:42:19.621 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 28.619468514000001, 2024-09-21T21:42:19.621 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079794697999999997, 2024-09-21T21:42:19.621 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:19.621 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:19.621 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:19.621 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:19.621 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:19.622 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:19.622 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13633 2024-09-21T21:42:19.622 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.622 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:19.622 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.622 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:19.622 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943403+0000", 2024-09-21T21:42:19.622 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:19.622 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.622 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.622 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:19.622 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943403+0000", 2024-09-21T21:42:19.622 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:19.623 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.623 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.623 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:19.623 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943403+0000", 2024-09-21T21:42:19.623 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.65e-07 2024-09-21T21:42:19.623 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.623 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.623 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:19.623 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943405+0000", 2024-09-21T21:42:19.623 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0020000000000001e-06 2024-09-21T21:42:19.623 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.623 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.623 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:19.624 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943412+0000", 2024-09-21T21:42:19.624 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.8379999999999996e-06 2024-09-21T21:42:19.624 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.624 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.624 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:19.624 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943417+0000", 2024-09-21T21:42:19.624 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7650000000000001e-06 2024-09-21T21:42:19.624 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.624 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.624 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:19.624 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022922+0000", 2024-09-21T21:42:19.624 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079504512999999999 2024-09-21T21:42:19.625 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.625 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.625 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:19.625 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023039+0000", 2024-09-21T21:42:19.625 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000117044 2024-09-21T21:42:19.625 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.625 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.625 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:19.625 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023198+0000", 2024-09-21T21:42:19.625 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000159171 2024-09-21T21:42:19.625 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:19.625 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:19.626 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:19.626 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.626 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.626 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13634 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7995392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:19.626 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943423+0000", 2024-09-21T21:42:19.626 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 28.619448806000001, 2024-09-21T21:42:19.626 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.080142358999999996, 2024-09-21T21:42:19.626 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:19.626 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:19.626 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:19.627 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:19.627 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:19.627 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:19.627 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13634 2024-09-21T21:42:19.627 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.627 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:19.627 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.627 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:19.627 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943423+0000", 2024-09-21T21:42:19.627 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:19.627 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.627 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.628 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:19.628 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943423+0000", 2024-09-21T21:42:19.628 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:19.628 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.628 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.628 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:19.628 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943423+0000", 2024-09-21T21:42:19.628 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.58e-07 2024-09-21T21:42:19.628 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.628 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.628 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:19.628 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943425+0000", 2024-09-21T21:42:19.628 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9180000000000001e-06 2024-09-21T21:42:19.629 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.629 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.629 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:19.629 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943431+0000", 2024-09-21T21:42:19.629 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4439999999999996e-06 2024-09-21T21:42:19.629 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.629 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.629 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:19.629 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943436+0000", 2024-09-21T21:42:19.629 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.887e-06 2024-09-21T21:42:19.629 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.629 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.629 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:19.630 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023266+0000", 2024-09-21T21:42:19.630 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079829622000000003 2024-09-21T21:42:19.630 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.630 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.630 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:19.630 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023384+0000", 2024-09-21T21:42:19.630 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000117658 2024-09-21T21:42:19.630 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.630 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.630 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:19.630 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023565+0000", 2024-09-21T21:42:19.630 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00018147200000000001 2024-09-21T21:42:19.630 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:19.631 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:19.631 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:19.631 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.631 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.631 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13635 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 8024064~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:19.631 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943447+0000", 2024-09-21T21:42:19.631 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 28.619424265999999, 2024-09-21T21:42:19.631 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.080417653000000006, 2024-09-21T21:42:19.631 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:19.631 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:19.631 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:19.631 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:19.631 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:19.631 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:19.632 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13635 2024-09-21T21:42:19.632 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.632 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:19.632 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.632 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:19.632 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943447+0000", 2024-09-21T21:42:19.632 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:19.632 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.632 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.632 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:19.632 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943447+0000", 2024-09-21T21:42:19.632 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:19.632 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.633 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.633 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:19.633 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943448+0000", 2024-09-21T21:42:19.633 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1699999999999999e-07 2024-09-21T21:42:19.633 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.633 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.633 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:19.633 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943450+0000", 2024-09-21T21:42:19.633 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1639999999999999e-06 2024-09-21T21:42:19.633 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.633 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.633 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:19.633 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943456+0000", 2024-09-21T21:42:19.633 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8379999999999998e-06 2024-09-21T21:42:19.634 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.634 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.634 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:19.634 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943460+0000", 2024-09-21T21:42:19.634 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4700000000000004e-06 2024-09-21T21:42:19.634 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.634 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.634 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:19.634 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023595+0000", 2024-09-21T21:42:19.634 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.080134869999999997 2024-09-21T21:42:19.634 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.634 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.634 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:19.635 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023688+0000", 2024-09-21T21:42:19.635 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2794000000000006e-05 2024-09-21T21:42:19.635 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.635 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.635 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:19.635 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023865+0000", 2024-09-21T21:42:19.635 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0001771 2024-09-21T21:42:19.635 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:19.635 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:19.635 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:19.635 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.674 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.674 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13636 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 8052736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:19.674 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943467+0000", 2024-09-21T21:42:19.674 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 28.619404529000001, 2024-09-21T21:42:19.675 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.080694399999999999, 2024-09-21T21:42:19.675 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:19.675 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:19.675 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:19.675 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:19.675 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:19.675 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:19.676 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13636 2024-09-21T21:42:19.676 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.677 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:19.677 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.677 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:19.678 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943467+0000", 2024-09-21T21:42:19.678 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:19.678 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.678 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.678 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:19.678 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943467+0000", 2024-09-21T21:42:19.679 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:19.679 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.679 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.679 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:19.679 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944940+0000", 2024-09-21T21:42:19.679 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0014725109999999999 2024-09-21T21:42:19.679 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.680 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.680 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:19.680 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944949+0000", 2024-09-21T21:42:19.680 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2930000000000006e-06 2024-09-21T21:42:19.680 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.680 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.680 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:19.680 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944958+0000", 2024-09-21T21:42:19.681 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.6899999999999998e-06 2024-09-21T21:42:19.681 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.681 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.681 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:19.681 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944966+0000", 2024-09-21T21:42:19.681 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0690000000000002e-06 2024-09-21T21:42:19.681 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.681 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.682 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:19.682 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023900+0000", 2024-09-21T21:42:19.682 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078934371000000003 2024-09-21T21:42:19.682 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.682 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.682 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:19.682 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024009+0000", 2024-09-21T21:42:19.682 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000108857 2024-09-21T21:42:19.682 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.682 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.683 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:19.683 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024161+0000", 2024-09-21T21:42:19.683 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000152609 2024-09-21T21:42:19.683 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:19.683 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:19.683 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:19.683 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.683 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.683 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13637 3.12 3.1dc31b72 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:19.683 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.944978+0000", 2024-09-21T21:42:19.683 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 28.617893594000002, 2024-09-21T21:42:19.683 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079232604999999998, 2024-09-21T21:42:19.683 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:19.683 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:19.684 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:42:19.684 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:19.684 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:19.684 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:19.684 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13637 2024-09-21T21:42:19.684 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.684 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:19.684 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.684 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:19.684 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944978+0000", 2024-09-21T21:42:19.684 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:19.684 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.684 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.684 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:19.684 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944978+0000", 2024-09-21T21:42:19.685 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:19.685 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.685 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.685 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:19.685 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944978+0000", 2024-09-21T21:42:19.685 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9100000000000004e-07 2024-09-21T21:42:19.685 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.685 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.685 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:19.685 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944981+0000", 2024-09-21T21:42:19.685 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3290000000000001e-06 2024-09-21T21:42:19.685 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.685 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.685 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:19.685 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944987+0000", 2024-09-21T21:42:19.686 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.28e-06 2024-09-21T21:42:19.686 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.686 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.686 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:19.686 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944992+0000", 2024-09-21T21:42:19.686 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8520000000000003e-06 2024-09-21T21:42:19.686 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.686 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.686 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:19.686 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024196+0000", 2024-09-21T21:42:19.686 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079203673000000002 2024-09-21T21:42:19.686 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.686 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.686 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:19.687 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024211+0000", 2024-09-21T21:42:19.687 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.4980000000000001e-05 2024-09-21T21:42:19.687 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:19.687 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:19.687 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:19.687 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.687 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.687 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13638 3.12 3.1dc31b72 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:19.687 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.945000+0000", 2024-09-21T21:42:19.687 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 28.617871466, 2024-09-21T21:42:19.687 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079257465999999999, 2024-09-21T21:42:19.687 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:19.687 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:19.687 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:42:19.688 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:19.688 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:19.688 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:19.688 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13638 2024-09-21T21:42:19.688 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.688 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:19.688 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.688 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:19.688 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945000+0000", 2024-09-21T21:42:19.688 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:19.688 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.688 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.688 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:19.688 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945000+0000", 2024-09-21T21:42:19.688 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:19.689 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.689 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.689 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:19.689 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945001+0000", 2024-09-21T21:42:19.689 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.32e-07 2024-09-21T21:42:19.689 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.689 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.689 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:19.689 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945003+0000", 2024-09-21T21:42:19.689 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2309999999999999e-06 2024-09-21T21:42:19.689 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.689 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.689 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:19.689 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945009+0000", 2024-09-21T21:42:19.689 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7450000000000001e-06 2024-09-21T21:42:19.689 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.689 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.690 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:19.690 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945013+0000", 2024-09-21T21:42:19.690 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.673e-06 2024-09-21T21:42:19.690 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.690 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.690 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:19.690 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024245+0000", 2024-09-21T21:42:19.690 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079232160999999995 2024-09-21T21:42:19.690 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.690 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.690 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:19.690 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024258+0000", 2024-09-21T21:42:19.690 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.2224e-05 2024-09-21T21:42:19.690 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:19.691 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:19.691 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:19.691 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.691 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.691 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13639 3.12 3.1dc31b72 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:19.691 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.945034+0000", 2024-09-21T21:42:19.691 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 28.617837667, 2024-09-21T21:42:19.691 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079260627, 2024-09-21T21:42:19.691 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:19.691 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:19.691 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:42:19.691 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:19.691 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:19.692 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:19.692 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13639 2024-09-21T21:42:19.692 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.692 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:19.692 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.692 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:19.692 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945034+0000", 2024-09-21T21:42:19.692 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:19.692 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.692 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.692 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:19.692 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945034+0000", 2024-09-21T21:42:19.692 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:19.692 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.693 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.693 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:19.693 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945034+0000", 2024-09-21T21:42:19.693 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3799999999999998e-07 2024-09-21T21:42:19.693 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.693 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.693 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:19.694 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945036+0000", 2024-09-21T21:42:19.694 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.052e-06 2024-09-21T21:42:19.694 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.694 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.694 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:19.694 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945040+0000", 2024-09-21T21:42:19.694 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1169999999999997e-06 2024-09-21T21:42:19.694 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.694 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.695 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:19.695 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945043+0000", 2024-09-21T21:42:19.695 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.8439999999999998e-06 2024-09-21T21:42:19.695 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.695 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.695 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:19.695 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024286+0000", 2024-09-21T21:42:19.695 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079242483000000002 2024-09-21T21:42:19.695 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.695 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.695 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:19.695 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024295+0000", 2024-09-21T21:42:19.695 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.7930000000000003e-06 2024-09-21T21:42:19.696 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:19.696 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:19.696 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:19.696 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.696 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.696 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13640 3.12 3.1dc31b72 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:19.696 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.945047+0000", 2024-09-21T21:42:19.696 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 28.617824145, 2024-09-21T21:42:19.696 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079412321999999994, 2024-09-21T21:42:19.696 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:19.696 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:19.696 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:42:19.696 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:19.696 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:19.697 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:19.697 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13640 2024-09-21T21:42:19.697 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.697 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:19.697 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.697 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:19.697 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945047+0000", 2024-09-21T21:42:19.697 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:19.697 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.697 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.697 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:19.697 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945047+0000", 2024-09-21T21:42:19.697 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:19.697 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.698 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.698 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:19.698 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945048+0000", 2024-09-21T21:42:19.698 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0499999999999999e-07 2024-09-21T21:42:19.698 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.698 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.698 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:19.698 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945049+0000", 2024-09-21T21:42:19.698 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.3939999999999999e-06 2024-09-21T21:42:19.698 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.699 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.699 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:19.699 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945053+0000", 2024-09-21T21:42:19.699 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1760000000000003e-06 2024-09-21T21:42:19.699 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.699 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.699 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:19.699 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945056+0000", 2024-09-21T21:42:19.699 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.6129999999999998e-06 2024-09-21T21:42:19.699 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.699 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.699 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:19.699 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024360+0000", 2024-09-21T21:42:19.699 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079303645000000006 2024-09-21T21:42:19.700 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:19.700 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:19.700 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:19.700 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024460+0000", 2024-09-21T21:42:19.700 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000100189 2024-09-21T21:42:19.700 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:19.700 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:19.700 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:19.700 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:19.700 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:19.700 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:19.702 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T21:42:19.702 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:19.700+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:19.801 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T21:42:19.802 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:19.801+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:19.848 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:19.848 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:42:19.848 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 0 2024-09-21T21:42:19.848 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:19.857 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T21:42:19.903 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:19.902+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:19.924 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0: {} 2024-09-21T21:42:19.925 INFO:teuthology.orchestra.run.smithi081.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T21:42:19.944 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1: {} 2024-09-21T21:42:19.944 INFO:teuthology.orchestra.run.smithi081.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T21:42:19.965 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2: {} 2024-09-21T21:42:19.965 INFO:teuthology.orchestra.run.smithi081.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T21:42:19.971 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:19.971 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:19.980 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T21:42:19.982 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3: {} 2024-09-21T21:42:19.983 INFO:teuthology.orchestra.run.smithi081.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T21:42:19.999 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4: {} 2024-09-21T21:42:19.999 INFO:teuthology.orchestra.run.smithi081.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T21:42:20.004 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:20.003+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:20.012 INFO:teuthology.orchestra.run.smithi164.stdout:checking clone #37, image image_client.0-clone37 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent38 2024-09-21T21:42:20.016 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5: {} 2024-09-21T21:42:20.016 INFO:teuthology.orchestra.run.smithi081.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T21:42:20.099 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:20.099 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:20.105 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:20.104+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:20.111 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T21:42:20.206 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:20.205+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:20.279 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:20.280 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:20.290 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T21:42:20.307 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:20.305+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:20.407 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:20.406+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:20.470 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:20.470 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:20.482 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T21:42:20.490 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T21:42:20.508 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:20.506+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:20.609 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:20.608+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:20.632 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:20.632 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:20.644 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T21:42:20.710 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:20.709+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:20.775 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:20.776 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:20.784 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T21:42:20.810 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:20.809+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:20.885 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:20.886 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:20.894 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T21:42:20.937 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:20.936+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:20.956 INFO:teuthology.orchestra.run.smithi164.stdout:5115 write 0x743c200 thru 0x743f277 (0x3078 bytes) 2024-09-21T21:42:20.958 INFO:teuthology.orchestra.run.smithi164.stdout:5120 write 0x3c15200 thru 0x3c18a02 (0x3803 bytes) 2024-09-21T21:42:20.958 INFO:teuthology.orchestra.run.smithi164.stdout:5121 read 0x1cf0000 thru 0x1cf0926 (0x927 bytes) 2024-09-21T21:42:20.959 INFO:teuthology.orchestra.run.smithi164.stdout:5122 write 0x94ede00 thru 0x94fa558 (0xc759 bytes) 2024-09-21T21:42:20.960 INFO:teuthology.orchestra.run.smithi164.stdout:5123 read 0x4709e00 thru 0x4712765 (0x8966 bytes) 2024-09-21T21:42:20.960 INFO:teuthology.orchestra.run.smithi164.stdout:5124 punch from 0xa133200 to 0xa13b000, (0x7e00 bytes) 2024-09-21T21:42:21.011 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:21.010+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:21.021 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:21.021 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:21.030 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T21:42:21.055 INFO:teuthology.orchestra.run.smithi164.stdout:5125 write 0x137ac00 thru 0x137e868 (0x3c69 bytes) 2024-09-21T21:42:21.056 INFO:teuthology.orchestra.run.smithi164.stdout:5126 read 0xab77a00 thru 0xab7fe79 (0x847a bytes) 2024-09-21T21:42:21.056 INFO:teuthology.orchestra.run.smithi164.stdout:5129 punch from 0x59c2200 to 0x59c7c00, (0x5a00 bytes) 2024-09-21T21:42:21.114 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:21.113+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:21.213 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:21.212+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:21.217 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:21.217 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:21.227 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T21:42:21.306 INFO:teuthology.orchestra.run.smithi164.stdout:5131 write 0xd8b8c00 thru 0xd8c2d4f (0xa150 bytes) 2024-09-21T21:42:21.314 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:21.313+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:21.346 INFO:teuthology.orchestra.run.smithi164.stdout:5132 write 0xdd2a000 thru 0xdd336cf (0x96d0 bytes) 2024-09-21T21:42:21.346 INFO:teuthology.orchestra.run.smithi164.stdout:5133 write 0x246ce00 thru 0x247bf8e (0xf18f bytes) 2024-09-21T21:42:21.347 INFO:teuthology.orchestra.run.smithi164.stdout:5135 write 0xcedec00 thru 0xcee24b8 (0x38b9 bytes) 2024-09-21T21:42:21.413 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:21.413 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:21.415 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:21.413+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:21.426 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T21:42:21.480 INFO:teuthology.orchestra.run.smithi164.stdout:5136 write 0xded9000 thru 0xdee4da7 (0xbda8 bytes) 2024-09-21T21:42:21.488 INFO:teuthology.orchestra.run.smithi164.stdout:5140 write 0x5000600 thru 0x5002c0e (0x260f bytes) 2024-09-21T21:42:21.493 INFO:teuthology.orchestra.run.smithi081.stderr:nodeep-scrub is unset 2024-09-21T21:42:21.503 INFO:teuthology.orchestra.run.smithi164.stdout:5145 trunc from 0xdee4e00 to 0x2c2ce00 2024-09-21T21:42:21.550 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:21.548+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:21.603 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:21.603 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:21.613 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T21:42:21.616 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:21.614+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:21.717 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:21.715+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:21.746 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:21.747 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:21.747 INFO:teuthology.orchestra.run.smithi164.stdout:5150 write 0x32e2800 thru 0x32e7913 (0x5114 bytes) 2024-09-21T21:42:21.756 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T21:42:21.757 INFO:teuthology.orchestra.run.smithi164.stdout:5152 trunc from 0x32e7a00 to 0xa011a00 2024-09-21T21:42:21.794 INFO:teuthology.orchestra.run.smithi164.stdout:5154 write 0xb4f1e00 thru 0xb4f7b12 (0x5d13 bytes) 2024-09-21T21:42:21.794 INFO:teuthology.orchestra.run.smithi164.stdout:5155 read 0x705a400 thru 0x705f5cf (0x51d0 bytes) 2024-09-21T21:42:21.796 INFO:teuthology.orchestra.run.smithi164.stdout:5156 trunc from 0xb4f7c00 to 0xc888800 2024-09-21T21:42:21.818 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:21.816+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:21.833 INFO:teuthology.orchestra.run.smithi164.stdout:5158 trunc from 0xc888800 to 0x642a800 2024-09-21T21:42:21.855 INFO:teuthology.orchestra.run.smithi164.stdout:5159 read 0x2e24c00 thru 0x2e2e62b (0x9a2c bytes) 2024-09-21T21:42:21.866 INFO:teuthology.orchestra.run.smithi164.stdout:5163 write 0x70be000 thru 0x70c95fd (0xb5fe bytes) 2024-09-21T21:42:21.904 INFO:teuthology.orchestra.run.smithi164.stdout:5164 write 0xe087e00 thru 0xe08be5e (0x405f bytes) 2024-09-21T21:42:21.905 INFO:teuthology.orchestra.run.smithi164.stdout:5165 write 0x62b4c00 thru 0x62bbe22 (0x7223 bytes) 2024-09-21T21:42:21.905 INFO:teuthology.orchestra.run.smithi164.stdout:5169 read 0x12a4c00 thru 0x12b143e (0xc83f bytes) 2024-09-21T21:42:21.906 INFO:teuthology.orchestra.run.smithi164.stdout:5171 punch from 0x7fcf600 to 0x7fd8400, (0x8e00 bytes) 2024-09-21T21:42:21.907 INFO:teuthology.orchestra.run.smithi164.stdout:5173 read 0x965400 thru 0x967b7b (0x277c bytes) 2024-09-21T21:42:21.908 INFO:teuthology.orchestra.run.smithi164.stdout:5177 read 0xb9da000 thru 0xb9dc82d (0x282e bytes) 2024-09-21T21:42:21.908 INFO:teuthology.orchestra.run.smithi164.stdout:5181 write 0xc70a000 thru 0xc70de46 (0x3e47 bytes) 2024-09-21T21:42:21.908 INFO:teuthology.orchestra.run.smithi164.stdout:5183 read 0x3ed4800 thru 0x3ed832a (0x3b2b bytes) 2024-09-21T21:42:21.908 INFO:teuthology.orchestra.run.smithi164.stdout:5184 write 0x4ed3200 thru 0x4edbd59 (0x8b5a bytes) 2024-09-21T21:42:21.911 INFO:teuthology.orchestra.run.smithi164.stdout:5185 trunc from 0xe08c000 to 0xb0bc600 2024-09-21T21:42:21.918 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:21.917+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:21.930 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:21.930 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:21.942 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T21:42:21.989 INFO:teuthology.orchestra.run.smithi164.stdout:5186 write 0x1787800 thru 0x17959bc (0xe1bd bytes) 2024-09-21T21:42:21.989 INFO:teuthology.orchestra.run.smithi164.stdout:5191 write 0x931a200 thru 0x932517a (0xaf7b bytes) 2024-09-21T21:42:21.989 INFO:teuthology.orchestra.run.smithi164.stdout:5192 read 0x175d200 thru 0x1767211 (0xa012 bytes) 2024-09-21T21:42:21.991 INFO:teuthology.orchestra.run.smithi164.stdout:5194 read 0x757b200 thru 0x7587402 (0xc203 bytes) 2024-09-21T21:42:21.991 INFO:teuthology.orchestra.run.smithi164.stdout:5195 read 0x5bf0e00 thru 0x5bf5919 (0x4b1a bytes) 2024-09-21T21:42:21.991 INFO:teuthology.orchestra.run.smithi164.stdout:5196 write 0x1c7da00 thru 0x1c84947 (0x6f48 bytes) 2024-09-21T21:42:21.992 INFO:teuthology.orchestra.run.smithi164.stdout:5197 punch from 0x9115400 to 0x911da00, (0x8600 bytes) 2024-09-21T21:42:22.020 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:22.018+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:22.029 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T21:42:22.094 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:22.094 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:22.108 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T21:42:22.120 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:22.119+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:22.173 INFO:teuthology.orchestra.run.smithi164.stdout:5200 trunc from 0xb0bc600 to 0xa720e00 2024-09-21T21:42:22.189 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0: {} 2024-09-21T21:42:22.190 INFO:teuthology.orchestra.run.smithi081.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T21:42:22.210 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1: {} 2024-09-21T21:42:22.210 INFO:teuthology.orchestra.run.smithi081.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T21:42:22.221 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:22.220+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:22.228 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2: {} 2024-09-21T21:42:22.229 INFO:teuthology.orchestra.run.smithi081.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T21:42:22.241 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:22.241 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:42:22.242 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 0 2024-09-21T21:42:22.242 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:22.245 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3: {} 2024-09-21T21:42:22.246 INFO:teuthology.orchestra.run.smithi081.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T21:42:22.252 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T21:42:22.258 INFO:teuthology.orchestra.run.smithi164.stdout:5201 write 0x7a54e00 thru 0x7a58c08 (0x3e09 bytes) 2024-09-21T21:42:22.260 INFO:teuthology.orchestra.run.smithi164.stdout:5203 trunc from 0xa720e00 to 0xb9ce000 2024-09-21T21:42:22.266 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4: {} 2024-09-21T21:42:22.266 INFO:teuthology.orchestra.run.smithi081.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T21:42:22.288 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5: {} 2024-09-21T21:42:22.288 INFO:teuthology.orchestra.run.smithi081.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T21:42:22.301 INFO:teuthology.orchestra.run.smithi164.stdout:5204 trunc from 0xb9ce000 to 0x334200 2024-09-21T21:42:22.322 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:22.321+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:22.382 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:22.382 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:42:22.382 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:42:22.382 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:42:22.382 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:22.391 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T21:42:22.423 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:22.422+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:22.488 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 5, 3] out_osds: [0, 4] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-21T21:42:22.489 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T21:42:22.489 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-21T21:42:22.489 INFO:tasks.thrashosds.thrasher:Setting osd 5 primary_affinity to 1.000000 2024-09-21T21:42:22.489 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-09-21T21:42:22.520 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:22.521 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 20, 2024-09-21T21:42:22.521 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 600, 2024-09-21T21:42:22.521 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:42:22.521 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.521 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11319 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 5492736~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:22.521 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.712385+0000", 2024-09-21T21:42:22.521 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 31.805257003000001, 2024-09-21T21:42:22.521 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078627889000000006, 2024-09-21T21:42:22.521 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:22.521 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:22.521 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:22.521 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:22.521 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:22.521 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:22.522 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11319 2024-09-21T21:42:22.522 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.522 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:22.522 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.522 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:22.522 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712385+0000", 2024-09-21T21:42:22.522 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.522 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.522 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.522 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:22.522 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712385+0000", 2024-09-21T21:42:22.522 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.522 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.523 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.523 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:22.523 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712385+0000", 2024-09-21T21:42:22.523 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2800000000000002e-07 2024-09-21T21:42:22.523 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.523 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.523 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:22.523 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712387+0000", 2024-09-21T21:42:22.523 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1569999999999998e-06 2024-09-21T21:42:22.523 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.523 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.523 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:22.523 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712393+0000", 2024-09-21T21:42:22.523 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9370000000000001e-06 2024-09-21T21:42:22.524 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.524 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.524 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:22.524 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712398+0000", 2024-09-21T21:42:22.524 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5020000000000003e-06 2024-09-21T21:42:22.524 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.524 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.524 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:22.524 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.790746+0000", 2024-09-21T21:42:22.524 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078348589999999996 2024-09-21T21:42:22.524 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.524 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.524 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:22.524 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.790844+0000", 2024-09-21T21:42:22.525 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.7526000000000001e-05 2024-09-21T21:42:22.525 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.525 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.525 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:22.525 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.791013+0000", 2024-09-21T21:42:22.525 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00016874899999999999 2024-09-21T21:42:22.525 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.525 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:22.525 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.525 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.525 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.525 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11320 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 6180864~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:22.525 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.712405+0000", 2024-09-21T21:42:22.525 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 31.805237043999998, 2024-09-21T21:42:22.526 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078882870999999993, 2024-09-21T21:42:22.526 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:22.526 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:22.526 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:22.526 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:22.526 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:22.526 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:22.526 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11320 2024-09-21T21:42:22.526 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.526 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:22.526 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.526 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:22.526 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712405+0000", 2024-09-21T21:42:22.526 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.527 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.527 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.527 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:22.527 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712405+0000", 2024-09-21T21:42:22.527 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.527 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.527 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.527 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:22.527 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.713942+0000", 2024-09-21T21:42:22.527 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.001536882 2024-09-21T21:42:22.527 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.527 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.527 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:22.528 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.713946+0000", 2024-09-21T21:42:22.528 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4549999999999997e-06 2024-09-21T21:42:22.528 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.528 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.528 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:22.528 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.713958+0000", 2024-09-21T21:42:22.528 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1775e-05 2024-09-21T21:42:22.528 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.528 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.528 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:22.528 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.713965+0000", 2024-09-21T21:42:22.528 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5100000000000001e-06 2024-09-21T21:42:22.528 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.528 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.529 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:22.529 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.791059+0000", 2024-09-21T21:42:22.529 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.077094036000000005 2024-09-21T21:42:22.529 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.529 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.529 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:22.529 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.791175+0000", 2024-09-21T21:42:22.529 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000115895 2024-09-21T21:42:22.529 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.529 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.529 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:22.529 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.791288+0000", 2024-09-21T21:42:22.529 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000112318 2024-09-21T21:42:22.529 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.530 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:22.530 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.530 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.530 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.530 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11325 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 5414912~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:22.530 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.714095+0000", 2024-09-21T21:42:22.530 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 31.803547048999999, 2024-09-21T21:42:22.530 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078631894999999993, 2024-09-21T21:42:22.530 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:22.530 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:22.530 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:22.530 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:22.530 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:22.530 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:22.530 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11325 2024-09-21T21:42:22.531 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.531 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:22.531 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.531 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:22.531 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714095+0000", 2024-09-21T21:42:22.531 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.531 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.531 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.531 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:22.531 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714095+0000", 2024-09-21T21:42:22.531 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.531 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.531 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.531 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:22.532 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714095+0000", 2024-09-21T21:42:22.532 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7500000000000001e-07 2024-09-21T21:42:22.532 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.532 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.532 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:22.532 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714097+0000", 2024-09-21T21:42:22.532 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.12e-06 2024-09-21T21:42:22.532 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.532 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.532 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:22.532 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714104+0000", 2024-09-21T21:42:22.532 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3520000000000003e-06 2024-09-21T21:42:22.532 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.532 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.532 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:22.533 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714108+0000", 2024-09-21T21:42:22.533 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8810000000000004e-06 2024-09-21T21:42:22.533 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.533 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.533 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:22.533 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792509+0000", 2024-09-21T21:42:22.533 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078400739999999997 2024-09-21T21:42:22.533 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.533 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.533 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:22.533 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792613+0000", 2024-09-21T21:42:22.533 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000103939 2024-09-21T21:42:22.533 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.533 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.534 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:22.534 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792727+0000", 2024-09-21T21:42:22.534 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011348799999999999 2024-09-21T21:42:22.534 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.534 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:22.534 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.534 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.534 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.534 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11326 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 6103040~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:22.534 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.714115+0000", 2024-09-21T21:42:22.534 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 31.803527021000001, 2024-09-21T21:42:22.534 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078963056000000004, 2024-09-21T21:42:22.534 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:22.534 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:22.535 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:22.535 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:22.535 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:22.535 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:22.535 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11326 2024-09-21T21:42:22.535 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.535 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:22.535 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.535 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:22.535 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714115+0000", 2024-09-21T21:42:22.535 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.535 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.535 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.535 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:22.536 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714115+0000", 2024-09-21T21:42:22.536 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.536 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.536 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.536 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:22.536 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.715940+0000", 2024-09-21T21:42:22.536 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0018254969999999999 2024-09-21T21:42:22.536 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.536 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.536 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:22.536 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.715945+0000", 2024-09-21T21:42:22.536 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6120000000000001e-06 2024-09-21T21:42:22.536 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.536 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.537 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:22.537 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.715954+0000", 2024-09-21T21:42:22.537 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.5899999999999997e-06 2024-09-21T21:42:22.537 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.537 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.537 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:22.537 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.715959+0000", 2024-09-21T21:42:22.537 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0849999999999996e-06 2024-09-21T21:42:22.537 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.537 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.537 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:22.537 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792771+0000", 2024-09-21T21:42:22.537 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.076811829999999998 2024-09-21T21:42:22.537 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.538 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.538 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:22.538 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792882+0000", 2024-09-21T21:42:22.538 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000110857 2024-09-21T21:42:22.538 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.538 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.538 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:22.538 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.793078+0000", 2024-09-21T21:42:22.538 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00019558499999999999 2024-09-21T21:42:22.538 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.538 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:22.538 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.538 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.538 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.539 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11332 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 7565312~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:22.539 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.716063+0000", 2024-09-21T21:42:22.539 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 31.801578487, 2024-09-21T21:42:22.539 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078787482000000006, 2024-09-21T21:42:22.539 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:22.539 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:22.539 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:22.539 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:22.539 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:22.539 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:22.539 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11332 2024-09-21T21:42:22.539 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.539 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:22.540 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.540 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:22.540 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.716063+0000", 2024-09-21T21:42:22.540 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.540 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.540 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.540 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:22.540 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.716063+0000", 2024-09-21T21:42:22.540 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.540 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.540 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.540 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:22.540 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.717944+0000", 2024-09-21T21:42:22.540 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0018812309999999999 2024-09-21T21:42:22.541 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.541 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.541 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:22.541 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.717949+0000", 2024-09-21T21:42:22.541 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9389999999999996e-06 2024-09-21T21:42:22.541 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.541 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.541 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:22.541 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.717963+0000", 2024-09-21T21:42:22.541 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.3899e-05 2024-09-21T21:42:22.541 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.541 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.541 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:22.542 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.717970+0000", 2024-09-21T21:42:22.542 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.426e-06 2024-09-21T21:42:22.542 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.542 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.542 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:22.542 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.794573+0000", 2024-09-21T21:42:22.542 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.076602990999999995 2024-09-21T21:42:22.542 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.542 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.542 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:22.542 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.794682+0000", 2024-09-21T21:42:22.542 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000109541 2024-09-21T21:42:22.542 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.542 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.543 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:22.543 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.794851+0000", 2024-09-21T21:42:22.543 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000168455 2024-09-21T21:42:22.543 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.543 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:22.543 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.543 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.543 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.543 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11339 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 6266880~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:22.543 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.718117+0000", 2024-09-21T21:42:22.543 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 31.799524461000001, 2024-09-21T21:42:22.543 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078703717000000006, 2024-09-21T21:42:22.543 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:22.543 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:22.543 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:22.544 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:22.544 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:22.544 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:22.544 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11339 2024-09-21T21:42:22.544 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.544 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:22.544 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.544 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:22.544 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.718117+0000", 2024-09-21T21:42:22.544 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.544 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.544 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.544 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:22.544 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.718117+0000", 2024-09-21T21:42:22.545 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.545 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.545 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.545 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:22.545 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720019+0000", 2024-09-21T21:42:22.545 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0019012549999999999 2024-09-21T21:42:22.545 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.545 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.545 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:22.545 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720024+0000", 2024-09-21T21:42:22.545 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1100000000000002e-06 2024-09-21T21:42:22.545 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.545 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.545 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:22.546 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720035+0000", 2024-09-21T21:42:22.546 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.094e-05 2024-09-21T21:42:22.546 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.546 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.546 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:22.546 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720040+0000", 2024-09-21T21:42:22.546 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1830000000000002e-06 2024-09-21T21:42:22.546 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.546 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.546 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:22.546 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796618+0000", 2024-09-21T21:42:22.546 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.076578033000000004 2024-09-21T21:42:22.546 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.546 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.547 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:22.547 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796696+0000", 2024-09-21T21:42:22.547 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7836000000000005e-05 2024-09-21T21:42:22.547 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.547 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.547 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:22.547 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796821+0000", 2024-09-21T21:42:22.547 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00012536000000000001 2024-09-21T21:42:22.547 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.547 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:22.547 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.547 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.547 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.547 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11345 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 5472256~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:22.547 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.720116+0000", 2024-09-21T21:42:22.548 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 31.797525561, 2024-09-21T21:42:22.548 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078667730000000005, 2024-09-21T21:42:22.548 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:22.548 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:22.548 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:22.548 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:22.548 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:22.548 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:22.548 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11345 2024-09-21T21:42:22.548 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.548 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:22.548 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.548 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:22.549 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720116+0000", 2024-09-21T21:42:22.549 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.549 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.549 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.549 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:22.549 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720116+0000", 2024-09-21T21:42:22.549 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.549 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.549 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.549 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:22.549 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720116+0000", 2024-09-21T21:42:22.549 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.04e-07 2024-09-21T21:42:22.549 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.550 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.550 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:22.550 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720118+0000", 2024-09-21T21:42:22.550 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.181e-06 2024-09-21T21:42:22.550 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.550 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.550 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:22.550 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720121+0000", 2024-09-21T21:42:22.550 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0199999999999999e-06 2024-09-21T21:42:22.550 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.550 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.550 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:22.550 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720123+0000", 2024-09-21T21:42:22.550 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.5469999999999999e-06 2024-09-21T21:42:22.551 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.551 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.551 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:22.551 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798626+0000", 2024-09-21T21:42:22.551 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078503137000000001 2024-09-21T21:42:22.551 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.551 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.551 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:22.551 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798677+0000", 2024-09-21T21:42:22.551 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0930999999999997e-05 2024-09-21T21:42:22.551 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.551 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.551 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:22.551 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798784+0000", 2024-09-21T21:42:22.552 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010671 2024-09-21T21:42:22.552 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.552 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:22.552 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.552 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.552 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.552 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11346 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 5500928~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:22.552 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.720126+0000", 2024-09-21T21:42:22.552 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 31.797515487999998, 2024-09-21T21:42:22.552 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078863421000000003, 2024-09-21T21:42:22.552 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:22.552 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:22.552 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:22.552 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:22.552 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:22.553 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:22.553 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11346 2024-09-21T21:42:22.553 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.553 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:22.553 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.553 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:22.553 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720126+0000", 2024-09-21T21:42:22.553 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.553 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.553 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.553 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:22.553 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720126+0000", 2024-09-21T21:42:22.553 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.553 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.553 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.554 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:22.554 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.722145+0000", 2024-09-21T21:42:22.554 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0020183219999999999 2024-09-21T21:42:22.554 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.554 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.554 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:22.554 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.722150+0000", 2024-09-21T21:42:22.554 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1760000000000001e-06 2024-09-21T21:42:22.554 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.554 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.554 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:22.554 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.722161+0000", 2024-09-21T21:42:22.554 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1566000000000001e-05 2024-09-21T21:42:22.555 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.555 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.555 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:22.555 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.722169+0000", 2024-09-21T21:42:22.555 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.2840000000000001e-06 2024-09-21T21:42:22.555 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.555 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.555 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:22.555 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798815+0000", 2024-09-21T21:42:22.555 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.076646607000000005 2024-09-21T21:42:22.555 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.555 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.555 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:22.555 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798870+0000", 2024-09-21T21:42:22.555 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4486e-05 2024-09-21T21:42:22.556 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.556 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.556 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:22.556 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798990+0000", 2024-09-21T21:42:22.556 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011998 2024-09-21T21:42:22.556 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.556 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:22.556 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.556 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.556 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.556 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13629 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7852032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:22.556 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943319+0000", 2024-09-21T21:42:22.556 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 31.574322846000001, 2024-09-21T21:42:22.556 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078610975999999999, 2024-09-21T21:42:22.557 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:22.557 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:22.557 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:22.557 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:22.557 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:22.557 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:22.557 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13629 2024-09-21T21:42:22.557 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.557 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:22.557 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.557 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:22.557 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943319+0000", 2024-09-21T21:42:22.557 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.557 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.558 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.558 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:22.558 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943319+0000", 2024-09-21T21:42:22.558 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.558 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.558 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.558 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:22.558 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943319+0000", 2024-09-21T21:42:22.558 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6199999999999999e-07 2024-09-21T21:42:22.558 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.558 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.558 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:22.558 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943321+0000", 2024-09-21T21:42:22.558 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1009999999999999e-06 2024-09-21T21:42:22.559 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.559 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.559 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:22.559 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943328+0000", 2024-09-21T21:42:22.559 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.1029999999999996e-06 2024-09-21T21:42:22.559 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.559 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.559 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:22.559 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943333+0000", 2024-09-21T21:42:22.559 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7709999999999997e-06 2024-09-21T21:42:22.559 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.559 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.559 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:22.559 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.021679+0000", 2024-09-21T21:42:22.559 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078345673000000005 2024-09-21T21:42:22.560 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.560 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.560 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:22.560 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.021771+0000", 2024-09-21T21:42:22.560 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2201999999999996e-05 2024-09-21T21:42:22.560 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.560 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.560 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:22.560 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.021930+0000", 2024-09-21T21:42:22.560 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00015876400000000001 2024-09-21T21:42:22.560 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.560 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:22.560 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.560 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.560 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.561 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13630 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7880704~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:22.561 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943339+0000", 2024-09-21T21:42:22.561 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 31.574302658000001, 2024-09-21T21:42:22.561 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078939410000000002, 2024-09-21T21:42:22.561 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:22.561 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:22.561 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:22.561 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:22.561 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:22.561 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:22.561 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13630 2024-09-21T21:42:22.561 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.561 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:22.562 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.562 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:22.562 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943339+0000", 2024-09-21T21:42:22.562 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.562 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.562 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.562 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:22.562 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943339+0000", 2024-09-21T21:42:22.562 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.562 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.562 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.562 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:22.562 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943339+0000", 2024-09-21T21:42:22.562 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8000000000000001e-07 2024-09-21T21:42:22.563 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.563 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.563 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:22.563 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943341+0000", 2024-09-21T21:42:22.563 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9850000000000001e-06 2024-09-21T21:42:22.563 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.563 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.563 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:22.563 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943347+0000", 2024-09-21T21:42:22.563 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2979999999999999e-06 2024-09-21T21:42:22.563 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.563 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.563 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:22.563 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943354+0000", 2024-09-21T21:42:22.563 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.2969999999999998e-06 2024-09-21T21:42:22.564 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.564 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.564 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:22.564 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.021996+0000", 2024-09-21T21:42:22.564 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078641680000000005 2024-09-21T21:42:22.564 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.564 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.564 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:22.564 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022104+0000", 2024-09-21T21:42:22.564 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000107902 2024-09-21T21:42:22.564 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.564 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.564 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:22.565 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022278+0000", 2024-09-21T21:42:22.565 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00017486800000000001 2024-09-21T21:42:22.565 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.565 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:22.565 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.565 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.565 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.565 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13631 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7909376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:22.565 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943362+0000", 2024-09-21T21:42:22.565 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 31.574279837999999, 2024-09-21T21:42:22.565 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079195644999999995, 2024-09-21T21:42:22.565 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:22.565 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:22.565 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:22.566 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:22.566 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:22.566 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:22.566 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13631 2024-09-21T21:42:22.566 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.566 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:22.566 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.566 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:22.566 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943362+0000", 2024-09-21T21:42:22.566 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.566 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.566 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.566 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:22.566 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943362+0000", 2024-09-21T21:42:22.566 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.567 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.567 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.567 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:22.567 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943362+0000", 2024-09-21T21:42:22.567 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8799999999999998e-07 2024-09-21T21:42:22.567 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.567 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.567 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:22.567 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943364+0000", 2024-09-21T21:42:22.567 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9850000000000001e-06 2024-09-21T21:42:22.567 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.567 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.567 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:22.568 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943371+0000", 2024-09-21T21:42:22.568 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0650000000000001e-06 2024-09-21T21:42:22.568 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.568 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.568 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:22.568 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943376+0000", 2024-09-21T21:42:22.568 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8250000000000004e-06 2024-09-21T21:42:22.568 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.568 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.568 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:22.568 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022329+0000", 2024-09-21T21:42:22.568 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078952888999999998 2024-09-21T21:42:22.568 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.568 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.568 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:22.569 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022419+0000", 2024-09-21T21:42:22.569 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.9738e-05 2024-09-21T21:42:22.569 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.569 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.569 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:22.569 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022558+0000", 2024-09-21T21:42:22.569 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00013875500000000001 2024-09-21T21:42:22.569 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.569 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:22.569 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.569 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.569 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.569 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13632 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7966720~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:22.569 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943382+0000", 2024-09-21T21:42:22.570 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 31.574259604000002, 2024-09-21T21:42:22.570 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079495727000000002, 2024-09-21T21:42:22.570 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:22.570 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:22.570 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:22.570 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:22.570 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:22.570 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:22.570 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13632 2024-09-21T21:42:22.570 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.570 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:22.570 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.570 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:22.570 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943382+0000", 2024-09-21T21:42:22.570 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.571 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.571 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.571 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:22.571 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943382+0000", 2024-09-21T21:42:22.571 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.571 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.571 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.571 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:22.571 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943382+0000", 2024-09-21T21:42:22.571 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7e-07 2024-09-21T21:42:22.571 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.571 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.571 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:22.571 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943385+0000", 2024-09-21T21:42:22.572 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2230000000000001e-06 2024-09-21T21:42:22.572 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.572 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.572 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:22.572 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943391+0000", 2024-09-21T21:42:22.572 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6810000000000001e-06 2024-09-21T21:42:22.572 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.572 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.572 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:22.572 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943397+0000", 2024-09-21T21:42:22.572 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6559999999999997e-06 2024-09-21T21:42:22.572 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.572 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.572 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:22.572 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022601+0000", 2024-09-21T21:42:22.573 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079204051999999997 2024-09-21T21:42:22.573 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.573 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.573 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:22.573 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022704+0000", 2024-09-21T21:42:22.573 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010329599999999999 2024-09-21T21:42:22.573 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.573 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.573 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:22.573 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022878+0000", 2024-09-21T21:42:22.573 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000173449 2024-09-21T21:42:22.573 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.573 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:22.573 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.573 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.574 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.574 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13633 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7938048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:22.574 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943403+0000", 2024-09-21T21:42:22.574 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 31.574238674, 2024-09-21T21:42:22.574 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079794697999999997, 2024-09-21T21:42:22.574 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:22.574 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:22.574 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:22.574 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:22.574 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:22.574 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:22.574 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13633 2024-09-21T21:42:22.574 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.574 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:22.575 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.575 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:22.575 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943403+0000", 2024-09-21T21:42:22.575 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.575 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.575 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.575 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:22.575 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943403+0000", 2024-09-21T21:42:22.575 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.575 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.575 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.575 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:22.575 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943403+0000", 2024-09-21T21:42:22.575 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.65e-07 2024-09-21T21:42:22.575 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.576 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.576 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:22.576 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943405+0000", 2024-09-21T21:42:22.576 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0020000000000001e-06 2024-09-21T21:42:22.576 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.576 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.576 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:22.576 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943412+0000", 2024-09-21T21:42:22.576 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.8379999999999996e-06 2024-09-21T21:42:22.576 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.576 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.576 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:22.576 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943417+0000", 2024-09-21T21:42:22.576 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7650000000000001e-06 2024-09-21T21:42:22.577 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.577 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.577 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:22.577 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022922+0000", 2024-09-21T21:42:22.577 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079504512999999999 2024-09-21T21:42:22.577 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.577 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.577 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:22.577 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023039+0000", 2024-09-21T21:42:22.577 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000117044 2024-09-21T21:42:22.577 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.577 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.577 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:22.577 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023198+0000", 2024-09-21T21:42:22.578 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000159171 2024-09-21T21:42:22.578 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.578 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:22.578 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.578 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.578 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.578 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13634 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7995392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:22.578 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943423+0000", 2024-09-21T21:42:22.578 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 31.574218966, 2024-09-21T21:42:22.578 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.080142358999999996, 2024-09-21T21:42:22.578 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:22.578 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:22.578 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:22.578 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:22.579 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:22.579 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:22.579 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13634 2024-09-21T21:42:22.579 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.579 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:22.579 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.579 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:22.579 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943423+0000", 2024-09-21T21:42:22.579 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.579 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.579 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.579 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:22.579 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943423+0000", 2024-09-21T21:42:22.579 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.580 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.580 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.580 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:22.580 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943423+0000", 2024-09-21T21:42:22.580 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.58e-07 2024-09-21T21:42:22.580 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.580 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.580 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:22.580 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943425+0000", 2024-09-21T21:42:22.580 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9180000000000001e-06 2024-09-21T21:42:22.580 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.580 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.580 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:22.580 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943431+0000", 2024-09-21T21:42:22.580 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4439999999999996e-06 2024-09-21T21:42:22.581 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.581 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.581 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:22.581 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943436+0000", 2024-09-21T21:42:22.581 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.887e-06 2024-09-21T21:42:22.581 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.581 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.581 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:22.581 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023266+0000", 2024-09-21T21:42:22.581 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079829622000000003 2024-09-21T21:42:22.581 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.581 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.581 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:22.581 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023384+0000", 2024-09-21T21:42:22.582 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000117658 2024-09-21T21:42:22.582 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.582 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.582 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:22.582 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023565+0000", 2024-09-21T21:42:22.582 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00018147200000000001 2024-09-21T21:42:22.582 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.582 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:22.582 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.582 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.582 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.582 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13635 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 8024064~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:22.582 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943447+0000", 2024-09-21T21:42:22.582 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 31.574194425999998, 2024-09-21T21:42:22.583 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.080417653000000006, 2024-09-21T21:42:22.583 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:22.583 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:22.583 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:22.583 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:22.583 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:22.583 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:22.583 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13635 2024-09-21T21:42:22.583 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.583 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:22.583 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.583 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:22.583 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943447+0000", 2024-09-21T21:42:22.583 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.584 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.584 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.584 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:22.584 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943447+0000", 2024-09-21T21:42:22.584 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.584 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.584 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.584 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:22.584 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943448+0000", 2024-09-21T21:42:22.584 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1699999999999999e-07 2024-09-21T21:42:22.584 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.584 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.584 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:22.584 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943450+0000", 2024-09-21T21:42:22.584 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1639999999999999e-06 2024-09-21T21:42:22.585 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.585 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.585 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:22.585 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943456+0000", 2024-09-21T21:42:22.585 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8379999999999998e-06 2024-09-21T21:42:22.585 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.585 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.585 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:22.585 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943460+0000", 2024-09-21T21:42:22.585 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4700000000000004e-06 2024-09-21T21:42:22.585 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.585 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.585 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:22.585 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023595+0000", 2024-09-21T21:42:22.585 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.080134869999999997 2024-09-21T21:42:22.586 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.586 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.586 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:22.586 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023688+0000", 2024-09-21T21:42:22.586 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2794000000000006e-05 2024-09-21T21:42:22.586 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.586 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.586 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:22.586 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023865+0000", 2024-09-21T21:42:22.586 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0001771 2024-09-21T21:42:22.586 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.586 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:22.586 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.586 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.586 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.587 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13636 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 8052736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:22.587 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943467+0000", 2024-09-21T21:42:22.587 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 31.574174688999999, 2024-09-21T21:42:22.587 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.080694399999999999, 2024-09-21T21:42:22.587 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:22.587 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:22.587 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:22.587 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:22.587 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:22.587 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:22.587 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13636 2024-09-21T21:42:22.587 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.587 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:22.587 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.588 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:22.588 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943467+0000", 2024-09-21T21:42:22.588 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.588 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.588 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.588 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:22.588 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943467+0000", 2024-09-21T21:42:22.588 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.588 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.588 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.588 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:22.588 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944940+0000", 2024-09-21T21:42:22.588 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0014725109999999999 2024-09-21T21:42:22.588 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.588 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.589 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:22.589 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944949+0000", 2024-09-21T21:42:22.589 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2930000000000006e-06 2024-09-21T21:42:22.589 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.589 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.589 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:22.589 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944958+0000", 2024-09-21T21:42:22.589 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.6899999999999998e-06 2024-09-21T21:42:22.589 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.589 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.589 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:22.589 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944966+0000", 2024-09-21T21:42:22.589 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0690000000000002e-06 2024-09-21T21:42:22.589 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.590 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.590 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:22.590 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023900+0000", 2024-09-21T21:42:22.590 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078934371000000003 2024-09-21T21:42:22.590 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.590 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.590 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:22.590 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024009+0000", 2024-09-21T21:42:22.590 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000108857 2024-09-21T21:42:22.590 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.590 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.590 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:22.590 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024161+0000", 2024-09-21T21:42:22.590 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000152609 2024-09-21T21:42:22.591 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.591 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:22.591 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.591 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.591 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.591 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13637 3.12 3.1dc31b72 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:22.591 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.944978+0000", 2024-09-21T21:42:22.591 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 31.572663754000001, 2024-09-21T21:42:22.591 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079232604999999998, 2024-09-21T21:42:22.591 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:22.591 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:22.591 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:42:22.591 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:22.591 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:22.591 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:22.592 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13637 2024-09-21T21:42:22.592 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.592 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:22.592 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.592 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:22.592 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944978+0000", 2024-09-21T21:42:22.592 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.592 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.592 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.592 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:22.592 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944978+0000", 2024-09-21T21:42:22.592 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.592 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.592 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.593 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:22.593 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944978+0000", 2024-09-21T21:42:22.593 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9100000000000004e-07 2024-09-21T21:42:22.593 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.593 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.593 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:22.593 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944981+0000", 2024-09-21T21:42:22.593 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3290000000000001e-06 2024-09-21T21:42:22.593 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.593 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.593 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:22.593 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944987+0000", 2024-09-21T21:42:22.593 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.28e-06 2024-09-21T21:42:22.593 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.593 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.594 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:22.594 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944992+0000", 2024-09-21T21:42:22.594 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8520000000000003e-06 2024-09-21T21:42:22.594 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.594 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.594 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:22.594 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024196+0000", 2024-09-21T21:42:22.594 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079203673000000002 2024-09-21T21:42:22.594 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.594 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.594 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:22.594 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024211+0000", 2024-09-21T21:42:22.594 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.4980000000000001e-05 2024-09-21T21:42:22.594 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.595 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:22.595 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.595 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.595 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.595 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13638 3.12 3.1dc31b72 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:22.595 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.945000+0000", 2024-09-21T21:42:22.595 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 31.572641625999999, 2024-09-21T21:42:22.595 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079257465999999999, 2024-09-21T21:42:22.595 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:22.595 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:22.595 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:42:22.595 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:22.595 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:22.595 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:22.596 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13638 2024-09-21T21:42:22.596 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.596 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:22.596 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.596 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:22.596 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945000+0000", 2024-09-21T21:42:22.596 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.596 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.596 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.596 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:22.596 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945000+0000", 2024-09-21T21:42:22.596 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.596 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.596 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.597 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:22.597 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945001+0000", 2024-09-21T21:42:22.597 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.32e-07 2024-09-21T21:42:22.597 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.597 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.597 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:22.597 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945003+0000", 2024-09-21T21:42:22.597 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2309999999999999e-06 2024-09-21T21:42:22.597 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.597 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.597 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:22.597 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945009+0000", 2024-09-21T21:42:22.597 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7450000000000001e-06 2024-09-21T21:42:22.597 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.597 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.598 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:22.598 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945013+0000", 2024-09-21T21:42:22.598 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.673e-06 2024-09-21T21:42:22.598 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.598 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.598 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:22.598 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024245+0000", 2024-09-21T21:42:22.598 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079232160999999995 2024-09-21T21:42:22.598 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.598 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.598 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:22.598 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024258+0000", 2024-09-21T21:42:22.598 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.2224e-05 2024-09-21T21:42:22.598 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.599 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:22.599 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.599 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.599 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.599 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13639 3.12 3.1dc31b72 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:22.599 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.945034+0000", 2024-09-21T21:42:22.599 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 31.572607826999999, 2024-09-21T21:42:22.599 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079260627, 2024-09-21T21:42:22.599 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:22.599 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:22.599 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:42:22.599 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:22.599 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:22.599 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:22.599 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13639 2024-09-21T21:42:22.600 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.600 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:22.600 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.600 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:22.600 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945034+0000", 2024-09-21T21:42:22.600 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.600 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.600 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.600 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:22.600 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945034+0000", 2024-09-21T21:42:22.600 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.600 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.600 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.600 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:22.600 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945034+0000", 2024-09-21T21:42:22.601 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3799999999999998e-07 2024-09-21T21:42:22.601 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.601 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.601 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:22.601 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945036+0000", 2024-09-21T21:42:22.601 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.052e-06 2024-09-21T21:42:22.601 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.601 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.601 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:22.601 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945040+0000", 2024-09-21T21:42:22.601 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1169999999999997e-06 2024-09-21T21:42:22.601 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.601 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.601 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:22.601 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945043+0000", 2024-09-21T21:42:22.602 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.8439999999999998e-06 2024-09-21T21:42:22.602 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.602 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.602 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:22.602 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024286+0000", 2024-09-21T21:42:22.602 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079242483000000002 2024-09-21T21:42:22.602 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.602 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.602 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:22.602 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024295+0000", 2024-09-21T21:42:22.602 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.7930000000000003e-06 2024-09-21T21:42:22.602 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.602 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:22.602 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.603 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.603 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.603 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13640 3.12 3.1dc31b72 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:22.603 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.945047+0000", 2024-09-21T21:42:22.603 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 31.572594304999999, 2024-09-21T21:42:22.603 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079412321999999994, 2024-09-21T21:42:22.603 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:22.603 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:22.603 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:42:22.603 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:22.603 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:22.603 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:22.603 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13640 2024-09-21T21:42:22.603 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.604 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:22.604 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.604 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:22.604 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945047+0000", 2024-09-21T21:42:22.604 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.604 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.604 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.604 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:22.604 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945047+0000", 2024-09-21T21:42:22.604 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.604 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.604 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.604 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:22.604 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945048+0000", 2024-09-21T21:42:22.604 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0499999999999999e-07 2024-09-21T21:42:22.605 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.605 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.605 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:22.605 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945049+0000", 2024-09-21T21:42:22.605 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.3939999999999999e-06 2024-09-21T21:42:22.605 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.605 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.605 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:22.605 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945053+0000", 2024-09-21T21:42:22.605 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1760000000000003e-06 2024-09-21T21:42:22.605 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.605 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.605 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:22.605 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945056+0000", 2024-09-21T21:42:22.606 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.6129999999999998e-06 2024-09-21T21:42:22.606 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.606 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.606 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:22.606 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024360+0000", 2024-09-21T21:42:22.606 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079303645000000006 2024-09-21T21:42:22.606 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.606 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.606 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:22.606 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024460+0000", 2024-09-21T21:42:22.606 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000100189 2024-09-21T21:42:22.606 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.606 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:22.606 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.607 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.607 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:22.607 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:22.609 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T21:42:22.609 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:22.606+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:22.611 INFO:teuthology.orchestra.run.smithi164.stdout:5205 trunc from 0x334200 to 0x3fcfa00 2024-09-21T21:42:22.611 INFO:teuthology.orchestra.run.smithi164.stdout:5208 trunc from 0x3fcfa00 to 0x554b400 2024-09-21T21:42:22.611 INFO:teuthology.orchestra.run.smithi164.stdout:5210 write 0xb58a200 thru 0xb58bb91 (0x1992 bytes) 2024-09-21T21:42:22.611 INFO:teuthology.orchestra.run.smithi164.stdout:5212 punch from 0x8cba000 to 0x8cc9800, (0xf800 bytes) 2024-09-21T21:42:22.611 INFO:teuthology.orchestra.run.smithi164.stdout:5214 punch from 0x53f4800 to 0x5400e00, (0xc600 bytes) 2024-09-21T21:42:22.611 INFO:teuthology.orchestra.run.smithi164.stdout:5215 read 0x97f0a00 thru 0x97ffd50 (0xf351 bytes) 2024-09-21T21:42:22.611 INFO:teuthology.orchestra.run.smithi164.stdout:5216 punch from 0xac31400 to 0xac38800, (0x7400 bytes) 2024-09-21T21:42:22.611 INFO:teuthology.orchestra.run.smithi164.stdout:5217 punch from 0x722a600 to 0x7238800, (0xe200 bytes) 2024-09-21T21:42:22.611 INFO:teuthology.orchestra.run.smithi164.stdout:5218 write 0x92f2600 thru 0x92fc930 (0xa331 bytes) 2024-09-21T21:42:22.611 INFO:teuthology.orchestra.run.smithi164.stdout:5219 punch from 0x4432200 to 0x4434200, (0x2000 bytes) 2024-09-21T21:42:22.611 INFO:teuthology.orchestra.run.smithi164.stdout:5223 read 0x7fe2c00 thru 0x7fefbbc (0xcfbd bytes) 2024-09-21T21:42:22.612 INFO:teuthology.orchestra.run.smithi164.stdout:5225 read 0x7984200 thru 0x798d0e2 (0x8ee3 bytes) 2024-09-21T21:42:22.612 INFO:teuthology.orchestra.run.smithi164.stdout:5228 read 0x47f8000 thru 0x47f833d (0x33e bytes) 2024-09-21T21:42:22.612 INFO:teuthology.orchestra.run.smithi164.stdout:5229 read 0x1f9f200 thru 0x1fa32d0 (0x40d1 bytes) 2024-09-21T21:42:22.612 INFO:teuthology.orchestra.run.smithi164.stdout:5230 write 0x74bc00 thru 0x755629 (0x9a2a bytes) 2024-09-21T21:42:22.612 INFO:teuthology.orchestra.run.smithi164.stdout:5232 read 0x7d54400 thru 0x7d626e0 (0xe2e1 bytes) 2024-09-21T21:42:22.612 INFO:teuthology.orchestra.run.smithi164.stdout:5234 read 0xb48800 thru 0xb48b29 (0x32a bytes) 2024-09-21T21:42:22.612 INFO:teuthology.orchestra.run.smithi164.stdout:5236 punch from 0x6524200 to 0x652c200, (0x8000 bytes) 2024-09-21T21:42:22.612 INFO:teuthology.orchestra.run.smithi164.stdout:5237 read 0x2e87600 thru 0x2e9433f (0xcd40 bytes) 2024-09-21T21:42:22.612 INFO:teuthology.orchestra.run.smithi164.stdout:5238 punch from 0xa3eb400 to 0xa3ef200, (0x3e00 bytes) 2024-09-21T21:42:22.612 INFO:teuthology.orchestra.run.smithi164.stdout:5240 punch from 0x5b31800 to 0x5b36600, (0x4e00 bytes) 2024-09-21T21:42:22.612 INFO:teuthology.orchestra.run.smithi164.stdout:5242 punch from 0x8a70000 to 0x8a7d000, (0xd000 bytes) 2024-09-21T21:42:22.612 INFO:teuthology.orchestra.run.smithi164.stdout:5243 read 0x2b91000 thru 0x2b95830 (0x4831 bytes) 2024-09-21T21:42:22.612 INFO:teuthology.orchestra.run.smithi164.stdout:5244 write 0x2706a00 thru 0x2713253 (0xc854 bytes) 2024-09-21T21:42:22.612 INFO:teuthology.orchestra.run.smithi164.stdout:5245 read 0x8da600 thru 0x8e9a20 (0xf421 bytes) 2024-09-21T21:42:22.708 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:22.707+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:22.711 INFO:teuthology.orchestra.run.smithi164.stdout:5246 write 0xca3f400 thru 0xca49222 (0x9e23 bytes) 2024-09-21T21:42:22.720 INFO:teuthology.orchestra.run.smithi164.stdout:5247 trunc from 0xca49400 to 0x543fe00 2024-09-21T21:42:22.784 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:22.784 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:42:22.784 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.784 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5716.0:25391 3.6 3.9f7f74e6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:22.784 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:22.781426+0000", 2024-09-21T21:42:22.784 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 4.2531000000000003e-05, 2024-09-21T21:42:22.784 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0464999999999998e-05, 2024-09-21T21:42:22.785 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:22.785 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:22.785 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:22.785 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:22.785 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5716", 2024-09-21T21:42:22.785 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:22.785 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 25391 2024-09-21T21:42:22.785 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.785 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:22.785 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.785 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:22.785 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.781426+0000", 2024-09-21T21:42:22.785 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.785 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.785 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.786 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:22.786 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.781426+0000", 2024-09-21T21:42:22.786 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.786 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.786 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.786 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:22.786 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.781427+0000", 2024-09-21T21:42:22.786 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.1399999999999996e-07 2024-09-21T21:42:22.786 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.786 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.786 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:22.786 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.781431+0000", 2024-09-21T21:42:22.786 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5999999999999998e-06 2024-09-21T21:42:22.786 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.786 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.787 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:22.787 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.781438+0000", 2024-09-21T21:42:22.787 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.3189999999999997e-06 2024-09-21T21:42:22.787 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.787 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.787 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:22.787 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.781443+0000", 2024-09-21T21:42:22.787 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0170000000000004e-06 2024-09-21T21:42:22.787 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.787 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:22.787 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.787 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.787 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.787 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5716.0:25393 3.6 3.9f7f74e6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:22.788 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:22.781457+0000", 2024-09-21T21:42:22.788 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 1.1581e-05, 2024-09-21T21:42:22.788 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.4728000000000006e-05, 2024-09-21T21:42:22.788 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:22.788 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:22.788 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:22.788 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:22.788 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5716", 2024-09-21T21:42:22.788 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:22.788 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 25393 2024-09-21T21:42:22.788 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.788 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:22.788 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.788 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:22.789 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.781457+0000", 2024-09-21T21:42:22.789 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.789 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.789 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.789 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:22.789 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.781457+0000", 2024-09-21T21:42:22.789 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.789 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.789 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.789 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:22.789 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.781458+0000", 2024-09-21T21:42:22.789 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3e-07 2024-09-21T21:42:22.789 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.790 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.790 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:22.790 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.781464+0000", 2024-09-21T21:42:22.790 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2720000000000003e-06 2024-09-21T21:42:22.790 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.790 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.790 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:22.790 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.781470+0000", 2024-09-21T21:42:22.790 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3600000000000001e-06 2024-09-21T21:42:22.790 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.790 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.790 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:22.790 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.781475+0000", 2024-09-21T21:42:22.790 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0080000000000002e-06 2024-09-21T21:42:22.791 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.791 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:22.791 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.791 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.791 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.791 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5716.0:25367 3.7 3:e7fbbd2e:::rbd_data.1121c0cf2471.0000000000000149:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:22.791 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:22.778124+0000", 2024-09-21T21:42:22.791 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.0033444299999999998, 2024-09-21T21:42:22.791 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0034769079999999999, 2024-09-21T21:42:22.791 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:22.791 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:22.791 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "waiting for sub ops", 2024-09-21T21:42:22.791 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:22.791 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5716", 2024-09-21T21:42:22.791 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:22.792 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 25367 2024-09-21T21:42:22.792 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.792 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:22.792 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.792 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:22.792 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.778124+0000", 2024-09-21T21:42:22.792 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.792 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.792 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.792 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:22.792 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.778124+0000", 2024-09-21T21:42:22.792 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.792 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.792 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.793 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:22.793 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.778125+0000", 2024-09-21T21:42:22.793 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3300000000000002e-07 2024-09-21T21:42:22.793 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.793 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.793 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:22.793 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.778127+0000", 2024-09-21T21:42:22.793 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.6560000000000001e-06 2024-09-21T21:42:22.793 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.793 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.793 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:22.793 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.778132+0000", 2024-09-21T21:42:22.793 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.848e-06 2024-09-21T21:42:22.793 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.793 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.794 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:22.794 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.778136+0000", 2024-09-21T21:42:22.794 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.783e-06 2024-09-21T21:42:22.794 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.794 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.794 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:22.794 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.778175+0000", 2024-09-21T21:42:22.794 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.9264999999999998e-05 2024-09-21T21:42:22.794 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.794 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.794 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:22.794 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.778245+0000", 2024-09-21T21:42:22.794 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9834999999999998e-05 2024-09-21T21:42:22.794 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.794 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.795 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 2", 2024-09-21T21:42:22.795 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.778351+0000", 2024-09-21T21:42:22.795 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000106085 2024-09-21T21:42:22.795 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.795 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:22.795 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.795 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.795 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.795 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5716.0:25369 3.0 3:02afe9b6:::rbd_data.1121c0cf2471.0000000000000154:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:22.795 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:22.778178+0000", 2024-09-21T21:42:22.795 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.003290355, 2024-09-21T21:42:22.795 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.003492579, 2024-09-21T21:42:22.795 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:22.795 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:22.795 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "waiting for sub ops", 2024-09-21T21:42:22.796 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:22.796 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5716", 2024-09-21T21:42:22.796 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:22.796 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 25369 2024-09-21T21:42:22.796 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.796 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:22.796 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.796 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:22.796 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.778178+0000", 2024-09-21T21:42:22.796 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.796 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.796 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.796 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:22.796 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.778178+0000", 2024-09-21T21:42:22.796 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.797 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.797 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.797 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:22.797 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.778179+0000", 2024-09-21T21:42:22.797 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1099999999999996e-07 2024-09-21T21:42:22.797 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.797 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.797 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:22.797 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.778182+0000", 2024-09-21T21:42:22.797 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.7360000000000001e-06 2024-09-21T21:42:22.797 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.797 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.797 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:22.797 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.778188+0000", 2024-09-21T21:42:22.798 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2750000000000001e-06 2024-09-21T21:42:22.798 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.798 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.798 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:22.798 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.778193+0000", 2024-09-21T21:42:22.798 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8040000000000001e-06 2024-09-21T21:42:22.798 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.798 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.798 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:22.798 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.778235+0000", 2024-09-21T21:42:22.798 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2734000000000001e-05 2024-09-21T21:42:22.798 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.798 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.798 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:22.799 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.778302+0000", 2024-09-21T21:42:22.799 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6458000000000006e-05 2024-09-21T21:42:22.799 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.799 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.799 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 2", 2024-09-21T21:42:22.799 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.778390+0000", 2024-09-21T21:42:22.799 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.7832999999999997e-05 2024-09-21T21:42:22.799 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.799 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:22.799 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.799 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.799 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.799 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5716.0:25371 3.6 3.9f7f74e6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:22.799 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:22.778796+0000", 2024-09-21T21:42:22.799 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.0026730740000000001, 2024-09-21T21:42:22.800 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0029383690000000001, 2024-09-21T21:42:22.800 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:22.800 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:22.800 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:42:22.800 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:22.800 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5716", 2024-09-21T21:42:22.800 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:22.800 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 25371 2024-09-21T21:42:22.800 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.800 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:22.800 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.800 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:22.800 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.778796+0000", 2024-09-21T21:42:22.800 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.800 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.801 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.801 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:22.801 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.778796+0000", 2024-09-21T21:42:22.801 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.801 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.801 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.801 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:22.801 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.778796+0000", 2024-09-21T21:42:22.801 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.09e-07 2024-09-21T21:42:22.801 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.801 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.801 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:22.801 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.778800+0000", 2024-09-21T21:42:22.801 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4120000000000001e-06 2024-09-21T21:42:22.801 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.802 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.802 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:22.802 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.778808+0000", 2024-09-21T21:42:22.802 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.9589999999999995e-06 2024-09-21T21:42:22.802 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.802 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.802 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:22.802 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.778813+0000", 2024-09-21T21:42:22.802 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6099999999999997e-06 2024-09-21T21:42:22.802 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.802 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.802 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:22.802 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.781722+0000", 2024-09-21T21:42:22.803 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0029083389999999998 2024-09-21T21:42:22.803 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.803 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:22.803 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.803 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.803 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.803 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5716.0:25373 3.6 3.9f7f74e6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:22.803 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:22.778822+0000", 2024-09-21T21:42:22.803 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.002646256, 2024-09-21T21:42:22.803 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0029745930000000002, 2024-09-21T21:42:22.803 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:22.804 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:22.804 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:22.804 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:22.804 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5716", 2024-09-21T21:42:22.804 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:22.804 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 25373 2024-09-21T21:42:22.804 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.804 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:22.804 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.804 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:22.804 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.778822+0000", 2024-09-21T21:42:22.804 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.804 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.804 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.805 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:22.805 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.778822+0000", 2024-09-21T21:42:22.805 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.805 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.805 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.805 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:22.805 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.778823+0000", 2024-09-21T21:42:22.805 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5200000000000002e-07 2024-09-21T21:42:22.805 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.805 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.805 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:22.805 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.778826+0000", 2024-09-21T21:42:22.805 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.7049999999999999e-06 2024-09-21T21:42:22.805 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.805 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.806 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:22.806 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.778832+0000", 2024-09-21T21:42:22.806 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.708e-06 2024-09-21T21:42:22.806 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.806 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.806 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:22.806 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.778838+0000", 2024-09-21T21:42:22.806 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1780000000000002e-06 2024-09-21T21:42:22.806 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.806 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:22.806 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.806 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.806 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.806 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5716.0:25375 3.6 3.9f7f74e6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:22.807 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:22.778846+0000", 2024-09-21T21:42:22.807 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.0026223650000000002, 2024-09-21T21:42:22.807 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0030073679999999998, 2024-09-21T21:42:22.807 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:22.807 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:22.807 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:22.807 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:22.807 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5716", 2024-09-21T21:42:22.807 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:22.807 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 25375 2024-09-21T21:42:22.807 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.807 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:22.807 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.807 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:22.808 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.778846+0000", 2024-09-21T21:42:22.808 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.808 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.808 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.808 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:22.808 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.778846+0000", 2024-09-21T21:42:22.808 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.808 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.808 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.808 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:22.808 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.778847+0000", 2024-09-21T21:42:22.808 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7899999999999999e-07 2024-09-21T21:42:22.808 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.808 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.808 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:22.809 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.778850+0000", 2024-09-21T21:42:22.809 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.835e-06 2024-09-21T21:42:22.809 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.809 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.809 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:22.809 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.778856+0000", 2024-09-21T21:42:22.809 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3380000000000001e-06 2024-09-21T21:42:22.809 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.809 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.809 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:22.809 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.778861+0000", 2024-09-21T21:42:22.809 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8010000000000003e-06 2024-09-21T21:42:22.809 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.809 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:22.809 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.810 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.810 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.810 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_repop(client.5716.0:25374 3.1b e105/100 3:de152ad4:::rbd_data.1121c0cf2471.0000000000000156:head v 105'536, mlcod=105'534)", 2024-09-21T21:42:22.810 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:22.778917+0000", 2024-09-21T21:42:22.810 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.0025515310000000001, 2024-09-21T21:42:22.810 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.002987918, 2024-09-21T21:42:22.810 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:22.810 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:22.810 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:22.810 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:22.810 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.810 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:22.810 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.778917+0000", 2024-09-21T21:42:22.810 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.810 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.811 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.811 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:22.811 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.778917+0000", 2024-09-21T21:42:22.811 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.811 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.811 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.811 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:22.811 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.778918+0000", 2024-09-21T21:42:22.811 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.75e-07 2024-09-21T21:42:22.811 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.811 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.811 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:22.811 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.778922+0000", 2024-09-21T21:42:22.811 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4900000000000002e-06 2024-09-21T21:42:22.811 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.812 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.812 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:22.812 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.778931+0000", 2024-09-21T21:42:22.812 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.6510000000000007e-06 2024-09-21T21:42:22.812 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.812 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.812 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:22.812 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.778941+0000", 2024-09-21T21:42:22.812 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.011e-05 2024-09-21T21:42:22.812 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.812 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.812 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:22.812 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.778991+0000", 2024-09-21T21:42:22.812 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9635999999999997e-05 2024-09-21T21:42:22.812 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.813 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.813 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:22.813 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.779009+0000", 2024-09-21T21:42:22.813 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.8121000000000001e-05 2024-09-21T21:42:22.813 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.813 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:22.813 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.813 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.813 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.813 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4385.0:21775 3.6 3.9f7f74e6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:22.813 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:22.779002+0000", 2024-09-21T21:42:22.813 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.0024670529999999999, 2024-09-21T21:42:22.813 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0029801979999999999, 2024-09-21T21:42:22.813 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:22.813 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:22.814 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:22.814 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:22.814 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4385", 2024-09-21T21:42:22.814 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:42:22.814 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 21775 2024-09-21T21:42:22.814 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.814 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:22.814 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.814 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:22.814 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.779002+0000", 2024-09-21T21:42:22.814 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.814 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.814 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.815 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:22.815 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.779002+0000", 2024-09-21T21:42:22.815 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.815 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.815 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.815 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:22.815 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.779003+0000", 2024-09-21T21:42:22.815 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.8599999999999997e-07 2024-09-21T21:42:22.815 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.815 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.815 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:22.815 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.779007+0000", 2024-09-21T21:42:22.815 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5719999999999996e-06 2024-09-21T21:42:22.815 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.815 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.816 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:22.816 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.779041+0000", 2024-09-21T21:42:22.816 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3558e-05 2024-09-21T21:42:22.816 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.816 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.816 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:22.816 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.779050+0000", 2024-09-21T21:42:22.816 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.85e-06 2024-09-21T21:42:22.816 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.816 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:22.816 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.816 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.816 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.816 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4385.0:21776 3.6 3.9f7f74e6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:22.816 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:22.779288+0000", 2024-09-21T21:42:22.817 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.0021807879999999999, 2024-09-21T21:42:22.817 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0027450460000000001, 2024-09-21T21:42:22.817 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:22.817 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:22.817 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:22.817 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:22.817 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4385", 2024-09-21T21:42:22.817 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:42:22.817 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 21776 2024-09-21T21:42:22.817 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.817 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:22.817 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.817 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:22.817 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.779288+0000", 2024-09-21T21:42:22.817 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.818 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.818 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.818 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:22.818 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.779288+0000", 2024-09-21T21:42:22.818 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.818 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.818 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.818 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:22.818 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.779289+0000", 2024-09-21T21:42:22.818 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.4900000000000005e-07 2024-09-21T21:42:22.818 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.818 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.818 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:22.818 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.779294+0000", 2024-09-21T21:42:22.819 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7469999999999996e-06 2024-09-21T21:42:22.819 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.819 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.819 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:22.819 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.779303+0000", 2024-09-21T21:42:22.819 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.5920000000000007e-06 2024-09-21T21:42:22.819 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.819 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.819 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:22.819 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.779310+0000", 2024-09-21T21:42:22.819 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.8009999999999998e-06 2024-09-21T21:42:22.819 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.819 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:22.819 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.819 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.820 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.820 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_repop(client.5716.0:25372 3.3 e105/99 3:c6621026:::rbd_data.1121c0cf2471.0000000000000155:head v 105'342, mlcod=105'340)", 2024-09-21T21:42:22.820 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:22.779296+0000", 2024-09-21T21:42:22.820 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.0021730790000000001, 2024-09-21T21:42:22.820 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0027905510000000001, 2024-09-21T21:42:22.820 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:22.820 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:22.820 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:22.820 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:22.820 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.820 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:22.820 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.779296+0000", 2024-09-21T21:42:22.820 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.820 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.820 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.821 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:22.821 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.779296+0000", 2024-09-21T21:42:22.821 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.821 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.821 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.821 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:22.821 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.779296+0000", 2024-09-21T21:42:22.821 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.46e-07 2024-09-21T21:42:22.821 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.821 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.821 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:22.821 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.779301+0000", 2024-09-21T21:42:22.821 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4610000000000001e-06 2024-09-21T21:42:22.821 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.821 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.822 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:22.822 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.779309+0000", 2024-09-21T21:42:22.822 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.3620000000000006e-06 2024-09-21T21:42:22.822 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.822 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.822 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:22.822 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.779315+0000", 2024-09-21T21:42:22.822 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0820000000000004e-06 2024-09-21T21:42:22.822 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.822 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.822 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:22.822 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.779360+0000", 2024-09-21T21:42:22.822 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4728000000000003e-05 2024-09-21T21:42:22.822 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.822 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.823 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:22.823 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.779379+0000", 2024-09-21T21:42:22.823 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9497999999999999e-05 2024-09-21T21:42:22.823 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.823 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:22.823 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.823 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.823 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.823 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5716.0:25376 3.6 3.9f7f74e6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:22.823 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:22.779626+0000", 2024-09-21T21:42:22.823 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.0018422740000000001, 2024-09-21T21:42:22.823 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0025009139999999999, 2024-09-21T21:42:22.823 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:22.823 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:22.824 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:22.824 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:22.824 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5716", 2024-09-21T21:42:22.824 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:22.824 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 25376 2024-09-21T21:42:22.824 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.824 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:22.824 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.824 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:22.824 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.779626+0000", 2024-09-21T21:42:22.824 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.824 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.824 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.824 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:22.824 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.779626+0000", 2024-09-21T21:42:22.824 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.825 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.825 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.825 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:22.825 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.779627+0000", 2024-09-21T21:42:22.825 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.1900000000000001e-07 2024-09-21T21:42:22.825 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.825 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.825 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:22.825 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.779631+0000", 2024-09-21T21:42:22.825 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.0470000000000004e-06 2024-09-21T21:42:22.825 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.825 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.825 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:22.825 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.779642+0000", 2024-09-21T21:42:22.826 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0397999999999999e-05 2024-09-21T21:42:22.826 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.826 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.826 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:22.826 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.779650+0000", 2024-09-21T21:42:22.826 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.4230000000000005e-06 2024-09-21T21:42:22.826 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.826 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:22.826 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.826 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.826 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.826 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5716.0:25377 3.6 3.9f7f74e6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:22.826 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:22.779661+0000", 2024-09-21T21:42:22.826 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.0018073080000000001, 2024-09-21T21:42:22.826 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0024938500000000001, 2024-09-21T21:42:22.827 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:22.827 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:22.827 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:22.827 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:22.827 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5716", 2024-09-21T21:42:22.827 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:22.827 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 25377 2024-09-21T21:42:22.827 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.827 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:22.827 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.827 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:22.827 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.779661+0000", 2024-09-21T21:42:22.827 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.827 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.827 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.828 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:22.828 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.779661+0000", 2024-09-21T21:42:22.828 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.828 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.828 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.828 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:22.828 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.779662+0000", 2024-09-21T21:42:22.828 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1199999999999998e-07 2024-09-21T21:42:22.828 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.828 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.828 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:22.828 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.779665+0000", 2024-09-21T21:42:22.828 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.897e-06 2024-09-21T21:42:22.828 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.829 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.829 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:22.829 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.779672+0000", 2024-09-21T21:42:22.829 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0840000000000003e-06 2024-09-21T21:42:22.829 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.829 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.829 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:22.829 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.779677+0000", 2024-09-21T21:42:22.829 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0799999999999996e-06 2024-09-21T21:42:22.829 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.829 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:22.829 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.829 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.829 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.829 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5716.0:25379 3.6 3.9f7f74e6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:22.830 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:22.779685+0000", 2024-09-21T21:42:22.830 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.0017841210000000001, 2024-09-21T21:42:22.830 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0024971619999999998, 2024-09-21T21:42:22.830 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:22.830 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:22.830 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:22.830 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:22.830 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5716", 2024-09-21T21:42:22.830 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:22.830 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 25379 2024-09-21T21:42:22.830 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.830 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:22.830 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.830 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:22.830 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.779685+0000", 2024-09-21T21:42:22.831 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.831 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.831 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.831 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:22.831 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.779685+0000", 2024-09-21T21:42:22.831 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.831 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.831 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.831 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:22.831 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.779685+0000", 2024-09-21T21:42:22.831 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6100000000000001e-07 2024-09-21T21:42:22.831 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.831 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.831 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:22.831 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.779688+0000", 2024-09-21T21:42:22.832 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.8009999999999999e-06 2024-09-21T21:42:22.832 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.832 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.832 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:22.832 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.779694+0000", 2024-09-21T21:42:22.832 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.455e-06 2024-09-21T21:42:22.832 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.832 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.832 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:22.832 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.779699+0000", 2024-09-21T21:42:22.832 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8450000000000002e-06 2024-09-21T21:42:22.832 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.832 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:22.832 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.832 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.833 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.833 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5716.0:25380 3.6 3.9f7f74e6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:22.833 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:22.779706+0000", 2024-09-21T21:42:22.833 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.0017629709999999999, 2024-09-21T21:42:22.833 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0025023110000000001, 2024-09-21T21:42:22.833 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:22.833 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:22.833 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:22.833 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:22.833 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5716", 2024-09-21T21:42:22.833 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:22.833 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 25380 2024-09-21T21:42:22.833 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.833 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:22.833 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.834 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:22.834 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.779706+0000", 2024-09-21T21:42:22.834 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.834 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.834 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.834 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:22.834 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.779706+0000", 2024-09-21T21:42:22.834 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.834 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.834 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.834 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:22.834 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.779706+0000", 2024-09-21T21:42:22.834 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.9499999999999998e-07 2024-09-21T21:42:22.834 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.834 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.835 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:22.835 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.779709+0000", 2024-09-21T21:42:22.835 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.7310000000000002e-06 2024-09-21T21:42:22.835 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.835 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.835 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:22.835 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.779715+0000", 2024-09-21T21:42:22.835 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5400000000000001e-06 2024-09-21T21:42:22.835 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.835 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.835 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:22.835 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.779720+0000", 2024-09-21T21:42:22.835 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8679999999999998e-06 2024-09-21T21:42:22.835 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.835 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:22.836 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.836 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.836 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.836 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5716.0:25381 3.1d 3:bc5f4586:::rbd_data.1121c0cf2471.0000000000000158:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:22.836 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:22.779759+0000", 2024-09-21T21:42:22.836 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.001709236, 2024-09-21T21:42:22.836 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0024768329999999999, 2024-09-21T21:42:22.836 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:22.836 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:22.836 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "waiting for sub ops", 2024-09-21T21:42:22.836 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:22.836 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5716", 2024-09-21T21:42:22.836 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:22.836 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 25381 2024-09-21T21:42:22.836 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.837 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:22.837 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.837 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:22.837 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.779759+0000", 2024-09-21T21:42:22.837 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.837 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.837 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.837 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:22.837 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.779759+0000", 2024-09-21T21:42:22.837 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.837 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.837 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.837 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:22.837 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.779760+0000", 2024-09-21T21:42:22.837 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7400000000000002e-07 2024-09-21T21:42:22.838 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.838 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.838 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:22.838 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.779763+0000", 2024-09-21T21:42:22.838 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.1149999999999998e-06 2024-09-21T21:42:22.838 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.838 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.838 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:22.838 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.779774+0000", 2024-09-21T21:42:22.838 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0581000000000001e-05 2024-09-21T21:42:22.838 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.838 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.838 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:22.838 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.779780+0000", 2024-09-21T21:42:22.839 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6230000000000002e-06 2024-09-21T21:42:22.839 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.839 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.839 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:22.839 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.779817+0000", 2024-09-21T21:42:22.839 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7061999999999999e-05 2024-09-21T21:42:22.839 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.839 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.839 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:22.839 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.779855+0000", 2024-09-21T21:42:22.839 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8343000000000002e-05 2024-09-21T21:42:22.839 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.839 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.839 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:22.839 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.779933+0000", 2024-09-21T21:42:22.840 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.8172999999999998e-05 2024-09-21T21:42:22.840 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.840 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:22.840 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.840 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.840 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.840 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5716.0:25382 3.6 3.9f7f74e6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:22.840 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:22.779793+0000", 2024-09-21T21:42:22.840 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.0016755100000000001, 2024-09-21T21:42:22.840 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.002482808, 2024-09-21T21:42:22.840 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:22.840 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:22.840 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:22.840 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:22.840 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5716", 2024-09-21T21:42:22.841 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:22.841 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 25382 2024-09-21T21:42:22.841 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.841 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:22.841 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.841 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:22.841 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.779793+0000", 2024-09-21T21:42:22.841 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.841 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.841 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.841 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:22.841 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.779793+0000", 2024-09-21T21:42:22.841 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.841 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.841 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.842 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:22.842 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.779794+0000", 2024-09-21T21:42:22.842 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3300000000000003e-07 2024-09-21T21:42:22.842 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.842 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.842 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:22.842 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.779796+0000", 2024-09-21T21:42:22.842 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.7970000000000001e-06 2024-09-21T21:42:22.842 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.842 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.842 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:22.842 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.779805+0000", 2024-09-21T21:42:22.842 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.1200000000000002e-06 2024-09-21T21:42:22.842 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.843 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.843 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:22.843 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.779810+0000", 2024-09-21T21:42:22.843 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8130000000000001e-06 2024-09-21T21:42:22.843 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.843 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:22.843 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.843 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.843 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.843 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5716.0:25383 3.6 3.9f7f74e6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:22.843 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:22.779818+0000", 2024-09-21T21:42:22.843 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.0016509319999999999, 2024-09-21T21:42:22.843 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0024905610000000001, 2024-09-21T21:42:22.843 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:22.843 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:22.844 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:22.844 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:22.844 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5716", 2024-09-21T21:42:22.844 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:22.844 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 25383 2024-09-21T21:42:22.844 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.844 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:22.844 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.844 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:22.844 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.779818+0000", 2024-09-21T21:42:22.844 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.844 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.844 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.844 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:22.844 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.779818+0000", 2024-09-21T21:42:22.845 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.845 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.845 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.845 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:22.845 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.779818+0000", 2024-09-21T21:42:22.845 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6499999999999999e-07 2024-09-21T21:42:22.845 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.845 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.845 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:22.845 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.779821+0000", 2024-09-21T21:42:22.845 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.7549999999999999e-06 2024-09-21T21:42:22.845 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.845 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.846 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:22.846 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.779827+0000", 2024-09-21T21:42:22.846 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3899999999999998e-06 2024-09-21T21:42:22.846 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.846 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.846 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:22.846 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.779833+0000", 2024-09-21T21:42:22.846 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3029999999999999e-06 2024-09-21T21:42:22.846 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.846 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:22.846 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.846 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.846 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.847 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5716.0:25384 3.6 3.9f7f74e6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:22.847 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:22.779840+0000", 2024-09-21T21:42:22.847 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.001629134, 2024-09-21T21:42:22.847 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0024956470000000001, 2024-09-21T21:42:22.847 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:22.847 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:22.847 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:22.847 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:22.847 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5716", 2024-09-21T21:42:22.847 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:22.847 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 25384 2024-09-21T21:42:22.847 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.847 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:22.847 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.848 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:22.848 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.779840+0000", 2024-09-21T21:42:22.848 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.848 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.848 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.848 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:22.848 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.779840+0000", 2024-09-21T21:42:22.848 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.848 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.848 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.848 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:22.848 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.779840+0000", 2024-09-21T21:42:22.848 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7300000000000001e-07 2024-09-21T21:42:22.848 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.848 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.849 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:22.849 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.779843+0000", 2024-09-21T21:42:22.849 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.5270000000000001e-06 2024-09-21T21:42:22.849 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.849 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.849 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:22.849 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.779850+0000", 2024-09-21T21:42:22.849 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0899999999999999e-06 2024-09-21T21:42:22.849 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.849 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.849 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:22.849 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.779854+0000", 2024-09-21T21:42:22.850 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7809999999999996e-06 2024-09-21T21:42:22.850 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.850 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:22.850 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.850 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.850 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.850 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5716.0:25385 3.4 3:201e046b:::rbd_data.1121c0cf2471.0000000000000159:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:22.850 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:22.780548+0000", 2024-09-21T21:42:22.850 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.00092027600000000004, 2024-09-21T21:42:22.850 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0018170490000000001, 2024-09-21T21:42:22.850 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:22.850 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:22.850 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "waiting for sub ops", 2024-09-21T21:42:22.850 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:22.851 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5716", 2024-09-21T21:42:22.851 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:22.851 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 25385 2024-09-21T21:42:22.851 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.851 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:22.851 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.851 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:22.851 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.780548+0000", 2024-09-21T21:42:22.851 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.851 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.851 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.851 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:22.851 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.780548+0000", 2024-09-21T21:42:22.851 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.852 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.852 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.852 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:22.852 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.780549+0000", 2024-09-21T21:42:22.852 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.5300000000000003e-07 2024-09-21T21:42:22.852 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.852 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.852 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:22.852 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.780554+0000", 2024-09-21T21:42:22.852 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5279999999999997e-06 2024-09-21T21:42:22.852 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.852 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.852 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:22.852 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.780565+0000", 2024-09-21T21:42:22.853 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1282e-05 2024-09-21T21:42:22.853 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.853 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.853 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:22.853 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.780572+0000", 2024-09-21T21:42:22.853 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.2189999999999998e-06 2024-09-21T21:42:22.853 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.853 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.853 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:22.853 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.780612+0000", 2024-09-21T21:42:22.853 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.9298999999999998e-05 2024-09-21T21:42:22.853 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.853 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.854 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:22.854 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.780658+0000", 2024-09-21T21:42:22.854 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6610999999999998e-05 2024-09-21T21:42:22.854 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.854 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.854 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 2", 2024-09-21T21:42:22.854 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.780711+0000", 2024-09-21T21:42:22.854 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2892000000000001e-05 2024-09-21T21:42:22.854 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.854 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:22.854 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.854 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.854 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.854 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5716.0:25386 3.6 3.9f7f74e6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:22.855 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:22.780595+0000", 2024-09-21T21:42:22.855 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.00087330100000000005, 2024-09-21T21:42:22.855 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0018050049999999999, 2024-09-21T21:42:22.855 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:22.855 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:22.855 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:22.855 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:22.855 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5716", 2024-09-21T21:42:22.855 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:22.855 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 25386 2024-09-21T21:42:22.855 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.855 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:22.855 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.855 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:22.855 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.780595+0000", 2024-09-21T21:42:22.856 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.856 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.856 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.856 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:22.856 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.780595+0000", 2024-09-21T21:42:22.856 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.856 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.856 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.856 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:22.856 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.780596+0000", 2024-09-21T21:42:22.856 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8400000000000004e-07 2024-09-21T21:42:22.856 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.856 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.856 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:22.856 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.780599+0000", 2024-09-21T21:42:22.857 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4630000000000001e-06 2024-09-21T21:42:22.857 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.857 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.857 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:22.857 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.780608+0000", 2024-09-21T21:42:22.857 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.4069999999999993e-06 2024-09-21T21:42:22.857 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.857 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.857 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:22.857 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.780613+0000", 2024-09-21T21:42:22.857 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5799999999999999e-06 2024-09-21T21:42:22.857 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.857 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:22.857 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.858 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.858 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.858 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5716.0:25387 3.6 3.9cba2d86 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:22.858 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:22.780624+0000", 2024-09-21T21:42:22.858 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.000845038, 2024-09-21T21:42:22.858 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0018033599999999999, 2024-09-21T21:42:22.858 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:22.858 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:22.858 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:22.858 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:22.858 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5716", 2024-09-21T21:42:22.858 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:22.858 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 25387 2024-09-21T21:42:22.858 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.859 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:22.859 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.859 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:22.859 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.780624+0000", 2024-09-21T21:42:22.859 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.859 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.859 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.859 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:22.859 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.780624+0000", 2024-09-21T21:42:22.859 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.859 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.860 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.860 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:22.860 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.780624+0000", 2024-09-21T21:42:22.860 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8999999999999997e-07 2024-09-21T21:42:22.860 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.860 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.860 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:22.860 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.780627+0000", 2024-09-21T21:42:22.860 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3640000000000002e-06 2024-09-21T21:42:22.860 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.860 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.860 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:22.860 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.780633+0000", 2024-09-21T21:42:22.860 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4320000000000004e-06 2024-09-21T21:42:22.861 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.861 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.861 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:22.861 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.780638+0000", 2024-09-21T21:42:22.861 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8779999999999997e-06 2024-09-21T21:42:22.861 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.861 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:22.861 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.861 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.861 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.861 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5716.0:25388 3.6 3.9f7f74e6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:22.861 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:22.780658+0000", 2024-09-21T21:42:22.861 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.00081050600000000001, 2024-09-21T21:42:22.861 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0017952420000000001, 2024-09-21T21:42:22.862 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:22.862 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:22.862 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:22.862 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:22.862 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5716", 2024-09-21T21:42:22.862 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:22.862 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 25388 2024-09-21T21:42:22.862 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.862 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:22.862 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.862 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:22.862 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.780658+0000", 2024-09-21T21:42:22.862 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.862 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.862 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.863 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:22.863 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.780658+0000", 2024-09-21T21:42:22.863 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.863 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.863 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.863 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:22.863 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.780659+0000", 2024-09-21T21:42:22.863 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8299999999999997e-07 2024-09-21T21:42:22.863 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.863 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.863 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:22.863 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.780662+0000", 2024-09-21T21:42:22.863 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.18e-06 2024-09-21T21:42:22.863 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.863 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.864 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:22.864 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.780670+0000", 2024-09-21T21:42:22.864 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7170000000000008e-06 2024-09-21T21:42:22.864 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.864 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.864 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:22.864 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.780675+0000", 2024-09-21T21:42:22.864 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3789999999999997e-06 2024-09-21T21:42:22.864 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.864 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:22.864 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.864 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.864 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.864 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4385.0:21777 3.6 3.9f7f74e6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:22.864 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:22.781248+0000", 2024-09-21T21:42:22.865 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.00022116200000000001, 2024-09-21T21:42:22.865 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.001232258, 2024-09-21T21:42:22.865 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:22.865 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:22.865 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:22.865 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:22.865 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4385", 2024-09-21T21:42:22.865 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:42:22.865 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 21777 2024-09-21T21:42:22.865 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.865 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:22.865 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.865 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:22.865 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.781248+0000", 2024-09-21T21:42:22.865 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.866 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.866 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.866 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:22.866 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.781248+0000", 2024-09-21T21:42:22.866 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.866 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.866 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.866 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:22.866 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.781249+0000", 2024-09-21T21:42:22.866 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.9300000000000006e-07 2024-09-21T21:42:22.866 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.866 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.866 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:22.866 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.781254+0000", 2024-09-21T21:42:22.866 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6160000000000001e-06 2024-09-21T21:42:22.867 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.867 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.867 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:22.867 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.781262+0000", 2024-09-21T21:42:22.867 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.977e-06 2024-09-21T21:42:22.867 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.867 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.867 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:22.867 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.781269+0000", 2024-09-21T21:42:22.867 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5799999999999997e-06 2024-09-21T21:42:22.867 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.867 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:22.867 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.867 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.867 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.868 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5716.0:25390 3.6 3.9f7f74e6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:22.868 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:22.781338+0000", 2024-09-21T21:42:22.868 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.0001304, 2024-09-21T21:42:22.868 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0011676740000000001, 2024-09-21T21:42:22.868 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:22.868 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:22.868 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:22.868 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:22.868 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5716", 2024-09-21T21:42:22.868 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:22.868 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 25390 2024-09-21T21:42:22.868 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.868 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:22.868 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.869 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:22.869 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.781338+0000", 2024-09-21T21:42:22.869 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.869 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.869 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.869 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:22.869 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.781338+0000", 2024-09-21T21:42:22.869 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:22.869 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.869 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.869 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:22.869 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.781339+0000", 2024-09-21T21:42:22.869 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7400000000000002e-07 2024-09-21T21:42:22.869 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.869 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.870 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:22.870 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.781343+0000", 2024-09-21T21:42:22.870 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3290000000000004e-06 2024-09-21T21:42:22.870 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.870 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.870 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:22.870 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.781353+0000", 2024-09-21T21:42:22.870 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.4730000000000006e-06 2024-09-21T21:42:22.870 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:22.870 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:22.870 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:22.870 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:22.781359+0000", 2024-09-21T21:42:22.870 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5440000000000004e-06 2024-09-21T21:42:22.870 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.871 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:22.871 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.871 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:22.871 INFO:teuthology.orchestra.run.smithi081.stdout: ], 2024-09-21T21:42:22.871 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 25 2024-09-21T21:42:22.871 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:22.872 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:22.870+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:22.873 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T21:42:22.874 INFO:teuthology.orchestra.run.smithi164.stdout:5250 read 0x12bca00 thru 0x12cc4cc (0xfacd bytes) 2024-09-21T21:42:22.874 INFO:teuthology.orchestra.run.smithi164.stdout:5252 read 0x1064400 thru 0x106f5aa (0xb1ab bytes) 2024-09-21T21:42:22.874 INFO:teuthology.orchestra.run.smithi164.stdout:5253 write 0xc1a9400 thru 0xc1b69bb (0xd5bc bytes) 2024-09-21T21:42:22.874 INFO:teuthology.orchestra.run.smithi164.stdout:5254 read 0xa326800 thru 0xa32852f (0x1d30 bytes) 2024-09-21T21:42:22.874 INFO:teuthology.orchestra.run.smithi164.stdout:5255 write 0xbca4000 thru 0xbcab1f0 (0x71f1 bytes) 2024-09-21T21:42:22.874 INFO:teuthology.orchestra.run.smithi164.stdout:5258 read 0x87c3400 thru 0x87d19ea (0xe5eb bytes) 2024-09-21T21:42:22.874 INFO:teuthology.orchestra.run.smithi164.stdout:5259 punch from 0xc019400 to 0xc01be00, (0x2a00 bytes) 2024-09-21T21:42:22.874 INFO:teuthology.orchestra.run.smithi164.stdout:5261 read 0xac0d200 thru 0xac18b2d (0xb92e bytes) 2024-09-21T21:42:22.874 INFO:teuthology.orchestra.run.smithi164.stdout:5263 trunc from 0xc1b6a00 to 0xae2ee00 2024-09-21T21:42:22.875 INFO:teuthology.orchestra.run.smithi164.stdout:5264 punch from 0x60cc600 to 0x60d6600, (0xa000 bytes) 2024-09-21T21:42:22.880 INFO:teuthology.orchestra.run.smithi164.stdout:5266 read 0x8dff000 thru 0x8e034d8 (0x44d9 bytes) 2024-09-21T21:42:22.890 INFO:teuthology.orchestra.run.smithi164.stdout:5267 trunc from 0xae2ee00 to 0x2885400 2024-09-21T21:42:22.915 INFO:teuthology.orchestra.run.smithi164.stdout:5268 read 0x2809800 thru 0x280ae62 (0x1663 bytes) 2024-09-21T21:42:22.922 INFO:teuthology.orchestra.run.smithi164.stdout:5271 read 0x1596400 thru 0x159a71f (0x4320 bytes) 2024-09-21T21:42:22.943 INFO:teuthology.orchestra.run.smithi164.stdout:5273 write 0xbb34400 thru 0xbb400a4 (0xbca5 bytes) 2024-09-21T21:42:22.950 INFO:teuthology.orchestra.run.smithi164.stdout:5277 punch from 0x19f1c00 to 0x1a01400, (0xf800 bytes) 2024-09-21T21:42:22.951 INFO:teuthology.orchestra.run.smithi164.stdout:5278 read 0x97aee00 thru 0x97b75d5 (0x87d6 bytes) 2024-09-21T21:42:22.951 INFO:teuthology.orchestra.run.smithi164.stdout:5280 write 0x4eb6a00 thru 0x4ec5ffe (0xf5ff bytes) 2024-09-21T21:42:22.952 INFO:teuthology.orchestra.run.smithi164.stdout:5284 write 0xaf5c800 thru 0xaf66572 (0x9d73 bytes) 2024-09-21T21:42:22.952 INFO:teuthology.orchestra.run.smithi164.stdout:5286 read 0x579be00 thru 0x57a65cd (0xa7ce bytes) 2024-09-21T21:42:22.952 INFO:teuthology.orchestra.run.smithi164.stdout:5287 write 0x1599400 thru 0x1599c60 (0x861 bytes) 2024-09-21T21:42:22.952 INFO:teuthology.orchestra.run.smithi164.stdout:5290 read 0x3ca8600 thru 0x3cb49a0 (0xc3a1 bytes) 2024-09-21T21:42:22.952 INFO:teuthology.orchestra.run.smithi164.stdout:5291 write 0x303200 thru 0x307c5f (0x4a60 bytes) 2024-09-21T21:42:22.952 INFO:teuthology.orchestra.run.smithi164.stdout:5292 punch from 0xae83a00 to 0xae90000, (0xc600 bytes) 2024-09-21T21:42:22.953 INFO:teuthology.orchestra.run.smithi164.stdout:5295 read 0x8448000 thru 0x844a39c (0x239d bytes) 2024-09-21T21:42:22.953 INFO:teuthology.orchestra.run.smithi164.stdout:5297 read 0x6094600 thru 0x6097e94 (0x3895 bytes) 2024-09-21T21:42:22.966 INFO:teuthology.orchestra.run.smithi164.stdout:5298 trunc from 0xbb40200 to 0xb5c400 2024-09-21T21:42:22.972 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:22.971+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:23.037 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:23.037 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:42:23.037 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:42:23.037 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:42:23.038 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:23.049 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T21:42:23.074 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:23.072+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:23.075 INFO:teuthology.orchestra.run.smithi164.stdout:5300 write 0x5d5d200 thru 0x5d6b800 (0xe601 bytes) 2024-09-21T21:42:23.081 INFO:teuthology.orchestra.run.smithi164.stdout:5305 trunc from 0x5d6ba00 to 0xadac000 2024-09-21T21:42:23.111 INFO:teuthology.orchestra.run.smithi164.stdout:5309 read 0x907a000 thru 0x907d66c (0x366d bytes) 2024-09-21T21:42:23.112 INFO:teuthology.orchestra.run.smithi164.stdout:5310 write 0x7245e00 thru 0x72532dd (0xd4de bytes) 2024-09-21T21:42:23.112 INFO:teuthology.orchestra.run.smithi164.stdout:5311 read 0x24fb400 thru 0x24ff50b (0x410c bytes) 2024-09-21T21:42:23.146 INFO:teuthology.orchestra.run.smithi164.stdout:5313 write 0xd1cfe00 thru 0xd1dbcd0 (0xbed1 bytes) 2024-09-21T21:42:23.146 INFO:teuthology.orchestra.run.smithi164.stdout:5314 punch from 0x9d99c00 to 0x9da4c00, (0xb000 bytes) 2024-09-21T21:42:23.147 INFO:teuthology.orchestra.run.smithi164.stdout:5316 read 0x841600 thru 0x851093 (0xfa94 bytes) 2024-09-21T21:42:23.165 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:23.166 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 20, 2024-09-21T21:42:23.166 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 600, 2024-09-21T21:42:23.166 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:42:23.166 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.166 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4824.0:1564 3.2 3:4647520c:::rbd_data.1121b97f2925.0000000000000006:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:42:23.166 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:42:23.166 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 81.527003811, 2024-09-21T21:42:23.166 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056097347999999998, 2024-09-21T21:42:23.166 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:23.166 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:23.166 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:23.166 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:23.166 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4824", 2024-09-21T21:42:23.166 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:42:23.167 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1564 2024-09-21T21:42:23.167 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.167 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:23.167 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.167 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:23.167 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:42:23.167 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:23.167 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.167 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.167 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:23.167 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:42:23.167 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:23.167 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.168 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.168 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:23.168 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:42:23.168 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6499999999999999e-07 2024-09-21T21:42:23.168 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.168 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.168 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:23.168 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636067+0000", 2024-09-21T21:42:23.168 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3879999999999998e-06 2024-09-21T21:42:23.168 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.168 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.168 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:23.168 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636075+0000", 2024-09-21T21:42:23.168 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5480000000000004e-06 2024-09-21T21:42:23.168 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.169 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.169 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:23.169 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636080+0000", 2024-09-21T21:42:23.169 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.339e-06 2024-09-21T21:42:23.169 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.169 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.169 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:23.169 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636133+0000", 2024-09-21T21:42:23.169 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2930000000000003e-05 2024-09-21T21:42:23.169 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.169 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.169 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:23.170 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636265+0000", 2024-09-21T21:42:23.170 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000132058 2024-09-21T21:42:23.170 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.170 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.170 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:23.170 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.692162+0000", 2024-09-21T21:42:23.170 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055896620000000001 2024-09-21T21:42:23.170 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.170 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:23.170 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.171 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.171 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.171 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3260 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 1437696~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:42:23.171 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.196297+0000", 2024-09-21T21:42:23.171 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 35.966771600999998, 2024-09-21T21:42:23.171 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056071022999999998, 2024-09-21T21:42:23.171 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:23.171 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:23.171 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:23.171 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:23.171 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:23.172 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:23.172 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3260 2024-09-21T21:42:23.172 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.172 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:23.172 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.172 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:23.172 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196297+0000", 2024-09-21T21:42:23.172 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:23.172 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.172 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.172 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:23.172 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196297+0000", 2024-09-21T21:42:23.172 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:23.172 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.173 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.173 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:23.173 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196918+0000", 2024-09-21T21:42:23.173 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00062148699999999999 2024-09-21T21:42:23.173 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.173 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.173 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:23.173 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196930+0000", 2024-09-21T21:42:23.173 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1657e-05 2024-09-21T21:42:23.173 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.173 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.173 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:23.173 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196944+0000", 2024-09-21T21:42:23.173 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.4549000000000001e-05 2024-09-21T21:42:23.174 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.174 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.174 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:23.174 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196951+0000", 2024-09-21T21:42:23.174 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6749999999999996e-06 2024-09-21T21:42:23.174 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.174 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.174 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:23.174 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252208+0000", 2024-09-21T21:42:23.174 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055256656000000001 2024-09-21T21:42:23.174 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.174 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.174 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:23.175 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252278+0000", 2024-09-21T21:42:23.175 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9882000000000006e-05 2024-09-21T21:42:23.175 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.175 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.175 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:23.175 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252368+0000", 2024-09-21T21:42:23.175 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.0117000000000005e-05 2024-09-21T21:42:23.175 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.175 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:23.175 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.175 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.175 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.175 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3263 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 3559424~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:42:23.175 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.197041+0000", 2024-09-21T21:42:23.175 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 35.966027787000002, 2024-09-21T21:42:23.176 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055935804999999998, 2024-09-21T21:42:23.176 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:23.176 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:23.176 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:23.176 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:23.176 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:23.176 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:23.176 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3263 2024-09-21T21:42:23.176 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.176 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:23.176 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.176 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:23.176 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197041+0000", 2024-09-21T21:42:23.176 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:23.176 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.177 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.177 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:23.177 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197041+0000", 2024-09-21T21:42:23.177 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:23.177 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.177 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.177 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:23.177 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197624+0000", 2024-09-21T21:42:23.177 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00058358100000000001 2024-09-21T21:42:23.177 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.177 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.177 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:23.177 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197629+0000", 2024-09-21T21:42:23.177 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5909999999999998e-06 2024-09-21T21:42:23.177 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.178 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.178 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:23.178 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197639+0000", 2024-09-21T21:42:23.178 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0257e-05 2024-09-21T21:42:23.178 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.178 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.178 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:23.178 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197646+0000", 2024-09-21T21:42:23.178 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.7229999999999998e-06 2024-09-21T21:42:23.178 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.178 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.178 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:23.178 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252804+0000", 2024-09-21T21:42:23.178 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055158134999999997 2024-09-21T21:42:23.179 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.179 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.179 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:23.179 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252882+0000", 2024-09-21T21:42:23.179 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7791000000000001e-05 2024-09-21T21:42:23.179 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.179 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.179 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:23.179 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252976+0000", 2024-09-21T21:42:23.179 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.4727000000000003e-05 2024-09-21T21:42:23.179 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.179 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:23.179 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.179 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.179 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.180 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3437 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 925696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:42:23.180 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.205888+0000", 2024-09-21T21:42:23.180 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 35.957180506, 2024-09-21T21:42:23.180 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055953808000000001, 2024-09-21T21:42:23.180 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:23.180 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:23.180 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:23.180 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:23.180 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:23.180 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:23.180 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3437 2024-09-21T21:42:23.180 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.180 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:23.180 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.181 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:23.181 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205888+0000", 2024-09-21T21:42:23.181 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:23.181 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.181 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.181 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:23.181 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205888+0000", 2024-09-21T21:42:23.181 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:23.181 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.181 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.182 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:23.182 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205888+0000", 2024-09-21T21:42:23.182 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.75e-07 2024-09-21T21:42:23.182 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.182 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.182 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:23.182 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205891+0000", 2024-09-21T21:42:23.182 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3750000000000001e-06 2024-09-21T21:42:23.182 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.182 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.182 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:23.182 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205897+0000", 2024-09-21T21:42:23.182 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.285e-06 2024-09-21T21:42:23.182 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.182 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.183 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:23.183 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205902+0000", 2024-09-21T21:42:23.183 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6290000000000001e-06 2024-09-21T21:42:23.183 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.183 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.183 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:23.183 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261686+0000", 2024-09-21T21:42:23.183 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055784586999999997 2024-09-21T21:42:23.183 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.183 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.183 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:23.183 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261764+0000", 2024-09-21T21:42:23.183 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7591999999999998e-05 2024-09-21T21:42:23.183 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.184 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.184 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:23.184 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261842+0000", 2024-09-21T21:42:23.184 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7865000000000001e-05 2024-09-21T21:42:23.184 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.184 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:23.184 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.184 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.184 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.184 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3438 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 954368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:42:23.184 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.205908+0000", 2024-09-21T21:42:23.184 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 35.957160641000002, 2024-09-21T21:42:23.184 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056083169000000002, 2024-09-21T21:42:23.184 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:23.185 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:23.185 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:23.185 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:23.185 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:23.185 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:23.185 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3438 2024-09-21T21:42:23.185 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.185 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:23.185 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.185 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:23.185 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205908+0000", 2024-09-21T21:42:23.185 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:23.185 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.185 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.185 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:23.186 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205908+0000", 2024-09-21T21:42:23.186 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:23.186 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.186 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.186 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:23.186 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206431+0000", 2024-09-21T21:42:23.186 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00052282100000000003 2024-09-21T21:42:23.186 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.186 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.186 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:23.186 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206442+0000", 2024-09-21T21:42:23.186 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1695000000000001e-05 2024-09-21T21:42:23.186 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.186 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.186 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:23.187 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206452+0000", 2024-09-21T21:42:23.187 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.4560000000000006e-06 2024-09-21T21:42:23.187 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.187 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.187 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:23.187 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206458+0000", 2024-09-21T21:42:23.187 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4670000000000001e-06 2024-09-21T21:42:23.187 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.187 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.187 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:23.187 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261871+0000", 2024-09-21T21:42:23.187 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055412712000000003 2024-09-21T21:42:23.187 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.187 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.187 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:23.188 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261928+0000", 2024-09-21T21:42:23.188 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6631e-05 2024-09-21T21:42:23.188 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.188 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.188 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:23.188 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261991+0000", 2024-09-21T21:42:23.188 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3386999999999995e-05 2024-09-21T21:42:23.188 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.188 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:23.188 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.188 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.188 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.188 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3444 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 2359296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:42:23.188 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.206533+0000", 2024-09-21T21:42:23.188 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 35.956535121000002, 2024-09-21T21:42:23.189 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055974239000000002, 2024-09-21T21:42:23.189 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:23.189 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:23.189 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:23.189 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:23.189 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:23.189 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:23.189 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3444 2024-09-21T21:42:23.189 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.189 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:23.189 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.189 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:23.189 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206533+0000", 2024-09-21T21:42:23.190 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:23.190 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.190 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.190 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:23.190 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206533+0000", 2024-09-21T21:42:23.190 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:23.190 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.190 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.190 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:23.190 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206710+0000", 2024-09-21T21:42:23.190 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000176293 2024-09-21T21:42:23.190 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.190 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.190 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:23.191 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206714+0000", 2024-09-21T21:42:23.191 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1989999999999999e-06 2024-09-21T21:42:23.191 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.191 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.191 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:23.191 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206723+0000", 2024-09-21T21:42:23.191 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.9199999999999993e-06 2024-09-21T21:42:23.191 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.191 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.191 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:23.191 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206729+0000", 2024-09-21T21:42:23.191 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0739999999999998e-06 2024-09-21T21:42:23.191 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.191 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.191 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:23.192 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262339+0000", 2024-09-21T21:42:23.192 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055610166000000003 2024-09-21T21:42:23.192 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.192 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.192 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:23.192 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262419+0000", 2024-09-21T21:42:23.192 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0432999999999994e-05 2024-09-21T21:42:23.192 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.192 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.192 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:23.192 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262507+0000", 2024-09-21T21:42:23.192 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.8153999999999997e-05 2024-09-21T21:42:23.192 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.192 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:23.192 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.192 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.193 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.193 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10362 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:23.193 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840386+0000", 2024-09-21T21:42:23.193 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.3226822770000002, 2024-09-21T21:42:23.193 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056546515999999998, 2024-09-21T21:42:23.193 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:23.193 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:23.193 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:23.193 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:23.193 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:23.193 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:23.193 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10362 2024-09-21T21:42:23.193 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.193 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:23.193 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.194 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:23.194 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840386+0000", 2024-09-21T21:42:23.194 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:23.194 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.194 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.194 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:23.194 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840386+0000", 2024-09-21T21:42:23.194 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:23.194 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.194 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.194 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:23.194 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840386+0000", 2024-09-21T21:42:23.194 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.01e-07 2024-09-21T21:42:23.194 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.195 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.195 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:23.195 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840389+0000", 2024-09-21T21:42:23.195 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.9739999999999998e-06 2024-09-21T21:42:23.195 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.195 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.195 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:23.195 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840396+0000", 2024-09-21T21:42:23.195 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2280000000000003e-06 2024-09-21T21:42:23.195 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.195 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.195 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:23.195 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840400+0000", 2024-09-21T21:42:23.195 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4429999999999996e-06 2024-09-21T21:42:23.195 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.196 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.196 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:23.196 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.890896+0000", 2024-09-21T21:42:23.196 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.050495717000000002 2024-09-21T21:42:23.196 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.196 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.196 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:23.196 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.891012+0000", 2024-09-21T21:42:23.196 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011602 2024-09-21T21:42:23.196 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.196 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.196 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:23.196 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.891274+0000", 2024-09-21T21:42:23.196 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00026240200000000001 2024-09-21T21:42:23.196 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.197 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.197 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:23.197 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.894810+0000", 2024-09-21T21:42:23.197 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0035353630000000001 2024-09-21T21:42:23.197 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.197 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.197 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:23.197 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.896818+0000", 2024-09-21T21:42:23.197 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.002008681 2024-09-21T21:42:23.197 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.197 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.197 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:23.197 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.896850+0000", 2024-09-21T21:42:23.197 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.1826999999999999e-05 2024-09-21T21:42:23.198 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.198 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.198 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:23.198 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.896933+0000", 2024-09-21T21:42:23.198 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.2460000000000007e-05 2024-09-21T21:42:23.198 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.198 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:23.198 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.198 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.198 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.198 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10363 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:23.198 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840606+0000", 2024-09-21T21:42:23.198 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.3224627669999993, 2024-09-21T21:42:23.198 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056546011, 2024-09-21T21:42:23.199 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:23.199 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:23.199 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:23.199 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:23.199 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:23.199 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:23.199 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10363 2024-09-21T21:42:23.199 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.199 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:23.199 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.199 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:23.199 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840606+0000", 2024-09-21T21:42:23.199 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:23.199 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.199 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.200 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:23.200 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840606+0000", 2024-09-21T21:42:23.200 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:23.200 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.200 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.200 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:23.200 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840606+0000", 2024-09-21T21:42:23.200 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.4700000000000001e-07 2024-09-21T21:42:23.200 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.200 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.200 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:23.200 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840611+0000", 2024-09-21T21:42:23.200 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4649999999999996e-06 2024-09-21T21:42:23.200 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.201 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.201 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:23.201 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840620+0000", 2024-09-21T21:42:23.201 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.0350000000000007e-06 2024-09-21T21:42:23.201 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.201 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.201 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:23.201 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840626+0000", 2024-09-21T21:42:23.201 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7180000000000001e-06 2024-09-21T21:42:23.201 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.201 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.201 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:23.201 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.891665+0000", 2024-09-21T21:42:23.201 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.051039924 2024-09-21T21:42:23.201 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.202 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.202 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:23.202 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.891770+0000", 2024-09-21T21:42:23.202 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010445700000000001 2024-09-21T21:42:23.202 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.202 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.202 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:23.202 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.892147+0000", 2024-09-21T21:42:23.202 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00037731800000000002 2024-09-21T21:42:23.202 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.202 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.202 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:23.202 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.896776+0000", 2024-09-21T21:42:23.202 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0046289759999999999 2024-09-21T21:42:23.203 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.203 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.203 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:23.203 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897003+0000", 2024-09-21T21:42:23.203 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00022641299999999999 2024-09-21T21:42:23.203 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.203 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.203 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:23.203 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897064+0000", 2024-09-21T21:42:23.203 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1571999999999999e-05 2024-09-21T21:42:23.203 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.203 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.203 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:23.203 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897152+0000", 2024-09-21T21:42:23.203 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.7386000000000007e-05 2024-09-21T21:42:23.204 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.204 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:23.204 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.204 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.204 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.204 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10364 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:23.204 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840641+0000", 2024-09-21T21:42:23.204 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.3224269470000003, 2024-09-21T21:42:23.204 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056671527999999999, 2024-09-21T21:42:23.204 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:23.204 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:23.204 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:23.204 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:23.204 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:23.204 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:23.205 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10364 2024-09-21T21:42:23.205 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.205 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:23.205 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.205 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:23.205 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840641+0000", 2024-09-21T21:42:23.205 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:23.205 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.205 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.205 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:23.205 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840641+0000", 2024-09-21T21:42:23.205 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:23.205 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.205 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.205 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:23.206 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840642+0000", 2024-09-21T21:42:23.206 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7899999999999999e-07 2024-09-21T21:42:23.206 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.206 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.206 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:23.206 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840645+0000", 2024-09-21T21:42:23.206 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0010000000000002e-06 2024-09-21T21:42:23.206 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.206 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.206 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:23.206 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840654+0000", 2024-09-21T21:42:23.206 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.8570000000000001e-06 2024-09-21T21:42:23.206 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.206 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.206 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:23.207 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840659+0000", 2024-09-21T21:42:23.207 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.558e-06 2024-09-21T21:42:23.207 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.207 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.207 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:23.207 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.892656+0000", 2024-09-21T21:42:23.207 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.051996586999999997 2024-09-21T21:42:23.207 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.207 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.207 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:23.207 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.892721+0000", 2024-09-21T21:42:23.207 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5010000000000003e-05 2024-09-21T21:42:23.207 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.207 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.208 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:23.208 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.893112+0000", 2024-09-21T21:42:23.208 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00039133899999999999 2024-09-21T21:42:23.208 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.208 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.208 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:23.208 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.896784+0000", 2024-09-21T21:42:23.208 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0036713679999999999 2024-09-21T21:42:23.208 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.208 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.208 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:23.208 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897204+0000", 2024-09-21T21:42:23.208 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00041991400000000002 2024-09-21T21:42:23.208 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.208 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.209 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:23.209 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897234+0000", 2024-09-21T21:42:23.209 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0307999999999998e-05 2024-09-21T21:42:23.209 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.209 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.209 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:23.209 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897313+0000", 2024-09-21T21:42:23.209 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.9107e-05 2024-09-21T21:42:23.209 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.209 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:23.209 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.209 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.209 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.209 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10365 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:23.209 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840668+0000", 2024-09-21T21:42:23.210 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.3224000819999997, 2024-09-21T21:42:23.210 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.058761225, 2024-09-21T21:42:23.210 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:23.210 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:23.210 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:23.210 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:23.210 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:23.210 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:23.210 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10365 2024-09-21T21:42:23.210 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.210 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:23.210 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.210 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:23.210 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840668+0000", 2024-09-21T21:42:23.210 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:23.211 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.211 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.211 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:23.211 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840668+0000", 2024-09-21T21:42:23.211 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:23.211 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.211 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.211 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:23.211 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840669+0000", 2024-09-21T21:42:23.211 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6300000000000001e-07 2024-09-21T21:42:23.211 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.211 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.211 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:23.211 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840672+0000", 2024-09-21T21:42:23.211 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.072e-06 2024-09-21T21:42:23.212 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.212 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.212 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:23.212 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840678+0000", 2024-09-21T21:42:23.212 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.7039999999999997e-06 2024-09-21T21:42:23.212 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.212 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.212 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:23.212 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840683+0000", 2024-09-21T21:42:23.212 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6759999999999998e-06 2024-09-21T21:42:23.212 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.212 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.212 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:23.212 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.893929+0000", 2024-09-21T21:42:23.212 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.053246136999999999 2024-09-21T21:42:23.213 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.213 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.213 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:23.213 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.894082+0000", 2024-09-21T21:42:23.213 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00015301799999999999 2024-09-21T21:42:23.213 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.213 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.213 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:23.213 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.894357+0000", 2024-09-21T21:42:23.213 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00027507999999999998 2024-09-21T21:42:23.213 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.213 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.213 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:23.213 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897161+0000", 2024-09-21T21:42:23.213 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0028036810000000001 2024-09-21T21:42:23.214 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.214 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.214 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:23.214 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.899243+0000", 2024-09-21T21:42:23.214 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0020816379999999998 2024-09-21T21:42:23.214 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.214 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.214 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:23.214 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.899306+0000", 2024-09-21T21:42:23.214 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2964000000000002e-05 2024-09-21T21:42:23.214 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.214 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.214 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:23.214 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.899430+0000", 2024-09-21T21:42:23.214 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00012389200000000001 2024-09-21T21:42:23.215 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.215 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:23.215 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.215 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.215 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.215 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10366 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:23.215 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840691+0000", 2024-09-21T21:42:23.215 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.3223773819999991, 2024-09-21T21:42:23.215 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.061540984999999999, 2024-09-21T21:42:23.215 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:23.215 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:23.215 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:23.215 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:23.215 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:23.216 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:23.216 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10366 2024-09-21T21:42:23.216 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.216 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:23.216 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.216 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:23.216 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840691+0000", 2024-09-21T21:42:23.216 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:23.216 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.216 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.216 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:23.216 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840691+0000", 2024-09-21T21:42:23.216 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:23.216 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.216 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.217 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:23.217 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840691+0000", 2024-09-21T21:42:23.217 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3099999999999999e-07 2024-09-21T21:42:23.217 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.217 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.217 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:23.217 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840694+0000", 2024-09-21T21:42:23.217 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.5780000000000001e-06 2024-09-21T21:42:23.217 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.217 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.217 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:23.217 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840700+0000", 2024-09-21T21:42:23.217 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9429999999999997e-06 2024-09-21T21:42:23.217 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.218 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.218 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:23.218 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840704+0000", 2024-09-21T21:42:23.218 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6380000000000003e-06 2024-09-21T21:42:23.218 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.218 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.218 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:23.218 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.894846+0000", 2024-09-21T21:42:23.218 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.054141392000000003 2024-09-21T21:42:23.218 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.218 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.218 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:23.218 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.894932+0000", 2024-09-21T21:42:23.218 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.5834000000000005e-05 2024-09-21T21:42:23.218 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.219 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.219 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:23.219 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.895302+0000", 2024-09-21T21:42:23.219 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000370737 2024-09-21T21:42:23.219 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.219 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.219 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:23.219 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897166+0000", 2024-09-21T21:42:23.219 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0018632620000000001 2024-09-21T21:42:23.219 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.219 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.219 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:23.219 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902053+0000", 2024-09-21T21:42:23.219 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0048868289999999997 2024-09-21T21:42:23.220 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.220 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.220 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:23.220 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902111+0000", 2024-09-21T21:42:23.220 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7983999999999999e-05 2024-09-21T21:42:23.220 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.220 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.220 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:23.220 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902232+0000", 2024-09-21T21:42:23.220 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000121457 2024-09-21T21:42:23.220 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.220 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:23.220 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.220 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.220 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.220 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10367 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:23.221 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840842+0000", 2024-09-21T21:42:23.221 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.3222265740000001, 2024-09-21T21:42:23.221 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.061549088000000002, 2024-09-21T21:42:23.221 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:23.221 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:23.221 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:23.221 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:23.221 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:23.221 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:23.221 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10367 2024-09-21T21:42:23.221 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.221 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:23.221 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.221 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:23.222 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840842+0000", 2024-09-21T21:42:23.222 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:23.222 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.222 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.222 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:23.222 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840842+0000", 2024-09-21T21:42:23.222 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:23.222 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.222 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.222 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:23.222 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840842+0000", 2024-09-21T21:42:23.222 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3099999999999997e-07 2024-09-21T21:42:23.222 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.222 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.222 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:23.223 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840846+0000", 2024-09-21T21:42:23.223 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.2890000000000002e-06 2024-09-21T21:42:23.223 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.223 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.223 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:23.223 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840855+0000", 2024-09-21T21:42:23.223 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.9619999999999999e-06 2024-09-21T21:42:23.223 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.223 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.223 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:23.223 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840860+0000", 2024-09-21T21:42:23.223 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3759999999999999e-06 2024-09-21T21:42:23.223 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.223 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.223 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:23.223 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.895799+0000", 2024-09-21T21:42:23.224 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.054938569999999999 2024-09-21T21:42:23.224 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.224 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.224 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:23.224 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.895906+0000", 2024-09-21T21:42:23.224 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000107593 2024-09-21T21:42:23.224 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.224 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.224 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:23.224 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.896356+0000", 2024-09-21T21:42:23.224 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00044941700000000002 2024-09-21T21:42:23.224 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.224 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.224 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:23.225 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.898131+0000", 2024-09-21T21:42:23.225 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.001775064 2024-09-21T21:42:23.225 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.225 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.225 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:23.225 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902283+0000", 2024-09-21T21:42:23.225 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0041524839999999997 2024-09-21T21:42:23.225 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.225 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.225 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:23.225 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902316+0000", 2024-09-21T21:42:23.225 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3002e-05 2024-09-21T21:42:23.225 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.225 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.225 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:23.225 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902391+0000", 2024-09-21T21:42:23.226 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.47e-05 2024-09-21T21:42:23.226 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.226 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:23.226 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.226 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.226 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.226 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10368 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:23.226 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840873+0000", 2024-09-21T21:42:23.226 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.3221956200000005, 2024-09-21T21:42:23.226 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.063395291000000006, 2024-09-21T21:42:23.226 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:23.226 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:23.226 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:23.226 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:23.227 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:23.227 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:23.227 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10368 2024-09-21T21:42:23.227 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.227 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:23.227 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.227 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:23.227 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840873+0000", 2024-09-21T21:42:23.227 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:23.227 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.227 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.227 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:23.227 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840873+0000", 2024-09-21T21:42:23.227 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:23.227 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.228 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.228 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:23.228 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840873+0000", 2024-09-21T21:42:23.228 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6699999999999999e-07 2024-09-21T21:42:23.228 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.228 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.228 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:23.228 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840876+0000", 2024-09-21T21:42:23.228 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.6410000000000002e-06 2024-09-21T21:42:23.228 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.228 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.228 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:23.228 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840883+0000", 2024-09-21T21:42:23.228 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0389999999999999e-06 2024-09-21T21:42:23.228 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.229 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.229 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:23.229 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840888+0000", 2024-09-21T21:42:23.229 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8679999999999998e-06 2024-09-21T21:42:23.229 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.229 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.229 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:23.229 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897345+0000", 2024-09-21T21:42:23.229 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056457734000000002 2024-09-21T21:42:23.229 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.229 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.229 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:23.229 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897407+0000", 2024-09-21T21:42:23.229 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1822000000000005e-05 2024-09-21T21:42:23.229 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.230 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.230 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:23.230 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897664+0000", 2024-09-21T21:42:23.230 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00025638499999999998 2024-09-21T21:42:23.230 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.230 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.230 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:23.230 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.901136+0000", 2024-09-21T21:42:23.230 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0034728070000000001 2024-09-21T21:42:23.230 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.230 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.230 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:23.230 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904090+0000", 2024-09-21T21:42:23.230 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0029535799999999999 2024-09-21T21:42:23.230 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.231 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.231 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:23.231 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904144+0000", 2024-09-21T21:42:23.231 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4369000000000001e-05 2024-09-21T21:42:23.231 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.231 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.231 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:23.231 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904268+0000", 2024-09-21T21:42:23.231 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00012367899999999999 2024-09-21T21:42:23.231 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.231 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:23.231 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.231 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.231 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.231 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10369 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:23.232 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840902+0000", 2024-09-21T21:42:23.232 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.3221659670000001, 2024-09-21T21:42:23.232 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.063538938000000003, 2024-09-21T21:42:23.232 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:23.232 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:23.232 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:23.232 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:23.232 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:23.232 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:23.232 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10369 2024-09-21T21:42:23.232 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.232 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:23.232 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.232 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:23.233 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840902+0000", 2024-09-21T21:42:23.233 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:23.233 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.233 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.233 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:23.233 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840902+0000", 2024-09-21T21:42:23.233 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:23.233 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.233 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.233 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:23.233 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840903+0000", 2024-09-21T21:42:23.233 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2100000000000002e-07 2024-09-21T21:42:23.233 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.233 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.233 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:23.234 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840913+0000", 2024-09-21T21:42:23.234 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.048e-05 2024-09-21T21:42:23.234 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.234 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.234 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:23.234 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840920+0000", 2024-09-21T21:42:23.234 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0330000000000003e-06 2024-09-21T21:42:23.234 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.234 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.234 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:23.234 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840925+0000", 2024-09-21T21:42:23.234 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0170000000000004e-06 2024-09-21T21:42:23.234 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.234 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.235 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:23.235 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.898179+0000", 2024-09-21T21:42:23.235 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.057253671999999999 2024-09-21T21:42:23.235 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.235 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.235 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:23.235 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.898284+0000", 2024-09-21T21:42:23.235 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000105202 2024-09-21T21:42:23.235 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.235 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.235 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:23.235 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.898729+0000", 2024-09-21T21:42:23.235 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00044459300000000002 2024-09-21T21:42:23.235 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.236 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.236 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:23.236 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902246+0000", 2024-09-21T21:42:23.236 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0035171629999999998 2024-09-21T21:42:23.236 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.236 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.236 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:23.236 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904310+0000", 2024-09-21T21:42:23.236 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.002063765 2024-09-21T21:42:23.236 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.236 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.236 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:23.236 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904342+0000", 2024-09-21T21:42:23.236 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.2091999999999997e-05 2024-09-21T21:42:23.237 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.237 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.237 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:23.237 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904441+0000", 2024-09-21T21:42:23.237 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.9500000000000006e-05 2024-09-21T21:42:23.237 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.237 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:23.237 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.237 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.237 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.237 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10370 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:23.237 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840944+0000", 2024-09-21T21:42:23.237 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.3221242540000002, 2024-09-21T21:42:23.237 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.064436853000000002, 2024-09-21T21:42:23.237 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:23.238 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:23.238 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:23.238 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:23.238 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:23.238 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:23.238 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10370 2024-09-21T21:42:23.238 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.238 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:23.238 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.238 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:23.238 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840944+0000", 2024-09-21T21:42:23.238 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:23.238 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.238 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.238 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:23.239 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840944+0000", 2024-09-21T21:42:23.239 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:23.239 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.239 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.239 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:23.239 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840945+0000", 2024-09-21T21:42:23.239 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2300000000000002e-07 2024-09-21T21:42:23.239 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.239 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.239 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:23.239 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840947+0000", 2024-09-21T21:42:23.239 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.8200000000000001e-06 2024-09-21T21:42:23.239 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.239 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.239 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:23.240 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840955+0000", 2024-09-21T21:42:23.240 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.8660000000000006e-06 2024-09-21T21:42:23.240 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.240 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.240 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:23.240 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840960+0000", 2024-09-21T21:42:23.240 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8670000000000002e-06 2024-09-21T21:42:23.240 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.240 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.240 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:23.240 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.899460+0000", 2024-09-21T21:42:23.240 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.058499922000000003 2024-09-21T21:42:23.240 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.240 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.241 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:23.241 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.899530+0000", 2024-09-21T21:42:23.241 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9558e-05 2024-09-21T21:42:23.241 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.241 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.241 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:23.241 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.899775+0000", 2024-09-21T21:42:23.241 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000245231 2024-09-21T21:42:23.241 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.241 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.241 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:23.241 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902254+0000", 2024-09-21T21:42:23.241 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0024790519999999998 2024-09-21T21:42:23.241 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.241 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.242 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:23.242 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905183+0000", 2024-09-21T21:42:23.242 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.002929156 2024-09-21T21:42:23.242 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.242 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.242 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:23.242 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905243+0000", 2024-09-21T21:42:23.242 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0055e-05 2024-09-21T21:42:23.242 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.242 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.242 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:23.242 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905381+0000", 2024-09-21T21:42:23.242 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00013790300000000001 2024-09-21T21:42:23.242 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.242 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:23.243 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.243 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.243 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.243 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10371 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:23.243 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840980+0000", 2024-09-21T21:42:23.243 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.3220885879999997, 2024-09-21T21:42:23.243 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.064554376999999996, 2024-09-21T21:42:23.243 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:23.243 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:23.243 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:23.243 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:23.243 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:23.243 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:23.243 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10371 2024-09-21T21:42:23.244 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.244 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:23.244 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.244 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:23.244 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840980+0000", 2024-09-21T21:42:23.244 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:23.244 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.244 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.244 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:23.244 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840980+0000", 2024-09-21T21:42:23.244 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:23.244 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.244 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.244 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:23.244 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840980+0000", 2024-09-21T21:42:23.245 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8999999999999997e-07 2024-09-21T21:42:23.245 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.245 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.245 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:23.245 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840983+0000", 2024-09-21T21:42:23.245 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.123e-06 2024-09-21T21:42:23.245 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.245 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.245 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:23.245 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840990+0000", 2024-09-21T21:42:23.245 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9129999999999997e-06 2024-09-21T21:42:23.245 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.245 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.245 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:23.246 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840995+0000", 2024-09-21T21:42:23.246 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0989999999999998e-06 2024-09-21T21:42:23.246 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.246 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.246 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:23.246 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.900168+0000", 2024-09-21T21:42:23.246 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.059172739000000002 2024-09-21T21:42:23.246 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.246 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.246 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:23.246 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.900274+0000", 2024-09-21T21:42:23.246 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010546499999999999 2024-09-21T21:42:23.246 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.246 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.247 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:23.247 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.900703+0000", 2024-09-21T21:42:23.247 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00042978200000000003 2024-09-21T21:42:23.247 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.247 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.247 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:23.247 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.903443+0000", 2024-09-21T21:42:23.247 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0027400749999999998 2024-09-21T21:42:23.247 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.247 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.247 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:23.247 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905430+0000", 2024-09-21T21:42:23.247 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0019868109999999998 2024-09-21T21:42:23.247 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.247 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.248 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:23.248 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905464+0000", 2024-09-21T21:42:23.248 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3599000000000001e-05 2024-09-21T21:42:23.248 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.248 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.248 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:23.248 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905534+0000", 2024-09-21T21:42:23.248 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0281e-05 2024-09-21T21:42:23.248 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.248 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:23.249 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.249 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.249 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.249 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10372 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:23.249 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.841004+0000", 2024-09-21T21:42:23.249 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.3220646990000002, 2024-09-21T21:42:23.249 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.064757812999999997, 2024-09-21T21:42:23.249 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:23.249 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:23.249 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:23.249 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:23.249 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:23.249 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:23.249 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10372 2024-09-21T21:42:23.250 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.250 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:23.250 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.250 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:23.250 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841004+0000", 2024-09-21T21:42:23.250 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:23.250 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.250 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.250 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:23.250 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841004+0000", 2024-09-21T21:42:23.250 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:23.250 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.250 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.250 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:23.251 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841004+0000", 2024-09-21T21:42:23.251 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8500000000000002e-07 2024-09-21T21:42:23.251 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.251 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.251 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:23.251 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841007+0000", 2024-09-21T21:42:23.251 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.6390000000000001e-06 2024-09-21T21:42:23.251 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.251 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.251 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:23.251 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841013+0000", 2024-09-21T21:42:23.251 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0070000000000003e-06 2024-09-21T21:42:23.251 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.251 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.251 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:23.252 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841018+0000", 2024-09-21T21:42:23.252 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9119999999999997e-06 2024-09-21T21:42:23.252 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.252 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.252 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:23.252 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.901182+0000", 2024-09-21T21:42:23.252 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.060164444999999997 2024-09-21T21:42:23.252 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.252 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.252 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:23.252 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.901297+0000", 2024-09-21T21:42:23.253 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011468699999999999 2024-09-21T21:42:23.253 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.253 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.253 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:23.253 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.901559+0000", 2024-09-21T21:42:23.253 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000261764 2024-09-21T21:42:23.253 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.253 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.253 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:23.253 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904281+0000", 2024-09-21T21:42:23.253 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0027229329999999999 2024-09-21T21:42:23.253 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.253 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.253 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:23.254 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905588+0000", 2024-09-21T21:42:23.254 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.001306438 2024-09-21T21:42:23.254 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.254 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.254 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:23.254 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905644+0000", 2024-09-21T21:42:23.254 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5747e-05 2024-09-21T21:42:23.254 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.254 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.254 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:23.254 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905761+0000", 2024-09-21T21:42:23.254 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000117856 2024-09-21T21:42:23.254 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.254 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:23.254 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.255 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.255 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.255 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10373 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:23.255 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.841027+0000", 2024-09-21T21:42:23.255 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.3220416149999998, 2024-09-21T21:42:23.255 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.066941839000000003, 2024-09-21T21:42:23.255 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:23.255 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:23.255 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:23.255 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:23.255 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:23.255 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:23.255 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10373 2024-09-21T21:42:23.255 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.255 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:23.256 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.256 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:23.256 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841027+0000", 2024-09-21T21:42:23.256 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:23.256 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.256 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.256 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:23.256 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841027+0000", 2024-09-21T21:42:23.256 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:23.256 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.256 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.256 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:23.257 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841027+0000", 2024-09-21T21:42:23.257 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6199999999999999e-07 2024-09-21T21:42:23.257 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.257 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.257 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:23.257 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841030+0000", 2024-09-21T21:42:23.257 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.729e-06 2024-09-21T21:42:23.257 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.257 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.257 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:23.257 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841036+0000", 2024-09-21T21:42:23.257 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1979999999999998e-06 2024-09-21T21:42:23.257 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.258 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.258 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:23.258 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841042+0000", 2024-09-21T21:42:23.258 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4289999999999998e-06 2024-09-21T21:42:23.258 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.258 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.258 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:23.258 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902433+0000", 2024-09-21T21:42:23.258 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.061390329 2024-09-21T21:42:23.258 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.258 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.258 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:23.258 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902541+0000", 2024-09-21T21:42:23.258 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000108349 2024-09-21T21:42:23.258 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.259 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.259 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:23.259 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902983+0000", 2024-09-21T21:42:23.259 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00044158000000000001 2024-09-21T21:42:23.259 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.259 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.259 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:23.259 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904490+0000", 2024-09-21T21:42:23.259 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0015074489999999999 2024-09-21T21:42:23.259 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.259 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.259 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:23.259 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.907671+0000", 2024-09-21T21:42:23.259 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.003180511 2024-09-21T21:42:23.260 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.260 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.260 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:23.260 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.907767+0000", 2024-09-21T21:42:23.260 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6379000000000007e-05 2024-09-21T21:42:23.260 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.260 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.260 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:23.260 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.907969+0000", 2024-09-21T21:42:23.260 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00020152399999999999 2024-09-21T21:42:23.260 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.260 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:23.260 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.260 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.260 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.260 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:12335 3.1a 3:5b387735:::rbd_data.1121faebdfe3.0000000000000005:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:42:23.261 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:14.411635+0000", 2024-09-21T21:42:23.261 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 8.7514337369999993, 2024-09-21T21:42:23.261 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.005973491, 2024-09-21T21:42:23.261 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:23.261 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:23.261 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:23.261 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:23.261 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:23.261 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:23.261 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 12335 2024-09-21T21:42:23.261 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.261 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:23.261 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.261 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:23.261 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411635+0000", 2024-09-21T21:42:23.262 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:23.262 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.262 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.262 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:23.262 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411635+0000", 2024-09-21T21:42:23.262 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:23.262 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.262 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.262 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:23.262 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411635+0000", 2024-09-21T21:42:23.262 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8500000000000001e-07 2024-09-21T21:42:23.262 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.262 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.262 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:23.263 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411639+0000", 2024-09-21T21:42:23.263 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.574e-06 2024-09-21T21:42:23.263 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.263 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.263 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:23.263 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411645+0000", 2024-09-21T21:42:23.263 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6869999999999997e-06 2024-09-21T21:42:23.263 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.263 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.263 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:23.263 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411649+0000", 2024-09-21T21:42:23.263 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.771e-06 2024-09-21T21:42:23.263 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.263 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.263 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:23.264 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411695+0000", 2024-09-21T21:42:23.264 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6e-05 2024-09-21T21:42:23.264 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.264 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.264 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op must wait for map", 2024-09-21T21:42:23.264 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411702+0000", 2024-09-21T21:42:23.264 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5590000000000002e-06 2024-09-21T21:42:23.264 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.264 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.264 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:23.264 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.417222+0000", 2024-09-21T21:42:23.264 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0055198219999999997 2024-09-21T21:42:23.264 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.264 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.264 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for peered", 2024-09-21T21:42:23.265 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.417230+0000", 2024-09-21T21:42:23.265 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.123e-06 2024-09-21T21:42:23.265 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.265 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.265 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:23.265 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.417348+0000", 2024-09-21T21:42:23.265 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.000118029 2024-09-21T21:42:23.265 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.265 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.265 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:23.265 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.417451+0000", 2024-09-21T21:42:23.265 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000103674 2024-09-21T21:42:23.265 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.265 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.265 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:23.265 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.417608+0000", 2024-09-21T21:42:23.266 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00015666700000000001 2024-09-21T21:42:23.266 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.266 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:23.266 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.266 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.266 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.266 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:12349 3.1a 3:5b387735:::rbd_data.1121faebdfe3.0000000000000005:head [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:42:23.266 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:14.411833+0000", 2024-09-21T21:42:23.266 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 8.7512349240000002, 2024-09-21T21:42:23.266 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0065064500000001, 2024-09-21T21:42:23.266 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:23.266 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:23.266 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:23.266 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:23.266 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:23.267 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:23.267 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 12349 2024-09-21T21:42:23.267 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.267 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:23.267 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.267 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:23.267 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411833+0000", 2024-09-21T21:42:23.267 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:23.267 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.267 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.267 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:23.267 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411833+0000", 2024-09-21T21:42:23.267 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:23.267 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.267 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.268 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:23.268 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411834+0000", 2024-09-21T21:42:23.268 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.03e-07 2024-09-21T21:42:23.268 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.268 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.268 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:23.268 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411836+0000", 2024-09-21T21:42:23.268 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.4700000000000001e-06 2024-09-21T21:42:23.268 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.268 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.268 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:23.268 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411843+0000", 2024-09-21T21:42:23.268 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4069999999999998e-06 2024-09-21T21:42:23.268 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.268 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.269 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:23.269 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411848+0000", 2024-09-21T21:42:23.269 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1580000000000004e-06 2024-09-21T21:42:23.269 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.269 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.269 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:23.269 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411901+0000", 2024-09-21T21:42:23.269 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2824000000000001e-05 2024-09-21T21:42:23.269 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.269 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.269 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting_for_map not empty", 2024-09-21T21:42:23.269 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411904+0000", 2024-09-21T21:42:23.269 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0639999999999998e-06 2024-09-21T21:42:23.269 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.269 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.269 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:23.270 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.417252+0000", 2024-09-21T21:42:23.270 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0053485060000000003 2024-09-21T21:42:23.270 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.270 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.270 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for peered", 2024-09-21T21:42:23.270 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.417259+0000", 2024-09-21T21:42:23.270 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.1149999999999996e-06 2024-09-21T21:42:23.270 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.270 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.270 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:23.270 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.418160+0000", 2024-09-21T21:42:23.270 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.000901083 2024-09-21T21:42:23.270 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.270 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.271 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:23.271 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.418236+0000", 2024-09-21T21:42:23.271 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5852000000000005e-05 2024-09-21T21:42:23.271 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.271 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.271 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:23.271 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.418340+0000", 2024-09-21T21:42:23.271 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000103568 2024-09-21T21:42:23.271 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.271 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:23.271 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.271 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.271 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:23.271 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:23.273 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:23.271+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:23.274 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T21:42:23.275 INFO:teuthology.orchestra.run.smithi164.stdout:5318 read 0x9ac5c00 thru 0x9ad0e8e (0xb28f bytes) 2024-09-21T21:42:23.275 INFO:teuthology.orchestra.run.smithi164.stdout:5319 punch from 0x5497e00 to 0x549ac00, (0x2e00 bytes) 2024-09-21T21:42:23.275 INFO:teuthology.orchestra.run.smithi164.stdout:5320 write 0xc695600 thru 0xc69648d (0xe8e bytes) 2024-09-21T21:42:23.275 INFO:teuthology.orchestra.run.smithi164.stdout:5322 read 0xa355000 thru 0xa35d426 (0x8427 bytes) 2024-09-21T21:42:23.275 INFO:teuthology.orchestra.run.smithi164.stdout:5325 read 0xd01200 thru 0xd03522 (0x2323 bytes) 2024-09-21T21:42:23.275 INFO:teuthology.orchestra.run.smithi164.stdout:5326 trunc from 0xd1dbe00 to 0x65d7200 2024-09-21T21:42:23.275 INFO:teuthology.orchestra.run.smithi164.stdout:5329 punch from 0x3fdb200 to 0x3fe1400, (0x6200 bytes) 2024-09-21T21:42:23.275 INFO:teuthology.orchestra.run.smithi164.stdout:5331 trunc from 0x65d7200 to 0xb2c9400 2024-09-21T21:42:23.276 INFO:teuthology.orchestra.run.smithi164.stdout:5332 read 0x7998a00 thru 0x79a3301 (0xa902 bytes) 2024-09-21T21:42:23.276 INFO:teuthology.orchestra.run.smithi164.stdout:5333 write 0xe515e00 thru 0xe51a8dd (0x4ade bytes) 2024-09-21T21:42:23.276 INFO:teuthology.orchestra.run.smithi164.stdout:5334 write 0x6d22a00 thru 0x6d28b96 (0x6197 bytes) 2024-09-21T21:42:23.276 INFO:teuthology.orchestra.run.smithi164.stdout:5337 write 0xd167600 thru 0xd16d39b (0x5d9c bytes) 2024-09-21T21:42:23.276 INFO:teuthology.orchestra.run.smithi164.stdout:5338 read 0x7e72600 thru 0x7e7a317 (0x7d18 bytes) 2024-09-21T21:42:23.276 INFO:teuthology.orchestra.run.smithi164.stdout:5339 read 0x82c9000 thru 0x82d56db (0xc6dc bytes) 2024-09-21T21:42:23.276 INFO:teuthology.orchestra.run.smithi164.stdout:5346 punch from 0x426c00 to 0x434200, (0xd600 bytes) 2024-09-21T21:42:23.276 INFO:teuthology.orchestra.run.smithi164.stdout:5347 read 0xa8eac00 thru 0xa8f8d76 (0xe177 bytes) 2024-09-21T21:42:23.276 INFO:teuthology.orchestra.run.smithi164.stdout:5348 punch from 0xa328600 to 0xa338200, (0xfc00 bytes) 2024-09-21T21:42:23.276 INFO:teuthology.orchestra.run.smithi164.stdout:5349 read 0xb225800 thru 0xb23202a (0xc82b bytes) 2024-09-21T21:42:23.276 INFO:teuthology.orchestra.run.smithi164.stdout:5350 read 0x1c01c00 thru 0x1c0e9e7 (0xcde8 bytes) 2024-09-21T21:42:23.276 INFO:teuthology.orchestra.run.smithi164.stdout:5351 trunc from 0xe51aa00 to 0x32e6e00 2024-09-21T21:42:23.353 INFO:teuthology.orchestra.run.smithi164.stdout:5352 punch from 0xd2a200 to 0xd35e00, (0xbc00 bytes) 2024-09-21T21:42:23.361 INFO:teuthology.orchestra.run.smithi164.stdout:5354 punch from 0xd98c00 to 0xda0a00, (0x7e00 bytes) 2024-09-21T21:42:23.373 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:23.371+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:23.395 INFO:teuthology.orchestra.run.smithi164.stdout:5355 write 0xcc1b800 thru 0xcc2857e (0xcd7f bytes) 2024-09-21T21:42:23.395 INFO:teuthology.orchestra.run.smithi164.stdout:5356 read 0x9601200 thru 0x960eab2 (0xd8b3 bytes) 2024-09-21T21:42:23.395 INFO:teuthology.orchestra.run.smithi164.stdout:5357 punch from 0x50a6400 to 0x50ae600, (0x8200 bytes) 2024-09-21T21:42:23.396 INFO:teuthology.orchestra.run.smithi164.stdout:5358 read 0xa621800 thru 0xa62b24d (0x9a4e bytes) 2024-09-21T21:42:23.396 INFO:teuthology.orchestra.run.smithi164.stdout:5359 write 0x844e600 thru 0x8454ff0 (0x69f1 bytes) 2024-09-21T21:42:23.403 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:23.404 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:42:23.404 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.404 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_repop(client.5716.0:28288 3.5 e105/55 3:a65b9360:::rbd_data.1121c0cf2471.0000000000000191:head v 105'499, mlcod=105'497)", 2024-09-21T21:42:23.404 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:23.402037+0000", 2024-09-21T21:42:23.404 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.00084908800000000001, 2024-09-21T21:42:23.404 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00086549700000000001, 2024-09-21T21:42:23.404 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:23.404 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:23.404 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:23.404 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:23.404 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.405 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:23.405 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:23.402037+0000", 2024-09-21T21:42:23.405 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:23.405 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.405 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.405 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:23.405 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:23.402037+0000", 2024-09-21T21:42:23.405 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:23.405 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.405 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.405 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:23.405 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:23.402038+0000", 2024-09-21T21:42:23.405 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.9999999999999996e-07 2024-09-21T21:42:23.405 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.405 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.406 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:23.406 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:23.402047+0000", 2024-09-21T21:42:23.406 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2499999999999995e-06 2024-09-21T21:42:23.406 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.406 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.406 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:23.406 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:23.402057+0000", 2024-09-21T21:42:23.406 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0128e-05 2024-09-21T21:42:23.406 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.406 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.406 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:23.406 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:23.402062+0000", 2024-09-21T21:42:23.406 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7559999999999999e-06 2024-09-21T21:42:23.406 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.407 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.407 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:23.407 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:23.402108+0000", 2024-09-21T21:42:23.407 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6414e-05 2024-09-21T21:42:23.407 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.407 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.407 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:23.407 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:23.402123+0000", 2024-09-21T21:42:23.407 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.5324000000000001e-05 2024-09-21T21:42:23.407 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.407 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:23.407 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.407 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.407 INFO:teuthology.orchestra.run.smithi081.stdout: ], 2024-09-21T21:42:23.408 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 1 2024-09-21T21:42:23.408 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:23.408 INFO:teuthology.orchestra.run.smithi164.stdout:5361 trunc from 0xcc28600 to 0x60a0200 2024-09-21T21:42:23.413 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T21:42:23.448 INFO:teuthology.orchestra.run.smithi164.stdout:5362 clone 40 order 19 su 262144 sc 13 2024-09-21T21:42:23.474 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:23.473+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:23.508 INFO:teuthology.orchestra.run.smithi081.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-21T21:42:23.508 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T21:42:23.564 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:23.564 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:42:23.564 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:42:23.564 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:42:23.564 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:23.574 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T21:42:23.616 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:23.614+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:23.677 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:23.675+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:23.721 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:23.721 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 20, 2024-09-21T21:42:23.721 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 600, 2024-09-21T21:42:23.722 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:42:23.722 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.722 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13296 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 4046848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:23.722 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.796643+0000", 2024-09-21T21:42:23.722 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 32.922449379, 2024-09-21T21:42:23.722 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071682019999999999, 2024-09-21T21:42:23.722 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:23.722 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:23.722 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:23.722 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:23.722 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:23.722 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:23.722 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13296 2024-09-21T21:42:23.723 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.723 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:23.723 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.723 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:23.723 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796643+0000", 2024-09-21T21:42:23.723 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:23.723 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.723 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.723 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:23.723 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796643+0000", 2024-09-21T21:42:23.723 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:23.723 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.723 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.723 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:23.724 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796643+0000", 2024-09-21T21:42:23.724 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.7199999999999998e-07 2024-09-21T21:42:23.724 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.724 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.724 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:23.724 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796647+0000", 2024-09-21T21:42:23.724 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6770000000000001e-06 2024-09-21T21:42:23.724 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.724 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.724 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:23.724 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796656+0000", 2024-09-21T21:42:23.724 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2059999999999996e-06 2024-09-21T21:42:23.724 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.725 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.725 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:23.725 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796664+0000", 2024-09-21T21:42:23.725 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7919999999999993e-06 2024-09-21T21:42:23.725 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.725 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.725 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:23.725 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.868186+0000", 2024-09-21T21:42:23.725 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071522273999999997 2024-09-21T21:42:23.725 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.725 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.725 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:23.725 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.868239+0000", 2024-09-21T21:42:23.726 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2787999999999997e-05 2024-09-21T21:42:23.726 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.726 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.726 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:23.726 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.868325+0000", 2024-09-21T21:42:23.726 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.5611000000000004e-05 2024-09-21T21:42:23.726 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.726 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:23.726 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.726 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.726 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.726 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13561 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 1728512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:23.726 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.823991+0000", 2024-09-21T21:42:23.726 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 32.895101078000003, 2024-09-21T21:42:23.727 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071654762999999996, 2024-09-21T21:42:23.727 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:23.727 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:23.727 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:23.727 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:23.727 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:23.727 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:23.727 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13561 2024-09-21T21:42:23.727 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.727 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:23.727 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.727 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:23.727 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.823991+0000", 2024-09-21T21:42:23.728 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:23.728 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.728 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.728 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:23.728 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.823991+0000", 2024-09-21T21:42:23.728 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:23.728 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.728 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.728 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:23.728 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.823991+0000", 2024-09-21T21:42:23.728 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9299999999999998e-07 2024-09-21T21:42:23.728 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.728 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.728 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:23.728 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.823994+0000", 2024-09-21T21:42:23.729 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2369999999999999e-06 2024-09-21T21:42:23.729 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.729 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.729 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:23.729 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824000+0000", 2024-09-21T21:42:23.729 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.55e-06 2024-09-21T21:42:23.729 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.729 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.729 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:23.729 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824005+0000", 2024-09-21T21:42:23.729 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4329999999999997e-06 2024-09-21T21:42:23.729 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.729 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.730 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:23.730 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.895360+0000", 2024-09-21T21:42:23.730 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071355426 2024-09-21T21:42:23.730 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.730 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.730 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:23.730 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.895477+0000", 2024-09-21T21:42:23.730 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0001171 2024-09-21T21:42:23.730 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.730 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.730 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:23.730 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.895646+0000", 2024-09-21T21:42:23.730 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000168524 2024-09-21T21:42:23.730 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.730 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:23.731 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.731 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.731 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.731 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13562 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 1757184~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:23.731 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.824011+0000", 2024-09-21T21:42:23.731 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 32.895081200999996, 2024-09-21T21:42:23.731 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072020453999999998, 2024-09-21T21:42:23.731 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:23.731 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:23.731 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:23.731 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:23.731 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:23.731 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:23.731 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13562 2024-09-21T21:42:23.731 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.732 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:23.732 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.732 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:23.732 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824011+0000", 2024-09-21T21:42:23.732 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:23.732 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.732 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.732 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:23.732 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824011+0000", 2024-09-21T21:42:23.732 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:23.732 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.732 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.732 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:23.732 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824011+0000", 2024-09-21T21:42:23.733 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5600000000000001e-07 2024-09-21T21:42:23.733 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.733 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.733 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:23.733 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824013+0000", 2024-09-21T21:42:23.733 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9800000000000001e-06 2024-09-21T21:42:23.733 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.733 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.733 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:23.733 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824019+0000", 2024-09-21T21:42:23.733 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7339999999999996e-06 2024-09-21T21:42:23.733 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.733 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.733 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:23.734 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824023+0000", 2024-09-21T21:42:23.734 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3680000000000004e-06 2024-09-21T21:42:23.734 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.734 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.734 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:23.734 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.895699+0000", 2024-09-21T21:42:23.734 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071675597999999993 2024-09-21T21:42:23.734 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.734 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.734 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:23.734 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.895830+0000", 2024-09-21T21:42:23.734 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000130858 2024-09-21T21:42:23.734 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.734 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.735 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:23.735 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896031+0000", 2024-09-21T21:42:23.735 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00020155999999999999 2024-09-21T21:42:23.735 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.735 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:23.735 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.735 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.735 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.735 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13563 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 323584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:23.735 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.824029+0000", 2024-09-21T21:42:23.735 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 32.895062760999998, 2024-09-21T21:42:23.735 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072307563000000005, 2024-09-21T21:42:23.735 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:23.735 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:23.736 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:23.736 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:23.736 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:23.736 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:23.736 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13563 2024-09-21T21:42:23.736 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.736 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:23.736 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.736 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:23.736 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824029+0000", 2024-09-21T21:42:23.736 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:23.736 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.736 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.736 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:23.736 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824029+0000", 2024-09-21T21:42:23.737 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:23.737 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.737 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.737 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:23.737 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824030+0000", 2024-09-21T21:42:23.737 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1699999999999999e-07 2024-09-21T21:42:23.737 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.737 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.737 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:23.737 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824032+0000", 2024-09-21T21:42:23.737 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0320000000000002e-06 2024-09-21T21:42:23.737 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.737 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.737 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:23.738 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824038+0000", 2024-09-21T21:42:23.738 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8769999999999998e-06 2024-09-21T21:42:23.738 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.738 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.738 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:23.738 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824042+0000", 2024-09-21T21:42:23.738 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5650000000000003e-06 2024-09-21T21:42:23.738 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.738 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.738 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:23.738 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896058+0000", 2024-09-21T21:42:23.738 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072015941 2024-09-21T21:42:23.738 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.739 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.739 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:23.739 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896164+0000", 2024-09-21T21:42:23.739 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010593499999999999 2024-09-21T21:42:23.739 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.739 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.739 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:23.739 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896337+0000", 2024-09-21T21:42:23.739 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00017279599999999999 2024-09-21T21:42:23.739 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.739 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:23.739 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.739 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.740 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.740 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13564 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 983040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:23.740 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.824048+0000", 2024-09-21T21:42:23.740 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 32.895043686000001, 2024-09-21T21:42:23.740 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072491193999999995, 2024-09-21T21:42:23.740 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:23.740 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:23.740 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:23.740 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:23.740 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:23.740 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:23.740 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13564 2024-09-21T21:42:23.740 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.741 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:23.741 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.741 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:23.741 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824048+0000", 2024-09-21T21:42:23.741 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:23.741 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.741 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.741 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:23.741 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824048+0000", 2024-09-21T21:42:23.741 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:23.741 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.741 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.741 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:23.741 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824663+0000", 2024-09-21T21:42:23.742 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00061494299999999996 2024-09-21T21:42:23.742 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.742 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.742 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:23.742 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824669+0000", 2024-09-21T21:42:23.742 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2900000000000002e-06 2024-09-21T21:42:23.742 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.742 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.742 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:23.742 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824679+0000", 2024-09-21T21:42:23.742 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0338e-05 2024-09-21T21:42:23.742 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.742 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.742 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:23.742 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824687+0000", 2024-09-21T21:42:23.743 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.9049999999999997e-06 2024-09-21T21:42:23.743 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.743 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.743 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:23.743 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896381+0000", 2024-09-21T21:42:23.743 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071694305999999999 2024-09-21T21:42:23.743 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.743 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.743 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:23.743 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896445+0000", 2024-09-21T21:42:23.743 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3869999999999997e-05 2024-09-21T21:42:23.743 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.743 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.744 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:23.744 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896540+0000", 2024-09-21T21:42:23.744 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.4542000000000004e-05 2024-09-21T21:42:23.744 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.744 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:23.744 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.744 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.744 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.744 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13565 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3190784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:23.744 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.824704+0000", 2024-09-21T21:42:23.744 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 32.894387815000002, 2024-09-21T21:42:23.744 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072014745000000005, 2024-09-21T21:42:23.744 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:23.744 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:23.745 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:23.745 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:23.745 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:23.745 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:23.745 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13565 2024-09-21T21:42:23.745 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.745 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:23.745 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.745 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:23.745 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824704+0000", 2024-09-21T21:42:23.745 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:23.745 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.745 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.745 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:23.745 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824704+0000", 2024-09-21T21:42:23.746 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:23.746 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.746 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.746 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:23.746 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824705+0000", 2024-09-21T21:42:23.746 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8999999999999997e-07 2024-09-21T21:42:23.746 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.746 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.746 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:23.746 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824707+0000", 2024-09-21T21:42:23.746 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.7669999999999999e-06 2024-09-21T21:42:23.746 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.746 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.746 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:23.747 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824715+0000", 2024-09-21T21:42:23.747 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7700000000000001e-06 2024-09-21T21:42:23.747 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.747 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.747 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:23.747 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824721+0000", 2024-09-21T21:42:23.747 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9560000000000002e-06 2024-09-21T21:42:23.747 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.747 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.747 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:23.747 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896570+0000", 2024-09-21T21:42:23.747 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071848870999999995 2024-09-21T21:42:23.747 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.747 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.748 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:23.748 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896626+0000", 2024-09-21T21:42:23.748 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5578000000000002e-05 2024-09-21T21:42:23.748 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.748 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.748 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:23.748 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896719+0000", 2024-09-21T21:42:23.748 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.3313000000000006e-05 2024-09-21T21:42:23.748 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.748 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:23.748 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.748 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.748 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.748 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13566 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3162112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:23.749 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.824729+0000", 2024-09-21T21:42:23.749 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 32.894362928, 2024-09-21T21:42:23.749 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072356907999999998, 2024-09-21T21:42:23.749 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:23.749 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:23.749 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:23.749 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:23.749 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:23.749 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:23.749 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13566 2024-09-21T21:42:23.749 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.749 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:23.749 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.749 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:23.749 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824729+0000", 2024-09-21T21:42:23.750 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:23.750 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.750 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.750 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:23.750 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824729+0000", 2024-09-21T21:42:23.750 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:23.750 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.750 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.750 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:23.750 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825630+0000", 2024-09-21T21:42:23.750 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00090139400000000002 2024-09-21T21:42:23.750 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.750 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.750 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:23.751 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825634+0000", 2024-09-21T21:42:23.751 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4960000000000001e-06 2024-09-21T21:42:23.751 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.751 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.751 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:23.751 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825644+0000", 2024-09-21T21:42:23.751 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.9650000000000003e-06 2024-09-21T21:42:23.751 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.751 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.751 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:23.751 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825649+0000", 2024-09-21T21:42:23.751 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0359999999999997e-06 2024-09-21T21:42:23.751 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.751 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.752 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:23.752 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896757+0000", 2024-09-21T21:42:23.752 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071107717000000001 2024-09-21T21:42:23.752 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.752 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.752 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:23.752 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896878+0000", 2024-09-21T21:42:23.752 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000120834 2024-09-21T21:42:23.752 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.752 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.752 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:23.752 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897086+0000", 2024-09-21T21:42:23.752 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00020846600000000001 2024-09-21T21:42:23.752 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.753 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:23.753 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.753 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.753 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.753 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13568 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3907584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:23.753 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.825671+0000", 2024-09-21T21:42:23.753 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 32.893420519000003, 2024-09-21T21:42:23.753 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071794595000000003, 2024-09-21T21:42:23.753 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:23.753 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:23.753 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:23.753 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:23.753 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:23.753 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:23.753 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13568 2024-09-21T21:42:23.754 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.754 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:23.754 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.754 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:23.754 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825671+0000", 2024-09-21T21:42:23.754 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:23.754 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.754 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.754 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:23.754 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825671+0000", 2024-09-21T21:42:23.754 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:23.754 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.754 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.754 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:23.755 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825672+0000", 2024-09-21T21:42:23.755 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2999999999999999e-07 2024-09-21T21:42:23.755 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.755 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.755 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:23.755 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825673+0000", 2024-09-21T21:42:23.755 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.246e-06 2024-09-21T21:42:23.755 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.755 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.755 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:23.755 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825693+0000", 2024-09-21T21:42:23.755 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9789999999999999e-05 2024-09-21T21:42:23.755 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.755 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.756 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:23.756 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825697+0000", 2024-09-21T21:42:23.756 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2379999999999999e-06 2024-09-21T21:42:23.756 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.756 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.756 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:23.756 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897284+0000", 2024-09-21T21:42:23.756 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071586710999999997 2024-09-21T21:42:23.756 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.756 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.756 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:23.756 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897337+0000", 2024-09-21T21:42:23.756 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2800000000000003e-05 2024-09-21T21:42:23.756 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.756 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.757 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:23.757 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897466+0000", 2024-09-21T21:42:23.757 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00012957999999999999 2024-09-21T21:42:23.757 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.757 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:23.757 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.757 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.757 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.757 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13574 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 352256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:23.757 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.825702+0000", 2024-09-21T21:42:23.757 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 32.893390371000002, 2024-09-21T21:42:23.757 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072080377000000001, 2024-09-21T21:42:23.757 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:23.757 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:23.758 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:23.758 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:23.758 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:23.758 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:23.758 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13574 2024-09-21T21:42:23.758 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.758 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:23.758 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.758 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:23.758 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825702+0000", 2024-09-21T21:42:23.758 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:23.758 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.758 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.758 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:23.759 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825702+0000", 2024-09-21T21:42:23.759 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:23.759 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.759 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.759 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:23.759 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825943+0000", 2024-09-21T21:42:23.759 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00024105200000000001 2024-09-21T21:42:23.759 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.759 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.759 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:23.759 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825946+0000", 2024-09-21T21:42:23.759 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.743e-06 2024-09-21T21:42:23.759 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.759 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.759 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:23.760 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825956+0000", 2024-09-21T21:42:23.760 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6609999999999995e-06 2024-09-21T21:42:23.760 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.760 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.760 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:23.760 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825963+0000", 2024-09-21T21:42:23.760 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4609999999999996e-06 2024-09-21T21:42:23.760 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.760 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.760 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:23.760 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897502+0000", 2024-09-21T21:42:23.760 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071539354999999999 2024-09-21T21:42:23.760 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.760 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.761 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:23.761 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897619+0000", 2024-09-21T21:42:23.761 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011665 2024-09-21T21:42:23.761 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.761 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.761 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:23.761 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897782+0000", 2024-09-21T21:42:23.761 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00016345500000000001 2024-09-21T21:42:23.761 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.761 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:23.761 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.761 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.761 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.761 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13575 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 380928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:23.762 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.825973+0000", 2024-09-21T21:42:23.762 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 32.893118934999997, 2024-09-21T21:42:23.762 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072012723000000001, 2024-09-21T21:42:23.762 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:23.762 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:23.762 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:23.762 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:23.762 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:23.762 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:23.762 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13575 2024-09-21T21:42:23.762 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.762 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:23.762 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.762 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:23.762 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825973+0000", 2024-09-21T21:42:23.763 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:23.763 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.763 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.763 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:23.763 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825973+0000", 2024-09-21T21:42:23.763 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:23.763 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.763 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.763 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:23.763 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826746+0000", 2024-09-21T21:42:23.763 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00077326800000000002 2024-09-21T21:42:23.763 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.763 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.763 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:23.764 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826766+0000", 2024-09-21T21:42:23.764 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9465000000000001e-05 2024-09-21T21:42:23.764 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.764 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.764 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:23.764 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826812+0000", 2024-09-21T21:42:23.764 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5927999999999998e-05 2024-09-21T21:42:23.764 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.764 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.764 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:23.764 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826833+0000", 2024-09-21T21:42:23.764 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1710999999999999e-05 2024-09-21T21:42:23.764 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.764 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.765 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:23.765 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897809+0000", 2024-09-21T21:42:23.765 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.070975229000000001 2024-09-21T21:42:23.765 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.765 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.765 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:23.765 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897864+0000", 2024-09-21T21:42:23.765 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5334999999999998e-05 2024-09-21T21:42:23.765 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.765 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.765 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:23.765 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897986+0000", 2024-09-21T21:42:23.765 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000121787 2024-09-21T21:42:23.765 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.766 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:23.766 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.766 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.766 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.766 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13577 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 1814528~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:23.766 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.826874+0000", 2024-09-21T21:42:23.766 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 32.892217901000002, 2024-09-21T21:42:23.766 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071639123999999998, 2024-09-21T21:42:23.766 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:23.766 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:23.766 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:23.766 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:23.766 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:23.766 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:23.767 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13577 2024-09-21T21:42:23.767 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.767 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:23.767 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.767 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:23.767 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826874+0000", 2024-09-21T21:42:23.767 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:23.767 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.767 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.767 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:23.767 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826874+0000", 2024-09-21T21:42:23.767 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:23.767 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.768 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.768 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:23.768 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826875+0000", 2024-09-21T21:42:23.768 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.82e-07 2024-09-21T21:42:23.768 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.768 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.768 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:23.768 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826877+0000", 2024-09-21T21:42:23.768 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.295e-06 2024-09-21T21:42:23.768 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.768 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.768 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:23.768 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826883+0000", 2024-09-21T21:42:23.768 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2609999999999999e-06 2024-09-21T21:42:23.768 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.769 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.769 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:23.769 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826889+0000", 2024-09-21T21:42:23.769 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5589999999999996e-06 2024-09-21T21:42:23.769 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.769 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.769 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:23.769 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898301+0000", 2024-09-21T21:42:23.769 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071412188000000001 2024-09-21T21:42:23.769 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.769 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.769 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:23.769 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898416+0000", 2024-09-21T21:42:23.769 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000115538 2024-09-21T21:42:23.770 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.770 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.770 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:23.770 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898513+0000", 2024-09-21T21:42:23.770 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6800999999999997e-05 2024-09-21T21:42:23.770 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.770 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:23.770 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.770 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.770 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.770 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13578 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 2502656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:23.770 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.826896+0000", 2024-09-21T21:42:23.770 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 32.892196445000003, 2024-09-21T21:42:23.771 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071789411999999997, 2024-09-21T21:42:23.771 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:23.771 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:23.771 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:23.771 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:23.771 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:23.771 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:23.771 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13578 2024-09-21T21:42:23.771 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.771 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:23.771 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.771 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:23.771 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826896+0000", 2024-09-21T21:42:23.771 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:23.772 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.772 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.772 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:23.772 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826896+0000", 2024-09-21T21:42:23.772 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:23.772 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.772 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.772 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:23.772 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827327+0000", 2024-09-21T21:42:23.772 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00043094500000000002 2024-09-21T21:42:23.772 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.772 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.772 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:23.772 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827331+0000", 2024-09-21T21:42:23.772 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.0969999999999999e-06 2024-09-21T21:42:23.773 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.773 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.773 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:23.773 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827341+0000", 2024-09-21T21:42:23.773 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0199e-05 2024-09-21T21:42:23.773 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.773 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.773 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:23.773 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827346+0000", 2024-09-21T21:42:23.773 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3210000000000004e-06 2024-09-21T21:42:23.773 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.773 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.773 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:23.773 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898540+0000", 2024-09-21T21:42:23.774 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071194246000000003 2024-09-21T21:42:23.774 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.774 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.774 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:23.774 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898598+0000", 2024-09-21T21:42:23.774 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7658999999999998e-05 2024-09-21T21:42:23.774 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.774 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.774 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:23.774 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898685+0000", 2024-09-21T21:42:23.774 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.6945000000000002e-05 2024-09-21T21:42:23.774 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.774 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:23.774 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.775 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.775 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.775 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13579 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 2531328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:23.775 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.827355+0000", 2024-09-21T21:42:23.775 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 32.891736913999999, 2024-09-21T21:42:23.775 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071650933, 2024-09-21T21:42:23.775 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:23.775 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:23.775 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:23.775 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:23.775 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:23.775 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:23.775 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13579 2024-09-21T21:42:23.775 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.776 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:23.776 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.776 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:23.776 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827355+0000", 2024-09-21T21:42:23.776 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:23.776 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.776 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.776 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:23.776 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827355+0000", 2024-09-21T21:42:23.776 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:23.776 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.776 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.776 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:23.776 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827486+0000", 2024-09-21T21:42:23.776 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00013066799999999999 2024-09-21T21:42:23.777 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.777 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.777 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:23.777 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827490+0000", 2024-09-21T21:42:23.777 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.9009999999999996e-06 2024-09-21T21:42:23.777 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.777 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.777 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:23.777 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827499+0000", 2024-09-21T21:42:23.777 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6209999999999999e-06 2024-09-21T21:42:23.777 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.777 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.777 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:23.777 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827506+0000", 2024-09-21T21:42:23.777 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6490000000000002e-06 2024-09-21T21:42:23.778 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.778 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.778 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:23.778 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898713+0000", 2024-09-21T21:42:23.778 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071206937999999997 2024-09-21T21:42:23.778 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.778 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.778 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:23.778 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898811+0000", 2024-09-21T21:42:23.778 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.8159999999999995e-05 2024-09-21T21:42:23.778 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.778 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.778 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:23.778 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899006+0000", 2024-09-21T21:42:23.779 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00019499599999999999 2024-09-21T21:42:23.779 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.779 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:23.779 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.779 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.779 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.779 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13580 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3219456~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:23.779 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.827522+0000", 2024-09-21T21:42:23.779 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 32.891570483000002, 2024-09-21T21:42:23.779 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071813298999999997, 2024-09-21T21:42:23.779 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:23.779 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:23.779 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:23.779 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:23.779 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:23.780 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:23.780 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13580 2024-09-21T21:42:23.780 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.780 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:23.780 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.780 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:23.780 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827522+0000", 2024-09-21T21:42:23.780 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:23.780 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.780 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.780 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:23.780 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827522+0000", 2024-09-21T21:42:23.780 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:23.780 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.781 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.781 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:23.781 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827898+0000", 2024-09-21T21:42:23.781 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000376876 2024-09-21T21:42:23.781 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.781 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.781 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:23.781 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827903+0000", 2024-09-21T21:42:23.781 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4129999999999999e-06 2024-09-21T21:42:23.781 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.781 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.781 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:23.781 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827911+0000", 2024-09-21T21:42:23.782 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.4540000000000007e-06 2024-09-21T21:42:23.782 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.782 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.782 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:23.782 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827917+0000", 2024-09-21T21:42:23.782 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2110000000000003e-06 2024-09-21T21:42:23.782 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.782 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.782 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:23.782 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899041+0000", 2024-09-21T21:42:23.782 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071123965999999997 2024-09-21T21:42:23.782 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.782 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.782 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:23.783 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899157+0000", 2024-09-21T21:42:23.783 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011573700000000001 2024-09-21T21:42:23.783 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.783 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.783 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:23.783 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899335+0000", 2024-09-21T21:42:23.783 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00017764199999999999 2024-09-21T21:42:23.783 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.783 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:23.783 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.783 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.783 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.783 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13581 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3248128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:23.783 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.827929+0000", 2024-09-21T21:42:23.784 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 32.891163083999999, 2024-09-21T21:42:23.784 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071730151000000006, 2024-09-21T21:42:23.784 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:23.784 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:23.784 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:23.784 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:23.784 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:23.784 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:23.784 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13581 2024-09-21T21:42:23.784 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.784 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:23.784 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.784 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:23.784 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827929+0000", 2024-09-21T21:42:23.785 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:23.785 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.785 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.785 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:23.785 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827929+0000", 2024-09-21T21:42:23.785 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:23.785 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.785 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.785 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:23.785 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.829943+0000", 2024-09-21T21:42:23.785 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0020143750000000001 2024-09-21T21:42:23.785 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.785 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.785 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:23.785 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.829948+0000", 2024-09-21T21:42:23.786 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9400000000000001e-06 2024-09-21T21:42:23.786 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.786 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.786 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:23.786 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.829960+0000", 2024-09-21T21:42:23.786 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.2123999999999999e-05 2024-09-21T21:42:23.786 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.786 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.786 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:23.786 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.829968+0000", 2024-09-21T21:42:23.786 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7000000000000008e-06 2024-09-21T21:42:23.786 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.786 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.786 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:23.787 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899370+0000", 2024-09-21T21:42:23.787 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.069401672999999997 2024-09-21T21:42:23.787 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.787 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.787 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:23.787 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899481+0000", 2024-09-21T21:42:23.787 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011145300000000001 2024-09-21T21:42:23.787 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.787 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.787 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:23.787 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899659+0000", 2024-09-21T21:42:23.787 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00017788600000000001 2024-09-21T21:42:23.787 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.787 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:23.788 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.788 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.788 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.788 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13595 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 1871872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:23.788 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.830180+0000", 2024-09-21T21:42:23.788 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 32.888911970000002, 2024-09-21T21:42:23.788 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071809244999999994, 2024-09-21T21:42:23.788 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:23.788 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:23.788 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:23.788 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:23.788 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:23.788 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:23.788 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13595 2024-09-21T21:42:23.789 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.789 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:23.789 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.789 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:23.789 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.830180+0000", 2024-09-21T21:42:23.789 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:23.789 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.789 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.789 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:23.789 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.830180+0000", 2024-09-21T21:42:23.789 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:23.789 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.789 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.789 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:23.790 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.831967+0000", 2024-09-21T21:42:23.790 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0017868350000000001 2024-09-21T21:42:23.790 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.790 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.790 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:23.790 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.831971+0000", 2024-09-21T21:42:23.790 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3950000000000003e-06 2024-09-21T21:42:23.790 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.790 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.790 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:23.790 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.831983+0000", 2024-09-21T21:42:23.790 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1815e-05 2024-09-21T21:42:23.790 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.790 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.791 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:23.791 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.831993+0000", 2024-09-21T21:42:23.791 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6339999999999996e-06 2024-09-21T21:42:23.791 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.791 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.791 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:23.791 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.901693+0000", 2024-09-21T21:42:23.791 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.069700082999999996 2024-09-21T21:42:23.791 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.791 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.791 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:23.791 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.901813+0000", 2024-09-21T21:42:23.791 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011971 2024-09-21T21:42:23.791 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.792 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.792 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:23.792 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.901989+0000", 2024-09-21T21:42:23.792 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000176773 2024-09-21T21:42:23.792 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.792 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:23.792 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.792 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.792 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.792 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13601 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3334144~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:23.792 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.832130+0000", 2024-09-21T21:42:23.792 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 32.886962097999998, 2024-09-21T21:42:23.792 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071861727, 2024-09-21T21:42:23.792 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:23.793 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:23.793 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:23.793 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:23.793 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:23.793 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:23.793 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13601 2024-09-21T21:42:23.793 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.793 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:23.793 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.793 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:23.793 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.832130+0000", 2024-09-21T21:42:23.793 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:23.793 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.793 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.793 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:23.794 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.832130+0000", 2024-09-21T21:42:23.794 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:23.794 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.794 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.794 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:23.794 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.833939+0000", 2024-09-21T21:42:23.794 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0018094400000000001 2024-09-21T21:42:23.794 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.794 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.794 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:23.794 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.833944+0000", 2024-09-21T21:42:23.794 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5979999999999999e-06 2024-09-21T21:42:23.794 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.794 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.795 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:23.795 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.833953+0000", 2024-09-21T21:42:23.795 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.1460000000000002e-06 2024-09-21T21:42:23.795 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.795 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.795 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:23.795 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.833960+0000", 2024-09-21T21:42:23.795 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.7989999999999996e-06 2024-09-21T21:42:23.795 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.795 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.795 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:23.795 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.903693+0000", 2024-09-21T21:42:23.795 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.069732901 2024-09-21T21:42:23.795 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.795 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.796 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:23.796 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.903814+0000", 2024-09-21T21:42:23.796 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000121585 2024-09-21T21:42:23.796 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.796 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.796 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:23.796 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.903992+0000", 2024-09-21T21:42:23.796 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00017725799999999999 2024-09-21T21:42:23.796 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.796 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:23.796 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.796 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.796 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.796 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:12342 3.1 3:87417293:::rbd_data.1121faebdfe3.0000000000000004:head [read 4096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:42:23.797 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:14.411725+0000", 2024-09-21T21:42:23.797 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.3073670320000002, 2024-09-21T21:42:23.797 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0089213029999999, 2024-09-21T21:42:23.797 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:23.797 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:23.797 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:23.797 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:23.797 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:23.797 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:23.797 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 12342 2024-09-21T21:42:23.797 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.797 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:23.797 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.797 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:23.797 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411725+0000", 2024-09-21T21:42:23.798 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:23.798 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.798 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.798 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:23.798 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411725+0000", 2024-09-21T21:42:23.798 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:23.798 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.798 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.798 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:23.798 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411726+0000", 2024-09-21T21:42:23.798 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.4169999999999999e-06 2024-09-21T21:42:23.798 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.798 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.798 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:23.799 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411734+0000", 2024-09-21T21:42:23.799 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.3799999999999996e-06 2024-09-21T21:42:23.799 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.799 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.799 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:23.799 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411750+0000", 2024-09-21T21:42:23.799 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.6429000000000001e-05 2024-09-21T21:42:23.799 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.799 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.799 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:23.799 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411760+0000", 2024-09-21T21:42:23.799 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.4239999999999999e-06 2024-09-21T21:42:23.799 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.799 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.800 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:23.800 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411828+0000", 2024-09-21T21:42:23.800 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.7935999999999995e-05 2024-09-21T21:42:23.800 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.800 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.800 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op must wait for map", 2024-09-21T21:42:23.800 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411837+0000", 2024-09-21T21:42:23.800 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.0100000000000001e-06 2024-09-21T21:42:23.800 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.800 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.800 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:23.800 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.414984+0000", 2024-09-21T21:42:23.800 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0031472050000000001 2024-09-21T21:42:23.800 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.801 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.801 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for peered", 2024-09-21T21:42:23.801 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.414994+0000", 2024-09-21T21:42:23.801 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0291e-05 2024-09-21T21:42:23.801 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.801 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.801 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:23.801 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.420353+0000", 2024-09-21T21:42:23.801 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.005358913 2024-09-21T21:42:23.801 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.801 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.801 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:23.801 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.420489+0000", 2024-09-21T21:42:23.801 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00013592699999999999 2024-09-21T21:42:23.801 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.802 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.802 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:23.802 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.420646+0000", 2024-09-21T21:42:23.802 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00015737100000000001 2024-09-21T21:42:23.802 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.802 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:23.802 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.802 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.802 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.802 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:12348 3.1 3:87417293:::rbd_data.1121faebdfe3.0000000000000004:head [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:42:23.802 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:14.411793+0000", 2024-09-21T21:42:23.802 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.3072994859999998, 2024-09-21T21:42:23.802 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.009298384, 2024-09-21T21:42:23.802 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:23.803 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:23.803 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:23.803 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:23.803 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:23.803 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:23.803 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 12348 2024-09-21T21:42:23.803 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.803 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:23.803 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.803 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:23.803 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411793+0000", 2024-09-21T21:42:23.803 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:23.803 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.804 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.804 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:23.804 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411793+0000", 2024-09-21T21:42:23.804 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:23.804 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.804 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.804 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:23.804 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411793+0000", 2024-09-21T21:42:23.804 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0999999999999998e-07 2024-09-21T21:42:23.804 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.804 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.804 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:23.804 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411797+0000", 2024-09-21T21:42:23.804 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.49e-06 2024-09-21T21:42:23.805 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.805 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.805 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:23.805 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411809+0000", 2024-09-21T21:42:23.805 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.2510999999999999e-05 2024-09-21T21:42:23.805 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.805 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.805 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:23.805 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411816+0000", 2024-09-21T21:42:23.805 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6420000000000001e-06 2024-09-21T21:42:23.805 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.805 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.805 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:23.805 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411963+0000", 2024-09-21T21:42:23.805 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00014700999999999999 2024-09-21T21:42:23.806 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.806 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.806 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting_for_map not empty", 2024-09-21T21:42:23.806 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411970+0000", 2024-09-21T21:42:23.806 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9800000000000001e-06 2024-09-21T21:42:23.806 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.806 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.806 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:23.806 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.415018+0000", 2024-09-21T21:42:23.806 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0030482299999999999 2024-09-21T21:42:23.806 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.806 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.806 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for peered", 2024-09-21T21:42:23.806 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.415025+0000", 2024-09-21T21:42:23.807 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.3749999999999997e-06 2024-09-21T21:42:23.807 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.807 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.807 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:23.807 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.420904+0000", 2024-09-21T21:42:23.807 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0058787339999999 2024-09-21T21:42:23.807 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.807 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.807 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:23.807 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.420988+0000", 2024-09-21T21:42:23.807 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.3955999999999994e-05 2024-09-21T21:42:23.807 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.807 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.808 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:23.808 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.421091+0000", 2024-09-21T21:42:23.808 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000102846 2024-09-21T21:42:23.808 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.808 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:23.808 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.808 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.808 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.808 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4385.0:20922 3.9 3:920c5256:::rbd_id.image_client.0-clone36:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-21T21:42:23.808 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:17.811324+0000", 2024-09-21T21:42:23.808 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 5.907767754, 2024-09-21T21:42:23.808 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.66951882299999999, 2024-09-21T21:42:23.808 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:23.808 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:23.809 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:23.809 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:23.809 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4385", 2024-09-21T21:42:23.809 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:42:23.809 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 20922 2024-09-21T21:42:23.809 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.809 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:23.809 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.809 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:23.809 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:17.811324+0000", 2024-09-21T21:42:23.809 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:23.809 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.809 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.810 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:23.810 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:17.811324+0000", 2024-09-21T21:42:23.810 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:23.810 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.810 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.810 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:23.810 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:17.811326+0000", 2024-09-21T21:42:23.810 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.536e-06 2024-09-21T21:42:23.810 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.810 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.810 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:23.810 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:17.811332+0000", 2024-09-21T21:42:23.810 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6699999999999997e-06 2024-09-21T21:42:23.810 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.810 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.811 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:23.811 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:17.811347+0000", 2024-09-21T21:42:23.811 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.4589999999999999e-05 2024-09-21T21:42:23.811 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.811 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.811 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:23.811 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:17.811359+0000", 2024-09-21T21:42:23.811 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1776e-05 2024-09-21T21:42:23.811 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.811 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.811 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:23.811 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:17.811424+0000", 2024-09-21T21:42:23.811 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4795000000000006e-05 2024-09-21T21:42:23.811 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.812 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.812 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for peered", 2024-09-21T21:42:23.812 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:17.811436+0000", 2024-09-21T21:42:23.812 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.221e-05 2024-09-21T21:42:23.812 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.812 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.812 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:23.812 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:18.480657+0000", 2024-09-21T21:42:23.812 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.66922088999999996 2024-09-21T21:42:23.812 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.812 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.812 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:23.812 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:18.480760+0000", 2024-09-21T21:42:23.812 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000102827 2024-09-21T21:42:23.813 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:23.813 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:23.813 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:23.813 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:18.480843+0000", 2024-09-21T21:42:23.813 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.3529000000000006e-05 2024-09-21T21:42:23.813 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.813 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:23.813 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.813 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:23.813 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:23.813 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:23.815 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T21:42:23.815 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:23.813+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:23.915 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:23.914+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:23.919 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:23.919 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:42:23.919 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-21T21:42:23.919 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:23.927 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T21:42:24.016 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:24.015+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:24.071 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:24.071 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:42:24.071 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:42:24.071 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:42:24.072 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:24.079 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T21:42:24.123 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:24.122+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:24.183 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:24.183 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-21T21:42:24.183 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-21T21:42:24.183 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:42:24.183 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.184 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19747 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 7905280~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:24.184 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273058+0000", 2024-09-21T21:42:24.184 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 32.907831409000003, 2024-09-21T21:42:24.184 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97338591399999996, 2024-09-21T21:42:24.184 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:24.184 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:24.184 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:24.184 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:24.184 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:24.184 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:24.184 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19747 2024-09-21T21:42:24.184 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.184 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:24.184 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.185 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:24.185 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273058+0000", 2024-09-21T21:42:24.185 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:24.185 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.185 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.185 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:24.185 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273058+0000", 2024-09-21T21:42:24.185 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:24.185 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.185 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.185 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:24.185 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273058+0000", 2024-09-21T21:42:24.185 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5900000000000003e-07 2024-09-21T21:42:24.185 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.186 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.186 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:24.186 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273061+0000", 2024-09-21T21:42:24.186 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3479999999999998e-06 2024-09-21T21:42:24.186 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.186 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.186 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:24.186 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273067+0000", 2024-09-21T21:42:24.186 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1469999999999998e-06 2024-09-21T21:42:24.186 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.186 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.186 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:24.186 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273071+0000", 2024-09-21T21:42:24.186 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7339999999999999e-06 2024-09-21T21:42:24.187 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.187 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.187 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:24.187 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277945+0000", 2024-09-21T21:42:24.187 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.004873474 2024-09-21T21:42:24.187 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.187 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.187 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:24.187 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277948+0000", 2024-09-21T21:42:24.187 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0299999999999998e-06 2024-09-21T21:42:24.187 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.187 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.187 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:24.187 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246269+0000", 2024-09-21T21:42:24.188 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96832060200000003 2024-09-21T21:42:24.188 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.188 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.188 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:24.188 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246362+0000", 2024-09-21T21:42:24.188 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3867000000000001e-05 2024-09-21T21:42:24.188 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.188 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.188 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:24.188 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246444+0000", 2024-09-21T21:42:24.189 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1353000000000005e-05 2024-09-21T21:42:24.189 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:24.189 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:24.189 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:24.189 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.189 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.189 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19755 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 5783552~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:24.189 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273077+0000", 2024-09-21T21:42:24.189 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 32.907811793, 2024-09-21T21:42:24.189 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97350203400000002, 2024-09-21T21:42:24.189 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:24.189 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:24.190 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:24.190 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:24.190 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:24.190 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:24.190 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19755 2024-09-21T21:42:24.190 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.190 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:24.190 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.190 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:24.190 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273077+0000", 2024-09-21T21:42:24.190 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:24.190 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.190 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.190 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:24.191 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273077+0000", 2024-09-21T21:42:24.191 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:24.191 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.191 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.191 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:24.191 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273078+0000", 2024-09-21T21:42:24.191 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5699999999999998e-07 2024-09-21T21:42:24.191 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.193 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.193 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:24.193 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273083+0000", 2024-09-21T21:42:24.193 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8670000000000002e-06 2024-09-21T21:42:24.194 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.194 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.194 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:24.194 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273088+0000", 2024-09-21T21:42:24.194 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8370000000000002e-06 2024-09-21T21:42:24.194 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.194 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.194 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:24.194 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273093+0000", 2024-09-21T21:42:24.194 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5709999999999999e-06 2024-09-21T21:42:24.194 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.194 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.194 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:24.194 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277968+0000", 2024-09-21T21:42:24.195 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048753909999999998 2024-09-21T21:42:24.195 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.195 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.195 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:24.195 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277971+0000", 2024-09-21T21:42:24.195 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0189999999999998e-06 2024-09-21T21:42:24.195 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.195 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.195 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:24.195 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246485+0000", 2024-09-21T21:42:24.195 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96851399299999996 2024-09-21T21:42:24.195 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.195 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.195 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:24.196 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246536+0000", 2024-09-21T21:42:24.196 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0430000000000003e-05 2024-09-21T21:42:24.196 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.196 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.196 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:24.196 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246579+0000", 2024-09-21T21:42:24.196 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3569000000000002e-05 2024-09-21T21:42:24.196 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:24.196 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:24.196 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:24.196 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.196 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.196 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19756 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6500352~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:24.196 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273099+0000", 2024-09-21T21:42:24.197 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 32.907790419999998, 2024-09-21T21:42:24.197 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97359272799999996, 2024-09-21T21:42:24.197 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:24.197 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:24.197 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:24.197 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:24.197 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:24.197 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:24.197 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19756 2024-09-21T21:42:24.197 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.197 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:24.197 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.197 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:24.197 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273099+0000", 2024-09-21T21:42:24.197 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:24.198 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.198 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.198 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:24.198 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273099+0000", 2024-09-21T21:42:24.198 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:24.198 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.198 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.198 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:24.198 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273099+0000", 2024-09-21T21:42:24.198 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0900000000000002e-07 2024-09-21T21:42:24.198 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.198 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.198 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:24.198 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273101+0000", 2024-09-21T21:42:24.199 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0320000000000002e-06 2024-09-21T21:42:24.199 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.199 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.199 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:24.199 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273107+0000", 2024-09-21T21:42:24.199 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2830000000000001e-06 2024-09-21T21:42:24.199 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.199 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.199 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:24.199 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273111+0000", 2024-09-21T21:42:24.199 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2230000000000001e-06 2024-09-21T21:42:24.199 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.199 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.199 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:24.200 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277994+0000", 2024-09-21T21:42:24.200 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048837070000000002 2024-09-21T21:42:24.200 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.200 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.200 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:24.200 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278000+0000", 2024-09-21T21:42:24.200 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3149999999999999e-06 2024-09-21T21:42:24.200 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.200 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.200 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:24.200 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246607+0000", 2024-09-21T21:42:24.200 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.968606839 2024-09-21T21:42:24.200 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.200 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.201 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:24.201 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246651+0000", 2024-09-21T21:42:24.201 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4734999999999998e-05 2024-09-21T21:42:24.201 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.201 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.201 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:24.201 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246692+0000", 2024-09-21T21:42:24.201 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0185000000000002e-05 2024-09-21T21:42:24.201 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:24.201 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:24.201 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:24.202 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.202 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.202 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19757 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 5812224~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:24.202 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273116+0000", 2024-09-21T21:42:24.202 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 32.907772762999997, 2024-09-21T21:42:24.202 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97367813999999997, 2024-09-21T21:42:24.202 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:24.202 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:24.202 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:24.202 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:24.202 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:24.202 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:24.202 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19757 2024-09-21T21:42:24.203 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.203 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:24.203 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.203 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:24.203 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273116+0000", 2024-09-21T21:42:24.203 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:24.203 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.203 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.203 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:24.203 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273116+0000", 2024-09-21T21:42:24.203 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:24.203 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.203 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.203 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:24.203 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273117+0000", 2024-09-21T21:42:24.204 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0999999999999999e-07 2024-09-21T21:42:24.204 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.204 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.204 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:24.204 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273119+0000", 2024-09-21T21:42:24.204 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9970000000000001e-06 2024-09-21T21:42:24.204 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.204 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.204 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:24.204 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273125+0000", 2024-09-21T21:42:24.204 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9540000000000001e-06 2024-09-21T21:42:24.204 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.204 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.204 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:24.205 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273135+0000", 2024-09-21T21:42:24.205 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0597999999999999e-05 2024-09-21T21:42:24.205 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.205 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.205 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:24.205 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278019+0000", 2024-09-21T21:42:24.205 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048834849999999999 2024-09-21T21:42:24.205 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.205 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.205 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:24.205 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278022+0000", 2024-09-21T21:42:24.205 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9560000000000002e-06 2024-09-21T21:42:24.205 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.205 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.206 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:24.206 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246716+0000", 2024-09-21T21:42:24.206 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.968694202 2024-09-21T21:42:24.206 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.206 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.206 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:24.206 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246758+0000", 2024-09-21T21:42:24.206 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1607000000000003e-05 2024-09-21T21:42:24.206 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.206 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.206 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:24.206 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246795+0000", 2024-09-21T21:42:24.206 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6931000000000003e-05 2024-09-21T21:42:24.206 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:24.207 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:24.207 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:24.207 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.207 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.207 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19758 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6529024~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:24.207 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273142+0000", 2024-09-21T21:42:24.207 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 32.907747542000003, 2024-09-21T21:42:24.207 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97375924599999997, 2024-09-21T21:42:24.207 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:24.207 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:24.207 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:24.207 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:24.207 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:24.207 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:24.208 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19758 2024-09-21T21:42:24.208 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.208 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:24.208 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.208 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:24.208 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273142+0000", 2024-09-21T21:42:24.208 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:24.208 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.208 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.208 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:24.208 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273142+0000", 2024-09-21T21:42:24.208 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:24.208 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.208 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.208 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:24.209 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273142+0000", 2024-09-21T21:42:24.209 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0200000000000003e-07 2024-09-21T21:42:24.209 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.209 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.209 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:24.209 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273144+0000", 2024-09-21T21:42:24.209 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1390000000000002e-06 2024-09-21T21:42:24.209 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.209 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.209 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:24.209 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273150+0000", 2024-09-21T21:42:24.209 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6409999999999999e-06 2024-09-21T21:42:24.209 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.209 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.210 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:24.210 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273157+0000", 2024-09-21T21:42:24.210 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1359999999999999e-06 2024-09-21T21:42:24.210 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.210 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.210 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:24.210 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278050+0000", 2024-09-21T21:42:24.210 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048926170000000001 2024-09-21T21:42:24.210 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.210 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.210 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:24.210 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278053+0000", 2024-09-21T21:42:24.210 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4879999999999999e-06 2024-09-21T21:42:24.210 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.210 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.211 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:24.211 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246819+0000", 2024-09-21T21:42:24.211 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96876590399999996 2024-09-21T21:42:24.211 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.211 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.211 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:24.211 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246856+0000", 2024-09-21T21:42:24.211 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7287999999999999e-05 2024-09-21T21:42:24.211 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.211 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.211 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:24.211 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246901+0000", 2024-09-21T21:42:24.211 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4631e-05 2024-09-21T21:42:24.211 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:24.212 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:24.212 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:24.212 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.212 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.212 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19759 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 7933952~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:24.212 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273172+0000", 2024-09-21T21:42:24.212 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 32.907717278, 2024-09-21T21:42:24.212 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97384789500000002, 2024-09-21T21:42:24.212 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:24.212 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:24.212 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:24.212 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:24.212 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:24.212 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:24.212 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19759 2024-09-21T21:42:24.213 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.213 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:24.213 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.213 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:24.213 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273172+0000", 2024-09-21T21:42:24.213 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:24.213 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.213 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.213 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:24.213 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273172+0000", 2024-09-21T21:42:24.213 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:24.213 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.213 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.213 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:24.214 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273172+0000", 2024-09-21T21:42:24.214 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6100000000000002e-07 2024-09-21T21:42:24.214 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.214 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.214 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:24.214 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273174+0000", 2024-09-21T21:42:24.214 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.691e-06 2024-09-21T21:42:24.214 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.214 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.214 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:24.214 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273178+0000", 2024-09-21T21:42:24.214 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7579999999999998e-06 2024-09-21T21:42:24.214 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.214 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.215 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:24.215 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273180+0000", 2024-09-21T21:42:24.215 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7199999999999998e-06 2024-09-21T21:42:24.215 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.215 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.215 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:24.215 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278077+0000", 2024-09-21T21:42:24.215 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048969529999999999 2024-09-21T21:42:24.215 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.215 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.215 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:24.215 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278080+0000", 2024-09-21T21:42:24.215 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.066e-06 2024-09-21T21:42:24.215 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.215 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.216 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:24.216 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246925+0000", 2024-09-21T21:42:24.216 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96884490199999995 2024-09-21T21:42:24.216 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.216 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.216 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:24.216 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246974+0000", 2024-09-21T21:42:24.216 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8387999999999998e-05 2024-09-21T21:42:24.216 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.216 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.216 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:24.216 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247020+0000", 2024-09-21T21:42:24.216 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6156000000000003e-05 2024-09-21T21:42:24.216 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:24.217 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:24.217 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:24.217 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.217 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.217 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19760 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 7991296~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:24.217 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273220+0000", 2024-09-21T21:42:24.217 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 32.907668764, 2024-09-21T21:42:24.217 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97403125499999998, 2024-09-21T21:42:24.217 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:24.217 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:24.217 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:24.217 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:24.217 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:24.217 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:24.217 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19760 2024-09-21T21:42:24.218 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.218 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:24.218 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.218 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:24.218 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273220+0000", 2024-09-21T21:42:24.218 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:24.218 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.218 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.218 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:24.218 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273220+0000", 2024-09-21T21:42:24.218 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:24.218 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.218 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.218 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:24.219 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273221+0000", 2024-09-21T21:42:24.219 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.65e-07 2024-09-21T21:42:24.219 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.219 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.219 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:24.219 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273228+0000", 2024-09-21T21:42:24.219 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6040000000000004e-06 2024-09-21T21:42:24.219 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.219 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.219 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:24.219 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273233+0000", 2024-09-21T21:42:24.219 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1569999999999999e-06 2024-09-21T21:42:24.219 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.219 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.219 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:24.220 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273238+0000", 2024-09-21T21:42:24.220 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0050000000000004e-06 2024-09-21T21:42:24.220 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.220 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.220 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:24.220 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278107+0000", 2024-09-21T21:42:24.220 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00486857 2024-09-21T21:42:24.220 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.220 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.220 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:24.220 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278111+0000", 2024-09-21T21:42:24.220 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6389999999999998e-06 2024-09-21T21:42:24.220 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.220 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.220 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:24.221 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247069+0000", 2024-09-21T21:42:24.221 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96895796899999997 2024-09-21T21:42:24.221 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.221 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.221 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:24.221 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247166+0000", 2024-09-21T21:42:24.221 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.7158999999999996e-05 2024-09-21T21:42:24.221 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.221 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.221 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:24.221 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247252+0000", 2024-09-21T21:42:24.221 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5886999999999999e-05 2024-09-21T21:42:24.221 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:24.221 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:24.222 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:24.222 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.222 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.222 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19761 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 7962624~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:24.222 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273246+0000", 2024-09-21T21:42:24.222 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 32.907643704000002, 2024-09-21T21:42:24.222 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97415972200000001, 2024-09-21T21:42:24.222 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:24.222 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:24.222 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:24.222 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:24.222 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:24.222 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:24.222 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19761 2024-09-21T21:42:24.223 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.223 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:24.223 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.223 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:24.223 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273246+0000", 2024-09-21T21:42:24.223 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:24.223 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.223 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.223 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:24.223 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273246+0000", 2024-09-21T21:42:24.223 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:24.223 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.223 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.223 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:24.224 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273246+0000", 2024-09-21T21:42:24.224 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5100000000000001e-07 2024-09-21T21:42:24.224 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.224 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.224 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:24.224 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273248+0000", 2024-09-21T21:42:24.224 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1550000000000001e-06 2024-09-21T21:42:24.224 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.224 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.224 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:24.224 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273254+0000", 2024-09-21T21:42:24.224 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.964e-06 2024-09-21T21:42:24.224 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.224 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.224 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:24.225 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273259+0000", 2024-09-21T21:42:24.225 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5499999999999996e-06 2024-09-21T21:42:24.225 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.225 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.225 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:24.225 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278130+0000", 2024-09-21T21:42:24.225 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048715779999999997 2024-09-21T21:42:24.225 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.225 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.225 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:24.225 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278133+0000", 2024-09-21T21:42:24.225 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8739999999999999e-06 2024-09-21T21:42:24.225 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.225 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.225 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:24.226 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247270+0000", 2024-09-21T21:42:24.226 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.969137099 2024-09-21T21:42:24.226 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.226 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.226 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:24.226 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247338+0000", 2024-09-21T21:42:24.226 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7717000000000002e-05 2024-09-21T21:42:24.226 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.226 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.226 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:24.226 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247405+0000", 2024-09-21T21:42:24.226 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7434000000000005e-05 2024-09-21T21:42:24.226 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:24.226 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:24.227 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:24.227 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.227 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.227 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19769 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 5840896~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:24.227 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273287+0000", 2024-09-21T21:42:24.227 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 32.907602420000003, 2024-09-21T21:42:24.227 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97435390200000005, 2024-09-21T21:42:24.227 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:24.227 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:24.227 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:24.227 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:24.227 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:24.227 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:24.227 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19769 2024-09-21T21:42:24.228 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.228 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:24.228 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.228 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:24.228 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273287+0000", 2024-09-21T21:42:24.228 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:24.228 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.228 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.228 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:24.228 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273287+0000", 2024-09-21T21:42:24.228 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:24.228 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.228 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.228 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:24.229 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273287+0000", 2024-09-21T21:42:24.229 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6699999999999999e-07 2024-09-21T21:42:24.229 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.229 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.229 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:24.229 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273289+0000", 2024-09-21T21:42:24.229 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.708e-06 2024-09-21T21:42:24.229 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.229 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.229 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:24.229 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273296+0000", 2024-09-21T21:42:24.229 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8109999999999997e-06 2024-09-21T21:42:24.229 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.229 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.230 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:24.230 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273299+0000", 2024-09-21T21:42:24.230 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0740000000000001e-06 2024-09-21T21:42:24.230 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.230 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.230 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:24.230 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278159+0000", 2024-09-21T21:42:24.230 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048603240000000001 2024-09-21T21:42:24.230 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.230 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.230 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:24.230 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278162+0000", 2024-09-21T21:42:24.230 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1930000000000002e-06 2024-09-21T21:42:24.230 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.231 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.231 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:24.231 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247444+0000", 2024-09-21T21:42:24.231 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96928199100000001 2024-09-21T21:42:24.231 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.231 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.231 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:24.231 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247533+0000", 2024-09-21T21:42:24.231 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8490000000000001e-05 2024-09-21T21:42:24.231 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.231 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.231 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:24.231 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247641+0000", 2024-09-21T21:42:24.231 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000107944 2024-09-21T21:42:24.232 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:24.232 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:24.232 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:24.232 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.232 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.232 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19770 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 5869568~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:24.232 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273313+0000", 2024-09-21T21:42:24.232 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 32.907576317999997, 2024-09-21T21:42:24.232 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97449038899999996, 2024-09-21T21:42:24.232 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:24.232 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:24.232 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:24.232 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:24.232 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:24.232 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:24.233 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19770 2024-09-21T21:42:24.233 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.233 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:24.233 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.233 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:24.233 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273313+0000", 2024-09-21T21:42:24.233 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:24.233 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.233 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.233 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:24.233 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273313+0000", 2024-09-21T21:42:24.233 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:24.233 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.233 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.233 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:24.234 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273313+0000", 2024-09-21T21:42:24.234 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5699999999999998e-07 2024-09-21T21:42:24.234 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.234 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.234 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:24.234 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273315+0000", 2024-09-21T21:42:24.234 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0099999999999998e-06 2024-09-21T21:42:24.234 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.234 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.234 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:24.234 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273321+0000", 2024-09-21T21:42:24.234 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7529999999999998e-06 2024-09-21T21:42:24.234 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.234 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.235 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:24.235 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273328+0000", 2024-09-21T21:42:24.235 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9009999999999997e-06 2024-09-21T21:42:24.235 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.235 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.235 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:24.235 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278183+0000", 2024-09-21T21:42:24.235 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048547770000000002 2024-09-21T21:42:24.235 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.235 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.235 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:24.235 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278186+0000", 2024-09-21T21:42:24.235 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3100000000000001e-06 2024-09-21T21:42:24.235 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.235 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.236 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:24.236 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247676+0000", 2024-09-21T21:42:24.236 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96949004299999997 2024-09-21T21:42:24.236 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.236 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.236 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:24.236 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247757+0000", 2024-09-21T21:42:24.236 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0479e-05 2024-09-21T21:42:24.236 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.236 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.236 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:24.236 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247803+0000", 2024-09-21T21:42:24.236 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6758999999999997e-05 2024-09-21T21:42:24.236 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:24.236 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:24.237 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:24.237 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.237 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.237 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19771 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6586368~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:24.237 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273335+0000", 2024-09-21T21:42:24.237 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 32.907554531000002, 2024-09-21T21:42:24.237 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97458947699999998, 2024-09-21T21:42:24.237 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:24.237 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:24.237 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:24.237 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:24.237 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:24.237 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:24.237 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19771 2024-09-21T21:42:24.237 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.238 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:24.238 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.238 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:24.238 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273335+0000", 2024-09-21T21:42:24.238 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:24.238 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.238 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.238 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:24.238 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273335+0000", 2024-09-21T21:42:24.238 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:24.238 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.238 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.238 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:24.238 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273335+0000", 2024-09-21T21:42:24.239 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3599999999999999e-07 2024-09-21T21:42:24.239 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.239 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.239 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:24.239 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273337+0000", 2024-09-21T21:42:24.239 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0789999999999999e-06 2024-09-21T21:42:24.239 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.239 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.239 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:24.239 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273343+0000", 2024-09-21T21:42:24.239 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6280000000000002e-06 2024-09-21T21:42:24.239 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.239 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.239 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:24.239 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273346+0000", 2024-09-21T21:42:24.240 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.507e-06 2024-09-21T21:42:24.240 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.240 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.240 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:24.240 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278206+0000", 2024-09-21T21:42:24.240 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048600969999999999 2024-09-21T21:42:24.240 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.240 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.240 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:24.240 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278212+0000", 2024-09-21T21:42:24.240 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2079999999999999e-06 2024-09-21T21:42:24.240 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.240 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.240 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:24.241 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247828+0000", 2024-09-21T21:42:24.241 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.969616547 2024-09-21T21:42:24.241 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.241 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.241 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:24.241 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247881+0000", 2024-09-21T21:42:24.241 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3180999999999998e-05 2024-09-21T21:42:24.241 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.241 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.241 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:24.241 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247924+0000", 2024-09-21T21:42:24.241 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2894e-05 2024-09-21T21:42:24.241 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:24.241 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:24.242 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:24.242 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.242 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.242 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19772 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6557696~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:24.242 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273358+0000", 2024-09-21T21:42:24.242 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 32.907531151000001, 2024-09-21T21:42:24.242 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97467031800000004, 2024-09-21T21:42:24.242 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:24.242 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:24.242 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:24.242 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:24.242 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:24.242 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:24.243 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19772 2024-09-21T21:42:24.243 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.243 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:24.243 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.243 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:24.243 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273358+0000", 2024-09-21T21:42:24.243 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:24.243 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.243 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.243 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:24.243 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273358+0000", 2024-09-21T21:42:24.243 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:24.243 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.243 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.243 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:24.244 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273358+0000", 2024-09-21T21:42:24.244 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4499999999999998e-07 2024-09-21T21:42:24.244 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.244 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.244 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:24.244 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273360+0000", 2024-09-21T21:42:24.244 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9190000000000002e-06 2024-09-21T21:42:24.244 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.244 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.244 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:24.244 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273366+0000", 2024-09-21T21:42:24.244 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.891e-06 2024-09-21T21:42:24.244 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.244 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.244 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:24.244 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273371+0000", 2024-09-21T21:42:24.245 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1880000000000001e-06 2024-09-21T21:42:24.245 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.245 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.245 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:24.245 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278237+0000", 2024-09-21T21:42:24.245 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.004865472 2024-09-21T21:42:24.245 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.245 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.245 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:24.245 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278240+0000", 2024-09-21T21:42:24.245 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9550000000000001e-06 2024-09-21T21:42:24.245 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.245 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.246 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:24.246 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247949+0000", 2024-09-21T21:42:24.246 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.969709127 2024-09-21T21:42:24.246 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.246 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.246 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:24.246 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247990+0000", 2024-09-21T21:42:24.246 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1202000000000003e-05 2024-09-21T21:42:24.246 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.246 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.246 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:24.246 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248028+0000", 2024-09-21T21:42:24.246 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8219000000000001e-05 2024-09-21T21:42:24.246 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:24.247 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:24.247 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:24.247 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.247 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.247 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19773 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6615040~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:24.247 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273390+0000", 2024-09-21T21:42:24.247 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 32.907499289999997, 2024-09-21T21:42:24.247 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97476086200000001, 2024-09-21T21:42:24.247 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:24.247 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:24.247 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:24.247 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:24.247 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:24.247 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:24.248 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19773 2024-09-21T21:42:24.248 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.248 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:24.248 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.248 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:24.248 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273390+0000", 2024-09-21T21:42:24.248 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:24.248 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.248 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.248 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:24.248 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273390+0000", 2024-09-21T21:42:24.248 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:24.248 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.248 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.249 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:24.249 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273390+0000", 2024-09-21T21:42:24.249 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9999999999999999e-07 2024-09-21T21:42:24.249 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.249 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.249 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:24.249 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273392+0000", 2024-09-21T21:42:24.249 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0640000000000001e-06 2024-09-21T21:42:24.249 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.249 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.249 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:24.249 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273397+0000", 2024-09-21T21:42:24.249 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.113e-06 2024-09-21T21:42:24.249 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.249 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.250 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:24.250 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273401+0000", 2024-09-21T21:42:24.250 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3119999999999998e-06 2024-09-21T21:42:24.250 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.250 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.250 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:24.250 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278271+0000", 2024-09-21T21:42:24.250 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048698880000000002 2024-09-21T21:42:24.250 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.250 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.250 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:24.250 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278278+0000", 2024-09-21T21:42:24.250 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1210000000000001e-06 2024-09-21T21:42:24.251 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.251 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.251 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:24.251 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248060+0000", 2024-09-21T21:42:24.251 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96978224999999996 2024-09-21T21:42:24.251 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.251 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.251 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:24.251 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248111+0000", 2024-09-21T21:42:24.251 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.096e-05 2024-09-21T21:42:24.251 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.251 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.251 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:24.251 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248151+0000", 2024-09-21T21:42:24.251 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9854000000000002e-05 2024-09-21T21:42:24.252 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:24.252 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:24.252 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:24.252 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.252 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.252 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19784 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6643712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:24.252 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273405+0000", 2024-09-21T21:42:24.252 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 32.907484568999998, 2024-09-21T21:42:24.252 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97491798900000004, 2024-09-21T21:42:24.252 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:24.252 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:24.252 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:24.252 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:24.252 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:24.252 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:24.253 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19784 2024-09-21T21:42:24.253 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.253 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:24.253 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.253 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:24.253 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273405+0000", 2024-09-21T21:42:24.253 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:24.253 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.253 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.253 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:24.253 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273405+0000", 2024-09-21T21:42:24.253 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:24.253 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.253 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.254 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:24.254 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273405+0000", 2024-09-21T21:42:24.254 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2100000000000001e-07 2024-09-21T21:42:24.254 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.254 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.254 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:24.254 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273406+0000", 2024-09-21T21:42:24.254 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2049999999999999e-06 2024-09-21T21:42:24.254 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.254 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.254 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:24.254 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273409+0000", 2024-09-21T21:42:24.255 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0599999999999999e-06 2024-09-21T21:42:24.255 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.255 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.255 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:24.255 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273412+0000", 2024-09-21T21:42:24.255 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0730000000000001e-06 2024-09-21T21:42:24.255 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.255 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.255 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:24.255 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278317+0000", 2024-09-21T21:42:24.255 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0049045520000000004 2024-09-21T21:42:24.255 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.255 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.256 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:24.256 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278321+0000", 2024-09-21T21:42:24.256 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6249999999999998e-06 2024-09-21T21:42:24.256 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.256 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.256 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:24.256 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248179+0000", 2024-09-21T21:42:24.256 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96985744500000004 2024-09-21T21:42:24.256 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.256 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.256 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:24.256 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248253+0000", 2024-09-21T21:42:24.256 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3829000000000001e-05 2024-09-21T21:42:24.256 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.257 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.257 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:24.257 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248323+0000", 2024-09-21T21:42:24.257 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9979000000000002e-05 2024-09-21T21:42:24.257 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:24.257 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:24.257 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:24.257 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.257 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.257 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19785 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6672384~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:24.257 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273415+0000", 2024-09-21T21:42:24.257 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 32.907473983000003, 2024-09-21T21:42:24.257 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97500777800000005, 2024-09-21T21:42:24.257 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:24.258 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:24.258 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:24.258 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:24.258 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:24.258 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:24.258 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19785 2024-09-21T21:42:24.258 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.258 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:24.258 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.258 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:24.258 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273415+0000", 2024-09-21T21:42:24.258 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:24.258 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.258 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.259 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:24.259 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273415+0000", 2024-09-21T21:42:24.259 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:24.259 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.259 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.259 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:24.259 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273415+0000", 2024-09-21T21:42:24.259 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0100000000000001e-07 2024-09-21T21:42:24.259 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.259 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.259 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:24.259 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273417+0000", 2024-09-21T21:42:24.259 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.114e-06 2024-09-21T21:42:24.259 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.260 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.260 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:24.260 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273420+0000", 2024-09-21T21:42:24.260 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6770000000000001e-06 2024-09-21T21:42:24.260 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.260 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.260 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:24.260 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273423+0000", 2024-09-21T21:42:24.260 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5450000000000002e-06 2024-09-21T21:42:24.260 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.260 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.260 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:24.260 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278355+0000", 2024-09-21T21:42:24.260 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0049318590000000002 2024-09-21T21:42:24.261 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.261 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.261 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:24.261 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278360+0000", 2024-09-21T21:42:24.261 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3079999999999998e-06 2024-09-21T21:42:24.261 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.261 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.261 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:24.261 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248349+0000", 2024-09-21T21:42:24.261 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96998909899999997 2024-09-21T21:42:24.261 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.261 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.261 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:24.261 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248387+0000", 2024-09-21T21:42:24.261 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7657000000000002e-05 2024-09-21T21:42:24.262 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.262 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.262 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:24.262 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248423+0000", 2024-09-21T21:42:24.262 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6318e-05 2024-09-21T21:42:24.262 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:24.262 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:24.262 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:24.262 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.262 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.262 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19842 3.d 3:b7fa2ca0:::rbd_data.112157a479c.000000000000004d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:24.262 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273426+0000", 2024-09-21T21:42:24.262 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 32.907462930999998, 2024-09-21T21:42:24.263 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.98136526800000001, 2024-09-21T21:42:24.263 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:24.263 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:24.263 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:24.263 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:24.263 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:24.263 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:24.263 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19842 2024-09-21T21:42:24.263 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.263 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:24.263 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.263 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:24.263 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273426+0000", 2024-09-21T21:42:24.263 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:24.264 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.264 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.264 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:24.264 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273426+0000", 2024-09-21T21:42:24.264 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:24.264 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.264 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.264 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:24.264 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273427+0000", 2024-09-21T21:42:24.264 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.72e-07 2024-09-21T21:42:24.264 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.264 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.264 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:24.264 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275836+0000", 2024-09-21T21:42:24.265 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00240909 2024-09-21T21:42:24.265 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.265 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.265 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:24.265 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275844+0000", 2024-09-21T21:42:24.265 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3750000000000003e-06 2024-09-21T21:42:24.265 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.265 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.265 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:24.265 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275849+0000", 2024-09-21T21:42:24.265 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0100000000000003e-06 2024-09-21T21:42:24.265 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.265 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.265 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:24.266 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278394+0000", 2024-09-21T21:42:24.266 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025449890000000001 2024-09-21T21:42:24.266 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.266 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.266 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:24.266 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278402+0000", 2024-09-21T21:42:24.266 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6359999999999994e-06 2024-09-21T21:42:24.266 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.266 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.266 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:24.266 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248454+0000", 2024-09-21T21:42:24.266 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97005259499999996 2024-09-21T21:42:24.266 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.266 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.266 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:24.267 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248576+0000", 2024-09-21T21:42:24.267 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000122127 2024-09-21T21:42:24.267 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.267 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.267 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 2", 2024-09-21T21:42:24.267 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248844+0000", 2024-09-21T21:42:24.267 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00026784699999999999 2024-09-21T21:42:24.267 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.267 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.267 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:24.267 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.252487+0000", 2024-09-21T21:42:24.267 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0036427920000000002 2024-09-21T21:42:24.267 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.267 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.268 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:24.268 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.254691+0000", 2024-09-21T21:42:24.268 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002203899 2024-09-21T21:42:24.268 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.268 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.268 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:24.268 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.254748+0000", 2024-09-21T21:42:24.268 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7163000000000001e-05 2024-09-21T21:42:24.268 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.268 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.268 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:24.268 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.254792+0000", 2024-09-21T21:42:24.268 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3473000000000002e-05 2024-09-21T21:42:24.268 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:24.269 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:24.269 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:24.269 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.269 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.269 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19843 3.d 3:b7fa2ca0:::rbd_data.112157a479c.000000000000004d:head [write 0~0] snapc 21=[21] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:24.269 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.275860+0000", 2024-09-21T21:42:24.269 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 32.905029030999998, 2024-09-21T21:42:24.269 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.98183651999999999, 2024-09-21T21:42:24.269 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:24.269 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:24.269 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:24.269 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:24.269 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:24.269 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:24.270 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19843 2024-09-21T21:42:24.270 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.270 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:24.270 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.270 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:24.270 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275860+0000", 2024-09-21T21:42:24.270 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:24.270 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.270 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.270 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:24.270 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275860+0000", 2024-09-21T21:42:24.270 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:24.270 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.270 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.271 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:24.271 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275869+0000", 2024-09-21T21:42:24.271 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6300000000000004e-06 2024-09-21T21:42:24.271 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.271 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.271 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:24.271 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275871+0000", 2024-09-21T21:42:24.271 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2060000000000001e-06 2024-09-21T21:42:24.271 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.271 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.271 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:24.271 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275876+0000", 2024-09-21T21:42:24.271 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8180000000000003e-06 2024-09-21T21:42:24.271 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.272 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.272 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:24.272 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275880+0000", 2024-09-21T21:42:24.272 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1829999999999996e-06 2024-09-21T21:42:24.272 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.272 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.272 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:24.272 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278416+0000", 2024-09-21T21:42:24.272 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025361149999999998 2024-09-21T21:42:24.272 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.272 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.272 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:24.272 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278419+0000", 2024-09-21T21:42:24.272 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.163e-06 2024-09-21T21:42:24.272 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.273 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.273 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:24.273 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.250249+0000", 2024-09-21T21:42:24.273 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97182984299999997 2024-09-21T21:42:24.273 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.273 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.273 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:24.273 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.250321+0000", 2024-09-21T21:42:24.273 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1656000000000006e-05 2024-09-21T21:42:24.273 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.273 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.273 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 2", 2024-09-21T21:42:24.273 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.250419+0000", 2024-09-21T21:42:24.273 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.8277999999999997e-05 2024-09-21T21:42:24.274 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.274 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.274 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:24.274 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.252516+0000", 2024-09-21T21:42:24.274 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002097224 2024-09-21T21:42:24.274 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.274 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.274 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:24.274 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.257643+0000", 2024-09-21T21:42:24.274 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0051262319999999997 2024-09-21T21:42:24.274 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.274 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.274 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:24.274 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.257673+0000", 2024-09-21T21:42:24.275 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0023e-05 2024-09-21T21:42:24.275 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.275 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.275 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:24.275 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.257697+0000", 2024-09-21T21:42:24.275 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4148999999999999e-05 2024-09-21T21:42:24.275 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:24.275 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:24.275 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:24.275 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.275 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.275 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5626.0:12334 3.3 3:c643a86c:::rbd_data.1121faebdfe3.0000000000000002:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:42:24.275 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:14.411628+0000", 2024-09-21T21:42:24.275 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 9.7692607319999993, 2024-09-21T21:42:24.276 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0086947559999999, 2024-09-21T21:42:24.276 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:24.276 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:24.276 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:24.276 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:24.276 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5626", 2024-09-21T21:42:24.276 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:24.276 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 12334 2024-09-21T21:42:24.276 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.276 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:24.276 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.276 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:24.276 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411628+0000", 2024-09-21T21:42:24.276 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:24.277 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.277 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.277 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:24.277 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411628+0000", 2024-09-21T21:42:24.277 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:24.277 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.277 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.277 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:24.277 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411629+0000", 2024-09-21T21:42:24.277 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1200000000000002e-07 2024-09-21T21:42:24.277 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.277 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.277 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:24.277 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411633+0000", 2024-09-21T21:42:24.278 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1699999999999999e-06 2024-09-21T21:42:24.278 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.278 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.278 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:24.278 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411645+0000", 2024-09-21T21:42:24.278 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1328999999999999e-05 2024-09-21T21:42:24.278 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.278 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.278 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:24.278 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411652+0000", 2024-09-21T21:42:24.278 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9990000000000002e-06 2024-09-21T21:42:24.278 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.278 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.278 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:24.279 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411717+0000", 2024-09-21T21:42:24.279 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5519999999999996e-05 2024-09-21T21:42:24.279 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.279 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.279 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op must wait for map", 2024-09-21T21:42:24.279 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411725+0000", 2024-09-21T21:42:24.279 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2609999999999996e-06 2024-09-21T21:42:24.279 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.279 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.279 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:24.279 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415904+0000", 2024-09-21T21:42:24.279 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0041792749999999997 2024-09-21T21:42:24.279 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.279 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.280 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:24.280 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415913+0000", 2024-09-21T21:42:24.280 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.054e-06 2024-09-21T21:42:24.280 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.280 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.280 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:24.280 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420096+0000", 2024-09-21T21:42:24.280 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0041835670000001 2024-09-21T21:42:24.280 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.280 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.280 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:24.280 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420216+0000", 2024-09-21T21:42:24.280 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000120025 2024-09-21T21:42:24.280 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.280 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.281 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:24.281 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420323+0000", 2024-09-21T21:42:24.281 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010674400000000001 2024-09-21T21:42:24.281 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:24.281 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:24.281 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:24.281 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.281 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.281 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5626.0:12339 3.3 3:c643a86c:::rbd_data.1121faebdfe3.0000000000000002:head [read 4096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:42:24.281 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:14.411749+0000", 2024-09-21T21:42:24.281 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 9.7691405459999991, 2024-09-21T21:42:24.281 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.008818056, 2024-09-21T21:42:24.281 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:24.281 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:24.281 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:24.282 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:24.282 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5626", 2024-09-21T21:42:24.282 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:24.282 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 12339 2024-09-21T21:42:24.282 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.282 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:24.282 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.282 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:24.282 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411749+0000", 2024-09-21T21:42:24.282 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:24.282 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.282 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.282 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:24.282 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411749+0000", 2024-09-21T21:42:24.283 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:24.283 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.283 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.283 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:24.283 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411749+0000", 2024-09-21T21:42:24.283 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0900000000000002e-07 2024-09-21T21:42:24.283 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.283 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.283 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:24.283 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411752+0000", 2024-09-21T21:42:24.283 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5610000000000001e-06 2024-09-21T21:42:24.283 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.283 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.283 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:24.284 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411758+0000", 2024-09-21T21:42:24.284 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1890000000000004e-06 2024-09-21T21:42:24.284 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.284 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.284 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:24.284 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411763+0000", 2024-09-21T21:42:24.284 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7589999999999997e-06 2024-09-21T21:42:24.284 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.284 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.284 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:24.284 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411803+0000", 2024-09-21T21:42:24.284 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9922999999999997e-05 2024-09-21T21:42:24.284 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.284 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.284 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting_for_map not empty", 2024-09-21T21:42:24.285 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411809+0000", 2024-09-21T21:42:24.285 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3779999999999998e-06 2024-09-21T21:42:24.285 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.285 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.285 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:24.285 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415937+0000", 2024-09-21T21:42:24.285 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0041276100000000003 2024-09-21T21:42:24.285 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.285 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.285 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:24.285 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415944+0000", 2024-09-21T21:42:24.285 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8329999999999994e-06 2024-09-21T21:42:24.285 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.285 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.286 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:24.286 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420411+0000", 2024-09-21T21:42:24.286 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0044667300000001 2024-09-21T21:42:24.286 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.286 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.286 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:24.286 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420475+0000", 2024-09-21T21:42:24.286 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4356000000000005e-05 2024-09-21T21:42:24.286 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.286 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.286 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:24.286 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420567+0000", 2024-09-21T21:42:24.286 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.1308000000000001e-05 2024-09-21T21:42:24.286 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:24.286 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:24.287 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:24.287 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.287 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.287 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5626.0:12345 3.3 3:c643a86c:::rbd_data.1121faebdfe3.0000000000000002:head [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:42:24.287 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:14.411831+0000", 2024-09-21T21:42:24.287 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 9.7690584929999993, 2024-09-21T21:42:24.287 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0089163649999999, 2024-09-21T21:42:24.287 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:24.287 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:24.287 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:24.287 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:24.287 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5626", 2024-09-21T21:42:24.287 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:24.287 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 12345 2024-09-21T21:42:24.288 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.288 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:24.288 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.288 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:24.288 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411831+0000", 2024-09-21T21:42:24.288 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:24.288 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.288 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.288 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:24.288 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411831+0000", 2024-09-21T21:42:24.288 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:24.288 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.288 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.288 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:24.288 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411831+0000", 2024-09-21T21:42:24.289 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.08e-07 2024-09-21T21:42:24.289 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.289 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.289 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:24.289 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411833+0000", 2024-09-21T21:42:24.289 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3089999999999998e-06 2024-09-21T21:42:24.289 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.289 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.289 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:24.289 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411840+0000", 2024-09-21T21:42:24.289 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.472e-06 2024-09-21T21:42:24.289 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.289 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.289 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:24.290 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411845+0000", 2024-09-21T21:42:24.290 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7910000000000004e-06 2024-09-21T21:42:24.290 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.290 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.290 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:24.290 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411890+0000", 2024-09-21T21:42:24.290 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5763000000000002e-05 2024-09-21T21:42:24.290 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.290 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.290 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting_for_map not empty", 2024-09-21T21:42:24.290 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411896+0000", 2024-09-21T21:42:24.290 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0190000000000003e-06 2024-09-21T21:42:24.290 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.290 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.290 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:24.291 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415967+0000", 2024-09-21T21:42:24.291 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0040705749999999999 2024-09-21T21:42:24.291 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.291 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.291 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:24.291 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415974+0000", 2024-09-21T21:42:24.291 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3200000000000002e-06 2024-09-21T21:42:24.291 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.291 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.291 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:24.291 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420594+0000", 2024-09-21T21:42:24.291 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.004620093 2024-09-21T21:42:24.291 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.291 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.291 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:24.292 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420658+0000", 2024-09-21T21:42:24.292 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3678999999999998e-05 2024-09-21T21:42:24.292 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:24.292 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:24.292 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:24.292 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420747+0000", 2024-09-21T21:42:24.292 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8936000000000004e-05 2024-09-21T21:42:24.292 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:24.292 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:24.292 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:24.292 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:24.292 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:24.292 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:24.294 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T21:42:24.294 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:24.292+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:24.301 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T21:42:24.398 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:24.393+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:24.421 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0: {} 2024-09-21T21:42:24.421 INFO:teuthology.orchestra.run.smithi081.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T21:42:24.438 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1: {} 2024-09-21T21:42:24.438 INFO:teuthology.orchestra.run.smithi081.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T21:42:24.439 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:24.439 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:42:24.439 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-21T21:42:24.439 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:24.447 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T21:42:24.453 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2: {} 2024-09-21T21:42:24.453 INFO:teuthology.orchestra.run.smithi081.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T21:42:24.470 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3: {} 2024-09-21T21:42:24.470 INFO:teuthology.orchestra.run.smithi081.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T21:42:24.486 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4: {} 2024-09-21T21:42:24.486 INFO:teuthology.orchestra.run.smithi081.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T21:42:24.502 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5: {} 2024-09-21T21:42:24.503 INFO:teuthology.orchestra.run.smithi081.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T21:42:24.515 INFO:teuthology.orchestra.run.smithi081.stderr:noscrub is set 2024-09-21T21:42:24.532 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:24.531+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:24.577 INFO:teuthology.orchestra.run.smithi164.stdout:flattening image image_client.0-clone39 2024-09-21T21:42:24.596 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:24.594+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:24.597 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:24.597 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:24.606 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T21:42:24.696 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:24.695+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:24.750 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:24.750 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:24.759 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T21:42:24.798 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:24.796+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:24.866 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:24.867 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:24.875 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T21:42:24.898 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:24.897+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:24.911 INFO:teuthology.orchestra.run.smithi164.stdout:checking clone #38, image image_client.0-clone38 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent39 2024-09-21T21:42:24.998 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:24.997+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:25.033 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:25.033 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:25.046 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T21:42:25.099 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:25.097+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:25.200 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:25.198+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:25.253 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:25.253 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:25.267 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T21:42:25.307 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:25.306+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:25.378 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:25.378 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:25.387 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T21:42:25.402 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:25.401+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:25.443 INFO:teuthology.orchestra.run.smithi164.stdout:5365 read 0x665600 thru 0x66babf (0x64c0 bytes) 2024-09-21T21:42:25.444 INFO:teuthology.orchestra.run.smithi164.stdout:5366 punch from 0x4fd2800 to 0x4fd3800, (0x1000 bytes) 2024-09-21T21:42:25.454 INFO:teuthology.orchestra.run.smithi164.stdout:5370 punch from 0x28fbe00 to 0x28fdc00, (0x1e00 bytes) 2024-09-21T21:42:25.459 INFO:teuthology.orchestra.run.smithi164.stdout:5371 punch from 0x1ea1800 to 0x1ea4000, (0x2800 bytes) 2024-09-21T21:42:25.463 INFO:teuthology.orchestra.run.smithi164.stdout:5374 punch from 0x593a00 to 0x5a1e00, (0xe400 bytes) 2024-09-21T21:42:25.483 INFO:teuthology.orchestra.run.smithi164.stdout:5376 write 0x9fc1800 thru 0x9fcc79f (0xafa0 bytes) 2024-09-21T21:42:25.484 INFO:teuthology.orchestra.run.smithi164.stdout:5377 write 0x9cea400 thru 0x9cefdd2 (0x59d3 bytes) 2024-09-21T21:42:25.484 INFO:teuthology.orchestra.run.smithi164.stdout:5378 read 0x9aed200 thru 0x9af0798 (0x3599 bytes) 2024-09-21T21:42:25.484 INFO:teuthology.orchestra.run.smithi164.stdout:5379 punch from 0x1fb0800 to 0x1fb5200, (0x4a00 bytes) 2024-09-21T21:42:25.485 INFO:teuthology.orchestra.run.smithi164.stdout:5380 punch from 0x3e40800 to 0x3e4e000, (0xd800 bytes) 2024-09-21T21:42:25.487 INFO:teuthology.orchestra.run.smithi164.stdout:5385 trunc from 0x9fcc800 to 0x855c400 2024-09-21T21:42:25.495 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:25.495 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:25.503 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:25.502+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:25.505 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T21:42:25.518 INFO:teuthology.orchestra.run.smithi164.stdout:5387 trunc from 0x855c400 to 0x1130200 2024-09-21T21:42:25.560 INFO:teuthology.orchestra.run.smithi164.stdout:5388 read 0x633a00 thru 0x639926 (0x5f27 bytes) 2024-09-21T21:42:25.562 INFO:teuthology.orchestra.run.smithi164.stdout:5389 punch from 0xf04000 to 0xf10e00, (0xce00 bytes) 2024-09-21T21:42:25.562 INFO:teuthology.orchestra.run.smithi164.stdout:5391 punch from 0x251000 to 0x25c400, (0xb400 bytes) 2024-09-21T21:42:25.574 INFO:teuthology.orchestra.run.smithi164.stdout:5393 trunc from 0x1130200 to 0xb3d7400 2024-09-21T21:42:25.583 INFO:teuthology.orchestra.run.smithi164.stdout:5396 punch from 0x133f800 to 0x1344c00, (0x5400 bytes) 2024-09-21T21:42:25.586 INFO:teuthology.orchestra.run.smithi164.stdout:5398 read 0x1bf9200 thru 0x1c00e26 (0x7c27 bytes) 2024-09-21T21:42:25.586 INFO:teuthology.orchestra.run.smithi164.stdout:5401 punch from 0x659bc00 to 0x65aa200, (0xe600 bytes) 2024-09-21T21:42:25.586 INFO:teuthology.orchestra.run.smithi164.stdout:5402 punch from 0x83e5a00 to 0x83e5e00, (0x400 bytes) 2024-09-21T21:42:25.586 INFO:teuthology.orchestra.run.smithi164.stdout:5409 read 0xdde800 thru 0xde6248 (0x7a49 bytes) 2024-09-21T21:42:25.586 INFO:teuthology.orchestra.run.smithi164.stdout:5410 write 0xa08ce00 thru 0xa09b625 (0xe826 bytes) 2024-09-21T21:42:25.586 INFO:teuthology.orchestra.run.smithi164.stdout:5411 punch from 0x1080200 to 0x1081000, (0xe00 bytes) 2024-09-21T21:42:25.586 INFO:teuthology.orchestra.run.smithi164.stdout:5413 read 0x52fb200 thru 0x530450c (0x930d bytes) 2024-09-21T21:42:25.587 INFO:teuthology.orchestra.run.smithi164.stdout:5414 write 0x93b0000 thru 0x93b92c4 (0x92c5 bytes) 2024-09-21T21:42:25.590 INFO:teuthology.orchestra.run.smithi164.stdout:5415 trunc from 0xb3d7400 to 0xdebe400 2024-09-21T21:42:25.604 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:25.602+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:25.611 INFO:teuthology.orchestra.run.smithi164.stdout:5416 trunc from 0xdebe400 to 0x734c400 2024-09-21T21:42:25.632 INFO:teuthology.orchestra.run.smithi164.stdout:5417 read 0x139e800 thru 0x13aa0f8 (0xb8f9 bytes) 2024-09-21T21:42:25.632 INFO:teuthology.orchestra.run.smithi164.stdout:5418 punch from 0x4f19400 to 0x4f28400, (0xf000 bytes) 2024-09-21T21:42:25.637 INFO:teuthology.orchestra.run.smithi164.stdout:5419 trunc from 0x734c400 to 0xba7ec00 2024-09-21T21:42:25.653 INFO:teuthology.orchestra.run.smithi164.stdout:5423 read 0x1ba5400 thru 0x1ba7581 (0x2182 bytes) 2024-09-21T21:42:25.653 INFO:teuthology.orchestra.run.smithi164.stdout:5424 read 0x9c7f200 thru 0x9c804f5 (0x12f6 bytes) 2024-09-21T21:42:25.657 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:25.657 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:25.666 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T21:42:25.667 INFO:teuthology.orchestra.run.smithi164.stdout:5428 write 0xe739600 thru 0xe741dcc (0x87cd bytes) 2024-09-21T21:42:25.669 INFO:teuthology.orchestra.run.smithi164.stdout:5430 punch from 0x13ebe00 to 0x13f4e00, (0x9000 bytes) 2024-09-21T21:42:25.669 INFO:teuthology.orchestra.run.smithi164.stdout:5431 write 0x9781200 thru 0x978f1f4 (0xdff5 bytes) 2024-09-21T21:42:25.670 INFO:teuthology.orchestra.run.smithi164.stdout:5433 write 0xcfa7e00 thru 0xcfafc4f (0x7e50 bytes) 2024-09-21T21:42:25.670 INFO:teuthology.orchestra.run.smithi164.stdout:5434 read 0x81d5a00 thru 0x81d5cbf (0x2c0 bytes) 2024-09-21T21:42:25.670 INFO:teuthology.orchestra.run.smithi164.stdout:5435 read 0x99c6000 thru 0x99cb3b1 (0x53b2 bytes) 2024-09-21T21:42:25.670 INFO:teuthology.orchestra.run.smithi164.stdout:5436 punch from 0xe0f6e00 to 0xe0fe800, (0x7a00 bytes) 2024-09-21T21:42:25.670 INFO:teuthology.orchestra.run.smithi164.stdout:5440 write 0x1b8e200 thru 0x1b9737e (0x917f bytes) 2024-09-21T21:42:25.686 INFO:teuthology.orchestra.run.smithi164.stdout:5443 trunc from 0xe741e00 to 0x267000 2024-09-21T21:42:25.705 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:25.703+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:25.730 INFO:teuthology.orchestra.run.smithi164.stdout:5444 read 0x1b4400 thru 0x1bdbe4 (0x97e5 bytes) 2024-09-21T21:42:25.731 INFO:teuthology.orchestra.run.smithi164.stdout:5447 read 0x1c0600 thru 0x1c694c (0x634d bytes) 2024-09-21T21:42:25.732 INFO:teuthology.orchestra.run.smithi164.stdout:5448 read 0xed800 thru 0xef2ee (0x1aef bytes) 2024-09-21T21:42:25.732 INFO:teuthology.orchestra.run.smithi164.stdout:5449 punch from 0xe4200 to 0xe6800, (0x2600 bytes) 2024-09-21T21:42:25.733 INFO:teuthology.orchestra.run.smithi164.stdout:5450 read 0x1d7000 thru 0x1e1bfd (0xabfe bytes) 2024-09-21T21:42:25.751 INFO:teuthology.orchestra.run.smithi164.stdout:5452 write 0x870fa00 thru 0x871a4ca (0xaacb bytes) 2024-09-21T21:42:25.752 INFO:teuthology.orchestra.run.smithi164.stdout:5455 read 0x6954c00 thru 0x69558bc (0xcbd bytes) 2024-09-21T21:42:25.753 INFO:teuthology.orchestra.run.smithi164.stdout:5458 trunc from 0x871a600 to 0x7d06a00 2024-09-21T21:42:25.783 INFO:teuthology.orchestra.run.smithi164.stdout:5459 write 0x80e8000 thru 0x80f2d1f (0xad20 bytes) 2024-09-21T21:42:25.803 INFO:teuthology.orchestra.run.smithi164.stdout:5460 write 0xe566800 thru 0xe576064 (0xf865 bytes) 2024-09-21T21:42:25.805 INFO:teuthology.orchestra.run.smithi164.stdout:5461 read 0x7c41400 thru 0x7c4edbc (0xd9bd bytes) 2024-09-21T21:42:25.805 INFO:teuthology.orchestra.run.smithi164.stdout:5463 read 0x8d46c00 thru 0x8d4b739 (0x4b3a bytes) 2024-09-21T21:42:25.805 INFO:teuthology.orchestra.run.smithi164.stdout:5464 write 0x1ec8200 thru 0x1eca8c4 (0x26c5 bytes) 2024-09-21T21:42:25.805 INFO:teuthology.orchestra.run.smithi164.stdout:5465 punch from 0xa93f600 to 0xa94a000, (0xaa00 bytes) 2024-09-21T21:42:25.805 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:25.803+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:25.807 INFO:teuthology.orchestra.run.smithi164.stdout:5467 punch from 0x4d98000 to 0x4d9f200, (0x7200 bytes) 2024-09-21T21:42:25.807 INFO:teuthology.orchestra.run.smithi164.stdout:5471 read 0x3ac8e00 thru 0x3ad5fe6 (0xd1e7 bytes) 2024-09-21T21:42:25.807 INFO:teuthology.orchestra.run.smithi164.stdout:5472 read 0x3c21a00 thru 0x3c23c77 (0x2278 bytes) 2024-09-21T21:42:25.807 INFO:teuthology.orchestra.run.smithi164.stdout:5473 punch from 0x542e400 to 0x543b400, (0xd000 bytes) 2024-09-21T21:42:25.807 INFO:teuthology.orchestra.run.smithi164.stdout:5475 write 0x9acea00 thru 0x9ad045a (0x1a5b bytes) 2024-09-21T21:42:25.807 INFO:teuthology.orchestra.run.smithi164.stdout:5476 write 0x4bec400 thru 0x4bf2e45 (0x6a46 bytes) 2024-09-21T21:42:25.807 INFO:teuthology.orchestra.run.smithi164.stdout:5477 read 0x6dd3800 thru 0x6dd38cb (0xcc bytes) 2024-09-21T21:42:25.814 INFO:teuthology.orchestra.run.smithi164.stdout:5482 trunc from 0xe576200 to 0x8056200 2024-09-21T21:42:25.818 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:25.818 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:25.830 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T21:42:25.834 INFO:teuthology.orchestra.run.smithi164.stdout:5483 read 0x1bc1400 thru 0x1bcb01b (0x9c1c bytes) 2024-09-21T21:42:25.835 INFO:teuthology.orchestra.run.smithi164.stdout:5484 write 0x19d0600 thru 0x19ddb6b (0xd56c bytes) 2024-09-21T21:42:25.835 INFO:teuthology.orchestra.run.smithi164.stdout:5486 write 0x56cce00 thru 0x56dbb05 (0xed06 bytes) 2024-09-21T21:42:25.836 INFO:teuthology.orchestra.run.smithi164.stdout:5487 punch from 0x3826600 to 0x382e400, (0x7e00 bytes) 2024-09-21T21:42:25.837 INFO:teuthology.orchestra.run.smithi164.stdout:5489 punch from 0x3f2fc00 to 0x3f37800, (0x7c00 bytes) 2024-09-21T21:42:25.837 INFO:teuthology.orchestra.run.smithi164.stdout:5490 write 0x381c200 thru 0x3826315 (0xa116 bytes) 2024-09-21T21:42:25.837 INFO:teuthology.orchestra.run.smithi164.stdout:5491 punch from 0x1cc2a00 to 0x1ccee00, (0xc400 bytes) 2024-09-21T21:42:25.837 INFO:teuthology.orchestra.run.smithi164.stdout:5492 write 0x22bc000 thru 0x22bca20 (0xa21 bytes) 2024-09-21T21:42:25.837 INFO:teuthology.orchestra.run.smithi164.stdout:5493 trunc from 0x8056200 to 0x7c54800 2024-09-21T21:42:25.865 INFO:teuthology.orchestra.run.smithi164.stdout:5497 read 0x47c2800 thru 0x47c6453 (0x3c54 bytes) 2024-09-21T21:42:25.867 INFO:teuthology.orchestra.run.smithi164.stdout:5498 trunc from 0x7c54800 to 0x65f0600 2024-09-21T21:42:25.877 INFO:teuthology.orchestra.run.smithi164.stdout:5500 read 0x482da00 thru 0x483d959 (0xff5a bytes) 2024-09-21T21:42:25.889 INFO:teuthology.orchestra.run.smithi164.stdout:5502 write 0x85f9000 thru 0x85ffd7b (0x6d7c bytes) 2024-09-21T21:42:25.906 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:25.904+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:25.908 INFO:teuthology.orchestra.run.smithi164.stdout:5504 write 0xc112800 thru 0xc122095 (0xf896 bytes) 2024-09-21T21:42:25.919 INFO:teuthology.orchestra.run.smithi164.stdout:5510 trunc from 0xc122200 to 0x3bfd200 2024-09-21T21:42:25.939 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:25.939 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:25.941 INFO:teuthology.orchestra.run.smithi164.stdout:5512 trunc from 0x3bfd200 to 0x2766200 2024-09-21T21:42:25.948 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T21:42:25.973 INFO:teuthology.orchestra.run.smithi164.stdout:5517 write 0xa90d200 thru 0xa915776 (0x8577 bytes) 2024-09-21T21:42:25.974 INFO:teuthology.orchestra.run.smithi164.stdout:5518 read 0x8f01000 thru 0x8f08f93 (0x7f94 bytes) 2024-09-21T21:42:25.975 INFO:teuthology.orchestra.run.smithi164.stdout:5519 punch from 0xe9600 to 0xed000, (0x3a00 bytes) 2024-09-21T21:42:25.976 INFO:teuthology.orchestra.run.smithi164.stdout:5521 read 0x9184e00 thru 0x918c2df (0x74e0 bytes) 2024-09-21T21:42:25.976 INFO:teuthology.orchestra.run.smithi164.stdout:5524 read 0x60e3000 thru 0x60ea8b2 (0x78b3 bytes) 2024-09-21T21:42:25.976 INFO:teuthology.orchestra.run.smithi164.stdout:5525 trunc from 0xa915800 to 0xa910a00 2024-09-21T21:42:25.988 INFO:teuthology.orchestra.run.smithi164.stdout:5528 punch from 0x24d9400 to 0x24e2400, (0x9000 bytes) 2024-09-21T21:42:25.997 INFO:teuthology.orchestra.run.smithi164.stdout:5530 punch from 0x4785e00 to 0x478ae00, (0x5000 bytes) 2024-09-21T21:42:26.007 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:26.005+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:26.010 INFO:teuthology.orchestra.run.smithi164.stdout:5531 write 0xbd1c200 thru 0xbd276af (0xb4b0 bytes) 2024-09-21T21:42:26.018 INFO:teuthology.orchestra.run.smithi164.stdout:5532 write 0x6d03e00 thru 0x6d09843 (0x5a44 bytes) 2024-09-21T21:42:26.019 INFO:teuthology.orchestra.run.smithi164.stdout:5533 write 0x1966200 thru 0x19677ca (0x15cb bytes) 2024-09-21T21:42:26.019 INFO:teuthology.orchestra.run.smithi164.stdout:5535 read 0x45baa00 thru 0x45c537e (0xa97f bytes) 2024-09-21T21:42:26.019 INFO:teuthology.orchestra.run.smithi164.stdout:5536 write 0xa89f800 thru 0xa8a44dd (0x4cde bytes) 2024-09-21T21:42:26.030 INFO:teuthology.orchestra.run.smithi164.stdout:5539 trunc from 0xbd27800 to 0x2688e00 2024-09-21T21:42:26.052 INFO:teuthology.orchestra.run.smithi164.stdout:5541 write 0x233ba00 thru 0x233caa9 (0x10aa bytes) 2024-09-21T21:42:26.053 INFO:teuthology.orchestra.run.smithi164.stdout:5543 read 0x1029c00 thru 0x102baf3 (0x1ef4 bytes) 2024-09-21T21:42:26.058 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:26.058 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:26.067 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T21:42:26.077 INFO:teuthology.orchestra.run.smithi164.stdout:5544 write 0xb2a4600 thru 0xb2a56b5 (0x10b6 bytes) 2024-09-21T21:42:26.078 INFO:teuthology.orchestra.run.smithi164.stdout:5545 write 0x6626800 thru 0x662d3dd (0x6bde bytes) 2024-09-21T21:42:26.081 INFO:teuthology.orchestra.run.smithi164.stdout:5548 trunc from 0xb2a5800 to 0xe559400 2024-09-21T21:42:26.095 INFO:teuthology.orchestra.run.smithi164.stdout:5550 read 0x737a800 thru 0x737ad35 (0x536 bytes) 2024-09-21T21:42:26.097 INFO:teuthology.orchestra.run.smithi164.stdout:5553 punch from 0xbf2ac00 to 0xbf3aa00, (0xfe00 bytes) 2024-09-21T21:42:26.098 INFO:teuthology.orchestra.run.smithi164.stdout:5554 write 0x2656800 thru 0x2659bec (0x33ed bytes) 2024-09-21T21:42:26.098 INFO:teuthology.orchestra.run.smithi164.stdout:5557 write 0x8639400 thru 0x863e542 (0x5143 bytes) 2024-09-21T21:42:26.099 INFO:teuthology.orchestra.run.smithi164.stdout:5566 punch from 0xf8b000 to 0xf9a800, (0xf800 bytes) 2024-09-21T21:42:26.099 INFO:teuthology.orchestra.run.smithi164.stdout:5569 read 0x4b1600 thru 0x4b2ace (0x14cf bytes) 2024-09-21T21:42:26.111 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:26.110+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:26.112 INFO:teuthology.orchestra.run.smithi164.stdout:5570 trunc from 0xe559400 to 0x38f5c00 2024-09-21T21:42:26.134 INFO:teuthology.orchestra.run.smithi164.stdout:5571 punch from 0x1ee1e00 to 0x1ee2a00, (0xc00 bytes) 2024-09-21T21:42:26.164 INFO:teuthology.orchestra.run.smithi164.stdout:5574 write 0xe547600 thru 0xe54f868 (0x8269 bytes) 2024-09-21T21:42:26.164 INFO:teuthology.orchestra.run.smithi164.stdout:5575 write 0xa827000 thru 0xa834477 (0xd478 bytes) 2024-09-21T21:42:26.164 INFO:teuthology.orchestra.run.smithi164.stdout:5576 punch from 0x33cc400 to 0x33d5200, (0x8e00 bytes) 2024-09-21T21:42:26.164 INFO:teuthology.orchestra.run.smithi164.stdout:5577 punch from 0x59bcc00 to 0x59cb600, (0xea00 bytes) 2024-09-21T21:42:26.164 INFO:teuthology.orchestra.run.smithi164.stdout:5578 write 0x8b30e00 thru 0x8b363ad (0x55ae bytes) 2024-09-21T21:42:26.174 INFO:teuthology.orchestra.run.smithi164.stdout:5579 trunc from 0xe54fa00 to 0x641ee00 2024-09-21T21:42:26.176 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:26.176 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:26.186 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T21:42:26.199 INFO:teuthology.orchestra.run.smithi164.stdout:5581 punch from 0x4dfde00 to 0x4e0a400, (0xc600 bytes) 2024-09-21T21:42:26.199 INFO:teuthology.orchestra.run.smithi164.stdout:5582 write 0x14c2c00 thru 0x14c9107 (0x6508 bytes) 2024-09-21T21:42:26.205 INFO:teuthology.orchestra.run.smithi164.stdout:5584 trunc from 0x641ee00 to 0x1447e00 2024-09-21T21:42:26.226 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:26.225+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:26.259 INFO:teuthology.orchestra.run.smithi164.stdout:5587 write 0xb6e7000 thru 0xb6ed2c1 (0x62c2 bytes) 2024-09-21T21:42:26.260 INFO:teuthology.orchestra.run.smithi164.stdout:5588 write 0x2831000 thru 0x283335a (0x235b bytes) 2024-09-21T21:42:26.260 INFO:teuthology.orchestra.run.smithi164.stdout:5592 punch from 0x68a4a00 to 0x68ab600, (0x6c00 bytes) 2024-09-21T21:42:26.261 INFO:teuthology.orchestra.run.smithi164.stdout:5594 read 0x3bb3e00 thru 0x3bb9a52 (0x5c53 bytes) 2024-09-21T21:42:26.274 INFO:teuthology.orchestra.run.smithi164.stdout:5599 write 0xc8a1200 thru 0xc8b1162 (0xff63 bytes) 2024-09-21T21:42:26.281 INFO:teuthology.orchestra.run.smithi164.stdout:5601 write 0x3493200 thru 0x349b301 (0x8102 bytes) 2024-09-21T21:42:26.294 INFO:teuthology.orchestra.run.smithi164.stdout:5605 trunc from 0xc8b1200 to 0x1b7da00 2024-09-21T21:42:26.309 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:26.308+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:26.335 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:26.335 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:26.338 INFO:teuthology.orchestra.run.smithi164.stdout:5609 write 0x6ca3600 thru 0x6caea4c (0xb44d bytes) 2024-09-21T21:42:26.344 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T21:42:26.344 INFO:teuthology.orchestra.run.smithi164.stdout:5613 write 0x5c43e00 thru 0x5c4a823 (0x6a24 bytes) 2024-09-21T21:42:26.350 INFO:teuthology.orchestra.run.smithi164.stdout:5614 trunc from 0x6caec00 to 0x1463600 2024-09-21T21:42:26.374 INFO:teuthology.orchestra.run.smithi164.stdout:5616 read 0x6afc00 thru 0x6b8b8e (0x8f8f bytes) 2024-09-21T21:42:26.381 INFO:teuthology.orchestra.run.smithi164.stdout:5618 read 0x30200 thru 0x3e546 (0xe347 bytes) 2024-09-21T21:42:26.383 INFO:teuthology.orchestra.run.smithi164.stdout:5620 read 0xddd800 thru 0xde39c0 (0x61c1 bytes) 2024-09-21T21:42:26.383 INFO:teuthology.orchestra.run.smithi164.stdout:5621 punch from 0xb0a400 to 0xb0e200, (0x3e00 bytes) 2024-09-21T21:42:26.383 INFO:teuthology.orchestra.run.smithi164.stdout:5625 punch from 0xad9400 to 0xadb200, (0x1e00 bytes) 2024-09-21T21:42:26.383 INFO:teuthology.orchestra.run.smithi164.stdout:5626 read 0x119b600 thru 0x11a7166 (0xbb67 bytes) 2024-09-21T21:42:26.405 INFO:teuthology.orchestra.run.smithi164.stdout:5627 write 0xb5f4600 thru 0xb5fd9ea (0x93eb bytes) 2024-09-21T21:42:26.410 INFO:teuthology.orchestra.run.smithi164.stdout:5628 read 0xb377800 thru 0xb38342e (0xbc2f bytes) 2024-09-21T21:42:26.422 INFO:teuthology.orchestra.run.smithi164.stdout:5629 trunc from 0xb5fda00 to 0x198e600 2024-09-21T21:42:26.427 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:26.426+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:26.446 INFO:teuthology.orchestra.run.smithi164.stdout:5631 trunc from 0x198e600 to 0x4b27400 2024-09-21T21:42:26.466 INFO:teuthology.orchestra.run.smithi164.stdout:5632 trunc from 0x4b27400 to 0xcda0c00 2024-09-21T21:42:26.477 INFO:teuthology.orchestra.run.smithi164.stdout:5633 punch from 0x1ea1600 to 0x1ea2000, (0xa00 bytes) 2024-09-21T21:42:26.485 INFO:teuthology.orchestra.run.smithi164.stdout:5637 write 0x5f99800 thru 0x5f9a123 (0x924 bytes) 2024-09-21T21:42:26.485 INFO:teuthology.orchestra.run.smithi164.stdout:5639 punch from 0x7cb4e00 to 0x7cc4000, (0xf200 bytes) 2024-09-21T21:42:26.485 INFO:teuthology.orchestra.run.smithi164.stdout:5640 punch from 0x331b200 to 0x3327400, (0xc200 bytes) 2024-09-21T21:42:26.492 INFO:teuthology.orchestra.run.smithi164.stdout:5641 trunc from 0xcda0c00 to 0x74a8c00 2024-09-21T21:42:26.495 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:26.495 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:26.504 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T21:42:26.513 INFO:teuthology.orchestra.run.smithi164.stdout:5646 trunc from 0x74a8c00 to 0xc219c00 2024-09-21T21:42:26.517 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T21:42:26.524 INFO:teuthology.orchestra.run.smithi164.stdout:5648 write 0x4856c00 thru 0x48598e3 (0x2ce4 bytes) 2024-09-21T21:42:26.530 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T21:42:26.531 INFO:teuthology.orchestra.run.smithi164.stdout:5650 read 0x83e6000 thru 0x83e75a8 (0x15a9 bytes) 2024-09-21T21:42:26.532 INFO:teuthology.orchestra.run.smithi164.stdout:5651 read 0x6a40600 thru 0x6a43d97 (0x3798 bytes) 2024-09-21T21:42:26.532 INFO:teuthology.orchestra.run.smithi164.stdout:5652 read 0x770ca00 thru 0x771206e (0x566f bytes) 2024-09-21T21:42:26.532 INFO:teuthology.orchestra.run.smithi164.stdout:5653 read 0x69c9800 thru 0x69d3f4f (0xa750 bytes) 2024-09-21T21:42:26.532 INFO:teuthology.orchestra.run.smithi164.stdout:5654 read 0x7d7b600 thru 0x7d8b2a3 (0xfca4 bytes) 2024-09-21T21:42:26.532 INFO:teuthology.orchestra.run.smithi164.stdout:5655 punch from 0xa30cc00 to 0xa310a00, (0x3e00 bytes) 2024-09-21T21:42:26.533 INFO:teuthology.orchestra.run.smithi164.stdout:5658 read 0x9fd600 thru 0x9ffa65 (0x2466 bytes) 2024-09-21T21:42:26.533 INFO:teuthology.orchestra.run.smithi164.stdout:5660 punch from 0x5cda800 to 0x5ce7800, (0xd000 bytes) 2024-09-21T21:42:26.533 INFO:teuthology.orchestra.run.smithi164.stdout:5663 punch from 0x2fc6000 to 0x2fcee00, (0x8e00 bytes) 2024-09-21T21:42:26.533 INFO:teuthology.orchestra.run.smithi164.stdout:5664 read 0x489c000 thru 0x48a7cce (0xbccf bytes) 2024-09-21T21:42:26.533 INFO:teuthology.orchestra.run.smithi164.stdout:5665 punch from 0x566ba00 to 0x5677400, (0xba00 bytes) 2024-09-21T21:42:26.533 INFO:teuthology.orchestra.run.smithi164.stdout:5666 punch from 0xa9a0e00 to 0xa9ad000, (0xc200 bytes) 2024-09-21T21:42:26.533 INFO:teuthology.orchestra.run.smithi164.stdout:5669 read 0xb874800 thru 0xb882bbd (0xe3be bytes) 2024-09-21T21:42:26.533 INFO:teuthology.orchestra.run.smithi164.stdout:5671 write 0x4946400 thru 0x494726f (0xe70 bytes) 2024-09-21T21:42:26.533 INFO:teuthology.orchestra.run.smithi164.stdout:5672 write 0x8085600 thru 0x808c287 (0x6c88 bytes) 2024-09-21T21:42:26.536 INFO:teuthology.orchestra.run.smithi164.stdout:5674 trunc from 0xc219c00 to 0x9491000 2024-09-21T21:42:26.548 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:26.547+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:26.557 INFO:teuthology.orchestra.run.smithi164.stdout:5677 read 0x7946600 thru 0x7951a75 (0xb476 bytes) 2024-09-21T21:42:26.559 INFO:teuthology.orchestra.run.smithi164.stdout:5678 read 0x356e400 thru 0x3578040 (0x9c41 bytes) 2024-09-21T21:42:26.559 INFO:teuthology.orchestra.run.smithi164.stdout:5679 read 0x25f5400 thru 0x25f96db (0x42dc bytes) 2024-09-21T21:42:26.559 INFO:teuthology.orchestra.run.smithi164.stdout:5680 read 0x8389600 thru 0x8395a22 (0xc423 bytes) 2024-09-21T21:42:26.559 INFO:teuthology.orchestra.run.smithi164.stdout:5682 read 0x5b2ac00 thru 0x5b2f919 (0x4d1a bytes) 2024-09-21T21:42:26.559 INFO:teuthology.orchestra.run.smithi164.stdout:5684 punch from 0x7d6ac00 to 0x7d6d600, (0x2a00 bytes) 2024-09-21T21:42:26.581 INFO:teuthology.orchestra.run.smithi164.stdout:5686 write 0xa3e5200 thru 0xa3f0fae (0xbdaf bytes) 2024-09-21T21:42:26.581 INFO:teuthology.orchestra.run.smithi164.stdout:5687 punch from 0x14ad200 to 0x14bce00, (0xfc00 bytes) 2024-09-21T21:42:26.581 INFO:teuthology.orchestra.run.smithi164.stdout:5689 write 0x771ac00 thru 0x77296a5 (0xeaa6 bytes) 2024-09-21T21:42:26.581 INFO:teuthology.orchestra.run.smithi164.stdout:5691 read 0x4294a00 thru 0x4299b6a (0x516b bytes) 2024-09-21T21:42:26.581 INFO:teuthology.orchestra.run.smithi164.stdout:5692 punch from 0x671400 to 0x674200, (0x2e00 bytes) 2024-09-21T21:42:26.604 INFO:teuthology.orchestra.run.smithi164.stdout:5693 write 0xb923600 thru 0xb932e3f (0xf840 bytes) 2024-09-21T21:42:26.604 INFO:teuthology.orchestra.run.smithi164.stdout:5694 punch from 0xc69e00 to 0xc71600, (0x7800 bytes) 2024-09-21T21:42:26.604 INFO:teuthology.orchestra.run.smithi164.stdout:5695 punch from 0x5a7e400 to 0x5a88c00, (0xa800 bytes) 2024-09-21T21:42:26.604 INFO:teuthology.orchestra.run.smithi164.stdout:5697 read 0x548ca00 thru 0x5490d69 (0x436a bytes) 2024-09-21T21:42:26.604 INFO:teuthology.orchestra.run.smithi164.stdout:5701 punch from 0x6294200 to 0x6299c00, (0x5a00 bytes) 2024-09-21T21:42:26.604 INFO:teuthology.orchestra.run.smithi164.stdout:5705 read 0x4af4a00 thru 0x4afd961 (0x8f62 bytes) 2024-09-21T21:42:26.604 INFO:teuthology.orchestra.run.smithi164.stdout:5706 write 0xa520c00 thru 0xa52126d (0x66e bytes) 2024-09-21T21:42:26.611 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:26.610+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:26.614 INFO:teuthology.orchestra.run.smithi164.stdout:5707 trunc from 0xb933000 to 0x28d1e00 2024-09-21T21:42:26.626 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:26.626 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:26.637 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T21:42:26.641 INFO:teuthology.orchestra.run.smithi164.stdout:5709 read 0x1247000 thru 0x124bcd3 (0x4cd4 bytes) 2024-09-21T21:42:26.646 INFO:teuthology.orchestra.run.smithi164.stdout:5710 punch from 0x1f71a00 to 0x1f7f600, (0xdc00 bytes) 2024-09-21T21:42:26.650 INFO:teuthology.orchestra.run.smithi164.stdout:5711 trunc from 0x28d1e00 to 0x6de0e00 2024-09-21T21:42:26.672 INFO:teuthology.orchestra.run.smithi164.stdout:5712 write 0xb3dde00 thru 0xb3ebd95 (0xdf96 bytes) 2024-09-21T21:42:26.673 INFO:teuthology.orchestra.run.smithi164.stdout:5713 write 0x5575200 thru 0x557fd70 (0xab71 bytes) 2024-09-21T21:42:26.673 INFO:teuthology.orchestra.run.smithi164.stdout:5714 read 0x984c400 thru 0x984fa9a (0x369b bytes) 2024-09-21T21:42:26.673 INFO:teuthology.orchestra.run.smithi164.stdout:5715 write 0x5b5ea00 thru 0x5b62d91 (0x4392 bytes) 2024-09-21T21:42:26.675 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0: {} 2024-09-21T21:42:26.675 INFO:teuthology.orchestra.run.smithi081.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T21:42:26.676 INFO:teuthology.orchestra.run.smithi164.stdout:5716 trunc from 0xb3ebe00 to 0xda2a400 2024-09-21T21:42:26.694 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1: {} 2024-09-21T21:42:26.694 INFO:teuthology.orchestra.run.smithi081.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T21:42:26.708 INFO:teuthology.orchestra.run.smithi164.stdout:5718 write 0xde9b400 thru 0xdea1cb2 (0x68b3 bytes) 2024-09-21T21:42:26.709 INFO:teuthology.orchestra.run.smithi164.stdout:5719 write 0xd687a00 thru 0xd6958a2 (0xdea3 bytes) 2024-09-21T21:42:26.711 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2: {} 2024-09-21T21:42:26.711 INFO:teuthology.orchestra.run.smithi081.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T21:42:26.712 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:26.710+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:26.723 INFO:teuthology.orchestra.run.smithi164.stdout:5721 trunc from 0xdea1e00 to 0x2769400 2024-09-21T21:42:26.728 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3: {} 2024-09-21T21:42:26.728 INFO:teuthology.orchestra.run.smithi081.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T21:42:26.745 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4: {} 2024-09-21T21:42:26.746 INFO:teuthology.orchestra.run.smithi081.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T21:42:26.749 INFO:teuthology.orchestra.run.smithi164.stdout:5722 read 0x1b8f800 thru 0x1b8f9ee (0x1ef bytes) 2024-09-21T21:42:26.750 INFO:teuthology.orchestra.run.smithi164.stdout:5726 read 0x10ce000 thru 0x10d9a44 (0xba45 bytes) 2024-09-21T21:42:26.750 INFO:teuthology.orchestra.run.smithi164.stdout:5729 write 0xa8000 thru 0xaecd3 (0x6cd4 bytes) 2024-09-21T21:42:26.762 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5: {} 2024-09-21T21:42:26.762 INFO:teuthology.orchestra.run.smithi081.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T21:42:26.764 INFO:teuthology.orchestra.run.smithi164.stdout:5730 trunc from 0x2769400 to 0xe1e0c00 2024-09-21T21:42:26.776 INFO:teuthology.orchestra.run.smithi164.stdout:5731 read 0x686600 thru 0x696496 (0xfe97 bytes) 2024-09-21T21:42:26.777 INFO:teuthology.orchestra.run.smithi164.stdout:5732 write 0x5072400 thru 0x5072d63 (0x964 bytes) 2024-09-21T21:42:26.777 INFO:teuthology.orchestra.run.smithi164.stdout:5733 read 0xcb72c00 thru 0xcb76caf (0x40b0 bytes) 2024-09-21T21:42:26.777 INFO:teuthology.orchestra.run.smithi164.stdout:5734 read 0x7f0ae00 thru 0x7f18e73 (0xe074 bytes) 2024-09-21T21:42:26.777 INFO:teuthology.orchestra.run.smithi164.stdout:5735 write 0xc93200 thru 0xc956ed (0x24ee bytes) 2024-09-21T21:42:26.777 INFO:teuthology.orchestra.run.smithi164.stdout:5736 read 0x3c51c00 thru 0x3c51d66 (0x167 bytes) 2024-09-21T21:42:26.778 INFO:teuthology.orchestra.run.smithi164.stdout:5737 read 0x861ba00 thru 0x86245b2 (0x8bb3 bytes) 2024-09-21T21:42:26.788 INFO:teuthology.orchestra.run.smithi164.stdout:5739 trunc from 0xe1e0c00 to 0x54b9600 2024-09-21T21:42:26.793 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:26.793 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:42:26.794 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:42:26.794 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:42:26.794 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:26.803 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T21:42:26.804 INFO:teuthology.orchestra.run.smithi164.stdout:5740 trunc from 0x54b9600 to 0x426fe00 2024-09-21T21:42:26.837 INFO:teuthology.orchestra.run.smithi164.stdout:5741 write 0x7a99c00 thru 0x7aa8d7c (0xf17d bytes) 2024-09-21T21:42:26.844 INFO:teuthology.orchestra.run.smithi164.stdout:5742 read 0x13baa00 thru 0x13c9ec7 (0xf4c8 bytes) 2024-09-21T21:42:26.844 INFO:teuthology.orchestra.run.smithi164.stdout:5748 read 0x6590e00 thru 0x6591a5b (0xc5c bytes) 2024-09-21T21:42:26.844 INFO:teuthology.orchestra.run.smithi164.stdout:5750 punch from 0x73a4000 to 0x73b2c00, (0xec00 bytes) 2024-09-21T21:42:26.845 INFO:teuthology.orchestra.run.smithi164.stdout:5752 read 0x762ec00 thru 0x7636cc2 (0x80c3 bytes) 2024-09-21T21:42:26.846 INFO:teuthology.orchestra.run.smithi164.stdout:5754 read 0x6cff800 thru 0x6d0c462 (0xcc63 bytes) 2024-09-21T21:42:26.847 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:26.846+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:26.852 INFO:teuthology.orchestra.run.smithi164.stdout:5755 trunc from 0x7aa8e00 to 0x24e0000 2024-09-21T21:42:26.866 INFO:teuthology.orchestra.run.smithi164.stdout:5756 trunc from 0x24e0000 to 0x1c79e00 2024-09-21T21:42:26.878 INFO:teuthology.orchestra.run.smithi164.stdout:5758 read 0x642400 thru 0x64d0d3 (0xacd4 bytes) 2024-09-21T21:42:26.879 INFO:teuthology.orchestra.run.smithi164.stdout:5760 read 0xff0400 thru 0xff9f92 (0x9b93 bytes) 2024-09-21T21:42:26.879 INFO:teuthology.orchestra.run.smithi164.stdout:5761 trunc from 0x1c79e00 to 0x1455000 2024-09-21T21:42:26.911 INFO:teuthology.orchestra.run.smithi164.stdout:5764 write 0x6a61a00 thru 0x6a66c9a (0x529b bytes) 2024-09-21T21:42:26.913 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:26.912+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:26.915 INFO:teuthology.orchestra.run.smithi164.stdout:5765 trunc from 0x6a66e00 to 0x97bd200 2024-09-21T21:42:26.927 INFO:teuthology.orchestra.run.smithi164.stdout:5769 write 0x5e0c400 thru 0x5e1339e (0x6f9f bytes) 2024-09-21T21:42:26.928 INFO:teuthology.orchestra.run.smithi164.stdout:5770 read 0x63f1e00 thru 0x63f3582 (0x1783 bytes) 2024-09-21T21:42:26.928 INFO:teuthology.orchestra.run.smithi164.stdout:5771 read 0x13d7200 thru 0x13de24c (0x704d bytes) 2024-09-21T21:42:26.928 INFO:teuthology.orchestra.run.smithi164.stdout:5772 read 0x4cd7c00 thru 0x4ce532a (0xd72b bytes) 2024-09-21T21:42:26.928 INFO:teuthology.orchestra.run.smithi164.stdout:5773 write 0x5d7ac00 thru 0x5d83758 (0x8b59 bytes) 2024-09-21T21:42:26.928 INFO:teuthology.orchestra.run.smithi164.stdout:5774 write 0x5340200 thru 0x534de39 (0xdc3a bytes) 2024-09-21T21:42:26.928 INFO:teuthology.orchestra.run.smithi164.stdout:5776 read 0x7624000 thru 0x76317e6 (0xd7e7 bytes) 2024-09-21T21:42:26.930 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:26.930 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-21T21:42:26.930 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-21T21:42:26.931 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:42:26.931 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.931 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19747 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 7905280~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:26.931 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273058+0000", 2024-09-21T21:42:26.931 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 35.654801141999997, 2024-09-21T21:42:26.931 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97338591399999996, 2024-09-21T21:42:26.931 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:26.931 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:26.931 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:26.931 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:26.931 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:26.931 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:26.931 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19747 2024-09-21T21:42:26.931 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.932 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:26.932 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.932 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:26.932 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273058+0000", 2024-09-21T21:42:26.932 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:26.932 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.932 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.932 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:26.932 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273058+0000", 2024-09-21T21:42:26.932 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:26.932 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.932 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.932 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:26.932 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273058+0000", 2024-09-21T21:42:26.932 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5900000000000003e-07 2024-09-21T21:42:26.933 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.933 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.933 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:26.933 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273061+0000", 2024-09-21T21:42:26.933 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3479999999999998e-06 2024-09-21T21:42:26.933 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.933 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.933 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:26.933 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273067+0000", 2024-09-21T21:42:26.933 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1469999999999998e-06 2024-09-21T21:42:26.933 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.933 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.933 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:26.933 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273071+0000", 2024-09-21T21:42:26.933 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7339999999999999e-06 2024-09-21T21:42:26.934 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.934 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.934 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:26.934 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277945+0000", 2024-09-21T21:42:26.934 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.004873474 2024-09-21T21:42:26.934 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.934 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.934 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:26.934 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277948+0000", 2024-09-21T21:42:26.934 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0299999999999998e-06 2024-09-21T21:42:26.934 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.934 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.934 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:26.934 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246269+0000", 2024-09-21T21:42:26.935 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96832060200000003 2024-09-21T21:42:26.935 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.935 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.935 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:26.935 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246362+0000", 2024-09-21T21:42:26.935 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3867000000000001e-05 2024-09-21T21:42:26.935 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.935 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.935 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:26.935 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246444+0000", 2024-09-21T21:42:26.935 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1353000000000005e-05 2024-09-21T21:42:26.935 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:26.935 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:26.935 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:26.935 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.936 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.936 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19755 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 5783552~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:26.936 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273077+0000", 2024-09-21T21:42:26.936 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 35.654781526000001, 2024-09-21T21:42:26.936 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97350203400000002, 2024-09-21T21:42:26.936 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:26.936 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:26.936 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:26.936 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:26.936 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:26.936 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:26.936 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19755 2024-09-21T21:42:26.936 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.936 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:26.937 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.937 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:26.937 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273077+0000", 2024-09-21T21:42:26.937 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:26.937 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.937 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.937 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:26.937 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273077+0000", 2024-09-21T21:42:26.937 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:26.937 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.937 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.937 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:26.937 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273078+0000", 2024-09-21T21:42:26.937 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5699999999999998e-07 2024-09-21T21:42:26.937 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.938 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.938 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:26.938 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273083+0000", 2024-09-21T21:42:26.938 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8670000000000002e-06 2024-09-21T21:42:26.938 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.938 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.938 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:26.938 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273088+0000", 2024-09-21T21:42:26.938 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8370000000000002e-06 2024-09-21T21:42:26.938 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.938 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.938 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:26.938 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273093+0000", 2024-09-21T21:42:26.938 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5709999999999999e-06 2024-09-21T21:42:26.938 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.939 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.939 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:26.939 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277968+0000", 2024-09-21T21:42:26.939 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048753909999999998 2024-09-21T21:42:26.939 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.939 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.939 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:26.939 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277971+0000", 2024-09-21T21:42:26.939 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0189999999999998e-06 2024-09-21T21:42:26.939 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.939 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.939 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:26.939 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246485+0000", 2024-09-21T21:42:26.939 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96851399299999996 2024-09-21T21:42:26.940 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.940 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.940 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:26.940 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246536+0000", 2024-09-21T21:42:26.940 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0430000000000003e-05 2024-09-21T21:42:26.940 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.940 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.940 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:26.940 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246579+0000", 2024-09-21T21:42:26.940 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3569000000000002e-05 2024-09-21T21:42:26.940 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:26.940 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:26.940 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:26.940 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.940 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.941 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19756 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6500352~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:26.941 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273099+0000", 2024-09-21T21:42:26.941 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 35.654760152999998, 2024-09-21T21:42:26.941 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97359272799999996, 2024-09-21T21:42:26.941 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:26.941 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:26.941 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:26.941 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:26.941 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:26.941 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:26.941 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19756 2024-09-21T21:42:26.941 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.941 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:26.941 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.941 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:26.942 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273099+0000", 2024-09-21T21:42:26.942 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:26.942 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.942 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.942 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:26.942 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273099+0000", 2024-09-21T21:42:26.942 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:26.942 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.942 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.942 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:26.942 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273099+0000", 2024-09-21T21:42:26.942 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0900000000000002e-07 2024-09-21T21:42:26.942 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.942 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.943 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:26.943 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273101+0000", 2024-09-21T21:42:26.943 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0320000000000002e-06 2024-09-21T21:42:26.943 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.943 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.943 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:26.943 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273107+0000", 2024-09-21T21:42:26.943 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2830000000000001e-06 2024-09-21T21:42:26.943 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.943 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.943 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:26.943 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273111+0000", 2024-09-21T21:42:26.943 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2230000000000001e-06 2024-09-21T21:42:26.943 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.943 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.944 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:26.944 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277994+0000", 2024-09-21T21:42:26.944 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048837070000000002 2024-09-21T21:42:26.944 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.944 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.944 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:26.944 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278000+0000", 2024-09-21T21:42:26.944 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3149999999999999e-06 2024-09-21T21:42:26.944 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.944 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.944 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:26.944 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246607+0000", 2024-09-21T21:42:26.944 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.968606839 2024-09-21T21:42:26.944 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.945 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.945 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:26.945 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246651+0000", 2024-09-21T21:42:26.945 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4734999999999998e-05 2024-09-21T21:42:26.945 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.945 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.945 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:26.945 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246692+0000", 2024-09-21T21:42:26.945 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0185000000000002e-05 2024-09-21T21:42:26.945 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:26.945 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:26.945 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:26.945 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.945 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.945 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19757 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 5812224~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:26.946 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273116+0000", 2024-09-21T21:42:26.946 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 35.654742495999997, 2024-09-21T21:42:26.946 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97367813999999997, 2024-09-21T21:42:26.946 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:26.946 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:26.946 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:26.946 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:26.946 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:26.946 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:26.946 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19757 2024-09-21T21:42:26.946 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.946 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:26.946 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.946 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:26.946 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273116+0000", 2024-09-21T21:42:26.947 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:26.947 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.947 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.947 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:26.947 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273116+0000", 2024-09-21T21:42:26.947 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:26.947 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.947 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.947 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:26.947 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273117+0000", 2024-09-21T21:42:26.947 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0999999999999999e-07 2024-09-21T21:42:26.947 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.947 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.947 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:26.947 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273119+0000", 2024-09-21T21:42:26.948 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9970000000000001e-06 2024-09-21T21:42:26.948 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.948 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.948 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:26.948 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273125+0000", 2024-09-21T21:42:26.948 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9540000000000001e-06 2024-09-21T21:42:26.948 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.948 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.948 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:26.948 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273135+0000", 2024-09-21T21:42:26.948 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0597999999999999e-05 2024-09-21T21:42:26.948 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.948 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.948 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:26.948 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278019+0000", 2024-09-21T21:42:26.949 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048834849999999999 2024-09-21T21:42:26.949 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.949 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.949 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:26.949 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278022+0000", 2024-09-21T21:42:26.949 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9560000000000002e-06 2024-09-21T21:42:26.949 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.949 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.949 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:26.949 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246716+0000", 2024-09-21T21:42:26.949 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.968694202 2024-09-21T21:42:26.949 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.949 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.949 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:26.949 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246758+0000", 2024-09-21T21:42:26.950 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1607000000000003e-05 2024-09-21T21:42:26.950 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.950 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.950 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:26.950 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246795+0000", 2024-09-21T21:42:26.950 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6931000000000003e-05 2024-09-21T21:42:26.950 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:26.950 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:26.950 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:26.950 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.950 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.950 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19758 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6529024~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:26.951 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273142+0000", 2024-09-21T21:42:26.951 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 35.654717275000003, 2024-09-21T21:42:26.951 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97375924599999997, 2024-09-21T21:42:26.951 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:26.951 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:26.951 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:26.951 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:26.951 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:26.951 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:26.951 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19758 2024-09-21T21:42:26.951 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.951 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:26.951 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.951 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:26.952 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273142+0000", 2024-09-21T21:42:26.952 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:26.952 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.952 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.952 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:26.952 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273142+0000", 2024-09-21T21:42:26.952 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:26.952 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.952 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.952 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:26.952 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273142+0000", 2024-09-21T21:42:26.952 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0200000000000003e-07 2024-09-21T21:42:26.952 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.952 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.952 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:26.953 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273144+0000", 2024-09-21T21:42:26.953 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1390000000000002e-06 2024-09-21T21:42:26.953 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.953 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.953 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:26.953 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273150+0000", 2024-09-21T21:42:26.953 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6409999999999999e-06 2024-09-21T21:42:26.953 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.953 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.953 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:26.953 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273157+0000", 2024-09-21T21:42:26.953 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1359999999999999e-06 2024-09-21T21:42:26.953 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.953 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.954 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:26.954 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278050+0000", 2024-09-21T21:42:26.954 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048926170000000001 2024-09-21T21:42:26.954 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.954 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.954 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:26.954 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278053+0000", 2024-09-21T21:42:26.954 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4879999999999999e-06 2024-09-21T21:42:26.954 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.954 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.954 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:26.954 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246819+0000", 2024-09-21T21:42:26.954 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96876590399999996 2024-09-21T21:42:26.954 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.954 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.955 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:26.955 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246856+0000", 2024-09-21T21:42:26.955 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7287999999999999e-05 2024-09-21T21:42:26.955 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.955 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.955 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:26.955 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246901+0000", 2024-09-21T21:42:26.955 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4631e-05 2024-09-21T21:42:26.955 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:26.955 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:26.955 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:26.955 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.955 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.955 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19759 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 7933952~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:26.956 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273172+0000", 2024-09-21T21:42:26.956 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 35.654687011, 2024-09-21T21:42:26.956 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97384789500000002, 2024-09-21T21:42:26.956 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:26.956 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:26.956 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:26.956 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:26.956 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:26.956 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:26.956 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19759 2024-09-21T21:42:26.956 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.956 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:26.956 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.956 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:26.956 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273172+0000", 2024-09-21T21:42:26.957 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:26.957 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.957 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.957 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:26.957 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273172+0000", 2024-09-21T21:42:26.957 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:26.957 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.957 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.957 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:26.957 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273172+0000", 2024-09-21T21:42:26.957 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6100000000000002e-07 2024-09-21T21:42:26.957 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.957 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.957 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:26.958 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273174+0000", 2024-09-21T21:42:26.958 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.691e-06 2024-09-21T21:42:26.958 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.958 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.958 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:26.958 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273178+0000", 2024-09-21T21:42:26.958 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7579999999999998e-06 2024-09-21T21:42:26.958 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.958 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.958 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:26.958 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273180+0000", 2024-09-21T21:42:26.958 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7199999999999998e-06 2024-09-21T21:42:26.958 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.958 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.959 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:26.959 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278077+0000", 2024-09-21T21:42:26.959 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048969529999999999 2024-09-21T21:42:26.959 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.959 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.959 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:26.959 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278080+0000", 2024-09-21T21:42:26.959 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.066e-06 2024-09-21T21:42:26.959 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.959 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.959 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:26.959 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246925+0000", 2024-09-21T21:42:26.959 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96884490199999995 2024-09-21T21:42:26.959 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.959 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.960 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:26.960 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246974+0000", 2024-09-21T21:42:26.960 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8387999999999998e-05 2024-09-21T21:42:26.960 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.960 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.960 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:26.960 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247020+0000", 2024-09-21T21:42:26.960 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6156000000000003e-05 2024-09-21T21:42:26.960 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:26.960 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:26.960 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:26.960 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.960 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.960 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19760 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 7991296~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:26.960 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273220+0000", 2024-09-21T21:42:26.961 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 35.654638497000001, 2024-09-21T21:42:26.961 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97403125499999998, 2024-09-21T21:42:26.961 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:26.961 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:26.961 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:26.961 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:26.961 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:26.961 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:26.961 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19760 2024-09-21T21:42:26.961 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.961 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:26.961 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.961 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:26.961 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273220+0000", 2024-09-21T21:42:26.961 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:26.962 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.962 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.962 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:26.962 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273220+0000", 2024-09-21T21:42:26.962 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:26.962 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.962 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.962 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:26.962 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273221+0000", 2024-09-21T21:42:26.962 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.65e-07 2024-09-21T21:42:26.962 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.962 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.962 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:26.962 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273228+0000", 2024-09-21T21:42:26.963 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6040000000000004e-06 2024-09-21T21:42:26.963 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.963 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.963 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:26.963 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273233+0000", 2024-09-21T21:42:26.963 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1569999999999999e-06 2024-09-21T21:42:26.963 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.963 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.963 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:26.963 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273238+0000", 2024-09-21T21:42:26.963 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0050000000000004e-06 2024-09-21T21:42:26.963 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.963 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.963 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:26.963 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278107+0000", 2024-09-21T21:42:26.964 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00486857 2024-09-21T21:42:26.964 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.964 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.964 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:26.964 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278111+0000", 2024-09-21T21:42:26.964 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6389999999999998e-06 2024-09-21T21:42:26.964 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.964 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.964 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:26.964 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247069+0000", 2024-09-21T21:42:26.964 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96895796899999997 2024-09-21T21:42:26.964 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.964 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.964 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:26.965 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247166+0000", 2024-09-21T21:42:26.965 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.7158999999999996e-05 2024-09-21T21:42:26.965 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.965 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.965 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:26.965 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247252+0000", 2024-09-21T21:42:26.965 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5886999999999999e-05 2024-09-21T21:42:26.965 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:26.965 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:26.965 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:26.965 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.965 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.965 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19761 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 7962624~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:26.965 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273246+0000", 2024-09-21T21:42:26.965 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 35.654613437000002, 2024-09-21T21:42:26.965 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97415972200000001, 2024-09-21T21:42:26.966 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:26.966 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:26.966 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:26.966 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:26.966 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:26.966 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:26.966 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19761 2024-09-21T21:42:26.966 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.966 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:26.966 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.966 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:26.966 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273246+0000", 2024-09-21T21:42:26.966 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:26.966 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.967 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.967 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:26.967 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273246+0000", 2024-09-21T21:42:26.967 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:26.967 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.967 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.967 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:26.967 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273246+0000", 2024-09-21T21:42:26.967 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5100000000000001e-07 2024-09-21T21:42:26.967 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.967 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.967 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:26.967 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273248+0000", 2024-09-21T21:42:26.967 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1550000000000001e-06 2024-09-21T21:42:26.967 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.968 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.968 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:26.968 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273254+0000", 2024-09-21T21:42:26.968 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.964e-06 2024-09-21T21:42:26.968 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.968 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.968 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:26.968 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273259+0000", 2024-09-21T21:42:26.968 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5499999999999996e-06 2024-09-21T21:42:26.968 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.968 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.968 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:26.968 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278130+0000", 2024-09-21T21:42:26.968 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048715779999999997 2024-09-21T21:42:26.968 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.969 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.969 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:26.969 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278133+0000", 2024-09-21T21:42:26.969 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8739999999999999e-06 2024-09-21T21:42:26.969 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.969 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.969 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:26.969 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247270+0000", 2024-09-21T21:42:26.969 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.969137099 2024-09-21T21:42:26.969 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.969 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.969 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:26.969 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247338+0000", 2024-09-21T21:42:26.969 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7717000000000002e-05 2024-09-21T21:42:26.969 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.970 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.970 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:26.970 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247405+0000", 2024-09-21T21:42:26.970 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7434000000000005e-05 2024-09-21T21:42:26.970 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:26.970 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:26.970 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:26.970 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.970 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.970 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19769 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 5840896~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:26.970 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273287+0000", 2024-09-21T21:42:26.970 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 35.654572152999997, 2024-09-21T21:42:26.970 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97435390200000005, 2024-09-21T21:42:26.970 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:26.971 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:26.971 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:26.971 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:26.971 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:26.971 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:26.971 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19769 2024-09-21T21:42:26.971 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.971 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:26.971 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.971 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:26.971 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273287+0000", 2024-09-21T21:42:26.971 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:26.971 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.971 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.971 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:26.972 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273287+0000", 2024-09-21T21:42:26.972 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:26.972 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.972 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.972 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:26.972 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273287+0000", 2024-09-21T21:42:26.972 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6699999999999999e-07 2024-09-21T21:42:26.972 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.972 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.972 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:26.972 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273289+0000", 2024-09-21T21:42:26.972 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.708e-06 2024-09-21T21:42:26.972 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.972 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.973 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:26.973 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273296+0000", 2024-09-21T21:42:26.973 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8109999999999997e-06 2024-09-21T21:42:26.973 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.973 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.973 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:26.973 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273299+0000", 2024-09-21T21:42:26.973 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0740000000000001e-06 2024-09-21T21:42:26.973 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.973 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.973 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:26.973 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278159+0000", 2024-09-21T21:42:26.973 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048603240000000001 2024-09-21T21:42:26.973 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.973 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.974 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:26.974 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278162+0000", 2024-09-21T21:42:26.974 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1930000000000002e-06 2024-09-21T21:42:26.974 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.974 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.974 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:26.974 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247444+0000", 2024-09-21T21:42:26.974 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96928199100000001 2024-09-21T21:42:26.974 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.974 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.975 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:26.975 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247533+0000", 2024-09-21T21:42:26.975 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8490000000000001e-05 2024-09-21T21:42:26.975 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.975 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.975 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:26.975 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247641+0000", 2024-09-21T21:42:26.975 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000107944 2024-09-21T21:42:26.975 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:26.975 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:26.975 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:26.975 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.975 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.975 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19770 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 5869568~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:26.976 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273313+0000", 2024-09-21T21:42:26.976 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 35.654546050999997, 2024-09-21T21:42:26.976 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97449038899999996, 2024-09-21T21:42:26.976 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:26.976 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:26.976 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:26.976 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:26.976 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:26.976 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:26.976 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19770 2024-09-21T21:42:26.976 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.976 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:26.976 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.976 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:26.976 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273313+0000", 2024-09-21T21:42:26.977 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:26.977 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.977 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.977 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:26.977 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273313+0000", 2024-09-21T21:42:26.977 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:26.977 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.977 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.977 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:26.977 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273313+0000", 2024-09-21T21:42:26.977 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5699999999999998e-07 2024-09-21T21:42:26.977 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.977 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.977 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:26.978 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273315+0000", 2024-09-21T21:42:26.978 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0099999999999998e-06 2024-09-21T21:42:26.978 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.978 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.978 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:26.978 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273321+0000", 2024-09-21T21:42:26.978 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7529999999999998e-06 2024-09-21T21:42:26.978 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.978 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.978 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:26.978 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273328+0000", 2024-09-21T21:42:26.978 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9009999999999997e-06 2024-09-21T21:42:26.978 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.978 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.979 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:26.979 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278183+0000", 2024-09-21T21:42:26.979 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048547770000000002 2024-09-21T21:42:26.979 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.979 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.979 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:26.979 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278186+0000", 2024-09-21T21:42:26.979 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3100000000000001e-06 2024-09-21T21:42:26.979 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.979 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.979 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:26.979 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247676+0000", 2024-09-21T21:42:26.979 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96949004299999997 2024-09-21T21:42:26.979 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.979 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.980 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:26.980 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247757+0000", 2024-09-21T21:42:26.980 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0479e-05 2024-09-21T21:42:26.980 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.980 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.980 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:26.980 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247803+0000", 2024-09-21T21:42:26.980 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6758999999999997e-05 2024-09-21T21:42:26.980 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:26.980 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:26.980 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:26.980 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.980 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.981 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19771 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6586368~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:26.981 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273335+0000", 2024-09-21T21:42:26.981 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 35.654524264000003, 2024-09-21T21:42:26.981 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97458947699999998, 2024-09-21T21:42:26.981 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:26.981 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:26.981 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:26.981 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:26.981 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:26.981 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:26.981 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19771 2024-09-21T21:42:26.981 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.981 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:26.981 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.981 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:26.982 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273335+0000", 2024-09-21T21:42:26.982 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:26.982 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.982 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.982 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:26.982 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273335+0000", 2024-09-21T21:42:26.982 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:26.982 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.982 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.982 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:26.982 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273335+0000", 2024-09-21T21:42:26.982 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3599999999999999e-07 2024-09-21T21:42:26.982 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.982 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.983 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:26.983 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273337+0000", 2024-09-21T21:42:26.983 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0789999999999999e-06 2024-09-21T21:42:26.983 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.983 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.983 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:26.983 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273343+0000", 2024-09-21T21:42:26.983 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6280000000000002e-06 2024-09-21T21:42:26.983 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.983 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.983 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:26.983 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273346+0000", 2024-09-21T21:42:26.983 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.507e-06 2024-09-21T21:42:26.983 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.983 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.984 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:26.984 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278206+0000", 2024-09-21T21:42:26.984 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048600969999999999 2024-09-21T21:42:26.984 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.984 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.984 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:26.984 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278212+0000", 2024-09-21T21:42:26.984 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2079999999999999e-06 2024-09-21T21:42:26.984 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.984 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.984 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:26.984 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247828+0000", 2024-09-21T21:42:26.984 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.969616547 2024-09-21T21:42:26.984 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.985 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.985 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:26.985 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247881+0000", 2024-09-21T21:42:26.985 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3180999999999998e-05 2024-09-21T21:42:26.985 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.985 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.985 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:26.985 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247924+0000", 2024-09-21T21:42:26.985 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2894e-05 2024-09-21T21:42:26.985 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:26.985 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:26.985 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:26.985 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.985 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.986 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19772 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6557696~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:26.986 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273358+0000", 2024-09-21T21:42:26.986 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 35.654500884000001, 2024-09-21T21:42:26.986 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97467031800000004, 2024-09-21T21:42:26.986 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:26.986 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:26.986 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:26.986 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:26.986 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:26.986 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:26.986 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19772 2024-09-21T21:42:26.986 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.986 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:26.986 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.987 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:26.987 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273358+0000", 2024-09-21T21:42:26.987 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:26.987 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.987 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.987 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:26.987 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273358+0000", 2024-09-21T21:42:26.987 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:26.987 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.987 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.987 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:26.987 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273358+0000", 2024-09-21T21:42:26.987 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4499999999999998e-07 2024-09-21T21:42:26.987 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.988 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.988 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:26.988 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273360+0000", 2024-09-21T21:42:26.988 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9190000000000002e-06 2024-09-21T21:42:26.988 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.988 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.988 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:26.988 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273366+0000", 2024-09-21T21:42:26.988 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.891e-06 2024-09-21T21:42:26.988 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.988 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.988 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:26.988 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273371+0000", 2024-09-21T21:42:26.988 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1880000000000001e-06 2024-09-21T21:42:26.989 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.989 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.989 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:26.989 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278237+0000", 2024-09-21T21:42:26.989 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.004865472 2024-09-21T21:42:26.989 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.989 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.989 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:26.989 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278240+0000", 2024-09-21T21:42:26.989 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9550000000000001e-06 2024-09-21T21:42:26.989 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.989 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.989 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:26.989 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247949+0000", 2024-09-21T21:42:26.989 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.969709127 2024-09-21T21:42:26.990 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.990 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.990 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:26.990 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247990+0000", 2024-09-21T21:42:26.990 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1202000000000003e-05 2024-09-21T21:42:26.990 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.990 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.990 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:26.990 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248028+0000", 2024-09-21T21:42:26.990 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8219000000000001e-05 2024-09-21T21:42:26.990 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:26.990 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:26.990 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:26.990 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.991 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.991 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19773 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6615040~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:26.991 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273390+0000", 2024-09-21T21:42:26.991 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 35.654469022999997, 2024-09-21T21:42:26.991 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97476086200000001, 2024-09-21T21:42:26.991 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:26.991 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:26.991 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:26.991 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:26.991 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:26.991 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:26.991 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19773 2024-09-21T21:42:26.991 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.991 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:26.991 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.992 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:26.992 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273390+0000", 2024-09-21T21:42:26.992 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:26.992 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.992 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.992 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:26.992 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273390+0000", 2024-09-21T21:42:26.992 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:26.992 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.992 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.992 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:26.992 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273390+0000", 2024-09-21T21:42:26.992 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9999999999999999e-07 2024-09-21T21:42:26.992 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.992 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.993 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:26.993 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273392+0000", 2024-09-21T21:42:26.993 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0640000000000001e-06 2024-09-21T21:42:26.993 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.993 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.993 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:26.993 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273397+0000", 2024-09-21T21:42:26.993 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.113e-06 2024-09-21T21:42:26.993 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.993 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.993 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:26.993 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273401+0000", 2024-09-21T21:42:26.993 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3119999999999998e-06 2024-09-21T21:42:26.993 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.993 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.994 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:26.994 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278271+0000", 2024-09-21T21:42:26.994 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048698880000000002 2024-09-21T21:42:26.994 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.994 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.994 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:26.994 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278278+0000", 2024-09-21T21:42:26.994 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1210000000000001e-06 2024-09-21T21:42:26.994 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.994 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.994 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:26.994 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248060+0000", 2024-09-21T21:42:26.994 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96978224999999996 2024-09-21T21:42:26.994 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.994 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.995 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:26.995 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248111+0000", 2024-09-21T21:42:26.995 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.096e-05 2024-09-21T21:42:26.995 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.995 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.995 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:26.995 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248151+0000", 2024-09-21T21:42:26.995 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9854000000000002e-05 2024-09-21T21:42:26.995 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:26.995 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:26.995 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:26.995 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.995 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.995 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19784 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6643712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:26.995 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273405+0000", 2024-09-21T21:42:26.996 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 35.654454301999998, 2024-09-21T21:42:26.996 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97491798900000004, 2024-09-21T21:42:26.996 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:26.996 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:26.996 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:26.996 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:26.996 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:26.996 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:26.996 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19784 2024-09-21T21:42:26.996 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.996 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:26.996 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.996 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:26.996 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273405+0000", 2024-09-21T21:42:26.997 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:26.997 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.997 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.997 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:26.997 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273405+0000", 2024-09-21T21:42:26.997 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:26.997 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.997 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.997 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:26.997 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273405+0000", 2024-09-21T21:42:26.997 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2100000000000001e-07 2024-09-21T21:42:26.997 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.997 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.997 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:26.997 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273406+0000", 2024-09-21T21:42:26.998 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2049999999999999e-06 2024-09-21T21:42:26.998 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.998 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.998 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:26.998 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273409+0000", 2024-09-21T21:42:26.998 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0599999999999999e-06 2024-09-21T21:42:26.998 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.998 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.998 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:26.998 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273412+0000", 2024-09-21T21:42:26.998 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0730000000000001e-06 2024-09-21T21:42:26.998 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.998 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.998 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:26.999 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278317+0000", 2024-09-21T21:42:26.999 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0049045520000000004 2024-09-21T21:42:26.999 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.999 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.999 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:26.999 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278321+0000", 2024-09-21T21:42:26.999 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6249999999999998e-06 2024-09-21T21:42:26.999 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.999 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.999 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:26.999 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248179+0000", 2024-09-21T21:42:26.999 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96985744500000004 2024-09-21T21:42:26.999 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:26.999 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:26.999 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:27.000 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248253+0000", 2024-09-21T21:42:27.000 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3829000000000001e-05 2024-09-21T21:42:27.000 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.000 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.000 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:27.000 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248323+0000", 2024-09-21T21:42:27.000 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9979000000000002e-05 2024-09-21T21:42:27.000 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:27.000 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:27.000 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:27.000 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.000 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.000 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19785 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6672384~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:27.000 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273415+0000", 2024-09-21T21:42:27.000 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 35.654443716000003, 2024-09-21T21:42:27.001 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97500777800000005, 2024-09-21T21:42:27.001 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:27.001 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:27.001 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:27.001 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:27.001 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:27.001 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:27.001 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19785 2024-09-21T21:42:27.001 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.001 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:27.001 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.001 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:27.001 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273415+0000", 2024-09-21T21:42:27.001 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:27.002 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.002 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.002 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:27.002 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273415+0000", 2024-09-21T21:42:27.002 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:27.002 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.002 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.002 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:27.002 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273415+0000", 2024-09-21T21:42:27.002 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0100000000000001e-07 2024-09-21T21:42:27.002 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.002 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.002 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:27.002 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273417+0000", 2024-09-21T21:42:27.003 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.114e-06 2024-09-21T21:42:27.003 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.003 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.003 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:27.003 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273420+0000", 2024-09-21T21:42:27.003 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6770000000000001e-06 2024-09-21T21:42:27.003 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.003 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.003 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:27.003 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273423+0000", 2024-09-21T21:42:27.003 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5450000000000002e-06 2024-09-21T21:42:27.003 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.003 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.003 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:27.003 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278355+0000", 2024-09-21T21:42:27.004 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0049318590000000002 2024-09-21T21:42:27.004 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.004 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.004 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:27.004 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278360+0000", 2024-09-21T21:42:27.004 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3079999999999998e-06 2024-09-21T21:42:27.004 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.004 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.004 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:27.004 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248349+0000", 2024-09-21T21:42:27.004 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96998909899999997 2024-09-21T21:42:27.004 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.004 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.004 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:27.004 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248387+0000", 2024-09-21T21:42:27.005 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7657000000000002e-05 2024-09-21T21:42:27.005 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.005 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.005 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:27.005 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248423+0000", 2024-09-21T21:42:27.005 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6318e-05 2024-09-21T21:42:27.005 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:27.005 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:27.005 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:27.005 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.005 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.005 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19842 3.d 3:b7fa2ca0:::rbd_data.112157a479c.000000000000004d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:27.005 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273426+0000", 2024-09-21T21:42:27.005 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 35.654432663999998, 2024-09-21T21:42:27.006 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.98136526800000001, 2024-09-21T21:42:27.006 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:27.006 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:27.006 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:27.006 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:27.006 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:27.006 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:27.006 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19842 2024-09-21T21:42:27.006 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.006 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:27.006 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.006 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:27.006 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273426+0000", 2024-09-21T21:42:27.006 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:27.007 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.007 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.007 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:27.007 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273426+0000", 2024-09-21T21:42:27.007 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:27.007 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.007 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.007 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:27.007 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273427+0000", 2024-09-21T21:42:27.007 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.72e-07 2024-09-21T21:42:27.007 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.007 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.007 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:27.008 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275836+0000", 2024-09-21T21:42:27.008 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00240909 2024-09-21T21:42:27.008 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.008 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.008 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:27.008 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275844+0000", 2024-09-21T21:42:27.008 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3750000000000003e-06 2024-09-21T21:42:27.008 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.008 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.008 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:27.008 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275849+0000", 2024-09-21T21:42:27.008 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0100000000000003e-06 2024-09-21T21:42:27.008 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.008 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.008 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:27.008 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278394+0000", 2024-09-21T21:42:27.008 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025449890000000001 2024-09-21T21:42:27.009 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.009 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.009 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:27.009 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278402+0000", 2024-09-21T21:42:27.009 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6359999999999994e-06 2024-09-21T21:42:27.009 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.009 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.009 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:27.009 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248454+0000", 2024-09-21T21:42:27.009 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97005259499999996 2024-09-21T21:42:27.009 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.009 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.009 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:27.009 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248576+0000", 2024-09-21T21:42:27.009 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000122127 2024-09-21T21:42:27.010 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.010 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.010 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 2", 2024-09-21T21:42:27.010 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248844+0000", 2024-09-21T21:42:27.010 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00026784699999999999 2024-09-21T21:42:27.010 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.010 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.010 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:27.010 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.252487+0000", 2024-09-21T21:42:27.010 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0036427920000000002 2024-09-21T21:42:27.010 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.010 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.010 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:27.011 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.254691+0000", 2024-09-21T21:42:27.011 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002203899 2024-09-21T21:42:27.011 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.011 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.011 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:27.011 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.254748+0000", 2024-09-21T21:42:27.011 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7163000000000001e-05 2024-09-21T21:42:27.011 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.011 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.011 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:27.011 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.254792+0000", 2024-09-21T21:42:27.011 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3473000000000002e-05 2024-09-21T21:42:27.011 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:27.011 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:27.012 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:27.012 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.012 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.012 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19843 3.d 3:b7fa2ca0:::rbd_data.112157a479c.000000000000004d:head [write 0~0] snapc 21=[21] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:27.012 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.275860+0000", 2024-09-21T21:42:27.012 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 35.651998763999998, 2024-09-21T21:42:27.012 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.98183651999999999, 2024-09-21T21:42:27.012 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:27.012 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:27.012 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:27.012 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:27.012 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:27.012 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:27.012 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19843 2024-09-21T21:42:27.012 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.013 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:27.013 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.013 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:27.013 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275860+0000", 2024-09-21T21:42:27.013 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:27.013 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.013 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.013 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:27.013 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275860+0000", 2024-09-21T21:42:27.013 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:27.013 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.013 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.013 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:27.013 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275869+0000", 2024-09-21T21:42:27.014 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6300000000000004e-06 2024-09-21T21:42:27.014 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.014 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.014 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:27.014 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275871+0000", 2024-09-21T21:42:27.014 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2060000000000001e-06 2024-09-21T21:42:27.014 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.014 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.014 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:27.014 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275876+0000", 2024-09-21T21:42:27.014 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8180000000000003e-06 2024-09-21T21:42:27.014 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.014 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.014 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:27.015 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275880+0000", 2024-09-21T21:42:27.015 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1829999999999996e-06 2024-09-21T21:42:27.015 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.015 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.015 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:27.015 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278416+0000", 2024-09-21T21:42:27.015 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025361149999999998 2024-09-21T21:42:27.015 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.015 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.015 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:27.015 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278419+0000", 2024-09-21T21:42:27.015 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.163e-06 2024-09-21T21:42:27.015 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.015 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.015 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:27.016 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.250249+0000", 2024-09-21T21:42:27.016 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97182984299999997 2024-09-21T21:42:27.016 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.016 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.016 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:27.016 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.250321+0000", 2024-09-21T21:42:27.016 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1656000000000006e-05 2024-09-21T21:42:27.016 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.016 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.016 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 2", 2024-09-21T21:42:27.016 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.250419+0000", 2024-09-21T21:42:27.016 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.8277999999999997e-05 2024-09-21T21:42:27.016 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.016 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.017 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:27.017 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.252516+0000", 2024-09-21T21:42:27.017 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002097224 2024-09-21T21:42:27.017 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.017 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.017 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:27.017 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.257643+0000", 2024-09-21T21:42:27.017 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0051262319999999997 2024-09-21T21:42:27.017 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.017 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.017 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:27.017 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.257673+0000", 2024-09-21T21:42:27.017 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0023e-05 2024-09-21T21:42:27.017 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.017 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.018 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:27.018 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.257697+0000", 2024-09-21T21:42:27.018 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4148999999999999e-05 2024-09-21T21:42:27.018 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:27.018 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:27.018 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:27.018 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.018 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.018 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5626.0:12334 3.3 3:c643a86c:::rbd_data.1121faebdfe3.0000000000000002:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:42:27.018 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:14.411628+0000", 2024-09-21T21:42:27.018 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.516230465, 2024-09-21T21:42:27.018 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0086947559999999, 2024-09-21T21:42:27.018 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:27.018 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:27.019 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:27.019 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:27.019 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5626", 2024-09-21T21:42:27.019 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:27.019 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 12334 2024-09-21T21:42:27.019 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.019 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:27.019 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.019 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:27.019 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411628+0000", 2024-09-21T21:42:27.019 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:27.019 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.019 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.019 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:27.020 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411628+0000", 2024-09-21T21:42:27.020 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:27.020 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.020 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.020 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:27.020 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411629+0000", 2024-09-21T21:42:27.020 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1200000000000002e-07 2024-09-21T21:42:27.020 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.020 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.020 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:27.020 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411633+0000", 2024-09-21T21:42:27.020 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1699999999999999e-06 2024-09-21T21:42:27.020 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.020 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.021 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:27.021 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411645+0000", 2024-09-21T21:42:27.021 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1328999999999999e-05 2024-09-21T21:42:27.021 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.021 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.021 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:27.021 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411652+0000", 2024-09-21T21:42:27.021 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9990000000000002e-06 2024-09-21T21:42:27.021 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.021 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.021 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:27.021 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411717+0000", 2024-09-21T21:42:27.021 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5519999999999996e-05 2024-09-21T21:42:27.022 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.022 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.022 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op must wait for map", 2024-09-21T21:42:27.022 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411725+0000", 2024-09-21T21:42:27.022 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2609999999999996e-06 2024-09-21T21:42:27.022 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.022 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.022 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:27.022 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415904+0000", 2024-09-21T21:42:27.022 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0041792749999999997 2024-09-21T21:42:27.022 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.022 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.022 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:27.022 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415913+0000", 2024-09-21T21:42:27.022 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.054e-06 2024-09-21T21:42:27.023 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.023 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.023 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:27.023 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420096+0000", 2024-09-21T21:42:27.023 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0041835670000001 2024-09-21T21:42:27.023 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.023 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.023 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:27.023 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420216+0000", 2024-09-21T21:42:27.023 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000120025 2024-09-21T21:42:27.023 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.023 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.023 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:27.023 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420323+0000", 2024-09-21T21:42:27.024 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010674400000000001 2024-09-21T21:42:27.024 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:27.024 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:27.024 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:27.024 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.024 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.024 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5626.0:12339 3.3 3:c643a86c:::rbd_data.1121faebdfe3.0000000000000002:head [read 4096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:42:27.024 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:14.411749+0000", 2024-09-21T21:42:27.024 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.516110278999999, 2024-09-21T21:42:27.024 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.008818056, 2024-09-21T21:42:27.024 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:27.024 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:27.024 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:27.024 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:27.024 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5626", 2024-09-21T21:42:27.025 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:27.025 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 12339 2024-09-21T21:42:27.025 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.025 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:27.025 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.025 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:27.025 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411749+0000", 2024-09-21T21:42:27.025 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:27.025 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.025 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.025 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:27.025 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411749+0000", 2024-09-21T21:42:27.025 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:27.025 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.026 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.026 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:27.026 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411749+0000", 2024-09-21T21:42:27.026 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0900000000000002e-07 2024-09-21T21:42:27.026 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.026 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.026 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:27.026 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411752+0000", 2024-09-21T21:42:27.026 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5610000000000001e-06 2024-09-21T21:42:27.026 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.026 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.026 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:27.026 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411758+0000", 2024-09-21T21:42:27.026 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1890000000000004e-06 2024-09-21T21:42:27.027 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.027 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.027 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:27.027 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411763+0000", 2024-09-21T21:42:27.027 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7589999999999997e-06 2024-09-21T21:42:27.027 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.027 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.027 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:27.027 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411803+0000", 2024-09-21T21:42:27.027 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9922999999999997e-05 2024-09-21T21:42:27.027 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.027 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.027 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting_for_map not empty", 2024-09-21T21:42:27.027 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411809+0000", 2024-09-21T21:42:27.027 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3779999999999998e-06 2024-09-21T21:42:27.028 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.028 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.028 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:27.028 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415937+0000", 2024-09-21T21:42:27.028 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0041276100000000003 2024-09-21T21:42:27.028 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.028 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.028 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:27.028 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415944+0000", 2024-09-21T21:42:27.028 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8329999999999994e-06 2024-09-21T21:42:27.028 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.028 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.028 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:27.028 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420411+0000", 2024-09-21T21:42:27.028 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0044667300000001 2024-09-21T21:42:27.029 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.029 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.029 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:27.029 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420475+0000", 2024-09-21T21:42:27.029 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4356000000000005e-05 2024-09-21T21:42:27.029 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.029 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.029 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:27.029 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420567+0000", 2024-09-21T21:42:27.029 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.1308000000000001e-05 2024-09-21T21:42:27.029 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:27.029 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:27.029 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:27.029 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.029 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.030 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5626.0:12345 3.3 3:c643a86c:::rbd_data.1121faebdfe3.0000000000000002:head [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:42:27.030 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:14.411831+0000", 2024-09-21T21:42:27.030 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.516028226, 2024-09-21T21:42:27.030 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0089163649999999, 2024-09-21T21:42:27.030 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:27.030 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:27.030 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:27.030 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:27.030 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5626", 2024-09-21T21:42:27.030 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:27.030 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 12345 2024-09-21T21:42:27.030 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.030 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:27.030 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.031 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:27.031 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411831+0000", 2024-09-21T21:42:27.031 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:27.031 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.031 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.031 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:27.031 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411831+0000", 2024-09-21T21:42:27.031 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:27.031 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.031 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.031 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:27.031 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411831+0000", 2024-09-21T21:42:27.031 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.08e-07 2024-09-21T21:42:27.031 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.032 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.032 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:27.032 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411833+0000", 2024-09-21T21:42:27.032 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3089999999999998e-06 2024-09-21T21:42:27.032 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.032 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.032 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:27.032 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411840+0000", 2024-09-21T21:42:27.032 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.472e-06 2024-09-21T21:42:27.032 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.032 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.032 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:27.032 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411845+0000", 2024-09-21T21:42:27.032 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7910000000000004e-06 2024-09-21T21:42:27.033 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.033 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.033 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:27.033 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411890+0000", 2024-09-21T21:42:27.033 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5763000000000002e-05 2024-09-21T21:42:27.033 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.033 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.033 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting_for_map not empty", 2024-09-21T21:42:27.033 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411896+0000", 2024-09-21T21:42:27.033 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0190000000000003e-06 2024-09-21T21:42:27.033 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.033 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.033 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:27.033 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415967+0000", 2024-09-21T21:42:27.033 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0040705749999999999 2024-09-21T21:42:27.034 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.034 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.034 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:27.034 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415974+0000", 2024-09-21T21:42:27.034 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3200000000000002e-06 2024-09-21T21:42:27.034 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.034 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.034 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:27.034 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420594+0000", 2024-09-21T21:42:27.034 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.004620093 2024-09-21T21:42:27.034 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.034 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.034 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:27.034 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420658+0000", 2024-09-21T21:42:27.034 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3678999999999998e-05 2024-09-21T21:42:27.035 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.035 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.035 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:27.035 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420747+0000", 2024-09-21T21:42:27.035 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8936000000000004e-05 2024-09-21T21:42:27.035 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:27.035 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:27.035 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:27.035 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:27.035 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:27.035 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:27.037 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T21:42:27.038 INFO:teuthology.orchestra.run.smithi164.stdout:5777 trunc from 0x97bd200 to 0x6ea9800 2024-09-21T21:42:27.038 INFO:teuthology.orchestra.run.smithi164.stdout:5778 punch from 0x298de00 to 0x298f800, (0x1a00 bytes) 2024-09-21T21:42:27.038 INFO:teuthology.orchestra.run.smithi164.stdout:5779 punch from 0x3396400 to 0x3398800, (0x2400 bytes) 2024-09-21T21:42:27.038 INFO:teuthology.orchestra.run.smithi164.stdout:5782 read 0x5933c00 thru 0x5940362 (0xc763 bytes) 2024-09-21T21:42:27.038 INFO:teuthology.orchestra.run.smithi164.stdout:5783 read 0x18d7a00 thru 0x18d9b78 (0x2179 bytes) 2024-09-21T21:42:27.038 INFO:teuthology.orchestra.run.smithi164.stdout:5784 trunc from 0x6ea9800 to 0x23e3c00 2024-09-21T21:42:27.039 INFO:teuthology.orchestra.run.smithi164.stdout:5786 read 0x1e49400 thru 0x1e4d6a5 (0x42a6 bytes) 2024-09-21T21:42:27.039 INFO:teuthology.orchestra.run.smithi164.stdout:5788 punch from 0x164aa00 to 0x1659200, (0xe800 bytes) 2024-09-21T21:42:27.039 INFO:teuthology.orchestra.run.smithi164.stdout:5791 read 0x1b2dc00 thru 0x1b2f279 (0x167a bytes) 2024-09-21T21:42:27.039 INFO:teuthology.orchestra.run.smithi164.stdout:5793 write 0xb226e00 thru 0xb231abf (0xacc0 bytes) 2024-09-21T21:42:27.039 INFO:teuthology.orchestra.run.smithi164.stdout:5794 trunc from 0xb231c00 to 0x940fe00 2024-09-21T21:42:27.039 INFO:teuthology.orchestra.run.smithi164.stdout:5795 write 0x6a05c00 thru 0x6a07b3b (0x1f3c bytes) 2024-09-21T21:42:27.039 INFO:teuthology.orchestra.run.smithi164.stdout:5796 read 0x574d200 thru 0x575969f (0xc4a0 bytes) 2024-09-21T21:42:27.039 INFO:teuthology.orchestra.run.smithi164.stdout:5798 clone 41 order 23 su 1048576 sc 10 2024-09-21T21:42:27.039 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:27.035+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:27.137 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:27.136+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:27.181 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:27.181 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:42:27.181 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-21T21:42:27.181 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:27.189 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T21:42:27.238 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:27.237+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:27.332 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:27.332 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:42:27.332 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:42:27.333 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:42:27.333 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:27.339 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:27.337+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:27.341 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T21:42:27.440 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:27.438+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:27.485 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:27.485 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-21T21:42:27.485 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-21T21:42:27.485 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:42:27.485 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.486 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6974 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5246976~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:27.486 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.922932+0000", 2024-09-21T21:42:27.486 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 57.560179783999999, 2024-09-21T21:42:27.486 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041773352999999999, 2024-09-21T21:42:27.486 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:27.486 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:27.486 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:27.486 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:27.486 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:27.486 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:27.486 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6974 2024-09-21T21:42:27.486 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.486 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:27.487 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.487 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:27.487 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922932+0000", 2024-09-21T21:42:27.487 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:27.487 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.487 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.487 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:27.487 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922932+0000", 2024-09-21T21:42:27.487 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:27.487 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.487 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.487 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:27.487 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922933+0000", 2024-09-21T21:42:27.487 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.46e-07 2024-09-21T21:42:27.488 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.488 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.488 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:27.488 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922940+0000", 2024-09-21T21:42:27.488 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6140000000000003e-06 2024-09-21T21:42:27.488 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.488 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.488 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:27.488 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922949+0000", 2024-09-21T21:42:27.488 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6899999999999998e-06 2024-09-21T21:42:27.488 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.488 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.488 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:27.488 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922955+0000", 2024-09-21T21:42:27.489 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5140000000000001e-06 2024-09-21T21:42:27.489 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.489 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.489 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:27.489 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964547+0000", 2024-09-21T21:42:27.489 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041592854999999998 2024-09-21T21:42:27.489 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.489 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.489 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:27.489 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964617+0000", 2024-09-21T21:42:27.489 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9058000000000002e-05 2024-09-21T21:42:27.489 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.489 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.489 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:27.489 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964706+0000", 2024-09-21T21:42:27.490 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9276000000000004e-05 2024-09-21T21:42:27.490 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:27.490 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:27.490 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:27.490 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.490 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.490 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6976 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5261312~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:27.490 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.922961+0000", 2024-09-21T21:42:27.490 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 57.560150739000001, 2024-09-21T21:42:27.490 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041863648000000003, 2024-09-21T21:42:27.490 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:27.490 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:27.490 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:27.491 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:27.491 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:27.491 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:27.491 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6976 2024-09-21T21:42:27.491 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.491 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:27.491 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.491 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:27.491 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922961+0000", 2024-09-21T21:42:27.491 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:27.492 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.492 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.492 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:27.492 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922961+0000", 2024-09-21T21:42:27.492 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:27.492 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.492 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.492 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:27.492 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922962+0000", 2024-09-21T21:42:27.492 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.89e-07 2024-09-21T21:42:27.492 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.492 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.492 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:27.493 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922964+0000", 2024-09-21T21:42:27.493 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1270000000000001e-06 2024-09-21T21:42:27.493 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.493 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.493 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:27.493 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922970+0000", 2024-09-21T21:42:27.493 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8030000000000002e-06 2024-09-21T21:42:27.493 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.493 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.493 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:27.493 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922979+0000", 2024-09-21T21:42:27.493 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5559999999999996e-06 2024-09-21T21:42:27.493 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.493 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.493 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:27.494 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964734+0000", 2024-09-21T21:42:27.494 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041754415000000003 2024-09-21T21:42:27.494 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.494 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.494 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:27.494 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964775+0000", 2024-09-21T21:42:27.494 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0806999999999997e-05 2024-09-21T21:42:27.494 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.494 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.494 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:27.494 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964825+0000", 2024-09-21T21:42:27.494 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0550999999999998e-05 2024-09-21T21:42:27.494 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:27.494 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:27.495 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:27.495 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.495 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.495 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6978 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5275648~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:27.495 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.922987+0000", 2024-09-21T21:42:27.495 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 57.560124954000003, 2024-09-21T21:42:27.495 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042043876000000001, 2024-09-21T21:42:27.495 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:27.495 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:27.495 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:27.495 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:27.495 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:27.495 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:27.496 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6978 2024-09-21T21:42:27.496 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.496 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:27.496 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.496 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:27.496 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922987+0000", 2024-09-21T21:42:27.496 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:27.496 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.496 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.496 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:27.496 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922987+0000", 2024-09-21T21:42:27.496 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:27.496 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.497 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.497 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:27.497 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922988+0000", 2024-09-21T21:42:27.497 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1199999999999998e-07 2024-09-21T21:42:27.497 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.497 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.497 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:27.497 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922990+0000", 2024-09-21T21:42:27.497 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.142e-06 2024-09-21T21:42:27.497 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.497 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.497 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:27.497 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922997+0000", 2024-09-21T21:42:27.497 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.199e-06 2024-09-21T21:42:27.498 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.498 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.498 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:27.498 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923004+0000", 2024-09-21T21:42:27.498 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3599999999999998e-06 2024-09-21T21:42:27.498 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.498 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.498 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:27.498 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964856+0000", 2024-09-21T21:42:27.498 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041851521000000003 2024-09-21T21:42:27.498 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.498 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.498 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:27.499 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964944+0000", 2024-09-21T21:42:27.499 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7859e-05 2024-09-21T21:42:27.499 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.499 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.499 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:27.499 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965031+0000", 2024-09-21T21:42:27.499 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7383e-05 2024-09-21T21:42:27.499 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:27.499 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:27.499 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:27.499 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.499 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.499 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6980 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5289984~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:27.499 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923011+0000", 2024-09-21T21:42:27.500 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 57.560101617000001, 2024-09-21T21:42:27.500 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042139337999999998, 2024-09-21T21:42:27.500 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:27.500 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:27.500 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:27.500 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:27.500 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:27.500 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:27.500 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6980 2024-09-21T21:42:27.500 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.500 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:27.500 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.500 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:27.500 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923011+0000", 2024-09-21T21:42:27.501 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:27.501 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.501 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.501 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:27.501 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923011+0000", 2024-09-21T21:42:27.501 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:27.501 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.501 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.501 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:27.501 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923011+0000", 2024-09-21T21:42:27.501 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6699999999999999e-07 2024-09-21T21:42:27.501 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.501 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.501 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:27.501 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923013+0000", 2024-09-21T21:42:27.501 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0279999999999999e-06 2024-09-21T21:42:27.502 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.502 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.502 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:27.502 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923018+0000", 2024-09-21T21:42:27.502 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4790000000000004e-06 2024-09-21T21:42:27.502 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.502 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.502 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:27.502 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923029+0000", 2024-09-21T21:42:27.502 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0601999999999999e-05 2024-09-21T21:42:27.502 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.502 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.502 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:27.502 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965059+0000", 2024-09-21T21:42:27.503 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042029947999999998 2024-09-21T21:42:27.503 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.503 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.503 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:27.503 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965100+0000", 2024-09-21T21:42:27.503 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1170000000000001e-05 2024-09-21T21:42:27.503 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.503 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.503 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:27.503 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965150+0000", 2024-09-21T21:42:27.503 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9743999999999997e-05 2024-09-21T21:42:27.503 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:27.503 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:27.503 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:27.504 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.504 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.504 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6982 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5304320~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:27.504 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923036+0000", 2024-09-21T21:42:27.504 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 57.560075796, 2024-09-21T21:42:27.504 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042225666000000002, 2024-09-21T21:42:27.504 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:27.504 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:27.504 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:27.504 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:27.504 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:27.504 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:27.504 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6982 2024-09-21T21:42:27.504 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.505 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:27.505 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.505 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:27.505 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923036+0000", 2024-09-21T21:42:27.505 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:27.505 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.505 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.505 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:27.505 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923036+0000", 2024-09-21T21:42:27.505 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:27.505 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.505 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.505 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:27.505 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923037+0000", 2024-09-21T21:42:27.506 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2800000000000002e-07 2024-09-21T21:42:27.506 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.506 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.506 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:27.506 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923039+0000", 2024-09-21T21:42:27.506 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1500000000000002e-06 2024-09-21T21:42:27.506 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.506 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.506 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:27.506 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923049+0000", 2024-09-21T21:42:27.506 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0236e-05 2024-09-21T21:42:27.506 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.506 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.506 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:27.507 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923056+0000", 2024-09-21T21:42:27.507 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2540000000000003e-06 2024-09-21T21:42:27.507 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.507 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.507 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:27.507 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965173+0000", 2024-09-21T21:42:27.507 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042116686 2024-09-21T21:42:27.507 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.507 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.507 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:27.507 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965215+0000", 2024-09-21T21:42:27.507 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1696000000000002e-05 2024-09-21T21:42:27.507 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.507 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.508 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:27.508 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965262+0000", 2024-09-21T21:42:27.508 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7216000000000003e-05 2024-09-21T21:42:27.508 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:27.508 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:27.508 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:27.508 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.508 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.508 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6984 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5318656~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:27.508 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923063+0000", 2024-09-21T21:42:27.508 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 57.560049141999997, 2024-09-21T21:42:27.508 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042318651999999998, 2024-09-21T21:42:27.508 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:27.508 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:27.508 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:27.509 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:27.509 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:27.509 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:27.509 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6984 2024-09-21T21:42:27.509 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.509 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:27.509 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.509 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:27.509 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923063+0000", 2024-09-21T21:42:27.509 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:27.509 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.509 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.509 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:27.509 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923063+0000", 2024-09-21T21:42:27.509 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:27.510 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.510 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.510 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:27.510 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923064+0000", 2024-09-21T21:42:27.510 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2e-07 2024-09-21T21:42:27.510 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.510 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.510 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:27.510 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923066+0000", 2024-09-21T21:42:27.510 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.057e-06 2024-09-21T21:42:27.510 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.510 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.510 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:27.510 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923071+0000", 2024-09-21T21:42:27.511 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8810000000000001e-06 2024-09-21T21:42:27.511 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.511 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.511 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:27.511 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923078+0000", 2024-09-21T21:42:27.511 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5479999999999998e-06 2024-09-21T21:42:27.511 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.511 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.511 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:27.511 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965288+0000", 2024-09-21T21:42:27.511 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042209576999999998 2024-09-21T21:42:27.511 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.511 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.511 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:27.512 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965332+0000", 2024-09-21T21:42:27.512 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4366999999999997e-05 2024-09-21T21:42:27.512 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.512 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.512 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:27.512 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965382+0000", 2024-09-21T21:42:27.512 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9802000000000002e-05 2024-09-21T21:42:27.512 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:27.512 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:27.512 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:27.513 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.513 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.513 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6987 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5332992~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:27.513 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923085+0000", 2024-09-21T21:42:27.513 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 57.560027617000003, 2024-09-21T21:42:27.513 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042412103999999999, 2024-09-21T21:42:27.513 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:27.513 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:27.513 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:27.513 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:27.513 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:27.513 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:27.513 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6987 2024-09-21T21:42:27.514 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.514 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:27.514 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.514 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:27.514 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923085+0000", 2024-09-21T21:42:27.514 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:27.514 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.514 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.514 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:27.514 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923085+0000", 2024-09-21T21:42:27.514 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:27.514 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.515 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.515 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:27.515 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923085+0000", 2024-09-21T21:42:27.515 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.89e-07 2024-09-21T21:42:27.515 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.515 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.515 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:27.515 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923087+0000", 2024-09-21T21:42:27.515 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0810000000000001e-06 2024-09-21T21:42:27.515 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.515 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.516 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:27.516 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923098+0000", 2024-09-21T21:42:27.516 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1299e-05 2024-09-21T21:42:27.516 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.516 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.516 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:27.516 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923104+0000", 2024-09-21T21:42:27.516 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9100000000000002e-06 2024-09-21T21:42:27.516 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.516 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.516 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:27.516 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965404+0000", 2024-09-21T21:42:27.516 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042299622000000002 2024-09-21T21:42:27.516 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.516 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.516 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:27.517 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965445+0000", 2024-09-21T21:42:27.517 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0701000000000002e-05 2024-09-21T21:42:27.517 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.517 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.517 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:27.517 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965497+0000", 2024-09-21T21:42:27.517 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2101999999999997e-05 2024-09-21T21:42:27.517 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:27.517 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:27.517 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:27.517 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.517 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.518 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6989 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5347328~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:27.518 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923111+0000", 2024-09-21T21:42:27.518 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 57.560001276999998, 2024-09-21T21:42:27.518 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042505188999999999, 2024-09-21T21:42:27.518 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:27.518 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:27.518 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:27.518 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:27.518 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:27.518 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:27.518 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6989 2024-09-21T21:42:27.518 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.518 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:27.518 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.519 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:27.519 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923111+0000", 2024-09-21T21:42:27.519 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:27.519 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.519 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.519 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:27.519 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923111+0000", 2024-09-21T21:42:27.519 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:27.519 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.519 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.519 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:27.519 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923111+0000", 2024-09-21T21:42:27.519 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4499999999999997e-07 2024-09-21T21:42:27.519 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.519 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.520 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:27.520 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923114+0000", 2024-09-21T21:42:27.520 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1569999999999998e-06 2024-09-21T21:42:27.520 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.520 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.520 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:27.520 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923120+0000", 2024-09-21T21:42:27.520 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1759999999999998e-06 2024-09-21T21:42:27.520 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.520 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.520 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:27.520 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923125+0000", 2024-09-21T21:42:27.520 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2560000000000002e-06 2024-09-21T21:42:27.520 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.520 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.520 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:27.521 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965523+0000", 2024-09-21T21:42:27.521 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042398094999999997 2024-09-21T21:42:27.521 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.521 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.521 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:27.521 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965566+0000", 2024-09-21T21:42:27.521 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3393999999999998e-05 2024-09-21T21:42:27.521 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.521 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.521 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:27.521 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965616+0000", 2024-09-21T21:42:27.521 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9666000000000002e-05 2024-09-21T21:42:27.521 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:27.521 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:27.521 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:27.522 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.522 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.522 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6991 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5361664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:27.522 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923132+0000", 2024-09-21T21:42:27.522 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 57.559979988000002, 2024-09-21T21:42:27.522 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042600779999999998, 2024-09-21T21:42:27.522 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:27.522 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:27.522 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:27.522 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:27.522 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:27.522 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:27.522 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6991 2024-09-21T21:42:27.522 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.522 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:27.523 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.523 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:27.523 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923132+0000", 2024-09-21T21:42:27.523 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:27.523 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.523 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.523 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:27.523 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923132+0000", 2024-09-21T21:42:27.523 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:27.523 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.524 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.524 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:27.524 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923132+0000", 2024-09-21T21:42:27.524 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4200000000000002e-07 2024-09-21T21:42:27.524 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.524 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.524 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:27.524 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923134+0000", 2024-09-21T21:42:27.524 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2270000000000001e-06 2024-09-21T21:42:27.524 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.524 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.524 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:27.524 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923137+0000", 2024-09-21T21:42:27.524 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1930000000000002e-06 2024-09-21T21:42:27.524 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.525 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.525 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:27.525 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923139+0000", 2024-09-21T21:42:27.525 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4710000000000001e-06 2024-09-21T21:42:27.525 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.525 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.525 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:27.525 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965639+0000", 2024-09-21T21:42:27.525 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.04249965 2024-09-21T21:42:27.525 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.525 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.525 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:27.525 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965679+0000", 2024-09-21T21:42:27.525 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9852999999999999e-05 2024-09-21T21:42:27.526 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.526 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.526 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:27.526 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965733+0000", 2024-09-21T21:42:27.526 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4144000000000002e-05 2024-09-21T21:42:27.526 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:27.526 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:27.526 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:27.526 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.526 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.526 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6993 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5384192~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:27.526 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923143+0000", 2024-09-21T21:42:27.526 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 57.559969193999997, 2024-09-21T21:42:27.526 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042708182999999997, 2024-09-21T21:42:27.526 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:27.527 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:27.527 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:27.527 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:27.527 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:27.527 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:27.527 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6993 2024-09-21T21:42:27.527 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.527 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:27.527 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.527 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:27.527 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923143+0000", 2024-09-21T21:42:27.527 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:27.527 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.527 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.528 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:27.528 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923143+0000", 2024-09-21T21:42:27.528 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:27.528 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.528 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.528 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:27.528 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923143+0000", 2024-09-21T21:42:27.528 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0800000000000001e-07 2024-09-21T21:42:27.528 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.528 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.528 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:27.528 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923144+0000", 2024-09-21T21:42:27.528 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.009e-06 2024-09-21T21:42:27.528 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.529 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.529 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:27.529 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923148+0000", 2024-09-21T21:42:27.529 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.737e-06 2024-09-21T21:42:27.529 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.529 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.529 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:27.529 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923150+0000", 2024-09-21T21:42:27.529 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4669999999999998e-06 2024-09-21T21:42:27.529 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.529 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.529 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:27.529 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965760+0000", 2024-09-21T21:42:27.529 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042609399999999999 2024-09-21T21:42:27.529 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.530 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.530 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:27.530 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965801+0000", 2024-09-21T21:42:27.530 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1399999999999997e-05 2024-09-21T21:42:27.530 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.530 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.530 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:27.530 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965851+0000", 2024-09-21T21:42:27.530 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9962e-05 2024-09-21T21:42:27.530 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:27.530 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:27.530 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:27.530 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.530 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.531 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6995 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5398528~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:27.531 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923153+0000", 2024-09-21T21:42:27.531 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 57.559958829999999, 2024-09-21T21:42:27.531 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042828034000000001, 2024-09-21T21:42:27.531 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:27.531 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:27.531 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:27.531 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:27.531 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:27.531 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:27.531 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6995 2024-09-21T21:42:27.531 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.531 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:27.531 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.531 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:27.532 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923153+0000", 2024-09-21T21:42:27.532 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:27.532 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.532 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.532 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:27.532 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923153+0000", 2024-09-21T21:42:27.532 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:27.532 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.532 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.532 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:27.532 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923154+0000", 2024-09-21T21:42:27.533 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2600000000000001e-07 2024-09-21T21:42:27.533 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.533 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.533 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:27.533 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923158+0000", 2024-09-21T21:42:27.533 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0559999999999998e-06 2024-09-21T21:42:27.533 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.533 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.533 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:27.533 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923161+0000", 2024-09-21T21:42:27.533 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6519999999999999e-06 2024-09-21T21:42:27.533 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.533 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.533 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:27.534 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923170+0000", 2024-09-21T21:42:27.534 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5720000000000003e-06 2024-09-21T21:42:27.534 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.534 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.534 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:27.534 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965887+0000", 2024-09-21T21:42:27.534 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042716863000000001 2024-09-21T21:42:27.534 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.534 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.534 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:27.534 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965927+0000", 2024-09-21T21:42:27.534 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0346999999999998e-05 2024-09-21T21:42:27.534 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.534 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.535 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:27.535 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965981+0000", 2024-09-21T21:42:27.535 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4317999999999997e-05 2024-09-21T21:42:27.535 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:27.535 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:27.535 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:27.535 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.535 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.535 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6997 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5412864~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:27.535 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923174+0000", 2024-09-21T21:42:27.535 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 57.559938674999998, 2024-09-21T21:42:27.535 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042928296999999997, 2024-09-21T21:42:27.535 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:27.535 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:27.535 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:27.535 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:27.536 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:27.536 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:27.536 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6997 2024-09-21T21:42:27.536 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.536 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:27.536 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.536 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:27.536 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923174+0000", 2024-09-21T21:42:27.536 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:27.536 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.536 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.536 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:27.536 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923174+0000", 2024-09-21T21:42:27.536 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:27.537 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.537 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.537 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:27.537 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923174+0000", 2024-09-21T21:42:27.537 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.67e-07 2024-09-21T21:42:27.537 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.537 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.537 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:27.537 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923178+0000", 2024-09-21T21:42:27.537 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1330000000000001e-06 2024-09-21T21:42:27.537 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.537 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.537 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:27.537 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923182+0000", 2024-09-21T21:42:27.537 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.794e-06 2024-09-21T21:42:27.537 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.538 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.538 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:27.538 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923184+0000", 2024-09-21T21:42:27.538 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.435e-06 2024-09-21T21:42:27.538 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.538 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.538 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:27.538 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966009+0000", 2024-09-21T21:42:27.538 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042824327000000002 2024-09-21T21:42:27.538 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.538 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.538 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:27.538 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966052+0000", 2024-09-21T21:42:27.538 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3034000000000002e-05 2024-09-21T21:42:27.539 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.539 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.539 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:27.539 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966102+0000", 2024-09-21T21:42:27.539 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0306999999999998e-05 2024-09-21T21:42:27.539 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:27.539 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:27.539 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:27.539 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.539 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.539 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6999 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5427200~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:27.539 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923187+0000", 2024-09-21T21:42:27.539 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 57.559924946999999, 2024-09-21T21:42:27.539 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043104267000000002, 2024-09-21T21:42:27.539 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:27.540 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:27.540 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:27.540 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:27.540 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:27.540 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:27.540 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6999 2024-09-21T21:42:27.540 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.540 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:27.540 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.540 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:27.540 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923187+0000", 2024-09-21T21:42:27.540 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:27.540 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.540 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.541 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:27.541 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923187+0000", 2024-09-21T21:42:27.541 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:27.541 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.541 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.541 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:27.541 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923188+0000", 2024-09-21T21:42:27.541 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3999999999999998e-07 2024-09-21T21:42:27.541 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.541 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.541 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:27.541 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923189+0000", 2024-09-21T21:42:27.541 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4169999999999999e-06 2024-09-21T21:42:27.541 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.541 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.542 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:27.542 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923194+0000", 2024-09-21T21:42:27.542 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1819999999999997e-06 2024-09-21T21:42:27.542 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.542 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.542 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:27.542 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923198+0000", 2024-09-21T21:42:27.542 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.143e-06 2024-09-21T21:42:27.542 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.542 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.542 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:27.542 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966135+0000", 2024-09-21T21:42:27.542 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042936297999999998 2024-09-21T21:42:27.542 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.543 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.543 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:27.543 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966204+0000", 2024-09-21T21:42:27.543 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8989e-05 2024-09-21T21:42:27.543 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.543 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.543 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:27.543 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966292+0000", 2024-09-21T21:42:27.543 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7997999999999993e-05 2024-09-21T21:42:27.543 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:27.543 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:27.543 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:27.543 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.544 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.544 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7001 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5441536~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:27.544 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923211+0000", 2024-09-21T21:42:27.544 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 57.559901156000002, 2024-09-21T21:42:27.544 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043196518000000003, 2024-09-21T21:42:27.544 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:27.544 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:27.544 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:27.544 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:27.544 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:27.544 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:27.544 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7001 2024-09-21T21:42:27.544 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.544 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:27.544 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.545 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:27.545 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923211+0000", 2024-09-21T21:42:27.545 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:27.545 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.545 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.545 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:27.545 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923211+0000", 2024-09-21T21:42:27.545 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:27.545 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.545 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.545 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:27.545 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923212+0000", 2024-09-21T21:42:27.545 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3900000000000005e-07 2024-09-21T21:42:27.545 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.545 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.546 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:27.546 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923214+0000", 2024-09-21T21:42:27.546 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9970000000000001e-06 2024-09-21T21:42:27.546 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.546 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.546 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:27.546 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923219+0000", 2024-09-21T21:42:27.546 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4310000000000002e-06 2024-09-21T21:42:27.546 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.546 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.546 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:27.546 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923224+0000", 2024-09-21T21:42:27.546 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6410000000000001e-06 2024-09-21T21:42:27.546 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.546 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.547 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:27.547 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966318+0000", 2024-09-21T21:42:27.547 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043093842 2024-09-21T21:42:27.547 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.547 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.547 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:27.547 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966359+0000", 2024-09-21T21:42:27.547 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1260999999999997e-05 2024-09-21T21:42:27.547 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.547 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.547 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:27.547 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966408+0000", 2024-09-21T21:42:27.547 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8807000000000002e-05 2024-09-21T21:42:27.547 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:27.548 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:27.548 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:27.548 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.548 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.548 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7002 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5455872~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:27.548 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923229+0000", 2024-09-21T21:42:27.548 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 57.559882746, 2024-09-21T21:42:27.548 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043364568999999999, 2024-09-21T21:42:27.548 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:27.548 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:27.548 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:27.548 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:27.548 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:27.548 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:27.549 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7002 2024-09-21T21:42:27.549 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.549 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:27.549 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.549 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:27.549 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923229+0000", 2024-09-21T21:42:27.549 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:27.549 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.549 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.549 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:27.549 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923229+0000", 2024-09-21T21:42:27.549 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:27.549 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.550 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.550 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:27.550 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923230+0000", 2024-09-21T21:42:27.550 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4700000000000002e-07 2024-09-21T21:42:27.550 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.550 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.550 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:27.550 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923232+0000", 2024-09-21T21:42:27.550 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0719999999999998e-06 2024-09-21T21:42:27.550 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.550 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.550 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:27.550 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923237+0000", 2024-09-21T21:42:27.550 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2290000000000002e-06 2024-09-21T21:42:27.550 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.551 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.551 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:27.551 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923245+0000", 2024-09-21T21:42:27.551 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2959999999999999e-06 2024-09-21T21:42:27.551 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.551 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.551 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:27.551 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966438+0000", 2024-09-21T21:42:27.551 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043193027000000002 2024-09-21T21:42:27.551 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.551 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.551 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:27.551 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966506+0000", 2024-09-21T21:42:27.551 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7707999999999996e-05 2024-09-21T21:42:27.551 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.552 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.552 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:27.552 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966594+0000", 2024-09-21T21:42:27.552 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.789e-05 2024-09-21T21:42:27.552 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:27.552 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:27.552 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:27.552 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.552 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.552 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7005 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5470208~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:27.552 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923254+0000", 2024-09-21T21:42:27.552 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 57.559858515000002, 2024-09-21T21:42:27.552 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043457176, 2024-09-21T21:42:27.552 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:27.552 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:27.553 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:27.553 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:27.553 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:27.553 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:27.553 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7005 2024-09-21T21:42:27.553 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.553 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:27.553 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.553 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:27.553 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923254+0000", 2024-09-21T21:42:27.553 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:27.553 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.553 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.553 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:27.553 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923254+0000", 2024-09-21T21:42:27.554 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:27.554 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.554 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.554 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:27.554 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923254+0000", 2024-09-21T21:42:27.554 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4700000000000002e-07 2024-09-21T21:42:27.554 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.554 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.554 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:27.554 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923256+0000", 2024-09-21T21:42:27.554 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.007e-06 2024-09-21T21:42:27.554 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.554 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.554 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:27.555 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923266+0000", 2024-09-21T21:42:27.555 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.4700000000000008e-06 2024-09-21T21:42:27.555 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.555 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.555 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:27.555 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923271+0000", 2024-09-21T21:42:27.555 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2809999999999999e-06 2024-09-21T21:42:27.555 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.555 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.555 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:27.555 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966619+0000", 2024-09-21T21:42:27.555 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043347951000000003 2024-09-21T21:42:27.555 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.555 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.555 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:27.556 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966661+0000", 2024-09-21T21:42:27.556 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2614999999999998e-05 2024-09-21T21:42:27.556 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.556 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.556 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:27.556 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966711+0000", 2024-09-21T21:42:27.556 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9505000000000001e-05 2024-09-21T21:42:27.556 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:27.556 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:27.556 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:27.556 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.556 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.556 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7008 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5484544~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:27.556 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923278+0000", 2024-09-21T21:42:27.556 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 57.559833867999998, 2024-09-21T21:42:27.557 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043631071, 2024-09-21T21:42:27.557 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:27.557 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:27.557 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:27.557 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:27.557 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:27.557 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:27.557 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7008 2024-09-21T21:42:27.557 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.557 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:27.557 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.557 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:27.557 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923278+0000", 2024-09-21T21:42:27.557 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:27.558 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.558 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.558 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:27.558 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923278+0000", 2024-09-21T21:42:27.558 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:27.558 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.558 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.558 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:27.558 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923279+0000", 2024-09-21T21:42:27.558 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.77e-07 2024-09-21T21:42:27.558 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.558 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.558 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:27.559 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923281+0000", 2024-09-21T21:42:27.559 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0219999999999999e-06 2024-09-21T21:42:27.559 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.559 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.559 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:27.559 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923287+0000", 2024-09-21T21:42:27.559 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7799999999999997e-06 2024-09-21T21:42:27.559 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.559 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.559 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:27.559 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923291+0000", 2024-09-21T21:42:27.559 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1699999999999999e-06 2024-09-21T21:42:27.559 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.559 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.559 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:27.559 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966742+0000", 2024-09-21T21:42:27.560 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043451344000000003 2024-09-21T21:42:27.560 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.560 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.560 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:27.560 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966809+0000", 2024-09-21T21:42:27.560 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7070999999999995e-05 2024-09-21T21:42:27.560 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.560 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.560 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:27.560 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966909+0000", 2024-09-21T21:42:27.560 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000100307 2024-09-21T21:42:27.560 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:27.560 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:27.560 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:27.560 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.560 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.561 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7010 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5498880~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:27.561 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923301+0000", 2024-09-21T21:42:27.561 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 57.559810972999998, 2024-09-21T21:42:27.561 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043717830999999999, 2024-09-21T21:42:27.561 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:27.561 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:27.561 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:27.561 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:27.561 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:27.561 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:27.561 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7010 2024-09-21T21:42:27.561 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.561 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:27.561 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.561 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:27.561 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923301+0000", 2024-09-21T21:42:27.562 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:27.562 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.562 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.562 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:27.562 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923301+0000", 2024-09-21T21:42:27.562 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:27.562 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.562 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.562 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:27.562 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923302+0000", 2024-09-21T21:42:27.562 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.41e-07 2024-09-21T21:42:27.562 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.562 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.562 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:27.562 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923304+0000", 2024-09-21T21:42:27.563 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0990000000000001e-06 2024-09-21T21:42:27.563 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.563 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.563 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:27.563 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923310+0000", 2024-09-21T21:42:27.563 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3860000000000004e-06 2024-09-21T21:42:27.563 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.563 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.563 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:27.563 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923315+0000", 2024-09-21T21:42:27.563 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8459999999999999e-06 2024-09-21T21:42:27.563 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.563 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.563 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:27.563 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966938+0000", 2024-09-21T21:42:27.563 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043622693999999997 2024-09-21T21:42:27.564 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.564 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.564 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:27.564 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966975+0000", 2024-09-21T21:42:27.564 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7608999999999999e-05 2024-09-21T21:42:27.564 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.564 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.564 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:27.564 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.967019+0000", 2024-09-21T21:42:27.564 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3856000000000002e-05 2024-09-21T21:42:27.564 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:27.564 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:27.564 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:27.564 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.564 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.564 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7354 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 2107392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:27.564 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.936932+0000", 2024-09-21T21:42:27.565 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 57.546180118000002, 2024-09-21T21:42:27.565 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041607708, 2024-09-21T21:42:27.565 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:27.565 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:27.565 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:27.565 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:27.565 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:27.565 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:27.565 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7354 2024-09-21T21:42:27.565 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.565 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:27.565 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.565 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:27.565 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.936932+0000", 2024-09-21T21:42:27.565 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:27.566 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.566 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.566 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:27.566 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.936932+0000", 2024-09-21T21:42:27.566 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:27.566 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.566 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.566 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:27.566 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.937276+0000", 2024-09-21T21:42:27.566 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00034344200000000001 2024-09-21T21:42:27.566 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.566 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.566 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:27.566 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.937279+0000", 2024-09-21T21:42:27.566 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.844e-06 2024-09-21T21:42:27.567 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.567 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.567 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:27.567 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.937288+0000", 2024-09-21T21:42:27.567 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2409999999999995e-06 2024-09-21T21:42:27.567 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.567 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.567 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:27.567 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.937293+0000", 2024-09-21T21:42:27.567 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6559999999999997e-06 2024-09-21T21:42:27.567 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.567 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.567 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:27.567 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.978448+0000", 2024-09-21T21:42:27.567 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041154621000000002 2024-09-21T21:42:27.568 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.568 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.568 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:27.568 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.978487+0000", 2024-09-21T21:42:27.568 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9254000000000001e-05 2024-09-21T21:42:27.568 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.568 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.568 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:27.568 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.978540+0000", 2024-09-21T21:42:27.568 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2649999999999999e-05 2024-09-21T21:42:27.568 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:27.568 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:27.568 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:27.568 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.568 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.569 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5215.0:1054 3.9 3:946d2401:::rbd_data.1121524d2522.000000000000000b:head [set-alloc-hint object_size 16777216 write_size 16777216,write 6615040~59392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-21T21:42:27.569 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:38.148979+0000", 2024-09-21T21:42:27.569 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 49.334133151000003, 2024-09-21T21:42:27.569 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047870388999999999, 2024-09-21T21:42:27.569 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:27.569 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:27.569 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:27.569 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:27.569 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5215", 2024-09-21T21:42:27.569 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1580282168", 2024-09-21T21:42:27.569 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1054 2024-09-21T21:42:27.569 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.569 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:27.569 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.570 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:27.570 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.148979+0000", 2024-09-21T21:42:27.570 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:27.570 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.570 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.570 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:27.570 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.148979+0000", 2024-09-21T21:42:27.570 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:27.570 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.570 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.570 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:27.570 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.148980+0000", 2024-09-21T21:42:27.570 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5799999999999999e-07 2024-09-21T21:42:27.570 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.570 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.571 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:27.571 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.149039+0000", 2024-09-21T21:42:27.571 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9151000000000003e-05 2024-09-21T21:42:27.571 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.571 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.571 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:27.571 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.149047+0000", 2024-09-21T21:42:27.571 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1000000000000004e-06 2024-09-21T21:42:27.571 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.571 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.571 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:27.571 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.149052+0000", 2024-09-21T21:42:27.571 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5959999999999997e-06 2024-09-21T21:42:27.571 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.572 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.572 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:27.572 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.195288+0000", 2024-09-21T21:42:27.572 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.046236248000000001 2024-09-21T21:42:27.572 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.572 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.572 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:27.572 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.195334+0000", 2024-09-21T21:42:27.572 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6575e-05 2024-09-21T21:42:27.572 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.572 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.572 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 2", 2024-09-21T21:42:27.572 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.195406+0000", 2024-09-21T21:42:27.572 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1715e-05 2024-09-21T21:42:27.572 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.573 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.573 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:27.573 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.196018+0000", 2024-09-21T21:42:27.573 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00061218999999999998 2024-09-21T21:42:27.573 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.573 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.573 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:27.573 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.196794+0000", 2024-09-21T21:42:27.573 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000775814 2024-09-21T21:42:27.573 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.573 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.573 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:27.573 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.196827+0000", 2024-09-21T21:42:27.573 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2419000000000003e-05 2024-09-21T21:42:27.573 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:27.573 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:27.574 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:27.574 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.196849+0000", 2024-09-21T21:42:27.574 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2923000000000001e-05 2024-09-21T21:42:27.574 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:27.574 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:27.574 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:27.574 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:27.574 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:27.574 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:27.576 INFO:teuthology.orchestra.run.smithi081.stderr:nodeep-scrub is set 2024-09-21T21:42:27.576 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T21:42:27.577 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:27.574+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:27.594 INFO:teuthology.orchestra.run.smithi164.stdout:flattening image image_client.0-clone40 2024-09-21T21:42:27.676 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:27.674+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:27.720 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:27.721 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:42:27.721 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-21T21:42:27.721 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:27.729 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T21:42:27.777 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:27.775+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:27.875 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:27.875 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:42:27.875 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:42:27.875 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:42:27.875 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:27.878 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:27.877+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:27.883 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T21:42:27.930 INFO:teuthology.orchestra.run.smithi164.stdout:checking clone #39, image image_client.0-clone39 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent40 2024-09-21T21:42:27.979 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:27.978+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:28.072 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:28.072 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-21T21:42:28.072 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-21T21:42:28.072 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:42:28.073 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.073 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4385.0:15202 3.16 3:6888948b:::rbd_header.1121493f6459:head [watch reconnect cookie 140325440202496 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-21T21:42:28.073 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:41.736601+0000", 2024-09-21T21:42:28.073 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 46.332046321999997, 2024-09-21T21:42:28.073 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5040056769999999, 2024-09-21T21:42:28.073 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:28.073 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:28.073 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:28.073 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:28.073 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4385", 2024-09-21T21:42:28.073 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:42:28.073 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 15202 2024-09-21T21:42:28.074 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.074 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:28.074 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.074 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:28.074 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736601+0000", 2024-09-21T21:42:28.074 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:28.074 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.074 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.074 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:28.074 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736601+0000", 2024-09-21T21:42:28.075 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:28.075 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.075 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.075 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:28.075 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736602+0000", 2024-09-21T21:42:28.075 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0699999999999999e-06 2024-09-21T21:42:28.075 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.075 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.075 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:28.075 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736607+0000", 2024-09-21T21:42:28.075 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2520000000000001e-06 2024-09-21T21:42:28.076 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.076 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.076 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:28.076 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736617+0000", 2024-09-21T21:42:28.078 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0074e-05 2024-09-21T21:42:28.078 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.078 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.078 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:28.078 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736622+0000", 2024-09-21T21:42:28.078 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.101e-06 2024-09-21T21:42:28.078 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.078 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.078 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:28.079 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736664+0000", 2024-09-21T21:42:28.079 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1869999999999997e-05 2024-09-21T21:42:28.079 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.079 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.079 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op must wait for map", 2024-09-21T21:42:28.079 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736667+0000", 2024-09-21T21:42:28.079 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4750000000000002e-06 2024-09-21T21:42:28.079 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.079 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.079 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:28.079 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742886+0000", 2024-09-21T21:42:28.079 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0062187579999999996 2024-09-21T21:42:28.080 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.080 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.080 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:28.080 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742892+0000", 2024-09-21T21:42:28.080 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0440000000000001e-06 2024-09-21T21:42:28.080 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.080 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.080 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:28.080 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210307+0000", 2024-09-21T21:42:28.080 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.467414781 2024-09-21T21:42:28.081 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.081 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.081 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:28.081 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210322+0000", 2024-09-21T21:42:28.081 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5311e-05 2024-09-21T21:42:28.081 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.081 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.081 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:28.081 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.237624+0000", 2024-09-21T21:42:28.081 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0273018920000001 2024-09-21T21:42:28.082 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.082 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.082 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:28.082 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.237870+0000", 2024-09-21T21:42:28.082 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00024622899999999999 2024-09-21T21:42:28.082 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.082 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.082 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:28.082 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.240607+0000", 2024-09-21T21:42:28.082 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0027368200000000001 2024-09-21T21:42:28.082 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:28.083 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:28.083 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:28.083 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.083 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.083 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5415.0:379 3.16 3:6888948b:::rbd_header.1121493f6459:head [watch reconnect cookie 139702602617920 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-21T21:42:28.083 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:41.739544+0000", 2024-09-21T21:42:28.083 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 46.329103893000003, 2024-09-21T21:42:28.083 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.501180645, 2024-09-21T21:42:28.083 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:28.083 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:28.083 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:28.083 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:28.083 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5415", 2024-09-21T21:42:28.084 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1212816607", 2024-09-21T21:42:28.084 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 379 2024-09-21T21:42:28.084 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.084 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:28.084 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.084 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:28.084 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739544+0000", 2024-09-21T21:42:28.084 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:28.084 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.084 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.084 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:28.084 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739544+0000", 2024-09-21T21:42:28.085 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:28.085 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.085 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.085 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:28.085 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739545+0000", 2024-09-21T21:42:28.085 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.065e-06 2024-09-21T21:42:28.085 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.085 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.085 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:28.085 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739550+0000", 2024-09-21T21:42:28.085 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9599999999999999e-06 2024-09-21T21:42:28.085 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.086 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.086 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:28.086 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739565+0000", 2024-09-21T21:42:28.086 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5709000000000002e-05 2024-09-21T21:42:28.086 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.086 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.086 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:28.086 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739573+0000", 2024-09-21T21:42:28.086 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7849999999999992e-06 2024-09-21T21:42:28.086 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.087 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.087 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:28.087 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739634+0000", 2024-09-21T21:42:28.087 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0732000000000001e-05 2024-09-21T21:42:28.087 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.087 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.087 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op must wait for map", 2024-09-21T21:42:28.087 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739640+0000", 2024-09-21T21:42:28.087 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0000000000000002e-06 2024-09-21T21:42:28.087 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.087 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.087 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:28.088 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742958+0000", 2024-09-21T21:42:28.088 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0033182390000000002 2024-09-21T21:42:28.088 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.088 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.088 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:28.088 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742966+0000", 2024-09-21T21:42:28.088 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1610000000000003e-06 2024-09-21T21:42:28.088 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.088 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.088 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:28.088 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210405+0000", 2024-09-21T21:42:28.088 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.467438725 2024-09-21T21:42:28.089 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.089 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.089 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:28.089 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210415+0000", 2024-09-21T21:42:28.089 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0234000000000001e-05 2024-09-21T21:42:28.089 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.089 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.089 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:28.089 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.238743+0000", 2024-09-21T21:42:28.089 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0283273070000001 2024-09-21T21:42:28.089 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.089 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.090 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:28.090 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.238842+0000", 2024-09-21T21:42:28.090 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.9896999999999996e-05 2024-09-21T21:42:28.090 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.090 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.090 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:28.090 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.240724+0000", 2024-09-21T21:42:28.090 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001881831 2024-09-21T21:42:28.090 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:28.090 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:28.090 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:28.090 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.091 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.091 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4385.0:15203 3.16 3:6888948b:::rbd_header.1121493f6459:head [call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-21T21:42:28.091 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:41.739737+0000", 2024-09-21T21:42:28.091 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 46.328910882000002, 2024-09-21T21:42:28.091 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4989621799999999, 2024-09-21T21:42:28.091 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:28.091 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:28.091 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:28.091 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:28.092 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4385", 2024-09-21T21:42:28.092 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:42:28.092 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 15203 2024-09-21T21:42:28.092 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.092 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:28.092 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.092 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:28.092 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739737+0000", 2024-09-21T21:42:28.092 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:28.092 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.092 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.093 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:28.093 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739737+0000", 2024-09-21T21:42:28.093 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:28.093 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.093 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.093 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:28.093 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739738+0000", 2024-09-21T21:42:28.093 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.7499999999999998e-07 2024-09-21T21:42:28.093 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.093 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.093 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:28.094 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739743+0000", 2024-09-21T21:42:28.094 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5049999999999998e-06 2024-09-21T21:42:28.094 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.094 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.094 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:28.094 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739757+0000", 2024-09-21T21:42:28.094 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3947e-05 2024-09-21T21:42:28.094 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.094 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.094 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:28.094 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739765+0000", 2024-09-21T21:42:28.094 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9310000000000008e-06 2024-09-21T21:42:28.095 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.095 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.095 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:28.095 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739816+0000", 2024-09-21T21:42:28.096 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0559999999999997e-05 2024-09-21T21:42:28.096 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.096 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.096 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting_for_map not empty", 2024-09-21T21:42:28.096 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739821+0000", 2024-09-21T21:42:28.096 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3539999999999999e-06 2024-09-21T21:42:28.096 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.096 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.097 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:28.097 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742928+0000", 2024-09-21T21:42:28.097 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0031074399999999999 2024-09-21T21:42:28.097 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.097 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.097 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:28.097 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742937+0000", 2024-09-21T21:42:28.097 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1289999999999996e-06 2024-09-21T21:42:28.097 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.097 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.097 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:28.097 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210350+0000", 2024-09-21T21:42:28.097 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.46741311299999999 2024-09-21T21:42:28.097 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.098 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.098 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:28.098 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210358+0000", 2024-09-21T21:42:28.098 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1869999999999997e-06 2024-09-21T21:42:28.098 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.098 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.098 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:28.098 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.238398+0000", 2024-09-21T21:42:28.098 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0280397830000001 2024-09-21T21:42:28.098 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.098 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.098 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:28.098 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.238489+0000", 2024-09-21T21:42:28.099 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.0958000000000006e-05 2024-09-21T21:42:28.099 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.099 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.099 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:28.099 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.238699+0000", 2024-09-21T21:42:28.099 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00021029800000000001 2024-09-21T21:42:28.099 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:28.099 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:28.099 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:28.099 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.099 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.099 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4385.0:15204 3.16 3:6888948b:::rbd_header.1121493f6459:head [watch reconnect cookie 140325440202496 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-21T21:42:28.099 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:42.210726+0000", 2024-09-21T21:42:28.099 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 45.857922004000002, 2024-09-21T21:42:28.100 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.030304557, 2024-09-21T21:42:28.100 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:28.100 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:28.100 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:28.100 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:28.100 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4385", 2024-09-21T21:42:28.100 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:42:28.100 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 15204 2024-09-21T21:42:28.100 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.100 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:28.100 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.100 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:28.100 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210726+0000", 2024-09-21T21:42:28.100 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:28.101 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.101 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.101 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:28.101 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210726+0000", 2024-09-21T21:42:28.101 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:28.101 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.101 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.101 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:28.101 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210727+0000", 2024-09-21T21:42:28.101 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.204e-06 2024-09-21T21:42:28.101 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.101 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.101 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:28.101 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210732+0000", 2024-09-21T21:42:28.101 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5249999999999996e-06 2024-09-21T21:42:28.102 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.102 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.102 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:28.102 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210744+0000", 2024-09-21T21:42:28.102 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1405e-05 2024-09-21T21:42:28.102 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.102 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.102 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:28.102 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210754+0000", 2024-09-21T21:42:28.102 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0122999999999999e-05 2024-09-21T21:42:28.102 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.102 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.102 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:28.102 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210798+0000", 2024-09-21T21:42:28.103 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4005000000000003e-05 2024-09-21T21:42:28.103 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.103 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.103 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:28.103 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210803+0000", 2024-09-21T21:42:28.103 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5130000000000004e-06 2024-09-21T21:42:28.103 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.103 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.103 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:28.103 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.239260+0000", 2024-09-21T21:42:28.103 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0284563040000001 2024-09-21T21:42:28.103 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.103 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.103 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:28.104 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.239318+0000", 2024-09-21T21:42:28.104 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8486000000000002e-05 2024-09-21T21:42:28.104 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.104 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.104 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:28.104 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.241030+0000", 2024-09-21T21:42:28.104 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0017119920000000001 2024-09-21T21:42:28.104 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:28.104 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:28.104 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:28.104 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.104 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.104 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5415.0:380 3.16 3:6888948b:::rbd_header.1121493f6459:head [watch reconnect cookie 139702602617920 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-21T21:42:28.104 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:42.210777+0000", 2024-09-21T21:42:28.104 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 45.857870224999999, 2024-09-21T21:42:28.105 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.030410566, 2024-09-21T21:42:28.105 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:28.105 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:28.105 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:28.105 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:28.105 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5415", 2024-09-21T21:42:28.105 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1212816607", 2024-09-21T21:42:28.105 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 380 2024-09-21T21:42:28.105 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.105 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:28.105 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.105 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:28.105 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210777+0000", 2024-09-21T21:42:28.105 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:28.105 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.105 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.106 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:28.106 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210777+0000", 2024-09-21T21:42:28.106 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:28.106 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.106 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.106 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:28.106 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210778+0000", 2024-09-21T21:42:28.106 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8999999999999996e-07 2024-09-21T21:42:28.106 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.106 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.106 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:28.106 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210780+0000", 2024-09-21T21:42:28.106 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.503e-06 2024-09-21T21:42:28.107 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.107 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.107 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:28.107 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210788+0000", 2024-09-21T21:42:28.107 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0999999999999998e-06 2024-09-21T21:42:28.107 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.107 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.107 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:28.107 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210799+0000", 2024-09-21T21:42:28.107 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1703e-05 2024-09-21T21:42:28.107 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.107 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.107 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:28.107 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210833+0000", 2024-09-21T21:42:28.107 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4008999999999999e-05 2024-09-21T21:42:28.108 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.108 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.108 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:28.108 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210838+0000", 2024-09-21T21:42:28.108 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7480000000000001e-06 2024-09-21T21:42:28.108 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.108 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.108 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:28.108 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.239612+0000", 2024-09-21T21:42:28.108 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.028773554 2024-09-21T21:42:28.108 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.108 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.108 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:28.108 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.239677+0000", 2024-09-21T21:42:28.108 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5053000000000002e-05 2024-09-21T21:42:28.109 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.109 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.109 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:28.109 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.241188+0000", 2024-09-21T21:42:28.109 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001511306 2024-09-21T21:42:28.109 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:28.109 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:28.109 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:28.109 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.109 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.109 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24794 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:28.109 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.584085+0000", 2024-09-21T21:42:28.109 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 5.4845630490000001, 2024-09-21T21:42:28.109 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.096209996000000006, 2024-09-21T21:42:28.109 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:28.110 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:28.110 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:28.110 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:28.110 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:42:28.110 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:28.110 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24794 2024-09-21T21:42:28.110 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.110 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:28.110 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.110 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:28.110 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584085+0000", 2024-09-21T21:42:28.110 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:28.110 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.110 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.111 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:28.111 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584085+0000", 2024-09-21T21:42:28.111 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:28.111 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.111 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.111 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:28.111 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584085+0000", 2024-09-21T21:42:28.111 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3200000000000001e-07 2024-09-21T21:42:28.111 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.111 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.111 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:28.111 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584086+0000", 2024-09-21T21:42:28.111 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.717e-06 2024-09-21T21:42:28.111 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.111 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.112 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:28.112 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584090+0000", 2024-09-21T21:42:28.112 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3450000000000002e-06 2024-09-21T21:42:28.112 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.112 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.112 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:28.112 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584092+0000", 2024-09-21T21:42:28.112 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6359999999999998e-06 2024-09-21T21:42:28.112 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.112 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.112 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:28.112 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.676497+0000", 2024-09-21T21:42:28.112 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.092404659 2024-09-21T21:42:28.112 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.112 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.113 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:28.113 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.676567+0000", 2024-09-21T21:42:28.113 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9627000000000003e-05 2024-09-21T21:42:28.113 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.113 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.113 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:42:28.113 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.676925+0000", 2024-09-21T21:42:28.113 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00035848500000000002 2024-09-21T21:42:28.113 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.113 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.113 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:28.113 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.678427+0000", 2024-09-21T21:42:28.113 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001501638 2024-09-21T21:42:28.113 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.113 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.113 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:28.114 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.680204+0000", 2024-09-21T21:42:28.114 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0017766979999999999 2024-09-21T21:42:28.114 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.114 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.114 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:28.114 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.680256+0000", 2024-09-21T21:42:28.114 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2549999999999997e-05 2024-09-21T21:42:28.114 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.114 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.114 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:28.114 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.680295+0000", 2024-09-21T21:42:28.114 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8408999999999998e-05 2024-09-21T21:42:28.114 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:28.114 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:28.115 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:28.115 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.115 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.115 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24796 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:28.115 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.584099+0000", 2024-09-21T21:42:28.115 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 5.4845484799999999, 2024-09-21T21:42:28.115 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.097297429000000005, 2024-09-21T21:42:28.115 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:28.115 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:28.115 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:28.115 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:28.115 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:42:28.116 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:28.116 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24796 2024-09-21T21:42:28.116 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.116 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:28.116 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.116 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:28.116 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584099+0000", 2024-09-21T21:42:28.116 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:28.116 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.116 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.116 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:28.116 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584099+0000", 2024-09-21T21:42:28.116 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:28.116 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.117 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.117 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:28.117 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584099+0000", 2024-09-21T21:42:28.117 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3999999999999998e-07 2024-09-21T21:42:28.117 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.117 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.117 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:28.117 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584101+0000", 2024-09-21T21:42:28.117 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5850000000000001e-06 2024-09-21T21:42:28.117 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.117 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.117 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:28.117 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584108+0000", 2024-09-21T21:42:28.117 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.498e-06 2024-09-21T21:42:28.117 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.118 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.118 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:28.118 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584112+0000", 2024-09-21T21:42:28.118 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.252e-06 2024-09-21T21:42:28.118 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.118 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.118 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:28.118 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.677336+0000", 2024-09-21T21:42:28.118 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.093224206000000004 2024-09-21T21:42:28.118 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.118 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.118 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:28.118 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.677411+0000", 2024-09-21T21:42:28.118 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5036000000000005e-05 2024-09-21T21:42:28.119 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.119 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.119 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:42:28.119 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.677782+0000", 2024-09-21T21:42:28.119 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00037153099999999998 2024-09-21T21:42:28.119 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.119 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.119 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:28.119 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.680360+0000", 2024-09-21T21:42:28.119 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025773020000000001 2024-09-21T21:42:28.119 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.119 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.119 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:28.119 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681313+0000", 2024-09-21T21:42:28.119 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00095341100000000004 2024-09-21T21:42:28.120 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.120 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.120 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:28.120 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681361+0000", 2024-09-21T21:42:28.120 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7497999999999998e-05 2024-09-21T21:42:28.120 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.120 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.120 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:28.120 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681397+0000", 2024-09-21T21:42:28.120 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.587e-05 2024-09-21T21:42:28.120 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:28.120 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:28.120 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:28.120 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.121 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.121 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24797 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:28.121 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.584116+0000", 2024-09-21T21:42:28.121 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 5.4845319909999999, 2024-09-21T21:42:28.121 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.097428257000000004, 2024-09-21T21:42:28.121 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:28.121 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:28.121 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:28.121 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:28.122 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:42:28.122 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:28.122 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24797 2024-09-21T21:42:28.122 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.122 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:28.122 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.122 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:28.122 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584116+0000", 2024-09-21T21:42:28.122 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:28.122 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.122 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.122 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:28.122 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584116+0000", 2024-09-21T21:42:28.122 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:28.123 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.123 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.123 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:28.123 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584116+0000", 2024-09-21T21:42:28.123 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.53e-07 2024-09-21T21:42:28.123 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.123 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.123 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:28.123 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584118+0000", 2024-09-21T21:42:28.123 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6810000000000001e-06 2024-09-21T21:42:28.123 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.123 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.123 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:28.123 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584121+0000", 2024-09-21T21:42:28.124 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1099999999999999e-06 2024-09-21T21:42:28.124 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.124 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.124 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:28.124 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584123+0000", 2024-09-21T21:42:28.124 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5890000000000001e-06 2024-09-21T21:42:28.124 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.124 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.124 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:28.124 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.678476+0000", 2024-09-21T21:42:28.124 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.094352320000000003 2024-09-21T21:42:28.124 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.124 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.124 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:28.125 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.678556+0000", 2024-09-21T21:42:28.125 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0291999999999996e-05 2024-09-21T21:42:28.125 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.125 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.125 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:42:28.125 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.678871+0000", 2024-09-21T21:42:28.125 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000314689 2024-09-21T21:42:28.125 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.125 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.125 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:28.125 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681452+0000", 2024-09-21T21:42:28.125 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025817890000000001 2024-09-21T21:42:28.125 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.125 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.125 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:28.126 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681467+0000", 2024-09-21T21:42:28.126 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4256e-05 2024-09-21T21:42:28.126 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.126 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.126 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:28.126 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681510+0000", 2024-09-21T21:42:28.126 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3560999999999998e-05 2024-09-21T21:42:28.126 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.126 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.126 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:28.126 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681544+0000", 2024-09-21T21:42:28.126 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3717000000000003e-05 2024-09-21T21:42:28.127 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:28.127 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:28.127 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:28.127 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.127 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.127 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24798 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:28.127 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.584130+0000", 2024-09-21T21:42:28.127 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 5.484517468, 2024-09-21T21:42:28.127 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.098350526999999993, 2024-09-21T21:42:28.127 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:28.127 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:28.127 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:28.127 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:28.127 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:42:28.127 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:28.128 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24798 2024-09-21T21:42:28.128 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.128 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:28.128 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.128 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:28.128 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584130+0000", 2024-09-21T21:42:28.128 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:28.128 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.128 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.128 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:28.128 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584130+0000", 2024-09-21T21:42:28.128 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:28.128 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.128 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.128 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:28.129 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584131+0000", 2024-09-21T21:42:28.129 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4700000000000001e-07 2024-09-21T21:42:28.129 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.129 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.129 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:28.129 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584133+0000", 2024-09-21T21:42:28.129 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.34e-06 2024-09-21T21:42:28.129 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.129 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.129 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:28.129 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584137+0000", 2024-09-21T21:42:28.129 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5609999999999999e-06 2024-09-21T21:42:28.129 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.129 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.129 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:28.130 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584139+0000", 2024-09-21T21:42:28.130 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7350000000000001e-06 2024-09-21T21:42:28.130 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.130 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.130 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:28.130 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.679274+0000", 2024-09-21T21:42:28.130 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.095134719000000006 2024-09-21T21:42:28.130 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.130 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.130 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:28.130 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.679358+0000", 2024-09-21T21:42:28.130 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3609999999999994e-05 2024-09-21T21:42:28.130 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.131 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.131 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:42:28.131 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.679685+0000", 2024-09-21T21:42:28.131 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00032732399999999999 2024-09-21T21:42:28.131 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.131 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.131 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:28.131 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681553+0000", 2024-09-21T21:42:28.131 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001868059 2024-09-21T21:42:28.131 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.131 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.131 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:28.131 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.682375+0000", 2024-09-21T21:42:28.131 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00082232099999999999 2024-09-21T21:42:28.132 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.132 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.132 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:28.132 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.682431+0000", 2024-09-21T21:42:28.132 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6051999999999998e-05 2024-09-21T21:42:28.132 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.132 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.132 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:28.132 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.682481+0000", 2024-09-21T21:42:28.132 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9258999999999997e-05 2024-09-21T21:42:28.132 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:28.132 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:28.132 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:28.132 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.132 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.132 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24799 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:28.133 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.584159+0000", 2024-09-21T21:42:28.133 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 5.4844884799999996, 2024-09-21T21:42:28.133 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.100021551, 2024-09-21T21:42:28.133 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:28.133 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:28.133 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:28.133 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:28.133 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:42:28.133 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:28.133 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24799 2024-09-21T21:42:28.133 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.133 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:28.133 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.133 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:28.134 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584159+0000", 2024-09-21T21:42:28.134 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:28.134 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.134 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.134 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:28.134 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584159+0000", 2024-09-21T21:42:28.134 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:28.134 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.134 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.134 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:28.134 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584160+0000", 2024-09-21T21:42:28.134 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1300000000000001e-07 2024-09-21T21:42:28.134 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.134 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.135 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:28.135 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584162+0000", 2024-09-21T21:42:28.135 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5409999999999999e-06 2024-09-21T21:42:28.135 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.135 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.135 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:28.135 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584169+0000", 2024-09-21T21:42:28.135 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7880000000000001e-06 2024-09-21T21:42:28.135 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.135 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.135 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:28.135 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584174+0000", 2024-09-21T21:42:28.135 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9590000000000003e-06 2024-09-21T21:42:28.135 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.135 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.136 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:28.136 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.680400+0000", 2024-09-21T21:42:28.136 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.096226514999999999 2024-09-21T21:42:28.136 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.136 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.136 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:28.136 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.680480+0000", 2024-09-21T21:42:28.136 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9589e-05 2024-09-21T21:42:28.136 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.136 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.136 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:42:28.136 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.680847+0000", 2024-09-21T21:42:28.136 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00036757700000000001 2024-09-21T21:42:28.136 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.136 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.137 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:28.137 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.682555+0000", 2024-09-21T21:42:28.137 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001707791 2024-09-21T21:42:28.137 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.137 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.137 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:28.137 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684093+0000", 2024-09-21T21:42:28.137 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0015377170000000001 2024-09-21T21:42:28.137 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.137 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.137 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:28.137 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684144+0000", 2024-09-21T21:42:28.137 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1208000000000002e-05 2024-09-21T21:42:28.137 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.137 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.138 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:28.138 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684181+0000", 2024-09-21T21:42:28.138 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6452999999999998e-05 2024-09-21T21:42:28.138 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:28.138 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:28.138 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:28.138 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.138 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.138 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24835 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:28.138 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586332+0000", 2024-09-21T21:42:28.138 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 5.4823158660000004, 2024-09-21T21:42:28.138 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.098730360000000003, 2024-09-21T21:42:28.138 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:28.138 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:28.139 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:28.139 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:28.139 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:42:28.139 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:28.139 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24835 2024-09-21T21:42:28.139 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.139 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:28.139 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.139 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:28.139 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586332+0000", 2024-09-21T21:42:28.139 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:28.139 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.139 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.139 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:28.139 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586332+0000", 2024-09-21T21:42:28.140 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:28.140 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.140 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.140 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:28.140 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586333+0000", 2024-09-21T21:42:28.140 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0200000000000001e-07 2024-09-21T21:42:28.140 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.140 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.140 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:28.140 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586346+0000", 2024-09-21T21:42:28.140 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3745e-05 2024-09-21T21:42:28.140 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.140 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.140 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:28.140 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586366+0000", 2024-09-21T21:42:28.141 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9272000000000001e-05 2024-09-21T21:42:28.141 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.141 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.141 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:28.141 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586377+0000", 2024-09-21T21:42:28.141 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1197e-05 2024-09-21T21:42:28.141 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.141 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.141 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:28.141 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681578+0000", 2024-09-21T21:42:28.141 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.095201420999999994 2024-09-21T21:42:28.141 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.141 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.141 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:28.142 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681640+0000", 2024-09-21T21:42:28.142 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1834000000000005e-05 2024-09-21T21:42:28.142 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.142 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.142 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:42:28.142 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681915+0000", 2024-09-21T21:42:28.142 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00027534199999999998 2024-09-21T21:42:28.142 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.142 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.142 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:28.142 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684230+0000", 2024-09-21T21:42:28.142 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00231424 2024-09-21T21:42:28.142 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.142 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.142 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:28.143 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684959+0000", 2024-09-21T21:42:28.143 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00072921399999999995 2024-09-21T21:42:28.143 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.143 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.143 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:28.143 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.685008+0000", 2024-09-21T21:42:28.143 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8860999999999999e-05 2024-09-21T21:42:28.143 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.143 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.143 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:28.143 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.685062+0000", 2024-09-21T21:42:28.143 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4432000000000003e-05 2024-09-21T21:42:28.143 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:28.143 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:28.143 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:28.144 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.144 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.144 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24837 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:28.144 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586395+0000", 2024-09-21T21:42:28.144 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 5.4822529309999997, 2024-09-21T21:42:28.144 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.099700522, 2024-09-21T21:42:28.144 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:28.144 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:28.144 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:28.144 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:28.144 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:42:28.144 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:28.144 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24837 2024-09-21T21:42:28.144 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.144 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:28.145 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.145 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:28.145 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586395+0000", 2024-09-21T21:42:28.145 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:28.145 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.145 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.145 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:28.145 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586395+0000", 2024-09-21T21:42:28.145 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:28.145 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.145 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.145 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:28.145 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586395+0000", 2024-09-21T21:42:28.145 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.06e-07 2024-09-21T21:42:28.145 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.146 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.146 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:28.146 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586401+0000", 2024-09-21T21:42:28.146 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0889999999999996e-06 2024-09-21T21:42:28.146 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.146 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.146 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:28.146 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586410+0000", 2024-09-21T21:42:28.146 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.918e-06 2024-09-21T21:42:28.146 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.146 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.146 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:28.146 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586424+0000", 2024-09-21T21:42:28.146 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3478999999999999e-05 2024-09-21T21:42:28.147 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.147 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.147 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:28.147 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.682590+0000", 2024-09-21T21:42:28.147 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.096166086999999997 2024-09-21T21:42:28.147 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.147 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.147 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:28.147 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.682657+0000", 2024-09-21T21:42:28.147 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6878000000000006e-05 2024-09-21T21:42:28.147 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.147 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.147 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:42:28.147 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.682879+0000", 2024-09-21T21:42:28.147 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00022238 2024-09-21T21:42:28.148 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.148 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.148 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:28.148 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.685073+0000", 2024-09-21T21:42:28.148 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021938119999999998 2024-09-21T21:42:28.148 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.148 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.148 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:28.148 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686038+0000", 2024-09-21T21:42:28.148 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00096539499999999997 2024-09-21T21:42:28.148 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.148 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.148 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:28.148 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686071+0000", 2024-09-21T21:42:28.148 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2514000000000001e-05 2024-09-21T21:42:28.149 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.149 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.149 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:28.149 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686095+0000", 2024-09-21T21:42:28.149 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4363999999999999e-05 2024-09-21T21:42:28.149 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:28.149 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:28.149 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:28.149 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.149 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.149 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24839 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:28.149 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586468+0000", 2024-09-21T21:42:28.149 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 5.4821792790000003, 2024-09-21T21:42:28.149 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.099739355000000002, 2024-09-21T21:42:28.149 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:28.150 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:28.150 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:28.150 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:28.150 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:42:28.150 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:28.150 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24839 2024-09-21T21:42:28.150 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.150 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:28.150 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.150 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:28.150 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586468+0000", 2024-09-21T21:42:28.150 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:28.150 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.150 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.150 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:28.151 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586468+0000", 2024-09-21T21:42:28.151 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:28.151 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.151 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.151 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:28.151 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586469+0000", 2024-09-21T21:42:28.151 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2200000000000004e-07 2024-09-21T21:42:28.151 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.151 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.151 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:28.151 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586472+0000", 2024-09-21T21:42:28.151 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9960000000000002e-06 2024-09-21T21:42:28.151 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.151 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.151 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:28.152 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586479+0000", 2024-09-21T21:42:28.152 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2699999999999999e-06 2024-09-21T21:42:28.152 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.152 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.152 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:28.152 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586485+0000", 2024-09-21T21:42:28.152 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6999999999999996e-06 2024-09-21T21:42:28.152 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.152 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.152 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:28.152 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.683276+0000", 2024-09-21T21:42:28.152 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.096791618999999995 2024-09-21T21:42:28.152 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.152 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.152 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:28.153 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.683326+0000", 2024-09-21T21:42:28.153 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9384e-05 2024-09-21T21:42:28.153 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.153 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.153 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:42:28.153 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.683655+0000", 2024-09-21T21:42:28.153 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00032965700000000002 2024-09-21T21:42:28.153 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.153 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.153 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:28.153 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686120+0000", 2024-09-21T21:42:28.153 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002464819 2024-09-21T21:42:28.153 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.153 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.153 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:28.154 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686159+0000", 2024-09-21T21:42:28.154 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9181000000000003e-05 2024-09-21T21:42:28.154 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.154 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.154 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:28.154 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686185+0000", 2024-09-21T21:42:28.154 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5907000000000001e-05 2024-09-21T21:42:28.154 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.154 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.154 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:28.154 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686208+0000", 2024-09-21T21:42:28.154 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.23e-05 2024-09-21T21:42:28.154 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:28.154 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:28.155 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:28.155 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.155 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.155 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24841 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:28.155 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586492+0000", 2024-09-21T21:42:28.155 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 5.4821551169999996, 2024-09-21T21:42:28.155 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.101672077, 2024-09-21T21:42:28.155 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:28.155 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:28.155 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:28.155 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:28.155 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:42:28.155 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:28.155 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24841 2024-09-21T21:42:28.155 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.156 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:28.156 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.156 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:28.156 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586492+0000", 2024-09-21T21:42:28.156 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:28.156 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.156 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.156 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:28.156 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586492+0000", 2024-09-21T21:42:28.156 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:28.156 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.156 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.156 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:28.156 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586493+0000", 2024-09-21T21:42:28.156 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5999999999999999e-07 2024-09-21T21:42:28.157 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.157 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.157 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:28.157 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586496+0000", 2024-09-21T21:42:28.157 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6089999999999999e-06 2024-09-21T21:42:28.157 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.157 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.157 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:28.157 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586502+0000", 2024-09-21T21:42:28.157 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6270000000000002e-06 2024-09-21T21:42:28.157 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.157 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.157 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:28.157 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586507+0000", 2024-09-21T21:42:28.157 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8810000000000004e-06 2024-09-21T21:42:28.158 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.158 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.158 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:28.158 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684273+0000", 2024-09-21T21:42:28.158 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.097766415999999995 2024-09-21T21:42:28.158 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.158 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.158 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:28.158 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684359+0000", 2024-09-21T21:42:28.158 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5574000000000004e-05 2024-09-21T21:42:28.158 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.158 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.158 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:42:28.158 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684568+0000", 2024-09-21T21:42:28.158 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000208529 2024-09-21T21:42:28.159 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.159 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.159 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:28.159 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.687186+0000", 2024-09-21T21:42:28.159 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0026180769999999999 2024-09-21T21:42:28.159 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.159 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.159 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:28.159 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688093+0000", 2024-09-21T21:42:28.159 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00090766999999999996 2024-09-21T21:42:28.159 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.159 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.159 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:28.159 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688142+0000", 2024-09-21T21:42:28.160 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8986000000000002e-05 2024-09-21T21:42:28.160 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.160 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.160 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:28.160 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688165+0000", 2024-09-21T21:42:28.160 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2248000000000001e-05 2024-09-21T21:42:28.160 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:28.160 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:28.160 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:28.160 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.160 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.160 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24843 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:28.160 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586514+0000", 2024-09-21T21:42:28.160 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 5.4821338309999996, 2024-09-21T21:42:28.160 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.101826449, 2024-09-21T21:42:28.161 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:28.161 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:28.161 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:28.161 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:28.161 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:42:28.161 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:28.161 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24843 2024-09-21T21:42:28.161 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.161 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:28.161 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.161 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:28.161 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586514+0000", 2024-09-21T21:42:28.161 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:28.161 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.162 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.162 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:28.162 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586514+0000", 2024-09-21T21:42:28.162 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:28.162 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.162 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.162 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:28.162 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586514+0000", 2024-09-21T21:42:28.162 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9999999999999998e-07 2024-09-21T21:42:28.162 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.162 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.162 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:28.162 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586517+0000", 2024-09-21T21:42:28.162 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4980000000000001e-06 2024-09-21T21:42:28.163 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.163 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.163 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:28.163 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586523+0000", 2024-09-21T21:42:28.163 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4609999999999996e-06 2024-09-21T21:42:28.163 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.163 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.163 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:28.163 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586528+0000", 2024-09-21T21:42:28.163 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2979999999999999e-06 2024-09-21T21:42:28.163 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.163 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.163 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:28.163 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.685104+0000", 2024-09-21T21:42:28.163 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.098575837999999999 2024-09-21T21:42:28.164 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.164 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.164 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:28.164 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.685184+0000", 2024-09-21T21:42:28.164 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9807000000000003e-05 2024-09-21T21:42:28.164 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.164 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.164 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:42:28.164 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.685496+0000", 2024-09-21T21:42:28.164 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00031225900000000001 2024-09-21T21:42:28.164 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.164 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.164 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:28.164 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688238+0000", 2024-09-21T21:42:28.165 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0027419409999999999 2024-09-21T21:42:28.165 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.165 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.165 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:28.165 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688253+0000", 2024-09-21T21:42:28.165 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4331000000000001e-05 2024-09-21T21:42:28.165 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.165 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.165 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:28.165 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688296+0000", 2024-09-21T21:42:28.165 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3801000000000003e-05 2024-09-21T21:42:28.165 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.165 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.165 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:28.166 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688340+0000", 2024-09-21T21:42:28.166 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3815e-05 2024-09-21T21:42:28.166 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:28.166 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:28.166 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:28.166 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.166 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.166 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24845 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:28.166 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586535+0000", 2024-09-21T21:42:28.166 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 5.4821129510000004, 2024-09-21T21:42:28.166 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.10285678600000001, 2024-09-21T21:42:28.166 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:28.166 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:28.166 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:28.167 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:28.167 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:42:28.167 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:28.167 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24845 2024-09-21T21:42:28.167 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.167 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:28.167 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.167 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:28.167 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586535+0000", 2024-09-21T21:42:28.167 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:28.167 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.167 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.167 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:28.167 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586535+0000", 2024-09-21T21:42:28.167 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:28.167 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.168 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.168 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:28.168 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586535+0000", 2024-09-21T21:42:28.168 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9000000000000002e-07 2024-09-21T21:42:28.168 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.168 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.168 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:28.168 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586537+0000", 2024-09-21T21:42:28.168 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4569999999999999e-06 2024-09-21T21:42:28.168 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.168 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.168 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:28.168 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586544+0000", 2024-09-21T21:42:28.168 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5849999999999996e-06 2024-09-21T21:42:28.169 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.169 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.169 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:28.169 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586549+0000", 2024-09-21T21:42:28.169 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0089999999999998e-06 2024-09-21T21:42:28.169 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.169 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.169 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:28.169 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686239+0000", 2024-09-21T21:42:28.169 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.099689531999999997 2024-09-21T21:42:28.169 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.169 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.169 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:28.169 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686289+0000", 2024-09-21T21:42:28.169 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.007e-05 2024-09-21T21:42:28.169 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.170 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.170 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:42:28.170 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686633+0000", 2024-09-21T21:42:28.170 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00034428699999999998 2024-09-21T21:42:28.170 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.170 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.170 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:28.170 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688370+0000", 2024-09-21T21:42:28.170 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001736741 2024-09-21T21:42:28.170 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.170 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.170 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:28.170 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.689295+0000", 2024-09-21T21:42:28.170 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00092517900000000004 2024-09-21T21:42:28.170 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.171 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.171 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:28.171 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.689354+0000", 2024-09-21T21:42:28.171 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8799999999999999e-05 2024-09-21T21:42:28.171 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.171 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.171 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:28.171 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.689391+0000", 2024-09-21T21:42:28.171 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7735999999999999e-05 2024-09-21T21:42:28.171 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:28.171 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:28.171 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:28.171 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.171 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.172 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24846 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:28.172 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586564+0000", 2024-09-21T21:42:28.172 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 5.4820839230000002, 2024-09-21T21:42:28.172 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.10375692, 2024-09-21T21:42:28.172 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:28.172 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:28.172 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:28.172 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:28.172 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:42:28.172 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:28.172 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24846 2024-09-21T21:42:28.172 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.172 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:28.172 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.172 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:28.173 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586564+0000", 2024-09-21T21:42:28.173 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:28.173 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.173 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.173 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:28.173 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586564+0000", 2024-09-21T21:42:28.173 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:28.173 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.173 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.173 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:28.173 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586564+0000", 2024-09-21T21:42:28.173 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4799999999999998e-07 2024-09-21T21:42:28.173 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.173 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.173 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:28.174 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586567+0000", 2024-09-21T21:42:28.174 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1669999999999999e-06 2024-09-21T21:42:28.174 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.174 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.174 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:28.174 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586578+0000", 2024-09-21T21:42:28.174 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0443e-05 2024-09-21T21:42:28.174 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.174 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.174 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:28.174 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586583+0000", 2024-09-21T21:42:28.174 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6790000000000002e-06 2024-09-21T21:42:28.174 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.174 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.175 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:28.175 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.687215+0000", 2024-09-21T21:42:28.175 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.100631973 2024-09-21T21:42:28.175 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.175 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.175 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:28.175 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.687311+0000", 2024-09-21T21:42:28.175 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5764e-05 2024-09-21T21:42:28.175 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.175 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.175 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:42:28.175 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.687663+0000", 2024-09-21T21:42:28.175 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00035167 2024-09-21T21:42:28.175 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.175 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.176 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:28.176 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690236+0000", 2024-09-21T21:42:28.176 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025733370000000002 2024-09-21T21:42:28.176 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.176 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.176 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:28.176 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690250+0000", 2024-09-21T21:42:28.176 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3548e-05 2024-09-21T21:42:28.176 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.176 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.176 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:28.176 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690298+0000", 2024-09-21T21:42:28.176 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7948999999999998e-05 2024-09-21T21:42:28.176 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.176 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.177 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:28.177 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690321+0000", 2024-09-21T21:42:28.177 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2841999999999999e-05 2024-09-21T21:42:28.177 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:28.177 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:28.177 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:28.177 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.177 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.177 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24849 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:28.177 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586633+0000", 2024-09-21T21:42:28.177 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 5.4820142570000003, 2024-09-21T21:42:28.177 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.104506548, 2024-09-21T21:42:28.177 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:28.177 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:28.178 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:28.178 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:28.178 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:42:28.178 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:28.178 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24849 2024-09-21T21:42:28.178 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.178 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:28.178 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.178 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:28.178 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586633+0000", 2024-09-21T21:42:28.178 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:28.178 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.178 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.178 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:28.178 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586633+0000", 2024-09-21T21:42:28.179 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:28.179 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.179 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.179 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:28.179 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586634+0000", 2024-09-21T21:42:28.179 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9100000000000004e-07 2024-09-21T21:42:28.179 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.179 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.179 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:28.179 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586637+0000", 2024-09-21T21:42:28.179 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1999999999999999e-06 2024-09-21T21:42:28.179 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.179 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.179 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:28.179 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586645+0000", 2024-09-21T21:42:28.180 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7759999999999997e-06 2024-09-21T21:42:28.180 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.180 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.180 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:28.180 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586652+0000", 2024-09-21T21:42:28.180 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3560000000000004e-06 2024-09-21T21:42:28.180 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.180 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.180 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:28.180 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688395+0000", 2024-09-21T21:42:28.180 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.101743239 2024-09-21T21:42:28.180 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.180 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.180 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:28.181 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688464+0000", 2024-09-21T21:42:28.181 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8273000000000002e-05 2024-09-21T21:42:28.181 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.181 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.181 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:42:28.181 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688786+0000", 2024-09-21T21:42:28.181 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000322064 2024-09-21T21:42:28.181 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.181 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.181 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:28.181 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690364+0000", 2024-09-21T21:42:28.181 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001578084 2024-09-21T21:42:28.181 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.181 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.181 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:28.182 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.691050+0000", 2024-09-21T21:42:28.182 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00068623399999999998 2024-09-21T21:42:28.182 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.182 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.182 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:28.182 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.691090+0000", 2024-09-21T21:42:28.182 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9406999999999997e-05 2024-09-21T21:42:28.182 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.182 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.182 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:28.182 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.691140+0000", 2024-09-21T21:42:28.182 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0324000000000001e-05 2024-09-21T21:42:28.182 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:28.182 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:28.183 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:28.183 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.183 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.183 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24851 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:28.183 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586661+0000", 2024-09-21T21:42:28.183 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 5.4819869709999995, 2024-09-21T21:42:28.183 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.10536891399999999, 2024-09-21T21:42:28.183 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:28.183 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:28.183 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:28.183 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:28.183 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:42:28.183 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:28.183 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24851 2024-09-21T21:42:28.183 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.184 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:28.184 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.184 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:28.184 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586661+0000", 2024-09-21T21:42:28.184 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:28.184 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.184 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.184 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:28.184 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586661+0000", 2024-09-21T21:42:28.184 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:28.184 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.184 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.184 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:28.184 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586661+0000", 2024-09-21T21:42:28.184 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4700000000000002e-07 2024-09-21T21:42:28.185 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.185 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.185 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:28.185 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586664+0000", 2024-09-21T21:42:28.185 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.672e-06 2024-09-21T21:42:28.185 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.185 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.185 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:28.185 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586671+0000", 2024-09-21T21:42:28.185 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9959999999999996e-06 2024-09-21T21:42:28.185 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.185 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.185 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:28.185 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586676+0000", 2024-09-21T21:42:28.185 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4020000000000001e-06 2024-09-21T21:42:28.186 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.186 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.186 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:28.186 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.689438+0000", 2024-09-21T21:42:28.186 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.102761503 2024-09-21T21:42:28.186 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.186 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.186 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:28.186 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.689524+0000", 2024-09-21T21:42:28.186 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6383000000000003e-05 2024-09-21T21:42:28.186 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.186 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.186 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:42:28.186 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.689769+0000", 2024-09-21T21:42:28.187 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00024504699999999997 2024-09-21T21:42:28.187 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.187 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.187 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:28.187 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.691936+0000", 2024-09-21T21:42:28.187 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021665130000000001 2024-09-21T21:42:28.187 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.187 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.187 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:28.187 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.691947+0000", 2024-09-21T21:42:28.187 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1198e-05 2024-09-21T21:42:28.187 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.187 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.187 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:28.187 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.691992+0000", 2024-09-21T21:42:28.188 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4774000000000002e-05 2024-09-21T21:42:28.188 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.188 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.188 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:28.188 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.692030+0000", 2024-09-21T21:42:28.188 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7978999999999997e-05 2024-09-21T21:42:28.188 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:28.188 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:28.188 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:28.188 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.188 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.188 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24853 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:28.188 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586684+0000", 2024-09-21T21:42:28.188 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 5.4819640139999999, 2024-09-21T21:42:28.188 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.105936852, 2024-09-21T21:42:28.189 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:28.189 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:28.189 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:28.189 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:28.189 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:42:28.189 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:28.189 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24853 2024-09-21T21:42:28.189 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.189 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:28.189 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.189 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:28.189 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586684+0000", 2024-09-21T21:42:28.189 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:28.189 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.189 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.190 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:28.190 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586684+0000", 2024-09-21T21:42:28.190 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:28.190 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.190 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.190 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:28.190 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586684+0000", 2024-09-21T21:42:28.190 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.68e-07 2024-09-21T21:42:28.190 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.190 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.190 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:28.190 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586687+0000", 2024-09-21T21:42:28.190 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.774e-06 2024-09-21T21:42:28.190 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.191 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.191 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:28.191 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586694+0000", 2024-09-21T21:42:28.191 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6959999999999999e-06 2024-09-21T21:42:28.191 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.191 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.191 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:28.191 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586698+0000", 2024-09-21T21:42:28.191 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7489999999999998e-06 2024-09-21T21:42:28.191 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.191 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.191 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:28.191 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690395+0000", 2024-09-21T21:42:28.191 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.103696645 2024-09-21T21:42:28.191 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.192 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.192 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:28.192 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690440+0000", 2024-09-21T21:42:28.192 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4835e-05 2024-09-21T21:42:28.192 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.192 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.192 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:42:28.192 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690654+0000", 2024-09-21T21:42:28.192 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000214114 2024-09-21T21:42:28.192 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.192 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.192 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:28.192 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.692041+0000", 2024-09-21T21:42:28.192 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0013866519999999999 2024-09-21T21:42:28.192 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.193 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.193 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:28.193 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.692558+0000", 2024-09-21T21:42:28.193 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00051717199999999999 2024-09-21T21:42:28.193 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.193 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.193 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:28.193 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.692586+0000", 2024-09-21T21:42:28.193 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8504e-05 2024-09-21T21:42:28.193 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:28.193 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:28.193 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:28.193 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.692620+0000", 2024-09-21T21:42:28.193 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4143000000000002e-05 2024-09-21T21:42:28.193 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:28.194 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:28.194 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:28.194 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:28.194 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:28.194 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:28.195 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T21:42:28.196 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:28.193+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:28.296 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:28.294+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:28.304 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:28.304 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:42:28.304 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 0 2024-09-21T21:42:28.304 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:28.313 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T21:42:28.371 INFO:teuthology.orchestra.run.smithi164.stdout:5799 trunc from 0x940fe00 to 0x8b91800 2024-09-21T21:42:28.398 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:28.397+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:28.406 INFO:teuthology.orchestra.run.smithi164.stdout:5801 trunc from 0x8b91800 to 0x6696800 2024-09-21T21:42:28.434 INFO:teuthology.orchestra.run.smithi164.stdout:5802 write 0x8e50a00 thru 0x8e5bae6 (0xb0e7 bytes) 2024-09-21T21:42:28.435 INFO:teuthology.orchestra.run.smithi164.stdout:5803 write 0x7b5dc00 thru 0x7b637ef (0x5bf0 bytes) 2024-09-21T21:42:28.444 INFO:teuthology.orchestra.run.smithi164.stdout:5804 trunc from 0x8e5bc00 to 0x16e2400 2024-09-21T21:42:28.463 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:28.463 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:42:28.464 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:42:28.464 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:42:28.464 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:28.473 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T21:42:28.492 INFO:teuthology.orchestra.run.smithi164.stdout:5811 read 0x1c2400 thru 0x1c5cc4 (0x38c5 bytes) 2024-09-21T21:42:28.493 INFO:teuthology.orchestra.run.smithi164.stdout:5812 read 0x8f5400 thru 0x8fdff1 (0x8bf2 bytes) 2024-09-21T21:42:28.497 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:28.496+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:28.509 INFO:teuthology.orchestra.run.smithi164.stdout:5813 write 0x7596a00 thru 0x759f5a4 (0x8ba5 bytes) 2024-09-21T21:42:28.510 INFO:teuthology.orchestra.run.smithi164.stdout:5814 write 0x3928e00 thru 0x3931546 (0x8747 bytes) 2024-09-21T21:42:28.511 INFO:teuthology.orchestra.run.smithi164.stdout:5816 trunc from 0x759f600 to 0x7238200 2024-09-21T21:42:28.525 INFO:teuthology.orchestra.run.smithi164.stdout:5819 clone 42 order 21 su 8192 sc 5 2024-09-21T21:42:28.527 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 5, 3] out_osds: [0, 4] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-21T21:42:28.527 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T21:42:28.528 INFO:tasks.thrashosds.thrasher:inject_pause on osd.0 2024-09-21T21:42:28.528 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-21T21:42:28.528 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-21T21:42:28.528 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok config set filestore_inject_stall 3 2024-09-21T21:42:28.600 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:28.599+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:28.622 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:28.622 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 20, 2024-09-21T21:42:28.622 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 600, 2024-09-21T21:42:28.622 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:42:28.622 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.622 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11319 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 5492736~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:28.622 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.712385+0000", 2024-09-21T21:42:28.622 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 37.906461505999999, 2024-09-21T21:42:28.622 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078627889000000006, 2024-09-21T21:42:28.622 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:28.622 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:28.622 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:28.623 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:28.623 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:28.623 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:28.623 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11319 2024-09-21T21:42:28.623 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.623 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:28.623 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.623 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:28.623 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712385+0000", 2024-09-21T21:42:28.623 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:28.623 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.623 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.623 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:28.623 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712385+0000", 2024-09-21T21:42:28.624 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:28.624 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.624 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.624 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:28.624 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712385+0000", 2024-09-21T21:42:28.624 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2800000000000002e-07 2024-09-21T21:42:28.624 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.624 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.624 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:28.624 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712387+0000", 2024-09-21T21:42:28.624 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1569999999999998e-06 2024-09-21T21:42:28.624 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.624 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.624 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:28.624 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712393+0000", 2024-09-21T21:42:28.625 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9370000000000001e-06 2024-09-21T21:42:28.625 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.625 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.625 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:28.625 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712398+0000", 2024-09-21T21:42:28.625 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5020000000000003e-06 2024-09-21T21:42:28.625 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.625 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.625 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:28.625 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.790746+0000", 2024-09-21T21:42:28.625 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078348589999999996 2024-09-21T21:42:28.625 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.625 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.626 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:28.626 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.790844+0000", 2024-09-21T21:42:28.626 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.7526000000000001e-05 2024-09-21T21:42:28.626 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.626 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.626 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:28.626 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.791013+0000", 2024-09-21T21:42:28.626 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00016874899999999999 2024-09-21T21:42:28.626 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:28.626 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:28.626 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:28.626 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.626 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.626 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11320 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 6180864~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:28.627 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.712405+0000", 2024-09-21T21:42:28.627 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 37.906441547, 2024-09-21T21:42:28.627 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078882870999999993, 2024-09-21T21:42:28.627 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:28.627 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:28.627 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:28.627 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:28.627 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:28.627 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:28.627 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11320 2024-09-21T21:42:28.627 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.627 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:28.627 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.627 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:28.628 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712405+0000", 2024-09-21T21:42:28.628 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:28.628 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.628 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.628 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:28.628 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712405+0000", 2024-09-21T21:42:28.628 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:28.628 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.628 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.628 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:28.628 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.713942+0000", 2024-09-21T21:42:28.628 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.001536882 2024-09-21T21:42:28.628 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.628 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.629 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:28.629 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.713946+0000", 2024-09-21T21:42:28.629 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4549999999999997e-06 2024-09-21T21:42:28.629 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.629 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.629 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:28.629 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.713958+0000", 2024-09-21T21:42:28.629 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1775e-05 2024-09-21T21:42:28.629 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.629 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.629 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:28.629 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.713965+0000", 2024-09-21T21:42:28.629 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5100000000000001e-06 2024-09-21T21:42:28.629 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.630 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.630 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:28.630 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.791059+0000", 2024-09-21T21:42:28.630 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.077094036000000005 2024-09-21T21:42:28.630 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.630 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.630 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:28.630 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.791175+0000", 2024-09-21T21:42:28.630 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000115895 2024-09-21T21:42:28.630 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.630 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.630 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:28.630 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.791288+0000", 2024-09-21T21:42:28.630 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000112318 2024-09-21T21:42:28.631 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:28.631 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:28.631 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:28.631 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.631 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.631 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11325 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 5414912~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:28.631 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.714095+0000", 2024-09-21T21:42:28.631 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 37.904751552, 2024-09-21T21:42:28.631 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078631894999999993, 2024-09-21T21:42:28.631 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:28.631 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:28.631 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:28.631 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:28.632 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:28.632 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:28.632 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11325 2024-09-21T21:42:28.632 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.632 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:28.632 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.632 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:28.632 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714095+0000", 2024-09-21T21:42:28.632 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:28.632 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.632 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.632 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:28.632 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714095+0000", 2024-09-21T21:42:28.632 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:28.632 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.633 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.633 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:28.633 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714095+0000", 2024-09-21T21:42:28.633 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7500000000000001e-07 2024-09-21T21:42:28.633 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.633 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.633 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:28.633 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714097+0000", 2024-09-21T21:42:28.633 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.12e-06 2024-09-21T21:42:28.633 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.633 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.633 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:28.633 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714104+0000", 2024-09-21T21:42:28.633 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3520000000000003e-06 2024-09-21T21:42:28.634 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.634 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.634 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:28.634 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714108+0000", 2024-09-21T21:42:28.634 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8810000000000004e-06 2024-09-21T21:42:28.634 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.634 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.634 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:28.634 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792509+0000", 2024-09-21T21:42:28.634 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078400739999999997 2024-09-21T21:42:28.634 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.634 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.634 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:28.634 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792613+0000", 2024-09-21T21:42:28.635 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000103939 2024-09-21T21:42:28.635 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.635 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.635 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:28.635 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792727+0000", 2024-09-21T21:42:28.635 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011348799999999999 2024-09-21T21:42:28.635 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:28.635 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:28.635 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:28.635 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.635 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.635 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11326 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 6103040~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:28.635 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.714115+0000", 2024-09-21T21:42:28.635 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 37.904731523999999, 2024-09-21T21:42:28.636 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078963056000000004, 2024-09-21T21:42:28.636 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:28.636 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:28.636 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:28.636 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:28.636 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:28.636 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:28.636 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11326 2024-09-21T21:42:28.636 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.636 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:28.636 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.636 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:28.636 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714115+0000", 2024-09-21T21:42:28.636 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:28.637 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.637 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.637 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:28.637 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714115+0000", 2024-09-21T21:42:28.637 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:28.637 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.637 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.637 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:28.637 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.715940+0000", 2024-09-21T21:42:28.637 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0018254969999999999 2024-09-21T21:42:28.637 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.637 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.637 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:28.637 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.715945+0000", 2024-09-21T21:42:28.637 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6120000000000001e-06 2024-09-21T21:42:28.638 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.638 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.638 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:28.638 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.715954+0000", 2024-09-21T21:42:28.638 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.5899999999999997e-06 2024-09-21T21:42:28.638 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.638 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.638 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:28.638 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.715959+0000", 2024-09-21T21:42:28.638 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0849999999999996e-06 2024-09-21T21:42:28.638 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.638 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.638 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:28.638 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792771+0000", 2024-09-21T21:42:28.639 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.076811829999999998 2024-09-21T21:42:28.639 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.639 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.639 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:28.639 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792882+0000", 2024-09-21T21:42:28.639 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000110857 2024-09-21T21:42:28.639 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.639 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.639 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:28.639 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.793078+0000", 2024-09-21T21:42:28.639 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00019558499999999999 2024-09-21T21:42:28.639 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:28.639 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:28.639 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:28.639 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.640 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.640 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11332 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 7565312~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:28.640 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.716063+0000", 2024-09-21T21:42:28.640 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 37.902782989999999, 2024-09-21T21:42:28.640 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078787482000000006, 2024-09-21T21:42:28.640 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:28.640 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:28.640 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:28.640 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:28.640 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:28.640 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:28.640 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11332 2024-09-21T21:42:28.640 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.640 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:28.640 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.641 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:28.641 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.716063+0000", 2024-09-21T21:42:28.641 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:28.641 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.641 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.641 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:28.641 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.716063+0000", 2024-09-21T21:42:28.641 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:28.641 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.641 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.641 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:28.641 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.717944+0000", 2024-09-21T21:42:28.641 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0018812309999999999 2024-09-21T21:42:28.641 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.642 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.642 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:28.642 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.717949+0000", 2024-09-21T21:42:28.642 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9389999999999996e-06 2024-09-21T21:42:28.642 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.642 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.642 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:28.642 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.717963+0000", 2024-09-21T21:42:28.642 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.3899e-05 2024-09-21T21:42:28.642 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.642 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.642 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:28.642 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.717970+0000", 2024-09-21T21:42:28.642 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.426e-06 2024-09-21T21:42:28.643 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.643 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.643 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:28.643 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.794573+0000", 2024-09-21T21:42:28.643 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.076602990999999995 2024-09-21T21:42:28.643 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.643 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.643 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:28.643 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.794682+0000", 2024-09-21T21:42:28.643 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000109541 2024-09-21T21:42:28.643 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.643 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.643 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:28.643 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.794851+0000", 2024-09-21T21:42:28.643 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000168455 2024-09-21T21:42:28.644 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:28.644 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:28.644 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:28.644 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.644 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.644 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11339 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 6266880~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:28.644 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.718117+0000", 2024-09-21T21:42:28.644 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 37.900728964000002, 2024-09-21T21:42:28.644 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078703717000000006, 2024-09-21T21:42:28.644 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:28.644 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:28.644 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:28.644 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:28.644 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:28.645 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:28.645 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11339 2024-09-21T21:42:28.645 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.645 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:28.645 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.645 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:28.645 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.718117+0000", 2024-09-21T21:42:28.645 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:28.645 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.645 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.645 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:28.645 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.718117+0000", 2024-09-21T21:42:28.645 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:28.645 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.645 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.646 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:28.646 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720019+0000", 2024-09-21T21:42:28.646 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0019012549999999999 2024-09-21T21:42:28.646 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.646 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.646 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:28.646 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720024+0000", 2024-09-21T21:42:28.646 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1100000000000002e-06 2024-09-21T21:42:28.646 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.646 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.646 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:28.646 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720035+0000", 2024-09-21T21:42:28.646 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.094e-05 2024-09-21T21:42:28.646 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.646 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.647 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:28.647 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720040+0000", 2024-09-21T21:42:28.647 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1830000000000002e-06 2024-09-21T21:42:28.647 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.647 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.647 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:28.647 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796618+0000", 2024-09-21T21:42:28.647 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.076578033000000004 2024-09-21T21:42:28.647 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.647 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.647 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:28.647 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796696+0000", 2024-09-21T21:42:28.647 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7836000000000005e-05 2024-09-21T21:42:28.647 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.647 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.648 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:28.648 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796821+0000", 2024-09-21T21:42:28.648 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00012536000000000001 2024-09-21T21:42:28.648 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:28.648 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:28.648 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:28.648 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.648 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.648 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11345 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 5472256~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:28.648 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.720116+0000", 2024-09-21T21:42:28.648 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 37.898730063999999, 2024-09-21T21:42:28.648 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078667730000000005, 2024-09-21T21:42:28.648 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:28.648 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:28.648 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:28.649 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:28.649 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:28.649 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:28.649 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11345 2024-09-21T21:42:28.649 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.649 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:28.649 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.649 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:28.649 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720116+0000", 2024-09-21T21:42:28.649 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:28.649 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.649 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.649 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:28.649 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720116+0000", 2024-09-21T21:42:28.650 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:28.650 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.650 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.650 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:28.650 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720116+0000", 2024-09-21T21:42:28.650 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.04e-07 2024-09-21T21:42:28.650 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.650 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.650 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:28.650 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720118+0000", 2024-09-21T21:42:28.650 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.181e-06 2024-09-21T21:42:28.650 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.650 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.650 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:28.651 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720121+0000", 2024-09-21T21:42:28.651 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0199999999999999e-06 2024-09-21T21:42:28.651 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.651 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.651 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:28.651 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720123+0000", 2024-09-21T21:42:28.651 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.5469999999999999e-06 2024-09-21T21:42:28.651 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.651 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.651 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:28.651 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798626+0000", 2024-09-21T21:42:28.651 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078503137000000001 2024-09-21T21:42:28.651 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.651 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.651 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:28.652 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798677+0000", 2024-09-21T21:42:28.652 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0930999999999997e-05 2024-09-21T21:42:28.652 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.652 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.652 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:28.652 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798784+0000", 2024-09-21T21:42:28.652 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010671 2024-09-21T21:42:28.652 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:28.652 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:28.652 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:28.652 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.652 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.652 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11346 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 5500928~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:28.652 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.720126+0000", 2024-09-21T21:42:28.653 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 37.898719991, 2024-09-21T21:42:28.653 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078863421000000003, 2024-09-21T21:42:28.653 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:28.653 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:28.653 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:28.653 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:28.653 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:28.653 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:28.653 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11346 2024-09-21T21:42:28.653 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.653 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:28.653 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.653 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:28.653 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720126+0000", 2024-09-21T21:42:28.653 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:28.654 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.654 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.654 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:28.654 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720126+0000", 2024-09-21T21:42:28.654 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:28.654 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.654 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.654 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:28.654 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.722145+0000", 2024-09-21T21:42:28.654 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0020183219999999999 2024-09-21T21:42:28.654 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.654 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.654 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:28.654 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.722150+0000", 2024-09-21T21:42:28.655 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1760000000000001e-06 2024-09-21T21:42:28.655 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.655 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.655 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:28.655 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.722161+0000", 2024-09-21T21:42:28.655 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1566000000000001e-05 2024-09-21T21:42:28.655 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.655 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.655 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:28.655 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.722169+0000", 2024-09-21T21:42:28.655 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.2840000000000001e-06 2024-09-21T21:42:28.655 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.655 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.655 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:28.656 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798815+0000", 2024-09-21T21:42:28.656 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.076646607000000005 2024-09-21T21:42:28.656 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.656 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.656 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:28.656 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798870+0000", 2024-09-21T21:42:28.656 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4486e-05 2024-09-21T21:42:28.656 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.656 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.656 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:28.656 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798990+0000", 2024-09-21T21:42:28.656 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011998 2024-09-21T21:42:28.656 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:28.656 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:28.656 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:28.657 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.657 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.657 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13629 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7852032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:28.657 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943319+0000", 2024-09-21T21:42:28.657 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 37.675527348999999, 2024-09-21T21:42:28.657 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078610975999999999, 2024-09-21T21:42:28.657 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:28.657 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:28.657 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:28.657 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:28.657 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:28.657 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:28.657 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13629 2024-09-21T21:42:28.658 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.658 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:28.658 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.658 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:28.658 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943319+0000", 2024-09-21T21:42:28.658 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:28.658 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.658 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.658 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:28.658 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943319+0000", 2024-09-21T21:42:28.658 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:28.658 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.658 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.658 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:28.658 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943319+0000", 2024-09-21T21:42:28.659 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6199999999999999e-07 2024-09-21T21:42:28.659 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.659 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.659 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:28.659 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943321+0000", 2024-09-21T21:42:28.659 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1009999999999999e-06 2024-09-21T21:42:28.659 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.659 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.659 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:28.659 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943328+0000", 2024-09-21T21:42:28.659 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.1029999999999996e-06 2024-09-21T21:42:28.659 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.659 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.659 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:28.660 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943333+0000", 2024-09-21T21:42:28.660 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7709999999999997e-06 2024-09-21T21:42:28.660 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.660 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.660 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:28.660 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.021679+0000", 2024-09-21T21:42:28.660 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078345673000000005 2024-09-21T21:42:28.660 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.660 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.660 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:28.660 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.021771+0000", 2024-09-21T21:42:28.660 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2201999999999996e-05 2024-09-21T21:42:28.660 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.660 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.661 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:28.661 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.021930+0000", 2024-09-21T21:42:28.661 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00015876400000000001 2024-09-21T21:42:28.661 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:28.661 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:28.661 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:28.661 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.661 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.661 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13630 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7880704~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:28.661 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943339+0000", 2024-09-21T21:42:28.661 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 37.675507160999999, 2024-09-21T21:42:28.661 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078939410000000002, 2024-09-21T21:42:28.661 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:28.661 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:28.661 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:28.662 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:28.662 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:28.662 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:28.662 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13630 2024-09-21T21:42:28.662 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.662 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:28.662 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.662 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:28.662 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943339+0000", 2024-09-21T21:42:28.662 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:28.662 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.662 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.662 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:28.663 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943339+0000", 2024-09-21T21:42:28.663 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:28.663 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.663 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.663 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:28.663 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943339+0000", 2024-09-21T21:42:28.663 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8000000000000001e-07 2024-09-21T21:42:28.663 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.663 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.663 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:28.663 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943341+0000", 2024-09-21T21:42:28.663 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9850000000000001e-06 2024-09-21T21:42:28.663 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.663 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.663 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:28.664 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943347+0000", 2024-09-21T21:42:28.664 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2979999999999999e-06 2024-09-21T21:42:28.664 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.664 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.664 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:28.664 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943354+0000", 2024-09-21T21:42:28.664 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.2969999999999998e-06 2024-09-21T21:42:28.664 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.664 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.664 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:28.664 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.021996+0000", 2024-09-21T21:42:28.664 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078641680000000005 2024-09-21T21:42:28.664 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.664 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.665 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:28.665 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022104+0000", 2024-09-21T21:42:28.665 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000107902 2024-09-21T21:42:28.665 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.665 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.665 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:28.665 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022278+0000", 2024-09-21T21:42:28.665 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00017486800000000001 2024-09-21T21:42:28.665 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:28.665 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:28.665 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:28.665 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.665 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.666 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13631 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7909376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:28.666 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943362+0000", 2024-09-21T21:42:28.666 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 37.675484341000001, 2024-09-21T21:42:28.666 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079195644999999995, 2024-09-21T21:42:28.666 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:28.666 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:28.666 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:28.666 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:28.666 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:28.666 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:28.666 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13631 2024-09-21T21:42:28.666 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.666 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:28.666 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.667 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:28.667 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943362+0000", 2024-09-21T21:42:28.667 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:28.667 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.667 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.667 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:28.667 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943362+0000", 2024-09-21T21:42:28.667 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:28.667 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.667 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.667 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:28.667 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943362+0000", 2024-09-21T21:42:28.667 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8799999999999998e-07 2024-09-21T21:42:28.667 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.667 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.668 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:28.668 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943364+0000", 2024-09-21T21:42:28.668 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9850000000000001e-06 2024-09-21T21:42:28.668 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.668 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.668 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:28.668 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943371+0000", 2024-09-21T21:42:28.668 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0650000000000001e-06 2024-09-21T21:42:28.668 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.668 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.668 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:28.668 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943376+0000", 2024-09-21T21:42:28.668 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8250000000000004e-06 2024-09-21T21:42:28.668 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.668 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.669 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:28.669 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022329+0000", 2024-09-21T21:42:28.669 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078952888999999998 2024-09-21T21:42:28.669 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.669 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.669 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:28.669 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022419+0000", 2024-09-21T21:42:28.669 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.9738e-05 2024-09-21T21:42:28.669 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.669 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.669 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:28.669 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022558+0000", 2024-09-21T21:42:28.669 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00013875500000000001 2024-09-21T21:42:28.669 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:28.670 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:28.670 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:28.670 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.670 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.670 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13632 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7966720~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:28.670 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943382+0000", 2024-09-21T21:42:28.670 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 37.675464107000003, 2024-09-21T21:42:28.670 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079495727000000002, 2024-09-21T21:42:28.670 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:28.670 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:28.670 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:28.670 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:28.670 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:28.671 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:28.671 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13632 2024-09-21T21:42:28.671 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.671 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:28.671 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.671 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:28.671 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943382+0000", 2024-09-21T21:42:28.671 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:28.671 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.671 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.671 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:28.671 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943382+0000", 2024-09-21T21:42:28.671 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:28.671 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.672 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.672 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:28.672 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943382+0000", 2024-09-21T21:42:28.672 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7e-07 2024-09-21T21:42:28.672 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.672 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.672 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:28.672 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943385+0000", 2024-09-21T21:42:28.672 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2230000000000001e-06 2024-09-21T21:42:28.672 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.672 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.672 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:28.672 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943391+0000", 2024-09-21T21:42:28.672 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6810000000000001e-06 2024-09-21T21:42:28.672 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.673 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.673 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:28.673 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943397+0000", 2024-09-21T21:42:28.673 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6559999999999997e-06 2024-09-21T21:42:28.673 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.673 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.673 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:28.673 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022601+0000", 2024-09-21T21:42:28.673 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079204051999999997 2024-09-21T21:42:28.673 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.673 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.673 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:28.673 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022704+0000", 2024-09-21T21:42:28.673 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010329599999999999 2024-09-21T21:42:28.674 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.674 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.674 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:28.674 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022878+0000", 2024-09-21T21:42:28.674 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000173449 2024-09-21T21:42:28.674 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:28.674 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:28.674 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:28.674 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.674 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.674 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13633 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7938048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:28.674 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943403+0000", 2024-09-21T21:42:28.674 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 37.675443176999998, 2024-09-21T21:42:28.674 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079794697999999997, 2024-09-21T21:42:28.675 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:28.675 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:28.675 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:28.675 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:28.675 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:28.675 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:28.675 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13633 2024-09-21T21:42:28.675 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.675 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:28.675 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.675 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:28.675 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943403+0000", 2024-09-21T21:42:28.675 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:28.675 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.676 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.676 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:28.676 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943403+0000", 2024-09-21T21:42:28.676 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:28.676 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.676 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.676 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:28.676 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943403+0000", 2024-09-21T21:42:28.676 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.65e-07 2024-09-21T21:42:28.676 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.676 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.676 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:28.676 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943405+0000", 2024-09-21T21:42:28.677 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0020000000000001e-06 2024-09-21T21:42:28.677 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.677 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.677 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:28.677 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943412+0000", 2024-09-21T21:42:28.677 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.8379999999999996e-06 2024-09-21T21:42:28.677 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.677 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.677 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:28.677 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943417+0000", 2024-09-21T21:42:28.677 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7650000000000001e-06 2024-09-21T21:42:28.677 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.677 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.677 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:28.678 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022922+0000", 2024-09-21T21:42:28.678 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079504512999999999 2024-09-21T21:42:28.678 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.678 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.678 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:28.678 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023039+0000", 2024-09-21T21:42:28.678 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000117044 2024-09-21T21:42:28.678 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.678 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.678 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:28.678 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023198+0000", 2024-09-21T21:42:28.678 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000159171 2024-09-21T21:42:28.678 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:28.678 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:28.679 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:28.679 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.679 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.679 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13634 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7995392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:28.679 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943423+0000", 2024-09-21T21:42:28.679 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 37.675423469000002, 2024-09-21T21:42:28.679 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.080142358999999996, 2024-09-21T21:42:28.679 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:28.679 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:28.679 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:28.679 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:28.679 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:28.679 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:28.679 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13634 2024-09-21T21:42:28.679 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.680 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:28.680 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.680 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:28.680 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943423+0000", 2024-09-21T21:42:28.680 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:28.680 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.680 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.680 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:28.680 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943423+0000", 2024-09-21T21:42:28.680 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:28.680 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.680 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.680 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:28.680 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943423+0000", 2024-09-21T21:42:28.681 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.58e-07 2024-09-21T21:42:28.681 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.681 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.681 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:28.681 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943425+0000", 2024-09-21T21:42:28.681 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9180000000000001e-06 2024-09-21T21:42:28.681 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.681 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.681 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:28.681 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943431+0000", 2024-09-21T21:42:28.681 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4439999999999996e-06 2024-09-21T21:42:28.681 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.681 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.681 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:28.681 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943436+0000", 2024-09-21T21:42:28.682 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.887e-06 2024-09-21T21:42:28.682 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.682 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.682 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:28.682 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023266+0000", 2024-09-21T21:42:28.682 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079829622000000003 2024-09-21T21:42:28.682 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.682 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.682 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:28.682 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023384+0000", 2024-09-21T21:42:28.682 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000117658 2024-09-21T21:42:28.682 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.682 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.683 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:28.683 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023565+0000", 2024-09-21T21:42:28.683 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00018147200000000001 2024-09-21T21:42:28.683 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:28.683 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:28.683 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:28.683 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.683 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.683 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13635 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 8024064~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:28.683 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943447+0000", 2024-09-21T21:42:28.683 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 37.675398928999996, 2024-09-21T21:42:28.683 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.080417653000000006, 2024-09-21T21:42:28.683 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:28.683 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:28.684 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:28.684 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:28.684 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:28.684 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:28.684 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13635 2024-09-21T21:42:28.684 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.684 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:28.684 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.684 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:28.684 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943447+0000", 2024-09-21T21:42:28.684 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:28.684 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.684 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.684 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:28.685 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943447+0000", 2024-09-21T21:42:28.685 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:28.685 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.685 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.685 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:28.685 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943448+0000", 2024-09-21T21:42:28.685 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1699999999999999e-07 2024-09-21T21:42:28.685 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.685 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.685 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:28.685 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943450+0000", 2024-09-21T21:42:28.685 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1639999999999999e-06 2024-09-21T21:42:28.685 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.685 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.686 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:28.686 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943456+0000", 2024-09-21T21:42:28.686 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8379999999999998e-06 2024-09-21T21:42:28.686 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.686 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.686 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:28.686 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943460+0000", 2024-09-21T21:42:28.686 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4700000000000004e-06 2024-09-21T21:42:28.686 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.686 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.686 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:28.686 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023595+0000", 2024-09-21T21:42:28.686 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.080134869999999997 2024-09-21T21:42:28.686 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.686 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.687 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:28.687 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023688+0000", 2024-09-21T21:42:28.687 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2794000000000006e-05 2024-09-21T21:42:28.687 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.687 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.687 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:28.687 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023865+0000", 2024-09-21T21:42:28.687 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0001771 2024-09-21T21:42:28.687 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:28.687 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:28.687 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:28.687 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.687 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.687 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13636 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 8052736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:28.688 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943467+0000", 2024-09-21T21:42:28.688 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 37.675379192000001, 2024-09-21T21:42:28.688 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.080694399999999999, 2024-09-21T21:42:28.688 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:28.688 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:28.688 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:28.688 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:28.688 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:28.688 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:28.688 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13636 2024-09-21T21:42:28.688 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.688 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:28.688 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.688 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:28.689 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943467+0000", 2024-09-21T21:42:28.689 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:28.689 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.689 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.689 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:28.689 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943467+0000", 2024-09-21T21:42:28.689 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:28.689 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.689 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.689 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:28.689 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944940+0000", 2024-09-21T21:42:28.689 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0014725109999999999 2024-09-21T21:42:28.689 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.689 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.689 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:28.690 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944949+0000", 2024-09-21T21:42:28.690 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2930000000000006e-06 2024-09-21T21:42:28.690 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.690 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.690 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:28.690 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944958+0000", 2024-09-21T21:42:28.690 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.6899999999999998e-06 2024-09-21T21:42:28.690 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.690 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.690 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:28.690 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944966+0000", 2024-09-21T21:42:28.690 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0690000000000002e-06 2024-09-21T21:42:28.690 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.691 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.691 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:28.691 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023900+0000", 2024-09-21T21:42:28.691 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078934371000000003 2024-09-21T21:42:28.691 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.691 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.691 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:28.691 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024009+0000", 2024-09-21T21:42:28.691 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000108857 2024-09-21T21:42:28.691 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.691 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.691 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:28.691 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024161+0000", 2024-09-21T21:42:28.691 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000152609 2024-09-21T21:42:28.692 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:28.692 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:28.692 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:28.692 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.692 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.692 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13637 3.12 3.1dc31b72 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:28.692 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.944978+0000", 2024-09-21T21:42:28.692 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 37.673868257000002, 2024-09-21T21:42:28.692 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079232604999999998, 2024-09-21T21:42:28.692 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:28.692 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:28.692 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:42:28.692 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:28.692 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:28.692 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:28.693 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13637 2024-09-21T21:42:28.693 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.693 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:28.693 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.693 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:28.693 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944978+0000", 2024-09-21T21:42:28.693 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:28.693 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.693 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.693 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:28.693 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944978+0000", 2024-09-21T21:42:28.693 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:28.693 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.693 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.694 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:28.694 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944978+0000", 2024-09-21T21:42:28.694 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9100000000000004e-07 2024-09-21T21:42:28.694 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.694 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.694 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:28.694 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944981+0000", 2024-09-21T21:42:28.694 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3290000000000001e-06 2024-09-21T21:42:28.694 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.694 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.694 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:28.694 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944987+0000", 2024-09-21T21:42:28.694 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.28e-06 2024-09-21T21:42:28.694 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.695 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.695 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:28.695 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944992+0000", 2024-09-21T21:42:28.695 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8520000000000003e-06 2024-09-21T21:42:28.695 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.695 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.695 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:28.695 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024196+0000", 2024-09-21T21:42:28.695 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079203673000000002 2024-09-21T21:42:28.695 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.695 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.695 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:28.695 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024211+0000", 2024-09-21T21:42:28.695 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.4980000000000001e-05 2024-09-21T21:42:28.696 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:28.696 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:28.696 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:28.696 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.696 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.696 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13638 3.12 3.1dc31b72 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:28.696 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.945000+0000", 2024-09-21T21:42:28.696 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 37.673846128999998, 2024-09-21T21:42:28.696 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079257465999999999, 2024-09-21T21:42:28.696 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:28.696 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:28.696 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:42:28.696 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:28.696 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:28.696 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:28.697 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13638 2024-09-21T21:42:28.697 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.697 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:28.697 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.697 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:28.697 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945000+0000", 2024-09-21T21:42:28.697 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:28.697 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.697 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.697 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:28.697 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945000+0000", 2024-09-21T21:42:28.697 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:28.697 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.697 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.697 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:28.698 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945001+0000", 2024-09-21T21:42:28.698 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.32e-07 2024-09-21T21:42:28.698 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.698 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.698 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:28.698 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945003+0000", 2024-09-21T21:42:28.698 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2309999999999999e-06 2024-09-21T21:42:28.698 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.698 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.698 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:28.698 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945009+0000", 2024-09-21T21:42:28.698 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7450000000000001e-06 2024-09-21T21:42:28.698 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.698 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.699 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:28.699 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945013+0000", 2024-09-21T21:42:28.699 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.673e-06 2024-09-21T21:42:28.699 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.699 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.699 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:28.699 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024245+0000", 2024-09-21T21:42:28.699 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079232160999999995 2024-09-21T21:42:28.699 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.699 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.699 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:28.699 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024258+0000", 2024-09-21T21:42:28.699 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.2224e-05 2024-09-21T21:42:28.699 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:28.700 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:28.700 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:28.700 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.700 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.700 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13639 3.12 3.1dc31b72 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:28.700 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.945034+0000", 2024-09-21T21:42:28.700 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 37.673812329999997, 2024-09-21T21:42:28.700 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079260627, 2024-09-21T21:42:28.700 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:28.700 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:28.700 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:42:28.700 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:28.700 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:28.700 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:28.700 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13639 2024-09-21T21:42:28.701 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.701 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:28.701 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.701 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:28.701 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945034+0000", 2024-09-21T21:42:28.701 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:28.701 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.701 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.701 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:28.701 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945034+0000", 2024-09-21T21:42:28.701 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:28.701 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.701 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.701 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:28.702 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945034+0000", 2024-09-21T21:42:28.702 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3799999999999998e-07 2024-09-21T21:42:28.702 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.702 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.702 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:28.702 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945036+0000", 2024-09-21T21:42:28.702 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.052e-06 2024-09-21T21:42:28.702 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.702 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.702 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:28.702 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945040+0000", 2024-09-21T21:42:28.702 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1169999999999997e-06 2024-09-21T21:42:28.702 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.702 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.703 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:28.703 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945043+0000", 2024-09-21T21:42:28.703 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.8439999999999998e-06 2024-09-21T21:42:28.703 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.703 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.703 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:28.703 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024286+0000", 2024-09-21T21:42:28.703 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079242483000000002 2024-09-21T21:42:28.703 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.703 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.703 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:28.703 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024295+0000", 2024-09-21T21:42:28.703 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.7930000000000003e-06 2024-09-21T21:42:28.703 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:28.704 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:28.704 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:28.704 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.704 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.704 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13640 3.12 3.1dc31b72 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:28.704 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.945047+0000", 2024-09-21T21:42:28.704 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 37.673798808000001, 2024-09-21T21:42:28.704 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079412321999999994, 2024-09-21T21:42:28.704 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:28.704 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:28.704 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:42:28.704 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:28.704 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:28.704 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:28.705 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13640 2024-09-21T21:42:28.705 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.705 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:28.705 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.705 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:28.705 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945047+0000", 2024-09-21T21:42:28.705 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:28.705 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.705 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.705 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:28.705 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945047+0000", 2024-09-21T21:42:28.705 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:28.705 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.705 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.705 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:28.706 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945048+0000", 2024-09-21T21:42:28.706 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0499999999999999e-07 2024-09-21T21:42:28.706 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.706 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.706 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:28.706 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945049+0000", 2024-09-21T21:42:28.706 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.3939999999999999e-06 2024-09-21T21:42:28.706 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.706 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.706 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:28.706 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945053+0000", 2024-09-21T21:42:28.706 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1760000000000003e-06 2024-09-21T21:42:28.706 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.706 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.707 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:28.707 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945056+0000", 2024-09-21T21:42:28.707 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.6129999999999998e-06 2024-09-21T21:42:28.707 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.707 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.707 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:28.707 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024360+0000", 2024-09-21T21:42:28.707 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079303645000000006 2024-09-21T21:42:28.707 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:28.707 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:28.707 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:28.707 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024460+0000", 2024-09-21T21:42:28.707 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000100189 2024-09-21T21:42:28.707 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:28.708 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:28.708 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:28.708 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:28.708 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:28.708 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:28.709 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T21:42:28.710 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:28.710 INFO:teuthology.orchestra.run.smithi081.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-21T21:42:28.710 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:28.711 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:28.708+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:28.774 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T21:42:28.810 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:28.809+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:28.855 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:28.855 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:42:28.855 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 0 2024-09-21T21:42:28.855 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:28.864 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T21:42:28.911 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:28.910+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:28.928 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0: {} 2024-09-21T21:42:28.928 INFO:teuthology.orchestra.run.smithi081.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T21:42:28.945 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1: {} 2024-09-21T21:42:28.946 INFO:teuthology.orchestra.run.smithi081.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T21:42:28.962 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2: {} 2024-09-21T21:42:28.962 INFO:teuthology.orchestra.run.smithi081.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T21:42:28.978 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3: {} 2024-09-21T21:42:28.979 INFO:teuthology.orchestra.run.smithi081.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T21:42:28.981 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:28.981 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:28.990 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T21:42:28.995 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4: {} 2024-09-21T21:42:28.996 INFO:teuthology.orchestra.run.smithi081.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T21:42:29.012 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5: {} 2024-09-21T21:42:29.013 INFO:teuthology.orchestra.run.smithi081.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T21:42:29.013 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:29.011+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:29.098 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:29.098 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:29.107 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T21:42:29.114 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:29.112+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:29.209 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:29.210 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:29.214 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:29.212+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:29.218 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T21:42:29.315 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:29.314+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:29.362 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:29.362 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:29.371 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T21:42:29.416 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:29.415+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:29.514 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:29.515 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:29.516 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:29.515+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:29.523 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T21:42:29.578 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T21:42:29.614 INFO:teuthology.orchestra.run.smithi164.stdout:truncating image image_client.0-clone41 from 0x7238200 (overlap 0x16e2400) to 0x0 2024-09-21T21:42:29.621 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:29.620+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:29.648 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:29.649 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:29.658 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T21:42:29.717 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:29.716+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:29.764 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:29.764 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:29.773 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T21:42:29.820 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:29.818+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:29.905 INFO:teuthology.orchestra.run.smithi164.stdout:checking clone #40, image image_client.0-clone40 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent41 2024-09-21T21:42:29.921 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:29.919+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:29.922 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:29.922 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:29.931 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T21:42:30.022 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:30.020+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:30.046 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:30.046 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:30.056 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T21:42:30.140 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:30.139+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:30.208 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:30.209 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:30.217 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T21:42:30.259 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:30.257+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:30.326 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:30.325+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:30.333 INFO:teuthology.orchestra.run.smithi164.stdout:5820 read 0x1fad400 thru 0x1fb7388 (0x9f89 bytes) 2024-09-21T21:42:30.339 INFO:teuthology.orchestra.run.smithi164.stdout:5824 trunc from 0x7238200 to 0xb47d800 2024-09-21T21:42:30.355 INFO:teuthology.orchestra.run.smithi164.stdout:5825 read 0x64af600 thru 0x64b077c (0x117d bytes) 2024-09-21T21:42:30.356 INFO:teuthology.orchestra.run.smithi164.stdout:5826 read 0x570c000 thru 0x5715101 (0x9102 bytes) 2024-09-21T21:42:30.365 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:30.365 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:30.373 INFO:teuthology.orchestra.run.smithi164.stdout:5828 write 0xe377400 thru 0xe37c48e (0x508f bytes) 2024-09-21T21:42:30.374 INFO:teuthology.orchestra.run.smithi164.stdout:5829 write 0x2ca5400 thru 0x2cad46c (0x806d bytes) 2024-09-21T21:42:30.374 INFO:teuthology.orchestra.run.smithi164.stdout:5832 write 0xc1e7a00 thru 0xc1f5371 (0xd972 bytes) 2024-09-21T21:42:30.374 INFO:teuthology.orchestra.run.smithi164.stdout:5833 clone 43 order 22 su 65536 sc 10 2024-09-21T21:42:30.376 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T21:42:30.427 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:30.425+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:30.523 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:30.524 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:30.528 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:30.527+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:30.532 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T21:42:30.539 INFO:teuthology.orchestra.run.smithi081.stderr:noscrub is unset 2024-09-21T21:42:30.616 INFO:teuthology.orchestra.run.smithi164.stdout:truncating image image_client.0-clone42 from 0xe37c600 (overlap 0x7238200) to 0x0 2024-09-21T21:42:30.629 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:30.627+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:30.659 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:30.659 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:30.673 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T21:42:30.758 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:30.757+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:30.830 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:30.829+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:30.874 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:30.874 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:30.887 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T21:42:30.931 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:30.930+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:31.005 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:31.005 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:31.014 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T21:42:31.024 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T21:42:31.032 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:31.031+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:31.133 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:31.132+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:31.162 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:31.163 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:31.172 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T21:42:31.177 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0: {} 2024-09-21T21:42:31.177 INFO:teuthology.orchestra.run.smithi081.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T21:42:31.194 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1: {} 2024-09-21T21:42:31.195 INFO:teuthology.orchestra.run.smithi081.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T21:42:31.211 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2: {} 2024-09-21T21:42:31.212 INFO:teuthology.orchestra.run.smithi081.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T21:42:31.229 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3: {} 2024-09-21T21:42:31.229 INFO:teuthology.orchestra.run.smithi081.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T21:42:31.234 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:31.233+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:31.246 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4: {} 2024-09-21T21:42:31.246 INFO:teuthology.orchestra.run.smithi081.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T21:42:31.264 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5: {} 2024-09-21T21:42:31.265 INFO:teuthology.orchestra.run.smithi081.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T21:42:31.287 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:31.288 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 20, 2024-09-21T21:42:31.288 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 600, 2024-09-21T21:42:31.288 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:42:31.288 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.288 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11319 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 5492736~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:31.288 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.712385+0000", 2024-09-21T21:42:31.288 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 40.573302579, 2024-09-21T21:42:31.288 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078627889000000006, 2024-09-21T21:42:31.288 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.288 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.288 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:31.289 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.289 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:31.289 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:31.289 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11319 2024-09-21T21:42:31.289 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.289 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.289 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.289 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.289 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712385+0000", 2024-09-21T21:42:31.289 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.289 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.289 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.289 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.289 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712385+0000", 2024-09-21T21:42:31.289 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.290 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.290 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.290 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.290 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712385+0000", 2024-09-21T21:42:31.290 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2800000000000002e-07 2024-09-21T21:42:31.290 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.290 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.290 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.290 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712387+0000", 2024-09-21T21:42:31.290 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1569999999999998e-06 2024-09-21T21:42:31.290 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.290 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.290 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.291 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712393+0000", 2024-09-21T21:42:31.291 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9370000000000001e-06 2024-09-21T21:42:31.291 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.291 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.291 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.291 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712398+0000", 2024-09-21T21:42:31.291 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5020000000000003e-06 2024-09-21T21:42:31.291 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.291 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.291 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:31.291 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.790746+0000", 2024-09-21T21:42:31.291 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078348589999999996 2024-09-21T21:42:31.291 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.291 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.292 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:31.292 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.790844+0000", 2024-09-21T21:42:31.292 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.7526000000000001e-05 2024-09-21T21:42:31.292 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.292 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.292 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:31.292 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.791013+0000", 2024-09-21T21:42:31.292 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00016874899999999999 2024-09-21T21:42:31.292 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.292 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.292 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.292 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.292 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.292 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11320 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 6180864~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:31.293 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.712405+0000", 2024-09-21T21:42:31.293 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 40.573282620000001, 2024-09-21T21:42:31.293 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078882870999999993, 2024-09-21T21:42:31.293 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.293 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.293 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:31.293 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.293 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:31.293 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:31.293 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11320 2024-09-21T21:42:31.293 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.293 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.293 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.293 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.294 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712405+0000", 2024-09-21T21:42:31.294 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.294 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.294 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.294 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.294 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712405+0000", 2024-09-21T21:42:31.294 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.294 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.294 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.294 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.294 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.713942+0000", 2024-09-21T21:42:31.294 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.001536882 2024-09-21T21:42:31.294 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.294 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.294 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.295 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.713946+0000", 2024-09-21T21:42:31.295 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4549999999999997e-06 2024-09-21T21:42:31.295 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.295 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.295 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.295 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.713958+0000", 2024-09-21T21:42:31.295 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1775e-05 2024-09-21T21:42:31.296 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.296 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.296 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.296 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.713965+0000", 2024-09-21T21:42:31.296 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5100000000000001e-06 2024-09-21T21:42:31.296 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.296 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.296 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:31.296 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.791059+0000", 2024-09-21T21:42:31.296 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.077094036000000005 2024-09-21T21:42:31.296 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.296 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.296 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:31.297 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.791175+0000", 2024-09-21T21:42:31.297 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000115895 2024-09-21T21:42:31.297 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.297 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.297 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:31.297 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.791288+0000", 2024-09-21T21:42:31.297 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000112318 2024-09-21T21:42:31.297 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.297 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.297 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.297 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.297 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.297 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11325 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 5414912~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:31.297 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.714095+0000", 2024-09-21T21:42:31.298 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 40.571592625000001, 2024-09-21T21:42:31.298 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078631894999999993, 2024-09-21T21:42:31.298 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.298 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.298 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:31.298 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.298 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:31.298 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:31.298 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11325 2024-09-21T21:42:31.298 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.298 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.298 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.298 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.298 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714095+0000", 2024-09-21T21:42:31.299 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.299 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.299 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.299 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.299 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714095+0000", 2024-09-21T21:42:31.299 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.299 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.299 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.299 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.299 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714095+0000", 2024-09-21T21:42:31.299 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7500000000000001e-07 2024-09-21T21:42:31.299 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.300 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.300 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.300 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714097+0000", 2024-09-21T21:42:31.300 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.12e-06 2024-09-21T21:42:31.300 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.300 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.300 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.300 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714104+0000", 2024-09-21T21:42:31.300 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3520000000000003e-06 2024-09-21T21:42:31.300 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.300 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.300 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.300 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714108+0000", 2024-09-21T21:42:31.300 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8810000000000004e-06 2024-09-21T21:42:31.300 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.301 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.301 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:31.301 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792509+0000", 2024-09-21T21:42:31.301 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078400739999999997 2024-09-21T21:42:31.301 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.301 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.301 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:31.301 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792613+0000", 2024-09-21T21:42:31.301 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000103939 2024-09-21T21:42:31.301 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.301 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.301 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:31.301 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792727+0000", 2024-09-21T21:42:31.301 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011348799999999999 2024-09-21T21:42:31.302 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.302 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.302 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.302 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.302 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.302 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11326 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 6103040~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:31.302 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.714115+0000", 2024-09-21T21:42:31.302 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 40.571572596999999, 2024-09-21T21:42:31.302 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078963056000000004, 2024-09-21T21:42:31.302 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.302 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.302 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:31.302 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.303 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:31.303 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:31.303 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11326 2024-09-21T21:42:31.303 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.303 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.303 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.303 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.303 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714115+0000", 2024-09-21T21:42:31.303 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.303 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.303 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.303 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.303 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714115+0000", 2024-09-21T21:42:31.303 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.304 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.304 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.304 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.304 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.715940+0000", 2024-09-21T21:42:31.304 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0018254969999999999 2024-09-21T21:42:31.304 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.304 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.304 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.304 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.715945+0000", 2024-09-21T21:42:31.304 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6120000000000001e-06 2024-09-21T21:42:31.304 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.304 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.304 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.305 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.715954+0000", 2024-09-21T21:42:31.305 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.5899999999999997e-06 2024-09-21T21:42:31.305 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.305 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.305 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.305 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.715959+0000", 2024-09-21T21:42:31.305 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0849999999999996e-06 2024-09-21T21:42:31.305 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.305 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.305 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:31.305 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792771+0000", 2024-09-21T21:42:31.305 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.076811829999999998 2024-09-21T21:42:31.305 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.306 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.306 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:31.306 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792882+0000", 2024-09-21T21:42:31.306 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000110857 2024-09-21T21:42:31.306 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.306 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.306 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:31.306 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.793078+0000", 2024-09-21T21:42:31.306 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00019558499999999999 2024-09-21T21:42:31.306 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.306 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.306 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.306 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.306 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.306 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11332 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 7565312~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:31.307 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.716063+0000", 2024-09-21T21:42:31.307 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 40.569624062999999, 2024-09-21T21:42:31.307 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078787482000000006, 2024-09-21T21:42:31.307 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.307 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.307 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:31.307 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.307 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:31.307 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:31.307 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11332 2024-09-21T21:42:31.307 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.307 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.307 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.308 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.308 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.716063+0000", 2024-09-21T21:42:31.308 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.308 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.308 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.308 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.308 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.716063+0000", 2024-09-21T21:42:31.308 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.308 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.308 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.308 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.308 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.717944+0000", 2024-09-21T21:42:31.308 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0018812309999999999 2024-09-21T21:42:31.308 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.309 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.309 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.309 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.717949+0000", 2024-09-21T21:42:31.309 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9389999999999996e-06 2024-09-21T21:42:31.309 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.309 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.309 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.309 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.717963+0000", 2024-09-21T21:42:31.309 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.3899e-05 2024-09-21T21:42:31.309 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.309 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.309 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.309 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.717970+0000", 2024-09-21T21:42:31.309 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.426e-06 2024-09-21T21:42:31.310 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.310 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.310 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:31.310 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.794573+0000", 2024-09-21T21:42:31.310 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.076602990999999995 2024-09-21T21:42:31.310 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.310 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.310 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:31.310 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.794682+0000", 2024-09-21T21:42:31.310 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000109541 2024-09-21T21:42:31.310 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.310 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.310 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:31.310 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.794851+0000", 2024-09-21T21:42:31.310 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000168455 2024-09-21T21:42:31.311 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.311 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.311 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.311 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.311 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.311 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11339 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 6266880~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:31.311 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.718117+0000", 2024-09-21T21:42:31.311 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 40.567570037000003, 2024-09-21T21:42:31.311 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078703717000000006, 2024-09-21T21:42:31.311 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.311 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.311 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:31.311 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.311 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:31.312 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:31.312 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11339 2024-09-21T21:42:31.312 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.312 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.312 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.312 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.312 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.718117+0000", 2024-09-21T21:42:31.312 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.312 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.312 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.312 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.312 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.718117+0000", 2024-09-21T21:42:31.312 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.312 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.312 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.312 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.313 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720019+0000", 2024-09-21T21:42:31.313 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0019012549999999999 2024-09-21T21:42:31.313 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.313 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.313 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.313 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720024+0000", 2024-09-21T21:42:31.313 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1100000000000002e-06 2024-09-21T21:42:31.313 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.313 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.313 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.313 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720035+0000", 2024-09-21T21:42:31.313 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.094e-05 2024-09-21T21:42:31.313 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.313 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.313 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.314 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720040+0000", 2024-09-21T21:42:31.314 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1830000000000002e-06 2024-09-21T21:42:31.314 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.314 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.314 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:31.314 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796618+0000", 2024-09-21T21:42:31.314 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.076578033000000004 2024-09-21T21:42:31.314 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.314 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.314 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:31.314 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796696+0000", 2024-09-21T21:42:31.314 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7836000000000005e-05 2024-09-21T21:42:31.314 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.314 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.314 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:31.315 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796821+0000", 2024-09-21T21:42:31.315 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00012536000000000001 2024-09-21T21:42:31.315 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.315 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.315 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.315 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.315 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.315 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11345 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 5472256~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:31.315 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.720116+0000", 2024-09-21T21:42:31.315 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 40.565571136999999, 2024-09-21T21:42:31.315 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078667730000000005, 2024-09-21T21:42:31.315 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.315 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.315 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:31.315 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.315 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:31.316 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:31.316 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11345 2024-09-21T21:42:31.316 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.316 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.316 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.316 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.316 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720116+0000", 2024-09-21T21:42:31.316 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.316 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.316 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.316 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.316 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720116+0000", 2024-09-21T21:42:31.316 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.316 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.316 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.317 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.317 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720116+0000", 2024-09-21T21:42:31.317 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.04e-07 2024-09-21T21:42:31.317 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.317 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.317 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.317 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720118+0000", 2024-09-21T21:42:31.317 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.181e-06 2024-09-21T21:42:31.317 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.317 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.317 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.317 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720121+0000", 2024-09-21T21:42:31.317 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0199999999999999e-06 2024-09-21T21:42:31.317 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.318 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.318 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.318 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720123+0000", 2024-09-21T21:42:31.318 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.5469999999999999e-06 2024-09-21T21:42:31.318 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.318 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.318 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:31.318 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798626+0000", 2024-09-21T21:42:31.318 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078503137000000001 2024-09-21T21:42:31.318 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.318 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.318 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:31.318 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798677+0000", 2024-09-21T21:42:31.319 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0930999999999997e-05 2024-09-21T21:42:31.319 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.319 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.319 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:31.319 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798784+0000", 2024-09-21T21:42:31.319 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010671 2024-09-21T21:42:31.319 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.319 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.319 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.319 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.319 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.319 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11346 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 5500928~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:31.319 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.720126+0000", 2024-09-21T21:42:31.319 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 40.565561064000001, 2024-09-21T21:42:31.319 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078863421000000003, 2024-09-21T21:42:31.319 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.320 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.320 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:31.320 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.320 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:31.320 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:31.320 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11346 2024-09-21T21:42:31.320 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.320 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.320 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.320 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.320 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720126+0000", 2024-09-21T21:42:31.320 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.320 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.320 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.320 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.321 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720126+0000", 2024-09-21T21:42:31.321 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.321 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.321 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.321 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.321 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.722145+0000", 2024-09-21T21:42:31.321 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0020183219999999999 2024-09-21T21:42:31.321 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.321 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.321 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.321 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.722150+0000", 2024-09-21T21:42:31.321 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1760000000000001e-06 2024-09-21T21:42:31.321 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.321 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.322 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.322 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.722161+0000", 2024-09-21T21:42:31.322 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1566000000000001e-05 2024-09-21T21:42:31.322 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.322 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.322 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.322 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.722169+0000", 2024-09-21T21:42:31.322 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.2840000000000001e-06 2024-09-21T21:42:31.322 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.322 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.322 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:31.322 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798815+0000", 2024-09-21T21:42:31.322 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.076646607000000005 2024-09-21T21:42:31.322 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.323 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.323 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:31.323 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798870+0000", 2024-09-21T21:42:31.323 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4486e-05 2024-09-21T21:42:31.323 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.323 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.323 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:31.323 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798990+0000", 2024-09-21T21:42:31.323 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011998 2024-09-21T21:42:31.323 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.323 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.323 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.323 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.323 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.323 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13629 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7852032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:31.324 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943319+0000", 2024-09-21T21:42:31.324 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 40.342368422, 2024-09-21T21:42:31.324 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078610975999999999, 2024-09-21T21:42:31.324 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.324 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.324 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:31.324 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.324 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:31.324 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:31.324 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13629 2024-09-21T21:42:31.324 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.324 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.324 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.324 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.325 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943319+0000", 2024-09-21T21:42:31.325 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.325 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.325 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.325 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.325 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943319+0000", 2024-09-21T21:42:31.325 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.325 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.325 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.325 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.325 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943319+0000", 2024-09-21T21:42:31.325 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6199999999999999e-07 2024-09-21T21:42:31.325 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.325 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.325 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.326 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943321+0000", 2024-09-21T21:42:31.326 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1009999999999999e-06 2024-09-21T21:42:31.326 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.326 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.326 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.326 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943328+0000", 2024-09-21T21:42:31.326 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.1029999999999996e-06 2024-09-21T21:42:31.326 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.326 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.326 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.326 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943333+0000", 2024-09-21T21:42:31.326 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7709999999999997e-06 2024-09-21T21:42:31.326 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.326 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.326 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:31.327 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.021679+0000", 2024-09-21T21:42:31.327 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078345673000000005 2024-09-21T21:42:31.327 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.327 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.327 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:31.327 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.021771+0000", 2024-09-21T21:42:31.327 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2201999999999996e-05 2024-09-21T21:42:31.327 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.327 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.327 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:31.327 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.021930+0000", 2024-09-21T21:42:31.327 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00015876400000000001 2024-09-21T21:42:31.327 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.327 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.328 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.328 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.328 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.328 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13630 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7880704~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:31.328 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943339+0000", 2024-09-21T21:42:31.328 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 40.342348233999999, 2024-09-21T21:42:31.328 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078939410000000002, 2024-09-21T21:42:31.328 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.328 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.328 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:31.328 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.328 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:31.328 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:31.328 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13630 2024-09-21T21:42:31.328 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.329 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.329 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.329 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.329 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943339+0000", 2024-09-21T21:42:31.329 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.329 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.329 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.329 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.329 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943339+0000", 2024-09-21T21:42:31.329 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.329 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.329 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.329 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.329 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943339+0000", 2024-09-21T21:42:31.330 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8000000000000001e-07 2024-09-21T21:42:31.330 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.330 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.330 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.330 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943341+0000", 2024-09-21T21:42:31.330 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9850000000000001e-06 2024-09-21T21:42:31.330 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.330 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.330 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.330 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943347+0000", 2024-09-21T21:42:31.330 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2979999999999999e-06 2024-09-21T21:42:31.330 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.330 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.330 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.331 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943354+0000", 2024-09-21T21:42:31.331 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.2969999999999998e-06 2024-09-21T21:42:31.331 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.331 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.331 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:31.331 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.021996+0000", 2024-09-21T21:42:31.331 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078641680000000005 2024-09-21T21:42:31.331 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.331 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.331 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:31.331 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022104+0000", 2024-09-21T21:42:31.331 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000107902 2024-09-21T21:42:31.331 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.331 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.331 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:31.332 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022278+0000", 2024-09-21T21:42:31.332 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00017486800000000001 2024-09-21T21:42:31.332 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.332 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.332 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.332 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.332 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.332 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13631 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7909376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:31.332 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943362+0000", 2024-09-21T21:42:31.332 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 40.342325414000001, 2024-09-21T21:42:31.332 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079195644999999995, 2024-09-21T21:42:31.332 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.332 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.332 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:31.333 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.333 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:31.333 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:31.333 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13631 2024-09-21T21:42:31.333 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.333 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.333 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.333 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.333 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943362+0000", 2024-09-21T21:42:31.333 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.333 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.333 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.333 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.333 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943362+0000", 2024-09-21T21:42:31.334 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.334 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.334 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.334 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.334 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943362+0000", 2024-09-21T21:42:31.334 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8799999999999998e-07 2024-09-21T21:42:31.334 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.334 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.334 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.334 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943364+0000", 2024-09-21T21:42:31.334 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9850000000000001e-06 2024-09-21T21:42:31.334 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.334 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.334 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.335 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943371+0000", 2024-09-21T21:42:31.335 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0650000000000001e-06 2024-09-21T21:42:31.335 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.335 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.335 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.335 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943376+0000", 2024-09-21T21:42:31.335 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8250000000000004e-06 2024-09-21T21:42:31.335 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.335 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.335 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:31.335 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022329+0000", 2024-09-21T21:42:31.335 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078952888999999998 2024-09-21T21:42:31.335 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.335 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.335 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:31.336 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022419+0000", 2024-09-21T21:42:31.336 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.9738e-05 2024-09-21T21:42:31.336 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.336 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.336 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:31.336 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022558+0000", 2024-09-21T21:42:31.336 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00013875500000000001 2024-09-21T21:42:31.336 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.336 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.336 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.336 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.336 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.336 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13632 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7966720~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:31.336 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943382+0000", 2024-09-21T21:42:31.337 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 40.342305179999997, 2024-09-21T21:42:31.337 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079495727000000002, 2024-09-21T21:42:31.337 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.337 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.337 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:31.337 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.337 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:31.337 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:31.337 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13632 2024-09-21T21:42:31.337 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.337 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.337 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.337 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.337 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943382+0000", 2024-09-21T21:42:31.337 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.338 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.338 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.338 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.338 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943382+0000", 2024-09-21T21:42:31.338 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.338 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.338 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.338 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.338 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943382+0000", 2024-09-21T21:42:31.338 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7e-07 2024-09-21T21:42:31.338 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.338 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.338 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.338 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943385+0000", 2024-09-21T21:42:31.338 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2230000000000001e-06 2024-09-21T21:42:31.339 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.339 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.339 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.339 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943391+0000", 2024-09-21T21:42:31.339 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6810000000000001e-06 2024-09-21T21:42:31.339 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.339 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.339 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.339 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943397+0000", 2024-09-21T21:42:31.339 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6559999999999997e-06 2024-09-21T21:42:31.339 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.340 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.340 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:31.340 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022601+0000", 2024-09-21T21:42:31.340 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079204051999999997 2024-09-21T21:42:31.340 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.340 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.340 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:31.340 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022704+0000", 2024-09-21T21:42:31.340 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010329599999999999 2024-09-21T21:42:31.340 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.340 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.340 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:31.340 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022878+0000", 2024-09-21T21:42:31.340 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000173449 2024-09-21T21:42:31.341 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.341 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.341 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.341 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.341 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.341 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13633 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7938048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:31.341 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943403+0000", 2024-09-21T21:42:31.341 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 40.342284249999999, 2024-09-21T21:42:31.341 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079794697999999997, 2024-09-21T21:42:31.341 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.341 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.341 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:31.341 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.341 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:31.342 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:31.342 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13633 2024-09-21T21:42:31.342 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.342 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.342 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.342 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.342 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943403+0000", 2024-09-21T21:42:31.342 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.342 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.342 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.342 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.342 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943403+0000", 2024-09-21T21:42:31.342 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.342 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.343 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.343 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.343 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943403+0000", 2024-09-21T21:42:31.343 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.65e-07 2024-09-21T21:42:31.343 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.343 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.343 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.343 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943405+0000", 2024-09-21T21:42:31.343 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0020000000000001e-06 2024-09-21T21:42:31.343 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.343 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.343 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.343 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943412+0000", 2024-09-21T21:42:31.343 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.8379999999999996e-06 2024-09-21T21:42:31.343 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.344 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.344 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.344 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943417+0000", 2024-09-21T21:42:31.344 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7650000000000001e-06 2024-09-21T21:42:31.344 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.344 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.344 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:31.344 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022922+0000", 2024-09-21T21:42:31.344 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079504512999999999 2024-09-21T21:42:31.344 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.344 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.344 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:31.344 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023039+0000", 2024-09-21T21:42:31.344 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000117044 2024-09-21T21:42:31.345 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.345 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.345 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:31.345 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023198+0000", 2024-09-21T21:42:31.345 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000159171 2024-09-21T21:42:31.345 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.345 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.345 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.345 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.345 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.345 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13634 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7995392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:31.345 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943423+0000", 2024-09-21T21:42:31.345 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 40.342264542000002, 2024-09-21T21:42:31.345 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.080142358999999996, 2024-09-21T21:42:31.345 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.346 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.346 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:31.346 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.346 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:31.346 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:31.346 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13634 2024-09-21T21:42:31.346 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.346 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.346 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.346 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.346 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943423+0000", 2024-09-21T21:42:31.346 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.346 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.346 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.350 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.350 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943423+0000", 2024-09-21T21:42:31.350 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.350 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.350 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.350 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.350 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943423+0000", 2024-09-21T21:42:31.350 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.58e-07 2024-09-21T21:42:31.350 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.350 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.351 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.351 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943425+0000", 2024-09-21T21:42:31.351 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9180000000000001e-06 2024-09-21T21:42:31.351 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.351 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.351 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.351 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943431+0000", 2024-09-21T21:42:31.351 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4439999999999996e-06 2024-09-21T21:42:31.351 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.351 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.351 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.351 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943436+0000", 2024-09-21T21:42:31.351 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.887e-06 2024-09-21T21:42:31.351 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.351 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.352 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:31.352 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023266+0000", 2024-09-21T21:42:31.352 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079829622000000003 2024-09-21T21:42:31.352 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.352 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.352 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:31.352 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023384+0000", 2024-09-21T21:42:31.352 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000117658 2024-09-21T21:42:31.352 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.352 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.352 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:31.352 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023565+0000", 2024-09-21T21:42:31.352 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00018147200000000001 2024-09-21T21:42:31.352 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.352 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.352 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.353 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.353 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.353 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13635 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 8024064~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:31.353 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943447+0000", 2024-09-21T21:42:31.353 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 40.342240001999997, 2024-09-21T21:42:31.353 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.080417653000000006, 2024-09-21T21:42:31.353 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.353 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.353 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:31.353 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.353 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:31.353 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:31.353 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13635 2024-09-21T21:42:31.353 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.354 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.354 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.354 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.354 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943447+0000", 2024-09-21T21:42:31.354 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.354 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.354 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.354 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.354 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943447+0000", 2024-09-21T21:42:31.354 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.354 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.354 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.354 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.354 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943448+0000", 2024-09-21T21:42:31.355 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1699999999999999e-07 2024-09-21T21:42:31.355 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.355 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.355 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.355 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943450+0000", 2024-09-21T21:42:31.355 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1639999999999999e-06 2024-09-21T21:42:31.355 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.355 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.355 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.355 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943456+0000", 2024-09-21T21:42:31.355 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8379999999999998e-06 2024-09-21T21:42:31.355 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.355 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.355 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.356 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943460+0000", 2024-09-21T21:42:31.356 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4700000000000004e-06 2024-09-21T21:42:31.356 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.356 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.356 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:31.356 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023595+0000", 2024-09-21T21:42:31.356 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.080134869999999997 2024-09-21T21:42:31.356 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.356 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.356 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:31.356 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023688+0000", 2024-09-21T21:42:31.356 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2794000000000006e-05 2024-09-21T21:42:31.356 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.357 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.357 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:31.357 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023865+0000", 2024-09-21T21:42:31.357 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0001771 2024-09-21T21:42:31.357 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.357 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.357 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.357 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.357 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.357 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13636 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 8052736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:31.357 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943467+0000", 2024-09-21T21:42:31.357 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 40.342220265000002, 2024-09-21T21:42:31.357 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.080694399999999999, 2024-09-21T21:42:31.357 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.358 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.358 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:31.358 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.358 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:31.358 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:31.358 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13636 2024-09-21T21:42:31.358 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.358 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.358 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.358 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.358 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943467+0000", 2024-09-21T21:42:31.358 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.358 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.358 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.358 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.359 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943467+0000", 2024-09-21T21:42:31.359 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.359 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.359 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.359 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.359 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944940+0000", 2024-09-21T21:42:31.359 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0014725109999999999 2024-09-21T21:42:31.359 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.359 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.359 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.359 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944949+0000", 2024-09-21T21:42:31.359 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2930000000000006e-06 2024-09-21T21:42:31.359 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.359 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.359 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.360 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944958+0000", 2024-09-21T21:42:31.360 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.6899999999999998e-06 2024-09-21T21:42:31.360 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.360 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.360 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.360 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944966+0000", 2024-09-21T21:42:31.360 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0690000000000002e-06 2024-09-21T21:42:31.360 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.360 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.360 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:31.360 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023900+0000", 2024-09-21T21:42:31.360 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078934371000000003 2024-09-21T21:42:31.360 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.360 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.361 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:31.361 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024009+0000", 2024-09-21T21:42:31.361 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000108857 2024-09-21T21:42:31.361 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.361 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.361 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:31.361 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024161+0000", 2024-09-21T21:42:31.361 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000152609 2024-09-21T21:42:31.361 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.361 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.361 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.361 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.361 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.361 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13637 3.12 3.1dc31b72 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:31.361 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.944978+0000", 2024-09-21T21:42:31.362 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 40.340709330000003, 2024-09-21T21:42:31.362 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079232604999999998, 2024-09-21T21:42:31.362 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.362 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.362 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:42:31.362 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.362 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:31.362 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:31.362 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13637 2024-09-21T21:42:31.362 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.362 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.362 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.362 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.362 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944978+0000", 2024-09-21T21:42:31.363 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.363 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.363 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.363 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.363 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944978+0000", 2024-09-21T21:42:31.363 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.363 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.363 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.363 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.363 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944978+0000", 2024-09-21T21:42:31.363 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9100000000000004e-07 2024-09-21T21:42:31.363 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.363 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.363 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.363 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944981+0000", 2024-09-21T21:42:31.364 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3290000000000001e-06 2024-09-21T21:42:31.364 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.364 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.364 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.364 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944987+0000", 2024-09-21T21:42:31.364 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.28e-06 2024-09-21T21:42:31.364 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.364 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.364 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.364 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944992+0000", 2024-09-21T21:42:31.364 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8520000000000003e-06 2024-09-21T21:42:31.364 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.364 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.364 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:31.365 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024196+0000", 2024-09-21T21:42:31.365 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079203673000000002 2024-09-21T21:42:31.365 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.365 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.365 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:31.365 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024211+0000", 2024-09-21T21:42:31.365 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.4980000000000001e-05 2024-09-21T21:42:31.365 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.365 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.365 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.365 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.365 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.365 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13638 3.12 3.1dc31b72 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:31.365 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.945000+0000", 2024-09-21T21:42:31.366 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 40.340687201999998, 2024-09-21T21:42:31.366 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079257465999999999, 2024-09-21T21:42:31.366 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.366 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.366 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:42:31.366 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.366 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:31.366 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:31.366 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13638 2024-09-21T21:42:31.366 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.366 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.366 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.366 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.366 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945000+0000", 2024-09-21T21:42:31.367 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.367 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.367 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.367 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.367 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945000+0000", 2024-09-21T21:42:31.367 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.367 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.367 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.367 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.367 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945001+0000", 2024-09-21T21:42:31.367 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.32e-07 2024-09-21T21:42:31.367 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.367 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.367 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.367 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945003+0000", 2024-09-21T21:42:31.368 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2309999999999999e-06 2024-09-21T21:42:31.368 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.368 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.368 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.368 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945009+0000", 2024-09-21T21:42:31.368 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7450000000000001e-06 2024-09-21T21:42:31.368 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.368 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.368 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.368 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945013+0000", 2024-09-21T21:42:31.368 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.673e-06 2024-09-21T21:42:31.368 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.368 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.368 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:31.369 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024245+0000", 2024-09-21T21:42:31.369 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079232160999999995 2024-09-21T21:42:31.369 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.369 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.369 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:31.369 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024258+0000", 2024-09-21T21:42:31.369 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.2224e-05 2024-09-21T21:42:31.369 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.369 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.369 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.369 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.369 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.369 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13639 3.12 3.1dc31b72 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:31.369 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.945034+0000", 2024-09-21T21:42:31.369 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 40.340653402999997, 2024-09-21T21:42:31.370 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079260627, 2024-09-21T21:42:31.370 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.370 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.370 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:42:31.370 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.370 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:31.370 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:31.370 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13639 2024-09-21T21:42:31.370 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.370 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.370 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.370 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.370 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945034+0000", 2024-09-21T21:42:31.370 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.371 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.371 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.371 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.371 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945034+0000", 2024-09-21T21:42:31.371 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.371 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.371 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.371 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.371 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945034+0000", 2024-09-21T21:42:31.371 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3799999999999998e-07 2024-09-21T21:42:31.371 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.371 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.372 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.372 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945036+0000", 2024-09-21T21:42:31.372 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.052e-06 2024-09-21T21:42:31.372 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.372 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.372 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.372 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945040+0000", 2024-09-21T21:42:31.372 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1169999999999997e-06 2024-09-21T21:42:31.372 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.372 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.372 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.372 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945043+0000", 2024-09-21T21:42:31.372 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.8439999999999998e-06 2024-09-21T21:42:31.372 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.372 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.373 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:31.373 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024286+0000", 2024-09-21T21:42:31.373 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079242483000000002 2024-09-21T21:42:31.373 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.373 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.373 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:31.373 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024295+0000", 2024-09-21T21:42:31.373 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.7930000000000003e-06 2024-09-21T21:42:31.373 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.373 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.373 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.373 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.373 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.373 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13640 3.12 3.1dc31b72 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:31.373 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.945047+0000", 2024-09-21T21:42:31.374 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 40.340639881000001, 2024-09-21T21:42:31.374 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079412321999999994, 2024-09-21T21:42:31.374 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.374 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.374 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:42:31.374 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.374 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:31.374 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:31.374 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13640 2024-09-21T21:42:31.374 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.374 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.374 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.374 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.374 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945047+0000", 2024-09-21T21:42:31.374 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.375 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.375 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.375 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.375 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945047+0000", 2024-09-21T21:42:31.375 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.375 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.375 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.375 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.375 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945048+0000", 2024-09-21T21:42:31.375 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0499999999999999e-07 2024-09-21T21:42:31.375 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.375 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.375 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.375 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945049+0000", 2024-09-21T21:42:31.375 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.3939999999999999e-06 2024-09-21T21:42:31.376 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.376 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.376 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.376 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945053+0000", 2024-09-21T21:42:31.376 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1760000000000003e-06 2024-09-21T21:42:31.376 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.376 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.376 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.376 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945056+0000", 2024-09-21T21:42:31.376 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.6129999999999998e-06 2024-09-21T21:42:31.376 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.376 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.376 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:31.376 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024360+0000", 2024-09-21T21:42:31.377 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079303645000000006 2024-09-21T21:42:31.377 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.377 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.377 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:31.377 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024460+0000", 2024-09-21T21:42:31.377 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000100189 2024-09-21T21:42:31.377 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.377 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.377 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.377 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.377 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.377 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:31.378 INFO:teuthology.orchestra.run.smithi164.stdout:checking clone #41, image image_client.0-clone41 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent42 2024-09-21T21:42:31.379 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T21:42:31.379 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:31.377+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:31.392 INFO:teuthology.orchestra.run.smithi164.stdout:5834 write 0x76ab200 thru 0x76ae2bb (0x30bc bytes) 2024-09-21T21:42:31.393 INFO:teuthology.orchestra.run.smithi164.stdout:5835 read 0x974ea00 thru 0x9755e23 (0x7424 bytes) 2024-09-21T21:42:31.393 INFO:teuthology.orchestra.run.smithi164.stdout:5836 write 0xdb29000 thru 0xdb33bec (0xabed bytes) 2024-09-21T21:42:31.413 INFO:teuthology.orchestra.run.smithi164.stdout:5839 trunc from 0xe37c600 to 0x1a61400 2024-09-21T21:42:31.479 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:31.478+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:31.573 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:31.574 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:42:31.574 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.574 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:3844 3.18 3.1f231578 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.574 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.562848+0000", 2024-09-21T21:42:31.574 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.0027781210000000001, 2024-09-21T21:42:31.574 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0028031750000000002, 2024-09-21T21:42:31.574 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.574 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.574 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.574 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.574 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.575 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.575 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3844 2024-09-21T21:42:31.575 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.575 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.575 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.575 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.575 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.562848+0000", 2024-09-21T21:42:31.575 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.575 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.575 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.575 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.575 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.562848+0000", 2024-09-21T21:42:31.575 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.575 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.575 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.576 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.576 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.562848+0000", 2024-09-21T21:42:31.576 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6399999999999998e-07 2024-09-21T21:42:31.576 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.576 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.576 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.576 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.562850+0000", 2024-09-21T21:42:31.576 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9700000000000002e-06 2024-09-21T21:42:31.576 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.576 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.576 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.576 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.562857+0000", 2024-09-21T21:42:31.576 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.516e-06 2024-09-21T21:42:31.576 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.576 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.577 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.577 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.562862+0000", 2024-09-21T21:42:31.577 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7249999999999997e-06 2024-09-21T21:42:31.577 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.577 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.577 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.577 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.577 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.577 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:4109 3.19 3.d0490659 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.577 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.564042+0000", 2024-09-21T21:42:31.577 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.001583898, 2024-09-21T21:42:31.577 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.001653404, 2024-09-21T21:42:31.577 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.577 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.578 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.578 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.578 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.578 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.578 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4109 2024-09-21T21:42:31.578 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.578 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.578 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.578 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.578 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564042+0000", 2024-09-21T21:42:31.579 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.579 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.579 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.579 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.579 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564042+0000", 2024-09-21T21:42:31.579 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.579 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.579 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.579 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.579 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564043+0000", 2024-09-21T21:42:31.579 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.34e-07 2024-09-21T21:42:31.579 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.579 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.579 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.580 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564044+0000", 2024-09-21T21:42:31.580 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.869e-06 2024-09-21T21:42:31.580 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.580 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.580 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.580 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564051+0000", 2024-09-21T21:42:31.580 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.652e-06 2024-09-21T21:42:31.580 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.580 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.580 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.580 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564056+0000", 2024-09-21T21:42:31.580 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7140000000000001e-06 2024-09-21T21:42:31.580 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.580 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.581 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.581 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.581 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.581 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:4195 3.18 3.1f231578 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.581 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.564541+0000", 2024-09-21T21:42:31.581 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.0010849550000000001, 2024-09-21T21:42:31.581 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0011835700000000001, 2024-09-21T21:42:31.581 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.581 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.581 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.581 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.581 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.581 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.581 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4195 2024-09-21T21:42:31.581 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.582 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.582 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.582 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.582 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564541+0000", 2024-09-21T21:42:31.582 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.582 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.582 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.582 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.582 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564541+0000", 2024-09-21T21:42:31.582 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.582 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.582 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.582 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.583 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564541+0000", 2024-09-21T21:42:31.583 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0900000000000001e-07 2024-09-21T21:42:31.583 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.583 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.583 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.583 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564542+0000", 2024-09-21T21:42:31.583 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0750000000000001e-06 2024-09-21T21:42:31.583 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.583 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.583 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.583 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564545+0000", 2024-09-21T21:42:31.583 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.756e-06 2024-09-21T21:42:31.583 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.583 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.583 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.584 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564548+0000", 2024-09-21T21:42:31.584 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.5349999999999999e-06 2024-09-21T21:42:31.584 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.584 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.584 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.584 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.584 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.584 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:4297 3.1a 3.51c7a5a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.584 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.565128+0000", 2024-09-21T21:42:31.584 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.00049824500000000002, 2024-09-21T21:42:31.584 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00063681899999999995, 2024-09-21T21:42:31.584 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.584 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.584 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.585 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.585 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.585 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.585 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4297 2024-09-21T21:42:31.585 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.585 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.585 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.585 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.585 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565128+0000", 2024-09-21T21:42:31.585 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.585 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.585 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.585 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.585 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565128+0000", 2024-09-21T21:42:31.585 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.586 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.586 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.586 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.586 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565128+0000", 2024-09-21T21:42:31.586 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.1899999999999998e-07 2024-09-21T21:42:31.586 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.586 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.586 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.586 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565130+0000", 2024-09-21T21:42:31.586 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.8330000000000001e-06 2024-09-21T21:42:31.586 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.586 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.586 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.586 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565136+0000", 2024-09-21T21:42:31.586 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8390000000000003e-06 2024-09-21T21:42:31.587 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.587 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.587 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.587 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565140+0000", 2024-09-21T21:42:31.587 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6e-06 2024-09-21T21:42:31.587 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.587 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.587 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.587 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.587 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.587 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:3845 3.18 3.1f231578 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.587 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.562868+0000", 2024-09-21T21:42:31.587 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.0027582510000000002, 2024-09-21T21:42:31.587 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.002932507, 2024-09-21T21:42:31.587 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.588 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.588 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.588 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.588 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.588 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.588 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3845 2024-09-21T21:42:31.588 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.588 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.588 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.588 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.588 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.562868+0000", 2024-09-21T21:42:31.589 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.589 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.589 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.589 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.589 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.562868+0000", 2024-09-21T21:42:31.589 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.589 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.589 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.589 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.589 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.562868+0000", 2024-09-21T21:42:31.589 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2799999999999996e-07 2024-09-21T21:42:31.589 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.589 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.589 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.590 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.562870+0000", 2024-09-21T21:42:31.590 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9460000000000001e-06 2024-09-21T21:42:31.590 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.590 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.590 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.590 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.562876+0000", 2024-09-21T21:42:31.590 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5319999999999997e-06 2024-09-21T21:42:31.590 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.590 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.590 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.590 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.562880+0000", 2024-09-21T21:42:31.590 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.51e-06 2024-09-21T21:42:31.590 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.590 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.591 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.591 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.591 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.591 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:4113 3.18 3.1f231578 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.591 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.564061+0000", 2024-09-21T21:42:31.591 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.0015649940000000001, 2024-09-21T21:42:31.591 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0017693679999999999, 2024-09-21T21:42:31.591 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.591 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.591 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.591 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.591 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.591 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.591 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4113 2024-09-21T21:42:31.591 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.592 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.592 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.592 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.592 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564061+0000", 2024-09-21T21:42:31.592 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.592 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.592 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.592 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.592 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564061+0000", 2024-09-21T21:42:31.592 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.592 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.592 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.592 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.593 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564062+0000", 2024-09-21T21:42:31.593 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4600000000000005e-07 2024-09-21T21:42:31.593 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.593 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.593 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.593 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564064+0000", 2024-09-21T21:42:31.593 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9360000000000002e-06 2024-09-21T21:42:31.593 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.593 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.593 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.593 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564070+0000", 2024-09-21T21:42:31.593 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9390000000000002e-06 2024-09-21T21:42:31.593 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.593 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.593 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.594 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564074+0000", 2024-09-21T21:42:31.594 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6469999999999997e-06 2024-09-21T21:42:31.594 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.594 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.594 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.594 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.594 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.594 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:4200 3.1a 3.51c7a5a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.594 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.564551+0000", 2024-09-21T21:42:31.594 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.0010756030000000001, 2024-09-21T21:42:31.594 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0013065959999999999, 2024-09-21T21:42:31.594 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.594 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.594 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.595 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.595 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.595 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.595 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4200 2024-09-21T21:42:31.595 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.595 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.595 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.595 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.595 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564551+0000", 2024-09-21T21:42:31.595 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.595 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.595 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.595 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.595 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564551+0000", 2024-09-21T21:42:31.596 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.596 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.596 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.596 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.596 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564551+0000", 2024-09-21T21:42:31.596 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.1699999999999999e-07 2024-09-21T21:42:31.596 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.596 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.596 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.596 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564552+0000", 2024-09-21T21:42:31.596 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.085e-06 2024-09-21T21:42:31.596 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.596 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.596 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.597 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564555+0000", 2024-09-21T21:42:31.597 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.7999999999999999e-06 2024-09-21T21:42:31.597 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.597 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.597 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.597 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564557+0000", 2024-09-21T21:42:31.597 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.5780000000000001e-06 2024-09-21T21:42:31.597 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.597 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.597 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.597 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.597 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.597 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:4298 3.1a 3.51c7a5a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.598 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.565147+0000", 2024-09-21T21:42:31.598 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.00047901299999999998, 2024-09-21T21:42:31.598 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000737549, 2024-09-21T21:42:31.598 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.598 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.598 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.598 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.598 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.598 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.598 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4298 2024-09-21T21:42:31.598 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.598 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.599 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.599 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.599 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565147+0000", 2024-09-21T21:42:31.599 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.599 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.599 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.599 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.599 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565147+0000", 2024-09-21T21:42:31.599 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.599 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.599 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.599 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.599 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565147+0000", 2024-09-21T21:42:31.599 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7899999999999999e-07 2024-09-21T21:42:31.600 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.600 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.600 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.600 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565153+0000", 2024-09-21T21:42:31.600 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3789999999999997e-06 2024-09-21T21:42:31.600 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.600 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.600 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.600 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565159+0000", 2024-09-21T21:42:31.600 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0090000000000004e-06 2024-09-21T21:42:31.600 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.600 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.600 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.600 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565164+0000", 2024-09-21T21:42:31.601 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8339999999999998e-06 2024-09-21T21:42:31.601 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.601 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.601 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.601 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.601 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.601 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:3873 3.18 3.1f231578 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.601 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.562898+0000", 2024-09-21T21:42:31.601 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.0027279650000000002, 2024-09-21T21:42:31.601 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0030138610000000001, 2024-09-21T21:42:31.601 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.601 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.601 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.602 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.602 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.602 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.602 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3873 2024-09-21T21:42:31.602 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.602 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.602 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.602 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.602 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.562898+0000", 2024-09-21T21:42:31.602 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.602 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.602 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.603 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.603 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.562898+0000", 2024-09-21T21:42:31.603 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.603 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.603 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.604 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.604 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.562899+0000", 2024-09-21T21:42:31.604 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5400000000000001e-07 2024-09-21T21:42:31.604 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.604 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.604 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.604 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.562901+0000", 2024-09-21T21:42:31.604 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1260000000000001e-06 2024-09-21T21:42:31.604 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.604 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.604 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.604 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.562906+0000", 2024-09-21T21:42:31.604 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6400000000000002e-06 2024-09-21T21:42:31.604 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.604 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.605 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.605 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.562911+0000", 2024-09-21T21:42:31.605 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6460000000000001e-06 2024-09-21T21:42:31.605 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.605 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.605 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.605 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.605 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.605 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:4025 3.18 3.1f231578 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.605 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.563589+0000", 2024-09-21T21:42:31.605 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.002036787, 2024-09-21T21:42:31.605 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0023564699999999998, 2024-09-21T21:42:31.605 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.605 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.605 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.606 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.606 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.606 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.606 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4025 2024-09-21T21:42:31.606 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.606 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.606 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.606 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.606 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563589+0000", 2024-09-21T21:42:31.606 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.606 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.606 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.606 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.607 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563589+0000", 2024-09-21T21:42:31.607 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.607 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.607 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.607 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.607 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563590+0000", 2024-09-21T21:42:31.607 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1099999999999995e-07 2024-09-21T21:42:31.607 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.607 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.607 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.607 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563592+0000", 2024-09-21T21:42:31.607 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9850000000000001e-06 2024-09-21T21:42:31.607 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.607 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.607 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.608 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563598+0000", 2024-09-21T21:42:31.608 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2949999999999999e-06 2024-09-21T21:42:31.608 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.608 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.608 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.608 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563603+0000", 2024-09-21T21:42:31.608 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5480000000000003e-06 2024-09-21T21:42:31.608 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.608 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.608 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.608 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.608 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.608 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:4114 3.18 3.1f231578 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.608 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.564080+0000", 2024-09-21T21:42:31.609 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.001546387, 2024-09-21T21:42:31.609 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.001892499, 2024-09-21T21:42:31.609 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.609 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.609 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.609 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.609 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.609 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.609 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4114 2024-09-21T21:42:31.609 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.609 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.609 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.609 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.609 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564080+0000", 2024-09-21T21:42:31.610 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.610 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.610 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.610 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.610 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564080+0000", 2024-09-21T21:42:31.610 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.610 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.610 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.610 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.610 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564080+0000", 2024-09-21T21:42:31.610 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4499999999999997e-07 2024-09-21T21:42:31.610 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.610 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.610 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.611 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564082+0000", 2024-09-21T21:42:31.611 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.793e-06 2024-09-21T21:42:31.611 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.611 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.611 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.611 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564088+0000", 2024-09-21T21:42:31.611 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7640000000000002e-06 2024-09-21T21:42:31.611 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.611 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.611 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.611 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564092+0000", 2024-09-21T21:42:31.611 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2390000000000004e-06 2024-09-21T21:42:31.611 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.611 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.611 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.612 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.612 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.612 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:4201 3.1a 3.51c7a5a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.612 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.564582+0000", 2024-09-21T21:42:31.612 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.001044016, 2024-09-21T21:42:31.612 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0014164169999999999, 2024-09-21T21:42:31.612 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.612 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.612 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.612 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.612 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.612 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.612 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4201 2024-09-21T21:42:31.612 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.612 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.613 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.613 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.613 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564582+0000", 2024-09-21T21:42:31.613 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.613 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.613 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.613 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.613 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564582+0000", 2024-09-21T21:42:31.613 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.613 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.613 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.613 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.613 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564583+0000", 2024-09-21T21:42:31.613 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3200000000000005e-07 2024-09-21T21:42:31.613 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.614 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.614 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.614 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564585+0000", 2024-09-21T21:42:31.614 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0980000000000001e-06 2024-09-21T21:42:31.614 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.614 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.614 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.614 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564591+0000", 2024-09-21T21:42:31.614 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9109999999999998e-06 2024-09-21T21:42:31.614 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.614 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.614 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.614 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564595+0000", 2024-09-21T21:42:31.614 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5920000000000002e-06 2024-09-21T21:42:31.615 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.615 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.615 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.615 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.615 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.615 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:4299 3.1a 3.51c7a5a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.615 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.565170+0000", 2024-09-21T21:42:31.615 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.00045613400000000002, 2024-09-21T21:42:31.615 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000855877, 2024-09-21T21:42:31.615 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.615 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.615 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.615 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.615 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.615 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.616 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4299 2024-09-21T21:42:31.616 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.616 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.616 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.616 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.616 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565170+0000", 2024-09-21T21:42:31.616 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.616 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.616 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.616 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.616 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565170+0000", 2024-09-21T21:42:31.616 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.616 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.616 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.617 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.617 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565170+0000", 2024-09-21T21:42:31.617 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8099999999999998e-07 2024-09-21T21:42:31.617 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.617 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.617 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.617 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565172+0000", 2024-09-21T21:42:31.617 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9149999999999999e-06 2024-09-21T21:42:31.617 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.617 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.617 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.617 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565178+0000", 2024-09-21T21:42:31.617 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7660000000000004e-06 2024-09-21T21:42:31.617 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.617 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.618 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.618 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565183+0000", 2024-09-21T21:42:31.618 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8049999999999997e-06 2024-09-21T21:42:31.618 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.618 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.618 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.618 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.618 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.618 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:3875 3.18 3.1f231578 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.618 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.562918+0000", 2024-09-21T21:42:31.618 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.002708542, 2024-09-21T21:42:31.618 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.003135802, 2024-09-21T21:42:31.618 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.618 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.619 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.619 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.619 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.619 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.619 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3875 2024-09-21T21:42:31.619 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.619 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.619 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.619 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.619 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.562918+0000", 2024-09-21T21:42:31.619 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.619 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.619 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.620 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.620 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.562918+0000", 2024-09-21T21:42:31.620 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.620 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.620 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.620 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.620 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.562918+0000", 2024-09-21T21:42:31.620 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5900000000000003e-07 2024-09-21T21:42:31.620 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.620 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.620 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.620 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.562920+0000", 2024-09-21T21:42:31.620 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.827e-06 2024-09-21T21:42:31.620 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.620 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.621 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.621 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.562925+0000", 2024-09-21T21:42:31.621 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6169999999999998e-06 2024-09-21T21:42:31.621 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.621 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.621 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.621 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.562930+0000", 2024-09-21T21:42:31.621 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5000000000000001e-06 2024-09-21T21:42:31.621 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.621 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.621 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.621 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.621 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.621 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:4116 3.18 3.1f231578 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.621 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.564097+0000", 2024-09-21T21:42:31.621 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.0015289290000000001, 2024-09-21T21:42:31.622 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0019902710000000001, 2024-09-21T21:42:31.622 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.622 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.622 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.622 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.622 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.622 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.622 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4116 2024-09-21T21:42:31.622 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.622 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.622 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.622 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.622 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564097+0000", 2024-09-21T21:42:31.622 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.622 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.623 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.623 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.623 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564097+0000", 2024-09-21T21:42:31.623 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.623 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.623 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.623 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.623 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564098+0000", 2024-09-21T21:42:31.623 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5400000000000001e-07 2024-09-21T21:42:31.623 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.623 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.623 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.623 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564100+0000", 2024-09-21T21:42:31.623 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.029e-06 2024-09-21T21:42:31.624 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.624 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.624 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.624 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564106+0000", 2024-09-21T21:42:31.624 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7960000000000001e-06 2024-09-21T21:42:31.624 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.624 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.624 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.624 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564110+0000", 2024-09-21T21:42:31.624 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7199999999999997e-06 2024-09-21T21:42:31.624 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.624 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.624 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.624 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.624 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.624 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:4202 3.1a 3.51c7a5a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.625 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.564601+0000", 2024-09-21T21:42:31.625 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.0010248869999999999, 2024-09-21T21:42:31.625 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0015393939999999999, 2024-09-21T21:42:31.625 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.625 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.625 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.625 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.625 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.625 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.625 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4202 2024-09-21T21:42:31.625 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.625 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.625 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.625 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.625 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564601+0000", 2024-09-21T21:42:31.626 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.626 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.626 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.626 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.626 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564601+0000", 2024-09-21T21:42:31.626 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.626 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.626 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.626 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.626 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564602+0000", 2024-09-21T21:42:31.626 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4200000000000001e-07 2024-09-21T21:42:31.626 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.626 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.626 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.626 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564604+0000", 2024-09-21T21:42:31.627 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.978e-06 2024-09-21T21:42:31.627 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.627 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.627 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.627 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564609+0000", 2024-09-21T21:42:31.627 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4399999999999996e-06 2024-09-21T21:42:31.627 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.627 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.627 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.627 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564613+0000", 2024-09-21T21:42:31.627 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.245e-06 2024-09-21T21:42:31.627 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.627 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.627 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.627 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.627 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.628 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:4305 3.19 3.d0490659 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.628 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.565203+0000", 2024-09-21T21:42:31.628 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.00042303499999999999, 2024-09-21T21:42:31.628 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00099276599999999996, 2024-09-21T21:42:31.628 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.628 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.628 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.628 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.628 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.628 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.628 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4305 2024-09-21T21:42:31.628 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.628 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.628 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.628 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.629 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565203+0000", 2024-09-21T21:42:31.629 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.629 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.629 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.629 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.629 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565203+0000", 2024-09-21T21:42:31.629 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.629 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.629 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.629 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.629 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565204+0000", 2024-09-21T21:42:31.629 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1699999999999998e-07 2024-09-21T21:42:31.629 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.629 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.630 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.630 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565206+0000", 2024-09-21T21:42:31.630 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1900000000000002e-06 2024-09-21T21:42:31.630 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.630 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.630 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.630 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565212+0000", 2024-09-21T21:42:31.630 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1310000000000003e-06 2024-09-21T21:42:31.630 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.630 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.630 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.630 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565217+0000", 2024-09-21T21:42:31.630 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8300000000000003e-06 2024-09-21T21:42:31.630 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.630 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.631 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.631 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.631 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.631 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:3876 3.18 3.1f231578 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.631 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.562959+0000", 2024-09-21T21:42:31.631 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.0026670349999999999, 2024-09-21T21:42:31.631 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0032951, 2024-09-21T21:42:31.631 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.631 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.631 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.631 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.631 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.631 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.631 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3876 2024-09-21T21:42:31.632 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.632 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.632 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.632 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.632 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.562959+0000", 2024-09-21T21:42:31.632 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.632 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.632 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.632 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.632 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.562959+0000", 2024-09-21T21:42:31.632 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.632 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.632 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.632 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.632 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.562959+0000", 2024-09-21T21:42:31.633 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2199999999999999e-07 2024-09-21T21:42:31.633 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.633 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.633 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.633 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.562962+0000", 2024-09-21T21:42:31.633 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.4399999999999999e-06 2024-09-21T21:42:31.633 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.633 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.633 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.633 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.562968+0000", 2024-09-21T21:42:31.633 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.365e-06 2024-09-21T21:42:31.633 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.633 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.634 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.634 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.562973+0000", 2024-09-21T21:42:31.634 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7090000000000001e-06 2024-09-21T21:42:31.634 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.634 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.634 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.634 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.634 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.634 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:4118 3.18 3.1f231578 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.634 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.564119+0000", 2024-09-21T21:42:31.634 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.0015072849999999999, 2024-09-21T21:42:31.634 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.002178213, 2024-09-21T21:42:31.634 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.634 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.634 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.635 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.635 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.635 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.635 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4118 2024-09-21T21:42:31.635 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.635 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.635 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.635 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.635 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564119+0000", 2024-09-21T21:42:31.635 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.635 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.635 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.635 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.635 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564119+0000", 2024-09-21T21:42:31.635 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.636 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.636 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.636 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.636 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564119+0000", 2024-09-21T21:42:31.636 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6199999999999999e-07 2024-09-21T21:42:31.636 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.636 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.636 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.636 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564121+0000", 2024-09-21T21:42:31.636 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9479999999999998e-06 2024-09-21T21:42:31.636 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.636 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.636 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.636 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564127+0000", 2024-09-21T21:42:31.636 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6550000000000001e-06 2024-09-21T21:42:31.637 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.637 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.637 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.637 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564131+0000", 2024-09-21T21:42:31.637 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.656e-06 2024-09-21T21:42:31.637 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.637 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.637 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.637 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.637 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.637 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:4204 3.1a 3.51c7a5a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.637 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.564621+0000", 2024-09-21T21:42:31.637 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.001004696, 2024-09-21T21:42:31.637 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.001703361, 2024-09-21T21:42:31.637 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.638 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.638 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.638 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.638 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.638 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.638 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4204 2024-09-21T21:42:31.638 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.638 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.638 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.638 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.638 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564621+0000", 2024-09-21T21:42:31.638 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.638 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.638 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.638 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.639 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564621+0000", 2024-09-21T21:42:31.639 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.639 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.639 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.639 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.639 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564622+0000", 2024-09-21T21:42:31.639 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4299999999999999e-07 2024-09-21T21:42:31.639 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.639 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.639 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.639 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564624+0000", 2024-09-21T21:42:31.639 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9659999999999999e-06 2024-09-21T21:42:31.639 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.639 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.640 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.640 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564629+0000", 2024-09-21T21:42:31.640 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4480000000000002e-06 2024-09-21T21:42:31.640 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.640 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.640 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.640 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564634+0000", 2024-09-21T21:42:31.640 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5469999999999998e-06 2024-09-21T21:42:31.640 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.640 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.640 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.640 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.640 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.640 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:4307 3.19 3.d0490659 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.640 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.565246+0000", 2024-09-21T21:42:31.641 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.00038022300000000001, 2024-09-21T21:42:31.641 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.001106178, 2024-09-21T21:42:31.641 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.641 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.641 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.641 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.641 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.641 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.641 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4307 2024-09-21T21:42:31.641 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.641 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.641 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.641 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.641 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565246+0000", 2024-09-21T21:42:31.642 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.642 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.642 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.642 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.642 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565246+0000", 2024-09-21T21:42:31.642 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.642 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.642 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.642 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.642 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565246+0000", 2024-09-21T21:42:31.642 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.1699999999999999e-07 2024-09-21T21:42:31.643 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.643 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.643 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.643 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565248+0000", 2024-09-21T21:42:31.643 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.99e-06 2024-09-21T21:42:31.643 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.643 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.643 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.643 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565252+0000", 2024-09-21T21:42:31.643 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.0810000000000004e-06 2024-09-21T21:42:31.643 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.643 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.643 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.643 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565255+0000", 2024-09-21T21:42:31.643 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.8289999999999999e-06 2024-09-21T21:42:31.644 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.644 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.644 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.644 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.644 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.644 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:3878 3.18 3.1f231578 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.644 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.562992+0000", 2024-09-21T21:42:31.644 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.0026344179999999999, 2024-09-21T21:42:31.644 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.003387475, 2024-09-21T21:42:31.644 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.644 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.644 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.645 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.645 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.645 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.645 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3878 2024-09-21T21:42:31.645 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.645 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.645 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.645 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.645 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.562992+0000", 2024-09-21T21:42:31.645 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.645 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.645 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.645 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.645 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.562992+0000", 2024-09-21T21:42:31.646 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.646 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.646 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.646 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.646 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.562992+0000", 2024-09-21T21:42:31.646 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0399999999999996e-07 2024-09-21T21:42:31.646 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.646 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.646 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.646 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.562994+0000", 2024-09-21T21:42:31.646 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.097e-06 2024-09-21T21:42:31.646 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.646 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.646 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.647 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563000+0000", 2024-09-21T21:42:31.647 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0390000000000001e-06 2024-09-21T21:42:31.647 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.647 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.647 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.647 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563005+0000", 2024-09-21T21:42:31.647 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7430000000000002e-06 2024-09-21T21:42:31.647 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.647 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.647 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.647 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.647 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.647 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:4210 3.19 3.d0490659 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.647 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.564639+0000", 2024-09-21T21:42:31.647 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.0009867229999999999, 2024-09-21T21:42:31.648 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.001766463, 2024-09-21T21:42:31.648 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.648 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.648 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.648 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.648 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.648 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.648 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4210 2024-09-21T21:42:31.648 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.648 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.648 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.648 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.648 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564639+0000", 2024-09-21T21:42:31.648 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.648 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.649 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.649 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.649 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564639+0000", 2024-09-21T21:42:31.649 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.649 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.649 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.649 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.649 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564640+0000", 2024-09-21T21:42:31.649 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7899999999999999e-07 2024-09-21T21:42:31.649 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.649 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.649 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.649 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564642+0000", 2024-09-21T21:42:31.649 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.99e-06 2024-09-21T21:42:31.649 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.650 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.650 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.650 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564647+0000", 2024-09-21T21:42:31.650 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6400000000000002e-06 2024-09-21T21:42:31.650 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.650 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.650 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.650 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564652+0000", 2024-09-21T21:42:31.650 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5759999999999999e-06 2024-09-21T21:42:31.650 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.650 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.650 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.650 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.650 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.651 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:4313 3.18 3.1f231578 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.651 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.565259+0000", 2024-09-21T21:42:31.651 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.00036712399999999998, 2024-09-21T21:42:31.651 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.001172934, 2024-09-21T21:42:31.651 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.651 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.651 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.651 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.651 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.651 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.651 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4313 2024-09-21T21:42:31.651 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.651 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.652 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.652 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.652 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565259+0000", 2024-09-21T21:42:31.652 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.652 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.652 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.652 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.652 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565259+0000", 2024-09-21T21:42:31.652 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.652 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.652 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.652 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.652 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565259+0000", 2024-09-21T21:42:31.652 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.15e-07 2024-09-21T21:42:31.652 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.653 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.653 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.653 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565261+0000", 2024-09-21T21:42:31.653 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.2190000000000001e-06 2024-09-21T21:42:31.653 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.653 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.653 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.653 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565264+0000", 2024-09-21T21:42:31.653 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7340000000000002e-06 2024-09-21T21:42:31.653 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.653 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.653 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.653 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565267+0000", 2024-09-21T21:42:31.653 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.5979999999999999e-06 2024-09-21T21:42:31.654 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.654 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.654 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.654 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.654 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.654 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:3905 3.18 3.1f231578 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.654 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.563012+0000", 2024-09-21T21:42:31.654 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.0026144570000000002, 2024-09-21T21:42:31.654 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0034665130000000001, 2024-09-21T21:42:31.654 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.654 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.654 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.654 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.654 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.654 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.655 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3905 2024-09-21T21:42:31.655 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.655 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.655 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.655 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.655 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563012+0000", 2024-09-21T21:42:31.655 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.655 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.655 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.655 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.655 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563012+0000", 2024-09-21T21:42:31.655 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.655 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.655 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.656 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.656 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563012+0000", 2024-09-21T21:42:31.656 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1800000000000001e-07 2024-09-21T21:42:31.656 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.656 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.656 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.656 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563014+0000", 2024-09-21T21:42:31.656 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0990000000000001e-06 2024-09-21T21:42:31.656 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.656 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.656 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.656 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563020+0000", 2024-09-21T21:42:31.656 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8200000000000002e-06 2024-09-21T21:42:31.656 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.657 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.657 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.657 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563025+0000", 2024-09-21T21:42:31.657 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6210000000000003e-06 2024-09-21T21:42:31.657 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.657 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.657 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.657 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.657 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.657 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:4211 3.19 3.d0490659 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.657 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.564658+0000", 2024-09-21T21:42:31.657 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.00096789800000000002, 2024-09-21T21:42:31.657 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0018761990000000001, 2024-09-21T21:42:31.658 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.658 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.658 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.658 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.658 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.658 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.658 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4211 2024-09-21T21:42:31.658 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.658 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.658 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.658 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.658 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564658+0000", 2024-09-21T21:42:31.658 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.658 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.658 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.659 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.659 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564658+0000", 2024-09-21T21:42:31.659 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.659 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.659 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.659 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.659 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564659+0000", 2024-09-21T21:42:31.659 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6300000000000001e-07 2024-09-21T21:42:31.659 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.659 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.659 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.659 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564661+0000", 2024-09-21T21:42:31.659 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9470000000000002e-06 2024-09-21T21:42:31.660 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.660 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.660 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.660 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564666+0000", 2024-09-21T21:42:31.660 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.643e-06 2024-09-21T21:42:31.660 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.660 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.660 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.660 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564671+0000", 2024-09-21T21:42:31.660 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8099999999999997e-06 2024-09-21T21:42:31.660 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.660 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.660 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.660 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.660 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.661 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:4314 3.1a 3.51c7a5a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.661 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.565270+0000", 2024-09-21T21:42:31.661 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.00035608299999999999, 2024-09-21T21:42:31.661 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0013176119999999999, 2024-09-21T21:42:31.661 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.661 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.661 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.661 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.661 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.661 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.661 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4314 2024-09-21T21:42:31.661 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.661 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.661 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.662 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.662 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565270+0000", 2024-09-21T21:42:31.662 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.662 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.662 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.662 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.662 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565270+0000", 2024-09-21T21:42:31.662 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.662 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.662 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.662 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.662 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565270+0000", 2024-09-21T21:42:31.662 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.36e-07 2024-09-21T21:42:31.662 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.663 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.663 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.663 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565271+0000", 2024-09-21T21:42:31.663 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.102e-06 2024-09-21T21:42:31.663 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.663 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.663 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.663 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565274+0000", 2024-09-21T21:42:31.663 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.027e-06 2024-09-21T21:42:31.663 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.663 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.663 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.663 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565277+0000", 2024-09-21T21:42:31.663 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.4619999999999999e-06 2024-09-21T21:42:31.663 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.663 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.664 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.664 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.664 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.664 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:3907 3.18 3.1f231578 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.664 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.563030+0000", 2024-09-21T21:42:31.664 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.0025962239999999998, 2024-09-21T21:42:31.664 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.003612915, 2024-09-21T21:42:31.664 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.664 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.664 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.664 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.664 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.664 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.664 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3907 2024-09-21T21:42:31.664 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.665 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.665 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.665 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.665 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563030+0000", 2024-09-21T21:42:31.665 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.665 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.665 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.665 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.665 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563030+0000", 2024-09-21T21:42:31.665 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.665 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.665 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.665 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.665 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563030+0000", 2024-09-21T21:42:31.666 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.1899999999999998e-07 2024-09-21T21:42:31.666 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.666 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.666 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.666 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563032+0000", 2024-09-21T21:42:31.666 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.7680000000000001e-06 2024-09-21T21:42:31.666 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.666 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.666 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.666 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563038+0000", 2024-09-21T21:42:31.666 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6019999999999999e-06 2024-09-21T21:42:31.666 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.666 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.666 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.667 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563042+0000", 2024-09-21T21:42:31.667 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3579999999999996e-06 2024-09-21T21:42:31.667 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.667 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.667 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.667 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.667 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.667 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:4039 3.18 3.1f231578 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.667 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.563659+0000", 2024-09-21T21:42:31.667 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.001966658, 2024-09-21T21:42:31.667 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0030167029999999999, 2024-09-21T21:42:31.667 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.667 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.667 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.667 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.668 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.668 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.668 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4039 2024-09-21T21:42:31.668 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.668 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.668 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.668 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.668 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563659+0000", 2024-09-21T21:42:31.668 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.668 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.668 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.668 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.668 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563659+0000", 2024-09-21T21:42:31.668 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.669 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.669 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.669 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.669 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563660+0000", 2024-09-21T21:42:31.669 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.11e-07 2024-09-21T21:42:31.669 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.669 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.669 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.669 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563661+0000", 2024-09-21T21:42:31.669 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1379999999999999e-06 2024-09-21T21:42:31.669 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.669 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.669 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.670 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563664+0000", 2024-09-21T21:42:31.670 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.067e-06 2024-09-21T21:42:31.670 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.670 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.670 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.670 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563666+0000", 2024-09-21T21:42:31.670 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.5739999999999998e-06 2024-09-21T21:42:31.670 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.670 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.670 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.670 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.670 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.670 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:4216 3.18 3.1f231578 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.670 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.564677+0000", 2024-09-21T21:42:31.671 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.00094931499999999995, 2024-09-21T21:42:31.671 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.002050451, 2024-09-21T21:42:31.671 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.671 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.671 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.671 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.671 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.671 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.671 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4216 2024-09-21T21:42:31.671 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.671 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.671 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.671 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.671 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564677+0000", 2024-09-21T21:42:31.671 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.672 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.672 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.672 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.672 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564677+0000", 2024-09-21T21:42:31.672 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.672 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.672 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.672 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.672 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564677+0000", 2024-09-21T21:42:31.672 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4700000000000001e-07 2024-09-21T21:42:31.672 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.672 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.672 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.672 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564679+0000", 2024-09-21T21:42:31.672 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0849999999999999e-06 2024-09-21T21:42:31.673 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.673 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.673 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.673 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564685+0000", 2024-09-21T21:42:31.673 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9349999999999999e-06 2024-09-21T21:42:31.673 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.673 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.673 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.673 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564690+0000", 2024-09-21T21:42:31.673 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9219999999999996e-06 2024-09-21T21:42:31.673 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.673 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.673 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.673 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.674 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.674 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:4316 3.1a 3.51c7a5a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.674 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.565280+0000", 2024-09-21T21:42:31.674 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.00034637999999999997, 2024-09-21T21:42:31.674 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.001500024, 2024-09-21T21:42:31.674 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.674 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.674 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.674 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.674 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.674 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.674 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4316 2024-09-21T21:42:31.674 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.674 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.675 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.675 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.675 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565280+0000", 2024-09-21T21:42:31.675 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.675 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.675 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.675 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.675 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565280+0000", 2024-09-21T21:42:31.675 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.675 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.675 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.675 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.675 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565280+0000", 2024-09-21T21:42:31.675 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.16e-07 2024-09-21T21:42:31.676 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.676 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.676 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.676 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565281+0000", 2024-09-21T21:42:31.676 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.2190000000000001e-06 2024-09-21T21:42:31.676 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.676 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.676 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.676 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565284+0000", 2024-09-21T21:42:31.676 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.892e-06 2024-09-21T21:42:31.676 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.676 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.676 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.676 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565287+0000", 2024-09-21T21:42:31.677 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.4700000000000001e-06 2024-09-21T21:42:31.677 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.677 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.677 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.677 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.677 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.677 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:3909 3.18 3.1f231578 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.677 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.563047+0000", 2024-09-21T21:42:31.677 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.0025790679999999999, 2024-09-21T21:42:31.677 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0037876820000000001, 2024-09-21T21:42:31.677 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.677 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.677 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.677 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.678 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.678 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.678 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3909 2024-09-21T21:42:31.678 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.678 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.678 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.678 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.678 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563047+0000", 2024-09-21T21:42:31.678 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.678 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.678 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.678 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.678 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563047+0000", 2024-09-21T21:42:31.678 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.678 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.679 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.679 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.679 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563047+0000", 2024-09-21T21:42:31.679 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.2800000000000003e-07 2024-09-21T21:42:31.679 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.679 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.679 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.679 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563049+0000", 2024-09-21T21:42:31.679 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.6750000000000001e-06 2024-09-21T21:42:31.680 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.680 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.680 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.680 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563054+0000", 2024-09-21T21:42:31.680 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4369999999999998e-06 2024-09-21T21:42:31.680 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.680 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.680 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.680 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563059+0000", 2024-09-21T21:42:31.680 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4299999999999999e-06 2024-09-21T21:42:31.680 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.680 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.680 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.680 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.681 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.681 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:4040 3.18 3.1f231578 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.681 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.563670+0000", 2024-09-21T21:42:31.681 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.0019560459999999999, 2024-09-21T21:42:31.681 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0032140139999999998, 2024-09-21T21:42:31.681 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.681 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.682 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.683 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.683 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.683 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.683 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4040 2024-09-21T21:42:31.683 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.683 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.683 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.683 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.683 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563670+0000", 2024-09-21T21:42:31.683 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.683 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.683 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.683 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.683 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563670+0000", 2024-09-21T21:42:31.684 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.684 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.684 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.684 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.684 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563670+0000", 2024-09-21T21:42:31.684 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1e-07 2024-09-21T21:42:31.684 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.684 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.684 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.684 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563671+0000", 2024-09-21T21:42:31.684 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1069999999999999e-06 2024-09-21T21:42:31.684 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.684 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.684 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.685 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563674+0000", 2024-09-21T21:42:31.685 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.9299999999999999e-06 2024-09-21T21:42:31.685 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.685 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.685 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.685 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563677+0000", 2024-09-21T21:42:31.685 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.5739999999999998e-06 2024-09-21T21:42:31.685 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.685 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.685 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.685 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.685 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.685 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:4218 3.1a 3.51c7a5a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.685 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.564696+0000", 2024-09-21T21:42:31.685 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.00093028700000000002, 2024-09-21T21:42:31.686 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0022173219999999999, 2024-09-21T21:42:31.686 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.686 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.686 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.686 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.686 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.686 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.686 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4218 2024-09-21T21:42:31.686 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.686 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.686 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.686 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.686 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564696+0000", 2024-09-21T21:42:31.686 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.687 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.687 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.687 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.687 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564696+0000", 2024-09-21T21:42:31.687 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.687 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.687 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.687 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.687 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564696+0000", 2024-09-21T21:42:31.687 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8099999999999998e-07 2024-09-21T21:42:31.687 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.687 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.687 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.687 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564698+0000", 2024-09-21T21:42:31.688 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9470000000000002e-06 2024-09-21T21:42:31.688 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.688 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.688 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.688 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564704+0000", 2024-09-21T21:42:31.688 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.789e-06 2024-09-21T21:42:31.688 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.688 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.688 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.688 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564709+0000", 2024-09-21T21:42:31.688 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8949999999999997e-06 2024-09-21T21:42:31.688 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.688 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.688 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.688 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.689 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.689 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:4322 3.19 3.d0490659 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.689 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.565289+0000", 2024-09-21T21:42:31.689 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.00033679000000000001, 2024-09-21T21:42:31.689 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0016910110000000001, 2024-09-21T21:42:31.689 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.689 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.689 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.689 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.689 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.690 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.690 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4322 2024-09-21T21:42:31.690 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.690 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.690 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.690 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.690 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565289+0000", 2024-09-21T21:42:31.690 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.690 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.690 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.690 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.690 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565289+0000", 2024-09-21T21:42:31.690 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.690 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.690 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.691 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.691 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565290+0000", 2024-09-21T21:42:31.691 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1199999999999999e-07 2024-09-21T21:42:31.691 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.691 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.691 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.691 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565291+0000", 2024-09-21T21:42:31.691 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0380000000000001e-06 2024-09-21T21:42:31.691 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.691 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.691 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.691 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565294+0000", 2024-09-21T21:42:31.691 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.056e-06 2024-09-21T21:42:31.691 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.692 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.692 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.692 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565296+0000", 2024-09-21T21:42:31.692 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.5280000000000002e-06 2024-09-21T21:42:31.692 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.692 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.692 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.692 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.692 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.692 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:3911 3.18 3.1f231578 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.692 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.563064+0000", 2024-09-21T21:42:31.692 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.0025622539999999999, 2024-09-21T21:42:31.692 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.003954979, 2024-09-21T21:42:31.693 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.693 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.693 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.693 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.693 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.693 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.693 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3911 2024-09-21T21:42:31.693 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.693 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.693 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.693 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.693 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563064+0000", 2024-09-21T21:42:31.693 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.693 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.693 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.694 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.694 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563064+0000", 2024-09-21T21:42:31.694 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.694 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.694 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.694 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.694 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563064+0000", 2024-09-21T21:42:31.694 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.1899999999999998e-07 2024-09-21T21:42:31.694 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.694 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.694 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.694 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563066+0000", 2024-09-21T21:42:31.694 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.6139999999999999e-06 2024-09-21T21:42:31.694 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.695 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.695 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.695 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563071+0000", 2024-09-21T21:42:31.695 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4040000000000003e-06 2024-09-21T21:42:31.695 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.695 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.695 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.695 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563076+0000", 2024-09-21T21:42:31.695 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0919999999999997e-06 2024-09-21T21:42:31.695 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.695 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.695 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.695 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.695 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.695 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:4219 3.1a 3.51c7a5a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.696 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.564714+0000", 2024-09-21T21:42:31.696 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.00091189699999999995, 2024-09-21T21:42:31.696 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0023572889999999998, 2024-09-21T21:42:31.696 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.696 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.696 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.696 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.696 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.696 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.696 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4219 2024-09-21T21:42:31.696 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.696 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.696 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.696 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.697 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564714+0000", 2024-09-21T21:42:31.697 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.697 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.697 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.697 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.697 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564714+0000", 2024-09-21T21:42:31.697 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.697 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.697 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.697 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.697 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564715+0000", 2024-09-21T21:42:31.697 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3099999999999998e-07 2024-09-21T21:42:31.697 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.697 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.697 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.698 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564720+0000", 2024-09-21T21:42:31.698 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5820000000000001e-06 2024-09-21T21:42:31.698 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.698 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.698 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.698 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564726+0000", 2024-09-21T21:42:31.698 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9240000000000004e-06 2024-09-21T21:42:31.698 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.698 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.698 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.698 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564732+0000", 2024-09-21T21:42:31.698 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2730000000000002e-06 2024-09-21T21:42:31.698 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.698 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.699 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.699 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.699 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.699 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:4323 3.19 3.d0490659 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.699 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.565299+0000", 2024-09-21T21:42:31.699 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.00032703499999999999, 2024-09-21T21:42:31.699 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0018287589999999999, 2024-09-21T21:42:31.699 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.699 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.699 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.699 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.699 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.699 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.699 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4323 2024-09-21T21:42:31.699 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.700 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.700 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.700 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.700 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565299+0000", 2024-09-21T21:42:31.700 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.700 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.700 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.700 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.700 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565299+0000", 2024-09-21T21:42:31.700 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.700 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.700 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.700 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.700 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565299+0000", 2024-09-21T21:42:31.700 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1400000000000001e-07 2024-09-21T21:42:31.701 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.701 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.701 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.701 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565300+0000", 2024-09-21T21:42:31.701 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0860000000000001e-06 2024-09-21T21:42:31.701 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.701 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.701 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.701 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565304+0000", 2024-09-21T21:42:31.701 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7850000000000002e-06 2024-09-21T21:42:31.701 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.701 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.701 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.701 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565307+0000", 2024-09-21T21:42:31.702 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.6539999999999999e-06 2024-09-21T21:42:31.702 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.702 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.702 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.702 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.702 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.702 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:3927 3.18 3.1f231578 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.702 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.563088+0000", 2024-09-21T21:42:31.702 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.0025384299999999999, 2024-09-21T21:42:31.702 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0040936619999999996, 2024-09-21T21:42:31.702 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.702 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.702 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.702 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.703 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.703 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.703 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3927 2024-09-21T21:42:31.703 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.703 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.703 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.703 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.703 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563088+0000", 2024-09-21T21:42:31.703 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.703 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.703 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.703 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.704 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563088+0000", 2024-09-21T21:42:31.704 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.704 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.704 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.704 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.704 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563088+0000", 2024-09-21T21:42:31.704 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3999999999999997e-07 2024-09-21T21:42:31.704 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.704 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.704 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.704 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563090+0000", 2024-09-21T21:42:31.704 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0990000000000001e-06 2024-09-21T21:42:31.704 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.704 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.705 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.705 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563096+0000", 2024-09-21T21:42:31.705 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3060000000000003e-06 2024-09-21T21:42:31.705 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.705 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.705 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.705 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563101+0000", 2024-09-21T21:42:31.705 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5530000000000003e-06 2024-09-21T21:42:31.705 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.705 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.705 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.705 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.705 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.705 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:4226 3.19 3.d0490659 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.705 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.564738+0000", 2024-09-21T21:42:31.706 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.00088847999999999998, 2024-09-21T21:42:31.706 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.002487038, 2024-09-21T21:42:31.706 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.706 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.706 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.706 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.706 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.706 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.706 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4226 2024-09-21T21:42:31.706 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.706 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.706 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.706 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.706 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564738+0000", 2024-09-21T21:42:31.707 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.707 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.707 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.707 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.707 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564738+0000", 2024-09-21T21:42:31.707 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.707 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.707 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.707 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.707 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564738+0000", 2024-09-21T21:42:31.707 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5400000000000002e-07 2024-09-21T21:42:31.707 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.707 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.707 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.708 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564740+0000", 2024-09-21T21:42:31.708 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.058e-06 2024-09-21T21:42:31.708 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.708 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.708 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.708 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564746+0000", 2024-09-21T21:42:31.708 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7880000000000003e-06 2024-09-21T21:42:31.708 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.708 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.708 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.708 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564750+0000", 2024-09-21T21:42:31.708 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2259999999999999e-06 2024-09-21T21:42:31.708 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.708 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.709 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.709 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.709 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.709 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:4328 3.18 3.1f231578 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.709 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.565310+0000", 2024-09-21T21:42:31.709 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.000316025, 2024-09-21T21:42:31.709 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.001942645, 2024-09-21T21:42:31.709 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.709 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.709 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.709 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.709 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.709 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.709 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4328 2024-09-21T21:42:31.710 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.710 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.710 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.710 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.710 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565310+0000", 2024-09-21T21:42:31.710 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.710 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.710 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.710 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.710 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565310+0000", 2024-09-21T21:42:31.710 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.710 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.710 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.711 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.711 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565310+0000", 2024-09-21T21:42:31.711 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.28e-07 2024-09-21T21:42:31.711 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.711 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.711 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.711 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565311+0000", 2024-09-21T21:42:31.711 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.096e-06 2024-09-21T21:42:31.711 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.711 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.711 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.711 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565315+0000", 2024-09-21T21:42:31.711 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.2600000000000001e-06 2024-09-21T21:42:31.711 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.711 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.712 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.712 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565317+0000", 2024-09-21T21:42:31.712 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.667e-06 2024-09-21T21:42:31.712 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.712 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.712 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.712 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.712 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.712 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:3928 3.18 3.1f231578 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.712 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.563120+0000", 2024-09-21T21:42:31.712 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.002506065, 2024-09-21T21:42:31.712 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0041627130000000002, 2024-09-21T21:42:31.712 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.713 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.713 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.713 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.713 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.713 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.713 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3928 2024-09-21T21:42:31.713 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.713 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.713 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.713 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.713 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563120+0000", 2024-09-21T21:42:31.713 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.713 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.713 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.713 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.714 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563120+0000", 2024-09-21T21:42:31.714 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.714 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.714 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.714 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.714 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563121+0000", 2024-09-21T21:42:31.714 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5000000000000003e-07 2024-09-21T21:42:31.714 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.714 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.714 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.714 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563123+0000", 2024-09-21T21:42:31.714 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1440000000000001e-06 2024-09-21T21:42:31.714 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.714 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.714 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.715 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563128+0000", 2024-09-21T21:42:31.715 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7250000000000002e-06 2024-09-21T21:42:31.715 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.715 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.715 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.715 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563133+0000", 2024-09-21T21:42:31.715 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6789999999999996e-06 2024-09-21T21:42:31.715 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.715 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.715 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.715 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.715 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.715 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:4227 3.19 3.d0490659 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.715 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.564763+0000", 2024-09-21T21:42:31.715 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.00086315100000000004, 2024-09-21T21:42:31.716 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0025474159999999998, 2024-09-21T21:42:31.716 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.716 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.716 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.716 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.716 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.716 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.716 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4227 2024-09-21T21:42:31.716 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.716 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.716 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.716 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.716 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564763+0000", 2024-09-21T21:42:31.716 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.716 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.716 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.717 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.717 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564763+0000", 2024-09-21T21:42:31.717 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.717 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.717 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.717 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.717 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564763+0000", 2024-09-21T21:42:31.717 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.4200000000000002e-07 2024-09-21T21:42:31.717 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.717 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.717 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.717 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564765+0000", 2024-09-21T21:42:31.717 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9089999999999998e-06 2024-09-21T21:42:31.717 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.717 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.717 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.718 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564769+0000", 2024-09-21T21:42:31.718 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5710000000000002e-06 2024-09-21T21:42:31.718 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.718 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.718 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.718 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564771+0000", 2024-09-21T21:42:31.718 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.7319999999999998e-06 2024-09-21T21:42:31.718 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.718 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.718 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.718 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.718 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.718 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:4330 3.1a 3.51c7a5a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.718 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.565320+0000", 2024-09-21T21:42:31.719 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.00030579799999999997, 2024-09-21T21:42:31.719 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0020322360000000002, 2024-09-21T21:42:31.719 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.719 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.719 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.719 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.719 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.719 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.719 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4330 2024-09-21T21:42:31.719 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.719 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.719 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.719 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.719 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565320+0000", 2024-09-21T21:42:31.720 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.720 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.720 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.720 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.720 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565320+0000", 2024-09-21T21:42:31.720 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.720 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.720 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.720 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.720 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565321+0000", 2024-09-21T21:42:31.720 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.41e-07 2024-09-21T21:42:31.720 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.720 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.720 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.720 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565322+0000", 2024-09-21T21:42:31.720 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.288e-06 2024-09-21T21:42:31.721 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.721 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.721 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.721 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565325+0000", 2024-09-21T21:42:31.721 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5410000000000001e-06 2024-09-21T21:42:31.721 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.721 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.721 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.721 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565328+0000", 2024-09-21T21:42:31.721 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.5720000000000001e-06 2024-09-21T21:42:31.721 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.721 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.721 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.721 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.722 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.722 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:3953 3.18 3.1f231578 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.722 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.563140+0000", 2024-09-21T21:42:31.722 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.0024865429999999999, 2024-09-21T21:42:31.722 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0042676900000000002, 2024-09-21T21:42:31.722 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.722 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.722 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.722 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.722 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.722 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.722 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3953 2024-09-21T21:42:31.722 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.722 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.722 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.723 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.723 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563140+0000", 2024-09-21T21:42:31.723 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.723 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.723 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.723 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.723 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563140+0000", 2024-09-21T21:42:31.723 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.723 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.723 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.723 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.723 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563140+0000", 2024-09-21T21:42:31.723 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.77e-07 2024-09-21T21:42:31.723 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.724 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.724 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.724 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563142+0000", 2024-09-21T21:42:31.724 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.854e-06 2024-09-21T21:42:31.724 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.724 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.724 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.724 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563148+0000", 2024-09-21T21:42:31.724 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3840000000000002e-06 2024-09-21T21:42:31.724 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.724 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.724 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.724 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563153+0000", 2024-09-21T21:42:31.724 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7090000000000001e-06 2024-09-21T21:42:31.725 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.725 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.725 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.725 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.725 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.725 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:4231 3.18 3.1f231578 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.725 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.564775+0000", 2024-09-21T21:42:31.725 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.00085096599999999998, 2024-09-21T21:42:31.725 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0026876019999999999, 2024-09-21T21:42:31.725 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.725 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.725 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.725 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.725 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.725 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.726 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4231 2024-09-21T21:42:31.726 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.726 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.726 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.726 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.726 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564775+0000", 2024-09-21T21:42:31.726 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.726 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.726 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.726 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.726 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564775+0000", 2024-09-21T21:42:31.726 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.726 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.726 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.726 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.727 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564775+0000", 2024-09-21T21:42:31.727 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.1899999999999998e-07 2024-09-21T21:42:31.727 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.727 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.727 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.727 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564777+0000", 2024-09-21T21:42:31.727 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.2559999999999999e-06 2024-09-21T21:42:31.727 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.727 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.727 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.727 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564780+0000", 2024-09-21T21:42:31.727 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.1410000000000001e-06 2024-09-21T21:42:31.727 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.727 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.728 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.728 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564782+0000", 2024-09-21T21:42:31.728 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.633e-06 2024-09-21T21:42:31.728 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.728 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.728 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.728 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.728 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.728 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:4331 3.1a 3.51c7a5a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.728 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.565351+0000", 2024-09-21T21:42:31.728 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.00027553700000000001, 2024-09-21T21:42:31.728 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0021673930000000001, 2024-09-21T21:42:31.728 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.728 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.729 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.729 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.729 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.729 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.729 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4331 2024-09-21T21:42:31.729 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.729 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.729 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.729 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.729 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565351+0000", 2024-09-21T21:42:31.729 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.729 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.729 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.729 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.729 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565351+0000", 2024-09-21T21:42:31.730 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.730 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.730 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.730 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.730 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565351+0000", 2024-09-21T21:42:31.730 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4099999999999999e-07 2024-09-21T21:42:31.730 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.730 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.730 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.730 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565352+0000", 2024-09-21T21:42:31.730 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.4610000000000001e-06 2024-09-21T21:42:31.730 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.730 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.730 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.731 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565356+0000", 2024-09-21T21:42:31.731 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.2440000000000002e-06 2024-09-21T21:42:31.731 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.731 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.731 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.731 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565358+0000", 2024-09-21T21:42:31.731 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.5150000000000001e-06 2024-09-21T21:42:31.731 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.731 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.731 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.731 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.731 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.731 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:3954 3.18 3.1f231578 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.731 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.563159+0000", 2024-09-21T21:42:31.731 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.0024674459999999999, 2024-09-21T21:42:31.732 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0043966379999999996, 2024-09-21T21:42:31.732 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.732 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.732 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.732 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.732 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.732 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.732 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3954 2024-09-21T21:42:31.732 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.732 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.732 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.732 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.732 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563159+0000", 2024-09-21T21:42:31.732 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.732 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.733 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.733 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.733 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563159+0000", 2024-09-21T21:42:31.733 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.733 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.733 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.733 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.733 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563159+0000", 2024-09-21T21:42:31.733 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.39e-07 2024-09-21T21:42:31.733 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.733 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.733 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.733 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563161+0000", 2024-09-21T21:42:31.733 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0899999999999999e-06 2024-09-21T21:42:31.734 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.734 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.734 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.734 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563167+0000", 2024-09-21T21:42:31.734 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6590000000000004e-06 2024-09-21T21:42:31.734 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.734 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.734 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.734 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563171+0000", 2024-09-21T21:42:31.734 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5859999999999998e-06 2024-09-21T21:42:31.734 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.734 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.734 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.734 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.735 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.735 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:4055 3.18 3.1f231578 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.735 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.563719+0000", 2024-09-21T21:42:31.735 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.001907216, 2024-09-21T21:42:31.735 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0038647989999999999, 2024-09-21T21:42:31.735 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.735 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.735 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.735 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.735 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.735 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.735 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4055 2024-09-21T21:42:31.735 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.735 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.735 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.736 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.736 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563719+0000", 2024-09-21T21:42:31.736 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.736 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.736 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.736 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.736 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563719+0000", 2024-09-21T21:42:31.736 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.736 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.736 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.736 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.736 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563719+0000", 2024-09-21T21:42:31.736 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6300000000000005e-07 2024-09-21T21:42:31.736 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.736 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.737 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.737 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563727+0000", 2024-09-21T21:42:31.737 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0320000000000003e-06 2024-09-21T21:42:31.737 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.737 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.737 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.737 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563734+0000", 2024-09-21T21:42:31.737 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6490000000000002e-06 2024-09-21T21:42:31.737 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.737 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.737 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.737 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563739+0000", 2024-09-21T21:42:31.737 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9529999999999998e-06 2024-09-21T21:42:31.737 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.737 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.738 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.738 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.738 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.738 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:4146 3.18 3.1f231578 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.738 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.564248+0000", 2024-09-21T21:42:31.738 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.001378038, 2024-09-21T21:42:31.738 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0033634619999999998, 2024-09-21T21:42:31.738 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.738 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.738 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.738 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.738 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.738 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.739 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4146 2024-09-21T21:42:31.739 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.739 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.739 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.739 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.739 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564248+0000", 2024-09-21T21:42:31.739 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.739 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.739 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.739 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.739 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564248+0000", 2024-09-21T21:42:31.739 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.739 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.739 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.740 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.740 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564248+0000", 2024-09-21T21:42:31.740 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.16e-07 2024-09-21T21:42:31.740 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.740 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.740 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.740 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564250+0000", 2024-09-21T21:42:31.740 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.2869999999999999e-06 2024-09-21T21:42:31.740 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.740 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.740 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.740 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564253+0000", 2024-09-21T21:42:31.740 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.9529999999999999e-06 2024-09-21T21:42:31.740 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.740 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.740 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.741 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564255+0000", 2024-09-21T21:42:31.741 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.402e-06 2024-09-21T21:42:31.741 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.741 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.741 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.741 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.741 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.741 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:4234 3.1a 3.51c7a5a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.741 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.564785+0000", 2024-09-21T21:42:31.741 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.000840631, 2024-09-21T21:42:31.741 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0028658889999999999, 2024-09-21T21:42:31.741 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.741 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.741 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.741 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.742 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.742 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.742 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4234 2024-09-21T21:42:31.742 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.742 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.742 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.742 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.742 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564785+0000", 2024-09-21T21:42:31.742 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.742 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.742 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.742 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.743 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564785+0000", 2024-09-21T21:42:31.743 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.743 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.743 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.743 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.743 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564786+0000", 2024-09-21T21:42:31.743 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0599999999999999e-07 2024-09-21T21:42:31.743 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.743 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.743 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.743 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564787+0000", 2024-09-21T21:42:31.743 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.3069999999999999e-06 2024-09-21T21:42:31.743 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.743 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.743 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.743 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564790+0000", 2024-09-21T21:42:31.744 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0369999999999999e-06 2024-09-21T21:42:31.744 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.744 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.744 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.744 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564793+0000", 2024-09-21T21:42:31.744 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.6299999999999998e-06 2024-09-21T21:42:31.744 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.744 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.744 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.744 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.744 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.744 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:4343 3.19 3.d0490659 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.744 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.565364+0000", 2024-09-21T21:42:31.744 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.000261957, 2024-09-21T21:42:31.745 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.002343557, 2024-09-21T21:42:31.745 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.745 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.745 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.745 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.745 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.745 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.745 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4343 2024-09-21T21:42:31.745 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.745 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.745 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.745 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.745 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565364+0000", 2024-09-21T21:42:31.745 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.746 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.746 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.746 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.746 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565364+0000", 2024-09-21T21:42:31.746 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.746 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.746 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.746 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.746 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565364+0000", 2024-09-21T21:42:31.746 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.22e-07 2024-09-21T21:42:31.746 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.746 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.746 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.746 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565366+0000", 2024-09-21T21:42:31.747 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.6899999999999999e-06 2024-09-21T21:42:31.747 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.747 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.747 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.747 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565369+0000", 2024-09-21T21:42:31.747 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.394e-06 2024-09-21T21:42:31.747 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.747 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.747 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.747 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565374+0000", 2024-09-21T21:42:31.747 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1930000000000003e-06 2024-09-21T21:42:31.747 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.747 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.747 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.747 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.748 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.748 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:3956 3.18 3.1f231578 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.748 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.563177+0000", 2024-09-21T21:42:31.748 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.0024490419999999998, 2024-09-21T21:42:31.748 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0045847809999999996, 2024-09-21T21:42:31.748 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.748 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.748 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.748 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.748 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.748 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.748 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3956 2024-09-21T21:42:31.748 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.748 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.749 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.749 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.749 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563177+0000", 2024-09-21T21:42:31.749 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.749 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.749 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.749 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.749 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563177+0000", 2024-09-21T21:42:31.749 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.749 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.749 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.749 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.749 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563177+0000", 2024-09-21T21:42:31.749 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5400000000000002e-07 2024-09-21T21:42:31.749 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.750 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.750 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.750 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563179+0000", 2024-09-21T21:42:31.750 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9099999999999999e-06 2024-09-21T21:42:31.750 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.750 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.750 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.750 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563185+0000", 2024-09-21T21:42:31.750 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7710000000000003e-06 2024-09-21T21:42:31.750 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.750 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.750 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.750 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563190+0000", 2024-09-21T21:42:31.750 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6079999999999998e-06 2024-09-21T21:42:31.751 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.751 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.751 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.751 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.751 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.751 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:4057 3.18 3.1f231578 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.751 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.563746+0000", 2024-09-21T21:42:31.751 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.0018802770000000001, 2024-09-21T21:42:31.751 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0040687170000000003, 2024-09-21T21:42:31.751 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.751 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.751 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.751 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.751 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.752 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.752 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4057 2024-09-21T21:42:31.752 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.752 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.752 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.752 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.752 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563746+0000", 2024-09-21T21:42:31.752 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.752 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.752 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.752 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.752 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563746+0000", 2024-09-21T21:42:31.752 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.752 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.752 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.753 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.753 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563746+0000", 2024-09-21T21:42:31.753 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0699999999999997e-07 2024-09-21T21:42:31.753 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.753 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.753 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.753 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563748+0000", 2024-09-21T21:42:31.753 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.7579999999999999e-06 2024-09-21T21:42:31.753 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.753 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.753 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.753 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563754+0000", 2024-09-21T21:42:31.753 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2489999999999998e-06 2024-09-21T21:42:31.753 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.753 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.754 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.754 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563759+0000", 2024-09-21T21:42:31.754 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.296e-06 2024-09-21T21:42:31.754 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.754 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.754 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.754 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.754 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.754 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:4147 3.18 3.1f231578 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.754 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.564258+0000", 2024-09-21T21:42:31.754 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.0013681769999999999, 2024-09-21T21:42:31.754 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0035948920000000001, 2024-09-21T21:42:31.754 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.754 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.754 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.755 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.755 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.755 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.755 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4147 2024-09-21T21:42:31.755 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.755 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.755 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.755 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.755 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564258+0000", 2024-09-21T21:42:31.755 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.755 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.755 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.755 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.755 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564258+0000", 2024-09-21T21:42:31.755 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.756 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.756 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.756 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.756 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564258+0000", 2024-09-21T21:42:31.756 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0800000000000001e-07 2024-09-21T21:42:31.756 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.756 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.756 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.756 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564259+0000", 2024-09-21T21:42:31.756 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1570000000000001e-06 2024-09-21T21:42:31.756 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.756 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.756 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.756 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564262+0000", 2024-09-21T21:42:31.756 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.9119999999999998e-06 2024-09-21T21:42:31.757 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.757 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.757 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.757 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564265+0000", 2024-09-21T21:42:31.757 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.8030000000000001e-06 2024-09-21T21:42:31.757 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.757 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.757 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.757 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.757 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.757 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:4235 3.1a 3.51c7a5a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.757 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.564795+0000", 2024-09-21T21:42:31.757 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.00083068900000000002, 2024-09-21T21:42:31.757 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0030844660000000001, 2024-09-21T21:42:31.758 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.758 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.758 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.758 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.758 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.758 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.758 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4235 2024-09-21T21:42:31.758 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.758 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.758 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.758 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.758 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564795+0000", 2024-09-21T21:42:31.758 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.759 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.759 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.759 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.759 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564795+0000", 2024-09-21T21:42:31.759 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.759 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.759 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.759 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.759 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564796+0000", 2024-09-21T21:42:31.759 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1199999999999999e-07 2024-09-21T21:42:31.759 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.759 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.759 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.759 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564797+0000", 2024-09-21T21:42:31.759 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.17e-06 2024-09-21T21:42:31.760 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.760 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.760 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.760 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564801+0000", 2024-09-21T21:42:31.760 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8199999999999998e-06 2024-09-21T21:42:31.760 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.760 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.760 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.760 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564804+0000", 2024-09-21T21:42:31.760 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.897e-06 2024-09-21T21:42:31.760 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.760 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.760 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.760 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.760 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.761 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:4345 3.19 3.d0490659 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.761 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.565380+0000", 2024-09-21T21:42:31.761 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.00024634899999999998, 2024-09-21T21:42:31.761 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0025341579999999999, 2024-09-21T21:42:31.761 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.761 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.761 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.761 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.761 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.761 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.761 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4345 2024-09-21T21:42:31.761 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.761 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.761 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.762 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.762 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565380+0000", 2024-09-21T21:42:31.762 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.762 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.762 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.762 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.762 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565380+0000", 2024-09-21T21:42:31.762 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.762 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.762 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.762 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.762 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565380+0000", 2024-09-21T21:42:31.762 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3399999999999999e-07 2024-09-21T21:42:31.762 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.763 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.763 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.763 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565382+0000", 2024-09-21T21:42:31.763 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.928e-06 2024-09-21T21:42:31.763 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.763 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.763 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.763 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565388+0000", 2024-09-21T21:42:31.763 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9279999999999998e-06 2024-09-21T21:42:31.763 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.763 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.763 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.763 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565393+0000", 2024-09-21T21:42:31.763 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4479999999999996e-06 2024-09-21T21:42:31.764 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.764 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.764 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.764 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.764 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.764 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:3958 3.18 3.1f231578 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.764 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.563195+0000", 2024-09-21T21:42:31.764 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.0024308300000000001, 2024-09-21T21:42:31.764 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0047653879999999997, 2024-09-21T21:42:31.764 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.764 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.764 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.764 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.764 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.764 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.765 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3958 2024-09-21T21:42:31.765 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.765 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.765 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.765 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.765 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563195+0000", 2024-09-21T21:42:31.765 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.765 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.765 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.765 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.765 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563195+0000", 2024-09-21T21:42:31.765 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.765 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.765 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.765 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.766 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563196+0000", 2024-09-21T21:42:31.766 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.1300000000000001e-07 2024-09-21T21:42:31.766 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.766 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.766 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.766 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563197+0000", 2024-09-21T21:42:31.766 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.7749999999999999e-06 2024-09-21T21:42:31.766 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.766 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.766 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.766 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563203+0000", 2024-09-21T21:42:31.766 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.908e-06 2024-09-21T21:42:31.766 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.766 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.767 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.767 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563208+0000", 2024-09-21T21:42:31.767 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5759999999999999e-06 2024-09-21T21:42:31.767 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.767 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.767 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.767 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.767 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.767 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:4148 3.18 3.1f231578 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.767 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.564268+0000", 2024-09-21T21:42:31.767 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.0013582819999999999, 2024-09-21T21:42:31.767 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0037230760000000001, 2024-09-21T21:42:31.767 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.767 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.767 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.767 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.768 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.768 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.768 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4148 2024-09-21T21:42:31.768 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.768 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.768 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.768 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.768 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564268+0000", 2024-09-21T21:42:31.768 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.768 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.768 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.768 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.768 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564268+0000", 2024-09-21T21:42:31.768 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.769 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.769 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.769 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.769 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564268+0000", 2024-09-21T21:42:31.769 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.36e-07 2024-09-21T21:42:31.769 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.769 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.769 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.769 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564269+0000", 2024-09-21T21:42:31.769 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1689999999999999e-06 2024-09-21T21:42:31.769 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.769 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.769 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.769 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564272+0000", 2024-09-21T21:42:31.770 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.937e-06 2024-09-21T21:42:31.770 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.770 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.770 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.770 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564275+0000", 2024-09-21T21:42:31.770 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.4380000000000002e-06 2024-09-21T21:42:31.770 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.770 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.770 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.770 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.770 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.770 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:4247 3.19 3.d0490659 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.770 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.564807+0000", 2024-09-21T21:42:31.770 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.00081918599999999996, 2024-09-21T21:42:31.770 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.003210882, 2024-09-21T21:42:31.771 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.771 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.771 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.771 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.771 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.771 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.771 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4247 2024-09-21T21:42:31.771 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.771 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.771 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.771 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.771 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564807+0000", 2024-09-21T21:42:31.771 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.771 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.771 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.772 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.772 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564807+0000", 2024-09-21T21:42:31.772 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.772 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.772 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.772 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.772 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564807+0000", 2024-09-21T21:42:31.772 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.16e-07 2024-09-21T21:42:31.772 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.772 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.772 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.772 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564808+0000", 2024-09-21T21:42:31.772 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.209e-06 2024-09-21T21:42:31.772 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.773 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.773 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.773 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564811+0000", 2024-09-21T21:42:31.773 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.9220000000000001e-06 2024-09-21T21:42:31.773 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.773 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.773 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.773 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564815+0000", 2024-09-21T21:42:31.773 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.371e-06 2024-09-21T21:42:31.773 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.773 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.773 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.773 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.773 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.774 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:4346 3.19 3.d0490659 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.774 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.565399+0000", 2024-09-21T21:42:31.774 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.00022701500000000001, 2024-09-21T21:42:31.774 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.002646401, 2024-09-21T21:42:31.774 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.774 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.774 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.774 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.774 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.774 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.774 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4346 2024-09-21T21:42:31.774 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.774 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.774 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.774 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.775 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565399+0000", 2024-09-21T21:42:31.775 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.775 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.775 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.775 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.775 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565399+0000", 2024-09-21T21:42:31.775 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.775 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.775 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.775 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.775 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565399+0000", 2024-09-21T21:42:31.775 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.2599999999999998e-07 2024-09-21T21:42:31.775 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.775 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.775 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.776 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565401+0000", 2024-09-21T21:42:31.776 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.962e-06 2024-09-21T21:42:31.776 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.776 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.776 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.776 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565408+0000", 2024-09-21T21:42:31.776 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3350000000000003e-06 2024-09-21T21:42:31.776 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.776 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.776 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.776 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565412+0000", 2024-09-21T21:42:31.776 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4610000000000001e-06 2024-09-21T21:42:31.776 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.776 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.777 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.777 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.777 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.777 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:4149 3.18 3.1f231578 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.777 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.564280+0000", 2024-09-21T21:42:31.777 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.0013459699999999999, 2024-09-21T21:42:31.777 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0037934940000000001, 2024-09-21T21:42:31.777 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.777 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.777 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.777 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.777 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.777 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.777 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4149 2024-09-21T21:42:31.777 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.777 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.778 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.778 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.778 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564280+0000", 2024-09-21T21:42:31.778 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.778 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.778 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.778 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.778 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564280+0000", 2024-09-21T21:42:31.778 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.778 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.778 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.778 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.778 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564281+0000", 2024-09-21T21:42:31.778 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8899999999999999e-07 2024-09-21T21:42:31.779 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.779 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.779 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.779 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564282+0000", 2024-09-21T21:42:31.779 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.2249999999999999e-06 2024-09-21T21:42:31.779 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.779 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.779 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.779 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564286+0000", 2024-09-21T21:42:31.779 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2150000000000003e-06 2024-09-21T21:42:31.779 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.779 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.779 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.779 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564291+0000", 2024-09-21T21:42:31.780 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3409999999999996e-06 2024-09-21T21:42:31.780 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.780 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.780 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.780 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.780 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.780 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:4249 3.19 3.d0490659 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.780 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.564818+0000", 2024-09-21T21:42:31.780 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.00080853800000000003, 2024-09-21T21:42:31.780 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0032839750000000002, 2024-09-21T21:42:31.780 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.780 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.780 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.780 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.780 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.781 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.781 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4249 2024-09-21T21:42:31.781 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.781 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.781 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.781 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.781 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564818+0000", 2024-09-21T21:42:31.781 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.781 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.781 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.781 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.781 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564818+0000", 2024-09-21T21:42:31.781 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.781 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.782 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.782 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.782 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564818+0000", 2024-09-21T21:42:31.782 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1799999999999999e-07 2024-09-21T21:42:31.782 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.782 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.782 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.782 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564819+0000", 2024-09-21T21:42:31.782 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0950000000000001e-06 2024-09-21T21:42:31.782 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.782 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.782 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.782 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564822+0000", 2024-09-21T21:42:31.782 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.9299999999999999e-06 2024-09-21T21:42:31.782 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.783 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.783 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.783 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564824+0000", 2024-09-21T21:42:31.783 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.6000000000000001e-06 2024-09-21T21:42:31.783 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.783 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.783 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.783 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.783 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.783 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:4348 3.19 3.d0490659 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.783 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.565419+0000", 2024-09-21T21:42:31.783 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.00020724, 2024-09-21T21:42:31.783 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0027093820000000002, 2024-09-21T21:42:31.783 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.784 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.784 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.784 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.784 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.784 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.784 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4348 2024-09-21T21:42:31.784 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.784 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.784 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.784 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.784 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565419+0000", 2024-09-21T21:42:31.784 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.784 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.784 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.784 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.785 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565419+0000", 2024-09-21T21:42:31.785 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.785 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.785 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.785 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.785 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565419+0000", 2024-09-21T21:42:31.785 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.89e-07 2024-09-21T21:42:31.785 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.785 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.785 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.785 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565421+0000", 2024-09-21T21:42:31.785 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.8330000000000001e-06 2024-09-21T21:42:31.785 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.785 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.786 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.786 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565427+0000", 2024-09-21T21:42:31.786 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9619999999999998e-06 2024-09-21T21:42:31.786 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.786 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.786 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.786 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565432+0000", 2024-09-21T21:42:31.786 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4159999999999997e-06 2024-09-21T21:42:31.786 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.786 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.786 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.786 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.786 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.786 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:4250 3.19 3.d0490659 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.787 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.564827+0000", 2024-09-21T21:42:31.787 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.00079871, 2024-09-21T21:42:31.787 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0033340119999999999, 2024-09-21T21:42:31.787 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.787 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.787 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.787 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.787 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.787 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.787 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4250 2024-09-21T21:42:31.787 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.787 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.787 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.787 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.787 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564827+0000", 2024-09-21T21:42:31.788 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.788 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.788 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.788 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.788 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564827+0000", 2024-09-21T21:42:31.788 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.788 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.788 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.788 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.788 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564828+0000", 2024-09-21T21:42:31.788 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3200000000000001e-07 2024-09-21T21:42:31.788 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.788 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.788 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.788 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564829+0000", 2024-09-21T21:42:31.789 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.519e-06 2024-09-21T21:42:31.789 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.789 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.789 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.789 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564834+0000", 2024-09-21T21:42:31.789 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1200000000000001e-06 2024-09-21T21:42:31.789 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.789 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.789 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.789 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564837+0000", 2024-09-21T21:42:31.789 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.942e-06 2024-09-21T21:42:31.789 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.789 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.789 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.789 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.790 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.790 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:4354 3.18 3.1f231578 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.790 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.565437+0000", 2024-09-21T21:42:31.790 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.00018893600000000001, 2024-09-21T21:42:31.790 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0027683880000000001, 2024-09-21T21:42:31.790 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.790 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.790 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.790 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.790 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.790 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.790 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4354 2024-09-21T21:42:31.790 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.790 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.790 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.791 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.791 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565437+0000", 2024-09-21T21:42:31.791 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.791 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.791 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.791 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.791 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565437+0000", 2024-09-21T21:42:31.791 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.791 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.791 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.791 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.791 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565438+0000", 2024-09-21T21:42:31.791 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.53e-07 2024-09-21T21:42:31.791 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.792 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.792 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.792 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565439+0000", 2024-09-21T21:42:31.792 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.8470000000000001e-06 2024-09-21T21:42:31.792 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.792 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.792 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.792 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565445+0000", 2024-09-21T21:42:31.792 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0469999999999999e-06 2024-09-21T21:42:31.792 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.792 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.792 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.792 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565450+0000", 2024-09-21T21:42:31.792 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5530000000000003e-06 2024-09-21T21:42:31.792 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.793 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.793 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.793 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.793 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.793 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:4253 3.19 3.d0490659 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.793 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.564841+0000", 2024-09-21T21:42:31.793 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.00078509200000000004, 2024-09-21T21:42:31.793 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0033990169999999998, 2024-09-21T21:42:31.793 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.793 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.793 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.793 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.793 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.793 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.794 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4253 2024-09-21T21:42:31.794 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.794 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.794 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.794 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.794 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564841+0000", 2024-09-21T21:42:31.794 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.794 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.794 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.794 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.794 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564841+0000", 2024-09-21T21:42:31.794 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.794 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.794 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.794 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.795 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564842+0000", 2024-09-21T21:42:31.795 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8999999999999997e-07 2024-09-21T21:42:31.795 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.795 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.795 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.795 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564843+0000", 2024-09-21T21:42:31.795 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.8360000000000001e-06 2024-09-21T21:42:31.795 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.795 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.795 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.795 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564849+0000", 2024-09-21T21:42:31.795 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5929999999999996e-06 2024-09-21T21:42:31.795 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.795 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.795 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.796 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564853+0000", 2024-09-21T21:42:31.796 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3270000000000002e-06 2024-09-21T21:42:31.796 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.796 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.796 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.796 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.796 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.796 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:4356 3.18 3.1f231578 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.796 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.565456+0000", 2024-09-21T21:42:31.796 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.00016989300000000001, 2024-09-21T21:42:31.796 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0028209369999999999, 2024-09-21T21:42:31.796 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.796 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.796 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.797 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.797 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.797 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.797 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4356 2024-09-21T21:42:31.797 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.797 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.797 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.797 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.797 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565456+0000", 2024-09-21T21:42:31.797 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.797 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.797 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.797 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.797 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565456+0000", 2024-09-21T21:42:31.798 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.798 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.798 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.798 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.798 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565457+0000", 2024-09-21T21:42:31.798 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.65e-07 2024-09-21T21:42:31.798 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.798 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.798 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.798 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565458+0000", 2024-09-21T21:42:31.798 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9180000000000001e-06 2024-09-21T21:42:31.798 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.798 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.798 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.798 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565464+0000", 2024-09-21T21:42:31.799 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.643e-06 2024-09-21T21:42:31.799 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.799 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.799 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.799 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565469+0000", 2024-09-21T21:42:31.799 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4900000000000002e-06 2024-09-21T21:42:31.799 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.799 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.799 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.799 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.799 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.799 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:3986 3.18 3.1f231578 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.799 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.563279+0000", 2024-09-21T21:42:31.800 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.0023473719999999999, 2024-09-21T21:42:31.800 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0050926820000000003, 2024-09-21T21:42:31.800 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.800 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.800 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:42:31.800 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.800 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.800 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.800 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3986 2024-09-21T21:42:31.800 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.800 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.800 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.800 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.800 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563279+0000", 2024-09-21T21:42:31.800 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.801 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.801 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.801 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.801 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563279+0000", 2024-09-21T21:42:31.801 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.801 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.801 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.801 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.801 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563279+0000", 2024-09-21T21:42:31.801 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7899999999999999e-07 2024-09-21T21:42:31.801 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.801 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.801 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.801 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563281+0000", 2024-09-21T21:42:31.801 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0109999999999999e-06 2024-09-21T21:42:31.802 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.802 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.802 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.802 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563287+0000", 2024-09-21T21:42:31.802 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5130000000000004e-06 2024-09-21T21:42:31.802 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.802 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.802 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.802 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563291+0000", 2024-09-21T21:42:31.802 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6670000000000004e-06 2024-09-21T21:42:31.802 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.802 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.802 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:31.802 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.568368+0000", 2024-09-21T21:42:31.802 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0050766129999999998 2024-09-21T21:42:31.802 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.803 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.803 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.803 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.803 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.803 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:4257 3.18 3.1f231578 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.803 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.564859+0000", 2024-09-21T21:42:31.803 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.00076745700000000003, 2024-09-21T21:42:31.803 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0035438029999999999, 2024-09-21T21:42:31.803 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.803 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.803 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.803 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.803 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.804 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.804 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4257 2024-09-21T21:42:31.804 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.804 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.804 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.804 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.804 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564859+0000", 2024-09-21T21:42:31.804 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.804 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.804 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.804 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.804 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564859+0000", 2024-09-21T21:42:31.804 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.804 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.804 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.805 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.805 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564859+0000", 2024-09-21T21:42:31.805 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7599999999999997e-07 2024-09-21T21:42:31.805 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.805 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.805 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.805 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564861+0000", 2024-09-21T21:42:31.805 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.804e-06 2024-09-21T21:42:31.805 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.805 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.805 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.805 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564867+0000", 2024-09-21T21:42:31.805 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.784e-06 2024-09-21T21:42:31.805 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.805 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.806 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.806 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564871+0000", 2024-09-21T21:42:31.806 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.481e-06 2024-09-21T21:42:31.806 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.806 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.806 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.806 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.806 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.806 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:4359 3.1a 3.51c7a5a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.806 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.565474+0000", 2024-09-21T21:42:31.806 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.00015170900000000001, 2024-09-21T21:42:31.806 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0029575199999999999, 2024-09-21T21:42:31.806 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.806 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.807 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.807 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.807 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.807 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.807 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4359 2024-09-21T21:42:31.807 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.807 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.807 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.807 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.807 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565474+0000", 2024-09-21T21:42:31.807 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.807 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.807 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.807 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.807 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565474+0000", 2024-09-21T21:42:31.808 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.808 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.808 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.808 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.808 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565475+0000", 2024-09-21T21:42:31.808 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8700000000000001e-07 2024-09-21T21:42:31.808 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.808 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.808 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.808 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565477+0000", 2024-09-21T21:42:31.808 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.875e-06 2024-09-21T21:42:31.808 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.808 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.808 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.808 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565483+0000", 2024-09-21T21:42:31.809 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0079999999999999e-06 2024-09-21T21:42:31.809 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.809 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.809 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.809 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565487+0000", 2024-09-21T21:42:31.809 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4789999999999998e-06 2024-09-21T21:42:31.809 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.809 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.809 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.809 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.809 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.809 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:3987 3.18 3.1f231578 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.809 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.563310+0000", 2024-09-21T21:42:31.809 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.0023162590000000002, 2024-09-21T21:42:31.809 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0051546129999999997, 2024-09-21T21:42:31.810 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.810 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.810 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.810 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.810 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.810 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.810 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3987 2024-09-21T21:42:31.810 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.810 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.810 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.810 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.810 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563310+0000", 2024-09-21T21:42:31.810 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.810 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.810 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.811 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.811 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563310+0000", 2024-09-21T21:42:31.811 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.811 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.811 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.811 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.811 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563310+0000", 2024-09-21T21:42:31.811 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.96e-07 2024-09-21T21:42:31.811 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.811 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.811 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.811 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563312+0000", 2024-09-21T21:42:31.811 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.8509999999999999e-06 2024-09-21T21:42:31.811 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.811 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.811 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.812 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563316+0000", 2024-09-21T21:42:31.812 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7330000000000001e-06 2024-09-21T21:42:31.812 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.812 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.812 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.812 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563319+0000", 2024-09-21T21:42:31.812 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.7779999999999999e-06 2024-09-21T21:42:31.812 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.812 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.812 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.812 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.812 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.812 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:4259 3.18 3.1f231578 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.812 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.564886+0000", 2024-09-21T21:42:31.813 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.00074052899999999997, 2024-09-21T21:42:31.813 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0036176490000000001, 2024-09-21T21:42:31.813 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.813 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.813 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.813 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.813 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.813 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.813 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4259 2024-09-21T21:42:31.813 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.813 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.813 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.813 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.813 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564886+0000", 2024-09-21T21:42:31.814 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.814 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.814 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.814 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.814 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564886+0000", 2024-09-21T21:42:31.814 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.814 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.814 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.814 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.814 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564886+0000", 2024-09-21T21:42:31.814 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.9999999999999998e-07 2024-09-21T21:42:31.814 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.814 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.814 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.814 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564888+0000", 2024-09-21T21:42:31.815 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.069e-06 2024-09-21T21:42:31.815 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.815 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.815 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.815 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564894+0000", 2024-09-21T21:42:31.815 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2129999999999996e-06 2024-09-21T21:42:31.815 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.815 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.815 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.815 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564899+0000", 2024-09-21T21:42:31.815 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9439999999999996e-06 2024-09-21T21:42:31.815 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.815 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.815 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.816 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.816 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.816 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:4361 3.1a 3.51c7a5a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.816 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.565495+0000", 2024-09-21T21:42:31.816 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.00013075900000000001, 2024-09-21T21:42:31.816 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0030374429999999999, 2024-09-21T21:42:31.816 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.816 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.816 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.816 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.816 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.816 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.816 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4361 2024-09-21T21:42:31.816 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.817 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.817 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.817 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.817 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565495+0000", 2024-09-21T21:42:31.817 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.817 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.817 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.817 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.817 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565495+0000", 2024-09-21T21:42:31.817 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.817 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.817 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.817 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.817 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565496+0000", 2024-09-21T21:42:31.818 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3600000000000004e-07 2024-09-21T21:42:31.818 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.818 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.818 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.818 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565498+0000", 2024-09-21T21:42:31.818 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.962e-06 2024-09-21T21:42:31.818 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.818 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.818 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.818 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565504+0000", 2024-09-21T21:42:31.818 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8930000000000002e-06 2024-09-21T21:42:31.818 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.818 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.818 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.819 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565508+0000", 2024-09-21T21:42:31.819 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.561e-06 2024-09-21T21:42:31.819 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.819 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.819 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.819 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.819 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.819 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:3988 3.18 3.1f231578 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.819 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.563322+0000", 2024-09-21T21:42:31.819 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.0023042050000000001, 2024-09-21T21:42:31.819 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.005242671, 2024-09-21T21:42:31.819 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.819 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.819 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.819 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.820 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.820 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.820 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3988 2024-09-21T21:42:31.820 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.820 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.820 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.820 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.820 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563322+0000", 2024-09-21T21:42:31.820 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.820 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.820 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.820 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.820 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563322+0000", 2024-09-21T21:42:31.820 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.821 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.821 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.821 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.821 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563322+0000", 2024-09-21T21:42:31.821 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0899999999999997e-07 2024-09-21T21:42:31.821 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.821 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.821 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.821 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563331+0000", 2024-09-21T21:42:31.821 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.3799999999999994e-06 2024-09-21T21:42:31.821 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.821 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.821 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.821 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563335+0000", 2024-09-21T21:42:31.821 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2549999999999999e-06 2024-09-21T21:42:31.822 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.822 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.822 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.822 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563338+0000", 2024-09-21T21:42:31.822 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.8159999999999998e-06 2024-09-21T21:42:31.822 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.822 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.822 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.822 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.822 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.822 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:4081 3.18 3.1f231578 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.822 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.563883+0000", 2024-09-21T21:42:31.822 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.001743514, 2024-09-21T21:42:31.822 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0047204380000000004, 2024-09-21T21:42:31.822 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.823 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.823 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.823 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.823 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.823 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.823 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4081 2024-09-21T21:42:31.823 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.823 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.823 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.823 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.823 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563883+0000", 2024-09-21T21:42:31.823 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.823 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.823 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.824 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.824 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563883+0000", 2024-09-21T21:42:31.824 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.824 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.824 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.824 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.824 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563883+0000", 2024-09-21T21:42:31.824 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4499999999999998e-07 2024-09-21T21:42:31.824 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.824 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.824 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.824 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563885+0000", 2024-09-21T21:42:31.824 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.793e-06 2024-09-21T21:42:31.824 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.824 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.825 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.825 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563890+0000", 2024-09-21T21:42:31.825 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5260000000000001e-06 2024-09-21T21:42:31.825 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.825 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.825 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.825 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563895+0000", 2024-09-21T21:42:31.825 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3470000000000001e-06 2024-09-21T21:42:31.825 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.825 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.825 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.825 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.825 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.825 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:4362 3.1a 3.51c7a5a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.825 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.565528+0000", 2024-09-21T21:42:31.826 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.8169000000000001e-05, 2024-09-21T21:42:31.826 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0031016429999999998, 2024-09-21T21:42:31.826 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.826 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.826 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.826 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.826 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.826 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.826 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4362 2024-09-21T21:42:31.826 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.826 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.826 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.826 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.826 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565528+0000", 2024-09-21T21:42:31.826 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.827 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.827 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.827 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.827 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565528+0000", 2024-09-21T21:42:31.827 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.827 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.827 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.827 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.827 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565528+0000", 2024-09-21T21:42:31.827 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4600000000000005e-07 2024-09-21T21:42:31.827 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.827 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.827 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.827 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565531+0000", 2024-09-21T21:42:31.828 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3590000000000002e-06 2024-09-21T21:42:31.828 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.828 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.828 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.828 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565536+0000", 2024-09-21T21:42:31.828 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5740000000000003e-06 2024-09-21T21:42:31.828 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.828 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.828 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.828 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565542+0000", 2024-09-21T21:42:31.828 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0860000000000001e-06 2024-09-21T21:42:31.828 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.828 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.828 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.828 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.829 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.829 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:3989 3.18 3.1f231578 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.829 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.563341+0000", 2024-09-21T21:42:31.829 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.0022848120000000002, 2024-09-21T21:42:31.829 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0053221079999999999, 2024-09-21T21:42:31.829 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.829 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.829 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.829 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.829 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.829 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.829 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3989 2024-09-21T21:42:31.829 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.829 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.829 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.830 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.830 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563341+0000", 2024-09-21T21:42:31.830 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.830 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.830 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.830 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.830 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563341+0000", 2024-09-21T21:42:31.830 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.830 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.830 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.830 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.830 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563342+0000", 2024-09-21T21:42:31.830 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.4999999999999999e-07 2024-09-21T21:42:31.830 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.831 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.831 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.831 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563343+0000", 2024-09-21T21:42:31.831 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.269e-06 2024-09-21T21:42:31.831 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.831 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.831 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.831 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563346+0000", 2024-09-21T21:42:31.831 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0699999999999998e-06 2024-09-21T21:42:31.831 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.831 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.831 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.831 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563348+0000", 2024-09-21T21:42:31.831 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.4990000000000001e-06 2024-09-21T21:42:31.831 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.832 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.832 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.832 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.832 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.832 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:4083 3.18 3.1f231578 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.832 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.563901+0000", 2024-09-21T21:42:31.832 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.00172533, 2024-09-21T21:42:31.832 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0048020199999999997, 2024-09-21T21:42:31.832 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.832 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.832 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.832 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.832 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.832 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.832 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4083 2024-09-21T21:42:31.833 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.833 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.833 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.833 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.833 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563901+0000", 2024-09-21T21:42:31.833 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.833 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.833 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.833 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.833 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563901+0000", 2024-09-21T21:42:31.833 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.833 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.833 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.833 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.833 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563901+0000", 2024-09-21T21:42:31.834 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1699999999999999e-07 2024-09-21T21:42:31.834 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.834 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.834 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.834 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563903+0000", 2024-09-21T21:42:31.834 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.739e-06 2024-09-21T21:42:31.834 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.834 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.834 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.834 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563907+0000", 2024-09-21T21:42:31.834 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4710000000000001e-06 2024-09-21T21:42:31.834 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.834 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.834 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.834 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563910+0000", 2024-09-21T21:42:31.835 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.7130000000000001e-06 2024-09-21T21:42:31.835 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.835 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.835 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.835 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.835 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.835 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:4364 3.1a 3.51c7a5a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.835 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.565548+0000", 2024-09-21T21:42:31.835 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 7.7707999999999995e-05, 2024-09-21T21:42:31.835 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.003182991, 2024-09-21T21:42:31.835 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.835 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.835 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.835 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.835 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.836 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.836 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4364 2024-09-21T21:42:31.836 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.836 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.836 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.836 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.836 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565548+0000", 2024-09-21T21:42:31.836 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.836 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.836 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.836 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.836 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565548+0000", 2024-09-21T21:42:31.836 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.836 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.836 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.836 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.837 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565549+0000", 2024-09-21T21:42:31.837 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7999999999999996e-07 2024-09-21T21:42:31.837 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.837 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.837 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.837 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565550+0000", 2024-09-21T21:42:31.837 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.466e-06 2024-09-21T21:42:31.837 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.837 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.837 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.837 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565554+0000", 2024-09-21T21:42:31.837 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4740000000000001e-06 2024-09-21T21:42:31.837 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.837 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.837 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.837 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565556+0000", 2024-09-21T21:42:31.838 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.5469999999999999e-06 2024-09-21T21:42:31.838 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.838 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.838 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.838 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.838 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.838 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:4084 3.18 3.1f231578 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.838 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.563914+0000", 2024-09-21T21:42:31.838 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.0017125109999999999, 2024-09-21T21:42:31.838 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00484987, 2024-09-21T21:42:31.838 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.838 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.838 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.838 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.839 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.839 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.839 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4084 2024-09-21T21:42:31.839 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.839 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.839 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.839 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.839 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563914+0000", 2024-09-21T21:42:31.839 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.839 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.839 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.839 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.839 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563914+0000", 2024-09-21T21:42:31.839 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.839 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.840 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.840 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.840 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563914+0000", 2024-09-21T21:42:31.840 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8200000000000001e-07 2024-09-21T21:42:31.840 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.840 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.840 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.840 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563915+0000", 2024-09-21T21:42:31.840 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1689999999999999e-06 2024-09-21T21:42:31.840 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.840 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.840 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.840 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563918+0000", 2024-09-21T21:42:31.840 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.9249999999999999e-06 2024-09-21T21:42:31.840 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.841 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.841 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.841 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563921+0000", 2024-09-21T21:42:31.841 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.61e-06 2024-09-21T21:42:31.841 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.841 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.841 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.841 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.841 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.841 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:4167 3.18 3.1f231578 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.841 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.564434+0000", 2024-09-21T21:42:31.841 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.001192193, 2024-09-21T21:42:31.841 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0043747150000000004, 2024-09-21T21:42:31.841 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.841 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.842 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.842 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.842 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.842 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.842 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4167 2024-09-21T21:42:31.842 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.842 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.842 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.842 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.842 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564434+0000", 2024-09-21T21:42:31.842 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.842 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.842 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.842 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.842 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564434+0000", 2024-09-21T21:42:31.843 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.843 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.843 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.843 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.843 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564434+0000", 2024-09-21T21:42:31.843 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.58e-07 2024-09-21T21:42:31.843 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.843 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.843 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.843 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564436+0000", 2024-09-21T21:42:31.843 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0229999999999999e-06 2024-09-21T21:42:31.843 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.843 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.843 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.843 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564442+0000", 2024-09-21T21:42:31.844 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3630000000000002e-06 2024-09-21T21:42:31.844 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.844 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.844 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.844 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564446+0000", 2024-09-21T21:42:31.844 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.605e-06 2024-09-21T21:42:31.844 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.844 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.844 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.844 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.844 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.844 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:4376 3.19 3.d0490659 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.844 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.565561+0000", 2024-09-21T21:42:31.844 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 6.5538999999999997e-05, 2024-09-21T21:42:31.844 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0033002219999999998, 2024-09-21T21:42:31.844 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.844 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.845 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.845 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.845 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.845 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.845 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4376 2024-09-21T21:42:31.845 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.845 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.845 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.845 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.845 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565561+0000", 2024-09-21T21:42:31.845 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.845 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.845 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.845 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.845 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565561+0000", 2024-09-21T21:42:31.846 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.846 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.846 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.846 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.846 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565561+0000", 2024-09-21T21:42:31.846 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.17e-07 2024-09-21T21:42:31.846 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.846 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.846 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.846 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565562+0000", 2024-09-21T21:42:31.846 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1829999999999999e-06 2024-09-21T21:42:31.846 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.846 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.846 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.846 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565565+0000", 2024-09-21T21:42:31.847 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0079999999999998e-06 2024-09-21T21:42:31.847 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.847 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.847 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.847 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565568+0000", 2024-09-21T21:42:31.847 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.5799999999999999e-06 2024-09-21T21:42:31.847 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.847 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.847 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.847 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.847 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.847 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:4086 3.18 3.1f231578 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.847 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.563923+0000", 2024-09-21T21:42:31.847 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.001702629, 2024-09-21T21:42:31.848 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0049997100000000001, 2024-09-21T21:42:31.848 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.848 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.848 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.848 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.848 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.848 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.848 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4086 2024-09-21T21:42:31.848 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.848 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.848 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.848 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.848 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563923+0000", 2024-09-21T21:42:31.848 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.849 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.849 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.849 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.849 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563923+0000", 2024-09-21T21:42:31.849 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.849 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.849 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.849 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.849 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563924+0000", 2024-09-21T21:42:31.849 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.17e-07 2024-09-21T21:42:31.849 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.849 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.849 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.849 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563925+0000", 2024-09-21T21:42:31.849 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.017e-06 2024-09-21T21:42:31.850 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.850 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.850 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.850 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563928+0000", 2024-09-21T21:42:31.850 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.8490000000000002e-06 2024-09-21T21:42:31.850 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.850 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.850 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.850 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563931+0000", 2024-09-21T21:42:31.850 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4209999999999999e-06 2024-09-21T21:42:31.850 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.850 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.850 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.850 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.851 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.851 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:4168 3.18 3.1f231578 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.851 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.564452+0000", 2024-09-21T21:42:31.851 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.0011736260000000001, 2024-09-21T21:42:31.851 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0045296360000000001, 2024-09-21T21:42:31.851 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.851 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.851 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.851 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.851 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.851 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.851 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4168 2024-09-21T21:42:31.851 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.851 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.851 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.852 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.852 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564452+0000", 2024-09-21T21:42:31.852 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.852 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.852 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.852 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.852 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564452+0000", 2024-09-21T21:42:31.852 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.852 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.852 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.852 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.852 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564453+0000", 2024-09-21T21:42:31.852 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.2500000000000001e-07 2024-09-21T21:42:31.852 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.853 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.853 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.853 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564455+0000", 2024-09-21T21:42:31.853 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.7600000000000001e-06 2024-09-21T21:42:31.853 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.853 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.853 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.853 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564458+0000", 2024-09-21T21:42:31.853 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.788e-06 2024-09-21T21:42:31.853 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.853 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.853 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.853 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564461+0000", 2024-09-21T21:42:31.853 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.57e-06 2024-09-21T21:42:31.853 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.854 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.854 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.854 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.854 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.854 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:4377 3.19 3.d0490659 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.854 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.565571+0000", 2024-09-21T21:42:31.854 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 5.5492999999999999e-05, 2024-09-21T21:42:31.854 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0034640029999999998, 2024-09-21T21:42:31.854 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.854 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.854 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.854 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.854 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.854 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.855 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4377 2024-09-21T21:42:31.855 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.855 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.855 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.855 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.855 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565571+0000", 2024-09-21T21:42:31.855 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.855 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.855 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.855 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.855 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565571+0000", 2024-09-21T21:42:31.855 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.855 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.855 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.856 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.856 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565571+0000", 2024-09-21T21:42:31.856 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2000000000000001e-07 2024-09-21T21:42:31.856 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.856 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.856 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.856 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565575+0000", 2024-09-21T21:42:31.856 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3579999999999996e-06 2024-09-21T21:42:31.856 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.856 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.856 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.856 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565578+0000", 2024-09-21T21:42:31.856 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.2739999999999999e-06 2024-09-21T21:42:31.856 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.857 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.857 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.857 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565581+0000", 2024-09-21T21:42:31.857 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.5610000000000001e-06 2024-09-21T21:42:31.857 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.857 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.857 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.857 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.857 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.857 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:4378 3.19 3.d0490659 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.857 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.565584+0000", 2024-09-21T21:42:31.857 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 4.1835999999999997e-05, 2024-09-21T21:42:31.857 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0034804409999999999, 2024-09-21T21:42:31.857 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.858 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.858 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.858 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.858 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.858 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.858 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4378 2024-09-21T21:42:31.858 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.858 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.858 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.858 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.858 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565584+0000", 2024-09-21T21:42:31.858 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.858 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.858 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.858 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.859 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565584+0000", 2024-09-21T21:42:31.859 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.859 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.859 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.859 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.859 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565585+0000", 2024-09-21T21:42:31.859 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.1100000000000002e-07 2024-09-21T21:42:31.859 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.859 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.859 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.859 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565586+0000", 2024-09-21T21:42:31.859 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1939999999999999e-06 2024-09-21T21:42:31.859 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.859 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.860 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.860 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565589+0000", 2024-09-21T21:42:31.860 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0280000000000001e-06 2024-09-21T21:42:31.860 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.860 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.860 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.860 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565591+0000", 2024-09-21T21:42:31.860 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.4540000000000001e-06 2024-09-21T21:42:31.860 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.860 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.860 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.860 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.860 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.860 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:4379 3.19 3.d0490659 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.860 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.565594+0000", 2024-09-21T21:42:31.861 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 3.2054999999999997e-05, 2024-09-21T21:42:31.861 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0034990899999999998, 2024-09-21T21:42:31.861 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.861 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.861 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.861 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.861 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.861 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.861 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4379 2024-09-21T21:42:31.861 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.861 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.861 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.861 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.861 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565594+0000", 2024-09-21T21:42:31.862 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.862 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.862 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.862 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.862 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565594+0000", 2024-09-21T21:42:31.862 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.862 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.862 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.862 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.862 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565594+0000", 2024-09-21T21:42:31.862 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1299999999999999e-07 2024-09-21T21:42:31.862 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.862 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.862 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.863 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565595+0000", 2024-09-21T21:42:31.863 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.096e-06 2024-09-21T21:42:31.863 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.863 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.863 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.863 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565599+0000", 2024-09-21T21:42:31.863 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.2229999999999999e-06 2024-09-21T21:42:31.863 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.863 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.863 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.863 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565601+0000", 2024-09-21T21:42:31.863 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.5670000000000002e-06 2024-09-21T21:42:31.863 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.863 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.863 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.864 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.864 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.864 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:4282 3.19 3:9a60920b:::rbd_data.112159f8e692.0000000000000004:2c [read 3031040~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.864 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.565029+0000", 2024-09-21T21:42:31.864 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.00059686199999999998, 2024-09-21T21:42:31.864 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0040953140000000001, 2024-09-21T21:42:31.864 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.864 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.864 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:31.864 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.864 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.864 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.864 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4282 2024-09-21T21:42:31.864 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.864 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.865 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.865 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.865 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565029+0000", 2024-09-21T21:42:31.865 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.865 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.865 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.865 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.865 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565029+0000", 2024-09-21T21:42:31.865 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.865 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.865 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.865 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.865 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565029+0000", 2024-09-21T21:42:31.865 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.48e-07 2024-09-21T21:42:31.866 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.866 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.866 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.866 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565031+0000", 2024-09-21T21:42:31.866 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.381e-06 2024-09-21T21:42:31.866 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.866 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.866 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.866 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565034+0000", 2024-09-21T21:42:31.866 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5480000000000002e-06 2024-09-21T21:42:31.866 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.866 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.866 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.866 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565037+0000", 2024-09-21T21:42:31.866 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0759999999999999e-06 2024-09-21T21:42:31.867 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.867 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.867 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:31.867 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.569031+0000", 2024-09-21T21:42:31.867 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0039934130000000003 2024-09-21T21:42:31.867 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.867 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.867 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:31.867 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.569096+0000", 2024-09-21T21:42:31.867 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5190000000000004e-05 2024-09-21T21:42:31.867 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.867 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.867 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.867 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.868 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.868 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:4385 3.18 3.1f231578 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.868 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.565605+0000", 2024-09-21T21:42:31.868 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 2.1498999999999999e-05, 2024-09-21T21:42:31.868 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0035535599999999999, 2024-09-21T21:42:31.868 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.868 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.868 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.868 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.868 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.868 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.868 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4385 2024-09-21T21:42:31.868 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.868 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.869 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.869 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.869 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565605+0000", 2024-09-21T21:42:31.869 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.869 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.869 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.869 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.869 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565605+0000", 2024-09-21T21:42:31.869 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.869 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.869 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.869 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.869 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565605+0000", 2024-09-21T21:42:31.869 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.16e-07 2024-09-21T21:42:31.870 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.870 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.870 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.870 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565606+0000", 2024-09-21T21:42:31.870 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1179999999999999e-06 2024-09-21T21:42:31.870 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.870 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.870 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.870 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565609+0000", 2024-09-21T21:42:31.870 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.9790000000000002e-06 2024-09-21T21:42:31.870 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.870 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.870 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.870 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565611+0000", 2024-09-21T21:42:31.870 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.5009999999999999e-06 2024-09-21T21:42:31.871 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.871 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.871 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.871 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.871 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.871 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:4285 3.19 3:9a60920b:::rbd_data.112159f8e692.0000000000000004:2c [read 3072000~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.871 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.565041+0000", 2024-09-21T21:42:31.871 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.00058558500000000001, 2024-09-21T21:42:31.871 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0041696069999999997, 2024-09-21T21:42:31.871 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.871 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.871 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:42:31.871 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.871 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.871 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.872 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4285 2024-09-21T21:42:31.872 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.872 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.872 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.872 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.872 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565041+0000", 2024-09-21T21:42:31.872 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.872 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.872 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.872 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.872 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565041+0000", 2024-09-21T21:42:31.872 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.872 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.872 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.872 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.873 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565041+0000", 2024-09-21T21:42:31.873 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1500000000000001e-07 2024-09-21T21:42:31.873 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.873 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.873 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.873 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565042+0000", 2024-09-21T21:42:31.873 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.17e-06 2024-09-21T21:42:31.873 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.873 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.873 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.873 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565045+0000", 2024-09-21T21:42:31.873 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0450000000000001e-06 2024-09-21T21:42:31.873 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.874 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.874 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.874 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565048+0000", 2024-09-21T21:42:31.874 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.6740000000000001e-06 2024-09-21T21:42:31.874 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.874 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.874 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:31.874 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.569188+0000", 2024-09-21T21:42:31.874 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0041404880000000003 2024-09-21T21:42:31.874 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.874 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.874 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.874 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.874 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.875 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:4387 3.18 3.1f231578 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.875 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.565614+0000", 2024-09-21T21:42:31.875 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 1.1769e-05, 2024-09-21T21:42:31.875 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0036577789999999999, 2024-09-21T21:42:31.875 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.875 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.875 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.875 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.875 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.875 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.875 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4387 2024-09-21T21:42:31.875 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.875 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.875 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.876 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.876 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565614+0000", 2024-09-21T21:42:31.876 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.876 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.876 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.876 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.876 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565614+0000", 2024-09-21T21:42:31.876 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.876 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.876 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.876 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.876 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565615+0000", 2024-09-21T21:42:31.876 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1799999999999999e-07 2024-09-21T21:42:31.877 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.877 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.877 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.877 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565616+0000", 2024-09-21T21:42:31.877 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.186e-06 2024-09-21T21:42:31.877 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.877 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.877 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.877 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565619+0000", 2024-09-21T21:42:31.877 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.9409999999999999e-06 2024-09-21T21:42:31.877 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.877 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.877 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.877 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565621+0000", 2024-09-21T21:42:31.877 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.475e-06 2024-09-21T21:42:31.878 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.878 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.878 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.878 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.878 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.878 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:4289 3.18 3.1f231578 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.878 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.565053+0000", 2024-09-21T21:42:31.878 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.00057320400000000001, 2024-09-21T21:42:31.878 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0042737440000000003, 2024-09-21T21:42:31.878 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.878 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.878 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.878 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.878 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.879 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.879 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4289 2024-09-21T21:42:31.879 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.879 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.879 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.879 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.879 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565053+0000", 2024-09-21T21:42:31.879 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.879 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.879 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.879 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.879 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565053+0000", 2024-09-21T21:42:31.879 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.879 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.880 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.880 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.880 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565053+0000", 2024-09-21T21:42:31.880 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2399999999999999e-07 2024-09-21T21:42:31.880 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.880 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.880 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.880 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565055+0000", 2024-09-21T21:42:31.880 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.781e-06 2024-09-21T21:42:31.880 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.880 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.880 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.880 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565061+0000", 2024-09-21T21:42:31.880 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.536e-06 2024-09-21T21:42:31.880 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.880 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.881 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.881 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565064+0000", 2024-09-21T21:42:31.881 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.411e-06 2024-09-21T21:42:31.881 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.881 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.881 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.881 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.881 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.881 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:4392 3.1a 3.51c7a5a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.881 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.565625+0000", 2024-09-21T21:42:31.881 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 1.2389999999999999e-06, 2024-09-21T21:42:31.881 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0037551630000000002, 2024-09-21T21:42:31.881 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.881 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.882 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.882 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.882 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.882 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.882 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4392 2024-09-21T21:42:31.882 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.882 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.882 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.882 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.882 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565625+0000", 2024-09-21T21:42:31.882 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.882 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.882 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.882 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.882 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565625+0000", 2024-09-21T21:42:31.883 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.883 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.883 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.883 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.883 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565625+0000", 2024-09-21T21:42:31.883 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.36e-07 2024-09-21T21:42:31.883 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.883 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.883 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.883 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565626+0000", 2024-09-21T21:42:31.883 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.15e-06 2024-09-21T21:42:31.883 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.883 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.883 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.884 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565629+0000", 2024-09-21T21:42:31.884 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.9919999999999999e-06 2024-09-21T21:42:31.884 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.884 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.884 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.884 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565632+0000", 2024-09-21T21:42:31.884 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.469e-06 2024-09-21T21:42:31.884 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.884 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.884 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.884 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.884 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.884 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:4291 3.18 3.1f231578 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.884 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.565090+0000", 2024-09-21T21:42:31.885 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.00053636700000000005, 2024-09-21T21:42:31.885 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.004324924, 2024-09-21T21:42:31.885 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.885 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.885 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.885 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.885 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.885 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.885 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4291 2024-09-21T21:42:31.885 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.885 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.885 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.885 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.885 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565090+0000", 2024-09-21T21:42:31.886 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.886 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.886 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.886 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.886 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565090+0000", 2024-09-21T21:42:31.886 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.886 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.886 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.886 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.886 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565090+0000", 2024-09-21T21:42:31.886 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.51e-07 2024-09-21T21:42:31.886 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.886 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.886 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.886 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565092+0000", 2024-09-21T21:42:31.887 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.046e-06 2024-09-21T21:42:31.887 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.887 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.887 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.887 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565098+0000", 2024-09-21T21:42:31.887 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8860000000000001e-06 2024-09-21T21:42:31.887 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.887 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.887 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.887 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565103+0000", 2024-09-21T21:42:31.887 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0390000000000004e-06 2024-09-21T21:42:31.887 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.887 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.887 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.888 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.888 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.888 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:4393 3.1a 3.51c7a5a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.888 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.565635+0000", 2024-09-21T21:42:31.888 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 4294967295.9999914, 2024-09-21T21:42:31.888 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0038233329999999999, 2024-09-21T21:42:31.888 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.888 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.888 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.888 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.888 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.888 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.888 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4393 2024-09-21T21:42:31.888 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.888 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.889 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.889 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.889 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565635+0000", 2024-09-21T21:42:31.889 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.889 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.889 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.889 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.889 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565635+0000", 2024-09-21T21:42:31.889 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.889 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.889 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.889 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.889 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565635+0000", 2024-09-21T21:42:31.889 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7500000000000001e-07 2024-09-21T21:42:31.890 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.890 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.890 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.890 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565636+0000", 2024-09-21T21:42:31.890 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.198e-06 2024-09-21T21:42:31.890 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.890 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.890 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.890 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565640+0000", 2024-09-21T21:42:31.890 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7450000000000001e-06 2024-09-21T21:42:31.890 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.890 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.890 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.890 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565643+0000", 2024-09-21T21:42:31.891 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.101e-06 2024-09-21T21:42:31.891 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.891 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.891 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.891 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.891 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.891 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:4020 3.18 3:1ea8c4f8:::rbd_data.112159f8e692.0000000000000007:2c [read 3096576~8192 out=8192b] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.891 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.563515+0000", 2024-09-21T21:42:31.891 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.0021114269999999999, 2024-09-21T21:42:31.891 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0059823949999999997, 2024-09-21T21:42:31.891 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.891 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.891 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:31.891 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.891 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.892 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.892 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4020 2024-09-21T21:42:31.892 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.892 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.892 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.892 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.892 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563515+0000", 2024-09-21T21:42:31.892 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.892 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.892 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.892 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.892 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563515+0000", 2024-09-21T21:42:31.892 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.892 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.893 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.893 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.893 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563515+0000", 2024-09-21T21:42:31.893 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.9999999999999999e-07 2024-09-21T21:42:31.893 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.893 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.893 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.893 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563517+0000", 2024-09-21T21:42:31.893 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.8300000000000001e-06 2024-09-21T21:42:31.893 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.893 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.893 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.893 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563523+0000", 2024-09-21T21:42:31.893 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3230000000000003e-06 2024-09-21T21:42:31.893 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.894 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.894 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.894 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.563528+0000", 2024-09-21T21:42:31.894 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8180000000000003e-06 2024-09-21T21:42:31.894 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.894 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.894 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:31.894 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.569385+0000", 2024-09-21T21:42:31.894 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0058574289999999999 2024-09-21T21:42:31.894 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.894 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.894 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:31.894 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.569455+0000", 2024-09-21T21:42:31.894 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9503000000000002e-05 2024-09-21T21:42:31.895 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.895 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.895 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.895 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.895 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.895 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:4194 3.18 3.1f231578 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.895 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.564531+0000", 2024-09-21T21:42:31.895 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.0010948030000000001, 2024-09-21T21:42:31.895 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0050307599999999996, 2024-09-21T21:42:31.895 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.895 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.895 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.895 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.895 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.896 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.896 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4194 2024-09-21T21:42:31.896 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.896 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.896 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.896 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.896 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564531+0000", 2024-09-21T21:42:31.896 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.896 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.896 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.896 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.896 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564531+0000", 2024-09-21T21:42:31.896 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.896 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.897 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.897 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.897 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564532+0000", 2024-09-21T21:42:31.897 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.17e-07 2024-09-21T21:42:31.897 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.897 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.897 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.897 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564533+0000", 2024-09-21T21:42:31.897 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0809999999999999e-06 2024-09-21T21:42:31.897 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.897 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.897 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.897 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564535+0000", 2024-09-21T21:42:31.897 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.892e-06 2024-09-21T21:42:31.897 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.898 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.898 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.898 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.564538+0000", 2024-09-21T21:42:31.898 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.7530000000000002e-06 2024-09-21T21:42:31.898 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.898 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.898 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.898 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.898 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.898 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5857.0:4394 3.1a 3.51c7a5a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e110)", 2024-09-21T21:42:31.898 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:31.565646+0000", 2024-09-21T21:42:31.898 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 4294967295.99998, 2024-09-21T21:42:31.898 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0039707650000000002, 2024-09-21T21:42:31.899 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:31.899 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:31.899 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "queued for pg", 2024-09-21T21:42:31.899 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:31.899 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5857", 2024-09-21T21:42:31.899 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/789972164", 2024-09-21T21:42:31.899 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 4394 2024-09-21T21:42:31.899 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.899 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:31.899 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.899 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:31.899 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565646+0000", 2024-09-21T21:42:31.899 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.899 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.899 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.900 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:31.900 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565646+0000", 2024-09-21T21:42:31.900 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:31.900 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.900 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.900 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:31.900 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565646+0000", 2024-09-21T21:42:31.900 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.4299999999999999e-07 2024-09-21T21:42:31.900 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.900 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.900 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:31.900 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565647+0000", 2024-09-21T21:42:31.900 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1429999999999999e-06 2024-09-21T21:42:31.900 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.900 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.901 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:31.901 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565652+0000", 2024-09-21T21:42:31.901 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3239999999999996e-06 2024-09-21T21:42:31.901 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:31.901 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:31.901 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:31.901 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:31.565656+0000", 2024-09-21T21:42:31.901 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4290000000000003e-06 2024-09-21T21:42:31.901 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.901 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:31.901 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.901 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:31.901 INFO:teuthology.orchestra.run.smithi081.stdout: ], 2024-09-21T21:42:31.901 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 99 2024-09-21T21:42:31.901 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:31.903 INFO:teuthology.orchestra.run.smithi164.stdout:5840 write 0x9145200 thru 0x9151ac4 (0xc8c5 bytes) 2024-09-21T21:42:31.903 INFO:teuthology.orchestra.run.smithi164.stdout:5844 clone 44 order 22 su 65536 sc 10 2024-09-21T21:42:31.903 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T21:42:31.904 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:31.901+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:32.004 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:32.003+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:32.006 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:32.006 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:42:32.006 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:42:32.006 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:42:32.006 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:32.015 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T21:42:32.105 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:32.104+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:32.119 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:32.120 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 20, 2024-09-21T21:42:32.120 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 600, 2024-09-21T21:42:32.120 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:42:32.120 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.120 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4824.0:1564 3.2 3:4647520c:::rbd_data.1121b97f2925.0000000000000006:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:42:32.120 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:42:32.120 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 90.481199228999998, 2024-09-21T21:42:32.120 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056097347999999998, 2024-09-21T21:42:32.120 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:32.120 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:32.120 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:32.121 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:32.121 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4824", 2024-09-21T21:42:32.121 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:42:32.121 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1564 2024-09-21T21:42:32.121 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.121 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:32.121 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.121 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:32.121 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:42:32.121 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:32.121 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.121 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.122 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:32.122 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:42:32.122 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:32.122 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.122 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.122 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:32.122 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:42:32.122 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6499999999999999e-07 2024-09-21T21:42:32.122 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.122 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.122 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:32.123 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636067+0000", 2024-09-21T21:42:32.123 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3879999999999998e-06 2024-09-21T21:42:32.123 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.123 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.123 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:32.123 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636075+0000", 2024-09-21T21:42:32.123 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5480000000000004e-06 2024-09-21T21:42:32.123 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.123 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.123 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:32.123 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636080+0000", 2024-09-21T21:42:32.123 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.339e-06 2024-09-21T21:42:32.124 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.124 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.124 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:32.124 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636133+0000", 2024-09-21T21:42:32.124 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2930000000000003e-05 2024-09-21T21:42:32.124 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.124 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.124 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:32.124 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636265+0000", 2024-09-21T21:42:32.124 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000132058 2024-09-21T21:42:32.124 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.124 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.125 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:32.125 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.692162+0000", 2024-09-21T21:42:32.125 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055896620000000001 2024-09-21T21:42:32.125 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:32.125 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:32.125 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:32.125 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.125 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.125 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3260 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 1437696~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:42:32.125 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.196297+0000", 2024-09-21T21:42:32.125 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 44.920967019000003, 2024-09-21T21:42:32.125 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056071022999999998, 2024-09-21T21:42:32.126 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:32.126 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:32.126 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:32.126 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:32.126 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:32.126 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:32.126 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3260 2024-09-21T21:42:32.126 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.126 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:32.126 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.126 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:32.126 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196297+0000", 2024-09-21T21:42:32.127 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:32.127 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.127 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.127 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:32.127 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196297+0000", 2024-09-21T21:42:32.127 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:32.127 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.127 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.127 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:32.127 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196918+0000", 2024-09-21T21:42:32.127 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00062148699999999999 2024-09-21T21:42:32.128 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.128 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.128 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:32.128 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196930+0000", 2024-09-21T21:42:32.128 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1657e-05 2024-09-21T21:42:32.128 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.128 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.128 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:32.128 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196944+0000", 2024-09-21T21:42:32.128 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.4549000000000001e-05 2024-09-21T21:42:32.128 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.128 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.129 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:32.129 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196951+0000", 2024-09-21T21:42:32.129 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6749999999999996e-06 2024-09-21T21:42:32.129 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.129 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.129 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:32.129 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252208+0000", 2024-09-21T21:42:32.129 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055256656000000001 2024-09-21T21:42:32.129 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.129 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.129 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:32.129 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252278+0000", 2024-09-21T21:42:32.130 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9882000000000006e-05 2024-09-21T21:42:32.130 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.130 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.130 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:32.130 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252368+0000", 2024-09-21T21:42:32.130 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.0117000000000005e-05 2024-09-21T21:42:32.130 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:32.130 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:32.130 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:32.130 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.130 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.131 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3263 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 3559424~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:42:32.131 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.197041+0000", 2024-09-21T21:42:32.131 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 44.920223204999999, 2024-09-21T21:42:32.131 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055935804999999998, 2024-09-21T21:42:32.131 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:32.131 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:32.131 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:32.131 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:32.131 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:32.131 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:32.131 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3263 2024-09-21T21:42:32.131 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.132 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:32.132 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.132 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:32.132 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197041+0000", 2024-09-21T21:42:32.132 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:32.132 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.132 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.132 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:32.132 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197041+0000", 2024-09-21T21:42:32.132 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:32.132 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.133 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.133 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:32.133 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197624+0000", 2024-09-21T21:42:32.133 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00058358100000000001 2024-09-21T21:42:32.133 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.133 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.133 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:32.133 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197629+0000", 2024-09-21T21:42:32.133 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5909999999999998e-06 2024-09-21T21:42:32.133 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.133 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.133 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:32.134 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197639+0000", 2024-09-21T21:42:32.134 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0257e-05 2024-09-21T21:42:32.134 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.134 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.134 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:32.134 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197646+0000", 2024-09-21T21:42:32.134 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.7229999999999998e-06 2024-09-21T21:42:32.134 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.134 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.134 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:32.134 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252804+0000", 2024-09-21T21:42:32.134 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055158134999999997 2024-09-21T21:42:32.135 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.135 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.135 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:32.135 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252882+0000", 2024-09-21T21:42:32.135 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7791000000000001e-05 2024-09-21T21:42:32.135 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.135 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.135 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:32.135 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252976+0000", 2024-09-21T21:42:32.135 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.4727000000000003e-05 2024-09-21T21:42:32.135 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:32.136 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:32.136 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:32.136 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.136 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.136 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3437 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 925696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:42:32.136 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.205888+0000", 2024-09-21T21:42:32.136 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 44.911375923999998, 2024-09-21T21:42:32.136 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055953808000000001, 2024-09-21T21:42:32.136 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:32.136 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:32.136 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:32.136 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:32.137 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:32.137 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:32.137 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3437 2024-09-21T21:42:32.137 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.137 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:32.137 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.137 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:32.137 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205888+0000", 2024-09-21T21:42:32.137 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:32.137 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.137 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.137 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:32.137 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205888+0000", 2024-09-21T21:42:32.138 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:32.138 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.138 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.138 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:32.138 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205888+0000", 2024-09-21T21:42:32.138 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.75e-07 2024-09-21T21:42:32.138 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.138 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.138 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:32.138 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205891+0000", 2024-09-21T21:42:32.138 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3750000000000001e-06 2024-09-21T21:42:32.138 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.139 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.139 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:32.139 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205897+0000", 2024-09-21T21:42:32.139 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.285e-06 2024-09-21T21:42:32.139 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.139 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.139 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:32.139 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205902+0000", 2024-09-21T21:42:32.139 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6290000000000001e-06 2024-09-21T21:42:32.139 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.139 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.139 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:32.140 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261686+0000", 2024-09-21T21:42:32.140 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055784586999999997 2024-09-21T21:42:32.140 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.140 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.140 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:32.140 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261764+0000", 2024-09-21T21:42:32.140 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7591999999999998e-05 2024-09-21T21:42:32.140 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.140 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.140 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:32.140 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261842+0000", 2024-09-21T21:42:32.140 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7865000000000001e-05 2024-09-21T21:42:32.140 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:32.141 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:32.141 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:32.141 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.141 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.141 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3438 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 954368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:42:32.141 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.205908+0000", 2024-09-21T21:42:32.141 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 44.911356058999999, 2024-09-21T21:42:32.141 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056083169000000002, 2024-09-21T21:42:32.141 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:32.141 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:32.141 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:32.141 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:32.142 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:32.142 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:32.142 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3438 2024-09-21T21:42:32.142 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.142 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:32.142 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.142 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:32.142 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205908+0000", 2024-09-21T21:42:32.142 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:32.142 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.142 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.142 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:32.143 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205908+0000", 2024-09-21T21:42:32.143 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:32.143 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.143 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.143 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:32.143 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206431+0000", 2024-09-21T21:42:32.143 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00052282100000000003 2024-09-21T21:42:32.143 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.143 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.143 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:32.143 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206442+0000", 2024-09-21T21:42:32.143 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1695000000000001e-05 2024-09-21T21:42:32.144 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.144 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.144 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:32.144 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206452+0000", 2024-09-21T21:42:32.144 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.4560000000000006e-06 2024-09-21T21:42:32.144 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.144 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.144 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:32.144 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206458+0000", 2024-09-21T21:42:32.144 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4670000000000001e-06 2024-09-21T21:42:32.144 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.144 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.145 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:32.145 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261871+0000", 2024-09-21T21:42:32.145 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055412712000000003 2024-09-21T21:42:32.145 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.145 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.145 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:32.145 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261928+0000", 2024-09-21T21:42:32.145 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6631e-05 2024-09-21T21:42:32.145 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.145 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.145 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:32.145 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261991+0000", 2024-09-21T21:42:32.146 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3386999999999995e-05 2024-09-21T21:42:32.146 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:32.146 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:32.146 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:32.146 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.146 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.146 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3444 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 2359296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:42:32.146 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.206533+0000", 2024-09-21T21:42:32.146 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 44.910730538999999, 2024-09-21T21:42:32.146 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055974239000000002, 2024-09-21T21:42:32.146 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:32.146 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:32.146 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:32.147 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:32.147 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:32.147 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:32.147 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3444 2024-09-21T21:42:32.147 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.147 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:32.147 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.147 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:32.147 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206533+0000", 2024-09-21T21:42:32.147 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:32.147 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.147 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.148 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:32.148 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206533+0000", 2024-09-21T21:42:32.148 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:32.148 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.148 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.148 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:32.148 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206710+0000", 2024-09-21T21:42:32.148 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000176293 2024-09-21T21:42:32.148 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.148 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.148 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:32.148 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206714+0000", 2024-09-21T21:42:32.149 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1989999999999999e-06 2024-09-21T21:42:32.149 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.149 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.149 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:32.149 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206723+0000", 2024-09-21T21:42:32.149 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.9199999999999993e-06 2024-09-21T21:42:32.149 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.149 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.149 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:32.149 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206729+0000", 2024-09-21T21:42:32.149 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0739999999999998e-06 2024-09-21T21:42:32.149 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.149 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.150 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:32.150 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262339+0000", 2024-09-21T21:42:32.150 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055610166000000003 2024-09-21T21:42:32.150 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.150 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.150 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:32.150 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262419+0000", 2024-09-21T21:42:32.150 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0432999999999994e-05 2024-09-21T21:42:32.150 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.150 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.150 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:32.150 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262507+0000", 2024-09-21T21:42:32.151 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.8153999999999997e-05 2024-09-21T21:42:32.151 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:32.151 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:32.151 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:32.151 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.151 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.151 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10362 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:32.151 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840386+0000", 2024-09-21T21:42:32.151 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 18.276877695, 2024-09-21T21:42:32.151 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056546515999999998, 2024-09-21T21:42:32.151 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:32.151 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:32.152 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:32.152 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:32.152 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:32.152 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:32.152 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10362 2024-09-21T21:42:32.152 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.152 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:32.152 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.152 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:32.152 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840386+0000", 2024-09-21T21:42:32.152 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:32.152 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.153 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.153 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:32.153 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840386+0000", 2024-09-21T21:42:32.153 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:32.153 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.153 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.153 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:32.153 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840386+0000", 2024-09-21T21:42:32.153 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.01e-07 2024-09-21T21:42:32.153 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.153 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.153 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:32.153 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840389+0000", 2024-09-21T21:42:32.154 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.9739999999999998e-06 2024-09-21T21:42:32.154 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.154 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.154 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:32.154 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840396+0000", 2024-09-21T21:42:32.154 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2280000000000003e-06 2024-09-21T21:42:32.154 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.154 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.154 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:32.154 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840400+0000", 2024-09-21T21:42:32.154 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4429999999999996e-06 2024-09-21T21:42:32.154 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.154 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.155 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:32.155 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.890896+0000", 2024-09-21T21:42:32.155 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.050495717000000002 2024-09-21T21:42:32.155 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.155 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.155 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:32.155 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.891012+0000", 2024-09-21T21:42:32.155 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011602 2024-09-21T21:42:32.155 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.155 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.155 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:32.155 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.891274+0000", 2024-09-21T21:42:32.156 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00026240200000000001 2024-09-21T21:42:32.156 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.156 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.156 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:32.156 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.894810+0000", 2024-09-21T21:42:32.156 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0035353630000000001 2024-09-21T21:42:32.156 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.156 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.156 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:32.156 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.896818+0000", 2024-09-21T21:42:32.156 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.002008681 2024-09-21T21:42:32.156 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.156 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.157 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:32.157 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.896850+0000", 2024-09-21T21:42:32.157 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.1826999999999999e-05 2024-09-21T21:42:32.157 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.157 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.157 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:32.157 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.896933+0000", 2024-09-21T21:42:32.157 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.2460000000000007e-05 2024-09-21T21:42:32.157 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:32.157 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:32.157 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:32.157 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.158 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.158 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10363 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:32.158 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840606+0000", 2024-09-21T21:42:32.158 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 18.276658184999999, 2024-09-21T21:42:32.158 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056546011, 2024-09-21T21:42:32.158 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:32.158 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:32.158 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:32.158 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:32.158 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:32.158 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:32.158 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10363 2024-09-21T21:42:32.159 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.159 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:32.159 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.159 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:32.159 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840606+0000", 2024-09-21T21:42:32.159 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:32.159 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.159 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.159 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:32.159 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840606+0000", 2024-09-21T21:42:32.159 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:32.159 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.159 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.160 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:32.160 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840606+0000", 2024-09-21T21:42:32.160 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.4700000000000001e-07 2024-09-21T21:42:32.160 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.160 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.160 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:32.160 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840611+0000", 2024-09-21T21:42:32.160 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4649999999999996e-06 2024-09-21T21:42:32.160 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.160 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.160 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:32.161 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840620+0000", 2024-09-21T21:42:32.161 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.0350000000000007e-06 2024-09-21T21:42:32.161 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.161 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.161 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:32.161 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840626+0000", 2024-09-21T21:42:32.161 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7180000000000001e-06 2024-09-21T21:42:32.161 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.161 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.161 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:32.161 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.891665+0000", 2024-09-21T21:42:32.161 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.051039924 2024-09-21T21:42:32.162 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.162 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.162 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:32.162 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.891770+0000", 2024-09-21T21:42:32.162 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010445700000000001 2024-09-21T21:42:32.162 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.162 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.162 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:32.162 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.892147+0000", 2024-09-21T21:42:32.162 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00037731800000000002 2024-09-21T21:42:32.162 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.162 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.163 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:32.163 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.896776+0000", 2024-09-21T21:42:32.163 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0046289759999999999 2024-09-21T21:42:32.163 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.163 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.164 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:32.164 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897003+0000", 2024-09-21T21:42:32.164 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00022641299999999999 2024-09-21T21:42:32.164 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.164 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.164 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:32.164 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897064+0000", 2024-09-21T21:42:32.164 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1571999999999999e-05 2024-09-21T21:42:32.164 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.164 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.164 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:32.164 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897152+0000", 2024-09-21T21:42:32.165 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.7386000000000007e-05 2024-09-21T21:42:32.165 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:32.165 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:32.165 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:32.165 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.165 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.165 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10364 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:32.165 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840641+0000", 2024-09-21T21:42:32.165 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 18.276622365000001, 2024-09-21T21:42:32.165 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056671527999999999, 2024-09-21T21:42:32.165 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:32.165 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:32.166 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:32.166 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:32.166 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:32.166 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:32.166 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10364 2024-09-21T21:42:32.166 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.166 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:32.166 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.166 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:32.166 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840641+0000", 2024-09-21T21:42:32.166 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:32.166 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.167 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.167 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:32.167 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840641+0000", 2024-09-21T21:42:32.167 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:32.167 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.167 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.167 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:32.167 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840642+0000", 2024-09-21T21:42:32.167 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7899999999999999e-07 2024-09-21T21:42:32.167 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.167 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.167 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:32.168 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840645+0000", 2024-09-21T21:42:32.168 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0010000000000002e-06 2024-09-21T21:42:32.168 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.168 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.168 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:32.168 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840654+0000", 2024-09-21T21:42:32.168 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.8570000000000001e-06 2024-09-21T21:42:32.168 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.168 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.168 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:32.168 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840659+0000", 2024-09-21T21:42:32.169 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.558e-06 2024-09-21T21:42:32.169 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.169 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.169 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:32.169 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.892656+0000", 2024-09-21T21:42:32.169 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.051996586999999997 2024-09-21T21:42:32.169 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.169 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.169 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:32.169 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.892721+0000", 2024-09-21T21:42:32.169 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5010000000000003e-05 2024-09-21T21:42:32.169 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.169 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.170 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:32.170 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.893112+0000", 2024-09-21T21:42:32.170 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00039133899999999999 2024-09-21T21:42:32.170 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.170 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.170 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:32.170 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.896784+0000", 2024-09-21T21:42:32.170 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0036713679999999999 2024-09-21T21:42:32.170 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.170 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.170 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:32.170 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897204+0000", 2024-09-21T21:42:32.171 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00041991400000000002 2024-09-21T21:42:32.171 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.171 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.171 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:32.171 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897234+0000", 2024-09-21T21:42:32.171 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0307999999999998e-05 2024-09-21T21:42:32.171 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.171 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.171 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:32.171 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897313+0000", 2024-09-21T21:42:32.171 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.9107e-05 2024-09-21T21:42:32.172 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:32.172 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:32.172 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:32.172 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.172 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.172 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10365 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:32.172 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840668+0000", 2024-09-21T21:42:32.172 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 18.276595499999999, 2024-09-21T21:42:32.172 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.058761225, 2024-09-21T21:42:32.172 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:32.172 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:32.172 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:32.173 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:32.173 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:32.173 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:32.173 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10365 2024-09-21T21:42:32.173 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.173 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:32.173 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.173 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:32.173 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840668+0000", 2024-09-21T21:42:32.173 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:32.173 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.173 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.173 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:32.174 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840668+0000", 2024-09-21T21:42:32.174 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:32.174 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.174 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.174 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:32.174 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840669+0000", 2024-09-21T21:42:32.174 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6300000000000001e-07 2024-09-21T21:42:32.174 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.174 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.174 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:32.174 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840672+0000", 2024-09-21T21:42:32.174 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.072e-06 2024-09-21T21:42:32.175 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.175 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.175 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:32.175 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840678+0000", 2024-09-21T21:42:32.175 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.7039999999999997e-06 2024-09-21T21:42:32.175 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.175 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.175 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:32.175 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840683+0000", 2024-09-21T21:42:32.175 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6759999999999998e-06 2024-09-21T21:42:32.175 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.176 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.176 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:32.176 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.893929+0000", 2024-09-21T21:42:32.176 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.053246136999999999 2024-09-21T21:42:32.176 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.176 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.176 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:32.176 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.894082+0000", 2024-09-21T21:42:32.176 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00015301799999999999 2024-09-21T21:42:32.176 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.176 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.176 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:32.176 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.894357+0000", 2024-09-21T21:42:32.177 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00027507999999999998 2024-09-21T21:42:32.177 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.177 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.177 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:32.177 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897161+0000", 2024-09-21T21:42:32.177 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0028036810000000001 2024-09-21T21:42:32.177 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.177 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.177 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:32.177 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.899243+0000", 2024-09-21T21:42:32.177 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0020816379999999998 2024-09-21T21:42:32.177 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.178 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.178 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:32.178 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.899306+0000", 2024-09-21T21:42:32.178 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2964000000000002e-05 2024-09-21T21:42:32.178 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.178 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.178 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:32.178 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.899430+0000", 2024-09-21T21:42:32.178 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00012389200000000001 2024-09-21T21:42:32.178 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:32.178 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:32.178 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:32.179 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.179 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.179 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10366 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:32.179 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840691+0000", 2024-09-21T21:42:32.179 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 18.2765728, 2024-09-21T21:42:32.179 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.061540984999999999, 2024-09-21T21:42:32.179 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:32.179 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:32.179 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:32.179 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:32.179 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:32.179 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:32.180 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10366 2024-09-21T21:42:32.180 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.180 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:32.180 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.180 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:32.180 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840691+0000", 2024-09-21T21:42:32.180 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:32.180 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.180 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.180 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:32.180 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840691+0000", 2024-09-21T21:42:32.180 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:32.180 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.181 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.181 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:32.181 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840691+0000", 2024-09-21T21:42:32.181 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3099999999999999e-07 2024-09-21T21:42:32.181 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.181 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.181 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:32.181 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840694+0000", 2024-09-21T21:42:32.181 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.5780000000000001e-06 2024-09-21T21:42:32.181 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.181 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.181 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:32.182 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840700+0000", 2024-09-21T21:42:32.182 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9429999999999997e-06 2024-09-21T21:42:32.182 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.182 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.182 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:32.182 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840704+0000", 2024-09-21T21:42:32.182 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6380000000000003e-06 2024-09-21T21:42:32.182 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.182 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.182 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:32.182 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.894846+0000", 2024-09-21T21:42:32.182 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.054141392000000003 2024-09-21T21:42:32.182 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.183 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.183 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:32.183 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.894932+0000", 2024-09-21T21:42:32.183 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.5834000000000005e-05 2024-09-21T21:42:32.183 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.183 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.183 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:32.183 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.895302+0000", 2024-09-21T21:42:32.183 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000370737 2024-09-21T21:42:32.183 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.183 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.183 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:32.184 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897166+0000", 2024-09-21T21:42:32.184 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0018632620000000001 2024-09-21T21:42:32.184 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.184 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.184 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:32.184 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902053+0000", 2024-09-21T21:42:32.184 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0048868289999999997 2024-09-21T21:42:32.184 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.184 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.184 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:32.185 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902111+0000", 2024-09-21T21:42:32.185 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7983999999999999e-05 2024-09-21T21:42:32.185 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.185 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.185 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:32.185 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902232+0000", 2024-09-21T21:42:32.185 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000121457 2024-09-21T21:42:32.185 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:32.185 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:32.185 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:32.185 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.185 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.186 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10367 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:32.186 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840842+0000", 2024-09-21T21:42:32.186 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 18.276421992, 2024-09-21T21:42:32.186 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.061549088000000002, 2024-09-21T21:42:32.186 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:32.186 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:32.186 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:32.186 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:32.186 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:32.186 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:32.186 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10367 2024-09-21T21:42:32.186 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.187 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:32.187 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.187 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:32.187 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840842+0000", 2024-09-21T21:42:32.187 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:32.187 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.187 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.187 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:32.187 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840842+0000", 2024-09-21T21:42:32.187 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:32.187 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.187 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.188 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:32.188 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840842+0000", 2024-09-21T21:42:32.188 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3099999999999997e-07 2024-09-21T21:42:32.188 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.188 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.188 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:32.188 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840846+0000", 2024-09-21T21:42:32.188 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.2890000000000002e-06 2024-09-21T21:42:32.188 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.188 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.188 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:32.189 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840855+0000", 2024-09-21T21:42:32.189 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.9619999999999999e-06 2024-09-21T21:42:32.189 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.189 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.189 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:32.189 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840860+0000", 2024-09-21T21:42:32.189 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3759999999999999e-06 2024-09-21T21:42:32.189 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.189 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.189 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:32.189 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.895799+0000", 2024-09-21T21:42:32.189 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.054938569999999999 2024-09-21T21:42:32.190 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.190 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.190 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:32.190 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.895906+0000", 2024-09-21T21:42:32.190 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000107593 2024-09-21T21:42:32.190 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.190 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.190 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:32.190 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.896356+0000", 2024-09-21T21:42:32.190 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00044941700000000002 2024-09-21T21:42:32.190 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.190 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.191 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:32.191 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.898131+0000", 2024-09-21T21:42:32.191 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.001775064 2024-09-21T21:42:32.191 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.191 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.191 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:32.191 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902283+0000", 2024-09-21T21:42:32.191 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0041524839999999997 2024-09-21T21:42:32.191 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.191 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.191 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:32.191 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902316+0000", 2024-09-21T21:42:32.192 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3002e-05 2024-09-21T21:42:32.192 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.192 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.192 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:32.192 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902391+0000", 2024-09-21T21:42:32.192 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.47e-05 2024-09-21T21:42:32.192 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:32.192 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:32.192 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:32.192 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.192 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.192 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10368 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:32.193 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840873+0000", 2024-09-21T21:42:32.193 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 18.276391038, 2024-09-21T21:42:32.193 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.063395291000000006, 2024-09-21T21:42:32.193 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:32.193 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:32.193 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:32.193 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:32.193 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:32.193 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:32.193 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10368 2024-09-21T21:42:32.193 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.193 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:32.194 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.194 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:32.194 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840873+0000", 2024-09-21T21:42:32.194 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:32.194 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.194 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.194 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:32.194 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840873+0000", 2024-09-21T21:42:32.194 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:32.194 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.194 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.194 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:32.195 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840873+0000", 2024-09-21T21:42:32.195 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6699999999999999e-07 2024-09-21T21:42:32.195 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.195 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.195 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:32.195 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840876+0000", 2024-09-21T21:42:32.195 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.6410000000000002e-06 2024-09-21T21:42:32.195 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.195 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.195 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:32.195 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840883+0000", 2024-09-21T21:42:32.195 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0389999999999999e-06 2024-09-21T21:42:32.195 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.196 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.196 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:32.196 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840888+0000", 2024-09-21T21:42:32.196 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8679999999999998e-06 2024-09-21T21:42:32.196 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.196 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.196 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:32.196 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897345+0000", 2024-09-21T21:42:32.196 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056457734000000002 2024-09-21T21:42:32.196 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.197 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.197 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:32.197 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897407+0000", 2024-09-21T21:42:32.197 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1822000000000005e-05 2024-09-21T21:42:32.197 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.197 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.197 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:32.197 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897664+0000", 2024-09-21T21:42:32.197 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00025638499999999998 2024-09-21T21:42:32.197 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.197 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.197 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:32.197 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.901136+0000", 2024-09-21T21:42:32.198 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0034728070000000001 2024-09-21T21:42:32.198 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.198 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.198 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:32.198 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904090+0000", 2024-09-21T21:42:32.198 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0029535799999999999 2024-09-21T21:42:32.198 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.198 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.198 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:32.198 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904144+0000", 2024-09-21T21:42:32.198 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4369000000000001e-05 2024-09-21T21:42:32.199 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.199 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.199 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:32.199 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904268+0000", 2024-09-21T21:42:32.199 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00012367899999999999 2024-09-21T21:42:32.199 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:32.199 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:32.199 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:32.199 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.199 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.199 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10369 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:32.200 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840902+0000", 2024-09-21T21:42:32.200 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 18.276361385000001, 2024-09-21T21:42:32.200 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.063538938000000003, 2024-09-21T21:42:32.200 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:32.200 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:32.200 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:32.200 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:32.200 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:32.200 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:32.200 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10369 2024-09-21T21:42:32.200 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.200 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:32.200 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.200 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:32.201 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840902+0000", 2024-09-21T21:42:32.201 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:32.201 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.201 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.201 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:32.201 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840902+0000", 2024-09-21T21:42:32.201 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:32.201 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.201 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.201 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:32.201 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840903+0000", 2024-09-21T21:42:32.201 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2100000000000002e-07 2024-09-21T21:42:32.202 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.202 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.202 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:32.202 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840913+0000", 2024-09-21T21:42:32.202 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.048e-05 2024-09-21T21:42:32.202 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.202 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.202 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:32.202 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840920+0000", 2024-09-21T21:42:32.202 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0330000000000003e-06 2024-09-21T21:42:32.202 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.202 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.203 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:32.203 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840925+0000", 2024-09-21T21:42:32.203 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0170000000000004e-06 2024-09-21T21:42:32.203 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.203 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.203 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:32.203 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.898179+0000", 2024-09-21T21:42:32.203 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.057253671999999999 2024-09-21T21:42:32.203 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.203 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.203 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:32.204 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.898284+0000", 2024-09-21T21:42:32.204 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000105202 2024-09-21T21:42:32.204 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.204 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.204 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:32.204 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.898729+0000", 2024-09-21T21:42:32.204 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00044459300000000002 2024-09-21T21:42:32.204 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.204 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.204 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:32.204 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902246+0000", 2024-09-21T21:42:32.204 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0035171629999999998 2024-09-21T21:42:32.205 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.205 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.205 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:32.205 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904310+0000", 2024-09-21T21:42:32.205 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.002063765 2024-09-21T21:42:32.205 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.205 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.205 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:32.205 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904342+0000", 2024-09-21T21:42:32.205 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.2091999999999997e-05 2024-09-21T21:42:32.205 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.205 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.205 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:32.206 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904441+0000", 2024-09-21T21:42:32.206 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.9500000000000006e-05 2024-09-21T21:42:32.206 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:32.206 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:32.206 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:32.206 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.206 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.206 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10370 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:32.206 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840944+0000", 2024-09-21T21:42:32.206 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 18.276319672, 2024-09-21T21:42:32.206 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.064436853000000002, 2024-09-21T21:42:32.206 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:32.207 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:32.207 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:32.207 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:32.207 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:32.207 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:32.207 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10370 2024-09-21T21:42:32.207 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.207 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:32.207 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.207 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:32.207 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840944+0000", 2024-09-21T21:42:32.207 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:32.207 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.208 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.208 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:32.208 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840944+0000", 2024-09-21T21:42:32.208 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:32.208 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.208 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.208 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:32.208 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840945+0000", 2024-09-21T21:42:32.208 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2300000000000002e-07 2024-09-21T21:42:32.208 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.208 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.208 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:32.208 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840947+0000", 2024-09-21T21:42:32.208 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.8200000000000001e-06 2024-09-21T21:42:32.209 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.209 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.209 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:32.209 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840955+0000", 2024-09-21T21:42:32.209 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.8660000000000006e-06 2024-09-21T21:42:32.209 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.209 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.209 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:32.209 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840960+0000", 2024-09-21T21:42:32.209 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8670000000000002e-06 2024-09-21T21:42:32.209 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.209 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.209 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:32.210 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.899460+0000", 2024-09-21T21:42:32.210 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.058499922000000003 2024-09-21T21:42:32.210 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.210 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.210 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:32.210 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.899530+0000", 2024-09-21T21:42:32.210 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9558e-05 2024-09-21T21:42:32.210 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.210 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.210 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:32.210 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.899775+0000", 2024-09-21T21:42:32.210 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000245231 2024-09-21T21:42:32.210 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.211 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.211 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:32.211 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902254+0000", 2024-09-21T21:42:32.211 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0024790519999999998 2024-09-21T21:42:32.211 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.211 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.211 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:32.211 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905183+0000", 2024-09-21T21:42:32.211 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.002929156 2024-09-21T21:42:32.211 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.211 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.212 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:32.212 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905243+0000", 2024-09-21T21:42:32.212 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0055e-05 2024-09-21T21:42:32.212 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.212 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.212 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:32.212 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905381+0000", 2024-09-21T21:42:32.212 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00013790300000000001 2024-09-21T21:42:32.212 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:32.212 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:32.212 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:32.212 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.212 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.213 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10371 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:32.213 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840980+0000", 2024-09-21T21:42:32.213 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 18.276284006000001, 2024-09-21T21:42:32.213 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.064554376999999996, 2024-09-21T21:42:32.213 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:32.213 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:32.213 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:32.213 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:32.213 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:32.213 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:32.213 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10371 2024-09-21T21:42:32.214 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.214 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:32.214 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.214 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:32.214 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840980+0000", 2024-09-21T21:42:32.214 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:32.214 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.214 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.214 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:32.214 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840980+0000", 2024-09-21T21:42:32.214 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:32.214 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.215 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.215 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:32.215 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840980+0000", 2024-09-21T21:42:32.215 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8999999999999997e-07 2024-09-21T21:42:32.215 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.215 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.215 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:32.215 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840983+0000", 2024-09-21T21:42:32.215 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.123e-06 2024-09-21T21:42:32.215 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.215 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.215 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:32.215 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840990+0000", 2024-09-21T21:42:32.216 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9129999999999997e-06 2024-09-21T21:42:32.216 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.216 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.216 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:32.216 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840995+0000", 2024-09-21T21:42:32.216 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0989999999999998e-06 2024-09-21T21:42:32.216 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.216 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.216 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:32.216 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.900168+0000", 2024-09-21T21:42:32.216 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.059172739000000002 2024-09-21T21:42:32.216 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.217 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.217 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:32.217 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.900274+0000", 2024-09-21T21:42:32.217 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010546499999999999 2024-09-21T21:42:32.217 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.217 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.217 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:32.217 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.900703+0000", 2024-09-21T21:42:32.217 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00042978200000000003 2024-09-21T21:42:32.217 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.217 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.217 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:32.217 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.903443+0000", 2024-09-21T21:42:32.218 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0027400749999999998 2024-09-21T21:42:32.218 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.218 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.218 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:32.218 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905430+0000", 2024-09-21T21:42:32.218 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0019868109999999998 2024-09-21T21:42:32.218 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.218 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.218 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:32.218 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905464+0000", 2024-09-21T21:42:32.218 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3599000000000001e-05 2024-09-21T21:42:32.218 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.219 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.219 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:32.219 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905534+0000", 2024-09-21T21:42:32.219 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0281e-05 2024-09-21T21:42:32.219 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:32.219 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:32.219 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:32.219 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.219 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.219 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10372 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:32.219 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.841004+0000", 2024-09-21T21:42:32.220 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 18.276260117, 2024-09-21T21:42:32.220 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.064757812999999997, 2024-09-21T21:42:32.220 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:32.220 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:32.220 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:32.220 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:32.220 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:32.220 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:32.220 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10372 2024-09-21T21:42:32.220 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.220 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:32.220 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.221 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:32.221 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841004+0000", 2024-09-21T21:42:32.221 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:32.221 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.221 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.221 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:32.221 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841004+0000", 2024-09-21T21:42:32.221 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:32.221 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.221 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.221 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:32.221 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841004+0000", 2024-09-21T21:42:32.222 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8500000000000002e-07 2024-09-21T21:42:32.222 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.222 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.222 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:32.222 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841007+0000", 2024-09-21T21:42:32.222 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.6390000000000001e-06 2024-09-21T21:42:32.222 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.222 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.222 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:32.222 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841013+0000", 2024-09-21T21:42:32.222 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0070000000000003e-06 2024-09-21T21:42:32.222 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.223 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.223 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:32.223 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841018+0000", 2024-09-21T21:42:32.223 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9119999999999997e-06 2024-09-21T21:42:32.223 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.223 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.223 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:32.223 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.901182+0000", 2024-09-21T21:42:32.223 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.060164444999999997 2024-09-21T21:42:32.223 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.223 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.224 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:32.224 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.901297+0000", 2024-09-21T21:42:32.224 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011468699999999999 2024-09-21T21:42:32.224 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.224 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.224 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:32.224 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.901559+0000", 2024-09-21T21:42:32.224 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000261764 2024-09-21T21:42:32.224 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.224 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.224 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:32.224 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904281+0000", 2024-09-21T21:42:32.225 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0027229329999999999 2024-09-21T21:42:32.225 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.225 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.225 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:32.225 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905588+0000", 2024-09-21T21:42:32.225 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.001306438 2024-09-21T21:42:32.225 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.225 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.225 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:32.225 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905644+0000", 2024-09-21T21:42:32.225 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5747e-05 2024-09-21T21:42:32.225 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.226 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.226 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:32.226 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905761+0000", 2024-09-21T21:42:32.226 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000117856 2024-09-21T21:42:32.226 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:32.226 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:32.226 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:32.226 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.226 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.226 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10373 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:32.226 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.841027+0000", 2024-09-21T21:42:32.226 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 18.276237033000001, 2024-09-21T21:42:32.227 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.066941839000000003, 2024-09-21T21:42:32.227 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:32.227 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:32.227 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:32.227 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:32.227 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:32.227 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:32.227 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10373 2024-09-21T21:42:32.227 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.227 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:32.227 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.227 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:32.228 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841027+0000", 2024-09-21T21:42:32.228 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:32.228 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.228 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.228 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:32.228 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841027+0000", 2024-09-21T21:42:32.228 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:32.228 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.228 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.228 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:32.228 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841027+0000", 2024-09-21T21:42:32.228 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6199999999999999e-07 2024-09-21T21:42:32.228 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.229 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.229 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:32.229 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841030+0000", 2024-09-21T21:42:32.229 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.729e-06 2024-09-21T21:42:32.229 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.229 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.229 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:32.229 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841036+0000", 2024-09-21T21:42:32.229 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1979999999999998e-06 2024-09-21T21:42:32.229 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.229 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.229 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:32.230 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841042+0000", 2024-09-21T21:42:32.230 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4289999999999998e-06 2024-09-21T21:42:32.230 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.230 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.230 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:32.230 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902433+0000", 2024-09-21T21:42:32.230 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.061390329 2024-09-21T21:42:32.230 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.230 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.230 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:32.230 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902541+0000", 2024-09-21T21:42:32.230 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000108349 2024-09-21T21:42:32.231 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.231 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.231 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:32.231 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902983+0000", 2024-09-21T21:42:32.231 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00044158000000000001 2024-09-21T21:42:32.231 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.231 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.231 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:32.231 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904490+0000", 2024-09-21T21:42:32.231 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0015074489999999999 2024-09-21T21:42:32.231 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.231 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.232 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:32.232 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.907671+0000", 2024-09-21T21:42:32.232 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.003180511 2024-09-21T21:42:32.232 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.232 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.232 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:32.232 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.907767+0000", 2024-09-21T21:42:32.232 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6379000000000007e-05 2024-09-21T21:42:32.232 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.232 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.232 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:32.232 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.907969+0000", 2024-09-21T21:42:32.233 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00020152399999999999 2024-09-21T21:42:32.233 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:32.233 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:32.233 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:32.233 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.233 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.233 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:12335 3.1a 3:5b387735:::rbd_data.1121faebdfe3.0000000000000005:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:42:32.233 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:14.411635+0000", 2024-09-21T21:42:32.233 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 17.705629155, 2024-09-21T21:42:32.233 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.005973491, 2024-09-21T21:42:32.233 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:32.233 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:32.234 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:32.234 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:32.234 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:32.234 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:32.234 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 12335 2024-09-21T21:42:32.234 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.234 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:32.234 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.234 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:32.234 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411635+0000", 2024-09-21T21:42:32.234 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:32.234 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.235 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.235 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:32.235 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411635+0000", 2024-09-21T21:42:32.235 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:32.235 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.235 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.235 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:32.235 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411635+0000", 2024-09-21T21:42:32.235 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8500000000000001e-07 2024-09-21T21:42:32.235 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.235 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.235 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:32.236 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411639+0000", 2024-09-21T21:42:32.236 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.574e-06 2024-09-21T21:42:32.236 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.236 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.236 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:32.236 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411645+0000", 2024-09-21T21:42:32.236 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6869999999999997e-06 2024-09-21T21:42:32.236 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.236 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.236 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:32.236 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411649+0000", 2024-09-21T21:42:32.236 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.771e-06 2024-09-21T21:42:32.237 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.237 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.237 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:32.237 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411695+0000", 2024-09-21T21:42:32.237 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6e-05 2024-09-21T21:42:32.237 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.237 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.237 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op must wait for map", 2024-09-21T21:42:32.237 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411702+0000", 2024-09-21T21:42:32.237 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5590000000000002e-06 2024-09-21T21:42:32.237 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.237 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.238 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:32.238 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.417222+0000", 2024-09-21T21:42:32.238 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0055198219999999997 2024-09-21T21:42:32.238 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.238 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.238 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for peered", 2024-09-21T21:42:32.238 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.417230+0000", 2024-09-21T21:42:32.238 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.123e-06 2024-09-21T21:42:32.238 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.238 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.238 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:32.238 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.417348+0000", 2024-09-21T21:42:32.238 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.000118029 2024-09-21T21:42:32.239 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.239 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.239 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:32.239 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.417451+0000", 2024-09-21T21:42:32.239 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000103674 2024-09-21T21:42:32.239 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.239 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.239 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:32.239 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.417608+0000", 2024-09-21T21:42:32.239 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00015666700000000001 2024-09-21T21:42:32.239 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:32.239 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:32.240 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:32.240 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.240 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.240 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:12349 3.1a 3:5b387735:::rbd_data.1121faebdfe3.0000000000000005:head [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:42:32.240 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:14.411833+0000", 2024-09-21T21:42:32.240 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 17.705430342, 2024-09-21T21:42:32.240 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0065064500000001, 2024-09-21T21:42:32.240 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:32.240 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:32.240 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:32.240 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:32.240 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:32.241 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:32.241 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 12349 2024-09-21T21:42:32.241 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.241 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:32.241 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.241 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:32.241 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411833+0000", 2024-09-21T21:42:32.241 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:32.241 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.241 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.241 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:32.241 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411833+0000", 2024-09-21T21:42:32.241 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:32.242 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.242 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.242 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:32.242 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411834+0000", 2024-09-21T21:42:32.242 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.03e-07 2024-09-21T21:42:32.242 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.242 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.242 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:32.242 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411836+0000", 2024-09-21T21:42:32.242 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.4700000000000001e-06 2024-09-21T21:42:32.242 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.242 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.243 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:32.243 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411843+0000", 2024-09-21T21:42:32.243 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4069999999999998e-06 2024-09-21T21:42:32.243 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.243 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.243 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:32.243 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411848+0000", 2024-09-21T21:42:32.243 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1580000000000004e-06 2024-09-21T21:42:32.243 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.243 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.243 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:32.243 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411901+0000", 2024-09-21T21:42:32.243 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2824000000000001e-05 2024-09-21T21:42:32.244 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.244 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.244 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting_for_map not empty", 2024-09-21T21:42:32.244 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411904+0000", 2024-09-21T21:42:32.244 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0639999999999998e-06 2024-09-21T21:42:32.244 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.244 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.244 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:32.244 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.417252+0000", 2024-09-21T21:42:32.244 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0053485060000000003 2024-09-21T21:42:32.244 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.244 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.244 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for peered", 2024-09-21T21:42:32.245 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.417259+0000", 2024-09-21T21:42:32.245 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.1149999999999996e-06 2024-09-21T21:42:32.245 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.245 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.245 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:32.245 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.418160+0000", 2024-09-21T21:42:32.245 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.000901083 2024-09-21T21:42:32.245 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.245 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.245 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:32.245 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.418236+0000", 2024-09-21T21:42:32.245 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5852000000000005e-05 2024-09-21T21:42:32.246 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.246 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.246 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:32.246 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.418340+0000", 2024-09-21T21:42:32.246 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000103568 2024-09-21T21:42:32.246 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:32.246 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:32.246 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:32.246 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:32.246 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:32.246 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:32.248 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T21:42:32.248 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:32.246+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:32.348 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:32.347+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:32.393 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:32.393 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:42:32.393 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 0 2024-09-21T21:42:32.393 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:32.401 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T21:42:32.449 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:32.448+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:32.504 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:32.504 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:42:32.505 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:42:32.505 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:42:32.505 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:32.513 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T21:42:32.549 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:32.548+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:32.554 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T21:42:32.623 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:32.623 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 20, 2024-09-21T21:42:32.623 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 600, 2024-09-21T21:42:32.623 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:42:32.623 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.623 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13296 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 4046848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:32.623 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.796643+0000", 2024-09-21T21:42:32.624 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 41.823957256, 2024-09-21T21:42:32.624 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071682019999999999, 2024-09-21T21:42:32.624 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:32.624 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:32.624 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:32.624 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:32.624 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:32.624 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:32.624 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13296 2024-09-21T21:42:32.624 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.624 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:32.624 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.624 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:32.625 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796643+0000", 2024-09-21T21:42:32.625 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:32.625 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.625 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.625 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:32.625 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796643+0000", 2024-09-21T21:42:32.625 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:32.625 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.625 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.625 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:32.625 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796643+0000", 2024-09-21T21:42:32.625 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.7199999999999998e-07 2024-09-21T21:42:32.625 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.626 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.626 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:32.626 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796647+0000", 2024-09-21T21:42:32.626 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6770000000000001e-06 2024-09-21T21:42:32.626 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.626 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.626 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:32.626 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796656+0000", 2024-09-21T21:42:32.626 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2059999999999996e-06 2024-09-21T21:42:32.626 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.626 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.626 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:32.627 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796664+0000", 2024-09-21T21:42:32.627 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7919999999999993e-06 2024-09-21T21:42:32.627 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.627 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.627 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:32.627 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.868186+0000", 2024-09-21T21:42:32.627 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071522273999999997 2024-09-21T21:42:32.627 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.627 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.627 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:32.627 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.868239+0000", 2024-09-21T21:42:32.628 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2787999999999997e-05 2024-09-21T21:42:32.628 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.628 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.628 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:32.628 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.868325+0000", 2024-09-21T21:42:32.628 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.5611000000000004e-05 2024-09-21T21:42:32.628 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:32.628 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:32.628 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:32.628 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.628 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.628 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13561 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 1728512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:32.629 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.823991+0000", 2024-09-21T21:42:32.629 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 41.796608954999996, 2024-09-21T21:42:32.629 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071654762999999996, 2024-09-21T21:42:32.629 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:32.629 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:32.629 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:32.629 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:32.629 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:32.629 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:32.629 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13561 2024-09-21T21:42:32.629 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.629 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:32.629 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.629 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:32.630 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.823991+0000", 2024-09-21T21:42:32.630 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:32.630 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.630 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.630 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:32.630 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.823991+0000", 2024-09-21T21:42:32.630 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:32.630 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.630 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.630 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:32.630 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.823991+0000", 2024-09-21T21:42:32.630 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9299999999999998e-07 2024-09-21T21:42:32.630 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.631 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.631 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:32.631 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.823994+0000", 2024-09-21T21:42:32.631 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2369999999999999e-06 2024-09-21T21:42:32.631 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.631 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.631 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:32.631 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824000+0000", 2024-09-21T21:42:32.631 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.55e-06 2024-09-21T21:42:32.631 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.631 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.631 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:32.632 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824005+0000", 2024-09-21T21:42:32.632 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4329999999999997e-06 2024-09-21T21:42:32.632 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.632 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.632 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:32.632 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.895360+0000", 2024-09-21T21:42:32.632 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071355426 2024-09-21T21:42:32.632 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.632 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.632 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:32.632 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.895477+0000", 2024-09-21T21:42:32.632 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0001171 2024-09-21T21:42:32.633 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.633 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.633 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:32.633 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.895646+0000", 2024-09-21T21:42:32.633 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000168524 2024-09-21T21:42:32.633 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:32.633 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:32.633 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:32.633 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.633 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.633 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13562 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 1757184~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:32.633 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.824011+0000", 2024-09-21T21:42:32.633 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 41.796589077999997, 2024-09-21T21:42:32.634 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072020453999999998, 2024-09-21T21:42:32.634 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:32.634 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:32.634 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:32.634 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:32.634 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:32.634 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:32.634 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13562 2024-09-21T21:42:32.634 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.634 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:32.634 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.634 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:32.635 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824011+0000", 2024-09-21T21:42:32.635 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:32.635 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.635 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.635 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:32.635 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824011+0000", 2024-09-21T21:42:32.635 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:32.635 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.635 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.635 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:32.635 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824011+0000", 2024-09-21T21:42:32.635 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5600000000000001e-07 2024-09-21T21:42:32.636 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.636 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.636 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:32.636 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824013+0000", 2024-09-21T21:42:32.636 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9800000000000001e-06 2024-09-21T21:42:32.636 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.636 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.636 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:32.636 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824019+0000", 2024-09-21T21:42:32.636 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7339999999999996e-06 2024-09-21T21:42:32.636 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.636 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.637 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:32.637 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824023+0000", 2024-09-21T21:42:32.637 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3680000000000004e-06 2024-09-21T21:42:32.637 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.637 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.637 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:32.637 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.895699+0000", 2024-09-21T21:42:32.637 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071675597999999993 2024-09-21T21:42:32.637 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.637 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.637 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:32.637 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.895830+0000", 2024-09-21T21:42:32.637 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000130858 2024-09-21T21:42:32.638 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.638 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.638 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:32.638 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896031+0000", 2024-09-21T21:42:32.638 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00020155999999999999 2024-09-21T21:42:32.638 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:32.638 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:32.638 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:32.638 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.638 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.638 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13563 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 323584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:32.638 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.824029+0000", 2024-09-21T21:42:32.639 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 41.796570637999999, 2024-09-21T21:42:32.639 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072307563000000005, 2024-09-21T21:42:32.639 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:32.639 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:32.639 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:32.639 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:32.639 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:32.639 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:32.639 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13563 2024-09-21T21:42:32.639 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.639 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:32.639 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.640 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:32.640 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824029+0000", 2024-09-21T21:42:32.640 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:32.640 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.640 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.640 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:32.640 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824029+0000", 2024-09-21T21:42:32.640 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:32.640 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.640 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.640 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:32.640 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824030+0000", 2024-09-21T21:42:32.641 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1699999999999999e-07 2024-09-21T21:42:32.641 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.641 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.641 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:32.641 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824032+0000", 2024-09-21T21:42:32.641 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0320000000000002e-06 2024-09-21T21:42:32.641 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.641 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.641 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:32.641 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824038+0000", 2024-09-21T21:42:32.641 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8769999999999998e-06 2024-09-21T21:42:32.641 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.642 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.642 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:32.642 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824042+0000", 2024-09-21T21:42:32.642 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5650000000000003e-06 2024-09-21T21:42:32.642 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.642 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.642 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:32.642 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896058+0000", 2024-09-21T21:42:32.642 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072015941 2024-09-21T21:42:32.642 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.642 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.642 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:32.643 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896164+0000", 2024-09-21T21:42:32.643 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010593499999999999 2024-09-21T21:42:32.643 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.643 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.643 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:32.643 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896337+0000", 2024-09-21T21:42:32.643 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00017279599999999999 2024-09-21T21:42:32.643 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:32.643 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:32.643 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:32.643 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.643 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.644 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13564 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 983040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:32.644 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.824048+0000", 2024-09-21T21:42:32.644 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 41.796551563000001, 2024-09-21T21:42:32.644 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072491193999999995, 2024-09-21T21:42:32.644 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:32.644 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:32.644 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:32.644 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:32.644 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:32.644 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:32.644 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13564 2024-09-21T21:42:32.644 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.645 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:32.645 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.645 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:32.645 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824048+0000", 2024-09-21T21:42:32.645 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:32.645 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.645 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.645 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:32.645 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824048+0000", 2024-09-21T21:42:32.645 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:32.645 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.646 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.646 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:32.646 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824663+0000", 2024-09-21T21:42:32.646 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00061494299999999996 2024-09-21T21:42:32.646 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.646 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.646 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:32.646 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824669+0000", 2024-09-21T21:42:32.646 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2900000000000002e-06 2024-09-21T21:42:32.646 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.646 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.646 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:32.646 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824679+0000", 2024-09-21T21:42:32.647 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0338e-05 2024-09-21T21:42:32.647 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.647 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.647 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:32.647 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824687+0000", 2024-09-21T21:42:32.647 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.9049999999999997e-06 2024-09-21T21:42:32.647 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.647 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.647 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:32.647 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896381+0000", 2024-09-21T21:42:32.647 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071694305999999999 2024-09-21T21:42:32.647 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.648 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.648 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:32.648 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896445+0000", 2024-09-21T21:42:32.648 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3869999999999997e-05 2024-09-21T21:42:32.648 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.648 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.648 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:32.648 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896540+0000", 2024-09-21T21:42:32.648 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.4542000000000004e-05 2024-09-21T21:42:32.648 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:32.648 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:32.648 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:32.649 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.649 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.649 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13565 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3190784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:32.649 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.824704+0000", 2024-09-21T21:42:32.649 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 41.795895692000002, 2024-09-21T21:42:32.649 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072014745000000005, 2024-09-21T21:42:32.649 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:32.649 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:32.649 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:32.649 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:32.649 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:32.649 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:32.649 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13565 2024-09-21T21:42:32.650 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.650 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:32.650 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.650 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:32.650 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824704+0000", 2024-09-21T21:42:32.650 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:32.650 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.650 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.650 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:32.650 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824704+0000", 2024-09-21T21:42:32.650 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:32.650 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.651 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.651 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:32.651 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824705+0000", 2024-09-21T21:42:32.651 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8999999999999997e-07 2024-09-21T21:42:32.651 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.651 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.651 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:32.651 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824707+0000", 2024-09-21T21:42:32.651 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.7669999999999999e-06 2024-09-21T21:42:32.651 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.651 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.651 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:32.652 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824715+0000", 2024-09-21T21:42:32.652 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7700000000000001e-06 2024-09-21T21:42:32.652 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.652 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.652 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:32.652 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824721+0000", 2024-09-21T21:42:32.652 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9560000000000002e-06 2024-09-21T21:42:32.652 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.652 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.652 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:32.652 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896570+0000", 2024-09-21T21:42:32.652 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071848870999999995 2024-09-21T21:42:32.652 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.653 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.653 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:32.653 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896626+0000", 2024-09-21T21:42:32.653 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5578000000000002e-05 2024-09-21T21:42:32.653 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.653 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.653 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:32.653 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896719+0000", 2024-09-21T21:42:32.653 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.3313000000000006e-05 2024-09-21T21:42:32.653 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:32.653 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:32.653 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:32.653 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.654 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.654 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13566 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3162112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:32.654 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.824729+0000", 2024-09-21T21:42:32.654 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 41.795870805, 2024-09-21T21:42:32.654 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072356907999999998, 2024-09-21T21:42:32.654 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:32.654 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:32.654 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:32.654 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:32.654 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:32.654 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:32.655 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13566 2024-09-21T21:42:32.655 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.655 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:32.655 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.655 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:32.655 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824729+0000", 2024-09-21T21:42:32.655 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:32.655 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.655 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.655 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:32.655 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824729+0000", 2024-09-21T21:42:32.655 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:32.655 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.656 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.656 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:32.656 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825630+0000", 2024-09-21T21:42:32.656 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00090139400000000002 2024-09-21T21:42:32.656 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.656 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.656 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:32.656 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825634+0000", 2024-09-21T21:42:32.656 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4960000000000001e-06 2024-09-21T21:42:32.656 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.656 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.657 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:32.657 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825644+0000", 2024-09-21T21:42:32.657 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.9650000000000003e-06 2024-09-21T21:42:32.657 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.657 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.657 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:32.657 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825649+0000", 2024-09-21T21:42:32.657 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0359999999999997e-06 2024-09-21T21:42:32.657 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.657 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.657 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:32.657 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896757+0000", 2024-09-21T21:42:32.657 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071107717000000001 2024-09-21T21:42:32.658 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.658 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.658 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:32.658 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896878+0000", 2024-09-21T21:42:32.658 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000120834 2024-09-21T21:42:32.658 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.658 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.658 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:32.658 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897086+0000", 2024-09-21T21:42:32.658 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00020846600000000001 2024-09-21T21:42:32.658 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:32.659 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:32.659 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:32.659 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.659 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.659 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13568 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3907584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:32.659 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.825671+0000", 2024-09-21T21:42:32.659 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 41.794928396000003, 2024-09-21T21:42:32.659 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071794595000000003, 2024-09-21T21:42:32.659 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:32.659 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:32.659 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:32.659 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:32.659 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:32.660 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:32.660 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13568 2024-09-21T21:42:32.660 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.660 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:32.660 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.660 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:32.660 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825671+0000", 2024-09-21T21:42:32.660 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:32.660 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.660 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.660 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:32.660 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825671+0000", 2024-09-21T21:42:32.661 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:32.661 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.661 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.661 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:32.661 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825672+0000", 2024-09-21T21:42:32.661 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2999999999999999e-07 2024-09-21T21:42:32.661 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.661 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.661 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:32.661 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825673+0000", 2024-09-21T21:42:32.661 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.246e-06 2024-09-21T21:42:32.661 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.662 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.662 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:32.662 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825693+0000", 2024-09-21T21:42:32.662 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9789999999999999e-05 2024-09-21T21:42:32.662 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.662 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.662 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:32.662 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825697+0000", 2024-09-21T21:42:32.662 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2379999999999999e-06 2024-09-21T21:42:32.662 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.662 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.662 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:32.663 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897284+0000", 2024-09-21T21:42:32.663 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071586710999999997 2024-09-21T21:42:32.663 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.663 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.663 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:32.663 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897337+0000", 2024-09-21T21:42:32.663 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2800000000000003e-05 2024-09-21T21:42:32.663 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.663 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.663 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:32.663 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897466+0000", 2024-09-21T21:42:32.663 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00012957999999999999 2024-09-21T21:42:32.664 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:32.664 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:32.664 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:32.664 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.664 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.664 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13574 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 352256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:32.664 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.825702+0000", 2024-09-21T21:42:32.664 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 41.794898248000003, 2024-09-21T21:42:32.664 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072080377000000001, 2024-09-21T21:42:32.664 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:32.664 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:32.664 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:32.665 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:32.665 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:32.665 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:32.665 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13574 2024-09-21T21:42:32.665 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.665 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:32.665 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.665 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:32.665 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825702+0000", 2024-09-21T21:42:32.665 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:32.665 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.665 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.666 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:32.666 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825702+0000", 2024-09-21T21:42:32.666 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:32.666 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.666 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.666 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:32.666 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825943+0000", 2024-09-21T21:42:32.666 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00024105200000000001 2024-09-21T21:42:32.666 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.666 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.666 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:32.667 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825946+0000", 2024-09-21T21:42:32.667 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.743e-06 2024-09-21T21:42:32.667 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.667 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.667 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:32.667 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825956+0000", 2024-09-21T21:42:32.667 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6609999999999995e-06 2024-09-21T21:42:32.667 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.667 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.667 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:32.667 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825963+0000", 2024-09-21T21:42:32.667 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4609999999999996e-06 2024-09-21T21:42:32.668 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.668 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.668 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:32.668 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897502+0000", 2024-09-21T21:42:32.668 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071539354999999999 2024-09-21T21:42:32.668 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.668 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.668 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:32.668 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897619+0000", 2024-09-21T21:42:32.668 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011665 2024-09-21T21:42:32.668 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.669 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.669 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:32.669 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897782+0000", 2024-09-21T21:42:32.669 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00016345500000000001 2024-09-21T21:42:32.669 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:32.669 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:32.669 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:32.669 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.669 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.669 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13575 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 380928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:32.669 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.825973+0000", 2024-09-21T21:42:32.669 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 41.794626811999997, 2024-09-21T21:42:32.670 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072012723000000001, 2024-09-21T21:42:32.670 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:32.670 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:32.670 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:32.670 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:32.670 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:32.670 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:32.670 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13575 2024-09-21T21:42:32.670 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.670 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:32.670 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.670 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:32.671 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825973+0000", 2024-09-21T21:42:32.671 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:32.671 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.671 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.671 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:32.671 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825973+0000", 2024-09-21T21:42:32.671 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:32.671 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.671 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.671 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:32.671 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826746+0000", 2024-09-21T21:42:32.671 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00077326800000000002 2024-09-21T21:42:32.672 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.672 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.672 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:32.672 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826766+0000", 2024-09-21T21:42:32.672 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9465000000000001e-05 2024-09-21T21:42:32.672 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.672 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.672 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:32.672 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826812+0000", 2024-09-21T21:42:32.672 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5927999999999998e-05 2024-09-21T21:42:32.672 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.672 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.673 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:32.673 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826833+0000", 2024-09-21T21:42:32.673 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1710999999999999e-05 2024-09-21T21:42:32.673 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.673 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.673 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:32.673 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897809+0000", 2024-09-21T21:42:32.673 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.070975229000000001 2024-09-21T21:42:32.673 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.673 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.673 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:32.673 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897864+0000", 2024-09-21T21:42:32.673 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5334999999999998e-05 2024-09-21T21:42:32.674 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.674 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.674 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:32.674 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897986+0000", 2024-09-21T21:42:32.674 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000121787 2024-09-21T21:42:32.674 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:32.674 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:32.674 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:32.674 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.674 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.674 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13577 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 1814528~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:32.674 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.826874+0000", 2024-09-21T21:42:32.675 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 41.793725778000002, 2024-09-21T21:42:32.675 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071639123999999998, 2024-09-21T21:42:32.675 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:32.675 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:32.675 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:32.675 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:32.675 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:32.675 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:32.675 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13577 2024-09-21T21:42:32.675 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.675 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:32.676 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.676 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:32.676 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826874+0000", 2024-09-21T21:42:32.676 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:32.676 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.676 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.676 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:32.676 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826874+0000", 2024-09-21T21:42:32.676 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:32.676 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.676 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.676 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:32.677 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826875+0000", 2024-09-21T21:42:32.677 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.82e-07 2024-09-21T21:42:32.677 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.677 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.677 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:32.677 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826877+0000", 2024-09-21T21:42:32.677 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.295e-06 2024-09-21T21:42:32.677 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.677 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.677 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:32.677 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826883+0000", 2024-09-21T21:42:32.677 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2609999999999999e-06 2024-09-21T21:42:32.678 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.678 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.678 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:32.678 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826889+0000", 2024-09-21T21:42:32.678 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5589999999999996e-06 2024-09-21T21:42:32.678 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.678 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.678 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:32.678 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898301+0000", 2024-09-21T21:42:32.678 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071412188000000001 2024-09-21T21:42:32.678 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.678 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.679 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:32.679 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898416+0000", 2024-09-21T21:42:32.679 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000115538 2024-09-21T21:42:32.679 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.679 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.679 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:32.679 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898513+0000", 2024-09-21T21:42:32.679 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6800999999999997e-05 2024-09-21T21:42:32.679 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:32.679 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:32.679 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:32.680 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.680 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.680 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13578 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 2502656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:32.680 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.826896+0000", 2024-09-21T21:42:32.680 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 41.793704321999996, 2024-09-21T21:42:32.680 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071789411999999997, 2024-09-21T21:42:32.680 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:32.680 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:32.680 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:32.680 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:32.680 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:32.680 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:32.681 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13578 2024-09-21T21:42:32.681 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.681 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:32.681 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.681 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:32.681 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826896+0000", 2024-09-21T21:42:32.681 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:32.681 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.681 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.681 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:32.681 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826896+0000", 2024-09-21T21:42:32.682 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:32.682 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.682 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.682 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:32.682 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827327+0000", 2024-09-21T21:42:32.682 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00043094500000000002 2024-09-21T21:42:32.682 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.682 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.682 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:32.682 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827331+0000", 2024-09-21T21:42:32.682 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.0969999999999999e-06 2024-09-21T21:42:32.682 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.683 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.683 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:32.683 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827341+0000", 2024-09-21T21:42:32.683 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0199e-05 2024-09-21T21:42:32.683 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.683 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.683 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:32.683 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827346+0000", 2024-09-21T21:42:32.683 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3210000000000004e-06 2024-09-21T21:42:32.683 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.683 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.684 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:32.684 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898540+0000", 2024-09-21T21:42:32.684 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071194246000000003 2024-09-21T21:42:32.684 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.684 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.684 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:32.684 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898598+0000", 2024-09-21T21:42:32.684 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7658999999999998e-05 2024-09-21T21:42:32.684 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.684 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.684 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:32.685 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898685+0000", 2024-09-21T21:42:32.685 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.6945000000000002e-05 2024-09-21T21:42:32.685 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:32.685 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:32.685 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:32.685 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.685 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.685 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13579 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 2531328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:32.685 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.827355+0000", 2024-09-21T21:42:32.685 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 41.793244790999999, 2024-09-21T21:42:32.685 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071650933, 2024-09-21T21:42:32.685 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:32.685 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:32.686 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:32.686 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:32.686 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:32.686 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:32.686 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13579 2024-09-21T21:42:32.686 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.686 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:32.686 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.686 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:32.686 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827355+0000", 2024-09-21T21:42:32.686 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:32.686 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.687 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.687 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:32.687 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827355+0000", 2024-09-21T21:42:32.687 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:32.687 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.687 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.687 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:32.687 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827486+0000", 2024-09-21T21:42:32.687 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00013066799999999999 2024-09-21T21:42:32.687 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.687 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.687 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:32.688 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827490+0000", 2024-09-21T21:42:32.688 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.9009999999999996e-06 2024-09-21T21:42:32.688 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.688 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.688 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:32.688 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827499+0000", 2024-09-21T21:42:32.688 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6209999999999999e-06 2024-09-21T21:42:32.688 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.688 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.688 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:32.688 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827506+0000", 2024-09-21T21:42:32.688 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6490000000000002e-06 2024-09-21T21:42:32.689 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.689 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.689 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:32.689 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898713+0000", 2024-09-21T21:42:32.689 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071206937999999997 2024-09-21T21:42:32.689 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.689 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.689 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:32.689 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898811+0000", 2024-09-21T21:42:32.689 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.8159999999999995e-05 2024-09-21T21:42:32.689 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.689 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.690 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:32.690 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899006+0000", 2024-09-21T21:42:32.690 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00019499599999999999 2024-09-21T21:42:32.690 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:32.690 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:32.690 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:32.690 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.690 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.690 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13580 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3219456~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:32.691 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.827522+0000", 2024-09-21T21:42:32.691 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 41.793078360000003, 2024-09-21T21:42:32.691 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071813298999999997, 2024-09-21T21:42:32.691 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:32.691 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:32.691 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:32.691 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:32.691 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:32.691 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:32.691 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13580 2024-09-21T21:42:32.691 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.691 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:32.692 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.692 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:32.692 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827522+0000", 2024-09-21T21:42:32.692 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:32.692 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.692 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.692 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:32.692 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827522+0000", 2024-09-21T21:42:32.692 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:32.692 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.692 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.692 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:32.692 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827898+0000", 2024-09-21T21:42:32.692 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000376876 2024-09-21T21:42:32.693 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.693 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.693 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:32.693 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827903+0000", 2024-09-21T21:42:32.693 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4129999999999999e-06 2024-09-21T21:42:32.693 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.693 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.693 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:32.693 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827911+0000", 2024-09-21T21:42:32.693 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.4540000000000007e-06 2024-09-21T21:42:32.693 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.693 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.694 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:32.694 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827917+0000", 2024-09-21T21:42:32.694 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2110000000000003e-06 2024-09-21T21:42:32.694 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.694 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.694 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:32.694 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899041+0000", 2024-09-21T21:42:32.694 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071123965999999997 2024-09-21T21:42:32.694 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.694 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.694 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:32.694 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899157+0000", 2024-09-21T21:42:32.695 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011573700000000001 2024-09-21T21:42:32.695 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.695 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.695 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:32.695 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899335+0000", 2024-09-21T21:42:32.695 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00017764199999999999 2024-09-21T21:42:32.695 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:32.695 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:32.695 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:32.695 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.695 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.695 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13581 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3248128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:32.696 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.827929+0000", 2024-09-21T21:42:32.696 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 41.792670960999999, 2024-09-21T21:42:32.696 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071730151000000006, 2024-09-21T21:42:32.696 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:32.696 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:32.696 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:32.696 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:32.696 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:32.696 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:32.696 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13581 2024-09-21T21:42:32.696 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.696 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:32.697 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.697 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:32.697 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827929+0000", 2024-09-21T21:42:32.697 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:32.697 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.697 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.697 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:32.697 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827929+0000", 2024-09-21T21:42:32.697 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:32.697 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.697 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.697 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:32.698 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.829943+0000", 2024-09-21T21:42:32.698 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0020143750000000001 2024-09-21T21:42:32.698 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.698 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.698 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:32.698 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.829948+0000", 2024-09-21T21:42:32.698 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9400000000000001e-06 2024-09-21T21:42:32.698 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.698 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.698 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:32.698 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.829960+0000", 2024-09-21T21:42:32.699 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.2123999999999999e-05 2024-09-21T21:42:32.699 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.699 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.699 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:32.699 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.829968+0000", 2024-09-21T21:42:32.699 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7000000000000008e-06 2024-09-21T21:42:32.699 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.699 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.699 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:32.699 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899370+0000", 2024-09-21T21:42:32.699 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.069401672999999997 2024-09-21T21:42:32.699 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.700 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.700 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:32.700 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899481+0000", 2024-09-21T21:42:32.700 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011145300000000001 2024-09-21T21:42:32.700 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.700 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.700 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:32.700 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899659+0000", 2024-09-21T21:42:32.700 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00017788600000000001 2024-09-21T21:42:32.700 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:32.700 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:32.701 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:32.701 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.701 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.701 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13595 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 1871872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:32.701 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.830180+0000", 2024-09-21T21:42:32.701 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 41.790419847000003, 2024-09-21T21:42:32.701 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071809244999999994, 2024-09-21T21:42:32.701 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:32.701 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:32.701 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:32.701 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:32.701 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:32.701 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:32.702 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13595 2024-09-21T21:42:32.702 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.702 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:32.702 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.702 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:32.702 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.830180+0000", 2024-09-21T21:42:32.702 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:32.702 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.702 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.702 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:32.702 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.830180+0000", 2024-09-21T21:42:32.702 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:32.702 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.703 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.703 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:32.703 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.831967+0000", 2024-09-21T21:42:32.703 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0017868350000000001 2024-09-21T21:42:32.703 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.703 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.703 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:32.703 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.831971+0000", 2024-09-21T21:42:32.703 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3950000000000003e-06 2024-09-21T21:42:32.703 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.703 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.703 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:32.704 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.831983+0000", 2024-09-21T21:42:32.704 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1815e-05 2024-09-21T21:42:32.704 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.704 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.704 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:32.704 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.831993+0000", 2024-09-21T21:42:32.704 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6339999999999996e-06 2024-09-21T21:42:32.704 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.704 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.704 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:32.704 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.901693+0000", 2024-09-21T21:42:32.704 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.069700082999999996 2024-09-21T21:42:32.705 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.705 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.705 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:32.705 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.901813+0000", 2024-09-21T21:42:32.705 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011971 2024-09-21T21:42:32.705 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.705 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.705 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:32.705 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.901989+0000", 2024-09-21T21:42:32.705 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000176773 2024-09-21T21:42:32.705 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:32.706 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:32.706 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:32.706 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.706 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.706 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13601 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3334144~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:32.706 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.832130+0000", 2024-09-21T21:42:32.706 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 41.788469974999998, 2024-09-21T21:42:32.706 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071861727, 2024-09-21T21:42:32.706 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:32.706 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:32.706 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:32.706 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:32.707 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:32.707 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:32.707 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13601 2024-09-21T21:42:32.707 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.707 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:32.707 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.707 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:32.707 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.832130+0000", 2024-09-21T21:42:32.707 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:32.707 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.707 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.707 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:32.708 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.832130+0000", 2024-09-21T21:42:32.708 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:32.708 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.708 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.708 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:32.708 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.833939+0000", 2024-09-21T21:42:32.708 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0018094400000000001 2024-09-21T21:42:32.708 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.708 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.708 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:32.708 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.833944+0000", 2024-09-21T21:42:32.708 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5979999999999999e-06 2024-09-21T21:42:32.709 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.709 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.709 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:32.709 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.833953+0000", 2024-09-21T21:42:32.709 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.1460000000000002e-06 2024-09-21T21:42:32.709 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.709 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.709 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:32.709 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.833960+0000", 2024-09-21T21:42:32.709 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.7989999999999996e-06 2024-09-21T21:42:32.709 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.709 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.710 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:32.710 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.903693+0000", 2024-09-21T21:42:32.710 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.069732901 2024-09-21T21:42:32.710 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.710 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.710 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:32.710 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.903814+0000", 2024-09-21T21:42:32.710 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000121585 2024-09-21T21:42:32.710 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.710 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.710 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:32.710 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.903992+0000", 2024-09-21T21:42:32.711 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00017725799999999999 2024-09-21T21:42:32.711 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:32.711 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:32.711 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:32.711 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.711 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.711 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:12342 3.1 3:87417293:::rbd_data.1121faebdfe3.0000000000000004:head [read 4096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:42:32.711 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:14.411725+0000", 2024-09-21T21:42:32.711 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 18.208874908999999, 2024-09-21T21:42:32.711 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0089213029999999, 2024-09-21T21:42:32.711 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:32.711 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:32.712 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:32.712 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:32.712 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:32.712 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:32.712 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 12342 2024-09-21T21:42:32.712 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.712 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:32.712 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.712 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:32.712 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411725+0000", 2024-09-21T21:42:32.712 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:32.712 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.713 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.713 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:32.713 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411725+0000", 2024-09-21T21:42:32.713 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:32.713 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.713 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.713 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:32.713 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411726+0000", 2024-09-21T21:42:32.713 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.4169999999999999e-06 2024-09-21T21:42:32.713 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.713 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.713 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:32.714 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411734+0000", 2024-09-21T21:42:32.714 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.3799999999999996e-06 2024-09-21T21:42:32.714 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.714 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.714 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:32.714 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411750+0000", 2024-09-21T21:42:32.714 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.6429000000000001e-05 2024-09-21T21:42:32.714 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.714 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.714 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:32.714 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411760+0000", 2024-09-21T21:42:32.714 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.4239999999999999e-06 2024-09-21T21:42:32.715 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.715 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.715 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:32.715 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411828+0000", 2024-09-21T21:42:32.715 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.7935999999999995e-05 2024-09-21T21:42:32.715 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.715 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.715 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op must wait for map", 2024-09-21T21:42:32.715 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411837+0000", 2024-09-21T21:42:32.715 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.0100000000000001e-06 2024-09-21T21:42:32.715 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.715 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.716 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:32.716 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.414984+0000", 2024-09-21T21:42:32.716 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0031472050000000001 2024-09-21T21:42:32.716 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.716 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.716 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for peered", 2024-09-21T21:42:32.716 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.414994+0000", 2024-09-21T21:42:32.716 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0291e-05 2024-09-21T21:42:32.716 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.716 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.716 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:32.716 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.420353+0000", 2024-09-21T21:42:32.717 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.005358913 2024-09-21T21:42:32.717 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.717 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.717 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:32.717 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.420489+0000", 2024-09-21T21:42:32.717 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00013592699999999999 2024-09-21T21:42:32.717 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.717 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.717 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:32.717 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.420646+0000", 2024-09-21T21:42:32.717 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00015737100000000001 2024-09-21T21:42:32.717 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:32.717 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:32.718 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:32.718 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.718 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.718 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:12348 3.1 3:87417293:::rbd_data.1121faebdfe3.0000000000000004:head [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:42:32.718 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:14.411793+0000", 2024-09-21T21:42:32.718 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 18.208807362999998, 2024-09-21T21:42:32.718 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.009298384, 2024-09-21T21:42:32.718 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:32.718 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:32.718 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:32.718 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:32.718 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:32.719 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:32.719 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 12348 2024-09-21T21:42:32.719 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.719 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:32.719 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.719 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:32.719 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411793+0000", 2024-09-21T21:42:32.719 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:32.719 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.719 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.719 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:32.719 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411793+0000", 2024-09-21T21:42:32.720 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:32.720 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.720 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.720 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:32.720 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411793+0000", 2024-09-21T21:42:32.720 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0999999999999998e-07 2024-09-21T21:42:32.720 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.720 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.720 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:32.720 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411797+0000", 2024-09-21T21:42:32.720 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.49e-06 2024-09-21T21:42:32.721 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.721 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.721 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:32.721 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411809+0000", 2024-09-21T21:42:32.721 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.2510999999999999e-05 2024-09-21T21:42:32.721 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.721 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.721 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:32.721 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411816+0000", 2024-09-21T21:42:32.721 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6420000000000001e-06 2024-09-21T21:42:32.721 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.721 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.721 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:32.722 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411963+0000", 2024-09-21T21:42:32.722 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00014700999999999999 2024-09-21T21:42:32.722 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.722 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.722 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting_for_map not empty", 2024-09-21T21:42:32.722 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411970+0000", 2024-09-21T21:42:32.722 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9800000000000001e-06 2024-09-21T21:42:32.722 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.722 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.722 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:32.722 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.415018+0000", 2024-09-21T21:42:32.722 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0030482299999999999 2024-09-21T21:42:32.723 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.723 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.723 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for peered", 2024-09-21T21:42:32.723 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.415025+0000", 2024-09-21T21:42:32.723 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.3749999999999997e-06 2024-09-21T21:42:32.723 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.723 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.723 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:32.723 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.420904+0000", 2024-09-21T21:42:32.723 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0058787339999999 2024-09-21T21:42:32.723 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.723 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.723 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:32.724 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.420988+0000", 2024-09-21T21:42:32.724 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.3955999999999994e-05 2024-09-21T21:42:32.724 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.724 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.724 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:32.724 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.421091+0000", 2024-09-21T21:42:32.724 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000102846 2024-09-21T21:42:32.724 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:32.724 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:32.724 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:32.724 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.724 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.724 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4385.0:20922 3.9 3:920c5256:::rbd_id.image_client.0-clone36:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-21T21:42:32.725 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:17.811324+0000", 2024-09-21T21:42:32.725 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 14.809275631, 2024-09-21T21:42:32.725 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.66951882299999999, 2024-09-21T21:42:32.725 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:32.725 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:32.725 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:32.725 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:32.725 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4385", 2024-09-21T21:42:32.725 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:42:32.725 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 20922 2024-09-21T21:42:32.725 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.725 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:32.725 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.726 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:32.726 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:17.811324+0000", 2024-09-21T21:42:32.726 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:32.726 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.726 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.726 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:32.726 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:17.811324+0000", 2024-09-21T21:42:32.726 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:32.726 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.726 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.726 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:32.726 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:17.811326+0000", 2024-09-21T21:42:32.727 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.536e-06 2024-09-21T21:42:32.727 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.727 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.727 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:32.727 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:17.811332+0000", 2024-09-21T21:42:32.727 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6699999999999997e-06 2024-09-21T21:42:32.727 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.727 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.727 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:32.727 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:17.811347+0000", 2024-09-21T21:42:32.727 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.4589999999999999e-05 2024-09-21T21:42:32.727 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.727 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.728 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:32.728 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:17.811359+0000", 2024-09-21T21:42:32.728 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1776e-05 2024-09-21T21:42:32.728 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.728 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.728 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:32.728 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:17.811424+0000", 2024-09-21T21:42:32.728 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4795000000000006e-05 2024-09-21T21:42:32.728 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.728 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.728 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for peered", 2024-09-21T21:42:32.728 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:17.811436+0000", 2024-09-21T21:42:32.729 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.221e-05 2024-09-21T21:42:32.729 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.729 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.729 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:32.729 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:18.480657+0000", 2024-09-21T21:42:32.729 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.66922088999999996 2024-09-21T21:42:32.729 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.729 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.729 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:32.729 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:18.480760+0000", 2024-09-21T21:42:32.729 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000102827 2024-09-21T21:42:32.729 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:32.729 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:32.730 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:32.730 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:18.480843+0000", 2024-09-21T21:42:32.730 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.3529000000000006e-05 2024-09-21T21:42:32.730 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:32.730 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:32.730 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:32.730 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:32.730 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:32.730 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:32.731 INFO:teuthology.orchestra.run.smithi164.stdout:truncating image image_client.0-clone43 from 0x9151c00 (overlap 0x1a61400) to 0x17bde00 2024-09-21T21:42:32.732 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T21:42:32.732 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:32.730+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:32.832 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:32.830+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:32.838 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:32.838 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:42:32.838 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-21T21:42:32.838 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:32.847 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T21:42:32.933 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:32.931+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:32.988 INFO:teuthology.orchestra.run.smithi164.stdout:checking clone #42, image image_client.0-clone42 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent43 2024-09-21T21:42:32.992 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:32.993 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:42:32.993 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:42:32.993 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:42:32.993 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:33.001 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T21:42:33.034 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:33.032+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:33.045 INFO:teuthology.orchestra.run.smithi164.stdout:5845 write 0x1808e00 thru 0x1818ce8 (0xfee9 bytes) 2024-09-21T21:42:33.071 INFO:teuthology.orchestra.run.smithi164.stdout:5846 write 0xddf8800 thru 0xddfc00d (0x380e bytes) 2024-09-21T21:42:33.072 INFO:teuthology.orchestra.run.smithi164.stdout:5848 write 0x59b3800 thru 0x59bf2fe (0xbaff bytes) 2024-09-21T21:42:33.072 INFO:teuthology.orchestra.run.smithi164.stdout:5850 clone 45 order 22 su 262144 sc 12 2024-09-21T21:42:33.109 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:33.110 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-21T21:42:33.110 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-21T21:42:33.110 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:42:33.110 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.110 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19747 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 7905280~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:33.110 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273058+0000", 2024-09-21T21:42:33.110 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 41.833910084000003, 2024-09-21T21:42:33.110 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97338591399999996, 2024-09-21T21:42:33.110 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:33.110 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:33.110 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:33.110 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:33.111 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:33.111 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:33.111 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19747 2024-09-21T21:42:33.111 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.111 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:33.111 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.111 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:33.111 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273058+0000", 2024-09-21T21:42:33.111 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:33.111 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.111 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.111 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:33.111 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273058+0000", 2024-09-21T21:42:33.111 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:33.112 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.112 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.112 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:33.112 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273058+0000", 2024-09-21T21:42:33.112 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5900000000000003e-07 2024-09-21T21:42:33.112 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.112 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.112 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:33.112 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273061+0000", 2024-09-21T21:42:33.112 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3479999999999998e-06 2024-09-21T21:42:33.112 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.112 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.112 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:33.112 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273067+0000", 2024-09-21T21:42:33.112 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1469999999999998e-06 2024-09-21T21:42:33.113 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.113 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.113 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:33.113 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273071+0000", 2024-09-21T21:42:33.113 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7339999999999999e-06 2024-09-21T21:42:33.113 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.113 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.113 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:33.113 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277945+0000", 2024-09-21T21:42:33.113 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.004873474 2024-09-21T21:42:33.113 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.113 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.114 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:33.114 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277948+0000", 2024-09-21T21:42:33.114 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0299999999999998e-06 2024-09-21T21:42:33.114 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.114 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.114 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:33.114 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246269+0000", 2024-09-21T21:42:33.114 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96832060200000003 2024-09-21T21:42:33.114 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.114 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.114 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:33.114 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246362+0000", 2024-09-21T21:42:33.114 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3867000000000001e-05 2024-09-21T21:42:33.114 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.115 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.115 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:33.115 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246444+0000", 2024-09-21T21:42:33.115 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1353000000000005e-05 2024-09-21T21:42:33.115 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:33.115 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:33.115 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:33.115 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.115 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.115 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19755 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 5783552~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:33.115 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273077+0000", 2024-09-21T21:42:33.115 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 41.833890468, 2024-09-21T21:42:33.115 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97350203400000002, 2024-09-21T21:42:33.115 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:33.115 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:33.116 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:33.116 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:33.116 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:33.116 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:33.116 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19755 2024-09-21T21:42:33.116 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.116 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:33.116 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.116 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:33.116 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273077+0000", 2024-09-21T21:42:33.116 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:33.116 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.116 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.116 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:33.117 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273077+0000", 2024-09-21T21:42:33.117 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:33.117 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.117 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.117 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:33.117 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273078+0000", 2024-09-21T21:42:33.117 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5699999999999998e-07 2024-09-21T21:42:33.117 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.117 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.117 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:33.117 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273083+0000", 2024-09-21T21:42:33.117 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8670000000000002e-06 2024-09-21T21:42:33.117 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.117 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.117 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:33.118 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273088+0000", 2024-09-21T21:42:33.118 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8370000000000002e-06 2024-09-21T21:42:33.118 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.118 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.118 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:33.118 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273093+0000", 2024-09-21T21:42:33.118 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5709999999999999e-06 2024-09-21T21:42:33.118 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.118 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.118 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:33.118 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277968+0000", 2024-09-21T21:42:33.118 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048753909999999998 2024-09-21T21:42:33.118 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.118 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.119 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:33.119 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277971+0000", 2024-09-21T21:42:33.119 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0189999999999998e-06 2024-09-21T21:42:33.119 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.119 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.119 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:33.119 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246485+0000", 2024-09-21T21:42:33.119 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96851399299999996 2024-09-21T21:42:33.119 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.119 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.119 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:33.119 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246536+0000", 2024-09-21T21:42:33.119 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0430000000000003e-05 2024-09-21T21:42:33.119 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.120 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.120 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:33.120 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246579+0000", 2024-09-21T21:42:33.120 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3569000000000002e-05 2024-09-21T21:42:33.120 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:33.120 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:33.120 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:33.120 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.120 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.120 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19756 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6500352~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:33.120 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273099+0000", 2024-09-21T21:42:33.120 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 41.833869094999997, 2024-09-21T21:42:33.120 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97359272799999996, 2024-09-21T21:42:33.120 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:33.120 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:33.121 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:33.121 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:33.121 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:33.121 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:33.121 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19756 2024-09-21T21:42:33.121 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.121 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:33.121 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.121 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:33.121 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273099+0000", 2024-09-21T21:42:33.121 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:33.121 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.121 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.121 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:33.122 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273099+0000", 2024-09-21T21:42:33.122 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:33.122 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.122 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.122 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:33.122 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273099+0000", 2024-09-21T21:42:33.122 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0900000000000002e-07 2024-09-21T21:42:33.122 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.122 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.122 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:33.122 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273101+0000", 2024-09-21T21:42:33.122 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0320000000000002e-06 2024-09-21T21:42:33.122 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.123 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.123 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:33.123 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273107+0000", 2024-09-21T21:42:33.123 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2830000000000001e-06 2024-09-21T21:42:33.123 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.123 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.123 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:33.123 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273111+0000", 2024-09-21T21:42:33.123 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2230000000000001e-06 2024-09-21T21:42:33.123 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.123 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.123 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:33.123 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277994+0000", 2024-09-21T21:42:33.124 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048837070000000002 2024-09-21T21:42:33.124 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.124 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.124 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:33.124 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278000+0000", 2024-09-21T21:42:33.124 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3149999999999999e-06 2024-09-21T21:42:33.124 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.124 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.124 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:33.124 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246607+0000", 2024-09-21T21:42:33.124 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.968606839 2024-09-21T21:42:33.124 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.124 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.124 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:33.125 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246651+0000", 2024-09-21T21:42:33.125 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4734999999999998e-05 2024-09-21T21:42:33.125 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.125 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.125 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:33.125 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246692+0000", 2024-09-21T21:42:33.125 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0185000000000002e-05 2024-09-21T21:42:33.125 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:33.125 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:33.125 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:33.125 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.125 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.125 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19757 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 5812224~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:33.126 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273116+0000", 2024-09-21T21:42:33.126 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 41.833851437999996, 2024-09-21T21:42:33.126 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97367813999999997, 2024-09-21T21:42:33.126 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:33.126 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:33.126 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:33.126 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:33.126 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:33.126 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:33.126 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19757 2024-09-21T21:42:33.126 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.126 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:33.126 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.126 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:33.127 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273116+0000", 2024-09-21T21:42:33.127 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:33.127 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.127 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.127 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:33.127 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273116+0000", 2024-09-21T21:42:33.127 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:33.127 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.127 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.127 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:33.127 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273117+0000", 2024-09-21T21:42:33.127 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0999999999999999e-07 2024-09-21T21:42:33.127 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.127 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.128 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:33.128 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273119+0000", 2024-09-21T21:42:33.128 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9970000000000001e-06 2024-09-21T21:42:33.128 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.128 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.128 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:33.128 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273125+0000", 2024-09-21T21:42:33.128 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9540000000000001e-06 2024-09-21T21:42:33.128 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.128 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.128 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:33.128 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273135+0000", 2024-09-21T21:42:33.128 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0597999999999999e-05 2024-09-21T21:42:33.128 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.129 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.129 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:33.129 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278019+0000", 2024-09-21T21:42:33.129 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048834849999999999 2024-09-21T21:42:33.129 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.129 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.129 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:33.129 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278022+0000", 2024-09-21T21:42:33.129 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9560000000000002e-06 2024-09-21T21:42:33.129 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.129 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.129 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:33.129 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246716+0000", 2024-09-21T21:42:33.130 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.968694202 2024-09-21T21:42:33.130 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.130 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.130 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:33.130 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246758+0000", 2024-09-21T21:42:33.130 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1607000000000003e-05 2024-09-21T21:42:33.130 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.130 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.130 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:33.130 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246795+0000", 2024-09-21T21:42:33.130 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6931000000000003e-05 2024-09-21T21:42:33.130 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:33.130 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:33.130 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:33.131 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.131 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.131 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19758 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6529024~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:33.131 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273142+0000", 2024-09-21T21:42:33.131 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 41.833826217000002, 2024-09-21T21:42:33.131 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97375924599999997, 2024-09-21T21:42:33.131 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:33.131 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:33.131 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:33.131 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:33.131 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:33.131 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:33.131 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19758 2024-09-21T21:42:33.131 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.132 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:33.132 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.132 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:33.132 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273142+0000", 2024-09-21T21:42:33.132 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:33.132 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.132 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.132 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:33.132 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273142+0000", 2024-09-21T21:42:33.132 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:33.132 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.132 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.132 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:33.132 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273142+0000", 2024-09-21T21:42:33.133 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0200000000000003e-07 2024-09-21T21:42:33.133 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.133 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.133 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:33.133 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273144+0000", 2024-09-21T21:42:33.133 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1390000000000002e-06 2024-09-21T21:42:33.133 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.133 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.133 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:33.133 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273150+0000", 2024-09-21T21:42:33.133 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6409999999999999e-06 2024-09-21T21:42:33.133 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.133 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.133 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:33.134 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273157+0000", 2024-09-21T21:42:33.134 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1359999999999999e-06 2024-09-21T21:42:33.134 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.134 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.134 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:33.134 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278050+0000", 2024-09-21T21:42:33.134 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048926170000000001 2024-09-21T21:42:33.134 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.134 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.134 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:33.134 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278053+0000", 2024-09-21T21:42:33.134 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4879999999999999e-06 2024-09-21T21:42:33.134 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.134 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.134 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:33.135 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246819+0000", 2024-09-21T21:42:33.135 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96876590399999996 2024-09-21T21:42:33.135 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.135 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.135 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:33.135 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246856+0000", 2024-09-21T21:42:33.135 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7287999999999999e-05 2024-09-21T21:42:33.135 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.135 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.135 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:33.135 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246901+0000", 2024-09-21T21:42:33.135 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4631e-05 2024-09-21T21:42:33.135 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:33.136 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:33.136 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:33.136 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.136 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.136 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19759 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 7933952~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:33.136 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273172+0000", 2024-09-21T21:42:33.136 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 41.833795952999999, 2024-09-21T21:42:33.136 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97384789500000002, 2024-09-21T21:42:33.136 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:33.136 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:33.136 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:33.136 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:33.136 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:33.136 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:33.137 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19759 2024-09-21T21:42:33.137 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.137 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:33.137 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.137 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:33.137 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273172+0000", 2024-09-21T21:42:33.137 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:33.137 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.137 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.137 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:33.137 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273172+0000", 2024-09-21T21:42:33.137 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:33.137 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.137 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.138 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:33.138 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273172+0000", 2024-09-21T21:42:33.138 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6100000000000002e-07 2024-09-21T21:42:33.138 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.138 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.138 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:33.138 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273174+0000", 2024-09-21T21:42:33.138 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.691e-06 2024-09-21T21:42:33.138 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.138 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.138 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:33.138 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273178+0000", 2024-09-21T21:42:33.138 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7579999999999998e-06 2024-09-21T21:42:33.139 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.139 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.139 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:33.139 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273180+0000", 2024-09-21T21:42:33.139 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7199999999999998e-06 2024-09-21T21:42:33.139 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.139 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.139 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:33.139 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278077+0000", 2024-09-21T21:42:33.139 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048969529999999999 2024-09-21T21:42:33.139 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.139 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.139 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:33.139 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278080+0000", 2024-09-21T21:42:33.140 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.066e-06 2024-09-21T21:42:33.140 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.140 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.140 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:33.140 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246925+0000", 2024-09-21T21:42:33.140 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96884490199999995 2024-09-21T21:42:33.140 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.140 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.140 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:33.140 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246974+0000", 2024-09-21T21:42:33.140 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8387999999999998e-05 2024-09-21T21:42:33.140 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.140 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.140 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:33.141 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247020+0000", 2024-09-21T21:42:33.141 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6156000000000003e-05 2024-09-21T21:42:33.141 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:33.141 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:33.141 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:33.141 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.141 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.141 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19760 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 7991296~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:33.141 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273220+0000", 2024-09-21T21:42:33.141 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 41.833747439, 2024-09-21T21:42:33.141 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97403125499999998, 2024-09-21T21:42:33.141 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:33.141 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:33.142 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:33.142 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:33.142 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:33.142 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:33.142 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19760 2024-09-21T21:42:33.142 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.142 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:33.142 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.142 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:33.142 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273220+0000", 2024-09-21T21:42:33.142 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:33.142 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.142 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.143 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:33.143 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273220+0000", 2024-09-21T21:42:33.143 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:33.143 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.143 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.143 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:33.143 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273221+0000", 2024-09-21T21:42:33.143 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.65e-07 2024-09-21T21:42:33.143 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.143 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.143 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:33.143 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273228+0000", 2024-09-21T21:42:33.143 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6040000000000004e-06 2024-09-21T21:42:33.143 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.144 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.144 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:33.144 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273233+0000", 2024-09-21T21:42:33.144 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1569999999999999e-06 2024-09-21T21:42:33.144 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.144 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.144 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:33.144 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273238+0000", 2024-09-21T21:42:33.144 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0050000000000004e-06 2024-09-21T21:42:33.144 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.144 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.144 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:33.144 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278107+0000", 2024-09-21T21:42:33.144 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00486857 2024-09-21T21:42:33.144 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.145 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.145 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:33.145 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278111+0000", 2024-09-21T21:42:33.145 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6389999999999998e-06 2024-09-21T21:42:33.145 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.145 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.145 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:33.145 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247069+0000", 2024-09-21T21:42:33.145 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96895796899999997 2024-09-21T21:42:33.145 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.145 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.145 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:33.145 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247166+0000", 2024-09-21T21:42:33.145 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.7158999999999996e-05 2024-09-21T21:42:33.146 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.146 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.146 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:33.146 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247252+0000", 2024-09-21T21:42:33.146 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5886999999999999e-05 2024-09-21T21:42:33.146 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:33.146 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:33.146 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:33.146 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.146 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.146 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19761 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 7962624~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:33.146 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273246+0000", 2024-09-21T21:42:33.146 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 41.833722379000001, 2024-09-21T21:42:33.146 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97415972200000001, 2024-09-21T21:42:33.147 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:33.147 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:33.147 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:33.147 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:33.147 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:33.147 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:33.147 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19761 2024-09-21T21:42:33.147 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.147 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:33.147 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.147 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:33.147 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273246+0000", 2024-09-21T21:42:33.147 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:33.147 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.147 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.148 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:33.148 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273246+0000", 2024-09-21T21:42:33.148 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:33.148 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.148 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.148 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:33.148 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273246+0000", 2024-09-21T21:42:33.148 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5100000000000001e-07 2024-09-21T21:42:33.148 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.148 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.148 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:33.148 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273248+0000", 2024-09-21T21:42:33.148 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1550000000000001e-06 2024-09-21T21:42:33.148 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.149 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.149 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:33.149 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273254+0000", 2024-09-21T21:42:33.149 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.964e-06 2024-09-21T21:42:33.149 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.149 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.149 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:33.149 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273259+0000", 2024-09-21T21:42:33.149 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5499999999999996e-06 2024-09-21T21:42:33.149 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.149 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.149 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:33.149 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278130+0000", 2024-09-21T21:42:33.149 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048715779999999997 2024-09-21T21:42:33.149 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.150 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.150 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:33.150 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278133+0000", 2024-09-21T21:42:33.150 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8739999999999999e-06 2024-09-21T21:42:33.150 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.150 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.150 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:33.150 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247270+0000", 2024-09-21T21:42:33.150 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.969137099 2024-09-21T21:42:33.150 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.150 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.150 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:33.150 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247338+0000", 2024-09-21T21:42:33.150 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7717000000000002e-05 2024-09-21T21:42:33.150 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.151 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.151 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:33.151 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247405+0000", 2024-09-21T21:42:33.151 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7434000000000005e-05 2024-09-21T21:42:33.151 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:33.151 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:33.151 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:33.151 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.151 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.151 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19769 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 5840896~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:33.151 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273287+0000", 2024-09-21T21:42:33.151 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 41.833681095000003, 2024-09-21T21:42:33.152 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97435390200000005, 2024-09-21T21:42:33.152 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:33.152 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:33.152 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:33.152 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:33.152 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:33.152 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:33.152 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19769 2024-09-21T21:42:33.152 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.152 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:33.152 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.152 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:33.152 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273287+0000", 2024-09-21T21:42:33.152 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:33.152 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.153 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.153 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:33.153 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273287+0000", 2024-09-21T21:42:33.153 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:33.153 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.153 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.153 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:33.153 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273287+0000", 2024-09-21T21:42:33.153 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6699999999999999e-07 2024-09-21T21:42:33.153 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.153 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.153 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:33.153 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273289+0000", 2024-09-21T21:42:33.153 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.708e-06 2024-09-21T21:42:33.154 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.154 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.154 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:33.154 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273296+0000", 2024-09-21T21:42:33.154 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8109999999999997e-06 2024-09-21T21:42:33.154 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.154 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.154 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:33.154 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273299+0000", 2024-09-21T21:42:33.154 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0740000000000001e-06 2024-09-21T21:42:33.154 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.154 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.154 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:33.154 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278159+0000", 2024-09-21T21:42:33.155 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048603240000000001 2024-09-21T21:42:33.155 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.155 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.155 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:33.155 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278162+0000", 2024-09-21T21:42:33.155 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1930000000000002e-06 2024-09-21T21:42:33.155 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.155 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.155 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:33.155 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247444+0000", 2024-09-21T21:42:33.155 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96928199100000001 2024-09-21T21:42:33.155 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.155 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.155 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:33.156 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247533+0000", 2024-09-21T21:42:33.156 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8490000000000001e-05 2024-09-21T21:42:33.156 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.156 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.156 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:33.156 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247641+0000", 2024-09-21T21:42:33.156 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000107944 2024-09-21T21:42:33.156 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:33.156 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:33.156 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:33.156 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.156 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.156 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19770 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 5869568~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:33.156 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273313+0000", 2024-09-21T21:42:33.156 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 41.833654993000003, 2024-09-21T21:42:33.156 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97449038899999996, 2024-09-21T21:42:33.157 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:33.157 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:33.157 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:33.157 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:33.157 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:33.157 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:33.157 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19770 2024-09-21T21:42:33.157 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.157 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:33.157 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.157 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:33.157 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273313+0000", 2024-09-21T21:42:33.157 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:33.157 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.157 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.157 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:33.158 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273313+0000", 2024-09-21T21:42:33.158 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:33.158 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.158 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.158 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:33.158 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273313+0000", 2024-09-21T21:42:33.158 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5699999999999998e-07 2024-09-21T21:42:33.158 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.158 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.158 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:33.158 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273315+0000", 2024-09-21T21:42:33.158 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0099999999999998e-06 2024-09-21T21:42:33.158 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.158 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.159 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:33.159 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273321+0000", 2024-09-21T21:42:33.159 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7529999999999998e-06 2024-09-21T21:42:33.159 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.159 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.159 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:33.159 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273328+0000", 2024-09-21T21:42:33.159 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9009999999999997e-06 2024-09-21T21:42:33.159 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.159 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.159 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:33.159 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278183+0000", 2024-09-21T21:42:33.159 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048547770000000002 2024-09-21T21:42:33.159 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.160 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.160 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:33.160 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278186+0000", 2024-09-21T21:42:33.160 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3100000000000001e-06 2024-09-21T21:42:33.160 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.160 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.160 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:33.160 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247676+0000", 2024-09-21T21:42:33.160 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96949004299999997 2024-09-21T21:42:33.160 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.160 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.160 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:33.160 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247757+0000", 2024-09-21T21:42:33.160 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0479e-05 2024-09-21T21:42:33.160 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.160 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.161 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:33.161 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247803+0000", 2024-09-21T21:42:33.161 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6758999999999997e-05 2024-09-21T21:42:33.161 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:33.161 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:33.161 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:33.161 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.161 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.161 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19771 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6586368~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:33.161 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273335+0000", 2024-09-21T21:42:33.161 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 41.833633206000002, 2024-09-21T21:42:33.161 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97458947699999998, 2024-09-21T21:42:33.161 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:33.162 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:33.162 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:33.162 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:33.162 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:33.162 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:33.162 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19771 2024-09-21T21:42:33.162 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.162 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:33.162 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.162 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:33.162 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273335+0000", 2024-09-21T21:42:33.162 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:33.162 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.162 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.163 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:33.163 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273335+0000", 2024-09-21T21:42:33.163 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:33.163 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.163 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.163 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:33.163 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273335+0000", 2024-09-21T21:42:33.163 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3599999999999999e-07 2024-09-21T21:42:33.163 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.163 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.163 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:33.163 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273337+0000", 2024-09-21T21:42:33.163 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0789999999999999e-06 2024-09-21T21:42:33.164 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.164 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.164 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:33.164 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273343+0000", 2024-09-21T21:42:33.164 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6280000000000002e-06 2024-09-21T21:42:33.164 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.164 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.164 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:33.164 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273346+0000", 2024-09-21T21:42:33.164 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.507e-06 2024-09-21T21:42:33.164 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.164 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.164 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:33.164 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278206+0000", 2024-09-21T21:42:33.164 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048600969999999999 2024-09-21T21:42:33.165 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.165 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.165 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:33.165 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278212+0000", 2024-09-21T21:42:33.165 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2079999999999999e-06 2024-09-21T21:42:33.165 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.165 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.165 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:33.165 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247828+0000", 2024-09-21T21:42:33.165 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.969616547 2024-09-21T21:42:33.165 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.165 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.165 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:33.165 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247881+0000", 2024-09-21T21:42:33.166 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3180999999999998e-05 2024-09-21T21:42:33.166 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.166 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.166 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:33.166 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247924+0000", 2024-09-21T21:42:33.166 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2894e-05 2024-09-21T21:42:33.166 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:33.166 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:33.166 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:33.166 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.166 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.166 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19772 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6557696~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:33.166 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273358+0000", 2024-09-21T21:42:33.166 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 41.833609826, 2024-09-21T21:42:33.167 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97467031800000004, 2024-09-21T21:42:33.167 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:33.167 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:33.167 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:33.167 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:33.167 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:33.167 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:33.167 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19772 2024-09-21T21:42:33.167 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.167 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:33.168 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.168 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:33.168 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273358+0000", 2024-09-21T21:42:33.168 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:33.168 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.168 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.168 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:33.168 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273358+0000", 2024-09-21T21:42:33.168 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:33.168 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.168 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.168 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:33.169 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273358+0000", 2024-09-21T21:42:33.169 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4499999999999998e-07 2024-09-21T21:42:33.169 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.169 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.169 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:33.169 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273360+0000", 2024-09-21T21:42:33.169 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9190000000000002e-06 2024-09-21T21:42:33.169 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.169 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.169 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:33.169 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273366+0000", 2024-09-21T21:42:33.169 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.891e-06 2024-09-21T21:42:33.169 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.169 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.169 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:33.170 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273371+0000", 2024-09-21T21:42:33.170 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1880000000000001e-06 2024-09-21T21:42:33.170 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.170 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.170 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:33.170 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278237+0000", 2024-09-21T21:42:33.170 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.004865472 2024-09-21T21:42:33.170 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.170 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.170 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:33.170 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278240+0000", 2024-09-21T21:42:33.170 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9550000000000001e-06 2024-09-21T21:42:33.170 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.170 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.171 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:33.171 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247949+0000", 2024-09-21T21:42:33.171 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.969709127 2024-09-21T21:42:33.171 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.171 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.171 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:33.171 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247990+0000", 2024-09-21T21:42:33.171 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1202000000000003e-05 2024-09-21T21:42:33.171 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.171 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.171 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:33.171 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248028+0000", 2024-09-21T21:42:33.171 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8219000000000001e-05 2024-09-21T21:42:33.171 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:33.172 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:33.172 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:33.172 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.172 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.172 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19773 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6615040~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:33.172 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273390+0000", 2024-09-21T21:42:33.172 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 41.833577965000003, 2024-09-21T21:42:33.172 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97476086200000001, 2024-09-21T21:42:33.172 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:33.172 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:33.172 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:33.172 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:33.172 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:33.172 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:33.172 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19773 2024-09-21T21:42:33.173 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.173 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:33.173 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.173 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:33.173 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273390+0000", 2024-09-21T21:42:33.173 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:33.173 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.173 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.173 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:33.173 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273390+0000", 2024-09-21T21:42:33.173 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:33.173 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.173 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.173 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:33.173 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273390+0000", 2024-09-21T21:42:33.174 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9999999999999999e-07 2024-09-21T21:42:33.174 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.174 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.174 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:33.174 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273392+0000", 2024-09-21T21:42:33.174 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0640000000000001e-06 2024-09-21T21:42:33.174 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.174 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.174 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:33.174 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273397+0000", 2024-09-21T21:42:33.174 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.113e-06 2024-09-21T21:42:33.174 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.174 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.174 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:33.174 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273401+0000", 2024-09-21T21:42:33.175 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3119999999999998e-06 2024-09-21T21:42:33.175 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.175 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.175 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:33.175 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278271+0000", 2024-09-21T21:42:33.175 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048698880000000002 2024-09-21T21:42:33.175 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.175 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.175 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:33.175 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278278+0000", 2024-09-21T21:42:33.175 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1210000000000001e-06 2024-09-21T21:42:33.175 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.175 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.175 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:33.176 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248060+0000", 2024-09-21T21:42:33.176 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96978224999999996 2024-09-21T21:42:33.176 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.176 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.176 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:33.176 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248111+0000", 2024-09-21T21:42:33.176 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.096e-05 2024-09-21T21:42:33.176 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.176 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.176 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:33.176 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248151+0000", 2024-09-21T21:42:33.176 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9854000000000002e-05 2024-09-21T21:42:33.176 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:33.176 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:33.176 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:33.177 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.177 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.177 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19784 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6643712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:33.177 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273405+0000", 2024-09-21T21:42:33.177 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 41.833563243999997, 2024-09-21T21:42:33.177 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97491798900000004, 2024-09-21T21:42:33.177 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:33.177 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:33.177 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:33.177 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:33.177 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:33.177 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:33.177 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19784 2024-09-21T21:42:33.177 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.177 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:33.178 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.178 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:33.178 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273405+0000", 2024-09-21T21:42:33.178 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:33.178 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.178 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.178 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:33.178 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273405+0000", 2024-09-21T21:42:33.178 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:33.178 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.178 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.178 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:33.178 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273405+0000", 2024-09-21T21:42:33.178 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2100000000000001e-07 2024-09-21T21:42:33.179 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.179 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.179 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:33.179 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273406+0000", 2024-09-21T21:42:33.179 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2049999999999999e-06 2024-09-21T21:42:33.179 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.179 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.179 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:33.179 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273409+0000", 2024-09-21T21:42:33.179 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0599999999999999e-06 2024-09-21T21:42:33.179 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.179 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.179 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:33.179 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273412+0000", 2024-09-21T21:42:33.179 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0730000000000001e-06 2024-09-21T21:42:33.180 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.180 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.180 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:33.180 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278317+0000", 2024-09-21T21:42:33.180 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0049045520000000004 2024-09-21T21:42:33.180 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.180 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.180 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:33.180 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278321+0000", 2024-09-21T21:42:33.180 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6249999999999998e-06 2024-09-21T21:42:33.180 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.180 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.180 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:33.180 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248179+0000", 2024-09-21T21:42:33.181 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96985744500000004 2024-09-21T21:42:33.181 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.181 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.181 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:33.181 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248253+0000", 2024-09-21T21:42:33.181 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3829000000000001e-05 2024-09-21T21:42:33.181 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.181 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.181 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:33.181 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248323+0000", 2024-09-21T21:42:33.181 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9979000000000002e-05 2024-09-21T21:42:33.181 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:33.181 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:33.181 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:33.181 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.182 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.182 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19785 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6672384~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:33.182 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273415+0000", 2024-09-21T21:42:33.182 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 41.833552658000002, 2024-09-21T21:42:33.182 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97500777800000005, 2024-09-21T21:42:33.182 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:33.182 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:33.182 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:33.182 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:33.182 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:33.182 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:33.182 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19785 2024-09-21T21:42:33.182 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.182 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:33.183 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.183 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:33.183 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273415+0000", 2024-09-21T21:42:33.183 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:33.183 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.183 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.183 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:33.183 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273415+0000", 2024-09-21T21:42:33.183 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:33.183 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.183 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.183 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:33.183 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273415+0000", 2024-09-21T21:42:33.183 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0100000000000001e-07 2024-09-21T21:42:33.183 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.184 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.184 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:33.184 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273417+0000", 2024-09-21T21:42:33.184 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.114e-06 2024-09-21T21:42:33.184 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.184 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.184 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:33.184 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273420+0000", 2024-09-21T21:42:33.184 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6770000000000001e-06 2024-09-21T21:42:33.184 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.184 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.184 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:33.184 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273423+0000", 2024-09-21T21:42:33.184 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5450000000000002e-06 2024-09-21T21:42:33.184 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.185 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.185 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:33.185 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278355+0000", 2024-09-21T21:42:33.185 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0049318590000000002 2024-09-21T21:42:33.185 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.185 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.185 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:33.185 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278360+0000", 2024-09-21T21:42:33.185 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3079999999999998e-06 2024-09-21T21:42:33.185 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.185 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.185 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:33.185 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248349+0000", 2024-09-21T21:42:33.186 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96998909899999997 2024-09-21T21:42:33.186 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.186 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.186 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:33.186 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248387+0000", 2024-09-21T21:42:33.186 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7657000000000002e-05 2024-09-21T21:42:33.186 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.186 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.186 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:33.186 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248423+0000", 2024-09-21T21:42:33.186 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6318e-05 2024-09-21T21:42:33.186 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:33.186 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:33.186 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:33.187 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.187 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.187 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19842 3.d 3:b7fa2ca0:::rbd_data.112157a479c.000000000000004d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:33.187 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273426+0000", 2024-09-21T21:42:33.187 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 41.833541605999997, 2024-09-21T21:42:33.187 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.98136526800000001, 2024-09-21T21:42:33.187 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:33.187 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:33.187 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:33.187 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:33.187 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:33.187 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:33.187 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19842 2024-09-21T21:42:33.187 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.188 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:33.188 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.188 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:33.188 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273426+0000", 2024-09-21T21:42:33.188 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:33.188 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.188 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.188 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:33.188 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273426+0000", 2024-09-21T21:42:33.188 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:33.188 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.188 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.188 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:33.188 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273427+0000", 2024-09-21T21:42:33.188 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.72e-07 2024-09-21T21:42:33.189 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.189 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.189 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:33.189 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275836+0000", 2024-09-21T21:42:33.189 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00240909 2024-09-21T21:42:33.189 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.189 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.189 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:33.189 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275844+0000", 2024-09-21T21:42:33.189 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3750000000000003e-06 2024-09-21T21:42:33.189 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.189 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.189 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:33.189 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275849+0000", 2024-09-21T21:42:33.190 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0100000000000003e-06 2024-09-21T21:42:33.190 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.190 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.190 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:33.190 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278394+0000", 2024-09-21T21:42:33.190 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025449890000000001 2024-09-21T21:42:33.190 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.190 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.190 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:33.190 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278402+0000", 2024-09-21T21:42:33.190 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6359999999999994e-06 2024-09-21T21:42:33.190 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.190 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.190 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:33.191 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248454+0000", 2024-09-21T21:42:33.191 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97005259499999996 2024-09-21T21:42:33.191 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.191 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.191 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:33.191 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248576+0000", 2024-09-21T21:42:33.191 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000122127 2024-09-21T21:42:33.191 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.191 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.191 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 2", 2024-09-21T21:42:33.191 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248844+0000", 2024-09-21T21:42:33.191 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00026784699999999999 2024-09-21T21:42:33.191 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.192 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.192 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:33.192 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.252487+0000", 2024-09-21T21:42:33.192 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0036427920000000002 2024-09-21T21:42:33.192 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.192 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.192 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:33.192 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.254691+0000", 2024-09-21T21:42:33.192 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002203899 2024-09-21T21:42:33.192 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.192 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.192 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:33.192 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.254748+0000", 2024-09-21T21:42:33.192 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7163000000000001e-05 2024-09-21T21:42:33.193 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.193 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.193 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:33.193 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.254792+0000", 2024-09-21T21:42:33.193 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3473000000000002e-05 2024-09-21T21:42:33.193 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:33.193 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:33.193 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:33.193 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.193 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.193 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19843 3.d 3:b7fa2ca0:::rbd_data.112157a479c.000000000000004d:head [write 0~0] snapc 21=[21] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:33.193 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.275860+0000", 2024-09-21T21:42:33.193 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 41.831107705999997, 2024-09-21T21:42:33.193 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.98183651999999999, 2024-09-21T21:42:33.194 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:33.194 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:33.194 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:33.194 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:33.194 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:33.194 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:33.194 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19843 2024-09-21T21:42:33.194 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.194 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:33.194 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.194 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:33.194 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275860+0000", 2024-09-21T21:42:33.194 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:33.194 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.195 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.195 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:33.195 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275860+0000", 2024-09-21T21:42:33.195 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:33.195 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.195 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.195 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:33.195 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275869+0000", 2024-09-21T21:42:33.195 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6300000000000004e-06 2024-09-21T21:42:33.195 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.195 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.195 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:33.195 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275871+0000", 2024-09-21T21:42:33.195 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2060000000000001e-06 2024-09-21T21:42:33.196 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.196 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.196 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:33.196 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275876+0000", 2024-09-21T21:42:33.196 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8180000000000003e-06 2024-09-21T21:42:33.196 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.196 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.196 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:33.196 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275880+0000", 2024-09-21T21:42:33.196 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1829999999999996e-06 2024-09-21T21:42:33.196 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.196 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.196 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:33.196 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278416+0000", 2024-09-21T21:42:33.197 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025361149999999998 2024-09-21T21:42:33.197 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.197 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.197 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:33.197 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278419+0000", 2024-09-21T21:42:33.197 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.163e-06 2024-09-21T21:42:33.197 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.197 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.197 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:33.197 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.250249+0000", 2024-09-21T21:42:33.197 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97182984299999997 2024-09-21T21:42:33.197 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.197 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.197 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:33.197 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.250321+0000", 2024-09-21T21:42:33.198 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1656000000000006e-05 2024-09-21T21:42:33.198 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.198 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.198 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 2", 2024-09-21T21:42:33.198 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.250419+0000", 2024-09-21T21:42:33.198 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.8277999999999997e-05 2024-09-21T21:42:33.198 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.198 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.198 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:33.198 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.252516+0000", 2024-09-21T21:42:33.198 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002097224 2024-09-21T21:42:33.198 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.198 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.198 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:33.198 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.257643+0000", 2024-09-21T21:42:33.199 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0051262319999999997 2024-09-21T21:42:33.199 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.199 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.199 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:33.199 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.257673+0000", 2024-09-21T21:42:33.199 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0023e-05 2024-09-21T21:42:33.199 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.199 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.199 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:33.199 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.257697+0000", 2024-09-21T21:42:33.199 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4148999999999999e-05 2024-09-21T21:42:33.199 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:33.199 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:33.200 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:33.200 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.200 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.200 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5626.0:12334 3.3 3:c643a86c:::rbd_data.1121faebdfe3.0000000000000002:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:42:33.200 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:14.411628+0000", 2024-09-21T21:42:33.200 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 18.695339406999999, 2024-09-21T21:42:33.200 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0086947559999999, 2024-09-21T21:42:33.200 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:33.200 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:33.200 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:33.200 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:33.200 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5626", 2024-09-21T21:42:33.200 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:33.200 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 12334 2024-09-21T21:42:33.200 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.201 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:33.201 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.201 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:33.201 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411628+0000", 2024-09-21T21:42:33.201 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:33.201 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.201 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.201 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:33.201 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411628+0000", 2024-09-21T21:42:33.201 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:33.201 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.201 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.201 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:33.201 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411629+0000", 2024-09-21T21:42:33.201 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1200000000000002e-07 2024-09-21T21:42:33.202 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.202 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.202 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:33.202 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411633+0000", 2024-09-21T21:42:33.202 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1699999999999999e-06 2024-09-21T21:42:33.202 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.202 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.202 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:33.202 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411645+0000", 2024-09-21T21:42:33.202 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1328999999999999e-05 2024-09-21T21:42:33.202 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.202 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.202 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:33.203 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411652+0000", 2024-09-21T21:42:33.203 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9990000000000002e-06 2024-09-21T21:42:33.203 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.203 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.203 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:33.203 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411717+0000", 2024-09-21T21:42:33.203 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5519999999999996e-05 2024-09-21T21:42:33.203 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.203 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.203 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op must wait for map", 2024-09-21T21:42:33.203 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411725+0000", 2024-09-21T21:42:33.203 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2609999999999996e-06 2024-09-21T21:42:33.203 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.203 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.203 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:33.204 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415904+0000", 2024-09-21T21:42:33.204 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0041792749999999997 2024-09-21T21:42:33.204 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.204 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.204 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:33.204 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415913+0000", 2024-09-21T21:42:33.204 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.054e-06 2024-09-21T21:42:33.204 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.204 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.204 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:33.204 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420096+0000", 2024-09-21T21:42:33.204 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0041835670000001 2024-09-21T21:42:33.204 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.204 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.205 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:33.205 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420216+0000", 2024-09-21T21:42:33.205 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000120025 2024-09-21T21:42:33.205 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.205 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.205 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:33.205 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420323+0000", 2024-09-21T21:42:33.205 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010674400000000001 2024-09-21T21:42:33.205 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:33.205 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:33.205 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:33.205 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.205 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.205 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5626.0:12339 3.3 3:c643a86c:::rbd_data.1121faebdfe3.0000000000000002:head [read 4096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:42:33.206 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:14.411749+0000", 2024-09-21T21:42:33.206 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 18.695219220999999, 2024-09-21T21:42:33.206 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.008818056, 2024-09-21T21:42:33.206 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:33.206 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:33.206 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:33.206 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:33.206 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5626", 2024-09-21T21:42:33.206 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:33.206 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 12339 2024-09-21T21:42:33.206 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.206 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:33.206 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.207 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:33.207 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411749+0000", 2024-09-21T21:42:33.207 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:33.207 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.207 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.207 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:33.207 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411749+0000", 2024-09-21T21:42:33.207 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:33.207 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.207 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.207 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:33.207 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411749+0000", 2024-09-21T21:42:33.207 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0900000000000002e-07 2024-09-21T21:42:33.207 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.208 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.208 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:33.208 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411752+0000", 2024-09-21T21:42:33.208 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5610000000000001e-06 2024-09-21T21:42:33.208 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.208 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.208 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:33.208 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411758+0000", 2024-09-21T21:42:33.208 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1890000000000004e-06 2024-09-21T21:42:33.208 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.208 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.208 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:33.208 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411763+0000", 2024-09-21T21:42:33.209 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7589999999999997e-06 2024-09-21T21:42:33.209 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.209 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.209 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:33.209 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411803+0000", 2024-09-21T21:42:33.209 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9922999999999997e-05 2024-09-21T21:42:33.209 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.209 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.209 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting_for_map not empty", 2024-09-21T21:42:33.209 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411809+0000", 2024-09-21T21:42:33.209 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3779999999999998e-06 2024-09-21T21:42:33.209 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.209 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.210 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:33.210 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415937+0000", 2024-09-21T21:42:33.210 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0041276100000000003 2024-09-21T21:42:33.210 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.210 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.210 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:33.210 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415944+0000", 2024-09-21T21:42:33.210 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8329999999999994e-06 2024-09-21T21:42:33.210 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.210 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.210 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:33.210 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420411+0000", 2024-09-21T21:42:33.210 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0044667300000001 2024-09-21T21:42:33.211 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.211 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.211 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:33.211 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420475+0000", 2024-09-21T21:42:33.211 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4356000000000005e-05 2024-09-21T21:42:33.211 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.211 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.211 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:33.211 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420567+0000", 2024-09-21T21:42:33.211 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.1308000000000001e-05 2024-09-21T21:42:33.211 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:33.211 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:33.211 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:33.211 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.211 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.212 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5626.0:12345 3.3 3:c643a86c:::rbd_data.1121faebdfe3.0000000000000002:head [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:42:33.212 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:14.411831+0000", 2024-09-21T21:42:33.212 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 18.695137167999999, 2024-09-21T21:42:33.212 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0089163649999999, 2024-09-21T21:42:33.212 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:33.212 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:33.212 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:33.212 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:33.212 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5626", 2024-09-21T21:42:33.212 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:33.212 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 12345 2024-09-21T21:42:33.212 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.212 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:33.212 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.212 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:33.212 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411831+0000", 2024-09-21T21:42:33.213 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:33.213 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.213 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.213 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:33.213 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411831+0000", 2024-09-21T21:42:33.213 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:33.213 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.213 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.213 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:33.213 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411831+0000", 2024-09-21T21:42:33.213 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.08e-07 2024-09-21T21:42:33.213 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.213 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.213 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:33.214 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411833+0000", 2024-09-21T21:42:33.214 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3089999999999998e-06 2024-09-21T21:42:33.214 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.214 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.214 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:33.214 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411840+0000", 2024-09-21T21:42:33.214 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.472e-06 2024-09-21T21:42:33.214 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.214 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.214 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:33.214 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411845+0000", 2024-09-21T21:42:33.214 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7910000000000004e-06 2024-09-21T21:42:33.214 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.214 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.214 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:33.215 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411890+0000", 2024-09-21T21:42:33.215 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5763000000000002e-05 2024-09-21T21:42:33.215 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.215 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.215 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting_for_map not empty", 2024-09-21T21:42:33.215 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411896+0000", 2024-09-21T21:42:33.215 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0190000000000003e-06 2024-09-21T21:42:33.215 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.215 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.215 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:33.215 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415967+0000", 2024-09-21T21:42:33.215 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0040705749999999999 2024-09-21T21:42:33.215 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.215 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.215 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:33.216 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415974+0000", 2024-09-21T21:42:33.216 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3200000000000002e-06 2024-09-21T21:42:33.216 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.216 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.216 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:33.216 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420594+0000", 2024-09-21T21:42:33.216 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.004620093 2024-09-21T21:42:33.216 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.216 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.216 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:33.216 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420658+0000", 2024-09-21T21:42:33.216 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3678999999999998e-05 2024-09-21T21:42:33.216 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:33.216 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:33.217 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:33.217 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420747+0000", 2024-09-21T21:42:33.217 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8936000000000004e-05 2024-09-21T21:42:33.217 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:33.217 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:33.217 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:33.217 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:33.217 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:33.217 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:33.219 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T21:42:33.219 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:33.217+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:33.277 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T21:42:33.319 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:33.318+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:33.362 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:33.362 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:42:33.362 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-21T21:42:33.362 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:33.370 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T21:42:33.396 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0: {} 2024-09-21T21:42:33.397 INFO:teuthology.orchestra.run.smithi081.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T21:42:33.413 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1: {} 2024-09-21T21:42:33.414 INFO:teuthology.orchestra.run.smithi081.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T21:42:33.420 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:33.418+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:33.429 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2: {} 2024-09-21T21:42:33.429 INFO:teuthology.orchestra.run.smithi081.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T21:42:33.446 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3: {} 2024-09-21T21:42:33.447 INFO:teuthology.orchestra.run.smithi081.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T21:42:33.463 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4: {} 2024-09-21T21:42:33.463 INFO:teuthology.orchestra.run.smithi081.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T21:42:33.481 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5: {} 2024-09-21T21:42:33.481 INFO:teuthology.orchestra.run.smithi081.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T21:42:33.517 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:33.517 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:33.525 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T21:42:33.532 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:33.531+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:33.546 INFO:teuthology.orchestra.run.smithi081.stderr:nodeep-scrub is unset 2024-09-21T21:42:33.622 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:33.620+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:33.634 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:33.634 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:33.643 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T21:42:33.646 INFO:teuthology.orchestra.run.smithi164.stdout:truncating image image_client.0-clone44 from 0xddfc200 (overlap 0x9151c00) to 0x0 2024-09-21T21:42:33.711 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 5, 3] out_osds: [0, 4] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-21T21:42:33.711 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T21:42:33.711 INFO:tasks.thrashosds.thrasher:inject_pause on osd.5 2024-09-21T21:42:33.711 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-21T21:42:33.711 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-21T21:42:33.712 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok config set filestore_inject_stall 3 2024-09-21T21:42:33.723 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:33.722+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:33.763 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:33.763 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:33.772 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T21:42:33.822 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:33.822 INFO:teuthology.orchestra.run.smithi122.stdout: "success": "" 2024-09-21T21:42:33.822 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:33.823 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:33.822+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:33.881 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:33.881 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:33.890 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T21:42:33.924 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:33.923+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:33.995 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:33.996 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:34.004 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T21:42:34.049 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:34.047+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:34.065 INFO:teuthology.orchestra.run.smithi164.stdout:checking clone #43, image image_client.0-clone43 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent44 2024-09-21T21:42:34.113 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:34.113 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:34.122 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T21:42:34.126 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:34.124+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:34.208 INFO:teuthology.orchestra.run.smithi164.stdout:5852 write 0x2fdc200 thru 0x2fde723 (0x2524 bytes) 2024-09-21T21:42:34.209 INFO:teuthology.orchestra.run.smithi164.stdout:5854 punch from 0x8b9f600 to 0x8ba6600, (0x7000 bytes) 2024-09-21T21:42:34.225 INFO:teuthology.orchestra.run.smithi164.stdout:5861 read 0x3eb6000 thru 0x3eb7d53 (0x1d54 bytes) 2024-09-21T21:42:34.225 INFO:teuthology.orchestra.run.smithi164.stdout:5862 read 0x5fc4600 thru 0x5fcb6c5 (0x70c6 bytes) 2024-09-21T21:42:34.225 INFO:teuthology.orchestra.run.smithi164.stdout:5864 read 0xc488400 thru 0xc48d3b8 (0x4fb9 bytes) 2024-09-21T21:42:34.226 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:34.225+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:34.227 INFO:teuthology.orchestra.run.smithi164.stdout:5866 trunc from 0xddfc200 to 0xc217200 2024-09-21T21:42:34.249 INFO:teuthology.orchestra.run.smithi164.stdout:5869 punch from 0x89c600 to 0x8a5a00, (0x9400 bytes) 2024-09-21T21:42:34.256 INFO:teuthology.orchestra.run.smithi164.stdout:5870 trunc from 0xc217200 to 0x8fa5400 2024-09-21T21:42:34.275 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:34.275 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:34.284 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T21:42:34.298 INFO:teuthology.orchestra.run.smithi164.stdout:5871 write 0xedaa000 thru 0xedb555d (0xb55e bytes) 2024-09-21T21:42:34.299 INFO:teuthology.orchestra.run.smithi164.stdout:5873 write 0x1b1ae00 thru 0x1b1dcba (0x2ebb bytes) 2024-09-21T21:42:34.299 INFO:teuthology.orchestra.run.smithi164.stdout:5874 clone 46 order 20 su 16384 sc 3 2024-09-21T21:42:34.327 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:34.325+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:34.427 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:34.427 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:34.427 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:34.425+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:34.436 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T21:42:34.528 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:34.526+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:34.583 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:34.583 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:34.592 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T21:42:34.635 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:34.634+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:34.659 INFO:teuthology.orchestra.run.smithi164.stdout:truncating image image_client.0-clone45 from 0xedb5600 (overlap 0x8fa5400) to 0x648d400 2024-09-21T21:42:34.699 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:34.699 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:34.709 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T21:42:34.730 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:34.729+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:34.831 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:34.830+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:34.854 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:34.854 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:34.863 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T21:42:34.947 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:34.946+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:35.009 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:35.010 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:35.020 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T21:42:35.033 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:35.032+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:35.082 INFO:teuthology.orchestra.run.smithi164.stdout:checking clone #44, image image_client.0-clone44 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent45 2024-09-21T21:42:35.134 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:35.133+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:35.139 INFO:teuthology.orchestra.run.smithi164.stdout:5877 write 0x4a04200 thru 0x4a04838 (0x639 bytes) 2024-09-21T21:42:35.140 INFO:teuthology.orchestra.run.smithi164.stdout:5878 trunc from 0xedb5600 to 0xda6c400 2024-09-21T21:42:35.167 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:35.167 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:35.175 INFO:teuthology.orchestra.run.smithi164.stdout:5882 trunc from 0xda6c400 to 0x8541800 2024-09-21T21:42:35.176 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T21:42:35.231 INFO:teuthology.orchestra.run.smithi164.stdout:5884 write 0xb586400 thru 0xb594b67 (0xe768 bytes) 2024-09-21T21:42:35.231 INFO:teuthology.orchestra.run.smithi164.stdout:5885 read 0x59ee400 thru 0x59fb385 (0xcf86 bytes) 2024-09-21T21:42:35.232 INFO:teuthology.orchestra.run.smithi164.stdout:5888 read 0x1aa6000 thru 0x1aa787e (0x187f bytes) 2024-09-21T21:42:35.235 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:35.233+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:35.243 INFO:teuthology.orchestra.run.smithi164.stdout:5890 trunc from 0xb594c00 to 0x231cc00 2024-09-21T21:42:35.289 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:35.289 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:35.299 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T21:42:35.302 INFO:teuthology.orchestra.run.smithi164.stdout:5891 write 0x2fdf400 thru 0x2fe9797 (0xa398 bytes) 2024-09-21T21:42:35.303 INFO:teuthology.orchestra.run.smithi164.stdout:5894 write 0x206c000 thru 0x206fc4e (0x3c4f bytes) 2024-09-21T21:42:35.303 INFO:teuthology.orchestra.run.smithi164.stdout:5895 punch from 0x1c89600 to 0x1c95200, (0xbc00 bytes) 2024-09-21T21:42:35.334 INFO:teuthology.orchestra.run.smithi164.stdout:5896 write 0x40d4600 thru 0x40e16e8 (0xd0e9 bytes) 2024-09-21T21:42:35.334 INFO:teuthology.orchestra.run.smithi164.stdout:5897 read 0x145ec00 thru 0x1464ac1 (0x5ec2 bytes) 2024-09-21T21:42:35.334 INFO:teuthology.orchestra.run.smithi164.stdout:5898 read 0x2568400 thru 0x2575583 (0xd184 bytes) 2024-09-21T21:42:35.334 INFO:teuthology.orchestra.run.smithi164.stdout:5899 read 0x2ab0800 thru 0x2ab1721 (0xf22 bytes) 2024-09-21T21:42:35.339 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:35.338+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:35.360 INFO:teuthology.orchestra.run.smithi164.stdout:5900 write 0x945d000 thru 0x9462f76 (0x5f77 bytes) 2024-09-21T21:42:35.363 INFO:teuthology.orchestra.run.smithi164.stdout:5902 trunc from 0x9463000 to 0xb7da200 2024-09-21T21:42:35.383 INFO:teuthology.orchestra.run.smithi164.stdout:5903 punch from 0x786ce00 to 0x7872a00, (0x5c00 bytes) 2024-09-21T21:42:35.383 INFO:teuthology.orchestra.run.smithi164.stdout:5905 write 0x615c00 thru 0x61f8db (0x9cdc bytes) 2024-09-21T21:42:35.383 INFO:teuthology.orchestra.run.smithi164.stdout:5910 write 0x50e200 thru 0x51cdad (0xebae bytes) 2024-09-21T21:42:35.384 INFO:teuthology.orchestra.run.smithi164.stdout:5911 write 0x9b77400 thru 0x9b811e4 (0x9de5 bytes) 2024-09-21T21:42:35.419 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:35.419 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:35.422 INFO:teuthology.orchestra.run.smithi164.stdout:5912 write 0xdb06800 thru 0xdb1396c (0xd16d bytes) 2024-09-21T21:42:35.423 INFO:teuthology.orchestra.run.smithi164.stdout:5914 read 0x26f5a00 thru 0x270567b (0xfc7c bytes) 2024-09-21T21:42:35.423 INFO:teuthology.orchestra.run.smithi164.stdout:5915 read 0xa603000 thru 0xa608649 (0x564a bytes) 2024-09-21T21:42:35.423 INFO:teuthology.orchestra.run.smithi164.stdout:5916 write 0xaf51200 thru 0xaf51cfc (0xafd bytes) 2024-09-21T21:42:35.423 INFO:teuthology.orchestra.run.smithi164.stdout:5918 write 0x67ea800 thru 0x67f4a63 (0xa264 bytes) 2024-09-21T21:42:35.423 INFO:teuthology.orchestra.run.smithi164.stdout:5920 write 0x4f6ea00 thru 0x4f74dcf (0x63d0 bytes) 2024-09-21T21:42:35.423 INFO:teuthology.orchestra.run.smithi164.stdout:5922 write 0x5e74600 thru 0x5e80e33 (0xc834 bytes) 2024-09-21T21:42:35.423 INFO:teuthology.orchestra.run.smithi164.stdout:5925 punch from 0x1e4be00 to 0x1e4e600, (0x2800 bytes) 2024-09-21T21:42:35.424 INFO:teuthology.orchestra.run.smithi164.stdout:5926 punch from 0x6726600 to 0x6728600, (0x2000 bytes) 2024-09-21T21:42:35.424 INFO:teuthology.orchestra.run.smithi164.stdout:5927 write 0x1207400 thru 0x1213fef (0xcbf0 bytes) 2024-09-21T21:42:35.429 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T21:42:35.430 INFO:teuthology.orchestra.run.smithi164.stdout:5928 trunc from 0xdb13a00 to 0xac4cc00 2024-09-21T21:42:35.436 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:35.435+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:35.472 INFO:teuthology.orchestra.run.smithi164.stdout:5930 write 0x95dea00 thru 0x95e177f (0x2d80 bytes) 2024-09-21T21:42:35.474 INFO:teuthology.orchestra.run.smithi164.stdout:5931 read 0x6939600 thru 0x693adc3 (0x17c4 bytes) 2024-09-21T21:42:35.474 INFO:teuthology.orchestra.run.smithi164.stdout:5932 write 0x20be600 thru 0x20c0728 (0x2129 bytes) 2024-09-21T21:42:35.474 INFO:teuthology.orchestra.run.smithi164.stdout:5936 punch from 0x7756000 to 0x7758c00, (0x2c00 bytes) 2024-09-21T21:42:35.475 INFO:teuthology.orchestra.run.smithi164.stdout:5937 write 0x65eb800 thru 0x65f749b (0xbc9c bytes) 2024-09-21T21:42:35.475 INFO:teuthology.orchestra.run.smithi164.stdout:5939 read 0x4c20200 thru 0x4c2c960 (0xc761 bytes) 2024-09-21T21:42:35.475 INFO:teuthology.orchestra.run.smithi164.stdout:5940 read 0x8755a00 thru 0x8763622 (0xdc23 bytes) 2024-09-21T21:42:35.475 INFO:teuthology.orchestra.run.smithi164.stdout:5948 punch from 0x9c08400 to 0x9c10e00, (0x8a00 bytes) 2024-09-21T21:42:35.476 INFO:teuthology.orchestra.run.smithi164.stdout:5961 read 0x7bd2a00 thru 0x7bd60f5 (0x36f6 bytes) 2024-09-21T21:42:35.479 INFO:teuthology.orchestra.run.smithi164.stdout:5962 trunc from 0xac4cc00 to 0xdb1ca00 2024-09-21T21:42:35.492 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T21:42:35.502 INFO:teuthology.orchestra.run.smithi164.stdout:5964 punch from 0x193f400 to 0x1946c00, (0x7800 bytes) 2024-09-21T21:42:35.506 INFO:teuthology.orchestra.run.smithi164.stdout:5965 read 0x1c05c00 thru 0x1c1199d (0xbd9e bytes) 2024-09-21T21:42:35.512 INFO:teuthology.orchestra.run.smithi164.stdout:5966 write 0xd501000 thru 0xd50f2ba (0xe2bb bytes) 2024-09-21T21:42:35.512 INFO:teuthology.orchestra.run.smithi164.stdout:5967 read 0x23c2600 thru 0x23c521f (0x2c20 bytes) 2024-09-21T21:42:35.512 INFO:teuthology.orchestra.run.smithi164.stdout:5969 punch from 0xf95c00 to 0xf9f400, (0x9800 bytes) 2024-09-21T21:42:35.514 INFO:teuthology.orchestra.run.smithi164.stdout:5970 read 0x6768800 thru 0x677516b (0xc96c bytes) 2024-09-21T21:42:35.519 INFO:teuthology.orchestra.run.smithi164.stdout:5972 write 0x5486200 thru 0x548a1eb (0x3fec bytes) 2024-09-21T21:42:35.519 INFO:teuthology.orchestra.run.smithi164.stdout:5974 punch from 0x9008c00 to 0x900fa00, (0x6e00 bytes) 2024-09-21T21:42:35.519 INFO:teuthology.orchestra.run.smithi164.stdout:5978 read 0x3fc9600 thru 0x3fcad68 (0x1769 bytes) 2024-09-21T21:42:35.520 INFO:teuthology.orchestra.run.smithi164.stdout:5979 read 0x2422e00 thru 0x242e4d2 (0xb6d3 bytes) 2024-09-21T21:42:35.535 INFO:teuthology.orchestra.run.smithi164.stdout:5980 trunc from 0xdb1ca00 to 0x83e200 2024-09-21T21:42:35.537 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:35.536+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:35.561 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T21:42:35.562 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:35.562 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:35.571 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T21:42:35.606 INFO:teuthology.orchestra.run.smithi164.stdout:5981 write 0xc58e800 thru 0xc594749 (0x5f4a bytes) 2024-09-21T21:42:35.607 INFO:teuthology.orchestra.run.smithi164.stdout:5982 read 0x85d8000 thru 0x85e4721 (0xc722 bytes) 2024-09-21T21:42:35.607 INFO:teuthology.orchestra.run.smithi164.stdout:5983 read 0xa341600 thru 0xa34e882 (0xd283 bytes) 2024-09-21T21:42:35.607 INFO:teuthology.orchestra.run.smithi164.stdout:5984 punch from 0x4a12600 to 0x4a14800, (0x2200 bytes) 2024-09-21T21:42:35.608 INFO:teuthology.orchestra.run.smithi164.stdout:5987 read 0xbc3f600 thru 0xbc3f9cb (0x3cc bytes) 2024-09-21T21:42:35.623 INFO:teuthology.orchestra.run.smithi164.stdout:5989 trunc from 0xc594800 to 0x2d2800 2024-09-21T21:42:35.634 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0: {} 2024-09-21T21:42:35.635 INFO:teuthology.orchestra.run.smithi081.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T21:42:35.638 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:35.637+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:35.660 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1: {} 2024-09-21T21:42:35.661 INFO:teuthology.orchestra.run.smithi081.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T21:42:35.665 INFO:teuthology.orchestra.run.smithi164.stdout:5992 read 0x2ae400 thru 0x2b561a (0x721b bytes) 2024-09-21T21:42:35.678 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2: {} 2024-09-21T21:42:35.678 INFO:teuthology.orchestra.run.smithi081.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T21:42:35.689 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:35.689 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:35.690 INFO:teuthology.orchestra.run.smithi164.stdout:5996 write 0x9edc600 thru 0x9ee8cf9 (0xc6fa bytes) 2024-09-21T21:42:35.692 INFO:teuthology.orchestra.run.smithi164.stdout:5997 write 0x7e4fa00 thru 0x7e52215 (0x2816 bytes) 2024-09-21T21:42:35.692 INFO:teuthology.orchestra.run.smithi164.stdout:5998 write 0x82b2400 thru 0x82b3f52 (0x1b53 bytes) 2024-09-21T21:42:35.695 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3: {} 2024-09-21T21:42:35.695 INFO:teuthology.orchestra.run.smithi081.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T21:42:35.699 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T21:42:35.714 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4: {} 2024-09-21T21:42:35.715 INFO:teuthology.orchestra.run.smithi081.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T21:42:35.718 INFO:teuthology.orchestra.run.smithi164.stdout:6000 write 0xddb6a00 thru 0xddc08bb (0x9ebc bytes) 2024-09-21T21:42:35.731 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5: {} 2024-09-21T21:42:35.731 INFO:teuthology.orchestra.run.smithi081.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T21:42:35.738 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:35.737+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:35.839 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:35.838+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:35.846 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:35.846 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:42:35.846 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-21T21:42:35.847 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:35.855 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T21:42:35.940 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:35.938+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:35.942 INFO:teuthology.orchestra.run.smithi164.stdout:checking clone #45, image image_client.0-clone45 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent46 2024-09-21T21:42:36.010 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:36.010 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:42:36.010 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:42:36.010 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:42:36.011 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:36.021 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T21:42:36.040 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:36.039+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:36.141 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:36.139+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:36.181 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:36.181 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-21T21:42:36.181 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-21T21:42:36.181 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:42:36.181 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.181 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6974 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5246976~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:36.181 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.922932+0000", 2024-09-21T21:42:36.182 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 66.25592134, 2024-09-21T21:42:36.182 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041773352999999999, 2024-09-21T21:42:36.182 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:36.182 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:36.182 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:36.182 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:36.182 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:36.182 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:36.182 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6974 2024-09-21T21:42:36.182 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.182 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:36.182 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.182 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:36.183 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922932+0000", 2024-09-21T21:42:36.183 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:36.183 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.183 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.183 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:36.183 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922932+0000", 2024-09-21T21:42:36.183 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:36.183 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.183 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.183 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:36.183 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922933+0000", 2024-09-21T21:42:36.183 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.46e-07 2024-09-21T21:42:36.183 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.184 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.184 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:36.184 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922940+0000", 2024-09-21T21:42:36.184 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6140000000000003e-06 2024-09-21T21:42:36.184 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.184 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.184 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:36.184 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922949+0000", 2024-09-21T21:42:36.184 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6899999999999998e-06 2024-09-21T21:42:36.184 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.184 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.184 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:36.184 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922955+0000", 2024-09-21T21:42:36.185 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5140000000000001e-06 2024-09-21T21:42:36.185 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.185 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.185 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:36.185 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964547+0000", 2024-09-21T21:42:36.185 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041592854999999998 2024-09-21T21:42:36.185 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.185 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.185 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:36.185 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964617+0000", 2024-09-21T21:42:36.185 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9058000000000002e-05 2024-09-21T21:42:36.185 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.185 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.185 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:36.186 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964706+0000", 2024-09-21T21:42:36.186 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9276000000000004e-05 2024-09-21T21:42:36.186 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.186 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:36.186 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.186 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.186 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.186 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6976 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5261312~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:36.186 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.922961+0000", 2024-09-21T21:42:36.186 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 66.255892294999995, 2024-09-21T21:42:36.186 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041863648000000003, 2024-09-21T21:42:36.186 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:36.186 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:36.187 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:36.187 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:36.187 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:36.187 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:36.187 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6976 2024-09-21T21:42:36.187 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.187 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:36.187 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.187 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:36.187 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922961+0000", 2024-09-21T21:42:36.187 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:36.187 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.187 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.187 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:36.188 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922961+0000", 2024-09-21T21:42:36.188 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:36.188 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.188 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.188 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:36.188 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922962+0000", 2024-09-21T21:42:36.188 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.89e-07 2024-09-21T21:42:36.188 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.188 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.188 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:36.188 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922964+0000", 2024-09-21T21:42:36.188 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1270000000000001e-06 2024-09-21T21:42:36.188 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.188 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.189 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:36.189 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922970+0000", 2024-09-21T21:42:36.189 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8030000000000002e-06 2024-09-21T21:42:36.189 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.189 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.189 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:36.189 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922979+0000", 2024-09-21T21:42:36.189 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5559999999999996e-06 2024-09-21T21:42:36.189 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.189 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.189 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:36.189 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964734+0000", 2024-09-21T21:42:36.189 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041754415000000003 2024-09-21T21:42:36.189 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.190 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.190 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:36.190 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964775+0000", 2024-09-21T21:42:36.190 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0806999999999997e-05 2024-09-21T21:42:36.190 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.190 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.190 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:36.190 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964825+0000", 2024-09-21T21:42:36.190 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0550999999999998e-05 2024-09-21T21:42:36.190 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.190 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:36.190 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.190 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.191 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.191 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6978 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5275648~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:36.191 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.922987+0000", 2024-09-21T21:42:36.191 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 66.255866510000004, 2024-09-21T21:42:36.191 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042043876000000001, 2024-09-21T21:42:36.191 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:36.191 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:36.191 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:36.191 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:36.191 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:36.191 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:36.191 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6978 2024-09-21T21:42:36.191 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.191 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:36.192 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.192 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:36.192 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922987+0000", 2024-09-21T21:42:36.192 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:36.192 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.192 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.192 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:36.192 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922987+0000", 2024-09-21T21:42:36.192 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:36.192 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.192 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.192 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:36.192 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922988+0000", 2024-09-21T21:42:36.192 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1199999999999998e-07 2024-09-21T21:42:36.193 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.193 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.193 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:36.193 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922990+0000", 2024-09-21T21:42:36.193 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.142e-06 2024-09-21T21:42:36.193 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.193 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.193 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:36.193 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922997+0000", 2024-09-21T21:42:36.193 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.199e-06 2024-09-21T21:42:36.193 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.193 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.193 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:36.193 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923004+0000", 2024-09-21T21:42:36.194 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3599999999999998e-06 2024-09-21T21:42:36.194 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.194 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.194 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:36.194 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964856+0000", 2024-09-21T21:42:36.194 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041851521000000003 2024-09-21T21:42:36.194 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.194 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.194 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:36.194 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964944+0000", 2024-09-21T21:42:36.194 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7859e-05 2024-09-21T21:42:36.194 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.195 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.195 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:36.195 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965031+0000", 2024-09-21T21:42:36.195 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7383e-05 2024-09-21T21:42:36.195 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.195 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:36.195 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.195 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.195 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.195 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6980 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5289984~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:36.195 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923011+0000", 2024-09-21T21:42:36.195 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 66.255843173000002, 2024-09-21T21:42:36.195 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042139337999999998, 2024-09-21T21:42:36.195 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:36.196 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:36.196 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:36.196 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:36.196 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:36.196 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:36.196 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6980 2024-09-21T21:42:36.196 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.196 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:36.196 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.196 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:36.196 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923011+0000", 2024-09-21T21:42:36.196 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:36.196 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.196 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.197 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:36.197 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923011+0000", 2024-09-21T21:42:36.197 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:36.197 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.197 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.197 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:36.197 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923011+0000", 2024-09-21T21:42:36.197 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6699999999999999e-07 2024-09-21T21:42:36.197 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.197 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.197 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:36.197 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923013+0000", 2024-09-21T21:42:36.197 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0279999999999999e-06 2024-09-21T21:42:36.197 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.198 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.198 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:36.198 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923018+0000", 2024-09-21T21:42:36.198 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4790000000000004e-06 2024-09-21T21:42:36.198 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.198 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.198 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:36.198 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923029+0000", 2024-09-21T21:42:36.198 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0601999999999999e-05 2024-09-21T21:42:36.198 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.198 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.198 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:36.198 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965059+0000", 2024-09-21T21:42:36.198 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042029947999999998 2024-09-21T21:42:36.198 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.199 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.199 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:36.199 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965100+0000", 2024-09-21T21:42:36.199 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1170000000000001e-05 2024-09-21T21:42:36.199 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.199 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.199 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:36.199 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965150+0000", 2024-09-21T21:42:36.199 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9743999999999997e-05 2024-09-21T21:42:36.199 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.199 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:36.199 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.199 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.200 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.200 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6982 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5304320~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:36.200 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923036+0000", 2024-09-21T21:42:36.200 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 66.255817351999994, 2024-09-21T21:42:36.200 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042225666000000002, 2024-09-21T21:42:36.200 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:36.200 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:36.200 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:36.200 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:36.200 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:36.200 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:36.201 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6982 2024-09-21T21:42:36.201 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.201 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:36.201 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.201 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:36.201 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923036+0000", 2024-09-21T21:42:36.201 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:36.201 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.201 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.201 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:36.201 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923036+0000", 2024-09-21T21:42:36.201 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:36.201 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.201 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.202 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:36.202 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923037+0000", 2024-09-21T21:42:36.202 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2800000000000002e-07 2024-09-21T21:42:36.202 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.202 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.202 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:36.202 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923039+0000", 2024-09-21T21:42:36.202 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1500000000000002e-06 2024-09-21T21:42:36.202 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.202 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.202 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:36.202 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923049+0000", 2024-09-21T21:42:36.202 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0236e-05 2024-09-21T21:42:36.203 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.203 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.203 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:36.203 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923056+0000", 2024-09-21T21:42:36.203 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2540000000000003e-06 2024-09-21T21:42:36.203 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.203 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.203 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:36.203 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965173+0000", 2024-09-21T21:42:36.203 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042116686 2024-09-21T21:42:36.203 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.203 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.203 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:36.203 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965215+0000", 2024-09-21T21:42:36.203 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1696000000000002e-05 2024-09-21T21:42:36.204 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.204 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.204 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:36.204 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965262+0000", 2024-09-21T21:42:36.204 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7216000000000003e-05 2024-09-21T21:42:36.204 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.204 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:36.204 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.204 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.204 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.204 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6984 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5318656~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:36.204 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923063+0000", 2024-09-21T21:42:36.204 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 66.255790697999998, 2024-09-21T21:42:36.204 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042318651999999998, 2024-09-21T21:42:36.204 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:36.205 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:36.205 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:36.205 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:36.205 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:36.205 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:36.205 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6984 2024-09-21T21:42:36.205 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.205 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:36.205 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.205 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:36.205 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923063+0000", 2024-09-21T21:42:36.205 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:36.205 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.205 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.205 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:36.205 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923063+0000", 2024-09-21T21:42:36.206 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:36.206 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.206 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.206 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:36.206 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923064+0000", 2024-09-21T21:42:36.206 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2e-07 2024-09-21T21:42:36.206 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.206 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.206 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:36.206 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923066+0000", 2024-09-21T21:42:36.206 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.057e-06 2024-09-21T21:42:36.207 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.207 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.207 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:36.207 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923071+0000", 2024-09-21T21:42:36.207 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8810000000000001e-06 2024-09-21T21:42:36.207 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.207 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.207 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:36.207 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923078+0000", 2024-09-21T21:42:36.207 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5479999999999998e-06 2024-09-21T21:42:36.207 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.207 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.207 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:36.207 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965288+0000", 2024-09-21T21:42:36.207 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042209576999999998 2024-09-21T21:42:36.207 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.208 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.208 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:36.208 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965332+0000", 2024-09-21T21:42:36.208 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4366999999999997e-05 2024-09-21T21:42:36.208 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.208 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.208 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:36.208 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965382+0000", 2024-09-21T21:42:36.208 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9802000000000002e-05 2024-09-21T21:42:36.208 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.208 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:36.208 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.208 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.208 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.208 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6987 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5332992~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:36.209 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923085+0000", 2024-09-21T21:42:36.209 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 66.255769173000004, 2024-09-21T21:42:36.209 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042412103999999999, 2024-09-21T21:42:36.209 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:36.209 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:36.209 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:36.209 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:36.209 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:36.209 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:36.209 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6987 2024-09-21T21:42:36.209 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.209 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:36.209 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.209 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:36.210 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923085+0000", 2024-09-21T21:42:36.210 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:36.210 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.210 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.210 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:36.210 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923085+0000", 2024-09-21T21:42:36.210 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:36.210 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.210 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.210 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:36.210 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923085+0000", 2024-09-21T21:42:36.210 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.89e-07 2024-09-21T21:42:36.210 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.210 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.210 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:36.211 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923087+0000", 2024-09-21T21:42:36.211 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0810000000000001e-06 2024-09-21T21:42:36.211 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.211 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.211 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:36.211 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923098+0000", 2024-09-21T21:42:36.211 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1299e-05 2024-09-21T21:42:36.211 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.211 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.211 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:36.211 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923104+0000", 2024-09-21T21:42:36.211 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9100000000000002e-06 2024-09-21T21:42:36.211 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.211 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.212 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:36.212 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965404+0000", 2024-09-21T21:42:36.212 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042299622000000002 2024-09-21T21:42:36.212 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.212 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.212 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:36.212 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965445+0000", 2024-09-21T21:42:36.212 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0701000000000002e-05 2024-09-21T21:42:36.212 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.212 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.212 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:36.212 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965497+0000", 2024-09-21T21:42:36.212 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2101999999999997e-05 2024-09-21T21:42:36.212 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.213 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:36.213 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.213 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.213 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.213 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6989 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5347328~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:36.213 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923111+0000", 2024-09-21T21:42:36.213 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 66.255742832999999, 2024-09-21T21:42:36.213 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042505188999999999, 2024-09-21T21:42:36.213 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:36.213 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:36.213 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:36.213 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:36.213 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:36.213 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:36.213 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6989 2024-09-21T21:42:36.214 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.214 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:36.214 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.214 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:36.214 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923111+0000", 2024-09-21T21:42:36.214 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:36.214 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.214 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.214 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:36.214 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923111+0000", 2024-09-21T21:42:36.214 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:36.214 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.214 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.214 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:36.215 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923111+0000", 2024-09-21T21:42:36.215 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4499999999999997e-07 2024-09-21T21:42:36.215 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.215 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.215 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:36.215 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923114+0000", 2024-09-21T21:42:36.215 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1569999999999998e-06 2024-09-21T21:42:36.215 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.215 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.215 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:36.215 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923120+0000", 2024-09-21T21:42:36.215 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1759999999999998e-06 2024-09-21T21:42:36.215 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.215 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.215 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:36.216 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923125+0000", 2024-09-21T21:42:36.216 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2560000000000002e-06 2024-09-21T21:42:36.216 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.216 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.216 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:36.216 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965523+0000", 2024-09-21T21:42:36.216 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042398094999999997 2024-09-21T21:42:36.216 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.216 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.216 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:36.216 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965566+0000", 2024-09-21T21:42:36.216 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3393999999999998e-05 2024-09-21T21:42:36.216 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.216 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.217 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:36.217 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965616+0000", 2024-09-21T21:42:36.217 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9666000000000002e-05 2024-09-21T21:42:36.217 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.217 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:36.217 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.217 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.217 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.217 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6991 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5361664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:36.217 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923132+0000", 2024-09-21T21:42:36.217 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 66.255721543999996, 2024-09-21T21:42:36.217 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042600779999999998, 2024-09-21T21:42:36.217 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:36.217 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:36.218 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:36.218 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:36.218 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:36.218 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:36.218 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6991 2024-09-21T21:42:36.218 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.218 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:36.218 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.218 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:36.218 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923132+0000", 2024-09-21T21:42:36.218 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:36.218 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.218 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.218 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:36.219 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923132+0000", 2024-09-21T21:42:36.219 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:36.219 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.219 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.219 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:36.219 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923132+0000", 2024-09-21T21:42:36.219 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4200000000000002e-07 2024-09-21T21:42:36.219 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.219 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.219 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:36.219 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923134+0000", 2024-09-21T21:42:36.219 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2270000000000001e-06 2024-09-21T21:42:36.219 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.219 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.219 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:36.220 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923137+0000", 2024-09-21T21:42:36.220 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1930000000000002e-06 2024-09-21T21:42:36.220 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.220 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.220 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:36.220 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923139+0000", 2024-09-21T21:42:36.220 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4710000000000001e-06 2024-09-21T21:42:36.220 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.220 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.220 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:36.220 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965639+0000", 2024-09-21T21:42:36.220 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.04249965 2024-09-21T21:42:36.220 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.220 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.221 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:36.221 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965679+0000", 2024-09-21T21:42:36.221 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9852999999999999e-05 2024-09-21T21:42:36.221 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.221 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.221 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:36.221 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965733+0000", 2024-09-21T21:42:36.221 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4144000000000002e-05 2024-09-21T21:42:36.221 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.221 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:36.221 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.221 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.221 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.221 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6993 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5384192~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:36.221 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923143+0000", 2024-09-21T21:42:36.222 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 66.255710750000006, 2024-09-21T21:42:36.222 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042708182999999997, 2024-09-21T21:42:36.222 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:36.222 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:36.222 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:36.222 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:36.222 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:36.222 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:36.222 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6993 2024-09-21T21:42:36.222 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.222 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:36.222 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.222 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:36.222 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923143+0000", 2024-09-21T21:42:36.223 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:36.223 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.223 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.223 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:36.223 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923143+0000", 2024-09-21T21:42:36.223 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:36.223 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.223 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.223 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:36.223 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923143+0000", 2024-09-21T21:42:36.223 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0800000000000001e-07 2024-09-21T21:42:36.223 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.223 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.223 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:36.224 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923144+0000", 2024-09-21T21:42:36.224 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.009e-06 2024-09-21T21:42:36.224 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.224 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.224 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:36.224 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923148+0000", 2024-09-21T21:42:36.224 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.737e-06 2024-09-21T21:42:36.224 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.224 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.224 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:36.224 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923150+0000", 2024-09-21T21:42:36.224 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4669999999999998e-06 2024-09-21T21:42:36.224 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.224 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.224 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:36.225 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965760+0000", 2024-09-21T21:42:36.225 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042609399999999999 2024-09-21T21:42:36.225 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.225 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.225 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:36.225 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965801+0000", 2024-09-21T21:42:36.225 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1399999999999997e-05 2024-09-21T21:42:36.225 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.225 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.225 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:36.226 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965851+0000", 2024-09-21T21:42:36.226 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9962e-05 2024-09-21T21:42:36.226 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.226 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:36.226 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.226 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.226 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.226 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6995 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5398528~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:36.226 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923153+0000", 2024-09-21T21:42:36.226 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 66.255700386000001, 2024-09-21T21:42:36.226 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042828034000000001, 2024-09-21T21:42:36.226 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:36.226 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:36.227 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:36.227 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:36.227 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:36.227 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:36.227 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6995 2024-09-21T21:42:36.227 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.227 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:36.227 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.227 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:36.227 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923153+0000", 2024-09-21T21:42:36.227 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:36.227 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.227 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.227 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:36.227 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923153+0000", 2024-09-21T21:42:36.228 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:36.228 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.228 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.228 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:36.228 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923154+0000", 2024-09-21T21:42:36.228 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2600000000000001e-07 2024-09-21T21:42:36.228 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.228 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.228 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:36.228 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923158+0000", 2024-09-21T21:42:36.228 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0559999999999998e-06 2024-09-21T21:42:36.228 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.228 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.229 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:36.229 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923161+0000", 2024-09-21T21:42:36.229 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6519999999999999e-06 2024-09-21T21:42:36.229 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.229 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.229 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:36.229 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923170+0000", 2024-09-21T21:42:36.229 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5720000000000003e-06 2024-09-21T21:42:36.229 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.229 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.229 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:36.230 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965887+0000", 2024-09-21T21:42:36.230 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042716863000000001 2024-09-21T21:42:36.230 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.230 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.230 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:36.230 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965927+0000", 2024-09-21T21:42:36.230 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0346999999999998e-05 2024-09-21T21:42:36.230 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.230 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.230 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:36.230 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965981+0000", 2024-09-21T21:42:36.230 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4317999999999997e-05 2024-09-21T21:42:36.231 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.231 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:36.231 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.231 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.231 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.231 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6997 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5412864~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:36.231 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923174+0000", 2024-09-21T21:42:36.231 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 66.255680230999999, 2024-09-21T21:42:36.231 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042928296999999997, 2024-09-21T21:42:36.231 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:36.231 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:36.231 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:36.231 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:36.231 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:36.232 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:36.232 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6997 2024-09-21T21:42:36.232 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.232 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:36.232 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.232 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:36.232 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923174+0000", 2024-09-21T21:42:36.232 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:36.232 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.232 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.232 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:36.232 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923174+0000", 2024-09-21T21:42:36.232 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:36.232 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.232 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.233 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:36.233 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923174+0000", 2024-09-21T21:42:36.233 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.67e-07 2024-09-21T21:42:36.233 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.233 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.233 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:36.233 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923178+0000", 2024-09-21T21:42:36.233 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1330000000000001e-06 2024-09-21T21:42:36.233 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.233 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.233 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:36.233 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923182+0000", 2024-09-21T21:42:36.233 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.794e-06 2024-09-21T21:42:36.233 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.233 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.234 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:36.234 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923184+0000", 2024-09-21T21:42:36.234 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.435e-06 2024-09-21T21:42:36.234 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.234 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.234 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:36.234 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966009+0000", 2024-09-21T21:42:36.234 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042824327000000002 2024-09-21T21:42:36.234 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.234 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.234 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:36.234 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966052+0000", 2024-09-21T21:42:36.234 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3034000000000002e-05 2024-09-21T21:42:36.234 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.235 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.235 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:36.235 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966102+0000", 2024-09-21T21:42:36.235 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0306999999999998e-05 2024-09-21T21:42:36.235 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.235 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:36.235 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.235 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.235 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.235 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6999 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5427200~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:36.235 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923187+0000", 2024-09-21T21:42:36.235 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 66.255666503, 2024-09-21T21:42:36.235 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043104267000000002, 2024-09-21T21:42:36.235 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:36.236 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:36.236 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:36.236 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:36.236 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:36.236 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:36.236 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6999 2024-09-21T21:42:36.236 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.236 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:36.236 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.236 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:36.236 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923187+0000", 2024-09-21T21:42:36.236 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:36.236 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.236 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.237 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:36.237 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923187+0000", 2024-09-21T21:42:36.237 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:36.237 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.237 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.237 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:36.237 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923188+0000", 2024-09-21T21:42:36.237 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3999999999999998e-07 2024-09-21T21:42:36.237 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.237 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.237 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:36.237 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923189+0000", 2024-09-21T21:42:36.237 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4169999999999999e-06 2024-09-21T21:42:36.237 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.237 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.237 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:36.238 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923194+0000", 2024-09-21T21:42:36.238 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1819999999999997e-06 2024-09-21T21:42:36.238 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.238 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.238 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:36.238 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923198+0000", 2024-09-21T21:42:36.238 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.143e-06 2024-09-21T21:42:36.238 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.238 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.238 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:36.238 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966135+0000", 2024-09-21T21:42:36.238 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042936297999999998 2024-09-21T21:42:36.238 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.239 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.239 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:36.239 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966204+0000", 2024-09-21T21:42:36.239 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8989e-05 2024-09-21T21:42:36.239 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.239 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.239 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:36.239 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966292+0000", 2024-09-21T21:42:36.239 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7997999999999993e-05 2024-09-21T21:42:36.239 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.239 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:36.239 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.239 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.239 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.240 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7001 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5441536~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:36.240 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923211+0000", 2024-09-21T21:42:36.240 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 66.255642711999997, 2024-09-21T21:42:36.240 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043196518000000003, 2024-09-21T21:42:36.240 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:36.240 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:36.240 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:36.240 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:36.240 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:36.240 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:36.240 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7001 2024-09-21T21:42:36.240 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.240 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:36.240 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.240 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:36.241 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923211+0000", 2024-09-21T21:42:36.241 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:36.241 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.241 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.241 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:36.241 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923211+0000", 2024-09-21T21:42:36.241 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:36.241 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.241 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.241 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:36.241 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923212+0000", 2024-09-21T21:42:36.241 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3900000000000005e-07 2024-09-21T21:42:36.241 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.241 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.242 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:36.242 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923214+0000", 2024-09-21T21:42:36.242 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9970000000000001e-06 2024-09-21T21:42:36.242 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.242 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.242 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:36.242 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923219+0000", 2024-09-21T21:42:36.242 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4310000000000002e-06 2024-09-21T21:42:36.242 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.242 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.242 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:36.242 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923224+0000", 2024-09-21T21:42:36.242 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6410000000000001e-06 2024-09-21T21:42:36.242 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.243 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.243 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:36.243 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966318+0000", 2024-09-21T21:42:36.243 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043093842 2024-09-21T21:42:36.243 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.243 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.243 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:36.243 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966359+0000", 2024-09-21T21:42:36.243 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1260999999999997e-05 2024-09-21T21:42:36.243 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.243 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.243 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:36.243 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966408+0000", 2024-09-21T21:42:36.243 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8807000000000002e-05 2024-09-21T21:42:36.244 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.244 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:36.244 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.244 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.244 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.244 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7002 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5455872~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:36.244 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923229+0000", 2024-09-21T21:42:36.244 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 66.255624302000001, 2024-09-21T21:42:36.244 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043364568999999999, 2024-09-21T21:42:36.244 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:36.244 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:36.244 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:36.244 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:36.244 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:36.245 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:36.245 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7002 2024-09-21T21:42:36.245 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.245 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:36.245 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.245 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:36.245 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923229+0000", 2024-09-21T21:42:36.245 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:36.245 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.245 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.245 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:36.245 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923229+0000", 2024-09-21T21:42:36.245 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:36.245 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.246 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.246 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:36.246 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923230+0000", 2024-09-21T21:42:36.246 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4700000000000002e-07 2024-09-21T21:42:36.246 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.246 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.246 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:36.246 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923232+0000", 2024-09-21T21:42:36.246 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0719999999999998e-06 2024-09-21T21:42:36.246 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.246 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.246 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:36.246 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923237+0000", 2024-09-21T21:42:36.246 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2290000000000002e-06 2024-09-21T21:42:36.247 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.247 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.247 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:36.247 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923245+0000", 2024-09-21T21:42:36.247 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2959999999999999e-06 2024-09-21T21:42:36.247 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.247 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.247 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:36.247 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966438+0000", 2024-09-21T21:42:36.247 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043193027000000002 2024-09-21T21:42:36.247 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.247 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.247 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:36.247 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966506+0000", 2024-09-21T21:42:36.248 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7707999999999996e-05 2024-09-21T21:42:36.248 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.248 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.248 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:36.248 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966594+0000", 2024-09-21T21:42:36.248 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.789e-05 2024-09-21T21:42:36.248 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.248 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:36.248 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.248 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.248 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.248 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7005 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5470208~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:36.248 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923254+0000", 2024-09-21T21:42:36.248 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 66.255600071000003, 2024-09-21T21:42:36.249 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043457176, 2024-09-21T21:42:36.249 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:36.249 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:36.249 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:36.249 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:36.249 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:36.249 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:36.249 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7005 2024-09-21T21:42:36.249 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.249 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:36.249 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.249 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:36.249 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923254+0000", 2024-09-21T21:42:36.249 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:36.249 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.250 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.250 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:36.250 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923254+0000", 2024-09-21T21:42:36.250 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:36.250 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.250 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.250 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:36.250 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923254+0000", 2024-09-21T21:42:36.250 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4700000000000002e-07 2024-09-21T21:42:36.250 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.250 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.250 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:36.250 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923256+0000", 2024-09-21T21:42:36.250 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.007e-06 2024-09-21T21:42:36.250 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.251 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.251 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:36.251 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923266+0000", 2024-09-21T21:42:36.251 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.4700000000000008e-06 2024-09-21T21:42:36.251 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.251 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.251 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:36.251 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923271+0000", 2024-09-21T21:42:36.251 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2809999999999999e-06 2024-09-21T21:42:36.251 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.251 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.251 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:36.251 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966619+0000", 2024-09-21T21:42:36.251 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043347951000000003 2024-09-21T21:42:36.251 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.252 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.252 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:36.252 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966661+0000", 2024-09-21T21:42:36.252 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2614999999999998e-05 2024-09-21T21:42:36.252 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.252 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.252 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:36.252 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966711+0000", 2024-09-21T21:42:36.252 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9505000000000001e-05 2024-09-21T21:42:36.252 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.252 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:36.252 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.252 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.252 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.252 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7008 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5484544~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:36.252 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923278+0000", 2024-09-21T21:42:36.252 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 66.255575424, 2024-09-21T21:42:36.253 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043631071, 2024-09-21T21:42:36.253 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:36.253 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:36.253 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:36.253 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:36.253 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:36.253 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:36.253 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7008 2024-09-21T21:42:36.253 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.253 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:36.253 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.253 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:36.253 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923278+0000", 2024-09-21T21:42:36.253 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:36.253 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.253 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.254 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:36.254 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923278+0000", 2024-09-21T21:42:36.254 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:36.254 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.254 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.254 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:36.254 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923279+0000", 2024-09-21T21:42:36.254 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.77e-07 2024-09-21T21:42:36.254 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.254 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.254 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:36.254 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923281+0000", 2024-09-21T21:42:36.254 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0219999999999999e-06 2024-09-21T21:42:36.254 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.254 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.255 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:36.255 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923287+0000", 2024-09-21T21:42:36.255 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7799999999999997e-06 2024-09-21T21:42:36.255 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.255 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.255 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:36.255 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923291+0000", 2024-09-21T21:42:36.255 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1699999999999999e-06 2024-09-21T21:42:36.255 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.255 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.255 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:36.255 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966742+0000", 2024-09-21T21:42:36.255 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043451344000000003 2024-09-21T21:42:36.255 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.255 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.256 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:36.256 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966809+0000", 2024-09-21T21:42:36.256 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7070999999999995e-05 2024-09-21T21:42:36.256 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.256 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.256 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:36.256 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966909+0000", 2024-09-21T21:42:36.256 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000100307 2024-09-21T21:42:36.256 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.256 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:36.256 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.256 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.256 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.256 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7010 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5498880~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:36.256 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923301+0000", 2024-09-21T21:42:36.257 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 66.255552528999999, 2024-09-21T21:42:36.257 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043717830999999999, 2024-09-21T21:42:36.257 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:36.257 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:36.257 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:36.257 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:36.257 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:36.257 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:36.257 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7010 2024-09-21T21:42:36.257 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.257 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:36.257 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.257 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:36.257 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923301+0000", 2024-09-21T21:42:36.257 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:36.258 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.258 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.258 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:36.258 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923301+0000", 2024-09-21T21:42:36.258 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:36.258 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.258 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.258 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:36.258 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923302+0000", 2024-09-21T21:42:36.258 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.41e-07 2024-09-21T21:42:36.258 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.258 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.258 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:36.258 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923304+0000", 2024-09-21T21:42:36.258 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0990000000000001e-06 2024-09-21T21:42:36.259 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.259 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.259 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:36.259 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923310+0000", 2024-09-21T21:42:36.259 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3860000000000004e-06 2024-09-21T21:42:36.259 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.259 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.259 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:36.259 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923315+0000", 2024-09-21T21:42:36.259 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8459999999999999e-06 2024-09-21T21:42:36.259 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.259 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.259 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:36.259 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966938+0000", 2024-09-21T21:42:36.259 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043622693999999997 2024-09-21T21:42:36.260 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.260 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.260 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:36.260 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966975+0000", 2024-09-21T21:42:36.260 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7608999999999999e-05 2024-09-21T21:42:36.260 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.260 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.260 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:36.260 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.967019+0000", 2024-09-21T21:42:36.260 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3856000000000002e-05 2024-09-21T21:42:36.260 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.260 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:36.260 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.261 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.261 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.261 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7354 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 2107392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:36.261 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.936932+0000", 2024-09-21T21:42:36.261 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 66.241921673999997, 2024-09-21T21:42:36.261 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041607708, 2024-09-21T21:42:36.261 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:36.261 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:36.261 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:36.261 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:36.261 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:36.261 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:36.261 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7354 2024-09-21T21:42:36.261 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.261 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:36.262 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.262 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:36.262 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.936932+0000", 2024-09-21T21:42:36.262 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:36.262 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.262 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.262 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:36.262 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.936932+0000", 2024-09-21T21:42:36.262 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:36.262 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.262 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.262 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:36.262 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.937276+0000", 2024-09-21T21:42:36.262 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00034344200000000001 2024-09-21T21:42:36.263 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.263 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.263 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:36.263 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.937279+0000", 2024-09-21T21:42:36.263 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.844e-06 2024-09-21T21:42:36.263 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.263 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.263 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:36.263 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.937288+0000", 2024-09-21T21:42:36.263 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2409999999999995e-06 2024-09-21T21:42:36.263 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.263 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.263 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:36.263 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.937293+0000", 2024-09-21T21:42:36.263 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6559999999999997e-06 2024-09-21T21:42:36.264 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.264 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.264 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:36.264 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.978448+0000", 2024-09-21T21:42:36.264 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041154621000000002 2024-09-21T21:42:36.264 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.264 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.264 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:36.264 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.978487+0000", 2024-09-21T21:42:36.264 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9254000000000001e-05 2024-09-21T21:42:36.264 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.264 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.265 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:36.266 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.978540+0000", 2024-09-21T21:42:36.266 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2649999999999999e-05 2024-09-21T21:42:36.266 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.266 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:36.266 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.266 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.266 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.266 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5215.0:1054 3.9 3:946d2401:::rbd_data.1121524d2522.000000000000000b:head [set-alloc-hint object_size 16777216 write_size 16777216,write 6615040~59392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-21T21:42:36.266 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:38.148979+0000", 2024-09-21T21:42:36.266 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 58.029874706999998, 2024-09-21T21:42:36.266 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047870388999999999, 2024-09-21T21:42:36.267 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:36.267 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:36.267 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:36.267 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:36.267 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5215", 2024-09-21T21:42:36.267 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1580282168", 2024-09-21T21:42:36.267 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1054 2024-09-21T21:42:36.267 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.267 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:36.267 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.267 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:36.267 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.148979+0000", 2024-09-21T21:42:36.267 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:36.267 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.268 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.268 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:36.268 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.148979+0000", 2024-09-21T21:42:36.268 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:36.268 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.268 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.268 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:36.268 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.148980+0000", 2024-09-21T21:42:36.268 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5799999999999999e-07 2024-09-21T21:42:36.268 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.268 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.268 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:36.268 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.149039+0000", 2024-09-21T21:42:36.269 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9151000000000003e-05 2024-09-21T21:42:36.269 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.269 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.269 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:36.269 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.149047+0000", 2024-09-21T21:42:36.269 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1000000000000004e-06 2024-09-21T21:42:36.269 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.269 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.269 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:36.269 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.149052+0000", 2024-09-21T21:42:36.269 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5959999999999997e-06 2024-09-21T21:42:36.269 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.269 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.269 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:36.270 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.195288+0000", 2024-09-21T21:42:36.270 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.046236248000000001 2024-09-21T21:42:36.270 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.270 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.270 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:36.270 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.195334+0000", 2024-09-21T21:42:36.270 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6575e-05 2024-09-21T21:42:36.270 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.270 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.270 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 2", 2024-09-21T21:42:36.270 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.195406+0000", 2024-09-21T21:42:36.270 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1715e-05 2024-09-21T21:42:36.270 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.270 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.270 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:36.270 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.196018+0000", 2024-09-21T21:42:36.271 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00061218999999999998 2024-09-21T21:42:36.271 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.271 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.271 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:36.271 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.196794+0000", 2024-09-21T21:42:36.271 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000775814 2024-09-21T21:42:36.271 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.271 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.271 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:36.271 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.196827+0000", 2024-09-21T21:42:36.271 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2419000000000003e-05 2024-09-21T21:42:36.271 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.271 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.271 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:36.271 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.196849+0000", 2024-09-21T21:42:36.271 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2923000000000001e-05 2024-09-21T21:42:36.272 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.272 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:36.272 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.272 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.272 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:36.272 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:36.273 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T21:42:36.274 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:36.272+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:36.374 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:36.372+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:36.424 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:36.424 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:42:36.424 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.424 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4385.0:25371 3.f 3:f6a9dd44:::rbd_header.11214027af96:head [watch watch cookie 140325441068688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-09-21T21:42:36.424 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:36.423273+0000", 2024-09-21T21:42:36.424 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 7.7018e-05, 2024-09-21T21:42:36.424 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3426999999999998e-05, 2024-09-21T21:42:36.424 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:36.425 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:36.425 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2024-09-21T21:42:36.425 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:36.425 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4385", 2024-09-21T21:42:36.425 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:42:36.425 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 25371 2024-09-21T21:42:36.425 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.425 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:36.425 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.425 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:36.425 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:36.423273+0000", 2024-09-21T21:42:36.425 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:36.425 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.426 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.426 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:36.426 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:36.423273+0000", 2024-09-21T21:42:36.426 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:36.426 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.426 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.426 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:36.426 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:36.423273+0000", 2024-09-21T21:42:36.426 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7000000000000005e-07 2024-09-21T21:42:36.426 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.426 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.426 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:36.426 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:36.423276+0000", 2024-09-21T21:42:36.427 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6989999999999999e-06 2024-09-21T21:42:36.427 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.427 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.427 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:36.427 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:36.423283+0000", 2024-09-21T21:42:36.427 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.674e-06 2024-09-21T21:42:36.427 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.427 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.427 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:36.427 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:36.423287+0000", 2024-09-21T21:42:36.427 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7249999999999999e-06 2024-09-21T21:42:36.427 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.427 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.427 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:36.427 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:36.423323+0000", 2024-09-21T21:42:36.428 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6328999999999997e-05 2024-09-21T21:42:36.428 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.428 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:36.428 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.428 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.428 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-21T21:42:36.428 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 1 2024-09-21T21:42:36.428 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:36.434 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T21:42:36.478 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:36.477+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:36.542 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:36.542 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:42:36.542 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:42:36.542 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:42:36.542 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:36.551 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T21:42:36.559 INFO:teuthology.orchestra.run.smithi081.stderr:noscrub is set 2024-09-21T21:42:36.591 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:36.590+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:36.676 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:36.674+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:36.705 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:36.705 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-21T21:42:36.705 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-21T21:42:36.705 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:42:36.705 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.705 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4385.0:15202 3.16 3:6888948b:::rbd_header.1121493f6459:head [watch reconnect cookie 140325440202496 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-21T21:42:36.705 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:41.736601+0000", 2024-09-21T21:42:36.705 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 54.965479418999998, 2024-09-21T21:42:36.705 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5040056769999999, 2024-09-21T21:42:36.705 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:36.705 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:36.706 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:36.706 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:36.706 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4385", 2024-09-21T21:42:36.706 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:42:36.706 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 15202 2024-09-21T21:42:36.706 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.706 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:36.706 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.706 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:36.706 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736601+0000", 2024-09-21T21:42:36.706 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:36.706 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.706 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.707 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:36.707 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736601+0000", 2024-09-21T21:42:36.707 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:36.707 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.707 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.707 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:36.707 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736602+0000", 2024-09-21T21:42:36.707 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0699999999999999e-06 2024-09-21T21:42:36.707 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.707 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.707 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:36.707 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736607+0000", 2024-09-21T21:42:36.707 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2520000000000001e-06 2024-09-21T21:42:36.707 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.708 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.708 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:36.708 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736617+0000", 2024-09-21T21:42:36.708 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0074e-05 2024-09-21T21:42:36.708 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.708 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.708 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:36.708 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736622+0000", 2024-09-21T21:42:36.708 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.101e-06 2024-09-21T21:42:36.708 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.708 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.708 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:36.708 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736664+0000", 2024-09-21T21:42:36.708 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1869999999999997e-05 2024-09-21T21:42:36.708 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.708 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.709 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op must wait for map", 2024-09-21T21:42:36.709 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736667+0000", 2024-09-21T21:42:36.709 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4750000000000002e-06 2024-09-21T21:42:36.709 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.709 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.709 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:36.709 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742886+0000", 2024-09-21T21:42:36.709 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0062187579999999996 2024-09-21T21:42:36.709 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.709 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.709 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:36.709 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742892+0000", 2024-09-21T21:42:36.709 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0440000000000001e-06 2024-09-21T21:42:36.709 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.710 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.710 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:36.710 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210307+0000", 2024-09-21T21:42:36.710 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.467414781 2024-09-21T21:42:36.710 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.710 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.710 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:36.710 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210322+0000", 2024-09-21T21:42:36.710 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5311e-05 2024-09-21T21:42:36.710 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.710 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.710 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:36.710 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.237624+0000", 2024-09-21T21:42:36.710 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0273018920000001 2024-09-21T21:42:36.711 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.711 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.711 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:36.711 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.237870+0000", 2024-09-21T21:42:36.711 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00024622899999999999 2024-09-21T21:42:36.711 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.711 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.711 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:36.711 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.240607+0000", 2024-09-21T21:42:36.711 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0027368200000000001 2024-09-21T21:42:36.711 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.711 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:36.711 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.711 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.711 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.712 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5415.0:379 3.16 3:6888948b:::rbd_header.1121493f6459:head [watch reconnect cookie 139702602617920 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-21T21:42:36.712 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:41.739544+0000", 2024-09-21T21:42:36.712 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 54.962536989999997, 2024-09-21T21:42:36.712 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.501180645, 2024-09-21T21:42:36.712 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:36.712 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:36.712 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:36.712 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:36.712 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5415", 2024-09-21T21:42:36.712 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1212816607", 2024-09-21T21:42:36.712 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 379 2024-09-21T21:42:36.712 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.713 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:36.713 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.713 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:36.713 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739544+0000", 2024-09-21T21:42:36.713 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:36.713 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.713 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.713 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:36.713 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739544+0000", 2024-09-21T21:42:36.713 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:36.713 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.713 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.713 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:36.713 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739545+0000", 2024-09-21T21:42:36.713 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.065e-06 2024-09-21T21:42:36.713 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.714 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.714 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:36.714 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739550+0000", 2024-09-21T21:42:36.714 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9599999999999999e-06 2024-09-21T21:42:36.714 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.714 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.714 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:36.714 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739565+0000", 2024-09-21T21:42:36.714 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5709000000000002e-05 2024-09-21T21:42:36.714 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.714 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.714 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:36.714 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739573+0000", 2024-09-21T21:42:36.715 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7849999999999992e-06 2024-09-21T21:42:36.715 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.715 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.715 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:36.715 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739634+0000", 2024-09-21T21:42:36.715 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0732000000000001e-05 2024-09-21T21:42:36.715 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.715 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.715 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op must wait for map", 2024-09-21T21:42:36.715 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739640+0000", 2024-09-21T21:42:36.715 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0000000000000002e-06 2024-09-21T21:42:36.715 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.715 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.715 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:36.716 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742958+0000", 2024-09-21T21:42:36.716 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0033182390000000002 2024-09-21T21:42:36.716 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.716 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.716 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:36.716 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742966+0000", 2024-09-21T21:42:36.716 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1610000000000003e-06 2024-09-21T21:42:36.716 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.716 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.716 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:36.716 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210405+0000", 2024-09-21T21:42:36.716 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.467438725 2024-09-21T21:42:36.716 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.716 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.716 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:36.717 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210415+0000", 2024-09-21T21:42:36.717 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0234000000000001e-05 2024-09-21T21:42:36.717 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.717 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.717 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:36.717 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.238743+0000", 2024-09-21T21:42:36.717 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0283273070000001 2024-09-21T21:42:36.717 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.717 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.717 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:36.717 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.238842+0000", 2024-09-21T21:42:36.717 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.9896999999999996e-05 2024-09-21T21:42:36.717 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.717 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.718 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:36.718 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.240724+0000", 2024-09-21T21:42:36.718 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001881831 2024-09-21T21:42:36.718 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.718 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:36.718 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.718 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.718 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.718 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4385.0:15203 3.16 3:6888948b:::rbd_header.1121493f6459:head [call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-21T21:42:36.718 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:41.739737+0000", 2024-09-21T21:42:36.718 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 54.962343979000003, 2024-09-21T21:42:36.718 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4989621799999999, 2024-09-21T21:42:36.718 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:36.718 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:36.719 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:36.719 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:36.719 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4385", 2024-09-21T21:42:36.719 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:42:36.719 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 15203 2024-09-21T21:42:36.719 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.719 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:36.719 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.719 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:36.719 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739737+0000", 2024-09-21T21:42:36.719 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:36.719 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.719 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.720 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:36.720 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739737+0000", 2024-09-21T21:42:36.720 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:36.720 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.720 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.720 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:36.720 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739738+0000", 2024-09-21T21:42:36.720 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.7499999999999998e-07 2024-09-21T21:42:36.720 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.720 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.720 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:36.720 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739743+0000", 2024-09-21T21:42:36.720 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5049999999999998e-06 2024-09-21T21:42:36.720 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.721 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.721 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:36.721 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739757+0000", 2024-09-21T21:42:36.721 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3947e-05 2024-09-21T21:42:36.721 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.721 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.721 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:36.721 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739765+0000", 2024-09-21T21:42:36.721 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9310000000000008e-06 2024-09-21T21:42:36.721 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.721 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.721 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:36.721 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739816+0000", 2024-09-21T21:42:36.721 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0559999999999997e-05 2024-09-21T21:42:36.721 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.722 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.722 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting_for_map not empty", 2024-09-21T21:42:36.722 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739821+0000", 2024-09-21T21:42:36.722 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3539999999999999e-06 2024-09-21T21:42:36.722 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.722 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.722 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:36.722 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742928+0000", 2024-09-21T21:42:36.722 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0031074399999999999 2024-09-21T21:42:36.722 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.722 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.722 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:36.722 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742937+0000", 2024-09-21T21:42:36.723 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1289999999999996e-06 2024-09-21T21:42:36.723 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.723 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.723 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:36.723 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210350+0000", 2024-09-21T21:42:36.723 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.46741311299999999 2024-09-21T21:42:36.723 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.723 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.723 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:36.723 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210358+0000", 2024-09-21T21:42:36.723 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1869999999999997e-06 2024-09-21T21:42:36.723 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.723 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.723 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:36.723 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.238398+0000", 2024-09-21T21:42:36.724 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0280397830000001 2024-09-21T21:42:36.724 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.724 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.724 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:36.724 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.238489+0000", 2024-09-21T21:42:36.724 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.0958000000000006e-05 2024-09-21T21:42:36.724 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.724 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.724 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:36.724 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.238699+0000", 2024-09-21T21:42:36.724 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00021029800000000001 2024-09-21T21:42:36.724 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.724 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:36.724 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.725 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.725 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.725 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4385.0:15204 3.16 3:6888948b:::rbd_header.1121493f6459:head [watch reconnect cookie 140325440202496 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-21T21:42:36.725 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:42.210726+0000", 2024-09-21T21:42:36.725 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 54.491355101000003, 2024-09-21T21:42:36.725 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.030304557, 2024-09-21T21:42:36.725 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:36.725 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:36.725 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:36.725 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:36.725 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4385", 2024-09-21T21:42:36.725 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:42:36.725 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 15204 2024-09-21T21:42:36.725 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.725 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:36.726 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.726 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:36.726 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210726+0000", 2024-09-21T21:42:36.726 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:36.726 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.726 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.726 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:36.726 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210726+0000", 2024-09-21T21:42:36.726 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:36.726 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.726 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.726 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:36.726 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210727+0000", 2024-09-21T21:42:36.726 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.204e-06 2024-09-21T21:42:36.727 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.727 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.727 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:36.727 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210732+0000", 2024-09-21T21:42:36.727 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5249999999999996e-06 2024-09-21T21:42:36.727 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.727 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.727 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:36.727 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210744+0000", 2024-09-21T21:42:36.727 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1405e-05 2024-09-21T21:42:36.727 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.727 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.727 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:36.727 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210754+0000", 2024-09-21T21:42:36.727 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0122999999999999e-05 2024-09-21T21:42:36.728 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.728 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.728 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:36.728 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210798+0000", 2024-09-21T21:42:36.728 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4005000000000003e-05 2024-09-21T21:42:36.728 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.728 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.728 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:36.728 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210803+0000", 2024-09-21T21:42:36.728 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5130000000000004e-06 2024-09-21T21:42:36.728 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.728 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.728 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:36.728 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.239260+0000", 2024-09-21T21:42:36.728 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0284563040000001 2024-09-21T21:42:36.729 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.729 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.729 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:36.729 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.239318+0000", 2024-09-21T21:42:36.729 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8486000000000002e-05 2024-09-21T21:42:36.729 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.729 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.729 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:36.729 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.241030+0000", 2024-09-21T21:42:36.729 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0017119920000000001 2024-09-21T21:42:36.729 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.729 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:36.729 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.729 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.729 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.730 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5415.0:380 3.16 3:6888948b:::rbd_header.1121493f6459:head [watch reconnect cookie 139702602617920 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-21T21:42:36.730 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:42.210777+0000", 2024-09-21T21:42:36.730 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 54.491303322, 2024-09-21T21:42:36.730 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.030410566, 2024-09-21T21:42:36.730 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:36.730 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:36.730 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:36.730 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:36.730 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5415", 2024-09-21T21:42:36.730 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1212816607", 2024-09-21T21:42:36.730 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 380 2024-09-21T21:42:36.730 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.730 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:36.730 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.730 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:36.731 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210777+0000", 2024-09-21T21:42:36.731 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:36.731 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.731 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.731 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:36.731 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210777+0000", 2024-09-21T21:42:36.731 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:36.731 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.731 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.731 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:36.731 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210778+0000", 2024-09-21T21:42:36.731 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8999999999999996e-07 2024-09-21T21:42:36.731 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.731 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.731 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:36.732 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210780+0000", 2024-09-21T21:42:36.732 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.503e-06 2024-09-21T21:42:36.732 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.732 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.732 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:36.732 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210788+0000", 2024-09-21T21:42:36.732 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0999999999999998e-06 2024-09-21T21:42:36.732 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.732 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.732 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:36.732 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210799+0000", 2024-09-21T21:42:36.732 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1703e-05 2024-09-21T21:42:36.732 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.732 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.733 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:36.733 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210833+0000", 2024-09-21T21:42:36.733 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4008999999999999e-05 2024-09-21T21:42:36.733 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.733 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.733 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:36.733 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210838+0000", 2024-09-21T21:42:36.733 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7480000000000001e-06 2024-09-21T21:42:36.733 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.733 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.733 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:36.733 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.239612+0000", 2024-09-21T21:42:36.733 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.028773554 2024-09-21T21:42:36.733 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.734 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.734 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:36.734 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.239677+0000", 2024-09-21T21:42:36.734 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5053000000000002e-05 2024-09-21T21:42:36.734 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.734 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.734 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:36.734 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.241188+0000", 2024-09-21T21:42:36.734 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001511306 2024-09-21T21:42:36.734 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.734 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:36.734 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.734 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.734 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.734 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24794 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:36.735 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.584085+0000", 2024-09-21T21:42:36.735 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 14.117996145999999, 2024-09-21T21:42:36.735 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.096209996000000006, 2024-09-21T21:42:36.735 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:36.735 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:36.735 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:36.735 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:36.735 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:42:36.735 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:36.735 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24794 2024-09-21T21:42:36.735 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.735 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:36.735 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.736 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:36.736 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584085+0000", 2024-09-21T21:42:36.736 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:36.736 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.736 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.736 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:36.736 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584085+0000", 2024-09-21T21:42:36.736 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:36.736 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.736 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.736 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:36.736 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584085+0000", 2024-09-21T21:42:36.736 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3200000000000001e-07 2024-09-21T21:42:36.736 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.736 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.737 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:36.737 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584086+0000", 2024-09-21T21:42:36.737 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.717e-06 2024-09-21T21:42:36.737 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.737 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.737 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:36.737 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584090+0000", 2024-09-21T21:42:36.737 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3450000000000002e-06 2024-09-21T21:42:36.737 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.737 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.737 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:36.737 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584092+0000", 2024-09-21T21:42:36.737 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6359999999999998e-06 2024-09-21T21:42:36.737 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.737 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.738 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:36.738 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.676497+0000", 2024-09-21T21:42:36.738 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.092404659 2024-09-21T21:42:36.738 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.738 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.738 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:36.738 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.676567+0000", 2024-09-21T21:42:36.738 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9627000000000003e-05 2024-09-21T21:42:36.738 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.738 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.738 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:42:36.738 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.676925+0000", 2024-09-21T21:42:36.738 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00035848500000000002 2024-09-21T21:42:36.738 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.738 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.739 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:36.739 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.678427+0000", 2024-09-21T21:42:36.739 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001501638 2024-09-21T21:42:36.739 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.739 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.739 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:36.739 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.680204+0000", 2024-09-21T21:42:36.739 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0017766979999999999 2024-09-21T21:42:36.739 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.739 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.739 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:36.739 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.680256+0000", 2024-09-21T21:42:36.739 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2549999999999997e-05 2024-09-21T21:42:36.739 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.740 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.740 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:36.740 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.680295+0000", 2024-09-21T21:42:36.740 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8408999999999998e-05 2024-09-21T21:42:36.740 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.740 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:36.740 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.740 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.740 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.740 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24796 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:36.740 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.584099+0000", 2024-09-21T21:42:36.740 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 14.117981577, 2024-09-21T21:42:36.740 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.097297429000000005, 2024-09-21T21:42:36.740 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:36.740 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:36.741 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:36.741 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:36.741 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:42:36.741 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:36.741 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24796 2024-09-21T21:42:36.741 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.741 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:36.741 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.741 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:36.741 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584099+0000", 2024-09-21T21:42:36.741 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:36.741 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.741 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.741 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:36.741 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584099+0000", 2024-09-21T21:42:36.742 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:36.742 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.742 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.742 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:36.742 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584099+0000", 2024-09-21T21:42:36.742 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3999999999999998e-07 2024-09-21T21:42:36.742 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.742 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.742 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:36.742 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584101+0000", 2024-09-21T21:42:36.742 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5850000000000001e-06 2024-09-21T21:42:36.742 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.742 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.742 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:36.742 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584108+0000", 2024-09-21T21:42:36.743 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.498e-06 2024-09-21T21:42:36.743 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.743 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.743 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:36.743 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584112+0000", 2024-09-21T21:42:36.743 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.252e-06 2024-09-21T21:42:36.743 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.743 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.743 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:36.743 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.677336+0000", 2024-09-21T21:42:36.743 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.093224206000000004 2024-09-21T21:42:36.744 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.744 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.744 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:36.744 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.677411+0000", 2024-09-21T21:42:36.744 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5036000000000005e-05 2024-09-21T21:42:36.744 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.744 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.744 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:42:36.744 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.677782+0000", 2024-09-21T21:42:36.744 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00037153099999999998 2024-09-21T21:42:36.744 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.744 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.744 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:36.745 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.680360+0000", 2024-09-21T21:42:36.745 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025773020000000001 2024-09-21T21:42:36.745 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.745 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.745 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:36.745 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681313+0000", 2024-09-21T21:42:36.745 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00095341100000000004 2024-09-21T21:42:36.745 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.745 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.745 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:36.745 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681361+0000", 2024-09-21T21:42:36.745 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7497999999999998e-05 2024-09-21T21:42:36.745 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.745 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.746 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:36.746 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681397+0000", 2024-09-21T21:42:36.746 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.587e-05 2024-09-21T21:42:36.746 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.746 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:36.746 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.746 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.746 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.746 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24797 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:36.746 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.584116+0000", 2024-09-21T21:42:36.746 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 14.117965088, 2024-09-21T21:42:36.746 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.097428257000000004, 2024-09-21T21:42:36.746 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:36.746 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:36.747 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:36.747 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:36.747 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:42:36.747 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:36.747 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24797 2024-09-21T21:42:36.747 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.747 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:36.747 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.747 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:36.747 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584116+0000", 2024-09-21T21:42:36.747 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:36.747 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.747 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.748 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:36.748 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584116+0000", 2024-09-21T21:42:36.748 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:36.748 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.748 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.748 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:36.748 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584116+0000", 2024-09-21T21:42:36.748 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.53e-07 2024-09-21T21:42:36.748 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.748 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.748 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:36.748 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584118+0000", 2024-09-21T21:42:36.748 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6810000000000001e-06 2024-09-21T21:42:36.748 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.749 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.749 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:36.749 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584121+0000", 2024-09-21T21:42:36.749 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1099999999999999e-06 2024-09-21T21:42:36.749 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.749 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.749 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:36.749 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584123+0000", 2024-09-21T21:42:36.749 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5890000000000001e-06 2024-09-21T21:42:36.749 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.749 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.749 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:36.749 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.678476+0000", 2024-09-21T21:42:36.749 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.094352320000000003 2024-09-21T21:42:36.750 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.750 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.750 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:36.750 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.678556+0000", 2024-09-21T21:42:36.750 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0291999999999996e-05 2024-09-21T21:42:36.750 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.750 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.750 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:42:36.750 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.678871+0000", 2024-09-21T21:42:36.750 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000314689 2024-09-21T21:42:36.750 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.750 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.750 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:36.751 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681452+0000", 2024-09-21T21:42:36.751 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025817890000000001 2024-09-21T21:42:36.751 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.751 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.751 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:36.751 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681467+0000", 2024-09-21T21:42:36.751 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4256e-05 2024-09-21T21:42:36.751 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.751 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.751 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:36.751 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681510+0000", 2024-09-21T21:42:36.751 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3560999999999998e-05 2024-09-21T21:42:36.751 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.751 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.751 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:36.752 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681544+0000", 2024-09-21T21:42:36.752 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3717000000000003e-05 2024-09-21T21:42:36.752 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.752 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:36.752 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.752 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.752 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.752 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24798 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:36.752 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.584130+0000", 2024-09-21T21:42:36.752 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 14.117950564999999, 2024-09-21T21:42:36.752 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.098350526999999993, 2024-09-21T21:42:36.752 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:36.752 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:36.752 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:36.752 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:36.753 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:42:36.753 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:36.753 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24798 2024-09-21T21:42:36.753 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.753 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:36.753 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.753 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:36.753 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584130+0000", 2024-09-21T21:42:36.753 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:36.753 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.753 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.753 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:36.753 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584130+0000", 2024-09-21T21:42:36.753 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:36.753 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.754 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.754 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:36.754 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584131+0000", 2024-09-21T21:42:36.754 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4700000000000001e-07 2024-09-21T21:42:36.754 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.754 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.754 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:36.754 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584133+0000", 2024-09-21T21:42:36.754 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.34e-06 2024-09-21T21:42:36.754 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.754 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.754 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:36.754 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584137+0000", 2024-09-21T21:42:36.754 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5609999999999999e-06 2024-09-21T21:42:36.755 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.755 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.755 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:36.755 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584139+0000", 2024-09-21T21:42:36.755 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7350000000000001e-06 2024-09-21T21:42:36.755 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.755 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.755 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:36.755 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.679274+0000", 2024-09-21T21:42:36.755 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.095134719000000006 2024-09-21T21:42:36.755 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.755 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.755 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:36.756 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.679358+0000", 2024-09-21T21:42:36.756 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3609999999999994e-05 2024-09-21T21:42:36.756 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.756 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.756 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:42:36.756 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.679685+0000", 2024-09-21T21:42:36.756 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00032732399999999999 2024-09-21T21:42:36.756 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.756 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.756 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:36.756 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681553+0000", 2024-09-21T21:42:36.756 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001868059 2024-09-21T21:42:36.756 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.756 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.757 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:36.757 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.682375+0000", 2024-09-21T21:42:36.757 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00082232099999999999 2024-09-21T21:42:36.757 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.757 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.757 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:36.757 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.682431+0000", 2024-09-21T21:42:36.757 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6051999999999998e-05 2024-09-21T21:42:36.757 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.757 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.757 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:36.757 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.682481+0000", 2024-09-21T21:42:36.757 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9258999999999997e-05 2024-09-21T21:42:36.757 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.757 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:36.758 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.758 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.758 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.758 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24799 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:36.758 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.584159+0000", 2024-09-21T21:42:36.758 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 14.117921577000001, 2024-09-21T21:42:36.758 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.100021551, 2024-09-21T21:42:36.758 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:36.758 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:36.758 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:36.758 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:36.758 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:42:36.758 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:36.759 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24799 2024-09-21T21:42:36.759 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.759 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:36.759 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.759 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:36.759 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584159+0000", 2024-09-21T21:42:36.759 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:36.759 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.759 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.759 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:36.759 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584159+0000", 2024-09-21T21:42:36.759 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:36.759 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.759 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.759 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:36.760 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584160+0000", 2024-09-21T21:42:36.760 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1300000000000001e-07 2024-09-21T21:42:36.760 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.760 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.760 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:36.760 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584162+0000", 2024-09-21T21:42:36.760 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5409999999999999e-06 2024-09-21T21:42:36.760 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.760 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.760 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:36.760 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584169+0000", 2024-09-21T21:42:36.760 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7880000000000001e-06 2024-09-21T21:42:36.760 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.760 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.760 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:36.761 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584174+0000", 2024-09-21T21:42:36.761 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9590000000000003e-06 2024-09-21T21:42:36.761 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.761 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.761 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:36.761 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.680400+0000", 2024-09-21T21:42:36.761 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.096226514999999999 2024-09-21T21:42:36.761 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.761 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.761 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:36.761 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.680480+0000", 2024-09-21T21:42:36.761 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9589e-05 2024-09-21T21:42:36.761 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.761 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.762 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:42:36.762 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.680847+0000", 2024-09-21T21:42:36.762 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00036757700000000001 2024-09-21T21:42:36.762 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.762 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.762 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:36.762 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.682555+0000", 2024-09-21T21:42:36.762 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001707791 2024-09-21T21:42:36.762 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.762 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.762 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:36.762 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684093+0000", 2024-09-21T21:42:36.762 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0015377170000000001 2024-09-21T21:42:36.762 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.763 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.763 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:36.763 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684144+0000", 2024-09-21T21:42:36.763 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1208000000000002e-05 2024-09-21T21:42:36.763 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.763 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.763 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:36.763 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684181+0000", 2024-09-21T21:42:36.763 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6452999999999998e-05 2024-09-21T21:42:36.763 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.763 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:36.763 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.763 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.763 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.763 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24835 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:36.763 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586332+0000", 2024-09-21T21:42:36.764 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 14.115748963, 2024-09-21T21:42:36.764 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.098730360000000003, 2024-09-21T21:42:36.764 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:36.764 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:36.764 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:36.764 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:36.764 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:42:36.764 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:36.764 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24835 2024-09-21T21:42:36.764 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.764 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:36.764 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.764 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:36.764 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586332+0000", 2024-09-21T21:42:36.764 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:36.765 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.765 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.765 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:36.765 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586332+0000", 2024-09-21T21:42:36.765 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:36.765 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.765 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.765 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:36.765 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586333+0000", 2024-09-21T21:42:36.765 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0200000000000001e-07 2024-09-21T21:42:36.765 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.765 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.765 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:36.765 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586346+0000", 2024-09-21T21:42:36.766 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3745e-05 2024-09-21T21:42:36.766 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.766 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.766 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:36.766 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586366+0000", 2024-09-21T21:42:36.766 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9272000000000001e-05 2024-09-21T21:42:36.766 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.766 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.766 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:36.766 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586377+0000", 2024-09-21T21:42:36.766 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1197e-05 2024-09-21T21:42:36.766 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.766 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.767 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:36.767 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681578+0000", 2024-09-21T21:42:36.767 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.095201420999999994 2024-09-21T21:42:36.767 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.767 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.767 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:36.767 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681640+0000", 2024-09-21T21:42:36.767 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1834000000000005e-05 2024-09-21T21:42:36.767 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.767 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.767 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:42:36.767 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681915+0000", 2024-09-21T21:42:36.768 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00027534199999999998 2024-09-21T21:42:36.768 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.768 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.768 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:36.768 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684230+0000", 2024-09-21T21:42:36.768 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00231424 2024-09-21T21:42:36.768 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.768 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.768 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:36.768 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684959+0000", 2024-09-21T21:42:36.768 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00072921399999999995 2024-09-21T21:42:36.768 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.768 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.769 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:36.769 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.685008+0000", 2024-09-21T21:42:36.769 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8860999999999999e-05 2024-09-21T21:42:36.769 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.769 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.769 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:36.769 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.685062+0000", 2024-09-21T21:42:36.769 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4432000000000003e-05 2024-09-21T21:42:36.769 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.769 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:36.769 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.769 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.769 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.769 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24837 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:36.769 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586395+0000", 2024-09-21T21:42:36.770 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 14.115686028000001, 2024-09-21T21:42:36.770 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.099700522, 2024-09-21T21:42:36.770 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:36.770 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:36.770 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:36.770 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:36.770 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:42:36.770 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:36.770 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24837 2024-09-21T21:42:36.770 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.770 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:36.770 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.771 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:36.771 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586395+0000", 2024-09-21T21:42:36.771 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:36.771 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.771 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.771 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:36.771 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586395+0000", 2024-09-21T21:42:36.771 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:36.771 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.771 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.771 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:36.771 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586395+0000", 2024-09-21T21:42:36.771 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.06e-07 2024-09-21T21:42:36.771 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.772 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.772 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:36.772 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586401+0000", 2024-09-21T21:42:36.772 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0889999999999996e-06 2024-09-21T21:42:36.772 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.772 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.772 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:36.772 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586410+0000", 2024-09-21T21:42:36.772 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.918e-06 2024-09-21T21:42:36.772 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.772 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.773 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:36.773 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586424+0000", 2024-09-21T21:42:36.773 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3478999999999999e-05 2024-09-21T21:42:36.773 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.773 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.773 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:36.773 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.682590+0000", 2024-09-21T21:42:36.773 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.096166086999999997 2024-09-21T21:42:36.773 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.773 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.773 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:36.773 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.682657+0000", 2024-09-21T21:42:36.773 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6878000000000006e-05 2024-09-21T21:42:36.773 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.774 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.774 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:42:36.774 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.682879+0000", 2024-09-21T21:42:36.774 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00022238 2024-09-21T21:42:36.774 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.774 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.774 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:36.774 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.685073+0000", 2024-09-21T21:42:36.774 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021938119999999998 2024-09-21T21:42:36.774 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.774 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.774 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:36.774 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686038+0000", 2024-09-21T21:42:36.774 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00096539499999999997 2024-09-21T21:42:36.775 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.775 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.775 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:36.775 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686071+0000", 2024-09-21T21:42:36.775 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2514000000000001e-05 2024-09-21T21:42:36.775 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.775 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.775 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:36.775 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686095+0000", 2024-09-21T21:42:36.775 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4363999999999999e-05 2024-09-21T21:42:36.775 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.775 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:36.775 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.775 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.775 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.776 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24839 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:36.776 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586468+0000", 2024-09-21T21:42:36.776 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 14.115612376, 2024-09-21T21:42:36.776 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.099739355000000002, 2024-09-21T21:42:36.776 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:36.776 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:36.776 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:36.776 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:36.776 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:42:36.776 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:36.776 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24839 2024-09-21T21:42:36.776 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.776 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:36.776 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.776 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:36.777 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586468+0000", 2024-09-21T21:42:36.777 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:36.777 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.777 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.777 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:36.777 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586468+0000", 2024-09-21T21:42:36.777 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:36.777 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.777 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.777 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:36.777 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586469+0000", 2024-09-21T21:42:36.777 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2200000000000004e-07 2024-09-21T21:42:36.777 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.777 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.778 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:36.778 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586472+0000", 2024-09-21T21:42:36.778 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9960000000000002e-06 2024-09-21T21:42:36.778 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.778 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.778 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:36.778 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586479+0000", 2024-09-21T21:42:36.778 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2699999999999999e-06 2024-09-21T21:42:36.778 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.778 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.778 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:36.778 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586485+0000", 2024-09-21T21:42:36.778 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6999999999999996e-06 2024-09-21T21:42:36.778 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.778 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.779 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:36.779 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.683276+0000", 2024-09-21T21:42:36.779 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.096791618999999995 2024-09-21T21:42:36.779 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.779 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.779 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:36.779 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.683326+0000", 2024-09-21T21:42:36.779 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9384e-05 2024-09-21T21:42:36.779 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.779 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.779 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:42:36.779 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.683655+0000", 2024-09-21T21:42:36.779 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00032965700000000002 2024-09-21T21:42:36.780 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.780 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.780 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:36.780 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686120+0000", 2024-09-21T21:42:36.780 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002464819 2024-09-21T21:42:36.780 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.780 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.780 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:36.780 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686159+0000", 2024-09-21T21:42:36.780 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9181000000000003e-05 2024-09-21T21:42:36.780 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.780 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.780 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:36.780 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686185+0000", 2024-09-21T21:42:36.781 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5907000000000001e-05 2024-09-21T21:42:36.781 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.781 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.781 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:36.781 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686208+0000", 2024-09-21T21:42:36.781 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.23e-05 2024-09-21T21:42:36.781 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.781 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:36.781 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.781 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.781 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.781 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24841 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:36.781 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586492+0000", 2024-09-21T21:42:36.781 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 14.115588214000001, 2024-09-21T21:42:36.782 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.101672077, 2024-09-21T21:42:36.782 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:36.782 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:36.782 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:36.782 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:36.782 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:42:36.782 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:36.782 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24841 2024-09-21T21:42:36.782 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.782 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:36.782 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.782 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:36.782 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586492+0000", 2024-09-21T21:42:36.783 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:36.783 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.783 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.783 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:36.783 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586492+0000", 2024-09-21T21:42:36.783 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:36.783 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.783 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.783 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:36.783 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586493+0000", 2024-09-21T21:42:36.783 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5999999999999999e-07 2024-09-21T21:42:36.783 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.783 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.783 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:36.784 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586496+0000", 2024-09-21T21:42:36.784 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6089999999999999e-06 2024-09-21T21:42:36.784 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.784 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.784 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:36.784 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586502+0000", 2024-09-21T21:42:36.784 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6270000000000002e-06 2024-09-21T21:42:36.784 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.784 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.784 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:36.784 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586507+0000", 2024-09-21T21:42:36.784 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8810000000000004e-06 2024-09-21T21:42:36.784 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.785 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.785 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:36.785 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684273+0000", 2024-09-21T21:42:36.785 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.097766415999999995 2024-09-21T21:42:36.785 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.785 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.785 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:36.785 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684359+0000", 2024-09-21T21:42:36.785 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5574000000000004e-05 2024-09-21T21:42:36.785 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.785 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.785 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:42:36.785 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684568+0000", 2024-09-21T21:42:36.786 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000208529 2024-09-21T21:42:36.786 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.786 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.786 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:36.786 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.687186+0000", 2024-09-21T21:42:36.786 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0026180769999999999 2024-09-21T21:42:36.786 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.786 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.786 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:36.786 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688093+0000", 2024-09-21T21:42:36.786 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00090766999999999996 2024-09-21T21:42:36.786 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.786 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.787 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:36.787 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688142+0000", 2024-09-21T21:42:36.787 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8986000000000002e-05 2024-09-21T21:42:36.787 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.787 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.787 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:36.787 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688165+0000", 2024-09-21T21:42:36.787 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2248000000000001e-05 2024-09-21T21:42:36.787 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.787 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:36.787 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.787 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.787 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.788 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24843 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:36.788 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586514+0000", 2024-09-21T21:42:36.788 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 14.115566928, 2024-09-21T21:42:36.788 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.101826449, 2024-09-21T21:42:36.788 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:36.788 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:36.788 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:36.788 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:36.788 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:42:36.788 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:36.788 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24843 2024-09-21T21:42:36.788 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.788 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:36.789 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.789 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:36.789 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586514+0000", 2024-09-21T21:42:36.789 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:36.789 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.789 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.789 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:36.789 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586514+0000", 2024-09-21T21:42:36.789 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:36.789 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.789 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.789 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:36.789 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586514+0000", 2024-09-21T21:42:36.789 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9999999999999998e-07 2024-09-21T21:42:36.790 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.790 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.790 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:36.790 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586517+0000", 2024-09-21T21:42:36.790 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4980000000000001e-06 2024-09-21T21:42:36.790 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.790 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.790 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:36.790 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586523+0000", 2024-09-21T21:42:36.790 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4609999999999996e-06 2024-09-21T21:42:36.790 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.790 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.790 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:36.790 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586528+0000", 2024-09-21T21:42:36.791 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2979999999999999e-06 2024-09-21T21:42:36.791 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.791 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.791 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:36.791 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.685104+0000", 2024-09-21T21:42:36.791 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.098575837999999999 2024-09-21T21:42:36.791 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.791 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.791 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:36.791 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.685184+0000", 2024-09-21T21:42:36.791 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9807000000000003e-05 2024-09-21T21:42:36.791 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.791 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.791 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:42:36.792 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.685496+0000", 2024-09-21T21:42:36.792 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00031225900000000001 2024-09-21T21:42:36.792 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.792 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.792 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:36.792 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688238+0000", 2024-09-21T21:42:36.792 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0027419409999999999 2024-09-21T21:42:36.792 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.792 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.792 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:36.792 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688253+0000", 2024-09-21T21:42:36.792 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4331000000000001e-05 2024-09-21T21:42:36.792 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.792 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.793 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:36.793 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688296+0000", 2024-09-21T21:42:36.793 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3801000000000003e-05 2024-09-21T21:42:36.793 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.793 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.793 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:36.793 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688340+0000", 2024-09-21T21:42:36.793 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3815e-05 2024-09-21T21:42:36.793 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.793 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:36.793 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.793 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.793 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.793 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24845 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:36.793 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586535+0000", 2024-09-21T21:42:36.794 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 14.115546048000001, 2024-09-21T21:42:36.794 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.10285678600000001, 2024-09-21T21:42:36.794 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:36.794 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:36.794 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:36.794 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:36.794 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:42:36.794 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:36.794 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24845 2024-09-21T21:42:36.794 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.794 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:36.794 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.794 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:36.794 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586535+0000", 2024-09-21T21:42:36.795 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:36.795 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.795 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.795 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:36.795 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586535+0000", 2024-09-21T21:42:36.795 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:36.795 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.795 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.795 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:36.795 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586535+0000", 2024-09-21T21:42:36.795 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9000000000000002e-07 2024-09-21T21:42:36.795 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.795 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.795 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:36.795 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586537+0000", 2024-09-21T21:42:36.795 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4569999999999999e-06 2024-09-21T21:42:36.796 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.796 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.796 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:36.796 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586544+0000", 2024-09-21T21:42:36.796 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5849999999999996e-06 2024-09-21T21:42:36.796 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.796 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.796 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:36.796 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586549+0000", 2024-09-21T21:42:36.796 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0089999999999998e-06 2024-09-21T21:42:36.796 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.796 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.796 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:36.796 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686239+0000", 2024-09-21T21:42:36.796 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.099689531999999997 2024-09-21T21:42:36.796 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.797 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.797 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:36.797 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686289+0000", 2024-09-21T21:42:36.797 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.007e-05 2024-09-21T21:42:36.797 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.797 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.797 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:42:36.797 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686633+0000", 2024-09-21T21:42:36.797 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00034428699999999998 2024-09-21T21:42:36.797 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.797 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.797 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:36.797 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688370+0000", 2024-09-21T21:42:36.797 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001736741 2024-09-21T21:42:36.797 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.798 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.798 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:36.798 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.689295+0000", 2024-09-21T21:42:36.798 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00092517900000000004 2024-09-21T21:42:36.798 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.798 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.798 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:36.798 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.689354+0000", 2024-09-21T21:42:36.798 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8799999999999999e-05 2024-09-21T21:42:36.798 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.798 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.798 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:36.798 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.689391+0000", 2024-09-21T21:42:36.798 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7735999999999999e-05 2024-09-21T21:42:36.799 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.799 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:36.799 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.799 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.799 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.799 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24846 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:36.799 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586564+0000", 2024-09-21T21:42:36.799 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 14.11551702, 2024-09-21T21:42:36.799 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.10375692, 2024-09-21T21:42:36.799 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:36.799 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:36.799 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:36.799 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:36.799 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:42:36.799 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:36.800 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24846 2024-09-21T21:42:36.800 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.800 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:36.800 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.800 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:36.800 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586564+0000", 2024-09-21T21:42:36.800 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:36.800 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.800 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.800 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:36.800 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586564+0000", 2024-09-21T21:42:36.800 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:36.800 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.800 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.800 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:36.801 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586564+0000", 2024-09-21T21:42:36.801 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4799999999999998e-07 2024-09-21T21:42:36.801 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.801 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.801 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:36.801 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586567+0000", 2024-09-21T21:42:36.801 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1669999999999999e-06 2024-09-21T21:42:36.801 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.801 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.801 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:36.801 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586578+0000", 2024-09-21T21:42:36.801 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0443e-05 2024-09-21T21:42:36.801 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.801 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.801 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:36.801 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586583+0000", 2024-09-21T21:42:36.802 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6790000000000002e-06 2024-09-21T21:42:36.802 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.802 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.802 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:36.802 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.687215+0000", 2024-09-21T21:42:36.802 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.100631973 2024-09-21T21:42:36.802 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.802 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.802 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:36.802 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.687311+0000", 2024-09-21T21:42:36.802 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5764e-05 2024-09-21T21:42:36.802 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.802 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.802 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:42:36.803 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.687663+0000", 2024-09-21T21:42:36.803 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00035167 2024-09-21T21:42:36.803 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.803 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.803 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:36.803 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690236+0000", 2024-09-21T21:42:36.803 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025733370000000002 2024-09-21T21:42:36.803 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.803 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.803 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:36.803 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690250+0000", 2024-09-21T21:42:36.803 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3548e-05 2024-09-21T21:42:36.803 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.804 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.804 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:36.804 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690298+0000", 2024-09-21T21:42:36.804 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7948999999999998e-05 2024-09-21T21:42:36.804 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.804 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.804 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:36.804 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690321+0000", 2024-09-21T21:42:36.804 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2841999999999999e-05 2024-09-21T21:42:36.804 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.804 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:36.804 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.804 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.804 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.804 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24849 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:36.805 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586633+0000", 2024-09-21T21:42:36.805 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 14.115447354000001, 2024-09-21T21:42:36.805 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.104506548, 2024-09-21T21:42:36.805 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:36.805 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:36.805 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:36.805 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:36.805 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:42:36.805 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:36.805 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24849 2024-09-21T21:42:36.805 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.805 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:36.805 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.805 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:36.806 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586633+0000", 2024-09-21T21:42:36.806 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:36.806 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.806 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.806 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:36.806 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586633+0000", 2024-09-21T21:42:36.806 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:36.806 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.806 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.806 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:36.806 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586634+0000", 2024-09-21T21:42:36.806 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9100000000000004e-07 2024-09-21T21:42:36.806 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.806 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.806 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:36.807 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586637+0000", 2024-09-21T21:42:36.807 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1999999999999999e-06 2024-09-21T21:42:36.807 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.807 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.807 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:36.807 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586645+0000", 2024-09-21T21:42:36.807 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7759999999999997e-06 2024-09-21T21:42:36.807 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.807 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.807 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:36.807 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586652+0000", 2024-09-21T21:42:36.807 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3560000000000004e-06 2024-09-21T21:42:36.807 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.808 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.808 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:36.808 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688395+0000", 2024-09-21T21:42:36.808 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.101743239 2024-09-21T21:42:36.808 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.808 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.808 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:36.808 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688464+0000", 2024-09-21T21:42:36.808 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8273000000000002e-05 2024-09-21T21:42:36.808 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.808 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.808 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:42:36.808 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688786+0000", 2024-09-21T21:42:36.808 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000322064 2024-09-21T21:42:36.808 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.809 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.809 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:36.809 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690364+0000", 2024-09-21T21:42:36.809 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001578084 2024-09-21T21:42:36.809 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.809 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.809 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:36.809 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.691050+0000", 2024-09-21T21:42:36.809 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00068623399999999998 2024-09-21T21:42:36.809 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.809 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.809 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:36.809 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.691090+0000", 2024-09-21T21:42:36.809 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9406999999999997e-05 2024-09-21T21:42:36.810 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.810 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.810 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:36.810 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.691140+0000", 2024-09-21T21:42:36.810 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0324000000000001e-05 2024-09-21T21:42:36.810 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.810 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:36.810 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.810 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.810 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.810 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24851 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:36.810 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586661+0000", 2024-09-21T21:42:36.810 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 14.115420068000001, 2024-09-21T21:42:36.810 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.10536891399999999, 2024-09-21T21:42:36.810 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:36.810 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:36.811 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:36.811 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:36.811 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:42:36.811 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:36.811 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24851 2024-09-21T21:42:36.811 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.811 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:36.811 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.811 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:36.811 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586661+0000", 2024-09-21T21:42:36.811 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:36.811 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.811 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.811 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:36.812 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586661+0000", 2024-09-21T21:42:36.812 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:36.812 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.812 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.812 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:36.812 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586661+0000", 2024-09-21T21:42:36.812 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4700000000000002e-07 2024-09-21T21:42:36.812 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.812 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.812 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:36.812 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586664+0000", 2024-09-21T21:42:36.812 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.672e-06 2024-09-21T21:42:36.812 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.812 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.812 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:36.813 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586671+0000", 2024-09-21T21:42:36.813 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9959999999999996e-06 2024-09-21T21:42:36.813 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.813 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.813 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:36.813 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586676+0000", 2024-09-21T21:42:36.813 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4020000000000001e-06 2024-09-21T21:42:36.813 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.813 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.813 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:36.813 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.689438+0000", 2024-09-21T21:42:36.813 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.102761503 2024-09-21T21:42:36.813 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.813 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.813 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:36.814 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.689524+0000", 2024-09-21T21:42:36.814 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6383000000000003e-05 2024-09-21T21:42:36.814 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.814 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.814 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:42:36.814 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.689769+0000", 2024-09-21T21:42:36.814 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00024504699999999997 2024-09-21T21:42:36.814 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.814 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.814 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:36.814 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.691936+0000", 2024-09-21T21:42:36.814 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021665130000000001 2024-09-21T21:42:36.814 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.815 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.815 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:36.815 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.691947+0000", 2024-09-21T21:42:36.815 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1198e-05 2024-09-21T21:42:36.815 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.815 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.815 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:36.815 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.691992+0000", 2024-09-21T21:42:36.815 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4774000000000002e-05 2024-09-21T21:42:36.815 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.815 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.815 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:36.815 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.692030+0000", 2024-09-21T21:42:36.815 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7978999999999997e-05 2024-09-21T21:42:36.815 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.816 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:36.816 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.816 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.816 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.816 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24853 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:36.816 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586684+0000", 2024-09-21T21:42:36.816 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 14.115397111, 2024-09-21T21:42:36.816 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.105936852, 2024-09-21T21:42:36.816 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:36.816 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:36.816 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:36.816 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:36.816 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:42:36.817 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:36.817 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24853 2024-09-21T21:42:36.817 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.817 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:36.817 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.817 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:36.817 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586684+0000", 2024-09-21T21:42:36.817 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:36.817 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.817 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.817 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:36.817 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586684+0000", 2024-09-21T21:42:36.817 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:36.817 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.818 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.818 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:36.818 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586684+0000", 2024-09-21T21:42:36.818 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.68e-07 2024-09-21T21:42:36.818 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.818 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.818 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:36.818 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586687+0000", 2024-09-21T21:42:36.818 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.774e-06 2024-09-21T21:42:36.818 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.818 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.818 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:36.818 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586694+0000", 2024-09-21T21:42:36.818 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6959999999999999e-06 2024-09-21T21:42:36.819 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.819 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.819 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:36.819 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586698+0000", 2024-09-21T21:42:36.819 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7489999999999998e-06 2024-09-21T21:42:36.819 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.819 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.819 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:36.819 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690395+0000", 2024-09-21T21:42:36.819 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.103696645 2024-09-21T21:42:36.819 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.819 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.819 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:36.820 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690440+0000", 2024-09-21T21:42:36.820 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4835e-05 2024-09-21T21:42:36.820 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.820 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.820 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:42:36.820 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690654+0000", 2024-09-21T21:42:36.820 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000214114 2024-09-21T21:42:36.820 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.820 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.820 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:36.820 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.692041+0000", 2024-09-21T21:42:36.820 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0013866519999999999 2024-09-21T21:42:36.820 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.820 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.821 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:36.821 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.692558+0000", 2024-09-21T21:42:36.821 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00051717199999999999 2024-09-21T21:42:36.821 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.821 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.821 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:36.821 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.692586+0000", 2024-09-21T21:42:36.821 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8504e-05 2024-09-21T21:42:36.821 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:36.821 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:36.821 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:36.821 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.692620+0000", 2024-09-21T21:42:36.821 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4143000000000002e-05 2024-09-21T21:42:36.821 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.821 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:36.821 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.822 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:36.822 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:36.822 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:36.823 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:36.822+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:36.823 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T21:42:36.924 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:36.923+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:36.932 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:36.932 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:42:36.932 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 0 2024-09-21T21:42:36.932 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:36.941 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T21:42:37.025 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:37.024+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:37.084 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:37.084 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:42:37.084 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:42:37.084 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:42:37.084 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:37.093 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T21:42:37.125 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:37.124+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:37.197 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:37.197 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 20, 2024-09-21T21:42:37.197 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 600, 2024-09-21T21:42:37.197 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:42:37.197 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.197 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11319 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 5492736~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:37.198 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.712385+0000", 2024-09-21T21:42:37.198 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 46.482682423999997, 2024-09-21T21:42:37.198 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078627889000000006, 2024-09-21T21:42:37.198 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:37.198 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:37.198 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:37.198 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:37.198 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:37.198 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:37.198 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11319 2024-09-21T21:42:37.198 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.198 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:37.198 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.199 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:37.199 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712385+0000", 2024-09-21T21:42:37.199 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:37.199 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.199 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.199 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:37.199 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712385+0000", 2024-09-21T21:42:37.199 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:37.199 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.199 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.199 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:37.199 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712385+0000", 2024-09-21T21:42:37.199 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2800000000000002e-07 2024-09-21T21:42:37.200 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.200 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.200 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:37.200 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712387+0000", 2024-09-21T21:42:37.200 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1569999999999998e-06 2024-09-21T21:42:37.200 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.200 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.200 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:37.200 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712393+0000", 2024-09-21T21:42:37.200 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9370000000000001e-06 2024-09-21T21:42:37.200 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.200 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.200 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:37.200 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712398+0000", 2024-09-21T21:42:37.201 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5020000000000003e-06 2024-09-21T21:42:37.201 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.201 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.201 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:37.201 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.790746+0000", 2024-09-21T21:42:37.201 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078348589999999996 2024-09-21T21:42:37.201 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.201 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.201 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:37.201 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.790844+0000", 2024-09-21T21:42:37.201 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.7526000000000001e-05 2024-09-21T21:42:37.201 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.201 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.202 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:37.202 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.791013+0000", 2024-09-21T21:42:37.202 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00016874899999999999 2024-09-21T21:42:37.202 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:37.202 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:37.202 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:37.202 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.202 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.202 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11320 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 6180864~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:37.202 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.712405+0000", 2024-09-21T21:42:37.202 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 46.482662464999997, 2024-09-21T21:42:37.202 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078882870999999993, 2024-09-21T21:42:37.203 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:37.203 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:37.203 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:37.203 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:37.203 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:37.203 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:37.203 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11320 2024-09-21T21:42:37.203 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.203 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:37.203 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.203 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:37.203 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712405+0000", 2024-09-21T21:42:37.203 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:37.204 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.204 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.204 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:37.204 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712405+0000", 2024-09-21T21:42:37.204 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:37.204 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.204 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.204 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:37.204 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.713942+0000", 2024-09-21T21:42:37.204 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.001536882 2024-09-21T21:42:37.204 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.204 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.204 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:37.204 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.713946+0000", 2024-09-21T21:42:37.205 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4549999999999997e-06 2024-09-21T21:42:37.205 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.205 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.205 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:37.205 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.713958+0000", 2024-09-21T21:42:37.205 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1775e-05 2024-09-21T21:42:37.205 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.205 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.205 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:37.205 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.713965+0000", 2024-09-21T21:42:37.205 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5100000000000001e-06 2024-09-21T21:42:37.205 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.205 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.205 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:37.206 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.791059+0000", 2024-09-21T21:42:37.206 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.077094036000000005 2024-09-21T21:42:37.206 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.206 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.206 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:37.206 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.791175+0000", 2024-09-21T21:42:37.206 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000115895 2024-09-21T21:42:37.206 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.206 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.206 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:37.206 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.791288+0000", 2024-09-21T21:42:37.206 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000112318 2024-09-21T21:42:37.206 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:37.207 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:37.207 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:37.207 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.207 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.207 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11325 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 5414912~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:37.207 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.714095+0000", 2024-09-21T21:42:37.207 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 46.480972469999998, 2024-09-21T21:42:37.207 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078631894999999993, 2024-09-21T21:42:37.207 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:37.207 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:37.207 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:37.207 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:37.207 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:37.207 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:37.208 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11325 2024-09-21T21:42:37.208 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.208 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:37.208 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.208 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:37.208 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714095+0000", 2024-09-21T21:42:37.208 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:37.208 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.208 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.208 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:37.208 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714095+0000", 2024-09-21T21:42:37.208 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:37.208 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.208 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.208 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:37.209 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714095+0000", 2024-09-21T21:42:37.209 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7500000000000001e-07 2024-09-21T21:42:37.209 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.209 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.209 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:37.209 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714097+0000", 2024-09-21T21:42:37.209 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.12e-06 2024-09-21T21:42:37.209 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.209 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.209 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:37.209 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714104+0000", 2024-09-21T21:42:37.209 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3520000000000003e-06 2024-09-21T21:42:37.209 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.209 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.209 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:37.210 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714108+0000", 2024-09-21T21:42:37.210 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8810000000000004e-06 2024-09-21T21:42:37.210 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.210 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.210 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:37.210 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792509+0000", 2024-09-21T21:42:37.210 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078400739999999997 2024-09-21T21:42:37.210 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.210 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.210 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:37.210 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792613+0000", 2024-09-21T21:42:37.210 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000103939 2024-09-21T21:42:37.210 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.210 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.210 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:37.211 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792727+0000", 2024-09-21T21:42:37.211 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011348799999999999 2024-09-21T21:42:37.211 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:37.211 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:37.211 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:37.211 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.211 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.211 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11326 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 6103040~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:37.211 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.714115+0000", 2024-09-21T21:42:37.211 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 46.480952442000003, 2024-09-21T21:42:37.211 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078963056000000004, 2024-09-21T21:42:37.211 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:37.211 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:37.212 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:37.212 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:37.212 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:37.212 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:37.212 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11326 2024-09-21T21:42:37.212 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.212 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:37.212 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.212 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:37.212 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714115+0000", 2024-09-21T21:42:37.212 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:37.212 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.212 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.212 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:37.213 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714115+0000", 2024-09-21T21:42:37.213 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:37.213 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.213 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.213 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:37.213 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.715940+0000", 2024-09-21T21:42:37.213 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0018254969999999999 2024-09-21T21:42:37.213 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.213 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.213 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:37.213 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.715945+0000", 2024-09-21T21:42:37.213 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6120000000000001e-06 2024-09-21T21:42:37.213 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.213 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.213 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:37.214 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.715954+0000", 2024-09-21T21:42:37.214 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.5899999999999997e-06 2024-09-21T21:42:37.214 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.214 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.214 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:37.214 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.715959+0000", 2024-09-21T21:42:37.214 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0849999999999996e-06 2024-09-21T21:42:37.214 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.214 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.214 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:37.214 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792771+0000", 2024-09-21T21:42:37.214 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.076811829999999998 2024-09-21T21:42:37.214 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.214 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.215 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:37.215 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792882+0000", 2024-09-21T21:42:37.215 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000110857 2024-09-21T21:42:37.215 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.215 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.215 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:37.215 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.793078+0000", 2024-09-21T21:42:37.215 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00019558499999999999 2024-09-21T21:42:37.215 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:37.215 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:37.215 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:37.215 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.215 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.215 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11332 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 7565312~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:37.215 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.716063+0000", 2024-09-21T21:42:37.216 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 46.479003908000003, 2024-09-21T21:42:37.216 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078787482000000006, 2024-09-21T21:42:37.216 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:37.216 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:37.216 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:37.216 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:37.216 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:37.216 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:37.216 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11332 2024-09-21T21:42:37.216 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.216 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:37.216 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.216 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:37.216 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.716063+0000", 2024-09-21T21:42:37.217 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:37.217 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.217 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.217 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:37.217 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.716063+0000", 2024-09-21T21:42:37.217 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:37.217 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.217 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.217 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:37.217 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.717944+0000", 2024-09-21T21:42:37.217 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0018812309999999999 2024-09-21T21:42:37.217 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.217 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.217 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:37.217 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.717949+0000", 2024-09-21T21:42:37.218 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9389999999999996e-06 2024-09-21T21:42:37.218 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.218 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.218 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:37.218 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.717963+0000", 2024-09-21T21:42:37.218 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.3899e-05 2024-09-21T21:42:37.218 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.218 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.218 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:37.218 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.717970+0000", 2024-09-21T21:42:37.218 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.426e-06 2024-09-21T21:42:37.218 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.218 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.218 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:37.219 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.794573+0000", 2024-09-21T21:42:37.219 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.076602990999999995 2024-09-21T21:42:37.219 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.219 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.219 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:37.219 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.794682+0000", 2024-09-21T21:42:37.219 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000109541 2024-09-21T21:42:37.219 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.219 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.219 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:37.219 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.794851+0000", 2024-09-21T21:42:37.219 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000168455 2024-09-21T21:42:37.219 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:37.220 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:37.220 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:37.220 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.220 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.220 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11339 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 6266880~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:37.220 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.718117+0000", 2024-09-21T21:42:37.220 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 46.476949882, 2024-09-21T21:42:37.220 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078703717000000006, 2024-09-21T21:42:37.220 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:37.220 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:37.220 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:37.220 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:37.220 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:37.220 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:37.221 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11339 2024-09-21T21:42:37.221 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.221 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:37.221 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.221 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:37.221 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.718117+0000", 2024-09-21T21:42:37.221 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:37.221 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.221 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.221 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:37.221 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.718117+0000", 2024-09-21T21:42:37.221 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:37.221 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.221 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.222 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:37.222 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720019+0000", 2024-09-21T21:42:37.222 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0019012549999999999 2024-09-21T21:42:37.222 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.222 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.222 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:37.222 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720024+0000", 2024-09-21T21:42:37.222 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1100000000000002e-06 2024-09-21T21:42:37.222 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.222 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.222 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:37.222 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720035+0000", 2024-09-21T21:42:37.222 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.094e-05 2024-09-21T21:42:37.222 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.223 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.223 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:37.223 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720040+0000", 2024-09-21T21:42:37.223 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1830000000000002e-06 2024-09-21T21:42:37.223 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.223 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.223 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:37.223 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796618+0000", 2024-09-21T21:42:37.223 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.076578033000000004 2024-09-21T21:42:37.223 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.223 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.223 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:37.223 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796696+0000", 2024-09-21T21:42:37.223 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7836000000000005e-05 2024-09-21T21:42:37.224 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.224 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.224 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:37.224 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796821+0000", 2024-09-21T21:42:37.224 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00012536000000000001 2024-09-21T21:42:37.224 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:37.224 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:37.224 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:37.224 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.224 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.224 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11345 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 5472256~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:37.224 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.720116+0000", 2024-09-21T21:42:37.224 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 46.474950982000003, 2024-09-21T21:42:37.224 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078667730000000005, 2024-09-21T21:42:37.225 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:37.225 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:37.225 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:37.225 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:37.225 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:37.225 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:37.225 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11345 2024-09-21T21:42:37.225 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.225 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:37.225 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.225 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:37.225 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720116+0000", 2024-09-21T21:42:37.225 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:37.226 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.226 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.226 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:37.226 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720116+0000", 2024-09-21T21:42:37.226 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:37.226 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.226 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.226 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:37.226 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720116+0000", 2024-09-21T21:42:37.226 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.04e-07 2024-09-21T21:42:37.226 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.226 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.226 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:37.227 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720118+0000", 2024-09-21T21:42:37.227 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.181e-06 2024-09-21T21:42:37.227 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.227 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.227 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:37.227 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720121+0000", 2024-09-21T21:42:37.227 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0199999999999999e-06 2024-09-21T21:42:37.227 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.227 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.227 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:37.227 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720123+0000", 2024-09-21T21:42:37.227 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.5469999999999999e-06 2024-09-21T21:42:37.227 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.227 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.228 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:37.228 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798626+0000", 2024-09-21T21:42:37.228 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078503137000000001 2024-09-21T21:42:37.228 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.228 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.228 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:37.228 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798677+0000", 2024-09-21T21:42:37.228 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0930999999999997e-05 2024-09-21T21:42:37.228 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.228 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.228 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:37.228 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798784+0000", 2024-09-21T21:42:37.228 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010671 2024-09-21T21:42:37.228 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:37.229 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:37.229 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:37.229 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.229 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.229 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11346 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 5500928~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:37.229 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.720126+0000", 2024-09-21T21:42:37.229 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 46.474940908999997, 2024-09-21T21:42:37.229 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078863421000000003, 2024-09-21T21:42:37.229 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:37.229 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:37.229 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:37.229 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:37.229 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:37.229 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:37.230 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11346 2024-09-21T21:42:37.230 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.230 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:37.230 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.230 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:37.230 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720126+0000", 2024-09-21T21:42:37.230 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:37.230 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.230 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.230 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:37.230 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720126+0000", 2024-09-21T21:42:37.230 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:37.230 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.231 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.231 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:37.231 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.722145+0000", 2024-09-21T21:42:37.231 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0020183219999999999 2024-09-21T21:42:37.231 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.231 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.231 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:37.231 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.722150+0000", 2024-09-21T21:42:37.231 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1760000000000001e-06 2024-09-21T21:42:37.231 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.231 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.231 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:37.231 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.722161+0000", 2024-09-21T21:42:37.231 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1566000000000001e-05 2024-09-21T21:42:37.231 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.232 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.232 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:37.232 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.722169+0000", 2024-09-21T21:42:37.232 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.2840000000000001e-06 2024-09-21T21:42:37.232 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.232 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.232 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:37.232 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798815+0000", 2024-09-21T21:42:37.232 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.076646607000000005 2024-09-21T21:42:37.232 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.232 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.232 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:37.232 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798870+0000", 2024-09-21T21:42:37.232 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4486e-05 2024-09-21T21:42:37.233 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.233 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.233 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:37.233 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798990+0000", 2024-09-21T21:42:37.233 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011998 2024-09-21T21:42:37.233 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:37.233 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:37.233 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:37.233 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.233 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.233 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13629 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7852032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:37.233 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943319+0000", 2024-09-21T21:42:37.233 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 46.251748267000004, 2024-09-21T21:42:37.233 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078610975999999999, 2024-09-21T21:42:37.234 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:37.234 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:37.234 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:37.234 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:37.234 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:37.234 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:37.234 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13629 2024-09-21T21:42:37.234 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.234 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:37.234 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.234 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:37.234 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943319+0000", 2024-09-21T21:42:37.234 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:37.235 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.235 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.235 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:37.235 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943319+0000", 2024-09-21T21:42:37.235 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:37.235 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.235 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.235 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:37.235 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943319+0000", 2024-09-21T21:42:37.235 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6199999999999999e-07 2024-09-21T21:42:37.235 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.235 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.235 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:37.236 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943321+0000", 2024-09-21T21:42:37.236 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1009999999999999e-06 2024-09-21T21:42:37.236 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.236 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.236 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:37.236 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943328+0000", 2024-09-21T21:42:37.236 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.1029999999999996e-06 2024-09-21T21:42:37.236 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.236 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.236 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:37.236 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943333+0000", 2024-09-21T21:42:37.236 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7709999999999997e-06 2024-09-21T21:42:37.236 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.236 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.236 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:37.237 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.021679+0000", 2024-09-21T21:42:37.237 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078345673000000005 2024-09-21T21:42:37.237 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.237 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.237 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:37.237 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.021771+0000", 2024-09-21T21:42:37.237 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2201999999999996e-05 2024-09-21T21:42:37.237 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.237 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.237 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:37.237 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.021930+0000", 2024-09-21T21:42:37.237 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00015876400000000001 2024-09-21T21:42:37.237 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:37.237 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:37.237 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:37.238 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.238 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.238 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13630 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7880704~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:37.238 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943339+0000", 2024-09-21T21:42:37.238 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 46.251728079000003, 2024-09-21T21:42:37.238 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078939410000000002, 2024-09-21T21:42:37.238 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:37.238 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:37.238 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:37.238 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:37.238 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:37.239 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:37.239 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13630 2024-09-21T21:42:37.239 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.239 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:37.239 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.239 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:37.239 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943339+0000", 2024-09-21T21:42:37.239 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:37.239 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.239 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.240 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:37.240 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943339+0000", 2024-09-21T21:42:37.240 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:37.240 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.240 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.240 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:37.240 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943339+0000", 2024-09-21T21:42:37.240 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8000000000000001e-07 2024-09-21T21:42:37.240 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.240 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.240 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:37.240 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943341+0000", 2024-09-21T21:42:37.240 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9850000000000001e-06 2024-09-21T21:42:37.240 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.241 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.241 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:37.241 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943347+0000", 2024-09-21T21:42:37.241 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2979999999999999e-06 2024-09-21T21:42:37.241 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.241 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.241 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:37.241 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943354+0000", 2024-09-21T21:42:37.241 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.2969999999999998e-06 2024-09-21T21:42:37.241 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.241 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.241 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:37.241 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.021996+0000", 2024-09-21T21:42:37.241 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078641680000000005 2024-09-21T21:42:37.241 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.242 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.242 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:37.242 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022104+0000", 2024-09-21T21:42:37.242 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000107902 2024-09-21T21:42:37.242 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.242 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.242 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:37.242 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022278+0000", 2024-09-21T21:42:37.242 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00017486800000000001 2024-09-21T21:42:37.242 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:37.242 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:37.242 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:37.242 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.242 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.243 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13631 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7909376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:37.243 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943362+0000", 2024-09-21T21:42:37.243 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 46.251705258999998, 2024-09-21T21:42:37.243 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079195644999999995, 2024-09-21T21:42:37.243 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:37.243 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:37.243 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:37.243 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:37.243 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:37.243 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:37.243 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13631 2024-09-21T21:42:37.243 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.244 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:37.244 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.244 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:37.244 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943362+0000", 2024-09-21T21:42:37.244 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:37.244 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.244 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.244 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:37.244 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943362+0000", 2024-09-21T21:42:37.245 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:37.245 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.245 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.245 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:37.245 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943362+0000", 2024-09-21T21:42:37.245 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8799999999999998e-07 2024-09-21T21:42:37.245 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.245 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.245 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:37.245 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943364+0000", 2024-09-21T21:42:37.245 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9850000000000001e-06 2024-09-21T21:42:37.246 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.246 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.246 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:37.246 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943371+0000", 2024-09-21T21:42:37.246 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0650000000000001e-06 2024-09-21T21:42:37.246 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.246 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.246 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:37.246 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943376+0000", 2024-09-21T21:42:37.246 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8250000000000004e-06 2024-09-21T21:42:37.246 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.246 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.246 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:37.246 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022329+0000", 2024-09-21T21:42:37.247 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078952888999999998 2024-09-21T21:42:37.247 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.247 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.247 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:37.247 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022419+0000", 2024-09-21T21:42:37.247 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.9738e-05 2024-09-21T21:42:37.247 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.247 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.247 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:37.247 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022558+0000", 2024-09-21T21:42:37.247 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00013875500000000001 2024-09-21T21:42:37.247 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:37.247 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:37.247 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:37.247 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.248 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.248 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13632 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7966720~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:37.248 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943382+0000", 2024-09-21T21:42:37.248 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 46.251685025, 2024-09-21T21:42:37.248 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079495727000000002, 2024-09-21T21:42:37.248 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:37.248 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:37.248 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:37.248 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:37.248 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:37.248 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:37.248 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13632 2024-09-21T21:42:37.248 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.248 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:37.249 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.249 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:37.249 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943382+0000", 2024-09-21T21:42:37.249 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:37.249 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.249 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.249 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:37.249 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943382+0000", 2024-09-21T21:42:37.249 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:37.249 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.249 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.249 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:37.249 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943382+0000", 2024-09-21T21:42:37.249 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7e-07 2024-09-21T21:42:37.250 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.250 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.250 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:37.250 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943385+0000", 2024-09-21T21:42:37.250 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2230000000000001e-06 2024-09-21T21:42:37.250 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.250 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.250 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:37.250 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943391+0000", 2024-09-21T21:42:37.250 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6810000000000001e-06 2024-09-21T21:42:37.250 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.250 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.250 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:37.251 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943397+0000", 2024-09-21T21:42:37.251 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6559999999999997e-06 2024-09-21T21:42:37.251 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.251 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.251 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:37.251 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022601+0000", 2024-09-21T21:42:37.251 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079204051999999997 2024-09-21T21:42:37.251 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.251 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.251 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:37.251 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022704+0000", 2024-09-21T21:42:37.251 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010329599999999999 2024-09-21T21:42:37.251 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.251 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.252 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:37.252 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022878+0000", 2024-09-21T21:42:37.252 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000173449 2024-09-21T21:42:37.252 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:37.252 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:37.252 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:37.252 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.252 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.252 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13633 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7938048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:37.252 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943403+0000", 2024-09-21T21:42:37.252 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 46.251664095000002, 2024-09-21T21:42:37.252 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079794697999999997, 2024-09-21T21:42:37.252 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:37.252 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:37.252 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:37.253 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:37.253 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:37.253 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:37.253 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13633 2024-09-21T21:42:37.253 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.253 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:37.253 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.253 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:37.253 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943403+0000", 2024-09-21T21:42:37.253 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:37.253 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.253 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.253 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:37.253 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943403+0000", 2024-09-21T21:42:37.254 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:37.254 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.254 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.254 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:37.254 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943403+0000", 2024-09-21T21:42:37.254 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.65e-07 2024-09-21T21:42:37.254 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.254 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.254 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:37.254 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943405+0000", 2024-09-21T21:42:37.254 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0020000000000001e-06 2024-09-21T21:42:37.254 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.254 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.255 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:37.255 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943412+0000", 2024-09-21T21:42:37.255 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.8379999999999996e-06 2024-09-21T21:42:37.255 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.255 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.255 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:37.255 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943417+0000", 2024-09-21T21:42:37.255 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7650000000000001e-06 2024-09-21T21:42:37.255 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.255 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.255 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:37.255 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022922+0000", 2024-09-21T21:42:37.255 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079504512999999999 2024-09-21T21:42:37.255 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.256 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.256 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:37.256 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023039+0000", 2024-09-21T21:42:37.256 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000117044 2024-09-21T21:42:37.256 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.256 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.256 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:37.256 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023198+0000", 2024-09-21T21:42:37.256 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000159171 2024-09-21T21:42:37.256 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:37.256 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:37.256 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:37.256 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.256 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.257 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13634 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7995392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:37.257 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943423+0000", 2024-09-21T21:42:37.257 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 46.251644386999999, 2024-09-21T21:42:37.257 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.080142358999999996, 2024-09-21T21:42:37.257 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:37.257 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:37.257 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:37.257 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:37.257 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:37.257 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:37.257 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13634 2024-09-21T21:42:37.257 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.257 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:37.257 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.258 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:37.258 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943423+0000", 2024-09-21T21:42:37.258 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:37.258 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.258 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.258 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:37.258 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943423+0000", 2024-09-21T21:42:37.258 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:37.258 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.258 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.258 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:37.258 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943423+0000", 2024-09-21T21:42:37.258 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.58e-07 2024-09-21T21:42:37.258 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.259 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.259 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:37.259 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943425+0000", 2024-09-21T21:42:37.259 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9180000000000001e-06 2024-09-21T21:42:37.259 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.259 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.259 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:37.259 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943431+0000", 2024-09-21T21:42:37.259 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4439999999999996e-06 2024-09-21T21:42:37.259 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.259 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.259 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:37.259 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943436+0000", 2024-09-21T21:42:37.259 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.887e-06 2024-09-21T21:42:37.260 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.260 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.260 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:37.260 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023266+0000", 2024-09-21T21:42:37.260 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079829622000000003 2024-09-21T21:42:37.260 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.260 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.260 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:37.260 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023384+0000", 2024-09-21T21:42:37.260 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000117658 2024-09-21T21:42:37.260 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.260 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.260 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:37.260 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023565+0000", 2024-09-21T21:42:37.260 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00018147200000000001 2024-09-21T21:42:37.261 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:37.261 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:37.261 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:37.261 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.261 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.261 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13635 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 8024064~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:37.261 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943447+0000", 2024-09-21T21:42:37.261 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 46.251619847000001, 2024-09-21T21:42:37.261 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.080417653000000006, 2024-09-21T21:42:37.261 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:37.261 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:37.261 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:37.261 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:37.261 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:37.261 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:37.262 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13635 2024-09-21T21:42:37.262 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.262 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:37.262 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.262 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:37.262 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943447+0000", 2024-09-21T21:42:37.262 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:37.262 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.262 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.262 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:37.262 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943447+0000", 2024-09-21T21:42:37.262 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:37.262 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.262 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.263 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:37.263 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943448+0000", 2024-09-21T21:42:37.263 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1699999999999999e-07 2024-09-21T21:42:37.263 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.263 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.263 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:37.263 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943450+0000", 2024-09-21T21:42:37.263 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1639999999999999e-06 2024-09-21T21:42:37.263 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.263 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.263 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:37.263 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943456+0000", 2024-09-21T21:42:37.263 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8379999999999998e-06 2024-09-21T21:42:37.264 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.264 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.264 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:37.264 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943460+0000", 2024-09-21T21:42:37.264 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4700000000000004e-06 2024-09-21T21:42:37.264 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.264 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.264 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:37.264 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023595+0000", 2024-09-21T21:42:37.264 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.080134869999999997 2024-09-21T21:42:37.264 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.264 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.264 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:37.264 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023688+0000", 2024-09-21T21:42:37.264 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2794000000000006e-05 2024-09-21T21:42:37.265 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.265 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.265 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:37.265 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023865+0000", 2024-09-21T21:42:37.265 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0001771 2024-09-21T21:42:37.265 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:37.265 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:37.265 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:37.265 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.265 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.265 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13636 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 8052736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:37.265 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943467+0000", 2024-09-21T21:42:37.265 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 46.251600109999998, 2024-09-21T21:42:37.265 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.080694399999999999, 2024-09-21T21:42:37.265 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:37.266 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:37.266 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:37.266 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:37.266 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:37.266 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:37.266 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13636 2024-09-21T21:42:37.266 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.266 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:37.266 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.266 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:37.266 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943467+0000", 2024-09-21T21:42:37.266 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:37.266 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.266 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.267 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:37.267 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943467+0000", 2024-09-21T21:42:37.267 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:37.267 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.267 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.267 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:37.267 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944940+0000", 2024-09-21T21:42:37.267 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0014725109999999999 2024-09-21T21:42:37.267 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.267 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.267 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:37.267 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944949+0000", 2024-09-21T21:42:37.267 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2930000000000006e-06 2024-09-21T21:42:37.267 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.267 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.268 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:37.268 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944958+0000", 2024-09-21T21:42:37.268 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.6899999999999998e-06 2024-09-21T21:42:37.268 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.268 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.268 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:37.268 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944966+0000", 2024-09-21T21:42:37.268 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0690000000000002e-06 2024-09-21T21:42:37.268 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.268 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.268 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:37.268 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023900+0000", 2024-09-21T21:42:37.268 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078934371000000003 2024-09-21T21:42:37.268 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.269 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.269 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:37.269 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024009+0000", 2024-09-21T21:42:37.269 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000108857 2024-09-21T21:42:37.269 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.269 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.269 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:37.269 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024161+0000", 2024-09-21T21:42:37.269 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000152609 2024-09-21T21:42:37.269 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:37.269 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:37.269 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:37.269 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.269 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.270 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13637 3.12 3.1dc31b72 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:37.270 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.944978+0000", 2024-09-21T21:42:37.270 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 46.250089174999999, 2024-09-21T21:42:37.270 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079232604999999998, 2024-09-21T21:42:37.270 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:37.270 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:37.270 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:42:37.270 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:37.270 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:37.270 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:37.270 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13637 2024-09-21T21:42:37.270 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.270 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:37.270 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.270 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:37.271 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944978+0000", 2024-09-21T21:42:37.271 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:37.271 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.271 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.271 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:37.271 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944978+0000", 2024-09-21T21:42:37.271 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:37.271 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.271 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.271 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:37.271 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944978+0000", 2024-09-21T21:42:37.271 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9100000000000004e-07 2024-09-21T21:42:37.271 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.271 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.272 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:37.272 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944981+0000", 2024-09-21T21:42:37.272 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3290000000000001e-06 2024-09-21T21:42:37.272 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.272 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.272 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:37.272 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944987+0000", 2024-09-21T21:42:37.272 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.28e-06 2024-09-21T21:42:37.272 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.272 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.272 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:37.272 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944992+0000", 2024-09-21T21:42:37.272 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8520000000000003e-06 2024-09-21T21:42:37.272 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.272 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.273 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:37.273 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024196+0000", 2024-09-21T21:42:37.273 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079203673000000002 2024-09-21T21:42:37.273 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.273 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.273 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:37.273 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024211+0000", 2024-09-21T21:42:37.273 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.4980000000000001e-05 2024-09-21T21:42:37.273 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:37.273 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:37.273 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:37.273 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.273 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.273 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13638 3.12 3.1dc31b72 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:37.274 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.945000+0000", 2024-09-21T21:42:37.274 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 46.250067047000002, 2024-09-21T21:42:37.274 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079257465999999999, 2024-09-21T21:42:37.274 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:37.274 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:37.274 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:42:37.274 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:37.274 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:37.274 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:37.274 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13638 2024-09-21T21:42:37.274 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.274 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:37.274 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.274 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:37.275 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945000+0000", 2024-09-21T21:42:37.275 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:37.275 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.275 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.275 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:37.275 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945000+0000", 2024-09-21T21:42:37.275 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:37.275 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.275 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.275 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:37.275 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945001+0000", 2024-09-21T21:42:37.275 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.32e-07 2024-09-21T21:42:37.275 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.275 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.275 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:37.276 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945003+0000", 2024-09-21T21:42:37.276 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2309999999999999e-06 2024-09-21T21:42:37.276 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.276 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.276 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:37.276 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945009+0000", 2024-09-21T21:42:37.276 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7450000000000001e-06 2024-09-21T21:42:37.276 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.276 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.276 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:37.276 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945013+0000", 2024-09-21T21:42:37.276 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.673e-06 2024-09-21T21:42:37.276 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.276 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.277 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:37.277 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024245+0000", 2024-09-21T21:42:37.277 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079232160999999995 2024-09-21T21:42:37.277 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.277 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.277 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:37.277 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024258+0000", 2024-09-21T21:42:37.277 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.2224e-05 2024-09-21T21:42:37.277 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:37.277 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:37.277 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:37.277 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.277 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.277 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13639 3.12 3.1dc31b72 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:37.277 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.945034+0000", 2024-09-21T21:42:37.278 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 46.250033248000001, 2024-09-21T21:42:37.278 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079260627, 2024-09-21T21:42:37.278 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:37.278 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:37.278 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:42:37.278 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:37.278 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:37.278 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:37.278 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13639 2024-09-21T21:42:37.278 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.278 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:37.278 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.278 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:37.278 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945034+0000", 2024-09-21T21:42:37.279 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:37.279 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.279 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.279 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:37.279 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945034+0000", 2024-09-21T21:42:37.279 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:37.279 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.279 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.279 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:37.279 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945034+0000", 2024-09-21T21:42:37.279 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3799999999999998e-07 2024-09-21T21:42:37.279 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.279 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.279 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:37.280 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945036+0000", 2024-09-21T21:42:37.280 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.052e-06 2024-09-21T21:42:37.280 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.280 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.280 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:37.280 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945040+0000", 2024-09-21T21:42:37.280 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1169999999999997e-06 2024-09-21T21:42:37.280 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.280 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.280 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:37.280 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945043+0000", 2024-09-21T21:42:37.280 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.8439999999999998e-06 2024-09-21T21:42:37.280 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.280 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.280 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:37.281 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024286+0000", 2024-09-21T21:42:37.281 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079242483000000002 2024-09-21T21:42:37.281 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.281 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.281 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:37.281 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024295+0000", 2024-09-21T21:42:37.281 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.7930000000000003e-06 2024-09-21T21:42:37.281 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:37.281 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:37.281 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:37.281 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.281 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.281 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13640 3.12 3.1dc31b72 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:37.281 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.945047+0000", 2024-09-21T21:42:37.282 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 46.250019725999998, 2024-09-21T21:42:37.282 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079412321999999994, 2024-09-21T21:42:37.282 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:37.282 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:37.282 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:42:37.282 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:37.282 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:37.282 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:37.282 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13640 2024-09-21T21:42:37.282 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.282 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:37.282 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.282 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:37.282 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945047+0000", 2024-09-21T21:42:37.283 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:37.283 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.283 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.283 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:37.283 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945047+0000", 2024-09-21T21:42:37.283 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:37.283 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.283 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.283 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:37.283 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945048+0000", 2024-09-21T21:42:37.283 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0499999999999999e-07 2024-09-21T21:42:37.283 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.283 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.283 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:37.283 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945049+0000", 2024-09-21T21:42:37.284 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.3939999999999999e-06 2024-09-21T21:42:37.284 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.284 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.284 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:37.284 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945053+0000", 2024-09-21T21:42:37.284 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1760000000000003e-06 2024-09-21T21:42:37.284 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.284 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.284 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:37.284 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945056+0000", 2024-09-21T21:42:37.284 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.6129999999999998e-06 2024-09-21T21:42:37.284 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.284 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.284 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:37.284 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024360+0000", 2024-09-21T21:42:37.285 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079303645000000006 2024-09-21T21:42:37.285 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.285 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.285 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:37.285 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024460+0000", 2024-09-21T21:42:37.285 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000100189 2024-09-21T21:42:37.285 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:37.285 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:37.285 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:37.285 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:37.285 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:37.285 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:37.287 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T21:42:37.287 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:37.285+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:37.388 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:37.386+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:37.389 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:37.389 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:42:37.389 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 0 2024-09-21T21:42:37.389 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:37.398 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T21:42:37.489 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:37.487+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:37.500 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:37.500 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:42:37.500 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:42:37.500 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:42:37.500 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:37.508 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T21:42:37.590 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:37.588+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:37.662 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:37.663 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 20, 2024-09-21T21:42:37.663 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 600, 2024-09-21T21:42:37.663 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:42:37.663 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.663 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4824.0:1564 3.2 3:4647520c:::rbd_data.1121b97f2925.0000000000000006:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:42:37.663 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:42:37.663 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 96.023957103000001, 2024-09-21T21:42:37.663 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056097347999999998, 2024-09-21T21:42:37.663 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:37.663 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:37.663 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:37.663 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:37.663 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4824", 2024-09-21T21:42:37.664 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:42:37.664 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1564 2024-09-21T21:42:37.664 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.664 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:37.664 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.664 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:37.664 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:42:37.664 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:37.664 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.664 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.664 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:37.664 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:42:37.664 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:37.664 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.665 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.665 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:37.665 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:42:37.665 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6499999999999999e-07 2024-09-21T21:42:37.665 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.665 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.665 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:37.665 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636067+0000", 2024-09-21T21:42:37.665 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3879999999999998e-06 2024-09-21T21:42:37.665 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.665 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.665 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:37.665 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636075+0000", 2024-09-21T21:42:37.665 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5480000000000004e-06 2024-09-21T21:42:37.665 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.666 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.666 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:37.666 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636080+0000", 2024-09-21T21:42:37.666 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.339e-06 2024-09-21T21:42:37.666 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.666 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.666 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:37.666 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636133+0000", 2024-09-21T21:42:37.666 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2930000000000003e-05 2024-09-21T21:42:37.666 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.666 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.666 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:37.666 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636265+0000", 2024-09-21T21:42:37.666 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000132058 2024-09-21T21:42:37.667 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.667 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.667 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:37.667 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.692162+0000", 2024-09-21T21:42:37.667 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055896620000000001 2024-09-21T21:42:37.667 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:37.667 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:37.667 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:37.667 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.667 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.667 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3260 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 1437696~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:42:37.667 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.196297+0000", 2024-09-21T21:42:37.667 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 50.463724892999998, 2024-09-21T21:42:37.668 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056071022999999998, 2024-09-21T21:42:37.668 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:37.668 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:37.668 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:37.668 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:37.668 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:37.668 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:37.668 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3260 2024-09-21T21:42:37.668 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.668 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:37.668 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.668 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:37.668 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196297+0000", 2024-09-21T21:42:37.668 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:37.669 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.669 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.669 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:37.669 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196297+0000", 2024-09-21T21:42:37.669 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:37.669 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.669 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.669 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:37.669 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196918+0000", 2024-09-21T21:42:37.669 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00062148699999999999 2024-09-21T21:42:37.669 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.669 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.669 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:37.669 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196930+0000", 2024-09-21T21:42:37.670 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1657e-05 2024-09-21T21:42:37.670 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.670 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.670 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:37.670 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196944+0000", 2024-09-21T21:42:37.670 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.4549000000000001e-05 2024-09-21T21:42:37.670 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.670 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.670 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:37.670 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196951+0000", 2024-09-21T21:42:37.670 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6749999999999996e-06 2024-09-21T21:42:37.670 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.670 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.670 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:37.671 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252208+0000", 2024-09-21T21:42:37.671 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055256656000000001 2024-09-21T21:42:37.671 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.671 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.671 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:37.671 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252278+0000", 2024-09-21T21:42:37.671 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9882000000000006e-05 2024-09-21T21:42:37.671 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.671 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.671 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:37.671 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252368+0000", 2024-09-21T21:42:37.671 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.0117000000000005e-05 2024-09-21T21:42:37.671 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:37.671 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:37.672 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:37.672 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.672 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.672 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3263 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 3559424~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:42:37.672 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.197041+0000", 2024-09-21T21:42:37.672 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 50.462981079000002, 2024-09-21T21:42:37.672 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055935804999999998, 2024-09-21T21:42:37.672 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:37.672 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:37.672 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:37.672 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:37.672 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:37.672 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:37.672 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3263 2024-09-21T21:42:37.672 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.673 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:37.673 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.673 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:37.673 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197041+0000", 2024-09-21T21:42:37.673 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:37.673 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.673 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.673 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:37.673 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197041+0000", 2024-09-21T21:42:37.673 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:37.673 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.673 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.673 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:37.673 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197624+0000", 2024-09-21T21:42:37.673 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00058358100000000001 2024-09-21T21:42:37.674 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.674 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.674 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:37.674 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197629+0000", 2024-09-21T21:42:37.674 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5909999999999998e-06 2024-09-21T21:42:37.674 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.674 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.674 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:37.674 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197639+0000", 2024-09-21T21:42:37.674 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0257e-05 2024-09-21T21:42:37.674 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.674 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.674 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:37.674 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197646+0000", 2024-09-21T21:42:37.675 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.7229999999999998e-06 2024-09-21T21:42:37.675 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.675 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.675 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:37.675 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252804+0000", 2024-09-21T21:42:37.675 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055158134999999997 2024-09-21T21:42:37.675 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.675 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.675 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:37.675 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252882+0000", 2024-09-21T21:42:37.675 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7791000000000001e-05 2024-09-21T21:42:37.675 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.675 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.675 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:37.676 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252976+0000", 2024-09-21T21:42:37.676 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.4727000000000003e-05 2024-09-21T21:42:37.676 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:37.676 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:37.676 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:37.676 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.676 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.676 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3437 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 925696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:42:37.676 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.205888+0000", 2024-09-21T21:42:37.676 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 50.454133798000001, 2024-09-21T21:42:37.676 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055953808000000001, 2024-09-21T21:42:37.676 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:37.676 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:37.676 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:37.677 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:37.677 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:37.677 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:37.677 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3437 2024-09-21T21:42:37.677 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.677 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:37.677 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.677 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:37.677 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205888+0000", 2024-09-21T21:42:37.677 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:37.677 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.677 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.677 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:37.677 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205888+0000", 2024-09-21T21:42:37.677 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:37.678 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.678 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.678 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:37.678 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205888+0000", 2024-09-21T21:42:37.678 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.75e-07 2024-09-21T21:42:37.678 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.678 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.678 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:37.678 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205891+0000", 2024-09-21T21:42:37.678 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3750000000000001e-06 2024-09-21T21:42:37.678 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.678 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.678 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:37.678 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205897+0000", 2024-09-21T21:42:37.679 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.285e-06 2024-09-21T21:42:37.679 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.679 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.679 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:37.679 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205902+0000", 2024-09-21T21:42:37.679 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6290000000000001e-06 2024-09-21T21:42:37.679 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.679 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.679 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:37.679 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261686+0000", 2024-09-21T21:42:37.679 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055784586999999997 2024-09-21T21:42:37.679 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.679 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.680 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:37.680 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261764+0000", 2024-09-21T21:42:37.680 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7591999999999998e-05 2024-09-21T21:42:37.680 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.680 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.680 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:37.680 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261842+0000", 2024-09-21T21:42:37.680 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7865000000000001e-05 2024-09-21T21:42:37.680 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:37.680 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:37.680 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:37.680 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.680 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.680 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3438 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 954368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:42:37.681 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.205908+0000", 2024-09-21T21:42:37.681 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 50.454113933000002, 2024-09-21T21:42:37.681 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056083169000000002, 2024-09-21T21:42:37.681 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:37.681 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:37.681 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:37.681 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:37.681 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:37.681 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:37.681 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3438 2024-09-21T21:42:37.681 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.681 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:37.681 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.681 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:37.681 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205908+0000", 2024-09-21T21:42:37.681 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:37.682 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.682 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.682 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:37.682 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205908+0000", 2024-09-21T21:42:37.682 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:37.682 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.682 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.682 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:37.682 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206431+0000", 2024-09-21T21:42:37.682 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00052282100000000003 2024-09-21T21:42:37.682 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.682 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.682 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:37.683 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206442+0000", 2024-09-21T21:42:37.683 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1695000000000001e-05 2024-09-21T21:42:37.683 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.683 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.683 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:37.683 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206452+0000", 2024-09-21T21:42:37.683 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.4560000000000006e-06 2024-09-21T21:42:37.683 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.683 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.683 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:37.683 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206458+0000", 2024-09-21T21:42:37.683 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4670000000000001e-06 2024-09-21T21:42:37.683 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.683 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.683 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:37.684 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261871+0000", 2024-09-21T21:42:37.684 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055412712000000003 2024-09-21T21:42:37.684 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.684 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.684 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:37.684 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261928+0000", 2024-09-21T21:42:37.684 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6631e-05 2024-09-21T21:42:37.684 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.684 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.684 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:37.684 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261991+0000", 2024-09-21T21:42:37.684 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3386999999999995e-05 2024-09-21T21:42:37.684 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:37.684 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:37.685 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:37.685 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.685 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.685 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3444 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 2359296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:42:37.685 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.206533+0000", 2024-09-21T21:42:37.685 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 50.453488413000002, 2024-09-21T21:42:37.685 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055974239000000002, 2024-09-21T21:42:37.685 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:37.685 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:37.685 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:37.685 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:37.685 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:37.685 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:37.685 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3444 2024-09-21T21:42:37.686 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.686 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:37.686 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.686 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:37.686 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206533+0000", 2024-09-21T21:42:37.686 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:37.686 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.686 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.686 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:37.686 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206533+0000", 2024-09-21T21:42:37.686 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:37.686 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.686 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.686 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:37.686 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206710+0000", 2024-09-21T21:42:37.687 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000176293 2024-09-21T21:42:37.687 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.687 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.687 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:37.687 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206714+0000", 2024-09-21T21:42:37.687 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1989999999999999e-06 2024-09-21T21:42:37.687 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.687 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.687 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:37.687 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206723+0000", 2024-09-21T21:42:37.687 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.9199999999999993e-06 2024-09-21T21:42:37.687 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.687 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.687 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:37.688 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206729+0000", 2024-09-21T21:42:37.688 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0739999999999998e-06 2024-09-21T21:42:37.688 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.688 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.688 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:37.688 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262339+0000", 2024-09-21T21:42:37.688 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055610166000000003 2024-09-21T21:42:37.688 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.688 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.688 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:37.688 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262419+0000", 2024-09-21T21:42:37.688 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0432999999999994e-05 2024-09-21T21:42:37.688 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.688 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.688 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:37.689 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262507+0000", 2024-09-21T21:42:37.689 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.8153999999999997e-05 2024-09-21T21:42:37.689 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:37.689 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:37.689 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:37.689 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.689 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.689 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10362 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:37.689 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840386+0000", 2024-09-21T21:42:37.689 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 23.819635568999999, 2024-09-21T21:42:37.689 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056546515999999998, 2024-09-21T21:42:37.689 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:37.689 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:37.689 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:37.690 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:37.690 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:37.690 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:37.690 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10362 2024-09-21T21:42:37.690 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.690 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:37.690 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.690 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:37.690 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840386+0000", 2024-09-21T21:42:37.690 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:37.690 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.690 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.690 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:37.690 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840386+0000", 2024-09-21T21:42:37.691 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:37.691 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.691 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.691 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:37.691 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840386+0000", 2024-09-21T21:42:37.691 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.01e-07 2024-09-21T21:42:37.691 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.691 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.691 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:37.691 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840389+0000", 2024-09-21T21:42:37.691 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.9739999999999998e-06 2024-09-21T21:42:37.691 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.691 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.691 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:37.692 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840396+0000", 2024-09-21T21:42:37.692 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2280000000000003e-06 2024-09-21T21:42:37.692 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.692 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.692 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:37.692 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840400+0000", 2024-09-21T21:42:37.692 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4429999999999996e-06 2024-09-21T21:42:37.692 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.692 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.692 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:37.692 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.890896+0000", 2024-09-21T21:42:37.692 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.050495717000000002 2024-09-21T21:42:37.692 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.692 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.692 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:37.693 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.891012+0000", 2024-09-21T21:42:37.693 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011602 2024-09-21T21:42:37.693 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.693 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.693 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:37.693 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.891274+0000", 2024-09-21T21:42:37.693 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00026240200000000001 2024-09-21T21:42:37.693 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.693 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.693 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:37.694 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.894810+0000", 2024-09-21T21:42:37.694 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0035353630000000001 2024-09-21T21:42:37.694 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.694 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.694 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:37.694 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.896818+0000", 2024-09-21T21:42:37.694 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.002008681 2024-09-21T21:42:37.694 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.694 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.694 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:37.694 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.896850+0000", 2024-09-21T21:42:37.694 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.1826999999999999e-05 2024-09-21T21:42:37.694 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.694 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.695 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:37.695 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.896933+0000", 2024-09-21T21:42:37.695 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.2460000000000007e-05 2024-09-21T21:42:37.695 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:37.695 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:37.695 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:37.695 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.695 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.695 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10363 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:37.695 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840606+0000", 2024-09-21T21:42:37.695 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 23.819416059000002, 2024-09-21T21:42:37.695 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056546011, 2024-09-21T21:42:37.695 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:37.695 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:37.696 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:37.696 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:37.696 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:37.696 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:37.696 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10363 2024-09-21T21:42:37.696 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.696 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:37.696 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.696 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:37.696 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840606+0000", 2024-09-21T21:42:37.696 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:37.696 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.696 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.696 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:37.696 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840606+0000", 2024-09-21T21:42:37.697 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:37.697 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.697 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.697 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:37.697 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840606+0000", 2024-09-21T21:42:37.697 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.4700000000000001e-07 2024-09-21T21:42:37.697 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.697 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.697 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:37.697 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840611+0000", 2024-09-21T21:42:37.697 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4649999999999996e-06 2024-09-21T21:42:37.697 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.697 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.697 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:37.698 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840620+0000", 2024-09-21T21:42:37.698 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.0350000000000007e-06 2024-09-21T21:42:37.698 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.698 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.698 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:37.698 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840626+0000", 2024-09-21T21:42:37.698 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7180000000000001e-06 2024-09-21T21:42:37.698 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.698 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.698 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:37.698 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.891665+0000", 2024-09-21T21:42:37.698 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.051039924 2024-09-21T21:42:37.698 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.698 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.698 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:37.699 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.891770+0000", 2024-09-21T21:42:37.699 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010445700000000001 2024-09-21T21:42:37.699 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.699 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.699 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:37.699 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.892147+0000", 2024-09-21T21:42:37.699 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00037731800000000002 2024-09-21T21:42:37.699 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.699 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.699 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:37.699 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.896776+0000", 2024-09-21T21:42:37.699 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0046289759999999999 2024-09-21T21:42:37.699 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.699 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.699 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:37.700 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897003+0000", 2024-09-21T21:42:37.700 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00022641299999999999 2024-09-21T21:42:37.700 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.700 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.700 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:37.700 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897064+0000", 2024-09-21T21:42:37.700 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1571999999999999e-05 2024-09-21T21:42:37.700 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.700 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.700 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:37.700 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897152+0000", 2024-09-21T21:42:37.701 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.7386000000000007e-05 2024-09-21T21:42:37.701 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:37.701 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:37.701 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:37.701 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.701 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.701 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10364 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:37.701 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840641+0000", 2024-09-21T21:42:37.701 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 23.819380239000001, 2024-09-21T21:42:37.701 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056671527999999999, 2024-09-21T21:42:37.701 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:37.701 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:37.701 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:37.701 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:37.702 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:37.702 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:37.702 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10364 2024-09-21T21:42:37.702 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.702 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:37.702 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.702 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:37.702 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840641+0000", 2024-09-21T21:42:37.702 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:37.702 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.702 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.702 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:37.702 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840641+0000", 2024-09-21T21:42:37.702 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:37.703 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.703 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.703 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:37.703 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840642+0000", 2024-09-21T21:42:37.703 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7899999999999999e-07 2024-09-21T21:42:37.703 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.703 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.703 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:37.703 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840645+0000", 2024-09-21T21:42:37.703 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0010000000000002e-06 2024-09-21T21:42:37.704 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.704 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.704 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:37.704 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840654+0000", 2024-09-21T21:42:37.704 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.8570000000000001e-06 2024-09-21T21:42:37.704 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.704 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.704 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:37.704 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840659+0000", 2024-09-21T21:42:37.704 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.558e-06 2024-09-21T21:42:37.704 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.704 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.704 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:37.704 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.892656+0000", 2024-09-21T21:42:37.705 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.051996586999999997 2024-09-21T21:42:37.705 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.705 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.705 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:37.705 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.892721+0000", 2024-09-21T21:42:37.705 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5010000000000003e-05 2024-09-21T21:42:37.705 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.705 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.705 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:37.705 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.893112+0000", 2024-09-21T21:42:37.705 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00039133899999999999 2024-09-21T21:42:37.705 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.705 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.705 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:37.706 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.896784+0000", 2024-09-21T21:42:37.706 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0036713679999999999 2024-09-21T21:42:37.706 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.706 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.706 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:37.706 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897204+0000", 2024-09-21T21:42:37.706 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00041991400000000002 2024-09-21T21:42:37.706 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.706 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.706 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:37.706 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897234+0000", 2024-09-21T21:42:37.706 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0307999999999998e-05 2024-09-21T21:42:37.707 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.707 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.707 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:37.707 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897313+0000", 2024-09-21T21:42:37.707 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.9107e-05 2024-09-21T21:42:37.707 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:37.707 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:37.707 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:37.707 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.707 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.707 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10365 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:37.707 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840668+0000", 2024-09-21T21:42:37.707 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 23.819353373999999, 2024-09-21T21:42:37.707 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.058761225, 2024-09-21T21:42:37.707 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:37.708 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:37.708 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:37.708 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:37.708 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:37.708 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:37.708 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10365 2024-09-21T21:42:37.708 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.708 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:37.708 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.708 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:37.708 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840668+0000", 2024-09-21T21:42:37.708 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:37.708 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.708 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.709 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:37.709 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840668+0000", 2024-09-21T21:42:37.709 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:37.709 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.709 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.709 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:37.709 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840669+0000", 2024-09-21T21:42:37.709 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6300000000000001e-07 2024-09-21T21:42:37.709 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.709 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.709 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:37.709 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840672+0000", 2024-09-21T21:42:37.709 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.072e-06 2024-09-21T21:42:37.709 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.710 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.710 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:37.710 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840678+0000", 2024-09-21T21:42:37.710 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.7039999999999997e-06 2024-09-21T21:42:37.710 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.710 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.710 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:37.710 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840683+0000", 2024-09-21T21:42:37.710 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6759999999999998e-06 2024-09-21T21:42:37.710 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.710 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.710 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:37.710 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.893929+0000", 2024-09-21T21:42:37.710 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.053246136999999999 2024-09-21T21:42:37.710 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.711 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.711 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:37.711 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.894082+0000", 2024-09-21T21:42:37.711 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00015301799999999999 2024-09-21T21:42:37.711 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.711 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.711 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:37.711 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.894357+0000", 2024-09-21T21:42:37.711 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00027507999999999998 2024-09-21T21:42:37.711 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.711 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.711 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:37.711 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897161+0000", 2024-09-21T21:42:37.711 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0028036810000000001 2024-09-21T21:42:37.712 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.712 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.712 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:37.712 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.899243+0000", 2024-09-21T21:42:37.712 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0020816379999999998 2024-09-21T21:42:37.712 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.712 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.712 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:37.712 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.899306+0000", 2024-09-21T21:42:37.712 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2964000000000002e-05 2024-09-21T21:42:37.712 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.712 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.712 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:37.712 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.899430+0000", 2024-09-21T21:42:37.712 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00012389200000000001 2024-09-21T21:42:37.713 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:37.713 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:37.713 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:37.713 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.713 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.713 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10366 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:37.713 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840691+0000", 2024-09-21T21:42:37.713 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 23.819330674, 2024-09-21T21:42:37.713 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.061540984999999999, 2024-09-21T21:42:37.713 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:37.713 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:37.713 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:37.713 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:37.713 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:37.714 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:37.714 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10366 2024-09-21T21:42:37.714 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.714 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:37.714 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.714 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:37.714 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840691+0000", 2024-09-21T21:42:37.714 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:37.714 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.714 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.714 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:37.714 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840691+0000", 2024-09-21T21:42:37.714 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:37.714 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.714 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.715 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:37.715 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840691+0000", 2024-09-21T21:42:37.715 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3099999999999999e-07 2024-09-21T21:42:37.715 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.715 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.715 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:37.715 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840694+0000", 2024-09-21T21:42:37.715 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.5780000000000001e-06 2024-09-21T21:42:37.715 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.715 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.715 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:37.715 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840700+0000", 2024-09-21T21:42:37.715 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9429999999999997e-06 2024-09-21T21:42:37.715 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.715 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.716 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:37.716 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840704+0000", 2024-09-21T21:42:37.716 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6380000000000003e-06 2024-09-21T21:42:37.716 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.716 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.716 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:37.716 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.894846+0000", 2024-09-21T21:42:37.716 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.054141392000000003 2024-09-21T21:42:37.716 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.716 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.716 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:37.716 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.894932+0000", 2024-09-21T21:42:37.716 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.5834000000000005e-05 2024-09-21T21:42:37.716 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.716 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.717 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:37.717 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.895302+0000", 2024-09-21T21:42:37.717 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000370737 2024-09-21T21:42:37.717 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.717 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.717 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:37.717 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897166+0000", 2024-09-21T21:42:37.717 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0018632620000000001 2024-09-21T21:42:37.717 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.717 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.717 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:37.717 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902053+0000", 2024-09-21T21:42:37.717 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0048868289999999997 2024-09-21T21:42:37.717 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.718 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.718 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:37.718 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902111+0000", 2024-09-21T21:42:37.718 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7983999999999999e-05 2024-09-21T21:42:37.718 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.718 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.718 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:37.718 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902232+0000", 2024-09-21T21:42:37.718 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000121457 2024-09-21T21:42:37.718 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:37.718 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:37.718 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:37.718 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.719 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.719 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10367 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:37.719 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840842+0000", 2024-09-21T21:42:37.719 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 23.819179865999999, 2024-09-21T21:42:37.719 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.061549088000000002, 2024-09-21T21:42:37.719 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:37.719 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:37.719 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:37.719 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:37.719 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:37.719 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:37.719 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10367 2024-09-21T21:42:37.719 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.719 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:37.720 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.720 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:37.720 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840842+0000", 2024-09-21T21:42:37.720 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:37.720 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.720 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.720 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:37.720 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840842+0000", 2024-09-21T21:42:37.720 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:37.720 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.720 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.720 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:37.720 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840842+0000", 2024-09-21T21:42:37.720 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3099999999999997e-07 2024-09-21T21:42:37.720 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.721 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.721 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:37.721 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840846+0000", 2024-09-21T21:42:37.721 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.2890000000000002e-06 2024-09-21T21:42:37.721 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.721 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.721 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:37.721 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840855+0000", 2024-09-21T21:42:37.721 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.9619999999999999e-06 2024-09-21T21:42:37.721 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.721 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.721 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:37.721 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840860+0000", 2024-09-21T21:42:37.721 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3759999999999999e-06 2024-09-21T21:42:37.721 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.722 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.722 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:37.722 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.895799+0000", 2024-09-21T21:42:37.722 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.054938569999999999 2024-09-21T21:42:37.722 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.722 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.722 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:37.722 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.895906+0000", 2024-09-21T21:42:37.722 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000107593 2024-09-21T21:42:37.722 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.722 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.722 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:37.722 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.896356+0000", 2024-09-21T21:42:37.722 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00044941700000000002 2024-09-21T21:42:37.723 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.723 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.723 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:37.723 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.898131+0000", 2024-09-21T21:42:37.723 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.001775064 2024-09-21T21:42:37.723 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.723 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.723 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:37.723 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902283+0000", 2024-09-21T21:42:37.723 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0041524839999999997 2024-09-21T21:42:37.723 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.723 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.723 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:37.723 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902316+0000", 2024-09-21T21:42:37.724 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3002e-05 2024-09-21T21:42:37.724 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.724 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.724 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:37.724 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902391+0000", 2024-09-21T21:42:37.724 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.47e-05 2024-09-21T21:42:37.724 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:37.724 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:37.724 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:37.724 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.724 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.724 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10368 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:37.724 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840873+0000", 2024-09-21T21:42:37.724 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 23.819148911999999, 2024-09-21T21:42:37.724 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.063395291000000006, 2024-09-21T21:42:37.725 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:37.725 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:37.725 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:37.725 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:37.725 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:37.725 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:37.725 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10368 2024-09-21T21:42:37.725 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.725 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:37.725 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.725 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:37.725 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840873+0000", 2024-09-21T21:42:37.725 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:37.725 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.725 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.726 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:37.726 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840873+0000", 2024-09-21T21:42:37.726 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:37.726 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.726 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.726 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:37.726 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840873+0000", 2024-09-21T21:42:37.726 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6699999999999999e-07 2024-09-21T21:42:37.726 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.726 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.726 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:37.726 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840876+0000", 2024-09-21T21:42:37.726 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.6410000000000002e-06 2024-09-21T21:42:37.726 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.727 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.727 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:37.727 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840883+0000", 2024-09-21T21:42:37.727 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0389999999999999e-06 2024-09-21T21:42:37.727 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.727 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.727 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:37.727 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840888+0000", 2024-09-21T21:42:37.727 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8679999999999998e-06 2024-09-21T21:42:37.727 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.727 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.727 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:37.727 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897345+0000", 2024-09-21T21:42:37.727 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056457734000000002 2024-09-21T21:42:37.727 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.728 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.728 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:37.728 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897407+0000", 2024-09-21T21:42:37.728 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1822000000000005e-05 2024-09-21T21:42:37.728 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.728 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.728 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:37.728 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897664+0000", 2024-09-21T21:42:37.728 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00025638499999999998 2024-09-21T21:42:37.728 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.728 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.728 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:37.728 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.901136+0000", 2024-09-21T21:42:37.728 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0034728070000000001 2024-09-21T21:42:37.728 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.729 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.729 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:37.729 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904090+0000", 2024-09-21T21:42:37.729 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0029535799999999999 2024-09-21T21:42:37.729 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.729 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.729 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:37.729 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904144+0000", 2024-09-21T21:42:37.729 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4369000000000001e-05 2024-09-21T21:42:37.729 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.729 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.729 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:37.729 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904268+0000", 2024-09-21T21:42:37.729 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00012367899999999999 2024-09-21T21:42:37.730 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:37.730 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:37.730 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:37.730 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.730 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.730 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10369 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:37.730 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840902+0000", 2024-09-21T21:42:37.730 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 23.819119259000001, 2024-09-21T21:42:37.730 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.063538938000000003, 2024-09-21T21:42:37.730 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:37.730 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:37.730 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:37.730 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:37.730 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:37.731 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:37.731 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10369 2024-09-21T21:42:37.731 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.731 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:37.731 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.731 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:37.731 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840902+0000", 2024-09-21T21:42:37.731 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:37.731 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.731 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.731 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:37.731 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840902+0000", 2024-09-21T21:42:37.731 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:37.731 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.731 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.732 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:37.732 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840903+0000", 2024-09-21T21:42:37.732 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2100000000000002e-07 2024-09-21T21:42:37.732 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.732 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.732 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:37.732 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840913+0000", 2024-09-21T21:42:37.732 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.048e-05 2024-09-21T21:42:37.732 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.732 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.732 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:37.732 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840920+0000", 2024-09-21T21:42:37.732 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0330000000000003e-06 2024-09-21T21:42:37.732 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.733 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.733 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:37.733 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840925+0000", 2024-09-21T21:42:37.733 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0170000000000004e-06 2024-09-21T21:42:37.733 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.733 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.733 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:37.733 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.898179+0000", 2024-09-21T21:42:37.733 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.057253671999999999 2024-09-21T21:42:37.733 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.733 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.733 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:37.733 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.898284+0000", 2024-09-21T21:42:37.733 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000105202 2024-09-21T21:42:37.734 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.734 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.734 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:37.734 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.898729+0000", 2024-09-21T21:42:37.734 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00044459300000000002 2024-09-21T21:42:37.734 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.734 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.734 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:37.734 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902246+0000", 2024-09-21T21:42:37.734 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0035171629999999998 2024-09-21T21:42:37.734 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.734 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.734 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:37.734 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904310+0000", 2024-09-21T21:42:37.734 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.002063765 2024-09-21T21:42:37.735 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.735 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.735 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:37.735 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904342+0000", 2024-09-21T21:42:37.735 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.2091999999999997e-05 2024-09-21T21:42:37.735 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.735 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.735 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:37.735 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904441+0000", 2024-09-21T21:42:37.735 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.9500000000000006e-05 2024-09-21T21:42:37.735 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:37.735 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:37.735 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:37.735 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.736 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.736 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10370 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:37.736 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840944+0000", 2024-09-21T21:42:37.736 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 23.819077545999999, 2024-09-21T21:42:37.736 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.064436853000000002, 2024-09-21T21:42:37.736 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:37.736 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:37.736 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:37.736 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:37.736 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:37.736 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:37.736 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10370 2024-09-21T21:42:37.736 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.736 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:37.737 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.737 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:37.737 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840944+0000", 2024-09-21T21:42:37.737 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:37.737 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.737 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.737 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:37.737 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840944+0000", 2024-09-21T21:42:37.737 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:37.737 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.737 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.737 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:37.737 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840945+0000", 2024-09-21T21:42:37.737 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2300000000000002e-07 2024-09-21T21:42:37.737 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.738 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.738 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:37.738 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840947+0000", 2024-09-21T21:42:37.738 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.8200000000000001e-06 2024-09-21T21:42:37.738 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.738 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.738 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:37.738 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840955+0000", 2024-09-21T21:42:37.738 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.8660000000000006e-06 2024-09-21T21:42:37.738 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.738 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.738 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:37.738 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840960+0000", 2024-09-21T21:42:37.738 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8670000000000002e-06 2024-09-21T21:42:37.739 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.739 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.739 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:37.739 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.899460+0000", 2024-09-21T21:42:37.739 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.058499922000000003 2024-09-21T21:42:37.739 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.739 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.739 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:37.739 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.899530+0000", 2024-09-21T21:42:37.739 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9558e-05 2024-09-21T21:42:37.739 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.739 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.739 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:37.739 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.899775+0000", 2024-09-21T21:42:37.740 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000245231 2024-09-21T21:42:37.740 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.740 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.740 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:37.740 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902254+0000", 2024-09-21T21:42:37.740 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0024790519999999998 2024-09-21T21:42:37.740 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.740 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.740 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:37.740 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905183+0000", 2024-09-21T21:42:37.740 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.002929156 2024-09-21T21:42:37.740 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.740 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.740 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:37.740 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905243+0000", 2024-09-21T21:42:37.741 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0055e-05 2024-09-21T21:42:37.741 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.741 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.741 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:37.741 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905381+0000", 2024-09-21T21:42:37.741 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00013790300000000001 2024-09-21T21:42:37.741 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:37.741 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:37.741 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:37.741 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.741 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.741 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10371 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:37.741 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840980+0000", 2024-09-21T21:42:37.741 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 23.81904188, 2024-09-21T21:42:37.742 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.064554376999999996, 2024-09-21T21:42:37.742 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:37.742 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:37.742 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:37.742 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:37.742 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:37.742 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:37.742 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10371 2024-09-21T21:42:37.742 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.742 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:37.742 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.742 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:37.742 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840980+0000", 2024-09-21T21:42:37.742 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:37.743 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.743 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.743 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:37.743 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840980+0000", 2024-09-21T21:42:37.743 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:37.743 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.743 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.743 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:37.743 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840980+0000", 2024-09-21T21:42:37.743 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8999999999999997e-07 2024-09-21T21:42:37.743 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.743 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.743 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:37.743 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840983+0000", 2024-09-21T21:42:37.744 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.123e-06 2024-09-21T21:42:37.744 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.744 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.744 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:37.744 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840990+0000", 2024-09-21T21:42:37.744 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9129999999999997e-06 2024-09-21T21:42:37.744 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.744 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.744 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:37.744 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840995+0000", 2024-09-21T21:42:37.744 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0989999999999998e-06 2024-09-21T21:42:37.744 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.744 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.744 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:37.744 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.900168+0000", 2024-09-21T21:42:37.745 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.059172739000000002 2024-09-21T21:42:37.745 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.745 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.745 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:37.745 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.900274+0000", 2024-09-21T21:42:37.745 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010546499999999999 2024-09-21T21:42:37.745 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.745 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.745 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:37.745 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.900703+0000", 2024-09-21T21:42:37.745 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00042978200000000003 2024-09-21T21:42:37.745 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.745 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.745 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:37.745 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.903443+0000", 2024-09-21T21:42:37.746 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0027400749999999998 2024-09-21T21:42:37.746 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.746 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.746 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:37.746 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905430+0000", 2024-09-21T21:42:37.746 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0019868109999999998 2024-09-21T21:42:37.746 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.746 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.746 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:37.746 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905464+0000", 2024-09-21T21:42:37.746 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3599000000000001e-05 2024-09-21T21:42:37.746 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.746 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.746 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:37.747 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905534+0000", 2024-09-21T21:42:37.747 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0281e-05 2024-09-21T21:42:37.747 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:37.747 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:37.747 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:37.747 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.747 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.747 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10372 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:37.747 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.841004+0000", 2024-09-21T21:42:37.747 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 23.819017990999999, 2024-09-21T21:42:37.747 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.064757812999999997, 2024-09-21T21:42:37.748 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:37.748 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:37.748 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:37.748 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:37.748 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:37.748 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:37.748 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10372 2024-09-21T21:42:37.748 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.748 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:37.748 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.748 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:37.748 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841004+0000", 2024-09-21T21:42:37.748 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:37.749 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.749 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.749 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:37.749 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841004+0000", 2024-09-21T21:42:37.749 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:37.749 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.749 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.749 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:37.749 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841004+0000", 2024-09-21T21:42:37.749 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8500000000000002e-07 2024-09-21T21:42:37.749 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.749 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.749 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:37.749 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841007+0000", 2024-09-21T21:42:37.749 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.6390000000000001e-06 2024-09-21T21:42:37.750 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.750 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.750 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:37.750 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841013+0000", 2024-09-21T21:42:37.750 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0070000000000003e-06 2024-09-21T21:42:37.750 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.750 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.750 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:37.750 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841018+0000", 2024-09-21T21:42:37.750 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9119999999999997e-06 2024-09-21T21:42:37.750 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.750 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.750 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:37.750 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.901182+0000", 2024-09-21T21:42:37.751 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.060164444999999997 2024-09-21T21:42:37.751 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.751 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.751 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:37.751 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.901297+0000", 2024-09-21T21:42:37.751 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011468699999999999 2024-09-21T21:42:37.751 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.751 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.751 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:37.751 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.901559+0000", 2024-09-21T21:42:37.751 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000261764 2024-09-21T21:42:37.751 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.751 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.751 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:37.752 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904281+0000", 2024-09-21T21:42:37.752 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0027229329999999999 2024-09-21T21:42:37.752 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.752 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.752 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:37.752 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905588+0000", 2024-09-21T21:42:37.752 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.001306438 2024-09-21T21:42:37.752 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.752 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.752 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:37.752 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905644+0000", 2024-09-21T21:42:37.752 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5747e-05 2024-09-21T21:42:37.752 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.752 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.753 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:37.753 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905761+0000", 2024-09-21T21:42:37.753 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000117856 2024-09-21T21:42:37.753 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:37.753 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:37.753 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:37.753 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.753 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.753 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10373 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:37.753 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.841027+0000", 2024-09-21T21:42:37.753 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 23.818994907, 2024-09-21T21:42:37.753 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.066941839000000003, 2024-09-21T21:42:37.753 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:37.753 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:37.754 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:37.754 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:37.754 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:37.754 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:37.754 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10373 2024-09-21T21:42:37.754 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.754 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:37.754 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.754 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:37.754 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841027+0000", 2024-09-21T21:42:37.754 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:37.754 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.755 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.755 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:37.755 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841027+0000", 2024-09-21T21:42:37.755 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:37.755 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.755 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.755 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:37.755 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841027+0000", 2024-09-21T21:42:37.755 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6199999999999999e-07 2024-09-21T21:42:37.755 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.755 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.755 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:37.755 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841030+0000", 2024-09-21T21:42:37.755 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.729e-06 2024-09-21T21:42:37.756 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.756 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.756 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:37.756 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841036+0000", 2024-09-21T21:42:37.756 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1979999999999998e-06 2024-09-21T21:42:37.756 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.756 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.756 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:37.756 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841042+0000", 2024-09-21T21:42:37.756 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4289999999999998e-06 2024-09-21T21:42:37.756 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.756 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.756 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:37.756 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902433+0000", 2024-09-21T21:42:37.757 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.061390329 2024-09-21T21:42:37.757 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.757 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.757 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:37.757 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902541+0000", 2024-09-21T21:42:37.757 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000108349 2024-09-21T21:42:37.757 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.757 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.757 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:37.757 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902983+0000", 2024-09-21T21:42:37.757 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00044158000000000001 2024-09-21T21:42:37.757 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.757 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.757 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:37.758 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904490+0000", 2024-09-21T21:42:37.758 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0015074489999999999 2024-09-21T21:42:37.758 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.758 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.758 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:37.758 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.907671+0000", 2024-09-21T21:42:37.758 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.003180511 2024-09-21T21:42:37.758 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.758 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.758 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:37.758 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.907767+0000", 2024-09-21T21:42:37.758 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6379000000000007e-05 2024-09-21T21:42:37.758 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.758 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.759 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:37.759 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.907969+0000", 2024-09-21T21:42:37.759 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00020152399999999999 2024-09-21T21:42:37.759 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:37.759 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:37.759 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:37.759 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.759 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.759 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:12335 3.1a 3:5b387735:::rbd_data.1121faebdfe3.0000000000000005:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:42:37.759 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:14.411635+0000", 2024-09-21T21:42:37.759 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 23.248387029, 2024-09-21T21:42:37.759 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.005973491, 2024-09-21T21:42:37.759 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:37.759 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:37.760 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:37.760 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:37.760 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:37.760 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:37.760 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 12335 2024-09-21T21:42:37.760 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.760 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:37.760 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.760 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:37.760 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411635+0000", 2024-09-21T21:42:37.760 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:37.760 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.760 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.760 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:37.760 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411635+0000", 2024-09-21T21:42:37.761 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:37.761 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.761 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.761 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:37.761 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411635+0000", 2024-09-21T21:42:37.761 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8500000000000001e-07 2024-09-21T21:42:37.761 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.761 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.761 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:37.761 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411639+0000", 2024-09-21T21:42:37.761 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.574e-06 2024-09-21T21:42:37.761 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.761 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.761 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:37.762 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411645+0000", 2024-09-21T21:42:37.762 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6869999999999997e-06 2024-09-21T21:42:37.762 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.762 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.762 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:37.762 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411649+0000", 2024-09-21T21:42:37.762 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.771e-06 2024-09-21T21:42:37.762 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.762 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.762 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:37.762 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411695+0000", 2024-09-21T21:42:37.762 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6e-05 2024-09-21T21:42:37.762 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.762 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.762 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op must wait for map", 2024-09-21T21:42:37.763 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411702+0000", 2024-09-21T21:42:37.763 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5590000000000002e-06 2024-09-21T21:42:37.763 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.763 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.763 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:37.763 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.417222+0000", 2024-09-21T21:42:37.763 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0055198219999999997 2024-09-21T21:42:37.763 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.763 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.763 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for peered", 2024-09-21T21:42:37.763 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.417230+0000", 2024-09-21T21:42:37.763 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.123e-06 2024-09-21T21:42:37.763 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.763 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.764 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:37.764 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.417348+0000", 2024-09-21T21:42:37.764 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.000118029 2024-09-21T21:42:37.764 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.764 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.764 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:37.764 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.417451+0000", 2024-09-21T21:42:37.764 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000103674 2024-09-21T21:42:37.764 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.764 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.764 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:37.764 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.417608+0000", 2024-09-21T21:42:37.764 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00015666700000000001 2024-09-21T21:42:37.765 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:37.765 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:37.765 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:37.765 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.765 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.765 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:12349 3.1a 3:5b387735:::rbd_data.1121faebdfe3.0000000000000005:head [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:42:37.765 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:14.411833+0000", 2024-09-21T21:42:37.765 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 23.248188215999999, 2024-09-21T21:42:37.765 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0065064500000001, 2024-09-21T21:42:37.765 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:37.765 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:37.765 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:37.765 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:37.765 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:37.766 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:37.766 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 12349 2024-09-21T21:42:37.766 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.766 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:37.766 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.766 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:37.766 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411833+0000", 2024-09-21T21:42:37.766 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:37.766 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.766 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.766 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:37.766 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411833+0000", 2024-09-21T21:42:37.766 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:37.766 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.767 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.767 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:37.767 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411834+0000", 2024-09-21T21:42:37.767 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.03e-07 2024-09-21T21:42:37.767 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.767 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.767 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:37.767 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411836+0000", 2024-09-21T21:42:37.767 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.4700000000000001e-06 2024-09-21T21:42:37.767 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.767 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.767 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:37.767 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411843+0000", 2024-09-21T21:42:37.767 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4069999999999998e-06 2024-09-21T21:42:37.768 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.768 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.768 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:37.768 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411848+0000", 2024-09-21T21:42:37.768 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1580000000000004e-06 2024-09-21T21:42:37.768 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.768 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.768 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:37.768 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411901+0000", 2024-09-21T21:42:37.768 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2824000000000001e-05 2024-09-21T21:42:37.768 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.768 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.768 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting_for_map not empty", 2024-09-21T21:42:37.768 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411904+0000", 2024-09-21T21:42:37.768 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0639999999999998e-06 2024-09-21T21:42:37.769 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.769 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.769 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:37.769 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.417252+0000", 2024-09-21T21:42:37.769 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0053485060000000003 2024-09-21T21:42:37.769 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.769 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.769 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for peered", 2024-09-21T21:42:37.769 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.417259+0000", 2024-09-21T21:42:37.769 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.1149999999999996e-06 2024-09-21T21:42:37.769 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.769 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.769 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:37.769 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.418160+0000", 2024-09-21T21:42:37.770 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.000901083 2024-09-21T21:42:37.770 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.770 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.770 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:37.770 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.418236+0000", 2024-09-21T21:42:37.770 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5852000000000005e-05 2024-09-21T21:42:37.770 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:37.770 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:37.770 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:37.770 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.418340+0000", 2024-09-21T21:42:37.770 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000103568 2024-09-21T21:42:37.770 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:37.770 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:37.770 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:37.771 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:37.771 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:37.771 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:37.771 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T21:42:37.772 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:37.771+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:37.773 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T21:42:37.873 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:37.872+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:37.923 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:37.923 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:42:37.924 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 0 2024-09-21T21:42:37.924 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:37.935 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T21:42:37.941 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0: {} 2024-09-21T21:42:37.942 INFO:teuthology.orchestra.run.smithi081.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T21:42:37.959 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1: {} 2024-09-21T21:42:37.959 INFO:teuthology.orchestra.run.smithi081.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T21:42:37.973 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:37.972+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:37.975 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2: {} 2024-09-21T21:42:37.976 INFO:teuthology.orchestra.run.smithi081.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T21:42:37.992 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3: {} 2024-09-21T21:42:37.992 INFO:teuthology.orchestra.run.smithi081.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T21:42:38.009 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4: {} 2024-09-21T21:42:38.009 INFO:teuthology.orchestra.run.smithi081.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T21:42:38.026 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5: {} 2024-09-21T21:42:38.026 INFO:teuthology.orchestra.run.smithi081.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T21:42:38.039 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:38.039 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:38.048 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T21:42:38.075 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:38.073+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:38.176 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:38.175+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:38.192 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:38.192 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:38.200 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T21:42:38.277 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:38.276+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:38.307 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:38.307 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:38.316 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T21:42:38.379 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:38.377+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:38.418 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:38.418 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:38.426 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T21:42:38.479 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:38.478+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:38.569 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:38.569 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:38.577 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T21:42:38.579 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T21:42:38.580 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:38.578+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:38.680 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:38.679+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:38.690 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:38.691 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:38.700 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T21:42:38.782 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:38.781+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:38.802 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:38.803 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:38.811 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T21:42:38.832 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 5, 3] out_osds: [0, 4] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-21T21:42:38.832 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T21:42:38.832 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T21:42:38.883 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:38.881+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:38.914 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:38.914 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:38.923 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T21:42:38.984 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:38.982+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:39.027 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:39.027 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:39.036 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T21:42:39.085 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:39.084+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:39.109 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:42:39.110 INFO:teuthology.orchestra.run.smithi081.stderr:dumped all 2024-09-21T21:42:39.121 INFO:teuthology.orchestra.run.smithi081.stdout:{"pg_ready":true,"pg_map":{"version":182,"stamp":"2024-09-21T21:42:38.570546+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1339206619,"num_objects":1922,"num_object_clones":440,"num_object_copies":3844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1922,"num_whiteouts":561,"num_read":394843,"num_read_kb":1445801,"num_write":25461,"num_write_kb":319218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":186957635,"num_keys_recovered":717,"num_objects_omap":47,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20930,"ondisk_log_size":20930,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":307820,"kb_used_data":62272,"kb_used_omap":6,"kb_used_meta":245433,"kb_avail":377179540,"statfs":{"total":386547056640,"available":386231848960,"internally_reserved":0,"allocated":63766528,"data_stored":479176702,"data_compressed":31787869,"data_compressed_allocated":34590720,"data_compressed_original":457875456,"omap_allocated":6378,"internal_metadata":251324182},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":36355135,"num_objects":89,"num_object_clones":18,"num_object_copies":178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89,"num_whiteouts":65,"num_read":15414,"num_read_kb":92810,"num_write":758,"num_write_kb":21068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.258258"},"pg_stats":[{"pgid":"3.1f","version":"112'458","reported_seq":6899,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T21:42:34.301155+0000","last_change":"2024-09-21T21:42:03.214391+0000","last_active":"2024-09-21T21:42:34.301155+0000","last_peered":"2024-09-21T21:42:34.301155+0000","last_clean":"2024-09-21T21:42:34.301155+0000","last_became_active":"2024-09-21T21:42:03.212171+0000","last_became_peered":"2024-09-21T21:42:03.212171+0000","last_unstale":"2024-09-21T21:42:34.301155+0000","last_undegraded":"2024-09-21T21:42:34.301155+0000","last_fullsized":"2024-09-21T21:42:34.301155+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":458,"log_dups_size":0,"ondisk_log_size":458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:38:12.333537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010985019999999999,"stat_sum":{"num_bytes":32916593,"num_objects":57,"num_object_clones":13,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":15,"num_read":7589,"num_read_kb":23979,"num_write":441,"num_write_kb":8479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":158,"num_bytes_recovered":3347584,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"112'903","reported_seq":14741,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T21:42:34.265141+0000","last_change":"2024-09-21T21:41:53.217241+0000","last_active":"2024-09-21T21:42:34.265141+0000","last_peered":"2024-09-21T21:42:34.265141+0000","last_clean":"2024-09-21T21:42:34.265141+0000","last_became_active":"2024-09-21T21:41:53.215776+0000","last_became_peered":"2024-09-21T21:41:53.215776+0000","last_unstale":"2024-09-21T21:42:34.265141+0000","last_undegraded":"2024-09-21T21:42:34.265141+0000","last_fullsized":"2024-09-21T21:42:34.265141+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":903,"log_dups_size":0,"ondisk_log_size":903,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:34:23.114794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042608099999999997,"stat_sum":{"num_bytes":42414151,"num_objects":64,"num_object_clones":13,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":21,"num_read":15911,"num_read_kb":72499,"num_write":1389,"num_write_kb":17377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":14291598,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"115'855","reported_seq":9856,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T21:42:37.717331+0000","last_change":"2024-09-21T21:42:37.573612+0000","last_active":"2024-09-21T21:42:37.717331+0000","last_peered":"2024-09-21T21:42:37.717331+0000","last_clean":"2024-09-21T21:42:37.717331+0000","last_became_active":"2024-09-21T21:42:24.517759+0000","last_became_peered":"2024-09-21T21:42:24.517759+0000","last_unstale":"2024-09-21T21:42:37.717331+0000","last_undegraded":"2024-09-21T21:42:37.717331+0000","last_fullsized":"2024-09-21T21:42:37.717331+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":855,"log_dups_size":0,"ondisk_log_size":855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:50:15.731193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098670899999999989,"stat_sum":{"num_bytes":39247258,"num_objects":59,"num_object_clones":11,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":19,"num_read":12718,"num_read_kb":43114,"num_write":1058,"num_write_kb":7887,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":92,"num_bytes_recovered":3929994,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2f","length":"2"}]},{"pgid":"3.1c","version":"115'747","reported_seq":12196,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T21:42:37.684906+0000","last_change":"2024-09-21T21:42:37.574219+0000","last_active":"2024-09-21T21:42:37.684906+0000","last_peered":"2024-09-21T21:42:37.684906+0000","last_clean":"2024-09-21T21:42:37.684906+0000","last_became_active":"2024-09-21T21:42:24.516469+0000","last_became_peered":"2024-09-21T21:42:24.516469+0000","last_unstale":"2024-09-21T21:42:37.684906+0000","last_undegraded":"2024-09-21T21:42:37.684906+0000","last_fullsized":"2024-09-21T21:42:37.684906+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":747,"log_dups_size":0,"ondisk_log_size":747,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:35:18.162405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013174580000000001,"stat_sum":{"num_bytes":32479322,"num_objects":44,"num_object_clones":10,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":14,"num_read":14564,"num_read_kb":34482,"num_write":773,"num_write_kb":7795,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":1229402,"num_keys_recovered":99,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2f","length":"2"}]},{"pgid":"3.1b","version":"112'606","reported_seq":7759,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T21:42:34.301117+0000","last_change":"2024-09-21T21:42:16.417817+0000","last_active":"2024-09-21T21:42:34.301117+0000","last_peered":"2024-09-21T21:42:34.301117+0000","last_clean":"2024-09-21T21:42:34.301117+0000","last_became_active":"2024-09-21T21:42:16.416438+0000","last_became_peered":"2024-09-21T21:42:16.416438+0000","last_unstale":"2024-09-21T21:42:34.301117+0000","last_undegraded":"2024-09-21T21:42:34.301117+0000","last_fullsized":"2024-09-21T21:42:34.301117+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":606,"log_dups_size":0,"ondisk_log_size":606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:15:02.839147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069216500000000001,"stat_sum":{"num_bytes":54121746,"num_objects":69,"num_object_clones":19,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":22,"num_read":7733,"num_read_kb":32854,"num_write":753,"num_write_kb":4909,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":223,"num_bytes_recovered":1599826,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"115'518","reported_seq":24538,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T21:42:37.699081+0000","last_change":"2024-09-21T21:42:37.576746+0000","last_active":"2024-09-21T21:42:37.699081+0000","last_peered":"2024-09-21T21:42:37.699081+0000","last_clean":"2024-09-21T21:42:37.699081+0000","last_became_active":"2024-09-21T21:42:15.416423+0000","last_became_peered":"2024-09-21T21:42:15.416423+0000","last_unstale":"2024-09-21T21:42:37.699081+0000","last_undegraded":"2024-09-21T21:42:37.699081+0000","last_fullsized":"2024-09-21T21:42:37.699081+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:04:04.552339+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.011043259,"stat_sum":{"num_bytes":36781213,"num_objects":56,"num_object_clones":8,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":16,"num_read":24395,"num_read_kb":58526,"num_write":503,"num_write_kb":7766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":4395677,"num_keys_recovered":22,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2f","length":"2"}]},{"pgid":"3.19","version":"113'723","reported_seq":13121,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T21:42:37.565661+0000","last_change":"2024-09-21T21:42:37.566173+0000","last_active":"2024-09-21T21:42:37.565661+0000","last_peered":"2024-09-21T21:42:37.565661+0000","last_clean":"2024-09-21T21:42:37.565661+0000","last_became_active":"2024-09-21T21:42:15.421101+0000","last_became_peered":"2024-09-21T21:42:15.421101+0000","last_unstale":"2024-09-21T21:42:37.565661+0000","last_undegraded":"2024-09-21T21:42:37.565661+0000","last_fullsized":"2024-09-21T21:42:37.565661+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":723,"log_dups_size":0,"ondisk_log_size":723,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:41:28.249312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001205315,"stat_sum":{"num_bytes":44894985,"num_objects":74,"num_object_clones":13,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":25,"num_read":15281,"num_read_kb":61080,"num_write":762,"num_write_kb":15357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":9861823,"num_keys_recovered":35,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2f","length":"2"}]},{"pgid":"3.18","version":"115'288","reported_seq":6573,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T21:42:37.718176+0000","last_change":"2024-09-21T21:42:37.566702+0000","last_active":"2024-09-21T21:42:37.718176+0000","last_peered":"2024-09-21T21:42:37.718176+0000","last_clean":"2024-09-21T21:42:37.718176+0000","last_became_active":"2024-09-21T21:42:03.211916+0000","last_became_peered":"2024-09-21T21:42:03.211916+0000","last_unstale":"2024-09-21T21:42:37.718176+0000","last_undegraded":"2024-09-21T21:42:37.718176+0000","last_fullsized":"2024-09-21T21:42:37.718176+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":288,"log_dups_size":0,"ondisk_log_size":288,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:47:55.519551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001118911,"stat_sum":{"num_bytes":47243296,"num_objects":48,"num_object_clones":14,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":15,"num_read":6155,"num_read_kb":33021,"num_write":306,"num_write_kb":8639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":59,"num_bytes_recovered":3271696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2f","length":"2"}]},{"pgid":"3.17","version":"115'403","reported_seq":5893,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T21:42:37.581928+0000","last_change":"2024-09-21T21:42:37.582204+0000","last_active":"2024-09-21T21:42:37.581928+0000","last_peered":"2024-09-21T21:42:37.581928+0000","last_clean":"2024-09-21T21:42:37.581928+0000","last_became_active":"2024-09-21T21:42:03.213418+0000","last_became_peered":"2024-09-21T21:42:03.213418+0000","last_unstale":"2024-09-21T21:42:37.581928+0000","last_undegraded":"2024-09-21T21:42:37.581928+0000","last_fullsized":"2024-09-21T21:42:37.581928+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":403,"log_dups_size":0,"ondisk_log_size":403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:45:19.151682+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0094614429999999999,"stat_sum":{"num_bytes":27399802,"num_objects":62,"num_object_clones":17,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":17,"num_read":6216,"num_read_kb":25745,"num_write":465,"num_write_kb":9168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":3723386,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2f","length":"2"}]},{"pgid":"3.16","version":"115'1109","reported_seq":9214,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T21:42:37.580112+0000","last_change":"2024-09-21T21:42:37.580359+0000","last_active":"2024-09-21T21:42:37.580112+0000","last_peered":"2024-09-21T21:42:37.580112+0000","last_clean":"2024-09-21T21:42:37.580112+0000","last_became_active":"2024-09-21T21:41:53.214270+0000","last_became_peered":"2024-09-21T21:41:53.214270+0000","last_unstale":"2024-09-21T21:42:37.580112+0000","last_undegraded":"2024-09-21T21:42:37.580112+0000","last_fullsized":"2024-09-21T21:42:37.580112+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":1109,"log_dups_size":0,"ondisk_log_size":1109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:13:04.104504+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0072705510000000001,"stat_sum":{"num_bytes":45410001,"num_objects":65,"num_object_clones":14,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":14,"num_read":12811,"num_read_kb":44024,"num_write":1702,"num_write_kb":12819,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":148,"num_bytes_recovered":4869268,"num_keys_recovered":47,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2f","length":"2"}]},{"pgid":"3.15","version":"111'663","reported_seq":8513,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T21:42:34.301181+0000","last_change":"2024-09-21T21:42:16.420053+0000","last_active":"2024-09-21T21:42:34.301181+0000","last_peered":"2024-09-21T21:42:34.301181+0000","last_clean":"2024-09-21T21:42:34.301181+0000","last_became_active":"2024-09-21T21:42:16.418172+0000","last_became_peered":"2024-09-21T21:42:16.418172+0000","last_unstale":"2024-09-21T21:42:34.301181+0000","last_undegraded":"2024-09-21T21:42:34.301181+0000","last_fullsized":"2024-09-21T21:42:34.301181+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":663,"log_dups_size":0,"ondisk_log_size":663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:05:34.054529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001006961,"stat_sum":{"num_bytes":37038678,"num_objects":52,"num_object_clones":10,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":12,"num_read":8886,"num_read_kb":54749,"num_write":916,"num_write_kb":7163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":385,"num_bytes_recovered":9565806,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"112'500","reported_seq":20915,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T21:42:33.658599+0000","last_change":"2024-09-21T21:42:16.418075+0000","last_active":"2024-09-21T21:42:33.658599+0000","last_peered":"2024-09-21T21:42:33.658599+0000","last_clean":"2024-09-21T21:42:33.658599+0000","last_became_active":"2024-09-21T21:42:16.416899+0000","last_became_peered":"2024-09-21T21:42:16.416899+0000","last_unstale":"2024-09-21T21:42:33.658599+0000","last_undegraded":"2024-09-21T21:42:33.658599+0000","last_fullsized":"2024-09-21T21:42:33.658599+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":500,"log_dups_size":0,"ondisk_log_size":500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:59:54.574427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093907599999999995,"stat_sum":{"num_bytes":43906724,"num_objects":54,"num_object_clones":13,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":15,"num_read":21495,"num_read_kb":74341,"num_write":434,"num_write_kb":10927,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":227,"num_bytes_recovered":15558124,"num_keys_recovered":84,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":85,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T21:42:33.547706+0000","last_change":"2024-09-21T21:42:12.214789+0000","last_active":"2024-09-21T21:42:33.547706+0000","last_peered":"2024-09-21T21:42:33.547706+0000","last_clean":"2024-09-21T21:42:33.547706+0000","last_became_active":"2024-09-21T21:42:12.214436+0000","last_became_peered":"2024-09-21T21:42:12.214436+0000","last_unstale":"2024-09-21T21:42:33.547706+0000","last_undegraded":"2024-09-21T21:42:33.547706+0000","last_fullsized":"2024-09-21T21:42:33.547706+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:45:03.040098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"115'1068","reported_seq":13604,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T21:42:37.574914+0000","last_change":"2024-09-21T21:42:37.574914+0000","last_active":"2024-09-21T21:42:37.574914+0000","last_peered":"2024-09-21T21:42:37.574914+0000","last_clean":"2024-09-21T21:42:37.574914+0000","last_became_active":"2024-09-21T21:42:15.419283+0000","last_became_peered":"2024-09-21T21:42:15.419283+0000","last_unstale":"2024-09-21T21:42:37.574914+0000","last_undegraded":"2024-09-21T21:42:37.574914+0000","last_fullsized":"2024-09-21T21:42:37.574914+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":1068,"log_dups_size":0,"ondisk_log_size":1068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:42:29.083722+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0092762569999999996,"stat_sum":{"num_bytes":45708908,"num_objects":67,"num_object_clones":20,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":27,"num_read":15380,"num_read_kb":65943,"num_write":1509,"num_write_kb":15970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":193,"num_bytes_recovered":6876276,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2f","length":"2"}]},{"pgid":"2.6","version":"0'0","reported_seq":222,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T21:42:37.565800+0000","last_change":"2024-09-21T21:41:49.266534+0000","last_active":"2024-09-21T21:42:37.565800+0000","last_peered":"2024-09-21T21:42:37.565800+0000","last_clean":"2024-09-21T21:42:37.565800+0000","last_became_active":"2024-09-21T21:41:49.266257+0000","last_became_peered":"2024-09-21T21:41:49.266257+0000","last_unstale":"2024-09-21T21:42:37.565800+0000","last_undegraded":"2024-09-21T21:42:37.565800+0000","last_fullsized":"2024-09-21T21:42:37.565800+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:38:31.063193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062280799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"114'445","reported_seq":8161,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T21:42:37.567401+0000","last_change":"2024-09-21T21:42:37.567963+0000","last_active":"2024-09-21T21:42:37.567401+0000","last_peered":"2024-09-21T21:42:37.567401+0000","last_clean":"2024-09-21T21:42:37.567401+0000","last_became_active":"2024-09-21T21:41:25.651977+0000","last_became_peered":"2024-09-21T21:41:25.651977+0000","last_unstale":"2024-09-21T21:42:37.567401+0000","last_undegraded":"2024-09-21T21:42:37.567401+0000","last_fullsized":"2024-09-21T21:42:37.567401+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":445,"log_dups_size":0,"ondisk_log_size":445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:20:11.871151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025709740000000002,"stat_sum":{"num_bytes":33515127,"num_objects":51,"num_object_clones":9,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":13,"num_read":10062,"num_read_kb":39991,"num_write":447,"num_write_kb":10588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2f","length":"2"}]},{"pgid":"2.5","version":"0'0","reported_seq":90,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T21:42:33.547564+0000","last_change":"2024-09-21T21:42:12.214946+0000","last_active":"2024-09-21T21:42:33.547564+0000","last_peered":"2024-09-21T21:42:33.547564+0000","last_clean":"2024-09-21T21:42:33.547564+0000","last_became_active":"2024-09-21T21:42:12.214648+0000","last_became_peered":"2024-09-21T21:42:12.214648+0000","last_unstale":"2024-09-21T21:42:33.547564+0000","last_undegraded":"2024-09-21T21:42:33.547564+0000","last_fullsized":"2024-09-21T21:42:33.547564+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:54:09.452995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"115'801","reported_seq":4299,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T21:42:37.666995+0000","last_change":"2024-09-21T21:42:37.567296+0000","last_active":"2024-09-21T21:42:37.666995+0000","last_peered":"2024-09-21T21:42:37.666995+0000","last_clean":"2024-09-21T21:42:37.666995+0000","last_became_active":"2024-09-21T21:40:37.419944+0000","last_became_peered":"2024-09-21T21:40:37.419944+0000","last_unstale":"2024-09-21T21:42:37.666995+0000","last_undegraded":"2024-09-21T21:42:37.666995+0000","last_fullsized":"2024-09-21T21:42:37.666995+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":801,"log_dups_size":0,"ondisk_log_size":801,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:18:39.453348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019928900000000002,"stat_sum":{"num_bytes":23613499,"num_objects":56,"num_object_clones":14,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":13,"num_read":5852,"num_read_kb":25203,"num_write":1209,"num_write_kb":11312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2f","length":"2"}]},{"pgid":"2.4","version":"0'0","reported_seq":222,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T21:42:37.565474+0000","last_change":"2024-09-21T21:42:12.214652+0000","last_active":"2024-09-21T21:42:37.565474+0000","last_peered":"2024-09-21T21:42:37.565474+0000","last_clean":"2024-09-21T21:42:37.565474+0000","last_became_active":"2024-09-21T21:42:12.214348+0000","last_became_peered":"2024-09-21T21:42:12.214348+0000","last_unstale":"2024-09-21T21:42:37.565474+0000","last_undegraded":"2024-09-21T21:42:37.565474+0000","last_fullsized":"2024-09-21T21:42:37.565474+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:55:56.386112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070669899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"113'539","reported_seq":9875,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T21:42:37.573433+0000","last_change":"2024-09-21T21:42:37.573932+0000","last_active":"2024-09-21T21:42:37.573433+0000","last_peered":"2024-09-21T21:42:37.573433+0000","last_clean":"2024-09-21T21:42:37.573433+0000","last_became_active":"2024-09-21T21:41:25.650379+0000","last_became_peered":"2024-09-21T21:41:25.650379+0000","last_unstale":"2024-09-21T21:42:37.573433+0000","last_undegraded":"2024-09-21T21:42:37.573433+0000","last_fullsized":"2024-09-21T21:42:37.573433+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":539,"log_dups_size":0,"ondisk_log_size":539,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:22:08.416162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001010557,"stat_sum":{"num_bytes":46469167,"num_objects":61,"num_object_clones":15,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":20,"num_read":10915,"num_read_kb":31732,"num_write":457,"num_write_kb":8327,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2f","length":"2"}]},{"pgid":"3.3","version":"112'383","reported_seq":10252,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T21:42:34.256527+0000","last_change":"2024-09-21T21:42:15.419928+0000","last_active":"2024-09-21T21:42:34.256527+0000","last_peered":"2024-09-21T21:42:34.256527+0000","last_clean":"2024-09-21T21:42:34.256527+0000","last_became_active":"2024-09-21T21:42:15.418642+0000","last_became_peered":"2024-09-21T21:42:15.418642+0000","last_unstale":"2024-09-21T21:42:34.256527+0000","last_undegraded":"2024-09-21T21:42:34.256527+0000","last_fullsized":"2024-09-21T21:42:34.256527+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:13:12.359442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041679500000000002,"stat_sum":{"num_bytes":16652816,"num_objects":49,"num_object_clones":11,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":12,"num_read":9305,"num_read_kb":35441,"num_write":403,"num_write_kb":9668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":171,"num_bytes_recovered":13125136,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":87,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T21:42:37.572486+0000","last_change":"2024-09-21T21:42:24.516929+0000","last_active":"2024-09-21T21:42:37.572486+0000","last_peered":"2024-09-21T21:42:37.572486+0000","last_clean":"2024-09-21T21:42:37.572486+0000","last_became_active":"2024-09-21T21:42:24.516623+0000","last_became_peered":"2024-09-21T21:42:24.516623+0000","last_unstale":"2024-09-21T21:42:37.572486+0000","last_undegraded":"2024-09-21T21:42:37.572486+0000","last_fullsized":"2024-09-21T21:42:37.572486+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:51:59.592292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000906268,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"115'702","reported_seq":10808,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T21:42:37.571237+0000","last_change":"2024-09-21T21:42:37.571237+0000","last_active":"2024-09-21T21:42:37.571237+0000","last_peered":"2024-09-21T21:42:37.571237+0000","last_clean":"2024-09-21T21:42:37.571237+0000","last_became_active":"2024-09-21T21:40:37.418973+0000","last_became_peered":"2024-09-21T21:40:37.418973+0000","last_unstale":"2024-09-21T21:42:37.571237+0000","last_undegraded":"2024-09-21T21:42:37.571237+0000","last_fullsized":"2024-09-21T21:42:37.571237+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":702,"log_dups_size":0,"ondisk_log_size":702,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:56:40.826132+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0060728600000000002,"stat_sum":{"num_bytes":43942565,"num_objects":70,"num_object_clones":22,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":22,"num_read":13015,"num_read_kb":27792,"num_write":544,"num_write_kb":6281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2f","length":"2"}]},{"pgid":"2.1","version":"0'0","reported_seq":129,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T21:42:37.564944+0000","last_change":"2024-09-21T21:41:25.652142+0000","last_active":"2024-09-21T21:42:37.564944+0000","last_peered":"2024-09-21T21:42:37.564944+0000","last_clean":"2024-09-21T21:42:37.564944+0000","last_became_active":"2024-09-21T21:41:25.651832+0000","last_became_peered":"2024-09-21T21:41:25.651832+0000","last_unstale":"2024-09-21T21:42:37.564944+0000","last_undegraded":"2024-09-21T21:42:37.564944+0000","last_fullsized":"2024-09-21T21:42:37.564944+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:37:21.188619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"113'331","reported_seq":16188,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T21:42:37.574264+0000","last_change":"2024-09-21T21:42:37.574704+0000","last_active":"2024-09-21T21:42:37.574264+0000","last_peered":"2024-09-21T21:42:37.574264+0000","last_clean":"2024-09-21T21:42:37.574264+0000","last_became_active":"2024-09-21T21:42:24.518786+0000","last_became_peered":"2024-09-21T21:42:24.518786+0000","last_unstale":"2024-09-21T21:42:37.574264+0000","last_undegraded":"2024-09-21T21:42:37.574264+0000","last_fullsized":"2024-09-21T21:42:37.574264+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":331,"log_dups_size":0,"ondisk_log_size":331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:46:57.049208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017676390000000001,"stat_sum":{"num_bytes":56085070,"num_objects":67,"num_object_clones":17,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":28,"num_read":15385,"num_read_kb":48499,"num_write":355,"num_write_kb":4859,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":161,"num_bytes_recovered":5746844,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2f","length":"2"}]},{"pgid":"2.0","version":"0'0","reported_seq":80,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T21:42:33.547270+0000","last_change":"2024-09-21T21:41:49.265465+0000","last_active":"2024-09-21T21:42:33.547270+0000","last_peered":"2024-09-21T21:42:33.547270+0000","last_clean":"2024-09-21T21:42:33.547270+0000","last_became_active":"2024-09-21T21:41:49.265182+0000","last_became_peered":"2024-09-21T21:41:49.265182+0000","last_unstale":"2024-09-21T21:42:33.547270+0000","last_undegraded":"2024-09-21T21:42:33.547270+0000","last_fullsized":"2024-09-21T21:42:33.547270+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:19:17.973195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"113'676","reported_seq":9664,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T21:42:37.575006+0000","last_change":"2024-09-21T21:42:37.575209+0000","last_active":"2024-09-21T21:42:37.575006+0000","last_peered":"2024-09-21T21:42:37.575006+0000","last_clean":"2024-09-21T21:42:37.575006+0000","last_became_active":"2024-09-21T21:40:37.421655+0000","last_became_peered":"2024-09-21T21:40:37.421655+0000","last_unstale":"2024-09-21T21:42:37.575006+0000","last_undegraded":"2024-09-21T21:42:37.575006+0000","last_fullsized":"2024-09-21T21:42:37.575006+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":676,"log_dups_size":0,"ondisk_log_size":676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:14:18.133394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0089506629999999993,"stat_sum":{"num_bytes":27928845,"num_objects":51,"num_object_clones":8,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":15,"num_read":12226,"num_read_kb":33963,"num_write":682,"num_write_kb":7482,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2f","length":"2"}]},{"pgid":"2.3","version":"17'1","reported_seq":185,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T21:42:37.572622+0000","last_change":"2024-09-21T21:42:24.518810+0000","last_active":"2024-09-21T21:42:37.572622+0000","last_peered":"2024-09-21T21:42:37.572622+0000","last_clean":"2024-09-21T21:42:37.572622+0000","last_became_active":"2024-09-21T21:42:24.518461+0000","last_became_peered":"2024-09-21T21:42:24.518461+0000","last_unstale":"2024-09-21T21:42:37.572622+0000","last_undegraded":"2024-09-21T21:42:37.572622+0000","last_fullsized":"2024-09-21T21:42:37.572622+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:17:47.593497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036803999999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":274,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T21:42:33.547385+0000","last_change":"2024-09-21T21:42:03.212447+0000","last_active":"2024-09-21T21:42:33.547385+0000","last_peered":"2024-09-21T21:42:33.547385+0000","last_clean":"2024-09-21T21:42:33.547385+0000","last_became_active":"2024-09-21T21:42:03.211881+0000","last_became_peered":"2024-09-21T21:42:03.211881+0000","last_unstale":"2024-09-21T21:42:33.547385+0000","last_undegraded":"2024-09-21T21:42:33.547385+0000","last_fullsized":"2024-09-21T21:42:33.547385+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:18.340430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:18.340430+0000","last_clean_scrub_stamp":"2024-09-21T21:40:18.340430+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:09:02.519349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"111'309","reported_seq":5659,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T21:42:34.301509+0000","last_change":"2024-09-21T21:41:52.214231+0000","last_active":"2024-09-21T21:42:34.301509+0000","last_peered":"2024-09-21T21:42:34.301509+0000","last_clean":"2024-09-21T21:42:34.301509+0000","last_became_active":"2024-09-21T21:41:52.213238+0000","last_became_peered":"2024-09-21T21:41:52.213238+0000","last_unstale":"2024-09-21T21:42:34.301509+0000","last_undegraded":"2024-09-21T21:42:34.301509+0000","last_fullsized":"2024-09-21T21:42:34.301509+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":309,"log_dups_size":0,"ondisk_log_size":309,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:07:39.494607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056255200000000002,"stat_sum":{"num_bytes":27186265,"num_objects":52,"num_object_clones":11,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":14,"num_read":5047,"num_read_kb":24589,"num_write":323,"num_write_kb":7251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":172,"num_bytes_recovered":2229995,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"113'481","reported_seq":8128,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-21T21:42:35.550777+0000","last_change":"2024-09-21T21:42:18.480957+0000","last_active":"2024-09-21T21:42:35.550777+0000","last_peered":"2024-09-21T21:42:35.550777+0000","last_clean":"2024-09-21T21:42:35.550777+0000","last_became_active":"2024-09-21T21:42:18.480134+0000","last_became_peered":"2024-09-21T21:42:18.480134+0000","last_unstale":"2024-09-21T21:42:35.550777+0000","last_undegraded":"2024-09-21T21:42:35.550777+0000","last_fullsized":"2024-09-21T21:42:35.550777+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":481,"log_dups_size":0,"ondisk_log_size":481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:08:09.499616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035788200000000001,"stat_sum":{"num_bytes":56720125,"num_objects":58,"num_object_clones":11,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":17,"num_read":8622,"num_read_kb":47238,"num_write":479,"num_write_kb":9428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":107,"num_bytes_recovered":6946045,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"112'910","reported_seq":10591,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T21:42:34.304040+0000","last_change":"2024-09-21T21:42:16.418562+0000","last_active":"2024-09-21T21:42:34.304040+0000","last_peered":"2024-09-21T21:42:34.304040+0000","last_clean":"2024-09-21T21:42:34.304040+0000","last_became_active":"2024-09-21T21:42:16.416788+0000","last_became_peered":"2024-09-21T21:42:16.416788+0000","last_unstale":"2024-09-21T21:42:34.304040+0000","last_undegraded":"2024-09-21T21:42:34.304040+0000","last_fullsized":"2024-09-21T21:42:34.304040+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":910,"log_dups_size":0,"ondisk_log_size":910,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:30:03.280136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065146799999999999,"stat_sum":{"num_bytes":60092658,"num_objects":60,"num_object_clones":11,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":13,"num_read":12642,"num_read_kb":48198,"num_write":1114,"num_write_kb":10610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":14636658,"num_keys_recovered":136,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"114'716","reported_seq":9583,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T21:42:37.568039+0000","last_change":"2024-09-21T21:42:37.568220+0000","last_active":"2024-09-21T21:42:37.568039+0000","last_peered":"2024-09-21T21:42:37.568039+0000","last_clean":"2024-09-21T21:42:37.568039+0000","last_became_active":"2024-09-21T21:42:16.416994+0000","last_became_peered":"2024-09-21T21:42:16.416994+0000","last_unstale":"2024-09-21T21:42:37.568039+0000","last_undegraded":"2024-09-21T21:42:37.568039+0000","last_fullsized":"2024-09-21T21:42:37.568039+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":716,"log_dups_size":0,"ondisk_log_size":716,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:12:47.490764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028242929999999999,"stat_sum":{"num_bytes":31207733,"num_objects":54,"num_object_clones":11,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":15,"num_read":10197,"num_read_kb":45337,"num_write":1093,"num_write_kb":14641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":189,"num_bytes_recovered":7872106,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2f","length":"2"}]},{"pgid":"3.c","version":"115'1484","reported_seq":13126,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T21:42:37.715361+0000","last_change":"2024-09-21T21:42:37.580702+0000","last_active":"2024-09-21T21:42:37.715361+0000","last_peered":"2024-09-21T21:42:37.715361+0000","last_clean":"2024-09-21T21:42:37.715361+0000","last_became_active":"2024-09-21T21:42:24.517955+0000","last_became_peered":"2024-09-21T21:42:24.517955+0000","last_unstale":"2024-09-21T21:42:37.715361+0000","last_undegraded":"2024-09-21T21:42:37.715361+0000","last_fullsized":"2024-09-21T21:42:37.715361+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":1484,"log_dups_size":0,"ondisk_log_size":1484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:25:52.179558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.007356619,"stat_sum":{"num_bytes":46455675,"num_objects":71,"num_object_clones":18,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":18,"num_read":18210,"num_read_kb":61017,"num_write":2247,"num_write_kb":11519,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":4225300,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2f","length":"2"}]},{"pgid":"3.d","version":"112'1262","reported_seq":16622,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T21:42:33.686152+0000","last_change":"2024-09-21T21:41:52.217062+0000","last_active":"2024-09-21T21:42:33.686152+0000","last_peered":"2024-09-21T21:42:33.686152+0000","last_clean":"2024-09-21T21:42:33.686152+0000","last_became_active":"2024-09-21T21:41:52.214022+0000","last_became_peered":"2024-09-21T21:41:52.214022+0000","last_unstale":"2024-09-21T21:42:33.686152+0000","last_undegraded":"2024-09-21T21:42:33.686152+0000","last_fullsized":"2024-09-21T21:42:33.686152+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":1262,"log_dups_size":0,"ondisk_log_size":1262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:13:25.256566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098736399999999991,"stat_sum":{"num_bytes":44398452,"num_objects":71,"num_object_clones":18,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":19,"num_read":20140,"num_read_kb":43573,"num_write":1713,"num_write_kb":13768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":149,"num_bytes_recovered":3462638,"num_keys_recovered":36,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"115'389","reported_seq":7187,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T21:42:37.575352+0000","last_change":"2024-09-21T21:42:37.575352+0000","last_active":"2024-09-21T21:42:37.575352+0000","last_peered":"2024-09-21T21:42:37.575352+0000","last_clean":"2024-09-21T21:42:37.575352+0000","last_became_active":"2024-09-21T21:42:24.517033+0000","last_became_peered":"2024-09-21T21:42:24.517033+0000","last_unstale":"2024-09-21T21:42:37.575352+0000","last_undegraded":"2024-09-21T21:42:37.575352+0000","last_fullsized":"2024-09-21T21:42:37.575352+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":389,"log_dups_size":0,"ondisk_log_size":389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:24:00.788978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021375750000000001,"stat_sum":{"num_bytes":37484672,"num_objects":52,"num_object_clones":10,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":19,"num_read":6496,"num_read_kb":34811,"num_write":421,"num_write_kb":6853,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":3302528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2f","length":"2"}]},{"pgid":"3.f","version":"114'607","reported_seq":16464,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T21:42:37.574796+0000","last_change":"2024-09-21T21:42:37.575051+0000","last_active":"2024-09-21T21:42:37.574796+0000","last_peered":"2024-09-21T21:42:37.574796+0000","last_clean":"2024-09-21T21:42:37.574796+0000","last_became_active":"2024-09-21T21:41:52.212206+0000","last_became_peered":"2024-09-21T21:41:52.212206+0000","last_unstale":"2024-09-21T21:42:37.574796+0000","last_undegraded":"2024-09-21T21:42:37.574796+0000","last_fullsized":"2024-09-21T21:42:37.574796+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":607,"log_dups_size":0,"ondisk_log_size":607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:19:20.899990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001885919,"stat_sum":{"num_bytes":30786231,"num_objects":77,"num_object_clones":21,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":18,"num_read":17837,"num_read_kb":41938,"num_write":589,"num_write_kb":9488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":2967886,"num_keys_recovered":48,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2f","length":"2"}]},{"pgid":"3.10","version":"114'355","reported_seq":13573,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-21T21:42:37.573689+0000","last_change":"2024-09-21T21:42:37.574284+0000","last_active":"2024-09-21T21:42:37.573689+0000","last_peered":"2024-09-21T21:42:37.573689+0000","last_clean":"2024-09-21T21:42:37.573689+0000","last_became_active":"2024-09-21T21:42:24.516842+0000","last_became_peered":"2024-09-21T21:42:24.516842+0000","last_unstale":"2024-09-21T21:42:37.573689+0000","last_undegraded":"2024-09-21T21:42:37.573689+0000","last_fullsized":"2024-09-21T21:42:37.573689+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":355,"log_dups_size":0,"ondisk_log_size":355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:42:46.269688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00137828,"stat_sum":{"num_bytes":58286111,"num_objects":55,"num_object_clones":14,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":16,"num_read":12968,"num_read_kb":55235,"num_write":399,"num_write_kb":11231,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":3599391,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2f","length":"2"}]},{"pgid":"3.11","version":"112'536","reported_seq":7009,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T21:42:34.306738+0000","last_change":"2024-09-21T21:42:15.420089+0000","last_active":"2024-09-21T21:42:34.306738+0000","last_peered":"2024-09-21T21:42:34.306738+0000","last_clean":"2024-09-21T21:42:34.306738+0000","last_became_active":"2024-09-21T21:42:15.418634+0000","last_became_peered":"2024-09-21T21:42:15.418634+0000","last_unstale":"2024-09-21T21:42:34.306738+0000","last_undegraded":"2024-09-21T21:42:34.306738+0000","last_fullsized":"2024-09-21T21:42:34.306738+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":536,"log_dups_size":0,"ondisk_log_size":536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:12:30.698126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087239000000000001,"stat_sum":{"num_bytes":42157674,"num_objects":63,"num_object_clones":12,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":17,"num_read":7101,"num_read_kb":29318,"num_write":653,"num_write_kb":6823,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":228,"num_bytes_recovered":3550319,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"112'643","reported_seq":18520,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T21:42:34.268424+0000","last_change":"2024-09-21T21:42:03.212866+0000","last_active":"2024-09-21T21:42:34.268424+0000","last_peered":"2024-09-21T21:42:34.268424+0000","last_clean":"2024-09-21T21:42:34.268424+0000","last_became_active":"2024-09-21T21:42:03.211865+0000","last_became_peered":"2024-09-21T21:42:03.211865+0000","last_unstale":"2024-09-21T21:42:34.268424+0000","last_undegraded":"2024-09-21T21:42:34.268424+0000","last_fullsized":"2024-09-21T21:42:34.268424+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":643,"log_dups_size":0,"ondisk_log_size":643,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:27:32.534905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046261000000000002,"stat_sum":{"num_bytes":64613411,"num_objects":71,"num_object_clones":19,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":23,"num_read":20027,"num_read_kb":71107,"num_write":703,"num_write_kb":9921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":211,"num_bytes_recovered":13613603,"num_keys_recovered":81,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"112'400","reported_seq":10132,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-21T21:42:34.286953+0000","last_change":"2024-09-21T21:41:52.210658+0000","last_active":"2024-09-21T21:42:34.286953+0000","last_peered":"2024-09-21T21:42:34.286953+0000","last_clean":"2024-09-21T21:42:34.286953+0000","last_became_active":"2024-09-21T21:41:52.209710+0000","last_became_peered":"2024-09-21T21:41:52.209710+0000","last_unstale":"2024-09-21T21:42:34.286953+0000","last_undegraded":"2024-09-21T21:42:34.286953+0000","last_fullsized":"2024-09-21T21:42:34.286953+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":400,"log_dups_size":0,"ondisk_log_size":400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:46:37.545769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070986300000000003,"stat_sum":{"num_bytes":61457459,"num_objects":58,"num_object_clones":13,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":17,"num_read":9546,"num_read_kb":76362,"num_write":458,"num_write_kb":12494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":146,"num_bytes_recovered":17387622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1338616232,"num_objects":1918,"num_object_clones":440,"num_object_copies":3836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1918,"num_whiteouts":561,"num_read":394727,"num_read_kb":1445701,"num_write":25300,"num_write_kb":316800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4200,"num_bytes_recovered":185186531,"num_keys_recovered":717,"num_objects_omap":47,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":76533760,"data_stored":617539405,"data_compressed":41280585,"data_compressed_allocated":44933120,"data_compressed_original":592072704,"omap_allocated":0,"internal_metadata":0},"log_size":20810,"ondisk_log_size":20810,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":1771104,"data_compressed":100455,"data_compressed_allocated":135168,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":4,"up_from":0,"seq":55834574878,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574878,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87224,"kb_used_data":15780,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94284616,"statfs":{"total":96636764160,"available":96547446784,"internally_reserved":0,"allocated":16158720,"data_stored":133030509,"data_compressed":9057911,"data_compressed_allocated":9306112,"data_compressed_original":127964672,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":12,"seq":51539607583,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83620,"kb_used_data":16332,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94288220,"statfs":{"total":96636764160,"available":96551137280,"internally_reserved":0,"allocated":16723968,"data_stored":126975018,"data_compressed":8497089,"data_compressed_allocated":9101312,"data_compressed_original":121383936,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607583,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66440,"kb_used_data":17184,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94305400,"statfs":{"total":96636764160,"available":96568729600,"internally_reserved":0,"allocated":17596416,"data_stored":120397284,"data_compressed":7953534,"data_compressed_allocated":9261056,"data_compressed_original":114359808,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":0,"seq":51539607584,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640287,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70536,"kb_used_data":12976,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94301304,"statfs":{"total":96636764160,"available":96564535296,"internally_reserved":0,"allocated":13287424,"data_stored":98773891,"data_compressed":6279335,"data_compressed_allocated":6922240,"data_compressed_original":94167040,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32395,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34030,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34030,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":7696384,"data_stored":53427314,"data_compressed":3500155,"data_compressed_allocated":3907584,"data_compressed_original":50207744,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16183296,"data_stored":119245567,"data_compressed":7919504,"data_compressed_allocated":9216000,"data_compressed_original":113769984,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":11952128,"data_stored":98223329,"data_compressed":6279335,"data_compressed_allocated":6922240,"data_compressed_original":94167040,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":14819328,"data_stored":132008031,"data_compressed":9028732,"data_compressed_allocated":9273344,"data_compressed_original":127477248,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":10522624,"data_stored":88221495,"data_compressed":6055770,"data_compressed_allocated":6512640,"data_compressed_original":85066752,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":15360000,"data_stored":126413669,"data_compressed":8497089,"data_compressed_allocated":9101312,"data_compressed_original":121383936,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T21:42:39.139 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:39.139 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:39.147 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T21:42:39.191 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:39.190+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:39.250 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:39.250 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:39.259 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T21:42:39.299 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:39.298+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:39.362 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:39.362 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:39.371 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T21:42:39.387 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:39.386+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:39.474 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:39.474 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:39.483 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T21:42:39.527 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:39.525+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:39.572 INFO:teuthology.orchestra.run.smithi081.stderr:nodeep-scrub is set 2024-09-21T21:42:39.589 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:39.588+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:39.592 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:39.592 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:39.603 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T21:42:39.690 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:39.688+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:39.748 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:39.748 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:39.757 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T21:42:39.791 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:39.790+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:39.892 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:39.891+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:39.899 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:39.899 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:39.908 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T21:42:39.993 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:39.992+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:40.038 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T21:42:40.053 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:40.053 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:40.062 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T21:42:40.094 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:40.093+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:40.175 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0: {} 2024-09-21T21:42:40.175 INFO:teuthology.orchestra.run.smithi081.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T21:42:40.192 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1: {} 2024-09-21T21:42:40.192 INFO:teuthology.orchestra.run.smithi081.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T21:42:40.195 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:40.193+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:40.208 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2: {} 2024-09-21T21:42:40.208 INFO:teuthology.orchestra.run.smithi081.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T21:42:40.220 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:40.220 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:42:40.220 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 0 2024-09-21T21:42:40.220 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:40.225 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3: {} 2024-09-21T21:42:40.225 INFO:teuthology.orchestra.run.smithi081.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T21:42:40.229 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T21:42:40.242 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4: {} 2024-09-21T21:42:40.242 INFO:teuthology.orchestra.run.smithi081.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T21:42:40.259 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5: {} 2024-09-21T21:42:40.259 INFO:teuthology.orchestra.run.smithi081.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T21:42:40.311 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:40.310+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:40.346 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:40.346 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:42:40.346 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:42:40.346 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:42:40.346 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:40.354 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T21:42:40.396 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:40.395+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:40.497 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:40.495+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:40.500 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:40.500 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 20, 2024-09-21T21:42:40.500 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 600, 2024-09-21T21:42:40.500 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:42:40.500 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.500 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13296 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 4046848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:40.500 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.796643+0000", 2024-09-21T21:42:40.500 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 49.701449019999998, 2024-09-21T21:42:40.501 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071682019999999999, 2024-09-21T21:42:40.501 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:40.501 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:40.501 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:40.501 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:40.501 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:40.501 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:40.501 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13296 2024-09-21T21:42:40.501 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.501 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:40.501 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.501 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:40.501 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796643+0000", 2024-09-21T21:42:40.501 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:40.502 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.502 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.502 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:40.502 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796643+0000", 2024-09-21T21:42:40.502 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:40.502 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.502 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.502 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:40.502 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796643+0000", 2024-09-21T21:42:40.502 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.7199999999999998e-07 2024-09-21T21:42:40.502 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.502 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.502 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:40.502 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796647+0000", 2024-09-21T21:42:40.503 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6770000000000001e-06 2024-09-21T21:42:40.503 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.503 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.503 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:40.503 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796656+0000", 2024-09-21T21:42:40.503 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2059999999999996e-06 2024-09-21T21:42:40.503 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.503 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.503 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:40.503 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796664+0000", 2024-09-21T21:42:40.503 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7919999999999993e-06 2024-09-21T21:42:40.503 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.503 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.503 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:40.504 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.868186+0000", 2024-09-21T21:42:40.504 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071522273999999997 2024-09-21T21:42:40.504 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.504 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.504 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:40.504 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.868239+0000", 2024-09-21T21:42:40.504 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2787999999999997e-05 2024-09-21T21:42:40.504 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.504 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.504 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:40.504 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.868325+0000", 2024-09-21T21:42:40.504 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.5611000000000004e-05 2024-09-21T21:42:40.504 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:40.504 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:40.504 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:40.505 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.505 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.505 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13561 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 1728512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:40.505 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.823991+0000", 2024-09-21T21:42:40.505 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 49.674100719000002, 2024-09-21T21:42:40.505 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071654762999999996, 2024-09-21T21:42:40.505 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:40.505 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:40.505 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:40.505 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:40.505 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:40.505 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:40.505 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13561 2024-09-21T21:42:40.505 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.506 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:40.506 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.506 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:40.506 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.823991+0000", 2024-09-21T21:42:40.506 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:40.506 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.506 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.506 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:40.506 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.823991+0000", 2024-09-21T21:42:40.506 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:40.506 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.506 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.506 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:40.506 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.823991+0000", 2024-09-21T21:42:40.507 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9299999999999998e-07 2024-09-21T21:42:40.507 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.507 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.507 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:40.507 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.823994+0000", 2024-09-21T21:42:40.507 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2369999999999999e-06 2024-09-21T21:42:40.507 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.507 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.507 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:40.507 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824000+0000", 2024-09-21T21:42:40.507 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.55e-06 2024-09-21T21:42:40.507 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.507 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.507 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:40.508 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824005+0000", 2024-09-21T21:42:40.508 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4329999999999997e-06 2024-09-21T21:42:40.508 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.508 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.508 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:40.508 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.895360+0000", 2024-09-21T21:42:40.508 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071355426 2024-09-21T21:42:40.508 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.508 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.508 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:40.508 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.895477+0000", 2024-09-21T21:42:40.508 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0001171 2024-09-21T21:42:40.508 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.508 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.509 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:40.509 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.895646+0000", 2024-09-21T21:42:40.509 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000168524 2024-09-21T21:42:40.509 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:40.509 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:40.509 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:40.509 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.509 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.509 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13562 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 1757184~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:40.509 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.824011+0000", 2024-09-21T21:42:40.509 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 49.674080842000002, 2024-09-21T21:42:40.509 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072020453999999998, 2024-09-21T21:42:40.509 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:40.509 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:40.510 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:40.510 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:40.510 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:40.510 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:40.510 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13562 2024-09-21T21:42:40.510 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.510 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:40.510 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.510 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:40.510 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824011+0000", 2024-09-21T21:42:40.510 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:40.510 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.510 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.511 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:40.511 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824011+0000", 2024-09-21T21:42:40.511 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:40.511 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.511 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.511 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:40.511 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824011+0000", 2024-09-21T21:42:40.511 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5600000000000001e-07 2024-09-21T21:42:40.511 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.511 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.511 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:40.511 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824013+0000", 2024-09-21T21:42:40.511 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9800000000000001e-06 2024-09-21T21:42:40.511 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.511 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.512 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:40.512 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824019+0000", 2024-09-21T21:42:40.512 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7339999999999996e-06 2024-09-21T21:42:40.512 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.512 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.512 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:40.512 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824023+0000", 2024-09-21T21:42:40.512 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3680000000000004e-06 2024-09-21T21:42:40.512 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.512 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.512 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:40.512 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.895699+0000", 2024-09-21T21:42:40.512 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071675597999999993 2024-09-21T21:42:40.512 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.512 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.513 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:40.513 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.895830+0000", 2024-09-21T21:42:40.513 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000130858 2024-09-21T21:42:40.513 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.513 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.513 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:40.513 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896031+0000", 2024-09-21T21:42:40.513 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00020155999999999999 2024-09-21T21:42:40.513 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:40.513 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:40.513 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:40.513 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.513 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.513 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13563 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 323584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:40.514 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.824029+0000", 2024-09-21T21:42:40.514 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 49.674062401999997, 2024-09-21T21:42:40.514 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072307563000000005, 2024-09-21T21:42:40.514 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:40.514 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:40.514 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:40.514 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:40.514 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:40.514 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:40.514 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13563 2024-09-21T21:42:40.514 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.514 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:40.514 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.514 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:40.515 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824029+0000", 2024-09-21T21:42:40.515 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:40.515 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.515 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.515 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:40.515 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824029+0000", 2024-09-21T21:42:40.515 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:40.515 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.515 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.515 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:40.515 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824030+0000", 2024-09-21T21:42:40.515 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1699999999999999e-07 2024-09-21T21:42:40.515 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.515 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.515 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:40.516 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824032+0000", 2024-09-21T21:42:40.516 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0320000000000002e-06 2024-09-21T21:42:40.516 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.516 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.516 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:40.516 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824038+0000", 2024-09-21T21:42:40.516 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8769999999999998e-06 2024-09-21T21:42:40.516 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.516 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.516 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:40.516 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824042+0000", 2024-09-21T21:42:40.516 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5650000000000003e-06 2024-09-21T21:42:40.516 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.516 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.516 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:40.517 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896058+0000", 2024-09-21T21:42:40.517 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072015941 2024-09-21T21:42:40.517 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.517 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.517 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:40.517 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896164+0000", 2024-09-21T21:42:40.517 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010593499999999999 2024-09-21T21:42:40.517 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.517 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.517 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:40.517 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896337+0000", 2024-09-21T21:42:40.517 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00017279599999999999 2024-09-21T21:42:40.517 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:40.517 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:40.517 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:40.517 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.518 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.518 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13564 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 983040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:40.518 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.824048+0000", 2024-09-21T21:42:40.518 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 49.674043327, 2024-09-21T21:42:40.518 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072491193999999995, 2024-09-21T21:42:40.518 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:40.518 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:40.518 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:40.518 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:40.518 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:40.518 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:40.518 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13564 2024-09-21T21:42:40.518 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.518 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:40.518 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.519 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:40.519 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824048+0000", 2024-09-21T21:42:40.519 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:40.519 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.519 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.519 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:40.519 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824048+0000", 2024-09-21T21:42:40.519 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:40.519 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.519 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.519 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:40.519 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824663+0000", 2024-09-21T21:42:40.519 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00061494299999999996 2024-09-21T21:42:40.519 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.520 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.520 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:40.520 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824669+0000", 2024-09-21T21:42:40.520 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2900000000000002e-06 2024-09-21T21:42:40.520 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.520 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.520 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:40.520 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824679+0000", 2024-09-21T21:42:40.520 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0338e-05 2024-09-21T21:42:40.520 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.520 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.520 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:40.520 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824687+0000", 2024-09-21T21:42:40.520 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.9049999999999997e-06 2024-09-21T21:42:40.520 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.521 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.521 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:40.521 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896381+0000", 2024-09-21T21:42:40.521 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071694305999999999 2024-09-21T21:42:40.521 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.521 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.521 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:40.521 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896445+0000", 2024-09-21T21:42:40.521 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3869999999999997e-05 2024-09-21T21:42:40.521 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.521 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.521 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:40.521 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896540+0000", 2024-09-21T21:42:40.521 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.4542000000000004e-05 2024-09-21T21:42:40.521 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:40.522 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:40.522 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:40.522 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.522 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.522 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13565 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3190784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:40.522 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.824704+0000", 2024-09-21T21:42:40.522 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 49.673387456, 2024-09-21T21:42:40.522 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072014745000000005, 2024-09-21T21:42:40.522 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:40.522 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:40.522 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:40.522 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:40.522 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:40.522 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:40.522 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13565 2024-09-21T21:42:40.523 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.523 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:40.523 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.523 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:40.523 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824704+0000", 2024-09-21T21:42:40.523 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:40.523 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.523 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.523 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:40.523 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824704+0000", 2024-09-21T21:42:40.523 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:40.523 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.523 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.523 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:40.524 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824705+0000", 2024-09-21T21:42:40.524 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8999999999999997e-07 2024-09-21T21:42:40.524 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.524 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.524 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:40.524 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824707+0000", 2024-09-21T21:42:40.524 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.7669999999999999e-06 2024-09-21T21:42:40.524 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.524 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.524 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:40.524 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824715+0000", 2024-09-21T21:42:40.524 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7700000000000001e-06 2024-09-21T21:42:40.524 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.524 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.524 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:40.524 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824721+0000", 2024-09-21T21:42:40.525 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9560000000000002e-06 2024-09-21T21:42:40.525 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.525 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.525 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:40.525 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896570+0000", 2024-09-21T21:42:40.525 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071848870999999995 2024-09-21T21:42:40.525 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.525 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.525 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:40.525 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896626+0000", 2024-09-21T21:42:40.525 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5578000000000002e-05 2024-09-21T21:42:40.525 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.525 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.525 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:40.525 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896719+0000", 2024-09-21T21:42:40.526 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.3313000000000006e-05 2024-09-21T21:42:40.526 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:40.526 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:40.526 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:40.526 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.526 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.526 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13566 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3162112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:40.526 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.824729+0000", 2024-09-21T21:42:40.526 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 49.673362568999998, 2024-09-21T21:42:40.526 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072356907999999998, 2024-09-21T21:42:40.526 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:40.526 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:40.526 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:40.526 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:40.526 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:40.527 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:40.527 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13566 2024-09-21T21:42:40.527 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.527 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:40.527 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.527 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:40.527 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824729+0000", 2024-09-21T21:42:40.527 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:40.527 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.527 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.527 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:40.527 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824729+0000", 2024-09-21T21:42:40.527 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:40.527 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.528 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.528 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:40.528 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825630+0000", 2024-09-21T21:42:40.528 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00090139400000000002 2024-09-21T21:42:40.529 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.529 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.529 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:40.530 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825634+0000", 2024-09-21T21:42:40.530 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4960000000000001e-06 2024-09-21T21:42:40.530 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.530 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.530 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:40.530 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825644+0000", 2024-09-21T21:42:40.530 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.9650000000000003e-06 2024-09-21T21:42:40.530 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.530 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.530 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:40.530 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825649+0000", 2024-09-21T21:42:40.530 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0359999999999997e-06 2024-09-21T21:42:40.530 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.530 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.531 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:40.531 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896757+0000", 2024-09-21T21:42:40.531 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071107717000000001 2024-09-21T21:42:40.531 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.531 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.531 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:40.531 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896878+0000", 2024-09-21T21:42:40.531 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000120834 2024-09-21T21:42:40.531 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.531 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.531 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:40.531 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897086+0000", 2024-09-21T21:42:40.531 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00020846600000000001 2024-09-21T21:42:40.531 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:40.531 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:40.532 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:40.532 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.532 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.532 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13568 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3907584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:40.532 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.825671+0000", 2024-09-21T21:42:40.532 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 49.672420160000001, 2024-09-21T21:42:40.532 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071794595000000003, 2024-09-21T21:42:40.532 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:40.532 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:40.532 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:40.532 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:40.532 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:40.532 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:40.532 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13568 2024-09-21T21:42:40.533 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.533 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:40.533 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.533 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:40.533 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825671+0000", 2024-09-21T21:42:40.533 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:40.533 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.533 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.533 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:40.533 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825671+0000", 2024-09-21T21:42:40.533 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:40.533 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.533 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.533 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:40.533 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825672+0000", 2024-09-21T21:42:40.533 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2999999999999999e-07 2024-09-21T21:42:40.534 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.534 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.534 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:40.534 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825673+0000", 2024-09-21T21:42:40.534 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.246e-06 2024-09-21T21:42:40.534 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.534 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.534 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:40.534 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825693+0000", 2024-09-21T21:42:40.534 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9789999999999999e-05 2024-09-21T21:42:40.534 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.534 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.534 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:40.534 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825697+0000", 2024-09-21T21:42:40.534 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2379999999999999e-06 2024-09-21T21:42:40.535 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.535 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.535 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:40.535 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897284+0000", 2024-09-21T21:42:40.535 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071586710999999997 2024-09-21T21:42:40.535 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.535 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.535 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:40.535 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897337+0000", 2024-09-21T21:42:40.535 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2800000000000003e-05 2024-09-21T21:42:40.535 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.535 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.535 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:40.535 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897466+0000", 2024-09-21T21:42:40.535 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00012957999999999999 2024-09-21T21:42:40.535 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:40.536 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:40.536 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:40.536 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.536 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.536 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13574 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 352256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:40.536 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.825702+0000", 2024-09-21T21:42:40.536 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 49.672390012000001, 2024-09-21T21:42:40.536 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072080377000000001, 2024-09-21T21:42:40.536 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:40.536 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:40.536 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:40.536 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:40.536 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:40.536 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:40.536 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13574 2024-09-21T21:42:40.537 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.537 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:40.537 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.537 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:40.537 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825702+0000", 2024-09-21T21:42:40.537 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:40.537 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.537 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.537 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:40.537 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825702+0000", 2024-09-21T21:42:40.537 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:40.537 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.537 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.537 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:40.537 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825943+0000", 2024-09-21T21:42:40.537 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00024105200000000001 2024-09-21T21:42:40.538 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.538 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.538 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:40.538 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825946+0000", 2024-09-21T21:42:40.538 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.743e-06 2024-09-21T21:42:40.538 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.538 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.538 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:40.538 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825956+0000", 2024-09-21T21:42:40.538 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6609999999999995e-06 2024-09-21T21:42:40.538 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.538 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.538 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:40.538 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825963+0000", 2024-09-21T21:42:40.539 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4609999999999996e-06 2024-09-21T21:42:40.539 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.539 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.539 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:40.539 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897502+0000", 2024-09-21T21:42:40.539 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071539354999999999 2024-09-21T21:42:40.539 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.539 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.539 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:40.539 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897619+0000", 2024-09-21T21:42:40.539 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011665 2024-09-21T21:42:40.539 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.539 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.539 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:40.540 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897782+0000", 2024-09-21T21:42:40.540 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00016345500000000001 2024-09-21T21:42:40.540 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:40.540 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:40.540 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:40.540 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.540 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.540 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13575 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 380928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:40.540 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.825973+0000", 2024-09-21T21:42:40.540 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 49.672118576000003, 2024-09-21T21:42:40.540 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072012723000000001, 2024-09-21T21:42:40.540 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:40.540 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:40.541 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:40.541 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:40.541 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:40.541 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:40.541 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13575 2024-09-21T21:42:40.541 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.541 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:40.541 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.541 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:40.541 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825973+0000", 2024-09-21T21:42:40.541 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:40.541 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.541 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.541 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:40.542 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825973+0000", 2024-09-21T21:42:40.542 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:40.542 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.542 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.542 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:40.542 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826746+0000", 2024-09-21T21:42:40.542 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00077326800000000002 2024-09-21T21:42:40.542 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.542 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.542 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:40.542 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826766+0000", 2024-09-21T21:42:40.542 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9465000000000001e-05 2024-09-21T21:42:40.542 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.543 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.543 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:40.543 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826812+0000", 2024-09-21T21:42:40.543 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5927999999999998e-05 2024-09-21T21:42:40.543 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.543 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.543 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:40.543 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826833+0000", 2024-09-21T21:42:40.543 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1710999999999999e-05 2024-09-21T21:42:40.543 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.543 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.543 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:40.543 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897809+0000", 2024-09-21T21:42:40.543 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.070975229000000001 2024-09-21T21:42:40.544 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.544 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.544 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:40.544 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897864+0000", 2024-09-21T21:42:40.544 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5334999999999998e-05 2024-09-21T21:42:40.544 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.544 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.544 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:40.544 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897986+0000", 2024-09-21T21:42:40.544 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000121787 2024-09-21T21:42:40.544 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:40.544 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:40.544 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:40.545 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.545 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.545 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13577 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 1814528~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:40.545 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.826874+0000", 2024-09-21T21:42:40.545 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 49.671217542000001, 2024-09-21T21:42:40.545 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071639123999999998, 2024-09-21T21:42:40.545 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:40.545 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:40.545 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:40.545 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:40.545 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:40.545 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:40.545 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13577 2024-09-21T21:42:40.545 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.546 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:40.546 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.546 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:40.546 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826874+0000", 2024-09-21T21:42:40.546 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:40.546 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.546 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.546 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:40.546 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826874+0000", 2024-09-21T21:42:40.546 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:40.546 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.546 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.546 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:40.546 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826875+0000", 2024-09-21T21:42:40.546 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.82e-07 2024-09-21T21:42:40.547 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.547 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.547 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:40.547 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826877+0000", 2024-09-21T21:42:40.547 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.295e-06 2024-09-21T21:42:40.547 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.547 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.547 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:40.547 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826883+0000", 2024-09-21T21:42:40.547 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2609999999999999e-06 2024-09-21T21:42:40.547 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.547 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.547 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:40.547 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826889+0000", 2024-09-21T21:42:40.547 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5589999999999996e-06 2024-09-21T21:42:40.548 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.548 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.548 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:40.548 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898301+0000", 2024-09-21T21:42:40.548 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071412188000000001 2024-09-21T21:42:40.548 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.548 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.548 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:40.548 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898416+0000", 2024-09-21T21:42:40.548 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000115538 2024-09-21T21:42:40.548 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.548 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.548 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:40.548 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898513+0000", 2024-09-21T21:42:40.549 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6800999999999997e-05 2024-09-21T21:42:40.549 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:40.549 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:40.549 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:40.549 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.549 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.549 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13578 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 2502656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:40.549 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.826896+0000", 2024-09-21T21:42:40.549 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 49.671196086000002, 2024-09-21T21:42:40.549 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071789411999999997, 2024-09-21T21:42:40.549 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:40.549 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:40.549 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:40.549 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:40.550 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:40.550 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:40.550 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13578 2024-09-21T21:42:40.550 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.550 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:40.550 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.550 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:40.550 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826896+0000", 2024-09-21T21:42:40.550 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:40.550 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.550 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.550 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:40.550 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826896+0000", 2024-09-21T21:42:40.550 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:40.551 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.551 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.551 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:40.551 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827327+0000", 2024-09-21T21:42:40.551 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00043094500000000002 2024-09-21T21:42:40.551 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.551 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.551 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:40.551 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827331+0000", 2024-09-21T21:42:40.551 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.0969999999999999e-06 2024-09-21T21:42:40.551 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.551 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.551 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:40.551 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827341+0000", 2024-09-21T21:42:40.552 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0199e-05 2024-09-21T21:42:40.552 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.552 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.552 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:40.552 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827346+0000", 2024-09-21T21:42:40.552 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3210000000000004e-06 2024-09-21T21:42:40.552 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.552 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.552 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:40.552 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898540+0000", 2024-09-21T21:42:40.552 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071194246000000003 2024-09-21T21:42:40.552 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.552 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.552 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:40.553 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898598+0000", 2024-09-21T21:42:40.553 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7658999999999998e-05 2024-09-21T21:42:40.553 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.553 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.553 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:40.553 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898685+0000", 2024-09-21T21:42:40.553 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.6945000000000002e-05 2024-09-21T21:42:40.553 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:40.553 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:40.553 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:40.553 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.553 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.553 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13579 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 2531328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:40.553 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.827355+0000", 2024-09-21T21:42:40.554 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 49.670736554999998, 2024-09-21T21:42:40.554 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071650933, 2024-09-21T21:42:40.554 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:40.554 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:40.554 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:40.554 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:40.554 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:40.554 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:40.554 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13579 2024-09-21T21:42:40.554 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.554 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:40.554 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.554 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:40.554 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827355+0000", 2024-09-21T21:42:40.555 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:40.555 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.555 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.555 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:40.555 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827355+0000", 2024-09-21T21:42:40.555 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:40.555 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.555 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.555 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:40.555 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827486+0000", 2024-09-21T21:42:40.555 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00013066799999999999 2024-09-21T21:42:40.555 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.555 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.555 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:40.556 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827490+0000", 2024-09-21T21:42:40.556 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.9009999999999996e-06 2024-09-21T21:42:40.556 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.556 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.556 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:40.556 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827499+0000", 2024-09-21T21:42:40.556 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6209999999999999e-06 2024-09-21T21:42:40.556 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.556 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.556 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:40.556 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827506+0000", 2024-09-21T21:42:40.556 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6490000000000002e-06 2024-09-21T21:42:40.556 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.556 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.556 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:40.557 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898713+0000", 2024-09-21T21:42:40.557 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071206937999999997 2024-09-21T21:42:40.557 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.557 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.557 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:40.557 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898811+0000", 2024-09-21T21:42:40.557 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.8159999999999995e-05 2024-09-21T21:42:40.557 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.557 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.557 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:40.557 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899006+0000", 2024-09-21T21:42:40.557 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00019499599999999999 2024-09-21T21:42:40.557 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:40.557 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:40.558 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:40.558 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.558 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.558 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13580 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3219456~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:40.558 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.827522+0000", 2024-09-21T21:42:40.558 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 49.670570124000001, 2024-09-21T21:42:40.558 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071813298999999997, 2024-09-21T21:42:40.558 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:40.558 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:40.558 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:40.558 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:40.558 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:40.558 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:40.558 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13580 2024-09-21T21:42:40.559 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.559 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:40.559 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.559 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:40.559 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827522+0000", 2024-09-21T21:42:40.559 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:40.559 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.559 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.559 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:40.559 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827522+0000", 2024-09-21T21:42:40.559 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:40.559 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.559 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.560 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:40.560 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827898+0000", 2024-09-21T21:42:40.560 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000376876 2024-09-21T21:42:40.560 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.560 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.560 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:40.560 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827903+0000", 2024-09-21T21:42:40.560 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4129999999999999e-06 2024-09-21T21:42:40.560 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.560 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.560 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:40.560 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827911+0000", 2024-09-21T21:42:40.560 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.4540000000000007e-06 2024-09-21T21:42:40.560 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.560 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.561 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:40.561 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827917+0000", 2024-09-21T21:42:40.561 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2110000000000003e-06 2024-09-21T21:42:40.561 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.561 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.561 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:40.561 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899041+0000", 2024-09-21T21:42:40.561 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071123965999999997 2024-09-21T21:42:40.561 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.561 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.561 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:40.561 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899157+0000", 2024-09-21T21:42:40.561 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011573700000000001 2024-09-21T21:42:40.561 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.562 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.562 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:40.562 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899335+0000", 2024-09-21T21:42:40.562 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00017764199999999999 2024-09-21T21:42:40.562 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:40.562 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:40.562 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:40.562 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.562 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.562 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13581 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3248128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:40.562 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.827929+0000", 2024-09-21T21:42:40.562 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 49.670162724999997, 2024-09-21T21:42:40.562 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071730151000000006, 2024-09-21T21:42:40.562 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:40.562 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:40.563 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:40.563 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:40.563 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:40.563 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:40.563 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13581 2024-09-21T21:42:40.563 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.563 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:40.563 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.563 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:40.563 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827929+0000", 2024-09-21T21:42:40.563 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:40.563 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.563 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.564 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:40.564 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827929+0000", 2024-09-21T21:42:40.564 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:40.564 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.564 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.564 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:40.564 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.829943+0000", 2024-09-21T21:42:40.564 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0020143750000000001 2024-09-21T21:42:40.564 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.564 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.564 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:40.564 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.829948+0000", 2024-09-21T21:42:40.564 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9400000000000001e-06 2024-09-21T21:42:40.564 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.564 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.565 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:40.565 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.829960+0000", 2024-09-21T21:42:40.565 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.2123999999999999e-05 2024-09-21T21:42:40.565 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.565 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.565 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:40.565 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.829968+0000", 2024-09-21T21:42:40.565 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7000000000000008e-06 2024-09-21T21:42:40.565 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.565 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.565 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:40.565 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899370+0000", 2024-09-21T21:42:40.565 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.069401672999999997 2024-09-21T21:42:40.565 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.565 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.566 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:40.566 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899481+0000", 2024-09-21T21:42:40.566 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011145300000000001 2024-09-21T21:42:40.566 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.566 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.566 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:40.566 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899659+0000", 2024-09-21T21:42:40.566 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00017788600000000001 2024-09-21T21:42:40.566 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:40.566 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:40.566 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:40.566 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.566 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.566 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13595 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 1871872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:40.567 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.830180+0000", 2024-09-21T21:42:40.567 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 49.667911611000001, 2024-09-21T21:42:40.567 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071809244999999994, 2024-09-21T21:42:40.567 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:40.567 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:40.567 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:40.567 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:40.567 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:40.567 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:40.567 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13595 2024-09-21T21:42:40.567 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.567 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:40.567 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.567 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:40.567 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.830180+0000", 2024-09-21T21:42:40.568 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:40.568 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.568 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.568 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:40.568 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.830180+0000", 2024-09-21T21:42:40.568 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:40.568 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.568 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.568 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:40.568 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.831967+0000", 2024-09-21T21:42:40.568 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0017868350000000001 2024-09-21T21:42:40.568 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.568 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.568 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:40.568 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.831971+0000", 2024-09-21T21:42:40.568 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3950000000000003e-06 2024-09-21T21:42:40.569 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.569 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.569 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:40.569 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.831983+0000", 2024-09-21T21:42:40.569 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1815e-05 2024-09-21T21:42:40.569 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.569 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.569 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:40.569 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.831993+0000", 2024-09-21T21:42:40.569 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6339999999999996e-06 2024-09-21T21:42:40.569 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.569 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.569 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:40.569 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.901693+0000", 2024-09-21T21:42:40.570 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.069700082999999996 2024-09-21T21:42:40.570 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.570 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.570 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:40.570 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.901813+0000", 2024-09-21T21:42:40.570 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011971 2024-09-21T21:42:40.570 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.570 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.570 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:40.570 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.901989+0000", 2024-09-21T21:42:40.570 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000176773 2024-09-21T21:42:40.570 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:40.570 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:40.570 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:40.570 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.571 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.571 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13601 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3334144~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:40.571 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.832130+0000", 2024-09-21T21:42:40.571 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 49.665961738999997, 2024-09-21T21:42:40.571 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071861727, 2024-09-21T21:42:40.571 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:40.571 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:40.571 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:40.571 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:40.571 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:40.571 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:40.571 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13601 2024-09-21T21:42:40.571 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.571 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:40.571 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.571 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:40.572 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.832130+0000", 2024-09-21T21:42:40.572 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:40.572 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.572 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.572 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:40.572 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.832130+0000", 2024-09-21T21:42:40.572 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:40.572 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.572 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.572 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:40.572 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.833939+0000", 2024-09-21T21:42:40.572 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0018094400000000001 2024-09-21T21:42:40.572 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.572 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.572 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:40.573 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.833944+0000", 2024-09-21T21:42:40.573 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5979999999999999e-06 2024-09-21T21:42:40.573 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.573 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.573 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:40.573 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.833953+0000", 2024-09-21T21:42:40.573 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.1460000000000002e-06 2024-09-21T21:42:40.573 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.573 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.573 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:40.573 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.833960+0000", 2024-09-21T21:42:40.573 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.7989999999999996e-06 2024-09-21T21:42:40.573 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.573 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.573 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:40.574 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.903693+0000", 2024-09-21T21:42:40.574 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.069732901 2024-09-21T21:42:40.574 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.574 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.574 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:40.574 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.903814+0000", 2024-09-21T21:42:40.574 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000121585 2024-09-21T21:42:40.574 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.574 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.574 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:40.574 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.903992+0000", 2024-09-21T21:42:40.574 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00017725799999999999 2024-09-21T21:42:40.574 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:40.574 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:40.575 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:40.575 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.575 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.575 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:12342 3.1 3:87417293:::rbd_data.1121faebdfe3.0000000000000004:head [read 4096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:42:40.575 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:14.411725+0000", 2024-09-21T21:42:40.575 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 26.086366673000001, 2024-09-21T21:42:40.575 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0089213029999999, 2024-09-21T21:42:40.575 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:40.575 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:40.575 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:40.575 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:40.575 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:40.575 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:40.575 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 12342 2024-09-21T21:42:40.576 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.576 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:40.576 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.576 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:40.576 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411725+0000", 2024-09-21T21:42:40.576 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:40.576 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.576 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.576 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:40.576 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411725+0000", 2024-09-21T21:42:40.576 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:40.576 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.576 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.576 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:40.577 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411726+0000", 2024-09-21T21:42:40.577 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.4169999999999999e-06 2024-09-21T21:42:40.577 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.577 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.577 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:40.577 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411734+0000", 2024-09-21T21:42:40.577 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.3799999999999996e-06 2024-09-21T21:42:40.577 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.577 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.577 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:40.577 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411750+0000", 2024-09-21T21:42:40.577 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.6429000000000001e-05 2024-09-21T21:42:40.577 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.577 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.578 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:40.578 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411760+0000", 2024-09-21T21:42:40.578 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.4239999999999999e-06 2024-09-21T21:42:40.578 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.578 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.578 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:40.578 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411828+0000", 2024-09-21T21:42:40.578 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.7935999999999995e-05 2024-09-21T21:42:40.578 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.578 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.578 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op must wait for map", 2024-09-21T21:42:40.578 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411837+0000", 2024-09-21T21:42:40.578 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.0100000000000001e-06 2024-09-21T21:42:40.578 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.579 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.579 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:40.579 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.414984+0000", 2024-09-21T21:42:40.579 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0031472050000000001 2024-09-21T21:42:40.579 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.579 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.579 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for peered", 2024-09-21T21:42:40.579 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.414994+0000", 2024-09-21T21:42:40.579 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0291e-05 2024-09-21T21:42:40.579 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.579 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.579 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:40.579 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.420353+0000", 2024-09-21T21:42:40.579 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.005358913 2024-09-21T21:42:40.580 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.580 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.580 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:40.580 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.420489+0000", 2024-09-21T21:42:40.580 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00013592699999999999 2024-09-21T21:42:40.580 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.580 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.580 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:40.580 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.420646+0000", 2024-09-21T21:42:40.580 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00015737100000000001 2024-09-21T21:42:40.580 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:40.580 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:40.580 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:40.580 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.580 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.581 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:12348 3.1 3:87417293:::rbd_data.1121faebdfe3.0000000000000004:head [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:42:40.581 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:14.411793+0000", 2024-09-21T21:42:40.581 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 26.086299127, 2024-09-21T21:42:40.581 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.009298384, 2024-09-21T21:42:40.581 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:40.581 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:40.581 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:40.581 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:40.581 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:40.581 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:40.581 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 12348 2024-09-21T21:42:40.581 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.581 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:40.581 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.582 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:40.582 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411793+0000", 2024-09-21T21:42:40.582 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:40.582 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.582 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.582 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:40.582 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411793+0000", 2024-09-21T21:42:40.582 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:40.582 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.582 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.582 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:40.582 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411793+0000", 2024-09-21T21:42:40.582 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0999999999999998e-07 2024-09-21T21:42:40.582 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.583 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.583 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:40.583 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411797+0000", 2024-09-21T21:42:40.583 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.49e-06 2024-09-21T21:42:40.583 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.583 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.583 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:40.583 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411809+0000", 2024-09-21T21:42:40.583 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.2510999999999999e-05 2024-09-21T21:42:40.583 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.583 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.583 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:40.583 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411816+0000", 2024-09-21T21:42:40.583 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6420000000000001e-06 2024-09-21T21:42:40.584 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.584 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.584 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:40.584 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411963+0000", 2024-09-21T21:42:40.584 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00014700999999999999 2024-09-21T21:42:40.584 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.584 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.584 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting_for_map not empty", 2024-09-21T21:42:40.584 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411970+0000", 2024-09-21T21:42:40.584 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9800000000000001e-06 2024-09-21T21:42:40.584 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.584 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.584 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:40.584 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.415018+0000", 2024-09-21T21:42:40.584 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0030482299999999999 2024-09-21T21:42:40.585 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.585 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.585 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for peered", 2024-09-21T21:42:40.585 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.415025+0000", 2024-09-21T21:42:40.585 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.3749999999999997e-06 2024-09-21T21:42:40.585 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.585 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.585 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:40.585 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.420904+0000", 2024-09-21T21:42:40.585 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0058787339999999 2024-09-21T21:42:40.585 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.585 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.585 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:40.586 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.420988+0000", 2024-09-21T21:42:40.586 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.3955999999999994e-05 2024-09-21T21:42:40.586 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.586 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.586 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:40.586 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.421091+0000", 2024-09-21T21:42:40.586 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000102846 2024-09-21T21:42:40.586 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:40.586 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:40.586 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:40.586 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.586 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.586 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4385.0:20922 3.9 3:920c5256:::rbd_id.image_client.0-clone36:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-21T21:42:40.586 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:17.811324+0000", 2024-09-21T21:42:40.586 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 22.686767395, 2024-09-21T21:42:40.587 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.66951882299999999, 2024-09-21T21:42:40.587 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:40.587 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:40.587 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:40.587 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:40.587 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4385", 2024-09-21T21:42:40.587 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:42:40.587 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 20922 2024-09-21T21:42:40.587 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.587 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:40.587 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.587 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:40.587 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:17.811324+0000", 2024-09-21T21:42:40.587 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:40.588 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.588 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.588 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:40.588 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:17.811324+0000", 2024-09-21T21:42:40.588 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:40.588 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.588 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.588 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:40.588 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:17.811326+0000", 2024-09-21T21:42:40.588 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.536e-06 2024-09-21T21:42:40.588 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.588 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.588 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:40.588 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:17.811332+0000", 2024-09-21T21:42:40.589 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6699999999999997e-06 2024-09-21T21:42:40.589 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.589 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.589 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:40.589 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:17.811347+0000", 2024-09-21T21:42:40.589 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.4589999999999999e-05 2024-09-21T21:42:40.589 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.589 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.589 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:40.589 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:17.811359+0000", 2024-09-21T21:42:40.589 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1776e-05 2024-09-21T21:42:40.589 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.589 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.589 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:40.590 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:17.811424+0000", 2024-09-21T21:42:40.590 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4795000000000006e-05 2024-09-21T21:42:40.590 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.590 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.590 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for peered", 2024-09-21T21:42:40.590 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:17.811436+0000", 2024-09-21T21:42:40.590 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.221e-05 2024-09-21T21:42:40.590 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.590 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.590 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:40.590 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:18.480657+0000", 2024-09-21T21:42:40.590 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.66922088999999996 2024-09-21T21:42:40.590 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.590 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.590 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:40.591 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:18.480760+0000", 2024-09-21T21:42:40.591 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000102827 2024-09-21T21:42:40.591 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:40.591 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:40.591 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:40.591 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:18.480843+0000", 2024-09-21T21:42:40.591 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.3529000000000006e-05 2024-09-21T21:42:40.591 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:40.591 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:40.591 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:40.591 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:40.591 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:40.591 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:40.593 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T21:42:40.636 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:40.635+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:40.697 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:40.696+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:40.708 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:40.709 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:42:40.709 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-21T21:42:40.709 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:40.718 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T21:42:40.802 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:40.801+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:40.861 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:40.862 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:42:40.862 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:42:40.862 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:42:40.862 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:40.870 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T21:42:40.911 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:40.910+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:41.000 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:40.998+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:41.014 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:41.014 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-21T21:42:41.014 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-21T21:42:41.014 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:42:41.014 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.014 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19747 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 7905280~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:41.014 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273058+0000", 2024-09-21T21:42:41.015 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 49.738856124000002, 2024-09-21T21:42:41.015 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97338591399999996, 2024-09-21T21:42:41.015 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:41.015 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:41.015 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:41.015 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:41.015 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:41.015 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:41.015 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19747 2024-09-21T21:42:41.015 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.015 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:41.016 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.016 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:41.016 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273058+0000", 2024-09-21T21:42:41.016 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:41.016 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.016 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.016 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:41.016 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273058+0000", 2024-09-21T21:42:41.016 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:41.016 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.016 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.016 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:41.017 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273058+0000", 2024-09-21T21:42:41.017 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5900000000000003e-07 2024-09-21T21:42:41.017 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.017 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.017 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:41.017 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273061+0000", 2024-09-21T21:42:41.017 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3479999999999998e-06 2024-09-21T21:42:41.017 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.017 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.017 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:41.017 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273067+0000", 2024-09-21T21:42:41.017 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1469999999999998e-06 2024-09-21T21:42:41.017 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.018 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.018 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:41.018 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273071+0000", 2024-09-21T21:42:41.018 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7339999999999999e-06 2024-09-21T21:42:41.018 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.018 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.018 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:41.018 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277945+0000", 2024-09-21T21:42:41.018 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.004873474 2024-09-21T21:42:41.018 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.018 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.019 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:41.019 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277948+0000", 2024-09-21T21:42:41.019 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0299999999999998e-06 2024-09-21T21:42:41.019 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.019 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.019 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:41.019 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246269+0000", 2024-09-21T21:42:41.019 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96832060200000003 2024-09-21T21:42:41.019 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.019 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.019 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:41.020 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246362+0000", 2024-09-21T21:42:41.020 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3867000000000001e-05 2024-09-21T21:42:41.020 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.020 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.020 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:41.020 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246444+0000", 2024-09-21T21:42:41.020 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1353000000000005e-05 2024-09-21T21:42:41.020 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:41.020 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:41.020 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:41.020 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.020 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.021 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19755 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 5783552~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:41.021 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273077+0000", 2024-09-21T21:42:41.021 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 49.738836507999999, 2024-09-21T21:42:41.021 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97350203400000002, 2024-09-21T21:42:41.021 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:41.021 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:41.021 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:41.021 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:41.021 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:41.021 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:41.021 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19755 2024-09-21T21:42:41.021 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.022 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:41.022 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.022 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:41.022 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273077+0000", 2024-09-21T21:42:41.022 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:41.022 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.022 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.022 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:41.022 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273077+0000", 2024-09-21T21:42:41.022 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:41.022 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.022 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.023 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:41.023 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273078+0000", 2024-09-21T21:42:41.023 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5699999999999998e-07 2024-09-21T21:42:41.023 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.023 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.023 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:41.023 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273083+0000", 2024-09-21T21:42:41.023 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8670000000000002e-06 2024-09-21T21:42:41.023 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.023 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.023 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:41.023 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273088+0000", 2024-09-21T21:42:41.023 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8370000000000002e-06 2024-09-21T21:42:41.024 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.024 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.024 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:41.024 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273093+0000", 2024-09-21T21:42:41.024 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5709999999999999e-06 2024-09-21T21:42:41.024 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.024 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.024 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:41.024 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277968+0000", 2024-09-21T21:42:41.024 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048753909999999998 2024-09-21T21:42:41.024 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.024 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.024 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:41.025 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277971+0000", 2024-09-21T21:42:41.025 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0189999999999998e-06 2024-09-21T21:42:41.025 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.025 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.025 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:41.025 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246485+0000", 2024-09-21T21:42:41.025 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96851399299999996 2024-09-21T21:42:41.025 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.025 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.025 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:41.025 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246536+0000", 2024-09-21T21:42:41.025 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0430000000000003e-05 2024-09-21T21:42:41.026 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.026 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.026 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:41.026 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246579+0000", 2024-09-21T21:42:41.026 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3569000000000002e-05 2024-09-21T21:42:41.026 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:41.026 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:41.026 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:41.026 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.026 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.026 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19756 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6500352~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:41.026 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273099+0000", 2024-09-21T21:42:41.026 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 49.738815135000003, 2024-09-21T21:42:41.027 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97359272799999996, 2024-09-21T21:42:41.027 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:41.027 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:41.027 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:41.027 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:41.027 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:41.027 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:41.027 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19756 2024-09-21T21:42:41.027 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.027 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:41.027 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.027 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:41.027 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273099+0000", 2024-09-21T21:42:41.028 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:41.028 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.028 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.028 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:41.028 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273099+0000", 2024-09-21T21:42:41.028 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:41.028 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.028 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.028 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:41.028 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273099+0000", 2024-09-21T21:42:41.028 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0900000000000002e-07 2024-09-21T21:42:41.028 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.029 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.029 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:41.029 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273101+0000", 2024-09-21T21:42:41.029 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0320000000000002e-06 2024-09-21T21:42:41.029 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.029 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.029 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:41.029 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273107+0000", 2024-09-21T21:42:41.029 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2830000000000001e-06 2024-09-21T21:42:41.029 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.029 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.029 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:41.029 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273111+0000", 2024-09-21T21:42:41.030 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2230000000000001e-06 2024-09-21T21:42:41.030 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.030 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.030 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:41.030 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277994+0000", 2024-09-21T21:42:41.030 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048837070000000002 2024-09-21T21:42:41.030 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.030 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.030 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:41.030 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278000+0000", 2024-09-21T21:42:41.030 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3149999999999999e-06 2024-09-21T21:42:41.031 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.031 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.031 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:41.031 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246607+0000", 2024-09-21T21:42:41.031 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.968606839 2024-09-21T21:42:41.031 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.031 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.031 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:41.031 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246651+0000", 2024-09-21T21:42:41.031 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4734999999999998e-05 2024-09-21T21:42:41.031 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.031 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.032 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:41.032 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246692+0000", 2024-09-21T21:42:41.032 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0185000000000002e-05 2024-09-21T21:42:41.032 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:41.032 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:41.032 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:41.032 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.032 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.032 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19757 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 5812224~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:41.032 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273116+0000", 2024-09-21T21:42:41.032 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 49.738797478000002, 2024-09-21T21:42:41.032 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97367813999999997, 2024-09-21T21:42:41.032 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:41.033 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:41.033 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:41.033 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:41.033 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:41.033 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:41.033 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19757 2024-09-21T21:42:41.033 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.033 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:41.033 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.033 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:41.033 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273116+0000", 2024-09-21T21:42:41.033 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:41.034 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.034 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.034 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:41.034 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273116+0000", 2024-09-21T21:42:41.034 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:41.034 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.034 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.034 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:41.034 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273117+0000", 2024-09-21T21:42:41.034 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0999999999999999e-07 2024-09-21T21:42:41.034 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.035 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.035 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:41.035 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273119+0000", 2024-09-21T21:42:41.035 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9970000000000001e-06 2024-09-21T21:42:41.035 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.035 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.035 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:41.035 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273125+0000", 2024-09-21T21:42:41.035 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9540000000000001e-06 2024-09-21T21:42:41.035 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.035 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.035 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:41.036 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273135+0000", 2024-09-21T21:42:41.036 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0597999999999999e-05 2024-09-21T21:42:41.036 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.036 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.036 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:41.036 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278019+0000", 2024-09-21T21:42:41.036 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048834849999999999 2024-09-21T21:42:41.036 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.036 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.036 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:41.036 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278022+0000", 2024-09-21T21:42:41.036 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9560000000000002e-06 2024-09-21T21:42:41.036 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.037 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.037 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:41.037 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246716+0000", 2024-09-21T21:42:41.037 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.968694202 2024-09-21T21:42:41.037 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.037 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.037 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:41.037 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246758+0000", 2024-09-21T21:42:41.037 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1607000000000003e-05 2024-09-21T21:42:41.037 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.037 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.037 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:41.038 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246795+0000", 2024-09-21T21:42:41.038 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6931000000000003e-05 2024-09-21T21:42:41.038 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:41.038 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:41.038 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:41.038 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.038 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.038 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19758 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6529024~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:41.038 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273142+0000", 2024-09-21T21:42:41.038 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 49.738772257000001, 2024-09-21T21:42:41.038 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97375924599999997, 2024-09-21T21:42:41.038 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:41.039 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:41.039 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:41.039 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:41.039 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:41.039 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:41.039 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19758 2024-09-21T21:42:41.039 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.039 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:41.039 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.039 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:41.039 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273142+0000", 2024-09-21T21:42:41.039 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:41.039 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.039 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.040 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:41.040 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273142+0000", 2024-09-21T21:42:41.040 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:41.040 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.040 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.040 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:41.040 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273142+0000", 2024-09-21T21:42:41.040 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0200000000000003e-07 2024-09-21T21:42:41.040 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.040 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.040 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:41.040 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273144+0000", 2024-09-21T21:42:41.040 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1390000000000002e-06 2024-09-21T21:42:41.041 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.041 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.041 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:41.041 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273150+0000", 2024-09-21T21:42:41.041 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6409999999999999e-06 2024-09-21T21:42:41.041 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.041 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.041 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:41.041 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273157+0000", 2024-09-21T21:42:41.041 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1359999999999999e-06 2024-09-21T21:42:41.041 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.041 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.042 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:41.042 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278050+0000", 2024-09-21T21:42:41.042 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048926170000000001 2024-09-21T21:42:41.042 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.042 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.042 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:41.042 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278053+0000", 2024-09-21T21:42:41.042 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4879999999999999e-06 2024-09-21T21:42:41.042 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.042 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.042 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:41.042 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246819+0000", 2024-09-21T21:42:41.043 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96876590399999996 2024-09-21T21:42:41.043 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.043 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.043 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:41.043 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246856+0000", 2024-09-21T21:42:41.043 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7287999999999999e-05 2024-09-21T21:42:41.043 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.043 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.043 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:41.043 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246901+0000", 2024-09-21T21:42:41.043 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4631e-05 2024-09-21T21:42:41.043 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:41.043 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:41.044 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:41.044 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.044 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.044 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19759 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 7933952~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:41.044 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273172+0000", 2024-09-21T21:42:41.044 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 49.738741992999998, 2024-09-21T21:42:41.044 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97384789500000002, 2024-09-21T21:42:41.044 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:41.044 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:41.044 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:41.044 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:41.044 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:41.044 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:41.044 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19759 2024-09-21T21:42:41.045 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.045 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:41.045 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.045 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:41.045 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273172+0000", 2024-09-21T21:42:41.045 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:41.045 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.045 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.045 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:41.045 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273172+0000", 2024-09-21T21:42:41.045 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:41.045 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.046 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.046 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:41.046 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273172+0000", 2024-09-21T21:42:41.046 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6100000000000002e-07 2024-09-21T21:42:41.046 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.046 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.046 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:41.046 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273174+0000", 2024-09-21T21:42:41.046 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.691e-06 2024-09-21T21:42:41.046 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.046 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.046 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:41.047 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273178+0000", 2024-09-21T21:42:41.047 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7579999999999998e-06 2024-09-21T21:42:41.047 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.047 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.047 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:41.047 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273180+0000", 2024-09-21T21:42:41.047 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7199999999999998e-06 2024-09-21T21:42:41.047 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.047 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.047 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:41.047 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278077+0000", 2024-09-21T21:42:41.047 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048969529999999999 2024-09-21T21:42:41.048 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.048 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.048 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:41.048 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278080+0000", 2024-09-21T21:42:41.048 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.066e-06 2024-09-21T21:42:41.048 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.048 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.048 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:41.048 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246925+0000", 2024-09-21T21:42:41.048 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96884490199999995 2024-09-21T21:42:41.048 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.048 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.049 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:41.049 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246974+0000", 2024-09-21T21:42:41.049 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8387999999999998e-05 2024-09-21T21:42:41.049 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.049 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.049 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:41.049 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247020+0000", 2024-09-21T21:42:41.049 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6156000000000003e-05 2024-09-21T21:42:41.049 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:41.049 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:41.049 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:41.049 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.050 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.050 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19760 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 7991296~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:41.050 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273220+0000", 2024-09-21T21:42:41.050 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 49.738693478999998, 2024-09-21T21:42:41.050 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97403125499999998, 2024-09-21T21:42:41.050 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:41.050 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:41.050 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:41.050 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:41.050 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:41.050 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:41.050 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19760 2024-09-21T21:42:41.050 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.051 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:41.051 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.051 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:41.051 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273220+0000", 2024-09-21T21:42:41.051 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:41.051 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.051 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.051 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:41.051 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273220+0000", 2024-09-21T21:42:41.051 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:41.051 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.051 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.052 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:41.052 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273221+0000", 2024-09-21T21:42:41.052 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.65e-07 2024-09-21T21:42:41.052 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.052 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.052 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:41.052 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273228+0000", 2024-09-21T21:42:41.052 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6040000000000004e-06 2024-09-21T21:42:41.052 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.052 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.052 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:41.052 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273233+0000", 2024-09-21T21:42:41.053 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1569999999999999e-06 2024-09-21T21:42:41.053 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.053 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.053 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:41.053 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273238+0000", 2024-09-21T21:42:41.053 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0050000000000004e-06 2024-09-21T21:42:41.053 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.053 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.053 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:41.053 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278107+0000", 2024-09-21T21:42:41.053 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00486857 2024-09-21T21:42:41.053 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.053 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.054 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:41.054 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278111+0000", 2024-09-21T21:42:41.054 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6389999999999998e-06 2024-09-21T21:42:41.054 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.054 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.054 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:41.054 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247069+0000", 2024-09-21T21:42:41.054 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96895796899999997 2024-09-21T21:42:41.054 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.054 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.054 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:41.055 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247166+0000", 2024-09-21T21:42:41.055 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.7158999999999996e-05 2024-09-21T21:42:41.055 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.055 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.055 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:41.055 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247252+0000", 2024-09-21T21:42:41.055 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5886999999999999e-05 2024-09-21T21:42:41.055 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:41.055 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:41.055 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:41.055 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.055 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.055 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19761 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 7962624~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:41.056 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273246+0000", 2024-09-21T21:42:41.056 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 49.738668419, 2024-09-21T21:42:41.056 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97415972200000001, 2024-09-21T21:42:41.056 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:41.056 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:41.056 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:41.056 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:41.056 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:41.056 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:41.056 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19761 2024-09-21T21:42:41.056 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.056 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:41.056 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.057 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:41.057 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273246+0000", 2024-09-21T21:42:41.057 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:41.057 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.057 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.057 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:41.057 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273246+0000", 2024-09-21T21:42:41.057 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:41.057 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.057 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.057 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:41.057 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273246+0000", 2024-09-21T21:42:41.058 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5100000000000001e-07 2024-09-21T21:42:41.058 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.058 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.058 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:41.058 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273248+0000", 2024-09-21T21:42:41.058 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1550000000000001e-06 2024-09-21T21:42:41.058 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.058 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.058 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:41.058 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273254+0000", 2024-09-21T21:42:41.058 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.964e-06 2024-09-21T21:42:41.058 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.059 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.059 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:41.059 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273259+0000", 2024-09-21T21:42:41.059 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5499999999999996e-06 2024-09-21T21:42:41.059 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.059 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.059 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:41.059 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278130+0000", 2024-09-21T21:42:41.059 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048715779999999997 2024-09-21T21:42:41.059 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.059 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.059 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:41.060 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278133+0000", 2024-09-21T21:42:41.060 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8739999999999999e-06 2024-09-21T21:42:41.060 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.060 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.060 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:41.060 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247270+0000", 2024-09-21T21:42:41.060 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.969137099 2024-09-21T21:42:41.060 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.060 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.060 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:41.060 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247338+0000", 2024-09-21T21:42:41.060 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7717000000000002e-05 2024-09-21T21:42:41.061 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.061 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.061 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:41.061 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247405+0000", 2024-09-21T21:42:41.061 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7434000000000005e-05 2024-09-21T21:42:41.061 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:41.061 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:41.061 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:41.061 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.061 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.061 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19769 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 5840896~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:41.061 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273287+0000", 2024-09-21T21:42:41.062 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 49.738627135000002, 2024-09-21T21:42:41.062 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97435390200000005, 2024-09-21T21:42:41.062 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:41.062 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:41.062 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:41.062 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:41.062 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:41.062 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:41.062 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19769 2024-09-21T21:42:41.062 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.062 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:41.062 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.063 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:41.063 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273287+0000", 2024-09-21T21:42:41.063 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:41.063 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.063 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.063 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:41.063 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273287+0000", 2024-09-21T21:42:41.063 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:41.063 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.063 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.063 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:41.064 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273287+0000", 2024-09-21T21:42:41.064 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6699999999999999e-07 2024-09-21T21:42:41.064 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.064 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.064 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:41.064 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273289+0000", 2024-09-21T21:42:41.064 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.708e-06 2024-09-21T21:42:41.064 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.064 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.064 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:41.064 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273296+0000", 2024-09-21T21:42:41.064 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8109999999999997e-06 2024-09-21T21:42:41.064 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.065 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.065 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:41.065 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273299+0000", 2024-09-21T21:42:41.065 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0740000000000001e-06 2024-09-21T21:42:41.065 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.065 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.065 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:41.065 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278159+0000", 2024-09-21T21:42:41.065 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048603240000000001 2024-09-21T21:42:41.065 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.065 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.065 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:41.065 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278162+0000", 2024-09-21T21:42:41.066 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1930000000000002e-06 2024-09-21T21:42:41.066 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.066 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.066 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:41.066 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247444+0000", 2024-09-21T21:42:41.066 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96928199100000001 2024-09-21T21:42:41.066 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.066 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.066 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:41.066 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247533+0000", 2024-09-21T21:42:41.066 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8490000000000001e-05 2024-09-21T21:42:41.066 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.067 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.067 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:41.067 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247641+0000", 2024-09-21T21:42:41.067 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000107944 2024-09-21T21:42:41.067 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:41.067 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:41.067 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:41.067 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.067 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.067 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19770 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 5869568~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:41.067 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273313+0000", 2024-09-21T21:42:41.067 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 49.738601033000002, 2024-09-21T21:42:41.067 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97449038899999996, 2024-09-21T21:42:41.068 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:41.068 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:41.068 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:41.068 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:41.068 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:41.068 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:41.068 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19770 2024-09-21T21:42:41.068 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.068 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:41.068 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.068 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:41.068 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273313+0000", 2024-09-21T21:42:41.068 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:41.069 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.069 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.069 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:41.069 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273313+0000", 2024-09-21T21:42:41.069 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:41.069 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.069 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.069 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:41.069 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273313+0000", 2024-09-21T21:42:41.069 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5699999999999998e-07 2024-09-21T21:42:41.069 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.069 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.070 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:41.070 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273315+0000", 2024-09-21T21:42:41.070 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0099999999999998e-06 2024-09-21T21:42:41.070 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.070 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.070 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:41.070 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273321+0000", 2024-09-21T21:42:41.070 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7529999999999998e-06 2024-09-21T21:42:41.070 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.070 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.070 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:41.071 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273328+0000", 2024-09-21T21:42:41.071 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9009999999999997e-06 2024-09-21T21:42:41.071 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.071 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.071 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:41.071 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278183+0000", 2024-09-21T21:42:41.071 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048547770000000002 2024-09-21T21:42:41.071 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.071 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.071 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:41.071 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278186+0000", 2024-09-21T21:42:41.072 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3100000000000001e-06 2024-09-21T21:42:41.072 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.072 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.072 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:41.072 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247676+0000", 2024-09-21T21:42:41.072 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96949004299999997 2024-09-21T21:42:41.072 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.072 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.072 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:41.072 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247757+0000", 2024-09-21T21:42:41.072 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0479e-05 2024-09-21T21:42:41.072 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.073 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.073 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:41.073 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247803+0000", 2024-09-21T21:42:41.073 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6758999999999997e-05 2024-09-21T21:42:41.073 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:41.073 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:41.073 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:41.073 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.073 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.073 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19771 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6586368~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:41.073 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273335+0000", 2024-09-21T21:42:41.073 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 49.738579246, 2024-09-21T21:42:41.074 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97458947699999998, 2024-09-21T21:42:41.074 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:41.074 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:41.074 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:41.074 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:41.074 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:41.074 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:41.074 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19771 2024-09-21T21:42:41.074 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.074 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:41.074 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.075 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:41.075 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273335+0000", 2024-09-21T21:42:41.075 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:41.075 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.075 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.075 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:41.075 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273335+0000", 2024-09-21T21:42:41.075 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:41.075 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.075 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.075 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:41.075 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273335+0000", 2024-09-21T21:42:41.076 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3599999999999999e-07 2024-09-21T21:42:41.076 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.076 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.076 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:41.076 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273337+0000", 2024-09-21T21:42:41.076 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0789999999999999e-06 2024-09-21T21:42:41.076 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.076 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.076 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:41.076 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273343+0000", 2024-09-21T21:42:41.076 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6280000000000002e-06 2024-09-21T21:42:41.076 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.077 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.077 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:41.077 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273346+0000", 2024-09-21T21:42:41.077 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.507e-06 2024-09-21T21:42:41.077 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.077 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.077 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:41.077 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278206+0000", 2024-09-21T21:42:41.077 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048600969999999999 2024-09-21T21:42:41.077 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.077 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.077 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:41.077 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278212+0000", 2024-09-21T21:42:41.078 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2079999999999999e-06 2024-09-21T21:42:41.078 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.078 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.078 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:41.078 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247828+0000", 2024-09-21T21:42:41.078 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.969616547 2024-09-21T21:42:41.078 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.078 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.078 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:41.078 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247881+0000", 2024-09-21T21:42:41.078 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3180999999999998e-05 2024-09-21T21:42:41.079 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.079 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.079 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:41.079 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247924+0000", 2024-09-21T21:42:41.079 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2894e-05 2024-09-21T21:42:41.079 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:41.079 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:41.079 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:41.079 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.079 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.079 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19772 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6557696~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:41.079 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273358+0000", 2024-09-21T21:42:41.080 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 49.738555865999999, 2024-09-21T21:42:41.080 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97467031800000004, 2024-09-21T21:42:41.080 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:41.080 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:41.080 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:41.080 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:41.080 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:41.080 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:41.080 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19772 2024-09-21T21:42:41.080 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.080 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:41.080 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.081 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:41.081 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273358+0000", 2024-09-21T21:42:41.081 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:41.081 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.081 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.081 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:41.081 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273358+0000", 2024-09-21T21:42:41.081 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:41.081 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.081 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.081 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:41.081 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273358+0000", 2024-09-21T21:42:41.082 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4499999999999998e-07 2024-09-21T21:42:41.082 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.082 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.082 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:41.082 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273360+0000", 2024-09-21T21:42:41.082 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9190000000000002e-06 2024-09-21T21:42:41.082 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.082 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.082 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:41.082 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273366+0000", 2024-09-21T21:42:41.082 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.891e-06 2024-09-21T21:42:41.082 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.083 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.083 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:41.083 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273371+0000", 2024-09-21T21:42:41.083 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1880000000000001e-06 2024-09-21T21:42:41.083 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.083 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.083 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:41.083 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278237+0000", 2024-09-21T21:42:41.083 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.004865472 2024-09-21T21:42:41.083 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.083 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.083 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:41.083 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278240+0000", 2024-09-21T21:42:41.084 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9550000000000001e-06 2024-09-21T21:42:41.084 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.084 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.084 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:41.084 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247949+0000", 2024-09-21T21:42:41.084 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.969709127 2024-09-21T21:42:41.084 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.084 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.084 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:41.084 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247990+0000", 2024-09-21T21:42:41.084 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1202000000000003e-05 2024-09-21T21:42:41.084 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.085 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.085 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:41.085 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248028+0000", 2024-09-21T21:42:41.085 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8219000000000001e-05 2024-09-21T21:42:41.085 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:41.085 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:41.085 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:41.085 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.085 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.085 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19773 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6615040~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:41.085 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273390+0000", 2024-09-21T21:42:41.085 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 49.738524005000002, 2024-09-21T21:42:41.085 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97476086200000001, 2024-09-21T21:42:41.086 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:41.086 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:41.086 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:41.086 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:41.086 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:41.086 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:41.086 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19773 2024-09-21T21:42:41.086 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.086 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:41.086 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.086 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:41.086 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273390+0000", 2024-09-21T21:42:41.087 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:41.087 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.087 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.087 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:41.087 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273390+0000", 2024-09-21T21:42:41.087 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:41.087 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.087 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.087 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:41.087 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273390+0000", 2024-09-21T21:42:41.087 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9999999999999999e-07 2024-09-21T21:42:41.087 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.088 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.088 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:41.088 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273392+0000", 2024-09-21T21:42:41.088 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0640000000000001e-06 2024-09-21T21:42:41.088 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.088 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.088 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:41.088 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273397+0000", 2024-09-21T21:42:41.088 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.113e-06 2024-09-21T21:42:41.088 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.088 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.088 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:41.089 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273401+0000", 2024-09-21T21:42:41.089 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3119999999999998e-06 2024-09-21T21:42:41.089 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.089 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.089 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:41.089 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278271+0000", 2024-09-21T21:42:41.089 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048698880000000002 2024-09-21T21:42:41.089 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.089 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.089 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:41.089 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278278+0000", 2024-09-21T21:42:41.089 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1210000000000001e-06 2024-09-21T21:42:41.090 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.090 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.090 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:41.090 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248060+0000", 2024-09-21T21:42:41.090 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96978224999999996 2024-09-21T21:42:41.090 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.090 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.090 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:41.090 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248111+0000", 2024-09-21T21:42:41.090 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.096e-05 2024-09-21T21:42:41.090 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.091 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.091 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:41.091 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248151+0000", 2024-09-21T21:42:41.091 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9854000000000002e-05 2024-09-21T21:42:41.091 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:41.091 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:41.091 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:41.091 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.091 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.091 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19784 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6643712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:41.091 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273405+0000", 2024-09-21T21:42:41.091 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 49.738509284000003, 2024-09-21T21:42:41.092 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97491798900000004, 2024-09-21T21:42:41.092 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:41.092 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:41.092 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:41.092 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:41.092 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:41.092 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:41.092 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19784 2024-09-21T21:42:41.092 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.092 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:41.092 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.092 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:41.093 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273405+0000", 2024-09-21T21:42:41.093 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:41.093 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.093 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.093 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:41.093 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273405+0000", 2024-09-21T21:42:41.093 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:41.093 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.093 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.093 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:41.093 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273405+0000", 2024-09-21T21:42:41.094 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2100000000000001e-07 2024-09-21T21:42:41.094 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.094 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.094 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:41.094 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273406+0000", 2024-09-21T21:42:41.094 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2049999999999999e-06 2024-09-21T21:42:41.094 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.094 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.094 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:41.094 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273409+0000", 2024-09-21T21:42:41.094 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0599999999999999e-06 2024-09-21T21:42:41.094 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.095 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.095 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:41.095 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273412+0000", 2024-09-21T21:42:41.095 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0730000000000001e-06 2024-09-21T21:42:41.095 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.095 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.095 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:41.095 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278317+0000", 2024-09-21T21:42:41.095 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0049045520000000004 2024-09-21T21:42:41.095 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.095 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.095 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:41.096 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278321+0000", 2024-09-21T21:42:41.096 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6249999999999998e-06 2024-09-21T21:42:41.096 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.096 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.096 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:41.096 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248179+0000", 2024-09-21T21:42:41.096 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96985744500000004 2024-09-21T21:42:41.096 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.096 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.096 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:41.096 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248253+0000", 2024-09-21T21:42:41.096 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3829000000000001e-05 2024-09-21T21:42:41.097 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.097 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.097 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:41.097 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248323+0000", 2024-09-21T21:42:41.097 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9979000000000002e-05 2024-09-21T21:42:41.097 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:41.097 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:41.097 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:41.097 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.097 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.097 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19785 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6672384~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:41.097 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273415+0000", 2024-09-21T21:42:41.098 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 49.738498698000001, 2024-09-21T21:42:41.098 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97500777800000005, 2024-09-21T21:42:41.098 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:41.098 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:41.098 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:41.098 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:41.098 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:41.098 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:41.098 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19785 2024-09-21T21:42:41.098 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.098 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:41.098 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.098 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:41.099 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273415+0000", 2024-09-21T21:42:41.099 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:41.099 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.099 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.099 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:41.099 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273415+0000", 2024-09-21T21:42:41.099 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:41.099 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.099 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.099 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:41.099 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273415+0000", 2024-09-21T21:42:41.099 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0100000000000001e-07 2024-09-21T21:42:41.100 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.100 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.100 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:41.100 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273417+0000", 2024-09-21T21:42:41.100 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.114e-06 2024-09-21T21:42:41.100 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.100 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.100 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:41.100 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273420+0000", 2024-09-21T21:42:41.100 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6770000000000001e-06 2024-09-21T21:42:41.100 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.100 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.100 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:41.101 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273423+0000", 2024-09-21T21:42:41.101 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5450000000000002e-06 2024-09-21T21:42:41.101 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.101 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.101 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:41.101 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278355+0000", 2024-09-21T21:42:41.101 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0049318590000000002 2024-09-21T21:42:41.101 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.101 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.101 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:41.101 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278360+0000", 2024-09-21T21:42:41.101 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3079999999999998e-06 2024-09-21T21:42:41.101 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.102 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.102 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:41.102 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248349+0000", 2024-09-21T21:42:41.102 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96998909899999997 2024-09-21T21:42:41.102 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.102 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.102 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:41.102 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248387+0000", 2024-09-21T21:42:41.102 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7657000000000002e-05 2024-09-21T21:42:41.102 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.102 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.102 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:41.103 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248423+0000", 2024-09-21T21:42:41.103 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6318e-05 2024-09-21T21:42:41.103 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:41.103 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:41.103 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:41.103 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.103 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.103 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19842 3.d 3:b7fa2ca0:::rbd_data.112157a479c.000000000000004d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:41.103 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273426+0000", 2024-09-21T21:42:41.103 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 49.738487646000003, 2024-09-21T21:42:41.103 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.98136526800000001, 2024-09-21T21:42:41.103 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:41.104 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:41.104 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:41.104 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:41.104 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:41.104 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:41.104 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19842 2024-09-21T21:42:41.104 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.104 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:41.104 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.104 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:41.104 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273426+0000", 2024-09-21T21:42:41.104 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:41.105 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.105 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.105 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:41.105 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273426+0000", 2024-09-21T21:42:41.105 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:41.105 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.105 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.105 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:41.105 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273427+0000", 2024-09-21T21:42:41.105 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.72e-07 2024-09-21T21:42:41.105 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.105 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.105 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:41.106 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275836+0000", 2024-09-21T21:42:41.106 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00240909 2024-09-21T21:42:41.106 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.106 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.106 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:41.106 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275844+0000", 2024-09-21T21:42:41.106 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3750000000000003e-06 2024-09-21T21:42:41.106 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.106 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.106 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:41.106 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275849+0000", 2024-09-21T21:42:41.106 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0100000000000003e-06 2024-09-21T21:42:41.107 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.107 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.107 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:41.107 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278394+0000", 2024-09-21T21:42:41.107 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025449890000000001 2024-09-21T21:42:41.107 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.107 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.107 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:41.107 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278402+0000", 2024-09-21T21:42:41.107 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6359999999999994e-06 2024-09-21T21:42:41.107 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.107 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.108 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:41.108 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248454+0000", 2024-09-21T21:42:41.108 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97005259499999996 2024-09-21T21:42:41.108 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.108 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.108 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:41.108 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248576+0000", 2024-09-21T21:42:41.108 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000122127 2024-09-21T21:42:41.108 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.108 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.108 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 2", 2024-09-21T21:42:41.108 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248844+0000", 2024-09-21T21:42:41.109 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00026784699999999999 2024-09-21T21:42:41.109 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.109 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.109 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:41.109 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.252487+0000", 2024-09-21T21:42:41.109 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0036427920000000002 2024-09-21T21:42:41.109 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.109 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.109 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:41.109 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.254691+0000", 2024-09-21T21:42:41.109 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002203899 2024-09-21T21:42:41.109 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.110 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.110 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:41.110 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.254748+0000", 2024-09-21T21:42:41.110 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7163000000000001e-05 2024-09-21T21:42:41.110 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.110 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.110 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:41.110 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.254792+0000", 2024-09-21T21:42:41.110 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3473000000000002e-05 2024-09-21T21:42:41.110 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:41.110 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:41.110 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:41.111 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.111 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.111 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19843 3.d 3:b7fa2ca0:::rbd_data.112157a479c.000000000000004d:head [write 0~0] snapc 21=[21] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:41.111 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.275860+0000", 2024-09-21T21:42:41.111 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 49.736053746000003, 2024-09-21T21:42:41.111 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.98183651999999999, 2024-09-21T21:42:41.111 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:41.111 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:41.111 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:41.111 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:41.111 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:41.111 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:41.112 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19843 2024-09-21T21:42:41.112 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.112 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:41.112 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.112 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:41.112 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275860+0000", 2024-09-21T21:42:41.112 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:41.112 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.112 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.112 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:41.112 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275860+0000", 2024-09-21T21:42:41.112 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:41.113 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.113 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.113 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:41.113 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275869+0000", 2024-09-21T21:42:41.113 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6300000000000004e-06 2024-09-21T21:42:41.113 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.113 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.113 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:41.113 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275871+0000", 2024-09-21T21:42:41.113 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2060000000000001e-06 2024-09-21T21:42:41.113 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.113 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.114 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:41.114 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275876+0000", 2024-09-21T21:42:41.114 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8180000000000003e-06 2024-09-21T21:42:41.114 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.114 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.114 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:41.114 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275880+0000", 2024-09-21T21:42:41.114 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1829999999999996e-06 2024-09-21T21:42:41.114 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.114 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.114 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:41.114 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278416+0000", 2024-09-21T21:42:41.115 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025361149999999998 2024-09-21T21:42:41.115 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.115 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.115 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:41.115 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278419+0000", 2024-09-21T21:42:41.115 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.163e-06 2024-09-21T21:42:41.115 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.115 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.115 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:41.115 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.250249+0000", 2024-09-21T21:42:41.115 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97182984299999997 2024-09-21T21:42:41.115 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.115 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.116 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:41.116 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.250321+0000", 2024-09-21T21:42:41.116 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1656000000000006e-05 2024-09-21T21:42:41.116 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.116 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.116 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 2", 2024-09-21T21:42:41.116 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.250419+0000", 2024-09-21T21:42:41.116 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.8277999999999997e-05 2024-09-21T21:42:41.116 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.116 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.116 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:41.116 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.252516+0000", 2024-09-21T21:42:41.117 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002097224 2024-09-21T21:42:41.117 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.117 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.117 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:41.117 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.257643+0000", 2024-09-21T21:42:41.117 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0051262319999999997 2024-09-21T21:42:41.117 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.117 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.117 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:41.117 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.257673+0000", 2024-09-21T21:42:41.117 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0023e-05 2024-09-21T21:42:41.117 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.117 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.118 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:41.118 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.257697+0000", 2024-09-21T21:42:41.118 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4148999999999999e-05 2024-09-21T21:42:41.118 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:41.118 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:41.118 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:41.118 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.118 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.118 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5626.0:12334 3.3 3:c643a86c:::rbd_data.1121faebdfe3.0000000000000002:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:42:41.118 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:14.411628+0000", 2024-09-21T21:42:41.118 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 26.600285447000001, 2024-09-21T21:42:41.118 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0086947559999999, 2024-09-21T21:42:41.119 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:41.119 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:41.119 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:41.119 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:41.119 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5626", 2024-09-21T21:42:41.119 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:41.119 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 12334 2024-09-21T21:42:41.119 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.119 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:41.119 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.119 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:41.119 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411628+0000", 2024-09-21T21:42:41.120 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:41.120 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.120 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.120 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:41.120 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411628+0000", 2024-09-21T21:42:41.120 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:41.120 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.120 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.120 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:41.120 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411629+0000", 2024-09-21T21:42:41.120 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1200000000000002e-07 2024-09-21T21:42:41.120 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.121 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.121 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:41.121 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411633+0000", 2024-09-21T21:42:41.121 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1699999999999999e-06 2024-09-21T21:42:41.121 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.121 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.121 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:41.121 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411645+0000", 2024-09-21T21:42:41.121 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1328999999999999e-05 2024-09-21T21:42:41.121 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.121 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.121 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:41.121 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411652+0000", 2024-09-21T21:42:41.121 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9990000000000002e-06 2024-09-21T21:42:41.122 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.122 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.122 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:41.122 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411717+0000", 2024-09-21T21:42:41.122 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5519999999999996e-05 2024-09-21T21:42:41.122 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.122 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.122 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op must wait for map", 2024-09-21T21:42:41.122 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411725+0000", 2024-09-21T21:42:41.122 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2609999999999996e-06 2024-09-21T21:42:41.122 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.122 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.122 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:41.123 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415904+0000", 2024-09-21T21:42:41.123 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0041792749999999997 2024-09-21T21:42:41.123 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.123 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.123 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:41.123 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415913+0000", 2024-09-21T21:42:41.123 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.054e-06 2024-09-21T21:42:41.123 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.123 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.123 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:41.123 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420096+0000", 2024-09-21T21:42:41.123 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0041835670000001 2024-09-21T21:42:41.123 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.124 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.124 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:41.124 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420216+0000", 2024-09-21T21:42:41.124 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000120025 2024-09-21T21:42:41.124 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.124 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.124 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:41.124 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420323+0000", 2024-09-21T21:42:41.124 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010674400000000001 2024-09-21T21:42:41.124 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:41.124 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:41.125 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:41.125 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.125 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.125 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5626.0:12339 3.3 3:c643a86c:::rbd_data.1121faebdfe3.0000000000000002:head [read 4096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:42:41.125 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:14.411749+0000", 2024-09-21T21:42:41.125 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 26.600165261000001, 2024-09-21T21:42:41.125 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.008818056, 2024-09-21T21:42:41.125 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:41.125 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:41.125 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:41.125 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:41.126 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5626", 2024-09-21T21:42:41.126 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:41.126 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 12339 2024-09-21T21:42:41.126 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.126 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:41.126 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.126 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:41.126 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411749+0000", 2024-09-21T21:42:41.126 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:41.126 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.126 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.126 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:41.127 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411749+0000", 2024-09-21T21:42:41.127 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:41.127 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.127 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.127 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:41.127 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411749+0000", 2024-09-21T21:42:41.127 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0900000000000002e-07 2024-09-21T21:42:41.127 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.127 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.127 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:41.127 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411752+0000", 2024-09-21T21:42:41.127 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5610000000000001e-06 2024-09-21T21:42:41.127 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.128 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.128 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:41.128 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411758+0000", 2024-09-21T21:42:41.128 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1890000000000004e-06 2024-09-21T21:42:41.128 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.128 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.128 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:41.128 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411763+0000", 2024-09-21T21:42:41.128 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7589999999999997e-06 2024-09-21T21:42:41.128 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.128 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.128 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:41.129 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411803+0000", 2024-09-21T21:42:41.129 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9922999999999997e-05 2024-09-21T21:42:41.129 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.129 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.129 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting_for_map not empty", 2024-09-21T21:42:41.129 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411809+0000", 2024-09-21T21:42:41.129 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3779999999999998e-06 2024-09-21T21:42:41.129 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.129 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.129 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:41.129 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415937+0000", 2024-09-21T21:42:41.130 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0041276100000000003 2024-09-21T21:42:41.130 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.130 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.130 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:41.130 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415944+0000", 2024-09-21T21:42:41.130 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8329999999999994e-06 2024-09-21T21:42:41.130 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.130 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.130 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:41.130 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420411+0000", 2024-09-21T21:42:41.130 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0044667300000001 2024-09-21T21:42:41.130 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.130 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.131 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:41.131 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420475+0000", 2024-09-21T21:42:41.131 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4356000000000005e-05 2024-09-21T21:42:41.131 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.131 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.131 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:41.131 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420567+0000", 2024-09-21T21:42:41.131 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.1308000000000001e-05 2024-09-21T21:42:41.131 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:41.131 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:41.131 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:41.132 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.132 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.132 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5626.0:12345 3.3 3:c643a86c:::rbd_data.1121faebdfe3.0000000000000002:head [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:42:41.132 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:14.411831+0000", 2024-09-21T21:42:41.132 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 26.600083208000001, 2024-09-21T21:42:41.132 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0089163649999999, 2024-09-21T21:42:41.132 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:41.132 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:41.132 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:41.132 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:41.132 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5626", 2024-09-21T21:42:41.132 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:41.132 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 12345 2024-09-21T21:42:41.133 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.133 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:41.133 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.133 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:41.133 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411831+0000", 2024-09-21T21:42:41.133 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:41.133 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.133 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.133 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:41.133 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411831+0000", 2024-09-21T21:42:41.133 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:41.133 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.133 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.134 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:41.134 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411831+0000", 2024-09-21T21:42:41.134 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.08e-07 2024-09-21T21:42:41.134 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.134 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.134 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:41.134 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411833+0000", 2024-09-21T21:42:41.134 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3089999999999998e-06 2024-09-21T21:42:41.134 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.134 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.134 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:41.134 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411840+0000", 2024-09-21T21:42:41.135 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.472e-06 2024-09-21T21:42:41.135 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.135 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.135 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:41.135 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411845+0000", 2024-09-21T21:42:41.135 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7910000000000004e-06 2024-09-21T21:42:41.135 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.135 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.135 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:41.135 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411890+0000", 2024-09-21T21:42:41.135 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5763000000000002e-05 2024-09-21T21:42:41.135 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.136 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.136 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting_for_map not empty", 2024-09-21T21:42:41.136 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411896+0000", 2024-09-21T21:42:41.136 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0190000000000003e-06 2024-09-21T21:42:41.136 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.136 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.136 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:41.136 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415967+0000", 2024-09-21T21:42:41.136 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0040705749999999999 2024-09-21T21:42:41.136 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.136 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.136 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:41.137 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415974+0000", 2024-09-21T21:42:41.137 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3200000000000002e-06 2024-09-21T21:42:41.137 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.137 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.137 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:41.137 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420594+0000", 2024-09-21T21:42:41.137 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.004620093 2024-09-21T21:42:41.137 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.137 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.137 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:41.137 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420658+0000", 2024-09-21T21:42:41.137 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3678999999999998e-05 2024-09-21T21:42:41.138 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.138 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.138 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:41.138 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420747+0000", 2024-09-21T21:42:41.138 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8936000000000004e-05 2024-09-21T21:42:41.138 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:41.138 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:41.138 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:41.138 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:41.138 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:41.138 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:41.140 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T21:42:41.140 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:41.139+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:41.240 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:41.239+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:41.283 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:41.283 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:42:41.283 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-21T21:42:41.283 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:41.291 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T21:42:41.342 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:41.341+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:41.393 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:41.393 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:42:41.393 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:42:41.393 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:42:41.393 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:41.402 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T21:42:41.443 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:41.441+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:41.506 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:41.506 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-21T21:42:41.506 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-21T21:42:41.506 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:42:41.506 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.506 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6974 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5246976~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:41.507 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.922932+0000", 2024-09-21T21:42:41.507 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 71.581193233999997, 2024-09-21T21:42:41.507 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041773352999999999, 2024-09-21T21:42:41.507 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:41.507 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:41.507 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:41.507 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:41.507 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:41.507 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:41.507 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6974 2024-09-21T21:42:41.507 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.507 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:41.507 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.508 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:41.508 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922932+0000", 2024-09-21T21:42:41.508 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:41.508 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.508 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.508 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:41.508 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922932+0000", 2024-09-21T21:42:41.508 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:41.508 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.508 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.508 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:41.508 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922933+0000", 2024-09-21T21:42:41.508 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.46e-07 2024-09-21T21:42:41.508 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.509 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.509 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:41.509 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922940+0000", 2024-09-21T21:42:41.509 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6140000000000003e-06 2024-09-21T21:42:41.509 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.509 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.509 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:41.509 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922949+0000", 2024-09-21T21:42:41.509 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6899999999999998e-06 2024-09-21T21:42:41.509 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.509 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.509 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:41.509 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922955+0000", 2024-09-21T21:42:41.509 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5140000000000001e-06 2024-09-21T21:42:41.510 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.510 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.510 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:41.510 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964547+0000", 2024-09-21T21:42:41.510 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041592854999999998 2024-09-21T21:42:41.510 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.510 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.510 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:41.510 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964617+0000", 2024-09-21T21:42:41.510 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9058000000000002e-05 2024-09-21T21:42:41.510 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.510 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.510 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:41.511 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964706+0000", 2024-09-21T21:42:41.511 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9276000000000004e-05 2024-09-21T21:42:41.511 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:41.511 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:41.511 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:41.511 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.511 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.511 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6976 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5261312~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:41.511 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.922961+0000", 2024-09-21T21:42:41.511 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 71.581164189000006, 2024-09-21T21:42:41.511 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041863648000000003, 2024-09-21T21:42:41.511 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:41.511 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:41.511 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:41.512 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:41.512 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:41.512 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:41.512 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6976 2024-09-21T21:42:41.512 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.512 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:41.512 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.512 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:41.512 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922961+0000", 2024-09-21T21:42:41.512 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:41.512 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.512 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.512 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:41.512 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922961+0000", 2024-09-21T21:42:41.512 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:41.513 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.513 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.513 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:41.513 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922962+0000", 2024-09-21T21:42:41.513 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.89e-07 2024-09-21T21:42:41.513 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.513 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.513 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:41.513 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922964+0000", 2024-09-21T21:42:41.513 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1270000000000001e-06 2024-09-21T21:42:41.513 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.513 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.513 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:41.513 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922970+0000", 2024-09-21T21:42:41.513 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8030000000000002e-06 2024-09-21T21:42:41.514 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.514 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.514 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:41.514 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922979+0000", 2024-09-21T21:42:41.514 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5559999999999996e-06 2024-09-21T21:42:41.514 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.514 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.514 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:41.514 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964734+0000", 2024-09-21T21:42:41.514 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041754415000000003 2024-09-21T21:42:41.514 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.514 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.514 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:41.514 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964775+0000", 2024-09-21T21:42:41.515 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0806999999999997e-05 2024-09-21T21:42:41.515 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.515 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.515 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:41.515 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964825+0000", 2024-09-21T21:42:41.515 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0550999999999998e-05 2024-09-21T21:42:41.515 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:41.515 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:41.515 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:41.515 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.515 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.515 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6978 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5275648~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:41.515 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.922987+0000", 2024-09-21T21:42:41.515 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 71.581138404000001, 2024-09-21T21:42:41.515 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042043876000000001, 2024-09-21T21:42:41.516 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:41.516 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:41.516 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:41.516 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:41.516 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:41.516 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:41.516 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6978 2024-09-21T21:42:41.516 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.516 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:41.516 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.516 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:41.516 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922987+0000", 2024-09-21T21:42:41.516 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:41.516 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.516 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.517 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:41.517 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922987+0000", 2024-09-21T21:42:41.517 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:41.517 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.517 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.517 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:41.517 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922988+0000", 2024-09-21T21:42:41.517 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1199999999999998e-07 2024-09-21T21:42:41.517 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.517 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.517 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:41.517 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922990+0000", 2024-09-21T21:42:41.517 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.142e-06 2024-09-21T21:42:41.517 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.517 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.518 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:41.518 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922997+0000", 2024-09-21T21:42:41.518 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.199e-06 2024-09-21T21:42:41.518 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.518 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.518 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:41.518 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923004+0000", 2024-09-21T21:42:41.518 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3599999999999998e-06 2024-09-21T21:42:41.518 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.518 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.518 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:41.518 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964856+0000", 2024-09-21T21:42:41.518 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041851521000000003 2024-09-21T21:42:41.518 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.519 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.519 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:41.519 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964944+0000", 2024-09-21T21:42:41.519 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7859e-05 2024-09-21T21:42:41.519 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.519 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.519 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:41.519 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965031+0000", 2024-09-21T21:42:41.519 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7383e-05 2024-09-21T21:42:41.519 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:41.519 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:41.519 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:41.519 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.519 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.519 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6980 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5289984~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:41.520 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923011+0000", 2024-09-21T21:42:41.520 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 71.581115066999999, 2024-09-21T21:42:41.520 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042139337999999998, 2024-09-21T21:42:41.520 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:41.520 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:41.520 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:41.520 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:41.520 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:41.520 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:41.520 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6980 2024-09-21T21:42:41.520 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.520 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:41.520 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.520 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:41.521 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923011+0000", 2024-09-21T21:42:41.521 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:41.521 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.521 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.521 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:41.521 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923011+0000", 2024-09-21T21:42:41.521 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:41.521 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.521 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.521 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:41.521 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923011+0000", 2024-09-21T21:42:41.521 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6699999999999999e-07 2024-09-21T21:42:41.521 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.521 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.521 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:41.522 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923013+0000", 2024-09-21T21:42:41.522 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0279999999999999e-06 2024-09-21T21:42:41.522 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.522 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.522 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:41.522 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923018+0000", 2024-09-21T21:42:41.522 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4790000000000004e-06 2024-09-21T21:42:41.522 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.522 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.522 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:41.522 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923029+0000", 2024-09-21T21:42:41.522 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0601999999999999e-05 2024-09-21T21:42:41.522 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.522 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.523 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:41.523 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965059+0000", 2024-09-21T21:42:41.523 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042029947999999998 2024-09-21T21:42:41.523 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.523 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.523 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:41.523 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965100+0000", 2024-09-21T21:42:41.523 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1170000000000001e-05 2024-09-21T21:42:41.523 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.523 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.523 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:41.523 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965150+0000", 2024-09-21T21:42:41.523 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9743999999999997e-05 2024-09-21T21:42:41.523 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:41.524 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:41.524 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:41.524 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.524 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.524 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6982 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5304320~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:41.524 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923036+0000", 2024-09-21T21:42:41.524 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 71.581089246000005, 2024-09-21T21:42:41.524 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042225666000000002, 2024-09-21T21:42:41.524 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:41.524 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:41.524 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:41.524 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:41.524 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:41.524 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:41.524 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6982 2024-09-21T21:42:41.525 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.525 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:41.525 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.525 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:41.525 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923036+0000", 2024-09-21T21:42:41.525 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:41.525 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.525 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.525 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:41.525 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923036+0000", 2024-09-21T21:42:41.525 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:41.525 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.525 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.526 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:41.526 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923037+0000", 2024-09-21T21:42:41.526 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2800000000000002e-07 2024-09-21T21:42:41.526 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.526 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.526 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:41.526 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923039+0000", 2024-09-21T21:42:41.526 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1500000000000002e-06 2024-09-21T21:42:41.526 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.526 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.526 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:41.526 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923049+0000", 2024-09-21T21:42:41.526 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0236e-05 2024-09-21T21:42:41.526 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.527 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.527 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:41.527 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923056+0000", 2024-09-21T21:42:41.527 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2540000000000003e-06 2024-09-21T21:42:41.527 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.527 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.527 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:41.527 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965173+0000", 2024-09-21T21:42:41.527 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042116686 2024-09-21T21:42:41.527 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.527 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.527 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:41.527 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965215+0000", 2024-09-21T21:42:41.527 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1696000000000002e-05 2024-09-21T21:42:41.528 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.528 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.528 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:41.528 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965262+0000", 2024-09-21T21:42:41.528 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7216000000000003e-05 2024-09-21T21:42:41.528 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:41.528 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:41.528 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:41.528 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.528 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.528 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6984 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5318656~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:41.528 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923063+0000", 2024-09-21T21:42:41.528 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 71.581062591999995, 2024-09-21T21:42:41.528 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042318651999999998, 2024-09-21T21:42:41.529 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:41.529 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:41.529 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:41.529 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:41.529 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:41.529 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:41.529 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6984 2024-09-21T21:42:41.529 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.529 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:41.529 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.529 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:41.529 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923063+0000", 2024-09-21T21:42:41.529 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:41.529 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.530 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.530 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:41.530 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923063+0000", 2024-09-21T21:42:41.530 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:41.530 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.530 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.530 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:41.530 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923064+0000", 2024-09-21T21:42:41.530 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2e-07 2024-09-21T21:42:41.530 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.530 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.530 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:41.530 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923066+0000", 2024-09-21T21:42:41.530 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.057e-06 2024-09-21T21:42:41.530 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.531 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.531 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:41.531 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923071+0000", 2024-09-21T21:42:41.531 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8810000000000001e-06 2024-09-21T21:42:41.531 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.531 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.531 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:41.531 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923078+0000", 2024-09-21T21:42:41.531 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5479999999999998e-06 2024-09-21T21:42:41.531 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.531 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.531 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:41.531 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965288+0000", 2024-09-21T21:42:41.531 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042209576999999998 2024-09-21T21:42:41.531 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.532 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.532 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:41.532 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965332+0000", 2024-09-21T21:42:41.532 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4366999999999997e-05 2024-09-21T21:42:41.532 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.532 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.532 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:41.532 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965382+0000", 2024-09-21T21:42:41.532 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9802000000000002e-05 2024-09-21T21:42:41.532 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:41.532 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:41.532 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:41.532 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.532 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.533 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6987 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5332992~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:41.533 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923085+0000", 2024-09-21T21:42:41.533 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 71.581041067000001, 2024-09-21T21:42:41.533 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042412103999999999, 2024-09-21T21:42:41.533 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:41.533 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:41.533 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:41.533 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:41.533 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:41.533 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:41.533 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6987 2024-09-21T21:42:41.533 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.533 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:41.533 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.533 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:41.534 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923085+0000", 2024-09-21T21:42:41.534 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:41.534 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.534 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.534 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:41.534 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923085+0000", 2024-09-21T21:42:41.534 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:41.534 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.534 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.534 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:41.534 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923085+0000", 2024-09-21T21:42:41.534 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.89e-07 2024-09-21T21:42:41.534 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.535 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.535 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:41.535 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923087+0000", 2024-09-21T21:42:41.535 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0810000000000001e-06 2024-09-21T21:42:41.535 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.535 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.535 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:41.535 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923098+0000", 2024-09-21T21:42:41.535 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1299e-05 2024-09-21T21:42:41.535 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.535 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.535 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:41.535 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923104+0000", 2024-09-21T21:42:41.536 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9100000000000002e-06 2024-09-21T21:42:41.536 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.536 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.536 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:41.536 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965404+0000", 2024-09-21T21:42:41.536 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042299622000000002 2024-09-21T21:42:41.536 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.536 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.536 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:41.536 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965445+0000", 2024-09-21T21:42:41.536 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0701000000000002e-05 2024-09-21T21:42:41.536 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.536 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.536 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:41.537 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965497+0000", 2024-09-21T21:42:41.537 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2101999999999997e-05 2024-09-21T21:42:41.537 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:41.537 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:41.537 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:41.537 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.537 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.537 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6989 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5347328~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:41.537 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923111+0000", 2024-09-21T21:42:41.537 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 71.581014726999996, 2024-09-21T21:42:41.537 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042505188999999999, 2024-09-21T21:42:41.537 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:41.537 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:41.537 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:41.538 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:41.538 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:41.538 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:41.538 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6989 2024-09-21T21:42:41.538 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.538 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:41.538 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.538 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:41.538 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923111+0000", 2024-09-21T21:42:41.538 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:41.538 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.538 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.538 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:41.538 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923111+0000", 2024-09-21T21:42:41.538 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:41.539 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.539 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.539 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:41.539 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923111+0000", 2024-09-21T21:42:41.539 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4499999999999997e-07 2024-09-21T21:42:41.539 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.539 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.539 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:41.539 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923114+0000", 2024-09-21T21:42:41.539 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1569999999999998e-06 2024-09-21T21:42:41.539 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.539 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.539 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:41.539 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923120+0000", 2024-09-21T21:42:41.540 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1759999999999998e-06 2024-09-21T21:42:41.540 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.540 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.540 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:41.540 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923125+0000", 2024-09-21T21:42:41.540 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2560000000000002e-06 2024-09-21T21:42:41.540 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.540 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.540 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:41.540 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965523+0000", 2024-09-21T21:42:41.540 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042398094999999997 2024-09-21T21:42:41.540 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.540 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.540 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:41.540 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965566+0000", 2024-09-21T21:42:41.541 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3393999999999998e-05 2024-09-21T21:42:41.541 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.541 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.541 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:41.541 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965616+0000", 2024-09-21T21:42:41.541 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9666000000000002e-05 2024-09-21T21:42:41.541 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:41.541 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:41.541 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:41.541 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.541 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.541 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6991 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5361664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:41.541 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923132+0000", 2024-09-21T21:42:41.541 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 71.580993437999993, 2024-09-21T21:42:41.541 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042600779999999998, 2024-09-21T21:42:41.541 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:41.542 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:41.542 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:41.542 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:41.542 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:41.542 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:41.542 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6991 2024-09-21T21:42:41.542 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.542 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:41.542 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.542 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:41.542 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923132+0000", 2024-09-21T21:42:41.542 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:41.542 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.542 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.543 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:41.543 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923132+0000", 2024-09-21T21:42:41.543 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:41.543 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.543 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.543 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:41.543 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923132+0000", 2024-09-21T21:42:41.543 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4200000000000002e-07 2024-09-21T21:42:41.543 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.543 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.543 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:41.543 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923134+0000", 2024-09-21T21:42:41.543 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2270000000000001e-06 2024-09-21T21:42:41.543 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.543 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.544 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:41.544 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923137+0000", 2024-09-21T21:42:41.544 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1930000000000002e-06 2024-09-21T21:42:41.544 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.544 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.544 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:41.544 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923139+0000", 2024-09-21T21:42:41.544 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4710000000000001e-06 2024-09-21T21:42:41.544 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.544 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.544 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:41.544 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965639+0000", 2024-09-21T21:42:41.544 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.04249965 2024-09-21T21:42:41.544 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.544 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.544 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:41.545 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965679+0000", 2024-09-21T21:42:41.545 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9852999999999999e-05 2024-09-21T21:42:41.545 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.545 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.545 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:41.545 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965733+0000", 2024-09-21T21:42:41.545 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4144000000000002e-05 2024-09-21T21:42:41.545 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:41.545 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:41.545 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:41.545 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.545 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.545 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6993 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5384192~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:41.545 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923143+0000", 2024-09-21T21:42:41.545 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 71.580982644000002, 2024-09-21T21:42:41.545 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042708182999999997, 2024-09-21T21:42:41.546 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:41.546 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:41.546 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:41.546 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:41.546 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:41.546 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:41.546 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6993 2024-09-21T21:42:41.546 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.546 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:41.546 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.546 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:41.546 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923143+0000", 2024-09-21T21:42:41.546 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:41.546 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.547 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.547 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:41.547 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923143+0000", 2024-09-21T21:42:41.547 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:41.547 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.547 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.547 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:41.547 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923143+0000", 2024-09-21T21:42:41.547 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0800000000000001e-07 2024-09-21T21:42:41.547 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.547 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.547 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:41.547 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923144+0000", 2024-09-21T21:42:41.547 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.009e-06 2024-09-21T21:42:41.547 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.547 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.548 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:41.548 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923148+0000", 2024-09-21T21:42:41.548 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.737e-06 2024-09-21T21:42:41.548 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.548 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.548 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:41.548 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923150+0000", 2024-09-21T21:42:41.548 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4669999999999998e-06 2024-09-21T21:42:41.548 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.548 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.548 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:41.548 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965760+0000", 2024-09-21T21:42:41.548 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042609399999999999 2024-09-21T21:42:41.548 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.548 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.549 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:41.549 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965801+0000", 2024-09-21T21:42:41.549 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1399999999999997e-05 2024-09-21T21:42:41.549 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.549 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.549 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:41.549 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965851+0000", 2024-09-21T21:42:41.549 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9962e-05 2024-09-21T21:42:41.549 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:41.549 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:41.549 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:41.549 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.549 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.549 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6995 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5398528~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:41.549 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923153+0000", 2024-09-21T21:42:41.550 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 71.580972279999997, 2024-09-21T21:42:41.550 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042828034000000001, 2024-09-21T21:42:41.550 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:41.550 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:41.550 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:41.550 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:41.550 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:41.550 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:41.550 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6995 2024-09-21T21:42:41.550 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.550 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:41.550 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.550 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:41.550 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923153+0000", 2024-09-21T21:42:41.551 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:41.551 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.551 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.551 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:41.551 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923153+0000", 2024-09-21T21:42:41.551 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:41.551 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.551 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.551 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:41.551 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923154+0000", 2024-09-21T21:42:41.551 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2600000000000001e-07 2024-09-21T21:42:41.551 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.551 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.551 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:41.551 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923158+0000", 2024-09-21T21:42:41.552 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0559999999999998e-06 2024-09-21T21:42:41.552 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.552 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.552 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:41.552 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923161+0000", 2024-09-21T21:42:41.552 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6519999999999999e-06 2024-09-21T21:42:41.552 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.552 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.552 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:41.552 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923170+0000", 2024-09-21T21:42:41.552 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5720000000000003e-06 2024-09-21T21:42:41.552 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.552 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.552 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:41.552 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965887+0000", 2024-09-21T21:42:41.552 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042716863000000001 2024-09-21T21:42:41.553 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.553 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.553 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:41.553 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965927+0000", 2024-09-21T21:42:41.553 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0346999999999998e-05 2024-09-21T21:42:41.553 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.553 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.553 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:41.553 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965981+0000", 2024-09-21T21:42:41.553 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4317999999999997e-05 2024-09-21T21:42:41.553 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:41.553 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:41.553 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:41.553 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.553 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.554 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6997 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5412864~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:41.554 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923174+0000", 2024-09-21T21:42:41.554 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 71.580952124999996, 2024-09-21T21:42:41.554 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042928296999999997, 2024-09-21T21:42:41.554 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:41.554 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:41.554 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:41.554 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:41.554 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:41.554 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:41.554 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6997 2024-09-21T21:42:41.554 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.554 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:41.554 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.554 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:41.555 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923174+0000", 2024-09-21T21:42:41.555 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:41.555 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.555 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.555 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:41.555 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923174+0000", 2024-09-21T21:42:41.555 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:41.555 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.555 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.555 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:41.555 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923174+0000", 2024-09-21T21:42:41.555 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.67e-07 2024-09-21T21:42:41.555 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.555 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.555 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:41.556 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923178+0000", 2024-09-21T21:42:41.556 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1330000000000001e-06 2024-09-21T21:42:41.556 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.556 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.556 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:41.556 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923182+0000", 2024-09-21T21:42:41.556 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.794e-06 2024-09-21T21:42:41.556 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.556 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.556 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:41.556 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923184+0000", 2024-09-21T21:42:41.556 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.435e-06 2024-09-21T21:42:41.556 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.556 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.556 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:41.557 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966009+0000", 2024-09-21T21:42:41.557 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042824327000000002 2024-09-21T21:42:41.557 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.557 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.557 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:41.557 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966052+0000", 2024-09-21T21:42:41.557 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3034000000000002e-05 2024-09-21T21:42:41.557 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.557 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.557 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:41.557 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966102+0000", 2024-09-21T21:42:41.557 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0306999999999998e-05 2024-09-21T21:42:41.557 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:41.557 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:41.558 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:41.558 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.558 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.558 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6999 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5427200~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:41.558 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923187+0000", 2024-09-21T21:42:41.558 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 71.580938396999997, 2024-09-21T21:42:41.558 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043104267000000002, 2024-09-21T21:42:41.558 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:41.558 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:41.558 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:41.558 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:41.558 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:41.558 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:41.558 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6999 2024-09-21T21:42:41.558 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.559 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:41.559 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.559 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:41.559 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923187+0000", 2024-09-21T21:42:41.559 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:41.559 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.559 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.559 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:41.559 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923187+0000", 2024-09-21T21:42:41.559 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:41.559 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.559 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.559 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:41.559 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923188+0000", 2024-09-21T21:42:41.559 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3999999999999998e-07 2024-09-21T21:42:41.560 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.560 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.560 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:41.560 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923189+0000", 2024-09-21T21:42:41.560 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4169999999999999e-06 2024-09-21T21:42:41.560 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.560 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.560 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:41.560 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923194+0000", 2024-09-21T21:42:41.560 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1819999999999997e-06 2024-09-21T21:42:41.560 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.560 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.560 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:41.561 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923198+0000", 2024-09-21T21:42:41.561 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.143e-06 2024-09-21T21:42:41.561 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.561 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.561 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:41.561 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966135+0000", 2024-09-21T21:42:41.561 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042936297999999998 2024-09-21T21:42:41.561 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.561 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.561 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:41.561 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966204+0000", 2024-09-21T21:42:41.561 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8989e-05 2024-09-21T21:42:41.561 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.561 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.562 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:41.562 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966292+0000", 2024-09-21T21:42:41.562 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7997999999999993e-05 2024-09-21T21:42:41.562 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:41.562 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:41.562 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:41.562 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.562 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.562 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7001 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5441536~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:41.562 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923211+0000", 2024-09-21T21:42:41.562 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 71.580914605999993, 2024-09-21T21:42:41.562 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043196518000000003, 2024-09-21T21:42:41.562 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:41.563 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:41.563 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:41.563 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:41.563 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:41.563 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:41.563 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7001 2024-09-21T21:42:41.563 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.563 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:41.563 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.563 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:41.563 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923211+0000", 2024-09-21T21:42:41.563 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:41.563 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.564 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.564 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:41.564 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923211+0000", 2024-09-21T21:42:41.564 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:41.564 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.564 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.564 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:41.564 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923212+0000", 2024-09-21T21:42:41.564 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3900000000000005e-07 2024-09-21T21:42:41.564 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.564 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.564 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:41.564 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923214+0000", 2024-09-21T21:42:41.565 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9970000000000001e-06 2024-09-21T21:42:41.565 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.565 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.565 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:41.565 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923219+0000", 2024-09-21T21:42:41.565 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4310000000000002e-06 2024-09-21T21:42:41.565 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.565 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.565 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:41.565 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923224+0000", 2024-09-21T21:42:41.565 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6410000000000001e-06 2024-09-21T21:42:41.565 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.565 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.565 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:41.565 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966318+0000", 2024-09-21T21:42:41.565 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043093842 2024-09-21T21:42:41.566 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.566 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.566 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:41.566 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966359+0000", 2024-09-21T21:42:41.566 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1260999999999997e-05 2024-09-21T21:42:41.566 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.566 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.566 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:41.566 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966408+0000", 2024-09-21T21:42:41.566 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8807000000000002e-05 2024-09-21T21:42:41.566 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:41.566 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:41.566 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:41.567 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.567 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.567 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7002 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5455872~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:41.567 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923229+0000", 2024-09-21T21:42:41.567 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 71.580896195999998, 2024-09-21T21:42:41.567 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043364568999999999, 2024-09-21T21:42:41.567 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:41.567 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:41.567 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:41.567 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:41.567 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:41.567 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:41.567 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7002 2024-09-21T21:42:41.567 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.568 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:41.568 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.568 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:41.568 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923229+0000", 2024-09-21T21:42:41.568 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:41.568 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.568 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.568 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:41.568 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923229+0000", 2024-09-21T21:42:41.568 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:41.568 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.568 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.568 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:41.568 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923230+0000", 2024-09-21T21:42:41.568 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4700000000000002e-07 2024-09-21T21:42:41.568 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.569 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.569 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:41.569 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923232+0000", 2024-09-21T21:42:41.569 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0719999999999998e-06 2024-09-21T21:42:41.569 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.569 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.569 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:41.569 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923237+0000", 2024-09-21T21:42:41.569 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2290000000000002e-06 2024-09-21T21:42:41.569 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.569 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.569 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:41.569 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923245+0000", 2024-09-21T21:42:41.569 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2959999999999999e-06 2024-09-21T21:42:41.569 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.570 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.570 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:41.570 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966438+0000", 2024-09-21T21:42:41.570 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043193027000000002 2024-09-21T21:42:41.570 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.570 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.570 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:41.570 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966506+0000", 2024-09-21T21:42:41.570 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7707999999999996e-05 2024-09-21T21:42:41.570 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.570 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.570 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:41.570 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966594+0000", 2024-09-21T21:42:41.570 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.789e-05 2024-09-21T21:42:41.570 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:41.571 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:41.571 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:41.571 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.571 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.571 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7005 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5470208~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:41.571 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923254+0000", 2024-09-21T21:42:41.571 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 71.580871965, 2024-09-21T21:42:41.571 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043457176, 2024-09-21T21:42:41.571 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:41.571 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:41.571 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:41.571 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:41.571 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:41.571 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:41.572 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7005 2024-09-21T21:42:41.572 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.572 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:41.572 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.572 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:41.572 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923254+0000", 2024-09-21T21:42:41.572 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:41.572 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.572 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.572 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:41.572 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923254+0000", 2024-09-21T21:42:41.572 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:41.572 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.572 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.572 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:41.572 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923254+0000", 2024-09-21T21:42:41.573 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4700000000000002e-07 2024-09-21T21:42:41.573 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.573 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.573 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:41.573 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923256+0000", 2024-09-21T21:42:41.573 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.007e-06 2024-09-21T21:42:41.573 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.573 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.573 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:41.573 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923266+0000", 2024-09-21T21:42:41.573 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.4700000000000008e-06 2024-09-21T21:42:41.573 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.573 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.573 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:41.573 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923271+0000", 2024-09-21T21:42:41.574 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2809999999999999e-06 2024-09-21T21:42:41.574 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.574 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.574 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:41.574 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966619+0000", 2024-09-21T21:42:41.574 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043347951000000003 2024-09-21T21:42:41.574 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.574 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.574 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:41.574 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966661+0000", 2024-09-21T21:42:41.574 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2614999999999998e-05 2024-09-21T21:42:41.574 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.574 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.574 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:41.575 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966711+0000", 2024-09-21T21:42:41.575 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9505000000000001e-05 2024-09-21T21:42:41.575 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:41.575 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:41.575 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:41.575 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.575 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.575 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7008 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5484544~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:41.575 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923278+0000", 2024-09-21T21:42:41.575 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 71.580847317999996, 2024-09-21T21:42:41.575 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043631071, 2024-09-21T21:42:41.575 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:41.575 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:41.575 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:41.575 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:41.576 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:41.576 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:41.576 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7008 2024-09-21T21:42:41.576 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.576 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:41.576 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.576 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:41.576 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923278+0000", 2024-09-21T21:42:41.576 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:41.576 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.576 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.576 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:41.576 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923278+0000", 2024-09-21T21:42:41.576 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:41.576 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.576 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.577 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:41.577 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923279+0000", 2024-09-21T21:42:41.577 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.77e-07 2024-09-21T21:42:41.577 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.577 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.577 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:41.577 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923281+0000", 2024-09-21T21:42:41.577 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0219999999999999e-06 2024-09-21T21:42:41.577 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.577 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.577 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:41.577 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923287+0000", 2024-09-21T21:42:41.577 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7799999999999997e-06 2024-09-21T21:42:41.577 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.577 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.578 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:41.578 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923291+0000", 2024-09-21T21:42:41.578 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1699999999999999e-06 2024-09-21T21:42:41.578 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.578 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.578 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:41.578 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966742+0000", 2024-09-21T21:42:41.578 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043451344000000003 2024-09-21T21:42:41.578 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.578 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.578 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:41.578 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966809+0000", 2024-09-21T21:42:41.578 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7070999999999995e-05 2024-09-21T21:42:41.578 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.579 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.579 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:41.579 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966909+0000", 2024-09-21T21:42:41.579 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000100307 2024-09-21T21:42:41.579 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:41.579 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:41.579 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:41.579 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.579 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.579 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7010 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5498880~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:41.579 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923301+0000", 2024-09-21T21:42:41.579 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 71.580824422999996, 2024-09-21T21:42:41.579 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043717830999999999, 2024-09-21T21:42:41.579 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:41.580 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:41.580 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:41.580 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:41.580 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:41.580 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:41.580 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7010 2024-09-21T21:42:41.580 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.580 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:41.580 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.580 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:41.580 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923301+0000", 2024-09-21T21:42:41.580 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:41.580 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.580 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.580 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:41.581 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923301+0000", 2024-09-21T21:42:41.581 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:41.581 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.581 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.581 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:41.581 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923302+0000", 2024-09-21T21:42:41.581 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.41e-07 2024-09-21T21:42:41.581 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.581 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.581 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:41.581 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923304+0000", 2024-09-21T21:42:41.581 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0990000000000001e-06 2024-09-21T21:42:41.581 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.581 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.581 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:41.582 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923310+0000", 2024-09-21T21:42:41.582 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3860000000000004e-06 2024-09-21T21:42:41.582 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.582 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.582 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:41.582 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923315+0000", 2024-09-21T21:42:41.582 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8459999999999999e-06 2024-09-21T21:42:41.582 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.582 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.582 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:41.582 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966938+0000", 2024-09-21T21:42:41.582 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043622693999999997 2024-09-21T21:42:41.582 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.582 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.583 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:41.583 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966975+0000", 2024-09-21T21:42:41.583 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7608999999999999e-05 2024-09-21T21:42:41.583 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.583 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.583 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:41.583 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.967019+0000", 2024-09-21T21:42:41.583 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3856000000000002e-05 2024-09-21T21:42:41.583 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:41.583 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:41.583 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:41.583 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.583 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.583 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7354 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 2107392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:41.584 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.936932+0000", 2024-09-21T21:42:41.584 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 71.567193567999993, 2024-09-21T21:42:41.584 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041607708, 2024-09-21T21:42:41.584 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:41.584 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:41.584 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:41.584 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:41.584 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:41.584 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:41.584 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7354 2024-09-21T21:42:41.584 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.584 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:41.584 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.584 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:41.584 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.936932+0000", 2024-09-21T21:42:41.584 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:41.585 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.585 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.585 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:41.585 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.936932+0000", 2024-09-21T21:42:41.585 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:41.585 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.585 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.585 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:41.585 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.937276+0000", 2024-09-21T21:42:41.585 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00034344200000000001 2024-09-21T21:42:41.585 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.585 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.585 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:41.585 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.937279+0000", 2024-09-21T21:42:41.585 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.844e-06 2024-09-21T21:42:41.586 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.586 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.586 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:41.586 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.937288+0000", 2024-09-21T21:42:41.586 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2409999999999995e-06 2024-09-21T21:42:41.586 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.586 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.586 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:41.586 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.937293+0000", 2024-09-21T21:42:41.586 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6559999999999997e-06 2024-09-21T21:42:41.586 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.586 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.586 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:41.586 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.978448+0000", 2024-09-21T21:42:41.587 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041154621000000002 2024-09-21T21:42:41.587 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.587 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.587 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:41.587 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.978487+0000", 2024-09-21T21:42:41.587 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9254000000000001e-05 2024-09-21T21:42:41.587 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.587 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.587 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:41.587 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.978540+0000", 2024-09-21T21:42:41.587 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2649999999999999e-05 2024-09-21T21:42:41.587 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:41.587 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:41.587 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:41.587 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.588 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.588 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5215.0:1054 3.9 3:946d2401:::rbd_data.1121524d2522.000000000000000b:head [set-alloc-hint object_size 16777216 write_size 16777216,write 6615040~59392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-21T21:42:41.588 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:38.148979+0000", 2024-09-21T21:42:41.588 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 63.355146601000001, 2024-09-21T21:42:41.588 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047870388999999999, 2024-09-21T21:42:41.588 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:41.588 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:41.588 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:41.588 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:41.588 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5215", 2024-09-21T21:42:41.588 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1580282168", 2024-09-21T21:42:41.588 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1054 2024-09-21T21:42:41.588 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.588 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:41.588 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.589 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:41.589 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.148979+0000", 2024-09-21T21:42:41.589 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:41.589 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.589 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.589 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:41.589 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.148979+0000", 2024-09-21T21:42:41.589 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:41.589 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.589 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.589 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:41.589 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.148980+0000", 2024-09-21T21:42:41.589 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5799999999999999e-07 2024-09-21T21:42:41.589 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.590 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.590 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:41.590 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.149039+0000", 2024-09-21T21:42:41.590 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9151000000000003e-05 2024-09-21T21:42:41.590 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.590 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.590 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:41.590 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.149047+0000", 2024-09-21T21:42:41.590 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1000000000000004e-06 2024-09-21T21:42:41.590 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.590 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.590 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:41.590 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.149052+0000", 2024-09-21T21:42:41.590 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5959999999999997e-06 2024-09-21T21:42:41.591 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.591 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.591 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:41.591 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.195288+0000", 2024-09-21T21:42:41.591 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.046236248000000001 2024-09-21T21:42:41.591 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.591 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.591 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:41.591 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.195334+0000", 2024-09-21T21:42:41.591 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6575e-05 2024-09-21T21:42:41.591 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.591 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.591 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 2", 2024-09-21T21:42:41.591 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.195406+0000", 2024-09-21T21:42:41.592 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1715e-05 2024-09-21T21:42:41.592 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.592 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.592 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:41.592 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.196018+0000", 2024-09-21T21:42:41.592 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00061218999999999998 2024-09-21T21:42:41.592 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.592 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.592 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:41.592 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.196794+0000", 2024-09-21T21:42:41.592 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000775814 2024-09-21T21:42:41.592 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.592 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.592 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:41.592 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.196827+0000", 2024-09-21T21:42:41.593 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2419000000000003e-05 2024-09-21T21:42:41.593 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:41.593 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:41.593 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:41.593 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.196849+0000", 2024-09-21T21:42:41.593 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2923000000000001e-05 2024-09-21T21:42:41.593 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:41.593 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:41.593 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:41.593 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:41.593 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:41.593 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:41.594 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T21:42:41.595 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T21:42:41.596 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:41.593+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:41.696 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:41.694+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:41.746 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:41.746 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:42:41.746 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-21T21:42:41.746 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:41.755 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T21:42:41.799 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:41.798+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:41.859 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:41.859 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:42:41.859 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:42:41.859 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:42:41.859 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:41.867 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T21:42:41.897 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:41.896+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:41.999 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:41.998+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:42.012 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:42.013 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-21T21:42:42.013 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-21T21:42:42.013 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:42:42.013 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.013 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4385.0:15202 3.16 3:6888948b:::rbd_header.1121493f6459:head [watch reconnect cookie 140325440202496 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-21T21:42:42.013 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:41.736601+0000", 2024-09-21T21:42:42.013 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 60.273294362000001, 2024-09-21T21:42:42.013 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5040056769999999, 2024-09-21T21:42:42.013 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:42.013 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:42.013 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:42.013 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:42.013 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4385", 2024-09-21T21:42:42.014 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:42:42.014 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 15202 2024-09-21T21:42:42.014 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.014 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:42.014 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.014 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:42.014 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736601+0000", 2024-09-21T21:42:42.014 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:42.014 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.014 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.014 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:42.014 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736601+0000", 2024-09-21T21:42:42.014 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:42.015 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.015 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.015 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:42.015 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736602+0000", 2024-09-21T21:42:42.015 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0699999999999999e-06 2024-09-21T21:42:42.015 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.015 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.015 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:42.015 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736607+0000", 2024-09-21T21:42:42.015 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2520000000000001e-06 2024-09-21T21:42:42.015 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.015 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.015 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:42.015 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736617+0000", 2024-09-21T21:42:42.015 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0074e-05 2024-09-21T21:42:42.016 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.016 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.016 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:42.016 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736622+0000", 2024-09-21T21:42:42.016 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.101e-06 2024-09-21T21:42:42.016 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.016 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.016 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:42.016 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736664+0000", 2024-09-21T21:42:42.016 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1869999999999997e-05 2024-09-21T21:42:42.016 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.016 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.016 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op must wait for map", 2024-09-21T21:42:42.017 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736667+0000", 2024-09-21T21:42:42.017 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4750000000000002e-06 2024-09-21T21:42:42.017 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.017 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.017 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:42.017 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742886+0000", 2024-09-21T21:42:42.017 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0062187579999999996 2024-09-21T21:42:42.017 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.017 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.017 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:42.017 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742892+0000", 2024-09-21T21:42:42.017 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0440000000000001e-06 2024-09-21T21:42:42.017 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.017 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.017 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:42.018 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210307+0000", 2024-09-21T21:42:42.018 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.467414781 2024-09-21T21:42:42.018 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.018 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.018 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:42.018 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210322+0000", 2024-09-21T21:42:42.018 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5311e-05 2024-09-21T21:42:42.018 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.018 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.018 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:42.018 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.237624+0000", 2024-09-21T21:42:42.018 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0273018920000001 2024-09-21T21:42:42.018 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.018 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.019 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:42.019 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.237870+0000", 2024-09-21T21:42:42.019 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00024622899999999999 2024-09-21T21:42:42.019 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.019 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.019 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:42.019 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.240607+0000", 2024-09-21T21:42:42.019 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0027368200000000001 2024-09-21T21:42:42.019 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:42.019 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:42.019 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:42.019 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.019 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.020 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5415.0:379 3.16 3:6888948b:::rbd_header.1121493f6459:head [watch reconnect cookie 139702602617920 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-21T21:42:42.020 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:41.739544+0000", 2024-09-21T21:42:42.020 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 60.270351933000001, 2024-09-21T21:42:42.020 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.501180645, 2024-09-21T21:42:42.020 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:42.020 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:42.020 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:42.020 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:42.020 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5415", 2024-09-21T21:42:42.020 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1212816607", 2024-09-21T21:42:42.020 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 379 2024-09-21T21:42:42.020 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.020 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:42.020 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.020 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:42.021 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739544+0000", 2024-09-21T21:42:42.021 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:42.021 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.021 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.021 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:42.021 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739544+0000", 2024-09-21T21:42:42.021 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:42.021 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.021 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.021 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:42.021 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739545+0000", 2024-09-21T21:42:42.021 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.065e-06 2024-09-21T21:42:42.021 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.022 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.022 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:42.022 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739550+0000", 2024-09-21T21:42:42.022 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9599999999999999e-06 2024-09-21T21:42:42.022 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.022 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.022 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:42.022 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739565+0000", 2024-09-21T21:42:42.022 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5709000000000002e-05 2024-09-21T21:42:42.022 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.022 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.022 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:42.022 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739573+0000", 2024-09-21T21:42:42.022 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7849999999999992e-06 2024-09-21T21:42:42.023 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.023 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.023 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:42.023 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739634+0000", 2024-09-21T21:42:42.023 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0732000000000001e-05 2024-09-21T21:42:42.023 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.023 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.023 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op must wait for map", 2024-09-21T21:42:42.023 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739640+0000", 2024-09-21T21:42:42.023 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0000000000000002e-06 2024-09-21T21:42:42.023 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.023 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.023 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:42.023 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742958+0000", 2024-09-21T21:42:42.024 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0033182390000000002 2024-09-21T21:42:42.024 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.024 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.024 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:42.024 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742966+0000", 2024-09-21T21:42:42.024 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1610000000000003e-06 2024-09-21T21:42:42.024 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.024 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.024 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:42.024 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210405+0000", 2024-09-21T21:42:42.024 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.467438725 2024-09-21T21:42:42.024 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.024 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.024 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:42.024 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210415+0000", 2024-09-21T21:42:42.024 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0234000000000001e-05 2024-09-21T21:42:42.025 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.025 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.025 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:42.025 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.238743+0000", 2024-09-21T21:42:42.025 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0283273070000001 2024-09-21T21:42:42.025 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.025 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.025 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:42.025 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.238842+0000", 2024-09-21T21:42:42.025 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.9896999999999996e-05 2024-09-21T21:42:42.025 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.025 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.025 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:42.025 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.240724+0000", 2024-09-21T21:42:42.026 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001881831 2024-09-21T21:42:42.026 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:42.026 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:42.026 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:42.026 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.026 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.026 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4385.0:15203 3.16 3:6888948b:::rbd_header.1121493f6459:head [call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-21T21:42:42.026 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:41.739737+0000", 2024-09-21T21:42:42.026 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 60.270158922, 2024-09-21T21:42:42.026 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4989621799999999, 2024-09-21T21:42:42.026 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:42.026 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:42.026 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:42.026 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:42.027 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4385", 2024-09-21T21:42:42.027 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:42:42.027 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 15203 2024-09-21T21:42:42.027 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.027 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:42.027 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.027 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:42.027 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739737+0000", 2024-09-21T21:42:42.027 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:42.027 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.027 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.027 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:42.027 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739737+0000", 2024-09-21T21:42:42.028 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:42.028 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.028 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.028 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:42.028 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739738+0000", 2024-09-21T21:42:42.028 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.7499999999999998e-07 2024-09-21T21:42:42.028 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.028 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.028 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:42.028 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739743+0000", 2024-09-21T21:42:42.028 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5049999999999998e-06 2024-09-21T21:42:42.028 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.028 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.028 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:42.029 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739757+0000", 2024-09-21T21:42:42.029 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3947e-05 2024-09-21T21:42:42.029 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.029 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.029 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:42.029 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739765+0000", 2024-09-21T21:42:42.029 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9310000000000008e-06 2024-09-21T21:42:42.029 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.029 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.029 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:42.029 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739816+0000", 2024-09-21T21:42:42.029 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0559999999999997e-05 2024-09-21T21:42:42.029 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.030 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.030 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting_for_map not empty", 2024-09-21T21:42:42.030 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739821+0000", 2024-09-21T21:42:42.030 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3539999999999999e-06 2024-09-21T21:42:42.030 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.030 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.030 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:42.030 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742928+0000", 2024-09-21T21:42:42.030 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0031074399999999999 2024-09-21T21:42:42.030 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.030 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.030 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:42.030 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742937+0000", 2024-09-21T21:42:42.030 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1289999999999996e-06 2024-09-21T21:42:42.031 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.031 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.031 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:42.031 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210350+0000", 2024-09-21T21:42:42.031 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.46741311299999999 2024-09-21T21:42:42.031 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.031 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.031 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:42.031 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210358+0000", 2024-09-21T21:42:42.031 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1869999999999997e-06 2024-09-21T21:42:42.031 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.031 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.031 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:42.031 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.238398+0000", 2024-09-21T21:42:42.032 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0280397830000001 2024-09-21T21:42:42.032 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.032 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.032 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:42.032 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.238489+0000", 2024-09-21T21:42:42.032 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.0958000000000006e-05 2024-09-21T21:42:42.032 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.032 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.032 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:42.032 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.238699+0000", 2024-09-21T21:42:42.032 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00021029800000000001 2024-09-21T21:42:42.032 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:42.032 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:42.032 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:42.032 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.033 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.033 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4385.0:15204 3.16 3:6888948b:::rbd_header.1121493f6459:head [watch reconnect cookie 140325440202496 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-21T21:42:42.033 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:42.210726+0000", 2024-09-21T21:42:42.033 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 59.799170044, 2024-09-21T21:42:42.033 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.030304557, 2024-09-21T21:42:42.033 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:42.033 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:42.033 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:42.033 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:42.033 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4385", 2024-09-21T21:42:42.033 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:42:42.033 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 15204 2024-09-21T21:42:42.033 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.033 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:42.034 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.034 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:42.034 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210726+0000", 2024-09-21T21:42:42.034 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:42.034 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.034 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.034 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:42.034 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210726+0000", 2024-09-21T21:42:42.034 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:42.034 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.034 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.034 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:42.034 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210727+0000", 2024-09-21T21:42:42.034 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.204e-06 2024-09-21T21:42:42.035 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.035 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.035 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:42.035 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210732+0000", 2024-09-21T21:42:42.035 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5249999999999996e-06 2024-09-21T21:42:42.035 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.035 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.035 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:42.035 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210744+0000", 2024-09-21T21:42:42.035 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1405e-05 2024-09-21T21:42:42.035 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.035 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.035 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:42.035 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210754+0000", 2024-09-21T21:42:42.036 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0122999999999999e-05 2024-09-21T21:42:42.036 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.036 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.036 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:42.036 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210798+0000", 2024-09-21T21:42:42.036 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4005000000000003e-05 2024-09-21T21:42:42.036 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.036 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.036 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:42.036 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210803+0000", 2024-09-21T21:42:42.036 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5130000000000004e-06 2024-09-21T21:42:42.036 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.036 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.036 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:42.036 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.239260+0000", 2024-09-21T21:42:42.037 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0284563040000001 2024-09-21T21:42:42.037 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.037 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.037 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:42.037 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.239318+0000", 2024-09-21T21:42:42.037 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8486000000000002e-05 2024-09-21T21:42:42.037 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.037 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.037 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:42.037 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.241030+0000", 2024-09-21T21:42:42.037 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0017119920000000001 2024-09-21T21:42:42.037 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:42.037 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:42.037 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:42.038 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.038 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.038 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5415.0:380 3.16 3:6888948b:::rbd_header.1121493f6459:head [watch reconnect cookie 139702602617920 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-21T21:42:42.038 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:42.210777+0000", 2024-09-21T21:42:42.038 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 59.799118264999997, 2024-09-21T21:42:42.038 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.030410566, 2024-09-21T21:42:42.038 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:42.038 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:42.038 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:42.038 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:42.038 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5415", 2024-09-21T21:42:42.038 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1212816607", 2024-09-21T21:42:42.038 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 380 2024-09-21T21:42:42.039 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.039 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:42.039 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.039 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:42.039 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210777+0000", 2024-09-21T21:42:42.039 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:42.039 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.039 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.039 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:42.039 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210777+0000", 2024-09-21T21:42:42.039 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:42.039 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.039 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.039 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:42.040 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210778+0000", 2024-09-21T21:42:42.040 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8999999999999996e-07 2024-09-21T21:42:42.040 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.040 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.040 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:42.040 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210780+0000", 2024-09-21T21:42:42.040 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.503e-06 2024-09-21T21:42:42.040 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.040 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.040 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:42.040 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210788+0000", 2024-09-21T21:42:42.040 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0999999999999998e-06 2024-09-21T21:42:42.040 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.040 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.041 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:42.041 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210799+0000", 2024-09-21T21:42:42.041 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1703e-05 2024-09-21T21:42:42.041 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.041 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.041 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:42.041 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210833+0000", 2024-09-21T21:42:42.041 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4008999999999999e-05 2024-09-21T21:42:42.041 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.041 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.041 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:42.041 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210838+0000", 2024-09-21T21:42:42.041 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7480000000000001e-06 2024-09-21T21:42:42.041 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.041 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.042 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:42.042 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.239612+0000", 2024-09-21T21:42:42.042 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.028773554 2024-09-21T21:42:42.042 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.042 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.042 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:42.042 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.239677+0000", 2024-09-21T21:42:42.042 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5053000000000002e-05 2024-09-21T21:42:42.042 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.042 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.042 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:42.042 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.241188+0000", 2024-09-21T21:42:42.042 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001511306 2024-09-21T21:42:42.042 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:42.043 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:42.043 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:42.043 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.043 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.043 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24794 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:42.043 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.584085+0000", 2024-09-21T21:42:42.043 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 19.425811089, 2024-09-21T21:42:42.043 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.096209996000000006, 2024-09-21T21:42:42.043 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:42.043 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:42.043 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:42.043 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:42.043 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:42:42.043 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:42.044 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24794 2024-09-21T21:42:42.044 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.044 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:42.044 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.044 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:42.044 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584085+0000", 2024-09-21T21:42:42.044 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:42.044 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.044 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.044 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:42.044 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584085+0000", 2024-09-21T21:42:42.044 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:42.044 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.044 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.045 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:42.045 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584085+0000", 2024-09-21T21:42:42.045 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3200000000000001e-07 2024-09-21T21:42:42.045 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.045 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.045 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:42.045 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584086+0000", 2024-09-21T21:42:42.045 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.717e-06 2024-09-21T21:42:42.045 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.045 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.045 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:42.045 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584090+0000", 2024-09-21T21:42:42.045 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3450000000000002e-06 2024-09-21T21:42:42.045 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.045 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.046 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:42.046 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584092+0000", 2024-09-21T21:42:42.046 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6359999999999998e-06 2024-09-21T21:42:42.046 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.046 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.046 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:42.046 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.676497+0000", 2024-09-21T21:42:42.046 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.092404659 2024-09-21T21:42:42.046 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.046 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.046 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:42.046 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.676567+0000", 2024-09-21T21:42:42.046 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9627000000000003e-05 2024-09-21T21:42:42.046 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.047 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.047 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:42:42.047 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.676925+0000", 2024-09-21T21:42:42.047 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00035848500000000002 2024-09-21T21:42:42.047 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.047 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.047 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:42.047 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.678427+0000", 2024-09-21T21:42:42.047 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001501638 2024-09-21T21:42:42.047 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.047 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.047 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:42.047 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.680204+0000", 2024-09-21T21:42:42.047 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0017766979999999999 2024-09-21T21:42:42.047 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.048 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.048 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:42.048 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.680256+0000", 2024-09-21T21:42:42.048 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2549999999999997e-05 2024-09-21T21:42:42.048 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.048 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.048 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:42.048 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.680295+0000", 2024-09-21T21:42:42.048 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8408999999999998e-05 2024-09-21T21:42:42.048 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:42.048 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:42.048 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:42.048 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.048 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.049 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24796 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:42.049 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.584099+0000", 2024-09-21T21:42:42.049 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 19.425796519999999, 2024-09-21T21:42:42.049 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.097297429000000005, 2024-09-21T21:42:42.049 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:42.049 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:42.049 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:42.049 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:42.049 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:42:42.049 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:42.049 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24796 2024-09-21T21:42:42.049 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.049 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:42.049 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.050 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:42.050 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584099+0000", 2024-09-21T21:42:42.050 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:42.050 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.050 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.050 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:42.050 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584099+0000", 2024-09-21T21:42:42.050 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:42.050 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.050 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.050 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:42.050 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584099+0000", 2024-09-21T21:42:42.050 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3999999999999998e-07 2024-09-21T21:42:42.050 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.051 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.051 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:42.051 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584101+0000", 2024-09-21T21:42:42.051 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5850000000000001e-06 2024-09-21T21:42:42.051 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.051 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.051 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:42.051 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584108+0000", 2024-09-21T21:42:42.051 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.498e-06 2024-09-21T21:42:42.051 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.051 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.051 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:42.051 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584112+0000", 2024-09-21T21:42:42.051 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.252e-06 2024-09-21T21:42:42.052 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.052 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.052 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:42.052 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.677336+0000", 2024-09-21T21:42:42.052 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.093224206000000004 2024-09-21T21:42:42.052 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.052 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.052 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:42.052 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.677411+0000", 2024-09-21T21:42:42.052 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5036000000000005e-05 2024-09-21T21:42:42.052 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.052 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.052 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:42:42.052 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.677782+0000", 2024-09-21T21:42:42.053 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00037153099999999998 2024-09-21T21:42:42.053 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.053 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.053 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:42.053 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.680360+0000", 2024-09-21T21:42:42.053 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025773020000000001 2024-09-21T21:42:42.053 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.053 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.053 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:42.053 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681313+0000", 2024-09-21T21:42:42.053 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00095341100000000004 2024-09-21T21:42:42.053 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.053 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.053 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:42.054 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681361+0000", 2024-09-21T21:42:42.054 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7497999999999998e-05 2024-09-21T21:42:42.054 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.054 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.054 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:42.054 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681397+0000", 2024-09-21T21:42:42.054 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.587e-05 2024-09-21T21:42:42.054 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:42.054 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:42.054 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:42.054 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.054 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.055 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24797 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:42.055 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.584116+0000", 2024-09-21T21:42:42.055 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 19.425780030999999, 2024-09-21T21:42:42.055 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.097428257000000004, 2024-09-21T21:42:42.055 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:42.055 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:42.055 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:42.055 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:42.055 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:42:42.055 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:42.055 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24797 2024-09-21T21:42:42.055 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.055 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:42.055 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.056 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:42.056 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584116+0000", 2024-09-21T21:42:42.056 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:42.056 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.056 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.056 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:42.056 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584116+0000", 2024-09-21T21:42:42.056 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:42.056 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.056 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.056 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:42.056 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584116+0000", 2024-09-21T21:42:42.056 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.53e-07 2024-09-21T21:42:42.057 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.057 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.057 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:42.057 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584118+0000", 2024-09-21T21:42:42.057 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6810000000000001e-06 2024-09-21T21:42:42.057 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.057 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.057 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:42.057 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584121+0000", 2024-09-21T21:42:42.057 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1099999999999999e-06 2024-09-21T21:42:42.057 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.057 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.057 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:42.057 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584123+0000", 2024-09-21T21:42:42.058 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5890000000000001e-06 2024-09-21T21:42:42.058 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.058 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.058 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:42.058 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.678476+0000", 2024-09-21T21:42:42.058 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.094352320000000003 2024-09-21T21:42:42.058 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.058 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.058 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:42.058 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.678556+0000", 2024-09-21T21:42:42.058 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0291999999999996e-05 2024-09-21T21:42:42.058 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.058 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.058 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:42:42.059 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.678871+0000", 2024-09-21T21:42:42.059 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000314689 2024-09-21T21:42:42.059 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.059 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.059 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:42.059 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681452+0000", 2024-09-21T21:42:42.059 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025817890000000001 2024-09-21T21:42:42.059 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.059 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.059 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:42.059 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681467+0000", 2024-09-21T21:42:42.059 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4256e-05 2024-09-21T21:42:42.059 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.059 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.060 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:42.060 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681510+0000", 2024-09-21T21:42:42.060 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3560999999999998e-05 2024-09-21T21:42:42.060 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.060 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.060 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:42.060 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681544+0000", 2024-09-21T21:42:42.060 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3717000000000003e-05 2024-09-21T21:42:42.060 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:42.060 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:42.060 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:42.060 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.060 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.060 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24798 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:42.060 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.584130+0000", 2024-09-21T21:42:42.061 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 19.425765508000001, 2024-09-21T21:42:42.061 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.098350526999999993, 2024-09-21T21:42:42.061 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:42.061 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:42.061 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:42.061 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:42.061 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:42:42.061 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:42.061 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24798 2024-09-21T21:42:42.061 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.061 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:42.061 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.061 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:42.061 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584130+0000", 2024-09-21T21:42:42.062 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:42.062 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.062 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.062 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:42.062 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584130+0000", 2024-09-21T21:42:42.062 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:42.062 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.062 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.062 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:42.062 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584131+0000", 2024-09-21T21:42:42.062 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4700000000000001e-07 2024-09-21T21:42:42.062 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.062 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.062 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:42.063 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584133+0000", 2024-09-21T21:42:42.063 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.34e-06 2024-09-21T21:42:42.063 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.063 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.063 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:42.063 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584137+0000", 2024-09-21T21:42:42.063 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5609999999999999e-06 2024-09-21T21:42:42.063 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.063 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.063 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:42.063 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584139+0000", 2024-09-21T21:42:42.063 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7350000000000001e-06 2024-09-21T21:42:42.063 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.063 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.063 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:42.064 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.679274+0000", 2024-09-21T21:42:42.064 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.095134719000000006 2024-09-21T21:42:42.064 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.064 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.064 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:42.064 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.679358+0000", 2024-09-21T21:42:42.064 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3609999999999994e-05 2024-09-21T21:42:42.064 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.064 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.064 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:42:42.064 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.679685+0000", 2024-09-21T21:42:42.064 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00032732399999999999 2024-09-21T21:42:42.064 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.064 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.064 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:42.065 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681553+0000", 2024-09-21T21:42:42.065 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001868059 2024-09-21T21:42:42.065 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.065 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.065 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:42.065 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.682375+0000", 2024-09-21T21:42:42.065 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00082232099999999999 2024-09-21T21:42:42.065 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.065 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.065 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:42.065 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.682431+0000", 2024-09-21T21:42:42.065 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6051999999999998e-05 2024-09-21T21:42:42.065 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.065 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.065 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:42.065 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.682481+0000", 2024-09-21T21:42:42.066 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9258999999999997e-05 2024-09-21T21:42:42.066 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:42.066 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:42.066 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:42.066 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.066 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.066 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24799 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:42.066 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.584159+0000", 2024-09-21T21:42:42.066 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 19.425736520000001, 2024-09-21T21:42:42.066 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.100021551, 2024-09-21T21:42:42.066 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:42.066 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:42.066 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:42.066 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:42.066 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:42:42.067 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:42.067 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24799 2024-09-21T21:42:42.067 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.067 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:42.067 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.067 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:42.067 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584159+0000", 2024-09-21T21:42:42.067 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:42.067 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.067 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.067 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:42.067 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584159+0000", 2024-09-21T21:42:42.067 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:42.068 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.068 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.068 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:42.068 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584160+0000", 2024-09-21T21:42:42.068 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1300000000000001e-07 2024-09-21T21:42:42.068 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.068 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.068 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:42.068 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584162+0000", 2024-09-21T21:42:42.068 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5409999999999999e-06 2024-09-21T21:42:42.068 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.068 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.068 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:42.068 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584169+0000", 2024-09-21T21:42:42.069 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7880000000000001e-06 2024-09-21T21:42:42.069 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.069 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.069 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:42.069 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584174+0000", 2024-09-21T21:42:42.069 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9590000000000003e-06 2024-09-21T21:42:42.069 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.069 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.069 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:42.069 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.680400+0000", 2024-09-21T21:42:42.069 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.096226514999999999 2024-09-21T21:42:42.069 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.069 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.069 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:42.069 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.680480+0000", 2024-09-21T21:42:42.070 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9589e-05 2024-09-21T21:42:42.070 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.070 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.070 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:42:42.070 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.680847+0000", 2024-09-21T21:42:42.070 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00036757700000000001 2024-09-21T21:42:42.070 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.070 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.070 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:42.070 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.682555+0000", 2024-09-21T21:42:42.070 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001707791 2024-09-21T21:42:42.070 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.070 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.070 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:42.070 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684093+0000", 2024-09-21T21:42:42.071 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0015377170000000001 2024-09-21T21:42:42.071 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.071 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.071 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:42.071 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684144+0000", 2024-09-21T21:42:42.071 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1208000000000002e-05 2024-09-21T21:42:42.071 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.071 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.071 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:42.071 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684181+0000", 2024-09-21T21:42:42.071 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6452999999999998e-05 2024-09-21T21:42:42.071 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:42.071 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:42.071 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:42.071 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.072 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.072 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24835 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:42.072 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586332+0000", 2024-09-21T21:42:42.072 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 19.423563905999998, 2024-09-21T21:42:42.072 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.098730360000000003, 2024-09-21T21:42:42.072 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:42.072 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:42.072 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:42.072 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:42.072 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:42:42.072 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:42.072 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24835 2024-09-21T21:42:42.072 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.072 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:42.072 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.073 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:42.073 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586332+0000", 2024-09-21T21:42:42.073 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:42.073 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.073 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.073 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:42.073 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586332+0000", 2024-09-21T21:42:42.073 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:42.073 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.073 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.073 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:42.073 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586333+0000", 2024-09-21T21:42:42.073 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0200000000000001e-07 2024-09-21T21:42:42.073 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.073 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.074 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:42.074 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586346+0000", 2024-09-21T21:42:42.074 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3745e-05 2024-09-21T21:42:42.074 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.074 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.074 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:42.074 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586366+0000", 2024-09-21T21:42:42.074 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9272000000000001e-05 2024-09-21T21:42:42.074 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.074 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.074 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:42.074 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586377+0000", 2024-09-21T21:42:42.074 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1197e-05 2024-09-21T21:42:42.074 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.075 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.075 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:42.075 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681578+0000", 2024-09-21T21:42:42.075 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.095201420999999994 2024-09-21T21:42:42.075 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.075 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.075 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:42.075 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681640+0000", 2024-09-21T21:42:42.075 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1834000000000005e-05 2024-09-21T21:42:42.075 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.075 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.075 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:42:42.075 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681915+0000", 2024-09-21T21:42:42.075 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00027534199999999998 2024-09-21T21:42:42.075 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.076 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.076 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:42.076 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684230+0000", 2024-09-21T21:42:42.076 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00231424 2024-09-21T21:42:42.076 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.076 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.076 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:42.076 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684959+0000", 2024-09-21T21:42:42.076 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00072921399999999995 2024-09-21T21:42:42.076 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.076 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.076 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:42.076 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.685008+0000", 2024-09-21T21:42:42.076 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8860999999999999e-05 2024-09-21T21:42:42.076 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.077 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.077 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:42.077 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.685062+0000", 2024-09-21T21:42:42.077 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4432000000000003e-05 2024-09-21T21:42:42.077 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:42.077 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:42.077 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:42.077 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.077 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.077 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24837 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:42.077 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586395+0000", 2024-09-21T21:42:42.077 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 19.423500970999999, 2024-09-21T21:42:42.077 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.099700522, 2024-09-21T21:42:42.077 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:42.078 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:42.078 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:42.078 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:42.078 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:42:42.078 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:42.078 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24837 2024-09-21T21:42:42.078 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.078 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:42.078 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.078 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:42.078 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586395+0000", 2024-09-21T21:42:42.078 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:42.078 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.078 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.079 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:42.079 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586395+0000", 2024-09-21T21:42:42.079 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:42.079 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.079 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.079 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:42.079 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586395+0000", 2024-09-21T21:42:42.079 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.06e-07 2024-09-21T21:42:42.079 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.079 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.079 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:42.079 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586401+0000", 2024-09-21T21:42:42.079 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0889999999999996e-06 2024-09-21T21:42:42.079 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.080 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.080 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:42.080 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586410+0000", 2024-09-21T21:42:42.080 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.918e-06 2024-09-21T21:42:42.080 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.080 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.080 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:42.080 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586424+0000", 2024-09-21T21:42:42.080 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3478999999999999e-05 2024-09-21T21:42:42.080 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.080 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.080 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:42.080 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.682590+0000", 2024-09-21T21:42:42.081 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.096166086999999997 2024-09-21T21:42:42.081 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.081 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.081 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:42.081 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.682657+0000", 2024-09-21T21:42:42.081 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6878000000000006e-05 2024-09-21T21:42:42.081 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.081 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.081 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:42:42.081 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.682879+0000", 2024-09-21T21:42:42.081 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00022238 2024-09-21T21:42:42.081 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.081 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.081 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:42.082 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.685073+0000", 2024-09-21T21:42:42.082 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021938119999999998 2024-09-21T21:42:42.082 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.082 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.082 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:42.082 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686038+0000", 2024-09-21T21:42:42.082 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00096539499999999997 2024-09-21T21:42:42.082 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.082 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.082 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:42.082 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686071+0000", 2024-09-21T21:42:42.082 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2514000000000001e-05 2024-09-21T21:42:42.082 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.082 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.083 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:42.083 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686095+0000", 2024-09-21T21:42:42.083 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4363999999999999e-05 2024-09-21T21:42:42.083 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:42.083 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:42.083 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:42.083 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.083 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.083 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24839 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:42.083 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586468+0000", 2024-09-21T21:42:42.083 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 19.423427319000002, 2024-09-21T21:42:42.083 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.099739355000000002, 2024-09-21T21:42:42.083 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:42.084 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:42.084 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:42.084 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:42.084 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:42:42.084 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:42.084 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24839 2024-09-21T21:42:42.084 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.084 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:42.084 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.084 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:42.084 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586468+0000", 2024-09-21T21:42:42.084 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:42.084 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.084 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.085 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:42.085 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586468+0000", 2024-09-21T21:42:42.085 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:42.085 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.085 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.085 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:42.085 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586469+0000", 2024-09-21T21:42:42.085 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2200000000000004e-07 2024-09-21T21:42:42.085 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.085 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.085 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:42.085 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586472+0000", 2024-09-21T21:42:42.085 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9960000000000002e-06 2024-09-21T21:42:42.085 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.086 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.086 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:42.086 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586479+0000", 2024-09-21T21:42:42.086 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2699999999999999e-06 2024-09-21T21:42:42.086 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.086 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.086 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:42.086 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586485+0000", 2024-09-21T21:42:42.086 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6999999999999996e-06 2024-09-21T21:42:42.086 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.086 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.086 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:42.086 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.683276+0000", 2024-09-21T21:42:42.086 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.096791618999999995 2024-09-21T21:42:42.087 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.087 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.087 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:42.087 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.683326+0000", 2024-09-21T21:42:42.087 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9384e-05 2024-09-21T21:42:42.087 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.087 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.087 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:42:42.087 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.683655+0000", 2024-09-21T21:42:42.087 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00032965700000000002 2024-09-21T21:42:42.087 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.087 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.087 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:42.087 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686120+0000", 2024-09-21T21:42:42.087 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002464819 2024-09-21T21:42:42.088 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.088 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.088 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:42.088 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686159+0000", 2024-09-21T21:42:42.088 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9181000000000003e-05 2024-09-21T21:42:42.088 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.088 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.088 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:42.088 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686185+0000", 2024-09-21T21:42:42.088 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5907000000000001e-05 2024-09-21T21:42:42.088 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.088 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.088 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:42.088 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686208+0000", 2024-09-21T21:42:42.089 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.23e-05 2024-09-21T21:42:42.089 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:42.089 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:42.089 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:42.089 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.089 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.089 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24841 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:42.089 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586492+0000", 2024-09-21T21:42:42.089 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 19.423403156999999, 2024-09-21T21:42:42.089 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.101672077, 2024-09-21T21:42:42.089 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:42.089 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:42.089 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:42.089 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:42.090 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:42:42.090 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:42.090 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24841 2024-09-21T21:42:42.090 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.090 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:42.090 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.090 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:42.090 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586492+0000", 2024-09-21T21:42:42.090 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:42.090 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.090 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.090 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:42.090 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586492+0000", 2024-09-21T21:42:42.090 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:42.091 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.091 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.091 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:42.091 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586493+0000", 2024-09-21T21:42:42.091 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5999999999999999e-07 2024-09-21T21:42:42.091 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.091 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.091 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:42.091 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586496+0000", 2024-09-21T21:42:42.091 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6089999999999999e-06 2024-09-21T21:42:42.091 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.091 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.091 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:42.091 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586502+0000", 2024-09-21T21:42:42.092 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6270000000000002e-06 2024-09-21T21:42:42.092 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.092 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.092 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:42.092 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586507+0000", 2024-09-21T21:42:42.092 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8810000000000004e-06 2024-09-21T21:42:42.092 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.092 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.092 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:42.092 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684273+0000", 2024-09-21T21:42:42.092 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.097766415999999995 2024-09-21T21:42:42.092 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.092 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.092 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:42.093 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684359+0000", 2024-09-21T21:42:42.093 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5574000000000004e-05 2024-09-21T21:42:42.093 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.093 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.093 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:42:42.093 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684568+0000", 2024-09-21T21:42:42.093 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000208529 2024-09-21T21:42:42.093 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.093 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.093 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:42.093 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.687186+0000", 2024-09-21T21:42:42.093 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0026180769999999999 2024-09-21T21:42:42.093 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.093 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.094 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:42.094 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688093+0000", 2024-09-21T21:42:42.094 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00090766999999999996 2024-09-21T21:42:42.094 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.094 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.094 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:42.094 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688142+0000", 2024-09-21T21:42:42.094 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8986000000000002e-05 2024-09-21T21:42:42.094 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.094 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.094 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:42.094 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688165+0000", 2024-09-21T21:42:42.094 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2248000000000001e-05 2024-09-21T21:42:42.095 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:42.095 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:42.095 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:42.095 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.095 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.095 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24843 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:42.095 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586514+0000", 2024-09-21T21:42:42.095 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 19.423381871, 2024-09-21T21:42:42.095 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.101826449, 2024-09-21T21:42:42.095 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:42.095 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:42.095 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:42.095 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:42.095 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:42:42.096 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:42.096 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24843 2024-09-21T21:42:42.096 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.096 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:42.096 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.096 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:42.096 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586514+0000", 2024-09-21T21:42:42.096 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:42.096 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.096 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.096 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:42.096 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586514+0000", 2024-09-21T21:42:42.096 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:42.096 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.097 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.097 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:42.097 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586514+0000", 2024-09-21T21:42:42.097 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9999999999999998e-07 2024-09-21T21:42:42.097 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.097 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.097 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:42.097 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586517+0000", 2024-09-21T21:42:42.097 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4980000000000001e-06 2024-09-21T21:42:42.097 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.097 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.097 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:42.097 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586523+0000", 2024-09-21T21:42:42.097 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4609999999999996e-06 2024-09-21T21:42:42.098 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.098 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.098 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:42.098 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586528+0000", 2024-09-21T21:42:42.098 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2979999999999999e-06 2024-09-21T21:42:42.098 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.098 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.098 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:42.098 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.685104+0000", 2024-09-21T21:42:42.098 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.098575837999999999 2024-09-21T21:42:42.098 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.098 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.098 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:42.098 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.685184+0000", 2024-09-21T21:42:42.098 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9807000000000003e-05 2024-09-21T21:42:42.099 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.099 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.099 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:42:42.099 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.685496+0000", 2024-09-21T21:42:42.099 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00031225900000000001 2024-09-21T21:42:42.099 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.099 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.099 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:42.099 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688238+0000", 2024-09-21T21:42:42.099 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0027419409999999999 2024-09-21T21:42:42.099 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.099 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.099 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:42.099 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688253+0000", 2024-09-21T21:42:42.100 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4331000000000001e-05 2024-09-21T21:42:42.100 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.100 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.100 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:42.100 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688296+0000", 2024-09-21T21:42:42.100 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3801000000000003e-05 2024-09-21T21:42:42.100 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.100 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.100 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:42.100 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688340+0000", 2024-09-21T21:42:42.100 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3815e-05 2024-09-21T21:42:42.100 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:42.100 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:42.100 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:42.101 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.101 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.101 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24845 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:42.101 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586535+0000", 2024-09-21T21:42:42.101 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 19.423360990999999, 2024-09-21T21:42:42.101 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.10285678600000001, 2024-09-21T21:42:42.101 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:42.101 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:42.101 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:42.101 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:42.101 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:42:42.101 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:42.101 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24845 2024-09-21T21:42:42.101 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.102 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:42.102 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.102 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:42.102 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586535+0000", 2024-09-21T21:42:42.102 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:42.102 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.102 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.102 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:42.102 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586535+0000", 2024-09-21T21:42:42.102 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:42.102 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.102 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.102 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:42.102 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586535+0000", 2024-09-21T21:42:42.103 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9000000000000002e-07 2024-09-21T21:42:42.103 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.103 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.103 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:42.103 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586537+0000", 2024-09-21T21:42:42.103 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4569999999999999e-06 2024-09-21T21:42:42.103 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.103 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.103 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:42.103 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586544+0000", 2024-09-21T21:42:42.103 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5849999999999996e-06 2024-09-21T21:42:42.103 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.103 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.103 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:42.104 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586549+0000", 2024-09-21T21:42:42.104 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0089999999999998e-06 2024-09-21T21:42:42.104 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.104 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.104 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:42.104 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686239+0000", 2024-09-21T21:42:42.104 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.099689531999999997 2024-09-21T21:42:42.104 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.104 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.104 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:42.104 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686289+0000", 2024-09-21T21:42:42.104 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.007e-05 2024-09-21T21:42:42.104 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.104 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.104 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:42:42.105 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686633+0000", 2024-09-21T21:42:42.105 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00034428699999999998 2024-09-21T21:42:42.105 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.105 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.105 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:42.105 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688370+0000", 2024-09-21T21:42:42.105 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001736741 2024-09-21T21:42:42.105 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.105 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.105 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:42.105 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.689295+0000", 2024-09-21T21:42:42.105 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00092517900000000004 2024-09-21T21:42:42.105 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.105 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.106 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:42.106 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.689354+0000", 2024-09-21T21:42:42.106 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8799999999999999e-05 2024-09-21T21:42:42.106 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.106 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.106 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:42.106 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.689391+0000", 2024-09-21T21:42:42.106 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7735999999999999e-05 2024-09-21T21:42:42.106 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:42.106 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:42.106 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:42.106 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.106 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.106 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24846 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:42.107 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586564+0000", 2024-09-21T21:42:42.107 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 19.423331962999999, 2024-09-21T21:42:42.107 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.10375692, 2024-09-21T21:42:42.107 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:42.107 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:42.107 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:42.107 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:42.107 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:42:42.107 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:42.107 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24846 2024-09-21T21:42:42.107 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.107 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:42.107 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.107 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:42.108 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586564+0000", 2024-09-21T21:42:42.108 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:42.108 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.108 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.108 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:42.108 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586564+0000", 2024-09-21T21:42:42.108 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:42.108 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.108 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.108 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:42.108 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586564+0000", 2024-09-21T21:42:42.108 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4799999999999998e-07 2024-09-21T21:42:42.108 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.108 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.109 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:42.109 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586567+0000", 2024-09-21T21:42:42.109 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1669999999999999e-06 2024-09-21T21:42:42.109 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.109 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.109 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:42.109 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586578+0000", 2024-09-21T21:42:42.109 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0443e-05 2024-09-21T21:42:42.109 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.109 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.109 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:42.109 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586583+0000", 2024-09-21T21:42:42.109 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6790000000000002e-06 2024-09-21T21:42:42.109 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.109 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.110 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:42.110 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.687215+0000", 2024-09-21T21:42:42.110 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.100631973 2024-09-21T21:42:42.110 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.110 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.110 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:42.110 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.687311+0000", 2024-09-21T21:42:42.110 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5764e-05 2024-09-21T21:42:42.110 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.110 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.110 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:42:42.110 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.687663+0000", 2024-09-21T21:42:42.110 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00035167 2024-09-21T21:42:42.110 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.111 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.111 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:42.111 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690236+0000", 2024-09-21T21:42:42.111 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025733370000000002 2024-09-21T21:42:42.111 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.111 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.111 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:42.111 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690250+0000", 2024-09-21T21:42:42.111 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3548e-05 2024-09-21T21:42:42.111 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.111 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.111 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:42.111 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690298+0000", 2024-09-21T21:42:42.111 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7948999999999998e-05 2024-09-21T21:42:42.111 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.112 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.112 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:42.112 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690321+0000", 2024-09-21T21:42:42.112 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2841999999999999e-05 2024-09-21T21:42:42.112 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:42.112 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:42.112 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:42.112 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.112 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.112 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24849 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:42.112 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586633+0000", 2024-09-21T21:42:42.112 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 19.423262297000001, 2024-09-21T21:42:42.112 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.104506548, 2024-09-21T21:42:42.112 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:42.113 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:42.113 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:42.113 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:42.113 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:42:42.113 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:42.113 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24849 2024-09-21T21:42:42.113 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.113 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:42.113 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.113 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:42.113 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586633+0000", 2024-09-21T21:42:42.113 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:42.113 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.113 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.113 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:42.114 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586633+0000", 2024-09-21T21:42:42.114 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:42.114 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.114 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.114 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:42.114 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586634+0000", 2024-09-21T21:42:42.114 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9100000000000004e-07 2024-09-21T21:42:42.114 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.114 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.114 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:42.114 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586637+0000", 2024-09-21T21:42:42.114 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1999999999999999e-06 2024-09-21T21:42:42.114 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.114 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.114 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:42.115 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586645+0000", 2024-09-21T21:42:42.115 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7759999999999997e-06 2024-09-21T21:42:42.115 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.115 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.115 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:42.115 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586652+0000", 2024-09-21T21:42:42.115 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3560000000000004e-06 2024-09-21T21:42:42.115 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.115 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.115 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:42.115 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688395+0000", 2024-09-21T21:42:42.115 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.101743239 2024-09-21T21:42:42.115 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.115 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.116 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:42.116 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688464+0000", 2024-09-21T21:42:42.116 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8273000000000002e-05 2024-09-21T21:42:42.116 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.116 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.116 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:42:42.116 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688786+0000", 2024-09-21T21:42:42.116 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000322064 2024-09-21T21:42:42.116 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.116 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.116 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:42.116 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690364+0000", 2024-09-21T21:42:42.116 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001578084 2024-09-21T21:42:42.116 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.116 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.117 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:42.117 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.691050+0000", 2024-09-21T21:42:42.117 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00068623399999999998 2024-09-21T21:42:42.117 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.117 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.117 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:42.117 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.691090+0000", 2024-09-21T21:42:42.117 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9406999999999997e-05 2024-09-21T21:42:42.117 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.117 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.117 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:42.117 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.691140+0000", 2024-09-21T21:42:42.117 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0324000000000001e-05 2024-09-21T21:42:42.117 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:42.118 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:42.118 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:42.118 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.118 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.118 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24851 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:42.118 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586661+0000", 2024-09-21T21:42:42.118 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 19.423235010999999, 2024-09-21T21:42:42.118 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.10536891399999999, 2024-09-21T21:42:42.118 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:42.118 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:42.118 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:42.118 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:42.118 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:42:42.118 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:42.119 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24851 2024-09-21T21:42:42.119 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.119 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:42.119 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.119 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:42.119 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586661+0000", 2024-09-21T21:42:42.119 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:42.119 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.119 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.119 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:42.119 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586661+0000", 2024-09-21T21:42:42.119 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:42.119 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.119 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.120 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:42.120 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586661+0000", 2024-09-21T21:42:42.120 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4700000000000002e-07 2024-09-21T21:42:42.120 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.120 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.120 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:42.120 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586664+0000", 2024-09-21T21:42:42.120 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.672e-06 2024-09-21T21:42:42.120 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.120 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.120 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:42.120 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586671+0000", 2024-09-21T21:42:42.120 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9959999999999996e-06 2024-09-21T21:42:42.120 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.120 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.121 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:42.121 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586676+0000", 2024-09-21T21:42:42.121 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4020000000000001e-06 2024-09-21T21:42:42.121 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.121 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.121 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:42.121 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.689438+0000", 2024-09-21T21:42:42.121 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.102761503 2024-09-21T21:42:42.121 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.121 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.121 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:42.121 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.689524+0000", 2024-09-21T21:42:42.121 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6383000000000003e-05 2024-09-21T21:42:42.121 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.122 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.122 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:42:42.122 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.689769+0000", 2024-09-21T21:42:42.122 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00024504699999999997 2024-09-21T21:42:42.122 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.122 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.122 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:42.122 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.691936+0000", 2024-09-21T21:42:42.122 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021665130000000001 2024-09-21T21:42:42.122 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.122 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.122 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:42.122 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.691947+0000", 2024-09-21T21:42:42.122 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1198e-05 2024-09-21T21:42:42.123 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.123 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.123 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:42.123 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.691992+0000", 2024-09-21T21:42:42.123 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4774000000000002e-05 2024-09-21T21:42:42.123 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.123 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.123 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:42.123 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.692030+0000", 2024-09-21T21:42:42.123 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7978999999999997e-05 2024-09-21T21:42:42.123 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:42.123 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:42.123 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:42.123 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.124 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.124 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24853 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:42.124 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586684+0000", 2024-09-21T21:42:42.124 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 19.423212054, 2024-09-21T21:42:42.124 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.105936852, 2024-09-21T21:42:42.124 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:42.124 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:42.124 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:42.124 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:42.124 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:42:42.124 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:42.124 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24853 2024-09-21T21:42:42.124 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.124 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:42.125 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.125 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:42.125 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586684+0000", 2024-09-21T21:42:42.125 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:42.125 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.125 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.125 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:42.125 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586684+0000", 2024-09-21T21:42:42.125 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:42.125 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.125 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.125 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:42.125 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586684+0000", 2024-09-21T21:42:42.125 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.68e-07 2024-09-21T21:42:42.126 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.126 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.126 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:42.126 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586687+0000", 2024-09-21T21:42:42.126 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.774e-06 2024-09-21T21:42:42.126 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.126 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.126 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:42.126 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586694+0000", 2024-09-21T21:42:42.126 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6959999999999999e-06 2024-09-21T21:42:42.126 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.126 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.126 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:42.126 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586698+0000", 2024-09-21T21:42:42.127 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7489999999999998e-06 2024-09-21T21:42:42.127 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.127 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.127 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:42.127 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690395+0000", 2024-09-21T21:42:42.127 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.103696645 2024-09-21T21:42:42.127 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.127 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.127 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:42.127 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690440+0000", 2024-09-21T21:42:42.127 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4835e-05 2024-09-21T21:42:42.127 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.127 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.127 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:42:42.128 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690654+0000", 2024-09-21T21:42:42.128 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000214114 2024-09-21T21:42:42.128 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.128 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.128 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:42.128 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.692041+0000", 2024-09-21T21:42:42.128 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0013866519999999999 2024-09-21T21:42:42.128 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.128 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.128 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:42.128 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.692558+0000", 2024-09-21T21:42:42.128 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00051717199999999999 2024-09-21T21:42:42.128 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.129 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.129 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:42.129 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.692586+0000", 2024-09-21T21:42:42.129 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8504e-05 2024-09-21T21:42:42.129 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:42.129 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:42.129 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:42.129 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.692620+0000", 2024-09-21T21:42:42.129 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4143000000000002e-05 2024-09-21T21:42:42.129 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:42.129 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:42.129 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:42.129 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:42.129 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:42.129 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:42.131 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:42.129+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:42.131 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T21:42:42.232 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:42.231+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:42.235 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:42.235 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:42:42.235 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 0 2024-09-21T21:42:42.235 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:42.243 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T21:42:42.272 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T21:42:42.333 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:42.331+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:42.349 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:42.349 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:42:42.349 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:42:42.349 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:42:42.349 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:42.358 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T21:42:42.406 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0: {} 2024-09-21T21:42:42.406 INFO:teuthology.orchestra.run.smithi081.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T21:42:42.424 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1: {} 2024-09-21T21:42:42.424 INFO:teuthology.orchestra.run.smithi081.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T21:42:42.433 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:42.432+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:42.455 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2: {} 2024-09-21T21:42:42.456 INFO:teuthology.orchestra.run.smithi081.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T21:42:42.473 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3: {} 2024-09-21T21:42:42.473 INFO:teuthology.orchestra.run.smithi081.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T21:42:42.489 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4: {} 2024-09-21T21:42:42.490 INFO:teuthology.orchestra.run.smithi081.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T21:42:42.506 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5: {} 2024-09-21T21:42:42.506 INFO:teuthology.orchestra.run.smithi081.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T21:42:42.514 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:42.515 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:42.524 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T21:42:42.534 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:42.533+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:42.607 INFO:teuthology.orchestra.run.smithi081.stderr:noscrub is unset 2024-09-21T21:42:42.635 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:42.634+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:42.679 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:42.679 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:42.690 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T21:42:42.737 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:42.735+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:42.797 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:42.798 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:42.807 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T21:42:42.838 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:42.837+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:42.907 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:42.908 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:42.917 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T21:42:42.961 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:42.960+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:43.019 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:43.019 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:43.028 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T21:42:43.040 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:43.039+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:43.141 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:43.140+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:43.171 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:43.171 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:43.180 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T21:42:43.264 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:43.263+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:43.323 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:43.324 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:43.332 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T21:42:43.376 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:43.375+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:43.442 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:43.442 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:43.443 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:43.443+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:43.451 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T21:42:43.544 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:43.543+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:43.595 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:43.595 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:43.604 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T21:42:43.646 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:43.645+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:43.747 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:43.745+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:43.753 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:43.754 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:43.763 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T21:42:43.850 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:43.849+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:43.865 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:43.865 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:43.874 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T21:42:43.958 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:43.957+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:44.018 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:44.019 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:44.027 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T21:42:44.069 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:44.067+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:44.123 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 5, 3] out_osds: [0, 4] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-21T21:42:44.123 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T21:42:44.123 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-21T21:42:44.124 INFO:tasks.thrashosds.thrasher:Setting osd 1 primary_affinity to 1.000000 2024-09-21T21:42:44.124 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-09-21T21:42:44.152 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:44.151+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:44.170 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:44.171 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:44.179 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T21:42:44.263 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:44.262+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:44.324 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:44.324 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:44.333 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T21:42:44.374 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:44.373+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:44.455 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:44.454+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:44.475 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:44.475 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:44.484 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T21:42:44.518 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T21:42:44.562 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:44.561+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:44.618 INFO:teuthology.orchestra.run.smithi081.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-21T21:42:44.622 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T21:42:44.629 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:44.629 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:44.639 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T21:42:44.641 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0: {} 2024-09-21T21:42:44.641 INFO:teuthology.orchestra.run.smithi081.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T21:42:44.657 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:44.655+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:44.660 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1: {} 2024-09-21T21:42:44.661 INFO:teuthology.orchestra.run.smithi081.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T21:42:44.679 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2: {} 2024-09-21T21:42:44.679 INFO:teuthology.orchestra.run.smithi081.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T21:42:44.697 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3: {} 2024-09-21T21:42:44.697 INFO:teuthology.orchestra.run.smithi081.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T21:42:44.715 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4: {} 2024-09-21T21:42:44.715 INFO:teuthology.orchestra.run.smithi081.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T21:42:44.741 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5: {} 2024-09-21T21:42:44.741 INFO:teuthology.orchestra.run.smithi081.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T21:42:44.763 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:44.763 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:42:44.764 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 0 2024-09-21T21:42:44.764 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:44.764 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:44.762+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:44.773 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T21:42:44.859 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:44.857+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:44.919 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:44.919 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:42:44.919 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:42:44.919 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:42:44.919 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:44.928 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T21:42:44.971 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:44.970+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:45.031 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:45.031 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 20, 2024-09-21T21:42:45.031 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 600, 2024-09-21T21:42:45.031 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:42:45.031 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.032 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11319 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 5492736~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:45.032 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.712385+0000", 2024-09-21T21:42:45.032 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 54.316639696000003, 2024-09-21T21:42:45.032 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078627889000000006, 2024-09-21T21:42:45.032 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:45.032 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:45.032 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:45.032 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:45.032 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:45.032 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:45.032 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11319 2024-09-21T21:42:45.033 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.033 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:45.033 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.033 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:45.033 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712385+0000", 2024-09-21T21:42:45.033 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:45.033 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.033 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.033 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:45.033 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712385+0000", 2024-09-21T21:42:45.033 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:45.033 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.033 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.034 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:45.034 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712385+0000", 2024-09-21T21:42:45.034 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2800000000000002e-07 2024-09-21T21:42:45.034 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.034 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.034 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:45.034 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712387+0000", 2024-09-21T21:42:45.034 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1569999999999998e-06 2024-09-21T21:42:45.034 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.034 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.034 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:45.034 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712393+0000", 2024-09-21T21:42:45.034 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9370000000000001e-06 2024-09-21T21:42:45.034 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.035 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.035 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:45.035 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712398+0000", 2024-09-21T21:42:45.035 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5020000000000003e-06 2024-09-21T21:42:45.035 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.035 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.035 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:45.035 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.790746+0000", 2024-09-21T21:42:45.035 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078348589999999996 2024-09-21T21:42:45.036 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.036 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.036 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:45.036 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.790844+0000", 2024-09-21T21:42:45.036 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.7526000000000001e-05 2024-09-21T21:42:45.036 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.036 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.036 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:45.036 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.791013+0000", 2024-09-21T21:42:45.036 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00016874899999999999 2024-09-21T21:42:45.036 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:45.036 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:45.036 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:45.036 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.037 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.037 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11320 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 6180864~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:45.037 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.712405+0000", 2024-09-21T21:42:45.037 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 54.316619737000003, 2024-09-21T21:42:45.037 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078882870999999993, 2024-09-21T21:42:45.037 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:45.037 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:45.037 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:45.037 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:45.037 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:45.037 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:45.037 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11320 2024-09-21T21:42:45.037 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.037 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:45.038 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.038 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:45.038 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712405+0000", 2024-09-21T21:42:45.038 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:45.038 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.038 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.038 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:45.038 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712405+0000", 2024-09-21T21:42:45.038 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:45.038 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.038 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.038 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:45.038 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.713942+0000", 2024-09-21T21:42:45.039 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.001536882 2024-09-21T21:42:45.039 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.039 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.039 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:45.039 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.713946+0000", 2024-09-21T21:42:45.039 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4549999999999997e-06 2024-09-21T21:42:45.039 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.039 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.039 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:45.039 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.713958+0000", 2024-09-21T21:42:45.039 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1775e-05 2024-09-21T21:42:45.039 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.039 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.040 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:45.040 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.713965+0000", 2024-09-21T21:42:45.040 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5100000000000001e-06 2024-09-21T21:42:45.040 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.040 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.040 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:45.040 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.791059+0000", 2024-09-21T21:42:45.040 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.077094036000000005 2024-09-21T21:42:45.040 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.040 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.040 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:45.040 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.791175+0000", 2024-09-21T21:42:45.040 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000115895 2024-09-21T21:42:45.040 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.040 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.041 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:45.041 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.791288+0000", 2024-09-21T21:42:45.041 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000112318 2024-09-21T21:42:45.041 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:45.041 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:45.041 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:45.041 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.041 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.041 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11325 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 5414912~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:45.041 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.714095+0000", 2024-09-21T21:42:45.041 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 54.314929741999997, 2024-09-21T21:42:45.041 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078631894999999993, 2024-09-21T21:42:45.041 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:45.041 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:45.041 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:45.042 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:45.042 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:45.042 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:45.042 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11325 2024-09-21T21:42:45.042 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.042 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:45.042 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.042 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:45.042 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714095+0000", 2024-09-21T21:42:45.042 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:45.042 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.042 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.042 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:45.043 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714095+0000", 2024-09-21T21:42:45.043 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:45.043 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.043 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.043 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:45.043 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714095+0000", 2024-09-21T21:42:45.043 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7500000000000001e-07 2024-09-21T21:42:45.043 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.043 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.043 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:45.043 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714097+0000", 2024-09-21T21:42:45.043 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.12e-06 2024-09-21T21:42:45.043 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.043 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.043 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:45.044 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714104+0000", 2024-09-21T21:42:45.044 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3520000000000003e-06 2024-09-21T21:42:45.044 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.044 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.044 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:45.044 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714108+0000", 2024-09-21T21:42:45.044 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8810000000000004e-06 2024-09-21T21:42:45.044 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.044 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.044 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:45.044 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792509+0000", 2024-09-21T21:42:45.044 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078400739999999997 2024-09-21T21:42:45.044 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.044 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.044 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:45.045 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792613+0000", 2024-09-21T21:42:45.045 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000103939 2024-09-21T21:42:45.045 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.045 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.045 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:45.045 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792727+0000", 2024-09-21T21:42:45.045 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011348799999999999 2024-09-21T21:42:45.045 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:45.045 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:45.045 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:45.045 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.045 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.045 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11326 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 6103040~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:45.046 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.714115+0000", 2024-09-21T21:42:45.046 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 54.314909714000002, 2024-09-21T21:42:45.046 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078963056000000004, 2024-09-21T21:42:45.046 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:45.046 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:45.046 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:45.046 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:45.046 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:45.046 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:45.046 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11326 2024-09-21T21:42:45.046 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.046 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:45.047 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.047 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:45.047 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714115+0000", 2024-09-21T21:42:45.047 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:45.047 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.047 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.047 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:45.047 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714115+0000", 2024-09-21T21:42:45.047 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:45.047 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.047 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.047 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:45.047 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.715940+0000", 2024-09-21T21:42:45.047 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0018254969999999999 2024-09-21T21:42:45.047 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.048 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.048 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:45.048 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.715945+0000", 2024-09-21T21:42:45.048 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6120000000000001e-06 2024-09-21T21:42:45.048 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.048 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.048 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:45.048 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.715954+0000", 2024-09-21T21:42:45.048 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.5899999999999997e-06 2024-09-21T21:42:45.048 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.048 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.048 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:45.049 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.715959+0000", 2024-09-21T21:42:45.049 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0849999999999996e-06 2024-09-21T21:42:45.049 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.049 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.049 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:45.049 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792771+0000", 2024-09-21T21:42:45.049 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.076811829999999998 2024-09-21T21:42:45.049 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.049 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.049 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:45.049 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792882+0000", 2024-09-21T21:42:45.049 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000110857 2024-09-21T21:42:45.050 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.050 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.050 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:45.050 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.793078+0000", 2024-09-21T21:42:45.050 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00019558499999999999 2024-09-21T21:42:45.050 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:45.050 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:45.050 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:45.050 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.050 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.050 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11332 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 7565312~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:45.050 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.716063+0000", 2024-09-21T21:42:45.050 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 54.312961180000002, 2024-09-21T21:42:45.050 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078787482000000006, 2024-09-21T21:42:45.051 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:45.051 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:45.051 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:45.051 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:45.051 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:45.051 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:45.051 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11332 2024-09-21T21:42:45.051 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.051 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:45.051 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.051 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:45.051 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.716063+0000", 2024-09-21T21:42:45.051 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:45.051 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.051 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.052 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:45.052 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.716063+0000", 2024-09-21T21:42:45.052 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:45.052 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.052 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.052 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:45.052 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.717944+0000", 2024-09-21T21:42:45.052 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0018812309999999999 2024-09-21T21:42:45.052 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.052 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.052 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:45.052 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.717949+0000", 2024-09-21T21:42:45.052 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9389999999999996e-06 2024-09-21T21:42:45.052 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.052 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.053 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:45.053 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.717963+0000", 2024-09-21T21:42:45.053 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.3899e-05 2024-09-21T21:42:45.053 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.053 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.053 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:45.053 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.717970+0000", 2024-09-21T21:42:45.053 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.426e-06 2024-09-21T21:42:45.053 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.053 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.053 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:45.053 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.794573+0000", 2024-09-21T21:42:45.053 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.076602990999999995 2024-09-21T21:42:45.053 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.053 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.054 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:45.054 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.794682+0000", 2024-09-21T21:42:45.054 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000109541 2024-09-21T21:42:45.054 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.054 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.054 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:45.054 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.794851+0000", 2024-09-21T21:42:45.054 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000168455 2024-09-21T21:42:45.054 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:45.054 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:45.054 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:45.054 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.054 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.054 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11339 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 6266880~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:45.055 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.718117+0000", 2024-09-21T21:42:45.055 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 54.310907153999999, 2024-09-21T21:42:45.055 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078703717000000006, 2024-09-21T21:42:45.055 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:45.055 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:45.055 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:45.055 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:45.055 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:45.055 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:45.055 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11339 2024-09-21T21:42:45.055 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.055 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:45.055 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.055 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:45.055 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.718117+0000", 2024-09-21T21:42:45.056 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:45.056 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.056 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.056 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:45.056 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.718117+0000", 2024-09-21T21:42:45.056 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:45.056 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.056 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.056 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:45.056 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720019+0000", 2024-09-21T21:42:45.056 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0019012549999999999 2024-09-21T21:42:45.056 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.056 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.057 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:45.057 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720024+0000", 2024-09-21T21:42:45.057 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1100000000000002e-06 2024-09-21T21:42:45.057 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.057 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.057 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:45.057 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720035+0000", 2024-09-21T21:42:45.057 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.094e-05 2024-09-21T21:42:45.057 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.057 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.057 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:45.057 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720040+0000", 2024-09-21T21:42:45.058 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1830000000000002e-06 2024-09-21T21:42:45.058 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.058 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.058 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:45.058 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796618+0000", 2024-09-21T21:42:45.058 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.076578033000000004 2024-09-21T21:42:45.058 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.058 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.058 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:45.058 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796696+0000", 2024-09-21T21:42:45.058 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7836000000000005e-05 2024-09-21T21:42:45.059 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.059 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.059 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:45.059 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796821+0000", 2024-09-21T21:42:45.059 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00012536000000000001 2024-09-21T21:42:45.059 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:45.059 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:45.059 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:45.059 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.059 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.059 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11345 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 5472256~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:45.059 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.720116+0000", 2024-09-21T21:42:45.059 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 54.308908254000002, 2024-09-21T21:42:45.060 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078667730000000005, 2024-09-21T21:42:45.060 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:45.060 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:45.060 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:45.060 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:45.060 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:45.060 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:45.060 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11345 2024-09-21T21:42:45.060 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.060 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:45.060 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.060 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:45.061 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720116+0000", 2024-09-21T21:42:45.061 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:45.061 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.061 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.061 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:45.061 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720116+0000", 2024-09-21T21:42:45.061 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:45.061 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.061 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.061 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:45.061 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720116+0000", 2024-09-21T21:42:45.061 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.04e-07 2024-09-21T21:42:45.061 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.061 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.062 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:45.062 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720118+0000", 2024-09-21T21:42:45.062 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.181e-06 2024-09-21T21:42:45.062 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.062 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.062 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:45.062 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720121+0000", 2024-09-21T21:42:45.062 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0199999999999999e-06 2024-09-21T21:42:45.062 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.062 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.062 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:45.062 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720123+0000", 2024-09-21T21:42:45.062 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.5469999999999999e-06 2024-09-21T21:42:45.062 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.063 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.063 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:45.063 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798626+0000", 2024-09-21T21:42:45.063 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078503137000000001 2024-09-21T21:42:45.063 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.063 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.063 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:45.063 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798677+0000", 2024-09-21T21:42:45.063 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0930999999999997e-05 2024-09-21T21:42:45.063 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.063 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.063 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:45.063 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798784+0000", 2024-09-21T21:42:45.063 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010671 2024-09-21T21:42:45.063 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:45.064 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:45.064 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:45.064 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.064 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.064 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11346 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 5500928~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:45.064 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.720126+0000", 2024-09-21T21:42:45.064 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 54.308898181000004, 2024-09-21T21:42:45.064 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078863421000000003, 2024-09-21T21:42:45.064 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:45.064 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:45.064 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:45.064 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:45.064 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:45.064 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:45.065 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11346 2024-09-21T21:42:45.065 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.065 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:45.065 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.065 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:45.065 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720126+0000", 2024-09-21T21:42:45.065 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:45.065 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.065 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.065 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:45.065 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720126+0000", 2024-09-21T21:42:45.065 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:45.065 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.065 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.065 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:45.065 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.722145+0000", 2024-09-21T21:42:45.066 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0020183219999999999 2024-09-21T21:42:45.066 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.066 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.066 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:45.066 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.722150+0000", 2024-09-21T21:42:45.066 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1760000000000001e-06 2024-09-21T21:42:45.066 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.066 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.066 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:45.066 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.722161+0000", 2024-09-21T21:42:45.066 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1566000000000001e-05 2024-09-21T21:42:45.066 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.067 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.067 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:45.067 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.722169+0000", 2024-09-21T21:42:45.067 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.2840000000000001e-06 2024-09-21T21:42:45.067 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.067 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.067 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:45.067 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798815+0000", 2024-09-21T21:42:45.067 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.076646607000000005 2024-09-21T21:42:45.067 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.067 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.067 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:45.067 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798870+0000", 2024-09-21T21:42:45.067 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4486e-05 2024-09-21T21:42:45.067 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.068 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.068 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:45.068 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798990+0000", 2024-09-21T21:42:45.068 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011998 2024-09-21T21:42:45.068 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:45.068 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:45.068 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:45.068 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.068 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.068 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13629 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7852032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:45.068 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943319+0000", 2024-09-21T21:42:45.068 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 54.085705539000003, 2024-09-21T21:42:45.068 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078610975999999999, 2024-09-21T21:42:45.068 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:45.068 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:45.069 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:45.069 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:45.069 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:45.069 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:45.069 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13629 2024-09-21T21:42:45.069 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.069 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:45.069 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.069 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:45.069 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943319+0000", 2024-09-21T21:42:45.069 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:45.069 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.069 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.069 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:45.069 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943319+0000", 2024-09-21T21:42:45.070 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:45.070 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.070 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.070 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:45.070 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943319+0000", 2024-09-21T21:42:45.070 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6199999999999999e-07 2024-09-21T21:42:45.070 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.070 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.070 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:45.070 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943321+0000", 2024-09-21T21:42:45.070 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1009999999999999e-06 2024-09-21T21:42:45.070 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.070 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.070 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:45.071 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943328+0000", 2024-09-21T21:42:45.071 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.1029999999999996e-06 2024-09-21T21:42:45.071 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.071 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.071 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:45.071 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943333+0000", 2024-09-21T21:42:45.071 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7709999999999997e-06 2024-09-21T21:42:45.071 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.071 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.071 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:45.071 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.021679+0000", 2024-09-21T21:42:45.071 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078345673000000005 2024-09-21T21:42:45.071 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.072 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.072 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:45.072 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.021771+0000", 2024-09-21T21:42:45.072 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2201999999999996e-05 2024-09-21T21:42:45.072 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.072 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.072 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:45.072 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.021930+0000", 2024-09-21T21:42:45.072 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00015876400000000001 2024-09-21T21:42:45.072 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:45.072 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:45.072 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:45.072 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.072 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.072 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13630 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7880704~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:45.073 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943339+0000", 2024-09-21T21:42:45.073 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 54.085685351000002, 2024-09-21T21:42:45.073 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078939410000000002, 2024-09-21T21:42:45.073 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:45.073 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:45.073 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:45.073 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:45.073 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:45.073 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:45.073 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13630 2024-09-21T21:42:45.073 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.073 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:45.073 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.073 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:45.074 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943339+0000", 2024-09-21T21:42:45.074 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:45.074 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.074 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.074 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:45.074 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943339+0000", 2024-09-21T21:42:45.074 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:45.074 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.074 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.074 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:45.074 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943339+0000", 2024-09-21T21:42:45.074 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8000000000000001e-07 2024-09-21T21:42:45.074 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.074 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.075 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:45.075 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943341+0000", 2024-09-21T21:42:45.075 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9850000000000001e-06 2024-09-21T21:42:45.075 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.075 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.075 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:45.075 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943347+0000", 2024-09-21T21:42:45.075 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2979999999999999e-06 2024-09-21T21:42:45.075 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.075 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.075 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:45.075 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943354+0000", 2024-09-21T21:42:45.075 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.2969999999999998e-06 2024-09-21T21:42:45.075 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.076 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.076 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:45.076 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.021996+0000", 2024-09-21T21:42:45.076 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078641680000000005 2024-09-21T21:42:45.076 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.076 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.076 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:45.076 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022104+0000", 2024-09-21T21:42:45.076 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000107902 2024-09-21T21:42:45.076 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.076 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.076 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:45.077 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022278+0000", 2024-09-21T21:42:45.077 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00017486800000000001 2024-09-21T21:42:45.077 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:45.077 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:45.077 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:45.077 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.077 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.077 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13631 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7909376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:45.077 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943362+0000", 2024-09-21T21:42:45.077 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 54.085662530999997, 2024-09-21T21:42:45.077 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079195644999999995, 2024-09-21T21:42:45.077 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:45.077 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:45.077 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:45.078 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:45.078 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:45.078 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:45.078 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13631 2024-09-21T21:42:45.078 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.078 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:45.078 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.078 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:45.078 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943362+0000", 2024-09-21T21:42:45.078 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:45.078 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.078 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.078 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:45.078 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943362+0000", 2024-09-21T21:42:45.079 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:45.079 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.079 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.079 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:45.079 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943362+0000", 2024-09-21T21:42:45.079 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8799999999999998e-07 2024-09-21T21:42:45.079 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.079 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.079 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:45.079 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943364+0000", 2024-09-21T21:42:45.079 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9850000000000001e-06 2024-09-21T21:42:45.080 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.080 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.080 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:45.080 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943371+0000", 2024-09-21T21:42:45.080 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0650000000000001e-06 2024-09-21T21:42:45.080 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.080 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.080 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:45.080 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943376+0000", 2024-09-21T21:42:45.080 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8250000000000004e-06 2024-09-21T21:42:45.080 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.080 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.080 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:45.080 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022329+0000", 2024-09-21T21:42:45.081 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078952888999999998 2024-09-21T21:42:45.081 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.081 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.081 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:45.081 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022419+0000", 2024-09-21T21:42:45.081 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.9738e-05 2024-09-21T21:42:45.081 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.081 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.081 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:45.081 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022558+0000", 2024-09-21T21:42:45.081 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00013875500000000001 2024-09-21T21:42:45.081 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:45.081 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:45.082 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:45.082 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.082 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.082 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13632 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7966720~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:45.082 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943382+0000", 2024-09-21T21:42:45.082 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 54.085642297, 2024-09-21T21:42:45.082 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079495727000000002, 2024-09-21T21:42:45.082 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:45.082 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:45.082 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:45.082 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:45.082 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:45.082 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:45.082 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13632 2024-09-21T21:42:45.083 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.083 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:45.083 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.083 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:45.083 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943382+0000", 2024-09-21T21:42:45.083 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:45.083 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.083 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.083 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:45.083 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943382+0000", 2024-09-21T21:42:45.083 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:45.083 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.083 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.083 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:45.083 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943382+0000", 2024-09-21T21:42:45.084 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7e-07 2024-09-21T21:42:45.084 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.084 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.084 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:45.084 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943385+0000", 2024-09-21T21:42:45.084 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2230000000000001e-06 2024-09-21T21:42:45.084 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.084 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.084 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:45.084 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943391+0000", 2024-09-21T21:42:45.084 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6810000000000001e-06 2024-09-21T21:42:45.084 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.084 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.084 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:45.085 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943397+0000", 2024-09-21T21:42:45.085 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6559999999999997e-06 2024-09-21T21:42:45.085 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.085 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.085 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:45.085 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022601+0000", 2024-09-21T21:42:45.085 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079204051999999997 2024-09-21T21:42:45.085 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.085 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.085 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:45.085 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022704+0000", 2024-09-21T21:42:45.085 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010329599999999999 2024-09-21T21:42:45.085 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.085 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.085 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:45.086 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022878+0000", 2024-09-21T21:42:45.086 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000173449 2024-09-21T21:42:45.086 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:45.086 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:45.086 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:45.086 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.086 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.086 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13633 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7938048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:45.086 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943403+0000", 2024-09-21T21:42:45.086 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 54.085621367000002, 2024-09-21T21:42:45.086 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079794697999999997, 2024-09-21T21:42:45.086 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:45.087 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:45.087 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:45.087 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:45.087 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:45.087 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:45.087 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13633 2024-09-21T21:42:45.087 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.087 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:45.087 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.087 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:45.087 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943403+0000", 2024-09-21T21:42:45.087 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:45.087 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.087 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.088 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:45.088 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943403+0000", 2024-09-21T21:42:45.088 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:45.088 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.088 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.088 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:45.088 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943403+0000", 2024-09-21T21:42:45.088 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.65e-07 2024-09-21T21:42:45.088 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.088 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.088 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:45.088 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943405+0000", 2024-09-21T21:42:45.088 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0020000000000001e-06 2024-09-21T21:42:45.089 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.089 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.089 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:45.089 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943412+0000", 2024-09-21T21:42:45.089 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.8379999999999996e-06 2024-09-21T21:42:45.089 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.089 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.089 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:45.089 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943417+0000", 2024-09-21T21:42:45.089 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7650000000000001e-06 2024-09-21T21:42:45.089 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.089 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.089 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:45.089 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022922+0000", 2024-09-21T21:42:45.090 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079504512999999999 2024-09-21T21:42:45.090 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.090 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.090 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:45.090 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023039+0000", 2024-09-21T21:42:45.090 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000117044 2024-09-21T21:42:45.090 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.090 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.090 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:45.090 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023198+0000", 2024-09-21T21:42:45.090 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000159171 2024-09-21T21:42:45.090 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:45.090 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:45.090 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:45.091 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.091 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.091 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13634 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7995392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:45.091 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943423+0000", 2024-09-21T21:42:45.091 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 54.085601658999998, 2024-09-21T21:42:45.091 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.080142358999999996, 2024-09-21T21:42:45.091 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:45.091 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:45.091 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:45.091 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:45.091 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:45.091 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:45.091 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13634 2024-09-21T21:42:45.091 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.092 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:45.092 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.092 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:45.092 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943423+0000", 2024-09-21T21:42:45.092 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:45.092 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.092 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.092 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:45.092 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943423+0000", 2024-09-21T21:42:45.092 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:45.092 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.092 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.092 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:45.093 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943423+0000", 2024-09-21T21:42:45.093 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.58e-07 2024-09-21T21:42:45.093 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.093 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.093 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:45.093 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943425+0000", 2024-09-21T21:42:45.093 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9180000000000001e-06 2024-09-21T21:42:45.093 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.093 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.093 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:45.093 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943431+0000", 2024-09-21T21:42:45.093 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4439999999999996e-06 2024-09-21T21:42:45.093 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.093 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.093 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:45.094 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943436+0000", 2024-09-21T21:42:45.094 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.887e-06 2024-09-21T21:42:45.094 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.094 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.094 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:45.094 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023266+0000", 2024-09-21T21:42:45.094 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079829622000000003 2024-09-21T21:42:45.094 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.094 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.094 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:45.094 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023384+0000", 2024-09-21T21:42:45.095 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000117658 2024-09-21T21:42:45.095 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.095 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.095 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:45.095 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023565+0000", 2024-09-21T21:42:45.095 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00018147200000000001 2024-09-21T21:42:45.095 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:45.095 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:45.095 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:45.095 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.095 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.095 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13635 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 8024064~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:45.096 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943447+0000", 2024-09-21T21:42:45.096 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 54.085577119, 2024-09-21T21:42:45.096 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.080417653000000006, 2024-09-21T21:42:45.096 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:45.096 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:45.096 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:45.096 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:45.096 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:45.096 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:45.096 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13635 2024-09-21T21:42:45.096 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.096 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:45.096 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.097 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:45.097 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943447+0000", 2024-09-21T21:42:45.097 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:45.097 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.097 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.097 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:45.097 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943447+0000", 2024-09-21T21:42:45.097 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:45.097 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.097 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.097 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:45.097 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943448+0000", 2024-09-21T21:42:45.098 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1699999999999999e-07 2024-09-21T21:42:45.098 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.098 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.098 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:45.098 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943450+0000", 2024-09-21T21:42:45.098 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1639999999999999e-06 2024-09-21T21:42:45.098 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.098 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.098 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:45.098 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943456+0000", 2024-09-21T21:42:45.098 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8379999999999998e-06 2024-09-21T21:42:45.098 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.098 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.099 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:45.099 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943460+0000", 2024-09-21T21:42:45.099 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4700000000000004e-06 2024-09-21T21:42:45.099 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.099 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.099 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:45.099 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023595+0000", 2024-09-21T21:42:45.099 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.080134869999999997 2024-09-21T21:42:45.099 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.099 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.099 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:45.099 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023688+0000", 2024-09-21T21:42:45.099 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2794000000000006e-05 2024-09-21T21:42:45.099 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.099 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.100 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:45.100 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023865+0000", 2024-09-21T21:42:45.100 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0001771 2024-09-21T21:42:45.100 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:45.100 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:45.100 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:45.100 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.100 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.100 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13636 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 8052736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:45.100 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943467+0000", 2024-09-21T21:42:45.100 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 54.085557381999998, 2024-09-21T21:42:45.100 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.080694399999999999, 2024-09-21T21:42:45.100 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:45.101 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:45.101 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:45.101 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:45.101 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:45.101 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:45.101 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13636 2024-09-21T21:42:45.101 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.101 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:45.101 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.101 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:45.101 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943467+0000", 2024-09-21T21:42:45.101 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:45.101 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.101 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.101 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:45.102 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943467+0000", 2024-09-21T21:42:45.102 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:45.102 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.102 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.102 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:45.102 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944940+0000", 2024-09-21T21:42:45.102 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0014725109999999999 2024-09-21T21:42:45.102 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.102 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.102 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:45.102 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944949+0000", 2024-09-21T21:42:45.102 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2930000000000006e-06 2024-09-21T21:42:45.102 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.103 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.103 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:45.103 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944958+0000", 2024-09-21T21:42:45.103 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.6899999999999998e-06 2024-09-21T21:42:45.103 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.103 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.103 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:45.103 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944966+0000", 2024-09-21T21:42:45.103 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0690000000000002e-06 2024-09-21T21:42:45.103 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.103 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.103 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:45.103 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023900+0000", 2024-09-21T21:42:45.104 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078934371000000003 2024-09-21T21:42:45.104 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.104 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.104 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:45.104 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024009+0000", 2024-09-21T21:42:45.104 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000108857 2024-09-21T21:42:45.104 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.104 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.104 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:45.104 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024161+0000", 2024-09-21T21:42:45.104 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000152609 2024-09-21T21:42:45.104 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:45.104 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:45.104 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:45.104 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.105 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.105 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13637 3.12 3.1dc31b72 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:45.105 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.944978+0000", 2024-09-21T21:42:45.105 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 54.084046446999999, 2024-09-21T21:42:45.105 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079232604999999998, 2024-09-21T21:42:45.105 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:45.105 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:45.105 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:42:45.105 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:45.105 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:45.105 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:45.106 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13637 2024-09-21T21:42:45.106 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.106 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:45.106 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.106 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:45.106 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944978+0000", 2024-09-21T21:42:45.106 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:45.107 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.107 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.107 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:45.107 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944978+0000", 2024-09-21T21:42:45.107 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:45.107 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.107 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.108 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:45.108 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944978+0000", 2024-09-21T21:42:45.108 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9100000000000004e-07 2024-09-21T21:42:45.108 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.108 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.108 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:45.109 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944981+0000", 2024-09-21T21:42:45.109 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3290000000000001e-06 2024-09-21T21:42:45.109 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.109 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.109 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:45.109 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944987+0000", 2024-09-21T21:42:45.109 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.28e-06 2024-09-21T21:42:45.110 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.110 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.110 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:45.110 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944992+0000", 2024-09-21T21:42:45.110 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8520000000000003e-06 2024-09-21T21:42:45.110 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.111 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.111 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:45.111 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024196+0000", 2024-09-21T21:42:45.111 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079203673000000002 2024-09-21T21:42:45.111 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.111 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.111 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:45.112 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024211+0000", 2024-09-21T21:42:45.112 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.4980000000000001e-05 2024-09-21T21:42:45.112 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:45.112 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:45.112 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:45.112 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.113 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.113 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13638 3.12 3.1dc31b72 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:45.113 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.945000+0000", 2024-09-21T21:42:45.113 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 54.084024319000001, 2024-09-21T21:42:45.113 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079257465999999999, 2024-09-21T21:42:45.113 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:45.113 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:45.114 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:42:45.114 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:45.114 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:45.114 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:45.114 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13638 2024-09-21T21:42:45.114 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.115 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:45.115 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.115 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:45.115 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945000+0000", 2024-09-21T21:42:45.115 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:45.115 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.115 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.116 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:45.116 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945000+0000", 2024-09-21T21:42:45.116 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:45.116 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.116 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.116 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:45.116 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945001+0000", 2024-09-21T21:42:45.117 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.32e-07 2024-09-21T21:42:45.117 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.117 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.117 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:45.117 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945003+0000", 2024-09-21T21:42:45.117 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2309999999999999e-06 2024-09-21T21:42:45.118 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.118 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.118 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:45.118 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945009+0000", 2024-09-21T21:42:45.118 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7450000000000001e-06 2024-09-21T21:42:45.118 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.119 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.119 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:45.119 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945013+0000", 2024-09-21T21:42:45.119 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.673e-06 2024-09-21T21:42:45.119 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.119 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.119 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:45.120 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024245+0000", 2024-09-21T21:42:45.120 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079232160999999995 2024-09-21T21:42:45.120 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.120 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.120 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:45.120 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024258+0000", 2024-09-21T21:42:45.121 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.2224e-05 2024-09-21T21:42:45.121 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:45.121 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:45.121 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:45.121 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.121 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.121 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13639 3.12 3.1dc31b72 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:45.122 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.945034+0000", 2024-09-21T21:42:45.122 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 54.08399052, 2024-09-21T21:42:45.122 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079260627, 2024-09-21T21:42:45.122 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:45.122 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:45.122 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:42:45.123 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:45.123 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:45.123 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:45.123 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13639 2024-09-21T21:42:45.123 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.123 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:45.123 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.123 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:45.124 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945034+0000", 2024-09-21T21:42:45.124 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:45.124 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.124 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.124 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:45.124 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945034+0000", 2024-09-21T21:42:45.125 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:45.125 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.125 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.125 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:45.125 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945034+0000", 2024-09-21T21:42:45.125 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3799999999999998e-07 2024-09-21T21:42:45.125 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.126 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.126 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:45.126 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945036+0000", 2024-09-21T21:42:45.126 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.052e-06 2024-09-21T21:42:45.126 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.126 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.126 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:45.126 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945040+0000", 2024-09-21T21:42:45.126 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1169999999999997e-06 2024-09-21T21:42:45.126 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.126 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.126 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:45.127 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945043+0000", 2024-09-21T21:42:45.127 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.8439999999999998e-06 2024-09-21T21:42:45.127 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.127 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.127 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:45.127 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024286+0000", 2024-09-21T21:42:45.127 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079242483000000002 2024-09-21T21:42:45.127 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.127 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.128 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:45.128 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024295+0000", 2024-09-21T21:42:45.128 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.7930000000000003e-06 2024-09-21T21:42:45.128 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:45.128 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:45.128 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:45.128 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.128 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.128 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13640 3.12 3.1dc31b72 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:45.128 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.945047+0000", 2024-09-21T21:42:45.128 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 54.083976997999997, 2024-09-21T21:42:45.128 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079412321999999994, 2024-09-21T21:42:45.128 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:45.128 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:45.129 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:42:45.129 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:45.129 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:45.129 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:45.129 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13640 2024-09-21T21:42:45.129 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.129 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:45.129 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.129 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:45.130 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945047+0000", 2024-09-21T21:42:45.130 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:45.130 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.130 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.130 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:45.130 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945047+0000", 2024-09-21T21:42:45.130 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:45.130 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.130 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.130 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:45.130 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945048+0000", 2024-09-21T21:42:45.130 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0499999999999999e-07 2024-09-21T21:42:45.130 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.130 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.131 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:45.131 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945049+0000", 2024-09-21T21:42:45.131 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.3939999999999999e-06 2024-09-21T21:42:45.131 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.131 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.131 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:45.131 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945053+0000", 2024-09-21T21:42:45.131 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1760000000000003e-06 2024-09-21T21:42:45.131 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.131 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.131 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:45.131 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945056+0000", 2024-09-21T21:42:45.131 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.6129999999999998e-06 2024-09-21T21:42:45.131 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.131 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.132 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:45.132 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024360+0000", 2024-09-21T21:42:45.132 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079303645000000006 2024-09-21T21:42:45.132 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.132 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.132 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:45.132 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024460+0000", 2024-09-21T21:42:45.132 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000100189 2024-09-21T21:42:45.132 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:45.132 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:45.132 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:45.132 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:45.132 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:45.132 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:45.134 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T21:42:45.134 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:45.133+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:45.234 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:45.233+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:45.278 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:45.278 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:42:45.278 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 0 2024-09-21T21:42:45.278 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:45.287 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T21:42:45.335 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:45.334+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:45.390 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:45.391 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:42:45.391 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:42:45.391 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:42:45.391 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:45.399 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T21:42:45.436 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:45.435+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:45.504 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:45.504 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 20, 2024-09-21T21:42:45.504 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 600, 2024-09-21T21:42:45.504 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:42:45.504 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.504 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4824.0:1564 3.2 3:4647520c:::rbd_data.1121b97f2925.0000000000000006:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:42:45.504 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:42:45.504 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 103.865598521, 2024-09-21T21:42:45.504 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056097347999999998, 2024-09-21T21:42:45.504 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:45.505 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:45.505 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:45.505 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:45.505 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4824", 2024-09-21T21:42:45.505 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:42:45.505 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1564 2024-09-21T21:42:45.505 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.505 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:45.505 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.505 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:45.505 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:42:45.505 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:45.505 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.505 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.506 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:45.506 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:42:45.506 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:45.506 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.506 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.506 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:45.506 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:42:45.506 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6499999999999999e-07 2024-09-21T21:42:45.506 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.506 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.506 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:45.506 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636067+0000", 2024-09-21T21:42:45.506 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3879999999999998e-06 2024-09-21T21:42:45.506 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.507 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.507 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:45.507 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636075+0000", 2024-09-21T21:42:45.507 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5480000000000004e-06 2024-09-21T21:42:45.507 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.507 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.507 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:45.507 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636080+0000", 2024-09-21T21:42:45.507 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.339e-06 2024-09-21T21:42:45.507 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.507 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.507 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:45.507 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636133+0000", 2024-09-21T21:42:45.508 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2930000000000003e-05 2024-09-21T21:42:45.508 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.508 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.508 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:45.508 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636265+0000", 2024-09-21T21:42:45.508 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000132058 2024-09-21T21:42:45.508 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.508 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.508 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:45.508 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.692162+0000", 2024-09-21T21:42:45.508 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055896620000000001 2024-09-21T21:42:45.508 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:45.508 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:45.508 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:45.509 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.509 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.509 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3260 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 1437696~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:42:45.509 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.196297+0000", 2024-09-21T21:42:45.509 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 58.305366311, 2024-09-21T21:42:45.509 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056071022999999998, 2024-09-21T21:42:45.509 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:45.509 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:45.509 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:45.509 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:45.509 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:45.509 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:45.509 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3260 2024-09-21T21:42:45.509 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.509 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:45.510 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.510 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:45.510 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196297+0000", 2024-09-21T21:42:45.510 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:45.510 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.510 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.510 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:45.510 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196297+0000", 2024-09-21T21:42:45.510 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:45.510 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.510 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.510 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:45.510 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196918+0000", 2024-09-21T21:42:45.510 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00062148699999999999 2024-09-21T21:42:45.511 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.511 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.511 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:45.511 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196930+0000", 2024-09-21T21:42:45.511 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1657e-05 2024-09-21T21:42:45.511 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.511 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.511 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:45.511 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196944+0000", 2024-09-21T21:42:45.511 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.4549000000000001e-05 2024-09-21T21:42:45.511 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.511 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.511 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:45.511 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196951+0000", 2024-09-21T21:42:45.512 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6749999999999996e-06 2024-09-21T21:42:45.512 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.512 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.512 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:45.512 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252208+0000", 2024-09-21T21:42:45.512 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055256656000000001 2024-09-21T21:42:45.512 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.512 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.512 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:45.512 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252278+0000", 2024-09-21T21:42:45.512 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9882000000000006e-05 2024-09-21T21:42:45.512 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.512 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.512 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:45.512 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252368+0000", 2024-09-21T21:42:45.513 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.0117000000000005e-05 2024-09-21T21:42:45.513 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:45.513 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:45.513 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:45.513 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.513 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.513 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3263 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 3559424~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:42:45.513 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.197041+0000", 2024-09-21T21:42:45.513 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 58.304622496999997, 2024-09-21T21:42:45.513 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055935804999999998, 2024-09-21T21:42:45.513 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:45.513 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:45.513 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:45.513 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:45.514 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:45.514 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:45.514 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3263 2024-09-21T21:42:45.514 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.514 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:45.514 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.514 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:45.514 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197041+0000", 2024-09-21T21:42:45.514 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:45.514 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.514 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.514 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:45.514 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197041+0000", 2024-09-21T21:42:45.514 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:45.514 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.515 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.515 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:45.515 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197624+0000", 2024-09-21T21:42:45.515 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00058358100000000001 2024-09-21T21:42:45.515 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.515 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.515 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:45.515 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197629+0000", 2024-09-21T21:42:45.515 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5909999999999998e-06 2024-09-21T21:42:45.515 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.515 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.515 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:45.515 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197639+0000", 2024-09-21T21:42:45.515 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0257e-05 2024-09-21T21:42:45.515 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.516 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.516 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:45.516 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197646+0000", 2024-09-21T21:42:45.516 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.7229999999999998e-06 2024-09-21T21:42:45.516 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.516 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.516 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:45.516 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252804+0000", 2024-09-21T21:42:45.516 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055158134999999997 2024-09-21T21:42:45.516 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.516 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.516 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:45.516 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252882+0000", 2024-09-21T21:42:45.516 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7791000000000001e-05 2024-09-21T21:42:45.516 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.517 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.517 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:45.517 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252976+0000", 2024-09-21T21:42:45.517 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.4727000000000003e-05 2024-09-21T21:42:45.517 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:45.517 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:45.517 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:45.517 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.517 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.517 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3437 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 925696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:42:45.517 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.205888+0000", 2024-09-21T21:42:45.517 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 58.295775216000003, 2024-09-21T21:42:45.517 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055953808000000001, 2024-09-21T21:42:45.517 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:45.517 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:45.517 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:45.518 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:45.518 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:45.518 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:45.518 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3437 2024-09-21T21:42:45.518 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.518 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:45.518 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.518 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:45.518 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205888+0000", 2024-09-21T21:42:45.518 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:45.518 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.518 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.518 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:45.518 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205888+0000", 2024-09-21T21:42:45.519 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:45.519 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.519 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.519 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:45.519 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205888+0000", 2024-09-21T21:42:45.519 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.75e-07 2024-09-21T21:42:45.519 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.519 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.519 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:45.519 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205891+0000", 2024-09-21T21:42:45.519 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3750000000000001e-06 2024-09-21T21:42:45.519 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.519 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.519 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:45.520 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205897+0000", 2024-09-21T21:42:45.520 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.285e-06 2024-09-21T21:42:45.520 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.520 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.520 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:45.520 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205902+0000", 2024-09-21T21:42:45.520 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6290000000000001e-06 2024-09-21T21:42:45.520 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.520 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.520 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:45.520 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261686+0000", 2024-09-21T21:42:45.520 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055784586999999997 2024-09-21T21:42:45.520 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.520 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.520 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:45.521 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261764+0000", 2024-09-21T21:42:45.521 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7591999999999998e-05 2024-09-21T21:42:45.521 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.521 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.521 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:45.521 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261842+0000", 2024-09-21T21:42:45.521 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7865000000000001e-05 2024-09-21T21:42:45.521 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:45.521 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:45.521 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:45.521 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.521 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.521 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3438 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 954368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:42:45.521 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.205908+0000", 2024-09-21T21:42:45.521 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 58.295755350999997, 2024-09-21T21:42:45.522 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056083169000000002, 2024-09-21T21:42:45.522 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:45.522 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:45.522 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:45.522 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:45.522 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:45.522 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:45.522 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3438 2024-09-21T21:42:45.522 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.522 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:45.522 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.522 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:45.522 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205908+0000", 2024-09-21T21:42:45.523 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:45.523 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.523 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.523 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:45.523 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205908+0000", 2024-09-21T21:42:45.523 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:45.523 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.523 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.523 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:45.523 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206431+0000", 2024-09-21T21:42:45.523 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00052282100000000003 2024-09-21T21:42:45.523 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.523 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.523 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:45.523 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206442+0000", 2024-09-21T21:42:45.524 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1695000000000001e-05 2024-09-21T21:42:45.524 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.524 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.524 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:45.524 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206452+0000", 2024-09-21T21:42:45.524 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.4560000000000006e-06 2024-09-21T21:42:45.524 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.524 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.524 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:45.524 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206458+0000", 2024-09-21T21:42:45.524 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4670000000000001e-06 2024-09-21T21:42:45.524 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.524 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.524 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:45.525 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261871+0000", 2024-09-21T21:42:45.525 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055412712000000003 2024-09-21T21:42:45.525 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.525 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.525 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:45.525 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261928+0000", 2024-09-21T21:42:45.525 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6631e-05 2024-09-21T21:42:45.525 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.525 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.525 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:45.525 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261991+0000", 2024-09-21T21:42:45.525 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3386999999999995e-05 2024-09-21T21:42:45.525 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:45.525 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:45.525 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:45.526 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.526 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.526 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3444 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 2359296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:42:45.526 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.206533+0000", 2024-09-21T21:42:45.526 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 58.295129830999997, 2024-09-21T21:42:45.526 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055974239000000002, 2024-09-21T21:42:45.526 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:45.526 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:45.526 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:45.526 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:45.526 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:45.526 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:45.526 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3444 2024-09-21T21:42:45.526 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.527 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:45.527 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.527 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:45.527 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206533+0000", 2024-09-21T21:42:45.527 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:45.527 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.527 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.527 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:45.527 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206533+0000", 2024-09-21T21:42:45.527 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:45.527 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.527 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.527 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:45.527 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206710+0000", 2024-09-21T21:42:45.527 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000176293 2024-09-21T21:42:45.528 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.528 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.528 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:45.528 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206714+0000", 2024-09-21T21:42:45.528 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1989999999999999e-06 2024-09-21T21:42:45.528 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.528 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.528 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:45.528 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206723+0000", 2024-09-21T21:42:45.528 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.9199999999999993e-06 2024-09-21T21:42:45.528 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.528 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.528 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:45.528 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206729+0000", 2024-09-21T21:42:45.528 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0739999999999998e-06 2024-09-21T21:42:45.529 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.529 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.529 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:45.529 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262339+0000", 2024-09-21T21:42:45.529 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055610166000000003 2024-09-21T21:42:45.529 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.529 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.529 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:45.529 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262419+0000", 2024-09-21T21:42:45.529 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0432999999999994e-05 2024-09-21T21:42:45.529 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.529 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.529 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:45.529 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262507+0000", 2024-09-21T21:42:45.530 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.8153999999999997e-05 2024-09-21T21:42:45.530 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:45.530 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:45.530 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:45.530 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.530 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.530 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10362 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:45.530 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840386+0000", 2024-09-21T21:42:45.530 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 31.661276987000001, 2024-09-21T21:42:45.530 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056546515999999998, 2024-09-21T21:42:45.530 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:45.530 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:45.530 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:45.530 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:45.530 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:45.531 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:45.531 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10362 2024-09-21T21:42:45.531 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.531 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:45.531 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.531 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:45.531 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840386+0000", 2024-09-21T21:42:45.531 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:45.531 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.531 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.531 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:45.531 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840386+0000", 2024-09-21T21:42:45.531 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:45.532 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.532 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.532 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:45.532 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840386+0000", 2024-09-21T21:42:45.532 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.01e-07 2024-09-21T21:42:45.532 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.532 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.532 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:45.532 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840389+0000", 2024-09-21T21:42:45.532 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.9739999999999998e-06 2024-09-21T21:42:45.532 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.532 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.532 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:45.532 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840396+0000", 2024-09-21T21:42:45.533 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2280000000000003e-06 2024-09-21T21:42:45.533 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.533 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.533 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:45.533 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840400+0000", 2024-09-21T21:42:45.533 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4429999999999996e-06 2024-09-21T21:42:45.533 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.533 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.533 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:45.533 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.890896+0000", 2024-09-21T21:42:45.533 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.050495717000000002 2024-09-21T21:42:45.533 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.533 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.533 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:45.533 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.891012+0000", 2024-09-21T21:42:45.534 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011602 2024-09-21T21:42:45.534 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.534 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.534 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:45.534 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.891274+0000", 2024-09-21T21:42:45.534 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00026240200000000001 2024-09-21T21:42:45.534 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.534 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.534 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:45.534 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.894810+0000", 2024-09-21T21:42:45.534 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0035353630000000001 2024-09-21T21:42:45.534 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.534 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.534 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:45.535 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.896818+0000", 2024-09-21T21:42:45.535 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.002008681 2024-09-21T21:42:45.535 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.535 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.535 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:45.535 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.896850+0000", 2024-09-21T21:42:45.535 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.1826999999999999e-05 2024-09-21T21:42:45.535 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.535 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.535 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:45.535 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.896933+0000", 2024-09-21T21:42:45.535 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.2460000000000007e-05 2024-09-21T21:42:45.535 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:45.535 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:45.536 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:45.536 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.536 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.536 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10363 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:45.536 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840606+0000", 2024-09-21T21:42:45.536 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 31.661057477, 2024-09-21T21:42:45.536 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056546011, 2024-09-21T21:42:45.536 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:45.536 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:45.536 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:45.536 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:45.536 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:45.536 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:45.536 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10363 2024-09-21T21:42:45.537 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.537 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:45.537 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.537 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:45.537 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840606+0000", 2024-09-21T21:42:45.537 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:45.537 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.537 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.537 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:45.537 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840606+0000", 2024-09-21T21:42:45.537 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:45.537 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.537 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.537 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:45.538 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840606+0000", 2024-09-21T21:42:45.538 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.4700000000000001e-07 2024-09-21T21:42:45.538 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.538 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.538 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:45.538 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840611+0000", 2024-09-21T21:42:45.538 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4649999999999996e-06 2024-09-21T21:42:45.538 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.538 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.538 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:45.538 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840620+0000", 2024-09-21T21:42:45.538 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.0350000000000007e-06 2024-09-21T21:42:45.538 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.538 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.539 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:45.539 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840626+0000", 2024-09-21T21:42:45.539 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7180000000000001e-06 2024-09-21T21:42:45.539 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.539 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.539 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:45.539 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.891665+0000", 2024-09-21T21:42:45.539 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.051039924 2024-09-21T21:42:45.539 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.539 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.539 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:45.539 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.891770+0000", 2024-09-21T21:42:45.539 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010445700000000001 2024-09-21T21:42:45.539 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.540 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.540 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:45.540 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.892147+0000", 2024-09-21T21:42:45.540 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00037731800000000002 2024-09-21T21:42:45.540 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.540 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.540 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:45.540 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.896776+0000", 2024-09-21T21:42:45.540 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0046289759999999999 2024-09-21T21:42:45.540 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.540 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.540 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:45.540 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897003+0000", 2024-09-21T21:42:45.540 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00022641299999999999 2024-09-21T21:42:45.541 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.541 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.541 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:45.541 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897064+0000", 2024-09-21T21:42:45.541 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1571999999999999e-05 2024-09-21T21:42:45.541 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.541 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.541 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:45.541 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897152+0000", 2024-09-21T21:42:45.541 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.7386000000000007e-05 2024-09-21T21:42:45.541 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:45.541 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:45.541 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:45.541 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.541 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.542 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10364 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:45.542 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840641+0000", 2024-09-21T21:42:45.542 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 31.661021656999999, 2024-09-21T21:42:45.542 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056671527999999999, 2024-09-21T21:42:45.542 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:45.542 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:45.542 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:45.542 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:45.542 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:45.542 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:45.542 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10364 2024-09-21T21:42:45.542 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.542 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:45.542 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.543 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:45.543 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840641+0000", 2024-09-21T21:42:45.543 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:45.543 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.543 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.543 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:45.543 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840641+0000", 2024-09-21T21:42:45.543 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:45.543 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.543 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.543 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:45.543 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840642+0000", 2024-09-21T21:42:45.543 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7899999999999999e-07 2024-09-21T21:42:45.543 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.544 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.544 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:45.544 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840645+0000", 2024-09-21T21:42:45.544 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0010000000000002e-06 2024-09-21T21:42:45.544 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.544 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.544 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:45.544 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840654+0000", 2024-09-21T21:42:45.544 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.8570000000000001e-06 2024-09-21T21:42:45.544 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.544 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.544 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:45.544 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840659+0000", 2024-09-21T21:42:45.544 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.558e-06 2024-09-21T21:42:45.545 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.545 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.545 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:45.545 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.892656+0000", 2024-09-21T21:42:45.545 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.051996586999999997 2024-09-21T21:42:45.545 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.545 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.545 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:45.545 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.892721+0000", 2024-09-21T21:42:45.545 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5010000000000003e-05 2024-09-21T21:42:45.545 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.545 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.545 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:45.545 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.893112+0000", 2024-09-21T21:42:45.545 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00039133899999999999 2024-09-21T21:42:45.546 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.546 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.546 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:45.546 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.896784+0000", 2024-09-21T21:42:45.546 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0036713679999999999 2024-09-21T21:42:45.546 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.546 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.546 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:45.546 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897204+0000", 2024-09-21T21:42:45.546 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00041991400000000002 2024-09-21T21:42:45.546 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.546 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.546 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:45.546 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897234+0000", 2024-09-21T21:42:45.546 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0307999999999998e-05 2024-09-21T21:42:45.547 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.547 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.547 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:45.547 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897313+0000", 2024-09-21T21:42:45.547 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.9107e-05 2024-09-21T21:42:45.547 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:45.547 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:45.547 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:45.547 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.547 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.547 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10365 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:45.547 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840668+0000", 2024-09-21T21:42:45.547 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 31.660994792, 2024-09-21T21:42:45.547 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.058761225, 2024-09-21T21:42:45.548 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:45.548 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:45.548 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:45.548 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:45.548 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:45.548 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:45.548 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10365 2024-09-21T21:42:45.548 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.548 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:45.548 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.548 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:45.548 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840668+0000", 2024-09-21T21:42:45.548 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:45.548 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.549 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.549 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:45.549 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840668+0000", 2024-09-21T21:42:45.549 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:45.549 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.549 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.549 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:45.549 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840669+0000", 2024-09-21T21:42:45.549 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6300000000000001e-07 2024-09-21T21:42:45.549 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.549 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.549 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:45.549 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840672+0000", 2024-09-21T21:42:45.549 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.072e-06 2024-09-21T21:42:45.550 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.550 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.550 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:45.550 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840678+0000", 2024-09-21T21:42:45.550 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.7039999999999997e-06 2024-09-21T21:42:45.550 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.550 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.550 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:45.550 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840683+0000", 2024-09-21T21:42:45.550 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6759999999999998e-06 2024-09-21T21:42:45.550 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.550 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.550 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:45.550 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.893929+0000", 2024-09-21T21:42:45.551 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.053246136999999999 2024-09-21T21:42:45.551 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.551 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.551 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:45.551 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.894082+0000", 2024-09-21T21:42:45.551 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00015301799999999999 2024-09-21T21:42:45.551 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.551 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.551 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:45.551 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.894357+0000", 2024-09-21T21:42:45.551 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00027507999999999998 2024-09-21T21:42:45.551 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.551 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.551 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:45.551 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897161+0000", 2024-09-21T21:42:45.552 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0028036810000000001 2024-09-21T21:42:45.552 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.552 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.552 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:45.552 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.899243+0000", 2024-09-21T21:42:45.552 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0020816379999999998 2024-09-21T21:42:45.552 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.552 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.552 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:45.552 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.899306+0000", 2024-09-21T21:42:45.552 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2964000000000002e-05 2024-09-21T21:42:45.552 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.552 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.552 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:45.552 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.899430+0000", 2024-09-21T21:42:45.553 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00012389200000000001 2024-09-21T21:42:45.553 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:45.553 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:45.553 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:45.553 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.553 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.553 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10366 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:45.553 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840691+0000", 2024-09-21T21:42:45.553 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 31.660972092000002, 2024-09-21T21:42:45.553 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.061540984999999999, 2024-09-21T21:42:45.553 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:45.553 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:45.553 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:45.553 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:45.554 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:45.554 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:45.554 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10366 2024-09-21T21:42:45.554 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.554 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:45.554 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.554 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:45.554 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840691+0000", 2024-09-21T21:42:45.554 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:45.554 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.554 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.554 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:45.554 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840691+0000", 2024-09-21T21:42:45.554 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:45.554 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.555 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.555 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:45.555 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840691+0000", 2024-09-21T21:42:45.555 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3099999999999999e-07 2024-09-21T21:42:45.555 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.555 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.555 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:45.555 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840694+0000", 2024-09-21T21:42:45.555 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.5780000000000001e-06 2024-09-21T21:42:45.555 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.555 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.555 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:45.555 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840700+0000", 2024-09-21T21:42:45.555 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9429999999999997e-06 2024-09-21T21:42:45.555 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.556 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.556 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:45.556 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840704+0000", 2024-09-21T21:42:45.556 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6380000000000003e-06 2024-09-21T21:42:45.556 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.556 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.556 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:45.556 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.894846+0000", 2024-09-21T21:42:45.556 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.054141392000000003 2024-09-21T21:42:45.556 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.556 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.556 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:45.556 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.894932+0000", 2024-09-21T21:42:45.556 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.5834000000000005e-05 2024-09-21T21:42:45.556 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.557 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.557 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:45.557 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.895302+0000", 2024-09-21T21:42:45.557 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000370737 2024-09-21T21:42:45.557 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.557 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.557 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:45.557 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897166+0000", 2024-09-21T21:42:45.557 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0018632620000000001 2024-09-21T21:42:45.557 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.557 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.557 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:45.557 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902053+0000", 2024-09-21T21:42:45.557 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0048868289999999997 2024-09-21T21:42:45.558 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.558 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.558 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:45.558 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902111+0000", 2024-09-21T21:42:45.558 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7983999999999999e-05 2024-09-21T21:42:45.558 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.558 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.558 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:45.558 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902232+0000", 2024-09-21T21:42:45.558 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000121457 2024-09-21T21:42:45.558 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:45.558 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:45.558 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:45.558 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.559 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.559 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10367 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:45.559 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840842+0000", 2024-09-21T21:42:45.559 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 31.660821284000001, 2024-09-21T21:42:45.559 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.061549088000000002, 2024-09-21T21:42:45.559 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:45.559 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:45.559 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:45.559 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:45.559 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:45.559 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:45.559 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10367 2024-09-21T21:42:45.559 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.559 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:45.560 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.560 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:45.560 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840842+0000", 2024-09-21T21:42:45.560 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:45.560 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.560 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.560 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:45.560 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840842+0000", 2024-09-21T21:42:45.560 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:45.560 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.560 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.560 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:45.560 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840842+0000", 2024-09-21T21:42:45.560 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3099999999999997e-07 2024-09-21T21:42:45.560 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.561 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.561 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:45.561 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840846+0000", 2024-09-21T21:42:45.561 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.2890000000000002e-06 2024-09-21T21:42:45.561 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.561 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.561 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:45.561 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840855+0000", 2024-09-21T21:42:45.561 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.9619999999999999e-06 2024-09-21T21:42:45.561 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.561 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.561 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:45.561 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840860+0000", 2024-09-21T21:42:45.561 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3759999999999999e-06 2024-09-21T21:42:45.562 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.562 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.562 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:45.562 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.895799+0000", 2024-09-21T21:42:45.562 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.054938569999999999 2024-09-21T21:42:45.562 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.562 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.562 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:45.562 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.895906+0000", 2024-09-21T21:42:45.562 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000107593 2024-09-21T21:42:45.562 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.562 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.562 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:45.562 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.896356+0000", 2024-09-21T21:42:45.563 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00044941700000000002 2024-09-21T21:42:45.563 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.563 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.563 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:45.563 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.898131+0000", 2024-09-21T21:42:45.563 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.001775064 2024-09-21T21:42:45.563 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.563 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.563 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:45.563 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902283+0000", 2024-09-21T21:42:45.563 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0041524839999999997 2024-09-21T21:42:45.563 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.563 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.563 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:45.564 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902316+0000", 2024-09-21T21:42:45.564 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3002e-05 2024-09-21T21:42:45.564 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.564 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.564 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:45.564 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902391+0000", 2024-09-21T21:42:45.564 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.47e-05 2024-09-21T21:42:45.564 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:45.564 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:45.564 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:45.564 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.564 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.564 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10368 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:45.564 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840873+0000", 2024-09-21T21:42:45.564 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 31.660790330000001, 2024-09-21T21:42:45.565 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.063395291000000006, 2024-09-21T21:42:45.565 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:45.565 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:45.565 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:45.565 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:45.565 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:45.565 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:45.565 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10368 2024-09-21T21:42:45.565 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.565 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:45.565 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.565 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:45.565 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840873+0000", 2024-09-21T21:42:45.565 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:45.566 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.566 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.566 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:45.566 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840873+0000", 2024-09-21T21:42:45.566 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:45.566 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.566 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.566 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:45.566 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840873+0000", 2024-09-21T21:42:45.566 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6699999999999999e-07 2024-09-21T21:42:45.566 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.566 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.566 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:45.566 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840876+0000", 2024-09-21T21:42:45.567 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.6410000000000002e-06 2024-09-21T21:42:45.567 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.567 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.567 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:45.567 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840883+0000", 2024-09-21T21:42:45.567 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0389999999999999e-06 2024-09-21T21:42:45.567 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.567 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.567 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:45.567 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840888+0000", 2024-09-21T21:42:45.567 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8679999999999998e-06 2024-09-21T21:42:45.567 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.567 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.567 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:45.568 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897345+0000", 2024-09-21T21:42:45.568 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056457734000000002 2024-09-21T21:42:45.568 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.568 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.568 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:45.568 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897407+0000", 2024-09-21T21:42:45.568 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1822000000000005e-05 2024-09-21T21:42:45.568 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.568 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.568 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:45.568 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897664+0000", 2024-09-21T21:42:45.568 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00025638499999999998 2024-09-21T21:42:45.568 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.568 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.568 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:45.569 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.901136+0000", 2024-09-21T21:42:45.569 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0034728070000000001 2024-09-21T21:42:45.569 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.569 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.569 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:45.569 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904090+0000", 2024-09-21T21:42:45.569 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0029535799999999999 2024-09-21T21:42:45.569 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.569 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.569 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:45.569 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904144+0000", 2024-09-21T21:42:45.569 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4369000000000001e-05 2024-09-21T21:42:45.569 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.569 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.570 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:45.570 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904268+0000", 2024-09-21T21:42:45.570 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00012367899999999999 2024-09-21T21:42:45.570 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:45.570 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:45.570 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:45.570 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.570 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.570 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10369 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:45.570 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840902+0000", 2024-09-21T21:42:45.570 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 31.660760676999999, 2024-09-21T21:42:45.570 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.063538938000000003, 2024-09-21T21:42:45.570 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:45.570 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:45.571 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:45.571 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:45.571 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:45.571 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:45.571 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10369 2024-09-21T21:42:45.571 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.571 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:45.571 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.571 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:45.571 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840902+0000", 2024-09-21T21:42:45.571 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:45.571 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.571 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.572 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:45.572 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840902+0000", 2024-09-21T21:42:45.572 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:45.572 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.572 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.572 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:45.572 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840903+0000", 2024-09-21T21:42:45.572 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2100000000000002e-07 2024-09-21T21:42:45.572 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.572 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.572 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:45.572 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840913+0000", 2024-09-21T21:42:45.572 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.048e-05 2024-09-21T21:42:45.572 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.572 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.573 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:45.573 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840920+0000", 2024-09-21T21:42:45.573 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0330000000000003e-06 2024-09-21T21:42:45.573 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.573 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.573 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:45.573 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840925+0000", 2024-09-21T21:42:45.573 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0170000000000004e-06 2024-09-21T21:42:45.573 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.573 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.573 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:45.573 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.898179+0000", 2024-09-21T21:42:45.573 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.057253671999999999 2024-09-21T21:42:45.574 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.574 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.574 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:45.574 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.898284+0000", 2024-09-21T21:42:45.574 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000105202 2024-09-21T21:42:45.574 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.574 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.574 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:45.574 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.898729+0000", 2024-09-21T21:42:45.574 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00044459300000000002 2024-09-21T21:42:45.574 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.574 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.574 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:45.574 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902246+0000", 2024-09-21T21:42:45.574 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0035171629999999998 2024-09-21T21:42:45.575 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.575 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.575 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:45.575 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904310+0000", 2024-09-21T21:42:45.575 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.002063765 2024-09-21T21:42:45.575 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.575 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.575 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:45.575 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904342+0000", 2024-09-21T21:42:45.575 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.2091999999999997e-05 2024-09-21T21:42:45.575 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.575 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.575 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:45.575 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904441+0000", 2024-09-21T21:42:45.576 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.9500000000000006e-05 2024-09-21T21:42:45.576 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:45.576 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:45.576 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:45.576 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.576 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.576 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10370 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:45.576 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840944+0000", 2024-09-21T21:42:45.576 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 31.660718964000001, 2024-09-21T21:42:45.576 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.064436853000000002, 2024-09-21T21:42:45.576 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:45.576 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:45.576 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:45.576 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:45.576 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:45.576 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:45.577 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10370 2024-09-21T21:42:45.577 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.577 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:45.577 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.577 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:45.577 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840944+0000", 2024-09-21T21:42:45.577 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:45.577 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.577 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.577 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:45.577 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840944+0000", 2024-09-21T21:42:45.577 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:45.577 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.577 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.578 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:45.578 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840945+0000", 2024-09-21T21:42:45.578 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2300000000000002e-07 2024-09-21T21:42:45.578 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.578 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.578 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:45.578 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840947+0000", 2024-09-21T21:42:45.578 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.8200000000000001e-06 2024-09-21T21:42:45.578 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.578 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.578 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:45.578 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840955+0000", 2024-09-21T21:42:45.578 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.8660000000000006e-06 2024-09-21T21:42:45.579 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.579 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.579 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:45.579 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840960+0000", 2024-09-21T21:42:45.579 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8670000000000002e-06 2024-09-21T21:42:45.579 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.579 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.579 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:45.579 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.899460+0000", 2024-09-21T21:42:45.579 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.058499922000000003 2024-09-21T21:42:45.579 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.579 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.579 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:45.580 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.899530+0000", 2024-09-21T21:42:45.580 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9558e-05 2024-09-21T21:42:45.580 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.580 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.580 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:45.580 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.899775+0000", 2024-09-21T21:42:45.580 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000245231 2024-09-21T21:42:45.580 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.580 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.580 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:45.580 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902254+0000", 2024-09-21T21:42:45.580 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0024790519999999998 2024-09-21T21:42:45.580 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.580 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.581 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:45.581 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905183+0000", 2024-09-21T21:42:45.581 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.002929156 2024-09-21T21:42:45.581 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.581 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.581 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:45.581 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905243+0000", 2024-09-21T21:42:45.581 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0055e-05 2024-09-21T21:42:45.581 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.581 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.581 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:45.581 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905381+0000", 2024-09-21T21:42:45.581 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00013790300000000001 2024-09-21T21:42:45.581 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:45.581 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:45.582 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:45.582 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.582 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.582 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10371 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:45.582 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840980+0000", 2024-09-21T21:42:45.582 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 31.660683297999999, 2024-09-21T21:42:45.582 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.064554376999999996, 2024-09-21T21:42:45.582 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:45.582 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:45.582 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:45.582 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:45.582 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:45.582 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:45.582 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10371 2024-09-21T21:42:45.583 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.583 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:45.583 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.583 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:45.583 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840980+0000", 2024-09-21T21:42:45.583 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:45.583 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.583 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.583 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:45.583 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840980+0000", 2024-09-21T21:42:45.583 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:45.583 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.583 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.583 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:45.584 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840980+0000", 2024-09-21T21:42:45.584 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8999999999999997e-07 2024-09-21T21:42:45.584 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.584 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.584 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:45.584 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840983+0000", 2024-09-21T21:42:45.584 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.123e-06 2024-09-21T21:42:45.584 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.584 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.584 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:45.584 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840990+0000", 2024-09-21T21:42:45.584 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9129999999999997e-06 2024-09-21T21:42:45.584 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.584 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.584 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:45.585 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840995+0000", 2024-09-21T21:42:45.585 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0989999999999998e-06 2024-09-21T21:42:45.585 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.585 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.585 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:45.585 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.900168+0000", 2024-09-21T21:42:45.585 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.059172739000000002 2024-09-21T21:42:45.585 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.585 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.585 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:45.585 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.900274+0000", 2024-09-21T21:42:45.585 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010546499999999999 2024-09-21T21:42:45.585 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.585 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.585 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:45.586 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.900703+0000", 2024-09-21T21:42:45.586 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00042978200000000003 2024-09-21T21:42:45.586 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.586 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.586 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:45.586 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.903443+0000", 2024-09-21T21:42:45.586 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0027400749999999998 2024-09-21T21:42:45.586 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.586 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.586 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:45.586 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905430+0000", 2024-09-21T21:42:45.586 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0019868109999999998 2024-09-21T21:42:45.586 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.586 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.586 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:45.587 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905464+0000", 2024-09-21T21:42:45.587 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3599000000000001e-05 2024-09-21T21:42:45.587 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.587 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.587 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:45.587 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905534+0000", 2024-09-21T21:42:45.587 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0281e-05 2024-09-21T21:42:45.587 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:45.587 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:45.587 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:45.587 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.587 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.587 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10372 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:45.587 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.841004+0000", 2024-09-21T21:42:45.587 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 31.660659409000001, 2024-09-21T21:42:45.588 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.064757812999999997, 2024-09-21T21:42:45.588 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:45.588 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:45.588 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:45.588 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:45.588 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:45.588 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:45.588 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10372 2024-09-21T21:42:45.588 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.588 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:45.588 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.588 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:45.588 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841004+0000", 2024-09-21T21:42:45.588 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:45.589 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.589 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.589 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:45.589 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841004+0000", 2024-09-21T21:42:45.589 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:45.589 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.589 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.589 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:45.589 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841004+0000", 2024-09-21T21:42:45.589 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8500000000000002e-07 2024-09-21T21:42:45.589 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.589 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.589 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:45.589 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841007+0000", 2024-09-21T21:42:45.590 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.6390000000000001e-06 2024-09-21T21:42:45.590 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.590 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.590 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:45.590 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841013+0000", 2024-09-21T21:42:45.590 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0070000000000003e-06 2024-09-21T21:42:45.590 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.590 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.590 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:45.590 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841018+0000", 2024-09-21T21:42:45.590 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9119999999999997e-06 2024-09-21T21:42:45.590 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.590 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.590 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:45.590 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.901182+0000", 2024-09-21T21:42:45.591 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.060164444999999997 2024-09-21T21:42:45.591 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.591 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.591 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:45.591 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.901297+0000", 2024-09-21T21:42:45.591 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011468699999999999 2024-09-21T21:42:45.591 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.591 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.591 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:45.591 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.901559+0000", 2024-09-21T21:42:45.591 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000261764 2024-09-21T21:42:45.591 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.591 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.591 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:45.592 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904281+0000", 2024-09-21T21:42:45.592 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0027229329999999999 2024-09-21T21:42:45.592 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.592 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.592 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:45.592 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905588+0000", 2024-09-21T21:42:45.592 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.001306438 2024-09-21T21:42:45.592 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.592 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.592 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:45.592 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905644+0000", 2024-09-21T21:42:45.592 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5747e-05 2024-09-21T21:42:45.592 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.593 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.593 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:45.593 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905761+0000", 2024-09-21T21:42:45.593 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000117856 2024-09-21T21:42:45.593 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:45.593 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:45.593 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:45.593 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.593 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.593 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10373 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:45.593 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.841027+0000", 2024-09-21T21:42:45.593 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 31.660636324999999, 2024-09-21T21:42:45.593 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.066941839000000003, 2024-09-21T21:42:45.594 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:45.594 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:45.594 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:45.594 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:45.594 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:45.594 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:45.594 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10373 2024-09-21T21:42:45.594 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.594 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:45.594 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.594 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:45.594 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841027+0000", 2024-09-21T21:42:45.594 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:45.594 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.595 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.595 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:45.595 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841027+0000", 2024-09-21T21:42:45.595 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:45.595 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.595 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.595 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:45.595 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841027+0000", 2024-09-21T21:42:45.595 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6199999999999999e-07 2024-09-21T21:42:45.595 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.595 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.595 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:45.595 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841030+0000", 2024-09-21T21:42:45.595 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.729e-06 2024-09-21T21:42:45.596 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.596 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.596 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:45.596 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841036+0000", 2024-09-21T21:42:45.596 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1979999999999998e-06 2024-09-21T21:42:45.596 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.596 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.596 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:45.596 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841042+0000", 2024-09-21T21:42:45.596 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4289999999999998e-06 2024-09-21T21:42:45.596 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.596 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.596 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:45.596 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902433+0000", 2024-09-21T21:42:45.596 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.061390329 2024-09-21T21:42:45.597 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.597 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.597 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:45.597 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902541+0000", 2024-09-21T21:42:45.597 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000108349 2024-09-21T21:42:45.597 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.597 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.597 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:45.597 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902983+0000", 2024-09-21T21:42:45.597 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00044158000000000001 2024-09-21T21:42:45.597 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.597 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.597 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:45.597 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904490+0000", 2024-09-21T21:42:45.597 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0015074489999999999 2024-09-21T21:42:45.598 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.598 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.598 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:45.598 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.907671+0000", 2024-09-21T21:42:45.598 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.003180511 2024-09-21T21:42:45.598 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.598 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.598 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:45.598 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.907767+0000", 2024-09-21T21:42:45.598 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6379000000000007e-05 2024-09-21T21:42:45.598 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.598 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.598 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:45.598 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.907969+0000", 2024-09-21T21:42:45.599 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00020152399999999999 2024-09-21T21:42:45.599 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:45.599 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:45.599 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:45.599 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.599 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.599 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:12335 3.1a 3:5b387735:::rbd_data.1121faebdfe3.0000000000000005:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:42:45.599 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:14.411635+0000", 2024-09-21T21:42:45.599 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 31.090028447000002, 2024-09-21T21:42:45.599 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.005973491, 2024-09-21T21:42:45.599 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:45.599 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:45.599 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:45.599 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:45.599 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:45.600 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:45.600 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 12335 2024-09-21T21:42:45.600 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.600 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:45.600 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.600 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:45.600 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411635+0000", 2024-09-21T21:42:45.600 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:45.600 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.600 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.600 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:45.600 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411635+0000", 2024-09-21T21:42:45.600 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:45.600 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.600 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.601 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:45.601 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411635+0000", 2024-09-21T21:42:45.601 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8500000000000001e-07 2024-09-21T21:42:45.601 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.601 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.601 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:45.601 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411639+0000", 2024-09-21T21:42:45.601 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.574e-06 2024-09-21T21:42:45.601 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.601 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.601 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:45.601 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411645+0000", 2024-09-21T21:42:45.601 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6869999999999997e-06 2024-09-21T21:42:45.602 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.602 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.602 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:45.602 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411649+0000", 2024-09-21T21:42:45.602 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.771e-06 2024-09-21T21:42:45.602 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.602 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.602 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:45.602 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411695+0000", 2024-09-21T21:42:45.602 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6e-05 2024-09-21T21:42:45.602 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.602 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.602 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op must wait for map", 2024-09-21T21:42:45.602 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411702+0000", 2024-09-21T21:42:45.602 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5590000000000002e-06 2024-09-21T21:42:45.603 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.603 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.603 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:45.603 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.417222+0000", 2024-09-21T21:42:45.603 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0055198219999999997 2024-09-21T21:42:45.603 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.603 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.603 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for peered", 2024-09-21T21:42:45.603 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.417230+0000", 2024-09-21T21:42:45.603 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.123e-06 2024-09-21T21:42:45.603 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.603 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.603 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:45.603 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.417348+0000", 2024-09-21T21:42:45.604 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.000118029 2024-09-21T21:42:45.604 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.604 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.604 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:45.604 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.417451+0000", 2024-09-21T21:42:45.604 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000103674 2024-09-21T21:42:45.604 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.604 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.604 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:45.604 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.417608+0000", 2024-09-21T21:42:45.604 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00015666700000000001 2024-09-21T21:42:45.604 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:45.604 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:45.604 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:45.605 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.605 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.605 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:12349 3.1a 3:5b387735:::rbd_data.1121faebdfe3.0000000000000005:head [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:42:45.605 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:14.411833+0000", 2024-09-21T21:42:45.605 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 31.089829634000001, 2024-09-21T21:42:45.605 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0065064500000001, 2024-09-21T21:42:45.605 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:45.605 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:45.605 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:45.605 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:45.605 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:45.605 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:45.605 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 12349 2024-09-21T21:42:45.605 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.605 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:45.606 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.606 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:45.606 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411833+0000", 2024-09-21T21:42:45.606 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:45.606 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.606 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.606 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:45.606 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411833+0000", 2024-09-21T21:42:45.606 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:45.606 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.606 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.606 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:45.606 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411834+0000", 2024-09-21T21:42:45.606 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.03e-07 2024-09-21T21:42:45.607 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.607 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.607 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:45.607 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411836+0000", 2024-09-21T21:42:45.607 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.4700000000000001e-06 2024-09-21T21:42:45.607 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.607 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.607 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:45.607 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411843+0000", 2024-09-21T21:42:45.607 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4069999999999998e-06 2024-09-21T21:42:45.607 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.607 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.607 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:45.607 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411848+0000", 2024-09-21T21:42:45.608 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1580000000000004e-06 2024-09-21T21:42:45.608 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.608 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.608 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:45.608 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411901+0000", 2024-09-21T21:42:45.608 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2824000000000001e-05 2024-09-21T21:42:45.608 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.608 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.608 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting_for_map not empty", 2024-09-21T21:42:45.608 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411904+0000", 2024-09-21T21:42:45.608 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0639999999999998e-06 2024-09-21T21:42:45.608 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.608 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.608 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:45.609 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.417252+0000", 2024-09-21T21:42:45.609 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0053485060000000003 2024-09-21T21:42:45.609 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.609 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.609 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for peered", 2024-09-21T21:42:45.609 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.417259+0000", 2024-09-21T21:42:45.609 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.1149999999999996e-06 2024-09-21T21:42:45.609 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.609 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.609 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:45.609 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.418160+0000", 2024-09-21T21:42:45.609 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.000901083 2024-09-21T21:42:45.609 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.609 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.610 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:45.610 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.418236+0000", 2024-09-21T21:42:45.610 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5852000000000005e-05 2024-09-21T21:42:45.610 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:45.610 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:45.610 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:45.610 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.418340+0000", 2024-09-21T21:42:45.610 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000103568 2024-09-21T21:42:45.610 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:45.610 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:45.610 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:45.610 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:45.610 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:45.610 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:45.612 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T21:42:45.612 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:45.611+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:45.617 INFO:teuthology.orchestra.run.smithi081.stderr:nodeep-scrub is unset 2024-09-21T21:42:45.712 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:45.711+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:45.723 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:45.724 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:42:45.724 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 0 2024-09-21T21:42:45.724 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:45.733 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T21:42:45.817 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:45.816+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:45.876 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:45.876 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:42:45.877 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:42:45.877 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:42:45.877 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:45.885 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T21:42:45.927 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:45.925+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:46.015 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:46.014+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:46.029 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:46.029 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 20, 2024-09-21T21:42:46.029 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 600, 2024-09-21T21:42:46.029 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:42:46.029 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.029 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13296 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 4046848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:46.029 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.796643+0000", 2024-09-21T21:42:46.029 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 55.229970229000003, 2024-09-21T21:42:46.029 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071682019999999999, 2024-09-21T21:42:46.029 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:46.029 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:46.030 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:46.030 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:46.030 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:46.030 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:46.030 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13296 2024-09-21T21:42:46.030 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.030 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:46.030 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.030 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:46.030 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796643+0000", 2024-09-21T21:42:46.030 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:46.030 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.030 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.030 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:46.031 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796643+0000", 2024-09-21T21:42:46.031 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:46.031 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.031 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.031 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:46.031 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796643+0000", 2024-09-21T21:42:46.031 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.7199999999999998e-07 2024-09-21T21:42:46.031 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.031 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.031 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:46.031 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796647+0000", 2024-09-21T21:42:46.031 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6770000000000001e-06 2024-09-21T21:42:46.031 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.031 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.031 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:46.032 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796656+0000", 2024-09-21T21:42:46.032 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2059999999999996e-06 2024-09-21T21:42:46.032 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.032 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.032 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:46.032 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796664+0000", 2024-09-21T21:42:46.032 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7919999999999993e-06 2024-09-21T21:42:46.032 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.032 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.032 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:46.032 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.868186+0000", 2024-09-21T21:42:46.032 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071522273999999997 2024-09-21T21:42:46.032 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.032 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.032 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:46.033 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.868239+0000", 2024-09-21T21:42:46.033 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2787999999999997e-05 2024-09-21T21:42:46.033 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.033 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.033 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:46.033 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.868325+0000", 2024-09-21T21:42:46.033 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.5611000000000004e-05 2024-09-21T21:42:46.033 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:46.033 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:46.033 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:46.033 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.033 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.033 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13561 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 1728512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:46.033 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.823991+0000", 2024-09-21T21:42:46.033 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 55.202621927999999, 2024-09-21T21:42:46.033 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071654762999999996, 2024-09-21T21:42:46.034 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:46.034 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:46.034 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:46.034 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:46.034 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:46.034 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:46.034 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13561 2024-09-21T21:42:46.034 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.034 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:46.034 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.034 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:46.034 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.823991+0000", 2024-09-21T21:42:46.034 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:46.034 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.035 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.035 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:46.035 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.823991+0000", 2024-09-21T21:42:46.035 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:46.035 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.035 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.035 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:46.035 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.823991+0000", 2024-09-21T21:42:46.035 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9299999999999998e-07 2024-09-21T21:42:46.035 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.035 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.035 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:46.035 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.823994+0000", 2024-09-21T21:42:46.035 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2369999999999999e-06 2024-09-21T21:42:46.035 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.036 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.036 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:46.036 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824000+0000", 2024-09-21T21:42:46.036 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.55e-06 2024-09-21T21:42:46.036 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.036 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.036 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:46.036 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824005+0000", 2024-09-21T21:42:46.036 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4329999999999997e-06 2024-09-21T21:42:46.036 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.036 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.036 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:46.036 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.895360+0000", 2024-09-21T21:42:46.036 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071355426 2024-09-21T21:42:46.037 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.037 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.037 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:46.037 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.895477+0000", 2024-09-21T21:42:46.037 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0001171 2024-09-21T21:42:46.037 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.037 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.037 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:46.037 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.895646+0000", 2024-09-21T21:42:46.037 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000168524 2024-09-21T21:42:46.037 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:46.037 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:46.037 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:46.037 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.038 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.038 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13562 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 1757184~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:46.038 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.824011+0000", 2024-09-21T21:42:46.038 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 55.202602051, 2024-09-21T21:42:46.038 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072020453999999998, 2024-09-21T21:42:46.038 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:46.038 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:46.038 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:46.038 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:46.038 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:46.038 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:46.038 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13562 2024-09-21T21:42:46.038 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.039 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:46.039 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.039 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:46.039 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824011+0000", 2024-09-21T21:42:46.039 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:46.039 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.039 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.039 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:46.039 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824011+0000", 2024-09-21T21:42:46.039 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:46.039 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.039 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.039 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:46.039 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824011+0000", 2024-09-21T21:42:46.039 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5600000000000001e-07 2024-09-21T21:42:46.040 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.040 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.040 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:46.040 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824013+0000", 2024-09-21T21:42:46.040 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9800000000000001e-06 2024-09-21T21:42:46.040 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.040 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.040 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:46.040 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824019+0000", 2024-09-21T21:42:46.040 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7339999999999996e-06 2024-09-21T21:42:46.040 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.040 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.040 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:46.040 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824023+0000", 2024-09-21T21:42:46.040 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3680000000000004e-06 2024-09-21T21:42:46.041 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.041 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.041 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:46.041 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.895699+0000", 2024-09-21T21:42:46.041 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071675597999999993 2024-09-21T21:42:46.041 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.041 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.041 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:46.041 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.895830+0000", 2024-09-21T21:42:46.041 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000130858 2024-09-21T21:42:46.041 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.041 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.041 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:46.041 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896031+0000", 2024-09-21T21:42:46.042 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00020155999999999999 2024-09-21T21:42:46.042 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:46.042 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:46.042 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:46.042 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.042 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.042 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13563 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 323584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:46.042 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.824029+0000", 2024-09-21T21:42:46.042 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 55.202583611000001, 2024-09-21T21:42:46.042 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072307563000000005, 2024-09-21T21:42:46.042 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:46.042 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:46.042 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:46.042 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:46.042 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:46.043 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:46.043 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13563 2024-09-21T21:42:46.043 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.043 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:46.043 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.043 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:46.043 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824029+0000", 2024-09-21T21:42:46.043 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:46.043 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.043 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.043 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:46.043 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824029+0000", 2024-09-21T21:42:46.043 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:46.043 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.043 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.043 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:46.044 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824030+0000", 2024-09-21T21:42:46.044 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1699999999999999e-07 2024-09-21T21:42:46.044 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.044 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.044 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:46.044 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824032+0000", 2024-09-21T21:42:46.044 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0320000000000002e-06 2024-09-21T21:42:46.044 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.044 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.044 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:46.044 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824038+0000", 2024-09-21T21:42:46.044 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8769999999999998e-06 2024-09-21T21:42:46.044 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.044 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.044 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:46.044 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824042+0000", 2024-09-21T21:42:46.045 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5650000000000003e-06 2024-09-21T21:42:46.045 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.045 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.045 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:46.045 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896058+0000", 2024-09-21T21:42:46.045 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072015941 2024-09-21T21:42:46.045 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.045 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.045 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:46.045 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896164+0000", 2024-09-21T21:42:46.045 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010593499999999999 2024-09-21T21:42:46.045 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.045 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.045 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:46.046 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896337+0000", 2024-09-21T21:42:46.046 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00017279599999999999 2024-09-21T21:42:46.046 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:46.046 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:46.046 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:46.046 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.046 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.046 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13564 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 983040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:46.046 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.824048+0000", 2024-09-21T21:42:46.046 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 55.202564535999997, 2024-09-21T21:42:46.046 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072491193999999995, 2024-09-21T21:42:46.046 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:46.046 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:46.046 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:46.047 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:46.047 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:46.047 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:46.047 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13564 2024-09-21T21:42:46.047 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.047 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:46.047 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.047 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:46.047 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824048+0000", 2024-09-21T21:42:46.047 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:46.047 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.047 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.047 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:46.047 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824048+0000", 2024-09-21T21:42:46.047 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:46.048 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.048 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.048 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:46.048 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824663+0000", 2024-09-21T21:42:46.048 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00061494299999999996 2024-09-21T21:42:46.048 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.048 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.048 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:46.048 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824669+0000", 2024-09-21T21:42:46.048 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2900000000000002e-06 2024-09-21T21:42:46.048 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.048 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.048 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:46.048 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824679+0000", 2024-09-21T21:42:46.048 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0338e-05 2024-09-21T21:42:46.049 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.049 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.049 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:46.049 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824687+0000", 2024-09-21T21:42:46.049 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.9049999999999997e-06 2024-09-21T21:42:46.049 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.049 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.049 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:46.049 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896381+0000", 2024-09-21T21:42:46.049 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071694305999999999 2024-09-21T21:42:46.049 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.049 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.049 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:46.049 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896445+0000", 2024-09-21T21:42:46.050 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3869999999999997e-05 2024-09-21T21:42:46.050 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.050 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.050 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:46.050 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896540+0000", 2024-09-21T21:42:46.050 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.4542000000000004e-05 2024-09-21T21:42:46.050 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:46.050 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:46.050 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:46.050 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.050 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.050 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13565 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3190784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:46.050 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.824704+0000", 2024-09-21T21:42:46.050 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 55.201908664999998, 2024-09-21T21:42:46.051 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072014745000000005, 2024-09-21T21:42:46.051 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:46.051 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:46.051 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:46.051 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:46.051 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:46.051 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:46.051 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13565 2024-09-21T21:42:46.051 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.051 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:46.051 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.051 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:46.051 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824704+0000", 2024-09-21T21:42:46.051 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:46.052 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.052 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.052 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:46.052 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824704+0000", 2024-09-21T21:42:46.052 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:46.052 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.052 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.052 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:46.052 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824705+0000", 2024-09-21T21:42:46.052 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8999999999999997e-07 2024-09-21T21:42:46.052 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.052 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.052 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:46.052 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824707+0000", 2024-09-21T21:42:46.052 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.7669999999999999e-06 2024-09-21T21:42:46.053 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.053 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.053 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:46.053 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824715+0000", 2024-09-21T21:42:46.053 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7700000000000001e-06 2024-09-21T21:42:46.053 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.053 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.053 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:46.053 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824721+0000", 2024-09-21T21:42:46.053 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9560000000000002e-06 2024-09-21T21:42:46.053 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.053 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.053 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:46.053 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896570+0000", 2024-09-21T21:42:46.054 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071848870999999995 2024-09-21T21:42:46.054 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.054 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.054 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:46.054 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896626+0000", 2024-09-21T21:42:46.054 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5578000000000002e-05 2024-09-21T21:42:46.054 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.054 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.054 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:46.054 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896719+0000", 2024-09-21T21:42:46.054 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.3313000000000006e-05 2024-09-21T21:42:46.054 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:46.054 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:46.054 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:46.055 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.055 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.055 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13566 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3162112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:46.055 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.824729+0000", 2024-09-21T21:42:46.055 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 55.201883778000003, 2024-09-21T21:42:46.055 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072356907999999998, 2024-09-21T21:42:46.055 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:46.055 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:46.055 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:46.055 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:46.055 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:46.055 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:46.055 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13566 2024-09-21T21:42:46.055 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.055 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:46.056 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.056 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:46.056 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824729+0000", 2024-09-21T21:42:46.056 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:46.056 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.056 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.056 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:46.056 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824729+0000", 2024-09-21T21:42:46.056 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:46.056 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.056 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.056 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:46.056 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825630+0000", 2024-09-21T21:42:46.057 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00090139400000000002 2024-09-21T21:42:46.057 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.057 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.057 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:46.057 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825634+0000", 2024-09-21T21:42:46.057 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4960000000000001e-06 2024-09-21T21:42:46.057 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.057 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.057 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:46.057 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825644+0000", 2024-09-21T21:42:46.057 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.9650000000000003e-06 2024-09-21T21:42:46.057 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.057 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.057 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:46.058 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825649+0000", 2024-09-21T21:42:46.058 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0359999999999997e-06 2024-09-21T21:42:46.058 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.058 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.058 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:46.058 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896757+0000", 2024-09-21T21:42:46.058 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071107717000000001 2024-09-21T21:42:46.058 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.058 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.058 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:46.058 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896878+0000", 2024-09-21T21:42:46.058 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000120834 2024-09-21T21:42:46.058 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.059 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.059 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:46.059 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897086+0000", 2024-09-21T21:42:46.059 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00020846600000000001 2024-09-21T21:42:46.059 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:46.059 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:46.059 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:46.059 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.059 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.059 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13568 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3907584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:46.059 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.825671+0000", 2024-09-21T21:42:46.059 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 55.200941368999999, 2024-09-21T21:42:46.059 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071794595000000003, 2024-09-21T21:42:46.060 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:46.060 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:46.060 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:46.060 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:46.060 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:46.060 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:46.060 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13568 2024-09-21T21:42:46.060 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.060 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:46.060 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.060 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:46.060 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825671+0000", 2024-09-21T21:42:46.060 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:46.060 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.061 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.061 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:46.061 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825671+0000", 2024-09-21T21:42:46.061 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:46.061 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.061 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.061 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:46.061 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825672+0000", 2024-09-21T21:42:46.061 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2999999999999999e-07 2024-09-21T21:42:46.061 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.061 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.061 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:46.061 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825673+0000", 2024-09-21T21:42:46.062 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.246e-06 2024-09-21T21:42:46.062 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.062 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.062 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:46.062 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825693+0000", 2024-09-21T21:42:46.062 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9789999999999999e-05 2024-09-21T21:42:46.062 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.062 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.062 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:46.062 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825697+0000", 2024-09-21T21:42:46.062 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2379999999999999e-06 2024-09-21T21:42:46.062 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.062 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.063 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:46.063 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897284+0000", 2024-09-21T21:42:46.063 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071586710999999997 2024-09-21T21:42:46.063 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.063 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.063 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:46.063 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897337+0000", 2024-09-21T21:42:46.063 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2800000000000003e-05 2024-09-21T21:42:46.063 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.063 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.063 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:46.063 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897466+0000", 2024-09-21T21:42:46.063 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00012957999999999999 2024-09-21T21:42:46.063 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:46.063 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:46.064 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:46.064 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.064 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.064 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13574 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 352256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:46.064 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.825702+0000", 2024-09-21T21:42:46.064 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 55.200911220999998, 2024-09-21T21:42:46.064 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072080377000000001, 2024-09-21T21:42:46.064 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:46.064 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:46.064 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:46.064 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:46.065 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:46.065 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:46.065 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13574 2024-09-21T21:42:46.065 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.065 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:46.065 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.065 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:46.065 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825702+0000", 2024-09-21T21:42:46.065 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:46.065 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.065 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.065 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:46.065 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825702+0000", 2024-09-21T21:42:46.065 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:46.065 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.065 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.066 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:46.066 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825943+0000", 2024-09-21T21:42:46.066 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00024105200000000001 2024-09-21T21:42:46.066 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.066 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.066 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:46.066 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825946+0000", 2024-09-21T21:42:46.066 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.743e-06 2024-09-21T21:42:46.066 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.066 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.066 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:46.066 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825956+0000", 2024-09-21T21:42:46.066 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6609999999999995e-06 2024-09-21T21:42:46.066 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.066 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.066 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:46.067 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825963+0000", 2024-09-21T21:42:46.067 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4609999999999996e-06 2024-09-21T21:42:46.067 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.067 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.067 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:46.067 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897502+0000", 2024-09-21T21:42:46.067 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071539354999999999 2024-09-21T21:42:46.067 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.067 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.067 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:46.067 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897619+0000", 2024-09-21T21:42:46.067 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011665 2024-09-21T21:42:46.067 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.067 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.068 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:46.068 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897782+0000", 2024-09-21T21:42:46.068 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00016345500000000001 2024-09-21T21:42:46.068 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:46.068 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:46.068 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:46.068 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.068 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.068 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13575 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 380928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:46.068 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.825973+0000", 2024-09-21T21:42:46.068 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 55.200639785, 2024-09-21T21:42:46.068 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072012723000000001, 2024-09-21T21:42:46.068 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:46.068 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:46.068 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:46.068 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:46.069 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:46.069 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:46.069 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13575 2024-09-21T21:42:46.069 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.069 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:46.069 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.069 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:46.069 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825973+0000", 2024-09-21T21:42:46.069 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:46.069 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.069 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.069 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:46.069 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825973+0000", 2024-09-21T21:42:46.070 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:46.070 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.070 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.070 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:46.070 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826746+0000", 2024-09-21T21:42:46.070 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00077326800000000002 2024-09-21T21:42:46.070 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.070 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.070 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:46.070 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826766+0000", 2024-09-21T21:42:46.070 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9465000000000001e-05 2024-09-21T21:42:46.070 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.070 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.070 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:46.070 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826812+0000", 2024-09-21T21:42:46.071 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5927999999999998e-05 2024-09-21T21:42:46.071 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.071 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.071 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:46.071 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826833+0000", 2024-09-21T21:42:46.071 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1710999999999999e-05 2024-09-21T21:42:46.071 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.071 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.071 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:46.071 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897809+0000", 2024-09-21T21:42:46.071 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.070975229000000001 2024-09-21T21:42:46.071 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.071 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.072 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:46.072 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897864+0000", 2024-09-21T21:42:46.072 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5334999999999998e-05 2024-09-21T21:42:46.072 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.072 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.072 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:46.072 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897986+0000", 2024-09-21T21:42:46.072 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000121787 2024-09-21T21:42:46.072 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:46.072 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:46.072 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:46.072 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.072 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.073 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13577 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 1814528~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:46.073 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.826874+0000", 2024-09-21T21:42:46.073 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 55.199738750999998, 2024-09-21T21:42:46.073 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071639123999999998, 2024-09-21T21:42:46.073 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:46.073 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:46.073 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:46.073 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:46.073 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:46.073 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:46.073 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13577 2024-09-21T21:42:46.073 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.073 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:46.074 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.074 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:46.074 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826874+0000", 2024-09-21T21:42:46.074 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:46.074 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.074 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.074 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:46.074 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826874+0000", 2024-09-21T21:42:46.074 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:46.074 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.074 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.074 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:46.074 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826875+0000", 2024-09-21T21:42:46.075 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.82e-07 2024-09-21T21:42:46.075 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.075 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.075 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:46.075 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826877+0000", 2024-09-21T21:42:46.075 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.295e-06 2024-09-21T21:42:46.075 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.075 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.075 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:46.075 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826883+0000", 2024-09-21T21:42:46.075 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2609999999999999e-06 2024-09-21T21:42:46.075 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.075 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.075 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:46.076 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826889+0000", 2024-09-21T21:42:46.076 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5589999999999996e-06 2024-09-21T21:42:46.076 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.076 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.076 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:46.076 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898301+0000", 2024-09-21T21:42:46.076 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071412188000000001 2024-09-21T21:42:46.076 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.076 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.076 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:46.076 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898416+0000", 2024-09-21T21:42:46.076 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000115538 2024-09-21T21:42:46.076 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.076 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.077 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:46.077 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898513+0000", 2024-09-21T21:42:46.077 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6800999999999997e-05 2024-09-21T21:42:46.077 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:46.077 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:46.077 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:46.077 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.077 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.077 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13578 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 2502656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:46.077 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.826896+0000", 2024-09-21T21:42:46.077 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 55.199717294999999, 2024-09-21T21:42:46.078 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071789411999999997, 2024-09-21T21:42:46.078 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:46.078 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:46.078 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:46.078 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:46.078 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:46.078 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:46.078 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13578 2024-09-21T21:42:46.078 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.078 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:46.078 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.078 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:46.078 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826896+0000", 2024-09-21T21:42:46.078 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:46.079 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.079 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.079 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:46.079 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826896+0000", 2024-09-21T21:42:46.079 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:46.079 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.079 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.079 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:46.079 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827327+0000", 2024-09-21T21:42:46.079 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00043094500000000002 2024-09-21T21:42:46.079 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.079 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.079 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:46.079 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827331+0000", 2024-09-21T21:42:46.080 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.0969999999999999e-06 2024-09-21T21:42:46.080 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.080 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.080 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:46.080 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827341+0000", 2024-09-21T21:42:46.080 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0199e-05 2024-09-21T21:42:46.080 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.080 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.080 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:46.080 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827346+0000", 2024-09-21T21:42:46.080 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3210000000000004e-06 2024-09-21T21:42:46.080 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.080 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.080 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:46.081 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898540+0000", 2024-09-21T21:42:46.081 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071194246000000003 2024-09-21T21:42:46.081 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.081 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.081 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:46.081 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898598+0000", 2024-09-21T21:42:46.081 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7658999999999998e-05 2024-09-21T21:42:46.081 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.081 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.081 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:46.081 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898685+0000", 2024-09-21T21:42:46.081 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.6945000000000002e-05 2024-09-21T21:42:46.081 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:46.081 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:46.081 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:46.082 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.082 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.082 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13579 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 2531328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:46.082 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.827355+0000", 2024-09-21T21:42:46.082 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 55.199257764000002, 2024-09-21T21:42:46.082 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071650933, 2024-09-21T21:42:46.082 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:46.082 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:46.082 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:46.082 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:46.082 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:46.082 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:46.082 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13579 2024-09-21T21:42:46.082 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.083 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:46.083 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.083 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:46.083 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827355+0000", 2024-09-21T21:42:46.083 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:46.083 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.083 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.083 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:46.083 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827355+0000", 2024-09-21T21:42:46.083 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:46.083 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.083 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.083 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:46.083 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827486+0000", 2024-09-21T21:42:46.083 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00013066799999999999 2024-09-21T21:42:46.084 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.084 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.084 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:46.084 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827490+0000", 2024-09-21T21:42:46.084 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.9009999999999996e-06 2024-09-21T21:42:46.084 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.084 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.084 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:46.084 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827499+0000", 2024-09-21T21:42:46.084 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6209999999999999e-06 2024-09-21T21:42:46.084 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.084 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.084 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:46.085 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827506+0000", 2024-09-21T21:42:46.085 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6490000000000002e-06 2024-09-21T21:42:46.085 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.085 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.085 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:46.085 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898713+0000", 2024-09-21T21:42:46.085 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071206937999999997 2024-09-21T21:42:46.085 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.085 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.085 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:46.085 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898811+0000", 2024-09-21T21:42:46.085 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.8159999999999995e-05 2024-09-21T21:42:46.085 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.085 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.085 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:46.086 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899006+0000", 2024-09-21T21:42:46.086 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00019499599999999999 2024-09-21T21:42:46.086 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:46.086 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:46.086 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:46.086 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.086 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.086 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13580 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3219456~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:46.086 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.827522+0000", 2024-09-21T21:42:46.086 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 55.199091332999998, 2024-09-21T21:42:46.086 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071813298999999997, 2024-09-21T21:42:46.086 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:46.086 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:46.086 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:46.087 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:46.087 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:46.087 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:46.087 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13580 2024-09-21T21:42:46.087 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.087 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:46.087 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.087 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:46.087 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827522+0000", 2024-09-21T21:42:46.087 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:46.087 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.087 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.087 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:46.087 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827522+0000", 2024-09-21T21:42:46.088 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:46.088 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.088 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.088 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:46.088 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827898+0000", 2024-09-21T21:42:46.088 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000376876 2024-09-21T21:42:46.088 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.088 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.088 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:46.088 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827903+0000", 2024-09-21T21:42:46.088 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4129999999999999e-06 2024-09-21T21:42:46.088 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.088 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.089 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:46.089 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827911+0000", 2024-09-21T21:42:46.089 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.4540000000000007e-06 2024-09-21T21:42:46.089 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.089 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.089 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:46.089 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827917+0000", 2024-09-21T21:42:46.089 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2110000000000003e-06 2024-09-21T21:42:46.089 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.089 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.089 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:46.089 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899041+0000", 2024-09-21T21:42:46.089 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071123965999999997 2024-09-21T21:42:46.090 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.090 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.090 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:46.090 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899157+0000", 2024-09-21T21:42:46.090 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011573700000000001 2024-09-21T21:42:46.090 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.090 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.090 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:46.090 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899335+0000", 2024-09-21T21:42:46.090 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00017764199999999999 2024-09-21T21:42:46.090 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:46.090 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:46.090 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:46.090 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.091 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.091 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13581 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3248128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:46.091 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.827929+0000", 2024-09-21T21:42:46.091 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 55.198683934000002, 2024-09-21T21:42:46.091 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071730151000000006, 2024-09-21T21:42:46.091 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:46.091 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:46.091 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:46.091 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:46.091 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:46.091 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:46.091 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13581 2024-09-21T21:42:46.091 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.092 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:46.092 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.092 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:46.092 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827929+0000", 2024-09-21T21:42:46.092 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:46.092 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.092 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.092 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:46.092 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827929+0000", 2024-09-21T21:42:46.092 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:46.092 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.092 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.092 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:46.092 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.829943+0000", 2024-09-21T21:42:46.093 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0020143750000000001 2024-09-21T21:42:46.093 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.093 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.093 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:46.093 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.829948+0000", 2024-09-21T21:42:46.093 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9400000000000001e-06 2024-09-21T21:42:46.093 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.093 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.093 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:46.093 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.829960+0000", 2024-09-21T21:42:46.093 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.2123999999999999e-05 2024-09-21T21:42:46.093 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.093 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.094 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:46.094 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.829968+0000", 2024-09-21T21:42:46.094 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7000000000000008e-06 2024-09-21T21:42:46.094 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.094 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.094 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:46.094 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899370+0000", 2024-09-21T21:42:46.094 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.069401672999999997 2024-09-21T21:42:46.094 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.094 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.094 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:46.094 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899481+0000", 2024-09-21T21:42:46.094 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011145300000000001 2024-09-21T21:42:46.095 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.095 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.095 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:46.095 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899659+0000", 2024-09-21T21:42:46.095 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00017788600000000001 2024-09-21T21:42:46.095 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:46.095 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:46.095 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:46.095 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.095 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.095 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13595 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 1871872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:46.095 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.830180+0000", 2024-09-21T21:42:46.095 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 55.196432819999998, 2024-09-21T21:42:46.095 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071809244999999994, 2024-09-21T21:42:46.095 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:46.096 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:46.096 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:46.096 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:46.096 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:46.096 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:46.096 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13595 2024-09-21T21:42:46.096 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.096 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:46.096 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.096 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:46.096 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.830180+0000", 2024-09-21T21:42:46.096 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:46.096 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.096 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.096 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:46.096 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.830180+0000", 2024-09-21T21:42:46.097 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:46.097 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.097 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.097 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:46.097 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.831967+0000", 2024-09-21T21:42:46.097 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0017868350000000001 2024-09-21T21:42:46.097 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.097 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.097 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:46.097 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.831971+0000", 2024-09-21T21:42:46.097 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3950000000000003e-06 2024-09-21T21:42:46.097 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.097 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.098 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:46.098 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.831983+0000", 2024-09-21T21:42:46.098 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1815e-05 2024-09-21T21:42:46.098 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.098 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.098 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:46.098 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.831993+0000", 2024-09-21T21:42:46.098 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6339999999999996e-06 2024-09-21T21:42:46.098 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.098 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.098 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:46.098 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.901693+0000", 2024-09-21T21:42:46.098 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.069700082999999996 2024-09-21T21:42:46.099 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.099 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.099 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:46.099 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.901813+0000", 2024-09-21T21:42:46.099 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011971 2024-09-21T21:42:46.099 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.099 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.099 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:46.099 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.901989+0000", 2024-09-21T21:42:46.099 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000176773 2024-09-21T21:42:46.099 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:46.099 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:46.099 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:46.099 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.099 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.100 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13601 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3334144~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:46.100 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.832130+0000", 2024-09-21T21:42:46.100 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 55.194482948000001, 2024-09-21T21:42:46.100 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071861727, 2024-09-21T21:42:46.100 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:46.101 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:46.101 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:46.101 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:46.102 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:46.102 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:46.102 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13601 2024-09-21T21:42:46.102 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.102 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:46.102 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.102 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:46.102 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.832130+0000", 2024-09-21T21:42:46.102 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:46.102 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.102 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.102 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:46.102 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.832130+0000", 2024-09-21T21:42:46.102 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:46.103 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.103 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.103 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:46.103 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.833939+0000", 2024-09-21T21:42:46.103 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0018094400000000001 2024-09-21T21:42:46.103 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.103 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.103 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:46.103 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.833944+0000", 2024-09-21T21:42:46.103 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5979999999999999e-06 2024-09-21T21:42:46.103 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.103 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.103 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:46.103 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.833953+0000", 2024-09-21T21:42:46.103 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.1460000000000002e-06 2024-09-21T21:42:46.104 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.104 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.104 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:46.104 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.833960+0000", 2024-09-21T21:42:46.104 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.7989999999999996e-06 2024-09-21T21:42:46.104 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.104 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.104 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:46.104 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.903693+0000", 2024-09-21T21:42:46.104 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.069732901 2024-09-21T21:42:46.104 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.104 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.104 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:46.104 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.903814+0000", 2024-09-21T21:42:46.104 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000121585 2024-09-21T21:42:46.104 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.105 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.105 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:46.105 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.903992+0000", 2024-09-21T21:42:46.105 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00017725799999999999 2024-09-21T21:42:46.105 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:46.105 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:46.105 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:46.105 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.105 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.105 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:12342 3.1 3:87417293:::rbd_data.1121faebdfe3.0000000000000004:head [read 4096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:42:46.105 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:14.411725+0000", 2024-09-21T21:42:46.105 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 31.614887882000001, 2024-09-21T21:42:46.105 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0089213029999999, 2024-09-21T21:42:46.105 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:46.105 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:46.106 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:46.106 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:46.106 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:46.106 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:46.106 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 12342 2024-09-21T21:42:46.106 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.106 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:46.106 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.106 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:46.106 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411725+0000", 2024-09-21T21:42:46.106 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:46.106 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.106 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.106 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:46.106 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411725+0000", 2024-09-21T21:42:46.107 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:46.107 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.107 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.107 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:46.107 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411726+0000", 2024-09-21T21:42:46.107 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.4169999999999999e-06 2024-09-21T21:42:46.107 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.107 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.107 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:46.107 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411734+0000", 2024-09-21T21:42:46.107 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.3799999999999996e-06 2024-09-21T21:42:46.107 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.107 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.107 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:46.107 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411750+0000", 2024-09-21T21:42:46.108 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.6429000000000001e-05 2024-09-21T21:42:46.108 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.108 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.108 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:46.108 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411760+0000", 2024-09-21T21:42:46.108 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.4239999999999999e-06 2024-09-21T21:42:46.108 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.108 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.108 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:46.108 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411828+0000", 2024-09-21T21:42:46.108 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.7935999999999995e-05 2024-09-21T21:42:46.108 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.108 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.108 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op must wait for map", 2024-09-21T21:42:46.109 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411837+0000", 2024-09-21T21:42:46.109 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.0100000000000001e-06 2024-09-21T21:42:46.109 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.109 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.109 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:46.109 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.414984+0000", 2024-09-21T21:42:46.109 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0031472050000000001 2024-09-21T21:42:46.109 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.109 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.109 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for peered", 2024-09-21T21:42:46.109 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.414994+0000", 2024-09-21T21:42:46.109 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0291e-05 2024-09-21T21:42:46.110 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.110 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.110 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:46.110 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.420353+0000", 2024-09-21T21:42:46.110 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.005358913 2024-09-21T21:42:46.110 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.110 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.110 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:46.110 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.420489+0000", 2024-09-21T21:42:46.110 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00013592699999999999 2024-09-21T21:42:46.110 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.110 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.110 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:46.111 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.420646+0000", 2024-09-21T21:42:46.111 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00015737100000000001 2024-09-21T21:42:46.111 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:46.111 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:46.111 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:46.111 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.111 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.111 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:12348 3.1 3:87417293:::rbd_data.1121faebdfe3.0000000000000004:head [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:42:46.111 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:14.411793+0000", 2024-09-21T21:42:46.111 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 31.614820336000001, 2024-09-21T21:42:46.111 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.009298384, 2024-09-21T21:42:46.111 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:46.111 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:46.111 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:46.112 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:46.112 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:46.112 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:46.112 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 12348 2024-09-21T21:42:46.112 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.112 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:46.112 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.112 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:46.112 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411793+0000", 2024-09-21T21:42:46.112 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:46.112 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.112 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.112 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:46.112 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411793+0000", 2024-09-21T21:42:46.112 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:46.113 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.113 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.113 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:46.113 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411793+0000", 2024-09-21T21:42:46.113 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0999999999999998e-07 2024-09-21T21:42:46.113 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.113 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.113 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:46.113 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411797+0000", 2024-09-21T21:42:46.113 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.49e-06 2024-09-21T21:42:46.113 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.113 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.113 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:46.113 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411809+0000", 2024-09-21T21:42:46.114 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.2510999999999999e-05 2024-09-21T21:42:46.114 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.114 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.114 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:46.114 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411816+0000", 2024-09-21T21:42:46.114 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6420000000000001e-06 2024-09-21T21:42:46.114 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.114 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.114 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:46.114 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411963+0000", 2024-09-21T21:42:46.114 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00014700999999999999 2024-09-21T21:42:46.114 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.114 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.114 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting_for_map not empty", 2024-09-21T21:42:46.115 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411970+0000", 2024-09-21T21:42:46.115 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9800000000000001e-06 2024-09-21T21:42:46.115 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.115 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.115 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:46.115 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.415018+0000", 2024-09-21T21:42:46.115 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0030482299999999999 2024-09-21T21:42:46.115 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.115 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.115 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for peered", 2024-09-21T21:42:46.115 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.415025+0000", 2024-09-21T21:42:46.115 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.3749999999999997e-06 2024-09-21T21:42:46.115 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.115 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.116 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:46.116 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.420904+0000", 2024-09-21T21:42:46.116 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0058787339999999 2024-09-21T21:42:46.116 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.116 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.116 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:46.116 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.420988+0000", 2024-09-21T21:42:46.116 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.3955999999999994e-05 2024-09-21T21:42:46.116 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.116 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.116 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:46.116 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.421091+0000", 2024-09-21T21:42:46.116 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000102846 2024-09-21T21:42:46.117 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:46.117 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:46.117 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:46.117 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.117 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.117 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4385.0:20922 3.9 3:920c5256:::rbd_id.image_client.0-clone36:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-21T21:42:46.117 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:17.811324+0000", 2024-09-21T21:42:46.117 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 28.215288604000001, 2024-09-21T21:42:46.117 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.66951882299999999, 2024-09-21T21:42:46.117 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:46.117 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:46.117 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:46.117 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:46.118 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4385", 2024-09-21T21:42:46.118 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:42:46.118 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 20922 2024-09-21T21:42:46.118 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.118 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:46.118 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.118 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:46.118 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:17.811324+0000", 2024-09-21T21:42:46.118 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:46.118 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.118 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.118 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:46.118 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:17.811324+0000", 2024-09-21T21:42:46.119 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:46.119 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.119 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.119 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:46.119 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:17.811326+0000", 2024-09-21T21:42:46.119 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.536e-06 2024-09-21T21:42:46.119 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.119 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.119 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:46.119 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:17.811332+0000", 2024-09-21T21:42:46.119 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6699999999999997e-06 2024-09-21T21:42:46.119 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.119 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.120 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:46.120 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:17.811347+0000", 2024-09-21T21:42:46.120 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.4589999999999999e-05 2024-09-21T21:42:46.120 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.120 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.120 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:46.120 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:17.811359+0000", 2024-09-21T21:42:46.120 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1776e-05 2024-09-21T21:42:46.120 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.120 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.120 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:46.120 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:17.811424+0000", 2024-09-21T21:42:46.120 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4795000000000006e-05 2024-09-21T21:42:46.120 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.121 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.121 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for peered", 2024-09-21T21:42:46.121 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:17.811436+0000", 2024-09-21T21:42:46.121 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.221e-05 2024-09-21T21:42:46.121 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.121 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.121 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:46.121 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:18.480657+0000", 2024-09-21T21:42:46.121 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.66922088999999996 2024-09-21T21:42:46.121 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.121 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.121 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:46.121 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:18.480760+0000", 2024-09-21T21:42:46.121 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000102827 2024-09-21T21:42:46.122 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:46.122 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:46.122 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:46.122 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:18.480843+0000", 2024-09-21T21:42:46.122 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.3529000000000006e-05 2024-09-21T21:42:46.122 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:46.122 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:46.122 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:46.122 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:46.122 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:46.122 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:46.124 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T21:42:46.124 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:46.122+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:46.224 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:46.224+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:46.241 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:46.241 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:42:46.241 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-21T21:42:46.241 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:46.250 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T21:42:46.326 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:46.324+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:46.355 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:46.355 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:42:46.355 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:42:46.355 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:42:46.355 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:46.364 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T21:42:46.426 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:46.425+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:46.469 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:46.469 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-21T21:42:46.470 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-21T21:42:46.470 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:42:46.470 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.470 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19747 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 7905280~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:46.470 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273058+0000", 2024-09-21T21:42:46.470 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 55.193355062000002, 2024-09-21T21:42:46.471 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97338591399999996, 2024-09-21T21:42:46.471 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:46.471 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:46.471 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:46.471 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:46.471 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:46.472 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:46.472 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19747 2024-09-21T21:42:46.472 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.472 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:46.472 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.472 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:46.472 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273058+0000", 2024-09-21T21:42:46.473 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:46.473 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.473 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.473 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:46.473 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273058+0000", 2024-09-21T21:42:46.473 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:46.473 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.474 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.474 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:46.474 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273058+0000", 2024-09-21T21:42:46.474 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5900000000000003e-07 2024-09-21T21:42:46.474 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.474 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.475 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:46.475 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273061+0000", 2024-09-21T21:42:46.475 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3479999999999998e-06 2024-09-21T21:42:46.475 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.475 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.475 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:46.476 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273067+0000", 2024-09-21T21:42:46.476 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1469999999999998e-06 2024-09-21T21:42:46.476 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.476 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.476 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:46.476 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273071+0000", 2024-09-21T21:42:46.476 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7339999999999999e-06 2024-09-21T21:42:46.477 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.477 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.477 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:46.477 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277945+0000", 2024-09-21T21:42:46.477 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.004873474 2024-09-21T21:42:46.477 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.478 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.478 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:46.478 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277948+0000", 2024-09-21T21:42:46.478 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0299999999999998e-06 2024-09-21T21:42:46.478 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.478 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.479 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:46.479 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246269+0000", 2024-09-21T21:42:46.479 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96832060200000003 2024-09-21T21:42:46.479 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.479 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.479 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:46.480 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246362+0000", 2024-09-21T21:42:46.480 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3867000000000001e-05 2024-09-21T21:42:46.480 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.480 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.480 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:46.480 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246444+0000", 2024-09-21T21:42:46.481 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1353000000000005e-05 2024-09-21T21:42:46.481 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:46.481 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:46.481 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:46.481 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.481 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.482 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19755 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 5783552~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:46.482 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273077+0000", 2024-09-21T21:42:46.482 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 55.193335445999999, 2024-09-21T21:42:46.482 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97350203400000002, 2024-09-21T21:42:46.482 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:46.482 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:46.483 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:46.483 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:46.483 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:46.483 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:46.483 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19755 2024-09-21T21:42:46.483 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.483 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:46.484 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.484 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:46.484 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273077+0000", 2024-09-21T21:42:46.484 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:46.484 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.484 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.484 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:46.485 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273077+0000", 2024-09-21T21:42:46.485 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:46.485 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.485 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.485 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:46.485 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273078+0000", 2024-09-21T21:42:46.485 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5699999999999998e-07 2024-09-21T21:42:46.485 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.485 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.486 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:46.486 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273083+0000", 2024-09-21T21:42:46.486 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8670000000000002e-06 2024-09-21T21:42:46.486 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.486 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.486 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:46.486 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273088+0000", 2024-09-21T21:42:46.486 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8370000000000002e-06 2024-09-21T21:42:46.486 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.486 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.486 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:46.486 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273093+0000", 2024-09-21T21:42:46.486 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5709999999999999e-06 2024-09-21T21:42:46.487 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.487 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.487 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:46.487 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277968+0000", 2024-09-21T21:42:46.487 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048753909999999998 2024-09-21T21:42:46.487 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.487 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.487 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:46.487 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277971+0000", 2024-09-21T21:42:46.487 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0189999999999998e-06 2024-09-21T21:42:46.487 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.487 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.487 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:46.487 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246485+0000", 2024-09-21T21:42:46.488 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96851399299999996 2024-09-21T21:42:46.488 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.488 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.488 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:46.488 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246536+0000", 2024-09-21T21:42:46.488 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0430000000000003e-05 2024-09-21T21:42:46.488 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.488 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.488 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:46.488 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246579+0000", 2024-09-21T21:42:46.488 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3569000000000002e-05 2024-09-21T21:42:46.488 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:46.488 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:46.488 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:46.489 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.489 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.489 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19756 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6500352~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:46.489 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273099+0000", 2024-09-21T21:42:46.489 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 55.193314073000003, 2024-09-21T21:42:46.489 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97359272799999996, 2024-09-21T21:42:46.489 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:46.489 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:46.489 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:46.489 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:46.489 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:46.489 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:46.489 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19756 2024-09-21T21:42:46.489 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.490 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:46.490 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.490 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:46.490 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273099+0000", 2024-09-21T21:42:46.490 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:46.490 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.490 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.490 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:46.491 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273099+0000", 2024-09-21T21:42:46.491 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:46.491 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.491 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.491 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:46.491 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273099+0000", 2024-09-21T21:42:46.491 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0900000000000002e-07 2024-09-21T21:42:46.491 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.491 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.491 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:46.491 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273101+0000", 2024-09-21T21:42:46.491 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0320000000000002e-06 2024-09-21T21:42:46.492 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.492 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.492 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:46.492 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273107+0000", 2024-09-21T21:42:46.492 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2830000000000001e-06 2024-09-21T21:42:46.492 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.492 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.492 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:46.492 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273111+0000", 2024-09-21T21:42:46.492 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2230000000000001e-06 2024-09-21T21:42:46.492 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.492 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.492 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:46.492 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277994+0000", 2024-09-21T21:42:46.493 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048837070000000002 2024-09-21T21:42:46.493 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.493 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.493 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:46.493 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278000+0000", 2024-09-21T21:42:46.493 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3149999999999999e-06 2024-09-21T21:42:46.493 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.493 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.493 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:46.493 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246607+0000", 2024-09-21T21:42:46.493 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.968606839 2024-09-21T21:42:46.493 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.493 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.493 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:46.494 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246651+0000", 2024-09-21T21:42:46.494 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4734999999999998e-05 2024-09-21T21:42:46.494 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.494 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.494 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:46.494 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246692+0000", 2024-09-21T21:42:46.494 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0185000000000002e-05 2024-09-21T21:42:46.494 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:46.494 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:46.494 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:46.494 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.494 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.495 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19757 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 5812224~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:46.495 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273116+0000", 2024-09-21T21:42:46.495 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 55.193296416000003, 2024-09-21T21:42:46.495 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97367813999999997, 2024-09-21T21:42:46.495 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:46.495 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:46.495 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:46.495 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:46.495 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:46.495 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:46.495 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19757 2024-09-21T21:42:46.495 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.495 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:46.495 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.496 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:46.496 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273116+0000", 2024-09-21T21:42:46.496 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:46.496 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.496 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.496 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:46.496 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273116+0000", 2024-09-21T21:42:46.496 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:46.496 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.496 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.496 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:46.496 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273117+0000", 2024-09-21T21:42:46.496 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0999999999999999e-07 2024-09-21T21:42:46.496 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.497 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.497 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:46.497 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273119+0000", 2024-09-21T21:42:46.497 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9970000000000001e-06 2024-09-21T21:42:46.497 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.497 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.497 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:46.497 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273125+0000", 2024-09-21T21:42:46.498 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9540000000000001e-06 2024-09-21T21:42:46.498 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.498 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.498 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:46.498 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273135+0000", 2024-09-21T21:42:46.498 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0597999999999999e-05 2024-09-21T21:42:46.498 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.498 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.498 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:46.498 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278019+0000", 2024-09-21T21:42:46.498 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048834849999999999 2024-09-21T21:42:46.498 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.498 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.499 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:46.499 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278022+0000", 2024-09-21T21:42:46.499 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9560000000000002e-06 2024-09-21T21:42:46.499 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.499 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.499 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:46.499 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246716+0000", 2024-09-21T21:42:46.499 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.968694202 2024-09-21T21:42:46.499 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.499 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.499 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:46.499 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246758+0000", 2024-09-21T21:42:46.499 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1607000000000003e-05 2024-09-21T21:42:46.499 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.500 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.500 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:46.500 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246795+0000", 2024-09-21T21:42:46.500 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6931000000000003e-05 2024-09-21T21:42:46.500 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:46.500 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:46.500 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:46.500 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.500 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.500 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19758 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6529024~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:46.501 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273142+0000", 2024-09-21T21:42:46.501 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 55.193271195000001, 2024-09-21T21:42:46.501 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97375924599999997, 2024-09-21T21:42:46.501 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:46.501 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:46.501 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:46.501 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:46.501 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:46.501 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:46.502 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19758 2024-09-21T21:42:46.502 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.502 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:46.502 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.502 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:46.502 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273142+0000", 2024-09-21T21:42:46.502 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:46.502 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.502 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.502 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:46.502 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273142+0000", 2024-09-21T21:42:46.502 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:46.502 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.503 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.503 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:46.503 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273142+0000", 2024-09-21T21:42:46.503 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0200000000000003e-07 2024-09-21T21:42:46.503 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.503 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.503 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:46.503 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273144+0000", 2024-09-21T21:42:46.503 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1390000000000002e-06 2024-09-21T21:42:46.503 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.503 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.503 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:46.503 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273150+0000", 2024-09-21T21:42:46.503 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6409999999999999e-06 2024-09-21T21:42:46.504 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.504 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.504 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:46.504 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273157+0000", 2024-09-21T21:42:46.504 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1359999999999999e-06 2024-09-21T21:42:46.504 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.504 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.504 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:46.504 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278050+0000", 2024-09-21T21:42:46.504 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048926170000000001 2024-09-21T21:42:46.504 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.504 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.504 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:46.504 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278053+0000", 2024-09-21T21:42:46.504 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4879999999999999e-06 2024-09-21T21:42:46.505 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.505 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.505 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:46.505 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246819+0000", 2024-09-21T21:42:46.505 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96876590399999996 2024-09-21T21:42:46.505 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.505 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.505 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:46.505 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246856+0000", 2024-09-21T21:42:46.505 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7287999999999999e-05 2024-09-21T21:42:46.505 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.505 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.505 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:46.505 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246901+0000", 2024-09-21T21:42:46.506 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4631e-05 2024-09-21T21:42:46.506 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:46.506 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:46.506 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:46.506 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.506 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.506 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19759 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 7933952~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:46.506 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273172+0000", 2024-09-21T21:42:46.506 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 55.193240930999998, 2024-09-21T21:42:46.506 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97384789500000002, 2024-09-21T21:42:46.506 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:46.506 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:46.506 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:46.506 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:46.507 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:46.507 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:46.507 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19759 2024-09-21T21:42:46.507 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.507 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:46.507 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.507 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:46.507 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273172+0000", 2024-09-21T21:42:46.507 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:46.507 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.507 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.507 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:46.507 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273172+0000", 2024-09-21T21:42:46.507 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:46.508 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.508 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.509 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:46.509 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273172+0000", 2024-09-21T21:42:46.509 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6100000000000002e-07 2024-09-21T21:42:46.509 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.509 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.510 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:46.510 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273174+0000", 2024-09-21T21:42:46.510 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.691e-06 2024-09-21T21:42:46.510 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.510 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.510 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:46.510 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273178+0000", 2024-09-21T21:42:46.510 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7579999999999998e-06 2024-09-21T21:42:46.510 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.510 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.510 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:46.510 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273180+0000", 2024-09-21T21:42:46.511 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7199999999999998e-06 2024-09-21T21:42:46.511 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.511 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.511 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:46.511 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278077+0000", 2024-09-21T21:42:46.511 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048969529999999999 2024-09-21T21:42:46.512 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.512 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.512 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:46.512 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278080+0000", 2024-09-21T21:42:46.512 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.066e-06 2024-09-21T21:42:46.512 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.512 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.513 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:46.513 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246925+0000", 2024-09-21T21:42:46.513 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96884490199999995 2024-09-21T21:42:46.513 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.513 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.513 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:46.513 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246974+0000", 2024-09-21T21:42:46.513 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8387999999999998e-05 2024-09-21T21:42:46.514 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.514 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.514 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:46.514 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247020+0000", 2024-09-21T21:42:46.514 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6156000000000003e-05 2024-09-21T21:42:46.514 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:46.515 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:46.515 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:46.515 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.515 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.515 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19760 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 7991296~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:46.515 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273220+0000", 2024-09-21T21:42:46.515 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 55.193192416999999, 2024-09-21T21:42:46.515 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97403125499999998, 2024-09-21T21:42:46.515 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:46.515 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:46.515 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:46.516 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:46.516 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:46.516 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:46.516 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19760 2024-09-21T21:42:46.516 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.516 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:46.516 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.516 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:46.516 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273220+0000", 2024-09-21T21:42:46.516 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:46.516 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.516 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.516 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:46.516 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273220+0000", 2024-09-21T21:42:46.517 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:46.517 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.517 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.517 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:46.517 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273221+0000", 2024-09-21T21:42:46.517 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.65e-07 2024-09-21T21:42:46.517 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.517 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.517 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:46.517 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273228+0000", 2024-09-21T21:42:46.517 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6040000000000004e-06 2024-09-21T21:42:46.517 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.517 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.517 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:46.518 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273233+0000", 2024-09-21T21:42:46.518 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1569999999999999e-06 2024-09-21T21:42:46.518 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.518 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.518 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:46.518 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273238+0000", 2024-09-21T21:42:46.518 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0050000000000004e-06 2024-09-21T21:42:46.518 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.518 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.519 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:46.519 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278107+0000", 2024-09-21T21:42:46.519 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00486857 2024-09-21T21:42:46.519 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.519 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.519 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:46.519 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278111+0000", 2024-09-21T21:42:46.519 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6389999999999998e-06 2024-09-21T21:42:46.519 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.519 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.519 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:46.520 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247069+0000", 2024-09-21T21:42:46.520 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96895796899999997 2024-09-21T21:42:46.520 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.520 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.520 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:46.520 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247166+0000", 2024-09-21T21:42:46.520 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.7158999999999996e-05 2024-09-21T21:42:46.520 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.520 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.520 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:46.520 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247252+0000", 2024-09-21T21:42:46.520 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5886999999999999e-05 2024-09-21T21:42:46.520 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:46.521 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:46.521 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:46.521 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.521 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.521 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19761 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 7962624~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:46.521 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273246+0000", 2024-09-21T21:42:46.521 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 55.193167357, 2024-09-21T21:42:46.521 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97415972200000001, 2024-09-21T21:42:46.521 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:46.521 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:46.521 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:46.521 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:46.521 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:46.521 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:46.521 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19761 2024-09-21T21:42:46.522 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.522 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:46.522 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.522 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:46.522 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273246+0000", 2024-09-21T21:42:46.522 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:46.522 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.522 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.522 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:46.522 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273246+0000", 2024-09-21T21:42:46.522 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:46.522 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.523 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.523 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:46.523 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273246+0000", 2024-09-21T21:42:46.523 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5100000000000001e-07 2024-09-21T21:42:46.523 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.523 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.523 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:46.523 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273248+0000", 2024-09-21T21:42:46.523 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1550000000000001e-06 2024-09-21T21:42:46.523 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.523 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.523 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:46.523 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273254+0000", 2024-09-21T21:42:46.523 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.964e-06 2024-09-21T21:42:46.524 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.524 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.524 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:46.524 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273259+0000", 2024-09-21T21:42:46.524 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5499999999999996e-06 2024-09-21T21:42:46.524 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.524 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.524 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:46.524 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278130+0000", 2024-09-21T21:42:46.524 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048715779999999997 2024-09-21T21:42:46.524 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.524 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.524 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:46.524 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278133+0000", 2024-09-21T21:42:46.525 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8739999999999999e-06 2024-09-21T21:42:46.525 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.525 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.525 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:46.525 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247270+0000", 2024-09-21T21:42:46.525 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.969137099 2024-09-21T21:42:46.525 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.525 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.525 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:46.525 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247338+0000", 2024-09-21T21:42:46.525 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7717000000000002e-05 2024-09-21T21:42:46.525 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.525 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.526 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:46.526 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247405+0000", 2024-09-21T21:42:46.526 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7434000000000005e-05 2024-09-21T21:42:46.526 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:46.526 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:46.526 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:46.526 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.526 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.526 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19769 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 5840896~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:46.526 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273287+0000", 2024-09-21T21:42:46.527 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 55.193126073000002, 2024-09-21T21:42:46.527 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97435390200000005, 2024-09-21T21:42:46.527 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:46.527 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:46.527 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:46.527 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:46.527 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:46.527 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:46.527 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19769 2024-09-21T21:42:46.527 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.527 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:46.527 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.527 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:46.527 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273287+0000", 2024-09-21T21:42:46.528 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:46.528 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.528 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.528 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:46.528 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273287+0000", 2024-09-21T21:42:46.528 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:46.528 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.528 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.528 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:46.528 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273287+0000", 2024-09-21T21:42:46.528 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6699999999999999e-07 2024-09-21T21:42:46.528 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.529 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.529 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:46.529 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273289+0000", 2024-09-21T21:42:46.529 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.708e-06 2024-09-21T21:42:46.529 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.529 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.529 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:46.529 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273296+0000", 2024-09-21T21:42:46.529 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8109999999999997e-06 2024-09-21T21:42:46.529 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.529 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.529 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:46.529 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273299+0000", 2024-09-21T21:42:46.529 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0740000000000001e-06 2024-09-21T21:42:46.530 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.530 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.530 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:46.530 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278159+0000", 2024-09-21T21:42:46.530 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048603240000000001 2024-09-21T21:42:46.530 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.530 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.530 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:46.530 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278162+0000", 2024-09-21T21:42:46.530 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1930000000000002e-06 2024-09-21T21:42:46.530 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.530 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.530 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:46.530 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247444+0000", 2024-09-21T21:42:46.531 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96928199100000001 2024-09-21T21:42:46.531 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.531 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.531 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:46.531 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247533+0000", 2024-09-21T21:42:46.531 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8490000000000001e-05 2024-09-21T21:42:46.531 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.531 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.531 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:46.531 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247641+0000", 2024-09-21T21:42:46.531 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000107944 2024-09-21T21:42:46.531 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:46.531 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:46.531 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:46.532 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.532 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.532 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19770 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 5869568~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:46.532 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273313+0000", 2024-09-21T21:42:46.532 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 55.193099971000002, 2024-09-21T21:42:46.532 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97449038899999996, 2024-09-21T21:42:46.532 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:46.532 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:46.532 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:46.532 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:46.532 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:46.532 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:46.532 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19770 2024-09-21T21:42:46.532 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.533 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:46.533 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.533 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:46.533 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273313+0000", 2024-09-21T21:42:46.533 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:46.533 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.533 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.533 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:46.533 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273313+0000", 2024-09-21T21:42:46.533 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:46.533 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.533 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.533 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:46.533 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273313+0000", 2024-09-21T21:42:46.533 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5699999999999998e-07 2024-09-21T21:42:46.533 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.533 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.534 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:46.534 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273315+0000", 2024-09-21T21:42:46.534 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0099999999999998e-06 2024-09-21T21:42:46.534 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.534 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.534 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:46.534 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273321+0000", 2024-09-21T21:42:46.534 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7529999999999998e-06 2024-09-21T21:42:46.534 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.534 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.534 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:46.534 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273328+0000", 2024-09-21T21:42:46.534 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9009999999999997e-06 2024-09-21T21:42:46.534 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.535 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.535 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:46.535 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278183+0000", 2024-09-21T21:42:46.535 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048547770000000002 2024-09-21T21:42:46.535 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.535 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.535 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:46.535 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278186+0000", 2024-09-21T21:42:46.535 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3100000000000001e-06 2024-09-21T21:42:46.535 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.535 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.535 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:46.535 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247676+0000", 2024-09-21T21:42:46.536 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96949004299999997 2024-09-21T21:42:46.536 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.536 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.536 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:46.536 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247757+0000", 2024-09-21T21:42:46.536 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0479e-05 2024-09-21T21:42:46.536 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.536 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.536 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:46.536 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247803+0000", 2024-09-21T21:42:46.536 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6758999999999997e-05 2024-09-21T21:42:46.536 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:46.536 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:46.537 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:46.537 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.537 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.537 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19771 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6586368~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:46.537 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273335+0000", 2024-09-21T21:42:46.537 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 55.193078184000001, 2024-09-21T21:42:46.537 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97458947699999998, 2024-09-21T21:42:46.537 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:46.537 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:46.537 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:46.537 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:46.537 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:46.537 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:46.537 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19771 2024-09-21T21:42:46.537 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.537 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:46.538 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.538 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:46.538 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273335+0000", 2024-09-21T21:42:46.538 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:46.538 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.538 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.538 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:46.538 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273335+0000", 2024-09-21T21:42:46.538 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:46.538 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.538 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.538 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:46.538 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273335+0000", 2024-09-21T21:42:46.538 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3599999999999999e-07 2024-09-21T21:42:46.539 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.539 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.539 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:46.539 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273337+0000", 2024-09-21T21:42:46.539 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0789999999999999e-06 2024-09-21T21:42:46.539 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.539 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.539 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:46.539 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273343+0000", 2024-09-21T21:42:46.539 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6280000000000002e-06 2024-09-21T21:42:46.539 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.539 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.539 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:46.539 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273346+0000", 2024-09-21T21:42:46.540 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.507e-06 2024-09-21T21:42:46.540 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.540 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.540 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:46.540 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278206+0000", 2024-09-21T21:42:46.540 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048600969999999999 2024-09-21T21:42:46.540 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.540 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.540 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:46.540 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278212+0000", 2024-09-21T21:42:46.540 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2079999999999999e-06 2024-09-21T21:42:46.540 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.540 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.540 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:46.540 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247828+0000", 2024-09-21T21:42:46.541 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.969616547 2024-09-21T21:42:46.541 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.541 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.541 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:46.541 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247881+0000", 2024-09-21T21:42:46.541 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3180999999999998e-05 2024-09-21T21:42:46.541 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.541 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.541 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:46.541 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247924+0000", 2024-09-21T21:42:46.541 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2894e-05 2024-09-21T21:42:46.541 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:46.541 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:46.541 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:46.541 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.542 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.542 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19772 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6557696~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:46.542 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273358+0000", 2024-09-21T21:42:46.542 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 55.193054803999999, 2024-09-21T21:42:46.542 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97467031800000004, 2024-09-21T21:42:46.542 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:46.542 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:46.542 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:46.543 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:46.543 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:46.543 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:46.543 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19772 2024-09-21T21:42:46.543 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.543 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:46.543 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.543 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:46.543 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273358+0000", 2024-09-21T21:42:46.543 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:46.543 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.543 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.543 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:46.543 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273358+0000", 2024-09-21T21:42:46.544 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:46.544 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.544 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.544 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:46.544 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273358+0000", 2024-09-21T21:42:46.544 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4499999999999998e-07 2024-09-21T21:42:46.544 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.544 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.544 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:46.544 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273360+0000", 2024-09-21T21:42:46.544 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9190000000000002e-06 2024-09-21T21:42:46.544 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.544 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.544 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:46.544 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273366+0000", 2024-09-21T21:42:46.544 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.891e-06 2024-09-21T21:42:46.545 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.545 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.545 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:46.545 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273371+0000", 2024-09-21T21:42:46.545 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1880000000000001e-06 2024-09-21T21:42:46.545 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.545 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.545 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:46.545 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278237+0000", 2024-09-21T21:42:46.545 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.004865472 2024-09-21T21:42:46.545 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.545 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.545 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:46.546 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278240+0000", 2024-09-21T21:42:46.546 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9550000000000001e-06 2024-09-21T21:42:46.546 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.546 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.546 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:46.546 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247949+0000", 2024-09-21T21:42:46.546 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.969709127 2024-09-21T21:42:46.546 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.546 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.546 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:46.547 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247990+0000", 2024-09-21T21:42:46.547 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1202000000000003e-05 2024-09-21T21:42:46.547 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.547 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.547 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:46.547 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248028+0000", 2024-09-21T21:42:46.547 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8219000000000001e-05 2024-09-21T21:42:46.547 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:46.547 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:46.547 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:46.547 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.547 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.548 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19773 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6615040~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:46.548 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273390+0000", 2024-09-21T21:42:46.548 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 55.193022943000003, 2024-09-21T21:42:46.548 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97476086200000001, 2024-09-21T21:42:46.548 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:46.548 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:46.548 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:46.548 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:46.548 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:46.548 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:46.548 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19773 2024-09-21T21:42:46.548 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.549 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:46.549 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.549 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:46.549 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273390+0000", 2024-09-21T21:42:46.549 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:46.549 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.549 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.549 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:46.549 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273390+0000", 2024-09-21T21:42:46.549 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:46.549 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.549 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.550 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:46.550 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273390+0000", 2024-09-21T21:42:46.550 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9999999999999999e-07 2024-09-21T21:42:46.550 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.550 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.550 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:46.550 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273392+0000", 2024-09-21T21:42:46.550 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0640000000000001e-06 2024-09-21T21:42:46.550 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.550 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.550 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:46.551 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273397+0000", 2024-09-21T21:42:46.551 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.113e-06 2024-09-21T21:42:46.551 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.551 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.551 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:46.551 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273401+0000", 2024-09-21T21:42:46.551 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3119999999999998e-06 2024-09-21T21:42:46.551 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.551 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.552 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:46.552 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278271+0000", 2024-09-21T21:42:46.552 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048698880000000002 2024-09-21T21:42:46.552 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.552 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.552 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:46.552 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278278+0000", 2024-09-21T21:42:46.552 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1210000000000001e-06 2024-09-21T21:42:46.552 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.552 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.552 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:46.552 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248060+0000", 2024-09-21T21:42:46.552 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96978224999999996 2024-09-21T21:42:46.552 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.553 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.553 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:46.553 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248111+0000", 2024-09-21T21:42:46.553 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.096e-05 2024-09-21T21:42:46.553 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.553 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.553 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:46.553 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248151+0000", 2024-09-21T21:42:46.553 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9854000000000002e-05 2024-09-21T21:42:46.553 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:46.553 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:46.553 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:46.553 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.553 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.554 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19784 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6643712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:46.554 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273405+0000", 2024-09-21T21:42:46.554 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 55.193008222000003, 2024-09-21T21:42:46.554 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97491798900000004, 2024-09-21T21:42:46.554 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:46.554 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:46.554 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:46.554 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:46.554 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:46.554 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:46.554 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19784 2024-09-21T21:42:46.555 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.555 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:46.555 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.555 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:46.555 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273405+0000", 2024-09-21T21:42:46.555 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:46.555 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.555 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.555 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:46.555 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273405+0000", 2024-09-21T21:42:46.555 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:46.555 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.555 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.556 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:46.556 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273405+0000", 2024-09-21T21:42:46.556 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2100000000000001e-07 2024-09-21T21:42:46.556 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.556 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.556 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:46.556 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273406+0000", 2024-09-21T21:42:46.556 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2049999999999999e-06 2024-09-21T21:42:46.556 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.556 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.556 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:46.556 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273409+0000", 2024-09-21T21:42:46.556 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0599999999999999e-06 2024-09-21T21:42:46.556 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.557 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.557 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:46.557 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273412+0000", 2024-09-21T21:42:46.557 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0730000000000001e-06 2024-09-21T21:42:46.557 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.557 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.557 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:46.557 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278317+0000", 2024-09-21T21:42:46.557 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0049045520000000004 2024-09-21T21:42:46.557 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.557 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.557 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:46.557 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278321+0000", 2024-09-21T21:42:46.557 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6249999999999998e-06 2024-09-21T21:42:46.557 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.558 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.558 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:46.558 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248179+0000", 2024-09-21T21:42:46.558 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96985744500000004 2024-09-21T21:42:46.558 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.558 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.558 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:46.558 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248253+0000", 2024-09-21T21:42:46.558 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3829000000000001e-05 2024-09-21T21:42:46.558 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.558 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.558 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:46.558 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248323+0000", 2024-09-21T21:42:46.559 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9979000000000002e-05 2024-09-21T21:42:46.559 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:46.559 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:46.559 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:46.559 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.559 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.559 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19785 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6672384~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:46.559 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273415+0000", 2024-09-21T21:42:46.559 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 55.192997636000001, 2024-09-21T21:42:46.559 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97500777800000005, 2024-09-21T21:42:46.559 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:46.559 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:46.559 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:46.560 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:46.560 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:46.560 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:46.560 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19785 2024-09-21T21:42:46.560 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.560 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:46.560 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.560 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:46.560 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273415+0000", 2024-09-21T21:42:46.560 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:46.560 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.560 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.560 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:46.560 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273415+0000", 2024-09-21T21:42:46.561 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:46.561 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.561 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.561 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:46.561 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273415+0000", 2024-09-21T21:42:46.561 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0100000000000001e-07 2024-09-21T21:42:46.561 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.561 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.562 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:46.562 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273417+0000", 2024-09-21T21:42:46.562 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.114e-06 2024-09-21T21:42:46.562 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.562 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.562 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:46.562 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273420+0000", 2024-09-21T21:42:46.562 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6770000000000001e-06 2024-09-21T21:42:46.562 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.562 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.562 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:46.562 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273423+0000", 2024-09-21T21:42:46.563 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5450000000000002e-06 2024-09-21T21:42:46.563 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.563 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.563 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:46.563 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278355+0000", 2024-09-21T21:42:46.563 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0049318590000000002 2024-09-21T21:42:46.563 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.563 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.563 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:46.563 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278360+0000", 2024-09-21T21:42:46.563 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3079999999999998e-06 2024-09-21T21:42:46.563 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.563 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.563 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:46.564 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248349+0000", 2024-09-21T21:42:46.564 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96998909899999997 2024-09-21T21:42:46.564 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.564 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.564 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:46.564 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248387+0000", 2024-09-21T21:42:46.564 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7657000000000002e-05 2024-09-21T21:42:46.564 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.564 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.564 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:46.564 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248423+0000", 2024-09-21T21:42:46.564 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6318e-05 2024-09-21T21:42:46.564 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:46.565 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:46.565 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:46.565 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.565 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.565 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19842 3.d 3:b7fa2ca0:::rbd_data.112157a479c.000000000000004d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:46.565 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273426+0000", 2024-09-21T21:42:46.565 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 55.192986584000003, 2024-09-21T21:42:46.565 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.98136526800000001, 2024-09-21T21:42:46.565 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:46.565 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:46.565 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:46.565 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:46.565 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:46.565 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:46.566 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19842 2024-09-21T21:42:46.566 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.566 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:46.566 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.566 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:46.566 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273426+0000", 2024-09-21T21:42:46.566 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:46.566 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.566 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.566 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:46.566 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273426+0000", 2024-09-21T21:42:46.566 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:46.566 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.566 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.567 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:46.567 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273427+0000", 2024-09-21T21:42:46.567 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.72e-07 2024-09-21T21:42:46.567 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.567 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.567 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:46.567 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275836+0000", 2024-09-21T21:42:46.567 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00240909 2024-09-21T21:42:46.567 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.567 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.567 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:46.567 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275844+0000", 2024-09-21T21:42:46.567 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3750000000000003e-06 2024-09-21T21:42:46.568 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.568 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.568 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:46.568 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275849+0000", 2024-09-21T21:42:46.568 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0100000000000003e-06 2024-09-21T21:42:46.568 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.568 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.568 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:46.568 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278394+0000", 2024-09-21T21:42:46.568 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025449890000000001 2024-09-21T21:42:46.568 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.568 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.568 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:46.568 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278402+0000", 2024-09-21T21:42:46.568 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6359999999999994e-06 2024-09-21T21:42:46.569 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.569 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.569 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:46.569 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248454+0000", 2024-09-21T21:42:46.569 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97005259499999996 2024-09-21T21:42:46.569 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.569 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.569 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:46.569 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248576+0000", 2024-09-21T21:42:46.569 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000122127 2024-09-21T21:42:46.569 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.569 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.569 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 2", 2024-09-21T21:42:46.569 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248844+0000", 2024-09-21T21:42:46.570 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00026784699999999999 2024-09-21T21:42:46.570 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.570 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.570 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:46.570 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.252487+0000", 2024-09-21T21:42:46.570 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0036427920000000002 2024-09-21T21:42:46.570 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.570 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.570 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:46.570 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.254691+0000", 2024-09-21T21:42:46.570 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002203899 2024-09-21T21:42:46.570 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.570 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.571 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:46.571 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.254748+0000", 2024-09-21T21:42:46.571 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7163000000000001e-05 2024-09-21T21:42:46.571 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.571 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.571 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:46.571 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.254792+0000", 2024-09-21T21:42:46.571 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3473000000000002e-05 2024-09-21T21:42:46.571 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:46.571 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:46.571 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:46.571 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.571 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.572 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19843 3.d 3:b7fa2ca0:::rbd_data.112157a479c.000000000000004d:head [write 0~0] snapc 21=[21] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:46.572 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.275860+0000", 2024-09-21T21:42:46.572 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 55.190552683999996, 2024-09-21T21:42:46.572 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.98183651999999999, 2024-09-21T21:42:46.572 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:46.572 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:46.572 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:46.572 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:46.572 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:46.572 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:46.572 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19843 2024-09-21T21:42:46.573 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.573 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:46.573 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.573 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:46.573 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275860+0000", 2024-09-21T21:42:46.573 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:46.573 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.573 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.573 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:46.573 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275860+0000", 2024-09-21T21:42:46.573 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:46.573 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.573 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.574 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:46.574 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275869+0000", 2024-09-21T21:42:46.574 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6300000000000004e-06 2024-09-21T21:42:46.574 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.574 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.574 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:46.574 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275871+0000", 2024-09-21T21:42:46.574 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2060000000000001e-06 2024-09-21T21:42:46.574 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.574 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.574 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:46.574 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275876+0000", 2024-09-21T21:42:46.574 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8180000000000003e-06 2024-09-21T21:42:46.575 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.575 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.575 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:46.575 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275880+0000", 2024-09-21T21:42:46.575 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1829999999999996e-06 2024-09-21T21:42:46.575 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.575 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.575 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:46.575 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278416+0000", 2024-09-21T21:42:46.575 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025361149999999998 2024-09-21T21:42:46.575 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.575 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.575 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:46.575 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278419+0000", 2024-09-21T21:42:46.576 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.163e-06 2024-09-21T21:42:46.576 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.576 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.576 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:46.576 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.250249+0000", 2024-09-21T21:42:46.576 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97182984299999997 2024-09-21T21:42:46.576 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.576 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.576 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:46.576 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.250321+0000", 2024-09-21T21:42:46.576 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1656000000000006e-05 2024-09-21T21:42:46.576 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.576 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.577 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 2", 2024-09-21T21:42:46.577 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.250419+0000", 2024-09-21T21:42:46.577 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.8277999999999997e-05 2024-09-21T21:42:46.577 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.577 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.577 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:46.577 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.252516+0000", 2024-09-21T21:42:46.578 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002097224 2024-09-21T21:42:46.578 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.578 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.578 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:46.578 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.257643+0000", 2024-09-21T21:42:46.578 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0051262319999999997 2024-09-21T21:42:46.578 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.579 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.579 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:46.579 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.257673+0000", 2024-09-21T21:42:46.579 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0023e-05 2024-09-21T21:42:46.579 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.579 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.579 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:46.579 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.257697+0000", 2024-09-21T21:42:46.579 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4148999999999999e-05 2024-09-21T21:42:46.579 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:46.579 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:46.579 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:46.579 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.579 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.580 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5626.0:12334 3.3 3:c643a86c:::rbd_data.1121faebdfe3.0000000000000002:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:42:46.580 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:14.411628+0000", 2024-09-21T21:42:46.580 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 32.054784384999998, 2024-09-21T21:42:46.580 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0086947559999999, 2024-09-21T21:42:46.580 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:46.580 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:46.580 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:46.580 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:46.580 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5626", 2024-09-21T21:42:46.580 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:46.580 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 12334 2024-09-21T21:42:46.581 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.581 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:46.581 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.581 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:46.581 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411628+0000", 2024-09-21T21:42:46.581 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:46.581 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.581 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.581 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:46.581 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411628+0000", 2024-09-21T21:42:46.581 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:46.581 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.581 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.581 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:46.582 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411629+0000", 2024-09-21T21:42:46.582 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1200000000000002e-07 2024-09-21T21:42:46.582 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.582 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.582 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:46.582 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411633+0000", 2024-09-21T21:42:46.582 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1699999999999999e-06 2024-09-21T21:42:46.582 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.582 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.582 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:46.583 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411645+0000", 2024-09-21T21:42:46.583 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1328999999999999e-05 2024-09-21T21:42:46.583 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.583 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.583 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:46.583 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411652+0000", 2024-09-21T21:42:46.583 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9990000000000002e-06 2024-09-21T21:42:46.583 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.583 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.583 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:46.583 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411717+0000", 2024-09-21T21:42:46.583 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5519999999999996e-05 2024-09-21T21:42:46.583 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.583 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.583 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op must wait for map", 2024-09-21T21:42:46.584 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411725+0000", 2024-09-21T21:42:46.584 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2609999999999996e-06 2024-09-21T21:42:46.584 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.584 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.584 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:46.584 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415904+0000", 2024-09-21T21:42:46.584 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0041792749999999997 2024-09-21T21:42:46.584 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.584 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.584 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:46.584 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415913+0000", 2024-09-21T21:42:46.584 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.054e-06 2024-09-21T21:42:46.584 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.584 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.584 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:46.584 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420096+0000", 2024-09-21T21:42:46.585 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0041835670000001 2024-09-21T21:42:46.585 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.585 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.585 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:46.585 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420216+0000", 2024-09-21T21:42:46.585 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000120025 2024-09-21T21:42:46.585 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.585 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.585 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:46.585 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420323+0000", 2024-09-21T21:42:46.585 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010674400000000001 2024-09-21T21:42:46.585 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:46.585 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:46.585 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:46.585 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.585 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.586 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5626.0:12339 3.3 3:c643a86c:::rbd_data.1121faebdfe3.0000000000000002:head [read 4096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:42:46.586 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:14.411749+0000", 2024-09-21T21:42:46.586 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 32.054664199000001, 2024-09-21T21:42:46.586 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.008818056, 2024-09-21T21:42:46.586 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:46.586 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:46.586 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:46.586 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:46.586 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5626", 2024-09-21T21:42:46.586 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:46.586 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 12339 2024-09-21T21:42:46.586 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.586 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:46.586 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.587 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:46.587 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411749+0000", 2024-09-21T21:42:46.587 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:46.587 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.587 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.587 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:46.587 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411749+0000", 2024-09-21T21:42:46.587 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:46.587 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.587 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.587 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:46.587 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411749+0000", 2024-09-21T21:42:46.587 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0900000000000002e-07 2024-09-21T21:42:46.587 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.587 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.587 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:46.588 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411752+0000", 2024-09-21T21:42:46.588 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5610000000000001e-06 2024-09-21T21:42:46.588 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.588 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.588 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:46.588 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411758+0000", 2024-09-21T21:42:46.588 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1890000000000004e-06 2024-09-21T21:42:46.588 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.588 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.588 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:46.588 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411763+0000", 2024-09-21T21:42:46.588 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7589999999999997e-06 2024-09-21T21:42:46.588 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.588 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.588 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:46.588 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411803+0000", 2024-09-21T21:42:46.589 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9922999999999997e-05 2024-09-21T21:42:46.589 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.589 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.589 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting_for_map not empty", 2024-09-21T21:42:46.589 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411809+0000", 2024-09-21T21:42:46.589 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3779999999999998e-06 2024-09-21T21:42:46.589 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.589 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.589 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:46.589 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415937+0000", 2024-09-21T21:42:46.589 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0041276100000000003 2024-09-21T21:42:46.589 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.589 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.589 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:46.589 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415944+0000", 2024-09-21T21:42:46.590 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8329999999999994e-06 2024-09-21T21:42:46.590 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.590 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.590 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:46.590 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420411+0000", 2024-09-21T21:42:46.590 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0044667300000001 2024-09-21T21:42:46.590 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.590 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.590 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:46.590 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420475+0000", 2024-09-21T21:42:46.590 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4356000000000005e-05 2024-09-21T21:42:46.590 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.590 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.590 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:46.591 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420567+0000", 2024-09-21T21:42:46.591 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.1308000000000001e-05 2024-09-21T21:42:46.591 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:46.591 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:46.591 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:46.591 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.591 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.591 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5626.0:12345 3.3 3:c643a86c:::rbd_data.1121faebdfe3.0000000000000002:head [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:42:46.591 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:14.411831+0000", 2024-09-21T21:42:46.591 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 32.054582146000001, 2024-09-21T21:42:46.591 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0089163649999999, 2024-09-21T21:42:46.591 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:46.591 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:46.591 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:46.591 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:46.592 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5626", 2024-09-21T21:42:46.592 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:46.592 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 12345 2024-09-21T21:42:46.592 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.592 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:46.592 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.592 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:46.592 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411831+0000", 2024-09-21T21:42:46.592 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:46.592 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.592 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.592 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:46.592 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411831+0000", 2024-09-21T21:42:46.592 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:46.592 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.593 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.593 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:46.593 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411831+0000", 2024-09-21T21:42:46.593 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.08e-07 2024-09-21T21:42:46.593 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.593 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.593 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:46.593 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411833+0000", 2024-09-21T21:42:46.593 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3089999999999998e-06 2024-09-21T21:42:46.593 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.593 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.593 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:46.594 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411840+0000", 2024-09-21T21:42:46.594 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.472e-06 2024-09-21T21:42:46.594 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.594 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.594 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:46.594 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411845+0000", 2024-09-21T21:42:46.594 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7910000000000004e-06 2024-09-21T21:42:46.594 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.594 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.594 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:46.594 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411890+0000", 2024-09-21T21:42:46.594 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5763000000000002e-05 2024-09-21T21:42:46.594 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.594 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.595 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting_for_map not empty", 2024-09-21T21:42:46.595 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411896+0000", 2024-09-21T21:42:46.595 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0190000000000003e-06 2024-09-21T21:42:46.595 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.595 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.595 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:46.595 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415967+0000", 2024-09-21T21:42:46.595 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0040705749999999999 2024-09-21T21:42:46.595 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.595 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.595 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:46.595 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415974+0000", 2024-09-21T21:42:46.595 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3200000000000002e-06 2024-09-21T21:42:46.595 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.595 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.596 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:46.596 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420594+0000", 2024-09-21T21:42:46.596 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.004620093 2024-09-21T21:42:46.596 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.596 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.596 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:46.596 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420658+0000", 2024-09-21T21:42:46.596 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3678999999999998e-05 2024-09-21T21:42:46.596 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:46.596 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:46.596 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:46.596 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420747+0000", 2024-09-21T21:42:46.596 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8936000000000004e-05 2024-09-21T21:42:46.596 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:46.596 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:46.596 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:46.597 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:46.597 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:46.597 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:46.598 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T21:42:46.599 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:46.597+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:46.699 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:46.697+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:46.750 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:46.750 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:42:46.750 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-21T21:42:46.750 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:46.757 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T21:42:46.760 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T21:42:46.804 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:46.803+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:46.864 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:46.864 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:42:46.864 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:42:46.865 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:42:46.865 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:46.873 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T21:42:46.874 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0: {} 2024-09-21T21:42:46.875 INFO:teuthology.orchestra.run.smithi081.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T21:42:46.891 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1: {} 2024-09-21T21:42:46.891 INFO:teuthology.orchestra.run.smithi081.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T21:42:46.900 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:46.899+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:46.907 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2: {} 2024-09-21T21:42:46.907 INFO:teuthology.orchestra.run.smithi081.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T21:42:46.924 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3: {} 2024-09-21T21:42:46.924 INFO:teuthology.orchestra.run.smithi081.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T21:42:46.940 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4: {} 2024-09-21T21:42:46.940 INFO:teuthology.orchestra.run.smithi081.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T21:42:46.956 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5: {} 2024-09-21T21:42:46.957 INFO:teuthology.orchestra.run.smithi081.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T21:42:47.001 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:47.000+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:47.017 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:47.017 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:47.026 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T21:42:47.110 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:47.109+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:47.169 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:47.169 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:47.178 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T21:42:47.203 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:47.202+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:47.304 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:47.302+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:47.344 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:47.345 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:47.354 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T21:42:47.404 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:47.403+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:47.459 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:47.459 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:47.467 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T21:42:47.511 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:47.510+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:47.571 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:47.571 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:47.580 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T21:42:47.621 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:47.620+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:47.636 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T21:42:47.708 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:47.707+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:47.725 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:47.725 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:47.734 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T21:42:47.809 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:47.807+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:47.843 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:47.843 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:47.853 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T21:42:47.910 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:47.909+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:47.956 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:47.956 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:47.965 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T21:42:48.050 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:48.048+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:48.109 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:48.109 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:48.112 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:48.111+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:48.118 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T21:42:48.213 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:48.212+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:48.220 INFO:teuthology.orchestra.run.smithi081.stderr:noscrub is set 2024-09-21T21:42:48.272 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:48.272 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:48.282 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T21:42:48.314 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:48.313+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:48.415 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:48.413+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:48.425 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:48.425 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:48.434 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T21:42:48.515 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:48.515+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:48.537 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:48.537 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:48.546 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T21:42:48.616 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:48.614+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:48.650 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:48.650 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:48.659 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T21:42:48.717 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:48.715+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:48.775 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:48.775 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:48.785 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T21:42:48.817 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:48.816+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:48.889 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:48.889 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:48.898 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T21:42:48.918 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:48.918+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:48.969 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T21:42:49.001 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:49.001 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:49.010 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T21:42:49.019 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:49.018+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:49.092 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0: {} 2024-09-21T21:42:49.092 INFO:teuthology.orchestra.run.smithi081.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T21:42:49.108 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1: {} 2024-09-21T21:42:49.109 INFO:teuthology.orchestra.run.smithi081.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T21:42:49.114 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:49.114 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:49.120 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:49.118+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:49.123 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T21:42:49.124 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2: {} 2024-09-21T21:42:49.124 INFO:teuthology.orchestra.run.smithi081.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T21:42:49.141 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3: {} 2024-09-21T21:42:49.141 INFO:teuthology.orchestra.run.smithi081.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T21:42:49.157 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4: {} 2024-09-21T21:42:49.157 INFO:teuthology.orchestra.run.smithi081.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T21:42:49.174 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5: {} 2024-09-21T21:42:49.174 INFO:teuthology.orchestra.run.smithi081.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T21:42:49.225 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:49.224+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:49.232 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:49.232 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:42:49.232 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:42:49.233 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:42:49.233 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:49.242 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T21:42:49.321 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:49.320+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:49.353 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:49.353 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-21T21:42:49.353 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-21T21:42:49.353 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:42:49.353 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.353 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6974 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5246976~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:49.354 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.922932+0000", 2024-09-21T21:42:49.354 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 79.428133067999994, 2024-09-21T21:42:49.354 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041773352999999999, 2024-09-21T21:42:49.354 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:49.354 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:49.354 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:49.354 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:49.354 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:49.354 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:49.354 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6974 2024-09-21T21:42:49.354 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.354 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:49.354 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.354 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:49.355 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922932+0000", 2024-09-21T21:42:49.355 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:49.355 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.355 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.355 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:49.355 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922932+0000", 2024-09-21T21:42:49.355 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:49.355 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.355 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.355 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:49.355 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922933+0000", 2024-09-21T21:42:49.355 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.46e-07 2024-09-21T21:42:49.355 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.356 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.356 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:49.356 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922940+0000", 2024-09-21T21:42:49.356 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6140000000000003e-06 2024-09-21T21:42:49.356 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.356 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.356 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:49.356 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922949+0000", 2024-09-21T21:42:49.356 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6899999999999998e-06 2024-09-21T21:42:49.356 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.356 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.356 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:49.356 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922955+0000", 2024-09-21T21:42:49.356 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5140000000000001e-06 2024-09-21T21:42:49.357 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.357 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.357 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:49.357 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964547+0000", 2024-09-21T21:42:49.357 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041592854999999998 2024-09-21T21:42:49.357 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.357 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.357 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:49.357 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964617+0000", 2024-09-21T21:42:49.357 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9058000000000002e-05 2024-09-21T21:42:49.357 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.357 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.357 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:49.357 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964706+0000", 2024-09-21T21:42:49.357 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9276000000000004e-05 2024-09-21T21:42:49.358 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:49.358 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:49.358 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:49.358 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.358 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.358 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6976 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5261312~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:49.358 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.922961+0000", 2024-09-21T21:42:49.358 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 79.428104023000003, 2024-09-21T21:42:49.358 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041863648000000003, 2024-09-21T21:42:49.358 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:49.358 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:49.358 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:49.358 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:49.358 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:49.359 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:49.359 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6976 2024-09-21T21:42:49.359 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.359 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:49.359 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.359 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:49.359 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922961+0000", 2024-09-21T21:42:49.359 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:49.359 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.359 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.359 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:49.359 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922961+0000", 2024-09-21T21:42:49.359 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:49.359 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.360 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.360 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:49.360 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922962+0000", 2024-09-21T21:42:49.360 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.89e-07 2024-09-21T21:42:49.360 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.360 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.360 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:49.360 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922964+0000", 2024-09-21T21:42:49.360 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1270000000000001e-06 2024-09-21T21:42:49.360 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.360 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.360 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:49.360 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922970+0000", 2024-09-21T21:42:49.360 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8030000000000002e-06 2024-09-21T21:42:49.361 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.361 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.361 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:49.361 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922979+0000", 2024-09-21T21:42:49.361 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5559999999999996e-06 2024-09-21T21:42:49.361 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.361 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.361 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:49.361 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964734+0000", 2024-09-21T21:42:49.361 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041754415000000003 2024-09-21T21:42:49.361 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.361 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.361 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:49.362 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964775+0000", 2024-09-21T21:42:49.362 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0806999999999997e-05 2024-09-21T21:42:49.362 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.362 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.362 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:49.362 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964825+0000", 2024-09-21T21:42:49.362 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0550999999999998e-05 2024-09-21T21:42:49.362 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:49.362 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:49.362 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:49.362 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.362 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.362 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6978 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5275648~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:49.362 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.922987+0000", 2024-09-21T21:42:49.363 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 79.428078237999998, 2024-09-21T21:42:49.363 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042043876000000001, 2024-09-21T21:42:49.363 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:49.363 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:49.363 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:49.363 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:49.363 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:49.363 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:49.363 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6978 2024-09-21T21:42:49.363 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.363 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:49.363 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.363 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:49.363 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922987+0000", 2024-09-21T21:42:49.363 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:49.363 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.364 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.364 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:49.364 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922987+0000", 2024-09-21T21:42:49.364 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:49.364 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.364 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.364 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:49.364 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922988+0000", 2024-09-21T21:42:49.364 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1199999999999998e-07 2024-09-21T21:42:49.364 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.364 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.364 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:49.364 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922990+0000", 2024-09-21T21:42:49.364 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.142e-06 2024-09-21T21:42:49.364 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.365 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.365 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:49.365 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922997+0000", 2024-09-21T21:42:49.365 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.199e-06 2024-09-21T21:42:49.365 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.365 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.365 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:49.365 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923004+0000", 2024-09-21T21:42:49.365 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3599999999999998e-06 2024-09-21T21:42:49.365 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.365 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.365 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:49.365 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964856+0000", 2024-09-21T21:42:49.365 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041851521000000003 2024-09-21T21:42:49.365 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.365 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.366 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:49.366 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964944+0000", 2024-09-21T21:42:49.366 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7859e-05 2024-09-21T21:42:49.366 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.366 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.366 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:49.366 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965031+0000", 2024-09-21T21:42:49.366 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7383e-05 2024-09-21T21:42:49.366 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:49.366 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:49.366 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:49.366 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.366 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.366 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6980 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5289984~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:49.366 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923011+0000", 2024-09-21T21:42:49.367 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 79.428054900999996, 2024-09-21T21:42:49.367 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042139337999999998, 2024-09-21T21:42:49.367 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:49.367 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:49.367 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:49.367 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:49.367 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:49.367 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:49.367 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6980 2024-09-21T21:42:49.367 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.367 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:49.367 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.367 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:49.367 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923011+0000", 2024-09-21T21:42:49.368 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:49.368 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.368 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.368 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:49.368 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923011+0000", 2024-09-21T21:42:49.368 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:49.368 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.368 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.368 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:49.368 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923011+0000", 2024-09-21T21:42:49.368 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6699999999999999e-07 2024-09-21T21:42:49.368 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.368 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.368 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:49.369 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923013+0000", 2024-09-21T21:42:49.369 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0279999999999999e-06 2024-09-21T21:42:49.369 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.369 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.369 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:49.369 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923018+0000", 2024-09-21T21:42:49.369 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4790000000000004e-06 2024-09-21T21:42:49.369 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.369 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.369 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:49.369 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923029+0000", 2024-09-21T21:42:49.369 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0601999999999999e-05 2024-09-21T21:42:49.369 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.369 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.369 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:49.370 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965059+0000", 2024-09-21T21:42:49.370 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042029947999999998 2024-09-21T21:42:49.370 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.370 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.370 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:49.370 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965100+0000", 2024-09-21T21:42:49.370 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1170000000000001e-05 2024-09-21T21:42:49.370 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.370 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.370 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:49.370 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965150+0000", 2024-09-21T21:42:49.370 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9743999999999997e-05 2024-09-21T21:42:49.370 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:49.371 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:49.371 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:49.371 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.371 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.371 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6982 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5304320~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:49.371 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923036+0000", 2024-09-21T21:42:49.371 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 79.428029080000002, 2024-09-21T21:42:49.371 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042225666000000002, 2024-09-21T21:42:49.371 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:49.371 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:49.371 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:49.371 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:49.371 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:49.371 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:49.372 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6982 2024-09-21T21:42:49.372 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.372 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:49.372 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.372 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:49.372 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923036+0000", 2024-09-21T21:42:49.372 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:49.372 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.372 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.372 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:49.372 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923036+0000", 2024-09-21T21:42:49.372 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:49.372 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.372 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.372 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:49.373 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923037+0000", 2024-09-21T21:42:49.373 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2800000000000002e-07 2024-09-21T21:42:49.373 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.373 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.373 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:49.373 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923039+0000", 2024-09-21T21:42:49.373 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1500000000000002e-06 2024-09-21T21:42:49.373 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.373 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.373 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:49.373 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923049+0000", 2024-09-21T21:42:49.373 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0236e-05 2024-09-21T21:42:49.373 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.373 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.374 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:49.374 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923056+0000", 2024-09-21T21:42:49.374 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2540000000000003e-06 2024-09-21T21:42:49.374 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.374 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.374 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:49.374 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965173+0000", 2024-09-21T21:42:49.374 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042116686 2024-09-21T21:42:49.374 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.374 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.374 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:49.374 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965215+0000", 2024-09-21T21:42:49.374 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1696000000000002e-05 2024-09-21T21:42:49.374 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.375 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.375 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:49.375 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965262+0000", 2024-09-21T21:42:49.375 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7216000000000003e-05 2024-09-21T21:42:49.375 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:49.375 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:49.375 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:49.375 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.375 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.375 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6984 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5318656~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:49.375 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923063+0000", 2024-09-21T21:42:49.375 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 79.428002426000006, 2024-09-21T21:42:49.375 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042318651999999998, 2024-09-21T21:42:49.375 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:49.376 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:49.376 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:49.376 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:49.376 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:49.376 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:49.377 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6984 2024-09-21T21:42:49.377 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.377 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:49.377 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.377 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:49.378 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923063+0000", 2024-09-21T21:42:49.378 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:49.378 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.378 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.378 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:49.378 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923063+0000", 2024-09-21T21:42:49.378 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:49.378 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.378 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.378 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:49.378 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923064+0000", 2024-09-21T21:42:49.378 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2e-07 2024-09-21T21:42:49.378 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.379 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.379 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:49.379 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923066+0000", 2024-09-21T21:42:49.379 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.057e-06 2024-09-21T21:42:49.379 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.379 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.379 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:49.379 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923071+0000", 2024-09-21T21:42:49.379 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8810000000000001e-06 2024-09-21T21:42:49.379 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.379 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.379 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:49.379 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923078+0000", 2024-09-21T21:42:49.379 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5479999999999998e-06 2024-09-21T21:42:49.380 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.380 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.380 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:49.380 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965288+0000", 2024-09-21T21:42:49.380 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042209576999999998 2024-09-21T21:42:49.380 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.380 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.380 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:49.380 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965332+0000", 2024-09-21T21:42:49.380 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4366999999999997e-05 2024-09-21T21:42:49.380 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.380 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.380 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:49.380 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965382+0000", 2024-09-21T21:42:49.380 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9802000000000002e-05 2024-09-21T21:42:49.381 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:49.381 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:49.381 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:49.381 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.381 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.381 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6987 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5332992~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:49.381 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923085+0000", 2024-09-21T21:42:49.381 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 79.427980900999998, 2024-09-21T21:42:49.381 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042412103999999999, 2024-09-21T21:42:49.381 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:49.381 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:49.381 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:49.381 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:49.381 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:49.381 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:49.382 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6987 2024-09-21T21:42:49.382 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.382 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:49.382 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.382 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:49.382 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923085+0000", 2024-09-21T21:42:49.382 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:49.382 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.382 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.382 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:49.382 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923085+0000", 2024-09-21T21:42:49.382 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:49.382 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.382 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.382 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:49.383 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923085+0000", 2024-09-21T21:42:49.383 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.89e-07 2024-09-21T21:42:49.383 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.383 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.383 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:49.383 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923087+0000", 2024-09-21T21:42:49.383 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0810000000000001e-06 2024-09-21T21:42:49.383 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.383 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.383 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:49.383 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923098+0000", 2024-09-21T21:42:49.383 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1299e-05 2024-09-21T21:42:49.383 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.383 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.383 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:49.384 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923104+0000", 2024-09-21T21:42:49.384 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9100000000000002e-06 2024-09-21T21:42:49.384 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.384 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.384 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:49.384 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965404+0000", 2024-09-21T21:42:49.384 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042299622000000002 2024-09-21T21:42:49.384 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.384 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.384 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:49.384 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965445+0000", 2024-09-21T21:42:49.384 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0701000000000002e-05 2024-09-21T21:42:49.384 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.384 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.384 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:49.385 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965497+0000", 2024-09-21T21:42:49.385 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2101999999999997e-05 2024-09-21T21:42:49.385 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:49.385 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:49.385 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:49.385 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.385 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.385 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6989 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5347328~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:49.385 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923111+0000", 2024-09-21T21:42:49.385 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 79.427954561000007, 2024-09-21T21:42:49.385 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042505188999999999, 2024-09-21T21:42:49.385 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:49.385 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:49.385 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:49.385 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:49.385 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:49.386 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:49.386 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6989 2024-09-21T21:42:49.386 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.386 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:49.386 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.386 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:49.386 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923111+0000", 2024-09-21T21:42:49.386 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:49.386 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.386 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.386 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:49.386 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923111+0000", 2024-09-21T21:42:49.386 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:49.386 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.387 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.387 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:49.387 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923111+0000", 2024-09-21T21:42:49.387 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4499999999999997e-07 2024-09-21T21:42:49.387 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.387 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.387 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:49.387 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923114+0000", 2024-09-21T21:42:49.388 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1569999999999998e-06 2024-09-21T21:42:49.388 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.388 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.388 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:49.388 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923120+0000", 2024-09-21T21:42:49.388 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1759999999999998e-06 2024-09-21T21:42:49.388 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.388 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.388 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:49.388 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923125+0000", 2024-09-21T21:42:49.388 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2560000000000002e-06 2024-09-21T21:42:49.388 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.389 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.389 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:49.389 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965523+0000", 2024-09-21T21:42:49.389 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042398094999999997 2024-09-21T21:42:49.389 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.389 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.389 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:49.389 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965566+0000", 2024-09-21T21:42:49.389 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3393999999999998e-05 2024-09-21T21:42:49.389 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.389 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.389 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:49.389 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965616+0000", 2024-09-21T21:42:49.389 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9666000000000002e-05 2024-09-21T21:42:49.390 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:49.390 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:49.390 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:49.390 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.390 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.390 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6991 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5361664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:49.390 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923132+0000", 2024-09-21T21:42:49.390 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 79.427933272000004, 2024-09-21T21:42:49.390 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042600779999999998, 2024-09-21T21:42:49.390 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:49.390 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:49.390 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:49.390 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:49.390 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:49.391 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:49.391 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6991 2024-09-21T21:42:49.391 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.391 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:49.391 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.391 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:49.391 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923132+0000", 2024-09-21T21:42:49.391 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:49.391 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.391 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.391 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:49.391 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923132+0000", 2024-09-21T21:42:49.391 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:49.391 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.392 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.392 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:49.392 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923132+0000", 2024-09-21T21:42:49.392 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4200000000000002e-07 2024-09-21T21:42:49.392 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.392 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.392 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:49.392 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923134+0000", 2024-09-21T21:42:49.392 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2270000000000001e-06 2024-09-21T21:42:49.392 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.392 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.392 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:49.392 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923137+0000", 2024-09-21T21:42:49.393 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1930000000000002e-06 2024-09-21T21:42:49.393 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.393 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.393 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:49.393 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923139+0000", 2024-09-21T21:42:49.393 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4710000000000001e-06 2024-09-21T21:42:49.393 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.393 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.393 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:49.393 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965639+0000", 2024-09-21T21:42:49.393 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.04249965 2024-09-21T21:42:49.393 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.393 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.393 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:49.393 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965679+0000", 2024-09-21T21:42:49.394 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9852999999999999e-05 2024-09-21T21:42:49.394 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.394 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.394 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:49.394 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965733+0000", 2024-09-21T21:42:49.394 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4144000000000002e-05 2024-09-21T21:42:49.394 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:49.394 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:49.394 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:49.394 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.394 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.394 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6993 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5384192~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:49.394 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923143+0000", 2024-09-21T21:42:49.395 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 79.427922477999999, 2024-09-21T21:42:49.395 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042708182999999997, 2024-09-21T21:42:49.395 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:49.395 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:49.395 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:49.395 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:49.395 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:49.395 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:49.395 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6993 2024-09-21T21:42:49.395 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.395 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:49.395 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.395 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:49.395 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923143+0000", 2024-09-21T21:42:49.395 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:49.396 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.396 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.396 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:49.396 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923143+0000", 2024-09-21T21:42:49.396 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:49.396 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.396 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.396 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:49.396 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923143+0000", 2024-09-21T21:42:49.396 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0800000000000001e-07 2024-09-21T21:42:49.396 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.396 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.396 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:49.396 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923144+0000", 2024-09-21T21:42:49.396 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.009e-06 2024-09-21T21:42:49.396 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.397 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.397 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:49.397 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923148+0000", 2024-09-21T21:42:49.397 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.737e-06 2024-09-21T21:42:49.397 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.397 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.397 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:49.397 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923150+0000", 2024-09-21T21:42:49.397 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4669999999999998e-06 2024-09-21T21:42:49.397 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.397 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.397 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:49.397 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965760+0000", 2024-09-21T21:42:49.397 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042609399999999999 2024-09-21T21:42:49.397 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.397 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.398 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:49.398 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965801+0000", 2024-09-21T21:42:49.398 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1399999999999997e-05 2024-09-21T21:42:49.398 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.398 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.398 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:49.398 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965851+0000", 2024-09-21T21:42:49.398 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9962e-05 2024-09-21T21:42:49.398 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:49.398 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:49.398 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:49.398 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.398 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.398 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6995 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5398528~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:49.398 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923153+0000", 2024-09-21T21:42:49.399 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 79.427912113999994, 2024-09-21T21:42:49.399 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042828034000000001, 2024-09-21T21:42:49.399 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:49.399 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:49.399 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:49.399 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:49.399 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:49.399 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:49.399 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6995 2024-09-21T21:42:49.399 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.399 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:49.399 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.399 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:49.399 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923153+0000", 2024-09-21T21:42:49.400 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:49.400 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.400 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.400 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:49.400 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923153+0000", 2024-09-21T21:42:49.400 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:49.400 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.400 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.400 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:49.400 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923154+0000", 2024-09-21T21:42:49.400 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2600000000000001e-07 2024-09-21T21:42:49.400 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.400 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.400 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:49.400 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923158+0000", 2024-09-21T21:42:49.400 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0559999999999998e-06 2024-09-21T21:42:49.401 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.401 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.401 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:49.401 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923161+0000", 2024-09-21T21:42:49.401 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6519999999999999e-06 2024-09-21T21:42:49.401 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.401 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.401 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:49.401 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923170+0000", 2024-09-21T21:42:49.401 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5720000000000003e-06 2024-09-21T21:42:49.401 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.401 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.401 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:49.401 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965887+0000", 2024-09-21T21:42:49.402 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042716863000000001 2024-09-21T21:42:49.402 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.402 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.402 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:49.402 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965927+0000", 2024-09-21T21:42:49.402 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0346999999999998e-05 2024-09-21T21:42:49.402 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.402 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.402 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:49.402 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965981+0000", 2024-09-21T21:42:49.402 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4317999999999997e-05 2024-09-21T21:42:49.402 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:49.402 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:49.402 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:49.403 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.403 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.403 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6997 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5412864~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:49.403 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923174+0000", 2024-09-21T21:42:49.403 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 79.427891958999993, 2024-09-21T21:42:49.403 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042928296999999997, 2024-09-21T21:42:49.403 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:49.403 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:49.403 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:49.403 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:49.403 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:49.403 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:49.403 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6997 2024-09-21T21:42:49.404 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.404 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:49.404 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.404 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:49.404 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923174+0000", 2024-09-21T21:42:49.404 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:49.404 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.404 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.404 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:49.404 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923174+0000", 2024-09-21T21:42:49.404 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:49.404 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.404 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.404 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:49.404 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923174+0000", 2024-09-21T21:42:49.405 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.67e-07 2024-09-21T21:42:49.405 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.405 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.405 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:49.405 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923178+0000", 2024-09-21T21:42:49.405 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1330000000000001e-06 2024-09-21T21:42:49.405 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.405 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.405 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:49.405 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923182+0000", 2024-09-21T21:42:49.405 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.794e-06 2024-09-21T21:42:49.405 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.405 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.405 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:49.406 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923184+0000", 2024-09-21T21:42:49.406 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.435e-06 2024-09-21T21:42:49.406 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.406 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.406 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:49.406 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966009+0000", 2024-09-21T21:42:49.406 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042824327000000002 2024-09-21T21:42:49.406 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.406 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.406 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:49.406 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966052+0000", 2024-09-21T21:42:49.406 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3034000000000002e-05 2024-09-21T21:42:49.406 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.406 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.407 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:49.407 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966102+0000", 2024-09-21T21:42:49.407 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0306999999999998e-05 2024-09-21T21:42:49.407 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:49.407 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:49.407 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:49.407 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.407 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.407 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6999 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5427200~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:49.407 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923187+0000", 2024-09-21T21:42:49.407 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 79.427878230999994, 2024-09-21T21:42:49.407 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043104267000000002, 2024-09-21T21:42:49.407 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:49.407 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:49.408 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:49.408 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:49.408 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:49.408 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:49.408 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6999 2024-09-21T21:42:49.408 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.408 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:49.408 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.408 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:49.408 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923187+0000", 2024-09-21T21:42:49.408 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:49.408 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.408 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.408 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:49.408 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923187+0000", 2024-09-21T21:42:49.409 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:49.409 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.409 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.409 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:49.409 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923188+0000", 2024-09-21T21:42:49.409 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3999999999999998e-07 2024-09-21T21:42:49.409 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.409 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.409 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:49.409 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923189+0000", 2024-09-21T21:42:49.409 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4169999999999999e-06 2024-09-21T21:42:49.409 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.409 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.409 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:49.410 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923194+0000", 2024-09-21T21:42:49.410 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1819999999999997e-06 2024-09-21T21:42:49.410 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.410 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.410 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:49.410 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923198+0000", 2024-09-21T21:42:49.410 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.143e-06 2024-09-21T21:42:49.410 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.410 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.410 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:49.410 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966135+0000", 2024-09-21T21:42:49.410 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042936297999999998 2024-09-21T21:42:49.410 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.411 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.411 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:49.411 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966204+0000", 2024-09-21T21:42:49.411 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8989e-05 2024-09-21T21:42:49.411 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.411 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.411 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:49.411 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966292+0000", 2024-09-21T21:42:49.411 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7997999999999993e-05 2024-09-21T21:42:49.411 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:49.411 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:49.411 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:49.411 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.411 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.411 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7001 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5441536~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:49.412 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923211+0000", 2024-09-21T21:42:49.412 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 79.427854440000004, 2024-09-21T21:42:49.412 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043196518000000003, 2024-09-21T21:42:49.412 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:49.412 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:49.412 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:49.412 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:49.412 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:49.412 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:49.412 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7001 2024-09-21T21:42:49.412 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.412 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:49.412 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.412 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:49.413 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923211+0000", 2024-09-21T21:42:49.413 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:49.413 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.413 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.413 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:49.413 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923211+0000", 2024-09-21T21:42:49.413 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:49.413 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.413 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.413 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:49.413 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923212+0000", 2024-09-21T21:42:49.413 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3900000000000005e-07 2024-09-21T21:42:49.413 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.413 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.414 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:49.414 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923214+0000", 2024-09-21T21:42:49.414 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9970000000000001e-06 2024-09-21T21:42:49.414 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.414 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.414 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:49.414 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923219+0000", 2024-09-21T21:42:49.414 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4310000000000002e-06 2024-09-21T21:42:49.414 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.414 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.414 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:49.414 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923224+0000", 2024-09-21T21:42:49.414 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6410000000000001e-06 2024-09-21T21:42:49.415 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.415 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.415 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:49.415 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966318+0000", 2024-09-21T21:42:49.415 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043093842 2024-09-21T21:42:49.415 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.415 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.415 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:49.415 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966359+0000", 2024-09-21T21:42:49.415 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1260999999999997e-05 2024-09-21T21:42:49.415 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.415 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.415 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:49.415 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966408+0000", 2024-09-21T21:42:49.415 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8807000000000002e-05 2024-09-21T21:42:49.416 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:49.416 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:49.416 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:49.416 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.416 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.416 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7002 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5455872~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:49.416 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923229+0000", 2024-09-21T21:42:49.416 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 79.427836029999995, 2024-09-21T21:42:49.416 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043364568999999999, 2024-09-21T21:42:49.416 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:49.416 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:49.416 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:49.416 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:49.416 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:49.417 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:49.417 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7002 2024-09-21T21:42:49.417 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.417 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:49.417 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.417 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:49.417 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923229+0000", 2024-09-21T21:42:49.417 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:49.417 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.417 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.417 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:49.417 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923229+0000", 2024-09-21T21:42:49.417 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:49.417 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.418 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.418 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:49.418 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923230+0000", 2024-09-21T21:42:49.418 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4700000000000002e-07 2024-09-21T21:42:49.418 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.418 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.418 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:49.418 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923232+0000", 2024-09-21T21:42:49.418 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0719999999999998e-06 2024-09-21T21:42:49.418 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.418 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.418 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:49.418 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923237+0000", 2024-09-21T21:42:49.419 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2290000000000002e-06 2024-09-21T21:42:49.419 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.419 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.419 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:49.419 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923245+0000", 2024-09-21T21:42:49.419 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2959999999999999e-06 2024-09-21T21:42:49.419 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.419 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.419 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:49.419 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966438+0000", 2024-09-21T21:42:49.419 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043193027000000002 2024-09-21T21:42:49.419 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.419 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.419 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:49.420 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966506+0000", 2024-09-21T21:42:49.420 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7707999999999996e-05 2024-09-21T21:42:49.420 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.420 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.420 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:49.420 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966594+0000", 2024-09-21T21:42:49.420 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.789e-05 2024-09-21T21:42:49.420 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:49.420 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:49.420 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:49.420 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.420 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.420 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7005 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5470208~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:49.420 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923254+0000", 2024-09-21T21:42:49.421 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 79.427811798999997, 2024-09-21T21:42:49.421 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043457176, 2024-09-21T21:42:49.421 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:49.421 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:49.421 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:49.421 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:49.421 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:49.421 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:49.421 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7005 2024-09-21T21:42:49.421 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.421 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:49.421 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.422 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:49.422 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923254+0000", 2024-09-21T21:42:49.422 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:49.422 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.422 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.422 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:49.422 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923254+0000", 2024-09-21T21:42:49.422 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:49.422 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.422 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.422 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:49.422 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923254+0000", 2024-09-21T21:42:49.422 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4700000000000002e-07 2024-09-21T21:42:49.423 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.423 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.423 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:49.423 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923256+0000", 2024-09-21T21:42:49.423 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.007e-06 2024-09-21T21:42:49.423 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.423 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.423 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:49.423 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923266+0000", 2024-09-21T21:42:49.423 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.4700000000000008e-06 2024-09-21T21:42:49.423 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.423 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.423 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:49.424 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923271+0000", 2024-09-21T21:42:49.424 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2809999999999999e-06 2024-09-21T21:42:49.424 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.424 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.424 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:49.424 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966619+0000", 2024-09-21T21:42:49.424 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043347951000000003 2024-09-21T21:42:49.424 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.424 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.424 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:49.424 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966661+0000", 2024-09-21T21:42:49.424 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2614999999999998e-05 2024-09-21T21:42:49.424 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.424 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.424 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:49.424 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966711+0000", 2024-09-21T21:42:49.425 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9505000000000001e-05 2024-09-21T21:42:49.425 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:49.425 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:49.425 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:49.425 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.425 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.425 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7008 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5484544~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:49.425 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923278+0000", 2024-09-21T21:42:49.425 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 79.427787151999993, 2024-09-21T21:42:49.425 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043631071, 2024-09-21T21:42:49.425 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:49.425 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:49.425 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:49.425 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:49.425 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:49.426 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:49.426 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7008 2024-09-21T21:42:49.426 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.426 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:49.426 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.426 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:49.426 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923278+0000", 2024-09-21T21:42:49.426 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:49.426 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.426 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.426 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:49.426 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923278+0000", 2024-09-21T21:42:49.426 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:49.426 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.427 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.427 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:49.427 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923279+0000", 2024-09-21T21:42:49.427 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.77e-07 2024-09-21T21:42:49.427 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.427 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.427 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:49.427 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923281+0000", 2024-09-21T21:42:49.427 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0219999999999999e-06 2024-09-21T21:42:49.427 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.427 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.427 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:49.427 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923287+0000", 2024-09-21T21:42:49.427 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7799999999999997e-06 2024-09-21T21:42:49.427 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.428 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.428 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:49.428 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923291+0000", 2024-09-21T21:42:49.428 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1699999999999999e-06 2024-09-21T21:42:49.428 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.428 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.428 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:49.428 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966742+0000", 2024-09-21T21:42:49.428 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043451344000000003 2024-09-21T21:42:49.428 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.428 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.428 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:49.428 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966809+0000", 2024-09-21T21:42:49.428 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7070999999999995e-05 2024-09-21T21:42:49.428 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.428 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.429 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:49.429 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966909+0000", 2024-09-21T21:42:49.429 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000100307 2024-09-21T21:42:49.429 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:49.429 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:49.429 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:49.429 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.429 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.429 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7010 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5498880~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:49.429 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923301+0000", 2024-09-21T21:42:49.429 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 79.427764257000007, 2024-09-21T21:42:49.429 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043717830999999999, 2024-09-21T21:42:49.429 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:49.429 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:49.429 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:49.429 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:49.430 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:49.430 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:49.430 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7010 2024-09-21T21:42:49.430 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.430 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:49.430 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.430 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:49.430 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923301+0000", 2024-09-21T21:42:49.430 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:49.430 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.430 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.430 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:49.430 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923301+0000", 2024-09-21T21:42:49.430 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:49.430 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.430 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.431 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:49.431 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923302+0000", 2024-09-21T21:42:49.431 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.41e-07 2024-09-21T21:42:49.431 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.431 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.431 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:49.431 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923304+0000", 2024-09-21T21:42:49.431 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0990000000000001e-06 2024-09-21T21:42:49.431 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.431 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.431 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:49.431 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923310+0000", 2024-09-21T21:42:49.431 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3860000000000004e-06 2024-09-21T21:42:49.431 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.432 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.432 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:49.432 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923315+0000", 2024-09-21T21:42:49.432 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8459999999999999e-06 2024-09-21T21:42:49.432 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.432 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.432 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:49.432 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966938+0000", 2024-09-21T21:42:49.432 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043622693999999997 2024-09-21T21:42:49.432 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.432 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.432 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:49.432 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966975+0000", 2024-09-21T21:42:49.432 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7608999999999999e-05 2024-09-21T21:42:49.432 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.432 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.433 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:49.433 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.967019+0000", 2024-09-21T21:42:49.433 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3856000000000002e-05 2024-09-21T21:42:49.433 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:49.433 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:49.433 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:49.433 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.433 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.433 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7354 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 2107392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:49.433 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.936932+0000", 2024-09-21T21:42:49.433 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 79.414133402000004, 2024-09-21T21:42:49.433 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041607708, 2024-09-21T21:42:49.433 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:49.433 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:49.433 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:49.434 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:49.434 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:49.434 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:49.434 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7354 2024-09-21T21:42:49.434 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.434 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:49.434 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.434 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:49.434 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.936932+0000", 2024-09-21T21:42:49.434 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:49.434 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.434 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.434 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:49.434 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.936932+0000", 2024-09-21T21:42:49.434 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:49.434 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.435 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.435 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:49.435 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.937276+0000", 2024-09-21T21:42:49.435 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00034344200000000001 2024-09-21T21:42:49.435 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.435 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.435 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:49.435 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.937279+0000", 2024-09-21T21:42:49.435 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.844e-06 2024-09-21T21:42:49.435 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.435 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.435 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:49.435 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.937288+0000", 2024-09-21T21:42:49.435 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2409999999999995e-06 2024-09-21T21:42:49.435 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.435 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.436 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:49.436 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.937293+0000", 2024-09-21T21:42:49.436 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6559999999999997e-06 2024-09-21T21:42:49.436 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.436 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.436 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:49.436 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.978448+0000", 2024-09-21T21:42:49.436 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041154621000000002 2024-09-21T21:42:49.436 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.436 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.436 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:49.436 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.978487+0000", 2024-09-21T21:42:49.436 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9254000000000001e-05 2024-09-21T21:42:49.436 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.436 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.437 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:49.437 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.978540+0000", 2024-09-21T21:42:49.437 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2649999999999999e-05 2024-09-21T21:42:49.437 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:49.437 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:49.437 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:49.437 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.437 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.437 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5215.0:1054 3.9 3:946d2401:::rbd_data.1121524d2522.000000000000000b:head [set-alloc-hint object_size 16777216 write_size 16777216,write 6615040~59392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-21T21:42:49.437 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:38.148979+0000", 2024-09-21T21:42:49.437 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 71.202086434999998, 2024-09-21T21:42:49.437 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047870388999999999, 2024-09-21T21:42:49.437 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:49.438 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:49.438 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:49.438 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:49.438 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5215", 2024-09-21T21:42:49.438 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1580282168", 2024-09-21T21:42:49.438 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1054 2024-09-21T21:42:49.438 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.438 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:49.438 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.438 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:49.438 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.148979+0000", 2024-09-21T21:42:49.438 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:49.438 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.439 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.439 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:49.439 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.148979+0000", 2024-09-21T21:42:49.439 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:49.439 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.439 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.439 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:49.439 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.148980+0000", 2024-09-21T21:42:49.439 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5799999999999999e-07 2024-09-21T21:42:49.439 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.439 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.439 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:49.439 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.149039+0000", 2024-09-21T21:42:49.439 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9151000000000003e-05 2024-09-21T21:42:49.439 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.440 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.440 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:49.440 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.149047+0000", 2024-09-21T21:42:49.440 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1000000000000004e-06 2024-09-21T21:42:49.440 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.440 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.440 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:49.440 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.149052+0000", 2024-09-21T21:42:49.440 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5959999999999997e-06 2024-09-21T21:42:49.440 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.440 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.440 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:49.440 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.195288+0000", 2024-09-21T21:42:49.440 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.046236248000000001 2024-09-21T21:42:49.440 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.440 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.440 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:49.441 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.195334+0000", 2024-09-21T21:42:49.441 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6575e-05 2024-09-21T21:42:49.441 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.441 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.441 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 2", 2024-09-21T21:42:49.441 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.195406+0000", 2024-09-21T21:42:49.441 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1715e-05 2024-09-21T21:42:49.441 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.441 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.441 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:49.441 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.196018+0000", 2024-09-21T21:42:49.441 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00061218999999999998 2024-09-21T21:42:49.441 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.441 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.441 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:49.442 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.196794+0000", 2024-09-21T21:42:49.442 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000775814 2024-09-21T21:42:49.442 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.442 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.442 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:49.442 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.196827+0000", 2024-09-21T21:42:49.442 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2419000000000003e-05 2024-09-21T21:42:49.442 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.442 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.442 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:49.442 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.196849+0000", 2024-09-21T21:42:49.442 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2923000000000001e-05 2024-09-21T21:42:49.442 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:49.442 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:49.442 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:49.442 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:49.443 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:49.443 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:49.444 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T21:42:49.445 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:49.443+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:49.545 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:49.544+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:49.589 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:49.589 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:42:49.589 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-21T21:42:49.589 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:49.599 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T21:42:49.634 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 5, 3] out_osds: [0, 4] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-21T21:42:49.634 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T21:42:49.634 INFO:tasks.thrashosds.thrasher:inject_pause on osd.3 2024-09-21T21:42:49.635 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-21T21:42:49.635 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-21T21:42:49.635 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok config set filestore_inject_stall 3 2024-09-21T21:42:49.646 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:49.645+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:49.704 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:49.704 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:42:49.704 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:42:49.704 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:42:49.704 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:49.713 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T21:42:49.746 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:49.746+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:49.748 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:49.748 INFO:teuthology.orchestra.run.smithi122.stdout: "success": "" 2024-09-21T21:42:49.748 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:49.819 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:49.819 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-21T21:42:49.819 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-21T21:42:49.819 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:42:49.819 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.820 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4385.0:15202 3.16 3:6888948b:::rbd_header.1121493f6459:head [watch reconnect cookie 140325440202496 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-21T21:42:49.820 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:41.736601+0000", 2024-09-21T21:42:49.820 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 68.079799170000001, 2024-09-21T21:42:49.820 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5040056769999999, 2024-09-21T21:42:49.820 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:49.820 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:49.820 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:49.820 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:49.820 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4385", 2024-09-21T21:42:49.820 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:42:49.820 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 15202 2024-09-21T21:42:49.820 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.820 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:49.821 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.821 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:49.821 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736601+0000", 2024-09-21T21:42:49.821 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:49.821 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.821 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.821 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:49.821 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736601+0000", 2024-09-21T21:42:49.821 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:49.821 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.821 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.821 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:49.821 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736602+0000", 2024-09-21T21:42:49.822 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0699999999999999e-06 2024-09-21T21:42:49.822 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.822 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.822 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:49.822 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736607+0000", 2024-09-21T21:42:49.822 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2520000000000001e-06 2024-09-21T21:42:49.822 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.822 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.822 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:49.822 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736617+0000", 2024-09-21T21:42:49.822 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0074e-05 2024-09-21T21:42:49.822 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.822 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.823 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:49.823 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736622+0000", 2024-09-21T21:42:49.823 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.101e-06 2024-09-21T21:42:49.823 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.823 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.823 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:49.823 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736664+0000", 2024-09-21T21:42:49.823 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1869999999999997e-05 2024-09-21T21:42:49.823 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.823 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.823 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op must wait for map", 2024-09-21T21:42:49.823 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736667+0000", 2024-09-21T21:42:49.823 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4750000000000002e-06 2024-09-21T21:42:49.824 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.824 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.824 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:49.824 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742886+0000", 2024-09-21T21:42:49.824 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0062187579999999996 2024-09-21T21:42:49.824 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.824 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.824 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:49.824 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742892+0000", 2024-09-21T21:42:49.824 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0440000000000001e-06 2024-09-21T21:42:49.824 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.824 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.824 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:49.824 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210307+0000", 2024-09-21T21:42:49.825 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.467414781 2024-09-21T21:42:49.825 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.825 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.825 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:49.825 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210322+0000", 2024-09-21T21:42:49.825 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5311e-05 2024-09-21T21:42:49.825 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.825 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.825 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:49.825 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.237624+0000", 2024-09-21T21:42:49.825 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0273018920000001 2024-09-21T21:42:49.825 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.825 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.825 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:49.825 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.237870+0000", 2024-09-21T21:42:49.826 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00024622899999999999 2024-09-21T21:42:49.826 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.826 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.826 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:49.826 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.240607+0000", 2024-09-21T21:42:49.826 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0027368200000000001 2024-09-21T21:42:49.826 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:49.826 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:49.826 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:49.826 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.826 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.826 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5415.0:379 3.16 3:6888948b:::rbd_header.1121493f6459:head [watch reconnect cookie 139702602617920 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-21T21:42:49.826 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:41.739544+0000", 2024-09-21T21:42:49.827 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 68.076856741, 2024-09-21T21:42:49.827 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.501180645, 2024-09-21T21:42:49.827 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:49.827 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:49.827 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:49.827 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:49.827 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5415", 2024-09-21T21:42:49.827 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1212816607", 2024-09-21T21:42:49.827 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 379 2024-09-21T21:42:49.827 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.827 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:49.827 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.827 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:49.827 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739544+0000", 2024-09-21T21:42:49.828 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:49.828 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.828 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.828 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:49.828 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739544+0000", 2024-09-21T21:42:49.828 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:49.828 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.828 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.828 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:49.828 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739545+0000", 2024-09-21T21:42:49.828 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.065e-06 2024-09-21T21:42:49.828 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.828 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.828 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:49.828 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739550+0000", 2024-09-21T21:42:49.828 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9599999999999999e-06 2024-09-21T21:42:49.829 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.829 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.829 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:49.829 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739565+0000", 2024-09-21T21:42:49.829 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5709000000000002e-05 2024-09-21T21:42:49.829 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.829 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.829 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:49.829 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739573+0000", 2024-09-21T21:42:49.829 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7849999999999992e-06 2024-09-21T21:42:49.829 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.829 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.829 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:49.829 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739634+0000", 2024-09-21T21:42:49.829 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0732000000000001e-05 2024-09-21T21:42:49.830 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.830 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.830 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op must wait for map", 2024-09-21T21:42:49.830 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739640+0000", 2024-09-21T21:42:49.830 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0000000000000002e-06 2024-09-21T21:42:49.830 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.830 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.830 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:49.830 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742958+0000", 2024-09-21T21:42:49.830 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0033182390000000002 2024-09-21T21:42:49.830 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.830 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.830 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:49.830 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742966+0000", 2024-09-21T21:42:49.830 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1610000000000003e-06 2024-09-21T21:42:49.831 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.831 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.831 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:49.831 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210405+0000", 2024-09-21T21:42:49.831 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.467438725 2024-09-21T21:42:49.831 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.831 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.831 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:49.831 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210415+0000", 2024-09-21T21:42:49.831 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0234000000000001e-05 2024-09-21T21:42:49.831 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.831 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.831 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:49.832 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.238743+0000", 2024-09-21T21:42:49.832 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0283273070000001 2024-09-21T21:42:49.832 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.832 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.832 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:49.832 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.238842+0000", 2024-09-21T21:42:49.832 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.9896999999999996e-05 2024-09-21T21:42:49.832 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.832 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.832 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:49.832 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.240724+0000", 2024-09-21T21:42:49.832 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001881831 2024-09-21T21:42:49.832 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:49.832 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:49.833 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:49.833 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.833 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.833 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4385.0:15203 3.16 3:6888948b:::rbd_header.1121493f6459:head [call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-21T21:42:49.833 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:41.739737+0000", 2024-09-21T21:42:49.833 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 68.076663730000007, 2024-09-21T21:42:49.833 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4989621799999999, 2024-09-21T21:42:49.833 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:49.833 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:49.833 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:49.833 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:49.833 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4385", 2024-09-21T21:42:49.833 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:42:49.834 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 15203 2024-09-21T21:42:49.834 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.834 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:49.834 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.834 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:49.834 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739737+0000", 2024-09-21T21:42:49.834 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:49.834 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.834 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.834 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:49.834 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739737+0000", 2024-09-21T21:42:49.834 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:49.834 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.835 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.835 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:49.835 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739738+0000", 2024-09-21T21:42:49.835 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.7499999999999998e-07 2024-09-21T21:42:49.835 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.835 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.835 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:49.835 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739743+0000", 2024-09-21T21:42:49.835 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5049999999999998e-06 2024-09-21T21:42:49.835 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.835 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.835 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:49.835 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739757+0000", 2024-09-21T21:42:49.835 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3947e-05 2024-09-21T21:42:49.835 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.836 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.836 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:49.836 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739765+0000", 2024-09-21T21:42:49.836 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9310000000000008e-06 2024-09-21T21:42:49.836 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.836 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.836 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:49.836 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739816+0000", 2024-09-21T21:42:49.836 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0559999999999997e-05 2024-09-21T21:42:49.836 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.836 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.836 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting_for_map not empty", 2024-09-21T21:42:49.836 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739821+0000", 2024-09-21T21:42:49.837 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3539999999999999e-06 2024-09-21T21:42:49.837 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.837 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.837 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:49.837 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742928+0000", 2024-09-21T21:42:49.837 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0031074399999999999 2024-09-21T21:42:49.837 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.837 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.837 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:49.837 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742937+0000", 2024-09-21T21:42:49.837 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1289999999999996e-06 2024-09-21T21:42:49.837 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.837 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.837 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:49.838 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210350+0000", 2024-09-21T21:42:49.838 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.46741311299999999 2024-09-21T21:42:49.838 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.838 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.838 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:49.838 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210358+0000", 2024-09-21T21:42:49.838 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1869999999999997e-06 2024-09-21T21:42:49.838 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.838 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.838 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:49.838 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.238398+0000", 2024-09-21T21:42:49.838 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0280397830000001 2024-09-21T21:42:49.838 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.839 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.839 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:49.839 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.238489+0000", 2024-09-21T21:42:49.839 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.0958000000000006e-05 2024-09-21T21:42:49.839 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.839 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.839 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:49.839 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.238699+0000", 2024-09-21T21:42:49.839 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00021029800000000001 2024-09-21T21:42:49.839 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:49.839 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:49.839 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:49.839 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.839 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.840 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4385.0:15204 3.16 3:6888948b:::rbd_header.1121493f6459:head [watch reconnect cookie 140325440202496 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-21T21:42:49.840 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:42.210726+0000", 2024-09-21T21:42:49.840 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 67.605674852000007, 2024-09-21T21:42:49.840 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.030304557, 2024-09-21T21:42:49.840 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:49.840 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:49.840 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:49.840 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:49.840 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4385", 2024-09-21T21:42:49.840 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:42:49.840 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 15204 2024-09-21T21:42:49.840 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.840 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:49.840 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.841 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:49.841 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210726+0000", 2024-09-21T21:42:49.841 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:49.841 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.841 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.841 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:49.841 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210726+0000", 2024-09-21T21:42:49.841 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:49.841 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.841 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.841 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:49.841 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210727+0000", 2024-09-21T21:42:49.841 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.204e-06 2024-09-21T21:42:49.841 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.842 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.842 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:49.842 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210732+0000", 2024-09-21T21:42:49.842 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5249999999999996e-06 2024-09-21T21:42:49.842 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.842 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.842 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:49.842 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210744+0000", 2024-09-21T21:42:49.842 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1405e-05 2024-09-21T21:42:49.842 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.842 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.842 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:49.842 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210754+0000", 2024-09-21T21:42:49.842 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0122999999999999e-05 2024-09-21T21:42:49.843 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.843 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.843 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:49.843 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210798+0000", 2024-09-21T21:42:49.843 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4005000000000003e-05 2024-09-21T21:42:49.843 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.843 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.843 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:49.843 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210803+0000", 2024-09-21T21:42:49.843 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5130000000000004e-06 2024-09-21T21:42:49.843 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.843 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.843 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:49.843 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.239260+0000", 2024-09-21T21:42:49.844 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0284563040000001 2024-09-21T21:42:49.844 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.844 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.844 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:49.844 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.239318+0000", 2024-09-21T21:42:49.844 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8486000000000002e-05 2024-09-21T21:42:49.844 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.844 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.844 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:49.844 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.241030+0000", 2024-09-21T21:42:49.844 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0017119920000000001 2024-09-21T21:42:49.844 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:49.844 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:49.844 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:49.844 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.845 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.845 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5415.0:380 3.16 3:6888948b:::rbd_header.1121493f6459:head [watch reconnect cookie 139702602617920 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-21T21:42:49.845 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:42.210777+0000", 2024-09-21T21:42:49.845 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 67.605623073000004, 2024-09-21T21:42:49.845 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.030410566, 2024-09-21T21:42:49.845 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:49.845 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:49.845 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:49.845 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:49.845 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5415", 2024-09-21T21:42:49.845 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1212816607", 2024-09-21T21:42:49.845 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 380 2024-09-21T21:42:49.845 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.845 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:49.846 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.846 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:49.846 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210777+0000", 2024-09-21T21:42:49.846 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:49.846 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.846 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.846 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:49.846 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210777+0000", 2024-09-21T21:42:49.846 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:49.846 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.846 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.846 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:49.847 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210778+0000", 2024-09-21T21:42:49.847 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8999999999999996e-07 2024-09-21T21:42:49.847 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.847 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.847 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:49.847 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210780+0000", 2024-09-21T21:42:49.847 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.503e-06 2024-09-21T21:42:49.847 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.847 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.847 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:49.847 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210788+0000", 2024-09-21T21:42:49.847 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0999999999999998e-06 2024-09-21T21:42:49.847 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.848 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.848 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:49.848 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210799+0000", 2024-09-21T21:42:49.848 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1703e-05 2024-09-21T21:42:49.848 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.848 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.848 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:49.848 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210833+0000", 2024-09-21T21:42:49.848 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4008999999999999e-05 2024-09-21T21:42:49.848 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.848 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.848 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:49.848 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210838+0000", 2024-09-21T21:42:49.849 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7480000000000001e-06 2024-09-21T21:42:49.849 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.849 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.849 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:49.849 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.239612+0000", 2024-09-21T21:42:49.849 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.028773554 2024-09-21T21:42:49.849 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.849 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.849 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:49.849 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.239677+0000", 2024-09-21T21:42:49.849 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5053000000000002e-05 2024-09-21T21:42:49.849 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.849 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.849 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:49.850 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.241188+0000", 2024-09-21T21:42:49.850 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001511306 2024-09-21T21:42:49.850 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:49.850 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:49.850 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:49.850 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.850 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.850 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24794 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:49.850 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.584085+0000", 2024-09-21T21:42:49.850 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 27.232315896999999, 2024-09-21T21:42:49.850 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.096209996000000006, 2024-09-21T21:42:49.850 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:49.850 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:49.850 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:49.851 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:49.851 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:42:49.851 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:49.851 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24794 2024-09-21T21:42:49.851 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.851 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:49.851 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.851 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:49.851 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584085+0000", 2024-09-21T21:42:49.851 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:49.851 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.851 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.851 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:49.851 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584085+0000", 2024-09-21T21:42:49.851 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:49.852 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.852 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.852 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:49.852 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584085+0000", 2024-09-21T21:42:49.852 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3200000000000001e-07 2024-09-21T21:42:49.852 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.852 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.852 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:49.852 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584086+0000", 2024-09-21T21:42:49.852 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.717e-06 2024-09-21T21:42:49.852 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.852 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.852 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:49.853 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584090+0000", 2024-09-21T21:42:49.853 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3450000000000002e-06 2024-09-21T21:42:49.853 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.853 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.853 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:49.853 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584092+0000", 2024-09-21T21:42:49.853 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6359999999999998e-06 2024-09-21T21:42:49.853 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.853 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.853 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:49.853 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.676497+0000", 2024-09-21T21:42:49.853 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.092404659 2024-09-21T21:42:49.853 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.853 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.854 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:49.854 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.676567+0000", 2024-09-21T21:42:49.854 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9627000000000003e-05 2024-09-21T21:42:49.854 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.854 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.854 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:42:49.854 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.676925+0000", 2024-09-21T21:42:49.854 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00035848500000000002 2024-09-21T21:42:49.854 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.854 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.854 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:49.854 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.678427+0000", 2024-09-21T21:42:49.854 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001501638 2024-09-21T21:42:49.854 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.854 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.855 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:49.855 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.680204+0000", 2024-09-21T21:42:49.855 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0017766979999999999 2024-09-21T21:42:49.855 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.855 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.855 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:49.855 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.680256+0000", 2024-09-21T21:42:49.855 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2549999999999997e-05 2024-09-21T21:42:49.855 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.855 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.855 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:49.855 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.680295+0000", 2024-09-21T21:42:49.855 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8408999999999998e-05 2024-09-21T21:42:49.855 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:49.856 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:49.856 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:49.856 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.856 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.856 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24796 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:49.856 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.584099+0000", 2024-09-21T21:42:49.856 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 27.232301327999998, 2024-09-21T21:42:49.856 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.097297429000000005, 2024-09-21T21:42:49.856 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:49.856 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:49.856 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:49.856 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:49.856 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:42:49.856 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:49.856 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24796 2024-09-21T21:42:49.857 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.857 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:49.857 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.857 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:49.857 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584099+0000", 2024-09-21T21:42:49.857 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:49.857 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.857 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.857 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:49.857 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584099+0000", 2024-09-21T21:42:49.857 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:49.857 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.857 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.857 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:49.857 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584099+0000", 2024-09-21T21:42:49.858 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3999999999999998e-07 2024-09-21T21:42:49.858 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.858 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.858 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:49.858 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584101+0000", 2024-09-21T21:42:49.858 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5850000000000001e-06 2024-09-21T21:42:49.858 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.858 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.858 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:49.858 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584108+0000", 2024-09-21T21:42:49.858 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.498e-06 2024-09-21T21:42:49.858 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.858 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.858 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:49.859 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584112+0000", 2024-09-21T21:42:49.859 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.252e-06 2024-09-21T21:42:49.859 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.859 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.859 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:49.859 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.677336+0000", 2024-09-21T21:42:49.859 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.093224206000000004 2024-09-21T21:42:49.859 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.859 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.859 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:49.859 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.677411+0000", 2024-09-21T21:42:49.859 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5036000000000005e-05 2024-09-21T21:42:49.859 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.859 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.859 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:42:49.860 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.677782+0000", 2024-09-21T21:42:49.860 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00037153099999999998 2024-09-21T21:42:49.860 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.860 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.860 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:49.860 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.680360+0000", 2024-09-21T21:42:49.860 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025773020000000001 2024-09-21T21:42:49.860 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.860 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.860 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:49.860 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681313+0000", 2024-09-21T21:42:49.860 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00095341100000000004 2024-09-21T21:42:49.860 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.860 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.861 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:49.861 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681361+0000", 2024-09-21T21:42:49.861 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7497999999999998e-05 2024-09-21T21:42:49.861 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.861 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.861 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:49.861 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681397+0000", 2024-09-21T21:42:49.861 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.587e-05 2024-09-21T21:42:49.861 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:49.861 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:49.861 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:49.861 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.861 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.861 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24797 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:49.862 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.584116+0000", 2024-09-21T21:42:49.862 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 27.232284838999998, 2024-09-21T21:42:49.862 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.097428257000000004, 2024-09-21T21:42:49.862 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:49.862 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:49.862 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:49.862 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:49.862 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:42:49.862 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:49.862 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24797 2024-09-21T21:42:49.862 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.862 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:49.862 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.862 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:49.863 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584116+0000", 2024-09-21T21:42:49.863 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:49.863 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.863 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.863 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:49.863 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584116+0000", 2024-09-21T21:42:49.863 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:49.863 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.863 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.863 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:49.863 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584116+0000", 2024-09-21T21:42:49.863 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.53e-07 2024-09-21T21:42:49.863 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.863 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.864 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:49.864 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584118+0000", 2024-09-21T21:42:49.864 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6810000000000001e-06 2024-09-21T21:42:49.864 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.864 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.864 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:49.864 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584121+0000", 2024-09-21T21:42:49.864 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1099999999999999e-06 2024-09-21T21:42:49.864 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.864 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.864 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:49.864 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584123+0000", 2024-09-21T21:42:49.864 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5890000000000001e-06 2024-09-21T21:42:49.864 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.865 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.865 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:49.865 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.678476+0000", 2024-09-21T21:42:49.865 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.094352320000000003 2024-09-21T21:42:49.865 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.865 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.865 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:49.865 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.678556+0000", 2024-09-21T21:42:49.865 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0291999999999996e-05 2024-09-21T21:42:49.865 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.865 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.865 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:42:49.865 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.678871+0000", 2024-09-21T21:42:49.865 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000314689 2024-09-21T21:42:49.866 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.866 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.866 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:49.866 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681452+0000", 2024-09-21T21:42:49.866 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025817890000000001 2024-09-21T21:42:49.866 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.866 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.866 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:49.866 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681467+0000", 2024-09-21T21:42:49.866 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4256e-05 2024-09-21T21:42:49.866 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.866 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.866 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:49.866 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681510+0000", 2024-09-21T21:42:49.867 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3560999999999998e-05 2024-09-21T21:42:49.867 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.867 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.867 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:49.867 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681544+0000", 2024-09-21T21:42:49.867 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3717000000000003e-05 2024-09-21T21:42:49.867 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:49.867 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:49.867 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:49.867 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.867 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.867 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24798 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:49.867 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.584130+0000", 2024-09-21T21:42:49.867 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 27.232270316000001, 2024-09-21T21:42:49.868 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.098350526999999993, 2024-09-21T21:42:49.868 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:49.868 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:49.868 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:49.868 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:49.868 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:42:49.868 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:49.868 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24798 2024-09-21T21:42:49.868 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.868 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:49.868 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.868 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:49.868 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584130+0000", 2024-09-21T21:42:49.868 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:49.869 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.869 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.869 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:49.869 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584130+0000", 2024-09-21T21:42:49.869 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:49.869 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.869 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.869 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:49.869 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584131+0000", 2024-09-21T21:42:49.869 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4700000000000001e-07 2024-09-21T21:42:49.869 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.869 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.870 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:49.870 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584133+0000", 2024-09-21T21:42:49.870 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.34e-06 2024-09-21T21:42:49.870 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.870 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.870 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:49.870 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584137+0000", 2024-09-21T21:42:49.870 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5609999999999999e-06 2024-09-21T21:42:49.870 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.870 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.870 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:49.870 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584139+0000", 2024-09-21T21:42:49.870 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7350000000000001e-06 2024-09-21T21:42:49.870 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.870 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.871 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:49.871 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.679274+0000", 2024-09-21T21:42:49.871 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.095134719000000006 2024-09-21T21:42:49.871 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.871 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.871 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:49.871 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.679358+0000", 2024-09-21T21:42:49.871 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3609999999999994e-05 2024-09-21T21:42:49.871 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.871 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.871 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:42:49.871 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.679685+0000", 2024-09-21T21:42:49.871 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00032732399999999999 2024-09-21T21:42:49.871 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.871 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.872 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:49.872 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681553+0000", 2024-09-21T21:42:49.872 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001868059 2024-09-21T21:42:49.872 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.872 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.872 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:49.872 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.682375+0000", 2024-09-21T21:42:49.872 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00082232099999999999 2024-09-21T21:42:49.872 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.872 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.872 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:49.872 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.682431+0000", 2024-09-21T21:42:49.872 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6051999999999998e-05 2024-09-21T21:42:49.872 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.873 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.873 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:49.873 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.682481+0000", 2024-09-21T21:42:49.873 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9258999999999997e-05 2024-09-21T21:42:49.873 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:49.873 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:49.873 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:49.873 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.873 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.873 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24799 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:49.873 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.584159+0000", 2024-09-21T21:42:49.873 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 27.232241328000001, 2024-09-21T21:42:49.873 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.100021551, 2024-09-21T21:42:49.873 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:49.874 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:49.874 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:49.874 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:49.874 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:42:49.874 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:49.874 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24799 2024-09-21T21:42:49.874 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.874 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:49.874 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.874 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:49.874 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584159+0000", 2024-09-21T21:42:49.874 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:49.874 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.875 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.875 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:49.875 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584159+0000", 2024-09-21T21:42:49.875 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:49.875 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.875 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.875 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:49.875 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584160+0000", 2024-09-21T21:42:49.875 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1300000000000001e-07 2024-09-21T21:42:49.875 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.875 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.875 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:49.876 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584162+0000", 2024-09-21T21:42:49.876 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5409999999999999e-06 2024-09-21T21:42:49.876 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.876 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.876 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:49.876 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584169+0000", 2024-09-21T21:42:49.876 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7880000000000001e-06 2024-09-21T21:42:49.876 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.876 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.876 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:49.876 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584174+0000", 2024-09-21T21:42:49.876 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9590000000000003e-06 2024-09-21T21:42:49.877 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.877 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.877 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:49.877 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.680400+0000", 2024-09-21T21:42:49.877 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.096226514999999999 2024-09-21T21:42:49.877 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.877 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.877 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:49.877 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.680480+0000", 2024-09-21T21:42:49.877 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9589e-05 2024-09-21T21:42:49.877 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.877 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.877 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:42:49.877 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.680847+0000", 2024-09-21T21:42:49.878 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00036757700000000001 2024-09-21T21:42:49.878 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.878 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.878 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:49.878 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.682555+0000", 2024-09-21T21:42:49.878 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001707791 2024-09-21T21:42:49.878 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.878 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.878 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:49.878 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684093+0000", 2024-09-21T21:42:49.878 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0015377170000000001 2024-09-21T21:42:49.878 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.879 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.879 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:49.879 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684144+0000", 2024-09-21T21:42:49.879 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1208000000000002e-05 2024-09-21T21:42:49.879 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.879 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.879 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:49.879 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684181+0000", 2024-09-21T21:42:49.879 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6452999999999998e-05 2024-09-21T21:42:49.879 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:49.879 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:49.879 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:49.879 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.880 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.880 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24835 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:49.880 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586332+0000", 2024-09-21T21:42:49.880 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 27.230068714000002, 2024-09-21T21:42:49.880 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.098730360000000003, 2024-09-21T21:42:49.880 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:49.880 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:49.880 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:49.880 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:49.880 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:42:49.880 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:49.880 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24835 2024-09-21T21:42:49.880 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.881 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:49.881 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.881 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:49.881 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586332+0000", 2024-09-21T21:42:49.881 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:49.881 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.881 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.881 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:49.881 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586332+0000", 2024-09-21T21:42:49.881 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:49.881 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.881 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.882 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:49.882 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586333+0000", 2024-09-21T21:42:49.882 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0200000000000001e-07 2024-09-21T21:42:49.882 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.882 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.882 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:49.882 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586346+0000", 2024-09-21T21:42:49.882 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3745e-05 2024-09-21T21:42:49.882 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.882 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.882 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:49.882 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586366+0000", 2024-09-21T21:42:49.883 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9272000000000001e-05 2024-09-21T21:42:49.883 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.883 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.883 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:49.883 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586377+0000", 2024-09-21T21:42:49.883 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1197e-05 2024-09-21T21:42:49.883 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.883 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.883 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:49.883 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681578+0000", 2024-09-21T21:42:49.883 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.095201420999999994 2024-09-21T21:42:49.883 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.883 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.883 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:49.884 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681640+0000", 2024-09-21T21:42:49.884 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1834000000000005e-05 2024-09-21T21:42:49.884 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.884 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.884 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:42:49.884 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681915+0000", 2024-09-21T21:42:49.884 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00027534199999999998 2024-09-21T21:42:49.884 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.884 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.884 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:49.884 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684230+0000", 2024-09-21T21:42:49.884 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00231424 2024-09-21T21:42:49.884 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.884 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.884 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:49.885 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684959+0000", 2024-09-21T21:42:49.885 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00072921399999999995 2024-09-21T21:42:49.885 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.885 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.885 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:49.885 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.685008+0000", 2024-09-21T21:42:49.885 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8860999999999999e-05 2024-09-21T21:42:49.885 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.885 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.885 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:49.885 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.685062+0000", 2024-09-21T21:42:49.885 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4432000000000003e-05 2024-09-21T21:42:49.885 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:49.885 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:49.886 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:49.886 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.886 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.886 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24837 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:49.886 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586395+0000", 2024-09-21T21:42:49.886 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 27.230005778999999, 2024-09-21T21:42:49.886 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.099700522, 2024-09-21T21:42:49.886 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:49.886 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:49.886 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:49.886 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:49.886 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:42:49.887 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:49.887 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24837 2024-09-21T21:42:49.887 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.887 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:49.887 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.887 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:49.887 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586395+0000", 2024-09-21T21:42:49.887 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:49.887 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.887 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.887 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:49.887 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586395+0000", 2024-09-21T21:42:49.887 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:49.887 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.887 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.887 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:49.888 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586395+0000", 2024-09-21T21:42:49.888 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.06e-07 2024-09-21T21:42:49.888 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.888 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.888 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:49.888 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586401+0000", 2024-09-21T21:42:49.888 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0889999999999996e-06 2024-09-21T21:42:49.888 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.888 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.888 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:49.888 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586410+0000", 2024-09-21T21:42:49.888 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.918e-06 2024-09-21T21:42:49.888 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.888 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.889 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:49.889 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586424+0000", 2024-09-21T21:42:49.889 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3478999999999999e-05 2024-09-21T21:42:49.889 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.889 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.889 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:49.889 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.682590+0000", 2024-09-21T21:42:49.889 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.096166086999999997 2024-09-21T21:42:49.889 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.889 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.889 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:49.889 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.682657+0000", 2024-09-21T21:42:49.889 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6878000000000006e-05 2024-09-21T21:42:49.889 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.890 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.890 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:42:49.890 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.682879+0000", 2024-09-21T21:42:49.890 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00022238 2024-09-21T21:42:49.890 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.890 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.890 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:49.890 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.685073+0000", 2024-09-21T21:42:49.890 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021938119999999998 2024-09-21T21:42:49.890 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.890 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.890 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:49.890 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686038+0000", 2024-09-21T21:42:49.890 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00096539499999999997 2024-09-21T21:42:49.890 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.891 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.891 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:49.891 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686071+0000", 2024-09-21T21:42:49.891 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2514000000000001e-05 2024-09-21T21:42:49.891 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.891 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.891 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:49.891 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686095+0000", 2024-09-21T21:42:49.891 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4363999999999999e-05 2024-09-21T21:42:49.891 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:49.891 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:49.891 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:49.891 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.891 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.891 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24839 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:49.892 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586468+0000", 2024-09-21T21:42:49.892 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 27.229932127000001, 2024-09-21T21:42:49.892 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.099739355000000002, 2024-09-21T21:42:49.892 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:49.892 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:49.892 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:49.892 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:49.892 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:42:49.892 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:49.892 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24839 2024-09-21T21:42:49.892 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.892 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:49.893 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.893 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:49.893 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586468+0000", 2024-09-21T21:42:49.893 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:49.893 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.893 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.893 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:49.893 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586468+0000", 2024-09-21T21:42:49.893 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:49.893 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.893 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.893 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:49.893 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586469+0000", 2024-09-21T21:42:49.893 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2200000000000004e-07 2024-09-21T21:42:49.893 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.894 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.894 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:49.894 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586472+0000", 2024-09-21T21:42:49.894 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9960000000000002e-06 2024-09-21T21:42:49.894 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.894 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.894 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:49.894 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586479+0000", 2024-09-21T21:42:49.894 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2699999999999999e-06 2024-09-21T21:42:49.894 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.894 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.894 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:49.894 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586485+0000", 2024-09-21T21:42:49.894 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6999999999999996e-06 2024-09-21T21:42:49.895 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.895 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.895 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:49.895 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.683276+0000", 2024-09-21T21:42:49.895 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.096791618999999995 2024-09-21T21:42:49.895 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.895 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.895 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:49.895 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.683326+0000", 2024-09-21T21:42:49.895 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9384e-05 2024-09-21T21:42:49.895 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.895 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.895 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:42:49.895 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.683655+0000", 2024-09-21T21:42:49.895 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00032965700000000002 2024-09-21T21:42:49.896 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.896 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.896 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:49.896 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686120+0000", 2024-09-21T21:42:49.896 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002464819 2024-09-21T21:42:49.896 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.896 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.896 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:49.896 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686159+0000", 2024-09-21T21:42:49.896 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9181000000000003e-05 2024-09-21T21:42:49.896 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.896 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.896 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:49.896 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686185+0000", 2024-09-21T21:42:49.897 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5907000000000001e-05 2024-09-21T21:42:49.897 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.897 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.897 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:49.897 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686208+0000", 2024-09-21T21:42:49.897 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.23e-05 2024-09-21T21:42:49.897 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:49.897 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:49.897 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:49.897 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.897 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.897 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24841 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:49.897 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586492+0000", 2024-09-21T21:42:49.897 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 27.229907964999999, 2024-09-21T21:42:49.897 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.101672077, 2024-09-21T21:42:49.898 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:49.898 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:49.898 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:49.898 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:49.898 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:42:49.898 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:49.898 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24841 2024-09-21T21:42:49.898 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.898 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:49.898 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.898 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:49.898 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586492+0000", 2024-09-21T21:42:49.898 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:49.898 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.899 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.899 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:49.899 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586492+0000", 2024-09-21T21:42:49.899 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:49.899 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.899 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.899 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:49.899 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586493+0000", 2024-09-21T21:42:49.899 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5999999999999999e-07 2024-09-21T21:42:49.899 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.899 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.899 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:49.899 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586496+0000", 2024-09-21T21:42:49.900 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6089999999999999e-06 2024-09-21T21:42:49.900 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.900 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.900 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:49.900 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586502+0000", 2024-09-21T21:42:49.900 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6270000000000002e-06 2024-09-21T21:42:49.900 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.900 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.900 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:49.900 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586507+0000", 2024-09-21T21:42:49.900 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8810000000000004e-06 2024-09-21T21:42:49.900 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.900 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.900 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:49.900 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684273+0000", 2024-09-21T21:42:49.901 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.097766415999999995 2024-09-21T21:42:49.901 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.901 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.901 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:49.901 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684359+0000", 2024-09-21T21:42:49.901 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5574000000000004e-05 2024-09-21T21:42:49.901 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.901 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.901 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:42:49.901 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684568+0000", 2024-09-21T21:42:49.901 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000208529 2024-09-21T21:42:49.901 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.901 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.901 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:49.902 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.687186+0000", 2024-09-21T21:42:49.902 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0026180769999999999 2024-09-21T21:42:49.902 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.902 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.902 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:49.902 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688093+0000", 2024-09-21T21:42:49.902 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00090766999999999996 2024-09-21T21:42:49.902 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.902 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.902 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:49.902 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688142+0000", 2024-09-21T21:42:49.902 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8986000000000002e-05 2024-09-21T21:42:49.902 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.903 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.903 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:49.903 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688165+0000", 2024-09-21T21:42:49.903 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2248000000000001e-05 2024-09-21T21:42:49.903 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:49.903 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:49.903 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:49.903 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.903 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.903 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24843 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:49.903 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586514+0000", 2024-09-21T21:42:49.903 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 27.229886679, 2024-09-21T21:42:49.903 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.101826449, 2024-09-21T21:42:49.903 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:49.904 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:49.904 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:49.904 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:49.904 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:42:49.904 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:49.904 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24843 2024-09-21T21:42:49.904 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.904 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:49.904 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.904 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:49.904 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586514+0000", 2024-09-21T21:42:49.904 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:49.904 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.904 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.905 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:49.905 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586514+0000", 2024-09-21T21:42:49.905 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:49.905 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.905 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.905 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:49.905 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586514+0000", 2024-09-21T21:42:49.905 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9999999999999998e-07 2024-09-21T21:42:49.905 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.905 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.905 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:49.905 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586517+0000", 2024-09-21T21:42:49.905 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4980000000000001e-06 2024-09-21T21:42:49.905 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.906 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.906 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:49.906 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586523+0000", 2024-09-21T21:42:49.906 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4609999999999996e-06 2024-09-21T21:42:49.906 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.906 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.906 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:49.906 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586528+0000", 2024-09-21T21:42:49.906 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2979999999999999e-06 2024-09-21T21:42:49.906 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.906 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.906 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:49.906 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.685104+0000", 2024-09-21T21:42:49.907 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.098575837999999999 2024-09-21T21:42:49.907 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.907 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.907 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:49.907 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.685184+0000", 2024-09-21T21:42:49.907 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9807000000000003e-05 2024-09-21T21:42:49.907 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.907 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.907 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:42:49.907 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.685496+0000", 2024-09-21T21:42:49.907 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00031225900000000001 2024-09-21T21:42:49.907 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.907 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.908 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:49.908 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688238+0000", 2024-09-21T21:42:49.908 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0027419409999999999 2024-09-21T21:42:49.908 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.908 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.908 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:49.908 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688253+0000", 2024-09-21T21:42:49.908 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4331000000000001e-05 2024-09-21T21:42:49.908 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.908 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.908 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:49.908 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688296+0000", 2024-09-21T21:42:49.908 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3801000000000003e-05 2024-09-21T21:42:49.909 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.909 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.909 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:49.909 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688340+0000", 2024-09-21T21:42:49.909 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3815e-05 2024-09-21T21:42:49.909 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:49.909 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:49.909 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:49.909 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.909 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.909 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24845 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:49.909 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586535+0000", 2024-09-21T21:42:49.909 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 27.229865798999999, 2024-09-21T21:42:49.910 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.10285678600000001, 2024-09-21T21:42:49.910 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:49.910 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:49.910 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:49.910 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:49.910 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:42:49.910 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:49.910 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24845 2024-09-21T21:42:49.910 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.910 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:49.910 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.910 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:49.910 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586535+0000", 2024-09-21T21:42:49.910 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:49.910 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.911 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.911 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:49.911 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586535+0000", 2024-09-21T21:42:49.911 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:49.911 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.911 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.911 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:49.911 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586535+0000", 2024-09-21T21:42:49.911 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9000000000000002e-07 2024-09-21T21:42:49.911 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.911 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.911 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:49.911 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586537+0000", 2024-09-21T21:42:49.911 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4569999999999999e-06 2024-09-21T21:42:49.912 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.912 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.912 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:49.912 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586544+0000", 2024-09-21T21:42:49.912 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5849999999999996e-06 2024-09-21T21:42:49.912 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.912 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.912 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:49.912 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586549+0000", 2024-09-21T21:42:49.912 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0089999999999998e-06 2024-09-21T21:42:49.912 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.912 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.912 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:49.912 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686239+0000", 2024-09-21T21:42:49.912 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.099689531999999997 2024-09-21T21:42:49.912 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.913 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.913 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:49.913 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686289+0000", 2024-09-21T21:42:49.913 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.007e-05 2024-09-21T21:42:49.913 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.913 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.913 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:42:49.913 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686633+0000", 2024-09-21T21:42:49.913 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00034428699999999998 2024-09-21T21:42:49.913 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.913 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.913 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:49.913 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688370+0000", 2024-09-21T21:42:49.913 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001736741 2024-09-21T21:42:49.913 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.914 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.914 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:49.914 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.689295+0000", 2024-09-21T21:42:49.914 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00092517900000000004 2024-09-21T21:42:49.914 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.914 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.914 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:49.914 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.689354+0000", 2024-09-21T21:42:49.914 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8799999999999999e-05 2024-09-21T21:42:49.914 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.914 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.914 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:49.914 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.689391+0000", 2024-09-21T21:42:49.914 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7735999999999999e-05 2024-09-21T21:42:49.915 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:49.915 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:49.915 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:49.915 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.915 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.915 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24846 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:49.915 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586564+0000", 2024-09-21T21:42:49.915 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 27.229836770999999, 2024-09-21T21:42:49.915 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.10375692, 2024-09-21T21:42:49.915 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:49.915 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:49.915 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:49.915 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:49.915 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:42:49.916 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:49.916 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24846 2024-09-21T21:42:49.916 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.916 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:49.916 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.916 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:49.916 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586564+0000", 2024-09-21T21:42:49.916 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:49.916 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.916 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.916 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:49.916 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586564+0000", 2024-09-21T21:42:49.916 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:49.916 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.916 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.917 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:49.917 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586564+0000", 2024-09-21T21:42:49.917 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4799999999999998e-07 2024-09-21T21:42:49.917 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.917 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.917 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:49.917 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586567+0000", 2024-09-21T21:42:49.917 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1669999999999999e-06 2024-09-21T21:42:49.917 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.917 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.917 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:49.917 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586578+0000", 2024-09-21T21:42:49.917 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0443e-05 2024-09-21T21:42:49.918 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.918 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.918 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:49.918 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586583+0000", 2024-09-21T21:42:49.918 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6790000000000002e-06 2024-09-21T21:42:49.918 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.918 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.918 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:49.918 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.687215+0000", 2024-09-21T21:42:49.918 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.100631973 2024-09-21T21:42:49.918 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.918 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.918 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:49.918 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.687311+0000", 2024-09-21T21:42:49.918 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5764e-05 2024-09-21T21:42:49.919 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.919 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.919 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:42:49.919 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.687663+0000", 2024-09-21T21:42:49.919 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00035167 2024-09-21T21:42:49.919 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.919 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.919 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:49.919 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690236+0000", 2024-09-21T21:42:49.919 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025733370000000002 2024-09-21T21:42:49.919 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.919 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.919 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:49.919 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690250+0000", 2024-09-21T21:42:49.920 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3548e-05 2024-09-21T21:42:49.920 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.920 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.920 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:49.920 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690298+0000", 2024-09-21T21:42:49.920 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7948999999999998e-05 2024-09-21T21:42:49.920 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.920 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.920 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:49.920 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690321+0000", 2024-09-21T21:42:49.920 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2841999999999999e-05 2024-09-21T21:42:49.920 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:49.920 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:49.920 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:49.921 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.921 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.921 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24849 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:49.921 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586633+0000", 2024-09-21T21:42:49.921 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 27.229767105000001, 2024-09-21T21:42:49.921 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.104506548, 2024-09-21T21:42:49.921 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:49.921 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:49.921 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:49.921 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:49.921 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:42:49.921 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:49.921 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24849 2024-09-21T21:42:49.922 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.922 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:49.922 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.922 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:49.922 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586633+0000", 2024-09-21T21:42:49.922 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:49.922 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.922 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.922 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:49.922 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586633+0000", 2024-09-21T21:42:49.922 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:49.922 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.922 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.922 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:49.923 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586634+0000", 2024-09-21T21:42:49.923 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9100000000000004e-07 2024-09-21T21:42:49.923 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.923 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.923 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:49.923 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586637+0000", 2024-09-21T21:42:49.923 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1999999999999999e-06 2024-09-21T21:42:49.923 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.923 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.923 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:49.923 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586645+0000", 2024-09-21T21:42:49.923 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7759999999999997e-06 2024-09-21T21:42:49.923 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.923 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.924 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:49.924 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586652+0000", 2024-09-21T21:42:49.924 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3560000000000004e-06 2024-09-21T21:42:49.924 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.924 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.924 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:49.924 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688395+0000", 2024-09-21T21:42:49.924 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.101743239 2024-09-21T21:42:49.924 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.924 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.924 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:49.924 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688464+0000", 2024-09-21T21:42:49.924 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8273000000000002e-05 2024-09-21T21:42:49.924 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.925 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.925 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:42:49.925 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688786+0000", 2024-09-21T21:42:49.925 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000322064 2024-09-21T21:42:49.925 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.925 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.925 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:49.925 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690364+0000", 2024-09-21T21:42:49.925 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001578084 2024-09-21T21:42:49.925 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.925 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.925 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:49.925 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.691050+0000", 2024-09-21T21:42:49.925 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00068623399999999998 2024-09-21T21:42:49.925 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.925 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.926 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:49.926 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.691090+0000", 2024-09-21T21:42:49.926 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9406999999999997e-05 2024-09-21T21:42:49.926 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.926 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.926 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:49.926 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.691140+0000", 2024-09-21T21:42:49.926 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0324000000000001e-05 2024-09-21T21:42:49.926 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:49.926 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:49.926 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:49.926 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.926 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.926 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24851 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:49.926 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586661+0000", 2024-09-21T21:42:49.927 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 27.229739818999999, 2024-09-21T21:42:49.927 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.10536891399999999, 2024-09-21T21:42:49.927 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:49.927 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:49.927 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:49.927 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:49.927 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:42:49.927 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:49.927 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24851 2024-09-21T21:42:49.927 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.927 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:49.927 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.927 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:49.927 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586661+0000", 2024-09-21T21:42:49.928 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:49.928 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.928 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.928 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:49.928 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586661+0000", 2024-09-21T21:42:49.928 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:49.928 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.928 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.928 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:49.928 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586661+0000", 2024-09-21T21:42:49.928 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4700000000000002e-07 2024-09-21T21:42:49.928 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.928 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.928 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:49.928 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586664+0000", 2024-09-21T21:42:49.929 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.672e-06 2024-09-21T21:42:49.929 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.929 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.929 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:49.929 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586671+0000", 2024-09-21T21:42:49.929 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9959999999999996e-06 2024-09-21T21:42:49.929 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.929 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.929 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:49.929 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586676+0000", 2024-09-21T21:42:49.929 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4020000000000001e-06 2024-09-21T21:42:49.929 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.929 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.929 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:49.930 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.689438+0000", 2024-09-21T21:42:49.930 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.102761503 2024-09-21T21:42:49.930 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.930 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.930 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:49.930 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.689524+0000", 2024-09-21T21:42:49.930 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6383000000000003e-05 2024-09-21T21:42:49.930 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.930 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.930 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:42:49.930 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.689769+0000", 2024-09-21T21:42:49.930 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00024504699999999997 2024-09-21T21:42:49.930 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.930 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.930 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:49.931 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.691936+0000", 2024-09-21T21:42:49.931 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021665130000000001 2024-09-21T21:42:49.931 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.931 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.931 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:49.931 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.691947+0000", 2024-09-21T21:42:49.931 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1198e-05 2024-09-21T21:42:49.931 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.931 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.931 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:49.931 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.691992+0000", 2024-09-21T21:42:49.931 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4774000000000002e-05 2024-09-21T21:42:49.931 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.932 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.932 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:49.932 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.692030+0000", 2024-09-21T21:42:49.932 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7978999999999997e-05 2024-09-21T21:42:49.932 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:49.932 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:49.932 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:49.932 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.932 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.932 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24853 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:49.932 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586684+0000", 2024-09-21T21:42:49.932 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 27.229716862, 2024-09-21T21:42:49.932 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.105936852, 2024-09-21T21:42:49.933 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:49.933 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:49.933 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:49.933 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:49.933 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:42:49.933 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:49.933 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24853 2024-09-21T21:42:49.933 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.933 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:49.933 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.933 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:49.933 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586684+0000", 2024-09-21T21:42:49.933 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:49.933 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.934 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.934 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:49.934 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586684+0000", 2024-09-21T21:42:49.934 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:49.934 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.934 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.934 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:49.934 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586684+0000", 2024-09-21T21:42:49.934 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.68e-07 2024-09-21T21:42:49.934 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.934 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.934 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:49.934 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586687+0000", 2024-09-21T21:42:49.935 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.774e-06 2024-09-21T21:42:49.935 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.935 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.935 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:49.935 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586694+0000", 2024-09-21T21:42:49.935 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6959999999999999e-06 2024-09-21T21:42:49.935 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.935 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.935 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:49.935 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586698+0000", 2024-09-21T21:42:49.935 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7489999999999998e-06 2024-09-21T21:42:49.935 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.935 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.936 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:49.936 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690395+0000", 2024-09-21T21:42:49.936 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.103696645 2024-09-21T21:42:49.936 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.936 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.936 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:49.936 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690440+0000", 2024-09-21T21:42:49.936 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4835e-05 2024-09-21T21:42:49.936 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.936 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.936 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:42:49.936 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690654+0000", 2024-09-21T21:42:49.936 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000214114 2024-09-21T21:42:49.936 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.936 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.937 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:49.937 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.692041+0000", 2024-09-21T21:42:49.937 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0013866519999999999 2024-09-21T21:42:49.937 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.937 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.937 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:49.937 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.692558+0000", 2024-09-21T21:42:49.937 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00051717199999999999 2024-09-21T21:42:49.937 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.937 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.937 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:49.937 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.692586+0000", 2024-09-21T21:42:49.937 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8504e-05 2024-09-21T21:42:49.937 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:49.937 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:49.938 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:49.938 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.692620+0000", 2024-09-21T21:42:49.938 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4143000000000002e-05 2024-09-21T21:42:49.938 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:49.938 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:49.938 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:49.938 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:49.938 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:49.938 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:49.940 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T21:42:49.940 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:49.938+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:50.040 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:50.039+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:50.045 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:50.045 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:42:50.045 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 0 2024-09-21T21:42:50.045 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:50.054 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T21:42:50.141 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:50.140+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:50.198 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:50.198 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:42:50.199 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:42:50.199 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:42:50.199 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:50.207 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T21:42:50.238 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T21:42:50.250 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:50.249+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:50.321 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:50.321 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 20, 2024-09-21T21:42:50.321 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 600, 2024-09-21T21:42:50.321 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:42:50.321 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.322 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11319 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 5492736~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:50.322 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.712385+0000", 2024-09-21T21:42:50.322 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 59.606186424999997, 2024-09-21T21:42:50.322 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078627889000000006, 2024-09-21T21:42:50.322 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:50.322 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:50.322 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:50.322 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:50.322 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:50.322 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:50.323 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11319 2024-09-21T21:42:50.323 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.323 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:50.323 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.323 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:50.323 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712385+0000", 2024-09-21T21:42:50.323 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:50.323 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.323 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.323 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:50.323 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712385+0000", 2024-09-21T21:42:50.324 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:50.324 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.324 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.324 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:50.324 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712385+0000", 2024-09-21T21:42:50.324 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2800000000000002e-07 2024-09-21T21:42:50.324 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.324 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.324 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:50.324 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712387+0000", 2024-09-21T21:42:50.324 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1569999999999998e-06 2024-09-21T21:42:50.324 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.325 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.325 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:50.325 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712393+0000", 2024-09-21T21:42:50.325 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9370000000000001e-06 2024-09-21T21:42:50.325 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.325 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.325 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:50.325 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712398+0000", 2024-09-21T21:42:50.325 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5020000000000003e-06 2024-09-21T21:42:50.325 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.325 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.325 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:50.325 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.790746+0000", 2024-09-21T21:42:50.325 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078348589999999996 2024-09-21T21:42:50.325 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.325 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.326 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:50.326 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.790844+0000", 2024-09-21T21:42:50.326 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.7526000000000001e-05 2024-09-21T21:42:50.326 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.326 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.326 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:50.326 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.791013+0000", 2024-09-21T21:42:50.326 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00016874899999999999 2024-09-21T21:42:50.326 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:50.326 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:50.326 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:50.326 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.326 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.327 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11320 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 6180864~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:50.327 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.712405+0000", 2024-09-21T21:42:50.327 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 59.606166465999998, 2024-09-21T21:42:50.327 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078882870999999993, 2024-09-21T21:42:50.327 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:50.327 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:50.327 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:50.327 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:50.327 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:50.327 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:50.327 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11320 2024-09-21T21:42:50.327 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.327 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:50.327 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.327 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:50.328 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712405+0000", 2024-09-21T21:42:50.328 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:50.328 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.328 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.328 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:50.328 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712405+0000", 2024-09-21T21:42:50.328 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:50.328 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.328 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.328 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:50.328 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.713942+0000", 2024-09-21T21:42:50.328 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.001536882 2024-09-21T21:42:50.328 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.328 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.328 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:50.328 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.713946+0000", 2024-09-21T21:42:50.329 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4549999999999997e-06 2024-09-21T21:42:50.329 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.329 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.329 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:50.329 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.713958+0000", 2024-09-21T21:42:50.329 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1775e-05 2024-09-21T21:42:50.329 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.329 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.329 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:50.329 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.713965+0000", 2024-09-21T21:42:50.329 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5100000000000001e-06 2024-09-21T21:42:50.329 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.329 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.329 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:50.330 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.791059+0000", 2024-09-21T21:42:50.330 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.077094036000000005 2024-09-21T21:42:50.330 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.330 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.330 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:50.330 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.791175+0000", 2024-09-21T21:42:50.330 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000115895 2024-09-21T21:42:50.330 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.330 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.330 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:50.330 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.791288+0000", 2024-09-21T21:42:50.330 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000112318 2024-09-21T21:42:50.330 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:50.330 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:50.331 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:50.331 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.331 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.331 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11325 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 5414912~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:50.331 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.714095+0000", 2024-09-21T21:42:50.331 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 59.604476470999998, 2024-09-21T21:42:50.331 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078631894999999993, 2024-09-21T21:42:50.331 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:50.331 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:50.331 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:50.332 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:50.332 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:50.332 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:50.332 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11325 2024-09-21T21:42:50.332 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.332 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:50.332 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.332 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:50.332 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714095+0000", 2024-09-21T21:42:50.332 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:50.332 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.333 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.333 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:50.333 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714095+0000", 2024-09-21T21:42:50.333 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:50.333 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.333 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.333 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:50.333 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714095+0000", 2024-09-21T21:42:50.333 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7500000000000001e-07 2024-09-21T21:42:50.333 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.333 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.334 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:50.334 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714097+0000", 2024-09-21T21:42:50.334 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.12e-06 2024-09-21T21:42:50.334 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.334 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.334 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:50.334 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714104+0000", 2024-09-21T21:42:50.334 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3520000000000003e-06 2024-09-21T21:42:50.334 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.334 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.334 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:50.335 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714108+0000", 2024-09-21T21:42:50.335 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8810000000000004e-06 2024-09-21T21:42:50.335 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.335 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.335 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:50.335 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792509+0000", 2024-09-21T21:42:50.335 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078400739999999997 2024-09-21T21:42:50.335 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.335 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.335 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:50.335 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792613+0000", 2024-09-21T21:42:50.335 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000103939 2024-09-21T21:42:50.335 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.336 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.336 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:50.336 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792727+0000", 2024-09-21T21:42:50.336 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011348799999999999 2024-09-21T21:42:50.336 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:50.336 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:50.336 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:50.336 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.336 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.336 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11326 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 6103040~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:50.336 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.714115+0000", 2024-09-21T21:42:50.336 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 59.604456442999997, 2024-09-21T21:42:50.336 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078963056000000004, 2024-09-21T21:42:50.336 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:50.337 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:50.337 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:50.337 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:50.337 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:50.337 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:50.337 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11326 2024-09-21T21:42:50.337 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.337 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:50.337 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.337 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:50.337 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714115+0000", 2024-09-21T21:42:50.337 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:50.337 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.337 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.337 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:50.337 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714115+0000", 2024-09-21T21:42:50.338 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:50.338 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.338 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.338 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:50.338 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.715940+0000", 2024-09-21T21:42:50.338 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0018254969999999999 2024-09-21T21:42:50.338 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.338 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.338 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:50.338 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.715945+0000", 2024-09-21T21:42:50.338 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6120000000000001e-06 2024-09-21T21:42:50.338 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.338 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.339 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:50.339 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.715954+0000", 2024-09-21T21:42:50.339 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.5899999999999997e-06 2024-09-21T21:42:50.339 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.339 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.339 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:50.339 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.715959+0000", 2024-09-21T21:42:50.339 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0849999999999996e-06 2024-09-21T21:42:50.339 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.339 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.339 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:50.339 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792771+0000", 2024-09-21T21:42:50.339 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.076811829999999998 2024-09-21T21:42:50.340 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.340 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.340 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:50.340 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792882+0000", 2024-09-21T21:42:50.340 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000110857 2024-09-21T21:42:50.340 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.340 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.340 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:50.340 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.793078+0000", 2024-09-21T21:42:50.340 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00019558499999999999 2024-09-21T21:42:50.340 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:50.340 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:50.340 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:50.340 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.340 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.341 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11332 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 7565312~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:50.341 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.716063+0000", 2024-09-21T21:42:50.341 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 59.602507909000003, 2024-09-21T21:42:50.341 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078787482000000006, 2024-09-21T21:42:50.341 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:50.341 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:50.341 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:50.341 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:50.341 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:50.341 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:50.341 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11332 2024-09-21T21:42:50.341 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.341 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:50.341 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.342 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:50.342 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.716063+0000", 2024-09-21T21:42:50.342 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:50.342 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.342 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.342 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:50.342 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.716063+0000", 2024-09-21T21:42:50.342 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:50.342 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.343 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.343 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:50.343 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.717944+0000", 2024-09-21T21:42:50.343 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0018812309999999999 2024-09-21T21:42:50.343 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.343 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.343 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:50.343 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.717949+0000", 2024-09-21T21:42:50.343 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9389999999999996e-06 2024-09-21T21:42:50.343 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.343 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.343 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:50.343 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.717963+0000", 2024-09-21T21:42:50.343 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.3899e-05 2024-09-21T21:42:50.344 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.344 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.344 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:50.344 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.717970+0000", 2024-09-21T21:42:50.344 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.426e-06 2024-09-21T21:42:50.344 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.344 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.344 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:50.344 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.794573+0000", 2024-09-21T21:42:50.344 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.076602990999999995 2024-09-21T21:42:50.344 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.344 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.345 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:50.345 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.794682+0000", 2024-09-21T21:42:50.345 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000109541 2024-09-21T21:42:50.345 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.345 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.345 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:50.345 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.794851+0000", 2024-09-21T21:42:50.345 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000168455 2024-09-21T21:42:50.345 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:50.345 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:50.345 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:50.345 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.345 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.345 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11339 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 6266880~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:50.346 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.718117+0000", 2024-09-21T21:42:50.346 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 59.600453883, 2024-09-21T21:42:50.346 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078703717000000006, 2024-09-21T21:42:50.346 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:50.346 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:50.346 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:50.346 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:50.346 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:50.346 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:50.346 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11339 2024-09-21T21:42:50.346 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.346 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:50.346 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.346 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:50.347 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.718117+0000", 2024-09-21T21:42:50.347 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:50.347 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.347 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.347 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:50.347 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.718117+0000", 2024-09-21T21:42:50.347 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:50.347 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.347 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.347 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:50.347 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720019+0000", 2024-09-21T21:42:50.347 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0019012549999999999 2024-09-21T21:42:50.348 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.348 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.348 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:50.348 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720024+0000", 2024-09-21T21:42:50.348 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1100000000000002e-06 2024-09-21T21:42:50.348 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.348 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.348 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:50.348 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720035+0000", 2024-09-21T21:42:50.348 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.094e-05 2024-09-21T21:42:50.348 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.348 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.348 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:50.348 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720040+0000", 2024-09-21T21:42:50.349 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1830000000000002e-06 2024-09-21T21:42:50.349 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.349 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.349 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:50.349 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796618+0000", 2024-09-21T21:42:50.349 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.076578033000000004 2024-09-21T21:42:50.349 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.349 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.349 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:50.349 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796696+0000", 2024-09-21T21:42:50.349 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7836000000000005e-05 2024-09-21T21:42:50.349 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.349 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.349 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:50.349 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796821+0000", 2024-09-21T21:42:50.350 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00012536000000000001 2024-09-21T21:42:50.350 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:50.350 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:50.350 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:50.350 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.350 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.350 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11345 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 5472256~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:50.350 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.720116+0000", 2024-09-21T21:42:50.350 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 59.598454983000003, 2024-09-21T21:42:50.350 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078667730000000005, 2024-09-21T21:42:50.350 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:50.350 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:50.350 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:50.350 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:50.350 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:50.351 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:50.351 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11345 2024-09-21T21:42:50.351 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.351 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:50.351 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.351 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:50.351 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720116+0000", 2024-09-21T21:42:50.351 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:50.351 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.442 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.443 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:50.443 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720116+0000", 2024-09-21T21:42:50.443 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:50.443 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.443 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.443 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:50.443 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720116+0000", 2024-09-21T21:42:50.443 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.04e-07 2024-09-21T21:42:50.443 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.443 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.443 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:50.443 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720118+0000", 2024-09-21T21:42:50.443 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.181e-06 2024-09-21T21:42:50.443 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.444 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.444 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:50.444 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720121+0000", 2024-09-21T21:42:50.444 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0199999999999999e-06 2024-09-21T21:42:50.444 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.444 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.444 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:50.444 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720123+0000", 2024-09-21T21:42:50.444 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.5469999999999999e-06 2024-09-21T21:42:50.444 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.444 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.444 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:50.445 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798626+0000", 2024-09-21T21:42:50.445 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078503137000000001 2024-09-21T21:42:50.445 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.445 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.445 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:50.445 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798677+0000", 2024-09-21T21:42:50.445 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0930999999999997e-05 2024-09-21T21:42:50.445 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.445 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.445 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:50.445 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798784+0000", 2024-09-21T21:42:50.445 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010671 2024-09-21T21:42:50.445 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:50.445 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:50.445 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:50.446 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.446 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.446 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11346 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 5500928~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:50.446 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.720126+0000", 2024-09-21T21:42:50.446 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 59.598444909999998, 2024-09-21T21:42:50.446 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078863421000000003, 2024-09-21T21:42:50.446 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:50.446 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:50.446 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:50.446 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:50.446 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:50.446 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:50.446 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11346 2024-09-21T21:42:50.446 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.447 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:50.447 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.447 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:50.447 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720126+0000", 2024-09-21T21:42:50.447 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:50.448 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.448 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.448 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:50.448 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720126+0000", 2024-09-21T21:42:50.448 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:50.448 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.448 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.448 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:50.448 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.722145+0000", 2024-09-21T21:42:50.448 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0020183219999999999 2024-09-21T21:42:50.448 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.448 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.448 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:50.448 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.722150+0000", 2024-09-21T21:42:50.448 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1760000000000001e-06 2024-09-21T21:42:50.449 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.449 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.449 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:50.449 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.722161+0000", 2024-09-21T21:42:50.449 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1566000000000001e-05 2024-09-21T21:42:50.449 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.449 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.449 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:50.449 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.722169+0000", 2024-09-21T21:42:50.449 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.2840000000000001e-06 2024-09-21T21:42:50.449 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.449 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.449 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:50.449 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798815+0000", 2024-09-21T21:42:50.450 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.076646607000000005 2024-09-21T21:42:50.450 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.450 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.450 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:50.450 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798870+0000", 2024-09-21T21:42:50.450 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4486e-05 2024-09-21T21:42:50.450 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.450 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.450 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:50.450 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798990+0000", 2024-09-21T21:42:50.450 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011998 2024-09-21T21:42:50.451 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:50.451 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:50.451 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:50.451 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.451 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.451 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13629 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7852032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:50.451 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943319+0000", 2024-09-21T21:42:50.451 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 59.375252267999997, 2024-09-21T21:42:50.451 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078610975999999999, 2024-09-21T21:42:50.451 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:50.451 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:50.451 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:50.451 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:50.451 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:50.452 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:50.452 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13629 2024-09-21T21:42:50.452 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.452 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:50.452 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.452 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:50.452 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943319+0000", 2024-09-21T21:42:50.452 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:50.452 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.452 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.452 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:50.452 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943319+0000", 2024-09-21T21:42:50.452 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:50.452 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.452 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.453 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:50.453 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943319+0000", 2024-09-21T21:42:50.453 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6199999999999999e-07 2024-09-21T21:42:50.453 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.453 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.453 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:50.453 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943321+0000", 2024-09-21T21:42:50.453 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1009999999999999e-06 2024-09-21T21:42:50.453 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.453 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.453 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:50.454 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943328+0000", 2024-09-21T21:42:50.454 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.1029999999999996e-06 2024-09-21T21:42:50.454 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.454 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.454 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:50.454 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943333+0000", 2024-09-21T21:42:50.454 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7709999999999997e-06 2024-09-21T21:42:50.454 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.454 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.454 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:50.455 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.021679+0000", 2024-09-21T21:42:50.455 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078345673000000005 2024-09-21T21:42:50.455 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.455 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.455 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:50.455 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.021771+0000", 2024-09-21T21:42:50.455 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2201999999999996e-05 2024-09-21T21:42:50.455 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.455 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.455 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:50.455 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.021930+0000", 2024-09-21T21:42:50.455 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00015876400000000001 2024-09-21T21:42:50.455 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:50.455 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:50.456 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:50.456 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.456 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.456 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13630 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7880704~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:50.456 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943339+0000", 2024-09-21T21:42:50.456 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 59.375232080000004, 2024-09-21T21:42:50.456 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078939410000000002, 2024-09-21T21:42:50.456 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:50.456 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:50.456 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:50.456 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:50.456 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:50.456 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:50.456 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13630 2024-09-21T21:42:50.457 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.457 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:50.457 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.457 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:50.457 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943339+0000", 2024-09-21T21:42:50.457 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:50.457 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.457 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.457 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:50.457 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943339+0000", 2024-09-21T21:42:50.457 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:50.458 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.458 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.458 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:50.458 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943339+0000", 2024-09-21T21:42:50.458 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8000000000000001e-07 2024-09-21T21:42:50.459 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.459 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.459 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:50.459 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943341+0000", 2024-09-21T21:42:50.459 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9850000000000001e-06 2024-09-21T21:42:50.459 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.459 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.459 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:50.459 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943347+0000", 2024-09-21T21:42:50.459 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2979999999999999e-06 2024-09-21T21:42:50.459 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.460 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.460 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:50.460 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943354+0000", 2024-09-21T21:42:50.460 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.2969999999999998e-06 2024-09-21T21:42:50.460 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.460 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.460 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:50.460 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.021996+0000", 2024-09-21T21:42:50.460 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078641680000000005 2024-09-21T21:42:50.460 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.460 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.460 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:50.460 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022104+0000", 2024-09-21T21:42:50.460 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000107902 2024-09-21T21:42:50.461 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.461 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.461 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:50.461 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022278+0000", 2024-09-21T21:42:50.461 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00017486800000000001 2024-09-21T21:42:50.461 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:50.461 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:50.461 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:50.461 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.461 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.461 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13631 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7909376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:50.461 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943362+0000", 2024-09-21T21:42:50.461 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 59.375209259999998, 2024-09-21T21:42:50.461 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079195644999999995, 2024-09-21T21:42:50.462 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:50.462 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:50.462 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:50.462 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:50.462 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:50.462 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:50.462 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13631 2024-09-21T21:42:50.462 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.462 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:50.462 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.462 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:50.462 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943362+0000", 2024-09-21T21:42:50.462 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:50.462 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.463 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.463 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:50.463 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943362+0000", 2024-09-21T21:42:50.463 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:50.463 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.463 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.463 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:50.463 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943362+0000", 2024-09-21T21:42:50.463 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8799999999999998e-07 2024-09-21T21:42:50.463 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.463 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.463 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:50.464 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943364+0000", 2024-09-21T21:42:50.464 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9850000000000001e-06 2024-09-21T21:42:50.464 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.464 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.464 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:50.464 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943371+0000", 2024-09-21T21:42:50.464 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0650000000000001e-06 2024-09-21T21:42:50.464 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.464 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.464 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:50.464 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943376+0000", 2024-09-21T21:42:50.464 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8250000000000004e-06 2024-09-21T21:42:50.464 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.464 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.465 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:50.465 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022329+0000", 2024-09-21T21:42:50.465 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078952888999999998 2024-09-21T21:42:50.465 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.465 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.465 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:50.465 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022419+0000", 2024-09-21T21:42:50.465 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.9738e-05 2024-09-21T21:42:50.465 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.465 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.466 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:50.466 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022558+0000", 2024-09-21T21:42:50.466 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00013875500000000001 2024-09-21T21:42:50.466 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:50.466 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:50.466 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:50.466 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.466 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.466 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13632 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7966720~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:50.466 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943382+0000", 2024-09-21T21:42:50.466 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 59.375189026000001, 2024-09-21T21:42:50.466 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079495727000000002, 2024-09-21T21:42:50.466 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:50.467 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:50.467 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:50.467 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:50.467 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:50.467 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:50.467 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13632 2024-09-21T21:42:50.467 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.467 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:50.467 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.467 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:50.467 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943382+0000", 2024-09-21T21:42:50.467 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:50.467 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.467 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.468 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:50.468 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943382+0000", 2024-09-21T21:42:50.468 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:50.468 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.468 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.468 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:50.468 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943382+0000", 2024-09-21T21:42:50.468 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7e-07 2024-09-21T21:42:50.468 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.468 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.468 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:50.468 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943385+0000", 2024-09-21T21:42:50.468 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2230000000000001e-06 2024-09-21T21:42:50.469 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.469 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.469 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:50.469 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943391+0000", 2024-09-21T21:42:50.469 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6810000000000001e-06 2024-09-21T21:42:50.469 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.469 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.469 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:50.469 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943397+0000", 2024-09-21T21:42:50.469 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6559999999999997e-06 2024-09-21T21:42:50.469 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.469 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.469 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:50.469 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022601+0000", 2024-09-21T21:42:50.469 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079204051999999997 2024-09-21T21:42:50.469 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.470 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.470 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:50.470 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022704+0000", 2024-09-21T21:42:50.470 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010329599999999999 2024-09-21T21:42:50.470 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.470 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.470 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:50.470 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022878+0000", 2024-09-21T21:42:50.470 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000173449 2024-09-21T21:42:50.470 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:50.470 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:50.470 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:50.470 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.470 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.470 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13633 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7938048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:50.471 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943403+0000", 2024-09-21T21:42:50.471 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 59.375168096000003, 2024-09-21T21:42:50.471 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079794697999999997, 2024-09-21T21:42:50.471 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:50.471 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:50.471 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:50.471 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:50.471 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:50.471 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:50.471 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13633 2024-09-21T21:42:50.471 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.472 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:50.472 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.472 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:50.472 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943403+0000", 2024-09-21T21:42:50.472 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:50.472 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.472 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.472 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:50.472 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943403+0000", 2024-09-21T21:42:50.472 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:50.472 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.472 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.472 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:50.472 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943403+0000", 2024-09-21T21:42:50.472 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.65e-07 2024-09-21T21:42:50.473 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.473 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.473 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:50.473 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943405+0000", 2024-09-21T21:42:50.473 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0020000000000001e-06 2024-09-21T21:42:50.473 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.473 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.473 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:50.473 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943412+0000", 2024-09-21T21:42:50.473 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.8379999999999996e-06 2024-09-21T21:42:50.473 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.473 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.473 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:50.474 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943417+0000", 2024-09-21T21:42:50.474 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7650000000000001e-06 2024-09-21T21:42:50.474 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.474 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.474 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:50.474 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022922+0000", 2024-09-21T21:42:50.474 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079504512999999999 2024-09-21T21:42:50.474 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.474 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.474 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:50.474 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023039+0000", 2024-09-21T21:42:50.474 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000117044 2024-09-21T21:42:50.474 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.475 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.475 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:50.475 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023198+0000", 2024-09-21T21:42:50.475 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000159171 2024-09-21T21:42:50.475 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:50.475 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:50.475 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:50.475 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.475 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.475 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13634 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7995392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:50.475 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943423+0000", 2024-09-21T21:42:50.476 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 59.375148387999999, 2024-09-21T21:42:50.476 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.080142358999999996, 2024-09-21T21:42:50.476 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:50.476 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:50.476 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:50.476 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:50.476 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:50.476 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:50.476 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13634 2024-09-21T21:42:50.476 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.476 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:50.476 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.476 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:50.476 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943423+0000", 2024-09-21T21:42:50.476 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:50.477 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.477 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.477 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:50.477 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943423+0000", 2024-09-21T21:42:50.477 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:50.477 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.477 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.477 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:50.477 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943423+0000", 2024-09-21T21:42:50.477 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.58e-07 2024-09-21T21:42:50.477 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.477 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.477 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:50.477 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943425+0000", 2024-09-21T21:42:50.477 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9180000000000001e-06 2024-09-21T21:42:50.478 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.478 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.478 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:50.478 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943431+0000", 2024-09-21T21:42:50.478 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4439999999999996e-06 2024-09-21T21:42:50.478 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.478 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.478 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:50.478 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943436+0000", 2024-09-21T21:42:50.478 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.887e-06 2024-09-21T21:42:50.478 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.478 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.478 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:50.478 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023266+0000", 2024-09-21T21:42:50.479 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079829622000000003 2024-09-21T21:42:50.479 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.479 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.479 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:50.479 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023384+0000", 2024-09-21T21:42:50.479 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000117658 2024-09-21T21:42:50.479 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.479 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.479 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:50.479 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023565+0000", 2024-09-21T21:42:50.479 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00018147200000000001 2024-09-21T21:42:50.479 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:50.480 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:50.480 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:50.480 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.480 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.480 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13635 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 8024064~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:50.480 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943447+0000", 2024-09-21T21:42:50.480 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 59.375123848000001, 2024-09-21T21:42:50.480 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.080417653000000006, 2024-09-21T21:42:50.480 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:50.480 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:50.480 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:50.480 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:50.480 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:50.480 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:50.481 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13635 2024-09-21T21:42:50.481 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.481 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:50.481 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.481 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:50.481 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943447+0000", 2024-09-21T21:42:50.481 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:50.481 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.481 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.481 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:50.481 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943447+0000", 2024-09-21T21:42:50.481 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:50.481 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.481 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.482 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:50.482 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943448+0000", 2024-09-21T21:42:50.482 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1699999999999999e-07 2024-09-21T21:42:50.482 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.482 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.482 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:50.482 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943450+0000", 2024-09-21T21:42:50.482 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1639999999999999e-06 2024-09-21T21:42:50.482 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.482 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.482 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:50.482 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943456+0000", 2024-09-21T21:42:50.482 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8379999999999998e-06 2024-09-21T21:42:50.483 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.483 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.483 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:50.483 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943460+0000", 2024-09-21T21:42:50.483 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4700000000000004e-06 2024-09-21T21:42:50.483 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.483 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.483 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:50.483 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023595+0000", 2024-09-21T21:42:50.483 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.080134869999999997 2024-09-21T21:42:50.483 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.483 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.483 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:50.484 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023688+0000", 2024-09-21T21:42:50.484 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2794000000000006e-05 2024-09-21T21:42:50.484 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.484 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.484 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:50.484 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023865+0000", 2024-09-21T21:42:50.484 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0001771 2024-09-21T21:42:50.484 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:50.484 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:50.484 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:50.484 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.484 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.484 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13636 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 8052736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:50.484 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943467+0000", 2024-09-21T21:42:50.485 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 59.375104110999999, 2024-09-21T21:42:50.485 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.080694399999999999, 2024-09-21T21:42:50.485 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:50.485 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:50.485 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:50.485 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:50.485 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:50.485 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:50.485 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13636 2024-09-21T21:42:50.485 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.486 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:50.486 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.486 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:50.486 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943467+0000", 2024-09-21T21:42:50.486 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:50.486 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.487 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.487 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:50.487 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943467+0000", 2024-09-21T21:42:50.487 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:50.487 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.487 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.488 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:50.488 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944940+0000", 2024-09-21T21:42:50.488 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0014725109999999999 2024-09-21T21:42:50.488 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.488 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.488 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:50.488 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944949+0000", 2024-09-21T21:42:50.488 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2930000000000006e-06 2024-09-21T21:42:50.489 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.489 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.489 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:50.489 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944958+0000", 2024-09-21T21:42:50.489 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.6899999999999998e-06 2024-09-21T21:42:50.489 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.489 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.489 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:50.490 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944966+0000", 2024-09-21T21:42:50.490 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0690000000000002e-06 2024-09-21T21:42:50.490 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.490 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.490 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:50.490 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023900+0000", 2024-09-21T21:42:50.491 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078934371000000003 2024-09-21T21:42:50.491 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.491 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.491 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:50.491 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024009+0000", 2024-09-21T21:42:50.491 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000108857 2024-09-21T21:42:50.491 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.492 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.492 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:50.492 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024161+0000", 2024-09-21T21:42:50.492 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000152609 2024-09-21T21:42:50.492 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:50.492 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:50.493 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:50.493 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.493 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.493 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13637 3.12 3.1dc31b72 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:50.493 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.944978+0000", 2024-09-21T21:42:50.493 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 59.373593176, 2024-09-21T21:42:50.493 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079232604999999998, 2024-09-21T21:42:50.493 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:50.493 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:50.494 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:42:50.494 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:50.494 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:50.494 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:50.494 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13637 2024-09-21T21:42:50.494 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.494 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:50.494 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.495 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:50.495 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944978+0000", 2024-09-21T21:42:50.495 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:50.495 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.495 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.495 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:50.495 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944978+0000", 2024-09-21T21:42:50.495 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:50.495 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.496 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.496 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:50.496 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944978+0000", 2024-09-21T21:42:50.496 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9100000000000004e-07 2024-09-21T21:42:50.496 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.496 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.496 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:50.496 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944981+0000", 2024-09-21T21:42:50.497 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3290000000000001e-06 2024-09-21T21:42:50.497 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.497 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.497 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:50.497 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944987+0000", 2024-09-21T21:42:50.497 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.28e-06 2024-09-21T21:42:50.497 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.497 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.497 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:50.497 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944992+0000", 2024-09-21T21:42:50.498 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8520000000000003e-06 2024-09-21T21:42:50.498 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.498 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.498 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:50.498 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024196+0000", 2024-09-21T21:42:50.498 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079203673000000002 2024-09-21T21:42:50.498 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.498 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.499 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:50.499 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024211+0000", 2024-09-21T21:42:50.499 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.4980000000000001e-05 2024-09-21T21:42:50.499 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:50.499 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:50.499 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:50.499 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.499 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.500 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13638 3.12 3.1dc31b72 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:50.500 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.945000+0000", 2024-09-21T21:42:50.500 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 59.373571048000002, 2024-09-21T21:42:50.500 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079257465999999999, 2024-09-21T21:42:50.500 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:50.500 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:50.500 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:42:50.500 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:50.500 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:50.501 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:50.501 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13638 2024-09-21T21:42:50.501 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.501 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:50.501 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.501 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:50.501 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945000+0000", 2024-09-21T21:42:50.501 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:50.501 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.502 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.502 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:50.502 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945000+0000", 2024-09-21T21:42:50.502 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:50.502 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.502 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.502 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:50.502 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945001+0000", 2024-09-21T21:42:50.502 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.32e-07 2024-09-21T21:42:50.503 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.503 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.503 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:50.503 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945003+0000", 2024-09-21T21:42:50.503 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2309999999999999e-06 2024-09-21T21:42:50.503 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.503 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.503 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:50.504 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945009+0000", 2024-09-21T21:42:50.504 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7450000000000001e-06 2024-09-21T21:42:50.504 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.504 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.504 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:50.504 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945013+0000", 2024-09-21T21:42:50.504 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.673e-06 2024-09-21T21:42:50.504 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.504 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.505 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:50.505 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024245+0000", 2024-09-21T21:42:50.505 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079232160999999995 2024-09-21T21:42:50.505 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.505 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.505 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:50.505 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024258+0000", 2024-09-21T21:42:50.505 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.2224e-05 2024-09-21T21:42:50.505 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:50.506 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:50.506 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:50.506 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.506 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.506 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13639 3.12 3.1dc31b72 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:50.506 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.945034+0000", 2024-09-21T21:42:50.506 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 59.373537249000002, 2024-09-21T21:42:50.506 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079260627, 2024-09-21T21:42:50.507 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:50.507 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:50.507 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:42:50.507 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:50.507 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:50.507 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:50.507 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13639 2024-09-21T21:42:50.507 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.507 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:50.507 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.507 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:50.508 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945034+0000", 2024-09-21T21:42:50.508 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:50.508 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.508 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.508 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:50.508 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945034+0000", 2024-09-21T21:42:50.508 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:50.508 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.508 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.508 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:50.508 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945034+0000", 2024-09-21T21:42:50.508 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3799999999999998e-07 2024-09-21T21:42:50.508 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.508 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.509 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:50.509 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945036+0000", 2024-09-21T21:42:50.509 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.052e-06 2024-09-21T21:42:50.509 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.509 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.509 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:50.509 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945040+0000", 2024-09-21T21:42:50.509 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1169999999999997e-06 2024-09-21T21:42:50.509 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.509 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.509 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:50.509 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945043+0000", 2024-09-21T21:42:50.509 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.8439999999999998e-06 2024-09-21T21:42:50.509 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.510 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.510 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:50.510 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024286+0000", 2024-09-21T21:42:50.510 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079242483000000002 2024-09-21T21:42:50.510 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.510 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.510 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:50.510 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024295+0000", 2024-09-21T21:42:50.510 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.7930000000000003e-06 2024-09-21T21:42:50.510 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:50.510 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:50.510 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:50.510 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.511 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.511 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13640 3.12 3.1dc31b72 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:50.511 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.945047+0000", 2024-09-21T21:42:50.511 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 59.373523726999998, 2024-09-21T21:42:50.511 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079412321999999994, 2024-09-21T21:42:50.511 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:50.511 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:50.511 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:42:50.511 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:50.511 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:50.511 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:50.511 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13640 2024-09-21T21:42:50.512 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.512 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:50.512 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.512 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:50.512 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945047+0000", 2024-09-21T21:42:50.512 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:50.512 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.512 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.512 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:50.512 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945047+0000", 2024-09-21T21:42:50.512 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:50.512 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.512 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.512 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:50.513 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945048+0000", 2024-09-21T21:42:50.513 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0499999999999999e-07 2024-09-21T21:42:50.513 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.513 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.513 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:50.513 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945049+0000", 2024-09-21T21:42:50.513 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.3939999999999999e-06 2024-09-21T21:42:50.513 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.513 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.513 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:50.513 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945053+0000", 2024-09-21T21:42:50.513 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1760000000000003e-06 2024-09-21T21:42:50.513 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.513 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.513 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:50.514 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945056+0000", 2024-09-21T21:42:50.514 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.6129999999999998e-06 2024-09-21T21:42:50.514 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.514 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.514 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:50.514 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024360+0000", 2024-09-21T21:42:50.514 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079303645000000006 2024-09-21T21:42:50.514 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.514 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.514 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:50.514 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024460+0000", 2024-09-21T21:42:50.514 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000100189 2024-09-21T21:42:50.514 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:50.514 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:50.515 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:50.515 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:50.515 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:50.515 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:50.516 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T21:42:50.517 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:50.514+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:50.617 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:50.616+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:50.660 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:50.660 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:42:50.660 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 0 2024-09-21T21:42:50.660 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:50.670 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T21:42:50.753 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:50.752+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:50.813 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:50.813 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:42:50.813 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:42:50.813 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:42:50.813 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:50.818 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:50.818+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:50.822 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T21:42:50.919 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:50.918+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:50.967 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:50.967 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 20, 2024-09-21T21:42:50.967 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 600, 2024-09-21T21:42:50.967 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:42:50.968 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.968 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4824.0:1564 3.2 3:4647520c:::rbd_data.1121b97f2925.0000000000000006:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:42:50.968 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:42:50.968 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 109.32895035200001, 2024-09-21T21:42:50.968 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056097347999999998, 2024-09-21T21:42:50.968 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:50.968 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:50.968 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:50.968 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:50.968 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4824", 2024-09-21T21:42:50.968 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:42:50.968 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1564 2024-09-21T21:42:50.968 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.968 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:50.969 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.969 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:50.969 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:42:50.969 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:50.969 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.969 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.969 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:50.969 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:42:50.969 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:50.969 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.969 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.969 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:50.969 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:42:50.970 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6499999999999999e-07 2024-09-21T21:42:50.970 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.970 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.970 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:50.970 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636067+0000", 2024-09-21T21:42:50.970 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3879999999999998e-06 2024-09-21T21:42:50.970 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.970 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.970 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:50.970 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636075+0000", 2024-09-21T21:42:50.970 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5480000000000004e-06 2024-09-21T21:42:50.970 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.970 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.970 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:50.970 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636080+0000", 2024-09-21T21:42:50.971 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.339e-06 2024-09-21T21:42:50.971 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.971 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.971 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:50.971 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636133+0000", 2024-09-21T21:42:50.971 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2930000000000003e-05 2024-09-21T21:42:50.971 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.971 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.971 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:50.971 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636265+0000", 2024-09-21T21:42:50.971 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000132058 2024-09-21T21:42:50.971 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.971 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.971 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:50.971 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.692162+0000", 2024-09-21T21:42:50.972 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055896620000000001 2024-09-21T21:42:50.972 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:50.972 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:50.972 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:50.972 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.972 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.972 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3260 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 1437696~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:42:50.972 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.196297+0000", 2024-09-21T21:42:50.972 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 63.768718141999997, 2024-09-21T21:42:50.972 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056071022999999998, 2024-09-21T21:42:50.972 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:50.972 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:50.972 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:50.972 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:50.972 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:50.973 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:50.973 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3260 2024-09-21T21:42:50.973 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.973 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:50.973 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.973 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:50.973 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196297+0000", 2024-09-21T21:42:50.973 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:50.973 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.973 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.973 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:50.973 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196297+0000", 2024-09-21T21:42:50.973 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:50.973 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.973 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.974 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:50.974 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196918+0000", 2024-09-21T21:42:50.974 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00062148699999999999 2024-09-21T21:42:50.974 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.974 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.974 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:50.974 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196930+0000", 2024-09-21T21:42:50.974 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1657e-05 2024-09-21T21:42:50.974 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.974 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.974 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:50.974 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196944+0000", 2024-09-21T21:42:50.974 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.4549000000000001e-05 2024-09-21T21:42:50.974 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.974 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.975 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:50.975 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196951+0000", 2024-09-21T21:42:50.975 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6749999999999996e-06 2024-09-21T21:42:50.975 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.975 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.975 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:50.975 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252208+0000", 2024-09-21T21:42:50.975 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055256656000000001 2024-09-21T21:42:50.975 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.975 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.975 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:50.975 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252278+0000", 2024-09-21T21:42:50.975 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9882000000000006e-05 2024-09-21T21:42:50.975 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.975 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.976 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:50.976 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252368+0000", 2024-09-21T21:42:50.976 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.0117000000000005e-05 2024-09-21T21:42:50.976 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:50.976 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:50.976 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:50.976 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.976 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.976 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3263 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 3559424~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:42:50.976 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.197041+0000", 2024-09-21T21:42:50.976 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 63.767974328000001, 2024-09-21T21:42:50.976 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055935804999999998, 2024-09-21T21:42:50.976 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:50.976 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:50.976 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:50.977 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:50.977 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:50.977 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:50.977 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3263 2024-09-21T21:42:50.977 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.977 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:50.977 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.977 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:50.977 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197041+0000", 2024-09-21T21:42:50.977 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:50.977 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.977 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.977 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:50.977 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197041+0000", 2024-09-21T21:42:50.977 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:50.978 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.978 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.978 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:50.978 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197624+0000", 2024-09-21T21:42:50.978 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00058358100000000001 2024-09-21T21:42:50.978 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.978 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.978 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:50.978 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197629+0000", 2024-09-21T21:42:50.978 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5909999999999998e-06 2024-09-21T21:42:50.978 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.978 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.978 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:50.978 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197639+0000", 2024-09-21T21:42:50.978 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0257e-05 2024-09-21T21:42:50.979 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.979 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.979 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:50.979 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197646+0000", 2024-09-21T21:42:50.979 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.7229999999999998e-06 2024-09-21T21:42:50.979 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.979 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.979 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:50.979 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252804+0000", 2024-09-21T21:42:50.979 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055158134999999997 2024-09-21T21:42:50.979 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.979 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.979 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:50.979 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252882+0000", 2024-09-21T21:42:50.979 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7791000000000001e-05 2024-09-21T21:42:50.979 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.980 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.980 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:50.980 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252976+0000", 2024-09-21T21:42:50.980 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.4727000000000003e-05 2024-09-21T21:42:50.980 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:50.980 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:50.980 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:50.980 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.980 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.980 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3437 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 925696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:42:50.980 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.205888+0000", 2024-09-21T21:42:50.980 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 63.759127047, 2024-09-21T21:42:50.980 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055953808000000001, 2024-09-21T21:42:50.980 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:50.980 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:50.981 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:50.981 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:50.981 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:50.981 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:50.981 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3437 2024-09-21T21:42:50.981 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.981 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:50.981 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.981 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:50.981 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205888+0000", 2024-09-21T21:42:50.981 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:50.981 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.981 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.981 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:50.981 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205888+0000", 2024-09-21T21:42:50.981 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:50.982 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.982 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.982 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:50.982 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205888+0000", 2024-09-21T21:42:50.982 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.75e-07 2024-09-21T21:42:50.982 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.982 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.982 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:50.982 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205891+0000", 2024-09-21T21:42:50.982 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3750000000000001e-06 2024-09-21T21:42:50.982 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.982 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.982 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:50.982 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205897+0000", 2024-09-21T21:42:50.982 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.285e-06 2024-09-21T21:42:50.983 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.983 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.983 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:50.983 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205902+0000", 2024-09-21T21:42:50.983 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6290000000000001e-06 2024-09-21T21:42:50.983 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.983 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.983 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:50.983 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261686+0000", 2024-09-21T21:42:50.983 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055784586999999997 2024-09-21T21:42:50.983 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.983 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.983 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:50.983 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261764+0000", 2024-09-21T21:42:50.983 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7591999999999998e-05 2024-09-21T21:42:50.983 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.984 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.984 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:50.984 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261842+0000", 2024-09-21T21:42:50.984 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7865000000000001e-05 2024-09-21T21:42:50.984 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:50.984 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:50.984 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:50.984 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.984 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.984 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3438 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 954368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:42:50.984 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.205908+0000", 2024-09-21T21:42:50.984 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 63.759107182000001, 2024-09-21T21:42:50.984 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056083169000000002, 2024-09-21T21:42:50.984 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:50.984 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:50.984 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:50.985 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:50.985 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:50.985 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:50.985 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3438 2024-09-21T21:42:50.985 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.985 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:50.985 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.985 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:50.985 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205908+0000", 2024-09-21T21:42:50.985 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:50.985 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.985 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.985 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:50.985 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205908+0000", 2024-09-21T21:42:50.985 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:50.985 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.986 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.986 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:50.986 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206431+0000", 2024-09-21T21:42:50.986 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00052282100000000003 2024-09-21T21:42:50.986 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.986 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.986 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:50.986 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206442+0000", 2024-09-21T21:42:50.986 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1695000000000001e-05 2024-09-21T21:42:50.986 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.986 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.986 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:50.987 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206452+0000", 2024-09-21T21:42:50.987 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.4560000000000006e-06 2024-09-21T21:42:50.987 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.987 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.987 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:50.987 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206458+0000", 2024-09-21T21:42:50.987 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4670000000000001e-06 2024-09-21T21:42:50.987 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.987 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.987 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:50.987 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261871+0000", 2024-09-21T21:42:50.987 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055412712000000003 2024-09-21T21:42:50.987 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.987 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.988 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:50.988 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261928+0000", 2024-09-21T21:42:50.988 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6631e-05 2024-09-21T21:42:50.988 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.988 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.988 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:50.988 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261991+0000", 2024-09-21T21:42:50.988 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3386999999999995e-05 2024-09-21T21:42:50.988 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:50.988 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:50.988 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:50.988 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.988 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.989 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3444 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 2359296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:42:50.989 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.206533+0000", 2024-09-21T21:42:50.989 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 63.758481662000001, 2024-09-21T21:42:50.989 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055974239000000002, 2024-09-21T21:42:50.989 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:50.989 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:50.989 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:50.989 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:50.989 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:50.989 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:50.989 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3444 2024-09-21T21:42:50.989 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.989 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:50.989 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.989 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:50.990 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206533+0000", 2024-09-21T21:42:50.990 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:50.990 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.990 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.990 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:50.990 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206533+0000", 2024-09-21T21:42:50.990 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:50.990 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.990 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.990 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:50.990 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206710+0000", 2024-09-21T21:42:50.990 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000176293 2024-09-21T21:42:50.990 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.990 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.991 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:50.991 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206714+0000", 2024-09-21T21:42:50.991 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1989999999999999e-06 2024-09-21T21:42:50.991 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.991 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.991 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:50.991 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206723+0000", 2024-09-21T21:42:50.991 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.9199999999999993e-06 2024-09-21T21:42:50.991 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.991 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.991 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:50.991 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206729+0000", 2024-09-21T21:42:50.992 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0739999999999998e-06 2024-09-21T21:42:50.992 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.992 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.992 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:50.992 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262339+0000", 2024-09-21T21:42:50.992 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055610166000000003 2024-09-21T21:42:50.992 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.992 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.992 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:50.992 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262419+0000", 2024-09-21T21:42:50.992 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0432999999999994e-05 2024-09-21T21:42:50.992 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.992 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.993 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:50.993 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262507+0000", 2024-09-21T21:42:50.993 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.8153999999999997e-05 2024-09-21T21:42:50.993 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:50.993 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:50.993 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:50.993 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.993 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.993 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10362 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:50.993 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840386+0000", 2024-09-21T21:42:50.993 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 37.124628817999998, 2024-09-21T21:42:50.993 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056546515999999998, 2024-09-21T21:42:50.993 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:50.994 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:50.994 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:50.994 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:50.994 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:50.994 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:50.994 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10362 2024-09-21T21:42:50.994 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.994 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:50.994 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.994 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:50.994 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840386+0000", 2024-09-21T21:42:50.994 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:50.994 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.994 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.994 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:50.995 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840386+0000", 2024-09-21T21:42:50.995 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:50.995 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.995 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.995 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:50.995 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840386+0000", 2024-09-21T21:42:50.995 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.01e-07 2024-09-21T21:42:50.995 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.995 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.995 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:50.995 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840389+0000", 2024-09-21T21:42:50.995 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.9739999999999998e-06 2024-09-21T21:42:50.995 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.995 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.995 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:50.996 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840396+0000", 2024-09-21T21:42:50.996 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2280000000000003e-06 2024-09-21T21:42:50.996 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.996 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.996 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:50.996 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840400+0000", 2024-09-21T21:42:50.996 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4429999999999996e-06 2024-09-21T21:42:50.996 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.996 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.996 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:50.996 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.890896+0000", 2024-09-21T21:42:50.996 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.050495717000000002 2024-09-21T21:42:50.996 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.996 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.997 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:50.997 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.891012+0000", 2024-09-21T21:42:50.997 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011602 2024-09-21T21:42:50.997 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.997 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.997 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:50.997 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.891274+0000", 2024-09-21T21:42:50.997 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00026240200000000001 2024-09-21T21:42:50.997 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.997 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.997 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:50.997 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.894810+0000", 2024-09-21T21:42:50.997 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0035353630000000001 2024-09-21T21:42:50.997 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.997 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.998 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:50.998 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.896818+0000", 2024-09-21T21:42:50.998 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.002008681 2024-09-21T21:42:50.998 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.998 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.998 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:50.998 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.896850+0000", 2024-09-21T21:42:50.998 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.1826999999999999e-05 2024-09-21T21:42:50.998 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.998 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.998 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:50.998 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.896933+0000", 2024-09-21T21:42:50.998 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.2460000000000007e-05 2024-09-21T21:42:50.998 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:50.998 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:50.999 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:50.999 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:50.999 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:50.999 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10363 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:50.999 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840606+0000", 2024-09-21T21:42:50.999 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 37.124409307999997, 2024-09-21T21:42:50.999 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056546011, 2024-09-21T21:42:50.999 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:50.999 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:50.999 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:50.999 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:50.999 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:50.999 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:51.000 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10363 2024-09-21T21:42:51.000 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.000 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:51.000 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.000 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:51.000 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840606+0000", 2024-09-21T21:42:51.000 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:51.000 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.000 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.000 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:51.000 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840606+0000", 2024-09-21T21:42:51.000 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:51.000 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.000 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.001 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:51.001 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840606+0000", 2024-09-21T21:42:51.001 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.4700000000000001e-07 2024-09-21T21:42:51.001 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.001 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.001 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:51.001 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840611+0000", 2024-09-21T21:42:51.001 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4649999999999996e-06 2024-09-21T21:42:51.001 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.001 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.001 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:51.001 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840620+0000", 2024-09-21T21:42:51.001 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.0350000000000007e-06 2024-09-21T21:42:51.002 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.002 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.002 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:51.002 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840626+0000", 2024-09-21T21:42:51.002 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7180000000000001e-06 2024-09-21T21:42:51.002 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.002 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.002 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:51.002 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.891665+0000", 2024-09-21T21:42:51.002 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.051039924 2024-09-21T21:42:51.002 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.002 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.002 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:51.002 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.891770+0000", 2024-09-21T21:42:51.003 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010445700000000001 2024-09-21T21:42:51.003 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.003 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.003 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:51.003 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.892147+0000", 2024-09-21T21:42:51.003 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00037731800000000002 2024-09-21T21:42:51.003 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.003 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.003 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:51.003 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.896776+0000", 2024-09-21T21:42:51.003 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0046289759999999999 2024-09-21T21:42:51.003 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.003 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.003 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:51.004 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897003+0000", 2024-09-21T21:42:51.004 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00022641299999999999 2024-09-21T21:42:51.004 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.004 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.004 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:51.004 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897064+0000", 2024-09-21T21:42:51.004 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1571999999999999e-05 2024-09-21T21:42:51.004 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.004 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.004 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:51.004 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897152+0000", 2024-09-21T21:42:51.004 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.7386000000000007e-05 2024-09-21T21:42:51.004 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:51.004 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:51.005 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:51.005 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.005 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.005 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10364 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:51.005 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840641+0000", 2024-09-21T21:42:51.005 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 37.124373488000003, 2024-09-21T21:42:51.005 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056671527999999999, 2024-09-21T21:42:51.005 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:51.005 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:51.005 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:51.005 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:51.005 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:51.005 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:51.005 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10364 2024-09-21T21:42:51.006 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.006 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:51.006 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.006 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:51.006 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840641+0000", 2024-09-21T21:42:51.006 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:51.006 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.006 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.006 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:51.006 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840641+0000", 2024-09-21T21:42:51.006 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:51.006 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.006 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.006 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:51.007 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840642+0000", 2024-09-21T21:42:51.007 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7899999999999999e-07 2024-09-21T21:42:51.007 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.007 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.007 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:51.007 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840645+0000", 2024-09-21T21:42:51.007 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0010000000000002e-06 2024-09-21T21:42:51.007 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.007 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.007 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:51.007 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840654+0000", 2024-09-21T21:42:51.007 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.8570000000000001e-06 2024-09-21T21:42:51.007 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.007 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.007 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:51.008 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840659+0000", 2024-09-21T21:42:51.008 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.558e-06 2024-09-21T21:42:51.008 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.008 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.008 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:51.008 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.892656+0000", 2024-09-21T21:42:51.008 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.051996586999999997 2024-09-21T21:42:51.008 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.008 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.008 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:51.008 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.892721+0000", 2024-09-21T21:42:51.008 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5010000000000003e-05 2024-09-21T21:42:51.008 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.009 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.009 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:51.009 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.893112+0000", 2024-09-21T21:42:51.009 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00039133899999999999 2024-09-21T21:42:51.009 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.009 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.009 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:51.009 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.896784+0000", 2024-09-21T21:42:51.009 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0036713679999999999 2024-09-21T21:42:51.009 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.009 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.009 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:51.009 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897204+0000", 2024-09-21T21:42:51.009 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00041991400000000002 2024-09-21T21:42:51.010 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.010 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.010 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:51.010 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897234+0000", 2024-09-21T21:42:51.010 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0307999999999998e-05 2024-09-21T21:42:51.010 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.010 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.010 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:51.010 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897313+0000", 2024-09-21T21:42:51.010 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.9107e-05 2024-09-21T21:42:51.010 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:51.010 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:51.010 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:51.010 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.011 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.011 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10365 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:51.011 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840668+0000", 2024-09-21T21:42:51.011 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 37.124346623000001, 2024-09-21T21:42:51.011 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.058761225, 2024-09-21T21:42:51.011 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:51.011 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:51.011 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:51.011 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:51.011 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:51.011 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:51.011 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10365 2024-09-21T21:42:51.011 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.011 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:51.012 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.012 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:51.012 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840668+0000", 2024-09-21T21:42:51.012 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:51.012 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.012 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.012 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:51.012 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840668+0000", 2024-09-21T21:42:51.012 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:51.012 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.012 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.012 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:51.012 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840669+0000", 2024-09-21T21:42:51.012 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6300000000000001e-07 2024-09-21T21:42:51.013 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.013 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.013 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:51.013 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840672+0000", 2024-09-21T21:42:51.013 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.072e-06 2024-09-21T21:42:51.013 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.013 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.013 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:51.013 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840678+0000", 2024-09-21T21:42:51.013 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.7039999999999997e-06 2024-09-21T21:42:51.013 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.013 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.013 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:51.013 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840683+0000", 2024-09-21T21:42:51.014 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6759999999999998e-06 2024-09-21T21:42:51.014 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.014 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.014 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:51.014 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.893929+0000", 2024-09-21T21:42:51.014 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.053246136999999999 2024-09-21T21:42:51.014 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.014 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.014 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:51.014 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.894082+0000", 2024-09-21T21:42:51.014 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00015301799999999999 2024-09-21T21:42:51.014 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.014 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.014 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:51.015 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.894357+0000", 2024-09-21T21:42:51.015 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00027507999999999998 2024-09-21T21:42:51.015 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.015 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.015 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:51.015 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897161+0000", 2024-09-21T21:42:51.015 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0028036810000000001 2024-09-21T21:42:51.015 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.015 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.015 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:51.015 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.899243+0000", 2024-09-21T21:42:51.015 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0020816379999999998 2024-09-21T21:42:51.015 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.015 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.016 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:51.016 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.899306+0000", 2024-09-21T21:42:51.016 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2964000000000002e-05 2024-09-21T21:42:51.016 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.016 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.016 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:51.016 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.899430+0000", 2024-09-21T21:42:51.016 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00012389200000000001 2024-09-21T21:42:51.016 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:51.016 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:51.016 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:51.016 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.017 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.017 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10366 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:51.017 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840691+0000", 2024-09-21T21:42:51.017 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 37.124323922999999, 2024-09-21T21:42:51.017 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.061540984999999999, 2024-09-21T21:42:51.017 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:51.017 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:51.017 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:51.017 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:51.017 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:51.017 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:51.017 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10366 2024-09-21T21:42:51.017 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.018 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:51.018 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.018 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:51.018 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840691+0000", 2024-09-21T21:42:51.018 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:51.018 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.018 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.018 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:51.018 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840691+0000", 2024-09-21T21:42:51.018 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:51.018 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.018 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.019 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:51.019 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840691+0000", 2024-09-21T21:42:51.019 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3099999999999999e-07 2024-09-21T21:42:51.019 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.019 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.019 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:51.019 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840694+0000", 2024-09-21T21:42:51.019 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.5780000000000001e-06 2024-09-21T21:42:51.019 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.019 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.019 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:51.019 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840700+0000", 2024-09-21T21:42:51.019 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9429999999999997e-06 2024-09-21T21:42:51.019 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.019 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.020 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:51.020 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840704+0000", 2024-09-21T21:42:51.020 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6380000000000003e-06 2024-09-21T21:42:51.020 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.020 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.020 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:51.020 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.894846+0000", 2024-09-21T21:42:51.020 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.054141392000000003 2024-09-21T21:42:51.020 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.020 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.020 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:51.020 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.894932+0000", 2024-09-21T21:42:51.020 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.5834000000000005e-05 2024-09-21T21:42:51.020 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.020 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.021 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:51.021 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.895302+0000", 2024-09-21T21:42:51.021 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000370737 2024-09-21T21:42:51.021 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.021 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.021 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:51.021 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897166+0000", 2024-09-21T21:42:51.021 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0018632620000000001 2024-09-21T21:42:51.021 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.021 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.021 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:51.021 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902053+0000", 2024-09-21T21:42:51.021 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0048868289999999997 2024-09-21T21:42:51.021 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.022 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.022 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:51.022 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902111+0000", 2024-09-21T21:42:51.022 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7983999999999999e-05 2024-09-21T21:42:51.022 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.022 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.022 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:51.022 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902232+0000", 2024-09-21T21:42:51.022 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000121457 2024-09-21T21:42:51.022 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:51.022 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:51.022 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:51.022 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.023 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.023 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10367 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:51.023 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840842+0000", 2024-09-21T21:42:51.023 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 37.124173114999998, 2024-09-21T21:42:51.023 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.061549088000000002, 2024-09-21T21:42:51.023 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:51.023 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:51.023 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:51.023 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:51.023 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:51.023 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:51.023 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10367 2024-09-21T21:42:51.023 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.023 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:51.023 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.024 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:51.024 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840842+0000", 2024-09-21T21:42:51.024 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:51.024 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.024 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.024 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:51.024 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840842+0000", 2024-09-21T21:42:51.024 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:51.024 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.024 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.024 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:51.024 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840842+0000", 2024-09-21T21:42:51.024 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3099999999999997e-07 2024-09-21T21:42:51.024 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.025 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.025 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:51.025 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840846+0000", 2024-09-21T21:42:51.025 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.2890000000000002e-06 2024-09-21T21:42:51.025 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.025 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.025 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:51.025 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840855+0000", 2024-09-21T21:42:51.025 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.9619999999999999e-06 2024-09-21T21:42:51.025 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.025 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.025 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:51.025 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840860+0000", 2024-09-21T21:42:51.025 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3759999999999999e-06 2024-09-21T21:42:51.025 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.025 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.026 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:51.026 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.895799+0000", 2024-09-21T21:42:51.026 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.054938569999999999 2024-09-21T21:42:51.026 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.026 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.026 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:51.026 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.895906+0000", 2024-09-21T21:42:51.026 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000107593 2024-09-21T21:42:51.026 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.026 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.026 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:51.026 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.896356+0000", 2024-09-21T21:42:51.026 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00044941700000000002 2024-09-21T21:42:51.026 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.027 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.027 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:51.027 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.898131+0000", 2024-09-21T21:42:51.027 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.001775064 2024-09-21T21:42:51.027 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.027 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.027 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:51.027 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902283+0000", 2024-09-21T21:42:51.027 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0041524839999999997 2024-09-21T21:42:51.027 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.027 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.027 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:51.027 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902316+0000", 2024-09-21T21:42:51.028 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3002e-05 2024-09-21T21:42:51.028 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.028 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.028 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:51.028 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902391+0000", 2024-09-21T21:42:51.028 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.47e-05 2024-09-21T21:42:51.028 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:51.028 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:51.028 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:51.028 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.028 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.028 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10368 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:51.029 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840873+0000", 2024-09-21T21:42:51.029 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 37.124142161000002, 2024-09-21T21:42:51.029 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.063395291000000006, 2024-09-21T21:42:51.029 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:51.029 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:51.029 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:51.029 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:51.029 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:51.029 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:51.029 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10368 2024-09-21T21:42:51.029 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.029 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:51.029 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.029 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:51.029 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840873+0000", 2024-09-21T21:42:51.030 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:51.030 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.030 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.030 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:51.030 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840873+0000", 2024-09-21T21:42:51.030 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:51.030 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.030 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.030 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:51.030 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840873+0000", 2024-09-21T21:42:51.030 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6699999999999999e-07 2024-09-21T21:42:51.030 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.030 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.031 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:51.031 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840876+0000", 2024-09-21T21:42:51.031 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.6410000000000002e-06 2024-09-21T21:42:51.031 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.031 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.031 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:51.031 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840883+0000", 2024-09-21T21:42:51.031 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0389999999999999e-06 2024-09-21T21:42:51.031 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.031 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.031 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:51.031 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840888+0000", 2024-09-21T21:42:51.031 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8679999999999998e-06 2024-09-21T21:42:51.031 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.031 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.032 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:51.032 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897345+0000", 2024-09-21T21:42:51.032 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056457734000000002 2024-09-21T21:42:51.032 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.032 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.032 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:51.032 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897407+0000", 2024-09-21T21:42:51.032 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1822000000000005e-05 2024-09-21T21:42:51.032 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.032 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.032 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:51.032 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897664+0000", 2024-09-21T21:42:51.033 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00025638499999999998 2024-09-21T21:42:51.033 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.033 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.033 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:51.033 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.901136+0000", 2024-09-21T21:42:51.033 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0034728070000000001 2024-09-21T21:42:51.033 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.033 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.033 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:51.033 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904090+0000", 2024-09-21T21:42:51.033 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0029535799999999999 2024-09-21T21:42:51.033 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.033 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.034 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:51.034 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904144+0000", 2024-09-21T21:42:51.034 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4369000000000001e-05 2024-09-21T21:42:51.034 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.034 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.034 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:51.034 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904268+0000", 2024-09-21T21:42:51.034 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00012367899999999999 2024-09-21T21:42:51.034 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:51.034 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:51.034 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:51.034 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.034 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.034 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10369 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:51.035 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840902+0000", 2024-09-21T21:42:51.035 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 37.124112508000003, 2024-09-21T21:42:51.035 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.063538938000000003, 2024-09-21T21:42:51.035 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:51.035 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:51.035 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:51.035 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:51.035 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:51.035 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:51.035 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10369 2024-09-21T21:42:51.035 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.035 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:51.035 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.035 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:51.036 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840902+0000", 2024-09-21T21:42:51.036 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:51.036 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.036 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.036 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:51.036 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840902+0000", 2024-09-21T21:42:51.036 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:51.036 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.036 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.036 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:51.036 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840903+0000", 2024-09-21T21:42:51.036 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2100000000000002e-07 2024-09-21T21:42:51.036 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.036 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.036 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:51.036 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840913+0000", 2024-09-21T21:42:51.037 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.048e-05 2024-09-21T21:42:51.037 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.037 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.037 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:51.037 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840920+0000", 2024-09-21T21:42:51.037 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0330000000000003e-06 2024-09-21T21:42:51.037 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.037 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.037 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:51.037 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840925+0000", 2024-09-21T21:42:51.037 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0170000000000004e-06 2024-09-21T21:42:51.037 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.037 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.037 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:51.037 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.898179+0000", 2024-09-21T21:42:51.037 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.057253671999999999 2024-09-21T21:42:51.038 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.038 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.038 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:51.038 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.898284+0000", 2024-09-21T21:42:51.038 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000105202 2024-09-21T21:42:51.038 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.038 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.038 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:51.038 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.898729+0000", 2024-09-21T21:42:51.038 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00044459300000000002 2024-09-21T21:42:51.038 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.038 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.038 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:51.038 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902246+0000", 2024-09-21T21:42:51.039 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0035171629999999998 2024-09-21T21:42:51.039 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.039 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.039 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:51.039 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904310+0000", 2024-09-21T21:42:51.039 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.002063765 2024-09-21T21:42:51.039 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.039 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.039 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:51.039 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904342+0000", 2024-09-21T21:42:51.039 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.2091999999999997e-05 2024-09-21T21:42:51.039 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.039 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.039 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:51.039 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904441+0000", 2024-09-21T21:42:51.040 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.9500000000000006e-05 2024-09-21T21:42:51.040 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:51.040 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:51.040 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:51.040 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.040 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.040 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10370 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:51.040 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840944+0000", 2024-09-21T21:42:51.040 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 37.124070795000002, 2024-09-21T21:42:51.040 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.064436853000000002, 2024-09-21T21:42:51.040 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:51.040 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:51.040 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:51.040 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:51.040 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:51.041 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:51.041 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10370 2024-09-21T21:42:51.041 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.041 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:51.041 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.041 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:51.041 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840944+0000", 2024-09-21T21:42:51.041 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:51.041 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.041 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.041 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:51.041 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840944+0000", 2024-09-21T21:42:51.041 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:51.041 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.041 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.042 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:51.042 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840945+0000", 2024-09-21T21:42:51.042 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2300000000000002e-07 2024-09-21T21:42:51.042 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.042 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.042 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:51.042 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840947+0000", 2024-09-21T21:42:51.042 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.8200000000000001e-06 2024-09-21T21:42:51.042 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.042 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.042 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:51.042 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840955+0000", 2024-09-21T21:42:51.042 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.8660000000000006e-06 2024-09-21T21:42:51.042 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.043 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.043 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:51.043 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840960+0000", 2024-09-21T21:42:51.043 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8670000000000002e-06 2024-09-21T21:42:51.043 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.043 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.043 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:51.043 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.899460+0000", 2024-09-21T21:42:51.043 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.058499922000000003 2024-09-21T21:42:51.043 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.043 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.043 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:51.043 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.899530+0000", 2024-09-21T21:42:51.043 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9558e-05 2024-09-21T21:42:51.043 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.043 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.044 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:51.044 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.899775+0000", 2024-09-21T21:42:51.044 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000245231 2024-09-21T21:42:51.044 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.044 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.044 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:51.044 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902254+0000", 2024-09-21T21:42:51.044 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0024790519999999998 2024-09-21T21:42:51.044 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.044 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.044 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:51.044 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905183+0000", 2024-09-21T21:42:51.044 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.002929156 2024-09-21T21:42:51.044 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.045 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.045 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:51.045 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905243+0000", 2024-09-21T21:42:51.045 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0055e-05 2024-09-21T21:42:51.045 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.045 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.045 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:51.045 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905381+0000", 2024-09-21T21:42:51.045 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00013790300000000001 2024-09-21T21:42:51.045 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:51.045 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:51.045 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:51.045 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.045 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.046 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10371 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:51.046 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840980+0000", 2024-09-21T21:42:51.046 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 37.124035128999999, 2024-09-21T21:42:51.046 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.064554376999999996, 2024-09-21T21:42:51.046 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:51.046 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:51.046 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:51.046 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:51.046 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:51.046 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:51.046 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10371 2024-09-21T21:42:51.046 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.046 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:51.046 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.046 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:51.047 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840980+0000", 2024-09-21T21:42:51.047 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:51.047 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.047 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.047 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:51.047 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840980+0000", 2024-09-21T21:42:51.047 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:51.047 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.047 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.047 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:51.047 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840980+0000", 2024-09-21T21:42:51.047 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8999999999999997e-07 2024-09-21T21:42:51.047 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.047 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.047 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:51.048 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840983+0000", 2024-09-21T21:42:51.048 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.123e-06 2024-09-21T21:42:51.048 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.048 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.048 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:51.048 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840990+0000", 2024-09-21T21:42:51.048 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9129999999999997e-06 2024-09-21T21:42:51.048 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.048 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.048 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:51.048 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840995+0000", 2024-09-21T21:42:51.048 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0989999999999998e-06 2024-09-21T21:42:51.048 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.049 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.049 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:51.049 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.900168+0000", 2024-09-21T21:42:51.049 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.059172739000000002 2024-09-21T21:42:51.049 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.049 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.049 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:51.049 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.900274+0000", 2024-09-21T21:42:51.049 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010546499999999999 2024-09-21T21:42:51.049 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.049 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.049 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:51.049 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.900703+0000", 2024-09-21T21:42:51.049 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00042978200000000003 2024-09-21T21:42:51.049 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.050 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.050 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:51.050 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.903443+0000", 2024-09-21T21:42:51.050 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0027400749999999998 2024-09-21T21:42:51.050 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.050 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.050 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:51.050 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905430+0000", 2024-09-21T21:42:51.050 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0019868109999999998 2024-09-21T21:42:51.050 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.050 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.050 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:51.050 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905464+0000", 2024-09-21T21:42:51.050 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3599000000000001e-05 2024-09-21T21:42:51.051 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.051 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.051 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:51.051 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905534+0000", 2024-09-21T21:42:51.051 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0281e-05 2024-09-21T21:42:51.051 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:51.051 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:51.051 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:51.051 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.051 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.051 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10372 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:51.051 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.841004+0000", 2024-09-21T21:42:51.051 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 37.124011240000002, 2024-09-21T21:42:51.051 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.064757812999999997, 2024-09-21T21:42:51.052 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:51.052 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:51.052 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:51.052 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:51.052 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:51.052 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:51.052 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10372 2024-09-21T21:42:51.052 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.052 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:51.052 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.052 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:51.052 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841004+0000", 2024-09-21T21:42:51.052 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:51.052 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.052 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.053 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:51.053 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841004+0000", 2024-09-21T21:42:51.053 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:51.053 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.053 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.053 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:51.053 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841004+0000", 2024-09-21T21:42:51.053 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8500000000000002e-07 2024-09-21T21:42:51.053 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.053 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.053 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:51.053 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841007+0000", 2024-09-21T21:42:51.053 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.6390000000000001e-06 2024-09-21T21:42:51.053 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.054 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.054 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:51.054 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841013+0000", 2024-09-21T21:42:51.054 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0070000000000003e-06 2024-09-21T21:42:51.054 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.054 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.054 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:51.054 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841018+0000", 2024-09-21T21:42:51.054 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9119999999999997e-06 2024-09-21T21:42:51.054 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.054 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.054 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:51.054 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.901182+0000", 2024-09-21T21:42:51.054 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.060164444999999997 2024-09-21T21:42:51.054 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.055 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.055 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:51.055 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.901297+0000", 2024-09-21T21:42:51.055 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011468699999999999 2024-09-21T21:42:51.055 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.055 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.055 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:51.055 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.901559+0000", 2024-09-21T21:42:51.055 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000261764 2024-09-21T21:42:51.055 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.055 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.055 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:51.055 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904281+0000", 2024-09-21T21:42:51.056 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0027229329999999999 2024-09-21T21:42:51.056 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.056 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.056 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:51.056 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905588+0000", 2024-09-21T21:42:51.056 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.001306438 2024-09-21T21:42:51.056 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.056 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.056 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:51.056 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905644+0000", 2024-09-21T21:42:51.056 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5747e-05 2024-09-21T21:42:51.056 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.056 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.056 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:51.056 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905761+0000", 2024-09-21T21:42:51.057 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000117856 2024-09-21T21:42:51.057 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:51.057 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:51.057 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:51.057 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.057 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.057 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10373 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:51.057 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.841027+0000", 2024-09-21T21:42:51.057 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 37.123988156000003, 2024-09-21T21:42:51.057 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.066941839000000003, 2024-09-21T21:42:51.057 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:51.057 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:51.057 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:51.057 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:51.058 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:51.058 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:51.058 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10373 2024-09-21T21:42:51.058 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.058 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:51.058 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.058 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:51.058 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841027+0000", 2024-09-21T21:42:51.058 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:51.058 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.058 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.058 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:51.058 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841027+0000", 2024-09-21T21:42:51.058 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:51.059 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.059 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.059 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:51.059 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841027+0000", 2024-09-21T21:42:51.059 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6199999999999999e-07 2024-09-21T21:42:51.059 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.059 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.059 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:51.059 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841030+0000", 2024-09-21T21:42:51.059 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.729e-06 2024-09-21T21:42:51.059 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.059 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.059 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:51.059 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841036+0000", 2024-09-21T21:42:51.060 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1979999999999998e-06 2024-09-21T21:42:51.060 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.060 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.060 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:51.060 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841042+0000", 2024-09-21T21:42:51.060 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4289999999999998e-06 2024-09-21T21:42:51.060 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.060 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.060 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:51.060 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902433+0000", 2024-09-21T21:42:51.060 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.061390329 2024-09-21T21:42:51.060 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.060 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.060 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:51.061 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902541+0000", 2024-09-21T21:42:51.061 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000108349 2024-09-21T21:42:51.061 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.061 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.061 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:51.061 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902983+0000", 2024-09-21T21:42:51.061 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00044158000000000001 2024-09-21T21:42:51.061 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.061 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.061 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:51.061 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904490+0000", 2024-09-21T21:42:51.061 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0015074489999999999 2024-09-21T21:42:51.061 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.061 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.061 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:51.062 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.907671+0000", 2024-09-21T21:42:51.062 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.003180511 2024-09-21T21:42:51.062 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.062 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.062 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:51.062 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.907767+0000", 2024-09-21T21:42:51.062 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6379000000000007e-05 2024-09-21T21:42:51.062 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.062 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.062 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:51.062 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.907969+0000", 2024-09-21T21:42:51.063 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00020152399999999999 2024-09-21T21:42:51.063 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:51.063 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:51.063 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:51.063 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.063 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.063 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:12335 3.1a 3:5b387735:::rbd_data.1121faebdfe3.0000000000000005:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:42:51.063 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:14.411635+0000", 2024-09-21T21:42:51.063 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 36.553380277999999, 2024-09-21T21:42:51.063 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.005973491, 2024-09-21T21:42:51.063 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:51.063 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:51.063 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:51.063 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:51.064 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:51.064 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:51.064 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 12335 2024-09-21T21:42:51.064 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.064 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:51.064 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.064 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:51.064 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411635+0000", 2024-09-21T21:42:51.064 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:51.064 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.064 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.064 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:51.064 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411635+0000", 2024-09-21T21:42:51.064 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:51.064 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.065 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.065 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:51.065 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411635+0000", 2024-09-21T21:42:51.065 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8500000000000001e-07 2024-09-21T21:42:51.065 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.065 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.065 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:51.065 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411639+0000", 2024-09-21T21:42:51.065 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.574e-06 2024-09-21T21:42:51.065 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.065 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.065 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:51.065 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411645+0000", 2024-09-21T21:42:51.066 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6869999999999997e-06 2024-09-21T21:42:51.066 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.066 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.066 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:51.066 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411649+0000", 2024-09-21T21:42:51.066 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.771e-06 2024-09-21T21:42:51.066 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.066 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.066 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:51.066 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411695+0000", 2024-09-21T21:42:51.066 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6e-05 2024-09-21T21:42:51.066 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.066 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.066 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op must wait for map", 2024-09-21T21:42:51.066 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411702+0000", 2024-09-21T21:42:51.067 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5590000000000002e-06 2024-09-21T21:42:51.067 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.067 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.067 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:51.067 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.417222+0000", 2024-09-21T21:42:51.067 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0055198219999999997 2024-09-21T21:42:51.067 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.067 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.067 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for peered", 2024-09-21T21:42:51.067 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.417230+0000", 2024-09-21T21:42:51.067 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.123e-06 2024-09-21T21:42:51.067 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.067 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.067 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:51.067 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.417348+0000", 2024-09-21T21:42:51.067 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.000118029 2024-09-21T21:42:51.068 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.068 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.068 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:51.068 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.417451+0000", 2024-09-21T21:42:51.068 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000103674 2024-09-21T21:42:51.068 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.068 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.068 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:51.068 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.417608+0000", 2024-09-21T21:42:51.068 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00015666700000000001 2024-09-21T21:42:51.068 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:51.068 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:51.068 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:51.069 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.069 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.069 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:12349 3.1a 3:5b387735:::rbd_data.1121faebdfe3.0000000000000005:head [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:42:51.069 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:14.411833+0000", 2024-09-21T21:42:51.069 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 36.553181465000002, 2024-09-21T21:42:51.069 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0065064500000001, 2024-09-21T21:42:51.069 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:51.069 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:51.069 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:51.069 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:51.069 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:51.069 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:51.069 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 12349 2024-09-21T21:42:51.069 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.070 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:51.070 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.070 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:51.070 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411833+0000", 2024-09-21T21:42:51.070 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:51.070 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.070 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.070 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:51.070 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411833+0000", 2024-09-21T21:42:51.070 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:51.070 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.070 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.070 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:51.070 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411834+0000", 2024-09-21T21:42:51.070 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.03e-07 2024-09-21T21:42:51.071 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.071 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.071 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:51.071 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411836+0000", 2024-09-21T21:42:51.071 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.4700000000000001e-06 2024-09-21T21:42:51.071 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.071 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.071 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:51.071 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411843+0000", 2024-09-21T21:42:51.071 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4069999999999998e-06 2024-09-21T21:42:51.071 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.071 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.071 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:51.071 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411848+0000", 2024-09-21T21:42:51.071 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1580000000000004e-06 2024-09-21T21:42:51.071 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.072 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.072 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:51.072 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411901+0000", 2024-09-21T21:42:51.072 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2824000000000001e-05 2024-09-21T21:42:51.072 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.072 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.072 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting_for_map not empty", 2024-09-21T21:42:51.072 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411904+0000", 2024-09-21T21:42:51.072 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0639999999999998e-06 2024-09-21T21:42:51.072 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.072 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.072 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:51.072 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.417252+0000", 2024-09-21T21:42:51.072 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0053485060000000003 2024-09-21T21:42:51.073 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.073 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.073 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for peered", 2024-09-21T21:42:51.073 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.417259+0000", 2024-09-21T21:42:51.073 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.1149999999999996e-06 2024-09-21T21:42:51.073 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.073 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.073 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:51.073 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.418160+0000", 2024-09-21T21:42:51.073 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.000901083 2024-09-21T21:42:51.073 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.073 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.073 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:51.073 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.418236+0000", 2024-09-21T21:42:51.074 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5852000000000005e-05 2024-09-21T21:42:51.074 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:51.074 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:51.074 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:51.074 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.418340+0000", 2024-09-21T21:42:51.074 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000103568 2024-09-21T21:42:51.074 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:51.074 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:51.074 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:51.074 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:51.074 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:51.074 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:51.076 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T21:42:51.076 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:51.075+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:51.176 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:51.175+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:51.186 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T21:42:51.223 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:51.223 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:42:51.223 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 0 2024-09-21T21:42:51.224 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:51.226 INFO:teuthology.orchestra.run.smithi081.stderr:nodeep-scrub is set 2024-09-21T21:42:51.238 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T21:42:51.288 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:51.286+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:51.320 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0: {} 2024-09-21T21:42:51.321 INFO:teuthology.orchestra.run.smithi081.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T21:42:51.343 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1: {} 2024-09-21T21:42:51.343 INFO:teuthology.orchestra.run.smithi081.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T21:42:51.360 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2: {} 2024-09-21T21:42:51.361 INFO:teuthology.orchestra.run.smithi081.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T21:42:51.377 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:51.376+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:51.398 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:51.398 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:51.408 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T21:42:51.478 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:51.476+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:51.552 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:51.552 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:51.561 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T21:42:51.578 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3: {} 2024-09-21T21:42:51.578 INFO:teuthology.orchestra.run.smithi081.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T21:42:51.594 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4: {} 2024-09-21T21:42:51.594 INFO:teuthology.orchestra.run.smithi081.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T21:42:51.605 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:51.603+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:51.611 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5: {} 2024-09-21T21:42:51.611 INFO:teuthology.orchestra.run.smithi081.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T21:42:51.665 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:51.665 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:51.674 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T21:42:51.716 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:51.713+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:51.780 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:51.779+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:51.816 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:51.816 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:51.825 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T21:42:51.881 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:51.878+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:51.927 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:51.927 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:51.935 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T21:42:52.019 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:52.017+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:52.078 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:52.078 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:52.083 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:52.082+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:52.087 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T21:42:52.184 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:52.181+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:52.235 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:52.235 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:52.245 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T21:42:52.285 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:52.282+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:52.386 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:52.384+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:52.391 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:52.391 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:52.400 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T21:42:52.486 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:52.485+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:52.503 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:52.503 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:52.512 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T21:42:52.587 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:52.585+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:52.613 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:52.614 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:52.622 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T21:42:52.689 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:52.686+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:52.726 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:52.726 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:52.734 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T21:42:52.790 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:52.787+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:52.841 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:52.841 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:52.849 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T21:42:52.891 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:52.888+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:52.991 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:52.989+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:52.992 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:52.992 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:53.001 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T21:42:53.092 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:53.090+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:53.103 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:53.103 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:53.112 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T21:42:53.196 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:53.194+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:53.246 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T21:42:53.261 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:53.261 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:53.271 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T21:42:53.294 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:53.292+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:53.395 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:53.393+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:53.420 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:53.420 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:53.430 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T21:42:53.513 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:53.511+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:53.573 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:53.573 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:53.582 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T21:42:53.597 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:53.596+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:53.623 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T21:42:53.698 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:53.696+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:53.730 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:53.730 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:53.739 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T21:42:53.743 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0: {} 2024-09-21T21:42:53.743 INFO:teuthology.orchestra.run.smithi081.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T21:42:53.761 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1: {} 2024-09-21T21:42:53.761 INFO:teuthology.orchestra.run.smithi081.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T21:42:53.777 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2: {} 2024-09-21T21:42:53.777 INFO:teuthology.orchestra.run.smithi081.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T21:42:53.794 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3: {} 2024-09-21T21:42:53.794 INFO:teuthology.orchestra.run.smithi081.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T21:42:53.799 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:53.797+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:53.811 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4: {} 2024-09-21T21:42:53.811 INFO:teuthology.orchestra.run.smithi081.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T21:42:53.827 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5: {} 2024-09-21T21:42:53.828 INFO:teuthology.orchestra.run.smithi081.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T21:42:53.844 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:53.844 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:42:53.844 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:42:53.844 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:42:53.845 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:53.853 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T21:42:53.900 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:53.898+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:53.998 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:53.999 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 20, 2024-09-21T21:42:53.999 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 600, 2024-09-21T21:42:53.999 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:42:53.999 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:53.999 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13296 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 4046848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:53.999 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.796643+0000", 2024-09-21T21:42:53.999 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 63.199766457999999, 2024-09-21T21:42:53.999 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071682019999999999, 2024-09-21T21:42:53.999 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:53.999 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:53.999 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:53.999 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:53.999 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:54.000 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:54.000 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13296 2024-09-21T21:42:54.000 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.000 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:54.000 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.000 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:54.000 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796643+0000", 2024-09-21T21:42:54.000 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:54.000 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.000 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.000 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:54.000 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796643+0000", 2024-09-21T21:42:54.000 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:54.000 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.001 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.001 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:54.001 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796643+0000", 2024-09-21T21:42:54.001 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.7199999999999998e-07 2024-09-21T21:42:54.001 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.001 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.001 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:54.001 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796647+0000", 2024-09-21T21:42:54.001 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6770000000000001e-06 2024-09-21T21:42:54.001 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.001 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.001 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:54.001 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796656+0000", 2024-09-21T21:42:54.002 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2059999999999996e-06 2024-09-21T21:42:54.002 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.002 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.002 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:54.002 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796664+0000", 2024-09-21T21:42:54.002 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7919999999999993e-06 2024-09-21T21:42:54.002 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.002 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.002 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:54.002 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.868186+0000", 2024-09-21T21:42:54.002 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071522273999999997 2024-09-21T21:42:54.002 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.003 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.003 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:54.003 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.868239+0000", 2024-09-21T21:42:54.003 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2787999999999997e-05 2024-09-21T21:42:54.003 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.003 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.003 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:54.004 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.868325+0000", 2024-09-21T21:42:54.004 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.5611000000000004e-05 2024-09-21T21:42:54.004 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:54.004 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:54.004 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:54.004 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.004 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.004 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13561 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 1728512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:54.004 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.823991+0000", 2024-09-21T21:42:54.004 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 63.172418157000003, 2024-09-21T21:42:54.004 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071654762999999996, 2024-09-21T21:42:54.004 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:54.005 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:54.005 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:54.005 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:54.005 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:54.005 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:54.005 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13561 2024-09-21T21:42:54.005 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.005 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:54.005 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.005 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:54.005 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.823991+0000", 2024-09-21T21:42:54.005 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:54.006 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.006 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.006 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:54.006 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.823991+0000", 2024-09-21T21:42:54.006 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:54.006 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.006 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.006 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:54.006 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.823991+0000", 2024-09-21T21:42:54.006 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9299999999999998e-07 2024-09-21T21:42:54.006 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.006 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.007 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:54.007 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.823994+0000", 2024-09-21T21:42:54.007 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2369999999999999e-06 2024-09-21T21:42:54.007 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.007 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.007 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:54.007 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824000+0000", 2024-09-21T21:42:54.007 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.55e-06 2024-09-21T21:42:54.007 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.007 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.007 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:54.007 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824005+0000", 2024-09-21T21:42:54.007 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4329999999999997e-06 2024-09-21T21:42:54.008 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.008 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.008 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:54.008 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.895360+0000", 2024-09-21T21:42:54.008 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071355426 2024-09-21T21:42:54.008 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.008 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.008 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:54.008 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.895477+0000", 2024-09-21T21:42:54.008 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0001171 2024-09-21T21:42:54.008 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.008 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.008 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:54.009 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.895646+0000", 2024-09-21T21:42:54.009 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000168524 2024-09-21T21:42:54.009 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:54.009 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:54.009 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:54.009 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.009 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.009 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13562 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 1757184~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:54.009 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.824011+0000", 2024-09-21T21:42:54.009 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 63.172398280000003, 2024-09-21T21:42:54.009 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072020453999999998, 2024-09-21T21:42:54.010 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:54.010 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:54.010 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:54.010 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:54.010 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:54.010 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:54.010 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13562 2024-09-21T21:42:54.010 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.010 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:54.010 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.010 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:54.010 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824011+0000", 2024-09-21T21:42:54.010 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:54.011 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.011 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.011 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:54.011 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824011+0000", 2024-09-21T21:42:54.011 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:54.011 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.011 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.011 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:54.012 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824011+0000", 2024-09-21T21:42:54.012 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5600000000000001e-07 2024-09-21T21:42:54.012 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.012 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.012 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:54.012 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824013+0000", 2024-09-21T21:42:54.012 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9800000000000001e-06 2024-09-21T21:42:54.012 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.012 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.012 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:54.012 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824019+0000", 2024-09-21T21:42:54.012 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7339999999999996e-06 2024-09-21T21:42:54.013 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.013 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.013 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:54.013 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824023+0000", 2024-09-21T21:42:54.013 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3680000000000004e-06 2024-09-21T21:42:54.013 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.013 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.013 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:54.013 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.895699+0000", 2024-09-21T21:42:54.013 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071675597999999993 2024-09-21T21:42:54.013 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.013 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.014 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:54.014 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.895830+0000", 2024-09-21T21:42:54.014 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000130858 2024-09-21T21:42:54.014 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.014 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.014 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:54.014 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896031+0000", 2024-09-21T21:42:54.014 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00020155999999999999 2024-09-21T21:42:54.014 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:54.014 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:54.014 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:54.014 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.014 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.015 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13563 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 323584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:54.015 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.824029+0000", 2024-09-21T21:42:54.015 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 63.172379839999998, 2024-09-21T21:42:54.015 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072307563000000005, 2024-09-21T21:42:54.015 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:54.015 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:54.015 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:54.015 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:54.015 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:54.015 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:54.015 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13563 2024-09-21T21:42:54.015 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.015 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:54.016 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.016 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:54.016 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824029+0000", 2024-09-21T21:42:54.016 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:54.016 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.016 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.016 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:54.016 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824029+0000", 2024-09-21T21:42:54.016 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:54.016 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.016 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.016 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:54.016 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824030+0000", 2024-09-21T21:42:54.017 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1699999999999999e-07 2024-09-21T21:42:54.017 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.017 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.017 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:54.017 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824032+0000", 2024-09-21T21:42:54.017 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0320000000000002e-06 2024-09-21T21:42:54.017 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.017 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.017 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:54.017 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824038+0000", 2024-09-21T21:42:54.017 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8769999999999998e-06 2024-09-21T21:42:54.017 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.017 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.017 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:54.018 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824042+0000", 2024-09-21T21:42:54.018 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5650000000000003e-06 2024-09-21T21:42:54.018 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.018 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.018 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:54.018 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896058+0000", 2024-09-21T21:42:54.018 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072015941 2024-09-21T21:42:54.018 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.018 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.018 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:54.018 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896164+0000", 2024-09-21T21:42:54.018 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010593499999999999 2024-09-21T21:42:54.018 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.018 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.019 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:54.019 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896337+0000", 2024-09-21T21:42:54.019 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00017279599999999999 2024-09-21T21:42:54.019 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:54.019 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:54.019 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:54.019 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.019 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.019 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13564 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 983040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:54.019 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.824048+0000", 2024-09-21T21:42:54.019 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 63.172360765000001, 2024-09-21T21:42:54.019 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072491193999999995, 2024-09-21T21:42:54.019 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:54.019 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:54.020 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:54.020 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:54.020 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:54.020 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:54.020 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13564 2024-09-21T21:42:54.020 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.020 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:54.020 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.020 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:54.020 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824048+0000", 2024-09-21T21:42:54.020 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:54.020 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.020 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.021 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:54.021 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824048+0000", 2024-09-21T21:42:54.021 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:54.021 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.021 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.021 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:54.021 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824663+0000", 2024-09-21T21:42:54.021 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00061494299999999996 2024-09-21T21:42:54.021 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.021 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.021 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:54.021 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824669+0000", 2024-09-21T21:42:54.021 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2900000000000002e-06 2024-09-21T21:42:54.021 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.022 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.022 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:54.022 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824679+0000", 2024-09-21T21:42:54.022 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0338e-05 2024-09-21T21:42:54.022 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.022 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.022 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:54.022 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824687+0000", 2024-09-21T21:42:54.022 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.9049999999999997e-06 2024-09-21T21:42:54.022 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.022 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.022 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:54.022 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896381+0000", 2024-09-21T21:42:54.022 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071694305999999999 2024-09-21T21:42:54.023 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.023 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.023 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:54.023 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896445+0000", 2024-09-21T21:42:54.023 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3869999999999997e-05 2024-09-21T21:42:54.023 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.023 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.023 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:54.023 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896540+0000", 2024-09-21T21:42:54.023 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.4542000000000004e-05 2024-09-21T21:42:54.024 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:54.024 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:54.024 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:54.024 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.024 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.024 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13565 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3190784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:54.024 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.824704+0000", 2024-09-21T21:42:54.024 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 63.171704894000001, 2024-09-21T21:42:54.024 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072014745000000005, 2024-09-21T21:42:54.024 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:54.024 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:54.024 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:54.024 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:54.024 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:54.025 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:54.025 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13565 2024-09-21T21:42:54.025 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.025 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:54.025 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.025 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:54.025 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824704+0000", 2024-09-21T21:42:54.025 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:54.025 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.025 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.025 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:54.025 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824704+0000", 2024-09-21T21:42:54.025 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:54.025 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.026 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.026 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:54.026 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824705+0000", 2024-09-21T21:42:54.026 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8999999999999997e-07 2024-09-21T21:42:54.026 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.026 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.026 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:54.026 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824707+0000", 2024-09-21T21:42:54.026 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.7669999999999999e-06 2024-09-21T21:42:54.026 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.026 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.026 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:54.027 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824715+0000", 2024-09-21T21:42:54.027 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7700000000000001e-06 2024-09-21T21:42:54.027 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.027 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.027 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:54.027 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824721+0000", 2024-09-21T21:42:54.027 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9560000000000002e-06 2024-09-21T21:42:54.027 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.027 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.027 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:54.027 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896570+0000", 2024-09-21T21:42:54.027 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071848870999999995 2024-09-21T21:42:54.028 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.028 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.028 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:54.028 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896626+0000", 2024-09-21T21:42:54.028 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5578000000000002e-05 2024-09-21T21:42:54.028 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.028 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.028 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:54.028 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896719+0000", 2024-09-21T21:42:54.029 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.3313000000000006e-05 2024-09-21T21:42:54.029 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:54.029 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:54.029 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:54.029 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.029 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.029 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13566 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3162112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:54.029 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.824729+0000", 2024-09-21T21:42:54.029 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 63.171680006999999, 2024-09-21T21:42:54.029 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072356907999999998, 2024-09-21T21:42:54.029 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:54.029 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:54.030 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:54.030 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:54.030 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:54.030 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:54.030 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13566 2024-09-21T21:42:54.030 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.030 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:54.030 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.030 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:54.031 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824729+0000", 2024-09-21T21:42:54.031 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:54.031 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.031 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.031 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:54.031 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824729+0000", 2024-09-21T21:42:54.031 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:54.031 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.031 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.031 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:54.031 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825630+0000", 2024-09-21T21:42:54.031 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00090139400000000002 2024-09-21T21:42:54.031 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.032 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.032 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:54.032 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825634+0000", 2024-09-21T21:42:54.032 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4960000000000001e-06 2024-09-21T21:42:54.032 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.032 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.032 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:54.032 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825644+0000", 2024-09-21T21:42:54.032 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.9650000000000003e-06 2024-09-21T21:42:54.032 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.032 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.032 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:54.032 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825649+0000", 2024-09-21T21:42:54.033 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0359999999999997e-06 2024-09-21T21:42:54.033 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.033 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.033 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:54.033 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896757+0000", 2024-09-21T21:42:54.033 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071107717000000001 2024-09-21T21:42:54.033 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.033 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.033 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:54.033 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896878+0000", 2024-09-21T21:42:54.033 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000120834 2024-09-21T21:42:54.033 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.033 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.033 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:54.034 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897086+0000", 2024-09-21T21:42:54.034 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00020846600000000001 2024-09-21T21:42:54.034 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:54.034 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:54.034 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:54.034 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.034 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.034 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13568 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3907584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:54.034 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.825671+0000", 2024-09-21T21:42:54.034 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 63.170737598000002, 2024-09-21T21:42:54.034 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071794595000000003, 2024-09-21T21:42:54.034 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:54.034 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:54.035 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:54.035 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:54.035 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:54.035 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:54.035 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13568 2024-09-21T21:42:54.035 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.035 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:54.035 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.035 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:54.035 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825671+0000", 2024-09-21T21:42:54.035 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:54.035 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.035 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.035 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:54.036 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825671+0000", 2024-09-21T21:42:54.036 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:54.036 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.036 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.036 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:54.036 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825672+0000", 2024-09-21T21:42:54.036 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2999999999999999e-07 2024-09-21T21:42:54.036 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.036 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.036 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:54.036 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825673+0000", 2024-09-21T21:42:54.036 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.246e-06 2024-09-21T21:42:54.036 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.036 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.037 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:54.037 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825693+0000", 2024-09-21T21:42:54.037 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9789999999999999e-05 2024-09-21T21:42:54.037 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.037 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.037 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:54.037 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825697+0000", 2024-09-21T21:42:54.037 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2379999999999999e-06 2024-09-21T21:42:54.037 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.037 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.037 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:54.037 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897284+0000", 2024-09-21T21:42:54.037 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071586710999999997 2024-09-21T21:42:54.037 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.037 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.038 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:54.038 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897337+0000", 2024-09-21T21:42:54.038 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2800000000000003e-05 2024-09-21T21:42:54.038 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.038 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.038 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:54.038 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897466+0000", 2024-09-21T21:42:54.038 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00012957999999999999 2024-09-21T21:42:54.038 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:54.038 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:54.038 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:54.038 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.038 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.039 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13574 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 352256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:54.039 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.825702+0000", 2024-09-21T21:42:54.039 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 63.170707450000002, 2024-09-21T21:42:54.039 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072080377000000001, 2024-09-21T21:42:54.039 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:54.039 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:54.039 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:54.039 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:54.039 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:54.039 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:54.039 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13574 2024-09-21T21:42:54.039 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.040 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:54.040 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.040 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:54.040 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825702+0000", 2024-09-21T21:42:54.040 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:54.040 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.040 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.040 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:54.040 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825702+0000", 2024-09-21T21:42:54.040 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:54.040 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.040 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.041 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:54.041 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825943+0000", 2024-09-21T21:42:54.041 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00024105200000000001 2024-09-21T21:42:54.041 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.041 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.041 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:54.041 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825946+0000", 2024-09-21T21:42:54.041 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.743e-06 2024-09-21T21:42:54.041 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.041 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.041 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:54.041 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825956+0000", 2024-09-21T21:42:54.041 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6609999999999995e-06 2024-09-21T21:42:54.041 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.042 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.042 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:54.042 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825963+0000", 2024-09-21T21:42:54.042 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4609999999999996e-06 2024-09-21T21:42:54.042 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.042 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.042 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:54.042 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897502+0000", 2024-09-21T21:42:54.042 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071539354999999999 2024-09-21T21:42:54.042 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.042 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.043 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:54.043 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897619+0000", 2024-09-21T21:42:54.043 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011665 2024-09-21T21:42:54.043 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.043 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.043 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:54.043 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897782+0000", 2024-09-21T21:42:54.043 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00016345500000000001 2024-09-21T21:42:54.043 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:54.043 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:54.043 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:54.043 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.044 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.044 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13575 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 380928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:54.044 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.825973+0000", 2024-09-21T21:42:54.044 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 63.170436014000003, 2024-09-21T21:42:54.044 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072012723000000001, 2024-09-21T21:42:54.044 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:54.044 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:54.044 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:54.044 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:54.044 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:54.044 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:54.044 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13575 2024-09-21T21:42:54.044 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.044 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:54.045 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.045 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:54.045 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825973+0000", 2024-09-21T21:42:54.045 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:54.045 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.045 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.045 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:54.045 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825973+0000", 2024-09-21T21:42:54.045 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:54.045 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.045 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.045 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:54.046 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826746+0000", 2024-09-21T21:42:54.046 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00077326800000000002 2024-09-21T21:42:54.046 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.046 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.046 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:54.046 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826766+0000", 2024-09-21T21:42:54.046 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9465000000000001e-05 2024-09-21T21:42:54.046 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.046 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.046 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:54.046 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826812+0000", 2024-09-21T21:42:54.046 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5927999999999998e-05 2024-09-21T21:42:54.046 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.047 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.047 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:54.047 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826833+0000", 2024-09-21T21:42:54.047 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1710999999999999e-05 2024-09-21T21:42:54.047 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.047 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.047 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:54.047 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897809+0000", 2024-09-21T21:42:54.047 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.070975229000000001 2024-09-21T21:42:54.047 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.047 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.047 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:54.048 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897864+0000", 2024-09-21T21:42:54.048 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5334999999999998e-05 2024-09-21T21:42:54.048 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.048 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.048 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:54.048 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897986+0000", 2024-09-21T21:42:54.048 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000121787 2024-09-21T21:42:54.048 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:54.048 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:54.048 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:54.048 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.048 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.048 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13577 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 1814528~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:54.048 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.826874+0000", 2024-09-21T21:42:54.049 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 63.169534980000002, 2024-09-21T21:42:54.049 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071639123999999998, 2024-09-21T21:42:54.049 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:54.049 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:54.049 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:54.049 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:54.049 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:54.049 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:54.049 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13577 2024-09-21T21:42:54.049 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.049 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:54.049 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.050 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:54.050 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826874+0000", 2024-09-21T21:42:54.050 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:54.050 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.050 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.050 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:54.050 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826874+0000", 2024-09-21T21:42:54.050 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:54.050 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.050 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.050 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:54.050 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826875+0000", 2024-09-21T21:42:54.050 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.82e-07 2024-09-21T21:42:54.051 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.051 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.051 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:54.051 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826877+0000", 2024-09-21T21:42:54.051 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.295e-06 2024-09-21T21:42:54.051 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.051 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.051 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:54.051 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826883+0000", 2024-09-21T21:42:54.051 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2609999999999999e-06 2024-09-21T21:42:54.051 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.051 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.051 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:54.052 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826889+0000", 2024-09-21T21:42:54.052 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5589999999999996e-06 2024-09-21T21:42:54.052 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.052 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.052 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:54.052 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898301+0000", 2024-09-21T21:42:54.052 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071412188000000001 2024-09-21T21:42:54.052 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.052 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.052 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:54.052 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898416+0000", 2024-09-21T21:42:54.052 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000115538 2024-09-21T21:42:54.052 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.053 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.053 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:54.053 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898513+0000", 2024-09-21T21:42:54.053 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6800999999999997e-05 2024-09-21T21:42:54.053 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:54.053 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:54.053 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:54.053 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.053 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.053 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13578 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 2502656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:54.053 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.826896+0000", 2024-09-21T21:42:54.053 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 63.169513524000003, 2024-09-21T21:42:54.053 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071789411999999997, 2024-09-21T21:42:54.053 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:54.054 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:54.054 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:54.054 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:54.054 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:54.054 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:54.054 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13578 2024-09-21T21:42:54.054 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.054 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:54.054 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.054 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:54.054 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826896+0000", 2024-09-21T21:42:54.054 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:54.054 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.054 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.055 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:54.055 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826896+0000", 2024-09-21T21:42:54.055 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:54.055 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.055 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.055 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:54.055 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827327+0000", 2024-09-21T21:42:54.055 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00043094500000000002 2024-09-21T21:42:54.055 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.055 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.055 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:54.055 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827331+0000", 2024-09-21T21:42:54.055 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.0969999999999999e-06 2024-09-21T21:42:54.056 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.056 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.056 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:54.056 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827341+0000", 2024-09-21T21:42:54.056 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0199e-05 2024-09-21T21:42:54.056 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.056 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.056 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:54.056 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827346+0000", 2024-09-21T21:42:54.056 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3210000000000004e-06 2024-09-21T21:42:54.056 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.056 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.056 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:54.056 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898540+0000", 2024-09-21T21:42:54.057 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071194246000000003 2024-09-21T21:42:54.057 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.057 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.057 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:54.057 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898598+0000", 2024-09-21T21:42:54.057 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7658999999999998e-05 2024-09-21T21:42:54.057 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.057 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.057 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:54.057 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898685+0000", 2024-09-21T21:42:54.057 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.6945000000000002e-05 2024-09-21T21:42:54.057 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:54.057 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:54.057 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:54.057 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.058 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.058 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13579 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 2531328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:54.058 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.827355+0000", 2024-09-21T21:42:54.058 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 63.169053992999999, 2024-09-21T21:42:54.058 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071650933, 2024-09-21T21:42:54.058 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:54.058 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:54.058 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:54.058 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:54.058 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:54.059 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:54.059 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13579 2024-09-21T21:42:54.059 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.059 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:54.059 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.059 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:54.059 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827355+0000", 2024-09-21T21:42:54.059 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:54.059 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.059 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.059 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:54.059 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827355+0000", 2024-09-21T21:42:54.059 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:54.059 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.060 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.060 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:54.060 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827486+0000", 2024-09-21T21:42:54.060 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00013066799999999999 2024-09-21T21:42:54.060 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.060 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.060 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:54.060 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827490+0000", 2024-09-21T21:42:54.060 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.9009999999999996e-06 2024-09-21T21:42:54.060 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.060 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.060 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:54.061 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827499+0000", 2024-09-21T21:42:54.061 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6209999999999999e-06 2024-09-21T21:42:54.061 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.061 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.061 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:54.061 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827506+0000", 2024-09-21T21:42:54.061 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6490000000000002e-06 2024-09-21T21:42:54.061 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.061 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.061 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:54.061 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898713+0000", 2024-09-21T21:42:54.062 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071206937999999997 2024-09-21T21:42:54.062 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.062 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.062 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:54.062 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898811+0000", 2024-09-21T21:42:54.062 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.8159999999999995e-05 2024-09-21T21:42:54.062 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.062 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.062 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:54.062 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899006+0000", 2024-09-21T21:42:54.062 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00019499599999999999 2024-09-21T21:42:54.062 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:54.063 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:54.063 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:54.063 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.063 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.063 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13580 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3219456~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:54.063 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.827522+0000", 2024-09-21T21:42:54.063 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 63.168887562000002, 2024-09-21T21:42:54.063 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071813298999999997, 2024-09-21T21:42:54.063 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:54.063 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:54.063 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:54.064 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:54.064 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:54.064 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:54.064 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13580 2024-09-21T21:42:54.064 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.064 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:54.064 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.064 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:54.064 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827522+0000", 2024-09-21T21:42:54.064 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:54.064 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.065 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.065 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:54.065 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827522+0000", 2024-09-21T21:42:54.065 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:54.065 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.065 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.065 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:54.065 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827898+0000", 2024-09-21T21:42:54.065 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000376876 2024-09-21T21:42:54.065 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.065 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.065 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:54.065 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827903+0000", 2024-09-21T21:42:54.065 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4129999999999999e-06 2024-09-21T21:42:54.066 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.066 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.066 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:54.066 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827911+0000", 2024-09-21T21:42:54.066 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.4540000000000007e-06 2024-09-21T21:42:54.066 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.066 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.066 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:54.066 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827917+0000", 2024-09-21T21:42:54.066 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2110000000000003e-06 2024-09-21T21:42:54.066 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.066 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.066 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:54.067 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899041+0000", 2024-09-21T21:42:54.067 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071123965999999997 2024-09-21T21:42:54.067 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.067 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.067 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:54.067 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899157+0000", 2024-09-21T21:42:54.067 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011573700000000001 2024-09-21T21:42:54.067 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.067 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.067 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:54.067 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899335+0000", 2024-09-21T21:42:54.067 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00017764199999999999 2024-09-21T21:42:54.067 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:54.067 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:54.068 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:54.068 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.068 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.068 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13581 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3248128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:54.068 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.827929+0000", 2024-09-21T21:42:54.068 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 63.168480162999998, 2024-09-21T21:42:54.068 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071730151000000006, 2024-09-21T21:42:54.068 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:54.068 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:54.068 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:54.068 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:54.068 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:54.068 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:54.069 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13581 2024-09-21T21:42:54.069 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.069 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:54.069 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.069 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:54.069 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827929+0000", 2024-09-21T21:42:54.069 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:54.069 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.069 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.069 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:54.069 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827929+0000", 2024-09-21T21:42:54.069 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:54.069 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.069 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.070 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:54.070 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.829943+0000", 2024-09-21T21:42:54.070 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0020143750000000001 2024-09-21T21:42:54.070 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.070 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.070 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:54.070 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.829948+0000", 2024-09-21T21:42:54.070 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9400000000000001e-06 2024-09-21T21:42:54.070 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.070 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.070 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:54.070 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.829960+0000", 2024-09-21T21:42:54.071 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.2123999999999999e-05 2024-09-21T21:42:54.071 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.071 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.071 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:54.071 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.829968+0000", 2024-09-21T21:42:54.071 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7000000000000008e-06 2024-09-21T21:42:54.071 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.071 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.071 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:54.071 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899370+0000", 2024-09-21T21:42:54.071 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.069401672999999997 2024-09-21T21:42:54.071 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.072 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.072 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:54.072 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899481+0000", 2024-09-21T21:42:54.072 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011145300000000001 2024-09-21T21:42:54.072 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.072 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.072 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:54.072 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899659+0000", 2024-09-21T21:42:54.072 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00017788600000000001 2024-09-21T21:42:54.072 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:54.072 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:54.072 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:54.072 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.072 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.072 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13595 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 1871872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:54.073 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.830180+0000", 2024-09-21T21:42:54.073 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 63.166229049000002, 2024-09-21T21:42:54.073 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071809244999999994, 2024-09-21T21:42:54.073 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:54.073 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:54.073 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:54.073 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:54.073 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:54.073 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:54.073 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13595 2024-09-21T21:42:54.073 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.073 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:54.074 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.074 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:54.074 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.830180+0000", 2024-09-21T21:42:54.074 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:54.074 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.074 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.074 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:54.074 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.830180+0000", 2024-09-21T21:42:54.074 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:54.074 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.074 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.074 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:54.075 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.831967+0000", 2024-09-21T21:42:54.075 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0017868350000000001 2024-09-21T21:42:54.075 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.075 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.075 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:54.075 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.831971+0000", 2024-09-21T21:42:54.075 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3950000000000003e-06 2024-09-21T21:42:54.075 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.075 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.075 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:54.075 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.831983+0000", 2024-09-21T21:42:54.075 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1815e-05 2024-09-21T21:42:54.076 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.076 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.076 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:54.076 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.831993+0000", 2024-09-21T21:42:54.076 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6339999999999996e-06 2024-09-21T21:42:54.076 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.076 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.076 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:54.076 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.901693+0000", 2024-09-21T21:42:54.076 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.069700082999999996 2024-09-21T21:42:54.076 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.076 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.077 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:54.077 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.901813+0000", 2024-09-21T21:42:54.077 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011971 2024-09-21T21:42:54.077 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.077 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.077 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:54.077 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.901989+0000", 2024-09-21T21:42:54.077 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000176773 2024-09-21T21:42:54.077 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:54.077 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:54.077 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:54.077 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.078 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.078 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13601 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3334144~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:54.078 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.832130+0000", 2024-09-21T21:42:54.078 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 63.164279176999997, 2024-09-21T21:42:54.078 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071861727, 2024-09-21T21:42:54.078 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:54.078 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:54.078 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:54.078 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:54.078 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:54.078 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:54.078 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13601 2024-09-21T21:42:54.079 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.079 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:54.079 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.079 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:54.079 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.832130+0000", 2024-09-21T21:42:54.079 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:54.079 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.079 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.079 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:54.079 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.832130+0000", 2024-09-21T21:42:54.079 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:54.079 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.079 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.080 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:54.080 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.833939+0000", 2024-09-21T21:42:54.080 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0018094400000000001 2024-09-21T21:42:54.081 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.081 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.081 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:54.081 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.833944+0000", 2024-09-21T21:42:54.082 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5979999999999999e-06 2024-09-21T21:42:54.082 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.082 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.082 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:54.082 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.833953+0000", 2024-09-21T21:42:54.082 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.1460000000000002e-06 2024-09-21T21:42:54.082 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.082 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.082 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:54.082 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.833960+0000", 2024-09-21T21:42:54.082 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.7989999999999996e-06 2024-09-21T21:42:54.082 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.082 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.083 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:54.083 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.903693+0000", 2024-09-21T21:42:54.083 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.069732901 2024-09-21T21:42:54.083 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.083 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.083 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:54.083 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.903814+0000", 2024-09-21T21:42:54.083 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000121585 2024-09-21T21:42:54.083 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.083 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.083 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:54.083 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.903992+0000", 2024-09-21T21:42:54.084 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00017725799999999999 2024-09-21T21:42:54.084 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:54.084 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:54.084 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:54.084 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.084 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.084 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:12342 3.1 3:87417293:::rbd_data.1121faebdfe3.0000000000000004:head [read 4096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:42:54.084 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:14.411725+0000", 2024-09-21T21:42:54.084 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 39.584684111000001, 2024-09-21T21:42:54.084 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0089213029999999, 2024-09-21T21:42:54.084 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:54.084 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:54.084 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:54.084 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:54.085 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:54.085 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:54.085 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 12342 2024-09-21T21:42:54.085 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.085 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:54.085 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.085 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:54.085 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411725+0000", 2024-09-21T21:42:54.085 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:54.085 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.085 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.085 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:54.085 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411725+0000", 2024-09-21T21:42:54.085 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:54.086 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.086 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.086 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:54.086 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411726+0000", 2024-09-21T21:42:54.086 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.4169999999999999e-06 2024-09-21T21:42:54.086 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.086 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.086 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:54.086 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411734+0000", 2024-09-21T21:42:54.086 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.3799999999999996e-06 2024-09-21T21:42:54.086 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.086 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.086 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:54.087 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411750+0000", 2024-09-21T21:42:54.087 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.6429000000000001e-05 2024-09-21T21:42:54.087 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.087 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.087 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:54.087 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411760+0000", 2024-09-21T21:42:54.087 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.4239999999999999e-06 2024-09-21T21:42:54.087 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.087 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.087 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:54.087 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411828+0000", 2024-09-21T21:42:54.087 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.7935999999999995e-05 2024-09-21T21:42:54.087 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.087 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.088 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op must wait for map", 2024-09-21T21:42:54.088 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411837+0000", 2024-09-21T21:42:54.088 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.0100000000000001e-06 2024-09-21T21:42:54.088 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.088 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.088 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:54.088 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.414984+0000", 2024-09-21T21:42:54.088 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0031472050000000001 2024-09-21T21:42:54.088 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.088 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.088 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for peered", 2024-09-21T21:42:54.088 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.414994+0000", 2024-09-21T21:42:54.088 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0291e-05 2024-09-21T21:42:54.089 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.089 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.089 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:54.089 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.420353+0000", 2024-09-21T21:42:54.089 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.005358913 2024-09-21T21:42:54.089 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.089 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.089 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:54.089 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.420489+0000", 2024-09-21T21:42:54.089 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00013592699999999999 2024-09-21T21:42:54.089 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.089 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.089 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:54.089 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.420646+0000", 2024-09-21T21:42:54.089 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00015737100000000001 2024-09-21T21:42:54.090 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:54.090 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:54.090 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:54.090 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.090 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.090 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:12348 3.1 3:87417293:::rbd_data.1121faebdfe3.0000000000000004:head [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:42:54.090 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:14.411793+0000", 2024-09-21T21:42:54.090 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 39.584616564999997, 2024-09-21T21:42:54.090 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.009298384, 2024-09-21T21:42:54.090 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:54.090 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:54.090 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:54.090 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:54.091 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:54.091 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:54.091 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 12348 2024-09-21T21:42:54.091 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.091 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:54.091 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.091 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:54.091 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411793+0000", 2024-09-21T21:42:54.091 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:54.091 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.092 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.092 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:54.092 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411793+0000", 2024-09-21T21:42:54.092 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:54.092 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.092 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.092 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:54.092 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411793+0000", 2024-09-21T21:42:54.092 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0999999999999998e-07 2024-09-21T21:42:54.092 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.092 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.092 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:54.092 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411797+0000", 2024-09-21T21:42:54.092 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.49e-06 2024-09-21T21:42:54.093 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.093 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.093 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:54.093 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411809+0000", 2024-09-21T21:42:54.093 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.2510999999999999e-05 2024-09-21T21:42:54.093 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.093 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.093 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:54.093 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411816+0000", 2024-09-21T21:42:54.093 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6420000000000001e-06 2024-09-21T21:42:54.094 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.094 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.094 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:54.094 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411963+0000", 2024-09-21T21:42:54.094 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00014700999999999999 2024-09-21T21:42:54.094 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.094 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.094 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting_for_map not empty", 2024-09-21T21:42:54.094 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411970+0000", 2024-09-21T21:42:54.094 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9800000000000001e-06 2024-09-21T21:42:54.094 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.094 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.094 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:54.094 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.415018+0000", 2024-09-21T21:42:54.095 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0030482299999999999 2024-09-21T21:42:54.095 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.095 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.095 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for peered", 2024-09-21T21:42:54.095 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.415025+0000", 2024-09-21T21:42:54.095 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.3749999999999997e-06 2024-09-21T21:42:54.095 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.095 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.095 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:54.095 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.420904+0000", 2024-09-21T21:42:54.096 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0058787339999999 2024-09-21T21:42:54.096 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.096 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.096 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:54.096 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.420988+0000", 2024-09-21T21:42:54.096 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.3955999999999994e-05 2024-09-21T21:42:54.096 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.096 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.096 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:54.096 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.421091+0000", 2024-09-21T21:42:54.096 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000102846 2024-09-21T21:42:54.096 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:54.096 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:54.096 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:54.097 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.097 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.097 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4385.0:20922 3.9 3:920c5256:::rbd_id.image_client.0-clone36:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-21T21:42:54.097 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:17.811324+0000", 2024-09-21T21:42:54.097 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 36.185084832999998, 2024-09-21T21:42:54.097 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.66951882299999999, 2024-09-21T21:42:54.097 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:54.097 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:54.097 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:54.097 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:54.097 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4385", 2024-09-21T21:42:54.097 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:42:54.097 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 20922 2024-09-21T21:42:54.097 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.098 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:54.098 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.098 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:54.098 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:17.811324+0000", 2024-09-21T21:42:54.098 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:54.098 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.098 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.098 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:54.098 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:17.811324+0000", 2024-09-21T21:42:54.098 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:54.098 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.098 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.098 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:54.099 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:17.811326+0000", 2024-09-21T21:42:54.099 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.536e-06 2024-09-21T21:42:54.099 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.099 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.099 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:54.099 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:17.811332+0000", 2024-09-21T21:42:54.099 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6699999999999997e-06 2024-09-21T21:42:54.099 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.099 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.099 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:54.099 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:17.811347+0000", 2024-09-21T21:42:54.099 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.4589999999999999e-05 2024-09-21T21:42:54.100 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.100 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.100 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:54.100 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:17.811359+0000", 2024-09-21T21:42:54.100 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1776e-05 2024-09-21T21:42:54.100 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.100 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.100 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:54.100 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:17.811424+0000", 2024-09-21T21:42:54.100 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4795000000000006e-05 2024-09-21T21:42:54.100 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.100 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.100 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for peered", 2024-09-21T21:42:54.100 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:17.811436+0000", 2024-09-21T21:42:54.101 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.221e-05 2024-09-21T21:42:54.101 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.101 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.101 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:54.101 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:18.480657+0000", 2024-09-21T21:42:54.101 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.66922088999999996 2024-09-21T21:42:54.101 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.101 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.101 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:54.101 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:18.480760+0000", 2024-09-21T21:42:54.101 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000102827 2024-09-21T21:42:54.101 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:54.101 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:54.101 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:54.102 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:18.480843+0000", 2024-09-21T21:42:54.102 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.3529000000000006e-05 2024-09-21T21:42:54.102 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:54.102 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:54.102 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:54.102 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:54.102 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:54.102 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:54.104 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T21:42:54.104 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:54.101+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:54.204 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:54.202+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:54.212 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:54.212 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:42:54.212 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-21T21:42:54.212 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:54.221 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T21:42:54.236 INFO:teuthology.orchestra.run.smithi081.stderr:noscrub is unset 2024-09-21T21:42:54.305 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:54.302+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:54.335 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:54.335 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:42:54.335 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:42:54.335 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:42:54.335 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:54.345 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T21:42:54.406 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:54.404+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:54.451 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:54.451 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-21T21:42:54.451 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-21T21:42:54.452 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:42:54.452 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.452 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19747 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 7905280~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:54.452 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273058+0000", 2024-09-21T21:42:54.452 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 63.175977113000002, 2024-09-21T21:42:54.452 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97338591399999996, 2024-09-21T21:42:54.452 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:54.452 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:54.452 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:54.452 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:54.452 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:54.452 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:54.452 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19747 2024-09-21T21:42:54.453 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.453 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:54.453 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.453 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:54.453 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273058+0000", 2024-09-21T21:42:54.453 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:54.453 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.453 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.453 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:54.453 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273058+0000", 2024-09-21T21:42:54.453 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:54.453 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.453 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.453 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:54.453 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273058+0000", 2024-09-21T21:42:54.454 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5900000000000003e-07 2024-09-21T21:42:54.454 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.454 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.454 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:54.454 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273061+0000", 2024-09-21T21:42:54.454 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3479999999999998e-06 2024-09-21T21:42:54.454 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.454 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.454 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:54.454 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273067+0000", 2024-09-21T21:42:54.454 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1469999999999998e-06 2024-09-21T21:42:54.454 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.454 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.454 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:54.455 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273071+0000", 2024-09-21T21:42:54.455 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7339999999999999e-06 2024-09-21T21:42:54.455 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.455 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.455 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:54.455 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277945+0000", 2024-09-21T21:42:54.455 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.004873474 2024-09-21T21:42:54.455 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.455 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.455 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:54.455 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277948+0000", 2024-09-21T21:42:54.455 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0299999999999998e-06 2024-09-21T21:42:54.455 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.455 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.456 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:54.456 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246269+0000", 2024-09-21T21:42:54.456 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96832060200000003 2024-09-21T21:42:54.456 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.456 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.456 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:54.456 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246362+0000", 2024-09-21T21:42:54.456 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3867000000000001e-05 2024-09-21T21:42:54.456 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.456 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.456 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:54.456 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246444+0000", 2024-09-21T21:42:54.456 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1353000000000005e-05 2024-09-21T21:42:54.456 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:54.456 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:54.457 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:54.457 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.457 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.457 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19755 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 5783552~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:54.457 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273077+0000", 2024-09-21T21:42:54.457 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 63.175957496999999, 2024-09-21T21:42:54.457 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97350203400000002, 2024-09-21T21:42:54.457 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:54.457 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:54.457 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:54.457 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:54.457 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:54.457 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:54.457 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19755 2024-09-21T21:42:54.457 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.458 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:54.458 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.458 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:54.458 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273077+0000", 2024-09-21T21:42:54.458 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:54.458 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.458 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.458 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:54.458 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273077+0000", 2024-09-21T21:42:54.458 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:54.458 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.458 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.458 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:54.458 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273078+0000", 2024-09-21T21:42:54.459 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5699999999999998e-07 2024-09-21T21:42:54.459 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.459 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.459 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:54.459 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273083+0000", 2024-09-21T21:42:54.459 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8670000000000002e-06 2024-09-21T21:42:54.459 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.459 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.459 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:54.459 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273088+0000", 2024-09-21T21:42:54.459 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8370000000000002e-06 2024-09-21T21:42:54.459 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.459 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.459 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:54.459 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273093+0000", 2024-09-21T21:42:54.460 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5709999999999999e-06 2024-09-21T21:42:54.460 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.460 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.460 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:54.460 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277968+0000", 2024-09-21T21:42:54.460 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048753909999999998 2024-09-21T21:42:54.460 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.460 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.460 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:54.460 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277971+0000", 2024-09-21T21:42:54.460 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0189999999999998e-06 2024-09-21T21:42:54.460 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.460 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.460 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:54.461 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246485+0000", 2024-09-21T21:42:54.461 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96851399299999996 2024-09-21T21:42:54.461 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.461 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.461 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:54.461 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246536+0000", 2024-09-21T21:42:54.461 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0430000000000003e-05 2024-09-21T21:42:54.461 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.461 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.461 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:54.461 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246579+0000", 2024-09-21T21:42:54.461 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3569000000000002e-05 2024-09-21T21:42:54.461 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:54.461 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:54.461 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:54.462 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.462 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.462 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19756 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6500352~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:54.462 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273099+0000", 2024-09-21T21:42:54.462 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 63.175936124000003, 2024-09-21T21:42:54.462 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97359272799999996, 2024-09-21T21:42:54.462 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:54.462 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:54.462 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:54.462 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:54.462 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:54.462 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:54.463 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19756 2024-09-21T21:42:54.463 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.463 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:54.463 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.463 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:54.463 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273099+0000", 2024-09-21T21:42:54.463 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:54.463 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.463 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.463 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:54.463 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273099+0000", 2024-09-21T21:42:54.463 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:54.463 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.464 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.464 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:54.464 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273099+0000", 2024-09-21T21:42:54.464 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0900000000000002e-07 2024-09-21T21:42:54.464 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.464 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.464 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:54.464 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273101+0000", 2024-09-21T21:42:54.464 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0320000000000002e-06 2024-09-21T21:42:54.464 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.464 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.465 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:54.465 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273107+0000", 2024-09-21T21:42:54.465 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2830000000000001e-06 2024-09-21T21:42:54.465 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.465 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.465 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:54.465 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273111+0000", 2024-09-21T21:42:54.465 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2230000000000001e-06 2024-09-21T21:42:54.465 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.465 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.465 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:54.465 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277994+0000", 2024-09-21T21:42:54.465 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048837070000000002 2024-09-21T21:42:54.465 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.466 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.466 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:54.466 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278000+0000", 2024-09-21T21:42:54.466 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3149999999999999e-06 2024-09-21T21:42:54.466 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.466 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.466 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:54.466 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246607+0000", 2024-09-21T21:42:54.466 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.968606839 2024-09-21T21:42:54.466 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.466 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.466 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:54.466 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246651+0000", 2024-09-21T21:42:54.466 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4734999999999998e-05 2024-09-21T21:42:54.467 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.467 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.467 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:54.467 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246692+0000", 2024-09-21T21:42:54.467 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0185000000000002e-05 2024-09-21T21:42:54.467 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:54.467 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:54.467 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:54.467 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.467 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.467 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19757 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 5812224~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:54.467 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273116+0000", 2024-09-21T21:42:54.467 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 63.175918467000002, 2024-09-21T21:42:54.467 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97367813999999997, 2024-09-21T21:42:54.467 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:54.468 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:54.468 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:54.468 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:54.468 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:54.468 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:54.468 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19757 2024-09-21T21:42:54.468 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.468 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:54.468 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.468 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:54.468 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273116+0000", 2024-09-21T21:42:54.468 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:54.468 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.468 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.469 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:54.469 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273116+0000", 2024-09-21T21:42:54.469 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:54.469 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.469 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.469 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:54.469 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273117+0000", 2024-09-21T21:42:54.469 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0999999999999999e-07 2024-09-21T21:42:54.469 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.469 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.469 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:54.469 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273119+0000", 2024-09-21T21:42:54.469 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9970000000000001e-06 2024-09-21T21:42:54.469 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.470 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.470 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:54.470 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273125+0000", 2024-09-21T21:42:54.470 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9540000000000001e-06 2024-09-21T21:42:54.470 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.470 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.470 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:54.470 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273135+0000", 2024-09-21T21:42:54.470 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0597999999999999e-05 2024-09-21T21:42:54.470 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.470 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.470 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:54.470 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278019+0000", 2024-09-21T21:42:54.470 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048834849999999999 2024-09-21T21:42:54.471 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.471 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.471 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:54.471 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278022+0000", 2024-09-21T21:42:54.471 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9560000000000002e-06 2024-09-21T21:42:54.471 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.471 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.471 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:54.471 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246716+0000", 2024-09-21T21:42:54.471 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.968694202 2024-09-21T21:42:54.471 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.471 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.471 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:54.471 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246758+0000", 2024-09-21T21:42:54.471 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1607000000000003e-05 2024-09-21T21:42:54.472 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.472 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.472 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:54.472 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246795+0000", 2024-09-21T21:42:54.472 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6931000000000003e-05 2024-09-21T21:42:54.472 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:54.472 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:54.473 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:54.473 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.473 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.473 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19758 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6529024~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:54.473 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273142+0000", 2024-09-21T21:42:54.473 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 63.175893246000001, 2024-09-21T21:42:54.473 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97375924599999997, 2024-09-21T21:42:54.473 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:54.473 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:54.473 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:54.473 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:54.473 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:54.474 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:54.474 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19758 2024-09-21T21:42:54.474 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.474 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:54.474 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.474 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:54.474 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273142+0000", 2024-09-21T21:42:54.474 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:54.474 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.474 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.474 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:54.474 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273142+0000", 2024-09-21T21:42:54.474 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:54.475 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.475 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.475 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:54.475 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273142+0000", 2024-09-21T21:42:54.475 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0200000000000003e-07 2024-09-21T21:42:54.475 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.475 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.475 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:54.475 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273144+0000", 2024-09-21T21:42:54.475 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1390000000000002e-06 2024-09-21T21:42:54.475 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.475 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.475 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:54.475 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273150+0000", 2024-09-21T21:42:54.475 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6409999999999999e-06 2024-09-21T21:42:54.476 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.476 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.476 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:54.476 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273157+0000", 2024-09-21T21:42:54.476 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1359999999999999e-06 2024-09-21T21:42:54.476 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.476 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.476 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:54.476 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278050+0000", 2024-09-21T21:42:54.476 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048926170000000001 2024-09-21T21:42:54.476 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.476 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.476 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:54.476 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278053+0000", 2024-09-21T21:42:54.477 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4879999999999999e-06 2024-09-21T21:42:54.477 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.477 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.477 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:54.477 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246819+0000", 2024-09-21T21:42:54.477 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96876590399999996 2024-09-21T21:42:54.477 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.477 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.477 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:54.477 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246856+0000", 2024-09-21T21:42:54.477 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7287999999999999e-05 2024-09-21T21:42:54.477 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.477 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.477 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:54.478 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246901+0000", 2024-09-21T21:42:54.478 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4631e-05 2024-09-21T21:42:54.478 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:54.478 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:54.478 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:54.478 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.478 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.478 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19759 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 7933952~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:54.478 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273172+0000", 2024-09-21T21:42:54.478 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 63.175862981999998, 2024-09-21T21:42:54.478 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97384789500000002, 2024-09-21T21:42:54.478 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:54.478 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:54.478 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:54.478 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:54.479 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:54.479 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:54.479 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19759 2024-09-21T21:42:54.479 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.479 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:54.479 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.479 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:54.479 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273172+0000", 2024-09-21T21:42:54.479 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:54.479 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.480 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.480 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:54.480 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273172+0000", 2024-09-21T21:42:54.480 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:54.480 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.480 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.480 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:54.480 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273172+0000", 2024-09-21T21:42:54.480 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6100000000000002e-07 2024-09-21T21:42:54.480 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.480 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.480 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:54.480 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273174+0000", 2024-09-21T21:42:54.480 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.691e-06 2024-09-21T21:42:54.481 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.481 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.481 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:54.481 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273178+0000", 2024-09-21T21:42:54.481 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7579999999999998e-06 2024-09-21T21:42:54.481 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.481 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.481 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:54.481 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273180+0000", 2024-09-21T21:42:54.481 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7199999999999998e-06 2024-09-21T21:42:54.481 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.481 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.481 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:54.481 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278077+0000", 2024-09-21T21:42:54.482 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048969529999999999 2024-09-21T21:42:54.482 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.482 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.482 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:54.482 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278080+0000", 2024-09-21T21:42:54.482 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.066e-06 2024-09-21T21:42:54.482 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.482 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.482 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:54.482 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246925+0000", 2024-09-21T21:42:54.482 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96884490199999995 2024-09-21T21:42:54.482 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.482 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.482 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:54.483 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246974+0000", 2024-09-21T21:42:54.483 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8387999999999998e-05 2024-09-21T21:42:54.483 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.483 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.483 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:54.483 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247020+0000", 2024-09-21T21:42:54.483 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6156000000000003e-05 2024-09-21T21:42:54.483 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:54.483 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:54.483 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:54.483 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.483 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.483 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19760 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 7991296~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:54.483 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273220+0000", 2024-09-21T21:42:54.484 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 63.175814467999999, 2024-09-21T21:42:54.484 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97403125499999998, 2024-09-21T21:42:54.484 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:54.484 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:54.484 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:54.484 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:54.484 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:54.484 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:54.484 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19760 2024-09-21T21:42:54.484 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.484 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:54.484 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.484 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:54.484 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273220+0000", 2024-09-21T21:42:54.484 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:54.485 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.485 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.485 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:54.485 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273220+0000", 2024-09-21T21:42:54.485 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:54.485 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.485 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.485 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:54.485 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273221+0000", 2024-09-21T21:42:54.485 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.65e-07 2024-09-21T21:42:54.485 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.485 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.485 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:54.485 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273228+0000", 2024-09-21T21:42:54.486 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6040000000000004e-06 2024-09-21T21:42:54.486 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.486 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.486 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:54.486 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273233+0000", 2024-09-21T21:42:54.486 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1569999999999999e-06 2024-09-21T21:42:54.486 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.486 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.486 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:54.486 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273238+0000", 2024-09-21T21:42:54.486 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0050000000000004e-06 2024-09-21T21:42:54.486 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.486 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.486 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:54.486 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278107+0000", 2024-09-21T21:42:54.489 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00486857 2024-09-21T21:42:54.489 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.489 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.489 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:54.489 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278111+0000", 2024-09-21T21:42:54.489 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6389999999999998e-06 2024-09-21T21:42:54.490 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.490 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.490 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:54.490 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247069+0000", 2024-09-21T21:42:54.490 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96895796899999997 2024-09-21T21:42:54.490 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.490 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.490 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:54.490 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247166+0000", 2024-09-21T21:42:54.490 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.7158999999999996e-05 2024-09-21T21:42:54.490 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.490 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.490 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:54.490 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247252+0000", 2024-09-21T21:42:54.491 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5886999999999999e-05 2024-09-21T21:42:54.491 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:54.491 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:54.491 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:54.491 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.491 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.491 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19761 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 7962624~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:54.491 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273246+0000", 2024-09-21T21:42:54.491 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 63.175789408, 2024-09-21T21:42:54.491 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97415972200000001, 2024-09-21T21:42:54.491 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:54.491 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:54.491 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:54.491 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:54.492 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:54.492 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:54.492 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19761 2024-09-21T21:42:54.492 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.492 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:54.492 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.492 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:54.492 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273246+0000", 2024-09-21T21:42:54.492 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:54.492 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.492 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.492 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:54.492 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273246+0000", 2024-09-21T21:42:54.492 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:54.492 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.493 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.493 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:54.493 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273246+0000", 2024-09-21T21:42:54.493 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5100000000000001e-07 2024-09-21T21:42:54.493 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.493 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.493 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:54.493 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273248+0000", 2024-09-21T21:42:54.493 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1550000000000001e-06 2024-09-21T21:42:54.493 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.493 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.493 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:54.493 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273254+0000", 2024-09-21T21:42:54.493 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.964e-06 2024-09-21T21:42:54.494 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.494 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.494 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:54.494 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273259+0000", 2024-09-21T21:42:54.494 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5499999999999996e-06 2024-09-21T21:42:54.494 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.494 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.494 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:54.494 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278130+0000", 2024-09-21T21:42:54.494 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048715779999999997 2024-09-21T21:42:54.494 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.495 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.495 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:54.495 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278133+0000", 2024-09-21T21:42:54.495 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8739999999999999e-06 2024-09-21T21:42:54.495 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.495 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.495 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:54.495 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247270+0000", 2024-09-21T21:42:54.495 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.969137099 2024-09-21T21:42:54.495 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.495 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.495 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:54.495 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247338+0000", 2024-09-21T21:42:54.495 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7717000000000002e-05 2024-09-21T21:42:54.496 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.496 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.496 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:54.496 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247405+0000", 2024-09-21T21:42:54.496 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7434000000000005e-05 2024-09-21T21:42:54.496 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:54.496 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:54.496 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:54.496 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.496 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.496 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19769 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 5840896~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:54.496 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273287+0000", 2024-09-21T21:42:54.496 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 63.175748124000002, 2024-09-21T21:42:54.496 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97435390200000005, 2024-09-21T21:42:54.496 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:54.497 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:54.497 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:54.497 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:54.497 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:54.497 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:54.497 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19769 2024-09-21T21:42:54.497 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.497 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:54.497 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.497 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:54.497 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273287+0000", 2024-09-21T21:42:54.497 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:54.497 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.497 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.498 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:54.498 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273287+0000", 2024-09-21T21:42:54.498 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:54.498 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.498 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.498 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:54.498 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273287+0000", 2024-09-21T21:42:54.498 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6699999999999999e-07 2024-09-21T21:42:54.498 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.498 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.498 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:54.498 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273289+0000", 2024-09-21T21:42:54.498 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.708e-06 2024-09-21T21:42:54.498 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.499 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.499 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:54.499 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273296+0000", 2024-09-21T21:42:54.499 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8109999999999997e-06 2024-09-21T21:42:54.499 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.499 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.499 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:54.499 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273299+0000", 2024-09-21T21:42:54.499 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0740000000000001e-06 2024-09-21T21:42:54.499 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.499 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.499 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:54.499 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278159+0000", 2024-09-21T21:42:54.499 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048603240000000001 2024-09-21T21:42:54.499 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.500 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.500 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:54.500 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278162+0000", 2024-09-21T21:42:54.500 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1930000000000002e-06 2024-09-21T21:42:54.500 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.500 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.500 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:54.500 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247444+0000", 2024-09-21T21:42:54.500 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96928199100000001 2024-09-21T21:42:54.500 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.500 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.500 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:54.500 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247533+0000", 2024-09-21T21:42:54.500 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8490000000000001e-05 2024-09-21T21:42:54.500 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.501 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.501 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:54.501 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247641+0000", 2024-09-21T21:42:54.501 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000107944 2024-09-21T21:42:54.501 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:54.501 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:54.501 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:54.501 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.501 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.501 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19770 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 5869568~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:54.501 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273313+0000", 2024-09-21T21:42:54.501 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 63.175722022000002, 2024-09-21T21:42:54.501 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97449038899999996, 2024-09-21T21:42:54.501 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:54.502 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:54.502 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:54.502 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:54.502 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:54.502 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:54.502 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19770 2024-09-21T21:42:54.502 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.502 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:54.502 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.502 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:54.502 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273313+0000", 2024-09-21T21:42:54.502 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:54.502 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.502 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.503 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:54.503 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273313+0000", 2024-09-21T21:42:54.503 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:54.503 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.503 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.503 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:54.503 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273313+0000", 2024-09-21T21:42:54.503 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5699999999999998e-07 2024-09-21T21:42:54.503 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.503 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.503 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:54.503 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273315+0000", 2024-09-21T21:42:54.503 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0099999999999998e-06 2024-09-21T21:42:54.503 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.503 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.503 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:54.504 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273321+0000", 2024-09-21T21:42:54.504 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7529999999999998e-06 2024-09-21T21:42:54.504 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.504 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.504 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:54.504 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273328+0000", 2024-09-21T21:42:54.504 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9009999999999997e-06 2024-09-21T21:42:54.504 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.504 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.504 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:54.504 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278183+0000", 2024-09-21T21:42:54.504 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048547770000000002 2024-09-21T21:42:54.504 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.504 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.504 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:54.505 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278186+0000", 2024-09-21T21:42:54.505 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3100000000000001e-06 2024-09-21T21:42:54.505 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.505 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.505 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:54.505 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247676+0000", 2024-09-21T21:42:54.505 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96949004299999997 2024-09-21T21:42:54.505 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.505 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.505 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:54.505 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247757+0000", 2024-09-21T21:42:54.505 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0479e-05 2024-09-21T21:42:54.505 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.505 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.505 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:54.506 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247803+0000", 2024-09-21T21:42:54.506 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6758999999999997e-05 2024-09-21T21:42:54.506 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:54.506 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:54.506 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:54.506 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.506 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.506 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19771 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6586368~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:54.506 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273335+0000", 2024-09-21T21:42:54.506 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 63.175700235000001, 2024-09-21T21:42:54.506 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97458947699999998, 2024-09-21T21:42:54.506 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:54.506 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:54.506 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:54.506 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:54.507 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:54.507 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:54.507 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19771 2024-09-21T21:42:54.507 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.507 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:54.507 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.507 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:54.507 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273335+0000", 2024-09-21T21:42:54.507 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:54.507 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.507 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.507 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:54.507 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273335+0000", 2024-09-21T21:42:54.507 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:54.508 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.508 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.508 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:54.508 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273335+0000", 2024-09-21T21:42:54.508 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3599999999999999e-07 2024-09-21T21:42:54.508 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.508 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.508 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:54.508 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273337+0000", 2024-09-21T21:42:54.508 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0789999999999999e-06 2024-09-21T21:42:54.508 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.508 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.508 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:54.508 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273343+0000", 2024-09-21T21:42:54.509 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6280000000000002e-06 2024-09-21T21:42:54.509 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.509 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.509 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:54.509 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273346+0000", 2024-09-21T21:42:54.509 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.507e-06 2024-09-21T21:42:54.509 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.509 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.509 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:54.509 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278206+0000", 2024-09-21T21:42:54.509 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048600969999999999 2024-09-21T21:42:54.509 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.509 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.510 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:54.510 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278212+0000", 2024-09-21T21:42:54.510 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2079999999999999e-06 2024-09-21T21:42:54.510 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.510 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.510 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:54.510 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247828+0000", 2024-09-21T21:42:54.510 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.969616547 2024-09-21T21:42:54.510 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.510 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.510 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:54.510 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247881+0000", 2024-09-21T21:42:54.510 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3180999999999998e-05 2024-09-21T21:42:54.510 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.510 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.511 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:54.511 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247924+0000", 2024-09-21T21:42:54.511 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2894e-05 2024-09-21T21:42:54.511 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:54.511 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:54.511 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:54.511 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.511 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.511 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19772 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6557696~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:54.511 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273358+0000", 2024-09-21T21:42:54.511 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 63.175676854999999, 2024-09-21T21:42:54.511 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97467031800000004, 2024-09-21T21:42:54.511 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:54.511 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:54.512 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:54.512 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:54.512 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:54.512 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:54.512 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19772 2024-09-21T21:42:54.512 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.512 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:54.512 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.512 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:54.512 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273358+0000", 2024-09-21T21:42:54.512 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:54.512 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.512 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.512 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:54.512 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273358+0000", 2024-09-21T21:42:54.513 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:54.513 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.513 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.513 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:54.513 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273358+0000", 2024-09-21T21:42:54.513 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4499999999999998e-07 2024-09-21T21:42:54.513 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.513 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.513 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:54.513 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273360+0000", 2024-09-21T21:42:54.513 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9190000000000002e-06 2024-09-21T21:42:54.513 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.513 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.513 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:54.514 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273366+0000", 2024-09-21T21:42:54.514 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.891e-06 2024-09-21T21:42:54.514 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.514 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.514 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:54.514 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273371+0000", 2024-09-21T21:42:54.514 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1880000000000001e-06 2024-09-21T21:42:54.514 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.514 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.514 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:54.514 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278237+0000", 2024-09-21T21:42:54.514 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.004865472 2024-09-21T21:42:54.515 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.515 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.515 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:54.515 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278240+0000", 2024-09-21T21:42:54.515 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9550000000000001e-06 2024-09-21T21:42:54.515 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.515 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.515 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:54.515 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247949+0000", 2024-09-21T21:42:54.515 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.969709127 2024-09-21T21:42:54.515 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.515 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.515 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:54.515 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247990+0000", 2024-09-21T21:42:54.515 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1202000000000003e-05 2024-09-21T21:42:54.516 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.516 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.516 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:54.516 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248028+0000", 2024-09-21T21:42:54.516 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8219000000000001e-05 2024-09-21T21:42:54.516 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:54.516 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:54.516 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:54.516 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.516 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.516 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19773 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6615040~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:54.516 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273390+0000", 2024-09-21T21:42:54.516 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 63.175644994000002, 2024-09-21T21:42:54.516 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97476086200000001, 2024-09-21T21:42:54.517 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:54.517 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:54.517 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:54.517 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:54.517 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:54.517 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:54.517 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19773 2024-09-21T21:42:54.517 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.517 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:54.517 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.517 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:54.517 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273390+0000", 2024-09-21T21:42:54.517 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:54.518 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.518 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.518 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:54.518 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273390+0000", 2024-09-21T21:42:54.518 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:54.518 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.518 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.518 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:54.518 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273390+0000", 2024-09-21T21:42:54.518 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9999999999999999e-07 2024-09-21T21:42:54.518 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.518 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.518 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:54.518 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273392+0000", 2024-09-21T21:42:54.518 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0640000000000001e-06 2024-09-21T21:42:54.519 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.519 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.519 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:54.519 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273397+0000", 2024-09-21T21:42:54.519 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.113e-06 2024-09-21T21:42:54.519 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.519 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.520 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:54.520 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273401+0000", 2024-09-21T21:42:54.520 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3119999999999998e-06 2024-09-21T21:42:54.520 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.520 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.520 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:54.520 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278271+0000", 2024-09-21T21:42:54.520 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048698880000000002 2024-09-21T21:42:54.520 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.520 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.520 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:54.520 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278278+0000", 2024-09-21T21:42:54.520 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1210000000000001e-06 2024-09-21T21:42:54.520 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.521 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.521 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:54.521 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248060+0000", 2024-09-21T21:42:54.521 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96978224999999996 2024-09-21T21:42:54.521 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.521 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.521 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:54.521 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248111+0000", 2024-09-21T21:42:54.521 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.096e-05 2024-09-21T21:42:54.521 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.521 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.521 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:54.521 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248151+0000", 2024-09-21T21:42:54.522 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9854000000000002e-05 2024-09-21T21:42:54.522 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:54.522 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:54.522 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:54.522 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.522 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.522 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19784 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6643712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:54.522 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273405+0000", 2024-09-21T21:42:54.522 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 63.175630273000003, 2024-09-21T21:42:54.522 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97491798900000004, 2024-09-21T21:42:54.522 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:54.522 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:54.522 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:54.522 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:54.523 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:54.523 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:54.523 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19784 2024-09-21T21:42:54.523 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.523 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:54.523 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.523 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:54.523 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273405+0000", 2024-09-21T21:42:54.523 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:54.523 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.523 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.523 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:54.523 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273405+0000", 2024-09-21T21:42:54.524 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:54.524 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.524 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.524 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:54.524 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273405+0000", 2024-09-21T21:42:54.524 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2100000000000001e-07 2024-09-21T21:42:54.524 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.524 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.524 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:54.524 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273406+0000", 2024-09-21T21:42:54.524 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2049999999999999e-06 2024-09-21T21:42:54.524 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.524 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.524 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:54.524 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273409+0000", 2024-09-21T21:42:54.525 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0599999999999999e-06 2024-09-21T21:42:54.525 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.525 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.525 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:54.525 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273412+0000", 2024-09-21T21:42:54.525 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0730000000000001e-06 2024-09-21T21:42:54.525 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.525 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.525 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:54.525 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278317+0000", 2024-09-21T21:42:54.525 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0049045520000000004 2024-09-21T21:42:54.525 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.525 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.525 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:54.526 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278321+0000", 2024-09-21T21:42:54.526 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6249999999999998e-06 2024-09-21T21:42:54.526 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.526 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.526 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:54.526 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248179+0000", 2024-09-21T21:42:54.526 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96985744500000004 2024-09-21T21:42:54.526 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.526 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.526 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:54.526 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248253+0000", 2024-09-21T21:42:54.526 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3829000000000001e-05 2024-09-21T21:42:54.526 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.526 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.527 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:54.527 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248323+0000", 2024-09-21T21:42:54.527 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9979000000000002e-05 2024-09-21T21:42:54.527 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:54.527 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:54.527 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:54.527 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.527 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.527 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19785 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6672384~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:54.527 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273415+0000", 2024-09-21T21:42:54.527 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 63.175619687000001, 2024-09-21T21:42:54.527 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97500777800000005, 2024-09-21T21:42:54.527 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:54.528 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:54.528 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:54.528 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:54.528 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:54.528 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:54.528 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19785 2024-09-21T21:42:54.528 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.528 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:54.528 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.528 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:54.528 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273415+0000", 2024-09-21T21:42:54.528 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:54.528 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.528 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.528 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:54.529 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273415+0000", 2024-09-21T21:42:54.529 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:54.529 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.529 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.529 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:54.529 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273415+0000", 2024-09-21T21:42:54.529 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0100000000000001e-07 2024-09-21T21:42:54.529 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.529 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.529 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:54.529 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273417+0000", 2024-09-21T21:42:54.529 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.114e-06 2024-09-21T21:42:54.529 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.529 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.529 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:54.530 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273420+0000", 2024-09-21T21:42:54.530 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6770000000000001e-06 2024-09-21T21:42:54.530 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.530 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.530 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:54.530 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273423+0000", 2024-09-21T21:42:54.530 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5450000000000002e-06 2024-09-21T21:42:54.530 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.530 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.530 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:54.530 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278355+0000", 2024-09-21T21:42:54.530 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0049318590000000002 2024-09-21T21:42:54.530 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.530 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.531 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:54.531 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278360+0000", 2024-09-21T21:42:54.531 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3079999999999998e-06 2024-09-21T21:42:54.531 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.531 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.531 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:54.531 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248349+0000", 2024-09-21T21:42:54.531 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96998909899999997 2024-09-21T21:42:54.531 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.531 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.531 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:54.531 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248387+0000", 2024-09-21T21:42:54.531 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7657000000000002e-05 2024-09-21T21:42:54.531 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.531 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.532 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:54.532 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248423+0000", 2024-09-21T21:42:54.532 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6318e-05 2024-09-21T21:42:54.532 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:54.532 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:54.532 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:54.532 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.532 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.532 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19842 3.d 3:b7fa2ca0:::rbd_data.112157a479c.000000000000004d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:54.532 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273426+0000", 2024-09-21T21:42:54.532 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 63.175608635000003, 2024-09-21T21:42:54.532 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.98136526800000001, 2024-09-21T21:42:54.532 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:54.532 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:54.532 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:54.533 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:54.533 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:54.533 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:54.533 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19842 2024-09-21T21:42:54.533 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.533 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:54.533 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.533 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:54.533 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273426+0000", 2024-09-21T21:42:54.533 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:54.533 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.533 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.533 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:54.533 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273426+0000", 2024-09-21T21:42:54.533 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:54.534 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.534 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.534 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:54.534 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273427+0000", 2024-09-21T21:42:54.534 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.72e-07 2024-09-21T21:42:54.534 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.534 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.534 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:54.534 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275836+0000", 2024-09-21T21:42:54.534 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00240909 2024-09-21T21:42:54.534 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.534 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.534 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:54.534 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275844+0000", 2024-09-21T21:42:54.535 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3750000000000003e-06 2024-09-21T21:42:54.535 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.535 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.535 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:54.535 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275849+0000", 2024-09-21T21:42:54.535 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0100000000000003e-06 2024-09-21T21:42:54.535 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.535 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.535 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:54.535 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278394+0000", 2024-09-21T21:42:54.535 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025449890000000001 2024-09-21T21:42:54.535 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.535 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.535 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:54.535 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278402+0000", 2024-09-21T21:42:54.536 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6359999999999994e-06 2024-09-21T21:42:54.536 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.536 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.536 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:54.536 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248454+0000", 2024-09-21T21:42:54.536 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97005259499999996 2024-09-21T21:42:54.536 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.536 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.536 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:54.536 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248576+0000", 2024-09-21T21:42:54.536 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000122127 2024-09-21T21:42:54.536 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.536 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.536 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 2", 2024-09-21T21:42:54.536 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248844+0000", 2024-09-21T21:42:54.537 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00026784699999999999 2024-09-21T21:42:54.537 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.537 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.537 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:54.537 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.252487+0000", 2024-09-21T21:42:54.537 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0036427920000000002 2024-09-21T21:42:54.537 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.537 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.537 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:54.537 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.254691+0000", 2024-09-21T21:42:54.537 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002203899 2024-09-21T21:42:54.537 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.537 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.537 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:54.538 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.254748+0000", 2024-09-21T21:42:54.538 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7163000000000001e-05 2024-09-21T21:42:54.538 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.538 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.538 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:54.538 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.254792+0000", 2024-09-21T21:42:54.538 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3473000000000002e-05 2024-09-21T21:42:54.538 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:54.538 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:54.538 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:54.538 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.538 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.538 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19843 3.d 3:b7fa2ca0:::rbd_data.112157a479c.000000000000004d:head [write 0~0] snapc 21=[21] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:42:54.538 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.275860+0000", 2024-09-21T21:42:54.538 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 63.173174735000003, 2024-09-21T21:42:54.539 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.98183651999999999, 2024-09-21T21:42:54.539 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:54.539 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:54.539 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:54.539 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:54.539 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:42:54.539 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:54.539 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19843 2024-09-21T21:42:54.539 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.539 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:54.539 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.539 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:54.539 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275860+0000", 2024-09-21T21:42:54.539 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:54.540 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.540 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.540 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:54.540 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275860+0000", 2024-09-21T21:42:54.540 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:54.540 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.540 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.540 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:54.540 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275869+0000", 2024-09-21T21:42:54.540 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6300000000000004e-06 2024-09-21T21:42:54.540 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.540 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.540 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:54.540 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275871+0000", 2024-09-21T21:42:54.540 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2060000000000001e-06 2024-09-21T21:42:54.541 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.541 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.541 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:54.541 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275876+0000", 2024-09-21T21:42:54.541 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8180000000000003e-06 2024-09-21T21:42:54.541 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.541 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.541 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:54.541 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275880+0000", 2024-09-21T21:42:54.541 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1829999999999996e-06 2024-09-21T21:42:54.541 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.541 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.541 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:54.541 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278416+0000", 2024-09-21T21:42:54.541 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025361149999999998 2024-09-21T21:42:54.542 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.542 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.542 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:54.542 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278419+0000", 2024-09-21T21:42:54.542 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.163e-06 2024-09-21T21:42:54.542 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.542 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.542 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:54.542 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.250249+0000", 2024-09-21T21:42:54.542 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97182984299999997 2024-09-21T21:42:54.542 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.542 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.542 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:54.542 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.250321+0000", 2024-09-21T21:42:54.543 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1656000000000006e-05 2024-09-21T21:42:54.543 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.543 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.543 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 2", 2024-09-21T21:42:54.543 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.250419+0000", 2024-09-21T21:42:54.543 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.8277999999999997e-05 2024-09-21T21:42:54.543 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.543 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.543 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:54.543 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.252516+0000", 2024-09-21T21:42:54.543 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002097224 2024-09-21T21:42:54.543 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.543 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.543 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:54.543 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.257643+0000", 2024-09-21T21:42:54.544 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0051262319999999997 2024-09-21T21:42:54.544 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.544 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.544 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:54.544 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.257673+0000", 2024-09-21T21:42:54.544 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0023e-05 2024-09-21T21:42:54.544 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.544 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.544 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:54.544 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.257697+0000", 2024-09-21T21:42:54.544 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4148999999999999e-05 2024-09-21T21:42:54.544 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:54.544 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:54.544 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:54.544 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.545 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.545 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5626.0:12334 3.3 3:c643a86c:::rbd_data.1121faebdfe3.0000000000000002:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:42:54.545 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:14.411628+0000", 2024-09-21T21:42:54.545 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 40.037406435999998, 2024-09-21T21:42:54.545 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0086947559999999, 2024-09-21T21:42:54.545 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:54.545 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:54.545 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:54.545 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:54.545 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5626", 2024-09-21T21:42:54.545 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:54.545 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 12334 2024-09-21T21:42:54.545 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.545 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:54.545 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.546 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:54.546 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411628+0000", 2024-09-21T21:42:54.546 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:54.546 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.546 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.546 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:54.546 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411628+0000", 2024-09-21T21:42:54.546 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:54.546 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.546 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.546 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:54.546 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411629+0000", 2024-09-21T21:42:54.546 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1200000000000002e-07 2024-09-21T21:42:54.546 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.547 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.547 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:54.547 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411633+0000", 2024-09-21T21:42:54.547 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1699999999999999e-06 2024-09-21T21:42:54.547 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.547 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.547 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:54.547 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411645+0000", 2024-09-21T21:42:54.547 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1328999999999999e-05 2024-09-21T21:42:54.547 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.547 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.547 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:54.547 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411652+0000", 2024-09-21T21:42:54.547 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9990000000000002e-06 2024-09-21T21:42:54.547 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.548 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.548 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:54.548 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411717+0000", 2024-09-21T21:42:54.548 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5519999999999996e-05 2024-09-21T21:42:54.548 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.548 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.548 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op must wait for map", 2024-09-21T21:42:54.548 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411725+0000", 2024-09-21T21:42:54.548 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2609999999999996e-06 2024-09-21T21:42:54.548 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.548 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.548 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:54.548 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415904+0000", 2024-09-21T21:42:54.548 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0041792749999999997 2024-09-21T21:42:54.548 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.549 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.549 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:54.549 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415913+0000", 2024-09-21T21:42:54.549 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.054e-06 2024-09-21T21:42:54.549 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.549 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.549 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:54.549 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420096+0000", 2024-09-21T21:42:54.549 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0041835670000001 2024-09-21T21:42:54.549 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.549 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.549 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:54.549 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420216+0000", 2024-09-21T21:42:54.549 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000120025 2024-09-21T21:42:54.549 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.550 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.550 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:54.550 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420323+0000", 2024-09-21T21:42:54.550 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010674400000000001 2024-09-21T21:42:54.550 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:54.550 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:54.550 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:54.550 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.550 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.550 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5626.0:12339 3.3 3:c643a86c:::rbd_data.1121faebdfe3.0000000000000002:head [read 4096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:42:54.550 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:14.411749+0000", 2024-09-21T21:42:54.550 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 40.037286250000001, 2024-09-21T21:42:54.550 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.008818056, 2024-09-21T21:42:54.551 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:54.551 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:54.551 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:54.551 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:54.551 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5626", 2024-09-21T21:42:54.551 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:54.551 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 12339 2024-09-21T21:42:54.551 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.551 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:54.551 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.551 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:54.551 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411749+0000", 2024-09-21T21:42:54.551 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:54.551 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.551 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.552 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:54.552 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411749+0000", 2024-09-21T21:42:54.552 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:54.552 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.552 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.552 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:54.552 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411749+0000", 2024-09-21T21:42:54.552 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0900000000000002e-07 2024-09-21T21:42:54.552 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.552 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.552 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:54.552 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411752+0000", 2024-09-21T21:42:54.552 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5610000000000001e-06 2024-09-21T21:42:54.552 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.553 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.553 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:54.553 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411758+0000", 2024-09-21T21:42:54.553 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1890000000000004e-06 2024-09-21T21:42:54.553 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.553 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.553 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:54.553 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411763+0000", 2024-09-21T21:42:54.553 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7589999999999997e-06 2024-09-21T21:42:54.553 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.553 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.553 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:54.553 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411803+0000", 2024-09-21T21:42:54.553 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9922999999999997e-05 2024-09-21T21:42:54.553 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.554 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.554 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting_for_map not empty", 2024-09-21T21:42:54.554 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411809+0000", 2024-09-21T21:42:54.554 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3779999999999998e-06 2024-09-21T21:42:54.554 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.554 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.554 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:54.554 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415937+0000", 2024-09-21T21:42:54.554 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0041276100000000003 2024-09-21T21:42:54.554 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.554 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.554 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:54.554 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415944+0000", 2024-09-21T21:42:54.554 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8329999999999994e-06 2024-09-21T21:42:54.555 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.555 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.555 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:54.555 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420411+0000", 2024-09-21T21:42:54.555 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0044667300000001 2024-09-21T21:42:54.555 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.555 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.555 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:54.555 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420475+0000", 2024-09-21T21:42:54.555 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4356000000000005e-05 2024-09-21T21:42:54.555 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.555 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.555 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:54.555 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420567+0000", 2024-09-21T21:42:54.555 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.1308000000000001e-05 2024-09-21T21:42:54.556 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:54.556 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:54.556 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:54.556 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.556 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.556 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5626.0:12345 3.3 3:c643a86c:::rbd_data.1121faebdfe3.0000000000000002:head [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:42:54.556 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:14.411831+0000", 2024-09-21T21:42:54.556 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 40.037204197000001, 2024-09-21T21:42:54.556 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0089163649999999, 2024-09-21T21:42:54.556 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:54.556 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:54.556 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:54.556 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:54.556 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5626", 2024-09-21T21:42:54.557 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:54.557 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 12345 2024-09-21T21:42:54.557 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.557 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:54.557 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.557 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:54.557 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411831+0000", 2024-09-21T21:42:54.557 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:54.557 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.557 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.557 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:54.557 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411831+0000", 2024-09-21T21:42:54.557 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:54.557 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.557 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.558 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:54.558 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411831+0000", 2024-09-21T21:42:54.558 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.08e-07 2024-09-21T21:42:54.558 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.558 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.558 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:54.558 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411833+0000", 2024-09-21T21:42:54.558 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3089999999999998e-06 2024-09-21T21:42:54.558 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.558 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.558 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:54.558 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411840+0000", 2024-09-21T21:42:54.558 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.472e-06 2024-09-21T21:42:54.558 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.559 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.559 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:54.559 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411845+0000", 2024-09-21T21:42:54.559 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7910000000000004e-06 2024-09-21T21:42:54.559 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.559 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.559 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:54.559 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411890+0000", 2024-09-21T21:42:54.559 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5763000000000002e-05 2024-09-21T21:42:54.559 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.559 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.559 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting_for_map not empty", 2024-09-21T21:42:54.559 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411896+0000", 2024-09-21T21:42:54.559 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0190000000000003e-06 2024-09-21T21:42:54.559 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.560 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.560 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:54.560 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415967+0000", 2024-09-21T21:42:54.560 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0040705749999999999 2024-09-21T21:42:54.560 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.560 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.560 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:54.560 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415974+0000", 2024-09-21T21:42:54.560 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3200000000000002e-06 2024-09-21T21:42:54.560 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.560 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.560 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:54.560 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420594+0000", 2024-09-21T21:42:54.560 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.004620093 2024-09-21T21:42:54.560 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.561 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.561 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:54.561 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420658+0000", 2024-09-21T21:42:54.561 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3678999999999998e-05 2024-09-21T21:42:54.561 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:54.561 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:54.561 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:54.561 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420747+0000", 2024-09-21T21:42:54.561 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8936000000000004e-05 2024-09-21T21:42:54.561 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:54.561 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:54.561 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:54.561 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:54.561 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:54.561 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:54.563 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T21:42:54.563 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:54.561+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:54.663 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:54.661+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:54.707 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:54.707 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:42:54.707 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-21T21:42:54.707 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:54.715 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T21:42:54.757 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 5, 3] out_osds: [0, 4] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-21T21:42:54.758 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T21:42:54.758 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-21T21:42:54.758 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-21T21:42:54.800 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:54.798+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:54.857 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:54.858 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:42:54.858 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:42:54.858 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:42:54.858 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:54.864 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:54.862+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:54.866 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T21:42:54.965 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:54.963+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:55.012 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:55.012 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-21T21:42:55.012 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-21T21:42:55.012 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:42:55.012 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.012 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6974 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5246976~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:55.012 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.922932+0000", 2024-09-21T21:42:55.012 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 85.086429089999996, 2024-09-21T21:42:55.012 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041773352999999999, 2024-09-21T21:42:55.013 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:55.013 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:55.013 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:55.013 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:55.013 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:55.013 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:55.013 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6974 2024-09-21T21:42:55.013 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.013 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:55.013 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.013 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:55.013 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922932+0000", 2024-09-21T21:42:55.013 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:55.013 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.014 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.014 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:55.014 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922932+0000", 2024-09-21T21:42:55.014 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:55.014 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.014 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.014 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:55.014 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922933+0000", 2024-09-21T21:42:55.014 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.46e-07 2024-09-21T21:42:55.014 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.014 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.014 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:55.014 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922940+0000", 2024-09-21T21:42:55.015 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6140000000000003e-06 2024-09-21T21:42:55.015 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.015 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.015 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:55.015 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922949+0000", 2024-09-21T21:42:55.015 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6899999999999998e-06 2024-09-21T21:42:55.015 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.015 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.015 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:55.015 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922955+0000", 2024-09-21T21:42:55.015 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5140000000000001e-06 2024-09-21T21:42:55.015 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.016 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.016 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:55.016 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964547+0000", 2024-09-21T21:42:55.016 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041592854999999998 2024-09-21T21:42:55.016 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.016 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.016 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:55.016 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964617+0000", 2024-09-21T21:42:55.016 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9058000000000002e-05 2024-09-21T21:42:55.016 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.016 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.016 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:55.017 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964706+0000", 2024-09-21T21:42:55.017 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9276000000000004e-05 2024-09-21T21:42:55.017 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.017 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:55.017 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.017 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.017 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.017 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6976 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5261312~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:55.017 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.922961+0000", 2024-09-21T21:42:55.017 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 85.086400045000005, 2024-09-21T21:42:55.017 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041863648000000003, 2024-09-21T21:42:55.017 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:55.017 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:55.017 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:55.018 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:55.018 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:55.018 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:55.018 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6976 2024-09-21T21:42:55.018 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.018 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:55.018 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.018 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:55.018 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922961+0000", 2024-09-21T21:42:55.018 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:55.018 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.018 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.018 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:55.019 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922961+0000", 2024-09-21T21:42:55.019 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:55.019 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.019 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.019 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:55.019 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922962+0000", 2024-09-21T21:42:55.019 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.89e-07 2024-09-21T21:42:55.019 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.019 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.019 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:55.019 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922964+0000", 2024-09-21T21:42:55.019 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1270000000000001e-06 2024-09-21T21:42:55.019 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.019 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.020 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:55.020 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922970+0000", 2024-09-21T21:42:55.020 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8030000000000002e-06 2024-09-21T21:42:55.020 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.020 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.020 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:55.020 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922979+0000", 2024-09-21T21:42:55.020 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5559999999999996e-06 2024-09-21T21:42:55.020 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.020 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.020 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:55.020 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964734+0000", 2024-09-21T21:42:55.020 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041754415000000003 2024-09-21T21:42:55.020 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.021 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.021 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:55.021 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964775+0000", 2024-09-21T21:42:55.021 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0806999999999997e-05 2024-09-21T21:42:55.021 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.021 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.021 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:55.021 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964825+0000", 2024-09-21T21:42:55.021 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0550999999999998e-05 2024-09-21T21:42:55.021 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.021 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:55.021 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.021 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.021 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.022 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6978 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5275648~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:55.022 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.922987+0000", 2024-09-21T21:42:55.022 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 85.086374259999999, 2024-09-21T21:42:55.022 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042043876000000001, 2024-09-21T21:42:55.022 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:55.022 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:55.022 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:55.022 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:55.022 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:55.022 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:55.022 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6978 2024-09-21T21:42:55.022 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.022 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:55.022 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.023 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:55.023 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922987+0000", 2024-09-21T21:42:55.023 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:55.023 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.023 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.023 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:55.023 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922987+0000", 2024-09-21T21:42:55.023 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:55.023 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.023 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.023 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:55.023 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922988+0000", 2024-09-21T21:42:55.023 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1199999999999998e-07 2024-09-21T21:42:55.024 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.024 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.024 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:55.024 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922990+0000", 2024-09-21T21:42:55.024 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.142e-06 2024-09-21T21:42:55.024 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.024 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.024 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:55.024 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922997+0000", 2024-09-21T21:42:55.024 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.199e-06 2024-09-21T21:42:55.024 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.024 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.024 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:55.024 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923004+0000", 2024-09-21T21:42:55.024 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3599999999999998e-06 2024-09-21T21:42:55.025 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.025 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.025 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:55.025 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964856+0000", 2024-09-21T21:42:55.025 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041851521000000003 2024-09-21T21:42:55.025 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.025 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.025 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:55.025 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964944+0000", 2024-09-21T21:42:55.025 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7859e-05 2024-09-21T21:42:55.025 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.025 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.025 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:55.025 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965031+0000", 2024-09-21T21:42:55.026 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7383e-05 2024-09-21T21:42:55.026 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.026 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:55.026 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.026 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.026 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.026 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6980 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5289984~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:55.026 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923011+0000", 2024-09-21T21:42:55.026 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 85.086350922999998, 2024-09-21T21:42:55.026 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042139337999999998, 2024-09-21T21:42:55.026 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:55.026 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:55.026 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:55.026 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:55.027 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:55.027 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:55.027 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6980 2024-09-21T21:42:55.027 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.027 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:55.027 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.027 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:55.027 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923011+0000", 2024-09-21T21:42:55.027 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:55.027 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.027 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.027 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:55.027 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923011+0000", 2024-09-21T21:42:55.027 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:55.028 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.028 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.028 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:55.028 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923011+0000", 2024-09-21T21:42:55.028 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6699999999999999e-07 2024-09-21T21:42:55.028 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.028 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.028 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:55.028 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923013+0000", 2024-09-21T21:42:55.028 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0279999999999999e-06 2024-09-21T21:42:55.028 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.028 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.028 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:55.028 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923018+0000", 2024-09-21T21:42:55.028 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4790000000000004e-06 2024-09-21T21:42:55.029 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.029 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.029 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:55.029 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923029+0000", 2024-09-21T21:42:55.029 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0601999999999999e-05 2024-09-21T21:42:55.029 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.029 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.029 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:55.029 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965059+0000", 2024-09-21T21:42:55.029 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042029947999999998 2024-09-21T21:42:55.029 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.029 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.029 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:55.029 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965100+0000", 2024-09-21T21:42:55.030 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1170000000000001e-05 2024-09-21T21:42:55.030 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.030 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.030 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:55.030 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965150+0000", 2024-09-21T21:42:55.030 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9743999999999997e-05 2024-09-21T21:42:55.030 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.030 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:55.030 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.030 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.030 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.030 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6982 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5304320~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:55.030 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923036+0000", 2024-09-21T21:42:55.030 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 85.086325102000004, 2024-09-21T21:42:55.031 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042225666000000002, 2024-09-21T21:42:55.031 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:55.031 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:55.031 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:55.031 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:55.031 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:55.031 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:55.031 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6982 2024-09-21T21:42:55.031 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.031 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:55.031 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.031 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:55.031 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923036+0000", 2024-09-21T21:42:55.032 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:55.032 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.032 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.032 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:55.032 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923036+0000", 2024-09-21T21:42:55.032 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:55.032 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.032 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.032 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:55.032 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923037+0000", 2024-09-21T21:42:55.032 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2800000000000002e-07 2024-09-21T21:42:55.032 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.032 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.032 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:55.033 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923039+0000", 2024-09-21T21:42:55.033 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1500000000000002e-06 2024-09-21T21:42:55.033 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.033 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.033 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:55.033 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923049+0000", 2024-09-21T21:42:55.033 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0236e-05 2024-09-21T21:42:55.033 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.033 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.033 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:55.033 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923056+0000", 2024-09-21T21:42:55.033 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2540000000000003e-06 2024-09-21T21:42:55.033 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.033 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.034 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:55.034 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965173+0000", 2024-09-21T21:42:55.034 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042116686 2024-09-21T21:42:55.034 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.034 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.034 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:55.034 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965215+0000", 2024-09-21T21:42:55.034 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1696000000000002e-05 2024-09-21T21:42:55.034 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.034 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.034 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:55.034 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965262+0000", 2024-09-21T21:42:55.034 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7216000000000003e-05 2024-09-21T21:42:55.034 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.035 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:55.035 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.035 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.035 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.035 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6984 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5318656~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:55.035 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923063+0000", 2024-09-21T21:42:55.035 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 85.086298447999994, 2024-09-21T21:42:55.035 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042318651999999998, 2024-09-21T21:42:55.035 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:55.035 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:55.035 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:55.035 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:55.035 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:55.035 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:55.036 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6984 2024-09-21T21:42:55.036 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.036 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:55.036 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.036 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:55.036 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923063+0000", 2024-09-21T21:42:55.036 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:55.036 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.036 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.036 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:55.036 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923063+0000", 2024-09-21T21:42:55.036 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:55.036 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.036 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.037 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:55.037 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923064+0000", 2024-09-21T21:42:55.037 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2e-07 2024-09-21T21:42:55.037 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.037 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.037 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:55.037 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923066+0000", 2024-09-21T21:42:55.037 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.057e-06 2024-09-21T21:42:55.037 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.037 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.037 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:55.037 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923071+0000", 2024-09-21T21:42:55.037 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8810000000000001e-06 2024-09-21T21:42:55.037 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.038 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.038 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:55.038 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923078+0000", 2024-09-21T21:42:55.038 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5479999999999998e-06 2024-09-21T21:42:55.038 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.038 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.038 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:55.038 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965288+0000", 2024-09-21T21:42:55.038 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042209576999999998 2024-09-21T21:42:55.038 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.038 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.038 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:55.038 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965332+0000", 2024-09-21T21:42:55.038 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4366999999999997e-05 2024-09-21T21:42:55.039 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.039 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.039 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:55.039 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965382+0000", 2024-09-21T21:42:55.039 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9802000000000002e-05 2024-09-21T21:42:55.039 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.039 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:55.039 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.039 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.039 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.039 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6987 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5332992~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:55.039 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923085+0000", 2024-09-21T21:42:55.039 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 85.086276923, 2024-09-21T21:42:55.039 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042412103999999999, 2024-09-21T21:42:55.040 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:55.040 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:55.040 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:55.040 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:55.040 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:55.040 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:55.040 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6987 2024-09-21T21:42:55.040 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.040 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:55.040 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.040 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:55.040 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923085+0000", 2024-09-21T21:42:55.040 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:55.041 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.041 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.041 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:55.041 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923085+0000", 2024-09-21T21:42:55.041 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:55.041 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.041 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.041 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:55.041 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923085+0000", 2024-09-21T21:42:55.041 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.89e-07 2024-09-21T21:42:55.041 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.041 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.041 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:55.042 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923087+0000", 2024-09-21T21:42:55.042 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0810000000000001e-06 2024-09-21T21:42:55.042 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.042 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.042 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:55.042 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923098+0000", 2024-09-21T21:42:55.042 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1299e-05 2024-09-21T21:42:55.042 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.042 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.042 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:55.042 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923104+0000", 2024-09-21T21:42:55.042 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9100000000000002e-06 2024-09-21T21:42:55.042 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.042 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.043 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:55.043 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965404+0000", 2024-09-21T21:42:55.043 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042299622000000002 2024-09-21T21:42:55.043 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.043 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.043 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:55.043 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965445+0000", 2024-09-21T21:42:55.043 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0701000000000002e-05 2024-09-21T21:42:55.043 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.043 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.043 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:55.043 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965497+0000", 2024-09-21T21:42:55.043 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2101999999999997e-05 2024-09-21T21:42:55.043 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.044 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:55.044 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.044 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.044 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.044 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6989 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5347328~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:55.044 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923111+0000", 2024-09-21T21:42:55.044 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 85.086250582999995, 2024-09-21T21:42:55.044 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042505188999999999, 2024-09-21T21:42:55.044 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:55.044 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:55.044 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:55.044 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:55.044 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:55.044 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:55.045 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6989 2024-09-21T21:42:55.045 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.045 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:55.045 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.045 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:55.045 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923111+0000", 2024-09-21T21:42:55.045 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:55.045 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.045 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.045 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:55.045 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923111+0000", 2024-09-21T21:42:55.045 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:55.045 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.045 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.046 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:55.046 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923111+0000", 2024-09-21T21:42:55.046 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4499999999999997e-07 2024-09-21T21:42:55.046 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.046 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.046 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:55.046 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923114+0000", 2024-09-21T21:42:55.046 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1569999999999998e-06 2024-09-21T21:42:55.046 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.046 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.046 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:55.046 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923120+0000", 2024-09-21T21:42:55.046 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1759999999999998e-06 2024-09-21T21:42:55.046 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.047 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.047 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:55.047 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923125+0000", 2024-09-21T21:42:55.047 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2560000000000002e-06 2024-09-21T21:42:55.047 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.047 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.047 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:55.047 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965523+0000", 2024-09-21T21:42:55.047 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042398094999999997 2024-09-21T21:42:55.047 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.047 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.047 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:55.047 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965566+0000", 2024-09-21T21:42:55.048 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3393999999999998e-05 2024-09-21T21:42:55.048 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.048 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.048 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:55.048 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965616+0000", 2024-09-21T21:42:55.048 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9666000000000002e-05 2024-09-21T21:42:55.048 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.048 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:55.048 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.048 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.048 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.048 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6991 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5361664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:55.048 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923132+0000", 2024-09-21T21:42:55.048 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 85.086229294000006, 2024-09-21T21:42:55.048 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042600779999999998, 2024-09-21T21:42:55.049 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:55.049 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:55.049 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:55.049 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:55.049 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:55.049 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:55.049 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6991 2024-09-21T21:42:55.049 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.049 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:55.049 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.049 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:55.049 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923132+0000", 2024-09-21T21:42:55.049 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:55.049 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.050 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.050 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:55.050 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923132+0000", 2024-09-21T21:42:55.050 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:55.050 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.050 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.050 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:55.050 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923132+0000", 2024-09-21T21:42:55.050 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4200000000000002e-07 2024-09-21T21:42:55.050 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.050 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.050 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:55.050 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923134+0000", 2024-09-21T21:42:55.050 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2270000000000001e-06 2024-09-21T21:42:55.050 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.051 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.051 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:55.051 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923137+0000", 2024-09-21T21:42:55.051 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1930000000000002e-06 2024-09-21T21:42:55.051 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.051 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.051 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:55.051 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923139+0000", 2024-09-21T21:42:55.051 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4710000000000001e-06 2024-09-21T21:42:55.051 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.051 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.051 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:55.051 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965639+0000", 2024-09-21T21:42:55.051 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.04249965 2024-09-21T21:42:55.051 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.052 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.052 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:55.052 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965679+0000", 2024-09-21T21:42:55.052 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9852999999999999e-05 2024-09-21T21:42:55.052 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.052 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.052 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:55.052 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965733+0000", 2024-09-21T21:42:55.052 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4144000000000002e-05 2024-09-21T21:42:55.052 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.052 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:55.052 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.052 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.052 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.053 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6993 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5384192~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:55.053 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923143+0000", 2024-09-21T21:42:55.053 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 85.086218500000001, 2024-09-21T21:42:55.053 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042708182999999997, 2024-09-21T21:42:55.053 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:55.053 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:55.053 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:55.053 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:55.053 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:55.053 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:55.053 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6993 2024-09-21T21:42:55.053 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.053 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:55.053 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.053 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:55.054 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923143+0000", 2024-09-21T21:42:55.054 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:55.054 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.054 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.054 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:55.054 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923143+0000", 2024-09-21T21:42:55.054 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:55.054 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.054 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.054 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:55.054 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923143+0000", 2024-09-21T21:42:55.054 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0800000000000001e-07 2024-09-21T21:42:55.054 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.054 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.055 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:55.055 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923144+0000", 2024-09-21T21:42:55.055 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.009e-06 2024-09-21T21:42:55.055 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.055 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.055 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:55.055 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923148+0000", 2024-09-21T21:42:55.055 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.737e-06 2024-09-21T21:42:55.055 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.055 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.055 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:55.055 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923150+0000", 2024-09-21T21:42:55.055 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4669999999999998e-06 2024-09-21T21:42:55.056 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.056 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.056 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:55.056 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965760+0000", 2024-09-21T21:42:55.056 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042609399999999999 2024-09-21T21:42:55.056 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.056 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.056 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:55.056 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965801+0000", 2024-09-21T21:42:55.056 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1399999999999997e-05 2024-09-21T21:42:55.056 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.056 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.056 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:55.056 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965851+0000", 2024-09-21T21:42:55.057 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9962e-05 2024-09-21T21:42:55.057 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.057 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:55.057 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.057 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.057 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.057 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6995 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5398528~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:55.057 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923153+0000", 2024-09-21T21:42:55.057 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 85.086208135999996, 2024-09-21T21:42:55.057 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042828034000000001, 2024-09-21T21:42:55.057 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:55.057 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:55.057 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:55.057 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:55.058 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:55.058 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:55.058 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6995 2024-09-21T21:42:55.058 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.058 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:55.058 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.058 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:55.058 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923153+0000", 2024-09-21T21:42:55.058 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:55.058 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.058 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.058 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:55.058 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923153+0000", 2024-09-21T21:42:55.058 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:55.059 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.059 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.059 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:55.059 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923154+0000", 2024-09-21T21:42:55.059 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2600000000000001e-07 2024-09-21T21:42:55.059 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.059 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.059 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:55.059 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923158+0000", 2024-09-21T21:42:55.059 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0559999999999998e-06 2024-09-21T21:42:55.059 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.059 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.059 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:55.059 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923161+0000", 2024-09-21T21:42:55.059 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6519999999999999e-06 2024-09-21T21:42:55.060 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.060 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.060 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:55.060 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923170+0000", 2024-09-21T21:42:55.060 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5720000000000003e-06 2024-09-21T21:42:55.060 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.060 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.060 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:55.060 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965887+0000", 2024-09-21T21:42:55.060 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042716863000000001 2024-09-21T21:42:55.060 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.060 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.060 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:55.060 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965927+0000", 2024-09-21T21:42:55.060 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0346999999999998e-05 2024-09-21T21:42:55.061 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.061 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.061 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:55.061 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965981+0000", 2024-09-21T21:42:55.061 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4317999999999997e-05 2024-09-21T21:42:55.061 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.061 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:55.061 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.061 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.061 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.061 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6997 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5412864~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:55.061 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923174+0000", 2024-09-21T21:42:55.061 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 85.086187980999995, 2024-09-21T21:42:55.061 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042928296999999997, 2024-09-21T21:42:55.062 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:55.062 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:55.062 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:55.062 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:55.062 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:55.062 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:55.062 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6997 2024-09-21T21:42:55.062 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.062 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:55.062 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.062 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:55.062 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923174+0000", 2024-09-21T21:42:55.062 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:55.062 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.063 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.063 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:55.063 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923174+0000", 2024-09-21T21:42:55.063 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:55.063 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.063 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.063 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:55.063 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923174+0000", 2024-09-21T21:42:55.063 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.67e-07 2024-09-21T21:42:55.063 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.063 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.063 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:55.063 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923178+0000", 2024-09-21T21:42:55.063 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1330000000000001e-06 2024-09-21T21:42:55.064 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.064 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.064 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:55.064 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923182+0000", 2024-09-21T21:42:55.064 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.794e-06 2024-09-21T21:42:55.064 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.064 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.064 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:55.064 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923184+0000", 2024-09-21T21:42:55.064 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.435e-06 2024-09-21T21:42:55.064 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.064 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.064 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:55.064 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966009+0000", 2024-09-21T21:42:55.065 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042824327000000002 2024-09-21T21:42:55.065 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.065 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.065 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:55.065 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966052+0000", 2024-09-21T21:42:55.065 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3034000000000002e-05 2024-09-21T21:42:55.065 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.065 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.065 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:55.065 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966102+0000", 2024-09-21T21:42:55.065 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0306999999999998e-05 2024-09-21T21:42:55.065 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.065 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:55.065 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.065 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.066 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.066 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6999 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5427200~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:55.066 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923187+0000", 2024-09-21T21:42:55.066 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 85.086174252999996, 2024-09-21T21:42:55.066 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043104267000000002, 2024-09-21T21:42:55.066 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:55.066 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:55.066 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:55.066 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:55.066 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:55.066 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:55.066 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6999 2024-09-21T21:42:55.066 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.066 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:55.067 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.067 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:55.067 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923187+0000", 2024-09-21T21:42:55.067 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:55.067 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.067 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.067 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:55.067 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923187+0000", 2024-09-21T21:42:55.067 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:55.067 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.067 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.067 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:55.067 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923188+0000", 2024-09-21T21:42:55.067 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3999999999999998e-07 2024-09-21T21:42:55.068 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.068 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.068 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:55.068 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923189+0000", 2024-09-21T21:42:55.068 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4169999999999999e-06 2024-09-21T21:42:55.068 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.068 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.068 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:55.068 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923194+0000", 2024-09-21T21:42:55.068 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1819999999999997e-06 2024-09-21T21:42:55.068 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.068 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.068 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:55.068 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923198+0000", 2024-09-21T21:42:55.069 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.143e-06 2024-09-21T21:42:55.069 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.069 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.069 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:55.069 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966135+0000", 2024-09-21T21:42:55.069 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042936297999999998 2024-09-21T21:42:55.069 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.069 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.069 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:55.069 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966204+0000", 2024-09-21T21:42:55.069 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8989e-05 2024-09-21T21:42:55.069 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.069 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.069 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:55.069 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966292+0000", 2024-09-21T21:42:55.070 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7997999999999993e-05 2024-09-21T21:42:55.070 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.070 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:55.070 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.070 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.070 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.070 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7001 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5441536~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:55.070 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923211+0000", 2024-09-21T21:42:55.070 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 85.086150462000006, 2024-09-21T21:42:55.070 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043196518000000003, 2024-09-21T21:42:55.070 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:55.070 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:55.070 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:55.071 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:55.071 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:55.071 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:55.071 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7001 2024-09-21T21:42:55.071 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.071 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:55.071 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.071 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:55.071 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923211+0000", 2024-09-21T21:42:55.071 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:55.071 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.071 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.071 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:55.071 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923211+0000", 2024-09-21T21:42:55.072 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:55.072 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.072 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.072 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:55.072 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923212+0000", 2024-09-21T21:42:55.072 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3900000000000005e-07 2024-09-21T21:42:55.072 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.072 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.072 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:55.072 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923214+0000", 2024-09-21T21:42:55.072 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9970000000000001e-06 2024-09-21T21:42:55.072 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.072 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.072 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:55.072 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923219+0000", 2024-09-21T21:42:55.073 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4310000000000002e-06 2024-09-21T21:42:55.073 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.073 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.073 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:55.073 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923224+0000", 2024-09-21T21:42:55.073 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6410000000000001e-06 2024-09-21T21:42:55.073 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.073 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.073 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:55.073 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966318+0000", 2024-09-21T21:42:55.073 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043093842 2024-09-21T21:42:55.073 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.073 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.073 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:55.074 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966359+0000", 2024-09-21T21:42:55.074 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1260999999999997e-05 2024-09-21T21:42:55.074 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.074 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.074 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:55.074 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966408+0000", 2024-09-21T21:42:55.074 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8807000000000002e-05 2024-09-21T21:42:55.074 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.074 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:55.074 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.074 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.074 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.074 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7002 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5455872~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:55.074 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923229+0000", 2024-09-21T21:42:55.075 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 85.086132051999996, 2024-09-21T21:42:55.075 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043364568999999999, 2024-09-21T21:42:55.075 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:55.075 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:55.075 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:55.075 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:55.075 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:55.075 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:55.075 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7002 2024-09-21T21:42:55.075 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.075 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:55.075 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.075 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:55.075 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923229+0000", 2024-09-21T21:42:55.076 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:55.076 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.076 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.076 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:55.076 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923229+0000", 2024-09-21T21:42:55.076 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:55.076 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.076 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.076 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:55.076 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923230+0000", 2024-09-21T21:42:55.076 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4700000000000002e-07 2024-09-21T21:42:55.076 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.076 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.076 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:55.076 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923232+0000", 2024-09-21T21:42:55.077 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0719999999999998e-06 2024-09-21T21:42:55.077 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.077 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.077 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:55.077 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923237+0000", 2024-09-21T21:42:55.077 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2290000000000002e-06 2024-09-21T21:42:55.077 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.077 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.077 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:55.077 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923245+0000", 2024-09-21T21:42:55.077 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2959999999999999e-06 2024-09-21T21:42:55.077 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.077 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.078 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:55.078 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966438+0000", 2024-09-21T21:42:55.078 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043193027000000002 2024-09-21T21:42:55.078 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.078 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.078 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:55.078 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966506+0000", 2024-09-21T21:42:55.078 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7707999999999996e-05 2024-09-21T21:42:55.078 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.078 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.078 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:55.078 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966594+0000", 2024-09-21T21:42:55.078 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.789e-05 2024-09-21T21:42:55.079 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.079 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:55.079 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.079 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.079 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.079 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7005 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5470208~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:55.079 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923254+0000", 2024-09-21T21:42:55.079 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 85.086107820999999, 2024-09-21T21:42:55.079 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043457176, 2024-09-21T21:42:55.079 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:55.079 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:55.079 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:55.079 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:55.079 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:55.080 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:55.080 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7005 2024-09-21T21:42:55.080 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.080 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:55.080 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.080 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:55.080 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923254+0000", 2024-09-21T21:42:55.080 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:55.080 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.080 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.080 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:55.080 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923254+0000", 2024-09-21T21:42:55.080 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:55.080 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.080 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.081 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:55.081 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923254+0000", 2024-09-21T21:42:55.081 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4700000000000002e-07 2024-09-21T21:42:55.081 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.081 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.081 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:55.081 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923256+0000", 2024-09-21T21:42:55.081 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.007e-06 2024-09-21T21:42:55.081 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.081 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.081 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:55.081 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923266+0000", 2024-09-21T21:42:55.081 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.4700000000000008e-06 2024-09-21T21:42:55.081 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.081 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.082 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:55.082 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923271+0000", 2024-09-21T21:42:55.082 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2809999999999999e-06 2024-09-21T21:42:55.082 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.082 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.082 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:55.082 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966619+0000", 2024-09-21T21:42:55.082 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043347951000000003 2024-09-21T21:42:55.082 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.082 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.082 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:55.082 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966661+0000", 2024-09-21T21:42:55.082 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2614999999999998e-05 2024-09-21T21:42:55.082 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.082 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.083 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:55.083 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966711+0000", 2024-09-21T21:42:55.083 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9505000000000001e-05 2024-09-21T21:42:55.083 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.083 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:55.083 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.083 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.083 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.083 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7008 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5484544~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:55.083 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923278+0000", 2024-09-21T21:42:55.083 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 85.086083173999995, 2024-09-21T21:42:55.084 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043631071, 2024-09-21T21:42:55.084 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:55.084 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:55.084 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:55.084 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:55.084 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:55.084 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:55.084 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7008 2024-09-21T21:42:55.084 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.084 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:55.084 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.084 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:55.084 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923278+0000", 2024-09-21T21:42:55.084 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:55.084 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.085 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.085 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:55.085 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923278+0000", 2024-09-21T21:42:55.085 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:55.085 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.085 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.085 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:55.085 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923279+0000", 2024-09-21T21:42:55.085 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.77e-07 2024-09-21T21:42:55.085 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.085 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.085 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:55.085 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923281+0000", 2024-09-21T21:42:55.085 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0219999999999999e-06 2024-09-21T21:42:55.086 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.086 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.086 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:55.086 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923287+0000", 2024-09-21T21:42:55.086 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7799999999999997e-06 2024-09-21T21:42:55.086 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.086 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.086 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:55.086 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923291+0000", 2024-09-21T21:42:55.086 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1699999999999999e-06 2024-09-21T21:42:55.086 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.086 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.086 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:55.087 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966742+0000", 2024-09-21T21:42:55.087 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043451344000000003 2024-09-21T21:42:55.087 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.087 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.087 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:55.087 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966809+0000", 2024-09-21T21:42:55.087 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7070999999999995e-05 2024-09-21T21:42:55.087 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.087 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.087 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:55.087 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966909+0000", 2024-09-21T21:42:55.087 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000100307 2024-09-21T21:42:55.087 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.087 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:55.087 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.088 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.088 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.088 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7010 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5498880~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:55.088 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923301+0000", 2024-09-21T21:42:55.088 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 85.086060278999994, 2024-09-21T21:42:55.088 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043717830999999999, 2024-09-21T21:42:55.088 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:55.088 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:55.088 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:55.088 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:55.088 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:55.088 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:55.088 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7010 2024-09-21T21:42:55.088 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.088 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:55.089 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.089 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:55.089 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923301+0000", 2024-09-21T21:42:55.089 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:55.089 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.089 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.089 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:55.089 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923301+0000", 2024-09-21T21:42:55.089 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:55.089 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.089 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.089 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:55.089 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923302+0000", 2024-09-21T21:42:55.089 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.41e-07 2024-09-21T21:42:55.090 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.090 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.090 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:55.090 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923304+0000", 2024-09-21T21:42:55.090 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0990000000000001e-06 2024-09-21T21:42:55.090 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.090 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.090 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:55.090 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923310+0000", 2024-09-21T21:42:55.090 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3860000000000004e-06 2024-09-21T21:42:55.090 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.090 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.090 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:55.090 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923315+0000", 2024-09-21T21:42:55.091 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8459999999999999e-06 2024-09-21T21:42:55.091 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.091 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.091 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:55.091 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966938+0000", 2024-09-21T21:42:55.091 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043622693999999997 2024-09-21T21:42:55.091 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.091 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.091 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:55.091 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966975+0000", 2024-09-21T21:42:55.091 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7608999999999999e-05 2024-09-21T21:42:55.091 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.091 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.091 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:55.092 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.967019+0000", 2024-09-21T21:42:55.092 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3856000000000002e-05 2024-09-21T21:42:55.092 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.092 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:55.092 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.092 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.092 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.092 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7354 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 2107392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:42:55.092 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.936932+0000", 2024-09-21T21:42:55.092 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 85.072429424000006, 2024-09-21T21:42:55.092 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041607708, 2024-09-21T21:42:55.092 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:55.092 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:55.092 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:55.092 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:55.093 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:42:55.093 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:42:55.093 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7354 2024-09-21T21:42:55.093 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.093 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:55.093 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.093 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:55.093 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.936932+0000", 2024-09-21T21:42:55.093 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:55.093 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.093 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.093 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:55.093 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.936932+0000", 2024-09-21T21:42:55.093 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:55.093 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.094 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.094 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:55.094 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.937276+0000", 2024-09-21T21:42:55.094 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00034344200000000001 2024-09-21T21:42:55.094 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.094 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.094 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:55.094 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.937279+0000", 2024-09-21T21:42:55.094 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.844e-06 2024-09-21T21:42:55.094 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.094 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.094 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:55.094 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.937288+0000", 2024-09-21T21:42:55.094 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2409999999999995e-06 2024-09-21T21:42:55.095 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.095 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.095 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:55.095 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.937293+0000", 2024-09-21T21:42:55.095 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6559999999999997e-06 2024-09-21T21:42:55.095 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.095 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.095 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:55.095 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.978448+0000", 2024-09-21T21:42:55.095 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041154621000000002 2024-09-21T21:42:55.095 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.095 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.095 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:55.096 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.978487+0000", 2024-09-21T21:42:55.096 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9254000000000001e-05 2024-09-21T21:42:55.096 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.096 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.096 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:55.096 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.978540+0000", 2024-09-21T21:42:55.096 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2649999999999999e-05 2024-09-21T21:42:55.096 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.096 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:55.096 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.096 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.096 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.096 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5215.0:1054 3.9 3:946d2401:::rbd_data.1121524d2522.000000000000000b:head [set-alloc-hint object_size 16777216 write_size 16777216,write 6615040~59392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-21T21:42:55.096 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:38.148979+0000", 2024-09-21T21:42:55.096 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 76.860382457, 2024-09-21T21:42:55.097 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047870388999999999, 2024-09-21T21:42:55.097 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:55.097 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:55.097 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:55.097 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:55.097 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5215", 2024-09-21T21:42:55.097 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1580282168", 2024-09-21T21:42:55.097 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1054 2024-09-21T21:42:55.097 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.097 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:55.097 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.097 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:55.097 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.148979+0000", 2024-09-21T21:42:55.097 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:55.097 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.098 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.098 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:55.098 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.148979+0000", 2024-09-21T21:42:55.098 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:55.098 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.098 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.098 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:55.098 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.148980+0000", 2024-09-21T21:42:55.098 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5799999999999999e-07 2024-09-21T21:42:55.098 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.098 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.098 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:55.098 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.149039+0000", 2024-09-21T21:42:55.098 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9151000000000003e-05 2024-09-21T21:42:55.099 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.099 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.099 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:55.099 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.149047+0000", 2024-09-21T21:42:55.099 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1000000000000004e-06 2024-09-21T21:42:55.099 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.099 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.099 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:55.099 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.149052+0000", 2024-09-21T21:42:55.099 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5959999999999997e-06 2024-09-21T21:42:55.099 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.099 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.099 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:55.099 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.195288+0000", 2024-09-21T21:42:55.099 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.046236248000000001 2024-09-21T21:42:55.100 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.100 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.100 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:55.100 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.195334+0000", 2024-09-21T21:42:55.101 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6575e-05 2024-09-21T21:42:55.101 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.101 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.101 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 2", 2024-09-21T21:42:55.101 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.195406+0000", 2024-09-21T21:42:55.102 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1715e-05 2024-09-21T21:42:55.102 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.102 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.102 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:55.102 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.196018+0000", 2024-09-21T21:42:55.102 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00061218999999999998 2024-09-21T21:42:55.102 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.102 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.102 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:55.102 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.196794+0000", 2024-09-21T21:42:55.102 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000775814 2024-09-21T21:42:55.102 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.102 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.103 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:55.103 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.196827+0000", 2024-09-21T21:42:55.103 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2419000000000003e-05 2024-09-21T21:42:55.103 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.103 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.103 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:55.103 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.196849+0000", 2024-09-21T21:42:55.103 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2923000000000001e-05 2024-09-21T21:42:55.103 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.103 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:55.103 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.103 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.103 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:55.103 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:55.106 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T21:42:55.106 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:55.103+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:55.106 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:42:55.106 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:55.107 INFO:teuthology.orchestra.run.smithi081.stdout: "epoch": 132, 2024-09-21T21:42:55.107 INFO:teuthology.orchestra.run.smithi081.stdout: "fsid": "a0fdcfc2-ddd2-4e50-81bb-876d940f3578", 2024-09-21T21:42:55.107 INFO:teuthology.orchestra.run.smithi081.stdout: "created": "2024-09-21T21:40:07.413917+0000", 2024-09-21T21:42:55.107 INFO:teuthology.orchestra.run.smithi081.stdout: "modified": "2024-09-21T21:42:54.229880+0000", 2024-09-21T21:42:55.107 INFO:teuthology.orchestra.run.smithi081.stdout: "last_up_change": "2024-09-21T21:40:18.340430+0000", 2024-09-21T21:42:55.107 INFO:teuthology.orchestra.run.smithi081.stdout: "last_in_change": "2024-09-21T21:42:11.387597+0000", 2024-09-21T21:42:55.107 INFO:teuthology.orchestra.run.smithi081.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-21T21:42:55.107 INFO:teuthology.orchestra.run.smithi081.stdout: "flags_num": 5804032, 2024-09-21T21:42:55.107 INFO:teuthology.orchestra.run.smithi081.stdout: "flags_set": [ 2024-09-21T21:42:55.107 INFO:teuthology.orchestra.run.smithi081.stdout: "nodeep-scrub", 2024-09-21T21:42:55.107 INFO:teuthology.orchestra.run.smithi081.stdout: "pglog_hardlimit", 2024-09-21T21:42:55.107 INFO:teuthology.orchestra.run.smithi081.stdout: "purged_snapdirs", 2024-09-21T21:42:55.107 INFO:teuthology.orchestra.run.smithi081.stdout: "recovery_deletes", 2024-09-21T21:42:55.108 INFO:teuthology.orchestra.run.smithi081.stdout: "sortbitwise" 2024-09-21T21:42:55.108 INFO:teuthology.orchestra.run.smithi081.stdout: ], 2024-09-21T21:42:55.108 INFO:teuthology.orchestra.run.smithi081.stdout: "crush_version": 5, 2024-09-21T21:42:55.108 INFO:teuthology.orchestra.run.smithi081.stdout: "full_ratio": 0.94999998807907104, 2024-09-21T21:42:55.108 INFO:teuthology.orchestra.run.smithi081.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-21T21:42:55.108 INFO:teuthology.orchestra.run.smithi081.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-21T21:42:55.108 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster_snapshot": "", 2024-09-21T21:42:55.108 INFO:teuthology.orchestra.run.smithi081.stdout: "pool_max": 3, 2024-09-21T21:42:55.108 INFO:teuthology.orchestra.run.smithi081.stdout: "max_osd": 6, 2024-09-21T21:42:55.108 INFO:teuthology.orchestra.run.smithi081.stdout: "require_min_compat_client": "luminous", 2024-09-21T21:42:55.108 INFO:teuthology.orchestra.run.smithi081.stdout: "min_compat_client": "luminous", 2024-09-21T21:42:55.108 INFO:teuthology.orchestra.run.smithi081.stdout: "require_osd_release": "squid", 2024-09-21T21:42:55.108 INFO:teuthology.orchestra.run.smithi081.stdout: "allow_crimson": false, 2024-09-21T21:42:55.108 INFO:teuthology.orchestra.run.smithi081.stdout: "pools": [ 2024-09-21T21:42:55.108 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:55.109 INFO:teuthology.orchestra.run.smithi081.stdout: "pool": 1, 2024-09-21T21:42:55.109 INFO:teuthology.orchestra.run.smithi081.stdout: "pool_name": ".mgr", 2024-09-21T21:42:55.109 INFO:teuthology.orchestra.run.smithi081.stdout: "create_time": "2024-09-21T21:40:18.321328+0000", 2024-09-21T21:42:55.109 INFO:teuthology.orchestra.run.smithi081.stdout: "flags": 1, 2024-09-21T21:42:55.109 INFO:teuthology.orchestra.run.smithi081.stdout: "flags_names": "hashpspool", 2024-09-21T21:42:55.109 INFO:teuthology.orchestra.run.smithi081.stdout: "type": 1, 2024-09-21T21:42:55.109 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 2, 2024-09-21T21:42:55.109 INFO:teuthology.orchestra.run.smithi081.stdout: "min_size": 1, 2024-09-21T21:42:55.109 INFO:teuthology.orchestra.run.smithi081.stdout: "crush_rule": 0, 2024-09-21T21:42:55.109 INFO:teuthology.orchestra.run.smithi081.stdout: "peering_crush_bucket_count": 0, 2024-09-21T21:42:55.109 INFO:teuthology.orchestra.run.smithi081.stdout: "peering_crush_bucket_target": 0, 2024-09-21T21:42:55.109 INFO:teuthology.orchestra.run.smithi081.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T21:42:55.109 INFO:teuthology.orchestra.run.smithi081.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T21:42:55.110 INFO:teuthology.orchestra.run.smithi081.stdout: "is_stretch_pool": false, 2024-09-21T21:42:55.110 INFO:teuthology.orchestra.run.smithi081.stdout: "object_hash": 2, 2024-09-21T21:42:55.110 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_autoscale_mode": "off", 2024-09-21T21:42:55.110 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_num": 1, 2024-09-21T21:42:55.110 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_placement_num": 1, 2024-09-21T21:42:55.110 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_placement_num_target": 1, 2024-09-21T21:42:55.110 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_num_target": 1, 2024-09-21T21:42:55.110 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_num_pending": 1, 2024-09-21T21:42:55.110 INFO:teuthology.orchestra.run.smithi081.stdout: "last_pg_merge_meta": { 2024-09-21T21:42:55.110 INFO:teuthology.orchestra.run.smithi081.stdout: "source_pgid": "0.0", 2024-09-21T21:42:55.110 INFO:teuthology.orchestra.run.smithi081.stdout: "ready_epoch": 0, 2024-09-21T21:42:55.110 INFO:teuthology.orchestra.run.smithi081.stdout: "last_epoch_started": 0, 2024-09-21T21:42:55.110 INFO:teuthology.orchestra.run.smithi081.stdout: "last_epoch_clean": 0, 2024-09-21T21:42:55.111 INFO:teuthology.orchestra.run.smithi081.stdout: "source_version": "0'0", 2024-09-21T21:42:55.111 INFO:teuthology.orchestra.run.smithi081.stdout: "target_version": "0'0" 2024-09-21T21:42:55.111 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:55.111 INFO:teuthology.orchestra.run.smithi081.stdout: "last_change": "15", 2024-09-21T21:42:55.111 INFO:teuthology.orchestra.run.smithi081.stdout: "last_force_op_resend": "0", 2024-09-21T21:42:55.111 INFO:teuthology.orchestra.run.smithi081.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T21:42:55.111 INFO:teuthology.orchestra.run.smithi081.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T21:42:55.111 INFO:teuthology.orchestra.run.smithi081.stdout: "auid": 0, 2024-09-21T21:42:55.111 INFO:teuthology.orchestra.run.smithi081.stdout: "snap_mode": "selfmanaged", 2024-09-21T21:42:55.111 INFO:teuthology.orchestra.run.smithi081.stdout: "snap_seq": 0, 2024-09-21T21:42:55.111 INFO:teuthology.orchestra.run.smithi081.stdout: "snap_epoch": 0, 2024-09-21T21:42:55.111 INFO:teuthology.orchestra.run.smithi081.stdout: "pool_snaps": [], 2024-09-21T21:42:55.111 INFO:teuthology.orchestra.run.smithi081.stdout: "removed_snaps": "[]", 2024-09-21T21:42:55.111 INFO:teuthology.orchestra.run.smithi081.stdout: "quota_max_bytes": 0, 2024-09-21T21:42:55.112 INFO:teuthology.orchestra.run.smithi081.stdout: "quota_max_objects": 0, 2024-09-21T21:42:55.112 INFO:teuthology.orchestra.run.smithi081.stdout: "tiers": [], 2024-09-21T21:42:55.112 INFO:teuthology.orchestra.run.smithi081.stdout: "tier_of": -1, 2024-09-21T21:42:55.112 INFO:teuthology.orchestra.run.smithi081.stdout: "read_tier": -1, 2024-09-21T21:42:55.112 INFO:teuthology.orchestra.run.smithi081.stdout: "write_tier": -1, 2024-09-21T21:42:55.112 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_mode": "none", 2024-09-21T21:42:55.112 INFO:teuthology.orchestra.run.smithi081.stdout: "target_max_bytes": 0, 2024-09-21T21:42:55.112 INFO:teuthology.orchestra.run.smithi081.stdout: "target_max_objects": 0, 2024-09-21T21:42:55.112 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T21:42:55.112 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T21:42:55.112 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T21:42:55.112 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_min_flush_age": 0, 2024-09-21T21:42:55.112 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_min_evict_age": 0, 2024-09-21T21:42:55.112 INFO:teuthology.orchestra.run.smithi081.stdout: "erasure_code_profile": "", 2024-09-21T21:42:55.113 INFO:teuthology.orchestra.run.smithi081.stdout: "hit_set_params": { 2024-09-21T21:42:55.113 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "none" 2024-09-21T21:42:55.113 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:55.113 INFO:teuthology.orchestra.run.smithi081.stdout: "hit_set_period": 0, 2024-09-21T21:42:55.113 INFO:teuthology.orchestra.run.smithi081.stdout: "hit_set_count": 0, 2024-09-21T21:42:55.113 INFO:teuthology.orchestra.run.smithi081.stdout: "use_gmt_hitset": true, 2024-09-21T21:42:55.113 INFO:teuthology.orchestra.run.smithi081.stdout: "min_read_recency_for_promote": 0, 2024-09-21T21:42:55.113 INFO:teuthology.orchestra.run.smithi081.stdout: "min_write_recency_for_promote": 0, 2024-09-21T21:42:55.113 INFO:teuthology.orchestra.run.smithi081.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T21:42:55.113 INFO:teuthology.orchestra.run.smithi081.stdout: "hit_set_search_last_n": 0, 2024-09-21T21:42:55.113 INFO:teuthology.orchestra.run.smithi081.stdout: "grade_table": [], 2024-09-21T21:42:55.113 INFO:teuthology.orchestra.run.smithi081.stdout: "stripe_width": 0, 2024-09-21T21:42:55.113 INFO:teuthology.orchestra.run.smithi081.stdout: "expected_num_objects": 0, 2024-09-21T21:42:55.113 INFO:teuthology.orchestra.run.smithi081.stdout: "fast_read": false, 2024-09-21T21:42:55.113 INFO:teuthology.orchestra.run.smithi081.stdout: "options": { 2024-09-21T21:42:55.114 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_num_max": 32, 2024-09-21T21:42:55.114 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_num_min": 1 2024-09-21T21:42:55.114 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:55.114 INFO:teuthology.orchestra.run.smithi081.stdout: "application_metadata": { 2024-09-21T21:42:55.114 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": {} 2024-09-21T21:42:55.114 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:55.114 INFO:teuthology.orchestra.run.smithi081.stdout: "read_balance": { 2024-09-21T21:42:55.114 INFO:teuthology.orchestra.run.smithi081.stdout: "score_type": "Fair distribution", 2024-09-21T21:42:55.114 INFO:teuthology.orchestra.run.smithi081.stdout: "score_acting": 6.059999942779541, 2024-09-21T21:42:55.114 INFO:teuthology.orchestra.run.smithi081.stdout: "score_stable": 6.059999942779541, 2024-09-21T21:42:55.114 INFO:teuthology.orchestra.run.smithi081.stdout: "optimal_score": 0.33000001311302185, 2024-09-21T21:42:55.114 INFO:teuthology.orchestra.run.smithi081.stdout: "raw_score_acting": 2, 2024-09-21T21:42:55.114 INFO:teuthology.orchestra.run.smithi081.stdout: "raw_score_stable": 2, 2024-09-21T21:42:55.114 INFO:teuthology.orchestra.run.smithi081.stdout: "primary_affinity_weighted": 0.88716894388198853, 2024-09-21T21:42:55.115 INFO:teuthology.orchestra.run.smithi081.stdout: "average_primary_affinity": 0.88999998569488525, 2024-09-21T21:42:55.115 INFO:teuthology.orchestra.run.smithi081.stdout: "average_primary_affinity_weighted": 0.88999998569488525 2024-09-21T21:42:55.115 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:55.115 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:55.115 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:55.115 INFO:teuthology.orchestra.run.smithi081.stdout: "pool": 2, 2024-09-21T21:42:55.115 INFO:teuthology.orchestra.run.smithi081.stdout: "pool_name": "rbd", 2024-09-21T21:42:55.115 INFO:teuthology.orchestra.run.smithi081.stdout: "create_time": "2024-09-21T21:40:24.235658+0000", 2024-09-21T21:42:55.115 INFO:teuthology.orchestra.run.smithi081.stdout: "flags": 8193, 2024-09-21T21:42:55.115 INFO:teuthology.orchestra.run.smithi081.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T21:42:55.115 INFO:teuthology.orchestra.run.smithi081.stdout: "type": 1, 2024-09-21T21:42:55.116 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 2, 2024-09-21T21:42:55.116 INFO:teuthology.orchestra.run.smithi081.stdout: "min_size": 1, 2024-09-21T21:42:55.116 INFO:teuthology.orchestra.run.smithi081.stdout: "crush_rule": 0, 2024-09-21T21:42:55.116 INFO:teuthology.orchestra.run.smithi081.stdout: "peering_crush_bucket_count": 0, 2024-09-21T21:42:55.116 INFO:teuthology.orchestra.run.smithi081.stdout: "peering_crush_bucket_target": 0, 2024-09-21T21:42:55.116 INFO:teuthology.orchestra.run.smithi081.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T21:42:55.116 INFO:teuthology.orchestra.run.smithi081.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T21:42:55.116 INFO:teuthology.orchestra.run.smithi081.stdout: "is_stretch_pool": false, 2024-09-21T21:42:55.116 INFO:teuthology.orchestra.run.smithi081.stdout: "object_hash": 2, 2024-09-21T21:42:55.116 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_autoscale_mode": "off", 2024-09-21T21:42:55.116 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_num": 8, 2024-09-21T21:42:55.116 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_placement_num": 8, 2024-09-21T21:42:55.116 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_placement_num_target": 8, 2024-09-21T21:42:55.117 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_num_target": 8, 2024-09-21T21:42:55.117 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_num_pending": 8, 2024-09-21T21:42:55.117 INFO:teuthology.orchestra.run.smithi081.stdout: "last_pg_merge_meta": { 2024-09-21T21:42:55.117 INFO:teuthology.orchestra.run.smithi081.stdout: "source_pgid": "0.0", 2024-09-21T21:42:55.117 INFO:teuthology.orchestra.run.smithi081.stdout: "ready_epoch": 0, 2024-09-21T21:42:55.117 INFO:teuthology.orchestra.run.smithi081.stdout: "last_epoch_started": 0, 2024-09-21T21:42:55.117 INFO:teuthology.orchestra.run.smithi081.stdout: "last_epoch_clean": 0, 2024-09-21T21:42:55.117 INFO:teuthology.orchestra.run.smithi081.stdout: "source_version": "0'0", 2024-09-21T21:42:55.117 INFO:teuthology.orchestra.run.smithi081.stdout: "target_version": "0'0" 2024-09-21T21:42:55.117 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:55.117 INFO:teuthology.orchestra.run.smithi081.stdout: "last_change": "19", 2024-09-21T21:42:55.117 INFO:teuthology.orchestra.run.smithi081.stdout: "last_force_op_resend": "0", 2024-09-21T21:42:55.117 INFO:teuthology.orchestra.run.smithi081.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T21:42:55.117 INFO:teuthology.orchestra.run.smithi081.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T21:42:55.118 INFO:teuthology.orchestra.run.smithi081.stdout: "auid": 0, 2024-09-21T21:42:55.118 INFO:teuthology.orchestra.run.smithi081.stdout: "snap_mode": "selfmanaged", 2024-09-21T21:42:55.118 INFO:teuthology.orchestra.run.smithi081.stdout: "snap_seq": 2, 2024-09-21T21:42:55.118 INFO:teuthology.orchestra.run.smithi081.stdout: "snap_epoch": 19, 2024-09-21T21:42:55.118 INFO:teuthology.orchestra.run.smithi081.stdout: "pool_snaps": [], 2024-09-21T21:42:55.118 INFO:teuthology.orchestra.run.smithi081.stdout: "removed_snaps": "[]", 2024-09-21T21:42:55.118 INFO:teuthology.orchestra.run.smithi081.stdout: "quota_max_bytes": 0, 2024-09-21T21:42:55.118 INFO:teuthology.orchestra.run.smithi081.stdout: "quota_max_objects": 0, 2024-09-21T21:42:55.118 INFO:teuthology.orchestra.run.smithi081.stdout: "tiers": [], 2024-09-21T21:42:55.118 INFO:teuthology.orchestra.run.smithi081.stdout: "tier_of": -1, 2024-09-21T21:42:55.118 INFO:teuthology.orchestra.run.smithi081.stdout: "read_tier": -1, 2024-09-21T21:42:55.118 INFO:teuthology.orchestra.run.smithi081.stdout: "write_tier": -1, 2024-09-21T21:42:55.118 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_mode": "none", 2024-09-21T21:42:55.118 INFO:teuthology.orchestra.run.smithi081.stdout: "target_max_bytes": 0, 2024-09-21T21:42:55.119 INFO:teuthology.orchestra.run.smithi081.stdout: "target_max_objects": 0, 2024-09-21T21:42:55.119 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T21:42:55.119 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T21:42:55.119 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T21:42:55.119 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_min_flush_age": 0, 2024-09-21T21:42:55.119 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_min_evict_age": 0, 2024-09-21T21:42:55.119 INFO:teuthology.orchestra.run.smithi081.stdout: "erasure_code_profile": "", 2024-09-21T21:42:55.119 INFO:teuthology.orchestra.run.smithi081.stdout: "hit_set_params": { 2024-09-21T21:42:55.119 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "none" 2024-09-21T21:42:55.119 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:55.119 INFO:teuthology.orchestra.run.smithi081.stdout: "hit_set_period": 0, 2024-09-21T21:42:55.119 INFO:teuthology.orchestra.run.smithi081.stdout: "hit_set_count": 0, 2024-09-21T21:42:55.119 INFO:teuthology.orchestra.run.smithi081.stdout: "use_gmt_hitset": true, 2024-09-21T21:42:55.120 INFO:teuthology.orchestra.run.smithi081.stdout: "min_read_recency_for_promote": 0, 2024-09-21T21:42:55.120 INFO:teuthology.orchestra.run.smithi081.stdout: "min_write_recency_for_promote": 0, 2024-09-21T21:42:55.120 INFO:teuthology.orchestra.run.smithi081.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T21:42:55.120 INFO:teuthology.orchestra.run.smithi081.stdout: "hit_set_search_last_n": 0, 2024-09-21T21:42:55.120 INFO:teuthology.orchestra.run.smithi081.stdout: "grade_table": [], 2024-09-21T21:42:55.120 INFO:teuthology.orchestra.run.smithi081.stdout: "stripe_width": 0, 2024-09-21T21:42:55.120 INFO:teuthology.orchestra.run.smithi081.stdout: "expected_num_objects": 0, 2024-09-21T21:42:55.120 INFO:teuthology.orchestra.run.smithi081.stdout: "fast_read": false, 2024-09-21T21:42:55.120 INFO:teuthology.orchestra.run.smithi081.stdout: "options": {}, 2024-09-21T21:42:55.120 INFO:teuthology.orchestra.run.smithi081.stdout: "application_metadata": { 2024-09-21T21:42:55.120 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd": {} 2024-09-21T21:42:55.120 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:55.120 INFO:teuthology.orchestra.run.smithi081.stdout: "read_balance": { 2024-09-21T21:42:55.120 INFO:teuthology.orchestra.run.smithi081.stdout: "score_type": "Fair distribution", 2024-09-21T21:42:55.121 INFO:teuthology.orchestra.run.smithi081.stdout: "score_acting": 2.2400000095367432, 2024-09-21T21:42:55.121 INFO:teuthology.orchestra.run.smithi081.stdout: "score_stable": 2.2400000095367432, 2024-09-21T21:42:55.121 INFO:teuthology.orchestra.run.smithi081.stdout: "optimal_score": 0.67000001668930054, 2024-09-21T21:42:55.121 INFO:teuthology.orchestra.run.smithi081.stdout: "raw_score_acting": 1.5, 2024-09-21T21:42:55.121 INFO:teuthology.orchestra.run.smithi081.stdout: "raw_score_stable": 1.5, 2024-09-21T21:42:55.121 INFO:teuthology.orchestra.run.smithi081.stdout: "primary_affinity_weighted": 0.88716894388198853, 2024-09-21T21:42:55.121 INFO:teuthology.orchestra.run.smithi081.stdout: "average_primary_affinity": 0.88999998569488525, 2024-09-21T21:42:55.121 INFO:teuthology.orchestra.run.smithi081.stdout: "average_primary_affinity_weighted": 0.88999998569488525 2024-09-21T21:42:55.121 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:55.121 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:55.121 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:55.121 INFO:teuthology.orchestra.run.smithi081.stdout: "pool": 3, 2024-09-21T21:42:55.121 INFO:teuthology.orchestra.run.smithi081.stdout: "pool_name": "pool_client.0", 2024-09-21T21:42:55.121 INFO:teuthology.orchestra.run.smithi081.stdout: "create_time": "2024-09-21T21:40:36.093691+0000", 2024-09-21T21:42:55.122 INFO:teuthology.orchestra.run.smithi081.stdout: "flags": 8193, 2024-09-21T21:42:55.122 INFO:teuthology.orchestra.run.smithi081.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T21:42:55.122 INFO:teuthology.orchestra.run.smithi081.stdout: "type": 1, 2024-09-21T21:42:55.122 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 2, 2024-09-21T21:42:55.122 INFO:teuthology.orchestra.run.smithi081.stdout: "min_size": 1, 2024-09-21T21:42:55.122 INFO:teuthology.orchestra.run.smithi081.stdout: "crush_rule": 0, 2024-09-21T21:42:55.122 INFO:teuthology.orchestra.run.smithi081.stdout: "peering_crush_bucket_count": 0, 2024-09-21T21:42:55.122 INFO:teuthology.orchestra.run.smithi081.stdout: "peering_crush_bucket_target": 0, 2024-09-21T21:42:55.122 INFO:teuthology.orchestra.run.smithi081.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T21:42:55.122 INFO:teuthology.orchestra.run.smithi081.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T21:42:55.122 INFO:teuthology.orchestra.run.smithi081.stdout: "is_stretch_pool": false, 2024-09-21T21:42:55.122 INFO:teuthology.orchestra.run.smithi081.stdout: "object_hash": 2, 2024-09-21T21:42:55.122 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_autoscale_mode": "off", 2024-09-21T21:42:55.122 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_num": 32, 2024-09-21T21:42:55.123 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_placement_num": 32, 2024-09-21T21:42:55.123 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_placement_num_target": 32, 2024-09-21T21:42:55.123 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_num_target": 32, 2024-09-21T21:42:55.123 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_num_pending": 32, 2024-09-21T21:42:55.123 INFO:teuthology.orchestra.run.smithi081.stdout: "last_pg_merge_meta": { 2024-09-21T21:42:55.123 INFO:teuthology.orchestra.run.smithi081.stdout: "source_pgid": "0.0", 2024-09-21T21:42:55.123 INFO:teuthology.orchestra.run.smithi081.stdout: "ready_epoch": 0, 2024-09-21T21:42:55.123 INFO:teuthology.orchestra.run.smithi081.stdout: "last_epoch_started": 0, 2024-09-21T21:42:55.123 INFO:teuthology.orchestra.run.smithi081.stdout: "last_epoch_clean": 0, 2024-09-21T21:42:55.123 INFO:teuthology.orchestra.run.smithi081.stdout: "source_version": "0'0", 2024-09-21T21:42:55.123 INFO:teuthology.orchestra.run.smithi081.stdout: "target_version": "0'0" 2024-09-21T21:42:55.123 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:55.123 INFO:teuthology.orchestra.run.smithi081.stdout: "last_change": "132", 2024-09-21T21:42:55.123 INFO:teuthology.orchestra.run.smithi081.stdout: "last_force_op_resend": "0", 2024-09-21T21:42:55.124 INFO:teuthology.orchestra.run.smithi081.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T21:42:55.124 INFO:teuthology.orchestra.run.smithi081.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T21:42:55.124 INFO:teuthology.orchestra.run.smithi081.stdout: "auid": 0, 2024-09-21T21:42:55.124 INFO:teuthology.orchestra.run.smithi081.stdout: "snap_mode": "selfmanaged", 2024-09-21T21:42:55.124 INFO:teuthology.orchestra.run.smithi081.stdout: "snap_seq": 48, 2024-09-21T21:42:55.124 INFO:teuthology.orchestra.run.smithi081.stdout: "snap_epoch": 132, 2024-09-21T21:42:55.124 INFO:teuthology.orchestra.run.smithi081.stdout: "pool_snaps": [], 2024-09-21T21:42:55.124 INFO:teuthology.orchestra.run.smithi081.stdout: "removed_snaps": "[]", 2024-09-21T21:42:55.124 INFO:teuthology.orchestra.run.smithi081.stdout: "quota_max_bytes": 0, 2024-09-21T21:42:55.124 INFO:teuthology.orchestra.run.smithi081.stdout: "quota_max_objects": 0, 2024-09-21T21:42:55.124 INFO:teuthology.orchestra.run.smithi081.stdout: "tiers": [], 2024-09-21T21:42:55.124 INFO:teuthology.orchestra.run.smithi081.stdout: "tier_of": -1, 2024-09-21T21:42:55.124 INFO:teuthology.orchestra.run.smithi081.stdout: "read_tier": -1, 2024-09-21T21:42:55.124 INFO:teuthology.orchestra.run.smithi081.stdout: "write_tier": -1, 2024-09-21T21:42:55.124 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_mode": "none", 2024-09-21T21:42:55.125 INFO:teuthology.orchestra.run.smithi081.stdout: "target_max_bytes": 0, 2024-09-21T21:42:55.125 INFO:teuthology.orchestra.run.smithi081.stdout: "target_max_objects": 0, 2024-09-21T21:42:55.125 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T21:42:55.125 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T21:42:55.125 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T21:42:55.125 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_min_flush_age": 0, 2024-09-21T21:42:55.125 INFO:teuthology.orchestra.run.smithi081.stdout: "cache_min_evict_age": 0, 2024-09-21T21:42:55.125 INFO:teuthology.orchestra.run.smithi081.stdout: "erasure_code_profile": "", 2024-09-21T21:42:55.125 INFO:teuthology.orchestra.run.smithi081.stdout: "hit_set_params": { 2024-09-21T21:42:55.125 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "none" 2024-09-21T21:42:55.125 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:55.125 INFO:teuthology.orchestra.run.smithi081.stdout: "hit_set_period": 0, 2024-09-21T21:42:55.125 INFO:teuthology.orchestra.run.smithi081.stdout: "hit_set_count": 0, 2024-09-21T21:42:55.125 INFO:teuthology.orchestra.run.smithi081.stdout: "use_gmt_hitset": true, 2024-09-21T21:42:55.126 INFO:teuthology.orchestra.run.smithi081.stdout: "min_read_recency_for_promote": 0, 2024-09-21T21:42:55.126 INFO:teuthology.orchestra.run.smithi081.stdout: "min_write_recency_for_promote": 0, 2024-09-21T21:42:55.126 INFO:teuthology.orchestra.run.smithi081.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T21:42:55.126 INFO:teuthology.orchestra.run.smithi081.stdout: "hit_set_search_last_n": 0, 2024-09-21T21:42:55.126 INFO:teuthology.orchestra.run.smithi081.stdout: "grade_table": [], 2024-09-21T21:42:55.126 INFO:teuthology.orchestra.run.smithi081.stdout: "stripe_width": 0, 2024-09-21T21:42:55.126 INFO:teuthology.orchestra.run.smithi081.stdout: "expected_num_objects": 0, 2024-09-21T21:42:55.126 INFO:teuthology.orchestra.run.smithi081.stdout: "fast_read": false, 2024-09-21T21:42:55.126 INFO:teuthology.orchestra.run.smithi081.stdout: "options": {}, 2024-09-21T21:42:55.126 INFO:teuthology.orchestra.run.smithi081.stdout: "application_metadata": { 2024-09-21T21:42:55.126 INFO:teuthology.orchestra.run.smithi081.stdout: "rbd": {} 2024-09-21T21:42:55.126 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:55.126 INFO:teuthology.orchestra.run.smithi081.stdout: "read_balance": { 2024-09-21T21:42:55.126 INFO:teuthology.orchestra.run.smithi081.stdout: "score_type": "Fair distribution", 2024-09-21T21:42:55.127 INFO:teuthology.orchestra.run.smithi081.stdout: "score_acting": 2.2400000095367432, 2024-09-21T21:42:55.127 INFO:teuthology.orchestra.run.smithi081.stdout: "score_stable": 2.2400000095367432, 2024-09-21T21:42:55.127 INFO:teuthology.orchestra.run.smithi081.stdout: "optimal_score": 0.67000001668930054, 2024-09-21T21:42:55.127 INFO:teuthology.orchestra.run.smithi081.stdout: "raw_score_acting": 1.5, 2024-09-21T21:42:55.127 INFO:teuthology.orchestra.run.smithi081.stdout: "raw_score_stable": 1.5, 2024-09-21T21:42:55.127 INFO:teuthology.orchestra.run.smithi081.stdout: "primary_affinity_weighted": 0.88716894388198853, 2024-09-21T21:42:55.127 INFO:teuthology.orchestra.run.smithi081.stdout: "average_primary_affinity": 0.88999998569488525, 2024-09-21T21:42:55.127 INFO:teuthology.orchestra.run.smithi081.stdout: "average_primary_affinity_weighted": 0.88999998569488525 2024-09-21T21:42:55.127 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:55.127 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:55.127 INFO:teuthology.orchestra.run.smithi081.stdout: ], 2024-09-21T21:42:55.127 INFO:teuthology.orchestra.run.smithi081.stdout: "osds": [ 2024-09-21T21:42:55.127 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:55.127 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": 0, 2024-09-21T21:42:55.128 INFO:teuthology.orchestra.run.smithi081.stdout: "uuid": "a9869830-8e74-4b02-963d-ab630a0ee286", 2024-09-21T21:42:55.128 INFO:teuthology.orchestra.run.smithi081.stdout: "up": 1, 2024-09-21T21:42:55.128 INFO:teuthology.orchestra.run.smithi081.stdout: "in": 0, 2024-09-21T21:42:55.128 INFO:teuthology.orchestra.run.smithi081.stdout: "weight": 0, 2024-09-21T21:42:55.128 INFO:teuthology.orchestra.run.smithi081.stdout: "primary_affinity": 1, 2024-09-21T21:42:55.128 INFO:teuthology.orchestra.run.smithi081.stdout: "last_clean_begin": 0, 2024-09-21T21:42:55.128 INFO:teuthology.orchestra.run.smithi081.stdout: "last_clean_end": 0, 2024-09-21T21:42:55.128 INFO:teuthology.orchestra.run.smithi081.stdout: "up_from": 12, 2024-09-21T21:42:55.128 INFO:teuthology.orchestra.run.smithi081.stdout: "up_thru": 88, 2024-09-21T21:42:55.128 INFO:teuthology.orchestra.run.smithi081.stdout: "down_at": 0, 2024-09-21T21:42:55.128 INFO:teuthology.orchestra.run.smithi081.stdout: "lost_at": 0, 2024-09-21T21:42:55.128 INFO:teuthology.orchestra.run.smithi081.stdout: "public_addrs": { 2024-09-21T21:42:55.128 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:42:55.128 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:55.129 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:42:55.129 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6808", 2024-09-21T21:42:55.129 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1657552338 2024-09-21T21:42:55.129 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:55.129 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:55.129 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:42:55.129 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6809", 2024-09-21T21:42:55.129 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1657552338 2024-09-21T21:42:55.129 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:55.129 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:55.129 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:55.129 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster_addrs": { 2024-09-21T21:42:55.129 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:42:55.129 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:55.129 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:42:55.130 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6810", 2024-09-21T21:42:55.130 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1657552338 2024-09-21T21:42:55.130 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:55.130 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:55.130 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:42:55.130 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6811", 2024-09-21T21:42:55.130 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1657552338 2024-09-21T21:42:55.130 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:55.130 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:55.130 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:55.130 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_back_addrs": { 2024-09-21T21:42:55.130 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:42:55.130 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:55.130 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:42:55.131 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6814", 2024-09-21T21:42:55.131 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1657552338 2024-09-21T21:42:55.131 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:55.131 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:55.131 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:42:55.131 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6815", 2024-09-21T21:42:55.131 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1657552338 2024-09-21T21:42:55.131 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:55.131 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:55.131 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:55.131 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_front_addrs": { 2024-09-21T21:42:55.131 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:42:55.131 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:55.131 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:42:55.132 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6812", 2024-09-21T21:42:55.132 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1657552338 2024-09-21T21:42:55.132 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:55.132 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:55.132 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:42:55.132 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6813", 2024-09-21T21:42:55.132 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1657552338 2024-09-21T21:42:55.132 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:55.132 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:55.132 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:55.132 INFO:teuthology.orchestra.run.smithi081.stdout: "public_addr": "172.21.15.81:6809/1657552338", 2024-09-21T21:42:55.132 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster_addr": "172.21.15.81:6811/1657552338", 2024-09-21T21:42:55.132 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_back_addr": "172.21.15.81:6815/1657552338", 2024-09-21T21:42:55.132 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_front_addr": "172.21.15.81:6813/1657552338", 2024-09-21T21:42:55.133 INFO:teuthology.orchestra.run.smithi081.stdout: "state": [ 2024-09-21T21:42:55.133 INFO:teuthology.orchestra.run.smithi081.stdout: "exists", 2024-09-21T21:42:55.133 INFO:teuthology.orchestra.run.smithi081.stdout: "up" 2024-09-21T21:42:55.133 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:55.133 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:55.133 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:55.133 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": 1, 2024-09-21T21:42:55.133 INFO:teuthology.orchestra.run.smithi081.stdout: "uuid": "b189819d-69a3-4e95-96e7-96dc4809ac80", 2024-09-21T21:42:55.133 INFO:teuthology.orchestra.run.smithi081.stdout: "up": 1, 2024-09-21T21:42:55.133 INFO:teuthology.orchestra.run.smithi081.stdout: "in": 1, 2024-09-21T21:42:55.133 INFO:teuthology.orchestra.run.smithi081.stdout: "weight": 1, 2024-09-21T21:42:55.133 INFO:teuthology.orchestra.run.smithi081.stdout: "primary_affinity": 1, 2024-09-21T21:42:55.133 INFO:teuthology.orchestra.run.smithi081.stdout: "last_clean_begin": 0, 2024-09-21T21:42:55.133 INFO:teuthology.orchestra.run.smithi081.stdout: "last_clean_end": 0, 2024-09-21T21:42:55.133 INFO:teuthology.orchestra.run.smithi081.stdout: "up_from": 12, 2024-09-21T21:42:55.134 INFO:teuthology.orchestra.run.smithi081.stdout: "up_thru": 100, 2024-09-21T21:42:55.134 INFO:teuthology.orchestra.run.smithi081.stdout: "down_at": 0, 2024-09-21T21:42:55.134 INFO:teuthology.orchestra.run.smithi081.stdout: "lost_at": 0, 2024-09-21T21:42:55.134 INFO:teuthology.orchestra.run.smithi081.stdout: "public_addrs": { 2024-09-21T21:42:55.134 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:42:55.134 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:55.134 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:42:55.134 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6816", 2024-09-21T21:42:55.134 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2864159502 2024-09-21T21:42:55.134 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:55.134 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:55.134 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:42:55.134 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6817", 2024-09-21T21:42:55.134 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2864159502 2024-09-21T21:42:55.135 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:55.135 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:55.135 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:55.135 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster_addrs": { 2024-09-21T21:42:55.135 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:42:55.135 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:55.135 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:42:55.135 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6818", 2024-09-21T21:42:55.135 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2864159502 2024-09-21T21:42:55.135 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:55.135 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:55.135 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:42:55.135 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6819", 2024-09-21T21:42:55.135 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2864159502 2024-09-21T21:42:55.135 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:55.136 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:55.136 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:55.136 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_back_addrs": { 2024-09-21T21:42:55.136 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:42:55.136 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:55.136 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:42:55.136 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6822", 2024-09-21T21:42:55.136 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2864159502 2024-09-21T21:42:55.136 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:55.136 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:55.136 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:42:55.136 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6823", 2024-09-21T21:42:55.136 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2864159502 2024-09-21T21:42:55.136 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:55.137 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:55.137 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:55.137 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_front_addrs": { 2024-09-21T21:42:55.137 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:42:55.137 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:55.137 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:42:55.137 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6820", 2024-09-21T21:42:55.137 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2864159502 2024-09-21T21:42:55.137 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:55.137 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:55.137 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:42:55.137 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6821", 2024-09-21T21:42:55.137 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2864159502 2024-09-21T21:42:55.137 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:55.137 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:55.138 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:55.138 INFO:teuthology.orchestra.run.smithi081.stdout: "public_addr": "172.21.15.81:6817/2864159502", 2024-09-21T21:42:55.138 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster_addr": "172.21.15.81:6819/2864159502", 2024-09-21T21:42:55.138 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_back_addr": "172.21.15.81:6823/2864159502", 2024-09-21T21:42:55.138 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_front_addr": "172.21.15.81:6821/2864159502", 2024-09-21T21:42:55.138 INFO:teuthology.orchestra.run.smithi081.stdout: "state": [ 2024-09-21T21:42:55.138 INFO:teuthology.orchestra.run.smithi081.stdout: "exists", 2024-09-21T21:42:55.138 INFO:teuthology.orchestra.run.smithi081.stdout: "up" 2024-09-21T21:42:55.138 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:55.138 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:55.138 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:55.138 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": 2, 2024-09-21T21:42:55.138 INFO:teuthology.orchestra.run.smithi081.stdout: "uuid": "8437b99a-1297-4903-ab66-95c5cba2b2bf", 2024-09-21T21:42:55.138 INFO:teuthology.orchestra.run.smithi081.stdout: "up": 1, 2024-09-21T21:42:55.139 INFO:teuthology.orchestra.run.smithi081.stdout: "in": 1, 2024-09-21T21:42:55.139 INFO:teuthology.orchestra.run.smithi081.stdout: "weight": 1, 2024-09-21T21:42:55.139 INFO:teuthology.orchestra.run.smithi081.stdout: "primary_affinity": 0.3230133056640625, 2024-09-21T21:42:55.139 INFO:teuthology.orchestra.run.smithi081.stdout: "last_clean_begin": 0, 2024-09-21T21:42:55.139 INFO:teuthology.orchestra.run.smithi081.stdout: "last_clean_end": 0, 2024-09-21T21:42:55.139 INFO:teuthology.orchestra.run.smithi081.stdout: "up_from": 11, 2024-09-21T21:42:55.139 INFO:teuthology.orchestra.run.smithi081.stdout: "up_thru": 102, 2024-09-21T21:42:55.139 INFO:teuthology.orchestra.run.smithi081.stdout: "down_at": 0, 2024-09-21T21:42:55.139 INFO:teuthology.orchestra.run.smithi081.stdout: "lost_at": 0, 2024-09-21T21:42:55.139 INFO:teuthology.orchestra.run.smithi081.stdout: "public_addrs": { 2024-09-21T21:42:55.139 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:42:55.139 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:55.139 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:42:55.139 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6800", 2024-09-21T21:42:55.139 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 3023160932 2024-09-21T21:42:55.140 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:55.140 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:55.140 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:42:55.140 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6801", 2024-09-21T21:42:55.140 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 3023160932 2024-09-21T21:42:55.140 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:55.140 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:55.140 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:55.140 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster_addrs": { 2024-09-21T21:42:55.140 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:42:55.140 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:55.140 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:42:55.140 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6802", 2024-09-21T21:42:55.140 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 3023160932 2024-09-21T21:42:55.141 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:55.141 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:55.141 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:42:55.141 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6803", 2024-09-21T21:42:55.141 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 3023160932 2024-09-21T21:42:55.141 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:55.141 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:55.141 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:55.141 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_back_addrs": { 2024-09-21T21:42:55.141 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:42:55.141 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:55.141 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:42:55.141 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6806", 2024-09-21T21:42:55.141 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 3023160932 2024-09-21T21:42:55.142 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:55.142 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:55.142 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:42:55.142 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6807", 2024-09-21T21:42:55.142 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 3023160932 2024-09-21T21:42:55.142 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:55.142 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:55.142 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:55.142 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_front_addrs": { 2024-09-21T21:42:55.142 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:42:55.142 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:55.142 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:42:55.142 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6804", 2024-09-21T21:42:55.142 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 3023160932 2024-09-21T21:42:55.143 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:55.143 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:55.143 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:42:55.143 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.81:6805", 2024-09-21T21:42:55.143 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 3023160932 2024-09-21T21:42:55.143 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:55.143 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:55.143 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:55.143 INFO:teuthology.orchestra.run.smithi081.stdout: "public_addr": "172.21.15.81:6801/3023160932", 2024-09-21T21:42:55.143 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster_addr": "172.21.15.81:6803/3023160932", 2024-09-21T21:42:55.143 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_back_addr": "172.21.15.81:6807/3023160932", 2024-09-21T21:42:55.143 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_front_addr": "172.21.15.81:6805/3023160932", 2024-09-21T21:42:55.143 INFO:teuthology.orchestra.run.smithi081.stdout: "state": [ 2024-09-21T21:42:55.143 INFO:teuthology.orchestra.run.smithi081.stdout: "exists", 2024-09-21T21:42:55.144 INFO:teuthology.orchestra.run.smithi081.stdout: "up" 2024-09-21T21:42:55.144 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:55.144 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:55.144 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:55.144 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": 3, 2024-09-21T21:42:55.144 INFO:teuthology.orchestra.run.smithi081.stdout: "uuid": "e8848768-c1aa-445b-9554-dc6aa4eefc8d", 2024-09-21T21:42:55.144 INFO:teuthology.orchestra.run.smithi081.stdout: "up": 1, 2024-09-21T21:42:55.144 INFO:teuthology.orchestra.run.smithi081.stdout: "in": 1, 2024-09-21T21:42:55.144 INFO:teuthology.orchestra.run.smithi081.stdout: "weight": 1, 2024-09-21T21:42:55.144 INFO:teuthology.orchestra.run.smithi081.stdout: "primary_affinity": 1, 2024-09-21T21:42:55.144 INFO:teuthology.orchestra.run.smithi081.stdout: "last_clean_begin": 0, 2024-09-21T21:42:55.144 INFO:teuthology.orchestra.run.smithi081.stdout: "last_clean_end": 0, 2024-09-21T21:42:55.144 INFO:teuthology.orchestra.run.smithi081.stdout: "up_from": 13, 2024-09-21T21:42:55.145 INFO:teuthology.orchestra.run.smithi081.stdout: "up_thru": 100, 2024-09-21T21:42:55.145 INFO:teuthology.orchestra.run.smithi081.stdout: "down_at": 0, 2024-09-21T21:42:55.145 INFO:teuthology.orchestra.run.smithi081.stdout: "lost_at": 0, 2024-09-21T21:42:55.145 INFO:teuthology.orchestra.run.smithi081.stdout: "public_addrs": { 2024-09-21T21:42:55.145 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:42:55.145 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:55.145 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:42:55.145 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6808", 2024-09-21T21:42:55.145 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2681016844 2024-09-21T21:42:55.145 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:55.145 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:55.145 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:42:55.145 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6809", 2024-09-21T21:42:55.145 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2681016844 2024-09-21T21:42:55.145 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:55.146 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:55.146 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:55.146 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster_addrs": { 2024-09-21T21:42:55.146 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:42:55.146 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:55.146 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:42:55.146 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6810", 2024-09-21T21:42:55.146 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2681016844 2024-09-21T21:42:55.146 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:55.146 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:55.146 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:42:55.146 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6811", 2024-09-21T21:42:55.146 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2681016844 2024-09-21T21:42:55.146 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:55.147 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:55.147 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:55.147 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_back_addrs": { 2024-09-21T21:42:55.147 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:42:55.147 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:55.147 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:42:55.147 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6814", 2024-09-21T21:42:55.147 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2681016844 2024-09-21T21:42:55.147 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:55.147 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:55.147 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:42:55.147 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6815", 2024-09-21T21:42:55.147 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2681016844 2024-09-21T21:42:55.147 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:55.148 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:55.148 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:55.148 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_front_addrs": { 2024-09-21T21:42:55.148 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:42:55.148 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:55.148 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:42:55.148 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6812", 2024-09-21T21:42:55.148 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2681016844 2024-09-21T21:42:55.148 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:55.148 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:55.148 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:42:55.148 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6813", 2024-09-21T21:42:55.148 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2681016844 2024-09-21T21:42:55.148 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:55.149 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:55.149 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:55.149 INFO:teuthology.orchestra.run.smithi081.stdout: "public_addr": "172.21.15.122:6809/2681016844", 2024-09-21T21:42:55.149 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster_addr": "172.21.15.122:6811/2681016844", 2024-09-21T21:42:55.149 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_back_addr": "172.21.15.122:6815/2681016844", 2024-09-21T21:42:55.149 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_front_addr": "172.21.15.122:6813/2681016844", 2024-09-21T21:42:55.149 INFO:teuthology.orchestra.run.smithi081.stdout: "state": [ 2024-09-21T21:42:55.149 INFO:teuthology.orchestra.run.smithi081.stdout: "exists", 2024-09-21T21:42:55.149 INFO:teuthology.orchestra.run.smithi081.stdout: "up" 2024-09-21T21:42:55.149 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:55.149 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:55.149 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:55.149 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": 4, 2024-09-21T21:42:55.149 INFO:teuthology.orchestra.run.smithi081.stdout: "uuid": "b1d8e85d-a207-4ba6-9322-61d07812083e", 2024-09-21T21:42:55.150 INFO:teuthology.orchestra.run.smithi081.stdout: "up": 1, 2024-09-21T21:42:55.150 INFO:teuthology.orchestra.run.smithi081.stdout: "in": 0, 2024-09-21T21:42:55.150 INFO:teuthology.orchestra.run.smithi081.stdout: "weight": 0, 2024-09-21T21:42:55.150 INFO:teuthology.orchestra.run.smithi081.stdout: "primary_affinity": 1, 2024-09-21T21:42:55.150 INFO:teuthology.orchestra.run.smithi081.stdout: "last_clean_begin": 0, 2024-09-21T21:42:55.150 INFO:teuthology.orchestra.run.smithi081.stdout: "last_clean_end": 0, 2024-09-21T21:42:55.150 INFO:teuthology.orchestra.run.smithi081.stdout: "up_from": 13, 2024-09-21T21:42:55.150 INFO:teuthology.orchestra.run.smithi081.stdout: "up_thru": 99, 2024-09-21T21:42:55.150 INFO:teuthology.orchestra.run.smithi081.stdout: "down_at": 0, 2024-09-21T21:42:55.150 INFO:teuthology.orchestra.run.smithi081.stdout: "lost_at": 0, 2024-09-21T21:42:55.150 INFO:teuthology.orchestra.run.smithi081.stdout: "public_addrs": { 2024-09-21T21:42:55.150 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:42:55.150 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:55.150 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:42:55.151 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6800", 2024-09-21T21:42:55.151 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1095023540 2024-09-21T21:42:55.151 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:55.151 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:55.151 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:42:55.151 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6801", 2024-09-21T21:42:55.151 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1095023540 2024-09-21T21:42:55.151 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:55.151 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:55.151 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:55.151 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster_addrs": { 2024-09-21T21:42:55.151 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:42:55.151 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:55.151 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:42:55.152 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6802", 2024-09-21T21:42:55.152 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1095023540 2024-09-21T21:42:55.152 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:55.152 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:55.152 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:42:55.152 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6803", 2024-09-21T21:42:55.152 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1095023540 2024-09-21T21:42:55.152 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:55.152 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:55.152 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:55.152 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_back_addrs": { 2024-09-21T21:42:55.152 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:42:55.152 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:55.152 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:42:55.153 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6806", 2024-09-21T21:42:55.153 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1095023540 2024-09-21T21:42:55.153 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:55.153 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:55.153 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:42:55.153 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6807", 2024-09-21T21:42:55.153 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1095023540 2024-09-21T21:42:55.153 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:55.153 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:55.153 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:55.153 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_front_addrs": { 2024-09-21T21:42:55.153 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:42:55.153 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:55.153 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:42:55.153 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6804", 2024-09-21T21:42:55.154 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1095023540 2024-09-21T21:42:55.154 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:55.154 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:55.154 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:42:55.154 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6805", 2024-09-21T21:42:55.154 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1095023540 2024-09-21T21:42:55.154 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:55.154 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:55.154 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:55.154 INFO:teuthology.orchestra.run.smithi081.stdout: "public_addr": "172.21.15.122:6801/1095023540", 2024-09-21T21:42:55.154 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster_addr": "172.21.15.122:6803/1095023540", 2024-09-21T21:42:55.154 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_back_addr": "172.21.15.122:6807/1095023540", 2024-09-21T21:42:55.154 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_front_addr": "172.21.15.122:6805/1095023540", 2024-09-21T21:42:55.154 INFO:teuthology.orchestra.run.smithi081.stdout: "state": [ 2024-09-21T21:42:55.155 INFO:teuthology.orchestra.run.smithi081.stdout: "exists", 2024-09-21T21:42:55.155 INFO:teuthology.orchestra.run.smithi081.stdout: "up" 2024-09-21T21:42:55.155 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:55.155 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:55.155 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:55.155 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": 5, 2024-09-21T21:42:55.155 INFO:teuthology.orchestra.run.smithi081.stdout: "uuid": "3ab0450b-d8fd-4ef2-a298-08d5aa2ea32f", 2024-09-21T21:42:55.155 INFO:teuthology.orchestra.run.smithi081.stdout: "up": 1, 2024-09-21T21:42:55.155 INFO:teuthology.orchestra.run.smithi081.stdout: "in": 1, 2024-09-21T21:42:55.155 INFO:teuthology.orchestra.run.smithi081.stdout: "weight": 1, 2024-09-21T21:42:55.155 INFO:teuthology.orchestra.run.smithi081.stdout: "primary_affinity": 1, 2024-09-21T21:42:55.155 INFO:teuthology.orchestra.run.smithi081.stdout: "last_clean_begin": 0, 2024-09-21T21:42:55.155 INFO:teuthology.orchestra.run.smithi081.stdout: "last_clean_end": 0, 2024-09-21T21:42:55.155 INFO:teuthology.orchestra.run.smithi081.stdout: "up_from": 12, 2024-09-21T21:42:55.155 INFO:teuthology.orchestra.run.smithi081.stdout: "up_thru": 106, 2024-09-21T21:42:55.156 INFO:teuthology.orchestra.run.smithi081.stdout: "down_at": 0, 2024-09-21T21:42:55.156 INFO:teuthology.orchestra.run.smithi081.stdout: "lost_at": 0, 2024-09-21T21:42:55.156 INFO:teuthology.orchestra.run.smithi081.stdout: "public_addrs": { 2024-09-21T21:42:55.156 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:42:55.156 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:55.156 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:42:55.156 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6816", 2024-09-21T21:42:55.156 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1974398333 2024-09-21T21:42:55.156 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:55.156 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:55.156 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:42:55.156 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6817", 2024-09-21T21:42:55.156 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1974398333 2024-09-21T21:42:55.156 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:55.157 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:55.157 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:55.157 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster_addrs": { 2024-09-21T21:42:55.157 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:42:55.157 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:55.157 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:42:55.157 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6818", 2024-09-21T21:42:55.157 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1974398333 2024-09-21T21:42:55.157 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:55.157 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:55.157 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:42:55.157 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6819", 2024-09-21T21:42:55.157 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1974398333 2024-09-21T21:42:55.157 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:55.157 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:55.158 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:55.158 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_back_addrs": { 2024-09-21T21:42:55.158 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:42:55.158 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:55.158 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:42:55.158 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6822", 2024-09-21T21:42:55.158 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1974398333 2024-09-21T21:42:55.158 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:55.158 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:55.158 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:42:55.158 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6823", 2024-09-21T21:42:55.158 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1974398333 2024-09-21T21:42:55.158 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:55.159 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:55.159 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:55.159 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_front_addrs": { 2024-09-21T21:42:55.159 INFO:teuthology.orchestra.run.smithi081.stdout: "addrvec": [ 2024-09-21T21:42:55.159 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:55.159 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v2", 2024-09-21T21:42:55.159 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6820", 2024-09-21T21:42:55.159 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1974398333 2024-09-21T21:42:55.159 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:55.159 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:55.159 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2024-09-21T21:42:55.159 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "172.21.15.122:6821", 2024-09-21T21:42:55.159 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1974398333 2024-09-21T21:42:55.159 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:55.159 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:55.160 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:55.160 INFO:teuthology.orchestra.run.smithi081.stdout: "public_addr": "172.21.15.122:6817/1974398333", 2024-09-21T21:42:55.160 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster_addr": "172.21.15.122:6819/1974398333", 2024-09-21T21:42:55.160 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_back_addr": "172.21.15.122:6823/1974398333", 2024-09-21T21:42:55.160 INFO:teuthology.orchestra.run.smithi081.stdout: "heartbeat_front_addr": "172.21.15.122:6821/1974398333", 2024-09-21T21:42:55.160 INFO:teuthology.orchestra.run.smithi081.stdout: "state": [ 2024-09-21T21:42:55.160 INFO:teuthology.orchestra.run.smithi081.stdout: "exists", 2024-09-21T21:42:55.160 INFO:teuthology.orchestra.run.smithi081.stdout: "up" 2024-09-21T21:42:55.160 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:55.160 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:55.160 INFO:teuthology.orchestra.run.smithi081.stdout: ], 2024-09-21T21:42:55.160 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_xinfo": [ 2024-09-21T21:42:55.160 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:55.160 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": 0, 2024-09-21T21:42:55.161 INFO:teuthology.orchestra.run.smithi081.stdout: "down_stamp": "0.000000", 2024-09-21T21:42:55.161 INFO:teuthology.orchestra.run.smithi081.stdout: "laggy_probability": 0, 2024-09-21T21:42:55.161 INFO:teuthology.orchestra.run.smithi081.stdout: "laggy_interval": 0, 2024-09-21T21:42:55.161 INFO:teuthology.orchestra.run.smithi081.stdout: "features": 4540701547738038271, 2024-09-21T21:42:55.161 INFO:teuthology.orchestra.run.smithi081.stdout: "old_weight": 65536, 2024-09-21T21:42:55.161 INFO:teuthology.orchestra.run.smithi081.stdout: "last_purged_snaps_scrub": "2024-09-21T21:40:15.766648+0000", 2024-09-21T21:42:55.161 INFO:teuthology.orchestra.run.smithi081.stdout: "dead_epoch": 0 2024-09-21T21:42:55.161 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:55.161 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:55.161 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": 1, 2024-09-21T21:42:55.161 INFO:teuthology.orchestra.run.smithi081.stdout: "down_stamp": "0.000000", 2024-09-21T21:42:55.161 INFO:teuthology.orchestra.run.smithi081.stdout: "laggy_probability": 0, 2024-09-21T21:42:55.161 INFO:teuthology.orchestra.run.smithi081.stdout: "laggy_interval": 0, 2024-09-21T21:42:55.161 INFO:teuthology.orchestra.run.smithi081.stdout: "features": 4540701547738038271, 2024-09-21T21:42:55.161 INFO:teuthology.orchestra.run.smithi081.stdout: "old_weight": 0, 2024-09-21T21:42:55.162 INFO:teuthology.orchestra.run.smithi081.stdout: "last_purged_snaps_scrub": "2024-09-21T21:40:15.457496+0000", 2024-09-21T21:42:55.162 INFO:teuthology.orchestra.run.smithi081.stdout: "dead_epoch": 0 2024-09-21T21:42:55.162 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:55.162 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:55.162 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": 2, 2024-09-21T21:42:55.162 INFO:teuthology.orchestra.run.smithi081.stdout: "down_stamp": "0.000000", 2024-09-21T21:42:55.162 INFO:teuthology.orchestra.run.smithi081.stdout: "laggy_probability": 0, 2024-09-21T21:42:55.162 INFO:teuthology.orchestra.run.smithi081.stdout: "laggy_interval": 0, 2024-09-21T21:42:55.162 INFO:teuthology.orchestra.run.smithi081.stdout: "features": 4540701547738038271, 2024-09-21T21:42:55.162 INFO:teuthology.orchestra.run.smithi081.stdout: "old_weight": 0, 2024-09-21T21:42:55.162 INFO:teuthology.orchestra.run.smithi081.stdout: "last_purged_snaps_scrub": "2024-09-21T21:40:14.767039+0000", 2024-09-21T21:42:55.162 INFO:teuthology.orchestra.run.smithi081.stdout: "dead_epoch": 0 2024-09-21T21:42:55.162 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:55.162 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:55.163 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": 3, 2024-09-21T21:42:55.163 INFO:teuthology.orchestra.run.smithi081.stdout: "down_stamp": "0.000000", 2024-09-21T21:42:55.163 INFO:teuthology.orchestra.run.smithi081.stdout: "laggy_probability": 0, 2024-09-21T21:42:55.163 INFO:teuthology.orchestra.run.smithi081.stdout: "laggy_interval": 0, 2024-09-21T21:42:55.163 INFO:teuthology.orchestra.run.smithi081.stdout: "features": 4540701547738038271, 2024-09-21T21:42:55.163 INFO:teuthology.orchestra.run.smithi081.stdout: "old_weight": 0, 2024-09-21T21:42:55.163 INFO:teuthology.orchestra.run.smithi081.stdout: "last_purged_snaps_scrub": "2024-09-21T21:40:15.355570+0000", 2024-09-21T21:42:55.163 INFO:teuthology.orchestra.run.smithi081.stdout: "dead_epoch": 0 2024-09-21T21:42:55.163 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:55.163 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:55.163 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": 4, 2024-09-21T21:42:55.163 INFO:teuthology.orchestra.run.smithi081.stdout: "down_stamp": "0.000000", 2024-09-21T21:42:55.163 INFO:teuthology.orchestra.run.smithi081.stdout: "laggy_probability": 0, 2024-09-21T21:42:55.163 INFO:teuthology.orchestra.run.smithi081.stdout: "laggy_interval": 0, 2024-09-21T21:42:55.164 INFO:teuthology.orchestra.run.smithi081.stdout: "features": 4540701547738038271, 2024-09-21T21:42:55.164 INFO:teuthology.orchestra.run.smithi081.stdout: "old_weight": 65536, 2024-09-21T21:42:55.164 INFO:teuthology.orchestra.run.smithi081.stdout: "last_purged_snaps_scrub": "2024-09-21T21:40:15.335729+0000", 2024-09-21T21:42:55.164 INFO:teuthology.orchestra.run.smithi081.stdout: "dead_epoch": 0 2024-09-21T21:42:55.164 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:55.164 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:55.164 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": 5, 2024-09-21T21:42:55.164 INFO:teuthology.orchestra.run.smithi081.stdout: "down_stamp": "0.000000", 2024-09-21T21:42:55.164 INFO:teuthology.orchestra.run.smithi081.stdout: "laggy_probability": 0, 2024-09-21T21:42:55.164 INFO:teuthology.orchestra.run.smithi081.stdout: "laggy_interval": 0, 2024-09-21T21:42:55.164 INFO:teuthology.orchestra.run.smithi081.stdout: "features": 4540701547738038271, 2024-09-21T21:42:55.164 INFO:teuthology.orchestra.run.smithi081.stdout: "old_weight": 0, 2024-09-21T21:42:55.164 INFO:teuthology.orchestra.run.smithi081.stdout: "last_purged_snaps_scrub": "2024-09-21T21:40:15.614733+0000", 2024-09-21T21:42:55.164 INFO:teuthology.orchestra.run.smithi081.stdout: "dead_epoch": 0 2024-09-21T21:42:55.165 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:55.165 INFO:teuthology.orchestra.run.smithi081.stdout: ], 2024-09-21T21:42:55.165 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_upmap": [ 2024-09-21T21:42:55.165 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:55.165 INFO:teuthology.orchestra.run.smithi081.stdout: "pgid": "3.8", 2024-09-21T21:42:55.165 INFO:teuthology.orchestra.run.smithi081.stdout: "osds": [ 2024-09-21T21:42:55.165 INFO:teuthology.orchestra.run.smithi081.stdout: 2, 2024-09-21T21:42:55.165 INFO:teuthology.orchestra.run.smithi081.stdout: 3 2024-09-21T21:42:55.165 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:55.165 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:55.165 INFO:teuthology.orchestra.run.smithi081.stdout: ], 2024-09-21T21:42:55.165 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_upmap_items": [], 2024-09-21T21:42:55.165 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_upmap_primaries": [], 2024-09-21T21:42:55.165 INFO:teuthology.orchestra.run.smithi081.stdout: "pg_temp": [], 2024-09-21T21:42:55.165 INFO:teuthology.orchestra.run.smithi081.stdout: "primary_temp": [], 2024-09-21T21:42:55.166 INFO:teuthology.orchestra.run.smithi081.stdout: "blocklist": {}, 2024-09-21T21:42:55.166 INFO:teuthology.orchestra.run.smithi081.stdout: "range_blocklist": {}, 2024-09-21T21:42:55.166 INFO:teuthology.orchestra.run.smithi081.stdout: "erasure_code_profiles": { 2024-09-21T21:42:55.166 INFO:teuthology.orchestra.run.smithi081.stdout: "default": { 2024-09-21T21:42:55.166 INFO:teuthology.orchestra.run.smithi081.stdout: "crush-failure-domain": "osd", 2024-09-21T21:42:55.166 INFO:teuthology.orchestra.run.smithi081.stdout: "k": "2", 2024-09-21T21:42:55.166 INFO:teuthology.orchestra.run.smithi081.stdout: "m": "1", 2024-09-21T21:42:55.166 INFO:teuthology.orchestra.run.smithi081.stdout: "plugin": "jerasure", 2024-09-21T21:42:55.166 INFO:teuthology.orchestra.run.smithi081.stdout: "technique": "reed_sol_van" 2024-09-21T21:42:55.166 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:55.166 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:55.166 INFO:teuthology.orchestra.run.smithi081.stdout: "removed_snaps_queue": [ 2024-09-21T21:42:55.166 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:55.166 INFO:teuthology.orchestra.run.smithi081.stdout: "pool": 3, 2024-09-21T21:42:55.167 INFO:teuthology.orchestra.run.smithi081.stdout: "snaps": [ 2024-09-21T21:42:55.167 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:55.167 INFO:teuthology.orchestra.run.smithi081.stdout: "begin": 30, 2024-09-21T21:42:55.167 INFO:teuthology.orchestra.run.smithi081.stdout: "length": 9 2024-09-21T21:42:55.167 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:55.167 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:55.167 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:55.167 INFO:teuthology.orchestra.run.smithi081.stdout: ], 2024-09-21T21:42:55.167 INFO:teuthology.orchestra.run.smithi081.stdout: "new_removed_snaps": [ 2024-09-21T21:42:55.167 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:55.167 INFO:teuthology.orchestra.run.smithi081.stdout: "pool": 3, 2024-09-21T21:42:55.167 INFO:teuthology.orchestra.run.smithi081.stdout: "snaps": [ 2024-09-21T21:42:55.167 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:55.167 INFO:teuthology.orchestra.run.smithi081.stdout: "begin": 30, 2024-09-21T21:42:55.168 INFO:teuthology.orchestra.run.smithi081.stdout: "length": 1 2024-09-21T21:42:55.168 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:55.168 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:55.168 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:55.168 INFO:teuthology.orchestra.run.smithi081.stdout: ], 2024-09-21T21:42:55.168 INFO:teuthology.orchestra.run.smithi081.stdout: "new_purged_snaps": [], 2024-09-21T21:42:55.168 INFO:teuthology.orchestra.run.smithi081.stdout: "crush_node_flags": {}, 2024-09-21T21:42:55.168 INFO:teuthology.orchestra.run.smithi081.stdout: "device_class_flags": {}, 2024-09-21T21:42:55.168 INFO:teuthology.orchestra.run.smithi081.stdout: "stretch_mode": { 2024-09-21T21:42:55.168 INFO:teuthology.orchestra.run.smithi081.stdout: "stretch_mode_enabled": false, 2024-09-21T21:42:55.168 INFO:teuthology.orchestra.run.smithi081.stdout: "stretch_bucket_count": 0, 2024-09-21T21:42:55.168 INFO:teuthology.orchestra.run.smithi081.stdout: "degraded_stretch_mode": 0, 2024-09-21T21:42:55.168 INFO:teuthology.orchestra.run.smithi081.stdout: "recovering_stretch_mode": 0, 2024-09-21T21:42:55.168 INFO:teuthology.orchestra.run.smithi081.stdout: "stretch_mode_bucket": 0 2024-09-21T21:42:55.169 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:55.169 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:55.169 INFO:tasks.thrashosds.thrasher:j is {'epoch': 132, 'fsid': 'a0fdcfc2-ddd2-4e50-81bb-876d940f3578', 'created': '2024-09-21T21:40:07.413917+0000', 'modified': '2024-09-21T21:42:54.229880+0000', 'last_up_change': '2024-09-21T21:40:18.340430+0000', 'last_in_change': '2024-09-21T21:42:11.387597+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T21:40:18.321328+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.8871689438819885, 'average_primary_affinity': 0.8899999856948853, 'average_primary_affinity_weighted': 0.8899999856948853}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-21T21:40:24.235658+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 19, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.240000009536743, 'score_stable': 2.240000009536743, 'optimal_score': 0.6700000166893005, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.8871689438819885, 'average_primary_affinity': 0.8899999856948853, 'average_primary_affinity_weighted': 0.8899999856948853}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-21T21:40:36.093691+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '132', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 48, 'snap_epoch': 132, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.240000009536743, 'score_stable': 2.240000009536743, 'optimal_score': 0.6700000166893005, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.8871689438819885, 'average_primary_affinity': 0.8899999856948853, 'average_primary_affinity_weighted': 0.8899999856948853}}], 'osds': [{'osd': 0, 'uuid': 'a9869830-8e74-4b02-963d-ab630a0ee286', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 88, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6808', 'nonce': 1657552338}, {'type': 'v1', 'addr': '172.21.15.81:6809', 'nonce': 1657552338}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6810', 'nonce': 1657552338}, {'type': 'v1', 'addr': '172.21.15.81:6811', 'nonce': 1657552338}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6814', 'nonce': 1657552338}, {'type': 'v1', 'addr': '172.21.15.81:6815', 'nonce': 1657552338}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6812', 'nonce': 1657552338}, {'type': 'v1', 'addr': '172.21.15.81:6813', 'nonce': 1657552338}]}, 'public_addr': '172.21.15.81:6809/1657552338', 'cluster_addr': '172.21.15.81:6811/1657552338', 'heartbeat_back_addr': '172.21.15.81:6815/1657552338', 'heartbeat_front_addr': '172.21.15.81:6813/1657552338', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'b189819d-69a3-4e95-96e7-96dc4809ac80', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 100, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6816', 'nonce': 2864159502}, {'type': 'v1', 'addr': '172.21.15.81:6817', 'nonce': 2864159502}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6818', 'nonce': 2864159502}, {'type': 'v1', 'addr': '172.21.15.81:6819', 'nonce': 2864159502}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6822', 'nonce': 2864159502}, {'type': 'v1', 'addr': '172.21.15.81:6823', 'nonce': 2864159502}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6820', 'nonce': 2864159502}, {'type': 'v1', 'addr': '172.21.15.81:6821', 'nonce': 2864159502}]}, 'public_addr': '172.21.15.81:6817/2864159502', 'cluster_addr': '172.21.15.81:6819/2864159502', 'heartbeat_back_addr': '172.21.15.81:6823/2864159502', 'heartbeat_front_addr': '172.21.15.81:6821/2864159502', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '8437b99a-1297-4903-ab66-95c5cba2b2bf', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.3230133056640625, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 102, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6800', 'nonce': 3023160932}, {'type': 'v1', 'addr': '172.21.15.81:6801', 'nonce': 3023160932}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6802', 'nonce': 3023160932}, {'type': 'v1', 'addr': '172.21.15.81:6803', 'nonce': 3023160932}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6806', 'nonce': 3023160932}, {'type': 'v1', 'addr': '172.21.15.81:6807', 'nonce': 3023160932}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6804', 'nonce': 3023160932}, {'type': 'v1', 'addr': '172.21.15.81:6805', 'nonce': 3023160932}]}, 'public_addr': '172.21.15.81:6801/3023160932', 'cluster_addr': '172.21.15.81:6803/3023160932', 'heartbeat_back_addr': '172.21.15.81:6807/3023160932', 'heartbeat_front_addr': '172.21.15.81:6805/3023160932', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'e8848768-c1aa-445b-9554-dc6aa4eefc8d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 100, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6808', 'nonce': 2681016844}, {'type': 'v1', 'addr': '172.21.15.122:6809', 'nonce': 2681016844}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6810', 'nonce': 2681016844}, {'type': 'v1', 'addr': '172.21.15.122:6811', 'nonce': 2681016844}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6814', 'nonce': 2681016844}, {'type': 'v1', 'addr': '172.21.15.122:6815', 'nonce': 2681016844}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6812', 'nonce': 2681016844}, {'type': 'v1', 'addr': '172.21.15.122:6813', 'nonce': 2681016844}]}, 'public_addr': '172.21.15.122:6809/2681016844', 'cluster_addr': '172.21.15.122:6811/2681016844', 'heartbeat_back_addr': '172.21.15.122:6815/2681016844', 'heartbeat_front_addr': '172.21.15.122:6813/2681016844', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'b1d8e85d-a207-4ba6-9322-61d07812083e', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 99, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6800', 'nonce': 1095023540}, {'type': 'v1', 'addr': '172.21.15.122:6801', 'nonce': 1095023540}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6802', 'nonce': 1095023540}, {'type': 'v1', 'addr': '172.21.15.122:6803', 'nonce': 1095023540}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6806', 'nonce': 1095023540}, {'type': 'v1', 'addr': '172.21.15.122:6807', 'nonce': 1095023540}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6804', 'nonce': 1095023540}, {'type': 'v1', 'addr': '172.21.15.122:6805', 'nonce': 1095023540}]}, 'public_addr': '172.21.15.122:6801/1095023540', 'cluster_addr': '172.21.15.122:6803/1095023540', 'heartbeat_back_addr': '172.21.15.122:6807/1095023540', 'heartbeat_front_addr': '172.21.15.122:6805/1095023540', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '3ab0450b-d8fd-4ef2-a298-08d5aa2ea32f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 106, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6816', 'nonce': 1974398333}, {'type': 'v1', 'addr': '172.21.15.122:6817', 'nonce': 1974398333}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6818', 'nonce': 1974398333}, {'type': 'v1', 'addr': '172.21.15.122:6819', 'nonce': 1974398333}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6822', 'nonce': 1974398333}, {'type': 'v1', 'addr': '172.21.15.122:6823', 'nonce': 1974398333}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6820', 'nonce': 1974398333}, {'type': 'v1', 'addr': '172.21.15.122:6821', 'nonce': 1974398333}]}, 'public_addr': '172.21.15.122:6817/1974398333', 'cluster_addr': '172.21.15.122:6819/1974398333', 'heartbeat_back_addr': '172.21.15.122:6823/1974398333', 'heartbeat_front_addr': '172.21.15.122:6821/1974398333', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-21T21:40:15.766648+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T21:40:15.457496+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T21:40:14.767039+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T21:40:15.355570+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-21T21:40:15.335729+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T21:40:15.614733+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.8', 'osds': [2, 3]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 30, 'length': 9}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 30, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-21T21:42:55.170 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T21:42:55.213 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:55.212+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:55.278 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:55.278 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:42:55.278 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.278 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4385.0:28833 3.1d 3:be5d5235:::rbd_object_map.1121524d2522:head [call lock.lock in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133)", 2024-09-21T21:42:55.278 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:55.274832+0000", 2024-09-21T21:42:55.279 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.002098019, 2024-09-21T21:42:55.279 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021154419999999999, 2024-09-21T21:42:55.279 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:55.279 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:55.279 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "waiting for sub ops", 2024-09-21T21:42:55.279 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:55.280 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4385", 2024-09-21T21:42:55.280 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:42:55.280 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 28833 2024-09-21T21:42:55.280 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.280 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:55.280 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.280 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:55.280 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:55.274832+0000", 2024-09-21T21:42:55.280 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:55.280 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.280 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.281 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:55.281 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:55.274832+0000", 2024-09-21T21:42:55.281 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:55.281 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.281 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.281 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:55.281 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:55.274832+0000", 2024-09-21T21:42:55.281 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9100000000000003e-07 2024-09-21T21:42:55.281 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.281 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.281 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:55.281 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:55.274837+0000", 2024-09-21T21:42:55.281 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3279999999999999e-06 2024-09-21T21:42:55.282 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.282 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.282 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:55.282 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:55.274845+0000", 2024-09-21T21:42:55.282 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5739999999999996e-06 2024-09-21T21:42:55.282 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.282 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.282 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:55.282 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:55.274850+0000", 2024-09-21T21:42:55.282 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.899e-06 2024-09-21T21:42:55.282 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.282 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.282 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:55.283 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:55.274899+0000", 2024-09-21T21:42:55.283 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9249000000000002e-05 2024-09-21T21:42:55.283 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.283 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.283 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:55.283 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:55.274981+0000", 2024-09-21T21:42:55.283 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1637999999999993e-05 2024-09-21T21:42:55.283 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.283 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.283 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2024-09-21T21:42:55.283 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:55.275156+0000", 2024-09-21T21:42:55.284 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000174723 2024-09-21T21:42:55.284 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.284 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.284 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:55.284 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:55.276586+0000", 2024-09-21T21:42:55.284 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001430218 2024-09-21T21:42:55.284 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.285 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:55.285 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.285 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.285 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-21T21:42:55.285 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 1 2024-09-21T21:42:55.285 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:55.287 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T21:42:55.307 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:55.304+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:55.395 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:55.395 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:42:55.395 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:42:55.395 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:42:55.396 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:55.404 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T21:42:55.447 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:55.445+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:55.469 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:42:55.469 INFO:teuthology.orchestra.run.smithi081.stderr:dumped all 2024-09-21T21:42:55.481 INFO:teuthology.orchestra.run.smithi081.stdout:{"pg_ready":true,"pg_map":{"version":207,"stamp":"2024-09-21T21:42:55.245630+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":891023762,"num_objects":1090,"num_object_clones":154,"num_object_copies":2180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1090,"num_whiteouts":332,"num_read":401867,"num_read_kb":1465261,"num_write":26480,"num_write_kb":321767,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4206,"num_bytes_recovered":186957635,"num_keys_recovered":717,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22591,"ondisk_log_size":22591,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":309168,"kb_used_data":41404,"kb_used_omap":6,"kb_used_meta":267641,"kb_avail":377178192,"statfs":{"total":386547056640,"available":386230468608,"internally_reserved":0,"allocated":42397696,"data_stored":243664539,"data_compressed":16681528,"data_compressed_allocated":19124224,"data_compressed_original":227124736,"omap_allocated":6378,"internal_metadata":274065174},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-117379055,"num_objects":-150,"num_object_clones":-52,"num_object_copies":-300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-150,"num_whiteouts":-21,"num_read":1473,"num_read_kb":1264,"num_write":243,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.023341"},"pg_stats":[{"pgid":"3.1f","version":"131'534","reported_seq":7285,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T21:42:54.242592+0000","last_change":"2024-09-21T21:42:54.242922+0000","last_active":"2024-09-21T21:42:54.242592+0000","last_peered":"2024-09-21T21:42:54.242592+0000","last_clean":"2024-09-21T21:42:54.242592+0000","last_became_active":"2024-09-21T21:42:03.212171+0000","last_became_peered":"2024-09-21T21:42:03.212171+0000","last_unstale":"2024-09-21T21:42:54.242592+0000","last_undegraded":"2024-09-21T21:42:54.242592+0000","last_fullsized":"2024-09-21T21:42:54.242592+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":534,"log_dups_size":0,"ondisk_log_size":534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:38:54.724730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001835557,"stat_sum":{"num_bytes":24982592,"num_objects":38,"num_object_clones":8,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":10,"num_read":8173,"num_read_kb":24585,"num_write":518,"num_write_kb":8551,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":158,"num_bytes_recovered":3347584,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1e","length":"9"}]},{"pgid":"3.1e","version":"129'938","reported_seq":14959,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T21:42:54.243561+0000","last_change":"2024-09-21T21:42:54.243788+0000","last_active":"2024-09-21T21:42:54.243561+0000","last_peered":"2024-09-21T21:42:54.243561+0000","last_clean":"2024-09-21T21:42:54.243561+0000","last_became_active":"2024-09-21T21:41:53.215776+0000","last_became_peered":"2024-09-21T21:41:53.215776+0000","last_unstale":"2024-09-21T21:42:54.243561+0000","last_undegraded":"2024-09-21T21:42:54.243561+0000","last_fullsized":"2024-09-21T21:42:54.243561+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":938,"log_dups_size":0,"ondisk_log_size":938,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:31:37.534523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022931710000000001,"stat_sum":{"num_bytes":29175623,"num_objects":43,"num_object_clones":6,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":17,"num_read":15976,"num_read_kb":72907,"num_write":1411,"num_write_kb":18385,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":14291598,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1e","length":"9"}]},{"pgid":"3.1d","version":"125'916","reported_seq":10153,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-21T21:42:53.237743+0000","last_change":"2024-09-21T21:42:53.238296+0000","last_active":"2024-09-21T21:42:53.237743+0000","last_peered":"2024-09-21T21:42:53.237743+0000","last_clean":"2024-09-21T21:42:53.237743+0000","last_became_active":"2024-09-21T21:42:24.517759+0000","last_became_peered":"2024-09-21T21:42:24.517759+0000","last_unstale":"2024-09-21T21:42:53.237743+0000","last_undegraded":"2024-09-21T21:42:53.237743+0000","last_fullsized":"2024-09-21T21:42:53.237743+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":916,"log_dups_size":0,"ondisk_log_size":916,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:55:09.080138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014805370000000001,"stat_sum":{"num_bytes":30245006,"num_objects":36,"num_object_clones":4,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":13,"num_read":12998,"num_read_kb":43357,"num_write":1094,"num_write_kb":7894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":92,"num_bytes_recovered":3929994,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1f","length":"8"}]},{"pgid":"3.1c","version":"131'780","reported_seq":12406,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-21T21:42:53.283846+0000","last_change":"2024-09-21T21:42:53.237520+0000","last_active":"2024-09-21T21:42:53.283846+0000","last_peered":"2024-09-21T21:42:53.283846+0000","last_clean":"2024-09-21T21:42:53.283846+0000","last_became_active":"2024-09-21T21:42:24.516469+0000","last_became_peered":"2024-09-21T21:42:24.516469+0000","last_unstale":"2024-09-21T21:42:53.283846+0000","last_undegraded":"2024-09-21T21:42:53.283846+0000","last_fullsized":"2024-09-21T21:42:53.283846+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":780,"log_dups_size":0,"ondisk_log_size":780,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:32:05.376459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001077061,"stat_sum":{"num_bytes":19661914,"num_objects":29,"num_object_clones":4,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":10,"num_read":14662,"num_read_kb":34580,"num_write":810,"num_write_kb":7795,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":1229402,"num_keys_recovered":99,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1f","length":"8"}]},{"pgid":"3.1b","version":"131'705","reported_seq":8465,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T21:42:54.242699+0000","last_change":"2024-09-21T21:42:54.243316+0000","last_active":"2024-09-21T21:42:54.242699+0000","last_peered":"2024-09-21T21:42:54.242699+0000","last_clean":"2024-09-21T21:42:54.242699+0000","last_became_active":"2024-09-21T21:42:16.416438+0000","last_became_peered":"2024-09-21T21:42:16.416438+0000","last_unstale":"2024-09-21T21:42:54.242699+0000","last_undegraded":"2024-09-21T21:42:54.242699+0000","last_fullsized":"2024-09-21T21:42:54.242699+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":705,"log_dups_size":0,"ondisk_log_size":705,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:47:18.320073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002158507,"stat_sum":{"num_bytes":37194320,"num_objects":25,"num_object_clones":3,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":7,"num_read":8588,"num_read_kb":37196,"num_write":815,"num_write_kb":4966,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":223,"num_bytes_recovered":1599826,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1e","length":"9"}]},{"pgid":"3.1a","version":"129'542","reported_seq":24788,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-21T21:42:53.289630+0000","last_change":"2024-09-21T21:42:53.245156+0000","last_active":"2024-09-21T21:42:53.289630+0000","last_peered":"2024-09-21T21:42:53.289630+0000","last_clean":"2024-09-21T21:42:53.289630+0000","last_became_active":"2024-09-21T21:42:15.416423+0000","last_became_peered":"2024-09-21T21:42:15.416423+0000","last_unstale":"2024-09-21T21:42:53.289630+0000","last_undegraded":"2024-09-21T21:42:53.289630+0000","last_fullsized":"2024-09-21T21:42:53.289630+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":542,"log_dups_size":0,"ondisk_log_size":542,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:22:03.273638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010780207,"stat_sum":{"num_bytes":23754871,"num_objects":34,"num_object_clones":2,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":12,"num_read":24513,"num_read_kb":58641,"num_write":515,"num_write_kb":7766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":4395677,"num_keys_recovered":22,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1f","length":"8"}]},{"pgid":"3.19","version":"131'772","reported_seq":13396,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-21T21:42:53.244361+0000","last_change":"2024-09-21T21:42:53.244943+0000","last_active":"2024-09-21T21:42:53.244361+0000","last_peered":"2024-09-21T21:42:53.244361+0000","last_clean":"2024-09-21T21:42:53.244361+0000","last_became_active":"2024-09-21T21:42:15.421101+0000","last_became_peered":"2024-09-21T21:42:15.421101+0000","last_unstale":"2024-09-21T21:42:53.244361+0000","last_undegraded":"2024-09-21T21:42:53.244361+0000","last_fullsized":"2024-09-21T21:42:53.244361+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":772,"log_dups_size":0,"ondisk_log_size":772,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:52:01.349419+0000","scrub_duration":0,"objects_trimmed":3,"snaptrim_duration":0.011223633,"stat_sum":{"num_bytes":27163798,"num_objects":49,"num_object_clones":5,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":19,"num_read":15601,"num_read_kb":61354,"num_write":789,"num_write_kb":15363,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":110,"num_bytes_recovered":9861823,"num_keys_recovered":35,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1f","length":"8"}]},{"pgid":"3.18","version":"128'307","reported_seq":6703,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-21T21:42:53.233532+0000","last_change":"2024-09-21T21:42:53.234206+0000","last_active":"2024-09-21T21:42:53.233532+0000","last_peered":"2024-09-21T21:42:53.233532+0000","last_clean":"2024-09-21T21:42:53.233532+0000","last_became_active":"2024-09-21T21:42:03.211916+0000","last_became_peered":"2024-09-21T21:42:03.211916+0000","last_unstale":"2024-09-21T21:42:53.233532+0000","last_undegraded":"2024-09-21T21:42:53.233532+0000","last_fullsized":"2024-09-21T21:42:53.233532+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":307,"log_dups_size":0,"ondisk_log_size":307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:39:20.005198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087651700000000005,"stat_sum":{"num_bytes":33874960,"num_objects":29,"num_object_clones":6,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":7,"num_read":6158,"num_read_kb":33022,"num_write":309,"num_write_kb":8639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":59,"num_bytes_recovered":3271696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1f","length":"8"}]},{"pgid":"3.17","version":"129'429","reported_seq":6026,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-21T21:42:53.237575+0000","last_change":"2024-09-21T21:42:53.238062+0000","last_active":"2024-09-21T21:42:53.237575+0000","last_peered":"2024-09-21T21:42:53.237575+0000","last_clean":"2024-09-21T21:42:53.237575+0000","last_became_active":"2024-09-21T21:42:03.213418+0000","last_became_peered":"2024-09-21T21:42:03.213418+0000","last_unstale":"2024-09-21T21:42:53.237575+0000","last_undegraded":"2024-09-21T21:42:53.237575+0000","last_fullsized":"2024-09-21T21:42:53.237575+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":429,"log_dups_size":0,"ondisk_log_size":429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:11:47.793334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013151339999999999,"stat_sum":{"num_bytes":20431994,"num_objects":38,"num_object_clones":7,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":9,"num_read":6216,"num_read_kb":25745,"num_write":471,"num_write_kb":9168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":3723386,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1f","length":"8"}]},{"pgid":"3.16","version":"131'1167","reported_seq":9459,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-21T21:42:53.317137+0000","last_change":"2024-09-21T21:42:53.239060+0000","last_active":"2024-09-21T21:42:53.317137+0000","last_peered":"2024-09-21T21:42:53.317137+0000","last_clean":"2024-09-21T21:42:53.317137+0000","last_became_active":"2024-09-21T21:41:53.214270+0000","last_became_peered":"2024-09-21T21:41:53.214270+0000","last_unstale":"2024-09-21T21:42:53.317137+0000","last_undegraded":"2024-09-21T21:42:53.317137+0000","last_fullsized":"2024-09-21T21:42:53.317137+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":1167,"log_dups_size":0,"ondisk_log_size":1167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:00:49.975862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020954429999999998,"stat_sum":{"num_bytes":30373962,"num_objects":38,"num_object_clones":6,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":10,"num_read":13028,"num_read_kb":44208,"num_write":1744,"num_write_kb":12830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":148,"num_bytes_recovered":4869268,"num_keys_recovered":47,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1f","length":"8"}]},{"pgid":"3.15","version":"129'706","reported_seq":8927,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T21:42:54.251015+0000","last_change":"2024-09-21T21:42:54.251246+0000","last_active":"2024-09-21T21:42:54.251015+0000","last_peered":"2024-09-21T21:42:54.251015+0000","last_clean":"2024-09-21T21:42:54.251015+0000","last_became_active":"2024-09-21T21:42:16.418172+0000","last_became_peered":"2024-09-21T21:42:16.418172+0000","last_unstale":"2024-09-21T21:42:54.251015+0000","last_undegraded":"2024-09-21T21:42:54.251015+0000","last_fullsized":"2024-09-21T21:42:54.251015+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":706,"log_dups_size":0,"ondisk_log_size":706,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:42:34.167914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0095812740000000007,"stat_sum":{"num_bytes":26579435,"num_objects":32,"num_object_clones":4,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":9,"num_read":9166,"num_read_kb":56836,"num_write":947,"num_write_kb":7167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":385,"num_bytes_recovered":9565806,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1e","length":"9"}]},{"pgid":"3.14","version":"129'558","reported_seq":21228,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T21:42:54.241657+0000","last_change":"2024-09-21T21:42:54.242060+0000","last_active":"2024-09-21T21:42:54.241657+0000","last_peered":"2024-09-21T21:42:54.241657+0000","last_clean":"2024-09-21T21:42:54.241657+0000","last_became_active":"2024-09-21T21:42:16.416899+0000","last_became_peered":"2024-09-21T21:42:16.416899+0000","last_unstale":"2024-09-21T21:42:54.241657+0000","last_undegraded":"2024-09-21T21:42:54.241657+0000","last_fullsized":"2024-09-21T21:42:54.241657+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":558,"log_dups_size":0,"ondisk_log_size":558,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:13:41.369787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001306495,"stat_sum":{"num_bytes":19497124,"num_objects":26,"num_object_clones":4,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":7,"num_read":21741,"num_read_kb":74871,"num_write":466,"num_write_kb":10968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":227,"num_bytes_recovered":15558124,"num_keys_recovered":84,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1e","length":"9"}]},{"pgid":"2.7","version":"0'0","reported_seq":125,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T21:42:54.241235+0000","last_change":"2024-09-21T21:42:12.214789+0000","last_active":"2024-09-21T21:42:54.241235+0000","last_peered":"2024-09-21T21:42:54.241235+0000","last_clean":"2024-09-21T21:42:54.241235+0000","last_became_active":"2024-09-21T21:42:12.214436+0000","last_became_peered":"2024-09-21T21:42:12.214436+0000","last_unstale":"2024-09-21T21:42:54.241235+0000","last_undegraded":"2024-09-21T21:42:54.241235+0000","last_fullsized":"2024-09-21T21:42:54.241235+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:45:03.040098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"131'1149","reported_seq":13868,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-21T21:42:53.240906+0000","last_change":"2024-09-21T21:42:53.241358+0000","last_active":"2024-09-21T21:42:53.240906+0000","last_peered":"2024-09-21T21:42:53.240906+0000","last_clean":"2024-09-21T21:42:53.240906+0000","last_became_active":"2024-09-21T21:42:15.419283+0000","last_became_peered":"2024-09-21T21:42:15.419283+0000","last_unstale":"2024-09-21T21:42:53.240906+0000","last_undegraded":"2024-09-21T21:42:53.240906+0000","last_fullsized":"2024-09-21T21:42:53.240906+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":1149,"log_dups_size":0,"ondisk_log_size":1149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:29:21.260384+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0079644309999999992,"stat_sum":{"num_bytes":34618368,"num_objects":33,"num_object_clones":6,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":13,"num_read":15587,"num_read_kb":66119,"num_write":1544,"num_write_kb":15981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":193,"num_bytes_recovered":6876276,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1f","length":"8"}]},{"pgid":"2.6","version":"0'0","reported_seq":254,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-21T21:42:53.234168+0000","last_change":"2024-09-21T21:41:49.266534+0000","last_active":"2024-09-21T21:42:53.234168+0000","last_peered":"2024-09-21T21:42:53.234168+0000","last_clean":"2024-09-21T21:42:53.234168+0000","last_became_active":"2024-09-21T21:41:49.266257+0000","last_became_peered":"2024-09-21T21:41:49.266257+0000","last_unstale":"2024-09-21T21:42:53.234168+0000","last_undegraded":"2024-09-21T21:42:53.234168+0000","last_fullsized":"2024-09-21T21:42:53.234168+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:38:31.063193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062280799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"131'491","reported_seq":8409,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-21T21:42:53.246836+0000","last_change":"2024-09-21T21:42:53.247083+0000","last_active":"2024-09-21T21:42:53.246836+0000","last_peered":"2024-09-21T21:42:53.246836+0000","last_clean":"2024-09-21T21:42:53.246836+0000","last_became_active":"2024-09-21T21:41:25.651977+0000","last_became_peered":"2024-09-21T21:41:25.651977+0000","last_unstale":"2024-09-21T21:42:53.246836+0000","last_undegraded":"2024-09-21T21:42:53.246836+0000","last_fullsized":"2024-09-21T21:42:53.246836+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":491,"log_dups_size":0,"ondisk_log_size":491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:57:16.419714+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.013068556,"stat_sum":{"num_bytes":18375205,"num_objects":28,"num_object_clones":3,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":9,"num_read":10282,"num_read_kb":40174,"num_write":476,"num_write_kb":10594,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1f","length":"8"}]},{"pgid":"2.5","version":"0'0","reported_seq":130,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T21:42:54.241281+0000","last_change":"2024-09-21T21:42:12.214946+0000","last_active":"2024-09-21T21:42:54.241281+0000","last_peered":"2024-09-21T21:42:54.241281+0000","last_clean":"2024-09-21T21:42:54.241281+0000","last_became_active":"2024-09-21T21:42:12.214648+0000","last_became_peered":"2024-09-21T21:42:12.214648+0000","last_unstale":"2024-09-21T21:42:54.241281+0000","last_undegraded":"2024-09-21T21:42:54.241281+0000","last_fullsized":"2024-09-21T21:42:54.241281+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:54:09.452995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"129'845","reported_seq":4459,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-21T21:42:53.241418+0000","last_change":"2024-09-21T21:42:53.244140+0000","last_active":"2024-09-21T21:42:53.241418+0000","last_peered":"2024-09-21T21:42:53.241418+0000","last_clean":"2024-09-21T21:42:53.241418+0000","last_became_active":"2024-09-21T21:40:37.419944+0000","last_became_peered":"2024-09-21T21:40:37.419944+0000","last_unstale":"2024-09-21T21:42:53.241418+0000","last_undegraded":"2024-09-21T21:42:53.241418+0000","last_fullsized":"2024-09-21T21:42:53.241418+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":845,"log_dups_size":0,"ondisk_log_size":845,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:25:29.728614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010427929000000001,"stat_sum":{"num_bytes":9993207,"num_objects":27,"num_object_clones":4,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":7,"num_read":5886,"num_read_kb":25228,"num_write":1234,"num_write_kb":11316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1f","length":"8"}]},{"pgid":"2.4","version":"0'0","reported_seq":254,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-21T21:42:53.233834+0000","last_change":"2024-09-21T21:42:12.214652+0000","last_active":"2024-09-21T21:42:53.233834+0000","last_peered":"2024-09-21T21:42:53.233834+0000","last_clean":"2024-09-21T21:42:53.233834+0000","last_became_active":"2024-09-21T21:42:12.214348+0000","last_became_peered":"2024-09-21T21:42:12.214348+0000","last_unstale":"2024-09-21T21:42:53.233834+0000","last_undegraded":"2024-09-21T21:42:53.233834+0000","last_fullsized":"2024-09-21T21:42:53.233834+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:55:56.386112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070669899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"131'587","reported_seq":10113,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-21T21:42:53.241464+0000","last_change":"2024-09-21T21:42:53.241839+0000","last_active":"2024-09-21T21:42:53.241464+0000","last_peered":"2024-09-21T21:42:53.241464+0000","last_clean":"2024-09-21T21:42:53.241464+0000","last_became_active":"2024-09-21T21:41:25.650379+0000","last_became_peered":"2024-09-21T21:41:25.650379+0000","last_unstale":"2024-09-21T21:42:53.241464+0000","last_undegraded":"2024-09-21T21:42:53.241464+0000","last_fullsized":"2024-09-21T21:42:53.241464+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":587,"log_dups_size":0,"ondisk_log_size":587,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:13:24.408550+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0049841549999999997,"stat_sum":{"num_bytes":38697007,"num_objects":35,"num_object_clones":4,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":10,"num_read":11123,"num_read_kb":31913,"num_write":478,"num_write_kb":8333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1f","length":"8"}]},{"pgid":"3.3","version":"132'498","reported_seq":10587,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T21:42:54.311627+0000","last_change":"2024-09-21T21:42:54.243522+0000","last_active":"2024-09-21T21:42:54.311627+0000","last_peered":"2024-09-21T21:42:54.311627+0000","last_clean":"2024-09-21T21:42:54.311627+0000","last_became_active":"2024-09-21T21:42:15.418642+0000","last_became_peered":"2024-09-21T21:42:15.418642+0000","last_unstale":"2024-09-21T21:42:54.311627+0000","last_undegraded":"2024-09-21T21:42:54.311627+0000","last_fullsized":"2024-09-21T21:42:54.311627+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":498,"log_dups_size":0,"ondisk_log_size":498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:44:08.793543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023681610000000001,"stat_sum":{"num_bytes":6427648,"num_objects":23,"num_object_clones":3,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":7,"num_read":9435,"num_read_kb":35587,"num_write":502,"num_write_kb":9733,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":171,"num_bytes_recovered":13125136,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1e","length":"9"}]},{"pgid":"2.2","version":"19'2","reported_seq":119,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-21T21:42:53.236046+0000","last_change":"2024-09-21T21:42:24.516929+0000","last_active":"2024-09-21T21:42:53.236046+0000","last_peered":"2024-09-21T21:42:53.236046+0000","last_clean":"2024-09-21T21:42:53.236046+0000","last_became_active":"2024-09-21T21:42:24.516623+0000","last_became_peered":"2024-09-21T21:42:24.516623+0000","last_unstale":"2024-09-21T21:42:53.236046+0000","last_undegraded":"2024-09-21T21:42:53.236046+0000","last_fullsized":"2024-09-21T21:42:53.236046+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:51:59.592292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000906268,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"130'736","reported_seq":10952,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-21T21:42:53.234950+0000","last_change":"2024-09-21T21:42:53.235392+0000","last_active":"2024-09-21T21:42:53.234950+0000","last_peered":"2024-09-21T21:42:53.234950+0000","last_clean":"2024-09-21T21:42:53.234950+0000","last_became_active":"2024-09-21T21:40:37.418973+0000","last_became_peered":"2024-09-21T21:40:37.418973+0000","last_unstale":"2024-09-21T21:42:53.234950+0000","last_undegraded":"2024-09-21T21:42:53.234950+0000","last_fullsized":"2024-09-21T21:42:53.234950+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":736,"log_dups_size":0,"ondisk_log_size":736,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:23:33.476781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001845803,"stat_sum":{"num_bytes":34270326,"num_objects":39,"num_object_clones":9,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":12,"num_read":13019,"num_read_kb":27794,"num_write":552,"num_write_kb":6281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1f","length":"8"}]},{"pgid":"2.1","version":"0'0","reported_seq":161,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-21T21:42:53.233767+0000","last_change":"2024-09-21T21:41:25.652142+0000","last_active":"2024-09-21T21:42:53.233767+0000","last_peered":"2024-09-21T21:42:53.233767+0000","last_clean":"2024-09-21T21:42:53.233767+0000","last_became_active":"2024-09-21T21:41:25.651832+0000","last_became_peered":"2024-09-21T21:41:25.651832+0000","last_unstale":"2024-09-21T21:42:53.233767+0000","last_undegraded":"2024-09-21T21:42:53.233767+0000","last_fullsized":"2024-09-21T21:42:53.233767+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:37:21.188619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"129'367","reported_seq":16334,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-21T21:42:53.236116+0000","last_change":"2024-09-21T21:42:53.237283+0000","last_active":"2024-09-21T21:42:53.236116+0000","last_peered":"2024-09-21T21:42:53.236116+0000","last_clean":"2024-09-21T21:42:53.236116+0000","last_became_active":"2024-09-21T21:42:24.518786+0000","last_became_peered":"2024-09-21T21:42:24.518786+0000","last_unstale":"2024-09-21T21:42:53.236116+0000","last_undegraded":"2024-09-21T21:42:53.236116+0000","last_fullsized":"2024-09-21T21:42:53.236116+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:52:53.018809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001342518,"stat_sum":{"num_bytes":40173134,"num_objects":34,"num_object_clones":4,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":18,"num_read":15385,"num_read_kb":48499,"num_write":365,"num_write_kb":4859,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":161,"num_bytes_recovered":5746844,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1f","length":"8"}]},{"pgid":"2.0","version":"0'0","reported_seq":120,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T21:42:54.240591+0000","last_change":"2024-09-21T21:41:49.265465+0000","last_active":"2024-09-21T21:42:54.240591+0000","last_peered":"2024-09-21T21:42:54.240591+0000","last_clean":"2024-09-21T21:42:54.240591+0000","last_became_active":"2024-09-21T21:41:49.265182+0000","last_became_peered":"2024-09-21T21:41:49.265182+0000","last_unstale":"2024-09-21T21:42:54.240591+0000","last_undegraded":"2024-09-21T21:42:54.240591+0000","last_fullsized":"2024-09-21T21:42:54.240591+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:19:17.973195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"129'720","reported_seq":9911,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-21T21:42:53.245223+0000","last_change":"2024-09-21T21:42:53.245675+0000","last_active":"2024-09-21T21:42:53.245223+0000","last_peered":"2024-09-21T21:42:53.245223+0000","last_clean":"2024-09-21T21:42:53.245223+0000","last_became_active":"2024-09-21T21:40:37.421655+0000","last_became_peered":"2024-09-21T21:40:37.421655+0000","last_unstale":"2024-09-21T21:42:53.245223+0000","last_undegraded":"2024-09-21T21:42:53.245223+0000","last_fullsized":"2024-09-21T21:42:53.245223+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":720,"log_dups_size":0,"ondisk_log_size":720,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:28:04.864611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011062621,"stat_sum":{"num_bytes":21689529,"num_objects":28,"num_object_clones":2,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":9,"num_read":12466,"num_read_kb":34170,"num_write":709,"num_write_kb":7488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1f","length":"8"}]},{"pgid":"2.3","version":"17'1","reported_seq":217,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-21T21:42:53.236185+0000","last_change":"2024-09-21T21:42:24.518810+0000","last_active":"2024-09-21T21:42:53.236185+0000","last_peered":"2024-09-21T21:42:53.236185+0000","last_clean":"2024-09-21T21:42:53.236185+0000","last_became_active":"2024-09-21T21:42:24.518461+0000","last_became_peered":"2024-09-21T21:42:24.518461+0000","last_unstale":"2024-09-21T21:42:53.236185+0000","last_undegraded":"2024-09-21T21:42:53.236185+0000","last_fullsized":"2024-09-21T21:42:53.236185+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:17:47.593497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036803999999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":314,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T21:42:54.240719+0000","last_change":"2024-09-21T21:42:03.212447+0000","last_active":"2024-09-21T21:42:54.240719+0000","last_peered":"2024-09-21T21:42:54.240719+0000","last_clean":"2024-09-21T21:42:54.240719+0000","last_became_active":"2024-09-21T21:42:03.211881+0000","last_became_peered":"2024-09-21T21:42:03.211881+0000","last_unstale":"2024-09-21T21:42:54.240719+0000","last_undegraded":"2024-09-21T21:42:54.240719+0000","last_fullsized":"2024-09-21T21:42:54.240719+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:18.340430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:18.340430+0000","last_clean_scrub_stamp":"2024-09-21T21:40:18.340430+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:09:02.519349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"131'334","reported_seq":6000,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T21:42:54.240938+0000","last_change":"2024-09-21T21:42:54.241372+0000","last_active":"2024-09-21T21:42:54.240938+0000","last_peered":"2024-09-21T21:42:54.240938+0000","last_clean":"2024-09-21T21:42:54.240938+0000","last_became_active":"2024-09-21T21:41:52.213238+0000","last_became_peered":"2024-09-21T21:41:52.213238+0000","last_unstale":"2024-09-21T21:42:54.240938+0000","last_undegraded":"2024-09-21T21:42:54.240938+0000","last_fullsized":"2024-09-21T21:42:54.240938+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":334,"log_dups_size":0,"ondisk_log_size":334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:41:07.620818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062345599999999999,"stat_sum":{"num_bytes":15578697,"num_objects":34,"num_object_clones":6,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":11,"num_read":5231,"num_read_kb":25946,"num_write":338,"num_write_kb":7274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":172,"num_bytes_recovered":2229995,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1e","length":"9"}]},{"pgid":"3.9","version":"129'498","reported_seq":8261,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-21T21:42:51.235264+0000","last_change":"2024-09-21T21:42:51.235523+0000","last_active":"2024-09-21T21:42:51.235264+0000","last_peered":"2024-09-21T21:42:51.235264+0000","last_clean":"2024-09-21T21:42:51.235264+0000","last_became_active":"2024-09-21T21:42:18.480134+0000","last_became_peered":"2024-09-21T21:42:18.480134+0000","last_unstale":"2024-09-21T21:42:51.235264+0000","last_undegraded":"2024-09-21T21:42:51.235264+0000","last_fullsized":"2024-09-21T21:42:51.235264+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":498,"log_dups_size":0,"ondisk_log_size":498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:27:31.859506+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0072582339999999997,"stat_sum":{"num_bytes":40670884,"num_objects":43,"num_object_clones":5,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":13,"num_read":8635,"num_read_kb":47244,"num_write":484,"num_write_kb":9428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":107,"num_bytes_recovered":6946045,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"21","length":"b"}]},{"pgid":"3.a","version":"131'982","reported_seq":10851,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T21:42:54.241611+0000","last_change":"2024-09-21T21:42:54.242665+0000","last_active":"2024-09-21T21:42:54.241611+0000","last_peered":"2024-09-21T21:42:54.241611+0000","last_clean":"2024-09-21T21:42:54.241611+0000","last_became_active":"2024-09-21T21:42:16.416788+0000","last_became_peered":"2024-09-21T21:42:16.416788+0000","last_unstale":"2024-09-21T21:42:54.241611+0000","last_undegraded":"2024-09-21T21:42:54.241611+0000","last_fullsized":"2024-09-21T21:42:54.241611+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":982,"log_dups_size":0,"ondisk_log_size":982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:49:50.667595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018958919999999999,"stat_sum":{"num_bytes":43136684,"num_objects":35,"num_object_clones":4,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":8,"num_read":12744,"num_read_kb":48760,"num_write":1167,"num_write_kb":10643,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":14636658,"num_keys_recovered":136,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1e","length":"9"}]},{"pgid":"3.b","version":"129'732","reported_seq":9711,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-21T21:42:53.234893+0000","last_change":"2024-09-21T21:42:53.234893+0000","last_active":"2024-09-21T21:42:53.234893+0000","last_peered":"2024-09-21T21:42:53.234893+0000","last_clean":"2024-09-21T21:42:53.234893+0000","last_became_active":"2024-09-21T21:42:16.416994+0000","last_became_peered":"2024-09-21T21:42:16.416994+0000","last_unstale":"2024-09-21T21:42:53.234893+0000","last_undegraded":"2024-09-21T21:42:53.234893+0000","last_fullsized":"2024-09-21T21:42:53.234893+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":732,"log_dups_size":0,"ondisk_log_size":732,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:23:04.202185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014868520000000001,"stat_sum":{"num_bytes":23798581,"num_objects":38,"num_object_clones":6,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":10,"num_read":10197,"num_read_kb":45337,"num_write":1099,"num_write_kb":14641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":189,"num_bytes_recovered":7872106,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1f","length":"8"}]},{"pgid":"3.c","version":"129'1571","reported_seq":13424,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-21T21:42:53.237700+0000","last_change":"2024-09-21T21:42:53.237700+0000","last_active":"2024-09-21T21:42:53.237700+0000","last_peered":"2024-09-21T21:42:53.237700+0000","last_clean":"2024-09-21T21:42:53.237700+0000","last_became_active":"2024-09-21T21:42:24.517955+0000","last_became_peered":"2024-09-21T21:42:24.517955+0000","last_unstale":"2024-09-21T21:42:53.237700+0000","last_undegraded":"2024-09-21T21:42:53.237700+0000","last_fullsized":"2024-09-21T21:42:53.237700+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":1571,"log_dups_size":0,"ondisk_log_size":1571,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:26:49.515326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001212124,"stat_sum":{"num_bytes":34539284,"num_objects":35,"num_object_clones":7,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":8,"num_read":18521,"num_read_kb":61279,"num_write":2309,"num_write_kb":11533,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":4225300,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1f","length":"8"}]},{"pgid":"3.d","version":"132'1382","reported_seq":17371,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T21:42:54.250773+0000","last_change":"2024-09-21T21:42:54.250973+0000","last_active":"2024-09-21T21:42:54.250773+0000","last_peered":"2024-09-21T21:42:54.250773+0000","last_clean":"2024-09-21T21:42:54.250773+0000","last_became_active":"2024-09-21T21:41:52.214022+0000","last_became_peered":"2024-09-21T21:41:52.214022+0000","last_unstale":"2024-09-21T21:42:54.250773+0000","last_undegraded":"2024-09-21T21:42:54.250773+0000","last_fullsized":"2024-09-21T21:42:54.250773+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":1382,"log_dups_size":0,"ondisk_log_size":1382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:26:40.264842+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0094351460000000002,"stat_sum":{"num_bytes":26429175,"num_objects":36,"num_object_clones":5,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":8,"num_read":20983,"num_read_kb":45388,"num_write":1794,"num_write_kb":14786,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":149,"num_bytes_recovered":3462638,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1e","length":"9"}]},{"pgid":"3.e","version":"131'420","reported_seq":7333,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-21T21:42:53.298857+0000","last_change":"2024-09-21T21:42:53.242198+0000","last_active":"2024-09-21T21:42:53.298857+0000","last_peered":"2024-09-21T21:42:53.298857+0000","last_clean":"2024-09-21T21:42:53.298857+0000","last_became_active":"2024-09-21T21:42:24.517033+0000","last_became_peered":"2024-09-21T21:42:24.517033+0000","last_unstale":"2024-09-21T21:42:53.298857+0000","last_undegraded":"2024-09-21T21:42:53.298857+0000","last_fullsized":"2024-09-21T21:42:53.298857+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":420,"log_dups_size":0,"ondisk_log_size":420,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:32:08.819594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0051787279999999996,"stat_sum":{"num_bytes":25635968,"num_objects":29,"num_object_clones":2,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":12,"num_read":6511,"num_read_kb":34822,"num_write":436,"num_write_kb":6854,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":3302528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1f","length":"8"}]},{"pgid":"3.f","version":"131'649","reported_seq":16618,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-21T21:42:53.242877+0000","last_change":"2024-09-21T21:42:53.242877+0000","last_active":"2024-09-21T21:42:53.242877+0000","last_peered":"2024-09-21T21:42:53.242877+0000","last_clean":"2024-09-21T21:42:53.242877+0000","last_became_active":"2024-09-21T21:41:52.212206+0000","last_became_peered":"2024-09-21T21:41:52.212206+0000","last_unstale":"2024-09-21T21:42:53.242877+0000","last_undegraded":"2024-09-21T21:42:53.242877+0000","last_fullsized":"2024-09-21T21:42:53.242877+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":649,"log_dups_size":0,"ondisk_log_size":649,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:28:05.212403+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0058979480000000001,"stat_sum":{"num_bytes":9309351,"num_objects":40,"num_object_clones":8,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":10,"num_read":17840,"num_read_kb":41939,"num_write":605,"num_write_kb":9488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":2967886,"num_keys_recovered":48,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1f","length":"8"}]},{"pgid":"3.10","version":"129'379","reported_seq":13708,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-21T21:42:53.236012+0000","last_change":"2024-09-21T21:42:53.236817+0000","last_active":"2024-09-21T21:42:53.236012+0000","last_peered":"2024-09-21T21:42:53.236012+0000","last_clean":"2024-09-21T21:42:53.236012+0000","last_became_active":"2024-09-21T21:42:24.516842+0000","last_became_peered":"2024-09-21T21:42:24.516842+0000","last_unstale":"2024-09-21T21:42:53.236012+0000","last_undegraded":"2024-09-21T21:42:53.236012+0000","last_fullsized":"2024-09-21T21:42:53.236012+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":379,"log_dups_size":0,"ondisk_log_size":379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:40:43.902132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098695399999999996,"stat_sum":{"num_bytes":46997520,"num_objects":33,"num_object_clones":5,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":9,"num_read":12971,"num_read_kb":55236,"num_write":405,"num_write_kb":11231,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":3599391,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1f","length":"8"}]},{"pgid":"3.11","version":"129'604","reported_seq":7681,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T21:42:54.242125+0000","last_change":"2024-09-21T21:42:54.242541+0000","last_active":"2024-09-21T21:42:54.242125+0000","last_peered":"2024-09-21T21:42:54.242125+0000","last_clean":"2024-09-21T21:42:54.242125+0000","last_became_active":"2024-09-21T21:42:15.418634+0000","last_became_peered":"2024-09-21T21:42:15.418634+0000","last_unstale":"2024-09-21T21:42:54.242125+0000","last_undegraded":"2024-09-21T21:42:54.242125+0000","last_fullsized":"2024-09-21T21:42:54.242125+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":604,"log_dups_size":0,"ondisk_log_size":604,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:58:17.390660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016105589999999999,"stat_sum":{"num_bytes":26224165,"num_objects":32,"num_object_clones":3,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":9,"num_read":7885,"num_read_kb":33887,"num_write":697,"num_write_kb":6863,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":228,"num_bytes_recovered":3550319,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1e","length":"9"}]},{"pgid":"3.12","version":"132'730","reported_seq":18985,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T21:42:54.318718+0000","last_change":"2024-09-21T21:42:54.253785+0000","last_active":"2024-09-21T21:42:54.318718+0000","last_peered":"2024-09-21T21:42:54.318718+0000","last_clean":"2024-09-21T21:42:54.318718+0000","last_became_active":"2024-09-21T21:42:03.211865+0000","last_became_peered":"2024-09-21T21:42:03.211865+0000","last_unstale":"2024-09-21T21:42:54.318718+0000","last_undegraded":"2024-09-21T21:42:54.318718+0000","last_fullsized":"2024-09-21T21:42:54.318718+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":730,"log_dups_size":0,"ondisk_log_size":730,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:26:03.135971+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.012216659,"stat_sum":{"num_bytes":41423872,"num_objects":39,"num_object_clones":7,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":12,"num_read":20617,"num_read_kb":71635,"num_write":760,"num_write_kb":9978,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":211,"num_bytes_recovered":13613603,"num_keys_recovered":81,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1e","length":"9"}]},{"pgid":"3.13","version":"132'443","reported_seq":10372,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T21:42:54.248588+0000","last_change":"2024-09-21T21:42:54.248849+0000","last_active":"2024-09-21T21:42:54.248588+0000","last_peered":"2024-09-21T21:42:54.248588+0000","last_clean":"2024-09-21T21:42:54.248588+0000","last_became_active":"2024-09-21T21:41:52.209710+0000","last_became_peered":"2024-09-21T21:41:52.209710+0000","last_unstale":"2024-09-21T21:42:54.248588+0000","last_undegraded":"2024-09-21T21:42:54.248588+0000","last_fullsized":"2024-09-21T21:42:54.248588+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":443,"log_dups_size":0,"ondisk_log_size":443,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:15:30.737024+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.007551894,"stat_sum":{"num_bytes":29509171,"num_objects":28,"num_object_clones":2,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":7,"num_read":9613,"num_read_kb":76902,"num_write":476,"num_write_kb":12553,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":146,"num_bytes_recovered":17387622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1e","length":"9"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":890433375,"num_objects":1086,"num_object_clones":154,"num_object_copies":2172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1086,"num_whiteouts":332,"num_read":401751,"num_read_kb":1465161,"num_write":26319,"num_write_kb":319349,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4200,"num_bytes_recovered":185186531,"num_keys_recovered":717,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":53686272,"data_stored":364478102,"data_compressed":25299625,"data_compressed_allocated":29118464,"data_compressed_original":344479232,"omap_allocated":0,"internal_metadata":0},"log_size":22471,"ondisk_log_size":22471,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":1771104,"data_compressed":100455,"data_compressed_allocated":135168,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":4,"up_from":0,"seq":55834574882,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574882,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":103592,"kb_used_data":9920,"kb_used_omap":1,"kb_used_meta":93630,"kb_avail":94268248,"statfs":{"total":96636764160,"available":96530685952,"internally_reserved":0,"allocated":10158080,"data_stored":63628559,"data_compressed":4663119,"data_compressed_allocated":5148672,"data_compressed_original":60166656,"omap_allocated":1593,"internal_metadata":95877575},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":12,"seq":51539607586,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78136,"kb_used_data":10848,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94293704,"statfs":{"total":96636764160,"available":96556752896,"internally_reserved":0,"allocated":11108352,"data_stored":60216429,"data_compressed":4154128,"data_compressed_allocated":4812800,"data_compressed_original":55620096,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607586,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60724,"kb_used_data":11480,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94311116,"statfs":{"total":96636764160,"available":96574582784,"internally_reserved":0,"allocated":11755520,"data_stored":58025739,"data_compressed":4051630,"data_compressed_allocated":5070848,"data_compressed_original":53298688,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":0,"seq":51539607587,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640290,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66716,"kb_used_data":9156,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94305124,"statfs":{"total":96636764160,"available":96568446976,"internally_reserved":0,"allocated":9375744,"data_stored":61793812,"data_compressed":3812651,"data_compressed_allocated":4091904,"data_compressed_original":58039296,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32395,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34030,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34030,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":7696384,"data_stored":53427314,"data_compressed":3500155,"data_compressed_allocated":3907584,"data_compressed_original":50207744,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9629696,"data_stored":51416118,"data_compressed":3772768,"data_compressed_allocated":4694016,"data_compressed_original":47515136,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":7856128,"data_stored":49628090,"data_compressed":3188307,"data_compressed_allocated":4091904,"data_compressed_original":46504960,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8429568,"data_stored":62274885,"data_compressed":4629089,"data_compressed_allocated":5103616,"data_compressed_original":59576832,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":10522624,"data_stored":88221495,"data_compressed":6055770,"data_compressed_allocated":6512640,"data_compressed_original":85066752,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":9551872,"data_stored":59510200,"data_compressed":4153536,"data_compressed_allocated":4808704,"data_compressed_original":55607808,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T21:42:55.483 INFO:tasks.thrashosds.thrasher:Setting 3.13 to [1, 4, 2, 5] 2024-09-21T21:42:55.483 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.13', '1', '4', '2', '5'] 2024-09-21T21:42:55.483 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.13 1 4 2 5 2024-09-21T21:42:55.508 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:55.508 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-21T21:42:55.508 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-21T21:42:55.508 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:42:55.508 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.508 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4385.0:15202 3.16 3:6888948b:::rbd_header.1121493f6459:head [watch reconnect cookie 140325440202496 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-21T21:42:55.508 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:41.736601+0000", 2024-09-21T21:42:55.509 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 73.769716590000002, 2024-09-21T21:42:55.509 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5040056769999999, 2024-09-21T21:42:55.509 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:55.509 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:55.509 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:55.509 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:55.509 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4385", 2024-09-21T21:42:55.509 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:42:55.509 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 15202 2024-09-21T21:42:55.509 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.509 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:55.509 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.509 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:55.510 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736601+0000", 2024-09-21T21:42:55.510 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:55.510 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.510 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.510 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:55.510 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736601+0000", 2024-09-21T21:42:55.510 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:55.510 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.510 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.510 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:55.510 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736602+0000", 2024-09-21T21:42:55.510 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0699999999999999e-06 2024-09-21T21:42:55.510 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.511 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.511 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:55.511 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736607+0000", 2024-09-21T21:42:55.511 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2520000000000001e-06 2024-09-21T21:42:55.511 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.511 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.511 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:55.511 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736617+0000", 2024-09-21T21:42:55.511 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0074e-05 2024-09-21T21:42:55.511 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.511 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.511 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:55.511 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736622+0000", 2024-09-21T21:42:55.511 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.101e-06 2024-09-21T21:42:55.512 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.512 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.512 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:55.512 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736664+0000", 2024-09-21T21:42:55.512 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1869999999999997e-05 2024-09-21T21:42:55.512 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.512 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.512 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op must wait for map", 2024-09-21T21:42:55.512 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736667+0000", 2024-09-21T21:42:55.512 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4750000000000002e-06 2024-09-21T21:42:55.512 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.512 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.512 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:55.512 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742886+0000", 2024-09-21T21:42:55.512 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0062187579999999996 2024-09-21T21:42:55.513 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.513 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.513 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:55.513 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742892+0000", 2024-09-21T21:42:55.513 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0440000000000001e-06 2024-09-21T21:42:55.513 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.513 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.513 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:55.513 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210307+0000", 2024-09-21T21:42:55.513 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.467414781 2024-09-21T21:42:55.513 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.513 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.513 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:55.513 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210322+0000", 2024-09-21T21:42:55.514 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5311e-05 2024-09-21T21:42:55.514 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.514 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.514 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:55.514 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.237624+0000", 2024-09-21T21:42:55.514 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0273018920000001 2024-09-21T21:42:55.514 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.514 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.514 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:55.514 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.237870+0000", 2024-09-21T21:42:55.514 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00024622899999999999 2024-09-21T21:42:55.514 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.514 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.515 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:55.515 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.240607+0000", 2024-09-21T21:42:55.515 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0027368200000000001 2024-09-21T21:42:55.515 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.515 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:55.515 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.515 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.515 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.515 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5415.0:379 3.16 3:6888948b:::rbd_header.1121493f6459:head [watch reconnect cookie 139702602617920 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-21T21:42:55.515 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:41.739544+0000", 2024-09-21T21:42:55.515 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 73.766774161000001, 2024-09-21T21:42:55.515 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.501180645, 2024-09-21T21:42:55.515 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:55.515 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:55.515 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:55.516 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:55.516 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5415", 2024-09-21T21:42:55.516 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1212816607", 2024-09-21T21:42:55.516 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 379 2024-09-21T21:42:55.516 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.516 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:55.516 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.516 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:55.516 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739544+0000", 2024-09-21T21:42:55.516 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:55.516 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.516 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.516 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:55.516 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739544+0000", 2024-09-21T21:42:55.517 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:55.517 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.517 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.517 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:55.517 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739545+0000", 2024-09-21T21:42:55.517 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.065e-06 2024-09-21T21:42:55.517 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.517 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.517 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:55.517 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739550+0000", 2024-09-21T21:42:55.517 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9599999999999999e-06 2024-09-21T21:42:55.517 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.517 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.517 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:55.518 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739565+0000", 2024-09-21T21:42:55.518 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5709000000000002e-05 2024-09-21T21:42:55.518 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.518 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.518 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:55.518 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739573+0000", 2024-09-21T21:42:55.518 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7849999999999992e-06 2024-09-21T21:42:55.518 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.518 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.518 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:55.518 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739634+0000", 2024-09-21T21:42:55.518 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0732000000000001e-05 2024-09-21T21:42:55.518 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.519 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.519 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op must wait for map", 2024-09-21T21:42:55.519 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739640+0000", 2024-09-21T21:42:55.519 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0000000000000002e-06 2024-09-21T21:42:55.519 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.519 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.519 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:55.519 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742958+0000", 2024-09-21T21:42:55.519 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0033182390000000002 2024-09-21T21:42:55.519 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.519 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.519 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:55.519 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742966+0000", 2024-09-21T21:42:55.519 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1610000000000003e-06 2024-09-21T21:42:55.520 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.520 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.520 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:55.520 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210405+0000", 2024-09-21T21:42:55.520 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.467438725 2024-09-21T21:42:55.520 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.520 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.520 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:55.520 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210415+0000", 2024-09-21T21:42:55.520 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0234000000000001e-05 2024-09-21T21:42:55.520 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.520 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.520 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:55.520 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.238743+0000", 2024-09-21T21:42:55.521 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0283273070000001 2024-09-21T21:42:55.521 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.521 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.521 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:55.521 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.238842+0000", 2024-09-21T21:42:55.521 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.9896999999999996e-05 2024-09-21T21:42:55.521 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.521 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.521 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:55.521 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.240724+0000", 2024-09-21T21:42:55.521 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001881831 2024-09-21T21:42:55.521 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.521 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:55.521 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.522 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.522 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.522 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4385.0:15203 3.16 3:6888948b:::rbd_header.1121493f6459:head [call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-21T21:42:55.522 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:41.739737+0000", 2024-09-21T21:42:55.522 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 73.766581149999993, 2024-09-21T21:42:55.522 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4989621799999999, 2024-09-21T21:42:55.522 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:55.522 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:55.522 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:55.522 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:55.522 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4385", 2024-09-21T21:42:55.523 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:42:55.523 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 15203 2024-09-21T21:42:55.523 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.523 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:55.523 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.523 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:55.523 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739737+0000", 2024-09-21T21:42:55.523 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:55.523 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.523 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.523 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:55.523 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739737+0000", 2024-09-21T21:42:55.523 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:55.523 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.524 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.524 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:55.524 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739738+0000", 2024-09-21T21:42:55.524 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.7499999999999998e-07 2024-09-21T21:42:55.524 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.524 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.524 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:55.524 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739743+0000", 2024-09-21T21:42:55.524 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5049999999999998e-06 2024-09-21T21:42:55.524 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.524 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.524 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:55.524 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739757+0000", 2024-09-21T21:42:55.524 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3947e-05 2024-09-21T21:42:55.524 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.525 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.525 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:55.525 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739765+0000", 2024-09-21T21:42:55.525 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9310000000000008e-06 2024-09-21T21:42:55.525 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.525 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.525 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:55.525 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739816+0000", 2024-09-21T21:42:55.525 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0559999999999997e-05 2024-09-21T21:42:55.525 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.525 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.525 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting_for_map not empty", 2024-09-21T21:42:55.525 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739821+0000", 2024-09-21T21:42:55.525 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3539999999999999e-06 2024-09-21T21:42:55.525 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.526 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.526 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:55.526 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742928+0000", 2024-09-21T21:42:55.526 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0031074399999999999 2024-09-21T21:42:55.526 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.526 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.526 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:55.526 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742937+0000", 2024-09-21T21:42:55.526 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1289999999999996e-06 2024-09-21T21:42:55.526 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.526 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.526 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:55.526 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210350+0000", 2024-09-21T21:42:55.527 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.46741311299999999 2024-09-21T21:42:55.527 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.527 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.527 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:55.527 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210358+0000", 2024-09-21T21:42:55.527 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1869999999999997e-06 2024-09-21T21:42:55.527 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.527 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.527 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:55.527 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.238398+0000", 2024-09-21T21:42:55.527 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0280397830000001 2024-09-21T21:42:55.527 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.527 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.527 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:55.528 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.238489+0000", 2024-09-21T21:42:55.528 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.0958000000000006e-05 2024-09-21T21:42:55.528 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.528 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.528 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:55.528 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.238699+0000", 2024-09-21T21:42:55.528 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00021029800000000001 2024-09-21T21:42:55.528 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.528 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:55.528 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.528 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.528 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.528 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4385.0:15204 3.16 3:6888948b:::rbd_header.1121493f6459:head [watch reconnect cookie 140325440202496 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-21T21:42:55.528 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:42.210726+0000", 2024-09-21T21:42:55.529 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 73.295592271999993, 2024-09-21T21:42:55.529 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.030304557, 2024-09-21T21:42:55.529 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:55.529 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:55.529 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:55.529 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:55.529 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4385", 2024-09-21T21:42:55.529 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:42:55.529 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 15204 2024-09-21T21:42:55.529 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.529 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:55.529 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.529 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:55.529 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210726+0000", 2024-09-21T21:42:55.530 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:55.530 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.530 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.530 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:55.530 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210726+0000", 2024-09-21T21:42:55.530 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:55.530 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.530 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.530 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:55.530 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210727+0000", 2024-09-21T21:42:55.530 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.204e-06 2024-09-21T21:42:55.530 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.530 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.530 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:55.531 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210732+0000", 2024-09-21T21:42:55.531 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5249999999999996e-06 2024-09-21T21:42:55.531 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.531 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.531 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:55.531 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210744+0000", 2024-09-21T21:42:55.531 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1405e-05 2024-09-21T21:42:55.531 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.531 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.531 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:55.531 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210754+0000", 2024-09-21T21:42:55.531 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0122999999999999e-05 2024-09-21T21:42:55.531 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.531 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.531 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:55.532 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210798+0000", 2024-09-21T21:42:55.532 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4005000000000003e-05 2024-09-21T21:42:55.532 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.532 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.532 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:55.532 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210803+0000", 2024-09-21T21:42:55.532 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5130000000000004e-06 2024-09-21T21:42:55.532 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.532 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.532 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:55.532 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.239260+0000", 2024-09-21T21:42:55.532 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0284563040000001 2024-09-21T21:42:55.532 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.532 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.533 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:55.533 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.239318+0000", 2024-09-21T21:42:55.533 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8486000000000002e-05 2024-09-21T21:42:55.533 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.533 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.533 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:55.533 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.241030+0000", 2024-09-21T21:42:55.533 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0017119920000000001 2024-09-21T21:42:55.533 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.533 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:55.533 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.533 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.533 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.533 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5415.0:380 3.16 3:6888948b:::rbd_header.1121493f6459:head [watch reconnect cookie 139702602617920 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-21T21:42:55.533 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:42.210777+0000", 2024-09-21T21:42:55.534 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 73.295540493000004, 2024-09-21T21:42:55.534 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.030410566, 2024-09-21T21:42:55.534 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:55.534 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:55.534 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:42:55.534 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:55.534 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5415", 2024-09-21T21:42:55.534 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1212816607", 2024-09-21T21:42:55.534 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 380 2024-09-21T21:42:55.534 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.534 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:55.534 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.534 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:55.534 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210777+0000", 2024-09-21T21:42:55.535 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:55.535 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.535 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.535 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:55.535 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210777+0000", 2024-09-21T21:42:55.535 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:55.535 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.535 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.535 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:55.535 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210778+0000", 2024-09-21T21:42:55.535 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8999999999999996e-07 2024-09-21T21:42:55.535 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.535 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.535 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:55.535 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210780+0000", 2024-09-21T21:42:55.536 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:55.505+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:55.537 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.503e-06 2024-09-21T21:42:55.537 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.537 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.537 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:55.537 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210788+0000", 2024-09-21T21:42:55.537 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0999999999999998e-06 2024-09-21T21:42:55.537 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.537 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.538 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:55.538 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210799+0000", 2024-09-21T21:42:55.538 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1703e-05 2024-09-21T21:42:55.538 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.538 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.538 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:55.538 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210833+0000", 2024-09-21T21:42:55.538 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4008999999999999e-05 2024-09-21T21:42:55.538 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.538 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.538 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:42:55.538 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210838+0000", 2024-09-21T21:42:55.538 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7480000000000001e-06 2024-09-21T21:42:55.539 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.539 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.539 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:55.539 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.239612+0000", 2024-09-21T21:42:55.539 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.028773554 2024-09-21T21:42:55.539 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.539 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.539 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:55.539 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.239677+0000", 2024-09-21T21:42:55.539 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5053000000000002e-05 2024-09-21T21:42:55.539 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.539 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.539 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:55.539 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.241188+0000", 2024-09-21T21:42:55.539 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001511306 2024-09-21T21:42:55.540 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.540 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:55.540 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.540 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.540 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.540 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24794 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:55.540 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.584085+0000", 2024-09-21T21:42:55.540 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 32.922233317, 2024-09-21T21:42:55.540 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.096209996000000006, 2024-09-21T21:42:55.540 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:55.540 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:55.540 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:55.540 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:55.540 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:42:55.540 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:55.541 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24794 2024-09-21T21:42:55.541 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.541 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:55.541 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.541 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:55.541 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584085+0000", 2024-09-21T21:42:55.541 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:55.541 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.541 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.541 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:55.541 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584085+0000", 2024-09-21T21:42:55.541 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:55.541 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.541 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.542 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:55.542 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584085+0000", 2024-09-21T21:42:55.542 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3200000000000001e-07 2024-09-21T21:42:55.542 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.542 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.542 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:55.542 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584086+0000", 2024-09-21T21:42:55.542 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.717e-06 2024-09-21T21:42:55.542 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.542 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.542 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:55.542 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584090+0000", 2024-09-21T21:42:55.542 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3450000000000002e-06 2024-09-21T21:42:55.542 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.542 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.543 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:55.543 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584092+0000", 2024-09-21T21:42:55.543 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6359999999999998e-06 2024-09-21T21:42:55.543 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.543 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.543 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:55.543 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.676497+0000", 2024-09-21T21:42:55.543 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.092404659 2024-09-21T21:42:55.543 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.543 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.543 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:55.543 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.676567+0000", 2024-09-21T21:42:55.543 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9627000000000003e-05 2024-09-21T21:42:55.543 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.544 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.544 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:42:55.544 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.676925+0000", 2024-09-21T21:42:55.544 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00035848500000000002 2024-09-21T21:42:55.544 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.544 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.544 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:55.544 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.678427+0000", 2024-09-21T21:42:55.544 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001501638 2024-09-21T21:42:55.544 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.544 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.544 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:55.544 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.680204+0000", 2024-09-21T21:42:55.544 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0017766979999999999 2024-09-21T21:42:55.544 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.545 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.545 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:55.545 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.680256+0000", 2024-09-21T21:42:55.545 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2549999999999997e-05 2024-09-21T21:42:55.545 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.545 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.545 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:55.545 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.680295+0000", 2024-09-21T21:42:55.545 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8408999999999998e-05 2024-09-21T21:42:55.545 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.545 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:55.545 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.545 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.545 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.545 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24796 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:55.546 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.584099+0000", 2024-09-21T21:42:55.546 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 32.922218747999999, 2024-09-21T21:42:55.546 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.097297429000000005, 2024-09-21T21:42:55.546 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:55.546 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:55.546 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:55.546 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:55.546 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:42:55.546 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:55.546 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24796 2024-09-21T21:42:55.546 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.546 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:55.546 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.546 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:55.547 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584099+0000", 2024-09-21T21:42:55.547 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:55.547 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.547 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.547 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:55.547 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584099+0000", 2024-09-21T21:42:55.547 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:55.547 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.547 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.547 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:55.547 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584099+0000", 2024-09-21T21:42:55.547 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3999999999999998e-07 2024-09-21T21:42:55.547 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.547 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.547 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:55.548 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584101+0000", 2024-09-21T21:42:55.548 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5850000000000001e-06 2024-09-21T21:42:55.548 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.548 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.548 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:55.548 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584108+0000", 2024-09-21T21:42:55.548 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.498e-06 2024-09-21T21:42:55.548 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.548 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.548 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:55.548 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584112+0000", 2024-09-21T21:42:55.548 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.252e-06 2024-09-21T21:42:55.548 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.548 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.548 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:55.549 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.677336+0000", 2024-09-21T21:42:55.549 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.093224206000000004 2024-09-21T21:42:55.549 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.549 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.549 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:55.549 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.677411+0000", 2024-09-21T21:42:55.549 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5036000000000005e-05 2024-09-21T21:42:55.549 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.549 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.549 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:42:55.549 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.677782+0000", 2024-09-21T21:42:55.549 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00037153099999999998 2024-09-21T21:42:55.549 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.549 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.550 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:55.550 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.680360+0000", 2024-09-21T21:42:55.550 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025773020000000001 2024-09-21T21:42:55.550 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.550 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.550 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:55.550 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681313+0000", 2024-09-21T21:42:55.550 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00095341100000000004 2024-09-21T21:42:55.550 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.550 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.550 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:55.550 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681361+0000", 2024-09-21T21:42:55.550 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7497999999999998e-05 2024-09-21T21:42:55.550 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.551 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.551 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:55.551 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681397+0000", 2024-09-21T21:42:55.551 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.587e-05 2024-09-21T21:42:55.551 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.551 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:55.551 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.551 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.551 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.551 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24797 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:55.551 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.584116+0000", 2024-09-21T21:42:55.551 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 32.922202259000002, 2024-09-21T21:42:55.551 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.097428257000000004, 2024-09-21T21:42:55.552 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:55.552 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:55.552 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:55.552 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:55.552 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:42:55.552 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:55.552 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24797 2024-09-21T21:42:55.552 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.552 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:55.552 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.552 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:55.552 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584116+0000", 2024-09-21T21:42:55.552 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:55.552 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.553 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.553 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:55.553 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584116+0000", 2024-09-21T21:42:55.553 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:55.553 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.553 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.553 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:55.553 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584116+0000", 2024-09-21T21:42:55.553 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.53e-07 2024-09-21T21:42:55.553 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.553 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.553 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:55.553 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584118+0000", 2024-09-21T21:42:55.553 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6810000000000001e-06 2024-09-21T21:42:55.554 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.554 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.554 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:55.554 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584121+0000", 2024-09-21T21:42:55.554 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1099999999999999e-06 2024-09-21T21:42:55.554 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.554 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.554 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:55.554 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584123+0000", 2024-09-21T21:42:55.554 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5890000000000001e-06 2024-09-21T21:42:55.554 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.554 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.554 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:55.554 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.678476+0000", 2024-09-21T21:42:55.555 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.094352320000000003 2024-09-21T21:42:55.555 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.555 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.555 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:55.555 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.678556+0000", 2024-09-21T21:42:55.555 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0291999999999996e-05 2024-09-21T21:42:55.555 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.555 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.555 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:42:55.555 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.678871+0000", 2024-09-21T21:42:55.555 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000314689 2024-09-21T21:42:55.555 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.555 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.555 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:55.555 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681452+0000", 2024-09-21T21:42:55.556 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025817890000000001 2024-09-21T21:42:55.556 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.556 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.556 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:55.556 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681467+0000", 2024-09-21T21:42:55.556 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4256e-05 2024-09-21T21:42:55.556 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.556 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.556 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:55.556 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681510+0000", 2024-09-21T21:42:55.556 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3560999999999998e-05 2024-09-21T21:42:55.556 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.556 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.556 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:55.556 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681544+0000", 2024-09-21T21:42:55.557 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3717000000000003e-05 2024-09-21T21:42:55.557 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.557 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:55.557 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.557 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.557 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.557 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24798 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:55.557 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.584130+0000", 2024-09-21T21:42:55.557 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 32.922187735999998, 2024-09-21T21:42:55.557 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.098350526999999993, 2024-09-21T21:42:55.557 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:55.557 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:55.557 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:55.557 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:55.558 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:42:55.558 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:55.558 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24798 2024-09-21T21:42:55.558 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.558 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:55.558 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.558 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:55.558 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584130+0000", 2024-09-21T21:42:55.558 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:55.558 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.558 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.558 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:55.558 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584130+0000", 2024-09-21T21:42:55.558 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:55.558 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.559 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.559 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:55.559 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584131+0000", 2024-09-21T21:42:55.559 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4700000000000001e-07 2024-09-21T21:42:55.559 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.559 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.559 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:55.559 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584133+0000", 2024-09-21T21:42:55.559 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.34e-06 2024-09-21T21:42:55.559 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.559 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.559 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:55.559 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584137+0000", 2024-09-21T21:42:55.559 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5609999999999999e-06 2024-09-21T21:42:55.560 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.560 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.560 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:55.560 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584139+0000", 2024-09-21T21:42:55.560 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7350000000000001e-06 2024-09-21T21:42:55.560 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.560 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.560 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:55.560 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.679274+0000", 2024-09-21T21:42:55.560 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.095134719000000006 2024-09-21T21:42:55.560 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.560 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.560 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:55.560 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.679358+0000", 2024-09-21T21:42:55.560 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3609999999999994e-05 2024-09-21T21:42:55.561 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.561 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.561 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:42:55.561 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.679685+0000", 2024-09-21T21:42:55.561 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00032732399999999999 2024-09-21T21:42:55.561 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.561 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.561 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:55.561 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681553+0000", 2024-09-21T21:42:55.561 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001868059 2024-09-21T21:42:55.561 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.561 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.561 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:55.561 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.682375+0000", 2024-09-21T21:42:55.561 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00082232099999999999 2024-09-21T21:42:55.562 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.562 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.562 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:55.562 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.682431+0000", 2024-09-21T21:42:55.562 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6051999999999998e-05 2024-09-21T21:42:55.562 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.562 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.562 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:55.562 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.682481+0000", 2024-09-21T21:42:55.562 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9258999999999997e-05 2024-09-21T21:42:55.562 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.562 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:55.562 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.562 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.562 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.563 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24799 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:55.563 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.584159+0000", 2024-09-21T21:42:55.563 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 32.922158748000001, 2024-09-21T21:42:55.563 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.100021551, 2024-09-21T21:42:55.563 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:55.563 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:55.563 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:55.563 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:55.563 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:42:55.563 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:55.563 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24799 2024-09-21T21:42:55.563 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.563 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:55.563 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.564 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:55.564 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584159+0000", 2024-09-21T21:42:55.564 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:55.564 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.564 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.564 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:55.564 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584159+0000", 2024-09-21T21:42:55.564 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:55.564 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.564 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.564 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:55.564 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584160+0000", 2024-09-21T21:42:55.564 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1300000000000001e-07 2024-09-21T21:42:55.564 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.564 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.565 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:55.565 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584162+0000", 2024-09-21T21:42:55.565 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5409999999999999e-06 2024-09-21T21:42:55.565 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.565 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.565 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:55.565 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584169+0000", 2024-09-21T21:42:55.565 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7880000000000001e-06 2024-09-21T21:42:55.565 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.565 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.565 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:55.565 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584174+0000", 2024-09-21T21:42:55.565 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9590000000000003e-06 2024-09-21T21:42:55.565 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.565 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.566 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:55.566 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.680400+0000", 2024-09-21T21:42:55.566 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.096226514999999999 2024-09-21T21:42:55.566 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.566 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.566 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:55.566 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.680480+0000", 2024-09-21T21:42:55.566 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9589e-05 2024-09-21T21:42:55.566 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.566 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.566 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:42:55.566 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.680847+0000", 2024-09-21T21:42:55.566 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00036757700000000001 2024-09-21T21:42:55.566 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.567 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.567 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:55.567 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.682555+0000", 2024-09-21T21:42:55.567 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001707791 2024-09-21T21:42:55.567 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.567 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.567 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:55.567 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684093+0000", 2024-09-21T21:42:55.567 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0015377170000000001 2024-09-21T21:42:55.567 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.567 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.567 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:55.567 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684144+0000", 2024-09-21T21:42:55.567 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1208000000000002e-05 2024-09-21T21:42:55.567 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.568 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.568 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:55.568 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684181+0000", 2024-09-21T21:42:55.568 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6452999999999998e-05 2024-09-21T21:42:55.568 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.568 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:55.568 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.568 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.568 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.568 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24835 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:55.568 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586332+0000", 2024-09-21T21:42:55.568 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 32.919986133999998, 2024-09-21T21:42:55.568 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.098730360000000003, 2024-09-21T21:42:55.568 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:55.568 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:55.569 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:55.569 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:55.569 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:42:55.569 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:55.569 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24835 2024-09-21T21:42:55.569 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.569 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:55.569 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.569 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:55.569 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586332+0000", 2024-09-21T21:42:55.569 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:55.570 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.570 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.570 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:55.570 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586332+0000", 2024-09-21T21:42:55.570 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:55.570 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.570 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.570 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:55.570 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586333+0000", 2024-09-21T21:42:55.570 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0200000000000001e-07 2024-09-21T21:42:55.570 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.570 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.570 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:55.571 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586346+0000", 2024-09-21T21:42:55.571 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3745e-05 2024-09-21T21:42:55.571 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.571 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.571 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:55.571 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586366+0000", 2024-09-21T21:42:55.571 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9272000000000001e-05 2024-09-21T21:42:55.571 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.571 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.571 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:55.571 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586377+0000", 2024-09-21T21:42:55.571 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1197e-05 2024-09-21T21:42:55.571 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.571 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.572 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:55.572 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681578+0000", 2024-09-21T21:42:55.572 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.095201420999999994 2024-09-21T21:42:55.572 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.572 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.572 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:55.572 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681640+0000", 2024-09-21T21:42:55.572 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1834000000000005e-05 2024-09-21T21:42:55.572 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.572 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.572 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:42:55.572 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681915+0000", 2024-09-21T21:42:55.572 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00027534199999999998 2024-09-21T21:42:55.572 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.573 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.573 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:55.573 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684230+0000", 2024-09-21T21:42:55.573 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00231424 2024-09-21T21:42:55.573 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.573 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.573 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:55.573 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684959+0000", 2024-09-21T21:42:55.573 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00072921399999999995 2024-09-21T21:42:55.573 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.573 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.573 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:55.573 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.685008+0000", 2024-09-21T21:42:55.574 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8860999999999999e-05 2024-09-21T21:42:55.574 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.574 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.574 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:55.574 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.685062+0000", 2024-09-21T21:42:55.574 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4432000000000003e-05 2024-09-21T21:42:55.574 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.574 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:55.574 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.574 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.574 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.574 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24837 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:55.574 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586395+0000", 2024-09-21T21:42:55.575 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 32.919923199000003, 2024-09-21T21:42:55.575 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.099700522, 2024-09-21T21:42:55.575 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:55.575 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:55.575 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:55.575 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:55.575 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:42:55.575 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:55.575 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24837 2024-09-21T21:42:55.575 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.575 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:55.575 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.575 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:55.575 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586395+0000", 2024-09-21T21:42:55.575 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:55.576 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.576 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.576 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:55.576 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586395+0000", 2024-09-21T21:42:55.576 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:55.576 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.576 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.576 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:55.576 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586395+0000", 2024-09-21T21:42:55.576 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.06e-07 2024-09-21T21:42:55.576 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.576 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.576 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:55.576 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586401+0000", 2024-09-21T21:42:55.577 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0889999999999996e-06 2024-09-21T21:42:55.577 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.577 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.577 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:55.577 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586410+0000", 2024-09-21T21:42:55.577 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.918e-06 2024-09-21T21:42:55.577 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.577 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.577 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:55.577 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586424+0000", 2024-09-21T21:42:55.577 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3478999999999999e-05 2024-09-21T21:42:55.577 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.577 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.577 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:55.578 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.682590+0000", 2024-09-21T21:42:55.578 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.096166086999999997 2024-09-21T21:42:55.578 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.578 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.578 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:55.578 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.682657+0000", 2024-09-21T21:42:55.578 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6878000000000006e-05 2024-09-21T21:42:55.578 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.578 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.578 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:42:55.578 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.682879+0000", 2024-09-21T21:42:55.578 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00022238 2024-09-21T21:42:55.578 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.578 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.579 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:55.579 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.685073+0000", 2024-09-21T21:42:55.579 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021938119999999998 2024-09-21T21:42:55.579 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.579 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.579 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:55.579 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686038+0000", 2024-09-21T21:42:55.579 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00096539499999999997 2024-09-21T21:42:55.579 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.579 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.579 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:55.579 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686071+0000", 2024-09-21T21:42:55.579 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2514000000000001e-05 2024-09-21T21:42:55.579 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.580 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.580 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:55.580 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686095+0000", 2024-09-21T21:42:55.580 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4363999999999999e-05 2024-09-21T21:42:55.580 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.580 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:55.580 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.580 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.580 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.580 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24839 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:55.580 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586468+0000", 2024-09-21T21:42:55.580 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 32.919849546999998, 2024-09-21T21:42:55.580 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.099739355000000002, 2024-09-21T21:42:55.580 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:55.580 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:55.581 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:55.581 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:55.581 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:42:55.581 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:55.581 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24839 2024-09-21T21:42:55.581 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.581 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:55.581 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.581 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:55.581 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586468+0000", 2024-09-21T21:42:55.581 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:55.581 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.581 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.581 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:55.582 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586468+0000", 2024-09-21T21:42:55.582 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:55.582 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.582 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.582 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:55.582 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586469+0000", 2024-09-21T21:42:55.582 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2200000000000004e-07 2024-09-21T21:42:55.582 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.582 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.582 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:55.582 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586472+0000", 2024-09-21T21:42:55.582 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9960000000000002e-06 2024-09-21T21:42:55.582 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.582 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.583 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:55.583 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586479+0000", 2024-09-21T21:42:55.583 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2699999999999999e-06 2024-09-21T21:42:55.583 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.583 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.583 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:55.583 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586485+0000", 2024-09-21T21:42:55.583 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6999999999999996e-06 2024-09-21T21:42:55.583 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.583 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.583 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:55.583 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.683276+0000", 2024-09-21T21:42:55.583 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.096791618999999995 2024-09-21T21:42:55.583 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.583 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.584 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:55.584 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.683326+0000", 2024-09-21T21:42:55.584 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9384e-05 2024-09-21T21:42:55.584 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.584 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.584 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:42:55.584 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.683655+0000", 2024-09-21T21:42:55.584 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00032965700000000002 2024-09-21T21:42:55.584 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.584 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.584 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:55.584 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686120+0000", 2024-09-21T21:42:55.584 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002464819 2024-09-21T21:42:55.584 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.585 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.585 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:55.585 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686159+0000", 2024-09-21T21:42:55.585 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9181000000000003e-05 2024-09-21T21:42:55.585 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.585 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.585 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:55.585 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686185+0000", 2024-09-21T21:42:55.585 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5907000000000001e-05 2024-09-21T21:42:55.585 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.585 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.585 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:55.585 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686208+0000", 2024-09-21T21:42:55.585 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.23e-05 2024-09-21T21:42:55.585 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.586 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:55.586 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.586 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.586 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.586 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24841 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:55.586 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586492+0000", 2024-09-21T21:42:55.586 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 32.919825385000003, 2024-09-21T21:42:55.586 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.101672077, 2024-09-21T21:42:55.586 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:55.586 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:55.586 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:55.586 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:55.586 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:42:55.586 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:55.587 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24841 2024-09-21T21:42:55.587 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.587 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:55.587 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.587 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:55.587 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586492+0000", 2024-09-21T21:42:55.587 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:55.587 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.587 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.587 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:55.587 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586492+0000", 2024-09-21T21:42:55.587 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:55.587 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.588 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.588 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:55.588 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586493+0000", 2024-09-21T21:42:55.588 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5999999999999999e-07 2024-09-21T21:42:55.588 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.588 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.588 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:55.588 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586496+0000", 2024-09-21T21:42:55.588 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6089999999999999e-06 2024-09-21T21:42:55.588 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.588 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.588 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:55.588 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586502+0000", 2024-09-21T21:42:55.588 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6270000000000002e-06 2024-09-21T21:42:55.588 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.589 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.589 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:55.589 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586507+0000", 2024-09-21T21:42:55.589 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8810000000000004e-06 2024-09-21T21:42:55.589 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.589 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.589 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:55.589 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684273+0000", 2024-09-21T21:42:55.589 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.097766415999999995 2024-09-21T21:42:55.589 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.589 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.589 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:55.589 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684359+0000", 2024-09-21T21:42:55.589 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5574000000000004e-05 2024-09-21T21:42:55.589 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.590 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.590 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:42:55.590 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684568+0000", 2024-09-21T21:42:55.590 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000208529 2024-09-21T21:42:55.590 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.590 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.590 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:55.590 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.687186+0000", 2024-09-21T21:42:55.590 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0026180769999999999 2024-09-21T21:42:55.590 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.590 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.590 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:55.590 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688093+0000", 2024-09-21T21:42:55.590 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00090766999999999996 2024-09-21T21:42:55.591 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.591 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.591 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:55.591 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688142+0000", 2024-09-21T21:42:55.591 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8986000000000002e-05 2024-09-21T21:42:55.591 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.591 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.591 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:55.591 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688165+0000", 2024-09-21T21:42:55.591 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2248000000000001e-05 2024-09-21T21:42:55.591 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.591 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:55.591 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.592 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.592 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.592 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24843 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:55.592 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586514+0000", 2024-09-21T21:42:55.592 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 32.919804098999997, 2024-09-21T21:42:55.592 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.101826449, 2024-09-21T21:42:55.592 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:55.592 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:55.592 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:55.592 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:55.592 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:42:55.592 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:55.592 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24843 2024-09-21T21:42:55.592 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.592 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:55.593 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.593 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:55.593 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586514+0000", 2024-09-21T21:42:55.593 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:55.593 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.593 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.593 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:55.593 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586514+0000", 2024-09-21T21:42:55.593 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:55.593 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.593 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.593 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:55.593 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586514+0000", 2024-09-21T21:42:55.593 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9999999999999998e-07 2024-09-21T21:42:55.594 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.594 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.594 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:55.594 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586517+0000", 2024-09-21T21:42:55.594 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4980000000000001e-06 2024-09-21T21:42:55.594 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.594 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.594 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:55.594 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586523+0000", 2024-09-21T21:42:55.594 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4609999999999996e-06 2024-09-21T21:42:55.594 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.594 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.594 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:55.594 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586528+0000", 2024-09-21T21:42:55.595 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2979999999999999e-06 2024-09-21T21:42:55.595 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.595 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.595 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:55.595 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.685104+0000", 2024-09-21T21:42:55.595 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.098575837999999999 2024-09-21T21:42:55.595 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.595 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.595 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:55.595 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.685184+0000", 2024-09-21T21:42:55.595 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9807000000000003e-05 2024-09-21T21:42:55.595 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.595 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.595 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:42:55.595 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.685496+0000", 2024-09-21T21:42:55.596 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00031225900000000001 2024-09-21T21:42:55.596 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.596 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.596 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:55.596 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688238+0000", 2024-09-21T21:42:55.596 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0027419409999999999 2024-09-21T21:42:55.596 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.596 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.596 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:55.596 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688253+0000", 2024-09-21T21:42:55.596 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4331000000000001e-05 2024-09-21T21:42:55.596 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.596 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.596 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:55.597 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688296+0000", 2024-09-21T21:42:55.597 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3801000000000003e-05 2024-09-21T21:42:55.597 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.597 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.597 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:55.597 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688340+0000", 2024-09-21T21:42:55.597 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3815e-05 2024-09-21T21:42:55.597 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.597 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:55.597 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.597 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.597 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.597 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24845 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:55.597 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586535+0000", 2024-09-21T21:42:55.598 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 32.919783219000003, 2024-09-21T21:42:55.598 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.10285678600000001, 2024-09-21T21:42:55.598 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:55.598 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:55.598 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:55.598 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:55.598 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:42:55.598 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:55.598 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24845 2024-09-21T21:42:55.598 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.598 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:55.598 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.598 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:55.598 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586535+0000", 2024-09-21T21:42:55.599 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:55.599 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.599 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.599 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:55.599 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586535+0000", 2024-09-21T21:42:55.599 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:55.599 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.599 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.599 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:55.599 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586535+0000", 2024-09-21T21:42:55.599 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9000000000000002e-07 2024-09-21T21:42:55.599 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.599 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.599 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:55.599 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586537+0000", 2024-09-21T21:42:55.600 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4569999999999999e-06 2024-09-21T21:42:55.600 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.600 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.600 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:55.600 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586544+0000", 2024-09-21T21:42:55.600 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5849999999999996e-06 2024-09-21T21:42:55.600 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.600 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.600 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:55.600 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586549+0000", 2024-09-21T21:42:55.600 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0089999999999998e-06 2024-09-21T21:42:55.600 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.600 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.600 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:55.601 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686239+0000", 2024-09-21T21:42:55.601 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.099689531999999997 2024-09-21T21:42:55.601 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.601 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.601 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:55.601 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686289+0000", 2024-09-21T21:42:55.601 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.007e-05 2024-09-21T21:42:55.601 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.601 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.601 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:42:55.601 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686633+0000", 2024-09-21T21:42:55.601 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00034428699999999998 2024-09-21T21:42:55.601 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.601 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.602 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:55.602 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688370+0000", 2024-09-21T21:42:55.602 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001736741 2024-09-21T21:42:55.602 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.602 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.602 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:55.602 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.689295+0000", 2024-09-21T21:42:55.602 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00092517900000000004 2024-09-21T21:42:55.602 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.602 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.602 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:55.602 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.689354+0000", 2024-09-21T21:42:55.602 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8799999999999999e-05 2024-09-21T21:42:55.602 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.603 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.603 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:55.603 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.689391+0000", 2024-09-21T21:42:55.603 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7735999999999999e-05 2024-09-21T21:42:55.603 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.603 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:55.603 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.603 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.603 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.603 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24846 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:55.603 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586564+0000", 2024-09-21T21:42:55.603 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 32.919754191000003, 2024-09-21T21:42:55.603 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.10375692, 2024-09-21T21:42:55.604 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:55.604 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:55.604 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:55.604 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:55.604 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:42:55.604 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:55.604 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24846 2024-09-21T21:42:55.604 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.604 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:55.604 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.604 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:55.604 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586564+0000", 2024-09-21T21:42:55.604 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:55.604 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.604 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.605 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:55.605 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586564+0000", 2024-09-21T21:42:55.605 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:55.605 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.605 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.605 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:55.605 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586564+0000", 2024-09-21T21:42:55.605 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4799999999999998e-07 2024-09-21T21:42:55.605 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.605 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.605 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:55.605 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586567+0000", 2024-09-21T21:42:55.605 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1669999999999999e-06 2024-09-21T21:42:55.605 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.606 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.606 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:55.606 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586578+0000", 2024-09-21T21:42:55.606 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0443e-05 2024-09-21T21:42:55.606 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.606 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.606 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:55.606 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586583+0000", 2024-09-21T21:42:55.606 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6790000000000002e-06 2024-09-21T21:42:55.606 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.606 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.606 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:55.606 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.687215+0000", 2024-09-21T21:42:55.606 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.100631973 2024-09-21T21:42:55.607 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.607 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.607 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:55.607 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.687311+0000", 2024-09-21T21:42:55.607 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5764e-05 2024-09-21T21:42:55.607 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.607 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.607 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:42:55.607 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.687663+0000", 2024-09-21T21:42:55.607 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00035167 2024-09-21T21:42:55.607 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.607 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.607 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:55.607 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690236+0000", 2024-09-21T21:42:55.608 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025733370000000002 2024-09-21T21:42:55.608 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.608 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.608 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:55.608 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690250+0000", 2024-09-21T21:42:55.608 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3548e-05 2024-09-21T21:42:55.608 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.608 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.608 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:55.608 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690298+0000", 2024-09-21T21:42:55.608 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7948999999999998e-05 2024-09-21T21:42:55.608 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.608 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.609 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:55.609 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690321+0000", 2024-09-21T21:42:55.609 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2841999999999999e-05 2024-09-21T21:42:55.609 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.609 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:55.609 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.609 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.609 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.609 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24849 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:55.609 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586633+0000", 2024-09-21T21:42:55.609 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 32.919684525000001, 2024-09-21T21:42:55.609 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.104506548, 2024-09-21T21:42:55.609 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:55.609 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:55.609 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:55.610 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:55.610 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:42:55.610 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:55.610 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24849 2024-09-21T21:42:55.610 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.610 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:55.610 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.610 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:55.610 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586633+0000", 2024-09-21T21:42:55.610 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:55.610 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.610 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.610 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:55.610 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586633+0000", 2024-09-21T21:42:55.611 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:55.611 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.611 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.611 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:55.611 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586634+0000", 2024-09-21T21:42:55.611 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9100000000000004e-07 2024-09-21T21:42:55.611 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.611 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.611 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:55.611 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586637+0000", 2024-09-21T21:42:55.611 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1999999999999999e-06 2024-09-21T21:42:55.611 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.611 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.611 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:55.611 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586645+0000", 2024-09-21T21:42:55.612 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7759999999999997e-06 2024-09-21T21:42:55.612 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.612 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.612 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:55.612 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586652+0000", 2024-09-21T21:42:55.612 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3560000000000004e-06 2024-09-21T21:42:55.612 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.612 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.612 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:55.612 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688395+0000", 2024-09-21T21:42:55.612 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.101743239 2024-09-21T21:42:55.612 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.612 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.612 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:55.613 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688464+0000", 2024-09-21T21:42:55.613 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8273000000000002e-05 2024-09-21T21:42:55.613 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.613 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.613 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:42:55.613 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688786+0000", 2024-09-21T21:42:55.613 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000322064 2024-09-21T21:42:55.613 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.613 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.613 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:55.613 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690364+0000", 2024-09-21T21:42:55.613 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001578084 2024-09-21T21:42:55.613 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.613 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.613 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:55.614 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.691050+0000", 2024-09-21T21:42:55.614 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00068623399999999998 2024-09-21T21:42:55.614 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.614 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.614 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:55.614 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.691090+0000", 2024-09-21T21:42:55.614 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9406999999999997e-05 2024-09-21T21:42:55.614 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.614 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.614 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:55.614 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.691140+0000", 2024-09-21T21:42:55.614 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0324000000000001e-05 2024-09-21T21:42:55.614 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.614 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:55.614 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.615 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.615 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.615 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24851 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:55.615 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586661+0000", 2024-09-21T21:42:55.615 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 32.919657239000003, 2024-09-21T21:42:55.615 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.10536891399999999, 2024-09-21T21:42:55.615 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:55.615 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:55.615 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:55.615 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:55.615 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:42:55.615 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:55.615 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24851 2024-09-21T21:42:55.615 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.615 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:55.616 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.616 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:55.616 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586661+0000", 2024-09-21T21:42:55.616 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:55.616 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.616 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.616 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:55.616 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586661+0000", 2024-09-21T21:42:55.616 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:55.616 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.616 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.616 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:55.616 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586661+0000", 2024-09-21T21:42:55.616 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4700000000000002e-07 2024-09-21T21:42:55.616 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.617 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.617 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:55.617 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586664+0000", 2024-09-21T21:42:55.617 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.672e-06 2024-09-21T21:42:55.617 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.617 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.617 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:55.617 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586671+0000", 2024-09-21T21:42:55.617 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9959999999999996e-06 2024-09-21T21:42:55.617 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.617 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.617 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:55.617 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586676+0000", 2024-09-21T21:42:55.617 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4020000000000001e-06 2024-09-21T21:42:55.618 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.618 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.618 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:55.618 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.689438+0000", 2024-09-21T21:42:55.618 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.102761503 2024-09-21T21:42:55.618 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.618 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.618 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:55.618 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.689524+0000", 2024-09-21T21:42:55.618 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6383000000000003e-05 2024-09-21T21:42:55.618 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.618 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.618 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:42:55.618 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.689769+0000", 2024-09-21T21:42:55.619 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00024504699999999997 2024-09-21T21:42:55.619 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.619 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.619 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:55.619 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.691936+0000", 2024-09-21T21:42:55.619 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021665130000000001 2024-09-21T21:42:55.619 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.619 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.619 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:55.619 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.691947+0000", 2024-09-21T21:42:55.619 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1198e-05 2024-09-21T21:42:55.619 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.619 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.619 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:55.619 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.691992+0000", 2024-09-21T21:42:55.619 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4774000000000002e-05 2024-09-21T21:42:55.620 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.620 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.620 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:55.620 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.692030+0000", 2024-09-21T21:42:55.620 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7978999999999997e-05 2024-09-21T21:42:55.620 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.620 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:55.620 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.620 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.620 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.620 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24853 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:42:55.620 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586684+0000", 2024-09-21T21:42:55.620 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 32.919634281999997, 2024-09-21T21:42:55.620 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.105936852, 2024-09-21T21:42:55.620 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:42:55.621 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:42:55.621 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:55.621 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:42:55.621 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:42:55.621 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:42:55.621 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24853 2024-09-21T21:42:55.621 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.621 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:42:55.621 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.621 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:42:55.621 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586684+0000", 2024-09-21T21:42:55.621 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:55.621 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.621 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.621 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:42:55.622 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586684+0000", 2024-09-21T21:42:55.622 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:42:55.622 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.622 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.622 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:42:55.622 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586684+0000", 2024-09-21T21:42:55.622 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.68e-07 2024-09-21T21:42:55.622 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.622 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.622 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:42:55.622 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586687+0000", 2024-09-21T21:42:55.622 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.774e-06 2024-09-21T21:42:55.622 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.622 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.622 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:42:55.623 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586694+0000", 2024-09-21T21:42:55.623 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6959999999999999e-06 2024-09-21T21:42:55.623 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.623 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.623 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:42:55.623 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586698+0000", 2024-09-21T21:42:55.623 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7489999999999998e-06 2024-09-21T21:42:55.623 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.623 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.623 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:42:55.623 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690395+0000", 2024-09-21T21:42:55.623 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.103696645 2024-09-21T21:42:55.623 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.623 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.624 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:42:55.624 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690440+0000", 2024-09-21T21:42:55.624 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4835e-05 2024-09-21T21:42:55.624 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.624 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.624 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:42:55.624 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690654+0000", 2024-09-21T21:42:55.624 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000214114 2024-09-21T21:42:55.624 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.624 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.624 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:42:55.624 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.692041+0000", 2024-09-21T21:42:55.624 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0013866519999999999 2024-09-21T21:42:55.624 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.624 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.625 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:55.625 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.692558+0000", 2024-09-21T21:42:55.625 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00051717199999999999 2024-09-21T21:42:55.625 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.625 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.625 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:42:55.625 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.692586+0000", 2024-09-21T21:42:55.625 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8504e-05 2024-09-21T21:42:55.625 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:42:55.625 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:42:55.625 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:42:55.625 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.692620+0000", 2024-09-21T21:42:55.625 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4143000000000002e-05 2024-09-21T21:42:55.625 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.625 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:55.626 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.626 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:42:55.626 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:42:55.626 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:55.626 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T21:42:55.628 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:55.625+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:55.729 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:55.727+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:55.734 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:55.734 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:42:55.734 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 0 2024-09-21T21:42:55.734 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:55.744 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T21:42:55.830 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:55.828+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:55.840 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T21:42:55.890 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:55.890 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:42:55.890 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:42:55.890 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:42:55.890 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:55.900 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T21:42:55.944 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:55.943+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:55.962 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0: {} 2024-09-21T21:42:55.962 INFO:teuthology.orchestra.run.smithi081.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T21:42:55.979 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1: {} 2024-09-21T21:42:55.979 INFO:teuthology.orchestra.run.smithi081.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T21:42:55.995 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2: {} 2024-09-21T21:42:55.995 INFO:teuthology.orchestra.run.smithi081.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T21:42:56.006 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:56.006 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:56.012 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3: {} 2024-09-21T21:42:56.012 INFO:teuthology.orchestra.run.smithi081.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T21:42:56.016 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T21:42:56.029 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4: {} 2024-09-21T21:42:56.030 INFO:teuthology.orchestra.run.smithi081.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T21:42:56.032 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:56.030+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:56.047 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5: {} 2024-09-21T21:42:56.047 INFO:teuthology.orchestra.run.smithi081.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T21:42:56.133 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:56.133 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:56.134 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:56.131+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:56.142 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T21:42:56.234 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:56.233+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:56.244 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:56.244 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:56.251 INFO:teuthology.orchestra.run.smithi081.stderr:set 3.13 pg_upmap_items mapping to [1->4,2->5] 2024-09-21T21:42:56.253 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T21:42:56.258 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T21:42:56.354 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:56.352+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:56.425 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:56.425 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:56.434 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T21:42:56.436 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:56.434+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:56.536 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:56.535+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:56.578 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:56.578 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:56.587 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T21:42:56.637 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:56.636+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:56.729 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:56.730 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:56.738 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T21:42:56.739 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:56.736+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:56.839 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:56.838+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:56.842 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:56.842 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:56.852 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T21:42:56.940 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:56.939+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:56.958 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:56.958 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:56.967 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T21:42:57.041 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:57.039+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:57.070 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:57.070 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:57.079 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T21:42:57.162 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:57.160+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:57.214 INFO:teuthology.orchestra.run.smithi081.stderr:nodeep-scrub is unset 2024-09-21T21:42:57.225 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:57.225 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:57.234 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T21:42:57.271 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:57.270+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:57.344 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:57.341+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:57.391 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:57.392 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:57.401 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T21:42:57.444 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:57.443+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:57.543 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:57.543 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:57.544 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:57.543+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:57.553 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T21:42:57.645 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:57.644+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:57.655 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:57.655 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:57.664 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T21:42:57.746 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:57.745+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:57.766 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:57.766 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:57.775 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T21:42:57.847 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:57.846+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:57.878 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:57.878 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:57.887 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T21:42:57.970 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:57.969+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:58.029 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:58.029 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:58.038 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T21:42:58.058 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T21:42:58.078 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:58.077+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:58.148 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:42:58.148 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:42:58.149 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:58.147+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:58.157 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T21:42:58.201 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0: {} 2024-09-21T21:42:58.201 INFO:teuthology.orchestra.run.smithi081.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T21:42:58.219 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1: {} 2024-09-21T21:42:58.219 INFO:teuthology.orchestra.run.smithi081.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T21:42:58.235 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2: {} 2024-09-21T21:42:58.235 INFO:teuthology.orchestra.run.smithi081.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T21:42:58.250 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:58.248+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:58.252 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3: {} 2024-09-21T21:42:58.252 INFO:teuthology.orchestra.run.smithi081.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T21:42:58.273 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4: {} 2024-09-21T21:42:58.273 INFO:teuthology.orchestra.run.smithi081.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T21:42:58.290 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5: {} 2024-09-21T21:42:58.290 INFO:teuthology.orchestra.run.smithi081.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T21:42:58.306 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:58.306 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:42:58.306 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:42:58.306 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:42:58.306 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:58.315 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T21:42:58.356 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:58.355+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:58.452 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:58.450+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:58.463 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:58.463 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 20, 2024-09-21T21:42:58.463 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 600, 2024-09-21T21:42:58.463 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:42:58.463 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.464 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11319 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 5492736~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:58.464 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.712385+0000", 2024-09-21T21:42:58.464 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 67.748985622000006, 2024-09-21T21:42:58.464 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078627889000000006, 2024-09-21T21:42:58.464 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:58.464 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:58.464 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:58.464 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:58.464 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:58.464 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:58.464 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11319 2024-09-21T21:42:58.464 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.464 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:58.465 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.465 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:58.465 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712385+0000", 2024-09-21T21:42:58.465 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:58.465 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.465 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.465 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:58.465 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712385+0000", 2024-09-21T21:42:58.465 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:58.465 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.465 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.465 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:58.465 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712385+0000", 2024-09-21T21:42:58.465 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2800000000000002e-07 2024-09-21T21:42:58.466 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.466 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.466 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:58.466 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712387+0000", 2024-09-21T21:42:58.466 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1569999999999998e-06 2024-09-21T21:42:58.466 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.466 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.466 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:58.466 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712393+0000", 2024-09-21T21:42:58.466 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9370000000000001e-06 2024-09-21T21:42:58.466 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.466 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.466 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:58.466 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712398+0000", 2024-09-21T21:42:58.467 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5020000000000003e-06 2024-09-21T21:42:58.467 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.467 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.467 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:58.467 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.790746+0000", 2024-09-21T21:42:58.467 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078348589999999996 2024-09-21T21:42:58.467 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.467 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.467 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:58.467 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.790844+0000", 2024-09-21T21:42:58.467 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.7526000000000001e-05 2024-09-21T21:42:58.467 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.467 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.467 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:58.467 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.791013+0000", 2024-09-21T21:42:58.467 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00016874899999999999 2024-09-21T21:42:58.468 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:58.468 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:58.468 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:58.468 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.468 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.468 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11320 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 6180864~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:58.468 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.712405+0000", 2024-09-21T21:42:58.468 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 67.748965663000007, 2024-09-21T21:42:58.468 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078882870999999993, 2024-09-21T21:42:58.468 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:58.468 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:58.468 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:58.468 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:58.468 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:58.469 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:58.469 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11320 2024-09-21T21:42:58.469 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.469 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:58.469 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.469 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:58.469 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712405+0000", 2024-09-21T21:42:58.469 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:58.469 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.469 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.469 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:58.469 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712405+0000", 2024-09-21T21:42:58.469 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:58.469 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.469 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.470 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:58.470 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.713942+0000", 2024-09-21T21:42:58.470 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.001536882 2024-09-21T21:42:58.470 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.470 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.470 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:58.470 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.713946+0000", 2024-09-21T21:42:58.470 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4549999999999997e-06 2024-09-21T21:42:58.470 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.470 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.470 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:58.470 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.713958+0000", 2024-09-21T21:42:58.470 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1775e-05 2024-09-21T21:42:58.470 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.470 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.471 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:58.471 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.713965+0000", 2024-09-21T21:42:58.471 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5100000000000001e-06 2024-09-21T21:42:58.471 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.471 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.471 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:58.471 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.791059+0000", 2024-09-21T21:42:58.471 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.077094036000000005 2024-09-21T21:42:58.471 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.471 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.471 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:58.471 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.791175+0000", 2024-09-21T21:42:58.471 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000115895 2024-09-21T21:42:58.471 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.471 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.472 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:58.472 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.791288+0000", 2024-09-21T21:42:58.472 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000112318 2024-09-21T21:42:58.472 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:58.472 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:58.472 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:58.472 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.472 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.472 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11325 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 5414912~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:58.472 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.714095+0000", 2024-09-21T21:42:58.472 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 67.747275668, 2024-09-21T21:42:58.472 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078631894999999993, 2024-09-21T21:42:58.472 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:58.472 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:58.472 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:58.473 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:58.473 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:58.473 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:58.473 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11325 2024-09-21T21:42:58.473 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.473 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:58.473 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.473 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:58.473 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714095+0000", 2024-09-21T21:42:58.473 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:58.473 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.473 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.473 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:58.473 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714095+0000", 2024-09-21T21:42:58.473 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:58.474 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.474 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.474 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:58.474 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714095+0000", 2024-09-21T21:42:58.474 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7500000000000001e-07 2024-09-21T21:42:58.474 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.474 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.474 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:58.474 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714097+0000", 2024-09-21T21:42:58.474 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.12e-06 2024-09-21T21:42:58.474 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.474 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.474 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:58.474 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714104+0000", 2024-09-21T21:42:58.474 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3520000000000003e-06 2024-09-21T21:42:58.474 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.475 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.475 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:58.475 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714108+0000", 2024-09-21T21:42:58.475 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8810000000000004e-06 2024-09-21T21:42:58.475 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.475 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.475 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:58.475 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792509+0000", 2024-09-21T21:42:58.475 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078400739999999997 2024-09-21T21:42:58.475 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.475 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.475 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:58.475 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792613+0000", 2024-09-21T21:42:58.475 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000103939 2024-09-21T21:42:58.476 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.476 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.476 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:58.476 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792727+0000", 2024-09-21T21:42:58.476 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011348799999999999 2024-09-21T21:42:58.476 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:58.476 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:58.476 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:58.476 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.476 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.476 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11326 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 6103040~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:58.476 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.714115+0000", 2024-09-21T21:42:58.476 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 67.747255640000006, 2024-09-21T21:42:58.476 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078963056000000004, 2024-09-21T21:42:58.477 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:58.477 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:58.477 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:58.477 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:58.477 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:58.477 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:58.477 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11326 2024-09-21T21:42:58.477 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.477 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:58.477 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.477 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:58.477 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714115+0000", 2024-09-21T21:42:58.477 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:58.477 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.477 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.477 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:58.478 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714115+0000", 2024-09-21T21:42:58.478 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:58.478 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.478 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.478 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:58.478 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.715940+0000", 2024-09-21T21:42:58.478 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0018254969999999999 2024-09-21T21:42:58.478 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.478 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.478 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:58.478 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.715945+0000", 2024-09-21T21:42:58.478 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6120000000000001e-06 2024-09-21T21:42:58.478 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.478 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.479 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:58.479 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.715954+0000", 2024-09-21T21:42:58.479 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.5899999999999997e-06 2024-09-21T21:42:58.479 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.479 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.479 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:58.479 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.715959+0000", 2024-09-21T21:42:58.479 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0849999999999996e-06 2024-09-21T21:42:58.479 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.479 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.479 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:58.479 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792771+0000", 2024-09-21T21:42:58.479 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.076811829999999998 2024-09-21T21:42:58.479 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.479 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.480 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:58.480 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792882+0000", 2024-09-21T21:42:58.480 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000110857 2024-09-21T21:42:58.480 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.480 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.480 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:58.480 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.793078+0000", 2024-09-21T21:42:58.480 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00019558499999999999 2024-09-21T21:42:58.480 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:58.480 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:58.480 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:58.480 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.480 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.480 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11332 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 7565312~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:58.480 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.716063+0000", 2024-09-21T21:42:58.480 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 67.745307105999998, 2024-09-21T21:42:58.481 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078787482000000006, 2024-09-21T21:42:58.481 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:58.481 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:58.481 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:58.481 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:58.481 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:58.481 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:58.481 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11332 2024-09-21T21:42:58.481 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.481 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:58.481 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.481 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:58.481 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.716063+0000", 2024-09-21T21:42:58.481 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:58.481 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.482 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.482 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:58.482 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.716063+0000", 2024-09-21T21:42:58.482 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:58.482 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.482 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.482 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:58.482 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.717944+0000", 2024-09-21T21:42:58.482 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0018812309999999999 2024-09-21T21:42:58.482 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.482 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.482 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:58.482 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.717949+0000", 2024-09-21T21:42:58.482 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9389999999999996e-06 2024-09-21T21:42:58.482 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.483 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.483 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:58.483 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.717963+0000", 2024-09-21T21:42:58.483 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.3899e-05 2024-09-21T21:42:58.483 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.483 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.483 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:58.483 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.717970+0000", 2024-09-21T21:42:58.483 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.426e-06 2024-09-21T21:42:58.483 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.483 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.483 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:58.483 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.794573+0000", 2024-09-21T21:42:58.483 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.076602990999999995 2024-09-21T21:42:58.484 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.484 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.484 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:58.484 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.794682+0000", 2024-09-21T21:42:58.484 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000109541 2024-09-21T21:42:58.484 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.484 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.484 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:58.484 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.794851+0000", 2024-09-21T21:42:58.484 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000168455 2024-09-21T21:42:58.484 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:58.484 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:58.484 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:58.484 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.484 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.484 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11339 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 6266880~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:58.485 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.718117+0000", 2024-09-21T21:42:58.485 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 67.743253080000002, 2024-09-21T21:42:58.485 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078703717000000006, 2024-09-21T21:42:58.485 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:58.485 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:58.485 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:58.485 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:58.485 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:58.485 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:58.485 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11339 2024-09-21T21:42:58.485 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.485 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:58.486 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.486 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:58.486 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.718117+0000", 2024-09-21T21:42:58.486 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:58.486 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.486 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.486 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:58.486 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.718117+0000", 2024-09-21T21:42:58.486 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:58.486 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.486 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.486 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:58.486 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720019+0000", 2024-09-21T21:42:58.486 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0019012549999999999 2024-09-21T21:42:58.487 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.487 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.487 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:58.487 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720024+0000", 2024-09-21T21:42:58.487 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1100000000000002e-06 2024-09-21T21:42:58.487 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.487 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.487 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:58.487 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720035+0000", 2024-09-21T21:42:58.487 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.094e-05 2024-09-21T21:42:58.487 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.487 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.487 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:58.487 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720040+0000", 2024-09-21T21:42:58.488 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1830000000000002e-06 2024-09-21T21:42:58.488 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.488 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.488 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:58.488 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796618+0000", 2024-09-21T21:42:58.488 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.076578033000000004 2024-09-21T21:42:58.488 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.488 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.488 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:58.488 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796696+0000", 2024-09-21T21:42:58.488 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7836000000000005e-05 2024-09-21T21:42:58.488 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.488 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.488 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:58.488 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796821+0000", 2024-09-21T21:42:58.488 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00012536000000000001 2024-09-21T21:42:58.489 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:58.489 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:58.489 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:58.489 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.489 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.489 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11345 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 5472256~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:58.489 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.720116+0000", 2024-09-21T21:42:58.489 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 67.741254179999999, 2024-09-21T21:42:58.489 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078667730000000005, 2024-09-21T21:42:58.489 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:58.489 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:58.489 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:58.489 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:58.489 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:58.490 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:58.490 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11345 2024-09-21T21:42:58.490 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.490 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:58.490 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.490 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:58.490 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720116+0000", 2024-09-21T21:42:58.490 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:58.490 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.490 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.490 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:58.490 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720116+0000", 2024-09-21T21:42:58.490 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:58.490 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.490 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.491 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:58.491 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720116+0000", 2024-09-21T21:42:58.491 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.04e-07 2024-09-21T21:42:58.491 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.491 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.491 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:58.491 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720118+0000", 2024-09-21T21:42:58.491 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.181e-06 2024-09-21T21:42:58.491 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.491 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.491 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:58.491 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720121+0000", 2024-09-21T21:42:58.491 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0199999999999999e-06 2024-09-21T21:42:58.491 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.491 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.492 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:58.492 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720123+0000", 2024-09-21T21:42:58.492 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.5469999999999999e-06 2024-09-21T21:42:58.492 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.492 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.492 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:58.492 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798626+0000", 2024-09-21T21:42:58.492 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078503137000000001 2024-09-21T21:42:58.492 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.492 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.492 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:58.492 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798677+0000", 2024-09-21T21:42:58.492 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0930999999999997e-05 2024-09-21T21:42:58.492 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.493 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.493 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:58.493 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798784+0000", 2024-09-21T21:42:58.493 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010671 2024-09-21T21:42:58.493 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:58.493 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:58.493 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:58.493 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.493 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.493 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11346 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 5500928~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:58.493 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.720126+0000", 2024-09-21T21:42:58.493 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 67.741244107, 2024-09-21T21:42:58.493 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078863421000000003, 2024-09-21T21:42:58.493 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:58.494 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:58.494 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:58.494 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:58.494 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:58.494 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:58.494 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11346 2024-09-21T21:42:58.494 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.494 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:58.494 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.494 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:58.494 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720126+0000", 2024-09-21T21:42:58.494 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:58.494 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.494 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.494 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:58.495 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720126+0000", 2024-09-21T21:42:58.495 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:58.495 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.495 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.495 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:58.495 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.722145+0000", 2024-09-21T21:42:58.495 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0020183219999999999 2024-09-21T21:42:58.495 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.495 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.495 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:58.495 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.722150+0000", 2024-09-21T21:42:58.495 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1760000000000001e-06 2024-09-21T21:42:58.495 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.495 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.495 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:58.496 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.722161+0000", 2024-09-21T21:42:58.496 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1566000000000001e-05 2024-09-21T21:42:58.496 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.496 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.496 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:58.496 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.722169+0000", 2024-09-21T21:42:58.496 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.2840000000000001e-06 2024-09-21T21:42:58.496 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.496 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.496 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:58.496 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798815+0000", 2024-09-21T21:42:58.496 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.076646607000000005 2024-09-21T21:42:58.496 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.496 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.496 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:58.497 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798870+0000", 2024-09-21T21:42:58.497 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4486e-05 2024-09-21T21:42:58.497 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.497 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.497 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:58.497 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798990+0000", 2024-09-21T21:42:58.497 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011998 2024-09-21T21:42:58.497 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:58.497 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:58.497 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:58.497 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.497 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.497 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13629 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7852032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:58.497 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943319+0000", 2024-09-21T21:42:58.497 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 67.518051464999999, 2024-09-21T21:42:58.498 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078610975999999999, 2024-09-21T21:42:58.498 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:58.498 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:58.498 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:58.498 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:58.498 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:58.498 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:58.498 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13629 2024-09-21T21:42:58.498 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.498 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:58.498 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.498 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:58.498 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943319+0000", 2024-09-21T21:42:58.498 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:58.498 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.499 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.499 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:58.499 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943319+0000", 2024-09-21T21:42:58.499 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:58.499 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.499 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.499 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:58.499 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943319+0000", 2024-09-21T21:42:58.499 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6199999999999999e-07 2024-09-21T21:42:58.499 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.499 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.499 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:58.499 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943321+0000", 2024-09-21T21:42:58.499 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1009999999999999e-06 2024-09-21T21:42:58.499 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.500 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.500 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:58.500 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943328+0000", 2024-09-21T21:42:58.500 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.1029999999999996e-06 2024-09-21T21:42:58.500 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.500 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.500 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:58.500 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943333+0000", 2024-09-21T21:42:58.500 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7709999999999997e-06 2024-09-21T21:42:58.500 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.500 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.500 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:58.500 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.021679+0000", 2024-09-21T21:42:58.500 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078345673000000005 2024-09-21T21:42:58.500 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.501 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.501 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:58.501 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.021771+0000", 2024-09-21T21:42:58.501 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2201999999999996e-05 2024-09-21T21:42:58.501 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.501 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.501 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:58.501 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.021930+0000", 2024-09-21T21:42:58.501 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00015876400000000001 2024-09-21T21:42:58.501 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:58.501 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:58.501 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:58.501 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.502 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.502 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13630 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7880704~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:58.502 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943339+0000", 2024-09-21T21:42:58.502 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 67.518031277000006, 2024-09-21T21:42:58.502 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078939410000000002, 2024-09-21T21:42:58.502 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:58.502 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:58.502 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:58.502 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:58.502 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:58.502 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:58.502 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13630 2024-09-21T21:42:58.502 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.502 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:58.503 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.503 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:58.503 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943339+0000", 2024-09-21T21:42:58.503 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:58.503 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.503 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.503 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:58.503 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943339+0000", 2024-09-21T21:42:58.503 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:58.503 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.503 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.503 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:58.503 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943339+0000", 2024-09-21T21:42:58.503 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8000000000000001e-07 2024-09-21T21:42:58.503 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.504 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.504 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:58.504 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943341+0000", 2024-09-21T21:42:58.504 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9850000000000001e-06 2024-09-21T21:42:58.504 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.504 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.504 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:58.504 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943347+0000", 2024-09-21T21:42:58.504 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2979999999999999e-06 2024-09-21T21:42:58.504 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.504 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.504 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:58.504 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943354+0000", 2024-09-21T21:42:58.504 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.2969999999999998e-06 2024-09-21T21:42:58.505 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.505 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.505 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:58.505 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.021996+0000", 2024-09-21T21:42:58.505 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078641680000000005 2024-09-21T21:42:58.505 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.505 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.505 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:58.505 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022104+0000", 2024-09-21T21:42:58.505 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000107902 2024-09-21T21:42:58.505 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.505 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.505 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:58.505 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022278+0000", 2024-09-21T21:42:58.506 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00017486800000000001 2024-09-21T21:42:58.506 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:58.506 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:58.506 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:58.506 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.506 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.506 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13631 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7909376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:58.506 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943362+0000", 2024-09-21T21:42:58.506 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 67.518008456999993, 2024-09-21T21:42:58.506 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079195644999999995, 2024-09-21T21:42:58.506 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:58.506 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:58.506 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:58.506 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:58.507 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:58.507 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:58.507 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13631 2024-09-21T21:42:58.507 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.507 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:58.507 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.507 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:58.507 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943362+0000", 2024-09-21T21:42:58.507 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:58.507 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.507 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.507 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:58.507 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943362+0000", 2024-09-21T21:42:58.507 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:58.507 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.508 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.508 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:58.508 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943362+0000", 2024-09-21T21:42:58.508 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8799999999999998e-07 2024-09-21T21:42:58.508 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.508 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.508 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:58.508 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943364+0000", 2024-09-21T21:42:58.508 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9850000000000001e-06 2024-09-21T21:42:58.508 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.508 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.508 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:58.508 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943371+0000", 2024-09-21T21:42:58.508 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0650000000000001e-06 2024-09-21T21:42:58.508 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.509 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.509 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:58.509 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943376+0000", 2024-09-21T21:42:58.509 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8250000000000004e-06 2024-09-21T21:42:58.509 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.509 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.509 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:58.509 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022329+0000", 2024-09-21T21:42:58.509 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078952888999999998 2024-09-21T21:42:58.509 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.509 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.509 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:58.509 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022419+0000", 2024-09-21T21:42:58.509 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.9738e-05 2024-09-21T21:42:58.509 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.509 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.510 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:58.510 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022558+0000", 2024-09-21T21:42:58.510 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00013875500000000001 2024-09-21T21:42:58.510 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:58.510 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:58.510 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:58.510 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.510 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.510 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13632 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7966720~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:58.510 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943382+0000", 2024-09-21T21:42:58.510 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 67.517988223000003, 2024-09-21T21:42:58.510 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079495727000000002, 2024-09-21T21:42:58.510 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:58.510 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:58.511 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:58.511 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:58.511 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:58.511 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:58.511 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13632 2024-09-21T21:42:58.511 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.511 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:58.511 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.511 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:58.511 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943382+0000", 2024-09-21T21:42:58.511 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:58.511 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.511 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.511 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:58.511 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943382+0000", 2024-09-21T21:42:58.512 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:58.512 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.512 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.512 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:58.512 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943382+0000", 2024-09-21T21:42:58.512 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7e-07 2024-09-21T21:42:58.513 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.513 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.513 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:58.513 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943385+0000", 2024-09-21T21:42:58.513 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2230000000000001e-06 2024-09-21T21:42:58.513 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.513 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.513 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:58.513 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943391+0000", 2024-09-21T21:42:58.513 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6810000000000001e-06 2024-09-21T21:42:58.513 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.513 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.513 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:58.513 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943397+0000", 2024-09-21T21:42:58.514 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6559999999999997e-06 2024-09-21T21:42:58.514 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.514 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.514 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:58.514 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022601+0000", 2024-09-21T21:42:58.514 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079204051999999997 2024-09-21T21:42:58.514 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.514 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.514 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:58.514 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022704+0000", 2024-09-21T21:42:58.514 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010329599999999999 2024-09-21T21:42:58.514 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.514 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.514 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:58.514 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022878+0000", 2024-09-21T21:42:58.515 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000173449 2024-09-21T21:42:58.515 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:58.515 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:58.515 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:58.515 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.515 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.515 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13633 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7938048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:58.515 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943403+0000", 2024-09-21T21:42:58.515 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 67.517967292999998, 2024-09-21T21:42:58.515 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079794697999999997, 2024-09-21T21:42:58.515 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:58.515 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:58.515 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:58.515 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:58.516 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:58.516 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:58.516 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13633 2024-09-21T21:42:58.516 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.516 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:58.516 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.516 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:58.516 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943403+0000", 2024-09-21T21:42:58.516 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:58.516 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.516 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.516 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:58.516 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943403+0000", 2024-09-21T21:42:58.516 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:58.516 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.517 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.517 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:58.517 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943403+0000", 2024-09-21T21:42:58.517 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.65e-07 2024-09-21T21:42:58.517 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.517 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.517 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:58.517 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943405+0000", 2024-09-21T21:42:58.517 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0020000000000001e-06 2024-09-21T21:42:58.517 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.517 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.517 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:58.517 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943412+0000", 2024-09-21T21:42:58.517 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.8379999999999996e-06 2024-09-21T21:42:58.518 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.518 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.518 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:58.518 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943417+0000", 2024-09-21T21:42:58.518 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7650000000000001e-06 2024-09-21T21:42:58.518 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.518 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.518 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:58.518 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022922+0000", 2024-09-21T21:42:58.518 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079504512999999999 2024-09-21T21:42:58.518 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.518 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.518 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:58.518 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023039+0000", 2024-09-21T21:42:58.519 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000117044 2024-09-21T21:42:58.519 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.519 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.519 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:58.519 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023198+0000", 2024-09-21T21:42:58.519 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000159171 2024-09-21T21:42:58.519 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:58.519 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:58.519 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:58.519 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.519 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.519 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13634 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7995392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:58.519 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943423+0000", 2024-09-21T21:42:58.519 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 67.517947585000002, 2024-09-21T21:42:58.519 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.080142358999999996, 2024-09-21T21:42:58.520 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:58.520 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:58.520 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:58.520 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:58.520 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:58.520 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:58.520 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13634 2024-09-21T21:42:58.520 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.520 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:58.520 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.520 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:58.520 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943423+0000", 2024-09-21T21:42:58.520 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:58.520 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.520 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.521 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:58.521 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943423+0000", 2024-09-21T21:42:58.521 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:58.521 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.521 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.521 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:58.521 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943423+0000", 2024-09-21T21:42:58.521 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.58e-07 2024-09-21T21:42:58.521 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.521 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.521 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:58.521 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943425+0000", 2024-09-21T21:42:58.521 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9180000000000001e-06 2024-09-21T21:42:58.521 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.522 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.522 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:58.522 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943431+0000", 2024-09-21T21:42:58.522 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4439999999999996e-06 2024-09-21T21:42:58.522 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.522 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.522 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:58.522 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943436+0000", 2024-09-21T21:42:58.522 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.887e-06 2024-09-21T21:42:58.522 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.522 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.522 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:58.522 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023266+0000", 2024-09-21T21:42:58.522 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079829622000000003 2024-09-21T21:42:58.522 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.523 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.523 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:58.523 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023384+0000", 2024-09-21T21:42:58.523 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000117658 2024-09-21T21:42:58.523 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.523 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.523 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:58.523 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023565+0000", 2024-09-21T21:42:58.523 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00018147200000000001 2024-09-21T21:42:58.523 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:58.523 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:58.523 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:58.523 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.523 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.523 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13635 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 8024064~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:58.524 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943447+0000", 2024-09-21T21:42:58.524 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 67.517923045000003, 2024-09-21T21:42:58.524 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.080417653000000006, 2024-09-21T21:42:58.524 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:58.524 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:58.524 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:58.524 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:58.524 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:58.524 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:58.524 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13635 2024-09-21T21:42:58.524 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.524 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:58.524 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.524 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:58.524 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943447+0000", 2024-09-21T21:42:58.525 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:58.525 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.525 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.525 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:58.525 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943447+0000", 2024-09-21T21:42:58.525 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:58.525 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.525 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.525 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:58.525 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943448+0000", 2024-09-21T21:42:58.525 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1699999999999999e-07 2024-09-21T21:42:58.525 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.525 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.525 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:58.525 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943450+0000", 2024-09-21T21:42:58.526 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1639999999999999e-06 2024-09-21T21:42:58.526 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.526 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.526 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:58.526 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943456+0000", 2024-09-21T21:42:58.526 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8379999999999998e-06 2024-09-21T21:42:58.526 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.526 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.526 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:58.526 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943460+0000", 2024-09-21T21:42:58.526 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4700000000000004e-06 2024-09-21T21:42:58.526 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.526 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.526 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:58.527 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023595+0000", 2024-09-21T21:42:58.527 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.080134869999999997 2024-09-21T21:42:58.527 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.527 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.527 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:58.527 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023688+0000", 2024-09-21T21:42:58.527 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2794000000000006e-05 2024-09-21T21:42:58.527 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.527 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.527 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:58.527 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023865+0000", 2024-09-21T21:42:58.527 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0001771 2024-09-21T21:42:58.527 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:58.527 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:58.527 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:58.528 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.528 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.528 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13636 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 8052736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:58.528 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943467+0000", 2024-09-21T21:42:58.528 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 67.517903308000001, 2024-09-21T21:42:58.528 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.080694399999999999, 2024-09-21T21:42:58.528 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:58.528 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:58.528 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:58.528 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:58.528 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:58.528 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:58.528 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13636 2024-09-21T21:42:58.528 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.528 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:58.529 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.529 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:58.529 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943467+0000", 2024-09-21T21:42:58.529 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:58.529 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.529 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.529 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:58.529 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943467+0000", 2024-09-21T21:42:58.529 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:58.529 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.529 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.529 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:58.529 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944940+0000", 2024-09-21T21:42:58.529 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0014725109999999999 2024-09-21T21:42:58.529 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.530 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.530 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:58.530 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944949+0000", 2024-09-21T21:42:58.530 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2930000000000006e-06 2024-09-21T21:42:58.530 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.530 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.530 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:58.530 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944958+0000", 2024-09-21T21:42:58.530 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.6899999999999998e-06 2024-09-21T21:42:58.530 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.530 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.530 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:58.530 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944966+0000", 2024-09-21T21:42:58.530 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0690000000000002e-06 2024-09-21T21:42:58.531 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.531 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.531 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:58.531 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023900+0000", 2024-09-21T21:42:58.531 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078934371000000003 2024-09-21T21:42:58.531 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.531 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.531 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:58.531 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024009+0000", 2024-09-21T21:42:58.531 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000108857 2024-09-21T21:42:58.531 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.531 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.531 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:58.531 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024161+0000", 2024-09-21T21:42:58.531 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000152609 2024-09-21T21:42:58.532 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:58.532 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:58.532 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:58.532 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.532 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.532 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13637 3.12 3.1dc31b72 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:58.532 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.944978+0000", 2024-09-21T21:42:58.532 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 67.516392373000002, 2024-09-21T21:42:58.532 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079232604999999998, 2024-09-21T21:42:58.532 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:58.532 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:58.532 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:42:58.532 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:58.532 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:58.532 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:58.533 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13637 2024-09-21T21:42:58.533 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.533 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:58.533 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.533 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:58.533 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944978+0000", 2024-09-21T21:42:58.533 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:58.533 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.533 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.533 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:58.533 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944978+0000", 2024-09-21T21:42:58.533 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:58.533 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.533 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.534 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:58.534 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944978+0000", 2024-09-21T21:42:58.534 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9100000000000004e-07 2024-09-21T21:42:58.534 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.534 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.534 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:58.534 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944981+0000", 2024-09-21T21:42:58.534 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3290000000000001e-06 2024-09-21T21:42:58.534 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.534 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.534 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:58.534 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944987+0000", 2024-09-21T21:42:58.534 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.28e-06 2024-09-21T21:42:58.534 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.534 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.535 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:58.535 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944992+0000", 2024-09-21T21:42:58.535 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8520000000000003e-06 2024-09-21T21:42:58.535 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.535 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.535 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:58.535 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024196+0000", 2024-09-21T21:42:58.535 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079203673000000002 2024-09-21T21:42:58.535 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.535 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.535 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:58.535 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024211+0000", 2024-09-21T21:42:58.535 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.4980000000000001e-05 2024-09-21T21:42:58.535 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:58.535 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:58.536 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:58.536 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.536 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.536 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13638 3.12 3.1dc31b72 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:58.536 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.945000+0000", 2024-09-21T21:42:58.536 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 67.516370245000005, 2024-09-21T21:42:58.536 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079257465999999999, 2024-09-21T21:42:58.536 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:58.536 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:58.536 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:42:58.536 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:58.536 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:58.536 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:58.536 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13638 2024-09-21T21:42:58.536 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.537 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:58.537 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.537 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:58.537 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945000+0000", 2024-09-21T21:42:58.537 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:58.537 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.537 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.537 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:58.537 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945000+0000", 2024-09-21T21:42:58.537 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:58.537 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.537 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.537 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:58.537 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945001+0000", 2024-09-21T21:42:58.537 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.32e-07 2024-09-21T21:42:58.538 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.538 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.538 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:58.538 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945003+0000", 2024-09-21T21:42:58.538 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2309999999999999e-06 2024-09-21T21:42:58.538 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.538 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.538 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:58.538 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945009+0000", 2024-09-21T21:42:58.538 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7450000000000001e-06 2024-09-21T21:42:58.538 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.538 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.538 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:58.538 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945013+0000", 2024-09-21T21:42:58.538 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.673e-06 2024-09-21T21:42:58.539 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.539 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.539 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:58.539 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024245+0000", 2024-09-21T21:42:58.539 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079232160999999995 2024-09-21T21:42:58.539 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.539 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.539 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:58.539 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024258+0000", 2024-09-21T21:42:58.539 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.2224e-05 2024-09-21T21:42:58.539 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:58.539 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:58.539 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:58.539 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.539 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.540 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13639 3.12 3.1dc31b72 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:58.540 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.945034+0000", 2024-09-21T21:42:58.540 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 67.516336445999997, 2024-09-21T21:42:58.540 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079260627, 2024-09-21T21:42:58.540 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:58.540 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:58.540 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:42:58.540 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:58.540 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:58.540 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:58.540 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13639 2024-09-21T21:42:58.540 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.540 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:58.540 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.540 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:58.541 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945034+0000", 2024-09-21T21:42:58.541 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:58.541 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.541 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.541 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:58.541 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945034+0000", 2024-09-21T21:42:58.541 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:58.541 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.541 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.541 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:58.541 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945034+0000", 2024-09-21T21:42:58.541 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3799999999999998e-07 2024-09-21T21:42:58.541 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.541 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.541 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:58.542 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945036+0000", 2024-09-21T21:42:58.542 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.052e-06 2024-09-21T21:42:58.542 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.542 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.542 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:58.542 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945040+0000", 2024-09-21T21:42:58.542 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1169999999999997e-06 2024-09-21T21:42:58.542 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.542 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.542 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:58.542 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945043+0000", 2024-09-21T21:42:58.542 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.8439999999999998e-06 2024-09-21T21:42:58.542 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.542 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.542 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:58.543 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024286+0000", 2024-09-21T21:42:58.543 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079242483000000002 2024-09-21T21:42:58.543 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.543 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.543 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:58.543 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024295+0000", 2024-09-21T21:42:58.543 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.7930000000000003e-06 2024-09-21T21:42:58.543 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:58.543 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:58.543 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:58.543 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.543 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.543 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13640 3.12 3.1dc31b72 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:58.543 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.945047+0000", 2024-09-21T21:42:58.543 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 67.516322923999994, 2024-09-21T21:42:58.544 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079412321999999994, 2024-09-21T21:42:58.544 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:58.544 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:58.544 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:42:58.544 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:58.544 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:58.544 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:58.544 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13640 2024-09-21T21:42:58.544 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.544 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:58.544 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.544 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:58.544 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945047+0000", 2024-09-21T21:42:58.544 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:58.544 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.545 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.545 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:58.545 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945047+0000", 2024-09-21T21:42:58.545 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:58.545 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.545 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.545 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:58.545 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945048+0000", 2024-09-21T21:42:58.545 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0499999999999999e-07 2024-09-21T21:42:58.545 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.545 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.545 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:58.545 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945049+0000", 2024-09-21T21:42:58.545 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.3939999999999999e-06 2024-09-21T21:42:58.545 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.545 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.546 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:58.546 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945053+0000", 2024-09-21T21:42:58.546 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1760000000000003e-06 2024-09-21T21:42:58.546 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.546 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.546 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:58.546 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945056+0000", 2024-09-21T21:42:58.546 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.6129999999999998e-06 2024-09-21T21:42:58.546 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.546 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.546 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:58.546 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024360+0000", 2024-09-21T21:42:58.546 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079303645000000006 2024-09-21T21:42:58.546 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.546 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.547 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:58.547 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024460+0000", 2024-09-21T21:42:58.547 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000100189 2024-09-21T21:42:58.547 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:58.547 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:58.547 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:58.547 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:58.547 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:58.547 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:58.548 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T21:42:58.590 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:42:58.588+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:42:58.653 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:58.652+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:58.692 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:58.692 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:42:58.692 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 0 2024-09-21T21:42:58.692 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:58.701 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T21:42:58.785 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:58.784+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:58.843 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:58.843 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:42:58.844 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:42:58.844 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:42:58.844 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:58.852 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T21:42:58.894 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:42:58.892+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:42:58.955 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:58.954+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:42:58.998 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:58.998 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 20, 2024-09-21T21:42:58.998 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 600, 2024-09-21T21:42:58.998 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:42:58.998 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.998 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4824.0:1564 3.2 3:4647520c:::rbd_data.1121b97f2925.0000000000000006:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:42:58.998 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:42:58.998 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 117.359334625, 2024-09-21T21:42:58.998 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056097347999999998, 2024-09-21T21:42:58.998 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:58.998 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:58.999 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:58.999 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:58.999 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4824", 2024-09-21T21:42:58.999 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:42:58.999 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1564 2024-09-21T21:42:58.999 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.999 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:58.999 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.999 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:58.999 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:42:58.999 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:58.999 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:58.999 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:58.999 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:59.000 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:42:59.000 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:59.000 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.000 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.000 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:59.000 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:42:59.000 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6499999999999999e-07 2024-09-21T21:42:59.000 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.000 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.000 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:59.000 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636067+0000", 2024-09-21T21:42:59.000 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3879999999999998e-06 2024-09-21T21:42:59.000 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.000 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.000 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:59.001 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636075+0000", 2024-09-21T21:42:59.001 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5480000000000004e-06 2024-09-21T21:42:59.001 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.001 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.001 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:59.001 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636080+0000", 2024-09-21T21:42:59.001 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.339e-06 2024-09-21T21:42:59.001 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.001 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.001 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:59.001 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636133+0000", 2024-09-21T21:42:59.001 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2930000000000003e-05 2024-09-21T21:42:59.001 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.001 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.002 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:59.002 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636265+0000", 2024-09-21T21:42:59.002 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000132058 2024-09-21T21:42:59.002 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.002 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.002 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:59.002 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.692162+0000", 2024-09-21T21:42:59.002 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055896620000000001 2024-09-21T21:42:59.002 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:59.002 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:59.002 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:59.002 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.002 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.002 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3260 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 1437696~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:42:59.003 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.196297+0000", 2024-09-21T21:42:59.003 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 71.799102414999993, 2024-09-21T21:42:59.003 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056071022999999998, 2024-09-21T21:42:59.003 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:59.003 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:59.003 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:59.003 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:59.003 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:59.003 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:59.003 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3260 2024-09-21T21:42:59.003 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.003 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:59.003 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.003 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:59.004 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196297+0000", 2024-09-21T21:42:59.004 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:59.004 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.004 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.004 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:59.004 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196297+0000", 2024-09-21T21:42:59.004 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:59.004 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.004 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.004 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:59.004 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196918+0000", 2024-09-21T21:42:59.004 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00062148699999999999 2024-09-21T21:42:59.004 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.004 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.004 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:59.005 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196930+0000", 2024-09-21T21:42:59.005 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1657e-05 2024-09-21T21:42:59.005 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.005 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.005 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:59.005 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196944+0000", 2024-09-21T21:42:59.005 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.4549000000000001e-05 2024-09-21T21:42:59.005 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.005 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.005 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:59.005 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196951+0000", 2024-09-21T21:42:59.005 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6749999999999996e-06 2024-09-21T21:42:59.005 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.005 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.005 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:59.006 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252208+0000", 2024-09-21T21:42:59.006 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055256656000000001 2024-09-21T21:42:59.006 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.006 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.006 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:59.006 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252278+0000", 2024-09-21T21:42:59.006 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9882000000000006e-05 2024-09-21T21:42:59.006 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.006 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.006 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:59.006 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252368+0000", 2024-09-21T21:42:59.006 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.0117000000000005e-05 2024-09-21T21:42:59.006 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:59.007 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:59.007 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:59.007 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.007 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.007 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3263 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 3559424~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:42:59.007 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.197041+0000", 2024-09-21T21:42:59.007 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 71.798358601000004, 2024-09-21T21:42:59.007 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055935804999999998, 2024-09-21T21:42:59.007 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:59.007 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:59.007 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:59.007 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:59.007 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:59.008 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:59.008 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3263 2024-09-21T21:42:59.008 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.008 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:59.008 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.008 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:59.008 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197041+0000", 2024-09-21T21:42:59.008 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:59.008 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.008 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.008 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:59.008 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197041+0000", 2024-09-21T21:42:59.008 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:59.008 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.008 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.009 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:59.009 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197624+0000", 2024-09-21T21:42:59.009 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00058358100000000001 2024-09-21T21:42:59.009 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.009 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.009 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:59.009 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197629+0000", 2024-09-21T21:42:59.009 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5909999999999998e-06 2024-09-21T21:42:59.009 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.009 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.009 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:59.009 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197639+0000", 2024-09-21T21:42:59.009 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0257e-05 2024-09-21T21:42:59.009 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.009 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.010 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:59.010 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.197646+0000", 2024-09-21T21:42:59.010 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.7229999999999998e-06 2024-09-21T21:42:59.010 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.010 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.010 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:59.010 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252804+0000", 2024-09-21T21:42:59.010 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055158134999999997 2024-09-21T21:42:59.010 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.010 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.010 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:59.010 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252882+0000", 2024-09-21T21:42:59.010 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7791000000000001e-05 2024-09-21T21:42:59.011 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.011 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.011 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:59.011 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252976+0000", 2024-09-21T21:42:59.011 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.4727000000000003e-05 2024-09-21T21:42:59.011 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:59.011 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:59.011 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:59.011 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.011 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.011 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3437 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 925696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:42:59.011 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.205888+0000", 2024-09-21T21:42:59.011 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 71.789511320000003, 2024-09-21T21:42:59.011 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055953808000000001, 2024-09-21T21:42:59.012 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:59.012 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:59.012 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:59.012 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:59.012 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:59.012 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:59.012 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3437 2024-09-21T21:42:59.012 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.012 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:59.012 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.012 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:59.012 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205888+0000", 2024-09-21T21:42:59.012 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:59.012 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.013 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.013 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:59.013 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205888+0000", 2024-09-21T21:42:59.013 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:59.013 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.013 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.013 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:59.013 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205888+0000", 2024-09-21T21:42:59.013 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.75e-07 2024-09-21T21:42:59.013 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.013 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.013 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:59.013 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205891+0000", 2024-09-21T21:42:59.013 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3750000000000001e-06 2024-09-21T21:42:59.014 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.014 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.014 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:59.014 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205897+0000", 2024-09-21T21:42:59.014 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.285e-06 2024-09-21T21:42:59.014 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.014 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.014 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:59.014 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205902+0000", 2024-09-21T21:42:59.014 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6290000000000001e-06 2024-09-21T21:42:59.014 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.014 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.014 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:59.014 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261686+0000", 2024-09-21T21:42:59.015 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055784586999999997 2024-09-21T21:42:59.015 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.015 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.015 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:59.015 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261764+0000", 2024-09-21T21:42:59.015 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7591999999999998e-05 2024-09-21T21:42:59.015 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.015 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.015 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:59.015 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261842+0000", 2024-09-21T21:42:59.015 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7865000000000001e-05 2024-09-21T21:42:59.015 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:59.015 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:59.015 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:59.016 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.016 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.016 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3438 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 954368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:42:59.016 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.205908+0000", 2024-09-21T21:42:59.016 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 71.789491455000004, 2024-09-21T21:42:59.016 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056083169000000002, 2024-09-21T21:42:59.016 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:59.016 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:59.016 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:59.016 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:59.016 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:59.016 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:59.016 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3438 2024-09-21T21:42:59.016 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.017 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:59.017 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.017 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:59.017 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205908+0000", 2024-09-21T21:42:59.017 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:59.017 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.017 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.017 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:59.017 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205908+0000", 2024-09-21T21:42:59.017 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:59.017 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.017 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.017 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:59.017 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206431+0000", 2024-09-21T21:42:59.018 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00052282100000000003 2024-09-21T21:42:59.018 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.018 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.018 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:59.018 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206442+0000", 2024-09-21T21:42:59.018 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1695000000000001e-05 2024-09-21T21:42:59.018 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.018 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.018 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:59.018 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206452+0000", 2024-09-21T21:42:59.018 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.4560000000000006e-06 2024-09-21T21:42:59.018 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.018 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.018 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:59.018 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206458+0000", 2024-09-21T21:42:59.019 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4670000000000001e-06 2024-09-21T21:42:59.019 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.019 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.019 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:59.019 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261871+0000", 2024-09-21T21:42:59.019 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055412712000000003 2024-09-21T21:42:59.019 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.019 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.019 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:59.019 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261928+0000", 2024-09-21T21:42:59.019 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6631e-05 2024-09-21T21:42:59.019 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.019 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.019 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:59.020 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261991+0000", 2024-09-21T21:42:59.020 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3386999999999995e-05 2024-09-21T21:42:59.020 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:59.020 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:59.020 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:59.020 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.020 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.020 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3444 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 2359296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:42:59.020 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.206533+0000", 2024-09-21T21:42:59.020 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 71.788865935000004, 2024-09-21T21:42:59.020 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055974239000000002, 2024-09-21T21:42:59.020 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:59.020 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:59.020 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:59.021 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:59.021 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:59.021 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:59.021 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3444 2024-09-21T21:42:59.021 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.021 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:59.021 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.021 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:59.021 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206533+0000", 2024-09-21T21:42:59.021 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:59.021 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.021 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.021 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:59.021 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206533+0000", 2024-09-21T21:42:59.022 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:59.022 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.022 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.022 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:59.022 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206710+0000", 2024-09-21T21:42:59.022 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000176293 2024-09-21T21:42:59.022 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.022 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.022 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:59.022 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206714+0000", 2024-09-21T21:42:59.022 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1989999999999999e-06 2024-09-21T21:42:59.022 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.022 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.022 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:59.023 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206723+0000", 2024-09-21T21:42:59.023 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.9199999999999993e-06 2024-09-21T21:42:59.023 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.023 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.023 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:59.023 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206729+0000", 2024-09-21T21:42:59.023 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0739999999999998e-06 2024-09-21T21:42:59.023 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.023 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.023 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:59.023 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262339+0000", 2024-09-21T21:42:59.023 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055610166000000003 2024-09-21T21:42:59.023 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.023 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.023 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:59.024 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262419+0000", 2024-09-21T21:42:59.024 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0432999999999994e-05 2024-09-21T21:42:59.024 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.024 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.024 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:59.024 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262507+0000", 2024-09-21T21:42:59.024 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.8153999999999997e-05 2024-09-21T21:42:59.024 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:59.024 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:59.024 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:59.024 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.024 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.024 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10362 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:59.024 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840386+0000", 2024-09-21T21:42:59.025 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 45.155013091000001, 2024-09-21T21:42:59.025 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056546515999999998, 2024-09-21T21:42:59.025 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:59.025 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:59.025 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:59.025 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:59.025 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:59.025 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:59.025 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10362 2024-09-21T21:42:59.025 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.025 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:59.025 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.025 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:59.026 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840386+0000", 2024-09-21T21:42:59.026 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:59.026 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.026 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.026 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:59.026 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840386+0000", 2024-09-21T21:42:59.026 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:59.026 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.026 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.026 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:59.026 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840386+0000", 2024-09-21T21:42:59.026 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.01e-07 2024-09-21T21:42:59.026 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.026 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.027 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:59.027 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840389+0000", 2024-09-21T21:42:59.027 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.9739999999999998e-06 2024-09-21T21:42:59.027 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.027 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.027 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:59.027 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840396+0000", 2024-09-21T21:42:59.027 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2280000000000003e-06 2024-09-21T21:42:59.027 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.027 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.027 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:59.027 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840400+0000", 2024-09-21T21:42:59.027 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4429999999999996e-06 2024-09-21T21:42:59.027 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.027 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.028 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:59.028 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.890896+0000", 2024-09-21T21:42:59.028 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.050495717000000002 2024-09-21T21:42:59.028 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.028 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.028 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:59.028 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.891012+0000", 2024-09-21T21:42:59.028 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011602 2024-09-21T21:42:59.028 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.028 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.028 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:59.028 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.891274+0000", 2024-09-21T21:42:59.028 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00026240200000000001 2024-09-21T21:42:59.028 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.028 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.029 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:59.029 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.894810+0000", 2024-09-21T21:42:59.029 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0035353630000000001 2024-09-21T21:42:59.029 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.029 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.029 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:59.029 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.896818+0000", 2024-09-21T21:42:59.029 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.002008681 2024-09-21T21:42:59.029 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.029 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.029 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:59.029 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.896850+0000", 2024-09-21T21:42:59.029 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.1826999999999999e-05 2024-09-21T21:42:59.029 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.030 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.030 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:59.030 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.896933+0000", 2024-09-21T21:42:59.030 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.2460000000000007e-05 2024-09-21T21:42:59.030 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:59.030 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:59.030 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:59.030 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.030 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.030 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10363 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:59.030 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840606+0000", 2024-09-21T21:42:59.030 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 45.154793581, 2024-09-21T21:42:59.030 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056546011, 2024-09-21T21:42:59.030 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:59.030 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:59.031 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:59.031 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:59.031 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:59.031 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:59.031 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10363 2024-09-21T21:42:59.031 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.031 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:59.031 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.031 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:59.031 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840606+0000", 2024-09-21T21:42:59.031 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:59.031 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.031 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.031 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:59.032 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840606+0000", 2024-09-21T21:42:59.032 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:59.032 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.032 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.032 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:59.032 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840606+0000", 2024-09-21T21:42:59.032 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.4700000000000001e-07 2024-09-21T21:42:59.032 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.032 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.032 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:59.032 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840611+0000", 2024-09-21T21:42:59.032 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4649999999999996e-06 2024-09-21T21:42:59.032 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.032 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.032 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:59.033 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840620+0000", 2024-09-21T21:42:59.033 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.0350000000000007e-06 2024-09-21T21:42:59.033 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.033 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.033 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:59.033 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840626+0000", 2024-09-21T21:42:59.033 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7180000000000001e-06 2024-09-21T21:42:59.033 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.033 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.033 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:59.033 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.891665+0000", 2024-09-21T21:42:59.033 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.051039924 2024-09-21T21:42:59.033 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.033 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.033 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:59.034 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.891770+0000", 2024-09-21T21:42:59.034 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010445700000000001 2024-09-21T21:42:59.034 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.034 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.034 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:59.034 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.892147+0000", 2024-09-21T21:42:59.034 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00037731800000000002 2024-09-21T21:42:59.034 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.034 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.034 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:59.034 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.896776+0000", 2024-09-21T21:42:59.034 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0046289759999999999 2024-09-21T21:42:59.034 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.034 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.034 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:59.035 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897003+0000", 2024-09-21T21:42:59.035 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00022641299999999999 2024-09-21T21:42:59.035 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.035 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.035 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:59.035 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897064+0000", 2024-09-21T21:42:59.035 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1571999999999999e-05 2024-09-21T21:42:59.035 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.035 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.035 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:59.035 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897152+0000", 2024-09-21T21:42:59.035 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.7386000000000007e-05 2024-09-21T21:42:59.035 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:59.035 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:59.035 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:59.036 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.036 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.036 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10364 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:59.036 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840641+0000", 2024-09-21T21:42:59.036 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 45.154757760999999, 2024-09-21T21:42:59.036 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056671527999999999, 2024-09-21T21:42:59.036 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:59.036 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:59.036 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:59.036 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:59.036 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:59.036 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:59.036 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10364 2024-09-21T21:42:59.036 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.036 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:59.037 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.037 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:59.037 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840641+0000", 2024-09-21T21:42:59.037 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:59.037 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.037 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.037 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:59.037 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840641+0000", 2024-09-21T21:42:59.037 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:59.037 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.037 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.037 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:59.037 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840642+0000", 2024-09-21T21:42:59.037 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7899999999999999e-07 2024-09-21T21:42:59.038 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.038 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.038 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:59.038 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840645+0000", 2024-09-21T21:42:59.038 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0010000000000002e-06 2024-09-21T21:42:59.038 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.038 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.038 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:59.038 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840654+0000", 2024-09-21T21:42:59.038 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.8570000000000001e-06 2024-09-21T21:42:59.038 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.038 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.038 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:59.039 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840659+0000", 2024-09-21T21:42:59.039 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.558e-06 2024-09-21T21:42:59.039 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.039 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.039 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:59.039 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.892656+0000", 2024-09-21T21:42:59.039 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.051996586999999997 2024-09-21T21:42:59.039 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.039 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.039 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:59.039 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.892721+0000", 2024-09-21T21:42:59.039 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5010000000000003e-05 2024-09-21T21:42:59.039 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.039 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.039 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:59.040 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.893112+0000", 2024-09-21T21:42:59.040 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00039133899999999999 2024-09-21T21:42:59.040 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.040 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.040 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:59.040 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.896784+0000", 2024-09-21T21:42:59.040 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0036713679999999999 2024-09-21T21:42:59.040 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.040 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.040 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:59.040 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897204+0000", 2024-09-21T21:42:59.040 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00041991400000000002 2024-09-21T21:42:59.040 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.040 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.041 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:59.041 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897234+0000", 2024-09-21T21:42:59.041 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0307999999999998e-05 2024-09-21T21:42:59.041 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.041 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.041 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:59.041 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897313+0000", 2024-09-21T21:42:59.041 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.9107e-05 2024-09-21T21:42:59.041 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:59.041 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:59.041 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:59.041 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.041 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.041 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10365 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:59.042 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840668+0000", 2024-09-21T21:42:59.042 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 45.154730895999997, 2024-09-21T21:42:59.042 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.058761225, 2024-09-21T21:42:59.042 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:59.042 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:59.042 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:59.042 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:59.042 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:59.042 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:59.042 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10365 2024-09-21T21:42:59.042 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.042 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:59.042 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.042 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:59.043 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840668+0000", 2024-09-21T21:42:59.043 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:59.043 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.043 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.043 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:59.043 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840668+0000", 2024-09-21T21:42:59.043 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:59.043 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.043 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.043 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:59.043 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840669+0000", 2024-09-21T21:42:59.043 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6300000000000001e-07 2024-09-21T21:42:59.043 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.043 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.044 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:59.044 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840672+0000", 2024-09-21T21:42:59.044 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.072e-06 2024-09-21T21:42:59.044 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.044 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.044 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:59.044 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840678+0000", 2024-09-21T21:42:59.044 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.7039999999999997e-06 2024-09-21T21:42:59.044 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.044 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.044 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:59.044 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840683+0000", 2024-09-21T21:42:59.044 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6759999999999998e-06 2024-09-21T21:42:59.044 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.045 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.045 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:59.045 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.893929+0000", 2024-09-21T21:42:59.045 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.053246136999999999 2024-09-21T21:42:59.045 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.045 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.045 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:59.045 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.894082+0000", 2024-09-21T21:42:59.045 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00015301799999999999 2024-09-21T21:42:59.045 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.045 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.045 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:59.045 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.894357+0000", 2024-09-21T21:42:59.045 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00027507999999999998 2024-09-21T21:42:59.046 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.046 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.046 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:59.046 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897161+0000", 2024-09-21T21:42:59.046 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0028036810000000001 2024-09-21T21:42:59.046 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.046 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.046 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:59.046 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.899243+0000", 2024-09-21T21:42:59.046 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0020816379999999998 2024-09-21T21:42:59.046 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.046 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.046 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:59.046 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.899306+0000", 2024-09-21T21:42:59.046 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2964000000000002e-05 2024-09-21T21:42:59.047 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.047 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.047 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:59.047 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.899430+0000", 2024-09-21T21:42:59.047 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00012389200000000001 2024-09-21T21:42:59.047 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:59.047 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:59.047 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:59.047 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.047 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.047 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10366 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:59.047 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840691+0000", 2024-09-21T21:42:59.047 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 45.154708196000001, 2024-09-21T21:42:59.047 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.061540984999999999, 2024-09-21T21:42:59.048 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:59.048 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:59.048 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:59.048 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:59.048 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:59.048 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:59.048 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10366 2024-09-21T21:42:59.048 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.048 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:59.048 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.048 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:59.048 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840691+0000", 2024-09-21T21:42:59.048 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:59.048 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.049 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.049 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:59.049 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840691+0000", 2024-09-21T21:42:59.049 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:59.049 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.049 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.049 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:59.049 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840691+0000", 2024-09-21T21:42:59.049 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3099999999999999e-07 2024-09-21T21:42:59.049 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.049 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.049 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:59.049 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840694+0000", 2024-09-21T21:42:59.049 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.5780000000000001e-06 2024-09-21T21:42:59.049 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.050 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.050 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:59.050 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840700+0000", 2024-09-21T21:42:59.050 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9429999999999997e-06 2024-09-21T21:42:59.050 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.050 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.050 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:59.050 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840704+0000", 2024-09-21T21:42:59.050 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6380000000000003e-06 2024-09-21T21:42:59.050 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.050 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.050 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:59.050 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.894846+0000", 2024-09-21T21:42:59.050 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.054141392000000003 2024-09-21T21:42:59.051 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.051 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.051 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:59.051 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.894932+0000", 2024-09-21T21:42:59.051 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.5834000000000005e-05 2024-09-21T21:42:59.051 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.051 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.051 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:59.051 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.895302+0000", 2024-09-21T21:42:59.051 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000370737 2024-09-21T21:42:59.051 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.051 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.051 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:59.052 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897166+0000", 2024-09-21T21:42:59.052 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0018632620000000001 2024-09-21T21:42:59.052 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.052 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.052 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:59.052 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902053+0000", 2024-09-21T21:42:59.052 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0048868289999999997 2024-09-21T21:42:59.052 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.052 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.052 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:59.052 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902111+0000", 2024-09-21T21:42:59.052 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7983999999999999e-05 2024-09-21T21:42:59.052 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.052 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.053 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:59.053 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902232+0000", 2024-09-21T21:42:59.053 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000121457 2024-09-21T21:42:59.053 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:59.053 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:59.053 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:59.053 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.053 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.053 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10367 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:59.053 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840842+0000", 2024-09-21T21:42:59.053 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 45.154557388000001, 2024-09-21T21:42:59.053 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.061549088000000002, 2024-09-21T21:42:59.053 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:59.053 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:59.053 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:59.054 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:59.054 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:59.054 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:59.054 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10367 2024-09-21T21:42:59.054 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.054 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:59.054 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.054 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:59.054 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840842+0000", 2024-09-21T21:42:59.054 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:59.055 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.055 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.055 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:59.055 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840842+0000", 2024-09-21T21:42:59.055 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:59.055 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.055 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.055 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:59.055 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840842+0000", 2024-09-21T21:42:59.055 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3099999999999997e-07 2024-09-21T21:42:59.056 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.056 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.056 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:59.056 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840846+0000", 2024-09-21T21:42:59.056 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.2890000000000002e-06 2024-09-21T21:42:59.056 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.056 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.056 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:59.056 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840855+0000", 2024-09-21T21:42:59.056 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.9619999999999999e-06 2024-09-21T21:42:59.056 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.056 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.056 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:59.056 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840860+0000", 2024-09-21T21:42:59.056 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3759999999999999e-06 2024-09-21T21:42:59.057 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.057 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.057 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:59.057 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.895799+0000", 2024-09-21T21:42:59.057 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.054938569999999999 2024-09-21T21:42:59.057 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.057 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.057 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:59.057 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.895906+0000", 2024-09-21T21:42:59.057 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000107593 2024-09-21T21:42:59.057 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.057 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.057 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:59.057 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.896356+0000", 2024-09-21T21:42:59.058 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00044941700000000002 2024-09-21T21:42:59.058 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.058 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.058 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:59.058 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.898131+0000", 2024-09-21T21:42:59.058 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.001775064 2024-09-21T21:42:59.058 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.058 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.058 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:59.058 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902283+0000", 2024-09-21T21:42:59.058 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0041524839999999997 2024-09-21T21:42:59.058 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.058 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.058 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:59.058 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902316+0000", 2024-09-21T21:42:59.059 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3002e-05 2024-09-21T21:42:59.059 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.059 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.059 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:59.059 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902391+0000", 2024-09-21T21:42:59.059 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.47e-05 2024-09-21T21:42:59.059 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:59.059 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:59.059 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:59.059 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.059 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.059 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10368 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:59.059 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840873+0000", 2024-09-21T21:42:59.059 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 45.154526433999997, 2024-09-21T21:42:59.059 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.063395291000000006, 2024-09-21T21:42:59.060 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:59.060 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:59.060 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:59.060 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:59.060 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:59.060 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:59.060 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10368 2024-09-21T21:42:59.060 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.060 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:59.060 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.060 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:59.060 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840873+0000", 2024-09-21T21:42:59.060 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:59.060 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.061 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.061 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:59.061 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840873+0000", 2024-09-21T21:42:59.061 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:59.061 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.061 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.061 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:59.061 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840873+0000", 2024-09-21T21:42:59.061 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6699999999999999e-07 2024-09-21T21:42:59.061 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.061 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.061 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:59.061 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840876+0000", 2024-09-21T21:42:59.061 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.6410000000000002e-06 2024-09-21T21:42:59.061 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.062 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.062 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:59.062 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840883+0000", 2024-09-21T21:42:59.062 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0389999999999999e-06 2024-09-21T21:42:59.062 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.062 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.062 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:59.062 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840888+0000", 2024-09-21T21:42:59.062 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8679999999999998e-06 2024-09-21T21:42:59.062 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.062 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.062 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:59.062 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897345+0000", 2024-09-21T21:42:59.062 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056457734000000002 2024-09-21T21:42:59.062 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.063 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.063 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:59.063 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897407+0000", 2024-09-21T21:42:59.063 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1822000000000005e-05 2024-09-21T21:42:59.063 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.063 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.063 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:59.063 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897664+0000", 2024-09-21T21:42:59.063 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00025638499999999998 2024-09-21T21:42:59.063 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.063 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.063 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:59.063 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.901136+0000", 2024-09-21T21:42:59.063 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0034728070000000001 2024-09-21T21:42:59.063 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.064 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.064 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:59.064 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904090+0000", 2024-09-21T21:42:59.064 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0029535799999999999 2024-09-21T21:42:59.064 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.064 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.064 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:59.064 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904144+0000", 2024-09-21T21:42:59.064 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4369000000000001e-05 2024-09-21T21:42:59.064 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.064 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.064 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:59.064 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904268+0000", 2024-09-21T21:42:59.064 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00012367899999999999 2024-09-21T21:42:59.065 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:59.065 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:59.065 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:59.065 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.065 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.065 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10369 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:59.065 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840902+0000", 2024-09-21T21:42:59.065 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 45.154496780999999, 2024-09-21T21:42:59.065 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.063538938000000003, 2024-09-21T21:42:59.065 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:59.065 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:59.065 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:59.065 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:59.065 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:59.066 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:59.066 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10369 2024-09-21T21:42:59.066 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.066 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:59.066 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.066 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:59.066 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840902+0000", 2024-09-21T21:42:59.066 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:59.066 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.066 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.066 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:59.066 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840902+0000", 2024-09-21T21:42:59.066 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:59.066 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.067 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.067 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:59.067 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840903+0000", 2024-09-21T21:42:59.067 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2100000000000002e-07 2024-09-21T21:42:59.067 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.067 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.067 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:59.067 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840913+0000", 2024-09-21T21:42:59.067 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.048e-05 2024-09-21T21:42:59.067 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.067 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.067 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:59.067 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840920+0000", 2024-09-21T21:42:59.067 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0330000000000003e-06 2024-09-21T21:42:59.068 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.068 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.068 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:59.068 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840925+0000", 2024-09-21T21:42:59.068 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0170000000000004e-06 2024-09-21T21:42:59.068 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.068 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.068 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:59.068 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.898179+0000", 2024-09-21T21:42:59.068 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.057253671999999999 2024-09-21T21:42:59.068 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.068 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.068 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:59.068 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.898284+0000", 2024-09-21T21:42:59.068 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000105202 2024-09-21T21:42:59.069 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.069 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.069 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:59.069 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.898729+0000", 2024-09-21T21:42:59.069 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00044459300000000002 2024-09-21T21:42:59.069 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.069 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.069 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:59.069 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902246+0000", 2024-09-21T21:42:59.069 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0035171629999999998 2024-09-21T21:42:59.069 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.069 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.069 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:59.069 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904310+0000", 2024-09-21T21:42:59.070 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.002063765 2024-09-21T21:42:59.070 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.070 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.070 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:59.070 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904342+0000", 2024-09-21T21:42:59.070 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.2091999999999997e-05 2024-09-21T21:42:59.070 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.070 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.070 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:59.070 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904441+0000", 2024-09-21T21:42:59.070 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.9500000000000006e-05 2024-09-21T21:42:59.070 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:59.070 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:59.070 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:59.070 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.071 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.071 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10370 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:59.071 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840944+0000", 2024-09-21T21:42:59.071 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 45.154455067999997, 2024-09-21T21:42:59.071 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.064436853000000002, 2024-09-21T21:42:59.071 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:59.071 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:59.071 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:59.071 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:59.071 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:59.071 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:59.071 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10370 2024-09-21T21:42:59.071 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.071 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:59.071 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.072 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:59.072 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840944+0000", 2024-09-21T21:42:59.072 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:59.072 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.072 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.072 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:59.072 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840944+0000", 2024-09-21T21:42:59.072 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:59.072 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.072 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.072 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:59.072 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840945+0000", 2024-09-21T21:42:59.072 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2300000000000002e-07 2024-09-21T21:42:59.072 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.073 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.073 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:59.073 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840947+0000", 2024-09-21T21:42:59.073 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.8200000000000001e-06 2024-09-21T21:42:59.073 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.073 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.073 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:59.073 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840955+0000", 2024-09-21T21:42:59.073 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.8660000000000006e-06 2024-09-21T21:42:59.073 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.073 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.073 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:59.073 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840960+0000", 2024-09-21T21:42:59.073 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8670000000000002e-06 2024-09-21T21:42:59.073 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.074 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.074 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:59.074 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.899460+0000", 2024-09-21T21:42:59.074 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.058499922000000003 2024-09-21T21:42:59.074 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.074 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.074 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:59.074 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.899530+0000", 2024-09-21T21:42:59.074 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9558e-05 2024-09-21T21:42:59.074 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.074 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.074 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:59.074 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.899775+0000", 2024-09-21T21:42:59.074 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000245231 2024-09-21T21:42:59.075 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.075 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.075 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:59.075 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902254+0000", 2024-09-21T21:42:59.075 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0024790519999999998 2024-09-21T21:42:59.075 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.075 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.075 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:59.075 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905183+0000", 2024-09-21T21:42:59.075 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.002929156 2024-09-21T21:42:59.075 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.075 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.075 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:59.075 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905243+0000", 2024-09-21T21:42:59.075 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0055e-05 2024-09-21T21:42:59.076 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.076 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.076 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:59.076 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905381+0000", 2024-09-21T21:42:59.076 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00013790300000000001 2024-09-21T21:42:59.076 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:59.076 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:59.076 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:59.076 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.076 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.076 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10371 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:59.076 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840980+0000", 2024-09-21T21:42:59.076 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 45.154419402000002, 2024-09-21T21:42:59.076 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.064554376999999996, 2024-09-21T21:42:59.076 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:59.077 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:59.077 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:59.077 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:59.077 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:59.077 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:59.077 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10371 2024-09-21T21:42:59.077 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.077 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:59.077 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.077 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:59.077 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840980+0000", 2024-09-21T21:42:59.077 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:59.077 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.077 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.078 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:59.078 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840980+0000", 2024-09-21T21:42:59.078 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:59.078 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.078 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.078 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:59.078 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840980+0000", 2024-09-21T21:42:59.078 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8999999999999997e-07 2024-09-21T21:42:59.078 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.078 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.078 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:59.078 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840983+0000", 2024-09-21T21:42:59.078 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.123e-06 2024-09-21T21:42:59.078 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.078 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.079 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:59.079 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840990+0000", 2024-09-21T21:42:59.079 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9129999999999997e-06 2024-09-21T21:42:59.079 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.079 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.079 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:59.079 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840995+0000", 2024-09-21T21:42:59.079 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0989999999999998e-06 2024-09-21T21:42:59.079 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.079 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.079 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:59.079 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.900168+0000", 2024-09-21T21:42:59.079 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.059172739000000002 2024-09-21T21:42:59.079 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.080 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.080 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:59.080 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.900274+0000", 2024-09-21T21:42:59.080 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010546499999999999 2024-09-21T21:42:59.080 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.080 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.080 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:59.080 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.900703+0000", 2024-09-21T21:42:59.080 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00042978200000000003 2024-09-21T21:42:59.080 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.080 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.080 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:59.080 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.903443+0000", 2024-09-21T21:42:59.080 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0027400749999999998 2024-09-21T21:42:59.081 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.081 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.081 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:59.081 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905430+0000", 2024-09-21T21:42:59.081 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0019868109999999998 2024-09-21T21:42:59.081 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.081 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.081 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:59.081 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905464+0000", 2024-09-21T21:42:59.081 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3599000000000001e-05 2024-09-21T21:42:59.081 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.081 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.081 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:59.081 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905534+0000", 2024-09-21T21:42:59.081 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0281e-05 2024-09-21T21:42:59.082 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:59.082 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:59.082 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:59.082 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.082 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.082 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10372 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:59.082 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.841004+0000", 2024-09-21T21:42:59.082 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 45.154395512999997, 2024-09-21T21:42:59.082 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.064757812999999997, 2024-09-21T21:42:59.082 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:59.082 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:59.082 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:59.082 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:59.083 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:59.083 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:59.083 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10372 2024-09-21T21:42:59.083 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.083 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:59.083 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.083 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:59.083 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841004+0000", 2024-09-21T21:42:59.083 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:59.083 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.083 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.083 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:59.083 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841004+0000", 2024-09-21T21:42:59.083 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:59.084 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.084 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.084 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:59.084 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841004+0000", 2024-09-21T21:42:59.084 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8500000000000002e-07 2024-09-21T21:42:59.084 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.084 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.084 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:59.084 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841007+0000", 2024-09-21T21:42:59.084 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.6390000000000001e-06 2024-09-21T21:42:59.084 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.084 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.084 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:59.084 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841013+0000", 2024-09-21T21:42:59.085 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0070000000000003e-06 2024-09-21T21:42:59.085 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.085 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.085 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:59.085 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841018+0000", 2024-09-21T21:42:59.085 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9119999999999997e-06 2024-09-21T21:42:59.085 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.085 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.085 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:59.085 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.901182+0000", 2024-09-21T21:42:59.085 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.060164444999999997 2024-09-21T21:42:59.085 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.085 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.085 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:59.085 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.901297+0000", 2024-09-21T21:42:59.085 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011468699999999999 2024-09-21T21:42:59.086 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.086 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.086 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:59.086 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.901559+0000", 2024-09-21T21:42:59.086 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000261764 2024-09-21T21:42:59.086 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.086 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.086 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:59.086 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904281+0000", 2024-09-21T21:42:59.086 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0027229329999999999 2024-09-21T21:42:59.086 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.086 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.086 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:59.086 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905588+0000", 2024-09-21T21:42:59.087 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.001306438 2024-09-21T21:42:59.087 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.087 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.087 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:59.087 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905644+0000", 2024-09-21T21:42:59.087 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5747e-05 2024-09-21T21:42:59.087 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.087 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.087 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:59.087 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905761+0000", 2024-09-21T21:42:59.087 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000117856 2024-09-21T21:42:59.087 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:59.087 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:59.087 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:59.088 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.088 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.088 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10373 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:42:59.088 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.841027+0000", 2024-09-21T21:42:59.088 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 45.154372428999999, 2024-09-21T21:42:59.088 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.066941839000000003, 2024-09-21T21:42:59.088 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:59.088 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:59.088 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:42:59.088 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:59.088 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:59.088 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:59.088 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10373 2024-09-21T21:42:59.088 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.088 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:59.089 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.089 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:59.089 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841027+0000", 2024-09-21T21:42:59.089 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:59.089 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.089 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.089 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:59.089 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841027+0000", 2024-09-21T21:42:59.089 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:59.089 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.089 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.089 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:59.089 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841027+0000", 2024-09-21T21:42:59.090 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6199999999999999e-07 2024-09-21T21:42:59.090 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.090 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.090 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:59.090 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841030+0000", 2024-09-21T21:42:59.090 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.729e-06 2024-09-21T21:42:59.090 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.090 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.090 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:59.090 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841036+0000", 2024-09-21T21:42:59.090 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1979999999999998e-06 2024-09-21T21:42:59.090 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.090 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.090 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:59.090 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841042+0000", 2024-09-21T21:42:59.091 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4289999999999998e-06 2024-09-21T21:42:59.091 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.091 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.091 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:59.091 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902433+0000", 2024-09-21T21:42:59.091 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.061390329 2024-09-21T21:42:59.091 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.091 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.091 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:59.091 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902541+0000", 2024-09-21T21:42:59.091 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000108349 2024-09-21T21:42:59.091 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.091 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.091 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:42:59.091 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902983+0000", 2024-09-21T21:42:59.092 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00044158000000000001 2024-09-21T21:42:59.092 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.092 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.092 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:42:59.092 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904490+0000", 2024-09-21T21:42:59.092 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0015074489999999999 2024-09-21T21:42:59.092 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.092 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.092 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:42:59.092 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.907671+0000", 2024-09-21T21:42:59.092 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.003180511 2024-09-21T21:42:59.092 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.092 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.092 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:42:59.093 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.907767+0000", 2024-09-21T21:42:59.093 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6379000000000007e-05 2024-09-21T21:42:59.093 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.093 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.093 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:59.093 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.907969+0000", 2024-09-21T21:42:59.093 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00020152399999999999 2024-09-21T21:42:59.093 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:59.093 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:59.093 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:59.093 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.093 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.093 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:12335 3.1a 3:5b387735:::rbd_data.1121faebdfe3.0000000000000005:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:42:59.093 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:14.411635+0000", 2024-09-21T21:42:59.094 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 44.583764551000002, 2024-09-21T21:42:59.094 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.005973491, 2024-09-21T21:42:59.094 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:59.094 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:59.094 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:59.094 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:59.094 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:59.094 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:59.094 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 12335 2024-09-21T21:42:59.094 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.094 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:59.094 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.094 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:59.094 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411635+0000", 2024-09-21T21:42:59.095 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:59.095 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.095 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.095 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:59.095 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411635+0000", 2024-09-21T21:42:59.095 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:59.095 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.095 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.095 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:59.095 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411635+0000", 2024-09-21T21:42:59.095 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8500000000000001e-07 2024-09-21T21:42:59.095 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.095 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.095 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:59.096 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411639+0000", 2024-09-21T21:42:59.096 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.574e-06 2024-09-21T21:42:59.096 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.096 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.096 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:59.096 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411645+0000", 2024-09-21T21:42:59.096 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6869999999999997e-06 2024-09-21T21:42:59.096 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.096 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.096 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:59.096 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411649+0000", 2024-09-21T21:42:59.096 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.771e-06 2024-09-21T21:42:59.096 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.096 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.096 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:59.097 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411695+0000", 2024-09-21T21:42:59.097 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6e-05 2024-09-21T21:42:59.097 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.097 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.097 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op must wait for map", 2024-09-21T21:42:59.097 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411702+0000", 2024-09-21T21:42:59.097 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5590000000000002e-06 2024-09-21T21:42:59.097 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.097 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.097 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:59.097 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.417222+0000", 2024-09-21T21:42:59.097 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0055198219999999997 2024-09-21T21:42:59.097 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.097 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.098 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for peered", 2024-09-21T21:42:59.098 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.417230+0000", 2024-09-21T21:42:59.098 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.123e-06 2024-09-21T21:42:59.098 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.098 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.098 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:59.098 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.417348+0000", 2024-09-21T21:42:59.098 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.000118029 2024-09-21T21:42:59.098 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.098 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.098 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:59.098 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.417451+0000", 2024-09-21T21:42:59.098 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000103674 2024-09-21T21:42:59.098 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.098 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.099 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:59.099 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.417608+0000", 2024-09-21T21:42:59.099 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00015666700000000001 2024-09-21T21:42:59.099 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:59.099 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:59.099 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:59.099 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.099 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.099 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:12349 3.1a 3:5b387735:::rbd_data.1121faebdfe3.0000000000000005:head [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:42:59.099 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:14.411833+0000", 2024-09-21T21:42:59.099 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 44.583565737999997, 2024-09-21T21:42:59.099 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0065064500000001, 2024-09-21T21:42:59.099 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:59.099 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:59.100 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:59.100 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:59.100 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:59.100 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:59.100 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 12349 2024-09-21T21:42:59.100 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.100 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:59.100 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.100 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:59.100 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411833+0000", 2024-09-21T21:42:59.100 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:59.100 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.100 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.100 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:59.100 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411833+0000", 2024-09-21T21:42:59.101 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:59.101 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.101 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.101 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:59.101 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411834+0000", 2024-09-21T21:42:59.101 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.03e-07 2024-09-21T21:42:59.101 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.101 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.101 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:59.101 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411836+0000", 2024-09-21T21:42:59.101 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.4700000000000001e-06 2024-09-21T21:42:59.101 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.101 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.101 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:59.101 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411843+0000", 2024-09-21T21:42:59.101 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4069999999999998e-06 2024-09-21T21:42:59.102 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.102 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.102 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:59.102 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411848+0000", 2024-09-21T21:42:59.102 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1580000000000004e-06 2024-09-21T21:42:59.102 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.102 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.102 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:59.102 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411901+0000", 2024-09-21T21:42:59.102 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2824000000000001e-05 2024-09-21T21:42:59.102 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.102 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.102 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting_for_map not empty", 2024-09-21T21:42:59.102 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411904+0000", 2024-09-21T21:42:59.103 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0639999999999998e-06 2024-09-21T21:42:59.103 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.103 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.103 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:59.103 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.417252+0000", 2024-09-21T21:42:59.103 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0053485060000000003 2024-09-21T21:42:59.103 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.103 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.103 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for peered", 2024-09-21T21:42:59.103 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.417259+0000", 2024-09-21T21:42:59.103 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.1149999999999996e-06 2024-09-21T21:42:59.103 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.103 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.103 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:59.103 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.418160+0000", 2024-09-21T21:42:59.104 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.000901083 2024-09-21T21:42:59.104 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.104 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.104 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:59.104 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.418236+0000", 2024-09-21T21:42:59.104 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5852000000000005e-05 2024-09-21T21:42:59.104 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.104 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.104 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:59.104 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.418340+0000", 2024-09-21T21:42:59.104 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000103568 2024-09-21T21:42:59.104 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:59.104 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:59.104 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:59.104 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:59.105 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:59.105 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:59.106 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T21:42:59.106 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:59.104+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:59.207 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:59.205+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:59.231 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T21:42:59.250 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:59.250 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:42:59.250 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 0 2024-09-21T21:42:59.251 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:59.260 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T21:42:59.308 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:59.306+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:59.408 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:59.407+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:59.412 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:59.412 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:42:59.412 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:42:59.413 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:42:59.413 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:59.422 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T21:42:59.509 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:59.507+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:59.567 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:42:59.567 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 20, 2024-09-21T21:42:59.567 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 600, 2024-09-21T21:42:59.567 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:42:59.567 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.567 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13296 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 4046848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:59.567 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.796643+0000", 2024-09-21T21:42:59.568 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 68.768068338999996, 2024-09-21T21:42:59.568 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071682019999999999, 2024-09-21T21:42:59.568 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:59.568 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:59.568 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:59.568 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:59.568 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:59.568 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:59.569 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13296 2024-09-21T21:42:59.569 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.569 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:59.569 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.569 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:59.569 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796643+0000", 2024-09-21T21:42:59.569 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:59.570 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.570 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.570 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:59.570 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796643+0000", 2024-09-21T21:42:59.570 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:59.570 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.570 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.571 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:59.571 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796643+0000", 2024-09-21T21:42:59.571 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.7199999999999998e-07 2024-09-21T21:42:59.571 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.571 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.571 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:59.571 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796647+0000", 2024-09-21T21:42:59.572 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6770000000000001e-06 2024-09-21T21:42:59.572 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.572 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.572 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:59.572 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796656+0000", 2024-09-21T21:42:59.572 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2059999999999996e-06 2024-09-21T21:42:59.573 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.573 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.573 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:59.573 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796664+0000", 2024-09-21T21:42:59.573 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7919999999999993e-06 2024-09-21T21:42:59.573 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.573 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.574 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:59.574 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.868186+0000", 2024-09-21T21:42:59.574 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071522273999999997 2024-09-21T21:42:59.574 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.574 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.574 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:59.574 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.868239+0000", 2024-09-21T21:42:59.575 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2787999999999997e-05 2024-09-21T21:42:59.575 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.575 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.575 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:59.575 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.868325+0000", 2024-09-21T21:42:59.575 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.5611000000000004e-05 2024-09-21T21:42:59.576 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:59.576 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:59.576 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:59.576 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.576 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.576 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13561 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 1728512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:59.577 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.823991+0000", 2024-09-21T21:42:59.577 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 68.740720038000006, 2024-09-21T21:42:59.577 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071654762999999996, 2024-09-21T21:42:59.577 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:59.577 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:59.577 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:59.577 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:59.578 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:59.578 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:59.578 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13561 2024-09-21T21:42:59.578 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.578 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:59.579 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.579 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:59.579 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.823991+0000", 2024-09-21T21:42:59.579 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:59.579 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.579 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.579 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:59.580 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.823991+0000", 2024-09-21T21:42:59.580 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:59.580 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.580 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.580 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:59.580 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.823991+0000", 2024-09-21T21:42:59.581 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9299999999999998e-07 2024-09-21T21:42:59.581 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.581 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.581 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:59.581 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.823994+0000", 2024-09-21T21:42:59.581 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2369999999999999e-06 2024-09-21T21:42:59.581 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.582 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.582 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:59.582 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824000+0000", 2024-09-21T21:42:59.582 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.55e-06 2024-09-21T21:42:59.582 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.582 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.583 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:59.583 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824005+0000", 2024-09-21T21:42:59.583 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4329999999999997e-06 2024-09-21T21:42:59.583 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.583 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.583 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:59.583 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.895360+0000", 2024-09-21T21:42:59.584 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071355426 2024-09-21T21:42:59.584 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.584 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.584 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:59.584 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.895477+0000", 2024-09-21T21:42:59.584 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0001171 2024-09-21T21:42:59.584 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.585 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.585 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:59.585 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.895646+0000", 2024-09-21T21:42:59.585 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000168524 2024-09-21T21:42:59.585 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:59.585 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:59.585 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:59.585 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.586 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.586 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13562 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 1757184~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:59.586 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.824011+0000", 2024-09-21T21:42:59.586 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 68.740700161000007, 2024-09-21T21:42:59.586 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072020453999999998, 2024-09-21T21:42:59.586 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:59.586 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:59.586 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:59.586 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:59.586 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:59.586 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:59.586 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13562 2024-09-21T21:42:59.586 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.587 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:59.587 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.587 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:59.587 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824011+0000", 2024-09-21T21:42:59.587 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:59.587 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.587 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.587 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:59.587 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824011+0000", 2024-09-21T21:42:59.587 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:59.587 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.587 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.587 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:59.588 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824011+0000", 2024-09-21T21:42:59.588 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5600000000000001e-07 2024-09-21T21:42:59.588 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.588 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.588 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:59.588 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824013+0000", 2024-09-21T21:42:59.588 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9800000000000001e-06 2024-09-21T21:42:59.588 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.588 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.588 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:59.588 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824019+0000", 2024-09-21T21:42:59.588 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7339999999999996e-06 2024-09-21T21:42:59.588 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.589 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.589 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:59.589 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824023+0000", 2024-09-21T21:42:59.589 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3680000000000004e-06 2024-09-21T21:42:59.589 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.589 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.589 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:59.589 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.895699+0000", 2024-09-21T21:42:59.589 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071675597999999993 2024-09-21T21:42:59.589 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.589 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.589 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:59.589 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.895830+0000", 2024-09-21T21:42:59.589 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000130858 2024-09-21T21:42:59.590 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.590 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.590 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:59.590 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896031+0000", 2024-09-21T21:42:59.590 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00020155999999999999 2024-09-21T21:42:59.590 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:59.590 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:59.590 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:59.590 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.590 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.590 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13563 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 323584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:59.590 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.824029+0000", 2024-09-21T21:42:59.590 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 68.740681721000001, 2024-09-21T21:42:59.591 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072307563000000005, 2024-09-21T21:42:59.591 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:59.591 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:59.591 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:59.591 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:59.591 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:59.591 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:59.591 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13563 2024-09-21T21:42:59.591 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.591 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:59.591 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.591 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:59.591 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824029+0000", 2024-09-21T21:42:59.591 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:59.592 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.592 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.592 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:59.592 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824029+0000", 2024-09-21T21:42:59.592 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:59.592 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.592 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.592 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:59.592 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824030+0000", 2024-09-21T21:42:59.592 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1699999999999999e-07 2024-09-21T21:42:59.592 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.592 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.592 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:59.592 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824032+0000", 2024-09-21T21:42:59.592 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0320000000000002e-06 2024-09-21T21:42:59.593 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.593 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.593 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:59.593 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824038+0000", 2024-09-21T21:42:59.593 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8769999999999998e-06 2024-09-21T21:42:59.593 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.593 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.593 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:59.593 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824042+0000", 2024-09-21T21:42:59.593 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5650000000000003e-06 2024-09-21T21:42:59.593 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.593 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.593 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:59.593 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896058+0000", 2024-09-21T21:42:59.594 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072015941 2024-09-21T21:42:59.594 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.594 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.594 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:59.594 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896164+0000", 2024-09-21T21:42:59.594 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010593499999999999 2024-09-21T21:42:59.594 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.594 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.594 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:59.594 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896337+0000", 2024-09-21T21:42:59.595 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00017279599999999999 2024-09-21T21:42:59.595 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:59.595 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:59.595 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:59.595 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.595 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.595 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13564 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 983040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:59.595 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.824048+0000", 2024-09-21T21:42:59.595 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 68.740662646000004, 2024-09-21T21:42:59.595 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072491193999999995, 2024-09-21T21:42:59.595 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:59.595 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:59.595 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:59.596 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:59.596 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:59.596 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:59.596 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13564 2024-09-21T21:42:59.596 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.596 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:59.596 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.596 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:59.596 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824048+0000", 2024-09-21T21:42:59.596 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:59.596 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.596 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.597 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:59.597 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824048+0000", 2024-09-21T21:42:59.597 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:59.597 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.597 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.597 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:59.597 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824663+0000", 2024-09-21T21:42:59.597 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00061494299999999996 2024-09-21T21:42:59.597 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.597 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.597 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:59.597 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824669+0000", 2024-09-21T21:42:59.597 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2900000000000002e-06 2024-09-21T21:42:59.598 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.598 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.598 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:59.598 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824679+0000", 2024-09-21T21:42:59.598 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0338e-05 2024-09-21T21:42:59.598 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.598 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.598 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:59.598 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824687+0000", 2024-09-21T21:42:59.598 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.9049999999999997e-06 2024-09-21T21:42:59.598 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.598 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.598 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:59.599 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896381+0000", 2024-09-21T21:42:59.599 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071694305999999999 2024-09-21T21:42:59.599 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.599 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.599 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:59.599 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896445+0000", 2024-09-21T21:42:59.599 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3869999999999997e-05 2024-09-21T21:42:59.599 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.599 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.599 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:59.599 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896540+0000", 2024-09-21T21:42:59.599 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.4542000000000004e-05 2024-09-21T21:42:59.599 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:59.601 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:59.601 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:59.601 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.602 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.602 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13565 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3190784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:59.602 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.824704+0000", 2024-09-21T21:42:59.602 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 68.740006774999998, 2024-09-21T21:42:59.602 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072014745000000005, 2024-09-21T21:42:59.602 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:59.602 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:59.602 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:59.602 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:59.602 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:59.602 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:59.603 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13565 2024-09-21T21:42:59.603 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.603 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:59.603 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.603 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:59.603 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824704+0000", 2024-09-21T21:42:59.603 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:59.603 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.603 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.603 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:59.603 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824704+0000", 2024-09-21T21:42:59.603 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:59.603 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.603 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.604 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:59.604 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824705+0000", 2024-09-21T21:42:59.604 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8999999999999997e-07 2024-09-21T21:42:59.604 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.604 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.604 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:59.604 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824707+0000", 2024-09-21T21:42:59.604 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.7669999999999999e-06 2024-09-21T21:42:59.604 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.604 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.604 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:59.604 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824715+0000", 2024-09-21T21:42:59.604 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7700000000000001e-06 2024-09-21T21:42:59.605 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.605 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.605 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:59.605 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824721+0000", 2024-09-21T21:42:59.605 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9560000000000002e-06 2024-09-21T21:42:59.605 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.605 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.605 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:59.605 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896570+0000", 2024-09-21T21:42:59.605 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071848870999999995 2024-09-21T21:42:59.605 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.605 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.605 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:59.605 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896626+0000", 2024-09-21T21:42:59.606 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5578000000000002e-05 2024-09-21T21:42:59.606 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.606 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.606 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:59.606 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896719+0000", 2024-09-21T21:42:59.606 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.3313000000000006e-05 2024-09-21T21:42:59.606 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:59.606 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:59.606 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:59.606 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.606 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.606 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13566 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3162112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:59.607 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.824729+0000", 2024-09-21T21:42:59.607 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 68.739981888000003, 2024-09-21T21:42:59.607 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072356907999999998, 2024-09-21T21:42:59.607 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:59.607 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:59.607 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:59.607 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:59.607 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:59.607 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:59.607 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13566 2024-09-21T21:42:59.607 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.607 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:59.607 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.607 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:59.608 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824729+0000", 2024-09-21T21:42:59.608 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:59.608 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.608 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.608 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:59.608 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824729+0000", 2024-09-21T21:42:59.608 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:59.608 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.608 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.608 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:59.608 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825630+0000", 2024-09-21T21:42:59.608 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00090139400000000002 2024-09-21T21:42:59.608 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.608 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.609 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:59.609 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825634+0000", 2024-09-21T21:42:59.609 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4960000000000001e-06 2024-09-21T21:42:59.609 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.609 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.609 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:59.609 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825644+0000", 2024-09-21T21:42:59.609 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.9650000000000003e-06 2024-09-21T21:42:59.609 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.609 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.609 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:59.609 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825649+0000", 2024-09-21T21:42:59.609 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0359999999999997e-06 2024-09-21T21:42:59.609 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.610 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.610 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:59.610 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896757+0000", 2024-09-21T21:42:59.610 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071107717000000001 2024-09-21T21:42:59.610 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.610 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.610 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:59.610 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896878+0000", 2024-09-21T21:42:59.610 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000120834 2024-09-21T21:42:59.610 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.610 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.610 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:59.610 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897086+0000", 2024-09-21T21:42:59.611 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00020846600000000001 2024-09-21T21:42:59.611 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:59.611 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:59.611 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:59.611 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.611 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.611 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13568 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3907584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:59.611 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.825671+0000", 2024-09-21T21:42:59.611 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 68.739039478999999, 2024-09-21T21:42:59.611 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071794595000000003, 2024-09-21T21:42:59.611 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:59.611 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:59.611 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:59.611 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:59.611 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:59.612 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:59.612 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13568 2024-09-21T21:42:59.612 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.612 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:59.612 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.612 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:59.612 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825671+0000", 2024-09-21T21:42:59.612 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:59.612 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.612 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.612 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:59.612 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825671+0000", 2024-09-21T21:42:59.612 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:59.612 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.613 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.613 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:59.613 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825672+0000", 2024-09-21T21:42:59.613 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2999999999999999e-07 2024-09-21T21:42:59.613 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.613 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.613 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:59.613 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825673+0000", 2024-09-21T21:42:59.613 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.246e-06 2024-09-21T21:42:59.613 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.613 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.613 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:59.613 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825693+0000", 2024-09-21T21:42:59.613 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9789999999999999e-05 2024-09-21T21:42:59.613 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.614 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.614 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:59.614 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825697+0000", 2024-09-21T21:42:59.614 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2379999999999999e-06 2024-09-21T21:42:59.614 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.614 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.614 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:59.614 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897284+0000", 2024-09-21T21:42:59.614 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071586710999999997 2024-09-21T21:42:59.614 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.614 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.614 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:59.614 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897337+0000", 2024-09-21T21:42:59.615 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2800000000000003e-05 2024-09-21T21:42:59.615 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.615 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.615 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:59.615 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897466+0000", 2024-09-21T21:42:59.615 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00012957999999999999 2024-09-21T21:42:59.615 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:59.615 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:59.615 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:59.615 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.615 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.615 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13574 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 352256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:59.615 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.825702+0000", 2024-09-21T21:42:59.615 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 68.739009331000005, 2024-09-21T21:42:59.616 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072080377000000001, 2024-09-21T21:42:59.616 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:59.616 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:59.616 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:59.616 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:59.616 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:59.616 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:59.616 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13574 2024-09-21T21:42:59.616 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.616 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:59.616 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.617 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:59.617 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825702+0000", 2024-09-21T21:42:59.617 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:59.617 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.617 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.617 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:59.617 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825702+0000", 2024-09-21T21:42:59.617 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:59.617 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.617 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.617 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:59.617 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825943+0000", 2024-09-21T21:42:59.617 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00024105200000000001 2024-09-21T21:42:59.617 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.617 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.618 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:59.618 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825946+0000", 2024-09-21T21:42:59.618 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.743e-06 2024-09-21T21:42:59.618 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.618 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.618 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:59.618 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825956+0000", 2024-09-21T21:42:59.618 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6609999999999995e-06 2024-09-21T21:42:59.618 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.618 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.618 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:59.618 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825963+0000", 2024-09-21T21:42:59.618 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4609999999999996e-06 2024-09-21T21:42:59.618 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.619 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.619 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:59.619 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897502+0000", 2024-09-21T21:42:59.619 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071539354999999999 2024-09-21T21:42:59.619 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.619 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.619 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:59.619 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897619+0000", 2024-09-21T21:42:59.619 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011665 2024-09-21T21:42:59.619 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.619 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.619 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:59.619 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897782+0000", 2024-09-21T21:42:59.619 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00016345500000000001 2024-09-21T21:42:59.620 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:59.620 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:59.620 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:59.620 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.620 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.620 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13575 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 380928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:59.620 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.825973+0000", 2024-09-21T21:42:59.620 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 68.738737895, 2024-09-21T21:42:59.620 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072012723000000001, 2024-09-21T21:42:59.620 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:59.620 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:59.620 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:59.620 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:59.620 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:59.620 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:59.621 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13575 2024-09-21T21:42:59.621 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.621 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:59.621 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.621 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:59.621 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825973+0000", 2024-09-21T21:42:59.621 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:59.621 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.621 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.621 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:59.621 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825973+0000", 2024-09-21T21:42:59.621 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:59.621 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.621 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.622 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:59.622 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826746+0000", 2024-09-21T21:42:59.622 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00077326800000000002 2024-09-21T21:42:59.622 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.622 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.622 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:59.622 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826766+0000", 2024-09-21T21:42:59.622 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9465000000000001e-05 2024-09-21T21:42:59.622 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.622 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.622 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:59.622 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826812+0000", 2024-09-21T21:42:59.622 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5927999999999998e-05 2024-09-21T21:42:59.623 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.623 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.623 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:59.623 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826833+0000", 2024-09-21T21:42:59.623 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1710999999999999e-05 2024-09-21T21:42:59.623 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.623 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.623 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:59.623 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897809+0000", 2024-09-21T21:42:59.623 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.070975229000000001 2024-09-21T21:42:59.623 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.623 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.623 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:59.623 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897864+0000", 2024-09-21T21:42:59.623 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5334999999999998e-05 2024-09-21T21:42:59.624 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.624 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.624 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:59.624 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897986+0000", 2024-09-21T21:42:59.624 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000121787 2024-09-21T21:42:59.624 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:59.624 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:59.624 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:59.624 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.624 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.624 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13577 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 1814528~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:59.624 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.826874+0000", 2024-09-21T21:42:59.624 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 68.737836861000005, 2024-09-21T21:42:59.624 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071639123999999998, 2024-09-21T21:42:59.625 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:59.625 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:59.625 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:59.625 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:59.625 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:59.625 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:59.625 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13577 2024-09-21T21:42:59.625 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.625 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:59.625 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.625 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:59.625 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826874+0000", 2024-09-21T21:42:59.626 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:59.626 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.626 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.626 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:59.626 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826874+0000", 2024-09-21T21:42:59.626 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:59.626 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.626 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.626 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:59.626 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826875+0000", 2024-09-21T21:42:59.626 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.82e-07 2024-09-21T21:42:59.627 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.627 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.627 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:59.627 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826877+0000", 2024-09-21T21:42:59.627 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.295e-06 2024-09-21T21:42:59.627 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.627 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.627 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:59.627 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826883+0000", 2024-09-21T21:42:59.627 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2609999999999999e-06 2024-09-21T21:42:59.627 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.627 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.627 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:59.627 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826889+0000", 2024-09-21T21:42:59.627 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5589999999999996e-06 2024-09-21T21:42:59.628 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.628 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.628 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:59.628 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898301+0000", 2024-09-21T21:42:59.628 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071412188000000001 2024-09-21T21:42:59.628 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.628 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.628 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:59.628 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898416+0000", 2024-09-21T21:42:59.628 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000115538 2024-09-21T21:42:59.628 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.628 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.628 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:59.628 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898513+0000", 2024-09-21T21:42:59.629 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6800999999999997e-05 2024-09-21T21:42:59.629 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:59.629 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:59.629 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:59.629 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.629 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.629 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13578 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 2502656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:59.629 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.826896+0000", 2024-09-21T21:42:59.629 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 68.737815405000006, 2024-09-21T21:42:59.629 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071789411999999997, 2024-09-21T21:42:59.629 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:59.629 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:59.629 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:59.629 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:59.630 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:59.630 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:59.630 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13578 2024-09-21T21:42:59.630 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.630 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:59.630 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.630 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:59.630 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826896+0000", 2024-09-21T21:42:59.630 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:59.630 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.630 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.630 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:59.630 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826896+0000", 2024-09-21T21:42:59.631 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:59.631 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.631 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.631 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:59.631 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827327+0000", 2024-09-21T21:42:59.631 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00043094500000000002 2024-09-21T21:42:59.631 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.631 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.631 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:59.631 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827331+0000", 2024-09-21T21:42:59.631 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.0969999999999999e-06 2024-09-21T21:42:59.631 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.631 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.631 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:59.632 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827341+0000", 2024-09-21T21:42:59.632 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0199e-05 2024-09-21T21:42:59.632 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.632 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.632 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:59.632 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827346+0000", 2024-09-21T21:42:59.632 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3210000000000004e-06 2024-09-21T21:42:59.632 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.632 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.632 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:59.632 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898540+0000", 2024-09-21T21:42:59.632 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071194246000000003 2024-09-21T21:42:59.633 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.633 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.633 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:59.633 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898598+0000", 2024-09-21T21:42:59.633 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7658999999999998e-05 2024-09-21T21:42:59.633 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.634 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.634 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:59.634 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898685+0000", 2024-09-21T21:42:59.634 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.6945000000000002e-05 2024-09-21T21:42:59.634 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:59.634 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:59.634 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:59.634 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.634 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.634 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13579 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 2531328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:59.634 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.827355+0000", 2024-09-21T21:42:59.634 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 68.737355874000002, 2024-09-21T21:42:59.635 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071650933, 2024-09-21T21:42:59.635 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:59.635 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:59.635 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:59.635 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:59.635 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:59.635 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:59.635 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13579 2024-09-21T21:42:59.635 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.635 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:59.635 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.635 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:59.635 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827355+0000", 2024-09-21T21:42:59.635 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:59.636 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.636 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.636 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:59.636 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827355+0000", 2024-09-21T21:42:59.636 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:59.636 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.636 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.636 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:59.636 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827486+0000", 2024-09-21T21:42:59.636 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00013066799999999999 2024-09-21T21:42:59.636 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.636 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.636 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:59.637 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827490+0000", 2024-09-21T21:42:59.637 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.9009999999999996e-06 2024-09-21T21:42:59.637 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.637 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.637 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:59.637 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827499+0000", 2024-09-21T21:42:59.637 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6209999999999999e-06 2024-09-21T21:42:59.637 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.637 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.637 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:59.637 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827506+0000", 2024-09-21T21:42:59.637 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6490000000000002e-06 2024-09-21T21:42:59.638 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.638 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.638 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:59.638 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898713+0000", 2024-09-21T21:42:59.638 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071206937999999997 2024-09-21T21:42:59.638 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.638 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.638 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:59.638 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898811+0000", 2024-09-21T21:42:59.638 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.8159999999999995e-05 2024-09-21T21:42:59.638 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.638 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.638 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:59.638 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899006+0000", 2024-09-21T21:42:59.638 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00019499599999999999 2024-09-21T21:42:59.639 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:59.639 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:59.639 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:59.639 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.639 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.639 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13580 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3219456~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:59.639 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.827522+0000", 2024-09-21T21:42:59.639 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 68.737189443000005, 2024-09-21T21:42:59.639 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071813298999999997, 2024-09-21T21:42:59.639 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:59.639 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:59.639 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:59.640 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:59.640 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:59.640 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:59.640 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13580 2024-09-21T21:42:59.640 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.640 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:59.640 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.640 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:59.640 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827522+0000", 2024-09-21T21:42:59.640 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:59.640 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.640 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.640 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:59.640 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827522+0000", 2024-09-21T21:42:59.640 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:59.641 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.641 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.641 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:59.641 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827898+0000", 2024-09-21T21:42:59.641 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000376876 2024-09-21T21:42:59.641 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.641 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.641 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:59.641 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827903+0000", 2024-09-21T21:42:59.641 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4129999999999999e-06 2024-09-21T21:42:59.641 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.641 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.641 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:59.641 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827911+0000", 2024-09-21T21:42:59.642 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.4540000000000007e-06 2024-09-21T21:42:59.642 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.642 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.642 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:59.642 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827917+0000", 2024-09-21T21:42:59.642 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2110000000000003e-06 2024-09-21T21:42:59.642 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.642 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.642 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:59.642 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899041+0000", 2024-09-21T21:42:59.642 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071123965999999997 2024-09-21T21:42:59.642 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.642 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.642 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:59.642 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899157+0000", 2024-09-21T21:42:59.643 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011573700000000001 2024-09-21T21:42:59.643 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.643 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.643 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:59.643 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899335+0000", 2024-09-21T21:42:59.643 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00017764199999999999 2024-09-21T21:42:59.643 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:59.643 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:59.643 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:59.643 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.643 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.643 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13581 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3248128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:59.643 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.827929+0000", 2024-09-21T21:42:59.643 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 68.736782043999995, 2024-09-21T21:42:59.644 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071730151000000006, 2024-09-21T21:42:59.644 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:59.644 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:59.644 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:59.644 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:59.644 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:59.644 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:59.644 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13581 2024-09-21T21:42:59.644 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.644 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:59.644 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.644 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:59.644 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827929+0000", 2024-09-21T21:42:59.644 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:59.644 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.645 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.645 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:59.645 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827929+0000", 2024-09-21T21:42:59.645 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:59.645 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.645 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.645 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:59.645 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.829943+0000", 2024-09-21T21:42:59.645 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0020143750000000001 2024-09-21T21:42:59.645 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.645 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.645 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:59.645 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.829948+0000", 2024-09-21T21:42:59.645 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9400000000000001e-06 2024-09-21T21:42:59.646 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.646 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.646 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:59.646 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.829960+0000", 2024-09-21T21:42:59.646 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.2123999999999999e-05 2024-09-21T21:42:59.646 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.646 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.646 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:59.646 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.829968+0000", 2024-09-21T21:42:59.646 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7000000000000008e-06 2024-09-21T21:42:59.646 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.646 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.646 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:59.646 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899370+0000", 2024-09-21T21:42:59.646 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.069401672999999997 2024-09-21T21:42:59.647 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.647 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.647 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:59.647 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899481+0000", 2024-09-21T21:42:59.647 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011145300000000001 2024-09-21T21:42:59.647 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.647 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.647 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:59.647 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899659+0000", 2024-09-21T21:42:59.647 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00017788600000000001 2024-09-21T21:42:59.647 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:59.647 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:59.647 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:59.647 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.648 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.648 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13595 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 1871872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:59.648 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.830180+0000", 2024-09-21T21:42:59.648 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 68.734530930000005, 2024-09-21T21:42:59.648 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071809244999999994, 2024-09-21T21:42:59.648 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:59.648 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:59.648 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:59.648 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:59.648 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:59.648 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:59.648 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13595 2024-09-21T21:42:59.648 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.648 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:59.648 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.649 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:59.649 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.830180+0000", 2024-09-21T21:42:59.649 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:59.649 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.649 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.649 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:59.649 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.830180+0000", 2024-09-21T21:42:59.649 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:59.649 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.649 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.649 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:59.649 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.831967+0000", 2024-09-21T21:42:59.649 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0017868350000000001 2024-09-21T21:42:59.649 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.650 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.650 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:59.650 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.831971+0000", 2024-09-21T21:42:59.650 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3950000000000003e-06 2024-09-21T21:42:59.650 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.650 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.650 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:59.650 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.831983+0000", 2024-09-21T21:42:59.650 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1815e-05 2024-09-21T21:42:59.650 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.650 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.651 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:59.651 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.831993+0000", 2024-09-21T21:42:59.651 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6339999999999996e-06 2024-09-21T21:42:59.651 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.651 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.651 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:59.651 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.901693+0000", 2024-09-21T21:42:59.651 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.069700082999999996 2024-09-21T21:42:59.651 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.651 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.651 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:59.651 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.901813+0000", 2024-09-21T21:42:59.651 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011971 2024-09-21T21:42:59.651 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.652 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.652 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:59.652 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.901989+0000", 2024-09-21T21:42:59.652 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000176773 2024-09-21T21:42:59.652 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:59.652 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:59.652 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:59.652 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.652 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.652 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13601 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3334144~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:42:59.652 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.832130+0000", 2024-09-21T21:42:59.652 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 68.732581057999994, 2024-09-21T21:42:59.652 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071861727, 2024-09-21T21:42:59.652 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:59.652 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:59.653 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:59.653 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:59.653 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:42:59.653 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:42:59.653 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13601 2024-09-21T21:42:59.653 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.653 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:59.653 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.653 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:59.653 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.832130+0000", 2024-09-21T21:42:59.653 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:59.653 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.653 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.653 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:59.654 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.832130+0000", 2024-09-21T21:42:59.654 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:59.654 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.654 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.654 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:59.654 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.833939+0000", 2024-09-21T21:42:59.654 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0018094400000000001 2024-09-21T21:42:59.654 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.654 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.654 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:59.654 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.833944+0000", 2024-09-21T21:42:59.654 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5979999999999999e-06 2024-09-21T21:42:59.654 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.654 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.655 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:59.655 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.833953+0000", 2024-09-21T21:42:59.655 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.1460000000000002e-06 2024-09-21T21:42:59.655 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.655 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.655 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:59.655 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.833960+0000", 2024-09-21T21:42:59.655 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.7989999999999996e-06 2024-09-21T21:42:59.655 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.655 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.655 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:59.655 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.903693+0000", 2024-09-21T21:42:59.655 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.069732901 2024-09-21T21:42:59.655 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.656 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.656 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:59.656 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.903814+0000", 2024-09-21T21:42:59.656 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000121585 2024-09-21T21:42:59.656 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.656 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.656 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:59.656 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.903992+0000", 2024-09-21T21:42:59.656 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00017725799999999999 2024-09-21T21:42:59.656 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:59.656 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:59.656 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:59.656 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.656 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.656 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:12342 3.1 3:87417293:::rbd_data.1121faebdfe3.0000000000000004:head [read 4096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:42:59.657 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:14.411725+0000", 2024-09-21T21:42:59.657 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 45.152985991999998, 2024-09-21T21:42:59.657 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0089213029999999, 2024-09-21T21:42:59.657 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:59.657 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:59.657 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:59.657 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:59.657 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:59.657 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:59.657 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 12342 2024-09-21T21:42:59.657 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.657 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:59.657 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.657 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:59.658 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411725+0000", 2024-09-21T21:42:59.658 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:59.658 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.658 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.658 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:59.658 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411725+0000", 2024-09-21T21:42:59.658 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:59.658 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.658 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.658 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:59.658 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411726+0000", 2024-09-21T21:42:59.658 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.4169999999999999e-06 2024-09-21T21:42:59.658 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.658 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.659 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:59.659 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411734+0000", 2024-09-21T21:42:59.659 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.3799999999999996e-06 2024-09-21T21:42:59.659 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.659 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.659 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:59.659 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411750+0000", 2024-09-21T21:42:59.659 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.6429000000000001e-05 2024-09-21T21:42:59.659 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.659 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.659 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:59.659 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411760+0000", 2024-09-21T21:42:59.659 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.4239999999999999e-06 2024-09-21T21:42:59.659 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.660 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.660 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:59.660 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411828+0000", 2024-09-21T21:42:59.660 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.7935999999999995e-05 2024-09-21T21:42:59.660 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.660 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.660 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op must wait for map", 2024-09-21T21:42:59.660 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411837+0000", 2024-09-21T21:42:59.660 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.0100000000000001e-06 2024-09-21T21:42:59.660 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.660 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.660 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:59.660 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.414984+0000", 2024-09-21T21:42:59.660 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0031472050000000001 2024-09-21T21:42:59.660 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.661 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.661 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for peered", 2024-09-21T21:42:59.661 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.414994+0000", 2024-09-21T21:42:59.661 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0291e-05 2024-09-21T21:42:59.661 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.661 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.661 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:59.661 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.420353+0000", 2024-09-21T21:42:59.661 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.005358913 2024-09-21T21:42:59.661 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.661 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.661 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:59.661 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.420489+0000", 2024-09-21T21:42:59.661 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00013592699999999999 2024-09-21T21:42:59.662 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.662 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.662 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:59.662 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.420646+0000", 2024-09-21T21:42:59.662 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00015737100000000001 2024-09-21T21:42:59.662 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:59.662 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:59.662 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:59.662 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.662 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.662 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:12348 3.1 3:87417293:::rbd_data.1121faebdfe3.0000000000000004:head [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:42:59.662 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:14.411793+0000", 2024-09-21T21:42:59.662 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 45.152918446000001, 2024-09-21T21:42:59.662 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.009298384, 2024-09-21T21:42:59.663 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:59.663 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:59.663 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:59.663 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:59.663 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:42:59.663 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:42:59.663 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 12348 2024-09-21T21:42:59.663 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.663 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:59.663 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.663 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:59.663 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411793+0000", 2024-09-21T21:42:59.663 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:59.663 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.664 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.664 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:59.664 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411793+0000", 2024-09-21T21:42:59.664 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:59.664 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.664 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.664 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:59.664 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411793+0000", 2024-09-21T21:42:59.664 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0999999999999998e-07 2024-09-21T21:42:59.664 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.664 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.664 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:59.664 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411797+0000", 2024-09-21T21:42:59.665 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.49e-06 2024-09-21T21:42:59.665 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.665 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.665 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:59.665 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411809+0000", 2024-09-21T21:42:59.665 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.2510999999999999e-05 2024-09-21T21:42:59.665 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.665 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.665 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:59.665 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411816+0000", 2024-09-21T21:42:59.665 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6420000000000001e-06 2024-09-21T21:42:59.665 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.665 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.666 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:59.666 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411963+0000", 2024-09-21T21:42:59.666 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00014700999999999999 2024-09-21T21:42:59.666 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.666 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.666 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting_for_map not empty", 2024-09-21T21:42:59.666 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411970+0000", 2024-09-21T21:42:59.666 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9800000000000001e-06 2024-09-21T21:42:59.666 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.666 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.666 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:59.666 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.415018+0000", 2024-09-21T21:42:59.666 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0030482299999999999 2024-09-21T21:42:59.667 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.667 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.667 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for peered", 2024-09-21T21:42:59.667 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.415025+0000", 2024-09-21T21:42:59.667 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.3749999999999997e-06 2024-09-21T21:42:59.667 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.667 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.667 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:59.667 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.420904+0000", 2024-09-21T21:42:59.667 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0058787339999999 2024-09-21T21:42:59.667 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.667 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.667 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:59.667 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.420988+0000", 2024-09-21T21:42:59.668 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.3955999999999994e-05 2024-09-21T21:42:59.668 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.668 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.668 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:59.668 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.421091+0000", 2024-09-21T21:42:59.668 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000102846 2024-09-21T21:42:59.668 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:59.668 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:59.668 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:59.668 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.668 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.668 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4385.0:20922 3.9 3:920c5256:::rbd_id.image_client.0-clone36:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-21T21:42:59.669 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:17.811324+0000", 2024-09-21T21:42:59.669 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 41.753386714000001, 2024-09-21T21:42:59.669 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.66951882299999999, 2024-09-21T21:42:59.669 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:42:59.669 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:42:59.669 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:42:59.669 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:42:59.669 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4385", 2024-09-21T21:42:59.669 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:42:59.669 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 20922 2024-09-21T21:42:59.669 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.669 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:42:59.669 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.670 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:42:59.670 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:17.811324+0000", 2024-09-21T21:42:59.670 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:59.670 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.670 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.670 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:42:59.670 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:17.811324+0000", 2024-09-21T21:42:59.670 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:42:59.670 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.670 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.670 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:42:59.670 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:17.811326+0000", 2024-09-21T21:42:59.670 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.536e-06 2024-09-21T21:42:59.670 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.671 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.671 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:42:59.671 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:17.811332+0000", 2024-09-21T21:42:59.671 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6699999999999997e-06 2024-09-21T21:42:59.671 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.671 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.671 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:42:59.671 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:17.811347+0000", 2024-09-21T21:42:59.671 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.4589999999999999e-05 2024-09-21T21:42:59.671 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.671 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.671 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:42:59.671 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:17.811359+0000", 2024-09-21T21:42:59.671 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1776e-05 2024-09-21T21:42:59.671 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.672 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.672 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:59.672 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:17.811424+0000", 2024-09-21T21:42:59.672 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4795000000000006e-05 2024-09-21T21:42:59.672 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.672 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.672 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for peered", 2024-09-21T21:42:59.672 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:17.811436+0000", 2024-09-21T21:42:59.672 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.221e-05 2024-09-21T21:42:59.672 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.672 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.672 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:42:59.672 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:18.480657+0000", 2024-09-21T21:42:59.672 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.66922088999999996 2024-09-21T21:42:59.672 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.673 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.673 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:42:59.673 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:18.480760+0000", 2024-09-21T21:42:59.673 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000102827 2024-09-21T21:42:59.673 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:42:59.673 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:42:59.673 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:42:59.673 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:18.480843+0000", 2024-09-21T21:42:59.673 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.3529000000000006e-05 2024-09-21T21:42:59.673 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:59.673 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:59.673 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:59.673 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:42:59.673 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:42:59.673 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:42:59.675 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T21:42:59.675 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:42:59.673+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:42:59.776 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:42:59.774+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:42:59.778 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:59.778 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:42:59.778 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-21T21:42:59.778 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:59.788 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T21:42:59.877 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:42:59.875+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:42:59.930 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:42:59.931 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:42:59.931 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:42:59.931 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:42:59.931 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:42:59.939 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T21:42:59.981 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:42:59.979+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:43:00.079 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:43:00.077+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:43:00.084 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:43:00.084 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-21T21:43:00.084 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-21T21:43:00.084 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:43:00.085 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.085 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19747 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 7905280~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:00.085 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273058+0000", 2024-09-21T21:43:00.085 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 68.809001562999995, 2024-09-21T21:43:00.085 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97338591399999996, 2024-09-21T21:43:00.085 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:00.085 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:00.085 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:00.085 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:00.085 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:00.085 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:00.085 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19747 2024-09-21T21:43:00.085 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.086 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:00.086 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.086 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:00.086 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273058+0000", 2024-09-21T21:43:00.086 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:00.086 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.086 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.086 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:00.086 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273058+0000", 2024-09-21T21:43:00.086 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:00.086 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.086 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.086 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:00.086 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273058+0000", 2024-09-21T21:43:00.087 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5900000000000003e-07 2024-09-21T21:43:00.087 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.087 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.087 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:00.087 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273061+0000", 2024-09-21T21:43:00.087 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3479999999999998e-06 2024-09-21T21:43:00.087 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.087 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.087 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:00.087 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273067+0000", 2024-09-21T21:43:00.087 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1469999999999998e-06 2024-09-21T21:43:00.087 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.087 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.087 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:00.088 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273071+0000", 2024-09-21T21:43:00.088 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7339999999999999e-06 2024-09-21T21:43:00.088 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.088 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.088 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:00.088 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277945+0000", 2024-09-21T21:43:00.088 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.004873474 2024-09-21T21:43:00.088 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.088 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.088 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:00.088 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277948+0000", 2024-09-21T21:43:00.088 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0299999999999998e-06 2024-09-21T21:43:00.088 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.088 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.089 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:00.089 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246269+0000", 2024-09-21T21:43:00.089 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96832060200000003 2024-09-21T21:43:00.089 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.089 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.089 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:00.089 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246362+0000", 2024-09-21T21:43:00.089 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3867000000000001e-05 2024-09-21T21:43:00.089 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.089 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.089 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:00.089 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246444+0000", 2024-09-21T21:43:00.089 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1353000000000005e-05 2024-09-21T21:43:00.090 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:00.090 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:00.090 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:00.090 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.090 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.090 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19755 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 5783552~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:00.090 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273077+0000", 2024-09-21T21:43:00.090 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 68.808981947000007, 2024-09-21T21:43:00.090 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97350203400000002, 2024-09-21T21:43:00.090 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:00.090 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:00.090 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:00.090 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:00.090 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:00.091 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:00.091 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19755 2024-09-21T21:43:00.091 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.091 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:00.091 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.091 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:00.091 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273077+0000", 2024-09-21T21:43:00.091 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:00.091 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.091 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.091 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:00.091 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273077+0000", 2024-09-21T21:43:00.091 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:00.092 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.092 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.092 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:00.092 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273078+0000", 2024-09-21T21:43:00.092 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5699999999999998e-07 2024-09-21T21:43:00.092 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.092 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.092 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:00.092 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273083+0000", 2024-09-21T21:43:00.092 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8670000000000002e-06 2024-09-21T21:43:00.092 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.092 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.092 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:00.092 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273088+0000", 2024-09-21T21:43:00.093 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8370000000000002e-06 2024-09-21T21:43:00.093 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.093 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.093 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:00.093 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273093+0000", 2024-09-21T21:43:00.093 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5709999999999999e-06 2024-09-21T21:43:00.093 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.093 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.093 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:00.093 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277968+0000", 2024-09-21T21:43:00.093 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048753909999999998 2024-09-21T21:43:00.093 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.093 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.093 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:00.094 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277971+0000", 2024-09-21T21:43:00.094 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0189999999999998e-06 2024-09-21T21:43:00.094 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.094 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.094 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:00.094 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246485+0000", 2024-09-21T21:43:00.094 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96851399299999996 2024-09-21T21:43:00.094 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.094 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.094 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:00.094 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246536+0000", 2024-09-21T21:43:00.094 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0430000000000003e-05 2024-09-21T21:43:00.094 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.095 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.095 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:00.095 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246579+0000", 2024-09-21T21:43:00.095 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3569000000000002e-05 2024-09-21T21:43:00.095 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:00.095 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:00.095 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:00.095 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.095 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.095 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19756 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6500352~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:00.095 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273099+0000", 2024-09-21T21:43:00.095 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 68.808960573999997, 2024-09-21T21:43:00.095 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97359272799999996, 2024-09-21T21:43:00.095 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:00.095 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:00.096 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:00.096 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:00.096 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:00.096 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:00.096 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19756 2024-09-21T21:43:00.096 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.096 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:00.096 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.096 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:00.096 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273099+0000", 2024-09-21T21:43:00.096 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:00.096 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.096 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.097 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:00.097 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273099+0000", 2024-09-21T21:43:00.097 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:00.097 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.097 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.097 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:00.097 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273099+0000", 2024-09-21T21:43:00.097 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0900000000000002e-07 2024-09-21T21:43:00.097 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.097 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.097 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:00.097 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273101+0000", 2024-09-21T21:43:00.097 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0320000000000002e-06 2024-09-21T21:43:00.097 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.098 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.098 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:00.098 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273107+0000", 2024-09-21T21:43:00.098 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2830000000000001e-06 2024-09-21T21:43:00.098 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.098 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.098 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:00.098 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273111+0000", 2024-09-21T21:43:00.098 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2230000000000001e-06 2024-09-21T21:43:00.098 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.098 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.098 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:00.098 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277994+0000", 2024-09-21T21:43:00.098 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048837070000000002 2024-09-21T21:43:00.099 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.099 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.099 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:00.099 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278000+0000", 2024-09-21T21:43:00.099 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3149999999999999e-06 2024-09-21T21:43:00.099 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.099 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.099 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:00.099 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246607+0000", 2024-09-21T21:43:00.099 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.968606839 2024-09-21T21:43:00.099 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.099 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.099 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:00.099 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246651+0000", 2024-09-21T21:43:00.100 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4734999999999998e-05 2024-09-21T21:43:00.100 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.100 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.100 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:00.100 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246692+0000", 2024-09-21T21:43:00.100 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0185000000000002e-05 2024-09-21T21:43:00.100 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:00.100 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:00.100 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:00.100 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.100 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.100 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19757 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 5812224~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:00.100 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273116+0000", 2024-09-21T21:43:00.101 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 68.808942916999996, 2024-09-21T21:43:00.101 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97367813999999997, 2024-09-21T21:43:00.101 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:00.101 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:00.101 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:00.101 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:00.101 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:00.101 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:00.101 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19757 2024-09-21T21:43:00.101 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.101 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:00.101 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.101 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:00.101 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273116+0000", 2024-09-21T21:43:00.101 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:00.102 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.102 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.102 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:00.102 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273116+0000", 2024-09-21T21:43:00.102 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:00.102 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.102 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.102 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:00.102 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273117+0000", 2024-09-21T21:43:00.102 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0999999999999999e-07 2024-09-21T21:43:00.102 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.103 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.103 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:00.103 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273119+0000", 2024-09-21T21:43:00.103 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9970000000000001e-06 2024-09-21T21:43:00.103 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.103 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.103 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:00.103 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273125+0000", 2024-09-21T21:43:00.103 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9540000000000001e-06 2024-09-21T21:43:00.103 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.103 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.103 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:00.104 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273135+0000", 2024-09-21T21:43:00.104 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0597999999999999e-05 2024-09-21T21:43:00.104 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.104 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.104 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:00.104 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278019+0000", 2024-09-21T21:43:00.104 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048834849999999999 2024-09-21T21:43:00.104 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.104 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.104 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:00.104 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278022+0000", 2024-09-21T21:43:00.104 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9560000000000002e-06 2024-09-21T21:43:00.104 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.104 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.105 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:00.105 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246716+0000", 2024-09-21T21:43:00.105 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.968694202 2024-09-21T21:43:00.105 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.105 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.105 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:00.105 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246758+0000", 2024-09-21T21:43:00.105 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1607000000000003e-05 2024-09-21T21:43:00.105 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.105 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.105 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:00.105 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246795+0000", 2024-09-21T21:43:00.105 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6931000000000003e-05 2024-09-21T21:43:00.106 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:00.106 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:00.106 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:00.106 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.106 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.106 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19758 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6529024~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:00.106 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273142+0000", 2024-09-21T21:43:00.106 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 68.808917695999995, 2024-09-21T21:43:00.106 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97375924599999997, 2024-09-21T21:43:00.106 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:00.106 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:00.106 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:00.106 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:00.107 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:00.107 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:00.107 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19758 2024-09-21T21:43:00.107 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.107 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:00.107 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.107 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:00.107 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273142+0000", 2024-09-21T21:43:00.107 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:00.107 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.107 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.107 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:00.107 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273142+0000", 2024-09-21T21:43:00.107 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:00.107 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.108 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.108 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:00.108 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273142+0000", 2024-09-21T21:43:00.108 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0200000000000003e-07 2024-09-21T21:43:00.108 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.108 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.108 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:00.108 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273144+0000", 2024-09-21T21:43:00.108 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1390000000000002e-06 2024-09-21T21:43:00.108 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.108 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.108 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:00.108 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273150+0000", 2024-09-21T21:43:00.108 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6409999999999999e-06 2024-09-21T21:43:00.109 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.109 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.109 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:00.109 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273157+0000", 2024-09-21T21:43:00.109 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1359999999999999e-06 2024-09-21T21:43:00.109 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.109 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.109 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:00.109 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278050+0000", 2024-09-21T21:43:00.109 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048926170000000001 2024-09-21T21:43:00.109 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.109 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.109 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:00.109 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278053+0000", 2024-09-21T21:43:00.109 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4879999999999999e-06 2024-09-21T21:43:00.110 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.110 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.110 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:00.110 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246819+0000", 2024-09-21T21:43:00.110 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96876590399999996 2024-09-21T21:43:00.110 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.110 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.110 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:00.110 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246856+0000", 2024-09-21T21:43:00.110 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7287999999999999e-05 2024-09-21T21:43:00.110 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.110 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.110 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:00.110 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246901+0000", 2024-09-21T21:43:00.111 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4631e-05 2024-09-21T21:43:00.111 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:00.111 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:00.111 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:00.111 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.111 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.111 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19759 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 7933952~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:00.111 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273172+0000", 2024-09-21T21:43:00.111 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 68.808887432000006, 2024-09-21T21:43:00.111 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97384789500000002, 2024-09-21T21:43:00.111 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:00.111 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:00.111 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:00.111 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:00.111 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:00.112 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:00.112 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19759 2024-09-21T21:43:00.112 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.112 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:00.112 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.112 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:00.112 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273172+0000", 2024-09-21T21:43:00.112 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:00.112 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.112 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.113 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:00.113 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273172+0000", 2024-09-21T21:43:00.113 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:00.113 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.113 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.113 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:00.113 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273172+0000", 2024-09-21T21:43:00.114 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6100000000000002e-07 2024-09-21T21:43:00.114 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.114 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.114 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:00.114 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273174+0000", 2024-09-21T21:43:00.114 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.691e-06 2024-09-21T21:43:00.114 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.115 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.115 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:00.115 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273178+0000", 2024-09-21T21:43:00.115 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7579999999999998e-06 2024-09-21T21:43:00.115 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.115 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.115 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:00.115 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273180+0000", 2024-09-21T21:43:00.116 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7199999999999998e-06 2024-09-21T21:43:00.116 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.116 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.116 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:00.116 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278077+0000", 2024-09-21T21:43:00.116 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048969529999999999 2024-09-21T21:43:00.116 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.116 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.116 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:00.117 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278080+0000", 2024-09-21T21:43:00.117 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.066e-06 2024-09-21T21:43:00.117 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.117 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.117 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:00.117 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246925+0000", 2024-09-21T21:43:00.117 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96884490199999995 2024-09-21T21:43:00.118 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.118 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.118 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:00.118 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246974+0000", 2024-09-21T21:43:00.118 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8387999999999998e-05 2024-09-21T21:43:00.118 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.118 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.119 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:00.119 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247020+0000", 2024-09-21T21:43:00.119 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6156000000000003e-05 2024-09-21T21:43:00.119 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:00.119 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:00.119 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:00.119 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.120 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.120 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19760 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 7991296~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:00.120 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273220+0000", 2024-09-21T21:43:00.120 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 68.808838918000006, 2024-09-21T21:43:00.120 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97403125499999998, 2024-09-21T21:43:00.120 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:00.120 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:00.120 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:00.120 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:00.120 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:00.120 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:00.121 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19760 2024-09-21T21:43:00.121 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.121 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:00.121 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.121 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:00.121 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273220+0000", 2024-09-21T21:43:00.121 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:00.121 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.121 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.121 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:00.121 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273220+0000", 2024-09-21T21:43:00.122 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:00.122 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.122 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.122 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:00.122 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273221+0000", 2024-09-21T21:43:00.122 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.65e-07 2024-09-21T21:43:00.122 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.122 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.122 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:00.122 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273228+0000", 2024-09-21T21:43:00.122 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6040000000000004e-06 2024-09-21T21:43:00.122 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.122 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.123 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:00.123 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273233+0000", 2024-09-21T21:43:00.123 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1569999999999999e-06 2024-09-21T21:43:00.123 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.123 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.123 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:00.123 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273238+0000", 2024-09-21T21:43:00.123 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0050000000000004e-06 2024-09-21T21:43:00.123 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.123 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.123 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:00.123 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278107+0000", 2024-09-21T21:43:00.123 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00486857 2024-09-21T21:43:00.123 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.124 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.124 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:00.124 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278111+0000", 2024-09-21T21:43:00.124 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6389999999999998e-06 2024-09-21T21:43:00.124 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.124 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.124 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:00.124 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247069+0000", 2024-09-21T21:43:00.124 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96895796899999997 2024-09-21T21:43:00.124 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.124 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.125 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:00.125 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247166+0000", 2024-09-21T21:43:00.125 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.7158999999999996e-05 2024-09-21T21:43:00.125 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.125 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.125 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:00.125 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247252+0000", 2024-09-21T21:43:00.125 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5886999999999999e-05 2024-09-21T21:43:00.125 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:00.125 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:00.125 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:00.125 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.125 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.126 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19761 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 7962624~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:00.126 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273246+0000", 2024-09-21T21:43:00.126 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 68.808813857999994, 2024-09-21T21:43:00.126 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97415972200000001, 2024-09-21T21:43:00.126 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:00.126 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:00.126 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:00.126 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:00.126 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:00.126 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:00.126 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19761 2024-09-21T21:43:00.126 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.126 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:00.127 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.127 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:00.127 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273246+0000", 2024-09-21T21:43:00.127 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:00.127 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.127 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.127 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:00.127 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273246+0000", 2024-09-21T21:43:00.127 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:00.127 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.127 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.127 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:00.127 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273246+0000", 2024-09-21T21:43:00.127 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5100000000000001e-07 2024-09-21T21:43:00.128 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.128 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.128 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:00.128 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273248+0000", 2024-09-21T21:43:00.128 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1550000000000001e-06 2024-09-21T21:43:00.128 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.128 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.128 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:00.128 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273254+0000", 2024-09-21T21:43:00.128 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.964e-06 2024-09-21T21:43:00.128 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.128 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.128 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:00.128 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273259+0000", 2024-09-21T21:43:00.129 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5499999999999996e-06 2024-09-21T21:43:00.129 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.129 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.129 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:00.129 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278130+0000", 2024-09-21T21:43:00.129 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048715779999999997 2024-09-21T21:43:00.129 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.129 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.129 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:00.129 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278133+0000", 2024-09-21T21:43:00.129 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8739999999999999e-06 2024-09-21T21:43:00.129 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.129 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.129 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:00.130 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247270+0000", 2024-09-21T21:43:00.130 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.969137099 2024-09-21T21:43:00.130 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.130 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.130 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:00.130 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247338+0000", 2024-09-21T21:43:00.130 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7717000000000002e-05 2024-09-21T21:43:00.130 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.130 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.130 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:00.130 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247405+0000", 2024-09-21T21:43:00.130 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7434000000000005e-05 2024-09-21T21:43:00.130 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:00.131 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:00.131 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:00.131 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.131 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.131 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19769 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 5840896~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:00.131 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273287+0000", 2024-09-21T21:43:00.131 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 68.808772574000002, 2024-09-21T21:43:00.131 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97435390200000005, 2024-09-21T21:43:00.131 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:00.131 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:00.131 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:00.131 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:00.131 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:00.131 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:00.131 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19769 2024-09-21T21:43:00.132 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.132 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:00.132 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.132 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:00.132 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273287+0000", 2024-09-21T21:43:00.132 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:00.132 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.132 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.132 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:00.132 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273287+0000", 2024-09-21T21:43:00.132 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:00.132 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.132 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.132 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:00.133 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273287+0000", 2024-09-21T21:43:00.133 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6699999999999999e-07 2024-09-21T21:43:00.133 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.133 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.133 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:00.133 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273289+0000", 2024-09-21T21:43:00.133 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.708e-06 2024-09-21T21:43:00.133 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.133 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.133 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:00.133 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273296+0000", 2024-09-21T21:43:00.133 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8109999999999997e-06 2024-09-21T21:43:00.133 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.133 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.134 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:00.134 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273299+0000", 2024-09-21T21:43:00.134 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0740000000000001e-06 2024-09-21T21:43:00.134 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.134 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.134 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:00.134 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278159+0000", 2024-09-21T21:43:00.134 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048603240000000001 2024-09-21T21:43:00.134 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.135 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.135 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:00.135 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278162+0000", 2024-09-21T21:43:00.135 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1930000000000002e-06 2024-09-21T21:43:00.135 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.135 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.135 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:00.135 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247444+0000", 2024-09-21T21:43:00.135 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96928199100000001 2024-09-21T21:43:00.135 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.136 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.136 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:00.136 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247533+0000", 2024-09-21T21:43:00.136 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8490000000000001e-05 2024-09-21T21:43:00.136 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.136 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.136 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:00.136 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247641+0000", 2024-09-21T21:43:00.136 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000107944 2024-09-21T21:43:00.136 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:00.136 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:00.136 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:00.136 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.137 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.137 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19770 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 5869568~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:00.137 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273313+0000", 2024-09-21T21:43:00.137 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 68.808746471999996, 2024-09-21T21:43:00.137 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97449038899999996, 2024-09-21T21:43:00.137 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:00.137 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:00.137 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:00.137 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:00.137 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:00.137 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:00.137 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19770 2024-09-21T21:43:00.137 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.137 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:00.138 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.138 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:00.138 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273313+0000", 2024-09-21T21:43:00.138 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:00.138 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.138 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.138 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:00.138 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273313+0000", 2024-09-21T21:43:00.138 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:00.138 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.138 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.138 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:00.138 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273313+0000", 2024-09-21T21:43:00.139 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5699999999999998e-07 2024-09-21T21:43:00.139 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.139 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.139 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:00.139 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273315+0000", 2024-09-21T21:43:00.139 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0099999999999998e-06 2024-09-21T21:43:00.139 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.139 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.139 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:00.139 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273321+0000", 2024-09-21T21:43:00.139 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7529999999999998e-06 2024-09-21T21:43:00.139 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.140 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.140 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:00.140 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273328+0000", 2024-09-21T21:43:00.140 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9009999999999997e-06 2024-09-21T21:43:00.140 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.140 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.140 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:00.140 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278183+0000", 2024-09-21T21:43:00.140 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048547770000000002 2024-09-21T21:43:00.140 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.140 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.140 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:00.140 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278186+0000", 2024-09-21T21:43:00.140 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3100000000000001e-06 2024-09-21T21:43:00.141 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.141 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.141 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:00.141 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247676+0000", 2024-09-21T21:43:00.141 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96949004299999997 2024-09-21T21:43:00.141 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.141 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.141 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:00.141 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247757+0000", 2024-09-21T21:43:00.141 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0479e-05 2024-09-21T21:43:00.141 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.141 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.141 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:00.142 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247803+0000", 2024-09-21T21:43:00.142 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6758999999999997e-05 2024-09-21T21:43:00.142 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:00.142 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:00.142 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:00.142 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.142 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.142 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19771 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6586368~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:00.142 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273335+0000", 2024-09-21T21:43:00.142 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 68.808724685000001, 2024-09-21T21:43:00.142 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97458947699999998, 2024-09-21T21:43:00.142 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:00.142 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:00.142 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:00.143 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:00.143 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:00.143 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:00.143 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19771 2024-09-21T21:43:00.143 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.143 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:00.143 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.143 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:00.143 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273335+0000", 2024-09-21T21:43:00.143 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:00.143 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.143 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.143 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:00.143 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273335+0000", 2024-09-21T21:43:00.144 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:00.144 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.144 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.144 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:00.144 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273335+0000", 2024-09-21T21:43:00.144 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3599999999999999e-07 2024-09-21T21:43:00.144 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.144 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.144 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:00.144 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273337+0000", 2024-09-21T21:43:00.144 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0789999999999999e-06 2024-09-21T21:43:00.144 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.144 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.144 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:00.144 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273343+0000", 2024-09-21T21:43:00.145 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6280000000000002e-06 2024-09-21T21:43:00.145 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.145 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.145 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:00.145 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273346+0000", 2024-09-21T21:43:00.145 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.507e-06 2024-09-21T21:43:00.145 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.145 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.145 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:00.145 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278206+0000", 2024-09-21T21:43:00.145 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048600969999999999 2024-09-21T21:43:00.145 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.146 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.146 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:00.146 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278212+0000", 2024-09-21T21:43:00.146 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2079999999999999e-06 2024-09-21T21:43:00.146 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.146 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.146 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:00.146 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247828+0000", 2024-09-21T21:43:00.146 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.969616547 2024-09-21T21:43:00.146 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.146 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.146 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:00.146 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247881+0000", 2024-09-21T21:43:00.147 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3180999999999998e-05 2024-09-21T21:43:00.147 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.147 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.147 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:00.147 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247924+0000", 2024-09-21T21:43:00.147 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2894e-05 2024-09-21T21:43:00.147 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:00.147 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:00.147 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:00.147 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.147 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.147 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19772 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6557696~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:00.148 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273358+0000", 2024-09-21T21:43:00.148 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 68.808701305, 2024-09-21T21:43:00.148 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97467031800000004, 2024-09-21T21:43:00.148 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:00.148 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:00.148 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:00.148 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:00.148 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:00.148 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:00.148 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19772 2024-09-21T21:43:00.148 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.148 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:00.148 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.148 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:00.149 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273358+0000", 2024-09-21T21:43:00.149 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:00.149 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.149 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.149 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:00.149 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273358+0000", 2024-09-21T21:43:00.149 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:00.149 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.149 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.149 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:00.149 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273358+0000", 2024-09-21T21:43:00.149 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4499999999999998e-07 2024-09-21T21:43:00.149 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.149 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.150 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:00.150 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273360+0000", 2024-09-21T21:43:00.150 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9190000000000002e-06 2024-09-21T21:43:00.150 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.150 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.150 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:00.150 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273366+0000", 2024-09-21T21:43:00.150 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.891e-06 2024-09-21T21:43:00.150 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.150 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.150 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:00.150 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273371+0000", 2024-09-21T21:43:00.150 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1880000000000001e-06 2024-09-21T21:43:00.151 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.151 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.151 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:00.151 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278237+0000", 2024-09-21T21:43:00.151 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.004865472 2024-09-21T21:43:00.151 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.151 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.151 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:00.151 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278240+0000", 2024-09-21T21:43:00.151 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9550000000000001e-06 2024-09-21T21:43:00.151 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.151 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.151 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:00.151 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247949+0000", 2024-09-21T21:43:00.151 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.969709127 2024-09-21T21:43:00.152 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.152 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.152 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:00.152 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247990+0000", 2024-09-21T21:43:00.152 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1202000000000003e-05 2024-09-21T21:43:00.152 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.152 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.152 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:00.152 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248028+0000", 2024-09-21T21:43:00.152 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8219000000000001e-05 2024-09-21T21:43:00.152 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:00.152 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:00.152 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:00.152 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.152 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.153 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19773 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6615040~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:00.153 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273390+0000", 2024-09-21T21:43:00.153 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 68.808669444000003, 2024-09-21T21:43:00.153 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97476086200000001, 2024-09-21T21:43:00.153 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:00.153 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:00.153 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:00.153 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:00.153 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:00.153 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:00.153 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19773 2024-09-21T21:43:00.153 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.153 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:00.153 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.153 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:00.154 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273390+0000", 2024-09-21T21:43:00.154 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:00.154 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.154 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.154 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:00.154 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273390+0000", 2024-09-21T21:43:00.154 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:00.154 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.154 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.154 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:00.154 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273390+0000", 2024-09-21T21:43:00.154 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9999999999999999e-07 2024-09-21T21:43:00.154 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.154 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.155 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:00.155 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273392+0000", 2024-09-21T21:43:00.155 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0640000000000001e-06 2024-09-21T21:43:00.155 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.155 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.155 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:00.155 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273397+0000", 2024-09-21T21:43:00.155 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.113e-06 2024-09-21T21:43:00.155 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.155 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.155 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:00.155 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273401+0000", 2024-09-21T21:43:00.156 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3119999999999998e-06 2024-09-21T21:43:00.156 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.156 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.156 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:00.156 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278271+0000", 2024-09-21T21:43:00.156 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048698880000000002 2024-09-21T21:43:00.156 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.156 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.156 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:00.156 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278278+0000", 2024-09-21T21:43:00.156 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1210000000000001e-06 2024-09-21T21:43:00.156 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.156 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.156 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:00.156 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248060+0000", 2024-09-21T21:43:00.157 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96978224999999996 2024-09-21T21:43:00.157 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.157 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.157 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:00.157 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248111+0000", 2024-09-21T21:43:00.157 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.096e-05 2024-09-21T21:43:00.157 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.157 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.157 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:00.157 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248151+0000", 2024-09-21T21:43:00.157 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9854000000000002e-05 2024-09-21T21:43:00.157 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:00.157 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:00.157 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:00.158 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.158 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.158 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19784 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6643712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:00.158 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273405+0000", 2024-09-21T21:43:00.158 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 68.808654723000004, 2024-09-21T21:43:00.158 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97491798900000004, 2024-09-21T21:43:00.158 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:00.158 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:00.158 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:00.158 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:00.158 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:00.158 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:00.158 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19784 2024-09-21T21:43:00.158 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.159 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:00.159 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.159 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:00.159 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273405+0000", 2024-09-21T21:43:00.159 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:00.159 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.159 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.159 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:00.159 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273405+0000", 2024-09-21T21:43:00.159 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:00.159 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.159 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.159 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:00.159 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273405+0000", 2024-09-21T21:43:00.160 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2100000000000001e-07 2024-09-21T21:43:00.160 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.160 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.160 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:00.160 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273406+0000", 2024-09-21T21:43:00.160 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2049999999999999e-06 2024-09-21T21:43:00.160 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.160 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.160 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:00.160 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273409+0000", 2024-09-21T21:43:00.160 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0599999999999999e-06 2024-09-21T21:43:00.160 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.160 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.160 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:00.161 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273412+0000", 2024-09-21T21:43:00.161 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0730000000000001e-06 2024-09-21T21:43:00.161 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.161 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.161 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:00.161 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278317+0000", 2024-09-21T21:43:00.161 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0049045520000000004 2024-09-21T21:43:00.161 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.161 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.161 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:00.161 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278321+0000", 2024-09-21T21:43:00.161 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6249999999999998e-06 2024-09-21T21:43:00.161 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.161 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.161 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:00.162 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248179+0000", 2024-09-21T21:43:00.162 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96985744500000004 2024-09-21T21:43:00.162 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.162 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.162 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:00.162 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248253+0000", 2024-09-21T21:43:00.162 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3829000000000001e-05 2024-09-21T21:43:00.162 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.162 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.162 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:00.162 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248323+0000", 2024-09-21T21:43:00.162 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9979000000000002e-05 2024-09-21T21:43:00.162 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:00.162 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:00.163 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:00.163 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.163 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.163 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19785 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6672384~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:00.163 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273415+0000", 2024-09-21T21:43:00.163 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 68.808644137000002, 2024-09-21T21:43:00.163 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97500777800000005, 2024-09-21T21:43:00.163 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:00.163 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:00.163 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:00.163 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:00.163 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:00.163 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:00.163 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19785 2024-09-21T21:43:00.164 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.164 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:00.164 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.164 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:00.164 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273415+0000", 2024-09-21T21:43:00.164 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:00.164 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.164 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.164 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:00.164 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273415+0000", 2024-09-21T21:43:00.164 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:00.164 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.164 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.164 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:00.164 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273415+0000", 2024-09-21T21:43:00.165 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0100000000000001e-07 2024-09-21T21:43:00.165 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.165 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.165 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:00.165 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273417+0000", 2024-09-21T21:43:00.165 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.114e-06 2024-09-21T21:43:00.165 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.165 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.165 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:00.165 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273420+0000", 2024-09-21T21:43:00.165 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6770000000000001e-06 2024-09-21T21:43:00.166 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.166 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.166 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:00.166 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273423+0000", 2024-09-21T21:43:00.166 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5450000000000002e-06 2024-09-21T21:43:00.166 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.166 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.166 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:00.166 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278355+0000", 2024-09-21T21:43:00.166 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0049318590000000002 2024-09-21T21:43:00.166 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.166 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.167 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:00.167 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278360+0000", 2024-09-21T21:43:00.167 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3079999999999998e-06 2024-09-21T21:43:00.167 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.167 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.167 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:00.167 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248349+0000", 2024-09-21T21:43:00.167 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96998909899999997 2024-09-21T21:43:00.167 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.167 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.167 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:00.167 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248387+0000", 2024-09-21T21:43:00.167 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7657000000000002e-05 2024-09-21T21:43:00.167 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.168 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.168 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:00.168 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248423+0000", 2024-09-21T21:43:00.168 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6318e-05 2024-09-21T21:43:00.168 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:00.168 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:00.168 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:00.168 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.168 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.168 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19842 3.d 3:b7fa2ca0:::rbd_data.112157a479c.000000000000004d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:00.168 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273426+0000", 2024-09-21T21:43:00.168 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 68.808633084999997, 2024-09-21T21:43:00.168 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.98136526800000001, 2024-09-21T21:43:00.169 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:00.169 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:00.169 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:00.169 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:00.169 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:00.169 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:00.169 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19842 2024-09-21T21:43:00.169 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.169 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:00.169 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.169 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:00.169 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273426+0000", 2024-09-21T21:43:00.169 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:00.170 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.170 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.170 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:00.170 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273426+0000", 2024-09-21T21:43:00.170 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:00.170 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.170 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.170 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:00.170 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273427+0000", 2024-09-21T21:43:00.170 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.72e-07 2024-09-21T21:43:00.171 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.171 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.171 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:00.171 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275836+0000", 2024-09-21T21:43:00.171 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00240909 2024-09-21T21:43:00.171 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.171 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.171 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:00.171 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275844+0000", 2024-09-21T21:43:00.171 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3750000000000003e-06 2024-09-21T21:43:00.171 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.172 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.172 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:00.172 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275849+0000", 2024-09-21T21:43:00.172 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0100000000000003e-06 2024-09-21T21:43:00.172 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.172 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.172 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:00.172 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278394+0000", 2024-09-21T21:43:00.172 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025449890000000001 2024-09-21T21:43:00.172 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.172 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.172 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:00.172 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278402+0000", 2024-09-21T21:43:00.172 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6359999999999994e-06 2024-09-21T21:43:00.173 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.173 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.173 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:00.173 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248454+0000", 2024-09-21T21:43:00.173 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97005259499999996 2024-09-21T21:43:00.173 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.173 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.173 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:00.173 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248576+0000", 2024-09-21T21:43:00.173 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000122127 2024-09-21T21:43:00.173 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.173 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.174 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 2", 2024-09-21T21:43:00.174 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248844+0000", 2024-09-21T21:43:00.174 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00026784699999999999 2024-09-21T21:43:00.174 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.174 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.174 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:00.174 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.252487+0000", 2024-09-21T21:43:00.174 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0036427920000000002 2024-09-21T21:43:00.174 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.174 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.174 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:00.174 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.254691+0000", 2024-09-21T21:43:00.174 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002203899 2024-09-21T21:43:00.175 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.175 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.175 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:00.175 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.254748+0000", 2024-09-21T21:43:00.175 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7163000000000001e-05 2024-09-21T21:43:00.175 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.175 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.175 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:00.175 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.254792+0000", 2024-09-21T21:43:00.175 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3473000000000002e-05 2024-09-21T21:43:00.175 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:00.175 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:00.175 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:00.175 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.176 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.176 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19843 3.d 3:b7fa2ca0:::rbd_data.112157a479c.000000000000004d:head [write 0~0] snapc 21=[21] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:00.176 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.275860+0000", 2024-09-21T21:43:00.176 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 68.806199184999997, 2024-09-21T21:43:00.176 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.98183651999999999, 2024-09-21T21:43:00.176 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:00.176 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:00.176 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:00.176 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:00.176 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:00.176 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:00.176 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19843 2024-09-21T21:43:00.177 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.177 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:00.177 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.177 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:00.177 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275860+0000", 2024-09-21T21:43:00.177 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:00.177 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.177 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.177 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:00.177 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275860+0000", 2024-09-21T21:43:00.177 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:00.177 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.178 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.178 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:00.178 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275869+0000", 2024-09-21T21:43:00.178 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6300000000000004e-06 2024-09-21T21:43:00.178 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.178 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.178 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:00.178 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275871+0000", 2024-09-21T21:43:00.178 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2060000000000001e-06 2024-09-21T21:43:00.178 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.178 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.178 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:00.179 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275876+0000", 2024-09-21T21:43:00.179 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8180000000000003e-06 2024-09-21T21:43:00.179 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.179 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.179 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:00.179 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275880+0000", 2024-09-21T21:43:00.179 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1829999999999996e-06 2024-09-21T21:43:00.179 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.179 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.180 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:00.180 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278416+0000", 2024-09-21T21:43:00.180 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025361149999999998 2024-09-21T21:43:00.180 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.180 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.180 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:00.180 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278419+0000", 2024-09-21T21:43:00.180 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.163e-06 2024-09-21T21:43:00.180 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.180 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.180 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:00.180 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.250249+0000", 2024-09-21T21:43:00.180 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97182984299999997 2024-09-21T21:43:00.180 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.180 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.180 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:00.181 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.250321+0000", 2024-09-21T21:43:00.181 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1656000000000006e-05 2024-09-21T21:43:00.181 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.181 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.181 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 2", 2024-09-21T21:43:00.181 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.250419+0000", 2024-09-21T21:43:00.181 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.8277999999999997e-05 2024-09-21T21:43:00.181 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.181 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.181 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:00.181 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.252516+0000", 2024-09-21T21:43:00.181 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002097224 2024-09-21T21:43:00.181 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.181 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.181 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:00.182 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.257643+0000", 2024-09-21T21:43:00.182 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0051262319999999997 2024-09-21T21:43:00.182 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.182 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.182 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:00.182 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.257673+0000", 2024-09-21T21:43:00.182 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0023e-05 2024-09-21T21:43:00.182 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.182 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.182 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:00.182 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.257697+0000", 2024-09-21T21:43:00.182 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4148999999999999e-05 2024-09-21T21:43:00.182 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:00.183 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:00.183 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:00.183 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.183 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.183 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5626.0:12334 3.3 3:c643a86c:::rbd_data.1121faebdfe3.0000000000000002:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:43:00.183 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:14.411628+0000", 2024-09-21T21:43:00.183 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 45.670430885999998, 2024-09-21T21:43:00.183 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0086947559999999, 2024-09-21T21:43:00.183 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:00.183 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:00.183 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:00.183 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:00.183 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5626", 2024-09-21T21:43:00.183 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:43:00.183 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 12334 2024-09-21T21:43:00.184 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.184 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:00.184 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.184 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:00.184 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411628+0000", 2024-09-21T21:43:00.184 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:00.184 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.184 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.184 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:00.184 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411628+0000", 2024-09-21T21:43:00.184 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:00.184 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.184 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.184 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:00.185 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411629+0000", 2024-09-21T21:43:00.185 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1200000000000002e-07 2024-09-21T21:43:00.185 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.185 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.185 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:00.185 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411633+0000", 2024-09-21T21:43:00.185 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1699999999999999e-06 2024-09-21T21:43:00.185 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.185 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.185 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:00.185 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411645+0000", 2024-09-21T21:43:00.186 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1328999999999999e-05 2024-09-21T21:43:00.186 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.186 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.186 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:00.186 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411652+0000", 2024-09-21T21:43:00.186 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9990000000000002e-06 2024-09-21T21:43:00.186 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.186 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.186 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:00.186 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411717+0000", 2024-09-21T21:43:00.186 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5519999999999996e-05 2024-09-21T21:43:00.186 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.187 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.187 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op must wait for map", 2024-09-21T21:43:00.187 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411725+0000", 2024-09-21T21:43:00.187 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2609999999999996e-06 2024-09-21T21:43:00.187 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.187 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.187 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:00.187 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415904+0000", 2024-09-21T21:43:00.187 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0041792749999999997 2024-09-21T21:43:00.187 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.187 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.187 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:00.187 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415913+0000", 2024-09-21T21:43:00.187 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.054e-06 2024-09-21T21:43:00.188 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.188 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.188 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:00.188 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420096+0000", 2024-09-21T21:43:00.188 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0041835670000001 2024-09-21T21:43:00.188 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.188 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.188 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:00.188 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420216+0000", 2024-09-21T21:43:00.188 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000120025 2024-09-21T21:43:00.188 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.188 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.188 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:00.188 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420323+0000", 2024-09-21T21:43:00.189 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010674400000000001 2024-09-21T21:43:00.189 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:00.189 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:00.189 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:00.189 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.189 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.189 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5626.0:12339 3.3 3:c643a86c:::rbd_data.1121faebdfe3.0000000000000002:head [read 4096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:43:00.189 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:14.411749+0000", 2024-09-21T21:43:00.189 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 45.670310700000002, 2024-09-21T21:43:00.189 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.008818056, 2024-09-21T21:43:00.190 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:00.190 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:00.190 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:00.190 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:00.190 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5626", 2024-09-21T21:43:00.190 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:43:00.190 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 12339 2024-09-21T21:43:00.191 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.191 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:00.191 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.191 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:00.191 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411749+0000", 2024-09-21T21:43:00.191 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:00.191 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.191 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.191 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:00.191 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411749+0000", 2024-09-21T21:43:00.191 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:00.191 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.192 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.192 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:00.192 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411749+0000", 2024-09-21T21:43:00.192 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0900000000000002e-07 2024-09-21T21:43:00.192 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.192 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.192 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:00.192 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411752+0000", 2024-09-21T21:43:00.192 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5610000000000001e-06 2024-09-21T21:43:00.193 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.193 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.193 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:00.193 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411758+0000", 2024-09-21T21:43:00.193 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1890000000000004e-06 2024-09-21T21:43:00.193 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.193 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.194 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:00.194 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411763+0000", 2024-09-21T21:43:00.194 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7589999999999997e-06 2024-09-21T21:43:00.194 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.194 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.194 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:00.194 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411803+0000", 2024-09-21T21:43:00.195 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9922999999999997e-05 2024-09-21T21:43:00.195 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.195 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.195 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting_for_map not empty", 2024-09-21T21:43:00.195 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411809+0000", 2024-09-21T21:43:00.195 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3779999999999998e-06 2024-09-21T21:43:00.196 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.196 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.196 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:00.196 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415937+0000", 2024-09-21T21:43:00.196 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0041276100000000003 2024-09-21T21:43:00.196 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.197 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.197 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:00.197 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415944+0000", 2024-09-21T21:43:00.197 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8329999999999994e-06 2024-09-21T21:43:00.197 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.197 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.197 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:00.198 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420411+0000", 2024-09-21T21:43:00.198 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0044667300000001 2024-09-21T21:43:00.198 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.198 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.198 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:00.198 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420475+0000", 2024-09-21T21:43:00.199 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4356000000000005e-05 2024-09-21T21:43:00.199 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.199 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.199 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:00.199 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420567+0000", 2024-09-21T21:43:00.199 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.1308000000000001e-05 2024-09-21T21:43:00.200 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:00.200 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:00.200 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:00.200 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.200 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.200 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5626.0:12345 3.3 3:c643a86c:::rbd_data.1121faebdfe3.0000000000000002:head [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:43:00.201 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:14.411831+0000", 2024-09-21T21:43:00.201 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 45.670228647000002, 2024-09-21T21:43:00.201 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0089163649999999, 2024-09-21T21:43:00.201 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:00.201 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:00.201 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:00.201 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:00.202 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5626", 2024-09-21T21:43:00.202 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:43:00.202 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 12345 2024-09-21T21:43:00.202 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.202 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:00.202 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.203 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:00.203 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411831+0000", 2024-09-21T21:43:00.203 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:00.203 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.203 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.203 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:00.204 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411831+0000", 2024-09-21T21:43:00.204 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:00.204 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.204 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.204 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:00.204 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411831+0000", 2024-09-21T21:43:00.205 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.08e-07 2024-09-21T21:43:00.205 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.205 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.205 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:00.205 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411833+0000", 2024-09-21T21:43:00.205 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3089999999999998e-06 2024-09-21T21:43:00.205 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.206 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.206 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:00.206 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411840+0000", 2024-09-21T21:43:00.206 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.472e-06 2024-09-21T21:43:00.206 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.206 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.207 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:00.207 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411845+0000", 2024-09-21T21:43:00.207 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7910000000000004e-06 2024-09-21T21:43:00.207 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.207 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.207 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:00.208 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411890+0000", 2024-09-21T21:43:00.208 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5763000000000002e-05 2024-09-21T21:43:00.208 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.208 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.208 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting_for_map not empty", 2024-09-21T21:43:00.208 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411896+0000", 2024-09-21T21:43:00.209 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0190000000000003e-06 2024-09-21T21:43:00.209 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.209 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.209 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:00.209 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415967+0000", 2024-09-21T21:43:00.209 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0040705749999999999 2024-09-21T21:43:00.209 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.210 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.210 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:00.210 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415974+0000", 2024-09-21T21:43:00.210 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3200000000000002e-06 2024-09-21T21:43:00.210 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.210 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.211 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:00.211 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420594+0000", 2024-09-21T21:43:00.211 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.004620093 2024-09-21T21:43:00.211 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.211 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.211 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:00.211 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420658+0000", 2024-09-21T21:43:00.211 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3678999999999998e-05 2024-09-21T21:43:00.211 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:00.211 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:00.212 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:00.212 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420747+0000", 2024-09-21T21:43:00.212 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8936000000000004e-05 2024-09-21T21:43:00.213 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:00.213 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:00.214 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:00.214 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:00.214 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:00.214 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:43:00.216 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T21:43:00.217 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:43:00.214+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:43:00.279 INFO:teuthology.orchestra.run.smithi081.stderr:noscrub is set 2024-09-21T21:43:00.303 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T21:43:00.338 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:43:00.337+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:43:00.375 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:43:00.375 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:43:00.375 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-21T21:43:00.376 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:43:00.385 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T21:43:00.429 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:43:00.428+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:43:00.459 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0: {} 2024-09-21T21:43:00.459 INFO:teuthology.orchestra.run.smithi081.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T21:43:00.475 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1: {} 2024-09-21T21:43:00.475 INFO:teuthology.orchestra.run.smithi081.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T21:43:00.488 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:43:00.488 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:43:00.488 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:43:00.488 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:43:00.488 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:43:00.491 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2: {} 2024-09-21T21:43:00.491 INFO:teuthology.orchestra.run.smithi081.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T21:43:00.497 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T21:43:00.507 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3: {} 2024-09-21T21:43:00.508 INFO:teuthology.orchestra.run.smithi081.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T21:43:00.524 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4: {} 2024-09-21T21:43:00.524 INFO:teuthology.orchestra.run.smithi081.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T21:43:00.537 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:43:00.536+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:43:00.540 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5: {} 2024-09-21T21:43:00.541 INFO:teuthology.orchestra.run.smithi081.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T21:43:00.618 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:43:00.617+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:43:00.639 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:00.639 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:00.648 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T21:43:00.719 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:43:00.718+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:43:00.792 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:00.792 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:00.801 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T21:43:00.820 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:43:00.819+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:43:00.904 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:00.904 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:00.913 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T21:43:00.921 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:43:00.920+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:43:01.017 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:01.017 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:01.022 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:43:01.021+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:43:01.026 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T21:43:01.123 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:43:01.122+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:43:01.130 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:01.130 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:01.139 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T21:43:01.224 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:43:01.223+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:43:01.242 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:01.242 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:01.251 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T21:43:01.270 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 5, 3] out_osds: [0, 4] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-21T21:43:01.271 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T21:43:01.271 INFO:tasks.thrashosds.thrasher:Adding osd 0 2024-09-21T21:43:01.271 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 0 2024-09-21T21:43:01.337 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:43:01.336+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:43:01.407 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:01.407 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:01.417 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T21:43:01.459 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:43:01.457+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:43:01.527 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:43:01.526+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:43:01.562 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:01.562 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:01.571 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T21:43:01.596 INFO:tasks.ceph.mon.a.smithi081.stderr:2024-09-21T21:43:01.595+0000 7f5d67cb4640 -1 mon.a@0(leader).osd e139 definitely_dead 0 2024-09-21T21:43:01.628 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:43:01.627+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:43:01.699 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:01.699 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:01.708 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T21:43:01.729 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:43:01.728+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:43:01.811 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:01.811 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:01.819 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T21:43:01.864 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:43:01.862+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:43:01.923 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:01.923 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:01.931 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T21:43:01.932 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:43:01.930+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:43:02.032 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:43:02.031+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:43:02.074 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:02.074 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:02.082 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T21:43:02.133 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:43:02.132+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:43:02.186 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:02.186 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:02.194 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T21:43:02.215 INFO:teuthology.orchestra.run.smithi081.stderr:marked in osd.0. 2024-09-21T21:43:02.234 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:43:02.232+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:43:02.235 INFO:tasks.thrashosds.thrasher:Added osd 0 2024-09-21T21:43:02.298 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T21:43:02.314 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:02.314 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:02.323 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T21:43:02.367 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:43:02.366+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:43:02.427 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:02.427 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:02.435 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:43:02.433+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:43:02.436 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T21:43:02.536 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:43:02.535+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:43:02.552 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T21:43:02.578 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:02.579 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:02.587 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T21:43:02.637 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:43:02.635+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:43:02.670 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0: {} 2024-09-21T21:43:02.670 INFO:teuthology.orchestra.run.smithi081.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T21:43:02.687 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1: {} 2024-09-21T21:43:02.687 INFO:teuthology.orchestra.run.smithi081.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T21:43:02.702 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2: {} 2024-09-21T21:43:02.702 INFO:teuthology.orchestra.run.smithi081.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T21:43:02.719 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3: {} 2024-09-21T21:43:02.719 INFO:teuthology.orchestra.run.smithi081.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T21:43:02.731 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:02.732 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:02.735 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4: {} 2024-09-21T21:43:02.735 INFO:teuthology.orchestra.run.smithi081.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T21:43:02.737 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:43:02.736+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:43:02.740 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T21:43:02.751 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5: {} 2024-09-21T21:43:02.752 INFO:teuthology.orchestra.run.smithi081.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T21:43:02.839 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:43:02.837+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:43:02.883 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:43:02.883 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:43:02.883 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:43:02.883 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:43:02.883 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:43:02.891 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T21:43:02.940 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:43:02.939+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:43:02.995 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:43:02.995 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-21T21:43:02.995 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-21T21:43:02.995 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:43:02.995 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:02.995 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6974 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5246976~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:02.995 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.922932+0000", 2024-09-21T21:43:02.995 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 93.069662717, 2024-09-21T21:43:02.996 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041773352999999999, 2024-09-21T21:43:02.996 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:02.996 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:02.996 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:02.996 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:02.996 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:02.996 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:02.996 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6974 2024-09-21T21:43:02.996 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:02.996 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:02.996 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:02.996 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:02.996 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922932+0000", 2024-09-21T21:43:02.996 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:02.997 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:02.997 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:02.997 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:02.997 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922932+0000", 2024-09-21T21:43:02.997 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:02.997 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:02.997 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:02.997 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:02.997 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922933+0000", 2024-09-21T21:43:02.997 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.46e-07 2024-09-21T21:43:02.997 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:02.997 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:02.997 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:02.997 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922940+0000", 2024-09-21T21:43:02.998 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6140000000000003e-06 2024-09-21T21:43:02.998 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:02.998 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:02.998 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:02.998 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922949+0000", 2024-09-21T21:43:02.998 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6899999999999998e-06 2024-09-21T21:43:02.998 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:02.998 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:02.998 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:02.998 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922955+0000", 2024-09-21T21:43:02.998 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5140000000000001e-06 2024-09-21T21:43:02.998 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:02.998 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:02.998 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:02.999 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964547+0000", 2024-09-21T21:43:02.999 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041592854999999998 2024-09-21T21:43:02.999 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:02.999 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:02.999 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:02.999 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964617+0000", 2024-09-21T21:43:02.999 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9058000000000002e-05 2024-09-21T21:43:02.999 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:02.999 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:02.999 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:02.999 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964706+0000", 2024-09-21T21:43:02.999 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9276000000000004e-05 2024-09-21T21:43:02.999 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:02.999 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:03.000 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:03.000 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.000 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.000 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6976 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5261312~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:03.000 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.922961+0000", 2024-09-21T21:43:03.000 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 93.069633671999995, 2024-09-21T21:43:03.000 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041863648000000003, 2024-09-21T21:43:03.000 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:03.000 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:03.000 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:03.000 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:03.000 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:03.000 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:03.000 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6976 2024-09-21T21:43:03.000 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.001 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:03.001 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.001 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:03.001 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922961+0000", 2024-09-21T21:43:03.001 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:03.001 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.001 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.001 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:03.001 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922961+0000", 2024-09-21T21:43:03.001 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:03.001 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.001 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.001 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:03.001 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922962+0000", 2024-09-21T21:43:03.001 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.89e-07 2024-09-21T21:43:03.001 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.002 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.002 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:03.002 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922964+0000", 2024-09-21T21:43:03.002 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1270000000000001e-06 2024-09-21T21:43:03.002 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.002 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.002 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:03.002 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922970+0000", 2024-09-21T21:43:03.002 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8030000000000002e-06 2024-09-21T21:43:03.002 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.002 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.002 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:03.002 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922979+0000", 2024-09-21T21:43:03.002 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5559999999999996e-06 2024-09-21T21:43:03.002 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.002 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.003 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:03.003 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964734+0000", 2024-09-21T21:43:03.003 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041754415000000003 2024-09-21T21:43:03.003 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.003 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.003 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:03.003 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964775+0000", 2024-09-21T21:43:03.003 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0806999999999997e-05 2024-09-21T21:43:03.003 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.003 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.003 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:03.003 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964825+0000", 2024-09-21T21:43:03.003 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0550999999999998e-05 2024-09-21T21:43:03.003 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:03.003 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:03.004 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:03.004 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.004 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.004 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6978 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5275648~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:03.004 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.922987+0000", 2024-09-21T21:43:03.004 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 93.069607887000004, 2024-09-21T21:43:03.004 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042043876000000001, 2024-09-21T21:43:03.004 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:03.004 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:03.004 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:03.004 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:03.004 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:03.004 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:03.004 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6978 2024-09-21T21:43:03.004 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.005 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:03.005 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.005 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:03.005 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922987+0000", 2024-09-21T21:43:03.005 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:03.005 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.005 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.005 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:03.005 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922987+0000", 2024-09-21T21:43:03.005 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:03.005 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.005 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.005 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:03.005 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922988+0000", 2024-09-21T21:43:03.005 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1199999999999998e-07 2024-09-21T21:43:03.005 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.006 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.006 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:03.006 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922990+0000", 2024-09-21T21:43:03.006 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.142e-06 2024-09-21T21:43:03.006 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.006 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.006 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:03.006 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922997+0000", 2024-09-21T21:43:03.006 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.199e-06 2024-09-21T21:43:03.006 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.006 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.006 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:03.006 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923004+0000", 2024-09-21T21:43:03.006 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3599999999999998e-06 2024-09-21T21:43:03.006 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.007 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.007 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:03.007 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964856+0000", 2024-09-21T21:43:03.007 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041851521000000003 2024-09-21T21:43:03.007 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.007 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.007 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:03.007 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964944+0000", 2024-09-21T21:43:03.007 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7859e-05 2024-09-21T21:43:03.007 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.007 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.007 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:03.007 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965031+0000", 2024-09-21T21:43:03.007 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7383e-05 2024-09-21T21:43:03.008 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:03.008 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:03.008 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:03.008 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.008 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.008 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6980 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5289984~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:03.008 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923011+0000", 2024-09-21T21:43:03.008 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 93.069584550000002, 2024-09-21T21:43:03.008 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042139337999999998, 2024-09-21T21:43:03.008 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:03.008 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:03.008 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:03.008 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:03.008 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:03.009 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:03.009 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6980 2024-09-21T21:43:03.009 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.009 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:03.009 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.009 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:03.009 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923011+0000", 2024-09-21T21:43:03.009 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:03.009 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.009 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.009 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:03.009 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923011+0000", 2024-09-21T21:43:03.009 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:03.009 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.009 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.010 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:03.010 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923011+0000", 2024-09-21T21:43:03.010 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6699999999999999e-07 2024-09-21T21:43:03.010 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.010 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.010 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:03.010 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923013+0000", 2024-09-21T21:43:03.010 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0279999999999999e-06 2024-09-21T21:43:03.010 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.010 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.010 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:03.010 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923018+0000", 2024-09-21T21:43:03.010 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4790000000000004e-06 2024-09-21T21:43:03.010 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.011 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.011 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:03.011 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923029+0000", 2024-09-21T21:43:03.011 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0601999999999999e-05 2024-09-21T21:43:03.011 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.011 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.011 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:03.011 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965059+0000", 2024-09-21T21:43:03.011 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042029947999999998 2024-09-21T21:43:03.011 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.011 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.011 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:03.011 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965100+0000", 2024-09-21T21:43:03.011 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1170000000000001e-05 2024-09-21T21:43:03.011 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.012 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.012 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:03.012 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965150+0000", 2024-09-21T21:43:03.012 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9743999999999997e-05 2024-09-21T21:43:03.012 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:03.012 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:03.012 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:03.012 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.012 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.012 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6982 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5304320~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:03.012 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923036+0000", 2024-09-21T21:43:03.012 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 93.069558728999993, 2024-09-21T21:43:03.012 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042225666000000002, 2024-09-21T21:43:03.012 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:03.012 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:03.013 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:03.013 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:03.013 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:03.013 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:03.013 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6982 2024-09-21T21:43:03.013 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.013 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:03.013 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.013 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:03.013 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923036+0000", 2024-09-21T21:43:03.013 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:03.013 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.013 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.013 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:03.014 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923036+0000", 2024-09-21T21:43:03.014 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:03.014 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.014 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.014 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:03.014 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923037+0000", 2024-09-21T21:43:03.014 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2800000000000002e-07 2024-09-21T21:43:03.014 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.014 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.014 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:03.014 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923039+0000", 2024-09-21T21:43:03.014 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1500000000000002e-06 2024-09-21T21:43:03.014 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.014 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.015 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:03.015 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923049+0000", 2024-09-21T21:43:03.015 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0236e-05 2024-09-21T21:43:03.015 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.015 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.015 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:03.015 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923056+0000", 2024-09-21T21:43:03.015 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2540000000000003e-06 2024-09-21T21:43:03.015 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.015 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.015 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:03.015 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965173+0000", 2024-09-21T21:43:03.015 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042116686 2024-09-21T21:43:03.015 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.015 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.016 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:03.016 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965215+0000", 2024-09-21T21:43:03.016 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1696000000000002e-05 2024-09-21T21:43:03.016 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.016 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.016 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:03.016 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965262+0000", 2024-09-21T21:43:03.016 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7216000000000003e-05 2024-09-21T21:43:03.016 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:03.016 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:03.016 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:03.016 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.016 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.016 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6984 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5318656~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:03.016 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923063+0000", 2024-09-21T21:43:03.017 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 93.069532074999998, 2024-09-21T21:43:03.017 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042318651999999998, 2024-09-21T21:43:03.017 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:03.017 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:03.017 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:03.017 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:03.017 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:03.017 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:03.017 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6984 2024-09-21T21:43:03.017 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.017 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:03.017 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.017 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:03.017 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923063+0000", 2024-09-21T21:43:03.017 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:03.018 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.018 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.018 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:03.018 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923063+0000", 2024-09-21T21:43:03.018 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:03.018 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.018 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.018 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:03.018 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923064+0000", 2024-09-21T21:43:03.018 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2e-07 2024-09-21T21:43:03.018 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.018 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.018 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:03.018 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923066+0000", 2024-09-21T21:43:03.018 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.057e-06 2024-09-21T21:43:03.019 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.019 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.019 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:03.019 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923071+0000", 2024-09-21T21:43:03.019 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8810000000000001e-06 2024-09-21T21:43:03.019 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.019 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.019 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:03.019 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923078+0000", 2024-09-21T21:43:03.019 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5479999999999998e-06 2024-09-21T21:43:03.019 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.019 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.019 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:03.019 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965288+0000", 2024-09-21T21:43:03.020 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042209576999999998 2024-09-21T21:43:03.020 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.020 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.020 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:03.020 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965332+0000", 2024-09-21T21:43:03.020 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4366999999999997e-05 2024-09-21T21:43:03.020 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.020 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.020 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:03.020 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965382+0000", 2024-09-21T21:43:03.020 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9802000000000002e-05 2024-09-21T21:43:03.020 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:03.020 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:03.020 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:03.020 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.021 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.021 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6987 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5332992~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:03.021 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923085+0000", 2024-09-21T21:43:03.021 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 93.069510550000004, 2024-09-21T21:43:03.021 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042412103999999999, 2024-09-21T21:43:03.021 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:03.021 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:03.021 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:03.021 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:03.021 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:03.021 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:03.021 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6987 2024-09-21T21:43:03.021 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.021 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:03.021 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.022 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:03.022 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923085+0000", 2024-09-21T21:43:03.022 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:03.022 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.022 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.022 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:03.022 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923085+0000", 2024-09-21T21:43:03.022 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:03.022 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.022 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.022 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:03.022 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923085+0000", 2024-09-21T21:43:03.022 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.89e-07 2024-09-21T21:43:03.022 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.023 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.023 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:03.023 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923087+0000", 2024-09-21T21:43:03.023 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0810000000000001e-06 2024-09-21T21:43:03.023 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.023 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.023 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:03.023 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923098+0000", 2024-09-21T21:43:03.023 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1299e-05 2024-09-21T21:43:03.023 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.023 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.023 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:03.023 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923104+0000", 2024-09-21T21:43:03.023 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9100000000000002e-06 2024-09-21T21:43:03.024 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.024 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.024 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:03.024 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965404+0000", 2024-09-21T21:43:03.024 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042299622000000002 2024-09-21T21:43:03.024 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.024 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.024 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:03.024 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965445+0000", 2024-09-21T21:43:03.024 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0701000000000002e-05 2024-09-21T21:43:03.024 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.024 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.024 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:03.024 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965497+0000", 2024-09-21T21:43:03.024 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2101999999999997e-05 2024-09-21T21:43:03.025 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:03.025 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:03.025 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:03.025 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.025 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.025 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6989 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5347328~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:03.025 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923111+0000", 2024-09-21T21:43:03.025 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 93.069484209999999, 2024-09-21T21:43:03.025 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042505188999999999, 2024-09-21T21:43:03.025 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:03.025 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:03.025 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:03.025 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:03.025 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:03.026 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:03.026 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6989 2024-09-21T21:43:03.026 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.026 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:03.026 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.026 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:03.026 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923111+0000", 2024-09-21T21:43:03.026 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:03.026 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.026 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.026 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:03.026 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923111+0000", 2024-09-21T21:43:03.026 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:03.026 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.026 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.027 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:03.027 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923111+0000", 2024-09-21T21:43:03.027 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4499999999999997e-07 2024-09-21T21:43:03.027 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.027 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.027 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:03.027 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923114+0000", 2024-09-21T21:43:03.027 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1569999999999998e-06 2024-09-21T21:43:03.027 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.027 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.027 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:03.027 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923120+0000", 2024-09-21T21:43:03.027 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1759999999999998e-06 2024-09-21T21:43:03.027 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.027 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.028 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:03.028 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923125+0000", 2024-09-21T21:43:03.028 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2560000000000002e-06 2024-09-21T21:43:03.028 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.028 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.028 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:03.028 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965523+0000", 2024-09-21T21:43:03.028 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042398094999999997 2024-09-21T21:43:03.028 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.028 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.028 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:03.028 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965566+0000", 2024-09-21T21:43:03.028 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3393999999999998e-05 2024-09-21T21:43:03.028 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.029 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.029 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:03.029 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965616+0000", 2024-09-21T21:43:03.029 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9666000000000002e-05 2024-09-21T21:43:03.029 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:03.029 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:03.029 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:03.029 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.029 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.029 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6991 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5361664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:03.029 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923132+0000", 2024-09-21T21:43:03.029 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 93.069462920999996, 2024-09-21T21:43:03.029 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042600779999999998, 2024-09-21T21:43:03.029 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:03.029 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:03.030 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:03.030 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:03.030 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:03.030 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:03.030 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6991 2024-09-21T21:43:03.030 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.030 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:03.030 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.030 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:03.030 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923132+0000", 2024-09-21T21:43:03.030 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:03.030 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.030 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.030 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:03.030 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923132+0000", 2024-09-21T21:43:03.031 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:03.031 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.031 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.031 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:03.031 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923132+0000", 2024-09-21T21:43:03.031 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4200000000000002e-07 2024-09-21T21:43:03.031 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.031 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.031 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:03.031 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923134+0000", 2024-09-21T21:43:03.031 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2270000000000001e-06 2024-09-21T21:43:03.031 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.031 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.031 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:03.032 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923137+0000", 2024-09-21T21:43:03.032 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1930000000000002e-06 2024-09-21T21:43:03.032 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.032 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.032 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:03.032 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923139+0000", 2024-09-21T21:43:03.032 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4710000000000001e-06 2024-09-21T21:43:03.032 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.032 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.032 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:03.032 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965639+0000", 2024-09-21T21:43:03.032 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.04249965 2024-09-21T21:43:03.032 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.032 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.032 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:03.033 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965679+0000", 2024-09-21T21:43:03.033 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9852999999999999e-05 2024-09-21T21:43:03.033 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.033 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.033 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:03.033 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965733+0000", 2024-09-21T21:43:03.033 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4144000000000002e-05 2024-09-21T21:43:03.033 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:03.033 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:03.033 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:03.033 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.033 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.033 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6993 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5384192~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:03.033 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923143+0000", 2024-09-21T21:43:03.034 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 93.069452127000005, 2024-09-21T21:43:03.034 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042708182999999997, 2024-09-21T21:43:03.034 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:03.034 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:03.034 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:03.034 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:03.034 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:03.034 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:03.034 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6993 2024-09-21T21:43:03.034 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.034 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:03.034 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.034 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:03.034 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923143+0000", 2024-09-21T21:43:03.034 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:03.035 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.035 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.035 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:03.035 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923143+0000", 2024-09-21T21:43:03.035 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:03.035 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.035 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.035 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:03.035 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923143+0000", 2024-09-21T21:43:03.035 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0800000000000001e-07 2024-09-21T21:43:03.035 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.035 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.035 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:03.035 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923144+0000", 2024-09-21T21:43:03.036 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.009e-06 2024-09-21T21:43:03.036 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.036 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.036 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:03.036 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923148+0000", 2024-09-21T21:43:03.036 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.737e-06 2024-09-21T21:43:03.036 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.036 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.036 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:03.036 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923150+0000", 2024-09-21T21:43:03.036 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4669999999999998e-06 2024-09-21T21:43:03.036 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.036 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.036 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:03.037 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965760+0000", 2024-09-21T21:43:03.037 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042609399999999999 2024-09-21T21:43:03.037 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.037 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.037 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:03.037 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965801+0000", 2024-09-21T21:43:03.037 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1399999999999997e-05 2024-09-21T21:43:03.037 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.037 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.037 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:03.037 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965851+0000", 2024-09-21T21:43:03.037 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9962e-05 2024-09-21T21:43:03.037 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:03.037 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:03.038 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:03.038 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.038 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.038 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6995 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5398528~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:03.038 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923153+0000", 2024-09-21T21:43:03.038 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 93.069441763, 2024-09-21T21:43:03.038 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042828034000000001, 2024-09-21T21:43:03.038 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:03.038 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:03.038 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:03.038 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:03.038 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:03.038 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:03.038 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6995 2024-09-21T21:43:03.038 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.039 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:03.039 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.039 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:03.039 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923153+0000", 2024-09-21T21:43:03.039 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:03.039 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.039 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.039 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:03.039 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923153+0000", 2024-09-21T21:43:03.039 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:03.039 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.039 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.039 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:03.040 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923154+0000", 2024-09-21T21:43:03.040 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2600000000000001e-07 2024-09-21T21:43:03.040 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.040 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.040 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:03.040 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923158+0000", 2024-09-21T21:43:03.040 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0559999999999998e-06 2024-09-21T21:43:03.040 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.040 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.040 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:03.040 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923161+0000", 2024-09-21T21:43:03.040 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6519999999999999e-06 2024-09-21T21:43:03.040 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.040 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.041 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:03.041 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923170+0000", 2024-09-21T21:43:03.041 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5720000000000003e-06 2024-09-21T21:43:03.041 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.041 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.041 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:03.041 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965887+0000", 2024-09-21T21:43:03.041 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042716863000000001 2024-09-21T21:43:03.041 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.041 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.041 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:03.041 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965927+0000", 2024-09-21T21:43:03.041 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0346999999999998e-05 2024-09-21T21:43:03.041 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.041 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.042 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:03.042 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965981+0000", 2024-09-21T21:43:03.042 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4317999999999997e-05 2024-09-21T21:43:03.042 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:03.042 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:03.042 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:03.042 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.042 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.042 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6997 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5412864~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:03.042 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923174+0000", 2024-09-21T21:43:03.042 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 93.069421607999999, 2024-09-21T21:43:03.042 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042928296999999997, 2024-09-21T21:43:03.042 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:03.042 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:03.042 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:03.043 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:03.043 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:03.043 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:03.043 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6997 2024-09-21T21:43:03.043 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.043 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:03.043 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.043 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:03.043 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923174+0000", 2024-09-21T21:43:03.043 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:03.043 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.043 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.043 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:03.044 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923174+0000", 2024-09-21T21:43:03.044 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:03.044 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.044 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.044 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:03.044 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923174+0000", 2024-09-21T21:43:03.044 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.67e-07 2024-09-21T21:43:03.044 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.044 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.044 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:03.044 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923178+0000", 2024-09-21T21:43:03.044 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1330000000000001e-06 2024-09-21T21:43:03.044 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.044 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.044 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:03.045 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923182+0000", 2024-09-21T21:43:03.045 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.794e-06 2024-09-21T21:43:03.045 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.045 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.045 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:03.045 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923184+0000", 2024-09-21T21:43:03.045 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.435e-06 2024-09-21T21:43:03.045 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.045 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.045 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:03.045 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966009+0000", 2024-09-21T21:43:03.045 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042824327000000002 2024-09-21T21:43:03.045 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.045 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.046 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:03.046 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966052+0000", 2024-09-21T21:43:03.046 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3034000000000002e-05 2024-09-21T21:43:03.046 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.046 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.046 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:03.046 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966102+0000", 2024-09-21T21:43:03.046 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0306999999999998e-05 2024-09-21T21:43:03.046 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:03.046 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:03.046 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:03.046 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.046 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.046 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6999 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5427200~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:03.046 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923187+0000", 2024-09-21T21:43:03.047 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 93.06940788, 2024-09-21T21:43:03.047 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043104267000000002, 2024-09-21T21:43:03.047 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:03.047 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:03.047 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:03.047 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:03.047 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:03.047 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:03.047 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6999 2024-09-21T21:43:03.047 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.047 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:03.047 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.047 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:03.047 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923187+0000", 2024-09-21T21:43:03.047 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:03.048 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.048 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.048 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:03.048 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923187+0000", 2024-09-21T21:43:03.048 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:03.048 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.048 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.048 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:03.048 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923188+0000", 2024-09-21T21:43:03.048 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3999999999999998e-07 2024-09-21T21:43:03.048 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.048 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.048 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:03.049 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923189+0000", 2024-09-21T21:43:03.049 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4169999999999999e-06 2024-09-21T21:43:03.049 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.049 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.049 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:03.049 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923194+0000", 2024-09-21T21:43:03.049 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1819999999999997e-06 2024-09-21T21:43:03.049 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.049 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.049 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:03.049 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923198+0000", 2024-09-21T21:43:03.049 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.143e-06 2024-09-21T21:43:03.049 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.049 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.049 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:03.050 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966135+0000", 2024-09-21T21:43:03.050 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042936297999999998 2024-09-21T21:43:03.050 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.050 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.050 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:03.050 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966204+0000", 2024-09-21T21:43:03.050 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8989e-05 2024-09-21T21:43:03.050 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.050 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.050 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:03.050 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966292+0000", 2024-09-21T21:43:03.050 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7997999999999993e-05 2024-09-21T21:43:03.050 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:03.050 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:03.051 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:03.051 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.051 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.051 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7001 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5441536~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:03.051 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923211+0000", 2024-09-21T21:43:03.051 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 93.069384088999996, 2024-09-21T21:43:03.051 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043196518000000003, 2024-09-21T21:43:03.051 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:03.051 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:03.051 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:03.051 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:03.051 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:03.051 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:03.051 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7001 2024-09-21T21:43:03.052 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.052 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:03.052 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.052 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:03.052 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923211+0000", 2024-09-21T21:43:03.052 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:03.052 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.052 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.052 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:03.052 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923211+0000", 2024-09-21T21:43:03.052 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:03.052 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.052 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.052 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:03.052 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923212+0000", 2024-09-21T21:43:03.053 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3900000000000005e-07 2024-09-21T21:43:03.053 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.053 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.053 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:03.053 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923214+0000", 2024-09-21T21:43:03.053 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9970000000000001e-06 2024-09-21T21:43:03.053 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.053 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.053 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:03.053 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923219+0000", 2024-09-21T21:43:03.053 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4310000000000002e-06 2024-09-21T21:43:03.053 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.053 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.053 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:03.054 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923224+0000", 2024-09-21T21:43:03.054 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6410000000000001e-06 2024-09-21T21:43:03.054 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.054 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.054 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:03.054 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966318+0000", 2024-09-21T21:43:03.054 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043093842 2024-09-21T21:43:03.054 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.054 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.054 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:03.054 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966359+0000", 2024-09-21T21:43:03.054 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1260999999999997e-05 2024-09-21T21:43:03.054 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.054 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.055 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:03.055 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966408+0000", 2024-09-21T21:43:03.055 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8807000000000002e-05 2024-09-21T21:43:03.055 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:03.055 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:03.055 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:03.055 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.055 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.055 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7002 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5455872~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:03.055 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923229+0000", 2024-09-21T21:43:03.055 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 93.069365679000001, 2024-09-21T21:43:03.055 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043364568999999999, 2024-09-21T21:43:03.055 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:03.055 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:03.056 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:03.056 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:03.056 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:03.056 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:03.056 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7002 2024-09-21T21:43:03.056 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.056 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:03.056 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.056 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:03.056 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923229+0000", 2024-09-21T21:43:03.056 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:03.056 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.056 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.056 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:03.057 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923229+0000", 2024-09-21T21:43:03.057 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:03.057 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.057 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.057 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:03.057 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923230+0000", 2024-09-21T21:43:03.057 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4700000000000002e-07 2024-09-21T21:43:03.057 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.057 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.057 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:03.057 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923232+0000", 2024-09-21T21:43:03.057 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0719999999999998e-06 2024-09-21T21:43:03.057 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.057 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.057 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:03.058 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923237+0000", 2024-09-21T21:43:03.058 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2290000000000002e-06 2024-09-21T21:43:03.058 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.058 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.058 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:03.058 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923245+0000", 2024-09-21T21:43:03.058 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2959999999999999e-06 2024-09-21T21:43:03.058 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.058 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.058 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:03.058 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966438+0000", 2024-09-21T21:43:03.059 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043193027000000002 2024-09-21T21:43:03.059 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.059 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.059 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:03.059 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966506+0000", 2024-09-21T21:43:03.059 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7707999999999996e-05 2024-09-21T21:43:03.059 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.059 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.059 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:03.059 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966594+0000", 2024-09-21T21:43:03.059 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.789e-05 2024-09-21T21:43:03.059 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:03.059 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:03.059 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:03.060 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.060 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.060 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7005 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5470208~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:03.060 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923254+0000", 2024-09-21T21:43:03.060 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 93.069341448000003, 2024-09-21T21:43:03.060 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043457176, 2024-09-21T21:43:03.060 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:03.060 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:03.060 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:03.060 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:03.060 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:03.060 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:03.060 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7005 2024-09-21T21:43:03.060 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.060 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:03.061 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.061 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:03.061 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923254+0000", 2024-09-21T21:43:03.061 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:03.061 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.061 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.061 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:03.061 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923254+0000", 2024-09-21T21:43:03.061 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:03.061 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.061 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.061 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:03.061 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923254+0000", 2024-09-21T21:43:03.061 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4700000000000002e-07 2024-09-21T21:43:03.061 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.062 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.062 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:03.062 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923256+0000", 2024-09-21T21:43:03.062 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.007e-06 2024-09-21T21:43:03.062 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.062 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.062 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:03.062 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923266+0000", 2024-09-21T21:43:03.062 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.4700000000000008e-06 2024-09-21T21:43:03.062 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.062 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.062 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:03.062 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923271+0000", 2024-09-21T21:43:03.062 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2809999999999999e-06 2024-09-21T21:43:03.063 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.063 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.063 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:03.063 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966619+0000", 2024-09-21T21:43:03.063 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043347951000000003 2024-09-21T21:43:03.063 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.063 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.063 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:03.063 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966661+0000", 2024-09-21T21:43:03.063 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2614999999999998e-05 2024-09-21T21:43:03.063 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.063 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.063 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:03.063 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966711+0000", 2024-09-21T21:43:03.064 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9505000000000001e-05 2024-09-21T21:43:03.064 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:03.064 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:03.064 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:03.064 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.064 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.064 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7008 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5484544~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:03.064 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923278+0000", 2024-09-21T21:43:03.064 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 93.069316800999999, 2024-09-21T21:43:03.064 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043631071, 2024-09-21T21:43:03.064 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:03.064 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:03.064 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:03.064 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:03.065 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:03.065 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:03.065 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7008 2024-09-21T21:43:03.065 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.065 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:03.065 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.065 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:03.065 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923278+0000", 2024-09-21T21:43:03.065 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:03.065 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.065 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.065 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:03.065 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923278+0000", 2024-09-21T21:43:03.065 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:03.065 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.066 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.066 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:03.066 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923279+0000", 2024-09-21T21:43:03.066 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.77e-07 2024-09-21T21:43:03.066 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.066 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.066 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:03.066 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923281+0000", 2024-09-21T21:43:03.066 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0219999999999999e-06 2024-09-21T21:43:03.066 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.066 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.066 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:03.066 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923287+0000", 2024-09-21T21:43:03.066 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7799999999999997e-06 2024-09-21T21:43:03.067 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.067 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.067 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:03.067 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923291+0000", 2024-09-21T21:43:03.067 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1699999999999999e-06 2024-09-21T21:43:03.067 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.067 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.067 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:03.067 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966742+0000", 2024-09-21T21:43:03.067 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043451344000000003 2024-09-21T21:43:03.067 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.067 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.067 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:03.067 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966809+0000", 2024-09-21T21:43:03.067 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7070999999999995e-05 2024-09-21T21:43:03.068 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.068 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.068 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:03.068 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966909+0000", 2024-09-21T21:43:03.068 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000100307 2024-09-21T21:43:03.068 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:03.068 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:03.068 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:03.068 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.068 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.068 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7010 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5498880~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:03.068 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923301+0000", 2024-09-21T21:43:03.068 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 93.069293905999999, 2024-09-21T21:43:03.068 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043717830999999999, 2024-09-21T21:43:03.068 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:03.069 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:03.069 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:03.069 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:03.069 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:03.069 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:03.069 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7010 2024-09-21T21:43:03.069 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.069 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:03.069 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.069 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:03.069 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923301+0000", 2024-09-21T21:43:03.069 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:03.069 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.069 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.070 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:03.070 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923301+0000", 2024-09-21T21:43:03.070 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:03.070 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.070 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.070 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:03.070 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923302+0000", 2024-09-21T21:43:03.070 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.41e-07 2024-09-21T21:43:03.070 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.070 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.070 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:03.070 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923304+0000", 2024-09-21T21:43:03.070 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0990000000000001e-06 2024-09-21T21:43:03.070 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.071 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.071 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:03.071 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923310+0000", 2024-09-21T21:43:03.071 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3860000000000004e-06 2024-09-21T21:43:03.071 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.071 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.071 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:03.071 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923315+0000", 2024-09-21T21:43:03.071 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8459999999999999e-06 2024-09-21T21:43:03.071 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.071 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.071 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:03.071 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966938+0000", 2024-09-21T21:43:03.071 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043622693999999997 2024-09-21T21:43:03.071 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.072 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.072 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:03.072 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966975+0000", 2024-09-21T21:43:03.072 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7608999999999999e-05 2024-09-21T21:43:03.072 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.072 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.072 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:03.072 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.967019+0000", 2024-09-21T21:43:03.072 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3856000000000002e-05 2024-09-21T21:43:03.072 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:03.072 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:03.072 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:03.072 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.072 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.072 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7354 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 2107392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:03.073 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.936932+0000", 2024-09-21T21:43:03.073 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 93.055663050999996, 2024-09-21T21:43:03.073 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041607708, 2024-09-21T21:43:03.073 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:03.073 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:03.073 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:03.073 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:03.073 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:03.073 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:03.073 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7354 2024-09-21T21:43:03.073 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.073 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:03.073 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.073 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:03.073 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.936932+0000", 2024-09-21T21:43:03.074 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:03.074 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.074 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.074 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:03.074 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.936932+0000", 2024-09-21T21:43:03.074 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:03.074 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.074 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.074 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:03.074 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.937276+0000", 2024-09-21T21:43:03.074 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00034344200000000001 2024-09-21T21:43:03.074 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.074 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.074 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:03.075 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.937279+0000", 2024-09-21T21:43:03.075 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.844e-06 2024-09-21T21:43:03.075 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.075 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.075 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:03.075 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.937288+0000", 2024-09-21T21:43:03.075 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2409999999999995e-06 2024-09-21T21:43:03.075 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.075 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.075 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:03.075 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.937293+0000", 2024-09-21T21:43:03.075 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6559999999999997e-06 2024-09-21T21:43:03.075 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.075 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.076 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:03.076 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.978448+0000", 2024-09-21T21:43:03.076 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041154621000000002 2024-09-21T21:43:03.076 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.076 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.076 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:03.076 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.978487+0000", 2024-09-21T21:43:03.076 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9254000000000001e-05 2024-09-21T21:43:03.076 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.076 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.076 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:03.076 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.978540+0000", 2024-09-21T21:43:03.076 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2649999999999999e-05 2024-09-21T21:43:03.076 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:03.077 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:03.077 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:03.077 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.077 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.077 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5215.0:1054 3.9 3:946d2401:::rbd_data.1121524d2522.000000000000000b:head [set-alloc-hint object_size 16777216 write_size 16777216,write 6615040~59392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-21T21:43:03.077 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:38.148979+0000", 2024-09-21T21:43:03.077 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 84.843616084000004, 2024-09-21T21:43:03.077 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047870388999999999, 2024-09-21T21:43:03.077 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:03.077 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:03.077 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:03.077 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:03.077 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5215", 2024-09-21T21:43:03.077 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1580282168", 2024-09-21T21:43:03.077 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1054 2024-09-21T21:43:03.078 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.078 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:03.078 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.078 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:03.078 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.148979+0000", 2024-09-21T21:43:03.078 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:03.078 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.078 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.078 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:03.078 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.148979+0000", 2024-09-21T21:43:03.078 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:03.078 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.078 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.078 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:03.079 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.148980+0000", 2024-09-21T21:43:03.079 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5799999999999999e-07 2024-09-21T21:43:03.079 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.079 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.079 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:03.079 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.149039+0000", 2024-09-21T21:43:03.079 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9151000000000003e-05 2024-09-21T21:43:03.079 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.079 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.079 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:03.079 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.149047+0000", 2024-09-21T21:43:03.079 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1000000000000004e-06 2024-09-21T21:43:03.079 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.079 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.079 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:03.080 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.149052+0000", 2024-09-21T21:43:03.080 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5959999999999997e-06 2024-09-21T21:43:03.080 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.080 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.080 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:03.080 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.195288+0000", 2024-09-21T21:43:03.080 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.046236248000000001 2024-09-21T21:43:03.080 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.080 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.080 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:03.080 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.195334+0000", 2024-09-21T21:43:03.080 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6575e-05 2024-09-21T21:43:03.080 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.080 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.080 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 2", 2024-09-21T21:43:03.081 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.195406+0000", 2024-09-21T21:43:03.081 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1715e-05 2024-09-21T21:43:03.081 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.081 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.081 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:03.081 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.196018+0000", 2024-09-21T21:43:03.081 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00061218999999999998 2024-09-21T21:43:03.081 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.081 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.081 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:03.081 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.196794+0000", 2024-09-21T21:43:03.081 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000775814 2024-09-21T21:43:03.081 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.081 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.081 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:03.082 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.196827+0000", 2024-09-21T21:43:03.082 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2419000000000003e-05 2024-09-21T21:43:03.082 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.082 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.082 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:03.082 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.196849+0000", 2024-09-21T21:43:03.082 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2923000000000001e-05 2024-09-21T21:43:03.082 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:03.082 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:03.082 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:03.082 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:03.082 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:03.082 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:43:03.084 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T21:43:03.084 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:43:03.082+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:43:03.184 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:43:03.183+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:43:03.221 INFO:teuthology.orchestra.run.smithi081.stderr:nodeep-scrub is set 2024-09-21T21:43:03.231 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:43:03.231 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:43:03.231 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-21T21:43:03.231 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:43:03.240 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T21:43:03.286 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:43:03.285+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:43:03.383 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:43:03.383 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:43:03.383 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:43:03.383 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:43:03.383 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:43:03.387 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:43:03.386+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:43:03.392 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T21:43:03.488 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:43:03.486+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:43:03.496 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:43:03.497 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-21T21:43:03.497 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-21T21:43:03.497 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:43:03.497 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.497 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4385.0:15202 3.16 3:6888948b:::rbd_header.1121493f6459:head [watch reconnect cookie 140325440202496 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-21T21:43:03.497 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:41.736601+0000", 2024-09-21T21:43:03.497 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 81.757565893999995, 2024-09-21T21:43:03.497 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5040056769999999, 2024-09-21T21:43:03.497 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:03.497 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:03.497 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:03.497 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:03.498 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4385", 2024-09-21T21:43:03.498 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:43:03.498 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 15202 2024-09-21T21:43:03.498 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.498 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:03.498 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.498 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:03.498 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736601+0000", 2024-09-21T21:43:03.498 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:03.498 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.498 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.498 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:03.499 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736601+0000", 2024-09-21T21:43:03.499 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:03.499 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.499 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.499 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:03.499 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736602+0000", 2024-09-21T21:43:03.499 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0699999999999999e-06 2024-09-21T21:43:03.499 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.499 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.499 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:03.499 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736607+0000", 2024-09-21T21:43:03.499 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2520000000000001e-06 2024-09-21T21:43:03.499 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.499 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.499 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:03.500 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736617+0000", 2024-09-21T21:43:03.500 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0074e-05 2024-09-21T21:43:03.500 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.500 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.500 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:03.500 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736622+0000", 2024-09-21T21:43:03.500 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.101e-06 2024-09-21T21:43:03.500 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.500 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.500 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:03.500 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736664+0000", 2024-09-21T21:43:03.500 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1869999999999997e-05 2024-09-21T21:43:03.500 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.500 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.501 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op must wait for map", 2024-09-21T21:43:03.501 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736667+0000", 2024-09-21T21:43:03.501 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4750000000000002e-06 2024-09-21T21:43:03.501 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.501 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.501 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:03.501 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742886+0000", 2024-09-21T21:43:03.501 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0062187579999999996 2024-09-21T21:43:03.501 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.501 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.501 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:03.501 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742892+0000", 2024-09-21T21:43:03.501 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0440000000000001e-06 2024-09-21T21:43:03.501 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.502 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.502 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:03.502 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210307+0000", 2024-09-21T21:43:03.502 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.467414781 2024-09-21T21:43:03.502 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.502 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.502 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:03.502 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210322+0000", 2024-09-21T21:43:03.502 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5311e-05 2024-09-21T21:43:03.502 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.502 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.502 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:03.502 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.237624+0000", 2024-09-21T21:43:03.502 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0273018920000001 2024-09-21T21:43:03.503 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.503 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.503 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:03.503 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.237870+0000", 2024-09-21T21:43:03.503 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00024622899999999999 2024-09-21T21:43:03.503 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.503 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.503 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:03.503 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.240607+0000", 2024-09-21T21:43:03.503 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0027368200000000001 2024-09-21T21:43:03.503 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:03.503 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:03.503 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:03.503 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.503 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.504 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5415.0:379 3.16 3:6888948b:::rbd_header.1121493f6459:head [watch reconnect cookie 139702602617920 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-21T21:43:03.504 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:41.739544+0000", 2024-09-21T21:43:03.504 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 81.754623464999995, 2024-09-21T21:43:03.504 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.501180645, 2024-09-21T21:43:03.504 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:03.504 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:03.504 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:03.504 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:03.504 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5415", 2024-09-21T21:43:03.504 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1212816607", 2024-09-21T21:43:03.504 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 379 2024-09-21T21:43:03.504 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.504 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:03.504 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.505 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:03.505 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739544+0000", 2024-09-21T21:43:03.505 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:03.505 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.505 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.505 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:03.505 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739544+0000", 2024-09-21T21:43:03.505 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:03.505 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.505 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.505 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:03.505 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739545+0000", 2024-09-21T21:43:03.505 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.065e-06 2024-09-21T21:43:03.505 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.505 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.506 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:03.506 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739550+0000", 2024-09-21T21:43:03.506 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9599999999999999e-06 2024-09-21T21:43:03.506 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.506 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.506 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:03.506 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739565+0000", 2024-09-21T21:43:03.506 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5709000000000002e-05 2024-09-21T21:43:03.506 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.506 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.506 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:03.506 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739573+0000", 2024-09-21T21:43:03.506 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7849999999999992e-06 2024-09-21T21:43:03.506 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.506 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.507 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:03.507 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739634+0000", 2024-09-21T21:43:03.507 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0732000000000001e-05 2024-09-21T21:43:03.507 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.507 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.507 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op must wait for map", 2024-09-21T21:43:03.507 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739640+0000", 2024-09-21T21:43:03.507 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0000000000000002e-06 2024-09-21T21:43:03.507 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.507 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.507 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:03.507 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742958+0000", 2024-09-21T21:43:03.507 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0033182390000000002 2024-09-21T21:43:03.507 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.507 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.508 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:03.508 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742966+0000", 2024-09-21T21:43:03.508 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1610000000000003e-06 2024-09-21T21:43:03.508 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.508 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.508 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:03.508 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210405+0000", 2024-09-21T21:43:03.508 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.467438725 2024-09-21T21:43:03.508 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.508 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.508 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:03.508 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210415+0000", 2024-09-21T21:43:03.508 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0234000000000001e-05 2024-09-21T21:43:03.508 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.509 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.509 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:03.509 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.238743+0000", 2024-09-21T21:43:03.509 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0283273070000001 2024-09-21T21:43:03.509 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.509 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.509 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:03.509 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.238842+0000", 2024-09-21T21:43:03.509 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.9896999999999996e-05 2024-09-21T21:43:03.509 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.509 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.509 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:03.509 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.240724+0000", 2024-09-21T21:43:03.509 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001881831 2024-09-21T21:43:03.509 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:03.510 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:03.510 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:03.510 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.510 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.510 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4385.0:15203 3.16 3:6888948b:::rbd_header.1121493f6459:head [call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-21T21:43:03.510 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:41.739737+0000", 2024-09-21T21:43:03.510 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 81.754430454000001, 2024-09-21T21:43:03.510 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4989621799999999, 2024-09-21T21:43:03.510 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:03.510 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:03.510 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:03.510 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:03.510 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4385", 2024-09-21T21:43:03.510 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:43:03.511 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 15203 2024-09-21T21:43:03.511 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.511 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:03.511 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.511 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:03.511 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739737+0000", 2024-09-21T21:43:03.511 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:03.511 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.511 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.511 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:03.511 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739737+0000", 2024-09-21T21:43:03.511 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:03.511 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.511 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.511 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:03.512 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739738+0000", 2024-09-21T21:43:03.512 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.7499999999999998e-07 2024-09-21T21:43:03.512 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.512 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.512 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:03.512 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739743+0000", 2024-09-21T21:43:03.512 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5049999999999998e-06 2024-09-21T21:43:03.512 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.512 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.512 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:03.512 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739757+0000", 2024-09-21T21:43:03.512 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3947e-05 2024-09-21T21:43:03.512 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.512 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.512 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:03.513 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739765+0000", 2024-09-21T21:43:03.513 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9310000000000008e-06 2024-09-21T21:43:03.513 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.513 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.513 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:03.513 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739816+0000", 2024-09-21T21:43:03.513 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0559999999999997e-05 2024-09-21T21:43:03.513 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.513 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.513 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting_for_map not empty", 2024-09-21T21:43:03.513 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739821+0000", 2024-09-21T21:43:03.513 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3539999999999999e-06 2024-09-21T21:43:03.513 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.513 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.513 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:03.514 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742928+0000", 2024-09-21T21:43:03.514 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0031074399999999999 2024-09-21T21:43:03.514 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.514 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.514 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:03.514 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742937+0000", 2024-09-21T21:43:03.514 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1289999999999996e-06 2024-09-21T21:43:03.514 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.514 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.514 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:03.514 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210350+0000", 2024-09-21T21:43:03.514 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.46741311299999999 2024-09-21T21:43:03.514 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.514 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.515 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:03.515 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210358+0000", 2024-09-21T21:43:03.515 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1869999999999997e-06 2024-09-21T21:43:03.515 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.515 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.515 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:03.515 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.238398+0000", 2024-09-21T21:43:03.515 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0280397830000001 2024-09-21T21:43:03.515 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.515 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.515 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:03.515 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.238489+0000", 2024-09-21T21:43:03.515 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.0958000000000006e-05 2024-09-21T21:43:03.515 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.515 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.516 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:03.516 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.238699+0000", 2024-09-21T21:43:03.516 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00021029800000000001 2024-09-21T21:43:03.516 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:03.516 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:03.516 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:03.516 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.516 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.516 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4385.0:15204 3.16 3:6888948b:::rbd_header.1121493f6459:head [watch reconnect cookie 140325440202496 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-21T21:43:03.516 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:42.210726+0000", 2024-09-21T21:43:03.516 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 81.283441576000001, 2024-09-21T21:43:03.516 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.030304557, 2024-09-21T21:43:03.516 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:03.516 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:03.516 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:03.517 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:03.517 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4385", 2024-09-21T21:43:03.517 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:43:03.517 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 15204 2024-09-21T21:43:03.517 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.517 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:03.517 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.517 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:03.517 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210726+0000", 2024-09-21T21:43:03.517 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:03.517 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.517 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.517 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:03.517 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210726+0000", 2024-09-21T21:43:03.517 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:03.518 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.518 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.518 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:03.518 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210727+0000", 2024-09-21T21:43:03.518 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.204e-06 2024-09-21T21:43:03.518 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.518 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.518 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:03.518 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210732+0000", 2024-09-21T21:43:03.518 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5249999999999996e-06 2024-09-21T21:43:03.518 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.518 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.518 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:03.518 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210744+0000", 2024-09-21T21:43:03.519 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1405e-05 2024-09-21T21:43:03.519 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.519 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.519 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:03.519 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210754+0000", 2024-09-21T21:43:03.519 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0122999999999999e-05 2024-09-21T21:43:03.519 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.519 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.519 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:03.519 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210798+0000", 2024-09-21T21:43:03.519 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4005000000000003e-05 2024-09-21T21:43:03.519 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.519 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.519 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:03.519 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210803+0000", 2024-09-21T21:43:03.520 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5130000000000004e-06 2024-09-21T21:43:03.520 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.520 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.520 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:03.520 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.239260+0000", 2024-09-21T21:43:03.520 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0284563040000001 2024-09-21T21:43:03.520 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.520 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.520 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:03.520 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.239318+0000", 2024-09-21T21:43:03.520 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8486000000000002e-05 2024-09-21T21:43:03.520 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.520 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.520 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:03.521 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.241030+0000", 2024-09-21T21:43:03.521 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0017119920000000001 2024-09-21T21:43:03.521 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:03.521 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:03.521 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:03.521 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.521 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.521 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5415.0:380 3.16 3:6888948b:::rbd_header.1121493f6459:head [watch reconnect cookie 139702602617920 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-21T21:43:03.521 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:42.210777+0000", 2024-09-21T21:43:03.521 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 81.283389796999998, 2024-09-21T21:43:03.521 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.030410566, 2024-09-21T21:43:03.521 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:03.521 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:03.521 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:03.521 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:03.522 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5415", 2024-09-21T21:43:03.522 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1212816607", 2024-09-21T21:43:03.522 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 380 2024-09-21T21:43:03.522 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.522 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:03.522 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.522 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:03.522 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210777+0000", 2024-09-21T21:43:03.522 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:03.522 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.522 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.522 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:03.522 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210777+0000", 2024-09-21T21:43:03.522 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:03.523 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.523 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.523 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:03.523 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210778+0000", 2024-09-21T21:43:03.523 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8999999999999996e-07 2024-09-21T21:43:03.523 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.523 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.523 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:03.523 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210780+0000", 2024-09-21T21:43:03.523 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.503e-06 2024-09-21T21:43:03.523 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.523 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.523 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:03.523 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210788+0000", 2024-09-21T21:43:03.523 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0999999999999998e-06 2024-09-21T21:43:03.524 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.524 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.524 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:03.524 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210799+0000", 2024-09-21T21:43:03.524 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1703e-05 2024-09-21T21:43:03.524 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.524 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.524 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:03.524 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210833+0000", 2024-09-21T21:43:03.524 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4008999999999999e-05 2024-09-21T21:43:03.524 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.524 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.524 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:03.524 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210838+0000", 2024-09-21T21:43:03.524 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7480000000000001e-06 2024-09-21T21:43:03.525 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.525 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.525 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:03.525 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.239612+0000", 2024-09-21T21:43:03.525 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.028773554 2024-09-21T21:43:03.525 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.525 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.525 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:03.525 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.239677+0000", 2024-09-21T21:43:03.525 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5053000000000002e-05 2024-09-21T21:43:03.525 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.525 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.525 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:03.525 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.241188+0000", 2024-09-21T21:43:03.526 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001511306 2024-09-21T21:43:03.526 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:03.526 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:03.526 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:03.526 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.526 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.526 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24794 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:03.526 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.584085+0000", 2024-09-21T21:43:03.526 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 40.910082621000001, 2024-09-21T21:43:03.526 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.096209996000000006, 2024-09-21T21:43:03.526 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:03.526 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:03.526 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:03.526 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:03.527 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:03.527 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:03.527 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24794 2024-09-21T21:43:03.527 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.527 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:03.527 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.527 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:03.527 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584085+0000", 2024-09-21T21:43:03.527 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:03.527 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.527 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.527 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:03.527 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584085+0000", 2024-09-21T21:43:03.527 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:03.527 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.528 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.528 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:03.528 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584085+0000", 2024-09-21T21:43:03.528 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3200000000000001e-07 2024-09-21T21:43:03.528 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.528 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.528 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:03.528 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584086+0000", 2024-09-21T21:43:03.528 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.717e-06 2024-09-21T21:43:03.528 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.528 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.528 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:03.528 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584090+0000", 2024-09-21T21:43:03.528 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3450000000000002e-06 2024-09-21T21:43:03.528 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.529 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.529 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:03.529 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584092+0000", 2024-09-21T21:43:03.529 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6359999999999998e-06 2024-09-21T21:43:03.529 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.529 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.529 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:03.529 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.676497+0000", 2024-09-21T21:43:03.529 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.092404659 2024-09-21T21:43:03.529 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.529 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.529 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:03.529 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.676567+0000", 2024-09-21T21:43:03.529 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9627000000000003e-05 2024-09-21T21:43:03.529 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.530 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.530 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:03.530 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.676925+0000", 2024-09-21T21:43:03.530 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00035848500000000002 2024-09-21T21:43:03.530 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.530 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.530 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:03.530 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.678427+0000", 2024-09-21T21:43:03.530 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001501638 2024-09-21T21:43:03.530 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.530 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.530 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:03.530 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.680204+0000", 2024-09-21T21:43:03.530 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0017766979999999999 2024-09-21T21:43:03.531 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.531 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.531 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:03.531 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.680256+0000", 2024-09-21T21:43:03.531 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2549999999999997e-05 2024-09-21T21:43:03.531 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.531 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.531 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:03.531 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.680295+0000", 2024-09-21T21:43:03.531 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8408999999999998e-05 2024-09-21T21:43:03.531 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:03.531 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:03.531 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:03.531 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.531 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.532 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24796 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:03.532 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.584099+0000", 2024-09-21T21:43:03.532 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 40.910068052, 2024-09-21T21:43:03.532 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.097297429000000005, 2024-09-21T21:43:03.532 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:03.532 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:03.532 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:03.532 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:03.532 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:03.532 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:03.532 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24796 2024-09-21T21:43:03.532 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.532 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:03.532 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.532 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:03.533 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584099+0000", 2024-09-21T21:43:03.533 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:03.533 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.533 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.533 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:03.533 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584099+0000", 2024-09-21T21:43:03.533 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:03.533 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.533 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.533 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:03.533 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584099+0000", 2024-09-21T21:43:03.533 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3999999999999998e-07 2024-09-21T21:43:03.533 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.533 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.533 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:03.534 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584101+0000", 2024-09-21T21:43:03.534 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5850000000000001e-06 2024-09-21T21:43:03.534 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.534 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.534 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:03.534 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584108+0000", 2024-09-21T21:43:03.534 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.498e-06 2024-09-21T21:43:03.534 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.534 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.534 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:03.534 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584112+0000", 2024-09-21T21:43:03.534 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.252e-06 2024-09-21T21:43:03.534 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.534 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.535 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:03.535 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.677336+0000", 2024-09-21T21:43:03.535 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.093224206000000004 2024-09-21T21:43:03.535 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.535 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.535 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:03.535 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.677411+0000", 2024-09-21T21:43:03.535 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5036000000000005e-05 2024-09-21T21:43:03.535 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.535 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.535 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:03.535 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.677782+0000", 2024-09-21T21:43:03.535 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00037153099999999998 2024-09-21T21:43:03.535 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.536 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.536 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:03.536 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.680360+0000", 2024-09-21T21:43:03.536 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025773020000000001 2024-09-21T21:43:03.536 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.536 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.536 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:03.536 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681313+0000", 2024-09-21T21:43:03.536 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00095341100000000004 2024-09-21T21:43:03.536 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.536 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.536 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:03.536 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681361+0000", 2024-09-21T21:43:03.536 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7497999999999998e-05 2024-09-21T21:43:03.536 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.537 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.537 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:03.537 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681397+0000", 2024-09-21T21:43:03.537 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.587e-05 2024-09-21T21:43:03.537 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:03.537 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:03.537 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:03.537 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.537 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.537 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24797 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:03.537 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.584116+0000", 2024-09-21T21:43:03.537 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 40.910051563000003, 2024-09-21T21:43:03.537 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.097428257000000004, 2024-09-21T21:43:03.537 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:03.538 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:03.538 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:03.538 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:03.538 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:03.538 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:03.538 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24797 2024-09-21T21:43:03.538 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.538 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:03.538 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.538 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:03.538 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584116+0000", 2024-09-21T21:43:03.538 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:03.538 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.538 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.538 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:03.539 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584116+0000", 2024-09-21T21:43:03.539 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:03.539 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.539 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.539 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:03.539 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584116+0000", 2024-09-21T21:43:03.539 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.53e-07 2024-09-21T21:43:03.539 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.539 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.539 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:03.539 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584118+0000", 2024-09-21T21:43:03.539 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6810000000000001e-06 2024-09-21T21:43:03.539 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.539 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.540 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:03.540 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584121+0000", 2024-09-21T21:43:03.540 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1099999999999999e-06 2024-09-21T21:43:03.540 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.540 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.540 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:03.540 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584123+0000", 2024-09-21T21:43:03.540 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5890000000000001e-06 2024-09-21T21:43:03.540 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.540 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.540 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:03.540 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.678476+0000", 2024-09-21T21:43:03.540 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.094352320000000003 2024-09-21T21:43:03.540 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.541 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.541 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:03.541 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.678556+0000", 2024-09-21T21:43:03.541 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0291999999999996e-05 2024-09-21T21:43:03.541 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.541 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.541 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:03.541 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.678871+0000", 2024-09-21T21:43:03.541 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000314689 2024-09-21T21:43:03.541 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.541 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.541 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:03.541 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681452+0000", 2024-09-21T21:43:03.541 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025817890000000001 2024-09-21T21:43:03.541 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.542 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.542 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:03.542 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681467+0000", 2024-09-21T21:43:03.542 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4256e-05 2024-09-21T21:43:03.542 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.542 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.542 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:03.542 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681510+0000", 2024-09-21T21:43:03.542 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3560999999999998e-05 2024-09-21T21:43:03.542 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.542 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.542 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:03.542 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681544+0000", 2024-09-21T21:43:03.542 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3717000000000003e-05 2024-09-21T21:43:03.542 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:03.543 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:03.543 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:03.543 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.543 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.543 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24798 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:03.543 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.584130+0000", 2024-09-21T21:43:03.543 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 40.910037039999999, 2024-09-21T21:43:03.543 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.098350526999999993, 2024-09-21T21:43:03.543 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:03.543 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:03.543 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:03.543 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:03.543 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:03.543 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:03.543 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24798 2024-09-21T21:43:03.544 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.544 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:03.544 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.544 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:03.544 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584130+0000", 2024-09-21T21:43:03.544 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:03.544 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.544 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.544 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:03.544 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584130+0000", 2024-09-21T21:43:03.544 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:03.544 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.544 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.544 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:03.544 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584131+0000", 2024-09-21T21:43:03.545 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4700000000000001e-07 2024-09-21T21:43:03.545 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.545 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.545 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:03.545 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584133+0000", 2024-09-21T21:43:03.545 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.34e-06 2024-09-21T21:43:03.545 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.545 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.545 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:03.545 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584137+0000", 2024-09-21T21:43:03.545 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5609999999999999e-06 2024-09-21T21:43:03.545 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.545 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.545 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:03.546 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584139+0000", 2024-09-21T21:43:03.546 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7350000000000001e-06 2024-09-21T21:43:03.546 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.546 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.546 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:03.546 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.679274+0000", 2024-09-21T21:43:03.546 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.095134719000000006 2024-09-21T21:43:03.546 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.546 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.546 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:03.546 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.679358+0000", 2024-09-21T21:43:03.546 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3609999999999994e-05 2024-09-21T21:43:03.546 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.546 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.546 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:03.547 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.679685+0000", 2024-09-21T21:43:03.547 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00032732399999999999 2024-09-21T21:43:03.547 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.547 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.547 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:03.547 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681553+0000", 2024-09-21T21:43:03.547 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001868059 2024-09-21T21:43:03.547 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.547 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.547 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:03.547 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.682375+0000", 2024-09-21T21:43:03.547 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00082232099999999999 2024-09-21T21:43:03.547 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.547 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.547 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:03.548 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.682431+0000", 2024-09-21T21:43:03.548 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6051999999999998e-05 2024-09-21T21:43:03.548 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.548 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.548 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:03.548 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.682481+0000", 2024-09-21T21:43:03.548 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9258999999999997e-05 2024-09-21T21:43:03.548 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:03.548 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:03.548 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:03.548 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.548 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.548 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24799 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:03.548 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.584159+0000", 2024-09-21T21:43:03.548 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 40.910008052000002, 2024-09-21T21:43:03.549 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.100021551, 2024-09-21T21:43:03.549 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:03.549 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:03.549 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:03.549 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:03.549 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:03.549 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:03.549 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24799 2024-09-21T21:43:03.549 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.549 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:03.549 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.549 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:03.549 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584159+0000", 2024-09-21T21:43:03.549 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:03.550 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.550 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.550 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:03.550 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584159+0000", 2024-09-21T21:43:03.550 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:03.550 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.550 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.550 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:03.550 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584160+0000", 2024-09-21T21:43:03.550 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1300000000000001e-07 2024-09-21T21:43:03.550 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.550 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.550 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:03.550 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584162+0000", 2024-09-21T21:43:03.551 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5409999999999999e-06 2024-09-21T21:43:03.551 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.551 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.551 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:03.551 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584169+0000", 2024-09-21T21:43:03.551 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7880000000000001e-06 2024-09-21T21:43:03.551 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.551 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.551 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:03.551 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584174+0000", 2024-09-21T21:43:03.551 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9590000000000003e-06 2024-09-21T21:43:03.551 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.551 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.551 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:03.551 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.680400+0000", 2024-09-21T21:43:03.552 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.096226514999999999 2024-09-21T21:43:03.552 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.552 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.552 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:03.552 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.680480+0000", 2024-09-21T21:43:03.552 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9589e-05 2024-09-21T21:43:03.552 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.552 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.552 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:03.552 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.680847+0000", 2024-09-21T21:43:03.552 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00036757700000000001 2024-09-21T21:43:03.552 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.552 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.552 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:03.552 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.682555+0000", 2024-09-21T21:43:03.553 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001707791 2024-09-21T21:43:03.553 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.553 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.553 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:03.553 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684093+0000", 2024-09-21T21:43:03.553 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0015377170000000001 2024-09-21T21:43:03.553 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.553 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.553 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:03.553 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684144+0000", 2024-09-21T21:43:03.553 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1208000000000002e-05 2024-09-21T21:43:03.553 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.553 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.553 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:03.553 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684181+0000", 2024-09-21T21:43:03.554 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6452999999999998e-05 2024-09-21T21:43:03.554 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:03.554 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:03.554 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:03.554 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.554 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.554 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24835 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:03.554 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586332+0000", 2024-09-21T21:43:03.554 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 40.907835437999999, 2024-09-21T21:43:03.554 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.098730360000000003, 2024-09-21T21:43:03.554 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:03.554 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:03.554 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:03.554 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:03.555 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:03.555 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:03.555 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24835 2024-09-21T21:43:03.555 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.555 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:03.555 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.555 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:03.555 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586332+0000", 2024-09-21T21:43:03.555 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:03.555 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.555 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.555 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:03.555 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586332+0000", 2024-09-21T21:43:03.555 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:03.555 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.556 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.556 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:03.556 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586333+0000", 2024-09-21T21:43:03.556 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0200000000000001e-07 2024-09-21T21:43:03.556 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.556 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.556 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:03.556 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586346+0000", 2024-09-21T21:43:03.556 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3745e-05 2024-09-21T21:43:03.556 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.556 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.556 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:03.556 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586366+0000", 2024-09-21T21:43:03.556 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9272000000000001e-05 2024-09-21T21:43:03.556 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.557 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.557 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:03.557 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586377+0000", 2024-09-21T21:43:03.557 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1197e-05 2024-09-21T21:43:03.557 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.557 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.557 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:03.557 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681578+0000", 2024-09-21T21:43:03.557 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.095201420999999994 2024-09-21T21:43:03.557 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.557 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.557 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:03.557 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681640+0000", 2024-09-21T21:43:03.557 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1834000000000005e-05 2024-09-21T21:43:03.558 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.558 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.558 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:03.558 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681915+0000", 2024-09-21T21:43:03.558 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00027534199999999998 2024-09-21T21:43:03.558 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.558 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.558 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:03.558 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684230+0000", 2024-09-21T21:43:03.558 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00231424 2024-09-21T21:43:03.558 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.558 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.558 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:03.558 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684959+0000", 2024-09-21T21:43:03.558 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00072921399999999995 2024-09-21T21:43:03.559 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.559 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.559 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:03.559 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.685008+0000", 2024-09-21T21:43:03.559 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8860999999999999e-05 2024-09-21T21:43:03.559 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.559 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.559 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:03.559 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.685062+0000", 2024-09-21T21:43:03.559 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4432000000000003e-05 2024-09-21T21:43:03.559 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:03.559 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:03.559 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:03.559 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.559 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.560 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24837 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:03.560 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586395+0000", 2024-09-21T21:43:03.560 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 40.907772502999997, 2024-09-21T21:43:03.560 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.099700522, 2024-09-21T21:43:03.560 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:03.560 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:03.560 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:03.560 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:03.560 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:03.560 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:03.560 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24837 2024-09-21T21:43:03.560 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.560 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:03.560 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.560 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:03.561 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586395+0000", 2024-09-21T21:43:03.561 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:03.561 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.561 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.561 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:03.561 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586395+0000", 2024-09-21T21:43:03.561 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:03.561 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.561 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.561 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:03.561 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586395+0000", 2024-09-21T21:43:03.561 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.06e-07 2024-09-21T21:43:03.561 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.561 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.561 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:03.562 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586401+0000", 2024-09-21T21:43:03.562 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0889999999999996e-06 2024-09-21T21:43:03.562 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.562 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.562 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:03.562 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586410+0000", 2024-09-21T21:43:03.562 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.918e-06 2024-09-21T21:43:03.562 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.562 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.562 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:03.562 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586424+0000", 2024-09-21T21:43:03.562 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3478999999999999e-05 2024-09-21T21:43:03.562 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.562 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.563 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:03.563 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.682590+0000", 2024-09-21T21:43:03.563 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.096166086999999997 2024-09-21T21:43:03.563 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.563 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.563 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:03.563 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.682657+0000", 2024-09-21T21:43:03.563 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6878000000000006e-05 2024-09-21T21:43:03.563 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.563 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.563 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:03.563 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.682879+0000", 2024-09-21T21:43:03.563 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00022238 2024-09-21T21:43:03.563 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.563 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.564 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:03.564 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.685073+0000", 2024-09-21T21:43:03.564 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021938119999999998 2024-09-21T21:43:03.564 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.564 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.564 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:03.564 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686038+0000", 2024-09-21T21:43:03.564 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00096539499999999997 2024-09-21T21:43:03.564 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.564 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.564 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:03.564 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686071+0000", 2024-09-21T21:43:03.564 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2514000000000001e-05 2024-09-21T21:43:03.564 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.564 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.565 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:03.565 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686095+0000", 2024-09-21T21:43:03.565 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4363999999999999e-05 2024-09-21T21:43:03.565 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:03.565 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:03.565 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:03.565 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.565 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.565 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24839 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:03.565 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586468+0000", 2024-09-21T21:43:03.565 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 40.907698850999999, 2024-09-21T21:43:03.565 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.099739355000000002, 2024-09-21T21:43:03.565 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:03.565 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:03.565 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:03.566 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:03.566 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:03.566 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:03.566 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24839 2024-09-21T21:43:03.566 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.566 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:03.566 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.566 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:03.566 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586468+0000", 2024-09-21T21:43:03.566 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:03.566 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.566 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.566 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:03.566 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586468+0000", 2024-09-21T21:43:03.567 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:03.567 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.567 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.567 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:03.567 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586469+0000", 2024-09-21T21:43:03.567 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2200000000000004e-07 2024-09-21T21:43:03.567 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.567 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.567 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:03.567 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586472+0000", 2024-09-21T21:43:03.567 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9960000000000002e-06 2024-09-21T21:43:03.567 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.567 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.567 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:03.567 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586479+0000", 2024-09-21T21:43:03.568 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2699999999999999e-06 2024-09-21T21:43:03.568 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.568 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.568 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:03.568 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586485+0000", 2024-09-21T21:43:03.568 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6999999999999996e-06 2024-09-21T21:43:03.568 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.568 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.568 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:03.568 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.683276+0000", 2024-09-21T21:43:03.568 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.096791618999999995 2024-09-21T21:43:03.568 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.568 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.568 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:03.568 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.683326+0000", 2024-09-21T21:43:03.569 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9384e-05 2024-09-21T21:43:03.569 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.569 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.569 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:03.569 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.683655+0000", 2024-09-21T21:43:03.569 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00032965700000000002 2024-09-21T21:43:03.569 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.569 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.569 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:03.569 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686120+0000", 2024-09-21T21:43:03.569 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002464819 2024-09-21T21:43:03.569 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.569 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.569 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:03.569 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686159+0000", 2024-09-21T21:43:03.570 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9181000000000003e-05 2024-09-21T21:43:03.570 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.570 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.570 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:03.570 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686185+0000", 2024-09-21T21:43:03.570 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5907000000000001e-05 2024-09-21T21:43:03.570 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.570 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.570 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:03.570 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686208+0000", 2024-09-21T21:43:03.570 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.23e-05 2024-09-21T21:43:03.570 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:03.570 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:03.570 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:03.571 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.571 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.571 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24841 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:03.571 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586492+0000", 2024-09-21T21:43:03.571 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 40.907674688999997, 2024-09-21T21:43:03.571 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.101672077, 2024-09-21T21:43:03.571 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:03.571 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:03.571 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:03.571 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:03.571 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:03.571 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:03.571 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24841 2024-09-21T21:43:03.571 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.571 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:03.572 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.572 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:03.572 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586492+0000", 2024-09-21T21:43:03.572 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:03.572 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.572 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.572 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:03.572 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586492+0000", 2024-09-21T21:43:03.572 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:03.572 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.572 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.572 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:03.572 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586493+0000", 2024-09-21T21:43:03.572 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5999999999999999e-07 2024-09-21T21:43:03.572 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.573 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.573 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:03.573 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586496+0000", 2024-09-21T21:43:03.573 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6089999999999999e-06 2024-09-21T21:43:03.573 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.573 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.573 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:03.573 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586502+0000", 2024-09-21T21:43:03.573 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6270000000000002e-06 2024-09-21T21:43:03.573 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.573 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.573 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:03.573 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586507+0000", 2024-09-21T21:43:03.573 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8810000000000004e-06 2024-09-21T21:43:03.573 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.574 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.574 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:03.574 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684273+0000", 2024-09-21T21:43:03.574 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.097766415999999995 2024-09-21T21:43:03.574 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.574 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.574 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:03.574 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684359+0000", 2024-09-21T21:43:03.574 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5574000000000004e-05 2024-09-21T21:43:03.574 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.574 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.574 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:03.574 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684568+0000", 2024-09-21T21:43:03.574 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000208529 2024-09-21T21:43:03.574 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.575 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.575 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:03.575 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.687186+0000", 2024-09-21T21:43:03.575 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0026180769999999999 2024-09-21T21:43:03.575 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.575 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.575 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:03.575 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688093+0000", 2024-09-21T21:43:03.575 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00090766999999999996 2024-09-21T21:43:03.575 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.575 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.575 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:03.575 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688142+0000", 2024-09-21T21:43:03.575 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8986000000000002e-05 2024-09-21T21:43:03.576 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.576 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.576 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:03.576 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688165+0000", 2024-09-21T21:43:03.576 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2248000000000001e-05 2024-09-21T21:43:03.576 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:03.576 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:03.576 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:03.576 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.576 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.576 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24843 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:03.576 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586514+0000", 2024-09-21T21:43:03.576 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 40.907653402999998, 2024-09-21T21:43:03.576 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.101826449, 2024-09-21T21:43:03.576 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:03.577 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:03.577 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:03.577 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:03.577 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:03.577 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:03.577 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24843 2024-09-21T21:43:03.577 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.577 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:03.577 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.577 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:03.577 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586514+0000", 2024-09-21T21:43:03.577 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:03.577 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.577 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.577 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:03.578 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586514+0000", 2024-09-21T21:43:03.578 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:03.578 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.578 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.578 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:03.578 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586514+0000", 2024-09-21T21:43:03.578 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9999999999999998e-07 2024-09-21T21:43:03.578 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.578 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.578 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:03.578 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586517+0000", 2024-09-21T21:43:03.578 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4980000000000001e-06 2024-09-21T21:43:03.578 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.578 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.578 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:03.579 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586523+0000", 2024-09-21T21:43:03.579 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4609999999999996e-06 2024-09-21T21:43:03.579 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.579 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.579 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:03.579 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586528+0000", 2024-09-21T21:43:03.579 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2979999999999999e-06 2024-09-21T21:43:03.579 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.579 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.579 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:03.579 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.685104+0000", 2024-09-21T21:43:03.579 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.098575837999999999 2024-09-21T21:43:03.579 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.579 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.580 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:03.580 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.685184+0000", 2024-09-21T21:43:03.580 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9807000000000003e-05 2024-09-21T21:43:03.580 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.580 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.580 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:03.580 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.685496+0000", 2024-09-21T21:43:03.580 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00031225900000000001 2024-09-21T21:43:03.580 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.580 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.580 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:03.580 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688238+0000", 2024-09-21T21:43:03.580 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0027419409999999999 2024-09-21T21:43:03.580 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.580 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.581 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:03.581 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688253+0000", 2024-09-21T21:43:03.581 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4331000000000001e-05 2024-09-21T21:43:03.581 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.581 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.581 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:03.581 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688296+0000", 2024-09-21T21:43:03.581 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3801000000000003e-05 2024-09-21T21:43:03.581 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.581 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.581 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:03.581 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688340+0000", 2024-09-21T21:43:03.581 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3815e-05 2024-09-21T21:43:03.581 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:03.581 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:03.582 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:03.582 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.582 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.582 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24845 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:03.582 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586535+0000", 2024-09-21T21:43:03.582 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 40.907632522999997, 2024-09-21T21:43:03.582 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.10285678600000001, 2024-09-21T21:43:03.582 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:03.582 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:03.582 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:03.582 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:03.582 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:03.582 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:03.582 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24845 2024-09-21T21:43:03.582 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.583 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:03.583 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.583 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:03.583 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586535+0000", 2024-09-21T21:43:03.583 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:03.583 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.583 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.583 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:03.583 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586535+0000", 2024-09-21T21:43:03.583 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:03.583 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.583 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.583 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:03.583 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586535+0000", 2024-09-21T21:43:03.584 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9000000000000002e-07 2024-09-21T21:43:03.584 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.584 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.584 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:03.584 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586537+0000", 2024-09-21T21:43:03.584 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4569999999999999e-06 2024-09-21T21:43:03.584 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.584 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.584 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:03.584 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586544+0000", 2024-09-21T21:43:03.584 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5849999999999996e-06 2024-09-21T21:43:03.584 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.584 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.584 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:03.584 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586549+0000", 2024-09-21T21:43:03.585 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0089999999999998e-06 2024-09-21T21:43:03.585 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.585 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.585 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:03.585 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686239+0000", 2024-09-21T21:43:03.585 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.099689531999999997 2024-09-21T21:43:03.585 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.585 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.585 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:03.585 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686289+0000", 2024-09-21T21:43:03.585 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.007e-05 2024-09-21T21:43:03.585 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.585 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.585 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:03.585 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686633+0000", 2024-09-21T21:43:03.586 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00034428699999999998 2024-09-21T21:43:03.586 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.586 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.586 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:03.586 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688370+0000", 2024-09-21T21:43:03.586 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001736741 2024-09-21T21:43:03.586 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.586 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.586 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:03.586 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.689295+0000", 2024-09-21T21:43:03.586 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00092517900000000004 2024-09-21T21:43:03.586 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.586 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.586 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:03.586 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.689354+0000", 2024-09-21T21:43:03.587 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8799999999999999e-05 2024-09-21T21:43:03.587 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.587 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.587 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:03.587 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.689391+0000", 2024-09-21T21:43:03.587 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7735999999999999e-05 2024-09-21T21:43:03.587 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:03.587 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:03.587 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:03.587 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.587 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.587 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24846 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:03.587 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586564+0000", 2024-09-21T21:43:03.587 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 40.907603495000004, 2024-09-21T21:43:03.587 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.10375692, 2024-09-21T21:43:03.588 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:03.588 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:03.588 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:03.588 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:03.588 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:03.588 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:03.588 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24846 2024-09-21T21:43:03.588 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.588 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:03.588 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.588 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:03.588 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586564+0000", 2024-09-21T21:43:03.588 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:03.588 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.589 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.589 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:03.589 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586564+0000", 2024-09-21T21:43:03.589 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:03.589 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.589 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.589 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:03.589 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586564+0000", 2024-09-21T21:43:03.589 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4799999999999998e-07 2024-09-21T21:43:03.589 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.589 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.589 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:03.589 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586567+0000", 2024-09-21T21:43:03.589 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1669999999999999e-06 2024-09-21T21:43:03.589 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.590 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.590 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:03.590 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586578+0000", 2024-09-21T21:43:03.590 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0443e-05 2024-09-21T21:43:03.590 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.590 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.590 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:03.590 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586583+0000", 2024-09-21T21:43:03.590 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6790000000000002e-06 2024-09-21T21:43:03.590 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.590 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.590 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:03.590 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.687215+0000", 2024-09-21T21:43:03.590 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.100631973 2024-09-21T21:43:03.590 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.591 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.591 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:03.591 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.687311+0000", 2024-09-21T21:43:03.591 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5764e-05 2024-09-21T21:43:03.591 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.591 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.591 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:03.591 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.687663+0000", 2024-09-21T21:43:03.591 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00035167 2024-09-21T21:43:03.591 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.591 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.591 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:03.591 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690236+0000", 2024-09-21T21:43:03.591 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025733370000000002 2024-09-21T21:43:03.592 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.592 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.592 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:03.592 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690250+0000", 2024-09-21T21:43:03.592 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3548e-05 2024-09-21T21:43:03.592 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.592 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.592 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:03.592 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690298+0000", 2024-09-21T21:43:03.592 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7948999999999998e-05 2024-09-21T21:43:03.592 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.592 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.592 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:03.592 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690321+0000", 2024-09-21T21:43:03.592 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2841999999999999e-05 2024-09-21T21:43:03.593 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:03.593 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:03.593 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:03.593 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.593 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.593 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24849 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:03.593 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586633+0000", 2024-09-21T21:43:03.593 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 40.907533829000002, 2024-09-21T21:43:03.593 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.104506548, 2024-09-21T21:43:03.593 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:03.593 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:03.593 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:03.594 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:03.594 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:03.594 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:03.594 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24849 2024-09-21T21:43:03.594 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.594 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:03.594 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.594 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:03.594 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586633+0000", 2024-09-21T21:43:03.594 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:03.594 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.595 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.595 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:03.595 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586633+0000", 2024-09-21T21:43:03.595 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:03.595 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.595 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.595 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:03.595 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586634+0000", 2024-09-21T21:43:03.595 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9100000000000004e-07 2024-09-21T21:43:03.595 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.595 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.595 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:03.595 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586637+0000", 2024-09-21T21:43:03.595 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1999999999999999e-06 2024-09-21T21:43:03.596 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.596 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.596 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:03.596 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586645+0000", 2024-09-21T21:43:03.596 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7759999999999997e-06 2024-09-21T21:43:03.596 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.596 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.596 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:03.596 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586652+0000", 2024-09-21T21:43:03.596 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3560000000000004e-06 2024-09-21T21:43:03.596 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.596 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.596 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:03.596 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688395+0000", 2024-09-21T21:43:03.596 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.101743239 2024-09-21T21:43:03.597 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.597 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.597 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:03.597 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688464+0000", 2024-09-21T21:43:03.597 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8273000000000002e-05 2024-09-21T21:43:03.597 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.597 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.597 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:03.597 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688786+0000", 2024-09-21T21:43:03.597 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000322064 2024-09-21T21:43:03.597 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.597 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.597 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:03.597 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690364+0000", 2024-09-21T21:43:03.597 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001578084 2024-09-21T21:43:03.598 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.598 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.598 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:03.598 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.691050+0000", 2024-09-21T21:43:03.598 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00068623399999999998 2024-09-21T21:43:03.598 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.598 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.598 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:03.598 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.691090+0000", 2024-09-21T21:43:03.598 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9406999999999997e-05 2024-09-21T21:43:03.598 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.598 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.598 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:03.598 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.691140+0000", 2024-09-21T21:43:03.598 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0324000000000001e-05 2024-09-21T21:43:03.599 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:03.599 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:03.599 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:03.599 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.599 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.599 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24851 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:03.599 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586661+0000", 2024-09-21T21:43:03.599 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 40.907506542999997, 2024-09-21T21:43:03.599 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.10536891399999999, 2024-09-21T21:43:03.599 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:03.599 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:03.599 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:03.599 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:03.599 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:03.599 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:03.600 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24851 2024-09-21T21:43:03.600 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.600 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:03.600 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.600 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:03.600 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586661+0000", 2024-09-21T21:43:03.600 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:03.600 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.600 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.600 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:03.600 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586661+0000", 2024-09-21T21:43:03.600 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:03.600 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.600 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.600 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:03.601 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586661+0000", 2024-09-21T21:43:03.601 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4700000000000002e-07 2024-09-21T21:43:03.601 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.601 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.601 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:03.601 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586664+0000", 2024-09-21T21:43:03.601 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.672e-06 2024-09-21T21:43:03.601 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.601 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.601 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:03.601 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586671+0000", 2024-09-21T21:43:03.601 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9959999999999996e-06 2024-09-21T21:43:03.601 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.601 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.602 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:03.602 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586676+0000", 2024-09-21T21:43:03.602 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4020000000000001e-06 2024-09-21T21:43:03.602 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.602 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.602 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:03.602 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.689438+0000", 2024-09-21T21:43:03.602 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.102761503 2024-09-21T21:43:03.602 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.602 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.602 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:03.602 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.689524+0000", 2024-09-21T21:43:03.602 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6383000000000003e-05 2024-09-21T21:43:03.602 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.603 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.603 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:03.603 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.689769+0000", 2024-09-21T21:43:03.603 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00024504699999999997 2024-09-21T21:43:03.603 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.603 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.603 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:03.603 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.691936+0000", 2024-09-21T21:43:03.603 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021665130000000001 2024-09-21T21:43:03.603 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.603 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.603 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:03.603 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.691947+0000", 2024-09-21T21:43:03.603 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1198e-05 2024-09-21T21:43:03.603 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.604 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.604 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:03.604 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.691992+0000", 2024-09-21T21:43:03.604 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4774000000000002e-05 2024-09-21T21:43:03.604 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.604 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.604 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:03.604 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.692030+0000", 2024-09-21T21:43:03.604 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7978999999999997e-05 2024-09-21T21:43:03.604 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:03.604 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:03.604 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:03.604 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.604 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.604 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24853 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:03.605 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586684+0000", 2024-09-21T21:43:03.605 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 40.907483585999998, 2024-09-21T21:43:03.605 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.105936852, 2024-09-21T21:43:03.605 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:03.605 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:03.605 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:03.605 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:03.605 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:03.605 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:03.605 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24853 2024-09-21T21:43:03.605 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.605 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:03.605 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.605 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:03.605 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586684+0000", 2024-09-21T21:43:03.606 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:03.606 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.606 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.606 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:03.606 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586684+0000", 2024-09-21T21:43:03.606 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:03.606 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.606 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.606 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:03.606 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586684+0000", 2024-09-21T21:43:03.606 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.68e-07 2024-09-21T21:43:03.606 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.606 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.606 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:03.606 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586687+0000", 2024-09-21T21:43:03.607 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.774e-06 2024-09-21T21:43:03.607 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.607 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.607 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:03.607 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586694+0000", 2024-09-21T21:43:03.607 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6959999999999999e-06 2024-09-21T21:43:03.607 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.607 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.607 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:03.607 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586698+0000", 2024-09-21T21:43:03.607 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7489999999999998e-06 2024-09-21T21:43:03.607 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.607 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.607 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:03.607 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690395+0000", 2024-09-21T21:43:03.608 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.103696645 2024-09-21T21:43:03.608 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.608 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.608 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:03.608 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690440+0000", 2024-09-21T21:43:03.608 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4835e-05 2024-09-21T21:43:03.608 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.608 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.608 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:03.608 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690654+0000", 2024-09-21T21:43:03.608 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000214114 2024-09-21T21:43:03.608 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.608 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.608 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:03.608 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.692041+0000", 2024-09-21T21:43:03.609 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0013866519999999999 2024-09-21T21:43:03.609 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.609 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.609 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:03.609 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.692558+0000", 2024-09-21T21:43:03.609 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00051717199999999999 2024-09-21T21:43:03.609 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.609 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.609 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:03.609 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.692586+0000", 2024-09-21T21:43:03.609 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8504e-05 2024-09-21T21:43:03.609 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:03.609 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:03.609 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:03.609 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.692620+0000", 2024-09-21T21:43:03.610 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4143000000000002e-05 2024-09-21T21:43:03.610 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:03.610 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:03.610 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:03.610 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:03.610 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:03.610 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:43:03.611 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:43:03.610+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:43:03.611 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T21:43:03.712 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:43:03.711+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:43:03.717 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:43:03.717 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:43:03.717 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 0 2024-09-21T21:43:03.718 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:43:03.726 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T21:43:03.814 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:43:03.812+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:43:03.827 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:43:03.827 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:43:03.827 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:43:03.828 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:43:03.828 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:43:03.836 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T21:43:03.915 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:43:03.913+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:43:03.940 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:43:03.940 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 20, 2024-09-21T21:43:03.940 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 600, 2024-09-21T21:43:03.941 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:43:03.941 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.941 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11319 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 5492736~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:03.941 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.712385+0000", 2024-09-21T21:43:03.941 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 73.225869415999995, 2024-09-21T21:43:03.941 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078627889000000006, 2024-09-21T21:43:03.941 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:03.941 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:03.941 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:03.941 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:03.942 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:03.942 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:03.942 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11319 2024-09-21T21:43:03.942 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.942 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:03.942 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.942 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:03.942 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712385+0000", 2024-09-21T21:43:03.942 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:03.942 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.942 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.942 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:03.942 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712385+0000", 2024-09-21T21:43:03.942 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:03.943 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.943 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.943 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:03.943 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712385+0000", 2024-09-21T21:43:03.943 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2800000000000002e-07 2024-09-21T21:43:03.943 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.943 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.943 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:03.943 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712387+0000", 2024-09-21T21:43:03.943 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1569999999999998e-06 2024-09-21T21:43:03.943 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.943 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.943 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:03.943 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712393+0000", 2024-09-21T21:43:03.943 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9370000000000001e-06 2024-09-21T21:43:03.944 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.944 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.944 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:03.944 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712398+0000", 2024-09-21T21:43:03.944 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5020000000000003e-06 2024-09-21T21:43:03.944 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.944 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.944 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:03.945 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.790746+0000", 2024-09-21T21:43:03.945 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078348589999999996 2024-09-21T21:43:03.945 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.945 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.945 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:03.945 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.790844+0000", 2024-09-21T21:43:03.945 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.7526000000000001e-05 2024-09-21T21:43:03.945 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.945 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.945 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:03.945 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.791013+0000", 2024-09-21T21:43:03.945 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00016874899999999999 2024-09-21T21:43:03.945 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:03.945 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:03.946 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:03.946 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.946 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.946 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11320 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 6180864~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:03.946 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.712405+0000", 2024-09-21T21:43:03.946 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 73.225849456999995, 2024-09-21T21:43:03.946 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078882870999999993, 2024-09-21T21:43:03.946 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:03.946 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:03.946 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:03.946 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:03.946 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:03.946 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:03.947 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11320 2024-09-21T21:43:03.947 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.947 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:03.947 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.947 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:03.947 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712405+0000", 2024-09-21T21:43:03.947 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:03.947 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.947 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.947 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:03.947 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712405+0000", 2024-09-21T21:43:03.947 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:03.947 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.947 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.948 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:03.948 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.713942+0000", 2024-09-21T21:43:03.948 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.001536882 2024-09-21T21:43:03.948 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.948 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.948 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:03.948 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.713946+0000", 2024-09-21T21:43:03.948 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4549999999999997e-06 2024-09-21T21:43:03.948 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.948 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.948 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:03.949 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.713958+0000", 2024-09-21T21:43:03.949 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1775e-05 2024-09-21T21:43:03.949 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.949 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.949 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:03.949 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.713965+0000", 2024-09-21T21:43:03.949 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5100000000000001e-06 2024-09-21T21:43:03.949 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.949 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.949 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:03.949 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.791059+0000", 2024-09-21T21:43:03.949 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.077094036000000005 2024-09-21T21:43:03.949 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.949 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.950 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:03.950 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.791175+0000", 2024-09-21T21:43:03.950 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000115895 2024-09-21T21:43:03.950 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.950 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.950 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:03.950 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.791288+0000", 2024-09-21T21:43:03.950 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000112318 2024-09-21T21:43:03.950 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:03.950 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:03.950 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:03.950 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.950 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.950 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11325 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 5414912~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:03.951 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.714095+0000", 2024-09-21T21:43:03.951 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 73.224159462000003, 2024-09-21T21:43:03.951 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078631894999999993, 2024-09-21T21:43:03.951 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:03.951 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:03.952 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:03.952 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:03.952 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:03.952 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:03.952 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11325 2024-09-21T21:43:03.952 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.952 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:03.952 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.952 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:03.952 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714095+0000", 2024-09-21T21:43:03.952 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:03.953 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.953 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.953 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:03.953 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714095+0000", 2024-09-21T21:43:03.953 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:03.953 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.953 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.953 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:03.953 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714095+0000", 2024-09-21T21:43:03.953 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7500000000000001e-07 2024-09-21T21:43:03.953 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.953 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.954 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:03.954 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714097+0000", 2024-09-21T21:43:03.954 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.12e-06 2024-09-21T21:43:03.954 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.954 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.954 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:03.954 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714104+0000", 2024-09-21T21:43:03.954 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3520000000000003e-06 2024-09-21T21:43:03.954 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.954 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.954 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:03.954 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714108+0000", 2024-09-21T21:43:03.954 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8810000000000004e-06 2024-09-21T21:43:03.955 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.955 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.955 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:03.955 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792509+0000", 2024-09-21T21:43:03.955 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078400739999999997 2024-09-21T21:43:03.955 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.956 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.956 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:03.956 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792613+0000", 2024-09-21T21:43:03.956 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000103939 2024-09-21T21:43:03.956 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.956 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.956 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:03.957 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792727+0000", 2024-09-21T21:43:03.957 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011348799999999999 2024-09-21T21:43:03.957 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:03.957 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:03.957 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:03.957 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.957 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.957 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11326 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 6103040~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:03.958 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.714115+0000", 2024-09-21T21:43:03.958 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 73.224139433999994, 2024-09-21T21:43:03.958 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078963056000000004, 2024-09-21T21:43:03.958 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:03.958 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:03.958 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:03.958 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:03.958 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:03.959 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:03.959 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11326 2024-09-21T21:43:03.959 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.959 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:03.959 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.959 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:03.959 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714115+0000", 2024-09-21T21:43:03.960 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:03.960 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.960 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.960 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:03.960 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714115+0000", 2024-09-21T21:43:03.960 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:03.960 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.961 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.961 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:03.961 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.715940+0000", 2024-09-21T21:43:03.961 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0018254969999999999 2024-09-21T21:43:03.961 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.961 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.961 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:03.962 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.715945+0000", 2024-09-21T21:43:03.962 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6120000000000001e-06 2024-09-21T21:43:03.962 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.962 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.962 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:03.962 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.715954+0000", 2024-09-21T21:43:03.962 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.5899999999999997e-06 2024-09-21T21:43:03.962 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.963 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.963 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:03.963 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.715959+0000", 2024-09-21T21:43:03.963 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0849999999999996e-06 2024-09-21T21:43:03.963 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.963 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.963 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:03.963 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792771+0000", 2024-09-21T21:43:03.963 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.076811829999999998 2024-09-21T21:43:03.963 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.963 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.963 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:03.963 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792882+0000", 2024-09-21T21:43:03.963 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000110857 2024-09-21T21:43:03.964 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.964 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.964 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:03.964 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.793078+0000", 2024-09-21T21:43:03.964 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00019558499999999999 2024-09-21T21:43:03.964 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:03.964 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:03.964 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:03.964 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.964 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.964 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11332 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 7565312~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:03.964 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.716063+0000", 2024-09-21T21:43:03.964 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 73.222190900000001, 2024-09-21T21:43:03.964 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078787482000000006, 2024-09-21T21:43:03.964 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:03.965 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:03.965 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:03.965 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:03.965 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:03.965 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:03.965 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11332 2024-09-21T21:43:03.965 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.965 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:03.965 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.965 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:03.965 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.716063+0000", 2024-09-21T21:43:03.965 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:03.965 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.966 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.966 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:03.966 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.716063+0000", 2024-09-21T21:43:03.966 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:03.966 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.966 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.966 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:03.966 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.717944+0000", 2024-09-21T21:43:03.966 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0018812309999999999 2024-09-21T21:43:03.966 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.966 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.966 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:03.966 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.717949+0000", 2024-09-21T21:43:03.966 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9389999999999996e-06 2024-09-21T21:43:03.967 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.967 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.967 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:03.967 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.717963+0000", 2024-09-21T21:43:03.967 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.3899e-05 2024-09-21T21:43:03.967 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.967 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.967 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:03.967 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.717970+0000", 2024-09-21T21:43:03.967 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.426e-06 2024-09-21T21:43:03.967 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.967 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.967 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:03.967 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.794573+0000", 2024-09-21T21:43:03.968 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.076602990999999995 2024-09-21T21:43:03.968 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.968 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.968 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:03.968 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.794682+0000", 2024-09-21T21:43:03.968 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000109541 2024-09-21T21:43:03.968 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.968 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.968 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:03.968 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.794851+0000", 2024-09-21T21:43:03.968 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000168455 2024-09-21T21:43:03.968 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:03.968 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:03.968 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:03.968 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.969 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.969 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11339 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 6266880~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:03.969 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.718117+0000", 2024-09-21T21:43:03.969 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 73.220136874000005, 2024-09-21T21:43:03.969 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078703717000000006, 2024-09-21T21:43:03.969 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:03.969 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:03.969 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:03.969 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:03.969 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:03.969 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:03.969 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11339 2024-09-21T21:43:03.969 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.969 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:03.969 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.970 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:03.970 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.718117+0000", 2024-09-21T21:43:03.970 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:03.970 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.970 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.970 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:03.970 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.718117+0000", 2024-09-21T21:43:03.970 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:03.970 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.970 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.970 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:03.970 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720019+0000", 2024-09-21T21:43:03.970 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0019012549999999999 2024-09-21T21:43:03.971 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.971 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.971 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:03.971 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720024+0000", 2024-09-21T21:43:03.971 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1100000000000002e-06 2024-09-21T21:43:03.971 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.971 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.971 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:03.971 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720035+0000", 2024-09-21T21:43:03.971 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.094e-05 2024-09-21T21:43:03.971 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.971 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.971 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:03.971 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720040+0000", 2024-09-21T21:43:03.971 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1830000000000002e-06 2024-09-21T21:43:03.972 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.972 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.972 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:03.972 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796618+0000", 2024-09-21T21:43:03.972 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.076578033000000004 2024-09-21T21:43:03.972 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.972 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.972 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:03.972 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796696+0000", 2024-09-21T21:43:03.972 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7836000000000005e-05 2024-09-21T21:43:03.972 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.972 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.972 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:03.973 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796821+0000", 2024-09-21T21:43:03.973 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00012536000000000001 2024-09-21T21:43:03.973 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:03.973 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:03.973 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:03.973 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.973 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.973 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11345 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 5472256~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:03.973 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.720116+0000", 2024-09-21T21:43:03.973 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 73.218137974000001, 2024-09-21T21:43:03.973 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078667730000000005, 2024-09-21T21:43:03.974 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:03.974 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:03.974 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:03.974 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:03.974 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:03.974 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:03.974 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11345 2024-09-21T21:43:03.974 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.975 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:03.975 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.975 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:03.975 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720116+0000", 2024-09-21T21:43:03.975 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:03.975 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.975 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.976 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:03.976 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720116+0000", 2024-09-21T21:43:03.976 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:03.976 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.976 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.976 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:03.976 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720116+0000", 2024-09-21T21:43:03.977 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.04e-07 2024-09-21T21:43:03.977 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.977 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.977 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:03.977 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720118+0000", 2024-09-21T21:43:03.977 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.181e-06 2024-09-21T21:43:03.977 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.977 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.977 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:03.978 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720121+0000", 2024-09-21T21:43:03.978 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0199999999999999e-06 2024-09-21T21:43:03.978 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.978 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.978 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:03.978 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720123+0000", 2024-09-21T21:43:03.978 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.5469999999999999e-06 2024-09-21T21:43:03.979 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.979 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.979 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:03.979 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798626+0000", 2024-09-21T21:43:03.979 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078503137000000001 2024-09-21T21:43:03.979 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.979 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.980 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:03.980 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798677+0000", 2024-09-21T21:43:03.980 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0930999999999997e-05 2024-09-21T21:43:03.980 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.980 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.980 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:03.980 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798784+0000", 2024-09-21T21:43:03.981 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010671 2024-09-21T21:43:03.981 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:03.981 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:03.981 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:03.981 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.981 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.981 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11346 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 5500928~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:03.981 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.720126+0000", 2024-09-21T21:43:03.982 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 73.218127901000003, 2024-09-21T21:43:03.982 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078863421000000003, 2024-09-21T21:43:03.982 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:03.982 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:03.982 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:03.982 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:03.982 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:03.982 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:03.982 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11346 2024-09-21T21:43:03.982 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.982 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:03.982 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.982 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:03.982 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720126+0000", 2024-09-21T21:43:03.983 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:03.983 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.983 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.983 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:03.983 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720126+0000", 2024-09-21T21:43:03.983 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:03.983 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.983 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.983 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:03.984 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.722145+0000", 2024-09-21T21:43:03.984 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0020183219999999999 2024-09-21T21:43:03.984 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.984 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.984 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:03.984 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.722150+0000", 2024-09-21T21:43:03.984 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1760000000000001e-06 2024-09-21T21:43:03.984 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.984 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.984 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:03.984 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.722161+0000", 2024-09-21T21:43:03.984 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1566000000000001e-05 2024-09-21T21:43:03.984 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.984 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.984 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:03.985 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.722169+0000", 2024-09-21T21:43:03.985 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.2840000000000001e-06 2024-09-21T21:43:03.985 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.985 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.985 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:03.985 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798815+0000", 2024-09-21T21:43:03.985 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.076646607000000005 2024-09-21T21:43:03.985 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.985 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.985 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:03.985 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798870+0000", 2024-09-21T21:43:03.985 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4486e-05 2024-09-21T21:43:03.985 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.985 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.986 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:03.986 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798990+0000", 2024-09-21T21:43:03.986 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011998 2024-09-21T21:43:03.986 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:03.986 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:03.986 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:03.986 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.986 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.986 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13629 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7852032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:03.986 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943319+0000", 2024-09-21T21:43:03.986 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 72.994935259000002, 2024-09-21T21:43:03.986 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078610975999999999, 2024-09-21T21:43:03.986 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:03.986 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:03.987 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:03.987 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:03.987 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:03.987 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:03.987 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13629 2024-09-21T21:43:03.987 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.987 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:03.987 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.987 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:03.987 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943319+0000", 2024-09-21T21:43:03.987 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:03.987 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.987 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.988 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:03.988 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943319+0000", 2024-09-21T21:43:03.988 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:03.988 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.988 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.988 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:03.988 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943319+0000", 2024-09-21T21:43:03.988 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6199999999999999e-07 2024-09-21T21:43:03.988 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.988 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.988 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:03.988 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943321+0000", 2024-09-21T21:43:03.988 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1009999999999999e-06 2024-09-21T21:43:03.989 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.989 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.989 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:03.989 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943328+0000", 2024-09-21T21:43:03.989 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.1029999999999996e-06 2024-09-21T21:43:03.989 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.989 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.989 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:03.989 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943333+0000", 2024-09-21T21:43:03.989 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7709999999999997e-06 2024-09-21T21:43:03.989 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.989 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.989 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:03.989 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.021679+0000", 2024-09-21T21:43:03.990 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078345673000000005 2024-09-21T21:43:03.990 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.990 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.990 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:03.990 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.021771+0000", 2024-09-21T21:43:03.990 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2201999999999996e-05 2024-09-21T21:43:03.990 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.990 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.990 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:03.990 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.021930+0000", 2024-09-21T21:43:03.990 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00015876400000000001 2024-09-21T21:43:03.990 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:03.990 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:03.990 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:03.991 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.991 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.991 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13630 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7880704~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:03.991 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943339+0000", 2024-09-21T21:43:03.991 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 72.994915070999994, 2024-09-21T21:43:03.991 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078939410000000002, 2024-09-21T21:43:03.991 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:03.991 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:03.991 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:03.991 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:03.991 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:03.991 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:03.991 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13630 2024-09-21T21:43:03.991 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.992 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:03.992 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.992 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:03.992 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943339+0000", 2024-09-21T21:43:03.992 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:03.992 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.992 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.992 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:03.992 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943339+0000", 2024-09-21T21:43:03.992 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:03.992 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.992 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.992 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:03.992 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943339+0000", 2024-09-21T21:43:03.992 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8000000000000001e-07 2024-09-21T21:43:03.993 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.993 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.993 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:03.993 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943341+0000", 2024-09-21T21:43:03.993 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9850000000000001e-06 2024-09-21T21:43:03.993 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.993 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.993 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:03.993 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943347+0000", 2024-09-21T21:43:03.993 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2979999999999999e-06 2024-09-21T21:43:03.993 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.993 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.993 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:03.994 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943354+0000", 2024-09-21T21:43:03.994 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.2969999999999998e-06 2024-09-21T21:43:03.994 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.994 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.994 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:03.994 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.021996+0000", 2024-09-21T21:43:03.994 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078641680000000005 2024-09-21T21:43:03.994 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.994 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.994 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:03.995 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022104+0000", 2024-09-21T21:43:03.995 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000107902 2024-09-21T21:43:03.995 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.995 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.995 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:03.995 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022278+0000", 2024-09-21T21:43:03.995 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00017486800000000001 2024-09-21T21:43:03.995 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:03.995 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:03.995 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:03.995 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.995 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.995 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13631 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7909376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:03.995 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943362+0000", 2024-09-21T21:43:03.996 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 72.994892250999996, 2024-09-21T21:43:03.996 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079195644999999995, 2024-09-21T21:43:03.996 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:03.996 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:03.996 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:03.996 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:03.996 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:03.996 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:03.996 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13631 2024-09-21T21:43:03.996 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.996 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:03.996 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.996 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:03.996 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943362+0000", 2024-09-21T21:43:03.996 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:03.997 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.997 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.997 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:03.997 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943362+0000", 2024-09-21T21:43:03.997 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:03.997 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.997 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.997 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:03.997 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943362+0000", 2024-09-21T21:43:03.997 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8799999999999998e-07 2024-09-21T21:43:03.997 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.997 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.997 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:03.998 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943364+0000", 2024-09-21T21:43:03.998 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9850000000000001e-06 2024-09-21T21:43:03.998 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.998 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.998 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:03.998 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943371+0000", 2024-09-21T21:43:03.998 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0650000000000001e-06 2024-09-21T21:43:03.998 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.998 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.998 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:03.999 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943376+0000", 2024-09-21T21:43:03.999 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8250000000000004e-06 2024-09-21T21:43:03.999 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.999 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.999 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:03.999 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022329+0000", 2024-09-21T21:43:03.999 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078952888999999998 2024-09-21T21:43:03.999 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.999 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:03.999 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:03.999 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022419+0000", 2024-09-21T21:43:03.999 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.9738e-05 2024-09-21T21:43:03.999 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:03.999 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.000 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:04.000 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022558+0000", 2024-09-21T21:43:04.000 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00013875500000000001 2024-09-21T21:43:04.000 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:04.000 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:04.000 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:04.001 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.001 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.001 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13632 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7966720~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:04.001 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943382+0000", 2024-09-21T21:43:04.001 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 72.994872017000006, 2024-09-21T21:43:04.001 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079495727000000002, 2024-09-21T21:43:04.001 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:04.001 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:04.002 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:04.002 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:04.002 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:04.002 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:04.002 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13632 2024-09-21T21:43:04.002 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.002 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:04.003 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.003 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:04.003 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943382+0000", 2024-09-21T21:43:04.003 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:04.003 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.003 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.003 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:04.003 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943382+0000", 2024-09-21T21:43:04.003 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:04.004 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.004 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.004 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:04.004 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943382+0000", 2024-09-21T21:43:04.004 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7e-07 2024-09-21T21:43:04.004 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.004 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.005 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:04.005 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943385+0000", 2024-09-21T21:43:04.005 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2230000000000001e-06 2024-09-21T21:43:04.005 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.005 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.005 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:04.005 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943391+0000", 2024-09-21T21:43:04.006 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6810000000000001e-06 2024-09-21T21:43:04.006 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.006 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.006 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:04.006 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943397+0000", 2024-09-21T21:43:04.006 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6559999999999997e-06 2024-09-21T21:43:04.007 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.007 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.007 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:04.007 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022601+0000", 2024-09-21T21:43:04.007 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079204051999999997 2024-09-21T21:43:04.007 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.007 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.007 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:04.007 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022704+0000", 2024-09-21T21:43:04.007 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010329599999999999 2024-09-21T21:43:04.008 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.008 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.008 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:04.008 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022878+0000", 2024-09-21T21:43:04.008 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000173449 2024-09-21T21:43:04.008 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:04.008 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:04.008 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:04.008 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.008 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.008 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13633 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7938048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:04.008 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943403+0000", 2024-09-21T21:43:04.008 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 72.994851087000001, 2024-09-21T21:43:04.009 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079794697999999997, 2024-09-21T21:43:04.009 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:04.009 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:04.009 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:04.009 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:04.009 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:04.009 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:04.009 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13633 2024-09-21T21:43:04.009 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.009 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:04.009 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.009 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:04.009 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943403+0000", 2024-09-21T21:43:04.009 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:04.010 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.010 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.010 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:04.010 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943403+0000", 2024-09-21T21:43:04.010 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:04.010 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.010 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.010 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:04.010 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943403+0000", 2024-09-21T21:43:04.010 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.65e-07 2024-09-21T21:43:04.010 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.010 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.010 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:04.010 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943405+0000", 2024-09-21T21:43:04.011 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0020000000000001e-06 2024-09-21T21:43:04.011 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.011 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.011 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:04.011 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943412+0000", 2024-09-21T21:43:04.011 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.8379999999999996e-06 2024-09-21T21:43:04.011 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.011 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.011 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:04.011 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943417+0000", 2024-09-21T21:43:04.011 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7650000000000001e-06 2024-09-21T21:43:04.011 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.011 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.011 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:04.012 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022922+0000", 2024-09-21T21:43:04.012 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079504512999999999 2024-09-21T21:43:04.012 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.012 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.012 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:04.012 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023039+0000", 2024-09-21T21:43:04.012 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000117044 2024-09-21T21:43:04.012 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.012 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.012 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:04.012 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023198+0000", 2024-09-21T21:43:04.012 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000159171 2024-09-21T21:43:04.012 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:04.012 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:04.013 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:04.013 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.013 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.013 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13634 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7995392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:04.013 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943423+0000", 2024-09-21T21:43:04.013 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 72.994831379000004, 2024-09-21T21:43:04.013 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.080142358999999996, 2024-09-21T21:43:04.013 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:04.013 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:04.013 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:04.013 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:04.013 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:04.013 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:04.013 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13634 2024-09-21T21:43:04.013 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.014 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:04.014 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.014 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:04.014 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943423+0000", 2024-09-21T21:43:04.014 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:04.014 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.014 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.014 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:04.014 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943423+0000", 2024-09-21T21:43:04.014 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:04.014 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.014 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.014 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:04.015 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943423+0000", 2024-09-21T21:43:04.015 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.58e-07 2024-09-21T21:43:04.015 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.015 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.015 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:04.015 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943425+0000", 2024-09-21T21:43:04.015 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9180000000000001e-06 2024-09-21T21:43:04.015 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.015 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.015 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:04.016 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943431+0000", 2024-09-21T21:43:04.016 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4439999999999996e-06 2024-09-21T21:43:04.016 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.016 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.016 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:04.016 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943436+0000", 2024-09-21T21:43:04.016 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.887e-06 2024-09-21T21:43:04.016 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.016 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.016 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:04.016 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023266+0000", 2024-09-21T21:43:04.016 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079829622000000003 2024-09-21T21:43:04.016 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.016 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.017 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:04.017 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023384+0000", 2024-09-21T21:43:04.017 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000117658 2024-09-21T21:43:04.017 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.017 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.017 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:04.017 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023565+0000", 2024-09-21T21:43:04.017 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00018147200000000001 2024-09-21T21:43:04.017 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:04.017 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:04.017 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:04.017 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.017 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.017 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13635 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 8024064~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:04.018 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943447+0000", 2024-09-21T21:43:04.018 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 72.994806839000006, 2024-09-21T21:43:04.018 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.080417653000000006, 2024-09-21T21:43:04.018 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:04.018 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:04.018 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:04.018 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:04.018 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:04.018 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:04.018 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13635 2024-09-21T21:43:04.018 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.018 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:04.018 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.018 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:04.019 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943447+0000", 2024-09-21T21:43:04.019 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:04.019 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.019 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.019 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:04.019 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943447+0000", 2024-09-21T21:43:04.019 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:04.019 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.019 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.019 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:04.019 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943448+0000", 2024-09-21T21:43:04.019 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1699999999999999e-07 2024-09-21T21:43:04.019 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.019 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.019 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:04.020 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943450+0000", 2024-09-21T21:43:04.020 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1639999999999999e-06 2024-09-21T21:43:04.020 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.020 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.020 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:04.020 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943456+0000", 2024-09-21T21:43:04.020 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8379999999999998e-06 2024-09-21T21:43:04.020 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.020 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.020 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:04.020 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943460+0000", 2024-09-21T21:43:04.020 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4700000000000004e-06 2024-09-21T21:43:04.020 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.020 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.021 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:04.021 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023595+0000", 2024-09-21T21:43:04.021 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.080134869999999997 2024-09-21T21:43:04.021 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.021 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.021 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:04.021 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023688+0000", 2024-09-21T21:43:04.021 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2794000000000006e-05 2024-09-21T21:43:04.021 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.021 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.021 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:04.021 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023865+0000", 2024-09-21T21:43:04.021 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0001771 2024-09-21T21:43:04.021 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:04.021 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:04.022 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:04.022 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.022 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.022 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13636 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 8052736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:04.022 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943467+0000", 2024-09-21T21:43:04.022 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 72.994787102000004, 2024-09-21T21:43:04.022 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.080694399999999999, 2024-09-21T21:43:04.022 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:04.022 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:04.022 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:04.022 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:04.022 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:04.022 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:04.023 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13636 2024-09-21T21:43:04.023 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.023 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:04.023 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.023 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:04.023 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943467+0000", 2024-09-21T21:43:04.023 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:04.023 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.023 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.023 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:04.023 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943467+0000", 2024-09-21T21:43:04.023 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:04.023 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.023 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.023 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:04.024 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944940+0000", 2024-09-21T21:43:04.024 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0014725109999999999 2024-09-21T21:43:04.024 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.024 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.024 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:04.024 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944949+0000", 2024-09-21T21:43:04.024 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2930000000000006e-06 2024-09-21T21:43:04.024 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.024 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.024 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:04.024 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944958+0000", 2024-09-21T21:43:04.024 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.6899999999999998e-06 2024-09-21T21:43:04.024 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.024 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.025 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:04.025 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944966+0000", 2024-09-21T21:43:04.025 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0690000000000002e-06 2024-09-21T21:43:04.025 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.025 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.025 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:04.025 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023900+0000", 2024-09-21T21:43:04.025 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078934371000000003 2024-09-21T21:43:04.025 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.025 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.025 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:04.025 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024009+0000", 2024-09-21T21:43:04.026 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000108857 2024-09-21T21:43:04.026 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.026 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.026 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:04.026 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024161+0000", 2024-09-21T21:43:04.026 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000152609 2024-09-21T21:43:04.026 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:04.026 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:04.026 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:04.026 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.026 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.027 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13637 3.12 3.1dc31b72 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:04.027 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.944978+0000", 2024-09-21T21:43:04.027 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 72.993276167000005, 2024-09-21T21:43:04.027 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079232604999999998, 2024-09-21T21:43:04.027 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:04.027 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:04.027 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:43:04.027 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:04.027 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:04.027 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:04.027 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13637 2024-09-21T21:43:04.027 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.027 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:04.027 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.028 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:04.028 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944978+0000", 2024-09-21T21:43:04.028 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:04.028 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.028 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.028 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:04.028 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944978+0000", 2024-09-21T21:43:04.028 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:04.028 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.028 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.028 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:04.028 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944978+0000", 2024-09-21T21:43:04.028 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9100000000000004e-07 2024-09-21T21:43:04.028 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.028 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.029 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:04.029 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944981+0000", 2024-09-21T21:43:04.029 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3290000000000001e-06 2024-09-21T21:43:04.029 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.029 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.029 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:04.029 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944987+0000", 2024-09-21T21:43:04.029 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.28e-06 2024-09-21T21:43:04.029 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.029 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.029 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:04.029 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944992+0000", 2024-09-21T21:43:04.029 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8520000000000003e-06 2024-09-21T21:43:04.029 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.030 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.030 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:04.030 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024196+0000", 2024-09-21T21:43:04.030 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079203673000000002 2024-09-21T21:43:04.030 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.030 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.030 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:04.030 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024211+0000", 2024-09-21T21:43:04.030 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.4980000000000001e-05 2024-09-21T21:43:04.030 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:04.030 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:04.030 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:04.030 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.030 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.031 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13638 3.12 3.1dc31b72 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:04.031 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.945000+0000", 2024-09-21T21:43:04.031 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 72.993254038999993, 2024-09-21T21:43:04.031 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079257465999999999, 2024-09-21T21:43:04.031 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:04.031 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:04.031 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:43:04.031 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:04.031 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:04.032 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:04.032 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13638 2024-09-21T21:43:04.032 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.032 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:04.032 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.032 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:04.032 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945000+0000", 2024-09-21T21:43:04.033 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:04.033 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.033 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.033 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:04.033 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945000+0000", 2024-09-21T21:43:04.033 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:04.033 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.034 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.034 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:04.034 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945001+0000", 2024-09-21T21:43:04.034 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.32e-07 2024-09-21T21:43:04.034 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.034 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.034 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:04.034 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945003+0000", 2024-09-21T21:43:04.035 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2309999999999999e-06 2024-09-21T21:43:04.035 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.035 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.035 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:04.035 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945009+0000", 2024-09-21T21:43:04.035 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7450000000000001e-06 2024-09-21T21:43:04.035 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.035 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.036 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:04.036 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945013+0000", 2024-09-21T21:43:04.036 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.673e-06 2024-09-21T21:43:04.036 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.036 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.036 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:04.036 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024245+0000", 2024-09-21T21:43:04.037 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079232160999999995 2024-09-21T21:43:04.037 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.037 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.037 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:04.037 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024258+0000", 2024-09-21T21:43:04.037 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.2224e-05 2024-09-21T21:43:04.037 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:04.038 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:04.038 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:04.038 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.038 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.038 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13639 3.12 3.1dc31b72 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:04.038 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.945034+0000", 2024-09-21T21:43:04.038 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 72.993220239999999, 2024-09-21T21:43:04.039 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079260627, 2024-09-21T21:43:04.039 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:04.039 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:04.039 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:43:04.039 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:04.039 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:04.039 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:04.039 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13639 2024-09-21T21:43:04.039 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.039 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:04.040 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.040 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:04.040 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945034+0000", 2024-09-21T21:43:04.040 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:04.040 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.040 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.040 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:04.040 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945034+0000", 2024-09-21T21:43:04.040 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:04.040 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.040 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.040 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:04.040 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945034+0000", 2024-09-21T21:43:04.040 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3799999999999998e-07 2024-09-21T21:43:04.041 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.041 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.041 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:04.041 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945036+0000", 2024-09-21T21:43:04.041 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.052e-06 2024-09-21T21:43:04.041 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.041 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.041 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:04.041 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945040+0000", 2024-09-21T21:43:04.041 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1169999999999997e-06 2024-09-21T21:43:04.041 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.041 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.041 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:04.041 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945043+0000", 2024-09-21T21:43:04.042 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.8439999999999998e-06 2024-09-21T21:43:04.042 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.042 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.042 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:04.042 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024286+0000", 2024-09-21T21:43:04.042 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079242483000000002 2024-09-21T21:43:04.042 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.042 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.042 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:04.042 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024295+0000", 2024-09-21T21:43:04.042 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.7930000000000003e-06 2024-09-21T21:43:04.042 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:04.042 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:04.042 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:04.043 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.043 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.043 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13640 3.12 3.1dc31b72 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:04.043 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.945047+0000", 2024-09-21T21:43:04.043 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 72.993206717999996, 2024-09-21T21:43:04.043 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079412321999999994, 2024-09-21T21:43:04.043 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:04.043 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:04.043 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:43:04.043 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:04.043 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:04.043 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:04.043 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13640 2024-09-21T21:43:04.043 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.044 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:04.044 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.044 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:04.044 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945047+0000", 2024-09-21T21:43:04.044 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:04.044 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.044 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.044 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:04.044 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945047+0000", 2024-09-21T21:43:04.044 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:04.044 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.044 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.044 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:04.044 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945048+0000", 2024-09-21T21:43:04.045 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0499999999999999e-07 2024-09-21T21:43:04.045 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.045 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.045 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:04.045 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945049+0000", 2024-09-21T21:43:04.045 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.3939999999999999e-06 2024-09-21T21:43:04.045 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.045 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.045 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:04.045 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945053+0000", 2024-09-21T21:43:04.045 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1760000000000003e-06 2024-09-21T21:43:04.045 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.045 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.045 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:04.045 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945056+0000", 2024-09-21T21:43:04.046 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.6129999999999998e-06 2024-09-21T21:43:04.046 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.046 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.046 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:04.046 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024360+0000", 2024-09-21T21:43:04.046 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079303645000000006 2024-09-21T21:43:04.046 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.046 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.046 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:04.046 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024460+0000", 2024-09-21T21:43:04.046 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000100189 2024-09-21T21:43:04.046 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:04.046 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:04.047 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:04.047 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:04.047 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:04.047 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:43:04.048 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T21:43:04.049 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:43:04.047+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:43:04.149 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:43:04.148+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:43:04.193 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:43:04.193 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:43:04.193 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.193 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4385.0:30053 3.6 3.b74753e6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e141)", 2024-09-21T21:43:04.193 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:43:03.223636+0000", 2024-09-21T21:43:04.193 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 0.96840802100000001, 2024-09-21T21:43:04.193 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.96842694600000001, 2024-09-21T21:43:04.193 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:04.193 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:04.193 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "waiting for peered", 2024-09-21T21:43:04.193 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:04.194 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4385", 2024-09-21T21:43:04.194 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:43:04.194 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 30053 2024-09-21T21:43:04.194 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.194 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:04.194 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.194 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:04.194 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:43:03.223636+0000", 2024-09-21T21:43:04.194 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:04.194 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.194 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.194 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:04.194 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:43:03.223636+0000", 2024-09-21T21:43:04.195 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:04.195 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.195 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.195 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:04.195 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:43:03.223638+0000", 2024-09-21T21:43:04.195 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.6080000000000001e-06 2024-09-21T21:43:04.195 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.195 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.195 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:04.195 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:43:03.223643+0000", 2024-09-21T21:43:04.195 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9110000000000001e-06 2024-09-21T21:43:04.195 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.196 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.196 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:04.196 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:43:03.223654+0000", 2024-09-21T21:43:04.196 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1029999999999999e-05 2024-09-21T21:43:04.196 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.196 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.196 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:04.196 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:43:03.223664+0000", 2024-09-21T21:43:04.196 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.9299999999999998e-06 2024-09-21T21:43:04.196 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.196 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.196 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:04.196 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:43:03.223856+0000", 2024-09-21T21:43:04.196 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00019216200000000001 2024-09-21T21:43:04.197 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.197 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.197 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for peered", 2024-09-21T21:43:04.197 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:43:03.223866+0000", 2024-09-21T21:43:04.197 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0356999999999999e-05 2024-09-21T21:43:04.197 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:04.197 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:04.197 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:04.197 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:04.197 INFO:teuthology.orchestra.run.smithi081.stdout: ], 2024-09-21T21:43:04.197 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 1 2024-09-21T21:43:04.197 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:43:04.201 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T21:43:04.250 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:43:04.248+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:43:04.334 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:43:04.335 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:43:04.335 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:43:04.335 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:43:04.335 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:43:04.346 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T21:43:04.387 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:43:04.385+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:43:04.452 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:43:04.451+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:43:04.502 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:43:04.502 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 20, 2024-09-21T21:43:04.502 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 600, 2024-09-21T21:43:04.502 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:43:04.502 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.503 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4824.0:1564 3.2 3:4647520c:::rbd_data.1121b97f2925.0000000000000006:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:43:04.503 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:43:04.503 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 122.863336723, 2024-09-21T21:43:04.503 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056097347999999998, 2024-09-21T21:43:04.503 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:04.503 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:04.503 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:04.503 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:04.503 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4824", 2024-09-21T21:43:04.503 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:43:04.504 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1564 2024-09-21T21:43:04.504 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.504 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:04.504 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.504 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:04.504 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:43:04.504 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:04.505 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.505 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.505 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:04.505 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:43:04.505 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:04.505 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.505 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.505 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:04.505 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:43:04.505 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6499999999999999e-07 2024-09-21T21:43:04.505 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.505 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.505 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:04.506 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636067+0000", 2024-09-21T21:43:04.506 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3879999999999998e-06 2024-09-21T21:43:04.506 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.506 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.506 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:04.506 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636075+0000", 2024-09-21T21:43:04.506 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5480000000000004e-06 2024-09-21T21:43:04.506 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.506 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.506 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:04.507 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636080+0000", 2024-09-21T21:43:04.507 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.339e-06 2024-09-21T21:43:04.507 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.507 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.507 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:04.507 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636133+0000", 2024-09-21T21:43:04.507 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2930000000000003e-05 2024-09-21T21:43:04.507 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.507 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.507 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:04.507 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636265+0000", 2024-09-21T21:43:04.507 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000132058 2024-09-21T21:43:04.507 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.508 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.508 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:04.508 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.692162+0000", 2024-09-21T21:43:04.508 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055896620000000001 2024-09-21T21:43:04.508 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:04.508 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:04.508 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:04.508 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.508 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.509 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3260 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 1437696~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:43:04.509 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.196297+0000", 2024-09-21T21:43:04.509 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 77.303104512999994, 2024-09-21T21:43:04.509 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056071022999999998, 2024-09-21T21:43:04.509 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:04.509 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:04.509 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:04.509 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:04.509 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:04.509 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:04.509 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3260 2024-09-21T21:43:04.509 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.509 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:04.509 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.509 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:04.510 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196297+0000", 2024-09-21T21:43:04.510 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:04.510 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.510 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.510 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:04.510 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196297+0000", 2024-09-21T21:43:04.510 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:04.510 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.510 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.510 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:04.511 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196918+0000", 2024-09-21T21:43:04.511 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00062148699999999999 2024-09-21T21:43:04.511 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.511 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.512 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:04.512 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196930+0000", 2024-09-21T21:43:04.512 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1657e-05 2024-09-21T21:43:04.514 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.514 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.514 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:04.515 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196944+0000", 2024-09-21T21:43:04.515 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.4549000000000001e-05 2024-09-21T21:43:04.515 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.515 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.515 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:04.515 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196951+0000", 2024-09-21T21:43:04.515 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6749999999999996e-06 2024-09-21T21:43:04.515 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.515 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.516 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:04.516 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252208+0000", 2024-09-21T21:43:04.516 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055256656000000001 2024-09-21T21:43:04.516 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.516 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.516 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:04.516 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252278+0000", 2024-09-21T21:43:04.516 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9882000000000006e-05 2024-09-21T21:43:04.516 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.516 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.516 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:04.516 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252368+0000", 2024-09-21T21:43:04.516 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.0117000000000005e-05 2024-09-21T21:43:04.517 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:04.517 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:04.517 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:04.517 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.517 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.517 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3437 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 925696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:43:04.517 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.205888+0000", 2024-09-21T21:43:04.517 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 77.293513418000003, 2024-09-21T21:43:04.517 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055953808000000001, 2024-09-21T21:43:04.518 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:04.518 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:04.518 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:04.518 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:04.518 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:04.518 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:04.518 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3437 2024-09-21T21:43:04.518 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.518 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:04.518 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.518 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:04.518 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205888+0000", 2024-09-21T21:43:04.518 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:04.518 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.519 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.519 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:04.519 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205888+0000", 2024-09-21T21:43:04.519 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:04.519 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.519 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.519 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:04.519 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205888+0000", 2024-09-21T21:43:04.519 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.75e-07 2024-09-21T21:43:04.519 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.519 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.519 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:04.519 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205891+0000", 2024-09-21T21:43:04.519 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3750000000000001e-06 2024-09-21T21:43:04.519 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.520 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.520 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:04.520 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205897+0000", 2024-09-21T21:43:04.520 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.285e-06 2024-09-21T21:43:04.520 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.520 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.520 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:04.520 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205902+0000", 2024-09-21T21:43:04.520 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6290000000000001e-06 2024-09-21T21:43:04.520 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.520 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.520 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:04.520 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261686+0000", 2024-09-21T21:43:04.521 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055784586999999997 2024-09-21T21:43:04.521 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.521 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.521 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:04.521 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261764+0000", 2024-09-21T21:43:04.522 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7591999999999998e-05 2024-09-21T21:43:04.522 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.522 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.522 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:04.522 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261842+0000", 2024-09-21T21:43:04.522 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7865000000000001e-05 2024-09-21T21:43:04.522 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:04.522 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:04.522 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:04.522 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.522 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.522 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3438 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 954368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:43:04.522 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.205908+0000", 2024-09-21T21:43:04.522 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 77.293493553000005, 2024-09-21T21:43:04.523 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056083169000000002, 2024-09-21T21:43:04.523 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:04.523 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:04.523 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:04.523 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:04.523 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:04.523 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:04.523 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3438 2024-09-21T21:43:04.523 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.523 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:04.523 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.523 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:04.523 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205908+0000", 2024-09-21T21:43:04.523 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:04.524 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.524 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.524 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:04.524 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205908+0000", 2024-09-21T21:43:04.524 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:04.524 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.524 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.524 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:04.524 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206431+0000", 2024-09-21T21:43:04.524 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00052282100000000003 2024-09-21T21:43:04.524 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.524 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.524 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:04.524 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206442+0000", 2024-09-21T21:43:04.524 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1695000000000001e-05 2024-09-21T21:43:04.525 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.525 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.525 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:04.525 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206452+0000", 2024-09-21T21:43:04.525 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.4560000000000006e-06 2024-09-21T21:43:04.525 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.525 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.525 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:04.525 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206458+0000", 2024-09-21T21:43:04.525 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4670000000000001e-06 2024-09-21T21:43:04.525 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.525 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.525 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:04.525 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261871+0000", 2024-09-21T21:43:04.525 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055412712000000003 2024-09-21T21:43:04.526 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.526 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.526 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:04.526 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261928+0000", 2024-09-21T21:43:04.526 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6631e-05 2024-09-21T21:43:04.526 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.526 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.526 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:04.526 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261991+0000", 2024-09-21T21:43:04.526 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3386999999999995e-05 2024-09-21T21:43:04.526 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:04.526 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:04.526 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:04.526 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.527 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.527 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3444 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 2359296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:43:04.527 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.206533+0000", 2024-09-21T21:43:04.527 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 77.292868033000005, 2024-09-21T21:43:04.527 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055974239000000002, 2024-09-21T21:43:04.527 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:04.527 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:04.527 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:04.527 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:04.527 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:04.527 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:04.527 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3444 2024-09-21T21:43:04.527 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.527 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:04.528 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.528 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:04.528 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206533+0000", 2024-09-21T21:43:04.528 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:04.528 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.528 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.528 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:04.528 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206533+0000", 2024-09-21T21:43:04.528 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:04.528 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.528 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.528 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:04.528 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206710+0000", 2024-09-21T21:43:04.528 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000176293 2024-09-21T21:43:04.528 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.529 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.529 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:04.529 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206714+0000", 2024-09-21T21:43:04.529 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1989999999999999e-06 2024-09-21T21:43:04.529 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.529 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.529 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:04.529 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206723+0000", 2024-09-21T21:43:04.529 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.9199999999999993e-06 2024-09-21T21:43:04.529 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.529 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.529 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:04.529 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206729+0000", 2024-09-21T21:43:04.530 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0739999999999998e-06 2024-09-21T21:43:04.530 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.530 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.530 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:04.530 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262339+0000", 2024-09-21T21:43:04.530 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055610166000000003 2024-09-21T21:43:04.530 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.530 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.530 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:04.530 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262419+0000", 2024-09-21T21:43:04.530 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0432999999999994e-05 2024-09-21T21:43:04.530 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.530 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.530 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:04.530 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262507+0000", 2024-09-21T21:43:04.531 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.8153999999999997e-05 2024-09-21T21:43:04.531 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:04.531 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:04.531 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:04.531 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.531 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.531 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10362 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:43:04.531 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840386+0000", 2024-09-21T21:43:04.531 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 50.659015189000002, 2024-09-21T21:43:04.531 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056546515999999998, 2024-09-21T21:43:04.531 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:04.531 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:04.531 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:04.532 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:04.532 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:43:04.532 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:43:04.532 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10362 2024-09-21T21:43:04.532 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.532 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:04.532 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.532 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:04.532 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840386+0000", 2024-09-21T21:43:04.532 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:04.532 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.532 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.532 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:04.532 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840386+0000", 2024-09-21T21:43:04.532 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:04.533 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.533 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.533 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:04.533 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840386+0000", 2024-09-21T21:43:04.533 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.01e-07 2024-09-21T21:43:04.533 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.533 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.533 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:04.533 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840389+0000", 2024-09-21T21:43:04.533 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.9739999999999998e-06 2024-09-21T21:43:04.533 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.533 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.533 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:04.533 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840396+0000", 2024-09-21T21:43:04.533 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2280000000000003e-06 2024-09-21T21:43:04.534 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.534 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.534 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:04.534 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840400+0000", 2024-09-21T21:43:04.534 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4429999999999996e-06 2024-09-21T21:43:04.534 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.534 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.534 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:04.534 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.890896+0000", 2024-09-21T21:43:04.534 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.050495717000000002 2024-09-21T21:43:04.534 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.534 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.534 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:04.534 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.891012+0000", 2024-09-21T21:43:04.535 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011602 2024-09-21T21:43:04.535 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.535 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.535 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:43:04.535 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.891274+0000", 2024-09-21T21:43:04.535 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00026240200000000001 2024-09-21T21:43:04.535 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.535 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.535 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:43:04.535 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.894810+0000", 2024-09-21T21:43:04.535 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0035353630000000001 2024-09-21T21:43:04.535 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.535 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.535 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:04.535 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.896818+0000", 2024-09-21T21:43:04.536 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.002008681 2024-09-21T21:43:04.536 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.536 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.536 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:43:04.536 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.896850+0000", 2024-09-21T21:43:04.536 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.1826999999999999e-05 2024-09-21T21:43:04.536 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.536 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.536 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:04.536 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.896933+0000", 2024-09-21T21:43:04.536 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.2460000000000007e-05 2024-09-21T21:43:04.536 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:04.536 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:04.536 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:04.537 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.537 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.537 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10363 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:43:04.537 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840606+0000", 2024-09-21T21:43:04.537 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 50.658795679000001, 2024-09-21T21:43:04.537 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056546011, 2024-09-21T21:43:04.537 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:04.537 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:04.537 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:04.537 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:04.537 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:43:04.537 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:43:04.537 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10363 2024-09-21T21:43:04.537 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.538 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:04.538 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.538 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:04.538 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840606+0000", 2024-09-21T21:43:04.538 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:04.538 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.538 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.538 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:04.538 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840606+0000", 2024-09-21T21:43:04.538 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:04.538 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.538 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.538 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:04.539 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840606+0000", 2024-09-21T21:43:04.539 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.4700000000000001e-07 2024-09-21T21:43:04.539 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.539 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.539 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:04.539 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840611+0000", 2024-09-21T21:43:04.539 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4649999999999996e-06 2024-09-21T21:43:04.539 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.539 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.539 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:04.539 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840620+0000", 2024-09-21T21:43:04.539 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.0350000000000007e-06 2024-09-21T21:43:04.539 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.539 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.539 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:04.540 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840626+0000", 2024-09-21T21:43:04.540 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7180000000000001e-06 2024-09-21T21:43:04.540 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.540 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.540 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:04.540 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.891665+0000", 2024-09-21T21:43:04.540 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.051039924 2024-09-21T21:43:04.540 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.540 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.540 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:04.540 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.891770+0000", 2024-09-21T21:43:04.540 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010445700000000001 2024-09-21T21:43:04.540 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.540 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.541 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:43:04.541 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.892147+0000", 2024-09-21T21:43:04.541 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00037731800000000002 2024-09-21T21:43:04.541 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.541 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.541 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:43:04.541 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.896776+0000", 2024-09-21T21:43:04.541 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0046289759999999999 2024-09-21T21:43:04.541 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.541 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.541 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:04.541 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897003+0000", 2024-09-21T21:43:04.541 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00022641299999999999 2024-09-21T21:43:04.541 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.541 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.542 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:43:04.542 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897064+0000", 2024-09-21T21:43:04.542 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1571999999999999e-05 2024-09-21T21:43:04.542 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.542 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.542 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:04.542 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897152+0000", 2024-09-21T21:43:04.542 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.7386000000000007e-05 2024-09-21T21:43:04.542 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:04.542 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:04.542 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:04.542 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.542 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.542 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10364 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:43:04.543 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840641+0000", 2024-09-21T21:43:04.543 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 50.658759859, 2024-09-21T21:43:04.543 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056671527999999999, 2024-09-21T21:43:04.543 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:04.543 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:04.543 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:04.543 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:04.543 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:43:04.543 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:43:04.543 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10364 2024-09-21T21:43:04.543 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.543 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:04.543 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.543 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:04.543 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840641+0000", 2024-09-21T21:43:04.544 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:04.544 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.544 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.544 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:04.544 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840641+0000", 2024-09-21T21:43:04.544 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:04.544 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.544 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.544 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:04.544 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840642+0000", 2024-09-21T21:43:04.544 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7899999999999999e-07 2024-09-21T21:43:04.544 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.544 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.544 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:04.544 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840645+0000", 2024-09-21T21:43:04.545 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0010000000000002e-06 2024-09-21T21:43:04.545 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.545 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.545 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:04.545 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840654+0000", 2024-09-21T21:43:04.545 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.8570000000000001e-06 2024-09-21T21:43:04.545 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.545 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.545 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:04.545 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840659+0000", 2024-09-21T21:43:04.545 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.558e-06 2024-09-21T21:43:04.545 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.545 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.545 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:04.545 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.892656+0000", 2024-09-21T21:43:04.546 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.051996586999999997 2024-09-21T21:43:04.546 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.546 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.546 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:04.546 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.892721+0000", 2024-09-21T21:43:04.546 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5010000000000003e-05 2024-09-21T21:43:04.546 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.546 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.546 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:43:04.546 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.893112+0000", 2024-09-21T21:43:04.546 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00039133899999999999 2024-09-21T21:43:04.546 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.546 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.547 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:43:04.547 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.896784+0000", 2024-09-21T21:43:04.547 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0036713679999999999 2024-09-21T21:43:04.547 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.547 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.547 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:04.547 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897204+0000", 2024-09-21T21:43:04.547 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00041991400000000002 2024-09-21T21:43:04.547 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.547 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.547 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:43:04.547 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897234+0000", 2024-09-21T21:43:04.547 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0307999999999998e-05 2024-09-21T21:43:04.547 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.547 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.548 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:04.548 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897313+0000", 2024-09-21T21:43:04.548 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.9107e-05 2024-09-21T21:43:04.548 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:04.548 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:04.548 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:04.548 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.548 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.548 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10365 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:43:04.548 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840668+0000", 2024-09-21T21:43:04.548 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 50.658732993999998, 2024-09-21T21:43:04.548 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.058761225, 2024-09-21T21:43:04.548 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:04.548 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:04.548 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:04.549 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:04.549 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:43:04.549 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:43:04.549 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10365 2024-09-21T21:43:04.549 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.549 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:04.549 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.549 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:04.549 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840668+0000", 2024-09-21T21:43:04.549 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:04.549 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.549 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.549 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:04.549 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840668+0000", 2024-09-21T21:43:04.550 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:04.550 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.550 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.550 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:04.550 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840669+0000", 2024-09-21T21:43:04.550 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6300000000000001e-07 2024-09-21T21:43:04.550 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.550 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.550 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:04.550 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840672+0000", 2024-09-21T21:43:04.550 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.072e-06 2024-09-21T21:43:04.550 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.550 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.550 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:04.551 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840678+0000", 2024-09-21T21:43:04.551 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.7039999999999997e-06 2024-09-21T21:43:04.551 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.551 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.551 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:04.551 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840683+0000", 2024-09-21T21:43:04.551 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6759999999999998e-06 2024-09-21T21:43:04.551 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.551 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.551 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:04.551 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.893929+0000", 2024-09-21T21:43:04.551 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.053246136999999999 2024-09-21T21:43:04.551 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.551 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.552 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:04.552 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.894082+0000", 2024-09-21T21:43:04.552 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00015301799999999999 2024-09-21T21:43:04.552 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.552 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.552 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:43:04.552 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.894357+0000", 2024-09-21T21:43:04.552 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00027507999999999998 2024-09-21T21:43:04.552 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.552 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.552 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:43:04.552 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897161+0000", 2024-09-21T21:43:04.552 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0028036810000000001 2024-09-21T21:43:04.552 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.552 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.553 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:04.553 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.899243+0000", 2024-09-21T21:43:04.553 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0020816379999999998 2024-09-21T21:43:04.553 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.553 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.553 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:43:04.553 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.899306+0000", 2024-09-21T21:43:04.553 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2964000000000002e-05 2024-09-21T21:43:04.553 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.553 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.553 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:04.553 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.899430+0000", 2024-09-21T21:43:04.553 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00012389200000000001 2024-09-21T21:43:04.553 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:04.553 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:04.554 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:04.554 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.554 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.554 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10366 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:43:04.554 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840691+0000", 2024-09-21T21:43:04.554 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 50.658710294000002, 2024-09-21T21:43:04.554 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.061540984999999999, 2024-09-21T21:43:04.554 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:04.554 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:04.554 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:04.554 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:04.554 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:43:04.554 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:43:04.554 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10366 2024-09-21T21:43:04.555 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.555 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:04.555 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.555 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:04.555 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840691+0000", 2024-09-21T21:43:04.555 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:04.555 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.555 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.555 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:04.555 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840691+0000", 2024-09-21T21:43:04.555 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:04.555 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.555 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.555 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:04.555 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840691+0000", 2024-09-21T21:43:04.556 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3099999999999999e-07 2024-09-21T21:43:04.556 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.556 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.556 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:04.556 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840694+0000", 2024-09-21T21:43:04.556 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.5780000000000001e-06 2024-09-21T21:43:04.556 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.556 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.556 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:04.556 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840700+0000", 2024-09-21T21:43:04.556 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9429999999999997e-06 2024-09-21T21:43:04.556 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.556 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.556 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:04.557 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840704+0000", 2024-09-21T21:43:04.557 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6380000000000003e-06 2024-09-21T21:43:04.557 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.557 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.557 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:04.557 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.894846+0000", 2024-09-21T21:43:04.557 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.054141392000000003 2024-09-21T21:43:04.557 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.557 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.557 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:04.557 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.894932+0000", 2024-09-21T21:43:04.557 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.5834000000000005e-05 2024-09-21T21:43:04.557 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.557 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.557 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:43:04.558 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.895302+0000", 2024-09-21T21:43:04.558 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000370737 2024-09-21T21:43:04.558 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.558 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.558 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:43:04.558 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897166+0000", 2024-09-21T21:43:04.558 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0018632620000000001 2024-09-21T21:43:04.558 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.558 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.558 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:04.558 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902053+0000", 2024-09-21T21:43:04.558 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0048868289999999997 2024-09-21T21:43:04.558 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.558 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.559 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:43:04.559 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902111+0000", 2024-09-21T21:43:04.559 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7983999999999999e-05 2024-09-21T21:43:04.559 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.559 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.559 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:04.559 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902232+0000", 2024-09-21T21:43:04.559 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000121457 2024-09-21T21:43:04.559 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:04.559 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:04.559 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:04.559 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.559 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.559 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10367 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:43:04.560 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840842+0000", 2024-09-21T21:43:04.560 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 50.658559486000001, 2024-09-21T21:43:04.560 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.061549088000000002, 2024-09-21T21:43:04.560 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:04.560 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:04.560 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:04.560 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:04.560 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:43:04.560 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:43:04.560 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10367 2024-09-21T21:43:04.560 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.560 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:04.560 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.560 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:04.560 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840842+0000", 2024-09-21T21:43:04.561 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:04.561 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.561 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.561 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:04.561 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840842+0000", 2024-09-21T21:43:04.561 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:04.561 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.561 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.561 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:04.561 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840842+0000", 2024-09-21T21:43:04.561 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3099999999999997e-07 2024-09-21T21:43:04.561 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.561 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.561 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:04.562 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840846+0000", 2024-09-21T21:43:04.562 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.2890000000000002e-06 2024-09-21T21:43:04.562 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.562 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.562 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:04.562 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840855+0000", 2024-09-21T21:43:04.562 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.9619999999999999e-06 2024-09-21T21:43:04.562 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.562 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.562 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:04.562 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840860+0000", 2024-09-21T21:43:04.562 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3759999999999999e-06 2024-09-21T21:43:04.563 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.563 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.563 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:04.563 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.895799+0000", 2024-09-21T21:43:04.563 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.054938569999999999 2024-09-21T21:43:04.563 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.563 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.563 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:04.563 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.895906+0000", 2024-09-21T21:43:04.563 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000107593 2024-09-21T21:43:04.563 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.563 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.563 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:43:04.563 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.896356+0000", 2024-09-21T21:43:04.564 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00044941700000000002 2024-09-21T21:43:04.564 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.564 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.564 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:43:04.564 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.898131+0000", 2024-09-21T21:43:04.564 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.001775064 2024-09-21T21:43:04.564 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.564 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.564 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:04.564 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902283+0000", 2024-09-21T21:43:04.564 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0041524839999999997 2024-09-21T21:43:04.564 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.564 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.564 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:43:04.564 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902316+0000", 2024-09-21T21:43:04.565 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3002e-05 2024-09-21T21:43:04.565 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.565 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.565 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:04.565 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902391+0000", 2024-09-21T21:43:04.565 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.47e-05 2024-09-21T21:43:04.565 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:04.565 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:04.565 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:04.565 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.565 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.565 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10368 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:43:04.565 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840873+0000", 2024-09-21T21:43:04.565 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 50.658528531999998, 2024-09-21T21:43:04.566 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.063395291000000006, 2024-09-21T21:43:04.566 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:04.566 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:04.566 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:04.566 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:04.566 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:43:04.566 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:43:04.566 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10368 2024-09-21T21:43:04.566 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.566 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:04.566 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.566 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:04.566 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840873+0000", 2024-09-21T21:43:04.566 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:04.566 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.567 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.567 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:04.567 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840873+0000", 2024-09-21T21:43:04.567 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:04.567 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.567 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.567 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:04.567 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840873+0000", 2024-09-21T21:43:04.567 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6699999999999999e-07 2024-09-21T21:43:04.567 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.567 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.567 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:04.567 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840876+0000", 2024-09-21T21:43:04.567 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.6410000000000002e-06 2024-09-21T21:43:04.568 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.568 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.568 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:04.568 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840883+0000", 2024-09-21T21:43:04.568 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0389999999999999e-06 2024-09-21T21:43:04.568 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.568 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.568 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:04.568 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840888+0000", 2024-09-21T21:43:04.568 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8679999999999998e-06 2024-09-21T21:43:04.568 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.568 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.568 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:04.568 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897345+0000", 2024-09-21T21:43:04.568 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056457734000000002 2024-09-21T21:43:04.569 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.569 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.569 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:04.569 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897407+0000", 2024-09-21T21:43:04.569 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1822000000000005e-05 2024-09-21T21:43:04.569 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.569 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.569 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:43:04.569 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897664+0000", 2024-09-21T21:43:04.569 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00025638499999999998 2024-09-21T21:43:04.569 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.569 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.569 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:43:04.569 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.901136+0000", 2024-09-21T21:43:04.569 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0034728070000000001 2024-09-21T21:43:04.570 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.570 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.570 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:04.570 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904090+0000", 2024-09-21T21:43:04.570 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0029535799999999999 2024-09-21T21:43:04.570 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.570 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.570 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:43:04.570 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904144+0000", 2024-09-21T21:43:04.570 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4369000000000001e-05 2024-09-21T21:43:04.570 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.570 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.570 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:04.570 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904268+0000", 2024-09-21T21:43:04.571 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00012367899999999999 2024-09-21T21:43:04.571 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:04.571 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:04.571 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:04.571 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.571 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.571 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10369 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:43:04.571 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840902+0000", 2024-09-21T21:43:04.571 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 50.658498879, 2024-09-21T21:43:04.571 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.063538938000000003, 2024-09-21T21:43:04.571 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:04.571 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:04.571 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:04.571 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:04.571 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:43:04.572 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:43:04.572 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10369 2024-09-21T21:43:04.572 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.572 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:04.572 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.572 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:04.572 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840902+0000", 2024-09-21T21:43:04.572 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:04.572 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.572 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.572 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:04.572 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840902+0000", 2024-09-21T21:43:04.572 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:04.572 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.572 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.573 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:04.573 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840903+0000", 2024-09-21T21:43:04.573 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2100000000000002e-07 2024-09-21T21:43:04.573 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.573 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.573 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:04.573 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840913+0000", 2024-09-21T21:43:04.573 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.048e-05 2024-09-21T21:43:04.573 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.573 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.573 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:04.573 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840920+0000", 2024-09-21T21:43:04.573 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0330000000000003e-06 2024-09-21T21:43:04.573 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.574 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.574 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:04.574 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840925+0000", 2024-09-21T21:43:04.574 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0170000000000004e-06 2024-09-21T21:43:04.574 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.574 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.574 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:04.574 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.898179+0000", 2024-09-21T21:43:04.574 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.057253671999999999 2024-09-21T21:43:04.574 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.574 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.574 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:04.574 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.898284+0000", 2024-09-21T21:43:04.574 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000105202 2024-09-21T21:43:04.574 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.575 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.575 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:43:04.575 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.898729+0000", 2024-09-21T21:43:04.575 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00044459300000000002 2024-09-21T21:43:04.575 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.575 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.575 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:43:04.575 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902246+0000", 2024-09-21T21:43:04.575 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0035171629999999998 2024-09-21T21:43:04.575 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.575 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.575 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:04.575 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904310+0000", 2024-09-21T21:43:04.576 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.002063765 2024-09-21T21:43:04.576 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.576 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.576 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:43:04.576 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904342+0000", 2024-09-21T21:43:04.576 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.2091999999999997e-05 2024-09-21T21:43:04.576 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.576 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.576 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:04.576 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904441+0000", 2024-09-21T21:43:04.576 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.9500000000000006e-05 2024-09-21T21:43:04.576 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:04.576 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:04.576 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:04.576 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.577 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.577 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10370 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:43:04.577 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840944+0000", 2024-09-21T21:43:04.577 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 50.658457165999998, 2024-09-21T21:43:04.577 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.064436853000000002, 2024-09-21T21:43:04.577 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:04.577 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:04.577 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:04.577 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:04.577 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:43:04.577 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:43:04.577 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10370 2024-09-21T21:43:04.577 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.577 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:04.578 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.578 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:04.578 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840944+0000", 2024-09-21T21:43:04.578 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:04.578 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.578 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.578 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:04.579 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840944+0000", 2024-09-21T21:43:04.579 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:04.579 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.579 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.579 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:04.579 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840945+0000", 2024-09-21T21:43:04.579 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2300000000000002e-07 2024-09-21T21:43:04.579 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.579 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.579 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:04.579 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840947+0000", 2024-09-21T21:43:04.579 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.8200000000000001e-06 2024-09-21T21:43:04.579 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.579 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.580 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:04.580 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840955+0000", 2024-09-21T21:43:04.580 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.8660000000000006e-06 2024-09-21T21:43:04.580 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.580 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.580 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:04.580 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840960+0000", 2024-09-21T21:43:04.580 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8670000000000002e-06 2024-09-21T21:43:04.580 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.580 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.580 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:04.580 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.899460+0000", 2024-09-21T21:43:04.580 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.058499922000000003 2024-09-21T21:43:04.580 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.580 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.581 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:04.581 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.899530+0000", 2024-09-21T21:43:04.581 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9558e-05 2024-09-21T21:43:04.581 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.581 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.581 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:43:04.581 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.899775+0000", 2024-09-21T21:43:04.581 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000245231 2024-09-21T21:43:04.581 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.581 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.581 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:43:04.581 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902254+0000", 2024-09-21T21:43:04.581 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0024790519999999998 2024-09-21T21:43:04.581 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.582 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.582 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:04.582 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905183+0000", 2024-09-21T21:43:04.582 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.002929156 2024-09-21T21:43:04.582 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.582 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.582 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:43:04.582 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905243+0000", 2024-09-21T21:43:04.582 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0055e-05 2024-09-21T21:43:04.582 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.582 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.582 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:04.582 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905381+0000", 2024-09-21T21:43:04.582 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00013790300000000001 2024-09-21T21:43:04.582 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:04.583 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:04.583 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:04.583 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.583 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.583 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10371 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:43:04.583 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840980+0000", 2024-09-21T21:43:04.583 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 50.658421500000003, 2024-09-21T21:43:04.583 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.064554376999999996, 2024-09-21T21:43:04.583 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:04.583 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:04.583 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:04.583 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:04.583 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:43:04.583 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:43:04.584 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10371 2024-09-21T21:43:04.584 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.584 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:04.584 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.584 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:04.584 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840980+0000", 2024-09-21T21:43:04.584 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:04.584 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.584 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.584 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:04.584 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840980+0000", 2024-09-21T21:43:04.584 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:04.584 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.584 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.584 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:04.585 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840980+0000", 2024-09-21T21:43:04.585 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8999999999999997e-07 2024-09-21T21:43:04.585 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.585 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.585 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:04.585 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840983+0000", 2024-09-21T21:43:04.585 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.123e-06 2024-09-21T21:43:04.585 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.585 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.585 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:04.585 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840990+0000", 2024-09-21T21:43:04.585 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9129999999999997e-06 2024-09-21T21:43:04.585 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.585 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.585 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:04.586 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840995+0000", 2024-09-21T21:43:04.586 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0989999999999998e-06 2024-09-21T21:43:04.586 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.586 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.586 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:04.586 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.900168+0000", 2024-09-21T21:43:04.586 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.059172739000000002 2024-09-21T21:43:04.586 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.586 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.586 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:04.586 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.900274+0000", 2024-09-21T21:43:04.586 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010546499999999999 2024-09-21T21:43:04.586 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.586 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.586 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:43:04.587 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.900703+0000", 2024-09-21T21:43:04.587 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00042978200000000003 2024-09-21T21:43:04.587 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.587 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.587 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:43:04.587 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.903443+0000", 2024-09-21T21:43:04.587 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0027400749999999998 2024-09-21T21:43:04.587 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.587 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.587 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:04.587 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905430+0000", 2024-09-21T21:43:04.587 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0019868109999999998 2024-09-21T21:43:04.587 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.587 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.587 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:43:04.588 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905464+0000", 2024-09-21T21:43:04.588 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3599000000000001e-05 2024-09-21T21:43:04.588 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.588 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.588 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:04.588 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905534+0000", 2024-09-21T21:43:04.588 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0281e-05 2024-09-21T21:43:04.588 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:04.588 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:04.588 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:04.588 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.588 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.588 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10372 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:43:04.588 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.841004+0000", 2024-09-21T21:43:04.588 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 50.658397610999998, 2024-09-21T21:43:04.589 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.064757812999999997, 2024-09-21T21:43:04.589 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:04.589 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:04.589 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:04.589 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:04.589 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:43:04.589 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:43:04.589 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10372 2024-09-21T21:43:04.589 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.589 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:04.589 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.589 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:04.589 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841004+0000", 2024-09-21T21:43:04.589 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:04.590 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.590 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.590 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:04.590 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841004+0000", 2024-09-21T21:43:04.590 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:04.590 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.590 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.590 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:04.590 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841004+0000", 2024-09-21T21:43:04.590 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8500000000000002e-07 2024-09-21T21:43:04.590 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.590 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.590 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:04.590 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841007+0000", 2024-09-21T21:43:04.591 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.6390000000000001e-06 2024-09-21T21:43:04.591 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.591 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.591 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:04.591 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841013+0000", 2024-09-21T21:43:04.591 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0070000000000003e-06 2024-09-21T21:43:04.591 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.591 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.591 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:04.591 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841018+0000", 2024-09-21T21:43:04.591 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9119999999999997e-06 2024-09-21T21:43:04.591 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.591 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.591 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:04.591 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.901182+0000", 2024-09-21T21:43:04.592 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.060164444999999997 2024-09-21T21:43:04.592 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.592 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.592 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:04.592 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.901297+0000", 2024-09-21T21:43:04.592 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011468699999999999 2024-09-21T21:43:04.592 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.592 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.592 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:43:04.592 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.901559+0000", 2024-09-21T21:43:04.592 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000261764 2024-09-21T21:43:04.592 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.592 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.592 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:43:04.592 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904281+0000", 2024-09-21T21:43:04.593 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0027229329999999999 2024-09-21T21:43:04.593 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.593 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.593 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:04.593 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905588+0000", 2024-09-21T21:43:04.593 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.001306438 2024-09-21T21:43:04.593 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.593 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.593 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:43:04.593 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905644+0000", 2024-09-21T21:43:04.593 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5747e-05 2024-09-21T21:43:04.593 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.593 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.593 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:04.594 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905761+0000", 2024-09-21T21:43:04.594 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000117856 2024-09-21T21:43:04.594 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:04.594 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:04.594 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:04.594 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.594 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.594 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10373 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:43:04.594 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.841027+0000", 2024-09-21T21:43:04.594 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 50.658374526999999, 2024-09-21T21:43:04.594 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.066941839000000003, 2024-09-21T21:43:04.594 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:04.594 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:04.594 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:04.594 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:04.595 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:43:04.595 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:43:04.595 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10373 2024-09-21T21:43:04.595 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.595 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:04.595 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.595 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:04.595 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841027+0000", 2024-09-21T21:43:04.595 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:04.595 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.595 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.595 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:04.596 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841027+0000", 2024-09-21T21:43:04.596 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:04.596 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.596 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.596 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:04.596 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841027+0000", 2024-09-21T21:43:04.596 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6199999999999999e-07 2024-09-21T21:43:04.596 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.596 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.596 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:04.596 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841030+0000", 2024-09-21T21:43:04.596 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.729e-06 2024-09-21T21:43:04.596 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.596 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.597 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:04.597 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841036+0000", 2024-09-21T21:43:04.597 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1979999999999998e-06 2024-09-21T21:43:04.597 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.597 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.597 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:04.597 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841042+0000", 2024-09-21T21:43:04.597 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4289999999999998e-06 2024-09-21T21:43:04.597 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.597 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.597 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:04.597 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902433+0000", 2024-09-21T21:43:04.597 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.061390329 2024-09-21T21:43:04.597 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.597 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.598 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:04.598 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902541+0000", 2024-09-21T21:43:04.598 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000108349 2024-09-21T21:43:04.598 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.598 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.598 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:43:04.598 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902983+0000", 2024-09-21T21:43:04.598 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00044158000000000001 2024-09-21T21:43:04.598 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.598 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.598 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:43:04.598 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904490+0000", 2024-09-21T21:43:04.598 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0015074489999999999 2024-09-21T21:43:04.598 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.599 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.599 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:04.599 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.907671+0000", 2024-09-21T21:43:04.599 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.003180511 2024-09-21T21:43:04.599 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.599 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.599 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:43:04.599 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.907767+0000", 2024-09-21T21:43:04.599 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6379000000000007e-05 2024-09-21T21:43:04.599 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.599 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.599 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:04.599 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.907969+0000", 2024-09-21T21:43:04.599 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00020152399999999999 2024-09-21T21:43:04.599 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:04.600 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:04.600 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:04.600 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.600 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.600 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:12335 3.1a 3:5b387735:::rbd_data.1121faebdfe3.0000000000000005:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:43:04.600 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:14.411635+0000", 2024-09-21T21:43:04.600 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 50.087766649000002, 2024-09-21T21:43:04.600 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.005973491, 2024-09-21T21:43:04.600 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:04.600 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:04.600 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:04.600 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:04.600 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:43:04.600 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:43:04.600 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 12335 2024-09-21T21:43:04.601 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.601 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:04.601 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.601 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:04.601 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411635+0000", 2024-09-21T21:43:04.601 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:04.601 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.601 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.601 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:04.601 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411635+0000", 2024-09-21T21:43:04.601 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:04.601 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.601 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.601 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:04.601 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411635+0000", 2024-09-21T21:43:04.602 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8500000000000001e-07 2024-09-21T21:43:04.602 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.602 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.602 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:04.602 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411639+0000", 2024-09-21T21:43:04.602 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.574e-06 2024-09-21T21:43:04.602 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.602 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.602 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:04.602 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411645+0000", 2024-09-21T21:43:04.602 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6869999999999997e-06 2024-09-21T21:43:04.602 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.602 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.603 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:04.603 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411649+0000", 2024-09-21T21:43:04.603 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.771e-06 2024-09-21T21:43:04.603 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.603 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.603 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:04.603 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411695+0000", 2024-09-21T21:43:04.603 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6e-05 2024-09-21T21:43:04.603 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.603 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.603 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op must wait for map", 2024-09-21T21:43:04.603 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411702+0000", 2024-09-21T21:43:04.603 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5590000000000002e-06 2024-09-21T21:43:04.603 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.603 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.604 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:04.604 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.417222+0000", 2024-09-21T21:43:04.604 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0055198219999999997 2024-09-21T21:43:04.604 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.604 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.604 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for peered", 2024-09-21T21:43:04.604 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.417230+0000", 2024-09-21T21:43:04.604 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.123e-06 2024-09-21T21:43:04.604 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.604 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.604 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:04.604 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.417348+0000", 2024-09-21T21:43:04.604 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.000118029 2024-09-21T21:43:04.604 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.605 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.605 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:04.605 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.417451+0000", 2024-09-21T21:43:04.605 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000103674 2024-09-21T21:43:04.605 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.605 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.605 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:04.605 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.417608+0000", 2024-09-21T21:43:04.605 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00015666700000000001 2024-09-21T21:43:04.605 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:04.605 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:04.605 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:04.605 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.605 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.606 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:12349 3.1a 3:5b387735:::rbd_data.1121faebdfe3.0000000000000005:head [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:43:04.606 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:14.411833+0000", 2024-09-21T21:43:04.606 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 50.087567835999998, 2024-09-21T21:43:04.606 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0065064500000001, 2024-09-21T21:43:04.606 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:04.606 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:04.606 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:04.606 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:04.606 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:43:04.606 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:43:04.606 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 12349 2024-09-21T21:43:04.606 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.606 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:04.606 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.607 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:04.607 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411833+0000", 2024-09-21T21:43:04.607 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:04.607 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.607 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.607 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:04.607 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411833+0000", 2024-09-21T21:43:04.607 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:04.607 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.607 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.607 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:04.607 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411834+0000", 2024-09-21T21:43:04.607 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.03e-07 2024-09-21T21:43:04.607 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.607 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.608 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:04.608 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411836+0000", 2024-09-21T21:43:04.608 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.4700000000000001e-06 2024-09-21T21:43:04.608 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.608 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.608 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:04.608 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411843+0000", 2024-09-21T21:43:04.608 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4069999999999998e-06 2024-09-21T21:43:04.608 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.608 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.608 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:04.608 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411848+0000", 2024-09-21T21:43:04.608 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1580000000000004e-06 2024-09-21T21:43:04.608 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.609 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.609 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:04.609 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411901+0000", 2024-09-21T21:43:04.609 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2824000000000001e-05 2024-09-21T21:43:04.609 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.609 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.609 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting_for_map not empty", 2024-09-21T21:43:04.609 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411904+0000", 2024-09-21T21:43:04.609 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0639999999999998e-06 2024-09-21T21:43:04.609 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.609 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.609 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:04.609 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.417252+0000", 2024-09-21T21:43:04.609 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0053485060000000003 2024-09-21T21:43:04.609 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.610 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.610 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for peered", 2024-09-21T21:43:04.610 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.417259+0000", 2024-09-21T21:43:04.610 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.1149999999999996e-06 2024-09-21T21:43:04.610 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.610 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.610 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:04.610 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.418160+0000", 2024-09-21T21:43:04.610 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.000901083 2024-09-21T21:43:04.610 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.610 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.610 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:04.610 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.418236+0000", 2024-09-21T21:43:04.610 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5852000000000005e-05 2024-09-21T21:43:04.610 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.611 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.611 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:04.611 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.418340+0000", 2024-09-21T21:43:04.611 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000103568 2024-09-21T21:43:04.611 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:04.611 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:04.611 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:04.611 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.611 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.611 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4385.0:30053 3.6 3:67cae2ed:::rbd_mirroring:head [list-watchers] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e141)", 2024-09-21T21:43:04.611 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:43:03.223636+0000", 2024-09-21T21:43:04.611 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 1.275765152, 2024-09-21T21:43:04.611 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.062115648, 2024-09-21T21:43:04.611 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:04.612 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:04.612 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:43:04.612 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:04.612 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4385", 2024-09-21T21:43:04.612 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:43:04.612 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 30053 2024-09-21T21:43:04.612 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.612 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:04.612 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.612 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:04.612 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:43:03.223636+0000", 2024-09-21T21:43:04.612 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:04.613 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.613 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.613 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:04.613 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:43:03.223636+0000", 2024-09-21T21:43:04.613 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:04.613 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.613 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.613 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:04.613 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:43:03.223638+0000", 2024-09-21T21:43:04.613 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.6080000000000001e-06 2024-09-21T21:43:04.613 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.613 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.613 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:04.613 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:43:03.223643+0000", 2024-09-21T21:43:04.613 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9110000000000001e-06 2024-09-21T21:43:04.614 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.614 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.614 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:04.614 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:43:03.223654+0000", 2024-09-21T21:43:04.614 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1029999999999999e-05 2024-09-21T21:43:04.614 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.614 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.614 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:04.614 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:43:03.223664+0000", 2024-09-21T21:43:04.614 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.9299999999999998e-06 2024-09-21T21:43:04.614 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.614 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.614 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:04.614 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:43:03.223856+0000", 2024-09-21T21:43:04.615 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00019216200000000001 2024-09-21T21:43:04.615 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.615 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.615 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for peered", 2024-09-21T21:43:04.615 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:43:03.223866+0000", 2024-09-21T21:43:04.615 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0356999999999999e-05 2024-09-21T21:43:04.615 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.615 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.615 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:04.615 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:43:04.285628+0000", 2024-09-21T21:43:04.615 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.061762211 2024-09-21T21:43:04.615 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:04.615 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:04.615 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:04.615 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:43:04.285752+0000", 2024-09-21T21:43:04.616 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00012343900000000001 2024-09-21T21:43:04.616 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:04.616 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:04.616 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:04.616 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:04.616 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:04.616 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:43:04.617 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T21:43:04.618 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:43:04.616+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:43:04.718 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:43:04.717+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:43:04.763 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T21:43:04.766 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:43:04.766 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:43:04.767 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 0 2024-09-21T21:43:04.767 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:43:04.777 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T21:43:04.819 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:43:04.818+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:43:04.895 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0: {} 2024-09-21T21:43:04.896 INFO:teuthology.orchestra.run.smithi081.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T21:43:04.914 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1: {} 2024-09-21T21:43:04.914 INFO:teuthology.orchestra.run.smithi081.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T21:43:04.920 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:43:04.919+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:43:04.931 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2: {} 2024-09-21T21:43:04.932 INFO:teuthology.orchestra.run.smithi081.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T21:43:04.932 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:04.932 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:04.941 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T21:43:04.950 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3: {} 2024-09-21T21:43:04.950 INFO:teuthology.orchestra.run.smithi081.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T21:43:04.970 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4: {} 2024-09-21T21:43:04.970 INFO:teuthology.orchestra.run.smithi081.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T21:43:04.989 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5: {} 2024-09-21T21:43:04.989 INFO:teuthology.orchestra.run.smithi081.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T21:43:05.041 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:43:05.040+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:43:05.061 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:05.061 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:05.070 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T21:43:05.122 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:43:05.121+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:43:05.179 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:05.179 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:05.188 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T21:43:05.223 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:43:05.222+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:43:05.238 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T21:43:05.325 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:43:05.323+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:43:05.340 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:05.340 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:05.350 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T21:43:05.426 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:43:05.425+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:43:05.455 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:05.455 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:05.464 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T21:43:05.527 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:43:05.525+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:43:05.610 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:05.610 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:05.619 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T21:43:05.628 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:43:05.626+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:43:05.720 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:05.720 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:05.729 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T21:43:05.729 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:43:05.727+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:43:05.830 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:43:05.828+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:43:05.872 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:05.872 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:05.880 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T21:43:05.930 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:43:05.929+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:43:05.989 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:05.989 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:05.999 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T21:43:06.039 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:43:06.038+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:43:06.133 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:43:06.131+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:43:06.141 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:06.142 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:06.150 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T21:43:06.224 INFO:teuthology.orchestra.run.smithi081.stderr:noscrub is unset 2024-09-21T21:43:06.234 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:43:06.233+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:43:06.260 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:06.260 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:06.269 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T21:43:06.335 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:43:06.333+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:43:06.396 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:06.396 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:06.405 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T21:43:06.436 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:43:06.434+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:43:06.537 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:43:06.536+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:43:06.561 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:06.561 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:06.571 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T21:43:06.638 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:43:06.637+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:43:06.680 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:06.680 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:06.691 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T21:43:06.739 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:43:06.738+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:43:06.801 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:06.801 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:06.811 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T21:43:06.841 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:43:06.839+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:43:06.920 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:06.921 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:06.930 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T21:43:06.941 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:43:06.940+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:43:07.001 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T21:43:07.043 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:07.043 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:07.043 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:43:07.041+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:43:07.053 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T21:43:07.136 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0: {} 2024-09-21T21:43:07.136 INFO:teuthology.orchestra.run.smithi081.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T21:43:07.143 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:43:07.142+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:43:07.154 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1: {} 2024-09-21T21:43:07.154 INFO:teuthology.orchestra.run.smithi081.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T21:43:07.171 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2: {} 2024-09-21T21:43:07.171 INFO:teuthology.orchestra.run.smithi081.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T21:43:07.188 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3: {} 2024-09-21T21:43:07.188 INFO:teuthology.orchestra.run.smithi081.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T21:43:07.207 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4: {} 2024-09-21T21:43:07.207 INFO:teuthology.orchestra.run.smithi081.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T21:43:07.211 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:43:07.212 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:43:07.212 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 0 2024-09-21T21:43:07.212 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:43:07.222 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T21:43:07.228 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5: {} 2024-09-21T21:43:07.228 INFO:teuthology.orchestra.run.smithi081.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T21:43:07.236 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 5, 3, 0] out_osds: [4] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-21T21:43:07.236 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T21:43:07.236 INFO:tasks.thrashosds.thrasher:Reweighting osd 3 to 0.9248624707287336 2024-09-21T21:43:07.236 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 0.9248624707287336 2024-09-21T21:43:07.244 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:43:07.242+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:43:07.345 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:43:07.343+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:43:07.351 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:43:07.351 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:43:07.351 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:43:07.351 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:43:07.351 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:43:07.362 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T21:43:07.446 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:43:07.444+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:43:07.495 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:43:07.496 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 20, 2024-09-21T21:43:07.496 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 600, 2024-09-21T21:43:07.496 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:43:07.496 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.496 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13296 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 4046848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:07.496 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.796643+0000", 2024-09-21T21:43:07.496 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 76.696455630000003, 2024-09-21T21:43:07.496 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071682019999999999, 2024-09-21T21:43:07.496 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:07.496 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:07.496 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:07.496 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:07.496 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:07.496 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:07.497 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13296 2024-09-21T21:43:07.497 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.497 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:07.497 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.497 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:07.497 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796643+0000", 2024-09-21T21:43:07.497 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:07.497 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.497 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.497 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:07.498 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796643+0000", 2024-09-21T21:43:07.498 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:07.498 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.498 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.498 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:07.498 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796643+0000", 2024-09-21T21:43:07.498 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.7199999999999998e-07 2024-09-21T21:43:07.498 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.498 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.498 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:07.499 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796647+0000", 2024-09-21T21:43:07.499 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6770000000000001e-06 2024-09-21T21:43:07.499 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.499 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.499 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:07.499 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796656+0000", 2024-09-21T21:43:07.499 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2059999999999996e-06 2024-09-21T21:43:07.499 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.499 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.499 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:07.499 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796664+0000", 2024-09-21T21:43:07.499 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7919999999999993e-06 2024-09-21T21:43:07.499 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.499 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.500 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:07.500 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.868186+0000", 2024-09-21T21:43:07.500 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071522273999999997 2024-09-21T21:43:07.500 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.500 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.500 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:07.500 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.868239+0000", 2024-09-21T21:43:07.500 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2787999999999997e-05 2024-09-21T21:43:07.500 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.500 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.500 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:07.500 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.868325+0000", 2024-09-21T21:43:07.500 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.5611000000000004e-05 2024-09-21T21:43:07.500 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:07.501 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:07.501 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:07.501 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.501 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.501 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13561 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 1728512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:07.501 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.823991+0000", 2024-09-21T21:43:07.501 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 76.669107328999999, 2024-09-21T21:43:07.501 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071654762999999996, 2024-09-21T21:43:07.501 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:07.501 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:07.501 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:07.501 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:07.501 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:07.502 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:07.502 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13561 2024-09-21T21:43:07.502 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.502 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:07.502 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.502 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:07.502 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.823991+0000", 2024-09-21T21:43:07.502 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:07.502 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.502 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.502 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:07.502 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.823991+0000", 2024-09-21T21:43:07.502 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:07.502 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.503 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.503 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:07.503 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.823991+0000", 2024-09-21T21:43:07.503 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9299999999999998e-07 2024-09-21T21:43:07.503 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.503 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.503 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:07.503 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.823994+0000", 2024-09-21T21:43:07.503 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2369999999999999e-06 2024-09-21T21:43:07.503 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.503 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.503 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:07.503 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824000+0000", 2024-09-21T21:43:07.504 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.55e-06 2024-09-21T21:43:07.504 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.504 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.504 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:07.504 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824005+0000", 2024-09-21T21:43:07.504 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4329999999999997e-06 2024-09-21T21:43:07.504 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.504 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.504 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:07.504 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.895360+0000", 2024-09-21T21:43:07.504 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071355426 2024-09-21T21:43:07.504 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.504 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.504 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:07.505 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.895477+0000", 2024-09-21T21:43:07.505 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0001171 2024-09-21T21:43:07.505 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.505 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.505 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:07.505 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.895646+0000", 2024-09-21T21:43:07.505 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000168524 2024-09-21T21:43:07.505 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:07.505 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:07.505 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:07.505 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.505 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.505 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13562 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 1757184~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:07.505 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.824011+0000", 2024-09-21T21:43:07.505 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 76.669087451999999, 2024-09-21T21:43:07.506 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072020453999999998, 2024-09-21T21:43:07.506 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:07.506 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:07.506 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:07.506 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:07.506 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:07.506 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:07.506 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13562 2024-09-21T21:43:07.506 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.506 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:07.506 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.506 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:07.506 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824011+0000", 2024-09-21T21:43:07.506 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:07.507 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.507 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.507 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:07.507 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824011+0000", 2024-09-21T21:43:07.507 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:07.507 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.508 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.508 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:07.508 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824011+0000", 2024-09-21T21:43:07.508 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5600000000000001e-07 2024-09-21T21:43:07.508 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.508 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.508 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:07.508 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824013+0000", 2024-09-21T21:43:07.508 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9800000000000001e-06 2024-09-21T21:43:07.509 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.509 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.509 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:07.509 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824019+0000", 2024-09-21T21:43:07.509 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7339999999999996e-06 2024-09-21T21:43:07.509 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.509 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.509 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:07.509 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824023+0000", 2024-09-21T21:43:07.509 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3680000000000004e-06 2024-09-21T21:43:07.509 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.509 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.509 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:07.509 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.895699+0000", 2024-09-21T21:43:07.510 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071675597999999993 2024-09-21T21:43:07.510 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.510 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.510 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:07.510 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.895830+0000", 2024-09-21T21:43:07.510 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000130858 2024-09-21T21:43:07.510 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.510 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.510 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:07.510 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896031+0000", 2024-09-21T21:43:07.510 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00020155999999999999 2024-09-21T21:43:07.510 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:07.510 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:07.510 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:07.511 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.511 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.511 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13563 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 323584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:07.511 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.824029+0000", 2024-09-21T21:43:07.511 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 76.669069011999994, 2024-09-21T21:43:07.511 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072307563000000005, 2024-09-21T21:43:07.511 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:07.511 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:07.511 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:07.511 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:07.511 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:07.511 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:07.511 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13563 2024-09-21T21:43:07.512 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.512 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:07.512 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.512 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:07.512 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824029+0000", 2024-09-21T21:43:07.512 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:07.512 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.512 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.512 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:07.512 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824029+0000", 2024-09-21T21:43:07.512 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:07.512 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.512 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.513 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:07.513 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824030+0000", 2024-09-21T21:43:07.513 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1699999999999999e-07 2024-09-21T21:43:07.513 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.513 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.513 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:07.513 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824032+0000", 2024-09-21T21:43:07.513 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0320000000000002e-06 2024-09-21T21:43:07.513 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.513 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.513 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:07.513 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824038+0000", 2024-09-21T21:43:07.513 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8769999999999998e-06 2024-09-21T21:43:07.513 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.514 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.514 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:07.514 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824042+0000", 2024-09-21T21:43:07.514 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5650000000000003e-06 2024-09-21T21:43:07.514 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.514 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.514 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:07.514 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896058+0000", 2024-09-21T21:43:07.514 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072015941 2024-09-21T21:43:07.514 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.514 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.514 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:07.514 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896164+0000", 2024-09-21T21:43:07.514 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010593499999999999 2024-09-21T21:43:07.515 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.515 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.515 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:07.515 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896337+0000", 2024-09-21T21:43:07.515 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00017279599999999999 2024-09-21T21:43:07.515 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:07.515 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:07.515 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:07.515 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.515 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.515 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13564 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 983040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:07.515 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.824048+0000", 2024-09-21T21:43:07.515 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 76.669049936999997, 2024-09-21T21:43:07.515 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072491193999999995, 2024-09-21T21:43:07.516 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:07.516 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:07.516 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:07.516 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:07.516 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:07.516 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:07.516 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13564 2024-09-21T21:43:07.516 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.516 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:07.516 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.516 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:07.516 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824048+0000", 2024-09-21T21:43:07.516 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:07.516 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.517 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.517 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:07.517 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824048+0000", 2024-09-21T21:43:07.517 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:07.517 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.517 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.517 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:07.517 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824663+0000", 2024-09-21T21:43:07.517 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00061494299999999996 2024-09-21T21:43:07.517 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.517 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.517 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:07.517 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824669+0000", 2024-09-21T21:43:07.518 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2900000000000002e-06 2024-09-21T21:43:07.518 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.519 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.519 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:07.519 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824679+0000", 2024-09-21T21:43:07.519 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0338e-05 2024-09-21T21:43:07.519 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.519 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.519 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:07.519 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824687+0000", 2024-09-21T21:43:07.519 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.9049999999999997e-06 2024-09-21T21:43:07.520 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.520 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.520 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:07.520 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896381+0000", 2024-09-21T21:43:07.520 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071694305999999999 2024-09-21T21:43:07.520 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.520 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.520 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:07.520 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896445+0000", 2024-09-21T21:43:07.520 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3869999999999997e-05 2024-09-21T21:43:07.520 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.520 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.520 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:07.520 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896540+0000", 2024-09-21T21:43:07.521 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.4542000000000004e-05 2024-09-21T21:43:07.521 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:07.521 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:07.521 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:07.521 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.521 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.521 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13565 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3190784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:07.521 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.824704+0000", 2024-09-21T21:43:07.521 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 76.668394066000005, 2024-09-21T21:43:07.521 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072014745000000005, 2024-09-21T21:43:07.521 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:07.521 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:07.521 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:07.521 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:07.521 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:07.522 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:07.522 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13565 2024-09-21T21:43:07.522 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.522 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:07.522 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.522 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:07.522 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824704+0000", 2024-09-21T21:43:07.522 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:07.522 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.522 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.522 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:07.522 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824704+0000", 2024-09-21T21:43:07.523 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:07.523 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.523 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.523 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:07.523 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824705+0000", 2024-09-21T21:43:07.523 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8999999999999997e-07 2024-09-21T21:43:07.523 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.523 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.523 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:07.523 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824707+0000", 2024-09-21T21:43:07.523 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.7669999999999999e-06 2024-09-21T21:43:07.523 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.523 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.524 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:07.524 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824715+0000", 2024-09-21T21:43:07.524 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7700000000000001e-06 2024-09-21T21:43:07.524 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.524 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.524 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:07.524 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824721+0000", 2024-09-21T21:43:07.524 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9560000000000002e-06 2024-09-21T21:43:07.524 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.524 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.524 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:07.524 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896570+0000", 2024-09-21T21:43:07.524 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071848870999999995 2024-09-21T21:43:07.524 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.524 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.525 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:07.525 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896626+0000", 2024-09-21T21:43:07.525 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5578000000000002e-05 2024-09-21T21:43:07.525 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.525 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.525 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:07.525 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896719+0000", 2024-09-21T21:43:07.525 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.3313000000000006e-05 2024-09-21T21:43:07.525 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:07.525 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:07.525 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:07.525 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.525 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.525 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13566 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3162112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:07.526 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.824729+0000", 2024-09-21T21:43:07.526 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 76.668369178999995, 2024-09-21T21:43:07.526 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072356907999999998, 2024-09-21T21:43:07.526 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:07.526 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:07.526 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:07.526 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:07.526 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:07.526 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:07.526 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13566 2024-09-21T21:43:07.526 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.526 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:07.526 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.526 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:07.527 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824729+0000", 2024-09-21T21:43:07.527 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:07.527 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.527 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.527 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:07.527 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824729+0000", 2024-09-21T21:43:07.527 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:07.527 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.527 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.527 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:07.527 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825630+0000", 2024-09-21T21:43:07.527 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00090139400000000002 2024-09-21T21:43:07.527 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.527 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.528 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:07.528 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825634+0000", 2024-09-21T21:43:07.528 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4960000000000001e-06 2024-09-21T21:43:07.528 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.528 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.528 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:07.528 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825644+0000", 2024-09-21T21:43:07.528 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.9650000000000003e-06 2024-09-21T21:43:07.528 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.528 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.528 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:07.528 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825649+0000", 2024-09-21T21:43:07.528 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0359999999999997e-06 2024-09-21T21:43:07.528 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.528 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.529 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:07.529 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896757+0000", 2024-09-21T21:43:07.529 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071107717000000001 2024-09-21T21:43:07.529 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.529 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.529 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:07.529 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896878+0000", 2024-09-21T21:43:07.529 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000120834 2024-09-21T21:43:07.529 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.529 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.529 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:07.529 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897086+0000", 2024-09-21T21:43:07.529 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00020846600000000001 2024-09-21T21:43:07.529 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:07.529 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:07.530 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:07.530 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.530 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.530 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13568 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3907584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:07.530 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.825671+0000", 2024-09-21T21:43:07.530 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 76.667426770000006, 2024-09-21T21:43:07.530 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071794595000000003, 2024-09-21T21:43:07.530 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:07.530 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:07.530 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:07.530 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:07.530 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:07.530 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:07.530 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13568 2024-09-21T21:43:07.530 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.531 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:07.531 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.531 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:07.531 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825671+0000", 2024-09-21T21:43:07.531 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:07.531 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.531 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.531 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:07.531 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825671+0000", 2024-09-21T21:43:07.531 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:07.531 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.531 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.531 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:07.531 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825672+0000", 2024-09-21T21:43:07.532 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2999999999999999e-07 2024-09-21T21:43:07.532 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.532 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.532 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:07.532 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825673+0000", 2024-09-21T21:43:07.532 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.246e-06 2024-09-21T21:43:07.532 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.532 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.532 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:07.532 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825693+0000", 2024-09-21T21:43:07.532 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9789999999999999e-05 2024-09-21T21:43:07.532 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.532 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.532 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:07.532 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825697+0000", 2024-09-21T21:43:07.533 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2379999999999999e-06 2024-09-21T21:43:07.533 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.533 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.533 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:07.533 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897284+0000", 2024-09-21T21:43:07.533 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071586710999999997 2024-09-21T21:43:07.533 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.533 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.533 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:07.533 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897337+0000", 2024-09-21T21:43:07.533 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2800000000000003e-05 2024-09-21T21:43:07.533 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.533 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.533 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:07.533 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897466+0000", 2024-09-21T21:43:07.534 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00012957999999999999 2024-09-21T21:43:07.534 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:07.534 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:07.534 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:07.534 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.534 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.534 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13574 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 352256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:07.534 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.825702+0000", 2024-09-21T21:43:07.534 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 76.667396621999998, 2024-09-21T21:43:07.534 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072080377000000001, 2024-09-21T21:43:07.534 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:07.534 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:07.534 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:07.534 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:07.535 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:07.535 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:07.535 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13574 2024-09-21T21:43:07.535 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.535 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:07.535 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.535 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:07.535 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825702+0000", 2024-09-21T21:43:07.535 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:07.535 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.535 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.535 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:07.535 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825702+0000", 2024-09-21T21:43:07.535 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:07.535 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.536 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.536 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:07.536 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825943+0000", 2024-09-21T21:43:07.536 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00024105200000000001 2024-09-21T21:43:07.536 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.536 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.536 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:07.536 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825946+0000", 2024-09-21T21:43:07.536 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.743e-06 2024-09-21T21:43:07.536 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.536 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.536 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:07.536 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825956+0000", 2024-09-21T21:43:07.536 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6609999999999995e-06 2024-09-21T21:43:07.537 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.537 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.537 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:07.537 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825963+0000", 2024-09-21T21:43:07.537 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4609999999999996e-06 2024-09-21T21:43:07.537 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.537 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.537 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:07.537 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897502+0000", 2024-09-21T21:43:07.537 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071539354999999999 2024-09-21T21:43:07.537 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.537 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.538 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:07.538 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897619+0000", 2024-09-21T21:43:07.538 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011665 2024-09-21T21:43:07.538 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.538 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.538 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:07.538 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897782+0000", 2024-09-21T21:43:07.538 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00016345500000000001 2024-09-21T21:43:07.538 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:07.538 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:07.538 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:07.538 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.538 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.538 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13575 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 380928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:07.539 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.825973+0000", 2024-09-21T21:43:07.539 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 76.667125186000007, 2024-09-21T21:43:07.539 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072012723000000001, 2024-09-21T21:43:07.539 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:07.539 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:07.539 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:07.539 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:07.539 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:07.539 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:07.539 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13575 2024-09-21T21:43:07.539 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.539 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:07.539 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.539 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:07.539 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825973+0000", 2024-09-21T21:43:07.540 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:07.540 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.540 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.540 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:07.540 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825973+0000", 2024-09-21T21:43:07.540 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:07.540 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.540 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.540 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:07.540 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826746+0000", 2024-09-21T21:43:07.540 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00077326800000000002 2024-09-21T21:43:07.540 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.540 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.540 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:07.540 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826766+0000", 2024-09-21T21:43:07.541 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9465000000000001e-05 2024-09-21T21:43:07.541 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.541 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.541 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:07.541 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826812+0000", 2024-09-21T21:43:07.541 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5927999999999998e-05 2024-09-21T21:43:07.541 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.541 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.541 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:07.541 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826833+0000", 2024-09-21T21:43:07.541 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1710999999999999e-05 2024-09-21T21:43:07.541 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.541 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.541 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:07.541 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897809+0000", 2024-09-21T21:43:07.542 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.070975229000000001 2024-09-21T21:43:07.542 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.542 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.542 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:07.542 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897864+0000", 2024-09-21T21:43:07.542 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5334999999999998e-05 2024-09-21T21:43:07.542 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.542 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.542 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:07.542 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897986+0000", 2024-09-21T21:43:07.542 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000121787 2024-09-21T21:43:07.542 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:07.542 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:07.542 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:07.543 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.543 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.543 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13577 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 1814528~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:07.543 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.826874+0000", 2024-09-21T21:43:07.543 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 76.666224151999998, 2024-09-21T21:43:07.543 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071639123999999998, 2024-09-21T21:43:07.543 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:07.543 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:07.543 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:07.543 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:07.543 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:07.543 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:07.543 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13577 2024-09-21T21:43:07.543 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.543 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:07.544 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.544 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:07.544 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826874+0000", 2024-09-21T21:43:07.544 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:07.544 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.544 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.544 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:07.544 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826874+0000", 2024-09-21T21:43:07.544 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:07.544 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.544 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.544 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:07.544 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826875+0000", 2024-09-21T21:43:07.544 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.82e-07 2024-09-21T21:43:07.544 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.545 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.545 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:07.545 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826877+0000", 2024-09-21T21:43:07.545 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.295e-06 2024-09-21T21:43:07.545 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.545 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.545 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:07.545 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826883+0000", 2024-09-21T21:43:07.545 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2609999999999999e-06 2024-09-21T21:43:07.545 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.545 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.545 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:07.545 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826889+0000", 2024-09-21T21:43:07.545 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5589999999999996e-06 2024-09-21T21:43:07.545 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.546 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.546 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:07.546 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898301+0000", 2024-09-21T21:43:07.546 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071412188000000001 2024-09-21T21:43:07.546 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.546 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.546 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:07.546 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898416+0000", 2024-09-21T21:43:07.546 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000115538 2024-09-21T21:43:07.546 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.546 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.546 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:07.546 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898513+0000", 2024-09-21T21:43:07.546 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6800999999999997e-05 2024-09-21T21:43:07.546 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:07.547 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:07.547 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:07.547 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.547 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.547 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13578 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 2502656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:07.547 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.826896+0000", 2024-09-21T21:43:07.547 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 76.666202695999999, 2024-09-21T21:43:07.547 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071789411999999997, 2024-09-21T21:43:07.547 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:07.547 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:07.547 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:07.547 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:07.547 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:07.547 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:07.547 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13578 2024-09-21T21:43:07.548 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.548 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:07.548 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.548 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:07.548 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826896+0000", 2024-09-21T21:43:07.548 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:07.548 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.548 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.548 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:07.548 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826896+0000", 2024-09-21T21:43:07.548 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:07.548 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.548 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.548 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:07.548 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827327+0000", 2024-09-21T21:43:07.549 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00043094500000000002 2024-09-21T21:43:07.549 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.549 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.549 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:07.549 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827331+0000", 2024-09-21T21:43:07.549 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.0969999999999999e-06 2024-09-21T21:43:07.549 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.549 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.549 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:07.549 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827341+0000", 2024-09-21T21:43:07.549 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0199e-05 2024-09-21T21:43:07.549 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.549 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.549 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:07.549 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827346+0000", 2024-09-21T21:43:07.550 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3210000000000004e-06 2024-09-21T21:43:07.550 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.550 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.550 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:07.550 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898540+0000", 2024-09-21T21:43:07.550 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071194246000000003 2024-09-21T21:43:07.550 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.550 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.550 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:07.550 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898598+0000", 2024-09-21T21:43:07.550 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7658999999999998e-05 2024-09-21T21:43:07.550 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.550 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.550 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:07.550 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898685+0000", 2024-09-21T21:43:07.551 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.6945000000000002e-05 2024-09-21T21:43:07.551 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:07.551 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:07.551 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:07.551 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.551 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.551 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13579 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 2531328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:07.551 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.827355+0000", 2024-09-21T21:43:07.551 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 76.665743164999995, 2024-09-21T21:43:07.551 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071650933, 2024-09-21T21:43:07.551 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:07.551 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:07.551 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:07.551 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:07.551 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:07.552 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:07.552 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13579 2024-09-21T21:43:07.552 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.552 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:07.552 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.552 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:07.552 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827355+0000", 2024-09-21T21:43:07.552 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:07.552 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.552 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.552 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:07.552 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827355+0000", 2024-09-21T21:43:07.552 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:07.552 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.552 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.553 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:07.553 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827486+0000", 2024-09-21T21:43:07.553 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00013066799999999999 2024-09-21T21:43:07.553 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.553 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.553 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:07.553 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827490+0000", 2024-09-21T21:43:07.553 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.9009999999999996e-06 2024-09-21T21:43:07.553 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.553 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.553 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:07.553 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827499+0000", 2024-09-21T21:43:07.553 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6209999999999999e-06 2024-09-21T21:43:07.553 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.554 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.554 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:07.554 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827506+0000", 2024-09-21T21:43:07.554 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6490000000000002e-06 2024-09-21T21:43:07.554 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.554 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.554 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:07.554 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898713+0000", 2024-09-21T21:43:07.554 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071206937999999997 2024-09-21T21:43:07.554 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.554 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.554 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:07.554 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898811+0000", 2024-09-21T21:43:07.554 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.8159999999999995e-05 2024-09-21T21:43:07.555 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.555 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.555 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:07.555 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899006+0000", 2024-09-21T21:43:07.555 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00019499599999999999 2024-09-21T21:43:07.555 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:07.555 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:07.555 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:07.555 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.555 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.555 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13580 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3219456~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:07.555 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.827522+0000", 2024-09-21T21:43:07.555 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 76.665576733999998, 2024-09-21T21:43:07.555 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071813298999999997, 2024-09-21T21:43:07.555 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:07.556 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:07.556 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:07.556 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:07.556 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:07.556 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:07.556 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13580 2024-09-21T21:43:07.556 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.556 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:07.556 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.556 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:07.556 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827522+0000", 2024-09-21T21:43:07.556 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:07.556 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.556 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.556 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:07.556 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827522+0000", 2024-09-21T21:43:07.557 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:07.557 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.557 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.557 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:07.557 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827898+0000", 2024-09-21T21:43:07.557 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000376876 2024-09-21T21:43:07.557 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.557 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.557 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:07.557 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827903+0000", 2024-09-21T21:43:07.557 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4129999999999999e-06 2024-09-21T21:43:07.557 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.557 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.557 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:07.557 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827911+0000", 2024-09-21T21:43:07.558 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.4540000000000007e-06 2024-09-21T21:43:07.558 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.558 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.558 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:07.558 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827917+0000", 2024-09-21T21:43:07.558 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2110000000000003e-06 2024-09-21T21:43:07.558 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.558 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.558 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:07.558 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899041+0000", 2024-09-21T21:43:07.558 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071123965999999997 2024-09-21T21:43:07.558 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.558 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.558 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:07.559 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899157+0000", 2024-09-21T21:43:07.559 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011573700000000001 2024-09-21T21:43:07.559 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.559 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.559 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:07.559 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899335+0000", 2024-09-21T21:43:07.559 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00017764199999999999 2024-09-21T21:43:07.559 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:07.559 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:07.559 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:07.559 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.559 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.559 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13581 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3248128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:07.559 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.827929+0000", 2024-09-21T21:43:07.559 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 76.665169335000002, 2024-09-21T21:43:07.560 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071730151000000006, 2024-09-21T21:43:07.560 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:07.560 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:07.560 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:07.560 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:07.560 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:07.560 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:07.560 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13581 2024-09-21T21:43:07.560 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.560 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:07.560 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.560 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:07.560 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827929+0000", 2024-09-21T21:43:07.560 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:07.560 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.561 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.561 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:07.561 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827929+0000", 2024-09-21T21:43:07.561 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:07.561 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.561 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.561 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:07.561 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.829943+0000", 2024-09-21T21:43:07.561 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0020143750000000001 2024-09-21T21:43:07.561 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.561 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.561 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:07.561 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.829948+0000", 2024-09-21T21:43:07.561 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9400000000000001e-06 2024-09-21T21:43:07.561 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.562 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.562 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:07.562 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.829960+0000", 2024-09-21T21:43:07.562 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.2123999999999999e-05 2024-09-21T21:43:07.562 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.562 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.562 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:07.562 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.829968+0000", 2024-09-21T21:43:07.562 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7000000000000008e-06 2024-09-21T21:43:07.562 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.562 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.562 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:07.562 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899370+0000", 2024-09-21T21:43:07.562 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.069401672999999997 2024-09-21T21:43:07.563 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.563 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.563 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:07.563 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899481+0000", 2024-09-21T21:43:07.563 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011145300000000001 2024-09-21T21:43:07.563 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.563 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.563 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:07.563 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899659+0000", 2024-09-21T21:43:07.563 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00017788600000000001 2024-09-21T21:43:07.563 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:07.563 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:07.563 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:07.563 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.563 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.564 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13595 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 1871872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:07.564 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.830180+0000", 2024-09-21T21:43:07.564 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 76.662918220999998, 2024-09-21T21:43:07.564 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071809244999999994, 2024-09-21T21:43:07.564 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:07.564 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:07.564 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:07.564 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:07.564 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:07.564 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:07.564 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13595 2024-09-21T21:43:07.564 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.564 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:07.564 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.564 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:07.565 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.830180+0000", 2024-09-21T21:43:07.565 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:07.565 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.565 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.565 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:07.565 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.830180+0000", 2024-09-21T21:43:07.565 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:07.565 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.565 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.565 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:07.565 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.831967+0000", 2024-09-21T21:43:07.565 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0017868350000000001 2024-09-21T21:43:07.565 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.565 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.566 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:07.566 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.831971+0000", 2024-09-21T21:43:07.566 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3950000000000003e-06 2024-09-21T21:43:07.566 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.566 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.566 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:07.566 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.831983+0000", 2024-09-21T21:43:07.566 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1815e-05 2024-09-21T21:43:07.566 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.566 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.566 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:07.566 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.831993+0000", 2024-09-21T21:43:07.566 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6339999999999996e-06 2024-09-21T21:43:07.566 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.567 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.567 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:07.567 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.901693+0000", 2024-09-21T21:43:07.567 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.069700082999999996 2024-09-21T21:43:07.567 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.567 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.567 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:07.567 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.901813+0000", 2024-09-21T21:43:07.567 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011971 2024-09-21T21:43:07.567 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.567 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.567 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:07.567 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.901989+0000", 2024-09-21T21:43:07.567 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000176773 2024-09-21T21:43:07.567 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:07.568 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:07.568 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:07.568 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.568 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.568 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13601 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3334144~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:07.568 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.832130+0000", 2024-09-21T21:43:07.568 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 76.660968349000001, 2024-09-21T21:43:07.568 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071861727, 2024-09-21T21:43:07.568 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:07.568 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:07.568 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:07.568 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:07.568 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:07.568 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:07.569 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13601 2024-09-21T21:43:07.569 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.569 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:07.569 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.569 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:07.569 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.832130+0000", 2024-09-21T21:43:07.569 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:07.569 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.569 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.569 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:07.569 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.832130+0000", 2024-09-21T21:43:07.569 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:07.569 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.569 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.569 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:07.569 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.833939+0000", 2024-09-21T21:43:07.570 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0018094400000000001 2024-09-21T21:43:07.570 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.570 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.570 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:07.570 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.833944+0000", 2024-09-21T21:43:07.570 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5979999999999999e-06 2024-09-21T21:43:07.570 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.570 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.570 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:07.570 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.833953+0000", 2024-09-21T21:43:07.570 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.1460000000000002e-06 2024-09-21T21:43:07.570 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.570 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.570 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:07.571 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.833960+0000", 2024-09-21T21:43:07.571 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.7989999999999996e-06 2024-09-21T21:43:07.571 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.571 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.571 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:07.571 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.903693+0000", 2024-09-21T21:43:07.571 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.069732901 2024-09-21T21:43:07.571 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.571 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.571 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:07.571 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.903814+0000", 2024-09-21T21:43:07.571 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000121585 2024-09-21T21:43:07.571 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.571 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.572 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:07.572 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.903992+0000", 2024-09-21T21:43:07.572 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00017725799999999999 2024-09-21T21:43:07.572 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:07.572 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:07.572 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:07.572 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.572 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.572 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:12342 3.1 3:87417293:::rbd_data.1121faebdfe3.0000000000000004:head [read 4096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:43:07.572 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:14.411725+0000", 2024-09-21T21:43:07.572 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 53.081373282999998, 2024-09-21T21:43:07.572 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0089213029999999, 2024-09-21T21:43:07.572 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:07.572 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:07.572 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:07.573 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:07.573 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:43:07.573 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:43:07.573 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 12342 2024-09-21T21:43:07.573 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.573 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:07.573 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.573 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:07.573 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411725+0000", 2024-09-21T21:43:07.573 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:07.573 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.573 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.573 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:07.573 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411725+0000", 2024-09-21T21:43:07.573 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:07.574 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.574 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.574 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:07.574 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411726+0000", 2024-09-21T21:43:07.574 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.4169999999999999e-06 2024-09-21T21:43:07.574 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.574 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.574 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:07.574 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411734+0000", 2024-09-21T21:43:07.574 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.3799999999999996e-06 2024-09-21T21:43:07.574 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.574 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.574 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:07.574 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411750+0000", 2024-09-21T21:43:07.575 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.6429000000000001e-05 2024-09-21T21:43:07.575 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.575 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.575 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:07.575 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411760+0000", 2024-09-21T21:43:07.575 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.4239999999999999e-06 2024-09-21T21:43:07.575 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.575 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.575 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:07.575 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411828+0000", 2024-09-21T21:43:07.575 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.7935999999999995e-05 2024-09-21T21:43:07.575 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.575 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.575 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op must wait for map", 2024-09-21T21:43:07.575 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411837+0000", 2024-09-21T21:43:07.575 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.0100000000000001e-06 2024-09-21T21:43:07.576 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.576 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.576 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:07.576 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.414984+0000", 2024-09-21T21:43:07.576 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0031472050000000001 2024-09-21T21:43:07.576 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.576 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.576 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for peered", 2024-09-21T21:43:07.576 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.414994+0000", 2024-09-21T21:43:07.576 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0291e-05 2024-09-21T21:43:07.576 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.576 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.576 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:07.576 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.420353+0000", 2024-09-21T21:43:07.576 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.005358913 2024-09-21T21:43:07.577 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.577 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.577 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:07.577 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.420489+0000", 2024-09-21T21:43:07.577 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00013592699999999999 2024-09-21T21:43:07.577 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.577 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.577 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:07.577 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.420646+0000", 2024-09-21T21:43:07.577 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00015737100000000001 2024-09-21T21:43:07.577 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:07.577 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:07.577 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:07.577 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.577 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.578 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:12348 3.1 3:87417293:::rbd_data.1121faebdfe3.0000000000000004:head [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:43:07.578 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:14.411793+0000", 2024-09-21T21:43:07.578 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 53.081305737000001, 2024-09-21T21:43:07.578 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.009298384, 2024-09-21T21:43:07.578 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:07.578 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:07.578 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:07.578 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:07.578 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:43:07.578 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:43:07.578 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 12348 2024-09-21T21:43:07.578 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.578 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:07.578 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.578 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:07.579 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411793+0000", 2024-09-21T21:43:07.579 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:07.579 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.579 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.579 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:07.579 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411793+0000", 2024-09-21T21:43:07.579 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:07.579 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.579 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.579 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:07.579 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411793+0000", 2024-09-21T21:43:07.579 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0999999999999998e-07 2024-09-21T21:43:07.579 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.579 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.580 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:07.580 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411797+0000", 2024-09-21T21:43:07.580 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.49e-06 2024-09-21T21:43:07.580 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.580 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.580 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:07.580 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411809+0000", 2024-09-21T21:43:07.580 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.2510999999999999e-05 2024-09-21T21:43:07.580 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.580 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.580 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:07.580 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411816+0000", 2024-09-21T21:43:07.580 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6420000000000001e-06 2024-09-21T21:43:07.580 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.580 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.581 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:07.581 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411963+0000", 2024-09-21T21:43:07.581 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00014700999999999999 2024-09-21T21:43:07.581 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.581 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.581 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting_for_map not empty", 2024-09-21T21:43:07.581 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411970+0000", 2024-09-21T21:43:07.581 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9800000000000001e-06 2024-09-21T21:43:07.581 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.581 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.581 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:07.581 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.415018+0000", 2024-09-21T21:43:07.581 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0030482299999999999 2024-09-21T21:43:07.581 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.582 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.582 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for peered", 2024-09-21T21:43:07.582 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.415025+0000", 2024-09-21T21:43:07.582 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.3749999999999997e-06 2024-09-21T21:43:07.582 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.582 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.582 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:07.582 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.420904+0000", 2024-09-21T21:43:07.582 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0058787339999999 2024-09-21T21:43:07.582 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.582 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.582 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:07.582 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.420988+0000", 2024-09-21T21:43:07.582 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.3955999999999994e-05 2024-09-21T21:43:07.583 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.583 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.583 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:07.583 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.421091+0000", 2024-09-21T21:43:07.583 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000102846 2024-09-21T21:43:07.583 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:07.583 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:07.583 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:07.583 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.583 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.583 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4385.0:20922 3.9 3:920c5256:::rbd_id.image_client.0-clone36:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-21T21:43:07.583 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:17.811324+0000", 2024-09-21T21:43:07.583 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 49.681774005000001, 2024-09-21T21:43:07.583 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.66951882299999999, 2024-09-21T21:43:07.583 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:07.584 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:07.584 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:07.584 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:07.584 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4385", 2024-09-21T21:43:07.584 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:43:07.584 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 20922 2024-09-21T21:43:07.584 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.584 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:07.584 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.584 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:07.584 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:17.811324+0000", 2024-09-21T21:43:07.584 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:07.584 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.584 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.585 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:07.585 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:17.811324+0000", 2024-09-21T21:43:07.585 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:07.585 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.585 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.585 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:07.585 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:17.811326+0000", 2024-09-21T21:43:07.585 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.536e-06 2024-09-21T21:43:07.585 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.585 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.585 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:07.585 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:17.811332+0000", 2024-09-21T21:43:07.585 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6699999999999997e-06 2024-09-21T21:43:07.585 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.585 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.585 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:07.586 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:17.811347+0000", 2024-09-21T21:43:07.586 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.4589999999999999e-05 2024-09-21T21:43:07.586 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.586 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.586 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:07.586 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:17.811359+0000", 2024-09-21T21:43:07.586 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1776e-05 2024-09-21T21:43:07.586 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.586 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.586 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:07.586 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:17.811424+0000", 2024-09-21T21:43:07.586 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4795000000000006e-05 2024-09-21T21:43:07.586 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.586 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.586 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for peered", 2024-09-21T21:43:07.587 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:17.811436+0000", 2024-09-21T21:43:07.587 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.221e-05 2024-09-21T21:43:07.587 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.587 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.587 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:07.587 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:18.480657+0000", 2024-09-21T21:43:07.587 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.66922088999999996 2024-09-21T21:43:07.587 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.587 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.587 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:07.587 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:18.480760+0000", 2024-09-21T21:43:07.587 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000102827 2024-09-21T21:43:07.587 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:07.587 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:07.587 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:07.588 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:18.480843+0000", 2024-09-21T21:43:07.588 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.3529000000000006e-05 2024-09-21T21:43:07.588 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:07.588 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:07.588 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:07.589 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:07.589 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:07.589 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:43:07.591 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T21:43:07.591 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:43:07.589+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:43:07.691 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:43:07.690+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:43:07.712 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:43:07.712 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:43:07.712 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-21T21:43:07.712 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:43:07.722 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T21:43:07.806 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:43:07.805+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:43:07.869 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:43:07.869 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:43:07.870 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:43:07.870 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:43:07.870 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:43:07.878 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T21:43:07.919 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:43:07.918+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:43:07.996 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:43:07.995+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:43:08.022 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:43:08.022 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-21T21:43:08.022 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-21T21:43:08.022 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:43:08.022 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.023 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19747 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 7905280~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:08.023 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273058+0000", 2024-09-21T21:43:08.023 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 76.746832616999995, 2024-09-21T21:43:08.023 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97338591399999996, 2024-09-21T21:43:08.023 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:08.023 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:08.023 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:08.023 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:08.023 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:08.023 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:08.023 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19747 2024-09-21T21:43:08.023 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.023 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:08.023 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.024 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:08.024 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273058+0000", 2024-09-21T21:43:08.024 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.024 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.024 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.024 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:08.024 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273058+0000", 2024-09-21T21:43:08.024 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.024 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.024 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.024 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:08.024 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273058+0000", 2024-09-21T21:43:08.024 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5900000000000003e-07 2024-09-21T21:43:08.024 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.025 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.025 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:08.025 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273061+0000", 2024-09-21T21:43:08.025 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3479999999999998e-06 2024-09-21T21:43:08.025 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.025 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.025 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:08.025 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273067+0000", 2024-09-21T21:43:08.025 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1469999999999998e-06 2024-09-21T21:43:08.025 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.025 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.025 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:08.025 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273071+0000", 2024-09-21T21:43:08.025 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7339999999999999e-06 2024-09-21T21:43:08.026 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.026 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.026 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:08.026 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277945+0000", 2024-09-21T21:43:08.026 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.004873474 2024-09-21T21:43:08.026 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.026 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.026 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:08.026 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277948+0000", 2024-09-21T21:43:08.026 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0299999999999998e-06 2024-09-21T21:43:08.026 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.026 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.026 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:08.027 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246269+0000", 2024-09-21T21:43:08.027 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96832060200000003 2024-09-21T21:43:08.027 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.027 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.027 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:08.027 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246362+0000", 2024-09-21T21:43:08.027 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3867000000000001e-05 2024-09-21T21:43:08.027 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.027 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.027 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:08.027 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246444+0000", 2024-09-21T21:43:08.027 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1353000000000005e-05 2024-09-21T21:43:08.027 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.028 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:08.028 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.028 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.028 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.028 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19755 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 5783552~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:08.028 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273077+0000", 2024-09-21T21:43:08.028 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 76.746813001000007, 2024-09-21T21:43:08.028 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97350203400000002, 2024-09-21T21:43:08.028 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:08.028 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:08.028 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:08.028 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:08.028 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:08.029 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:08.029 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19755 2024-09-21T21:43:08.029 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.029 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:08.029 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.029 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:08.029 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273077+0000", 2024-09-21T21:43:08.029 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.029 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.029 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.029 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:08.029 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273077+0000", 2024-09-21T21:43:08.029 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.029 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.029 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.030 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:08.030 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273078+0000", 2024-09-21T21:43:08.030 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5699999999999998e-07 2024-09-21T21:43:08.030 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.030 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.030 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:08.030 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273083+0000", 2024-09-21T21:43:08.030 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8670000000000002e-06 2024-09-21T21:43:08.030 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.030 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.030 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:08.030 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273088+0000", 2024-09-21T21:43:08.030 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8370000000000002e-06 2024-09-21T21:43:08.030 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.031 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.031 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:08.031 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273093+0000", 2024-09-21T21:43:08.031 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5709999999999999e-06 2024-09-21T21:43:08.031 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.031 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.031 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:08.031 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277968+0000", 2024-09-21T21:43:08.031 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048753909999999998 2024-09-21T21:43:08.031 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.031 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.031 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:08.031 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277971+0000", 2024-09-21T21:43:08.031 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0189999999999998e-06 2024-09-21T21:43:08.031 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.032 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.032 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:08.032 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246485+0000", 2024-09-21T21:43:08.032 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96851399299999996 2024-09-21T21:43:08.032 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.032 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.032 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:08.032 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246536+0000", 2024-09-21T21:43:08.032 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0430000000000003e-05 2024-09-21T21:43:08.032 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.032 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.032 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:08.032 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246579+0000", 2024-09-21T21:43:08.032 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3569000000000002e-05 2024-09-21T21:43:08.033 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.033 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:08.033 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.033 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.033 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.033 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19756 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6500352~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:08.033 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273099+0000", 2024-09-21T21:43:08.033 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 76.746791627999997, 2024-09-21T21:43:08.033 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97359272799999996, 2024-09-21T21:43:08.033 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:08.033 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:08.033 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:08.033 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:08.033 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:08.033 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:08.034 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19756 2024-09-21T21:43:08.034 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.034 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:08.034 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.034 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:08.034 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273099+0000", 2024-09-21T21:43:08.034 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.034 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.034 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.034 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:08.034 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273099+0000", 2024-09-21T21:43:08.034 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.034 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.034 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.035 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:08.035 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273099+0000", 2024-09-21T21:43:08.035 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0900000000000002e-07 2024-09-21T21:43:08.035 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.035 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.035 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:08.035 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273101+0000", 2024-09-21T21:43:08.035 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0320000000000002e-06 2024-09-21T21:43:08.035 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.035 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.035 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:08.035 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273107+0000", 2024-09-21T21:43:08.035 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2830000000000001e-06 2024-09-21T21:43:08.035 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.035 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.036 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:08.036 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273111+0000", 2024-09-21T21:43:08.036 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2230000000000001e-06 2024-09-21T21:43:08.036 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.036 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.036 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:08.036 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277994+0000", 2024-09-21T21:43:08.036 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048837070000000002 2024-09-21T21:43:08.036 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.036 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.036 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:08.036 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278000+0000", 2024-09-21T21:43:08.036 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3149999999999999e-06 2024-09-21T21:43:08.036 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.037 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.037 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:08.037 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246607+0000", 2024-09-21T21:43:08.037 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.968606839 2024-09-21T21:43:08.037 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.037 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.037 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:08.037 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246651+0000", 2024-09-21T21:43:08.037 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4734999999999998e-05 2024-09-21T21:43:08.037 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.037 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.037 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:08.037 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246692+0000", 2024-09-21T21:43:08.037 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0185000000000002e-05 2024-09-21T21:43:08.037 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.038 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:08.038 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.038 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.038 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.038 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19757 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 5812224~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:08.038 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273116+0000", 2024-09-21T21:43:08.038 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 76.746773970999996, 2024-09-21T21:43:08.038 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97367813999999997, 2024-09-21T21:43:08.038 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:08.038 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:08.038 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:08.038 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:08.038 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:08.038 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:08.038 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19757 2024-09-21T21:43:08.039 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.039 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:08.039 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.039 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:08.039 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273116+0000", 2024-09-21T21:43:08.039 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.039 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.039 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.039 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:08.039 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273116+0000", 2024-09-21T21:43:08.039 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.039 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.039 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.039 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:08.040 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273117+0000", 2024-09-21T21:43:08.040 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0999999999999999e-07 2024-09-21T21:43:08.040 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.040 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.040 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:08.040 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273119+0000", 2024-09-21T21:43:08.040 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9970000000000001e-06 2024-09-21T21:43:08.040 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.040 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.040 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:08.040 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273125+0000", 2024-09-21T21:43:08.040 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9540000000000001e-06 2024-09-21T21:43:08.040 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.040 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.041 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:08.041 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273135+0000", 2024-09-21T21:43:08.041 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0597999999999999e-05 2024-09-21T21:43:08.041 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.041 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.041 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:08.041 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278019+0000", 2024-09-21T21:43:08.041 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048834849999999999 2024-09-21T21:43:08.041 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.041 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.041 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:08.041 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278022+0000", 2024-09-21T21:43:08.041 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9560000000000002e-06 2024-09-21T21:43:08.041 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.041 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.042 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:08.042 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246716+0000", 2024-09-21T21:43:08.042 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.968694202 2024-09-21T21:43:08.042 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.042 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.042 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:08.042 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246758+0000", 2024-09-21T21:43:08.042 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1607000000000003e-05 2024-09-21T21:43:08.042 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.042 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.042 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:08.042 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246795+0000", 2024-09-21T21:43:08.042 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6931000000000003e-05 2024-09-21T21:43:08.042 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.043 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:08.043 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.043 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.043 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.043 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19758 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6529024~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:08.043 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273142+0000", 2024-09-21T21:43:08.043 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 76.746748749999995, 2024-09-21T21:43:08.043 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97375924599999997, 2024-09-21T21:43:08.043 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:08.043 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:08.043 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:08.043 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:08.043 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:08.043 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:08.044 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19758 2024-09-21T21:43:08.044 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.044 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:08.044 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.044 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:08.044 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273142+0000", 2024-09-21T21:43:08.044 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.044 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.044 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.044 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:08.044 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273142+0000", 2024-09-21T21:43:08.044 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.044 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.044 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.045 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:08.045 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273142+0000", 2024-09-21T21:43:08.045 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0200000000000003e-07 2024-09-21T21:43:08.045 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.045 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.045 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:08.045 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273144+0000", 2024-09-21T21:43:08.045 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1390000000000002e-06 2024-09-21T21:43:08.045 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.045 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.045 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:08.045 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273150+0000", 2024-09-21T21:43:08.045 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6409999999999999e-06 2024-09-21T21:43:08.045 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.046 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.046 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:08.046 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273157+0000", 2024-09-21T21:43:08.046 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1359999999999999e-06 2024-09-21T21:43:08.046 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.046 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.046 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:08.046 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278050+0000", 2024-09-21T21:43:08.046 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048926170000000001 2024-09-21T21:43:08.046 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.046 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.046 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:08.046 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278053+0000", 2024-09-21T21:43:08.046 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4879999999999999e-06 2024-09-21T21:43:08.047 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.047 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.047 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:08.047 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246819+0000", 2024-09-21T21:43:08.047 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96876590399999996 2024-09-21T21:43:08.047 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.047 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.047 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:08.047 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246856+0000", 2024-09-21T21:43:08.047 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7287999999999999e-05 2024-09-21T21:43:08.047 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.047 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.047 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:08.047 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246901+0000", 2024-09-21T21:43:08.047 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4631e-05 2024-09-21T21:43:08.048 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.048 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:08.048 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.048 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.048 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.048 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19759 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 7933952~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:08.048 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273172+0000", 2024-09-21T21:43:08.048 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 76.746718486000006, 2024-09-21T21:43:08.048 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97384789500000002, 2024-09-21T21:43:08.048 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:08.048 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:08.048 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:08.048 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:08.048 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:08.048 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:08.049 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19759 2024-09-21T21:43:08.049 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.049 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:08.049 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.049 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:08.049 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273172+0000", 2024-09-21T21:43:08.049 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.049 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.049 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.049 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:08.049 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273172+0000", 2024-09-21T21:43:08.049 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.049 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.049 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.050 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:08.050 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273172+0000", 2024-09-21T21:43:08.050 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6100000000000002e-07 2024-09-21T21:43:08.050 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.050 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.050 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:08.050 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273174+0000", 2024-09-21T21:43:08.050 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.691e-06 2024-09-21T21:43:08.050 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.050 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.050 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:08.050 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273178+0000", 2024-09-21T21:43:08.050 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7579999999999998e-06 2024-09-21T21:43:08.050 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.050 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.051 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:08.051 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273180+0000", 2024-09-21T21:43:08.051 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7199999999999998e-06 2024-09-21T21:43:08.051 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.051 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.051 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:08.051 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278077+0000", 2024-09-21T21:43:08.051 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048969529999999999 2024-09-21T21:43:08.051 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.051 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.051 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:08.051 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278080+0000", 2024-09-21T21:43:08.051 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.066e-06 2024-09-21T21:43:08.052 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.052 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.052 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:08.052 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246925+0000", 2024-09-21T21:43:08.052 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96884490199999995 2024-09-21T21:43:08.052 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.052 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.052 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:08.052 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246974+0000", 2024-09-21T21:43:08.052 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8387999999999998e-05 2024-09-21T21:43:08.052 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.052 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.052 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:08.052 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247020+0000", 2024-09-21T21:43:08.053 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6156000000000003e-05 2024-09-21T21:43:08.053 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.053 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:08.053 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.053 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.053 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.053 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19760 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 7991296~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:08.053 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273220+0000", 2024-09-21T21:43:08.053 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 76.746669972000007, 2024-09-21T21:43:08.053 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97403125499999998, 2024-09-21T21:43:08.053 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:08.053 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:08.053 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:08.053 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:08.054 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:08.054 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:08.054 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19760 2024-09-21T21:43:08.054 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.054 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:08.054 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.054 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:08.054 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273220+0000", 2024-09-21T21:43:08.054 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.054 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.054 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.054 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:08.054 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273220+0000", 2024-09-21T21:43:08.054 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.055 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.055 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.055 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:08.055 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273221+0000", 2024-09-21T21:43:08.055 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.65e-07 2024-09-21T21:43:08.055 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.055 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.055 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:08.055 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273228+0000", 2024-09-21T21:43:08.055 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6040000000000004e-06 2024-09-21T21:43:08.055 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.055 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.055 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:08.055 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273233+0000", 2024-09-21T21:43:08.056 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1569999999999999e-06 2024-09-21T21:43:08.056 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.056 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.056 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:08.056 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273238+0000", 2024-09-21T21:43:08.056 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0050000000000004e-06 2024-09-21T21:43:08.056 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.056 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.056 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:08.056 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278107+0000", 2024-09-21T21:43:08.056 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00486857 2024-09-21T21:43:08.056 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.056 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.057 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:08.057 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278111+0000", 2024-09-21T21:43:08.057 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6389999999999998e-06 2024-09-21T21:43:08.057 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.057 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.057 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:08.057 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247069+0000", 2024-09-21T21:43:08.057 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96895796899999997 2024-09-21T21:43:08.057 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.057 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.057 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:08.057 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247166+0000", 2024-09-21T21:43:08.057 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.7158999999999996e-05 2024-09-21T21:43:08.057 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.057 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.058 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:08.058 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247252+0000", 2024-09-21T21:43:08.058 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5886999999999999e-05 2024-09-21T21:43:08.058 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.058 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:08.058 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.058 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.058 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.058 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19761 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 7962624~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:08.058 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273246+0000", 2024-09-21T21:43:08.058 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 76.746644911999994, 2024-09-21T21:43:08.058 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97415972200000001, 2024-09-21T21:43:08.058 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:08.059 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:08.059 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:08.059 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:08.059 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:08.059 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:08.059 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19761 2024-09-21T21:43:08.059 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.059 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:08.059 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.059 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:08.059 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273246+0000", 2024-09-21T21:43:08.059 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.060 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.060 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.060 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:08.060 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273246+0000", 2024-09-21T21:43:08.060 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.060 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.060 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.060 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:08.060 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273246+0000", 2024-09-21T21:43:08.060 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5100000000000001e-07 2024-09-21T21:43:08.060 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.060 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.060 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:08.060 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273248+0000", 2024-09-21T21:43:08.061 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1550000000000001e-06 2024-09-21T21:43:08.061 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.061 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.061 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:08.061 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273254+0000", 2024-09-21T21:43:08.061 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.964e-06 2024-09-21T21:43:08.061 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.061 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.061 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:08.061 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273259+0000", 2024-09-21T21:43:08.061 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5499999999999996e-06 2024-09-21T21:43:08.061 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.061 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.061 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:08.062 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278130+0000", 2024-09-21T21:43:08.062 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048715779999999997 2024-09-21T21:43:08.062 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.062 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.062 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:08.062 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278133+0000", 2024-09-21T21:43:08.062 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8739999999999999e-06 2024-09-21T21:43:08.062 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.062 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.062 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:08.062 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247270+0000", 2024-09-21T21:43:08.062 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.969137099 2024-09-21T21:43:08.062 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.062 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.063 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:08.063 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247338+0000", 2024-09-21T21:43:08.063 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7717000000000002e-05 2024-09-21T21:43:08.063 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.063 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.063 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:08.063 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247405+0000", 2024-09-21T21:43:08.063 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7434000000000005e-05 2024-09-21T21:43:08.063 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.063 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:08.063 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.063 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.063 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.063 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19769 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 5840896~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:08.064 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273287+0000", 2024-09-21T21:43:08.064 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 76.746603628000003, 2024-09-21T21:43:08.064 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97435390200000005, 2024-09-21T21:43:08.064 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:08.064 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:08.064 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:08.064 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:08.064 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:08.064 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:08.064 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19769 2024-09-21T21:43:08.064 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.064 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:08.064 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.064 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:08.065 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273287+0000", 2024-09-21T21:43:08.065 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.065 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.065 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.065 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:08.065 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273287+0000", 2024-09-21T21:43:08.065 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.065 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.065 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.065 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:08.065 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273287+0000", 2024-09-21T21:43:08.065 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6699999999999999e-07 2024-09-21T21:43:08.065 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.065 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.066 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:08.066 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273289+0000", 2024-09-21T21:43:08.066 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.708e-06 2024-09-21T21:43:08.066 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.066 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.066 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:08.066 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273296+0000", 2024-09-21T21:43:08.066 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8109999999999997e-06 2024-09-21T21:43:08.066 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.066 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.066 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:08.066 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273299+0000", 2024-09-21T21:43:08.066 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0740000000000001e-06 2024-09-21T21:43:08.066 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.067 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.067 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:08.067 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278159+0000", 2024-09-21T21:43:08.067 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048603240000000001 2024-09-21T21:43:08.067 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.067 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.067 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:08.067 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278162+0000", 2024-09-21T21:43:08.067 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1930000000000002e-06 2024-09-21T21:43:08.067 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.067 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.067 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:08.067 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247444+0000", 2024-09-21T21:43:08.067 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96928199100000001 2024-09-21T21:43:08.068 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.068 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.068 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:08.068 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247533+0000", 2024-09-21T21:43:08.068 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8490000000000001e-05 2024-09-21T21:43:08.068 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.068 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.068 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:08.068 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247641+0000", 2024-09-21T21:43:08.068 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000107944 2024-09-21T21:43:08.068 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.068 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:08.068 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.068 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.069 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.069 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19770 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 5869568~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:08.069 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273313+0000", 2024-09-21T21:43:08.069 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 76.746577525999996, 2024-09-21T21:43:08.069 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97449038899999996, 2024-09-21T21:43:08.069 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:08.069 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:08.069 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:08.069 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:08.069 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:08.069 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:08.069 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19770 2024-09-21T21:43:08.069 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.069 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:08.070 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.070 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:08.070 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273313+0000", 2024-09-21T21:43:08.070 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.070 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.070 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.070 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:08.070 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273313+0000", 2024-09-21T21:43:08.070 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.070 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.070 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.070 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:08.070 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273313+0000", 2024-09-21T21:43:08.070 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5699999999999998e-07 2024-09-21T21:43:08.071 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.071 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.071 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:08.071 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273315+0000", 2024-09-21T21:43:08.071 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0099999999999998e-06 2024-09-21T21:43:08.071 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.071 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.071 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:08.071 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273321+0000", 2024-09-21T21:43:08.071 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7529999999999998e-06 2024-09-21T21:43:08.071 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.071 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.071 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:08.071 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273328+0000", 2024-09-21T21:43:08.072 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9009999999999997e-06 2024-09-21T21:43:08.072 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.072 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.072 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:08.072 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278183+0000", 2024-09-21T21:43:08.072 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048547770000000002 2024-09-21T21:43:08.072 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.072 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.072 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:08.072 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278186+0000", 2024-09-21T21:43:08.072 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3100000000000001e-06 2024-09-21T21:43:08.072 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.072 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.072 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:08.073 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247676+0000", 2024-09-21T21:43:08.073 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96949004299999997 2024-09-21T21:43:08.073 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.073 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.073 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:08.073 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247757+0000", 2024-09-21T21:43:08.073 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0479e-05 2024-09-21T21:43:08.073 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.073 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.073 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:08.073 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247803+0000", 2024-09-21T21:43:08.073 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6758999999999997e-05 2024-09-21T21:43:08.073 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.073 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:08.074 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.074 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.074 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.074 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19771 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6586368~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:08.074 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273335+0000", 2024-09-21T21:43:08.074 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 76.746555739000001, 2024-09-21T21:43:08.074 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97458947699999998, 2024-09-21T21:43:08.074 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:08.074 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:08.074 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:08.074 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:08.074 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:08.074 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:08.074 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19771 2024-09-21T21:43:08.075 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.075 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:08.075 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.075 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:08.075 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273335+0000", 2024-09-21T21:43:08.075 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.075 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.075 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.075 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:08.075 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273335+0000", 2024-09-21T21:43:08.075 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.075 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.075 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.075 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:08.076 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273335+0000", 2024-09-21T21:43:08.076 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3599999999999999e-07 2024-09-21T21:43:08.076 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.076 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.076 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:08.076 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273337+0000", 2024-09-21T21:43:08.076 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0789999999999999e-06 2024-09-21T21:43:08.076 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.076 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.076 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:08.076 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273343+0000", 2024-09-21T21:43:08.076 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6280000000000002e-06 2024-09-21T21:43:08.076 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.076 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.076 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:08.077 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273346+0000", 2024-09-21T21:43:08.077 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.507e-06 2024-09-21T21:43:08.077 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.077 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.077 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:08.077 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278206+0000", 2024-09-21T21:43:08.077 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048600969999999999 2024-09-21T21:43:08.077 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.077 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.077 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:08.077 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278212+0000", 2024-09-21T21:43:08.077 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2079999999999999e-06 2024-09-21T21:43:08.078 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.078 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.078 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:08.078 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247828+0000", 2024-09-21T21:43:08.078 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.969616547 2024-09-21T21:43:08.078 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.078 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.078 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:08.078 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247881+0000", 2024-09-21T21:43:08.078 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3180999999999998e-05 2024-09-21T21:43:08.078 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.078 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.078 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:08.078 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247924+0000", 2024-09-21T21:43:08.079 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2894e-05 2024-09-21T21:43:08.079 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.079 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:08.079 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.079 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.079 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.079 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19772 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6557696~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:08.079 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273358+0000", 2024-09-21T21:43:08.079 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 76.746532359, 2024-09-21T21:43:08.079 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97467031800000004, 2024-09-21T21:43:08.079 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:08.079 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:08.079 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:08.079 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:08.079 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:08.080 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:08.080 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19772 2024-09-21T21:43:08.080 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.080 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:08.080 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.080 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:08.080 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273358+0000", 2024-09-21T21:43:08.080 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.080 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.080 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.080 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:08.080 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273358+0000", 2024-09-21T21:43:08.080 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.080 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.081 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.081 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:08.081 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273358+0000", 2024-09-21T21:43:08.081 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4499999999999998e-07 2024-09-21T21:43:08.081 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.081 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.081 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:08.081 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273360+0000", 2024-09-21T21:43:08.081 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9190000000000002e-06 2024-09-21T21:43:08.081 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.081 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.081 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:08.081 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273366+0000", 2024-09-21T21:43:08.081 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.891e-06 2024-09-21T21:43:08.082 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.082 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.082 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:08.082 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273371+0000", 2024-09-21T21:43:08.082 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1880000000000001e-06 2024-09-21T21:43:08.082 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.082 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.082 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:08.082 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278237+0000", 2024-09-21T21:43:08.082 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.004865472 2024-09-21T21:43:08.082 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.082 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.082 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:08.082 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278240+0000", 2024-09-21T21:43:08.083 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9550000000000001e-06 2024-09-21T21:43:08.083 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.083 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.083 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:08.083 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247949+0000", 2024-09-21T21:43:08.083 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.969709127 2024-09-21T21:43:08.083 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.083 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.083 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:08.083 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247990+0000", 2024-09-21T21:43:08.083 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1202000000000003e-05 2024-09-21T21:43:08.083 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.083 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.083 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:08.083 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248028+0000", 2024-09-21T21:43:08.084 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8219000000000001e-05 2024-09-21T21:43:08.084 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.084 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:08.084 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.084 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.084 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.084 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19773 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6615040~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:08.084 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273390+0000", 2024-09-21T21:43:08.084 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 76.746500498000003, 2024-09-21T21:43:08.084 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97476086200000001, 2024-09-21T21:43:08.084 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:08.084 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:08.084 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:08.084 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:08.085 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:08.085 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:08.085 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19773 2024-09-21T21:43:08.085 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.085 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:08.085 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.085 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:08.085 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273390+0000", 2024-09-21T21:43:08.085 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.085 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.085 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.085 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:08.085 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273390+0000", 2024-09-21T21:43:08.085 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.085 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.086 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.086 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:08.086 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273390+0000", 2024-09-21T21:43:08.086 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9999999999999999e-07 2024-09-21T21:43:08.086 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.086 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.086 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:08.086 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273392+0000", 2024-09-21T21:43:08.086 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0640000000000001e-06 2024-09-21T21:43:08.086 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.086 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.086 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:08.086 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273397+0000", 2024-09-21T21:43:08.086 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.113e-06 2024-09-21T21:43:08.087 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.087 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.087 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:08.087 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273401+0000", 2024-09-21T21:43:08.087 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3119999999999998e-06 2024-09-21T21:43:08.087 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.087 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.087 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:08.087 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278271+0000", 2024-09-21T21:43:08.087 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048698880000000002 2024-09-21T21:43:08.087 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.087 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.087 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:08.087 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278278+0000", 2024-09-21T21:43:08.088 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1210000000000001e-06 2024-09-21T21:43:08.088 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.088 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.088 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:08.088 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248060+0000", 2024-09-21T21:43:08.088 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96978224999999996 2024-09-21T21:43:08.088 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.088 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.088 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:08.088 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248111+0000", 2024-09-21T21:43:08.088 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.096e-05 2024-09-21T21:43:08.088 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.088 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.089 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:08.089 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248151+0000", 2024-09-21T21:43:08.089 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9854000000000002e-05 2024-09-21T21:43:08.089 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.089 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:08.089 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.089 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.089 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.089 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19784 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6643712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:08.089 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273405+0000", 2024-09-21T21:43:08.089 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 76.746485777000004, 2024-09-21T21:43:08.089 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97491798900000004, 2024-09-21T21:43:08.089 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:08.089 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:08.090 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:08.090 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:08.090 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:08.090 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:08.090 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19784 2024-09-21T21:43:08.090 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.090 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:08.090 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.090 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:08.090 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273405+0000", 2024-09-21T21:43:08.090 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.090 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.090 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.090 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:08.091 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273405+0000", 2024-09-21T21:43:08.091 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.091 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.091 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.091 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:08.091 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273405+0000", 2024-09-21T21:43:08.091 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2100000000000001e-07 2024-09-21T21:43:08.091 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.091 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.091 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:08.091 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273406+0000", 2024-09-21T21:43:08.091 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2049999999999999e-06 2024-09-21T21:43:08.091 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.091 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.091 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:08.092 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273409+0000", 2024-09-21T21:43:08.092 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0599999999999999e-06 2024-09-21T21:43:08.092 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.092 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.092 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:08.092 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273412+0000", 2024-09-21T21:43:08.092 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0730000000000001e-06 2024-09-21T21:43:08.092 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.092 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.092 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:08.092 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278317+0000", 2024-09-21T21:43:08.092 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0049045520000000004 2024-09-21T21:43:08.092 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.092 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.093 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:08.093 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278321+0000", 2024-09-21T21:43:08.093 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6249999999999998e-06 2024-09-21T21:43:08.093 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.093 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.093 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:08.093 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248179+0000", 2024-09-21T21:43:08.093 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96985744500000004 2024-09-21T21:43:08.093 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.093 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.093 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:08.093 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248253+0000", 2024-09-21T21:43:08.093 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3829000000000001e-05 2024-09-21T21:43:08.093 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.093 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.094 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:08.094 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248323+0000", 2024-09-21T21:43:08.094 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9979000000000002e-05 2024-09-21T21:43:08.094 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.094 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:08.094 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.094 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.094 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.094 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19785 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6672384~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:08.094 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273415+0000", 2024-09-21T21:43:08.094 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 76.746475191000002, 2024-09-21T21:43:08.094 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97500777800000005, 2024-09-21T21:43:08.094 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:08.094 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:08.095 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:08.095 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:08.095 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:08.095 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:08.095 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19785 2024-09-21T21:43:08.095 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.095 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:08.095 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.095 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:08.095 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273415+0000", 2024-09-21T21:43:08.095 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.095 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.095 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.095 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:08.096 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273415+0000", 2024-09-21T21:43:08.096 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.096 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.096 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.096 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:08.096 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273415+0000", 2024-09-21T21:43:08.096 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0100000000000001e-07 2024-09-21T21:43:08.096 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.096 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.096 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:08.096 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273417+0000", 2024-09-21T21:43:08.096 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.114e-06 2024-09-21T21:43:08.096 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.096 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.096 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:08.097 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273420+0000", 2024-09-21T21:43:08.097 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6770000000000001e-06 2024-09-21T21:43:08.097 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.097 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.097 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:08.097 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273423+0000", 2024-09-21T21:43:08.097 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5450000000000002e-06 2024-09-21T21:43:08.097 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.097 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.097 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:08.097 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278355+0000", 2024-09-21T21:43:08.097 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0049318590000000002 2024-09-21T21:43:08.097 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.097 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.097 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:08.098 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278360+0000", 2024-09-21T21:43:08.098 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3079999999999998e-06 2024-09-21T21:43:08.098 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.098 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.098 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:08.098 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248349+0000", 2024-09-21T21:43:08.098 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96998909899999997 2024-09-21T21:43:08.098 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.098 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.098 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:08.098 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248387+0000", 2024-09-21T21:43:08.098 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7657000000000002e-05 2024-09-21T21:43:08.098 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.098 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.099 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:08.099 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248423+0000", 2024-09-21T21:43:08.099 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6318e-05 2024-09-21T21:43:08.099 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.099 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:08.099 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.099 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.099 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.099 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19842 3.d 3:b7fa2ca0:::rbd_data.112157a479c.000000000000004d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:08.099 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273426+0000", 2024-09-21T21:43:08.099 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 76.746464138999997, 2024-09-21T21:43:08.099 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.98136526800000001, 2024-09-21T21:43:08.099 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:08.099 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:08.100 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:08.100 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:08.100 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:08.100 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:08.100 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19842 2024-09-21T21:43:08.100 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.100 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:08.100 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.100 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:08.100 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273426+0000", 2024-09-21T21:43:08.100 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.100 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.100 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.100 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:08.100 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273426+0000", 2024-09-21T21:43:08.101 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.101 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.101 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.101 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:08.101 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273427+0000", 2024-09-21T21:43:08.101 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.72e-07 2024-09-21T21:43:08.101 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.101 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.101 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:08.101 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275836+0000", 2024-09-21T21:43:08.101 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00240909 2024-09-21T21:43:08.101 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.101 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.101 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:08.101 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275844+0000", 2024-09-21T21:43:08.102 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3750000000000003e-06 2024-09-21T21:43:08.102 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.102 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.102 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:08.102 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275849+0000", 2024-09-21T21:43:08.102 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0100000000000003e-06 2024-09-21T21:43:08.102 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.102 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.102 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:08.102 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278394+0000", 2024-09-21T21:43:08.102 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025449890000000001 2024-09-21T21:43:08.102 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.102 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.102 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:08.103 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278402+0000", 2024-09-21T21:43:08.103 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6359999999999994e-06 2024-09-21T21:43:08.103 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.103 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.103 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:08.103 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248454+0000", 2024-09-21T21:43:08.103 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97005259499999996 2024-09-21T21:43:08.103 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.103 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.103 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:08.103 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248576+0000", 2024-09-21T21:43:08.103 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000122127 2024-09-21T21:43:08.103 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.103 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.103 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 2", 2024-09-21T21:43:08.104 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248844+0000", 2024-09-21T21:43:08.104 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00026784699999999999 2024-09-21T21:43:08.104 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.104 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.104 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:08.104 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.252487+0000", 2024-09-21T21:43:08.104 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0036427920000000002 2024-09-21T21:43:08.104 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.104 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.104 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:08.104 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.254691+0000", 2024-09-21T21:43:08.104 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002203899 2024-09-21T21:43:08.104 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.104 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.105 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:08.105 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.254748+0000", 2024-09-21T21:43:08.105 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7163000000000001e-05 2024-09-21T21:43:08.105 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.105 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.105 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:08.105 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.254792+0000", 2024-09-21T21:43:08.105 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3473000000000002e-05 2024-09-21T21:43:08.105 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.105 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:08.105 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.105 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.105 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.106 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19843 3.d 3:b7fa2ca0:::rbd_data.112157a479c.000000000000004d:head [write 0~0] snapc 21=[21] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:08.106 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.275860+0000", 2024-09-21T21:43:08.106 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 76.744030238999997, 2024-09-21T21:43:08.106 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.98183651999999999, 2024-09-21T21:43:08.106 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:08.106 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:08.106 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:08.106 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:08.106 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:08.106 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:08.106 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19843 2024-09-21T21:43:08.106 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.106 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:08.106 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.106 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:08.107 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275860+0000", 2024-09-21T21:43:08.107 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.107 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.107 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.107 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:08.107 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275860+0000", 2024-09-21T21:43:08.107 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.107 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.107 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.107 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:08.107 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275869+0000", 2024-09-21T21:43:08.107 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6300000000000004e-06 2024-09-21T21:43:08.107 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.107 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.108 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:08.108 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275871+0000", 2024-09-21T21:43:08.108 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2060000000000001e-06 2024-09-21T21:43:08.108 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.108 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.108 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:08.108 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275876+0000", 2024-09-21T21:43:08.108 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8180000000000003e-06 2024-09-21T21:43:08.108 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.108 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.108 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:08.108 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275880+0000", 2024-09-21T21:43:08.109 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1829999999999996e-06 2024-09-21T21:43:08.109 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.109 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.109 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:08.109 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278416+0000", 2024-09-21T21:43:08.109 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025361149999999998 2024-09-21T21:43:08.109 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.109 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.109 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:08.109 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278419+0000", 2024-09-21T21:43:08.109 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.163e-06 2024-09-21T21:43:08.109 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.109 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.109 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:08.109 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.250249+0000", 2024-09-21T21:43:08.110 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97182984299999997 2024-09-21T21:43:08.110 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.110 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.110 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:08.110 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.250321+0000", 2024-09-21T21:43:08.110 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1656000000000006e-05 2024-09-21T21:43:08.110 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.110 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.110 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 2", 2024-09-21T21:43:08.110 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.250419+0000", 2024-09-21T21:43:08.110 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.8277999999999997e-05 2024-09-21T21:43:08.110 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.110 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.111 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:08.111 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.252516+0000", 2024-09-21T21:43:08.111 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002097224 2024-09-21T21:43:08.111 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.111 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.111 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:08.111 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.257643+0000", 2024-09-21T21:43:08.111 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0051262319999999997 2024-09-21T21:43:08.111 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.111 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.111 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:08.111 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.257673+0000", 2024-09-21T21:43:08.111 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0023e-05 2024-09-21T21:43:08.111 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.112 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.112 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:08.112 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.257697+0000", 2024-09-21T21:43:08.112 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4148999999999999e-05 2024-09-21T21:43:08.112 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.112 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:08.112 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.112 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.112 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.112 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5626.0:12334 3.3 3:c643a86c:::rbd_data.1121faebdfe3.0000000000000002:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:43:08.112 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:14.411628+0000", 2024-09-21T21:43:08.112 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 53.608261939999998, 2024-09-21T21:43:08.112 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0086947559999999, 2024-09-21T21:43:08.112 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:08.113 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:08.113 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:08.113 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:08.113 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5626", 2024-09-21T21:43:08.113 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:43:08.113 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 12334 2024-09-21T21:43:08.113 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.113 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:08.113 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.113 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:08.113 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411628+0000", 2024-09-21T21:43:08.113 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.113 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.113 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.113 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:08.114 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411628+0000", 2024-09-21T21:43:08.114 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.114 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.114 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.114 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:08.114 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411629+0000", 2024-09-21T21:43:08.114 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1200000000000002e-07 2024-09-21T21:43:08.114 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.114 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.114 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:08.114 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411633+0000", 2024-09-21T21:43:08.114 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1699999999999999e-06 2024-09-21T21:43:08.114 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.114 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.115 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:08.115 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411645+0000", 2024-09-21T21:43:08.115 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1328999999999999e-05 2024-09-21T21:43:08.115 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.115 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.115 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:08.115 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411652+0000", 2024-09-21T21:43:08.115 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9990000000000002e-06 2024-09-21T21:43:08.115 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.115 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.115 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:08.115 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411717+0000", 2024-09-21T21:43:08.115 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5519999999999996e-05 2024-09-21T21:43:08.115 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.115 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.116 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op must wait for map", 2024-09-21T21:43:08.116 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411725+0000", 2024-09-21T21:43:08.116 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2609999999999996e-06 2024-09-21T21:43:08.116 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.116 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.116 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:08.116 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415904+0000", 2024-09-21T21:43:08.116 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0041792749999999997 2024-09-21T21:43:08.116 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.116 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.116 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:08.116 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415913+0000", 2024-09-21T21:43:08.116 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.054e-06 2024-09-21T21:43:08.116 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.116 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.117 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:08.117 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420096+0000", 2024-09-21T21:43:08.117 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0041835670000001 2024-09-21T21:43:08.117 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.117 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.117 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:08.117 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420216+0000", 2024-09-21T21:43:08.117 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000120025 2024-09-21T21:43:08.117 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.117 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.117 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:08.117 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420323+0000", 2024-09-21T21:43:08.117 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010674400000000001 2024-09-21T21:43:08.117 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.118 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:08.118 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.118 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.118 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.118 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5626.0:12339 3.3 3:c643a86c:::rbd_data.1121faebdfe3.0000000000000002:head [read 4096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:43:08.118 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:14.411749+0000", 2024-09-21T21:43:08.118 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 53.608141754000002, 2024-09-21T21:43:08.118 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.008818056, 2024-09-21T21:43:08.118 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:08.118 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:08.118 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:08.118 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:08.118 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5626", 2024-09-21T21:43:08.119 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:43:08.119 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 12339 2024-09-21T21:43:08.119 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.119 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:08.119 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.119 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:08.119 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411749+0000", 2024-09-21T21:43:08.119 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.119 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.119 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.119 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:08.119 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411749+0000", 2024-09-21T21:43:08.119 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.119 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.120 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.120 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:08.120 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411749+0000", 2024-09-21T21:43:08.120 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0900000000000002e-07 2024-09-21T21:43:08.120 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.120 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.120 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:08.120 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411752+0000", 2024-09-21T21:43:08.120 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5610000000000001e-06 2024-09-21T21:43:08.120 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.120 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.120 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:08.120 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411758+0000", 2024-09-21T21:43:08.120 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1890000000000004e-06 2024-09-21T21:43:08.120 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.120 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.121 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:08.121 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411763+0000", 2024-09-21T21:43:08.121 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7589999999999997e-06 2024-09-21T21:43:08.121 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.121 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.121 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:08.121 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411803+0000", 2024-09-21T21:43:08.121 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9922999999999997e-05 2024-09-21T21:43:08.121 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.121 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.121 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting_for_map not empty", 2024-09-21T21:43:08.121 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411809+0000", 2024-09-21T21:43:08.121 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3779999999999998e-06 2024-09-21T21:43:08.122 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.122 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.122 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:08.122 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415937+0000", 2024-09-21T21:43:08.122 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0041276100000000003 2024-09-21T21:43:08.122 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.122 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.122 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:08.122 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415944+0000", 2024-09-21T21:43:08.122 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8329999999999994e-06 2024-09-21T21:43:08.122 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.122 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.122 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:08.122 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420411+0000", 2024-09-21T21:43:08.123 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0044667300000001 2024-09-21T21:43:08.123 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.123 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.123 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:08.123 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420475+0000", 2024-09-21T21:43:08.123 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4356000000000005e-05 2024-09-21T21:43:08.123 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.123 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.123 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:08.123 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420567+0000", 2024-09-21T21:43:08.123 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.1308000000000001e-05 2024-09-21T21:43:08.123 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.123 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:08.123 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.123 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.124 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.124 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5626.0:12345 3.3 3:c643a86c:::rbd_data.1121faebdfe3.0000000000000002:head [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:43:08.124 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:14.411831+0000", 2024-09-21T21:43:08.124 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 53.608059701000002, 2024-09-21T21:43:08.124 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0089163649999999, 2024-09-21T21:43:08.124 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:08.124 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:08.124 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:08.124 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:08.124 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5626", 2024-09-21T21:43:08.124 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:43:08.124 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 12345 2024-09-21T21:43:08.124 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.124 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:08.125 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.125 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:08.125 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411831+0000", 2024-09-21T21:43:08.125 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.125 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.125 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.125 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:08.125 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411831+0000", 2024-09-21T21:43:08.125 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.125 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.125 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.125 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:08.125 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411831+0000", 2024-09-21T21:43:08.125 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.08e-07 2024-09-21T21:43:08.125 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.126 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.126 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:08.126 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411833+0000", 2024-09-21T21:43:08.126 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3089999999999998e-06 2024-09-21T21:43:08.126 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.126 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.126 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:08.126 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411840+0000", 2024-09-21T21:43:08.126 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.472e-06 2024-09-21T21:43:08.126 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.126 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.126 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:08.126 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411845+0000", 2024-09-21T21:43:08.126 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7910000000000004e-06 2024-09-21T21:43:08.127 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.127 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.127 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:08.127 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411890+0000", 2024-09-21T21:43:08.127 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5763000000000002e-05 2024-09-21T21:43:08.127 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.127 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.127 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting_for_map not empty", 2024-09-21T21:43:08.127 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411896+0000", 2024-09-21T21:43:08.127 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0190000000000003e-06 2024-09-21T21:43:08.127 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.127 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.127 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:08.127 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415967+0000", 2024-09-21T21:43:08.127 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0040705749999999999 2024-09-21T21:43:08.128 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.128 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.128 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:08.128 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415974+0000", 2024-09-21T21:43:08.128 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3200000000000002e-06 2024-09-21T21:43:08.128 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.128 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.128 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:08.128 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420594+0000", 2024-09-21T21:43:08.128 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.004620093 2024-09-21T21:43:08.128 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.128 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.128 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:08.128 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420658+0000", 2024-09-21T21:43:08.128 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3678999999999998e-05 2024-09-21T21:43:08.129 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.129 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.129 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:08.129 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420747+0000", 2024-09-21T21:43:08.129 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8936000000000004e-05 2024-09-21T21:43:08.129 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.129 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:08.129 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.129 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.129 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:08.129 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:43:08.131 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T21:43:08.131 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:43:08.129+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:43:08.219 INFO:teuthology.orchestra.run.smithi081.stderr:reweighted osd.3 to 0.924862 (ecc3) 2024-09-21T21:43:08.231 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:43:08.229+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:43:08.244 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T21:43:08.279 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:43:08.279 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:43:08.280 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-21T21:43:08.280 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:43:08.288 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T21:43:08.331 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:43:08.330+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:43:08.431 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:43:08.431 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:43:08.431 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:43:08.431 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:43:08.431 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:43:08.432 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:43:08.431+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:43:08.440 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T21:43:08.533 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:43:08.532+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:43:08.585 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:43:08.585 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-21T21:43:08.585 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-21T21:43:08.585 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:43:08.585 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.585 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6974 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5246976~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:08.586 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.922932+0000", 2024-09-21T21:43:08.586 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 98.660093345000007, 2024-09-21T21:43:08.586 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041773352999999999, 2024-09-21T21:43:08.586 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:08.586 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:08.586 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:08.586 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:08.586 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:08.586 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:08.586 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6974 2024-09-21T21:43:08.586 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.586 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:08.586 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.586 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:08.587 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922932+0000", 2024-09-21T21:43:08.587 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.587 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.587 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.587 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:08.587 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922932+0000", 2024-09-21T21:43:08.587 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.587 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.587 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.587 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:08.587 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922933+0000", 2024-09-21T21:43:08.587 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.46e-07 2024-09-21T21:43:08.587 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.587 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.587 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:08.588 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922940+0000", 2024-09-21T21:43:08.588 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6140000000000003e-06 2024-09-21T21:43:08.588 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.588 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.588 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:08.588 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922949+0000", 2024-09-21T21:43:08.588 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6899999999999998e-06 2024-09-21T21:43:08.588 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.588 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.588 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:08.589 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922955+0000", 2024-09-21T21:43:08.589 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5140000000000001e-06 2024-09-21T21:43:08.589 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.589 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.589 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:08.589 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964547+0000", 2024-09-21T21:43:08.589 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041592854999999998 2024-09-21T21:43:08.589 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.589 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.589 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:08.589 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964617+0000", 2024-09-21T21:43:08.589 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9058000000000002e-05 2024-09-21T21:43:08.589 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.589 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.590 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:08.590 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964706+0000", 2024-09-21T21:43:08.590 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9276000000000004e-05 2024-09-21T21:43:08.590 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.590 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:08.590 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.590 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.590 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.590 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6976 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5261312~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:08.590 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.922961+0000", 2024-09-21T21:43:08.590 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 98.660064300000002, 2024-09-21T21:43:08.590 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041863648000000003, 2024-09-21T21:43:08.590 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:08.590 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:08.591 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:08.591 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:08.591 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:08.591 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:08.591 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6976 2024-09-21T21:43:08.591 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.591 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:08.591 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.591 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:08.591 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922961+0000", 2024-09-21T21:43:08.591 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.591 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.591 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.591 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:08.592 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922961+0000", 2024-09-21T21:43:08.592 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.592 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.592 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.592 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:08.592 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922962+0000", 2024-09-21T21:43:08.592 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.89e-07 2024-09-21T21:43:08.592 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.592 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.592 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:08.592 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922964+0000", 2024-09-21T21:43:08.592 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1270000000000001e-06 2024-09-21T21:43:08.592 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.592 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.592 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:08.593 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922970+0000", 2024-09-21T21:43:08.593 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8030000000000002e-06 2024-09-21T21:43:08.593 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.593 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.593 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:08.593 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922979+0000", 2024-09-21T21:43:08.593 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5559999999999996e-06 2024-09-21T21:43:08.593 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.593 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.593 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:08.593 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964734+0000", 2024-09-21T21:43:08.593 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041754415000000003 2024-09-21T21:43:08.593 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.593 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.593 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:08.594 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964775+0000", 2024-09-21T21:43:08.594 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0806999999999997e-05 2024-09-21T21:43:08.594 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.594 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.594 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:08.594 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964825+0000", 2024-09-21T21:43:08.594 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0550999999999998e-05 2024-09-21T21:43:08.594 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.594 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:08.594 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.594 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.594 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.594 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6978 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5275648~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:08.594 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.922987+0000", 2024-09-21T21:43:08.595 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 98.660038514999997, 2024-09-21T21:43:08.595 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042043876000000001, 2024-09-21T21:43:08.595 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:08.595 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:08.595 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:08.595 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:08.595 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:08.595 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:08.595 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6978 2024-09-21T21:43:08.595 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.595 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:08.595 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.595 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:08.595 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922987+0000", 2024-09-21T21:43:08.596 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.596 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.596 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.596 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:08.596 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922987+0000", 2024-09-21T21:43:08.596 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.596 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.596 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.596 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:08.596 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922988+0000", 2024-09-21T21:43:08.596 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1199999999999998e-07 2024-09-21T21:43:08.596 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.596 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.596 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:08.596 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922990+0000", 2024-09-21T21:43:08.597 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.142e-06 2024-09-21T21:43:08.597 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.597 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.597 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:08.597 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922997+0000", 2024-09-21T21:43:08.597 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.199e-06 2024-09-21T21:43:08.597 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.597 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.597 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:08.597 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923004+0000", 2024-09-21T21:43:08.597 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3599999999999998e-06 2024-09-21T21:43:08.597 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.597 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.597 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:08.597 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964856+0000", 2024-09-21T21:43:08.598 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041851521000000003 2024-09-21T21:43:08.598 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.598 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.598 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:08.598 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964944+0000", 2024-09-21T21:43:08.598 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7859e-05 2024-09-21T21:43:08.598 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.598 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.598 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:08.598 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965031+0000", 2024-09-21T21:43:08.598 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7383e-05 2024-09-21T21:43:08.599 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.599 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:08.599 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.599 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.599 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.599 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6980 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5289984~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:08.599 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923011+0000", 2024-09-21T21:43:08.599 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 98.660015177999995, 2024-09-21T21:43:08.599 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042139337999999998, 2024-09-21T21:43:08.599 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:08.599 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:08.599 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:08.599 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:08.600 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:08.600 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:08.600 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6980 2024-09-21T21:43:08.600 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.600 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:08.600 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.600 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:08.600 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923011+0000", 2024-09-21T21:43:08.600 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.600 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.600 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.600 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:08.600 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923011+0000", 2024-09-21T21:43:08.600 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.600 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.601 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.601 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:08.601 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923011+0000", 2024-09-21T21:43:08.601 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6699999999999999e-07 2024-09-21T21:43:08.601 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.601 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.601 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:08.601 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923013+0000", 2024-09-21T21:43:08.601 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0279999999999999e-06 2024-09-21T21:43:08.601 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.601 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.601 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:08.601 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923018+0000", 2024-09-21T21:43:08.601 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4790000000000004e-06 2024-09-21T21:43:08.601 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.602 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.602 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:08.602 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923029+0000", 2024-09-21T21:43:08.602 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0601999999999999e-05 2024-09-21T21:43:08.602 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.602 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.602 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:08.602 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965059+0000", 2024-09-21T21:43:08.602 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042029947999999998 2024-09-21T21:43:08.602 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.602 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.602 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:08.602 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965100+0000", 2024-09-21T21:43:08.603 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1170000000000001e-05 2024-09-21T21:43:08.603 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.603 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.603 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:08.603 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965150+0000", 2024-09-21T21:43:08.603 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9743999999999997e-05 2024-09-21T21:43:08.603 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.603 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:08.603 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.603 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.603 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.603 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6982 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5304320~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:08.603 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923036+0000", 2024-09-21T21:43:08.604 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 98.659989357000001, 2024-09-21T21:43:08.604 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042225666000000002, 2024-09-21T21:43:08.604 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:08.604 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:08.604 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:08.604 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:08.604 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:08.604 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:08.604 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6982 2024-09-21T21:43:08.604 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.604 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:08.604 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.604 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:08.604 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923036+0000", 2024-09-21T21:43:08.605 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.605 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.605 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.605 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:08.605 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923036+0000", 2024-09-21T21:43:08.605 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.605 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.605 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.605 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:08.605 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923037+0000", 2024-09-21T21:43:08.605 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2800000000000002e-07 2024-09-21T21:43:08.605 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.605 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.605 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:08.606 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923039+0000", 2024-09-21T21:43:08.606 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1500000000000002e-06 2024-09-21T21:43:08.606 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.606 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.606 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:08.606 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923049+0000", 2024-09-21T21:43:08.606 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0236e-05 2024-09-21T21:43:08.606 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.606 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.606 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:08.606 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923056+0000", 2024-09-21T21:43:08.606 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2540000000000003e-06 2024-09-21T21:43:08.606 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.606 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.607 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:08.607 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965173+0000", 2024-09-21T21:43:08.607 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042116686 2024-09-21T21:43:08.607 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.607 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.607 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:08.607 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965215+0000", 2024-09-21T21:43:08.607 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1696000000000002e-05 2024-09-21T21:43:08.607 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.607 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.607 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:08.607 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965262+0000", 2024-09-21T21:43:08.607 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7216000000000003e-05 2024-09-21T21:43:08.607 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.608 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:08.608 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.608 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.608 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.608 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6984 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5318656~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:08.608 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923063+0000", 2024-09-21T21:43:08.608 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 98.659962703000005, 2024-09-21T21:43:08.608 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042318651999999998, 2024-09-21T21:43:08.608 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:08.608 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:08.608 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:08.608 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:08.608 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:08.608 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:08.608 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6984 2024-09-21T21:43:08.609 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.609 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:08.609 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.609 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:08.609 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923063+0000", 2024-09-21T21:43:08.609 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.609 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.609 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.609 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:08.609 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923063+0000", 2024-09-21T21:43:08.609 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.609 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.610 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.610 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:08.610 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923064+0000", 2024-09-21T21:43:08.610 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2e-07 2024-09-21T21:43:08.610 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.610 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.610 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:08.610 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923066+0000", 2024-09-21T21:43:08.610 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.057e-06 2024-09-21T21:43:08.610 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.610 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.610 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:08.610 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923071+0000", 2024-09-21T21:43:08.610 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8810000000000001e-06 2024-09-21T21:43:08.610 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.611 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.611 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:08.611 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923078+0000", 2024-09-21T21:43:08.611 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5479999999999998e-06 2024-09-21T21:43:08.611 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.611 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.611 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:08.611 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965288+0000", 2024-09-21T21:43:08.611 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042209576999999998 2024-09-21T21:43:08.611 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.611 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.611 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:08.611 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965332+0000", 2024-09-21T21:43:08.611 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4366999999999997e-05 2024-09-21T21:43:08.611 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.612 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.612 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:08.612 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965382+0000", 2024-09-21T21:43:08.612 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9802000000000002e-05 2024-09-21T21:43:08.612 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.612 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:08.612 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.612 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.612 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.612 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6987 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5332992~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:08.612 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923085+0000", 2024-09-21T21:43:08.612 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 98.659941177999997, 2024-09-21T21:43:08.612 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042412103999999999, 2024-09-21T21:43:08.612 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:08.613 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:08.613 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:08.613 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:08.613 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:08.613 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:08.613 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6987 2024-09-21T21:43:08.613 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.613 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:08.613 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.613 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:08.613 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923085+0000", 2024-09-21T21:43:08.613 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.613 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.613 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.613 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:08.614 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923085+0000", 2024-09-21T21:43:08.614 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.614 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.614 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.614 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:08.614 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923085+0000", 2024-09-21T21:43:08.614 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.89e-07 2024-09-21T21:43:08.614 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.614 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.614 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:08.614 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923087+0000", 2024-09-21T21:43:08.614 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0810000000000001e-06 2024-09-21T21:43:08.614 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.614 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.615 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:08.615 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923098+0000", 2024-09-21T21:43:08.615 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1299e-05 2024-09-21T21:43:08.615 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.615 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.615 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:08.615 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923104+0000", 2024-09-21T21:43:08.615 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9100000000000002e-06 2024-09-21T21:43:08.615 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.615 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.615 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:08.615 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965404+0000", 2024-09-21T21:43:08.615 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042299622000000002 2024-09-21T21:43:08.615 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.616 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.616 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:08.616 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965445+0000", 2024-09-21T21:43:08.616 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0701000000000002e-05 2024-09-21T21:43:08.616 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.616 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.616 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:08.616 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965497+0000", 2024-09-21T21:43:08.616 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2101999999999997e-05 2024-09-21T21:43:08.616 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.616 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:08.616 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.616 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.616 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.616 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6989 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5347328~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:08.617 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923111+0000", 2024-09-21T21:43:08.617 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 98.659914838000006, 2024-09-21T21:43:08.617 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042505188999999999, 2024-09-21T21:43:08.617 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:08.617 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:08.617 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:08.617 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:08.617 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:08.617 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:08.617 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6989 2024-09-21T21:43:08.617 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.617 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:08.617 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.617 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:08.617 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923111+0000", 2024-09-21T21:43:08.618 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.618 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.618 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.618 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:08.618 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923111+0000", 2024-09-21T21:43:08.618 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.618 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.618 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.618 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:08.618 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923111+0000", 2024-09-21T21:43:08.618 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4499999999999997e-07 2024-09-21T21:43:08.618 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.618 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.618 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:08.618 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923114+0000", 2024-09-21T21:43:08.619 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1569999999999998e-06 2024-09-21T21:43:08.619 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.619 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.619 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:08.619 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923120+0000", 2024-09-21T21:43:08.619 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1759999999999998e-06 2024-09-21T21:43:08.619 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.619 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.619 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:08.619 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923125+0000", 2024-09-21T21:43:08.619 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2560000000000002e-06 2024-09-21T21:43:08.619 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.619 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.619 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:08.619 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965523+0000", 2024-09-21T21:43:08.620 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042398094999999997 2024-09-21T21:43:08.620 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.620 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.620 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:08.620 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965566+0000", 2024-09-21T21:43:08.620 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3393999999999998e-05 2024-09-21T21:43:08.620 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.620 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.620 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:08.620 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965616+0000", 2024-09-21T21:43:08.620 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9666000000000002e-05 2024-09-21T21:43:08.620 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.620 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:08.620 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.620 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.621 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.621 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6991 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5361664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:08.621 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923132+0000", 2024-09-21T21:43:08.621 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 98.659893549000003, 2024-09-21T21:43:08.621 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042600779999999998, 2024-09-21T21:43:08.621 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:08.621 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:08.621 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:08.621 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:08.621 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:08.621 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:08.621 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6991 2024-09-21T21:43:08.621 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.621 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:08.621 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.622 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:08.622 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923132+0000", 2024-09-21T21:43:08.622 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.622 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.622 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.622 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:08.622 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923132+0000", 2024-09-21T21:43:08.622 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.622 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.622 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.622 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:08.622 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923132+0000", 2024-09-21T21:43:08.622 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4200000000000002e-07 2024-09-21T21:43:08.622 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.622 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.623 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:08.623 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923134+0000", 2024-09-21T21:43:08.623 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2270000000000001e-06 2024-09-21T21:43:08.623 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.623 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.623 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:08.623 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923137+0000", 2024-09-21T21:43:08.623 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1930000000000002e-06 2024-09-21T21:43:08.623 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.623 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.623 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:08.623 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923139+0000", 2024-09-21T21:43:08.623 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4710000000000001e-06 2024-09-21T21:43:08.623 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.623 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.624 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:08.624 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965639+0000", 2024-09-21T21:43:08.624 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.04249965 2024-09-21T21:43:08.624 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.624 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.624 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:08.624 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965679+0000", 2024-09-21T21:43:08.624 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9852999999999999e-05 2024-09-21T21:43:08.624 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.624 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.624 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:08.624 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965733+0000", 2024-09-21T21:43:08.624 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4144000000000002e-05 2024-09-21T21:43:08.624 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.624 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:08.624 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.625 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.625 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.625 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6993 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5384192~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:08.625 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923143+0000", 2024-09-21T21:43:08.625 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 98.659882754999998, 2024-09-21T21:43:08.625 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042708182999999997, 2024-09-21T21:43:08.625 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:08.625 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:08.625 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:08.625 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:08.625 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:08.625 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:08.625 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6993 2024-09-21T21:43:08.625 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.625 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:08.626 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.626 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:08.626 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923143+0000", 2024-09-21T21:43:08.626 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.626 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.626 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.626 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:08.626 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923143+0000", 2024-09-21T21:43:08.626 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.626 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.626 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.626 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:08.626 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923143+0000", 2024-09-21T21:43:08.626 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0800000000000001e-07 2024-09-21T21:43:08.626 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.627 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.627 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:08.627 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923144+0000", 2024-09-21T21:43:08.627 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.009e-06 2024-09-21T21:43:08.627 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.627 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.627 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:08.627 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923148+0000", 2024-09-21T21:43:08.627 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.737e-06 2024-09-21T21:43:08.627 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.627 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.627 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:08.627 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923150+0000", 2024-09-21T21:43:08.627 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4669999999999998e-06 2024-09-21T21:43:08.627 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.628 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.628 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:08.628 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965760+0000", 2024-09-21T21:43:08.628 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042609399999999999 2024-09-21T21:43:08.628 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.628 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.628 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:08.628 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965801+0000", 2024-09-21T21:43:08.628 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1399999999999997e-05 2024-09-21T21:43:08.628 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.628 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.629 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:08.629 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965851+0000", 2024-09-21T21:43:08.629 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9962e-05 2024-09-21T21:43:08.629 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.629 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:08.629 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.629 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.629 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.629 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6995 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5398528~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:08.629 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923153+0000", 2024-09-21T21:43:08.629 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 98.659872390999993, 2024-09-21T21:43:08.629 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042828034000000001, 2024-09-21T21:43:08.629 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:08.629 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:08.630 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:08.630 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:08.630 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:08.630 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:08.630 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6995 2024-09-21T21:43:08.630 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.630 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:08.630 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.630 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:08.630 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923153+0000", 2024-09-21T21:43:08.630 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.630 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.630 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.630 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:08.631 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923153+0000", 2024-09-21T21:43:08.631 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.631 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.631 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.631 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:08.631 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923154+0000", 2024-09-21T21:43:08.631 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2600000000000001e-07 2024-09-21T21:43:08.631 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.631 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.631 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:08.631 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923158+0000", 2024-09-21T21:43:08.631 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0559999999999998e-06 2024-09-21T21:43:08.631 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.631 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.631 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:08.632 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923161+0000", 2024-09-21T21:43:08.632 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6519999999999999e-06 2024-09-21T21:43:08.632 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.632 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.632 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:08.632 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923170+0000", 2024-09-21T21:43:08.632 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5720000000000003e-06 2024-09-21T21:43:08.632 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.632 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.632 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:08.632 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965887+0000", 2024-09-21T21:43:08.632 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042716863000000001 2024-09-21T21:43:08.632 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.632 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.632 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:08.633 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965927+0000", 2024-09-21T21:43:08.633 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0346999999999998e-05 2024-09-21T21:43:08.633 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.633 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.633 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:08.633 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965981+0000", 2024-09-21T21:43:08.633 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4317999999999997e-05 2024-09-21T21:43:08.633 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.633 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:08.633 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.633 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.633 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.633 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6997 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5412864~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:08.633 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923174+0000", 2024-09-21T21:43:08.633 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 98.659852236000006, 2024-09-21T21:43:08.634 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042928296999999997, 2024-09-21T21:43:08.634 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:08.634 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:08.634 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:08.634 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:08.634 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:08.634 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:08.634 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6997 2024-09-21T21:43:08.634 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.634 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:08.634 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.634 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:08.634 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923174+0000", 2024-09-21T21:43:08.634 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.634 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.635 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.635 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:08.635 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923174+0000", 2024-09-21T21:43:08.635 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.635 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.635 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.635 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:08.635 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923174+0000", 2024-09-21T21:43:08.635 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.67e-07 2024-09-21T21:43:08.635 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.635 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.635 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:08.635 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923178+0000", 2024-09-21T21:43:08.635 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1330000000000001e-06 2024-09-21T21:43:08.635 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.636 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.636 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:08.636 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923182+0000", 2024-09-21T21:43:08.636 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.794e-06 2024-09-21T21:43:08.636 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.636 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.636 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:08.636 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923184+0000", 2024-09-21T21:43:08.636 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.435e-06 2024-09-21T21:43:08.636 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.636 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.636 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:08.636 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966009+0000", 2024-09-21T21:43:08.636 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042824327000000002 2024-09-21T21:43:08.636 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.636 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.637 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:08.637 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966052+0000", 2024-09-21T21:43:08.637 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3034000000000002e-05 2024-09-21T21:43:08.637 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.637 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.637 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:08.637 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966102+0000", 2024-09-21T21:43:08.637 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0306999999999998e-05 2024-09-21T21:43:08.637 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.637 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:08.637 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.637 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.637 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.637 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6999 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5427200~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:08.637 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923187+0000", 2024-09-21T21:43:08.638 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 98.659838508000007, 2024-09-21T21:43:08.638 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043104267000000002, 2024-09-21T21:43:08.638 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:08.638 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:08.638 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:08.638 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:08.638 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:08.638 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:08.638 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6999 2024-09-21T21:43:08.638 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.638 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:08.638 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.638 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:08.638 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923187+0000", 2024-09-21T21:43:08.639 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.639 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.639 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.639 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:08.639 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923187+0000", 2024-09-21T21:43:08.639 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.639 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.639 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.639 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:08.639 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923188+0000", 2024-09-21T21:43:08.639 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3999999999999998e-07 2024-09-21T21:43:08.639 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.639 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.639 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:08.639 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923189+0000", 2024-09-21T21:43:08.640 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4169999999999999e-06 2024-09-21T21:43:08.640 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.640 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.640 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:08.640 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923194+0000", 2024-09-21T21:43:08.640 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1819999999999997e-06 2024-09-21T21:43:08.640 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.640 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.640 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:08.640 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923198+0000", 2024-09-21T21:43:08.640 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.143e-06 2024-09-21T21:43:08.640 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.640 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.640 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:08.640 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966135+0000", 2024-09-21T21:43:08.641 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042936297999999998 2024-09-21T21:43:08.641 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.641 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.641 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:08.641 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966204+0000", 2024-09-21T21:43:08.641 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8989e-05 2024-09-21T21:43:08.641 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.641 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.641 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:08.641 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966292+0000", 2024-09-21T21:43:08.641 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7997999999999993e-05 2024-09-21T21:43:08.641 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.641 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:08.641 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.641 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.642 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.642 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7001 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5441536~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:08.642 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923211+0000", 2024-09-21T21:43:08.642 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 98.659814717000003, 2024-09-21T21:43:08.642 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043196518000000003, 2024-09-21T21:43:08.642 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:08.642 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:08.642 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:08.642 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:08.642 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:08.642 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:08.642 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7001 2024-09-21T21:43:08.642 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.642 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:08.642 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.643 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:08.643 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923211+0000", 2024-09-21T21:43:08.643 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.643 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.643 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.643 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:08.643 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923211+0000", 2024-09-21T21:43:08.643 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.643 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.643 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.643 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:08.643 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923212+0000", 2024-09-21T21:43:08.643 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3900000000000005e-07 2024-09-21T21:43:08.643 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.643 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.644 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:08.644 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923214+0000", 2024-09-21T21:43:08.644 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9970000000000001e-06 2024-09-21T21:43:08.644 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.644 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.644 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:08.644 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923219+0000", 2024-09-21T21:43:08.644 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4310000000000002e-06 2024-09-21T21:43:08.644 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.644 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.644 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:08.644 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923224+0000", 2024-09-21T21:43:08.644 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6410000000000001e-06 2024-09-21T21:43:08.644 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.644 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.645 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:08.645 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966318+0000", 2024-09-21T21:43:08.645 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043093842 2024-09-21T21:43:08.645 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.645 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.645 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:08.645 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966359+0000", 2024-09-21T21:43:08.645 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1260999999999997e-05 2024-09-21T21:43:08.645 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.645 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.645 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:08.645 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966408+0000", 2024-09-21T21:43:08.645 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8807000000000002e-05 2024-09-21T21:43:08.645 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.645 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:08.646 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.646 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.646 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.646 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7002 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5455872~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:08.646 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923229+0000", 2024-09-21T21:43:08.646 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 98.659796306999993, 2024-09-21T21:43:08.646 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043364568999999999, 2024-09-21T21:43:08.646 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:08.646 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:08.646 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:08.646 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:08.646 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:08.646 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:08.646 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7002 2024-09-21T21:43:08.647 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.647 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:08.647 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.647 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:08.647 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923229+0000", 2024-09-21T21:43:08.647 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.647 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.647 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.647 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:08.647 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923229+0000", 2024-09-21T21:43:08.647 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.647 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.647 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.647 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:08.647 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923230+0000", 2024-09-21T21:43:08.648 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4700000000000002e-07 2024-09-21T21:43:08.648 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.648 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.648 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:08.648 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923232+0000", 2024-09-21T21:43:08.648 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0719999999999998e-06 2024-09-21T21:43:08.648 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.648 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.648 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:08.648 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923237+0000", 2024-09-21T21:43:08.648 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2290000000000002e-06 2024-09-21T21:43:08.648 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.648 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.648 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:08.648 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923245+0000", 2024-09-21T21:43:08.649 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2959999999999999e-06 2024-09-21T21:43:08.649 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.649 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.649 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:08.649 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966438+0000", 2024-09-21T21:43:08.649 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043193027000000002 2024-09-21T21:43:08.649 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.649 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.649 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:08.649 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966506+0000", 2024-09-21T21:43:08.649 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7707999999999996e-05 2024-09-21T21:43:08.649 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.649 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.649 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:08.649 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966594+0000", 2024-09-21T21:43:08.649 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.789e-05 2024-09-21T21:43:08.650 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.650 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:08.650 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.650 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.650 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.650 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7005 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5470208~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:08.650 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923254+0000", 2024-09-21T21:43:08.650 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 98.659772075999996, 2024-09-21T21:43:08.650 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043457176, 2024-09-21T21:43:08.650 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:08.650 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:08.650 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:08.650 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:08.650 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:08.650 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:08.651 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7005 2024-09-21T21:43:08.651 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.651 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:08.651 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.651 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:08.651 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923254+0000", 2024-09-21T21:43:08.651 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.651 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.651 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.651 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:08.651 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923254+0000", 2024-09-21T21:43:08.651 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.651 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.651 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.651 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:08.652 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923254+0000", 2024-09-21T21:43:08.652 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4700000000000002e-07 2024-09-21T21:43:08.652 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.652 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.652 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:08.652 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923256+0000", 2024-09-21T21:43:08.652 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.007e-06 2024-09-21T21:43:08.652 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.652 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.652 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:08.652 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923266+0000", 2024-09-21T21:43:08.652 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.4700000000000008e-06 2024-09-21T21:43:08.652 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.652 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.652 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:08.653 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923271+0000", 2024-09-21T21:43:08.653 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2809999999999999e-06 2024-09-21T21:43:08.653 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.653 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.653 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:08.653 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966619+0000", 2024-09-21T21:43:08.653 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043347951000000003 2024-09-21T21:43:08.653 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.653 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.653 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:08.653 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966661+0000", 2024-09-21T21:43:08.653 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2614999999999998e-05 2024-09-21T21:43:08.653 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.653 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.653 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:08.654 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966711+0000", 2024-09-21T21:43:08.654 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9505000000000001e-05 2024-09-21T21:43:08.654 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.654 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:08.654 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.654 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.654 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.654 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7008 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5484544~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:08.654 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923278+0000", 2024-09-21T21:43:08.654 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 98.659747429000006, 2024-09-21T21:43:08.654 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043631071, 2024-09-21T21:43:08.654 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:08.654 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:08.654 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:08.655 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:08.655 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:08.655 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:08.655 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7008 2024-09-21T21:43:08.655 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.655 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:08.655 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.655 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:08.655 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923278+0000", 2024-09-21T21:43:08.655 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.655 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.655 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.655 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:08.655 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923278+0000", 2024-09-21T21:43:08.655 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.656 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.656 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.656 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:08.656 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923279+0000", 2024-09-21T21:43:08.656 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.77e-07 2024-09-21T21:43:08.656 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.656 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.656 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:08.656 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923281+0000", 2024-09-21T21:43:08.656 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0219999999999999e-06 2024-09-21T21:43:08.656 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.656 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.656 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:08.656 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923287+0000", 2024-09-21T21:43:08.656 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7799999999999997e-06 2024-09-21T21:43:08.657 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.657 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.657 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:08.657 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923291+0000", 2024-09-21T21:43:08.657 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1699999999999999e-06 2024-09-21T21:43:08.657 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.657 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.657 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:08.657 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966742+0000", 2024-09-21T21:43:08.657 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043451344000000003 2024-09-21T21:43:08.657 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.657 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.657 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:08.657 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966809+0000", 2024-09-21T21:43:08.657 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7070999999999995e-05 2024-09-21T21:43:08.658 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.658 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.658 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:08.658 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966909+0000", 2024-09-21T21:43:08.658 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000100307 2024-09-21T21:43:08.658 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.658 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:08.658 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.658 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.658 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.658 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7010 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5498880~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:08.658 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923301+0000", 2024-09-21T21:43:08.658 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 98.659724534000006, 2024-09-21T21:43:08.658 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043717830999999999, 2024-09-21T21:43:08.659 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:08.659 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:08.659 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:08.659 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:08.659 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:08.659 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:08.659 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7010 2024-09-21T21:43:08.659 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.659 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:08.659 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.659 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:08.659 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923301+0000", 2024-09-21T21:43:08.659 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.659 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.659 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.660 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:08.660 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923301+0000", 2024-09-21T21:43:08.660 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.660 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.660 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.660 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:08.660 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923302+0000", 2024-09-21T21:43:08.660 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.41e-07 2024-09-21T21:43:08.660 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.660 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.660 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:08.660 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923304+0000", 2024-09-21T21:43:08.660 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0990000000000001e-06 2024-09-21T21:43:08.660 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.660 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.661 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:08.661 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923310+0000", 2024-09-21T21:43:08.661 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3860000000000004e-06 2024-09-21T21:43:08.661 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.661 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.661 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:08.661 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923315+0000", 2024-09-21T21:43:08.661 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8459999999999999e-06 2024-09-21T21:43:08.661 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.661 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.661 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:08.661 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966938+0000", 2024-09-21T21:43:08.661 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043622693999999997 2024-09-21T21:43:08.661 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.662 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.662 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:08.662 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966975+0000", 2024-09-21T21:43:08.662 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7608999999999999e-05 2024-09-21T21:43:08.662 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.662 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.662 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:08.662 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.967019+0000", 2024-09-21T21:43:08.662 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3856000000000002e-05 2024-09-21T21:43:08.662 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.662 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:08.662 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.662 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.662 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.663 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7354 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 2107392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:08.663 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.936932+0000", 2024-09-21T21:43:08.663 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 98.646093679000003, 2024-09-21T21:43:08.663 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041607708, 2024-09-21T21:43:08.663 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:08.663 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:08.663 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:08.663 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:08.663 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:08.663 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:08.663 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7354 2024-09-21T21:43:08.663 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.663 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:08.663 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.664 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:08.664 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.936932+0000", 2024-09-21T21:43:08.664 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.664 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.664 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.664 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:08.664 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.936932+0000", 2024-09-21T21:43:08.664 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.664 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.664 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.664 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:08.664 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.937276+0000", 2024-09-21T21:43:08.664 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00034344200000000001 2024-09-21T21:43:08.664 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.664 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.665 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:08.665 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.937279+0000", 2024-09-21T21:43:08.665 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.844e-06 2024-09-21T21:43:08.665 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.665 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.665 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:08.665 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.937288+0000", 2024-09-21T21:43:08.665 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2409999999999995e-06 2024-09-21T21:43:08.665 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.665 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.665 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:08.665 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.937293+0000", 2024-09-21T21:43:08.665 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6559999999999997e-06 2024-09-21T21:43:08.665 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.665 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.666 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:08.666 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.978448+0000", 2024-09-21T21:43:08.666 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041154621000000002 2024-09-21T21:43:08.666 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.666 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.666 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:08.666 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.978487+0000", 2024-09-21T21:43:08.666 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9254000000000001e-05 2024-09-21T21:43:08.666 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.666 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.666 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:08.666 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.978540+0000", 2024-09-21T21:43:08.666 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2649999999999999e-05 2024-09-21T21:43:08.666 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.666 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:08.667 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.667 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.667 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.667 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5215.0:1054 3.9 3:946d2401:::rbd_data.1121524d2522.000000000000000b:head [set-alloc-hint object_size 16777216 write_size 16777216,write 6615040~59392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-21T21:43:08.667 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:38.148979+0000", 2024-09-21T21:43:08.667 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 90.434046711999997, 2024-09-21T21:43:08.667 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047870388999999999, 2024-09-21T21:43:08.667 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:08.667 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:08.667 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:08.667 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:08.667 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5215", 2024-09-21T21:43:08.667 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1580282168", 2024-09-21T21:43:08.667 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1054 2024-09-21T21:43:08.668 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.668 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:08.668 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.668 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:08.668 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.148979+0000", 2024-09-21T21:43:08.668 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.668 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.668 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.668 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:08.668 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.148979+0000", 2024-09-21T21:43:08.668 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.668 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.668 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.668 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:08.668 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.148980+0000", 2024-09-21T21:43:08.669 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5799999999999999e-07 2024-09-21T21:43:08.669 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.669 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.669 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:08.669 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.149039+0000", 2024-09-21T21:43:08.669 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9151000000000003e-05 2024-09-21T21:43:08.669 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.669 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.669 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:08.669 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.149047+0000", 2024-09-21T21:43:08.669 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1000000000000004e-06 2024-09-21T21:43:08.669 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.669 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.669 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:08.669 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.149052+0000", 2024-09-21T21:43:08.670 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5959999999999997e-06 2024-09-21T21:43:08.670 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.670 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.670 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:08.670 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.195288+0000", 2024-09-21T21:43:08.670 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.046236248000000001 2024-09-21T21:43:08.670 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.670 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.670 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:08.670 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.195334+0000", 2024-09-21T21:43:08.670 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6575e-05 2024-09-21T21:43:08.670 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.670 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.671 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 2", 2024-09-21T21:43:08.671 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.195406+0000", 2024-09-21T21:43:08.671 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1715e-05 2024-09-21T21:43:08.671 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.671 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.671 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:08.671 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.196018+0000", 2024-09-21T21:43:08.671 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00061218999999999998 2024-09-21T21:43:08.671 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.671 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.671 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:08.671 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.196794+0000", 2024-09-21T21:43:08.672 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000775814 2024-09-21T21:43:08.672 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.672 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.672 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:08.672 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.196827+0000", 2024-09-21T21:43:08.672 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2419000000000003e-05 2024-09-21T21:43:08.672 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.672 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.672 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:08.672 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.196849+0000", 2024-09-21T21:43:08.672 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2923000000000001e-05 2024-09-21T21:43:08.672 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.672 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:08.672 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.672 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.673 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:08.673 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:43:08.674 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T21:43:08.674 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:43:08.672+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:43:08.775 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:43:08.773+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:43:08.818 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:43:08.818 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:43:08.818 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.819 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4385.0:30375 3.14 3.fc4e13b4 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e146)", 2024-09-21T21:43:08.819 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:43:08.225561+0000", 2024-09-21T21:43:08.819 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.59206474099999995, 2024-09-21T21:43:08.819 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.59208501999999996, 2024-09-21T21:43:08.819 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:08.819 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:08.819 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "waiting for peered", 2024-09-21T21:43:08.819 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:08.819 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4385", 2024-09-21T21:43:08.819 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:43:08.819 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 30375 2024-09-21T21:43:08.819 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.819 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:08.819 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.820 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:08.820 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:43:08.225561+0000", 2024-09-21T21:43:08.820 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.820 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.820 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.820 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:08.820 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:43:08.225561+0000", 2024-09-21T21:43:08.820 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.820 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.820 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.820 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:08.820 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:43:08.225562+0000", 2024-09-21T21:43:08.820 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4980000000000001e-06 2024-09-21T21:43:08.820 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.821 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.821 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:08.821 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:43:08.225570+0000", 2024-09-21T21:43:08.821 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9300000000000003e-06 2024-09-21T21:43:08.821 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.821 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.821 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:08.821 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:43:08.225581+0000", 2024-09-21T21:43:08.821 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1147e-05 2024-09-21T21:43:08.821 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.821 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.821 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:08.821 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:43:08.225590+0000", 2024-09-21T21:43:08.821 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5949999999999999e-06 2024-09-21T21:43:08.822 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.822 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.822 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:08.822 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:43:08.225671+0000", 2024-09-21T21:43:08.822 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1216000000000003e-05 2024-09-21T21:43:08.822 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.822 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.822 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:08.822 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:43:08.225680+0000", 2024-09-21T21:43:08.822 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.0510000000000002e-06 2024-09-21T21:43:08.822 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.823 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:08.823 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.823 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.823 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.823 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4385.0:30376 3.14 3.fc4e13b4 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e146)", 2024-09-21T21:43:08.823 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:43:08.225609+0000", 2024-09-21T21:43:08.823 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.59201705900000001, 2024-09-21T21:43:08.823 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.59207737699999996, 2024-09-21T21:43:08.823 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:08.823 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:08.823 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "waiting for peered", 2024-09-21T21:43:08.823 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:08.823 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4385", 2024-09-21T21:43:08.823 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:43:08.823 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 30376 2024-09-21T21:43:08.824 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.824 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:08.824 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.824 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:08.824 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:43:08.225609+0000", 2024-09-21T21:43:08.824 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.824 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.824 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.824 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:08.824 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:43:08.225609+0000", 2024-09-21T21:43:08.824 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:08.824 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.824 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.824 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:08.825 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:43:08.225609+0000", 2024-09-21T21:43:08.825 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6000000000000004e-07 2024-09-21T21:43:08.825 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.825 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.825 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:08.825 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:43:08.225612+0000", 2024-09-21T21:43:08.825 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3390000000000001e-06 2024-09-21T21:43:08.825 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.825 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.825 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:08.825 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:43:08.225619+0000", 2024-09-21T21:43:08.825 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9709999999999998e-06 2024-09-21T21:43:08.825 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.825 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.825 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:08.826 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:43:08.225625+0000", 2024-09-21T21:43:08.826 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5369999999999997e-06 2024-09-21T21:43:08.826 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.826 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.826 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:08.826 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:43:08.225702+0000", 2024-09-21T21:43:08.826 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6901000000000001e-05 2024-09-21T21:43:08.826 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:08.826 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:08.826 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:08.826 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:43:08.225708+0000", 2024-09-21T21:43:08.826 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1059999999999996e-06 2024-09-21T21:43:08.826 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.827 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:08.827 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.827 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:08.827 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-21T21:43:08.827 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 2 2024-09-21T21:43:08.827 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:43:08.828 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T21:43:08.912 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:43:08.911+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:43:08.971 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:43:08.971 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:43:08.971 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:43:08.971 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:43:08.971 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:43:08.976 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:43:08.974+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:43:08.980 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T21:43:09.076 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:43:09.075+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:43:09.126 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:43:09.126 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-21T21:43:09.126 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-21T21:43:09.126 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:43:09.126 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.126 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4385.0:15202 3.16 3:6888948b:::rbd_header.1121493f6459:head [watch reconnect cookie 140325440202496 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-21T21:43:09.126 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:41.736601+0000", 2024-09-21T21:43:09.126 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 87.386781945999999, 2024-09-21T21:43:09.127 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5040056769999999, 2024-09-21T21:43:09.127 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:09.127 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:09.127 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:09.127 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:09.127 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4385", 2024-09-21T21:43:09.127 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:43:09.127 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 15202 2024-09-21T21:43:09.127 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.127 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:09.127 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.127 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:09.127 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736601+0000", 2024-09-21T21:43:09.127 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:09.127 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.128 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.128 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:09.128 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736601+0000", 2024-09-21T21:43:09.128 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:09.128 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.128 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.128 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:09.128 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736602+0000", 2024-09-21T21:43:09.128 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0699999999999999e-06 2024-09-21T21:43:09.128 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.128 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.128 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:09.128 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736607+0000", 2024-09-21T21:43:09.129 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2520000000000001e-06 2024-09-21T21:43:09.129 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.129 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.129 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:09.129 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736617+0000", 2024-09-21T21:43:09.129 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0074e-05 2024-09-21T21:43:09.129 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.129 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.129 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:09.129 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736622+0000", 2024-09-21T21:43:09.129 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.101e-06 2024-09-21T21:43:09.129 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.129 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.129 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:09.130 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736664+0000", 2024-09-21T21:43:09.130 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1869999999999997e-05 2024-09-21T21:43:09.130 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.130 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.130 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op must wait for map", 2024-09-21T21:43:09.130 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736667+0000", 2024-09-21T21:43:09.130 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4750000000000002e-06 2024-09-21T21:43:09.130 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.130 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.130 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:09.130 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742886+0000", 2024-09-21T21:43:09.130 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0062187579999999996 2024-09-21T21:43:09.130 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.130 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.131 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:09.131 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742892+0000", 2024-09-21T21:43:09.131 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0440000000000001e-06 2024-09-21T21:43:09.131 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.131 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.131 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:09.131 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210307+0000", 2024-09-21T21:43:09.131 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.467414781 2024-09-21T21:43:09.131 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.131 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.131 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:09.131 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210322+0000", 2024-09-21T21:43:09.131 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5311e-05 2024-09-21T21:43:09.131 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.131 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.132 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:09.132 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.237624+0000", 2024-09-21T21:43:09.132 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0273018920000001 2024-09-21T21:43:09.132 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.132 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.132 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:09.132 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.237870+0000", 2024-09-21T21:43:09.132 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00024622899999999999 2024-09-21T21:43:09.132 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.132 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.132 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:09.132 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.240607+0000", 2024-09-21T21:43:09.132 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0027368200000000001 2024-09-21T21:43:09.132 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:09.132 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:09.133 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:09.133 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.133 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.133 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5415.0:379 3.16 3:6888948b:::rbd_header.1121493f6459:head [watch reconnect cookie 139702602617920 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-21T21:43:09.133 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:41.739544+0000", 2024-09-21T21:43:09.133 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 87.383839516999998, 2024-09-21T21:43:09.133 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.501180645, 2024-09-21T21:43:09.133 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:09.133 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:09.133 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:09.133 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:09.133 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5415", 2024-09-21T21:43:09.133 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1212816607", 2024-09-21T21:43:09.133 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 379 2024-09-21T21:43:09.134 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.134 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:09.134 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.134 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:09.134 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739544+0000", 2024-09-21T21:43:09.134 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:09.134 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.134 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.134 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:09.134 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739544+0000", 2024-09-21T21:43:09.134 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:09.134 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.134 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.134 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:09.134 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739545+0000", 2024-09-21T21:43:09.135 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.065e-06 2024-09-21T21:43:09.135 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.135 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.135 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:09.135 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739550+0000", 2024-09-21T21:43:09.135 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9599999999999999e-06 2024-09-21T21:43:09.135 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.135 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.135 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:09.135 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739565+0000", 2024-09-21T21:43:09.135 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5709000000000002e-05 2024-09-21T21:43:09.135 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.135 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.135 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:09.135 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739573+0000", 2024-09-21T21:43:09.136 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7849999999999992e-06 2024-09-21T21:43:09.136 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.136 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.136 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:09.136 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739634+0000", 2024-09-21T21:43:09.136 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0732000000000001e-05 2024-09-21T21:43:09.136 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.136 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.136 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op must wait for map", 2024-09-21T21:43:09.136 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739640+0000", 2024-09-21T21:43:09.136 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0000000000000002e-06 2024-09-21T21:43:09.136 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.136 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.136 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:09.136 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742958+0000", 2024-09-21T21:43:09.137 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0033182390000000002 2024-09-21T21:43:09.137 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.137 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.137 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:09.137 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742966+0000", 2024-09-21T21:43:09.137 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1610000000000003e-06 2024-09-21T21:43:09.137 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.137 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.137 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:09.137 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210405+0000", 2024-09-21T21:43:09.137 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.467438725 2024-09-21T21:43:09.137 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.137 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.137 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:09.137 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210415+0000", 2024-09-21T21:43:09.138 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0234000000000001e-05 2024-09-21T21:43:09.138 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.138 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.138 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:09.138 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.238743+0000", 2024-09-21T21:43:09.138 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0283273070000001 2024-09-21T21:43:09.138 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.138 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.138 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:09.138 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.238842+0000", 2024-09-21T21:43:09.138 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.9896999999999996e-05 2024-09-21T21:43:09.138 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.138 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.138 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:09.139 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.240724+0000", 2024-09-21T21:43:09.139 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001881831 2024-09-21T21:43:09.139 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:09.139 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:09.139 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:09.139 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.139 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.139 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4385.0:15203 3.16 3:6888948b:::rbd_header.1121493f6459:head [call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-21T21:43:09.139 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:41.739737+0000", 2024-09-21T21:43:09.139 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 87.383646506000005, 2024-09-21T21:43:09.139 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4989621799999999, 2024-09-21T21:43:09.139 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:09.139 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:09.139 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:09.139 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:09.140 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4385", 2024-09-21T21:43:09.140 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:43:09.140 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 15203 2024-09-21T21:43:09.140 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.140 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:09.140 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.140 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:09.140 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739737+0000", 2024-09-21T21:43:09.140 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:09.140 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.140 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.140 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:09.140 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739737+0000", 2024-09-21T21:43:09.140 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:09.140 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.141 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.141 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:09.141 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739738+0000", 2024-09-21T21:43:09.141 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.7499999999999998e-07 2024-09-21T21:43:09.141 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.141 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.141 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:09.141 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739743+0000", 2024-09-21T21:43:09.141 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5049999999999998e-06 2024-09-21T21:43:09.141 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.141 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.141 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:09.141 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739757+0000", 2024-09-21T21:43:09.141 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3947e-05 2024-09-21T21:43:09.141 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.142 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.142 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:09.142 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739765+0000", 2024-09-21T21:43:09.142 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9310000000000008e-06 2024-09-21T21:43:09.142 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.142 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.142 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:09.142 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739816+0000", 2024-09-21T21:43:09.142 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0559999999999997e-05 2024-09-21T21:43:09.142 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.142 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.142 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting_for_map not empty", 2024-09-21T21:43:09.143 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739821+0000", 2024-09-21T21:43:09.143 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3539999999999999e-06 2024-09-21T21:43:09.143 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.143 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.143 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:09.143 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742928+0000", 2024-09-21T21:43:09.143 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0031074399999999999 2024-09-21T21:43:09.143 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.143 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.143 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:09.143 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742937+0000", 2024-09-21T21:43:09.143 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1289999999999996e-06 2024-09-21T21:43:09.143 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.143 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.144 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:09.144 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210350+0000", 2024-09-21T21:43:09.144 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.46741311299999999 2024-09-21T21:43:09.144 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.144 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.144 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:09.144 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210358+0000", 2024-09-21T21:43:09.144 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1869999999999997e-06 2024-09-21T21:43:09.144 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.144 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.144 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:09.144 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.238398+0000", 2024-09-21T21:43:09.144 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0280397830000001 2024-09-21T21:43:09.144 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.144 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.145 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:09.145 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.238489+0000", 2024-09-21T21:43:09.145 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.0958000000000006e-05 2024-09-21T21:43:09.145 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.145 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.145 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:09.145 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.238699+0000", 2024-09-21T21:43:09.145 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00021029800000000001 2024-09-21T21:43:09.145 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:09.145 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:09.145 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:09.145 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.145 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.145 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4385.0:15204 3.16 3:6888948b:::rbd_header.1121493f6459:head [watch reconnect cookie 140325440202496 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-21T21:43:09.145 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:42.210726+0000", 2024-09-21T21:43:09.146 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 86.912657628000005, 2024-09-21T21:43:09.146 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.030304557, 2024-09-21T21:43:09.146 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:09.146 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:09.146 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:09.146 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:09.146 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4385", 2024-09-21T21:43:09.146 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:43:09.146 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 15204 2024-09-21T21:43:09.146 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.146 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:09.146 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.146 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:09.146 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210726+0000", 2024-09-21T21:43:09.146 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:09.147 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.147 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.147 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:09.147 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210726+0000", 2024-09-21T21:43:09.147 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:09.147 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.147 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.147 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:09.147 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210727+0000", 2024-09-21T21:43:09.147 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.204e-06 2024-09-21T21:43:09.147 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.147 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.147 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:09.147 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210732+0000", 2024-09-21T21:43:09.148 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5249999999999996e-06 2024-09-21T21:43:09.148 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.148 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.148 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:09.148 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210744+0000", 2024-09-21T21:43:09.148 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1405e-05 2024-09-21T21:43:09.148 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.148 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.148 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:09.148 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210754+0000", 2024-09-21T21:43:09.148 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0122999999999999e-05 2024-09-21T21:43:09.148 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.148 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.149 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:09.149 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210798+0000", 2024-09-21T21:43:09.149 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4005000000000003e-05 2024-09-21T21:43:09.149 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.149 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.149 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:09.149 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210803+0000", 2024-09-21T21:43:09.149 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5130000000000004e-06 2024-09-21T21:43:09.149 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.149 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.149 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:09.149 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.239260+0000", 2024-09-21T21:43:09.149 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0284563040000001 2024-09-21T21:43:09.149 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.149 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.150 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:09.150 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.239318+0000", 2024-09-21T21:43:09.150 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8486000000000002e-05 2024-09-21T21:43:09.150 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.150 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.150 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:09.150 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.241030+0000", 2024-09-21T21:43:09.150 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0017119920000000001 2024-09-21T21:43:09.150 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:09.150 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:09.150 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:09.150 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.150 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.150 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5415.0:380 3.16 3:6888948b:::rbd_header.1121493f6459:head [watch reconnect cookie 139702602617920 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-21T21:43:09.151 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:42.210777+0000", 2024-09-21T21:43:09.151 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 86.912605849000002, 2024-09-21T21:43:09.151 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.030410566, 2024-09-21T21:43:09.151 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:09.151 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:09.151 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:09.151 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:09.151 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5415", 2024-09-21T21:43:09.151 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1212816607", 2024-09-21T21:43:09.151 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 380 2024-09-21T21:43:09.151 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.151 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:09.151 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.151 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:09.151 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210777+0000", 2024-09-21T21:43:09.152 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:09.152 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.152 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.152 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:09.152 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210777+0000", 2024-09-21T21:43:09.152 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:09.152 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.152 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.152 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:09.152 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210778+0000", 2024-09-21T21:43:09.152 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8999999999999996e-07 2024-09-21T21:43:09.152 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.152 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.152 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:09.153 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210780+0000", 2024-09-21T21:43:09.153 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.503e-06 2024-09-21T21:43:09.153 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.153 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.153 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:09.153 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210788+0000", 2024-09-21T21:43:09.153 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0999999999999998e-06 2024-09-21T21:43:09.153 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.153 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.153 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:09.153 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210799+0000", 2024-09-21T21:43:09.153 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1703e-05 2024-09-21T21:43:09.153 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.153 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.154 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:09.154 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210833+0000", 2024-09-21T21:43:09.154 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4008999999999999e-05 2024-09-21T21:43:09.154 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.154 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.154 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:09.154 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210838+0000", 2024-09-21T21:43:09.154 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7480000000000001e-06 2024-09-21T21:43:09.154 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.154 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.154 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:09.154 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.239612+0000", 2024-09-21T21:43:09.154 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.028773554 2024-09-21T21:43:09.154 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.155 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.155 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:09.155 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.239677+0000", 2024-09-21T21:43:09.155 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5053000000000002e-05 2024-09-21T21:43:09.155 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.155 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.155 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:09.155 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.241188+0000", 2024-09-21T21:43:09.155 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001511306 2024-09-21T21:43:09.155 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:09.155 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:09.155 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:09.155 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.156 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.156 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24794 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:09.156 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.584085+0000", 2024-09-21T21:43:09.156 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 46.539298672999998, 2024-09-21T21:43:09.156 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.096209996000000006, 2024-09-21T21:43:09.156 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:09.156 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:09.156 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:09.156 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:09.156 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:09.156 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:09.156 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24794 2024-09-21T21:43:09.156 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.156 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:09.157 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.157 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:09.157 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584085+0000", 2024-09-21T21:43:09.157 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:09.157 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.157 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.157 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:09.157 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584085+0000", 2024-09-21T21:43:09.157 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:09.157 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.157 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.157 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:09.157 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584085+0000", 2024-09-21T21:43:09.157 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3200000000000001e-07 2024-09-21T21:43:09.157 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.158 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.158 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:09.158 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584086+0000", 2024-09-21T21:43:09.158 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.717e-06 2024-09-21T21:43:09.158 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.158 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.158 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:09.158 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584090+0000", 2024-09-21T21:43:09.158 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3450000000000002e-06 2024-09-21T21:43:09.158 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.158 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.158 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:09.158 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584092+0000", 2024-09-21T21:43:09.158 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6359999999999998e-06 2024-09-21T21:43:09.159 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.159 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.159 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:09.159 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.676497+0000", 2024-09-21T21:43:09.159 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.092404659 2024-09-21T21:43:09.159 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.159 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.159 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:09.159 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.676567+0000", 2024-09-21T21:43:09.159 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9627000000000003e-05 2024-09-21T21:43:09.159 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.159 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.159 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:09.159 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.676925+0000", 2024-09-21T21:43:09.160 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00035848500000000002 2024-09-21T21:43:09.160 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.160 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.160 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:09.160 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.678427+0000", 2024-09-21T21:43:09.160 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001501638 2024-09-21T21:43:09.160 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.160 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.160 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:09.160 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.680204+0000", 2024-09-21T21:43:09.160 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0017766979999999999 2024-09-21T21:43:09.160 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.160 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.160 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:09.160 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.680256+0000", 2024-09-21T21:43:09.161 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2549999999999997e-05 2024-09-21T21:43:09.161 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.161 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.161 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:09.161 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.680295+0000", 2024-09-21T21:43:09.161 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8408999999999998e-05 2024-09-21T21:43:09.161 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:09.161 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:09.161 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:09.161 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.161 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.161 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24796 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:09.161 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.584099+0000", 2024-09-21T21:43:09.161 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 46.539284104000004, 2024-09-21T21:43:09.162 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.097297429000000005, 2024-09-21T21:43:09.162 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:09.162 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:09.162 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:09.162 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:09.162 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:09.162 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:09.162 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24796 2024-09-21T21:43:09.162 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.162 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:09.162 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.162 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:09.162 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584099+0000", 2024-09-21T21:43:09.162 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:09.162 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.163 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.163 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:09.163 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584099+0000", 2024-09-21T21:43:09.163 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:09.163 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.163 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.163 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:09.163 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584099+0000", 2024-09-21T21:43:09.163 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3999999999999998e-07 2024-09-21T21:43:09.163 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.163 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.163 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:09.163 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584101+0000", 2024-09-21T21:43:09.163 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5850000000000001e-06 2024-09-21T21:43:09.164 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.164 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.164 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:09.164 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584108+0000", 2024-09-21T21:43:09.164 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.498e-06 2024-09-21T21:43:09.164 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.164 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.164 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:09.164 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584112+0000", 2024-09-21T21:43:09.164 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.252e-06 2024-09-21T21:43:09.164 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.164 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.164 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:09.164 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.677336+0000", 2024-09-21T21:43:09.165 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.093224206000000004 2024-09-21T21:43:09.165 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.165 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.165 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:09.165 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.677411+0000", 2024-09-21T21:43:09.165 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5036000000000005e-05 2024-09-21T21:43:09.165 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.165 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.165 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:09.165 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.677782+0000", 2024-09-21T21:43:09.165 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00037153099999999998 2024-09-21T21:43:09.165 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.165 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.165 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:09.165 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.680360+0000", 2024-09-21T21:43:09.166 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025773020000000001 2024-09-21T21:43:09.166 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.166 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.166 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:09.166 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681313+0000", 2024-09-21T21:43:09.166 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00095341100000000004 2024-09-21T21:43:09.166 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.166 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.166 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:09.166 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681361+0000", 2024-09-21T21:43:09.166 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7497999999999998e-05 2024-09-21T21:43:09.166 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.166 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.166 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:09.167 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681397+0000", 2024-09-21T21:43:09.167 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.587e-05 2024-09-21T21:43:09.167 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:09.167 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:09.167 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:09.167 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.167 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.167 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24797 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:09.167 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.584116+0000", 2024-09-21T21:43:09.167 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 46.539267615, 2024-09-21T21:43:09.167 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.097428257000000004, 2024-09-21T21:43:09.167 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:09.167 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:09.167 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:09.168 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:09.168 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:09.168 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:09.168 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24797 2024-09-21T21:43:09.168 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.168 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:09.168 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.168 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:09.168 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584116+0000", 2024-09-21T21:43:09.168 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:09.168 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.168 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.168 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:09.168 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584116+0000", 2024-09-21T21:43:09.168 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:09.169 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.169 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.169 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:09.169 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584116+0000", 2024-09-21T21:43:09.169 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.53e-07 2024-09-21T21:43:09.169 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.169 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.169 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:09.169 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584118+0000", 2024-09-21T21:43:09.169 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6810000000000001e-06 2024-09-21T21:43:09.169 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.169 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.169 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:09.169 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584121+0000", 2024-09-21T21:43:09.170 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1099999999999999e-06 2024-09-21T21:43:09.170 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.170 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.170 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:09.170 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584123+0000", 2024-09-21T21:43:09.170 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5890000000000001e-06 2024-09-21T21:43:09.170 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.170 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.170 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:09.170 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.678476+0000", 2024-09-21T21:43:09.170 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.094352320000000003 2024-09-21T21:43:09.170 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.170 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.170 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:09.171 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.678556+0000", 2024-09-21T21:43:09.171 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0291999999999996e-05 2024-09-21T21:43:09.171 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.171 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.171 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:09.171 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.678871+0000", 2024-09-21T21:43:09.171 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000314689 2024-09-21T21:43:09.171 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.171 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.171 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:09.171 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681452+0000", 2024-09-21T21:43:09.171 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025817890000000001 2024-09-21T21:43:09.171 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.171 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.171 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:09.172 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681467+0000", 2024-09-21T21:43:09.172 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4256e-05 2024-09-21T21:43:09.172 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.172 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.172 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:09.172 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681510+0000", 2024-09-21T21:43:09.172 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3560999999999998e-05 2024-09-21T21:43:09.172 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.172 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.172 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:09.172 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681544+0000", 2024-09-21T21:43:09.172 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3717000000000003e-05 2024-09-21T21:43:09.172 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:09.172 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:09.172 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:09.173 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.173 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.173 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24798 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:09.173 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.584130+0000", 2024-09-21T21:43:09.173 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 46.539253092000003, 2024-09-21T21:43:09.173 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.098350526999999993, 2024-09-21T21:43:09.173 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:09.173 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:09.173 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:09.173 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:09.173 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:09.173 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:09.173 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24798 2024-09-21T21:43:09.173 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.173 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:09.174 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.174 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:09.174 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584130+0000", 2024-09-21T21:43:09.174 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:09.174 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.174 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.174 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:09.174 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584130+0000", 2024-09-21T21:43:09.174 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:09.174 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.174 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.174 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:09.174 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584131+0000", 2024-09-21T21:43:09.174 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4700000000000001e-07 2024-09-21T21:43:09.174 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.175 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.175 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:09.175 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584133+0000", 2024-09-21T21:43:09.175 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.34e-06 2024-09-21T21:43:09.175 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.175 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.175 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:09.175 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584137+0000", 2024-09-21T21:43:09.175 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5609999999999999e-06 2024-09-21T21:43:09.175 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.175 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.175 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:09.175 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584139+0000", 2024-09-21T21:43:09.175 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7350000000000001e-06 2024-09-21T21:43:09.175 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.176 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.176 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:09.176 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.679274+0000", 2024-09-21T21:43:09.176 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.095134719000000006 2024-09-21T21:43:09.176 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.176 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.176 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:09.176 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.679358+0000", 2024-09-21T21:43:09.176 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3609999999999994e-05 2024-09-21T21:43:09.176 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.176 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.176 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:09.176 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.679685+0000", 2024-09-21T21:43:09.176 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00032732399999999999 2024-09-21T21:43:09.176 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.176 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.177 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:09.177 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681553+0000", 2024-09-21T21:43:09.177 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001868059 2024-09-21T21:43:09.177 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.177 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.177 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:09.177 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.682375+0000", 2024-09-21T21:43:09.177 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00082232099999999999 2024-09-21T21:43:09.177 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.177 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.177 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:09.177 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.682431+0000", 2024-09-21T21:43:09.177 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6051999999999998e-05 2024-09-21T21:43:09.177 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.177 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.178 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:09.178 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.682481+0000", 2024-09-21T21:43:09.178 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9258999999999997e-05 2024-09-21T21:43:09.178 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:09.178 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:09.178 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:09.178 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.178 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.178 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24799 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:09.178 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.584159+0000", 2024-09-21T21:43:09.178 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 46.539224103999999, 2024-09-21T21:43:09.178 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.100021551, 2024-09-21T21:43:09.178 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:09.178 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:09.178 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:09.179 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:09.179 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:09.179 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:09.179 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24799 2024-09-21T21:43:09.179 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.179 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:09.179 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.179 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:09.179 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584159+0000", 2024-09-21T21:43:09.179 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:09.179 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.179 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.179 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:09.179 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584159+0000", 2024-09-21T21:43:09.179 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:09.180 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.180 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.180 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:09.180 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584160+0000", 2024-09-21T21:43:09.180 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1300000000000001e-07 2024-09-21T21:43:09.180 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.180 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.180 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:09.180 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584162+0000", 2024-09-21T21:43:09.180 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5409999999999999e-06 2024-09-21T21:43:09.180 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.180 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.180 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:09.180 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584169+0000", 2024-09-21T21:43:09.180 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7880000000000001e-06 2024-09-21T21:43:09.181 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.181 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.181 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:09.181 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584174+0000", 2024-09-21T21:43:09.181 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9590000000000003e-06 2024-09-21T21:43:09.181 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.181 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.181 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:09.181 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.680400+0000", 2024-09-21T21:43:09.181 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.096226514999999999 2024-09-21T21:43:09.181 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.181 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.181 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:09.181 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.680480+0000", 2024-09-21T21:43:09.181 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9589e-05 2024-09-21T21:43:09.182 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.182 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.182 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:09.182 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.680847+0000", 2024-09-21T21:43:09.182 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00036757700000000001 2024-09-21T21:43:09.182 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.182 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.182 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:09.182 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.682555+0000", 2024-09-21T21:43:09.182 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001707791 2024-09-21T21:43:09.182 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.182 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.182 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:09.182 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684093+0000", 2024-09-21T21:43:09.182 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0015377170000000001 2024-09-21T21:43:09.183 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.183 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.183 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:09.183 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684144+0000", 2024-09-21T21:43:09.183 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1208000000000002e-05 2024-09-21T21:43:09.183 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.183 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.183 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:09.183 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684181+0000", 2024-09-21T21:43:09.183 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6452999999999998e-05 2024-09-21T21:43:09.183 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:09.183 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:09.183 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:09.183 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.183 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.184 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24835 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:09.184 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586332+0000", 2024-09-21T21:43:09.184 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 46.537051490000003, 2024-09-21T21:43:09.184 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.098730360000000003, 2024-09-21T21:43:09.184 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:09.184 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:09.184 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:09.184 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:09.184 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:09.184 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:09.184 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24835 2024-09-21T21:43:09.184 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.184 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:09.184 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.184 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:09.185 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586332+0000", 2024-09-21T21:43:09.185 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:09.185 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.185 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.185 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:09.185 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586332+0000", 2024-09-21T21:43:09.185 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:09.185 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.185 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.185 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:09.185 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586333+0000", 2024-09-21T21:43:09.185 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0200000000000001e-07 2024-09-21T21:43:09.185 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.185 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.185 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:09.186 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586346+0000", 2024-09-21T21:43:09.186 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3745e-05 2024-09-21T21:43:09.186 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.186 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.186 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:09.186 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586366+0000", 2024-09-21T21:43:09.186 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9272000000000001e-05 2024-09-21T21:43:09.186 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.186 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.186 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:09.186 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586377+0000", 2024-09-21T21:43:09.186 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1197e-05 2024-09-21T21:43:09.186 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.186 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.186 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:09.187 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681578+0000", 2024-09-21T21:43:09.187 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.095201420999999994 2024-09-21T21:43:09.187 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.187 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.187 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:09.187 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681640+0000", 2024-09-21T21:43:09.187 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1834000000000005e-05 2024-09-21T21:43:09.187 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.187 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.187 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:09.187 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681915+0000", 2024-09-21T21:43:09.187 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00027534199999999998 2024-09-21T21:43:09.187 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.187 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.187 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:09.188 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684230+0000", 2024-09-21T21:43:09.188 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00231424 2024-09-21T21:43:09.188 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.188 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.188 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:09.188 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684959+0000", 2024-09-21T21:43:09.188 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00072921399999999995 2024-09-21T21:43:09.188 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.188 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.188 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:09.188 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.685008+0000", 2024-09-21T21:43:09.188 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8860999999999999e-05 2024-09-21T21:43:09.188 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.188 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.188 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:09.189 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.685062+0000", 2024-09-21T21:43:09.189 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4432000000000003e-05 2024-09-21T21:43:09.189 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:09.189 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:09.189 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:09.189 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.189 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.189 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24837 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:09.189 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586395+0000", 2024-09-21T21:43:09.189 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 46.536988555000001, 2024-09-21T21:43:09.189 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.099700522, 2024-09-21T21:43:09.189 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:09.189 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:09.189 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:09.189 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:09.190 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:09.190 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:09.190 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24837 2024-09-21T21:43:09.190 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.190 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:09.190 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.190 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:09.190 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586395+0000", 2024-09-21T21:43:09.190 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:09.190 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.190 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.190 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:09.190 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586395+0000", 2024-09-21T21:43:09.190 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:09.190 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.191 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.191 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:09.191 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586395+0000", 2024-09-21T21:43:09.191 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.06e-07 2024-09-21T21:43:09.191 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.191 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.191 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:09.191 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586401+0000", 2024-09-21T21:43:09.191 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0889999999999996e-06 2024-09-21T21:43:09.191 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.191 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.191 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:09.191 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586410+0000", 2024-09-21T21:43:09.191 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.918e-06 2024-09-21T21:43:09.191 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.192 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.192 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:09.192 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586424+0000", 2024-09-21T21:43:09.192 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3478999999999999e-05 2024-09-21T21:43:09.192 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.192 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.192 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:09.192 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.682590+0000", 2024-09-21T21:43:09.192 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.096166086999999997 2024-09-21T21:43:09.192 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.192 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.192 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:09.192 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.682657+0000", 2024-09-21T21:43:09.192 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6878000000000006e-05 2024-09-21T21:43:09.193 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.193 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.193 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:09.193 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.682879+0000", 2024-09-21T21:43:09.193 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00022238 2024-09-21T21:43:09.193 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.193 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.193 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:09.193 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.685073+0000", 2024-09-21T21:43:09.193 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021938119999999998 2024-09-21T21:43:09.193 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.193 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.193 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:09.193 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686038+0000", 2024-09-21T21:43:09.193 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00096539499999999997 2024-09-21T21:43:09.194 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.194 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.194 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:09.194 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686071+0000", 2024-09-21T21:43:09.194 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2514000000000001e-05 2024-09-21T21:43:09.194 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.194 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.194 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:09.194 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686095+0000", 2024-09-21T21:43:09.194 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4363999999999999e-05 2024-09-21T21:43:09.194 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:09.194 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:09.194 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:09.194 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.194 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.195 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24839 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:09.195 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586468+0000", 2024-09-21T21:43:09.195 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 46.536914903000003, 2024-09-21T21:43:09.195 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.099739355000000002, 2024-09-21T21:43:09.195 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:09.195 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:09.195 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:09.195 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:09.195 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:09.195 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:09.195 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24839 2024-09-21T21:43:09.195 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.195 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:09.195 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.195 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:09.196 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586468+0000", 2024-09-21T21:43:09.196 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:09.196 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.196 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.196 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:09.196 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586468+0000", 2024-09-21T21:43:09.196 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:09.196 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.196 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.196 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:09.196 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586469+0000", 2024-09-21T21:43:09.196 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2200000000000004e-07 2024-09-21T21:43:09.196 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.196 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.196 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:09.197 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586472+0000", 2024-09-21T21:43:09.197 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9960000000000002e-06 2024-09-21T21:43:09.197 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.197 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.197 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:09.197 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586479+0000", 2024-09-21T21:43:09.197 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2699999999999999e-06 2024-09-21T21:43:09.197 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.197 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.197 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:09.197 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586485+0000", 2024-09-21T21:43:09.197 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6999999999999996e-06 2024-09-21T21:43:09.197 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.197 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.197 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:09.198 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.683276+0000", 2024-09-21T21:43:09.198 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.096791618999999995 2024-09-21T21:43:09.198 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.198 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.198 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:09.198 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.683326+0000", 2024-09-21T21:43:09.198 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9384e-05 2024-09-21T21:43:09.198 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.198 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.198 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:09.198 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.683655+0000", 2024-09-21T21:43:09.198 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00032965700000000002 2024-09-21T21:43:09.198 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.198 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.198 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:09.199 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686120+0000", 2024-09-21T21:43:09.199 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002464819 2024-09-21T21:43:09.199 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.199 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.199 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:09.199 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686159+0000", 2024-09-21T21:43:09.199 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9181000000000003e-05 2024-09-21T21:43:09.199 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.199 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.199 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:09.199 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686185+0000", 2024-09-21T21:43:09.199 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5907000000000001e-05 2024-09-21T21:43:09.199 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.199 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.199 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:09.200 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686208+0000", 2024-09-21T21:43:09.200 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.23e-05 2024-09-21T21:43:09.200 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:09.200 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:09.200 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:09.200 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.200 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.200 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24841 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:09.200 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586492+0000", 2024-09-21T21:43:09.200 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 46.536890741000001, 2024-09-21T21:43:09.200 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.101672077, 2024-09-21T21:43:09.200 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:09.200 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:09.200 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:09.200 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:09.201 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:09.201 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:09.201 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24841 2024-09-21T21:43:09.201 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.201 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:09.201 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.201 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:09.201 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586492+0000", 2024-09-21T21:43:09.201 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:09.201 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.201 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.201 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:09.201 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586492+0000", 2024-09-21T21:43:09.201 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:09.201 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.202 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.202 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:09.202 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586493+0000", 2024-09-21T21:43:09.202 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5999999999999999e-07 2024-09-21T21:43:09.202 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.202 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.202 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:09.202 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586496+0000", 2024-09-21T21:43:09.202 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6089999999999999e-06 2024-09-21T21:43:09.202 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.202 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.202 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:09.202 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586502+0000", 2024-09-21T21:43:09.202 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6270000000000002e-06 2024-09-21T21:43:09.203 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.203 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.203 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:09.203 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586507+0000", 2024-09-21T21:43:09.203 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8810000000000004e-06 2024-09-21T21:43:09.203 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.203 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.203 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:09.203 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684273+0000", 2024-09-21T21:43:09.203 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.097766415999999995 2024-09-21T21:43:09.203 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.203 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.203 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:09.203 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684359+0000", 2024-09-21T21:43:09.203 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5574000000000004e-05 2024-09-21T21:43:09.204 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.204 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.204 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:09.204 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684568+0000", 2024-09-21T21:43:09.204 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000208529 2024-09-21T21:43:09.204 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.204 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.204 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:09.204 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.687186+0000", 2024-09-21T21:43:09.204 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0026180769999999999 2024-09-21T21:43:09.204 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.204 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.204 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:09.204 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688093+0000", 2024-09-21T21:43:09.205 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00090766999999999996 2024-09-21T21:43:09.205 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.205 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.205 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:09.205 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688142+0000", 2024-09-21T21:43:09.205 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8986000000000002e-05 2024-09-21T21:43:09.205 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.205 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.205 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:09.205 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688165+0000", 2024-09-21T21:43:09.205 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2248000000000001e-05 2024-09-21T21:43:09.205 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:09.205 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:09.205 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:09.205 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.206 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.206 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24843 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:09.206 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586514+0000", 2024-09-21T21:43:09.206 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 46.536869455000001, 2024-09-21T21:43:09.206 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.101826449, 2024-09-21T21:43:09.206 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:09.206 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:09.206 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:09.206 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:09.206 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:09.206 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:09.206 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24843 2024-09-21T21:43:09.206 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.206 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:09.206 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.207 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:09.207 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586514+0000", 2024-09-21T21:43:09.207 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:09.207 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.207 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.207 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:09.207 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586514+0000", 2024-09-21T21:43:09.207 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:09.207 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.207 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.207 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:09.207 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586514+0000", 2024-09-21T21:43:09.207 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9999999999999998e-07 2024-09-21T21:43:09.207 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.207 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.208 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:09.208 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586517+0000", 2024-09-21T21:43:09.208 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4980000000000001e-06 2024-09-21T21:43:09.208 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.208 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.208 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:09.208 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586523+0000", 2024-09-21T21:43:09.208 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4609999999999996e-06 2024-09-21T21:43:09.208 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.208 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.208 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:09.208 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586528+0000", 2024-09-21T21:43:09.208 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2979999999999999e-06 2024-09-21T21:43:09.208 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.208 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.209 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:09.209 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.685104+0000", 2024-09-21T21:43:09.209 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.098575837999999999 2024-09-21T21:43:09.209 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.209 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.209 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:09.209 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.685184+0000", 2024-09-21T21:43:09.209 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9807000000000003e-05 2024-09-21T21:43:09.209 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.209 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.209 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:09.209 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.685496+0000", 2024-09-21T21:43:09.209 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00031225900000000001 2024-09-21T21:43:09.209 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.209 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.210 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:09.210 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688238+0000", 2024-09-21T21:43:09.210 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0027419409999999999 2024-09-21T21:43:09.210 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.210 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.210 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:09.210 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688253+0000", 2024-09-21T21:43:09.210 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4331000000000001e-05 2024-09-21T21:43:09.210 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.210 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.210 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:09.210 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688296+0000", 2024-09-21T21:43:09.210 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3801000000000003e-05 2024-09-21T21:43:09.210 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.210 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.211 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:09.211 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688340+0000", 2024-09-21T21:43:09.211 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3815e-05 2024-09-21T21:43:09.211 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:09.211 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:09.211 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:09.211 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.211 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.211 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24845 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:09.211 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586535+0000", 2024-09-21T21:43:09.211 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 46.536848575, 2024-09-21T21:43:09.211 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.10285678600000001, 2024-09-21T21:43:09.211 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:09.211 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:09.211 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:09.212 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:09.212 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:09.212 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:09.212 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24845 2024-09-21T21:43:09.212 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.212 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:09.212 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.212 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:09.212 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586535+0000", 2024-09-21T21:43:09.212 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:09.212 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.212 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.212 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:09.212 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586535+0000", 2024-09-21T21:43:09.212 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:09.213 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.213 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.213 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:09.213 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586535+0000", 2024-09-21T21:43:09.213 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9000000000000002e-07 2024-09-21T21:43:09.213 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.213 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.213 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:09.213 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586537+0000", 2024-09-21T21:43:09.213 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4569999999999999e-06 2024-09-21T21:43:09.213 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.213 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.213 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:09.213 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586544+0000", 2024-09-21T21:43:09.213 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5849999999999996e-06 2024-09-21T21:43:09.214 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.214 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.214 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:09.214 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586549+0000", 2024-09-21T21:43:09.214 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0089999999999998e-06 2024-09-21T21:43:09.214 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.214 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.214 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:09.214 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686239+0000", 2024-09-21T21:43:09.214 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.099689531999999997 2024-09-21T21:43:09.214 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.214 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.214 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:09.214 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686289+0000", 2024-09-21T21:43:09.214 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.007e-05 2024-09-21T21:43:09.215 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.215 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.215 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:09.215 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686633+0000", 2024-09-21T21:43:09.215 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00034428699999999998 2024-09-21T21:43:09.215 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.215 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.215 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:09.215 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688370+0000", 2024-09-21T21:43:09.215 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001736741 2024-09-21T21:43:09.215 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.215 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.215 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:09.215 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.689295+0000", 2024-09-21T21:43:09.215 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00092517900000000004 2024-09-21T21:43:09.216 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.216 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.216 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:09.216 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.689354+0000", 2024-09-21T21:43:09.216 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8799999999999999e-05 2024-09-21T21:43:09.216 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.216 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.216 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:09.216 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.689391+0000", 2024-09-21T21:43:09.216 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7735999999999999e-05 2024-09-21T21:43:09.216 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:09.216 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:09.216 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:09.216 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.216 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.216 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24846 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:09.217 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586564+0000", 2024-09-21T21:43:09.217 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 46.536819547, 2024-09-21T21:43:09.217 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.10375692, 2024-09-21T21:43:09.217 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:09.217 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:09.217 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:09.217 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:09.217 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:09.217 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:09.217 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24846 2024-09-21T21:43:09.217 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.217 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:09.217 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.217 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:09.218 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586564+0000", 2024-09-21T21:43:09.218 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:09.218 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.218 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.218 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:09.218 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586564+0000", 2024-09-21T21:43:09.218 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:09.218 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.218 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.218 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:09.218 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586564+0000", 2024-09-21T21:43:09.218 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4799999999999998e-07 2024-09-21T21:43:09.218 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.218 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.219 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:09.219 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586567+0000", 2024-09-21T21:43:09.219 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1669999999999999e-06 2024-09-21T21:43:09.219 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.219 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.219 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:09.219 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586578+0000", 2024-09-21T21:43:09.219 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0443e-05 2024-09-21T21:43:09.219 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.219 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.219 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:09.219 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586583+0000", 2024-09-21T21:43:09.219 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6790000000000002e-06 2024-09-21T21:43:09.219 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.219 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.220 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:09.220 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.687215+0000", 2024-09-21T21:43:09.220 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.100631973 2024-09-21T21:43:09.220 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.220 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.221 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:09.221 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.687311+0000", 2024-09-21T21:43:09.221 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5764e-05 2024-09-21T21:43:09.221 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.221 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.222 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:09.222 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.687663+0000", 2024-09-21T21:43:09.222 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00035167 2024-09-21T21:43:09.222 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.222 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.222 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:09.222 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690236+0000", 2024-09-21T21:43:09.222 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025733370000000002 2024-09-21T21:43:09.222 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.222 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.222 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:09.222 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690250+0000", 2024-09-21T21:43:09.222 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3548e-05 2024-09-21T21:43:09.222 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.223 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.223 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:09.223 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690298+0000", 2024-09-21T21:43:09.223 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7948999999999998e-05 2024-09-21T21:43:09.223 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.223 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.223 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:09.223 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690321+0000", 2024-09-21T21:43:09.223 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2841999999999999e-05 2024-09-21T21:43:09.223 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:09.223 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:09.223 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:09.223 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.223 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.223 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24849 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:09.224 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586633+0000", 2024-09-21T21:43:09.224 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 46.536749880999999, 2024-09-21T21:43:09.224 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.104506548, 2024-09-21T21:43:09.224 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:09.224 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:09.224 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:09.224 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:09.224 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:09.224 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:09.224 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24849 2024-09-21T21:43:09.224 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.224 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:09.224 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.224 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:09.224 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586633+0000", 2024-09-21T21:43:09.225 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:09.225 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.225 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.225 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:09.225 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586633+0000", 2024-09-21T21:43:09.225 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:09.225 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.225 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.225 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:09.225 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586634+0000", 2024-09-21T21:43:09.225 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9100000000000004e-07 2024-09-21T21:43:09.225 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.225 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.225 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:09.225 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586637+0000", 2024-09-21T21:43:09.226 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1999999999999999e-06 2024-09-21T21:43:09.226 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.226 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.226 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:09.226 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586645+0000", 2024-09-21T21:43:09.226 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7759999999999997e-06 2024-09-21T21:43:09.226 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.226 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.226 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:09.226 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586652+0000", 2024-09-21T21:43:09.226 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3560000000000004e-06 2024-09-21T21:43:09.226 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.226 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.226 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:09.226 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688395+0000", 2024-09-21T21:43:09.227 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.101743239 2024-09-21T21:43:09.227 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.227 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.227 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:09.227 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688464+0000", 2024-09-21T21:43:09.227 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8273000000000002e-05 2024-09-21T21:43:09.227 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.227 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.227 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:09.227 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688786+0000", 2024-09-21T21:43:09.227 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000322064 2024-09-21T21:43:09.227 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.227 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.227 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:09.227 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690364+0000", 2024-09-21T21:43:09.228 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001578084 2024-09-21T21:43:09.228 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.228 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.228 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:09.228 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.691050+0000", 2024-09-21T21:43:09.228 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00068623399999999998 2024-09-21T21:43:09.228 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.228 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.228 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:09.228 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.691090+0000", 2024-09-21T21:43:09.228 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9406999999999997e-05 2024-09-21T21:43:09.228 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.228 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.228 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:09.228 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.691140+0000", 2024-09-21T21:43:09.228 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0324000000000001e-05 2024-09-21T21:43:09.229 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:09.229 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:09.229 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:09.229 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.229 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.229 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24851 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:09.229 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586661+0000", 2024-09-21T21:43:09.229 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 46.536722595000001, 2024-09-21T21:43:09.229 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.10536891399999999, 2024-09-21T21:43:09.229 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:09.229 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:09.229 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:09.229 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:09.229 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:09.229 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:09.230 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24851 2024-09-21T21:43:09.230 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.230 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:09.230 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.230 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:09.230 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586661+0000", 2024-09-21T21:43:09.230 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:09.230 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.230 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.230 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:09.230 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586661+0000", 2024-09-21T21:43:09.230 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:09.230 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.230 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.231 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:09.231 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586661+0000", 2024-09-21T21:43:09.231 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4700000000000002e-07 2024-09-21T21:43:09.231 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.231 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.231 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:09.231 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586664+0000", 2024-09-21T21:43:09.231 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.672e-06 2024-09-21T21:43:09.231 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.231 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.231 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:09.231 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586671+0000", 2024-09-21T21:43:09.231 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9959999999999996e-06 2024-09-21T21:43:09.231 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.231 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.232 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:09.232 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586676+0000", 2024-09-21T21:43:09.232 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4020000000000001e-06 2024-09-21T21:43:09.232 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.232 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.232 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:09.232 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.689438+0000", 2024-09-21T21:43:09.232 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.102761503 2024-09-21T21:43:09.232 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.232 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.232 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:09.232 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.689524+0000", 2024-09-21T21:43:09.232 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6383000000000003e-05 2024-09-21T21:43:09.232 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.232 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.233 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:09.233 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.689769+0000", 2024-09-21T21:43:09.233 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00024504699999999997 2024-09-21T21:43:09.233 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.233 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.233 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:09.233 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.691936+0000", 2024-09-21T21:43:09.233 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021665130000000001 2024-09-21T21:43:09.233 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.233 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.233 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:09.233 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.691947+0000", 2024-09-21T21:43:09.233 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1198e-05 2024-09-21T21:43:09.233 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.233 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.234 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:09.234 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.691992+0000", 2024-09-21T21:43:09.234 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4774000000000002e-05 2024-09-21T21:43:09.234 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.234 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.234 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:09.234 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.692030+0000", 2024-09-21T21:43:09.234 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7978999999999997e-05 2024-09-21T21:43:09.234 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:09.234 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:09.234 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:09.234 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.234 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.234 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24853 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:09.235 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586684+0000", 2024-09-21T21:43:09.235 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 46.536699638000002, 2024-09-21T21:43:09.235 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.105936852, 2024-09-21T21:43:09.235 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:09.235 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:09.235 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:09.235 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:09.235 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:09.235 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:09.235 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24853 2024-09-21T21:43:09.235 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.235 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:09.235 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.235 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:09.235 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586684+0000", 2024-09-21T21:43:09.236 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:09.236 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.236 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.236 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:09.236 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586684+0000", 2024-09-21T21:43:09.236 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:09.236 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.236 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.236 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:09.236 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586684+0000", 2024-09-21T21:43:09.236 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.68e-07 2024-09-21T21:43:09.236 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.236 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.236 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:09.236 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586687+0000", 2024-09-21T21:43:09.237 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.774e-06 2024-09-21T21:43:09.237 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.237 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.237 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:09.237 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586694+0000", 2024-09-21T21:43:09.237 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6959999999999999e-06 2024-09-21T21:43:09.237 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.237 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.237 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:09.237 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586698+0000", 2024-09-21T21:43:09.237 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7489999999999998e-06 2024-09-21T21:43:09.237 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.237 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.237 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:09.237 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690395+0000", 2024-09-21T21:43:09.238 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.103696645 2024-09-21T21:43:09.238 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.238 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.238 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:09.238 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690440+0000", 2024-09-21T21:43:09.238 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4835e-05 2024-09-21T21:43:09.238 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.238 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.238 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:09.238 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690654+0000", 2024-09-21T21:43:09.238 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000214114 2024-09-21T21:43:09.238 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.238 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.238 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:09.238 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.692041+0000", 2024-09-21T21:43:09.239 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0013866519999999999 2024-09-21T21:43:09.239 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.239 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.239 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:09.239 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.692558+0000", 2024-09-21T21:43:09.239 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00051717199999999999 2024-09-21T21:43:09.239 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.239 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.239 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:09.239 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.692586+0000", 2024-09-21T21:43:09.239 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8504e-05 2024-09-21T21:43:09.239 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:09.239 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:09.239 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:09.240 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.692620+0000", 2024-09-21T21:43:09.240 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4143000000000002e-05 2024-09-21T21:43:09.240 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:09.240 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:09.240 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:09.240 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:09.240 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:09.240 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:43:09.241 INFO:teuthology.orchestra.run.smithi081.stderr:nodeep-scrub is unset 2024-09-21T21:43:09.242 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:43:09.240+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:43:09.242 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T21:43:09.245 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T21:43:09.342 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:43:09.341+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:43:09.352 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:43:09.352 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:43:09.352 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 0 2024-09-21T21:43:09.352 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:43:09.362 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T21:43:09.408 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0: {} 2024-09-21T21:43:09.409 INFO:teuthology.orchestra.run.smithi081.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T21:43:09.426 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1: {} 2024-09-21T21:43:09.427 INFO:teuthology.orchestra.run.smithi081.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T21:43:09.443 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2: {} 2024-09-21T21:43:09.443 INFO:teuthology.orchestra.run.smithi081.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T21:43:09.443 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:43:09.441+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:43:09.459 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3: {} 2024-09-21T21:43:09.460 INFO:teuthology.orchestra.run.smithi081.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T21:43:09.476 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4: {} 2024-09-21T21:43:09.476 INFO:teuthology.orchestra.run.smithi081.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T21:43:09.493 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5: {} 2024-09-21T21:43:09.493 INFO:teuthology.orchestra.run.smithi081.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T21:43:09.509 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:09.509 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:09.519 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T21:43:09.544 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:43:09.543+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:43:09.645 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:43:09.644+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:43:09.661 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:09.661 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:09.670 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T21:43:09.746 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:43:09.745+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:43:09.813 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:09.813 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:09.821 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T21:43:09.865 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:43:09.864+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:43:09.924 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:09.924 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:09.932 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T21:43:09.948 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:43:09.947+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:43:10.049 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:43:10.048+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:43:10.074 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:10.074 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:10.083 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T21:43:10.150 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:43:10.149+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:43:10.186 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:10.186 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:10.195 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T21:43:10.278 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:43:10.277+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:43:10.348 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:10.349 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:10.352 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:43:10.350+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:43:10.358 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T21:43:10.453 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:43:10.451+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:43:10.507 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:10.507 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:10.515 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T21:43:10.554 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:43:10.553+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:43:10.619 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:10.619 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:10.628 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T21:43:10.654 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:43:10.653+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:43:10.731 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:10.731 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:10.740 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T21:43:10.755 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:43:10.754+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:43:10.843 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:10.843 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:10.851 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T21:43:10.896 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:43:10.895+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:43:10.954 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:10.954 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:10.959 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:43:10.957+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:43:10.963 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T21:43:11.059 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:43:11.058+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:43:11.107 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:11.107 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:11.115 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T21:43:11.199 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:43:11.198+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:43:11.247 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T21:43:11.262 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:11.263 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:11.273 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T21:43:11.290 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:43:11.289+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:43:11.361 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:43:11.359+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:43:11.419 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:11.419 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:11.428 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T21:43:11.462 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:43:11.460+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:43:11.504 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T21:43:11.562 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:43:11.561+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:43:11.570 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:11.571 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:11.579 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T21:43:11.622 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0: {} 2024-09-21T21:43:11.622 INFO:teuthology.orchestra.run.smithi081.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T21:43:11.639 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1: {} 2024-09-21T21:43:11.640 INFO:teuthology.orchestra.run.smithi081.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T21:43:11.655 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2: {} 2024-09-21T21:43:11.656 INFO:teuthology.orchestra.run.smithi081.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T21:43:11.663 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:43:11.662+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:43:11.672 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3: {} 2024-09-21T21:43:11.673 INFO:teuthology.orchestra.run.smithi081.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T21:43:11.688 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4: {} 2024-09-21T21:43:11.688 INFO:teuthology.orchestra.run.smithi081.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T21:43:11.705 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5: {} 2024-09-21T21:43:11.705 INFO:teuthology.orchestra.run.smithi081.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T21:43:11.725 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:43:11.725 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-21T21:43:11.726 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-21T21:43:11.726 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:43:11.726 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.726 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4385.0:15202 3.16 3:6888948b:::rbd_header.1121493f6459:head [watch reconnect cookie 140325440202496 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-21T21:43:11.726 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:41.736601+0000", 2024-09-21T21:43:11.726 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 89.986540249000001, 2024-09-21T21:43:11.726 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5040056769999999, 2024-09-21T21:43:11.726 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:11.726 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:11.726 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:11.726 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:11.726 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4385", 2024-09-21T21:43:11.726 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:43:11.727 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 15202 2024-09-21T21:43:11.727 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.727 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:11.727 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.727 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:11.727 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736601+0000", 2024-09-21T21:43:11.727 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:11.727 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.727 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.727 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:11.727 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736601+0000", 2024-09-21T21:43:11.727 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:11.727 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.727 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.727 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:11.727 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736602+0000", 2024-09-21T21:43:11.728 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0699999999999999e-06 2024-09-21T21:43:11.728 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.728 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.728 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:11.728 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736607+0000", 2024-09-21T21:43:11.728 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2520000000000001e-06 2024-09-21T21:43:11.728 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.728 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.728 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:11.728 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736617+0000", 2024-09-21T21:43:11.728 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0074e-05 2024-09-21T21:43:11.728 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.728 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.728 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:11.729 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736622+0000", 2024-09-21T21:43:11.729 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.101e-06 2024-09-21T21:43:11.729 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.729 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.729 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:11.729 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736664+0000", 2024-09-21T21:43:11.729 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1869999999999997e-05 2024-09-21T21:43:11.729 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.729 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.729 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op must wait for map", 2024-09-21T21:43:11.729 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736667+0000", 2024-09-21T21:43:11.729 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4750000000000002e-06 2024-09-21T21:43:11.729 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.729 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.729 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:11.730 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742886+0000", 2024-09-21T21:43:11.730 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0062187579999999996 2024-09-21T21:43:11.730 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.730 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.730 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:11.730 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742892+0000", 2024-09-21T21:43:11.730 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0440000000000001e-06 2024-09-21T21:43:11.730 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.730 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.730 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:11.730 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210307+0000", 2024-09-21T21:43:11.730 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.467414781 2024-09-21T21:43:11.730 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.730 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.730 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:11.731 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210322+0000", 2024-09-21T21:43:11.731 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5311e-05 2024-09-21T21:43:11.731 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.731 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.731 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:11.731 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.237624+0000", 2024-09-21T21:43:11.731 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0273018920000001 2024-09-21T21:43:11.731 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.731 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.731 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:11.731 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.237870+0000", 2024-09-21T21:43:11.731 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00024622899999999999 2024-09-21T21:43:11.731 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.731 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.732 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:11.732 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.240607+0000", 2024-09-21T21:43:11.732 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0027368200000000001 2024-09-21T21:43:11.732 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:11.732 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:11.732 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:11.732 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.732 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.732 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5415.0:379 3.16 3:6888948b:::rbd_header.1121493f6459:head [watch reconnect cookie 139702602617920 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-21T21:43:11.732 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:41.739544+0000", 2024-09-21T21:43:11.732 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 89.98359782, 2024-09-21T21:43:11.732 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.501180645, 2024-09-21T21:43:11.732 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:11.732 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:11.732 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:11.732 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:11.733 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5415", 2024-09-21T21:43:11.733 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1212816607", 2024-09-21T21:43:11.733 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 379 2024-09-21T21:43:11.733 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.733 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:11.733 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.733 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:11.733 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739544+0000", 2024-09-21T21:43:11.733 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:11.733 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.733 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.733 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:11.733 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739544+0000", 2024-09-21T21:43:11.733 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:11.733 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.734 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.734 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:11.734 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739545+0000", 2024-09-21T21:43:11.734 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.065e-06 2024-09-21T21:43:11.734 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.734 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.734 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:11.734 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739550+0000", 2024-09-21T21:43:11.734 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9599999999999999e-06 2024-09-21T21:43:11.734 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.734 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.734 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:11.734 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739565+0000", 2024-09-21T21:43:11.735 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5709000000000002e-05 2024-09-21T21:43:11.735 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.735 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.735 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:11.735 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739573+0000", 2024-09-21T21:43:11.735 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7849999999999992e-06 2024-09-21T21:43:11.735 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.735 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.735 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:11.735 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739634+0000", 2024-09-21T21:43:11.735 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0732000000000001e-05 2024-09-21T21:43:11.735 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.735 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.735 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op must wait for map", 2024-09-21T21:43:11.735 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739640+0000", 2024-09-21T21:43:11.736 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0000000000000002e-06 2024-09-21T21:43:11.736 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.736 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.736 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:11.736 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742958+0000", 2024-09-21T21:43:11.736 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0033182390000000002 2024-09-21T21:43:11.736 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.736 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.736 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:11.736 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742966+0000", 2024-09-21T21:43:11.736 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1610000000000003e-06 2024-09-21T21:43:11.736 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.736 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.736 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:11.736 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210405+0000", 2024-09-21T21:43:11.736 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.467438725 2024-09-21T21:43:11.737 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.737 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.737 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:11.737 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210415+0000", 2024-09-21T21:43:11.737 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0234000000000001e-05 2024-09-21T21:43:11.737 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.737 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.737 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:11.737 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.238743+0000", 2024-09-21T21:43:11.737 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0283273070000001 2024-09-21T21:43:11.737 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.737 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.737 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:11.737 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.238842+0000", 2024-09-21T21:43:11.737 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.9896999999999996e-05 2024-09-21T21:43:11.738 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.738 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.738 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:11.738 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.240724+0000", 2024-09-21T21:43:11.738 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001881831 2024-09-21T21:43:11.738 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:11.738 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:11.738 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:11.738 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.738 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.738 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4385.0:15203 3.16 3:6888948b:::rbd_header.1121493f6459:head [call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-21T21:43:11.738 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:41.739737+0000", 2024-09-21T21:43:11.738 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 89.983404809000007, 2024-09-21T21:43:11.738 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4989621799999999, 2024-09-21T21:43:11.739 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:11.739 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:11.739 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:11.739 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:11.739 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4385", 2024-09-21T21:43:11.739 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:43:11.739 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 15203 2024-09-21T21:43:11.739 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.739 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:11.739 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.739 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:11.739 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739737+0000", 2024-09-21T21:43:11.739 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:11.739 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.740 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.740 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:11.740 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739737+0000", 2024-09-21T21:43:11.740 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:11.740 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.740 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.740 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:11.740 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739738+0000", 2024-09-21T21:43:11.740 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.7499999999999998e-07 2024-09-21T21:43:11.740 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.740 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.740 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:11.740 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739743+0000", 2024-09-21T21:43:11.740 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5049999999999998e-06 2024-09-21T21:43:11.740 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.741 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.741 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:11.741 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739757+0000", 2024-09-21T21:43:11.741 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3947e-05 2024-09-21T21:43:11.741 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.741 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.741 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:11.741 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739765+0000", 2024-09-21T21:43:11.741 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9310000000000008e-06 2024-09-21T21:43:11.741 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.741 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.741 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:11.741 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739816+0000", 2024-09-21T21:43:11.741 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0559999999999997e-05 2024-09-21T21:43:11.741 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.742 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.742 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting_for_map not empty", 2024-09-21T21:43:11.742 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739821+0000", 2024-09-21T21:43:11.742 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3539999999999999e-06 2024-09-21T21:43:11.742 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.742 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.742 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:11.742 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742928+0000", 2024-09-21T21:43:11.742 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0031074399999999999 2024-09-21T21:43:11.742 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.742 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.742 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:11.742 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742937+0000", 2024-09-21T21:43:11.742 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1289999999999996e-06 2024-09-21T21:43:11.742 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.743 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.743 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:11.743 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210350+0000", 2024-09-21T21:43:11.743 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.46741311299999999 2024-09-21T21:43:11.743 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.743 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.743 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:11.743 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210358+0000", 2024-09-21T21:43:11.743 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1869999999999997e-06 2024-09-21T21:43:11.743 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.743 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.743 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:11.743 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.238398+0000", 2024-09-21T21:43:11.743 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0280397830000001 2024-09-21T21:43:11.743 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.744 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.744 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:11.744 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.238489+0000", 2024-09-21T21:43:11.744 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.0958000000000006e-05 2024-09-21T21:43:11.744 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.744 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.744 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:11.744 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.238699+0000", 2024-09-21T21:43:11.744 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00021029800000000001 2024-09-21T21:43:11.744 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:11.744 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:11.744 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:11.744 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.744 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.744 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4385.0:15204 3.16 3:6888948b:::rbd_header.1121493f6459:head [watch reconnect cookie 140325440202496 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-21T21:43:11.745 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:42.210726+0000", 2024-09-21T21:43:11.745 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 89.512415931000007, 2024-09-21T21:43:11.745 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.030304557, 2024-09-21T21:43:11.745 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:11.745 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:11.745 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:11.745 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:11.745 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4385", 2024-09-21T21:43:11.745 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:43:11.745 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 15204 2024-09-21T21:43:11.745 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.745 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:11.745 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.745 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:11.745 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210726+0000", 2024-09-21T21:43:11.746 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:11.746 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.746 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.746 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:11.746 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210726+0000", 2024-09-21T21:43:11.746 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:11.746 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.746 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.746 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:11.746 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210727+0000", 2024-09-21T21:43:11.746 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.204e-06 2024-09-21T21:43:11.746 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.746 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.746 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:11.747 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210732+0000", 2024-09-21T21:43:11.747 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5249999999999996e-06 2024-09-21T21:43:11.747 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.747 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.747 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:11.747 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210744+0000", 2024-09-21T21:43:11.747 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1405e-05 2024-09-21T21:43:11.747 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.747 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.747 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:11.747 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210754+0000", 2024-09-21T21:43:11.747 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0122999999999999e-05 2024-09-21T21:43:11.747 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.747 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.747 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:11.748 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210798+0000", 2024-09-21T21:43:11.748 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4005000000000003e-05 2024-09-21T21:43:11.748 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.748 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.748 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:11.748 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210803+0000", 2024-09-21T21:43:11.748 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5130000000000004e-06 2024-09-21T21:43:11.748 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.748 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.748 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:11.748 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.239260+0000", 2024-09-21T21:43:11.748 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0284563040000001 2024-09-21T21:43:11.748 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.748 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.748 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:11.748 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.239318+0000", 2024-09-21T21:43:11.749 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8486000000000002e-05 2024-09-21T21:43:11.749 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.749 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.749 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:11.749 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.241030+0000", 2024-09-21T21:43:11.749 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0017119920000000001 2024-09-21T21:43:11.749 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:11.749 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:11.749 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:11.749 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.749 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.749 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5415.0:380 3.16 3:6888948b:::rbd_header.1121493f6459:head [watch reconnect cookie 139702602617920 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-21T21:43:11.749 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:42.210777+0000", 2024-09-21T21:43:11.749 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 89.512364152000004, 2024-09-21T21:43:11.750 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.030410566, 2024-09-21T21:43:11.750 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:11.750 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:11.750 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:11.750 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:11.750 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5415", 2024-09-21T21:43:11.750 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1212816607", 2024-09-21T21:43:11.750 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 380 2024-09-21T21:43:11.750 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.750 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:11.750 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.750 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:11.750 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210777+0000", 2024-09-21T21:43:11.750 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:11.751 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.751 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.751 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:11.751 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210777+0000", 2024-09-21T21:43:11.751 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:11.751 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.751 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.751 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:11.751 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210778+0000", 2024-09-21T21:43:11.751 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8999999999999996e-07 2024-09-21T21:43:11.751 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.751 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.751 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:11.751 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210780+0000", 2024-09-21T21:43:11.752 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.503e-06 2024-09-21T21:43:11.752 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.752 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.752 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:11.752 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210788+0000", 2024-09-21T21:43:11.752 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0999999999999998e-06 2024-09-21T21:43:11.752 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.752 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.752 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:11.752 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210799+0000", 2024-09-21T21:43:11.752 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1703e-05 2024-09-21T21:43:11.752 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.752 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.752 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:11.752 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210833+0000", 2024-09-21T21:43:11.753 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4008999999999999e-05 2024-09-21T21:43:11.753 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.753 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.753 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:11.753 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210838+0000", 2024-09-21T21:43:11.753 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7480000000000001e-06 2024-09-21T21:43:11.753 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.753 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.753 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:11.753 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.239612+0000", 2024-09-21T21:43:11.753 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.028773554 2024-09-21T21:43:11.753 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.753 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.753 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:11.753 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.239677+0000", 2024-09-21T21:43:11.754 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5053000000000002e-05 2024-09-21T21:43:11.754 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.754 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.754 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:11.754 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.241188+0000", 2024-09-21T21:43:11.754 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001511306 2024-09-21T21:43:11.754 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:11.754 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:11.754 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:11.754 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.754 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.754 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24794 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:11.754 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.584085+0000", 2024-09-21T21:43:11.754 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 49.139056975999999, 2024-09-21T21:43:11.754 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.096209996000000006, 2024-09-21T21:43:11.755 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:11.755 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:11.755 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:11.755 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:11.755 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:11.755 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:11.755 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24794 2024-09-21T21:43:11.755 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.755 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:11.755 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.755 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:11.755 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584085+0000", 2024-09-21T21:43:11.755 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:11.755 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.756 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.756 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:11.756 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584085+0000", 2024-09-21T21:43:11.756 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:11.756 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.756 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.756 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:11.756 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584085+0000", 2024-09-21T21:43:11.756 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3200000000000001e-07 2024-09-21T21:43:11.756 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.756 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.756 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:11.756 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584086+0000", 2024-09-21T21:43:11.756 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.717e-06 2024-09-21T21:43:11.756 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.757 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.757 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:11.757 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584090+0000", 2024-09-21T21:43:11.757 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3450000000000002e-06 2024-09-21T21:43:11.757 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.757 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.757 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:11.757 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584092+0000", 2024-09-21T21:43:11.757 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6359999999999998e-06 2024-09-21T21:43:11.757 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.757 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.757 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:11.757 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.676497+0000", 2024-09-21T21:43:11.757 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.092404659 2024-09-21T21:43:11.757 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.757 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.758 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:11.758 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.676567+0000", 2024-09-21T21:43:11.758 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9627000000000003e-05 2024-09-21T21:43:11.758 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.758 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.758 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:11.758 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.676925+0000", 2024-09-21T21:43:11.758 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00035848500000000002 2024-09-21T21:43:11.758 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.758 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.758 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:11.758 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.678427+0000", 2024-09-21T21:43:11.758 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001501638 2024-09-21T21:43:11.758 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.759 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.759 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:11.759 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.680204+0000", 2024-09-21T21:43:11.759 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0017766979999999999 2024-09-21T21:43:11.759 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.759 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.759 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:11.759 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.680256+0000", 2024-09-21T21:43:11.759 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2549999999999997e-05 2024-09-21T21:43:11.759 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.759 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.759 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:11.759 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.680295+0000", 2024-09-21T21:43:11.759 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8408999999999998e-05 2024-09-21T21:43:11.759 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:11.760 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:11.760 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:11.760 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.760 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.760 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24796 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:11.760 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.584099+0000", 2024-09-21T21:43:11.760 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 49.139042406999998, 2024-09-21T21:43:11.760 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.097297429000000005, 2024-09-21T21:43:11.760 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:11.760 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:11.760 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:11.760 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:11.760 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:11.760 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:11.760 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24796 2024-09-21T21:43:11.761 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.761 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:11.761 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.761 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:11.761 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584099+0000", 2024-09-21T21:43:11.761 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:11.761 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.761 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.761 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:11.761 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584099+0000", 2024-09-21T21:43:11.761 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:11.761 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.761 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.761 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:11.761 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584099+0000", 2024-09-21T21:43:11.762 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3999999999999998e-07 2024-09-21T21:43:11.762 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.762 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.762 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:11.762 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584101+0000", 2024-09-21T21:43:11.762 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5850000000000001e-06 2024-09-21T21:43:11.762 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.762 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.762 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:11.762 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584108+0000", 2024-09-21T21:43:11.762 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.498e-06 2024-09-21T21:43:11.762 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.762 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.762 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:11.762 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584112+0000", 2024-09-21T21:43:11.763 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.252e-06 2024-09-21T21:43:11.763 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.763 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.763 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:11.763 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.677336+0000", 2024-09-21T21:43:11.763 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.093224206000000004 2024-09-21T21:43:11.763 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.763 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.763 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:11.763 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.677411+0000", 2024-09-21T21:43:11.763 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5036000000000005e-05 2024-09-21T21:43:11.763 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.763 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.763 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:11.764 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.677782+0000", 2024-09-21T21:43:11.764 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00037153099999999998 2024-09-21T21:43:11.764 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.764 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.764 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:11.764 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.680360+0000", 2024-09-21T21:43:11.764 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025773020000000001 2024-09-21T21:43:11.764 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.764 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.764 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:11.764 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681313+0000", 2024-09-21T21:43:11.764 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00095341100000000004 2024-09-21T21:43:11.764 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.764 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.765 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:11.765 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681361+0000", 2024-09-21T21:43:11.765 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7497999999999998e-05 2024-09-21T21:43:11.765 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.765 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.765 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:11.765 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681397+0000", 2024-09-21T21:43:11.765 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.587e-05 2024-09-21T21:43:11.765 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:11.765 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:11.765 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:11.765 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.765 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.765 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24797 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:11.765 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.584116+0000", 2024-09-21T21:43:11.766 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 49.139025918000002, 2024-09-21T21:43:11.766 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.097428257000000004, 2024-09-21T21:43:11.766 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:11.766 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:11.766 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:11.766 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:11.766 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:11.766 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:11.766 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24797 2024-09-21T21:43:11.766 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.766 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:11.766 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.766 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:11.766 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584116+0000", 2024-09-21T21:43:11.767 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:11.767 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.767 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.767 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:11.767 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584116+0000", 2024-09-21T21:43:11.767 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:11.767 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.767 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.767 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:11.767 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584116+0000", 2024-09-21T21:43:11.767 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.53e-07 2024-09-21T21:43:11.767 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.767 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.767 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:11.768 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584118+0000", 2024-09-21T21:43:11.768 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6810000000000001e-06 2024-09-21T21:43:11.768 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.768 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.768 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:11.768 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584121+0000", 2024-09-21T21:43:11.768 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1099999999999999e-06 2024-09-21T21:43:11.768 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.768 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.768 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:11.768 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584123+0000", 2024-09-21T21:43:11.768 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5890000000000001e-06 2024-09-21T21:43:11.768 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.768 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.768 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:11.769 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.678476+0000", 2024-09-21T21:43:11.769 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.094352320000000003 2024-09-21T21:43:11.769 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.769 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.769 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:11.769 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.678556+0000", 2024-09-21T21:43:11.769 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0291999999999996e-05 2024-09-21T21:43:11.769 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.769 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.769 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:11.769 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.678871+0000", 2024-09-21T21:43:11.769 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000314689 2024-09-21T21:43:11.769 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.769 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.769 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:11.770 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681452+0000", 2024-09-21T21:43:11.770 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025817890000000001 2024-09-21T21:43:11.770 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.770 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.770 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:11.770 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681467+0000", 2024-09-21T21:43:11.770 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4256e-05 2024-09-21T21:43:11.770 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.770 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.770 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:11.770 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681510+0000", 2024-09-21T21:43:11.770 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3560999999999998e-05 2024-09-21T21:43:11.770 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.771 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.771 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:11.771 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681544+0000", 2024-09-21T21:43:11.771 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3717000000000003e-05 2024-09-21T21:43:11.771 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:11.771 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:11.771 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:11.771 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.771 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.771 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24798 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:11.771 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.584130+0000", 2024-09-21T21:43:11.771 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 49.139011394999997, 2024-09-21T21:43:11.771 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.098350526999999993, 2024-09-21T21:43:11.771 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:11.771 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:11.772 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:11.772 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:11.772 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:11.772 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:11.772 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24798 2024-09-21T21:43:11.772 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.772 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:11.772 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.772 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:11.772 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584130+0000", 2024-09-21T21:43:11.772 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:11.772 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.772 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.772 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:11.772 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584130+0000", 2024-09-21T21:43:11.773 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:11.773 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.773 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.773 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:11.773 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584131+0000", 2024-09-21T21:43:11.773 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4700000000000001e-07 2024-09-21T21:43:11.773 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.773 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.773 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:11.773 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584133+0000", 2024-09-21T21:43:11.773 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.34e-06 2024-09-21T21:43:11.773 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.773 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.773 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:11.773 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584137+0000", 2024-09-21T21:43:11.774 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5609999999999999e-06 2024-09-21T21:43:11.774 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.774 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.774 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:11.774 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584139+0000", 2024-09-21T21:43:11.774 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7350000000000001e-06 2024-09-21T21:43:11.774 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.774 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.774 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:11.774 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.679274+0000", 2024-09-21T21:43:11.774 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.095134719000000006 2024-09-21T21:43:11.774 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.774 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.774 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:11.774 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.679358+0000", 2024-09-21T21:43:11.775 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3609999999999994e-05 2024-09-21T21:43:11.775 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.775 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.775 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:11.775 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.679685+0000", 2024-09-21T21:43:11.775 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00032732399999999999 2024-09-21T21:43:11.775 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.775 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.775 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:11.775 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681553+0000", 2024-09-21T21:43:11.775 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001868059 2024-09-21T21:43:11.775 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.775 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.775 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:11.775 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.682375+0000", 2024-09-21T21:43:11.776 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00082232099999999999 2024-09-21T21:43:11.776 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.776 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.776 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:11.776 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.682431+0000", 2024-09-21T21:43:11.776 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6051999999999998e-05 2024-09-21T21:43:11.776 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.776 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.776 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:11.776 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.682481+0000", 2024-09-21T21:43:11.776 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9258999999999997e-05 2024-09-21T21:43:11.776 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:11.776 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:11.776 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:11.776 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.777 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.777 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24799 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:11.777 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.584159+0000", 2024-09-21T21:43:11.777 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 49.138982407, 2024-09-21T21:43:11.777 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.100021551, 2024-09-21T21:43:11.777 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:11.777 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:11.777 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:11.777 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:11.777 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:11.777 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:11.777 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24799 2024-09-21T21:43:11.777 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.777 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:11.778 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.778 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:11.778 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584159+0000", 2024-09-21T21:43:11.778 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:11.778 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.778 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.778 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:11.778 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584159+0000", 2024-09-21T21:43:11.778 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:11.778 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.778 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.778 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:11.778 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584160+0000", 2024-09-21T21:43:11.778 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1300000000000001e-07 2024-09-21T21:43:11.778 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.779 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.779 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:11.779 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584162+0000", 2024-09-21T21:43:11.779 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5409999999999999e-06 2024-09-21T21:43:11.779 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.779 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.779 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:11.779 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584169+0000", 2024-09-21T21:43:11.779 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7880000000000001e-06 2024-09-21T21:43:11.779 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.779 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.779 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:11.779 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584174+0000", 2024-09-21T21:43:11.779 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9590000000000003e-06 2024-09-21T21:43:11.779 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.780 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.780 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:11.780 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.680400+0000", 2024-09-21T21:43:11.780 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.096226514999999999 2024-09-21T21:43:11.780 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.780 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.780 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:11.780 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.680480+0000", 2024-09-21T21:43:11.780 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9589e-05 2024-09-21T21:43:11.780 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.780 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.780 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:11.780 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.680847+0000", 2024-09-21T21:43:11.780 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00036757700000000001 2024-09-21T21:43:11.780 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.781 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.781 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:11.781 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.682555+0000", 2024-09-21T21:43:11.781 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001707791 2024-09-21T21:43:11.781 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.781 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.781 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:11.781 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684093+0000", 2024-09-21T21:43:11.781 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0015377170000000001 2024-09-21T21:43:11.781 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.781 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.781 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:11.781 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684144+0000", 2024-09-21T21:43:11.781 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1208000000000002e-05 2024-09-21T21:43:11.781 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.782 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.782 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:11.782 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684181+0000", 2024-09-21T21:43:11.782 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6452999999999998e-05 2024-09-21T21:43:11.782 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:11.782 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:11.782 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:11.782 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.782 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.782 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24835 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:11.782 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586332+0000", 2024-09-21T21:43:11.782 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 49.136809792999998, 2024-09-21T21:43:11.782 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.098730360000000003, 2024-09-21T21:43:11.782 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:11.783 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:11.783 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:11.783 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:11.783 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:11.783 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:11.783 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24835 2024-09-21T21:43:11.783 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.783 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:11.783 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.783 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:11.783 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586332+0000", 2024-09-21T21:43:11.783 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:11.783 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.783 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.783 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:11.784 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586332+0000", 2024-09-21T21:43:11.784 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:11.784 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.784 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.784 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:11.784 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586333+0000", 2024-09-21T21:43:11.784 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0200000000000001e-07 2024-09-21T21:43:11.784 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.784 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.784 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:11.784 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586346+0000", 2024-09-21T21:43:11.784 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3745e-05 2024-09-21T21:43:11.784 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.784 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.784 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:11.785 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586366+0000", 2024-09-21T21:43:11.785 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9272000000000001e-05 2024-09-21T21:43:11.785 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.785 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.785 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:11.785 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586377+0000", 2024-09-21T21:43:11.785 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1197e-05 2024-09-21T21:43:11.785 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.785 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.785 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:11.785 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681578+0000", 2024-09-21T21:43:11.785 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.095201420999999994 2024-09-21T21:43:11.785 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.785 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.786 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:11.786 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681640+0000", 2024-09-21T21:43:11.786 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1834000000000005e-05 2024-09-21T21:43:11.786 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.786 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.786 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:11.786 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681915+0000", 2024-09-21T21:43:11.786 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00027534199999999998 2024-09-21T21:43:11.786 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.786 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.786 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:11.786 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684230+0000", 2024-09-21T21:43:11.786 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00231424 2024-09-21T21:43:11.786 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.786 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.787 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:11.787 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684959+0000", 2024-09-21T21:43:11.787 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00072921399999999995 2024-09-21T21:43:11.787 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.787 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.787 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:11.787 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.685008+0000", 2024-09-21T21:43:11.787 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8860999999999999e-05 2024-09-21T21:43:11.787 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.787 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.787 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:11.787 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.685062+0000", 2024-09-21T21:43:11.787 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4432000000000003e-05 2024-09-21T21:43:11.787 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:11.787 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:11.787 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:11.788 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.788 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.788 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24837 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:11.788 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586395+0000", 2024-09-21T21:43:11.788 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 49.136746858000002, 2024-09-21T21:43:11.788 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.099700522, 2024-09-21T21:43:11.788 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:11.788 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:11.788 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:11.788 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:11.788 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:11.788 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:11.788 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24837 2024-09-21T21:43:11.788 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.788 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:11.789 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.789 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:11.789 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586395+0000", 2024-09-21T21:43:11.789 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:11.789 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.789 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.789 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:11.789 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586395+0000", 2024-09-21T21:43:11.789 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:11.789 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.789 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.789 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:11.789 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586395+0000", 2024-09-21T21:43:11.789 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.06e-07 2024-09-21T21:43:11.789 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.790 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.790 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:11.790 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586401+0000", 2024-09-21T21:43:11.790 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0889999999999996e-06 2024-09-21T21:43:11.790 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.790 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.790 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:11.790 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586410+0000", 2024-09-21T21:43:11.790 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.918e-06 2024-09-21T21:43:11.790 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.790 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.790 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:11.790 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586424+0000", 2024-09-21T21:43:11.790 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3478999999999999e-05 2024-09-21T21:43:11.791 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.791 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.791 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:11.791 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.682590+0000", 2024-09-21T21:43:11.791 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.096166086999999997 2024-09-21T21:43:11.791 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.791 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.791 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:11.791 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.682657+0000", 2024-09-21T21:43:11.791 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6878000000000006e-05 2024-09-21T21:43:11.791 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.791 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.791 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:11.791 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.682879+0000", 2024-09-21T21:43:11.791 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00022238 2024-09-21T21:43:11.792 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.792 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.792 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:11.792 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.685073+0000", 2024-09-21T21:43:11.792 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021938119999999998 2024-09-21T21:43:11.792 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.792 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.792 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:11.792 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686038+0000", 2024-09-21T21:43:11.792 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00096539499999999997 2024-09-21T21:43:11.792 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.792 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.792 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:11.792 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686071+0000", 2024-09-21T21:43:11.793 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2514000000000001e-05 2024-09-21T21:43:11.793 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.793 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.793 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:11.793 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686095+0000", 2024-09-21T21:43:11.793 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4363999999999999e-05 2024-09-21T21:43:11.793 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:11.793 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:11.793 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:11.793 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.793 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.793 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24839 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:11.793 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586468+0000", 2024-09-21T21:43:11.793 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 49.136673205999998, 2024-09-21T21:43:11.793 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.099739355000000002, 2024-09-21T21:43:11.794 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:11.794 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:11.794 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:11.794 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:11.794 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:11.794 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:11.794 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24839 2024-09-21T21:43:11.794 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.794 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:11.794 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.794 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:11.794 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586468+0000", 2024-09-21T21:43:11.794 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:11.794 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.795 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.795 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:11.795 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586468+0000", 2024-09-21T21:43:11.795 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:11.795 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.795 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.795 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:11.795 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586469+0000", 2024-09-21T21:43:11.795 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2200000000000004e-07 2024-09-21T21:43:11.795 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.795 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.795 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:11.795 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586472+0000", 2024-09-21T21:43:11.795 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9960000000000002e-06 2024-09-21T21:43:11.795 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.796 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.796 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:11.796 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586479+0000", 2024-09-21T21:43:11.796 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2699999999999999e-06 2024-09-21T21:43:11.796 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.796 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.796 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:11.796 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586485+0000", 2024-09-21T21:43:11.796 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6999999999999996e-06 2024-09-21T21:43:11.796 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.796 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.796 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:11.796 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.683276+0000", 2024-09-21T21:43:11.796 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.096791618999999995 2024-09-21T21:43:11.797 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.797 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.797 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:11.797 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.683326+0000", 2024-09-21T21:43:11.797 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9384e-05 2024-09-21T21:43:11.797 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.797 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.797 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:11.797 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.683655+0000", 2024-09-21T21:43:11.797 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00032965700000000002 2024-09-21T21:43:11.797 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.797 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.797 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:11.797 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686120+0000", 2024-09-21T21:43:11.797 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002464819 2024-09-21T21:43:11.798 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.798 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.798 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:11.798 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686159+0000", 2024-09-21T21:43:11.798 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9181000000000003e-05 2024-09-21T21:43:11.798 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.798 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.798 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:11.798 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686185+0000", 2024-09-21T21:43:11.798 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5907000000000001e-05 2024-09-21T21:43:11.798 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.798 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.798 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:11.798 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686208+0000", 2024-09-21T21:43:11.799 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.23e-05 2024-09-21T21:43:11.799 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:11.799 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:11.799 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:11.799 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.799 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.799 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24841 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:11.799 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586492+0000", 2024-09-21T21:43:11.799 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 49.136649044000002, 2024-09-21T21:43:11.799 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.101672077, 2024-09-21T21:43:11.799 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:11.799 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:11.799 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:11.799 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:11.800 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:11.800 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:11.800 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24841 2024-09-21T21:43:11.800 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.800 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:11.800 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.800 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:11.800 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586492+0000", 2024-09-21T21:43:11.800 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:11.800 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.800 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.800 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:11.800 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586492+0000", 2024-09-21T21:43:11.800 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:11.800 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.801 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.801 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:11.801 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586493+0000", 2024-09-21T21:43:11.801 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5999999999999999e-07 2024-09-21T21:43:11.801 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.801 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.801 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:11.801 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586496+0000", 2024-09-21T21:43:11.801 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6089999999999999e-06 2024-09-21T21:43:11.801 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.801 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.801 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:11.801 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586502+0000", 2024-09-21T21:43:11.801 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6270000000000002e-06 2024-09-21T21:43:11.801 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.802 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.802 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:11.802 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586507+0000", 2024-09-21T21:43:11.802 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8810000000000004e-06 2024-09-21T21:43:11.802 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.802 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.802 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:11.802 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684273+0000", 2024-09-21T21:43:11.802 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.097766415999999995 2024-09-21T21:43:11.802 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.802 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.802 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:11.802 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684359+0000", 2024-09-21T21:43:11.802 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5574000000000004e-05 2024-09-21T21:43:11.802 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.803 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.803 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:11.803 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684568+0000", 2024-09-21T21:43:11.803 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000208529 2024-09-21T21:43:11.803 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.803 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.803 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:11.803 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.687186+0000", 2024-09-21T21:43:11.803 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0026180769999999999 2024-09-21T21:43:11.803 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.803 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.803 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:11.803 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688093+0000", 2024-09-21T21:43:11.803 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00090766999999999996 2024-09-21T21:43:11.803 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.804 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.804 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:11.804 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688142+0000", 2024-09-21T21:43:11.804 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8986000000000002e-05 2024-09-21T21:43:11.804 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.804 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.804 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:11.804 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688165+0000", 2024-09-21T21:43:11.804 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2248000000000001e-05 2024-09-21T21:43:11.804 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:11.804 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:11.804 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:11.804 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.804 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.804 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24843 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:11.805 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586514+0000", 2024-09-21T21:43:11.805 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 49.136627758000003, 2024-09-21T21:43:11.805 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.101826449, 2024-09-21T21:43:11.805 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:11.805 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:11.805 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:11.805 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:11.805 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:11.805 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:11.805 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24843 2024-09-21T21:43:11.805 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.805 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:11.805 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.805 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:11.806 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586514+0000", 2024-09-21T21:43:11.806 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:11.806 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.806 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.806 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:11.806 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586514+0000", 2024-09-21T21:43:11.806 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:11.806 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.806 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.806 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:11.806 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586514+0000", 2024-09-21T21:43:11.806 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9999999999999998e-07 2024-09-21T21:43:11.806 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.806 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.807 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:11.807 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586517+0000", 2024-09-21T21:43:11.807 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4980000000000001e-06 2024-09-21T21:43:11.807 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.807 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.807 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:11.807 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586523+0000", 2024-09-21T21:43:11.807 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4609999999999996e-06 2024-09-21T21:43:11.807 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.807 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.807 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:11.807 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586528+0000", 2024-09-21T21:43:11.807 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2979999999999999e-06 2024-09-21T21:43:11.807 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.807 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.808 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:11.808 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.685104+0000", 2024-09-21T21:43:11.808 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.098575837999999999 2024-09-21T21:43:11.808 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.808 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.808 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:11.808 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.685184+0000", 2024-09-21T21:43:11.808 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9807000000000003e-05 2024-09-21T21:43:11.808 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.808 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.808 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:11.808 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.685496+0000", 2024-09-21T21:43:11.808 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00031225900000000001 2024-09-21T21:43:11.808 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.808 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.809 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:11.809 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688238+0000", 2024-09-21T21:43:11.809 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0027419409999999999 2024-09-21T21:43:11.809 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.809 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.809 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:11.809 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688253+0000", 2024-09-21T21:43:11.809 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4331000000000001e-05 2024-09-21T21:43:11.809 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.809 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.809 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:11.809 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688296+0000", 2024-09-21T21:43:11.809 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3801000000000003e-05 2024-09-21T21:43:11.809 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.809 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.810 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:11.810 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688340+0000", 2024-09-21T21:43:11.810 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3815e-05 2024-09-21T21:43:11.810 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:11.810 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:11.810 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:11.810 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.810 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.810 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24845 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:11.810 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586535+0000", 2024-09-21T21:43:11.810 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 49.136606878000002, 2024-09-21T21:43:11.810 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.10285678600000001, 2024-09-21T21:43:11.810 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:11.810 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:11.811 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:11.811 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:11.811 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:11.811 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:11.811 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24845 2024-09-21T21:43:11.811 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.811 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:11.811 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.811 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:11.811 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586535+0000", 2024-09-21T21:43:11.811 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:11.811 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.811 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.811 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:11.811 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586535+0000", 2024-09-21T21:43:11.812 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:11.812 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.812 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.812 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:11.812 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586535+0000", 2024-09-21T21:43:11.812 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9000000000000002e-07 2024-09-21T21:43:11.812 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.812 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.812 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:11.812 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586537+0000", 2024-09-21T21:43:11.812 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4569999999999999e-06 2024-09-21T21:43:11.812 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.812 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.812 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:11.813 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586544+0000", 2024-09-21T21:43:11.813 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5849999999999996e-06 2024-09-21T21:43:11.813 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.813 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.813 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:11.813 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586549+0000", 2024-09-21T21:43:11.813 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0089999999999998e-06 2024-09-21T21:43:11.813 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.813 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.813 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:11.813 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686239+0000", 2024-09-21T21:43:11.813 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.099689531999999997 2024-09-21T21:43:11.813 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.813 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.813 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:11.814 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686289+0000", 2024-09-21T21:43:11.814 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.007e-05 2024-09-21T21:43:11.814 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.814 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.814 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:11.814 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686633+0000", 2024-09-21T21:43:11.814 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00034428699999999998 2024-09-21T21:43:11.814 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.814 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.814 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:11.814 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688370+0000", 2024-09-21T21:43:11.814 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001736741 2024-09-21T21:43:11.814 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.814 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.814 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:11.815 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.689295+0000", 2024-09-21T21:43:11.815 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00092517900000000004 2024-09-21T21:43:11.815 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.815 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.815 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:11.815 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.689354+0000", 2024-09-21T21:43:11.815 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8799999999999999e-05 2024-09-21T21:43:11.815 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.815 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.815 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:11.815 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.689391+0000", 2024-09-21T21:43:11.815 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7735999999999999e-05 2024-09-21T21:43:11.815 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:11.815 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:11.815 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:11.816 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.816 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.816 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24846 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:11.816 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586564+0000", 2024-09-21T21:43:11.816 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 49.136577850000002, 2024-09-21T21:43:11.816 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.10375692, 2024-09-21T21:43:11.816 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:11.816 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:11.816 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:11.816 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:11.816 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:11.816 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:11.816 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24846 2024-09-21T21:43:11.816 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.817 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:11.817 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.817 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:11.817 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586564+0000", 2024-09-21T21:43:11.817 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:11.817 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.817 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.817 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:11.817 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586564+0000", 2024-09-21T21:43:11.817 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:11.817 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.817 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.817 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:11.817 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586564+0000", 2024-09-21T21:43:11.817 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4799999999999998e-07 2024-09-21T21:43:11.818 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.818 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.818 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:11.818 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586567+0000", 2024-09-21T21:43:11.818 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1669999999999999e-06 2024-09-21T21:43:11.818 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.818 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.818 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:11.818 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586578+0000", 2024-09-21T21:43:11.818 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0443e-05 2024-09-21T21:43:11.818 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.818 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.818 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:11.819 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586583+0000", 2024-09-21T21:43:11.819 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6790000000000002e-06 2024-09-21T21:43:11.819 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.819 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.819 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:11.819 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.687215+0000", 2024-09-21T21:43:11.819 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.100631973 2024-09-21T21:43:11.819 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.819 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.819 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:11.819 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.687311+0000", 2024-09-21T21:43:11.819 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5764e-05 2024-09-21T21:43:11.819 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.819 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.819 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:11.820 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.687663+0000", 2024-09-21T21:43:11.820 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00035167 2024-09-21T21:43:11.820 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.820 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.820 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:11.820 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690236+0000", 2024-09-21T21:43:11.820 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025733370000000002 2024-09-21T21:43:11.820 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.820 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.820 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:11.820 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690250+0000", 2024-09-21T21:43:11.820 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3548e-05 2024-09-21T21:43:11.820 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.820 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.820 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:11.821 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690298+0000", 2024-09-21T21:43:11.821 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7948999999999998e-05 2024-09-21T21:43:11.821 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.821 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.821 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:11.821 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690321+0000", 2024-09-21T21:43:11.821 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2841999999999999e-05 2024-09-21T21:43:11.821 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:11.821 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:11.821 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:11.821 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.821 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.821 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24849 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:11.821 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586633+0000", 2024-09-21T21:43:11.821 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 49.136508184, 2024-09-21T21:43:11.822 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.104506548, 2024-09-21T21:43:11.822 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:11.822 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:11.822 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:11.822 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:11.822 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:11.822 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:11.822 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24849 2024-09-21T21:43:11.822 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.822 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:11.822 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.822 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:11.822 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586633+0000", 2024-09-21T21:43:11.822 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:11.822 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.823 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.823 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:11.823 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586633+0000", 2024-09-21T21:43:11.823 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:11.823 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.823 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.823 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:11.823 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586634+0000", 2024-09-21T21:43:11.823 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9100000000000004e-07 2024-09-21T21:43:11.823 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.823 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.823 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:11.823 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586637+0000", 2024-09-21T21:43:11.823 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1999999999999999e-06 2024-09-21T21:43:11.823 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.824 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.824 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:11.824 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586645+0000", 2024-09-21T21:43:11.824 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7759999999999997e-06 2024-09-21T21:43:11.824 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.824 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.824 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:11.824 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586652+0000", 2024-09-21T21:43:11.824 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3560000000000004e-06 2024-09-21T21:43:11.824 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.824 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.824 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:11.824 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688395+0000", 2024-09-21T21:43:11.824 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.101743239 2024-09-21T21:43:11.824 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.825 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.825 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:11.825 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688464+0000", 2024-09-21T21:43:11.825 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8273000000000002e-05 2024-09-21T21:43:11.825 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.825 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.825 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:11.825 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688786+0000", 2024-09-21T21:43:11.825 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000322064 2024-09-21T21:43:11.825 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.825 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.825 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:11.825 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690364+0000", 2024-09-21T21:43:11.825 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001578084 2024-09-21T21:43:11.825 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.826 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.826 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:11.826 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.691050+0000", 2024-09-21T21:43:11.826 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00068623399999999998 2024-09-21T21:43:11.826 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.826 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.826 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:11.826 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.691090+0000", 2024-09-21T21:43:11.826 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9406999999999997e-05 2024-09-21T21:43:11.826 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.826 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.826 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:11.826 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.691140+0000", 2024-09-21T21:43:11.826 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0324000000000001e-05 2024-09-21T21:43:11.826 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:11.827 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:11.827 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:11.827 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.827 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.827 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24851 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:11.827 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586661+0000", 2024-09-21T21:43:11.827 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 49.136480898000002, 2024-09-21T21:43:11.827 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.10536891399999999, 2024-09-21T21:43:11.827 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:11.827 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:11.827 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:11.827 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:11.827 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:11.827 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:11.827 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24851 2024-09-21T21:43:11.828 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.828 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:11.828 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.828 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:11.828 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586661+0000", 2024-09-21T21:43:11.828 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:11.828 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.828 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.828 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:11.828 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586661+0000", 2024-09-21T21:43:11.828 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:11.828 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.828 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.828 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:11.828 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586661+0000", 2024-09-21T21:43:11.829 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4700000000000002e-07 2024-09-21T21:43:11.829 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.829 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.829 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:11.829 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586664+0000", 2024-09-21T21:43:11.829 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.672e-06 2024-09-21T21:43:11.829 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.829 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.829 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:11.829 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586671+0000", 2024-09-21T21:43:11.829 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9959999999999996e-06 2024-09-21T21:43:11.829 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.829 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.829 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:11.829 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586676+0000", 2024-09-21T21:43:11.830 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4020000000000001e-06 2024-09-21T21:43:11.830 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.830 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.830 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:11.830 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.689438+0000", 2024-09-21T21:43:11.830 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.102761503 2024-09-21T21:43:11.830 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.830 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.830 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:11.830 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.689524+0000", 2024-09-21T21:43:11.830 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6383000000000003e-05 2024-09-21T21:43:11.830 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.830 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.830 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:11.830 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.689769+0000", 2024-09-21T21:43:11.831 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00024504699999999997 2024-09-21T21:43:11.831 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.831 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.831 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:11.831 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.691936+0000", 2024-09-21T21:43:11.831 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021665130000000001 2024-09-21T21:43:11.831 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.831 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.831 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:11.831 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.691947+0000", 2024-09-21T21:43:11.831 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1198e-05 2024-09-21T21:43:11.831 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.831 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.831 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:11.831 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.691992+0000", 2024-09-21T21:43:11.832 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4774000000000002e-05 2024-09-21T21:43:11.832 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.832 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.832 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:11.832 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.692030+0000", 2024-09-21T21:43:11.832 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7978999999999997e-05 2024-09-21T21:43:11.832 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:11.832 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:11.832 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:11.832 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.832 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.832 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24853 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:11.832 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586684+0000", 2024-09-21T21:43:11.832 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 49.136457941000003, 2024-09-21T21:43:11.832 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.105936852, 2024-09-21T21:43:11.833 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:11.833 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:11.833 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:11.833 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:11.833 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:11.833 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:11.833 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24853 2024-09-21T21:43:11.833 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.833 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:11.833 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.833 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:11.833 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586684+0000", 2024-09-21T21:43:11.833 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:11.833 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.833 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.834 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:11.834 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586684+0000", 2024-09-21T21:43:11.834 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:11.834 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.834 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.834 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:11.834 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586684+0000", 2024-09-21T21:43:11.834 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.68e-07 2024-09-21T21:43:11.834 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.834 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.834 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:11.834 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586687+0000", 2024-09-21T21:43:11.834 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.774e-06 2024-09-21T21:43:11.834 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.834 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.835 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:11.835 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586694+0000", 2024-09-21T21:43:11.835 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6959999999999999e-06 2024-09-21T21:43:11.835 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.835 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.835 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:11.835 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586698+0000", 2024-09-21T21:43:11.835 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7489999999999998e-06 2024-09-21T21:43:11.835 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.835 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.835 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:11.835 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690395+0000", 2024-09-21T21:43:11.835 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.103696645 2024-09-21T21:43:11.835 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.835 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.836 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:11.836 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690440+0000", 2024-09-21T21:43:11.836 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4835e-05 2024-09-21T21:43:11.836 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.836 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.836 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:11.836 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690654+0000", 2024-09-21T21:43:11.836 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000214114 2024-09-21T21:43:11.836 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.836 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.836 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:11.836 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.692041+0000", 2024-09-21T21:43:11.836 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0013866519999999999 2024-09-21T21:43:11.836 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.837 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.837 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:11.837 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.692558+0000", 2024-09-21T21:43:11.837 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00051717199999999999 2024-09-21T21:43:11.837 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.837 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.837 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:11.837 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.692586+0000", 2024-09-21T21:43:11.837 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8504e-05 2024-09-21T21:43:11.837 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:11.837 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:11.837 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:11.837 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.692620+0000", 2024-09-21T21:43:11.837 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4143000000000002e-05 2024-09-21T21:43:11.837 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:11.838 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:11.838 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:11.838 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:11.838 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:11.838 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:43:11.839 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T21:43:11.840 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:43:11.837+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:43:11.940 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:43:11.938+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:43:11.942 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:43:11.943 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:43:11.943 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 0 2024-09-21T21:43:11.943 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:43:11.951 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T21:43:12.041 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:43:12.039+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:43:12.054 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:43:12.054 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:43:12.054 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:43:12.054 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:43:12.055 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:43:12.063 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T21:43:12.147 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:43:12.146+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:43:12.208 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:43:12.209 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 20, 2024-09-21T21:43:12.209 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 600, 2024-09-21T21:43:12.209 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:43:12.209 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.209 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11319 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 5492736~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:12.209 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.712385+0000", 2024-09-21T21:43:12.209 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 81.494015082000004, 2024-09-21T21:43:12.209 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078627889000000006, 2024-09-21T21:43:12.209 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:12.209 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:12.210 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:12.210 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:12.210 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:12.210 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:12.210 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11319 2024-09-21T21:43:12.210 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.210 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:12.210 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.210 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:12.210 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712385+0000", 2024-09-21T21:43:12.210 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:12.210 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.210 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.210 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:12.211 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712385+0000", 2024-09-21T21:43:12.211 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:12.211 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.211 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.211 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:12.211 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712385+0000", 2024-09-21T21:43:12.211 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2800000000000002e-07 2024-09-21T21:43:12.211 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.211 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.211 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:12.211 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712387+0000", 2024-09-21T21:43:12.211 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1569999999999998e-06 2024-09-21T21:43:12.212 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.212 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.212 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:12.212 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712393+0000", 2024-09-21T21:43:12.212 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9370000000000001e-06 2024-09-21T21:43:12.212 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.212 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.212 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:12.212 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712398+0000", 2024-09-21T21:43:12.212 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5020000000000003e-06 2024-09-21T21:43:12.212 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.212 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.213 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:12.213 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.790746+0000", 2024-09-21T21:43:12.213 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078348589999999996 2024-09-21T21:43:12.213 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.213 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.213 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:12.213 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.790844+0000", 2024-09-21T21:43:12.213 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.7526000000000001e-05 2024-09-21T21:43:12.213 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.213 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.213 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:12.213 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.791013+0000", 2024-09-21T21:43:12.213 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00016874899999999999 2024-09-21T21:43:12.213 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:12.213 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:12.214 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:12.214 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.214 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.214 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11320 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 6180864~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:12.214 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.712405+0000", 2024-09-21T21:43:12.214 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 81.493995123000005, 2024-09-21T21:43:12.214 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078882870999999993, 2024-09-21T21:43:12.214 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:12.214 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:12.214 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:12.214 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:12.214 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:12.214 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:12.215 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11320 2024-09-21T21:43:12.215 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.215 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:12.215 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.215 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:12.215 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712405+0000", 2024-09-21T21:43:12.215 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:12.215 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.215 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.215 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:12.215 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712405+0000", 2024-09-21T21:43:12.215 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:12.215 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.215 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.216 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:12.216 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.713942+0000", 2024-09-21T21:43:12.216 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.001536882 2024-09-21T21:43:12.216 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.216 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.216 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:12.216 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.713946+0000", 2024-09-21T21:43:12.216 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4549999999999997e-06 2024-09-21T21:43:12.216 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.216 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.216 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:12.216 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.713958+0000", 2024-09-21T21:43:12.216 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1775e-05 2024-09-21T21:43:12.216 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.216 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.217 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:12.217 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.713965+0000", 2024-09-21T21:43:12.217 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5100000000000001e-06 2024-09-21T21:43:12.217 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.217 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.217 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:12.217 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.791059+0000", 2024-09-21T21:43:12.217 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.077094036000000005 2024-09-21T21:43:12.217 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.217 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.217 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:12.217 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.791175+0000", 2024-09-21T21:43:12.217 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000115895 2024-09-21T21:43:12.217 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.217 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.218 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:12.218 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.791288+0000", 2024-09-21T21:43:12.218 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000112318 2024-09-21T21:43:12.218 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:12.218 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:12.218 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:12.218 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.218 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.218 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11325 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 5414912~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:12.218 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.714095+0000", 2024-09-21T21:43:12.219 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 81.492305127999998, 2024-09-21T21:43:12.219 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078631894999999993, 2024-09-21T21:43:12.219 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:12.219 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:12.219 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:12.219 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:12.219 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:12.219 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:12.219 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11325 2024-09-21T21:43:12.219 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.219 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:12.219 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.219 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:12.219 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714095+0000", 2024-09-21T21:43:12.219 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:12.220 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.220 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.220 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:12.220 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714095+0000", 2024-09-21T21:43:12.220 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:12.220 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.220 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.220 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:12.220 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714095+0000", 2024-09-21T21:43:12.220 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7500000000000001e-07 2024-09-21T21:43:12.220 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.220 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.220 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:12.220 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714097+0000", 2024-09-21T21:43:12.221 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.12e-06 2024-09-21T21:43:12.221 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.221 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.221 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:12.221 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714104+0000", 2024-09-21T21:43:12.221 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3520000000000003e-06 2024-09-21T21:43:12.221 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.221 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.221 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:12.221 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714108+0000", 2024-09-21T21:43:12.221 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8810000000000004e-06 2024-09-21T21:43:12.222 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.222 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.222 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:12.222 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792509+0000", 2024-09-21T21:43:12.222 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078400739999999997 2024-09-21T21:43:12.222 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.222 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.222 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:12.222 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792613+0000", 2024-09-21T21:43:12.222 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000103939 2024-09-21T21:43:12.222 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.222 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.223 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:12.223 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792727+0000", 2024-09-21T21:43:12.223 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011348799999999999 2024-09-21T21:43:12.223 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:12.223 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:12.223 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:12.223 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.223 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.223 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11326 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 6103040~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:12.223 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.714115+0000", 2024-09-21T21:43:12.223 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 81.492285100000004, 2024-09-21T21:43:12.223 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078963056000000004, 2024-09-21T21:43:12.223 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:12.223 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:12.224 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:12.224 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:12.224 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:12.224 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:12.224 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11326 2024-09-21T21:43:12.224 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.224 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:12.224 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.224 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:12.224 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714115+0000", 2024-09-21T21:43:12.224 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:12.224 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.224 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.224 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:12.224 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714115+0000", 2024-09-21T21:43:12.225 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:12.225 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.225 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.225 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:12.225 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.715940+0000", 2024-09-21T21:43:12.225 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0018254969999999999 2024-09-21T21:43:12.225 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.225 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.225 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:12.225 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.715945+0000", 2024-09-21T21:43:12.225 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6120000000000001e-06 2024-09-21T21:43:12.225 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.225 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.225 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:12.226 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.715954+0000", 2024-09-21T21:43:12.226 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.5899999999999997e-06 2024-09-21T21:43:12.226 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.226 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.226 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:12.226 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.715959+0000", 2024-09-21T21:43:12.226 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0849999999999996e-06 2024-09-21T21:43:12.226 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.226 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.227 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:12.227 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792771+0000", 2024-09-21T21:43:12.227 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.076811829999999998 2024-09-21T21:43:12.227 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.227 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.227 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:12.227 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792882+0000", 2024-09-21T21:43:12.227 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000110857 2024-09-21T21:43:12.227 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.227 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.227 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:12.227 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.793078+0000", 2024-09-21T21:43:12.227 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00019558499999999999 2024-09-21T21:43:12.227 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:12.228 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:12.228 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:12.228 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.228 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.228 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11332 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 7565312~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:12.228 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.716063+0000", 2024-09-21T21:43:12.228 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 81.490336565999996, 2024-09-21T21:43:12.228 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078787482000000006, 2024-09-21T21:43:12.228 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:12.228 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:12.228 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:12.229 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:12.229 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:12.229 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:12.229 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11332 2024-09-21T21:43:12.229 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.229 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:12.229 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.229 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:12.229 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.716063+0000", 2024-09-21T21:43:12.229 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:12.229 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.229 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.230 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:12.230 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.716063+0000", 2024-09-21T21:43:12.230 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:12.230 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.230 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.230 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:12.230 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.717944+0000", 2024-09-21T21:43:12.230 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0018812309999999999 2024-09-21T21:43:12.230 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.230 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.230 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:12.230 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.717949+0000", 2024-09-21T21:43:12.231 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9389999999999996e-06 2024-09-21T21:43:12.231 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.231 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.231 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:12.231 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.717963+0000", 2024-09-21T21:43:12.231 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.3899e-05 2024-09-21T21:43:12.231 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.231 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.231 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:12.231 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.717970+0000", 2024-09-21T21:43:12.231 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.426e-06 2024-09-21T21:43:12.232 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.232 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.232 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:12.232 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.794573+0000", 2024-09-21T21:43:12.232 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.076602990999999995 2024-09-21T21:43:12.232 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.232 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.232 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:12.232 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.794682+0000", 2024-09-21T21:43:12.232 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000109541 2024-09-21T21:43:12.232 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.233 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.233 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:12.233 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.794851+0000", 2024-09-21T21:43:12.233 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000168455 2024-09-21T21:43:12.233 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:12.233 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:12.233 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:12.233 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.233 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.233 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11339 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 6266880~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:12.233 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.718117+0000", 2024-09-21T21:43:12.233 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 81.48828254, 2024-09-21T21:43:12.233 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078703717000000006, 2024-09-21T21:43:12.234 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:12.234 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:12.234 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:12.234 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:12.234 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:12.234 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:12.235 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11339 2024-09-21T21:43:12.235 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.235 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:12.235 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.236 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:12.236 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.718117+0000", 2024-09-21T21:43:12.236 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:12.236 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.236 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.236 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:12.236 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.718117+0000", 2024-09-21T21:43:12.236 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:12.236 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.236 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.236 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:12.236 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720019+0000", 2024-09-21T21:43:12.236 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0019012549999999999 2024-09-21T21:43:12.236 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.237 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.237 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:12.237 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720024+0000", 2024-09-21T21:43:12.237 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1100000000000002e-06 2024-09-21T21:43:12.237 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.237 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.237 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:12.237 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720035+0000", 2024-09-21T21:43:12.237 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.094e-05 2024-09-21T21:43:12.237 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.237 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.237 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:12.237 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720040+0000", 2024-09-21T21:43:12.237 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1830000000000002e-06 2024-09-21T21:43:12.237 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.238 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.238 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:12.238 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796618+0000", 2024-09-21T21:43:12.238 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.076578033000000004 2024-09-21T21:43:12.238 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.238 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.238 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:12.238 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796696+0000", 2024-09-21T21:43:12.238 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7836000000000005e-05 2024-09-21T21:43:12.238 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.238 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.238 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:12.238 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796821+0000", 2024-09-21T21:43:12.238 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00012536000000000001 2024-09-21T21:43:12.238 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:12.239 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:12.239 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:12.239 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.239 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.239 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11345 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 5472256~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:12.239 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.720116+0000", 2024-09-21T21:43:12.239 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 81.486283639999996, 2024-09-21T21:43:12.239 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078667730000000005, 2024-09-21T21:43:12.239 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:12.239 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:12.239 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:12.239 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:12.239 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:12.239 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:12.240 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11345 2024-09-21T21:43:12.240 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.240 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:12.240 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.240 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:12.240 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720116+0000", 2024-09-21T21:43:12.240 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:12.240 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.240 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.240 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:12.240 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720116+0000", 2024-09-21T21:43:12.240 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:12.240 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.240 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.240 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:12.241 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720116+0000", 2024-09-21T21:43:12.241 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.04e-07 2024-09-21T21:43:12.241 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.241 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.241 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:12.241 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720118+0000", 2024-09-21T21:43:12.241 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.181e-06 2024-09-21T21:43:12.241 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.241 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.241 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:12.241 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720121+0000", 2024-09-21T21:43:12.241 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0199999999999999e-06 2024-09-21T21:43:12.241 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.241 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.241 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:12.242 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720123+0000", 2024-09-21T21:43:12.242 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.5469999999999999e-06 2024-09-21T21:43:12.242 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.242 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.242 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:12.242 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798626+0000", 2024-09-21T21:43:12.242 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078503137000000001 2024-09-21T21:43:12.242 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.242 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.242 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:12.242 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798677+0000", 2024-09-21T21:43:12.242 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0930999999999997e-05 2024-09-21T21:43:12.242 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.242 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.243 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:12.243 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798784+0000", 2024-09-21T21:43:12.243 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010671 2024-09-21T21:43:12.243 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:12.243 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:12.243 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:12.243 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.243 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.243 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11346 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 5500928~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:12.243 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.720126+0000", 2024-09-21T21:43:12.243 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 81.486273566999998, 2024-09-21T21:43:12.243 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078863421000000003, 2024-09-21T21:43:12.243 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:12.243 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:12.243 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:12.244 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:12.244 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:12.244 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:12.244 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11346 2024-09-21T21:43:12.244 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.244 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:12.244 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.244 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:12.244 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720126+0000", 2024-09-21T21:43:12.244 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:12.244 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.244 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.244 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:12.244 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720126+0000", 2024-09-21T21:43:12.244 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:12.245 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.245 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.245 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:12.245 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.722145+0000", 2024-09-21T21:43:12.245 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0020183219999999999 2024-09-21T21:43:12.245 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.245 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.245 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:12.245 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.722150+0000", 2024-09-21T21:43:12.245 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1760000000000001e-06 2024-09-21T21:43:12.245 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.245 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.245 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:12.245 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.722161+0000", 2024-09-21T21:43:12.245 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1566000000000001e-05 2024-09-21T21:43:12.246 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.246 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.246 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:12.246 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.722169+0000", 2024-09-21T21:43:12.246 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.2840000000000001e-06 2024-09-21T21:43:12.246 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.246 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.246 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:12.246 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798815+0000", 2024-09-21T21:43:12.246 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.076646607000000005 2024-09-21T21:43:12.246 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.246 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.246 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:12.246 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798870+0000", 2024-09-21T21:43:12.247 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4486e-05 2024-09-21T21:43:12.247 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.247 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.247 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:12.247 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798990+0000", 2024-09-21T21:43:12.247 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011998 2024-09-21T21:43:12.247 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:12.247 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:12.247 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:12.247 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.247 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.247 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13629 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7852032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:12.247 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943319+0000", 2024-09-21T21:43:12.247 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 81.263080924999997, 2024-09-21T21:43:12.247 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078610975999999999, 2024-09-21T21:43:12.248 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:12.248 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:12.248 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:12.248 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:12.248 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:12.248 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:12.248 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13629 2024-09-21T21:43:12.248 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.248 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:12.248 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.248 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:12.248 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943319+0000", 2024-09-21T21:43:12.248 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:12.248 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.248 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.249 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:12.249 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943319+0000", 2024-09-21T21:43:12.249 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:12.249 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.249 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.249 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:12.249 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943319+0000", 2024-09-21T21:43:12.249 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6199999999999999e-07 2024-09-21T21:43:12.249 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.249 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.249 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:12.249 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943321+0000", 2024-09-21T21:43:12.249 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1009999999999999e-06 2024-09-21T21:43:12.249 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.250 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.250 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:12.250 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943328+0000", 2024-09-21T21:43:12.250 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.1029999999999996e-06 2024-09-21T21:43:12.250 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.250 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.250 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:12.250 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943333+0000", 2024-09-21T21:43:12.250 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7709999999999997e-06 2024-09-21T21:43:12.250 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.250 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.250 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:12.250 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.021679+0000", 2024-09-21T21:43:12.250 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078345673000000005 2024-09-21T21:43:12.251 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.251 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.251 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:12.251 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.021771+0000", 2024-09-21T21:43:12.251 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2201999999999996e-05 2024-09-21T21:43:12.251 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.251 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.251 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:12.251 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.021930+0000", 2024-09-21T21:43:12.251 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00015876400000000001 2024-09-21T21:43:12.251 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:12.251 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:12.251 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:12.251 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.251 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.252 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13630 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7880704~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:12.252 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943339+0000", 2024-09-21T21:43:12.252 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 81.263060737000004, 2024-09-21T21:43:12.252 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078939410000000002, 2024-09-21T21:43:12.252 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:12.252 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:12.252 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:12.252 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:12.252 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:12.252 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:12.252 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13630 2024-09-21T21:43:12.252 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.252 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:12.253 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.253 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:12.253 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943339+0000", 2024-09-21T21:43:12.253 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:12.253 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.253 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.253 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:12.253 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943339+0000", 2024-09-21T21:43:12.253 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:12.253 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.253 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.253 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:12.253 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943339+0000", 2024-09-21T21:43:12.253 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8000000000000001e-07 2024-09-21T21:43:12.253 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.254 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.254 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:12.254 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943341+0000", 2024-09-21T21:43:12.254 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9850000000000001e-06 2024-09-21T21:43:12.254 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.254 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.254 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:12.254 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943347+0000", 2024-09-21T21:43:12.254 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2979999999999999e-06 2024-09-21T21:43:12.254 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.254 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.254 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:12.254 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943354+0000", 2024-09-21T21:43:12.254 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.2969999999999998e-06 2024-09-21T21:43:12.255 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.255 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.255 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:12.255 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.021996+0000", 2024-09-21T21:43:12.255 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078641680000000005 2024-09-21T21:43:12.255 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.255 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.255 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:12.255 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022104+0000", 2024-09-21T21:43:12.255 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000107902 2024-09-21T21:43:12.255 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.255 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.255 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:12.256 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022278+0000", 2024-09-21T21:43:12.256 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00017486800000000001 2024-09-21T21:43:12.256 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:12.256 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:12.256 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:12.256 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.256 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.256 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13631 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7909376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:12.256 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943362+0000", 2024-09-21T21:43:12.256 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 81.263037917000005, 2024-09-21T21:43:12.256 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079195644999999995, 2024-09-21T21:43:12.256 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:12.256 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:12.257 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:12.257 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:12.257 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:12.257 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:12.257 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13631 2024-09-21T21:43:12.257 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.257 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:12.257 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.257 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:12.257 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943362+0000", 2024-09-21T21:43:12.257 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:12.257 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.257 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.257 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:12.257 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943362+0000", 2024-09-21T21:43:12.258 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:12.258 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.258 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.258 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:12.258 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943362+0000", 2024-09-21T21:43:12.258 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8799999999999998e-07 2024-09-21T21:43:12.258 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.258 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.258 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:12.258 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943364+0000", 2024-09-21T21:43:12.258 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9850000000000001e-06 2024-09-21T21:43:12.258 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.258 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.258 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:12.259 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943371+0000", 2024-09-21T21:43:12.259 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0650000000000001e-06 2024-09-21T21:43:12.259 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.259 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.259 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:12.259 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943376+0000", 2024-09-21T21:43:12.259 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8250000000000004e-06 2024-09-21T21:43:12.259 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.259 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.259 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:12.259 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022329+0000", 2024-09-21T21:43:12.259 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078952888999999998 2024-09-21T21:43:12.259 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.259 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.259 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:12.260 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022419+0000", 2024-09-21T21:43:12.260 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.9738e-05 2024-09-21T21:43:12.260 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.260 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.260 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:12.260 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022558+0000", 2024-09-21T21:43:12.260 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00013875500000000001 2024-09-21T21:43:12.260 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:12.260 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:12.260 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:12.260 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.260 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.260 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13632 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7966720~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:12.260 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943382+0000", 2024-09-21T21:43:12.261 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 81.263017683000001, 2024-09-21T21:43:12.261 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079495727000000002, 2024-09-21T21:43:12.261 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:12.261 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:12.261 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:12.261 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:12.261 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:12.261 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:12.261 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13632 2024-09-21T21:43:12.261 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.261 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:12.261 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.261 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:12.261 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943382+0000", 2024-09-21T21:43:12.262 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:12.262 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.262 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.262 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:12.262 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943382+0000", 2024-09-21T21:43:12.262 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:12.262 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.262 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.262 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:12.262 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943382+0000", 2024-09-21T21:43:12.262 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7e-07 2024-09-21T21:43:12.262 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.262 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.262 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:12.262 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943385+0000", 2024-09-21T21:43:12.263 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2230000000000001e-06 2024-09-21T21:43:12.263 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.263 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.263 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:12.263 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943391+0000", 2024-09-21T21:43:12.263 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6810000000000001e-06 2024-09-21T21:43:12.263 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.263 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.263 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:12.263 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943397+0000", 2024-09-21T21:43:12.263 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6559999999999997e-06 2024-09-21T21:43:12.263 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.263 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.263 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:12.264 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022601+0000", 2024-09-21T21:43:12.264 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079204051999999997 2024-09-21T21:43:12.264 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.264 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.264 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:12.264 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022704+0000", 2024-09-21T21:43:12.264 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010329599999999999 2024-09-21T21:43:12.264 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.264 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.264 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:12.264 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022878+0000", 2024-09-21T21:43:12.264 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000173449 2024-09-21T21:43:12.264 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:12.264 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:12.264 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:12.265 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.265 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.265 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13633 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7938048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:12.265 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943403+0000", 2024-09-21T21:43:12.265 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 81.262996752999996, 2024-09-21T21:43:12.265 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079794697999999997, 2024-09-21T21:43:12.265 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:12.265 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:12.265 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:12.265 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:12.265 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:12.265 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:12.265 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13633 2024-09-21T21:43:12.265 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.265 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:12.266 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.266 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:12.266 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943403+0000", 2024-09-21T21:43:12.266 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:12.266 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.266 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.266 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:12.266 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943403+0000", 2024-09-21T21:43:12.266 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:12.266 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.266 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.266 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:12.266 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943403+0000", 2024-09-21T21:43:12.266 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.65e-07 2024-09-21T21:43:12.267 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.267 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.267 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:12.267 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943405+0000", 2024-09-21T21:43:12.267 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0020000000000001e-06 2024-09-21T21:43:12.267 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.267 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.267 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:12.267 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943412+0000", 2024-09-21T21:43:12.267 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.8379999999999996e-06 2024-09-21T21:43:12.267 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.267 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.267 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:12.267 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943417+0000", 2024-09-21T21:43:12.267 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7650000000000001e-06 2024-09-21T21:43:12.268 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.268 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.268 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:12.268 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022922+0000", 2024-09-21T21:43:12.268 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079504512999999999 2024-09-21T21:43:12.268 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.268 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.268 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:12.268 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023039+0000", 2024-09-21T21:43:12.268 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000117044 2024-09-21T21:43:12.268 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.268 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.268 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:12.268 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023198+0000", 2024-09-21T21:43:12.268 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000159171 2024-09-21T21:43:12.269 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:12.269 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:12.269 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:12.269 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.269 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.269 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13634 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7995392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:12.269 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943423+0000", 2024-09-21T21:43:12.269 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 81.262977045, 2024-09-21T21:43:12.269 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.080142358999999996, 2024-09-21T21:43:12.269 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:12.269 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:12.269 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:12.269 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:12.269 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:12.270 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:12.270 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13634 2024-09-21T21:43:12.270 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.270 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:12.270 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.270 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:12.270 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943423+0000", 2024-09-21T21:43:12.270 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:12.270 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.270 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.270 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:12.270 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943423+0000", 2024-09-21T21:43:12.270 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:12.270 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.271 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.271 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:12.271 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943423+0000", 2024-09-21T21:43:12.271 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.58e-07 2024-09-21T21:43:12.271 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.271 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.271 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:12.271 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943425+0000", 2024-09-21T21:43:12.271 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9180000000000001e-06 2024-09-21T21:43:12.271 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.271 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.272 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:12.272 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943431+0000", 2024-09-21T21:43:12.272 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4439999999999996e-06 2024-09-21T21:43:12.272 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.272 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.272 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:12.272 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943436+0000", 2024-09-21T21:43:12.272 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.887e-06 2024-09-21T21:43:12.272 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.272 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.272 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:12.272 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023266+0000", 2024-09-21T21:43:12.272 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079829622000000003 2024-09-21T21:43:12.272 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.273 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.273 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:12.273 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023384+0000", 2024-09-21T21:43:12.273 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000117658 2024-09-21T21:43:12.273 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.273 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.273 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:12.273 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023565+0000", 2024-09-21T21:43:12.273 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00018147200000000001 2024-09-21T21:43:12.273 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:12.273 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:12.273 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:12.273 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.273 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.273 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13635 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 8024064~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:12.273 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943447+0000", 2024-09-21T21:43:12.274 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 81.262952505000001, 2024-09-21T21:43:12.274 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.080417653000000006, 2024-09-21T21:43:12.274 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:12.274 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:12.274 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:12.274 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:12.274 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:12.274 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:12.274 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13635 2024-09-21T21:43:12.274 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.274 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:12.274 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.274 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:12.274 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943447+0000", 2024-09-21T21:43:12.275 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:12.275 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.275 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.275 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:12.275 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943447+0000", 2024-09-21T21:43:12.275 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:12.275 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.275 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.275 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:12.275 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943448+0000", 2024-09-21T21:43:12.275 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1699999999999999e-07 2024-09-21T21:43:12.275 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.275 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.275 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:12.275 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943450+0000", 2024-09-21T21:43:12.276 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1639999999999999e-06 2024-09-21T21:43:12.276 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.276 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.276 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:12.276 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943456+0000", 2024-09-21T21:43:12.276 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8379999999999998e-06 2024-09-21T21:43:12.276 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.276 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.276 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:12.276 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943460+0000", 2024-09-21T21:43:12.276 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4700000000000004e-06 2024-09-21T21:43:12.276 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.276 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.276 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:12.277 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023595+0000", 2024-09-21T21:43:12.277 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.080134869999999997 2024-09-21T21:43:12.277 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.277 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.277 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:12.277 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023688+0000", 2024-09-21T21:43:12.277 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2794000000000006e-05 2024-09-21T21:43:12.277 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.277 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.277 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:12.277 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023865+0000", 2024-09-21T21:43:12.277 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0001771 2024-09-21T21:43:12.277 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:12.277 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:12.277 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:12.278 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.278 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.278 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13636 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 8052736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:12.278 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943467+0000", 2024-09-21T21:43:12.278 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 81.262932767999999, 2024-09-21T21:43:12.278 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.080694399999999999, 2024-09-21T21:43:12.278 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:12.278 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:12.278 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:12.278 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:12.278 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:12.278 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:12.278 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13636 2024-09-21T21:43:12.278 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.279 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:12.279 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.279 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:12.279 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943467+0000", 2024-09-21T21:43:12.279 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:12.279 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.279 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.279 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:12.279 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943467+0000", 2024-09-21T21:43:12.279 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:12.279 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.279 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.279 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:12.279 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944940+0000", 2024-09-21T21:43:12.279 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0014725109999999999 2024-09-21T21:43:12.280 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.280 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.280 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:12.280 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944949+0000", 2024-09-21T21:43:12.280 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2930000000000006e-06 2024-09-21T21:43:12.280 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.280 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.280 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:12.280 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944958+0000", 2024-09-21T21:43:12.280 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.6899999999999998e-06 2024-09-21T21:43:12.280 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.280 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.280 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:12.280 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944966+0000", 2024-09-21T21:43:12.280 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0690000000000002e-06 2024-09-21T21:43:12.281 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.281 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.281 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:12.281 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023900+0000", 2024-09-21T21:43:12.281 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078934371000000003 2024-09-21T21:43:12.281 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.281 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.281 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:12.281 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024009+0000", 2024-09-21T21:43:12.281 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000108857 2024-09-21T21:43:12.281 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.281 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.281 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:12.281 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024161+0000", 2024-09-21T21:43:12.281 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000152609 2024-09-21T21:43:12.282 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:12.282 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:12.282 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:12.282 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.282 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.282 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13637 3.12 3.1dc31b72 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:12.282 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.944978+0000", 2024-09-21T21:43:12.282 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 81.261421833, 2024-09-21T21:43:12.282 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079232604999999998, 2024-09-21T21:43:12.282 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:12.282 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:12.282 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:43:12.282 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:12.282 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:12.283 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:12.283 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13637 2024-09-21T21:43:12.283 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.283 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:12.283 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.283 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:12.283 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944978+0000", 2024-09-21T21:43:12.283 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:12.283 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.283 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.283 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:12.283 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944978+0000", 2024-09-21T21:43:12.283 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:12.283 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.283 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.283 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:12.284 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944978+0000", 2024-09-21T21:43:12.284 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9100000000000004e-07 2024-09-21T21:43:12.284 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.284 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.284 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:12.285 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944981+0000", 2024-09-21T21:43:12.286 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3290000000000001e-06 2024-09-21T21:43:12.286 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.286 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.286 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:12.286 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944987+0000", 2024-09-21T21:43:12.286 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.28e-06 2024-09-21T21:43:12.286 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.286 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.286 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:12.286 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944992+0000", 2024-09-21T21:43:12.286 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8520000000000003e-06 2024-09-21T21:43:12.286 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.286 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.286 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:12.286 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024196+0000", 2024-09-21T21:43:12.287 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079203673000000002 2024-09-21T21:43:12.287 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.287 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.287 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:12.287 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024211+0000", 2024-09-21T21:43:12.287 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.4980000000000001e-05 2024-09-21T21:43:12.287 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:12.287 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:12.287 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:12.287 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.287 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.287 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13638 3.12 3.1dc31b72 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:12.287 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.945000+0000", 2024-09-21T21:43:12.287 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 81.261399705000002, 2024-09-21T21:43:12.287 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079257465999999999, 2024-09-21T21:43:12.288 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:12.288 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:12.288 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:43:12.288 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:12.288 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:12.288 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:12.288 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13638 2024-09-21T21:43:12.288 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.288 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:12.288 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.288 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:12.288 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945000+0000", 2024-09-21T21:43:12.288 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:12.288 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.288 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.289 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:12.289 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945000+0000", 2024-09-21T21:43:12.289 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:12.289 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.289 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.289 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:12.289 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945001+0000", 2024-09-21T21:43:12.289 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.32e-07 2024-09-21T21:43:12.289 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.289 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.289 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:12.289 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945003+0000", 2024-09-21T21:43:12.289 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2309999999999999e-06 2024-09-21T21:43:12.289 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.289 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.290 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:12.290 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945009+0000", 2024-09-21T21:43:12.290 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7450000000000001e-06 2024-09-21T21:43:12.290 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.290 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.290 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:12.290 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945013+0000", 2024-09-21T21:43:12.290 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.673e-06 2024-09-21T21:43:12.290 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.290 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.290 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:12.290 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024245+0000", 2024-09-21T21:43:12.290 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079232160999999995 2024-09-21T21:43:12.290 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.291 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.291 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:12.291 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024258+0000", 2024-09-21T21:43:12.291 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.2224e-05 2024-09-21T21:43:12.291 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:12.291 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:12.291 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:12.291 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.291 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.291 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13639 3.12 3.1dc31b72 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:12.291 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.945034+0000", 2024-09-21T21:43:12.291 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 81.261365905999995, 2024-09-21T21:43:12.291 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079260627, 2024-09-21T21:43:12.291 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:12.291 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:12.291 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:43:12.292 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:12.292 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:12.292 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:12.292 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13639 2024-09-21T21:43:12.292 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.292 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:12.292 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.292 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:12.292 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945034+0000", 2024-09-21T21:43:12.292 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:12.292 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.292 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.292 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:12.292 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945034+0000", 2024-09-21T21:43:12.293 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:12.293 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.293 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.293 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:12.293 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945034+0000", 2024-09-21T21:43:12.293 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3799999999999998e-07 2024-09-21T21:43:12.293 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.293 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.293 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:12.293 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945036+0000", 2024-09-21T21:43:12.293 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.052e-06 2024-09-21T21:43:12.293 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.293 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.293 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:12.293 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945040+0000", 2024-09-21T21:43:12.294 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1169999999999997e-06 2024-09-21T21:43:12.294 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.294 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.294 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:12.294 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945043+0000", 2024-09-21T21:43:12.294 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.8439999999999998e-06 2024-09-21T21:43:12.294 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.294 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.294 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:12.294 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024286+0000", 2024-09-21T21:43:12.294 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079242483000000002 2024-09-21T21:43:12.294 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.294 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.295 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:12.295 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024295+0000", 2024-09-21T21:43:12.295 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.7930000000000003e-06 2024-09-21T21:43:12.295 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:12.295 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:12.295 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:12.295 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.295 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.295 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13640 3.12 3.1dc31b72 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:12.295 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.945047+0000", 2024-09-21T21:43:12.295 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 81.261352384000006, 2024-09-21T21:43:12.295 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079412321999999994, 2024-09-21T21:43:12.295 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:12.295 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:12.296 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:43:12.296 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:12.296 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:12.296 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:12.296 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13640 2024-09-21T21:43:12.296 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.296 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:12.296 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.296 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:12.296 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945047+0000", 2024-09-21T21:43:12.296 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:12.296 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.296 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.296 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:12.297 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945047+0000", 2024-09-21T21:43:12.297 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:12.297 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.297 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.297 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:12.297 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945048+0000", 2024-09-21T21:43:12.297 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0499999999999999e-07 2024-09-21T21:43:12.297 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.297 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.297 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:12.297 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945049+0000", 2024-09-21T21:43:12.297 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.3939999999999999e-06 2024-09-21T21:43:12.297 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.297 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.298 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:12.298 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945053+0000", 2024-09-21T21:43:12.298 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1760000000000003e-06 2024-09-21T21:43:12.298 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.298 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.298 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:12.298 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945056+0000", 2024-09-21T21:43:12.298 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.6129999999999998e-06 2024-09-21T21:43:12.298 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.298 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.298 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:12.298 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024360+0000", 2024-09-21T21:43:12.298 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079303645000000006 2024-09-21T21:43:12.298 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.298 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.299 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:12.299 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024460+0000", 2024-09-21T21:43:12.299 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000100189 2024-09-21T21:43:12.299 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:12.299 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:12.299 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:12.299 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:12.299 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:12.299 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:43:12.301 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T21:43:12.302 INFO:teuthology.orchestra.run.smithi081.stderr:noscrub is set 2024-09-21T21:43:12.302 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:43:12.299+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:43:12.401 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:43:12.399+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:43:12.449 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:43:12.449 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:43:12.450 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 0 2024-09-21T21:43:12.450 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:43:12.458 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T21:43:12.543 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:43:12.541+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:43:12.600 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:43:12.601 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:43:12.601 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:43:12.601 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:43:12.601 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:43:12.603 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:43:12.602+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:43:12.609 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T21:43:12.704 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:43:12.703+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:43:12.755 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:43:12.755 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 20, 2024-09-21T21:43:12.755 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 600, 2024-09-21T21:43:12.755 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:43:12.755 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.755 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4824.0:1564 3.2 3:4647520c:::rbd_data.1121b97f2925.0000000000000006:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:43:12.755 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:43:12.755 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 131.116180649, 2024-09-21T21:43:12.755 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056097347999999998, 2024-09-21T21:43:12.755 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:12.755 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:12.755 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:12.756 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:12.756 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4824", 2024-09-21T21:43:12.756 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:43:12.756 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1564 2024-09-21T21:43:12.756 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.756 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:12.756 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.756 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:12.756 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:43:12.756 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:12.756 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.756 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.756 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:12.756 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:43:12.756 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:12.757 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.757 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.757 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:12.757 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:43:12.757 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6499999999999999e-07 2024-09-21T21:43:12.757 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.757 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.757 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:12.757 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636067+0000", 2024-09-21T21:43:12.757 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3879999999999998e-06 2024-09-21T21:43:12.757 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.757 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.757 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:12.757 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636075+0000", 2024-09-21T21:43:12.757 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5480000000000004e-06 2024-09-21T21:43:12.758 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.758 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.758 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:12.758 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636080+0000", 2024-09-21T21:43:12.758 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.339e-06 2024-09-21T21:43:12.758 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.758 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.758 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:12.758 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636133+0000", 2024-09-21T21:43:12.758 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2930000000000003e-05 2024-09-21T21:43:12.758 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.758 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.758 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:12.758 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636265+0000", 2024-09-21T21:43:12.759 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000132058 2024-09-21T21:43:12.759 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.759 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.759 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:12.759 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.692162+0000", 2024-09-21T21:43:12.759 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055896620000000001 2024-09-21T21:43:12.759 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:12.759 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:12.759 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:12.759 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.759 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.759 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3260 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 1437696~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:43:12.759 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.196297+0000", 2024-09-21T21:43:12.759 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 85.555948439000005, 2024-09-21T21:43:12.759 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056071022999999998, 2024-09-21T21:43:12.760 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:12.760 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:12.760 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:12.760 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:12.760 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:12.760 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:12.760 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3260 2024-09-21T21:43:12.760 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.760 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:12.760 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.760 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:12.760 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196297+0000", 2024-09-21T21:43:12.760 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:12.760 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.760 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.761 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:12.761 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196297+0000", 2024-09-21T21:43:12.761 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:12.761 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.761 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.761 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:12.761 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196918+0000", 2024-09-21T21:43:12.761 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00062148699999999999 2024-09-21T21:43:12.761 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.761 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.761 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:12.761 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196930+0000", 2024-09-21T21:43:12.761 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1657e-05 2024-09-21T21:43:12.761 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.761 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.761 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:12.762 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196944+0000", 2024-09-21T21:43:12.762 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.4549000000000001e-05 2024-09-21T21:43:12.762 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.762 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.762 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:12.762 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196951+0000", 2024-09-21T21:43:12.762 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6749999999999996e-06 2024-09-21T21:43:12.762 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.762 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.762 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:12.762 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252208+0000", 2024-09-21T21:43:12.762 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055256656000000001 2024-09-21T21:43:12.762 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.762 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.763 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:12.763 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252278+0000", 2024-09-21T21:43:12.763 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9882000000000006e-05 2024-09-21T21:43:12.763 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.763 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.763 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:12.763 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252368+0000", 2024-09-21T21:43:12.763 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.0117000000000005e-05 2024-09-21T21:43:12.763 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:12.763 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:12.763 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:12.763 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.763 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.763 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3437 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 925696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:43:12.763 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.205888+0000", 2024-09-21T21:43:12.764 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 85.546357344, 2024-09-21T21:43:12.764 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055953808000000001, 2024-09-21T21:43:12.764 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:12.764 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:12.764 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:12.764 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:12.764 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:12.764 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:12.764 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3437 2024-09-21T21:43:12.764 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.764 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:12.764 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.764 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:12.764 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205888+0000", 2024-09-21T21:43:12.764 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:12.765 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.765 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.765 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:12.765 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205888+0000", 2024-09-21T21:43:12.765 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:12.765 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.765 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.765 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:12.765 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205888+0000", 2024-09-21T21:43:12.765 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.75e-07 2024-09-21T21:43:12.765 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.765 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.765 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:12.765 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205891+0000", 2024-09-21T21:43:12.765 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3750000000000001e-06 2024-09-21T21:43:12.766 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.766 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.766 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:12.766 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205897+0000", 2024-09-21T21:43:12.766 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.285e-06 2024-09-21T21:43:12.766 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.766 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.766 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:12.766 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205902+0000", 2024-09-21T21:43:12.766 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6290000000000001e-06 2024-09-21T21:43:12.766 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.766 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.766 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:12.766 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261686+0000", 2024-09-21T21:43:12.766 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055784586999999997 2024-09-21T21:43:12.767 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.767 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.767 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:12.767 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261764+0000", 2024-09-21T21:43:12.767 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7591999999999998e-05 2024-09-21T21:43:12.767 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.767 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.767 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:12.767 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261842+0000", 2024-09-21T21:43:12.767 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7865000000000001e-05 2024-09-21T21:43:12.767 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:12.767 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:12.767 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:12.767 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.767 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.768 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3438 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 954368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:43:12.768 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.205908+0000", 2024-09-21T21:43:12.768 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 85.546337479000002, 2024-09-21T21:43:12.768 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056083169000000002, 2024-09-21T21:43:12.768 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:12.768 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:12.768 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:12.768 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:12.768 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:12.768 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:12.768 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3438 2024-09-21T21:43:12.768 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.768 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:12.768 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.769 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:12.769 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205908+0000", 2024-09-21T21:43:12.769 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:12.769 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.769 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.769 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:12.769 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205908+0000", 2024-09-21T21:43:12.769 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:12.769 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.769 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.769 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:12.769 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206431+0000", 2024-09-21T21:43:12.769 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00052282100000000003 2024-09-21T21:43:12.769 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.769 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.770 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:12.770 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206442+0000", 2024-09-21T21:43:12.770 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1695000000000001e-05 2024-09-21T21:43:12.770 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.770 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.770 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:12.770 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206452+0000", 2024-09-21T21:43:12.770 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.4560000000000006e-06 2024-09-21T21:43:12.770 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.770 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.770 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:12.770 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206458+0000", 2024-09-21T21:43:12.770 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4670000000000001e-06 2024-09-21T21:43:12.770 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.771 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.771 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:12.771 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261871+0000", 2024-09-21T21:43:12.771 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055412712000000003 2024-09-21T21:43:12.771 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.771 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.771 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:12.771 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261928+0000", 2024-09-21T21:43:12.771 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6631e-05 2024-09-21T21:43:12.771 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.771 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.771 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:12.771 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261991+0000", 2024-09-21T21:43:12.771 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3386999999999995e-05 2024-09-21T21:43:12.772 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:12.772 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:12.772 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:12.772 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.772 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.772 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3444 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 2359296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:43:12.772 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.206533+0000", 2024-09-21T21:43:12.772 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 85.545711959000002, 2024-09-21T21:43:12.772 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055974239000000002, 2024-09-21T21:43:12.772 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:12.772 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:12.772 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:12.772 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:12.772 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:12.772 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:12.773 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3444 2024-09-21T21:43:12.773 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.773 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:12.773 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.773 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:12.773 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206533+0000", 2024-09-21T21:43:12.773 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:12.773 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.773 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.773 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:12.773 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206533+0000", 2024-09-21T21:43:12.773 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:12.773 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.773 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.773 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:12.774 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206710+0000", 2024-09-21T21:43:12.774 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000176293 2024-09-21T21:43:12.774 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.774 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.774 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:12.774 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206714+0000", 2024-09-21T21:43:12.774 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1989999999999999e-06 2024-09-21T21:43:12.774 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.774 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.774 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:12.774 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206723+0000", 2024-09-21T21:43:12.774 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.9199999999999993e-06 2024-09-21T21:43:12.774 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.774 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.775 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:12.775 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206729+0000", 2024-09-21T21:43:12.775 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0739999999999998e-06 2024-09-21T21:43:12.775 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.775 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.775 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:12.775 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262339+0000", 2024-09-21T21:43:12.775 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055610166000000003 2024-09-21T21:43:12.775 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.775 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.775 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:12.776 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262419+0000", 2024-09-21T21:43:12.776 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0432999999999994e-05 2024-09-21T21:43:12.776 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.776 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.776 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:12.776 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262507+0000", 2024-09-21T21:43:12.776 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.8153999999999997e-05 2024-09-21T21:43:12.776 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:12.776 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:12.776 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:12.776 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.776 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.776 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10362 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:43:12.776 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840386+0000", 2024-09-21T21:43:12.776 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 58.911859114999999, 2024-09-21T21:43:12.777 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056546515999999998, 2024-09-21T21:43:12.777 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:12.777 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:12.777 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:12.777 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:12.777 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:43:12.777 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:43:12.777 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10362 2024-09-21T21:43:12.777 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.777 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:12.777 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.777 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:12.777 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840386+0000", 2024-09-21T21:43:12.777 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:12.778 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.778 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.778 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:12.778 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840386+0000", 2024-09-21T21:43:12.778 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:12.778 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.778 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.778 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:12.778 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840386+0000", 2024-09-21T21:43:12.778 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.01e-07 2024-09-21T21:43:12.778 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.778 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.778 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:12.778 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840389+0000", 2024-09-21T21:43:12.779 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.9739999999999998e-06 2024-09-21T21:43:12.779 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.779 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.779 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:12.779 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840396+0000", 2024-09-21T21:43:12.779 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2280000000000003e-06 2024-09-21T21:43:12.779 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.779 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.779 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:12.779 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840400+0000", 2024-09-21T21:43:12.779 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4429999999999996e-06 2024-09-21T21:43:12.779 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.779 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.779 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:12.779 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.890896+0000", 2024-09-21T21:43:12.780 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.050495717000000002 2024-09-21T21:43:12.780 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.780 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.780 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:12.780 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.891012+0000", 2024-09-21T21:43:12.780 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011602 2024-09-21T21:43:12.780 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.780 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.780 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:43:12.780 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.891274+0000", 2024-09-21T21:43:12.780 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00026240200000000001 2024-09-21T21:43:12.780 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.780 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.780 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:43:12.780 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.894810+0000", 2024-09-21T21:43:12.780 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0035353630000000001 2024-09-21T21:43:12.781 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.781 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.781 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:12.781 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.896818+0000", 2024-09-21T21:43:12.781 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.002008681 2024-09-21T21:43:12.781 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.781 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.781 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:43:12.781 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.896850+0000", 2024-09-21T21:43:12.781 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.1826999999999999e-05 2024-09-21T21:43:12.781 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.781 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.781 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:12.781 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.896933+0000", 2024-09-21T21:43:12.781 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.2460000000000007e-05 2024-09-21T21:43:12.782 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:12.782 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:12.782 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:12.782 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.782 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.782 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10363 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:43:12.782 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840606+0000", 2024-09-21T21:43:12.782 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 58.911639604999998, 2024-09-21T21:43:12.782 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056546011, 2024-09-21T21:43:12.782 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:12.782 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:12.782 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:12.782 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:12.782 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:43:12.782 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:43:12.783 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10363 2024-09-21T21:43:12.783 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.783 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:12.783 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.783 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:12.783 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840606+0000", 2024-09-21T21:43:12.783 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:12.783 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.783 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.783 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:12.783 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840606+0000", 2024-09-21T21:43:12.783 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:12.783 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.783 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.784 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:12.784 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840606+0000", 2024-09-21T21:43:12.784 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.4700000000000001e-07 2024-09-21T21:43:12.784 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.784 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.784 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:12.784 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840611+0000", 2024-09-21T21:43:12.784 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4649999999999996e-06 2024-09-21T21:43:12.784 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.784 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.784 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:12.784 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840620+0000", 2024-09-21T21:43:12.784 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.0350000000000007e-06 2024-09-21T21:43:12.784 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.785 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.785 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:12.785 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840626+0000", 2024-09-21T21:43:12.785 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7180000000000001e-06 2024-09-21T21:43:12.785 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.785 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.785 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:12.785 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.891665+0000", 2024-09-21T21:43:12.785 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.051039924 2024-09-21T21:43:12.785 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.785 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.785 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:12.785 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.891770+0000", 2024-09-21T21:43:12.785 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010445700000000001 2024-09-21T21:43:12.785 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.786 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.786 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:43:12.786 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.892147+0000", 2024-09-21T21:43:12.786 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00037731800000000002 2024-09-21T21:43:12.786 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.786 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.786 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:43:12.786 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.896776+0000", 2024-09-21T21:43:12.786 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0046289759999999999 2024-09-21T21:43:12.786 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.786 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.786 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:12.786 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897003+0000", 2024-09-21T21:43:12.786 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00022641299999999999 2024-09-21T21:43:12.786 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.787 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.787 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:43:12.787 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897064+0000", 2024-09-21T21:43:12.787 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1571999999999999e-05 2024-09-21T21:43:12.787 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.787 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.787 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:12.787 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897152+0000", 2024-09-21T21:43:12.787 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.7386000000000007e-05 2024-09-21T21:43:12.787 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:12.787 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:12.787 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:12.787 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.787 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.788 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10364 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:43:12.788 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840641+0000", 2024-09-21T21:43:12.788 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 58.911603784999997, 2024-09-21T21:43:12.788 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056671527999999999, 2024-09-21T21:43:12.788 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:12.788 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:12.788 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:12.788 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:12.788 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:43:12.788 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:43:12.788 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10364 2024-09-21T21:43:12.788 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.788 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:12.788 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.789 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:12.789 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840641+0000", 2024-09-21T21:43:12.789 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:12.789 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.789 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.789 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:12.789 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840641+0000", 2024-09-21T21:43:12.789 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:12.789 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.789 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.789 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:12.789 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840642+0000", 2024-09-21T21:43:12.789 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7899999999999999e-07 2024-09-21T21:43:12.789 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.789 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.790 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:12.790 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840645+0000", 2024-09-21T21:43:12.790 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0010000000000002e-06 2024-09-21T21:43:12.790 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.790 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.790 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:12.790 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840654+0000", 2024-09-21T21:43:12.790 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.8570000000000001e-06 2024-09-21T21:43:12.790 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.790 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.790 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:12.790 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840659+0000", 2024-09-21T21:43:12.790 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.558e-06 2024-09-21T21:43:12.790 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.790 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.791 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:12.791 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.892656+0000", 2024-09-21T21:43:12.791 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.051996586999999997 2024-09-21T21:43:12.791 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.791 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.791 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:12.791 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.892721+0000", 2024-09-21T21:43:12.791 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5010000000000003e-05 2024-09-21T21:43:12.791 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.791 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.791 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:43:12.791 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.893112+0000", 2024-09-21T21:43:12.791 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00039133899999999999 2024-09-21T21:43:12.791 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.792 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.792 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:43:12.792 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.896784+0000", 2024-09-21T21:43:12.792 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0036713679999999999 2024-09-21T21:43:12.792 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.792 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.792 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:12.792 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897204+0000", 2024-09-21T21:43:12.792 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00041991400000000002 2024-09-21T21:43:12.792 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.792 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.792 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:43:12.792 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897234+0000", 2024-09-21T21:43:12.792 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0307999999999998e-05 2024-09-21T21:43:12.792 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.793 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.793 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:12.793 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897313+0000", 2024-09-21T21:43:12.793 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.9107e-05 2024-09-21T21:43:12.793 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:12.793 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:12.793 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:12.793 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.793 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.793 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10365 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:43:12.793 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840668+0000", 2024-09-21T21:43:12.793 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 58.911576920000002, 2024-09-21T21:43:12.793 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.058761225, 2024-09-21T21:43:12.793 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:12.793 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:12.794 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:12.794 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:12.794 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:43:12.794 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:43:12.794 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10365 2024-09-21T21:43:12.794 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.794 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:12.794 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.794 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:12.794 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840668+0000", 2024-09-21T21:43:12.794 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:12.794 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.794 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.794 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:12.795 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840668+0000", 2024-09-21T21:43:12.795 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:12.795 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.795 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.795 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:12.795 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840669+0000", 2024-09-21T21:43:12.795 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6300000000000001e-07 2024-09-21T21:43:12.795 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.795 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.795 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:12.795 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840672+0000", 2024-09-21T21:43:12.795 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.072e-06 2024-09-21T21:43:12.795 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.795 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.796 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:12.796 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840678+0000", 2024-09-21T21:43:12.796 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.7039999999999997e-06 2024-09-21T21:43:12.796 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.796 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.796 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:12.796 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840683+0000", 2024-09-21T21:43:12.796 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6759999999999998e-06 2024-09-21T21:43:12.796 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.796 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.796 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:12.796 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.893929+0000", 2024-09-21T21:43:12.796 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.053246136999999999 2024-09-21T21:43:12.796 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.796 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.797 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:12.797 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.894082+0000", 2024-09-21T21:43:12.797 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00015301799999999999 2024-09-21T21:43:12.797 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.797 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.797 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:43:12.797 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.894357+0000", 2024-09-21T21:43:12.797 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00027507999999999998 2024-09-21T21:43:12.797 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.797 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.797 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:43:12.797 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897161+0000", 2024-09-21T21:43:12.797 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0028036810000000001 2024-09-21T21:43:12.797 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.797 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.798 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:12.798 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.899243+0000", 2024-09-21T21:43:12.798 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0020816379999999998 2024-09-21T21:43:12.798 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.798 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.798 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:43:12.798 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.899306+0000", 2024-09-21T21:43:12.798 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2964000000000002e-05 2024-09-21T21:43:12.798 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.798 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.798 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:12.798 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.899430+0000", 2024-09-21T21:43:12.798 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00012389200000000001 2024-09-21T21:43:12.798 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:12.799 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:12.799 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:12.799 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.799 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.799 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10366 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:43:12.799 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840691+0000", 2024-09-21T21:43:12.799 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 58.911554219999999, 2024-09-21T21:43:12.799 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.061540984999999999, 2024-09-21T21:43:12.799 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:12.799 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:12.799 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:12.799 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:12.799 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:43:12.799 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:43:12.799 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10366 2024-09-21T21:43:12.800 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.800 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:12.800 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.800 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:12.800 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840691+0000", 2024-09-21T21:43:12.800 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:12.800 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.800 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.800 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:12.800 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840691+0000", 2024-09-21T21:43:12.800 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:12.800 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.800 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.800 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:12.801 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840691+0000", 2024-09-21T21:43:12.801 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3099999999999999e-07 2024-09-21T21:43:12.801 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.801 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.801 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:12.801 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840694+0000", 2024-09-21T21:43:12.801 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.5780000000000001e-06 2024-09-21T21:43:12.801 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.801 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.801 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:12.801 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840700+0000", 2024-09-21T21:43:12.801 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9429999999999997e-06 2024-09-21T21:43:12.801 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.801 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.802 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:12.802 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840704+0000", 2024-09-21T21:43:12.802 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6380000000000003e-06 2024-09-21T21:43:12.802 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.802 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.802 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:12.802 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.894846+0000", 2024-09-21T21:43:12.802 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.054141392000000003 2024-09-21T21:43:12.802 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.802 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.802 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:12.802 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.894932+0000", 2024-09-21T21:43:12.802 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.5834000000000005e-05 2024-09-21T21:43:12.802 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.803 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.803 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:43:12.803 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.895302+0000", 2024-09-21T21:43:12.803 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000370737 2024-09-21T21:43:12.803 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.803 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.803 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:43:12.803 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897166+0000", 2024-09-21T21:43:12.803 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0018632620000000001 2024-09-21T21:43:12.803 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.803 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.803 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:12.803 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902053+0000", 2024-09-21T21:43:12.803 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0048868289999999997 2024-09-21T21:43:12.804 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.804 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.804 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:43:12.804 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902111+0000", 2024-09-21T21:43:12.804 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7983999999999999e-05 2024-09-21T21:43:12.804 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.804 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.804 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:12.804 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902232+0000", 2024-09-21T21:43:12.804 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000121457 2024-09-21T21:43:12.804 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:12.804 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:12.804 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:12.804 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.804 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.805 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10367 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:43:12.805 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840842+0000", 2024-09-21T21:43:12.805 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 58.911403411999999, 2024-09-21T21:43:12.805 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.061549088000000002, 2024-09-21T21:43:12.805 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:12.805 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:12.805 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:12.805 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:12.805 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:43:12.805 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:43:12.805 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10367 2024-09-21T21:43:12.805 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.805 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:12.805 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.805 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:12.806 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840842+0000", 2024-09-21T21:43:12.806 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:12.806 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.806 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.806 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:12.806 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840842+0000", 2024-09-21T21:43:12.806 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:12.806 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.806 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.806 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:12.806 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840842+0000", 2024-09-21T21:43:12.806 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3099999999999997e-07 2024-09-21T21:43:12.806 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.806 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.807 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:12.807 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840846+0000", 2024-09-21T21:43:12.807 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.2890000000000002e-06 2024-09-21T21:43:12.807 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.807 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.807 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:12.807 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840855+0000", 2024-09-21T21:43:12.807 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.9619999999999999e-06 2024-09-21T21:43:12.807 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.807 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.807 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:12.807 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840860+0000", 2024-09-21T21:43:12.807 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3759999999999999e-06 2024-09-21T21:43:12.807 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.807 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.808 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:12.808 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.895799+0000", 2024-09-21T21:43:12.808 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.054938569999999999 2024-09-21T21:43:12.808 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.808 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.808 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:12.808 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.895906+0000", 2024-09-21T21:43:12.808 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000107593 2024-09-21T21:43:12.808 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.808 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.808 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:43:12.808 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.896356+0000", 2024-09-21T21:43:12.808 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00044941700000000002 2024-09-21T21:43:12.808 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.808 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.809 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:43:12.809 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.898131+0000", 2024-09-21T21:43:12.809 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.001775064 2024-09-21T21:43:12.809 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.809 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.809 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:12.809 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902283+0000", 2024-09-21T21:43:12.809 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0041524839999999997 2024-09-21T21:43:12.809 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.809 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.809 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:43:12.809 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902316+0000", 2024-09-21T21:43:12.809 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3002e-05 2024-09-21T21:43:12.809 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.809 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.810 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:12.810 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902391+0000", 2024-09-21T21:43:12.810 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.47e-05 2024-09-21T21:43:12.810 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:12.810 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:12.810 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:12.810 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.810 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.810 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10368 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:43:12.810 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840873+0000", 2024-09-21T21:43:12.810 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 58.911372458000002, 2024-09-21T21:43:12.810 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.063395291000000006, 2024-09-21T21:43:12.810 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:12.810 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:12.811 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:12.811 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:12.811 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:43:12.811 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:43:12.811 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10368 2024-09-21T21:43:12.811 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.811 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:12.811 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.811 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:12.811 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840873+0000", 2024-09-21T21:43:12.811 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:12.811 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.811 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.811 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:12.812 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840873+0000", 2024-09-21T21:43:12.812 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:12.812 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.812 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.812 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:12.812 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840873+0000", 2024-09-21T21:43:12.812 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6699999999999999e-07 2024-09-21T21:43:12.812 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.812 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.812 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:12.812 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840876+0000", 2024-09-21T21:43:12.812 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.6410000000000002e-06 2024-09-21T21:43:12.812 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.812 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.813 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:12.813 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840883+0000", 2024-09-21T21:43:12.813 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0389999999999999e-06 2024-09-21T21:43:12.813 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.813 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.813 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:12.813 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840888+0000", 2024-09-21T21:43:12.813 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8679999999999998e-06 2024-09-21T21:43:12.813 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.813 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.813 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:12.813 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897345+0000", 2024-09-21T21:43:12.813 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056457734000000002 2024-09-21T21:43:12.813 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.813 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.813 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:12.814 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897407+0000", 2024-09-21T21:43:12.814 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1822000000000005e-05 2024-09-21T21:43:12.814 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.814 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.814 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:43:12.814 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897664+0000", 2024-09-21T21:43:12.814 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00025638499999999998 2024-09-21T21:43:12.814 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.814 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.814 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:43:12.814 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.901136+0000", 2024-09-21T21:43:12.814 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0034728070000000001 2024-09-21T21:43:12.814 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.814 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.815 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:12.815 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904090+0000", 2024-09-21T21:43:12.815 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0029535799999999999 2024-09-21T21:43:12.815 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.815 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.815 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:43:12.815 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904144+0000", 2024-09-21T21:43:12.815 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4369000000000001e-05 2024-09-21T21:43:12.815 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.815 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.815 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:12.815 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904268+0000", 2024-09-21T21:43:12.815 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00012367899999999999 2024-09-21T21:43:12.815 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:12.816 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:12.816 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:12.816 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.816 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.816 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10369 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:43:12.816 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840902+0000", 2024-09-21T21:43:12.816 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 58.911342804999997, 2024-09-21T21:43:12.816 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.063538938000000003, 2024-09-21T21:43:12.816 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:12.816 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:12.816 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:12.816 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:12.816 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:43:12.816 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:43:12.817 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10369 2024-09-21T21:43:12.817 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.817 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:12.817 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.817 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:12.817 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840902+0000", 2024-09-21T21:43:12.817 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:12.817 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.817 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.817 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:12.817 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840902+0000", 2024-09-21T21:43:12.817 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:12.817 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.817 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.818 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:12.818 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840903+0000", 2024-09-21T21:43:12.818 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2100000000000002e-07 2024-09-21T21:43:12.818 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.818 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.818 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:12.818 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840913+0000", 2024-09-21T21:43:12.818 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.048e-05 2024-09-21T21:43:12.818 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.818 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.818 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:12.818 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840920+0000", 2024-09-21T21:43:12.818 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0330000000000003e-06 2024-09-21T21:43:12.818 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.819 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.819 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:12.819 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840925+0000", 2024-09-21T21:43:12.819 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0170000000000004e-06 2024-09-21T21:43:12.819 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.819 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.819 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:12.819 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.898179+0000", 2024-09-21T21:43:12.819 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.057253671999999999 2024-09-21T21:43:12.819 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.819 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.819 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:12.819 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.898284+0000", 2024-09-21T21:43:12.819 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000105202 2024-09-21T21:43:12.819 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.820 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.820 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:43:12.820 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.898729+0000", 2024-09-21T21:43:12.820 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00044459300000000002 2024-09-21T21:43:12.820 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.820 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.820 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:43:12.820 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902246+0000", 2024-09-21T21:43:12.820 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0035171629999999998 2024-09-21T21:43:12.820 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.820 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.820 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:12.820 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904310+0000", 2024-09-21T21:43:12.820 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.002063765 2024-09-21T21:43:12.820 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.821 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.821 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:43:12.821 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904342+0000", 2024-09-21T21:43:12.821 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.2091999999999997e-05 2024-09-21T21:43:12.821 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.821 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.821 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:12.821 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904441+0000", 2024-09-21T21:43:12.821 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.9500000000000006e-05 2024-09-21T21:43:12.821 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:12.821 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:12.821 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:12.821 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.821 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.822 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10370 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:43:12.822 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840944+0000", 2024-09-21T21:43:12.822 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 58.911301092000002, 2024-09-21T21:43:12.822 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.064436853000000002, 2024-09-21T21:43:12.822 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:12.822 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:12.822 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:12.822 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:12.822 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:43:12.822 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:43:12.822 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10370 2024-09-21T21:43:12.822 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.822 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:12.822 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.823 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:12.823 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840944+0000", 2024-09-21T21:43:12.823 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:12.823 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.823 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.823 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:12.823 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840944+0000", 2024-09-21T21:43:12.823 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:12.823 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.823 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.823 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:12.823 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840945+0000", 2024-09-21T21:43:12.823 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2300000000000002e-07 2024-09-21T21:43:12.823 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.824 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.824 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:12.824 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840947+0000", 2024-09-21T21:43:12.824 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.8200000000000001e-06 2024-09-21T21:43:12.824 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.824 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.824 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:12.824 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840955+0000", 2024-09-21T21:43:12.824 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.8660000000000006e-06 2024-09-21T21:43:12.824 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.824 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.824 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:12.824 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840960+0000", 2024-09-21T21:43:12.824 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8670000000000002e-06 2024-09-21T21:43:12.824 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.824 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.825 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:12.825 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.899460+0000", 2024-09-21T21:43:12.825 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.058499922000000003 2024-09-21T21:43:12.825 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.825 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.825 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:12.825 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.899530+0000", 2024-09-21T21:43:12.825 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9558e-05 2024-09-21T21:43:12.825 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.825 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.825 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:43:12.825 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.899775+0000", 2024-09-21T21:43:12.825 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000245231 2024-09-21T21:43:12.825 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.826 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.826 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:43:12.826 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902254+0000", 2024-09-21T21:43:12.826 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0024790519999999998 2024-09-21T21:43:12.826 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.826 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.826 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:12.826 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905183+0000", 2024-09-21T21:43:12.826 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.002929156 2024-09-21T21:43:12.826 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.826 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.826 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:43:12.826 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905243+0000", 2024-09-21T21:43:12.826 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0055e-05 2024-09-21T21:43:12.826 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.827 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.827 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:12.827 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905381+0000", 2024-09-21T21:43:12.827 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00013790300000000001 2024-09-21T21:43:12.827 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:12.827 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:12.827 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:12.827 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.827 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.827 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10371 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:43:12.827 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840980+0000", 2024-09-21T21:43:12.827 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 58.911265426, 2024-09-21T21:43:12.827 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.064554376999999996, 2024-09-21T21:43:12.827 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:12.828 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:12.828 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:12.828 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:12.828 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:43:12.828 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:43:12.828 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10371 2024-09-21T21:43:12.828 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.828 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:12.828 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.828 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:12.828 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840980+0000", 2024-09-21T21:43:12.828 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:12.828 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.828 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.829 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:12.829 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840980+0000", 2024-09-21T21:43:12.829 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:12.829 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.829 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.829 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:12.829 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840980+0000", 2024-09-21T21:43:12.829 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8999999999999997e-07 2024-09-21T21:43:12.829 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.829 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.829 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:12.829 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840983+0000", 2024-09-21T21:43:12.829 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.123e-06 2024-09-21T21:43:12.829 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.830 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.830 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:12.830 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840990+0000", 2024-09-21T21:43:12.830 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9129999999999997e-06 2024-09-21T21:43:12.830 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.830 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.830 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:12.830 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840995+0000", 2024-09-21T21:43:12.830 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0989999999999998e-06 2024-09-21T21:43:12.830 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.830 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.830 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:12.830 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.900168+0000", 2024-09-21T21:43:12.830 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.059172739000000002 2024-09-21T21:43:12.831 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.831 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.831 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:12.831 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.900274+0000", 2024-09-21T21:43:12.831 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010546499999999999 2024-09-21T21:43:12.831 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.831 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.831 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:43:12.831 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.900703+0000", 2024-09-21T21:43:12.831 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00042978200000000003 2024-09-21T21:43:12.831 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.831 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.831 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:43:12.831 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.903443+0000", 2024-09-21T21:43:12.831 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0027400749999999998 2024-09-21T21:43:12.832 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.832 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.832 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:12.832 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905430+0000", 2024-09-21T21:43:12.832 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0019868109999999998 2024-09-21T21:43:12.832 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.832 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.832 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:43:12.832 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905464+0000", 2024-09-21T21:43:12.832 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3599000000000001e-05 2024-09-21T21:43:12.832 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.832 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.832 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:12.832 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905534+0000", 2024-09-21T21:43:12.832 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0281e-05 2024-09-21T21:43:12.832 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:12.833 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:12.833 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:12.833 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.833 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.833 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10372 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:43:12.833 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.841004+0000", 2024-09-21T21:43:12.833 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 58.911241537000002, 2024-09-21T21:43:12.833 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.064757812999999997, 2024-09-21T21:43:12.833 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:12.833 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:12.833 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:12.833 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:12.833 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:43:12.833 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:43:12.833 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10372 2024-09-21T21:43:12.833 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.833 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:12.834 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.834 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:12.834 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841004+0000", 2024-09-21T21:43:12.834 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:12.834 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.834 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.834 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:12.834 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841004+0000", 2024-09-21T21:43:12.834 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:12.834 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.834 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.834 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:12.834 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841004+0000", 2024-09-21T21:43:12.834 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8500000000000002e-07 2024-09-21T21:43:12.834 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.835 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.835 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:12.835 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841007+0000", 2024-09-21T21:43:12.835 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.6390000000000001e-06 2024-09-21T21:43:12.835 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.835 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.835 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:12.835 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841013+0000", 2024-09-21T21:43:12.835 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0070000000000003e-06 2024-09-21T21:43:12.835 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.835 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.835 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:12.835 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841018+0000", 2024-09-21T21:43:12.835 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9119999999999997e-06 2024-09-21T21:43:12.836 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.836 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.836 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:12.836 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.901182+0000", 2024-09-21T21:43:12.836 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.060164444999999997 2024-09-21T21:43:12.836 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.836 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.836 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:12.836 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.901297+0000", 2024-09-21T21:43:12.836 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011468699999999999 2024-09-21T21:43:12.836 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.836 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.836 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:43:12.836 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.901559+0000", 2024-09-21T21:43:12.836 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000261764 2024-09-21T21:43:12.837 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.837 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.837 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:43:12.837 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904281+0000", 2024-09-21T21:43:12.837 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0027229329999999999 2024-09-21T21:43:12.837 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.837 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.837 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:12.837 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905588+0000", 2024-09-21T21:43:12.837 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.001306438 2024-09-21T21:43:12.837 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.837 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.837 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:43:12.837 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905644+0000", 2024-09-21T21:43:12.837 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5747e-05 2024-09-21T21:43:12.837 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.838 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.838 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:12.838 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905761+0000", 2024-09-21T21:43:12.838 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000117856 2024-09-21T21:43:12.838 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:12.838 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:12.838 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:12.838 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.838 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.838 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10373 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:43:12.838 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.841027+0000", 2024-09-21T21:43:12.838 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 58.911218453000004, 2024-09-21T21:43:12.838 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.066941839000000003, 2024-09-21T21:43:12.838 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:12.838 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:12.839 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:12.839 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:12.839 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:43:12.839 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:43:12.839 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10373 2024-09-21T21:43:12.839 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.839 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:12.839 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.839 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:12.839 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841027+0000", 2024-09-21T21:43:12.839 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:12.839 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.839 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.839 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:12.839 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841027+0000", 2024-09-21T21:43:12.840 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:12.840 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.840 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.840 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:12.840 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841027+0000", 2024-09-21T21:43:12.840 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6199999999999999e-07 2024-09-21T21:43:12.840 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.840 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.840 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:12.840 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841030+0000", 2024-09-21T21:43:12.840 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.729e-06 2024-09-21T21:43:12.840 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.840 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.840 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:12.840 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841036+0000", 2024-09-21T21:43:12.841 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1979999999999998e-06 2024-09-21T21:43:12.841 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.841 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.841 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:12.841 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841042+0000", 2024-09-21T21:43:12.841 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4289999999999998e-06 2024-09-21T21:43:12.841 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.841 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.841 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:12.841 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902433+0000", 2024-09-21T21:43:12.841 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.061390329 2024-09-21T21:43:12.841 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.841 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.841 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:12.841 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902541+0000", 2024-09-21T21:43:12.841 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000108349 2024-09-21T21:43:12.841 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.841 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.842 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:43:12.842 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902983+0000", 2024-09-21T21:43:12.842 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00044158000000000001 2024-09-21T21:43:12.842 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.842 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.842 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:43:12.842 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904490+0000", 2024-09-21T21:43:12.842 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0015074489999999999 2024-09-21T21:43:12.842 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.842 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.842 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:12.842 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.907671+0000", 2024-09-21T21:43:12.842 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.003180511 2024-09-21T21:43:12.843 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.843 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.843 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:43:12.843 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.907767+0000", 2024-09-21T21:43:12.843 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6379000000000007e-05 2024-09-21T21:43:12.843 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.843 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.843 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:12.843 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.907969+0000", 2024-09-21T21:43:12.843 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00020152399999999999 2024-09-21T21:43:12.843 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:12.843 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:12.843 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:12.843 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.843 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.843 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:12335 3.1a 3:5b387735:::rbd_data.1121faebdfe3.0000000000000005:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:43:12.843 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:14.411635+0000", 2024-09-21T21:43:12.844 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 58.340610574999999, 2024-09-21T21:43:12.844 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.005973491, 2024-09-21T21:43:12.844 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:12.844 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:12.844 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:12.844 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:12.844 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:43:12.844 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:43:12.844 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 12335 2024-09-21T21:43:12.844 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.844 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:12.844 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.844 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:12.844 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411635+0000", 2024-09-21T21:43:12.844 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:12.844 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.845 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.845 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:12.845 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411635+0000", 2024-09-21T21:43:12.845 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:12.845 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.845 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.845 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:12.845 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411635+0000", 2024-09-21T21:43:12.845 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8500000000000001e-07 2024-09-21T21:43:12.845 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.845 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.845 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:12.845 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411639+0000", 2024-09-21T21:43:12.845 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.574e-06 2024-09-21T21:43:12.845 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.846 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.846 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:12.846 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411645+0000", 2024-09-21T21:43:12.846 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6869999999999997e-06 2024-09-21T21:43:12.846 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.846 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.846 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:12.846 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411649+0000", 2024-09-21T21:43:12.846 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.771e-06 2024-09-21T21:43:12.846 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.846 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.846 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:12.846 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411695+0000", 2024-09-21T21:43:12.846 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6e-05 2024-09-21T21:43:12.846 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.847 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.847 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op must wait for map", 2024-09-21T21:43:12.847 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411702+0000", 2024-09-21T21:43:12.847 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5590000000000002e-06 2024-09-21T21:43:12.847 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.847 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.847 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:12.847 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.417222+0000", 2024-09-21T21:43:12.847 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0055198219999999997 2024-09-21T21:43:12.847 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.847 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.847 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for peered", 2024-09-21T21:43:12.847 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.417230+0000", 2024-09-21T21:43:12.847 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.123e-06 2024-09-21T21:43:12.847 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.848 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.848 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:12.848 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.417348+0000", 2024-09-21T21:43:12.848 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.000118029 2024-09-21T21:43:12.848 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.848 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.848 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:12.848 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.417451+0000", 2024-09-21T21:43:12.848 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000103674 2024-09-21T21:43:12.848 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.848 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.848 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:12.848 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.417608+0000", 2024-09-21T21:43:12.848 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00015666700000000001 2024-09-21T21:43:12.848 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:12.849 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:12.849 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:12.849 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.849 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.849 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:12349 3.1a 3:5b387735:::rbd_data.1121faebdfe3.0000000000000005:head [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:43:12.849 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:14.411833+0000", 2024-09-21T21:43:12.849 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 58.340411762000002, 2024-09-21T21:43:12.849 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0065064500000001, 2024-09-21T21:43:12.849 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:12.849 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:12.849 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:12.850 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:12.850 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:43:12.850 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:43:12.850 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 12349 2024-09-21T21:43:12.850 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.850 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:12.850 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.850 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:12.850 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411833+0000", 2024-09-21T21:43:12.850 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:12.850 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.850 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.850 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:12.851 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411833+0000", 2024-09-21T21:43:12.851 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:12.851 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.851 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.851 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:12.851 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411834+0000", 2024-09-21T21:43:12.851 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.03e-07 2024-09-21T21:43:12.851 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.851 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.851 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:12.851 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411836+0000", 2024-09-21T21:43:12.851 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.4700000000000001e-06 2024-09-21T21:43:12.851 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.851 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.852 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:12.852 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411843+0000", 2024-09-21T21:43:12.852 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4069999999999998e-06 2024-09-21T21:43:12.852 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.852 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.852 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:12.852 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411848+0000", 2024-09-21T21:43:12.852 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1580000000000004e-06 2024-09-21T21:43:12.852 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.852 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.852 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:12.852 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411901+0000", 2024-09-21T21:43:12.852 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2824000000000001e-05 2024-09-21T21:43:12.852 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.852 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.853 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting_for_map not empty", 2024-09-21T21:43:12.853 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411904+0000", 2024-09-21T21:43:12.853 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0639999999999998e-06 2024-09-21T21:43:12.853 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.853 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.853 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:12.853 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.417252+0000", 2024-09-21T21:43:12.853 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0053485060000000003 2024-09-21T21:43:12.853 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.853 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.853 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for peered", 2024-09-21T21:43:12.853 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.417259+0000", 2024-09-21T21:43:12.853 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.1149999999999996e-06 2024-09-21T21:43:12.853 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.853 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.854 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:12.854 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.418160+0000", 2024-09-21T21:43:12.854 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.000901083 2024-09-21T21:43:12.854 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.854 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.854 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:12.854 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.418236+0000", 2024-09-21T21:43:12.854 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5852000000000005e-05 2024-09-21T21:43:12.854 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.854 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.854 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:12.854 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.418340+0000", 2024-09-21T21:43:12.854 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000103568 2024-09-21T21:43:12.854 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:12.854 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:12.855 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:12.855 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.855 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.855 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4385.0:30053 3.6 3:67cae2ed:::rbd_mirroring:head [list-watchers] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e141)", 2024-09-21T21:43:12.855 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:43:03.223636+0000", 2024-09-21T21:43:12.855 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 9.5286090780000006, 2024-09-21T21:43:12.855 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.062115648, 2024-09-21T21:43:12.855 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:12.855 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:12.855 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:43:12.855 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:12.855 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4385", 2024-09-21T21:43:12.855 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:43:12.856 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 30053 2024-09-21T21:43:12.856 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.856 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:12.856 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.856 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:12.856 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:43:03.223636+0000", 2024-09-21T21:43:12.856 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:12.856 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.856 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.856 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:12.856 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:43:03.223636+0000", 2024-09-21T21:43:12.856 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:12.856 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.856 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.856 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:12.857 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:43:03.223638+0000", 2024-09-21T21:43:12.857 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.6080000000000001e-06 2024-09-21T21:43:12.857 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.857 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.857 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:12.857 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:43:03.223643+0000", 2024-09-21T21:43:12.857 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9110000000000001e-06 2024-09-21T21:43:12.857 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.857 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.857 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:12.857 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:43:03.223654+0000", 2024-09-21T21:43:12.857 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1029999999999999e-05 2024-09-21T21:43:12.857 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.857 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.857 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:12.857 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:43:03.223664+0000", 2024-09-21T21:43:12.857 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.9299999999999998e-06 2024-09-21T21:43:12.858 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.858 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.858 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:12.858 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:43:03.223856+0000", 2024-09-21T21:43:12.858 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00019216200000000001 2024-09-21T21:43:12.858 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.858 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.858 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for peered", 2024-09-21T21:43:12.858 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:43:03.223866+0000", 2024-09-21T21:43:12.858 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0356999999999999e-05 2024-09-21T21:43:12.858 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.858 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.858 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:12.858 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:43:04.285628+0000", 2024-09-21T21:43:12.859 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.061762211 2024-09-21T21:43:12.859 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:12.859 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:12.859 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:12.859 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:43:04.285752+0000", 2024-09-21T21:43:12.859 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00012343900000000001 2024-09-21T21:43:12.859 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:12.859 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:12.859 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:12.859 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:12.859 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:12.859 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:43:12.861 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T21:43:12.861 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:43:12.859+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:43:12.962 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:43:12.960+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:43:13.006 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:43:13.006 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:43:13.006 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 0 2024-09-21T21:43:13.006 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:43:13.014 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T21:43:13.099 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:43:13.098+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:43:13.157 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:43:13.158 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:43:13.158 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:43:13.158 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:43:13.158 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:43:13.163 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:43:13.162+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:43:13.166 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T21:43:13.244 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 5, 3, 0] out_osds: [4] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-21T21:43:13.244 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T21:43:13.244 INFO:tasks.thrashosds.thrasher:Adding osd 4 2024-09-21T21:43:13.245 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 4 2024-09-21T21:43:13.294 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:43:13.293+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:43:13.320 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:43:13.320 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 20, 2024-09-21T21:43:13.320 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 600, 2024-09-21T21:43:13.321 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:43:13.321 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.321 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13296 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 4046848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:13.321 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.796643+0000", 2024-09-21T21:43:13.321 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 82.520862996000005, 2024-09-21T21:43:13.321 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071682019999999999, 2024-09-21T21:43:13.321 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:13.321 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:13.321 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:13.321 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:13.321 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:13.321 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:13.322 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13296 2024-09-21T21:43:13.322 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.322 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:13.322 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.322 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:13.322 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796643+0000", 2024-09-21T21:43:13.322 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:13.322 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.322 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.322 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:13.323 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796643+0000", 2024-09-21T21:43:13.323 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:13.323 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.323 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.323 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:13.323 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796643+0000", 2024-09-21T21:43:13.323 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.7199999999999998e-07 2024-09-21T21:43:13.323 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.323 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.323 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:13.323 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796647+0000", 2024-09-21T21:43:13.323 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6770000000000001e-06 2024-09-21T21:43:13.324 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.324 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.324 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:13.324 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796656+0000", 2024-09-21T21:43:13.324 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2059999999999996e-06 2024-09-21T21:43:13.324 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.324 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.324 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:13.324 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796664+0000", 2024-09-21T21:43:13.324 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7919999999999993e-06 2024-09-21T21:43:13.324 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.324 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.324 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:13.324 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.868186+0000", 2024-09-21T21:43:13.325 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071522273999999997 2024-09-21T21:43:13.325 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.325 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.325 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:13.325 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.868239+0000", 2024-09-21T21:43:13.325 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2787999999999997e-05 2024-09-21T21:43:13.325 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.325 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.325 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:13.325 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.868325+0000", 2024-09-21T21:43:13.325 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.5611000000000004e-05 2024-09-21T21:43:13.325 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:13.325 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:13.325 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:13.326 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.326 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.326 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13561 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 1728512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:13.326 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.823991+0000", 2024-09-21T21:43:13.326 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 82.493514695000002, 2024-09-21T21:43:13.326 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071654762999999996, 2024-09-21T21:43:13.326 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:13.326 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:13.326 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:13.326 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:13.326 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:13.326 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:13.326 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13561 2024-09-21T21:43:13.326 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.327 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:13.327 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.327 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:13.327 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.823991+0000", 2024-09-21T21:43:13.327 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:13.327 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.327 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.327 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:13.327 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.823991+0000", 2024-09-21T21:43:13.327 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:13.327 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.327 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.327 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:13.327 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.823991+0000", 2024-09-21T21:43:13.328 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9299999999999998e-07 2024-09-21T21:43:13.328 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.328 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.328 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:13.328 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.823994+0000", 2024-09-21T21:43:13.328 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2369999999999999e-06 2024-09-21T21:43:13.328 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.328 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.328 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:13.328 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824000+0000", 2024-09-21T21:43:13.328 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.55e-06 2024-09-21T21:43:13.328 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.328 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.328 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:13.328 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824005+0000", 2024-09-21T21:43:13.329 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4329999999999997e-06 2024-09-21T21:43:13.329 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.329 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.329 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:13.329 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.895360+0000", 2024-09-21T21:43:13.329 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071355426 2024-09-21T21:43:13.329 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.329 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.329 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:13.329 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.895477+0000", 2024-09-21T21:43:13.329 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0001171 2024-09-21T21:43:13.329 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.329 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.329 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:13.330 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.895646+0000", 2024-09-21T21:43:13.330 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000168524 2024-09-21T21:43:13.330 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:13.330 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:13.330 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:13.330 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.330 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.330 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13562 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 1757184~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:13.330 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.824011+0000", 2024-09-21T21:43:13.330 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 82.493494818000002, 2024-09-21T21:43:13.330 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072020453999999998, 2024-09-21T21:43:13.330 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:13.330 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:13.330 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:13.330 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:13.331 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:13.331 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:13.331 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13562 2024-09-21T21:43:13.331 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.331 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:13.331 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.331 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:13.331 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824011+0000", 2024-09-21T21:43:13.331 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:13.331 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.331 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.331 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:13.331 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824011+0000", 2024-09-21T21:43:13.331 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:13.332 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.332 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.332 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:13.332 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824011+0000", 2024-09-21T21:43:13.332 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5600000000000001e-07 2024-09-21T21:43:13.332 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.332 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.332 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:13.332 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824013+0000", 2024-09-21T21:43:13.332 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9800000000000001e-06 2024-09-21T21:43:13.332 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.332 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.332 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:13.332 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824019+0000", 2024-09-21T21:43:13.332 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7339999999999996e-06 2024-09-21T21:43:13.333 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.333 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.333 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:13.333 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824023+0000", 2024-09-21T21:43:13.333 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3680000000000004e-06 2024-09-21T21:43:13.333 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.333 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.333 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:13.333 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.895699+0000", 2024-09-21T21:43:13.333 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071675597999999993 2024-09-21T21:43:13.333 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.333 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.333 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:13.333 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.895830+0000", 2024-09-21T21:43:13.333 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000130858 2024-09-21T21:43:13.334 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.334 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.334 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:13.334 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896031+0000", 2024-09-21T21:43:13.334 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00020155999999999999 2024-09-21T21:43:13.334 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:13.334 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:13.334 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:13.334 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.334 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.334 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13563 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 323584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:13.334 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.824029+0000", 2024-09-21T21:43:13.334 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 82.493476377999997, 2024-09-21T21:43:13.334 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072307563000000005, 2024-09-21T21:43:13.335 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:13.335 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:13.335 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:13.335 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:13.335 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:13.335 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:13.335 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13563 2024-09-21T21:43:13.335 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.335 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:13.335 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.335 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:13.335 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824029+0000", 2024-09-21T21:43:13.335 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:13.336 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.336 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.336 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:13.336 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824029+0000", 2024-09-21T21:43:13.336 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:13.336 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.336 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.336 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:13.336 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824030+0000", 2024-09-21T21:43:13.336 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1699999999999999e-07 2024-09-21T21:43:13.336 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.336 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.336 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:13.336 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824032+0000", 2024-09-21T21:43:13.336 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0320000000000002e-06 2024-09-21T21:43:13.337 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.337 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.337 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:13.337 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824038+0000", 2024-09-21T21:43:13.337 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8769999999999998e-06 2024-09-21T21:43:13.337 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.337 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.337 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:13.337 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824042+0000", 2024-09-21T21:43:13.337 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5650000000000003e-06 2024-09-21T21:43:13.337 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.337 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.337 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:13.337 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896058+0000", 2024-09-21T21:43:13.337 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072015941 2024-09-21T21:43:13.338 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.338 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.338 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:13.338 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896164+0000", 2024-09-21T21:43:13.338 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010593499999999999 2024-09-21T21:43:13.338 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.338 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.338 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:13.338 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896337+0000", 2024-09-21T21:43:13.338 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00017279599999999999 2024-09-21T21:43:13.338 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:13.338 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:13.338 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:13.339 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.339 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.339 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13564 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 983040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:13.339 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.824048+0000", 2024-09-21T21:43:13.339 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 82.493457303, 2024-09-21T21:43:13.339 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072491193999999995, 2024-09-21T21:43:13.339 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:13.339 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:13.339 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:13.339 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:13.339 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:13.339 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:13.339 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13564 2024-09-21T21:43:13.340 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.340 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:13.340 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.340 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:13.340 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824048+0000", 2024-09-21T21:43:13.340 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:13.340 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.340 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.340 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:13.340 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824048+0000", 2024-09-21T21:43:13.340 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:13.340 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.340 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.341 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:13.341 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824663+0000", 2024-09-21T21:43:13.341 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00061494299999999996 2024-09-21T21:43:13.341 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.341 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.341 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:13.341 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824669+0000", 2024-09-21T21:43:13.341 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2900000000000002e-06 2024-09-21T21:43:13.341 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.341 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.341 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:13.341 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824679+0000", 2024-09-21T21:43:13.341 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0338e-05 2024-09-21T21:43:13.341 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.341 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.342 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:13.342 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824687+0000", 2024-09-21T21:43:13.342 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.9049999999999997e-06 2024-09-21T21:43:13.342 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.342 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.342 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:13.342 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896381+0000", 2024-09-21T21:43:13.342 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071694305999999999 2024-09-21T21:43:13.342 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.342 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.342 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:13.342 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896445+0000", 2024-09-21T21:43:13.343 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3869999999999997e-05 2024-09-21T21:43:13.343 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.343 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.343 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:13.343 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896540+0000", 2024-09-21T21:43:13.343 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.4542000000000004e-05 2024-09-21T21:43:13.343 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:13.343 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:13.343 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:13.343 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.343 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.343 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13565 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3190784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:13.343 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.824704+0000", 2024-09-21T21:43:13.343 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 82.492801431999993, 2024-09-21T21:43:13.344 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072014745000000005, 2024-09-21T21:43:13.344 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:13.344 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:13.344 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:13.344 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:13.344 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:13.344 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:13.344 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13565 2024-09-21T21:43:13.344 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.344 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:13.344 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.345 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:13.345 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824704+0000", 2024-09-21T21:43:13.345 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:13.345 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.345 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.345 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:13.345 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824704+0000", 2024-09-21T21:43:13.345 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:13.345 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.345 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.345 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:13.345 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824705+0000", 2024-09-21T21:43:13.345 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8999999999999997e-07 2024-09-21T21:43:13.345 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.346 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.346 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:13.346 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824707+0000", 2024-09-21T21:43:13.346 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.7669999999999999e-06 2024-09-21T21:43:13.346 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.346 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.346 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:13.346 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824715+0000", 2024-09-21T21:43:13.346 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7700000000000001e-06 2024-09-21T21:43:13.346 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.346 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.346 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:13.347 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824721+0000", 2024-09-21T21:43:13.347 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9560000000000002e-06 2024-09-21T21:43:13.347 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.347 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.347 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:13.347 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896570+0000", 2024-09-21T21:43:13.347 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071848870999999995 2024-09-21T21:43:13.347 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.347 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.347 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:13.347 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896626+0000", 2024-09-21T21:43:13.348 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5578000000000002e-05 2024-09-21T21:43:13.348 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.348 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.348 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:13.348 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896719+0000", 2024-09-21T21:43:13.348 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.3313000000000006e-05 2024-09-21T21:43:13.348 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:13.348 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:13.348 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:13.348 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.348 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.348 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13566 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3162112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:13.349 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.824729+0000", 2024-09-21T21:43:13.349 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 82.492776544999998, 2024-09-21T21:43:13.349 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072356907999999998, 2024-09-21T21:43:13.349 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:13.349 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:13.349 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:13.349 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:13.349 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:13.349 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:13.349 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13566 2024-09-21T21:43:13.350 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.350 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:13.350 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.350 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:13.350 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824729+0000", 2024-09-21T21:43:13.350 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:13.350 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.350 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.350 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:13.350 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824729+0000", 2024-09-21T21:43:13.350 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:13.351 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.351 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.351 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:13.351 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825630+0000", 2024-09-21T21:43:13.351 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00090139400000000002 2024-09-21T21:43:13.351 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.351 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.351 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:13.351 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825634+0000", 2024-09-21T21:43:13.352 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4960000000000001e-06 2024-09-21T21:43:13.352 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.352 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.352 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:13.352 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825644+0000", 2024-09-21T21:43:13.352 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.9650000000000003e-06 2024-09-21T21:43:13.352 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.352 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.352 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:13.352 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825649+0000", 2024-09-21T21:43:13.353 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0359999999999997e-06 2024-09-21T21:43:13.353 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.353 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.353 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:13.353 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896757+0000", 2024-09-21T21:43:13.353 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071107717000000001 2024-09-21T21:43:13.353 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.353 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.353 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:13.354 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896878+0000", 2024-09-21T21:43:13.354 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000120834 2024-09-21T21:43:13.354 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.354 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.354 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:13.354 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897086+0000", 2024-09-21T21:43:13.354 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00020846600000000001 2024-09-21T21:43:13.354 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:13.354 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:13.354 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:13.355 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.355 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.355 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13568 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3907584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:13.355 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.825671+0000", 2024-09-21T21:43:13.355 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 82.491834135999994, 2024-09-21T21:43:13.355 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071794595000000003, 2024-09-21T21:43:13.355 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:13.355 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:13.355 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:13.355 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:13.355 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:13.355 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:13.355 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13568 2024-09-21T21:43:13.355 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.356 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:13.356 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.356 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:13.356 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825671+0000", 2024-09-21T21:43:13.356 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:13.356 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.356 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.356 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:13.356 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825671+0000", 2024-09-21T21:43:13.356 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:13.356 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.356 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.357 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:13.357 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825672+0000", 2024-09-21T21:43:13.357 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2999999999999999e-07 2024-09-21T21:43:13.357 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.357 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.357 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:13.357 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825673+0000", 2024-09-21T21:43:13.357 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.246e-06 2024-09-21T21:43:13.357 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.357 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.357 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:13.357 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825693+0000", 2024-09-21T21:43:13.358 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9789999999999999e-05 2024-09-21T21:43:13.358 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.358 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.358 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:13.358 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825697+0000", 2024-09-21T21:43:13.358 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2379999999999999e-06 2024-09-21T21:43:13.358 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.358 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.358 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:13.358 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897284+0000", 2024-09-21T21:43:13.358 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071586710999999997 2024-09-21T21:43:13.358 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.358 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.359 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:13.359 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897337+0000", 2024-09-21T21:43:13.359 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2800000000000003e-05 2024-09-21T21:43:13.359 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.359 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.359 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:13.359 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897466+0000", 2024-09-21T21:43:13.359 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00012957999999999999 2024-09-21T21:43:13.359 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:13.359 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:13.359 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:13.360 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.360 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.360 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13574 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 352256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:13.360 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.825702+0000", 2024-09-21T21:43:13.360 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 82.491803988000001, 2024-09-21T21:43:13.360 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072080377000000001, 2024-09-21T21:43:13.360 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:13.360 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:13.360 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:13.360 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:13.360 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:13.360 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:13.360 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13574 2024-09-21T21:43:13.360 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.361 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:13.361 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.361 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:13.361 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825702+0000", 2024-09-21T21:43:13.361 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:13.361 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.361 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.361 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:13.361 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825702+0000", 2024-09-21T21:43:13.361 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:13.361 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.361 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.361 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:13.362 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825943+0000", 2024-09-21T21:43:13.362 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00024105200000000001 2024-09-21T21:43:13.362 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.362 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.362 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:13.362 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825946+0000", 2024-09-21T21:43:13.362 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.743e-06 2024-09-21T21:43:13.362 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.362 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.362 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:13.362 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825956+0000", 2024-09-21T21:43:13.362 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6609999999999995e-06 2024-09-21T21:43:13.362 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.362 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.362 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:13.363 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825963+0000", 2024-09-21T21:43:13.363 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4609999999999996e-06 2024-09-21T21:43:13.363 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.363 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.363 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:13.363 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897502+0000", 2024-09-21T21:43:13.363 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071539354999999999 2024-09-21T21:43:13.363 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.363 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.363 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:13.363 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897619+0000", 2024-09-21T21:43:13.363 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011665 2024-09-21T21:43:13.363 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.363 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.364 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:13.364 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897782+0000", 2024-09-21T21:43:13.364 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00016345500000000001 2024-09-21T21:43:13.364 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:13.364 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:13.364 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:13.364 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.364 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.364 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13575 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 380928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:13.364 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.825973+0000", 2024-09-21T21:43:13.364 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 82.491532551999995, 2024-09-21T21:43:13.364 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072012723000000001, 2024-09-21T21:43:13.364 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:13.365 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:13.365 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:13.365 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:13.365 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:13.365 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:13.365 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13575 2024-09-21T21:43:13.365 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.365 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:13.365 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.365 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:13.365 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825973+0000", 2024-09-21T21:43:13.365 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:13.365 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.365 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.366 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:13.366 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825973+0000", 2024-09-21T21:43:13.366 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:13.366 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.366 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.366 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:13.366 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826746+0000", 2024-09-21T21:43:13.366 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00077326800000000002 2024-09-21T21:43:13.366 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.366 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.366 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:13.366 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826766+0000", 2024-09-21T21:43:13.366 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9465000000000001e-05 2024-09-21T21:43:13.367 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.367 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.367 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:13.367 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826812+0000", 2024-09-21T21:43:13.367 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5927999999999998e-05 2024-09-21T21:43:13.367 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.367 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.367 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:13.367 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826833+0000", 2024-09-21T21:43:13.367 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1710999999999999e-05 2024-09-21T21:43:13.367 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.367 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.367 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:13.367 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897809+0000", 2024-09-21T21:43:13.367 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.070975229000000001 2024-09-21T21:43:13.368 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.368 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.368 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:13.368 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897864+0000", 2024-09-21T21:43:13.368 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5334999999999998e-05 2024-09-21T21:43:13.368 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.368 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.368 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:13.368 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897986+0000", 2024-09-21T21:43:13.368 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000121787 2024-09-21T21:43:13.368 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:13.368 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:13.368 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:13.368 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.369 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.369 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13577 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 1814528~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:13.369 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.826874+0000", 2024-09-21T21:43:13.369 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 82.490631518000001, 2024-09-21T21:43:13.369 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071639123999999998, 2024-09-21T21:43:13.369 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:13.369 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:13.369 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:13.369 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:13.369 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:13.369 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:13.369 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13577 2024-09-21T21:43:13.369 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.369 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:13.369 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.370 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:13.370 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826874+0000", 2024-09-21T21:43:13.370 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:13.370 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.370 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.370 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:13.370 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826874+0000", 2024-09-21T21:43:13.370 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:13.370 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.370 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.370 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:13.370 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826875+0000", 2024-09-21T21:43:13.370 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.82e-07 2024-09-21T21:43:13.371 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.371 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.371 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:13.371 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826877+0000", 2024-09-21T21:43:13.371 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.295e-06 2024-09-21T21:43:13.371 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.371 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.371 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:13.371 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826883+0000", 2024-09-21T21:43:13.371 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2609999999999999e-06 2024-09-21T21:43:13.371 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.371 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.371 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:13.371 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826889+0000", 2024-09-21T21:43:13.371 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5589999999999996e-06 2024-09-21T21:43:13.372 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.372 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.372 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:13.372 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898301+0000", 2024-09-21T21:43:13.372 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071412188000000001 2024-09-21T21:43:13.372 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.372 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.372 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:13.372 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898416+0000", 2024-09-21T21:43:13.372 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000115538 2024-09-21T21:43:13.372 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.373 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.373 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:13.373 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898513+0000", 2024-09-21T21:43:13.373 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6800999999999997e-05 2024-09-21T21:43:13.373 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:13.373 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:13.373 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:13.373 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.373 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.373 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13578 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 2502656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:13.373 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.826896+0000", 2024-09-21T21:43:13.373 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 82.490610062000002, 2024-09-21T21:43:13.373 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071789411999999997, 2024-09-21T21:43:13.374 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:13.374 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:13.374 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:13.374 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:13.374 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:13.374 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:13.374 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13578 2024-09-21T21:43:13.374 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.374 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:13.374 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.374 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:13.374 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826896+0000", 2024-09-21T21:43:13.375 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:13.375 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.375 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.375 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:13.375 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826896+0000", 2024-09-21T21:43:13.375 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:13.375 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.375 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.375 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:13.375 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827327+0000", 2024-09-21T21:43:13.376 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00043094500000000002 2024-09-21T21:43:13.376 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.376 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.376 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:13.376 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827331+0000", 2024-09-21T21:43:13.376 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.0969999999999999e-06 2024-09-21T21:43:13.376 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.376 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.376 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:13.376 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827341+0000", 2024-09-21T21:43:13.376 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0199e-05 2024-09-21T21:43:13.376 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.377 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.377 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:13.377 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827346+0000", 2024-09-21T21:43:13.377 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3210000000000004e-06 2024-09-21T21:43:13.377 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.377 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.377 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:13.377 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898540+0000", 2024-09-21T21:43:13.377 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071194246000000003 2024-09-21T21:43:13.378 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.378 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.378 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:13.378 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898598+0000", 2024-09-21T21:43:13.378 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7658999999999998e-05 2024-09-21T21:43:13.378 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.378 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.378 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:13.378 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898685+0000", 2024-09-21T21:43:13.379 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.6945000000000002e-05 2024-09-21T21:43:13.379 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:13.379 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:13.379 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:13.379 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.379 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.379 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13579 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 2531328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:13.379 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.827355+0000", 2024-09-21T21:43:13.379 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 82.490150530999998, 2024-09-21T21:43:13.379 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071650933, 2024-09-21T21:43:13.379 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:13.379 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:13.380 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:13.380 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:13.380 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:13.380 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:13.380 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13579 2024-09-21T21:43:13.380 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.380 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:13.380 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.380 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:13.380 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827355+0000", 2024-09-21T21:43:13.380 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:13.381 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.381 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.381 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:13.381 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827355+0000", 2024-09-21T21:43:13.381 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:13.381 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.381 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.381 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:13.381 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827486+0000", 2024-09-21T21:43:13.381 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00013066799999999999 2024-09-21T21:43:13.381 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.382 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.382 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:13.382 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827490+0000", 2024-09-21T21:43:13.382 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.9009999999999996e-06 2024-09-21T21:43:13.382 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.382 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.382 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:13.383 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827499+0000", 2024-09-21T21:43:13.383 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6209999999999999e-06 2024-09-21T21:43:13.383 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.383 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.383 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:13.383 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827506+0000", 2024-09-21T21:43:13.383 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6490000000000002e-06 2024-09-21T21:43:13.383 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.384 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.384 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:13.384 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898713+0000", 2024-09-21T21:43:13.384 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071206937999999997 2024-09-21T21:43:13.384 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.384 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.384 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:13.384 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898811+0000", 2024-09-21T21:43:13.384 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.8159999999999995e-05 2024-09-21T21:43:13.384 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.385 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.385 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:13.385 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899006+0000", 2024-09-21T21:43:13.385 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00019499599999999999 2024-09-21T21:43:13.385 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:13.385 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:13.385 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:13.385 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.385 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.385 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13580 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3219456~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:13.386 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.827522+0000", 2024-09-21T21:43:13.386 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 82.489984100000001, 2024-09-21T21:43:13.386 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071813298999999997, 2024-09-21T21:43:13.386 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:13.386 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:13.386 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:13.386 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:13.387 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:13.387 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:13.387 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13580 2024-09-21T21:43:13.387 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.387 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:13.387 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.387 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:13.387 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827522+0000", 2024-09-21T21:43:13.387 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:13.387 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.388 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.388 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:13.388 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827522+0000", 2024-09-21T21:43:13.388 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:13.388 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.388 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.388 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:13.388 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827898+0000", 2024-09-21T21:43:13.388 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000376876 2024-09-21T21:43:13.388 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.389 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.389 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:13.389 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827903+0000", 2024-09-21T21:43:13.389 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4129999999999999e-06 2024-09-21T21:43:13.389 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.389 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.389 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:13.389 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827911+0000", 2024-09-21T21:43:13.389 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.4540000000000007e-06 2024-09-21T21:43:13.389 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.389 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.389 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:13.389 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827917+0000", 2024-09-21T21:43:13.390 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2110000000000003e-06 2024-09-21T21:43:13.390 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.390 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.390 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:13.390 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899041+0000", 2024-09-21T21:43:13.390 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071123965999999997 2024-09-21T21:43:13.390 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.390 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.390 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:13.390 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899157+0000", 2024-09-21T21:43:13.390 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011573700000000001 2024-09-21T21:43:13.390 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.390 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.391 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:13.391 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899335+0000", 2024-09-21T21:43:13.391 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00017764199999999999 2024-09-21T21:43:13.391 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:13.391 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:13.391 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:13.391 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.391 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.391 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13581 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3248128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:13.391 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.827929+0000", 2024-09-21T21:43:13.391 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 82.489576701000004, 2024-09-21T21:43:13.392 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071730151000000006, 2024-09-21T21:43:13.392 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:13.392 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:13.392 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:13.392 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:13.392 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:13.392 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:13.392 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13581 2024-09-21T21:43:13.392 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.392 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:13.392 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.392 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:13.392 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827929+0000", 2024-09-21T21:43:13.392 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:13.393 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.393 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.393 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:13.393 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827929+0000", 2024-09-21T21:43:13.393 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:13.393 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.393 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.393 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:13.393 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.829943+0000", 2024-09-21T21:43:13.393 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0020143750000000001 2024-09-21T21:43:13.393 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.393 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.393 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:13.393 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.829948+0000", 2024-09-21T21:43:13.393 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9400000000000001e-06 2024-09-21T21:43:13.394 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.394 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.394 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:13.394 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.829960+0000", 2024-09-21T21:43:13.394 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.2123999999999999e-05 2024-09-21T21:43:13.394 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.394 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.394 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:13.394 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.829968+0000", 2024-09-21T21:43:13.394 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7000000000000008e-06 2024-09-21T21:43:13.394 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.394 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.394 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:13.394 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899370+0000", 2024-09-21T21:43:13.394 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.069401672999999997 2024-09-21T21:43:13.395 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.395 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.395 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:13.395 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899481+0000", 2024-09-21T21:43:13.395 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011145300000000001 2024-09-21T21:43:13.395 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.395 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.395 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:13.395 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899659+0000", 2024-09-21T21:43:13.395 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00017788600000000001 2024-09-21T21:43:13.395 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:13.395 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:13.395 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:13.395 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.395 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.395 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13595 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 1871872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:13.396 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.830180+0000", 2024-09-21T21:43:13.396 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 82.487325587000001, 2024-09-21T21:43:13.396 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071809244999999994, 2024-09-21T21:43:13.396 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:13.396 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:13.396 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:13.396 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:13.396 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:13.396 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:13.396 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13595 2024-09-21T21:43:13.396 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.396 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:13.396 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.396 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:13.396 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.830180+0000", 2024-09-21T21:43:13.397 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:13.397 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.397 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.397 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:13.397 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.830180+0000", 2024-09-21T21:43:13.397 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:13.397 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.397 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.397 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:13.397 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.831967+0000", 2024-09-21T21:43:13.397 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0017868350000000001 2024-09-21T21:43:13.397 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.397 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.397 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:13.397 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.831971+0000", 2024-09-21T21:43:13.398 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3950000000000003e-06 2024-09-21T21:43:13.398 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.398 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.398 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:13.398 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.831983+0000", 2024-09-21T21:43:13.398 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1815e-05 2024-09-21T21:43:13.398 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.398 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.398 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:13.398 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.831993+0000", 2024-09-21T21:43:13.398 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6339999999999996e-06 2024-09-21T21:43:13.398 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.398 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.398 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:13.399 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.901693+0000", 2024-09-21T21:43:13.399 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.069700082999999996 2024-09-21T21:43:13.399 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.399 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.399 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:13.399 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.901813+0000", 2024-09-21T21:43:13.399 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011971 2024-09-21T21:43:13.399 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.399 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.399 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:13.399 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.901989+0000", 2024-09-21T21:43:13.399 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000176773 2024-09-21T21:43:13.399 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:13.399 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:13.399 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:13.400 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.400 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.400 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13601 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3334144~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:13.400 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.832130+0000", 2024-09-21T21:43:13.400 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 82.485375715000004, 2024-09-21T21:43:13.400 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071861727, 2024-09-21T21:43:13.400 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:13.400 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:13.400 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:13.400 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:13.400 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:13.400 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:13.400 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13601 2024-09-21T21:43:13.400 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.400 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:13.401 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.401 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:13.401 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.832130+0000", 2024-09-21T21:43:13.401 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:13.401 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.401 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.401 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:13.401 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.832130+0000", 2024-09-21T21:43:13.401 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:13.401 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.401 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.401 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:13.401 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.833939+0000", 2024-09-21T21:43:13.401 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0018094400000000001 2024-09-21T21:43:13.401 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.402 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.402 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:13.402 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.833944+0000", 2024-09-21T21:43:13.402 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5979999999999999e-06 2024-09-21T21:43:13.402 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.402 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.402 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:13.402 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.833953+0000", 2024-09-21T21:43:13.402 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.1460000000000002e-06 2024-09-21T21:43:13.402 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.402 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.402 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:13.402 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.833960+0000", 2024-09-21T21:43:13.402 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.7989999999999996e-06 2024-09-21T21:43:13.402 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.403 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.403 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:13.403 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.903693+0000", 2024-09-21T21:43:13.403 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.069732901 2024-09-21T21:43:13.403 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.403 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.403 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:13.403 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.903814+0000", 2024-09-21T21:43:13.403 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000121585 2024-09-21T21:43:13.403 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.403 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.403 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:13.403 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.903992+0000", 2024-09-21T21:43:13.403 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00017725799999999999 2024-09-21T21:43:13.403 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:13.404 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:13.404 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:13.404 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.404 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.404 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:12342 3.1 3:87417293:::rbd_data.1121faebdfe3.0000000000000004:head [read 4096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:43:13.404 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:14.411725+0000", 2024-09-21T21:43:13.404 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 58.905780649, 2024-09-21T21:43:13.404 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0089213029999999, 2024-09-21T21:43:13.404 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:13.404 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:13.404 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:13.404 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:13.404 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:43:13.404 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:43:13.404 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 12342 2024-09-21T21:43:13.405 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.405 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:13.405 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.405 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:13.405 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411725+0000", 2024-09-21T21:43:13.405 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:13.405 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.405 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.405 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:13.405 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411725+0000", 2024-09-21T21:43:13.405 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:13.405 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.405 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.405 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:13.405 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411726+0000", 2024-09-21T21:43:13.406 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.4169999999999999e-06 2024-09-21T21:43:13.406 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.406 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.406 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:13.406 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411734+0000", 2024-09-21T21:43:13.406 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.3799999999999996e-06 2024-09-21T21:43:13.406 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.406 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.406 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:13.406 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411750+0000", 2024-09-21T21:43:13.406 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.6429000000000001e-05 2024-09-21T21:43:13.406 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.406 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.406 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:13.407 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411760+0000", 2024-09-21T21:43:13.407 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.4239999999999999e-06 2024-09-21T21:43:13.407 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.407 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.407 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:13.407 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411828+0000", 2024-09-21T21:43:13.407 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.7935999999999995e-05 2024-09-21T21:43:13.407 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.407 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.407 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op must wait for map", 2024-09-21T21:43:13.407 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411837+0000", 2024-09-21T21:43:13.407 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.0100000000000001e-06 2024-09-21T21:43:13.407 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.407 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.408 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:13.408 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.414984+0000", 2024-09-21T21:43:13.408 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0031472050000000001 2024-09-21T21:43:13.408 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.408 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.408 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for peered", 2024-09-21T21:43:13.408 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.414994+0000", 2024-09-21T21:43:13.408 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0291e-05 2024-09-21T21:43:13.408 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.408 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.408 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:13.408 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.420353+0000", 2024-09-21T21:43:13.409 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.005358913 2024-09-21T21:43:13.409 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.409 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.409 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:13.409 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.420489+0000", 2024-09-21T21:43:13.409 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00013592699999999999 2024-09-21T21:43:13.409 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.409 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.409 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:13.409 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.420646+0000", 2024-09-21T21:43:13.409 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00015737100000000001 2024-09-21T21:43:13.410 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:13.410 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:13.410 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:13.410 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.410 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.410 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:12348 3.1 3:87417293:::rbd_data.1121faebdfe3.0000000000000004:head [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:43:13.410 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:14.411793+0000", 2024-09-21T21:43:13.410 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 58.905713102999997, 2024-09-21T21:43:13.410 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.009298384, 2024-09-21T21:43:13.410 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:13.410 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:13.410 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:13.410 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:13.411 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:43:13.411 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:43:13.411 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 12348 2024-09-21T21:43:13.411 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.411 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:13.411 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.411 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:13.411 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411793+0000", 2024-09-21T21:43:13.411 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:13.411 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.411 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.411 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:13.412 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411793+0000", 2024-09-21T21:43:13.412 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:13.412 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.412 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.412 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:13.412 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411793+0000", 2024-09-21T21:43:13.412 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0999999999999998e-07 2024-09-21T21:43:13.412 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.412 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.412 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:13.412 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411797+0000", 2024-09-21T21:43:13.412 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.49e-06 2024-09-21T21:43:13.412 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.412 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.413 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:13.413 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411809+0000", 2024-09-21T21:43:13.413 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.2510999999999999e-05 2024-09-21T21:43:13.413 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.413 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.413 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:13.413 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411816+0000", 2024-09-21T21:43:13.413 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6420000000000001e-06 2024-09-21T21:43:13.413 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.413 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.413 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:13.414 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411963+0000", 2024-09-21T21:43:13.414 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00014700999999999999 2024-09-21T21:43:13.414 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.414 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.414 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting_for_map not empty", 2024-09-21T21:43:13.414 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411970+0000", 2024-09-21T21:43:13.414 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9800000000000001e-06 2024-09-21T21:43:13.414 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.414 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.414 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:13.414 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.415018+0000", 2024-09-21T21:43:13.414 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0030482299999999999 2024-09-21T21:43:13.414 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.415 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.415 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for peered", 2024-09-21T21:43:13.415 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.415025+0000", 2024-09-21T21:43:13.415 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.3749999999999997e-06 2024-09-21T21:43:13.415 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.415 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.415 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:13.415 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.420904+0000", 2024-09-21T21:43:13.415 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0058787339999999 2024-09-21T21:43:13.415 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.415 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.415 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:13.415 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.420988+0000", 2024-09-21T21:43:13.416 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.3955999999999994e-05 2024-09-21T21:43:13.416 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.416 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.416 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:13.416 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.421091+0000", 2024-09-21T21:43:13.416 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000102846 2024-09-21T21:43:13.416 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:13.417 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:13.417 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:13.417 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.417 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.417 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4385.0:20922 3.9 3:920c5256:::rbd_id.image_client.0-clone36:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-21T21:43:13.417 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:17.811324+0000", 2024-09-21T21:43:13.417 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 55.506181370999997, 2024-09-21T21:43:13.417 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.66951882299999999, 2024-09-21T21:43:13.417 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:13.417 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:13.417 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:13.417 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:13.418 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4385", 2024-09-21T21:43:13.418 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:43:13.418 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 20922 2024-09-21T21:43:13.418 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.418 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:13.418 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.418 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:13.418 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:17.811324+0000", 2024-09-21T21:43:13.418 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:13.418 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.419 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.419 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:13.419 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:17.811324+0000", 2024-09-21T21:43:13.419 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:13.419 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.419 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.419 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:13.419 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:17.811326+0000", 2024-09-21T21:43:13.419 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.536e-06 2024-09-21T21:43:13.419 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.419 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.419 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:13.419 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:17.811332+0000", 2024-09-21T21:43:13.420 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6699999999999997e-06 2024-09-21T21:43:13.420 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.420 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.420 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:13.420 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:17.811347+0000", 2024-09-21T21:43:13.420 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.4589999999999999e-05 2024-09-21T21:43:13.420 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.420 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.420 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:13.420 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:17.811359+0000", 2024-09-21T21:43:13.420 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1776e-05 2024-09-21T21:43:13.420 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.420 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.421 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:13.421 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:17.811424+0000", 2024-09-21T21:43:13.421 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4795000000000006e-05 2024-09-21T21:43:13.421 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.421 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.421 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for peered", 2024-09-21T21:43:13.421 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:17.811436+0000", 2024-09-21T21:43:13.421 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.221e-05 2024-09-21T21:43:13.421 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.421 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.421 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:13.421 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:18.480657+0000", 2024-09-21T21:43:13.421 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.66922088999999996 2024-09-21T21:43:13.421 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.422 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.422 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:13.422 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:18.480760+0000", 2024-09-21T21:43:13.422 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000102827 2024-09-21T21:43:13.422 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:13.422 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:13.422 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:13.422 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:18.480843+0000", 2024-09-21T21:43:13.422 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.3529000000000006e-05 2024-09-21T21:43:13.422 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:13.422 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:13.422 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:13.422 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:13.423 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:13.423 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:43:13.424 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T21:43:13.425 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:43:13.422+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:43:13.525 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:43:13.524+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:43:13.537 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:43:13.537 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:43:13.537 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-21T21:43:13.537 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:43:13.546 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T21:43:13.560 INFO:tasks.ceph.mon.a.smithi081.stderr:2024-09-21T21:43:13.559+0000 7f5d67cb4640 -1 mon.a@0(leader).osd e151 definitely_dead 0 2024-09-21T21:43:13.626 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:43:13.624+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:43:13.648 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:43:13.648 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:43:13.648 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:43:13.648 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:43:13.649 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:43:13.657 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T21:43:13.716 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T21:43:13.740 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:43:13.739+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:43:13.801 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:43:13.802 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-21T21:43:13.802 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-21T21:43:13.802 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:43:13.802 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.802 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19747 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 7905280~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:13.802 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273058+0000", 2024-09-21T21:43:13.802 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 82.526294824999994, 2024-09-21T21:43:13.802 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97338591399999996, 2024-09-21T21:43:13.802 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:13.802 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:13.803 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:13.803 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:13.803 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:13.803 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:13.803 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19747 2024-09-21T21:43:13.803 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.803 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:13.803 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.803 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:13.803 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273058+0000", 2024-09-21T21:43:13.803 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:13.803 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.803 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.804 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:13.804 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273058+0000", 2024-09-21T21:43:13.804 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:13.804 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.804 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.804 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:13.804 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273058+0000", 2024-09-21T21:43:13.804 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5900000000000003e-07 2024-09-21T21:43:13.804 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.804 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.804 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:13.804 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273061+0000", 2024-09-21T21:43:13.804 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3479999999999998e-06 2024-09-21T21:43:13.804 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.805 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.805 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:13.805 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273067+0000", 2024-09-21T21:43:13.805 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1469999999999998e-06 2024-09-21T21:43:13.805 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.805 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.805 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:13.805 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273071+0000", 2024-09-21T21:43:13.805 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7339999999999999e-06 2024-09-21T21:43:13.805 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.805 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.805 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:13.805 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277945+0000", 2024-09-21T21:43:13.806 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.004873474 2024-09-21T21:43:13.806 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.806 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.806 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:13.806 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277948+0000", 2024-09-21T21:43:13.806 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0299999999999998e-06 2024-09-21T21:43:13.806 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.806 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.806 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:13.806 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246269+0000", 2024-09-21T21:43:13.806 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96832060200000003 2024-09-21T21:43:13.806 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.806 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.806 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:13.807 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246362+0000", 2024-09-21T21:43:13.807 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3867000000000001e-05 2024-09-21T21:43:13.807 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.807 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.807 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:13.807 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246444+0000", 2024-09-21T21:43:13.807 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1353000000000005e-05 2024-09-21T21:43:13.807 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:13.807 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:13.807 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:13.807 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.807 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.807 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19755 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 5783552~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:13.807 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273077+0000", 2024-09-21T21:43:13.807 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 82.526275209000005, 2024-09-21T21:43:13.808 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97350203400000002, 2024-09-21T21:43:13.808 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:13.808 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:13.808 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:13.808 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:13.808 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:13.808 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:13.808 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19755 2024-09-21T21:43:13.808 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.808 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:13.808 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.808 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:13.808 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273077+0000", 2024-09-21T21:43:13.808 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:13.809 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.809 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.809 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:13.809 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273077+0000", 2024-09-21T21:43:13.809 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:13.809 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.809 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.809 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:13.809 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273078+0000", 2024-09-21T21:43:13.809 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5699999999999998e-07 2024-09-21T21:43:13.809 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.809 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.809 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:13.809 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273083+0000", 2024-09-21T21:43:13.810 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8670000000000002e-06 2024-09-21T21:43:13.810 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.810 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.810 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:13.810 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273088+0000", 2024-09-21T21:43:13.810 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8370000000000002e-06 2024-09-21T21:43:13.810 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.810 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.810 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:13.810 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273093+0000", 2024-09-21T21:43:13.810 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5709999999999999e-06 2024-09-21T21:43:13.810 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.810 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.810 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:13.811 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277968+0000", 2024-09-21T21:43:13.811 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048753909999999998 2024-09-21T21:43:13.811 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.811 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.811 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:13.811 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277971+0000", 2024-09-21T21:43:13.811 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0189999999999998e-06 2024-09-21T21:43:13.811 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.811 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.811 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:13.811 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246485+0000", 2024-09-21T21:43:13.811 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96851399299999996 2024-09-21T21:43:13.811 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.811 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.811 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:13.812 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246536+0000", 2024-09-21T21:43:13.812 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0430000000000003e-05 2024-09-21T21:43:13.812 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.812 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.812 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:13.812 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246579+0000", 2024-09-21T21:43:13.812 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3569000000000002e-05 2024-09-21T21:43:13.812 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:13.812 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:13.812 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:13.812 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.812 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.812 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19756 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6500352~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:13.812 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273099+0000", 2024-09-21T21:43:13.813 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 82.526253835999995, 2024-09-21T21:43:13.813 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97359272799999996, 2024-09-21T21:43:13.813 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:13.813 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:13.813 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:13.813 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:13.813 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:13.813 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:13.813 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19756 2024-09-21T21:43:13.813 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.813 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:13.813 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.813 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:13.813 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273099+0000", 2024-09-21T21:43:13.813 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:13.814 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.814 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.814 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:13.814 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273099+0000", 2024-09-21T21:43:13.814 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:13.814 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.814 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.814 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:13.814 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273099+0000", 2024-09-21T21:43:13.814 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0900000000000002e-07 2024-09-21T21:43:13.814 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.814 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.814 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:13.814 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273101+0000", 2024-09-21T21:43:13.815 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0320000000000002e-06 2024-09-21T21:43:13.815 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.815 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.815 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:13.815 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273107+0000", 2024-09-21T21:43:13.815 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2830000000000001e-06 2024-09-21T21:43:13.815 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.815 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.815 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:13.815 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273111+0000", 2024-09-21T21:43:13.815 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2230000000000001e-06 2024-09-21T21:43:13.815 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.815 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.815 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:13.815 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277994+0000", 2024-09-21T21:43:13.816 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048837070000000002 2024-09-21T21:43:13.816 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.816 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.816 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:13.816 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278000+0000", 2024-09-21T21:43:13.816 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3149999999999999e-06 2024-09-21T21:43:13.816 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.816 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.816 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:13.816 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246607+0000", 2024-09-21T21:43:13.816 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.968606839 2024-09-21T21:43:13.816 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.816 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.816 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:13.817 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246651+0000", 2024-09-21T21:43:13.817 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4734999999999998e-05 2024-09-21T21:43:13.817 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.817 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.817 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:13.817 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246692+0000", 2024-09-21T21:43:13.817 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0185000000000002e-05 2024-09-21T21:43:13.817 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:13.817 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:13.818 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:13.818 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.818 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.818 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19757 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 5812224~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:13.818 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273116+0000", 2024-09-21T21:43:13.818 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 82.526236178999994, 2024-09-21T21:43:13.818 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97367813999999997, 2024-09-21T21:43:13.818 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:13.818 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:13.818 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:13.818 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:13.818 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:13.818 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:13.818 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19757 2024-09-21T21:43:13.819 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.819 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:13.819 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.819 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:13.819 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273116+0000", 2024-09-21T21:43:13.819 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:13.819 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.819 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.819 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:13.819 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273116+0000", 2024-09-21T21:43:13.819 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:13.820 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.820 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.820 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:13.820 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273117+0000", 2024-09-21T21:43:13.820 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0999999999999999e-07 2024-09-21T21:43:13.820 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.820 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.820 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:13.820 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273119+0000", 2024-09-21T21:43:13.820 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9970000000000001e-06 2024-09-21T21:43:13.820 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.820 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.820 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:13.820 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273125+0000", 2024-09-21T21:43:13.821 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9540000000000001e-06 2024-09-21T21:43:13.821 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.821 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.821 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:13.821 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273135+0000", 2024-09-21T21:43:13.821 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0597999999999999e-05 2024-09-21T21:43:13.821 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.821 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.821 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:13.821 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278019+0000", 2024-09-21T21:43:13.822 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048834849999999999 2024-09-21T21:43:13.822 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.822 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.822 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:13.822 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278022+0000", 2024-09-21T21:43:13.822 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9560000000000002e-06 2024-09-21T21:43:13.822 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.822 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.822 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:13.822 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246716+0000", 2024-09-21T21:43:13.822 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.968694202 2024-09-21T21:43:13.822 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.822 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.822 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:13.823 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246758+0000", 2024-09-21T21:43:13.823 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1607000000000003e-05 2024-09-21T21:43:13.823 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.823 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.823 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:13.823 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246795+0000", 2024-09-21T21:43:13.823 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6931000000000003e-05 2024-09-21T21:43:13.823 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:13.823 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:13.823 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:13.823 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.824 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.824 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19758 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6529024~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:13.824 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273142+0000", 2024-09-21T21:43:13.824 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 82.526210957999993, 2024-09-21T21:43:13.824 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97375924599999997, 2024-09-21T21:43:13.824 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:13.825 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:13.825 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:13.825 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:13.825 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:13.825 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:13.825 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19758 2024-09-21T21:43:13.825 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.825 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:13.825 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.825 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:13.825 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273142+0000", 2024-09-21T21:43:13.826 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:13.826 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.826 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.826 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:13.826 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273142+0000", 2024-09-21T21:43:13.826 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:13.826 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.826 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.826 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:13.826 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273142+0000", 2024-09-21T21:43:13.826 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0200000000000003e-07 2024-09-21T21:43:13.826 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.826 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.826 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:13.827 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273144+0000", 2024-09-21T21:43:13.827 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1390000000000002e-06 2024-09-21T21:43:13.827 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.827 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.827 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:13.827 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273150+0000", 2024-09-21T21:43:13.827 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6409999999999999e-06 2024-09-21T21:43:13.827 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.827 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.827 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:13.827 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273157+0000", 2024-09-21T21:43:13.827 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1359999999999999e-06 2024-09-21T21:43:13.827 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.827 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.828 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:13.828 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278050+0000", 2024-09-21T21:43:13.828 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048926170000000001 2024-09-21T21:43:13.828 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.828 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.828 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:13.828 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278053+0000", 2024-09-21T21:43:13.828 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4879999999999999e-06 2024-09-21T21:43:13.828 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.828 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.828 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:13.828 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246819+0000", 2024-09-21T21:43:13.828 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96876590399999996 2024-09-21T21:43:13.828 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.829 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.829 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:13.829 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246856+0000", 2024-09-21T21:43:13.829 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7287999999999999e-05 2024-09-21T21:43:13.829 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.829 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.829 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:13.829 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246901+0000", 2024-09-21T21:43:13.829 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4631e-05 2024-09-21T21:43:13.829 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:13.829 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:13.829 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:13.829 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.830 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.830 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19759 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 7933952~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:13.830 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273172+0000", 2024-09-21T21:43:13.830 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 82.526180694000004, 2024-09-21T21:43:13.830 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97384789500000002, 2024-09-21T21:43:13.830 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:13.830 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:13.830 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:13.830 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:13.830 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:13.830 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:13.830 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19759 2024-09-21T21:43:13.830 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.830 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:13.831 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.831 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:13.831 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273172+0000", 2024-09-21T21:43:13.831 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:13.831 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.831 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.831 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:13.831 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273172+0000", 2024-09-21T21:43:13.831 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:13.831 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.831 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.831 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:13.832 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273172+0000", 2024-09-21T21:43:13.832 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6100000000000002e-07 2024-09-21T21:43:13.832 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.832 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.832 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:13.832 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273174+0000", 2024-09-21T21:43:13.832 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.691e-06 2024-09-21T21:43:13.832 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.832 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.832 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:13.832 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273178+0000", 2024-09-21T21:43:13.832 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7579999999999998e-06 2024-09-21T21:43:13.832 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.832 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.832 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:13.833 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273180+0000", 2024-09-21T21:43:13.833 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7199999999999998e-06 2024-09-21T21:43:13.833 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.833 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.833 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:13.833 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278077+0000", 2024-09-21T21:43:13.833 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048969529999999999 2024-09-21T21:43:13.833 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.833 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.833 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:13.833 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278080+0000", 2024-09-21T21:43:13.833 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.066e-06 2024-09-21T21:43:13.833 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.834 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.834 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:13.834 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246925+0000", 2024-09-21T21:43:13.834 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96884490199999995 2024-09-21T21:43:13.834 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.834 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.834 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:13.834 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246974+0000", 2024-09-21T21:43:13.834 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8387999999999998e-05 2024-09-21T21:43:13.834 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.834 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.834 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:13.834 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247020+0000", 2024-09-21T21:43:13.835 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6156000000000003e-05 2024-09-21T21:43:13.835 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:13.835 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:13.835 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:13.835 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.835 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.835 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19760 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 7991296~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:13.835 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273220+0000", 2024-09-21T21:43:13.835 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 82.526132180000005, 2024-09-21T21:43:13.835 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97403125499999998, 2024-09-21T21:43:13.835 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:13.835 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:13.835 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:13.836 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:13.836 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:13.836 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:13.836 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19760 2024-09-21T21:43:13.836 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.836 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:13.836 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.836 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:13.836 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273220+0000", 2024-09-21T21:43:13.836 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:13.836 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.836 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.836 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:13.837 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273220+0000", 2024-09-21T21:43:13.837 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:13.837 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.837 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.837 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:13.837 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273221+0000", 2024-09-21T21:43:13.837 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.65e-07 2024-09-21T21:43:13.837 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.837 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.837 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:13.837 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273228+0000", 2024-09-21T21:43:13.837 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6040000000000004e-06 2024-09-21T21:43:13.837 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.838 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.838 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:13.838 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273233+0000", 2024-09-21T21:43:13.838 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1569999999999999e-06 2024-09-21T21:43:13.838 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.838 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.838 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:13.838 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273238+0000", 2024-09-21T21:43:13.838 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0050000000000004e-06 2024-09-21T21:43:13.838 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.838 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.838 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:13.838 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278107+0000", 2024-09-21T21:43:13.838 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00486857 2024-09-21T21:43:13.838 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.839 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.839 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:13.839 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278111+0000", 2024-09-21T21:43:13.839 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6389999999999998e-06 2024-09-21T21:43:13.839 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.839 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.839 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:13.839 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247069+0000", 2024-09-21T21:43:13.839 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96895796899999997 2024-09-21T21:43:13.839 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.839 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.839 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:13.839 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247166+0000", 2024-09-21T21:43:13.839 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.7158999999999996e-05 2024-09-21T21:43:13.840 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.840 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.840 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:13.840 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247252+0000", 2024-09-21T21:43:13.840 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5886999999999999e-05 2024-09-21T21:43:13.840 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:13.840 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:13.840 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:13.840 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.840 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.840 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19761 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 7962624~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:13.840 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273246+0000", 2024-09-21T21:43:13.840 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 82.526107120000006, 2024-09-21T21:43:13.840 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97415972200000001, 2024-09-21T21:43:13.840 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:13.841 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:13.841 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:13.841 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:13.841 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:13.841 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:13.841 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19761 2024-09-21T21:43:13.841 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.841 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:13.841 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.841 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:13.841 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273246+0000", 2024-09-21T21:43:13.841 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:13.841 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.841 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.841 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:13.842 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273246+0000", 2024-09-21T21:43:13.842 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:13.842 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.842 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.842 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:13.842 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273246+0000", 2024-09-21T21:43:13.842 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5100000000000001e-07 2024-09-21T21:43:13.842 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.842 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.842 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:13.842 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273248+0000", 2024-09-21T21:43:13.842 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1550000000000001e-06 2024-09-21T21:43:13.842 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.842 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.843 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:13.843 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273254+0000", 2024-09-21T21:43:13.843 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.964e-06 2024-09-21T21:43:13.843 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.843 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.843 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:13.843 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273259+0000", 2024-09-21T21:43:13.843 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5499999999999996e-06 2024-09-21T21:43:13.843 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.843 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.843 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:13.843 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278130+0000", 2024-09-21T21:43:13.843 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048715779999999997 2024-09-21T21:43:13.843 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.843 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.844 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:13.844 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278133+0000", 2024-09-21T21:43:13.844 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8739999999999999e-06 2024-09-21T21:43:13.844 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.844 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.844 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:13.844 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247270+0000", 2024-09-21T21:43:13.844 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.969137099 2024-09-21T21:43:13.844 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.844 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.844 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:13.844 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247338+0000", 2024-09-21T21:43:13.844 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7717000000000002e-05 2024-09-21T21:43:13.844 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.844 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.845 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:13.845 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247405+0000", 2024-09-21T21:43:13.845 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7434000000000005e-05 2024-09-21T21:43:13.845 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:13.845 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:13.845 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:13.845 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.845 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.845 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19769 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 5840896~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:13.845 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273287+0000", 2024-09-21T21:43:13.845 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 82.526065836000001, 2024-09-21T21:43:13.845 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97435390200000005, 2024-09-21T21:43:13.845 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:13.845 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:13.846 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:13.846 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:13.846 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:13.846 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:13.846 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19769 2024-09-21T21:43:13.846 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.846 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:13.846 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.846 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:13.846 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273287+0000", 2024-09-21T21:43:13.846 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:13.846 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.846 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.846 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:13.847 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273287+0000", 2024-09-21T21:43:13.847 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:13.847 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.847 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.847 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:13.847 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273287+0000", 2024-09-21T21:43:13.847 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6699999999999999e-07 2024-09-21T21:43:13.847 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.847 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.847 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:13.847 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273289+0000", 2024-09-21T21:43:13.847 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.708e-06 2024-09-21T21:43:13.847 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.847 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.848 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:13.848 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273296+0000", 2024-09-21T21:43:13.848 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8109999999999997e-06 2024-09-21T21:43:13.848 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.848 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.848 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:13.848 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273299+0000", 2024-09-21T21:43:13.848 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0740000000000001e-06 2024-09-21T21:43:13.848 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.848 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.848 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:13.848 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278159+0000", 2024-09-21T21:43:13.848 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048603240000000001 2024-09-21T21:43:13.848 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.848 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.849 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:13.849 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278162+0000", 2024-09-21T21:43:13.849 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1930000000000002e-06 2024-09-21T21:43:13.849 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.849 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.849 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:13.849 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247444+0000", 2024-09-21T21:43:13.849 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96928199100000001 2024-09-21T21:43:13.849 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.849 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.849 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:13.849 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247533+0000", 2024-09-21T21:43:13.849 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8490000000000001e-05 2024-09-21T21:43:13.849 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.849 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.850 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:13.850 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247641+0000", 2024-09-21T21:43:13.850 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000107944 2024-09-21T21:43:13.850 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:13.850 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:13.850 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:13.850 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.850 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.850 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19770 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 5869568~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:13.851 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273313+0000", 2024-09-21T21:43:13.851 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 82.526039733999994, 2024-09-21T21:43:13.851 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97449038899999996, 2024-09-21T21:43:13.851 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:13.851 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:13.851 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:13.851 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:13.851 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:13.851 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:13.851 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19770 2024-09-21T21:43:13.851 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.851 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:13.851 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.851 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:13.851 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273313+0000", 2024-09-21T21:43:13.852 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:13.852 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.852 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.852 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:13.852 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273313+0000", 2024-09-21T21:43:13.852 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:13.852 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.852 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.852 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:13.852 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273313+0000", 2024-09-21T21:43:13.853 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5699999999999998e-07 2024-09-21T21:43:13.853 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.853 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.853 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:13.853 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273315+0000", 2024-09-21T21:43:13.853 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0099999999999998e-06 2024-09-21T21:43:13.853 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.853 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.853 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:13.853 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273321+0000", 2024-09-21T21:43:13.853 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7529999999999998e-06 2024-09-21T21:43:13.853 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.853 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.853 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:13.854 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273328+0000", 2024-09-21T21:43:13.854 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9009999999999997e-06 2024-09-21T21:43:13.854 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.854 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.854 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:13.854 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278183+0000", 2024-09-21T21:43:13.854 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048547770000000002 2024-09-21T21:43:13.854 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.854 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.854 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:13.855 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278186+0000", 2024-09-21T21:43:13.855 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3100000000000001e-06 2024-09-21T21:43:13.855 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.855 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.855 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:13.855 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247676+0000", 2024-09-21T21:43:13.855 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96949004299999997 2024-09-21T21:43:13.855 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.855 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.855 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:13.855 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247757+0000", 2024-09-21T21:43:13.855 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0479e-05 2024-09-21T21:43:13.855 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.855 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.856 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:13.856 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247803+0000", 2024-09-21T21:43:13.856 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6758999999999997e-05 2024-09-21T21:43:13.856 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:13.856 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:13.856 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:13.856 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.856 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.856 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19771 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6586368~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:13.856 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273335+0000", 2024-09-21T21:43:13.856 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 82.526017947, 2024-09-21T21:43:13.856 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97458947699999998, 2024-09-21T21:43:13.856 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:13.856 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:13.856 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:13.857 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:13.857 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:13.857 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:13.857 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19771 2024-09-21T21:43:13.857 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.857 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:13.857 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.857 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:13.857 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273335+0000", 2024-09-21T21:43:13.857 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:13.857 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.857 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.857 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:13.857 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273335+0000", 2024-09-21T21:43:13.857 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:13.858 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.858 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.858 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:13.858 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273335+0000", 2024-09-21T21:43:13.858 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3599999999999999e-07 2024-09-21T21:43:13.858 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.858 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.858 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:13.858 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273337+0000", 2024-09-21T21:43:13.858 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0789999999999999e-06 2024-09-21T21:43:13.858 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.858 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.858 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:13.859 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273343+0000", 2024-09-21T21:43:13.859 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6280000000000002e-06 2024-09-21T21:43:13.859 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.859 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.859 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:13.859 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273346+0000", 2024-09-21T21:43:13.859 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.507e-06 2024-09-21T21:43:13.859 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.859 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.860 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:13.860 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278206+0000", 2024-09-21T21:43:13.860 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048600969999999999 2024-09-21T21:43:13.860 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.860 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.860 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:13.860 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278212+0000", 2024-09-21T21:43:13.860 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2079999999999999e-06 2024-09-21T21:43:13.860 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.860 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.860 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:13.860 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247828+0000", 2024-09-21T21:43:13.860 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.969616547 2024-09-21T21:43:13.861 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.861 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.861 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:13.861 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247881+0000", 2024-09-21T21:43:13.861 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3180999999999998e-05 2024-09-21T21:43:13.861 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.861 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.861 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:13.861 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247924+0000", 2024-09-21T21:43:13.861 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2894e-05 2024-09-21T21:43:13.861 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:13.861 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:13.862 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:13.862 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.862 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.862 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19772 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6557696~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:13.862 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273358+0000", 2024-09-21T21:43:13.862 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 82.525994566999998, 2024-09-21T21:43:13.862 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97467031800000004, 2024-09-21T21:43:13.862 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:13.862 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:13.862 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:13.862 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:13.862 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:13.862 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:13.863 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19772 2024-09-21T21:43:13.863 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.863 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:13.863 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.863 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:13.863 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273358+0000", 2024-09-21T21:43:13.863 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:13.863 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.863 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.863 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:13.863 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273358+0000", 2024-09-21T21:43:13.863 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:13.863 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.863 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.863 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:13.864 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273358+0000", 2024-09-21T21:43:13.864 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4499999999999998e-07 2024-09-21T21:43:13.864 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.864 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.864 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:13.864 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273360+0000", 2024-09-21T21:43:13.864 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9190000000000002e-06 2024-09-21T21:43:13.864 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.864 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.864 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:13.864 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273366+0000", 2024-09-21T21:43:13.864 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.891e-06 2024-09-21T21:43:13.866 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.866 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.867 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:13.867 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273371+0000", 2024-09-21T21:43:13.867 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1880000000000001e-06 2024-09-21T21:43:13.867 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.867 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.867 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:13.867 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278237+0000", 2024-09-21T21:43:13.867 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.004865472 2024-09-21T21:43:13.867 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.867 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.867 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:13.867 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278240+0000", 2024-09-21T21:43:13.867 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9550000000000001e-06 2024-09-21T21:43:13.867 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.868 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.868 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:13.868 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247949+0000", 2024-09-21T21:43:13.868 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.969709127 2024-09-21T21:43:13.868 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.868 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.868 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:13.868 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247990+0000", 2024-09-21T21:43:13.868 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1202000000000003e-05 2024-09-21T21:43:13.868 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.868 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.868 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:13.868 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248028+0000", 2024-09-21T21:43:13.868 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8219000000000001e-05 2024-09-21T21:43:13.868 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:13.869 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:13.869 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:13.869 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.869 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.869 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19773 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6615040~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:13.869 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273390+0000", 2024-09-21T21:43:13.869 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 82.525962706000001, 2024-09-21T21:43:13.869 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97476086200000001, 2024-09-21T21:43:13.869 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:13.869 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:13.869 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:13.869 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:13.869 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:13.869 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:13.870 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19773 2024-09-21T21:43:13.870 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.870 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:13.870 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.870 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:13.870 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273390+0000", 2024-09-21T21:43:13.870 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:13.870 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.870 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.870 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:13.870 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273390+0000", 2024-09-21T21:43:13.870 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:13.870 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.871 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.871 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:13.871 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273390+0000", 2024-09-21T21:43:13.871 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9999999999999999e-07 2024-09-21T21:43:13.871 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.871 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.871 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:13.871 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273392+0000", 2024-09-21T21:43:13.871 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0640000000000001e-06 2024-09-21T21:43:13.871 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.871 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.871 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:13.871 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273397+0000", 2024-09-21T21:43:13.871 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.113e-06 2024-09-21T21:43:13.872 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.872 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.872 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:13.872 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273401+0000", 2024-09-21T21:43:13.872 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3119999999999998e-06 2024-09-21T21:43:13.872 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.872 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.872 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:13.872 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278271+0000", 2024-09-21T21:43:13.872 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048698880000000002 2024-09-21T21:43:13.872 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.872 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.872 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:13.872 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278278+0000", 2024-09-21T21:43:13.872 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1210000000000001e-06 2024-09-21T21:43:13.872 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.873 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.873 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:13.873 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248060+0000", 2024-09-21T21:43:13.873 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96978224999999996 2024-09-21T21:43:13.873 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.873 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.873 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:13.873 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248111+0000", 2024-09-21T21:43:13.873 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.096e-05 2024-09-21T21:43:13.873 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.873 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.873 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:13.873 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248151+0000", 2024-09-21T21:43:13.873 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9854000000000002e-05 2024-09-21T21:43:13.873 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:13.874 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:13.874 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:13.874 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.874 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.874 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19784 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6643712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:13.874 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273405+0000", 2024-09-21T21:43:13.874 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 82.525947985000002, 2024-09-21T21:43:13.874 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97491798900000004, 2024-09-21T21:43:13.874 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:13.874 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:13.874 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:13.874 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:13.874 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:13.874 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:13.874 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19784 2024-09-21T21:43:13.875 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.875 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:13.875 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.875 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:13.875 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273405+0000", 2024-09-21T21:43:13.875 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:13.875 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.875 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.875 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:13.875 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273405+0000", 2024-09-21T21:43:13.875 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:13.875 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.875 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.875 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:13.875 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273405+0000", 2024-09-21T21:43:13.876 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2100000000000001e-07 2024-09-21T21:43:13.876 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.876 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.876 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:13.876 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273406+0000", 2024-09-21T21:43:13.876 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2049999999999999e-06 2024-09-21T21:43:13.876 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.876 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.876 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:13.876 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273409+0000", 2024-09-21T21:43:13.876 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0599999999999999e-06 2024-09-21T21:43:13.876 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.876 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.877 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:13.877 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273412+0000", 2024-09-21T21:43:13.877 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0730000000000001e-06 2024-09-21T21:43:13.877 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.877 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.877 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:13.877 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278317+0000", 2024-09-21T21:43:13.877 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0049045520000000004 2024-09-21T21:43:13.877 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.877 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.877 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:13.877 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278321+0000", 2024-09-21T21:43:13.877 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6249999999999998e-06 2024-09-21T21:43:13.877 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.877 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.877 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:13.878 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248179+0000", 2024-09-21T21:43:13.878 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96985744500000004 2024-09-21T21:43:13.878 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.878 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.878 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:13.878 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248253+0000", 2024-09-21T21:43:13.878 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3829000000000001e-05 2024-09-21T21:43:13.878 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.878 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.878 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:13.879 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248323+0000", 2024-09-21T21:43:13.879 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9979000000000002e-05 2024-09-21T21:43:13.879 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:13.879 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:13.879 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:13.879 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.879 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.879 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19785 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6672384~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:13.879 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273415+0000", 2024-09-21T21:43:13.879 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 82.525937399, 2024-09-21T21:43:13.880 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97500777800000005, 2024-09-21T21:43:13.880 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:13.880 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:13.880 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:13.880 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:13.880 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:13.880 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:13.880 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19785 2024-09-21T21:43:13.880 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.880 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:13.880 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.880 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:13.880 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273415+0000", 2024-09-21T21:43:13.880 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:13.880 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.880 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.881 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:13.881 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273415+0000", 2024-09-21T21:43:13.881 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:13.881 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.881 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.881 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:13.881 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273415+0000", 2024-09-21T21:43:13.881 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0100000000000001e-07 2024-09-21T21:43:13.881 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.881 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.881 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:13.881 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273417+0000", 2024-09-21T21:43:13.881 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.114e-06 2024-09-21T21:43:13.881 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.881 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.882 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:13.882 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273420+0000", 2024-09-21T21:43:13.882 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6770000000000001e-06 2024-09-21T21:43:13.882 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.882 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.882 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:13.882 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273423+0000", 2024-09-21T21:43:13.882 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5450000000000002e-06 2024-09-21T21:43:13.882 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.882 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.882 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:13.882 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278355+0000", 2024-09-21T21:43:13.882 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0049318590000000002 2024-09-21T21:43:13.882 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.882 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.883 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:13.883 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278360+0000", 2024-09-21T21:43:13.883 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3079999999999998e-06 2024-09-21T21:43:13.883 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.883 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.883 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:13.883 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248349+0000", 2024-09-21T21:43:13.883 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96998909899999997 2024-09-21T21:43:13.883 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.883 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.883 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:13.883 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248387+0000", 2024-09-21T21:43:13.883 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7657000000000002e-05 2024-09-21T21:43:13.883 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.883 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.884 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:13.884 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248423+0000", 2024-09-21T21:43:13.884 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6318e-05 2024-09-21T21:43:13.884 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:13.884 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:13.884 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:13.884 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.884 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.884 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19842 3.d 3:b7fa2ca0:::rbd_data.112157a479c.000000000000004d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:13.884 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273426+0000", 2024-09-21T21:43:13.884 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 82.525926346999995, 2024-09-21T21:43:13.884 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.98136526800000001, 2024-09-21T21:43:13.884 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:13.884 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:13.884 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:13.885 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:13.885 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:13.885 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:13.885 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19842 2024-09-21T21:43:13.885 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.885 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:13.885 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.885 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:13.885 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273426+0000", 2024-09-21T21:43:13.885 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:13.885 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.885 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.885 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:13.885 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273426+0000", 2024-09-21T21:43:13.885 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:13.886 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.886 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.886 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:13.886 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273427+0000", 2024-09-21T21:43:13.886 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.72e-07 2024-09-21T21:43:13.886 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.886 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.886 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:13.886 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275836+0000", 2024-09-21T21:43:13.886 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00240909 2024-09-21T21:43:13.886 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.886 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.886 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:13.886 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275844+0000", 2024-09-21T21:43:13.887 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3750000000000003e-06 2024-09-21T21:43:13.887 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.887 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.887 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:13.887 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275849+0000", 2024-09-21T21:43:13.887 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0100000000000003e-06 2024-09-21T21:43:13.887 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.887 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.887 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:13.887 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278394+0000", 2024-09-21T21:43:13.887 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025449890000000001 2024-09-21T21:43:13.887 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.887 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.887 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:13.887 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278402+0000", 2024-09-21T21:43:13.888 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6359999999999994e-06 2024-09-21T21:43:13.888 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.888 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.888 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:13.888 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248454+0000", 2024-09-21T21:43:13.888 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97005259499999996 2024-09-21T21:43:13.888 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.888 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.888 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:13.888 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248576+0000", 2024-09-21T21:43:13.888 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000122127 2024-09-21T21:43:13.888 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.888 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.888 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 2", 2024-09-21T21:43:13.888 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248844+0000", 2024-09-21T21:43:13.889 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00026784699999999999 2024-09-21T21:43:13.889 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.889 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.889 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:13.889 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.252487+0000", 2024-09-21T21:43:13.889 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0036427920000000002 2024-09-21T21:43:13.889 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.889 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.889 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:13.889 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.254691+0000", 2024-09-21T21:43:13.889 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002203899 2024-09-21T21:43:13.889 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.889 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.889 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:13.889 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.254748+0000", 2024-09-21T21:43:13.889 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7163000000000001e-05 2024-09-21T21:43:13.890 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.890 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.890 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:13.890 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.254792+0000", 2024-09-21T21:43:13.890 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3473000000000002e-05 2024-09-21T21:43:13.890 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:13.890 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:13.890 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:13.890 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.890 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.890 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19843 3.d 3:b7fa2ca0:::rbd_data.112157a479c.000000000000004d:head [write 0~0] snapc 21=[21] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:13.890 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.275860+0000", 2024-09-21T21:43:13.890 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 82.523492446999995, 2024-09-21T21:43:13.891 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.98183651999999999, 2024-09-21T21:43:13.891 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:13.891 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:13.891 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:13.891 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:13.891 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:13.891 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:13.891 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19843 2024-09-21T21:43:13.891 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.891 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:13.891 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.891 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:13.891 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275860+0000", 2024-09-21T21:43:13.892 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:13.892 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.892 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.892 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:13.892 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275860+0000", 2024-09-21T21:43:13.892 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:13.892 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.892 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.892 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:13.892 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275869+0000", 2024-09-21T21:43:13.892 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6300000000000004e-06 2024-09-21T21:43:13.892 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.892 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.892 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:13.893 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275871+0000", 2024-09-21T21:43:13.893 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2060000000000001e-06 2024-09-21T21:43:13.893 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.893 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.893 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:13.893 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275876+0000", 2024-09-21T21:43:13.893 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8180000000000003e-06 2024-09-21T21:43:13.893 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.893 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.893 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:13.893 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275880+0000", 2024-09-21T21:43:13.893 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1829999999999996e-06 2024-09-21T21:43:13.893 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.893 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.894 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:13.894 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278416+0000", 2024-09-21T21:43:13.894 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025361149999999998 2024-09-21T21:43:13.894 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.894 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.894 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:13.894 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278419+0000", 2024-09-21T21:43:13.894 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.163e-06 2024-09-21T21:43:13.894 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.894 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.894 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:13.894 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.250249+0000", 2024-09-21T21:43:13.894 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97182984299999997 2024-09-21T21:43:13.894 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.895 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.895 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:13.895 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.250321+0000", 2024-09-21T21:43:13.895 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1656000000000006e-05 2024-09-21T21:43:13.895 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.895 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.895 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 2", 2024-09-21T21:43:13.895 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.250419+0000", 2024-09-21T21:43:13.895 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.8277999999999997e-05 2024-09-21T21:43:13.895 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.895 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.895 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:13.895 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.252516+0000", 2024-09-21T21:43:13.895 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002097224 2024-09-21T21:43:13.896 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.896 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.896 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:13.896 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.257643+0000", 2024-09-21T21:43:13.896 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0051262319999999997 2024-09-21T21:43:13.896 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.896 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.896 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:13.896 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.257673+0000", 2024-09-21T21:43:13.896 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0023e-05 2024-09-21T21:43:13.896 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.896 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.896 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:13.896 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.257697+0000", 2024-09-21T21:43:13.897 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4148999999999999e-05 2024-09-21T21:43:13.897 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:13.897 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:13.897 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:13.897 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.897 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.897 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5626.0:12334 3.3 3:c643a86c:::rbd_data.1121faebdfe3.0000000000000002:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:43:13.897 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:14.411628+0000", 2024-09-21T21:43:13.897 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 59.387724147999997, 2024-09-21T21:43:13.897 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0086947559999999, 2024-09-21T21:43:13.897 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:13.897 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:13.897 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:13.897 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:13.897 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5626", 2024-09-21T21:43:13.898 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:43:13.898 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 12334 2024-09-21T21:43:13.898 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.898 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:13.898 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.898 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:13.898 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411628+0000", 2024-09-21T21:43:13.898 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:13.898 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.898 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.898 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:13.898 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411628+0000", 2024-09-21T21:43:13.898 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:13.898 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.899 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.899 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:13.899 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411629+0000", 2024-09-21T21:43:13.899 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1200000000000002e-07 2024-09-21T21:43:13.899 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.899 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.899 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:13.899 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411633+0000", 2024-09-21T21:43:13.899 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1699999999999999e-06 2024-09-21T21:43:13.899 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.899 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.899 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:13.899 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411645+0000", 2024-09-21T21:43:13.899 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1328999999999999e-05 2024-09-21T21:43:13.900 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.900 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.900 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:13.900 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411652+0000", 2024-09-21T21:43:13.900 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9990000000000002e-06 2024-09-21T21:43:13.900 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.900 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.900 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:13.900 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411717+0000", 2024-09-21T21:43:13.900 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5519999999999996e-05 2024-09-21T21:43:13.900 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.900 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.900 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op must wait for map", 2024-09-21T21:43:13.900 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411725+0000", 2024-09-21T21:43:13.901 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2609999999999996e-06 2024-09-21T21:43:13.901 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.901 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.901 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:13.901 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415904+0000", 2024-09-21T21:43:13.901 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0041792749999999997 2024-09-21T21:43:13.901 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.901 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.901 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:13.901 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415913+0000", 2024-09-21T21:43:13.901 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.054e-06 2024-09-21T21:43:13.901 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.901 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.901 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:13.901 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420096+0000", 2024-09-21T21:43:13.902 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0041835670000001 2024-09-21T21:43:13.902 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.902 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.902 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:13.902 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420216+0000", 2024-09-21T21:43:13.902 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000120025 2024-09-21T21:43:13.902 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.902 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.902 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:13.902 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420323+0000", 2024-09-21T21:43:13.902 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010674400000000001 2024-09-21T21:43:13.902 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:13.902 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:13.903 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:13.903 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.903 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.903 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5626.0:12339 3.3 3:c643a86c:::rbd_data.1121faebdfe3.0000000000000002:head [read 4096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:43:13.903 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:14.411749+0000", 2024-09-21T21:43:13.903 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 59.387603962, 2024-09-21T21:43:13.903 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.008818056, 2024-09-21T21:43:13.903 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:13.903 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:13.903 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:13.903 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:13.903 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5626", 2024-09-21T21:43:13.903 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:43:13.903 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 12339 2024-09-21T21:43:13.904 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.904 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:13.904 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.904 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:13.904 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411749+0000", 2024-09-21T21:43:13.904 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:13.904 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.904 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.904 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:13.904 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411749+0000", 2024-09-21T21:43:13.904 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:13.904 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.904 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.904 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:13.904 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411749+0000", 2024-09-21T21:43:13.905 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0900000000000002e-07 2024-09-21T21:43:13.905 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.905 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.905 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:13.905 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411752+0000", 2024-09-21T21:43:13.905 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5610000000000001e-06 2024-09-21T21:43:13.905 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.905 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.905 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:13.905 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411758+0000", 2024-09-21T21:43:13.905 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1890000000000004e-06 2024-09-21T21:43:13.905 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.905 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.905 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:13.906 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411763+0000", 2024-09-21T21:43:13.906 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7589999999999997e-06 2024-09-21T21:43:13.906 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.906 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.906 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:13.906 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411803+0000", 2024-09-21T21:43:13.906 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9922999999999997e-05 2024-09-21T21:43:13.906 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.906 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.906 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting_for_map not empty", 2024-09-21T21:43:13.906 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411809+0000", 2024-09-21T21:43:13.906 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3779999999999998e-06 2024-09-21T21:43:13.906 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.906 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.907 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:13.907 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415937+0000", 2024-09-21T21:43:13.907 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0041276100000000003 2024-09-21T21:43:13.907 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.907 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.907 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:13.907 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415944+0000", 2024-09-21T21:43:13.907 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8329999999999994e-06 2024-09-21T21:43:13.907 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.907 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.907 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:13.907 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420411+0000", 2024-09-21T21:43:13.907 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0044667300000001 2024-09-21T21:43:13.907 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.908 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.908 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:13.908 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420475+0000", 2024-09-21T21:43:13.908 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4356000000000005e-05 2024-09-21T21:43:13.908 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.908 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.908 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:13.908 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420567+0000", 2024-09-21T21:43:13.908 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.1308000000000001e-05 2024-09-21T21:43:13.908 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:13.908 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:13.908 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:13.908 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.908 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.909 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5626.0:12345 3.3 3:c643a86c:::rbd_data.1121faebdfe3.0000000000000002:head [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:43:13.909 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:14.411831+0000", 2024-09-21T21:43:13.909 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 59.387521909, 2024-09-21T21:43:13.909 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0089163649999999, 2024-09-21T21:43:13.909 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:13.909 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:13.909 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:13.909 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:13.909 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5626", 2024-09-21T21:43:13.909 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:43:13.909 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 12345 2024-09-21T21:43:13.909 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.909 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:13.909 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.910 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:13.910 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411831+0000", 2024-09-21T21:43:13.910 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:13.910 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.910 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.910 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:13.910 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411831+0000", 2024-09-21T21:43:13.910 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:13.910 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.910 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.910 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:13.910 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411831+0000", 2024-09-21T21:43:13.910 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.08e-07 2024-09-21T21:43:13.910 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.911 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.911 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:13.911 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411833+0000", 2024-09-21T21:43:13.911 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3089999999999998e-06 2024-09-21T21:43:13.911 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.911 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.911 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:13.911 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411840+0000", 2024-09-21T21:43:13.911 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.472e-06 2024-09-21T21:43:13.911 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.911 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.911 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:13.911 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411845+0000", 2024-09-21T21:43:13.911 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7910000000000004e-06 2024-09-21T21:43:13.911 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.912 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.912 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:13.912 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411890+0000", 2024-09-21T21:43:13.912 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5763000000000002e-05 2024-09-21T21:43:13.912 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.912 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.912 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting_for_map not empty", 2024-09-21T21:43:13.912 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411896+0000", 2024-09-21T21:43:13.912 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0190000000000003e-06 2024-09-21T21:43:13.912 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.912 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.912 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:13.912 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415967+0000", 2024-09-21T21:43:13.912 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0040705749999999999 2024-09-21T21:43:13.913 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.913 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.913 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:13.913 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415974+0000", 2024-09-21T21:43:13.913 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3200000000000002e-06 2024-09-21T21:43:13.913 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.913 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.913 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:13.913 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420594+0000", 2024-09-21T21:43:13.913 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.004620093 2024-09-21T21:43:13.913 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.913 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.913 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:13.913 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420658+0000", 2024-09-21T21:43:13.914 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3678999999999998e-05 2024-09-21T21:43:13.914 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:13.914 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:13.914 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:13.914 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420747+0000", 2024-09-21T21:43:13.914 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8936000000000004e-05 2024-09-21T21:43:13.914 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:13.914 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:13.914 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:13.914 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:13.914 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:13.914 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:43:13.915 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0: {} 2024-09-21T21:43:13.916 INFO:teuthology.orchestra.run.smithi081.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T21:43:13.916 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1: {} 2024-09-21T21:43:13.917 INFO:teuthology.orchestra.run.smithi081.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T21:43:13.917 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T21:43:13.917 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:43:13.914+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:43:13.918 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2: {} 2024-09-21T21:43:13.918 INFO:teuthology.orchestra.run.smithi081.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T21:43:13.918 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3: {} 2024-09-21T21:43:13.918 INFO:teuthology.orchestra.run.smithi081.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T21:43:13.919 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4: {} 2024-09-21T21:43:13.919 INFO:teuthology.orchestra.run.smithi081.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T21:43:13.919 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5: {} 2024-09-21T21:43:13.919 INFO:teuthology.orchestra.run.smithi081.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T21:43:14.016 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:43:14.015+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:43:14.060 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:14.060 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:14.068 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T21:43:14.152 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:43:14.151+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:43:14.212 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:14.213 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:14.219 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:43:14.217+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:43:14.221 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T21:43:14.303 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T21:43:14.306 INFO:teuthology.orchestra.run.smithi081.stderr:marked in osd.4. 2024-09-21T21:43:14.319 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:43:14.318+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:43:14.326 INFO:tasks.thrashosds.thrasher:Added osd 4 2024-09-21T21:43:14.370 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:14.370 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:14.378 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T21:43:14.421 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:43:14.419+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:43:14.521 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:43:14.520+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:43:14.522 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:14.522 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:14.531 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T21:43:14.622 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:43:14.621+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:43:14.674 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:14.675 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:14.683 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T21:43:14.726 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:43:14.725+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:43:14.785 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:14.785 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:14.793 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T21:43:14.834 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:43:14.833+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:43:14.896 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:14.896 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:14.905 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T21:43:14.926 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:43:14.925+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:43:15.007 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:15.007 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:15.016 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T21:43:15.027 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:43:15.026+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:43:15.119 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:15.119 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:15.129 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T21:43:15.129 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:43:15.127+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:43:15.229 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:43:15.228+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:43:15.271 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:15.272 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:15.280 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T21:43:15.314 INFO:teuthology.orchestra.run.smithi081.stderr:nodeep-scrub is set 2024-09-21T21:43:15.330 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:43:15.328+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:43:15.389 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:15.389 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:15.397 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T21:43:15.440 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:43:15.439+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:43:15.499 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:15.499 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:15.508 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T21:43:15.550 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:43:15.548+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:43:15.632 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:43:15.631+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:43:15.652 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:15.652 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:15.660 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T21:43:15.744 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:43:15.743+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:43:15.803 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:15.804 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:15.813 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T21:43:15.853 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:43:15.852+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:43:15.928 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T21:43:15.935 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:43:15.934+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:43:15.956 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:15.956 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:15.965 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T21:43:16.035 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:43:16.034+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:43:16.047 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0: {} 2024-09-21T21:43:16.047 INFO:teuthology.orchestra.run.smithi081.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T21:43:16.064 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1: {} 2024-09-21T21:43:16.064 INFO:teuthology.orchestra.run.smithi081.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T21:43:16.080 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2: {} 2024-09-21T21:43:16.080 INFO:teuthology.orchestra.run.smithi081.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T21:43:16.085 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:16.085 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:16.095 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T21:43:16.096 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3: {} 2024-09-21T21:43:16.096 INFO:teuthology.orchestra.run.smithi081.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T21:43:16.113 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4: {} 2024-09-21T21:43:16.113 INFO:teuthology.orchestra.run.smithi081.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T21:43:16.130 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5: {} 2024-09-21T21:43:16.130 INFO:teuthology.orchestra.run.smithi081.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T21:43:16.138 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:43:16.137+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:43:16.201 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:43:16.201 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:43:16.201 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:43:16.202 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:43:16.202 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:43:16.210 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T21:43:16.237 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:43:16.236+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:43:16.338 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:43:16.337+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:43:16.362 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:43:16.362 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-21T21:43:16.362 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-21T21:43:16.362 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:43:16.362 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.362 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19747 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 7905280~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:16.362 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273058+0000", 2024-09-21T21:43:16.362 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 85.086270518999996, 2024-09-21T21:43:16.362 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97338591399999996, 2024-09-21T21:43:16.362 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:16.362 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:16.363 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:16.363 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:16.363 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:16.363 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:16.363 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19747 2024-09-21T21:43:16.363 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.363 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:16.363 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.363 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:16.363 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273058+0000", 2024-09-21T21:43:16.363 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:16.363 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.363 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.363 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:16.364 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273058+0000", 2024-09-21T21:43:16.364 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:16.364 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.364 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.364 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:16.364 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273058+0000", 2024-09-21T21:43:16.364 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5900000000000003e-07 2024-09-21T21:43:16.364 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.364 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.364 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:16.364 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273061+0000", 2024-09-21T21:43:16.364 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3479999999999998e-06 2024-09-21T21:43:16.364 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.364 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.365 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:16.365 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273067+0000", 2024-09-21T21:43:16.365 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1469999999999998e-06 2024-09-21T21:43:16.365 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.365 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.365 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:16.365 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273071+0000", 2024-09-21T21:43:16.365 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7339999999999999e-06 2024-09-21T21:43:16.365 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.365 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.365 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:16.365 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277945+0000", 2024-09-21T21:43:16.365 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.004873474 2024-09-21T21:43:16.365 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.365 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.366 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:16.366 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277948+0000", 2024-09-21T21:43:16.366 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0299999999999998e-06 2024-09-21T21:43:16.366 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.366 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.366 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:16.366 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246269+0000", 2024-09-21T21:43:16.366 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96832060200000003 2024-09-21T21:43:16.366 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.366 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.367 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:16.367 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246362+0000", 2024-09-21T21:43:16.367 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3867000000000001e-05 2024-09-21T21:43:16.367 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.367 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.367 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:16.367 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246444+0000", 2024-09-21T21:43:16.367 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1353000000000005e-05 2024-09-21T21:43:16.367 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:16.367 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:16.367 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:16.367 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.367 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.367 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19755 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 5783552~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:16.368 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273077+0000", 2024-09-21T21:43:16.368 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 85.086250903000007, 2024-09-21T21:43:16.368 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97350203400000002, 2024-09-21T21:43:16.368 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:16.368 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:16.368 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:16.368 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:16.368 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:16.368 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:16.368 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19755 2024-09-21T21:43:16.368 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.368 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:16.369 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.369 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:16.369 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273077+0000", 2024-09-21T21:43:16.369 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:16.369 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.369 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.369 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:16.369 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273077+0000", 2024-09-21T21:43:16.369 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:16.369 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.369 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.370 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:16.370 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273078+0000", 2024-09-21T21:43:16.370 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5699999999999998e-07 2024-09-21T21:43:16.370 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.370 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.370 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:16.370 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273083+0000", 2024-09-21T21:43:16.370 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8670000000000002e-06 2024-09-21T21:43:16.370 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.370 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.370 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:16.370 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273088+0000", 2024-09-21T21:43:16.370 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8370000000000002e-06 2024-09-21T21:43:16.371 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.371 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.371 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:16.371 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273093+0000", 2024-09-21T21:43:16.371 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5709999999999999e-06 2024-09-21T21:43:16.371 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.371 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.371 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:16.371 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277968+0000", 2024-09-21T21:43:16.371 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048753909999999998 2024-09-21T21:43:16.371 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.371 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.371 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:16.371 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277971+0000", 2024-09-21T21:43:16.372 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0189999999999998e-06 2024-09-21T21:43:16.372 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.372 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.372 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:16.372 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246485+0000", 2024-09-21T21:43:16.372 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96851399299999996 2024-09-21T21:43:16.372 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.372 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.372 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:16.372 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246536+0000", 2024-09-21T21:43:16.372 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0430000000000003e-05 2024-09-21T21:43:16.373 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.373 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.373 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:16.373 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246579+0000", 2024-09-21T21:43:16.373 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3569000000000002e-05 2024-09-21T21:43:16.373 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:16.373 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:16.373 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:16.373 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.373 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.373 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19756 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6500352~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:16.373 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273099+0000", 2024-09-21T21:43:16.373 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 85.086229529999997, 2024-09-21T21:43:16.373 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97359272799999996, 2024-09-21T21:43:16.374 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:16.374 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:16.374 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:16.374 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:16.374 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:16.374 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:16.374 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19756 2024-09-21T21:43:16.374 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.374 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:16.374 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.374 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:16.374 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273099+0000", 2024-09-21T21:43:16.375 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:16.375 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.375 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.375 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:16.375 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273099+0000", 2024-09-21T21:43:16.375 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:16.375 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.375 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.375 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:16.375 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273099+0000", 2024-09-21T21:43:16.375 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0900000000000002e-07 2024-09-21T21:43:16.375 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.375 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.375 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:16.375 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273101+0000", 2024-09-21T21:43:16.376 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0320000000000002e-06 2024-09-21T21:43:16.376 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.376 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.376 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:16.376 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273107+0000", 2024-09-21T21:43:16.376 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2830000000000001e-06 2024-09-21T21:43:16.376 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.376 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.376 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:16.376 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273111+0000", 2024-09-21T21:43:16.376 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2230000000000001e-06 2024-09-21T21:43:16.376 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.376 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.377 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:16.377 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277994+0000", 2024-09-21T21:43:16.377 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048837070000000002 2024-09-21T21:43:16.377 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.377 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.377 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:16.377 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278000+0000", 2024-09-21T21:43:16.377 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3149999999999999e-06 2024-09-21T21:43:16.377 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.377 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.377 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:16.377 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246607+0000", 2024-09-21T21:43:16.377 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.968606839 2024-09-21T21:43:16.377 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.377 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.378 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:16.378 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246651+0000", 2024-09-21T21:43:16.378 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4734999999999998e-05 2024-09-21T21:43:16.378 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.378 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.378 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:16.378 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246692+0000", 2024-09-21T21:43:16.378 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0185000000000002e-05 2024-09-21T21:43:16.378 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:16.378 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:16.378 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:16.378 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.378 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.379 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19757 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 5812224~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:16.379 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273116+0000", 2024-09-21T21:43:16.379 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 85.086211872999996, 2024-09-21T21:43:16.379 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97367813999999997, 2024-09-21T21:43:16.379 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:16.379 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:16.379 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:16.379 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:16.379 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:16.379 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:16.379 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19757 2024-09-21T21:43:16.379 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.379 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:16.379 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.379 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:16.380 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273116+0000", 2024-09-21T21:43:16.380 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:16.380 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.380 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.380 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:16.380 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273116+0000", 2024-09-21T21:43:16.380 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:16.380 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.380 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.380 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:16.380 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273117+0000", 2024-09-21T21:43:16.380 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0999999999999999e-07 2024-09-21T21:43:16.381 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.381 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.381 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:16.381 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273119+0000", 2024-09-21T21:43:16.381 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9970000000000001e-06 2024-09-21T21:43:16.381 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.381 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.381 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:16.381 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273125+0000", 2024-09-21T21:43:16.381 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9540000000000001e-06 2024-09-21T21:43:16.381 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.381 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.381 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:16.381 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273135+0000", 2024-09-21T21:43:16.382 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0597999999999999e-05 2024-09-21T21:43:16.382 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.382 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.382 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:16.382 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278019+0000", 2024-09-21T21:43:16.382 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048834849999999999 2024-09-21T21:43:16.382 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.382 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.382 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:16.382 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278022+0000", 2024-09-21T21:43:16.382 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9560000000000002e-06 2024-09-21T21:43:16.382 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.382 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.383 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:16.383 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246716+0000", 2024-09-21T21:43:16.383 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.968694202 2024-09-21T21:43:16.383 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.383 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.383 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:16.383 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246758+0000", 2024-09-21T21:43:16.383 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1607000000000003e-05 2024-09-21T21:43:16.383 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.383 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.383 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:16.383 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246795+0000", 2024-09-21T21:43:16.383 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6931000000000003e-05 2024-09-21T21:43:16.383 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:16.384 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:16.384 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:16.384 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.384 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.384 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19758 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6529024~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:16.384 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273142+0000", 2024-09-21T21:43:16.384 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 85.086186651999995, 2024-09-21T21:43:16.384 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97375924599999997, 2024-09-21T21:43:16.384 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:16.384 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:16.384 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:16.384 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:16.384 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:16.384 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:16.385 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19758 2024-09-21T21:43:16.385 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.385 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:16.385 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.385 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:16.385 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273142+0000", 2024-09-21T21:43:16.385 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:16.385 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.385 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.385 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:16.386 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273142+0000", 2024-09-21T21:43:16.386 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:16.386 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.386 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.386 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:16.386 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273142+0000", 2024-09-21T21:43:16.386 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0200000000000003e-07 2024-09-21T21:43:16.386 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.386 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.386 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:16.386 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273144+0000", 2024-09-21T21:43:16.386 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1390000000000002e-06 2024-09-21T21:43:16.387 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.387 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.387 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:16.387 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273150+0000", 2024-09-21T21:43:16.387 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6409999999999999e-06 2024-09-21T21:43:16.387 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.387 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.387 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:16.387 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273157+0000", 2024-09-21T21:43:16.387 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1359999999999999e-06 2024-09-21T21:43:16.387 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.388 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.388 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:16.388 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278050+0000", 2024-09-21T21:43:16.388 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048926170000000001 2024-09-21T21:43:16.388 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.388 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.388 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:16.388 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278053+0000", 2024-09-21T21:43:16.388 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4879999999999999e-06 2024-09-21T21:43:16.389 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.389 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.389 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:16.389 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246819+0000", 2024-09-21T21:43:16.389 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96876590399999996 2024-09-21T21:43:16.389 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.389 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.389 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:16.389 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246856+0000", 2024-09-21T21:43:16.389 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7287999999999999e-05 2024-09-21T21:43:16.389 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.389 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.389 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:16.390 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246901+0000", 2024-09-21T21:43:16.390 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4631e-05 2024-09-21T21:43:16.390 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:16.390 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:16.390 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:16.390 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.390 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.390 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19759 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 7933952~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:16.390 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273172+0000", 2024-09-21T21:43:16.390 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 85.086156388000006, 2024-09-21T21:43:16.390 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97384789500000002, 2024-09-21T21:43:16.391 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:16.391 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:16.391 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:16.391 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:16.391 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:16.391 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:16.391 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19759 2024-09-21T21:43:16.391 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.391 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:16.391 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.391 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:16.392 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273172+0000", 2024-09-21T21:43:16.392 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:16.392 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.392 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.392 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:16.392 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273172+0000", 2024-09-21T21:43:16.392 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:16.392 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.392 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.392 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:16.392 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273172+0000", 2024-09-21T21:43:16.393 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6100000000000002e-07 2024-09-21T21:43:16.393 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.393 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.393 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:16.393 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273174+0000", 2024-09-21T21:43:16.393 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.691e-06 2024-09-21T21:43:16.393 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.393 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.393 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:16.393 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273178+0000", 2024-09-21T21:43:16.393 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7579999999999998e-06 2024-09-21T21:43:16.394 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.394 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.394 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:16.394 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273180+0000", 2024-09-21T21:43:16.394 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7199999999999998e-06 2024-09-21T21:43:16.394 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.394 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.394 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:16.394 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278077+0000", 2024-09-21T21:43:16.394 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048969529999999999 2024-09-21T21:43:16.395 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.395 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.395 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:16.395 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278080+0000", 2024-09-21T21:43:16.395 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.066e-06 2024-09-21T21:43:16.395 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.395 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.395 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:16.395 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246925+0000", 2024-09-21T21:43:16.395 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96884490199999995 2024-09-21T21:43:16.395 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.395 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.396 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:16.396 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246974+0000", 2024-09-21T21:43:16.396 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8387999999999998e-05 2024-09-21T21:43:16.396 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.396 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.396 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:16.396 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247020+0000", 2024-09-21T21:43:16.396 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6156000000000003e-05 2024-09-21T21:43:16.396 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:16.396 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:16.396 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:16.396 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.396 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.396 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19760 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 7991296~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:16.397 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273220+0000", 2024-09-21T21:43:16.397 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 85.086107874000007, 2024-09-21T21:43:16.397 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97403125499999998, 2024-09-21T21:43:16.397 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:16.397 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:16.397 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:16.397 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:16.397 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:16.397 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:16.397 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19760 2024-09-21T21:43:16.397 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.397 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:16.397 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.398 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:16.398 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273220+0000", 2024-09-21T21:43:16.398 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:16.398 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.398 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.398 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:16.398 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273220+0000", 2024-09-21T21:43:16.398 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:16.398 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.398 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.398 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:16.398 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273221+0000", 2024-09-21T21:43:16.398 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.65e-07 2024-09-21T21:43:16.398 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.399 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.401 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:16.401 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273228+0000", 2024-09-21T21:43:16.401 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6040000000000004e-06 2024-09-21T21:43:16.401 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.402 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.402 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:16.402 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273233+0000", 2024-09-21T21:43:16.402 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1569999999999999e-06 2024-09-21T21:43:16.402 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.402 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.402 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:16.402 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273238+0000", 2024-09-21T21:43:16.402 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0050000000000004e-06 2024-09-21T21:43:16.402 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.402 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.402 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:16.403 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278107+0000", 2024-09-21T21:43:16.403 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00486857 2024-09-21T21:43:16.403 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.403 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.403 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:16.403 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278111+0000", 2024-09-21T21:43:16.403 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6389999999999998e-06 2024-09-21T21:43:16.403 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.403 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.403 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:16.403 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247069+0000", 2024-09-21T21:43:16.403 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96895796899999997 2024-09-21T21:43:16.403 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.404 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.404 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:16.404 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247166+0000", 2024-09-21T21:43:16.404 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.7158999999999996e-05 2024-09-21T21:43:16.404 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.404 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.404 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:16.404 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247252+0000", 2024-09-21T21:43:16.404 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5886999999999999e-05 2024-09-21T21:43:16.404 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:16.404 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:16.404 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:16.405 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.405 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.405 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19761 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 7962624~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:16.405 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273246+0000", 2024-09-21T21:43:16.405 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 85.086082813999994, 2024-09-21T21:43:16.405 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97415972200000001, 2024-09-21T21:43:16.405 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:16.405 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:16.405 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:16.405 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:16.405 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:16.405 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:16.406 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19761 2024-09-21T21:43:16.406 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.406 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:16.406 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.406 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:16.406 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273246+0000", 2024-09-21T21:43:16.406 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:16.406 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.406 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.406 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:16.406 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273246+0000", 2024-09-21T21:43:16.406 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:16.406 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.407 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.407 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:16.407 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273246+0000", 2024-09-21T21:43:16.407 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5100000000000001e-07 2024-09-21T21:43:16.407 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.407 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.407 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:16.407 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273248+0000", 2024-09-21T21:43:16.407 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1550000000000001e-06 2024-09-21T21:43:16.407 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.407 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.408 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:16.408 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273254+0000", 2024-09-21T21:43:16.408 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.964e-06 2024-09-21T21:43:16.408 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.408 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.408 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:16.408 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273259+0000", 2024-09-21T21:43:16.408 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5499999999999996e-06 2024-09-21T21:43:16.408 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.408 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.408 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:16.408 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278130+0000", 2024-09-21T21:43:16.408 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048715779999999997 2024-09-21T21:43:16.409 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.409 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.409 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:16.409 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278133+0000", 2024-09-21T21:43:16.409 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8739999999999999e-06 2024-09-21T21:43:16.409 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.409 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.409 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:16.409 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247270+0000", 2024-09-21T21:43:16.409 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.969137099 2024-09-21T21:43:16.409 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.410 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.410 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:16.410 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247338+0000", 2024-09-21T21:43:16.410 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7717000000000002e-05 2024-09-21T21:43:16.410 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.410 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.410 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:16.410 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247405+0000", 2024-09-21T21:43:16.410 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7434000000000005e-05 2024-09-21T21:43:16.410 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:16.410 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:16.410 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:16.411 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.411 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.411 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19769 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 5840896~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:16.411 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273287+0000", 2024-09-21T21:43:16.411 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 85.086041530000003, 2024-09-21T21:43:16.411 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97435390200000005, 2024-09-21T21:43:16.411 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:16.411 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:16.411 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:16.411 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:16.411 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:16.411 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:16.412 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19769 2024-09-21T21:43:16.412 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.412 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:16.412 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.412 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:16.412 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273287+0000", 2024-09-21T21:43:16.412 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:16.412 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.412 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.412 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:16.412 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273287+0000", 2024-09-21T21:43:16.413 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:16.413 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.413 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.413 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:16.413 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273287+0000", 2024-09-21T21:43:16.413 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6699999999999999e-07 2024-09-21T21:43:16.413 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.413 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.413 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:16.413 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273289+0000", 2024-09-21T21:43:16.413 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.708e-06 2024-09-21T21:43:16.413 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.413 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.413 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:16.414 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273296+0000", 2024-09-21T21:43:16.414 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8109999999999997e-06 2024-09-21T21:43:16.414 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.414 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.414 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:16.414 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273299+0000", 2024-09-21T21:43:16.414 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0740000000000001e-06 2024-09-21T21:43:16.414 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.414 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.414 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:16.414 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278159+0000", 2024-09-21T21:43:16.414 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048603240000000001 2024-09-21T21:43:16.414 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.415 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.415 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:16.415 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278162+0000", 2024-09-21T21:43:16.415 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1930000000000002e-06 2024-09-21T21:43:16.415 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.415 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.415 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:16.415 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247444+0000", 2024-09-21T21:43:16.415 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96928199100000001 2024-09-21T21:43:16.415 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.415 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.415 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:16.415 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247533+0000", 2024-09-21T21:43:16.415 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8490000000000001e-05 2024-09-21T21:43:16.416 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.416 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.416 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:16.416 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247641+0000", 2024-09-21T21:43:16.416 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000107944 2024-09-21T21:43:16.416 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:16.416 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:16.416 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:16.416 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.416 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.416 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19770 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 5869568~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:16.416 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273313+0000", 2024-09-21T21:43:16.416 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 85.086015427999996, 2024-09-21T21:43:16.417 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97449038899999996, 2024-09-21T21:43:16.417 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:16.417 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:16.417 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:16.417 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:16.417 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:16.417 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:16.417 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19770 2024-09-21T21:43:16.417 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.417 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:16.417 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.417 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:16.417 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273313+0000", 2024-09-21T21:43:16.417 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:16.418 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.418 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.418 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:16.418 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273313+0000", 2024-09-21T21:43:16.418 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:16.418 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.418 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.418 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:16.418 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273313+0000", 2024-09-21T21:43:16.418 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5699999999999998e-07 2024-09-21T21:43:16.418 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.418 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.418 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:16.418 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273315+0000", 2024-09-21T21:43:16.419 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0099999999999998e-06 2024-09-21T21:43:16.419 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.419 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.419 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:16.419 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273321+0000", 2024-09-21T21:43:16.419 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7529999999999998e-06 2024-09-21T21:43:16.419 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.419 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.419 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:16.419 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273328+0000", 2024-09-21T21:43:16.419 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9009999999999997e-06 2024-09-21T21:43:16.419 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.419 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.420 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:16.420 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278183+0000", 2024-09-21T21:43:16.420 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048547770000000002 2024-09-21T21:43:16.420 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.420 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.420 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:16.420 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278186+0000", 2024-09-21T21:43:16.420 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3100000000000001e-06 2024-09-21T21:43:16.420 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.420 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.420 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:16.420 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247676+0000", 2024-09-21T21:43:16.420 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96949004299999997 2024-09-21T21:43:16.421 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.421 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.421 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:16.421 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247757+0000", 2024-09-21T21:43:16.421 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0479e-05 2024-09-21T21:43:16.421 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.421 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.421 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:16.421 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247803+0000", 2024-09-21T21:43:16.421 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6758999999999997e-05 2024-09-21T21:43:16.421 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:16.421 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:16.421 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:16.422 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.422 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.422 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19771 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6586368~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:16.422 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273335+0000", 2024-09-21T21:43:16.422 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 85.085993641000002, 2024-09-21T21:43:16.422 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97458947699999998, 2024-09-21T21:43:16.422 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:16.422 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:16.422 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:16.422 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:16.422 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:16.422 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:16.422 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19771 2024-09-21T21:43:16.423 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.423 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:16.423 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.423 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:16.423 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273335+0000", 2024-09-21T21:43:16.423 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:16.423 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.423 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.423 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:16.423 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273335+0000", 2024-09-21T21:43:16.423 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:16.423 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.423 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.424 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:16.424 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273335+0000", 2024-09-21T21:43:16.424 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3599999999999999e-07 2024-09-21T21:43:16.424 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.424 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.424 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:16.424 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273337+0000", 2024-09-21T21:43:16.424 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0789999999999999e-06 2024-09-21T21:43:16.424 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.425 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.425 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:16.425 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273343+0000", 2024-09-21T21:43:16.425 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6280000000000002e-06 2024-09-21T21:43:16.425 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.425 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.425 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:16.425 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273346+0000", 2024-09-21T21:43:16.425 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.507e-06 2024-09-21T21:43:16.425 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.425 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.425 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:16.425 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278206+0000", 2024-09-21T21:43:16.426 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048600969999999999 2024-09-21T21:43:16.426 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.426 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.426 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:16.426 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278212+0000", 2024-09-21T21:43:16.426 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2079999999999999e-06 2024-09-21T21:43:16.426 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.426 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.426 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:16.426 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247828+0000", 2024-09-21T21:43:16.427 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.969616547 2024-09-21T21:43:16.427 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.427 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.427 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:16.427 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247881+0000", 2024-09-21T21:43:16.427 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3180999999999998e-05 2024-09-21T21:43:16.427 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.427 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.427 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:16.427 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247924+0000", 2024-09-21T21:43:16.427 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2894e-05 2024-09-21T21:43:16.427 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:16.428 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:16.428 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:16.428 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.428 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.428 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19772 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6557696~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:16.428 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273358+0000", 2024-09-21T21:43:16.428 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 85.085970261, 2024-09-21T21:43:16.428 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97467031800000004, 2024-09-21T21:43:16.428 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:16.428 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:16.428 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:16.429 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:16.429 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:16.429 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:16.429 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19772 2024-09-21T21:43:16.429 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.429 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:16.429 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.429 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:16.429 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273358+0000", 2024-09-21T21:43:16.429 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:16.429 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.429 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.430 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:16.430 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273358+0000", 2024-09-21T21:43:16.430 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:16.430 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.430 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.430 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:16.430 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273358+0000", 2024-09-21T21:43:16.430 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4499999999999998e-07 2024-09-21T21:43:16.430 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.430 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.431 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:16.431 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273360+0000", 2024-09-21T21:43:16.431 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9190000000000002e-06 2024-09-21T21:43:16.431 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.431 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.431 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:16.431 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273366+0000", 2024-09-21T21:43:16.431 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.891e-06 2024-09-21T21:43:16.431 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.431 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.431 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:16.431 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273371+0000", 2024-09-21T21:43:16.432 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1880000000000001e-06 2024-09-21T21:43:16.432 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.432 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.432 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:16.432 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278237+0000", 2024-09-21T21:43:16.432 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.004865472 2024-09-21T21:43:16.432 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.432 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.432 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:16.432 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278240+0000", 2024-09-21T21:43:16.432 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9550000000000001e-06 2024-09-21T21:43:16.432 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.433 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.433 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:16.433 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247949+0000", 2024-09-21T21:43:16.433 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.969709127 2024-09-21T21:43:16.433 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.433 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.433 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:16.433 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247990+0000", 2024-09-21T21:43:16.433 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1202000000000003e-05 2024-09-21T21:43:16.433 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.433 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.433 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:16.433 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248028+0000", 2024-09-21T21:43:16.434 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8219000000000001e-05 2024-09-21T21:43:16.434 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:16.434 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:16.434 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:16.434 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.434 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.434 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19773 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6615040~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:16.434 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273390+0000", 2024-09-21T21:43:16.434 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 85.085938400000003, 2024-09-21T21:43:16.434 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97476086200000001, 2024-09-21T21:43:16.434 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:16.434 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:16.435 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:16.435 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:16.435 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:16.435 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:16.435 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19773 2024-09-21T21:43:16.435 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.435 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:16.435 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.435 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:16.435 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273390+0000", 2024-09-21T21:43:16.435 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:16.435 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.436 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.436 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:16.436 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273390+0000", 2024-09-21T21:43:16.436 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:16.436 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.436 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.436 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:16.436 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273390+0000", 2024-09-21T21:43:16.436 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9999999999999999e-07 2024-09-21T21:43:16.436 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.436 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.436 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:16.436 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273392+0000", 2024-09-21T21:43:16.436 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0640000000000001e-06 2024-09-21T21:43:16.437 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.437 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.437 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:16.437 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273397+0000", 2024-09-21T21:43:16.437 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.113e-06 2024-09-21T21:43:16.437 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.437 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.437 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:16.437 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273401+0000", 2024-09-21T21:43:16.437 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3119999999999998e-06 2024-09-21T21:43:16.437 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.438 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.438 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:16.438 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278271+0000", 2024-09-21T21:43:16.438 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048698880000000002 2024-09-21T21:43:16.438 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.438 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.438 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:16.438 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278278+0000", 2024-09-21T21:43:16.438 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1210000000000001e-06 2024-09-21T21:43:16.438 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.438 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.438 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:16.439 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248060+0000", 2024-09-21T21:43:16.439 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96978224999999996 2024-09-21T21:43:16.439 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.439 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.439 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:16.439 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248111+0000", 2024-09-21T21:43:16.439 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.096e-05 2024-09-21T21:43:16.439 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.439 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.439 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:16.439 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248151+0000", 2024-09-21T21:43:16.439 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9854000000000002e-05 2024-09-21T21:43:16.439 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:16.440 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:16.440 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:16.440 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.440 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.440 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19784 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6643712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:16.440 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273405+0000", 2024-09-21T21:43:16.440 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 85.085923679000004, 2024-09-21T21:43:16.440 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97491798900000004, 2024-09-21T21:43:16.440 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:16.440 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:16.440 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:16.441 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:16.441 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:16.441 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:16.441 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19784 2024-09-21T21:43:16.441 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.441 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:16.441 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.441 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:16.441 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273405+0000", 2024-09-21T21:43:16.441 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:16.441 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.441 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.442 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:16.442 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273405+0000", 2024-09-21T21:43:16.442 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:16.442 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.442 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.442 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:16.442 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273405+0000", 2024-09-21T21:43:16.442 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2100000000000001e-07 2024-09-21T21:43:16.442 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.442 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.442 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:16.442 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273406+0000", 2024-09-21T21:43:16.442 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2049999999999999e-06 2024-09-21T21:43:16.443 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.443 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.443 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:16.443 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273409+0000", 2024-09-21T21:43:16.443 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0599999999999999e-06 2024-09-21T21:43:16.443 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.443 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.443 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:16.443 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273412+0000", 2024-09-21T21:43:16.443 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0730000000000001e-06 2024-09-21T21:43:16.443 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.444 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.444 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:16.444 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278317+0000", 2024-09-21T21:43:16.444 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0049045520000000004 2024-09-21T21:43:16.444 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.444 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.444 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:16.444 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278321+0000", 2024-09-21T21:43:16.444 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6249999999999998e-06 2024-09-21T21:43:16.444 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.444 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.444 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:16.444 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248179+0000", 2024-09-21T21:43:16.445 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96985744500000004 2024-09-21T21:43:16.445 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.445 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.445 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:16.445 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248253+0000", 2024-09-21T21:43:16.445 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3829000000000001e-05 2024-09-21T21:43:16.445 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.445 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.445 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:16.445 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248323+0000", 2024-09-21T21:43:16.445 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9979000000000002e-05 2024-09-21T21:43:16.445 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:16.445 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:16.446 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:16.446 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.446 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.446 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19785 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6672384~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:16.446 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273415+0000", 2024-09-21T21:43:16.446 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 85.085913093000002, 2024-09-21T21:43:16.446 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97500777800000005, 2024-09-21T21:43:16.446 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:16.446 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:16.446 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:16.446 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:16.446 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:16.446 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:16.447 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19785 2024-09-21T21:43:16.447 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.447 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:16.447 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.447 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:16.447 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273415+0000", 2024-09-21T21:43:16.447 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:16.447 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.447 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.447 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:16.447 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273415+0000", 2024-09-21T21:43:16.447 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:16.447 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.447 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.448 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:16.448 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273415+0000", 2024-09-21T21:43:16.448 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0100000000000001e-07 2024-09-21T21:43:16.448 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.448 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.448 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:16.448 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273417+0000", 2024-09-21T21:43:16.448 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.114e-06 2024-09-21T21:43:16.448 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.448 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.448 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:16.448 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273420+0000", 2024-09-21T21:43:16.448 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6770000000000001e-06 2024-09-21T21:43:16.449 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.449 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.449 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:16.449 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273423+0000", 2024-09-21T21:43:16.449 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5450000000000002e-06 2024-09-21T21:43:16.449 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.449 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.449 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:16.449 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278355+0000", 2024-09-21T21:43:16.449 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0049318590000000002 2024-09-21T21:43:16.449 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.449 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.449 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:16.449 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278360+0000", 2024-09-21T21:43:16.450 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3079999999999998e-06 2024-09-21T21:43:16.450 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.450 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.450 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:16.450 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248349+0000", 2024-09-21T21:43:16.450 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96998909899999997 2024-09-21T21:43:16.450 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.450 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.450 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:16.450 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248387+0000", 2024-09-21T21:43:16.450 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7657000000000002e-05 2024-09-21T21:43:16.450 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.450 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.450 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:16.451 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248423+0000", 2024-09-21T21:43:16.451 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6318e-05 2024-09-21T21:43:16.451 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:16.451 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:16.451 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:16.451 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.451 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.451 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19842 3.d 3:b7fa2ca0:::rbd_data.112157a479c.000000000000004d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:16.451 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273426+0000", 2024-09-21T21:43:16.451 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 85.085902040999997, 2024-09-21T21:43:16.451 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.98136526800000001, 2024-09-21T21:43:16.451 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:16.451 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:16.451 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:16.452 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:16.452 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:16.452 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:16.452 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19842 2024-09-21T21:43:16.452 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.452 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:16.452 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.452 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:16.452 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273426+0000", 2024-09-21T21:43:16.452 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:16.452 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.452 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.452 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:16.453 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273426+0000", 2024-09-21T21:43:16.453 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:16.453 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.453 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.453 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:16.453 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273427+0000", 2024-09-21T21:43:16.453 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.72e-07 2024-09-21T21:43:16.453 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.453 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.453 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:16.453 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275836+0000", 2024-09-21T21:43:16.453 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00240909 2024-09-21T21:43:16.453 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.453 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.453 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:16.454 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275844+0000", 2024-09-21T21:43:16.454 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3750000000000003e-06 2024-09-21T21:43:16.454 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.454 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.454 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:16.454 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275849+0000", 2024-09-21T21:43:16.454 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0100000000000003e-06 2024-09-21T21:43:16.454 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.454 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.454 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:16.454 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278394+0000", 2024-09-21T21:43:16.454 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025449890000000001 2024-09-21T21:43:16.454 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.455 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.455 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:16.455 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278402+0000", 2024-09-21T21:43:16.455 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6359999999999994e-06 2024-09-21T21:43:16.455 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.455 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.455 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:16.455 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248454+0000", 2024-09-21T21:43:16.455 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97005259499999996 2024-09-21T21:43:16.455 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.455 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.455 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:16.456 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248576+0000", 2024-09-21T21:43:16.456 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000122127 2024-09-21T21:43:16.456 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.456 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.456 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 2", 2024-09-21T21:43:16.456 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248844+0000", 2024-09-21T21:43:16.456 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00026784699999999999 2024-09-21T21:43:16.456 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.456 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.456 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:16.456 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.252487+0000", 2024-09-21T21:43:16.456 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0036427920000000002 2024-09-21T21:43:16.456 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.456 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.457 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:16.457 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.254691+0000", 2024-09-21T21:43:16.457 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002203899 2024-09-21T21:43:16.457 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.457 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.457 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:16.457 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.254748+0000", 2024-09-21T21:43:16.457 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7163000000000001e-05 2024-09-21T21:43:16.457 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.457 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.458 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:16.458 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.254792+0000", 2024-09-21T21:43:16.458 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3473000000000002e-05 2024-09-21T21:43:16.458 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:16.458 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:16.458 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:16.458 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.458 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.458 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19843 3.d 3:b7fa2ca0:::rbd_data.112157a479c.000000000000004d:head [write 0~0] snapc 21=[21] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:16.458 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.275860+0000", 2024-09-21T21:43:16.458 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 85.083468140999997, 2024-09-21T21:43:16.459 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.98183651999999999, 2024-09-21T21:43:16.459 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:16.459 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:16.459 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:16.459 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:16.459 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:16.459 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:16.459 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19843 2024-09-21T21:43:16.459 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.459 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:16.459 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.459 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:16.460 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275860+0000", 2024-09-21T21:43:16.460 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:16.460 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.460 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.460 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:16.460 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275860+0000", 2024-09-21T21:43:16.460 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:16.460 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.460 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.460 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:16.460 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275869+0000", 2024-09-21T21:43:16.461 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6300000000000004e-06 2024-09-21T21:43:16.461 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.461 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.461 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:16.461 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275871+0000", 2024-09-21T21:43:16.461 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2060000000000001e-06 2024-09-21T21:43:16.461 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.461 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.461 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:16.461 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275876+0000", 2024-09-21T21:43:16.461 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8180000000000003e-06 2024-09-21T21:43:16.461 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.462 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.462 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:16.462 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275880+0000", 2024-09-21T21:43:16.462 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1829999999999996e-06 2024-09-21T21:43:16.462 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.462 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.462 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:16.462 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278416+0000", 2024-09-21T21:43:16.462 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025361149999999998 2024-09-21T21:43:16.462 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.463 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.463 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:16.463 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278419+0000", 2024-09-21T21:43:16.463 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.163e-06 2024-09-21T21:43:16.463 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.463 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.463 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:16.463 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.250249+0000", 2024-09-21T21:43:16.463 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97182984299999997 2024-09-21T21:43:16.463 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.463 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.463 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:16.463 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.250321+0000", 2024-09-21T21:43:16.464 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1656000000000006e-05 2024-09-21T21:43:16.464 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.464 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.464 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 2", 2024-09-21T21:43:16.464 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.250419+0000", 2024-09-21T21:43:16.464 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.8277999999999997e-05 2024-09-21T21:43:16.464 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.464 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.464 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:16.464 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.252516+0000", 2024-09-21T21:43:16.465 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002097224 2024-09-21T21:43:16.465 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.465 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.465 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:16.465 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.257643+0000", 2024-09-21T21:43:16.465 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0051262319999999997 2024-09-21T21:43:16.465 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.465 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.465 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:16.465 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.257673+0000", 2024-09-21T21:43:16.465 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0023e-05 2024-09-21T21:43:16.466 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.466 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.466 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:16.466 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.257697+0000", 2024-09-21T21:43:16.466 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4148999999999999e-05 2024-09-21T21:43:16.466 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:16.466 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:16.466 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:16.466 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.466 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.466 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5626.0:12334 3.3 3:c643a86c:::rbd_data.1121faebdfe3.0000000000000002:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:43:16.467 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:14.411628+0000", 2024-09-21T21:43:16.467 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 61.947699841999999, 2024-09-21T21:43:16.467 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0086947559999999, 2024-09-21T21:43:16.467 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:16.467 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:16.467 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:16.467 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:16.467 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5626", 2024-09-21T21:43:16.467 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:43:16.467 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 12334 2024-09-21T21:43:16.467 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.467 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:16.467 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.467 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:16.468 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411628+0000", 2024-09-21T21:43:16.468 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:16.468 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.468 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.468 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:16.468 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411628+0000", 2024-09-21T21:43:16.468 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:16.468 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.468 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.468 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:16.468 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411629+0000", 2024-09-21T21:43:16.468 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1200000000000002e-07 2024-09-21T21:43:16.468 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.469 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.469 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:16.469 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411633+0000", 2024-09-21T21:43:16.469 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1699999999999999e-06 2024-09-21T21:43:16.469 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.469 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.469 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:16.469 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411645+0000", 2024-09-21T21:43:16.469 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1328999999999999e-05 2024-09-21T21:43:16.469 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.469 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.469 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:16.469 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411652+0000", 2024-09-21T21:43:16.469 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9990000000000002e-06 2024-09-21T21:43:16.470 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.470 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.470 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:16.470 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411717+0000", 2024-09-21T21:43:16.470 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5519999999999996e-05 2024-09-21T21:43:16.470 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.470 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.470 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op must wait for map", 2024-09-21T21:43:16.470 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411725+0000", 2024-09-21T21:43:16.470 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2609999999999996e-06 2024-09-21T21:43:16.470 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.470 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.470 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:16.470 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415904+0000", 2024-09-21T21:43:16.471 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0041792749999999997 2024-09-21T21:43:16.471 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.471 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.471 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:16.471 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415913+0000", 2024-09-21T21:43:16.471 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.054e-06 2024-09-21T21:43:16.471 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.471 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.471 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:16.471 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420096+0000", 2024-09-21T21:43:16.471 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0041835670000001 2024-09-21T21:43:16.471 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.471 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.472 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:16.472 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420216+0000", 2024-09-21T21:43:16.472 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000120025 2024-09-21T21:43:16.472 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.472 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.472 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:16.472 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420323+0000", 2024-09-21T21:43:16.472 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010674400000000001 2024-09-21T21:43:16.472 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:16.472 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:16.472 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:16.472 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.473 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.473 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5626.0:12339 3.3 3:c643a86c:::rbd_data.1121faebdfe3.0000000000000002:head [read 4096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:43:16.473 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:14.411749+0000", 2024-09-21T21:43:16.473 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 61.947579656000002, 2024-09-21T21:43:16.473 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.008818056, 2024-09-21T21:43:16.473 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:16.473 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:16.473 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:16.473 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:16.473 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5626", 2024-09-21T21:43:16.473 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:43:16.473 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 12339 2024-09-21T21:43:16.473 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.473 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:16.474 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.474 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:16.474 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411749+0000", 2024-09-21T21:43:16.474 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:16.474 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.474 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.474 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:16.474 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411749+0000", 2024-09-21T21:43:16.474 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:16.474 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.474 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.474 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:16.475 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411749+0000", 2024-09-21T21:43:16.475 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0900000000000002e-07 2024-09-21T21:43:16.475 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.475 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.475 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:16.475 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411752+0000", 2024-09-21T21:43:16.475 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5610000000000001e-06 2024-09-21T21:43:16.475 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.475 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.475 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:16.475 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411758+0000", 2024-09-21T21:43:16.475 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1890000000000004e-06 2024-09-21T21:43:16.475 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.476 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.476 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:16.476 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411763+0000", 2024-09-21T21:43:16.476 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7589999999999997e-06 2024-09-21T21:43:16.476 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.476 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.476 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:16.476 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411803+0000", 2024-09-21T21:43:16.476 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9922999999999997e-05 2024-09-21T21:43:16.476 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.476 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.476 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting_for_map not empty", 2024-09-21T21:43:16.476 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411809+0000", 2024-09-21T21:43:16.477 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3779999999999998e-06 2024-09-21T21:43:16.477 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.477 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.477 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:16.477 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415937+0000", 2024-09-21T21:43:16.477 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0041276100000000003 2024-09-21T21:43:16.477 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.477 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.477 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:16.477 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415944+0000", 2024-09-21T21:43:16.477 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8329999999999994e-06 2024-09-21T21:43:16.477 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.478 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.478 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:16.478 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420411+0000", 2024-09-21T21:43:16.478 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0044667300000001 2024-09-21T21:43:16.478 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.478 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.478 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:16.478 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420475+0000", 2024-09-21T21:43:16.478 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4356000000000005e-05 2024-09-21T21:43:16.478 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.478 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.478 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:16.479 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420567+0000", 2024-09-21T21:43:16.479 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.1308000000000001e-05 2024-09-21T21:43:16.479 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:16.479 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:16.479 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:16.479 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.479 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.479 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5626.0:12345 3.3 3:c643a86c:::rbd_data.1121faebdfe3.0000000000000002:head [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:43:16.479 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:14.411831+0000", 2024-09-21T21:43:16.479 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 61.947497603000002, 2024-09-21T21:43:16.479 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0089163649999999, 2024-09-21T21:43:16.479 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:16.479 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:16.479 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:16.479 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:16.480 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5626", 2024-09-21T21:43:16.480 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:43:16.480 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 12345 2024-09-21T21:43:16.480 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.480 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:16.480 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.480 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:16.480 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411831+0000", 2024-09-21T21:43:16.480 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:16.480 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.480 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.480 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:16.480 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411831+0000", 2024-09-21T21:43:16.481 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:16.481 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.481 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.481 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:16.481 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411831+0000", 2024-09-21T21:43:16.481 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.08e-07 2024-09-21T21:43:16.481 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.481 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.481 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:16.481 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411833+0000", 2024-09-21T21:43:16.481 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3089999999999998e-06 2024-09-21T21:43:16.481 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.481 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.481 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:16.482 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411840+0000", 2024-09-21T21:43:16.482 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.472e-06 2024-09-21T21:43:16.482 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.482 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.482 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:16.482 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411845+0000", 2024-09-21T21:43:16.482 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7910000000000004e-06 2024-09-21T21:43:16.482 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.482 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.482 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:16.482 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411890+0000", 2024-09-21T21:43:16.482 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5763000000000002e-05 2024-09-21T21:43:16.482 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.482 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.483 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting_for_map not empty", 2024-09-21T21:43:16.483 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411896+0000", 2024-09-21T21:43:16.483 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0190000000000003e-06 2024-09-21T21:43:16.483 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.483 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.483 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:16.483 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415967+0000", 2024-09-21T21:43:16.483 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0040705749999999999 2024-09-21T21:43:16.483 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.483 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.483 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:16.483 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415974+0000", 2024-09-21T21:43:16.483 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3200000000000002e-06 2024-09-21T21:43:16.483 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.483 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.484 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:16.484 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420594+0000", 2024-09-21T21:43:16.484 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.004620093 2024-09-21T21:43:16.484 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.484 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.484 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:16.484 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420658+0000", 2024-09-21T21:43:16.484 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3678999999999998e-05 2024-09-21T21:43:16.484 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.484 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.484 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:16.484 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420747+0000", 2024-09-21T21:43:16.484 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8936000000000004e-05 2024-09-21T21:43:16.484 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:16.485 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:16.485 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:16.485 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:16.485 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:16.485 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:43:16.486 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T21:43:16.487 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:43:16.484+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:43:16.587 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:43:16.585+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:43:16.660 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:43:16.660 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:43:16.660 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-21T21:43:16.661 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:43:16.671 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T21:43:16.688 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:43:16.686+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:43:16.788 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:43:16.787+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:43:16.791 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:43:16.791 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:43:16.791 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:43:16.791 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:43:16.791 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:43:16.801 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T21:43:16.890 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:43:16.888+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:43:16.953 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:43:16.954 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-21T21:43:16.954 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-21T21:43:16.954 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:43:16.954 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.954 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6974 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5246976~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:16.954 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.922932+0000", 2024-09-21T21:43:16.954 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 107.028420235, 2024-09-21T21:43:16.954 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041773352999999999, 2024-09-21T21:43:16.954 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:16.954 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:16.955 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:16.955 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:16.955 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:16.955 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:16.955 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6974 2024-09-21T21:43:16.955 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.955 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:16.955 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.956 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:16.956 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922932+0000", 2024-09-21T21:43:16.956 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:16.956 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.956 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.956 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:16.956 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922932+0000", 2024-09-21T21:43:16.956 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:16.956 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.956 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.956 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:16.956 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922933+0000", 2024-09-21T21:43:16.957 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.46e-07 2024-09-21T21:43:16.957 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.957 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.957 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:16.957 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922940+0000", 2024-09-21T21:43:16.957 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6140000000000003e-06 2024-09-21T21:43:16.957 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.957 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.957 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:16.957 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922949+0000", 2024-09-21T21:43:16.957 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6899999999999998e-06 2024-09-21T21:43:16.957 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.957 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.958 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:16.958 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922955+0000", 2024-09-21T21:43:16.958 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5140000000000001e-06 2024-09-21T21:43:16.958 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.958 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.958 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:16.958 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964547+0000", 2024-09-21T21:43:16.958 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041592854999999998 2024-09-21T21:43:16.958 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.958 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.959 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:16.959 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964617+0000", 2024-09-21T21:43:16.959 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9058000000000002e-05 2024-09-21T21:43:16.959 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.959 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.959 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:16.959 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964706+0000", 2024-09-21T21:43:16.959 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9276000000000004e-05 2024-09-21T21:43:16.959 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:16.959 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:16.959 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:16.959 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.959 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.959 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6976 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5261312~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:16.960 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.922961+0000", 2024-09-21T21:43:16.960 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 107.02839118999999, 2024-09-21T21:43:16.960 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041863648000000003, 2024-09-21T21:43:16.960 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:16.960 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:16.960 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:16.960 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:16.960 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:16.960 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:16.960 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6976 2024-09-21T21:43:16.960 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.960 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:16.960 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.960 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:16.960 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922961+0000", 2024-09-21T21:43:16.960 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:16.961 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.961 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.961 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:16.961 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922961+0000", 2024-09-21T21:43:16.961 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:16.961 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.961 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.961 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:16.961 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922962+0000", 2024-09-21T21:43:16.961 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.89e-07 2024-09-21T21:43:16.961 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.961 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.961 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:16.961 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922964+0000", 2024-09-21T21:43:16.961 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1270000000000001e-06 2024-09-21T21:43:16.962 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.962 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.962 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:16.962 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922970+0000", 2024-09-21T21:43:16.962 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8030000000000002e-06 2024-09-21T21:43:16.962 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.962 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.962 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:16.962 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922979+0000", 2024-09-21T21:43:16.962 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5559999999999996e-06 2024-09-21T21:43:16.962 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.962 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.963 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:16.963 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964734+0000", 2024-09-21T21:43:16.963 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041754415000000003 2024-09-21T21:43:16.963 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.963 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.963 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:16.963 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964775+0000", 2024-09-21T21:43:16.963 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0806999999999997e-05 2024-09-21T21:43:16.963 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.964 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.964 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:16.964 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964825+0000", 2024-09-21T21:43:16.964 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0550999999999998e-05 2024-09-21T21:43:16.964 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:16.964 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:16.964 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:16.964 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.964 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.964 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6978 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5275648~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:16.964 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.922987+0000", 2024-09-21T21:43:16.965 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 107.028365405, 2024-09-21T21:43:16.965 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042043876000000001, 2024-09-21T21:43:16.965 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:16.965 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:16.965 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:16.965 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:16.965 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:16.965 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:16.965 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6978 2024-09-21T21:43:16.966 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.966 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:16.966 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.966 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:16.966 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922987+0000", 2024-09-21T21:43:16.966 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:16.966 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.966 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.966 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:16.966 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922987+0000", 2024-09-21T21:43:16.966 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:16.967 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.967 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.967 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:16.967 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922988+0000", 2024-09-21T21:43:16.967 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1199999999999998e-07 2024-09-21T21:43:16.967 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.967 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.967 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:16.967 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922990+0000", 2024-09-21T21:43:16.967 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.142e-06 2024-09-21T21:43:16.967 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.967 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.967 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:16.967 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922997+0000", 2024-09-21T21:43:16.968 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.199e-06 2024-09-21T21:43:16.968 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.968 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.968 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:16.968 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923004+0000", 2024-09-21T21:43:16.968 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3599999999999998e-06 2024-09-21T21:43:16.968 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.968 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.968 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:16.968 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964856+0000", 2024-09-21T21:43:16.968 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041851521000000003 2024-09-21T21:43:16.968 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.968 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.969 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:16.969 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964944+0000", 2024-09-21T21:43:16.969 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7859e-05 2024-09-21T21:43:16.969 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.969 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.969 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:16.969 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965031+0000", 2024-09-21T21:43:16.969 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7383e-05 2024-09-21T21:43:16.969 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:16.969 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:16.969 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:16.969 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.969 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.969 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6980 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5289984~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:16.969 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923011+0000", 2024-09-21T21:43:16.970 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 107.028342068, 2024-09-21T21:43:16.970 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042139337999999998, 2024-09-21T21:43:16.970 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:16.970 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:16.970 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:16.970 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:16.970 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:16.970 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:16.970 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6980 2024-09-21T21:43:16.970 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.970 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:16.970 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.971 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:16.971 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923011+0000", 2024-09-21T21:43:16.971 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:16.971 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.971 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.971 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:16.971 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923011+0000", 2024-09-21T21:43:16.971 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:16.971 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.971 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.971 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:16.971 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923011+0000", 2024-09-21T21:43:16.971 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6699999999999999e-07 2024-09-21T21:43:16.971 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.971 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.972 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:16.972 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923013+0000", 2024-09-21T21:43:16.972 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0279999999999999e-06 2024-09-21T21:43:16.972 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.972 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.972 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:16.972 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923018+0000", 2024-09-21T21:43:16.972 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4790000000000004e-06 2024-09-21T21:43:16.972 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.972 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.972 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:16.972 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923029+0000", 2024-09-21T21:43:16.972 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0601999999999999e-05 2024-09-21T21:43:16.972 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.972 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.972 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:16.973 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965059+0000", 2024-09-21T21:43:16.973 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042029947999999998 2024-09-21T21:43:16.973 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.973 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.973 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:16.973 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965100+0000", 2024-09-21T21:43:16.973 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1170000000000001e-05 2024-09-21T21:43:16.973 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.973 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.973 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:16.973 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965150+0000", 2024-09-21T21:43:16.973 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9743999999999997e-05 2024-09-21T21:43:16.973 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:16.973 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:16.973 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:16.973 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.974 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.974 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6982 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5304320~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:16.974 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923036+0000", 2024-09-21T21:43:16.974 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 107.02831624700001, 2024-09-21T21:43:16.974 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042225666000000002, 2024-09-21T21:43:16.974 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:16.974 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:16.974 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:16.974 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:16.974 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:16.974 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:16.975 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6982 2024-09-21T21:43:16.975 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.975 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:16.975 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.975 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:16.975 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923036+0000", 2024-09-21T21:43:16.975 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:16.975 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.975 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.975 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:16.975 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923036+0000", 2024-09-21T21:43:16.975 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:16.975 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.975 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.975 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:16.976 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923037+0000", 2024-09-21T21:43:16.976 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2800000000000002e-07 2024-09-21T21:43:16.976 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.976 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.976 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:16.976 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923039+0000", 2024-09-21T21:43:16.976 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1500000000000002e-06 2024-09-21T21:43:16.976 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.976 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.976 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:16.976 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923049+0000", 2024-09-21T21:43:16.976 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0236e-05 2024-09-21T21:43:16.976 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.976 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.976 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:16.976 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923056+0000", 2024-09-21T21:43:16.977 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2540000000000003e-06 2024-09-21T21:43:16.977 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.977 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.977 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:16.977 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965173+0000", 2024-09-21T21:43:16.977 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042116686 2024-09-21T21:43:16.977 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.977 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.977 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:16.977 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965215+0000", 2024-09-21T21:43:16.977 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1696000000000002e-05 2024-09-21T21:43:16.977 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.977 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.977 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:16.978 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965262+0000", 2024-09-21T21:43:16.978 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7216000000000003e-05 2024-09-21T21:43:16.978 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:16.978 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:16.978 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:16.978 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.978 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.978 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6984 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5318656~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:16.978 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923063+0000", 2024-09-21T21:43:16.978 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 107.028289593, 2024-09-21T21:43:16.978 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042318651999999998, 2024-09-21T21:43:16.978 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:16.978 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:16.978 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:16.979 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:16.979 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:16.979 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:16.979 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6984 2024-09-21T21:43:16.979 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.979 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:16.979 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.979 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:16.979 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923063+0000", 2024-09-21T21:43:16.979 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:16.979 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.980 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.980 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:16.980 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923063+0000", 2024-09-21T21:43:16.980 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:16.980 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.980 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.980 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:16.980 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923064+0000", 2024-09-21T21:43:16.980 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2e-07 2024-09-21T21:43:16.980 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.980 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.980 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:16.980 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923066+0000", 2024-09-21T21:43:16.980 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.057e-06 2024-09-21T21:43:16.981 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.981 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.981 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:16.981 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923071+0000", 2024-09-21T21:43:16.981 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8810000000000001e-06 2024-09-21T21:43:16.981 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.981 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.981 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:16.981 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923078+0000", 2024-09-21T21:43:16.981 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5479999999999998e-06 2024-09-21T21:43:16.981 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.981 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.981 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:16.981 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965288+0000", 2024-09-21T21:43:16.982 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042209576999999998 2024-09-21T21:43:16.982 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.982 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.982 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:16.982 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965332+0000", 2024-09-21T21:43:16.982 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4366999999999997e-05 2024-09-21T21:43:16.982 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.983 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.983 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:16.983 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965382+0000", 2024-09-21T21:43:16.983 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9802000000000002e-05 2024-09-21T21:43:16.983 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:16.983 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:16.983 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:16.983 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.983 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.983 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6987 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5332992~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:16.983 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923085+0000", 2024-09-21T21:43:16.983 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 107.028268068, 2024-09-21T21:43:16.983 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042412103999999999, 2024-09-21T21:43:16.983 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:16.984 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:16.984 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:16.984 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:16.984 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:16.984 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:16.984 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6987 2024-09-21T21:43:16.984 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.984 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:16.985 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.985 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:16.985 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923085+0000", 2024-09-21T21:43:16.985 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:16.985 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.985 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.985 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:16.985 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923085+0000", 2024-09-21T21:43:16.985 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:16.985 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.985 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.985 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:16.985 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923085+0000", 2024-09-21T21:43:16.986 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.89e-07 2024-09-21T21:43:16.986 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.986 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.986 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:16.986 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923087+0000", 2024-09-21T21:43:16.986 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0810000000000001e-06 2024-09-21T21:43:16.986 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.986 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.986 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:16.987 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923098+0000", 2024-09-21T21:43:16.987 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1299e-05 2024-09-21T21:43:16.987 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.987 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.987 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:16.987 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923104+0000", 2024-09-21T21:43:16.987 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9100000000000002e-06 2024-09-21T21:43:16.987 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.987 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.987 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:16.987 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965404+0000", 2024-09-21T21:43:16.988 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042299622000000002 2024-09-21T21:43:16.988 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.988 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.988 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:16.988 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965445+0000", 2024-09-21T21:43:16.988 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0701000000000002e-05 2024-09-21T21:43:16.988 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.988 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.989 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:16.989 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965497+0000", 2024-09-21T21:43:16.989 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2101999999999997e-05 2024-09-21T21:43:16.989 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:16.989 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:16.989 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:16.989 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.989 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.989 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6989 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5347328~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:16.989 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923111+0000", 2024-09-21T21:43:16.989 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 107.028241728, 2024-09-21T21:43:16.989 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042505188999999999, 2024-09-21T21:43:16.990 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:16.990 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:16.990 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:16.990 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:16.990 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:16.990 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:16.990 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6989 2024-09-21T21:43:16.990 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.990 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:16.990 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.990 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:16.990 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923111+0000", 2024-09-21T21:43:16.991 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:16.991 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.991 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.991 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:16.992 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923111+0000", 2024-09-21T21:43:16.992 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:16.992 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.992 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.992 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:16.992 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923111+0000", 2024-09-21T21:43:16.992 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4499999999999997e-07 2024-09-21T21:43:16.992 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.992 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.992 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:16.992 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923114+0000", 2024-09-21T21:43:16.992 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1569999999999998e-06 2024-09-21T21:43:16.993 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.993 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.993 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:16.993 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923120+0000", 2024-09-21T21:43:16.993 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1759999999999998e-06 2024-09-21T21:43:16.993 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.993 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.993 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:16.993 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923125+0000", 2024-09-21T21:43:16.993 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2560000000000002e-06 2024-09-21T21:43:16.993 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.993 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.993 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:16.993 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965523+0000", 2024-09-21T21:43:16.994 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042398094999999997 2024-09-21T21:43:16.994 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.994 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.994 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:16.994 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965566+0000", 2024-09-21T21:43:16.994 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3393999999999998e-05 2024-09-21T21:43:16.994 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.994 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.994 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:16.994 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965616+0000", 2024-09-21T21:43:16.994 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9666000000000002e-05 2024-09-21T21:43:16.994 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:16.994 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:16.995 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:16.995 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.995 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.995 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6991 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5361664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:16.995 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923132+0000", 2024-09-21T21:43:16.995 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 107.02822043899999, 2024-09-21T21:43:16.995 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042600779999999998, 2024-09-21T21:43:16.995 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:16.995 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:16.995 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:16.995 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:16.995 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:16.996 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:16.996 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6991 2024-09-21T21:43:16.996 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.996 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:16.996 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.996 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:16.996 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923132+0000", 2024-09-21T21:43:16.996 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:16.996 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.996 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.996 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:16.996 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923132+0000", 2024-09-21T21:43:16.996 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:16.996 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.997 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.997 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:16.997 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923132+0000", 2024-09-21T21:43:16.997 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4200000000000002e-07 2024-09-21T21:43:16.997 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.997 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.997 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:16.997 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923134+0000", 2024-09-21T21:43:16.997 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2270000000000001e-06 2024-09-21T21:43:16.997 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.997 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.997 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:16.998 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923137+0000", 2024-09-21T21:43:16.998 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1930000000000002e-06 2024-09-21T21:43:16.998 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.998 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.998 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:16.998 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923139+0000", 2024-09-21T21:43:16.998 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4710000000000001e-06 2024-09-21T21:43:16.998 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.998 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.998 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:16.998 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965639+0000", 2024-09-21T21:43:16.998 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.04249965 2024-09-21T21:43:16.998 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.998 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.998 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:16.999 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965679+0000", 2024-09-21T21:43:16.999 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9852999999999999e-05 2024-09-21T21:43:16.999 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:16.999 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:16.999 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:16.999 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965733+0000", 2024-09-21T21:43:16.999 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4144000000000002e-05 2024-09-21T21:43:16.999 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:16.999 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:16.999 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:16.999 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.000 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.000 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6993 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5384192~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:17.000 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923143+0000", 2024-09-21T21:43:17.000 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 107.028209645, 2024-09-21T21:43:17.000 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042708182999999997, 2024-09-21T21:43:17.000 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:17.000 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:17.000 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:17.000 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:17.000 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:17.000 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:17.000 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6993 2024-09-21T21:43:17.000 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.000 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:17.000 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.001 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:17.001 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923143+0000", 2024-09-21T21:43:17.001 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:17.001 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.001 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.001 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:17.001 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923143+0000", 2024-09-21T21:43:17.001 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:17.001 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.001 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.001 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:17.001 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923143+0000", 2024-09-21T21:43:17.002 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0800000000000001e-07 2024-09-21T21:43:17.002 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.002 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.002 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:17.002 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923144+0000", 2024-09-21T21:43:17.002 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.009e-06 2024-09-21T21:43:17.002 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.002 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.002 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:17.002 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923148+0000", 2024-09-21T21:43:17.002 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.737e-06 2024-09-21T21:43:17.002 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.003 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.003 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:17.003 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923150+0000", 2024-09-21T21:43:17.003 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4669999999999998e-06 2024-09-21T21:43:17.003 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.003 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.003 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:17.003 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965760+0000", 2024-09-21T21:43:17.003 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042609399999999999 2024-09-21T21:43:17.003 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.003 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.003 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:17.003 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965801+0000", 2024-09-21T21:43:17.004 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1399999999999997e-05 2024-09-21T21:43:17.004 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.004 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.004 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:17.004 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965851+0000", 2024-09-21T21:43:17.004 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9962e-05 2024-09-21T21:43:17.004 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:17.004 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:17.004 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:17.004 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.004 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.004 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6995 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5398528~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:17.004 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923153+0000", 2024-09-21T21:43:17.005 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 107.028199281, 2024-09-21T21:43:17.005 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042828034000000001, 2024-09-21T21:43:17.005 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:17.005 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:17.005 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:17.005 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:17.005 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:17.005 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:17.005 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6995 2024-09-21T21:43:17.005 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.005 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:17.005 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.005 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:17.005 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923153+0000", 2024-09-21T21:43:17.006 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:17.006 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.006 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.006 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:17.006 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923153+0000", 2024-09-21T21:43:17.006 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:17.006 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.006 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.006 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:17.006 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923154+0000", 2024-09-21T21:43:17.006 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2600000000000001e-07 2024-09-21T21:43:17.006 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.007 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.007 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:17.007 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923158+0000", 2024-09-21T21:43:17.007 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0559999999999998e-06 2024-09-21T21:43:17.007 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.007 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.007 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:17.007 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923161+0000", 2024-09-21T21:43:17.007 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6519999999999999e-06 2024-09-21T21:43:17.007 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.007 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.007 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:17.007 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923170+0000", 2024-09-21T21:43:17.007 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5720000000000003e-06 2024-09-21T21:43:17.008 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.008 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.008 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:17.008 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965887+0000", 2024-09-21T21:43:17.008 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042716863000000001 2024-09-21T21:43:17.008 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.008 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.008 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:17.008 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965927+0000", 2024-09-21T21:43:17.008 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0346999999999998e-05 2024-09-21T21:43:17.008 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.008 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.008 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:17.008 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965981+0000", 2024-09-21T21:43:17.008 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4317999999999997e-05 2024-09-21T21:43:17.009 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:17.009 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:17.009 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:17.009 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.009 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.009 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6997 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5412864~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:17.009 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923174+0000", 2024-09-21T21:43:17.009 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 107.028179126, 2024-09-21T21:43:17.009 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042928296999999997, 2024-09-21T21:43:17.009 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:17.009 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:17.009 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:17.009 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:17.009 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:17.010 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:17.010 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6997 2024-09-21T21:43:17.010 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.010 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:17.010 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.010 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:17.010 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923174+0000", 2024-09-21T21:43:17.010 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:17.010 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.010 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.010 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:17.010 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923174+0000", 2024-09-21T21:43:17.010 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:17.010 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.010 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.011 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:17.011 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923174+0000", 2024-09-21T21:43:17.011 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.67e-07 2024-09-21T21:43:17.011 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.011 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.011 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:17.011 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923178+0000", 2024-09-21T21:43:17.011 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1330000000000001e-06 2024-09-21T21:43:17.011 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.011 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.011 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:17.011 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923182+0000", 2024-09-21T21:43:17.011 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.794e-06 2024-09-21T21:43:17.011 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.011 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.012 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:17.012 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923184+0000", 2024-09-21T21:43:17.012 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.435e-06 2024-09-21T21:43:17.012 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.012 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.012 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:17.012 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966009+0000", 2024-09-21T21:43:17.012 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042824327000000002 2024-09-21T21:43:17.012 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.012 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.012 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:17.012 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966052+0000", 2024-09-21T21:43:17.012 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3034000000000002e-05 2024-09-21T21:43:17.012 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.012 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.013 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:17.013 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966102+0000", 2024-09-21T21:43:17.013 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0306999999999998e-05 2024-09-21T21:43:17.013 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:17.013 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:17.013 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:17.013 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.013 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.013 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6999 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5427200~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:17.013 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923187+0000", 2024-09-21T21:43:17.013 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 107.028165398, 2024-09-21T21:43:17.013 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043104267000000002, 2024-09-21T21:43:17.013 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:17.013 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:17.013 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:17.014 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:17.014 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:17.014 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:17.014 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6999 2024-09-21T21:43:17.014 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.014 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:17.014 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.014 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:17.014 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923187+0000", 2024-09-21T21:43:17.014 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:17.014 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.014 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.014 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:17.014 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923187+0000", 2024-09-21T21:43:17.015 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:17.015 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.015 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.015 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:17.015 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923188+0000", 2024-09-21T21:43:17.015 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3999999999999998e-07 2024-09-21T21:43:17.015 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.015 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.015 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:17.016 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923189+0000", 2024-09-21T21:43:17.016 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4169999999999999e-06 2024-09-21T21:43:17.016 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.016 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.017 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:17.017 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923194+0000", 2024-09-21T21:43:17.017 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1819999999999997e-06 2024-09-21T21:43:17.018 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.018 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.018 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:17.018 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923198+0000", 2024-09-21T21:43:17.018 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.143e-06 2024-09-21T21:43:17.018 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.018 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.019 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:17.019 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966135+0000", 2024-09-21T21:43:17.019 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042936297999999998 2024-09-21T21:43:17.019 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.019 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.019 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:17.019 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966204+0000", 2024-09-21T21:43:17.020 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8989e-05 2024-09-21T21:43:17.020 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.020 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.020 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:17.020 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966292+0000", 2024-09-21T21:43:17.021 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7997999999999993e-05 2024-09-21T21:43:17.021 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:17.021 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:17.021 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:17.021 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.021 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.021 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7001 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5441536~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:17.021 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923211+0000", 2024-09-21T21:43:17.021 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 107.02814160699999, 2024-09-21T21:43:17.021 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043196518000000003, 2024-09-21T21:43:17.021 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:17.022 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:17.022 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:17.022 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:17.022 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:17.022 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:17.022 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7001 2024-09-21T21:43:17.022 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.022 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:17.022 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.022 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:17.022 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923211+0000", 2024-09-21T21:43:17.022 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:17.022 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.022 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.023 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:17.023 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923211+0000", 2024-09-21T21:43:17.023 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:17.023 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.023 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.023 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:17.023 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923212+0000", 2024-09-21T21:43:17.023 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3900000000000005e-07 2024-09-21T21:43:17.023 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.024 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.024 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:17.024 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923214+0000", 2024-09-21T21:43:17.024 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9970000000000001e-06 2024-09-21T21:43:17.024 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.024 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.024 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:17.025 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923219+0000", 2024-09-21T21:43:17.025 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4310000000000002e-06 2024-09-21T21:43:17.025 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.025 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.025 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:17.025 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923224+0000", 2024-09-21T21:43:17.025 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6410000000000001e-06 2024-09-21T21:43:17.025 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.026 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.026 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:17.026 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966318+0000", 2024-09-21T21:43:17.026 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043093842 2024-09-21T21:43:17.026 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.026 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.026 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:17.027 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966359+0000", 2024-09-21T21:43:17.027 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1260999999999997e-05 2024-09-21T21:43:17.027 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.027 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.027 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:17.027 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966408+0000", 2024-09-21T21:43:17.027 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8807000000000002e-05 2024-09-21T21:43:17.027 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:17.027 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:17.027 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:17.027 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.027 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.027 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7002 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5455872~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:17.028 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923229+0000", 2024-09-21T21:43:17.028 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 107.028123197, 2024-09-21T21:43:17.028 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043364568999999999, 2024-09-21T21:43:17.028 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:17.028 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:17.028 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:17.028 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:17.028 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:17.028 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:17.028 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7002 2024-09-21T21:43:17.028 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.028 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:17.028 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.028 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:17.029 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923229+0000", 2024-09-21T21:43:17.029 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:17.029 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.029 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.029 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:17.029 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923229+0000", 2024-09-21T21:43:17.029 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:17.029 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.029 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.029 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:17.029 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923230+0000", 2024-09-21T21:43:17.029 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4700000000000002e-07 2024-09-21T21:43:17.029 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.029 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.030 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:17.030 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923232+0000", 2024-09-21T21:43:17.030 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0719999999999998e-06 2024-09-21T21:43:17.030 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.030 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.030 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:17.030 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923237+0000", 2024-09-21T21:43:17.030 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2290000000000002e-06 2024-09-21T21:43:17.030 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.030 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.030 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:17.030 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923245+0000", 2024-09-21T21:43:17.030 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2959999999999999e-06 2024-09-21T21:43:17.030 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.031 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.031 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:17.031 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966438+0000", 2024-09-21T21:43:17.031 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043193027000000002 2024-09-21T21:43:17.031 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.031 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.031 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:17.031 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966506+0000", 2024-09-21T21:43:17.031 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7707999999999996e-05 2024-09-21T21:43:17.031 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.031 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.031 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:17.032 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966594+0000", 2024-09-21T21:43:17.032 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.789e-05 2024-09-21T21:43:17.032 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:17.032 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:17.032 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:17.032 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.032 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.032 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7005 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5470208~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:17.032 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923254+0000", 2024-09-21T21:43:17.032 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 107.028098966, 2024-09-21T21:43:17.032 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043457176, 2024-09-21T21:43:17.032 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:17.032 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:17.032 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:17.032 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:17.033 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:17.033 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:17.033 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7005 2024-09-21T21:43:17.033 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.033 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:17.033 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.033 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:17.034 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923254+0000", 2024-09-21T21:43:17.034 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:17.034 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.034 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.034 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:17.034 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923254+0000", 2024-09-21T21:43:17.034 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:17.034 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.034 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.034 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:17.034 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923254+0000", 2024-09-21T21:43:17.035 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4700000000000002e-07 2024-09-21T21:43:17.035 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.035 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.035 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:17.035 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923256+0000", 2024-09-21T21:43:17.035 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.007e-06 2024-09-21T21:43:17.035 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.035 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.035 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:17.035 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923266+0000", 2024-09-21T21:43:17.035 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.4700000000000008e-06 2024-09-21T21:43:17.035 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.035 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.036 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:17.036 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923271+0000", 2024-09-21T21:43:17.036 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2809999999999999e-06 2024-09-21T21:43:17.036 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.036 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.036 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:17.036 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966619+0000", 2024-09-21T21:43:17.036 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043347951000000003 2024-09-21T21:43:17.036 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.036 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.036 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:17.036 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966661+0000", 2024-09-21T21:43:17.037 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2614999999999998e-05 2024-09-21T21:43:17.037 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.037 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.037 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:17.037 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966711+0000", 2024-09-21T21:43:17.037 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9505000000000001e-05 2024-09-21T21:43:17.037 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:17.037 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:17.037 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:17.037 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.037 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.037 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7008 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5484544~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:17.037 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923278+0000", 2024-09-21T21:43:17.038 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 107.028074319, 2024-09-21T21:43:17.038 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043631071, 2024-09-21T21:43:17.038 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:17.038 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:17.038 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:17.038 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:17.038 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:17.038 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:17.038 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7008 2024-09-21T21:43:17.038 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.038 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:17.038 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.039 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:17.039 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923278+0000", 2024-09-21T21:43:17.039 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:17.039 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.039 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.039 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:17.039 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923278+0000", 2024-09-21T21:43:17.039 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:17.039 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.039 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.039 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:17.039 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923279+0000", 2024-09-21T21:43:17.039 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.77e-07 2024-09-21T21:43:17.040 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.040 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.040 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:17.040 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923281+0000", 2024-09-21T21:43:17.040 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0219999999999999e-06 2024-09-21T21:43:17.040 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.040 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.040 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:17.040 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923287+0000", 2024-09-21T21:43:17.040 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7799999999999997e-06 2024-09-21T21:43:17.040 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.040 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.041 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:17.041 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923291+0000", 2024-09-21T21:43:17.041 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1699999999999999e-06 2024-09-21T21:43:17.041 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.041 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.041 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:17.041 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966742+0000", 2024-09-21T21:43:17.041 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043451344000000003 2024-09-21T21:43:17.041 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.041 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.041 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:17.041 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966809+0000", 2024-09-21T21:43:17.041 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7070999999999995e-05 2024-09-21T21:43:17.042 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.042 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.042 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:17.042 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966909+0000", 2024-09-21T21:43:17.042 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000100307 2024-09-21T21:43:17.042 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:17.042 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:17.042 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:17.042 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.042 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.042 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7010 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5498880~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:17.042 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923301+0000", 2024-09-21T21:43:17.043 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 107.028051424, 2024-09-21T21:43:17.043 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043717830999999999, 2024-09-21T21:43:17.043 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:17.043 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:17.043 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:17.043 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:17.043 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:17.043 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:17.043 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7010 2024-09-21T21:43:17.043 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.043 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:17.043 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.043 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:17.043 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923301+0000", 2024-09-21T21:43:17.043 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:17.044 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.044 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.044 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:17.044 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923301+0000", 2024-09-21T21:43:17.044 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:17.044 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.044 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.044 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:17.044 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923302+0000", 2024-09-21T21:43:17.044 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.41e-07 2024-09-21T21:43:17.044 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.044 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.044 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:17.044 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923304+0000", 2024-09-21T21:43:17.045 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0990000000000001e-06 2024-09-21T21:43:17.045 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.045 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.045 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:17.045 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923310+0000", 2024-09-21T21:43:17.045 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3860000000000004e-06 2024-09-21T21:43:17.045 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.045 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.045 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:17.045 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923315+0000", 2024-09-21T21:43:17.045 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8459999999999999e-06 2024-09-21T21:43:17.045 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.045 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.045 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:17.046 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966938+0000", 2024-09-21T21:43:17.046 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043622693999999997 2024-09-21T21:43:17.046 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.046 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.046 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:17.046 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966975+0000", 2024-09-21T21:43:17.046 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7608999999999999e-05 2024-09-21T21:43:17.046 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.046 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.046 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:17.046 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.967019+0000", 2024-09-21T21:43:17.046 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3856000000000002e-05 2024-09-21T21:43:17.046 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:17.046 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:17.047 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:17.047 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.047 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.047 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7354 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 2107392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:17.047 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.936932+0000", 2024-09-21T21:43:17.047 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 107.01442056899999, 2024-09-21T21:43:17.047 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041607708, 2024-09-21T21:43:17.047 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:17.047 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:17.047 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:17.047 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:17.047 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:17.047 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:17.047 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7354 2024-09-21T21:43:17.047 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.048 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:17.048 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.048 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:17.048 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.936932+0000", 2024-09-21T21:43:17.048 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:17.048 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.048 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.048 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:17.048 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.936932+0000", 2024-09-21T21:43:17.048 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:17.048 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.048 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.048 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:17.048 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.937276+0000", 2024-09-21T21:43:17.049 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00034344200000000001 2024-09-21T21:43:17.049 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.049 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.049 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:17.049 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.937279+0000", 2024-09-21T21:43:17.049 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.844e-06 2024-09-21T21:43:17.049 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.049 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.049 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:17.049 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.937288+0000", 2024-09-21T21:43:17.049 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2409999999999995e-06 2024-09-21T21:43:17.049 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.049 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.049 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:17.049 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.937293+0000", 2024-09-21T21:43:17.050 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6559999999999997e-06 2024-09-21T21:43:17.050 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.050 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.050 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:17.050 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.978448+0000", 2024-09-21T21:43:17.050 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041154621000000002 2024-09-21T21:43:17.050 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.050 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.050 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:17.050 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.978487+0000", 2024-09-21T21:43:17.050 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9254000000000001e-05 2024-09-21T21:43:17.051 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.051 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.051 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:17.051 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.978540+0000", 2024-09-21T21:43:17.051 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2649999999999999e-05 2024-09-21T21:43:17.051 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:17.051 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:17.051 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:17.051 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.051 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.051 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5215.0:1054 3.9 3:946d2401:::rbd_data.1121524d2522.000000000000000b:head [set-alloc-hint object_size 16777216 write_size 16777216,write 6615040~59392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-21T21:43:17.051 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:38.148979+0000", 2024-09-21T21:43:17.051 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 98.802373602000003, 2024-09-21T21:43:17.052 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047870388999999999, 2024-09-21T21:43:17.052 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:17.052 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:17.052 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:17.052 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:17.052 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5215", 2024-09-21T21:43:17.052 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1580282168", 2024-09-21T21:43:17.052 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1054 2024-09-21T21:43:17.052 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.052 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:17.052 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.052 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:17.052 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.148979+0000", 2024-09-21T21:43:17.052 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:17.053 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.053 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.053 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:17.053 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.148979+0000", 2024-09-21T21:43:17.053 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:17.053 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.053 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.053 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:17.053 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.148980+0000", 2024-09-21T21:43:17.053 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5799999999999999e-07 2024-09-21T21:43:17.053 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.053 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.053 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:17.053 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.149039+0000", 2024-09-21T21:43:17.054 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9151000000000003e-05 2024-09-21T21:43:17.054 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.054 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.054 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:17.054 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.149047+0000", 2024-09-21T21:43:17.054 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1000000000000004e-06 2024-09-21T21:43:17.054 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.054 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.054 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:17.054 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.149052+0000", 2024-09-21T21:43:17.054 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5959999999999997e-06 2024-09-21T21:43:17.054 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.054 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.055 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:17.055 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.195288+0000", 2024-09-21T21:43:17.055 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.046236248000000001 2024-09-21T21:43:17.055 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.055 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.055 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:17.055 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.195334+0000", 2024-09-21T21:43:17.055 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6575e-05 2024-09-21T21:43:17.055 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.055 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.055 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 2", 2024-09-21T21:43:17.055 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.195406+0000", 2024-09-21T21:43:17.055 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1715e-05 2024-09-21T21:43:17.055 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.056 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.056 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:17.056 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.196018+0000", 2024-09-21T21:43:17.056 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00061218999999999998 2024-09-21T21:43:17.056 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.056 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.056 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:17.056 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.196794+0000", 2024-09-21T21:43:17.056 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000775814 2024-09-21T21:43:17.056 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.056 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.056 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:17.056 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.196827+0000", 2024-09-21T21:43:17.056 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2419000000000003e-05 2024-09-21T21:43:17.057 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.057 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.057 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:17.057 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.196849+0000", 2024-09-21T21:43:17.057 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2923000000000001e-05 2024-09-21T21:43:17.057 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:17.057 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:17.057 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:17.057 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:17.057 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:17.057 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:43:17.059 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T21:43:17.059 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:43:17.057+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:43:17.159 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:43:17.157+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:43:17.207 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:43:17.207 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:43:17.207 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-21T21:43:17.207 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:43:17.217 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T21:43:17.260 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:43:17.258+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:43:17.331 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T21:43:17.367 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:43:17.367 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:43:17.367 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:43:17.367 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:43:17.367 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:43:17.375 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:43:17.374+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:43:17.377 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T21:43:17.461 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:43:17.459+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:43:17.495 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:43:17.495 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-21T21:43:17.495 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-21T21:43:17.495 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:43:17.495 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.495 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4385.0:15202 3.16 3:6888948b:::rbd_header.1121493f6459:head [watch reconnect cookie 140325440202496 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-21T21:43:17.495 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:41.736601+0000", 2024-09-21T21:43:17.495 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 95.755555732000005, 2024-09-21T21:43:17.495 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5040056769999999, 2024-09-21T21:43:17.495 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:17.496 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:17.496 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:17.496 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:17.496 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4385", 2024-09-21T21:43:17.496 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:43:17.496 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 15202 2024-09-21T21:43:17.496 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.496 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:17.496 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.496 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:17.496 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736601+0000", 2024-09-21T21:43:17.496 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:17.496 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.496 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.496 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:17.497 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736601+0000", 2024-09-21T21:43:17.497 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:17.497 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.497 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.497 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:17.497 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736602+0000", 2024-09-21T21:43:17.497 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0699999999999999e-06 2024-09-21T21:43:17.497 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.497 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.497 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:17.497 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736607+0000", 2024-09-21T21:43:17.498 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2520000000000001e-06 2024-09-21T21:43:17.498 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.498 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.498 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:17.498 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736617+0000", 2024-09-21T21:43:17.498 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0074e-05 2024-09-21T21:43:17.498 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.498 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.498 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:17.498 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736622+0000", 2024-09-21T21:43:17.498 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.101e-06 2024-09-21T21:43:17.498 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.498 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.499 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:17.499 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736664+0000", 2024-09-21T21:43:17.499 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1869999999999997e-05 2024-09-21T21:43:17.499 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.499 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.499 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op must wait for map", 2024-09-21T21:43:17.499 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736667+0000", 2024-09-21T21:43:17.499 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4750000000000002e-06 2024-09-21T21:43:17.499 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.499 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.499 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:17.499 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742886+0000", 2024-09-21T21:43:17.499 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0062187579999999996 2024-09-21T21:43:17.499 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.500 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.500 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:17.500 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742892+0000", 2024-09-21T21:43:17.500 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0440000000000001e-06 2024-09-21T21:43:17.500 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.500 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.500 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:17.500 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210307+0000", 2024-09-21T21:43:17.500 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.467414781 2024-09-21T21:43:17.500 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.500 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.500 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:17.500 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210322+0000", 2024-09-21T21:43:17.500 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5311e-05 2024-09-21T21:43:17.501 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.501 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.501 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:17.501 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.237624+0000", 2024-09-21T21:43:17.501 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0273018920000001 2024-09-21T21:43:17.501 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.501 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.501 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:17.501 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.237870+0000", 2024-09-21T21:43:17.501 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00024622899999999999 2024-09-21T21:43:17.501 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.501 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.501 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:17.501 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.240607+0000", 2024-09-21T21:43:17.501 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0027368200000000001 2024-09-21T21:43:17.502 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:17.502 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:17.502 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:17.502 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.502 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.502 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5415.0:379 3.16 3:6888948b:::rbd_header.1121493f6459:head [watch reconnect cookie 139702602617920 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-21T21:43:17.502 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:41.739544+0000", 2024-09-21T21:43:17.502 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 95.752613303000004, 2024-09-21T21:43:17.502 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.501180645, 2024-09-21T21:43:17.502 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:17.502 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:17.502 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:17.502 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:17.502 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5415", 2024-09-21T21:43:17.503 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1212816607", 2024-09-21T21:43:17.503 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 379 2024-09-21T21:43:17.503 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.503 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:17.503 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.503 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:17.503 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739544+0000", 2024-09-21T21:43:17.503 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:17.503 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.503 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.503 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:17.503 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739544+0000", 2024-09-21T21:43:17.503 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:17.503 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.503 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.504 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:17.504 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739545+0000", 2024-09-21T21:43:17.504 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.065e-06 2024-09-21T21:43:17.504 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.504 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.504 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:17.504 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739550+0000", 2024-09-21T21:43:17.504 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9599999999999999e-06 2024-09-21T21:43:17.504 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.504 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.504 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:17.504 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739565+0000", 2024-09-21T21:43:17.504 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5709000000000002e-05 2024-09-21T21:43:17.504 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.505 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.505 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:17.505 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739573+0000", 2024-09-21T21:43:17.505 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7849999999999992e-06 2024-09-21T21:43:17.505 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.505 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.505 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:17.505 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739634+0000", 2024-09-21T21:43:17.505 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0732000000000001e-05 2024-09-21T21:43:17.505 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.505 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.505 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op must wait for map", 2024-09-21T21:43:17.505 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739640+0000", 2024-09-21T21:43:17.506 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0000000000000002e-06 2024-09-21T21:43:17.506 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.506 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.506 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:17.506 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742958+0000", 2024-09-21T21:43:17.506 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0033182390000000002 2024-09-21T21:43:17.506 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.506 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.506 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:17.506 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742966+0000", 2024-09-21T21:43:17.506 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1610000000000003e-06 2024-09-21T21:43:17.506 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.506 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.506 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:17.507 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210405+0000", 2024-09-21T21:43:17.507 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.467438725 2024-09-21T21:43:17.507 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.507 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.507 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:17.507 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210415+0000", 2024-09-21T21:43:17.507 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0234000000000001e-05 2024-09-21T21:43:17.507 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.507 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.507 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:17.507 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.238743+0000", 2024-09-21T21:43:17.507 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0283273070000001 2024-09-21T21:43:17.507 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.507 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.507 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:17.508 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.238842+0000", 2024-09-21T21:43:17.508 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.9896999999999996e-05 2024-09-21T21:43:17.508 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.508 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.508 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:17.508 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.240724+0000", 2024-09-21T21:43:17.508 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001881831 2024-09-21T21:43:17.508 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:17.508 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:17.508 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:17.508 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.508 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.508 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4385.0:15203 3.16 3:6888948b:::rbd_header.1121493f6459:head [call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-21T21:43:17.508 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:41.739737+0000", 2024-09-21T21:43:17.509 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 95.752420291999996, 2024-09-21T21:43:17.509 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4989621799999999, 2024-09-21T21:43:17.509 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:17.509 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:17.509 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:17.509 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:17.509 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4385", 2024-09-21T21:43:17.509 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:43:17.509 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 15203 2024-09-21T21:43:17.509 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.509 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:17.509 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.509 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:17.509 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739737+0000", 2024-09-21T21:43:17.509 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:17.510 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.510 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.510 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:17.510 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739737+0000", 2024-09-21T21:43:17.510 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:17.510 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.510 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.510 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:17.510 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739738+0000", 2024-09-21T21:43:17.510 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.7499999999999998e-07 2024-09-21T21:43:17.510 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.510 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.510 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:17.510 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739743+0000", 2024-09-21T21:43:17.511 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5049999999999998e-06 2024-09-21T21:43:17.511 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.511 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.511 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:17.511 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739757+0000", 2024-09-21T21:43:17.511 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3947e-05 2024-09-21T21:43:17.511 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.511 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.511 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:17.511 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739765+0000", 2024-09-21T21:43:17.511 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9310000000000008e-06 2024-09-21T21:43:17.511 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.511 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.511 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:17.512 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739816+0000", 2024-09-21T21:43:17.512 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0559999999999997e-05 2024-09-21T21:43:17.512 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.512 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.512 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting_for_map not empty", 2024-09-21T21:43:17.512 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739821+0000", 2024-09-21T21:43:17.512 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3539999999999999e-06 2024-09-21T21:43:17.512 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.512 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.512 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:17.512 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742928+0000", 2024-09-21T21:43:17.512 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0031074399999999999 2024-09-21T21:43:17.512 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.512 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.513 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:17.513 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742937+0000", 2024-09-21T21:43:17.513 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1289999999999996e-06 2024-09-21T21:43:17.513 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.513 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.513 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:17.513 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210350+0000", 2024-09-21T21:43:17.513 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.46741311299999999 2024-09-21T21:43:17.513 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.513 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.514 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:17.514 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210358+0000", 2024-09-21T21:43:17.514 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1869999999999997e-06 2024-09-21T21:43:17.514 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.514 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.514 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:17.514 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.238398+0000", 2024-09-21T21:43:17.514 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0280397830000001 2024-09-21T21:43:17.514 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.514 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.514 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:17.514 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.238489+0000", 2024-09-21T21:43:17.514 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.0958000000000006e-05 2024-09-21T21:43:17.514 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.515 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.515 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:17.515 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.238699+0000", 2024-09-21T21:43:17.515 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00021029800000000001 2024-09-21T21:43:17.515 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:17.515 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:17.515 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:17.515 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.515 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.515 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4385.0:15204 3.16 3:6888948b:::rbd_header.1121493f6459:head [watch reconnect cookie 140325440202496 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-21T21:43:17.515 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:42.210726+0000", 2024-09-21T21:43:17.515 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 95.281431413999996, 2024-09-21T21:43:17.515 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.030304557, 2024-09-21T21:43:17.515 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:17.515 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:17.516 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:17.516 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:17.516 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4385", 2024-09-21T21:43:17.516 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:43:17.516 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 15204 2024-09-21T21:43:17.516 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.516 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:17.516 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.516 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:17.516 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210726+0000", 2024-09-21T21:43:17.516 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:17.516 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.516 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.516 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:17.516 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210726+0000", 2024-09-21T21:43:17.517 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:17.517 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.517 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.517 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:17.517 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210727+0000", 2024-09-21T21:43:17.517 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.204e-06 2024-09-21T21:43:17.517 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.517 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.517 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:17.517 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210732+0000", 2024-09-21T21:43:17.517 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5249999999999996e-06 2024-09-21T21:43:17.517 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.517 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.517 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:17.517 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210744+0000", 2024-09-21T21:43:17.518 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1405e-05 2024-09-21T21:43:17.518 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.518 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.518 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:17.518 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210754+0000", 2024-09-21T21:43:17.518 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0122999999999999e-05 2024-09-21T21:43:17.518 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.518 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.518 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:17.518 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210798+0000", 2024-09-21T21:43:17.518 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4005000000000003e-05 2024-09-21T21:43:17.518 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.518 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.518 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:17.519 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210803+0000", 2024-09-21T21:43:17.519 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5130000000000004e-06 2024-09-21T21:43:17.519 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.519 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.519 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:17.519 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.239260+0000", 2024-09-21T21:43:17.519 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0284563040000001 2024-09-21T21:43:17.519 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.519 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.519 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:17.519 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.239318+0000", 2024-09-21T21:43:17.519 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8486000000000002e-05 2024-09-21T21:43:17.519 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.519 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.519 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:17.519 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.241030+0000", 2024-09-21T21:43:17.520 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0017119920000000001 2024-09-21T21:43:17.520 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:17.520 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:17.520 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:17.520 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.520 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.520 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5415.0:380 3.16 3:6888948b:::rbd_header.1121493f6459:head [watch reconnect cookie 139702602617920 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-21T21:43:17.520 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:42.210777+0000", 2024-09-21T21:43:17.520 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 95.281379634999993, 2024-09-21T21:43:17.520 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.030410566, 2024-09-21T21:43:17.520 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:17.520 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:17.520 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:17.520 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:17.521 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5415", 2024-09-21T21:43:17.521 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1212816607", 2024-09-21T21:43:17.521 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 380 2024-09-21T21:43:17.521 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.521 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:17.521 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.521 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:17.521 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210777+0000", 2024-09-21T21:43:17.521 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:17.521 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.521 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.521 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:17.521 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210777+0000", 2024-09-21T21:43:17.521 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:17.521 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.522 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.522 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:17.522 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210778+0000", 2024-09-21T21:43:17.522 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8999999999999996e-07 2024-09-21T21:43:17.522 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.522 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.522 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:17.522 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210780+0000", 2024-09-21T21:43:17.522 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.503e-06 2024-09-21T21:43:17.522 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.522 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.522 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:17.522 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210788+0000", 2024-09-21T21:43:17.523 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0999999999999998e-06 2024-09-21T21:43:17.523 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.523 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.523 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:17.523 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210799+0000", 2024-09-21T21:43:17.523 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1703e-05 2024-09-21T21:43:17.523 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.523 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.523 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:17.523 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210833+0000", 2024-09-21T21:43:17.523 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4008999999999999e-05 2024-09-21T21:43:17.523 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.523 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.523 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:17.524 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210838+0000", 2024-09-21T21:43:17.524 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7480000000000001e-06 2024-09-21T21:43:17.524 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.524 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.524 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:17.524 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.239612+0000", 2024-09-21T21:43:17.524 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.028773554 2024-09-21T21:43:17.524 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.524 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.524 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:17.524 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.239677+0000", 2024-09-21T21:43:17.524 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5053000000000002e-05 2024-09-21T21:43:17.524 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.525 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.525 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:17.525 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.241188+0000", 2024-09-21T21:43:17.525 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001511306 2024-09-21T21:43:17.525 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:17.525 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:17.525 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:17.525 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.525 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.525 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24794 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:17.525 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.584085+0000", 2024-09-21T21:43:17.525 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 54.908072459000003, 2024-09-21T21:43:17.525 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.096209996000000006, 2024-09-21T21:43:17.525 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:17.526 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:17.526 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:17.526 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:17.526 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:17.526 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:17.526 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24794 2024-09-21T21:43:17.526 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.526 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:17.526 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.526 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:17.526 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584085+0000", 2024-09-21T21:43:17.526 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:17.526 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.527 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.527 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:17.527 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584085+0000", 2024-09-21T21:43:17.527 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:17.527 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.527 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.527 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:17.527 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584085+0000", 2024-09-21T21:43:17.527 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3200000000000001e-07 2024-09-21T21:43:17.527 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.527 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.527 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:17.527 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584086+0000", 2024-09-21T21:43:17.527 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.717e-06 2024-09-21T21:43:17.528 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.528 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.528 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:17.528 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584090+0000", 2024-09-21T21:43:17.528 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3450000000000002e-06 2024-09-21T21:43:17.528 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.528 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.528 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:17.528 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584092+0000", 2024-09-21T21:43:17.528 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6359999999999998e-06 2024-09-21T21:43:17.528 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.528 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.528 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:17.528 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.676497+0000", 2024-09-21T21:43:17.529 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.092404659 2024-09-21T21:43:17.529 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.529 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.529 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:17.529 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.676567+0000", 2024-09-21T21:43:17.529 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9627000000000003e-05 2024-09-21T21:43:17.529 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.529 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.529 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:17.529 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.676925+0000", 2024-09-21T21:43:17.529 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00035848500000000002 2024-09-21T21:43:17.529 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.529 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.529 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:17.530 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.678427+0000", 2024-09-21T21:43:17.530 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001501638 2024-09-21T21:43:17.530 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.530 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.530 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:17.530 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.680204+0000", 2024-09-21T21:43:17.530 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0017766979999999999 2024-09-21T21:43:17.530 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.530 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.530 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:17.530 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.680256+0000", 2024-09-21T21:43:17.530 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2549999999999997e-05 2024-09-21T21:43:17.530 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.531 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.531 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:17.531 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.680295+0000", 2024-09-21T21:43:17.531 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8408999999999998e-05 2024-09-21T21:43:17.531 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:17.531 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:17.531 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:17.531 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.531 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.531 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24796 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:17.531 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.584099+0000", 2024-09-21T21:43:17.531 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 54.908057890000002, 2024-09-21T21:43:17.531 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.097297429000000005, 2024-09-21T21:43:17.531 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:17.531 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:17.532 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:17.532 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:17.532 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:17.532 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:17.532 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24796 2024-09-21T21:43:17.532 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.532 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:17.532 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.532 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:17.532 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584099+0000", 2024-09-21T21:43:17.532 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:17.532 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.532 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.533 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:17.533 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584099+0000", 2024-09-21T21:43:17.533 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:17.533 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.533 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.533 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:17.533 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584099+0000", 2024-09-21T21:43:17.533 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3999999999999998e-07 2024-09-21T21:43:17.533 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.533 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.533 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:17.534 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584101+0000", 2024-09-21T21:43:17.534 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5850000000000001e-06 2024-09-21T21:43:17.534 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.534 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.534 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:17.534 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584108+0000", 2024-09-21T21:43:17.534 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.498e-06 2024-09-21T21:43:17.534 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.534 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.534 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:17.534 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584112+0000", 2024-09-21T21:43:17.534 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.252e-06 2024-09-21T21:43:17.534 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.534 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.535 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:17.535 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.677336+0000", 2024-09-21T21:43:17.535 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.093224206000000004 2024-09-21T21:43:17.535 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.535 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.535 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:17.535 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.677411+0000", 2024-09-21T21:43:17.535 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5036000000000005e-05 2024-09-21T21:43:17.535 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.535 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.535 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:17.535 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.677782+0000", 2024-09-21T21:43:17.535 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00037153099999999998 2024-09-21T21:43:17.535 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.535 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.536 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:17.536 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.680360+0000", 2024-09-21T21:43:17.536 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025773020000000001 2024-09-21T21:43:17.536 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.536 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.536 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:17.536 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681313+0000", 2024-09-21T21:43:17.536 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00095341100000000004 2024-09-21T21:43:17.536 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.536 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.536 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:17.536 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681361+0000", 2024-09-21T21:43:17.536 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7497999999999998e-05 2024-09-21T21:43:17.537 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.537 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.537 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:17.537 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681397+0000", 2024-09-21T21:43:17.537 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.587e-05 2024-09-21T21:43:17.537 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:17.537 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:17.537 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:17.537 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.537 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.537 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24797 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:17.537 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.584116+0000", 2024-09-21T21:43:17.537 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 54.908041400999998, 2024-09-21T21:43:17.537 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.097428257000000004, 2024-09-21T21:43:17.537 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:17.538 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:17.538 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:17.538 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:17.538 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:17.538 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:17.538 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24797 2024-09-21T21:43:17.538 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.538 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:17.538 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.538 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:17.538 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584116+0000", 2024-09-21T21:43:17.538 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:17.538 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.538 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.539 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:17.539 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584116+0000", 2024-09-21T21:43:17.539 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:17.539 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.539 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.539 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:17.539 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584116+0000", 2024-09-21T21:43:17.539 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.53e-07 2024-09-21T21:43:17.539 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.539 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.539 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:17.539 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584118+0000", 2024-09-21T21:43:17.539 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6810000000000001e-06 2024-09-21T21:43:17.539 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.540 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.540 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:17.540 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584121+0000", 2024-09-21T21:43:17.540 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1099999999999999e-06 2024-09-21T21:43:17.540 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.540 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.540 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:17.540 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584123+0000", 2024-09-21T21:43:17.540 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5890000000000001e-06 2024-09-21T21:43:17.540 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.540 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.540 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:17.540 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.678476+0000", 2024-09-21T21:43:17.540 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.094352320000000003 2024-09-21T21:43:17.540 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.541 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.541 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:17.541 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.678556+0000", 2024-09-21T21:43:17.541 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0291999999999996e-05 2024-09-21T21:43:17.541 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.541 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.541 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:17.541 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.678871+0000", 2024-09-21T21:43:17.541 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000314689 2024-09-21T21:43:17.541 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.541 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.541 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:17.541 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681452+0000", 2024-09-21T21:43:17.541 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025817890000000001 2024-09-21T21:43:17.541 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.542 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.542 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:17.542 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681467+0000", 2024-09-21T21:43:17.542 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4256e-05 2024-09-21T21:43:17.542 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.542 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.542 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:17.542 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681510+0000", 2024-09-21T21:43:17.542 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3560999999999998e-05 2024-09-21T21:43:17.542 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.542 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.542 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:17.542 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681544+0000", 2024-09-21T21:43:17.542 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3717000000000003e-05 2024-09-21T21:43:17.542 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:17.543 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:17.543 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:17.543 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.543 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.543 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24798 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:17.543 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.584130+0000", 2024-09-21T21:43:17.543 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 54.908026878000001, 2024-09-21T21:43:17.543 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.098350526999999993, 2024-09-21T21:43:17.543 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:17.543 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:17.543 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:17.543 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:17.543 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:17.543 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:17.543 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24798 2024-09-21T21:43:17.544 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.544 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:17.544 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.544 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:17.544 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584130+0000", 2024-09-21T21:43:17.544 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:17.544 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.544 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.544 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:17.544 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584130+0000", 2024-09-21T21:43:17.544 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:17.544 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.544 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.544 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:17.544 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584131+0000", 2024-09-21T21:43:17.545 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4700000000000001e-07 2024-09-21T21:43:17.545 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.545 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.545 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:17.545 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584133+0000", 2024-09-21T21:43:17.545 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.34e-06 2024-09-21T21:43:17.545 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.545 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.545 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:17.545 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584137+0000", 2024-09-21T21:43:17.545 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5609999999999999e-06 2024-09-21T21:43:17.545 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.545 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.545 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:17.546 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584139+0000", 2024-09-21T21:43:17.546 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7350000000000001e-06 2024-09-21T21:43:17.546 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.546 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.546 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:17.546 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.679274+0000", 2024-09-21T21:43:17.546 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.095134719000000006 2024-09-21T21:43:17.546 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.546 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.546 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:17.546 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.679358+0000", 2024-09-21T21:43:17.546 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3609999999999994e-05 2024-09-21T21:43:17.546 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.546 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.547 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:17.547 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.679685+0000", 2024-09-21T21:43:17.547 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00032732399999999999 2024-09-21T21:43:17.547 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.547 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.547 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:17.547 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681553+0000", 2024-09-21T21:43:17.547 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001868059 2024-09-21T21:43:17.547 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.547 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.547 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:17.547 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.682375+0000", 2024-09-21T21:43:17.547 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00082232099999999999 2024-09-21T21:43:17.547 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.547 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.547 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:17.548 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.682431+0000", 2024-09-21T21:43:17.548 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6051999999999998e-05 2024-09-21T21:43:17.548 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.548 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.548 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:17.548 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.682481+0000", 2024-09-21T21:43:17.548 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9258999999999997e-05 2024-09-21T21:43:17.548 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:17.548 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:17.548 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:17.548 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.548 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.548 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24799 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:17.548 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.584159+0000", 2024-09-21T21:43:17.548 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 54.907997889999997, 2024-09-21T21:43:17.549 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.100021551, 2024-09-21T21:43:17.550 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:17.550 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:17.550 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:17.550 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:17.550 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:17.550 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:17.550 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24799 2024-09-21T21:43:17.550 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.550 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:17.550 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.550 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:17.551 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584159+0000", 2024-09-21T21:43:17.551 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:17.551 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.551 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.551 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:17.551 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584159+0000", 2024-09-21T21:43:17.551 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:17.551 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.551 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.551 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:17.551 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584160+0000", 2024-09-21T21:43:17.551 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1300000000000001e-07 2024-09-21T21:43:17.551 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.551 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.552 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:17.552 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584162+0000", 2024-09-21T21:43:17.552 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5409999999999999e-06 2024-09-21T21:43:17.552 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.552 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.552 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:17.552 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584169+0000", 2024-09-21T21:43:17.552 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7880000000000001e-06 2024-09-21T21:43:17.552 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.552 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.552 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:17.552 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584174+0000", 2024-09-21T21:43:17.552 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9590000000000003e-06 2024-09-21T21:43:17.552 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.553 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.553 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:17.553 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.680400+0000", 2024-09-21T21:43:17.553 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.096226514999999999 2024-09-21T21:43:17.553 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.553 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.553 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:17.553 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.680480+0000", 2024-09-21T21:43:17.553 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9589e-05 2024-09-21T21:43:17.553 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.553 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.553 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:17.553 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.680847+0000", 2024-09-21T21:43:17.553 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00036757700000000001 2024-09-21T21:43:17.553 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.554 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.554 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:17.554 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.682555+0000", 2024-09-21T21:43:17.554 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001707791 2024-09-21T21:43:17.554 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.554 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.554 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:17.554 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684093+0000", 2024-09-21T21:43:17.554 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0015377170000000001 2024-09-21T21:43:17.554 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.554 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.554 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:17.554 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684144+0000", 2024-09-21T21:43:17.554 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1208000000000002e-05 2024-09-21T21:43:17.555 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.555 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.555 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:17.555 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684181+0000", 2024-09-21T21:43:17.555 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6452999999999998e-05 2024-09-21T21:43:17.555 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:17.555 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:17.555 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:17.555 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.555 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.555 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24835 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:17.555 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586332+0000", 2024-09-21T21:43:17.555 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 54.905825276000002, 2024-09-21T21:43:17.555 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.098730360000000003, 2024-09-21T21:43:17.556 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:17.556 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:17.556 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:17.556 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:17.556 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:17.556 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:17.556 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24835 2024-09-21T21:43:17.556 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.556 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:17.556 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.556 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:17.556 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586332+0000", 2024-09-21T21:43:17.556 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:17.556 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.557 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.557 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:17.557 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586332+0000", 2024-09-21T21:43:17.557 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:17.557 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.557 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.557 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:17.557 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586333+0000", 2024-09-21T21:43:17.557 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0200000000000001e-07 2024-09-21T21:43:17.557 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.557 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.557 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:17.557 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586346+0000", 2024-09-21T21:43:17.557 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3745e-05 2024-09-21T21:43:17.558 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.558 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.558 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:17.558 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586366+0000", 2024-09-21T21:43:17.558 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9272000000000001e-05 2024-09-21T21:43:17.558 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.558 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.558 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:17.558 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586377+0000", 2024-09-21T21:43:17.558 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1197e-05 2024-09-21T21:43:17.558 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.558 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.558 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:17.558 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681578+0000", 2024-09-21T21:43:17.559 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.095201420999999994 2024-09-21T21:43:17.559 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.559 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.559 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:17.559 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681640+0000", 2024-09-21T21:43:17.559 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1834000000000005e-05 2024-09-21T21:43:17.559 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.559 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.559 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:17.559 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681915+0000", 2024-09-21T21:43:17.559 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00027534199999999998 2024-09-21T21:43:17.559 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.559 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.559 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:17.560 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684230+0000", 2024-09-21T21:43:17.560 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00231424 2024-09-21T21:43:17.560 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.560 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.560 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:17.560 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684959+0000", 2024-09-21T21:43:17.560 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00072921399999999995 2024-09-21T21:43:17.560 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.560 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.560 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:17.560 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.685008+0000", 2024-09-21T21:43:17.560 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8860999999999999e-05 2024-09-21T21:43:17.560 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.560 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.560 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:17.561 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.685062+0000", 2024-09-21T21:43:17.561 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4432000000000003e-05 2024-09-21T21:43:17.561 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:17.561 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:17.561 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:17.561 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.561 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.561 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24837 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:17.561 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586395+0000", 2024-09-21T21:43:17.561 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 54.905762340999999, 2024-09-21T21:43:17.561 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.099700522, 2024-09-21T21:43:17.561 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:17.561 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:17.561 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:17.562 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:17.562 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:17.562 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:17.562 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24837 2024-09-21T21:43:17.562 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.562 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:17.562 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.562 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:17.562 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586395+0000", 2024-09-21T21:43:17.562 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:17.562 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.562 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.562 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:17.562 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586395+0000", 2024-09-21T21:43:17.562 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:17.563 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.563 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.563 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:17.563 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586395+0000", 2024-09-21T21:43:17.563 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.06e-07 2024-09-21T21:43:17.563 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.563 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.563 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:17.563 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586401+0000", 2024-09-21T21:43:17.563 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0889999999999996e-06 2024-09-21T21:43:17.563 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.563 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.563 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:17.563 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586410+0000", 2024-09-21T21:43:17.564 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.918e-06 2024-09-21T21:43:17.564 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.564 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.564 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:17.564 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586424+0000", 2024-09-21T21:43:17.564 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3478999999999999e-05 2024-09-21T21:43:17.564 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.564 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.564 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:17.564 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.682590+0000", 2024-09-21T21:43:17.564 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.096166086999999997 2024-09-21T21:43:17.564 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.564 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.564 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:17.564 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.682657+0000", 2024-09-21T21:43:17.565 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6878000000000006e-05 2024-09-21T21:43:17.565 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.565 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.565 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:17.565 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.682879+0000", 2024-09-21T21:43:17.565 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00022238 2024-09-21T21:43:17.565 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.565 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.565 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:17.565 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.685073+0000", 2024-09-21T21:43:17.565 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021938119999999998 2024-09-21T21:43:17.565 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.565 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.565 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:17.566 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686038+0000", 2024-09-21T21:43:17.566 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00096539499999999997 2024-09-21T21:43:17.566 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.566 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.566 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:17.566 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686071+0000", 2024-09-21T21:43:17.566 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2514000000000001e-05 2024-09-21T21:43:17.566 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.566 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.566 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:17.566 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686095+0000", 2024-09-21T21:43:17.566 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4363999999999999e-05 2024-09-21T21:43:17.566 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:17.566 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:17.567 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:17.567 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.567 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.567 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24839 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:17.567 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586468+0000", 2024-09-21T21:43:17.567 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 54.905688689000002, 2024-09-21T21:43:17.567 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.099739355000000002, 2024-09-21T21:43:17.567 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:17.567 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:17.567 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:17.567 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:17.567 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:17.567 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:17.567 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24839 2024-09-21T21:43:17.568 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.568 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:17.568 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.568 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:17.568 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586468+0000", 2024-09-21T21:43:17.568 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:17.568 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.568 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.568 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:17.568 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586468+0000", 2024-09-21T21:43:17.568 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:17.568 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.568 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.568 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:17.568 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586469+0000", 2024-09-21T21:43:17.569 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2200000000000004e-07 2024-09-21T21:43:17.569 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.569 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.569 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:17.569 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586472+0000", 2024-09-21T21:43:17.569 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9960000000000002e-06 2024-09-21T21:43:17.569 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.569 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.569 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:17.569 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586479+0000", 2024-09-21T21:43:17.569 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2699999999999999e-06 2024-09-21T21:43:17.569 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.569 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.569 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:17.570 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586485+0000", 2024-09-21T21:43:17.570 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6999999999999996e-06 2024-09-21T21:43:17.570 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.570 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.570 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:17.570 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.683276+0000", 2024-09-21T21:43:17.570 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.096791618999999995 2024-09-21T21:43:17.570 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.570 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.570 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:17.570 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.683326+0000", 2024-09-21T21:43:17.570 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9384e-05 2024-09-21T21:43:17.570 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.570 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.570 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:17.571 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.683655+0000", 2024-09-21T21:43:17.571 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00032965700000000002 2024-09-21T21:43:17.571 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.571 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.571 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:17.571 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686120+0000", 2024-09-21T21:43:17.571 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002464819 2024-09-21T21:43:17.571 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.571 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.571 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:17.571 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686159+0000", 2024-09-21T21:43:17.571 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9181000000000003e-05 2024-09-21T21:43:17.571 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.571 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.572 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:17.572 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686185+0000", 2024-09-21T21:43:17.572 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5907000000000001e-05 2024-09-21T21:43:17.572 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.572 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.572 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:17.572 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686208+0000", 2024-09-21T21:43:17.572 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.23e-05 2024-09-21T21:43:17.572 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:17.572 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:17.572 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:17.572 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.572 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.572 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24841 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:17.573 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586492+0000", 2024-09-21T21:43:17.573 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 54.905664526999999, 2024-09-21T21:43:17.573 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.101672077, 2024-09-21T21:43:17.573 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:17.573 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:17.573 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:17.573 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:17.573 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:17.573 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:17.573 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24841 2024-09-21T21:43:17.573 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.573 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:17.573 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.573 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:17.573 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586492+0000", 2024-09-21T21:43:17.574 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:17.574 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.574 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.574 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:17.574 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586492+0000", 2024-09-21T21:43:17.574 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:17.574 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.574 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.574 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:17.574 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586493+0000", 2024-09-21T21:43:17.574 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5999999999999999e-07 2024-09-21T21:43:17.574 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.574 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.574 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:17.574 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586496+0000", 2024-09-21T21:43:17.575 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6089999999999999e-06 2024-09-21T21:43:17.575 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.575 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.575 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:17.575 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586502+0000", 2024-09-21T21:43:17.575 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6270000000000002e-06 2024-09-21T21:43:17.575 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.575 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.575 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:17.575 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586507+0000", 2024-09-21T21:43:17.575 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8810000000000004e-06 2024-09-21T21:43:17.575 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.575 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.576 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:17.576 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684273+0000", 2024-09-21T21:43:17.576 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.097766415999999995 2024-09-21T21:43:17.576 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.576 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.576 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:17.576 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684359+0000", 2024-09-21T21:43:17.576 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5574000000000004e-05 2024-09-21T21:43:17.576 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.576 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.576 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:17.576 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684568+0000", 2024-09-21T21:43:17.576 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000208529 2024-09-21T21:43:17.576 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.576 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.577 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:17.577 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.687186+0000", 2024-09-21T21:43:17.577 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0026180769999999999 2024-09-21T21:43:17.577 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.577 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.577 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:17.577 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688093+0000", 2024-09-21T21:43:17.577 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00090766999999999996 2024-09-21T21:43:17.577 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.577 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.577 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:17.577 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688142+0000", 2024-09-21T21:43:17.577 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8986000000000002e-05 2024-09-21T21:43:17.577 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.578 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.578 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:17.578 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688165+0000", 2024-09-21T21:43:17.578 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2248000000000001e-05 2024-09-21T21:43:17.578 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:17.578 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:17.578 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:17.578 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.578 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.578 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24843 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:17.578 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586514+0000", 2024-09-21T21:43:17.578 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 54.905643241, 2024-09-21T21:43:17.578 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.101826449, 2024-09-21T21:43:17.578 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:17.579 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:17.579 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:17.579 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:17.579 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:17.579 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:17.579 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24843 2024-09-21T21:43:17.579 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.579 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:17.579 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.579 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:17.579 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586514+0000", 2024-09-21T21:43:17.579 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:17.579 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.579 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.580 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:17.580 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586514+0000", 2024-09-21T21:43:17.580 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:17.580 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.580 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.580 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:17.580 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586514+0000", 2024-09-21T21:43:17.580 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9999999999999998e-07 2024-09-21T21:43:17.580 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.580 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.580 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:17.580 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586517+0000", 2024-09-21T21:43:17.580 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4980000000000001e-06 2024-09-21T21:43:17.580 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.581 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.581 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:17.581 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586523+0000", 2024-09-21T21:43:17.581 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4609999999999996e-06 2024-09-21T21:43:17.581 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.581 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.581 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:17.581 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586528+0000", 2024-09-21T21:43:17.581 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2979999999999999e-06 2024-09-21T21:43:17.581 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.581 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.581 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:17.581 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.685104+0000", 2024-09-21T21:43:17.581 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.098575837999999999 2024-09-21T21:43:17.581 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.582 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.582 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:17.582 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.685184+0000", 2024-09-21T21:43:17.582 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9807000000000003e-05 2024-09-21T21:43:17.582 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.582 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.582 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:17.582 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.685496+0000", 2024-09-21T21:43:17.582 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00031225900000000001 2024-09-21T21:43:17.582 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.582 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.582 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:17.582 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688238+0000", 2024-09-21T21:43:17.583 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0027419409999999999 2024-09-21T21:43:17.583 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.583 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.583 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:17.583 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688253+0000", 2024-09-21T21:43:17.583 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4331000000000001e-05 2024-09-21T21:43:17.583 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.583 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.583 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:17.583 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688296+0000", 2024-09-21T21:43:17.583 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3801000000000003e-05 2024-09-21T21:43:17.583 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.583 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.583 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:17.583 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688340+0000", 2024-09-21T21:43:17.584 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3815e-05 2024-09-21T21:43:17.584 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:17.584 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:17.584 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:17.584 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.584 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.584 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24845 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:17.584 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586535+0000", 2024-09-21T21:43:17.584 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 54.905622360999999, 2024-09-21T21:43:17.584 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.10285678600000001, 2024-09-21T21:43:17.584 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:17.584 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:17.584 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:17.584 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:17.585 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:17.585 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:17.585 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24845 2024-09-21T21:43:17.585 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.585 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:17.585 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.585 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:17.585 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586535+0000", 2024-09-21T21:43:17.585 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:17.585 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.585 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.585 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:17.585 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586535+0000", 2024-09-21T21:43:17.585 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:17.585 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.586 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.586 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:17.586 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586535+0000", 2024-09-21T21:43:17.586 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9000000000000002e-07 2024-09-21T21:43:17.586 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.586 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.586 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:17.586 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586537+0000", 2024-09-21T21:43:17.586 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4569999999999999e-06 2024-09-21T21:43:17.586 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.586 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.586 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:17.586 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586544+0000", 2024-09-21T21:43:17.586 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5849999999999996e-06 2024-09-21T21:43:17.587 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.587 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.587 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:17.587 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586549+0000", 2024-09-21T21:43:17.587 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0089999999999998e-06 2024-09-21T21:43:17.587 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.587 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.587 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:17.587 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686239+0000", 2024-09-21T21:43:17.587 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.099689531999999997 2024-09-21T21:43:17.587 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.587 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.587 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:17.587 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686289+0000", 2024-09-21T21:43:17.588 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.007e-05 2024-09-21T21:43:17.588 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.588 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.588 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:17.588 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686633+0000", 2024-09-21T21:43:17.588 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00034428699999999998 2024-09-21T21:43:17.588 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.588 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.588 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:17.588 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688370+0000", 2024-09-21T21:43:17.588 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001736741 2024-09-21T21:43:17.588 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.588 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.588 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:17.589 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.689295+0000", 2024-09-21T21:43:17.589 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00092517900000000004 2024-09-21T21:43:17.589 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.589 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.589 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:17.589 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.689354+0000", 2024-09-21T21:43:17.589 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8799999999999999e-05 2024-09-21T21:43:17.589 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.589 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.589 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:17.589 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.689391+0000", 2024-09-21T21:43:17.589 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7735999999999999e-05 2024-09-21T21:43:17.589 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:17.589 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:17.589 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:17.590 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.590 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.590 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24846 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:17.590 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586564+0000", 2024-09-21T21:43:17.590 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 54.905593332999999, 2024-09-21T21:43:17.590 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.10375692, 2024-09-21T21:43:17.590 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:17.590 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:17.590 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:17.590 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:17.590 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:17.590 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:17.590 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24846 2024-09-21T21:43:17.591 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.591 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:17.591 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.591 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:17.591 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586564+0000", 2024-09-21T21:43:17.591 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:17.591 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.591 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.591 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:17.591 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586564+0000", 2024-09-21T21:43:17.591 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:17.591 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.591 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.591 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:17.592 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586564+0000", 2024-09-21T21:43:17.592 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4799999999999998e-07 2024-09-21T21:43:17.592 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.592 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.592 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:17.592 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586567+0000", 2024-09-21T21:43:17.592 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1669999999999999e-06 2024-09-21T21:43:17.592 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.592 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.592 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:17.592 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586578+0000", 2024-09-21T21:43:17.592 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0443e-05 2024-09-21T21:43:17.592 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.592 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.592 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:17.593 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586583+0000", 2024-09-21T21:43:17.593 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6790000000000002e-06 2024-09-21T21:43:17.593 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.593 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.593 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:17.593 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.687215+0000", 2024-09-21T21:43:17.593 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.100631973 2024-09-21T21:43:17.593 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.593 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.593 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:17.593 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.687311+0000", 2024-09-21T21:43:17.593 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5764e-05 2024-09-21T21:43:17.593 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.594 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.594 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:17.594 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.687663+0000", 2024-09-21T21:43:17.594 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00035167 2024-09-21T21:43:17.594 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.594 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.594 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:17.594 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690236+0000", 2024-09-21T21:43:17.594 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025733370000000002 2024-09-21T21:43:17.594 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.594 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.594 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:17.594 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690250+0000", 2024-09-21T21:43:17.595 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3548e-05 2024-09-21T21:43:17.595 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.595 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.595 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:17.595 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690298+0000", 2024-09-21T21:43:17.595 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7948999999999998e-05 2024-09-21T21:43:17.595 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.595 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.595 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:17.595 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690321+0000", 2024-09-21T21:43:17.595 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2841999999999999e-05 2024-09-21T21:43:17.595 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:17.595 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:17.595 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:17.596 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.596 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.596 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24849 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:17.596 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586633+0000", 2024-09-21T21:43:17.596 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 54.905523666999997, 2024-09-21T21:43:17.596 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.104506548, 2024-09-21T21:43:17.596 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:17.596 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:17.596 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:17.596 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:17.596 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:17.596 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:17.596 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24849 2024-09-21T21:43:17.596 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.596 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:17.597 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.597 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:17.597 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586633+0000", 2024-09-21T21:43:17.597 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:17.597 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.597 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.597 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:17.597 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586633+0000", 2024-09-21T21:43:17.597 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:17.597 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.597 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.597 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:17.597 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586634+0000", 2024-09-21T21:43:17.597 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9100000000000004e-07 2024-09-21T21:43:17.598 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.598 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.598 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:17.598 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586637+0000", 2024-09-21T21:43:17.598 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1999999999999999e-06 2024-09-21T21:43:17.598 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.598 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.598 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:17.598 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586645+0000", 2024-09-21T21:43:17.598 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7759999999999997e-06 2024-09-21T21:43:17.598 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.598 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.598 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:17.598 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586652+0000", 2024-09-21T21:43:17.599 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3560000000000004e-06 2024-09-21T21:43:17.599 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.599 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.599 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:17.599 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688395+0000", 2024-09-21T21:43:17.599 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.101743239 2024-09-21T21:43:17.599 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.599 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.599 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:17.599 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688464+0000", 2024-09-21T21:43:17.599 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8273000000000002e-05 2024-09-21T21:43:17.599 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.599 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.600 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:17.600 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688786+0000", 2024-09-21T21:43:17.600 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000322064 2024-09-21T21:43:17.600 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.600 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.600 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:17.600 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690364+0000", 2024-09-21T21:43:17.600 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001578084 2024-09-21T21:43:17.600 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.600 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.600 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:17.600 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.691050+0000", 2024-09-21T21:43:17.600 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00068623399999999998 2024-09-21T21:43:17.600 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.600 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.601 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:17.601 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.691090+0000", 2024-09-21T21:43:17.601 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9406999999999997e-05 2024-09-21T21:43:17.601 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.601 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.601 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:17.601 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.691140+0000", 2024-09-21T21:43:17.601 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0324000000000001e-05 2024-09-21T21:43:17.601 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:17.601 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:17.601 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:17.601 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.601 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.601 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24851 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:17.601 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586661+0000", 2024-09-21T21:43:17.602 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 54.905496380999999, 2024-09-21T21:43:17.602 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.10536891399999999, 2024-09-21T21:43:17.602 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:17.602 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:17.602 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:17.602 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:17.602 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:17.602 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:17.602 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24851 2024-09-21T21:43:17.602 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.602 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:17.602 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.602 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:17.602 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586661+0000", 2024-09-21T21:43:17.603 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:17.603 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.603 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.603 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:17.603 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586661+0000", 2024-09-21T21:43:17.603 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:17.603 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.603 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.603 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:17.603 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586661+0000", 2024-09-21T21:43:17.603 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4700000000000002e-07 2024-09-21T21:43:17.603 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.603 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.603 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:17.603 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586664+0000", 2024-09-21T21:43:17.604 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.672e-06 2024-09-21T21:43:17.604 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.604 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.604 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:17.604 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586671+0000", 2024-09-21T21:43:17.604 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9959999999999996e-06 2024-09-21T21:43:17.604 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.604 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.604 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:17.604 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586676+0000", 2024-09-21T21:43:17.604 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4020000000000001e-06 2024-09-21T21:43:17.604 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.604 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.604 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:17.605 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.689438+0000", 2024-09-21T21:43:17.605 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.102761503 2024-09-21T21:43:17.605 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.605 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.605 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:17.605 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.689524+0000", 2024-09-21T21:43:17.605 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6383000000000003e-05 2024-09-21T21:43:17.605 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.605 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.605 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:17.605 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.689769+0000", 2024-09-21T21:43:17.605 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00024504699999999997 2024-09-21T21:43:17.605 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.605 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.605 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:17.606 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.691936+0000", 2024-09-21T21:43:17.606 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021665130000000001 2024-09-21T21:43:17.606 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.606 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.606 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:17.606 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.691947+0000", 2024-09-21T21:43:17.606 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1198e-05 2024-09-21T21:43:17.606 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.606 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.606 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:17.606 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.691992+0000", 2024-09-21T21:43:17.606 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4774000000000002e-05 2024-09-21T21:43:17.606 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.606 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.607 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:17.607 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.692030+0000", 2024-09-21T21:43:17.607 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7978999999999997e-05 2024-09-21T21:43:17.607 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:17.607 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:17.607 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:17.607 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.607 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.607 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24853 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:17.607 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586684+0000", 2024-09-21T21:43:17.607 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 54.905473424, 2024-09-21T21:43:17.607 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.105936852, 2024-09-21T21:43:17.607 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:17.607 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:17.608 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:17.608 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:17.608 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:17.608 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:17.608 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24853 2024-09-21T21:43:17.608 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.608 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:17.608 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.608 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:17.608 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586684+0000", 2024-09-21T21:43:17.608 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:17.608 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.608 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.608 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:17.608 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586684+0000", 2024-09-21T21:43:17.609 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:17.609 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.609 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.609 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:17.609 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586684+0000", 2024-09-21T21:43:17.609 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.68e-07 2024-09-21T21:43:17.609 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.609 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.609 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:17.609 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586687+0000", 2024-09-21T21:43:17.609 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.774e-06 2024-09-21T21:43:17.609 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.609 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.609 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:17.610 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586694+0000", 2024-09-21T21:43:17.610 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6959999999999999e-06 2024-09-21T21:43:17.610 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.610 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.610 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:17.610 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586698+0000", 2024-09-21T21:43:17.610 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7489999999999998e-06 2024-09-21T21:43:17.610 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.610 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.610 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:17.610 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690395+0000", 2024-09-21T21:43:17.610 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.103696645 2024-09-21T21:43:17.610 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.611 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.611 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:17.611 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690440+0000", 2024-09-21T21:43:17.611 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4835e-05 2024-09-21T21:43:17.611 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.611 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.611 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:17.611 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690654+0000", 2024-09-21T21:43:17.611 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000214114 2024-09-21T21:43:17.611 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.611 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.611 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:17.611 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.692041+0000", 2024-09-21T21:43:17.611 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0013866519999999999 2024-09-21T21:43:17.611 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.611 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.612 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:17.612 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.692558+0000", 2024-09-21T21:43:17.612 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00051717199999999999 2024-09-21T21:43:17.612 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.612 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.612 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:17.612 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.692586+0000", 2024-09-21T21:43:17.612 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8504e-05 2024-09-21T21:43:17.612 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:17.612 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:17.612 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:17.612 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.692620+0000", 2024-09-21T21:43:17.612 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4143000000000002e-05 2024-09-21T21:43:17.612 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:17.612 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:17.612 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:17.613 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:17.613 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:17.613 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:43:17.614 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:43:17.612+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:43:17.614 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T21:43:17.715 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:43:17.713+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:43:17.717 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:43:17.717 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:43:17.717 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 0 2024-09-21T21:43:17.717 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:43:17.726 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T21:43:17.816 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:43:17.815+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:43:17.828 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:43:17.828 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:43:17.828 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:43:17.828 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:43:17.829 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:43:17.838 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T21:43:17.916 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:43:17.915+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:43:17.943 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:43:17.944 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 20, 2024-09-21T21:43:17.944 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 600, 2024-09-21T21:43:17.944 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:43:17.944 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.944 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11319 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 5492736~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:17.944 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.712385+0000", 2024-09-21T21:43:17.944 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 87.229245066000004, 2024-09-21T21:43:17.944 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078627889000000006, 2024-09-21T21:43:17.944 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:17.944 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:17.944 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:17.944 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:17.944 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:17.945 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:17.945 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11319 2024-09-21T21:43:17.945 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.945 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:17.945 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.945 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:17.945 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712385+0000", 2024-09-21T21:43:17.945 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:17.945 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.945 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.945 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:17.945 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712385+0000", 2024-09-21T21:43:17.945 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:17.945 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.946 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.946 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:17.946 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712385+0000", 2024-09-21T21:43:17.946 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2800000000000002e-07 2024-09-21T21:43:17.946 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.946 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.946 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:17.946 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712387+0000", 2024-09-21T21:43:17.946 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1569999999999998e-06 2024-09-21T21:43:17.946 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.946 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.946 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:17.946 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712393+0000", 2024-09-21T21:43:17.946 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9370000000000001e-06 2024-09-21T21:43:17.947 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.947 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.947 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:17.947 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712398+0000", 2024-09-21T21:43:17.947 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5020000000000003e-06 2024-09-21T21:43:17.947 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.947 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.947 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:17.947 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.790746+0000", 2024-09-21T21:43:17.947 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078348589999999996 2024-09-21T21:43:17.947 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.947 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.947 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:17.947 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.790844+0000", 2024-09-21T21:43:17.948 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.7526000000000001e-05 2024-09-21T21:43:17.948 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.948 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.948 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:17.948 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.791013+0000", 2024-09-21T21:43:17.948 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00016874899999999999 2024-09-21T21:43:17.948 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:17.948 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:17.948 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:17.948 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.948 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.948 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11320 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 6180864~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:17.948 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.712405+0000", 2024-09-21T21:43:17.948 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 87.229225107000005, 2024-09-21T21:43:17.948 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078882870999999993, 2024-09-21T21:43:17.949 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:17.949 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:17.949 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:17.949 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:17.949 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:17.949 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:17.949 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11320 2024-09-21T21:43:17.949 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.949 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:17.949 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.949 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:17.949 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712405+0000", 2024-09-21T21:43:17.949 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:17.949 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.949 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.950 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:17.950 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712405+0000", 2024-09-21T21:43:17.950 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:17.950 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.950 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.950 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:17.950 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.713942+0000", 2024-09-21T21:43:17.950 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.001536882 2024-09-21T21:43:17.950 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.950 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.950 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:17.950 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.713946+0000", 2024-09-21T21:43:17.950 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4549999999999997e-06 2024-09-21T21:43:17.950 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.951 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.951 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:17.951 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.713958+0000", 2024-09-21T21:43:17.951 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1775e-05 2024-09-21T21:43:17.951 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.951 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.951 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:17.951 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.713965+0000", 2024-09-21T21:43:17.951 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5100000000000001e-06 2024-09-21T21:43:17.951 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.951 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.951 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:17.951 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.791059+0000", 2024-09-21T21:43:17.951 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.077094036000000005 2024-09-21T21:43:17.951 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.952 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.952 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:17.952 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.791175+0000", 2024-09-21T21:43:17.952 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000115895 2024-09-21T21:43:17.952 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.952 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.952 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:17.952 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.791288+0000", 2024-09-21T21:43:17.952 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000112318 2024-09-21T21:43:17.952 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:17.952 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:17.952 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:17.952 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.952 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.953 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11325 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 5414912~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:17.953 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.714095+0000", 2024-09-21T21:43:17.953 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 87.227535111999998, 2024-09-21T21:43:17.953 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078631894999999993, 2024-09-21T21:43:17.953 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:17.953 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:17.953 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:17.953 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:17.953 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:17.953 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:17.953 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11325 2024-09-21T21:43:17.953 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.953 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:17.953 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.953 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:17.954 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714095+0000", 2024-09-21T21:43:17.954 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:17.954 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.954 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.954 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:17.954 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714095+0000", 2024-09-21T21:43:17.954 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:17.954 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.954 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.954 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:17.954 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714095+0000", 2024-09-21T21:43:17.954 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7500000000000001e-07 2024-09-21T21:43:17.954 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.954 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.955 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:17.955 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714097+0000", 2024-09-21T21:43:17.955 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.12e-06 2024-09-21T21:43:17.955 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.955 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.955 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:17.955 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714104+0000", 2024-09-21T21:43:17.955 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3520000000000003e-06 2024-09-21T21:43:17.955 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.955 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.955 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:17.955 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714108+0000", 2024-09-21T21:43:17.955 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8810000000000004e-06 2024-09-21T21:43:17.955 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.955 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.956 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:17.956 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792509+0000", 2024-09-21T21:43:17.956 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078400739999999997 2024-09-21T21:43:17.956 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.956 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.956 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:17.956 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792613+0000", 2024-09-21T21:43:17.956 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000103939 2024-09-21T21:43:17.956 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.956 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.956 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:17.956 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792727+0000", 2024-09-21T21:43:17.956 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011348799999999999 2024-09-21T21:43:17.956 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:17.956 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:17.957 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:17.957 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.957 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.957 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11326 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 6103040~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:17.957 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.714115+0000", 2024-09-21T21:43:17.957 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 87.227515084000004, 2024-09-21T21:43:17.957 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078963056000000004, 2024-09-21T21:43:17.957 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:17.957 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:17.957 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:17.957 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:17.957 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:17.957 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:17.958 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11326 2024-09-21T21:43:17.958 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.958 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:17.958 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.958 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:17.958 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714115+0000", 2024-09-21T21:43:17.958 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:17.958 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.958 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.958 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:17.958 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714115+0000", 2024-09-21T21:43:17.958 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:17.958 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.958 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.959 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:17.959 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.715940+0000", 2024-09-21T21:43:17.959 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0018254969999999999 2024-09-21T21:43:17.959 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.959 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.959 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:17.959 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.715945+0000", 2024-09-21T21:43:17.959 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6120000000000001e-06 2024-09-21T21:43:17.959 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.959 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.959 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:17.959 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.715954+0000", 2024-09-21T21:43:17.959 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.5899999999999997e-06 2024-09-21T21:43:17.959 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.960 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.960 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:17.960 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.715959+0000", 2024-09-21T21:43:17.960 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0849999999999996e-06 2024-09-21T21:43:17.960 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.960 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.960 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:17.960 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792771+0000", 2024-09-21T21:43:17.960 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.076811829999999998 2024-09-21T21:43:17.960 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.960 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.960 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:17.960 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792882+0000", 2024-09-21T21:43:17.960 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000110857 2024-09-21T21:43:17.961 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.961 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.961 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:17.961 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.793078+0000", 2024-09-21T21:43:17.961 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00019558499999999999 2024-09-21T21:43:17.961 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:17.961 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:17.961 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:17.961 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.961 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.961 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11332 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 7565312~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:17.961 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.716063+0000", 2024-09-21T21:43:17.961 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 87.225566549999996, 2024-09-21T21:43:17.961 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078787482000000006, 2024-09-21T21:43:17.962 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:17.962 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:17.962 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:17.962 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:17.962 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:17.962 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:17.962 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11332 2024-09-21T21:43:17.962 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.962 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:17.962 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.962 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:17.962 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.716063+0000", 2024-09-21T21:43:17.962 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:17.962 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.963 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.963 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:17.963 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.716063+0000", 2024-09-21T21:43:17.963 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:17.963 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.963 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.963 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:17.963 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.717944+0000", 2024-09-21T21:43:17.963 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0018812309999999999 2024-09-21T21:43:17.963 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.963 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.963 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:17.963 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.717949+0000", 2024-09-21T21:43:17.963 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9389999999999996e-06 2024-09-21T21:43:17.964 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.964 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.964 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:17.964 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.717963+0000", 2024-09-21T21:43:17.964 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.3899e-05 2024-09-21T21:43:17.964 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.964 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.964 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:17.964 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.717970+0000", 2024-09-21T21:43:17.964 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.426e-06 2024-09-21T21:43:17.964 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.964 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.964 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:17.964 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.794573+0000", 2024-09-21T21:43:17.965 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.076602990999999995 2024-09-21T21:43:17.965 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.965 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.965 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:17.965 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.794682+0000", 2024-09-21T21:43:17.965 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000109541 2024-09-21T21:43:17.965 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.965 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.965 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:17.965 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.794851+0000", 2024-09-21T21:43:17.965 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000168455 2024-09-21T21:43:17.965 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:17.965 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:17.965 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:17.965 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.966 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.966 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11339 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 6266880~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:17.966 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.718117+0000", 2024-09-21T21:43:17.966 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 87.223512524, 2024-09-21T21:43:17.966 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078703717000000006, 2024-09-21T21:43:17.966 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:17.966 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:17.966 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:17.966 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:17.966 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:17.966 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:17.966 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11339 2024-09-21T21:43:17.966 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.966 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:17.967 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.967 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:17.967 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.718117+0000", 2024-09-21T21:43:17.967 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:17.967 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.967 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.967 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:17.967 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.718117+0000", 2024-09-21T21:43:17.967 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:17.967 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.967 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.967 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:17.967 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720019+0000", 2024-09-21T21:43:17.967 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0019012549999999999 2024-09-21T21:43:17.967 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.967 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.968 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:17.968 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720024+0000", 2024-09-21T21:43:17.968 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1100000000000002e-06 2024-09-21T21:43:17.968 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.968 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.968 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:17.968 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720035+0000", 2024-09-21T21:43:17.968 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.094e-05 2024-09-21T21:43:17.968 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.968 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.968 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:17.968 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720040+0000", 2024-09-21T21:43:17.968 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1830000000000002e-06 2024-09-21T21:43:17.968 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.968 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.969 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:17.969 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796618+0000", 2024-09-21T21:43:17.969 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.076578033000000004 2024-09-21T21:43:17.969 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.969 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.969 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:17.969 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796696+0000", 2024-09-21T21:43:17.969 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7836000000000005e-05 2024-09-21T21:43:17.969 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.969 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.969 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:17.969 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796821+0000", 2024-09-21T21:43:17.969 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00012536000000000001 2024-09-21T21:43:17.969 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:17.970 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:17.970 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:17.970 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.970 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.970 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11345 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 5472256~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:17.970 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.720116+0000", 2024-09-21T21:43:17.970 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 87.221513623999996, 2024-09-21T21:43:17.970 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078667730000000005, 2024-09-21T21:43:17.970 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:17.970 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:17.970 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:17.970 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:17.970 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:17.970 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:17.971 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11345 2024-09-21T21:43:17.971 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.971 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:17.971 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.971 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:17.971 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720116+0000", 2024-09-21T21:43:17.971 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:17.971 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.971 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.971 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:17.971 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720116+0000", 2024-09-21T21:43:17.971 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:17.971 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.971 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.972 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:17.972 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720116+0000", 2024-09-21T21:43:17.972 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.04e-07 2024-09-21T21:43:17.972 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.972 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.972 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:17.972 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720118+0000", 2024-09-21T21:43:17.972 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.181e-06 2024-09-21T21:43:17.972 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.972 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.972 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:17.972 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720121+0000", 2024-09-21T21:43:17.972 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0199999999999999e-06 2024-09-21T21:43:17.972 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.973 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.973 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:17.973 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720123+0000", 2024-09-21T21:43:17.973 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.5469999999999999e-06 2024-09-21T21:43:17.973 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.973 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.973 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:17.973 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798626+0000", 2024-09-21T21:43:17.973 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078503137000000001 2024-09-21T21:43:17.973 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.973 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.973 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:17.973 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798677+0000", 2024-09-21T21:43:17.973 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0930999999999997e-05 2024-09-21T21:43:17.973 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.974 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.974 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:17.974 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798784+0000", 2024-09-21T21:43:17.974 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010671 2024-09-21T21:43:17.974 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:17.974 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:17.974 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:17.974 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.974 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.974 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11346 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 5500928~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:17.974 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.720126+0000", 2024-09-21T21:43:17.974 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 87.221503550999998, 2024-09-21T21:43:17.974 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078863421000000003, 2024-09-21T21:43:17.974 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:17.974 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:17.975 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:17.975 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:17.975 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:17.975 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:17.975 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11346 2024-09-21T21:43:17.975 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.975 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:17.975 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.975 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:17.975 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720126+0000", 2024-09-21T21:43:17.975 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:17.975 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.975 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.975 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:17.976 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720126+0000", 2024-09-21T21:43:17.976 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:17.976 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.976 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.976 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:17.976 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.722145+0000", 2024-09-21T21:43:17.976 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0020183219999999999 2024-09-21T21:43:17.976 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.976 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.976 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:17.976 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.722150+0000", 2024-09-21T21:43:17.976 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1760000000000001e-06 2024-09-21T21:43:17.976 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.976 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.976 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:17.977 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.722161+0000", 2024-09-21T21:43:17.977 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1566000000000001e-05 2024-09-21T21:43:17.977 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.977 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.977 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:17.977 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.722169+0000", 2024-09-21T21:43:17.977 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.2840000000000001e-06 2024-09-21T21:43:17.977 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.977 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.977 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:17.977 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798815+0000", 2024-09-21T21:43:17.977 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.076646607000000005 2024-09-21T21:43:17.977 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.977 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.978 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:17.978 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798870+0000", 2024-09-21T21:43:17.978 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4486e-05 2024-09-21T21:43:17.978 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.978 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.978 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:17.978 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798990+0000", 2024-09-21T21:43:17.978 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011998 2024-09-21T21:43:17.978 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:17.978 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:17.978 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:17.978 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.978 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.978 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13629 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7852032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:17.979 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943319+0000", 2024-09-21T21:43:17.979 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 86.998310908999997, 2024-09-21T21:43:17.979 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078610975999999999, 2024-09-21T21:43:17.979 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:17.979 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:17.979 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:17.979 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:17.979 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:17.979 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:17.979 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13629 2024-09-21T21:43:17.979 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.979 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:17.979 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.979 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:17.980 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943319+0000", 2024-09-21T21:43:17.980 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:17.980 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.980 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.980 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:17.980 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943319+0000", 2024-09-21T21:43:17.980 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:17.980 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.980 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.980 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:17.980 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943319+0000", 2024-09-21T21:43:17.980 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6199999999999999e-07 2024-09-21T21:43:17.980 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.980 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.981 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:17.981 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943321+0000", 2024-09-21T21:43:17.981 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1009999999999999e-06 2024-09-21T21:43:17.981 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.981 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.981 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:17.981 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943328+0000", 2024-09-21T21:43:17.981 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.1029999999999996e-06 2024-09-21T21:43:17.981 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.981 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.981 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:17.981 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943333+0000", 2024-09-21T21:43:17.981 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7709999999999997e-06 2024-09-21T21:43:17.981 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.981 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.982 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:17.982 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.021679+0000", 2024-09-21T21:43:17.982 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078345673000000005 2024-09-21T21:43:17.982 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.982 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.982 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:17.982 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.021771+0000", 2024-09-21T21:43:17.982 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2201999999999996e-05 2024-09-21T21:43:17.982 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.982 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.982 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:17.982 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.021930+0000", 2024-09-21T21:43:17.982 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00015876400000000001 2024-09-21T21:43:17.982 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:17.982 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:17.983 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:17.983 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.983 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.983 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13630 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7880704~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:17.983 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943339+0000", 2024-09-21T21:43:17.983 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 86.998290721000004, 2024-09-21T21:43:17.983 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078939410000000002, 2024-09-21T21:43:17.983 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:17.983 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:17.983 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:17.983 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:17.983 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:17.983 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:17.983 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13630 2024-09-21T21:43:17.984 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.984 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:17.984 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.984 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:17.984 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943339+0000", 2024-09-21T21:43:17.984 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:17.984 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.984 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.984 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:17.984 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943339+0000", 2024-09-21T21:43:17.984 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:17.984 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.984 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.984 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:17.984 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943339+0000", 2024-09-21T21:43:17.985 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8000000000000001e-07 2024-09-21T21:43:17.985 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.985 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.985 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:17.985 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943341+0000", 2024-09-21T21:43:17.985 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9850000000000001e-06 2024-09-21T21:43:17.985 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.985 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.985 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:17.985 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943347+0000", 2024-09-21T21:43:17.985 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2979999999999999e-06 2024-09-21T21:43:17.985 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.985 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.985 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:17.986 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943354+0000", 2024-09-21T21:43:17.986 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.2969999999999998e-06 2024-09-21T21:43:17.986 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.986 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.986 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:17.986 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.021996+0000", 2024-09-21T21:43:17.986 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078641680000000005 2024-09-21T21:43:17.986 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.986 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.986 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:17.986 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022104+0000", 2024-09-21T21:43:17.986 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000107902 2024-09-21T21:43:17.986 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.986 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.987 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:17.987 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022278+0000", 2024-09-21T21:43:17.987 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00017486800000000001 2024-09-21T21:43:17.987 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:17.987 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:17.987 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:17.987 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.987 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.987 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13631 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7909376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:17.987 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943362+0000", 2024-09-21T21:43:17.987 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 86.998267901000006, 2024-09-21T21:43:17.987 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079195644999999995, 2024-09-21T21:43:17.987 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:17.987 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:17.987 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:17.988 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:17.988 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:17.988 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:17.988 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13631 2024-09-21T21:43:17.988 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.988 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:17.988 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.988 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:17.988 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943362+0000", 2024-09-21T21:43:17.988 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:17.988 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.988 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.988 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:17.988 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943362+0000", 2024-09-21T21:43:17.988 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:17.989 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.989 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.989 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:17.989 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943362+0000", 2024-09-21T21:43:17.989 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8799999999999998e-07 2024-09-21T21:43:17.989 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.989 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.989 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:17.989 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943364+0000", 2024-09-21T21:43:17.989 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9850000000000001e-06 2024-09-21T21:43:17.989 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.989 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.989 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:17.989 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943371+0000", 2024-09-21T21:43:17.989 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0650000000000001e-06 2024-09-21T21:43:17.990 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.990 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.990 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:17.990 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943376+0000", 2024-09-21T21:43:17.990 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8250000000000004e-06 2024-09-21T21:43:17.990 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.990 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.990 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:17.990 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022329+0000", 2024-09-21T21:43:17.990 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078952888999999998 2024-09-21T21:43:17.990 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.990 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.990 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:17.990 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022419+0000", 2024-09-21T21:43:17.991 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.9738e-05 2024-09-21T21:43:17.991 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.991 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.991 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:17.991 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022558+0000", 2024-09-21T21:43:17.991 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00013875500000000001 2024-09-21T21:43:17.991 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:17.991 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:17.991 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:17.991 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.991 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.991 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13632 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7966720~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:17.991 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943382+0000", 2024-09-21T21:43:17.991 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 86.998247667000001, 2024-09-21T21:43:17.992 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079495727000000002, 2024-09-21T21:43:17.992 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:17.992 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:17.992 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:17.992 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:17.992 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:17.992 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:17.992 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13632 2024-09-21T21:43:17.992 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.992 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:17.992 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.992 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:17.992 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943382+0000", 2024-09-21T21:43:17.992 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:17.992 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.993 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.993 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:17.993 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943382+0000", 2024-09-21T21:43:17.993 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:17.993 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.993 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.993 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:17.993 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943382+0000", 2024-09-21T21:43:17.993 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7e-07 2024-09-21T21:43:17.993 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.993 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.993 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:17.993 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943385+0000", 2024-09-21T21:43:17.993 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2230000000000001e-06 2024-09-21T21:43:17.994 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.994 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.994 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:17.994 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943391+0000", 2024-09-21T21:43:17.994 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6810000000000001e-06 2024-09-21T21:43:17.994 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.994 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.994 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:17.994 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943397+0000", 2024-09-21T21:43:17.994 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6559999999999997e-06 2024-09-21T21:43:17.994 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.994 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.994 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:17.994 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022601+0000", 2024-09-21T21:43:17.995 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079204051999999997 2024-09-21T21:43:17.995 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.995 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.995 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:17.995 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022704+0000", 2024-09-21T21:43:17.995 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010329599999999999 2024-09-21T21:43:17.995 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.995 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.995 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:17.995 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022878+0000", 2024-09-21T21:43:17.995 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000173449 2024-09-21T21:43:17.995 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:17.995 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:17.995 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:17.996 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.996 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.996 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13633 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7938048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:17.996 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943403+0000", 2024-09-21T21:43:17.996 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 86.998226736999996, 2024-09-21T21:43:17.996 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079794697999999997, 2024-09-21T21:43:17.996 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:17.996 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:17.996 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:17.996 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:17.996 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:17.996 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:17.996 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13633 2024-09-21T21:43:17.996 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.997 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:17.997 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.997 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:17.997 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943403+0000", 2024-09-21T21:43:17.997 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:17.997 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.997 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.997 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:17.997 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943403+0000", 2024-09-21T21:43:17.997 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:17.997 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.997 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.997 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:17.997 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943403+0000", 2024-09-21T21:43:17.998 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.65e-07 2024-09-21T21:43:17.998 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.998 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.998 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:17.998 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943405+0000", 2024-09-21T21:43:17.998 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0020000000000001e-06 2024-09-21T21:43:17.998 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.998 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.998 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:17.998 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943412+0000", 2024-09-21T21:43:17.998 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.8379999999999996e-06 2024-09-21T21:43:17.998 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.998 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.998 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:17.999 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943417+0000", 2024-09-21T21:43:17.999 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7650000000000001e-06 2024-09-21T21:43:17.999 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.999 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.999 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:17.999 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022922+0000", 2024-09-21T21:43:17.999 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079504512999999999 2024-09-21T21:43:17.999 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:17.999 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:17.999 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:17.999 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023039+0000", 2024-09-21T21:43:17.999 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000117044 2024-09-21T21:43:17.999 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:18.000 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:18.000 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:18.000 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023198+0000", 2024-09-21T21:43:18.000 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000159171 2024-09-21T21:43:18.000 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:18.000 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:18.000 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:18.000 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:18.000 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:18.000 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13634 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7995392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:18.000 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943423+0000", 2024-09-21T21:43:18.000 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 86.998207029, 2024-09-21T21:43:18.000 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.080142358999999996, 2024-09-21T21:43:18.000 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:18.000 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:18.001 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:18.001 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:18.001 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:18.001 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:18.001 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13634 2024-09-21T21:43:18.001 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:18.001 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:18.001 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:18.001 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:18.001 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943423+0000", 2024-09-21T21:43:18.001 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:18.001 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:18.001 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:18.001 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:18.002 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943423+0000", 2024-09-21T21:43:18.002 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:18.002 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:18.002 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:18.002 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:18.002 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943423+0000", 2024-09-21T21:43:18.002 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.58e-07 2024-09-21T21:43:18.002 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:18.002 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:18.002 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:18.002 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943425+0000", 2024-09-21T21:43:18.002 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9180000000000001e-06 2024-09-21T21:43:18.002 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:18.002 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:18.003 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:18.003 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943431+0000", 2024-09-21T21:43:18.003 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4439999999999996e-06 2024-09-21T21:43:18.003 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:18.003 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:18.003 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:18.003 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943436+0000", 2024-09-21T21:43:18.003 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.887e-06 2024-09-21T21:43:18.003 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:18.003 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:18.003 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:18.003 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023266+0000", 2024-09-21T21:43:18.003 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079829622000000003 2024-09-21T21:43:18.003 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:18.004 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:18.004 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:18.004 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023384+0000", 2024-09-21T21:43:18.004 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000117658 2024-09-21T21:43:18.004 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:18.004 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:18.004 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:18.004 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023565+0000", 2024-09-21T21:43:18.004 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00018147200000000001 2024-09-21T21:43:18.004 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:18.004 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:18.004 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:18.004 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:18.005 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:18.005 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13635 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 8024064~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:18.005 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943447+0000", 2024-09-21T21:43:18.005 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 86.998182489000001, 2024-09-21T21:43:18.005 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.080417653000000006, 2024-09-21T21:43:18.005 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:18.005 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:18.005 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:18.005 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:18.005 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:18.005 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:18.005 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13635 2024-09-21T21:43:18.005 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:18.005 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:18.005 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:18.005 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:18.006 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943447+0000", 2024-09-21T21:43:18.006 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:18.006 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:18.006 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:18.006 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:18.006 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943447+0000", 2024-09-21T21:43:18.006 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:18.006 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:18.006 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:18.006 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:18.006 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943448+0000", 2024-09-21T21:43:18.006 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1699999999999999e-07 2024-09-21T21:43:18.006 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:18.006 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:18.007 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:18.007 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943450+0000", 2024-09-21T21:43:18.007 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1639999999999999e-06 2024-09-21T21:43:18.007 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:18.007 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:18.007 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:18.007 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943456+0000", 2024-09-21T21:43:18.007 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8379999999999998e-06 2024-09-21T21:43:18.007 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:18.007 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:18.007 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:18.007 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943460+0000", 2024-09-21T21:43:18.007 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4700000000000004e-06 2024-09-21T21:43:18.008 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:18.008 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:18.008 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:18.008 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023595+0000", 2024-09-21T21:43:18.008 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.080134869999999997 2024-09-21T21:43:18.008 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:18.008 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:18.008 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:18.008 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023688+0000", 2024-09-21T21:43:18.008 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2794000000000006e-05 2024-09-21T21:43:18.008 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:18.008 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:18.008 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:18.008 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023865+0000", 2024-09-21T21:43:18.009 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0001771 2024-09-21T21:43:18.009 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:18.009 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:18.009 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:18.009 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:18.009 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:18.009 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13636 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 8052736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:18.009 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943467+0000", 2024-09-21T21:43:18.009 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 86.998162751999999, 2024-09-21T21:43:18.009 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.080694399999999999, 2024-09-21T21:43:18.009 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:18.009 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:18.009 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:18.009 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:18.009 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:18.010 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:18.010 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13636 2024-09-21T21:43:18.010 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:18.010 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:18.010 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:18.010 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:18.010 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943467+0000", 2024-09-21T21:43:18.010 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:18.010 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:18.010 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:18.010 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:18.010 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943467+0000", 2024-09-21T21:43:18.010 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:18.011 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:18.011 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:18.011 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:18.011 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944940+0000", 2024-09-21T21:43:18.011 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0014725109999999999 2024-09-21T21:43:18.011 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:18.011 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:18.011 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:18.011 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944949+0000", 2024-09-21T21:43:18.011 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2930000000000006e-06 2024-09-21T21:43:18.011 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:18.011 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:18.011 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:18.011 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944958+0000", 2024-09-21T21:43:18.011 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.6899999999999998e-06 2024-09-21T21:43:18.012 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:18.012 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:18.012 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:18.012 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944966+0000", 2024-09-21T21:43:18.012 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0690000000000002e-06 2024-09-21T21:43:18.012 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:18.012 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:18.012 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:18.012 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023900+0000", 2024-09-21T21:43:18.012 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078934371000000003 2024-09-21T21:43:18.012 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:18.012 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:18.012 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:18.012 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024009+0000", 2024-09-21T21:43:18.013 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000108857 2024-09-21T21:43:18.013 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:18.013 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:18.013 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:18.013 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024161+0000", 2024-09-21T21:43:18.013 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000152609 2024-09-21T21:43:18.013 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:18.013 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:18.013 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:18.013 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:18.013 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:18.013 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13637 3.12 3.1dc31b72 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:18.013 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.944978+0000", 2024-09-21T21:43:18.013 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 86.996651817, 2024-09-21T21:43:18.013 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079232604999999998, 2024-09-21T21:43:18.014 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:18.014 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:18.014 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:43:18.014 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:18.014 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:18.014 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:18.014 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13637 2024-09-21T21:43:18.014 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:18.014 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:18.014 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:18.014 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:18.014 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944978+0000", 2024-09-21T21:43:18.014 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:18.014 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:18.015 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:18.015 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:18.015 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944978+0000", 2024-09-21T21:43:18.015 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:18.015 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:18.015 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:18.015 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:18.015 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944978+0000", 2024-09-21T21:43:18.015 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9100000000000004e-07 2024-09-21T21:43:18.015 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:18.015 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:18.015 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:18.015 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944981+0000", 2024-09-21T21:43:18.015 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3290000000000001e-06 2024-09-21T21:43:18.016 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:18.016 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:18.016 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:18.016 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944987+0000", 2024-09-21T21:43:18.016 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.28e-06 2024-09-21T21:43:18.016 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:18.016 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:18.016 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:18.016 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944992+0000", 2024-09-21T21:43:18.016 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8520000000000003e-06 2024-09-21T21:43:18.016 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:18.016 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:18.016 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:18.016 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024196+0000", 2024-09-21T21:43:18.016 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079203673000000002 2024-09-21T21:43:18.017 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:18.017 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:18.017 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:18.017 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024211+0000", 2024-09-21T21:43:18.017 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.4980000000000001e-05 2024-09-21T21:43:18.017 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:18.017 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:18.017 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:18.017 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:18.017 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:18.017 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13638 3.12 3.1dc31b72 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:18.017 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.945000+0000", 2024-09-21T21:43:18.017 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 86.996629689000002, 2024-09-21T21:43:18.017 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079257465999999999, 2024-09-21T21:43:18.018 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:18.018 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:18.018 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:43:18.018 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:18.018 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:18.018 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:18.018 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13638 2024-09-21T21:43:18.018 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:18.018 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:18.018 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:18.018 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:18.018 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945000+0000", 2024-09-21T21:43:18.018 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:18.018 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:18.019 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:18.019 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:18.019 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945000+0000", 2024-09-21T21:43:18.019 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:18.019 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:18.019 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:18.019 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:18.019 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945001+0000", 2024-09-21T21:43:18.019 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.32e-07 2024-09-21T21:43:18.019 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:18.019 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:18.019 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:18.019 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945003+0000", 2024-09-21T21:43:18.019 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2309999999999999e-06 2024-09-21T21:43:18.019 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:18.020 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:18.020 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:18.020 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945009+0000", 2024-09-21T21:43:18.020 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7450000000000001e-06 2024-09-21T21:43:18.020 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:18.020 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:18.020 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:18.020 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945013+0000", 2024-09-21T21:43:18.020 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.673e-06 2024-09-21T21:43:18.020 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:18.020 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:18.020 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:18.020 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024245+0000", 2024-09-21T21:43:18.020 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079232160999999995 2024-09-21T21:43:18.021 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:18.021 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:18.021 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:18.021 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024258+0000", 2024-09-21T21:43:18.021 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.2224e-05 2024-09-21T21:43:18.021 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:18.021 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:18.021 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:18.021 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:18.021 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:18.021 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13639 3.12 3.1dc31b72 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:18.021 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.945034+0000", 2024-09-21T21:43:18.021 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 86.996595889999995, 2024-09-21T21:43:18.021 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079260627, 2024-09-21T21:43:18.021 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:18.022 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:18.022 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:43:18.022 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:18.022 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:18.022 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:18.022 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13639 2024-09-21T21:43:18.022 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:18.022 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:18.022 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:18.022 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:18.022 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945034+0000", 2024-09-21T21:43:18.022 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:18.022 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:18.022 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:18.023 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:18.023 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945034+0000", 2024-09-21T21:43:18.023 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:18.023 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:18.023 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:18.023 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:18.023 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945034+0000", 2024-09-21T21:43:18.023 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3799999999999998e-07 2024-09-21T21:43:18.023 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:18.023 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:18.023 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:18.023 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945036+0000", 2024-09-21T21:43:18.023 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.052e-06 2024-09-21T21:43:18.023 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:18.024 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:18.024 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:18.024 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945040+0000", 2024-09-21T21:43:18.024 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1169999999999997e-06 2024-09-21T21:43:18.024 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:18.024 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:18.024 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:18.024 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945043+0000", 2024-09-21T21:43:18.024 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.8439999999999998e-06 2024-09-21T21:43:18.024 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:18.024 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:18.024 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:18.024 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024286+0000", 2024-09-21T21:43:18.024 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079242483000000002 2024-09-21T21:43:18.025 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:18.025 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:18.025 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:18.025 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024295+0000", 2024-09-21T21:43:18.025 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.7930000000000003e-06 2024-09-21T21:43:18.025 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:18.025 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:18.025 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:18.025 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:18.025 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:18.025 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13640 3.12 3.1dc31b72 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:18.025 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.945047+0000", 2024-09-21T21:43:18.025 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 86.996582368000006, 2024-09-21T21:43:18.025 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079412321999999994, 2024-09-21T21:43:18.026 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:18.026 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:18.026 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:43:18.026 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:18.026 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:18.026 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:18.026 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13640 2024-09-21T21:43:18.026 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:18.026 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:18.026 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:18.026 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:18.026 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945047+0000", 2024-09-21T21:43:18.026 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:18.027 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:18.027 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:18.027 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:18.027 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945047+0000", 2024-09-21T21:43:18.027 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:18.027 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:18.027 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:18.027 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:18.027 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945048+0000", 2024-09-21T21:43:18.027 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0499999999999999e-07 2024-09-21T21:43:18.027 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:18.027 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:18.027 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:18.027 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945049+0000", 2024-09-21T21:43:18.028 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.3939999999999999e-06 2024-09-21T21:43:18.028 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:18.028 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:18.028 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:18.028 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945053+0000", 2024-09-21T21:43:18.028 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1760000000000003e-06 2024-09-21T21:43:18.028 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:18.028 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:18.028 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:18.028 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945056+0000", 2024-09-21T21:43:18.028 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.6129999999999998e-06 2024-09-21T21:43:18.028 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:18.028 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:18.028 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:18.029 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024360+0000", 2024-09-21T21:43:18.029 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079303645000000006 2024-09-21T21:43:18.029 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:18.029 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:18.029 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:18.029 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024460+0000", 2024-09-21T21:43:18.029 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000100189 2024-09-21T21:43:18.029 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:18.029 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:18.029 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:18.029 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:18.029 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:18.029 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:43:18.031 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T21:43:18.031 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:43:18.029+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:43:18.132 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:43:18.130+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:43:18.141 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T21:43:18.184 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:43:18.184 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:43:18.184 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 0 2024-09-21T21:43:18.185 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:43:18.194 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T21:43:18.218 INFO:teuthology.orchestra.run.smithi081.stderr:noscrub is unset 2024-09-21T21:43:18.232 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:43:18.231+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:43:18.291 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0: {} 2024-09-21T21:43:18.291 INFO:teuthology.orchestra.run.smithi081.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T21:43:18.311 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1: {} 2024-09-21T21:43:18.311 INFO:teuthology.orchestra.run.smithi081.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T21:43:18.320 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:18.321 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:18.330 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T21:43:18.331 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2: {} 2024-09-21T21:43:18.331 INFO:teuthology.orchestra.run.smithi081.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T21:43:18.333 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:43:18.332+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:43:18.348 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3: {} 2024-09-21T21:43:18.349 INFO:teuthology.orchestra.run.smithi081.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T21:43:18.368 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4: {} 2024-09-21T21:43:18.368 INFO:teuthology.orchestra.run.smithi081.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T21:43:18.388 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5: {} 2024-09-21T21:43:18.388 INFO:teuthology.orchestra.run.smithi081.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T21:43:18.434 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:43:18.433+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:43:18.440 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:18.440 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:18.449 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T21:43:18.535 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:43:18.534+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:43:18.592 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:18.593 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:18.601 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T21:43:18.636 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:43:18.634+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:43:18.704 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:18.704 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:18.713 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T21:43:18.757 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:43:18.756+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:43:18.817 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:18.817 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:18.826 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T21:43:18.867 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:43:18.866+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:43:18.934 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:18.935 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:18.938 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:43:18.936+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:43:18.944 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T21:43:19.038 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:43:19.037+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:43:19.049 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:19.049 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:19.059 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T21:43:19.143 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:43:19.142+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:43:19.210 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:19.210 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:19.219 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T21:43:19.260 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:43:19.259+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:43:19.326 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 5, 3, 0, 4] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-21T21:43:19.326 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T21:43:19.327 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-21T21:43:19.327 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-21T21:43:19.363 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:19.363 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:19.372 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T21:43:19.372 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:43:19.370+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:43:19.441 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:43:19.440+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:43:19.456 INFO:teuthology.orchestra.run.smithi081.stderr:no valid command found; 10 closest matches: 2024-09-21T21:43:19.457 INFO:teuthology.orchestra.run.smithi081.stderr:0 2024-09-21T21:43:19.457 INFO:teuthology.orchestra.run.smithi081.stderr:1 2024-09-21T21:43:19.457 INFO:teuthology.orchestra.run.smithi081.stderr:2 2024-09-21T21:43:19.457 INFO:teuthology.orchestra.run.smithi081.stderr:abort 2024-09-21T21:43:19.457 INFO:teuthology.orchestra.run.smithi081.stderr:assert 2024-09-21T21:43:19.457 INFO:teuthology.orchestra.run.smithi081.stderr:bench [] [] [] [] 2024-09-21T21:43:19.457 INFO:teuthology.orchestra.run.smithi081.stderr:bluefs debug_inject_read_zeros 2024-09-21T21:43:19.457 INFO:teuthology.orchestra.run.smithi081.stderr:bluefs files list 2024-09-21T21:43:19.457 INFO:teuthology.orchestra.run.smithi081.stderr:bluefs stats 2024-09-21T21:43:19.457 INFO:teuthology.orchestra.run.smithi081.stderr:bluestore allocator dump block 2024-09-21T21:43:19.457 INFO:teuthology.orchestra.run.smithi081.stderr:Error EINVAL: invalid command 2024-09-21T21:43:19.459 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-21T21:43:19.460 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-21T21:43:19.460 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-21T21:43:19.475 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:19.475 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:19.484 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T21:43:19.544 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:43:19.543+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:43:19.586 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:19.586 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:19.595 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T21:43:19.630 INFO:teuthology.orchestra.run.smithi081.stderr:no valid command found; 10 closest matches: 2024-09-21T21:43:19.630 INFO:teuthology.orchestra.run.smithi081.stderr:0 2024-09-21T21:43:19.630 INFO:teuthology.orchestra.run.smithi081.stderr:1 2024-09-21T21:43:19.630 INFO:teuthology.orchestra.run.smithi081.stderr:2 2024-09-21T21:43:19.630 INFO:teuthology.orchestra.run.smithi081.stderr:abort 2024-09-21T21:43:19.630 INFO:teuthology.orchestra.run.smithi081.stderr:assert 2024-09-21T21:43:19.630 INFO:teuthology.orchestra.run.smithi081.stderr:bench [] [] [] [] 2024-09-21T21:43:19.630 INFO:teuthology.orchestra.run.smithi081.stderr:bluefs debug_inject_read_zeros 2024-09-21T21:43:19.630 INFO:teuthology.orchestra.run.smithi081.stderr:bluefs files list 2024-09-21T21:43:19.631 INFO:teuthology.orchestra.run.smithi081.stderr:bluefs stats 2024-09-21T21:43:19.631 INFO:teuthology.orchestra.run.smithi081.stderr:bluestore allocator dump block 2024-09-21T21:43:19.631 INFO:teuthology.orchestra.run.smithi081.stderr:Error EINVAL: invalid command 2024-09-21T21:43:19.633 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-21T21:43:19.634 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-21T21:43:19.634 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 'trim stale osdmaps' 2024-09-21T21:43:19.675 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:43:19.674+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:43:19.697 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:19.697 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:19.706 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T21:43:19.744 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:43:19.742+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:43:19.803 INFO:teuthology.orchestra.run.smithi081.stderr:no valid command found; 10 closest matches: 2024-09-21T21:43:19.803 INFO:teuthology.orchestra.run.smithi081.stderr:0 2024-09-21T21:43:19.803 INFO:teuthology.orchestra.run.smithi081.stderr:1 2024-09-21T21:43:19.804 INFO:teuthology.orchestra.run.smithi081.stderr:2 2024-09-21T21:43:19.804 INFO:teuthology.orchestra.run.smithi081.stderr:abort 2024-09-21T21:43:19.804 INFO:teuthology.orchestra.run.smithi081.stderr:assert 2024-09-21T21:43:19.804 INFO:teuthology.orchestra.run.smithi081.stderr:bench [] [] [] [] 2024-09-21T21:43:19.804 INFO:teuthology.orchestra.run.smithi081.stderr:bluefs debug_inject_read_zeros 2024-09-21T21:43:19.804 INFO:teuthology.orchestra.run.smithi081.stderr:bluefs files list 2024-09-21T21:43:19.804 INFO:teuthology.orchestra.run.smithi081.stderr:bluefs stats 2024-09-21T21:43:19.804 INFO:teuthology.orchestra.run.smithi081.stderr:bluestore allocator dump block 2024-09-21T21:43:19.804 INFO:teuthology.orchestra.run.smithi081.stderr:Error EINVAL: invalid command 2024-09-21T21:43:19.807 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-21T21:43:19.807 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-21T21:43:19.807 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-21T21:43:19.809 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:19.809 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:19.818 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T21:43:19.848 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:43:19.847+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:43:19.921 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:19.921 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:19.929 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T21:43:19.945 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:43:19.943+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:43:19.977 INFO:teuthology.orchestra.run.smithi081.stderr:no valid command found; 10 closest matches: 2024-09-21T21:43:19.977 INFO:teuthology.orchestra.run.smithi081.stderr:0 2024-09-21T21:43:19.978 INFO:teuthology.orchestra.run.smithi081.stderr:1 2024-09-21T21:43:19.978 INFO:teuthology.orchestra.run.smithi081.stderr:2 2024-09-21T21:43:19.978 INFO:teuthology.orchestra.run.smithi081.stderr:abort 2024-09-21T21:43:19.978 INFO:teuthology.orchestra.run.smithi081.stderr:assert 2024-09-21T21:43:19.978 INFO:teuthology.orchestra.run.smithi081.stderr:bench [] [] [] [] 2024-09-21T21:43:19.978 INFO:teuthology.orchestra.run.smithi081.stderr:bluefs debug_inject_read_zeros 2024-09-21T21:43:19.978 INFO:teuthology.orchestra.run.smithi081.stderr:bluefs files list 2024-09-21T21:43:19.978 INFO:teuthology.orchestra.run.smithi081.stderr:bluefs stats 2024-09-21T21:43:19.978 INFO:teuthology.orchestra.run.smithi081.stderr:bluestore allocator dump block 2024-09-21T21:43:19.978 INFO:teuthology.orchestra.run.smithi081.stderr:Error EINVAL: invalid command 2024-09-21T21:43:19.980 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-21T21:43:19.981 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-21T21:43:19.981 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-21T21:43:20.035 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:20.036 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:20.045 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T21:43:20.046 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:43:20.044+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:43:20.113 INFO:teuthology.orchestra.run.smithi081.stderr:no valid command found; 10 closest matches: 2024-09-21T21:43:20.113 INFO:teuthology.orchestra.run.smithi081.stderr:0 2024-09-21T21:43:20.113 INFO:teuthology.orchestra.run.smithi081.stderr:1 2024-09-21T21:43:20.113 INFO:teuthology.orchestra.run.smithi081.stderr:2 2024-09-21T21:43:20.113 INFO:teuthology.orchestra.run.smithi081.stderr:abort 2024-09-21T21:43:20.113 INFO:teuthology.orchestra.run.smithi081.stderr:assert 2024-09-21T21:43:20.113 INFO:teuthology.orchestra.run.smithi081.stderr:bench [] [] [] [] 2024-09-21T21:43:20.114 INFO:teuthology.orchestra.run.smithi081.stderr:bluefs debug_inject_read_zeros 2024-09-21T21:43:20.114 INFO:teuthology.orchestra.run.smithi081.stderr:bluefs files list 2024-09-21T21:43:20.114 INFO:teuthology.orchestra.run.smithi081.stderr:bluefs stats 2024-09-21T21:43:20.114 INFO:teuthology.orchestra.run.smithi081.stderr:bluestore allocator dump block 2024-09-21T21:43:20.114 INFO:teuthology.orchestra.run.smithi081.stderr:Error EINVAL: invalid command 2024-09-21T21:43:20.117 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-21T21:43:20.117 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-21T21:43:20.117 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 'trim stale osdmaps' 2024-09-21T21:43:20.146 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:43:20.145+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:43:20.152 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:20.152 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:20.161 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T21:43:20.238 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T21:43:20.247 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:43:20.245+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:43:20.260 INFO:teuthology.orchestra.run.smithi081.stderr:no valid command found; 10 closest matches: 2024-09-21T21:43:20.260 INFO:teuthology.orchestra.run.smithi081.stderr:0 2024-09-21T21:43:20.260 INFO:teuthology.orchestra.run.smithi081.stderr:1 2024-09-21T21:43:20.260 INFO:teuthology.orchestra.run.smithi081.stderr:2 2024-09-21T21:43:20.260 INFO:teuthology.orchestra.run.smithi081.stderr:abort 2024-09-21T21:43:20.260 INFO:teuthology.orchestra.run.smithi081.stderr:assert 2024-09-21T21:43:20.260 INFO:teuthology.orchestra.run.smithi081.stderr:bench [] [] [] [] 2024-09-21T21:43:20.260 INFO:teuthology.orchestra.run.smithi081.stderr:bluefs debug_inject_read_zeros 2024-09-21T21:43:20.260 INFO:teuthology.orchestra.run.smithi081.stderr:bluefs files list 2024-09-21T21:43:20.261 INFO:teuthology.orchestra.run.smithi081.stderr:bluefs stats 2024-09-21T21:43:20.261 INFO:teuthology.orchestra.run.smithi081.stderr:bluestore allocator dump block 2024-09-21T21:43:20.261 INFO:teuthology.orchestra.run.smithi081.stderr:Error EINVAL: invalid command 2024-09-21T21:43:20.263 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-21T21:43:20.264 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-21T21:43:20.318 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:20.318 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:20.328 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T21:43:20.348 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:43:20.347+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:43:20.400 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T21:43:20.434 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:20.434 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:20.443 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T21:43:20.449 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:43:20.448+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:43:20.523 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0: {} 2024-09-21T21:43:20.524 INFO:teuthology.orchestra.run.smithi081.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T21:43:20.540 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1: {} 2024-09-21T21:43:20.540 INFO:teuthology.orchestra.run.smithi081.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T21:43:20.550 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:43:20.550+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:43:20.557 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2: {} 2024-09-21T21:43:20.557 INFO:teuthology.orchestra.run.smithi081.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T21:43:20.574 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3: {} 2024-09-21T21:43:20.574 INFO:teuthology.orchestra.run.smithi081.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T21:43:20.591 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4: {} 2024-09-21T21:43:20.591 INFO:teuthology.orchestra.run.smithi081.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T21:43:20.591 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:43:20.592 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:43:20.592 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:43:20.592 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:43:20.592 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:43:20.601 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T21:43:20.608 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5: {} 2024-09-21T21:43:20.608 INFO:teuthology.orchestra.run.smithi081.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T21:43:20.659 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:43:20.658+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:43:20.713 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:43:20.713 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 20, 2024-09-21T21:43:20.713 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 600, 2024-09-21T21:43:20.713 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:43:20.713 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.714 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4824.0:1564 3.2 3:4647520c:::rbd_data.1121b97f2925.0000000000000006:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:43:20.714 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:43:20.714 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 139.07458606500001, 2024-09-21T21:43:20.714 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056097347999999998, 2024-09-21T21:43:20.714 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:20.714 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:20.714 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:20.714 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:20.714 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4824", 2024-09-21T21:43:20.714 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:43:20.714 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1564 2024-09-21T21:43:20.714 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.714 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:20.714 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.715 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:20.715 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:43:20.715 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:20.715 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.715 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.715 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:20.715 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:43:20.715 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:20.715 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.715 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.715 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:20.715 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:43:20.715 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6499999999999999e-07 2024-09-21T21:43:20.715 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.716 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.716 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:20.716 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636067+0000", 2024-09-21T21:43:20.716 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3879999999999998e-06 2024-09-21T21:43:20.716 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.716 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.716 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:20.716 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636075+0000", 2024-09-21T21:43:20.716 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5480000000000004e-06 2024-09-21T21:43:20.716 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.716 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.716 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:20.716 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636080+0000", 2024-09-21T21:43:20.716 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.339e-06 2024-09-21T21:43:20.717 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.717 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.717 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:20.717 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636133+0000", 2024-09-21T21:43:20.717 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2930000000000003e-05 2024-09-21T21:43:20.717 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.717 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.717 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:20.717 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636265+0000", 2024-09-21T21:43:20.717 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000132058 2024-09-21T21:43:20.717 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.717 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.717 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:20.717 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.692162+0000", 2024-09-21T21:43:20.718 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055896620000000001 2024-09-21T21:43:20.718 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:20.718 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:20.718 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:20.718 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.718 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.718 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3260 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 1437696~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:43:20.718 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.196297+0000", 2024-09-21T21:43:20.718 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 93.514353854999996, 2024-09-21T21:43:20.718 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056071022999999998, 2024-09-21T21:43:20.718 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:20.718 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:20.719 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:20.719 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:20.719 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:20.719 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:20.719 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3260 2024-09-21T21:43:20.719 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.719 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:20.719 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.719 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:20.719 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196297+0000", 2024-09-21T21:43:20.719 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:20.719 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.719 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.719 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:20.719 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196297+0000", 2024-09-21T21:43:20.720 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:20.720 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.720 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.720 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:20.720 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196918+0000", 2024-09-21T21:43:20.720 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00062148699999999999 2024-09-21T21:43:20.720 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.720 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.720 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:20.720 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196930+0000", 2024-09-21T21:43:20.720 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1657e-05 2024-09-21T21:43:20.720 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.720 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.720 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:20.721 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196944+0000", 2024-09-21T21:43:20.721 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.4549000000000001e-05 2024-09-21T21:43:20.721 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.721 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.721 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:20.721 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196951+0000", 2024-09-21T21:43:20.721 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6749999999999996e-06 2024-09-21T21:43:20.721 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.721 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.721 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:20.721 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252208+0000", 2024-09-21T21:43:20.721 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055256656000000001 2024-09-21T21:43:20.721 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.721 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.721 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:20.722 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252278+0000", 2024-09-21T21:43:20.722 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9882000000000006e-05 2024-09-21T21:43:20.722 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.722 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.722 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:20.722 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252368+0000", 2024-09-21T21:43:20.722 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.0117000000000005e-05 2024-09-21T21:43:20.722 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:20.722 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:20.722 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:20.723 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.723 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.723 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3437 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 925696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:43:20.723 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.205888+0000", 2024-09-21T21:43:20.723 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 93.504762760000006, 2024-09-21T21:43:20.723 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055953808000000001, 2024-09-21T21:43:20.723 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:20.723 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:20.723 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:20.723 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:20.723 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:20.723 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:20.723 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3437 2024-09-21T21:43:20.723 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.723 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:20.724 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.724 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:20.724 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205888+0000", 2024-09-21T21:43:20.724 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:20.724 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.724 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.724 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:20.724 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205888+0000", 2024-09-21T21:43:20.724 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:20.724 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.724 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.724 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:20.724 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205888+0000", 2024-09-21T21:43:20.725 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.75e-07 2024-09-21T21:43:20.725 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.725 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.725 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:20.725 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205891+0000", 2024-09-21T21:43:20.726 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3750000000000001e-06 2024-09-21T21:43:20.726 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.726 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.726 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:20.726 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205897+0000", 2024-09-21T21:43:20.726 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.285e-06 2024-09-21T21:43:20.726 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.726 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.726 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:20.726 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205902+0000", 2024-09-21T21:43:20.726 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6290000000000001e-06 2024-09-21T21:43:20.726 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.726 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.727 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:20.727 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261686+0000", 2024-09-21T21:43:20.727 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055784586999999997 2024-09-21T21:43:20.727 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.727 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.727 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:20.727 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261764+0000", 2024-09-21T21:43:20.727 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7591999999999998e-05 2024-09-21T21:43:20.727 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.727 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.727 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:20.727 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261842+0000", 2024-09-21T21:43:20.727 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7865000000000001e-05 2024-09-21T21:43:20.727 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:20.728 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:20.728 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:20.728 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.728 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.728 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3438 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 954368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:43:20.728 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.205908+0000", 2024-09-21T21:43:20.728 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 93.504742895000007, 2024-09-21T21:43:20.728 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056083169000000002, 2024-09-21T21:43:20.728 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:20.728 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:20.728 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:20.728 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:20.728 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:20.728 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:20.729 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3438 2024-09-21T21:43:20.729 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.729 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:20.729 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.729 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:20.729 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205908+0000", 2024-09-21T21:43:20.729 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:20.729 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.729 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.729 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:20.729 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205908+0000", 2024-09-21T21:43:20.729 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:20.729 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.729 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.730 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:20.730 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206431+0000", 2024-09-21T21:43:20.730 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00052282100000000003 2024-09-21T21:43:20.730 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.730 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.730 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:20.730 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206442+0000", 2024-09-21T21:43:20.730 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1695000000000001e-05 2024-09-21T21:43:20.730 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.730 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.730 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:20.730 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206452+0000", 2024-09-21T21:43:20.730 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.4560000000000006e-06 2024-09-21T21:43:20.731 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.731 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.731 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:20.731 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206458+0000", 2024-09-21T21:43:20.731 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4670000000000001e-06 2024-09-21T21:43:20.731 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.731 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.731 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:20.731 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261871+0000", 2024-09-21T21:43:20.731 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055412712000000003 2024-09-21T21:43:20.731 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.731 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.731 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:20.732 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261928+0000", 2024-09-21T21:43:20.732 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6631e-05 2024-09-21T21:43:20.732 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.732 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.732 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:20.732 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261991+0000", 2024-09-21T21:43:20.732 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3386999999999995e-05 2024-09-21T21:43:20.732 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:20.732 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:20.732 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:20.732 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.732 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.732 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3444 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 2359296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:43:20.732 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.206533+0000", 2024-09-21T21:43:20.733 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 93.504117375000007, 2024-09-21T21:43:20.733 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055974239000000002, 2024-09-21T21:43:20.733 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:20.733 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:20.733 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:20.733 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:20.733 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:20.733 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:20.733 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3444 2024-09-21T21:43:20.733 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.733 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:20.733 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.733 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:20.733 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206533+0000", 2024-09-21T21:43:20.733 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:20.734 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.734 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.734 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:20.734 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206533+0000", 2024-09-21T21:43:20.734 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:20.734 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.734 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.734 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:20.734 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206710+0000", 2024-09-21T21:43:20.734 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000176293 2024-09-21T21:43:20.734 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.734 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.734 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:20.734 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206714+0000", 2024-09-21T21:43:20.735 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1989999999999999e-06 2024-09-21T21:43:20.735 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.735 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.735 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:20.735 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206723+0000", 2024-09-21T21:43:20.735 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.9199999999999993e-06 2024-09-21T21:43:20.735 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.735 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.735 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:20.735 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206729+0000", 2024-09-21T21:43:20.735 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0739999999999998e-06 2024-09-21T21:43:20.735 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.735 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.735 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:20.736 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262339+0000", 2024-09-21T21:43:20.736 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055610166000000003 2024-09-21T21:43:20.736 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.736 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.736 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:20.736 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262419+0000", 2024-09-21T21:43:20.736 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0432999999999994e-05 2024-09-21T21:43:20.736 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.736 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.736 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:20.736 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262507+0000", 2024-09-21T21:43:20.736 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.8153999999999997e-05 2024-09-21T21:43:20.736 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:20.736 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:20.737 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:20.737 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.737 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.737 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10362 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:43:20.737 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840386+0000", 2024-09-21T21:43:20.737 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 66.870264531000004, 2024-09-21T21:43:20.737 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056546515999999998, 2024-09-21T21:43:20.737 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:20.737 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:20.737 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:20.737 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:20.737 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:43:20.738 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:43:20.738 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10362 2024-09-21T21:43:20.738 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.738 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:20.738 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.738 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:20.738 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840386+0000", 2024-09-21T21:43:20.738 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:20.738 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.738 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.738 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:20.738 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840386+0000", 2024-09-21T21:43:20.738 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:20.738 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.739 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.739 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:20.739 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840386+0000", 2024-09-21T21:43:20.739 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.01e-07 2024-09-21T21:43:20.739 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.739 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.739 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:20.739 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840389+0000", 2024-09-21T21:43:20.739 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.9739999999999998e-06 2024-09-21T21:43:20.739 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.739 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.739 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:20.739 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840396+0000", 2024-09-21T21:43:20.739 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2280000000000003e-06 2024-09-21T21:43:20.739 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.740 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.740 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:20.740 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840400+0000", 2024-09-21T21:43:20.740 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4429999999999996e-06 2024-09-21T21:43:20.740 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.740 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.740 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:20.740 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.890896+0000", 2024-09-21T21:43:20.740 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.050495717000000002 2024-09-21T21:43:20.740 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.740 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.740 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:20.740 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.891012+0000", 2024-09-21T21:43:20.740 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011602 2024-09-21T21:43:20.741 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.741 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.741 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:43:20.741 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.891274+0000", 2024-09-21T21:43:20.741 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00026240200000000001 2024-09-21T21:43:20.741 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.741 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.741 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:43:20.741 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.894810+0000", 2024-09-21T21:43:20.741 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0035353630000000001 2024-09-21T21:43:20.741 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.741 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.741 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:20.742 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.896818+0000", 2024-09-21T21:43:20.742 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.002008681 2024-09-21T21:43:20.742 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.742 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.742 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:43:20.742 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.896850+0000", 2024-09-21T21:43:20.742 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.1826999999999999e-05 2024-09-21T21:43:20.742 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.742 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.742 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:20.742 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.896933+0000", 2024-09-21T21:43:20.742 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.2460000000000007e-05 2024-09-21T21:43:20.742 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:20.743 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:20.743 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:20.743 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.743 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.743 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10363 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:43:20.743 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840606+0000", 2024-09-21T21:43:20.743 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 66.870045020999996, 2024-09-21T21:43:20.743 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056546011, 2024-09-21T21:43:20.743 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:20.743 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:20.744 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:20.744 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:20.744 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:43:20.744 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:43:20.744 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10363 2024-09-21T21:43:20.744 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.745 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:20.745 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.745 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:20.745 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840606+0000", 2024-09-21T21:43:20.745 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:20.745 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.745 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.745 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:20.745 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840606+0000", 2024-09-21T21:43:20.745 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:20.745 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.745 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.745 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:20.746 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840606+0000", 2024-09-21T21:43:20.746 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.4700000000000001e-07 2024-09-21T21:43:20.746 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.746 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.746 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:20.746 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840611+0000", 2024-09-21T21:43:20.746 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4649999999999996e-06 2024-09-21T21:43:20.746 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.746 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.746 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:20.746 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840620+0000", 2024-09-21T21:43:20.746 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.0350000000000007e-06 2024-09-21T21:43:20.746 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.747 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.747 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:20.747 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840626+0000", 2024-09-21T21:43:20.747 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7180000000000001e-06 2024-09-21T21:43:20.747 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.747 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.747 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:20.747 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.891665+0000", 2024-09-21T21:43:20.747 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.051039924 2024-09-21T21:43:20.747 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.747 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.747 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:20.747 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.891770+0000", 2024-09-21T21:43:20.748 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010445700000000001 2024-09-21T21:43:20.748 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.748 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.748 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:43:20.748 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.892147+0000", 2024-09-21T21:43:20.748 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00037731800000000002 2024-09-21T21:43:20.748 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.748 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.748 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:43:20.748 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.896776+0000", 2024-09-21T21:43:20.748 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0046289759999999999 2024-09-21T21:43:20.748 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.749 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.749 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:20.749 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897003+0000", 2024-09-21T21:43:20.749 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00022641299999999999 2024-09-21T21:43:20.749 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.749 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.749 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:43:20.749 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897064+0000", 2024-09-21T21:43:20.749 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1571999999999999e-05 2024-09-21T21:43:20.749 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.749 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.749 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:20.749 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897152+0000", 2024-09-21T21:43:20.749 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.7386000000000007e-05 2024-09-21T21:43:20.750 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:20.750 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:20.750 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:20.750 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.750 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.750 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10364 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:43:20.750 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840641+0000", 2024-09-21T21:43:20.750 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 66.870009201000002, 2024-09-21T21:43:20.750 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056671527999999999, 2024-09-21T21:43:20.750 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:20.750 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:20.750 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:20.750 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:20.751 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:43:20.751 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:43:20.751 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10364 2024-09-21T21:43:20.751 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.751 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:20.751 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.751 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:20.751 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840641+0000", 2024-09-21T21:43:20.751 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:20.751 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.751 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.751 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:20.751 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840641+0000", 2024-09-21T21:43:20.751 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:20.752 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.752 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.752 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:20.752 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840642+0000", 2024-09-21T21:43:20.752 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7899999999999999e-07 2024-09-21T21:43:20.752 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.752 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.752 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:20.752 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840645+0000", 2024-09-21T21:43:20.752 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0010000000000002e-06 2024-09-21T21:43:20.752 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.752 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.752 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:20.752 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840654+0000", 2024-09-21T21:43:20.753 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.8570000000000001e-06 2024-09-21T21:43:20.753 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.753 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.753 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:20.753 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840659+0000", 2024-09-21T21:43:20.753 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.558e-06 2024-09-21T21:43:20.753 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.753 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.753 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:20.753 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.892656+0000", 2024-09-21T21:43:20.753 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.051996586999999997 2024-09-21T21:43:20.753 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.753 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.753 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:20.753 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.892721+0000", 2024-09-21T21:43:20.754 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5010000000000003e-05 2024-09-21T21:43:20.754 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.754 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.754 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:43:20.754 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.893112+0000", 2024-09-21T21:43:20.754 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00039133899999999999 2024-09-21T21:43:20.754 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.754 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.754 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:43:20.754 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.896784+0000", 2024-09-21T21:43:20.754 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0036713679999999999 2024-09-21T21:43:20.754 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.754 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.754 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:20.755 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897204+0000", 2024-09-21T21:43:20.755 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00041991400000000002 2024-09-21T21:43:20.755 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.755 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.755 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:43:20.755 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897234+0000", 2024-09-21T21:43:20.755 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0307999999999998e-05 2024-09-21T21:43:20.755 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.755 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.755 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:20.755 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897313+0000", 2024-09-21T21:43:20.755 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.9107e-05 2024-09-21T21:43:20.755 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:20.755 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:20.755 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:20.756 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.756 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.756 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10365 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:43:20.756 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840668+0000", 2024-09-21T21:43:20.756 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 66.869982336000007, 2024-09-21T21:43:20.756 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.058761225, 2024-09-21T21:43:20.756 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:20.756 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:20.756 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:20.756 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:20.756 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:43:20.756 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:43:20.756 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10365 2024-09-21T21:43:20.756 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.756 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:20.757 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.757 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:20.757 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840668+0000", 2024-09-21T21:43:20.757 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:20.757 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.757 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.757 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:20.757 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840668+0000", 2024-09-21T21:43:20.757 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:20.757 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.757 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.757 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:20.757 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840669+0000", 2024-09-21T21:43:20.757 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6300000000000001e-07 2024-09-21T21:43:20.758 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.758 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.758 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:20.758 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840672+0000", 2024-09-21T21:43:20.758 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.072e-06 2024-09-21T21:43:20.758 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.758 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.758 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:20.758 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840678+0000", 2024-09-21T21:43:20.758 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.7039999999999997e-06 2024-09-21T21:43:20.758 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.758 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.758 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:20.758 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840683+0000", 2024-09-21T21:43:20.759 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6759999999999998e-06 2024-09-21T21:43:20.759 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.759 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.759 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:20.759 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.893929+0000", 2024-09-21T21:43:20.759 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.053246136999999999 2024-09-21T21:43:20.759 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.759 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.759 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:20.759 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.894082+0000", 2024-09-21T21:43:20.759 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00015301799999999999 2024-09-21T21:43:20.759 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.759 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.759 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:43:20.759 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.894357+0000", 2024-09-21T21:43:20.760 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00027507999999999998 2024-09-21T21:43:20.760 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.760 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.760 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:43:20.760 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897161+0000", 2024-09-21T21:43:20.760 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0028036810000000001 2024-09-21T21:43:20.760 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.760 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.760 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:20.760 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.899243+0000", 2024-09-21T21:43:20.760 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0020816379999999998 2024-09-21T21:43:20.760 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.760 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.761 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:43:20.761 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.899306+0000", 2024-09-21T21:43:20.761 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2964000000000002e-05 2024-09-21T21:43:20.761 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.761 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.761 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:20.761 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.899430+0000", 2024-09-21T21:43:20.761 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00012389200000000001 2024-09-21T21:43:20.761 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:20.761 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:20.761 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:20.761 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.761 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.761 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10366 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:43:20.762 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840691+0000", 2024-09-21T21:43:20.762 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 66.869959636000004, 2024-09-21T21:43:20.762 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.061540984999999999, 2024-09-21T21:43:20.762 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:20.762 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:20.762 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:20.762 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:20.762 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:43:20.762 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:43:20.762 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10366 2024-09-21T21:43:20.762 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.762 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:20.762 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.763 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:20.763 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840691+0000", 2024-09-21T21:43:20.763 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:20.763 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.763 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.763 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:20.763 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840691+0000", 2024-09-21T21:43:20.763 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:20.763 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.763 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.763 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:20.763 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840691+0000", 2024-09-21T21:43:20.763 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3099999999999999e-07 2024-09-21T21:43:20.763 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.763 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.764 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:20.764 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840694+0000", 2024-09-21T21:43:20.764 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.5780000000000001e-06 2024-09-21T21:43:20.764 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.764 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.764 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:20.764 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840700+0000", 2024-09-21T21:43:20.764 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9429999999999997e-06 2024-09-21T21:43:20.764 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.764 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.764 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:20.764 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840704+0000", 2024-09-21T21:43:20.764 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6380000000000003e-06 2024-09-21T21:43:20.764 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.765 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.765 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:20.765 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.894846+0000", 2024-09-21T21:43:20.765 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.054141392000000003 2024-09-21T21:43:20.765 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.765 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.765 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:20.765 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.894932+0000", 2024-09-21T21:43:20.765 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.5834000000000005e-05 2024-09-21T21:43:20.765 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.765 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.765 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:43:20.765 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.895302+0000", 2024-09-21T21:43:20.765 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000370737 2024-09-21T21:43:20.766 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.766 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.766 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:43:20.766 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897166+0000", 2024-09-21T21:43:20.766 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0018632620000000001 2024-09-21T21:43:20.766 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.766 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.766 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:20.766 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902053+0000", 2024-09-21T21:43:20.766 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0048868289999999997 2024-09-21T21:43:20.766 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.766 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.766 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:43:20.766 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902111+0000", 2024-09-21T21:43:20.766 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7983999999999999e-05 2024-09-21T21:43:20.767 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.767 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.767 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:20.767 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902232+0000", 2024-09-21T21:43:20.767 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000121457 2024-09-21T21:43:20.767 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:20.767 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:20.767 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:20.767 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.767 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.767 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10367 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:43:20.767 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840842+0000", 2024-09-21T21:43:20.768 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 66.869808828000004, 2024-09-21T21:43:20.768 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.061549088000000002, 2024-09-21T21:43:20.768 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:20.768 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:20.768 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:20.768 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:20.768 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:43:20.768 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:43:20.768 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10367 2024-09-21T21:43:20.768 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.768 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:20.768 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.768 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:20.769 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840842+0000", 2024-09-21T21:43:20.769 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:20.769 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.769 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.769 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:20.769 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840842+0000", 2024-09-21T21:43:20.769 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:20.769 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.769 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.769 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:20.769 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840842+0000", 2024-09-21T21:43:20.769 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3099999999999997e-07 2024-09-21T21:43:20.769 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.769 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.770 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:20.770 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840846+0000", 2024-09-21T21:43:20.770 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.2890000000000002e-06 2024-09-21T21:43:20.770 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.770 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.770 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:20.770 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840855+0000", 2024-09-21T21:43:20.770 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.9619999999999999e-06 2024-09-21T21:43:20.770 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.770 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.770 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:20.770 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840860+0000", 2024-09-21T21:43:20.770 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3759999999999999e-06 2024-09-21T21:43:20.770 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.771 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.771 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:20.771 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.895799+0000", 2024-09-21T21:43:20.771 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.054938569999999999 2024-09-21T21:43:20.771 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.771 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.771 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:20.771 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.895906+0000", 2024-09-21T21:43:20.771 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000107593 2024-09-21T21:43:20.771 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.771 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.771 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:43:20.771 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.896356+0000", 2024-09-21T21:43:20.771 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00044941700000000002 2024-09-21T21:43:20.771 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.772 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.772 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:43:20.772 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.898131+0000", 2024-09-21T21:43:20.772 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.001775064 2024-09-21T21:43:20.772 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.772 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.772 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:20.772 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902283+0000", 2024-09-21T21:43:20.772 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0041524839999999997 2024-09-21T21:43:20.772 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.772 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.772 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:43:20.772 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902316+0000", 2024-09-21T21:43:20.772 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3002e-05 2024-09-21T21:43:20.773 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.773 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.773 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:20.773 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902391+0000", 2024-09-21T21:43:20.773 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.47e-05 2024-09-21T21:43:20.773 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:20.773 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:20.773 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:20.773 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.773 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.773 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10368 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:43:20.773 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840873+0000", 2024-09-21T21:43:20.773 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 66.869777873999993, 2024-09-21T21:43:20.773 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.063395291000000006, 2024-09-21T21:43:20.773 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:20.774 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:20.774 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:20.774 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:20.774 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:43:20.774 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:43:20.774 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10368 2024-09-21T21:43:20.774 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.774 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:20.774 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.774 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:20.774 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840873+0000", 2024-09-21T21:43:20.774 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:20.774 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.774 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.775 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:20.775 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840873+0000", 2024-09-21T21:43:20.775 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:20.775 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.775 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.775 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:20.775 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840873+0000", 2024-09-21T21:43:20.775 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6699999999999999e-07 2024-09-21T21:43:20.775 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.775 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.775 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:20.775 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840876+0000", 2024-09-21T21:43:20.775 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.6410000000000002e-06 2024-09-21T21:43:20.775 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.775 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.776 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:20.776 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840883+0000", 2024-09-21T21:43:20.776 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0389999999999999e-06 2024-09-21T21:43:20.776 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.776 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.776 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:20.776 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840888+0000", 2024-09-21T21:43:20.776 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8679999999999998e-06 2024-09-21T21:43:20.776 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.776 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.776 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:20.776 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897345+0000", 2024-09-21T21:43:20.776 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056457734000000002 2024-09-21T21:43:20.776 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.777 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.777 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:20.777 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897407+0000", 2024-09-21T21:43:20.777 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1822000000000005e-05 2024-09-21T21:43:20.777 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.777 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.777 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:43:20.777 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897664+0000", 2024-09-21T21:43:20.777 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00025638499999999998 2024-09-21T21:43:20.777 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.777 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.777 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:43:20.777 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.901136+0000", 2024-09-21T21:43:20.777 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0034728070000000001 2024-09-21T21:43:20.778 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.778 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.778 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:20.778 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904090+0000", 2024-09-21T21:43:20.778 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0029535799999999999 2024-09-21T21:43:20.778 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.778 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.778 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:43:20.778 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904144+0000", 2024-09-21T21:43:20.778 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4369000000000001e-05 2024-09-21T21:43:20.778 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.778 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.778 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:20.778 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904268+0000", 2024-09-21T21:43:20.779 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00012367899999999999 2024-09-21T21:43:20.779 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:20.779 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:20.779 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:20.779 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.779 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.779 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10369 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:43:20.779 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840902+0000", 2024-09-21T21:43:20.779 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 66.869748220999995, 2024-09-21T21:43:20.779 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.063538938000000003, 2024-09-21T21:43:20.779 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:20.779 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:20.779 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:20.779 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:20.780 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:43:20.780 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:43:20.780 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10369 2024-09-21T21:43:20.780 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.780 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:20.780 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.780 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:20.780 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840902+0000", 2024-09-21T21:43:20.780 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:20.780 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.780 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.780 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:20.780 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840902+0000", 2024-09-21T21:43:20.780 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:20.780 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.781 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.781 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:20.781 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840903+0000", 2024-09-21T21:43:20.781 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2100000000000002e-07 2024-09-21T21:43:20.781 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.781 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.781 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:20.781 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840913+0000", 2024-09-21T21:43:20.781 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.048e-05 2024-09-21T21:43:20.781 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.781 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.781 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:20.781 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840920+0000", 2024-09-21T21:43:20.781 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0330000000000003e-06 2024-09-21T21:43:20.781 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.782 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.782 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:20.782 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840925+0000", 2024-09-21T21:43:20.782 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0170000000000004e-06 2024-09-21T21:43:20.782 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.782 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.782 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:20.782 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.898179+0000", 2024-09-21T21:43:20.782 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.057253671999999999 2024-09-21T21:43:20.782 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.782 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.782 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:20.782 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.898284+0000", 2024-09-21T21:43:20.783 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000105202 2024-09-21T21:43:20.783 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.783 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.783 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:43:20.783 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.898729+0000", 2024-09-21T21:43:20.783 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00044459300000000002 2024-09-21T21:43:20.783 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.783 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.783 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:43:20.783 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902246+0000", 2024-09-21T21:43:20.783 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0035171629999999998 2024-09-21T21:43:20.783 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.783 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.783 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:20.784 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904310+0000", 2024-09-21T21:43:20.784 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.002063765 2024-09-21T21:43:20.784 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.784 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.784 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:43:20.784 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904342+0000", 2024-09-21T21:43:20.784 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.2091999999999997e-05 2024-09-21T21:43:20.784 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.784 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.784 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:20.784 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904441+0000", 2024-09-21T21:43:20.784 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.9500000000000006e-05 2024-09-21T21:43:20.785 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:20.785 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:20.785 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:20.785 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.785 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.785 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10370 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:43:20.785 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840944+0000", 2024-09-21T21:43:20.785 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 66.869706507999993, 2024-09-21T21:43:20.785 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.064436853000000002, 2024-09-21T21:43:20.785 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:20.785 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:20.785 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:20.785 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:20.786 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:43:20.786 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:43:20.786 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10370 2024-09-21T21:43:20.786 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.786 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:20.786 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.786 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:20.786 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840944+0000", 2024-09-21T21:43:20.786 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:20.786 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.786 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.786 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:20.786 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840944+0000", 2024-09-21T21:43:20.786 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:20.787 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.787 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.787 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:20.787 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840945+0000", 2024-09-21T21:43:20.787 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2300000000000002e-07 2024-09-21T21:43:20.787 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.787 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.787 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:20.787 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840947+0000", 2024-09-21T21:43:20.787 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.8200000000000001e-06 2024-09-21T21:43:20.787 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.787 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.787 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:20.788 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840955+0000", 2024-09-21T21:43:20.788 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.8660000000000006e-06 2024-09-21T21:43:20.788 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.788 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.788 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:20.788 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840960+0000", 2024-09-21T21:43:20.788 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8670000000000002e-06 2024-09-21T21:43:20.788 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.788 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.788 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:20.788 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.899460+0000", 2024-09-21T21:43:20.788 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.058499922000000003 2024-09-21T21:43:20.788 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.789 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.789 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:20.789 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.899530+0000", 2024-09-21T21:43:20.789 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9558e-05 2024-09-21T21:43:20.789 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.789 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.789 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:43:20.789 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.899775+0000", 2024-09-21T21:43:20.789 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000245231 2024-09-21T21:43:20.789 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.789 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.789 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:43:20.789 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902254+0000", 2024-09-21T21:43:20.789 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0024790519999999998 2024-09-21T21:43:20.790 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.790 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.790 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:20.790 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905183+0000", 2024-09-21T21:43:20.790 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.002929156 2024-09-21T21:43:20.790 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.790 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.790 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:43:20.790 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905243+0000", 2024-09-21T21:43:20.790 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0055e-05 2024-09-21T21:43:20.790 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.790 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.790 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:20.790 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905381+0000", 2024-09-21T21:43:20.791 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00013790300000000001 2024-09-21T21:43:20.791 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:20.791 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:20.791 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:20.791 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.791 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.791 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10371 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:43:20.791 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840980+0000", 2024-09-21T21:43:20.791 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 66.869670842000005, 2024-09-21T21:43:20.791 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.064554376999999996, 2024-09-21T21:43:20.791 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:20.791 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:20.791 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:20.792 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:20.792 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:43:20.792 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:43:20.792 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10371 2024-09-21T21:43:20.792 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.792 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:20.792 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.792 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:20.792 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840980+0000", 2024-09-21T21:43:20.792 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:20.792 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.792 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.792 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:20.792 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840980+0000", 2024-09-21T21:43:20.793 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:20.793 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.793 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.793 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:20.793 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840980+0000", 2024-09-21T21:43:20.793 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8999999999999997e-07 2024-09-21T21:43:20.793 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.793 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.793 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:20.793 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840983+0000", 2024-09-21T21:43:20.793 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.123e-06 2024-09-21T21:43:20.793 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.793 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.794 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:20.794 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840990+0000", 2024-09-21T21:43:20.794 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9129999999999997e-06 2024-09-21T21:43:20.794 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.794 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.794 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:20.794 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840995+0000", 2024-09-21T21:43:20.794 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0989999999999998e-06 2024-09-21T21:43:20.794 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.794 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.794 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:20.794 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.900168+0000", 2024-09-21T21:43:20.794 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.059172739000000002 2024-09-21T21:43:20.794 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.795 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.795 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:20.795 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.900274+0000", 2024-09-21T21:43:20.795 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010546499999999999 2024-09-21T21:43:20.795 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.795 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.795 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:43:20.795 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.900703+0000", 2024-09-21T21:43:20.795 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00042978200000000003 2024-09-21T21:43:20.795 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.795 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.795 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:43:20.795 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.903443+0000", 2024-09-21T21:43:20.795 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0027400749999999998 2024-09-21T21:43:20.796 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.796 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.796 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:20.796 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905430+0000", 2024-09-21T21:43:20.796 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0019868109999999998 2024-09-21T21:43:20.796 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.796 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.796 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:43:20.796 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905464+0000", 2024-09-21T21:43:20.796 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3599000000000001e-05 2024-09-21T21:43:20.796 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.796 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.796 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:20.796 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905534+0000", 2024-09-21T21:43:20.797 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0281e-05 2024-09-21T21:43:20.797 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:20.797 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:20.797 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:20.797 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.797 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.797 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10372 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:43:20.797 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.841004+0000", 2024-09-21T21:43:20.797 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 66.869646953, 2024-09-21T21:43:20.797 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.064757812999999997, 2024-09-21T21:43:20.797 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:20.797 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:20.797 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:20.797 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:20.798 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:43:20.798 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:43:20.798 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10372 2024-09-21T21:43:20.798 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.798 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:20.798 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.798 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:20.798 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841004+0000", 2024-09-21T21:43:20.798 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:20.798 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.798 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.798 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:20.798 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841004+0000", 2024-09-21T21:43:20.799 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:20.799 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.799 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.799 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:20.799 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841004+0000", 2024-09-21T21:43:20.799 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8500000000000002e-07 2024-09-21T21:43:20.799 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.799 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.799 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:20.799 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841007+0000", 2024-09-21T21:43:20.799 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.6390000000000001e-06 2024-09-21T21:43:20.800 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.800 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.800 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:20.800 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841013+0000", 2024-09-21T21:43:20.800 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0070000000000003e-06 2024-09-21T21:43:20.800 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.800 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.800 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:20.800 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841018+0000", 2024-09-21T21:43:20.800 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9119999999999997e-06 2024-09-21T21:43:20.800 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.800 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.800 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:20.800 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.901182+0000", 2024-09-21T21:43:20.801 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.060164444999999997 2024-09-21T21:43:20.801 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.801 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.801 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:20.801 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.901297+0000", 2024-09-21T21:43:20.801 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011468699999999999 2024-09-21T21:43:20.801 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.801 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.801 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:43:20.801 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.901559+0000", 2024-09-21T21:43:20.801 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000261764 2024-09-21T21:43:20.801 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.801 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.801 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:43:20.801 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904281+0000", 2024-09-21T21:43:20.802 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0027229329999999999 2024-09-21T21:43:20.802 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.802 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.802 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:20.802 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905588+0000", 2024-09-21T21:43:20.802 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.001306438 2024-09-21T21:43:20.802 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.802 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.802 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:43:20.802 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905644+0000", 2024-09-21T21:43:20.802 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5747e-05 2024-09-21T21:43:20.802 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.802 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.802 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:20.803 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905761+0000", 2024-09-21T21:43:20.803 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000117856 2024-09-21T21:43:20.803 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:20.803 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:20.803 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:20.803 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.803 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.803 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10373 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:43:20.803 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.841027+0000", 2024-09-21T21:43:20.803 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 66.869623868999994, 2024-09-21T21:43:20.803 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.066941839000000003, 2024-09-21T21:43:20.803 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:20.803 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:20.803 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:20.803 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:20.804 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:43:20.804 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:43:20.804 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10373 2024-09-21T21:43:20.804 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.804 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:20.804 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.804 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:20.804 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841027+0000", 2024-09-21T21:43:20.804 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:20.804 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.804 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.804 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:20.804 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841027+0000", 2024-09-21T21:43:20.804 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:20.804 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.805 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.805 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:20.805 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841027+0000", 2024-09-21T21:43:20.805 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6199999999999999e-07 2024-09-21T21:43:20.805 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.805 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.805 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:20.805 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841030+0000", 2024-09-21T21:43:20.805 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.729e-06 2024-09-21T21:43:20.805 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.805 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.805 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:20.805 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841036+0000", 2024-09-21T21:43:20.805 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1979999999999998e-06 2024-09-21T21:43:20.806 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.806 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.806 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:20.806 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841042+0000", 2024-09-21T21:43:20.806 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4289999999999998e-06 2024-09-21T21:43:20.806 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.806 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.806 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:20.806 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902433+0000", 2024-09-21T21:43:20.806 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.061390329 2024-09-21T21:43:20.806 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.806 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.806 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:20.806 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902541+0000", 2024-09-21T21:43:20.807 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000108349 2024-09-21T21:43:20.807 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.807 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.807 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:43:20.807 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902983+0000", 2024-09-21T21:43:20.807 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00044158000000000001 2024-09-21T21:43:20.807 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.807 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.807 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:43:20.807 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904490+0000", 2024-09-21T21:43:20.807 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0015074489999999999 2024-09-21T21:43:20.807 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.807 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.807 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:20.807 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.907671+0000", 2024-09-21T21:43:20.808 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.003180511 2024-09-21T21:43:20.808 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.808 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.808 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:43:20.808 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.907767+0000", 2024-09-21T21:43:20.808 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6379000000000007e-05 2024-09-21T21:43:20.808 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.808 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.808 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:20.808 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.907969+0000", 2024-09-21T21:43:20.808 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00020152399999999999 2024-09-21T21:43:20.808 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:20.808 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:20.808 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:20.809 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.809 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.809 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:12335 3.1a 3:5b387735:::rbd_data.1121faebdfe3.0000000000000005:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:43:20.809 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:14.411635+0000", 2024-09-21T21:43:20.809 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 66.299015991000005, 2024-09-21T21:43:20.809 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.005973491, 2024-09-21T21:43:20.809 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:20.809 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:20.809 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:20.809 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:20.809 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:43:20.809 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:43:20.809 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 12335 2024-09-21T21:43:20.810 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.810 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:20.810 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.810 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:20.810 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411635+0000", 2024-09-21T21:43:20.810 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:20.814 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.814 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.814 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:20.814 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411635+0000", 2024-09-21T21:43:20.814 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:20.814 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.814 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.814 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:20.815 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411635+0000", 2024-09-21T21:43:20.815 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8500000000000001e-07 2024-09-21T21:43:20.815 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.815 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.815 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:20.815 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411639+0000", 2024-09-21T21:43:20.815 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.574e-06 2024-09-21T21:43:20.815 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.815 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.815 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:20.815 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411645+0000", 2024-09-21T21:43:20.815 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6869999999999997e-06 2024-09-21T21:43:20.815 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.815 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.816 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:20.816 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411649+0000", 2024-09-21T21:43:20.816 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.771e-06 2024-09-21T21:43:20.816 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.816 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.816 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:20.816 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411695+0000", 2024-09-21T21:43:20.816 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6e-05 2024-09-21T21:43:20.816 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.816 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.816 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op must wait for map", 2024-09-21T21:43:20.816 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411702+0000", 2024-09-21T21:43:20.816 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5590000000000002e-06 2024-09-21T21:43:20.816 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.816 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.817 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:20.817 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.417222+0000", 2024-09-21T21:43:20.817 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0055198219999999997 2024-09-21T21:43:20.817 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.817 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.817 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for peered", 2024-09-21T21:43:20.817 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.417230+0000", 2024-09-21T21:43:20.817 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.123e-06 2024-09-21T21:43:20.817 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.817 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.817 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:20.817 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.417348+0000", 2024-09-21T21:43:20.817 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.000118029 2024-09-21T21:43:20.817 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.818 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.818 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:20.818 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.417451+0000", 2024-09-21T21:43:20.818 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000103674 2024-09-21T21:43:20.818 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.818 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.818 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:20.818 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.417608+0000", 2024-09-21T21:43:20.818 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00015666700000000001 2024-09-21T21:43:20.818 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:20.818 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:20.818 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:20.818 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.818 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.818 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:12349 3.1a 3:5b387735:::rbd_data.1121faebdfe3.0000000000000005:head [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:43:20.819 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:14.411833+0000", 2024-09-21T21:43:20.819 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 66.298817177999993, 2024-09-21T21:43:20.819 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0065064500000001, 2024-09-21T21:43:20.819 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:20.819 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:20.819 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:20.819 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:20.819 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:43:20.819 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:43:20.819 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 12349 2024-09-21T21:43:20.819 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.819 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:20.819 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.819 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:20.819 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411833+0000", 2024-09-21T21:43:20.820 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:20.820 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.820 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.820 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:20.820 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411833+0000", 2024-09-21T21:43:20.820 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:20.820 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.820 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.820 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:20.820 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411834+0000", 2024-09-21T21:43:20.820 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.03e-07 2024-09-21T21:43:20.820 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.820 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.820 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:20.821 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411836+0000", 2024-09-21T21:43:20.821 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.4700000000000001e-06 2024-09-21T21:43:20.821 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.821 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.821 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:20.821 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411843+0000", 2024-09-21T21:43:20.821 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4069999999999998e-06 2024-09-21T21:43:20.821 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.821 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.821 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:20.821 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411848+0000", 2024-09-21T21:43:20.821 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1580000000000004e-06 2024-09-21T21:43:20.821 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.821 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.821 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:20.822 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411901+0000", 2024-09-21T21:43:20.822 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2824000000000001e-05 2024-09-21T21:43:20.822 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.822 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.822 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting_for_map not empty", 2024-09-21T21:43:20.822 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411904+0000", 2024-09-21T21:43:20.822 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0639999999999998e-06 2024-09-21T21:43:20.822 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.822 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.822 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:20.822 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.417252+0000", 2024-09-21T21:43:20.822 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0053485060000000003 2024-09-21T21:43:20.823 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.823 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.823 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for peered", 2024-09-21T21:43:20.823 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.417259+0000", 2024-09-21T21:43:20.823 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.1149999999999996e-06 2024-09-21T21:43:20.823 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.823 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.823 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:20.823 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.418160+0000", 2024-09-21T21:43:20.823 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.000901083 2024-09-21T21:43:20.823 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.823 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.823 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:20.823 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.418236+0000", 2024-09-21T21:43:20.823 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5852000000000005e-05 2024-09-21T21:43:20.824 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.824 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.824 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:20.824 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.418340+0000", 2024-09-21T21:43:20.824 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000103568 2024-09-21T21:43:20.824 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:20.824 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:20.824 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:20.824 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.824 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.824 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4385.0:30053 3.6 3:67cae2ed:::rbd_mirroring:head [list-watchers] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e141)", 2024-09-21T21:43:20.824 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:43:03.223636+0000", 2024-09-21T21:43:20.824 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 17.487014494, 2024-09-21T21:43:20.824 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.062115648, 2024-09-21T21:43:20.825 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:20.825 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:20.825 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:43:20.825 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:20.825 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4385", 2024-09-21T21:43:20.825 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:43:20.825 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 30053 2024-09-21T21:43:20.825 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.825 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:20.825 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.825 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:20.825 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:43:03.223636+0000", 2024-09-21T21:43:20.825 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:20.825 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.826 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.826 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:20.826 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:43:03.223636+0000", 2024-09-21T21:43:20.826 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:20.826 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.826 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.826 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:20.826 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:43:03.223638+0000", 2024-09-21T21:43:20.826 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.6080000000000001e-06 2024-09-21T21:43:20.826 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.826 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.826 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:20.826 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:43:03.223643+0000", 2024-09-21T21:43:20.826 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9110000000000001e-06 2024-09-21T21:43:20.826 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.827 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.827 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:20.827 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:43:03.223654+0000", 2024-09-21T21:43:20.827 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1029999999999999e-05 2024-09-21T21:43:20.827 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.827 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.827 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:20.827 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:43:03.223664+0000", 2024-09-21T21:43:20.827 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.9299999999999998e-06 2024-09-21T21:43:20.827 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.827 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.827 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:20.827 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:43:03.223856+0000", 2024-09-21T21:43:20.828 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00019216200000000001 2024-09-21T21:43:20.828 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.828 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.828 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for peered", 2024-09-21T21:43:20.828 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:43:03.223866+0000", 2024-09-21T21:43:20.828 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0356999999999999e-05 2024-09-21T21:43:20.828 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.828 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.828 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:20.828 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:43:04.285628+0000", 2024-09-21T21:43:20.828 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.061762211 2024-09-21T21:43:20.828 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:20.828 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:20.828 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:20.829 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:43:04.285752+0000", 2024-09-21T21:43:20.829 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00012343900000000001 2024-09-21T21:43:20.829 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:20.829 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:20.829 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:20.829 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:20.829 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:20.829 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:43:20.831 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T21:43:20.831 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:43:20.829+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:43:20.931 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:43:20.930+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:43:20.974 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:43:20.974 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:43:20.974 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 0 2024-09-21T21:43:20.974 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:43:20.982 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T21:43:21.032 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:43:21.030+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:43:21.085 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:43:21.301 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:43:21.302 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:43:21.302 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:43:21.302 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:43:21.304 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T21:43:21.305 INFO:teuthology.orchestra.run.smithi081.stderr:nodeep-scrub is unset 2024-09-21T21:43:21.306 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:43:21.302+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:43:21.405 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:43:21.403+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:43:21.452 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:43:21.453 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 20, 2024-09-21T21:43:21.454 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 600, 2024-09-21T21:43:21.454 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:43:21.454 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.454 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13296 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 4046848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:21.454 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.796643+0000", 2024-09-21T21:43:21.454 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 90.652887242000006, 2024-09-21T21:43:21.455 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071682019999999999, 2024-09-21T21:43:21.455 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:21.455 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:21.455 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:21.455 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:21.455 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:21.455 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:21.455 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13296 2024-09-21T21:43:21.455 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.456 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:21.456 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.456 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:21.457 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796643+0000", 2024-09-21T21:43:21.457 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:21.457 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.457 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.457 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:21.457 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796643+0000", 2024-09-21T21:43:21.457 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:21.458 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.458 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.458 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:21.458 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796643+0000", 2024-09-21T21:43:21.458 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.7199999999999998e-07 2024-09-21T21:43:21.458 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.459 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.459 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:21.459 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796647+0000", 2024-09-21T21:43:21.459 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6770000000000001e-06 2024-09-21T21:43:21.459 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.459 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.459 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:21.459 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796656+0000", 2024-09-21T21:43:21.460 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2059999999999996e-06 2024-09-21T21:43:21.460 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.460 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.460 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:21.460 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796664+0000", 2024-09-21T21:43:21.460 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7919999999999993e-06 2024-09-21T21:43:21.460 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.460 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.461 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:21.462 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.868186+0000", 2024-09-21T21:43:21.462 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071522273999999997 2024-09-21T21:43:21.462 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.462 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.462 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:21.462 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.868239+0000", 2024-09-21T21:43:21.462 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2787999999999997e-05 2024-09-21T21:43:21.463 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.463 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.463 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:21.463 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.868325+0000", 2024-09-21T21:43:21.463 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.5611000000000004e-05 2024-09-21T21:43:21.463 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:21.464 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:21.464 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:21.464 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.464 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.464 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13561 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 1728512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:21.464 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.823991+0000", 2024-09-21T21:43:21.464 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 90.625538941000002, 2024-09-21T21:43:21.465 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071654762999999996, 2024-09-21T21:43:21.465 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:21.465 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:21.465 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:21.465 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:21.465 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:21.465 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:21.466 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13561 2024-09-21T21:43:21.466 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.467 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:21.467 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.467 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:21.467 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.823991+0000", 2024-09-21T21:43:21.467 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:21.467 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.467 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.467 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:21.467 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.823991+0000", 2024-09-21T21:43:21.467 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:21.467 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.468 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.468 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:21.468 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.823991+0000", 2024-09-21T21:43:21.468 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9299999999999998e-07 2024-09-21T21:43:21.468 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.468 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.468 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:21.468 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.823994+0000", 2024-09-21T21:43:21.468 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2369999999999999e-06 2024-09-21T21:43:21.468 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.468 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.468 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:21.468 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824000+0000", 2024-09-21T21:43:21.468 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.55e-06 2024-09-21T21:43:21.468 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.469 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.469 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:21.469 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824005+0000", 2024-09-21T21:43:21.469 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4329999999999997e-06 2024-09-21T21:43:21.469 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.469 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.469 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:21.469 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.895360+0000", 2024-09-21T21:43:21.469 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071355426 2024-09-21T21:43:21.469 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.469 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.469 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:21.469 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.895477+0000", 2024-09-21T21:43:21.469 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0001171 2024-09-21T21:43:21.470 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.470 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.470 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:21.470 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.895646+0000", 2024-09-21T21:43:21.470 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000168524 2024-09-21T21:43:21.470 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:21.470 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:21.470 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:21.470 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.470 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.470 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13562 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 1757184~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:21.470 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.824011+0000", 2024-09-21T21:43:21.470 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 90.625519064000002, 2024-09-21T21:43:21.470 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072020453999999998, 2024-09-21T21:43:21.470 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:21.471 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:21.471 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:21.471 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:21.471 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:21.471 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:21.471 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13562 2024-09-21T21:43:21.471 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.471 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:21.471 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.471 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:21.471 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824011+0000", 2024-09-21T21:43:21.471 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:21.471 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.472 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.472 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:21.472 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824011+0000", 2024-09-21T21:43:21.472 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:21.472 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.472 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.472 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:21.473 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824011+0000", 2024-09-21T21:43:21.473 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5600000000000001e-07 2024-09-21T21:43:21.473 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.473 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.473 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:21.473 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824013+0000", 2024-09-21T21:43:21.473 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9800000000000001e-06 2024-09-21T21:43:21.473 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.473 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.473 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:21.473 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824019+0000", 2024-09-21T21:43:21.473 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7339999999999996e-06 2024-09-21T21:43:21.473 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.474 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.474 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:21.474 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824023+0000", 2024-09-21T21:43:21.474 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3680000000000004e-06 2024-09-21T21:43:21.474 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.474 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.474 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:21.474 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.895699+0000", 2024-09-21T21:43:21.474 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071675597999999993 2024-09-21T21:43:21.475 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.475 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.475 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:21.475 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.895830+0000", 2024-09-21T21:43:21.475 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000130858 2024-09-21T21:43:21.475 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.475 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.475 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:21.475 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896031+0000", 2024-09-21T21:43:21.475 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00020155999999999999 2024-09-21T21:43:21.475 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:21.476 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:21.476 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:21.476 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.476 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.476 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13563 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 323584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:21.476 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.824029+0000", 2024-09-21T21:43:21.476 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 90.625500623999997, 2024-09-21T21:43:21.476 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072307563000000005, 2024-09-21T21:43:21.476 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:21.476 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:21.476 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:21.476 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:21.477 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:21.477 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:21.477 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13563 2024-09-21T21:43:21.477 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.477 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:21.477 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.477 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:21.477 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824029+0000", 2024-09-21T21:43:21.478 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:21.478 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.478 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.478 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:21.478 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824029+0000", 2024-09-21T21:43:21.478 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:21.478 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.478 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.478 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:21.479 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824030+0000", 2024-09-21T21:43:21.479 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1699999999999999e-07 2024-09-21T21:43:21.479 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.479 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.479 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:21.479 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824032+0000", 2024-09-21T21:43:21.479 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0320000000000002e-06 2024-09-21T21:43:21.479 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.479 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.479 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:21.479 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824038+0000", 2024-09-21T21:43:21.479 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8769999999999998e-06 2024-09-21T21:43:21.479 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.480 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.480 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:21.480 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824042+0000", 2024-09-21T21:43:21.480 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5650000000000003e-06 2024-09-21T21:43:21.480 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.480 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.480 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:21.480 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896058+0000", 2024-09-21T21:43:21.480 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072015941 2024-09-21T21:43:21.480 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.480 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.480 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:21.480 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896164+0000", 2024-09-21T21:43:21.481 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010593499999999999 2024-09-21T21:43:21.481 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.481 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.481 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:21.481 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896337+0000", 2024-09-21T21:43:21.481 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00017279599999999999 2024-09-21T21:43:21.481 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:21.481 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:21.481 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:21.481 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.481 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.482 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13564 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 983040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:21.482 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.824048+0000", 2024-09-21T21:43:21.482 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 90.625481549, 2024-09-21T21:43:21.482 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072491193999999995, 2024-09-21T21:43:21.482 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:21.482 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:21.482 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:21.482 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:21.482 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:21.482 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:21.483 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13564 2024-09-21T21:43:21.483 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.483 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:21.483 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.483 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:21.483 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824048+0000", 2024-09-21T21:43:21.483 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:21.483 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.483 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.483 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:21.483 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824048+0000", 2024-09-21T21:43:21.483 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:21.484 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.484 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.484 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:21.484 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824663+0000", 2024-09-21T21:43:21.484 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00061494299999999996 2024-09-21T21:43:21.484 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.484 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.484 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:21.484 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824669+0000", 2024-09-21T21:43:21.484 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2900000000000002e-06 2024-09-21T21:43:21.484 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.484 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.484 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:21.484 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824679+0000", 2024-09-21T21:43:21.485 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0338e-05 2024-09-21T21:43:21.485 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.485 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.485 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:21.485 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824687+0000", 2024-09-21T21:43:21.485 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.9049999999999997e-06 2024-09-21T21:43:21.485 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.485 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.485 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:21.485 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896381+0000", 2024-09-21T21:43:21.485 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071694305999999999 2024-09-21T21:43:21.485 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.485 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.485 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:21.486 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896445+0000", 2024-09-21T21:43:21.486 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3869999999999997e-05 2024-09-21T21:43:21.486 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.486 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.486 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:21.486 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896540+0000", 2024-09-21T21:43:21.486 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.4542000000000004e-05 2024-09-21T21:43:21.486 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:21.491 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:21.491 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:21.491 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.491 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.491 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13565 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3190784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:21.491 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.824704+0000", 2024-09-21T21:43:21.491 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 90.624825677999993, 2024-09-21T21:43:21.491 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072014745000000005, 2024-09-21T21:43:21.492 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:21.492 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:21.492 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:21.492 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:21.492 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:21.492 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:21.492 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13565 2024-09-21T21:43:21.492 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.492 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:21.493 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.493 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:21.493 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824704+0000", 2024-09-21T21:43:21.493 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:21.493 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.494 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.494 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:21.494 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824704+0000", 2024-09-21T21:43:21.494 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:21.494 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.494 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.494 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:21.495 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824705+0000", 2024-09-21T21:43:21.495 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8999999999999997e-07 2024-09-21T21:43:21.495 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.495 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.495 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:21.495 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824707+0000", 2024-09-21T21:43:21.495 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.7669999999999999e-06 2024-09-21T21:43:21.495 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.495 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.495 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:21.495 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824715+0000", 2024-09-21T21:43:21.495 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7700000000000001e-06 2024-09-21T21:43:21.495 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.496 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.496 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:21.496 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824721+0000", 2024-09-21T21:43:21.496 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9560000000000002e-06 2024-09-21T21:43:21.496 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.496 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.496 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:21.496 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896570+0000", 2024-09-21T21:43:21.496 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071848870999999995 2024-09-21T21:43:21.496 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.496 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.496 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:21.496 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896626+0000", 2024-09-21T21:43:21.496 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5578000000000002e-05 2024-09-21T21:43:21.497 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.497 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.497 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:21.497 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896719+0000", 2024-09-21T21:43:21.497 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.3313000000000006e-05 2024-09-21T21:43:21.497 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:21.497 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:21.497 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:21.497 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.497 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.497 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13566 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3162112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:21.497 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.824729+0000", 2024-09-21T21:43:21.497 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 90.624800790999998, 2024-09-21T21:43:21.497 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072356907999999998, 2024-09-21T21:43:21.497 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:21.498 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:21.498 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:21.498 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:21.498 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:21.498 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:21.498 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13566 2024-09-21T21:43:21.498 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.498 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:21.498 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.498 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:21.498 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824729+0000", 2024-09-21T21:43:21.498 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:21.498 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.499 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.499 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:21.499 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824729+0000", 2024-09-21T21:43:21.499 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:21.499 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.499 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.499 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:21.499 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825630+0000", 2024-09-21T21:43:21.499 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00090139400000000002 2024-09-21T21:43:21.499 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.499 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.499 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:21.499 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825634+0000", 2024-09-21T21:43:21.499 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4960000000000001e-06 2024-09-21T21:43:21.500 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.500 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.500 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:21.500 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825644+0000", 2024-09-21T21:43:21.500 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.9650000000000003e-06 2024-09-21T21:43:21.500 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.500 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.500 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:21.500 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825649+0000", 2024-09-21T21:43:21.500 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0359999999999997e-06 2024-09-21T21:43:21.500 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.500 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.500 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:21.500 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896757+0000", 2024-09-21T21:43:21.501 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071107717000000001 2024-09-21T21:43:21.501 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.501 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.501 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:21.501 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896878+0000", 2024-09-21T21:43:21.501 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000120834 2024-09-21T21:43:21.501 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.501 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.501 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:21.501 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897086+0000", 2024-09-21T21:43:21.501 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00020846600000000001 2024-09-21T21:43:21.501 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:21.501 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:21.501 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:21.502 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.502 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.502 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13568 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3907584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:21.502 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.825671+0000", 2024-09-21T21:43:21.502 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 90.623858381999995, 2024-09-21T21:43:21.502 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071794595000000003, 2024-09-21T21:43:21.502 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:21.502 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:21.502 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:21.502 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:21.502 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:21.502 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:21.502 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13568 2024-09-21T21:43:21.502 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.503 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:21.503 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.503 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:21.503 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825671+0000", 2024-09-21T21:43:21.503 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:21.503 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.503 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.503 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:21.503 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825671+0000", 2024-09-21T21:43:21.503 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:21.503 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.503 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.503 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:21.503 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825672+0000", 2024-09-21T21:43:21.504 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2999999999999999e-07 2024-09-21T21:43:21.504 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.504 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.504 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:21.504 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825673+0000", 2024-09-21T21:43:21.504 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.246e-06 2024-09-21T21:43:21.504 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.504 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.504 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:21.504 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825693+0000", 2024-09-21T21:43:21.504 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9789999999999999e-05 2024-09-21T21:43:21.504 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.504 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.504 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:21.505 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825697+0000", 2024-09-21T21:43:21.505 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2379999999999999e-06 2024-09-21T21:43:21.505 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.505 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.505 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:21.505 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897284+0000", 2024-09-21T21:43:21.505 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071586710999999997 2024-09-21T21:43:21.505 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.505 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.505 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:21.505 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897337+0000", 2024-09-21T21:43:21.505 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2800000000000003e-05 2024-09-21T21:43:21.505 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.505 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.505 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:21.506 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897466+0000", 2024-09-21T21:43:21.506 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00012957999999999999 2024-09-21T21:43:21.506 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:21.506 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:21.506 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:21.506 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.506 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.506 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13574 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 352256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:21.506 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.825702+0000", 2024-09-21T21:43:21.506 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 90.623828234000001, 2024-09-21T21:43:21.506 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072080377000000001, 2024-09-21T21:43:21.506 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:21.506 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:21.507 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:21.507 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:21.507 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:21.507 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:21.507 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13574 2024-09-21T21:43:21.507 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.507 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:21.507 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.507 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:21.507 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825702+0000", 2024-09-21T21:43:21.507 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:21.507 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.507 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.507 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:21.508 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825702+0000", 2024-09-21T21:43:21.508 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:21.508 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.508 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.508 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:21.508 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825943+0000", 2024-09-21T21:43:21.508 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00024105200000000001 2024-09-21T21:43:21.508 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.508 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.508 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:21.508 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825946+0000", 2024-09-21T21:43:21.508 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.743e-06 2024-09-21T21:43:21.508 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.509 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.509 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:21.509 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825956+0000", 2024-09-21T21:43:21.509 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6609999999999995e-06 2024-09-21T21:43:21.509 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.509 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.509 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:21.509 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825963+0000", 2024-09-21T21:43:21.509 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4609999999999996e-06 2024-09-21T21:43:21.509 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.509 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.509 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:21.509 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897502+0000", 2024-09-21T21:43:21.509 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071539354999999999 2024-09-21T21:43:21.510 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.510 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.510 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:21.510 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897619+0000", 2024-09-21T21:43:21.510 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011665 2024-09-21T21:43:21.510 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.510 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.510 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:21.510 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897782+0000", 2024-09-21T21:43:21.510 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00016345500000000001 2024-09-21T21:43:21.510 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:21.510 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:21.510 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:21.511 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.511 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.511 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13575 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 380928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:21.511 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.825973+0000", 2024-09-21T21:43:21.511 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 90.623556797999996, 2024-09-21T21:43:21.511 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072012723000000001, 2024-09-21T21:43:21.511 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:21.511 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:21.511 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:21.511 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:21.511 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:21.511 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:21.511 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13575 2024-09-21T21:43:21.511 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.511 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:21.512 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.512 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:21.512 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825973+0000", 2024-09-21T21:43:21.512 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:21.512 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.512 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.512 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:21.512 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825973+0000", 2024-09-21T21:43:21.512 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:21.512 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.512 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.512 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:21.512 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826746+0000", 2024-09-21T21:43:21.512 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00077326800000000002 2024-09-21T21:43:21.513 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.513 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.513 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:21.513 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826766+0000", 2024-09-21T21:43:21.513 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9465000000000001e-05 2024-09-21T21:43:21.513 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.513 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.513 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:21.513 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826812+0000", 2024-09-21T21:43:21.513 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5927999999999998e-05 2024-09-21T21:43:21.513 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.513 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.513 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:21.513 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826833+0000", 2024-09-21T21:43:21.514 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1710999999999999e-05 2024-09-21T21:43:21.514 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.514 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.514 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:21.514 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897809+0000", 2024-09-21T21:43:21.514 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.070975229000000001 2024-09-21T21:43:21.514 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.514 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.514 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:21.514 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897864+0000", 2024-09-21T21:43:21.514 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5334999999999998e-05 2024-09-21T21:43:21.514 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.514 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.514 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:21.515 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897986+0000", 2024-09-21T21:43:21.515 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000121787 2024-09-21T21:43:21.515 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:21.515 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:21.515 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:21.515 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.515 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.515 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13577 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 1814528~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:21.515 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.826874+0000", 2024-09-21T21:43:21.515 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 90.622655764000001, 2024-09-21T21:43:21.515 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071639123999999998, 2024-09-21T21:43:21.515 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:21.515 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:21.515 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:21.516 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:21.516 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:21.516 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:21.516 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13577 2024-09-21T21:43:21.516 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.516 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:21.516 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.516 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:21.516 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826874+0000", 2024-09-21T21:43:21.516 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:21.516 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.516 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.516 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:21.516 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826874+0000", 2024-09-21T21:43:21.517 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:21.517 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.517 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.517 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:21.517 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826875+0000", 2024-09-21T21:43:21.517 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.82e-07 2024-09-21T21:43:21.517 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.517 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.517 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:21.517 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826877+0000", 2024-09-21T21:43:21.517 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.295e-06 2024-09-21T21:43:21.517 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.517 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.517 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:21.517 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826883+0000", 2024-09-21T21:43:21.518 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2609999999999999e-06 2024-09-21T21:43:21.518 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.518 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.518 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:21.518 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826889+0000", 2024-09-21T21:43:21.518 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5589999999999996e-06 2024-09-21T21:43:21.518 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.518 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.518 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:21.518 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898301+0000", 2024-09-21T21:43:21.518 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071412188000000001 2024-09-21T21:43:21.518 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.518 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.518 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:21.519 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898416+0000", 2024-09-21T21:43:21.519 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000115538 2024-09-21T21:43:21.519 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.519 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.519 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:21.519 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898513+0000", 2024-09-21T21:43:21.519 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6800999999999997e-05 2024-09-21T21:43:21.519 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:21.519 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:21.519 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:21.519 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.519 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.519 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13578 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 2502656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:21.519 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.826896+0000", 2024-09-21T21:43:21.520 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 90.622634308000002, 2024-09-21T21:43:21.520 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071789411999999997, 2024-09-21T21:43:21.520 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:21.520 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:21.520 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:21.520 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:21.520 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:21.520 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:21.520 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13578 2024-09-21T21:43:21.520 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.520 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:21.520 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.520 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:21.520 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826896+0000", 2024-09-21T21:43:21.521 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:21.521 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.521 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.521 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:21.521 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826896+0000", 2024-09-21T21:43:21.521 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:21.521 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.521 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.521 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:21.521 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827327+0000", 2024-09-21T21:43:21.521 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00043094500000000002 2024-09-21T21:43:21.521 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.521 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.522 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:21.522 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827331+0000", 2024-09-21T21:43:21.522 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.0969999999999999e-06 2024-09-21T21:43:21.522 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.522 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.522 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:21.522 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827341+0000", 2024-09-21T21:43:21.522 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0199e-05 2024-09-21T21:43:21.522 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.522 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.522 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:21.522 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827346+0000", 2024-09-21T21:43:21.522 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3210000000000004e-06 2024-09-21T21:43:21.523 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.523 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.523 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:21.523 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898540+0000", 2024-09-21T21:43:21.523 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071194246000000003 2024-09-21T21:43:21.523 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.523 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.523 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:21.523 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898598+0000", 2024-09-21T21:43:21.523 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7658999999999998e-05 2024-09-21T21:43:21.523 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.523 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.523 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:21.523 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898685+0000", 2024-09-21T21:43:21.523 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.6945000000000002e-05 2024-09-21T21:43:21.524 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:21.524 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:21.524 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:21.524 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.524 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.524 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13579 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 2531328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:21.524 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.827355+0000", 2024-09-21T21:43:21.524 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 90.622174776999998, 2024-09-21T21:43:21.524 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071650933, 2024-09-21T21:43:21.524 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:21.524 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:21.524 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:21.524 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:21.524 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:21.525 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:21.525 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13579 2024-09-21T21:43:21.525 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.525 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:21.525 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.525 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:21.525 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827355+0000", 2024-09-21T21:43:21.525 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:21.525 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.525 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.525 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:21.525 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827355+0000", 2024-09-21T21:43:21.526 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:21.526 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.526 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.526 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:21.526 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827486+0000", 2024-09-21T21:43:21.526 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00013066799999999999 2024-09-21T21:43:21.526 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.526 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.526 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:21.526 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827490+0000", 2024-09-21T21:43:21.526 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.9009999999999996e-06 2024-09-21T21:43:21.526 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.527 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.527 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:21.527 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827499+0000", 2024-09-21T21:43:21.527 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6209999999999999e-06 2024-09-21T21:43:21.527 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.527 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.527 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:21.527 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827506+0000", 2024-09-21T21:43:21.527 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6490000000000002e-06 2024-09-21T21:43:21.527 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.527 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.527 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:21.527 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898713+0000", 2024-09-21T21:43:21.527 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071206937999999997 2024-09-21T21:43:21.528 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.528 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.528 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:21.528 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898811+0000", 2024-09-21T21:43:21.528 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.8159999999999995e-05 2024-09-21T21:43:21.528 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.528 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.528 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:21.528 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899006+0000", 2024-09-21T21:43:21.528 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00019499599999999999 2024-09-21T21:43:21.528 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:21.528 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:21.528 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:21.528 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.529 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.529 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13580 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3219456~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:21.529 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.827522+0000", 2024-09-21T21:43:21.529 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 90.622008346000001, 2024-09-21T21:43:21.529 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071813298999999997, 2024-09-21T21:43:21.529 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:21.529 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:21.529 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:21.529 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:21.529 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:21.529 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:21.529 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13580 2024-09-21T21:43:21.529 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.530 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:21.530 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.530 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:21.530 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827522+0000", 2024-09-21T21:43:21.530 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:21.530 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.530 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.530 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:21.530 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827522+0000", 2024-09-21T21:43:21.530 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:21.530 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.530 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.530 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:21.530 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827898+0000", 2024-09-21T21:43:21.531 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000376876 2024-09-21T21:43:21.531 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.531 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.531 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:21.531 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827903+0000", 2024-09-21T21:43:21.531 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4129999999999999e-06 2024-09-21T21:43:21.531 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.531 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.531 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:21.531 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827911+0000", 2024-09-21T21:43:21.531 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.4540000000000007e-06 2024-09-21T21:43:21.531 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.531 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.531 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:21.532 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827917+0000", 2024-09-21T21:43:21.532 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2110000000000003e-06 2024-09-21T21:43:21.532 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.532 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.532 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:21.532 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899041+0000", 2024-09-21T21:43:21.532 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071123965999999997 2024-09-21T21:43:21.532 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.532 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.532 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:21.532 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899157+0000", 2024-09-21T21:43:21.532 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011573700000000001 2024-09-21T21:43:21.532 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.532 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.533 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:21.533 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899335+0000", 2024-09-21T21:43:21.533 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00017764199999999999 2024-09-21T21:43:21.533 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:21.533 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:21.533 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:21.533 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.533 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.533 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13581 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3248128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:21.533 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.827929+0000", 2024-09-21T21:43:21.533 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 90.621600947000005, 2024-09-21T21:43:21.533 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071730151000000006, 2024-09-21T21:43:21.533 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:21.533 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:21.534 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:21.534 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:21.534 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:21.534 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:21.534 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13581 2024-09-21T21:43:21.534 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.534 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:21.534 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.534 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:21.534 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827929+0000", 2024-09-21T21:43:21.534 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:21.534 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.534 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.534 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:21.535 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827929+0000", 2024-09-21T21:43:21.535 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:21.535 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.535 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.535 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:21.535 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.829943+0000", 2024-09-21T21:43:21.535 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0020143750000000001 2024-09-21T21:43:21.535 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.535 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.535 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:21.535 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.829948+0000", 2024-09-21T21:43:21.535 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9400000000000001e-06 2024-09-21T21:43:21.535 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.535 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.536 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:21.536 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.829960+0000", 2024-09-21T21:43:21.536 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.2123999999999999e-05 2024-09-21T21:43:21.536 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.536 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.536 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:21.536 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.829968+0000", 2024-09-21T21:43:21.536 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7000000000000008e-06 2024-09-21T21:43:21.536 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.536 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.536 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:21.536 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899370+0000", 2024-09-21T21:43:21.536 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.069401672999999997 2024-09-21T21:43:21.536 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.537 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.537 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:21.537 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899481+0000", 2024-09-21T21:43:21.537 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011145300000000001 2024-09-21T21:43:21.537 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.537 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.537 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:21.537 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899659+0000", 2024-09-21T21:43:21.537 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00017788600000000001 2024-09-21T21:43:21.537 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:21.537 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:21.537 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:21.537 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.537 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.538 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13595 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 1871872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:21.538 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.830180+0000", 2024-09-21T21:43:21.538 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 90.619349833000001, 2024-09-21T21:43:21.538 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071809244999999994, 2024-09-21T21:43:21.538 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:21.538 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:21.538 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:21.538 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:21.538 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:21.538 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:21.538 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13595 2024-09-21T21:43:21.538 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.538 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:21.538 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.539 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:21.539 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.830180+0000", 2024-09-21T21:43:21.539 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:21.539 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.539 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.539 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:21.539 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.830180+0000", 2024-09-21T21:43:21.539 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:21.539 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.539 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.539 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:21.539 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.831967+0000", 2024-09-21T21:43:21.539 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0017868350000000001 2024-09-21T21:43:21.539 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.540 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.540 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:21.540 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.831971+0000", 2024-09-21T21:43:21.540 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3950000000000003e-06 2024-09-21T21:43:21.540 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.540 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.540 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:21.540 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.831983+0000", 2024-09-21T21:43:21.540 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1815e-05 2024-09-21T21:43:21.540 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.540 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.540 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:21.540 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.831993+0000", 2024-09-21T21:43:21.540 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6339999999999996e-06 2024-09-21T21:43:21.541 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.541 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.541 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:21.541 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.901693+0000", 2024-09-21T21:43:21.541 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.069700082999999996 2024-09-21T21:43:21.541 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.541 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.541 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:21.541 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.901813+0000", 2024-09-21T21:43:21.541 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011971 2024-09-21T21:43:21.541 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.541 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.541 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:21.541 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.901989+0000", 2024-09-21T21:43:21.542 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000176773 2024-09-21T21:43:21.542 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:21.542 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:21.542 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:21.542 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.542 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.542 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13601 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3334144~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:21.542 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.832130+0000", 2024-09-21T21:43:21.542 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 90.617399961000004, 2024-09-21T21:43:21.542 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071861727, 2024-09-21T21:43:21.542 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:21.542 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:21.542 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:21.542 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:21.543 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:21.543 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:21.543 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13601 2024-09-21T21:43:21.543 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.543 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:21.543 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.543 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:21.543 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.832130+0000", 2024-09-21T21:43:21.543 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:21.543 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.543 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.543 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:21.543 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.832130+0000", 2024-09-21T21:43:21.543 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:21.544 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.544 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.544 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:21.544 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.833939+0000", 2024-09-21T21:43:21.544 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0018094400000000001 2024-09-21T21:43:21.544 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.544 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.544 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:21.544 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.833944+0000", 2024-09-21T21:43:21.544 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5979999999999999e-06 2024-09-21T21:43:21.544 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.544 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.544 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:21.544 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.833953+0000", 2024-09-21T21:43:21.544 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.1460000000000002e-06 2024-09-21T21:43:21.545 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.545 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.545 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:21.545 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.833960+0000", 2024-09-21T21:43:21.545 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.7989999999999996e-06 2024-09-21T21:43:21.545 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.545 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.545 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:21.545 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.903693+0000", 2024-09-21T21:43:21.545 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.069732901 2024-09-21T21:43:21.545 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.545 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.545 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:21.545 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.903814+0000", 2024-09-21T21:43:21.545 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000121585 2024-09-21T21:43:21.546 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.546 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.546 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:21.546 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.903992+0000", 2024-09-21T21:43:21.546 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00017725799999999999 2024-09-21T21:43:21.546 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:21.546 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:21.546 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:21.546 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.546 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.546 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:12342 3.1 3:87417293:::rbd_data.1121faebdfe3.0000000000000004:head [read 4096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:43:21.546 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:14.411725+0000", 2024-09-21T21:43:21.546 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 67.037804894999994, 2024-09-21T21:43:21.546 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0089213029999999, 2024-09-21T21:43:21.547 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:21.547 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:21.547 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:21.547 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:21.547 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:43:21.547 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:43:21.547 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 12342 2024-09-21T21:43:21.547 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.547 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:21.547 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.547 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:21.548 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411725+0000", 2024-09-21T21:43:21.548 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:21.548 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.548 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.548 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:21.548 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411725+0000", 2024-09-21T21:43:21.548 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:21.548 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.548 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.548 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:21.548 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411726+0000", 2024-09-21T21:43:21.548 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.4169999999999999e-06 2024-09-21T21:43:21.548 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.548 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.549 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:21.549 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411734+0000", 2024-09-21T21:43:21.549 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.3799999999999996e-06 2024-09-21T21:43:21.549 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.549 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.549 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:21.549 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411750+0000", 2024-09-21T21:43:21.549 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.6429000000000001e-05 2024-09-21T21:43:21.549 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.549 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.549 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:21.549 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411760+0000", 2024-09-21T21:43:21.549 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.4239999999999999e-06 2024-09-21T21:43:21.549 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.549 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.550 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:21.550 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411828+0000", 2024-09-21T21:43:21.550 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.7935999999999995e-05 2024-09-21T21:43:21.550 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.550 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.550 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op must wait for map", 2024-09-21T21:43:21.550 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411837+0000", 2024-09-21T21:43:21.550 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.0100000000000001e-06 2024-09-21T21:43:21.550 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.550 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.550 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:21.550 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.414984+0000", 2024-09-21T21:43:21.550 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0031472050000000001 2024-09-21T21:43:21.550 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.551 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.551 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for peered", 2024-09-21T21:43:21.551 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.414994+0000", 2024-09-21T21:43:21.551 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0291e-05 2024-09-21T21:43:21.551 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.551 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.551 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:21.551 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.420353+0000", 2024-09-21T21:43:21.551 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.005358913 2024-09-21T21:43:21.551 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.551 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.551 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:21.551 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.420489+0000", 2024-09-21T21:43:21.551 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00013592699999999999 2024-09-21T21:43:21.551 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.552 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.552 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:21.552 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.420646+0000", 2024-09-21T21:43:21.552 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00015737100000000001 2024-09-21T21:43:21.552 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:21.552 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:21.552 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:21.552 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.552 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.552 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:12348 3.1 3:87417293:::rbd_data.1121faebdfe3.0000000000000004:head [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:43:21.552 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:14.411793+0000", 2024-09-21T21:43:21.552 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 67.037737348999997, 2024-09-21T21:43:21.552 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.009298384, 2024-09-21T21:43:21.552 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:21.553 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:21.553 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:21.553 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:21.553 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:43:21.553 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:43:21.553 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 12348 2024-09-21T21:43:21.553 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.553 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:21.553 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.553 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:21.553 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411793+0000", 2024-09-21T21:43:21.553 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:21.553 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.553 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.553 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:21.554 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411793+0000", 2024-09-21T21:43:21.554 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:21.554 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.554 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.554 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:21.554 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411793+0000", 2024-09-21T21:43:21.554 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0999999999999998e-07 2024-09-21T21:43:21.554 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.554 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.554 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:21.554 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411797+0000", 2024-09-21T21:43:21.554 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.49e-06 2024-09-21T21:43:21.554 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.554 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.555 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:21.555 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411809+0000", 2024-09-21T21:43:21.555 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.2510999999999999e-05 2024-09-21T21:43:21.555 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.555 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.555 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:21.555 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411816+0000", 2024-09-21T21:43:21.555 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6420000000000001e-06 2024-09-21T21:43:21.555 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.555 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.555 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:21.555 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411963+0000", 2024-09-21T21:43:21.555 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00014700999999999999 2024-09-21T21:43:21.556 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.556 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.556 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting_for_map not empty", 2024-09-21T21:43:21.556 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411970+0000", 2024-09-21T21:43:21.556 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9800000000000001e-06 2024-09-21T21:43:21.556 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.556 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.556 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:21.556 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.415018+0000", 2024-09-21T21:43:21.556 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0030482299999999999 2024-09-21T21:43:21.556 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.556 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.556 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for peered", 2024-09-21T21:43:21.556 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.415025+0000", 2024-09-21T21:43:21.556 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.3749999999999997e-06 2024-09-21T21:43:21.557 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.557 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.557 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:21.557 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.420904+0000", 2024-09-21T21:43:21.557 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0058787339999999 2024-09-21T21:43:21.557 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.557 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.557 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:21.557 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.420988+0000", 2024-09-21T21:43:21.557 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.3955999999999994e-05 2024-09-21T21:43:21.557 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.557 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.557 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:21.557 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.421091+0000", 2024-09-21T21:43:21.558 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000102846 2024-09-21T21:43:21.558 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:21.558 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:21.558 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:21.558 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.558 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.558 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4385.0:20922 3.9 3:920c5256:::rbd_id.image_client.0-clone36:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-21T21:43:21.558 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:17.811324+0000", 2024-09-21T21:43:21.558 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 63.638205616999997, 2024-09-21T21:43:21.558 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.66951882299999999, 2024-09-21T21:43:21.558 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:21.558 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:21.558 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:21.558 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:21.558 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4385", 2024-09-21T21:43:21.559 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:43:21.559 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 20922 2024-09-21T21:43:21.559 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.559 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:21.559 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.559 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:21.559 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:17.811324+0000", 2024-09-21T21:43:21.559 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:21.559 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.559 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.559 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:21.559 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:17.811324+0000", 2024-09-21T21:43:21.559 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:21.559 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.560 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.560 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:21.560 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:17.811326+0000", 2024-09-21T21:43:21.560 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.536e-06 2024-09-21T21:43:21.560 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.560 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.560 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:21.560 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:17.811332+0000", 2024-09-21T21:43:21.560 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6699999999999997e-06 2024-09-21T21:43:21.560 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.560 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.560 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:21.560 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:17.811347+0000", 2024-09-21T21:43:21.560 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.4589999999999999e-05 2024-09-21T21:43:21.561 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.561 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.561 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:21.561 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:17.811359+0000", 2024-09-21T21:43:21.561 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1776e-05 2024-09-21T21:43:21.561 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.561 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.561 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:21.561 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:17.811424+0000", 2024-09-21T21:43:21.561 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4795000000000006e-05 2024-09-21T21:43:21.561 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.561 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.561 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for peered", 2024-09-21T21:43:21.561 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:17.811436+0000", 2024-09-21T21:43:21.561 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.221e-05 2024-09-21T21:43:21.562 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.562 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.562 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:21.562 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:18.480657+0000", 2024-09-21T21:43:21.562 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.66922088999999996 2024-09-21T21:43:21.562 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.562 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.562 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:21.562 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:18.480760+0000", 2024-09-21T21:43:21.562 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000102827 2024-09-21T21:43:21.562 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:21.562 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:21.562 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:21.562 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:18.480843+0000", 2024-09-21T21:43:21.563 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.3529000000000006e-05 2024-09-21T21:43:21.563 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:21.563 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:21.563 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:21.563 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:21.563 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:21.563 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:43:21.564 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T21:43:21.565 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:43:21.563+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:43:21.665 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:43:21.664+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:43:21.667 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:43:21.667 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:43:21.667 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-21T21:43:21.667 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:43:21.676 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T21:43:21.766 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:43:21.765+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:43:21.778 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:43:21.778 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:43:21.778 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:43:21.778 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:43:21.778 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:43:21.787 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T21:43:21.867 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:43:21.866+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:43:21.892 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:43:21.892 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-21T21:43:21.892 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-21T21:43:21.892 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:43:21.893 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.893 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19747 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 7905280~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:21.893 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273058+0000", 2024-09-21T21:43:21.893 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 90.617027109000006, 2024-09-21T21:43:21.893 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97338591399999996, 2024-09-21T21:43:21.893 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:21.893 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:21.893 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:21.893 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:21.893 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:21.893 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:21.893 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19747 2024-09-21T21:43:21.893 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.894 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:21.894 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.894 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:21.894 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273058+0000", 2024-09-21T21:43:21.894 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:21.894 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.894 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.894 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:21.894 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273058+0000", 2024-09-21T21:43:21.894 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:21.894 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.894 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.894 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:21.894 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273058+0000", 2024-09-21T21:43:21.895 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5900000000000003e-07 2024-09-21T21:43:21.895 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.895 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.895 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:21.895 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273061+0000", 2024-09-21T21:43:21.895 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3479999999999998e-06 2024-09-21T21:43:21.895 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.895 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.895 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:21.895 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273067+0000", 2024-09-21T21:43:21.895 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1469999999999998e-06 2024-09-21T21:43:21.895 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.895 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.895 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:21.896 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273071+0000", 2024-09-21T21:43:21.896 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7339999999999999e-06 2024-09-21T21:43:21.896 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.896 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.896 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:21.896 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277945+0000", 2024-09-21T21:43:21.896 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.004873474 2024-09-21T21:43:21.896 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.896 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.896 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:21.896 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277948+0000", 2024-09-21T21:43:21.896 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0299999999999998e-06 2024-09-21T21:43:21.896 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.896 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.897 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:21.897 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246269+0000", 2024-09-21T21:43:21.897 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96832060200000003 2024-09-21T21:43:21.897 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.897 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.897 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:21.897 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246362+0000", 2024-09-21T21:43:21.897 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3867000000000001e-05 2024-09-21T21:43:21.897 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.897 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.897 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:21.897 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246444+0000", 2024-09-21T21:43:21.897 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1353000000000005e-05 2024-09-21T21:43:21.897 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:21.898 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:21.898 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:21.898 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.898 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.898 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19755 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 5783552~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:21.898 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273077+0000", 2024-09-21T21:43:21.898 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 90.617007493000003, 2024-09-21T21:43:21.898 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97350203400000002, 2024-09-21T21:43:21.898 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:21.898 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:21.898 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:21.898 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:21.898 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:21.898 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:21.898 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19755 2024-09-21T21:43:21.899 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.899 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:21.899 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.899 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:21.899 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273077+0000", 2024-09-21T21:43:21.899 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:21.899 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.899 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.899 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:21.899 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273077+0000", 2024-09-21T21:43:21.899 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:21.899 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.899 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.899 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:21.899 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273078+0000", 2024-09-21T21:43:21.900 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5699999999999998e-07 2024-09-21T21:43:21.900 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.900 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.900 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:21.900 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273083+0000", 2024-09-21T21:43:21.900 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8670000000000002e-06 2024-09-21T21:43:21.900 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.900 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.900 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:21.900 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273088+0000", 2024-09-21T21:43:21.900 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8370000000000002e-06 2024-09-21T21:43:21.900 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.900 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.900 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:21.900 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273093+0000", 2024-09-21T21:43:21.901 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5709999999999999e-06 2024-09-21T21:43:21.901 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.901 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.901 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:21.901 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277968+0000", 2024-09-21T21:43:21.901 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048753909999999998 2024-09-21T21:43:21.901 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.901 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.901 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:21.901 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277971+0000", 2024-09-21T21:43:21.901 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0189999999999998e-06 2024-09-21T21:43:21.901 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.901 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.901 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:21.902 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246485+0000", 2024-09-21T21:43:21.902 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96851399299999996 2024-09-21T21:43:21.902 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.902 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.902 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:21.902 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246536+0000", 2024-09-21T21:43:21.902 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0430000000000003e-05 2024-09-21T21:43:21.902 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.902 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.902 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:21.902 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246579+0000", 2024-09-21T21:43:21.902 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3569000000000002e-05 2024-09-21T21:43:21.902 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:21.902 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:21.903 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:21.903 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.903 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.903 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19756 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6500352~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:21.903 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273099+0000", 2024-09-21T21:43:21.903 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 90.616986120000007, 2024-09-21T21:43:21.903 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97359272799999996, 2024-09-21T21:43:21.903 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:21.903 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:21.903 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:21.903 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:21.903 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:21.903 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:21.903 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19756 2024-09-21T21:43:21.903 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.904 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:21.904 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.904 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:21.904 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273099+0000", 2024-09-21T21:43:21.904 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:21.904 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.904 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.904 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:21.904 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273099+0000", 2024-09-21T21:43:21.904 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:21.904 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.904 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.904 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:21.904 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273099+0000", 2024-09-21T21:43:21.905 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0900000000000002e-07 2024-09-21T21:43:21.905 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.905 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.905 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:21.905 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273101+0000", 2024-09-21T21:43:21.905 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0320000000000002e-06 2024-09-21T21:43:21.905 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.905 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.905 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:21.905 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273107+0000", 2024-09-21T21:43:21.905 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2830000000000001e-06 2024-09-21T21:43:21.905 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.905 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.905 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:21.905 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273111+0000", 2024-09-21T21:43:21.906 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2230000000000001e-06 2024-09-21T21:43:21.906 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.906 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.906 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:21.906 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277994+0000", 2024-09-21T21:43:21.906 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048837070000000002 2024-09-21T21:43:21.906 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.906 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.906 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:21.906 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278000+0000", 2024-09-21T21:43:21.906 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3149999999999999e-06 2024-09-21T21:43:21.906 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.906 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.906 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:21.907 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246607+0000", 2024-09-21T21:43:21.907 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.968606839 2024-09-21T21:43:21.907 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.907 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.907 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:21.907 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246651+0000", 2024-09-21T21:43:21.907 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4734999999999998e-05 2024-09-21T21:43:21.907 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.907 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.907 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:21.907 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246692+0000", 2024-09-21T21:43:21.907 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0185000000000002e-05 2024-09-21T21:43:21.907 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:21.907 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:21.908 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:21.908 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.908 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.908 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19757 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 5812224~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:21.908 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273116+0000", 2024-09-21T21:43:21.908 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 90.616968463000006, 2024-09-21T21:43:21.908 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97367813999999997, 2024-09-21T21:43:21.908 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:21.908 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:21.908 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:21.908 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:21.908 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:21.908 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:21.908 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19757 2024-09-21T21:43:21.909 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.909 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:21.909 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.909 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:21.909 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273116+0000", 2024-09-21T21:43:21.909 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:21.909 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.909 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.909 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:21.909 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273116+0000", 2024-09-21T21:43:21.909 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:21.909 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.909 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.909 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:21.909 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273117+0000", 2024-09-21T21:43:21.910 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0999999999999999e-07 2024-09-21T21:43:21.910 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.910 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.910 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:21.910 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273119+0000", 2024-09-21T21:43:21.910 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9970000000000001e-06 2024-09-21T21:43:21.910 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.910 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.910 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:21.910 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273125+0000", 2024-09-21T21:43:21.910 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9540000000000001e-06 2024-09-21T21:43:21.910 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.910 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.910 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:21.910 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273135+0000", 2024-09-21T21:43:21.911 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0597999999999999e-05 2024-09-21T21:43:21.911 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.911 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.911 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:21.911 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278019+0000", 2024-09-21T21:43:21.911 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048834849999999999 2024-09-21T21:43:21.911 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.911 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.911 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:21.911 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278022+0000", 2024-09-21T21:43:21.911 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9560000000000002e-06 2024-09-21T21:43:21.911 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.911 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.911 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:21.912 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246716+0000", 2024-09-21T21:43:21.912 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.968694202 2024-09-21T21:43:21.912 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.912 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.912 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:21.912 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246758+0000", 2024-09-21T21:43:21.912 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1607000000000003e-05 2024-09-21T21:43:21.912 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.912 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.912 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:21.912 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246795+0000", 2024-09-21T21:43:21.912 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6931000000000003e-05 2024-09-21T21:43:21.912 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:21.912 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:21.912 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:21.913 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.913 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.913 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19758 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6529024~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:21.913 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273142+0000", 2024-09-21T21:43:21.913 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 90.616943242000005, 2024-09-21T21:43:21.913 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97375924599999997, 2024-09-21T21:43:21.913 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:21.913 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:21.913 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:21.913 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:21.913 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:21.913 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:21.913 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19758 2024-09-21T21:43:21.913 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.914 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:21.914 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.914 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:21.914 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273142+0000", 2024-09-21T21:43:21.914 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:21.914 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.914 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.914 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:21.914 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273142+0000", 2024-09-21T21:43:21.914 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:21.914 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.914 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.914 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:21.914 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273142+0000", 2024-09-21T21:43:21.915 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0200000000000003e-07 2024-09-21T21:43:21.915 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.915 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.915 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:21.915 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273144+0000", 2024-09-21T21:43:21.915 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1390000000000002e-06 2024-09-21T21:43:21.915 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.915 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.915 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:21.915 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273150+0000", 2024-09-21T21:43:21.915 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6409999999999999e-06 2024-09-21T21:43:21.915 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.916 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.916 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:21.916 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273157+0000", 2024-09-21T21:43:21.916 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1359999999999999e-06 2024-09-21T21:43:21.916 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.916 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.916 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:21.916 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278050+0000", 2024-09-21T21:43:21.916 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048926170000000001 2024-09-21T21:43:21.916 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.916 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.916 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:21.916 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278053+0000", 2024-09-21T21:43:21.916 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4879999999999999e-06 2024-09-21T21:43:21.917 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.917 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.917 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:21.917 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246819+0000", 2024-09-21T21:43:21.917 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96876590399999996 2024-09-21T21:43:21.917 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.917 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.917 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:21.917 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246856+0000", 2024-09-21T21:43:21.917 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7287999999999999e-05 2024-09-21T21:43:21.917 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.917 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.917 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:21.917 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246901+0000", 2024-09-21T21:43:21.918 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4631e-05 2024-09-21T21:43:21.918 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:21.918 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:21.918 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:21.918 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.918 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.918 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19759 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 7933952~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:21.918 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273172+0000", 2024-09-21T21:43:21.918 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 90.616912978000002, 2024-09-21T21:43:21.918 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97384789500000002, 2024-09-21T21:43:21.918 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:21.918 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:21.918 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:21.918 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:21.919 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:21.919 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:21.919 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19759 2024-09-21T21:43:21.919 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.919 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:21.919 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.919 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:21.919 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273172+0000", 2024-09-21T21:43:21.919 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:21.919 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.919 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.919 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:21.919 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273172+0000", 2024-09-21T21:43:21.919 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:21.920 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.920 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.920 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:21.920 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273172+0000", 2024-09-21T21:43:21.920 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6100000000000002e-07 2024-09-21T21:43:21.920 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.920 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.920 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:21.920 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273174+0000", 2024-09-21T21:43:21.920 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.691e-06 2024-09-21T21:43:21.920 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.920 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.920 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:21.920 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273178+0000", 2024-09-21T21:43:21.920 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7579999999999998e-06 2024-09-21T21:43:21.921 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.921 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.921 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:21.921 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273180+0000", 2024-09-21T21:43:21.921 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7199999999999998e-06 2024-09-21T21:43:21.921 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.921 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.921 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:21.921 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278077+0000", 2024-09-21T21:43:21.921 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048969529999999999 2024-09-21T21:43:21.921 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.921 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.921 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:21.921 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278080+0000", 2024-09-21T21:43:21.922 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.066e-06 2024-09-21T21:43:21.922 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.922 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.922 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:21.922 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246925+0000", 2024-09-21T21:43:21.922 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96884490199999995 2024-09-21T21:43:21.922 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.922 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.922 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:21.922 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246974+0000", 2024-09-21T21:43:21.922 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8387999999999998e-05 2024-09-21T21:43:21.922 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.922 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.922 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:21.922 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247020+0000", 2024-09-21T21:43:21.923 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6156000000000003e-05 2024-09-21T21:43:21.923 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:21.923 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:21.923 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:21.923 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.923 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.923 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19760 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 7991296~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:21.923 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273220+0000", 2024-09-21T21:43:21.923 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 90.616864464000002, 2024-09-21T21:43:21.923 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97403125499999998, 2024-09-21T21:43:21.923 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:21.923 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:21.923 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:21.923 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:21.923 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:21.924 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:21.924 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19760 2024-09-21T21:43:21.924 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.924 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:21.924 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.924 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:21.924 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273220+0000", 2024-09-21T21:43:21.924 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:21.924 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.924 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.924 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:21.924 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273220+0000", 2024-09-21T21:43:21.924 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:21.924 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.924 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.925 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:21.925 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273221+0000", 2024-09-21T21:43:21.925 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.65e-07 2024-09-21T21:43:21.925 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.925 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.925 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:21.925 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273228+0000", 2024-09-21T21:43:21.925 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6040000000000004e-06 2024-09-21T21:43:21.925 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.925 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.925 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:21.925 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273233+0000", 2024-09-21T21:43:21.925 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1569999999999999e-06 2024-09-21T21:43:21.925 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.925 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.926 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:21.926 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273238+0000", 2024-09-21T21:43:21.926 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0050000000000004e-06 2024-09-21T21:43:21.926 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.926 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.926 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:21.926 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278107+0000", 2024-09-21T21:43:21.926 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00486857 2024-09-21T21:43:21.926 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.926 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.926 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:21.926 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278111+0000", 2024-09-21T21:43:21.926 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6389999999999998e-06 2024-09-21T21:43:21.926 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.927 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.927 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:21.927 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247069+0000", 2024-09-21T21:43:21.927 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96895796899999997 2024-09-21T21:43:21.927 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.927 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.927 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:21.927 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247166+0000", 2024-09-21T21:43:21.927 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.7158999999999996e-05 2024-09-21T21:43:21.927 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.927 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.927 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:21.927 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247252+0000", 2024-09-21T21:43:21.928 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5886999999999999e-05 2024-09-21T21:43:21.928 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:21.928 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:21.928 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:21.928 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.928 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.928 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19761 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 7962624~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:21.928 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273246+0000", 2024-09-21T21:43:21.928 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 90.616839404000004, 2024-09-21T21:43:21.928 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97415972200000001, 2024-09-21T21:43:21.928 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:21.928 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:21.928 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:21.928 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:21.928 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:21.929 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:21.929 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19761 2024-09-21T21:43:21.929 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.929 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:21.929 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.929 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:21.929 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273246+0000", 2024-09-21T21:43:21.929 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:21.929 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.929 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.929 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:21.929 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273246+0000", 2024-09-21T21:43:21.929 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:21.929 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.930 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.930 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:21.930 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273246+0000", 2024-09-21T21:43:21.930 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5100000000000001e-07 2024-09-21T21:43:21.930 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.930 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.930 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:21.930 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273248+0000", 2024-09-21T21:43:21.930 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1550000000000001e-06 2024-09-21T21:43:21.930 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.930 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.930 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:21.930 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273254+0000", 2024-09-21T21:43:21.930 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.964e-06 2024-09-21T21:43:21.931 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.931 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.931 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:21.931 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273259+0000", 2024-09-21T21:43:21.931 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5499999999999996e-06 2024-09-21T21:43:21.931 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.931 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.931 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:21.931 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278130+0000", 2024-09-21T21:43:21.931 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048715779999999997 2024-09-21T21:43:21.931 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.931 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.931 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:21.931 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278133+0000", 2024-09-21T21:43:21.932 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8739999999999999e-06 2024-09-21T21:43:21.932 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.932 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.932 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:21.932 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247270+0000", 2024-09-21T21:43:21.932 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.969137099 2024-09-21T21:43:21.932 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.932 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.932 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:21.932 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247338+0000", 2024-09-21T21:43:21.932 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7717000000000002e-05 2024-09-21T21:43:21.932 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.932 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.932 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:21.932 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247405+0000", 2024-09-21T21:43:21.933 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7434000000000005e-05 2024-09-21T21:43:21.933 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:21.933 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:21.933 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:21.933 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.933 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.933 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19769 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 5840896~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:21.933 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273287+0000", 2024-09-21T21:43:21.933 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 90.616798119999999, 2024-09-21T21:43:21.933 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97435390200000005, 2024-09-21T21:43:21.933 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:21.933 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:21.933 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:21.933 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:21.934 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:21.934 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:21.934 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19769 2024-09-21T21:43:21.934 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.934 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:21.934 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.934 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:21.934 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273287+0000", 2024-09-21T21:43:21.934 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:21.934 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.934 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.934 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:21.934 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273287+0000", 2024-09-21T21:43:21.934 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:21.935 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.935 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.935 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:21.935 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273287+0000", 2024-09-21T21:43:21.935 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6699999999999999e-07 2024-09-21T21:43:21.935 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.935 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.935 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:21.935 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273289+0000", 2024-09-21T21:43:21.935 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.708e-06 2024-09-21T21:43:21.935 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.935 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.935 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:21.936 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273296+0000", 2024-09-21T21:43:21.936 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8109999999999997e-06 2024-09-21T21:43:21.936 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.936 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.936 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:21.936 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273299+0000", 2024-09-21T21:43:21.936 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0740000000000001e-06 2024-09-21T21:43:21.936 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.936 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.936 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:21.936 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278159+0000", 2024-09-21T21:43:21.936 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048603240000000001 2024-09-21T21:43:21.936 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.936 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.937 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:21.937 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278162+0000", 2024-09-21T21:43:21.937 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1930000000000002e-06 2024-09-21T21:43:21.937 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.937 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.937 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:21.937 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247444+0000", 2024-09-21T21:43:21.937 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96928199100000001 2024-09-21T21:43:21.937 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.937 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.937 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:21.937 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247533+0000", 2024-09-21T21:43:21.937 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8490000000000001e-05 2024-09-21T21:43:21.937 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.938 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.938 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:21.938 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247641+0000", 2024-09-21T21:43:21.938 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000107944 2024-09-21T21:43:21.938 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:21.938 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:21.938 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:21.938 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.938 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.938 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19770 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 5869568~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:21.938 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273313+0000", 2024-09-21T21:43:21.938 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 90.616772018000006, 2024-09-21T21:43:21.938 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97449038899999996, 2024-09-21T21:43:21.938 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:21.939 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:21.939 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:21.939 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:21.939 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:21.939 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:21.939 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19770 2024-09-21T21:43:21.939 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.939 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:21.939 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.939 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:21.939 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273313+0000", 2024-09-21T21:43:21.939 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:21.939 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.939 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.940 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:21.940 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273313+0000", 2024-09-21T21:43:21.940 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:21.940 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.940 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.940 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:21.940 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273313+0000", 2024-09-21T21:43:21.940 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5699999999999998e-07 2024-09-21T21:43:21.940 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.940 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.940 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:21.940 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273315+0000", 2024-09-21T21:43:21.940 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0099999999999998e-06 2024-09-21T21:43:21.940 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.940 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.941 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:21.941 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273321+0000", 2024-09-21T21:43:21.941 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7529999999999998e-06 2024-09-21T21:43:21.941 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.941 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.941 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:21.941 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273328+0000", 2024-09-21T21:43:21.941 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9009999999999997e-06 2024-09-21T21:43:21.941 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.941 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.941 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:21.941 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278183+0000", 2024-09-21T21:43:21.941 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048547770000000002 2024-09-21T21:43:21.941 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.942 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.942 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:21.942 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278186+0000", 2024-09-21T21:43:21.942 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3100000000000001e-06 2024-09-21T21:43:21.942 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.942 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.942 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:21.942 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247676+0000", 2024-09-21T21:43:21.942 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96949004299999997 2024-09-21T21:43:21.942 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.942 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.942 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:21.942 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247757+0000", 2024-09-21T21:43:21.942 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0479e-05 2024-09-21T21:43:21.943 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.943 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.943 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:21.943 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247803+0000", 2024-09-21T21:43:21.943 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6758999999999997e-05 2024-09-21T21:43:21.943 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:21.943 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:21.943 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:21.943 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.943 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.943 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19771 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6586368~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:21.943 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273335+0000", 2024-09-21T21:43:21.943 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 90.616750230999997, 2024-09-21T21:43:21.943 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97458947699999998, 2024-09-21T21:43:21.944 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:21.944 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:21.944 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:21.944 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:21.944 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:21.944 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:21.944 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19771 2024-09-21T21:43:21.944 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.944 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:21.944 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.944 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:21.944 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273335+0000", 2024-09-21T21:43:21.944 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:21.944 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.945 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.945 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:21.945 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273335+0000", 2024-09-21T21:43:21.945 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:21.945 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.945 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.945 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:21.945 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273335+0000", 2024-09-21T21:43:21.945 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3599999999999999e-07 2024-09-21T21:43:21.945 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.945 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.945 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:21.945 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273337+0000", 2024-09-21T21:43:21.945 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0789999999999999e-06 2024-09-21T21:43:21.945 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.946 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.946 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:21.946 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273343+0000", 2024-09-21T21:43:21.946 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6280000000000002e-06 2024-09-21T21:43:21.946 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.946 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.946 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:21.946 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273346+0000", 2024-09-21T21:43:21.946 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.507e-06 2024-09-21T21:43:21.946 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.946 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.946 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:21.946 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278206+0000", 2024-09-21T21:43:21.947 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048600969999999999 2024-09-21T21:43:21.947 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.947 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.947 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:21.947 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278212+0000", 2024-09-21T21:43:21.947 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2079999999999999e-06 2024-09-21T21:43:21.947 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.947 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.947 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:21.947 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247828+0000", 2024-09-21T21:43:21.947 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.969616547 2024-09-21T21:43:21.947 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.947 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.947 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:21.947 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247881+0000", 2024-09-21T21:43:21.948 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3180999999999998e-05 2024-09-21T21:43:21.948 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.948 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.948 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:21.948 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247924+0000", 2024-09-21T21:43:21.948 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2894e-05 2024-09-21T21:43:21.948 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:21.948 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:21.948 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:21.948 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.948 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.948 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19772 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6557696~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:21.948 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273358+0000", 2024-09-21T21:43:21.948 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 90.616726850999996, 2024-09-21T21:43:21.949 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97467031800000004, 2024-09-21T21:43:21.949 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:21.949 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:21.949 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:21.949 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:21.949 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:21.949 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:21.949 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19772 2024-09-21T21:43:21.949 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.949 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:21.949 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.949 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:21.949 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273358+0000", 2024-09-21T21:43:21.949 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:21.950 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.950 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.950 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:21.950 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273358+0000", 2024-09-21T21:43:21.950 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:21.950 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.950 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.950 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:21.950 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273358+0000", 2024-09-21T21:43:21.950 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4499999999999998e-07 2024-09-21T21:43:21.950 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.950 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.950 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:21.950 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273360+0000", 2024-09-21T21:43:21.950 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9190000000000002e-06 2024-09-21T21:43:21.951 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.951 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.951 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:21.951 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273366+0000", 2024-09-21T21:43:21.951 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.891e-06 2024-09-21T21:43:21.951 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.951 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.951 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:21.951 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273371+0000", 2024-09-21T21:43:21.951 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1880000000000001e-06 2024-09-21T21:43:21.951 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.951 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.951 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:21.951 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278237+0000", 2024-09-21T21:43:21.951 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.004865472 2024-09-21T21:43:21.952 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.952 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.952 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:21.952 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278240+0000", 2024-09-21T21:43:21.952 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9550000000000001e-06 2024-09-21T21:43:21.952 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.952 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.952 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:21.952 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247949+0000", 2024-09-21T21:43:21.952 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.969709127 2024-09-21T21:43:21.952 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.952 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.952 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:21.952 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247990+0000", 2024-09-21T21:43:21.952 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1202000000000003e-05 2024-09-21T21:43:21.953 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.953 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.953 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:21.953 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248028+0000", 2024-09-21T21:43:21.953 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8219000000000001e-05 2024-09-21T21:43:21.953 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:21.953 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:21.953 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:21.953 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.953 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.953 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19773 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6615040~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:21.953 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273390+0000", 2024-09-21T21:43:21.953 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 90.616694989999999, 2024-09-21T21:43:21.953 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97476086200000001, 2024-09-21T21:43:21.953 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:21.954 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:21.954 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:21.954 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:21.954 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:21.954 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:21.954 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19773 2024-09-21T21:43:21.954 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.954 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:21.954 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.954 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:21.954 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273390+0000", 2024-09-21T21:43:21.954 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:21.954 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.954 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.955 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:21.955 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273390+0000", 2024-09-21T21:43:21.955 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:21.955 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.955 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.955 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:21.955 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273390+0000", 2024-09-21T21:43:21.955 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9999999999999999e-07 2024-09-21T21:43:21.955 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.955 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.955 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:21.955 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273392+0000", 2024-09-21T21:43:21.955 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0640000000000001e-06 2024-09-21T21:43:21.955 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.955 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.956 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:21.956 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273397+0000", 2024-09-21T21:43:21.956 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.113e-06 2024-09-21T21:43:21.956 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.956 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.956 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:21.956 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273401+0000", 2024-09-21T21:43:21.956 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3119999999999998e-06 2024-09-21T21:43:21.956 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.956 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.956 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:21.956 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278271+0000", 2024-09-21T21:43:21.956 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048698880000000002 2024-09-21T21:43:21.956 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.956 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.957 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:21.957 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278278+0000", 2024-09-21T21:43:21.957 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1210000000000001e-06 2024-09-21T21:43:21.957 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.957 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.957 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:21.957 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248060+0000", 2024-09-21T21:43:21.957 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96978224999999996 2024-09-21T21:43:21.957 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.957 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.957 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:21.957 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248111+0000", 2024-09-21T21:43:21.957 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.096e-05 2024-09-21T21:43:21.957 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.958 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.958 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:21.958 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248151+0000", 2024-09-21T21:43:21.958 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9854000000000002e-05 2024-09-21T21:43:21.958 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:21.958 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:21.958 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:21.958 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.958 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.958 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19784 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6643712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:21.958 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273405+0000", 2024-09-21T21:43:21.958 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 90.616680269, 2024-09-21T21:43:21.958 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97491798900000004, 2024-09-21T21:43:21.958 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:21.958 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:21.959 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:21.959 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:21.959 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:21.959 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:21.959 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19784 2024-09-21T21:43:21.959 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.959 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:21.959 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.959 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:21.959 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273405+0000", 2024-09-21T21:43:21.959 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:21.959 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.959 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.959 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:21.959 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273405+0000", 2024-09-21T21:43:21.960 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:21.960 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.960 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.960 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:21.960 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273405+0000", 2024-09-21T21:43:21.960 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2100000000000001e-07 2024-09-21T21:43:21.960 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.960 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.960 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:21.960 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273406+0000", 2024-09-21T21:43:21.960 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2049999999999999e-06 2024-09-21T21:43:21.960 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.960 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.960 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:21.961 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273409+0000", 2024-09-21T21:43:21.961 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0599999999999999e-06 2024-09-21T21:43:21.961 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.961 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.961 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:21.961 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273412+0000", 2024-09-21T21:43:21.961 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0730000000000001e-06 2024-09-21T21:43:21.961 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.961 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.961 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:21.961 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278317+0000", 2024-09-21T21:43:21.961 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0049045520000000004 2024-09-21T21:43:21.961 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.961 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.961 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:21.962 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278321+0000", 2024-09-21T21:43:21.962 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6249999999999998e-06 2024-09-21T21:43:21.962 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.962 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.962 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:21.962 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248179+0000", 2024-09-21T21:43:21.962 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96985744500000004 2024-09-21T21:43:21.962 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.962 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.962 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:21.962 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248253+0000", 2024-09-21T21:43:21.962 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3829000000000001e-05 2024-09-21T21:43:21.962 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.962 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.963 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:21.963 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248323+0000", 2024-09-21T21:43:21.963 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9979000000000002e-05 2024-09-21T21:43:21.963 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:21.963 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:21.963 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:21.963 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.963 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.963 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19785 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6672384~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:21.963 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273415+0000", 2024-09-21T21:43:21.963 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 90.616669682999998, 2024-09-21T21:43:21.963 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97500777800000005, 2024-09-21T21:43:21.963 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:21.963 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:21.963 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:21.964 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:21.964 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:21.964 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:21.964 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19785 2024-09-21T21:43:21.964 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.964 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:21.964 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.964 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:21.964 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273415+0000", 2024-09-21T21:43:21.964 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:21.964 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.964 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.964 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:21.964 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273415+0000", 2024-09-21T21:43:21.964 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:21.965 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.965 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.965 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:21.965 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273415+0000", 2024-09-21T21:43:21.965 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0100000000000001e-07 2024-09-21T21:43:21.965 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.965 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.965 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:21.965 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273417+0000", 2024-09-21T21:43:21.965 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.114e-06 2024-09-21T21:43:21.965 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.965 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.965 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:21.965 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273420+0000", 2024-09-21T21:43:21.965 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6770000000000001e-06 2024-09-21T21:43:21.966 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.966 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.966 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:21.966 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273423+0000", 2024-09-21T21:43:21.966 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5450000000000002e-06 2024-09-21T21:43:21.966 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.966 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.966 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:21.966 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278355+0000", 2024-09-21T21:43:21.966 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0049318590000000002 2024-09-21T21:43:21.966 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.966 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.966 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:21.966 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278360+0000", 2024-09-21T21:43:21.967 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3079999999999998e-06 2024-09-21T21:43:21.967 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.967 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.967 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:21.967 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248349+0000", 2024-09-21T21:43:21.967 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96998909899999997 2024-09-21T21:43:21.967 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.967 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.967 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:21.967 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248387+0000", 2024-09-21T21:43:21.967 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7657000000000002e-05 2024-09-21T21:43:21.967 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.967 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.967 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:21.967 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248423+0000", 2024-09-21T21:43:21.968 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6318e-05 2024-09-21T21:43:21.968 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:21.968 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:21.968 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:21.968 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.968 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.968 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19842 3.d 3:b7fa2ca0:::rbd_data.112157a479c.000000000000004d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:21.968 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273426+0000", 2024-09-21T21:43:21.968 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 90.616658631000007, 2024-09-21T21:43:21.968 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.98136526800000001, 2024-09-21T21:43:21.968 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:21.968 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:21.968 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:21.968 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:21.969 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:21.969 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:21.969 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19842 2024-09-21T21:43:21.969 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.969 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:21.969 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.969 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:21.969 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273426+0000", 2024-09-21T21:43:21.969 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:21.969 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.969 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.969 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:21.969 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273426+0000", 2024-09-21T21:43:21.969 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:21.969 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.970 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.970 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:21.970 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273427+0000", 2024-09-21T21:43:21.970 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.72e-07 2024-09-21T21:43:21.970 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.970 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.970 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:21.970 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275836+0000", 2024-09-21T21:43:21.970 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00240909 2024-09-21T21:43:21.970 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.970 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.970 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:21.970 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275844+0000", 2024-09-21T21:43:21.970 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3750000000000003e-06 2024-09-21T21:43:21.971 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.971 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.971 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:21.971 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275849+0000", 2024-09-21T21:43:21.971 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0100000000000003e-06 2024-09-21T21:43:21.971 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.971 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.971 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:21.971 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278394+0000", 2024-09-21T21:43:21.971 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025449890000000001 2024-09-21T21:43:21.971 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.971 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.971 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:21.971 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278402+0000", 2024-09-21T21:43:21.971 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6359999999999994e-06 2024-09-21T21:43:21.972 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.972 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.972 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:21.972 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248454+0000", 2024-09-21T21:43:21.972 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97005259499999996 2024-09-21T21:43:21.972 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.972 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.972 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:21.972 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248576+0000", 2024-09-21T21:43:21.972 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000122127 2024-09-21T21:43:21.972 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.972 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.972 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 2", 2024-09-21T21:43:21.972 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248844+0000", 2024-09-21T21:43:21.973 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00026784699999999999 2024-09-21T21:43:21.973 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.973 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.973 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:21.973 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.252487+0000", 2024-09-21T21:43:21.973 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0036427920000000002 2024-09-21T21:43:21.973 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.973 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.973 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:21.973 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.254691+0000", 2024-09-21T21:43:21.973 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002203899 2024-09-21T21:43:21.973 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.973 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.973 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:21.973 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.254748+0000", 2024-09-21T21:43:21.974 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7163000000000001e-05 2024-09-21T21:43:21.974 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.974 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.974 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:21.974 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.254792+0000", 2024-09-21T21:43:21.974 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3473000000000002e-05 2024-09-21T21:43:21.974 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:21.974 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:21.974 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:21.974 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.974 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.974 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19843 3.d 3:b7fa2ca0:::rbd_data.112157a479c.000000000000004d:head [write 0~0] snapc 21=[21] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:21.974 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.275860+0000", 2024-09-21T21:43:21.974 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 90.614224730999993, 2024-09-21T21:43:21.975 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.98183651999999999, 2024-09-21T21:43:21.975 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:21.975 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:21.975 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:21.975 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:21.975 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:21.975 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:21.975 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19843 2024-09-21T21:43:21.975 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.975 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:21.975 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.975 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:21.975 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275860+0000", 2024-09-21T21:43:21.975 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:21.975 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.976 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.976 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:21.976 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275860+0000", 2024-09-21T21:43:21.976 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:21.976 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.976 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.976 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:21.976 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275869+0000", 2024-09-21T21:43:21.976 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6300000000000004e-06 2024-09-21T21:43:21.976 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.976 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.976 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:21.976 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275871+0000", 2024-09-21T21:43:21.976 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2060000000000001e-06 2024-09-21T21:43:21.977 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.977 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.977 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:21.977 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275876+0000", 2024-09-21T21:43:21.977 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8180000000000003e-06 2024-09-21T21:43:21.977 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.977 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.977 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:21.977 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275880+0000", 2024-09-21T21:43:21.977 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1829999999999996e-06 2024-09-21T21:43:21.977 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.977 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.977 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:21.977 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278416+0000", 2024-09-21T21:43:21.977 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025361149999999998 2024-09-21T21:43:21.978 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.978 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.978 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:21.978 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278419+0000", 2024-09-21T21:43:21.978 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.163e-06 2024-09-21T21:43:21.978 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.978 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.978 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:21.978 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.250249+0000", 2024-09-21T21:43:21.978 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97182984299999997 2024-09-21T21:43:21.978 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.978 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.978 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:21.978 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.250321+0000", 2024-09-21T21:43:21.979 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1656000000000006e-05 2024-09-21T21:43:21.979 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.979 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.979 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 2", 2024-09-21T21:43:21.979 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.250419+0000", 2024-09-21T21:43:21.979 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.8277999999999997e-05 2024-09-21T21:43:21.979 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.979 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.979 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:21.979 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.252516+0000", 2024-09-21T21:43:21.979 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002097224 2024-09-21T21:43:21.979 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.979 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.979 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:21.979 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.257643+0000", 2024-09-21T21:43:21.980 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0051262319999999997 2024-09-21T21:43:21.980 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.980 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.980 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:21.980 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.257673+0000", 2024-09-21T21:43:21.980 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0023e-05 2024-09-21T21:43:21.980 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.980 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.980 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:21.980 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.257697+0000", 2024-09-21T21:43:21.980 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4148999999999999e-05 2024-09-21T21:43:21.980 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:21.980 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:21.980 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:21.980 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.981 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.981 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5626.0:12334 3.3 3:c643a86c:::rbd_data.1121faebdfe3.0000000000000002:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:43:21.981 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:14.411628+0000", 2024-09-21T21:43:21.981 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 67.478456432000002, 2024-09-21T21:43:21.981 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0086947559999999, 2024-09-21T21:43:21.981 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:21.981 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:21.981 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:21.981 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:21.981 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5626", 2024-09-21T21:43:21.981 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:43:21.981 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 12334 2024-09-21T21:43:21.981 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.981 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:21.981 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.982 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:21.982 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411628+0000", 2024-09-21T21:43:21.982 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:21.982 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.982 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.982 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:21.982 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411628+0000", 2024-09-21T21:43:21.982 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:21.982 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.982 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.982 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:21.982 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411629+0000", 2024-09-21T21:43:21.982 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1200000000000002e-07 2024-09-21T21:43:21.982 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.983 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.983 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:21.983 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411633+0000", 2024-09-21T21:43:21.983 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1699999999999999e-06 2024-09-21T21:43:21.983 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.983 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.983 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:21.983 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411645+0000", 2024-09-21T21:43:21.983 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1328999999999999e-05 2024-09-21T21:43:21.983 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.983 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.983 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:21.983 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411652+0000", 2024-09-21T21:43:21.983 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9990000000000002e-06 2024-09-21T21:43:21.984 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.984 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.984 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:21.984 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411717+0000", 2024-09-21T21:43:21.984 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5519999999999996e-05 2024-09-21T21:43:21.984 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.984 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.984 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op must wait for map", 2024-09-21T21:43:21.984 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411725+0000", 2024-09-21T21:43:21.984 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2609999999999996e-06 2024-09-21T21:43:21.984 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.984 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.984 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:21.984 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415904+0000", 2024-09-21T21:43:21.985 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0041792749999999997 2024-09-21T21:43:21.985 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.985 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.985 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:21.985 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415913+0000", 2024-09-21T21:43:21.985 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.054e-06 2024-09-21T21:43:21.985 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.985 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.985 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:21.985 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420096+0000", 2024-09-21T21:43:21.985 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0041835670000001 2024-09-21T21:43:21.985 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.985 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.985 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:21.985 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420216+0000", 2024-09-21T21:43:21.985 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000120025 2024-09-21T21:43:21.986 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.986 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.986 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:21.986 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420323+0000", 2024-09-21T21:43:21.986 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010674400000000001 2024-09-21T21:43:21.986 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:21.986 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:21.986 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:21.986 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.986 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.986 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5626.0:12339 3.3 3:c643a86c:::rbd_data.1121faebdfe3.0000000000000002:head [read 4096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:43:21.986 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:14.411749+0000", 2024-09-21T21:43:21.986 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 67.478336245999998, 2024-09-21T21:43:21.986 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.008818056, 2024-09-21T21:43:21.987 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:21.987 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:21.987 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:21.987 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:21.987 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5626", 2024-09-21T21:43:21.987 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:43:21.987 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 12339 2024-09-21T21:43:21.987 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.987 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:21.987 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.987 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:21.987 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411749+0000", 2024-09-21T21:43:21.987 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:21.987 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.988 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.988 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:21.988 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411749+0000", 2024-09-21T21:43:21.988 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:21.988 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.988 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.988 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:21.988 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411749+0000", 2024-09-21T21:43:21.988 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0900000000000002e-07 2024-09-21T21:43:21.988 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.988 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.988 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:21.988 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411752+0000", 2024-09-21T21:43:21.988 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5610000000000001e-06 2024-09-21T21:43:21.988 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.988 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.989 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:21.989 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411758+0000", 2024-09-21T21:43:21.989 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1890000000000004e-06 2024-09-21T21:43:21.989 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.989 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.989 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:21.989 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411763+0000", 2024-09-21T21:43:21.989 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7589999999999997e-06 2024-09-21T21:43:21.989 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.989 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.989 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:21.989 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411803+0000", 2024-09-21T21:43:21.989 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9922999999999997e-05 2024-09-21T21:43:21.989 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.989 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.990 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting_for_map not empty", 2024-09-21T21:43:21.990 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411809+0000", 2024-09-21T21:43:21.990 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3779999999999998e-06 2024-09-21T21:43:21.990 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.990 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.990 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:21.990 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415937+0000", 2024-09-21T21:43:21.990 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0041276100000000003 2024-09-21T21:43:21.990 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.990 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.990 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:21.990 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415944+0000", 2024-09-21T21:43:21.990 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8329999999999994e-06 2024-09-21T21:43:21.991 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.991 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.991 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:21.991 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420411+0000", 2024-09-21T21:43:21.991 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0044667300000001 2024-09-21T21:43:21.991 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.991 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.991 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:21.991 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420475+0000", 2024-09-21T21:43:21.991 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4356000000000005e-05 2024-09-21T21:43:21.991 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.991 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.991 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:21.991 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420567+0000", 2024-09-21T21:43:21.991 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.1308000000000001e-05 2024-09-21T21:43:21.991 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:21.992 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:21.992 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:21.992 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.992 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.992 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5626.0:12345 3.3 3:c643a86c:::rbd_data.1121faebdfe3.0000000000000002:head [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:43:21.992 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:14.411831+0000", 2024-09-21T21:43:21.992 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 67.478254192999998, 2024-09-21T21:43:21.992 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0089163649999999, 2024-09-21T21:43:21.992 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:21.992 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:21.992 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:21.992 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:21.992 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5626", 2024-09-21T21:43:21.992 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:43:21.993 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 12345 2024-09-21T21:43:21.993 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.993 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:21.993 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.993 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:21.993 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411831+0000", 2024-09-21T21:43:21.993 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:21.993 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.993 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.993 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:21.993 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411831+0000", 2024-09-21T21:43:21.993 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:21.993 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.993 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.993 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:21.994 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411831+0000", 2024-09-21T21:43:21.994 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.08e-07 2024-09-21T21:43:21.994 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.994 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.994 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:21.994 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411833+0000", 2024-09-21T21:43:21.994 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3089999999999998e-06 2024-09-21T21:43:21.994 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.994 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.994 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:21.994 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411840+0000", 2024-09-21T21:43:21.994 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.472e-06 2024-09-21T21:43:21.994 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.994 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.995 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:21.995 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411845+0000", 2024-09-21T21:43:21.995 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7910000000000004e-06 2024-09-21T21:43:21.995 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.995 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.995 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:21.995 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411890+0000", 2024-09-21T21:43:21.995 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5763000000000002e-05 2024-09-21T21:43:21.995 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.995 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.995 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting_for_map not empty", 2024-09-21T21:43:21.995 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411896+0000", 2024-09-21T21:43:21.995 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0190000000000003e-06 2024-09-21T21:43:21.995 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.995 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.996 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:21.996 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415967+0000", 2024-09-21T21:43:21.996 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0040705749999999999 2024-09-21T21:43:21.996 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.996 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.996 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:21.996 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415974+0000", 2024-09-21T21:43:21.996 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3200000000000002e-06 2024-09-21T21:43:21.996 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.996 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.996 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:21.996 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420594+0000", 2024-09-21T21:43:21.996 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.004620093 2024-09-21T21:43:21.996 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.996 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.997 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:21.997 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420658+0000", 2024-09-21T21:43:21.997 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3678999999999998e-05 2024-09-21T21:43:21.997 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:21.997 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:21.997 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:21.997 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420747+0000", 2024-09-21T21:43:21.997 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8936000000000004e-05 2024-09-21T21:43:21.997 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:21.997 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:21.997 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:21.997 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:21.997 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:21.997 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:43:21.999 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T21:43:21.999 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:43:21.997+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:43:22.099 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:43:22.098+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:43:22.102 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:43:22.102 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:43:22.102 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-21T21:43:22.102 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:43:22.111 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T21:43:22.200 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:43:22.199+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:43:22.214 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:43:22.214 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:43:22.214 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:43:22.214 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:43:22.214 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:43:22.224 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T21:43:22.309 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:43:22.308+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:43:22.375 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:43:22.375 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-21T21:43:22.375 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-21T21:43:22.375 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:43:22.375 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.375 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6974 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5246976~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:22.375 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.922932+0000", 2024-09-21T21:43:22.376 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 112.449977739, 2024-09-21T21:43:22.376 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041773352999999999, 2024-09-21T21:43:22.376 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:22.376 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:22.376 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:22.376 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:22.376 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:22.376 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:22.376 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6974 2024-09-21T21:43:22.376 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.376 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:22.376 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.376 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:22.376 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922932+0000", 2024-09-21T21:43:22.376 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:22.377 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.377 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.377 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:22.377 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922932+0000", 2024-09-21T21:43:22.377 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:22.377 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.377 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.377 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:22.377 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922933+0000", 2024-09-21T21:43:22.377 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.46e-07 2024-09-21T21:43:22.377 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.377 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.377 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:22.377 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922940+0000", 2024-09-21T21:43:22.377 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6140000000000003e-06 2024-09-21T21:43:22.378 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.378 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.378 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:22.378 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922949+0000", 2024-09-21T21:43:22.378 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6899999999999998e-06 2024-09-21T21:43:22.378 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.378 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.378 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:22.378 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922955+0000", 2024-09-21T21:43:22.378 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5140000000000001e-06 2024-09-21T21:43:22.378 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.378 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.378 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:22.378 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964547+0000", 2024-09-21T21:43:22.379 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041592854999999998 2024-09-21T21:43:22.379 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.379 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.379 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:22.379 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964617+0000", 2024-09-21T21:43:22.379 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9058000000000002e-05 2024-09-21T21:43:22.379 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.379 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.379 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:22.379 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964706+0000", 2024-09-21T21:43:22.379 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9276000000000004e-05 2024-09-21T21:43:22.379 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:22.379 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:22.380 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:22.380 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.380 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.380 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6976 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5261312~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:22.380 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.922961+0000", 2024-09-21T21:43:22.380 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 112.449948694, 2024-09-21T21:43:22.380 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041863648000000003, 2024-09-21T21:43:22.381 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:22.381 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:22.381 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:22.381 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:22.381 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:22.381 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:22.381 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6976 2024-09-21T21:43:22.381 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.381 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:22.381 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.381 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:22.382 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922961+0000", 2024-09-21T21:43:22.382 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:22.382 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.382 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.382 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:22.382 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922961+0000", 2024-09-21T21:43:22.382 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:22.382 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.382 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.382 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:22.382 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922962+0000", 2024-09-21T21:43:22.382 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.89e-07 2024-09-21T21:43:22.382 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.383 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.383 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:22.383 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922964+0000", 2024-09-21T21:43:22.383 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1270000000000001e-06 2024-09-21T21:43:22.383 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.383 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.383 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:22.383 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922970+0000", 2024-09-21T21:43:22.383 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8030000000000002e-06 2024-09-21T21:43:22.383 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.383 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.384 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:22.384 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922979+0000", 2024-09-21T21:43:22.384 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5559999999999996e-06 2024-09-21T21:43:22.384 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.384 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.384 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:22.384 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964734+0000", 2024-09-21T21:43:22.384 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041754415000000003 2024-09-21T21:43:22.384 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.384 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.384 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:22.384 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964775+0000", 2024-09-21T21:43:22.384 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0806999999999997e-05 2024-09-21T21:43:22.384 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.385 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.385 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:22.385 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964825+0000", 2024-09-21T21:43:22.385 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0550999999999998e-05 2024-09-21T21:43:22.385 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:22.385 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:22.385 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:22.385 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.385 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.385 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6978 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5275648~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:22.385 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.922987+0000", 2024-09-21T21:43:22.385 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 112.44992290899999, 2024-09-21T21:43:22.385 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042043876000000001, 2024-09-21T21:43:22.385 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:22.386 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:22.386 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:22.386 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:22.386 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:22.386 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:22.386 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6978 2024-09-21T21:43:22.386 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.386 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:22.386 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.386 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:22.386 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922987+0000", 2024-09-21T21:43:22.386 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:22.386 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.387 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.387 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:22.387 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922987+0000", 2024-09-21T21:43:22.387 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:22.387 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.387 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.387 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:22.387 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922988+0000", 2024-09-21T21:43:22.387 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1199999999999998e-07 2024-09-21T21:43:22.387 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.387 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.387 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:22.387 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922990+0000", 2024-09-21T21:43:22.387 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.142e-06 2024-09-21T21:43:22.387 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.388 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.388 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:22.388 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922997+0000", 2024-09-21T21:43:22.388 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.199e-06 2024-09-21T21:43:22.388 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.388 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.388 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:22.388 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923004+0000", 2024-09-21T21:43:22.388 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3599999999999998e-06 2024-09-21T21:43:22.388 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.388 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.388 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:22.388 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964856+0000", 2024-09-21T21:43:22.388 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041851521000000003 2024-09-21T21:43:22.388 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.389 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.389 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:22.389 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964944+0000", 2024-09-21T21:43:22.389 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7859e-05 2024-09-21T21:43:22.389 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.389 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.389 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:22.389 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965031+0000", 2024-09-21T21:43:22.389 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7383e-05 2024-09-21T21:43:22.389 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:22.389 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:22.389 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:22.389 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.389 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.389 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6980 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5289984~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:22.390 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923011+0000", 2024-09-21T21:43:22.390 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 112.44989957200001, 2024-09-21T21:43:22.390 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042139337999999998, 2024-09-21T21:43:22.390 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:22.390 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:22.390 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:22.390 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:22.390 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:22.390 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:22.390 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6980 2024-09-21T21:43:22.390 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.391 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:22.391 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.391 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:22.391 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923011+0000", 2024-09-21T21:43:22.391 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:22.391 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.391 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.391 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:22.391 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923011+0000", 2024-09-21T21:43:22.391 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:22.391 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.392 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.392 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:22.392 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923011+0000", 2024-09-21T21:43:22.392 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6699999999999999e-07 2024-09-21T21:43:22.392 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.392 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.392 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:22.392 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923013+0000", 2024-09-21T21:43:22.392 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0279999999999999e-06 2024-09-21T21:43:22.393 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.393 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.393 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:22.393 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923018+0000", 2024-09-21T21:43:22.393 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4790000000000004e-06 2024-09-21T21:43:22.393 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.393 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.393 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:22.393 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923029+0000", 2024-09-21T21:43:22.393 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0601999999999999e-05 2024-09-21T21:43:22.393 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.394 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.394 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:22.394 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965059+0000", 2024-09-21T21:43:22.394 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042029947999999998 2024-09-21T21:43:22.394 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.394 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.394 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:22.394 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965100+0000", 2024-09-21T21:43:22.394 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1170000000000001e-05 2024-09-21T21:43:22.394 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.394 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.394 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:22.394 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965150+0000", 2024-09-21T21:43:22.395 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9743999999999997e-05 2024-09-21T21:43:22.395 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:22.395 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:22.395 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:22.395 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.395 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.395 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6982 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5304320~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:22.395 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923036+0000", 2024-09-21T21:43:22.395 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 112.449873751, 2024-09-21T21:43:22.395 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042225666000000002, 2024-09-21T21:43:22.396 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:22.396 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:22.396 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:22.396 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:22.396 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:22.396 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:22.396 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6982 2024-09-21T21:43:22.396 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.396 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:22.396 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.396 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:22.396 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923036+0000", 2024-09-21T21:43:22.396 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:22.396 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.397 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.397 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:22.397 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923036+0000", 2024-09-21T21:43:22.397 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:22.397 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.397 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.397 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:22.397 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923037+0000", 2024-09-21T21:43:22.397 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2800000000000002e-07 2024-09-21T21:43:22.397 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.397 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.397 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:22.398 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923039+0000", 2024-09-21T21:43:22.398 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1500000000000002e-06 2024-09-21T21:43:22.398 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.398 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.398 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:22.398 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923049+0000", 2024-09-21T21:43:22.398 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0236e-05 2024-09-21T21:43:22.398 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.398 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.398 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:22.398 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923056+0000", 2024-09-21T21:43:22.398 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2540000000000003e-06 2024-09-21T21:43:22.399 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.399 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.399 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:22.399 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965173+0000", 2024-09-21T21:43:22.399 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042116686 2024-09-21T21:43:22.399 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.399 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.399 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:22.399 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965215+0000", 2024-09-21T21:43:22.399 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1696000000000002e-05 2024-09-21T21:43:22.399 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.400 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.400 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:22.400 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965262+0000", 2024-09-21T21:43:22.400 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7216000000000003e-05 2024-09-21T21:43:22.400 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:22.400 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:22.400 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:22.400 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.400 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.400 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6984 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5318656~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:22.400 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923063+0000", 2024-09-21T21:43:22.400 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 112.449847097, 2024-09-21T21:43:22.401 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042318651999999998, 2024-09-21T21:43:22.401 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:22.401 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:22.401 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:22.401 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:22.401 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:22.401 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:22.401 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6984 2024-09-21T21:43:22.401 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.402 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:22.402 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.402 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:22.402 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923063+0000", 2024-09-21T21:43:22.402 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:22.402 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.402 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.402 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:22.402 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923063+0000", 2024-09-21T21:43:22.402 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:22.403 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.403 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.403 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:22.403 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923064+0000", 2024-09-21T21:43:22.403 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2e-07 2024-09-21T21:43:22.403 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.403 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.403 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:22.403 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923066+0000", 2024-09-21T21:43:22.403 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.057e-06 2024-09-21T21:43:22.403 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.403 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.403 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:22.403 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923071+0000", 2024-09-21T21:43:22.404 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8810000000000001e-06 2024-09-21T21:43:22.404 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.404 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.404 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:22.404 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923078+0000", 2024-09-21T21:43:22.404 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5479999999999998e-06 2024-09-21T21:43:22.404 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.404 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.404 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:22.404 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965288+0000", 2024-09-21T21:43:22.404 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042209576999999998 2024-09-21T21:43:22.404 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.404 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.404 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:22.404 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965332+0000", 2024-09-21T21:43:22.405 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4366999999999997e-05 2024-09-21T21:43:22.405 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.405 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.405 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:22.405 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965382+0000", 2024-09-21T21:43:22.405 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9802000000000002e-05 2024-09-21T21:43:22.405 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:22.405 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:22.405 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:22.405 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.405 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.405 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6987 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5332992~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:22.405 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923085+0000", 2024-09-21T21:43:22.405 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 112.44982557199999, 2024-09-21T21:43:22.405 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042412103999999999, 2024-09-21T21:43:22.406 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:22.406 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:22.406 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:22.406 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:22.406 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:22.406 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:22.406 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6987 2024-09-21T21:43:22.406 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.406 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:22.406 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.406 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:22.406 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923085+0000", 2024-09-21T21:43:22.406 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:22.406 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.406 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.407 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:22.407 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923085+0000", 2024-09-21T21:43:22.407 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:22.407 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.407 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.407 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:22.407 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923085+0000", 2024-09-21T21:43:22.407 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.89e-07 2024-09-21T21:43:22.407 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.407 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.407 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:22.407 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923087+0000", 2024-09-21T21:43:22.407 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0810000000000001e-06 2024-09-21T21:43:22.407 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.407 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.407 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:22.408 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923098+0000", 2024-09-21T21:43:22.408 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1299e-05 2024-09-21T21:43:22.408 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.408 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.408 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:22.408 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923104+0000", 2024-09-21T21:43:22.408 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9100000000000002e-06 2024-09-21T21:43:22.408 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.408 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.408 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:22.408 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965404+0000", 2024-09-21T21:43:22.408 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042299622000000002 2024-09-21T21:43:22.408 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.408 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.408 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:22.409 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965445+0000", 2024-09-21T21:43:22.409 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0701000000000002e-05 2024-09-21T21:43:22.409 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.409 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.409 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:22.409 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965497+0000", 2024-09-21T21:43:22.409 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2101999999999997e-05 2024-09-21T21:43:22.409 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:22.409 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:22.409 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:22.409 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.409 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.409 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6989 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5347328~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:22.409 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923111+0000", 2024-09-21T21:43:22.409 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 112.449799232, 2024-09-21T21:43:22.410 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042505188999999999, 2024-09-21T21:43:22.410 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:22.410 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:22.410 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:22.410 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:22.410 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:22.410 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:22.410 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6989 2024-09-21T21:43:22.410 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.410 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:22.410 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.410 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:22.410 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923111+0000", 2024-09-21T21:43:22.410 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:22.410 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.411 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.411 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:22.411 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923111+0000", 2024-09-21T21:43:22.411 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:22.411 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.411 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.411 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:22.411 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923111+0000", 2024-09-21T21:43:22.411 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4499999999999997e-07 2024-09-21T21:43:22.411 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.411 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.411 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:22.411 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923114+0000", 2024-09-21T21:43:22.411 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1569999999999998e-06 2024-09-21T21:43:22.411 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.412 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.412 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:22.412 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923120+0000", 2024-09-21T21:43:22.412 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1759999999999998e-06 2024-09-21T21:43:22.412 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.412 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.412 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:22.412 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923125+0000", 2024-09-21T21:43:22.412 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2560000000000002e-06 2024-09-21T21:43:22.412 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.412 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.412 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:22.412 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965523+0000", 2024-09-21T21:43:22.412 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042398094999999997 2024-09-21T21:43:22.413 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.413 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.413 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:22.413 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965566+0000", 2024-09-21T21:43:22.413 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3393999999999998e-05 2024-09-21T21:43:22.413 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.413 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.413 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:22.413 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965616+0000", 2024-09-21T21:43:22.413 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9666000000000002e-05 2024-09-21T21:43:22.413 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:22.413 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:22.413 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:22.413 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.413 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.414 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6991 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5361664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:22.414 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923132+0000", 2024-09-21T21:43:22.414 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 112.449777943, 2024-09-21T21:43:22.414 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042600779999999998, 2024-09-21T21:43:22.414 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:22.414 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:22.414 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:22.414 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:22.414 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:22.414 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:22.414 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6991 2024-09-21T21:43:22.414 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.414 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:22.414 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.415 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:22.415 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923132+0000", 2024-09-21T21:43:22.415 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:22.415 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.415 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.415 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:22.415 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923132+0000", 2024-09-21T21:43:22.415 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:22.415 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.415 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.415 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:22.415 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923132+0000", 2024-09-21T21:43:22.415 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4200000000000002e-07 2024-09-21T21:43:22.415 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.415 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.416 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:22.416 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923134+0000", 2024-09-21T21:43:22.416 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2270000000000001e-06 2024-09-21T21:43:22.416 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.416 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.416 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:22.416 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923137+0000", 2024-09-21T21:43:22.416 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1930000000000002e-06 2024-09-21T21:43:22.416 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.416 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.416 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:22.416 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923139+0000", 2024-09-21T21:43:22.416 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4710000000000001e-06 2024-09-21T21:43:22.416 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.417 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.417 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:22.417 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965639+0000", 2024-09-21T21:43:22.417 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.04249965 2024-09-21T21:43:22.417 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.417 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.417 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:22.417 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965679+0000", 2024-09-21T21:43:22.417 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9852999999999999e-05 2024-09-21T21:43:22.417 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.417 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.417 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:22.417 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965733+0000", 2024-09-21T21:43:22.417 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4144000000000002e-05 2024-09-21T21:43:22.418 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:22.418 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:22.418 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:22.418 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.418 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.418 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6993 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5384192~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:22.418 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923143+0000", 2024-09-21T21:43:22.418 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 112.449767149, 2024-09-21T21:43:22.418 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042708182999999997, 2024-09-21T21:43:22.418 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:22.418 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:22.418 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:22.418 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:22.418 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:22.419 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:22.419 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6993 2024-09-21T21:43:22.419 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.419 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:22.419 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.419 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:22.419 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923143+0000", 2024-09-21T21:43:22.419 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:22.419 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.419 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.419 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:22.419 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923143+0000", 2024-09-21T21:43:22.419 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:22.419 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.420 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.420 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:22.420 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923143+0000", 2024-09-21T21:43:22.420 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0800000000000001e-07 2024-09-21T21:43:22.420 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.420 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.420 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:22.420 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923144+0000", 2024-09-21T21:43:22.420 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.009e-06 2024-09-21T21:43:22.420 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.420 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.420 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:22.420 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923148+0000", 2024-09-21T21:43:22.420 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.737e-06 2024-09-21T21:43:22.420 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.421 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.421 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:22.421 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923150+0000", 2024-09-21T21:43:22.421 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4669999999999998e-06 2024-09-21T21:43:22.421 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.421 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.421 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:22.421 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965760+0000", 2024-09-21T21:43:22.421 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042609399999999999 2024-09-21T21:43:22.421 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.421 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.421 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:22.421 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965801+0000", 2024-09-21T21:43:22.421 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1399999999999997e-05 2024-09-21T21:43:22.422 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.422 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.422 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:22.422 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965851+0000", 2024-09-21T21:43:22.422 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9962e-05 2024-09-21T21:43:22.422 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:22.422 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:22.422 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:22.422 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.422 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.422 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6995 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5398528~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:22.422 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923153+0000", 2024-09-21T21:43:22.422 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 112.44975678500001, 2024-09-21T21:43:22.423 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042828034000000001, 2024-09-21T21:43:22.423 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:22.423 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:22.423 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:22.423 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:22.423 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:22.423 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:22.423 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6995 2024-09-21T21:43:22.423 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.423 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:22.423 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.423 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:22.423 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923153+0000", 2024-09-21T21:43:22.423 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:22.424 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.424 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.424 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:22.424 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923153+0000", 2024-09-21T21:43:22.424 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:22.424 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.424 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.424 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:22.424 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923154+0000", 2024-09-21T21:43:22.424 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2600000000000001e-07 2024-09-21T21:43:22.424 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.424 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.424 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:22.424 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923158+0000", 2024-09-21T21:43:22.424 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0559999999999998e-06 2024-09-21T21:43:22.425 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.425 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.425 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:22.425 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923161+0000", 2024-09-21T21:43:22.425 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6519999999999999e-06 2024-09-21T21:43:22.425 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.425 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.425 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:22.425 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923170+0000", 2024-09-21T21:43:22.425 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5720000000000003e-06 2024-09-21T21:43:22.425 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.425 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.425 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:22.425 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965887+0000", 2024-09-21T21:43:22.425 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042716863000000001 2024-09-21T21:43:22.425 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.426 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.426 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:22.426 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965927+0000", 2024-09-21T21:43:22.426 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0346999999999998e-05 2024-09-21T21:43:22.426 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.426 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.426 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:22.426 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965981+0000", 2024-09-21T21:43:22.426 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4317999999999997e-05 2024-09-21T21:43:22.426 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:22.426 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:22.426 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:22.426 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.426 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.427 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6997 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5412864~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:22.427 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923174+0000", 2024-09-21T21:43:22.427 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 112.44973663, 2024-09-21T21:43:22.427 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042928296999999997, 2024-09-21T21:43:22.427 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:22.427 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:22.427 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:22.427 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:22.427 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:22.427 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:22.427 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6997 2024-09-21T21:43:22.427 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.427 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:22.427 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.428 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:22.428 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923174+0000", 2024-09-21T21:43:22.428 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:22.428 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.428 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.428 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:22.428 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923174+0000", 2024-09-21T21:43:22.428 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:22.428 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.428 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.428 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:22.428 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923174+0000", 2024-09-21T21:43:22.429 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.67e-07 2024-09-21T21:43:22.429 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.429 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.429 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:22.429 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923178+0000", 2024-09-21T21:43:22.429 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1330000000000001e-06 2024-09-21T21:43:22.429 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.429 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.429 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:22.429 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923182+0000", 2024-09-21T21:43:22.429 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.794e-06 2024-09-21T21:43:22.429 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.429 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.429 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:22.429 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923184+0000", 2024-09-21T21:43:22.430 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.435e-06 2024-09-21T21:43:22.430 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.430 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.430 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:22.430 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966009+0000", 2024-09-21T21:43:22.430 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042824327000000002 2024-09-21T21:43:22.430 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.430 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.430 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:22.430 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966052+0000", 2024-09-21T21:43:22.430 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3034000000000002e-05 2024-09-21T21:43:22.430 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.430 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.430 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:22.431 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966102+0000", 2024-09-21T21:43:22.431 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0306999999999998e-05 2024-09-21T21:43:22.431 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:22.431 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:22.431 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:22.431 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.431 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.431 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6999 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5427200~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:22.431 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923187+0000", 2024-09-21T21:43:22.431 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 112.449722902, 2024-09-21T21:43:22.431 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043104267000000002, 2024-09-21T21:43:22.431 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:22.431 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:22.431 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:22.431 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:22.432 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:22.432 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:22.432 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6999 2024-09-21T21:43:22.432 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.432 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:22.432 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.432 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:22.432 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923187+0000", 2024-09-21T21:43:22.432 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:22.432 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.432 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.432 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:22.432 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923187+0000", 2024-09-21T21:43:22.432 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:22.432 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.433 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.433 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:22.433 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923188+0000", 2024-09-21T21:43:22.433 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3999999999999998e-07 2024-09-21T21:43:22.433 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.433 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.433 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:22.433 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923189+0000", 2024-09-21T21:43:22.433 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4169999999999999e-06 2024-09-21T21:43:22.433 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.433 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.433 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:22.433 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923194+0000", 2024-09-21T21:43:22.433 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1819999999999997e-06 2024-09-21T21:43:22.433 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.434 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.434 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:22.434 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923198+0000", 2024-09-21T21:43:22.434 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.143e-06 2024-09-21T21:43:22.434 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.434 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.434 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:22.434 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966135+0000", 2024-09-21T21:43:22.434 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042936297999999998 2024-09-21T21:43:22.434 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.434 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.434 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:22.434 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966204+0000", 2024-09-21T21:43:22.434 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8989e-05 2024-09-21T21:43:22.435 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.435 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.435 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:22.435 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966292+0000", 2024-09-21T21:43:22.435 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7997999999999993e-05 2024-09-21T21:43:22.435 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:22.435 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:22.435 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:22.435 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.435 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.435 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7001 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5441536~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:22.435 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923211+0000", 2024-09-21T21:43:22.435 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 112.449699111, 2024-09-21T21:43:22.435 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043196518000000003, 2024-09-21T21:43:22.435 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:22.436 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:22.436 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:22.436 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:22.436 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:22.436 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:22.436 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7001 2024-09-21T21:43:22.436 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.436 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:22.436 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.436 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:22.436 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923211+0000", 2024-09-21T21:43:22.436 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:22.436 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.436 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.436 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:22.437 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923211+0000", 2024-09-21T21:43:22.437 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:22.437 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.437 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.437 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:22.437 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923212+0000", 2024-09-21T21:43:22.437 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3900000000000005e-07 2024-09-21T21:43:22.437 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.437 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.437 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:22.437 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923214+0000", 2024-09-21T21:43:22.437 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9970000000000001e-06 2024-09-21T21:43:22.437 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.437 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.437 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:22.438 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923219+0000", 2024-09-21T21:43:22.438 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4310000000000002e-06 2024-09-21T21:43:22.438 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.438 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.438 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:22.438 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923224+0000", 2024-09-21T21:43:22.438 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6410000000000001e-06 2024-09-21T21:43:22.438 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.438 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.438 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:22.438 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966318+0000", 2024-09-21T21:43:22.438 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043093842 2024-09-21T21:43:22.438 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.438 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.439 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:22.439 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966359+0000", 2024-09-21T21:43:22.439 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1260999999999997e-05 2024-09-21T21:43:22.439 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.439 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.439 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:22.439 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966408+0000", 2024-09-21T21:43:22.439 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8807000000000002e-05 2024-09-21T21:43:22.439 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:22.439 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:22.439 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:22.439 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.439 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.439 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7002 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5455872~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:22.439 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923229+0000", 2024-09-21T21:43:22.440 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 112.44968070100001, 2024-09-21T21:43:22.440 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043364568999999999, 2024-09-21T21:43:22.440 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:22.440 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:22.440 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:22.440 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:22.440 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:22.440 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:22.440 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7002 2024-09-21T21:43:22.440 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.440 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:22.440 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.440 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:22.440 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923229+0000", 2024-09-21T21:43:22.440 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:22.441 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.441 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.441 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:22.441 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923229+0000", 2024-09-21T21:43:22.441 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:22.441 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.441 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.441 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:22.441 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923230+0000", 2024-09-21T21:43:22.441 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4700000000000002e-07 2024-09-21T21:43:22.441 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.441 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.441 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:22.441 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923232+0000", 2024-09-21T21:43:22.442 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0719999999999998e-06 2024-09-21T21:43:22.442 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.442 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.442 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:22.442 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923237+0000", 2024-09-21T21:43:22.442 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2290000000000002e-06 2024-09-21T21:43:22.442 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.442 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.442 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:22.442 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923245+0000", 2024-09-21T21:43:22.442 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2959999999999999e-06 2024-09-21T21:43:22.442 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.442 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.442 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:22.442 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966438+0000", 2024-09-21T21:43:22.443 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043193027000000002 2024-09-21T21:43:22.443 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.443 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.443 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:22.443 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966506+0000", 2024-09-21T21:43:22.443 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7707999999999996e-05 2024-09-21T21:43:22.443 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.443 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.443 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:22.443 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966594+0000", 2024-09-21T21:43:22.443 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.789e-05 2024-09-21T21:43:22.443 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:22.443 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:22.443 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:22.444 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.444 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.444 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7005 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5470208~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:22.444 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923254+0000", 2024-09-21T21:43:22.444 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 112.44965646999999, 2024-09-21T21:43:22.444 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043457176, 2024-09-21T21:43:22.444 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:22.444 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:22.444 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:22.444 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:22.444 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:22.444 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:22.444 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7005 2024-09-21T21:43:22.445 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.445 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:22.445 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.445 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:22.445 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923254+0000", 2024-09-21T21:43:22.445 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:22.445 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.445 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.445 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:22.445 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923254+0000", 2024-09-21T21:43:22.445 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:22.445 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.445 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.445 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:22.445 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923254+0000", 2024-09-21T21:43:22.446 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4700000000000002e-07 2024-09-21T21:43:22.446 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.446 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.446 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:22.446 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923256+0000", 2024-09-21T21:43:22.446 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.007e-06 2024-09-21T21:43:22.446 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.446 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.446 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:22.446 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923266+0000", 2024-09-21T21:43:22.446 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.4700000000000008e-06 2024-09-21T21:43:22.446 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.446 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.446 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:22.447 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923271+0000", 2024-09-21T21:43:22.447 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2809999999999999e-06 2024-09-21T21:43:22.447 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.447 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.447 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:22.447 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966619+0000", 2024-09-21T21:43:22.447 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043347951000000003 2024-09-21T21:43:22.447 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.447 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.447 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:22.447 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966661+0000", 2024-09-21T21:43:22.447 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2614999999999998e-05 2024-09-21T21:43:22.447 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.447 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.448 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:22.448 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966711+0000", 2024-09-21T21:43:22.448 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9505000000000001e-05 2024-09-21T21:43:22.448 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:22.448 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:22.448 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:22.448 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.448 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.448 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7008 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5484544~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:22.448 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923278+0000", 2024-09-21T21:43:22.448 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 112.449631823, 2024-09-21T21:43:22.448 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043631071, 2024-09-21T21:43:22.448 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:22.448 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:22.448 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:22.449 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:22.449 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:22.449 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:22.449 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7008 2024-09-21T21:43:22.449 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.449 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:22.449 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.449 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:22.449 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923278+0000", 2024-09-21T21:43:22.449 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:22.449 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.449 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.449 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:22.449 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923278+0000", 2024-09-21T21:43:22.449 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:22.450 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.450 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.450 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:22.450 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923279+0000", 2024-09-21T21:43:22.450 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.77e-07 2024-09-21T21:43:22.450 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.450 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.450 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:22.450 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923281+0000", 2024-09-21T21:43:22.450 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0219999999999999e-06 2024-09-21T21:43:22.450 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.450 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.450 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:22.450 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923287+0000", 2024-09-21T21:43:22.451 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7799999999999997e-06 2024-09-21T21:43:22.451 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.451 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.451 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:22.451 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923291+0000", 2024-09-21T21:43:22.451 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1699999999999999e-06 2024-09-21T21:43:22.451 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.451 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.451 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:22.451 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966742+0000", 2024-09-21T21:43:22.451 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043451344000000003 2024-09-21T21:43:22.451 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.451 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.451 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:22.451 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966809+0000", 2024-09-21T21:43:22.452 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7070999999999995e-05 2024-09-21T21:43:22.452 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.452 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.452 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:22.452 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966909+0000", 2024-09-21T21:43:22.452 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000100307 2024-09-21T21:43:22.452 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:22.452 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:22.452 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:22.452 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.452 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.452 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7010 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5498880~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:22.452 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923301+0000", 2024-09-21T21:43:22.452 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 112.449608928, 2024-09-21T21:43:22.452 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043717830999999999, 2024-09-21T21:43:22.452 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:22.453 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:22.453 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:22.453 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:22.453 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:22.453 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:22.453 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7010 2024-09-21T21:43:22.453 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.453 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:22.453 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.453 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:22.453 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923301+0000", 2024-09-21T21:43:22.453 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:22.453 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.453 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.454 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:22.454 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923301+0000", 2024-09-21T21:43:22.454 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:22.454 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.454 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.454 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:22.454 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923302+0000", 2024-09-21T21:43:22.454 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.41e-07 2024-09-21T21:43:22.454 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.454 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.454 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:22.454 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923304+0000", 2024-09-21T21:43:22.454 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0990000000000001e-06 2024-09-21T21:43:22.454 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.455 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.455 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:22.455 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923310+0000", 2024-09-21T21:43:22.455 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3860000000000004e-06 2024-09-21T21:43:22.455 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.455 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.455 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:22.455 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923315+0000", 2024-09-21T21:43:22.455 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8459999999999999e-06 2024-09-21T21:43:22.455 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.455 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.455 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:22.455 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966938+0000", 2024-09-21T21:43:22.455 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043622693999999997 2024-09-21T21:43:22.456 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.456 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.456 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:22.456 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966975+0000", 2024-09-21T21:43:22.456 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7608999999999999e-05 2024-09-21T21:43:22.456 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.456 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.456 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:22.456 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.967019+0000", 2024-09-21T21:43:22.456 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3856000000000002e-05 2024-09-21T21:43:22.456 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:22.456 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:22.456 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:22.456 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.456 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.457 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5215.0:1054 3.9 3:946d2401:::rbd_data.1121524d2522.000000000000000b:head [set-alloc-hint object_size 16777216 write_size 16777216,write 6615040~59392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-21T21:43:22.457 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:38.148979+0000", 2024-09-21T21:43:22.457 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 104.22393110599999, 2024-09-21T21:43:22.457 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047870388999999999, 2024-09-21T21:43:22.457 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:22.457 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:22.457 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:22.457 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:22.457 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5215", 2024-09-21T21:43:22.457 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1580282168", 2024-09-21T21:43:22.457 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1054 2024-09-21T21:43:22.457 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.457 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:22.457 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.458 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:22.458 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.148979+0000", 2024-09-21T21:43:22.458 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:22.458 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.458 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.458 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:22.458 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.148979+0000", 2024-09-21T21:43:22.458 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:22.458 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.458 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.458 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:22.458 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.148980+0000", 2024-09-21T21:43:22.458 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5799999999999999e-07 2024-09-21T21:43:22.458 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.458 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.459 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:22.459 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.149039+0000", 2024-09-21T21:43:22.459 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9151000000000003e-05 2024-09-21T21:43:22.459 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.459 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.459 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:22.459 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.149047+0000", 2024-09-21T21:43:22.459 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1000000000000004e-06 2024-09-21T21:43:22.459 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.459 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.459 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:22.459 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.149052+0000", 2024-09-21T21:43:22.459 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5959999999999997e-06 2024-09-21T21:43:22.459 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.459 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.460 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:22.460 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.195288+0000", 2024-09-21T21:43:22.460 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.046236248000000001 2024-09-21T21:43:22.460 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.460 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.460 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:22.460 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.195334+0000", 2024-09-21T21:43:22.460 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6575e-05 2024-09-21T21:43:22.460 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.460 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.460 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 2", 2024-09-21T21:43:22.460 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.195406+0000", 2024-09-21T21:43:22.460 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1715e-05 2024-09-21T21:43:22.460 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.460 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.461 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:22.461 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.196018+0000", 2024-09-21T21:43:22.461 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00061218999999999998 2024-09-21T21:43:22.461 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.461 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.461 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:22.461 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.196794+0000", 2024-09-21T21:43:22.461 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000775814 2024-09-21T21:43:22.461 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.461 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.461 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:22.461 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.196827+0000", 2024-09-21T21:43:22.461 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2419000000000003e-05 2024-09-21T21:43:22.461 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.462 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.462 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:22.462 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.196849+0000", 2024-09-21T21:43:22.462 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2923000000000001e-05 2024-09-21T21:43:22.462 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:22.462 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:22.462 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:22.462 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.462 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.462 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4385.0:31260 3.3 3:c4c92e5a:::rbd_trash:head [call rbd.trash_add in=12b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e155)", 2024-09-21T21:43:22.462 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:43:17.285720+0000", 2024-09-21T21:43:22.462 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 5.087189875, 2024-09-21T21:43:22.462 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.945360706, 2024-09-21T21:43:22.462 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:22.462 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:22.463 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:22.463 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:22.463 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4385", 2024-09-21T21:43:22.463 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:43:22.463 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 31260 2024-09-21T21:43:22.463 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.463 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:22.463 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.463 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:22.463 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:43:17.285720+0000", 2024-09-21T21:43:22.463 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:22.463 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.463 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.463 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:22.463 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:43:17.285720+0000", 2024-09-21T21:43:22.464 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:22.464 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.464 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.464 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:22.464 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:43:17.285721+0000", 2024-09-21T21:43:22.464 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1540000000000001e-06 2024-09-21T21:43:22.464 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.464 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.464 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:22.464 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:43:17.285728+0000", 2024-09-21T21:43:22.464 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5899999999999996e-06 2024-09-21T21:43:22.464 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.464 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.464 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:22.464 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:43:17.285740+0000", 2024-09-21T21:43:22.465 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1539000000000001e-05 2024-09-21T21:43:22.465 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.465 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.465 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:22.465 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:43:17.285746+0000", 2024-09-21T21:43:22.465 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.849e-06 2024-09-21T21:43:22.465 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.465 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.465 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:22.465 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:43:17.285796+0000", 2024-09-21T21:43:22.465 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9678000000000001e-05 2024-09-21T21:43:22.465 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.465 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.465 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:22.465 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:43:17.285805+0000", 2024-09-21T21:43:22.466 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5320000000000006e-06 2024-09-21T21:43:22.466 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.466 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.466 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:22.466 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:43:18.228162+0000", 2024-09-21T21:43:22.466 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.94235735499999995 2024-09-21T21:43:22.466 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.466 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.466 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:22.466 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:43:18.228263+0000", 2024-09-21T21:43:22.466 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010129 2024-09-21T21:43:22.466 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.466 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.466 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:43:22.467 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:43:18.228409+0000", 2024-09-21T21:43:22.467 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000145778 2024-09-21T21:43:22.467 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.467 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.467 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:22.467 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:43:18.229542+0000", 2024-09-21T21:43:22.467 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001133046 2024-09-21T21:43:22.467 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.467 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.467 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:22.467 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:43:18.230991+0000", 2024-09-21T21:43:22.467 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001448866 2024-09-21T21:43:22.467 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.467 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.467 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:22.467 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:43:18.231040+0000", 2024-09-21T21:43:22.468 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9515999999999998e-05 2024-09-21T21:43:22.468 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:22.468 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:22.468 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:22.468 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:43:18.231081+0000", 2024-09-21T21:43:22.468 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0513000000000003e-05 2024-09-21T21:43:22.468 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:22.468 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:22.468 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:22.468 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:22.468 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:22.468 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:43:22.470 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T21:43:22.470 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:43:22.468+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:43:22.570 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:43:22.569+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:43:22.613 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:43:22.613 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:43:22.614 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-21T21:43:22.614 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:43:22.619 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T21:43:22.622 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T21:43:22.670 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:43:22.669+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:43:22.725 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:43:22.726 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:43:22.726 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:43:22.726 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:43:22.726 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:43:22.734 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T21:43:22.739 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0: {} 2024-09-21T21:43:22.739 INFO:teuthology.orchestra.run.smithi081.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T21:43:22.755 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1: {} 2024-09-21T21:43:22.756 INFO:teuthology.orchestra.run.smithi081.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T21:43:22.771 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2: {} 2024-09-21T21:43:22.772 INFO:teuthology.orchestra.run.smithi081.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T21:43:22.775 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:43:22.774+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:43:22.788 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3: {} 2024-09-21T21:43:22.788 INFO:teuthology.orchestra.run.smithi081.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T21:43:22.804 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4: {} 2024-09-21T21:43:22.804 INFO:teuthology.orchestra.run.smithi081.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T21:43:22.820 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5: {} 2024-09-21T21:43:22.820 INFO:teuthology.orchestra.run.smithi081.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T21:43:22.873 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:43:22.872+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:43:22.878 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:22.878 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:22.887 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T21:43:22.974 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:43:22.972+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:43:22.990 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:22.990 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:23.000 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T21:43:23.074 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:43:23.073+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:43:23.142 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:23.142 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:23.150 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T21:43:23.175 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:43:23.173+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:43:23.257 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:23.258 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:23.269 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T21:43:23.306 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-21T21:43:23.312 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:43:23.311+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:43:23.376 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:43:23.375+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:43:23.382 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:23.382 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:23.391 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T21:43:23.477 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:43:23.476+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:43:23.536 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:23.536 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:23.545 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T21:43:23.578 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:43:23.577+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:43:23.648 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:23.648 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:23.657 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T21:43:23.700 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:43:23.699+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:43:23.759 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:23.759 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:23.768 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T21:43:23.779 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:43:23.778+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:43:23.880 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:43:23.878+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:43:23.910 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:23.911 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:23.919 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T21:43:24.003 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:43:24.002+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:43:24.062 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:24.062 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:24.070 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T21:43:24.111 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:43:24.110+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:43:24.176 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:24.177 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:24.182 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:43:24.180+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:43:24.185 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T21:43:24.255 INFO:teuthology.orchestra.run.smithi081.stderr:noscrub is set 2024-09-21T21:43:24.282 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:43:24.281+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:43:24.293 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:24.293 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:24.303 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T21:43:24.388 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:43:24.387+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:43:24.456 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:24.456 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:24.464 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T21:43:24.505 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:43:24.504+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:43:24.585 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:43:24.584+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:43:24.607 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:24.607 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:24.616 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T21:43:24.686 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:43:24.685+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:43:24.718 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:24.718 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:24.727 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T21:43:24.787 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:43:24.785+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:43:24.829 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:24.829 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:24.832 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T21:43:24.838 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T21:43:24.888 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:43:24.887+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:43:24.952 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0: {} 2024-09-21T21:43:24.952 INFO:teuthology.orchestra.run.smithi081.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T21:43:24.968 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1: {} 2024-09-21T21:43:24.969 INFO:teuthology.orchestra.run.smithi081.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T21:43:24.983 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:24.983 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:24.984 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2: {} 2024-09-21T21:43:24.984 INFO:teuthology.orchestra.run.smithi081.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T21:43:24.989 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:43:24.988+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:43:24.992 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T21:43:25.001 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3: {} 2024-09-21T21:43:25.001 INFO:teuthology.orchestra.run.smithi081.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T21:43:25.017 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4: {} 2024-09-21T21:43:25.017 INFO:teuthology.orchestra.run.smithi081.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T21:43:25.034 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5: {} 2024-09-21T21:43:25.034 INFO:teuthology.orchestra.run.smithi081.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T21:43:25.091 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:43:25.089+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:43:25.095 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:43:25.095 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:43:25.095 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:43:25.095 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:43:25.095 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:43:25.104 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T21:43:25.192 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:43:25.190+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:43:25.250 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:43:25.250 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-21T21:43:25.250 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-21T21:43:25.250 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:43:25.250 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.250 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4385.0:15202 3.16 3:6888948b:::rbd_header.1121493f6459:head [watch reconnect cookie 140325440202496 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-21T21:43:25.251 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:41.736601+0000", 2024-09-21T21:43:25.251 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 103.510860678, 2024-09-21T21:43:25.251 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5040056769999999, 2024-09-21T21:43:25.251 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:25.251 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:25.251 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:25.251 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:25.251 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4385", 2024-09-21T21:43:25.251 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:43:25.251 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 15202 2024-09-21T21:43:25.251 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.251 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:25.251 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.251 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:25.251 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736601+0000", 2024-09-21T21:43:25.252 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:25.252 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.252 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.252 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:25.252 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736601+0000", 2024-09-21T21:43:25.252 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:25.252 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.252 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.252 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:25.252 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736602+0000", 2024-09-21T21:43:25.252 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0699999999999999e-06 2024-09-21T21:43:25.252 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.252 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.252 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:25.253 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736607+0000", 2024-09-21T21:43:25.253 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2520000000000001e-06 2024-09-21T21:43:25.253 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.253 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.253 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:25.253 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736617+0000", 2024-09-21T21:43:25.253 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0074e-05 2024-09-21T21:43:25.253 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.253 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.253 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:25.253 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736622+0000", 2024-09-21T21:43:25.253 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.101e-06 2024-09-21T21:43:25.253 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.253 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.253 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:25.254 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736664+0000", 2024-09-21T21:43:25.254 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1869999999999997e-05 2024-09-21T21:43:25.254 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.254 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.254 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op must wait for map", 2024-09-21T21:43:25.254 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736667+0000", 2024-09-21T21:43:25.254 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4750000000000002e-06 2024-09-21T21:43:25.254 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.254 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.254 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:25.254 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742886+0000", 2024-09-21T21:43:25.254 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0062187579999999996 2024-09-21T21:43:25.255 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.255 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.255 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:25.255 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742892+0000", 2024-09-21T21:43:25.255 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0440000000000001e-06 2024-09-21T21:43:25.255 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.255 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.255 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:25.255 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210307+0000", 2024-09-21T21:43:25.255 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.467414781 2024-09-21T21:43:25.255 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.255 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.255 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:25.255 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210322+0000", 2024-09-21T21:43:25.256 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5311e-05 2024-09-21T21:43:25.256 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.256 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.256 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:25.256 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.237624+0000", 2024-09-21T21:43:25.256 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0273018920000001 2024-09-21T21:43:25.256 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.256 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.256 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:25.256 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.237870+0000", 2024-09-21T21:43:25.256 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00024622899999999999 2024-09-21T21:43:25.256 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.256 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.256 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:25.256 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.240607+0000", 2024-09-21T21:43:25.257 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0027368200000000001 2024-09-21T21:43:25.257 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:25.257 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:25.257 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:25.257 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.257 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.257 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5415.0:379 3.16 3:6888948b:::rbd_header.1121493f6459:head [watch reconnect cookie 139702602617920 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-21T21:43:25.257 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:41.739544+0000", 2024-09-21T21:43:25.257 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 103.507918249, 2024-09-21T21:43:25.257 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.501180645, 2024-09-21T21:43:25.257 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:25.257 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:25.257 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:25.257 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:25.258 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5415", 2024-09-21T21:43:25.258 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1212816607", 2024-09-21T21:43:25.258 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 379 2024-09-21T21:43:25.258 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.258 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:25.258 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.258 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:25.258 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739544+0000", 2024-09-21T21:43:25.258 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:25.258 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.258 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.258 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:25.258 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739544+0000", 2024-09-21T21:43:25.258 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:25.259 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.259 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.259 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:25.259 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739545+0000", 2024-09-21T21:43:25.259 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.065e-06 2024-09-21T21:43:25.259 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.259 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.259 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:25.259 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739550+0000", 2024-09-21T21:43:25.259 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9599999999999999e-06 2024-09-21T21:43:25.259 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.259 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.259 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:25.259 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739565+0000", 2024-09-21T21:43:25.259 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5709000000000002e-05 2024-09-21T21:43:25.260 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.260 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.260 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:25.260 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739573+0000", 2024-09-21T21:43:25.260 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7849999999999992e-06 2024-09-21T21:43:25.260 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.260 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.260 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:25.260 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739634+0000", 2024-09-21T21:43:25.260 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0732000000000001e-05 2024-09-21T21:43:25.260 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.260 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.260 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op must wait for map", 2024-09-21T21:43:25.260 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739640+0000", 2024-09-21T21:43:25.261 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0000000000000002e-06 2024-09-21T21:43:25.261 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.261 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.261 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:25.261 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742958+0000", 2024-09-21T21:43:25.261 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0033182390000000002 2024-09-21T21:43:25.261 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.261 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.261 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:25.261 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742966+0000", 2024-09-21T21:43:25.261 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1610000000000003e-06 2024-09-21T21:43:25.261 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.261 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.261 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:25.261 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210405+0000", 2024-09-21T21:43:25.262 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.467438725 2024-09-21T21:43:25.262 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.262 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.262 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:25.262 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210415+0000", 2024-09-21T21:43:25.262 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0234000000000001e-05 2024-09-21T21:43:25.262 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.262 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.262 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:25.262 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.238743+0000", 2024-09-21T21:43:25.262 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0283273070000001 2024-09-21T21:43:25.262 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.262 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.262 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:25.263 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.238842+0000", 2024-09-21T21:43:25.263 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.9896999999999996e-05 2024-09-21T21:43:25.263 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.263 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.263 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:25.263 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.240724+0000", 2024-09-21T21:43:25.263 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001881831 2024-09-21T21:43:25.263 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:25.263 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:25.263 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:25.263 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.263 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.263 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4385.0:15203 3.16 3:6888948b:::rbd_header.1121493f6459:head [call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-21T21:43:25.263 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:41.739737+0000", 2024-09-21T21:43:25.263 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 103.50772523800001, 2024-09-21T21:43:25.264 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4989621799999999, 2024-09-21T21:43:25.264 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:25.264 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:25.264 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:25.264 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:25.264 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4385", 2024-09-21T21:43:25.264 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:43:25.264 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 15203 2024-09-21T21:43:25.264 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.264 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:25.264 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.264 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:25.264 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739737+0000", 2024-09-21T21:43:25.264 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:25.264 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.265 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.265 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:25.265 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739737+0000", 2024-09-21T21:43:25.265 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:25.265 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.265 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.265 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:25.265 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739738+0000", 2024-09-21T21:43:25.265 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.7499999999999998e-07 2024-09-21T21:43:25.265 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.265 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.265 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:25.265 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739743+0000", 2024-09-21T21:43:25.265 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5049999999999998e-06 2024-09-21T21:43:25.265 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.266 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.266 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:25.266 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739757+0000", 2024-09-21T21:43:25.266 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3947e-05 2024-09-21T21:43:25.266 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.266 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.266 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:25.266 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739765+0000", 2024-09-21T21:43:25.266 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9310000000000008e-06 2024-09-21T21:43:25.266 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.266 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.266 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:25.266 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739816+0000", 2024-09-21T21:43:25.266 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0559999999999997e-05 2024-09-21T21:43:25.267 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.267 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.267 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting_for_map not empty", 2024-09-21T21:43:25.267 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739821+0000", 2024-09-21T21:43:25.267 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3539999999999999e-06 2024-09-21T21:43:25.267 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.267 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.267 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:25.267 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742928+0000", 2024-09-21T21:43:25.267 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0031074399999999999 2024-09-21T21:43:25.267 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.267 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.267 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:25.267 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742937+0000", 2024-09-21T21:43:25.268 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1289999999999996e-06 2024-09-21T21:43:25.268 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.268 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.268 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:25.268 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210350+0000", 2024-09-21T21:43:25.268 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.46741311299999999 2024-09-21T21:43:25.268 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.268 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.268 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:25.268 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210358+0000", 2024-09-21T21:43:25.268 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1869999999999997e-06 2024-09-21T21:43:25.268 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.268 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.268 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:25.268 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.238398+0000", 2024-09-21T21:43:25.268 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0280397830000001 2024-09-21T21:43:25.269 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.269 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.269 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:25.269 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.238489+0000", 2024-09-21T21:43:25.269 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.0958000000000006e-05 2024-09-21T21:43:25.269 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.269 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.269 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:25.269 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.238699+0000", 2024-09-21T21:43:25.269 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00021029800000000001 2024-09-21T21:43:25.269 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:25.269 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:25.269 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:25.269 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.269 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.270 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4385.0:15204 3.16 3:6888948b:::rbd_header.1121493f6459:head [watch reconnect cookie 140325440202496 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-21T21:43:25.270 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:42.210726+0000", 2024-09-21T21:43:25.270 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 103.03673636000001, 2024-09-21T21:43:25.270 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.030304557, 2024-09-21T21:43:25.270 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:25.270 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:25.270 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:25.270 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:25.270 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4385", 2024-09-21T21:43:25.270 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:43:25.270 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 15204 2024-09-21T21:43:25.270 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.270 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:25.271 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.271 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:25.271 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210726+0000", 2024-09-21T21:43:25.271 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:25.271 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.271 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.271 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:25.271 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210726+0000", 2024-09-21T21:43:25.271 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:25.271 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.271 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.271 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:25.271 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210727+0000", 2024-09-21T21:43:25.271 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.204e-06 2024-09-21T21:43:25.271 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.271 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.272 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:25.272 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210732+0000", 2024-09-21T21:43:25.272 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5249999999999996e-06 2024-09-21T21:43:25.272 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.272 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.272 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:25.272 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210744+0000", 2024-09-21T21:43:25.272 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1405e-05 2024-09-21T21:43:25.272 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.272 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.272 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:25.272 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210754+0000", 2024-09-21T21:43:25.272 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0122999999999999e-05 2024-09-21T21:43:25.272 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.272 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.273 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:25.273 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210798+0000", 2024-09-21T21:43:25.273 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4005000000000003e-05 2024-09-21T21:43:25.273 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.273 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.273 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:25.273 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210803+0000", 2024-09-21T21:43:25.273 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5130000000000004e-06 2024-09-21T21:43:25.273 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.273 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.273 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:25.273 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.239260+0000", 2024-09-21T21:43:25.273 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0284563040000001 2024-09-21T21:43:25.273 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.274 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.274 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:25.274 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.239318+0000", 2024-09-21T21:43:25.274 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8486000000000002e-05 2024-09-21T21:43:25.274 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.274 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.274 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:25.274 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.241030+0000", 2024-09-21T21:43:25.274 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0017119920000000001 2024-09-21T21:43:25.274 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:25.274 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:25.274 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:25.274 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.274 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.275 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5415.0:380 3.16 3:6888948b:::rbd_header.1121493f6459:head [watch reconnect cookie 139702602617920 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-21T21:43:25.275 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:42.210777+0000", 2024-09-21T21:43:25.275 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 103.036684581, 2024-09-21T21:43:25.275 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.030410566, 2024-09-21T21:43:25.275 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:25.275 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:25.275 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:25.275 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:25.275 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5415", 2024-09-21T21:43:25.275 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1212816607", 2024-09-21T21:43:25.275 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 380 2024-09-21T21:43:25.275 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.275 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:25.275 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.275 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:25.275 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210777+0000", 2024-09-21T21:43:25.276 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:25.276 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.276 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.276 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:25.276 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210777+0000", 2024-09-21T21:43:25.276 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:25.276 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.276 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.276 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:25.276 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210778+0000", 2024-09-21T21:43:25.276 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8999999999999996e-07 2024-09-21T21:43:25.277 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.277 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.277 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:25.277 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210780+0000", 2024-09-21T21:43:25.277 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.503e-06 2024-09-21T21:43:25.277 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.277 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.277 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:25.277 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210788+0000", 2024-09-21T21:43:25.277 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0999999999999998e-06 2024-09-21T21:43:25.277 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.277 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.277 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:25.278 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210799+0000", 2024-09-21T21:43:25.278 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1703e-05 2024-09-21T21:43:25.278 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.278 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.278 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:25.278 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210833+0000", 2024-09-21T21:43:25.278 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4008999999999999e-05 2024-09-21T21:43:25.278 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.278 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.278 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:25.278 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210838+0000", 2024-09-21T21:43:25.278 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7480000000000001e-06 2024-09-21T21:43:25.278 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.278 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.279 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:25.279 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.239612+0000", 2024-09-21T21:43:25.279 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.028773554 2024-09-21T21:43:25.279 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.279 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.279 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:25.279 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.239677+0000", 2024-09-21T21:43:25.279 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5053000000000002e-05 2024-09-21T21:43:25.279 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.279 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.279 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:25.279 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.241188+0000", 2024-09-21T21:43:25.279 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001511306 2024-09-21T21:43:25.279 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:25.280 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:25.280 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:25.280 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.280 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.280 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24794 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:25.280 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.584085+0000", 2024-09-21T21:43:25.280 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 62.663377404999999, 2024-09-21T21:43:25.280 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.096209996000000006, 2024-09-21T21:43:25.280 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:25.280 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:25.280 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:25.280 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:25.280 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:25.280 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:25.281 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24794 2024-09-21T21:43:25.281 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.281 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:25.281 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.281 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:25.281 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584085+0000", 2024-09-21T21:43:25.281 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:25.281 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.281 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.281 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:25.281 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584085+0000", 2024-09-21T21:43:25.281 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:25.281 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.282 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.282 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:25.282 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584085+0000", 2024-09-21T21:43:25.282 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3200000000000001e-07 2024-09-21T21:43:25.282 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.282 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.282 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:25.282 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584086+0000", 2024-09-21T21:43:25.282 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.717e-06 2024-09-21T21:43:25.282 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.282 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.282 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:25.282 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584090+0000", 2024-09-21T21:43:25.283 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3450000000000002e-06 2024-09-21T21:43:25.283 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.283 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.283 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:25.283 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584092+0000", 2024-09-21T21:43:25.283 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6359999999999998e-06 2024-09-21T21:43:25.283 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.283 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.283 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:25.283 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.676497+0000", 2024-09-21T21:43:25.283 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.092404659 2024-09-21T21:43:25.283 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.283 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.284 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:25.284 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.676567+0000", 2024-09-21T21:43:25.284 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9627000000000003e-05 2024-09-21T21:43:25.284 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.284 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.284 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:25.284 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.676925+0000", 2024-09-21T21:43:25.284 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00035848500000000002 2024-09-21T21:43:25.284 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.284 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.284 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:25.284 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.678427+0000", 2024-09-21T21:43:25.284 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001501638 2024-09-21T21:43:25.284 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.285 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.285 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:25.285 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.680204+0000", 2024-09-21T21:43:25.285 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0017766979999999999 2024-09-21T21:43:25.285 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.285 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.285 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:25.285 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.680256+0000", 2024-09-21T21:43:25.285 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2549999999999997e-05 2024-09-21T21:43:25.285 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.285 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.285 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:25.285 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.680295+0000", 2024-09-21T21:43:25.285 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8408999999999998e-05 2024-09-21T21:43:25.286 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:25.286 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:25.286 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:25.286 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.286 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.286 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24796 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:25.286 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.584099+0000", 2024-09-21T21:43:25.286 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 62.663362835999997, 2024-09-21T21:43:25.286 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.097297429000000005, 2024-09-21T21:43:25.286 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:25.286 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:25.286 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:25.287 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:25.287 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:25.287 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:25.287 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24796 2024-09-21T21:43:25.287 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.287 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:25.287 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.287 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:25.287 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584099+0000", 2024-09-21T21:43:25.287 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:25.287 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.287 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.287 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:25.287 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584099+0000", 2024-09-21T21:43:25.288 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:25.288 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.288 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.288 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:25.288 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584099+0000", 2024-09-21T21:43:25.288 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3999999999999998e-07 2024-09-21T21:43:25.288 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.288 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.288 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:25.288 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584101+0000", 2024-09-21T21:43:25.288 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5850000000000001e-06 2024-09-21T21:43:25.288 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.288 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.288 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:25.289 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584108+0000", 2024-09-21T21:43:25.289 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.498e-06 2024-09-21T21:43:25.289 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.289 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.289 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:25.289 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584112+0000", 2024-09-21T21:43:25.289 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.252e-06 2024-09-21T21:43:25.289 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.289 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.289 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:25.289 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.677336+0000", 2024-09-21T21:43:25.289 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.093224206000000004 2024-09-21T21:43:25.289 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.290 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.290 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:25.290 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.677411+0000", 2024-09-21T21:43:25.290 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5036000000000005e-05 2024-09-21T21:43:25.290 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.290 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.290 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:25.290 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.677782+0000", 2024-09-21T21:43:25.290 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00037153099999999998 2024-09-21T21:43:25.290 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.290 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.290 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:25.290 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.680360+0000", 2024-09-21T21:43:25.291 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025773020000000001 2024-09-21T21:43:25.291 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.291 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.291 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:25.291 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681313+0000", 2024-09-21T21:43:25.291 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00095341100000000004 2024-09-21T21:43:25.291 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.291 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.291 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:25.291 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681361+0000", 2024-09-21T21:43:25.291 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7497999999999998e-05 2024-09-21T21:43:25.291 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.292 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.292 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:25.292 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681397+0000", 2024-09-21T21:43:25.292 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.587e-05 2024-09-21T21:43:25.292 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:25.292 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:25.292 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:25.292 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.292 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.292 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24797 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:25.292 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.584116+0000", 2024-09-21T21:43:25.292 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 62.663346347000001, 2024-09-21T21:43:25.292 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.097428257000000004, 2024-09-21T21:43:25.292 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:25.292 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:25.293 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:25.293 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:25.293 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:25.293 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:25.293 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24797 2024-09-21T21:43:25.293 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.293 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:25.293 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.293 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:25.293 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584116+0000", 2024-09-21T21:43:25.293 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:25.293 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.293 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.293 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:25.293 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584116+0000", 2024-09-21T21:43:25.294 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:25.294 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.294 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.294 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:25.294 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584116+0000", 2024-09-21T21:43:25.294 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.53e-07 2024-09-21T21:43:25.294 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.294 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.294 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:25.294 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584118+0000", 2024-09-21T21:43:25.294 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6810000000000001e-06 2024-09-21T21:43:25.294 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.294 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.294 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:25.295 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584121+0000", 2024-09-21T21:43:25.295 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1099999999999999e-06 2024-09-21T21:43:25.295 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.295 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.295 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:25.295 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584123+0000", 2024-09-21T21:43:25.295 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5890000000000001e-06 2024-09-21T21:43:25.295 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.295 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.295 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:25.295 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.678476+0000", 2024-09-21T21:43:25.295 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.094352320000000003 2024-09-21T21:43:25.296 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.296 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.296 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:25.296 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.678556+0000", 2024-09-21T21:43:25.296 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0291999999999996e-05 2024-09-21T21:43:25.296 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.296 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.296 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:25.296 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.678871+0000", 2024-09-21T21:43:25.296 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000314689 2024-09-21T21:43:25.296 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.296 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.296 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:25.296 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681452+0000", 2024-09-21T21:43:25.296 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025817890000000001 2024-09-21T21:43:25.297 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.297 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.297 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:25.297 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681467+0000", 2024-09-21T21:43:25.297 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4256e-05 2024-09-21T21:43:25.297 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.297 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.297 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:25.297 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681510+0000", 2024-09-21T21:43:25.297 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3560999999999998e-05 2024-09-21T21:43:25.297 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.297 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.297 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:25.297 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681544+0000", 2024-09-21T21:43:25.297 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3717000000000003e-05 2024-09-21T21:43:25.298 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:25.298 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:25.298 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:25.298 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.298 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.298 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24798 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:25.298 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.584130+0000", 2024-09-21T21:43:25.298 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 62.663331823999997, 2024-09-21T21:43:25.298 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.098350526999999993, 2024-09-21T21:43:25.298 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:25.298 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:25.298 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:25.298 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:25.298 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:25.299 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:25.299 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24798 2024-09-21T21:43:25.299 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.299 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:25.299 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.299 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:25.299 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584130+0000", 2024-09-21T21:43:25.299 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:25.299 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.299 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.299 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:25.299 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584130+0000", 2024-09-21T21:43:25.299 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:25.299 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.300 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.300 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:25.300 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584131+0000", 2024-09-21T21:43:25.300 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4700000000000001e-07 2024-09-21T21:43:25.300 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.300 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.300 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:25.300 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584133+0000", 2024-09-21T21:43:25.300 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.34e-06 2024-09-21T21:43:25.300 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.300 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.300 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:25.300 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584137+0000", 2024-09-21T21:43:25.301 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5609999999999999e-06 2024-09-21T21:43:25.301 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.301 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.301 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:25.301 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584139+0000", 2024-09-21T21:43:25.301 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7350000000000001e-06 2024-09-21T21:43:25.301 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.301 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.301 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:25.301 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.679274+0000", 2024-09-21T21:43:25.301 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.095134719000000006 2024-09-21T21:43:25.301 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.301 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.301 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:25.302 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.679358+0000", 2024-09-21T21:43:25.302 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3609999999999994e-05 2024-09-21T21:43:25.302 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.302 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.302 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:25.302 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.679685+0000", 2024-09-21T21:43:25.302 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00032732399999999999 2024-09-21T21:43:25.302 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.302 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.302 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:25.302 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681553+0000", 2024-09-21T21:43:25.302 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001868059 2024-09-21T21:43:25.302 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.302 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.303 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:25.303 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.682375+0000", 2024-09-21T21:43:25.303 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00082232099999999999 2024-09-21T21:43:25.303 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.303 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.303 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:25.303 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.682431+0000", 2024-09-21T21:43:25.303 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6051999999999998e-05 2024-09-21T21:43:25.303 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.303 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.303 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:25.303 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.682481+0000", 2024-09-21T21:43:25.303 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9258999999999997e-05 2024-09-21T21:43:25.303 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:25.304 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:25.304 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:25.304 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.304 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.304 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24799 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:25.304 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.584159+0000", 2024-09-21T21:43:25.304 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 62.663302836, 2024-09-21T21:43:25.304 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.100021551, 2024-09-21T21:43:25.304 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:25.304 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:25.304 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:25.304 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:25.304 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:25.304 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:25.304 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24799 2024-09-21T21:43:25.305 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.305 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:25.305 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.305 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:25.305 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584159+0000", 2024-09-21T21:43:25.305 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:25.305 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.305 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.305 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:25.305 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584159+0000", 2024-09-21T21:43:25.305 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:25.305 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.305 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.305 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:25.305 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584160+0000", 2024-09-21T21:43:25.306 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1300000000000001e-07 2024-09-21T21:43:25.306 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.306 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.306 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:25.306 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584162+0000", 2024-09-21T21:43:25.306 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5409999999999999e-06 2024-09-21T21:43:25.306 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.306 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.306 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:25.306 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584169+0000", 2024-09-21T21:43:25.306 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7880000000000001e-06 2024-09-21T21:43:25.306 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.306 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.306 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:25.307 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584174+0000", 2024-09-21T21:43:25.307 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9590000000000003e-06 2024-09-21T21:43:25.307 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.307 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.307 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:25.307 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.680400+0000", 2024-09-21T21:43:25.307 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.096226514999999999 2024-09-21T21:43:25.307 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.307 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.307 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:25.307 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.680480+0000", 2024-09-21T21:43:25.307 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9589e-05 2024-09-21T21:43:25.307 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.307 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.307 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:25.308 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.680847+0000", 2024-09-21T21:43:25.308 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00036757700000000001 2024-09-21T21:43:25.308 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.308 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.308 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:25.308 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.682555+0000", 2024-09-21T21:43:25.308 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001707791 2024-09-21T21:43:25.308 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.308 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.308 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:25.308 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684093+0000", 2024-09-21T21:43:25.308 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0015377170000000001 2024-09-21T21:43:25.308 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.308 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.308 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:25.309 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684144+0000", 2024-09-21T21:43:25.309 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1208000000000002e-05 2024-09-21T21:43:25.309 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.309 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.309 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:25.309 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684181+0000", 2024-09-21T21:43:25.309 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6452999999999998e-05 2024-09-21T21:43:25.309 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:25.309 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:25.309 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:25.309 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.309 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.309 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24835 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:25.309 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586332+0000", 2024-09-21T21:43:25.309 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 62.661130221999997, 2024-09-21T21:43:25.310 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.098730360000000003, 2024-09-21T21:43:25.310 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:25.310 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:25.310 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:25.310 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:25.310 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:25.310 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:25.310 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24835 2024-09-21T21:43:25.310 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.310 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:25.310 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.310 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:25.310 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586332+0000", 2024-09-21T21:43:25.311 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:25.311 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.311 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.311 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:25.311 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586332+0000", 2024-09-21T21:43:25.311 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:25.311 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.311 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.311 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:25.311 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586333+0000", 2024-09-21T21:43:25.311 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0200000000000001e-07 2024-09-21T21:43:25.311 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.312 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.312 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:25.312 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586346+0000", 2024-09-21T21:43:25.312 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3745e-05 2024-09-21T21:43:25.312 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.312 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.312 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:25.312 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586366+0000", 2024-09-21T21:43:25.312 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9272000000000001e-05 2024-09-21T21:43:25.312 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.312 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.312 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:25.312 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586377+0000", 2024-09-21T21:43:25.312 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1197e-05 2024-09-21T21:43:25.313 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.313 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.313 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:25.313 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681578+0000", 2024-09-21T21:43:25.313 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.095201420999999994 2024-09-21T21:43:25.313 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.313 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.313 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:25.313 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681640+0000", 2024-09-21T21:43:25.313 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1834000000000005e-05 2024-09-21T21:43:25.313 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.313 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.313 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:25.314 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681915+0000", 2024-09-21T21:43:25.314 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00027534199999999998 2024-09-21T21:43:25.314 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.314 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.314 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:25.314 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684230+0000", 2024-09-21T21:43:25.314 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00231424 2024-09-21T21:43:25.314 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.314 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.314 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:25.314 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684959+0000", 2024-09-21T21:43:25.314 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00072921399999999995 2024-09-21T21:43:25.314 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.315 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.315 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:25.315 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.685008+0000", 2024-09-21T21:43:25.315 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8860999999999999e-05 2024-09-21T21:43:25.315 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.315 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.315 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:25.315 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.685062+0000", 2024-09-21T21:43:25.315 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4432000000000003e-05 2024-09-21T21:43:25.315 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:25.315 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:25.315 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:25.315 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.315 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.316 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24837 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:25.316 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586395+0000", 2024-09-21T21:43:25.316 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 62.661067287000002, 2024-09-21T21:43:25.316 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.099700522, 2024-09-21T21:43:25.316 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:25.316 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:25.316 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:25.316 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:25.316 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:25.316 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:25.316 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24837 2024-09-21T21:43:25.316 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.316 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:25.316 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.317 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:25.317 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586395+0000", 2024-09-21T21:43:25.317 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:25.317 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.317 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.317 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:25.317 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586395+0000", 2024-09-21T21:43:25.317 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:25.317 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.317 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.317 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:25.317 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586395+0000", 2024-09-21T21:43:25.317 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.06e-07 2024-09-21T21:43:25.317 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.317 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.318 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:25.318 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586401+0000", 2024-09-21T21:43:25.318 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0889999999999996e-06 2024-09-21T21:43:25.318 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.318 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.318 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:25.318 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586410+0000", 2024-09-21T21:43:25.318 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.918e-06 2024-09-21T21:43:25.318 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.318 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.318 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:25.318 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586424+0000", 2024-09-21T21:43:25.318 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3478999999999999e-05 2024-09-21T21:43:25.319 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.319 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.319 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:25.319 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.682590+0000", 2024-09-21T21:43:25.319 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.096166086999999997 2024-09-21T21:43:25.319 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.319 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.319 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:25.319 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.682657+0000", 2024-09-21T21:43:25.319 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6878000000000006e-05 2024-09-21T21:43:25.319 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.319 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.320 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:25.320 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.682879+0000", 2024-09-21T21:43:25.320 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00022238 2024-09-21T21:43:25.320 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.320 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.320 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:25.320 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.685073+0000", 2024-09-21T21:43:25.320 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021938119999999998 2024-09-21T21:43:25.320 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.320 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.320 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:25.320 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686038+0000", 2024-09-21T21:43:25.320 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00096539499999999997 2024-09-21T21:43:25.321 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.321 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.321 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:25.321 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686071+0000", 2024-09-21T21:43:25.321 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2514000000000001e-05 2024-09-21T21:43:25.321 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.321 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.321 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:25.321 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686095+0000", 2024-09-21T21:43:25.321 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4363999999999999e-05 2024-09-21T21:43:25.321 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:25.321 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:25.321 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:25.321 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.322 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.322 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24839 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:25.322 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586468+0000", 2024-09-21T21:43:25.322 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 62.660993634999997, 2024-09-21T21:43:25.322 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.099739355000000002, 2024-09-21T21:43:25.322 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:25.322 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:25.322 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:25.322 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:25.322 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:25.322 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:25.322 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24839 2024-09-21T21:43:25.322 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.323 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:25.323 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.323 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:25.323 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586468+0000", 2024-09-21T21:43:25.323 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:25.323 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.323 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.323 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:25.323 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586468+0000", 2024-09-21T21:43:25.323 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:25.323 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.323 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.323 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:25.323 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586469+0000", 2024-09-21T21:43:25.323 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2200000000000004e-07 2024-09-21T21:43:25.324 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.324 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.324 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:25.324 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586472+0000", 2024-09-21T21:43:25.324 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9960000000000002e-06 2024-09-21T21:43:25.324 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.324 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.324 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:25.324 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586479+0000", 2024-09-21T21:43:25.324 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2699999999999999e-06 2024-09-21T21:43:25.324 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.324 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.324 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:25.325 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586485+0000", 2024-09-21T21:43:25.325 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6999999999999996e-06 2024-09-21T21:43:25.325 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.325 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.325 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:25.325 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.683276+0000", 2024-09-21T21:43:25.325 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.096791618999999995 2024-09-21T21:43:25.325 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.325 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.325 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:25.325 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.683326+0000", 2024-09-21T21:43:25.326 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9384e-05 2024-09-21T21:43:25.326 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.326 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.326 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:25.326 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.683655+0000", 2024-09-21T21:43:25.326 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00032965700000000002 2024-09-21T21:43:25.326 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.326 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.326 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:25.326 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686120+0000", 2024-09-21T21:43:25.326 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002464819 2024-09-21T21:43:25.326 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.326 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.327 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:25.327 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686159+0000", 2024-09-21T21:43:25.327 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9181000000000003e-05 2024-09-21T21:43:25.327 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.327 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.327 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:25.327 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686185+0000", 2024-09-21T21:43:25.327 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5907000000000001e-05 2024-09-21T21:43:25.327 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.327 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.327 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:25.327 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686208+0000", 2024-09-21T21:43:25.327 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.23e-05 2024-09-21T21:43:25.327 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:25.328 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:25.328 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:25.328 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.328 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.328 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24841 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:25.328 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586492+0000", 2024-09-21T21:43:25.328 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 62.660969473000002, 2024-09-21T21:43:25.328 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.101672077, 2024-09-21T21:43:25.328 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:25.328 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:25.328 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:25.328 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:25.328 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:25.328 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:25.329 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24841 2024-09-21T21:43:25.329 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.329 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:25.329 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.329 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:25.329 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586492+0000", 2024-09-21T21:43:25.329 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:25.329 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.329 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.329 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:25.329 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586492+0000", 2024-09-21T21:43:25.329 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:25.329 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.329 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.329 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:25.330 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586493+0000", 2024-09-21T21:43:25.330 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5999999999999999e-07 2024-09-21T21:43:25.330 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.330 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.330 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:25.330 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586496+0000", 2024-09-21T21:43:25.330 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6089999999999999e-06 2024-09-21T21:43:25.330 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.330 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.330 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:25.330 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586502+0000", 2024-09-21T21:43:25.330 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6270000000000002e-06 2024-09-21T21:43:25.330 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.331 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.331 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:25.331 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586507+0000", 2024-09-21T21:43:25.331 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8810000000000004e-06 2024-09-21T21:43:25.331 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.331 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.331 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:25.331 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684273+0000", 2024-09-21T21:43:25.331 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.097766415999999995 2024-09-21T21:43:25.331 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.331 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.331 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:25.331 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684359+0000", 2024-09-21T21:43:25.331 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5574000000000004e-05 2024-09-21T21:43:25.332 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.332 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.332 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:25.332 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684568+0000", 2024-09-21T21:43:25.332 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000208529 2024-09-21T21:43:25.332 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.332 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.332 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:25.332 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.687186+0000", 2024-09-21T21:43:25.332 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0026180769999999999 2024-09-21T21:43:25.332 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.332 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.332 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:25.332 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688093+0000", 2024-09-21T21:43:25.333 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00090766999999999996 2024-09-21T21:43:25.333 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.333 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.333 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:25.333 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688142+0000", 2024-09-21T21:43:25.333 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8986000000000002e-05 2024-09-21T21:43:25.333 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.333 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.333 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:25.333 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688165+0000", 2024-09-21T21:43:25.333 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2248000000000001e-05 2024-09-21T21:43:25.333 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:25.333 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:25.333 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:25.334 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.334 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.334 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24843 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:25.334 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586514+0000", 2024-09-21T21:43:25.334 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 62.660948187000002, 2024-09-21T21:43:25.334 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.101826449, 2024-09-21T21:43:25.334 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:25.334 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:25.334 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:25.334 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:25.334 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:25.334 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:25.334 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24843 2024-09-21T21:43:25.334 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.335 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:25.335 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.335 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:25.335 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586514+0000", 2024-09-21T21:43:25.335 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:25.335 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.335 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.335 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:25.335 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586514+0000", 2024-09-21T21:43:25.335 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:25.335 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.335 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.335 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:25.335 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586514+0000", 2024-09-21T21:43:25.336 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9999999999999998e-07 2024-09-21T21:43:25.336 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.336 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.336 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:25.336 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586517+0000", 2024-09-21T21:43:25.336 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4980000000000001e-06 2024-09-21T21:43:25.336 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.336 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.336 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:25.336 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586523+0000", 2024-09-21T21:43:25.336 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4609999999999996e-06 2024-09-21T21:43:25.336 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.336 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.336 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:25.337 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586528+0000", 2024-09-21T21:43:25.337 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2979999999999999e-06 2024-09-21T21:43:25.337 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.337 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.337 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:25.337 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.685104+0000", 2024-09-21T21:43:25.337 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.098575837999999999 2024-09-21T21:43:25.337 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.337 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.337 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:25.337 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.685184+0000", 2024-09-21T21:43:25.337 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9807000000000003e-05 2024-09-21T21:43:25.337 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.337 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.337 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:25.338 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.685496+0000", 2024-09-21T21:43:25.338 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00031225900000000001 2024-09-21T21:43:25.338 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.338 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.338 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:25.338 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688238+0000", 2024-09-21T21:43:25.338 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0027419409999999999 2024-09-21T21:43:25.338 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.338 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.338 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:25.338 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688253+0000", 2024-09-21T21:43:25.338 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4331000000000001e-05 2024-09-21T21:43:25.338 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.338 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.339 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:25.339 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688296+0000", 2024-09-21T21:43:25.339 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3801000000000003e-05 2024-09-21T21:43:25.339 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.339 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.339 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:25.339 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688340+0000", 2024-09-21T21:43:25.339 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3815e-05 2024-09-21T21:43:25.339 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:25.339 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:25.339 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:25.339 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.339 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.339 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24845 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:25.339 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586535+0000", 2024-09-21T21:43:25.340 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 62.660927307000001, 2024-09-21T21:43:25.340 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.10285678600000001, 2024-09-21T21:43:25.340 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:25.340 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:25.340 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:25.340 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:25.340 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:25.340 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:25.340 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24845 2024-09-21T21:43:25.340 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.340 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:25.340 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.340 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:25.340 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586535+0000", 2024-09-21T21:43:25.341 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:25.341 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.341 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.341 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:25.341 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586535+0000", 2024-09-21T21:43:25.341 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:25.341 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.341 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.341 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:25.341 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586535+0000", 2024-09-21T21:43:25.341 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9000000000000002e-07 2024-09-21T21:43:25.341 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.341 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.341 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:25.341 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586537+0000", 2024-09-21T21:43:25.342 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4569999999999999e-06 2024-09-21T21:43:25.342 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.342 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.342 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:25.342 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586544+0000", 2024-09-21T21:43:25.342 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5849999999999996e-06 2024-09-21T21:43:25.342 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.342 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.342 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:25.342 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586549+0000", 2024-09-21T21:43:25.342 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0089999999999998e-06 2024-09-21T21:43:25.342 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.342 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.343 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:25.343 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686239+0000", 2024-09-21T21:43:25.343 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.099689531999999997 2024-09-21T21:43:25.343 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.343 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.343 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:25.343 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686289+0000", 2024-09-21T21:43:25.343 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.007e-05 2024-09-21T21:43:25.343 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.343 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.343 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:25.343 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686633+0000", 2024-09-21T21:43:25.343 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00034428699999999998 2024-09-21T21:43:25.343 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.344 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.344 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:25.344 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688370+0000", 2024-09-21T21:43:25.344 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001736741 2024-09-21T21:43:25.344 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.344 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.344 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:25.344 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.689295+0000", 2024-09-21T21:43:25.344 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00092517900000000004 2024-09-21T21:43:25.344 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.344 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.344 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:25.344 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.689354+0000", 2024-09-21T21:43:25.345 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8799999999999999e-05 2024-09-21T21:43:25.345 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.345 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.345 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:25.345 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.689391+0000", 2024-09-21T21:43:25.345 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7735999999999999e-05 2024-09-21T21:43:25.345 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:25.345 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:25.345 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:25.345 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.345 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.345 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24846 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:25.345 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586564+0000", 2024-09-21T21:43:25.345 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 62.660898279000001, 2024-09-21T21:43:25.346 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.10375692, 2024-09-21T21:43:25.346 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:25.346 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:25.346 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:25.346 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:25.346 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:25.346 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:25.346 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24846 2024-09-21T21:43:25.346 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.346 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:25.346 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.346 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:25.346 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586564+0000", 2024-09-21T21:43:25.347 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:25.347 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.347 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.347 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:25.347 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586564+0000", 2024-09-21T21:43:25.347 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:25.347 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.347 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.347 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:25.347 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586564+0000", 2024-09-21T21:43:25.347 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4799999999999998e-07 2024-09-21T21:43:25.347 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.347 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.348 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:25.348 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586567+0000", 2024-09-21T21:43:25.348 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1669999999999999e-06 2024-09-21T21:43:25.348 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.348 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.348 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:25.348 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586578+0000", 2024-09-21T21:43:25.348 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0443e-05 2024-09-21T21:43:25.348 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.348 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.348 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:25.348 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586583+0000", 2024-09-21T21:43:25.348 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6790000000000002e-06 2024-09-21T21:43:25.348 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.349 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.349 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:25.349 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.687215+0000", 2024-09-21T21:43:25.349 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.100631973 2024-09-21T21:43:25.349 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.349 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.349 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:25.349 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.687311+0000", 2024-09-21T21:43:25.349 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5764e-05 2024-09-21T21:43:25.349 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.349 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.349 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:25.349 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.687663+0000", 2024-09-21T21:43:25.349 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00035167 2024-09-21T21:43:25.350 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.350 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.350 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:25.350 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690236+0000", 2024-09-21T21:43:25.350 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025733370000000002 2024-09-21T21:43:25.350 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.350 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.350 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:25.350 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690250+0000", 2024-09-21T21:43:25.350 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3548e-05 2024-09-21T21:43:25.350 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.350 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.350 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:25.350 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690298+0000", 2024-09-21T21:43:25.351 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7948999999999998e-05 2024-09-21T21:43:25.351 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.351 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.351 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:25.351 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690321+0000", 2024-09-21T21:43:25.351 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2841999999999999e-05 2024-09-21T21:43:25.351 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:25.351 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:25.351 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:25.351 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.351 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.351 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24849 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:25.351 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586633+0000", 2024-09-21T21:43:25.351 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 62.660828613, 2024-09-21T21:43:25.351 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.104506548, 2024-09-21T21:43:25.351 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:25.352 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:25.352 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:25.352 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:25.352 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:25.352 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:25.352 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24849 2024-09-21T21:43:25.352 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.352 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:25.352 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.352 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:25.352 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586633+0000", 2024-09-21T21:43:25.352 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:25.352 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.352 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.352 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:25.353 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586633+0000", 2024-09-21T21:43:25.353 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:25.353 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.353 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.353 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:25.353 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586634+0000", 2024-09-21T21:43:25.353 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9100000000000004e-07 2024-09-21T21:43:25.353 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.353 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.353 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:25.353 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586637+0000", 2024-09-21T21:43:25.353 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1999999999999999e-06 2024-09-21T21:43:25.353 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.353 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.354 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:25.354 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586645+0000", 2024-09-21T21:43:25.354 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7759999999999997e-06 2024-09-21T21:43:25.354 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.354 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.354 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:25.354 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586652+0000", 2024-09-21T21:43:25.354 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3560000000000004e-06 2024-09-21T21:43:25.354 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.354 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.354 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:25.354 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688395+0000", 2024-09-21T21:43:25.354 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.101743239 2024-09-21T21:43:25.354 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.354 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.355 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:25.355 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688464+0000", 2024-09-21T21:43:25.355 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8273000000000002e-05 2024-09-21T21:43:25.355 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.355 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.355 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:25.355 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688786+0000", 2024-09-21T21:43:25.355 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000322064 2024-09-21T21:43:25.355 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.355 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.355 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:25.355 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690364+0000", 2024-09-21T21:43:25.355 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001578084 2024-09-21T21:43:25.355 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.355 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.356 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:25.356 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.691050+0000", 2024-09-21T21:43:25.356 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00068623399999999998 2024-09-21T21:43:25.356 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.356 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.356 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:25.356 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.691090+0000", 2024-09-21T21:43:25.356 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9406999999999997e-05 2024-09-21T21:43:25.356 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.356 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.356 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:25.356 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.691140+0000", 2024-09-21T21:43:25.356 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0324000000000001e-05 2024-09-21T21:43:25.356 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:25.356 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:25.357 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:25.357 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.357 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.357 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24851 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:25.357 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586661+0000", 2024-09-21T21:43:25.357 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 62.660801327000001, 2024-09-21T21:43:25.357 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.10536891399999999, 2024-09-21T21:43:25.357 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:25.357 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:25.357 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:25.357 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:25.357 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:25.357 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:25.357 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24851 2024-09-21T21:43:25.357 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.357 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:25.358 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.358 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:25.358 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586661+0000", 2024-09-21T21:43:25.358 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:25.358 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.358 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.358 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:25.358 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586661+0000", 2024-09-21T21:43:25.358 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:25.358 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.358 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.358 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:25.358 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586661+0000", 2024-09-21T21:43:25.358 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4700000000000002e-07 2024-09-21T21:43:25.358 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.359 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.359 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:25.359 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586664+0000", 2024-09-21T21:43:25.359 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.672e-06 2024-09-21T21:43:25.359 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.359 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.359 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:25.359 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586671+0000", 2024-09-21T21:43:25.359 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9959999999999996e-06 2024-09-21T21:43:25.359 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.359 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.359 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:25.359 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586676+0000", 2024-09-21T21:43:25.360 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4020000000000001e-06 2024-09-21T21:43:25.360 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.360 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.360 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:25.360 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.689438+0000", 2024-09-21T21:43:25.360 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.102761503 2024-09-21T21:43:25.360 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.360 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.360 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:25.360 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.689524+0000", 2024-09-21T21:43:25.360 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6383000000000003e-05 2024-09-21T21:43:25.360 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.360 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.360 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:25.360 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.689769+0000", 2024-09-21T21:43:25.361 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00024504699999999997 2024-09-21T21:43:25.361 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.361 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.361 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:25.361 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.691936+0000", 2024-09-21T21:43:25.361 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021665130000000001 2024-09-21T21:43:25.361 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.361 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.361 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:25.361 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.691947+0000", 2024-09-21T21:43:25.361 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1198e-05 2024-09-21T21:43:25.361 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.361 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.362 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:25.362 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.691992+0000", 2024-09-21T21:43:25.362 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4774000000000002e-05 2024-09-21T21:43:25.362 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.362 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.362 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:25.362 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.692030+0000", 2024-09-21T21:43:25.362 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7978999999999997e-05 2024-09-21T21:43:25.362 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:25.362 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:25.362 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:25.362 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.362 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.362 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24853 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:25.363 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586684+0000", 2024-09-21T21:43:25.363 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 62.660778370000003, 2024-09-21T21:43:25.363 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.105936852, 2024-09-21T21:43:25.363 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:25.363 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:25.363 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:25.363 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:25.363 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:25.363 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:25.363 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24853 2024-09-21T21:43:25.363 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.363 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:25.363 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.364 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:25.364 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586684+0000", 2024-09-21T21:43:25.364 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:25.364 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.364 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.364 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:25.364 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586684+0000", 2024-09-21T21:43:25.364 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:25.364 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.364 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.364 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:25.364 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586684+0000", 2024-09-21T21:43:25.364 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.68e-07 2024-09-21T21:43:25.364 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.365 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.365 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:25.365 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586687+0000", 2024-09-21T21:43:25.365 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.774e-06 2024-09-21T21:43:25.365 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.365 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.365 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:25.365 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586694+0000", 2024-09-21T21:43:25.365 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6959999999999999e-06 2024-09-21T21:43:25.365 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.365 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.365 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:25.365 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586698+0000", 2024-09-21T21:43:25.365 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7489999999999998e-06 2024-09-21T21:43:25.366 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.366 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.366 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:25.366 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690395+0000", 2024-09-21T21:43:25.366 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.103696645 2024-09-21T21:43:25.366 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.366 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.366 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:25.366 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690440+0000", 2024-09-21T21:43:25.366 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4835e-05 2024-09-21T21:43:25.366 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.366 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.366 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:25.367 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690654+0000", 2024-09-21T21:43:25.367 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000214114 2024-09-21T21:43:25.367 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.367 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.367 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:25.367 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.692041+0000", 2024-09-21T21:43:25.367 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0013866519999999999 2024-09-21T21:43:25.367 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.367 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.367 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:25.367 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.692558+0000", 2024-09-21T21:43:25.367 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00051717199999999999 2024-09-21T21:43:25.367 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.368 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.368 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:25.368 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.692586+0000", 2024-09-21T21:43:25.368 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8504e-05 2024-09-21T21:43:25.368 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:25.368 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:25.368 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:25.368 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.692620+0000", 2024-09-21T21:43:25.368 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4143000000000002e-05 2024-09-21T21:43:25.368 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:25.368 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:25.368 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:25.368 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:25.369 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:25.369 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:43:25.369 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 5, 3, 0, 4] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-21T21:43:25.369 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T21:43:25.369 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-21T21:43:25.369 INFO:tasks.thrashosds.thrasher:Setting osd 4 primary_affinity to 0.000000 2024-09-21T21:43:25.369 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 0 2024-09-21T21:43:25.371 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:43:25.369+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:43:25.372 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T21:43:25.472 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:43:25.470+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:43:25.475 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:43:25.475 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:43:25.476 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 0 2024-09-21T21:43:25.476 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:43:25.485 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T21:43:25.573 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:43:25.572+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:43:25.628 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:43:25.629 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:43:25.629 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:43:25.629 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:43:25.629 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:43:25.637 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T21:43:25.680 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:43:25.679+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:43:25.741 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:43:25.741 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 20, 2024-09-21T21:43:25.741 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 600, 2024-09-21T21:43:25.741 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:43:25.741 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.741 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11319 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 5492736~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:25.741 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.712385+0000", 2024-09-21T21:43:25.742 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 95.026591698999994, 2024-09-21T21:43:25.742 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078627889000000006, 2024-09-21T21:43:25.742 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:25.742 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:25.742 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:25.742 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:25.742 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:25.742 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:25.742 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11319 2024-09-21T21:43:25.742 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.742 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:25.742 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.742 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:25.742 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712385+0000", 2024-09-21T21:43:25.743 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:25.743 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.743 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.743 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:25.743 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712385+0000", 2024-09-21T21:43:25.743 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:25.743 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.743 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.743 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:25.743 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712385+0000", 2024-09-21T21:43:25.743 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2800000000000002e-07 2024-09-21T21:43:25.743 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.743 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.744 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:25.744 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712387+0000", 2024-09-21T21:43:25.744 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1569999999999998e-06 2024-09-21T21:43:25.744 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.744 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.744 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:25.744 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712393+0000", 2024-09-21T21:43:25.744 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9370000000000001e-06 2024-09-21T21:43:25.744 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.744 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.744 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:25.744 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712398+0000", 2024-09-21T21:43:25.744 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5020000000000003e-06 2024-09-21T21:43:25.744 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.744 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.745 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:25.745 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.790746+0000", 2024-09-21T21:43:25.745 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078348589999999996 2024-09-21T21:43:25.745 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.745 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.745 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:25.745 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.790844+0000", 2024-09-21T21:43:25.745 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.7526000000000001e-05 2024-09-21T21:43:25.745 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.745 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.745 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:25.745 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.791013+0000", 2024-09-21T21:43:25.745 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00016874899999999999 2024-09-21T21:43:25.745 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:25.745 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:25.746 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:25.746 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.746 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.746 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11320 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 6180864~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:25.746 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.712405+0000", 2024-09-21T21:43:25.746 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 95.026571739999994, 2024-09-21T21:43:25.746 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078882870999999993, 2024-09-21T21:43:25.746 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:25.746 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:25.746 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:25.746 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:25.746 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:25.746 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:25.747 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11320 2024-09-21T21:43:25.747 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.747 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:25.747 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.747 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:25.747 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712405+0000", 2024-09-21T21:43:25.747 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:25.747 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.747 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.747 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:25.747 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712405+0000", 2024-09-21T21:43:25.747 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:25.747 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.747 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.747 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:25.748 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.713942+0000", 2024-09-21T21:43:25.748 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.001536882 2024-09-21T21:43:25.748 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.748 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.748 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:25.748 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.713946+0000", 2024-09-21T21:43:25.748 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4549999999999997e-06 2024-09-21T21:43:25.748 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.748 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.748 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:25.748 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.713958+0000", 2024-09-21T21:43:25.748 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1775e-05 2024-09-21T21:43:25.748 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.748 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.749 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:25.749 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.713965+0000", 2024-09-21T21:43:25.749 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5100000000000001e-06 2024-09-21T21:43:25.749 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.749 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.749 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:25.749 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.791059+0000", 2024-09-21T21:43:25.749 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.077094036000000005 2024-09-21T21:43:25.749 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.749 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.749 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:25.749 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.791175+0000", 2024-09-21T21:43:25.749 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000115895 2024-09-21T21:43:25.749 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.749 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.750 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:25.750 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.791288+0000", 2024-09-21T21:43:25.750 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000112318 2024-09-21T21:43:25.750 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:25.750 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:25.750 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:25.750 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.750 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.750 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11325 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 5414912~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:25.750 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.714095+0000", 2024-09-21T21:43:25.750 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 95.024881745000002, 2024-09-21T21:43:25.750 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078631894999999993, 2024-09-21T21:43:25.750 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:25.750 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:25.750 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:25.751 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:25.751 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:25.751 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:25.751 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11325 2024-09-21T21:43:25.751 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.751 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:25.751 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.751 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:25.751 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714095+0000", 2024-09-21T21:43:25.751 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:25.751 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.751 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.751 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:25.751 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714095+0000", 2024-09-21T21:43:25.751 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:25.752 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.752 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.752 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:25.752 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714095+0000", 2024-09-21T21:43:25.752 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7500000000000001e-07 2024-09-21T21:43:25.752 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.752 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.752 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:25.752 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714097+0000", 2024-09-21T21:43:25.752 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.12e-06 2024-09-21T21:43:25.752 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.752 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.752 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:25.752 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714104+0000", 2024-09-21T21:43:25.752 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3520000000000003e-06 2024-09-21T21:43:25.753 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.753 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.753 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:25.753 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714108+0000", 2024-09-21T21:43:25.753 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8810000000000004e-06 2024-09-21T21:43:25.753 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.753 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.753 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:25.753 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792509+0000", 2024-09-21T21:43:25.753 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078400739999999997 2024-09-21T21:43:25.753 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.753 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.753 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:25.753 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792613+0000", 2024-09-21T21:43:25.753 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000103939 2024-09-21T21:43:25.754 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.754 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.754 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:25.754 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792727+0000", 2024-09-21T21:43:25.754 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011348799999999999 2024-09-21T21:43:25.754 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:25.754 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:25.754 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:25.754 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.754 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.754 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11326 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 6103040~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:25.754 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.714115+0000", 2024-09-21T21:43:25.754 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 95.024861716999993, 2024-09-21T21:43:25.754 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078963056000000004, 2024-09-21T21:43:25.755 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:25.755 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:25.755 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:25.755 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:25.755 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:25.755 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:25.755 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11326 2024-09-21T21:43:25.755 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.755 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:25.755 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.755 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:25.755 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714115+0000", 2024-09-21T21:43:25.755 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:25.755 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.755 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.756 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:25.756 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714115+0000", 2024-09-21T21:43:25.756 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:25.756 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.756 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.756 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:25.756 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.715940+0000", 2024-09-21T21:43:25.756 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0018254969999999999 2024-09-21T21:43:25.756 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.756 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.756 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:25.756 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.715945+0000", 2024-09-21T21:43:25.757 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6120000000000001e-06 2024-09-21T21:43:25.757 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.757 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.757 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:25.757 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.715954+0000", 2024-09-21T21:43:25.757 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.5899999999999997e-06 2024-09-21T21:43:25.757 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.757 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.757 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:25.757 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.715959+0000", 2024-09-21T21:43:25.757 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0849999999999996e-06 2024-09-21T21:43:25.757 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.757 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.757 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:25.758 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792771+0000", 2024-09-21T21:43:25.758 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.076811829999999998 2024-09-21T21:43:25.758 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.758 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.758 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:25.758 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792882+0000", 2024-09-21T21:43:25.758 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000110857 2024-09-21T21:43:25.758 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.758 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.758 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:25.758 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.793078+0000", 2024-09-21T21:43:25.758 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00019558499999999999 2024-09-21T21:43:25.758 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:25.759 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:25.759 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:25.759 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.759 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.759 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11332 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 7565312~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:25.759 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.716063+0000", 2024-09-21T21:43:25.759 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 95.022913183, 2024-09-21T21:43:25.759 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078787482000000006, 2024-09-21T21:43:25.759 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:25.759 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:25.759 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:25.759 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:25.759 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:25.759 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:25.759 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11332 2024-09-21T21:43:25.760 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.760 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:25.760 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.760 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:25.760 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.716063+0000", 2024-09-21T21:43:25.760 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:25.760 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.760 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.760 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:25.760 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.716063+0000", 2024-09-21T21:43:25.760 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:25.760 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.760 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.760 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:25.761 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.717944+0000", 2024-09-21T21:43:25.761 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0018812309999999999 2024-09-21T21:43:25.761 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.761 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.761 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:25.761 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.717949+0000", 2024-09-21T21:43:25.761 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9389999999999996e-06 2024-09-21T21:43:25.761 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.761 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.761 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:25.761 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.717963+0000", 2024-09-21T21:43:25.762 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.3899e-05 2024-09-21T21:43:25.762 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.762 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.762 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:25.762 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.717970+0000", 2024-09-21T21:43:25.762 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.426e-06 2024-09-21T21:43:25.762 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.762 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.762 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:25.762 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.794573+0000", 2024-09-21T21:43:25.762 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.076602990999999995 2024-09-21T21:43:25.762 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.762 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.762 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:25.763 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.794682+0000", 2024-09-21T21:43:25.763 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000109541 2024-09-21T21:43:25.763 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.763 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.763 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:25.763 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.794851+0000", 2024-09-21T21:43:25.763 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000168455 2024-09-21T21:43:25.763 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:25.763 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:25.763 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:25.763 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.763 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.763 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11339 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 6266880~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:25.763 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.718117+0000", 2024-09-21T21:43:25.764 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 95.020859157000004, 2024-09-21T21:43:25.764 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078703717000000006, 2024-09-21T21:43:25.764 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:25.764 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:25.764 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:25.764 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:25.764 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:25.764 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:25.764 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11339 2024-09-21T21:43:25.764 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.764 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:25.764 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.764 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:25.764 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.718117+0000", 2024-09-21T21:43:25.765 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:25.765 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.765 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.765 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:25.765 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.718117+0000", 2024-09-21T21:43:25.765 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:25.765 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.765 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.765 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:25.765 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720019+0000", 2024-09-21T21:43:25.765 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0019012549999999999 2024-09-21T21:43:25.765 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.765 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.765 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:25.765 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720024+0000", 2024-09-21T21:43:25.766 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1100000000000002e-06 2024-09-21T21:43:25.766 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.766 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.766 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:25.766 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720035+0000", 2024-09-21T21:43:25.766 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.094e-05 2024-09-21T21:43:25.766 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.766 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.766 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:25.766 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720040+0000", 2024-09-21T21:43:25.766 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1830000000000002e-06 2024-09-21T21:43:25.766 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.766 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.766 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:25.766 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796618+0000", 2024-09-21T21:43:25.767 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.076578033000000004 2024-09-21T21:43:25.767 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.767 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.767 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:25.767 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796696+0000", 2024-09-21T21:43:25.767 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7836000000000005e-05 2024-09-21T21:43:25.767 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.767 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.767 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:25.767 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796821+0000", 2024-09-21T21:43:25.767 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00012536000000000001 2024-09-21T21:43:25.767 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:25.768 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:25.768 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:25.768 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.768 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.768 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11345 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 5472256~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:25.768 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.720116+0000", 2024-09-21T21:43:25.768 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 95.018860257, 2024-09-21T21:43:25.768 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078667730000000005, 2024-09-21T21:43:25.768 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:25.768 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:25.768 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:25.768 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:25.768 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:25.768 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:25.768 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11345 2024-09-21T21:43:25.769 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.769 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:25.769 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.769 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:25.769 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720116+0000", 2024-09-21T21:43:25.769 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:25.769 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.769 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.769 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:25.769 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720116+0000", 2024-09-21T21:43:25.769 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:25.769 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.769 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.769 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:25.770 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720116+0000", 2024-09-21T21:43:25.770 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.04e-07 2024-09-21T21:43:25.770 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.770 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.770 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:25.770 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720118+0000", 2024-09-21T21:43:25.770 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.181e-06 2024-09-21T21:43:25.770 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.770 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.770 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:25.770 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720121+0000", 2024-09-21T21:43:25.770 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0199999999999999e-06 2024-09-21T21:43:25.771 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.771 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.771 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:25.771 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720123+0000", 2024-09-21T21:43:25.771 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.5469999999999999e-06 2024-09-21T21:43:25.771 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.771 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.771 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:25.771 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798626+0000", 2024-09-21T21:43:25.771 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078503137000000001 2024-09-21T21:43:25.771 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.771 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.771 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:25.771 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798677+0000", 2024-09-21T21:43:25.771 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0930999999999997e-05 2024-09-21T21:43:25.772 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.772 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.772 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:25.772 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798784+0000", 2024-09-21T21:43:25.772 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010671 2024-09-21T21:43:25.772 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:25.772 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:25.772 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:25.772 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.772 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.772 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11346 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 5500928~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:25.772 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.720126+0000", 2024-09-21T21:43:25.772 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 95.018850184000001, 2024-09-21T21:43:25.772 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078863421000000003, 2024-09-21T21:43:25.773 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:25.773 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:25.773 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:25.773 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:25.773 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:25.773 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:25.773 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11346 2024-09-21T21:43:25.773 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.773 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:25.773 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.773 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:25.773 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720126+0000", 2024-09-21T21:43:25.773 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:25.773 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.773 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.774 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:25.774 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720126+0000", 2024-09-21T21:43:25.774 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:25.774 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.774 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.774 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:25.774 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.722145+0000", 2024-09-21T21:43:25.774 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0020183219999999999 2024-09-21T21:43:25.774 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.774 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.774 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:25.774 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.722150+0000", 2024-09-21T21:43:25.774 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1760000000000001e-06 2024-09-21T21:43:25.774 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.775 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.775 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:25.775 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.722161+0000", 2024-09-21T21:43:25.775 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1566000000000001e-05 2024-09-21T21:43:25.775 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.775 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.775 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:25.775 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.722169+0000", 2024-09-21T21:43:25.775 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.2840000000000001e-06 2024-09-21T21:43:25.775 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.775 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.775 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:25.775 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798815+0000", 2024-09-21T21:43:25.776 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.076646607000000005 2024-09-21T21:43:25.776 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.776 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.776 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:25.776 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798870+0000", 2024-09-21T21:43:25.776 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4486e-05 2024-09-21T21:43:25.776 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.776 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.776 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:25.776 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798990+0000", 2024-09-21T21:43:25.776 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011998 2024-09-21T21:43:25.776 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:25.776 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:25.776 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:25.776 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.777 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.777 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13629 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7852032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:25.777 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943319+0000", 2024-09-21T21:43:25.777 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 94.795657542000001, 2024-09-21T21:43:25.777 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078610975999999999, 2024-09-21T21:43:25.777 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:25.777 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:25.777 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:25.777 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:25.777 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:25.777 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:25.777 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13629 2024-09-21T21:43:25.777 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.777 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:25.777 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.778 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:25.778 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943319+0000", 2024-09-21T21:43:25.778 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:25.778 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.778 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.778 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:25.778 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943319+0000", 2024-09-21T21:43:25.778 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:25.778 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.778 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.778 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:25.778 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943319+0000", 2024-09-21T21:43:25.778 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6199999999999999e-07 2024-09-21T21:43:25.778 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.779 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.779 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:25.779 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943321+0000", 2024-09-21T21:43:25.779 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1009999999999999e-06 2024-09-21T21:43:25.779 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.779 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.779 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:25.779 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943328+0000", 2024-09-21T21:43:25.779 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.1029999999999996e-06 2024-09-21T21:43:25.779 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.779 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.779 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:25.779 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943333+0000", 2024-09-21T21:43:25.779 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7709999999999997e-06 2024-09-21T21:43:25.780 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.780 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.780 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:25.780 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.021679+0000", 2024-09-21T21:43:25.780 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078345673000000005 2024-09-21T21:43:25.780 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.780 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.780 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:25.780 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.021771+0000", 2024-09-21T21:43:25.780 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2201999999999996e-05 2024-09-21T21:43:25.781 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.781 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.781 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:25.781 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.021930+0000", 2024-09-21T21:43:25.781 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00015876400000000001 2024-09-21T21:43:25.781 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:25.781 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:25.781 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:25.781 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.781 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.781 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13630 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7880704~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:25.781 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943339+0000", 2024-09-21T21:43:25.781 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 94.795637353999993, 2024-09-21T21:43:25.781 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078939410000000002, 2024-09-21T21:43:25.782 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:25.782 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:25.782 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:25.782 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:25.782 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:25.782 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:25.782 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13630 2024-09-21T21:43:25.782 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.782 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:25.782 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.782 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:25.782 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943339+0000", 2024-09-21T21:43:25.782 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:25.782 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.782 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.783 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:25.783 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943339+0000", 2024-09-21T21:43:25.783 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:25.783 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.783 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.783 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:25.783 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943339+0000", 2024-09-21T21:43:25.783 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8000000000000001e-07 2024-09-21T21:43:25.783 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.783 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.783 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:25.783 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943341+0000", 2024-09-21T21:43:25.783 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9850000000000001e-06 2024-09-21T21:43:25.783 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.784 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.784 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:25.784 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943347+0000", 2024-09-21T21:43:25.784 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2979999999999999e-06 2024-09-21T21:43:25.784 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.784 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.784 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:25.784 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943354+0000", 2024-09-21T21:43:25.784 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.2969999999999998e-06 2024-09-21T21:43:25.784 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.784 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.784 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:25.784 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.021996+0000", 2024-09-21T21:43:25.784 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078641680000000005 2024-09-21T21:43:25.784 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.784 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.785 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:25.785 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022104+0000", 2024-09-21T21:43:25.785 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000107902 2024-09-21T21:43:25.785 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.785 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.785 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:25.785 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022278+0000", 2024-09-21T21:43:25.785 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00017486800000000001 2024-09-21T21:43:25.785 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:25.785 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:25.785 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:25.785 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.785 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.785 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13631 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7909376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:25.785 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943362+0000", 2024-09-21T21:43:25.786 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 94.795614533999995, 2024-09-21T21:43:25.786 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079195644999999995, 2024-09-21T21:43:25.786 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:25.786 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:25.786 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:25.786 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:25.786 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:25.786 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:25.786 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13631 2024-09-21T21:43:25.786 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.786 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:25.786 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.786 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:25.786 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943362+0000", 2024-09-21T21:43:25.786 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:25.787 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.787 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.787 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:25.787 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943362+0000", 2024-09-21T21:43:25.787 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:25.787 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.787 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.787 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:25.787 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943362+0000", 2024-09-21T21:43:25.787 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8799999999999998e-07 2024-09-21T21:43:25.787 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.787 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.787 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:25.787 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943364+0000", 2024-09-21T21:43:25.787 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9850000000000001e-06 2024-09-21T21:43:25.788 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.788 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.788 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:25.788 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943371+0000", 2024-09-21T21:43:25.788 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0650000000000001e-06 2024-09-21T21:43:25.788 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.788 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.788 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:25.788 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943376+0000", 2024-09-21T21:43:25.788 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8250000000000004e-06 2024-09-21T21:43:25.788 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.788 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.788 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:25.789 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022329+0000", 2024-09-21T21:43:25.789 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078952888999999998 2024-09-21T21:43:25.789 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.789 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.789 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:25.789 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022419+0000", 2024-09-21T21:43:25.789 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.9738e-05 2024-09-21T21:43:25.789 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.789 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.789 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:25.789 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022558+0000", 2024-09-21T21:43:25.789 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00013875500000000001 2024-09-21T21:43:25.789 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:25.789 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:25.789 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:25.790 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.790 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.790 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13632 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7966720~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:25.790 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943382+0000", 2024-09-21T21:43:25.790 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 94.795594300000005, 2024-09-21T21:43:25.790 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079495727000000002, 2024-09-21T21:43:25.790 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:25.790 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:25.790 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:25.790 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:25.790 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:25.790 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:25.790 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13632 2024-09-21T21:43:25.790 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.791 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:25.791 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.791 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:25.791 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943382+0000", 2024-09-21T21:43:25.791 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:25.791 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.791 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.791 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:25.791 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943382+0000", 2024-09-21T21:43:25.791 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:25.791 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.791 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.791 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:25.791 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943382+0000", 2024-09-21T21:43:25.792 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7e-07 2024-09-21T21:43:25.792 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.792 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.792 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:25.792 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943385+0000", 2024-09-21T21:43:25.792 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2230000000000001e-06 2024-09-21T21:43:25.792 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.792 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.792 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:25.792 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943391+0000", 2024-09-21T21:43:25.792 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6810000000000001e-06 2024-09-21T21:43:25.792 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.792 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.792 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:25.792 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943397+0000", 2024-09-21T21:43:25.793 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6559999999999997e-06 2024-09-21T21:43:25.793 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.793 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.793 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:25.793 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022601+0000", 2024-09-21T21:43:25.793 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079204051999999997 2024-09-21T21:43:25.793 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.793 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.793 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:25.793 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022704+0000", 2024-09-21T21:43:25.793 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010329599999999999 2024-09-21T21:43:25.793 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.793 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.793 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:25.793 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022878+0000", 2024-09-21T21:43:25.793 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000173449 2024-09-21T21:43:25.794 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:25.794 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:25.794 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:25.794 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.794 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.794 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13633 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7938048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:25.794 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943403+0000", 2024-09-21T21:43:25.794 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 94.79557337, 2024-09-21T21:43:25.794 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079794697999999997, 2024-09-21T21:43:25.794 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:25.794 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:25.794 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:25.794 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:25.794 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:25.795 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:25.795 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13633 2024-09-21T21:43:25.795 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.795 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:25.795 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.795 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:25.795 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943403+0000", 2024-09-21T21:43:25.795 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:25.795 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.795 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.795 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:25.795 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943403+0000", 2024-09-21T21:43:25.795 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:25.795 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.795 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.796 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:25.796 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943403+0000", 2024-09-21T21:43:25.796 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.65e-07 2024-09-21T21:43:25.796 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.796 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.796 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:25.796 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943405+0000", 2024-09-21T21:43:25.796 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0020000000000001e-06 2024-09-21T21:43:25.796 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.796 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.796 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:25.796 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943412+0000", 2024-09-21T21:43:25.796 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.8379999999999996e-06 2024-09-21T21:43:25.796 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.797 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.797 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:25.797 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943417+0000", 2024-09-21T21:43:25.797 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7650000000000001e-06 2024-09-21T21:43:25.797 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.797 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.797 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:25.797 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022922+0000", 2024-09-21T21:43:25.797 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079504512999999999 2024-09-21T21:43:25.797 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.797 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.797 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:25.797 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023039+0000", 2024-09-21T21:43:25.797 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000117044 2024-09-21T21:43:25.797 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.798 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.798 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:25.798 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023198+0000", 2024-09-21T21:43:25.798 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000159171 2024-09-21T21:43:25.798 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:25.798 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:25.798 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:25.798 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.798 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.798 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13634 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7995392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:25.798 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943423+0000", 2024-09-21T21:43:25.798 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 94.795553662000003, 2024-09-21T21:43:25.798 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.080142358999999996, 2024-09-21T21:43:25.798 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:25.799 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:25.799 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:25.799 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:25.799 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:25.799 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:25.799 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13634 2024-09-21T21:43:25.799 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.799 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:25.799 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.799 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:25.799 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943423+0000", 2024-09-21T21:43:25.799 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:25.799 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.799 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.800 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:25.800 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943423+0000", 2024-09-21T21:43:25.800 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:25.800 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.800 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.800 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:25.800 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943423+0000", 2024-09-21T21:43:25.800 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.58e-07 2024-09-21T21:43:25.800 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.800 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.800 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:25.800 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943425+0000", 2024-09-21T21:43:25.800 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9180000000000001e-06 2024-09-21T21:43:25.800 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.800 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.800 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:25.801 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943431+0000", 2024-09-21T21:43:25.801 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4439999999999996e-06 2024-09-21T21:43:25.801 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.801 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.801 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:25.801 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943436+0000", 2024-09-21T21:43:25.801 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.887e-06 2024-09-21T21:43:25.801 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.801 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.801 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:25.801 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023266+0000", 2024-09-21T21:43:25.801 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079829622000000003 2024-09-21T21:43:25.801 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.801 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.801 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:25.802 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023384+0000", 2024-09-21T21:43:25.802 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000117658 2024-09-21T21:43:25.802 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.802 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.802 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:25.802 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023565+0000", 2024-09-21T21:43:25.802 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00018147200000000001 2024-09-21T21:43:25.802 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:25.802 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:25.802 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:25.802 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.802 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.802 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13635 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 8024064~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:25.802 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943447+0000", 2024-09-21T21:43:25.802 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 94.795529122000005, 2024-09-21T21:43:25.803 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.080417653000000006, 2024-09-21T21:43:25.803 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:25.803 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:25.803 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:25.803 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:25.803 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:25.803 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:25.803 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13635 2024-09-21T21:43:25.803 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.803 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:25.803 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.803 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:25.803 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943447+0000", 2024-09-21T21:43:25.803 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:25.804 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.804 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.804 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:25.804 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943447+0000", 2024-09-21T21:43:25.804 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:25.804 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.804 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.804 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:25.804 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943448+0000", 2024-09-21T21:43:25.804 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1699999999999999e-07 2024-09-21T21:43:25.804 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.804 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.804 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:25.804 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943450+0000", 2024-09-21T21:43:25.805 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1639999999999999e-06 2024-09-21T21:43:25.805 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.805 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.805 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:25.805 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943456+0000", 2024-09-21T21:43:25.805 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8379999999999998e-06 2024-09-21T21:43:25.805 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.805 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.805 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:25.805 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943460+0000", 2024-09-21T21:43:25.805 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4700000000000004e-06 2024-09-21T21:43:25.805 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.805 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.805 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:25.805 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023595+0000", 2024-09-21T21:43:25.806 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.080134869999999997 2024-09-21T21:43:25.806 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.806 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.806 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:25.806 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023688+0000", 2024-09-21T21:43:25.806 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2794000000000006e-05 2024-09-21T21:43:25.806 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.806 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.806 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:25.806 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023865+0000", 2024-09-21T21:43:25.806 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0001771 2024-09-21T21:43:25.806 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:25.806 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:25.806 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:25.806 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.807 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.807 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13636 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 8052736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:25.807 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943467+0000", 2024-09-21T21:43:25.807 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 94.795509385000003, 2024-09-21T21:43:25.807 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.080694399999999999, 2024-09-21T21:43:25.807 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:25.807 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:25.807 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:25.807 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:25.807 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:25.807 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:25.807 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13636 2024-09-21T21:43:25.807 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.807 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:25.807 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.807 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:25.808 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943467+0000", 2024-09-21T21:43:25.808 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:25.808 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.808 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.808 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:25.808 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943467+0000", 2024-09-21T21:43:25.808 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:25.808 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.808 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.808 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:25.808 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944940+0000", 2024-09-21T21:43:25.808 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0014725109999999999 2024-09-21T21:43:25.808 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.808 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.808 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:25.809 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944949+0000", 2024-09-21T21:43:25.809 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2930000000000006e-06 2024-09-21T21:43:25.809 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.809 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.809 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:25.809 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944958+0000", 2024-09-21T21:43:25.809 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.6899999999999998e-06 2024-09-21T21:43:25.809 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.809 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.809 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:25.809 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944966+0000", 2024-09-21T21:43:25.809 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0690000000000002e-06 2024-09-21T21:43:25.809 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.809 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.809 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:25.809 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023900+0000", 2024-09-21T21:43:25.810 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078934371000000003 2024-09-21T21:43:25.810 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.810 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.810 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:25.810 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024009+0000", 2024-09-21T21:43:25.810 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000108857 2024-09-21T21:43:25.810 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.810 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.810 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:25.810 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024161+0000", 2024-09-21T21:43:25.810 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000152609 2024-09-21T21:43:25.810 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:25.810 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:25.810 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:25.810 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.811 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.811 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13637 3.12 3.1dc31b72 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:25.811 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.944978+0000", 2024-09-21T21:43:25.811 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 94.793998450000004, 2024-09-21T21:43:25.811 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079232604999999998, 2024-09-21T21:43:25.811 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:25.811 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:25.811 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:43:25.811 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:25.811 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:25.811 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:25.811 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13637 2024-09-21T21:43:25.811 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.811 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:25.811 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.812 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:25.812 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944978+0000", 2024-09-21T21:43:25.812 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:25.812 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.812 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.812 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:25.812 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944978+0000", 2024-09-21T21:43:25.812 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:25.812 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.812 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.812 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:25.812 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944978+0000", 2024-09-21T21:43:25.812 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9100000000000004e-07 2024-09-21T21:43:25.812 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.813 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.813 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:25.813 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944981+0000", 2024-09-21T21:43:25.813 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3290000000000001e-06 2024-09-21T21:43:25.813 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.813 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.813 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:25.813 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944987+0000", 2024-09-21T21:43:25.813 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.28e-06 2024-09-21T21:43:25.813 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.813 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.813 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:25.813 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944992+0000", 2024-09-21T21:43:25.813 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8520000000000003e-06 2024-09-21T21:43:25.813 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.813 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.814 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:25.814 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024196+0000", 2024-09-21T21:43:25.814 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079203673000000002 2024-09-21T21:43:25.814 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.814 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.814 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:25.814 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024211+0000", 2024-09-21T21:43:25.814 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.4980000000000001e-05 2024-09-21T21:43:25.814 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:25.814 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:25.814 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:25.814 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.814 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.814 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13638 3.12 3.1dc31b72 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:25.815 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.945000+0000", 2024-09-21T21:43:25.815 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 94.793976322000006, 2024-09-21T21:43:25.815 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079257465999999999, 2024-09-21T21:43:25.815 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:25.815 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:25.815 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:43:25.815 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:25.815 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:25.815 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:25.815 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13638 2024-09-21T21:43:25.815 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.815 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:25.815 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.815 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:25.816 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945000+0000", 2024-09-21T21:43:25.816 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:25.816 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.816 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.816 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:25.816 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945000+0000", 2024-09-21T21:43:25.816 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:25.816 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.816 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.816 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:25.816 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945001+0000", 2024-09-21T21:43:25.816 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.32e-07 2024-09-21T21:43:25.816 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.816 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.817 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:25.817 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945003+0000", 2024-09-21T21:43:25.817 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2309999999999999e-06 2024-09-21T21:43:25.817 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.817 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.817 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:25.817 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945009+0000", 2024-09-21T21:43:25.817 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7450000000000001e-06 2024-09-21T21:43:25.817 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.817 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.817 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:25.817 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945013+0000", 2024-09-21T21:43:25.817 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.673e-06 2024-09-21T21:43:25.817 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.817 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.818 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:25.818 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024245+0000", 2024-09-21T21:43:25.818 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079232160999999995 2024-09-21T21:43:25.818 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.818 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.818 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:25.818 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024258+0000", 2024-09-21T21:43:25.818 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.2224e-05 2024-09-21T21:43:25.818 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:25.818 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:25.818 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:25.818 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.818 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.818 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13639 3.12 3.1dc31b72 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:25.819 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.945034+0000", 2024-09-21T21:43:25.819 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 94.793942522999998, 2024-09-21T21:43:25.819 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079260627, 2024-09-21T21:43:25.819 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:25.819 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:25.819 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:43:25.819 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:25.819 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:25.819 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:25.819 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13639 2024-09-21T21:43:25.819 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.819 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:25.819 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.819 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:25.820 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945034+0000", 2024-09-21T21:43:25.820 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:25.820 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.820 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.820 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:25.820 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945034+0000", 2024-09-21T21:43:25.820 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:25.820 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.820 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.820 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:25.820 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945034+0000", 2024-09-21T21:43:25.820 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3799999999999998e-07 2024-09-21T21:43:25.820 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.820 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.820 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:25.821 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945036+0000", 2024-09-21T21:43:25.821 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.052e-06 2024-09-21T21:43:25.821 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.821 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.821 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:25.821 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945040+0000", 2024-09-21T21:43:25.821 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1169999999999997e-06 2024-09-21T21:43:25.821 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.821 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.821 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:25.821 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945043+0000", 2024-09-21T21:43:25.821 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.8439999999999998e-06 2024-09-21T21:43:25.821 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.821 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.822 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:25.822 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024286+0000", 2024-09-21T21:43:25.822 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079242483000000002 2024-09-21T21:43:25.822 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.822 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.822 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:25.822 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024295+0000", 2024-09-21T21:43:25.822 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.7930000000000003e-06 2024-09-21T21:43:25.822 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:25.822 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:25.822 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:25.822 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.822 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.822 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13640 3.12 3.1dc31b72 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:25.822 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.945047+0000", 2024-09-21T21:43:25.823 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 94.793929000999995, 2024-09-21T21:43:25.823 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079412321999999994, 2024-09-21T21:43:25.823 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:25.823 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:25.823 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:43:25.823 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:25.823 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:25.823 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:25.823 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13640 2024-09-21T21:43:25.823 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.823 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:25.823 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.823 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:25.823 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945047+0000", 2024-09-21T21:43:25.823 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:25.824 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.824 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.824 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:25.824 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945047+0000", 2024-09-21T21:43:25.824 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:25.824 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.824 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.824 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:25.824 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945048+0000", 2024-09-21T21:43:25.824 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0499999999999999e-07 2024-09-21T21:43:25.824 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.824 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.824 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:25.824 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945049+0000", 2024-09-21T21:43:25.824 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.3939999999999999e-06 2024-09-21T21:43:25.825 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.826 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.826 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:25.826 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945053+0000", 2024-09-21T21:43:25.826 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1760000000000003e-06 2024-09-21T21:43:25.826 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.827 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.827 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:25.827 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945056+0000", 2024-09-21T21:43:25.827 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.6129999999999998e-06 2024-09-21T21:43:25.827 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.827 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.827 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:25.827 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024360+0000", 2024-09-21T21:43:25.827 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079303645000000006 2024-09-21T21:43:25.827 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:25.827 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:25.827 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:25.827 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024460+0000", 2024-09-21T21:43:25.827 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000100189 2024-09-21T21:43:25.827 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:25.828 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:25.828 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:25.828 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:25.828 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:25.828 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:43:25.829 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T21:43:25.830 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:43:25.828+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:43:25.930 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:43:25.929+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:43:25.972 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:43:25.972 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:43:25.972 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 0 2024-09-21T21:43:25.972 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:43:25.980 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T21:43:26.031 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:43:26.030+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:43:26.083 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:43:26.084 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:43:26.084 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:43:26.084 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:43:26.084 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:43:26.092 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T21:43:26.133 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:43:26.132+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:43:26.235 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:43:26.233+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:43:26.237 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:43:26.237 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 20, 2024-09-21T21:43:26.237 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 600, 2024-09-21T21:43:26.237 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:43:26.237 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.237 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4824.0:1564 3.2 3:4647520c:::rbd_data.1121b97f2925.0000000000000006:head [stat,delete] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-21T21:43:26.237 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:43:26.237 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 144.59835698800001, 2024-09-21T21:43:26.238 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056097347999999998, 2024-09-21T21:43:26.238 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:26.238 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:26.238 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:26.238 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:26.238 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4824", 2024-09-21T21:43:26.238 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3663922554", 2024-09-21T21:43:26.238 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 1564 2024-09-21T21:43:26.238 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.238 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:26.238 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.238 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:26.238 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:43:26.239 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:26.239 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.239 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.239 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:26.239 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:43:26.239 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:26.239 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.239 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.239 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:26.239 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636065+0000", 2024-09-21T21:43:26.239 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6499999999999999e-07 2024-09-21T21:43:26.239 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.239 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.239 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:26.240 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636067+0000", 2024-09-21T21:43:26.240 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3879999999999998e-06 2024-09-21T21:43:26.240 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.240 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.240 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:26.240 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636075+0000", 2024-09-21T21:43:26.240 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5480000000000004e-06 2024-09-21T21:43:26.240 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.240 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.240 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:26.240 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636080+0000", 2024-09-21T21:43:26.240 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.339e-06 2024-09-21T21:43:26.240 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.241 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.241 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:26.241 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636133+0000", 2024-09-21T21:43:26.241 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2930000000000003e-05 2024-09-21T21:43:26.241 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.241 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.241 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:26.241 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.636265+0000", 2024-09-21T21:43:26.241 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000132058 2024-09-21T21:43:26.241 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.241 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.241 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:26.241 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:01.692162+0000", 2024-09-21T21:43:26.241 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055896620000000001 2024-09-21T21:43:26.242 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:26.242 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:26.242 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:26.242 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.242 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.242 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3260 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 1437696~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:43:26.242 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.196297+0000", 2024-09-21T21:43:26.242 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 99.038124777999997, 2024-09-21T21:43:26.242 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056071022999999998, 2024-09-21T21:43:26.242 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:26.242 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:26.242 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:26.242 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:26.242 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:26.243 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:26.243 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3260 2024-09-21T21:43:26.243 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.243 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:26.243 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.243 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:26.243 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196297+0000", 2024-09-21T21:43:26.243 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:26.243 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.243 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.243 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:26.243 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196297+0000", 2024-09-21T21:43:26.243 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:26.243 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.244 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.244 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:26.244 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196918+0000", 2024-09-21T21:43:26.244 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00062148699999999999 2024-09-21T21:43:26.244 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.244 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.244 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:26.244 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196930+0000", 2024-09-21T21:43:26.244 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1657e-05 2024-09-21T21:43:26.244 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.244 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.244 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:26.244 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196944+0000", 2024-09-21T21:43:26.244 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.4549000000000001e-05 2024-09-21T21:43:26.245 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.245 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.245 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:26.245 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.196951+0000", 2024-09-21T21:43:26.245 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6749999999999996e-06 2024-09-21T21:43:26.245 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.245 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.245 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:26.245 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252208+0000", 2024-09-21T21:43:26.245 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055256656000000001 2024-09-21T21:43:26.245 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.245 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.245 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:26.245 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252278+0000", 2024-09-21T21:43:26.246 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9882000000000006e-05 2024-09-21T21:43:26.246 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.246 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.246 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:26.246 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.252368+0000", 2024-09-21T21:43:26.246 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.0117000000000005e-05 2024-09-21T21:43:26.246 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:26.246 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:26.246 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:26.246 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.246 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.246 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3437 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 925696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:43:26.246 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.205888+0000", 2024-09-21T21:43:26.247 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 99.028533683000006, 2024-09-21T21:43:26.247 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055953808000000001, 2024-09-21T21:43:26.247 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:26.247 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:26.247 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:26.247 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:26.247 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:26.247 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:26.247 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3437 2024-09-21T21:43:26.247 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.247 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:26.247 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.247 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:26.248 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205888+0000", 2024-09-21T21:43:26.248 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:26.248 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.248 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.248 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:26.248 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205888+0000", 2024-09-21T21:43:26.248 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:26.248 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.248 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.248 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:26.248 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205888+0000", 2024-09-21T21:43:26.248 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.75e-07 2024-09-21T21:43:26.248 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.248 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.249 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:26.249 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205891+0000", 2024-09-21T21:43:26.249 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3750000000000001e-06 2024-09-21T21:43:26.249 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.249 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.249 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:26.249 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205897+0000", 2024-09-21T21:43:26.249 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.285e-06 2024-09-21T21:43:26.249 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.249 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.249 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:26.249 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205902+0000", 2024-09-21T21:43:26.249 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6290000000000001e-06 2024-09-21T21:43:26.250 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.250 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.250 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:26.250 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261686+0000", 2024-09-21T21:43:26.250 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055784586999999997 2024-09-21T21:43:26.250 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.250 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.250 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:26.250 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261764+0000", 2024-09-21T21:43:26.250 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7591999999999998e-05 2024-09-21T21:43:26.250 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.250 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.250 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:26.250 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261842+0000", 2024-09-21T21:43:26.251 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7865000000000001e-05 2024-09-21T21:43:26.251 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:26.251 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:26.251 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:26.251 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.251 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.251 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3438 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 954368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:43:26.252 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.205908+0000", 2024-09-21T21:43:26.252 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 99.028513817999993, 2024-09-21T21:43:26.252 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056083169000000002, 2024-09-21T21:43:26.252 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:26.252 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:26.252 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:26.252 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:26.252 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:26.252 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:26.252 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3438 2024-09-21T21:43:26.252 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.252 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:26.252 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.252 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:26.253 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205908+0000", 2024-09-21T21:43:26.253 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:26.253 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.253 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.253 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:26.253 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.205908+0000", 2024-09-21T21:43:26.253 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:26.253 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.253 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.253 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:26.253 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206431+0000", 2024-09-21T21:43:26.253 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00052282100000000003 2024-09-21T21:43:26.253 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.253 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.254 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:26.254 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206442+0000", 2024-09-21T21:43:26.254 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1695000000000001e-05 2024-09-21T21:43:26.254 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.254 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.254 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:26.254 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206452+0000", 2024-09-21T21:43:26.254 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.4560000000000006e-06 2024-09-21T21:43:26.254 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.254 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.254 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:26.254 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206458+0000", 2024-09-21T21:43:26.254 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4670000000000001e-06 2024-09-21T21:43:26.255 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.255 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.255 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:26.255 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261871+0000", 2024-09-21T21:43:26.255 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055412712000000003 2024-09-21T21:43:26.255 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.255 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.255 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:26.255 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261928+0000", 2024-09-21T21:43:26.255 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6631e-05 2024-09-21T21:43:26.255 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.255 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.255 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:26.255 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.261991+0000", 2024-09-21T21:43:26.256 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3386999999999995e-05 2024-09-21T21:43:26.256 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:26.256 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:26.256 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:26.256 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.256 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.256 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:3444 3.b 3:d3249882:::rbd_data.11216538973.0000000000000000:1e [read 2359296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-21T21:43:26.256 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:47.206533+0000", 2024-09-21T21:43:26.256 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 99.027888297999993, 2024-09-21T21:43:26.256 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055974239000000002, 2024-09-21T21:43:26.256 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:26.256 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:26.256 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:26.256 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:26.257 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:26.257 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:26.257 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 3444 2024-09-21T21:43:26.257 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.257 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:26.257 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.257 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:26.257 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206533+0000", 2024-09-21T21:43:26.257 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:26.257 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.257 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.257 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:26.257 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206533+0000", 2024-09-21T21:43:26.257 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:26.258 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.258 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.258 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:26.258 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206710+0000", 2024-09-21T21:43:26.258 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000176293 2024-09-21T21:43:26.258 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.258 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.258 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:26.258 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206714+0000", 2024-09-21T21:43:26.258 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1989999999999999e-06 2024-09-21T21:43:26.258 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.258 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.258 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:26.259 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206723+0000", 2024-09-21T21:43:26.259 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.9199999999999993e-06 2024-09-21T21:43:26.259 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.259 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.259 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:26.259 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.206729+0000", 2024-09-21T21:43:26.259 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0739999999999998e-06 2024-09-21T21:43:26.259 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.259 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.259 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:26.259 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262339+0000", 2024-09-21T21:43:26.259 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.055610166000000003 2024-09-21T21:43:26.259 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.259 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.259 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:26.260 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262419+0000", 2024-09-21T21:43:26.260 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0432999999999994e-05 2024-09-21T21:43:26.260 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.260 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.260 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:26.260 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:47.262507+0000", 2024-09-21T21:43:26.260 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.8153999999999997e-05 2024-09-21T21:43:26.260 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:26.260 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:26.260 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:26.260 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.260 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.261 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10362 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:43:26.261 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840386+0000", 2024-09-21T21:43:26.261 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 72.394035454000004, 2024-09-21T21:43:26.261 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056546515999999998, 2024-09-21T21:43:26.261 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:26.261 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:26.261 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:26.261 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:26.261 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:43:26.261 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:43:26.261 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10362 2024-09-21T21:43:26.261 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.261 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:26.261 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.262 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:26.262 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840386+0000", 2024-09-21T21:43:26.262 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:26.262 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.262 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.262 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:26.262 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840386+0000", 2024-09-21T21:43:26.262 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:26.262 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.262 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.262 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:26.262 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840386+0000", 2024-09-21T21:43:26.262 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.01e-07 2024-09-21T21:43:26.262 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.263 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.263 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:26.263 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840389+0000", 2024-09-21T21:43:26.263 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.9739999999999998e-06 2024-09-21T21:43:26.263 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.263 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.263 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:26.263 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840396+0000", 2024-09-21T21:43:26.263 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2280000000000003e-06 2024-09-21T21:43:26.263 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.263 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.263 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:26.264 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840400+0000", 2024-09-21T21:43:26.264 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4429999999999996e-06 2024-09-21T21:43:26.264 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.264 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.264 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:26.264 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.890896+0000", 2024-09-21T21:43:26.264 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.050495717000000002 2024-09-21T21:43:26.264 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.264 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.264 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:26.264 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.891012+0000", 2024-09-21T21:43:26.264 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011602 2024-09-21T21:43:26.264 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.264 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.265 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:43:26.265 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.891274+0000", 2024-09-21T21:43:26.265 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00026240200000000001 2024-09-21T21:43:26.265 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.265 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.265 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:43:26.265 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.894810+0000", 2024-09-21T21:43:26.265 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0035353630000000001 2024-09-21T21:43:26.265 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.265 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.265 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:26.279 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.896818+0000", 2024-09-21T21:43:26.279 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.002008681 2024-09-21T21:43:26.279 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.279 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.279 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:43:26.279 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.896850+0000", 2024-09-21T21:43:26.279 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.1826999999999999e-05 2024-09-21T21:43:26.280 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.280 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.280 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:26.280 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.896933+0000", 2024-09-21T21:43:26.280 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.2460000000000007e-05 2024-09-21T21:43:26.280 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:26.280 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:26.280 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:26.280 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.280 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.280 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10363 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:43:26.280 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840606+0000", 2024-09-21T21:43:26.280 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 72.393815943999996, 2024-09-21T21:43:26.280 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056546011, 2024-09-21T21:43:26.281 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:26.281 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:26.281 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:26.281 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:26.281 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:43:26.281 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:43:26.281 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10363 2024-09-21T21:43:26.281 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.281 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:26.281 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.281 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:26.281 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840606+0000", 2024-09-21T21:43:26.281 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:26.281 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.281 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.282 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:26.282 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840606+0000", 2024-09-21T21:43:26.282 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:26.282 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.282 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.282 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:26.282 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840606+0000", 2024-09-21T21:43:26.282 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.4700000000000001e-07 2024-09-21T21:43:26.282 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.282 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.282 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:26.282 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840611+0000", 2024-09-21T21:43:26.282 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4649999999999996e-06 2024-09-21T21:43:26.283 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.283 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.283 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:26.283 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840620+0000", 2024-09-21T21:43:26.283 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.0350000000000007e-06 2024-09-21T21:43:26.283 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.283 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.283 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:26.284 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840626+0000", 2024-09-21T21:43:26.284 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7180000000000001e-06 2024-09-21T21:43:26.284 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.284 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.284 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:26.284 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.891665+0000", 2024-09-21T21:43:26.284 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.051039924 2024-09-21T21:43:26.284 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.284 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.284 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:26.284 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.891770+0000", 2024-09-21T21:43:26.284 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010445700000000001 2024-09-21T21:43:26.284 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.284 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.285 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:43:26.285 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.892147+0000", 2024-09-21T21:43:26.285 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00037731800000000002 2024-09-21T21:43:26.285 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.285 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.285 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:43:26.285 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.896776+0000", 2024-09-21T21:43:26.285 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0046289759999999999 2024-09-21T21:43:26.285 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.285 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.285 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:26.285 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897003+0000", 2024-09-21T21:43:26.285 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00022641299999999999 2024-09-21T21:43:26.285 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.286 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.286 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:43:26.286 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897064+0000", 2024-09-21T21:43:26.286 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1571999999999999e-05 2024-09-21T21:43:26.286 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.286 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.286 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:26.286 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897152+0000", 2024-09-21T21:43:26.286 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.7386000000000007e-05 2024-09-21T21:43:26.286 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:26.286 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:26.286 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:26.286 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.287 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.287 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10364 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:43:26.287 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840641+0000", 2024-09-21T21:43:26.287 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 72.393780124000003, 2024-09-21T21:43:26.287 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056671527999999999, 2024-09-21T21:43:26.287 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:26.287 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:26.287 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:26.287 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:26.287 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:43:26.287 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:43:26.287 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10364 2024-09-21T21:43:26.287 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.287 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:26.288 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.288 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:26.288 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840641+0000", 2024-09-21T21:43:26.288 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:26.288 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.288 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.288 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:26.288 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840641+0000", 2024-09-21T21:43:26.288 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:26.288 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.288 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.288 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:26.288 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840642+0000", 2024-09-21T21:43:26.288 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7899999999999999e-07 2024-09-21T21:43:26.288 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.289 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.289 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:26.289 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840645+0000", 2024-09-21T21:43:26.289 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0010000000000002e-06 2024-09-21T21:43:26.289 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.289 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.289 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:26.289 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840654+0000", 2024-09-21T21:43:26.289 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.8570000000000001e-06 2024-09-21T21:43:26.289 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.289 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.289 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:26.290 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840659+0000", 2024-09-21T21:43:26.290 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.558e-06 2024-09-21T21:43:26.290 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.290 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.290 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:26.290 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.892656+0000", 2024-09-21T21:43:26.290 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.051996586999999997 2024-09-21T21:43:26.290 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.290 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.291 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:26.291 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.892721+0000", 2024-09-21T21:43:26.291 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5010000000000003e-05 2024-09-21T21:43:26.291 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.291 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.291 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:43:26.291 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.893112+0000", 2024-09-21T21:43:26.291 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00039133899999999999 2024-09-21T21:43:26.291 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.291 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.291 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:43:26.291 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.896784+0000", 2024-09-21T21:43:26.291 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0036713679999999999 2024-09-21T21:43:26.291 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.292 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.292 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:26.292 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897204+0000", 2024-09-21T21:43:26.292 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00041991400000000002 2024-09-21T21:43:26.292 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.292 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.292 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:43:26.292 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897234+0000", 2024-09-21T21:43:26.292 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0307999999999998e-05 2024-09-21T21:43:26.292 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.292 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.292 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:26.292 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897313+0000", 2024-09-21T21:43:26.293 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.9107e-05 2024-09-21T21:43:26.293 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:26.293 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:26.293 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:26.293 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.293 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.293 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10365 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:43:26.293 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840668+0000", 2024-09-21T21:43:26.293 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 72.393753258999993, 2024-09-21T21:43:26.293 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.058761225, 2024-09-21T21:43:26.293 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:26.293 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:26.294 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:26.294 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:26.294 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:43:26.294 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:43:26.294 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10365 2024-09-21T21:43:26.294 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.294 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:26.294 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.294 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:26.294 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840668+0000", 2024-09-21T21:43:26.294 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:26.294 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.295 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.295 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:26.295 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840668+0000", 2024-09-21T21:43:26.295 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:26.295 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.295 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.295 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:26.295 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840669+0000", 2024-09-21T21:43:26.295 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6300000000000001e-07 2024-09-21T21:43:26.295 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.295 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.295 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:26.296 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840672+0000", 2024-09-21T21:43:26.296 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.072e-06 2024-09-21T21:43:26.296 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.296 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.296 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:26.296 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840678+0000", 2024-09-21T21:43:26.296 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.7039999999999997e-06 2024-09-21T21:43:26.296 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.296 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.296 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:26.296 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840683+0000", 2024-09-21T21:43:26.296 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6759999999999998e-06 2024-09-21T21:43:26.296 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.296 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.297 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:26.297 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.893929+0000", 2024-09-21T21:43:26.297 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.053246136999999999 2024-09-21T21:43:26.297 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.297 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.297 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:26.297 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.894082+0000", 2024-09-21T21:43:26.297 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00015301799999999999 2024-09-21T21:43:26.297 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.297 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.297 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:43:26.297 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.894357+0000", 2024-09-21T21:43:26.297 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00027507999999999998 2024-09-21T21:43:26.298 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.298 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.298 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:43:26.298 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897161+0000", 2024-09-21T21:43:26.298 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0028036810000000001 2024-09-21T21:43:26.298 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.298 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.298 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:26.298 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.899243+0000", 2024-09-21T21:43:26.298 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0020816379999999998 2024-09-21T21:43:26.298 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.298 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.298 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:43:26.298 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.899306+0000", 2024-09-21T21:43:26.299 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2964000000000002e-05 2024-09-21T21:43:26.299 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.299 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.299 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:26.299 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.899430+0000", 2024-09-21T21:43:26.299 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00012389200000000001 2024-09-21T21:43:26.299 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:26.299 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:26.299 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:26.299 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.299 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.299 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10366 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:43:26.299 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840691+0000", 2024-09-21T21:43:26.300 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 72.393730559000005, 2024-09-21T21:43:26.300 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.061540984999999999, 2024-09-21T21:43:26.300 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:26.300 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:26.300 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:26.300 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:26.300 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:43:26.300 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:43:26.300 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10366 2024-09-21T21:43:26.300 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.300 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:26.300 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.300 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:26.301 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840691+0000", 2024-09-21T21:43:26.301 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:26.301 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.301 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.301 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:26.301 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840691+0000", 2024-09-21T21:43:26.301 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:26.301 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.301 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.301 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:26.301 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840691+0000", 2024-09-21T21:43:26.301 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3099999999999999e-07 2024-09-21T21:43:26.301 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.302 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.302 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:26.302 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840694+0000", 2024-09-21T21:43:26.302 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.5780000000000001e-06 2024-09-21T21:43:26.302 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.302 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.302 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:26.302 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840700+0000", 2024-09-21T21:43:26.302 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9429999999999997e-06 2024-09-21T21:43:26.302 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.302 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.302 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:26.302 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840704+0000", 2024-09-21T21:43:26.302 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6380000000000003e-06 2024-09-21T21:43:26.302 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.303 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.303 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:26.303 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.894846+0000", 2024-09-21T21:43:26.303 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.054141392000000003 2024-09-21T21:43:26.303 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.303 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.303 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:26.303 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.894932+0000", 2024-09-21T21:43:26.303 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.5834000000000005e-05 2024-09-21T21:43:26.303 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.303 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.303 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:43:26.303 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.895302+0000", 2024-09-21T21:43:26.303 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000370737 2024-09-21T21:43:26.304 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.304 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.304 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:43:26.304 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897166+0000", 2024-09-21T21:43:26.304 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0018632620000000001 2024-09-21T21:43:26.304 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.304 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.304 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:26.304 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902053+0000", 2024-09-21T21:43:26.304 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0048868289999999997 2024-09-21T21:43:26.304 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.304 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.304 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:43:26.304 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902111+0000", 2024-09-21T21:43:26.305 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7983999999999999e-05 2024-09-21T21:43:26.305 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.305 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.305 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:26.305 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902232+0000", 2024-09-21T21:43:26.305 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000121457 2024-09-21T21:43:26.305 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:26.305 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:26.305 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:26.305 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.305 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.305 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10367 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:43:26.305 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840842+0000", 2024-09-21T21:43:26.305 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 72.393579751000004, 2024-09-21T21:43:26.305 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.061549088000000002, 2024-09-21T21:43:26.306 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:26.306 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:26.306 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:26.306 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:26.306 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:43:26.306 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:43:26.306 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10367 2024-09-21T21:43:26.306 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.306 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:26.306 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.306 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:26.306 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840842+0000", 2024-09-21T21:43:26.306 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:26.306 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.307 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.307 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:26.307 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840842+0000", 2024-09-21T21:43:26.307 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:26.307 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.307 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.307 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:26.307 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840842+0000", 2024-09-21T21:43:26.307 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3099999999999997e-07 2024-09-21T21:43:26.307 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.307 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.307 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:26.307 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840846+0000", 2024-09-21T21:43:26.307 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.2890000000000002e-06 2024-09-21T21:43:26.308 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.308 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.308 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:26.308 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840855+0000", 2024-09-21T21:43:26.308 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.9619999999999999e-06 2024-09-21T21:43:26.308 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.308 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.308 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:26.308 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840860+0000", 2024-09-21T21:43:26.308 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3759999999999999e-06 2024-09-21T21:43:26.308 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.308 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.308 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:26.308 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.895799+0000", 2024-09-21T21:43:26.309 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.054938569999999999 2024-09-21T21:43:26.309 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.309 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.309 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:26.309 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.895906+0000", 2024-09-21T21:43:26.309 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000107593 2024-09-21T21:43:26.309 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.309 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.309 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:43:26.309 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.896356+0000", 2024-09-21T21:43:26.309 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00044941700000000002 2024-09-21T21:43:26.309 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.309 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.309 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:43:26.310 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.898131+0000", 2024-09-21T21:43:26.310 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.001775064 2024-09-21T21:43:26.310 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.310 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.310 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:26.310 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902283+0000", 2024-09-21T21:43:26.310 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0041524839999999997 2024-09-21T21:43:26.310 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.310 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.310 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:43:26.310 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902316+0000", 2024-09-21T21:43:26.310 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3002e-05 2024-09-21T21:43:26.310 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.310 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.311 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:26.311 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902391+0000", 2024-09-21T21:43:26.311 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.47e-05 2024-09-21T21:43:26.311 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:26.311 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:26.311 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:26.311 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.311 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.311 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10368 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:43:26.311 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840873+0000", 2024-09-21T21:43:26.311 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 72.393548796999994, 2024-09-21T21:43:26.311 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.063395291000000006, 2024-09-21T21:43:26.311 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:26.311 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:26.311 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:26.312 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:26.312 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:43:26.312 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:43:26.312 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10368 2024-09-21T21:43:26.312 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.312 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:26.312 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.312 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:26.312 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840873+0000", 2024-09-21T21:43:26.312 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:26.312 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.312 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.312 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:26.312 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840873+0000", 2024-09-21T21:43:26.313 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:26.313 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.313 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.313 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:26.313 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840873+0000", 2024-09-21T21:43:26.313 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6699999999999999e-07 2024-09-21T21:43:26.313 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.313 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.313 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:26.313 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840876+0000", 2024-09-21T21:43:26.313 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.6410000000000002e-06 2024-09-21T21:43:26.313 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.313 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.313 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:26.313 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840883+0000", 2024-09-21T21:43:26.314 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0389999999999999e-06 2024-09-21T21:43:26.314 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.314 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.314 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:26.314 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840888+0000", 2024-09-21T21:43:26.314 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8679999999999998e-06 2024-09-21T21:43:26.314 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.314 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.314 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:26.314 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897345+0000", 2024-09-21T21:43:26.314 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.056457734000000002 2024-09-21T21:43:26.314 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.314 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.315 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:26.315 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897407+0000", 2024-09-21T21:43:26.315 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1822000000000005e-05 2024-09-21T21:43:26.315 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.315 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.315 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:43:26.315 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.897664+0000", 2024-09-21T21:43:26.315 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00025638499999999998 2024-09-21T21:43:26.315 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.315 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.315 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:43:26.315 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.901136+0000", 2024-09-21T21:43:26.315 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0034728070000000001 2024-09-21T21:43:26.316 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.316 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.316 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:26.316 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904090+0000", 2024-09-21T21:43:26.316 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0029535799999999999 2024-09-21T21:43:26.316 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.316 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.316 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:43:26.316 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904144+0000", 2024-09-21T21:43:26.316 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4369000000000001e-05 2024-09-21T21:43:26.316 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.316 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.316 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:26.317 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904268+0000", 2024-09-21T21:43:26.317 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00012367899999999999 2024-09-21T21:43:26.317 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:26.317 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:26.317 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:26.317 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.317 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.317 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10369 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:43:26.317 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840902+0000", 2024-09-21T21:43:26.317 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 72.393519143999995, 2024-09-21T21:43:26.317 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.063538938000000003, 2024-09-21T21:43:26.317 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:26.317 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:26.317 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:26.317 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:26.318 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:43:26.318 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:43:26.318 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10369 2024-09-21T21:43:26.318 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.318 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:26.318 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.318 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:26.318 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840902+0000", 2024-09-21T21:43:26.318 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:26.318 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.318 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.318 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:26.318 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840902+0000", 2024-09-21T21:43:26.318 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:26.319 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.319 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.319 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:26.319 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840903+0000", 2024-09-21T21:43:26.319 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2100000000000002e-07 2024-09-21T21:43:26.319 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.319 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.319 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:26.319 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840913+0000", 2024-09-21T21:43:26.319 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.048e-05 2024-09-21T21:43:26.319 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.319 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.319 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:26.320 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840920+0000", 2024-09-21T21:43:26.320 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0330000000000003e-06 2024-09-21T21:43:26.320 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.320 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.320 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:26.320 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840925+0000", 2024-09-21T21:43:26.320 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0170000000000004e-06 2024-09-21T21:43:26.320 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.320 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.320 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:26.320 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.898179+0000", 2024-09-21T21:43:26.320 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.057253671999999999 2024-09-21T21:43:26.320 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.320 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.320 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:26.321 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.898284+0000", 2024-09-21T21:43:26.321 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000105202 2024-09-21T21:43:26.321 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.321 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.321 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:43:26.321 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.898729+0000", 2024-09-21T21:43:26.321 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00044459300000000002 2024-09-21T21:43:26.321 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.321 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.321 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:43:26.321 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902246+0000", 2024-09-21T21:43:26.321 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0035171629999999998 2024-09-21T21:43:26.321 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.321 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.322 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:26.322 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904310+0000", 2024-09-21T21:43:26.322 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.002063765 2024-09-21T21:43:26.322 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.322 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.322 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:43:26.322 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904342+0000", 2024-09-21T21:43:26.322 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.2091999999999997e-05 2024-09-21T21:43:26.322 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.322 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.322 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:26.322 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904441+0000", 2024-09-21T21:43:26.322 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.9500000000000006e-05 2024-09-21T21:43:26.322 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:26.323 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:26.323 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:26.323 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.323 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.323 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10370 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:43:26.323 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840944+0000", 2024-09-21T21:43:26.323 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 72.393477430999994, 2024-09-21T21:43:26.323 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.064436853000000002, 2024-09-21T21:43:26.323 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:26.323 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:26.323 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:26.323 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:26.323 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:43:26.323 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:43:26.324 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10370 2024-09-21T21:43:26.324 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.324 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:26.324 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.324 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:26.324 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840944+0000", 2024-09-21T21:43:26.324 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:26.324 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.324 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.324 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:26.324 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840944+0000", 2024-09-21T21:43:26.324 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:26.324 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.324 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.324 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:26.325 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840945+0000", 2024-09-21T21:43:26.325 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2300000000000002e-07 2024-09-21T21:43:26.325 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.325 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.325 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:26.325 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840947+0000", 2024-09-21T21:43:26.325 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.8200000000000001e-06 2024-09-21T21:43:26.325 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.325 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.325 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:26.325 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840955+0000", 2024-09-21T21:43:26.325 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.8660000000000006e-06 2024-09-21T21:43:26.325 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.326 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.326 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:26.326 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840960+0000", 2024-09-21T21:43:26.326 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8670000000000002e-06 2024-09-21T21:43:26.326 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.326 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.326 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:26.326 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.899460+0000", 2024-09-21T21:43:26.326 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.058499922000000003 2024-09-21T21:43:26.326 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.326 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.326 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:26.326 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.899530+0000", 2024-09-21T21:43:26.326 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9558e-05 2024-09-21T21:43:26.327 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.327 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.327 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:43:26.327 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.899775+0000", 2024-09-21T21:43:26.327 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000245231 2024-09-21T21:43:26.327 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.327 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.327 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:43:26.327 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902254+0000", 2024-09-21T21:43:26.327 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0024790519999999998 2024-09-21T21:43:26.327 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.327 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.327 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:26.327 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905183+0000", 2024-09-21T21:43:26.328 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.002929156 2024-09-21T21:43:26.328 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.328 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.328 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:43:26.328 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905243+0000", 2024-09-21T21:43:26.328 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0055e-05 2024-09-21T21:43:26.328 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.328 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.328 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:26.328 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905381+0000", 2024-09-21T21:43:26.328 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00013790300000000001 2024-09-21T21:43:26.328 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:26.328 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:26.328 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:26.329 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.329 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.329 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10371 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:43:26.329 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.840980+0000", 2024-09-21T21:43:26.329 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 72.393441765000006, 2024-09-21T21:43:26.329 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.064554376999999996, 2024-09-21T21:43:26.329 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:26.329 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:26.329 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:26.329 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:26.329 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:43:26.329 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:43:26.329 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10371 2024-09-21T21:43:26.329 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.330 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:26.330 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.330 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:26.330 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840980+0000", 2024-09-21T21:43:26.330 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:26.330 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.330 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.330 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:26.330 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840980+0000", 2024-09-21T21:43:26.330 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:26.330 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.330 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.330 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:26.330 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840980+0000", 2024-09-21T21:43:26.330 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8999999999999997e-07 2024-09-21T21:43:26.331 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.331 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.331 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:26.331 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840983+0000", 2024-09-21T21:43:26.331 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.123e-06 2024-09-21T21:43:26.331 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.331 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.331 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:26.331 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840990+0000", 2024-09-21T21:43:26.331 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9129999999999997e-06 2024-09-21T21:43:26.331 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.331 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.332 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:26.332 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.840995+0000", 2024-09-21T21:43:26.332 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0989999999999998e-06 2024-09-21T21:43:26.332 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.332 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.332 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:26.332 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.900168+0000", 2024-09-21T21:43:26.332 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.059172739000000002 2024-09-21T21:43:26.332 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.332 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.332 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:26.332 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.900274+0000", 2024-09-21T21:43:26.332 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010546499999999999 2024-09-21T21:43:26.333 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.333 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.333 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:43:26.333 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.900703+0000", 2024-09-21T21:43:26.333 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00042978200000000003 2024-09-21T21:43:26.333 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.333 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.333 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:43:26.333 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.903443+0000", 2024-09-21T21:43:26.333 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0027400749999999998 2024-09-21T21:43:26.333 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.333 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.333 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:26.333 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905430+0000", 2024-09-21T21:43:26.333 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0019868109999999998 2024-09-21T21:43:26.334 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.334 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.334 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:43:26.334 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905464+0000", 2024-09-21T21:43:26.334 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3599000000000001e-05 2024-09-21T21:43:26.334 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.334 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.334 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:26.334 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905534+0000", 2024-09-21T21:43:26.334 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0281e-05 2024-09-21T21:43:26.335 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:26.335 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:26.335 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:26.335 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.335 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.335 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10372 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:43:26.335 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.841004+0000", 2024-09-21T21:43:26.335 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 72.393417876000001, 2024-09-21T21:43:26.335 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.064757812999999997, 2024-09-21T21:43:26.335 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:26.335 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:26.335 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:26.335 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:26.336 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:43:26.336 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:43:26.336 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10372 2024-09-21T21:43:26.336 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.336 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:26.336 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.336 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:26.336 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841004+0000", 2024-09-21T21:43:26.336 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:26.336 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.336 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.336 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:26.336 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841004+0000", 2024-09-21T21:43:26.336 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:26.337 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.337 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.337 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:26.337 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841004+0000", 2024-09-21T21:43:26.337 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8500000000000002e-07 2024-09-21T21:43:26.337 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.337 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.337 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:26.337 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841007+0000", 2024-09-21T21:43:26.337 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.6390000000000001e-06 2024-09-21T21:43:26.337 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.337 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.337 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:26.337 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841013+0000", 2024-09-21T21:43:26.338 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0070000000000003e-06 2024-09-21T21:43:26.338 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.338 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.338 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:26.338 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841018+0000", 2024-09-21T21:43:26.338 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9119999999999997e-06 2024-09-21T21:43:26.338 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.338 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.338 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:26.338 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.901182+0000", 2024-09-21T21:43:26.338 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.060164444999999997 2024-09-21T21:43:26.338 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.338 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.338 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:26.339 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.901297+0000", 2024-09-21T21:43:26.339 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011468699999999999 2024-09-21T21:43:26.339 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.339 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.339 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:43:26.339 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.901559+0000", 2024-09-21T21:43:26.339 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000261764 2024-09-21T21:43:26.339 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.339 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.339 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:43:26.339 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904281+0000", 2024-09-21T21:43:26.339 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0027229329999999999 2024-09-21T21:43:26.340 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.340 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.340 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:26.340 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905588+0000", 2024-09-21T21:43:26.340 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.001306438 2024-09-21T21:43:26.340 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.340 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.340 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:43:26.340 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905644+0000", 2024-09-21T21:43:26.340 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5747e-05 2024-09-21T21:43:26.340 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.340 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.340 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:26.341 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.905761+0000", 2024-09-21T21:43:26.341 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000117856 2024-09-21T21:43:26.341 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:26.341 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:26.341 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:26.341 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.341 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.341 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:10373 3.6 3:616d9006:::rbd_object_map.1121faebdfe3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-21T21:43:26.341 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:13.841027+0000", 2024-09-21T21:43:26.341 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 72.393394791999995, 2024-09-21T21:43:26.341 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.066941839000000003, 2024-09-21T21:43:26.341 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:26.341 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:26.341 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:26.342 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:26.342 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:43:26.342 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:43:26.342 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 10373 2024-09-21T21:43:26.342 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.342 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:26.342 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.342 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:26.342 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841027+0000", 2024-09-21T21:43:26.342 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:26.342 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.342 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.342 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:26.343 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841027+0000", 2024-09-21T21:43:26.343 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:26.343 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.343 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.343 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:26.343 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841027+0000", 2024-09-21T21:43:26.343 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6199999999999999e-07 2024-09-21T21:43:26.343 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.343 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.343 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:26.343 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841030+0000", 2024-09-21T21:43:26.343 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.729e-06 2024-09-21T21:43:26.343 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.343 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.344 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:26.344 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841036+0000", 2024-09-21T21:43:26.344 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.1979999999999998e-06 2024-09-21T21:43:26.344 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.344 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.344 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:26.344 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.841042+0000", 2024-09-21T21:43:26.344 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4289999999999998e-06 2024-09-21T21:43:26.344 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.344 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.344 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:26.344 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902433+0000", 2024-09-21T21:43:26.344 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.061390329 2024-09-21T21:43:26.344 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.345 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.345 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:26.345 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902541+0000", 2024-09-21T21:43:26.345 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000108349 2024-09-21T21:43:26.345 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.345 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.345 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for subops from 5", 2024-09-21T21:43:26.345 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.902983+0000", 2024-09-21T21:43:26.345 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00044158000000000001 2024-09-21T21:43:26.345 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.345 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.345 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op_commit", 2024-09-21T21:43:26.345 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.904490+0000", 2024-09-21T21:43:26.345 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0015074489999999999 2024-09-21T21:43:26.346 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.346 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.346 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:26.346 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.907671+0000", 2024-09-21T21:43:26.346 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.003180511 2024-09-21T21:43:26.346 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.346 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.346 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "commit_sent", 2024-09-21T21:43:26.346 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.907767+0000", 2024-09-21T21:43:26.346 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6379000000000007e-05 2024-09-21T21:43:26.346 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.346 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.346 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:26.346 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:13.907969+0000", 2024-09-21T21:43:26.347 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00020152399999999999 2024-09-21T21:43:26.347 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:26.347 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:26.347 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:26.347 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.347 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.347 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:12335 3.1a 3:5b387735:::rbd_data.1121faebdfe3.0000000000000005:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:43:26.347 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:14.411635+0000", 2024-09-21T21:43:26.347 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 71.822786914000005, 2024-09-21T21:43:26.347 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.005973491, 2024-09-21T21:43:26.347 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:26.347 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:26.347 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:26.347 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:26.348 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:43:26.348 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:43:26.348 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 12335 2024-09-21T21:43:26.348 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.348 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:26.348 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.348 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:26.348 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411635+0000", 2024-09-21T21:43:26.348 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:26.348 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.348 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.348 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:26.349 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411635+0000", 2024-09-21T21:43:26.349 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:26.349 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.349 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.349 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:26.349 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411635+0000", 2024-09-21T21:43:26.349 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8500000000000001e-07 2024-09-21T21:43:26.349 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.349 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.349 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:26.349 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411639+0000", 2024-09-21T21:43:26.349 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.574e-06 2024-09-21T21:43:26.349 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.349 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.350 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:26.350 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411645+0000", 2024-09-21T21:43:26.350 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6869999999999997e-06 2024-09-21T21:43:26.350 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.350 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.350 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:26.350 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411649+0000", 2024-09-21T21:43:26.350 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.771e-06 2024-09-21T21:43:26.350 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.350 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.350 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:26.350 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411695+0000", 2024-09-21T21:43:26.351 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6e-05 2024-09-21T21:43:26.351 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.351 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.351 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op must wait for map", 2024-09-21T21:43:26.351 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411702+0000", 2024-09-21T21:43:26.351 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.5590000000000002e-06 2024-09-21T21:43:26.351 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.351 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.351 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:26.351 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.417222+0000", 2024-09-21T21:43:26.351 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0055198219999999997 2024-09-21T21:43:26.351 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.351 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.351 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for peered", 2024-09-21T21:43:26.352 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.417230+0000", 2024-09-21T21:43:26.352 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.123e-06 2024-09-21T21:43:26.352 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.352 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.352 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:26.352 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.417348+0000", 2024-09-21T21:43:26.352 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.000118029 2024-09-21T21:43:26.352 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.352 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.352 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:26.352 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.417451+0000", 2024-09-21T21:43:26.352 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000103674 2024-09-21T21:43:26.352 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.352 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.353 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:26.353 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.417608+0000", 2024-09-21T21:43:26.353 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00015666700000000001 2024-09-21T21:43:26.353 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:26.353 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:26.353 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:26.353 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.353 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.353 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:12349 3.1a 3:5b387735:::rbd_data.1121faebdfe3.0000000000000005:head [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:43:26.353 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:14.411833+0000", 2024-09-21T21:43:26.353 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 71.822588100999994, 2024-09-21T21:43:26.353 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0065064500000001, 2024-09-21T21:43:26.353 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:26.353 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:26.354 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:26.354 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:26.354 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:43:26.354 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:43:26.354 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 12349 2024-09-21T21:43:26.354 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.354 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:26.354 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.354 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:26.354 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411833+0000", 2024-09-21T21:43:26.354 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:26.354 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.354 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.354 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:26.355 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411833+0000", 2024-09-21T21:43:26.355 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:26.355 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.355 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.355 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:26.355 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411834+0000", 2024-09-21T21:43:26.355 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.03e-07 2024-09-21T21:43:26.355 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.355 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.355 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:26.355 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411836+0000", 2024-09-21T21:43:26.355 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.4700000000000001e-06 2024-09-21T21:43:26.355 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.355 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.356 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:26.356 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411843+0000", 2024-09-21T21:43:26.356 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4069999999999998e-06 2024-09-21T21:43:26.356 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.356 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.356 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:26.356 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411848+0000", 2024-09-21T21:43:26.356 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1580000000000004e-06 2024-09-21T21:43:26.356 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.356 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.356 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:26.356 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411901+0000", 2024-09-21T21:43:26.356 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2824000000000001e-05 2024-09-21T21:43:26.356 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.357 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.357 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting_for_map not empty", 2024-09-21T21:43:26.357 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411904+0000", 2024-09-21T21:43:26.357 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0639999999999998e-06 2024-09-21T21:43:26.357 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.357 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.357 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:26.357 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.417252+0000", 2024-09-21T21:43:26.357 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0053485060000000003 2024-09-21T21:43:26.357 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.357 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.357 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for peered", 2024-09-21T21:43:26.357 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.417259+0000", 2024-09-21T21:43:26.358 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.1149999999999996e-06 2024-09-21T21:43:26.358 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.358 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.358 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:26.358 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.418160+0000", 2024-09-21T21:43:26.358 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.000901083 2024-09-21T21:43:26.358 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.358 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.358 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:26.358 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.418236+0000", 2024-09-21T21:43:26.358 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5852000000000005e-05 2024-09-21T21:43:26.358 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.358 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.358 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:26.358 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.418340+0000", 2024-09-21T21:43:26.359 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000103568 2024-09-21T21:43:26.359 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:26.359 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:26.359 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:26.359 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.359 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.359 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4385.0:30053 3.6 3:67cae2ed:::rbd_mirroring:head [list-watchers] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e141)", 2024-09-21T21:43:26.359 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:43:03.223636+0000", 2024-09-21T21:43:26.359 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 23.010785417000001, 2024-09-21T21:43:26.359 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.062115648, 2024-09-21T21:43:26.359 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:26.359 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:26.359 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:43:26.360 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:26.360 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4385", 2024-09-21T21:43:26.360 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:43:26.360 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 30053 2024-09-21T21:43:26.360 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.360 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:26.360 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.360 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:26.360 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:43:03.223636+0000", 2024-09-21T21:43:26.360 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:26.360 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.360 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.360 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:26.360 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:43:03.223636+0000", 2024-09-21T21:43:26.360 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:26.361 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.361 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.361 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:26.361 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:43:03.223638+0000", 2024-09-21T21:43:26.361 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.6080000000000001e-06 2024-09-21T21:43:26.361 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.361 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.361 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:26.361 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:43:03.223643+0000", 2024-09-21T21:43:26.361 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9110000000000001e-06 2024-09-21T21:43:26.361 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.361 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.361 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:26.361 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:43:03.223654+0000", 2024-09-21T21:43:26.362 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1029999999999999e-05 2024-09-21T21:43:26.362 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.362 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.362 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:26.362 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:43:03.223664+0000", 2024-09-21T21:43:26.362 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.9299999999999998e-06 2024-09-21T21:43:26.362 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.362 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.362 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:26.362 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:43:03.223856+0000", 2024-09-21T21:43:26.362 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00019216200000000001 2024-09-21T21:43:26.362 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.362 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.362 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for peered", 2024-09-21T21:43:26.362 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:43:03.223866+0000", 2024-09-21T21:43:26.363 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0356999999999999e-05 2024-09-21T21:43:26.363 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.363 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.363 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:26.363 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:43:04.285628+0000", 2024-09-21T21:43:26.363 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.061762211 2024-09-21T21:43:26.363 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.363 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.363 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:26.363 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:43:04.285752+0000", 2024-09-21T21:43:26.363 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00012343900000000001 2024-09-21T21:43:26.363 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:26.363 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:26.364 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:26.364 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:26.364 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:26.364 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:43:26.364 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-21T21:43:26.366 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T21:43:26.366 INFO:teuthology.orchestra.run.smithi081.stderr:set osd.4 primary-affinity to 0 (802) 2024-09-21T21:43:26.367 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:43:26.365+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:43:26.468 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:43:26.465+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:43:26.484 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:43:26.484 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:43:26.484 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 0 2024-09-21T21:43:26.484 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:43:26.498 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T21:43:26.586 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:43:26.566+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:43:26.617 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:43:26.618 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:43:26.618 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:43:26.618 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:43:26.618 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:43:26.620 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T21:43:26.669 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:43:26.668+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:43:26.769 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:43:26.768+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:43:26.771 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:43:26.771 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 20, 2024-09-21T21:43:26.771 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 600, 2024-09-21T21:43:26.772 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:43:26.772 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.772 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13296 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 4046848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:26.772 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.796643+0000", 2024-09-21T21:43:26.772 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 95.972751740999996, 2024-09-21T21:43:26.772 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071682019999999999, 2024-09-21T21:43:26.772 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:26.772 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:26.772 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:26.772 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:26.772 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:26.772 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:26.772 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13296 2024-09-21T21:43:26.772 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.772 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:26.773 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.773 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:26.773 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796643+0000", 2024-09-21T21:43:26.773 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:26.773 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.773 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.773 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:26.773 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796643+0000", 2024-09-21T21:43:26.773 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:26.773 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.773 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.773 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:26.773 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796643+0000", 2024-09-21T21:43:26.773 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.7199999999999998e-07 2024-09-21T21:43:26.773 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.774 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.774 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:26.774 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796647+0000", 2024-09-21T21:43:26.774 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6770000000000001e-06 2024-09-21T21:43:26.774 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.774 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.774 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:26.774 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796656+0000", 2024-09-21T21:43:26.774 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2059999999999996e-06 2024-09-21T21:43:26.774 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.774 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.774 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:26.774 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796664+0000", 2024-09-21T21:43:26.774 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7919999999999993e-06 2024-09-21T21:43:26.775 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.775 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.775 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:26.775 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.868186+0000", 2024-09-21T21:43:26.775 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071522273999999997 2024-09-21T21:43:26.775 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.775 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.775 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:26.775 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.868239+0000", 2024-09-21T21:43:26.775 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2787999999999997e-05 2024-09-21T21:43:26.775 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.775 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.775 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:26.776 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.868325+0000", 2024-09-21T21:43:26.776 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.5611000000000004e-05 2024-09-21T21:43:26.776 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:26.776 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:26.776 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:26.776 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.776 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.776 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13561 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 1728512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:26.776 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.823991+0000", 2024-09-21T21:43:26.776 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 95.945403440000007, 2024-09-21T21:43:26.776 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071654762999999996, 2024-09-21T21:43:26.776 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:26.776 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:26.776 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:26.777 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:26.777 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:26.777 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:26.777 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13561 2024-09-21T21:43:26.777 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.777 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:26.777 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.777 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:26.777 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.823991+0000", 2024-09-21T21:43:26.777 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:26.777 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.777 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.777 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:26.777 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.823991+0000", 2024-09-21T21:43:26.777 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:26.778 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.778 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.778 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:26.778 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.823991+0000", 2024-09-21T21:43:26.778 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9299999999999998e-07 2024-09-21T21:43:26.778 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.778 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.778 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:26.778 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.823994+0000", 2024-09-21T21:43:26.778 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2369999999999999e-06 2024-09-21T21:43:26.779 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.779 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.779 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:26.779 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824000+0000", 2024-09-21T21:43:26.779 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.55e-06 2024-09-21T21:43:26.780 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.780 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.780 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:26.780 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824005+0000", 2024-09-21T21:43:26.780 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4329999999999997e-06 2024-09-21T21:43:26.780 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.780 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.780 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:26.780 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.895360+0000", 2024-09-21T21:43:26.781 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071355426 2024-09-21T21:43:26.781 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.781 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.781 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:26.781 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.895477+0000", 2024-09-21T21:43:26.781 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0001171 2024-09-21T21:43:26.781 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.781 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.782 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:26.782 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.895646+0000", 2024-09-21T21:43:26.782 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000168524 2024-09-21T21:43:26.782 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:26.782 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:26.782 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:26.782 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.783 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.783 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13562 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 1757184~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:26.783 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.824011+0000", 2024-09-21T21:43:26.783 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 95.945383563000007, 2024-09-21T21:43:26.783 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072020453999999998, 2024-09-21T21:43:26.783 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:26.783 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:26.783 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:26.783 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:26.783 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:26.783 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:26.783 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13562 2024-09-21T21:43:26.784 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.784 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:26.784 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.784 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:26.784 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824011+0000", 2024-09-21T21:43:26.784 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:26.784 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.784 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.784 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:26.784 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824011+0000", 2024-09-21T21:43:26.784 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:26.784 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.784 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.785 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:26.785 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824011+0000", 2024-09-21T21:43:26.785 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.5600000000000001e-07 2024-09-21T21:43:26.785 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.785 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.785 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:26.785 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824013+0000", 2024-09-21T21:43:26.785 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9800000000000001e-06 2024-09-21T21:43:26.785 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.785 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.786 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:26.786 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824019+0000", 2024-09-21T21:43:26.786 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7339999999999996e-06 2024-09-21T21:43:26.786 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.786 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.786 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:26.786 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824023+0000", 2024-09-21T21:43:26.786 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3680000000000004e-06 2024-09-21T21:43:26.786 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.786 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.786 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:26.786 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.895699+0000", 2024-09-21T21:43:26.786 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071675597999999993 2024-09-21T21:43:26.787 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.787 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.787 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:26.787 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.895830+0000", 2024-09-21T21:43:26.787 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000130858 2024-09-21T21:43:26.787 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.787 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.787 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:26.787 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896031+0000", 2024-09-21T21:43:26.787 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00020155999999999999 2024-09-21T21:43:26.788 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:26.788 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:26.788 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:26.788 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.788 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.788 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13563 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 323584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:26.788 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.824029+0000", 2024-09-21T21:43:26.788 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 95.945365123000002, 2024-09-21T21:43:26.788 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072307563000000005, 2024-09-21T21:43:26.788 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:26.788 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:26.789 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:26.789 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:26.789 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:26.789 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:26.789 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13563 2024-09-21T21:43:26.789 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.789 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:26.789 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.789 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:26.789 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824029+0000", 2024-09-21T21:43:26.789 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:26.789 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.789 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.789 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:26.790 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824029+0000", 2024-09-21T21:43:26.790 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:26.790 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.790 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.790 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:26.790 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824030+0000", 2024-09-21T21:43:26.790 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1699999999999999e-07 2024-09-21T21:43:26.790 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.790 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.790 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:26.790 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824032+0000", 2024-09-21T21:43:26.790 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0320000000000002e-06 2024-09-21T21:43:26.791 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.791 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.791 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:26.791 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824038+0000", 2024-09-21T21:43:26.791 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8769999999999998e-06 2024-09-21T21:43:26.791 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.791 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.791 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:26.791 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824042+0000", 2024-09-21T21:43:26.791 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5650000000000003e-06 2024-09-21T21:43:26.791 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.791 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.791 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:26.792 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896058+0000", 2024-09-21T21:43:26.792 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072015941 2024-09-21T21:43:26.792 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.792 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.792 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:26.792 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896164+0000", 2024-09-21T21:43:26.792 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010593499999999999 2024-09-21T21:43:26.792 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.792 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.792 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:26.792 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896337+0000", 2024-09-21T21:43:26.792 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00017279599999999999 2024-09-21T21:43:26.793 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:26.793 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:26.793 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:26.793 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.793 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.793 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13564 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 983040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:26.793 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.824048+0000", 2024-09-21T21:43:26.793 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 95.945346048000005, 2024-09-21T21:43:26.793 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072491193999999995, 2024-09-21T21:43:26.793 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:26.793 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:26.793 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:26.794 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:26.794 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:26.794 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:26.794 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13564 2024-09-21T21:43:26.795 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.796 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:26.796 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.796 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:26.796 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824048+0000", 2024-09-21T21:43:26.796 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:26.796 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.796 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.797 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:26.797 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824048+0000", 2024-09-21T21:43:26.797 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:26.797 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.797 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.797 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:26.797 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824663+0000", 2024-09-21T21:43:26.798 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00061494299999999996 2024-09-21T21:43:26.798 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.798 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.798 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:26.798 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824669+0000", 2024-09-21T21:43:26.798 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2900000000000002e-06 2024-09-21T21:43:26.798 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.799 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.799 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:26.800 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824679+0000", 2024-09-21T21:43:26.800 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0338e-05 2024-09-21T21:43:26.800 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.801 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.801 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:26.801 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824687+0000", 2024-09-21T21:43:26.802 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.9049999999999997e-06 2024-09-21T21:43:26.802 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.802 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.802 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:26.802 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896381+0000", 2024-09-21T21:43:26.803 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071694305999999999 2024-09-21T21:43:26.803 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.803 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.803 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:26.803 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896445+0000", 2024-09-21T21:43:26.806 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3869999999999997e-05 2024-09-21T21:43:26.807 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.807 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.807 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:26.807 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896540+0000", 2024-09-21T21:43:26.807 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.4542000000000004e-05 2024-09-21T21:43:26.807 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:26.808 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:26.808 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:26.808 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.809 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.809 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13565 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3190784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:26.809 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.824704+0000", 2024-09-21T21:43:26.810 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 95.944690176999998, 2024-09-21T21:43:26.811 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072014745000000005, 2024-09-21T21:43:26.811 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:26.812 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:26.812 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:26.812 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:26.812 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:26.812 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:26.812 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13565 2024-09-21T21:43:26.812 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.814 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:26.815 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.815 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:26.815 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824704+0000", 2024-09-21T21:43:26.817 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:26.817 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.817 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.817 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:26.817 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824704+0000", 2024-09-21T21:43:26.818 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:26.818 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.818 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.818 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:26.818 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824705+0000", 2024-09-21T21:43:26.818 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8999999999999997e-07 2024-09-21T21:43:26.818 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.818 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.818 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:26.818 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824707+0000", 2024-09-21T21:43:26.818 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.7669999999999999e-06 2024-09-21T21:43:26.818 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.818 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.818 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:26.819 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824715+0000", 2024-09-21T21:43:26.819 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7700000000000001e-06 2024-09-21T21:43:26.819 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.819 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.819 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:26.819 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824721+0000", 2024-09-21T21:43:26.819 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9560000000000002e-06 2024-09-21T21:43:26.819 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.819 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.819 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:26.819 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896570+0000", 2024-09-21T21:43:26.819 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071848870999999995 2024-09-21T21:43:26.819 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.819 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.820 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:26.820 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896626+0000", 2024-09-21T21:43:26.820 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5578000000000002e-05 2024-09-21T21:43:26.820 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.820 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.820 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:26.820 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896719+0000", 2024-09-21T21:43:26.820 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.3313000000000006e-05 2024-09-21T21:43:26.820 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:26.820 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:26.820 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:26.820 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.820 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.820 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13566 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3162112~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:26.821 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.824729+0000", 2024-09-21T21:43:26.821 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 95.944665290000003, 2024-09-21T21:43:26.821 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072356907999999998, 2024-09-21T21:43:26.821 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:26.821 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:26.821 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:26.821 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:26.821 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:26.821 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:26.821 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13566 2024-09-21T21:43:26.821 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.821 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:26.821 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.822 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:26.822 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824729+0000", 2024-09-21T21:43:26.822 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:26.822 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.822 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.822 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:26.822 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.824729+0000", 2024-09-21T21:43:26.822 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:26.822 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.822 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.822 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:26.822 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825630+0000", 2024-09-21T21:43:26.822 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00090139400000000002 2024-09-21T21:43:26.822 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.823 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.823 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:26.823 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825634+0000", 2024-09-21T21:43:26.823 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.4960000000000001e-06 2024-09-21T21:43:26.823 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.823 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.823 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:26.823 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825644+0000", 2024-09-21T21:43:26.823 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.9650000000000003e-06 2024-09-21T21:43:26.823 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.823 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.823 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:26.823 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825649+0000", 2024-09-21T21:43:26.823 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0359999999999997e-06 2024-09-21T21:43:26.823 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.824 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.824 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:26.824 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896757+0000", 2024-09-21T21:43:26.824 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071107717000000001 2024-09-21T21:43:26.824 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.824 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.824 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:26.824 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.896878+0000", 2024-09-21T21:43:26.824 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000120834 2024-09-21T21:43:26.824 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.824 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.824 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:26.824 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897086+0000", 2024-09-21T21:43:26.824 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00020846600000000001 2024-09-21T21:43:26.825 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:26.825 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:26.825 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:26.825 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.825 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.825 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13568 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3907584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:26.825 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.825671+0000", 2024-09-21T21:43:26.825 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 95.943722880999999, 2024-09-21T21:43:26.825 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071794595000000003, 2024-09-21T21:43:26.825 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:26.825 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:26.825 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:26.825 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:26.825 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:26.825 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:26.826 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13568 2024-09-21T21:43:26.826 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.826 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:26.826 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.826 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:26.826 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825671+0000", 2024-09-21T21:43:26.826 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:26.826 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.826 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.826 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:26.826 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825671+0000", 2024-09-21T21:43:26.826 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:26.826 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.826 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.827 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:26.827 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825672+0000", 2024-09-21T21:43:26.827 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2999999999999999e-07 2024-09-21T21:43:26.827 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.827 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.827 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:26.827 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825673+0000", 2024-09-21T21:43:26.827 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.246e-06 2024-09-21T21:43:26.827 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.827 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.827 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:26.827 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825693+0000", 2024-09-21T21:43:26.827 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9789999999999999e-05 2024-09-21T21:43:26.827 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.828 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.828 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:26.828 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825697+0000", 2024-09-21T21:43:26.828 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2379999999999999e-06 2024-09-21T21:43:26.828 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.828 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.828 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:26.828 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897284+0000", 2024-09-21T21:43:26.828 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071586710999999997 2024-09-21T21:43:26.828 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.828 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.828 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:26.828 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897337+0000", 2024-09-21T21:43:26.828 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2800000000000003e-05 2024-09-21T21:43:26.828 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.829 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.829 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:26.829 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897466+0000", 2024-09-21T21:43:26.829 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00012957999999999999 2024-09-21T21:43:26.829 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:26.829 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:26.829 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:26.829 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.829 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.829 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13574 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 352256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:26.829 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.825702+0000", 2024-09-21T21:43:26.829 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 95.943692733000006, 2024-09-21T21:43:26.829 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072080377000000001, 2024-09-21T21:43:26.829 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:26.829 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:26.830 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:26.830 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:26.830 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:26.830 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:26.830 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13574 2024-09-21T21:43:26.830 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.830 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:26.830 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.830 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:26.830 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825702+0000", 2024-09-21T21:43:26.830 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:26.830 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.830 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.830 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:26.830 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825702+0000", 2024-09-21T21:43:26.831 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:26.831 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.831 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.831 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:26.831 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825943+0000", 2024-09-21T21:43:26.831 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00024105200000000001 2024-09-21T21:43:26.831 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.831 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.831 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:26.831 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825946+0000", 2024-09-21T21:43:26.831 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.743e-06 2024-09-21T21:43:26.831 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.831 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.831 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:26.832 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825956+0000", 2024-09-21T21:43:26.832 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6609999999999995e-06 2024-09-21T21:43:26.832 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.832 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.832 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:26.832 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825963+0000", 2024-09-21T21:43:26.832 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4609999999999996e-06 2024-09-21T21:43:26.832 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.832 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.832 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:26.832 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897502+0000", 2024-09-21T21:43:26.832 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071539354999999999 2024-09-21T21:43:26.832 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.832 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.832 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:26.833 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897619+0000", 2024-09-21T21:43:26.833 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011665 2024-09-21T21:43:26.833 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.833 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.833 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:26.833 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897782+0000", 2024-09-21T21:43:26.833 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00016345500000000001 2024-09-21T21:43:26.833 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:26.833 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:26.833 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:26.833 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.833 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.833 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13575 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 380928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:26.833 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.825973+0000", 2024-09-21T21:43:26.833 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 95.943421297, 2024-09-21T21:43:26.834 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.072012723000000001, 2024-09-21T21:43:26.834 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:26.834 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:26.834 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:26.834 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:26.834 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:26.834 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:26.834 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13575 2024-09-21T21:43:26.834 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.834 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:26.834 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.834 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:26.834 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825973+0000", 2024-09-21T21:43:26.834 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:26.834 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.835 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.835 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:26.835 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.825973+0000", 2024-09-21T21:43:26.835 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:26.835 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.835 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.835 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:26.835 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826746+0000", 2024-09-21T21:43:26.835 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00077326800000000002 2024-09-21T21:43:26.835 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.835 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.835 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:26.836 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826766+0000", 2024-09-21T21:43:26.836 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9465000000000001e-05 2024-09-21T21:43:26.836 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.836 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.836 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:26.836 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826812+0000", 2024-09-21T21:43:26.836 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5927999999999998e-05 2024-09-21T21:43:26.836 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.836 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.836 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:26.836 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826833+0000", 2024-09-21T21:43:26.836 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1710999999999999e-05 2024-09-21T21:43:26.836 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.836 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.836 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:26.837 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897809+0000", 2024-09-21T21:43:26.837 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.070975229000000001 2024-09-21T21:43:26.837 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.837 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.837 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:26.837 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897864+0000", 2024-09-21T21:43:26.837 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5334999999999998e-05 2024-09-21T21:43:26.837 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.837 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.837 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:26.837 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.897986+0000", 2024-09-21T21:43:26.837 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000121787 2024-09-21T21:43:26.837 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:26.837 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:26.838 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:26.838 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.838 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.838 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13577 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 1814528~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:26.838 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.826874+0000", 2024-09-21T21:43:26.838 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 95.942520263000006, 2024-09-21T21:43:26.838 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071639123999999998, 2024-09-21T21:43:26.838 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:26.838 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:26.838 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:26.838 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:26.838 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:26.838 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:26.838 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13577 2024-09-21T21:43:26.839 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.839 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:26.839 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.839 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:26.839 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826874+0000", 2024-09-21T21:43:26.839 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:26.839 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.839 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.839 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:26.839 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826874+0000", 2024-09-21T21:43:26.839 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:26.839 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.840 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.840 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:26.840 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826875+0000", 2024-09-21T21:43:26.840 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.82e-07 2024-09-21T21:43:26.840 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.840 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.840 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:26.840 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826877+0000", 2024-09-21T21:43:26.840 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.295e-06 2024-09-21T21:43:26.840 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.840 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.840 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:26.841 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826883+0000", 2024-09-21T21:43:26.841 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2609999999999999e-06 2024-09-21T21:43:26.841 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.841 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.841 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:26.841 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826889+0000", 2024-09-21T21:43:26.841 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.5589999999999996e-06 2024-09-21T21:43:26.841 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.841 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.841 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:26.841 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898301+0000", 2024-09-21T21:43:26.841 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071412188000000001 2024-09-21T21:43:26.841 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.841 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.842 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:26.842 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898416+0000", 2024-09-21T21:43:26.842 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000115538 2024-09-21T21:43:26.842 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.842 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.842 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:26.842 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898513+0000", 2024-09-21T21:43:26.842 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6800999999999997e-05 2024-09-21T21:43:26.842 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:26.842 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:26.842 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:26.842 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.842 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.842 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13578 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 2502656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:26.842 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.826896+0000", 2024-09-21T21:43:26.843 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 95.942498807000007, 2024-09-21T21:43:26.843 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071789411999999997, 2024-09-21T21:43:26.843 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:26.843 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:26.843 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:26.843 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:26.843 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:26.843 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:26.843 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13578 2024-09-21T21:43:26.843 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.843 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:26.843 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.843 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:26.843 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826896+0000", 2024-09-21T21:43:26.844 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:26.844 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.844 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.844 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:26.844 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.826896+0000", 2024-09-21T21:43:26.844 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:26.844 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.844 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.844 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:26.844 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827327+0000", 2024-09-21T21:43:26.844 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00043094500000000002 2024-09-21T21:43:26.844 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.844 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.844 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:26.844 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827331+0000", 2024-09-21T21:43:26.845 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.0969999999999999e-06 2024-09-21T21:43:26.845 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.845 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.845 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:26.845 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827341+0000", 2024-09-21T21:43:26.845 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0199e-05 2024-09-21T21:43:26.845 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.845 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.845 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:26.845 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827346+0000", 2024-09-21T21:43:26.845 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.3210000000000004e-06 2024-09-21T21:43:26.845 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.845 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.845 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:26.845 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898540+0000", 2024-09-21T21:43:26.846 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071194246000000003 2024-09-21T21:43:26.846 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.846 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.846 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:26.846 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898598+0000", 2024-09-21T21:43:26.846 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7658999999999998e-05 2024-09-21T21:43:26.846 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.846 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.846 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:26.846 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898685+0000", 2024-09-21T21:43:26.846 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.6945000000000002e-05 2024-09-21T21:43:26.846 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:26.846 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:26.846 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:26.846 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.847 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.847 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13579 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 2531328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:26.847 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.827355+0000", 2024-09-21T21:43:26.847 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 95.942039276000003, 2024-09-21T21:43:26.847 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071650933, 2024-09-21T21:43:26.847 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:26.847 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:26.847 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:26.847 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:26.847 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:26.847 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:26.847 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13579 2024-09-21T21:43:26.847 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.847 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:26.847 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.848 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:26.848 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827355+0000", 2024-09-21T21:43:26.848 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:26.848 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.848 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.848 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:26.848 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827355+0000", 2024-09-21T21:43:26.848 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:26.848 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.848 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.848 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:26.848 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827486+0000", 2024-09-21T21:43:26.848 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00013066799999999999 2024-09-21T21:43:26.848 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.848 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.849 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:26.849 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827490+0000", 2024-09-21T21:43:26.849 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.9009999999999996e-06 2024-09-21T21:43:26.849 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.849 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.849 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:26.849 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827499+0000", 2024-09-21T21:43:26.849 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6209999999999999e-06 2024-09-21T21:43:26.849 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.849 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.849 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:26.849 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827506+0000", 2024-09-21T21:43:26.849 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6490000000000002e-06 2024-09-21T21:43:26.849 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.850 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.850 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:26.850 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898713+0000", 2024-09-21T21:43:26.850 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071206937999999997 2024-09-21T21:43:26.850 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.850 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.850 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:26.850 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.898811+0000", 2024-09-21T21:43:26.850 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.8159999999999995e-05 2024-09-21T21:43:26.850 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.850 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.850 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:26.850 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899006+0000", 2024-09-21T21:43:26.850 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00019499599999999999 2024-09-21T21:43:26.851 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:26.851 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:26.851 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:26.851 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.851 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.851 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13580 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3219456~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:26.851 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.827522+0000", 2024-09-21T21:43:26.851 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 95.941872845000006, 2024-09-21T21:43:26.851 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071813298999999997, 2024-09-21T21:43:26.851 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:26.851 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:26.851 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:26.851 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:26.851 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:26.852 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:26.852 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13580 2024-09-21T21:43:26.852 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.852 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:26.852 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.852 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:26.852 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827522+0000", 2024-09-21T21:43:26.852 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:26.852 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.852 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.852 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:26.852 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827522+0000", 2024-09-21T21:43:26.852 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:26.852 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.852 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.853 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:26.853 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827898+0000", 2024-09-21T21:43:26.853 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000376876 2024-09-21T21:43:26.853 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.853 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.853 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:26.853 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827903+0000", 2024-09-21T21:43:26.853 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4129999999999999e-06 2024-09-21T21:43:26.853 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.853 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.853 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:26.853 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827911+0000", 2024-09-21T21:43:26.853 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.4540000000000007e-06 2024-09-21T21:43:26.853 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.853 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.854 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:26.854 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827917+0000", 2024-09-21T21:43:26.854 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.2110000000000003e-06 2024-09-21T21:43:26.854 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.854 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.854 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:26.854 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899041+0000", 2024-09-21T21:43:26.854 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071123965999999997 2024-09-21T21:43:26.854 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.854 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.854 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:26.854 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899157+0000", 2024-09-21T21:43:26.854 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011573700000000001 2024-09-21T21:43:26.854 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.855 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.855 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:26.855 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899335+0000", 2024-09-21T21:43:26.855 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00017764199999999999 2024-09-21T21:43:26.855 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:26.855 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:26.855 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:26.855 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.855 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.855 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13581 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3248128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:26.855 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.827929+0000", 2024-09-21T21:43:26.855 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 95.941465445999995, 2024-09-21T21:43:26.855 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071730151000000006, 2024-09-21T21:43:26.855 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:26.855 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:26.856 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:26.856 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:26.856 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:26.856 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:26.856 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13581 2024-09-21T21:43:26.856 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.856 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:26.856 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.856 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:26.856 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827929+0000", 2024-09-21T21:43:26.856 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:26.856 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.856 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.856 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:26.857 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.827929+0000", 2024-09-21T21:43:26.857 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:26.857 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.857 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.857 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:26.857 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.829943+0000", 2024-09-21T21:43:26.857 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0020143750000000001 2024-09-21T21:43:26.857 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.857 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.857 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:26.857 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.829948+0000", 2024-09-21T21:43:26.857 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9400000000000001e-06 2024-09-21T21:43:26.857 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.857 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.857 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:26.858 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.829960+0000", 2024-09-21T21:43:26.858 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.2123999999999999e-05 2024-09-21T21:43:26.858 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.858 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.858 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:26.858 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.829968+0000", 2024-09-21T21:43:26.858 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7000000000000008e-06 2024-09-21T21:43:26.858 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.858 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.858 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:26.858 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899370+0000", 2024-09-21T21:43:26.858 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.069401672999999997 2024-09-21T21:43:26.858 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.858 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.858 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:26.859 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899481+0000", 2024-09-21T21:43:26.859 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011145300000000001 2024-09-21T21:43:26.859 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.859 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.859 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:26.859 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.899659+0000", 2024-09-21T21:43:26.859 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00017788600000000001 2024-09-21T21:43:26.859 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:26.859 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:26.859 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:26.859 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.859 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.859 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13595 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 1871872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:26.859 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.830180+0000", 2024-09-21T21:43:26.859 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 95.939214332000006, 2024-09-21T21:43:26.860 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071809244999999994, 2024-09-21T21:43:26.860 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:26.866 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:26.866 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:26.866 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:26.866 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:26.866 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:26.867 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13595 2024-09-21T21:43:26.867 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.867 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:26.867 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.867 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:26.867 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.830180+0000", 2024-09-21T21:43:26.867 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:26.867 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.867 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.867 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:26.867 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.830180+0000", 2024-09-21T21:43:26.867 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:26.867 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.867 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.868 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:26.868 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.831967+0000", 2024-09-21T21:43:26.868 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0017868350000000001 2024-09-21T21:43:26.868 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.868 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.868 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:26.868 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.831971+0000", 2024-09-21T21:43:26.868 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.3950000000000003e-06 2024-09-21T21:43:26.868 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.868 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.868 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:26.868 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.831983+0000", 2024-09-21T21:43:26.868 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1815e-05 2024-09-21T21:43:26.868 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.868 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.869 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:26.869 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.831993+0000", 2024-09-21T21:43:26.869 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.6339999999999996e-06 2024-09-21T21:43:26.869 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.869 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.869 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:26.869 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.901693+0000", 2024-09-21T21:43:26.869 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.069700082999999996 2024-09-21T21:43:26.869 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.869 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.869 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:26.869 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.901813+0000", 2024-09-21T21:43:26.869 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011971 2024-09-21T21:43:26.869 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.870 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.870 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:26.870 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.901989+0000", 2024-09-21T21:43:26.870 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000176773 2024-09-21T21:43:26.870 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:26.870 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:26.870 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:26.870 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.870 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.870 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13601 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 3334144~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:26.870 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.832130+0000", 2024-09-21T21:43:26.870 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 95.937264459999994, 2024-09-21T21:43:26.870 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.071861727, 2024-09-21T21:43:26.871 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:26.871 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:26.871 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:26.871 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:26.871 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:26.871 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:26.871 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13601 2024-09-21T21:43:26.871 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.871 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:26.871 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.871 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:26.871 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.832130+0000", 2024-09-21T21:43:26.871 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:26.871 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.871 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.872 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:26.872 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.832130+0000", 2024-09-21T21:43:26.872 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:26.872 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.872 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.872 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:26.872 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.833939+0000", 2024-09-21T21:43:26.872 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0018094400000000001 2024-09-21T21:43:26.872 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.872 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.872 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:26.872 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.833944+0000", 2024-09-21T21:43:26.872 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5979999999999999e-06 2024-09-21T21:43:26.872 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.872 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.873 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:26.873 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.833953+0000", 2024-09-21T21:43:26.873 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.1460000000000002e-06 2024-09-21T21:43:26.873 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.873 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.873 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:26.873 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.833960+0000", 2024-09-21T21:43:26.873 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.7989999999999996e-06 2024-09-21T21:43:26.873 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.873 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.873 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:26.873 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.903693+0000", 2024-09-21T21:43:26.873 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.069732901 2024-09-21T21:43:26.873 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.874 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.874 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:26.874 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.903814+0000", 2024-09-21T21:43:26.874 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000121585 2024-09-21T21:43:26.874 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.874 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.874 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:26.874 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.903992+0000", 2024-09-21T21:43:26.874 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00017725799999999999 2024-09-21T21:43:26.874 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:26.874 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:26.874 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:26.874 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.874 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.875 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:12342 3.1 3:87417293:::rbd_data.1121faebdfe3.0000000000000004:head [read 4096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:43:26.875 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:14.411725+0000", 2024-09-21T21:43:26.875 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 72.357669393999998, 2024-09-21T21:43:26.875 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0089213029999999, 2024-09-21T21:43:26.875 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:26.875 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:26.875 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:26.875 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:26.875 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:43:26.875 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:43:26.875 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 12342 2024-09-21T21:43:26.875 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.875 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:26.875 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.876 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:26.876 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411725+0000", 2024-09-21T21:43:26.876 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:26.876 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.876 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.876 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:26.876 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411725+0000", 2024-09-21T21:43:26.876 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:26.876 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.876 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.876 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:26.876 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411726+0000", 2024-09-21T21:43:26.876 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.4169999999999999e-06 2024-09-21T21:43:26.876 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.876 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.877 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:26.877 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411734+0000", 2024-09-21T21:43:26.877 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.3799999999999996e-06 2024-09-21T21:43:26.877 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.877 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.877 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:26.877 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411750+0000", 2024-09-21T21:43:26.877 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.6429000000000001e-05 2024-09-21T21:43:26.877 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.877 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.877 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:26.877 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411760+0000", 2024-09-21T21:43:26.877 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.4239999999999999e-06 2024-09-21T21:43:26.877 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.878 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.878 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:26.878 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411828+0000", 2024-09-21T21:43:26.878 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.7935999999999995e-05 2024-09-21T21:43:26.878 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.878 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.878 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "op must wait for map", 2024-09-21T21:43:26.878 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411837+0000", 2024-09-21T21:43:26.878 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.0100000000000001e-06 2024-09-21T21:43:26.878 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.878 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.878 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:26.878 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.414984+0000", 2024-09-21T21:43:26.878 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0031472050000000001 2024-09-21T21:43:26.878 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.879 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.879 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for peered", 2024-09-21T21:43:26.879 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.414994+0000", 2024-09-21T21:43:26.879 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0291e-05 2024-09-21T21:43:26.879 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.879 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.879 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:26.879 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.420353+0000", 2024-09-21T21:43:26.879 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.005358913 2024-09-21T21:43:26.879 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.879 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.879 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:26.879 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.420489+0000", 2024-09-21T21:43:26.879 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00013592699999999999 2024-09-21T21:43:26.879 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.880 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.880 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:26.880 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.420646+0000", 2024-09-21T21:43:26.880 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00015737100000000001 2024-09-21T21:43:26.880 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:26.880 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:26.880 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:26.880 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.880 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.880 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5626.0:12348 3.1 3:87417293:::rbd_data.1121faebdfe3.0000000000000004:head [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:43:26.880 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:14.411793+0000", 2024-09-21T21:43:26.880 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 72.357601848000002, 2024-09-21T21:43:26.880 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.009298384, 2024-09-21T21:43:26.880 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:26.880 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:26.881 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:26.881 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:26.881 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5626", 2024-09-21T21:43:26.881 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:43:26.881 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 12348 2024-09-21T21:43:26.881 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.881 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:26.881 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.881 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:26.881 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411793+0000", 2024-09-21T21:43:26.881 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:26.881 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.881 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.881 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:26.881 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411793+0000", 2024-09-21T21:43:26.882 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:26.882 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.882 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.882 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:26.882 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411793+0000", 2024-09-21T21:43:26.882 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.0999999999999998e-07 2024-09-21T21:43:26.882 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.882 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.882 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:26.882 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411797+0000", 2024-09-21T21:43:26.882 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.49e-06 2024-09-21T21:43:26.882 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.882 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.883 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:26.883 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411809+0000", 2024-09-21T21:43:26.883 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.2510999999999999e-05 2024-09-21T21:43:26.883 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.883 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.883 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:26.883 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411816+0000", 2024-09-21T21:43:26.883 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6420000000000001e-06 2024-09-21T21:43:26.883 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.883 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.883 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:26.883 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411963+0000", 2024-09-21T21:43:26.883 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00014700999999999999 2024-09-21T21:43:26.883 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.883 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.884 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting_for_map not empty", 2024-09-21T21:43:26.884 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.411970+0000", 2024-09-21T21:43:26.884 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.9800000000000001e-06 2024-09-21T21:43:26.884 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.884 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.884 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:26.884 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.415018+0000", 2024-09-21T21:43:26.884 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0030482299999999999 2024-09-21T21:43:26.884 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.884 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.884 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for peered", 2024-09-21T21:43:26.884 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:14.415025+0000", 2024-09-21T21:43:26.884 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.3749999999999997e-06 2024-09-21T21:43:26.884 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.885 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.885 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:26.885 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.420904+0000", 2024-09-21T21:43:26.885 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.0058787339999999 2024-09-21T21:43:26.885 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.885 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.885 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:26.885 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.420988+0000", 2024-09-21T21:43:26.885 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.3955999999999994e-05 2024-09-21T21:43:26.885 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.885 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.885 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:26.885 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:15.421091+0000", 2024-09-21T21:43:26.885 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000102846 2024-09-21T21:43:26.885 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:26.886 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:26.886 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:26.886 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.886 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.886 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.4385.0:20922 3.9 3:920c5256:::rbd_id.image_client.0-clone36:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-09-21T21:43:26.886 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:42:17.811324+0000", 2024-09-21T21:43:26.886 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 68.958070116000002, 2024-09-21T21:43:26.886 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.66951882299999999, 2024-09-21T21:43:26.886 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:26.886 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:26.886 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:26.886 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:26.886 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.4385", 2024-09-21T21:43:26.886 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:43:26.886 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 20922 2024-09-21T21:43:26.887 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.887 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:26.887 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.887 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:26.887 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:17.811324+0000", 2024-09-21T21:43:26.887 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:26.887 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.887 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.887 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:26.887 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:17.811324+0000", 2024-09-21T21:43:26.887 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:26.887 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.887 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.887 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:26.888 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:17.811326+0000", 2024-09-21T21:43:26.888 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.536e-06 2024-09-21T21:43:26.888 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.888 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.888 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:26.888 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:17.811332+0000", 2024-09-21T21:43:26.888 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6699999999999997e-06 2024-09-21T21:43:26.888 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.888 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.888 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:26.888 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:17.811347+0000", 2024-09-21T21:43:26.888 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.4589999999999999e-05 2024-09-21T21:43:26.888 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.888 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.888 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:26.888 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:17.811359+0000", 2024-09-21T21:43:26.889 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1776e-05 2024-09-21T21:43:26.889 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.889 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.889 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:26.889 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:17.811424+0000", 2024-09-21T21:43:26.889 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4795000000000006e-05 2024-09-21T21:43:26.889 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.889 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.889 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "waiting for peered", 2024-09-21T21:43:26.889 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:17.811436+0000", 2024-09-21T21:43:26.889 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.221e-05 2024-09-21T21:43:26.889 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.889 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.889 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:26.889 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:18.480657+0000", 2024-09-21T21:43:26.890 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.66922088999999996 2024-09-21T21:43:26.890 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.890 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.890 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:26.890 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:18.480760+0000", 2024-09-21T21:43:26.890 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000102827 2024-09-21T21:43:26.890 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:26.890 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:26.890 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:26.890 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:42:18.480843+0000", 2024-09-21T21:43:26.890 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.3529000000000006e-05 2024-09-21T21:43:26.890 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:26.890 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:26.890 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:26.891 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:26.891 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:26.891 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:43:26.892 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T21:43:26.893 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:43:26.892+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:43:26.995 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:43:26.994+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:43:27.020 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:43:27.020 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:43:27.020 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-21T21:43:27.020 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:43:27.030 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T21:43:27.045 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T21:43:27.094 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:43:27.092+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:43:27.139 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:43:27.139 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:43:27.139 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:43:27.139 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:43:27.139 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:43:27.147 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T21:43:27.168 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0: {} 2024-09-21T21:43:27.169 INFO:teuthology.orchestra.run.smithi081.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T21:43:27.185 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1: {} 2024-09-21T21:43:27.185 INFO:teuthology.orchestra.run.smithi081.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T21:43:27.196 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:43:27.195+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:43:27.201 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2: {} 2024-09-21T21:43:27.201 INFO:teuthology.orchestra.run.smithi081.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T21:43:27.217 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3: {} 2024-09-21T21:43:27.217 INFO:teuthology.orchestra.run.smithi081.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T21:43:27.234 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4: {} 2024-09-21T21:43:27.234 INFO:teuthology.orchestra.run.smithi081.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T21:43:27.250 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5: {} 2024-09-21T21:43:27.250 INFO:teuthology.orchestra.run.smithi081.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T21:43:27.255 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:27.255 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:27.264 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T21:43:27.268 INFO:teuthology.orchestra.run.smithi081.stderr:nodeep-scrub is set 2024-09-21T21:43:27.306 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:43:27.304+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:43:27.397 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:43:27.396+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:43:27.416 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:27.417 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:27.425 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T21:43:27.498 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:43:27.496+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:43:27.570 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:27.570 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:27.580 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T21:43:27.599 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:43:27.598+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:43:27.683 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:27.683 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:27.692 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T21:43:27.700 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:43:27.699+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:43:27.796 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:27.796 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:27.800 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:43:27.799+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:43:27.805 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T21:43:27.902 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:43:27.900+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:43:27.948 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:27.948 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:27.957 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T21:43:28.003 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:43:28.001+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:43:28.101 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:28.101 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:28.103 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:43:28.102+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:43:28.110 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T21:43:28.204 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:43:28.203+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:43:28.224 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:28.224 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:28.234 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T21:43:28.305 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:43:28.303+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:43:28.350 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:28.350 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:28.360 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T21:43:28.406 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:43:28.405+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:43:28.503 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:28.503 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:28.507 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:43:28.506+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:43:28.512 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T21:43:28.608 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:43:28.607+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:43:28.615 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:28.615 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:28.624 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T21:43:28.709 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:43:28.708+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:43:28.726 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:28.727 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:28.735 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T21:43:28.810 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:43:28.808+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:43:28.838 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:28.838 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:28.846 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T21:43:28.911 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:43:28.909+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:43:28.950 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:28.950 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:28.958 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T21:43:29.012 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:43:29.011+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:43:29.102 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:29.102 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:29.109 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T21:43:29.113 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:43:29.112+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:43:29.212 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:29.212 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:29.214 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:43:29.213+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:43:29.220 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T21:43:29.262 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T21:43:29.285 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T21:43:29.314 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:43:29.313+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:43:29.344 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:29.345 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:29.354 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T21:43:29.389 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0: {} 2024-09-21T21:43:29.390 INFO:teuthology.orchestra.run.smithi081.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T21:43:29.407 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1: {} 2024-09-21T21:43:29.407 INFO:teuthology.orchestra.run.smithi081.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T21:43:29.415 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:43:29.414+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:43:29.426 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2: {} 2024-09-21T21:43:29.426 INFO:teuthology.orchestra.run.smithi081.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T21:43:29.444 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3: {} 2024-09-21T21:43:29.445 INFO:teuthology.orchestra.run.smithi081.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T21:43:29.462 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4: {} 2024-09-21T21:43:29.462 INFO:teuthology.orchestra.run.smithi081.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T21:43:29.478 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5: {} 2024-09-21T21:43:29.479 INFO:teuthology.orchestra.run.smithi081.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T21:43:29.500 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:43:29.500 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:43:29.500 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:43:29.500 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:43:29.500 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:43:29.509 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T21:43:29.553 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:43:29.552+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:43:29.613 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:43:29.613 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-21T21:43:29.613 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-21T21:43:29.614 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:43:29.614 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.614 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19747 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 7905280~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:29.614 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273058+0000", 2024-09-21T21:43:29.614 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 98.337965169, 2024-09-21T21:43:29.614 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97338591399999996, 2024-09-21T21:43:29.614 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:29.614 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:29.614 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:29.614 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:29.614 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:29.614 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:29.614 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19747 2024-09-21T21:43:29.614 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.615 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:29.615 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.615 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:29.615 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273058+0000", 2024-09-21T21:43:29.615 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:29.615 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.615 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.615 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:29.615 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273058+0000", 2024-09-21T21:43:29.615 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:29.615 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.615 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.615 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:29.615 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273058+0000", 2024-09-21T21:43:29.616 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5900000000000003e-07 2024-09-21T21:43:29.616 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.616 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.616 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:29.616 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273061+0000", 2024-09-21T21:43:29.616 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3479999999999998e-06 2024-09-21T21:43:29.616 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.616 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.616 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:29.616 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273067+0000", 2024-09-21T21:43:29.616 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1469999999999998e-06 2024-09-21T21:43:29.616 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.616 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.616 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:29.617 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273071+0000", 2024-09-21T21:43:29.617 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7339999999999999e-06 2024-09-21T21:43:29.617 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.617 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.617 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:29.617 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277945+0000", 2024-09-21T21:43:29.617 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.004873474 2024-09-21T21:43:29.617 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.617 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.617 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:29.618 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277948+0000", 2024-09-21T21:43:29.618 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0299999999999998e-06 2024-09-21T21:43:29.618 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.618 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.618 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:29.618 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246269+0000", 2024-09-21T21:43:29.618 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96832060200000003 2024-09-21T21:43:29.618 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.618 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.618 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:29.618 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246362+0000", 2024-09-21T21:43:29.618 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3867000000000001e-05 2024-09-21T21:43:29.619 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.619 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.619 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:29.619 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246444+0000", 2024-09-21T21:43:29.619 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1353000000000005e-05 2024-09-21T21:43:29.619 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:29.619 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:29.619 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:29.619 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.619 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.619 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19755 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 5783552~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:29.619 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273077+0000", 2024-09-21T21:43:29.619 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 98.337945552999997, 2024-09-21T21:43:29.619 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97350203400000002, 2024-09-21T21:43:29.620 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:29.620 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:29.620 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:29.620 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:29.620 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:29.620 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:29.620 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19755 2024-09-21T21:43:29.620 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.620 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:29.620 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.620 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:29.620 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273077+0000", 2024-09-21T21:43:29.620 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:29.620 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.620 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.621 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:29.621 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273077+0000", 2024-09-21T21:43:29.621 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:29.621 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.621 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.621 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:29.621 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273078+0000", 2024-09-21T21:43:29.621 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5699999999999998e-07 2024-09-21T21:43:29.621 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.621 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.621 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:29.621 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273083+0000", 2024-09-21T21:43:29.621 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8670000000000002e-06 2024-09-21T21:43:29.621 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.622 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.622 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:29.622 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273088+0000", 2024-09-21T21:43:29.622 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8370000000000002e-06 2024-09-21T21:43:29.622 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.622 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.622 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:29.622 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273093+0000", 2024-09-21T21:43:29.622 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5709999999999999e-06 2024-09-21T21:43:29.622 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.622 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.622 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:29.622 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277968+0000", 2024-09-21T21:43:29.622 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048753909999999998 2024-09-21T21:43:29.622 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.623 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.623 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:29.623 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277971+0000", 2024-09-21T21:43:29.623 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0189999999999998e-06 2024-09-21T21:43:29.623 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.623 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.623 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:29.623 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246485+0000", 2024-09-21T21:43:29.623 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96851399299999996 2024-09-21T21:43:29.623 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.623 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.623 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:29.623 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246536+0000", 2024-09-21T21:43:29.623 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0430000000000003e-05 2024-09-21T21:43:29.624 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.624 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.624 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:29.624 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246579+0000", 2024-09-21T21:43:29.624 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3569000000000002e-05 2024-09-21T21:43:29.624 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:29.624 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:29.624 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:29.624 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.624 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.624 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19756 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6500352~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:29.624 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273099+0000", 2024-09-21T21:43:29.624 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 98.337924180000002, 2024-09-21T21:43:29.624 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97359272799999996, 2024-09-21T21:43:29.624 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:29.625 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:29.625 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:29.625 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:29.625 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:29.625 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:29.625 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19756 2024-09-21T21:43:29.625 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.625 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:29.625 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.625 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:29.626 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273099+0000", 2024-09-21T21:43:29.626 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:29.626 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.626 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.626 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:29.626 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273099+0000", 2024-09-21T21:43:29.626 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:29.626 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.626 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.626 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:29.626 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273099+0000", 2024-09-21T21:43:29.626 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0900000000000002e-07 2024-09-21T21:43:29.626 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.627 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.627 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:29.627 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273101+0000", 2024-09-21T21:43:29.627 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0320000000000002e-06 2024-09-21T21:43:29.627 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.627 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.627 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:29.627 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273107+0000", 2024-09-21T21:43:29.627 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2830000000000001e-06 2024-09-21T21:43:29.627 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.627 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.627 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:29.627 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273111+0000", 2024-09-21T21:43:29.627 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2230000000000001e-06 2024-09-21T21:43:29.627 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.628 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.628 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:29.628 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.277994+0000", 2024-09-21T21:43:29.628 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048837070000000002 2024-09-21T21:43:29.628 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.628 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.628 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:29.628 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278000+0000", 2024-09-21T21:43:29.628 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3149999999999999e-06 2024-09-21T21:43:29.628 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.628 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.628 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:29.628 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246607+0000", 2024-09-21T21:43:29.629 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.968606839 2024-09-21T21:43:29.629 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.629 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.629 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:29.629 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246651+0000", 2024-09-21T21:43:29.629 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4734999999999998e-05 2024-09-21T21:43:29.629 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.629 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.629 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:29.629 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246692+0000", 2024-09-21T21:43:29.629 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0185000000000002e-05 2024-09-21T21:43:29.629 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:29.629 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:29.629 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:29.630 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.630 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.630 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19757 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 5812224~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:29.630 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273116+0000", 2024-09-21T21:43:29.630 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 98.337906523000001, 2024-09-21T21:43:29.630 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97367813999999997, 2024-09-21T21:43:29.630 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:29.630 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:29.630 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:29.630 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:29.630 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:29.630 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:29.630 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19757 2024-09-21T21:43:29.630 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.631 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:29.631 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.631 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:29.631 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273116+0000", 2024-09-21T21:43:29.631 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:29.631 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.631 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.631 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:29.631 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273116+0000", 2024-09-21T21:43:29.631 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:29.631 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.631 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.631 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:29.631 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273117+0000", 2024-09-21T21:43:29.632 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0999999999999999e-07 2024-09-21T21:43:29.632 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.632 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.632 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:29.632 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273119+0000", 2024-09-21T21:43:29.632 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9970000000000001e-06 2024-09-21T21:43:29.632 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.632 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.632 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:29.632 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273125+0000", 2024-09-21T21:43:29.632 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9540000000000001e-06 2024-09-21T21:43:29.632 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.632 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.632 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:29.633 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273135+0000", 2024-09-21T21:43:29.633 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0597999999999999e-05 2024-09-21T21:43:29.633 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.633 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.633 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:29.633 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278019+0000", 2024-09-21T21:43:29.633 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048834849999999999 2024-09-21T21:43:29.633 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.633 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.633 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:29.633 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278022+0000", 2024-09-21T21:43:29.633 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9560000000000002e-06 2024-09-21T21:43:29.633 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.633 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.633 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:29.634 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246716+0000", 2024-09-21T21:43:29.634 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.968694202 2024-09-21T21:43:29.634 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.634 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.634 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:29.634 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246758+0000", 2024-09-21T21:43:29.634 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1607000000000003e-05 2024-09-21T21:43:29.634 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.634 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.634 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:29.634 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246795+0000", 2024-09-21T21:43:29.634 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6931000000000003e-05 2024-09-21T21:43:29.634 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:29.634 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:29.635 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:29.635 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.635 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.635 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19758 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6529024~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:29.635 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273142+0000", 2024-09-21T21:43:29.635 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 98.337881302, 2024-09-21T21:43:29.635 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97375924599999997, 2024-09-21T21:43:29.635 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:29.635 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:29.635 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:29.635 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:29.635 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:29.635 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:29.636 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19758 2024-09-21T21:43:29.636 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.636 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:29.636 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.636 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:29.636 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273142+0000", 2024-09-21T21:43:29.636 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:29.636 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.636 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.636 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:29.636 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273142+0000", 2024-09-21T21:43:29.636 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:29.636 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.636 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.637 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:29.637 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273142+0000", 2024-09-21T21:43:29.637 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0200000000000003e-07 2024-09-21T21:43:29.637 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.637 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.637 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:29.637 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273144+0000", 2024-09-21T21:43:29.637 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1390000000000002e-06 2024-09-21T21:43:29.637 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.637 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.637 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:29.637 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273150+0000", 2024-09-21T21:43:29.637 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6409999999999999e-06 2024-09-21T21:43:29.637 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.638 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.638 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:29.638 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273157+0000", 2024-09-21T21:43:29.638 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1359999999999999e-06 2024-09-21T21:43:29.638 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.638 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.638 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:29.638 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278050+0000", 2024-09-21T21:43:29.638 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048926170000000001 2024-09-21T21:43:29.638 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.638 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.638 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:29.639 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278053+0000", 2024-09-21T21:43:29.639 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4879999999999999e-06 2024-09-21T21:43:29.639 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.639 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.639 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:29.639 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246819+0000", 2024-09-21T21:43:29.639 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96876590399999996 2024-09-21T21:43:29.639 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.639 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.639 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:29.639 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246856+0000", 2024-09-21T21:43:29.639 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7287999999999999e-05 2024-09-21T21:43:29.639 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.639 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.640 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:29.640 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246901+0000", 2024-09-21T21:43:29.640 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4631e-05 2024-09-21T21:43:29.640 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:29.640 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:29.640 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:29.640 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.640 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.640 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19759 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 7933952~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:29.640 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273172+0000", 2024-09-21T21:43:29.640 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 98.337851037999997, 2024-09-21T21:43:29.640 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97384789500000002, 2024-09-21T21:43:29.640 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:29.640 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:29.640 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:29.641 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:29.641 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:29.641 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:29.641 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19759 2024-09-21T21:43:29.641 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.641 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:29.641 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.641 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:29.641 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273172+0000", 2024-09-21T21:43:29.641 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:29.641 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.641 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.641 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:29.641 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273172+0000", 2024-09-21T21:43:29.642 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:29.642 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.642 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.642 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:29.642 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273172+0000", 2024-09-21T21:43:29.642 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6100000000000002e-07 2024-09-21T21:43:29.642 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.642 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.642 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:29.642 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273174+0000", 2024-09-21T21:43:29.642 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.691e-06 2024-09-21T21:43:29.642 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.642 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.642 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:29.643 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273178+0000", 2024-09-21T21:43:29.643 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7579999999999998e-06 2024-09-21T21:43:29.643 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.643 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.643 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:29.643 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273180+0000", 2024-09-21T21:43:29.643 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7199999999999998e-06 2024-09-21T21:43:29.643 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.643 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.643 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:29.643 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278077+0000", 2024-09-21T21:43:29.643 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048969529999999999 2024-09-21T21:43:29.643 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.643 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.644 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:29.644 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278080+0000", 2024-09-21T21:43:29.644 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.066e-06 2024-09-21T21:43:29.644 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.644 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.644 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:29.644 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246925+0000", 2024-09-21T21:43:29.644 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96884490199999995 2024-09-21T21:43:29.644 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.644 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.644 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:29.644 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.246974+0000", 2024-09-21T21:43:29.644 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8387999999999998e-05 2024-09-21T21:43:29.644 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.644 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.645 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:29.645 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247020+0000", 2024-09-21T21:43:29.645 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6156000000000003e-05 2024-09-21T21:43:29.645 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:29.645 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:29.645 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:29.645 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.645 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.645 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19760 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 7991296~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:29.645 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273220+0000", 2024-09-21T21:43:29.645 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 98.337802523999997, 2024-09-21T21:43:29.645 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97403125499999998, 2024-09-21T21:43:29.645 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:29.645 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:29.646 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:29.646 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:29.646 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:29.646 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:29.646 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19760 2024-09-21T21:43:29.646 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.646 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:29.646 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.646 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:29.646 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273220+0000", 2024-09-21T21:43:29.646 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:29.646 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.646 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.647 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:29.647 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273220+0000", 2024-09-21T21:43:29.647 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:29.647 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.647 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.647 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:29.647 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273221+0000", 2024-09-21T21:43:29.647 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.65e-07 2024-09-21T21:43:29.647 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.647 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.647 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:29.647 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273228+0000", 2024-09-21T21:43:29.647 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6040000000000004e-06 2024-09-21T21:43:29.647 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.648 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.648 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:29.648 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273233+0000", 2024-09-21T21:43:29.648 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1569999999999999e-06 2024-09-21T21:43:29.648 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.648 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.648 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:29.648 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273238+0000", 2024-09-21T21:43:29.648 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0050000000000004e-06 2024-09-21T21:43:29.648 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.648 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.648 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:29.648 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278107+0000", 2024-09-21T21:43:29.648 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00486857 2024-09-21T21:43:29.649 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.649 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.649 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:29.649 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278111+0000", 2024-09-21T21:43:29.649 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6389999999999998e-06 2024-09-21T21:43:29.649 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.649 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.649 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:29.649 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247069+0000", 2024-09-21T21:43:29.649 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96895796899999997 2024-09-21T21:43:29.649 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.649 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.649 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:29.649 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247166+0000", 2024-09-21T21:43:29.650 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.7158999999999996e-05 2024-09-21T21:43:29.650 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.650 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.650 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:29.650 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247252+0000", 2024-09-21T21:43:29.650 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5886999999999999e-05 2024-09-21T21:43:29.650 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:29.650 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:29.650 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:29.650 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.651 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.651 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19761 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 7962624~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:29.651 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273246+0000", 2024-09-21T21:43:29.651 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 98.337777463999998, 2024-09-21T21:43:29.651 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97415972200000001, 2024-09-21T21:43:29.651 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:29.651 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:29.651 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:29.651 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:29.651 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:29.651 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:29.651 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19761 2024-09-21T21:43:29.651 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.652 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:29.652 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.652 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:29.652 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273246+0000", 2024-09-21T21:43:29.652 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:29.652 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.652 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.652 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:29.652 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273246+0000", 2024-09-21T21:43:29.652 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:29.652 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.652 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.652 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:29.652 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273246+0000", 2024-09-21T21:43:29.653 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5100000000000001e-07 2024-09-21T21:43:29.653 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.653 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.653 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:29.653 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273248+0000", 2024-09-21T21:43:29.653 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1550000000000001e-06 2024-09-21T21:43:29.653 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.653 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.653 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:29.654 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273254+0000", 2024-09-21T21:43:29.654 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.964e-06 2024-09-21T21:43:29.654 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.654 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.654 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:29.654 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273259+0000", 2024-09-21T21:43:29.654 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5499999999999996e-06 2024-09-21T21:43:29.655 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.655 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.655 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:29.655 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278130+0000", 2024-09-21T21:43:29.655 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048715779999999997 2024-09-21T21:43:29.655 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.655 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.655 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:29.655 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278133+0000", 2024-09-21T21:43:29.655 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8739999999999999e-06 2024-09-21T21:43:29.655 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.655 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.655 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:29.655 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247270+0000", 2024-09-21T21:43:29.656 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.969137099 2024-09-21T21:43:29.656 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.656 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.656 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:29.656 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247338+0000", 2024-09-21T21:43:29.656 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7717000000000002e-05 2024-09-21T21:43:29.656 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.656 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.656 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:29.656 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247405+0000", 2024-09-21T21:43:29.656 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7434000000000005e-05 2024-09-21T21:43:29.656 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:29.656 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:29.656 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:29.657 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.657 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.657 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19769 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 5840896~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:29.657 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273287+0000", 2024-09-21T21:43:29.657 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 98.337736179999993, 2024-09-21T21:43:29.657 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97435390200000005, 2024-09-21T21:43:29.657 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:29.657 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:29.657 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:29.657 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:29.657 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:29.657 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:29.657 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19769 2024-09-21T21:43:29.657 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.657 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:29.658 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.658 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:29.658 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273287+0000", 2024-09-21T21:43:29.658 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:29.658 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.658 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.658 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:29.658 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273287+0000", 2024-09-21T21:43:29.658 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:29.658 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.658 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.658 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:29.658 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273287+0000", 2024-09-21T21:43:29.658 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6699999999999999e-07 2024-09-21T21:43:29.659 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.659 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.659 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:29.659 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273289+0000", 2024-09-21T21:43:29.659 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.708e-06 2024-09-21T21:43:29.659 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.660 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.660 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:29.660 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273296+0000", 2024-09-21T21:43:29.660 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8109999999999997e-06 2024-09-21T21:43:29.660 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.660 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.660 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:29.660 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273299+0000", 2024-09-21T21:43:29.660 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0740000000000001e-06 2024-09-21T21:43:29.660 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.660 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.660 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:29.660 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278159+0000", 2024-09-21T21:43:29.660 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048603240000000001 2024-09-21T21:43:29.661 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.661 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.661 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:29.661 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278162+0000", 2024-09-21T21:43:29.661 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1930000000000002e-06 2024-09-21T21:43:29.661 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.661 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.661 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:29.661 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247444+0000", 2024-09-21T21:43:29.661 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96928199100000001 2024-09-21T21:43:29.661 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.661 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.661 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:29.661 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247533+0000", 2024-09-21T21:43:29.662 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8490000000000001e-05 2024-09-21T21:43:29.662 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.662 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.662 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:29.662 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247641+0000", 2024-09-21T21:43:29.662 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000107944 2024-09-21T21:43:29.662 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:29.662 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:29.662 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:29.662 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.662 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.662 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19770 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 5869568~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:29.662 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273313+0000", 2024-09-21T21:43:29.662 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 98.337710078000001, 2024-09-21T21:43:29.663 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97449038899999996, 2024-09-21T21:43:29.663 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:29.663 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:29.663 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:29.663 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:29.663 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:29.663 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:29.663 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19770 2024-09-21T21:43:29.663 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.663 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:29.663 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.663 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:29.663 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273313+0000", 2024-09-21T21:43:29.664 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:29.664 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.664 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.664 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:29.664 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273313+0000", 2024-09-21T21:43:29.664 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:29.664 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.664 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.664 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:29.664 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273313+0000", 2024-09-21T21:43:29.664 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5699999999999998e-07 2024-09-21T21:43:29.664 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.664 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.664 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:29.665 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273315+0000", 2024-09-21T21:43:29.665 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0099999999999998e-06 2024-09-21T21:43:29.665 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.665 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.665 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:29.665 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273321+0000", 2024-09-21T21:43:29.665 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7529999999999998e-06 2024-09-21T21:43:29.665 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.665 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.665 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:29.665 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273328+0000", 2024-09-21T21:43:29.665 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9009999999999997e-06 2024-09-21T21:43:29.665 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.665 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.665 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:29.666 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278183+0000", 2024-09-21T21:43:29.666 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048547770000000002 2024-09-21T21:43:29.666 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.666 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.666 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:29.666 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278186+0000", 2024-09-21T21:43:29.666 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3100000000000001e-06 2024-09-21T21:43:29.666 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.666 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.666 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:29.666 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247676+0000", 2024-09-21T21:43:29.667 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96949004299999997 2024-09-21T21:43:29.667 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.667 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.667 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:29.667 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247757+0000", 2024-09-21T21:43:29.667 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0479e-05 2024-09-21T21:43:29.667 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.667 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.667 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:29.667 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247803+0000", 2024-09-21T21:43:29.667 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6758999999999997e-05 2024-09-21T21:43:29.667 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:29.667 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:29.667 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:29.668 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.668 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.668 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19771 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6586368~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:29.668 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273335+0000", 2024-09-21T21:43:29.668 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 98.337688291000006, 2024-09-21T21:43:29.668 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97458947699999998, 2024-09-21T21:43:29.668 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:29.668 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:29.668 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:29.668 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:29.668 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:29.668 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:29.668 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19771 2024-09-21T21:43:29.668 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.669 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:29.669 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.669 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:29.669 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273335+0000", 2024-09-21T21:43:29.669 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:29.669 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.669 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.669 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:29.669 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273335+0000", 2024-09-21T21:43:29.669 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:29.669 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.669 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.669 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:29.669 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273335+0000", 2024-09-21T21:43:29.669 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3599999999999999e-07 2024-09-21T21:43:29.670 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.670 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.670 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:29.670 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273337+0000", 2024-09-21T21:43:29.670 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0789999999999999e-06 2024-09-21T21:43:29.670 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.670 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.670 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:29.670 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273343+0000", 2024-09-21T21:43:29.671 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6280000000000002e-06 2024-09-21T21:43:29.671 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.671 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.671 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:29.671 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273346+0000", 2024-09-21T21:43:29.671 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.507e-06 2024-09-21T21:43:29.671 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.671 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.671 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:29.671 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278206+0000", 2024-09-21T21:43:29.671 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048600969999999999 2024-09-21T21:43:29.671 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.671 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.672 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:29.672 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278212+0000", 2024-09-21T21:43:29.672 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2079999999999999e-06 2024-09-21T21:43:29.672 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.672 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.672 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:29.672 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247828+0000", 2024-09-21T21:43:29.672 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.969616547 2024-09-21T21:43:29.672 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.672 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.672 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:29.672 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247881+0000", 2024-09-21T21:43:29.672 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3180999999999998e-05 2024-09-21T21:43:29.672 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.672 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.673 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:29.673 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247924+0000", 2024-09-21T21:43:29.673 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2894e-05 2024-09-21T21:43:29.673 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:29.673 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:29.673 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:29.673 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.673 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.673 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19772 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6557696~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:29.673 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273358+0000", 2024-09-21T21:43:29.673 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 98.337664911000005, 2024-09-21T21:43:29.673 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97467031800000004, 2024-09-21T21:43:29.673 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:29.673 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:29.674 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:29.674 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:29.674 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:29.674 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:29.674 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19772 2024-09-21T21:43:29.674 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.674 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:29.674 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.674 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:29.674 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273358+0000", 2024-09-21T21:43:29.674 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:29.674 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.674 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.675 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:29.675 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273358+0000", 2024-09-21T21:43:29.675 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:29.675 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.675 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.675 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:29.675 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273358+0000", 2024-09-21T21:43:29.675 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4499999999999998e-07 2024-09-21T21:43:29.675 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.675 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.675 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:29.675 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273360+0000", 2024-09-21T21:43:29.675 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9190000000000002e-06 2024-09-21T21:43:29.675 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.675 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.676 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:29.676 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273366+0000", 2024-09-21T21:43:29.676 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.891e-06 2024-09-21T21:43:29.676 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.676 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.676 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:29.676 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273371+0000", 2024-09-21T21:43:29.676 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1880000000000001e-06 2024-09-21T21:43:29.676 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.676 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.676 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:29.676 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278237+0000", 2024-09-21T21:43:29.676 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.004865472 2024-09-21T21:43:29.676 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.677 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.677 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:29.677 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278240+0000", 2024-09-21T21:43:29.677 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9550000000000001e-06 2024-09-21T21:43:29.677 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.677 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.677 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:29.677 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247949+0000", 2024-09-21T21:43:29.677 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.969709127 2024-09-21T21:43:29.677 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.677 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.677 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:29.677 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.247990+0000", 2024-09-21T21:43:29.677 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1202000000000003e-05 2024-09-21T21:43:29.677 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.678 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.678 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:29.678 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248028+0000", 2024-09-21T21:43:29.678 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8219000000000001e-05 2024-09-21T21:43:29.678 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:29.678 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:29.678 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:29.678 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.678 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.678 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19773 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6615040~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:29.678 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273390+0000", 2024-09-21T21:43:29.678 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 98.337633049999994, 2024-09-21T21:43:29.678 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97476086200000001, 2024-09-21T21:43:29.678 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:29.679 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:29.679 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:29.679 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:29.679 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:29.679 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:29.679 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19773 2024-09-21T21:43:29.679 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.679 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:29.679 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.679 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:29.679 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273390+0000", 2024-09-21T21:43:29.679 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:29.679 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.679 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.679 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:29.680 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273390+0000", 2024-09-21T21:43:29.680 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:29.680 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.680 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.680 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:29.680 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273390+0000", 2024-09-21T21:43:29.680 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9999999999999999e-07 2024-09-21T21:43:29.680 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.680 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.680 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:29.680 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273392+0000", 2024-09-21T21:43:29.680 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0640000000000001e-06 2024-09-21T21:43:29.680 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.680 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.681 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:29.681 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273397+0000", 2024-09-21T21:43:29.681 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.113e-06 2024-09-21T21:43:29.681 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.681 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.681 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:29.681 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273401+0000", 2024-09-21T21:43:29.681 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3119999999999998e-06 2024-09-21T21:43:29.681 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.681 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.681 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:29.681 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278271+0000", 2024-09-21T21:43:29.681 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048698880000000002 2024-09-21T21:43:29.681 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.681 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.682 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:29.682 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278278+0000", 2024-09-21T21:43:29.682 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1210000000000001e-06 2024-09-21T21:43:29.682 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.682 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.682 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:29.682 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248060+0000", 2024-09-21T21:43:29.682 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96978224999999996 2024-09-21T21:43:29.682 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.682 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.682 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:29.682 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248111+0000", 2024-09-21T21:43:29.682 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.096e-05 2024-09-21T21:43:29.682 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.682 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.683 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:29.683 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248151+0000", 2024-09-21T21:43:29.683 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9854000000000002e-05 2024-09-21T21:43:29.683 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:29.683 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:29.683 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:29.683 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.683 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.683 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19784 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6643712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:29.683 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273405+0000", 2024-09-21T21:43:29.683 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 98.337618328999994, 2024-09-21T21:43:29.683 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97491798900000004, 2024-09-21T21:43:29.683 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:29.683 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:29.684 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:29.684 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:29.684 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:29.684 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:29.684 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19784 2024-09-21T21:43:29.684 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.684 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:29.684 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.684 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:29.684 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273405+0000", 2024-09-21T21:43:29.684 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:29.684 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.684 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.684 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:29.684 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273405+0000", 2024-09-21T21:43:29.685 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:29.685 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.685 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.685 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:29.685 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273405+0000", 2024-09-21T21:43:29.685 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2100000000000001e-07 2024-09-21T21:43:29.685 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.685 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.685 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:29.685 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273406+0000", 2024-09-21T21:43:29.685 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2049999999999999e-06 2024-09-21T21:43:29.685 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.685 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.685 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:29.686 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273409+0000", 2024-09-21T21:43:29.686 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0599999999999999e-06 2024-09-21T21:43:29.686 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.686 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.686 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:29.686 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273412+0000", 2024-09-21T21:43:29.686 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0730000000000001e-06 2024-09-21T21:43:29.686 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.686 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.686 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:29.686 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278317+0000", 2024-09-21T21:43:29.686 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0049045520000000004 2024-09-21T21:43:29.686 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.686 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.686 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:29.687 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278321+0000", 2024-09-21T21:43:29.687 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6249999999999998e-06 2024-09-21T21:43:29.687 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.687 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.687 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:29.687 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248179+0000", 2024-09-21T21:43:29.687 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96985744500000004 2024-09-21T21:43:29.687 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.687 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.687 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:29.687 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248253+0000", 2024-09-21T21:43:29.687 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3829000000000001e-05 2024-09-21T21:43:29.687 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.687 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.688 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:29.688 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248323+0000", 2024-09-21T21:43:29.688 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9979000000000002e-05 2024-09-21T21:43:29.688 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:29.688 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:29.688 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:29.688 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.688 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.688 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19785 3.d 3:b538ede9:::rbd_data.11216538973.0000000000000009:1e [read 6672384~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:29.688 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273415+0000", 2024-09-21T21:43:29.688 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 98.337607743000007, 2024-09-21T21:43:29.688 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97500777800000005, 2024-09-21T21:43:29.688 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:29.688 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:29.688 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:29.689 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:29.689 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:29.689 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:29.689 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19785 2024-09-21T21:43:29.689 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.689 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:29.689 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.689 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:29.689 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273415+0000", 2024-09-21T21:43:29.689 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:29.689 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.689 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.689 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:29.689 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273415+0000", 2024-09-21T21:43:29.689 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:29.690 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.690 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.690 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:29.690 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273415+0000", 2024-09-21T21:43:29.690 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0100000000000001e-07 2024-09-21T21:43:29.690 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.690 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.690 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:29.690 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273417+0000", 2024-09-21T21:43:29.690 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.114e-06 2024-09-21T21:43:29.690 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.690 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.690 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:29.690 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273420+0000", 2024-09-21T21:43:29.691 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6770000000000001e-06 2024-09-21T21:43:29.691 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.691 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.691 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:29.691 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273423+0000", 2024-09-21T21:43:29.691 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5450000000000002e-06 2024-09-21T21:43:29.691 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.691 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.691 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:29.691 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278355+0000", 2024-09-21T21:43:29.691 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0049318590000000002 2024-09-21T21:43:29.691 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.691 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.691 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:29.691 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278360+0000", 2024-09-21T21:43:29.692 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3079999999999998e-06 2024-09-21T21:43:29.692 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.692 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.692 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:29.692 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248349+0000", 2024-09-21T21:43:29.692 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.96998909899999997 2024-09-21T21:43:29.692 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.692 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.692 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:29.692 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248387+0000", 2024-09-21T21:43:29.692 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7657000000000002e-05 2024-09-21T21:43:29.692 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.692 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.692 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:29.693 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248423+0000", 2024-09-21T21:43:29.693 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6318e-05 2024-09-21T21:43:29.693 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:29.693 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:29.693 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:29.693 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.693 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.693 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19842 3.d 3:b7fa2ca0:::rbd_data.112157a479c.000000000000004d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:29.693 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.273426+0000", 2024-09-21T21:43:29.693 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 98.337596691000002, 2024-09-21T21:43:29.693 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.98136526800000001, 2024-09-21T21:43:29.693 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:29.693 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:29.693 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:29.693 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:29.694 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:29.694 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:29.694 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19842 2024-09-21T21:43:29.694 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.694 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:29.694 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.694 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:29.694 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273426+0000", 2024-09-21T21:43:29.694 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:29.694 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.694 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.694 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:29.694 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273426+0000", 2024-09-21T21:43:29.694 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:29.695 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.695 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.695 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:29.695 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.273427+0000", 2024-09-21T21:43:29.695 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.72e-07 2024-09-21T21:43:29.695 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.695 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.695 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:29.695 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275836+0000", 2024-09-21T21:43:29.695 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00240909 2024-09-21T21:43:29.695 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.695 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.695 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:29.695 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275844+0000", 2024-09-21T21:43:29.696 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3750000000000003e-06 2024-09-21T21:43:29.696 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.696 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.696 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:29.696 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275849+0000", 2024-09-21T21:43:29.696 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0100000000000003e-06 2024-09-21T21:43:29.696 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.696 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.696 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:29.696 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278394+0000", 2024-09-21T21:43:29.696 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025449890000000001 2024-09-21T21:43:29.696 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.696 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.696 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:29.696 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278402+0000", 2024-09-21T21:43:29.697 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6359999999999994e-06 2024-09-21T21:43:29.697 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.697 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.697 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:29.697 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248454+0000", 2024-09-21T21:43:29.697 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97005259499999996 2024-09-21T21:43:29.697 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.697 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.697 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:29.697 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248576+0000", 2024-09-21T21:43:29.697 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000122127 2024-09-21T21:43:29.697 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.697 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.697 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 2", 2024-09-21T21:43:29.697 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.248844+0000", 2024-09-21T21:43:29.698 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00026784699999999999 2024-09-21T21:43:29.698 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.698 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.698 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:29.698 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.252487+0000", 2024-09-21T21:43:29.698 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0036427920000000002 2024-09-21T21:43:29.698 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.698 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.698 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:29.698 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.254691+0000", 2024-09-21T21:43:29.698 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002203899 2024-09-21T21:43:29.698 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.698 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.698 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:29.699 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.254748+0000", 2024-09-21T21:43:29.699 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7163000000000001e-05 2024-09-21T21:43:29.699 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.699 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.699 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:29.699 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.254792+0000", 2024-09-21T21:43:29.699 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3473000000000002e-05 2024-09-21T21:43:29.699 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:29.699 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:29.699 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:29.699 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.699 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.699 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5177.0:19843 3.d 3:b7fa2ca0:::rbd_data.112157a479c.000000000000004d:head [write 0~0] snapc 21=[21] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-21T21:43:29.699 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:51.275860+0000", 2024-09-21T21:43:29.700 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 98.335162791000002, 2024-09-21T21:43:29.700 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.98183651999999999, 2024-09-21T21:43:29.700 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:29.700 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:29.700 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:29.700 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:29.700 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5177", 2024-09-21T21:43:29.700 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:29.700 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 19843 2024-09-21T21:43:29.700 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.700 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:29.700 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.700 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:29.700 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275860+0000", 2024-09-21T21:43:29.701 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:29.701 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.701 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.701 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:29.701 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275860+0000", 2024-09-21T21:43:29.701 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:29.701 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.701 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.701 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:29.701 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275869+0000", 2024-09-21T21:43:29.701 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6300000000000004e-06 2024-09-21T21:43:29.701 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.701 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.701 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:29.701 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275871+0000", 2024-09-21T21:43:29.702 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2060000000000001e-06 2024-09-21T21:43:29.702 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.702 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.702 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:29.702 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275876+0000", 2024-09-21T21:43:29.702 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8180000000000003e-06 2024-09-21T21:43:29.702 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.702 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.702 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:29.702 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.275880+0000", 2024-09-21T21:43:29.702 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1829999999999996e-06 2024-09-21T21:43:29.702 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.702 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.702 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:29.703 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278416+0000", 2024-09-21T21:43:29.703 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025361149999999998 2024-09-21T21:43:29.703 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.703 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.703 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:29.703 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:51.278419+0000", 2024-09-21T21:43:29.703 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.163e-06 2024-09-21T21:43:29.703 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.703 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.703 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:29.703 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.250249+0000", 2024-09-21T21:43:29.703 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.97182984299999997 2024-09-21T21:43:29.703 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.703 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.704 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:29.704 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.250321+0000", 2024-09-21T21:43:29.704 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1656000000000006e-05 2024-09-21T21:43:29.704 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.704 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.704 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 2", 2024-09-21T21:43:29.704 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.250419+0000", 2024-09-21T21:43:29.704 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.8277999999999997e-05 2024-09-21T21:43:29.704 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.704 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.704 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:29.704 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.252516+0000", 2024-09-21T21:43:29.704 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002097224 2024-09-21T21:43:29.704 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.704 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.705 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:29.705 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.257643+0000", 2024-09-21T21:43:29.705 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0051262319999999997 2024-09-21T21:43:29.705 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.705 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.705 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:29.705 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.257673+0000", 2024-09-21T21:43:29.705 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0023e-05 2024-09-21T21:43:29.705 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.705 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.705 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:29.705 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:52.257697+0000", 2024-09-21T21:43:29.705 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4148999999999999e-05 2024-09-21T21:43:29.705 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:29.706 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:29.706 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:29.706 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.706 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.706 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5626.0:12334 3.3 3:c643a86c:::rbd_data.1121faebdfe3.0000000000000002:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:43:29.706 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:14.411628+0000", 2024-09-21T21:43:29.706 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 75.199394491999996, 2024-09-21T21:43:29.706 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0086947559999999, 2024-09-21T21:43:29.706 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:29.706 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:29.706 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:29.706 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:29.706 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5626", 2024-09-21T21:43:29.706 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:43:29.707 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 12334 2024-09-21T21:43:29.707 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.707 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:29.707 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.707 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:29.707 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411628+0000", 2024-09-21T21:43:29.707 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:29.707 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.707 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.707 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:29.707 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411628+0000", 2024-09-21T21:43:29.707 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:29.707 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.707 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.707 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:29.708 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411629+0000", 2024-09-21T21:43:29.708 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1200000000000002e-07 2024-09-21T21:43:29.708 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.708 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.708 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:29.708 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411633+0000", 2024-09-21T21:43:29.708 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1699999999999999e-06 2024-09-21T21:43:29.708 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.708 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.708 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:29.708 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411645+0000", 2024-09-21T21:43:29.708 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1328999999999999e-05 2024-09-21T21:43:29.708 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.708 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.709 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:29.709 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411652+0000", 2024-09-21T21:43:29.709 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9990000000000002e-06 2024-09-21T21:43:29.709 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.709 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.709 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:29.709 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411717+0000", 2024-09-21T21:43:29.709 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5519999999999996e-05 2024-09-21T21:43:29.709 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.709 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.709 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op must wait for map", 2024-09-21T21:43:29.709 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411725+0000", 2024-09-21T21:43:29.709 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2609999999999996e-06 2024-09-21T21:43:29.709 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.709 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.710 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:29.710 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415904+0000", 2024-09-21T21:43:29.710 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0041792749999999997 2024-09-21T21:43:29.710 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.710 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.710 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:29.710 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415913+0000", 2024-09-21T21:43:29.710 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.054e-06 2024-09-21T21:43:29.710 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.710 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.710 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:29.710 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420096+0000", 2024-09-21T21:43:29.710 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0041835670000001 2024-09-21T21:43:29.710 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.710 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.711 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:29.711 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420216+0000", 2024-09-21T21:43:29.711 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000120025 2024-09-21T21:43:29.711 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.711 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.711 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:29.711 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420323+0000", 2024-09-21T21:43:29.711 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010674400000000001 2024-09-21T21:43:29.711 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:29.711 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:29.711 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:29.711 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.711 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.711 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5626.0:12339 3.3 3:c643a86c:::rbd_data.1121faebdfe3.0000000000000002:head [read 4096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:43:29.712 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:14.411749+0000", 2024-09-21T21:43:29.712 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 75.199274306000007, 2024-09-21T21:43:29.712 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.008818056, 2024-09-21T21:43:29.712 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:29.712 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:29.712 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:29.712 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:29.712 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5626", 2024-09-21T21:43:29.712 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:43:29.712 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 12339 2024-09-21T21:43:29.712 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.712 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:29.712 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.712 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:29.712 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411749+0000", 2024-09-21T21:43:29.713 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:29.713 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.713 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.713 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:29.713 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411749+0000", 2024-09-21T21:43:29.713 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:29.713 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.713 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.713 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:29.713 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411749+0000", 2024-09-21T21:43:29.713 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0900000000000002e-07 2024-09-21T21:43:29.713 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.713 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.713 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:29.714 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411752+0000", 2024-09-21T21:43:29.714 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5610000000000001e-06 2024-09-21T21:43:29.714 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.714 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.714 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:29.714 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411758+0000", 2024-09-21T21:43:29.714 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1890000000000004e-06 2024-09-21T21:43:29.714 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.714 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.714 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:29.714 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411763+0000", 2024-09-21T21:43:29.714 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7589999999999997e-06 2024-09-21T21:43:29.714 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.714 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.714 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:29.715 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411803+0000", 2024-09-21T21:43:29.715 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9922999999999997e-05 2024-09-21T21:43:29.715 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.715 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.715 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting_for_map not empty", 2024-09-21T21:43:29.715 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411809+0000", 2024-09-21T21:43:29.715 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3779999999999998e-06 2024-09-21T21:43:29.715 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.715 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.715 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:29.715 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415937+0000", 2024-09-21T21:43:29.715 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0041276100000000003 2024-09-21T21:43:29.715 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.715 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.716 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:29.716 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415944+0000", 2024-09-21T21:43:29.716 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8329999999999994e-06 2024-09-21T21:43:29.716 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.716 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.716 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:29.716 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420411+0000", 2024-09-21T21:43:29.716 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0044667300000001 2024-09-21T21:43:29.716 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.716 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.716 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:29.716 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420475+0000", 2024-09-21T21:43:29.716 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4356000000000005e-05 2024-09-21T21:43:29.716 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.716 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.717 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:29.717 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420567+0000", 2024-09-21T21:43:29.717 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.1308000000000001e-05 2024-09-21T21:43:29.717 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:29.717 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:29.717 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:29.717 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.717 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.717 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5626.0:12345 3.3 3:c643a86c:::rbd_data.1121faebdfe3.0000000000000002:head [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99)", 2024-09-21T21:43:29.717 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:14.411831+0000", 2024-09-21T21:43:29.717 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 75.199192253000007, 2024-09-21T21:43:29.717 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0089163649999999, 2024-09-21T21:43:29.717 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:29.717 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:29.718 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:29.718 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:29.718 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5626", 2024-09-21T21:43:29.718 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/592092813", 2024-09-21T21:43:29.718 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 12345 2024-09-21T21:43:29.718 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.718 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:29.718 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.718 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:29.718 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411831+0000", 2024-09-21T21:43:29.718 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:29.718 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.718 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.718 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:29.719 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411831+0000", 2024-09-21T21:43:29.719 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:29.719 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.719 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.719 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:29.719 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411831+0000", 2024-09-21T21:43:29.719 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.08e-07 2024-09-21T21:43:29.719 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.719 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.719 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:29.719 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411833+0000", 2024-09-21T21:43:29.719 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3089999999999998e-06 2024-09-21T21:43:29.719 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.719 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.719 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:29.720 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411840+0000", 2024-09-21T21:43:29.720 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.472e-06 2024-09-21T21:43:29.720 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.720 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.720 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:29.720 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411845+0000", 2024-09-21T21:43:29.720 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7910000000000004e-06 2024-09-21T21:43:29.720 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.720 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.720 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:29.720 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411890+0000", 2024-09-21T21:43:29.720 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5763000000000002e-05 2024-09-21T21:43:29.720 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.720 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.721 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting_for_map not empty", 2024-09-21T21:43:29.721 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.411896+0000", 2024-09-21T21:43:29.721 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0190000000000003e-06 2024-09-21T21:43:29.721 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.721 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.721 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:29.721 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415967+0000", 2024-09-21T21:43:29.721 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0040705749999999999 2024-09-21T21:43:29.721 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.721 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.721 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:29.721 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:14.415974+0000", 2024-09-21T21:43:29.721 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3200000000000002e-06 2024-09-21T21:43:29.721 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.721 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.722 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:29.722 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420594+0000", 2024-09-21T21:43:29.722 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.004620093 2024-09-21T21:43:29.722 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.722 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.722 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:29.722 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420658+0000", 2024-09-21T21:43:29.722 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3678999999999998e-05 2024-09-21T21:43:29.722 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:29.722 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:29.722 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:29.722 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:15.420747+0000", 2024-09-21T21:43:29.722 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8936000000000004e-05 2024-09-21T21:43:29.722 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:29.723 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:29.723 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:29.723 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:29.723 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:29.723 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:43:29.724 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T21:43:29.725 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:43:29.723+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:43:29.825 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:43:29.824+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:43:29.868 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:43:29.868 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:43:29.868 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-21T21:43:29.868 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:43:29.878 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T21:43:29.926 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:43:29.925+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:43:29.981 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:43:29.981 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:43:29.981 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:43:29.981 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:43:29.981 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:43:29.989 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T21:43:30.030 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:43:30.029+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:43:30.128 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:43:30.127+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:43:30.134 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:43:30.134 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-21T21:43:30.134 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-21T21:43:30.134 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:43:30.134 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.134 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6974 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5246976~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:30.134 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.922932+0000", 2024-09-21T21:43:30.134 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 120.208971625, 2024-09-21T21:43:30.135 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041773352999999999, 2024-09-21T21:43:30.135 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:30.135 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:30.135 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:30.135 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:30.135 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:30.135 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:30.135 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6974 2024-09-21T21:43:30.135 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.135 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:30.135 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.135 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:30.136 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922932+0000", 2024-09-21T21:43:30.136 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:30.136 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.136 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.136 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:30.136 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922932+0000", 2024-09-21T21:43:30.136 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:30.136 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.136 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.136 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:30.136 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922933+0000", 2024-09-21T21:43:30.136 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.46e-07 2024-09-21T21:43:30.136 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.137 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.137 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:30.137 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922940+0000", 2024-09-21T21:43:30.137 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6140000000000003e-06 2024-09-21T21:43:30.137 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.137 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.137 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:30.137 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922949+0000", 2024-09-21T21:43:30.137 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6899999999999998e-06 2024-09-21T21:43:30.137 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.137 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.137 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:30.138 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922955+0000", 2024-09-21T21:43:30.138 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5140000000000001e-06 2024-09-21T21:43:30.138 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.138 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.138 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:30.138 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964547+0000", 2024-09-21T21:43:30.138 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041592854999999998 2024-09-21T21:43:30.138 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.138 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.138 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:30.138 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964617+0000", 2024-09-21T21:43:30.138 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9058000000000002e-05 2024-09-21T21:43:30.139 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.139 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.139 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:30.139 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964706+0000", 2024-09-21T21:43:30.139 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9276000000000004e-05 2024-09-21T21:43:30.139 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.139 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:30.139 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.139 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.139 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.139 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6976 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5261312~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:30.139 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.922961+0000", 2024-09-21T21:43:30.139 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 120.20894258, 2024-09-21T21:43:30.140 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041863648000000003, 2024-09-21T21:43:30.140 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:30.140 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:30.140 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:30.140 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:30.140 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:30.140 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:30.140 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6976 2024-09-21T21:43:30.140 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.140 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:30.140 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.140 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:30.140 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922961+0000", 2024-09-21T21:43:30.141 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:30.141 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.141 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.141 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:30.141 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922961+0000", 2024-09-21T21:43:30.141 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:30.141 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.141 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.141 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:30.141 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922962+0000", 2024-09-21T21:43:30.142 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.89e-07 2024-09-21T21:43:30.142 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.142 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.142 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:30.142 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922964+0000", 2024-09-21T21:43:30.142 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1270000000000001e-06 2024-09-21T21:43:30.142 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.142 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.142 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:30.142 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922970+0000", 2024-09-21T21:43:30.142 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8030000000000002e-06 2024-09-21T21:43:30.142 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.143 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.143 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:30.143 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922979+0000", 2024-09-21T21:43:30.143 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5559999999999996e-06 2024-09-21T21:43:30.143 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.143 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.143 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:30.143 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964734+0000", 2024-09-21T21:43:30.143 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041754415000000003 2024-09-21T21:43:30.143 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.143 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.143 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:30.143 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964775+0000", 2024-09-21T21:43:30.144 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0806999999999997e-05 2024-09-21T21:43:30.144 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.144 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.144 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:30.144 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964825+0000", 2024-09-21T21:43:30.144 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0550999999999998e-05 2024-09-21T21:43:30.144 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.144 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:30.144 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.144 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.144 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.144 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6978 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5275648~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:30.144 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.922987+0000", 2024-09-21T21:43:30.145 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 120.20891679499999, 2024-09-21T21:43:30.145 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042043876000000001, 2024-09-21T21:43:30.145 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:30.145 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:30.145 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:30.145 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:30.145 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:30.145 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:30.145 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6978 2024-09-21T21:43:30.145 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.145 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:30.145 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.145 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:30.146 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922987+0000", 2024-09-21T21:43:30.146 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:30.146 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.146 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.146 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:30.146 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922987+0000", 2024-09-21T21:43:30.146 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:30.146 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.146 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.146 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:30.146 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922988+0000", 2024-09-21T21:43:30.146 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1199999999999998e-07 2024-09-21T21:43:30.147 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.147 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.147 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:30.147 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922990+0000", 2024-09-21T21:43:30.147 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.142e-06 2024-09-21T21:43:30.147 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.147 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.147 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:30.147 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.922997+0000", 2024-09-21T21:43:30.147 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.199e-06 2024-09-21T21:43:30.147 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.147 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.147 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:30.148 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923004+0000", 2024-09-21T21:43:30.148 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3599999999999998e-06 2024-09-21T21:43:30.148 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.148 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.148 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:30.148 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964856+0000", 2024-09-21T21:43:30.148 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.041851521000000003 2024-09-21T21:43:30.148 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.148 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.148 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:30.148 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.964944+0000", 2024-09-21T21:43:30.148 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7859e-05 2024-09-21T21:43:30.148 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.149 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.149 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:30.149 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965031+0000", 2024-09-21T21:43:30.149 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7383e-05 2024-09-21T21:43:30.149 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.149 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:30.149 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.149 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.149 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.149 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6980 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5289984~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:30.149 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923011+0000", 2024-09-21T21:43:30.149 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 120.20889345800001, 2024-09-21T21:43:30.149 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042139337999999998, 2024-09-21T21:43:30.150 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:30.150 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:30.150 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:30.150 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:30.150 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:30.150 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:30.150 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6980 2024-09-21T21:43:30.150 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.150 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:30.151 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.151 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:30.151 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923011+0000", 2024-09-21T21:43:30.151 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:30.151 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.151 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.151 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:30.151 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923011+0000", 2024-09-21T21:43:30.151 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:30.151 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.151 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.151 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:30.151 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923011+0000", 2024-09-21T21:43:30.152 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6699999999999999e-07 2024-09-21T21:43:30.152 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.152 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.152 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:30.152 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923013+0000", 2024-09-21T21:43:30.152 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0279999999999999e-06 2024-09-21T21:43:30.152 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.152 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.152 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:30.152 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923018+0000", 2024-09-21T21:43:30.152 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4790000000000004e-06 2024-09-21T21:43:30.152 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.152 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.153 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:30.153 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923029+0000", 2024-09-21T21:43:30.153 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0601999999999999e-05 2024-09-21T21:43:30.153 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.153 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.153 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:30.153 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965059+0000", 2024-09-21T21:43:30.153 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042029947999999998 2024-09-21T21:43:30.153 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.153 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.153 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:30.153 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965100+0000", 2024-09-21T21:43:30.153 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1170000000000001e-05 2024-09-21T21:43:30.154 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.154 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.154 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:30.154 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965150+0000", 2024-09-21T21:43:30.154 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9743999999999997e-05 2024-09-21T21:43:30.154 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.154 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:30.154 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.154 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.154 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.154 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6982 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5304320~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:30.155 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923036+0000", 2024-09-21T21:43:30.155 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 120.208867637, 2024-09-21T21:43:30.155 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042225666000000002, 2024-09-21T21:43:30.155 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:30.155 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:30.155 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:30.155 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:30.155 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:30.155 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:30.155 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6982 2024-09-21T21:43:30.155 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.155 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:30.155 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.156 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:30.156 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923036+0000", 2024-09-21T21:43:30.156 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:30.156 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.156 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.156 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:30.156 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923036+0000", 2024-09-21T21:43:30.156 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:30.156 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.156 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.156 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:30.156 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923037+0000", 2024-09-21T21:43:30.157 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2800000000000002e-07 2024-09-21T21:43:30.157 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.157 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.157 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:30.157 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923039+0000", 2024-09-21T21:43:30.157 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1500000000000002e-06 2024-09-21T21:43:30.157 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.157 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.157 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:30.157 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923049+0000", 2024-09-21T21:43:30.157 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0236e-05 2024-09-21T21:43:30.157 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.157 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.158 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:30.158 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923056+0000", 2024-09-21T21:43:30.158 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2540000000000003e-06 2024-09-21T21:43:30.158 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.158 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.158 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:30.158 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965173+0000", 2024-09-21T21:43:30.158 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042116686 2024-09-21T21:43:30.158 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.159 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.159 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:30.159 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965215+0000", 2024-09-21T21:43:30.159 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1696000000000002e-05 2024-09-21T21:43:30.160 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.160 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.160 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:30.160 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965262+0000", 2024-09-21T21:43:30.160 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7216000000000003e-05 2024-09-21T21:43:30.160 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.160 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:30.160 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.160 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.160 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.160 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6984 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5318656~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:30.160 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923063+0000", 2024-09-21T21:43:30.160 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 120.208840983, 2024-09-21T21:43:30.161 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042318651999999998, 2024-09-21T21:43:30.161 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:30.161 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:30.161 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:30.161 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:30.161 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:30.161 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:30.161 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6984 2024-09-21T21:43:30.161 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.161 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:30.161 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.161 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:30.161 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923063+0000", 2024-09-21T21:43:30.162 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:30.162 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.162 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.162 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:30.162 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923063+0000", 2024-09-21T21:43:30.162 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:30.162 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.162 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.162 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:30.162 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923064+0000", 2024-09-21T21:43:30.162 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2e-07 2024-09-21T21:43:30.162 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.162 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.162 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:30.163 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923066+0000", 2024-09-21T21:43:30.163 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.057e-06 2024-09-21T21:43:30.163 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.163 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.163 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:30.163 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923071+0000", 2024-09-21T21:43:30.163 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8810000000000001e-06 2024-09-21T21:43:30.163 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.163 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.163 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:30.163 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923078+0000", 2024-09-21T21:43:30.163 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5479999999999998e-06 2024-09-21T21:43:30.163 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.164 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.164 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:30.164 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965288+0000", 2024-09-21T21:43:30.164 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042209576999999998 2024-09-21T21:43:30.164 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.164 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.164 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:30.164 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965332+0000", 2024-09-21T21:43:30.164 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4366999999999997e-05 2024-09-21T21:43:30.164 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.165 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.165 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:30.165 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965382+0000", 2024-09-21T21:43:30.165 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9802000000000002e-05 2024-09-21T21:43:30.165 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.165 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:30.165 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.165 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.165 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.165 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6987 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5332992~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:30.165 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923085+0000", 2024-09-21T21:43:30.165 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 120.20881945799999, 2024-09-21T21:43:30.165 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042412103999999999, 2024-09-21T21:43:30.165 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:30.166 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:30.166 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:30.166 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:30.166 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:30.166 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:30.166 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6987 2024-09-21T21:43:30.166 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.166 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:30.166 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.166 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:30.166 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923085+0000", 2024-09-21T21:43:30.166 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:30.166 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.166 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.166 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:30.167 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923085+0000", 2024-09-21T21:43:30.167 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:30.167 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.167 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.167 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:30.167 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923085+0000", 2024-09-21T21:43:30.167 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.89e-07 2024-09-21T21:43:30.167 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.167 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.167 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:30.167 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923087+0000", 2024-09-21T21:43:30.167 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0810000000000001e-06 2024-09-21T21:43:30.167 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.167 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.168 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:30.168 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923098+0000", 2024-09-21T21:43:30.168 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1299e-05 2024-09-21T21:43:30.168 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.168 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.168 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:30.168 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923104+0000", 2024-09-21T21:43:30.168 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9100000000000002e-06 2024-09-21T21:43:30.168 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.168 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.168 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:30.168 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965404+0000", 2024-09-21T21:43:30.168 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042299622000000002 2024-09-21T21:43:30.168 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.168 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.169 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:30.169 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965445+0000", 2024-09-21T21:43:30.169 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0701000000000002e-05 2024-09-21T21:43:30.169 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.169 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.169 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:30.169 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965497+0000", 2024-09-21T21:43:30.169 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2101999999999997e-05 2024-09-21T21:43:30.169 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.169 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:30.169 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.169 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.169 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.169 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6989 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5347328~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:30.170 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923111+0000", 2024-09-21T21:43:30.170 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 120.208793118, 2024-09-21T21:43:30.170 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042505188999999999, 2024-09-21T21:43:30.170 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:30.170 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:30.170 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:30.170 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:30.170 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:30.170 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:30.171 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6989 2024-09-21T21:43:30.171 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.171 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:30.171 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.171 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:30.171 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923111+0000", 2024-09-21T21:43:30.171 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:30.171 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.171 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.171 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:30.171 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923111+0000", 2024-09-21T21:43:30.171 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:30.171 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.171 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.171 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:30.172 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923111+0000", 2024-09-21T21:43:30.172 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4499999999999997e-07 2024-09-21T21:43:30.172 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.172 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.172 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:30.172 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923114+0000", 2024-09-21T21:43:30.172 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1569999999999998e-06 2024-09-21T21:43:30.172 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.172 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.172 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:30.172 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923120+0000", 2024-09-21T21:43:30.172 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1759999999999998e-06 2024-09-21T21:43:30.172 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.172 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.172 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:30.173 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923125+0000", 2024-09-21T21:43:30.173 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2560000000000002e-06 2024-09-21T21:43:30.173 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.173 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.173 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:30.173 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965523+0000", 2024-09-21T21:43:30.173 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042398094999999997 2024-09-21T21:43:30.173 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.173 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.173 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:30.173 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965566+0000", 2024-09-21T21:43:30.173 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3393999999999998e-05 2024-09-21T21:43:30.173 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.173 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.173 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:30.174 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965616+0000", 2024-09-21T21:43:30.174 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9666000000000002e-05 2024-09-21T21:43:30.174 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.174 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:30.174 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.174 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.174 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.174 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6991 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5361664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:30.174 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923132+0000", 2024-09-21T21:43:30.174 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 120.208771829, 2024-09-21T21:43:30.174 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042600779999999998, 2024-09-21T21:43:30.174 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:30.174 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:30.174 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:30.175 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:30.175 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:30.175 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:30.175 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6991 2024-09-21T21:43:30.175 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.175 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:30.175 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.175 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:30.175 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923132+0000", 2024-09-21T21:43:30.175 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:30.175 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.175 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.175 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:30.175 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923132+0000", 2024-09-21T21:43:30.175 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:30.176 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.176 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.176 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:30.176 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923132+0000", 2024-09-21T21:43:30.176 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4200000000000002e-07 2024-09-21T21:43:30.176 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.176 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.176 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:30.176 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923134+0000", 2024-09-21T21:43:30.176 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2270000000000001e-06 2024-09-21T21:43:30.176 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.176 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.176 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:30.176 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923137+0000", 2024-09-21T21:43:30.176 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1930000000000002e-06 2024-09-21T21:43:30.177 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.177 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.177 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:30.177 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923139+0000", 2024-09-21T21:43:30.177 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4710000000000001e-06 2024-09-21T21:43:30.177 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.177 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.177 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:30.177 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965639+0000", 2024-09-21T21:43:30.177 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.04249965 2024-09-21T21:43:30.177 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.177 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.177 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:30.177 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965679+0000", 2024-09-21T21:43:30.177 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9852999999999999e-05 2024-09-21T21:43:30.178 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.178 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.178 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:30.178 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965733+0000", 2024-09-21T21:43:30.178 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4144000000000002e-05 2024-09-21T21:43:30.178 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.178 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:30.178 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.178 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.178 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.178 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6993 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5384192~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:30.178 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923143+0000", 2024-09-21T21:43:30.178 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 120.20876103499999, 2024-09-21T21:43:30.178 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042708182999999997, 2024-09-21T21:43:30.178 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:30.179 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:30.179 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:30.179 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:30.179 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:30.179 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:30.179 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6993 2024-09-21T21:43:30.179 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.179 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:30.179 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.179 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:30.179 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923143+0000", 2024-09-21T21:43:30.179 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:30.179 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.179 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.180 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:30.180 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923143+0000", 2024-09-21T21:43:30.180 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:30.180 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.180 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.180 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:30.180 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923143+0000", 2024-09-21T21:43:30.180 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0800000000000001e-07 2024-09-21T21:43:30.180 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.180 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.180 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:30.180 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923144+0000", 2024-09-21T21:43:30.180 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.009e-06 2024-09-21T21:43:30.180 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.181 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.181 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:30.181 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923148+0000", 2024-09-21T21:43:30.181 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.737e-06 2024-09-21T21:43:30.181 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.181 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.181 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:30.181 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923150+0000", 2024-09-21T21:43:30.181 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4669999999999998e-06 2024-09-21T21:43:30.181 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.181 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.181 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:30.181 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965760+0000", 2024-09-21T21:43:30.181 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042609399999999999 2024-09-21T21:43:30.182 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.182 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.182 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:30.182 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965801+0000", 2024-09-21T21:43:30.182 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1399999999999997e-05 2024-09-21T21:43:30.182 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.182 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.182 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:30.182 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965851+0000", 2024-09-21T21:43:30.182 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9962e-05 2024-09-21T21:43:30.182 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.182 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:30.182 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.182 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.182 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.183 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6995 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5398528~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:30.183 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923153+0000", 2024-09-21T21:43:30.183 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 120.208750671, 2024-09-21T21:43:30.183 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042828034000000001, 2024-09-21T21:43:30.183 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:30.183 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:30.183 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:30.183 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:30.183 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:30.183 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:30.183 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6995 2024-09-21T21:43:30.183 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.183 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:30.183 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.184 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:30.184 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923153+0000", 2024-09-21T21:43:30.184 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:30.184 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.184 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.184 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:30.184 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923153+0000", 2024-09-21T21:43:30.184 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:30.184 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.184 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.184 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:30.184 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923154+0000", 2024-09-21T21:43:30.184 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2600000000000001e-07 2024-09-21T21:43:30.184 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.184 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.184 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:30.185 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923158+0000", 2024-09-21T21:43:30.185 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0559999999999998e-06 2024-09-21T21:43:30.185 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.185 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.185 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:30.185 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923161+0000", 2024-09-21T21:43:30.185 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6519999999999999e-06 2024-09-21T21:43:30.185 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.185 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.185 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:30.185 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923170+0000", 2024-09-21T21:43:30.185 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5720000000000003e-06 2024-09-21T21:43:30.185 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.185 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.185 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:30.186 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965887+0000", 2024-09-21T21:43:30.186 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042716863000000001 2024-09-21T21:43:30.186 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.186 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.186 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:30.186 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965927+0000", 2024-09-21T21:43:30.186 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0346999999999998e-05 2024-09-21T21:43:30.186 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.186 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.186 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:30.186 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.965981+0000", 2024-09-21T21:43:30.186 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4317999999999997e-05 2024-09-21T21:43:30.186 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.186 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:30.187 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.187 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.187 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.187 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6997 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5412864~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:30.187 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923174+0000", 2024-09-21T21:43:30.187 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 120.208730516, 2024-09-21T21:43:30.187 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042928296999999997, 2024-09-21T21:43:30.187 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:30.187 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:30.187 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:30.187 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:30.187 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:30.187 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:30.187 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6997 2024-09-21T21:43:30.187 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.188 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:30.188 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.188 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:30.188 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923174+0000", 2024-09-21T21:43:30.188 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:30.188 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.188 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.188 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:30.188 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923174+0000", 2024-09-21T21:43:30.188 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:30.188 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.188 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.188 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:30.188 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923174+0000", 2024-09-21T21:43:30.188 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.67e-07 2024-09-21T21:43:30.189 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.189 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.189 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:30.189 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923178+0000", 2024-09-21T21:43:30.189 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1330000000000001e-06 2024-09-21T21:43:30.189 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.189 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.189 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:30.189 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923182+0000", 2024-09-21T21:43:30.189 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.794e-06 2024-09-21T21:43:30.189 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.189 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.189 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:30.189 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923184+0000", 2024-09-21T21:43:30.189 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.435e-06 2024-09-21T21:43:30.189 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.190 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.190 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:30.190 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966009+0000", 2024-09-21T21:43:30.190 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042824327000000002 2024-09-21T21:43:30.190 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.190 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.190 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:30.190 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966052+0000", 2024-09-21T21:43:30.190 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3034000000000002e-05 2024-09-21T21:43:30.190 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.190 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.191 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:30.191 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966102+0000", 2024-09-21T21:43:30.191 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0306999999999998e-05 2024-09-21T21:43:30.191 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.191 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:30.191 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.191 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.191 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.191 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:6999 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5427200~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:30.191 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923187+0000", 2024-09-21T21:43:30.191 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 120.208716788, 2024-09-21T21:43:30.191 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043104267000000002, 2024-09-21T21:43:30.191 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:30.192 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:30.192 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:30.192 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:30.192 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:30.192 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:30.192 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 6999 2024-09-21T21:43:30.192 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.192 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:30.192 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.192 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:30.192 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923187+0000", 2024-09-21T21:43:30.192 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:30.192 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.192 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.193 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:30.193 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923187+0000", 2024-09-21T21:43:30.193 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:30.193 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.193 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.193 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:30.193 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923188+0000", 2024-09-21T21:43:30.193 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3999999999999998e-07 2024-09-21T21:43:30.193 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.193 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.193 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:30.194 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923189+0000", 2024-09-21T21:43:30.194 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4169999999999999e-06 2024-09-21T21:43:30.194 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.194 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.194 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:30.194 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923194+0000", 2024-09-21T21:43:30.194 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1819999999999997e-06 2024-09-21T21:43:30.194 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.194 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.194 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:30.194 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923198+0000", 2024-09-21T21:43:30.194 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.143e-06 2024-09-21T21:43:30.194 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.194 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.195 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:30.195 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966135+0000", 2024-09-21T21:43:30.195 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.042936297999999998 2024-09-21T21:43:30.195 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.195 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.195 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:30.195 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966204+0000", 2024-09-21T21:43:30.195 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8989e-05 2024-09-21T21:43:30.195 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.195 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.195 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:30.195 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966292+0000", 2024-09-21T21:43:30.195 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.7997999999999993e-05 2024-09-21T21:43:30.195 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.195 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:30.196 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.196 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.196 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.196 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7001 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5441536~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:30.196 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923211+0000", 2024-09-21T21:43:30.196 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 120.208692997, 2024-09-21T21:43:30.196 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043196518000000003, 2024-09-21T21:43:30.196 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:30.196 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:30.196 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:30.196 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:30.196 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:30.196 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:30.196 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7001 2024-09-21T21:43:30.196 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.197 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:30.197 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.197 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:30.197 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923211+0000", 2024-09-21T21:43:30.197 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:30.197 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.197 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.197 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:30.197 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923211+0000", 2024-09-21T21:43:30.197 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:30.197 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.197 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.197 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:30.197 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923212+0000", 2024-09-21T21:43:30.197 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3900000000000005e-07 2024-09-21T21:43:30.198 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.198 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.198 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:30.198 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923214+0000", 2024-09-21T21:43:30.198 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9970000000000001e-06 2024-09-21T21:43:30.198 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.198 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.198 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:30.198 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923219+0000", 2024-09-21T21:43:30.198 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4310000000000002e-06 2024-09-21T21:43:30.198 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.198 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.198 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:30.198 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923224+0000", 2024-09-21T21:43:30.198 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6410000000000001e-06 2024-09-21T21:43:30.199 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.199 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.199 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:30.199 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966318+0000", 2024-09-21T21:43:30.199 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043093842 2024-09-21T21:43:30.199 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.199 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.199 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:30.199 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966359+0000", 2024-09-21T21:43:30.199 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1260999999999997e-05 2024-09-21T21:43:30.199 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.199 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.199 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:30.199 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966408+0000", 2024-09-21T21:43:30.199 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8807000000000002e-05 2024-09-21T21:43:30.200 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.200 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:30.200 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.200 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.200 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.200 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7002 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5455872~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:30.200 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923229+0000", 2024-09-21T21:43:30.200 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 120.208674587, 2024-09-21T21:43:30.200 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043364568999999999, 2024-09-21T21:43:30.200 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:30.200 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:30.200 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:30.200 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:30.200 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:30.200 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:30.201 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7002 2024-09-21T21:43:30.201 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.201 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:30.201 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.201 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:30.201 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923229+0000", 2024-09-21T21:43:30.201 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:30.201 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.201 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.201 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:30.201 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923229+0000", 2024-09-21T21:43:30.201 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:30.201 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.201 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.201 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:30.202 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923230+0000", 2024-09-21T21:43:30.202 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4700000000000002e-07 2024-09-21T21:43:30.202 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.202 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.202 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:30.202 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923232+0000", 2024-09-21T21:43:30.202 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0719999999999998e-06 2024-09-21T21:43:30.202 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.202 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.202 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:30.202 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923237+0000", 2024-09-21T21:43:30.202 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2290000000000002e-06 2024-09-21T21:43:30.202 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.202 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.202 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:30.203 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923245+0000", 2024-09-21T21:43:30.203 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2959999999999999e-06 2024-09-21T21:43:30.203 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.203 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.203 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:30.203 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966438+0000", 2024-09-21T21:43:30.203 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043193027000000002 2024-09-21T21:43:30.203 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.203 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.203 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:30.203 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966506+0000", 2024-09-21T21:43:30.203 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7707999999999996e-05 2024-09-21T21:43:30.203 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.203 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.203 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:30.204 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966594+0000", 2024-09-21T21:43:30.204 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.789e-05 2024-09-21T21:43:30.204 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.204 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:30.204 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.204 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.204 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.204 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7005 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5470208~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:30.204 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923254+0000", 2024-09-21T21:43:30.204 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 120.20865035600001, 2024-09-21T21:43:30.204 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043457176, 2024-09-21T21:43:30.204 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:30.204 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:30.204 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:30.204 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:30.205 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:30.205 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:30.205 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7005 2024-09-21T21:43:30.205 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.205 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:30.205 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.205 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:30.205 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923254+0000", 2024-09-21T21:43:30.205 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:30.205 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.205 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.205 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:30.205 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923254+0000", 2024-09-21T21:43:30.205 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:30.205 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.206 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.206 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:30.206 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923254+0000", 2024-09-21T21:43:30.206 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4700000000000002e-07 2024-09-21T21:43:30.206 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.206 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.206 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:30.206 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923256+0000", 2024-09-21T21:43:30.206 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.007e-06 2024-09-21T21:43:30.206 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.206 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.206 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:30.206 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923266+0000", 2024-09-21T21:43:30.206 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.4700000000000008e-06 2024-09-21T21:43:30.207 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.207 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.207 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:30.207 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923271+0000", 2024-09-21T21:43:30.207 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2809999999999999e-06 2024-09-21T21:43:30.207 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.207 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.207 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:30.207 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966619+0000", 2024-09-21T21:43:30.207 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043347951000000003 2024-09-21T21:43:30.207 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.207 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.207 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:30.207 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966661+0000", 2024-09-21T21:43:30.207 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2614999999999998e-05 2024-09-21T21:43:30.208 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.208 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.208 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:30.208 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966711+0000", 2024-09-21T21:43:30.208 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9505000000000001e-05 2024-09-21T21:43:30.208 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.208 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:30.208 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.208 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.208 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.208 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7008 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5484544~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:30.208 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923278+0000", 2024-09-21T21:43:30.208 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 120.208625709, 2024-09-21T21:43:30.208 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043631071, 2024-09-21T21:43:30.209 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:30.209 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:30.209 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:30.209 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:30.209 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:30.209 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:30.209 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7008 2024-09-21T21:43:30.209 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.209 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:30.209 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.209 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:30.209 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923278+0000", 2024-09-21T21:43:30.209 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:30.209 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.209 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.210 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:30.210 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923278+0000", 2024-09-21T21:43:30.210 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:30.210 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.210 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.210 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:30.210 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923279+0000", 2024-09-21T21:43:30.210 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.77e-07 2024-09-21T21:43:30.210 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.210 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.210 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:30.210 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923281+0000", 2024-09-21T21:43:30.210 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0219999999999999e-06 2024-09-21T21:43:30.210 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.211 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.211 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:30.211 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923287+0000", 2024-09-21T21:43:30.211 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7799999999999997e-06 2024-09-21T21:43:30.211 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.211 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.211 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:30.211 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923291+0000", 2024-09-21T21:43:30.211 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1699999999999999e-06 2024-09-21T21:43:30.211 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.211 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.211 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:30.211 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966742+0000", 2024-09-21T21:43:30.211 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043451344000000003 2024-09-21T21:43:30.211 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.212 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.212 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:30.212 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966809+0000", 2024-09-21T21:43:30.212 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7070999999999995e-05 2024-09-21T21:43:30.212 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.212 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.212 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:30.212 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966909+0000", 2024-09-21T21:43:30.212 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000100307 2024-09-21T21:43:30.212 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.212 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:30.212 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.212 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.212 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.212 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4952.0:7010 3.1a 3:5f5109db:::rbd_data.1121a3eabc61.0000000000000001:17 [read 5498880~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-21T21:43:30.213 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:29.923301+0000", 2024-09-21T21:43:30.213 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 120.208602814, 2024-09-21T21:43:30.213 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043717830999999999, 2024-09-21T21:43:30.213 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:30.213 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:30.213 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:30.213 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:30.213 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4952", 2024-09-21T21:43:30.213 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/3202420572", 2024-09-21T21:43:30.213 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 7010 2024-09-21T21:43:30.213 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.213 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:30.213 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.213 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:30.213 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923301+0000", 2024-09-21T21:43:30.214 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:30.214 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.214 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.214 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:30.214 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923301+0000", 2024-09-21T21:43:30.214 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:30.214 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.214 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.214 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:30.214 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923302+0000", 2024-09-21T21:43:30.214 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.41e-07 2024-09-21T21:43:30.214 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.214 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.214 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:30.214 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923304+0000", 2024-09-21T21:43:30.215 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0990000000000001e-06 2024-09-21T21:43:30.215 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.215 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.215 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:30.215 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923310+0000", 2024-09-21T21:43:30.215 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3860000000000004e-06 2024-09-21T21:43:30.215 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.215 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.215 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:30.215 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.923315+0000", 2024-09-21T21:43:30.215 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8459999999999999e-06 2024-09-21T21:43:30.215 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.215 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.215 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:30.216 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966938+0000", 2024-09-21T21:43:30.216 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.043622693999999997 2024-09-21T21:43:30.216 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.216 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.216 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:30.216 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.966975+0000", 2024-09-21T21:43:30.216 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7608999999999999e-05 2024-09-21T21:43:30.216 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.216 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.216 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:30.216 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:29.967019+0000", 2024-09-21T21:43:30.216 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3856000000000002e-05 2024-09-21T21:43:30.216 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.216 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:30.216 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.216 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.217 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.217 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5215.0:1054 3.9 3:946d2401:::rbd_data.1121524d2522.000000000000000b:head [set-alloc-hint object_size 16777216 write_size 16777216,write 6615040~59392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-21T21:43:30.217 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:38.148979+0000", 2024-09-21T21:43:30.217 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 111.98292499199999, 2024-09-21T21:43:30.217 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.047870388999999999, 2024-09-21T21:43:30.217 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:30.217 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:30.217 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:30.217 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:30.217 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5215", 2024-09-21T21:43:30.217 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1580282168", 2024-09-21T21:43:30.217 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1054 2024-09-21T21:43:30.217 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.217 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:30.217 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.218 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:30.218 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.148979+0000", 2024-09-21T21:43:30.218 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:30.218 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.218 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.218 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:30.218 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.148979+0000", 2024-09-21T21:43:30.218 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:30.218 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.218 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.218 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:30.218 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.148980+0000", 2024-09-21T21:43:30.218 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5799999999999999e-07 2024-09-21T21:43:30.218 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.218 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.219 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:30.219 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.149039+0000", 2024-09-21T21:43:30.219 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9151000000000003e-05 2024-09-21T21:43:30.219 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.219 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.219 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:30.219 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.149047+0000", 2024-09-21T21:43:30.219 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1000000000000004e-06 2024-09-21T21:43:30.219 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.219 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.219 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:30.219 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.149052+0000", 2024-09-21T21:43:30.219 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5959999999999997e-06 2024-09-21T21:43:30.219 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.220 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.220 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:30.220 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.195288+0000", 2024-09-21T21:43:30.220 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.046236248000000001 2024-09-21T21:43:30.220 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.220 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.220 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:30.220 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.195334+0000", 2024-09-21T21:43:30.220 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6575e-05 2024-09-21T21:43:30.220 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.220 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.220 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 2", 2024-09-21T21:43:30.220 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.195406+0000", 2024-09-21T21:43:30.221 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1715e-05 2024-09-21T21:43:30.221 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.221 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.221 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:30.221 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.196018+0000", 2024-09-21T21:43:30.221 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00061218999999999998 2024-09-21T21:43:30.221 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.221 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.221 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:30.221 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.196794+0000", 2024-09-21T21:43:30.221 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000775814 2024-09-21T21:43:30.221 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.221 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.221 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:30.221 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.196827+0000", 2024-09-21T21:43:30.222 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2419000000000003e-05 2024-09-21T21:43:30.222 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.222 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.222 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:30.222 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:38.196849+0000", 2024-09-21T21:43:30.222 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2923000000000001e-05 2024-09-21T21:43:30.222 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.222 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:30.222 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.222 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.222 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.222 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4385.0:31260 3.3 3:c4c92e5a:::rbd_trash:head [call rbd.trash_add in=12b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e155)", 2024-09-21T21:43:30.222 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:43:17.285720+0000", 2024-09-21T21:43:30.222 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 12.846183761000001, 2024-09-21T21:43:30.222 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.945360706, 2024-09-21T21:43:30.223 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:30.223 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:30.223 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:30.223 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:30.223 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4385", 2024-09-21T21:43:30.223 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:43:30.223 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 31260 2024-09-21T21:43:30.223 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.223 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:30.223 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.223 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:30.223 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:43:17.285720+0000", 2024-09-21T21:43:30.223 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:30.223 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.223 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.224 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:30.224 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:43:17.285720+0000", 2024-09-21T21:43:30.224 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:30.224 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.224 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.224 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:30.224 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:43:17.285721+0000", 2024-09-21T21:43:30.224 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1540000000000001e-06 2024-09-21T21:43:30.224 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.224 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.224 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:30.224 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:43:17.285728+0000", 2024-09-21T21:43:30.224 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5899999999999996e-06 2024-09-21T21:43:30.224 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.224 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.225 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:30.225 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:43:17.285740+0000", 2024-09-21T21:43:30.225 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1539000000000001e-05 2024-09-21T21:43:30.225 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.225 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.225 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:30.225 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:43:17.285746+0000", 2024-09-21T21:43:30.225 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.849e-06 2024-09-21T21:43:30.225 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.225 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.225 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:30.225 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:43:17.285796+0000", 2024-09-21T21:43:30.225 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9678000000000001e-05 2024-09-21T21:43:30.225 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.225 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.226 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:30.226 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:43:17.285805+0000", 2024-09-21T21:43:30.226 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5320000000000006e-06 2024-09-21T21:43:30.226 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.226 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.226 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:30.226 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:43:18.228162+0000", 2024-09-21T21:43:30.226 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.94235735499999995 2024-09-21T21:43:30.226 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.226 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.226 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:30.226 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:43:18.228263+0000", 2024-09-21T21:43:30.226 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010129 2024-09-21T21:43:30.226 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.226 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.226 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-21T21:43:30.227 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:43:18.228409+0000", 2024-09-21T21:43:30.227 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000145778 2024-09-21T21:43:30.227 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.227 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.227 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:30.227 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:43:18.229542+0000", 2024-09-21T21:43:30.227 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001133046 2024-09-21T21:43:30.227 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.227 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.227 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:30.227 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:43:18.230991+0000", 2024-09-21T21:43:30.227 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001448866 2024-09-21T21:43:30.227 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.227 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.227 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:30.228 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:43:18.231040+0000", 2024-09-21T21:43:30.228 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9515999999999998e-05 2024-09-21T21:43:30.228 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.228 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.228 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:30.228 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:43:18.231081+0000", 2024-09-21T21:43:30.228 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0513000000000003e-05 2024-09-21T21:43:30.228 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.228 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:30.228 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.228 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.228 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:30.228 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:43:30.230 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T21:43:30.230 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:43:30.229+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:43:30.280 INFO:teuthology.orchestra.run.smithi081.stderr:noscrub is unset 2024-09-21T21:43:30.330 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:43:30.329+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:43:30.383 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:43:30.384 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:43:30.384 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.384 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(client.4385.0:32966 3.17 e168/144 3:ef5716b5:::rbd_header.1121dee34a14:head v 168'492, mlcod=168'490)", 2024-09-21T21:43:30.384 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:43:30.382624+0000", 2024-09-21T21:43:30.384 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.000112778, 2024-09-21T21:43:30.384 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000128789, 2024-09-21T21:43:30.384 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:30.384 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:30.384 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:30.384 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:30.384 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.384 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:30.384 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:43:30.382624+0000", 2024-09-21T21:43:30.384 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:30.385 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.385 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.385 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:30.385 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:43:30.382624+0000", 2024-09-21T21:43:30.385 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:30.385 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.385 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.385 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:30.385 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:43:30.382625+0000", 2024-09-21T21:43:30.385 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7199999999999998e-07 2024-09-21T21:43:30.385 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.385 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.385 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:30.385 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:43:30.382631+0000", 2024-09-21T21:43:30.385 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5029999999999997e-06 2024-09-21T21:43:30.386 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.386 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.386 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:30.386 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:43:30.382640+0000", 2024-09-21T21:43:30.386 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.1370000000000008e-06 2024-09-21T21:43:30.386 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.386 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.386 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:30.386 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:43:30.382645+0000", 2024-09-21T21:43:30.386 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8300000000000003e-06 2024-09-21T21:43:30.386 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.386 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.386 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:30.386 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:43:30.382688+0000", 2024-09-21T21:43:30.386 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3272000000000002e-05 2024-09-21T21:43:30.387 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.387 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.387 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:30.387 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:43:30.382701+0000", 2024-09-21T21:43:30.387 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3033e-05 2024-09-21T21:43:30.387 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.387 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:30.387 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.387 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.387 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-21T21:43:30.387 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 1 2024-09-21T21:43:30.388 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:43:30.393 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T21:43:30.431 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:43:30.430+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:43:30.532 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:43:30.531+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:43:30.536 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:43:30.536 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-21T21:43:30.536 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 30, 2024-09-21T21:43:30.536 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-21T21:43:30.536 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:43:30.546 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T21:43:30.633 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:43:30.632+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:43:30.691 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-21T21:43:30.691 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-21T21:43:30.691 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-21T21:43:30.691 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-21T21:43:30.691 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.691 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4385.0:15202 3.16 3:6888948b:::rbd_header.1121493f6459:head [watch reconnect cookie 140325440202496 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-21T21:43:30.691 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:41.736601+0000", 2024-09-21T21:43:30.692 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 108.951754033, 2024-09-21T21:43:30.692 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5040056769999999, 2024-09-21T21:43:30.692 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:30.692 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:30.692 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:30.692 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:30.692 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4385", 2024-09-21T21:43:30.692 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:43:30.692 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 15202 2024-09-21T21:43:30.692 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.692 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:30.692 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.692 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:30.692 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736601+0000", 2024-09-21T21:43:30.693 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:30.693 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.693 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.693 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:30.693 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736601+0000", 2024-09-21T21:43:30.693 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:30.693 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.693 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.693 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:30.693 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736602+0000", 2024-09-21T21:43:30.693 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0699999999999999e-06 2024-09-21T21:43:30.693 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.693 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.693 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:30.694 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736607+0000", 2024-09-21T21:43:30.694 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2520000000000001e-06 2024-09-21T21:43:30.694 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.694 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.694 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:30.694 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736617+0000", 2024-09-21T21:43:30.694 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0074e-05 2024-09-21T21:43:30.694 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.694 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.694 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:30.694 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736622+0000", 2024-09-21T21:43:30.695 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.101e-06 2024-09-21T21:43:30.695 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.695 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.695 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:30.695 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736664+0000", 2024-09-21T21:43:30.695 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1869999999999997e-05 2024-09-21T21:43:30.695 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.695 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.695 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op must wait for map", 2024-09-21T21:43:30.695 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.736667+0000", 2024-09-21T21:43:30.695 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4750000000000002e-06 2024-09-21T21:43:30.695 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.695 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.696 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:30.696 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742886+0000", 2024-09-21T21:43:30.696 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0062187579999999996 2024-09-21T21:43:30.696 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.696 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.696 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:30.696 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742892+0000", 2024-09-21T21:43:30.696 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0440000000000001e-06 2024-09-21T21:43:30.696 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.696 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.696 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:30.696 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210307+0000", 2024-09-21T21:43:30.696 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.467414781 2024-09-21T21:43:30.696 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.696 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.697 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:30.697 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210322+0000", 2024-09-21T21:43:30.697 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5311e-05 2024-09-21T21:43:30.697 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.697 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.697 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:30.697 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.237624+0000", 2024-09-21T21:43:30.697 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0273018920000001 2024-09-21T21:43:30.697 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.697 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.697 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:30.697 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.237870+0000", 2024-09-21T21:43:30.697 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00024622899999999999 2024-09-21T21:43:30.698 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.698 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.698 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:30.698 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.240607+0000", 2024-09-21T21:43:30.698 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0027368200000000001 2024-09-21T21:43:30.698 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.698 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:30.698 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.698 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.698 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.698 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5415.0:379 3.16 3:6888948b:::rbd_header.1121493f6459:head [watch reconnect cookie 139702602617920 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-21T21:43:30.698 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:41.739544+0000", 2024-09-21T21:43:30.698 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 108.948811604, 2024-09-21T21:43:30.698 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.501180645, 2024-09-21T21:43:30.699 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:30.699 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:30.699 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:30.699 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:30.699 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5415", 2024-09-21T21:43:30.699 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1212816607", 2024-09-21T21:43:30.699 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 379 2024-09-21T21:43:30.699 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.699 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:30.699 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.699 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:30.699 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739544+0000", 2024-09-21T21:43:30.699 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:30.700 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.700 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.700 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:30.700 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739544+0000", 2024-09-21T21:43:30.700 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:30.700 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.700 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.700 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:30.700 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739545+0000", 2024-09-21T21:43:30.700 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.065e-06 2024-09-21T21:43:30.700 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.700 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.700 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:30.700 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739550+0000", 2024-09-21T21:43:30.701 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9599999999999999e-06 2024-09-21T21:43:30.701 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.701 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.701 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:30.701 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739565+0000", 2024-09-21T21:43:30.701 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5709000000000002e-05 2024-09-21T21:43:30.701 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.701 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.701 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:30.701 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739573+0000", 2024-09-21T21:43:30.701 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7849999999999992e-06 2024-09-21T21:43:30.701 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.701 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.702 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:30.702 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739634+0000", 2024-09-21T21:43:30.702 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0732000000000001e-05 2024-09-21T21:43:30.702 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.702 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.702 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op must wait for map", 2024-09-21T21:43:30.702 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739640+0000", 2024-09-21T21:43:30.702 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0000000000000002e-06 2024-09-21T21:43:30.702 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.702 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.702 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:30.702 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742958+0000", 2024-09-21T21:43:30.702 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0033182390000000002 2024-09-21T21:43:30.702 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.703 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.703 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:30.703 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742966+0000", 2024-09-21T21:43:30.703 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1610000000000003e-06 2024-09-21T21:43:30.703 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.703 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.703 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:30.703 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210405+0000", 2024-09-21T21:43:30.703 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.467438725 2024-09-21T21:43:30.703 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.703 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.703 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:30.703 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210415+0000", 2024-09-21T21:43:30.703 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0234000000000001e-05 2024-09-21T21:43:30.704 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.704 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.704 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:30.704 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.238743+0000", 2024-09-21T21:43:30.704 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0283273070000001 2024-09-21T21:43:30.704 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.704 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.704 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:30.704 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.238842+0000", 2024-09-21T21:43:30.704 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.9896999999999996e-05 2024-09-21T21:43:30.704 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.704 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.704 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:30.704 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.240724+0000", 2024-09-21T21:43:30.704 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001881831 2024-09-21T21:43:30.705 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.705 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:30.705 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.705 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.705 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.705 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4385.0:15203 3.16 3:6888948b:::rbd_header.1121493f6459:head [call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-21T21:43:30.705 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:41.739737+0000", 2024-09-21T21:43:30.705 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 108.94861859300001, 2024-09-21T21:43:30.705 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4989621799999999, 2024-09-21T21:43:30.705 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:30.705 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:30.705 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:30.706 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:30.706 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4385", 2024-09-21T21:43:30.706 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:43:30.706 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 15203 2024-09-21T21:43:30.706 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.706 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:30.706 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.706 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:30.706 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739737+0000", 2024-09-21T21:43:30.706 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:30.706 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.706 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.706 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:30.706 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739737+0000", 2024-09-21T21:43:30.707 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:30.707 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.707 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.707 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:30.707 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739738+0000", 2024-09-21T21:43:30.707 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.7499999999999998e-07 2024-09-21T21:43:30.707 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.707 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.707 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:30.707 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739743+0000", 2024-09-21T21:43:30.707 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5049999999999998e-06 2024-09-21T21:43:30.707 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.707 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.707 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:30.708 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739757+0000", 2024-09-21T21:43:30.708 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3947e-05 2024-09-21T21:43:30.708 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.708 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.708 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:30.708 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739765+0000", 2024-09-21T21:43:30.708 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9310000000000008e-06 2024-09-21T21:43:30.708 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.708 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.708 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:30.708 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739816+0000", 2024-09-21T21:43:30.708 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0559999999999997e-05 2024-09-21T21:43:30.708 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.708 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.708 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting_for_map not empty", 2024-09-21T21:43:30.709 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.739821+0000", 2024-09-21T21:43:30.709 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3539999999999999e-06 2024-09-21T21:43:30.709 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.709 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.709 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:30.709 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742928+0000", 2024-09-21T21:43:30.709 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0031074399999999999 2024-09-21T21:43:30.709 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.709 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.709 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:30.709 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:41.742937+0000", 2024-09-21T21:43:30.709 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1289999999999996e-06 2024-09-21T21:43:30.709 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.709 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.710 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:30.710 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210350+0000", 2024-09-21T21:43:30.710 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.46741311299999999 2024-09-21T21:43:30.710 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.710 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.710 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:30.710 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210358+0000", 2024-09-21T21:43:30.710 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1869999999999997e-06 2024-09-21T21:43:30.710 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.710 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.710 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:30.710 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.238398+0000", 2024-09-21T21:43:30.710 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0280397830000001 2024-09-21T21:43:30.710 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.711 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.711 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:30.711 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.238489+0000", 2024-09-21T21:43:30.711 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.0958000000000006e-05 2024-09-21T21:43:30.711 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.711 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.711 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:30.711 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.238699+0000", 2024-09-21T21:43:30.711 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00021029800000000001 2024-09-21T21:43:30.711 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.711 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:30.711 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.711 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.711 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.711 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4385.0:15204 3.16 3:6888948b:::rbd_header.1121493f6459:head [watch reconnect cookie 140325440202496 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-21T21:43:30.712 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:42.210726+0000", 2024-09-21T21:43:30.712 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 108.47762971500001, 2024-09-21T21:43:30.712 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.030304557, 2024-09-21T21:43:30.712 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:30.712 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:30.712 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:30.712 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:30.712 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4385", 2024-09-21T21:43:30.712 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/878134532", 2024-09-21T21:43:30.712 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 15204 2024-09-21T21:43:30.712 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.712 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:30.712 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.712 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:30.713 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210726+0000", 2024-09-21T21:43:30.713 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:30.713 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.713 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.713 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:30.713 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210726+0000", 2024-09-21T21:43:30.713 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:30.713 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.713 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.713 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:30.713 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210727+0000", 2024-09-21T21:43:30.713 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.204e-06 2024-09-21T21:43:30.713 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.713 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.713 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:30.714 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210732+0000", 2024-09-21T21:43:30.714 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5249999999999996e-06 2024-09-21T21:43:30.714 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.714 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.714 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:30.714 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210744+0000", 2024-09-21T21:43:30.714 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1405e-05 2024-09-21T21:43:30.714 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.714 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.714 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:30.714 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210754+0000", 2024-09-21T21:43:30.715 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0122999999999999e-05 2024-09-21T21:43:30.715 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.715 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.715 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:30.715 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210798+0000", 2024-09-21T21:43:30.715 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4005000000000003e-05 2024-09-21T21:43:30.715 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.715 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.715 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:30.715 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210803+0000", 2024-09-21T21:43:30.715 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5130000000000004e-06 2024-09-21T21:43:30.715 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.715 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.715 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:30.715 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.239260+0000", 2024-09-21T21:43:30.716 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0284563040000001 2024-09-21T21:43:30.716 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.716 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.716 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:30.716 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.239318+0000", 2024-09-21T21:43:30.716 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8486000000000002e-05 2024-09-21T21:43:30.716 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.716 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.716 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:30.716 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.241030+0000", 2024-09-21T21:43:30.716 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0017119920000000001 2024-09-21T21:43:30.716 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.716 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:30.716 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.716 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.717 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.717 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5415.0:380 3.16 3:6888948b:::rbd_header.1121493f6459:head [watch reconnect cookie 139702602617920 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71)", 2024-09-21T21:43:30.717 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:41:42.210777+0000", 2024-09-21T21:43:30.717 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 108.477577936, 2024-09-21T21:43:30.717 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.030410566, 2024-09-21T21:43:30.717 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:30.717 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:30.717 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-21T21:43:30.717 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:30.717 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5415", 2024-09-21T21:43:30.717 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/1212816607", 2024-09-21T21:43:30.717 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 380 2024-09-21T21:43:30.717 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.717 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:30.717 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.718 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:30.718 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210777+0000", 2024-09-21T21:43:30.718 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:30.718 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.718 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.718 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:30.718 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210777+0000", 2024-09-21T21:43:30.718 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:30.718 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.718 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.718 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:30.718 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210778+0000", 2024-09-21T21:43:30.718 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8999999999999996e-07 2024-09-21T21:43:30.718 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.718 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.719 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:30.719 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210780+0000", 2024-09-21T21:43:30.719 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.503e-06 2024-09-21T21:43:30.719 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.719 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.719 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:30.719 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210788+0000", 2024-09-21T21:43:30.719 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0999999999999998e-06 2024-09-21T21:43:30.719 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.719 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.719 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:30.720 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210799+0000", 2024-09-21T21:43:30.720 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1703e-05 2024-09-21T21:43:30.720 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.720 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.720 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:30.720 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210833+0000", 2024-09-21T21:43:30.720 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4008999999999999e-05 2024-09-21T21:43:30.720 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.720 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.720 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-21T21:43:30.720 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:42.210838+0000", 2024-09-21T21:43:30.720 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7480000000000001e-06 2024-09-21T21:43:30.720 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.720 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.720 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:30.721 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.239612+0000", 2024-09-21T21:43:30.721 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.028773554 2024-09-21T21:43:30.721 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.721 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.721 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:30.721 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.239677+0000", 2024-09-21T21:43:30.721 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5053000000000002e-05 2024-09-21T21:43:30.721 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.721 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.721 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:30.721 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:41:43.241188+0000", 2024-09-21T21:43:30.721 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001511306 2024-09-21T21:43:30.721 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.721 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:30.722 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.722 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.722 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.722 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24794 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:30.722 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.584085+0000", 2024-09-21T21:43:30.722 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 68.104270760000006, 2024-09-21T21:43:30.722 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.096209996000000006, 2024-09-21T21:43:30.722 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:30.722 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:30.722 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:30.722 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:30.722 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:30.722 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:30.722 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24794 2024-09-21T21:43:30.722 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.723 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:30.723 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.723 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:30.723 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584085+0000", 2024-09-21T21:43:30.723 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:30.723 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.723 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.723 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:30.723 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584085+0000", 2024-09-21T21:43:30.723 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:30.723 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.723 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.723 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:30.723 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584085+0000", 2024-09-21T21:43:30.724 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3200000000000001e-07 2024-09-21T21:43:30.724 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.724 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.724 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:30.724 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584086+0000", 2024-09-21T21:43:30.724 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.717e-06 2024-09-21T21:43:30.724 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.724 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.724 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:30.724 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584090+0000", 2024-09-21T21:43:30.724 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3450000000000002e-06 2024-09-21T21:43:30.724 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.724 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.724 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:30.724 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584092+0000", 2024-09-21T21:43:30.725 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6359999999999998e-06 2024-09-21T21:43:30.725 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.725 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.725 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:30.725 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.676497+0000", 2024-09-21T21:43:30.725 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.092404659 2024-09-21T21:43:30.725 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.725 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.725 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:30.725 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.676567+0000", 2024-09-21T21:43:30.725 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9627000000000003e-05 2024-09-21T21:43:30.725 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.725 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.725 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:30.725 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.676925+0000", 2024-09-21T21:43:30.726 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00035848500000000002 2024-09-21T21:43:30.726 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.726 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.726 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:30.726 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.678427+0000", 2024-09-21T21:43:30.726 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001501638 2024-09-21T21:43:30.726 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.726 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.726 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:30.726 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.680204+0000", 2024-09-21T21:43:30.726 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0017766979999999999 2024-09-21T21:43:30.726 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.727 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.727 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:30.727 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.680256+0000", 2024-09-21T21:43:30.727 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2549999999999997e-05 2024-09-21T21:43:30.727 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.727 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.727 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:30.727 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.680295+0000", 2024-09-21T21:43:30.727 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8408999999999998e-05 2024-09-21T21:43:30.728 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.728 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:30.728 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.728 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.728 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.728 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24796 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:30.728 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.584099+0000", 2024-09-21T21:43:30.728 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 68.104256191000005, 2024-09-21T21:43:30.728 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.097297429000000005, 2024-09-21T21:43:30.728 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:30.728 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:30.728 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:30.728 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:30.728 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:30.729 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:30.729 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24796 2024-09-21T21:43:30.729 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.729 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:30.729 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.729 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:30.729 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584099+0000", 2024-09-21T21:43:30.729 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:30.729 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.729 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.729 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:30.729 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584099+0000", 2024-09-21T21:43:30.729 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:30.729 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.729 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.730 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:30.730 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584099+0000", 2024-09-21T21:43:30.730 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3999999999999998e-07 2024-09-21T21:43:30.730 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.730 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.730 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:30.730 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584101+0000", 2024-09-21T21:43:30.730 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5850000000000001e-06 2024-09-21T21:43:30.730 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.730 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.730 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:30.730 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584108+0000", 2024-09-21T21:43:30.730 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.498e-06 2024-09-21T21:43:30.730 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.731 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.731 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:30.731 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584112+0000", 2024-09-21T21:43:30.731 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.252e-06 2024-09-21T21:43:30.731 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.731 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.731 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:30.731 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.677336+0000", 2024-09-21T21:43:30.731 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.093224206000000004 2024-09-21T21:43:30.731 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.731 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.731 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:30.731 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.677411+0000", 2024-09-21T21:43:30.731 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5036000000000005e-05 2024-09-21T21:43:30.732 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.732 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.732 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:30.732 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.677782+0000", 2024-09-21T21:43:30.732 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00037153099999999998 2024-09-21T21:43:30.732 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.732 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.732 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:30.732 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.680360+0000", 2024-09-21T21:43:30.732 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025773020000000001 2024-09-21T21:43:30.732 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.732 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.732 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:30.732 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681313+0000", 2024-09-21T21:43:30.733 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00095341100000000004 2024-09-21T21:43:30.733 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.733 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.733 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:30.733 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681361+0000", 2024-09-21T21:43:30.733 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7497999999999998e-05 2024-09-21T21:43:30.733 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.733 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.733 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:30.733 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681397+0000", 2024-09-21T21:43:30.733 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.587e-05 2024-09-21T21:43:30.733 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.733 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:30.734 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.734 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.734 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.734 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24797 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:30.734 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.584116+0000", 2024-09-21T21:43:30.734 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 68.104239702000001, 2024-09-21T21:43:30.734 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.097428257000000004, 2024-09-21T21:43:30.734 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:30.734 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:30.734 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:30.735 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:30.735 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:30.735 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:30.735 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24797 2024-09-21T21:43:30.735 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.735 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:30.735 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.735 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:30.735 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584116+0000", 2024-09-21T21:43:30.735 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:30.735 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.735 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.735 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:30.735 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584116+0000", 2024-09-21T21:43:30.735 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:30.736 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.736 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.736 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:30.736 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584116+0000", 2024-09-21T21:43:30.736 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.53e-07 2024-09-21T21:43:30.736 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.736 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.736 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:30.736 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584118+0000", 2024-09-21T21:43:30.736 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6810000000000001e-06 2024-09-21T21:43:30.736 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.736 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.736 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:30.736 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584121+0000", 2024-09-21T21:43:30.736 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1099999999999999e-06 2024-09-21T21:43:30.737 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.737 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.737 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:30.737 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584123+0000", 2024-09-21T21:43:30.737 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5890000000000001e-06 2024-09-21T21:43:30.737 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.737 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.737 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:30.737 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.678476+0000", 2024-09-21T21:43:30.737 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.094352320000000003 2024-09-21T21:43:30.737 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.737 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.737 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:30.737 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.678556+0000", 2024-09-21T21:43:30.738 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0291999999999996e-05 2024-09-21T21:43:30.738 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.738 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.738 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:30.738 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.678871+0000", 2024-09-21T21:43:30.738 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000314689 2024-09-21T21:43:30.738 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.738 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.738 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:30.738 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681452+0000", 2024-09-21T21:43:30.738 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025817890000000001 2024-09-21T21:43:30.738 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.738 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.738 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:30.739 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681467+0000", 2024-09-21T21:43:30.739 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4256e-05 2024-09-21T21:43:30.739 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.739 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.739 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:30.739 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681510+0000", 2024-09-21T21:43:30.739 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3560999999999998e-05 2024-09-21T21:43:30.739 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.739 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.739 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:30.739 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681544+0000", 2024-09-21T21:43:30.739 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3717000000000003e-05 2024-09-21T21:43:30.739 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.739 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:30.740 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.740 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.740 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.740 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24798 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:30.740 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.584130+0000", 2024-09-21T21:43:30.740 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 68.104225178999997, 2024-09-21T21:43:30.740 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.098350526999999993, 2024-09-21T21:43:30.740 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:30.740 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:30.740 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:30.740 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:30.740 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:30.740 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:30.740 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24798 2024-09-21T21:43:30.740 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.741 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:30.741 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.741 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:30.741 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584130+0000", 2024-09-21T21:43:30.741 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:30.741 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.741 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.741 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:30.741 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584130+0000", 2024-09-21T21:43:30.741 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:30.742 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.742 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.742 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:30.742 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584131+0000", 2024-09-21T21:43:30.742 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4700000000000001e-07 2024-09-21T21:43:30.742 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.742 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.742 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:30.742 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584133+0000", 2024-09-21T21:43:30.743 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.34e-06 2024-09-21T21:43:30.743 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.743 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.743 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:30.743 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584137+0000", 2024-09-21T21:43:30.743 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5609999999999999e-06 2024-09-21T21:43:30.743 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.743 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.743 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:30.743 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584139+0000", 2024-09-21T21:43:30.743 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7350000000000001e-06 2024-09-21T21:43:30.743 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.743 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.743 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:30.743 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.679274+0000", 2024-09-21T21:43:30.744 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.095134719000000006 2024-09-21T21:43:30.744 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.744 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.744 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:30.744 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.679358+0000", 2024-09-21T21:43:30.744 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3609999999999994e-05 2024-09-21T21:43:30.744 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.744 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.744 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:30.744 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.679685+0000", 2024-09-21T21:43:30.744 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00032732399999999999 2024-09-21T21:43:30.744 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.744 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.744 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:30.744 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681553+0000", 2024-09-21T21:43:30.745 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001868059 2024-09-21T21:43:30.745 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.745 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.745 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:30.745 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.682375+0000", 2024-09-21T21:43:30.745 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00082232099999999999 2024-09-21T21:43:30.745 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.745 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.745 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:30.745 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.682431+0000", 2024-09-21T21:43:30.745 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6051999999999998e-05 2024-09-21T21:43:30.745 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.745 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.745 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:30.745 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.682481+0000", 2024-09-21T21:43:30.746 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9258999999999997e-05 2024-09-21T21:43:30.746 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.746 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:30.746 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.746 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.746 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.746 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24799 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:30.746 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.584159+0000", 2024-09-21T21:43:30.746 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 68.104196191, 2024-09-21T21:43:30.746 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.100021551, 2024-09-21T21:43:30.746 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:30.746 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:30.746 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:30.746 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:30.746 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:30.747 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:30.747 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24799 2024-09-21T21:43:30.747 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.747 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:30.747 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.747 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:30.747 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584159+0000", 2024-09-21T21:43:30.747 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:30.747 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.747 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.748 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:30.748 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584159+0000", 2024-09-21T21:43:30.748 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:30.749 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.749 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.750 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:30.750 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584160+0000", 2024-09-21T21:43:30.750 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1300000000000001e-07 2024-09-21T21:43:30.750 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.750 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.750 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:30.750 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584162+0000", 2024-09-21T21:43:30.750 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5409999999999999e-06 2024-09-21T21:43:30.750 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.750 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.750 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:30.750 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584169+0000", 2024-09-21T21:43:30.750 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7880000000000001e-06 2024-09-21T21:43:30.750 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.751 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.751 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:30.751 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.584174+0000", 2024-09-21T21:43:30.751 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9590000000000003e-06 2024-09-21T21:43:30.751 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.751 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.751 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:30.751 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.680400+0000", 2024-09-21T21:43:30.751 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.096226514999999999 2024-09-21T21:43:30.751 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.751 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.751 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:30.751 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.680480+0000", 2024-09-21T21:43:30.751 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9589e-05 2024-09-21T21:43:30.751 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.752 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.752 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:30.752 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.680847+0000", 2024-09-21T21:43:30.752 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00036757700000000001 2024-09-21T21:43:30.752 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.752 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.752 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:30.752 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.682555+0000", 2024-09-21T21:43:30.752 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001707791 2024-09-21T21:43:30.752 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.752 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.752 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:30.752 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684093+0000", 2024-09-21T21:43:30.752 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0015377170000000001 2024-09-21T21:43:30.753 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.753 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.753 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:30.753 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684144+0000", 2024-09-21T21:43:30.753 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1208000000000002e-05 2024-09-21T21:43:30.753 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.753 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.753 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:30.753 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684181+0000", 2024-09-21T21:43:30.753 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6452999999999998e-05 2024-09-21T21:43:30.753 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.753 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:30.753 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.753 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.753 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.753 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24835 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:30.754 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586332+0000", 2024-09-21T21:43:30.754 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 68.102023576999997, 2024-09-21T21:43:30.754 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.098730360000000003, 2024-09-21T21:43:30.754 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:30.754 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:30.754 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:30.754 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:30.754 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:30.754 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:30.754 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24835 2024-09-21T21:43:30.754 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.754 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:30.754 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.754 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:30.755 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586332+0000", 2024-09-21T21:43:30.755 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:30.755 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.755 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.755 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:30.755 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586332+0000", 2024-09-21T21:43:30.755 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:30.755 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.755 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.755 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:30.755 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586333+0000", 2024-09-21T21:43:30.755 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0200000000000001e-07 2024-09-21T21:43:30.755 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.755 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.755 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:30.755 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586346+0000", 2024-09-21T21:43:30.756 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3745e-05 2024-09-21T21:43:30.756 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.756 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.756 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:30.756 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586366+0000", 2024-09-21T21:43:30.756 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9272000000000001e-05 2024-09-21T21:43:30.756 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.756 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.756 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:30.756 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586377+0000", 2024-09-21T21:43:30.756 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1197e-05 2024-09-21T21:43:30.756 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.756 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.756 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:30.756 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681578+0000", 2024-09-21T21:43:30.757 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.095201420999999994 2024-09-21T21:43:30.757 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.757 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.757 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:30.757 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681640+0000", 2024-09-21T21:43:30.757 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1834000000000005e-05 2024-09-21T21:43:30.757 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.757 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.757 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:30.757 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.681915+0000", 2024-09-21T21:43:30.757 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00027534199999999998 2024-09-21T21:43:30.757 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.757 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.757 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:30.757 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684230+0000", 2024-09-21T21:43:30.758 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00231424 2024-09-21T21:43:30.758 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.758 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.758 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:30.758 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684959+0000", 2024-09-21T21:43:30.758 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00072921399999999995 2024-09-21T21:43:30.758 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.758 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.758 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:30.758 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.685008+0000", 2024-09-21T21:43:30.758 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8860999999999999e-05 2024-09-21T21:43:30.758 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.758 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.758 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:30.758 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.685062+0000", 2024-09-21T21:43:30.758 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4432000000000003e-05 2024-09-21T21:43:30.759 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.759 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:30.759 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.759 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.759 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.759 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24837 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:30.759 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586395+0000", 2024-09-21T21:43:30.759 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 68.101960641999995, 2024-09-21T21:43:30.759 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.099700522, 2024-09-21T21:43:30.759 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:30.759 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:30.759 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:30.759 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:30.759 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:30.759 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:30.760 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24837 2024-09-21T21:43:30.760 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.760 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:30.760 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.760 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:30.760 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586395+0000", 2024-09-21T21:43:30.760 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:30.760 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.760 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.760 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:30.760 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586395+0000", 2024-09-21T21:43:30.760 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:30.760 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.760 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.760 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:30.761 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586395+0000", 2024-09-21T21:43:30.761 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.06e-07 2024-09-21T21:43:30.761 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.761 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.761 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:30.761 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586401+0000", 2024-09-21T21:43:30.761 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0889999999999996e-06 2024-09-21T21:43:30.761 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.761 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.761 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:30.761 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586410+0000", 2024-09-21T21:43:30.761 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.918e-06 2024-09-21T21:43:30.761 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.761 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.761 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:30.762 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586424+0000", 2024-09-21T21:43:30.762 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3478999999999999e-05 2024-09-21T21:43:30.762 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.762 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.762 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:30.762 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.682590+0000", 2024-09-21T21:43:30.762 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.096166086999999997 2024-09-21T21:43:30.762 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.762 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.762 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:30.762 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.682657+0000", 2024-09-21T21:43:30.762 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6878000000000006e-05 2024-09-21T21:43:30.762 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.762 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.763 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:30.763 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.682879+0000", 2024-09-21T21:43:30.763 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00022238 2024-09-21T21:43:30.763 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.763 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.763 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:30.763 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.685073+0000", 2024-09-21T21:43:30.763 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021938119999999998 2024-09-21T21:43:30.763 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.763 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.763 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:30.763 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686038+0000", 2024-09-21T21:43:30.764 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00096539499999999997 2024-09-21T21:43:30.764 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.764 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.764 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:30.764 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686071+0000", 2024-09-21T21:43:30.764 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2514000000000001e-05 2024-09-21T21:43:30.764 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.764 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.764 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:30.764 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686095+0000", 2024-09-21T21:43:30.764 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4363999999999999e-05 2024-09-21T21:43:30.764 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.764 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:30.764 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.764 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.765 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.765 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24839 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:30.765 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586468+0000", 2024-09-21T21:43:30.765 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 68.101886989999997, 2024-09-21T21:43:30.765 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.099739355000000002, 2024-09-21T21:43:30.765 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:30.765 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:30.765 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:30.765 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:30.765 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:30.765 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:30.765 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24839 2024-09-21T21:43:30.765 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.765 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:30.765 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.766 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:30.766 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586468+0000", 2024-09-21T21:43:30.766 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:30.766 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.766 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.766 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:30.766 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586468+0000", 2024-09-21T21:43:30.766 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:30.766 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.766 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.766 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:30.766 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586469+0000", 2024-09-21T21:43:30.766 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2200000000000004e-07 2024-09-21T21:43:30.766 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.766 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.767 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:30.767 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586472+0000", 2024-09-21T21:43:30.767 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9960000000000002e-06 2024-09-21T21:43:30.767 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.767 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.767 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:30.767 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586479+0000", 2024-09-21T21:43:30.767 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2699999999999999e-06 2024-09-21T21:43:30.767 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.767 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.767 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:30.767 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586485+0000", 2024-09-21T21:43:30.767 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6999999999999996e-06 2024-09-21T21:43:30.767 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.767 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.768 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:30.768 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.683276+0000", 2024-09-21T21:43:30.768 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.096791618999999995 2024-09-21T21:43:30.768 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.768 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.768 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:30.768 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.683326+0000", 2024-09-21T21:43:30.768 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9384e-05 2024-09-21T21:43:30.768 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.768 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.768 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:30.768 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.683655+0000", 2024-09-21T21:43:30.768 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00032965700000000002 2024-09-21T21:43:30.768 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.768 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.769 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:30.769 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686120+0000", 2024-09-21T21:43:30.769 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002464819 2024-09-21T21:43:30.769 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.769 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.769 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:30.769 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686159+0000", 2024-09-21T21:43:30.769 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9181000000000003e-05 2024-09-21T21:43:30.769 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.769 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.769 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:30.769 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686185+0000", 2024-09-21T21:43:30.769 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5907000000000001e-05 2024-09-21T21:43:30.769 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.769 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.770 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:30.770 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686208+0000", 2024-09-21T21:43:30.770 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.23e-05 2024-09-21T21:43:30.770 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.770 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:30.770 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.770 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.770 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.770 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24841 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:30.770 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586492+0000", 2024-09-21T21:43:30.770 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 68.101862827999994, 2024-09-21T21:43:30.770 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.101672077, 2024-09-21T21:43:30.770 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:30.770 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:30.770 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:30.771 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:30.771 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:30.771 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:30.771 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24841 2024-09-21T21:43:30.771 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.771 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:30.771 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.771 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:30.771 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586492+0000", 2024-09-21T21:43:30.771 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:30.771 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.771 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.771 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:30.771 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586492+0000", 2024-09-21T21:43:30.772 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:30.772 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.772 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.772 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:30.772 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586493+0000", 2024-09-21T21:43:30.772 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5999999999999999e-07 2024-09-21T21:43:30.772 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.772 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.772 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:30.772 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586496+0000", 2024-09-21T21:43:30.772 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6089999999999999e-06 2024-09-21T21:43:30.772 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.772 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.772 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:30.772 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586502+0000", 2024-09-21T21:43:30.772 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6270000000000002e-06 2024-09-21T21:43:30.773 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.773 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.773 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:30.773 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586507+0000", 2024-09-21T21:43:30.773 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8810000000000004e-06 2024-09-21T21:43:30.773 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.773 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.773 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:30.773 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684273+0000", 2024-09-21T21:43:30.773 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.097766415999999995 2024-09-21T21:43:30.773 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.773 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.773 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:30.773 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684359+0000", 2024-09-21T21:43:30.773 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5574000000000004e-05 2024-09-21T21:43:30.774 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.774 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.774 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:30.774 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.684568+0000", 2024-09-21T21:43:30.774 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000208529 2024-09-21T21:43:30.774 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.774 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.774 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:30.774 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.687186+0000", 2024-09-21T21:43:30.774 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0026180769999999999 2024-09-21T21:43:30.774 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.774 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.774 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:30.774 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688093+0000", 2024-09-21T21:43:30.774 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00090766999999999996 2024-09-21T21:43:30.775 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.775 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.775 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:30.775 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688142+0000", 2024-09-21T21:43:30.775 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8986000000000002e-05 2024-09-21T21:43:30.775 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.775 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.775 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:30.775 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688165+0000", 2024-09-21T21:43:30.775 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2248000000000001e-05 2024-09-21T21:43:30.775 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.775 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:30.775 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.775 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.775 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.776 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24843 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:30.776 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586514+0000", 2024-09-21T21:43:30.776 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 68.101841542000003, 2024-09-21T21:43:30.776 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.101826449, 2024-09-21T21:43:30.776 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:30.776 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:30.776 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:30.776 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:30.776 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:30.776 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:30.776 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24843 2024-09-21T21:43:30.776 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.776 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:30.777 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.777 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:30.777 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586514+0000", 2024-09-21T21:43:30.777 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:30.777 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.777 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.777 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:30.777 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586514+0000", 2024-09-21T21:43:30.777 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:30.777 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.777 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.777 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:30.777 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586514+0000", 2024-09-21T21:43:30.777 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9999999999999998e-07 2024-09-21T21:43:30.777 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.778 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.778 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:30.778 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586517+0000", 2024-09-21T21:43:30.778 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4980000000000001e-06 2024-09-21T21:43:30.778 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.778 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.778 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:30.778 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586523+0000", 2024-09-21T21:43:30.778 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4609999999999996e-06 2024-09-21T21:43:30.778 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.778 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.778 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:30.778 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586528+0000", 2024-09-21T21:43:30.778 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2979999999999999e-06 2024-09-21T21:43:30.778 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.779 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.779 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:30.779 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.685104+0000", 2024-09-21T21:43:30.779 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.098575837999999999 2024-09-21T21:43:30.779 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.779 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.779 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:30.779 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.685184+0000", 2024-09-21T21:43:30.779 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9807000000000003e-05 2024-09-21T21:43:30.779 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.779 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.779 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:30.779 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.685496+0000", 2024-09-21T21:43:30.779 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00031225900000000001 2024-09-21T21:43:30.779 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.780 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.780 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:30.780 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688238+0000", 2024-09-21T21:43:30.780 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0027419409999999999 2024-09-21T21:43:30.780 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.780 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.780 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:30.780 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688253+0000", 2024-09-21T21:43:30.780 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4331000000000001e-05 2024-09-21T21:43:30.780 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.780 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.780 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:30.780 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688296+0000", 2024-09-21T21:43:30.780 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3801000000000003e-05 2024-09-21T21:43:30.780 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.781 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.781 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:30.781 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688340+0000", 2024-09-21T21:43:30.781 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3815e-05 2024-09-21T21:43:30.781 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.781 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:30.781 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.781 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.781 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.781 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24845 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:30.781 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586535+0000", 2024-09-21T21:43:30.781 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 68.101820661999994, 2024-09-21T21:43:30.781 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.10285678600000001, 2024-09-21T21:43:30.781 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:30.781 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:30.782 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:30.782 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:30.782 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:30.782 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:30.782 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24845 2024-09-21T21:43:30.782 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.782 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:30.782 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.782 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:30.782 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586535+0000", 2024-09-21T21:43:30.782 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:30.782 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.782 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.782 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:30.782 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586535+0000", 2024-09-21T21:43:30.783 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:30.783 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.783 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.783 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:30.783 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586535+0000", 2024-09-21T21:43:30.783 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9000000000000002e-07 2024-09-21T21:43:30.783 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.783 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.783 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:30.783 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586537+0000", 2024-09-21T21:43:30.783 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4569999999999999e-06 2024-09-21T21:43:30.783 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.783 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.783 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:30.783 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586544+0000", 2024-09-21T21:43:30.784 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5849999999999996e-06 2024-09-21T21:43:30.784 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.784 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.784 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:30.784 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586549+0000", 2024-09-21T21:43:30.784 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0089999999999998e-06 2024-09-21T21:43:30.784 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.784 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.784 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:30.784 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686239+0000", 2024-09-21T21:43:30.784 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.099689531999999997 2024-09-21T21:43:30.784 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.784 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.784 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:30.784 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686289+0000", 2024-09-21T21:43:30.785 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.007e-05 2024-09-21T21:43:30.785 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.785 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.785 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:30.785 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.686633+0000", 2024-09-21T21:43:30.785 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00034428699999999998 2024-09-21T21:43:30.785 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.785 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.785 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:30.785 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688370+0000", 2024-09-21T21:43:30.785 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001736741 2024-09-21T21:43:30.785 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.785 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.785 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:30.785 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.689295+0000", 2024-09-21T21:43:30.786 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00092517900000000004 2024-09-21T21:43:30.786 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.786 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.786 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:30.786 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.689354+0000", 2024-09-21T21:43:30.786 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8799999999999999e-05 2024-09-21T21:43:30.786 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.786 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.786 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:30.786 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.689391+0000", 2024-09-21T21:43:30.786 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7735999999999999e-05 2024-09-21T21:43:30.786 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.786 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:30.786 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.786 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.787 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.787 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24846 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:30.787 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586564+0000", 2024-09-21T21:43:30.787 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 68.101791633999994, 2024-09-21T21:43:30.787 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.10375692, 2024-09-21T21:43:30.787 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:30.787 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:30.787 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:30.787 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:30.787 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:30.787 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:30.787 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24846 2024-09-21T21:43:30.787 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.787 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:30.787 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.788 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:30.788 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586564+0000", 2024-09-21T21:43:30.788 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:30.788 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.788 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.788 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:30.788 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586564+0000", 2024-09-21T21:43:30.788 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:30.788 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.788 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.788 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:30.788 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586564+0000", 2024-09-21T21:43:30.788 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4799999999999998e-07 2024-09-21T21:43:30.788 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.788 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.789 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:30.789 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586567+0000", 2024-09-21T21:43:30.789 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1669999999999999e-06 2024-09-21T21:43:30.789 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.789 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.789 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:30.789 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586578+0000", 2024-09-21T21:43:30.789 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0443e-05 2024-09-21T21:43:30.789 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.789 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.789 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:30.789 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586583+0000", 2024-09-21T21:43:30.789 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6790000000000002e-06 2024-09-21T21:43:30.789 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.789 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.790 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:30.790 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.687215+0000", 2024-09-21T21:43:30.790 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.100631973 2024-09-21T21:43:30.790 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.790 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.790 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:30.790 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.687311+0000", 2024-09-21T21:43:30.790 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5764e-05 2024-09-21T21:43:30.790 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.790 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.790 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:30.790 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.687663+0000", 2024-09-21T21:43:30.790 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00035167 2024-09-21T21:43:30.790 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.790 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.791 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:30.791 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690236+0000", 2024-09-21T21:43:30.791 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025733370000000002 2024-09-21T21:43:30.791 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.791 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.791 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:30.791 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690250+0000", 2024-09-21T21:43:30.791 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3548e-05 2024-09-21T21:43:30.791 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.791 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.791 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:30.791 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690298+0000", 2024-09-21T21:43:30.791 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7948999999999998e-05 2024-09-21T21:43:30.791 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.792 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.792 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:30.792 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690321+0000", 2024-09-21T21:43:30.792 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2841999999999999e-05 2024-09-21T21:43:30.792 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.792 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:30.792 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.792 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.792 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.792 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24849 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:30.792 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586633+0000", 2024-09-21T21:43:30.792 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 68.101721968000007, 2024-09-21T21:43:30.792 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.104506548, 2024-09-21T21:43:30.792 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:30.793 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:30.793 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:30.793 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:30.793 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:30.793 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:30.793 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24849 2024-09-21T21:43:30.793 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.793 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:30.793 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.793 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:30.793 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586633+0000", 2024-09-21T21:43:30.793 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:30.793 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.793 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.794 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:30.794 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586633+0000", 2024-09-21T21:43:30.794 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:30.794 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.794 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.794 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:30.794 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586634+0000", 2024-09-21T21:43:30.794 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9100000000000004e-07 2024-09-21T21:43:30.794 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.794 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.794 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:30.794 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586637+0000", 2024-09-21T21:43:30.795 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1999999999999999e-06 2024-09-21T21:43:30.795 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.795 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.795 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:30.795 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586645+0000", 2024-09-21T21:43:30.795 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7759999999999997e-06 2024-09-21T21:43:30.795 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.795 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.795 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:30.795 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586652+0000", 2024-09-21T21:43:30.795 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3560000000000004e-06 2024-09-21T21:43:30.795 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.795 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.796 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:30.796 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688395+0000", 2024-09-21T21:43:30.796 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.101743239 2024-09-21T21:43:30.796 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.796 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.796 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:30.796 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688464+0000", 2024-09-21T21:43:30.796 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8273000000000002e-05 2024-09-21T21:43:30.796 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.796 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.796 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:30.796 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.688786+0000", 2024-09-21T21:43:30.796 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000322064 2024-09-21T21:43:30.796 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.796 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.797 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:30.797 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690364+0000", 2024-09-21T21:43:30.797 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001578084 2024-09-21T21:43:30.797 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.797 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.797 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:30.797 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.691050+0000", 2024-09-21T21:43:30.797 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00068623399999999998 2024-09-21T21:43:30.797 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.797 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.797 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:30.797 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.691090+0000", 2024-09-21T21:43:30.797 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9406999999999997e-05 2024-09-21T21:43:30.797 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.797 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.798 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:30.798 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.691140+0000", 2024-09-21T21:43:30.798 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0324000000000001e-05 2024-09-21T21:43:30.798 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.798 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:30.798 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.798 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.798 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.798 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24851 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:30.798 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586661+0000", 2024-09-21T21:43:30.798 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 68.101694682000002, 2024-09-21T21:43:30.798 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.10536891399999999, 2024-09-21T21:43:30.798 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:30.798 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:30.799 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:30.799 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:30.799 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:30.799 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:30.799 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24851 2024-09-21T21:43:30.799 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.799 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:30.799 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.799 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:30.799 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586661+0000", 2024-09-21T21:43:30.799 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:30.799 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.799 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.799 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:30.800 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586661+0000", 2024-09-21T21:43:30.800 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:30.800 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.800 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.800 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:30.800 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586661+0000", 2024-09-21T21:43:30.800 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4700000000000002e-07 2024-09-21T21:43:30.800 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.800 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.800 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:30.800 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586664+0000", 2024-09-21T21:43:30.800 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.672e-06 2024-09-21T21:43:30.800 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.800 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.801 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:30.801 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586671+0000", 2024-09-21T21:43:30.801 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9959999999999996e-06 2024-09-21T21:43:30.801 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.801 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.801 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:30.801 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586676+0000", 2024-09-21T21:43:30.801 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4020000000000001e-06 2024-09-21T21:43:30.801 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.801 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.801 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:30.801 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.689438+0000", 2024-09-21T21:43:30.801 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.102761503 2024-09-21T21:43:30.802 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.802 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.802 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:30.802 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.689524+0000", 2024-09-21T21:43:30.802 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6383000000000003e-05 2024-09-21T21:43:30.802 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.802 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.802 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:30.802 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.689769+0000", 2024-09-21T21:43:30.802 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00024504699999999997 2024-09-21T21:43:30.802 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.802 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.802 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:30.802 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.691936+0000", 2024-09-21T21:43:30.802 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021665130000000001 2024-09-21T21:43:30.803 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.803 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.803 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:30.803 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.691947+0000", 2024-09-21T21:43:30.803 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1198e-05 2024-09-21T21:43:30.803 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.803 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.803 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:30.803 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.691992+0000", 2024-09-21T21:43:30.803 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4774000000000002e-05 2024-09-21T21:43:30.803 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.803 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.803 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:30.803 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.692030+0000", 2024-09-21T21:43:30.803 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7978999999999997e-05 2024-09-21T21:43:30.804 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.804 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:30.804 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.804 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.804 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.804 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.5716.0:24853 3.16 3:6a320e7d:::rbd_object_map.1121c0cf2471:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T21:43:30.804 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-21T21:42:22.586684+0000", 2024-09-21T21:43:30.804 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 68.101671725000003, 2024-09-21T21:43:30.804 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.105936852, 2024-09-21T21:43:30.804 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-21T21:43:30.804 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-21T21:43:30.804 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T21:43:30.804 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-21T21:43:30.804 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.5716", 2024-09-21T21:43:30.804 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.164:0/802865234", 2024-09-21T21:43:30.805 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 24853 2024-09-21T21:43:30.805 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.805 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-21T21:43:30.805 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.805 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-21T21:43:30.805 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586684+0000", 2024-09-21T21:43:30.805 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:30.805 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.805 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.805 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-21T21:43:30.805 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586684+0000", 2024-09-21T21:43:30.805 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-21T21:43:30.805 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.805 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.805 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-21T21:43:30.806 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586684+0000", 2024-09-21T21:43:30.806 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.68e-07 2024-09-21T21:43:30.806 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.806 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.806 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-21T21:43:30.806 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586687+0000", 2024-09-21T21:43:30.806 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.774e-06 2024-09-21T21:43:30.806 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.806 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.806 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-21T21:43:30.806 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586694+0000", 2024-09-21T21:43:30.806 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6959999999999999e-06 2024-09-21T21:43:30.806 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.806 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.806 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-21T21:43:30.807 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.586698+0000", 2024-09-21T21:43:30.807 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7489999999999998e-06 2024-09-21T21:43:30.807 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.807 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.807 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-21T21:43:30.807 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690395+0000", 2024-09-21T21:43:30.807 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.103696645 2024-09-21T21:43:30.807 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.807 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.807 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-21T21:43:30.807 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690440+0000", 2024-09-21T21:43:30.807 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4835e-05 2024-09-21T21:43:30.807 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.807 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.807 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-21T21:43:30.808 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.690654+0000", 2024-09-21T21:43:30.808 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000214114 2024-09-21T21:43:30.808 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.808 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.808 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-21T21:43:30.808 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.692041+0000", 2024-09-21T21:43:30.808 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0013866519999999999 2024-09-21T21:43:30.808 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.808 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.808 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-21T21:43:30.808 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.692558+0000", 2024-09-21T21:43:30.808 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00051717199999999999 2024-09-21T21:43:30.808 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.808 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.808 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-21T21:43:30.808 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.692586+0000", 2024-09-21T21:43:30.809 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8504e-05 2024-09-21T21:43:30.809 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-21T21:43:30.809 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-21T21:43:30.809 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-21T21:43:30.809 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-21T21:42:22.692620+0000", 2024-09-21T21:43:30.809 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4143000000000002e-05 2024-09-21T21:43:30.809 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.809 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:30.809 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.809 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-21T21:43:30.809 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-21T21:43:30.809 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-21T21:43:30.811 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-21T21:43:30.811 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:43:30.809+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:43:30.911 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:43:30.910+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:43:30.918 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:43:30.918 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:43:30.918 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 0 2024-09-21T21:43:30.918 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:43:30.927 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-21T21:43:31.013 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:43:31.010+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:43:31.070 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:43:31.071 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:43:31.071 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:43:31.071 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:43:31.071 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:43:31.080 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-21T21:43:31.120 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:43:31.119+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:43:31.214 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:43:31.213+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:43:31.223 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:43:31.223 INFO:teuthology.orchestra.run.smithi081.stdout: "size": 20, 2024-09-21T21:43:31.223 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 600, 2024-09-21T21:43:31.223 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [ 2024-09-21T21:43:31.223 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.223 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11319 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 5492736~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:31.223 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.712385+0000", 2024-09-21T21:43:31.224 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 100.50821078200001, 2024-09-21T21:43:31.224 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078627889000000006, 2024-09-21T21:43:31.224 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:31.224 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:31.224 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:31.224 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:31.224 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:31.224 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:31.224 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11319 2024-09-21T21:43:31.225 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.225 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:31.225 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.225 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:31.225 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712385+0000", 2024-09-21T21:43:31.225 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:31.225 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.225 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.225 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:31.225 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712385+0000", 2024-09-21T21:43:31.226 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:31.226 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.226 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.226 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:31.226 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712385+0000", 2024-09-21T21:43:31.226 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.2800000000000002e-07 2024-09-21T21:43:31.226 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.226 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.226 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:31.226 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712387+0000", 2024-09-21T21:43:31.226 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1569999999999998e-06 2024-09-21T21:43:31.226 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.227 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.227 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:31.227 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712393+0000", 2024-09-21T21:43:31.227 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.9370000000000001e-06 2024-09-21T21:43:31.227 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.227 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.227 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:31.227 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712398+0000", 2024-09-21T21:43:31.228 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.5020000000000003e-06 2024-09-21T21:43:31.228 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.228 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.228 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:31.228 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.790746+0000", 2024-09-21T21:43:31.228 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078348589999999996 2024-09-21T21:43:31.228 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.228 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.228 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:31.229 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.790844+0000", 2024-09-21T21:43:31.229 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.7526000000000001e-05 2024-09-21T21:43:31.229 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.229 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.229 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:31.229 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.791013+0000", 2024-09-21T21:43:31.229 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00016874899999999999 2024-09-21T21:43:31.229 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:31.229 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:31.229 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:31.230 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.230 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.230 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11320 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 6180864~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:31.230 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.712405+0000", 2024-09-21T21:43:31.230 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 100.50819082300001, 2024-09-21T21:43:31.230 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078882870999999993, 2024-09-21T21:43:31.230 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:31.230 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:31.230 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:31.230 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:31.230 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:31.231 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:31.231 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11320 2024-09-21T21:43:31.231 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.231 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:31.231 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.231 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:31.231 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712405+0000", 2024-09-21T21:43:31.231 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:31.231 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.232 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.232 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:31.232 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.712405+0000", 2024-09-21T21:43:31.232 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:31.232 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.232 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.232 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:31.232 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.713942+0000", 2024-09-21T21:43:31.232 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.001536882 2024-09-21T21:43:31.232 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.232 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.232 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:31.233 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.713946+0000", 2024-09-21T21:43:31.233 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4549999999999997e-06 2024-09-21T21:43:31.233 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.233 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.233 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:31.233 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.713958+0000", 2024-09-21T21:43:31.233 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1775e-05 2024-09-21T21:43:31.233 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.233 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.234 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:31.234 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.713965+0000", 2024-09-21T21:43:31.234 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.5100000000000001e-06 2024-09-21T21:43:31.234 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.234 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.234 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:31.234 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.791059+0000", 2024-09-21T21:43:31.234 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.077094036000000005 2024-09-21T21:43:31.234 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.234 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.235 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:31.235 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.791175+0000", 2024-09-21T21:43:31.235 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000115895 2024-09-21T21:43:31.235 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.235 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.235 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:31.235 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.791288+0000", 2024-09-21T21:43:31.235 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000112318 2024-09-21T21:43:31.235 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:31.236 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:31.236 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:31.236 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.236 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.236 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11325 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 5414912~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:31.236 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.714095+0000", 2024-09-21T21:43:31.236 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 100.506500828, 2024-09-21T21:43:31.236 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078631894999999993, 2024-09-21T21:43:31.236 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:31.237 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:31.237 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:31.237 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:31.237 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:31.237 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:31.237 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11325 2024-09-21T21:43:31.237 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.237 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:31.237 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.237 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:31.237 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714095+0000", 2024-09-21T21:43:31.238 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:31.238 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.238 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.238 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:31.238 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714095+0000", 2024-09-21T21:43:31.238 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:31.238 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.238 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.238 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:31.239 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714095+0000", 2024-09-21T21:43:31.239 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7500000000000001e-07 2024-09-21T21:43:31.239 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.239 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.239 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:31.239 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714097+0000", 2024-09-21T21:43:31.239 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.12e-06 2024-09-21T21:43:31.239 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.239 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.239 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:31.240 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714104+0000", 2024-09-21T21:43:31.240 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.3520000000000003e-06 2024-09-21T21:43:31.240 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.240 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.240 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:31.240 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714108+0000", 2024-09-21T21:43:31.240 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8810000000000004e-06 2024-09-21T21:43:31.240 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.240 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.240 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:31.241 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792509+0000", 2024-09-21T21:43:31.241 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078400739999999997 2024-09-21T21:43:31.241 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.241 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.241 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:31.241 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792613+0000", 2024-09-21T21:43:31.241 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000103939 2024-09-21T21:43:31.241 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.241 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.241 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:31.242 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792727+0000", 2024-09-21T21:43:31.242 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011348799999999999 2024-09-21T21:43:31.242 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:31.242 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:31.242 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:31.242 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.242 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.242 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11326 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 6103040~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:31.243 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.714115+0000", 2024-09-21T21:43:31.243 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 100.50648080000001, 2024-09-21T21:43:31.243 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078963056000000004, 2024-09-21T21:43:31.243 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:31.243 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:31.243 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:31.243 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:31.243 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:31.243 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:31.244 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11326 2024-09-21T21:43:31.244 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.244 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:31.244 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.244 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:31.244 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714115+0000", 2024-09-21T21:43:31.244 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:31.244 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.244 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.244 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:31.245 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.714115+0000", 2024-09-21T21:43:31.245 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:31.245 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.245 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.245 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:31.245 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.715940+0000", 2024-09-21T21:43:31.245 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0018254969999999999 2024-09-21T21:43:31.245 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.245 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.245 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:31.245 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.715945+0000", 2024-09-21T21:43:31.246 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.6120000000000001e-06 2024-09-21T21:43:31.246 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.246 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.246 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:31.246 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.715954+0000", 2024-09-21T21:43:31.246 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.5899999999999997e-06 2024-09-21T21:43:31.246 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.246 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.246 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:31.246 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.715959+0000", 2024-09-21T21:43:31.246 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0849999999999996e-06 2024-09-21T21:43:31.247 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.247 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.247 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:31.247 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792771+0000", 2024-09-21T21:43:31.247 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.076811829999999998 2024-09-21T21:43:31.247 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.247 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.247 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:31.247 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.792882+0000", 2024-09-21T21:43:31.247 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000110857 2024-09-21T21:43:31.247 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.248 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.248 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:31.248 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.793078+0000", 2024-09-21T21:43:31.248 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00019558499999999999 2024-09-21T21:43:31.248 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:31.248 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:31.248 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:31.248 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.248 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.248 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11332 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 7565312~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:31.248 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.716063+0000", 2024-09-21T21:43:31.248 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 100.504532266, 2024-09-21T21:43:31.249 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078787482000000006, 2024-09-21T21:43:31.249 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:31.249 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:31.249 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:31.249 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:31.249 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:31.249 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:31.249 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11332 2024-09-21T21:43:31.249 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.249 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:31.249 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.249 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:31.250 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.716063+0000", 2024-09-21T21:43:31.250 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:31.250 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.250 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.250 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:31.250 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.716063+0000", 2024-09-21T21:43:31.250 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:31.250 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.250 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.250 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:31.250 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.717944+0000", 2024-09-21T21:43:31.251 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0018812309999999999 2024-09-21T21:43:31.251 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.251 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.251 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:31.251 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.717949+0000", 2024-09-21T21:43:31.251 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9389999999999996e-06 2024-09-21T21:43:31.251 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.251 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.251 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:31.251 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.717963+0000", 2024-09-21T21:43:31.251 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.3899e-05 2024-09-21T21:43:31.251 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.252 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.252 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:31.252 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.717970+0000", 2024-09-21T21:43:31.252 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.426e-06 2024-09-21T21:43:31.252 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.252 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.252 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:31.252 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.794573+0000", 2024-09-21T21:43:31.252 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.076602990999999995 2024-09-21T21:43:31.252 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.252 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.252 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:31.252 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.794682+0000", 2024-09-21T21:43:31.252 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000109541 2024-09-21T21:43:31.252 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.253 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.253 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:31.253 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.794851+0000", 2024-09-21T21:43:31.253 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000168455 2024-09-21T21:43:31.253 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:31.253 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:31.253 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:31.253 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.253 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.253 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11339 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 6266880~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:31.253 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.718117+0000", 2024-09-21T21:43:31.253 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 100.50247824, 2024-09-21T21:43:31.254 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078703717000000006, 2024-09-21T21:43:31.254 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:31.254 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:31.254 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:31.254 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:31.254 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:31.254 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:31.254 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11339 2024-09-21T21:43:31.254 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.254 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:31.254 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.254 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:31.255 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.718117+0000", 2024-09-21T21:43:31.255 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:31.255 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.255 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.255 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:31.255 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.718117+0000", 2024-09-21T21:43:31.255 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:31.255 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.255 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.255 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:31.255 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720019+0000", 2024-09-21T21:43:31.255 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0019012549999999999 2024-09-21T21:43:31.256 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.256 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.256 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:31.256 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720024+0000", 2024-09-21T21:43:31.256 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1100000000000002e-06 2024-09-21T21:43:31.256 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.256 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.256 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:31.256 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720035+0000", 2024-09-21T21:43:31.256 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.094e-05 2024-09-21T21:43:31.256 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.256 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.257 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:31.257 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720040+0000", 2024-09-21T21:43:31.257 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1830000000000002e-06 2024-09-21T21:43:31.257 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.257 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.257 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:31.257 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796618+0000", 2024-09-21T21:43:31.257 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.076578033000000004 2024-09-21T21:43:31.257 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.257 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.258 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:31.258 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796696+0000", 2024-09-21T21:43:31.258 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.7836000000000005e-05 2024-09-21T21:43:31.258 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.258 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.258 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:31.258 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.796821+0000", 2024-09-21T21:43:31.258 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00012536000000000001 2024-09-21T21:43:31.258 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:31.258 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:31.258 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:31.258 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.258 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.259 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11345 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 5472256~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:31.259 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.720116+0000", 2024-09-21T21:43:31.259 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 100.50047934, 2024-09-21T21:43:31.259 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078667730000000005, 2024-09-21T21:43:31.259 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:31.259 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:31.259 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:31.259 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:31.259 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:31.259 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:31.259 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11345 2024-09-21T21:43:31.259 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.260 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:31.260 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.260 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:31.260 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720116+0000", 2024-09-21T21:43:31.260 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:31.260 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.260 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.260 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:31.260 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720116+0000", 2024-09-21T21:43:31.260 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:31.260 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.261 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.261 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:31.261 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720116+0000", 2024-09-21T21:43:31.261 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.04e-07 2024-09-21T21:43:31.261 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.261 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.261 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:31.261 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720118+0000", 2024-09-21T21:43:31.261 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.181e-06 2024-09-21T21:43:31.261 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.261 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.261 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:31.261 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720121+0000", 2024-09-21T21:43:31.262 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0199999999999999e-06 2024-09-21T21:43:31.262 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.262 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.262 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:31.262 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720123+0000", 2024-09-21T21:43:31.262 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.5469999999999999e-06 2024-09-21T21:43:31.262 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.262 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.262 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:31.262 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798626+0000", 2024-09-21T21:43:31.263 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078503137000000001 2024-09-21T21:43:31.263 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.263 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.263 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:31.263 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798677+0000", 2024-09-21T21:43:31.263 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.0930999999999997e-05 2024-09-21T21:43:31.263 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.263 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.263 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:31.263 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798784+0000", 2024-09-21T21:43:31.263 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010671 2024-09-21T21:43:31.264 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:31.264 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:31.264 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:31.264 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.264 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.264 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:11346 3.12 3:4a7244b3:::rbd_data.1121524d2522.0000000000000000:1d [read 5500928~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:31.264 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.720126+0000", 2024-09-21T21:43:31.264 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 100.500469267, 2024-09-21T21:43:31.264 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078863421000000003, 2024-09-21T21:43:31.264 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:31.264 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:31.265 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:31.265 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:31.265 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:31.265 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:31.265 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 11346 2024-09-21T21:43:31.265 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.265 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:31.265 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.265 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:31.265 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720126+0000", 2024-09-21T21:43:31.265 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:31.266 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.266 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.266 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:31.266 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.720126+0000", 2024-09-21T21:43:31.266 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:31.266 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.266 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.266 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:31.266 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.722145+0000", 2024-09-21T21:43:31.266 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0020183219999999999 2024-09-21T21:43:31.266 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.267 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.267 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:31.267 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.722150+0000", 2024-09-21T21:43:31.267 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.1760000000000001e-06 2024-09-21T21:43:31.267 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.267 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.267 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:31.267 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.722161+0000", 2024-09-21T21:43:31.267 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.1566000000000001e-05 2024-09-21T21:43:31.267 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.267 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.268 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:31.268 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.722169+0000", 2024-09-21T21:43:31.268 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.2840000000000001e-06 2024-09-21T21:43:31.268 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.268 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.268 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:31.268 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798815+0000", 2024-09-21T21:43:31.268 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.076646607000000005 2024-09-21T21:43:31.268 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.268 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.268 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:31.268 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798870+0000", 2024-09-21T21:43:31.268 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.4486e-05 2024-09-21T21:43:31.269 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.269 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.269 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:31.269 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.798990+0000", 2024-09-21T21:43:31.269 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00011998 2024-09-21T21:43:31.269 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:31.269 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:31.269 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:31.269 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.269 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.269 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13629 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7852032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:31.270 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943319+0000", 2024-09-21T21:43:31.270 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 100.277276625, 2024-09-21T21:43:31.270 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078610975999999999, 2024-09-21T21:43:31.270 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:31.270 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:31.270 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:31.270 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:31.270 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:31.270 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:31.270 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13629 2024-09-21T21:43:31.270 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.270 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:31.270 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.270 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:31.270 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943319+0000", 2024-09-21T21:43:31.271 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:31.271 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.271 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.271 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:31.271 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943319+0000", 2024-09-21T21:43:31.271 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:31.271 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.271 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.271 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:31.271 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943319+0000", 2024-09-21T21:43:31.271 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.6199999999999999e-07 2024-09-21T21:43:31.271 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.271 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.271 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:31.271 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943321+0000", 2024-09-21T21:43:31.272 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1009999999999999e-06 2024-09-21T21:43:31.272 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.272 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.272 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:31.272 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943328+0000", 2024-09-21T21:43:31.272 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.1029999999999996e-06 2024-09-21T21:43:31.272 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.272 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.272 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:31.272 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943333+0000", 2024-09-21T21:43:31.272 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7709999999999997e-06 2024-09-21T21:43:31.272 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.272 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.272 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:31.272 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.021679+0000", 2024-09-21T21:43:31.273 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078345673000000005 2024-09-21T21:43:31.273 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.273 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.273 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:31.273 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.021771+0000", 2024-09-21T21:43:31.273 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2201999999999996e-05 2024-09-21T21:43:31.273 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.273 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.273 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:31.273 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.021930+0000", 2024-09-21T21:43:31.273 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00015876400000000001 2024-09-21T21:43:31.273 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:31.273 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:31.273 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:31.273 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.274 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.274 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13630 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7880704~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:31.274 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943339+0000", 2024-09-21T21:43:31.274 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 100.27725643700001, 2024-09-21T21:43:31.274 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078939410000000002, 2024-09-21T21:43:31.274 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:31.274 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:31.274 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:31.274 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:31.274 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:31.274 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:31.274 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13630 2024-09-21T21:43:31.274 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.274 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:31.275 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.275 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:31.275 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943339+0000", 2024-09-21T21:43:31.275 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:31.275 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.275 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.275 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:31.275 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943339+0000", 2024-09-21T21:43:31.275 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:31.275 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.275 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.275 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:31.275 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943339+0000", 2024-09-21T21:43:31.275 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8000000000000001e-07 2024-09-21T21:43:31.275 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.276 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.276 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:31.276 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943341+0000", 2024-09-21T21:43:31.276 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9850000000000001e-06 2024-09-21T21:43:31.276 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.276 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.276 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:31.276 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943347+0000", 2024-09-21T21:43:31.276 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.2979999999999999e-06 2024-09-21T21:43:31.276 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.276 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.276 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:31.276 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943354+0000", 2024-09-21T21:43:31.276 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.2969999999999998e-06 2024-09-21T21:43:31.276 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.277 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.277 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:31.277 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.021996+0000", 2024-09-21T21:43:31.277 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078641680000000005 2024-09-21T21:43:31.277 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.277 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.277 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:31.277 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022104+0000", 2024-09-21T21:43:31.277 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000107902 2024-09-21T21:43:31.277 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.277 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.277 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:31.277 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022278+0000", 2024-09-21T21:43:31.277 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00017486800000000001 2024-09-21T21:43:31.277 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:31.278 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:31.278 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:31.278 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.278 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.278 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13631 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7909376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:31.278 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943362+0000", 2024-09-21T21:43:31.278 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 100.27723361699999, 2024-09-21T21:43:31.278 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079195644999999995, 2024-09-21T21:43:31.278 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:31.278 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:31.278 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:31.278 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:31.278 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:31.278 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:31.279 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13631 2024-09-21T21:43:31.279 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.279 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:31.279 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.279 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:31.279 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943362+0000", 2024-09-21T21:43:31.279 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:31.279 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.279 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.279 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:31.279 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943362+0000", 2024-09-21T21:43:31.279 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:31.279 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.279 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.279 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:31.280 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943362+0000", 2024-09-21T21:43:31.280 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.8799999999999998e-07 2024-09-21T21:43:31.280 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.280 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.280 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:31.280 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943364+0000", 2024-09-21T21:43:31.280 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9850000000000001e-06 2024-09-21T21:43:31.280 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.280 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.280 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:31.280 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943371+0000", 2024-09-21T21:43:31.280 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 7.0650000000000001e-06 2024-09-21T21:43:31.280 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.280 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.281 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:31.281 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943376+0000", 2024-09-21T21:43:31.281 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8250000000000004e-06 2024-09-21T21:43:31.281 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.281 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.281 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:31.281 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022329+0000", 2024-09-21T21:43:31.281 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078952888999999998 2024-09-21T21:43:31.281 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.281 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.281 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:31.281 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022419+0000", 2024-09-21T21:43:31.281 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.9738e-05 2024-09-21T21:43:31.281 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.281 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.282 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:31.282 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022558+0000", 2024-09-21T21:43:31.282 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00013875500000000001 2024-09-21T21:43:31.282 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:31.282 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:31.282 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:31.282 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.282 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.282 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13632 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7966720~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:31.282 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943382+0000", 2024-09-21T21:43:31.282 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 100.277213383, 2024-09-21T21:43:31.282 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079495727000000002, 2024-09-21T21:43:31.282 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:31.282 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:31.283 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:31.283 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:31.283 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:31.283 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:31.283 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13632 2024-09-21T21:43:31.283 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.283 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:31.283 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.283 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:31.283 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943382+0000", 2024-09-21T21:43:31.283 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:31.283 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.283 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.283 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:31.284 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943382+0000", 2024-09-21T21:43:31.284 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:31.284 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.284 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.284 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:31.284 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943382+0000", 2024-09-21T21:43:31.284 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.7e-07 2024-09-21T21:43:31.284 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.284 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.284 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:31.284 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943385+0000", 2024-09-21T21:43:31.284 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2230000000000001e-06 2024-09-21T21:43:31.284 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.284 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.284 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:31.285 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943391+0000", 2024-09-21T21:43:31.285 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.6810000000000001e-06 2024-09-21T21:43:31.285 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.285 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.285 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:31.285 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943397+0000", 2024-09-21T21:43:31.285 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.6559999999999997e-06 2024-09-21T21:43:31.285 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.285 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.285 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:31.285 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022601+0000", 2024-09-21T21:43:31.285 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079204051999999997 2024-09-21T21:43:31.285 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.285 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.285 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:31.286 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022704+0000", 2024-09-21T21:43:31.286 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00010329599999999999 2024-09-21T21:43:31.286 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.286 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.286 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:31.286 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022878+0000", 2024-09-21T21:43:31.286 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000173449 2024-09-21T21:43:31.286 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:31.286 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:31.286 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:31.286 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.286 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.286 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13633 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7938048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:31.286 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943403+0000", 2024-09-21T21:43:31.287 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 100.277192453, 2024-09-21T21:43:31.287 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079794697999999997, 2024-09-21T21:43:31.287 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:31.287 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:31.287 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:31.287 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:31.287 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:31.287 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:31.287 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13633 2024-09-21T21:43:31.287 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.287 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:31.287 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.287 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:31.287 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943403+0000", 2024-09-21T21:43:31.287 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:31.288 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.288 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.288 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:31.288 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943403+0000", 2024-09-21T21:43:31.288 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:31.288 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.288 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.288 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:31.288 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943403+0000", 2024-09-21T21:43:31.288 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.65e-07 2024-09-21T21:43:31.288 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.288 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.288 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:31.288 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943405+0000", 2024-09-21T21:43:31.289 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.0020000000000001e-06 2024-09-21T21:43:31.289 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.289 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.289 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:31.289 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943412+0000", 2024-09-21T21:43:31.289 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.8379999999999996e-06 2024-09-21T21:43:31.289 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.289 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.289 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:31.289 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943417+0000", 2024-09-21T21:43:31.289 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.7650000000000001e-06 2024-09-21T21:43:31.289 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.289 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.289 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:31.289 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.022922+0000", 2024-09-21T21:43:31.290 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079504512999999999 2024-09-21T21:43:31.290 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.290 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.290 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:31.290 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023039+0000", 2024-09-21T21:43:31.290 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000117044 2024-09-21T21:43:31.290 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.290 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.290 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:31.290 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023198+0000", 2024-09-21T21:43:31.290 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000159171 2024-09-21T21:43:31.290 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:31.290 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:31.290 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:31.290 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.291 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.291 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13634 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 7995392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:31.291 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943423+0000", 2024-09-21T21:43:31.291 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 100.277172745, 2024-09-21T21:43:31.291 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.080142358999999996, 2024-09-21T21:43:31.291 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:31.291 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:31.291 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:31.291 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:31.291 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:31.291 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:31.291 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13634 2024-09-21T21:43:31.291 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.291 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:31.292 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.292 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:31.292 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943423+0000", 2024-09-21T21:43:31.292 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:31.292 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.292 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.292 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:31.292 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943423+0000", 2024-09-21T21:43:31.292 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:31.292 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.292 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.292 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:31.292 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943423+0000", 2024-09-21T21:43:31.292 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.58e-07 2024-09-21T21:43:31.292 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.293 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.293 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:31.293 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943425+0000", 2024-09-21T21:43:31.293 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.9180000000000001e-06 2024-09-21T21:43:31.293 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.293 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.293 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:31.293 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943431+0000", 2024-09-21T21:43:31.293 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.4439999999999996e-06 2024-09-21T21:43:31.293 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.293 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.293 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:31.293 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943436+0000", 2024-09-21T21:43:31.293 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.887e-06 2024-09-21T21:43:31.293 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.294 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.294 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:31.294 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023266+0000", 2024-09-21T21:43:31.294 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079829622000000003 2024-09-21T21:43:31.294 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.294 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.294 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:31.294 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023384+0000", 2024-09-21T21:43:31.294 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000117658 2024-09-21T21:43:31.294 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.294 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.294 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:31.294 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023565+0000", 2024-09-21T21:43:31.294 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.00018147200000000001 2024-09-21T21:43:31.295 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:31.295 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:31.295 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:31.295 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.295 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.295 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13635 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 8024064~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:31.295 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943447+0000", 2024-09-21T21:43:31.295 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 100.277148205, 2024-09-21T21:43:31.295 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.080417653000000006, 2024-09-21T21:43:31.295 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:31.295 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:31.295 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:31.295 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:31.295 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:31.296 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:31.296 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13635 2024-09-21T21:43:31.296 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.296 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:31.296 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.296 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:31.296 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943447+0000", 2024-09-21T21:43:31.296 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:31.296 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.296 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.296 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:31.296 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943447+0000", 2024-09-21T21:43:31.296 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:31.296 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.297 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.297 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:31.297 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943448+0000", 2024-09-21T21:43:31.297 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1699999999999999e-07 2024-09-21T21:43:31.297 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.297 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.297 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:31.297 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943450+0000", 2024-09-21T21:43:31.297 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.1639999999999999e-06 2024-09-21T21:43:31.297 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.297 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.297 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:31.297 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943456+0000", 2024-09-21T21:43:31.298 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.8379999999999998e-06 2024-09-21T21:43:31.298 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.298 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.298 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:31.298 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943460+0000", 2024-09-21T21:43:31.298 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.4700000000000004e-06 2024-09-21T21:43:31.298 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.298 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.298 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:31.298 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023595+0000", 2024-09-21T21:43:31.298 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.080134869999999997 2024-09-21T21:43:31.298 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.298 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.298 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:31.299 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023688+0000", 2024-09-21T21:43:31.299 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2794000000000006e-05 2024-09-21T21:43:31.299 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.299 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.299 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:31.299 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023865+0000", 2024-09-21T21:43:31.299 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0001771 2024-09-21T21:43:31.299 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:31.299 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:31.299 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:31.299 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.299 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.299 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13636 3.12 3:4ed8c3b8:::rbd_data.1121524d2522.0000000000000001:1d [read 8052736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:31.299 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.943467+0000", 2024-09-21T21:43:31.299 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 100.277128468, 2024-09-21T21:43:31.300 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.080694399999999999, 2024-09-21T21:43:31.300 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:31.300 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:31.300 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "started", 2024-09-21T21:43:31.300 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:31.300 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:31.300 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:31.300 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13636 2024-09-21T21:43:31.300 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.300 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:31.300 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.300 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:31.300 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943467+0000", 2024-09-21T21:43:31.300 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:31.301 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.301 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.301 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:31.301 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.943467+0000", 2024-09-21T21:43:31.301 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:31.301 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.301 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.301 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:31.301 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944940+0000", 2024-09-21T21:43:31.301 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.0014725109999999999 2024-09-21T21:43:31.301 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.301 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.301 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:31.301 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944949+0000", 2024-09-21T21:43:31.302 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 9.2930000000000006e-06 2024-09-21T21:43:31.302 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.302 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.302 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:31.302 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944958+0000", 2024-09-21T21:43:31.302 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.6899999999999998e-06 2024-09-21T21:43:31.302 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.302 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.302 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:31.302 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944966+0000", 2024-09-21T21:43:31.302 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.0690000000000002e-06 2024-09-21T21:43:31.302 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.302 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.303 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:31.303 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.023900+0000", 2024-09-21T21:43:31.303 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.078934371000000003 2024-09-21T21:43:31.303 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.303 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.303 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "started", 2024-09-21T21:43:31.303 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024009+0000", 2024-09-21T21:43:31.303 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000108857 2024-09-21T21:43:31.303 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.303 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.303 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:31.303 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024161+0000", 2024-09-21T21:43:31.303 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000152609 2024-09-21T21:43:31.303 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:31.303 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:31.304 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:31.304 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.304 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.304 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13637 3.12 3.1dc31b72 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:31.304 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.944978+0000", 2024-09-21T21:43:31.304 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 100.275617533, 2024-09-21T21:43:31.304 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079232604999999998, 2024-09-21T21:43:31.304 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:31.304 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:31.304 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:43:31.304 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:31.304 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:31.304 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:31.304 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13637 2024-09-21T21:43:31.304 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.305 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:31.305 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.305 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:31.305 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944978+0000", 2024-09-21T21:43:31.305 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:31.305 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.305 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.305 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:31.305 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944978+0000", 2024-09-21T21:43:31.305 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:31.305 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.305 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.305 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:31.305 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944978+0000", 2024-09-21T21:43:31.305 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.9100000000000004e-07 2024-09-21T21:43:31.306 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.306 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.306 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:31.306 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944981+0000", 2024-09-21T21:43:31.306 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.3290000000000001e-06 2024-09-21T21:43:31.306 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.306 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.306 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:31.306 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944987+0000", 2024-09-21T21:43:31.306 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 6.28e-06 2024-09-21T21:43:31.306 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.306 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.306 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:31.306 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.944992+0000", 2024-09-21T21:43:31.306 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.8520000000000003e-06 2024-09-21T21:43:31.307 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.307 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.307 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:31.307 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024196+0000", 2024-09-21T21:43:31.307 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079203673000000002 2024-09-21T21:43:31.307 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.307 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.307 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:31.307 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024211+0000", 2024-09-21T21:43:31.307 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.4980000000000001e-05 2024-09-21T21:43:31.307 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:31.307 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:31.307 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:31.307 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.308 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.308 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13638 3.12 3.1dc31b72 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:31.308 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.945000+0000", 2024-09-21T21:43:31.308 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 100.275595405, 2024-09-21T21:43:31.308 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079257465999999999, 2024-09-21T21:43:31.308 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:31.308 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:31.308 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:43:31.308 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:31.308 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:31.308 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:31.308 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13638 2024-09-21T21:43:31.308 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.308 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:31.309 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.309 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:31.309 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945000+0000", 2024-09-21T21:43:31.309 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:31.309 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.309 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.309 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:31.309 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945000+0000", 2024-09-21T21:43:31.309 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:31.309 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.309 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.309 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:31.309 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945001+0000", 2024-09-21T21:43:31.309 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.32e-07 2024-09-21T21:43:31.309 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.310 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.310 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:31.310 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945003+0000", 2024-09-21T21:43:31.310 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.2309999999999999e-06 2024-09-21T21:43:31.310 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.310 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.310 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:31.310 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945009+0000", 2024-09-21T21:43:31.310 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 5.7450000000000001e-06 2024-09-21T21:43:31.310 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.310 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.310 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:31.310 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945013+0000", 2024-09-21T21:43:31.310 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.673e-06 2024-09-21T21:43:31.310 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.311 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.311 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:31.311 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024245+0000", 2024-09-21T21:43:31.311 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079232160999999995 2024-09-21T21:43:31.311 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.311 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.311 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:31.311 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024258+0000", 2024-09-21T21:43:31.311 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.2224e-05 2024-09-21T21:43:31.311 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:31.311 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:31.311 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:31.311 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.311 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.311 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13639 3.12 3.1dc31b72 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:31.312 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.945034+0000", 2024-09-21T21:43:31.312 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 100.275561606, 2024-09-21T21:43:31.312 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079260627, 2024-09-21T21:43:31.312 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:31.312 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:31.312 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:43:31.312 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:31.312 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:31.312 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:31.312 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13639 2024-09-21T21:43:31.312 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.312 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:31.312 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.312 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:31.312 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945034+0000", 2024-09-21T21:43:31.313 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:31.313 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.313 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.313 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:31.313 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945034+0000", 2024-09-21T21:43:31.313 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:31.313 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.313 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.313 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:31.313 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945034+0000", 2024-09-21T21:43:31.313 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.3799999999999998e-07 2024-09-21T21:43:31.313 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.313 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.313 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:31.313 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945036+0000", 2024-09-21T21:43:31.313 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.052e-06 2024-09-21T21:43:31.314 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.314 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.314 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:31.314 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945040+0000", 2024-09-21T21:43:31.314 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1169999999999997e-06 2024-09-21T21:43:31.314 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.314 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.314 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:31.314 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945043+0000", 2024-09-21T21:43:31.314 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.8439999999999998e-06 2024-09-21T21:43:31.314 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.314 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.314 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:31.314 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024286+0000", 2024-09-21T21:43:31.315 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079242483000000002 2024-09-21T21:43:31.315 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.315 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.315 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:31.315 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024295+0000", 2024-09-21T21:43:31.315 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 8.7930000000000003e-06 2024-09-21T21:43:31.315 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:31.315 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:31.315 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:31.315 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.315 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.315 INFO:teuthology.orchestra.run.smithi081.stdout: "description": "osd_op(client.5177.0:13640 3.12 3.1dc31b72 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-21T21:43:31.315 INFO:teuthology.orchestra.run.smithi081.stdout: "initiated_at": "2024-09-21T21:41:50.945047+0000", 2024-09-21T21:43:31.315 INFO:teuthology.orchestra.run.smithi081.stdout: "age": 100.27554808399999, 2024-09-21T21:43:31.315 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079412321999999994, 2024-09-21T21:43:31.315 INFO:teuthology.orchestra.run.smithi081.stdout: "continuous": false, 2024-09-21T21:43:31.316 INFO:teuthology.orchestra.run.smithi081.stdout: "type_data": { 2024-09-21T21:43:31.316 INFO:teuthology.orchestra.run.smithi081.stdout: "flag_point": "reached pg", 2024-09-21T21:43:31.316 INFO:teuthology.orchestra.run.smithi081.stdout: "client_info": { 2024-09-21T21:43:31.316 INFO:teuthology.orchestra.run.smithi081.stdout: "client": "client.5177", 2024-09-21T21:43:31.316 INFO:teuthology.orchestra.run.smithi081.stdout: "client_addr": "172.21.15.164:0/3782311581", 2024-09-21T21:43:31.316 INFO:teuthology.orchestra.run.smithi081.stdout: "tid": 13640 2024-09-21T21:43:31.316 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.316 INFO:teuthology.orchestra.run.smithi081.stdout: "events": [ 2024-09-21T21:43:31.316 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.316 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "initiated", 2024-09-21T21:43:31.316 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945047+0000", 2024-09-21T21:43:31.316 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:31.316 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.316 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.316 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "header_read", 2024-09-21T21:43:31.317 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945047+0000", 2024-09-21T21:43:31.317 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0 2024-09-21T21:43:31.317 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.317 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.317 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "throttled", 2024-09-21T21:43:31.317 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945048+0000", 2024-09-21T21:43:31.317 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 3.0499999999999999e-07 2024-09-21T21:43:31.317 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.317 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.317 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "all_read", 2024-09-21T21:43:31.317 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945049+0000", 2024-09-21T21:43:31.317 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 1.3939999999999999e-06 2024-09-21T21:43:31.317 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.317 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.318 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "dispatched", 2024-09-21T21:43:31.318 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945053+0000", 2024-09-21T21:43:31.318 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 4.1760000000000003e-06 2024-09-21T21:43:31.318 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.318 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.318 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "queued_for_pg", 2024-09-21T21:43:31.318 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:50.945056+0000", 2024-09-21T21:43:31.318 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 2.6129999999999998e-06 2024-09-21T21:43:31.318 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.318 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.318 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "reached_pg", 2024-09-21T21:43:31.318 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024360+0000", 2024-09-21T21:43:31.318 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.079303645000000006 2024-09-21T21:43:31.318 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-09-21T21:43:31.318 INFO:teuthology.orchestra.run.smithi081.stdout: { 2024-09-21T21:43:31.319 INFO:teuthology.orchestra.run.smithi081.stdout: "event": "done", 2024-09-21T21:43:31.319 INFO:teuthology.orchestra.run.smithi081.stdout: "time": "2024-09-21T21:41:51.024460+0000", 2024-09-21T21:43:31.319 INFO:teuthology.orchestra.run.smithi081.stdout: "duration": 0.000100189 2024-09-21T21:43:31.319 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:31.319 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:31.319 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:31.319 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-09-21T21:43:31.319 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-09-21T21:43:31.319 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:43:31.320 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:43:31.319+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:43:31.321 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-21T21:43:31.367 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 5, 3, 0, 4] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-21T21:43:31.367 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T21:43:31.368 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-21T21:43:31.368 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-21T21:43:31.421 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:43:31.420+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:43:31.430 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:43:31.430 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:43:31.430 INFO:teuthology.orchestra.run.smithi081.stdout: "num_ops": 0 2024-09-21T21:43:31.430 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:43:31.440 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-21T21:43:31.490 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T21:43:31.502 INFO:teuthology.orchestra.run.smithi081.stderr:no valid command found; 10 closest matches: 2024-09-21T21:43:31.502 INFO:teuthology.orchestra.run.smithi081.stderr:0 2024-09-21T21:43:31.503 INFO:teuthology.orchestra.run.smithi081.stderr:1 2024-09-21T21:43:31.503 INFO:teuthology.orchestra.run.smithi081.stderr:2 2024-09-21T21:43:31.503 INFO:teuthology.orchestra.run.smithi081.stderr:abort 2024-09-21T21:43:31.503 INFO:teuthology.orchestra.run.smithi081.stderr:assert 2024-09-21T21:43:31.503 INFO:teuthology.orchestra.run.smithi081.stderr:bench [] [] [] [] 2024-09-21T21:43:31.503 INFO:teuthology.orchestra.run.smithi081.stderr:bluefs debug_inject_read_zeros 2024-09-21T21:43:31.503 INFO:teuthology.orchestra.run.smithi081.stderr:bluefs files list 2024-09-21T21:43:31.503 INFO:teuthology.orchestra.run.smithi081.stderr:bluefs stats 2024-09-21T21:43:31.503 INFO:teuthology.orchestra.run.smithi081.stderr:bluestore allocator dump block 2024-09-21T21:43:31.503 INFO:teuthology.orchestra.run.smithi081.stderr:Error EINVAL: invalid command 2024-09-21T21:43:31.506 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-21T21:43:31.506 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-21T21:43:31.506 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-21T21:43:31.522 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:43:31.521+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:43:31.546 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:43:31.546 INFO:teuthology.orchestra.run.smithi081.stdout: "ops": [], 2024-09-21T21:43:31.546 INFO:teuthology.orchestra.run.smithi081.stdout: "complaint_time": 30, 2024-09-21T21:43:31.546 INFO:teuthology.orchestra.run.smithi081.stdout: "num_blocked_ops": 0 2024-09-21T21:43:31.546 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:43:31.555 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-21T21:43:31.628 INFO:teuthology.orchestra.run.smithi081.stdout:osd.0: {} 2024-09-21T21:43:31.628 INFO:teuthology.orchestra.run.smithi081.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T21:43:31.629 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:43:31.627+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:43:31.645 INFO:teuthology.orchestra.run.smithi081.stdout:osd.1: {} 2024-09-21T21:43:31.645 INFO:teuthology.orchestra.run.smithi081.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T21:43:31.661 INFO:teuthology.orchestra.run.smithi081.stdout:osd.2: {} 2024-09-21T21:43:31.662 INFO:teuthology.orchestra.run.smithi081.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T21:43:31.679 INFO:teuthology.orchestra.run.smithi081.stdout:osd.3: {} 2024-09-21T21:43:31.679 INFO:teuthology.orchestra.run.smithi081.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T21:43:31.690 INFO:teuthology.orchestra.run.smithi081.stderr:no valid command found; 10 closest matches: 2024-09-21T21:43:31.691 INFO:teuthology.orchestra.run.smithi081.stderr:0 2024-09-21T21:43:31.691 INFO:teuthology.orchestra.run.smithi081.stderr:1 2024-09-21T21:43:31.691 INFO:teuthology.orchestra.run.smithi081.stderr:2 2024-09-21T21:43:31.691 INFO:teuthology.orchestra.run.smithi081.stderr:abort 2024-09-21T21:43:31.691 INFO:teuthology.orchestra.run.smithi081.stderr:assert 2024-09-21T21:43:31.691 INFO:teuthology.orchestra.run.smithi081.stderr:bench [] [] [] [] 2024-09-21T21:43:31.691 INFO:teuthology.orchestra.run.smithi081.stderr:bluefs debug_inject_read_zeros 2024-09-21T21:43:31.691 INFO:teuthology.orchestra.run.smithi081.stderr:bluefs files list 2024-09-21T21:43:31.691 INFO:teuthology.orchestra.run.smithi081.stderr:bluefs stats 2024-09-21T21:43:31.691 INFO:teuthology.orchestra.run.smithi081.stderr:bluestore allocator dump block 2024-09-21T21:43:31.691 INFO:teuthology.orchestra.run.smithi081.stderr:Error EINVAL: invalid command 2024-09-21T21:43:31.693 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-21T21:43:31.694 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-21T21:43:31.694 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 'trim stale osdmaps' 2024-09-21T21:43:31.695 INFO:teuthology.orchestra.run.smithi081.stdout:osd.4: {} 2024-09-21T21:43:31.696 INFO:teuthology.orchestra.run.smithi081.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T21:43:31.702 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:31.703 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:31.713 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-21T21:43:31.715 INFO:teuthology.orchestra.run.smithi081.stdout:osd.5: {} 2024-09-21T21:43:31.715 INFO:teuthology.orchestra.run.smithi081.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T21:43:31.727 INFO:tasks.ceph.osd.2.smithi081.stderr:2024-09-21T21:43:31.725+0000 7fd4a3dee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 31965) UID: 0 2024-09-21T21:43:31.820 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:31.820 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:31.825 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:43:31.824+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:43:31.828 INFO:teuthology.orchestra.run.smithi081.stderr:no valid command found; 10 closest matches: 2024-09-21T21:43:31.828 INFO:teuthology.orchestra.run.smithi081.stderr:0 2024-09-21T21:43:31.828 INFO:teuthology.orchestra.run.smithi081.stderr:1 2024-09-21T21:43:31.828 INFO:teuthology.orchestra.run.smithi081.stderr:2 2024-09-21T21:43:31.828 INFO:teuthology.orchestra.run.smithi081.stderr:abort 2024-09-21T21:43:31.828 INFO:teuthology.orchestra.run.smithi081.stderr:assert 2024-09-21T21:43:31.828 INFO:teuthology.orchestra.run.smithi081.stderr:bench [] [] [] [] 2024-09-21T21:43:31.828 INFO:teuthology.orchestra.run.smithi081.stderr:bluefs debug_inject_read_zeros 2024-09-21T21:43:31.828 INFO:teuthology.orchestra.run.smithi081.stderr:bluefs files list 2024-09-21T21:43:31.829 INFO:teuthology.orchestra.run.smithi081.stderr:bluefs stats 2024-09-21T21:43:31.829 INFO:teuthology.orchestra.run.smithi081.stderr:bluestore allocator dump block 2024-09-21T21:43:31.829 INFO:teuthology.orchestra.run.smithi081.stderr:Error EINVAL: invalid command 2024-09-21T21:43:31.830 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-21T21:43:31.830 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-21T21:43:31.831 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-21T21:43:31.831 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-21T21:43:31.926 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:43:31.924+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:43:31.977 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:31.977 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:31.986 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-21T21:43:32.004 INFO:teuthology.orchestra.run.smithi081.stderr:no valid command found; 10 closest matches: 2024-09-21T21:43:32.004 INFO:teuthology.orchestra.run.smithi081.stderr:0 2024-09-21T21:43:32.004 INFO:teuthology.orchestra.run.smithi081.stderr:1 2024-09-21T21:43:32.004 INFO:teuthology.orchestra.run.smithi081.stderr:2 2024-09-21T21:43:32.004 INFO:teuthology.orchestra.run.smithi081.stderr:abort 2024-09-21T21:43:32.004 INFO:teuthology.orchestra.run.smithi081.stderr:assert 2024-09-21T21:43:32.004 INFO:teuthology.orchestra.run.smithi081.stderr:bench [] [] [] [] 2024-09-21T21:43:32.004 INFO:teuthology.orchestra.run.smithi081.stderr:bluefs debug_inject_read_zeros 2024-09-21T21:43:32.004 INFO:teuthology.orchestra.run.smithi081.stderr:bluefs files list 2024-09-21T21:43:32.004 INFO:teuthology.orchestra.run.smithi081.stderr:bluefs stats 2024-09-21T21:43:32.005 INFO:teuthology.orchestra.run.smithi081.stderr:bluestore allocator dump block 2024-09-21T21:43:32.005 INFO:teuthology.orchestra.run.smithi081.stderr:Error EINVAL: invalid command 2024-09-21T21:43:32.007 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-21T21:43:32.007 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-21T21:43:32.007 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-21T21:43:32.027 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:43:32.026+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:43:32.109 INFO:teuthology.orchestra.run.smithi081.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:32.109 INFO:teuthology.orchestra.run.smithi081.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:32.119 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-21T21:43:32.162 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:43:32.161+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:43:32.178 INFO:teuthology.orchestra.run.smithi081.stderr:no valid command found; 10 closest matches: 2024-09-21T21:43:32.178 INFO:teuthology.orchestra.run.smithi081.stderr:0 2024-09-21T21:43:32.178 INFO:teuthology.orchestra.run.smithi081.stderr:1 2024-09-21T21:43:32.179 INFO:teuthology.orchestra.run.smithi081.stderr:2 2024-09-21T21:43:32.179 INFO:teuthology.orchestra.run.smithi081.stderr:abort 2024-09-21T21:43:32.179 INFO:teuthology.orchestra.run.smithi081.stderr:assert 2024-09-21T21:43:32.179 INFO:teuthology.orchestra.run.smithi081.stderr:bench [] [] [] [] 2024-09-21T21:43:32.179 INFO:teuthology.orchestra.run.smithi081.stderr:bluefs debug_inject_read_zeros 2024-09-21T21:43:32.179 INFO:teuthology.orchestra.run.smithi081.stderr:bluefs files list 2024-09-21T21:43:32.179 INFO:teuthology.orchestra.run.smithi081.stderr:bluefs stats 2024-09-21T21:43:32.179 INFO:teuthology.orchestra.run.smithi081.stderr:bluestore allocator dump block 2024-09-21T21:43:32.179 INFO:teuthology.orchestra.run.smithi081.stderr:Error EINVAL: invalid command 2024-09-21T21:43:32.181 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-21T21:43:32.181 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-21T21:43:32.181 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 'trim stale osdmaps' 2024-09-21T21:43:32.224 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:32.224 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:32.228 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:43:32.227+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:43:32.234 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-21T21:43:32.301 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T21:43:32.327 INFO:teuthology.orchestra.run.smithi081.stderr:no valid command found; 10 closest matches: 2024-09-21T21:43:32.327 INFO:teuthology.orchestra.run.smithi081.stderr:0 2024-09-21T21:43:32.328 INFO:teuthology.orchestra.run.smithi081.stderr:1 2024-09-21T21:43:32.328 INFO:teuthology.orchestra.run.smithi081.stderr:2 2024-09-21T21:43:32.328 INFO:teuthology.orchestra.run.smithi081.stderr:abort 2024-09-21T21:43:32.328 INFO:teuthology.orchestra.run.smithi081.stderr:assert 2024-09-21T21:43:32.328 INFO:teuthology.orchestra.run.smithi081.stderr:bench [] [] [] [] 2024-09-21T21:43:32.328 INFO:teuthology.orchestra.run.smithi081.stderr:bluefs debug_inject_read_zeros 2024-09-21T21:43:32.328 INFO:teuthology.orchestra.run.smithi081.stderr:bluefs files list 2024-09-21T21:43:32.328 INFO:teuthology.orchestra.run.smithi081.stderr:bluefs stats 2024-09-21T21:43:32.328 INFO:teuthology.orchestra.run.smithi081.stderr:bluestore allocator dump block 2024-09-21T21:43:32.328 INFO:teuthology.orchestra.run.smithi081.stderr:Error EINVAL: invalid command 2024-09-21T21:43:32.329 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:43:32.328+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:43:32.330 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-21T21:43:32.331 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-21T21:43:32.383 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:32.383 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:32.392 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-21T21:43:32.432 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:43:32.431+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:43:32.531 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-09-21T21:43:32.529+0000 7f0434d29640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32207) UID: 0 2024-09-21T21:43:32.535 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:32.535 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:32.544 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-21T21:43:32.632 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:43:32.631+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:43:32.688 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:32.689 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:32.697 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-21T21:43:32.738 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:43:32.737+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:43:32.834 INFO:tasks.ceph.osd.0.smithi081.stderr:2024-09-21T21:43:32.833+0000 7faedeb33640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 31955) UID: 0 2024-09-21T21:43:32.840 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:32.840 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:32.849 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-21T21:43:32.935 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:43:32.934+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:43:32.950 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:32.951 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:32.959 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-21T21:43:33.043 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:43:33.042+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:43:33.103 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:33.103 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:33.112 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-21T21:43:33.154 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-21T21:43:33.152+0000 7fc96df4c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32210) UID: 0 2024-09-21T21:43:33.238 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-21T21:43:33.236+0000 7f7fb64ef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32212) UID: 0 2024-09-21T21:43:33.256 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:33.257 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:33.265 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-21T21:43:33.292 INFO:teuthology.orchestra.run.smithi081.stderr:nodeep-scrub is unset 2024-09-21T21:43:33.348 INFO:tasks.ceph.osd.1.smithi081.stderr:2024-09-21T21:43:33.347+0000 7ff409aef640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 31961) UID: 0 2024-09-21T21:43:33.398 INFO:teuthology.orchestra.run.smithi164.stdout:All operations completed A-OK! 2024-09-21T21:43:33.409 DEBUG:teuthology.parallel:result is None 2024-09-21T21:43:33.409 DEBUG:teuthology.run_tasks:Unwinding manager rbd_fsx 2024-09-21T21:43:33.426 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-09-21T21:43:33.482 INFO:tasks.thrashosds:joining thrashosds 2024-09-21T21:43:33.483 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-21T21:43:33.483 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T21:43:35.308 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-21T21:43:36.304 INFO:teuthology.orchestra.run.smithi081.stderr:noscrub is unset 2024-09-21T21:43:36.320 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-21T21:43:37.220 INFO:teuthology.orchestra.run.smithi081.stderr:nodeep-scrub is unset 2024-09-21T21:43:37.332 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-09-21T21:43:37.446 INFO:teuthology.orchestra.run.smithi081.stdout:mon.a: {} 2024-09-21T21:43:37.446 INFO:teuthology.orchestra.run.smithi081.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-21T21:43:37.456 INFO:teuthology.orchestra.run.smithi081.stdout:mon.b: {} 2024-09-21T21:43:37.456 INFO:teuthology.orchestra.run.smithi081.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-21T21:43:37.464 INFO:teuthology.orchestra.run.smithi081.stdout:mon.c: {} 2024-09-21T21:43:37.465 INFO:teuthology.orchestra.run.smithi081.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-21T21:43:37.475 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-21T21:43:38.312 INFO:teuthology.orchestra.run.smithi081.stderr:reweighted osd.0 to 1 (10000) 2024-09-21T21:43:38.328 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-09-21T21:43:39.317 INFO:teuthology.orchestra.run.smithi081.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-21T21:43:39.332 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-21T21:43:40.320 INFO:teuthology.orchestra.run.smithi081.stderr:reweighted osd.1 to 1 (10000) 2024-09-21T21:43:40.336 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-09-21T21:43:41.330 INFO:teuthology.orchestra.run.smithi081.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-21T21:43:41.348 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-21T21:43:42.221 INFO:teuthology.orchestra.run.smithi081.stderr:reweighted osd.2 to 1 (10000) 2024-09-21T21:43:42.238 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-09-21T21:43:43.347 INFO:teuthology.orchestra.run.smithi081.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-21T21:43:43.364 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-21T21:43:44.356 INFO:teuthology.orchestra.run.smithi081.stderr:reweighted osd.3 to 1 (10000) 2024-09-21T21:43:44.371 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-09-21T21:43:45.353 INFO:teuthology.orchestra.run.smithi081.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-21T21:43:45.369 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-21T21:43:46.368 INFO:teuthology.orchestra.run.smithi081.stderr:reweighted osd.4 to 1 (10000) 2024-09-21T21:43:46.383 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-09-21T21:43:47.222 INFO:teuthology.orchestra.run.smithi081.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-21T21:43:47.238 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-21T21:43:48.224 INFO:teuthology.orchestra.run.smithi081.stderr:reweighted osd.5 to 1 (10000) 2024-09-21T21:43:48.241 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-09-21T21:43:49.370 INFO:teuthology.orchestra.run.smithi081.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-21T21:43:49.387 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-09-21T21:43:49.387 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-09-21T21:43:49.387 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-09-21T21:43:49.387 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-09-21T21:43:49.387 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-21T21:43:49.388 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T21:43:49.668 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:43:49.669 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":185,"fsid":"a0fdcfc2-ddd2-4e50-81bb-876d940f3578","created":"2024-09-21T21:40:07.413917+0000","modified":"2024-09-21T21:43:49.359995+0000","last_up_change":"2024-09-21T21:40:18.340430+0000","last_in_change":"2024-09-21T21:43:14.301223+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T21:40:18.321328+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-21T21:40:24.235658+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":19,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-21T21:40:36.093691+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":48,"snap_epoch":171,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9869830-8e74-4b02-963d-ab630a0ee286","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1657552338},{"type":"v1","addr":"172.21.15.81:6809","nonce":1657552338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1657552338},{"type":"v1","addr":"172.21.15.81:6811","nonce":1657552338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1657552338},{"type":"v1","addr":"172.21.15.81:6815","nonce":1657552338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1657552338},{"type":"v1","addr":"172.21.15.81:6813","nonce":1657552338}]},"public_addr":"172.21.15.81:6809/1657552338","cluster_addr":"172.21.15.81:6811/1657552338","heartbeat_back_addr":"172.21.15.81:6815/1657552338","heartbeat_front_addr":"172.21.15.81:6813/1657552338","state":["exists","up"]},{"osd":1,"uuid":"b189819d-69a3-4e95-96e7-96dc4809ac80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2864159502},{"type":"v1","addr":"172.21.15.81:6817","nonce":2864159502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2864159502},{"type":"v1","addr":"172.21.15.81:6819","nonce":2864159502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2864159502},{"type":"v1","addr":"172.21.15.81:6823","nonce":2864159502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2864159502},{"type":"v1","addr":"172.21.15.81:6821","nonce":2864159502}]},"public_addr":"172.21.15.81:6817/2864159502","cluster_addr":"172.21.15.81:6819/2864159502","heartbeat_back_addr":"172.21.15.81:6823/2864159502","heartbeat_front_addr":"172.21.15.81:6821/2864159502","state":["exists","up"]},{"osd":2,"uuid":"8437b99a-1297-4903-ab66-95c5cba2b2bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3023160932},{"type":"v1","addr":"172.21.15.81:6801","nonce":3023160932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3023160932},{"type":"v1","addr":"172.21.15.81:6803","nonce":3023160932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3023160932},{"type":"v1","addr":"172.21.15.81:6807","nonce":3023160932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3023160932},{"type":"v1","addr":"172.21.15.81:6805","nonce":3023160932}]},"public_addr":"172.21.15.81:6801/3023160932","cluster_addr":"172.21.15.81:6803/3023160932","heartbeat_back_addr":"172.21.15.81:6807/3023160932","heartbeat_front_addr":"172.21.15.81:6805/3023160932","state":["exists","up"]},{"osd":3,"uuid":"e8848768-c1aa-445b-9554-dc6aa4eefc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":2681016844},{"type":"v1","addr":"172.21.15.122:6809","nonce":2681016844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2681016844},{"type":"v1","addr":"172.21.15.122:6811","nonce":2681016844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2681016844},{"type":"v1","addr":"172.21.15.122:6815","nonce":2681016844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2681016844},{"type":"v1","addr":"172.21.15.122:6813","nonce":2681016844}]},"public_addr":"172.21.15.122:6809/2681016844","cluster_addr":"172.21.15.122:6811/2681016844","heartbeat_back_addr":"172.21.15.122:6815/2681016844","heartbeat_front_addr":"172.21.15.122:6813/2681016844","state":["exists","up"]},{"osd":4,"uuid":"b1d8e85d-a207-4ba6-9322-61d07812083e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":1095023540},{"type":"v1","addr":"172.21.15.122:6801","nonce":1095023540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":1095023540},{"type":"v1","addr":"172.21.15.122:6803","nonce":1095023540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":1095023540},{"type":"v1","addr":"172.21.15.122:6807","nonce":1095023540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":1095023540},{"type":"v1","addr":"172.21.15.122:6805","nonce":1095023540}]},"public_addr":"172.21.15.122:6801/1095023540","cluster_addr":"172.21.15.122:6803/1095023540","heartbeat_back_addr":"172.21.15.122:6807/1095023540","heartbeat_front_addr":"172.21.15.122:6805/1095023540","state":["exists","up"]},{"osd":5,"uuid":"3ab0450b-d8fd-4ef2-a298-08d5aa2ea32f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1974398333},{"type":"v1","addr":"172.21.15.122:6817","nonce":1974398333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1974398333},{"type":"v1","addr":"172.21.15.122:6819","nonce":1974398333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1974398333},{"type":"v1","addr":"172.21.15.122:6823","nonce":1974398333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1974398333},{"type":"v1","addr":"172.21.15.122:6821","nonce":1974398333}]},"public_addr":"172.21.15.122:6817/1974398333","cluster_addr":"172.21.15.122:6819/1974398333","heartbeat_back_addr":"172.21.15.122:6823/1974398333","heartbeat_front_addr":"172.21.15.122:6821/1974398333","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T21:40:15.766648+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T21:40:15.457496+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T21:40:14.767039+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T21:40:15.355570+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T21:40:15.335729+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T21:40:15.614733+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.8","osds":[2,3]}],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T21:43:49.682 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-21T21:43:49.682 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T21:43:49.959 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:43:49.959 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":185,"fsid":"a0fdcfc2-ddd2-4e50-81bb-876d940f3578","created":"2024-09-21T21:40:07.413917+0000","modified":"2024-09-21T21:43:49.359995+0000","last_up_change":"2024-09-21T21:40:18.340430+0000","last_in_change":"2024-09-21T21:43:14.301223+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T21:40:18.321328+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-21T21:40:24.235658+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":19,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-21T21:40:36.093691+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":48,"snap_epoch":171,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9869830-8e74-4b02-963d-ab630a0ee286","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1657552338},{"type":"v1","addr":"172.21.15.81:6809","nonce":1657552338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1657552338},{"type":"v1","addr":"172.21.15.81:6811","nonce":1657552338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1657552338},{"type":"v1","addr":"172.21.15.81:6815","nonce":1657552338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1657552338},{"type":"v1","addr":"172.21.15.81:6813","nonce":1657552338}]},"public_addr":"172.21.15.81:6809/1657552338","cluster_addr":"172.21.15.81:6811/1657552338","heartbeat_back_addr":"172.21.15.81:6815/1657552338","heartbeat_front_addr":"172.21.15.81:6813/1657552338","state":["exists","up"]},{"osd":1,"uuid":"b189819d-69a3-4e95-96e7-96dc4809ac80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2864159502},{"type":"v1","addr":"172.21.15.81:6817","nonce":2864159502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2864159502},{"type":"v1","addr":"172.21.15.81:6819","nonce":2864159502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2864159502},{"type":"v1","addr":"172.21.15.81:6823","nonce":2864159502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2864159502},{"type":"v1","addr":"172.21.15.81:6821","nonce":2864159502}]},"public_addr":"172.21.15.81:6817/2864159502","cluster_addr":"172.21.15.81:6819/2864159502","heartbeat_back_addr":"172.21.15.81:6823/2864159502","heartbeat_front_addr":"172.21.15.81:6821/2864159502","state":["exists","up"]},{"osd":2,"uuid":"8437b99a-1297-4903-ab66-95c5cba2b2bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3023160932},{"type":"v1","addr":"172.21.15.81:6801","nonce":3023160932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3023160932},{"type":"v1","addr":"172.21.15.81:6803","nonce":3023160932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3023160932},{"type":"v1","addr":"172.21.15.81:6807","nonce":3023160932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3023160932},{"type":"v1","addr":"172.21.15.81:6805","nonce":3023160932}]},"public_addr":"172.21.15.81:6801/3023160932","cluster_addr":"172.21.15.81:6803/3023160932","heartbeat_back_addr":"172.21.15.81:6807/3023160932","heartbeat_front_addr":"172.21.15.81:6805/3023160932","state":["exists","up"]},{"osd":3,"uuid":"e8848768-c1aa-445b-9554-dc6aa4eefc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":2681016844},{"type":"v1","addr":"172.21.15.122:6809","nonce":2681016844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2681016844},{"type":"v1","addr":"172.21.15.122:6811","nonce":2681016844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2681016844},{"type":"v1","addr":"172.21.15.122:6815","nonce":2681016844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2681016844},{"type":"v1","addr":"172.21.15.122:6813","nonce":2681016844}]},"public_addr":"172.21.15.122:6809/2681016844","cluster_addr":"172.21.15.122:6811/2681016844","heartbeat_back_addr":"172.21.15.122:6815/2681016844","heartbeat_front_addr":"172.21.15.122:6813/2681016844","state":["exists","up"]},{"osd":4,"uuid":"b1d8e85d-a207-4ba6-9322-61d07812083e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":1095023540},{"type":"v1","addr":"172.21.15.122:6801","nonce":1095023540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":1095023540},{"type":"v1","addr":"172.21.15.122:6803","nonce":1095023540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":1095023540},{"type":"v1","addr":"172.21.15.122:6807","nonce":1095023540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":1095023540},{"type":"v1","addr":"172.21.15.122:6805","nonce":1095023540}]},"public_addr":"172.21.15.122:6801/1095023540","cluster_addr":"172.21.15.122:6803/1095023540","heartbeat_back_addr":"172.21.15.122:6807/1095023540","heartbeat_front_addr":"172.21.15.122:6805/1095023540","state":["exists","up"]},{"osd":5,"uuid":"3ab0450b-d8fd-4ef2-a298-08d5aa2ea32f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1974398333},{"type":"v1","addr":"172.21.15.122:6817","nonce":1974398333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1974398333},{"type":"v1","addr":"172.21.15.122:6819","nonce":1974398333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1974398333},{"type":"v1","addr":"172.21.15.122:6823","nonce":1974398333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1974398333},{"type":"v1","addr":"172.21.15.122:6821","nonce":1974398333}]},"public_addr":"172.21.15.122:6817/1974398333","cluster_addr":"172.21.15.122:6819/1974398333","heartbeat_back_addr":"172.21.15.122:6823/1974398333","heartbeat_front_addr":"172.21.15.122:6821/1974398333","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T21:40:15.766648+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T21:40:15.457496+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T21:40:14.767039+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T21:40:15.355570+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T21:40:15.335729+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T21:40:15.614733+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.8","osds":[2,3]}],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T21:43:49.972 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-21T21:43:49.972 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-21T21:43:49.973 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-21T21:43:49.973 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-21T21:43:49.973 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-21T21:43:49.974 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-21T21:43:50.126 INFO:teuthology.orchestra.run.smithi081.stdout:51539607598 2024-09-21T21:43:50.127 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-21T21:43:50.135 INFO:teuthology.orchestra.run.smithi081.stdout:55834574894 2024-09-21T21:43:50.135 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-21T21:43:50.174 INFO:teuthology.orchestra.run.smithi081.stdout:51539607598 2024-09-21T21:43:50.174 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-21T21:43:50.176 INFO:teuthology.orchestra.run.smithi081.stdout:47244640302 2024-09-21T21:43:50.176 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-21T21:43:50.182 INFO:teuthology.orchestra.run.smithi081.stdout:55834574894 2024-09-21T21:43:50.182 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-21T21:43:50.188 INFO:teuthology.orchestra.run.smithi081.stdout:51539607599 2024-09-21T21:43:50.188 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-21T21:43:50.503 INFO:teuthology.orchestra.run.smithi081.stdout:55834574894 2024-09-21T21:43:50.511 INFO:teuthology.orchestra.run.smithi081.stdout:51539607598 2024-09-21T21:43:50.515 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574894 got 55834574894 for osd.3 2024-09-21T21:43:50.515 DEBUG:teuthology.parallel:result is None 2024-09-21T21:43:50.525 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607598 got 51539607598 for osd.5 2024-09-21T21:43:50.525 DEBUG:teuthology.parallel:result is None 2024-09-21T21:43:50.600 INFO:teuthology.orchestra.run.smithi081.stdout:47244640302 2024-09-21T21:43:50.602 INFO:teuthology.orchestra.run.smithi081.stdout:51539607598 2024-09-21T21:43:50.617 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640302 got 47244640302 for osd.2 2024-09-21T21:43:50.617 DEBUG:teuthology.parallel:result is None 2024-09-21T21:43:50.618 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607598 got 51539607598 for osd.1 2024-09-21T21:43:50.618 DEBUG:teuthology.parallel:result is None 2024-09-21T21:43:50.665 INFO:teuthology.orchestra.run.smithi081.stdout:51539607599 2024-09-21T21:43:50.665 INFO:teuthology.orchestra.run.smithi081.stdout:55834574894 2024-09-21T21:43:50.679 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607599 got 51539607599 for osd.0 2024-09-21T21:43:50.679 DEBUG:teuthology.parallel:result is None 2024-09-21T21:43:50.680 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574894 got 55834574894 for osd.4 2024-09-21T21:43:50.680 DEBUG:teuthology.parallel:result is None 2024-09-21T21:43:50.680 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-21T21:43:50.680 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T21:43:50.958 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:43:50.958 INFO:teuthology.orchestra.run.smithi081.stderr:dumped all 2024-09-21T21:43:50.970 INFO:teuthology.orchestra.run.smithi081.stdout:{"pg_ready":true,"pg_map":{"version":287,"stamp":"2024-09-21T21:43:50.324770+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":245,"num_object_clones":0,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":237,"num_read":408969,"num_read_kb":1471330,"num_write":28035,"num_write_kb":322125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7832,"num_bytes_recovered":222563409,"num_keys_recovered":844,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24566,"ondisk_log_size":24566,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":366800,"kb_used_data":13072,"kb_used_omap":9,"kb_used_meta":353590,"kb_avail":565864240,"statfs":{"total":579820584960,"available":579444981760,"internally_reserved":0,"allocated":13385728,"data_stored":6760954,"data_compressed":68060,"data_compressed_allocated":90112,"data_compressed_original":1179648,"omap_allocated":9568,"internal_metadata":362076832},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":-11,"num_read_kb":-11,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":125,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000777"},"pg_stats":[{"pgid":"3.1f","version":"171'591","reported_seq":7658,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.374605+0000","last_change":"2024-09-21T21:43:33.322591+0000","last_active":"2024-09-21T21:43:49.374605+0000","last_peered":"2024-09-21T21:43:49.374605+0000","last_clean":"2024-09-21T21:43:49.374605+0000","last_became_active":"2024-09-21T21:43:08.230509+0000","last_became_peered":"2024-09-21T21:43:08.230509+0000","last_unstale":"2024-09-21T21:43:49.374605+0000","last_undegraded":"2024-09-21T21:43:49.374605+0000","last_fullsized":"2024-09-21T21:43:49.374605+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":591,"log_dups_size":0,"ondisk_log_size":591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:50:51.925851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013233089999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":8384,"num_read_kb":24764,"num_write":554,"num_write_kb":8557,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":293,"num_bytes_recovered":3779776,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"168'1013","reported_seq":15326,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-21T21:43:48.228069+0000","last_change":"2024-09-21T21:43:44.357820+0000","last_active":"2024-09-21T21:43:48.228069+0000","last_peered":"2024-09-21T21:43:48.228069+0000","last_clean":"2024-09-21T21:43:48.228069+0000","last_became_active":"2024-09-21T21:43:44.356029+0000","last_became_peered":"2024-09-21T21:43:44.356029+0000","last_unstale":"2024-09-21T21:43:48.228069+0000","last_undegraded":"2024-09-21T21:43:48.228069+0000","last_fullsized":"2024-09-21T21:43:48.228069+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":1013,"log_dups_size":0,"ondisk_log_size":1013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:20:13.278523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001190569,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":16219,"num_read_kb":73109,"num_write":1473,"num_write_kb":18401,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":14291598,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"168'985","reported_seq":10752,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.623764+0000","last_change":"2024-09-21T21:43:33.302894+0000","last_active":"2024-09-21T21:43:49.623764+0000","last_peered":"2024-09-21T21:43:49.623764+0000","last_clean":"2024-09-21T21:43:49.623764+0000","last_became_active":"2024-09-21T21:43:20.225483+0000","last_became_peered":"2024-09-21T21:43:20.225483+0000","last_unstale":"2024-09-21T21:43:49.623764+0000","last_undegraded":"2024-09-21T21:43:49.623764+0000","last_fullsized":"2024-09-21T21:43:49.623764+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":985,"log_dups_size":0,"ondisk_log_size":985,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:13:56.033873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077268300000000003,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":13110,"num_read_kb":43443,"num_write":1158,"num_write_kb":7907,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":4009384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"171'884","reported_seq":13301,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.623722+0000","last_change":"2024-09-21T21:43:33.304149+0000","last_active":"2024-09-21T21:43:49.623722+0000","last_peered":"2024-09-21T21:43:49.623722+0000","last_clean":"2024-09-21T21:43:49.623722+0000","last_became_active":"2024-09-21T21:43:19.225435+0000","last_became_peered":"2024-09-21T21:43:19.225435+0000","last_unstale":"2024-09-21T21:43:49.623722+0000","last_undegraded":"2024-09-21T21:43:49.623722+0000","last_fullsized":"2024-09-21T21:43:49.623722+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":884,"log_dups_size":0,"ondisk_log_size":884,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:45:28.185309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001856124,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":15417,"num_read_kb":35250,"num_write":925,"num_write_kb":7812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":178,"num_bytes_recovered":1721983,"num_keys_recovered":148,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"171'737","reported_seq":9064,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.374605+0000","last_change":"2024-09-21T21:43:33.322211+0000","last_active":"2024-09-21T21:43:49.374605+0000","last_peered":"2024-09-21T21:43:49.374605+0000","last_clean":"2024-09-21T21:43:49.374605+0000","last_became_active":"2024-09-21T21:43:19.225304+0000","last_became_peered":"2024-09-21T21:43:19.225304+0000","last_unstale":"2024-09-21T21:43:49.374605+0000","last_undegraded":"2024-09-21T21:43:49.374605+0000","last_fullsized":"2024-09-21T21:43:49.374605+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":737,"log_dups_size":0,"ondisk_log_size":737,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:07:04.863691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001146085,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":8621,"num_read_kb":37221,"num_write":843,"num_write_kb":4971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":547,"num_bytes_recovered":1792422,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"170'618","reported_seq":25431,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:50.091426+0000","last_change":"2024-09-21T21:43:48.232831+0000","last_active":"2024-09-21T21:43:50.091426+0000","last_peered":"2024-09-21T21:43:50.091426+0000","last_clean":"2024-09-21T21:43:50.091426+0000","last_became_active":"2024-09-21T21:43:48.230767+0000","last_became_peered":"2024-09-21T21:43:48.230767+0000","last_unstale":"2024-09-21T21:43:50.091426+0000","last_undegraded":"2024-09-21T21:43:50.091426+0000","last_fullsized":"2024-09-21T21:43:50.091426+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":618,"log_dups_size":0,"ondisk_log_size":618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:47:54.923891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0044588529999999996,"stat_sum":{"num_bytes":19,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":25138,"num_read_kb":59201,"num_write":578,"num_write_kb":7782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":203,"num_bytes_recovered":4464880,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"170'860","reported_seq":14187,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.373846+0000","last_change":"2024-09-21T21:43:33.291493+0000","last_active":"2024-09-21T21:43:49.373846+0000","last_peered":"2024-09-21T21:43:49.373846+0000","last_clean":"2024-09-21T21:43:49.373846+0000","last_became_active":"2024-09-21T21:43:20.224845+0000","last_became_peered":"2024-09-21T21:43:20.224845+0000","last_unstale":"2024-09-21T21:43:49.373846+0000","last_undegraded":"2024-09-21T21:43:49.373846+0000","last_fullsized":"2024-09-21T21:43:49.373846+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":860,"log_dups_size":0,"ondisk_log_size":860,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:55:13.229277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018111119999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":16015,"num_read_kb":61704,"num_write":858,"num_write_kb":15379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":9899814,"num_keys_recovered":59,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"168'335","reported_seq":6955,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-21T21:43:48.228040+0000","last_change":"2024-09-21T21:43:44.358159+0000","last_active":"2024-09-21T21:43:48.228040+0000","last_peered":"2024-09-21T21:43:48.228040+0000","last_clean":"2024-09-21T21:43:48.228040+0000","last_became_active":"2024-09-21T21:43:44.356203+0000","last_became_peered":"2024-09-21T21:43:44.356203+0000","last_unstale":"2024-09-21T21:43:48.228040+0000","last_undegraded":"2024-09-21T21:43:48.228040+0000","last_fullsized":"2024-09-21T21:43:48.228040+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:53:07.504981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012910630000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":6161,"num_read_kb":33023,"num_write":325,"num_write_kb":8639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":5854224,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"171'497","reported_seq":6419,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.374509+0000","last_change":"2024-09-21T21:43:33.322741+0000","last_active":"2024-09-21T21:43:49.374509+0000","last_peered":"2024-09-21T21:43:49.374509+0000","last_clean":"2024-09-21T21:43:49.374509+0000","last_became_active":"2024-09-21T21:43:07.232590+0000","last_became_peered":"2024-09-21T21:43:07.232590+0000","last_unstale":"2024-09-21T21:43:49.374509+0000","last_undegraded":"2024-09-21T21:43:49.374509+0000","last_fullsized":"2024-09-21T21:43:49.374509+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":497,"log_dups_size":0,"ondisk_log_size":497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:04:00.513414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001467229,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":6465,"num_read_kb":25955,"num_write":521,"num_write_kb":9178,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":203,"num_bytes_recovered":4149457,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"171'1238","reported_seq":9868,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.623839+0000","last_change":"2024-09-21T21:43:33.305145+0000","last_active":"2024-09-21T21:43:49.623839+0000","last_peered":"2024-09-21T21:43:49.623839+0000","last_clean":"2024-09-21T21:43:49.623839+0000","last_became_active":"2024-09-21T21:41:53.214270+0000","last_became_peered":"2024-09-21T21:41:53.214270+0000","last_unstale":"2024-09-21T21:43:49.623839+0000","last_undegraded":"2024-09-21T21:43:49.623839+0000","last_fullsized":"2024-09-21T21:43:49.623839+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":1238,"log_dups_size":0,"ondisk_log_size":1238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:51:58.701104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022124750000000002,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":13255,"num_read_kb":44399,"num_write":1796,"num_write_kb":12840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":148,"num_bytes_recovered":4869268,"num_keys_recovered":47,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"170'744","reported_seq":9490,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.371529+0000","last_change":"2024-09-21T21:43:33.293155+0000","last_active":"2024-09-21T21:43:49.371529+0000","last_peered":"2024-09-21T21:43:49.371529+0000","last_clean":"2024-09-21T21:43:49.371529+0000","last_became_active":"2024-09-21T21:43:07.228504+0000","last_became_peered":"2024-09-21T21:43:07.228504+0000","last_unstale":"2024-09-21T21:43:49.371529+0000","last_undegraded":"2024-09-21T21:43:49.371529+0000","last_fullsized":"2024-09-21T21:43:49.371529+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":744,"log_dups_size":0,"ondisk_log_size":744,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:55:34.005752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001650253,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":9201,"num_read_kb":56862,"num_write":979,"num_write_kb":7172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":520,"num_bytes_recovered":10162533,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"167'626","reported_seq":21896,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.371520+0000","last_change":"2024-09-21T21:43:48.230374+0000","last_active":"2024-09-21T21:43:49.371520+0000","last_peered":"2024-09-21T21:43:49.371520+0000","last_clean":"2024-09-21T21:43:49.371520+0000","last_became_active":"2024-09-21T21:43:48.228528+0000","last_became_peered":"2024-09-21T21:43:48.228528+0000","last_unstale":"2024-09-21T21:43:49.371520+0000","last_undegraded":"2024-09-21T21:43:49.371520+0000","last_fullsized":"2024-09-21T21:43:49.371520+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":626,"log_dups_size":0,"ondisk_log_size":626,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:52:06.587738+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.00063765000000000002,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":22083,"num_read_kb":75165,"num_write":513,"num_write_kb":10980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":602,"num_bytes_recovered":22337258,"num_keys_recovered":84,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":236,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.371668+0000","last_change":"2024-09-21T21:43:15.316851+0000","last_active":"2024-09-21T21:43:49.371668+0000","last_peered":"2024-09-21T21:43:49.371668+0000","last_clean":"2024-09-21T21:43:49.371668+0000","last_became_active":"2024-09-21T21:43:15.316533+0000","last_became_peered":"2024-09-21T21:43:15.316533+0000","last_unstale":"2024-09-21T21:43:49.371668+0000","last_undegraded":"2024-09-21T21:43:49.371668+0000","last_fullsized":"2024-09-21T21:43:49.371668+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:00:30.781100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"171'1225","reported_seq":14273,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.374367+0000","last_change":"2024-09-21T21:43:33.321874+0000","last_active":"2024-09-21T21:43:49.374367+0000","last_peered":"2024-09-21T21:43:49.374367+0000","last_clean":"2024-09-21T21:43:49.374367+0000","last_became_active":"2024-09-21T21:43:09.230849+0000","last_became_peered":"2024-09-21T21:43:09.230849+0000","last_unstale":"2024-09-21T21:43:49.374367+0000","last_undegraded":"2024-09-21T21:43:49.374367+0000","last_fullsized":"2024-09-21T21:43:49.374367+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":1225,"log_dups_size":0,"ondisk_log_size":1225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:50:50.908667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096363700000000004,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":15823,"num_read_kb":66323,"num_write":1574,"num_write_kb":15987,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":349,"num_bytes_recovered":9257588,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":362,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.374023+0000","last_change":"2024-09-21T21:41:49.266534+0000","last_active":"2024-09-21T21:43:49.374023+0000","last_peered":"2024-09-21T21:43:49.374023+0000","last_clean":"2024-09-21T21:43:49.374023+0000","last_became_active":"2024-09-21T21:41:49.266257+0000","last_became_peered":"2024-09-21T21:41:49.266257+0000","last_unstale":"2024-09-21T21:43:49.374023+0000","last_undegraded":"2024-09-21T21:43:49.374023+0000","last_fullsized":"2024-09-21T21:43:49.374023+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:38:31.063193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062280799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"170'555","reported_seq":8886,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-21T21:43:48.227944+0000","last_change":"2024-09-21T21:43:44.357970+0000","last_active":"2024-09-21T21:43:48.227944+0000","last_peered":"2024-09-21T21:43:48.227944+0000","last_clean":"2024-09-21T21:43:48.227944+0000","last_became_active":"2024-09-21T21:43:44.356543+0000","last_became_peered":"2024-09-21T21:43:44.356543+0000","last_unstale":"2024-09-21T21:43:48.227944+0000","last_undegraded":"2024-09-21T21:43:48.227944+0000","last_fullsized":"2024-09-21T21:43:48.227944+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":555,"log_dups_size":0,"ondisk_log_size":555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:10:07.726600+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.00099396100000000002,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":10770,"num_read_kb":40597,"num_write":524,"num_write_kb":10606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":241,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.371492+0000","last_change":"2024-09-21T21:43:03.224822+0000","last_active":"2024-09-21T21:43:49.371492+0000","last_peered":"2024-09-21T21:43:49.371492+0000","last_clean":"2024-09-21T21:43:49.371492+0000","last_became_active":"2024-09-21T21:43:03.224515+0000","last_became_peered":"2024-09-21T21:43:03.224515+0000","last_unstale":"2024-09-21T21:43:49.371492+0000","last_undegraded":"2024-09-21T21:43:49.371492+0000","last_fullsized":"2024-09-21T21:43:49.371492+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:08:18.537918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"170'880","reported_seq":4751,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.373958+0000","last_change":"2024-09-21T21:43:33.291832+0000","last_active":"2024-09-21T21:43:49.373958+0000","last_peered":"2024-09-21T21:43:49.373958+0000","last_clean":"2024-09-21T21:43:49.373958+0000","last_became_active":"2024-09-21T21:40:37.419944+0000","last_became_peered":"2024-09-21T21:40:37.419944+0000","last_unstale":"2024-09-21T21:43:49.373958+0000","last_undegraded":"2024-09-21T21:43:49.373958+0000","last_fullsized":"2024-09-21T21:43:49.373958+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":880,"log_dups_size":0,"ondisk_log_size":880,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:53:04.189791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020772899999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":5927,"num_read_kb":25257,"num_write":1263,"num_write_kb":11321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":367,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.374070+0000","last_change":"2024-09-21T21:43:03.224909+0000","last_active":"2024-09-21T21:43:49.374070+0000","last_peered":"2024-09-21T21:43:49.374070+0000","last_clean":"2024-09-21T21:43:49.374070+0000","last_became_active":"2024-09-21T21:43:03.224572+0000","last_became_peered":"2024-09-21T21:43:03.224572+0000","last_unstale":"2024-09-21T21:43:49.374070+0000","last_undegraded":"2024-09-21T21:43:49.374070+0000","last_fullsized":"2024-09-21T21:43:49.374070+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:10:47.109676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070669899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"168'637","reported_seq":10497,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.623794+0000","last_change":"2024-09-21T21:43:33.303937+0000","last_active":"2024-09-21T21:43:49.623794+0000","last_peered":"2024-09-21T21:43:49.623794+0000","last_clean":"2024-09-21T21:43:49.623794+0000","last_became_active":"2024-09-21T21:41:25.650379+0000","last_became_peered":"2024-09-21T21:41:25.650379+0000","last_unstale":"2024-09-21T21:43:49.623794+0000","last_undegraded":"2024-09-21T21:43:49.623794+0000","last_fullsized":"2024-09-21T21:43:49.623794+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":637,"log_dups_size":0,"ondisk_log_size":637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:48:59.774761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001695787,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":11334,"num_read_kb":32091,"num_write":515,"num_write_kb":8339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"171'646","reported_seq":11168,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:50.091476+0000","last_change":"2024-09-21T21:43:48.233737+0000","last_active":"2024-09-21T21:43:50.091476+0000","last_peered":"2024-09-21T21:43:50.091476+0000","last_clean":"2024-09-21T21:43:50.091476+0000","last_became_active":"2024-09-21T21:43:48.231457+0000","last_became_peered":"2024-09-21T21:43:48.231457+0000","last_unstale":"2024-09-21T21:43:50.091476+0000","last_undegraded":"2024-09-21T21:43:50.091476+0000","last_fullsized":"2024-09-21T21:43:50.091476+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:10:37.262673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081362800000000005,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":9832,"num_read_kb":35952,"num_write":639,"num_write_kb":9820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":437,"num_bytes_recovered":19367440,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":226,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.623813+0000","last_change":"2024-09-21T21:42:24.516929+0000","last_active":"2024-09-21T21:43:49.623813+0000","last_peered":"2024-09-21T21:43:49.623813+0000","last_clean":"2024-09-21T21:43:49.623813+0000","last_became_active":"2024-09-21T21:42:24.516623+0000","last_became_peered":"2024-09-21T21:42:24.516623+0000","last_unstale":"2024-09-21T21:43:49.623813+0000","last_undegraded":"2024-09-21T21:43:49.623813+0000","last_fullsized":"2024-09-21T21:43:49.623813+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:51:59.592292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000906268,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"171'815","reported_seq":11441,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.373829+0000","last_change":"2024-09-21T21:43:33.290813+0000","last_active":"2024-09-21T21:43:49.373829+0000","last_peered":"2024-09-21T21:43:49.373829+0000","last_clean":"2024-09-21T21:43:49.373829+0000","last_became_active":"2024-09-21T21:40:37.418973+0000","last_became_peered":"2024-09-21T21:40:37.418973+0000","last_unstale":"2024-09-21T21:43:49.373829+0000","last_undegraded":"2024-09-21T21:43:49.373829+0000","last_fullsized":"2024-09-21T21:43:49.373829+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":815,"log_dups_size":0,"ondisk_log_size":815,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:31:26.860045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001131546,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":13425,"num_read_kb":28144,"num_write":602,"num_write_kb":6293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":105,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-21T21:43:48.228311+0000","last_change":"2024-09-21T21:43:44.357753+0000","last_active":"2024-09-21T21:43:48.228311+0000","last_peered":"2024-09-21T21:43:48.228311+0000","last_clean":"2024-09-21T21:43:48.228311+0000","last_became_active":"2024-09-21T21:43:44.357116+0000","last_became_peered":"2024-09-21T21:43:44.357116+0000","last_unstale":"2024-09-21T21:43:48.228311+0000","last_undegraded":"2024-09-21T21:43:48.228311+0000","last_fullsized":"2024-09-21T21:43:48.228311+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:22:20.440763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"168'387","reported_seq":16883,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.373980+0000","last_change":"2024-09-21T21:43:33.321435+0000","last_active":"2024-09-21T21:43:49.373980+0000","last_peered":"2024-09-21T21:43:49.373980+0000","last_clean":"2024-09-21T21:43:49.373980+0000","last_became_active":"2024-09-21T21:43:20.224689+0000","last_became_peered":"2024-09-21T21:43:20.224689+0000","last_unstale":"2024-09-21T21:43:49.373980+0000","last_undegraded":"2024-09-21T21:43:49.373980+0000","last_fullsized":"2024-09-21T21:43:49.373980+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":387,"log_dups_size":0,"ondisk_log_size":387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:15:32.359340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061419100000000004,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":15397,"num_read_kb":48505,"num_write":377,"num_write_kb":4859,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":429,"num_bytes_recovered":6598898,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":227,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.371346+0000","last_change":"2024-09-21T21:41:49.265465+0000","last_active":"2024-09-21T21:43:49.371346+0000","last_peered":"2024-09-21T21:43:49.371346+0000","last_clean":"2024-09-21T21:43:49.371346+0000","last_became_active":"2024-09-21T21:41:49.265182+0000","last_became_peered":"2024-09-21T21:41:49.265182+0000","last_unstale":"2024-09-21T21:43:49.371346+0000","last_undegraded":"2024-09-21T21:43:49.371346+0000","last_fullsized":"2024-09-21T21:43:49.371346+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:19:17.973195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"171'773","reported_seq":10302,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.374490+0000","last_change":"2024-09-21T21:43:33.293418+0000","last_active":"2024-09-21T21:43:49.374490+0000","last_peered":"2024-09-21T21:43:49.374490+0000","last_clean":"2024-09-21T21:43:49.374490+0000","last_became_active":"2024-09-21T21:40:37.421655+0000","last_became_peered":"2024-09-21T21:40:37.421655+0000","last_unstale":"2024-09-21T21:43:49.374490+0000","last_undegraded":"2024-09-21T21:43:49.374490+0000","last_fullsized":"2024-09-21T21:43:49.374490+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":773,"log_dups_size":0,"ondisk_log_size":773,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:43:34.040974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031354970000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":12713,"num_read_kb":34378,"num_write":754,"num_write_kb":7498,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":324,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.623702+0000","last_change":"2024-09-21T21:42:24.518810+0000","last_active":"2024-09-21T21:43:49.623702+0000","last_peered":"2024-09-21T21:43:49.623702+0000","last_clean":"2024-09-21T21:43:49.623702+0000","last_became_active":"2024-09-21T21:42:24.518461+0000","last_became_peered":"2024-09-21T21:42:24.518461+0000","last_unstale":"2024-09-21T21:43:49.623702+0000","last_undegraded":"2024-09-21T21:43:49.623702+0000","last_fullsized":"2024-09-21T21:43:49.623702+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:17:47.593497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036803999999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":433,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.371463+0000","last_change":"2024-09-21T21:43:06.231540+0000","last_active":"2024-09-21T21:43:49.371463+0000","last_peered":"2024-09-21T21:43:49.371463+0000","last_clean":"2024-09-21T21:43:49.371463+0000","last_became_active":"2024-09-21T21:43:06.231241+0000","last_became_peered":"2024-09-21T21:43:06.231241+0000","last_unstale":"2024-09-21T21:43:49.371463+0000","last_undegraded":"2024-09-21T21:43:49.371463+0000","last_fullsized":"2024-09-21T21:43:49.371463+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:18.340430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:18.340430+0000","last_clean_scrub_stamp":"2024-09-21T21:40:18.340430+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:50:08.187886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"170'374","reported_seq":6270,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-21T21:43:48.228114+0000","last_change":"2024-09-21T21:43:44.358111+0000","last_active":"2024-09-21T21:43:48.228114+0000","last_peered":"2024-09-21T21:43:48.228114+0000","last_clean":"2024-09-21T21:43:48.228114+0000","last_became_active":"2024-09-21T21:43:44.356024+0000","last_became_peered":"2024-09-21T21:43:44.356024+0000","last_unstale":"2024-09-21T21:43:48.228114+0000","last_undegraded":"2024-09-21T21:43:48.228114+0000","last_fullsized":"2024-09-21T21:43:48.228114+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":374,"log_dups_size":0,"ondisk_log_size":374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:17:43.127988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094280000000000004,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":5268,"num_read_kb":25972,"num_write":367,"num_write_kb":7278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":172,"num_bytes_recovered":2229995,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"170'554","reported_seq":8626,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:50.091421+0000","last_change":"2024-09-21T21:43:48.233080+0000","last_active":"2024-09-21T21:43:50.091421+0000","last_peered":"2024-09-21T21:43:50.091421+0000","last_clean":"2024-09-21T21:43:50.091421+0000","last_became_active":"2024-09-21T21:43:48.230703+0000","last_became_peered":"2024-09-21T21:43:48.230703+0000","last_unstale":"2024-09-21T21:43:50.091421+0000","last_undegraded":"2024-09-21T21:43:50.091421+0000","last_fullsized":"2024-09-21T21:43:50.091421+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":554,"log_dups_size":0,"ondisk_log_size":554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:39:12.422453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071450699999999997,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":8813,"num_read_kb":47394,"num_write":525,"num_write_kb":9434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":241,"num_bytes_recovered":7579476,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"171'1111","reported_seq":11670,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.371409+0000","last_change":"2024-09-21T21:43:33.292652+0000","last_active":"2024-09-21T21:43:49.371409+0000","last_peered":"2024-09-21T21:43:49.371409+0000","last_clean":"2024-09-21T21:43:49.371409+0000","last_became_active":"2024-09-21T21:43:19.224644+0000","last_became_peered":"2024-09-21T21:43:19.224644+0000","last_unstale":"2024-09-21T21:43:49.371409+0000","last_undegraded":"2024-09-21T21:43:49.371409+0000","last_fullsized":"2024-09-21T21:43:49.371409+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":1111,"log_dups_size":0,"ondisk_log_size":1111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:19:31.244395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001226878,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":5,"num_read":13231,"num_read_kb":49174,"num_write":1280,"num_write_kb":10665,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":398,"num_bytes_recovered":16632560,"num_keys_recovered":144,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"170'777","reported_seq":10308,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.374283+0000","last_change":"2024-09-21T21:43:33.292265+0000","last_active":"2024-09-21T21:43:49.374283+0000","last_peered":"2024-09-21T21:43:49.374283+0000","last_clean":"2024-09-21T21:43:49.374283+0000","last_became_active":"2024-09-21T21:43:09.227569+0000","last_became_peered":"2024-09-21T21:43:09.227569+0000","last_unstale":"2024-09-21T21:43:49.374283+0000","last_undegraded":"2024-09-21T21:43:49.374283+0000","last_fullsized":"2024-09-21T21:43:49.374283+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":777,"log_dups_size":0,"ondisk_log_size":777,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:20:34.944827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024712610000000002,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":10244,"num_read_kb":45368,"num_write":1134,"num_write_kb":14646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":331,"num_bytes_recovered":8931668,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"168'1625","reported_seq":14031,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.623830+0000","last_change":"2024-09-21T21:43:33.304566+0000","last_active":"2024-09-21T21:43:49.623830+0000","last_peered":"2024-09-21T21:43:49.623830+0000","last_clean":"2024-09-21T21:43:49.623830+0000","last_became_active":"2024-09-21T21:43:07.230148+0000","last_became_peered":"2024-09-21T21:43:07.230148+0000","last_unstale":"2024-09-21T21:43:49.623830+0000","last_undegraded":"2024-09-21T21:43:49.623830+0000","last_fullsized":"2024-09-21T21:43:49.623830+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":1625,"log_dups_size":0,"ondisk_log_size":1625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:42:46.324537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021897029999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":18573,"num_read_kb":61318,"num_write":2350,"num_write_kb":11538,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":225,"num_bytes_recovered":7091105,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"171'1447","reported_seq":17761,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.371616+0000","last_change":"2024-09-21T21:43:33.293137+0000","last_active":"2024-09-21T21:43:49.371616+0000","last_peered":"2024-09-21T21:43:49.371616+0000","last_clean":"2024-09-21T21:43:49.371616+0000","last_became_active":"2024-09-21T21:41:52.214022+0000","last_became_peered":"2024-09-21T21:41:52.214022+0000","last_unstale":"2024-09-21T21:43:49.371616+0000","last_undegraded":"2024-09-21T21:43:49.371616+0000","last_fullsized":"2024-09-21T21:43:49.371616+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":1447,"log_dups_size":0,"ondisk_log_size":1447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:42:14.727563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016156199999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":21216,"num_read_kb":45587,"num_write":1846,"num_write_kb":14797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":149,"num_bytes_recovered":3462638,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"168'454","reported_seq":7909,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.623841+0000","last_change":"2024-09-21T21:43:33.304631+0000","last_active":"2024-09-21T21:43:49.623841+0000","last_peered":"2024-09-21T21:43:49.623841+0000","last_clean":"2024-09-21T21:43:49.623841+0000","last_became_active":"2024-09-21T21:43:18.223755+0000","last_became_peered":"2024-09-21T21:43:18.223755+0000","last_unstale":"2024-09-21T21:43:49.623841+0000","last_undegraded":"2024-09-21T21:43:49.623841+0000","last_fullsized":"2024-09-21T21:43:49.623841+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":454,"log_dups_size":0,"ondisk_log_size":454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:43:32.324890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018835950000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":6571,"num_read_kb":34865,"num_write":469,"num_write_kb":6862,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":238,"num_bytes_recovered":4045989,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"169'741","reported_seq":17136,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.623910+0000","last_change":"2024-09-21T21:43:33.303579+0000","last_active":"2024-09-21T21:43:49.623910+0000","last_peered":"2024-09-21T21:43:49.623910+0000","last_clean":"2024-09-21T21:43:49.623910+0000","last_became_active":"2024-09-21T21:41:52.212206+0000","last_became_peered":"2024-09-21T21:41:52.212206+0000","last_unstale":"2024-09-21T21:43:49.623910+0000","last_undegraded":"2024-09-21T21:43:49.623910+0000","last_fullsized":"2024-09-21T21:43:49.623910+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":741,"log_dups_size":0,"ondisk_log_size":741,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:10:57.220429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013653929999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":18315,"num_read_kb":42344,"num_write":672,"num_write_kb":9505,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":2967886,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"167'408","reported_seq":14252,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.623662+0000","last_change":"2024-09-21T21:43:33.302904+0000","last_active":"2024-09-21T21:43:49.623662+0000","last_peered":"2024-09-21T21:43:49.623662+0000","last_clean":"2024-09-21T21:43:49.623662+0000","last_became_active":"2024-09-21T21:43:06.233222+0000","last_became_peered":"2024-09-21T21:43:06.233222+0000","last_unstale":"2024-09-21T21:43:49.623662+0000","last_undegraded":"2024-09-21T21:43:49.623662+0000","last_fullsized":"2024-09-21T21:43:49.623662+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:54:36.083839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085383799999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":12974,"num_read_kb":55237,"num_write":424,"num_write_kb":11231,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":205,"num_bytes_recovered":6455839,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"168'641","reported_seq":8507,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.371363+0000","last_change":"2024-09-21T21:43:33.292136+0000","last_active":"2024-09-21T21:43:49.371363+0000","last_peered":"2024-09-21T21:43:49.371363+0000","last_clean":"2024-09-21T21:43:49.371363+0000","last_became_active":"2024-09-21T21:43:20.224840+0000","last_became_peered":"2024-09-21T21:43:20.224840+0000","last_unstale":"2024-09-21T21:43:49.371363+0000","last_undegraded":"2024-09-21T21:43:49.371363+0000","last_fullsized":"2024-09-21T21:43:49.371363+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":641,"log_dups_size":0,"ondisk_log_size":641,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:13:32.197836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091468500000000004,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":7916,"num_read_kb":33911,"num_write":730,"num_write_kb":6868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":492,"num_bytes_recovered":3936953,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"170'791","reported_seq":19355,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.374375+0000","last_change":"2024-09-21T21:43:33.321430+0000","last_active":"2024-09-21T21:43:49.374375+0000","last_peered":"2024-09-21T21:43:49.374375+0000","last_clean":"2024-09-21T21:43:49.374375+0000","last_became_active":"2024-09-21T21:43:07.229497+0000","last_became_peered":"2024-09-21T21:43:07.229497+0000","last_unstale":"2024-09-21T21:43:49.374375+0000","last_undegraded":"2024-09-21T21:43:49.374375+0000","last_fullsized":"2024-09-21T21:43:49.374375+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":791,"log_dups_size":0,"ondisk_log_size":791,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:10:05.347980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061499099999999995,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":20795,"num_read_kb":71790,"num_write":798,"num_write_kb":9984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":352,"num_bytes_recovered":16923683,"num_keys_recovered":92,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"168'477","reported_seq":10652,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.371417+0000","last_change":"2024-09-21T21:43:33.292617+0000","last_active":"2024-09-21T21:43:49.371417+0000","last_peered":"2024-09-21T21:43:49.371417+0000","last_clean":"2024-09-21T21:43:49.371417+0000","last_became_active":"2024-09-21T21:41:52.209710+0000","last_became_peered":"2024-09-21T21:41:52.209710+0000","last_unstale":"2024-09-21T21:43:49.371417+0000","last_undegraded":"2024-09-21T21:43:49.371417+0000","last_fullsized":"2024-09-21T21:43:49.371417+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":477,"log_dups_size":0,"ondisk_log_size":477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:20:59.585304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013513189999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":9647,"num_read_kb":76927,"num_write":508,"num_write_kb":12558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":146,"num_bytes_recovered":17387622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":241,"num_object_clones":0,"num_object_copies":482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":241,"num_whiteouts":237,"num_read":408853,"num_read_kb":1471230,"num_write":27874,"num_write_kb":319707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7824,"num_bytes_recovered":220201937,"num_keys_recovered":844,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24446,"ondisk_log_size":24446,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":102090,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":4,"up_from":13,"seq":55834574894,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55556,"kb_used_data":2160,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316284,"statfs":{"total":96636764160,"available":96579874816,"internally_reserved":0,"allocated":2211840,"data_stored":928386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574894,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87536,"kb_used_data":2204,"kb_used_omap":1,"kb_used_meta":85310,"kb_avail":94284304,"statfs":{"total":96636764160,"available":96547127296,"internally_reserved":0,"allocated":2256896,"data_stored":1518735,"data_compressed":34030,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1593,"internal_metadata":87357895},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":12,"seq":51539607598,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73604,"kb_used_data":2160,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298236,"statfs":{"total":96636764160,"available":96561393664,"internally_reserved":0,"allocated":2211840,"data_stored":928386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607598,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55576,"kb_used_data":2176,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316264,"statfs":{"total":96636764160,"available":96579854336,"internally_reserved":0,"allocated":2228224,"data_stored":933375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607599,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47296,"kb_used_data":2216,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324544,"statfs":{"total":96636764160,"available":96588333056,"internally_reserved":0,"allocated":2269184,"data_stored":1523705,"data_compressed":34030,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640302,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47232,"kb_used_data":2156,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324608,"statfs":{"total":96636764160,"available":96588398592,"internally_reserved":0,"allocated":2207744,"data_stored":928367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34030,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34030,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34030,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T21:43:50.972 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T21:43:51.248 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:43:51.249 INFO:teuthology.orchestra.run.smithi081.stderr:dumped all 2024-09-21T21:43:51.260 INFO:teuthology.orchestra.run.smithi081.stdout:{"pg_ready":true,"pg_map":{"version":287,"stamp":"2024-09-21T21:43:50.324770+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":245,"num_object_clones":0,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":237,"num_read":408969,"num_read_kb":1471330,"num_write":28035,"num_write_kb":322125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7832,"num_bytes_recovered":222563409,"num_keys_recovered":844,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24566,"ondisk_log_size":24566,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":366800,"kb_used_data":13072,"kb_used_omap":9,"kb_used_meta":353590,"kb_avail":565864240,"statfs":{"total":579820584960,"available":579444981760,"internally_reserved":0,"allocated":13385728,"data_stored":6760954,"data_compressed":68060,"data_compressed_allocated":90112,"data_compressed_original":1179648,"omap_allocated":9568,"internal_metadata":362076832},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":-11,"num_read_kb":-11,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":125,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000777"},"pg_stats":[{"pgid":"3.1f","version":"171'591","reported_seq":7658,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.374605+0000","last_change":"2024-09-21T21:43:33.322591+0000","last_active":"2024-09-21T21:43:49.374605+0000","last_peered":"2024-09-21T21:43:49.374605+0000","last_clean":"2024-09-21T21:43:49.374605+0000","last_became_active":"2024-09-21T21:43:08.230509+0000","last_became_peered":"2024-09-21T21:43:08.230509+0000","last_unstale":"2024-09-21T21:43:49.374605+0000","last_undegraded":"2024-09-21T21:43:49.374605+0000","last_fullsized":"2024-09-21T21:43:49.374605+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":591,"log_dups_size":0,"ondisk_log_size":591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:50:51.925851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013233089999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":8384,"num_read_kb":24764,"num_write":554,"num_write_kb":8557,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":293,"num_bytes_recovered":3779776,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"168'1013","reported_seq":15326,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-21T21:43:48.228069+0000","last_change":"2024-09-21T21:43:44.357820+0000","last_active":"2024-09-21T21:43:48.228069+0000","last_peered":"2024-09-21T21:43:48.228069+0000","last_clean":"2024-09-21T21:43:48.228069+0000","last_became_active":"2024-09-21T21:43:44.356029+0000","last_became_peered":"2024-09-21T21:43:44.356029+0000","last_unstale":"2024-09-21T21:43:48.228069+0000","last_undegraded":"2024-09-21T21:43:48.228069+0000","last_fullsized":"2024-09-21T21:43:48.228069+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":1013,"log_dups_size":0,"ondisk_log_size":1013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:20:13.278523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001190569,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":16219,"num_read_kb":73109,"num_write":1473,"num_write_kb":18401,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":14291598,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"168'985","reported_seq":10752,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.623764+0000","last_change":"2024-09-21T21:43:33.302894+0000","last_active":"2024-09-21T21:43:49.623764+0000","last_peered":"2024-09-21T21:43:49.623764+0000","last_clean":"2024-09-21T21:43:49.623764+0000","last_became_active":"2024-09-21T21:43:20.225483+0000","last_became_peered":"2024-09-21T21:43:20.225483+0000","last_unstale":"2024-09-21T21:43:49.623764+0000","last_undegraded":"2024-09-21T21:43:49.623764+0000","last_fullsized":"2024-09-21T21:43:49.623764+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":985,"log_dups_size":0,"ondisk_log_size":985,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:13:56.033873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077268300000000003,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":13110,"num_read_kb":43443,"num_write":1158,"num_write_kb":7907,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":4009384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"171'884","reported_seq":13301,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.623722+0000","last_change":"2024-09-21T21:43:33.304149+0000","last_active":"2024-09-21T21:43:49.623722+0000","last_peered":"2024-09-21T21:43:49.623722+0000","last_clean":"2024-09-21T21:43:49.623722+0000","last_became_active":"2024-09-21T21:43:19.225435+0000","last_became_peered":"2024-09-21T21:43:19.225435+0000","last_unstale":"2024-09-21T21:43:49.623722+0000","last_undegraded":"2024-09-21T21:43:49.623722+0000","last_fullsized":"2024-09-21T21:43:49.623722+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":884,"log_dups_size":0,"ondisk_log_size":884,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:45:28.185309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001856124,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":15417,"num_read_kb":35250,"num_write":925,"num_write_kb":7812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":178,"num_bytes_recovered":1721983,"num_keys_recovered":148,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"171'737","reported_seq":9064,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.374605+0000","last_change":"2024-09-21T21:43:33.322211+0000","last_active":"2024-09-21T21:43:49.374605+0000","last_peered":"2024-09-21T21:43:49.374605+0000","last_clean":"2024-09-21T21:43:49.374605+0000","last_became_active":"2024-09-21T21:43:19.225304+0000","last_became_peered":"2024-09-21T21:43:19.225304+0000","last_unstale":"2024-09-21T21:43:49.374605+0000","last_undegraded":"2024-09-21T21:43:49.374605+0000","last_fullsized":"2024-09-21T21:43:49.374605+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":737,"log_dups_size":0,"ondisk_log_size":737,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:07:04.863691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001146085,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":8621,"num_read_kb":37221,"num_write":843,"num_write_kb":4971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":547,"num_bytes_recovered":1792422,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"170'618","reported_seq":25431,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:50.091426+0000","last_change":"2024-09-21T21:43:48.232831+0000","last_active":"2024-09-21T21:43:50.091426+0000","last_peered":"2024-09-21T21:43:50.091426+0000","last_clean":"2024-09-21T21:43:50.091426+0000","last_became_active":"2024-09-21T21:43:48.230767+0000","last_became_peered":"2024-09-21T21:43:48.230767+0000","last_unstale":"2024-09-21T21:43:50.091426+0000","last_undegraded":"2024-09-21T21:43:50.091426+0000","last_fullsized":"2024-09-21T21:43:50.091426+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":618,"log_dups_size":0,"ondisk_log_size":618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:47:54.923891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0044588529999999996,"stat_sum":{"num_bytes":19,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":25138,"num_read_kb":59201,"num_write":578,"num_write_kb":7782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":203,"num_bytes_recovered":4464880,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"170'860","reported_seq":14187,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.373846+0000","last_change":"2024-09-21T21:43:33.291493+0000","last_active":"2024-09-21T21:43:49.373846+0000","last_peered":"2024-09-21T21:43:49.373846+0000","last_clean":"2024-09-21T21:43:49.373846+0000","last_became_active":"2024-09-21T21:43:20.224845+0000","last_became_peered":"2024-09-21T21:43:20.224845+0000","last_unstale":"2024-09-21T21:43:49.373846+0000","last_undegraded":"2024-09-21T21:43:49.373846+0000","last_fullsized":"2024-09-21T21:43:49.373846+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":860,"log_dups_size":0,"ondisk_log_size":860,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:55:13.229277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018111119999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":16015,"num_read_kb":61704,"num_write":858,"num_write_kb":15379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":9899814,"num_keys_recovered":59,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"168'335","reported_seq":6955,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-21T21:43:48.228040+0000","last_change":"2024-09-21T21:43:44.358159+0000","last_active":"2024-09-21T21:43:48.228040+0000","last_peered":"2024-09-21T21:43:48.228040+0000","last_clean":"2024-09-21T21:43:48.228040+0000","last_became_active":"2024-09-21T21:43:44.356203+0000","last_became_peered":"2024-09-21T21:43:44.356203+0000","last_unstale":"2024-09-21T21:43:48.228040+0000","last_undegraded":"2024-09-21T21:43:48.228040+0000","last_fullsized":"2024-09-21T21:43:48.228040+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:53:07.504981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012910630000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":6161,"num_read_kb":33023,"num_write":325,"num_write_kb":8639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":5854224,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"171'497","reported_seq":6419,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.374509+0000","last_change":"2024-09-21T21:43:33.322741+0000","last_active":"2024-09-21T21:43:49.374509+0000","last_peered":"2024-09-21T21:43:49.374509+0000","last_clean":"2024-09-21T21:43:49.374509+0000","last_became_active":"2024-09-21T21:43:07.232590+0000","last_became_peered":"2024-09-21T21:43:07.232590+0000","last_unstale":"2024-09-21T21:43:49.374509+0000","last_undegraded":"2024-09-21T21:43:49.374509+0000","last_fullsized":"2024-09-21T21:43:49.374509+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":497,"log_dups_size":0,"ondisk_log_size":497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:04:00.513414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001467229,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":6465,"num_read_kb":25955,"num_write":521,"num_write_kb":9178,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":203,"num_bytes_recovered":4149457,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"171'1238","reported_seq":9868,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.623839+0000","last_change":"2024-09-21T21:43:33.305145+0000","last_active":"2024-09-21T21:43:49.623839+0000","last_peered":"2024-09-21T21:43:49.623839+0000","last_clean":"2024-09-21T21:43:49.623839+0000","last_became_active":"2024-09-21T21:41:53.214270+0000","last_became_peered":"2024-09-21T21:41:53.214270+0000","last_unstale":"2024-09-21T21:43:49.623839+0000","last_undegraded":"2024-09-21T21:43:49.623839+0000","last_fullsized":"2024-09-21T21:43:49.623839+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":1238,"log_dups_size":0,"ondisk_log_size":1238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:51:58.701104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022124750000000002,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":13255,"num_read_kb":44399,"num_write":1796,"num_write_kb":12840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":148,"num_bytes_recovered":4869268,"num_keys_recovered":47,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"170'744","reported_seq":9490,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.371529+0000","last_change":"2024-09-21T21:43:33.293155+0000","last_active":"2024-09-21T21:43:49.371529+0000","last_peered":"2024-09-21T21:43:49.371529+0000","last_clean":"2024-09-21T21:43:49.371529+0000","last_became_active":"2024-09-21T21:43:07.228504+0000","last_became_peered":"2024-09-21T21:43:07.228504+0000","last_unstale":"2024-09-21T21:43:49.371529+0000","last_undegraded":"2024-09-21T21:43:49.371529+0000","last_fullsized":"2024-09-21T21:43:49.371529+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":744,"log_dups_size":0,"ondisk_log_size":744,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:55:34.005752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001650253,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":9201,"num_read_kb":56862,"num_write":979,"num_write_kb":7172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":520,"num_bytes_recovered":10162533,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"167'626","reported_seq":21896,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.371520+0000","last_change":"2024-09-21T21:43:48.230374+0000","last_active":"2024-09-21T21:43:49.371520+0000","last_peered":"2024-09-21T21:43:49.371520+0000","last_clean":"2024-09-21T21:43:49.371520+0000","last_became_active":"2024-09-21T21:43:48.228528+0000","last_became_peered":"2024-09-21T21:43:48.228528+0000","last_unstale":"2024-09-21T21:43:49.371520+0000","last_undegraded":"2024-09-21T21:43:49.371520+0000","last_fullsized":"2024-09-21T21:43:49.371520+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":626,"log_dups_size":0,"ondisk_log_size":626,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:52:06.587738+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.00063765000000000002,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":22083,"num_read_kb":75165,"num_write":513,"num_write_kb":10980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":602,"num_bytes_recovered":22337258,"num_keys_recovered":84,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":236,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.371668+0000","last_change":"2024-09-21T21:43:15.316851+0000","last_active":"2024-09-21T21:43:49.371668+0000","last_peered":"2024-09-21T21:43:49.371668+0000","last_clean":"2024-09-21T21:43:49.371668+0000","last_became_active":"2024-09-21T21:43:15.316533+0000","last_became_peered":"2024-09-21T21:43:15.316533+0000","last_unstale":"2024-09-21T21:43:49.371668+0000","last_undegraded":"2024-09-21T21:43:49.371668+0000","last_fullsized":"2024-09-21T21:43:49.371668+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:00:30.781100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"171'1225","reported_seq":14273,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.374367+0000","last_change":"2024-09-21T21:43:33.321874+0000","last_active":"2024-09-21T21:43:49.374367+0000","last_peered":"2024-09-21T21:43:49.374367+0000","last_clean":"2024-09-21T21:43:49.374367+0000","last_became_active":"2024-09-21T21:43:09.230849+0000","last_became_peered":"2024-09-21T21:43:09.230849+0000","last_unstale":"2024-09-21T21:43:49.374367+0000","last_undegraded":"2024-09-21T21:43:49.374367+0000","last_fullsized":"2024-09-21T21:43:49.374367+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":1225,"log_dups_size":0,"ondisk_log_size":1225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:50:50.908667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096363700000000004,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":15823,"num_read_kb":66323,"num_write":1574,"num_write_kb":15987,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":349,"num_bytes_recovered":9257588,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":362,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.374023+0000","last_change":"2024-09-21T21:41:49.266534+0000","last_active":"2024-09-21T21:43:49.374023+0000","last_peered":"2024-09-21T21:43:49.374023+0000","last_clean":"2024-09-21T21:43:49.374023+0000","last_became_active":"2024-09-21T21:41:49.266257+0000","last_became_peered":"2024-09-21T21:41:49.266257+0000","last_unstale":"2024-09-21T21:43:49.374023+0000","last_undegraded":"2024-09-21T21:43:49.374023+0000","last_fullsized":"2024-09-21T21:43:49.374023+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:38:31.063193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062280799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"170'555","reported_seq":8886,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-21T21:43:48.227944+0000","last_change":"2024-09-21T21:43:44.357970+0000","last_active":"2024-09-21T21:43:48.227944+0000","last_peered":"2024-09-21T21:43:48.227944+0000","last_clean":"2024-09-21T21:43:48.227944+0000","last_became_active":"2024-09-21T21:43:44.356543+0000","last_became_peered":"2024-09-21T21:43:44.356543+0000","last_unstale":"2024-09-21T21:43:48.227944+0000","last_undegraded":"2024-09-21T21:43:48.227944+0000","last_fullsized":"2024-09-21T21:43:48.227944+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":555,"log_dups_size":0,"ondisk_log_size":555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:10:07.726600+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.00099396100000000002,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":10770,"num_read_kb":40597,"num_write":524,"num_write_kb":10606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":241,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.371492+0000","last_change":"2024-09-21T21:43:03.224822+0000","last_active":"2024-09-21T21:43:49.371492+0000","last_peered":"2024-09-21T21:43:49.371492+0000","last_clean":"2024-09-21T21:43:49.371492+0000","last_became_active":"2024-09-21T21:43:03.224515+0000","last_became_peered":"2024-09-21T21:43:03.224515+0000","last_unstale":"2024-09-21T21:43:49.371492+0000","last_undegraded":"2024-09-21T21:43:49.371492+0000","last_fullsized":"2024-09-21T21:43:49.371492+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:08:18.537918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"170'880","reported_seq":4751,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.373958+0000","last_change":"2024-09-21T21:43:33.291832+0000","last_active":"2024-09-21T21:43:49.373958+0000","last_peered":"2024-09-21T21:43:49.373958+0000","last_clean":"2024-09-21T21:43:49.373958+0000","last_became_active":"2024-09-21T21:40:37.419944+0000","last_became_peered":"2024-09-21T21:40:37.419944+0000","last_unstale":"2024-09-21T21:43:49.373958+0000","last_undegraded":"2024-09-21T21:43:49.373958+0000","last_fullsized":"2024-09-21T21:43:49.373958+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":880,"log_dups_size":0,"ondisk_log_size":880,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:53:04.189791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020772899999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":5927,"num_read_kb":25257,"num_write":1263,"num_write_kb":11321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":367,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.374070+0000","last_change":"2024-09-21T21:43:03.224909+0000","last_active":"2024-09-21T21:43:49.374070+0000","last_peered":"2024-09-21T21:43:49.374070+0000","last_clean":"2024-09-21T21:43:49.374070+0000","last_became_active":"2024-09-21T21:43:03.224572+0000","last_became_peered":"2024-09-21T21:43:03.224572+0000","last_unstale":"2024-09-21T21:43:49.374070+0000","last_undegraded":"2024-09-21T21:43:49.374070+0000","last_fullsized":"2024-09-21T21:43:49.374070+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:10:47.109676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070669899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"168'637","reported_seq":10497,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.623794+0000","last_change":"2024-09-21T21:43:33.303937+0000","last_active":"2024-09-21T21:43:49.623794+0000","last_peered":"2024-09-21T21:43:49.623794+0000","last_clean":"2024-09-21T21:43:49.623794+0000","last_became_active":"2024-09-21T21:41:25.650379+0000","last_became_peered":"2024-09-21T21:41:25.650379+0000","last_unstale":"2024-09-21T21:43:49.623794+0000","last_undegraded":"2024-09-21T21:43:49.623794+0000","last_fullsized":"2024-09-21T21:43:49.623794+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":637,"log_dups_size":0,"ondisk_log_size":637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:48:59.774761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001695787,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":11334,"num_read_kb":32091,"num_write":515,"num_write_kb":8339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"171'646","reported_seq":11168,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:50.091476+0000","last_change":"2024-09-21T21:43:48.233737+0000","last_active":"2024-09-21T21:43:50.091476+0000","last_peered":"2024-09-21T21:43:50.091476+0000","last_clean":"2024-09-21T21:43:50.091476+0000","last_became_active":"2024-09-21T21:43:48.231457+0000","last_became_peered":"2024-09-21T21:43:48.231457+0000","last_unstale":"2024-09-21T21:43:50.091476+0000","last_undegraded":"2024-09-21T21:43:50.091476+0000","last_fullsized":"2024-09-21T21:43:50.091476+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:10:37.262673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081362800000000005,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":9832,"num_read_kb":35952,"num_write":639,"num_write_kb":9820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":437,"num_bytes_recovered":19367440,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":226,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.623813+0000","last_change":"2024-09-21T21:42:24.516929+0000","last_active":"2024-09-21T21:43:49.623813+0000","last_peered":"2024-09-21T21:43:49.623813+0000","last_clean":"2024-09-21T21:43:49.623813+0000","last_became_active":"2024-09-21T21:42:24.516623+0000","last_became_peered":"2024-09-21T21:42:24.516623+0000","last_unstale":"2024-09-21T21:43:49.623813+0000","last_undegraded":"2024-09-21T21:43:49.623813+0000","last_fullsized":"2024-09-21T21:43:49.623813+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:51:59.592292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000906268,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"171'815","reported_seq":11441,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.373829+0000","last_change":"2024-09-21T21:43:33.290813+0000","last_active":"2024-09-21T21:43:49.373829+0000","last_peered":"2024-09-21T21:43:49.373829+0000","last_clean":"2024-09-21T21:43:49.373829+0000","last_became_active":"2024-09-21T21:40:37.418973+0000","last_became_peered":"2024-09-21T21:40:37.418973+0000","last_unstale":"2024-09-21T21:43:49.373829+0000","last_undegraded":"2024-09-21T21:43:49.373829+0000","last_fullsized":"2024-09-21T21:43:49.373829+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":815,"log_dups_size":0,"ondisk_log_size":815,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:31:26.860045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001131546,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":13425,"num_read_kb":28144,"num_write":602,"num_write_kb":6293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":105,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-21T21:43:48.228311+0000","last_change":"2024-09-21T21:43:44.357753+0000","last_active":"2024-09-21T21:43:48.228311+0000","last_peered":"2024-09-21T21:43:48.228311+0000","last_clean":"2024-09-21T21:43:48.228311+0000","last_became_active":"2024-09-21T21:43:44.357116+0000","last_became_peered":"2024-09-21T21:43:44.357116+0000","last_unstale":"2024-09-21T21:43:48.228311+0000","last_undegraded":"2024-09-21T21:43:48.228311+0000","last_fullsized":"2024-09-21T21:43:48.228311+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:22:20.440763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"168'387","reported_seq":16883,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.373980+0000","last_change":"2024-09-21T21:43:33.321435+0000","last_active":"2024-09-21T21:43:49.373980+0000","last_peered":"2024-09-21T21:43:49.373980+0000","last_clean":"2024-09-21T21:43:49.373980+0000","last_became_active":"2024-09-21T21:43:20.224689+0000","last_became_peered":"2024-09-21T21:43:20.224689+0000","last_unstale":"2024-09-21T21:43:49.373980+0000","last_undegraded":"2024-09-21T21:43:49.373980+0000","last_fullsized":"2024-09-21T21:43:49.373980+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":387,"log_dups_size":0,"ondisk_log_size":387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:15:32.359340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061419100000000004,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":15397,"num_read_kb":48505,"num_write":377,"num_write_kb":4859,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":429,"num_bytes_recovered":6598898,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":227,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.371346+0000","last_change":"2024-09-21T21:41:49.265465+0000","last_active":"2024-09-21T21:43:49.371346+0000","last_peered":"2024-09-21T21:43:49.371346+0000","last_clean":"2024-09-21T21:43:49.371346+0000","last_became_active":"2024-09-21T21:41:49.265182+0000","last_became_peered":"2024-09-21T21:41:49.265182+0000","last_unstale":"2024-09-21T21:43:49.371346+0000","last_undegraded":"2024-09-21T21:43:49.371346+0000","last_fullsized":"2024-09-21T21:43:49.371346+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:19:17.973195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"171'773","reported_seq":10302,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.374490+0000","last_change":"2024-09-21T21:43:33.293418+0000","last_active":"2024-09-21T21:43:49.374490+0000","last_peered":"2024-09-21T21:43:49.374490+0000","last_clean":"2024-09-21T21:43:49.374490+0000","last_became_active":"2024-09-21T21:40:37.421655+0000","last_became_peered":"2024-09-21T21:40:37.421655+0000","last_unstale":"2024-09-21T21:43:49.374490+0000","last_undegraded":"2024-09-21T21:43:49.374490+0000","last_fullsized":"2024-09-21T21:43:49.374490+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":773,"log_dups_size":0,"ondisk_log_size":773,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:43:34.040974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031354970000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":12713,"num_read_kb":34378,"num_write":754,"num_write_kb":7498,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":324,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.623702+0000","last_change":"2024-09-21T21:42:24.518810+0000","last_active":"2024-09-21T21:43:49.623702+0000","last_peered":"2024-09-21T21:43:49.623702+0000","last_clean":"2024-09-21T21:43:49.623702+0000","last_became_active":"2024-09-21T21:42:24.518461+0000","last_became_peered":"2024-09-21T21:42:24.518461+0000","last_unstale":"2024-09-21T21:43:49.623702+0000","last_undegraded":"2024-09-21T21:43:49.623702+0000","last_fullsized":"2024-09-21T21:43:49.623702+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:17:47.593497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036803999999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":433,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.371463+0000","last_change":"2024-09-21T21:43:06.231540+0000","last_active":"2024-09-21T21:43:49.371463+0000","last_peered":"2024-09-21T21:43:49.371463+0000","last_clean":"2024-09-21T21:43:49.371463+0000","last_became_active":"2024-09-21T21:43:06.231241+0000","last_became_peered":"2024-09-21T21:43:06.231241+0000","last_unstale":"2024-09-21T21:43:49.371463+0000","last_undegraded":"2024-09-21T21:43:49.371463+0000","last_fullsized":"2024-09-21T21:43:49.371463+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:18.340430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:18.340430+0000","last_clean_scrub_stamp":"2024-09-21T21:40:18.340430+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:50:08.187886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"170'374","reported_seq":6270,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-21T21:43:48.228114+0000","last_change":"2024-09-21T21:43:44.358111+0000","last_active":"2024-09-21T21:43:48.228114+0000","last_peered":"2024-09-21T21:43:48.228114+0000","last_clean":"2024-09-21T21:43:48.228114+0000","last_became_active":"2024-09-21T21:43:44.356024+0000","last_became_peered":"2024-09-21T21:43:44.356024+0000","last_unstale":"2024-09-21T21:43:48.228114+0000","last_undegraded":"2024-09-21T21:43:48.228114+0000","last_fullsized":"2024-09-21T21:43:48.228114+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":374,"log_dups_size":0,"ondisk_log_size":374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:17:43.127988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094280000000000004,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":5268,"num_read_kb":25972,"num_write":367,"num_write_kb":7278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":172,"num_bytes_recovered":2229995,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"170'554","reported_seq":8626,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:50.091421+0000","last_change":"2024-09-21T21:43:48.233080+0000","last_active":"2024-09-21T21:43:50.091421+0000","last_peered":"2024-09-21T21:43:50.091421+0000","last_clean":"2024-09-21T21:43:50.091421+0000","last_became_active":"2024-09-21T21:43:48.230703+0000","last_became_peered":"2024-09-21T21:43:48.230703+0000","last_unstale":"2024-09-21T21:43:50.091421+0000","last_undegraded":"2024-09-21T21:43:50.091421+0000","last_fullsized":"2024-09-21T21:43:50.091421+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":554,"log_dups_size":0,"ondisk_log_size":554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:39:12.422453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071450699999999997,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":8813,"num_read_kb":47394,"num_write":525,"num_write_kb":9434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":241,"num_bytes_recovered":7579476,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"171'1111","reported_seq":11670,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.371409+0000","last_change":"2024-09-21T21:43:33.292652+0000","last_active":"2024-09-21T21:43:49.371409+0000","last_peered":"2024-09-21T21:43:49.371409+0000","last_clean":"2024-09-21T21:43:49.371409+0000","last_became_active":"2024-09-21T21:43:19.224644+0000","last_became_peered":"2024-09-21T21:43:19.224644+0000","last_unstale":"2024-09-21T21:43:49.371409+0000","last_undegraded":"2024-09-21T21:43:49.371409+0000","last_fullsized":"2024-09-21T21:43:49.371409+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":1111,"log_dups_size":0,"ondisk_log_size":1111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:19:31.244395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001226878,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":5,"num_read":13231,"num_read_kb":49174,"num_write":1280,"num_write_kb":10665,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":398,"num_bytes_recovered":16632560,"num_keys_recovered":144,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"170'777","reported_seq":10308,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.374283+0000","last_change":"2024-09-21T21:43:33.292265+0000","last_active":"2024-09-21T21:43:49.374283+0000","last_peered":"2024-09-21T21:43:49.374283+0000","last_clean":"2024-09-21T21:43:49.374283+0000","last_became_active":"2024-09-21T21:43:09.227569+0000","last_became_peered":"2024-09-21T21:43:09.227569+0000","last_unstale":"2024-09-21T21:43:49.374283+0000","last_undegraded":"2024-09-21T21:43:49.374283+0000","last_fullsized":"2024-09-21T21:43:49.374283+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":777,"log_dups_size":0,"ondisk_log_size":777,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:20:34.944827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024712610000000002,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":10244,"num_read_kb":45368,"num_write":1134,"num_write_kb":14646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":331,"num_bytes_recovered":8931668,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"168'1625","reported_seq":14031,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.623830+0000","last_change":"2024-09-21T21:43:33.304566+0000","last_active":"2024-09-21T21:43:49.623830+0000","last_peered":"2024-09-21T21:43:49.623830+0000","last_clean":"2024-09-21T21:43:49.623830+0000","last_became_active":"2024-09-21T21:43:07.230148+0000","last_became_peered":"2024-09-21T21:43:07.230148+0000","last_unstale":"2024-09-21T21:43:49.623830+0000","last_undegraded":"2024-09-21T21:43:49.623830+0000","last_fullsized":"2024-09-21T21:43:49.623830+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":1625,"log_dups_size":0,"ondisk_log_size":1625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:42:46.324537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021897029999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":18573,"num_read_kb":61318,"num_write":2350,"num_write_kb":11538,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":225,"num_bytes_recovered":7091105,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"171'1447","reported_seq":17761,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.371616+0000","last_change":"2024-09-21T21:43:33.293137+0000","last_active":"2024-09-21T21:43:49.371616+0000","last_peered":"2024-09-21T21:43:49.371616+0000","last_clean":"2024-09-21T21:43:49.371616+0000","last_became_active":"2024-09-21T21:41:52.214022+0000","last_became_peered":"2024-09-21T21:41:52.214022+0000","last_unstale":"2024-09-21T21:43:49.371616+0000","last_undegraded":"2024-09-21T21:43:49.371616+0000","last_fullsized":"2024-09-21T21:43:49.371616+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":1447,"log_dups_size":0,"ondisk_log_size":1447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:42:14.727563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016156199999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":21216,"num_read_kb":45587,"num_write":1846,"num_write_kb":14797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":149,"num_bytes_recovered":3462638,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"168'454","reported_seq":7909,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.623841+0000","last_change":"2024-09-21T21:43:33.304631+0000","last_active":"2024-09-21T21:43:49.623841+0000","last_peered":"2024-09-21T21:43:49.623841+0000","last_clean":"2024-09-21T21:43:49.623841+0000","last_became_active":"2024-09-21T21:43:18.223755+0000","last_became_peered":"2024-09-21T21:43:18.223755+0000","last_unstale":"2024-09-21T21:43:49.623841+0000","last_undegraded":"2024-09-21T21:43:49.623841+0000","last_fullsized":"2024-09-21T21:43:49.623841+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":454,"log_dups_size":0,"ondisk_log_size":454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:43:32.324890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018835950000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":6571,"num_read_kb":34865,"num_write":469,"num_write_kb":6862,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":238,"num_bytes_recovered":4045989,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"169'741","reported_seq":17136,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.623910+0000","last_change":"2024-09-21T21:43:33.303579+0000","last_active":"2024-09-21T21:43:49.623910+0000","last_peered":"2024-09-21T21:43:49.623910+0000","last_clean":"2024-09-21T21:43:49.623910+0000","last_became_active":"2024-09-21T21:41:52.212206+0000","last_became_peered":"2024-09-21T21:41:52.212206+0000","last_unstale":"2024-09-21T21:43:49.623910+0000","last_undegraded":"2024-09-21T21:43:49.623910+0000","last_fullsized":"2024-09-21T21:43:49.623910+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":741,"log_dups_size":0,"ondisk_log_size":741,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:10:57.220429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013653929999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":18315,"num_read_kb":42344,"num_write":672,"num_write_kb":9505,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":2967886,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"167'408","reported_seq":14252,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.623662+0000","last_change":"2024-09-21T21:43:33.302904+0000","last_active":"2024-09-21T21:43:49.623662+0000","last_peered":"2024-09-21T21:43:49.623662+0000","last_clean":"2024-09-21T21:43:49.623662+0000","last_became_active":"2024-09-21T21:43:06.233222+0000","last_became_peered":"2024-09-21T21:43:06.233222+0000","last_unstale":"2024-09-21T21:43:49.623662+0000","last_undegraded":"2024-09-21T21:43:49.623662+0000","last_fullsized":"2024-09-21T21:43:49.623662+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:54:36.083839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085383799999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":12974,"num_read_kb":55237,"num_write":424,"num_write_kb":11231,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":205,"num_bytes_recovered":6455839,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"168'641","reported_seq":8507,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.371363+0000","last_change":"2024-09-21T21:43:33.292136+0000","last_active":"2024-09-21T21:43:49.371363+0000","last_peered":"2024-09-21T21:43:49.371363+0000","last_clean":"2024-09-21T21:43:49.371363+0000","last_became_active":"2024-09-21T21:43:20.224840+0000","last_became_peered":"2024-09-21T21:43:20.224840+0000","last_unstale":"2024-09-21T21:43:49.371363+0000","last_undegraded":"2024-09-21T21:43:49.371363+0000","last_fullsized":"2024-09-21T21:43:49.371363+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":641,"log_dups_size":0,"ondisk_log_size":641,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:13:32.197836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091468500000000004,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":7916,"num_read_kb":33911,"num_write":730,"num_write_kb":6868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":492,"num_bytes_recovered":3936953,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"170'791","reported_seq":19355,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.374375+0000","last_change":"2024-09-21T21:43:33.321430+0000","last_active":"2024-09-21T21:43:49.374375+0000","last_peered":"2024-09-21T21:43:49.374375+0000","last_clean":"2024-09-21T21:43:49.374375+0000","last_became_active":"2024-09-21T21:43:07.229497+0000","last_became_peered":"2024-09-21T21:43:07.229497+0000","last_unstale":"2024-09-21T21:43:49.374375+0000","last_undegraded":"2024-09-21T21:43:49.374375+0000","last_fullsized":"2024-09-21T21:43:49.374375+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":791,"log_dups_size":0,"ondisk_log_size":791,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:10:05.347980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061499099999999995,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":20795,"num_read_kb":71790,"num_write":798,"num_write_kb":9984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":352,"num_bytes_recovered":16923683,"num_keys_recovered":92,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"168'477","reported_seq":10652,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.371417+0000","last_change":"2024-09-21T21:43:33.292617+0000","last_active":"2024-09-21T21:43:49.371417+0000","last_peered":"2024-09-21T21:43:49.371417+0000","last_clean":"2024-09-21T21:43:49.371417+0000","last_became_active":"2024-09-21T21:41:52.209710+0000","last_became_peered":"2024-09-21T21:41:52.209710+0000","last_unstale":"2024-09-21T21:43:49.371417+0000","last_undegraded":"2024-09-21T21:43:49.371417+0000","last_fullsized":"2024-09-21T21:43:49.371417+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":477,"log_dups_size":0,"ondisk_log_size":477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:20:59.585304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013513189999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":9647,"num_read_kb":76927,"num_write":508,"num_write_kb":12558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":146,"num_bytes_recovered":17387622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":241,"num_object_clones":0,"num_object_copies":482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":241,"num_whiteouts":237,"num_read":408853,"num_read_kb":1471230,"num_write":27874,"num_write_kb":319707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7824,"num_bytes_recovered":220201937,"num_keys_recovered":844,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24446,"ondisk_log_size":24446,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":102090,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":4,"up_from":13,"seq":55834574894,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55556,"kb_used_data":2160,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316284,"statfs":{"total":96636764160,"available":96579874816,"internally_reserved":0,"allocated":2211840,"data_stored":928386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574894,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87536,"kb_used_data":2204,"kb_used_omap":1,"kb_used_meta":85310,"kb_avail":94284304,"statfs":{"total":96636764160,"available":96547127296,"internally_reserved":0,"allocated":2256896,"data_stored":1518735,"data_compressed":34030,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1593,"internal_metadata":87357895},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":12,"seq":51539607598,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73604,"kb_used_data":2160,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298236,"statfs":{"total":96636764160,"available":96561393664,"internally_reserved":0,"allocated":2211840,"data_stored":928386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607598,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55576,"kb_used_data":2176,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316264,"statfs":{"total":96636764160,"available":96579854336,"internally_reserved":0,"allocated":2228224,"data_stored":933375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607599,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47296,"kb_used_data":2216,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324544,"statfs":{"total":96636764160,"available":96588333056,"internally_reserved":0,"allocated":2269184,"data_stored":1523705,"data_compressed":34030,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640302,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47232,"kb_used_data":2156,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324608,"statfs":{"total":96636764160,"available":96588398592,"internally_reserved":0,"allocated":2207744,"data_stored":928367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34030,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34030,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34030,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T21:43:51.262 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-21T21:43:51.262 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-09-21T21:43:51.558 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-21T21:43:51.567 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T21:43:51.567 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T21:43:51.840 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:43:51.840 INFO:teuthology.orchestra.run.smithi081.stderr:dumped all 2024-09-21T21:43:51.852 INFO:teuthology.orchestra.run.smithi081.stdout:{"pg_ready":true,"pg_map":{"version":287,"stamp":"2024-09-21T21:43:50.324770+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":245,"num_object_clones":0,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":237,"num_read":408969,"num_read_kb":1471330,"num_write":28035,"num_write_kb":322125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7832,"num_bytes_recovered":222563409,"num_keys_recovered":844,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24566,"ondisk_log_size":24566,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":366800,"kb_used_data":13072,"kb_used_omap":9,"kb_used_meta":353590,"kb_avail":565864240,"statfs":{"total":579820584960,"available":579444981760,"internally_reserved":0,"allocated":13385728,"data_stored":6760954,"data_compressed":68060,"data_compressed_allocated":90112,"data_compressed_original":1179648,"omap_allocated":9568,"internal_metadata":362076832},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":-11,"num_read_kb":-11,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":125,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000777"},"pg_stats":[{"pgid":"3.1f","version":"171'591","reported_seq":7658,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.374605+0000","last_change":"2024-09-21T21:43:33.322591+0000","last_active":"2024-09-21T21:43:49.374605+0000","last_peered":"2024-09-21T21:43:49.374605+0000","last_clean":"2024-09-21T21:43:49.374605+0000","last_became_active":"2024-09-21T21:43:08.230509+0000","last_became_peered":"2024-09-21T21:43:08.230509+0000","last_unstale":"2024-09-21T21:43:49.374605+0000","last_undegraded":"2024-09-21T21:43:49.374605+0000","last_fullsized":"2024-09-21T21:43:49.374605+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":591,"log_dups_size":0,"ondisk_log_size":591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:50:51.925851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013233089999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":8384,"num_read_kb":24764,"num_write":554,"num_write_kb":8557,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":293,"num_bytes_recovered":3779776,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"168'1013","reported_seq":15326,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-21T21:43:48.228069+0000","last_change":"2024-09-21T21:43:44.357820+0000","last_active":"2024-09-21T21:43:48.228069+0000","last_peered":"2024-09-21T21:43:48.228069+0000","last_clean":"2024-09-21T21:43:48.228069+0000","last_became_active":"2024-09-21T21:43:44.356029+0000","last_became_peered":"2024-09-21T21:43:44.356029+0000","last_unstale":"2024-09-21T21:43:48.228069+0000","last_undegraded":"2024-09-21T21:43:48.228069+0000","last_fullsized":"2024-09-21T21:43:48.228069+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":1013,"log_dups_size":0,"ondisk_log_size":1013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:20:13.278523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001190569,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":16219,"num_read_kb":73109,"num_write":1473,"num_write_kb":18401,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":14291598,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"168'985","reported_seq":10752,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.623764+0000","last_change":"2024-09-21T21:43:33.302894+0000","last_active":"2024-09-21T21:43:49.623764+0000","last_peered":"2024-09-21T21:43:49.623764+0000","last_clean":"2024-09-21T21:43:49.623764+0000","last_became_active":"2024-09-21T21:43:20.225483+0000","last_became_peered":"2024-09-21T21:43:20.225483+0000","last_unstale":"2024-09-21T21:43:49.623764+0000","last_undegraded":"2024-09-21T21:43:49.623764+0000","last_fullsized":"2024-09-21T21:43:49.623764+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":985,"log_dups_size":0,"ondisk_log_size":985,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:13:56.033873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077268300000000003,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":13110,"num_read_kb":43443,"num_write":1158,"num_write_kb":7907,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":4009384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"171'884","reported_seq":13301,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.623722+0000","last_change":"2024-09-21T21:43:33.304149+0000","last_active":"2024-09-21T21:43:49.623722+0000","last_peered":"2024-09-21T21:43:49.623722+0000","last_clean":"2024-09-21T21:43:49.623722+0000","last_became_active":"2024-09-21T21:43:19.225435+0000","last_became_peered":"2024-09-21T21:43:19.225435+0000","last_unstale":"2024-09-21T21:43:49.623722+0000","last_undegraded":"2024-09-21T21:43:49.623722+0000","last_fullsized":"2024-09-21T21:43:49.623722+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":884,"log_dups_size":0,"ondisk_log_size":884,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:45:28.185309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001856124,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":15417,"num_read_kb":35250,"num_write":925,"num_write_kb":7812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":178,"num_bytes_recovered":1721983,"num_keys_recovered":148,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"171'737","reported_seq":9064,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.374605+0000","last_change":"2024-09-21T21:43:33.322211+0000","last_active":"2024-09-21T21:43:49.374605+0000","last_peered":"2024-09-21T21:43:49.374605+0000","last_clean":"2024-09-21T21:43:49.374605+0000","last_became_active":"2024-09-21T21:43:19.225304+0000","last_became_peered":"2024-09-21T21:43:19.225304+0000","last_unstale":"2024-09-21T21:43:49.374605+0000","last_undegraded":"2024-09-21T21:43:49.374605+0000","last_fullsized":"2024-09-21T21:43:49.374605+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":737,"log_dups_size":0,"ondisk_log_size":737,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:07:04.863691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001146085,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":8621,"num_read_kb":37221,"num_write":843,"num_write_kb":4971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":547,"num_bytes_recovered":1792422,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"170'618","reported_seq":25431,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:50.091426+0000","last_change":"2024-09-21T21:43:48.232831+0000","last_active":"2024-09-21T21:43:50.091426+0000","last_peered":"2024-09-21T21:43:50.091426+0000","last_clean":"2024-09-21T21:43:50.091426+0000","last_became_active":"2024-09-21T21:43:48.230767+0000","last_became_peered":"2024-09-21T21:43:48.230767+0000","last_unstale":"2024-09-21T21:43:50.091426+0000","last_undegraded":"2024-09-21T21:43:50.091426+0000","last_fullsized":"2024-09-21T21:43:50.091426+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":618,"log_dups_size":0,"ondisk_log_size":618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:47:54.923891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0044588529999999996,"stat_sum":{"num_bytes":19,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":25138,"num_read_kb":59201,"num_write":578,"num_write_kb":7782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":203,"num_bytes_recovered":4464880,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"170'860","reported_seq":14187,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.373846+0000","last_change":"2024-09-21T21:43:33.291493+0000","last_active":"2024-09-21T21:43:49.373846+0000","last_peered":"2024-09-21T21:43:49.373846+0000","last_clean":"2024-09-21T21:43:49.373846+0000","last_became_active":"2024-09-21T21:43:20.224845+0000","last_became_peered":"2024-09-21T21:43:20.224845+0000","last_unstale":"2024-09-21T21:43:49.373846+0000","last_undegraded":"2024-09-21T21:43:49.373846+0000","last_fullsized":"2024-09-21T21:43:49.373846+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":860,"log_dups_size":0,"ondisk_log_size":860,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:55:13.229277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018111119999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":16015,"num_read_kb":61704,"num_write":858,"num_write_kb":15379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":9899814,"num_keys_recovered":59,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"168'335","reported_seq":6955,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-21T21:43:48.228040+0000","last_change":"2024-09-21T21:43:44.358159+0000","last_active":"2024-09-21T21:43:48.228040+0000","last_peered":"2024-09-21T21:43:48.228040+0000","last_clean":"2024-09-21T21:43:48.228040+0000","last_became_active":"2024-09-21T21:43:44.356203+0000","last_became_peered":"2024-09-21T21:43:44.356203+0000","last_unstale":"2024-09-21T21:43:48.228040+0000","last_undegraded":"2024-09-21T21:43:48.228040+0000","last_fullsized":"2024-09-21T21:43:48.228040+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:53:07.504981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012910630000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":6161,"num_read_kb":33023,"num_write":325,"num_write_kb":8639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":5854224,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"171'497","reported_seq":6419,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.374509+0000","last_change":"2024-09-21T21:43:33.322741+0000","last_active":"2024-09-21T21:43:49.374509+0000","last_peered":"2024-09-21T21:43:49.374509+0000","last_clean":"2024-09-21T21:43:49.374509+0000","last_became_active":"2024-09-21T21:43:07.232590+0000","last_became_peered":"2024-09-21T21:43:07.232590+0000","last_unstale":"2024-09-21T21:43:49.374509+0000","last_undegraded":"2024-09-21T21:43:49.374509+0000","last_fullsized":"2024-09-21T21:43:49.374509+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":497,"log_dups_size":0,"ondisk_log_size":497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:04:00.513414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001467229,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":6465,"num_read_kb":25955,"num_write":521,"num_write_kb":9178,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":203,"num_bytes_recovered":4149457,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"171'1238","reported_seq":9868,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.623839+0000","last_change":"2024-09-21T21:43:33.305145+0000","last_active":"2024-09-21T21:43:49.623839+0000","last_peered":"2024-09-21T21:43:49.623839+0000","last_clean":"2024-09-21T21:43:49.623839+0000","last_became_active":"2024-09-21T21:41:53.214270+0000","last_became_peered":"2024-09-21T21:41:53.214270+0000","last_unstale":"2024-09-21T21:43:49.623839+0000","last_undegraded":"2024-09-21T21:43:49.623839+0000","last_fullsized":"2024-09-21T21:43:49.623839+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":1238,"log_dups_size":0,"ondisk_log_size":1238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:51:58.701104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022124750000000002,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":13255,"num_read_kb":44399,"num_write":1796,"num_write_kb":12840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":148,"num_bytes_recovered":4869268,"num_keys_recovered":47,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"170'744","reported_seq":9490,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.371529+0000","last_change":"2024-09-21T21:43:33.293155+0000","last_active":"2024-09-21T21:43:49.371529+0000","last_peered":"2024-09-21T21:43:49.371529+0000","last_clean":"2024-09-21T21:43:49.371529+0000","last_became_active":"2024-09-21T21:43:07.228504+0000","last_became_peered":"2024-09-21T21:43:07.228504+0000","last_unstale":"2024-09-21T21:43:49.371529+0000","last_undegraded":"2024-09-21T21:43:49.371529+0000","last_fullsized":"2024-09-21T21:43:49.371529+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":744,"log_dups_size":0,"ondisk_log_size":744,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:55:34.005752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001650253,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":9201,"num_read_kb":56862,"num_write":979,"num_write_kb":7172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":520,"num_bytes_recovered":10162533,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"167'626","reported_seq":21896,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.371520+0000","last_change":"2024-09-21T21:43:48.230374+0000","last_active":"2024-09-21T21:43:49.371520+0000","last_peered":"2024-09-21T21:43:49.371520+0000","last_clean":"2024-09-21T21:43:49.371520+0000","last_became_active":"2024-09-21T21:43:48.228528+0000","last_became_peered":"2024-09-21T21:43:48.228528+0000","last_unstale":"2024-09-21T21:43:49.371520+0000","last_undegraded":"2024-09-21T21:43:49.371520+0000","last_fullsized":"2024-09-21T21:43:49.371520+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":626,"log_dups_size":0,"ondisk_log_size":626,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:52:06.587738+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.00063765000000000002,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":22083,"num_read_kb":75165,"num_write":513,"num_write_kb":10980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":602,"num_bytes_recovered":22337258,"num_keys_recovered":84,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":236,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.371668+0000","last_change":"2024-09-21T21:43:15.316851+0000","last_active":"2024-09-21T21:43:49.371668+0000","last_peered":"2024-09-21T21:43:49.371668+0000","last_clean":"2024-09-21T21:43:49.371668+0000","last_became_active":"2024-09-21T21:43:15.316533+0000","last_became_peered":"2024-09-21T21:43:15.316533+0000","last_unstale":"2024-09-21T21:43:49.371668+0000","last_undegraded":"2024-09-21T21:43:49.371668+0000","last_fullsized":"2024-09-21T21:43:49.371668+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:00:30.781100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"171'1225","reported_seq":14273,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.374367+0000","last_change":"2024-09-21T21:43:33.321874+0000","last_active":"2024-09-21T21:43:49.374367+0000","last_peered":"2024-09-21T21:43:49.374367+0000","last_clean":"2024-09-21T21:43:49.374367+0000","last_became_active":"2024-09-21T21:43:09.230849+0000","last_became_peered":"2024-09-21T21:43:09.230849+0000","last_unstale":"2024-09-21T21:43:49.374367+0000","last_undegraded":"2024-09-21T21:43:49.374367+0000","last_fullsized":"2024-09-21T21:43:49.374367+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":1225,"log_dups_size":0,"ondisk_log_size":1225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:50:50.908667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096363700000000004,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":15823,"num_read_kb":66323,"num_write":1574,"num_write_kb":15987,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":349,"num_bytes_recovered":9257588,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":362,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.374023+0000","last_change":"2024-09-21T21:41:49.266534+0000","last_active":"2024-09-21T21:43:49.374023+0000","last_peered":"2024-09-21T21:43:49.374023+0000","last_clean":"2024-09-21T21:43:49.374023+0000","last_became_active":"2024-09-21T21:41:49.266257+0000","last_became_peered":"2024-09-21T21:41:49.266257+0000","last_unstale":"2024-09-21T21:43:49.374023+0000","last_undegraded":"2024-09-21T21:43:49.374023+0000","last_fullsized":"2024-09-21T21:43:49.374023+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:38:31.063193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062280799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"170'555","reported_seq":8886,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-21T21:43:48.227944+0000","last_change":"2024-09-21T21:43:44.357970+0000","last_active":"2024-09-21T21:43:48.227944+0000","last_peered":"2024-09-21T21:43:48.227944+0000","last_clean":"2024-09-21T21:43:48.227944+0000","last_became_active":"2024-09-21T21:43:44.356543+0000","last_became_peered":"2024-09-21T21:43:44.356543+0000","last_unstale":"2024-09-21T21:43:48.227944+0000","last_undegraded":"2024-09-21T21:43:48.227944+0000","last_fullsized":"2024-09-21T21:43:48.227944+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":555,"log_dups_size":0,"ondisk_log_size":555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:10:07.726600+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.00099396100000000002,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":10770,"num_read_kb":40597,"num_write":524,"num_write_kb":10606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":241,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.371492+0000","last_change":"2024-09-21T21:43:03.224822+0000","last_active":"2024-09-21T21:43:49.371492+0000","last_peered":"2024-09-21T21:43:49.371492+0000","last_clean":"2024-09-21T21:43:49.371492+0000","last_became_active":"2024-09-21T21:43:03.224515+0000","last_became_peered":"2024-09-21T21:43:03.224515+0000","last_unstale":"2024-09-21T21:43:49.371492+0000","last_undegraded":"2024-09-21T21:43:49.371492+0000","last_fullsized":"2024-09-21T21:43:49.371492+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:08:18.537918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"170'880","reported_seq":4751,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.373958+0000","last_change":"2024-09-21T21:43:33.291832+0000","last_active":"2024-09-21T21:43:49.373958+0000","last_peered":"2024-09-21T21:43:49.373958+0000","last_clean":"2024-09-21T21:43:49.373958+0000","last_became_active":"2024-09-21T21:40:37.419944+0000","last_became_peered":"2024-09-21T21:40:37.419944+0000","last_unstale":"2024-09-21T21:43:49.373958+0000","last_undegraded":"2024-09-21T21:43:49.373958+0000","last_fullsized":"2024-09-21T21:43:49.373958+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":880,"log_dups_size":0,"ondisk_log_size":880,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:53:04.189791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020772899999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":5927,"num_read_kb":25257,"num_write":1263,"num_write_kb":11321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":367,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.374070+0000","last_change":"2024-09-21T21:43:03.224909+0000","last_active":"2024-09-21T21:43:49.374070+0000","last_peered":"2024-09-21T21:43:49.374070+0000","last_clean":"2024-09-21T21:43:49.374070+0000","last_became_active":"2024-09-21T21:43:03.224572+0000","last_became_peered":"2024-09-21T21:43:03.224572+0000","last_unstale":"2024-09-21T21:43:49.374070+0000","last_undegraded":"2024-09-21T21:43:49.374070+0000","last_fullsized":"2024-09-21T21:43:49.374070+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:10:47.109676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070669899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"168'637","reported_seq":10497,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.623794+0000","last_change":"2024-09-21T21:43:33.303937+0000","last_active":"2024-09-21T21:43:49.623794+0000","last_peered":"2024-09-21T21:43:49.623794+0000","last_clean":"2024-09-21T21:43:49.623794+0000","last_became_active":"2024-09-21T21:41:25.650379+0000","last_became_peered":"2024-09-21T21:41:25.650379+0000","last_unstale":"2024-09-21T21:43:49.623794+0000","last_undegraded":"2024-09-21T21:43:49.623794+0000","last_fullsized":"2024-09-21T21:43:49.623794+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":637,"log_dups_size":0,"ondisk_log_size":637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:48:59.774761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001695787,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":11334,"num_read_kb":32091,"num_write":515,"num_write_kb":8339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"171'646","reported_seq":11168,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:50.091476+0000","last_change":"2024-09-21T21:43:48.233737+0000","last_active":"2024-09-21T21:43:50.091476+0000","last_peered":"2024-09-21T21:43:50.091476+0000","last_clean":"2024-09-21T21:43:50.091476+0000","last_became_active":"2024-09-21T21:43:48.231457+0000","last_became_peered":"2024-09-21T21:43:48.231457+0000","last_unstale":"2024-09-21T21:43:50.091476+0000","last_undegraded":"2024-09-21T21:43:50.091476+0000","last_fullsized":"2024-09-21T21:43:50.091476+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:10:37.262673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081362800000000005,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":9832,"num_read_kb":35952,"num_write":639,"num_write_kb":9820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":437,"num_bytes_recovered":19367440,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":226,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.623813+0000","last_change":"2024-09-21T21:42:24.516929+0000","last_active":"2024-09-21T21:43:49.623813+0000","last_peered":"2024-09-21T21:43:49.623813+0000","last_clean":"2024-09-21T21:43:49.623813+0000","last_became_active":"2024-09-21T21:42:24.516623+0000","last_became_peered":"2024-09-21T21:42:24.516623+0000","last_unstale":"2024-09-21T21:43:49.623813+0000","last_undegraded":"2024-09-21T21:43:49.623813+0000","last_fullsized":"2024-09-21T21:43:49.623813+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:51:59.592292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000906268,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"171'815","reported_seq":11441,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.373829+0000","last_change":"2024-09-21T21:43:33.290813+0000","last_active":"2024-09-21T21:43:49.373829+0000","last_peered":"2024-09-21T21:43:49.373829+0000","last_clean":"2024-09-21T21:43:49.373829+0000","last_became_active":"2024-09-21T21:40:37.418973+0000","last_became_peered":"2024-09-21T21:40:37.418973+0000","last_unstale":"2024-09-21T21:43:49.373829+0000","last_undegraded":"2024-09-21T21:43:49.373829+0000","last_fullsized":"2024-09-21T21:43:49.373829+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":815,"log_dups_size":0,"ondisk_log_size":815,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:31:26.860045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001131546,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":13425,"num_read_kb":28144,"num_write":602,"num_write_kb":6293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":105,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-21T21:43:48.228311+0000","last_change":"2024-09-21T21:43:44.357753+0000","last_active":"2024-09-21T21:43:48.228311+0000","last_peered":"2024-09-21T21:43:48.228311+0000","last_clean":"2024-09-21T21:43:48.228311+0000","last_became_active":"2024-09-21T21:43:44.357116+0000","last_became_peered":"2024-09-21T21:43:44.357116+0000","last_unstale":"2024-09-21T21:43:48.228311+0000","last_undegraded":"2024-09-21T21:43:48.228311+0000","last_fullsized":"2024-09-21T21:43:48.228311+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:22:20.440763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"168'387","reported_seq":16883,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.373980+0000","last_change":"2024-09-21T21:43:33.321435+0000","last_active":"2024-09-21T21:43:49.373980+0000","last_peered":"2024-09-21T21:43:49.373980+0000","last_clean":"2024-09-21T21:43:49.373980+0000","last_became_active":"2024-09-21T21:43:20.224689+0000","last_became_peered":"2024-09-21T21:43:20.224689+0000","last_unstale":"2024-09-21T21:43:49.373980+0000","last_undegraded":"2024-09-21T21:43:49.373980+0000","last_fullsized":"2024-09-21T21:43:49.373980+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":387,"log_dups_size":0,"ondisk_log_size":387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:15:32.359340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061419100000000004,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":15397,"num_read_kb":48505,"num_write":377,"num_write_kb":4859,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":429,"num_bytes_recovered":6598898,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":227,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.371346+0000","last_change":"2024-09-21T21:41:49.265465+0000","last_active":"2024-09-21T21:43:49.371346+0000","last_peered":"2024-09-21T21:43:49.371346+0000","last_clean":"2024-09-21T21:43:49.371346+0000","last_became_active":"2024-09-21T21:41:49.265182+0000","last_became_peered":"2024-09-21T21:41:49.265182+0000","last_unstale":"2024-09-21T21:43:49.371346+0000","last_undegraded":"2024-09-21T21:43:49.371346+0000","last_fullsized":"2024-09-21T21:43:49.371346+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:19:17.973195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"171'773","reported_seq":10302,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.374490+0000","last_change":"2024-09-21T21:43:33.293418+0000","last_active":"2024-09-21T21:43:49.374490+0000","last_peered":"2024-09-21T21:43:49.374490+0000","last_clean":"2024-09-21T21:43:49.374490+0000","last_became_active":"2024-09-21T21:40:37.421655+0000","last_became_peered":"2024-09-21T21:40:37.421655+0000","last_unstale":"2024-09-21T21:43:49.374490+0000","last_undegraded":"2024-09-21T21:43:49.374490+0000","last_fullsized":"2024-09-21T21:43:49.374490+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":773,"log_dups_size":0,"ondisk_log_size":773,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:43:34.040974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031354970000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":12713,"num_read_kb":34378,"num_write":754,"num_write_kb":7498,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":324,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.623702+0000","last_change":"2024-09-21T21:42:24.518810+0000","last_active":"2024-09-21T21:43:49.623702+0000","last_peered":"2024-09-21T21:43:49.623702+0000","last_clean":"2024-09-21T21:43:49.623702+0000","last_became_active":"2024-09-21T21:42:24.518461+0000","last_became_peered":"2024-09-21T21:42:24.518461+0000","last_unstale":"2024-09-21T21:43:49.623702+0000","last_undegraded":"2024-09-21T21:43:49.623702+0000","last_fullsized":"2024-09-21T21:43:49.623702+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:17:47.593497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036803999999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":433,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.371463+0000","last_change":"2024-09-21T21:43:06.231540+0000","last_active":"2024-09-21T21:43:49.371463+0000","last_peered":"2024-09-21T21:43:49.371463+0000","last_clean":"2024-09-21T21:43:49.371463+0000","last_became_active":"2024-09-21T21:43:06.231241+0000","last_became_peered":"2024-09-21T21:43:06.231241+0000","last_unstale":"2024-09-21T21:43:49.371463+0000","last_undegraded":"2024-09-21T21:43:49.371463+0000","last_fullsized":"2024-09-21T21:43:49.371463+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:18.340430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:18.340430+0000","last_clean_scrub_stamp":"2024-09-21T21:40:18.340430+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:50:08.187886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"170'374","reported_seq":6270,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-21T21:43:48.228114+0000","last_change":"2024-09-21T21:43:44.358111+0000","last_active":"2024-09-21T21:43:48.228114+0000","last_peered":"2024-09-21T21:43:48.228114+0000","last_clean":"2024-09-21T21:43:48.228114+0000","last_became_active":"2024-09-21T21:43:44.356024+0000","last_became_peered":"2024-09-21T21:43:44.356024+0000","last_unstale":"2024-09-21T21:43:48.228114+0000","last_undegraded":"2024-09-21T21:43:48.228114+0000","last_fullsized":"2024-09-21T21:43:48.228114+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":374,"log_dups_size":0,"ondisk_log_size":374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:17:43.127988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094280000000000004,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":5268,"num_read_kb":25972,"num_write":367,"num_write_kb":7278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":172,"num_bytes_recovered":2229995,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"170'554","reported_seq":8626,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:50.091421+0000","last_change":"2024-09-21T21:43:48.233080+0000","last_active":"2024-09-21T21:43:50.091421+0000","last_peered":"2024-09-21T21:43:50.091421+0000","last_clean":"2024-09-21T21:43:50.091421+0000","last_became_active":"2024-09-21T21:43:48.230703+0000","last_became_peered":"2024-09-21T21:43:48.230703+0000","last_unstale":"2024-09-21T21:43:50.091421+0000","last_undegraded":"2024-09-21T21:43:50.091421+0000","last_fullsized":"2024-09-21T21:43:50.091421+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":554,"log_dups_size":0,"ondisk_log_size":554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:39:12.422453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071450699999999997,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":8813,"num_read_kb":47394,"num_write":525,"num_write_kb":9434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":241,"num_bytes_recovered":7579476,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"171'1111","reported_seq":11670,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.371409+0000","last_change":"2024-09-21T21:43:33.292652+0000","last_active":"2024-09-21T21:43:49.371409+0000","last_peered":"2024-09-21T21:43:49.371409+0000","last_clean":"2024-09-21T21:43:49.371409+0000","last_became_active":"2024-09-21T21:43:19.224644+0000","last_became_peered":"2024-09-21T21:43:19.224644+0000","last_unstale":"2024-09-21T21:43:49.371409+0000","last_undegraded":"2024-09-21T21:43:49.371409+0000","last_fullsized":"2024-09-21T21:43:49.371409+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":1111,"log_dups_size":0,"ondisk_log_size":1111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:19:31.244395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001226878,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":5,"num_read":13231,"num_read_kb":49174,"num_write":1280,"num_write_kb":10665,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":398,"num_bytes_recovered":16632560,"num_keys_recovered":144,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"170'777","reported_seq":10308,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.374283+0000","last_change":"2024-09-21T21:43:33.292265+0000","last_active":"2024-09-21T21:43:49.374283+0000","last_peered":"2024-09-21T21:43:49.374283+0000","last_clean":"2024-09-21T21:43:49.374283+0000","last_became_active":"2024-09-21T21:43:09.227569+0000","last_became_peered":"2024-09-21T21:43:09.227569+0000","last_unstale":"2024-09-21T21:43:49.374283+0000","last_undegraded":"2024-09-21T21:43:49.374283+0000","last_fullsized":"2024-09-21T21:43:49.374283+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":777,"log_dups_size":0,"ondisk_log_size":777,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:20:34.944827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024712610000000002,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":10244,"num_read_kb":45368,"num_write":1134,"num_write_kb":14646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":331,"num_bytes_recovered":8931668,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"168'1625","reported_seq":14031,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.623830+0000","last_change":"2024-09-21T21:43:33.304566+0000","last_active":"2024-09-21T21:43:49.623830+0000","last_peered":"2024-09-21T21:43:49.623830+0000","last_clean":"2024-09-21T21:43:49.623830+0000","last_became_active":"2024-09-21T21:43:07.230148+0000","last_became_peered":"2024-09-21T21:43:07.230148+0000","last_unstale":"2024-09-21T21:43:49.623830+0000","last_undegraded":"2024-09-21T21:43:49.623830+0000","last_fullsized":"2024-09-21T21:43:49.623830+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":1625,"log_dups_size":0,"ondisk_log_size":1625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:42:46.324537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021897029999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":18573,"num_read_kb":61318,"num_write":2350,"num_write_kb":11538,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":225,"num_bytes_recovered":7091105,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"171'1447","reported_seq":17761,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.371616+0000","last_change":"2024-09-21T21:43:33.293137+0000","last_active":"2024-09-21T21:43:49.371616+0000","last_peered":"2024-09-21T21:43:49.371616+0000","last_clean":"2024-09-21T21:43:49.371616+0000","last_became_active":"2024-09-21T21:41:52.214022+0000","last_became_peered":"2024-09-21T21:41:52.214022+0000","last_unstale":"2024-09-21T21:43:49.371616+0000","last_undegraded":"2024-09-21T21:43:49.371616+0000","last_fullsized":"2024-09-21T21:43:49.371616+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":1447,"log_dups_size":0,"ondisk_log_size":1447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:42:14.727563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016156199999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":21216,"num_read_kb":45587,"num_write":1846,"num_write_kb":14797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":149,"num_bytes_recovered":3462638,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"168'454","reported_seq":7909,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.623841+0000","last_change":"2024-09-21T21:43:33.304631+0000","last_active":"2024-09-21T21:43:49.623841+0000","last_peered":"2024-09-21T21:43:49.623841+0000","last_clean":"2024-09-21T21:43:49.623841+0000","last_became_active":"2024-09-21T21:43:18.223755+0000","last_became_peered":"2024-09-21T21:43:18.223755+0000","last_unstale":"2024-09-21T21:43:49.623841+0000","last_undegraded":"2024-09-21T21:43:49.623841+0000","last_fullsized":"2024-09-21T21:43:49.623841+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":454,"log_dups_size":0,"ondisk_log_size":454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:43:32.324890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018835950000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":6571,"num_read_kb":34865,"num_write":469,"num_write_kb":6862,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":238,"num_bytes_recovered":4045989,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"169'741","reported_seq":17136,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.623910+0000","last_change":"2024-09-21T21:43:33.303579+0000","last_active":"2024-09-21T21:43:49.623910+0000","last_peered":"2024-09-21T21:43:49.623910+0000","last_clean":"2024-09-21T21:43:49.623910+0000","last_became_active":"2024-09-21T21:41:52.212206+0000","last_became_peered":"2024-09-21T21:41:52.212206+0000","last_unstale":"2024-09-21T21:43:49.623910+0000","last_undegraded":"2024-09-21T21:43:49.623910+0000","last_fullsized":"2024-09-21T21:43:49.623910+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":741,"log_dups_size":0,"ondisk_log_size":741,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:10:57.220429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013653929999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":18315,"num_read_kb":42344,"num_write":672,"num_write_kb":9505,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":2967886,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"167'408","reported_seq":14252,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.623662+0000","last_change":"2024-09-21T21:43:33.302904+0000","last_active":"2024-09-21T21:43:49.623662+0000","last_peered":"2024-09-21T21:43:49.623662+0000","last_clean":"2024-09-21T21:43:49.623662+0000","last_became_active":"2024-09-21T21:43:06.233222+0000","last_became_peered":"2024-09-21T21:43:06.233222+0000","last_unstale":"2024-09-21T21:43:49.623662+0000","last_undegraded":"2024-09-21T21:43:49.623662+0000","last_fullsized":"2024-09-21T21:43:49.623662+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:54:36.083839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085383799999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":12974,"num_read_kb":55237,"num_write":424,"num_write_kb":11231,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":205,"num_bytes_recovered":6455839,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"168'641","reported_seq":8507,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.371363+0000","last_change":"2024-09-21T21:43:33.292136+0000","last_active":"2024-09-21T21:43:49.371363+0000","last_peered":"2024-09-21T21:43:49.371363+0000","last_clean":"2024-09-21T21:43:49.371363+0000","last_became_active":"2024-09-21T21:43:20.224840+0000","last_became_peered":"2024-09-21T21:43:20.224840+0000","last_unstale":"2024-09-21T21:43:49.371363+0000","last_undegraded":"2024-09-21T21:43:49.371363+0000","last_fullsized":"2024-09-21T21:43:49.371363+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":641,"log_dups_size":0,"ondisk_log_size":641,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:13:32.197836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091468500000000004,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":7916,"num_read_kb":33911,"num_write":730,"num_write_kb":6868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":492,"num_bytes_recovered":3936953,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"170'791","reported_seq":19355,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.374375+0000","last_change":"2024-09-21T21:43:33.321430+0000","last_active":"2024-09-21T21:43:49.374375+0000","last_peered":"2024-09-21T21:43:49.374375+0000","last_clean":"2024-09-21T21:43:49.374375+0000","last_became_active":"2024-09-21T21:43:07.229497+0000","last_became_peered":"2024-09-21T21:43:07.229497+0000","last_unstale":"2024-09-21T21:43:49.374375+0000","last_undegraded":"2024-09-21T21:43:49.374375+0000","last_fullsized":"2024-09-21T21:43:49.374375+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":791,"log_dups_size":0,"ondisk_log_size":791,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:10:05.347980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061499099999999995,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":20795,"num_read_kb":71790,"num_write":798,"num_write_kb":9984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":352,"num_bytes_recovered":16923683,"num_keys_recovered":92,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"168'477","reported_seq":10652,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.371417+0000","last_change":"2024-09-21T21:43:33.292617+0000","last_active":"2024-09-21T21:43:49.371417+0000","last_peered":"2024-09-21T21:43:49.371417+0000","last_clean":"2024-09-21T21:43:49.371417+0000","last_became_active":"2024-09-21T21:41:52.209710+0000","last_became_peered":"2024-09-21T21:41:52.209710+0000","last_unstale":"2024-09-21T21:43:49.371417+0000","last_undegraded":"2024-09-21T21:43:49.371417+0000","last_fullsized":"2024-09-21T21:43:49.371417+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":477,"log_dups_size":0,"ondisk_log_size":477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:20:59.585304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013513189999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":9647,"num_read_kb":76927,"num_write":508,"num_write_kb":12558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":146,"num_bytes_recovered":17387622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":241,"num_object_clones":0,"num_object_copies":482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":241,"num_whiteouts":237,"num_read":408853,"num_read_kb":1471230,"num_write":27874,"num_write_kb":319707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7824,"num_bytes_recovered":220201937,"num_keys_recovered":844,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24446,"ondisk_log_size":24446,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":102090,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":4,"up_from":13,"seq":55834574894,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55556,"kb_used_data":2160,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316284,"statfs":{"total":96636764160,"available":96579874816,"internally_reserved":0,"allocated":2211840,"data_stored":928386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574894,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87536,"kb_used_data":2204,"kb_used_omap":1,"kb_used_meta":85310,"kb_avail":94284304,"statfs":{"total":96636764160,"available":96547127296,"internally_reserved":0,"allocated":2256896,"data_stored":1518735,"data_compressed":34030,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1593,"internal_metadata":87357895},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":12,"seq":51539607598,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73604,"kb_used_data":2160,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298236,"statfs":{"total":96636764160,"available":96561393664,"internally_reserved":0,"allocated":2211840,"data_stored":928386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607598,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55576,"kb_used_data":2176,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316264,"statfs":{"total":96636764160,"available":96579854336,"internally_reserved":0,"allocated":2228224,"data_stored":933375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607599,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47296,"kb_used_data":2216,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324544,"statfs":{"total":96636764160,"available":96588333056,"internally_reserved":0,"allocated":2269184,"data_stored":1523705,"data_compressed":34030,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640302,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47232,"kb_used_data":2156,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324608,"statfs":{"total":96636764160,"available":96588398592,"internally_reserved":0,"allocated":2207744,"data_stored":928367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34030,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34030,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34030,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T21:43:51.854 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T21:43:52.125 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:43:52.126 INFO:teuthology.orchestra.run.smithi081.stderr:dumped all 2024-09-21T21:43:52.137 INFO:teuthology.orchestra.run.smithi081.stdout:{"pg_ready":true,"pg_map":{"version":287,"stamp":"2024-09-21T21:43:50.324770+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":245,"num_object_clones":0,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":237,"num_read":408969,"num_read_kb":1471330,"num_write":28035,"num_write_kb":322125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7832,"num_bytes_recovered":222563409,"num_keys_recovered":844,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24566,"ondisk_log_size":24566,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":366800,"kb_used_data":13072,"kb_used_omap":9,"kb_used_meta":353590,"kb_avail":565864240,"statfs":{"total":579820584960,"available":579444981760,"internally_reserved":0,"allocated":13385728,"data_stored":6760954,"data_compressed":68060,"data_compressed_allocated":90112,"data_compressed_original":1179648,"omap_allocated":9568,"internal_metadata":362076832},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":-11,"num_read_kb":-11,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":125,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000777"},"pg_stats":[{"pgid":"3.1f","version":"171'591","reported_seq":7658,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.374605+0000","last_change":"2024-09-21T21:43:33.322591+0000","last_active":"2024-09-21T21:43:49.374605+0000","last_peered":"2024-09-21T21:43:49.374605+0000","last_clean":"2024-09-21T21:43:49.374605+0000","last_became_active":"2024-09-21T21:43:08.230509+0000","last_became_peered":"2024-09-21T21:43:08.230509+0000","last_unstale":"2024-09-21T21:43:49.374605+0000","last_undegraded":"2024-09-21T21:43:49.374605+0000","last_fullsized":"2024-09-21T21:43:49.374605+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":591,"log_dups_size":0,"ondisk_log_size":591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:50:51.925851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013233089999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":8384,"num_read_kb":24764,"num_write":554,"num_write_kb":8557,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":293,"num_bytes_recovered":3779776,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"168'1013","reported_seq":15326,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-21T21:43:48.228069+0000","last_change":"2024-09-21T21:43:44.357820+0000","last_active":"2024-09-21T21:43:48.228069+0000","last_peered":"2024-09-21T21:43:48.228069+0000","last_clean":"2024-09-21T21:43:48.228069+0000","last_became_active":"2024-09-21T21:43:44.356029+0000","last_became_peered":"2024-09-21T21:43:44.356029+0000","last_unstale":"2024-09-21T21:43:48.228069+0000","last_undegraded":"2024-09-21T21:43:48.228069+0000","last_fullsized":"2024-09-21T21:43:48.228069+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":1013,"log_dups_size":0,"ondisk_log_size":1013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:20:13.278523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001190569,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":16219,"num_read_kb":73109,"num_write":1473,"num_write_kb":18401,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":14291598,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"168'985","reported_seq":10752,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.623764+0000","last_change":"2024-09-21T21:43:33.302894+0000","last_active":"2024-09-21T21:43:49.623764+0000","last_peered":"2024-09-21T21:43:49.623764+0000","last_clean":"2024-09-21T21:43:49.623764+0000","last_became_active":"2024-09-21T21:43:20.225483+0000","last_became_peered":"2024-09-21T21:43:20.225483+0000","last_unstale":"2024-09-21T21:43:49.623764+0000","last_undegraded":"2024-09-21T21:43:49.623764+0000","last_fullsized":"2024-09-21T21:43:49.623764+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":985,"log_dups_size":0,"ondisk_log_size":985,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:13:56.033873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077268300000000003,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":13110,"num_read_kb":43443,"num_write":1158,"num_write_kb":7907,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":4009384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"171'884","reported_seq":13301,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.623722+0000","last_change":"2024-09-21T21:43:33.304149+0000","last_active":"2024-09-21T21:43:49.623722+0000","last_peered":"2024-09-21T21:43:49.623722+0000","last_clean":"2024-09-21T21:43:49.623722+0000","last_became_active":"2024-09-21T21:43:19.225435+0000","last_became_peered":"2024-09-21T21:43:19.225435+0000","last_unstale":"2024-09-21T21:43:49.623722+0000","last_undegraded":"2024-09-21T21:43:49.623722+0000","last_fullsized":"2024-09-21T21:43:49.623722+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":884,"log_dups_size":0,"ondisk_log_size":884,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:45:28.185309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001856124,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":15417,"num_read_kb":35250,"num_write":925,"num_write_kb":7812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":178,"num_bytes_recovered":1721983,"num_keys_recovered":148,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"171'737","reported_seq":9064,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.374605+0000","last_change":"2024-09-21T21:43:33.322211+0000","last_active":"2024-09-21T21:43:49.374605+0000","last_peered":"2024-09-21T21:43:49.374605+0000","last_clean":"2024-09-21T21:43:49.374605+0000","last_became_active":"2024-09-21T21:43:19.225304+0000","last_became_peered":"2024-09-21T21:43:19.225304+0000","last_unstale":"2024-09-21T21:43:49.374605+0000","last_undegraded":"2024-09-21T21:43:49.374605+0000","last_fullsized":"2024-09-21T21:43:49.374605+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":737,"log_dups_size":0,"ondisk_log_size":737,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:07:04.863691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001146085,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":8621,"num_read_kb":37221,"num_write":843,"num_write_kb":4971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":547,"num_bytes_recovered":1792422,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"170'618","reported_seq":25431,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:50.091426+0000","last_change":"2024-09-21T21:43:48.232831+0000","last_active":"2024-09-21T21:43:50.091426+0000","last_peered":"2024-09-21T21:43:50.091426+0000","last_clean":"2024-09-21T21:43:50.091426+0000","last_became_active":"2024-09-21T21:43:48.230767+0000","last_became_peered":"2024-09-21T21:43:48.230767+0000","last_unstale":"2024-09-21T21:43:50.091426+0000","last_undegraded":"2024-09-21T21:43:50.091426+0000","last_fullsized":"2024-09-21T21:43:50.091426+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":618,"log_dups_size":0,"ondisk_log_size":618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:47:54.923891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0044588529999999996,"stat_sum":{"num_bytes":19,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":25138,"num_read_kb":59201,"num_write":578,"num_write_kb":7782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":203,"num_bytes_recovered":4464880,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"170'860","reported_seq":14187,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.373846+0000","last_change":"2024-09-21T21:43:33.291493+0000","last_active":"2024-09-21T21:43:49.373846+0000","last_peered":"2024-09-21T21:43:49.373846+0000","last_clean":"2024-09-21T21:43:49.373846+0000","last_became_active":"2024-09-21T21:43:20.224845+0000","last_became_peered":"2024-09-21T21:43:20.224845+0000","last_unstale":"2024-09-21T21:43:49.373846+0000","last_undegraded":"2024-09-21T21:43:49.373846+0000","last_fullsized":"2024-09-21T21:43:49.373846+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":860,"log_dups_size":0,"ondisk_log_size":860,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:55:13.229277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018111119999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":16015,"num_read_kb":61704,"num_write":858,"num_write_kb":15379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":9899814,"num_keys_recovered":59,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"168'335","reported_seq":6955,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-21T21:43:48.228040+0000","last_change":"2024-09-21T21:43:44.358159+0000","last_active":"2024-09-21T21:43:48.228040+0000","last_peered":"2024-09-21T21:43:48.228040+0000","last_clean":"2024-09-21T21:43:48.228040+0000","last_became_active":"2024-09-21T21:43:44.356203+0000","last_became_peered":"2024-09-21T21:43:44.356203+0000","last_unstale":"2024-09-21T21:43:48.228040+0000","last_undegraded":"2024-09-21T21:43:48.228040+0000","last_fullsized":"2024-09-21T21:43:48.228040+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:53:07.504981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012910630000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":6161,"num_read_kb":33023,"num_write":325,"num_write_kb":8639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":5854224,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"171'497","reported_seq":6419,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.374509+0000","last_change":"2024-09-21T21:43:33.322741+0000","last_active":"2024-09-21T21:43:49.374509+0000","last_peered":"2024-09-21T21:43:49.374509+0000","last_clean":"2024-09-21T21:43:49.374509+0000","last_became_active":"2024-09-21T21:43:07.232590+0000","last_became_peered":"2024-09-21T21:43:07.232590+0000","last_unstale":"2024-09-21T21:43:49.374509+0000","last_undegraded":"2024-09-21T21:43:49.374509+0000","last_fullsized":"2024-09-21T21:43:49.374509+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":497,"log_dups_size":0,"ondisk_log_size":497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:04:00.513414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001467229,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":6465,"num_read_kb":25955,"num_write":521,"num_write_kb":9178,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":203,"num_bytes_recovered":4149457,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"171'1238","reported_seq":9868,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.623839+0000","last_change":"2024-09-21T21:43:33.305145+0000","last_active":"2024-09-21T21:43:49.623839+0000","last_peered":"2024-09-21T21:43:49.623839+0000","last_clean":"2024-09-21T21:43:49.623839+0000","last_became_active":"2024-09-21T21:41:53.214270+0000","last_became_peered":"2024-09-21T21:41:53.214270+0000","last_unstale":"2024-09-21T21:43:49.623839+0000","last_undegraded":"2024-09-21T21:43:49.623839+0000","last_fullsized":"2024-09-21T21:43:49.623839+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":1238,"log_dups_size":0,"ondisk_log_size":1238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:51:58.701104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022124750000000002,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":13255,"num_read_kb":44399,"num_write":1796,"num_write_kb":12840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":148,"num_bytes_recovered":4869268,"num_keys_recovered":47,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"170'744","reported_seq":9490,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.371529+0000","last_change":"2024-09-21T21:43:33.293155+0000","last_active":"2024-09-21T21:43:49.371529+0000","last_peered":"2024-09-21T21:43:49.371529+0000","last_clean":"2024-09-21T21:43:49.371529+0000","last_became_active":"2024-09-21T21:43:07.228504+0000","last_became_peered":"2024-09-21T21:43:07.228504+0000","last_unstale":"2024-09-21T21:43:49.371529+0000","last_undegraded":"2024-09-21T21:43:49.371529+0000","last_fullsized":"2024-09-21T21:43:49.371529+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":744,"log_dups_size":0,"ondisk_log_size":744,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:55:34.005752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001650253,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":9201,"num_read_kb":56862,"num_write":979,"num_write_kb":7172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":520,"num_bytes_recovered":10162533,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"167'626","reported_seq":21896,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.371520+0000","last_change":"2024-09-21T21:43:48.230374+0000","last_active":"2024-09-21T21:43:49.371520+0000","last_peered":"2024-09-21T21:43:49.371520+0000","last_clean":"2024-09-21T21:43:49.371520+0000","last_became_active":"2024-09-21T21:43:48.228528+0000","last_became_peered":"2024-09-21T21:43:48.228528+0000","last_unstale":"2024-09-21T21:43:49.371520+0000","last_undegraded":"2024-09-21T21:43:49.371520+0000","last_fullsized":"2024-09-21T21:43:49.371520+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":626,"log_dups_size":0,"ondisk_log_size":626,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:52:06.587738+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.00063765000000000002,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":22083,"num_read_kb":75165,"num_write":513,"num_write_kb":10980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":602,"num_bytes_recovered":22337258,"num_keys_recovered":84,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":236,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.371668+0000","last_change":"2024-09-21T21:43:15.316851+0000","last_active":"2024-09-21T21:43:49.371668+0000","last_peered":"2024-09-21T21:43:49.371668+0000","last_clean":"2024-09-21T21:43:49.371668+0000","last_became_active":"2024-09-21T21:43:15.316533+0000","last_became_peered":"2024-09-21T21:43:15.316533+0000","last_unstale":"2024-09-21T21:43:49.371668+0000","last_undegraded":"2024-09-21T21:43:49.371668+0000","last_fullsized":"2024-09-21T21:43:49.371668+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:00:30.781100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"171'1225","reported_seq":14273,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.374367+0000","last_change":"2024-09-21T21:43:33.321874+0000","last_active":"2024-09-21T21:43:49.374367+0000","last_peered":"2024-09-21T21:43:49.374367+0000","last_clean":"2024-09-21T21:43:49.374367+0000","last_became_active":"2024-09-21T21:43:09.230849+0000","last_became_peered":"2024-09-21T21:43:09.230849+0000","last_unstale":"2024-09-21T21:43:49.374367+0000","last_undegraded":"2024-09-21T21:43:49.374367+0000","last_fullsized":"2024-09-21T21:43:49.374367+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":1225,"log_dups_size":0,"ondisk_log_size":1225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:50:50.908667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096363700000000004,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":15823,"num_read_kb":66323,"num_write":1574,"num_write_kb":15987,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":349,"num_bytes_recovered":9257588,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":362,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.374023+0000","last_change":"2024-09-21T21:41:49.266534+0000","last_active":"2024-09-21T21:43:49.374023+0000","last_peered":"2024-09-21T21:43:49.374023+0000","last_clean":"2024-09-21T21:43:49.374023+0000","last_became_active":"2024-09-21T21:41:49.266257+0000","last_became_peered":"2024-09-21T21:41:49.266257+0000","last_unstale":"2024-09-21T21:43:49.374023+0000","last_undegraded":"2024-09-21T21:43:49.374023+0000","last_fullsized":"2024-09-21T21:43:49.374023+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:38:31.063193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062280799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"170'555","reported_seq":8886,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-21T21:43:48.227944+0000","last_change":"2024-09-21T21:43:44.357970+0000","last_active":"2024-09-21T21:43:48.227944+0000","last_peered":"2024-09-21T21:43:48.227944+0000","last_clean":"2024-09-21T21:43:48.227944+0000","last_became_active":"2024-09-21T21:43:44.356543+0000","last_became_peered":"2024-09-21T21:43:44.356543+0000","last_unstale":"2024-09-21T21:43:48.227944+0000","last_undegraded":"2024-09-21T21:43:48.227944+0000","last_fullsized":"2024-09-21T21:43:48.227944+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":555,"log_dups_size":0,"ondisk_log_size":555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:10:07.726600+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.00099396100000000002,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":10770,"num_read_kb":40597,"num_write":524,"num_write_kb":10606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":241,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.371492+0000","last_change":"2024-09-21T21:43:03.224822+0000","last_active":"2024-09-21T21:43:49.371492+0000","last_peered":"2024-09-21T21:43:49.371492+0000","last_clean":"2024-09-21T21:43:49.371492+0000","last_became_active":"2024-09-21T21:43:03.224515+0000","last_became_peered":"2024-09-21T21:43:03.224515+0000","last_unstale":"2024-09-21T21:43:49.371492+0000","last_undegraded":"2024-09-21T21:43:49.371492+0000","last_fullsized":"2024-09-21T21:43:49.371492+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:08:18.537918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"170'880","reported_seq":4751,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.373958+0000","last_change":"2024-09-21T21:43:33.291832+0000","last_active":"2024-09-21T21:43:49.373958+0000","last_peered":"2024-09-21T21:43:49.373958+0000","last_clean":"2024-09-21T21:43:49.373958+0000","last_became_active":"2024-09-21T21:40:37.419944+0000","last_became_peered":"2024-09-21T21:40:37.419944+0000","last_unstale":"2024-09-21T21:43:49.373958+0000","last_undegraded":"2024-09-21T21:43:49.373958+0000","last_fullsized":"2024-09-21T21:43:49.373958+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":880,"log_dups_size":0,"ondisk_log_size":880,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:53:04.189791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020772899999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":5927,"num_read_kb":25257,"num_write":1263,"num_write_kb":11321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":367,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.374070+0000","last_change":"2024-09-21T21:43:03.224909+0000","last_active":"2024-09-21T21:43:49.374070+0000","last_peered":"2024-09-21T21:43:49.374070+0000","last_clean":"2024-09-21T21:43:49.374070+0000","last_became_active":"2024-09-21T21:43:03.224572+0000","last_became_peered":"2024-09-21T21:43:03.224572+0000","last_unstale":"2024-09-21T21:43:49.374070+0000","last_undegraded":"2024-09-21T21:43:49.374070+0000","last_fullsized":"2024-09-21T21:43:49.374070+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:10:47.109676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070669899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"168'637","reported_seq":10497,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.623794+0000","last_change":"2024-09-21T21:43:33.303937+0000","last_active":"2024-09-21T21:43:49.623794+0000","last_peered":"2024-09-21T21:43:49.623794+0000","last_clean":"2024-09-21T21:43:49.623794+0000","last_became_active":"2024-09-21T21:41:25.650379+0000","last_became_peered":"2024-09-21T21:41:25.650379+0000","last_unstale":"2024-09-21T21:43:49.623794+0000","last_undegraded":"2024-09-21T21:43:49.623794+0000","last_fullsized":"2024-09-21T21:43:49.623794+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":637,"log_dups_size":0,"ondisk_log_size":637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:48:59.774761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001695787,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":11334,"num_read_kb":32091,"num_write":515,"num_write_kb":8339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"171'646","reported_seq":11168,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:50.091476+0000","last_change":"2024-09-21T21:43:48.233737+0000","last_active":"2024-09-21T21:43:50.091476+0000","last_peered":"2024-09-21T21:43:50.091476+0000","last_clean":"2024-09-21T21:43:50.091476+0000","last_became_active":"2024-09-21T21:43:48.231457+0000","last_became_peered":"2024-09-21T21:43:48.231457+0000","last_unstale":"2024-09-21T21:43:50.091476+0000","last_undegraded":"2024-09-21T21:43:50.091476+0000","last_fullsized":"2024-09-21T21:43:50.091476+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:10:37.262673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081362800000000005,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":9832,"num_read_kb":35952,"num_write":639,"num_write_kb":9820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":437,"num_bytes_recovered":19367440,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":226,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.623813+0000","last_change":"2024-09-21T21:42:24.516929+0000","last_active":"2024-09-21T21:43:49.623813+0000","last_peered":"2024-09-21T21:43:49.623813+0000","last_clean":"2024-09-21T21:43:49.623813+0000","last_became_active":"2024-09-21T21:42:24.516623+0000","last_became_peered":"2024-09-21T21:42:24.516623+0000","last_unstale":"2024-09-21T21:43:49.623813+0000","last_undegraded":"2024-09-21T21:43:49.623813+0000","last_fullsized":"2024-09-21T21:43:49.623813+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:51:59.592292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000906268,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"171'815","reported_seq":11441,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.373829+0000","last_change":"2024-09-21T21:43:33.290813+0000","last_active":"2024-09-21T21:43:49.373829+0000","last_peered":"2024-09-21T21:43:49.373829+0000","last_clean":"2024-09-21T21:43:49.373829+0000","last_became_active":"2024-09-21T21:40:37.418973+0000","last_became_peered":"2024-09-21T21:40:37.418973+0000","last_unstale":"2024-09-21T21:43:49.373829+0000","last_undegraded":"2024-09-21T21:43:49.373829+0000","last_fullsized":"2024-09-21T21:43:49.373829+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":815,"log_dups_size":0,"ondisk_log_size":815,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:31:26.860045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001131546,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":13425,"num_read_kb":28144,"num_write":602,"num_write_kb":6293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":105,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-21T21:43:48.228311+0000","last_change":"2024-09-21T21:43:44.357753+0000","last_active":"2024-09-21T21:43:48.228311+0000","last_peered":"2024-09-21T21:43:48.228311+0000","last_clean":"2024-09-21T21:43:48.228311+0000","last_became_active":"2024-09-21T21:43:44.357116+0000","last_became_peered":"2024-09-21T21:43:44.357116+0000","last_unstale":"2024-09-21T21:43:48.228311+0000","last_undegraded":"2024-09-21T21:43:48.228311+0000","last_fullsized":"2024-09-21T21:43:48.228311+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:22:20.440763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"168'387","reported_seq":16883,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.373980+0000","last_change":"2024-09-21T21:43:33.321435+0000","last_active":"2024-09-21T21:43:49.373980+0000","last_peered":"2024-09-21T21:43:49.373980+0000","last_clean":"2024-09-21T21:43:49.373980+0000","last_became_active":"2024-09-21T21:43:20.224689+0000","last_became_peered":"2024-09-21T21:43:20.224689+0000","last_unstale":"2024-09-21T21:43:49.373980+0000","last_undegraded":"2024-09-21T21:43:49.373980+0000","last_fullsized":"2024-09-21T21:43:49.373980+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":387,"log_dups_size":0,"ondisk_log_size":387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:15:32.359340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061419100000000004,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":15397,"num_read_kb":48505,"num_write":377,"num_write_kb":4859,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":429,"num_bytes_recovered":6598898,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":227,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.371346+0000","last_change":"2024-09-21T21:41:49.265465+0000","last_active":"2024-09-21T21:43:49.371346+0000","last_peered":"2024-09-21T21:43:49.371346+0000","last_clean":"2024-09-21T21:43:49.371346+0000","last_became_active":"2024-09-21T21:41:49.265182+0000","last_became_peered":"2024-09-21T21:41:49.265182+0000","last_unstale":"2024-09-21T21:43:49.371346+0000","last_undegraded":"2024-09-21T21:43:49.371346+0000","last_fullsized":"2024-09-21T21:43:49.371346+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:19:17.973195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"171'773","reported_seq":10302,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.374490+0000","last_change":"2024-09-21T21:43:33.293418+0000","last_active":"2024-09-21T21:43:49.374490+0000","last_peered":"2024-09-21T21:43:49.374490+0000","last_clean":"2024-09-21T21:43:49.374490+0000","last_became_active":"2024-09-21T21:40:37.421655+0000","last_became_peered":"2024-09-21T21:40:37.421655+0000","last_unstale":"2024-09-21T21:43:49.374490+0000","last_undegraded":"2024-09-21T21:43:49.374490+0000","last_fullsized":"2024-09-21T21:43:49.374490+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":773,"log_dups_size":0,"ondisk_log_size":773,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:43:34.040974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031354970000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":12713,"num_read_kb":34378,"num_write":754,"num_write_kb":7498,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":324,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.623702+0000","last_change":"2024-09-21T21:42:24.518810+0000","last_active":"2024-09-21T21:43:49.623702+0000","last_peered":"2024-09-21T21:43:49.623702+0000","last_clean":"2024-09-21T21:43:49.623702+0000","last_became_active":"2024-09-21T21:42:24.518461+0000","last_became_peered":"2024-09-21T21:42:24.518461+0000","last_unstale":"2024-09-21T21:43:49.623702+0000","last_undegraded":"2024-09-21T21:43:49.623702+0000","last_fullsized":"2024-09-21T21:43:49.623702+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:17:47.593497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036803999999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":433,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.371463+0000","last_change":"2024-09-21T21:43:06.231540+0000","last_active":"2024-09-21T21:43:49.371463+0000","last_peered":"2024-09-21T21:43:49.371463+0000","last_clean":"2024-09-21T21:43:49.371463+0000","last_became_active":"2024-09-21T21:43:06.231241+0000","last_became_peered":"2024-09-21T21:43:06.231241+0000","last_unstale":"2024-09-21T21:43:49.371463+0000","last_undegraded":"2024-09-21T21:43:49.371463+0000","last_fullsized":"2024-09-21T21:43:49.371463+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:18.340430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:18.340430+0000","last_clean_scrub_stamp":"2024-09-21T21:40:18.340430+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:50:08.187886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"170'374","reported_seq":6270,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-21T21:43:48.228114+0000","last_change":"2024-09-21T21:43:44.358111+0000","last_active":"2024-09-21T21:43:48.228114+0000","last_peered":"2024-09-21T21:43:48.228114+0000","last_clean":"2024-09-21T21:43:48.228114+0000","last_became_active":"2024-09-21T21:43:44.356024+0000","last_became_peered":"2024-09-21T21:43:44.356024+0000","last_unstale":"2024-09-21T21:43:48.228114+0000","last_undegraded":"2024-09-21T21:43:48.228114+0000","last_fullsized":"2024-09-21T21:43:48.228114+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":374,"log_dups_size":0,"ondisk_log_size":374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:17:43.127988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094280000000000004,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":5268,"num_read_kb":25972,"num_write":367,"num_write_kb":7278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":172,"num_bytes_recovered":2229995,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"170'554","reported_seq":8626,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:50.091421+0000","last_change":"2024-09-21T21:43:48.233080+0000","last_active":"2024-09-21T21:43:50.091421+0000","last_peered":"2024-09-21T21:43:50.091421+0000","last_clean":"2024-09-21T21:43:50.091421+0000","last_became_active":"2024-09-21T21:43:48.230703+0000","last_became_peered":"2024-09-21T21:43:48.230703+0000","last_unstale":"2024-09-21T21:43:50.091421+0000","last_undegraded":"2024-09-21T21:43:50.091421+0000","last_fullsized":"2024-09-21T21:43:50.091421+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":554,"log_dups_size":0,"ondisk_log_size":554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:39:12.422453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071450699999999997,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":8813,"num_read_kb":47394,"num_write":525,"num_write_kb":9434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":241,"num_bytes_recovered":7579476,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"171'1111","reported_seq":11670,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.371409+0000","last_change":"2024-09-21T21:43:33.292652+0000","last_active":"2024-09-21T21:43:49.371409+0000","last_peered":"2024-09-21T21:43:49.371409+0000","last_clean":"2024-09-21T21:43:49.371409+0000","last_became_active":"2024-09-21T21:43:19.224644+0000","last_became_peered":"2024-09-21T21:43:19.224644+0000","last_unstale":"2024-09-21T21:43:49.371409+0000","last_undegraded":"2024-09-21T21:43:49.371409+0000","last_fullsized":"2024-09-21T21:43:49.371409+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":1111,"log_dups_size":0,"ondisk_log_size":1111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:19:31.244395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001226878,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":5,"num_read":13231,"num_read_kb":49174,"num_write":1280,"num_write_kb":10665,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":398,"num_bytes_recovered":16632560,"num_keys_recovered":144,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"170'777","reported_seq":10308,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.374283+0000","last_change":"2024-09-21T21:43:33.292265+0000","last_active":"2024-09-21T21:43:49.374283+0000","last_peered":"2024-09-21T21:43:49.374283+0000","last_clean":"2024-09-21T21:43:49.374283+0000","last_became_active":"2024-09-21T21:43:09.227569+0000","last_became_peered":"2024-09-21T21:43:09.227569+0000","last_unstale":"2024-09-21T21:43:49.374283+0000","last_undegraded":"2024-09-21T21:43:49.374283+0000","last_fullsized":"2024-09-21T21:43:49.374283+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":777,"log_dups_size":0,"ondisk_log_size":777,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:20:34.944827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024712610000000002,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":10244,"num_read_kb":45368,"num_write":1134,"num_write_kb":14646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":331,"num_bytes_recovered":8931668,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"168'1625","reported_seq":14031,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.623830+0000","last_change":"2024-09-21T21:43:33.304566+0000","last_active":"2024-09-21T21:43:49.623830+0000","last_peered":"2024-09-21T21:43:49.623830+0000","last_clean":"2024-09-21T21:43:49.623830+0000","last_became_active":"2024-09-21T21:43:07.230148+0000","last_became_peered":"2024-09-21T21:43:07.230148+0000","last_unstale":"2024-09-21T21:43:49.623830+0000","last_undegraded":"2024-09-21T21:43:49.623830+0000","last_fullsized":"2024-09-21T21:43:49.623830+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":1625,"log_dups_size":0,"ondisk_log_size":1625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:42:46.324537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021897029999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":18573,"num_read_kb":61318,"num_write":2350,"num_write_kb":11538,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":225,"num_bytes_recovered":7091105,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"171'1447","reported_seq":17761,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.371616+0000","last_change":"2024-09-21T21:43:33.293137+0000","last_active":"2024-09-21T21:43:49.371616+0000","last_peered":"2024-09-21T21:43:49.371616+0000","last_clean":"2024-09-21T21:43:49.371616+0000","last_became_active":"2024-09-21T21:41:52.214022+0000","last_became_peered":"2024-09-21T21:41:52.214022+0000","last_unstale":"2024-09-21T21:43:49.371616+0000","last_undegraded":"2024-09-21T21:43:49.371616+0000","last_fullsized":"2024-09-21T21:43:49.371616+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":1447,"log_dups_size":0,"ondisk_log_size":1447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:42:14.727563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016156199999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":21216,"num_read_kb":45587,"num_write":1846,"num_write_kb":14797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":149,"num_bytes_recovered":3462638,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"168'454","reported_seq":7909,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.623841+0000","last_change":"2024-09-21T21:43:33.304631+0000","last_active":"2024-09-21T21:43:49.623841+0000","last_peered":"2024-09-21T21:43:49.623841+0000","last_clean":"2024-09-21T21:43:49.623841+0000","last_became_active":"2024-09-21T21:43:18.223755+0000","last_became_peered":"2024-09-21T21:43:18.223755+0000","last_unstale":"2024-09-21T21:43:49.623841+0000","last_undegraded":"2024-09-21T21:43:49.623841+0000","last_fullsized":"2024-09-21T21:43:49.623841+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":454,"log_dups_size":0,"ondisk_log_size":454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:43:32.324890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018835950000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":6571,"num_read_kb":34865,"num_write":469,"num_write_kb":6862,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":238,"num_bytes_recovered":4045989,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"169'741","reported_seq":17136,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.623910+0000","last_change":"2024-09-21T21:43:33.303579+0000","last_active":"2024-09-21T21:43:49.623910+0000","last_peered":"2024-09-21T21:43:49.623910+0000","last_clean":"2024-09-21T21:43:49.623910+0000","last_became_active":"2024-09-21T21:41:52.212206+0000","last_became_peered":"2024-09-21T21:41:52.212206+0000","last_unstale":"2024-09-21T21:43:49.623910+0000","last_undegraded":"2024-09-21T21:43:49.623910+0000","last_fullsized":"2024-09-21T21:43:49.623910+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":741,"log_dups_size":0,"ondisk_log_size":741,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:10:57.220429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013653929999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":18315,"num_read_kb":42344,"num_write":672,"num_write_kb":9505,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":2967886,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"167'408","reported_seq":14252,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.623662+0000","last_change":"2024-09-21T21:43:33.302904+0000","last_active":"2024-09-21T21:43:49.623662+0000","last_peered":"2024-09-21T21:43:49.623662+0000","last_clean":"2024-09-21T21:43:49.623662+0000","last_became_active":"2024-09-21T21:43:06.233222+0000","last_became_peered":"2024-09-21T21:43:06.233222+0000","last_unstale":"2024-09-21T21:43:49.623662+0000","last_undegraded":"2024-09-21T21:43:49.623662+0000","last_fullsized":"2024-09-21T21:43:49.623662+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:54:36.083839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085383799999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":12974,"num_read_kb":55237,"num_write":424,"num_write_kb":11231,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":205,"num_bytes_recovered":6455839,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"168'641","reported_seq":8507,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.371363+0000","last_change":"2024-09-21T21:43:33.292136+0000","last_active":"2024-09-21T21:43:49.371363+0000","last_peered":"2024-09-21T21:43:49.371363+0000","last_clean":"2024-09-21T21:43:49.371363+0000","last_became_active":"2024-09-21T21:43:20.224840+0000","last_became_peered":"2024-09-21T21:43:20.224840+0000","last_unstale":"2024-09-21T21:43:49.371363+0000","last_undegraded":"2024-09-21T21:43:49.371363+0000","last_fullsized":"2024-09-21T21:43:49.371363+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":641,"log_dups_size":0,"ondisk_log_size":641,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:13:32.197836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091468500000000004,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":7916,"num_read_kb":33911,"num_write":730,"num_write_kb":6868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":492,"num_bytes_recovered":3936953,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"170'791","reported_seq":19355,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.374375+0000","last_change":"2024-09-21T21:43:33.321430+0000","last_active":"2024-09-21T21:43:49.374375+0000","last_peered":"2024-09-21T21:43:49.374375+0000","last_clean":"2024-09-21T21:43:49.374375+0000","last_became_active":"2024-09-21T21:43:07.229497+0000","last_became_peered":"2024-09-21T21:43:07.229497+0000","last_unstale":"2024-09-21T21:43:49.374375+0000","last_undegraded":"2024-09-21T21:43:49.374375+0000","last_fullsized":"2024-09-21T21:43:49.374375+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":791,"log_dups_size":0,"ondisk_log_size":791,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:10:05.347980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061499099999999995,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":20795,"num_read_kb":71790,"num_write":798,"num_write_kb":9984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":352,"num_bytes_recovered":16923683,"num_keys_recovered":92,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"168'477","reported_seq":10652,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.371417+0000","last_change":"2024-09-21T21:43:33.292617+0000","last_active":"2024-09-21T21:43:49.371417+0000","last_peered":"2024-09-21T21:43:49.371417+0000","last_clean":"2024-09-21T21:43:49.371417+0000","last_became_active":"2024-09-21T21:41:52.209710+0000","last_became_peered":"2024-09-21T21:41:52.209710+0000","last_unstale":"2024-09-21T21:43:49.371417+0000","last_undegraded":"2024-09-21T21:43:49.371417+0000","last_fullsized":"2024-09-21T21:43:49.371417+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":477,"log_dups_size":0,"ondisk_log_size":477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:20:59.585304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013513189999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":9647,"num_read_kb":76927,"num_write":508,"num_write_kb":12558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":146,"num_bytes_recovered":17387622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":241,"num_object_clones":0,"num_object_copies":482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":241,"num_whiteouts":237,"num_read":408853,"num_read_kb":1471230,"num_write":27874,"num_write_kb":319707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7824,"num_bytes_recovered":220201937,"num_keys_recovered":844,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24446,"ondisk_log_size":24446,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":102090,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":4,"up_from":13,"seq":55834574894,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55556,"kb_used_data":2160,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316284,"statfs":{"total":96636764160,"available":96579874816,"internally_reserved":0,"allocated":2211840,"data_stored":928386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574894,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87536,"kb_used_data":2204,"kb_used_omap":1,"kb_used_meta":85310,"kb_avail":94284304,"statfs":{"total":96636764160,"available":96547127296,"internally_reserved":0,"allocated":2256896,"data_stored":1518735,"data_compressed":34030,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1593,"internal_metadata":87357895},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":12,"seq":51539607598,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73604,"kb_used_data":2160,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298236,"statfs":{"total":96636764160,"available":96561393664,"internally_reserved":0,"allocated":2211840,"data_stored":928386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607598,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55576,"kb_used_data":2176,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316264,"statfs":{"total":96636764160,"available":96579854336,"internally_reserved":0,"allocated":2228224,"data_stored":933375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607599,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47296,"kb_used_data":2216,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324544,"statfs":{"total":96636764160,"available":96588333056,"internally_reserved":0,"allocated":2269184,"data_stored":1523705,"data_compressed":34030,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640302,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47232,"kb_used_data":2156,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324608,"statfs":{"total":96636764160,"available":96588398592,"internally_reserved":0,"allocated":2207744,"data_stored":928367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34030,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34030,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34030,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T21:43:52.139 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T21:43:52.139 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T21:43:52.415 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:43:52.416 INFO:teuthology.orchestra.run.smithi081.stderr:dumped all 2024-09-21T21:43:52.427 INFO:teuthology.orchestra.run.smithi081.stdout:{"pg_ready":true,"pg_map":{"version":288,"stamp":"2024-09-21T21:43:52.325202+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":245,"num_object_clones":0,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":237,"num_read":408969,"num_read_kb":1471330,"num_write":28035,"num_write_kb":322125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7832,"num_bytes_recovered":222563409,"num_keys_recovered":844,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24566,"ondisk_log_size":24566,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":366812,"kb_used_data":13084,"kb_used_omap":9,"kb_used_meta":353590,"kb_avail":565864228,"statfs":{"total":579820584960,"available":579444969472,"internally_reserved":0,"allocated":13398016,"data_stored":6765924,"data_compressed":68060,"data_compressed_allocated":90112,"data_compressed_original":1179648,"omap_allocated":9568,"internal_metadata":362076832},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":-11,"num_read_kb":-11,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":125,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.966914"},"pg_stats":[{"pgid":"3.1f","version":"171'591","reported_seq":7658,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.374605+0000","last_change":"2024-09-21T21:43:33.322591+0000","last_active":"2024-09-21T21:43:49.374605+0000","last_peered":"2024-09-21T21:43:49.374605+0000","last_clean":"2024-09-21T21:43:49.374605+0000","last_became_active":"2024-09-21T21:43:08.230509+0000","last_became_peered":"2024-09-21T21:43:08.230509+0000","last_unstale":"2024-09-21T21:43:49.374605+0000","last_undegraded":"2024-09-21T21:43:49.374605+0000","last_fullsized":"2024-09-21T21:43:49.374605+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":591,"log_dups_size":0,"ondisk_log_size":591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:50:51.925851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013233089999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":8384,"num_read_kb":24764,"num_write":554,"num_write_kb":8557,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":293,"num_bytes_recovered":3779776,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"168'1013","reported_seq":15328,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:50.272658+0000","last_change":"2024-09-21T21:43:44.357820+0000","last_active":"2024-09-21T21:43:50.272658+0000","last_peered":"2024-09-21T21:43:50.272658+0000","last_clean":"2024-09-21T21:43:50.272658+0000","last_became_active":"2024-09-21T21:43:44.356029+0000","last_became_peered":"2024-09-21T21:43:44.356029+0000","last_unstale":"2024-09-21T21:43:50.272658+0000","last_undegraded":"2024-09-21T21:43:50.272658+0000","last_fullsized":"2024-09-21T21:43:50.272658+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":1013,"log_dups_size":0,"ondisk_log_size":1013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:20:13.278523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001190569,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":16219,"num_read_kb":73109,"num_write":1473,"num_write_kb":18401,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":14291598,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"168'985","reported_seq":10752,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.623764+0000","last_change":"2024-09-21T21:43:33.302894+0000","last_active":"2024-09-21T21:43:49.623764+0000","last_peered":"2024-09-21T21:43:49.623764+0000","last_clean":"2024-09-21T21:43:49.623764+0000","last_became_active":"2024-09-21T21:43:20.225483+0000","last_became_peered":"2024-09-21T21:43:20.225483+0000","last_unstale":"2024-09-21T21:43:49.623764+0000","last_undegraded":"2024-09-21T21:43:49.623764+0000","last_fullsized":"2024-09-21T21:43:49.623764+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":985,"log_dups_size":0,"ondisk_log_size":985,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:13:56.033873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077268300000000003,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":13110,"num_read_kb":43443,"num_write":1158,"num_write_kb":7907,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":4009384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"171'884","reported_seq":13301,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.623722+0000","last_change":"2024-09-21T21:43:33.304149+0000","last_active":"2024-09-21T21:43:49.623722+0000","last_peered":"2024-09-21T21:43:49.623722+0000","last_clean":"2024-09-21T21:43:49.623722+0000","last_became_active":"2024-09-21T21:43:19.225435+0000","last_became_peered":"2024-09-21T21:43:19.225435+0000","last_unstale":"2024-09-21T21:43:49.623722+0000","last_undegraded":"2024-09-21T21:43:49.623722+0000","last_fullsized":"2024-09-21T21:43:49.623722+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":884,"log_dups_size":0,"ondisk_log_size":884,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:45:28.185309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001856124,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":15417,"num_read_kb":35250,"num_write":925,"num_write_kb":7812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":178,"num_bytes_recovered":1721983,"num_keys_recovered":148,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"171'737","reported_seq":9064,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.374605+0000","last_change":"2024-09-21T21:43:33.322211+0000","last_active":"2024-09-21T21:43:49.374605+0000","last_peered":"2024-09-21T21:43:49.374605+0000","last_clean":"2024-09-21T21:43:49.374605+0000","last_became_active":"2024-09-21T21:43:19.225304+0000","last_became_peered":"2024-09-21T21:43:19.225304+0000","last_unstale":"2024-09-21T21:43:49.374605+0000","last_undegraded":"2024-09-21T21:43:49.374605+0000","last_fullsized":"2024-09-21T21:43:49.374605+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":737,"log_dups_size":0,"ondisk_log_size":737,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:07:04.863691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001146085,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":8621,"num_read_kb":37221,"num_write":843,"num_write_kb":4971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":547,"num_bytes_recovered":1792422,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"170'618","reported_seq":25431,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:50.091426+0000","last_change":"2024-09-21T21:43:48.232831+0000","last_active":"2024-09-21T21:43:50.091426+0000","last_peered":"2024-09-21T21:43:50.091426+0000","last_clean":"2024-09-21T21:43:50.091426+0000","last_became_active":"2024-09-21T21:43:48.230767+0000","last_became_peered":"2024-09-21T21:43:48.230767+0000","last_unstale":"2024-09-21T21:43:50.091426+0000","last_undegraded":"2024-09-21T21:43:50.091426+0000","last_fullsized":"2024-09-21T21:43:50.091426+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":618,"log_dups_size":0,"ondisk_log_size":618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:47:54.923891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0044588529999999996,"stat_sum":{"num_bytes":19,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":25138,"num_read_kb":59201,"num_write":578,"num_write_kb":7782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":203,"num_bytes_recovered":4464880,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"170'860","reported_seq":14187,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.373846+0000","last_change":"2024-09-21T21:43:33.291493+0000","last_active":"2024-09-21T21:43:49.373846+0000","last_peered":"2024-09-21T21:43:49.373846+0000","last_clean":"2024-09-21T21:43:49.373846+0000","last_became_active":"2024-09-21T21:43:20.224845+0000","last_became_peered":"2024-09-21T21:43:20.224845+0000","last_unstale":"2024-09-21T21:43:49.373846+0000","last_undegraded":"2024-09-21T21:43:49.373846+0000","last_fullsized":"2024-09-21T21:43:49.373846+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":860,"log_dups_size":0,"ondisk_log_size":860,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:55:13.229277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018111119999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":16015,"num_read_kb":61704,"num_write":858,"num_write_kb":15379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":9899814,"num_keys_recovered":59,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"168'335","reported_seq":6957,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:50.272765+0000","last_change":"2024-09-21T21:43:44.358159+0000","last_active":"2024-09-21T21:43:50.272765+0000","last_peered":"2024-09-21T21:43:50.272765+0000","last_clean":"2024-09-21T21:43:50.272765+0000","last_became_active":"2024-09-21T21:43:44.356203+0000","last_became_peered":"2024-09-21T21:43:44.356203+0000","last_unstale":"2024-09-21T21:43:50.272765+0000","last_undegraded":"2024-09-21T21:43:50.272765+0000","last_fullsized":"2024-09-21T21:43:50.272765+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:53:07.504981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012910630000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":6161,"num_read_kb":33023,"num_write":325,"num_write_kb":8639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":5854224,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"171'497","reported_seq":6419,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.374509+0000","last_change":"2024-09-21T21:43:33.322741+0000","last_active":"2024-09-21T21:43:49.374509+0000","last_peered":"2024-09-21T21:43:49.374509+0000","last_clean":"2024-09-21T21:43:49.374509+0000","last_became_active":"2024-09-21T21:43:07.232590+0000","last_became_peered":"2024-09-21T21:43:07.232590+0000","last_unstale":"2024-09-21T21:43:49.374509+0000","last_undegraded":"2024-09-21T21:43:49.374509+0000","last_fullsized":"2024-09-21T21:43:49.374509+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":497,"log_dups_size":0,"ondisk_log_size":497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:04:00.513414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001467229,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":6465,"num_read_kb":25955,"num_write":521,"num_write_kb":9178,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":203,"num_bytes_recovered":4149457,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"171'1238","reported_seq":9868,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.623839+0000","last_change":"2024-09-21T21:43:33.305145+0000","last_active":"2024-09-21T21:43:49.623839+0000","last_peered":"2024-09-21T21:43:49.623839+0000","last_clean":"2024-09-21T21:43:49.623839+0000","last_became_active":"2024-09-21T21:41:53.214270+0000","last_became_peered":"2024-09-21T21:41:53.214270+0000","last_unstale":"2024-09-21T21:43:49.623839+0000","last_undegraded":"2024-09-21T21:43:49.623839+0000","last_fullsized":"2024-09-21T21:43:49.623839+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":1238,"log_dups_size":0,"ondisk_log_size":1238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:51:58.701104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022124750000000002,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":13255,"num_read_kb":44399,"num_write":1796,"num_write_kb":12840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":148,"num_bytes_recovered":4869268,"num_keys_recovered":47,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"170'744","reported_seq":9490,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.371529+0000","last_change":"2024-09-21T21:43:33.293155+0000","last_active":"2024-09-21T21:43:49.371529+0000","last_peered":"2024-09-21T21:43:49.371529+0000","last_clean":"2024-09-21T21:43:49.371529+0000","last_became_active":"2024-09-21T21:43:07.228504+0000","last_became_peered":"2024-09-21T21:43:07.228504+0000","last_unstale":"2024-09-21T21:43:49.371529+0000","last_undegraded":"2024-09-21T21:43:49.371529+0000","last_fullsized":"2024-09-21T21:43:49.371529+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":744,"log_dups_size":0,"ondisk_log_size":744,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:55:34.005752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001650253,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":9201,"num_read_kb":56862,"num_write":979,"num_write_kb":7172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":520,"num_bytes_recovered":10162533,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"167'626","reported_seq":21896,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.371520+0000","last_change":"2024-09-21T21:43:48.230374+0000","last_active":"2024-09-21T21:43:49.371520+0000","last_peered":"2024-09-21T21:43:49.371520+0000","last_clean":"2024-09-21T21:43:49.371520+0000","last_became_active":"2024-09-21T21:43:48.228528+0000","last_became_peered":"2024-09-21T21:43:48.228528+0000","last_unstale":"2024-09-21T21:43:49.371520+0000","last_undegraded":"2024-09-21T21:43:49.371520+0000","last_fullsized":"2024-09-21T21:43:49.371520+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":626,"log_dups_size":0,"ondisk_log_size":626,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:52:06.587738+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.00063765000000000002,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":22083,"num_read_kb":75165,"num_write":513,"num_write_kb":10980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":602,"num_bytes_recovered":22337258,"num_keys_recovered":84,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":236,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.371668+0000","last_change":"2024-09-21T21:43:15.316851+0000","last_active":"2024-09-21T21:43:49.371668+0000","last_peered":"2024-09-21T21:43:49.371668+0000","last_clean":"2024-09-21T21:43:49.371668+0000","last_became_active":"2024-09-21T21:43:15.316533+0000","last_became_peered":"2024-09-21T21:43:15.316533+0000","last_unstale":"2024-09-21T21:43:49.371668+0000","last_undegraded":"2024-09-21T21:43:49.371668+0000","last_fullsized":"2024-09-21T21:43:49.371668+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:00:30.781100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"171'1225","reported_seq":14273,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.374367+0000","last_change":"2024-09-21T21:43:33.321874+0000","last_active":"2024-09-21T21:43:49.374367+0000","last_peered":"2024-09-21T21:43:49.374367+0000","last_clean":"2024-09-21T21:43:49.374367+0000","last_became_active":"2024-09-21T21:43:09.230849+0000","last_became_peered":"2024-09-21T21:43:09.230849+0000","last_unstale":"2024-09-21T21:43:49.374367+0000","last_undegraded":"2024-09-21T21:43:49.374367+0000","last_fullsized":"2024-09-21T21:43:49.374367+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":1225,"log_dups_size":0,"ondisk_log_size":1225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:50:50.908667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096363700000000004,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":15823,"num_read_kb":66323,"num_write":1574,"num_write_kb":15987,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":349,"num_bytes_recovered":9257588,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":362,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.374023+0000","last_change":"2024-09-21T21:41:49.266534+0000","last_active":"2024-09-21T21:43:49.374023+0000","last_peered":"2024-09-21T21:43:49.374023+0000","last_clean":"2024-09-21T21:43:49.374023+0000","last_became_active":"2024-09-21T21:41:49.266257+0000","last_became_peered":"2024-09-21T21:41:49.266257+0000","last_unstale":"2024-09-21T21:43:49.374023+0000","last_undegraded":"2024-09-21T21:43:49.374023+0000","last_fullsized":"2024-09-21T21:43:49.374023+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:38:31.063193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062280799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"170'555","reported_seq":8888,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:50.272682+0000","last_change":"2024-09-21T21:43:44.357970+0000","last_active":"2024-09-21T21:43:50.272682+0000","last_peered":"2024-09-21T21:43:50.272682+0000","last_clean":"2024-09-21T21:43:50.272682+0000","last_became_active":"2024-09-21T21:43:44.356543+0000","last_became_peered":"2024-09-21T21:43:44.356543+0000","last_unstale":"2024-09-21T21:43:50.272682+0000","last_undegraded":"2024-09-21T21:43:50.272682+0000","last_fullsized":"2024-09-21T21:43:50.272682+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":555,"log_dups_size":0,"ondisk_log_size":555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:10:07.726600+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.00099396100000000002,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":10770,"num_read_kb":40597,"num_write":524,"num_write_kb":10606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":241,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.371492+0000","last_change":"2024-09-21T21:43:03.224822+0000","last_active":"2024-09-21T21:43:49.371492+0000","last_peered":"2024-09-21T21:43:49.371492+0000","last_clean":"2024-09-21T21:43:49.371492+0000","last_became_active":"2024-09-21T21:43:03.224515+0000","last_became_peered":"2024-09-21T21:43:03.224515+0000","last_unstale":"2024-09-21T21:43:49.371492+0000","last_undegraded":"2024-09-21T21:43:49.371492+0000","last_fullsized":"2024-09-21T21:43:49.371492+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:08:18.537918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"170'880","reported_seq":4751,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.373958+0000","last_change":"2024-09-21T21:43:33.291832+0000","last_active":"2024-09-21T21:43:49.373958+0000","last_peered":"2024-09-21T21:43:49.373958+0000","last_clean":"2024-09-21T21:43:49.373958+0000","last_became_active":"2024-09-21T21:40:37.419944+0000","last_became_peered":"2024-09-21T21:40:37.419944+0000","last_unstale":"2024-09-21T21:43:49.373958+0000","last_undegraded":"2024-09-21T21:43:49.373958+0000","last_fullsized":"2024-09-21T21:43:49.373958+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":880,"log_dups_size":0,"ondisk_log_size":880,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:53:04.189791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020772899999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":5927,"num_read_kb":25257,"num_write":1263,"num_write_kb":11321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":367,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.374070+0000","last_change":"2024-09-21T21:43:03.224909+0000","last_active":"2024-09-21T21:43:49.374070+0000","last_peered":"2024-09-21T21:43:49.374070+0000","last_clean":"2024-09-21T21:43:49.374070+0000","last_became_active":"2024-09-21T21:43:03.224572+0000","last_became_peered":"2024-09-21T21:43:03.224572+0000","last_unstale":"2024-09-21T21:43:49.374070+0000","last_undegraded":"2024-09-21T21:43:49.374070+0000","last_fullsized":"2024-09-21T21:43:49.374070+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:10:47.109676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070669899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"168'637","reported_seq":10497,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.623794+0000","last_change":"2024-09-21T21:43:33.303937+0000","last_active":"2024-09-21T21:43:49.623794+0000","last_peered":"2024-09-21T21:43:49.623794+0000","last_clean":"2024-09-21T21:43:49.623794+0000","last_became_active":"2024-09-21T21:41:25.650379+0000","last_became_peered":"2024-09-21T21:41:25.650379+0000","last_unstale":"2024-09-21T21:43:49.623794+0000","last_undegraded":"2024-09-21T21:43:49.623794+0000","last_fullsized":"2024-09-21T21:43:49.623794+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":637,"log_dups_size":0,"ondisk_log_size":637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:48:59.774761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001695787,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":11334,"num_read_kb":32091,"num_write":515,"num_write_kb":8339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"171'646","reported_seq":11168,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:50.091476+0000","last_change":"2024-09-21T21:43:48.233737+0000","last_active":"2024-09-21T21:43:50.091476+0000","last_peered":"2024-09-21T21:43:50.091476+0000","last_clean":"2024-09-21T21:43:50.091476+0000","last_became_active":"2024-09-21T21:43:48.231457+0000","last_became_peered":"2024-09-21T21:43:48.231457+0000","last_unstale":"2024-09-21T21:43:50.091476+0000","last_undegraded":"2024-09-21T21:43:50.091476+0000","last_fullsized":"2024-09-21T21:43:50.091476+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:10:37.262673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081362800000000005,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":9832,"num_read_kb":35952,"num_write":639,"num_write_kb":9820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":437,"num_bytes_recovered":19367440,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":226,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.623813+0000","last_change":"2024-09-21T21:42:24.516929+0000","last_active":"2024-09-21T21:43:49.623813+0000","last_peered":"2024-09-21T21:43:49.623813+0000","last_clean":"2024-09-21T21:43:49.623813+0000","last_became_active":"2024-09-21T21:42:24.516623+0000","last_became_peered":"2024-09-21T21:42:24.516623+0000","last_unstale":"2024-09-21T21:43:49.623813+0000","last_undegraded":"2024-09-21T21:43:49.623813+0000","last_fullsized":"2024-09-21T21:43:49.623813+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:51:59.592292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000906268,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"171'815","reported_seq":11441,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.373829+0000","last_change":"2024-09-21T21:43:33.290813+0000","last_active":"2024-09-21T21:43:49.373829+0000","last_peered":"2024-09-21T21:43:49.373829+0000","last_clean":"2024-09-21T21:43:49.373829+0000","last_became_active":"2024-09-21T21:40:37.418973+0000","last_became_peered":"2024-09-21T21:40:37.418973+0000","last_unstale":"2024-09-21T21:43:49.373829+0000","last_undegraded":"2024-09-21T21:43:49.373829+0000","last_fullsized":"2024-09-21T21:43:49.373829+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":815,"log_dups_size":0,"ondisk_log_size":815,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:31:26.860045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001131546,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":13425,"num_read_kb":28144,"num_write":602,"num_write_kb":6293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":107,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:50.272528+0000","last_change":"2024-09-21T21:43:44.357753+0000","last_active":"2024-09-21T21:43:50.272528+0000","last_peered":"2024-09-21T21:43:50.272528+0000","last_clean":"2024-09-21T21:43:50.272528+0000","last_became_active":"2024-09-21T21:43:44.357116+0000","last_became_peered":"2024-09-21T21:43:44.357116+0000","last_unstale":"2024-09-21T21:43:50.272528+0000","last_undegraded":"2024-09-21T21:43:50.272528+0000","last_fullsized":"2024-09-21T21:43:50.272528+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:22:20.440763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"168'387","reported_seq":16883,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.373980+0000","last_change":"2024-09-21T21:43:33.321435+0000","last_active":"2024-09-21T21:43:49.373980+0000","last_peered":"2024-09-21T21:43:49.373980+0000","last_clean":"2024-09-21T21:43:49.373980+0000","last_became_active":"2024-09-21T21:43:20.224689+0000","last_became_peered":"2024-09-21T21:43:20.224689+0000","last_unstale":"2024-09-21T21:43:49.373980+0000","last_undegraded":"2024-09-21T21:43:49.373980+0000","last_fullsized":"2024-09-21T21:43:49.373980+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":387,"log_dups_size":0,"ondisk_log_size":387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:15:32.359340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061419100000000004,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":15397,"num_read_kb":48505,"num_write":377,"num_write_kb":4859,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":429,"num_bytes_recovered":6598898,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":227,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.371346+0000","last_change":"2024-09-21T21:41:49.265465+0000","last_active":"2024-09-21T21:43:49.371346+0000","last_peered":"2024-09-21T21:43:49.371346+0000","last_clean":"2024-09-21T21:43:49.371346+0000","last_became_active":"2024-09-21T21:41:49.265182+0000","last_became_peered":"2024-09-21T21:41:49.265182+0000","last_unstale":"2024-09-21T21:43:49.371346+0000","last_undegraded":"2024-09-21T21:43:49.371346+0000","last_fullsized":"2024-09-21T21:43:49.371346+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:19:17.973195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"171'773","reported_seq":10302,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.374490+0000","last_change":"2024-09-21T21:43:33.293418+0000","last_active":"2024-09-21T21:43:49.374490+0000","last_peered":"2024-09-21T21:43:49.374490+0000","last_clean":"2024-09-21T21:43:49.374490+0000","last_became_active":"2024-09-21T21:40:37.421655+0000","last_became_peered":"2024-09-21T21:40:37.421655+0000","last_unstale":"2024-09-21T21:43:49.374490+0000","last_undegraded":"2024-09-21T21:43:49.374490+0000","last_fullsized":"2024-09-21T21:43:49.374490+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":773,"log_dups_size":0,"ondisk_log_size":773,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:43:34.040974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031354970000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":12713,"num_read_kb":34378,"num_write":754,"num_write_kb":7498,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":324,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.623702+0000","last_change":"2024-09-21T21:42:24.518810+0000","last_active":"2024-09-21T21:43:49.623702+0000","last_peered":"2024-09-21T21:43:49.623702+0000","last_clean":"2024-09-21T21:43:49.623702+0000","last_became_active":"2024-09-21T21:42:24.518461+0000","last_became_peered":"2024-09-21T21:42:24.518461+0000","last_unstale":"2024-09-21T21:43:49.623702+0000","last_undegraded":"2024-09-21T21:43:49.623702+0000","last_fullsized":"2024-09-21T21:43:49.623702+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:17:47.593497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036803999999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":433,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.371463+0000","last_change":"2024-09-21T21:43:06.231540+0000","last_active":"2024-09-21T21:43:49.371463+0000","last_peered":"2024-09-21T21:43:49.371463+0000","last_clean":"2024-09-21T21:43:49.371463+0000","last_became_active":"2024-09-21T21:43:06.231241+0000","last_became_peered":"2024-09-21T21:43:06.231241+0000","last_unstale":"2024-09-21T21:43:49.371463+0000","last_undegraded":"2024-09-21T21:43:49.371463+0000","last_fullsized":"2024-09-21T21:43:49.371463+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:18.340430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:18.340430+0000","last_clean_scrub_stamp":"2024-09-21T21:40:18.340430+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:50:08.187886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"170'374","reported_seq":6272,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:50.272852+0000","last_change":"2024-09-21T21:43:44.358111+0000","last_active":"2024-09-21T21:43:50.272852+0000","last_peered":"2024-09-21T21:43:50.272852+0000","last_clean":"2024-09-21T21:43:50.272852+0000","last_became_active":"2024-09-21T21:43:44.356024+0000","last_became_peered":"2024-09-21T21:43:44.356024+0000","last_unstale":"2024-09-21T21:43:50.272852+0000","last_undegraded":"2024-09-21T21:43:50.272852+0000","last_fullsized":"2024-09-21T21:43:50.272852+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":374,"log_dups_size":0,"ondisk_log_size":374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:17:43.127988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094280000000000004,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":5268,"num_read_kb":25972,"num_write":367,"num_write_kb":7278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":172,"num_bytes_recovered":2229995,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"170'554","reported_seq":8626,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:50.091421+0000","last_change":"2024-09-21T21:43:48.233080+0000","last_active":"2024-09-21T21:43:50.091421+0000","last_peered":"2024-09-21T21:43:50.091421+0000","last_clean":"2024-09-21T21:43:50.091421+0000","last_became_active":"2024-09-21T21:43:48.230703+0000","last_became_peered":"2024-09-21T21:43:48.230703+0000","last_unstale":"2024-09-21T21:43:50.091421+0000","last_undegraded":"2024-09-21T21:43:50.091421+0000","last_fullsized":"2024-09-21T21:43:50.091421+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":554,"log_dups_size":0,"ondisk_log_size":554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:39:12.422453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071450699999999997,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":8813,"num_read_kb":47394,"num_write":525,"num_write_kb":9434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":241,"num_bytes_recovered":7579476,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"171'1111","reported_seq":11670,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.371409+0000","last_change":"2024-09-21T21:43:33.292652+0000","last_active":"2024-09-21T21:43:49.371409+0000","last_peered":"2024-09-21T21:43:49.371409+0000","last_clean":"2024-09-21T21:43:49.371409+0000","last_became_active":"2024-09-21T21:43:19.224644+0000","last_became_peered":"2024-09-21T21:43:19.224644+0000","last_unstale":"2024-09-21T21:43:49.371409+0000","last_undegraded":"2024-09-21T21:43:49.371409+0000","last_fullsized":"2024-09-21T21:43:49.371409+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":1111,"log_dups_size":0,"ondisk_log_size":1111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:19:31.244395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001226878,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":5,"num_read":13231,"num_read_kb":49174,"num_write":1280,"num_write_kb":10665,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":398,"num_bytes_recovered":16632560,"num_keys_recovered":144,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"170'777","reported_seq":10308,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.374283+0000","last_change":"2024-09-21T21:43:33.292265+0000","last_active":"2024-09-21T21:43:49.374283+0000","last_peered":"2024-09-21T21:43:49.374283+0000","last_clean":"2024-09-21T21:43:49.374283+0000","last_became_active":"2024-09-21T21:43:09.227569+0000","last_became_peered":"2024-09-21T21:43:09.227569+0000","last_unstale":"2024-09-21T21:43:49.374283+0000","last_undegraded":"2024-09-21T21:43:49.374283+0000","last_fullsized":"2024-09-21T21:43:49.374283+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":777,"log_dups_size":0,"ondisk_log_size":777,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:20:34.944827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024712610000000002,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":10244,"num_read_kb":45368,"num_write":1134,"num_write_kb":14646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":331,"num_bytes_recovered":8931668,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"168'1625","reported_seq":14031,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.623830+0000","last_change":"2024-09-21T21:43:33.304566+0000","last_active":"2024-09-21T21:43:49.623830+0000","last_peered":"2024-09-21T21:43:49.623830+0000","last_clean":"2024-09-21T21:43:49.623830+0000","last_became_active":"2024-09-21T21:43:07.230148+0000","last_became_peered":"2024-09-21T21:43:07.230148+0000","last_unstale":"2024-09-21T21:43:49.623830+0000","last_undegraded":"2024-09-21T21:43:49.623830+0000","last_fullsized":"2024-09-21T21:43:49.623830+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":1625,"log_dups_size":0,"ondisk_log_size":1625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:42:46.324537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021897029999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":18573,"num_read_kb":61318,"num_write":2350,"num_write_kb":11538,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":225,"num_bytes_recovered":7091105,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"171'1447","reported_seq":17761,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.371616+0000","last_change":"2024-09-21T21:43:33.293137+0000","last_active":"2024-09-21T21:43:49.371616+0000","last_peered":"2024-09-21T21:43:49.371616+0000","last_clean":"2024-09-21T21:43:49.371616+0000","last_became_active":"2024-09-21T21:41:52.214022+0000","last_became_peered":"2024-09-21T21:41:52.214022+0000","last_unstale":"2024-09-21T21:43:49.371616+0000","last_undegraded":"2024-09-21T21:43:49.371616+0000","last_fullsized":"2024-09-21T21:43:49.371616+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":1447,"log_dups_size":0,"ondisk_log_size":1447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:42:14.727563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016156199999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":21216,"num_read_kb":45587,"num_write":1846,"num_write_kb":14797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":149,"num_bytes_recovered":3462638,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"168'454","reported_seq":7909,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.623841+0000","last_change":"2024-09-21T21:43:33.304631+0000","last_active":"2024-09-21T21:43:49.623841+0000","last_peered":"2024-09-21T21:43:49.623841+0000","last_clean":"2024-09-21T21:43:49.623841+0000","last_became_active":"2024-09-21T21:43:18.223755+0000","last_became_peered":"2024-09-21T21:43:18.223755+0000","last_unstale":"2024-09-21T21:43:49.623841+0000","last_undegraded":"2024-09-21T21:43:49.623841+0000","last_fullsized":"2024-09-21T21:43:49.623841+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":454,"log_dups_size":0,"ondisk_log_size":454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:43:32.324890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018835950000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":6571,"num_read_kb":34865,"num_write":469,"num_write_kb":6862,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":238,"num_bytes_recovered":4045989,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"169'741","reported_seq":17136,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.623910+0000","last_change":"2024-09-21T21:43:33.303579+0000","last_active":"2024-09-21T21:43:49.623910+0000","last_peered":"2024-09-21T21:43:49.623910+0000","last_clean":"2024-09-21T21:43:49.623910+0000","last_became_active":"2024-09-21T21:41:52.212206+0000","last_became_peered":"2024-09-21T21:41:52.212206+0000","last_unstale":"2024-09-21T21:43:49.623910+0000","last_undegraded":"2024-09-21T21:43:49.623910+0000","last_fullsized":"2024-09-21T21:43:49.623910+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":741,"log_dups_size":0,"ondisk_log_size":741,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:10:57.220429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013653929999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":18315,"num_read_kb":42344,"num_write":672,"num_write_kb":9505,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":2967886,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"167'408","reported_seq":14252,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.623662+0000","last_change":"2024-09-21T21:43:33.302904+0000","last_active":"2024-09-21T21:43:49.623662+0000","last_peered":"2024-09-21T21:43:49.623662+0000","last_clean":"2024-09-21T21:43:49.623662+0000","last_became_active":"2024-09-21T21:43:06.233222+0000","last_became_peered":"2024-09-21T21:43:06.233222+0000","last_unstale":"2024-09-21T21:43:49.623662+0000","last_undegraded":"2024-09-21T21:43:49.623662+0000","last_fullsized":"2024-09-21T21:43:49.623662+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:54:36.083839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085383799999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":12974,"num_read_kb":55237,"num_write":424,"num_write_kb":11231,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":205,"num_bytes_recovered":6455839,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"168'641","reported_seq":8507,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.371363+0000","last_change":"2024-09-21T21:43:33.292136+0000","last_active":"2024-09-21T21:43:49.371363+0000","last_peered":"2024-09-21T21:43:49.371363+0000","last_clean":"2024-09-21T21:43:49.371363+0000","last_became_active":"2024-09-21T21:43:20.224840+0000","last_became_peered":"2024-09-21T21:43:20.224840+0000","last_unstale":"2024-09-21T21:43:49.371363+0000","last_undegraded":"2024-09-21T21:43:49.371363+0000","last_fullsized":"2024-09-21T21:43:49.371363+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":641,"log_dups_size":0,"ondisk_log_size":641,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:13:32.197836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091468500000000004,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":7916,"num_read_kb":33911,"num_write":730,"num_write_kb":6868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":492,"num_bytes_recovered":3936953,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"170'791","reported_seq":19355,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.374375+0000","last_change":"2024-09-21T21:43:33.321430+0000","last_active":"2024-09-21T21:43:49.374375+0000","last_peered":"2024-09-21T21:43:49.374375+0000","last_clean":"2024-09-21T21:43:49.374375+0000","last_became_active":"2024-09-21T21:43:07.229497+0000","last_became_peered":"2024-09-21T21:43:07.229497+0000","last_unstale":"2024-09-21T21:43:49.374375+0000","last_undegraded":"2024-09-21T21:43:49.374375+0000","last_fullsized":"2024-09-21T21:43:49.374375+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":791,"log_dups_size":0,"ondisk_log_size":791,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:10:05.347980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061499099999999995,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":20795,"num_read_kb":71790,"num_write":798,"num_write_kb":9984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":352,"num_bytes_recovered":16923683,"num_keys_recovered":92,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"168'477","reported_seq":10652,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.371417+0000","last_change":"2024-09-21T21:43:33.292617+0000","last_active":"2024-09-21T21:43:49.371417+0000","last_peered":"2024-09-21T21:43:49.371417+0000","last_clean":"2024-09-21T21:43:49.371417+0000","last_became_active":"2024-09-21T21:41:52.209710+0000","last_became_peered":"2024-09-21T21:41:52.209710+0000","last_unstale":"2024-09-21T21:43:49.371417+0000","last_undegraded":"2024-09-21T21:43:49.371417+0000","last_fullsized":"2024-09-21T21:43:49.371417+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":477,"log_dups_size":0,"ondisk_log_size":477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:20:59.585304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013513189999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":9647,"num_read_kb":76927,"num_write":508,"num_write_kb":12558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":146,"num_bytes_recovered":17387622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":241,"num_object_clones":0,"num_object_copies":482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":241,"num_whiteouts":237,"num_read":408853,"num_read_kb":1471230,"num_write":27874,"num_write_kb":319707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7824,"num_bytes_recovered":220201937,"num_keys_recovered":844,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24446,"ondisk_log_size":24446,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":102090,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":4,"up_from":13,"seq":55834574894,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55556,"kb_used_data":2160,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316284,"statfs":{"total":96636764160,"available":96579874816,"internally_reserved":0,"allocated":2211840,"data_stored":928386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574894,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87536,"kb_used_data":2204,"kb_used_omap":1,"kb_used_meta":85310,"kb_avail":94284304,"statfs":{"total":96636764160,"available":96547127296,"internally_reserved":0,"allocated":2256896,"data_stored":1518735,"data_compressed":34030,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1593,"internal_metadata":87357895},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":12,"seq":51539607598,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73604,"kb_used_data":2160,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298236,"statfs":{"total":96636764160,"available":96561393664,"internally_reserved":0,"allocated":2211840,"data_stored":928386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607598,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55576,"kb_used_data":2176,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316264,"statfs":{"total":96636764160,"available":96579854336,"internally_reserved":0,"allocated":2228224,"data_stored":933375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607599,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47296,"kb_used_data":2216,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324544,"statfs":{"total":96636764160,"available":96588333056,"internally_reserved":0,"allocated":2269184,"data_stored":1523705,"data_compressed":34030,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640303,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47244,"kb_used_data":2168,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324596,"statfs":{"total":96636764160,"available":96588386304,"internally_reserved":0,"allocated":2220032,"data_stored":933337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34030,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34030,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34030,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T21:43:52.429 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T21:43:52.707 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:43:52.708 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":185,"fsid":"a0fdcfc2-ddd2-4e50-81bb-876d940f3578","created":"2024-09-21T21:40:07.413917+0000","modified":"2024-09-21T21:43:49.359995+0000","last_up_change":"2024-09-21T21:40:18.340430+0000","last_in_change":"2024-09-21T21:43:14.301223+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T21:40:18.321328+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-21T21:40:24.235658+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":19,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-21T21:40:36.093691+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":48,"snap_epoch":171,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a9869830-8e74-4b02-963d-ab630a0ee286","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1657552338},{"type":"v1","addr":"172.21.15.81:6809","nonce":1657552338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1657552338},{"type":"v1","addr":"172.21.15.81:6811","nonce":1657552338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1657552338},{"type":"v1","addr":"172.21.15.81:6815","nonce":1657552338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1657552338},{"type":"v1","addr":"172.21.15.81:6813","nonce":1657552338}]},"public_addr":"172.21.15.81:6809/1657552338","cluster_addr":"172.21.15.81:6811/1657552338","heartbeat_back_addr":"172.21.15.81:6815/1657552338","heartbeat_front_addr":"172.21.15.81:6813/1657552338","state":["exists","up"]},{"osd":1,"uuid":"b189819d-69a3-4e95-96e7-96dc4809ac80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2864159502},{"type":"v1","addr":"172.21.15.81:6817","nonce":2864159502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2864159502},{"type":"v1","addr":"172.21.15.81:6819","nonce":2864159502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2864159502},{"type":"v1","addr":"172.21.15.81:6823","nonce":2864159502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2864159502},{"type":"v1","addr":"172.21.15.81:6821","nonce":2864159502}]},"public_addr":"172.21.15.81:6817/2864159502","cluster_addr":"172.21.15.81:6819/2864159502","heartbeat_back_addr":"172.21.15.81:6823/2864159502","heartbeat_front_addr":"172.21.15.81:6821/2864159502","state":["exists","up"]},{"osd":2,"uuid":"8437b99a-1297-4903-ab66-95c5cba2b2bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":3023160932},{"type":"v1","addr":"172.21.15.81:6801","nonce":3023160932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":3023160932},{"type":"v1","addr":"172.21.15.81:6803","nonce":3023160932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":3023160932},{"type":"v1","addr":"172.21.15.81:6807","nonce":3023160932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":3023160932},{"type":"v1","addr":"172.21.15.81:6805","nonce":3023160932}]},"public_addr":"172.21.15.81:6801/3023160932","cluster_addr":"172.21.15.81:6803/3023160932","heartbeat_back_addr":"172.21.15.81:6807/3023160932","heartbeat_front_addr":"172.21.15.81:6805/3023160932","state":["exists","up"]},{"osd":3,"uuid":"e8848768-c1aa-445b-9554-dc6aa4eefc8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":2681016844},{"type":"v1","addr":"172.21.15.122:6809","nonce":2681016844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2681016844},{"type":"v1","addr":"172.21.15.122:6811","nonce":2681016844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2681016844},{"type":"v1","addr":"172.21.15.122:6815","nonce":2681016844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2681016844},{"type":"v1","addr":"172.21.15.122:6813","nonce":2681016844}]},"public_addr":"172.21.15.122:6809/2681016844","cluster_addr":"172.21.15.122:6811/2681016844","heartbeat_back_addr":"172.21.15.122:6815/2681016844","heartbeat_front_addr":"172.21.15.122:6813/2681016844","state":["exists","up"]},{"osd":4,"uuid":"b1d8e85d-a207-4ba6-9322-61d07812083e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":1095023540},{"type":"v1","addr":"172.21.15.122:6801","nonce":1095023540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":1095023540},{"type":"v1","addr":"172.21.15.122:6803","nonce":1095023540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":1095023540},{"type":"v1","addr":"172.21.15.122:6807","nonce":1095023540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":1095023540},{"type":"v1","addr":"172.21.15.122:6805","nonce":1095023540}]},"public_addr":"172.21.15.122:6801/1095023540","cluster_addr":"172.21.15.122:6803/1095023540","heartbeat_back_addr":"172.21.15.122:6807/1095023540","heartbeat_front_addr":"172.21.15.122:6805/1095023540","state":["exists","up"]},{"osd":5,"uuid":"3ab0450b-d8fd-4ef2-a298-08d5aa2ea32f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1974398333},{"type":"v1","addr":"172.21.15.122:6817","nonce":1974398333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1974398333},{"type":"v1","addr":"172.21.15.122:6819","nonce":1974398333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1974398333},{"type":"v1","addr":"172.21.15.122:6823","nonce":1974398333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1974398333},{"type":"v1","addr":"172.21.15.122:6821","nonce":1974398333}]},"public_addr":"172.21.15.122:6817/1974398333","cluster_addr":"172.21.15.122:6819/1974398333","heartbeat_back_addr":"172.21.15.122:6823/1974398333","heartbeat_front_addr":"172.21.15.122:6821/1974398333","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T21:40:15.766648+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T21:40:15.457496+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T21:40:14.767039+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T21:40:15.355570+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T21:40:15.335729+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T21:40:15.614733+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.8","osds":[2,3]}],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T21:43:53.721 INFO:tasks.ceph:Scrubbing osd.0 2024-09-21T21:43:53.721 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-21T21:43:53.855 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:43:53.856 INFO:teuthology.orchestra.run.smithi081.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T21:43:53.856 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:43:53.867 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-21T21:43:54.139 INFO:teuthology.orchestra.run.smithi081.stderr:instructed osd(s) 0 to deep-scrub 2024-09-21T21:43:54.152 INFO:tasks.ceph:Scrubbing osd.1 2024-09-21T21:43:54.152 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-21T21:43:54.273 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:43:54.273 INFO:teuthology.orchestra.run.smithi081.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T21:43:54.273 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:43:54.284 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-21T21:43:54.563 INFO:teuthology.orchestra.run.smithi081.stderr:instructed osd(s) 1 to deep-scrub 2024-09-21T21:43:54.576 INFO:tasks.ceph:Scrubbing osd.2 2024-09-21T21:43:54.576 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-21T21:43:54.698 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:43:54.698 INFO:teuthology.orchestra.run.smithi081.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T21:43:54.698 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:43:54.709 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-21T21:43:54.986 INFO:teuthology.orchestra.run.smithi081.stderr:instructed osd(s) 2 to deep-scrub 2024-09-21T21:43:54.999 INFO:tasks.ceph:Scrubbing osd.3 2024-09-21T21:43:54.999 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-21T21:43:55.122 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:43:55.122 INFO:teuthology.orchestra.run.smithi081.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T21:43:55.122 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:43:55.131 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-21T21:43:55.408 INFO:teuthology.orchestra.run.smithi081.stderr:instructed osd(s) 3 to deep-scrub 2024-09-21T21:43:55.421 INFO:tasks.ceph:Scrubbing osd.4 2024-09-21T21:43:55.421 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-21T21:43:55.543 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:43:55.543 INFO:teuthology.orchestra.run.smithi081.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T21:43:55.543 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:43:55.555 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-21T21:43:55.830 INFO:teuthology.orchestra.run.smithi081.stderr:instructed osd(s) 4 to deep-scrub 2024-09-21T21:43:55.843 INFO:tasks.ceph:Scrubbing osd.5 2024-09-21T21:43:55.843 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-21T21:43:55.966 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-09-21T21:43:55.966 INFO:teuthology.orchestra.run.smithi081.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T21:43:55.966 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-09-21T21:43:55.977 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-21T21:43:56.252 INFO:teuthology.orchestra.run.smithi081.stderr:instructed osd(s) 5 to deep-scrub 2024-09-21T21:43:56.265 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T21:43:56.540 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:43:56.540 INFO:teuthology.orchestra.run.smithi081.stderr:dumped all 2024-09-21T21:43:56.552 INFO:teuthology.orchestra.run.smithi081.stdout:{"pg_ready":true,"pg_map":{"version":290,"stamp":"2024-09-21T21:43:56.326056+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":245,"num_object_clones":0,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":237,"num_read":408969,"num_read_kb":1471330,"num_write":28035,"num_write_kb":322125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7832,"num_bytes_recovered":222563409,"num_keys_recovered":844,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24577,"ondisk_log_size":24577,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":366856,"kb_used_data":13120,"kb_used_omap":9,"kb_used_meta":353590,"kb_avail":565864184,"statfs":{"total":579820584960,"available":579444924416,"internally_reserved":0,"allocated":13434880,"data_stored":6780834,"data_compressed":68060,"data_compressed_allocated":90112,"data_compressed_original":1179648,"omap_allocated":9568,"internal_metadata":362076832},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":-11,"num_read_kb":-11,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":125,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.102268"},"pg_stats":[{"pgid":"3.1f","version":"171'591","reported_seq":7658,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.374605+0000","last_change":"2024-09-21T21:43:33.322591+0000","last_active":"2024-09-21T21:43:49.374605+0000","last_peered":"2024-09-21T21:43:49.374605+0000","last_clean":"2024-09-21T21:43:49.374605+0000","last_became_active":"2024-09-21T21:43:08.230509+0000","last_became_peered":"2024-09-21T21:43:08.230509+0000","last_unstale":"2024-09-21T21:43:49.374605+0000","last_undegraded":"2024-09-21T21:43:49.374605+0000","last_fullsized":"2024-09-21T21:43:49.374605+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":591,"log_dups_size":0,"ondisk_log_size":591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:50:51.925851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013233089999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":8384,"num_read_kb":24764,"num_write":554,"num_write_kb":8557,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":293,"num_bytes_recovered":3779776,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"185'1024","reported_seq":15348,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:56.254134+0000","last_change":"2024-09-21T21:43:56.254180+0000","last_active":"2024-09-21T21:43:56.254134+0000","last_peered":"2024-09-21T21:43:56.254134+0000","last_clean":"2024-09-21T21:43:56.254134+0000","last_became_active":"2024-09-21T21:43:44.356029+0000","last_became_peered":"2024-09-21T21:43:44.356029+0000","last_unstale":"2024-09-21T21:43:56.254134+0000","last_undegraded":"2024-09-21T21:43:56.254134+0000","last_fullsized":"2024-09-21T21:43:56.254134+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"185'1024","last_scrub_stamp":"2024-09-21T21:43:56.254044+0000","last_deep_scrub":"185'1024","last_deep_scrub_stamp":"2024-09-21T21:43:56.254044+0000","last_clean_scrub_stamp":"2024-09-21T21:43:56.254044+0000","objects_scrubbed":11,"log_size":1024,"log_dups_size":0,"ondisk_log_size":1024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:46:19.752736+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0.001190569,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":16219,"num_read_kb":73109,"num_write":1473,"num_write_kb":18401,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":14291598,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"168'985","reported_seq":10752,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.623764+0000","last_change":"2024-09-21T21:43:33.302894+0000","last_active":"2024-09-21T21:43:49.623764+0000","last_peered":"2024-09-21T21:43:49.623764+0000","last_clean":"2024-09-21T21:43:49.623764+0000","last_became_active":"2024-09-21T21:43:20.225483+0000","last_became_peered":"2024-09-21T21:43:20.225483+0000","last_unstale":"2024-09-21T21:43:49.623764+0000","last_undegraded":"2024-09-21T21:43:49.623764+0000","last_fullsized":"2024-09-21T21:43:49.623764+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":985,"log_dups_size":0,"ondisk_log_size":985,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:13:56.033873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077268300000000003,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":13110,"num_read_kb":43443,"num_write":1158,"num_write_kb":7907,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":4009384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"171'884","reported_seq":13301,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.623722+0000","last_change":"2024-09-21T21:43:33.304149+0000","last_active":"2024-09-21T21:43:49.623722+0000","last_peered":"2024-09-21T21:43:49.623722+0000","last_clean":"2024-09-21T21:43:49.623722+0000","last_became_active":"2024-09-21T21:43:19.225435+0000","last_became_peered":"2024-09-21T21:43:19.225435+0000","last_unstale":"2024-09-21T21:43:49.623722+0000","last_undegraded":"2024-09-21T21:43:49.623722+0000","last_fullsized":"2024-09-21T21:43:49.623722+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":884,"log_dups_size":0,"ondisk_log_size":884,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:45:28.185309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001856124,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":15417,"num_read_kb":35250,"num_write":925,"num_write_kb":7812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":178,"num_bytes_recovered":1721983,"num_keys_recovered":148,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"171'737","reported_seq":9064,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.374605+0000","last_change":"2024-09-21T21:43:33.322211+0000","last_active":"2024-09-21T21:43:49.374605+0000","last_peered":"2024-09-21T21:43:49.374605+0000","last_clean":"2024-09-21T21:43:49.374605+0000","last_became_active":"2024-09-21T21:43:19.225304+0000","last_became_peered":"2024-09-21T21:43:19.225304+0000","last_unstale":"2024-09-21T21:43:49.374605+0000","last_undegraded":"2024-09-21T21:43:49.374605+0000","last_fullsized":"2024-09-21T21:43:49.374605+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":737,"log_dups_size":0,"ondisk_log_size":737,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:07:04.863691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001146085,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":8621,"num_read_kb":37221,"num_write":843,"num_write_kb":4971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":547,"num_bytes_recovered":1792422,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"170'618","reported_seq":25431,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:50.091426+0000","last_change":"2024-09-21T21:43:48.232831+0000","last_active":"2024-09-21T21:43:50.091426+0000","last_peered":"2024-09-21T21:43:50.091426+0000","last_clean":"2024-09-21T21:43:50.091426+0000","last_became_active":"2024-09-21T21:43:48.230767+0000","last_became_peered":"2024-09-21T21:43:48.230767+0000","last_unstale":"2024-09-21T21:43:50.091426+0000","last_undegraded":"2024-09-21T21:43:50.091426+0000","last_fullsized":"2024-09-21T21:43:50.091426+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":618,"log_dups_size":0,"ondisk_log_size":618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:47:54.923891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0044588529999999996,"stat_sum":{"num_bytes":19,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":25138,"num_read_kb":59201,"num_write":578,"num_write_kb":7782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":203,"num_bytes_recovered":4464880,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"170'860","reported_seq":14187,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.373846+0000","last_change":"2024-09-21T21:43:33.291493+0000","last_active":"2024-09-21T21:43:49.373846+0000","last_peered":"2024-09-21T21:43:49.373846+0000","last_clean":"2024-09-21T21:43:49.373846+0000","last_became_active":"2024-09-21T21:43:20.224845+0000","last_became_peered":"2024-09-21T21:43:20.224845+0000","last_unstale":"2024-09-21T21:43:49.373846+0000","last_undegraded":"2024-09-21T21:43:49.373846+0000","last_fullsized":"2024-09-21T21:43:49.373846+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":860,"log_dups_size":0,"ondisk_log_size":860,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:55:13.229277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018111119999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":16015,"num_read_kb":61704,"num_write":858,"num_write_kb":15379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":9899814,"num_keys_recovered":59,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"168'335","reported_seq":6958,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:50.272805+0000","last_change":"2024-09-21T21:43:44.358159+0000","last_active":"2024-09-21T21:43:50.272805+0000","last_peered":"2024-09-21T21:43:50.272805+0000","last_clean":"2024-09-21T21:43:50.272805+0000","last_became_active":"2024-09-21T21:43:44.356203+0000","last_became_peered":"2024-09-21T21:43:44.356203+0000","last_unstale":"2024-09-21T21:43:50.272805+0000","last_undegraded":"2024-09-21T21:43:50.272805+0000","last_fullsized":"2024-09-21T21:43:50.272805+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012910630000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":6161,"num_read_kb":33023,"num_write":325,"num_write_kb":8639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":5854224,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"171'497","reported_seq":6419,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.374509+0000","last_change":"2024-09-21T21:43:33.322741+0000","last_active":"2024-09-21T21:43:49.374509+0000","last_peered":"2024-09-21T21:43:49.374509+0000","last_clean":"2024-09-21T21:43:49.374509+0000","last_became_active":"2024-09-21T21:43:07.232590+0000","last_became_peered":"2024-09-21T21:43:07.232590+0000","last_unstale":"2024-09-21T21:43:49.374509+0000","last_undegraded":"2024-09-21T21:43:49.374509+0000","last_fullsized":"2024-09-21T21:43:49.374509+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":497,"log_dups_size":0,"ondisk_log_size":497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:04:00.513414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001467229,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":6465,"num_read_kb":25955,"num_write":521,"num_write_kb":9178,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":203,"num_bytes_recovered":4149457,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"171'1238","reported_seq":9868,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.623839+0000","last_change":"2024-09-21T21:43:33.305145+0000","last_active":"2024-09-21T21:43:49.623839+0000","last_peered":"2024-09-21T21:43:49.623839+0000","last_clean":"2024-09-21T21:43:49.623839+0000","last_became_active":"2024-09-21T21:41:53.214270+0000","last_became_peered":"2024-09-21T21:41:53.214270+0000","last_unstale":"2024-09-21T21:43:49.623839+0000","last_undegraded":"2024-09-21T21:43:49.623839+0000","last_fullsized":"2024-09-21T21:43:49.623839+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":1238,"log_dups_size":0,"ondisk_log_size":1238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:51:58.701104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022124750000000002,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":13255,"num_read_kb":44399,"num_write":1796,"num_write_kb":12840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":148,"num_bytes_recovered":4869268,"num_keys_recovered":47,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"170'744","reported_seq":9490,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.371529+0000","last_change":"2024-09-21T21:43:33.293155+0000","last_active":"2024-09-21T21:43:49.371529+0000","last_peered":"2024-09-21T21:43:49.371529+0000","last_clean":"2024-09-21T21:43:49.371529+0000","last_became_active":"2024-09-21T21:43:07.228504+0000","last_became_peered":"2024-09-21T21:43:07.228504+0000","last_unstale":"2024-09-21T21:43:49.371529+0000","last_undegraded":"2024-09-21T21:43:49.371529+0000","last_fullsized":"2024-09-21T21:43:49.371529+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":744,"log_dups_size":0,"ondisk_log_size":744,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:55:34.005752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001650253,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":9201,"num_read_kb":56862,"num_write":979,"num_write_kb":7172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":520,"num_bytes_recovered":10162533,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"167'626","reported_seq":21896,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.371520+0000","last_change":"2024-09-21T21:43:48.230374+0000","last_active":"2024-09-21T21:43:49.371520+0000","last_peered":"2024-09-21T21:43:49.371520+0000","last_clean":"2024-09-21T21:43:49.371520+0000","last_became_active":"2024-09-21T21:43:48.228528+0000","last_became_peered":"2024-09-21T21:43:48.228528+0000","last_unstale":"2024-09-21T21:43:49.371520+0000","last_undegraded":"2024-09-21T21:43:49.371520+0000","last_fullsized":"2024-09-21T21:43:49.371520+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":626,"log_dups_size":0,"ondisk_log_size":626,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:52:06.587738+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.00063765000000000002,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":22083,"num_read_kb":75165,"num_write":513,"num_write_kb":10980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":602,"num_bytes_recovered":22337258,"num_keys_recovered":84,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":236,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.371668+0000","last_change":"2024-09-21T21:43:15.316851+0000","last_active":"2024-09-21T21:43:49.371668+0000","last_peered":"2024-09-21T21:43:49.371668+0000","last_clean":"2024-09-21T21:43:49.371668+0000","last_became_active":"2024-09-21T21:43:15.316533+0000","last_became_peered":"2024-09-21T21:43:15.316533+0000","last_unstale":"2024-09-21T21:43:49.371668+0000","last_undegraded":"2024-09-21T21:43:49.371668+0000","last_fullsized":"2024-09-21T21:43:49.371668+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:00:30.781100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"171'1225","reported_seq":14273,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.374367+0000","last_change":"2024-09-21T21:43:33.321874+0000","last_active":"2024-09-21T21:43:49.374367+0000","last_peered":"2024-09-21T21:43:49.374367+0000","last_clean":"2024-09-21T21:43:49.374367+0000","last_became_active":"2024-09-21T21:43:09.230849+0000","last_became_peered":"2024-09-21T21:43:09.230849+0000","last_unstale":"2024-09-21T21:43:49.374367+0000","last_undegraded":"2024-09-21T21:43:49.374367+0000","last_fullsized":"2024-09-21T21:43:49.374367+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":1225,"log_dups_size":0,"ondisk_log_size":1225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:50:50.908667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096363700000000004,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":15823,"num_read_kb":66323,"num_write":1574,"num_write_kb":15987,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":349,"num_bytes_recovered":9257588,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":362,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.374023+0000","last_change":"2024-09-21T21:41:49.266534+0000","last_active":"2024-09-21T21:43:49.374023+0000","last_peered":"2024-09-21T21:43:49.374023+0000","last_clean":"2024-09-21T21:43:49.374023+0000","last_became_active":"2024-09-21T21:41:49.266257+0000","last_became_peered":"2024-09-21T21:41:49.266257+0000","last_unstale":"2024-09-21T21:43:49.374023+0000","last_undegraded":"2024-09-21T21:43:49.374023+0000","last_fullsized":"2024-09-21T21:43:49.374023+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:38:31.063193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062280799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"170'555","reported_seq":8889,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:50.272723+0000","last_change":"2024-09-21T21:43:44.357970+0000","last_active":"2024-09-21T21:43:50.272723+0000","last_peered":"2024-09-21T21:43:50.272723+0000","last_clean":"2024-09-21T21:43:50.272723+0000","last_became_active":"2024-09-21T21:43:44.356543+0000","last_became_peered":"2024-09-21T21:43:44.356543+0000","last_unstale":"2024-09-21T21:43:50.272723+0000","last_undegraded":"2024-09-21T21:43:50.272723+0000","last_fullsized":"2024-09-21T21:43:50.272723+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":555,"log_dups_size":0,"ondisk_log_size":555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.00099396100000000002,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":10770,"num_read_kb":40597,"num_write":524,"num_write_kb":10606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":242,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.371558+0000","last_change":"2024-09-21T21:43:03.224822+0000","last_active":"2024-09-21T21:43:49.371558+0000","last_peered":"2024-09-21T21:43:49.371558+0000","last_clean":"2024-09-21T21:43:49.371558+0000","last_became_active":"2024-09-21T21:43:03.224515+0000","last_became_peered":"2024-09-21T21:43:03.224515+0000","last_unstale":"2024-09-21T21:43:49.371558+0000","last_undegraded":"2024-09-21T21:43:49.371558+0000","last_fullsized":"2024-09-21T21:43:49.371558+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:08:18.537918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"170'880","reported_seq":4751,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.373958+0000","last_change":"2024-09-21T21:43:33.291832+0000","last_active":"2024-09-21T21:43:49.373958+0000","last_peered":"2024-09-21T21:43:49.373958+0000","last_clean":"2024-09-21T21:43:49.373958+0000","last_became_active":"2024-09-21T21:40:37.419944+0000","last_became_peered":"2024-09-21T21:40:37.419944+0000","last_unstale":"2024-09-21T21:43:49.373958+0000","last_undegraded":"2024-09-21T21:43:49.373958+0000","last_fullsized":"2024-09-21T21:43:49.373958+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":880,"log_dups_size":0,"ondisk_log_size":880,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:53:04.189791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020772899999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":5927,"num_read_kb":25257,"num_write":1263,"num_write_kb":11321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":367,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.374070+0000","last_change":"2024-09-21T21:43:03.224909+0000","last_active":"2024-09-21T21:43:49.374070+0000","last_peered":"2024-09-21T21:43:49.374070+0000","last_clean":"2024-09-21T21:43:49.374070+0000","last_became_active":"2024-09-21T21:43:03.224572+0000","last_became_peered":"2024-09-21T21:43:03.224572+0000","last_unstale":"2024-09-21T21:43:49.374070+0000","last_undegraded":"2024-09-21T21:43:49.374070+0000","last_fullsized":"2024-09-21T21:43:49.374070+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:10:47.109676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070669899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"168'637","reported_seq":10497,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.623794+0000","last_change":"2024-09-21T21:43:33.303937+0000","last_active":"2024-09-21T21:43:49.623794+0000","last_peered":"2024-09-21T21:43:49.623794+0000","last_clean":"2024-09-21T21:43:49.623794+0000","last_became_active":"2024-09-21T21:41:25.650379+0000","last_became_peered":"2024-09-21T21:41:25.650379+0000","last_unstale":"2024-09-21T21:43:49.623794+0000","last_undegraded":"2024-09-21T21:43:49.623794+0000","last_fullsized":"2024-09-21T21:43:49.623794+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":637,"log_dups_size":0,"ondisk_log_size":637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:48:59.774761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001695787,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":11334,"num_read_kb":32091,"num_write":515,"num_write_kb":8339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"171'646","reported_seq":11168,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:50.091476+0000","last_change":"2024-09-21T21:43:48.233737+0000","last_active":"2024-09-21T21:43:50.091476+0000","last_peered":"2024-09-21T21:43:50.091476+0000","last_clean":"2024-09-21T21:43:50.091476+0000","last_became_active":"2024-09-21T21:43:48.231457+0000","last_became_peered":"2024-09-21T21:43:48.231457+0000","last_unstale":"2024-09-21T21:43:50.091476+0000","last_undegraded":"2024-09-21T21:43:50.091476+0000","last_fullsized":"2024-09-21T21:43:50.091476+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:10:37.262673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081362800000000005,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":9832,"num_read_kb":35952,"num_write":639,"num_write_kb":9820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":437,"num_bytes_recovered":19367440,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":226,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.623813+0000","last_change":"2024-09-21T21:42:24.516929+0000","last_active":"2024-09-21T21:43:49.623813+0000","last_peered":"2024-09-21T21:43:49.623813+0000","last_clean":"2024-09-21T21:43:49.623813+0000","last_became_active":"2024-09-21T21:42:24.516623+0000","last_became_peered":"2024-09-21T21:42:24.516623+0000","last_unstale":"2024-09-21T21:43:49.623813+0000","last_undegraded":"2024-09-21T21:43:49.623813+0000","last_fullsized":"2024-09-21T21:43:49.623813+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:51:59.592292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000906268,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"171'815","reported_seq":11441,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.373829+0000","last_change":"2024-09-21T21:43:33.290813+0000","last_active":"2024-09-21T21:43:49.373829+0000","last_peered":"2024-09-21T21:43:49.373829+0000","last_clean":"2024-09-21T21:43:49.373829+0000","last_became_active":"2024-09-21T21:40:37.418973+0000","last_became_peered":"2024-09-21T21:40:37.418973+0000","last_unstale":"2024-09-21T21:43:49.373829+0000","last_undegraded":"2024-09-21T21:43:49.373829+0000","last_fullsized":"2024-09-21T21:43:49.373829+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":815,"log_dups_size":0,"ondisk_log_size":815,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:31:26.860045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001131546,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":13425,"num_read_kb":28144,"num_write":602,"num_write_kb":6293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":116,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:55.285108+0000","last_change":"2024-09-21T21:43:55.285149+0000","last_active":"2024-09-21T21:43:55.285108+0000","last_peered":"2024-09-21T21:43:55.285108+0000","last_clean":"2024-09-21T21:43:55.285108+0000","last_became_active":"2024-09-21T21:43:44.357116+0000","last_became_peered":"2024-09-21T21:43:44.357116+0000","last_unstale":"2024-09-21T21:43:55.285108+0000","last_undegraded":"2024-09-21T21:43:55.285108+0000","last_fullsized":"2024-09-21T21:43:55.285108+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:43:55.285016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:43:55.285016+0000","last_clean_scrub_stamp":"2024-09-21T21:43:55.285016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:12:30.936177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"168'387","reported_seq":16883,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.373980+0000","last_change":"2024-09-21T21:43:33.321435+0000","last_active":"2024-09-21T21:43:49.373980+0000","last_peered":"2024-09-21T21:43:49.373980+0000","last_clean":"2024-09-21T21:43:49.373980+0000","last_became_active":"2024-09-21T21:43:20.224689+0000","last_became_peered":"2024-09-21T21:43:20.224689+0000","last_unstale":"2024-09-21T21:43:49.373980+0000","last_undegraded":"2024-09-21T21:43:49.373980+0000","last_fullsized":"2024-09-21T21:43:49.373980+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":387,"log_dups_size":0,"ondisk_log_size":387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:15:32.359340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061419100000000004,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":15397,"num_read_kb":48505,"num_write":377,"num_write_kb":4859,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":429,"num_bytes_recovered":6598898,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":227,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.371346+0000","last_change":"2024-09-21T21:41:49.265465+0000","last_active":"2024-09-21T21:43:49.371346+0000","last_peered":"2024-09-21T21:43:49.371346+0000","last_clean":"2024-09-21T21:43:49.371346+0000","last_became_active":"2024-09-21T21:41:49.265182+0000","last_became_peered":"2024-09-21T21:41:49.265182+0000","last_unstale":"2024-09-21T21:43:49.371346+0000","last_undegraded":"2024-09-21T21:43:49.371346+0000","last_fullsized":"2024-09-21T21:43:49.371346+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:19:17.973195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"171'773","reported_seq":10302,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.374490+0000","last_change":"2024-09-21T21:43:33.293418+0000","last_active":"2024-09-21T21:43:49.374490+0000","last_peered":"2024-09-21T21:43:49.374490+0000","last_clean":"2024-09-21T21:43:49.374490+0000","last_became_active":"2024-09-21T21:40:37.421655+0000","last_became_peered":"2024-09-21T21:40:37.421655+0000","last_unstale":"2024-09-21T21:43:49.374490+0000","last_undegraded":"2024-09-21T21:43:49.374490+0000","last_fullsized":"2024-09-21T21:43:49.374490+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":773,"log_dups_size":0,"ondisk_log_size":773,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:43:34.040974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031354970000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":12713,"num_read_kb":34378,"num_write":754,"num_write_kb":7498,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":324,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.623702+0000","last_change":"2024-09-21T21:42:24.518810+0000","last_active":"2024-09-21T21:43:49.623702+0000","last_peered":"2024-09-21T21:43:49.623702+0000","last_clean":"2024-09-21T21:43:49.623702+0000","last_became_active":"2024-09-21T21:42:24.518461+0000","last_became_peered":"2024-09-21T21:42:24.518461+0000","last_unstale":"2024-09-21T21:43:49.623702+0000","last_undegraded":"2024-09-21T21:43:49.623702+0000","last_fullsized":"2024-09-21T21:43:49.623702+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:24.357997+0000","last_clean_scrub_stamp":"2024-09-21T21:40:24.357997+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:17:47.593497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036803999999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'117","reported_seq":434,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.371518+0000","last_change":"2024-09-21T21:43:06.231540+0000","last_active":"2024-09-21T21:43:49.371518+0000","last_peered":"2024-09-21T21:43:49.371518+0000","last_clean":"2024-09-21T21:43:49.371518+0000","last_became_active":"2024-09-21T21:43:06.231241+0000","last_became_peered":"2024-09-21T21:43:06.231241+0000","last_unstale":"2024-09-21T21:43:49.371518+0000","last_undegraded":"2024-09-21T21:43:49.371518+0000","last_fullsized":"2024-09-21T21:43:49.371518+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:18.340430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:18.340430+0000","last_clean_scrub_stamp":"2024-09-21T21:40:18.340430+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:50:08.187886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"170'374","reported_seq":6273,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:50.272893+0000","last_change":"2024-09-21T21:43:44.358111+0000","last_active":"2024-09-21T21:43:50.272893+0000","last_peered":"2024-09-21T21:43:50.272893+0000","last_clean":"2024-09-21T21:43:50.272893+0000","last_became_active":"2024-09-21T21:43:44.356024+0000","last_became_peered":"2024-09-21T21:43:44.356024+0000","last_unstale":"2024-09-21T21:43:50.272893+0000","last_undegraded":"2024-09-21T21:43:50.272893+0000","last_fullsized":"2024-09-21T21:43:50.272893+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":374,"log_dups_size":0,"ondisk_log_size":374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094280000000000004,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":5268,"num_read_kb":25972,"num_write":367,"num_write_kb":7278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":172,"num_bytes_recovered":2229995,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"170'554","reported_seq":8626,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:50.091421+0000","last_change":"2024-09-21T21:43:48.233080+0000","last_active":"2024-09-21T21:43:50.091421+0000","last_peered":"2024-09-21T21:43:50.091421+0000","last_clean":"2024-09-21T21:43:50.091421+0000","last_became_active":"2024-09-21T21:43:48.230703+0000","last_became_peered":"2024-09-21T21:43:48.230703+0000","last_unstale":"2024-09-21T21:43:50.091421+0000","last_undegraded":"2024-09-21T21:43:50.091421+0000","last_fullsized":"2024-09-21T21:43:50.091421+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":554,"log_dups_size":0,"ondisk_log_size":554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:39:12.422453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071450699999999997,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":8813,"num_read_kb":47394,"num_write":525,"num_write_kb":9434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":241,"num_bytes_recovered":7579476,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"171'1111","reported_seq":11670,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.371409+0000","last_change":"2024-09-21T21:43:33.292652+0000","last_active":"2024-09-21T21:43:49.371409+0000","last_peered":"2024-09-21T21:43:49.371409+0000","last_clean":"2024-09-21T21:43:49.371409+0000","last_became_active":"2024-09-21T21:43:19.224644+0000","last_became_peered":"2024-09-21T21:43:19.224644+0000","last_unstale":"2024-09-21T21:43:49.371409+0000","last_undegraded":"2024-09-21T21:43:49.371409+0000","last_fullsized":"2024-09-21T21:43:49.371409+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":1111,"log_dups_size":0,"ondisk_log_size":1111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:19:31.244395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001226878,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":5,"num_read":13231,"num_read_kb":49174,"num_write":1280,"num_write_kb":10665,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":398,"num_bytes_recovered":16632560,"num_keys_recovered":144,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"170'777","reported_seq":10308,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.374283+0000","last_change":"2024-09-21T21:43:33.292265+0000","last_active":"2024-09-21T21:43:49.374283+0000","last_peered":"2024-09-21T21:43:49.374283+0000","last_clean":"2024-09-21T21:43:49.374283+0000","last_became_active":"2024-09-21T21:43:09.227569+0000","last_became_peered":"2024-09-21T21:43:09.227569+0000","last_unstale":"2024-09-21T21:43:49.374283+0000","last_undegraded":"2024-09-21T21:43:49.374283+0000","last_fullsized":"2024-09-21T21:43:49.374283+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":777,"log_dups_size":0,"ondisk_log_size":777,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:20:34.944827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024712610000000002,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":10244,"num_read_kb":45368,"num_write":1134,"num_write_kb":14646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":331,"num_bytes_recovered":8931668,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"168'1625","reported_seq":14031,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.623830+0000","last_change":"2024-09-21T21:43:33.304566+0000","last_active":"2024-09-21T21:43:49.623830+0000","last_peered":"2024-09-21T21:43:49.623830+0000","last_clean":"2024-09-21T21:43:49.623830+0000","last_became_active":"2024-09-21T21:43:07.230148+0000","last_became_peered":"2024-09-21T21:43:07.230148+0000","last_unstale":"2024-09-21T21:43:49.623830+0000","last_undegraded":"2024-09-21T21:43:49.623830+0000","last_fullsized":"2024-09-21T21:43:49.623830+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":1625,"log_dups_size":0,"ondisk_log_size":1625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:42:46.324537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021897029999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":18573,"num_read_kb":61318,"num_write":2350,"num_write_kb":11538,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":225,"num_bytes_recovered":7091105,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"171'1447","reported_seq":17761,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.371616+0000","last_change":"2024-09-21T21:43:33.293137+0000","last_active":"2024-09-21T21:43:49.371616+0000","last_peered":"2024-09-21T21:43:49.371616+0000","last_clean":"2024-09-21T21:43:49.371616+0000","last_became_active":"2024-09-21T21:41:52.214022+0000","last_became_peered":"2024-09-21T21:41:52.214022+0000","last_unstale":"2024-09-21T21:43:49.371616+0000","last_undegraded":"2024-09-21T21:43:49.371616+0000","last_fullsized":"2024-09-21T21:43:49.371616+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":1447,"log_dups_size":0,"ondisk_log_size":1447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:42:14.727563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016156199999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":21216,"num_read_kb":45587,"num_write":1846,"num_write_kb":14797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":149,"num_bytes_recovered":3462638,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"168'454","reported_seq":7909,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.623841+0000","last_change":"2024-09-21T21:43:33.304631+0000","last_active":"2024-09-21T21:43:49.623841+0000","last_peered":"2024-09-21T21:43:49.623841+0000","last_clean":"2024-09-21T21:43:49.623841+0000","last_became_active":"2024-09-21T21:43:18.223755+0000","last_became_peered":"2024-09-21T21:43:18.223755+0000","last_unstale":"2024-09-21T21:43:49.623841+0000","last_undegraded":"2024-09-21T21:43:49.623841+0000","last_fullsized":"2024-09-21T21:43:49.623841+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":454,"log_dups_size":0,"ondisk_log_size":454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:43:32.324890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018835950000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":6571,"num_read_kb":34865,"num_write":469,"num_write_kb":6862,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":238,"num_bytes_recovered":4045989,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"169'741","reported_seq":17136,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.623910+0000","last_change":"2024-09-21T21:43:33.303579+0000","last_active":"2024-09-21T21:43:49.623910+0000","last_peered":"2024-09-21T21:43:49.623910+0000","last_clean":"2024-09-21T21:43:49.623910+0000","last_became_active":"2024-09-21T21:41:52.212206+0000","last_became_peered":"2024-09-21T21:41:52.212206+0000","last_unstale":"2024-09-21T21:43:49.623910+0000","last_undegraded":"2024-09-21T21:43:49.623910+0000","last_fullsized":"2024-09-21T21:43:49.623910+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":741,"log_dups_size":0,"ondisk_log_size":741,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:10:57.220429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013653929999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":18315,"num_read_kb":42344,"num_write":672,"num_write_kb":9505,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":2967886,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"167'408","reported_seq":14252,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.623662+0000","last_change":"2024-09-21T21:43:33.302904+0000","last_active":"2024-09-21T21:43:49.623662+0000","last_peered":"2024-09-21T21:43:49.623662+0000","last_clean":"2024-09-21T21:43:49.623662+0000","last_became_active":"2024-09-21T21:43:06.233222+0000","last_became_peered":"2024-09-21T21:43:06.233222+0000","last_unstale":"2024-09-21T21:43:49.623662+0000","last_undegraded":"2024-09-21T21:43:49.623662+0000","last_fullsized":"2024-09-21T21:43:49.623662+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:54:36.083839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085383799999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":12974,"num_read_kb":55237,"num_write":424,"num_write_kb":11231,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":205,"num_bytes_recovered":6455839,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"168'641","reported_seq":8507,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.371363+0000","last_change":"2024-09-21T21:43:33.292136+0000","last_active":"2024-09-21T21:43:49.371363+0000","last_peered":"2024-09-21T21:43:49.371363+0000","last_clean":"2024-09-21T21:43:49.371363+0000","last_became_active":"2024-09-21T21:43:20.224840+0000","last_became_peered":"2024-09-21T21:43:20.224840+0000","last_unstale":"2024-09-21T21:43:49.371363+0000","last_undegraded":"2024-09-21T21:43:49.371363+0000","last_fullsized":"2024-09-21T21:43:49.371363+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":641,"log_dups_size":0,"ondisk_log_size":641,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:13:32.197836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091468500000000004,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":7916,"num_read_kb":33911,"num_write":730,"num_write_kb":6868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":492,"num_bytes_recovered":3936953,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"170'791","reported_seq":19355,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.374375+0000","last_change":"2024-09-21T21:43:33.321430+0000","last_active":"2024-09-21T21:43:49.374375+0000","last_peered":"2024-09-21T21:43:49.374375+0000","last_clean":"2024-09-21T21:43:49.374375+0000","last_became_active":"2024-09-21T21:43:07.229497+0000","last_became_peered":"2024-09-21T21:43:07.229497+0000","last_unstale":"2024-09-21T21:43:49.374375+0000","last_undegraded":"2024-09-21T21:43:49.374375+0000","last_fullsized":"2024-09-21T21:43:49.374375+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":791,"log_dups_size":0,"ondisk_log_size":791,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:10:05.347980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061499099999999995,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":20795,"num_read_kb":71790,"num_write":798,"num_write_kb":9984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":352,"num_bytes_recovered":16923683,"num_keys_recovered":92,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"168'477","reported_seq":10652,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:49.371417+0000","last_change":"2024-09-21T21:43:33.292617+0000","last_active":"2024-09-21T21:43:49.371417+0000","last_peered":"2024-09-21T21:43:49.371417+0000","last_clean":"2024-09-21T21:43:49.371417+0000","last_became_active":"2024-09-21T21:41:52.209710+0000","last_became_peered":"2024-09-21T21:41:52.209710+0000","last_unstale":"2024-09-21T21:43:49.371417+0000","last_undegraded":"2024-09-21T21:43:49.371417+0000","last_fullsized":"2024-09-21T21:43:49.371417+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:40:36.400971+0000","last_clean_scrub_stamp":"2024-09-21T21:40:36.400971+0000","objects_scrubbed":0,"log_size":477,"log_dups_size":0,"ondisk_log_size":477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:20:59.585304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013513189999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":9647,"num_read_kb":76927,"num_write":508,"num_write_kb":12558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":146,"num_bytes_recovered":17387622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":241,"num_object_clones":0,"num_object_copies":482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":241,"num_whiteouts":237,"num_read":408853,"num_read_kb":1471230,"num_write":27874,"num_write_kb":319707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7824,"num_bytes_recovered":220201937,"num_keys_recovered":844,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24457,"ondisk_log_size":24457,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":102090,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":4,"up_from":13,"seq":55834574895,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55572,"kb_used_data":2172,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316268,"statfs":{"total":96636764160,"available":96579858432,"internally_reserved":0,"allocated":2224128,"data_stored":933356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574895,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87552,"kb_used_data":2216,"kb_used_omap":1,"kb_used_meta":85310,"kb_avail":94284288,"statfs":{"total":96636764160,"available":96547110912,"internally_reserved":0,"allocated":2269184,"data_stored":1523705,"data_compressed":34030,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1593,"internal_metadata":87357895},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":12,"seq":51539607599,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73620,"kb_used_data":2172,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298220,"statfs":{"total":96636764160,"available":96561377280,"internally_reserved":0,"allocated":2224128,"data_stored":933356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607599,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55576,"kb_used_data":2176,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316264,"statfs":{"total":96636764160,"available":96579854336,"internally_reserved":0,"allocated":2228224,"data_stored":933375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607600,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47296,"kb_used_data":2216,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324544,"statfs":{"total":96636764160,"available":96588333056,"internally_reserved":0,"allocated":2269184,"data_stored":1523705,"data_compressed":34030,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640304,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47240,"kb_used_data":2168,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324600,"statfs":{"total":96636764160,"available":96588390400,"internally_reserved":0,"allocated":2220032,"data_stored":933337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34030,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34030,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34030,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T21:43:56.556 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-09-21T21:40:36.400971+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=21, tm_min=40, tm_sec=36, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=21, tm_min=43, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T21:43:56.556 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-09-21T21:40:36.400971+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=21, tm_min=40, tm_sec=36, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=21, tm_min=43, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T21:43:56.556 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-09-21T21:40:36.400971+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=21, tm_min=40, tm_sec=36, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=21, tm_min=43, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T21:43:56.556 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-09-21T21:40:36.400971+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=21, tm_min=40, tm_sec=36, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=21, tm_min=43, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T21:43:56.556 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-09-21T21:40:36.400971+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=21, tm_min=40, tm_sec=36, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=21, tm_min=43, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T21:43:56.557 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-09-21T21:40:36.400971+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=21, tm_min=40, tm_sec=36, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=21, tm_min=43, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T21:43:56.557 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-09-21T21:40:36.400971+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=21, tm_min=40, tm_sec=36, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=21, tm_min=43, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T21:43:56.557 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-09-21T21:40:36.400971+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=21, tm_min=40, tm_sec=36, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=21, tm_min=43, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T21:43:56.557 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-09-21T21:40:36.400971+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=21, tm_min=40, tm_sec=36, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=21, tm_min=43, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T21:43:56.557 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-09-21T21:40:36.400971+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=21, tm_min=40, tm_sec=36, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=21, tm_min=43, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T21:43:56.557 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-09-21T21:40:36.400971+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=21, tm_min=40, tm_sec=36, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=21, tm_min=43, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T21:43:56.557 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-21T21:40:24.357997+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=21, tm_min=40, tm_sec=24, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=21, tm_min=43, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T21:43:56.557 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-09-21T21:40:36.400971+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=21, tm_min=40, tm_sec=36, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=21, tm_min=43, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T21:43:56.557 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-21T21:40:24.357997+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=21, tm_min=40, tm_sec=24, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=21, tm_min=43, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T21:43:56.557 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-09-21T21:40:36.400971+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=21, tm_min=40, tm_sec=36, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=21, tm_min=43, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T21:43:56.558 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-21T21:40:24.357997+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=21, tm_min=40, tm_sec=24, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=21, tm_min=43, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T21:43:56.558 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-09-21T21:40:36.400971+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=21, tm_min=40, tm_sec=36, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=21, tm_min=43, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T21:43:56.558 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-21T21:40:24.357997+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=21, tm_min=40, tm_sec=24, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=21, tm_min=43, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T21:43:56.558 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-09-21T21:40:36.400971+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=21, tm_min=40, tm_sec=36, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=21, tm_min=43, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T21:43:56.558 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-09-21T21:40:36.400971+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=21, tm_min=40, tm_sec=36, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=21, tm_min=43, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T21:43:56.558 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-21T21:40:24.357997+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=21, tm_min=40, tm_sec=24, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=21, tm_min=43, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T21:43:56.558 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-09-21T21:40:36.400971+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=21, tm_min=40, tm_sec=36, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=21, tm_min=43, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T21:43:56.558 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-09-21T21:40:36.400971+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=21, tm_min=40, tm_sec=36, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=21, tm_min=43, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T21:43:56.558 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-21T21:40:24.357997+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=21, tm_min=40, tm_sec=24, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=21, tm_min=43, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T21:43:56.559 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-09-21T21:40:36.400971+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=21, tm_min=40, tm_sec=36, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=21, tm_min=43, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T21:43:56.559 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-21T21:40:24.357997+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=21, tm_min=40, tm_sec=24, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=21, tm_min=43, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T21:43:56.559 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-21T21:40:18.340430+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=21, tm_min=40, tm_sec=18, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=21, tm_min=43, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T21:43:56.559 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-09-21T21:40:36.400971+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=21, tm_min=40, tm_sec=36, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=21, tm_min=43, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T21:43:56.559 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-09-21T21:40:36.400971+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=21, tm_min=40, tm_sec=36, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=21, tm_min=43, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T21:43:56.559 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-09-21T21:40:36.400971+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=21, tm_min=40, tm_sec=36, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=21, tm_min=43, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T21:43:56.559 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-09-21T21:40:36.400971+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=21, tm_min=40, tm_sec=36, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=21, tm_min=43, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T21:43:56.559 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-09-21T21:40:36.400971+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=21, tm_min=40, tm_sec=36, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=21, tm_min=43, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T21:43:56.559 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-09-21T21:40:36.400971+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=21, tm_min=40, tm_sec=36, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=21, tm_min=43, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T21:43:56.559 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-09-21T21:40:36.400971+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=21, tm_min=40, tm_sec=36, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=21, tm_min=43, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T21:43:56.560 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-09-21T21:40:36.400971+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=21, tm_min=40, tm_sec=36, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=21, tm_min=43, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T21:43:56.560 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-09-21T21:40:36.400971+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=21, tm_min=40, tm_sec=36, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=21, tm_min=43, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T21:43:56.560 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-09-21T21:40:36.400971+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=21, tm_min=40, tm_sec=36, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=21, tm_min=43, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T21:43:56.560 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-09-21T21:40:36.400971+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=21, tm_min=40, tm_sec=36, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=21, tm_min=43, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T21:43:56.560 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-09-21T21:40:36.400971+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=21, tm_min=40, tm_sec=36, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=21, tm_min=43, tm_sec=52, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T21:43:56.560 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-21T21:44:16.561 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T21:44:16.851 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-09-21T21:44:16.851 INFO:teuthology.orchestra.run.smithi081.stderr:dumped all 2024-09-21T21:44:16.863 INFO:teuthology.orchestra.run.smithi081.stdout:{"pg_ready":true,"pg_map":{"version":300,"stamp":"2024-09-21T21:44:16.330784+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":245,"num_object_clones":0,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":237,"num_read":408969,"num_read_kb":1471330,"num_write":28035,"num_write_kb":322125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7832,"num_bytes_recovered":222563409,"num_keys_recovered":844,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24810,"ondisk_log_size":24810,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":370976,"kb_used_data":13120,"kb_used_omap":9,"kb_used_meta":357750,"kb_avail":565860064,"statfs":{"total":579820584960,"available":579440705536,"internally_reserved":0,"allocated":13434880,"data_stored":6780834,"data_compressed":68060,"data_compressed_allocated":90112,"data_compressed_original":1179648,"omap_allocated":9568,"internal_metadata":366336672},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002746"},"pg_stats":[{"pgid":"3.1f","version":"185'597","reported_seq":7673,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:57.047799+0000","last_change":"2024-09-21T21:43:57.047855+0000","last_active":"2024-09-21T21:43:57.047799+0000","last_peered":"2024-09-21T21:43:57.047799+0000","last_clean":"2024-09-21T21:43:57.047799+0000","last_became_active":"2024-09-21T21:43:08.230509+0000","last_became_peered":"2024-09-21T21:43:08.230509+0000","last_unstale":"2024-09-21T21:43:57.047799+0000","last_undegraded":"2024-09-21T21:43:57.047799+0000","last_fullsized":"2024-09-21T21:43:57.047799+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"185'597","last_scrub_stamp":"2024-09-21T21:43:57.047715+0000","last_deep_scrub":"185'597","last_deep_scrub_stamp":"2024-09-21T21:43:57.047715+0000","last_clean_scrub_stamp":"2024-09-21T21:43:57.047715+0000","objects_scrubbed":6,"log_size":597,"log_dups_size":0,"ondisk_log_size":597,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:37:58.195920+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.0013233089999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":8384,"num_read_kb":24764,"num_write":554,"num_write_kb":8557,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":293,"num_bytes_recovered":3779776,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"185'1024","reported_seq":15348,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:56.254134+0000","last_change":"2024-09-21T21:43:56.254180+0000","last_active":"2024-09-21T21:43:56.254134+0000","last_peered":"2024-09-21T21:43:56.254134+0000","last_clean":"2024-09-21T21:43:56.254134+0000","last_became_active":"2024-09-21T21:43:44.356029+0000","last_became_peered":"2024-09-21T21:43:44.356029+0000","last_unstale":"2024-09-21T21:43:56.254134+0000","last_undegraded":"2024-09-21T21:43:56.254134+0000","last_fullsized":"2024-09-21T21:43:56.254134+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"185'1024","last_scrub_stamp":"2024-09-21T21:43:56.254044+0000","last_deep_scrub":"185'1024","last_deep_scrub_stamp":"2024-09-21T21:43:56.254044+0000","last_clean_scrub_stamp":"2024-09-21T21:43:56.254044+0000","objects_scrubbed":11,"log_size":1024,"log_dups_size":0,"ondisk_log_size":1024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:46:19.752736+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0.001190569,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":16219,"num_read_kb":73109,"num_write":1473,"num_write_kb":18401,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":14291598,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"185'996","reported_seq":10772,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:44:05.830366+0000","last_change":"2024-09-21T21:44:05.830412+0000","last_active":"2024-09-21T21:44:05.830366+0000","last_peered":"2024-09-21T21:44:05.830366+0000","last_clean":"2024-09-21T21:44:05.830366+0000","last_became_active":"2024-09-21T21:43:20.225483+0000","last_became_peered":"2024-09-21T21:43:20.225483+0000","last_unstale":"2024-09-21T21:44:05.830366+0000","last_undegraded":"2024-09-21T21:44:05.830366+0000","last_fullsized":"2024-09-21T21:44:05.830366+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"185'996","last_scrub_stamp":"2024-09-21T21:44:05.830268+0000","last_deep_scrub":"185'996","last_deep_scrub_stamp":"2024-09-21T21:44:05.830268+0000","last_clean_scrub_stamp":"2024-09-21T21:44:05.830268+0000","objects_scrubbed":11,"log_size":996,"log_dups_size":0,"ondisk_log_size":996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:19:25.560904+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0.00077268300000000003,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":13110,"num_read_kb":43443,"num_write":1158,"num_write_kb":7907,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":4009384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"185'892","reported_seq":13318,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:44:04.831591+0000","last_change":"2024-09-21T21:44:04.831633+0000","last_active":"2024-09-21T21:44:04.831591+0000","last_peered":"2024-09-21T21:44:04.831591+0000","last_clean":"2024-09-21T21:44:04.831591+0000","last_became_active":"2024-09-21T21:43:19.225435+0000","last_became_peered":"2024-09-21T21:43:19.225435+0000","last_unstale":"2024-09-21T21:44:04.831591+0000","last_undegraded":"2024-09-21T21:44:04.831591+0000","last_fullsized":"2024-09-21T21:44:04.831591+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"185'892","last_scrub_stamp":"2024-09-21T21:44:04.831501+0000","last_deep_scrub":"185'892","last_deep_scrub_stamp":"2024-09-21T21:44:04.831501+0000","last_clean_scrub_stamp":"2024-09-21T21:44:04.831501+0000","objects_scrubbed":8,"log_size":892,"log_dups_size":0,"ondisk_log_size":892,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:25:00.775430+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.001856124,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":15417,"num_read_kb":35250,"num_write":925,"num_write_kb":7812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":178,"num_bytes_recovered":1721983,"num_keys_recovered":148,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"185'743","reported_seq":9079,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:59.015590+0000","last_change":"2024-09-21T21:43:59.015622+0000","last_active":"2024-09-21T21:43:59.015590+0000","last_peered":"2024-09-21T21:43:59.015590+0000","last_clean":"2024-09-21T21:43:59.015590+0000","last_became_active":"2024-09-21T21:43:19.225304+0000","last_became_peered":"2024-09-21T21:43:19.225304+0000","last_unstale":"2024-09-21T21:43:59.015590+0000","last_undegraded":"2024-09-21T21:43:59.015590+0000","last_fullsized":"2024-09-21T21:43:59.015590+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"185'743","last_scrub_stamp":"2024-09-21T21:43:59.015487+0000","last_deep_scrub":"185'743","last_deep_scrub_stamp":"2024-09-21T21:43:59.015487+0000","last_clean_scrub_stamp":"2024-09-21T21:43:59.015487+0000","objects_scrubbed":6,"log_size":743,"log_dups_size":0,"ondisk_log_size":743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:26:00.002243+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0.001146085,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":8621,"num_read_kb":37221,"num_write":843,"num_write_kb":4971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":547,"num_bytes_recovered":1792422,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"185'630","reported_seq":25452,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:55.912177+0000","last_change":"2024-09-21T21:43:55.912228+0000","last_active":"2024-09-21T21:43:55.912177+0000","last_peered":"2024-09-21T21:43:55.912177+0000","last_clean":"2024-09-21T21:43:55.912177+0000","last_became_active":"2024-09-21T21:43:48.230767+0000","last_became_peered":"2024-09-21T21:43:48.230767+0000","last_unstale":"2024-09-21T21:43:55.912177+0000","last_undegraded":"2024-09-21T21:43:55.912177+0000","last_fullsized":"2024-09-21T21:43:55.912177+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"185'630","last_scrub_stamp":"2024-09-21T21:43:55.912088+0000","last_deep_scrub":"185'630","last_deep_scrub_stamp":"2024-09-21T21:43:55.912088+0000","last_clean_scrub_stamp":"2024-09-21T21:43:55.912088+0000","objects_scrubbed":12,"log_size":630,"log_dups_size":0,"ondisk_log_size":630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:56:19.164114+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0.0044588529999999996,"stat_sum":{"num_bytes":19,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":11,"num_read":25138,"num_read_kb":59201,"num_write":578,"num_write_kb":7782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":203,"num_bytes_recovered":4464880,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"185'875","reported_seq":14211,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:44:00.635465+0000","last_change":"2024-09-21T21:44:00.635515+0000","last_active":"2024-09-21T21:44:00.635465+0000","last_peered":"2024-09-21T21:44:00.635465+0000","last_clean":"2024-09-21T21:44:00.635465+0000","last_became_active":"2024-09-21T21:43:20.224845+0000","last_became_peered":"2024-09-21T21:43:20.224845+0000","last_unstale":"2024-09-21T21:44:00.635465+0000","last_undegraded":"2024-09-21T21:44:00.635465+0000","last_fullsized":"2024-09-21T21:44:00.635465+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"185'875","last_scrub_stamp":"2024-09-21T21:44:00.635380+0000","last_deep_scrub":"185'875","last_deep_scrub_stamp":"2024-09-21T21:44:00.635380+0000","last_clean_scrub_stamp":"2024-09-21T21:44:00.635380+0000","objects_scrubbed":15,"log_size":875,"log_dups_size":0,"ondisk_log_size":875,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:52:10.322341+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.0018111119999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":16015,"num_read_kb":61704,"num_write":858,"num_write_kb":15379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":9899814,"num_keys_recovered":59,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"185'339","reported_seq":6970,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:59.220256+0000","last_change":"2024-09-21T21:43:59.220288+0000","last_active":"2024-09-21T21:43:59.220256+0000","last_peered":"2024-09-21T21:43:59.220256+0000","last_clean":"2024-09-21T21:43:59.220256+0000","last_became_active":"2024-09-21T21:43:44.356203+0000","last_became_peered":"2024-09-21T21:43:44.356203+0000","last_unstale":"2024-09-21T21:43:59.220256+0000","last_undegraded":"2024-09-21T21:43:59.220256+0000","last_fullsized":"2024-09-21T21:43:59.220256+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"185'339","last_scrub_stamp":"2024-09-21T21:43:59.220166+0000","last_deep_scrub":"185'339","last_deep_scrub_stamp":"2024-09-21T21:43:59.220166+0000","last_clean_scrub_stamp":"2024-09-21T21:43:59.220166+0000","objects_scrubbed":4,"log_size":339,"log_dups_size":0,"ondisk_log_size":339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:28:36.638649+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.0012910630000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":6161,"num_read_kb":33023,"num_write":325,"num_write_kb":8639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":5854224,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"185'502","reported_seq":6433,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:56.009738+0000","last_change":"2024-09-21T21:43:56.009787+0000","last_active":"2024-09-21T21:43:56.009738+0000","last_peered":"2024-09-21T21:43:56.009738+0000","last_clean":"2024-09-21T21:43:56.009738+0000","last_became_active":"2024-09-21T21:43:07.232590+0000","last_became_peered":"2024-09-21T21:43:07.232590+0000","last_unstale":"2024-09-21T21:43:56.009738+0000","last_undegraded":"2024-09-21T21:43:56.009738+0000","last_fullsized":"2024-09-21T21:43:56.009738+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"185'502","last_scrub_stamp":"2024-09-21T21:43:56.009628+0000","last_deep_scrub":"185'502","last_deep_scrub_stamp":"2024-09-21T21:43:56.009628+0000","last_clean_scrub_stamp":"2024-09-21T21:43:56.009628+0000","objects_scrubbed":5,"log_size":502,"log_dups_size":0,"ondisk_log_size":502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:03:17.369124+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.001467229,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":6465,"num_read_kb":25955,"num_write":521,"num_write_kb":9178,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":203,"num_bytes_recovered":4149457,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"185'1244","reported_seq":9883,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:58.707786+0000","last_change":"2024-09-21T21:43:58.707820+0000","last_active":"2024-09-21T21:43:58.707786+0000","last_peered":"2024-09-21T21:43:58.707786+0000","last_clean":"2024-09-21T21:43:58.707786+0000","last_became_active":"2024-09-21T21:41:53.214270+0000","last_became_peered":"2024-09-21T21:41:53.214270+0000","last_unstale":"2024-09-21T21:43:58.707786+0000","last_undegraded":"2024-09-21T21:43:58.707786+0000","last_fullsized":"2024-09-21T21:43:58.707786+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"185'1244","last_scrub_stamp":"2024-09-21T21:43:58.707699+0000","last_deep_scrub":"185'1244","last_deep_scrub_stamp":"2024-09-21T21:43:58.707699+0000","last_clean_scrub_stamp":"2024-09-21T21:43:58.707699+0000","objects_scrubbed":6,"log_size":1244,"log_dups_size":0,"ondisk_log_size":1244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T01:18:01.841925+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0.0022124750000000002,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":13255,"num_read_kb":44399,"num_write":1796,"num_write_kb":12840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":148,"num_bytes_recovered":4869268,"num_keys_recovered":47,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"185'751","reported_seq":9506,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:44:00.983621+0000","last_change":"2024-09-21T21:44:00.983656+0000","last_active":"2024-09-21T21:44:00.983621+0000","last_peered":"2024-09-21T21:44:00.983621+0000","last_clean":"2024-09-21T21:44:00.983621+0000","last_became_active":"2024-09-21T21:43:07.228504+0000","last_became_peered":"2024-09-21T21:43:07.228504+0000","last_unstale":"2024-09-21T21:44:00.983621+0000","last_undegraded":"2024-09-21T21:44:00.983621+0000","last_fullsized":"2024-09-21T21:44:00.983621+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"185'751","last_scrub_stamp":"2024-09-21T21:44:00.983526+0000","last_deep_scrub":"185'751","last_deep_scrub_stamp":"2024-09-21T21:44:00.983526+0000","last_clean_scrub_stamp":"2024-09-21T21:44:00.983526+0000","objects_scrubbed":7,"log_size":751,"log_dups_size":0,"ondisk_log_size":751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T05:34:26.397632+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0.001650253,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":9201,"num_read_kb":56862,"num_write":979,"num_write_kb":7172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":520,"num_bytes_recovered":10162533,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"185'631","reported_seq":21910,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:44:04.940160+0000","last_change":"2024-09-21T21:44:04.940194+0000","last_active":"2024-09-21T21:44:04.940160+0000","last_peered":"2024-09-21T21:44:04.940160+0000","last_clean":"2024-09-21T21:44:04.940160+0000","last_became_active":"2024-09-21T21:43:48.228528+0000","last_became_peered":"2024-09-21T21:43:48.228528+0000","last_unstale":"2024-09-21T21:44:04.940160+0000","last_undegraded":"2024-09-21T21:44:04.940160+0000","last_fullsized":"2024-09-21T21:44:04.940160+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"185'631","last_scrub_stamp":"2024-09-21T21:44:04.940077+0000","last_deep_scrub":"185'631","last_deep_scrub_stamp":"2024-09-21T21:44:04.940077+0000","last_clean_scrub_stamp":"2024-09-21T21:44:04.940077+0000","objects_scrubbed":5,"log_size":631,"log_dups_size":0,"ondisk_log_size":631,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:03:47.428510+0000","scrub_duration":5,"objects_trimmed":2,"snaptrim_duration":0.00063765000000000002,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":22083,"num_read_kb":75165,"num_write":513,"num_write_kb":10980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":602,"num_bytes_recovered":22337258,"num_keys_recovered":84,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":246,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:44:01.934275+0000","last_change":"2024-09-21T21:44:01.934297+0000","last_active":"2024-09-21T21:44:01.934275+0000","last_peered":"2024-09-21T21:44:01.934275+0000","last_clean":"2024-09-21T21:44:01.934275+0000","last_became_active":"2024-09-21T21:43:15.316533+0000","last_became_peered":"2024-09-21T21:43:15.316533+0000","last_unstale":"2024-09-21T21:44:01.934275+0000","last_undegraded":"2024-09-21T21:44:01.934275+0000","last_fullsized":"2024-09-21T21:44:01.934275+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:44:01.934183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:44:01.934183+0000","last_clean_scrub_stamp":"2024-09-21T21:44:01.934183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:07:36.451057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"185'1233","reported_seq":14290,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:58.007968+0000","last_change":"2024-09-21T21:43:58.008011+0000","last_active":"2024-09-21T21:43:58.007968+0000","last_peered":"2024-09-21T21:43:58.007968+0000","last_clean":"2024-09-21T21:43:58.007968+0000","last_became_active":"2024-09-21T21:43:09.230849+0000","last_became_peered":"2024-09-21T21:43:09.230849+0000","last_unstale":"2024-09-21T21:43:58.007968+0000","last_undegraded":"2024-09-21T21:43:58.007968+0000","last_fullsized":"2024-09-21T21:43:58.007968+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"185'1233","last_scrub_stamp":"2024-09-21T21:43:58.007873+0000","last_deep_scrub":"185'1233","last_deep_scrub_stamp":"2024-09-21T21:43:58.007873+0000","last_clean_scrub_stamp":"2024-09-21T21:43:58.007873+0000","objects_scrubbed":8,"log_size":1233,"log_dups_size":0,"ondisk_log_size":1233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:22:36.914179+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.00096363700000000004,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":15823,"num_read_kb":66323,"num_write":1574,"num_write_kb":15987,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":349,"num_bytes_recovered":9257588,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":371,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:57.639746+0000","last_change":"2024-09-21T21:43:57.639773+0000","last_active":"2024-09-21T21:43:57.639746+0000","last_peered":"2024-09-21T21:43:57.639746+0000","last_clean":"2024-09-21T21:43:57.639746+0000","last_became_active":"2024-09-21T21:41:49.266257+0000","last_became_peered":"2024-09-21T21:41:49.266257+0000","last_unstale":"2024-09-21T21:43:57.639746+0000","last_undegraded":"2024-09-21T21:43:57.639746+0000","last_fullsized":"2024-09-21T21:43:57.639746+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:43:57.639651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:43:57.639651+0000","last_clean_scrub_stamp":"2024-09-21T21:43:57.639651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:02:13.072716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062280799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"185'561","reported_seq":8903,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:57.226124+0000","last_change":"2024-09-21T21:43:57.226157+0000","last_active":"2024-09-21T21:43:57.226124+0000","last_peered":"2024-09-21T21:43:57.226124+0000","last_clean":"2024-09-21T21:43:57.226124+0000","last_became_active":"2024-09-21T21:43:44.356543+0000","last_became_peered":"2024-09-21T21:43:44.356543+0000","last_unstale":"2024-09-21T21:43:57.226124+0000","last_undegraded":"2024-09-21T21:43:57.226124+0000","last_fullsized":"2024-09-21T21:43:57.226124+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"185'561","last_scrub_stamp":"2024-09-21T21:43:57.226046+0000","last_deep_scrub":"185'561","last_deep_scrub_stamp":"2024-09-21T21:43:57.226046+0000","last_clean_scrub_stamp":"2024-09-21T21:43:57.226046+0000","objects_scrubbed":6,"log_size":561,"log_dups_size":0,"ondisk_log_size":561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:32:05.888071+0000","scrub_duration":5,"objects_trimmed":2,"snaptrim_duration":0.00099396100000000002,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":10770,"num_read_kb":40597,"num_write":524,"num_write_kb":10606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":251,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:59.009807+0000","last_change":"2024-09-21T21:43:59.009833+0000","last_active":"2024-09-21T21:43:59.009807+0000","last_peered":"2024-09-21T21:43:59.009807+0000","last_clean":"2024-09-21T21:43:59.009807+0000","last_became_active":"2024-09-21T21:43:03.224515+0000","last_became_peered":"2024-09-21T21:43:03.224515+0000","last_unstale":"2024-09-21T21:43:59.009807+0000","last_undegraded":"2024-09-21T21:43:59.009807+0000","last_fullsized":"2024-09-21T21:43:59.009807+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:43:59.009714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:43:59.009714+0000","last_clean_scrub_stamp":"2024-09-21T21:43:59.009714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:22:27.015184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"185'884","reported_seq":4764,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:55.644217+0000","last_change":"2024-09-21T21:43:55.644244+0000","last_active":"2024-09-21T21:43:55.644217+0000","last_peered":"2024-09-21T21:43:55.644217+0000","last_clean":"2024-09-21T21:43:55.644217+0000","last_became_active":"2024-09-21T21:40:37.419944+0000","last_became_peered":"2024-09-21T21:40:37.419944+0000","last_unstale":"2024-09-21T21:43:55.644217+0000","last_undegraded":"2024-09-21T21:43:55.644217+0000","last_fullsized":"2024-09-21T21:43:55.644217+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"185'884","last_scrub_stamp":"2024-09-21T21:43:55.644147+0000","last_deep_scrub":"185'884","last_deep_scrub_stamp":"2024-09-21T21:43:55.644147+0000","last_clean_scrub_stamp":"2024-09-21T21:43:55.644147+0000","objects_scrubbed":4,"log_size":884,"log_dups_size":0,"ondisk_log_size":884,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:10:02.048604+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.0020772899999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":5927,"num_read_kb":25257,"num_write":1263,"num_write_kb":11321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":376,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:58.593575+0000","last_change":"2024-09-21T21:43:58.593613+0000","last_active":"2024-09-21T21:43:58.593575+0000","last_peered":"2024-09-21T21:43:58.593575+0000","last_clean":"2024-09-21T21:43:58.593575+0000","last_became_active":"2024-09-21T21:43:03.224572+0000","last_became_peered":"2024-09-21T21:43:03.224572+0000","last_unstale":"2024-09-21T21:43:58.593575+0000","last_undegraded":"2024-09-21T21:43:58.593575+0000","last_fullsized":"2024-09-21T21:43:58.593575+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:43:58.593480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:43:58.593480+0000","last_clean_scrub_stamp":"2024-09-21T21:43:58.593480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:13:44.721846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070669899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"185'645","reported_seq":10514,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:56.658570+0000","last_change":"2024-09-21T21:43:56.658638+0000","last_active":"2024-09-21T21:43:56.658570+0000","last_peered":"2024-09-21T21:43:56.658570+0000","last_clean":"2024-09-21T21:43:56.658570+0000","last_became_active":"2024-09-21T21:41:25.650379+0000","last_became_peered":"2024-09-21T21:41:25.650379+0000","last_unstale":"2024-09-21T21:43:56.658570+0000","last_undegraded":"2024-09-21T21:43:56.658570+0000","last_fullsized":"2024-09-21T21:43:56.658570+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"185'645","last_scrub_stamp":"2024-09-21T21:43:56.658436+0000","last_deep_scrub":"185'645","last_deep_scrub_stamp":"2024-09-21T21:43:56.658436+0000","last_clean_scrub_stamp":"2024-09-21T21:43:56.658436+0000","objects_scrubbed":8,"log_size":645,"log_dups_size":0,"ondisk_log_size":645,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T08:40:47.051148+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0.001695787,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":11334,"num_read_kb":32091,"num_write":515,"num_write_kb":8339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"185'652","reported_seq":11183,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:57.948257+0000","last_change":"2024-09-21T21:43:57.948296+0000","last_active":"2024-09-21T21:43:57.948257+0000","last_peered":"2024-09-21T21:43:57.948257+0000","last_clean":"2024-09-21T21:43:57.948257+0000","last_became_active":"2024-09-21T21:43:48.231457+0000","last_became_peered":"2024-09-21T21:43:48.231457+0000","last_unstale":"2024-09-21T21:43:57.948257+0000","last_undegraded":"2024-09-21T21:43:57.948257+0000","last_fullsized":"2024-09-21T21:43:57.948257+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"185'652","last_scrub_stamp":"2024-09-21T21:43:57.948176+0000","last_deep_scrub":"185'652","last_deep_scrub_stamp":"2024-09-21T21:43:57.948176+0000","last_clean_scrub_stamp":"2024-09-21T21:43:57.948176+0000","objects_scrubbed":7,"log_size":652,"log_dups_size":0,"ondisk_log_size":652,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T00:08:19.400899+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0.00081362800000000005,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":9832,"num_read_kb":35952,"num_write":639,"num_write_kb":9820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":437,"num_bytes_recovered":19367440,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"185'3","reported_seq":236,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:59.719750+0000","last_change":"2024-09-21T21:43:59.719802+0000","last_active":"2024-09-21T21:43:59.719750+0000","last_peered":"2024-09-21T21:43:59.719750+0000","last_clean":"2024-09-21T21:43:59.719750+0000","last_became_active":"2024-09-21T21:42:24.516623+0000","last_became_peered":"2024-09-21T21:42:24.516623+0000","last_unstale":"2024-09-21T21:43:59.719750+0000","last_undegraded":"2024-09-21T21:43:59.719750+0000","last_fullsized":"2024-09-21T21:43:59.719750+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"185'3","last_scrub_stamp":"2024-09-21T21:43:59.719649+0000","last_deep_scrub":"185'3","last_deep_scrub_stamp":"2024-09-21T21:43:59.719649+0000","last_clean_scrub_stamp":"2024-09-21T21:43:59.719649+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:35:50.053302+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.000906268,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"185'821","reported_seq":11456,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:54.616601+0000","last_change":"2024-09-21T21:43:54.616635+0000","last_active":"2024-09-21T21:43:54.616601+0000","last_peered":"2024-09-21T21:43:54.616601+0000","last_clean":"2024-09-21T21:43:54.616601+0000","last_became_active":"2024-09-21T21:40:37.418973+0000","last_became_peered":"2024-09-21T21:40:37.418973+0000","last_unstale":"2024-09-21T21:43:54.616601+0000","last_undegraded":"2024-09-21T21:43:54.616601+0000","last_fullsized":"2024-09-21T21:43:54.616601+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"185'821","last_scrub_stamp":"2024-09-21T21:43:54.616526+0000","last_deep_scrub":"185'821","last_deep_scrub_stamp":"2024-09-21T21:43:54.616526+0000","last_clean_scrub_stamp":"2024-09-21T21:43:54.616526+0000","objects_scrubbed":6,"log_size":821,"log_dups_size":0,"ondisk_log_size":821,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:51:20.767680+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.001131546,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":13425,"num_read_kb":28144,"num_write":602,"num_write_kb":6293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":116,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:55.285108+0000","last_change":"2024-09-21T21:43:55.285149+0000","last_active":"2024-09-21T21:43:55.285108+0000","last_peered":"2024-09-21T21:43:55.285108+0000","last_clean":"2024-09-21T21:43:55.285108+0000","last_became_active":"2024-09-21T21:43:44.357116+0000","last_became_peered":"2024-09-21T21:43:44.357116+0000","last_unstale":"2024-09-21T21:43:55.285108+0000","last_undegraded":"2024-09-21T21:43:55.285108+0000","last_fullsized":"2024-09-21T21:43:55.285108+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:43:55.285016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:43:55.285016+0000","last_clean_scrub_stamp":"2024-09-21T21:43:55.285016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:12:30.936177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"185'402","reported_seq":16907,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:44:00.049740+0000","last_change":"2024-09-21T21:44:00.049799+0000","last_active":"2024-09-21T21:44:00.049740+0000","last_peered":"2024-09-21T21:44:00.049740+0000","last_clean":"2024-09-21T21:44:00.049740+0000","last_became_active":"2024-09-21T21:43:20.224689+0000","last_became_peered":"2024-09-21T21:43:20.224689+0000","last_unstale":"2024-09-21T21:44:00.049740+0000","last_undegraded":"2024-09-21T21:44:00.049740+0000","last_fullsized":"2024-09-21T21:44:00.049740+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"185'402","last_scrub_stamp":"2024-09-21T21:44:00.049645+0000","last_deep_scrub":"185'402","last_deep_scrub_stamp":"2024-09-21T21:44:00.049645+0000","last_clean_scrub_stamp":"2024-09-21T21:44:00.049645+0000","objects_scrubbed":15,"log_size":402,"log_dups_size":0,"ondisk_log_size":402,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:39:59.074597+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0.00061419100000000004,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":15397,"num_read_kb":48505,"num_write":377,"num_write_kb":4859,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":429,"num_bytes_recovered":6598898,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":236,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:56.046180+0000","last_change":"2024-09-21T21:43:56.046226+0000","last_active":"2024-09-21T21:43:56.046180+0000","last_peered":"2024-09-21T21:43:56.046180+0000","last_clean":"2024-09-21T21:43:56.046180+0000","last_became_active":"2024-09-21T21:41:49.265182+0000","last_became_peered":"2024-09-21T21:41:49.265182+0000","last_unstale":"2024-09-21T21:43:56.046180+0000","last_undegraded":"2024-09-21T21:43:56.046180+0000","last_fullsized":"2024-09-21T21:43:56.046180+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T21:43:56.046077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T21:43:56.046077+0000","last_clean_scrub_stamp":"2024-09-21T21:43:56.046077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:18:59.003965+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"185'781","reported_seq":10319,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:56.659434+0000","last_change":"2024-09-21T21:43:56.659463+0000","last_active":"2024-09-21T21:43:56.659434+0000","last_peered":"2024-09-21T21:43:56.659434+0000","last_clean":"2024-09-21T21:43:56.659434+0000","last_became_active":"2024-09-21T21:40:37.421655+0000","last_became_peered":"2024-09-21T21:40:37.421655+0000","last_unstale":"2024-09-21T21:43:56.659434+0000","last_undegraded":"2024-09-21T21:43:56.659434+0000","last_fullsized":"2024-09-21T21:43:56.659434+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"185'781","last_scrub_stamp":"2024-09-21T21:43:56.659359+0000","last_deep_scrub":"185'781","last_deep_scrub_stamp":"2024-09-21T21:43:56.659359+0000","last_clean_scrub_stamp":"2024-09-21T21:43:56.659359+0000","objects_scrubbed":8,"log_size":781,"log_dups_size":0,"ondisk_log_size":781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:06:39.236159+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.0031354970000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":12713,"num_read_kb":34378,"num_write":754,"num_write_kb":7498,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"185'2","reported_seq":334,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:44:00.761823+0000","last_change":"2024-09-21T21:44:00.761853+0000","last_active":"2024-09-21T21:44:00.761823+0000","last_peered":"2024-09-21T21:44:00.761823+0000","last_clean":"2024-09-21T21:44:00.761823+0000","last_became_active":"2024-09-21T21:42:24.518461+0000","last_became_peered":"2024-09-21T21:42:24.518461+0000","last_unstale":"2024-09-21T21:44:00.761823+0000","last_undegraded":"2024-09-21T21:44:00.761823+0000","last_fullsized":"2024-09-21T21:44:00.761823+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"185'2","last_scrub_stamp":"2024-09-21T21:44:00.761710+0000","last_deep_scrub":"185'2","last_deep_scrub_stamp":"2024-09-21T21:44:00.761710+0000","last_clean_scrub_stamp":"2024-09-21T21:44:00.761710+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T21:57:42.819855+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.00036803999999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"185'119","reported_seq":445,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:59.986239+0000","last_change":"2024-09-21T21:43:59.986289+0000","last_active":"2024-09-21T21:43:59.986239+0000","last_peered":"2024-09-21T21:43:59.986239+0000","last_clean":"2024-09-21T21:43:59.986239+0000","last_became_active":"2024-09-21T21:43:06.231241+0000","last_became_peered":"2024-09-21T21:43:06.231241+0000","last_unstale":"2024-09-21T21:43:59.986239+0000","last_undegraded":"2024-09-21T21:43:59.986239+0000","last_fullsized":"2024-09-21T21:43:59.986239+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"185'119","last_scrub_stamp":"2024-09-21T21:43:59.986153+0000","last_deep_scrub":"185'119","last_deep_scrub_stamp":"2024-09-21T21:43:59.986153+0000","last_clean_scrub_stamp":"2024-09-21T21:43:59.986153+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:00:52.409885+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"185'381","reported_seq":6288,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:58.203807+0000","last_change":"2024-09-21T21:43:58.203852+0000","last_active":"2024-09-21T21:43:58.203807+0000","last_peered":"2024-09-21T21:43:58.203807+0000","last_clean":"2024-09-21T21:43:58.203807+0000","last_became_active":"2024-09-21T21:43:44.356024+0000","last_became_peered":"2024-09-21T21:43:44.356024+0000","last_unstale":"2024-09-21T21:43:58.203807+0000","last_undegraded":"2024-09-21T21:43:58.203807+0000","last_fullsized":"2024-09-21T21:43:58.203807+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"185'381","last_scrub_stamp":"2024-09-21T21:43:58.203719+0000","last_deep_scrub":"185'381","last_deep_scrub_stamp":"2024-09-21T21:43:58.203719+0000","last_clean_scrub_stamp":"2024-09-21T21:43:58.203719+0000","objects_scrubbed":7,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T02:31:59.336527+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.00094280000000000004,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":5268,"num_read_kb":25972,"num_write":367,"num_write_kb":7278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":172,"num_bytes_recovered":2229995,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"185'563","reported_seq":8644,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:56.908066+0000","last_change":"2024-09-21T21:43:56.908105+0000","last_active":"2024-09-21T21:43:56.908066+0000","last_peered":"2024-09-21T21:43:56.908066+0000","last_clean":"2024-09-21T21:43:56.908066+0000","last_became_active":"2024-09-21T21:43:48.230703+0000","last_became_peered":"2024-09-21T21:43:48.230703+0000","last_unstale":"2024-09-21T21:43:56.908066+0000","last_undegraded":"2024-09-21T21:43:56.908066+0000","last_fullsized":"2024-09-21T21:43:56.908066+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"185'563","last_scrub_stamp":"2024-09-21T21:43:56.907983+0000","last_deep_scrub":"185'563","last_deep_scrub_stamp":"2024-09-21T21:43:56.907983+0000","last_clean_scrub_stamp":"2024-09-21T21:43:56.907983+0000","objects_scrubbed":9,"log_size":563,"log_dups_size":0,"ondisk_log_size":563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T22:23:56.363105+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0.00071450699999999997,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":8813,"num_read_kb":47394,"num_write":525,"num_write_kb":9434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":241,"num_bytes_recovered":7579476,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"185'1117","reported_seq":11686,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:44:02.903966+0000","last_change":"2024-09-21T21:44:02.903966+0000","last_active":"2024-09-21T21:44:02.903966+0000","last_peered":"2024-09-21T21:44:02.903966+0000","last_clean":"2024-09-21T21:44:02.903966+0000","last_became_active":"2024-09-21T21:43:19.224644+0000","last_became_peered":"2024-09-21T21:43:19.224644+0000","last_unstale":"2024-09-21T21:44:02.903966+0000","last_undegraded":"2024-09-21T21:44:02.903966+0000","last_fullsized":"2024-09-21T21:44:02.903966+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"185'1117","last_scrub_stamp":"2024-09-21T21:44:02.903793+0000","last_deep_scrub":"185'1117","last_deep_scrub_stamp":"2024-09-21T21:44:02.903793+0000","last_clean_scrub_stamp":"2024-09-21T21:44:02.903793+0000","objects_scrubbed":6,"log_size":1117,"log_dups_size":0,"ondisk_log_size":1117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:31:36.468040+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.001226878,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":5,"num_read":13231,"num_read_kb":49174,"num_write":1280,"num_write_kb":10665,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":398,"num_bytes_recovered":16632560,"num_keys_recovered":144,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"185'784","reported_seq":10324,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:59.641822+0000","last_change":"2024-09-21T21:43:59.641862+0000","last_active":"2024-09-21T21:43:59.641822+0000","last_peered":"2024-09-21T21:43:59.641822+0000","last_clean":"2024-09-21T21:43:59.641822+0000","last_became_active":"2024-09-21T21:43:09.227569+0000","last_became_peered":"2024-09-21T21:43:09.227569+0000","last_unstale":"2024-09-21T21:43:59.641822+0000","last_undegraded":"2024-09-21T21:43:59.641822+0000","last_fullsized":"2024-09-21T21:43:59.641822+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"185'784","last_scrub_stamp":"2024-09-21T21:43:59.641731+0000","last_deep_scrub":"185'784","last_deep_scrub_stamp":"2024-09-21T21:43:59.641731+0000","last_clean_scrub_stamp":"2024-09-21T21:43:59.641731+0000","objects_scrubbed":7,"log_size":784,"log_dups_size":0,"ondisk_log_size":784,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T03:41:28.312286+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.0024712610000000002,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":10244,"num_read_kb":45368,"num_write":1134,"num_write_kb":14646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":331,"num_bytes_recovered":8931668,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"185'1628","reported_seq":14043,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:44:02.834832+0000","last_change":"2024-09-21T21:44:02.834913+0000","last_active":"2024-09-21T21:44:02.834832+0000","last_peered":"2024-09-21T21:44:02.834832+0000","last_clean":"2024-09-21T21:44:02.834832+0000","last_became_active":"2024-09-21T21:43:07.230148+0000","last_became_peered":"2024-09-21T21:43:07.230148+0000","last_unstale":"2024-09-21T21:44:02.834832+0000","last_undegraded":"2024-09-21T21:44:02.834832+0000","last_fullsized":"2024-09-21T21:44:02.834832+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"185'1628","last_scrub_stamp":"2024-09-21T21:44:02.834706+0000","last_deep_scrub":"185'1628","last_deep_scrub_stamp":"2024-09-21T21:44:02.834706+0000","last_clean_scrub_stamp":"2024-09-21T21:44:02.834706+0000","objects_scrubbed":3,"log_size":1628,"log_dups_size":0,"ondisk_log_size":1628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T07:48:33.818273+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.0021897029999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":18573,"num_read_kb":61318,"num_write":2350,"num_write_kb":11538,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":225,"num_bytes_recovered":7091105,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"185'1455","reported_seq":17779,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:58.001304+0000","last_change":"2024-09-21T21:43:58.001304+0000","last_active":"2024-09-21T21:43:58.001304+0000","last_peered":"2024-09-21T21:43:58.001304+0000","last_clean":"2024-09-21T21:43:58.001304+0000","last_became_active":"2024-09-21T21:41:52.214022+0000","last_became_peered":"2024-09-21T21:41:52.214022+0000","last_unstale":"2024-09-21T21:43:58.001304+0000","last_undegraded":"2024-09-21T21:43:58.001304+0000","last_fullsized":"2024-09-21T21:43:58.001304+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"185'1455","last_scrub_stamp":"2024-09-21T21:43:58.001174+0000","last_deep_scrub":"185'1455","last_deep_scrub_stamp":"2024-09-21T21:43:58.001174+0000","last_clean_scrub_stamp":"2024-09-21T21:43:58.001174+0000","objects_scrubbed":8,"log_size":1455,"log_dups_size":0,"ondisk_log_size":1455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:22:46.763848+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0.0016156199999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":21216,"num_read_kb":45587,"num_write":1846,"num_write_kb":14797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":149,"num_bytes_recovered":3462638,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"185'465","reported_seq":7929,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:44:03.828347+0000","last_change":"2024-09-21T21:44:03.828397+0000","last_active":"2024-09-21T21:44:03.828347+0000","last_peered":"2024-09-21T21:44:03.828347+0000","last_clean":"2024-09-21T21:44:03.828347+0000","last_became_active":"2024-09-21T21:43:18.223755+0000","last_became_peered":"2024-09-21T21:43:18.223755+0000","last_unstale":"2024-09-21T21:44:03.828347+0000","last_undegraded":"2024-09-21T21:44:03.828347+0000","last_fullsized":"2024-09-21T21:44:03.828347+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"185'465","last_scrub_stamp":"2024-09-21T21:44:03.828255+0000","last_deep_scrub":"185'465","last_deep_scrub_stamp":"2024-09-21T21:44:03.828255+0000","last_clean_scrub_stamp":"2024-09-21T21:44:03.828255+0000","objects_scrubbed":11,"log_size":465,"log_dups_size":0,"ondisk_log_size":465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T04:58:05.820529+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.0018835950000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":6571,"num_read_kb":34865,"num_write":469,"num_write_kb":6862,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":238,"num_bytes_recovered":4045989,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"185'745","reported_seq":17149,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:57.686708+0000","last_change":"2024-09-21T21:43:57.686741+0000","last_active":"2024-09-21T21:43:57.686708+0000","last_peered":"2024-09-21T21:43:57.686708+0000","last_clean":"2024-09-21T21:43:57.686708+0000","last_became_active":"2024-09-21T21:41:52.212206+0000","last_became_peered":"2024-09-21T21:41:52.212206+0000","last_unstale":"2024-09-21T21:43:57.686708+0000","last_undegraded":"2024-09-21T21:43:57.686708+0000","last_fullsized":"2024-09-21T21:43:57.686708+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"185'745","last_scrub_stamp":"2024-09-21T21:43:57.686624+0000","last_deep_scrub":"185'745","last_deep_scrub_stamp":"2024-09-21T21:43:57.686624+0000","last_clean_scrub_stamp":"2024-09-21T21:43:57.686624+0000","objects_scrubbed":4,"log_size":745,"log_dups_size":0,"ondisk_log_size":745,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T23:00:33.090809+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0013653929999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":18315,"num_read_kb":42344,"num_write":672,"num_write_kb":9505,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":173,"num_bytes_recovered":2967886,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"185'414","reported_seq":14267,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:44:01.789434+0000","last_change":"2024-09-21T21:44:01.789473+0000","last_active":"2024-09-21T21:44:01.789434+0000","last_peered":"2024-09-21T21:44:01.789434+0000","last_clean":"2024-09-21T21:44:01.789434+0000","last_became_active":"2024-09-21T21:43:06.233222+0000","last_became_peered":"2024-09-21T21:43:06.233222+0000","last_unstale":"2024-09-21T21:44:01.789434+0000","last_undegraded":"2024-09-21T21:44:01.789434+0000","last_fullsized":"2024-09-21T21:44:01.789434+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"185'414","last_scrub_stamp":"2024-09-21T21:44:01.789344+0000","last_deep_scrub":"185'414","last_deep_scrub_stamp":"2024-09-21T21:44:01.789344+0000","last_clean_scrub_stamp":"2024-09-21T21:44:01.789344+0000","objects_scrubbed":6,"log_size":414,"log_dups_size":0,"ondisk_log_size":414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:41:16.953420+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0.00085383799999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":12974,"num_read_kb":55237,"num_write":424,"num_write_kb":11231,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":205,"num_bytes_recovered":6455839,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"185'647","reported_seq":8523,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:44:03.947679+0000","last_change":"2024-09-21T21:44:03.947679+0000","last_active":"2024-09-21T21:44:03.947679+0000","last_peered":"2024-09-21T21:44:03.947679+0000","last_clean":"2024-09-21T21:44:03.947679+0000","last_became_active":"2024-09-21T21:43:20.224840+0000","last_became_peered":"2024-09-21T21:43:20.224840+0000","last_unstale":"2024-09-21T21:44:03.947679+0000","last_undegraded":"2024-09-21T21:44:03.947679+0000","last_fullsized":"2024-09-21T21:44:03.947679+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"185'647","last_scrub_stamp":"2024-09-21T21:44:03.947504+0000","last_deep_scrub":"185'647","last_deep_scrub_stamp":"2024-09-21T21:44:03.947504+0000","last_clean_scrub_stamp":"2024-09-21T21:44:03.947504+0000","objects_scrubbed":6,"log_size":647,"log_dups_size":0,"ondisk_log_size":647,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T09:04:08.460617+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0.00091468500000000004,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":7916,"num_read_kb":33911,"num_write":730,"num_write_kb":6868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":492,"num_bytes_recovered":3936953,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"185'800","reported_seq":19373,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:54.976489+0000","last_change":"2024-09-21T21:43:54.976518+0000","last_active":"2024-09-21T21:43:54.976489+0000","last_peered":"2024-09-21T21:43:54.976489+0000","last_clean":"2024-09-21T21:43:54.976489+0000","last_became_active":"2024-09-21T21:43:07.229497+0000","last_became_peered":"2024-09-21T21:43:07.229497+0000","last_unstale":"2024-09-21T21:43:54.976489+0000","last_undegraded":"2024-09-21T21:43:54.976489+0000","last_fullsized":"2024-09-21T21:43:54.976489+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"185'800","last_scrub_stamp":"2024-09-21T21:43:54.976420+0000","last_deep_scrub":"185'800","last_deep_scrub_stamp":"2024-09-21T21:43:54.976420+0000","last_clean_scrub_stamp":"2024-09-21T21:43:54.976420+0000","objects_scrubbed":9,"log_size":800,"log_dups_size":0,"ondisk_log_size":800,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:27:26.181818+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.00061499099999999995,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":20795,"num_read_kb":71790,"num_write":798,"num_write_kb":9984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":352,"num_bytes_recovered":16923683,"num_keys_recovered":92,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"185'484","reported_seq":10669,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-21T21:43:57.026342+0000","last_change":"2024-09-21T21:43:57.026342+0000","last_active":"2024-09-21T21:43:57.026342+0000","last_peered":"2024-09-21T21:43:57.026342+0000","last_clean":"2024-09-21T21:43:57.026342+0000","last_became_active":"2024-09-21T21:41:52.209710+0000","last_became_peered":"2024-09-21T21:41:52.209710+0000","last_unstale":"2024-09-21T21:43:57.026342+0000","last_undegraded":"2024-09-21T21:43:57.026342+0000","last_fullsized":"2024-09-21T21:43:57.026342+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"185'484","last_scrub_stamp":"2024-09-21T21:43:57.026205+0000","last_deep_scrub":"185'484","last_deep_scrub_stamp":"2024-09-21T21:43:57.026205+0000","last_clean_scrub_stamp":"2024-09-21T21:43:57.026205+0000","objects_scrubbed":7,"log_size":484,"log_dups_size":0,"ondisk_log_size":484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-23T06:13:31.856518+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.0013513189999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":9647,"num_read_kb":76927,"num_write":508,"num_write_kb":12558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":146,"num_bytes_recovered":17387622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":241,"num_object_clones":0,"num_object_copies":482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":241,"num_whiteouts":237,"num_read":408853,"num_read_kb":1471230,"num_write":27874,"num_write_kb":319707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7824,"num_bytes_recovered":220201937,"num_keys_recovered":844,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24686,"ondisk_log_size":24686,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":102090,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":4,"up_from":13,"seq":55834574899,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55564,"kb_used_data":2172,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316276,"statfs":{"total":96636764160,"available":96579866624,"internally_reserved":0,"allocated":2224128,"data_stored":933356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574899,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87544,"kb_used_data":2216,"kb_used_omap":1,"kb_used_meta":85310,"kb_avail":94284296,"statfs":{"total":96636764160,"available":96547119104,"internally_reserved":0,"allocated":2269184,"data_stored":1523705,"data_compressed":34030,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1593,"internal_metadata":87357895},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":12,"seq":51539607603,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73612,"kb_used_data":2172,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298228,"statfs":{"total":96636764160,"available":96561385472,"internally_reserved":0,"allocated":2224128,"data_stored":933356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607603,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55568,"kb_used_data":2176,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316272,"statfs":{"total":96636764160,"available":96579862528,"internally_reserved":0,"allocated":2228224,"data_stored":933375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607604,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51448,"kb_used_data":2216,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320392,"statfs":{"total":96636764160,"available":96584081408,"internally_reserved":0,"allocated":2269184,"data_stored":1523705,"data_compressed":34030,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640309,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47240,"kb_used_data":2168,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324600,"statfs":{"total":96636764160,"available":96588390400,"internally_reserved":0,"allocated":2220032,"data_stored":933337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34030,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34030,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34030,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T21:44:16.866 DEBUG:teuthology.orchestra.run.smithi081:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-21T21:44:17.156 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-21T21:44:17.157 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-21T21:44:17.157 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-21T21:44:17.157 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T21:44:17.199 INFO:tasks.ceph.osd.0:Stopped 2024-09-21T21:44:17.199 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-21T21:44:17.200 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T21:44:17.256 INFO:tasks.ceph.osd.1:Stopped 2024-09-21T21:44:17.256 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-21T21:44:17.256 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T21:44:17.303 INFO:tasks.ceph.osd.2:Stopped 2024-09-21T21:44:17.303 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-21T21:44:17.303 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T21:44:17.346 INFO:tasks.ceph.osd.3:Stopped 2024-09-21T21:44:17.347 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-21T21:44:17.347 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T21:44:17.393 INFO:tasks.ceph.osd.4:Stopped 2024-09-21T21:44:17.393 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-21T21:44:17.393 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T21:44:17.433 INFO:tasks.ceph.osd.5:Stopped 2024-09-21T21:44:17.433 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-21T21:44:17.433 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-21T21:44:17.433 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T21:44:17.501 INFO:tasks.ceph.mgr.x:Stopped 2024-09-21T21:44:17.501 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-21T21:44:17.501 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-21T21:44:17.501 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T21:44:17.513 INFO:tasks.ceph.mon.a:Stopped 2024-09-21T21:44:17.513 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-21T21:44:17.513 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T21:44:17.563 INFO:tasks.ceph.mon.c:Stopped 2024-09-21T21:44:17.564 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-21T21:44:17.564 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T21:44:17.576 INFO:tasks.ceph.mon.b:Stopped 2024-09-21T21:44:17.576 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-21T21:44:17.576 DEBUG:teuthology.orchestra.run.smithi081:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v MON_DOWN | egrep -v OSDMAP_FLAGS | egrep -v OSD_DOWN | egrep -v PG_DEGRADED | egrep -v PG_AVAILABILITY | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'mons down' | egrep -v 'mon down' | egrep -v 'out of quorum' | egrep -v noscrub | egrep -v nodeep-scrub | head -n 1 2024-09-21T21:44:17.644 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi081.front.sepia.ceph.com 2024-09-21T21:44:17.644 DEBUG:teuthology.orchestra.run.smithi081:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-21T21:44:17.867 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi081.front.sepia.ceph.com 2024-09-21T21:44:17.867 DEBUG:teuthology.orchestra.run.smithi081:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-21T21:44:17.956 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi081.front.sepia.ceph.com 2024-09-21T21:44:17.956 DEBUG:teuthology.orchestra.run.smithi081:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-21T21:44:18.012 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi122.front.sepia.ceph.com 2024-09-21T21:44:18.012 DEBUG:teuthology.orchestra.run.smithi122:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-21T21:44:18.310 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi122.front.sepia.ceph.com 2024-09-21T21:44:18.310 DEBUG:teuthology.orchestra.run.smithi122:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-21T21:44:18.390 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi122.front.sepia.ceph.com 2024-09-21T21:44:18.390 DEBUG:teuthology.orchestra.run.smithi122:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-21T21:44:18.447 INFO:tasks.ceph:Archiving mon data... 2024-09-21T21:44:18.448 DEBUG:teuthology.misc:Transferring archived files from smithi081:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914423/data/mon.a.tgz 2024-09-21T21:44:18.448 DEBUG:teuthology.orchestra.run.smithi081:> mktemp 2024-09-21T21:44:18.464 INFO:teuthology.orchestra.run.smithi081.stdout:/tmp/tmp.DtlmsSwiVO 2024-09-21T21:44:18.464 DEBUG:teuthology.orchestra.run.smithi081:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.DtlmsSwiVO 2024-09-21T21:44:18.614 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod 0666 /tmp/tmp.DtlmsSwiVO 2024-09-21T21:44:18.697 DEBUG:teuthology.orchestra.remote:smithi081:/tmp/tmp.DtlmsSwiVO is 351KB 2024-09-21T21:44:18.792 DEBUG:teuthology.orchestra.run.smithi081:> rm -fr /tmp/tmp.DtlmsSwiVO 2024-09-21T21:44:18.807 DEBUG:teuthology.misc:Transferring archived files from smithi081:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914423/data/mon.c.tgz 2024-09-21T21:44:18.807 DEBUG:teuthology.orchestra.run.smithi081:> mktemp 2024-09-21T21:44:18.862 INFO:teuthology.orchestra.run.smithi081.stdout:/tmp/tmp.fUcBL6Yzqr 2024-09-21T21:44:18.862 DEBUG:teuthology.orchestra.run.smithi081:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.fUcBL6Yzqr 2024-09-21T21:44:19.023 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod 0666 /tmp/tmp.fUcBL6Yzqr 2024-09-21T21:44:19.107 DEBUG:teuthology.orchestra.remote:smithi081:/tmp/tmp.fUcBL6Yzqr is 476KB 2024-09-21T21:44:19.203 DEBUG:teuthology.orchestra.run.smithi081:> rm -fr /tmp/tmp.fUcBL6Yzqr 2024-09-21T21:44:19.218 DEBUG:teuthology.misc:Transferring archived files from smithi122:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914423/data/mon.b.tgz 2024-09-21T21:44:19.218 DEBUG:teuthology.orchestra.run.smithi122:> mktemp 2024-09-21T21:44:19.236 INFO:teuthology.orchestra.run.smithi122.stdout:/tmp/tmp.xdOrnJZAAx 2024-09-21T21:44:19.236 DEBUG:teuthology.orchestra.run.smithi122:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.xdOrnJZAAx 2024-09-21T21:44:19.415 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 0666 /tmp/tmp.xdOrnJZAAx 2024-09-21T21:44:19.498 DEBUG:teuthology.orchestra.remote:smithi122:/tmp/tmp.xdOrnJZAAx is 476KB 2024-09-21T21:44:19.595 DEBUG:teuthology.orchestra.run.smithi122:> rm -fr /tmp/tmp.xdOrnJZAAx 2024-09-21T21:44:19.610 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-21T21:44:19.610 DEBUG:teuthology.orchestra.run.smithi081:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-21T21:44:19.612 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-21T21:44:19.652 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-21T21:44:19.700 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-21T21:44:19.730 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-21T21:44:19.748 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-21T21:44:19.748 INFO:tasks.ceph:Archiving crash dumps... 2024-09-21T21:44:19.749 DEBUG:teuthology.misc:Transferring archived files from smithi081:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914423/remote/smithi081/crash 2024-09-21T21:44:19.750 DEBUG:teuthology.orchestra.run.smithi081:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-21T21:44:19.775 DEBUG:teuthology.misc:Transferring archived files from smithi122:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914423/remote/smithi122/crash 2024-09-21T21:44:19.776 DEBUG:teuthology.orchestra.run.smithi122:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-21T21:44:19.799 DEBUG:teuthology.misc:Transferring archived files from smithi164:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914423/remote/smithi164/crash 2024-09-21T21:44:19.800 DEBUG:teuthology.orchestra.run.smithi164:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-21T21:44:19.844 INFO:tasks.ceph:Compressing logs... 2024-09-21T21:44:19.844 DEBUG:teuthology.orchestra.run.smithi081:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T21:44:19.847 DEBUG:teuthology.orchestra.run.smithi122:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T21:44:19.849 DEBUG:teuthology.orchestra.run.smithi164:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T21:44:19.869 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-21T21:44:19.870 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55380.log 2024-09-21T21:44:19.870 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49472.log/var/log/ceph/ceph-osd.0.log: 2024-09-21T21:44:19.877 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-21T21:44:19.877 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-21T21:44:19.877 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31918.log 2024-09-21T21:44:19.877 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-osd.4.log: /var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32054.log 2024-09-21T21:44:19.877 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.31918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31958.log 2024-09-21T21:44:19.877 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31918.log.gz 2024-09-21T21:44:19.877 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.32054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32014.log 2024-09-21T21:44:19.877 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32054.log.gz 2024-09-21T21:44:19.877 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.31958.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.32134.log -- replaced with /var/log/ceph/ceph-client.admin.31958.log.gz 2024-09-21T21:44:19.878 INFO:teuthology.orchestra.run.smithi122.stderr: 2024-09-21T21:44:19.878 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.32014.log: 89.1% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-21T21:44:19.878 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.32014.log.gz 2024-09-21T21:44:19.878 INFO:teuthology.orchestra.run.smithi122.stderr: /var/log/ceph/ceph-osd.5.log 2024-09-21T21:44:19.878 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.32134.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-21T21:44:19.878 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32134.log.gz 2024-09-21T21:44:19.878 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-21T21:44:19.878 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32094.log 2024-09-21T21:44:19.878 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-21T21:44:19.878 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.32094.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-21T21:44:19.878 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32094.log.gz 2024-09-21T21:44:19.878 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-mon.b.log: 84.7% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-21T21:44:19.954 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-osd.3.log: 91.7% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-21T21:44:20.030 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30402.log 2024-09-21T21:44:20.030 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29596.log 2024-09-21T21:44:20.030 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27493.log 2024-09-21T21:44:20.030 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.30402.log: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.30402.log.gz 2024-09-21T21:44:20.031 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.29596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28552.log 2024-09-21T21:44:20.031 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29596.log.gz 2024-09-21T21:44:20.031 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28938.log 2024-09-21T21:44:20.031 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.27493.log.gz 2024-09-21T21:44:20.031 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29991.log 2024-09-21T21:44:20.031 INFO:teuthology.orchestra.run.smithi164.stderr: 25.3% -- replaced with /var/log/ceph/ceph-client.admin.28552.log.gz 2024-09-21T21:44:20.031 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28938.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28972.log 2024-09-21T21:44:20.031 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.29991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27529.log 2024-09-21T21:44:20.031 INFO:teuthology.orchestra.run.smithi164.stderr: 95.5% -- replaced with /var/log/ceph/ceph-client.admin.28938.log.gz 2024-09-21T21:44:20.031 INFO:teuthology.orchestra.run.smithi164.stderr: 95.2% -- replaced with /var/log/ceph/ceph-client.admin.29991.log.gz 2024-09-21T21:44:20.031 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29243.log 2024-09-21T21:44:20.031 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28972.log: 26.5% -- replaced with /var/log/ceph/ceph-client.admin.28972.log.gz 2024-09-21T21:44:20.031 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30506.log 2024-09-21T21:44:20.031 INFO:teuthology.orchestra.run.smithi164.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.27529.log.gz 2024-09-21T21:44:20.032 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.29243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28098.log 2024-09-21T21:44:20.032 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29243.log.gz 2024-09-21T21:44:20.032 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.30506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27669.log 2024-09-21T21:44:20.032 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.30506.log.gz 2024-09-21T21:44:20.032 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27992.log 2024-09-21T21:44:20.032 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29033.log 2024-09-21T21:44:20.032 INFO:teuthology.orchestra.run.smithi164.stderr: 95.4% -- replaced with /var/log/ceph/ceph-client.admin.28098.log.gz 2024-09-21T21:44:20.032 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.27669.log.gz 2024-09-21T21:44:20.032 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27992.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29736.log 2024-09-21T21:44:20.032 INFO:teuthology.orchestra.run.smithi164.stderr: 84.0% -- replaced with /var/log/ceph/ceph-client.admin.27992.log.gz 2024-09-21T21:44:20.032 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.29033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30411.log 2024-09-21T21:44:20.032 INFO:teuthology.orchestra.run.smithi164.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.29033.log.gz 2024-09-21T21:44:20.032 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.29736.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28719.log 2024-09-21T21:44:20.032 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29736.log.gz 2024-09-21T21:44:20.032 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.30411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27388.log 2024-09-21T21:44:20.033 INFO:teuthology.orchestra.run.smithi164.stderr: 84.9% -- replaced with /var/log/ceph/ceph-client.admin.30411.log.gz 2024-09-21T21:44:20.033 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.28719.log: /var/log/ceph/ceph-client.admin.27459.log 2024-09-21T21:44:20.033 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28719.log.gz 2024-09-21T21:44:20.033 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29745.log 2024-09-21T21:44:20.033 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.27388.log.gz 2024-09-21T21:44:20.033 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29139.log 2024-09-21T21:44:20.033 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.27459.log.gz 2024-09-21T21:44:20.033 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.29745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29885.log 2024-09-21T21:44:20.033 INFO:teuthology.orchestra.run.smithi164.stderr: 54.4% -- replaced with /var/log/ceph/ceph-client.admin.29745.log.gz 2024-09-21T21:44:20.033 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.29139.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29851.log 2024-09-21T21:44:20.033 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29139.log.gz 2024-09-21T21:44:20.033 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.29885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29148.log 2024-09-21T21:44:20.033 INFO:teuthology.orchestra.run.smithi164.stderr: 25.3% -- replaced with /var/log/ceph/ceph-client.admin.29885.log.gz 2024-09-21T21:44:20.033 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.29851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27468.log 2024-09-21T21:44:20.034 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.29148.log: gzip -5 --verbose 94.9% -- /var/log/ceph/ceph-client.admin.29042.log -- replaced with /var/log/ceph/ceph-client.admin.29851.log.gz 2024-09-21T21:44:20.034 INFO:teuthology.orchestra.run.smithi164.stderr: 2024-09-21T21:44:20.034 INFO:teuthology.orchestra.run.smithi164.stderr: 53.2% -- replaced with /var/log/ceph/ceph-client.admin.29148.log.gz 2024-09-21T21:44:20.034 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27468.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28929.log 2024-09-21T21:44:20.034 INFO:teuthology.orchestra.run.smithi164.stderr: 93.8% -- replaced with /var/log/ceph/ceph-client.admin.27468.log.gz 2024-09-21T21:44:20.034 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.29042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27633.log 2024-09-21T21:44:20.034 INFO:teuthology.orchestra.run.smithi164.stderr: 72.3% -- replaced with /var/log/ceph/ceph-client.admin.29042.log.gz 2024-09-21T21:44:20.034 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28728.log 2024-09-21T21:44:20.034 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28929.log.gz 2024-09-21T21:44:20.034 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28448.log 2024-09-21T21:44:20.034 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.27633.log.gz 2024-09-21T21:44:20.034 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29534.log 2024-09-21T21:44:20.034 INFO:teuthology.orchestra.run.smithi164.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.28728.log.gz 2024-09-21T21:44:20.034 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28448.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29455.log 2024-09-21T21:44:20.034 INFO:teuthology.orchestra.run.smithi164.stderr: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.28448.log.gz 2024-09-21T21:44:20.035 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.29534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29781.log 2024-09-21T21:44:20.035 INFO:teuthology.orchestra.run.smithi164.stderr: 82.2% -- replaced with /var/log/ceph/ceph-client.admin.29534.log.gz 2024-09-21T21:44:20.035 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.29455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28999.log 2024-09-21T21:44:20.035 INFO:teuthology.orchestra.run.smithi164.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.29455.log.gz 2024-09-21T21:44:20.035 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.29781.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28859.log 2024-09-21T21:44:20.035 INFO:teuthology.orchestra.run.smithi164.stderr: 94.4% -- replaced with /var/log/ceph/ceph-client.admin.29781.log.gz 2024-09-21T21:44:20.035 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28999.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28168.log 2024-09-21T21:44:20.035 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28999.log.gz 2024-09-21T21:44:20.035 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29641.log 2024-09-21T21:44:20.035 INFO:teuthology.orchestra.run.smithi164.stderr: 18.2% -- replaced with /var/log/ceph/ceph-client.admin.28859.log.gz 2024-09-21T21:44:20.035 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27739.log 2024-09-21T21:44:20.035 INFO:teuthology.orchestra.run.smithi164.stderr: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.28168.log.gz 2024-09-21T21:44:20.035 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.29641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27703.log 2024-09-21T21:44:20.035 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27739.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27397.log 2024-09-21T21:44:20.035 INFO:teuthology.orchestra.run.smithi164.stderr: 95.2% -- replaced with /var/log/ceph/ceph-client.admin.29641.log.gz 2024-09-21T21:44:20.036 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.27739.log.gz 2024-09-21T21:44:20.036 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27269.log 2024-09-21T21:44:20.036 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.27703.log.gz 2024-09-21T21:44:20.036 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28159.log 2024-09-21T21:44:20.036 INFO:teuthology.orchestra.run.smithi164.stderr: 52.9% -- replaced with /var/log/ceph/ceph-client.admin.27397.log.gz 2024-09-21T21:44:20.036 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28132.log 2024-09-21T21:44:20.036 INFO:teuthology.orchestra.run.smithi164.stderr: 56.8% -- replaced with /var/log/ceph/ceph-client.admin.27269.log.gz 2024-09-21T21:44:20.036 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28159.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29815.log 2024-09-21T21:44:20.036 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28159.log.gz 2024-09-21T21:44:20.036 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30366.log 2024-09-21T21:44:20.036 INFO:teuthology.orchestra.run.smithi164.stderr: 53.3% -- replaced with /var/log/ceph/ceph-client.admin.28132.log.gz 2024-09-21T21:44:20.036 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.29815.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28649.log 2024-09-21T21:44:20.036 INFO:teuthology.orchestra.run.smithi164.stderr: 25.9% -- replaced with /var/log/ceph/ceph-client.admin.29815.log.gz 2024-09-21T21:44:20.036 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.30366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28263.log 2024-09-21T21:44:20.036 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.30366.log.gz 2024-09-21T21:44:20.037 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28518.log 2024-09-21T21:44:20.037 INFO:teuthology.orchestra.run.smithi164.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.28649.log.gz 2024-09-21T21:44:20.037 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28412.log 2024-09-21T21:44:20.037 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28263.log.gz 2024-09-21T21:44:20.037 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29562.log 2024-09-21T21:44:20.037 INFO:teuthology.orchestra.run.smithi164.stderr: 83.8% -- replaced with /var/log/ceph/ceph-client.admin.28518.log.gz 2024-09-21T21:44:20.037 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29571.log 2024-09-21T21:44:20.037 INFO:teuthology.orchestra.run.smithi164.stderr: 83.9% -- replaced with /var/log/ceph/ceph-client.admin.28412.log.gz 2024-09-21T21:44:20.037 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.29562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29946.log 2024-09-21T21:44:20.037 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29562.log.gz 2024-09-21T21:44:20.037 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.29571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28193.log 2024-09-21T21:44:20.037 INFO:teuthology.orchestra.run.smithi164.stderr: 84.3% -- replaced with /var/log/ceph/ceph-client.admin.29571.log.gz 2024-09-21T21:44:20.037 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.29946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28062.log 2024-09-21T21:44:20.037 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29946.log.gz 2024-09-21T21:44:20.037 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28193.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28753.log 2024-09-21T21:44:20.038 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28193.log.gz 2024-09-21T21:44:20.038 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30131.log 2024-09-21T21:44:20.038 INFO:teuthology.orchestra.run.smithi164.stderr: 25.3% -- replaced with /var/log/ceph/ceph-client.admin.28062.log.gz 2024-09-21T21:44:20.038 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28692.log 2024-09-21T21:44:20.038 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28753.log.gz 2024-09-21T21:44:20.038 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.30131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27642.log 2024-09-21T21:44:20.038 INFO:teuthology.orchestra.run.smithi164.stderr: 92.9% -- replaced with /var/log/ceph/ceph-client.admin.30131.log.gz 2024-09-21T21:44:20.038 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28832.log 2024-09-21T21:44:20.038 INFO:teuthology.orchestra.run.smithi164.stderr: 78.7% -- replaced with /var/log/ceph/ceph-client.admin.28692.log.gz 2024-09-21T21:44:20.038 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28789.log 2024-09-21T21:44:20.038 INFO:teuthology.orchestra.run.smithi164.stderr: 75.1% -- replaced with /var/log/ceph/ceph-client.admin.27642.log.gz 2024-09-21T21:44:20.038 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29314.log 2024-09-21T21:44:20.038 INFO:teuthology.orchestra.run.smithi164.stderr: 51.8% -- replaced with /var/log/ceph/ceph-client.admin.28832.log.gz 2024-09-21T21:44:20.038 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28789.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30296.log 2024-09-21T21:44:20.039 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28789.log.gz 2024-09-21T21:44:20.039 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.29314.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28342.log 2024-09-21T21:44:20.039 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29314.log.gz 2024-09-21T21:44:20.039 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.30296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28202.log 2024-09-21T21:44:20.039 INFO:teuthology.orchestra.run.smithi164.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.30296.log.gz 2024-09-21T21:44:20.039 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27260.log 2024-09-21T21:44:20.039 INFO:teuthology.orchestra.run.smithi164.stderr: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.28342.log.gz 2024-09-21T21:44:20.039 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29675.log 2024-09-21T21:44:20.039 INFO:teuthology.orchestra.run.smithi164.stderr: 53.6% -- replaced with /var/log/ceph/ceph-client.admin.28202.log.gz 2024-09-21T21:44:20.039 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27260.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28378.log 2024-09-21T21:44:20.039 INFO:teuthology.orchestra.run.smithi164.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.27260.log.gz 2024-09-21T21:44:20.039 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.29675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28333.log 2024-09-21T21:44:20.039 INFO:teuthology.orchestra.run.smithi164.stderr: 57.4% -- replaced with /var/log/ceph/ceph-client.admin.29675.log.gz 2024-09-21T21:44:20.039 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28378.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30515.log 2024-09-21T21:44:20.039 INFO:teuthology.orchestra.run.smithi164.stderr: 94.9% -- replaced with /var/log/ceph/ceph-client.admin.28378.log.gz 2024-09-21T21:44:20.039 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27809.log 2024-09-21T21:44:20.040 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28333.log.gz 2024-09-21T21:44:20.040 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.30515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29772.log 2024-09-21T21:44:20.040 INFO:teuthology.orchestra.run.smithi164.stderr: 25.3% -- replaced with /var/log/ceph/ceph-client.admin.30515.log.gz 2024-09-21T21:44:20.040 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27748.log 2024-09-21T21:44:20.040 INFO:teuthology.orchestra.run.smithi164.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.27809.log.gz 2024-09-21T21:44:20.040 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.29772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30262.log 2024-09-21T21:44:20.040 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29772.log.gz 2024-09-21T21:44:20.040 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27748.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29103.log 2024-09-21T21:44:20.040 INFO:teuthology.orchestra.run.smithi164.stderr: 93.2% -- replaced with /var/log/ceph/ceph-client.admin.27748.log.gz 2024-09-21T21:44:20.040 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.30262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30542.log 2024-09-21T21:44:20.040 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.30262.log.gz 2024-09-21T21:44:20.040 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.29103.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27572.log 2024-09-21T21:44:20.040 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29103.log.gz 2024-09-21T21:44:20.040 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.30542.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27843.log 2024-09-21T21:44:20.041 INFO:teuthology.orchestra.run.smithi164.stderr: 18.2% -- replaced with /var/log/ceph/ceph-client.admin.30542.log.gz 2024-09-21T21:44:20.041 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27572.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29112.log 2024-09-21T21:44:20.041 INFO:teuthology.orchestra.run.smithi164.stderr: 51.5% -- replaced with /var/log/ceph/ceph-client.admin.27572.log.gz 2024-09-21T21:44:20.041 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27888.log 2024-09-21T21:44:20.041 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.27843.log.gz 2024-09-21T21:44:20.041 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.29112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27983.log 2024-09-21T21:44:20.041 INFO:teuthology.orchestra.run.smithi164.stderr: 84.1% -- replaced with /var/log/ceph/ceph-client.admin.29112.log.gz 2024-09-21T21:44:20.041 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29323.log 2024-09-21T21:44:20.041 INFO:teuthology.orchestra.run.smithi164.stderr: 84.9% -- replaced with /var/log/ceph/ceph-client.admin.27888.log.gz 2024-09-21T21:44:20.041 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.27983.log: -- /var/log/ceph/ceph-client.admin.29182.log 2024-09-21T21:44:20.041 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.27983.log.gz 2024-09-21T21:44:20.041 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.29323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28622.log 2024-09-21T21:44:20.041 INFO:teuthology.orchestra.run.smithi164.stderr: 55.8% -- replaced with /var/log/ceph/ceph-client.admin.29323.log.gz 2024-09-21T21:44:20.041 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.29182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29430.log 2024-09-21T21:44:20.042 INFO:teuthology.orchestra.run.smithi164.stderr: 53.9% -- replaced with /var/log/ceph/ceph-client.admin.29182.log.gz 2024-09-21T21:44:20.042 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28622.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29280.log 2024-09-21T21:44:20.042 INFO:teuthology.orchestra.run.smithi164.stderr: 54.9% -- replaced with /var/log/ceph/ceph-client.admin.28622.log.gz 2024-09-21T21:44:20.042 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.29430.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29876.log 2024-09-21T21:44:20.042 INFO:teuthology.orchestra.run.smithi164.stderr: 56.2% -- replaced with /var/log/ceph/ceph-client.admin.29430.log.gz 2024-09-21T21:44:20.042 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.29280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29982.log 2024-09-21T21:44:20.042 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29280.log.gz 2024-09-21T21:44:20.042 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.29876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28299.log 2024-09-21T21:44:20.042 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29876.log.gz 2024-09-21T21:44:20.042 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.29982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27563.log 2024-09-21T21:44:20.042 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29982.log.gz 2024-09-21T21:44:20.042 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28902.log 2024-09-21T21:44:20.042 INFO:teuthology.orchestra.run.smithi164.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.28299.log.gz 2024-09-21T21:44:20.042 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27353.log 2024-09-21T21:44:20.042 INFO:teuthology.orchestra.run.smithi164.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.27563.log.gz 2024-09-21T21:44:20.043 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27362.log 2024-09-21T21:44:20.043 INFO:teuthology.orchestra.run.smithi164.stderr: 57.0% -- replaced with /var/log/ceph/ceph-client.admin.28902.log.gz 2024-09-21T21:44:20.043 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27353.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30445.log 2024-09-21T21:44:20.043 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.27353.log.gz 2024-09-21T21:44:20.043 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29921.log 2024-09-21T21:44:20.043 INFO:teuthology.orchestra.run.smithi164.stderr: 56.6% -- replaced with /var/log/ceph/ceph-client.admin.27362.log.gz 2024-09-21T21:44:20.043 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.30445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29912.log 2024-09-21T21:44:20.043 INFO:teuthology.orchestra.run.smithi164.stderr: 57.1% -- replaced with /var/log/ceph/ceph-client.admin.30445.log.gz 2024-09-21T21:44:20.043 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.29921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29525.log 2024-09-21T21:44:20.043 INFO:teuthology.orchestra.run.smithi164.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.29921.log.gz 2024-09-21T21:44:20.043 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.29912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29173.log 2024-09-21T21:44:20.043 INFO:teuthology.orchestra.run.smithi164.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.29912.log.gz 2024-09-21T21:44:20.043 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.29525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28823.log 2024-09-21T21:44:20.043 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29525.log.gz 2024-09-21T21:44:20.044 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.29173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27949.log 2024-09-21T21:44:20.044 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29173.log.gz 2024-09-21T21:44:20.044 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28823.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29289.log 2024-09-21T21:44:20.044 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28823.log.gz 2024-09-21T21:44:20.044 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27949.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27913.log 2024-09-21T21:44:20.044 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.27949.log.gz 2024-09-21T21:44:20.044 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.29289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28089.log 2024-09-21T21:44:20.044 INFO:teuthology.orchestra.run.smithi164.stderr: 53.2% -- replaced with /var/log/ceph/ceph-client.admin.29289.log.gz 2024-09-21T21:44:20.044 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27773.log 2024-09-21T21:44:20.044 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.27913.log.gz 2024-09-21T21:44:20.044 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30375.log 2024-09-21T21:44:20.044 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28089.log.gz 2024-09-21T21:44:20.044 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27773.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28369.log 2024-09-21T21:44:20.044 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.27773.log.gz 2024-09-21T21:44:20.044 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.30375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29218.log 2024-09-21T21:44:20.045 INFO:teuthology.orchestra.run.smithi164.stderr: 24.1% -- replaced with /var/log/ceph/ceph-client.admin.30375.log.gz 2024-09-21T21:44:20.045 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28439.log 2024-09-21T21:44:20.045 INFO:teuthology.orchestra.run.smithi164.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.28369.log.gz 2024-09-21T21:44:20.045 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.29218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29666.log 2024-09-21T21:44:20.045 INFO:teuthology.orchestra.run.smithi164.stderr: 51.2% -- replaced with /var/log/ceph/ceph-client.admin.29218.log.gz 2024-09-21T21:44:20.045 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29711.log 2024-09-21T21:44:20.045 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28439.log.gz 2024-09-21T21:44:20.045 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.29666.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30341.log 2024-09-21T21:44:20.045 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29666.log.gz 2024-09-21T21:44:20.045 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.29711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30472.log 2024-09-21T21:44:20.045 INFO:teuthology.orchestra.run.smithi164.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.29711.log.gz 2024-09-21T21:44:20.045 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.30341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30086.log 2024-09-21T21:44:20.045 INFO:teuthology.orchestra.run.smithi164.stderr: 55.0% -- replaced with /var/log/ceph/ceph-client.admin.30341.log.gz 2024-09-21T21:44:20.046 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.30472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30436.log 2024-09-21T21:44:20.046 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.30472.log.gz 2024-09-21T21:44:20.046 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.30086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27879.log 2024-09-21T21:44:20.046 INFO:teuthology.orchestra.run.smithi164.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.30086.log.gz 2024-09-21T21:44:20.046 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.30436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28123.log 2024-09-21T21:44:20.046 INFO:teuthology.orchestra.run.smithi164.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.30436.log.gz 2024-09-21T21:44:20.046 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27879.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27502.log 2024-09-21T21:44:20.046 INFO:teuthology.orchestra.run.smithi164.stderr: 18.2% -- replaced with /var/log/ceph/ceph-client.admin.27879.log.gz 2024-09-21T21:44:20.046 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30165.log 2024-09-21T21:44:20.046 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28123.log.gz 2024-09-21T21:44:20.046 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27502.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29842.log 2024-09-21T21:44:20.046 INFO:teuthology.orchestra.run.smithi164.stderr: 53.6% -- replaced with /var/log/ceph/ceph-client.admin.27502.log.gz 2024-09-21T21:44:20.046 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.30165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30095.log 2024-09-21T21:44:20.046 INFO:teuthology.orchestra.run.smithi164.stderr: 82.0% -- replaced with /var/log/ceph/ceph-client.admin.30165.log.gz 2024-09-21T21:44:20.046 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.29842.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28683.log 2024-09-21T21:44:20.047 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29842.log.gz 2024-09-21T21:44:20.047 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.30095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29491.log 2024-09-21T21:44:20.047 INFO:teuthology.orchestra.run.smithi164.stderr: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.30095.log.gz 2024-09-21T21:44:20.047 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28019.log 2024-09-21T21:44:20.047 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28683.log.gz 2024-09-21T21:44:20.047 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.29491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27782.log 2024-09-21T21:44:20.047 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29491.log.gz 2024-09-21T21:44:20.047 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30481.log 2024-09-21T21:44:20.047 INFO:teuthology.orchestra.run.smithi164.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.28019.log.gz 2024-09-21T21:44:20.047 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27782.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28868.log 2024-09-21T21:44:20.047 INFO:teuthology.orchestra.run.smithi164.stderr: 76.1% -- replaced with /var/log/ceph/ceph-client.admin.27782.log.gz 2024-09-21T21:44:20.047 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.30481.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28272.log 2024-09-21T21:44:20.047 INFO:teuthology.orchestra.run.smithi164.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.30481.log.gz 2024-09-21T21:44:20.047 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29421.log 2024-09-21T21:44:20.047 INFO:teuthology.orchestra.run.smithi164.stderr: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.28868.log.gz 2024-09-21T21:44:20.048 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28272.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27712.log 2024-09-21T21:44:20.048 INFO:teuthology.orchestra.run.smithi164.stderr: 81.0% -- replaced with /var/log/ceph/ceph-client.admin.28272.log.gz 2024-09-21T21:44:20.048 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.29421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29008.log 2024-09-21T21:44:20.048 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29421.log.gz 2024-09-21T21:44:20.048 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27712.log: 56.1% -- replaced with /var/log/ceph/ceph-client.admin.27712.log.gz 2024-09-21T21:44:20.048 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30226.log 2024-09-21T21:44:20.048 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.29008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28963.log 2024-09-21T21:44:20.048 INFO:teuthology.orchestra.run.smithi164.stderr: 94.5% -- replaced with /var/log/ceph/ceph-client.admin.29008.log.gz 2024-09-21T21:44:20.048 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.30226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28588.log 2024-09-21T21:44:20.048 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.30226.log.gz 2024-09-21T21:44:20.048 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30235.log 2024-09-21T21:44:20.048 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28963.log: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.28963.log.gz 2024-09-21T21:44:20.048 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29384.log 2024-09-21T21:44:20.048 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.30235.log: 95.7% -- replaced with /var/log/ceph/ceph-client.admin.28588.log.gz 2024-09-21T21:44:20.048 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28308.log 2024-09-21T21:44:20.048 INFO:teuthology.orchestra.run.smithi164.stderr: 54.6% -- replaced with /var/log/ceph/ceph-client.admin.30235.log.gz 2024-09-21T21:44:20.049 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.29384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30025.log 2024-09-21T21:44:20.049 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29384.log.gz 2024-09-21T21:44:20.049 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30156.log 2024-09-21T21:44:20.049 INFO:teuthology.orchestra.run.smithi164.stderr: 93.9% -- replaced with /var/log/ceph/ceph-client.admin.28308.log.gz 2024-09-21T21:44:20.049 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.30025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30551.log 2024-09-21T21:44:20.049 INFO:teuthology.orchestra.run.smithi164.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.30025.log.gz 2024-09-21T21:44:20.049 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.30156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28238.log 2024-09-21T21:44:20.049 INFO:teuthology.orchestra.run.smithi164.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.30156.log.gz 2024-09-21T21:44:20.049 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.30551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29350.log 2024-09-21T21:44:20.049 INFO:teuthology.orchestra.run.smithi164.stderr: 53.2% -- replaced with /var/log/ceph/ceph-client.admin.30551.log.gz 2024-09-21T21:44:20.049 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28238.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28509.log 2024-09-21T21:44:20.049 INFO:teuthology.orchestra.run.smithi164.stderr: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.28238.log.gz 2024-09-21T21:44:20.049 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.29350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30271.log 2024-09-21T21:44:20.049 INFO:teuthology.orchestra.run.smithi164.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.29350.log.gz 2024-09-21T21:44:20.049 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30122.log 2024-09-21T21:44:20.050 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28509.log.gz 2024-09-21T21:44:20.050 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.30271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28473.log 2024-09-21T21:44:20.050 INFO:teuthology.orchestra.run.smithi164.stderr: 77.9% -- replaced with /var/log/ceph/ceph-client.admin.30271.log.gz 2024-09-21T21:44:20.050 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.30122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29955.log 2024-09-21T21:44:20.050 INFO:teuthology.orchestra.run.smithi164.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.30122.log.gz 2024-09-21T21:44:20.050 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28473.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28613.log 2024-09-21T21:44:20.050 INFO:teuthology.orchestra.run.smithi164.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.28473.log.gz 2024-09-21T21:44:20.050 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.29955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28482.log 2024-09-21T21:44:20.050 INFO:teuthology.orchestra.run.smithi164.stderr: 52.7% -- replaced with /var/log/ceph/ceph-client.admin.29955.log.gz 2024-09-21T21:44:20.050 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28893.log 2024-09-21T21:44:20.050 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28613.log.gz 2024-09-21T21:44:20.050 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28482.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27422.log 2024-09-21T21:44:20.050 INFO:teuthology.orchestra.run.smithi164.stderr: 25.9% -- replaced with /var/log/ceph/ceph-client.admin.28482.log.gz 2024-09-21T21:44:20.050 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30192.log 2024-09-21T21:44:20.051 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28893.log.gz 2024-09-21T21:44:20.051 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29806.log 2024-09-21T21:44:20.051 INFO:teuthology.orchestra.run.smithi164.stderr: 18.2% -- replaced with /var/log/ceph/ceph-client.admin.27422.log.gz 2024-09-21T21:44:20.051 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30332.log 2024-09-21T21:44:20.051 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.30192.log: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.30192.log.gz 2024-09-21T21:44:20.051 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29393.log 2024-09-21T21:44:20.051 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.29806.log: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29806.log.gz 2024-09-21T21:44:20.051 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.30332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29209.log 2024-09-21T21:44:20.051 INFO:teuthology.orchestra.run.smithi164.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.30332.log.gz 2024-09-21T21:44:20.051 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.29393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29632.log 2024-09-21T21:44:20.051 INFO:teuthology.orchestra.run.smithi164.stderr: 52.4% -- replaced with /var/log/ceph/ceph-client.admin.29393.log.gz 2024-09-21T21:44:20.051 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30305.log 2024-09-21T21:44:20.051 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.29209.log: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.29209.log.gz 2024-09-21T21:44:20.051 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.29632.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29605.log 2024-09-21T21:44:20.051 INFO:teuthology.orchestra.run.smithi164.stderr: 18.2% -- replaced with /var/log/ceph/ceph-client.admin.29632.log.gz 2024-09-21T21:44:20.052 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.30305.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28543.log 2024-09-21T21:44:20.052 INFO:teuthology.orchestra.run.smithi164.stderr: 25.9% -- replaced with /var/log/ceph/ceph-client.admin.30305.log.gz 2024-09-21T21:44:20.052 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28403.log 2024-09-21T21:44:20.052 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.29605.log: 57.5% -- replaced with /var/log/ceph/ceph-client.admin.29605.log.gz 2024-09-21T21:44:20.052 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28798.log 2024-09-21T21:44:20.052 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28543.log.gz 2024-09-21T21:44:20.052 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27852.log 2024-09-21T21:44:20.052 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28403.log.gz 2024-09-21T21:44:20.052 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28762.log 2024-09-21T21:44:20.052 INFO:teuthology.orchestra.run.smithi164.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.28798.log.gz 2024-09-21T21:44:20.052 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27852.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29359.log 2024-09-21T21:44:20.052 INFO:teuthology.orchestra.run.smithi164.stderr: 76.3% -- replaced with /var/log/ceph/ceph-client.admin.27852.log.gz 2024-09-21T21:44:20.052 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29702.log 2024-09-21T21:44:20.052 INFO:teuthology.orchestra.run.smithi164.stderr: 54.2% -- replaced with /var/log/ceph/ceph-client.admin.28762.log.gz 2024-09-21T21:44:20.052 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30201.log 2024-09-21T21:44:20.053 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.29359.log: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.29359.log.gz 2024-09-21T21:44:20.053 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.29702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30016.log 2024-09-21T21:44:20.053 INFO:teuthology.orchestra.run.smithi164.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.29702.log.gz 2024-09-21T21:44:20.053 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.30201.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29078.log 2024-09-21T21:44:20.053 INFO:teuthology.orchestra.run.smithi164.stderr: 84.0% -- replaced with /var/log/ceph/ceph-client.admin.30201.log.gz 2024-09-21T21:44:20.053 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27599.log 2024-09-21T21:44:20.053 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.30016.log: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.30016.log.gz 2024-09-21T21:44:20.053 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.29078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27678.log 2024-09-21T21:44:20.053 INFO:teuthology.orchestra.run.smithi164.stderr: 94.9% -- replaced with /var/log/ceph/ceph-client.admin.29078.log.gz 2024-09-21T21:44:20.053 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27958.log 2024-09-21T21:44:20.053 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.27599.log.gz 2024-09-21T21:44:20.053 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29069.log 2024-09-21T21:44:20.053 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27958.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27818.log 2024-09-21T21:44:20.053 INFO:teuthology.orchestra.run.smithi164.stderr: 96.1% -- replaced with /var/log/ceph/ceph-client.admin.27678.log.gz 2024-09-21T21:44:20.053 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.29069.log: 95.8% -- replaced with /var/log/ceph/ceph-client.admin.27958.log.gz 2024-09-21T21:44:20.054 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30052.log 2024-09-21T21:44:20.054 INFO:teuthology.orchestra.run.smithi164.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.29069.log.gz 2024-09-21T21:44:20.054 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28053.log 2024-09-21T21:44:20.054 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.30052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.27226.log 2024-09-21T21:44:20.054 INFO:teuthology.orchestra.run.smithi164.stderr: 95.1% -- replaced with /var/log/ceph/ceph-client.admin.27818.log.gz 2024-09-21T21:44:20.054 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.30052.log.gz 2024-09-21T21:44:20.054 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30061.log 2024-09-21T21:44:20.054 INFO:teuthology.orchestra.run.smithi164.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28053.log.gz 2024-09-21T21:44:20.054 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.0.27226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28658.log 2024-09-21T21:44:20.054 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.30061.log: 93.4% -- replaced with /var/log/ceph/ceph-client.0.27226.log.gz 2024-09-21T21:44:20.054 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28028.log 2024-09-21T21:44:20.054 INFO:teuthology.orchestra.run.smithi164.stderr: 94.9%/var/log/ceph/ceph-client.admin.28658.log: -- replaced with /var/log/ceph/ceph-client.admin.30061.log.gz 2024-09-21T21:44:20.054 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29464.log 2024-09-21T21:44:20.054 INFO:teuthology.orchestra.run.smithi164.stderr: 94.3% -- replaced with /var/log/ceph/ceph-client.admin.28658.log.gz 2024-09-21T21:44:20.055 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27922.log 2024-09-21T21:44:20.055 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.29464.log: 95.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.28028.log.gz 2024-09-21T21:44:20.055 INFO:teuthology.orchestra.run.smithi164.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.27608.log 2024-09-21T21:44:20.055 INFO:teuthology.orchestra.run.smithi164.stderr: 81.9% -- replaced with /var/log/ceph/ceph-client.admin.29464.log.gz 2024-09-21T21:44:20.055 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28229.log 2024-09-21T21:44:20.055 INFO:teuthology.orchestra.run.smithi164.stderr: 52.2% -- replaced with /var/log/ceph/ceph-client.admin.27922.log.gz 2024-09-21T21:44:20.055 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27431.log 2024-09-21T21:44:20.055 INFO:teuthology.orchestra.run.smithi164.stderr: 94.7% -- replaced with /var/log/ceph/ceph-client.admin.27608.log.gz 2024-09-21T21:44:20.055 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28579.log 2024-09-21T21:44:20.055 INFO:teuthology.orchestra.run.smithi164.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.28229.log.gz 2024-09-21T21:44:20.055 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27431.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29500.log 2024-09-21T21:44:20.055 INFO:teuthology.orchestra.run.smithi164.stderr: 57.6% -- replaced with /var/log/ceph/ceph-client.admin.27431.log.gz 2024-09-21T21:44:20.055 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29252.log 2024-09-21T21:44:20.055 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.28579.log: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28579.log.gz 2024-09-21T21:44:20.055 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.29500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27538.log 2024-09-21T21:44:20.055 INFO:teuthology.orchestra.run.smithi164.stderr: 93.8% -- replaced with /var/log/ceph/ceph-client.admin.29500.log.gz 2024-09-21T21:44:20.056 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.29252.log: 54.3% -- replaced with /var/log/ceph/ceph-client.admin.29252.log.gz 2024-09-21T21:44:20.056 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.27538.log: 92.8% -- replaced with /var/log/ceph/ceph-client.admin.27538.log.gz 2024-09-21T21:44:20.056 INFO:teuthology.orchestra.run.smithi164.stderr: 2024-09-21T21:44:20.056 INFO:teuthology.orchestra.run.smithi164.stderr:real 0m0.065s 2024-09-21T21:44:20.056 INFO:teuthology.orchestra.run.smithi164.stderr:user 0m0.075s 2024-09-21T21:44:20.056 INFO:teuthology.orchestra.run.smithi164.stderr:sys 0m0.085s 2024-09-21T21:44:20.057 INFO:teuthology.orchestra.run.smithi122.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-21T21:44:20.058 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.55380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55933.log 2024-09-21T21:44:20.058 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55380.log.gz 2024-09-21T21:44:20.058 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.49472.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49472.log.gz 2024-09-21T21:44:20.058 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53163.log 2024-09-21T21:44:20.058 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36858.log 2024-09-21T21:44:20.059 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.55933.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55933.log.gz 2024-09-21T21:44:20.059 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.53163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46217.log 2024-09-21T21:44:20.059 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53163.log.gz 2024-09-21T21:44:20.059 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.36858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55698.log 2024-09-21T21:44:20.059 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36858.log.gz 2024-09-21T21:44:20.059 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.46217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56523.log 2024-09-21T21:44:20.059 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46217.log.gz 2024-09-21T21:44:20.059 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.55698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57993.log 2024-09-21T21:44:20.059 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55698.log.gz 2024-09-21T21:44:20.059 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.56523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56817.log 2024-09-21T21:44:20.059 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56523.log.gz 2024-09-21T21:44:20.059 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.57993.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57951.log 2024-09-21T21:44:20.059 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57993.log.gz 2024-09-21T21:44:20.059 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.56817.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56817.log.gz 2024-09-21T21:44:20.059 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36110.log 2024-09-21T21:44:20.060 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.57951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39822.log 2024-09-21T21:44:20.060 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57951.log.gz 2024-09-21T21:44:20.060 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.36110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57123.log 2024-09-21T21:44:20.060 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36110.log.gz 2024-09-21T21:44:20.060 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.39822.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38902.log 2024-09-21T21:44:20.060 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39822.log.gz 2024-09-21T21:44:20.060 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.57123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37684.log 2024-09-21T21:44:20.060 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57123.log.gz 2024-09-21T21:44:20.060 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.38902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35548.log 2024-09-21T21:44:20.060 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38902.log.gz 2024-09-21T21:44:20.060 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.37684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57126.log 2024-09-21T21:44:20.060 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37684.log.gz 2024-09-21T21:44:20.060 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.35548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57127.log 2024-09-21T21:44:20.060 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35548.log.gz 2024-09-21T21:44:20.060 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.57126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54478.log 2024-09-21T21:44:20.061 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57126.log.gz 2024-09-21T21:44:20.061 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.57127.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35304.log 2024-09-21T21:44:20.061 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57127.log.gz 2024-09-21T21:44:20.061 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.54478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57741.log 2024-09-21T21:44:20.061 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54478.log.gz 2024-09-21T21:44:20.061 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.35304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39728.log 2024-09-21T21:44:20.061 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35304.log.gz 2024-09-21T21:44:20.061 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.57741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51372.log 2024-09-21T21:44:20.061 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57741.log.gz 2024-09-21T21:44:20.061 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.39728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54438.log 2024-09-21T21:44:20.061 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39728.log.gz 2024-09-21T21:44:20.061 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.51372.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54030.log 2024-09-21T21:44:20.061 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51372.log.gz 2024-09-21T21:44:20.061 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.54438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35448.log 2024-09-21T21:44:20.062 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54438.log.gz 2024-09-21T21:44:20.062 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.54030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51846.log 2024-09-21T21:44:20.062 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54030.log.gz 2024-09-21T21:44:20.062 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.35448.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52753.log 2024-09-21T21:44:20.062 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35448.log.gz 2024-09-21T21:44:20.062 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.51846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48983.log 2024-09-21T21:44:20.062 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51846.log.gz 2024-09-21T21:44:20.062 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.52753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38626.log 2024-09-21T21:44:20.062 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52753.log.gz 2024-09-21T21:44:20.062 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.48983.log: gzip -5 --verbose 0.0% -- /var/log/ceph/ceph-client.admin.50465.log -- replaced with /var/log/ceph/ceph-client.admin.48983.log.gz 2024-09-21T21:44:20.062 INFO:teuthology.orchestra.run.smithi081.stderr: 2024-09-21T21:44:20.062 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.38626.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40819.log 2024-09-21T21:44:20.062 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38626.log.gz 2024-09-21T21:44:20.062 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.50465.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-21T21:44:20.062 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50465.log.gz 2024-09-21T21:44:20.063 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.40819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38214.log 2024-09-21T21:44:20.063 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40819.log.gz 2024-09-21T21:44:20.063 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49820.log 2024-09-21T21:44:20.063 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.38214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45679.log 2024-09-21T21:44:20.063 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38214.log.gz 2024-09-21T21:44:20.063 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.49820.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57825.log 2024-09-21T21:44:20.063 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49820.log.gz 2024-09-21T21:44:20.063 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.45679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35038.log 2024-09-21T21:44:20.063 INFO:teuthology.orchestra.run.smithi081.stderr: 28.1% -- replaced with /var/log/ceph/ceph-client.admin.45679.log.gz 2024-09-21T21:44:20.063 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.57825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39177.log 2024-09-21T21:44:20.063 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57825.log.gz 2024-09-21T21:44:20.063 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.35038.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35038.log.gz 2024-09-21T21:44:20.063 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41819.log 2024-09-21T21:44:20.063 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.39177.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43617.log 2024-09-21T21:44:20.063 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39177.log.gz 2024-09-21T21:44:20.063 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.41819.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.41819.log.gz -- 2024-09-21T21:44:20.064 INFO:teuthology.orchestra.run.smithi081.stderr: /var/log/ceph/ceph-client.admin.49218.log 2024-09-21T21:44:20.064 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.43617.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43617.log.gz 2024-09-21T21:44:20.064 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56439.log 2024-09-21T21:44:20.064 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.49218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49686.log 2024-09-21T21:44:20.064 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49218.log.gz 2024-09-21T21:44:20.064 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.56439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57307.log 2024-09-21T21:44:20.064 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56439.log.gz 2024-09-21T21:44:20.064 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.49686.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49686.log.gz 2024-09-21T21:44:20.064 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50413.log 2024-09-21T21:44:20.064 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.57307.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57783.log 2024-09-21T21:44:20.064 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57307.log.gz 2024-09-21T21:44:20.064 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.50413.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50413.log.gz 2024-09-21T21:44:20.064 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58077.log 2024-09-21T21:44:20.064 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.57783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35964.log 2024-09-21T21:44:20.064 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57783.log.gz 2024-09-21T21:44:20.065 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.58077.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58077.log.gz 2024-09-21T21:44:20.065 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39272.log 2024-09-21T21:44:20.065 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.35964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41268.log 2024-09-21T21:44:20.065 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35964.log.gz 2024-09-21T21:44:20.065 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.39272.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39272.log.gz 2024-09-21T21:44:20.065 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49780.log 2024-09-21T21:44:20.065 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.41268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56392.log 2024-09-21T21:44:20.065 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41268.log.gz 2024-09-21T21:44:20.065 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.49780.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49780.log.gz 2024-09-21T21:44:20.065 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35297.log 2024-09-21T21:44:20.065 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.56392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37173.log 2024-09-21T21:44:20.065 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56392.log.gz 2024-09-21T21:44:20.065 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.35297.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35297.log.gz 2024-09-21T21:44:20.065 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45404.log 2024-09-21T21:44:20.066 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.37173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34686.log 2024-09-21T21:44:20.066 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37173.log.gz 2024-09-21T21:44:20.066 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.45404.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45404.log.gz 2024-09-21T21:44:20.066 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31970.log 2024-09-21T21:44:20.066 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.34686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39218.log 2024-09-21T21:44:20.066 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34686.log.gz 2024-09-21T21:44:20.066 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.31970.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31970.log.gz 2024-09-21T21:44:20.066 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51671.log 2024-09-21T21:44:20.066 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.39218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57124.log 2024-09-21T21:44:20.066 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39218.log.gz 2024-09-21T21:44:20.066 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.51671.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51671.log.gz 2024-09-21T21:44:20.066 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54598.log 2024-09-21T21:44:20.066 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.57124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48117.log 2024-09-21T21:44:20.066 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57124.log.gz 2024-09-21T21:44:20.066 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.54598.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54598.log.gz 2024-09-21T21:44:20.067 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47217.log 2024-09-21T21:44:20.067 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.48117.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34902.log 2024-09-21T21:44:20.067 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48117.log.gz 2024-09-21T21:44:20.067 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.47217.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47217.log.gz 2024-09-21T21:44:20.067 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47613.log 2024-09-21T21:44:20.067 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.34902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52846.log 2024-09-21T21:44:20.067 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34902.log.gz 2024-09-21T21:44:20.067 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.47613.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.47613.log.gz 2024-09-21T21:44:20.067 INFO:teuthology.orchestra.run.smithi081.stderr: /var/log/ceph/ceph-client.admin.44914.log 2024-09-21T21:44:20.067 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56226.log 2024-09-21T21:44:20.067 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.52846.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52846.log.gz 2024-09-21T21:44:20.067 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.44914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35236.log 2024-09-21T21:44:20.067 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44914.log.gz 2024-09-21T21:44:20.067 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.56226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35040.log 0.0% 2024-09-21T21:44:20.068 INFO:teuthology.orchestra.run.smithi081.stderr: -- replaced with /var/log/ceph/ceph-client.admin.56226.log.gz 2024-09-21T21:44:20.068 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.35236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35041.log 2024-09-21T21:44:20.068 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35236.log.gz 2024-09-21T21:44:20.068 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.35040.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.35040.log.gz -5 2024-09-21T21:44:20.068 INFO:teuthology.orchestra.run.smithi081.stderr: --verbose -- /var/log/ceph/ceph-client.admin.38748.log 2024-09-21T21:44:20.068 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.35041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46512.log 2024-09-21T21:44:20.068 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35041.log.gz 2024-09-21T21:44:20.068 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53438.log 2024-09-21T21:44:20.068 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.38748.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38748.log.gz 2024-09-21T21:44:20.068 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.46512.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46512.log.gz 2024-09-21T21:44:20.068 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38541.log 2024-09-21T21:44:20.068 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.53438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48573.log 2024-09-21T21:44:20.068 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53438.log.gz 2024-09-21T21:44:20.068 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.38541.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38541.log.gz 2024-09-21T21:44:20.069 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35796.log 2024-09-21T21:44:20.069 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.48573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43246.log 2024-09-21T21:44:20.069 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48573.log.gz 2024-09-21T21:44:20.069 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.35796.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35796.log.gz 2024-09-21T21:44:20.069 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56775.log 2024-09-21T21:44:20.069 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.43246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57119.log 2024-09-21T21:44:20.069 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43246.log.gz 2024-09-21T21:44:20.069 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.56775.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56775.log.gz 2024-09-21T21:44:20.069 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35468.log 2024-09-21T21:44:20.069 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.57119.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36211.log 2024-09-21T21:44:20.069 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57119.log.gz 2024-09-21T21:44:20.069 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.35468.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35468.log.gz 2024-09-21T21:44:20.069 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47809.log 2024-09-21T21:44:20.069 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.36211.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-21T21:44:20.069 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36211.log.gz 2024-09-21T21:44:20.070 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.47809.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47809.log.gz 2024-09-21T21:44:20.070 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47358.log 2024-09-21T21:44:20.070 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52558.log 2024-09-21T21:44:20.070 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.47358.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47358.log.gz 2024-09-21T21:44:20.070 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41575.log 2024-09-21T21:44:20.070 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41227.log 2024-09-21T21:44:20.070 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.52558.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52558.log.gz 2024-09-21T21:44:20.070 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.41575.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57699.log 2024-09-21T21:44:20.070 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41575.log.gz 2024-09-21T21:44:20.070 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.41227.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41227.log.gz 2024-09-21T21:44:20.070 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57615.log 2024-09-21T21:44:20.070 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.57699.log: /var/log/ceph/ceph-client.admin.54358.log 2024-09-21T21:44:20.070 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57699.log.gz 2024-09-21T21:44:20.070 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.57615.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57615.log.gz 2024-09-21T21:44:20.071 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48004.log 2024-09-21T21:44:20.071 INFO:teuthology.orchestra.run.smithi081.stderr: 93.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-21T21:44:20.071 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53808.log 2024-09-21T21:44:20.071 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.54358.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54358.log.gz 2024-09-21T21:44:20.071 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.48004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52243.log 2024-09-21T21:44:20.071 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48004.log.gz 2024-09-21T21:44:20.071 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.53808.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.53808.log.gz --verbose 2024-09-21T21:44:20.071 INFO:teuthology.orchestra.run.smithi081.stderr: -- /var/log/ceph/ceph-client.admin.58371.log 2024-09-21T21:44:20.071 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.52243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55220.log 2024-09-21T21:44:20.071 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52243.log.gz 2024-09-21T21:44:20.071 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.58371.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.58371.log.gz 2024-09-21T21:44:20.071 INFO:teuthology.orchestra.run.smithi081.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.36523.log 2024-09-21T21:44:20.071 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.55220.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58119.log 2024-09-21T21:44:20.071 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55220.log.gz 2024-09-21T21:44:20.071 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.36523.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36523.log.gz -5 2024-09-21T21:44:20.072 INFO:teuthology.orchestra.run.smithi081.stderr: --verbose -- /var/log/ceph/ceph-client.admin.45128.log 2024-09-21T21:44:20.072 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.58119.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53008.log 2024-09-21T21:44:20.072 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58119.log.gz 2024-09-21T21:44:20.072 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.45128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35234.log 0.0% 2024-09-21T21:44:20.072 INFO:teuthology.orchestra.run.smithi081.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45128.log.gz 2024-09-21T21:44:20.072 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.53008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42442.log 2024-09-21T21:44:20.072 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53008.log.gz 2024-09-21T21:44:20.072 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.35234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45342.log 2024-09-21T21:44:20.072 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35234.log.gz 2024-09-21T21:44:20.072 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.42442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52161.log 2024-09-21T21:44:20.072 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42442.log.gz 2024-09-21T21:44:20.072 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.45342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35629.log 2024-09-21T21:44:20.072 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45342.log.gz 2024-09-21T21:44:20.072 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.52161.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52161.log.gz 2024-09-21T21:44:20.073 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51472.log 2024-09-21T21:44:20.073 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.35629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57531.log 2024-09-21T21:44:20.073 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35629.log.gz 2024-09-21T21:44:20.073 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.51472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55718.log 2024-09-21T21:44:20.073 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51472.log.gz 2024-09-21T21:44:20.073 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.57531.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57531.log.gz -5 2024-09-21T21:44:20.073 INFO:teuthology.orchestra.run.smithi081.stderr: --verbose -- /var/log/ceph/ceph-client.admin.32042.log 2024-09-21T21:44:20.073 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.55718.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42555.log 2024-09-21T21:44:20.073 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55718.log.gz 2024-09-21T21:44:20.073 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.32042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46808.log 0.0% 2024-09-21T21:44:20.073 INFO:teuthology.orchestra.run.smithi081.stderr: -- replaced with /var/log/ceph/ceph-client.admin.32042.log.gz 2024-09-21T21:44:20.073 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.42555.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42555.log.gz 2024-09-21T21:44:20.073 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51567.log 2024-09-21T21:44:20.073 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.46808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53754.log 2024-09-21T21:44:20.073 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46808.log.gz 2024-09-21T21:44:20.074 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.51567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43503.log 2024-09-21T21:44:20.074 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51567.log.gz 2024-09-21T21:44:20.074 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.53754.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53754.log.gz 2024-09-21T21:44:20.074 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44484.log 2024-09-21T21:44:20.074 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.43503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56052.log 2024-09-21T21:44:20.074 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43503.log.gz 2024-09-21T21:44:20.074 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.44484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53478.log 2024-09-21T21:44:20.074 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44484.log.gz 2024-09-21T21:44:20.074 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.56052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46091.log 2024-09-21T21:44:20.074 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56052.log.gz 2024-09-21T21:44:20.074 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.53478.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53478.log.gz 2024-09-21T21:44:20.074 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51731.log 2024-09-21T21:44:20.074 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.46091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35752.log 2024-09-21T21:44:20.074 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46091.log.gz 2024-09-21T21:44:20.075 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.51731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52518.log 2024-09-21T21:44:20.075 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51731.log.gz 2024-09-21T21:44:20.075 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.35752.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46049.log 2024-09-21T21:44:20.075 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35752.log.gz 2024-09-21T21:44:20.075 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.52518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56649.log 0.0% 2024-09-21T21:44:20.075 INFO:teuthology.orchestra.run.smithi081.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52518.log.gz 2024-09-21T21:44:20.075 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.46049.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44126.log 2024-09-21T21:44:20.075 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46049.log.gz 2024-09-21T21:44:20.075 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.56649.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56649.log.gz 2024-09-21T21:44:20.075 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43934.log 2024-09-21T21:44:20.075 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.44126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56985.log 2024-09-21T21:44:20.075 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44126.log.gz 2024-09-21T21:44:20.075 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.43934.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37348.log 2024-09-21T21:44:20.075 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43934.log.gz 2024-09-21T21:44:20.076 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.56985.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50178.log 2024-09-21T21:44:20.076 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56985.log.gz 2024-09-21T21:44:20.076 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.37348.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.37348.log.gz 2024-09-21T21:44:20.076 INFO:teuthology.orchestra.run.smithi081.stderr: /var/log/ceph/ceph-client.admin.56859.log 2024-09-21T21:44:20.076 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.50178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35306.log 2024-09-21T21:44:20.076 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50178.log.gz 2024-09-21T21:44:20.076 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.56859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57125.log 2024-09-21T21:44:20.076 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56859.log.gz 2024-09-21T21:44:20.076 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.35306.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-21T21:44:20.076 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35306.log.gz 2024-09-21T21:44:20.076 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.57125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54873.log 2024-09-21T21:44:20.076 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57125.log.gz 2024-09-21T21:44:20.076 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34598.log 2024-09-21T21:44:20.076 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.54873.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47962.log 2024-09-21T21:44:20.076 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54873.log.gz 2024-09-21T21:44:20.077 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.34598.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34598.log.gz 2024-09-21T21:44:20.077 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56481.log 2024-09-21T21:44:20.077 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53101.log 2024-09-21T21:44:20.077 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.47962.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47962.log.gz 2024-09-21T21:44:20.077 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.56481.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38448.log 2024-09-21T21:44:20.077 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56481.log.gz 2024-09-21T21:44:20.077 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.53101.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53101.log.gz 2024-09-21T21:44:20.077 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57867.log 2024-09-21T21:44:20.077 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37490.log 2024-09-21T21:44:20.077 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.38448.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38448.log.gz 2024-09-21T21:44:20.077 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.57867.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.57867.log.gz -- 2024-09-21T21:44:20.077 INFO:teuthology.orchestra.run.smithi081.stderr: /var/log/ceph/ceph-client.admin.35711.log 2024-09-21T21:44:20.077 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.37490.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37490.log.gz 2024-09-21T21:44:20.077 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54204.log 2024-09-21T21:44:20.078 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38964.log 2024-09-21T21:44:20.078 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.35711.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35711.log.gz 2024-09-21T21:44:20.078 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.54204.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54398.log 2024-09-21T21:44:20.078 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54204.log.gz 2024-09-21T21:44:20.078 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.38964.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38964.log.gz 2024-09-21T21:44:20.078 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45445.log 2024-09-21T21:44:20.078 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42892.log 2024-09-21T21:44:20.078 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.54398.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54398.log.gz 2024-09-21T21:44:20.078 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.45445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57400.log 2024-09-21T21:44:20.078 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45445.log.gz 2024-09-21T21:44:20.078 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47084.log 2024-09-21T21:44:20.078 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.42892.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42892.log.gz 2024-09-21T21:44:20.078 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.57400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44208.log 2024-09-21T21:44:20.078 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57400.log.gz 2024-09-21T21:44:20.078 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.47084.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47084.log.gz 2024-09-21T21:44:20.079 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39924.log 2024-09-21T21:44:20.079 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54538.log 2024-09-21T21:44:20.079 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.44208.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44208.log.gz 2024-09-21T21:44:20.079 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.39924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35044.log 2024-09-21T21:44:20.079 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39924.log.gz 2024-09-21T21:44:20.079 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.54538.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54538.log.gz 2024-09-21T21:44:20.079 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56350.log 2024-09-21T21:44:20.079 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35568.log 2024-09-21T21:44:20.079 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.35044.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35044.log.gz 2024-09-21T21:44:20.079 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.56350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51609.log 2024-09-21T21:44:20.079 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56350.log.gz 2024-09-21T21:44:20.079 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.35568.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35568.log.gz 2024-09-21T21:44:20.079 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-21T21:44:20.079 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.51609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37858.log 2024-09-21T21:44:20.080 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51609.log.gz 2024-09-21T21:44:20.080 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48259.log 2024-09-21T21:44:20.080 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.37858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42750.log 2024-09-21T21:44:20.080 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37858.log.gz 2024-09-21T21:44:20.080 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.48259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58287.log 2024-09-21T21:44:20.080 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48259.log.gz 2024-09-21T21:44:20.080 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54658.log 2024-09-21T21:44:20.080 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.42750.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42750.log.gz 2024-09-21T21:44:20.080 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.58287.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53225.log 2024-09-21T21:44:20.080 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58287.log.gz 2024-09-21T21:44:20.080 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.54658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35922.log 2024-09-21T21:44:20.080 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54658.log.gz 2024-09-21T21:44:20.080 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.53225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40753.log 2024-09-21T21:44:20.080 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53225.log.gz 2024-09-21T21:44:20.080 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.35922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43187.log 2024-09-21T21:44:20.081 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35922.log.gz 2024-09-21T21:44:20.081 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.40753.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40753.log.gz 2024-09-21T21:44:20.081 INFO:teuthology.orchestra.run.smithi081.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.56126.log 2024-09-21T21:44:20.081 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.43187.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43187.log.gz 2024-09-21T21:44:20.081 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54083.log 2024-09-21T21:44:20.081 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.56126.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56126.log.gz 2024-09-21T21:44:20.081 INFO:teuthology.orchestra.run.smithi081.stderr: --verbose -- /var/log/ceph/ceph-client.admin.55464.log 2024-09-21T21:44:20.081 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.54083.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54083.log.gz 2024-09-21T21:44:20.081 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46592.log 2024-09-21T21:44:20.081 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.55464.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55464.log.gz 2024-09-21T21:44:20.081 INFO:teuthology.orchestra.run.smithi081.stderr: --verbose -- /var/log/ceph/ceph-client.admin.55873.log 2024-09-21T21:44:20.081 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.46592.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46592.log.gz 2024-09-21T21:44:20.081 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57274.log 2024-09-21T21:44:20.081 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.55873.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-21T21:44:20.082 INFO:teuthology.orchestra.run.smithi081.stderr: -- replaced with /var/log/ceph/ceph-client.admin.55873.log.gz 2024-09-21T21:44:20.082 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.57274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31741.log 2024-09-21T21:44:20.082 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57274.log.gz 2024-09-21T21:44:20.082 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57657.log 2024-09-21T21:44:20.082 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.31741.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31741.log.gz 2024-09-21T21:44:20.082 INFO:teuthology.orchestra.run.smithi081.stderr: --verbose -- /var/log/ceph/ceph-client.admin.55148.log 2024-09-21T21:44:20.082 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.57657.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57657.log.gz 2024-09-21T21:44:20.082 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.27357.log 2024-09-21T21:44:20.082 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.55148.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55148.log.gz 2024-09-21T21:44:20.082 INFO:teuthology.orchestra.run.smithi081.stderr: --verbose -- /var/log/ceph/ceph-client.admin.54718.log 2024-09-21T21:44:20.082 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph.tmp-client.admin.27357.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.27357.log.gz 2024-09-21T21:44:20.082 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34767.log 2024-09-21T21:44:20.082 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.54718.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54718.log.gz 2024-09-21T21:44:20.082 INFO:teuthology.orchestra.run.smithi081.stderr: --verbose -- /var/log/ceph/ceph-client.admin.57573.log 2024-09-21T21:44:20.082 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.34767.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34767.log.gz 2024-09-21T21:44:20.083 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40223.log 2024-09-21T21:44:20.083 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.57573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34860.log 2024-09-21T21:44:20.083 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57573.log.gz 2024-09-21T21:44:20.083 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.40223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56733.log 2024-09-21T21:44:20.083 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40223.log.gz 2024-09-21T21:44:20.083 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.34860.log: -- /var/log/ceph/ceph-client.admin.40951.log 2024-09-21T21:44:20.083 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34860.log.gz 2024-09-21T21:44:20.083 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.56733.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56733.log.gz 2024-09-21T21:44:20.083 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51198.log 2024-09-21T21:44:20.083 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.40951.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40951.log.gz 2024-09-21T21:44:20.083 INFO:teuthology.orchestra.run.smithi081.stderr: --verbose -- /var/log/ceph/ceph-client.admin.58035.log 2024-09-21T21:44:20.083 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.51198.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51198.log.gz 2024-09-21T21:44:20.083 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47674.log 2024-09-21T21:44:20.083 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.58035.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58035.log.gz 2024-09-21T21:44:20.083 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45087.log 2024-09-21T21:44:20.084 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.47674.log: gzip -5 0.0% --verbose -- /var/log/ceph/ceph.log 2024-09-21T21:44:20.084 INFO:teuthology.orchestra.run.smithi081.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47674.log.gz 2024-09-21T21:44:20.084 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.45087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41872.log 2024-09-21T21:44:20.084 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45087.log.gz 2024-09-21T21:44:20.084 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5/var/log/ceph/ceph.log: --verbose -- /var/log/ceph/ceph-client.admin.47432.log 2024-09-21T21:44:20.084 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.41872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42247.log 2024-09-21T21:44:20.084 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41872.log.gz 2024-09-21T21:44:20.084 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.47432.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47432.log.gz 2024-09-21T21:44:20.084 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39984.log 2024-09-21T21:44:20.084 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50689.log 2024-09-21T21:44:20.084 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.42247.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42247.log.gz 2024-09-21T21:44:20.084 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.39984.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39984.log.gz 2024-09-21T21:44:20.084 INFO:teuthology.orchestra.run.smithi081.stderr:gzip 91.6% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-21T21:44:20.084 INFO:teuthology.orchestra.run.smithi081.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.42054.log 2024-09-21T21:44:20.085 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49984.log 2024-09-21T21:44:20.085 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.50689.log: /var/log/ceph/ceph-client.admin.42054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45997.log 2024-09-21T21:44:20.085 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50689.log.gz 2024-09-21T21:44:20.085 INFO:teuthology.orchestra.run.smithi081.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42054.log.gz 2024-09-21T21:44:20.085 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.49984.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49984.log.gz 2024-09-21T21:44:20.085 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46133.log 2024-09-21T21:44:20.085 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40868.log 2024-09-21T21:44:20.085 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.45997.log: /var/log/ceph/ceph-client.admin.46133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56901.log 2024-09-21T21:44:20.085 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45997.log.gz 2024-09-21T21:44:20.085 INFO:teuthology.orchestra.run.smithi081.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46133.log.gz 2024-09-21T21:44:20.085 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.40868.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40868.log.gz 2024-09-21T21:44:20.085 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57399.log 2024-09-21T21:44:20.085 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37020.log 2024-09-21T21:44:20.085 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.56901.log: /var/log/ceph/ceph-client.admin.57399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36212.log 2024-09-21T21:44:20.085 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56901.log.gz 2024-09-21T21:44:20.086 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57399.log.gz 2024-09-21T21:44:20.086 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.37020.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37020.log.gz 2024-09-21T21:44:20.086 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39494.log 2024-09-21T21:44:20.086 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35192.log 2024-09-21T21:44:20.086 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.36212.log: /var/log/ceph/ceph-client.admin.39494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50372.log 2024-09-21T21:44:20.086 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36212.log.gz 2024-09-21T21:44:20.086 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39494.log.gz 2024-09-21T21:44:20.086 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.35192.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35192.log.gz 2024-09-21T21:44:20.086 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42187.log 2024-09-21T21:44:20.086 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44249.log 2024-09-21T21:44:20.086 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.50372.log: /var/log/ceph/ceph-client.admin.42187.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40264.log 2024-09-21T21:44:20.086 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50372.log.gz 2024-09-21T21:44:20.086 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42187.log.gz 2024-09-21T21:44:20.086 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.44249.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44249.log.gz 2024-09-21T21:44:20.086 INFO:teuthology.orchestra.run.smithi081.stderr: --verbose -- /var/log/ceph/ceph-client.admin.49177.log 2024-09-21T21:44:20.087 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.40264.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40264.log.gz 2024-09-21T21:44:20.087 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43679.log 2024-09-21T21:44:20.087 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.49177.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49177.log.gz 2024-09-21T21:44:20.087 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43014.log 2024-09-21T21:44:20.087 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.43679.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43679.log.gz 2024-09-21T21:44:20.087 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56186.log 2024-09-21T21:44:20.087 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.43014.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43014.log.gz 2024-09-21T21:44:20.087 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46767.log 2024-09-21T21:44:20.087 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.56186.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56186.log.gz 2024-09-21T21:44:20.087 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48902.log 2024-09-21T21:44:20.087 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.46767.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46767.log.gz 2024-09-21T21:44:20.087 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36048.log 2024-09-21T21:44:20.087 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.48902.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48902.log.gz 2024-09-21T21:44:20.087 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40498.log 2024-09-21T21:44:20.087 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.36048.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36048.log.gz 2024-09-21T21:44:20.088 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43382.log 2024-09-21T21:44:20.088 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.40498.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40498.log.gz 2024-09-21T21:44:20.088 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52356.log 2024-09-21T21:44:20.088 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.43382.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43382.log.gz 2024-09-21T21:44:20.088 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45732.log 2024-09-21T21:44:20.088 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.52356.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52356.log.gz 2024-09-21T21:44:20.088 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46175.log 2024-09-21T21:44:20.088 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.45732.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45732.log.gz 2024-09-21T21:44:20.088 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34818.log 2024-09-21T21:44:20.088 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.46175.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46175.log.gz 2024-09-21T21:44:20.088 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36006.log 2024-09-21T21:44:20.088 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57909.log 2024-09-21T21:44:20.088 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.34818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58245.log 2024-09-21T21:44:20.088 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.36006.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34818.log.gz 2024-09-21T21:44:20.088 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36006.log.gz 2024-09-21T21:44:20.089 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.57909.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41555.log 2024-09-21T21:44:20.089 INFO:teuthology.orchestra.run.smithi081.stderr: -- replaced with /var/log/ceph/ceph-client.admin.57909.log.gz 2024-09-21T21:44:20.089 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.58245.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58245.log.gz 2024-09-21T21:44:20.089 INFO:teuthology.orchestra.run.smithi081.stderr: --verbose -- /var/log/ceph/ceph-client.admin.58411.log 2024-09-21T21:44:20.089 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.41555.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41555.log.gz 2024-09-21T21:44:20.089 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56565.log 2024-09-21T21:44:20.089 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.58411.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58411.log.gz 2024-09-21T21:44:20.089 INFO:teuthology.orchestra.run.smithi081.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.58161.log 2024-09-21T21:44:20.089 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.56565.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56565.log.gz 2024-09-21T21:44:20.089 INFO:teuthology.orchestra.run.smithi081.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.40551.log 2024-09-21T21:44:20.089 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.58161.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58161.log.gz 2024-09-21T21:44:20.089 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57339.log 2024-09-21T21:44:20.089 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.40551.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40551.log.gz 2024-09-21T21:44:20.089 INFO:teuthology.orchestra.run.smithi081.stderr: --verbose -- /var/log/ceph/ceph-client.admin.35880.log 2024-09-21T21:44:20.089 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.57339.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57339.log.gz 2024-09-21T21:44:20.089 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43872.log 2024-09-21T21:44:20.090 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.35880.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35880.log.gz 2024-09-21T21:44:20.090 INFO:teuthology.orchestra.run.smithi081.stderr: --verbose -- /var/log/ceph/ceph-client.admin.34742.log 2024-09-21T21:44:20.090 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.43872.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43872.log.gz 2024-09-21T21:44:20.090 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50964.log 2024-09-21T21:44:20.090 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.34742.log: gzip -5 --verbose 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34742.log.gz 2024-09-21T21:44:20.090 INFO:teuthology.orchestra.run.smithi081.stderr: -- /var/log/ceph/ceph-client.admin.49452.log 2024-09-21T21:44:20.090 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.50964.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50964.log.gz 2024-09-21T21:44:20.090 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42697.log 2024-09-21T21:44:20.090 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.49452.log: gzip -5 0.0% --verbose -- /var/log/ceph/ceph-client.admin.44529.log 2024-09-21T21:44:20.090 INFO:teuthology.orchestra.run.smithi081.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49452.log.gz 2024-09-21T21:44:20.090 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.42697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55006.log 2024-09-21T21:44:20.090 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42697.log.gz 2024-09-21T21:44:20.090 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.44529.log: -- /var/log/ceph/ceph-client.admin.56266.log 2024-09-21T21:44:20.090 INFO:teuthology.orchestra.run.smithi081.stderr: 29.1% -- replaced with /var/log/ceph/ceph-client.admin.44529.log.gz 2024-09-21T21:44:20.091 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.55006.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55006.log.gz 2024-09-21T21:44:20.091 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40910.log 2024-09-21T21:44:20.091 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.56266.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56266.log.gz 2024-09-21T21:44:20.091 INFO:teuthology.orchestra.run.smithi081.stderr: --verbose -- /var/log/ceph/ceph-client.admin.35547.log 2024-09-21T21:44:20.091 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.40910.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40910.log.gz 2024-09-21T21:44:20.091 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46902.log 2024-09-21T21:44:20.091 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.35547.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35547.log.gz 2024-09-21T21:44:20.091 INFO:teuthology.orchestra.run.smithi081.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.38584.log 2024-09-21T21:44:20.091 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.46902.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39547.log 2024-09-21T21:44:20.091 INFO:teuthology.orchestra.run.smithi081.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46902.log.gz 2024-09-21T21:44:20.091 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.38584.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51988.log 2024-09-21T21:44:20.091 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38584.log.gz 2024-09-21T21:44:20.091 INFO:teuthology.orchestra.run.smithi081.stderr:gzip/var/log/ceph/ceph-client.admin.39547.log: -5 --verbose -- 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39547.log.gz 2024-09-21T21:44:20.091 INFO:teuthology.orchestra.run.smithi081.stderr: /var/log/ceph/ceph-client.admin.35036.log 2024-09-21T21:44:20.092 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.51988.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51988.log.gz 2024-09-21T21:44:20.092 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37583.log 2024-09-21T21:44:20.092 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.35036.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35036.log.gz 2024-09-21T21:44:20.092 INFO:teuthology.orchestra.run.smithi081.stderr: --verbose -- /var/log/ceph/ceph-client.admin.58329.log 2024-09-21T21:44:20.092 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.37583.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37583.log.gz 2024-09-21T21:44:20.092 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37254.log 2024-09-21T21:44:20.092 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.58329.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58329.log.gz 2024-09-21T21:44:20.092 INFO:teuthology.orchestra.run.smithi081.stderr: --verbose -- /var/log/ceph/ceph-client.admin.40079.log 2024-09-21T21:44:20.092 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.37254.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36450.log 2024-09-21T21:44:20.092 INFO:teuthology.orchestra.run.smithi081.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37254.log.gz 2024-09-21T21:44:20.092 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.40079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35838.log 2024-09-21T21:44:20.092 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40079.log.gz 2024-09-21T21:44:20.092 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36725.log 2024-09-21T21:44:20.092 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.36450.log: /var/log/ceph/ceph-client.admin.35838.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.36450.log.gz -- 2024-09-21T21:44:20.092 INFO:teuthology.orchestra.run.smithi081.stderr: /var/log/ceph/ceph-client.admin.55953.log 2024-09-21T21:44:20.093 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35838.log.gz 2024-09-21T21:44:20.093 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56943.log 2024-09-21T21:44:20.093 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.36725.log: /var/log/ceph/ceph-client.admin.55953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41502.log 2024-09-21T21:44:20.093 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36725.log.gz 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55953.log.gz 2024-09-21T21:44:20.093 INFO:teuthology.orchestra.run.smithi081.stderr: 2024-09-21T21:44:20.093 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.56943.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56943.log.gz 2024-09-21T21:44:20.093 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58203.log 2024-09-21T21:44:20.093 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.41502.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41502.log.gz 2024-09-21T21:44:20.093 INFO:teuthology.orchestra.run.smithi081.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.38501.log 2024-09-21T21:44:20.093 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.58203.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58203.log.gz 2024-09-21T21:44:20.093 INFO:teuthology.orchestra.run.smithi081.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.56308.log 2024-09-21T21:44:20.093 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.38501.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38501.log.gz 2024-09-21T21:44:20.093 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57336.log 2024-09-21T21:44:20.093 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.56308.log: gzip -5 --verbose 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56308.log.gz 2024-09-21T21:44:20.093 INFO:teuthology.orchestra.run.smithi081.stderr: -- /var/log/ceph/ceph-client.admin.51077.log 2024-09-21T21:44:20.094 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.57336.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57336.log.gz 2024-09-21T21:44:20.094 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37940.log 2024-09-21T21:44:20.094 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.51077.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51077.log.gz 2024-09-21T21:44:20.094 INFO:teuthology.orchestra.run.smithi081.stderr: --verbose -- /var/log/ceph/ceph-client.admin.45954.log 2024-09-21T21:44:20.094 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.37940.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37940.log.gz 2024-09-21T21:44:20.094 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50097.log 2024-09-21T21:44:20.094 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.45954.log: gzip -5 --verbose 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45954.log.gz 2024-09-21T21:44:20.094 INFO:teuthology.orchestra.run.smithi081.stderr: -- /var/log/ceph/ceph-client.admin.48574.log 2024-09-21T21:44:20.094 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.50097.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50097.log.gz 2024-09-21T21:44:20.094 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44699.log 2024-09-21T21:44:20.094 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.48574.log: gzip -5 0.0% --verbose -- /var/log/ceph/ceph-client.admin.48749.log 2024-09-21T21:44:20.094 INFO:teuthology.orchestra.run.smithi081.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48574.log.gz 2024-09-21T21:44:20.094 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.44699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46359.log 2024-09-21T21:44:20.094 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44699.log.gz 2024-09-21T21:44:20.094 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53713.log/var/log/ceph/ceph-client.admin.48749.log: 2024-09-21T21:44:20.095 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48749.log.gz 2024-09-21T21:44:20.095 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.46359.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46359.log.gz 2024-09-21T21:44:20.095 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48320.log 2024-09-21T21:44:20.095 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5/var/log/ceph/ceph-client.admin.53713.log: --verbose -- /var/log/ceph/ceph-client.admin.38173.log 2024-09-21T21:44:20.095 INFO:teuthology.orchestra.run.smithi081.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53713.log.gz 2024-09-21T21:44:20.095 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.48320.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48320.log.gz 2024-09-21T21:44:20.095 INFO:teuthology.orchestra.run.smithi081.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.56691.log 2024-09-21T21:44:20.095 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.38173.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38173.log.gz 2024-09-21T21:44:20.095 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41094.log 2024-09-21T21:44:20.095 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.56691.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56691.log.gz 2024-09-21T21:44:20.095 INFO:teuthology.orchestra.run.smithi081.stderr: --verbose -- /var/log/ceph/ceph-client.admin.40160.log 2024-09-21T21:44:20.095 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.41094.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41094.log.gz 2024-09-21T21:44:20.095 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50782.log 2024-09-21T21:44:20.095 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.40160.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40160.log.gz 2024-09-21T21:44:20.095 INFO:teuthology.orchestra.run.smithi081.stderr: --verbose -- /var/log/ceph/ceph-client.admin.35043.log 2024-09-21T21:44:20.096 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.50782.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50782.log.gz 2024-09-21T21:44:20.096 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56607.log 2024-09-21T21:44:20.096 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.35043.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35043.log.gz 2024-09-21T21:44:20.096 INFO:teuthology.orchestra.run.smithi081.stderr: --verbose -- /var/log/ceph/ceph-client.admin.44812.log 2024-09-21T21:44:20.096 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.56607.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56607.log.gz 2024-09-21T21:44:20.096 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.44812.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44812.log.gz 2024-09-21T21:44:20.096 INFO:teuthology.orchestra.run.smithi081.stderr: 91.0% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-21T21:44:20.183 INFO:teuthology.orchestra.run.smithi081.stderr: 90.5% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-21T21:44:23.359 INFO:teuthology.orchestra.run.smithi081.stderr: 95.6% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-21T21:44:24.584 INFO:teuthology.orchestra.run.smithi122.stderr: 95.7% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-21T21:44:25.756 INFO:teuthology.orchestra.run.smithi081.stderr: 95.9% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-21T21:44:27.378 INFO:teuthology.orchestra.run.smithi122.stderr: 95.7% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-21T21:44:28.617 INFO:teuthology.orchestra.run.smithi081.stderr: 95.5% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-21T21:44:28.619 INFO:teuthology.orchestra.run.smithi081.stderr: 2024-09-21T21:44:28.619 INFO:teuthology.orchestra.run.smithi081.stderr:real 0m8.760s 2024-09-21T21:44:28.619 INFO:teuthology.orchestra.run.smithi081.stderr:user 0m17.790s 2024-09-21T21:44:28.619 INFO:teuthology.orchestra.run.smithi081.stderr:sys 0m0.882s 2024-09-21T21:44:30.660 INFO:teuthology.orchestra.run.smithi122.stderr: 95.7% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-21T21:44:30.662 INFO:teuthology.orchestra.run.smithi122.stderr: 2024-09-21T21:44:30.662 INFO:teuthology.orchestra.run.smithi122.stderr:real 0m10.801s 2024-09-21T21:44:30.662 INFO:teuthology.orchestra.run.smithi122.stderr:user 0m22.301s 2024-09-21T21:44:30.662 INFO:teuthology.orchestra.run.smithi122.stderr:sys 0m0.950s 2024-09-21T21:44:30.663 INFO:tasks.ceph:Archiving logs... 2024-09-21T21:44:30.663 DEBUG:teuthology.misc:Transferring archived files from smithi081:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914423/remote/smithi081/log 2024-09-21T21:44:30.664 DEBUG:teuthology.orchestra.run.smithi081:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T21:44:32.961 DEBUG:teuthology.misc:Transferring archived files from smithi122:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914423/remote/smithi122/log 2024-09-21T21:44:32.962 DEBUG:teuthology.orchestra.run.smithi122:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T21:44:35.521 DEBUG:teuthology.misc:Transferring archived files from smithi164:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914423/remote/smithi164/log 2024-09-21T21:44:35.522 DEBUG:teuthology.orchestra.run.smithi164:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T21:44:35.671 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-21T21:44:35.690 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-21T21:44:35.690 DEBUG:teuthology.orchestra.run.smithi081:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T21:44:35.693 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T21:44:35.695 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T21:44:35.796 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-21T21:44:35.805 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-21T21:44:35.805 DEBUG:teuthology.orchestra.run.smithi081:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T21:44:35.808 DEBUG:teuthology.orchestra.run.smithi122:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T21:44:35.810 DEBUG:teuthology.orchestra.run.smithi164:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T21:44:35.823 INFO:teuthology.orchestra.run.smithi081.stderr:bash: line 1: ntpq: command not found 2024-09-21T21:44:35.825 INFO:teuthology.orchestra.run.smithi122.stderr:bash: line 1: ntpq: command not found 2024-09-21T21:44:35.827 INFO:teuthology.orchestra.run.smithi081.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T21:44:35.827 INFO:teuthology.orchestra.run.smithi081.stdout:=============================================================================== 2024-09-21T21:44:35.827 INFO:teuthology.orchestra.run.smithi081.stdout:^* hv01.front.sepia.ceph.com 3 6 377 44 +253us[ +256us] +/- 48ms 2024-09-21T21:44:35.827 INFO:teuthology.orchestra.run.smithi081.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 45 -1129us[-1126us] +/- 66ms 2024-09-21T21:44:35.827 INFO:teuthology.orchestra.run.smithi081.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 44 +753us[ +753us] +/- 63ms 2024-09-21T21:44:35.827 INFO:teuthology.orchestra.run.smithi081.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T21:44:35.828 INFO:teuthology.orchestra.run.smithi164.stderr:bash: line 1: ntpq: command not found 2024-09-21T21:44:35.829 INFO:teuthology.orchestra.run.smithi122.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T21:44:35.829 INFO:teuthology.orchestra.run.smithi122.stdout:=============================================================================== 2024-09-21T21:44:35.829 INFO:teuthology.orchestra.run.smithi122.stdout:^* hv01.front.sepia.ceph.com 3 6 377 42 +252us[ +253us] +/- 49ms 2024-09-21T21:44:35.829 INFO:teuthology.orchestra.run.smithi122.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 44 -1124us[-1122us] +/- 66ms 2024-09-21T21:44:35.829 INFO:teuthology.orchestra.run.smithi122.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 44 +755us[ +757us] +/- 63ms 2024-09-21T21:44:35.829 INFO:teuthology.orchestra.run.smithi122.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T21:44:35.832 INFO:teuthology.orchestra.run.smithi164.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T21:44:35.832 INFO:teuthology.orchestra.run.smithi164.stdout:=============================================================================== 2024-09-21T21:44:35.832 INFO:teuthology.orchestra.run.smithi164.stdout:^* hv01.front.sepia.ceph.com 3 6 377 43 +265us[ +252us] +/- 48ms 2024-09-21T21:44:35.832 INFO:teuthology.orchestra.run.smithi164.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 43 -1123us[-1135us] +/- 66ms 2024-09-21T21:44:35.832 INFO:teuthology.orchestra.run.smithi164.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 44 +775us[ +763us] +/- 63ms 2024-09-21T21:44:35.832 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T21:44:35.833 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-21T21:44:35.842 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-21T21:44:35.843 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-21T21:44:35.867 DEBUG:teuthology.orchestra.run.smithi081:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T21:44:35.954 DEBUG:teuthology.orchestra.run.smithi122:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T21:44:36.033 DEBUG:teuthology.orchestra.run.smithi164:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T21:44:36.098 DEBUG:teuthology.orchestra.run.smithi081:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-21T21:44:36.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T21:44:36.129 DEBUG:teuthology.orchestra.run.smithi122:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-21T21:44:36.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T21:44:36.160 DEBUG:teuthology.orchestra.run.smithi164:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-21T21:44:36.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T21:44:36.190 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-21T21:44:36.204 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-21T21:44:36.225 INFO:teuthology.task.internal:Duration was 663.278278 seconds 2024-09-21T21:44:36.226 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-21T21:44:36.260 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-21T21:44:36.260 DEBUG:teuthology.orchestra.run.smithi081:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T21:44:36.263 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T21:44:36.266 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T21:44:36.303 INFO:teuthology.orchestra.run.smithi081.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T21:44:36.305 INFO:teuthology.orchestra.run.smithi122.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T21:44:36.307 INFO:teuthology.orchestra.run.smithi164.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T21:44:36.620 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-21T21:44:36.620 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi081.front.sepia.ceph.com 2024-09-21T21:44:36.621 DEBUG:teuthology.orchestra.run.smithi081:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T21:44:36.684 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi122.front.sepia.ceph.com 2024-09-21T21:44:36.684 DEBUG:teuthology.orchestra.run.smithi122:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T21:44:36.709 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi164.front.sepia.ceph.com 2024-09-21T21:44:36.710 DEBUG:teuthology.orchestra.run.smithi164:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T21:44:36.736 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-21T21:44:36.736 DEBUG:teuthology.orchestra.run.smithi081:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T21:44:36.739 DEBUG:teuthology.orchestra.run.smithi122:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T21:44:36.752 DEBUG:teuthology.orchestra.run.smithi164:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T21:44:36.804 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-21T21:44:36.805 DEBUG:teuthology.orchestra.run.smithi081:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T21:44:36.808 DEBUG:teuthology.orchestra.run.smithi122:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T21:44:36.821 DEBUG:teuthology.orchestra.run.smithi164:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T21:44:36.950 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-21T21:44:36.965 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-21T21:44:36.965 DEBUG:teuthology.orchestra.run.smithi081:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T21:44:36.990 DEBUG:teuthology.orchestra.run.smithi122:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T21:44:37.015 DEBUG:teuthology.orchestra.run.smithi164:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T21:44:37.041 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-21T21:44:37.050 DEBUG:teuthology.orchestra.run.smithi081:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T21:44:37.053 DEBUG:teuthology.orchestra.run.smithi122:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T21:44:37.057 DEBUG:teuthology.orchestra.run.smithi164:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T21:44:37.073 INFO:teuthology.orchestra.run.smithi081.stdout:kernel.core_pattern = core 2024-09-21T21:44:37.078 INFO:teuthology.orchestra.run.smithi122.stdout:kernel.core_pattern = core 2024-09-21T21:44:37.105 INFO:teuthology.orchestra.run.smithi164.stdout:kernel.core_pattern = core 2024-09-21T21:44:37.133 DEBUG:teuthology.orchestra.run.smithi081:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T21:44:37.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T21:44:37.156 DEBUG:teuthology.orchestra.run.smithi122:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T21:44:37.170 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T21:44:37.170 DEBUG:teuthology.orchestra.run.smithi164:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T21:44:37.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T21:44:37.188 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-21T21:44:37.198 INFO:teuthology.task.internal:Transferring archived files... 2024-09-21T21:44:37.198 DEBUG:teuthology.misc:Transferring archived files from smithi081:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914423/remote/smithi081 2024-09-21T21:44:37.199 DEBUG:teuthology.orchestra.run.smithi081:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T21:44:37.233 DEBUG:teuthology.misc:Transferring archived files from smithi122:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914423/remote/smithi122 2024-09-21T21:44:37.233 DEBUG:teuthology.orchestra.run.smithi122:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T21:44:37.267 DEBUG:teuthology.misc:Transferring archived files from smithi164:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-21_21:16:02-rbd-squid-distro-default-smithi/7914423/remote/smithi164 2024-09-21T21:44:37.268 DEBUG:teuthology.orchestra.run.smithi164:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T21:44:37.305 INFO:teuthology.task.internal:Removing archive directory... 2024-09-21T21:44:37.305 DEBUG:teuthology.orchestra.run.smithi081:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T21:44:37.308 DEBUG:teuthology.orchestra.run.smithi122:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T21:44:37.310 DEBUG:teuthology.orchestra.run.smithi164:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T21:44:37.353 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-21T21:44:37.365 INFO:teuthology.task.internal:Not uploading archives. 2024-09-21T21:44:37.365 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-21T21:44:37.392 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-21T21:44:37.392 DEBUG:teuthology.orchestra.run.smithi081:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T21:44:37.395 DEBUG:teuthology.orchestra.run.smithi122:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T21:44:37.397 DEBUG:teuthology.orchestra.run.smithi164:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T21:44:37.407 INFO:teuthology.orchestra.run.smithi081.stdout: 265324 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 21:44 /home/ubuntu/cephtest 2024-09-21T21:44:37.409 INFO:teuthology.orchestra.run.smithi122.stdout: 265970 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 21:44 /home/ubuntu/cephtest 2024-09-21T21:44:37.411 INFO:teuthology.orchestra.run.smithi164.stdout: 266088 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 21:44 /home/ubuntu/cephtest 2024-09-21T21:44:37.413 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-21T21:44:37.426 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-21T21:44:37.497 INFO:teuthology.run:Summary data: description: rbd/device/{base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-snappy supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_nbd} duration: 663.2782776355743 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-21T21:44:37.498 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T21:44:37.582 INFO:teuthology.run:pass